Dear Gambit-List,
I've been accumulating a library of functions for simulating population processes (called "genxic"), and I'm running into some pesky errors. This code includes quite a lot of foreign functions, interfacing with the GNU Scientific Library.
1) When running numerically intense simulations, I'll get a segfault:
,---- | Process scheme segmentation fault `----
This might be after 50 iterations of a simulation when I'd planned to do 1152.
Where do I go from there? I've tried running gsc and gsi in GDB, but pretty often I can't get a backtrace after the segfault. The conditions under which the segfault occurs varies. Sometimes I get it after loading the library.
2) I've gotten this bizarre error from one of my procedures, and would like to know where to go to debug it:
,---- | *** ERROR IN | genxic.o1#31| -- (Argument 1) F64VECTOR expected | (f64vector-ref | '#u16(47806 51966 47806 51966 47806 51966 47806 51966 47806 51966 47806 519... | 1) `----
I am not initializing any u16 vectors (that I know of), so how can I tell where this comes from, and how do I investigate, or find what procedure genxic.o1#31 is? I tried repeating the compilation, and this is what I got
Dumping: #<primitive | genxic.o2|> #<primitive | genxic.o2#0|> #<primitive | genxic.o2#1|> #<primitive | genxic.o2#2|> #<primitive | genxic.o2#3|> ... #<primitive | genxic.o2#30|> #<primitive | genxic.o2#31|> #<procedure 1+> #<procedure 1-> #<procedure atom?> #<procedure xcons> #<procedure make-list>...
How are these procedures labeled, and how do I figure out which one it is?
Thanks, Joel
Afficher les réponses par date
2008/9/4 Joel J. Adamson adamsonj@email.unc.edu adamsonj@email.unc.edu
Recompile Gambit with debug options flipped on:
Unpack the Gambit sources.
Run ./configure --enable-single-host
(Should we flip on --enable-debug here also ?? )
Replace the two occurrences -O1 in Makefile with -O0 -g (more important than -O0 is -g, I heard a rumor that debugging info may be better if you do -O1 -g but have no idea).
make; make install
Ensure your sourcecode is compiled with (declare (safe)) (that's the default setting). I don't remember if the -O0 -g will be reflected to gcc/g++ when you do compile-file from within your app, so consider adding this to cc-opts of compile-file.
Using the instructions above, this should be fixed.
In order to track down the source of your bugs, it may be important for you to note that
- Throwing exceptions with no handler/catcher may sigsegv your app.
- You should presuppose that ___argXX variables inside Scheme-to-C calls are trashed after C-to-Scheme calls. I.e.: If you have declared a c-define or c-lambda that takes parameters, these will appear as ___arg0, ___arg1 etc. in the C code of the c-define / c-lambda. These will have correct contents all until you invoke Scheme functions from the C code. not afterwards. So back them up to the stack if you do.
- Ensure you didn't call the version of any Gambit-internal function with broken parameters. The names of them start with ##. Very simple example:
(cadr 4)
*** ERROR IN (console)@3.1 -- (Argument 1) PAIR expected (cadr 4) 1>
(##cadr 4)
/usr/local/bin/gsc: line 5: 17999 Segmentation fault
If your problems persist, ask the list further.
M
Thanks so much for the tips; I have some follow-up questions:
"MM" == Mikael More mikael.more@gmail.com writes:
MM> - Throwing exceptions with no handler/catcher may sigsegv your MM> app.
Good to know, I was unaware of this.
MM> - You should presuppose that ___argXX variables inside MM> Scheme-to-C calls are trashed after C-to-Scheme calls. I.e.: If MM> you have declared a c-define or c-lambda that takes parameters, MM> these will appear as ___arg0, ___arg1 etc. in the C code of the MM> c-define / c-lambda. These will have correct contents all until MM> you invoke Scheme functions from the C code. not afterwards.
Okay, take the following functions, that may be the source of the problem:
/* C function for transferring vectors */ int f64vector_length (___SCMOBJ); double f64vector_ref (___SCMOBJ, int); double * scm_vector_to_C (___SCMOBJ f64vec) { /* initialize the length of the vector */ int len = f64vector_length (f64vec); /* initializes the new vector */ double * newCvec = malloc (len*sizeof (double)); /* declare a counter */ int i; /* iterate over the length of the vector copying each object into the new vector */ for (i = 0; i<len; i++) { /* could be re-written using pointer arithmetic */ newCvec[i] = f64vector_ref (f64vec, i); } return newCvec; }
and then the corresponding "c-define"s:
;; Scheme function for getting data out of f64vectors (c-define (proc1 f64vec i) (scheme-object int) double "f64vector_ref" "" (f64vector-ref f64vec i)) (c-define (proc2 f64vec) (scheme-object) int "f64vector_length" "" (f64vector-length f64vec))
MM> So back them up to the stack if you do.
Forgive my ignorance of C, but how do I do that? (where the heck is this stack that everyone speaks of?)
Now, I also noticed this passage in the C-interface chapter of the manual:
,---- | Within the C code the variables `___arg1', `___arg2', etc. can be | referenced to access the converted arguments. Similarly, the result to | be returned from the call should be assigned to the variable `___result' | except when the result is of type `struct', `union', `type', `pointer', | `nonnull-pointer', `function' or `nonnull-function' in which case a | pointer must be assigned to the variable `___result_voidstar' which is | of type `void*'. For results of type `pointer', `nonnull-pointer', | `function' and `nonnull-function', the value assigned to the variable | `___result_voidstar' must be the pointer or function cast to `void*'. `----
However, I have functions like this:
(define make-gsl-matrix (c-lambda (int int scheme-object) gsl-matrix* "gsl_matrix * ___matrix = gsl_matrix_alloc (___arg1, ___arg2); ___matrix->data = scm_vector_to_C (___arg3); ___matrix->size1 = ___arg1; ___matrix->size2 = ___arg2; ___result_voidstar = ___matrix;"))
It seems that this is seriously wrong, according to the paragraph above: the last line should be
___result_voidstar = (void*)___matrix;
and the return type should be void* and not gsl-matrix*, defined as follows:
(c-define-type gsl-matrix* (pointer "gsl_matrix" gsl-matrix*))
But then the question becomes how do I access the result I need? I would much rather have a function that returns a matrix as a result (functional style) rather than having it imperatively modify an existing variable, a la C.
Thanks for your help! Joel
On 4-Sep-08, at 10:54 PM, Joel J. Adamson wrote:
Indeed your code violates some invariants of Gambit. The correct code is attached below.
The source of the problem is that you are storing references to Scheme objects (i.e. the ___SCMOBJ type) in places that the garbage collector has no knowledge of (i.e. C variables), so when there is a GC these references are not updated when the object referenced is moved by the GC. [Side note: amusingly the code actually works for f64vectors of length greater or equal to 128 because those vectors are allocated as "still" objects that aren't moved by the GC!]
Here is the sequence of events that cause the problem to occur:
1) scm_vector_to_C receives the parameter f64vec which is a movable f64vector (lets call it V) 2) the code calls the C functions f64vector_length and f64vector_ref which jump back into Scheme 3) the Scheme code executed when f64vector_ref is called allocates a flonum 4) at some point the heap fills up with these flonums and the GC is called 5) ***the GC moves V but does not update f64vec in scm_vector_to_C*** 6) when the for loop in scm_vector_to_C continues it calls f64vector_ref with f64vec, which has a stale reference (it points to where V used to be)
There are two solutions:
1) call the macro ___F64VECTORREF (from gambit.h) which does not allocate a flonum, and hence does not invoke the GC
2) use an inline code c-lambda where the vector is passed in one of the ___argxxx variables. These variables ***are know to the GC and will be updated properly***. But don't store their content in another C variable!
Marc
;;;;;; SOLUTION 1
(c-declare #<<end-of-c-declare
#include <stdlib.h>
#ifdef ___ORIGINAL_CODE___ /* C function for transferring vectors */ int f64vector_length (___SCMOBJ); double f64vector_ref (___SCMOBJ, int); #endif
double * scm_vector_to_C (___SCMOBJ f64vec) { /* initialize the length of the vector */ #ifdef ___ORIGINAL_CODE___ int len = f64vector_length (f64vec); #else int len = ___INT(___F64VECTORLENGTH(f64vec)); #endif /* initializes the new vector */ double * newCvec = malloc (len*sizeof (double)); /* declare a counter */ int i; /* iterate over the length of the vector copying each object into the new vector */ for (i = 0; i<len; i++) { /* could be re-written using pointer arithmetic */ #ifdef ___ORIGINAL_CODE___ newCvec[i] = f64vector_ref (f64vec, i); #else newCvec[i] = ___F64VECTORREF(f64vec, ___FIX(i)); #endif } return newCvec; } end-of-c-declare )
;;;;;;;; SOLUTION 2
(define foo (c-lambda (scheme-object) (pointer double) #<<end-of-code { /* initialize the length of the vector */ int len = f64vector_length (___arg1); /* initializes the new vector */ double * newCvec = malloc (len*sizeof (double)); /* declare a counter */ int i; /* iterate over the length of the vector copying each object into the new vector */ for (i = 0; i<len; i++) { /* could be re-written using pointer arithmetic */ newCvec[i] = f64vector_ref (___arg1, i); } ___result_voidstar = newCvec; } end-of-code ))
"Marc" == Marc Feeley feeley@iro.umontreal.ca writes:
Marc> The source of the problem is that you are storing references Marc> to Scheme objects (i.e. the ___SCMOBJ type) in places that the Marc> garbage collector has no knowledge of (i.e. C variables), so Marc> when there is a GC these references are not updated when the Marc> object referenced is moved by the GC.
I had a feeling this might be the problem, but not the vocabulary to express it ;)
Marc> [Side note: amusingly the code actually works for f64vectors Marc> of length greater or equal to 128 because those vectors are Marc> allocated as "still" objects that aren't moved by the GC!]
Yeah, I tried testing it with larger and larger vectors and couldn't get the segfault ;)
Marc> 1) call the macro ___F64VECTORREF (from gambit.h) which does Marc> not allocate a flonum, and hence does not invoke the GC
Okay, duh: I guess I'll check gambit.h next time I write a C function.
Marc> 2) use an inline code c-lambda where the vector is passed in Marc> one of the ___argxxx variables. These variables ***are know Marc> to the GC and will be updated properly***. But don't store Marc> their content in another C variable!
Good to know the difference.
Now I'm getting a segfault at mark_continuation() (mem.c:2430).
if (ra1 == ___GSTATE->internal_return) { ___WORD actual_ra = ___FP_STK(fp,___RETI_RA); ___RETI_GET_FS_LINK_GCMAP(actual_ra,fs,link,gcmap,nextgcmap) ___COVER_MARK_CONTINUATION_RETI; } else { here===> ___RETN_GET_FS_LINK_GCMAP(ra1,fs,link,gcmap,nextgcmap) ___COVER_MARK_CONTINUATION_RETN; }
This happened before when I was accumulating a list instead of outputting and then throwing away each run of the simulation --- am I just using up memory and the GC can't handle it? (is that even possible?) Or does it mean that I have other unreachable objects unknown to the garbage collector?
Joel
On 5-Sep-08, at 12:35 PM, Joel J. Adamson adamsonj@email.unc.edu wrote:
Hard to say without access to your code... What other c-lambdas and c- defines do you have?
Marc
"Marc" == Marc Feeley feeley@iro.umontreal.ca writes:
Marc> Hard to say without access to your code...
The code is available for svn checkout with
# svn co http://chondestes.bio.unc.edu/svn/genxic/trunk/ genxic
To build the loadable library, you will need the GNU Scientific Library.
Marc> What other c-lambdas and c- defines do you have?
Plenty of c-lambda, spread out over all the files that start with "gsl-", but here are all the c-defines and c-define-types:
;; -*- mode:scheme -*- ;;$Id$ ;; ;; gsl-genx.scm
;; This file is part of genXic ;; Copyright (C) 2008 Joel J. Adamson
;; genXic is free software; you can redistribute it and/or modify it ;; under the terms of the GNU General Public License as published by ;; the Free Software Foundation; either version 3, or (at your option) ;; any later version.
;; This program is distributed in the hope that it will be useful, but ;; WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; General Public License for more details.
;; You should have received a copy of the GNU General Public License ;; along with this program (see the file COPYING); if not, write to ;; the Free Software Foundation, Inc., 51 Franklin Street, Fifth ;; Floor, Boston, MA 02110-1301, USA.
;; include header file for GSL API (c-declare "#include "../c/gsl_genx.h"") ;; Scheme function for getting data out of f64vectors (c-define (proc1 f64vec i) (scheme-object int) double "f64vector_ref" "" (f64vector-ref f64vec i)) (c-define (proc2 f64vec) (scheme-object) int "f64vector_length" "" (f64vector-length f64vec))
;; declare types
;; gsl-matrix pointer (c-define-type gsl-matrix* (pointer "gsl_matrix" gsl-matrix* ;; release function "GENXIC_RELEASE_gsl_obj"))
;; gsl-vector pointer (c-define-type gsl-vector* (pointer "gsl_vector" gsl-vector* "GENXIC_RELEASE_gsl_obj"))
(c-define-type gsl_rng* (pointer "gsl_rng" gsl_rng* "GENXIC_RELEASE_gsl_obj"))
;; another important release function: ;; ;; "VADER_RELEASE_HIM"
(define-macro (gsl-pred tag obj) `(and (foreign? ,obj) (equal? (foreign-tags ,obj) '(,tag))))
;; boolean for gsl-vector pointer (define (gsl-vector*? gsl-object) (gsl-pred gsl-vector* gsl-object)) ;; boolean for gsl-matrix pointer (define (gsl-matrix*? gsl-object) (gsl-pred gsl-matrix* gsl-object)) (define (gsl-rng*? gsl-object) (gsl-pred gsl-rng* gsl-object))
(include "gsl-structures.scm") (include "gsl-lalgebra.scm") (include "gsl-wrapper.scm") (include "gsl-stochastic.scm") ;; end of gsl-genx.scm
Thanks for any suggestions ;) Joel
"Marc" == Marc Feeley feeley@iro.umontreal.ca writes:
Marc> Hard to say without access to your code... What other Marc> c-lambdas and c- defines do you have?
I forgot to mention, this segfault happens when I'm running the simulation that you can check out here:
# svn co http://chondestes.bio.unc.edu/svn/models/agjones agjones
Here's what happens:
============================================================ (gdb) run Starting program: /usr/local/Gambit-C/v4.2.8/bin/gsc Gambit v4.2.8
(load "genxic")
"/home/joel/lisp/scm/genxic/scm/genxic.o1"
(gsl-variance (f64vector 0.1 2.5))
2.88
(current-directory "/home/joel/lisp/scm/agjones") (load "main.scm")
*** WARNING -- Rebinding global variable "vector-length" to an interpreted procedure *** WARNING -- Rebinding global variable "f64vector-length" to an interpreted procedure "/home/joel/lisp/scm/agjones/main.scm"
(run-sim)
*** STOPPED IN run-sim, "data.scm"@133.6 1> ,e stream-initialize = #!unbound stream-consume = #!unbound 1> ,i #<procedure #2 run-sim> = (lambda () (letrec ((stream-initialize (lambda () (gen-sim-data (vector-length *model-param-vector*)))) (stream-consume (lambda (a b) (call-with-current-continuation (lambda (done) (set! *done* done) (if (null? a) (*done* "Done!") (summarize-sim-data a)) (call-with-values (lambda () (force b)) stream-consume)))))) (call-with-values stream-initialize stream-consume))) 1> ,c | > (vector-length '#((100 50 16 2 1000. .5 0. 0. 0. 1e-4 1. .25 true) (100 5... | 1152 | > (gen-sim-data 1152) .| #<unknown> | > (f64vector-length '#f64(1000.1137275451473 1000.1113046603153 1000.179049... | 41 | > (f64vector-length '#f64(1000. 1000. 1000. 1000. 1000. 1000. 1000. 1000. 1... | 59 ...snip... | > (f64vector-length '#f64(0. 0. 0. 0. 1000.2159945301612 1000.1360547757907... | 46 | > (f64vector-length '#f64(1000. 1000. 1000. 1000. 1000. 1000. 1000. 1000. 1... | 54 | > (gen-sim-data 1053) . Program received signal SIGSEGV, Segmentation fault. 0xb78815d4 in mark_continuation () at mem.c:2430 (gdb) backtrace #0 0xb78815d4 in mark_continuation () at mem.c:2430 #1 0xb7883ccf in ___garbage_collect (nonmovable_words_needed=0) at mem.c:3938 #2 0xb78a9b57 in ___H__20___kernel (___ps=0xb7c99820) at _kernel.c:7687 #3 0xb787cec5 in ___call (nargs=0, proc=-1212572783, stack_marker=134579429) at setup.c:1821 #4 0xb787e7ec in ___setup (setup_params=0xbfe353b4) at setup.c:2961 #5 0xb787a578 in ___main (linker=0x8050628 <____20___gsc__>) at main.c:582 #6 0xb788d6fd in ___main_char (argc=1, argv=0xbfe35554, linker=0x8050628 <____20___gsc__>, script_line=0x0) at os_base.c:338 #7 0x08050669 in main (argc=-1218998865, argv=0xb75efbd1) at _gsc_.c:12043 (gdb) break mem.c:2430 Breakpoint 2 at 0xb78815cf: file mem.c, line 2430. (gdb) run The program being debugged has been started already. Start it from the beginning? (y or n) y Starting program: /usr/local/Gambit-C/v4.2.8/bin/gsc
Breakpoint 2, mark_continuation () at mem.c:2430 (gdb) backtrace #0 mark_continuation () at mem.c:2430 #1 0xb7920ccf in ___garbage_collect (nonmovable_words_needed=0) at mem.c:3938 #2 0xb793c9dc in ___H__20___kernel (___ps=0xb7d36820) at _kernel.c:3246 #3 0xb7919e41 in ___call (nargs=0, proc=-1211929711, stack_marker=134579429) at setup.c:1817 #4 0xb791b7ec in ___setup (setup_params=0xbfcd2254) at setup.c:2961 #5 0xb7917578 in ___main (linker=0x8050628 <____20___gsc__>) at main.c:582 #6 0xb792a6fd in ___main_char (argc=1, argv=0xbfcd23f4, linker=0x8050628 <____20___gsc__>, script_line=0x0) at os_base.c:338 #7 0x08050669 in main (argc=3198, argv=0xf01d) at _gsc_.c:12043 (gdb)
============================================================ What does mark_continuation() do?
Joel
Thanks for all your help: it seems adjusting the heap size (-:m100000 or so) helps a lot. I've already run two full sets of simulations.
Thanks, Joel
Mikael More wrote:
This will switch on debug logging (it will write logging messages to a file "console" in the current directory wherever you start Gambit).
No, -O0 is better, because it doesn't optimize the C code. -O1 will eliminate a number of local variables so you won't be able to inspect their values, but that's not a problem usually.
I suggest someone start writing a wiki page. I'm starting to get tired of explaining things. I'll then add more details there.
Christian.