Janvier 2006 - Gambit-list - mailman.iro.umontreal.ca

Fwd: Still can't seem to get 64-bit shared libraries to work on Mac OS 10.4.4
by Bradley Lucier 18 Jan '06

18 Jan '06

I forgot to send this to the list. Begin forwarded message: > From: Bradley Lucier <lucier(a)math.purdue.edu> > Date: January 17, 2006 4:33:28 PM CST > To: Marc Feeley <feeley(a)iro.umontreal.ca> > Cc: Bradley Lucier <lucier(a)math.purdue.edu> > Subject: Still can't seem to get 64-bit shared libraries to work on > Mac OS 10.4.4 > > Marc: > > I downloaded the new Xtools programs (released January 12), and the > following still won't work: > > 73 16:09 env CC='gcc -mcpu=970 -m64 -g' ./configure --enable- > shared > 74 16:10 make -j > 75 16:12 sudo make install > 76 16:12 gsc > > It crashes with and invalid instruction trying to load _num.o. > > Can you try this and tell me what I'm doing wrong? > > Brad > > PS: Here's my environment. > > [lindv2:~/programs/gambc40b17] lucier% printenv > PATH=/sw/bin:/sw/sbin:/usr/local/Gambit-C/bin:/pkgs/dejagnu/bin:/ > pkgs/teTeX/bin/:/usr/local/Gambit-C/bin:/pkgs/dejagnu/bin:/pkgs/ > teTeX/bin/:/usr/bin:/bin:/usr/sbin:/sbin:/usr/X11R6/bin > SECURITYSESSIONID=89be80 > HOME=/Users/lucier > SHELL=/bin/tcsh > USER=lucier > __CF_USER_TEXT_ENCODING=0x1F5:0:0 > DISPLAY=:0.0 > MANPATH=/sw/share/man:/usr/share/man:/sw/lib/perl5/5.8.6/man:/usr/ > X11R6/man > HOSTTYPE=powermac > VENDOR=apple > OSTYPE=darwin > MACHTYPE=powerpc > SHLVL=2 > PWD=/Users/lucier/programs/gambc40b17 > LOGNAME=lucier > GROUP=lucier > HOST=lindv2.ima.umn.edu > CVSROOT=:pserver:anoncvs@gcc.gnu.org:/cvs/gcc > TEXINPUTS=.:/Users/lucier/text/papers/zakon: > INFOPATH=/sw/share/info:/sw/info:/usr/share/info > PERL5LIB=/sw/lib/perl5:/sw/lib/perl5/darwin > DYLD_FALLBACK_LIBRARY_PATH=/usr/local/Gambit-C/lib/: > XML_CATALOG_FILES=/sw/etc/xml/catalog > SGML_CATALOG_FILES=/sw/etc/sgml/catalog > TERM=xterm > WINDOWID=6291470

1 0

Error and value handling style question (long)
by Christian 18 Jan '06

18 Jan '06

Hello I've got a general Scheme programming style question. Since there may be different preferences around a particular Scheme implementation, I'm asking here. Actually, depending on the decision on which style to choose, I would vote for an efficiency improvement in gambit (see more below on keyword arguments). Especially since scheme is using runtime type checking, there are frequently "error cases" when accessing data structures, i.e. the data to be read is not available or has the wrong type. There are multiple approaches to handle this: a) call the current error handler, which usually transfers control to a captured continuation. b) (optionally) call a user-provided function instead, which may do the same. (i.e. the error function is lexically provided instead of dynamically). c) return a special value (optionally user-provided), which the caller has to check for. With solution a, the user has to install a new error handler if he wants to catch this, which is both tedious and relatively expensive. Solution b improves upon this by using the more efficient (and slightly less inconvenient) approach of providing the error handler through a lexical parameter instead of a dynamic parameter. But, unless also a success handler can be given, the user still has to use call/cc to differentiate between success and failure. Solution c may be the one looking as the most efficient, and most simple from the perspective of the writer of the library. But the library user then always needs to check the error values, it's a duplication of branches (first, there are branches in the library code to decide upon the value to be returned, then there have to be branches to dispatch upon them). Maybe with a nice pattern matching language, that issue at least doesn't appear to be a problem on the surface anymore? (But there may also be cases where multiple kinds of errors can happen--how to solve this, provide multiple special values as arguments? Or, introduce an error object hierarchy instead, where the caller can ask (exception? value) and handle depending on the result (errors as first-class values). Of course returning errors or other items meant to make the caller dispatch on them, might interfere with storing those objects validly into the data structure (example: hash tables). (How do other languages (Haskell?) solve this?)) There is also the case, where the library wants to return multiple values. The implementation of R5RS values/call-with-values in Gambit is basically a tuples approach (afaik the same approach as taken by ML and Haskell). This makes it more elegant (in my opinion) than Schemes which require continuations to expect exactly the number of values being returned to them, since it's more flexible ("normal" generic one-value handling Scheme code can handle them). But since they allocate memory (I think ML can optimize that away partially), this is not very efficient. And, this is intertwined with the above question of how to dispatch on data errors: a function should have a way to return to different places, some of them taking multiple values, some of them one, some of them none ("manual CPS style"). For these reasons, I think I would happily dispense with multi-value-bind (expect for compatibility reasons) and provide the continuations manually. My only issues with this are: - I want that to be as "standard" as possible (syntax-wise), over all of my code and as much as possible other people's code. This means, using the same function naming conventions and the same ordering of arguments. - Using keyword arguments may make it easier to achieve that goal: something like: (foo-bar/baz value on-success: (lambda (bar baz) ... ) on-error: (lambda (error-object) ...)) could be used as a standard approach. But this is (currently) not very efficient in Gambit: for one, the keyword parsing code doesn't seem to be very fast, and worse, it prevents the compiler from doing inlining (and maybe other optimizations?) in block-compilation mode. - Does specifying two (or maybe more) such continuations necessarily require multiple closure allocations (unless optimized away in block-compilation mode)? That would make it possibly slower than the allocate-a-values-tuple approach in some cases. - Instead of using keyword arguments, settle on a function naming approach: Using the with- naming as hint that this function will take a success continuation as it's second argument, and optionally an error continuatoin as third: (with-foo-bar a-foo (lambda (bar) ...) (lambda (error-object) ..)) Without the with-, expect it to take an optional error continuation as the second argument, and optionally a success continuation as third: (let ((bar (foo-bar a-foo (lambda (error-object) ...)))) ..) or (foo-bar a-foo (lambda (error-object) ...) (lambda (bar) ....)) as the user desires. Below, I've pasted an excerpt of a library I'm working on for handling Gambit syntax objects, which finally gave me the motivation to ask about this style question here. (Maybe writing such libs could be made easier with the help of some macros, once a 'standard' is decided.) Thanks for your suggestions Christian. (define (syntax-pos syn) ;; combined line/col (vector-ref syn 3)) (define (make-syntax-error message) (lambda (erroneous-object) (error message erroneous-object))) (define identity (lambda (v) v)) (define (syntax-line syn #!optional (error (make-syntax-error "syntax-line: no position info:")) (success identity)) (cond ((syntax-pos syn) => (lambda (pos) (success (+ 1 (bitwise-and pos 65535))))) (else (error syn)))) (define (syntax-col syn #!optional (error (make-syntax-error "syntax-col: no position info:")) (success identity)) (cond ((syntax-pos syn) => (lambda (pos) (success (+ 1 (quotient pos 65536))))) (else (error syn)))) (define (with-syntax-line/col syn success #!optional (error (make-syntax-error "with-syntax-line/col: no position info:"))) (cond ((syntax-pos syn) => (lambda (pos) (success (+ 1 (bitwise-and pos 65535)) (+ 1 (quotient pos 65536))))) (else (error syn))))

1 0

chjmodule update
by Christian 14 Jan '06

14 Jan '06

Hello I've put up an updated version of chjmodule which works with beta 17. Basically nothing new except the necessary fixes. http://scheme.mine.nu/gambit/chjmodule/ About the future: I'm short on time, but I've got a bunch of ideas to try with chjmodule. I'll get back at them slowly but probably steadily. - use the c#expand-source hook instead of writing temporary files for adding the necessary namespace magic to the source. Actually I'm thinking about also providing a new source expander, which allows a version of define-macro that may handle source location annotations. - solve the problems with the import speed and with the compiler optimizations: instead of letting the loader transfer bindings, let the client module do that itself: prepend this for each binding to the code of the client module (by use of the expander hook): (define car (let () (##namespace ("")) car)) (define rest (let () (##namespace ("")) cdr)) (define sxml-foo (let () (##namespace ("sxml#")) foo)) etc. - declarations (declare ..) shall be put into the -module.scm files. - binary-versioning of code: a module shall be rebuilt if an object file has been compiled for a different gambit release than the currently running (since otherwise they may crash the scheme system upon loading). Thus: put object and .c files into subdirectories which contain the gambit version. (- versioning of code in memory: I'm thinking of keeping separate versions of code in the same scheme system process: e.g. one on which one is debugging (for example interpreted), another one which is used live (for example compiled), using separate namespaces for each.) - specializing: I've got an idea of "merge" module files, which create a new specially compiled module from existing ones: Given a module F which, by default, depends on functions and macros from modules D and E. Now there's a merge module file which defines a module F' which depends on functions and macros from modules E and G, replacing the functionality originally provided by module D with the one provided by module G. Example: srfi-1 defines many list processing functions. I want to change the srfi-1 code to use |delay| in the right positions to make them output streams. But by default, the |delay| syntax would be a noop defined in module D. And by default, it would take |car|,|cdr| etc. from module E (which in that case would actually be from namespace "", the gambit builtins). But then I would define a merge module named srfi-1-stream, which takes the exact same srfi-1 module, but replaces the dependency D with G which defines |delay| to be the gambit builtin one, and it replaces the dependency E with one named H which defines |car|, |cdr| etc. to be functions which |force| their argument, so that the functions can also take streams as inputs. Then one could also write a module merge file named srfi-1-syntax, which redefines |car|,|cdr| etc. to be operations which operate on syntax trees carrying source location annotations (well, probably that one won't be so easy if the aim is to carry over as much location information to the output as possible). Ok I'm sure this one is an old idea. I've never really looked at the SML or S48 module systems, is this how they work? Does the R6RS module system (aim to) provide this? - possibly work with/on the object system (CLOS-like, with MI) which has been started by Ken Dickey, and think about integration with the module system. (Probably easy as long as only one global place for generics exists.) Christian.

1 0

Cosmic convergence
by Bradley Lucier 14 Jan '06

14 Jan '06

With the purchase of my 8-core, shared-memory, SMP Opteron machine and Marc's possible interest (see http://mailman.iro.umontreal.ca/ pipermail/gambit-list/2006-January/000560.html) in extending Gambit's runtime to support SMP, three threads of thought (obsessions?) have converged. It would seem to be difficult to develop any one of them without the other two, and one of them involves Gambit development directly, so please forgive the limited relevance of this message to this list. 1. My current image processing research. I'm not going to explain the details here, except to say that the problems are highly parallelizable, involves minimizing functions over high-dimensional spaces (say, 40 dimensions), and evaluating the function to be minimized can take a few minutes on a single Opteron core. We don't yet know a provably correct algorithm for the solution of the problem, so we're doing exploratory calculations, and we're still at the "get the right answer, see if that answer's interesting, and if so, see if we can speed up the algorithm to make it practical" stage. Each test calculation takes hours to days to complete. SMP would speed up the process tremendously. 2. SMP in Gambit. 'Nuff said. 3. A new framework for array processing, with applications to image processing. People who develop new algorithms for image processing often start in Matlab because it can be interactive, it operates on entire matrices at a time, and it has a lot of built-in functions. The latter part is the killer app aspect of Matlab---if Matlab has the functions you want built in, then you're golden, the code runs fast, it's easy to program, etc. If, however, you're developing new algorithms, rather than just trying to use a known algorithm, it's possible, or even likely, that Matlab won't have the built-in functions you need. Then you're reduced to "writing Fortran/C code in Matlab", writing loops that operate on one number/pixel/voxel/... at a time. Then you find out that, not only is it difficult to write code in this style, Matlab is incredibly slow executing code like that. So then the graduate student rewrites the code in C, which is even more difficult, but now it runs quickly, but it's nearly impossible to debug or change. (And it's usually the graduate student going this route, because they won't believe from the beginning that Matlab won't cut it, and then they think that the only way to get the code to run in 20 seconds instead of two hours is to write it in C.) So, I want to fix this problem. My goal is a framework in Scheme for developing new algorithms in array/image processing that has the following properties: A. The speed of programs developed with this framework are within an order of magnitude of the speed of similar programs written in C. B. It's at least as easy to develop new algorithms in this framework as it is to work in Matlab when Matlab's built-in functions suffice. I've been writing all my image processing algorithms in this evolving framework for at least five years now. In functional programming terms, the basic operations are various flavors of map, reduce, curry, and, when you're really in a pickle, for-each, on multi- dimensional arrays containing heterogeneous or homogeneous data types. (You use curry when you want to slice and dice multi- dimensional images in various ways, e.g., to view a three-dimensional image of a body in a different image plane than the one in which the body was scanned originally.) I'm working on an SRFI submission. Now here's the convergence. Some people at Google just made a big deal about their map-reduce algorithms that they use to process multiple gigabytes of data over thousands of processors. To have this work, they need two things, which can be explained in SRFI-1 terms: first, in (map f l1 l2 ...) then it can't matter in what order f is applied to the elements of the lists, and, in fact, f has to be "thread safe", any parallel executions of f applied to the elements of the list must give the same answer, and second, in (reduce op id l) the same must be true of op, and op must be associative. (The property needed for "map" is more restrictive than the R5RS property for argument evaluation, which is that R5RS is allowed to assume that the arguments to a function end up with the same value no matter in what order they are computed, but they must be computed sequentially in some order, no parallelism, no threads, etc.) So you realize that if you want to parallelize multi-dimensional array versions of these functions, you need the same properties. On the other hand, an array for-each, say, (array-for-each f a1 a2 a3 ...) must apply f in some fixed order, and sequentially. And you realize that either you supply versions of array-map and array-reduce that have the same property, or you expect that the times that array-map and array-reduce need these properties are so few and far between that you leave it to the users to cobble together what they need from array-for-each. (The only time I've needed this property so far is to read and write image files.) So, there we go. Without an SMP version of Gambit, I couldn't develop and test our new parallel image processing framework and speed up executing our new algorithms by a factor of 8 or so. Without our image processing research applications, I wouldn't have anything with which to test SMP Gambit or my new array-processing framework. And without the new image-processing framework, I doubt that I could easily develop fast, SMP-capable, image-processing algorithms for my research. So, yes, "We want our SMP!" Brad

2 2

Debugging
by Christian 14 Jan '06

14 Jan '06

Hello In beta 17, it seems that the leap operation in the debugger still doesn't work as it should, right? Can this be fixed? Ideas on some other things which could be useful for a good debugging experience: - easily redefine functions which have been defined in the interpreter to use non-tail calls: something like (define (function-proper-tail-calls-set! name true?) (let ((code (##decompile (eval name)))) (if (pair? code) (parameterize ((generate-proper-tail-calls true?)) (eval `(define ,name ,code))) (error "not an interpreted procedure:" name code)))) ; > (define (f n) (if (<= n 0) (error "fin") (f (- n 1)))) ; > (f 10) ; *** ERROR IN (console)(a)10.1 -- fin ; 1> ; ctl-d ; > (function-proper-tail-calls-set! 'f #f) ; > (f 10) ; *** ERROR IN f -- fin ; 1> Ok this works so far, but it would be nice if it would keep location information, and probably also if it would act on the lambda itself, instead of on one particular binding of the lambda. - could regions of code, either lexically or dynamically scoped, be made un-stepped? I'm currently using compilation to prevent the debugger from single-stepping code which I know works, and in general this makes sense. But in some cases (like when adding new code to an existing module) I can't compile the old/working/uninteresting functions without moving them to another file, which may be tedious, so a simple (function-step-set! foo #f) or something similar to achieve the same effect would be nice. - keep the continuations of each step of a calculation in a list which can be inspected. For "going back in time" (will work sensibly only for purely functional code, of course). Something like: ;; switch on tracking: (step/track) ;; or (step/track 10) for limiting the length of the backtrack to 10 ;; steps ;; or (track 10) for tracking the execution path without switching ;; on single stepping (run-some-code) ... ;; re-run program from 10 steps before now: ((list-ref (current-track-head) 10) "someval") Well those "continuations" might not actually take a value [by default] but store the value from the previous calculation automatically. Once that is in place, the debugger might even provide a back-step operation. Thanks Christian. (BTW note that (with b17) unlike ctrl-d, entering ,s into the repl terminates the running system without safety question. Ok, ,q does that too. Might be friendlier? But ok, I guess I'll go for running repl's in separate threads anyway in the future (attach to a running system through a socket), probably, then it won't matter.) PS. can I compile gambit so that it works safely in all places and doesn't crash if I feed the wrong data to some internal function? And can I compile it with the '(debug) compiler option enabled?

2 2

Bill Clementson's Blog: Concurrent/Parallel Programming - The Next Generation - Part 2
by Bradley Lucier 12 Jan '06

12 Jan '06

Bill Clementson talks about concurrent "functional" programming in his blog, and laments the lack of public progress on Termite: http://bc.tech.coop/blog/060111.html

2 1

Fwd: Gambit-C 4.0b17 - Softpedia 100% CLEAN Award
by Marc Feeley 12 Jan '06

12 Jan '06

Does anyone know what this is about? Is Softpedia a reputable organization? Marc Begin forwarded message: > Hello, > > Your product "Gambit-C 4.0b17" has been tested by the Softpedia > labs and found to be > completely clean of adware/spyware components. > > We are impressed with the quality of your product and encourage you > to keep > this high standards in the future. > > To assure our visitors that "Gambit-C 4.0b17" is clean, we have > granted it with the > "100% CLEAN" Softpedia award. Moreover, to let your users know > about this > certification, you may display this award on your website, on > software boxes > or inside your product.

1 0

Reading data from mixed ASCII/binary files
by Bradley Lucier 12 Jan '06

12 Jan '06

Marc: The file formats used by the image library netpbm (pbm, pgm, ppm, pnm, ...) are mixed ascii/binary format; the first few lines are ASCII that have a magic number for the file, the size of the image (rows and columns) and the maximum value of the pixels (for greyscale images). Then, after a single newline, comes the binary data. So: [lindv2:Arrays/srfi2/stacey] lucier% head -3 2314.pgm P5 1605 2244 255 <binary data follows here> Now, in beta 16 and beta 17, I get the error: > [lindv2:Arrays/srfi2/stacey] lucier% gsc > Gambit Version 4.0 beta 17 > > > (load "convex2.scm") > *** ERROR IN #<procedure #2>, "../generic-arrays.scm"@1171.44 -- > Input port character buffer is not empty > (read-u8 '#<input-port #3 "/Users/lucier/lang/scheme/Arrays/srfi2/ > stacey/2314.pgm">) > 1> ,b > 0 #<procedure #2> "../generic-arrays.scm"@1171:44 > (domain-getter i j) > 1 #<procedure #4> "../generic-arrays.scm"@283:9 (f i j) > 2 Array->Fixed-array "../generic-arrays.scm"@1169:6 > (Interval-for-each (case (##Interval-dimension domain) ((1) (lambda > (i) (range-setter (domain-... > 3 #<procedure #5> "../pgm.scm"@44:5 (Array->Fixed- > array generic-array-manipulators (build-Array (build-Interval '#(0 > 0) (vector rows colu... > 4 call-with-input-file > 5 (interaction) "convex2.scm"@77:21 (read-pgm > (string-append name ".pgm")) > 6 ##load > 7 ##load > 8 (interaction) (console)@1:1 (load > "convex2.scm") > 1> ,e > i = 0 > j = 0 > range-setter = (lambda (value i j) (manipulator-setter body > (indexer i j) value)) > domain-getter = (lambda (i j) (read-u8 port)) > result = '#<Array-base #6 domain: #<Interval #7 lower-bounds: #(0 > 0) upper-bounds: #(2244 1605)> getter: #<procedure #8> setter: > #<procedure #9> manipulat... > domain = '#<Interval #7 lower-bounds: #(0 0) upper-bounds: #(2244 > 1605)> > result-manipulators = '#<Array-manipulators #10 getter: #<procedure > #11> setter: #<procedure #12> maker: #<procedure #13 make-vector> > length: #<procedure ... > array = '#<Array-base #15 domain: #<Interval #7 lower-bounds: #(0 > 0) upper-bounds: #(2244 1605)> getter: #<procedure #16> setter: #f > manipulators: #f body... > (current-exception-handler) = primordial-exception-handler > (current-input-port) = '#<input-output-port #17 (console)> > (current-output-port) = '#<input-output-port #17 (console)> > (current-directory) = "/Users/lucier/lang/scheme/Arrays/srfi2/stacey/" > 1> and I read from the manual: > procedure: read-u8 [port] > > This procedure reads the byte input-port port and returns the > byte at the current byte location and advances the current byte > location to the next byte, unless the port is already at end-of- > file in which case read-u8 returns the end-of-file object. If it is > not specified, port defaults to the current input-port. > > This procedure must be called before any use of the port in a > character input operation (i.e. a call to the procedures read, read- > char, peek-char, etc) because otherwise the character-stream and > byte-stream may be out of sync due to the port buffering. So, how do I deal with meta-data like this? Read the u8 numbers, call integer->char, put them all into a string, call read on the string, ...? Why should I have to do this instead of the runtime doing this? Or perhaps you have a better suggestion. Brad

2 2

Web site update
by Guillaume Germain 11 Jan '06

11 Jan '06

Hi all, I finally took the time to begin creating the new web site for Gambit. It's available at the usual address: http://www.iro.umontreal.ca/~gambit/ So I'm the one to blame if there's anything wrong with the web site. I plan on adding more content in the future. Amongst other things, I want to build some kind of library repository. I'd appreciate if people having good quality libraries for Gambit (like SRFI implementations, bindings, etc.) could send me an email and tell me where to get them. It would also be great to have some sane semi-automatic installation mechanism for libraries. In the meantime, I offer the following way to install SRFI-1 for "punks who are feeling lucky", and have write access to the Gambit installation directory: $ lynx --source http://www.iro.umontreal.ca/~germaing/srfi-1.sh | sh I'm sure we can do better :) Guillaume

1 0

--enable-profile
by Sven.Hartrumpf＠FernUni-Hagen.de 11 Jan '06

11 Jan '06

Hi again. The configure option --enable-profile should automatically disable -fomit-frame-pointer. Otherwise gcc will stop because the options -pg and -fomit-frame-pointer are incompatible. Ciao Sven P.S. Are there any profiling tools available for use with Gambit besides plain old gprof?

4 4