- Gambit-list - mailman.iro.umontreal.ca

Fwd: fast csv parser (libcsv wrapped for gambit)
by david rush 12 Fév '07

12 Fév '07

On 2/10/07, Bradley Lucier <lucier(a)math.purdue.edu> wrote: > These two don't agree on what the fields should be in all cases. For > example, on the 97th record, I get > > (...2006 Formula D\303\251 Gen Con Tournament qualifier" > ..."Some Experience Needed" > "Formula D\303\251" > "all Advanced Rules except time trials" > "4.50" > "" > "40" > "12") > > for bewig and > > ("..."2006 Formula D\351 Gen Con Tournament qualifier" > ..."Some Experience Needed" > "Formula D\351" > "all Advanced Rules except time trials" > "4.50" > "" > "40" > "12") > > for libcsv. I don't know which is correct (but the \303 characters > are in the data file). Well without seeing it in hex (and feeling far too lazy and comfortable in front of my lovely fireplace to bother checking it myself) I'd say that this looks like a classic UTF-8 coding issue... david -- Once you label me, you negate me - Soren Kierkegaard

2 1

Possible new compilation strategy for read-char and write-char
by Bradley Lucier 12 Fév '07

12 Fév '07

Marc: macro-read-char is a fair amount of code to inline at each site of read-char. We have with the original -rw-r--r-- 1 lucier pfaculty 107585 Feb 10 12:47 bewig2.c -rwxr-xr-x 1 lucier pfaculty 52001 Feb 10 12:47 bewig2.o1* -rw-r--r-- 1 lucier pfaculty 10799 Feb 10 11:36 bewig2.scm If we replace all 16 instances of read-char with macro-read-char, we get -rw-r--r-- 1 lucier pfaculty 175125 Feb 10 13:38 bewig2+macro-read- char.c -rwxr-xr-x 1 lucier pfaculty 71575 Feb 10 13:38 bewig2+macro-read- char.o2* -rw-r--r-- 1 lucier pfaculty 10925 Feb 10 13:38 bewig2+macro-read- char.scm Part of the problem here is that, if function A calls function B, Gambit's inlining heuristic looks at the size of the macro-expanded function A before deciding whether to inline function B into A; if macro-expansion increases the size of A a lot (say, by expanding macro-read-char ;-), then B is more likely to be inlined into A. This happened with two of the state-machine functions in read-csv- record. If, instead, we add a simple function my-read-char (define (my-read-char port) (macro-read-char port)) and then replace calls to read-char by calls to my-read-char, we get -rw-r--r-- 1 lucier pfaculty 134833 Feb 11 11:50 bewig3.c -rwxr-xr-x 1 lucier pfaculty 59349 Feb 11 11:50 bewig3.o1* -rw-r--r-- 1 lucier pfaculty 10929 Feb 11 11:50 bewig3.scm and the times are almost as good as when macro-read-char is inlined at all call sites of read-char, 113ms on Philip's benchmark instead of 111ms. Recall that the time for using read-char is 435ms, so we're about four times as fast. So perhaps you should add another compilation strategy---for files that have calls to read-char (or to any functions that are big enough, or have a big enough fast path, that you don't want to inline them into each call site), put in a local function that has the code for macro-read-char, and redirect calls to read-char to this local function (under suitable circumstances, of course). Brad

1 0

Re: [gambit-list] fast csv parser (libcsv wrapped for gambit)
by Phil Dawes 10 Fév '07

10 Fév '07

Robby Findler wrote: > I know that this has turned into yet another Scheme benchmarking > extravaganza, but I wonder if anyone has considered trying to use PADS > to solve the original problem? > > http://www.padsproj.org/ > > I'd be curious to know how well it does (I've only read their papers, > not looked at the system itself.) > Hi Robby, Thanks for the link. I had a quick look but unfortunately I can't get on with the licence: http://www.padsproj.org/License.html (I need to be able to use this at work, in code which could end up as part of a service so GPL/LGPL is better for me) Cheers, Phil

1 0

Re: [gambit-list] SSAX-SXML port to Gambit-C
by schemeway＠sympatico.ca 10 Fév '07

10 Fév '07

Guillaume, > which version did you get? I remember there are a few hanging around, but > Kirill Lisovsky has one that works well with Gambit out of the box. I had to patch ssax-sxml-gambit-06052z,9.tg i.e. the latest version for Gambit-C on SourceForge. > In fact, after looking up "ssax Gambit" on Google, I found the following > link ;) > > https://webmail.iro.umontreal.ca/pipermail/gambit-list/2006-October/000884.… But if you read carefully (;-) you'll notice that it explicitly says it's for Gambit-C 3.0 (!), not 4.0. Also, Marc changed the default compilation settings since last June, which breaks the build process. Maybe it's because I'm on Windows/cygwin? I haven't tried it on Linux. Cheers! Dominique

1 0

Re: [gambit-list] fast csv parser (libcsv wrapped for gambit)
by david rush 09 Fév '07

09 Fév '07

---------- Forwarded message ---------- From: david rush <kumoyuki(a)gmail.com> Date: Feb 9, 2007 8:25 AM Subject: Re: [gambit-list] fast csv parser (libcsv wrapped for gambit) To: Marc Feeley <feeley(a)iro.umontreal.ca> On 2/9/07, Marc Feeley <feeley(a)iro.umontreal.ca> wrote: > On 8-Feb-07, at 4:03 PM, Phil Dawes wrote: > > I needed a fast csv parser for parsing large (multi-gig) files so I've > > wrapped the libcsv c code by Robert Gamble[1]. The first cut of > > csv.scm > Interesting. But why would you do it in C (in 600 lines of code) > when you can do it in 20 lines of Scheme? I was wondering about this as well. I had assumed that it had to do with features available in libcsv, but since I didn't have the time to do the homework, I didn't say anything. I certainly think you could meet (or beat) a C-library performance in Gambit, even if libcsv is using mmap to read the file in the first place... david -- Once you label me, you negate me - Soren Kierkegaard -- Once you label me, you negate me - Soren Kierkegaard

1 0

calling scheme with a dlopen in c
by Sean D'Epagnier 09 Fév '07

09 Fév '07

3 4

Gambit FFI reference counting
by Phil Dawes 09 Fév '07

09 Fév '07

The trickiest thing I found writing Gambit FFI code[1] was managing memory. After flailing around a bit scratching my head over memory leaks and crashes I found the best technique was to check the reference counts of objects as much as possible. Gambit doesn't appear to have a function to peek the refcount of a "STILL" allocated object, so I created a simple one in my code: int _peek_refcount(___SCMOBJ s){ #define ___STILL_BODY_OFS 6 #define ___STILL_REFCOUNT_OFS 1 return ___UNTAG(s)[___BODY_OFS - ___STILL_BODY_OFS + ___STILL_REFCOUNT_OFS]; } I then littered the code with assert calls. e.g: assert(1 == _peek_refcount(myobj)); Hope this helps somebody, Phil [1] https://webmail.iro.umontreal.ca/pipermail/gambit-list/2007-February/001112…

1 0

Re: [gambit-list] good language for real time application?
by Patrick D. Logan 09 Fév '07

09 Fév '07

Vincent wrote .. > I think it would be possible to write it using > statically allocated data structures. Since the boot > CD would use the computer as a single purpose machine, > I could just take all the memory available and manage > it myself rather than using the OS to allocate or > deallocate. Yes, "real-time" generally means "perform within some predictable limits" for time, space, whatever. I am aware of a few real-time systems built with commercial Smalltalk implementations that do essentially pre-allocation of pools of things and put other kinds of controls on the dynamic nature of things. Erlang is another example. So it comes down to the kinds of limits you have and can you put enough control over things that may blow past those limits. -Patrick

4 5

good language for real time application?
by Vincent 08 Fév '07

08 Fév '07

Hello, I want to write a router with real time performance. I am trying to find the best language to use for it, and would like to use a functional language if possible. Erlang might work, but I want to distribute the router incorporated into a Linux live boot CD, so I think that maybe whatever I distribute has to be GPL compatible. I don't know if I can distribute the Erlang runtime with that license condition. I tried to join the Erlang questions list to ask about that, but so far have not been able to join the list. Is Gambit Scheme suitable for real time applications? Does it have a real time garbage collector? Vincent ____________________________________________________________________________________ Finding fabulous fares is fun. Let Yahoo! FareChase search your favorite travel sites to find flight and hotel bargains. http://farechase.yahoo.com/promo-generic-14795097

3 3

Re: [gambit-list] Passing lambdas as callbacks to C functions
by schemeway＠sympatico.ca 05 Fév '07

05 Fév '07

Phil, >> I ran into problems using lambda expressions as callbacks when wrapping >> libcsv and I've boiled it down to this testcase. Can anybody tell me why >> this crashes?: >... >> looper is a c function which takes a scheme lambda and repeated invokes >> it using the 'call_lambda' bridging function. >> >> When I gsc compile it and run '(main)' after a bunch of looping and >> printing it crashes with: >> ERROR IN looper -- Operator is not a PROCEDURE >> ('#<return #2>) >> >> Any ideas? (or am I being stupid?!) > I'm just guessing here, but could it be that, since your lambda > expression is not reference anywhere else in your Scheme code, it > gets GC'd? I'm not so sure about my diagnostic anymore... I've able to reproduce your problem and it only shows up when started from the OS shell. If I first load your (compiled) code in gsi and then evaluate '(main)', I don't get the problem at all. That's strange... Marc, any idea? Dominique

1 0