Problem with FFI & Memory Corruption
Hail, brothers and sisters in Scheme. I've been working with Gambit's FFI recently, trying to create bindings for libxml2. Currently I'm trying to use libxml2 to parse a file, and produce structures representing the XML nodes. Reading Gambit's lib/os.c for some examples of how to produce such things (I started with service-info and worked down from there) I developed some code that pretty much works. But there are problems with memory corruption that result in "odd" looking results (for example, a list that should have some 95,000 items is instead two large fixnums consed together) or segfaults. Finally I teased out an XML file that fails pretty reliably, at least on my desktop. First, the code is on Bitbucket --- https://bitbucket.org/taylor_venable/gambit-xml/. The default Makefile target will build xml.o1 - you will probably have to invoke it as "make GSC=gsc" (the Makefile uses "GSC=gambitc" by default, which is the name used by the Gambit compiler on Arch Linux since the "gsc" program is from Ghostscript). Second, the test XML file is on my website --- http://metasyntax.net/test.xml.gz --- warning about clicking that in your browser, it's 141 MB uncompressed. Here's how I'm using things: $ make gcc -c -o os_xml.o -I/usr/include/libxml2 -g -O0 -Wall -Wno-unused -fPIC os_xml.c gambitc -debug -link -flat -o xml.o1.c xml.scm *** WARNING -- "##direct-structure-ref" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "##direct-structure-set!" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "##structure" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "##structure-instance-of?" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "##structure-type-set!" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "##subtype-set!" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "error" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "fx=" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "map" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "vector-ref" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") gcc -c -o xml.o -I/usr/include/libxml2 -D___DYNAMIC -g -O0 -Wall -Wno-unused -fPIC xml.c gcc -c -o xml.o1.o -I/usr/include/libxml2 -D___DYNAMIC -g -O0 -Wall -Wno-unused -fPIC xml.o1.c gcc -shared -o xml.o1 -lxml2 os_xml.o xml.o xml.o1.o $ gsi Gambit v4.6.3
(load "xml") "/home/taylor/Repos/gambit-xml/xml.o1" (define xml (read-xml "./test.xml")) (length (xml-element-children (xml-document-root xml))) Segmentation fault
Sometimes I can do this: $ gsi Gambit v4.6.3
(load "xml") "/home/taylor/Repos/gambit-xml/xml.o1" (define xml (read-xml "./test.xml")) (length (xml-element-children (xml-document-root xml))) *** ERROR IN (console)@4.9 -- (Argument 1) Instance of #<type #2 xml-element> expected (xml-element-children '#<gc-hash-table #3>) 1> xml #<xml-document #4 location: "./test.xml" root: #<gc-hash-table #3>>
Or this: $ gsi Gambit v4.6.3
(load "xml") "/home/taylor/Repos/gambit-xml/xml.o1" (define xml (read-xml "/home/taylor/sis-conv-input.xml")) (length (xml-element-children (xml-document-root xml))) *** ERROR IN (console)@3.1 -- (Argument 1) LIST expected (length '(23220778 . 22958618)) 1> xml #<xml-document #2 location: "/home/taylor/sis-conv-input.xml" root: #<xml-element #3 name: "enterprise" attrs: () children: (23220778 . 22958618)>>
It varies. Sometimes I get the error because children is a something like '(23220778 . 22958618), then when I print out the xml value, the root is printed as #<gc-hash-table #3>. So it looks to me like a memory corruption problem, but I don't know why. When I use gdb to look for a cause: $ gdb gsi GNU gdb (GDB) 7.4 Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-unknown-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/bin/gsi...(no debugging symbols found)...done. (gdb) run Starting program: /usr/bin/gsi Gambit v4.6.3
(load "xml") "/home/taylor/Repos/gambit-xml/xml.o1" (define xml (read-xml "./test.xml")) (length (xml-element-children (xml-document-root xml)))
Program received signal SIGSEGV, Segmentation fault. 0x00007ffff702b064 in ___H_xml_2d_element_2d_children (___ps=0xa4ce20) at xml.c:6232 6232 ___IF(___STRUCTUREDIOP(___R1,___R2)) (gdb) bt #0 0x00007ffff702b064 in ___H_xml_2d_element_2d_children (___ps=0xa4ce20) at xml.c:6232 #1 0x00000000004ad51f in ___call () #2 0x00000000004af14f in ___setup () #3 0x00000000004c3ef2 in ___main () #4 0x00000000004b8436 in ___main_char () #5 0x00000000004ac6a9 in main () (gdb) print ___ps->r $1 = {10259873, 11115697, 140737339703681, 11115697, 140737339703681} The problem manifests in code generated by Gambit, but I doubt that's where the problem originates. I don't know enough about the Gambit internals to make it any further on my own. I hope somebody can help guide me in debugging what's wrong, and hopefully point out whether my memory handling in the os_xml.c file is correct or not. There are a few places where I have doubts about whether I should use ___release_scmobj() or not. All of these problems happen fairly regularly on my desktop, which is an Intel Core i7-870 with 8GB of memory running Arch Linux x86_64 kernel 3.2.4 --- but I haven't yet had them on my laptop, which is an Intel Core 2 Duo P8800 with 4GB of memory running the same kernel. Mighty thanks for any assistance! -- Taylor C. Venable http://metasyntax.net/
Afficher les réponses par date
Thanks to Valgrind, I found my mistake. I made a vector with size two, then put three items into it. Specifically, the root node went into index 2, which is why that field of the xml-document was the one with the bug. That silly problem fixed, things seem to be working fine. But if anybody has a spare minute and would be willing to check over my code, I would still appreciate any feedback about whether I'm doing things the right way. Especially when it comes to memory management. Thanks! Sent from my mobile. On Feb 9, 2012 6:33 PM, "Taylor Venable" <taylor@metasyntax.net> wrote:
Hail, brothers and sisters in Scheme.
I've been working with Gambit's FFI recently, trying to create bindings for libxml2. Currently I'm trying to use libxml2 to parse a file, and produce structures representing the XML nodes. Reading Gambit's lib/os.c for some examples of how to produce such things (I started with service-info and worked down from there) I developed some code that pretty much works. But there are problems with memory corruption that result in "odd" looking results (for example, a list that should have some 95,000 items is instead two large fixnums consed together) or segfaults. Finally I teased out an XML file that fails pretty reliably, at least on my desktop.
First, the code is on Bitbucket --- https://bitbucket.org/taylor_venable/gambit-xml/. The default Makefile target will build xml.o1 - you will probably have to invoke it as "make GSC=gsc" (the Makefile uses "GSC=gambitc" by default, which is the name used by the Gambit compiler on Arch Linux since the "gsc" program is from Ghostscript).
Second, the test XML file is on my website --- http://metasyntax.net/test.xml.gz --- warning about clicking that in your browser, it's 141 MB uncompressed.
Here's how I'm using things:
$ make gcc -c -o os_xml.o -I/usr/include/libxml2 -g -O0 -Wall -Wno-unused -fPIC os_xml.c gambitc -debug -link -flat -o xml.o1.c xml.scm *** WARNING -- "##direct-structure-ref" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "##direct-structure-set!" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "##structure" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "##structure-instance-of?" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "##structure-type-set!" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "##subtype-set!" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "error" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "fx=" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "map" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") *** WARNING -- "vector-ref" is not defined, *** referenced in: ("/home/taylor/Repos/gambit-xml/xml.c") gcc -c -o xml.o -I/usr/include/libxml2 -D___DYNAMIC -g -O0 -Wall -Wno-unused -fPIC xml.c gcc -c -o xml.o1.o -I/usr/include/libxml2 -D___DYNAMIC -g -O0 -Wall -Wno-unused -fPIC xml.o1.c gcc -shared -o xml.o1 -lxml2 os_xml.o xml.o xml.o1.o $ gsi Gambit v4.6.3
(load "xml") "/home/taylor/Repos/gambit-xml/xml.o1" (define xml (read-xml "./test.xml")) (length (xml-element-children (xml-document-root xml))) Segmentation fault
Sometimes I can do this:
$ gsi Gambit v4.6.3
(load "xml") "/home/taylor/Repos/gambit-xml/xml.o1" (define xml (read-xml "./test.xml")) (length (xml-element-children (xml-document-root xml))) *** ERROR IN (console)@4.9 -- (Argument 1) Instance of #<type #2 xml-element> expected (xml-element-children '#<gc-hash-table #3>) 1> xml #<xml-document #4 location: "./test.xml" root: #<gc-hash-table #3>>
Or this:
$ gsi Gambit v4.6.3
(load "xml") "/home/taylor/Repos/gambit-xml/xml.o1" (define xml (read-xml "/home/taylor/sis-conv-input.xml")) (length (xml-element-children (xml-document-root xml))) *** ERROR IN (console)@3.1 -- (Argument 1) LIST expected (length '(23220778 . 22958618)) 1> xml #<xml-document #2 location: "/home/taylor/sis-conv-input.xml" root: #<xml-element #3 name: "enterprise" attrs: () children: (23220778 . 22958618)>>
It varies. Sometimes I get the error because children is a something like '(23220778 . 22958618), then when I print out the xml value, the root is printed as #<gc-hash-table #3>. So it looks to me like a memory corruption problem, but I don't know why. When I use gdb to look for a cause:
$ gdb gsi GNU gdb (GDB) 7.4 Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later < http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-unknown-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/bin/gsi...(no debugging symbols found)...done. (gdb) run Starting program: /usr/bin/gsi Gambit v4.6.3
(load "xml") "/home/taylor/Repos/gambit-xml/xml.o1" (define xml (read-xml "./test.xml")) (length (xml-element-children (xml-document-root xml)))
Program received signal SIGSEGV, Segmentation fault. 0x00007ffff702b064 in ___H_xml_2d_element_2d_children (___ps=0xa4ce20) at xml.c:6232 6232 ___IF(___STRUCTUREDIOP(___R1,___R2)) (gdb) bt #0 0x00007ffff702b064 in ___H_xml_2d_element_2d_children (___ps=0xa4ce20) at xml.c:6232 #1 0x00000000004ad51f in ___call () #2 0x00000000004af14f in ___setup () #3 0x00000000004c3ef2 in ___main () #4 0x00000000004b8436 in ___main_char () #5 0x00000000004ac6a9 in main () (gdb) print ___ps->r $1 = {10259873, 11115697, 140737339703681, 11115697, 140737339703681}
The problem manifests in code generated by Gambit, but I doubt that's where the problem originates. I don't know enough about the Gambit internals to make it any further on my own. I hope somebody can help guide me in debugging what's wrong, and hopefully point out whether my memory handling in the os_xml.c file is correct or not. There are a few places where I have doubts about whether I should use ___release_scmobj() or not.
All of these problems happen fairly regularly on my desktop, which is an Intel Core i7-870 with 8GB of memory running Arch Linux x86_64 kernel 3.2.4 --- but I haven't yet had them on my laptop, which is an Intel Core 2 Duo P8800 with 4GB of memory running the same kernel.
Mighty thanks for any assistance!
-- Taylor C. Venable http://metasyntax.net/
On 2012-02-10, at 2:01 PM, Taylor Venable wrote:
Thanks to Valgrind, I found my mistake. I made a vector with size two, then put three items into it. Specifically, the root node went into index 2, which is why that field of the xml-document was the one with the bug. That silly problem fixed, things seem to be working fine. But if anybody has a spare minute and would be willing to check over my code, I would still appreciate any feedback about whether I'm doing things the right way. Especially when it comes to memory management. Thanks!
I'm glad you found the bug. I looked at your code yesterday and all the allocations seem correct (use of ___STILL objects, and appropriate reference count management). It seems I missed the "out of bound" index which was the problem. Marc
Hi Taylor, Do you plan on making full bindings for libxml2? I'd be interested in packaging it as a Blackhole module when you are ready. Also, when following your steps I got this error:
(load "xml") *** WARNING -- Could not find C function: "____20_xml_2e_o1" *** ERROR IN (console)@1.1 -- /home/alvatar/gambit-xml/xml.o1: undefined symbol: ___release_scmobj
Using gambit 4.6.3 Best regards, Álvaro On Fri, Feb 10, 2012 at 8:09 PM, Marc Feeley <feeley@iro.umontreal.ca>wrote:
On 2012-02-10, at 2:01 PM, Taylor Venable wrote:
Thanks to Valgrind, I found my mistake. I made a vector with size two, then put three items into it. Specifically, the root node went into index 2, which is why that field of the xml-document was the one with the bug. That silly problem fixed, things seem to be working fine. But if anybody has a spare minute and would be willing to check over my code, I would still appreciate any feedback about whether I'm doing things the right way. Especially when it comes to memory management. Thanks!
I'm glad you found the bug. I looked at your code yesterday and all the allocations seem correct (use of ___STILL objects, and appropriate reference count management). It seems I missed the "out of bound" index which was the problem.
Marc
_______________________________________________ Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
On 2012-02-12, at 8:43 AM, Álvaro Castro-Castilla wrote:
Hi Taylor,
Do you plan on making full bindings for libxml2? I'd be interested in packaging it as a Blackhole module when you are ready. Also, when following your steps I got this error:
(load "xml") *** WARNING -- Could not find C function: "____20_xml_2e_o1" *** ERROR IN (console)@1.1 -- /home/alvatar/gambit-xml/xml.o1: undefined symbol: ___release_scmobj
Using gambit 4.6.3
Best regards,
Álvaro
I'm not sure this is the cause of the bug, but I have noticed that Taylor's code is not completely portable. It contains references to Gambit runtime functions that are not wrapped in a call to ___EXT. In other words, the calls vect = ___make_vector(4, ___FAL, ___STILL); ___release_scmobj(x); tmp = ___make_pair(x, lst, ___STILL); should be written vect = ___EXT(___make_vector)(4, ___FAL, ___STILL); ___EXT(___release_scmobj)(x); tmp = ___EXT(___make_pair)(x, lst, ___STILL); This is needed on some operating systems to link with the Gambit runtime system functions. On the operating systems that don't need anything special, the call ___EXT(f) is replaced with f, so there is no overhead. By the way, on which OS did this problem occur? Marc
Marc, if you mean my error, it was on Linux 3.1 (Gentoo), and happens also on latest Ubuntu on VirtualBox. 2012/2/13 Marc Feeley <feeley@iro.umontreal.ca>
On 2012-02-12, at 8:43 AM, Álvaro Castro-Castilla wrote:
Hi Taylor,
Do you plan on making full bindings for libxml2? I'd be interested in packaging it as a Blackhole module when you are ready. Also, when following your steps I got this error:
(load "xml") *** WARNING -- Could not find C function: "____20_xml_2e_o1" *** ERROR IN (console)@1.1 -- /home/alvatar/gambit-xml/xml.o1: undefined symbol: ___release_scmobj
Using gambit 4.6.3
Best regards,
Álvaro
I'm not sure this is the cause of the bug, but I have noticed that Taylor's code is not completely portable. It contains references to Gambit runtime functions that are not wrapped in a call to ___EXT. In other words, the calls
vect = ___make_vector(4, ___FAL, ___STILL); ___release_scmobj(x); tmp = ___make_pair(x, lst, ___STILL);
should be written
vect = ___EXT(___make_vector)(4, ___FAL, ___STILL); ___EXT(___release_scmobj)(x); tmp = ___EXT(___make_pair)(x, lst, ___STILL);
This is needed on some operating systems to link with the Gambit runtime system functions. On the operating systems that don't need anything special, the call ___EXT(f) is replaced with f, so there is no overhead.
By the way, on which OS did this problem occur?
Marc
Can you try the change I suggest to see if the problem goes away? Marc On 2012-02-13, at 5:59 AM, Álvaro Castro-Castilla wrote:
Marc, if you mean my error, it was on Linux 3.1 (Gentoo), and happens also on latest Ubuntu on VirtualBox.
2012/2/13 Marc Feeley <feeley@iro.umontreal.ca>
On 2012-02-12, at 8:43 AM, Álvaro Castro-Castilla wrote:
Hi Taylor,
Do you plan on making full bindings for libxml2? I'd be interested in packaging it as a Blackhole module when you are ready. Also, when following your steps I got this error:
(load "xml") *** WARNING -- Could not find C function: "____20_xml_2e_o1" *** ERROR IN (console)@1.1 -- /home/alvatar/gambit-xml/xml.o1: undefined symbol: ___release_scmobj
Using gambit 4.6.3
Best regards,
Álvaro
I'm not sure this is the cause of the bug, but I have noticed that Taylor's code is not completely portable. It contains references to Gambit runtime functions that are not wrapped in a call to ___EXT. In other words, the calls
vect = ___make_vector(4, ___FAL, ___STILL); ___release_scmobj(x); tmp = ___make_pair(x, lst, ___STILL);
should be written
vect = ___EXT(___make_vector)(4, ___FAL, ___STILL); ___EXT(___release_scmobj)(x); tmp = ___EXT(___make_pair)(x, lst, ___STILL);
This is needed on some operating systems to link with the Gambit runtime system functions. On the operating systems that don't need anything special, the call ___EXT(f) is replaced with f, so there is no overhead.
By the way, on which OS did this problem occur?
Marc
hum... I still get the same *** WARNING -- Could not find C function: "____20_xml_2e_o1" *** ERROR IN (console)@1.1 -- /home/alvatar/gambit-xml/xml.o1: undefined symbol: ___release_scmobj I've surrounded all those calls with ___EXT() 2012/2/13 Marc Feeley <feeley@iro.umontreal.ca>
Can you try the change I suggest to see if the problem goes away?
Marc
On 2012-02-13, at 5:59 AM, Álvaro Castro-Castilla wrote:
Marc, if you mean my error, it was on Linux 3.1 (Gentoo), and happens also on latest Ubuntu on VirtualBox.
2012/2/13 Marc Feeley <feeley@iro.umontreal.ca>
On 2012-02-12, at 8:43 AM, Álvaro Castro-Castilla wrote:
Hi Taylor,
Do you plan on making full bindings for libxml2? I'd be interested in packaging it as a Blackhole module when you are ready. Also, when following your steps I got this error:
(load "xml") *** WARNING -- Could not find C function: "____20_xml_2e_o1" *** ERROR IN (console)@1.1 -- /home/alvatar/gambit-xml/xml.o1: undefined symbol: ___release_scmobj
Using gambit 4.6.3
Best regards,
Álvaro
I'm not sure this is the cause of the bug, but I have noticed that Taylor's code is not completely portable. It contains references to Gambit runtime functions that are not wrapped in a call to ___EXT. In other words, the calls
vect = ___make_vector(4, ___FAL, ___STILL); ___release_scmobj(x); tmp = ___make_pair(x, lst, ___STILL);
should be written
vect = ___EXT(___make_vector)(4, ___FAL, ___STILL); ___EXT(___release_scmobj)(x); tmp = ___EXT(___make_pair)(x, lst, ___STILL);
This is needed on some operating systems to link with the Gambit runtime system functions. On the operating systems that don't need anything special, the call ___EXT(f) is replaced with f, so there is no overhead.
By the way, on which OS did this problem occur?
Marc
Can you pass a dump of what C compiler& linker invocations are being made? You do this by ensuring that in Gambit's OS environment there's the var GAMBC_CC_VERBOSE set to true, so in bash, GAMBC_CC_VERBOSE=TRUE gsc (The verbosity stuff is effectuated by the gambc-cc shellscript.) Den 13 februari 2012 22:00 skrev Álvaro Castro-Castilla < alvaro.castro.castilla@gmail.com>:
hum... I still get the same *** WARNING -- Could not find C function: "____20_xml_2e_o1" *** ERROR IN (console)@1.1 -- /home/alvatar/gambit-xml/xml.o1: undefined symbol: ___release_scmobj
I've surrounded all those calls with ___EXT()
2012/2/13 Marc Feeley <feeley@iro.umontreal.ca>
Can you try the change I suggest to see if the problem goes away?
Marc
On 2012-02-13, at 5:59 AM, Álvaro Castro-Castilla wrote:
Marc, if you mean my error, it was on Linux 3.1 (Gentoo), and happens also on latest Ubuntu on VirtualBox.
2012/2/13 Marc Feeley <feeley@iro.umontreal.ca>
On 2012-02-12, at 8:43 AM, Álvaro Castro-Castilla wrote:
Hi Taylor,
Do you plan on making full bindings for libxml2? I'd be interested in packaging it as a Blackhole module when you are ready. Also, when following your steps I got this error:
(load "xml") *** WARNING -- Could not find C function: "____20_xml_2e_o1" *** ERROR IN (console)@1.1 -- /home/alvatar/gambit-xml/xml.o1: undefined symbol: ___release_scmobj
Using gambit 4.6.3
Best regards,
Álvaro
I'm not sure this is the cause of the bug, but I have noticed that Taylor's code is not completely portable. It contains references to Gambit runtime functions that are not wrapped in a call to ___EXT. In other words, the calls
vect = ___make_vector(4, ___FAL, ___STILL); ___release_scmobj(x); tmp = ___make_pair(x, lst, ___STILL);
should be written
vect = ___EXT(___make_vector)(4, ___FAL, ___STILL); ___EXT(___release_scmobj)(x); tmp = ___EXT(___make_pair)(x, lst, ___STILL);
This is needed on some operating systems to link with the Gambit runtime system functions. On the operating systems that don't need anything special, the call ___EXT(f) is replaced with f, so there is no overhead.
By the way, on which OS did this problem occur?
Marc
_______________________________________________ Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
On 2012-02-13, at 3:00 PM, Álvaro Castro-Castilla wrote:
hum... I still get the same *** WARNING -- Could not find C function: "____20_xml_2e_o1" *** ERROR IN (console)@1.1 -- /home/alvatar/gambit-xml/xml.o1: undefined symbol: ___release_scmobj
I've surrounded all those calls with ___EXT()
It sounds like you compiled Gambit with a C++ compiler, and your file with a C compiler. I'm just guessing. Marc
You are right, that was it. Thank you. 2012/2/13 Marc Feeley <feeley@iro.umontreal.ca>
On 2012-02-13, at 3:00 PM, Álvaro Castro-Castilla wrote:
hum... I still get the same *** WARNING -- Could not find C function: "____20_xml_2e_o1" *** ERROR IN (console)@1.1 -- /home/alvatar/gambit-xml/xml.o1: undefined symbol: ___release_scmobj
I've surrounded all those calls with ___EXT()
It sounds like you compiled Gambit with a C++ compiler, and your file with a C compiler. I'm just guessing.
Marc
2012/2/12 Álvaro Castro-Castilla <alvaro.castro.castilla@gmail.com>:
Do you plan on making full bindings for libxml2? I'd be interested in packaging it as a Blackhole module when you are ready.
Right now I'm focusing on simple parsing, printing, and walking the tree. But I think that going into the other areas (e.g. xpath) and expanding the interface are very worthy goals. Having a fast, solid XML library for Gambit is a good thing, even if it's based in C. Another thing that would be nice to have is pure Scheme versions of the libxml2 code, so that if you wanted not to deal with compiling libxml2 you can just use only Scheme code. Sort of a pluggable backend for the heavy lifting. I haven't looked much at Black Hole but having support for it would probably be good. Thanks for your interest. -- Taylor C. Venable http://metasyntax.net/
Great! So, what's different from http://ssax.sourceforge.net/? There is a version for Gambit somewhere, and I've been using it with Blackhole: https://github.com/alvatarc/caad-research/tree/master/web/parse/ssax-sxml 2012/2/14 Taylor Venable <taylor@metasyntax.net>
2012/2/12 Álvaro Castro-Castilla <alvaro.castro.castilla@gmail.com>:
Do you plan on making full bindings for libxml2? I'd be interested in packaging it as a Blackhole module when you are ready.
Right now I'm focusing on simple parsing, printing, and walking the tree. But I think that going into the other areas (e.g. xpath) and expanding the interface are very worthy goals. Having a fast, solid XML library for Gambit is a good thing, even if it's based in C.
Another thing that would be nice to have is pure Scheme versions of the libxml2 code, so that if you wanted not to deal with compiling libxml2 you can just use only Scheme code. Sort of a pluggable backend for the heavy lifting.
I haven't looked much at Black Hole but having support for it would probably be good.
Thanks for your interest.
-- Taylor C. Venable http://metasyntax.net/
2012/2/14 Álvaro Castro-Castilla <alvaro.castro.castilla@gmail.com>:
So, what's different from http://ssax.sourceforge.net/?
I'm really sorry, I must have missed this message when it came in. Projecting out a bit, I would say that: * This will eventually support more features of XML than SSAX, because from what I can tell libxml2 can handle more (e.g. xinclude) than SSAX. * This should be faster and use less memory for parsing large XML files, even today, because the parser is written entirely in C. Although that's purely speculation and I might be wrong. * Today, SSAX is a much better choice than what I'm working on, for 99.9% of all applications. :-) Also, there's no event-driven model of parsing with my code currently --- so SSAX right now is the only one that does... SAX. I'd like to work on that over the next couple weeks. -- Taylor C. Venable http://metasyntax.net/
On Fri, Feb 10, 2012 at 14:09, Marc Feeley <feeley@iro.umontreal.ca> wrote:
I'm glad you found the bug. I looked at your code yesterday and all the allocations seem correct (use of ___STILL objects, and appropriate reference count management). It seems I missed the "out of bound" index which was the problem.
Thanks for the review! Do you think it would be a good idea to update the wiki with some documentation about using the C macros/functions like ___TRU, ___FAL, ___FIX, ___make_pair, ___make_vector, ___release_scmobj, etc. so that others can have an easier time building more complicated FFI code? I'd be happy to do it. These seem to be fairly lightly documented in the manual. The method of turning vectors into structures is especially useful, but I only found out about it by reading lib/os.c Best regards, -- Taylor C. Venable http://metasyntax.net/
participants (4)
-
Marc Feeley -
Mikael -
Taylor Venable -
Álvaro Castro-Castilla