Hi Snowballers, (I sent a message to this list a while ago about this but the thread died.) Oleg Kiselyov has written an excellent XML parsing system for Scheme, and it's been ported to a number of interpreters -- in fact, pretty much every interpreter that Snow currently supports (with the exception, maybe, of STklos). I think it'd be really useful to have some XML parsing packages included in Snow, and SSAX is pretty ubiquitous in the Scheme world. When I brought it up on this list a few months ago, Marc Feeley suggested that I get in touch with Oleg and ask him what his preferences are for porting. His response was:
"I guess more like `hands off'... I try to keep the core SSAX relatively portable. [...] It's up to the developers of a particular Scheme system and its community to decide how to treat SSAX."
Perhaps it might be useful to contact the maintainers of the various ports. They are, as nearly as I can figure out:
* Dmitry Lizorkin (dmitry.lizorkin@gmail.com) - Bigloo; Chicken; Gambit; PLT/MzScheme * Shiro Kawai (shiro@lava.net) - Gauche * Andy Wingo (wingo@pobox.com) - Guile * Mike Sperber (sperber@informatik.uni-tuebingen.de) - SCSH; Scheme48
If these maintainers want to participate, that'd be great -- but I recently took a look at some of the different ports with the intent to make the Snowball myself, and they're all pretty divergent in terms of the exports they provide and how the code is organized. The problem, I think, is that SSAX is not just an XML parser, it's a collection of functions for making parsers and reading text, and different distributions have made different decisions as to how to present the parsing system to the user. It's not clear to me how to proceed, but I'd love to discuss this further...
Regards, Julian
Afficher les réponses par date
I'd love to see SXML/SSAX as a snow package. Being new to scheme, it tooked me a lot of time to figure out how to install it for SCSH and Scheme48.
All-things that get to root of making W3C-ish stuff better in portable Scheme are, in some sense, worth a heck of a lot.
Scheme traditionally spends a lot of effort getting right very general, widely applicable numerical data; XML data and the protocols associated with it deserve just about as much attention, and for similar reasons (in my opinion).
-t
Ezequiel Birman wrote:
I'd love to see SXML/SSAX as a snow package. Being new to scheme, it tooked me a lot of time to figure out how to install it for SCSH and Scheme48.
Glad you guys agree -- can I perhaps interest you in a discussion of the technical details of such a package?
Given that there's a disparity in terms of exports among the various distributions for different Scheme platforms, I think it might be wise to choose a particular distribution and then re-port it to the requisite interpreters. The distribution I'm most familiar with is the one in guile-lib (http://home.gna.org/guile-lib/), but, unfortunately, it's a bit messy under the hood, since it includes, I think (could be wrong), an old stable release of SSAX coupled with some upstream work, so I don't know if it makes the easiest base for doing a port. Does anyone have any experience with any of the other ones, like PLT's?
The SSAX functionality that's most important to me is:
* ssax:xml->sxml (obviously) * The ssax:make-parser syntax, for customizing parser behavior * The utility functions used by the default parser, such as ssax:skip-internal-dtd, ssax:read-pi-body-as-string, etc., since these make customizing the parser much, much easier.
Thoughts?
On 7/12/07, Thomas Lord lord@emf.net wrote:
All-things that get to root of making W3C-ish stuff better in portable Scheme are, in some sense, worth a heck of a lot.
Scheme traditionally spends a lot of effort getting right very general, widely applicable numerical data; XML data and the protocols associated with it deserve just about as much attention, and for similar reasons (in my opinion).
-t
Ezequiel Birman wrote:
I'd love to see SXML/SSAX as a snow package. Being new to scheme, it tooked me a lot of time to figure out how to install it for SCSH and Scheme48.
Snow-users-list mailing list Snow-users-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/snow-users-list
Julian Graham wrote:
Glad you guys agree -- can I perhaps interest you in a discussion of the technical details of such a package?
See, it was on purpose I didn't go there. I didn't want to intimidate you or scare you away from SSAX which I'm sure can be helpful almost regardless of what it does.
But, on my wish list:
~ both streaming and DOMish APIs
~ get the Unicode processing within such perfect
~ get namespace handling perfect
~ don't forget to properly handle processing instructions and comments
~ handle insignificant whitespace well (see below about representation-agnostic...)
~ get the parsing and printing perfect
~ include an accurate XML pretty-printer
~ performance, space and time, are critical
~ DOM and SAX models should be representation-agnostic: ~~ handle in-core, abstractly represented datums ~~ make it possible to read and pretty print datums while PRESERVING insignificant whitespace! ~~ allow DOM and SAX models to refer to external, lazily-read resources -- e.g., I want to use DOM over a datum that is still largely on disk
~ XPATH-based accessors are critical. They should work well for XPATHs generated on-the-fly at run-time. They should work even better if the query is known ahead of time and can be compiled.
~ XSLT and XQUERY: these matter, a lot, and in theory Scheme engines should really shine here (perhaps see what Per Bothner's been up to)
~ Schema checking (DTD and XSchema) matters a lot -- and should be implemented very accurately
~ A loosey-goosey "approximate HTML" parser is critical
~ An HTML pretty-printer that satisfies common browsers is critical -- c.f. the html output mode of XSLT processors
~ the nasty hard part: interoperability with foreign functions matters (but probably can't be done perfectly). Have a look at Xerxes and XQilla, for example. This is a real hard problem for HLLs that like to be combined with foreign C and C++ libraries: too many same-but-different DOM APIs in one process.
~ it should all be very fast
~ probably will take a week or two to write :-)
Given that there's a disparity in terms of exports among the various distributions for different Scheme platforms, I think it might be wise to choose a particular distribution and then re-port it to the requisite interpreters. The distribution I'm most familiar with is the one in guile-lib (http://home.gna.org/guile-lib/), but, unfortunately, it's a bit messy under the hood, since it includes, I think (could be wrong), an old stable release of SSAX coupled with some upstream work, so I don't know if it makes the easiest base for doing a port. Does anyone have any experience with any of the other ones, like PLT's?
The SSAX functionality that's most important to me is:
- ssax:xml->sxml (obviously)
- The ssax:make-parser syntax, for customizing parser behavior
- The utility functions used by the default parser, such as
ssax:skip-internal-dtd, ssax:read-pi-body-as-string, etc., since these make customizing the parser much, much easier.
Thoughts?
I have experience with a few 80% solutions in other languages. I have a decent amount of experience with XML generally.
You frighten me with your eagerness to customize parsers -- that seems to rather miss the point of an exchange format.
80% solutions can be used for lots of stuff -- almost anything you do will be useful.
80% solutions tend to fall down hard as soon as you run into items from my requirements list above.
Perhaps my serious suggestion is: don't worry so much. Just get a package out there that does any damn thing that might be useful and then go from there. Separate concerns: a snowball for XML processing that happens to have some mud in it is a good place to start -- then separately, start making that snowball really top-notch. Don't make "getting XML in Scheme perfect" a pre-condition for "here's a snowball for XML processors".
-t
On 7/12/07, Thomas Lord lord@emf.net wrote:
All-things that get to root of making W3C-ish stuff better in portable Scheme are, in some sense, worth a heck of a lot.
Scheme traditionally spends a lot of effort getting right very general, widely applicable numerical data; XML data and the protocols associated with it deserve just about as much attention, and for similar reasons (in my opinion).
-t
Ezequiel Birman wrote:
I'd love to see SXML/SSAX as a snow package. Being new to scheme, it tooked me a lot of time to figure out how to install it for SCSH and Scheme48.
Snow-users-list mailing list Snow-users-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/snow-users-list
Snow-users-list mailing list Snow-users-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/snow-users-list
Well said! And I'd go a step further: do it for a single implementation first (your favorite one, or Gambit-C if you don't have one ;-), with an eye on portability.
Dominique
Perhaps my serious suggestion is: don't worry so much. Just
get a package out there that does any damn thing that might
be useful and then go from there. Separate concerns: a snowball
for XML processing that happens to have some mud in it is a
good place to start -- then separately, start making that snowball
really top-notch. Don't make "getting XML in Scheme perfect"
a pre-condition for "here's a snowball for XML processors".
Woah, woah, woah. Slow down. I want to be clear here -- I'm in favor of porting Oleg Kiselyov's SSAX/SXML package (http://okmij.org/ftp/Scheme/xml.html) to Snow. Yes, I realize XML parsing correctness / efficiency / functionality is very important, and that SSAX isn't going to suit the needs of every last-20-percenter out there. But: SSAX is already a very good XML parsing package, and it already works. I don't see any reason it doesn't belong in Snow, especially given that it is kind of the de facto Scheme XML parsing implementation.
And it (or nearby packages) already does a lot of the last-20-percent stuff -- to address a few of the items on Tom Lord's wish list:
* SSAX has a pretty streamy API already, and I've written a DOMish API that goes on top of that (SDOM: http://www.nongnu.org/sdom/). * SSAX's default namespace handling, while a little frustrating, is technically correct. * SSAX's parser is also, I believe, pretty adherent as far as correctness goes. SDOM includes an implementation of the Load/Save DOM recommendation (including doing pretty-prints), though it's not quite complete yet. * SSAX doesn't come with DTD support, but SDOM contains a partial DTD-parsing implementation. * There are Scheme implementations for things like XPATH and XQUERY floating around in SSAX's orbit -- see Oleg's site for links. * For permissive HTML parsing, there's always Neil Van Dyke's HtmlPrag (http://www.neilvandyke.org/htmlprag/), which is SXML-compatible.
I am *way* not arguing for "custom" XML parsers or any kind of standards shear -- those SSAX customization features I was requesting to be present in a Snow port are already features of SSAX's implementation. As I mentioned earlier, it's not just an XML parser, it's also kind of an example of how to build your own parser. The make-parser syntax, if exported, lets users (like me) implement useful (but not technically mandated) things that are lacking in the SSAX distribution, such as DTD parsing and more flexible namespace management, and then plug them into SSAX's vanilla parser.
I actually think we're on the same page here -- unless you seriously object to having SSAX as Snow's go-to XML parser. At the same time, given that, by virtue of the nature of SSAX's codebase, any effort to get it into Snow is going to require some decisions to be made about what parts of it to include, I think it's worth discussing some requirements.
Furthermore, I can't speak for the Snow maintainers, but I suspect that since XML parsing is such a core feature of a code repository like Snow -- which is primarily useful in that it presents a closed system of library dependencies -- that it doesn't make a whole lot of sense to have a bunch of competing implementations in there, each likely to have different syntax, etc.
On 7/13/07, Dominique Boucher dominique.boucher@nuecho.com wrote:
Well said! And I'd go a step further: do it for a single implementation first (your favorite one, or Gambit-C if you don't have one ;-), with an eye on portability.
Dominique
Perhaps my serious suggestion is: don't worry so much. Just
get a package out there that does any damn thing that might
be useful and then go from there. Separate concerns: a snowball
for XML processing that happens to have some mud in it is a
good place to start -- then separately, start making that snowball
really top-notch. Don't make "getting XML in Scheme perfect"
a pre-condition for "here's a snowball for XML processors".
Furthermore, I can't speak for the Snow maintainers, but I suspect that since XML parsing is such a core feature of a code repository like Snow -- which is primarily useful in that it presents a closed system of library dependencies -- that it doesn't make a whole lot of sense to have a bunch of competing implementations in there, each likely to have different syntax, etc.
Please don't think that way.
Given the uncertainties about the designing an XML library, the opposite is probably true: a bunch of competing, needs-driven implementations makes a lot of sense (while we wait for a Scheme genius to plunk down the Right Thing to which everyone naturally chooses to migrate).
I don't mean to argue in favor of sloppiness or design error. I'm just saying that you should think of a package system not as a library of pristine things but rather as a communications medium through which things are exchanged.
Like: if there are design questions about XML libs, and you want to put those before the community, a snowball containing a strawman XML lib is worth 10x an email question about what kind of XML lib the package system should have. At least that's how it works in CPAN et al.
-t
Julian Graham wrote:
Woah, woah, woah. Slow down. I want to be clear here -- I'm in favor of porting Oleg Kiselyov's SSAX/SXML package (http://okmij.org/ftp/Scheme/xml.html) to Snow. Yes, I realize XML parsing correctness / efficiency / functionality is very important, and that SSAX isn't going to suit the needs of every last-20-percenter out there. But: SSAX is already a very good XML parsing package, and it already works. I don't see any reason it doesn't belong in Snow, especially given that it is kind of the de facto Scheme XML parsing implementation.
And it (or nearby packages) already does a lot of the last-20-percent stuff -- to address a few of the items on Tom Lord's wish list:
- SSAX has a pretty streamy API already, and I've written a DOMish API
that goes on top of that (SDOM: http://www.nongnu.org/sdom/).
- SSAX's default namespace handling, while a little frustrating, is
technically correct.
- SSAX's parser is also, I believe, pretty adherent as far as
correctness goes. SDOM includes an implementation of the Load/Save DOM recommendation (including doing pretty-prints), though it's not quite complete yet.
- SSAX doesn't come with DTD support, but SDOM contains a partial
DTD-parsing implementation.
- There are Scheme implementations for things like XPATH and XQUERY
floating around in SSAX's orbit -- see Oleg's site for links.
- For permissive HTML parsing, there's always Neil Van Dyke's HtmlPrag
(http://www.neilvandyke.org/htmlprag/), which is SXML-compatible.
I am *way* not arguing for "custom" XML parsers or any kind of standards shear -- those SSAX customization features I was requesting to be present in a Snow port are already features of SSAX's implementation. As I mentioned earlier, it's not just an XML parser, it's also kind of an example of how to build your own parser. The make-parser syntax, if exported, lets users (like me) implement useful (but not technically mandated) things that are lacking in the SSAX distribution, such as DTD parsing and more flexible namespace management, and then plug them into SSAX's vanilla parser.
I actually think we're on the same page here -- unless you seriously object to having SSAX as Snow's go-to XML parser. At the same time, given that, by virtue of the nature of SSAX's codebase, any effort to get it into Snow is going to require some decisions to be made about what parts of it to include, I think it's worth discussing some requirements.
Furthermore, I can't speak for the Snow maintainers, but I suspect that since XML parsing is such a core feature of a code repository like Snow -- which is primarily useful in that it presents a closed system of library dependencies -- that it doesn't make a whole lot of sense to have a bunch of competing implementations in there, each likely to have different syntax, etc.
On 7/13/07, Dominique Boucher dominique.boucher@nuecho.com wrote:
Well said! And I'd go a step further: do it for a single implementation first (your favorite one, or Gambit-C if you don't have one ;-), with an eye on portability.
Dominique
Perhaps my serious suggestion is: don't worry so much. Just
get a package out there that does any damn thing that might
be useful and then go from there. Separate concerns: a snowball
for XML processing that happens to have some mud in it is a
good place to start -- then separately, start making that snowball
really top-notch. Don't make "getting XML in Scheme perfect"
a pre-condition for "here's a snowball for XML processors".
Snow-users-list mailing list Snow-users-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/snow-users-list
Fair enough, but no one's designing an XML library here. We have an XML library; it's pretty much the only XML library for Scheme; it's been in development and use for at least six years. The only thing I was soliciting was information on which ports of it were the tidiest and which module exports people wanted to see.
On 7/13/07, Thomas Lord lord@emf.net wrote:
Furthermore, I can't speak for the Snow maintainers, but I suspect that since XML parsing is such a core feature of a code repository like Snow -- which is primarily useful in that it presents a closed system of library dependencies -- that it doesn't make a whole lot of sense to have a bunch of competing implementations in there, each likely to have different syntax, etc.
Please don't think that way.
Given the uncertainties about the designing an XML library, the opposite is probably true: a bunch of competing, needs-driven implementations makes a lot of sense (while we wait for a Scheme genius to plunk down the Right Thing to which everyone naturally chooses to migrate).
I don't mean to argue in favor of sloppiness or design error. I'm just saying that you should think of a package system not as a library of pristine things but rather as a communications medium through which things are exchanged.
Like: if there are design questions about XML libs, and you want to put those before the community, a snowball containing a strawman XML lib is worth 10x an email question about what kind of XML lib the package system should have. At least that's how it works in CPAN et al.
-t
Julian Graham wrote:
Woah, woah, woah. Slow down. I want to be clear here -- I'm in favor of porting Oleg Kiselyov's SSAX/SXML package (http://okmij.org/ftp/Scheme/xml.html) to Snow. Yes, I realize XML parsing correctness / efficiency / functionality is very important, and that SSAX isn't going to suit the needs of every last-20-percenter out there. But: SSAX is already a very good XML parsing package, and it already works. I don't see any reason it doesn't belong in Snow, especially given that it is kind of the de facto Scheme XML parsing implementation.
And it (or nearby packages) already does a lot of the last-20-percent stuff -- to address a few of the items on Tom Lord's wish list:
- SSAX has a pretty streamy API already, and I've written a DOMish API
that goes on top of that (SDOM: http://www.nongnu.org/sdom/).
- SSAX's default namespace handling, while a little frustrating, is
technically correct.
- SSAX's parser is also, I believe, pretty adherent as far as
correctness goes. SDOM includes an implementation of the Load/Save DOM recommendation (including doing pretty-prints), though it's not quite complete yet.
- SSAX doesn't come with DTD support, but SDOM contains a partial
DTD-parsing implementation.
- There are Scheme implementations for things like XPATH and XQUERY
floating around in SSAX's orbit -- see Oleg's site for links.
- For permissive HTML parsing, there's always Neil Van Dyke's HtmlPrag
(http://www.neilvandyke.org/htmlprag/), which is SXML-compatible.
I am *way* not arguing for "custom" XML parsers or any kind of standards shear -- those SSAX customization features I was requesting to be present in a Snow port are already features of SSAX's implementation. As I mentioned earlier, it's not just an XML parser, it's also kind of an example of how to build your own parser. The make-parser syntax, if exported, lets users (like me) implement useful (but not technically mandated) things that are lacking in the SSAX distribution, such as DTD parsing and more flexible namespace management, and then plug them into SSAX's vanilla parser.
I actually think we're on the same page here -- unless you seriously object to having SSAX as Snow's go-to XML parser. At the same time, given that, by virtue of the nature of SSAX's codebase, any effort to get it into Snow is going to require some decisions to be made about what parts of it to include, I think it's worth discussing some requirements.
Furthermore, I can't speak for the Snow maintainers, but I suspect that since XML parsing is such a core feature of a code repository like Snow -- which is primarily useful in that it presents a closed system of library dependencies -- that it doesn't make a whole lot of sense to have a bunch of competing implementations in there, each likely to have different syntax, etc.
On 7/13/07, Dominique Boucher dominique.boucher@nuecho.com wrote:
Well said! And I'd go a step further: do it for a single implementation first (your favorite one, or Gambit-C if you don't have one ;-), with an eye on portability.
Dominique
Perhaps my serious suggestion is: don't worry so much. Just
get a package out there that does any damn thing that might
be useful and then go from there. Separate concerns: a snowball
for XML processing that happens to have some mud in it is a
good place to start -- then separately, start making that snowball
really top-notch. Don't make "getting XML in Scheme perfect"
a pre-condition for "here's a snowball for XML processors".
Snow-users-list mailing list Snow-users-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/snow-users-list
Hello,
I've a plan to snowball refactored SSAX-SXML package.
It'll likely include the following components:
1. XML parsing - SSAX, W3C compliant
2. HTML parsing - Neil van Dyke's HtmlPrag
3. SXPath - including full support for W3C XPath syntax and location-step functions in Scheme. Provides expressive power of XQuery.
4. SXLink - W3C compliant, but without support for points and ranges
5. XQuery Update Facility - no XQuery, just updates with Patrick Lehti syntax
6. XSLT-like transformation - but XSLT compatibility is unlikely
7. Descriptive XML schema - DataGuides
8. Serialization to XML and HTML, including pretty printing
9. ... and more :-)
Almost all of the code exists and is used in production for years, so it's mostly about refactoring/optimization/better packaging/easy of use.
API would be streamlined and much smaller comparing to current SSAX-SXML packages.
The package is currently developed and deployed as Chicken egg, but it should also support Bigloo, Gambit and PLT, and Snow looks like a reasonable way to achieve this.
The latest Chicken package at SourceForge http://sourceforge.net/project/showfiles.php?group_id=30687&package_id=1... provides very similar functionality.
We collaborate with Dmitry Lizorkin, and this package may be considered as a lightweight and streamlined version of the packages maintained by him.
It's worth a note that packaging SSAX as a separate package is quite possible and even may be a good idea. In the past SXML-tools and SSAX were packaged as two separate packages, but SXML-tools heavily depended on SSAX because SSAX package included "common library" code, such as string processing and even I/O. The code is very portable but it even comes with its own "local" SRFI-13 :-) I'd love to change this with Snow!
So, hopefully, with Snow this support code may be moved out of SSAX, and SSAX-SXML package may be decomposed as three different packages: 1. SSAX 2. SSAX-independent SXML-tools 3. Other SXML-tools (such as SXLink), this package depends on two others.
If you are interested in snowballing SSAX - please let me know, Dmitry and I will be glad to collaborate.
Best regards, Kirill Lisovsky.
--------------------------------- It's here! Your new message! Get new email alerts with the free Yahoo! Toolbar.
snow-users-list@iro.umontreal.ca