New to Gambit, old to Scheme and Lisp (bit I've been away from actively using them for decades, and much has changed).
I'm refamiliarising myself, taking a small project that I could write in several other languages in an hour, but doing it as learning project instead of a quickie implementation, finding out what's available nowadays.
The project is to take some overly dictatorial HTML and liberate it. You know the kind of thing, take out all the excess <br>'s and  .'s, figure out the real structure they're hiding, and reemit it using <p> and <h5> and other proper tags.
Now I can do all this from scratch, reading characters from ports and writing code to lex it and parse it and recognise idioms and such ... but that's so nose-to-the-grindstone.
Isn't the whole point of these new Scheme systems that we can work at a more conceptual level?
So I ask. What tools and libraries are already there to make this kind of task easier? Or more elaborate tasks of this kind -- because I will run into them later. I look in the web and find myself lost in beginner documentation. I guess I'm looking for links to heavier stuff.
-- hendrik
Afficher les réponses par date
On 2011-06-04, at 4:20 PM, Hendrik Boom wrote:
New to Gambit, old to Scheme and Lisp (bit I've been away from actively using them for decades, and much has changed).
I'm refamiliarising myself, taking a small project that I could write in several other languages in an hour, but doing it as learning project instead of a quickie implementation, finding out what's available nowadays.
The project is to take some overly dictatorial HTML and liberate it. You know the kind of thing, take out all the excess <br>'s and  .'s, figure out the real structure they're hiding, and reemit it using <p> and <h5> and other proper tags.
Now I can do all this from scratch, reading characters from ports and writing code to lex it and parse it and recognise idioms and such ... but that's so nose-to-the-grindstone.
Isn't the whole point of these new Scheme systems that we can work at a more conceptual level?
So I ask. What tools and libraries are already there to make this kind of task easier? Or more elaborate tasks of this kind -- because I will run into them later. I look in the web and find myself lost in beginner documentation. I guess I'm looking for links to heavier stuff.
-- hendrik
If you want to parse HTML in Scheme, let me recommend htmlprag (http://www.neilvandyke.org/htmlprag/) which is very portable.
If you want to generate a lexer or parser yourself, check out Silex and LALR-SCM (Scheme equivalents of lex and yacc), which were developed at the University of Montreal by Danny Dubé and Dominique Boucher respectively (http://www.iro.umontreal.ca/~dube/ and http://code.google.com/p/lalr-scm/).
Marc
On Sat, Jun 04, 2011 at 06:44:57PM -0400, Marc Feeley wrote:
If you want to parse HTML in Scheme, let me recommend htmlprag (http://www.neilvandyke.org/htmlprag/) which is very portable.
This looks like the right kind of thing. Thanks.
There's also, I've been advised, the html package for PLT Scheme, which seems to make extensive use of some kind of inherited object types. htmlprag looks simpler.
Next I suppose I get to wonder whether all this data structure will fit in memory. I have about 8 megabytes of the stuff, and a 1G machine, but the largest individual file seems about 350K. Probably OK, unless gambit is very wasteful of space. I remember Java was terrible at storing small strings. Scheme should be able to do better.
-- hendrik
If you want to generate a lexer or parser yourself, check out Silex and LALR-SCM (Scheme equivalents of lex and yacc), which were developed at the University of Montreal by Danny Dubé and Dominique Boucher respectively (http://www.iro.umontreal.ca/~dube/ and http://code.google.com/p/lalr-scm/).
No, that's not what I'm after -- not this time, anyway.
Marc
On Sat, Jun 04, 2011 at 10:09:52PM -0400, Hendrik Boom wrote:
On Sat, Jun 04, 2011 at 06:44:57PM -0400, Marc Feeley wrote:
If you want to parse HTML in Scheme, let me recommend htmlprag (http://www.neilvandyke.org/htmlprag/) which is very portable.
This looks like the right kind of thing. Thanks.
There's also, I've been advised, the html package for PLT Scheme, which seems to make extensive use of some kind of inherited object types. htmlprag looks simpler.
Next I suppose I get to wonder whether all this data structure will fit in memory. I have about 8 megabytes of the stuff, and a 1G machine, but the largest individual file seems about 350K. Probably OK, unless gambit is very wasteful of space. I remember Java was terrible at storing small strings. Scheme should be able to do better.
-- hendrik
For the record, I ended up with a sed script, followed by hand-editing the input to the sed script. This wasn't what I had in mind at all when I phrased the original query here, but it worked because the horrible HTML had a very simple pattern.
I don't think it'll work for the next conversion I have to do, though, which looks like it has complex CSS applied to complicated HTML or XML. The CSS is aimed at making things look good in a web browser, with links and stuff. EPUB will probably want it all to be different. Maybe some of these parsers I've been pointed to will still see some real use.
-- hendrik
If you want to generate a lexer or parser yourself, check out Silex and LALR-SCM (Scheme equivalents of lex and yacc), which were developed at the University of Montreal by Danny Dubé and Dominique Boucher respectively (http://www.iro.umontreal.ca/~dube/ and http://code.google.com/p/lalr-scm/).
No, that's not what I'm after -- not this time, anyway.
Marc
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
On Thu, Jul 21, 2011 at 10:43:59PM -0400, Hendrik Boom wrote:
I don't think sed will work for the next conversion I have to do, though, which looks like it has complex CSS applied to complicated HTML or XML. The CSS is aimed at making things look good in a web browser, with links and stuff. EPUB will probably want it all to be different. Maybe some of these parsers I've been pointed to will still see some real use.
What I suspect I'll want is something like regular expresssions, but applied to a stream of 'things' instead of characters. Initially, the things will be incoming tokens of various kinds (such as HTML tags) but as the process proceeds, those will be replaced by more complicatted things, such as XMLisg tree structures.
Ideallly, such a tool should be designed to be more or less independent of the kinds of things that cound as 'things'.
Of course I could be just as much off the mark with this as I was thinking I wanted to use Scheme instead of sed for the first conversion. But I'm pretty sure this time that sed won't do.
But I'm sure that there are other conversin tools around too, such as calibre. Baybe I still won't get around to usiing Scheme. :-(
-- hendrik
-
Hendrik Boom -
Marc Feeley