Rouan van Dalen wrote:

I was wondering what your thoughts are on this and if something like this already exists. I definitely need support for the latest MySQL database server and I am thinking of writing an HTTP server specifically for Gambit to form part of the web framework.

Choosing Gambit for me has been the result of looking for the best language and implementation for rewriting an existing web application (and writing new ones).

The first thing that has been missing has been a module system which supports runtime (re)loading of code into the application. That's why I did write "chjmodule" (note my recent comment about me wanting to release a new version and give access to my repo soon).

I also took Marc's http server example and split it into several "chjmodule" modules.

Sadly, the customer whose web application I wanted to rewrite, choose to give it into other hands instead (not for technical reasons, as has been said explicitely). As a result of this, my work life shifted from writing web applications to other things. While I still would like to 'finish' a web application framework (and would like to write a few small things in the next few months), the priorities have changed. I'll now rather invest more time to get things right (the way I want them) than create a complete framework quickly.

Actually we (me and a co-worker) wrote a tiny application using not the http server, but fastcgi instead, by going through Perl's fastcgi support (through the use of "gperl", my Gambit perl bindings).

I did spend quite some time thinking about how to design the web framework. My aim is to have an http server in one Gambit process (using Gambit's builtin threading), but do the actual processing in a slew of separate "worker" processes (possibly running under different unix uid's). The reasons are: not to block other clients while calling C code, not to affect other clients when a process crashes, to make use of multiple processors, and to increase security (by using a separate uid for each logged-in user, or switching off traceability under Linux). The http front end process would parse the request and decide upon the possibly contained user id and continuation id which worker process to dispatch it to. The worker processes are created by a worker supervisor (which is (possibly) running as root, forks and then changes uid to the respective user)). Communication occurs through a command pipe or a (unix/IP) socket (possibly using Termite, maybe with modifications), as a message (containing the request header) and possibly a separate body pipe/socket (containing the request body). The worker process will then create the response and feed it back to the front end either through a pipe/socket (if it's endless data) or through a temporary file (tmpfs) if the length is finite and small enough; the latter approach is better for freeing up the worker child quickly by being able to spool data faster than the client accepts (to keep the number of separate unix processes small), and optionally allows for caching the response at the same time; the http frontend would poll the file (or possibly use inotify/dnotify) for new chunks to be delivered, and get an end-of-request back from the child when it's finished, this is fast and allows the worker process to use normal IO routines (including charset conversions), without worrying about http chunking etc. (I actually also expect this to be rather more efficient than trying to pipeline page creation and delivery (e.g. using object pipes) in one process). BTW recently I've read that Paul Graham used about the same approach for Viaweb (using another clisp process for each client). This did actually surprise me because until now I thought of common lisp as "forget about the host OS, do everything in the CL system" way of doing things, and have got my worries about how I'd debug an application where processing is distributed across several OS processes (maybe even machines); seems Graham&Co. already solved that problem, and Erlang and Termite will have found or will find ways to solve that too.

I'm coming from an XML/XSLT background (AxKit); we did write a helper pseudo-XSLT library which we used to rewrite existing XSLT stylesheets to Scheme almost 1:1 (I've been aware of SXSLT library but writing that library ourselves seemed quicker than trying to understand SXSLT and was a good learning experience). I did also write an SXML serializer tuned for speed.

In the pseudo-XSLT helper library, we're making use of delayed evaluation (streams). This made me realize that in the longer term, I want lazy-by-default scheme modules (using a transformer as chjmodule plugin, with the ability to see and debug the resulting strict scheme code).

1. Is there any interest in something like this?

Yes there is, albeit I've got my own ideas how things should work as pointed out above, but I'll always be glad if I can use some piece of existing software (if it doesn't mean it locks me in or requires big time investments to get to know without paying off).

Note that Chicken has got quite many eggs in the web app space recently. You may want to look at what's there (and maybe [help] turn them into Snow packages, or at least get an overview on what's going on). I also think Askemos is interesting (but I haven't gotten around looking at it closely yet).

2. Does MySQL5 bindings exist for Gambit? (if not I will gladly write them)

You should be aware that in Gambit (as in many other language implementations), other Gambit threads aren't being scheduled while you run C code. But with my above described design that's no problem.

A more serious drawback of MySQL (except if they changed that in release 5) for me is that once a query has been issued, no new query can be issued before the previous result set has been read completely (and there are (have been?) no cursors); if you want to use functional streams for representing result sets (lazy lists) like me, that's a no-go (there's no point using lazy evaluation if you have to read in everything anyway).

See also:

http://bendiken.net/2006/06/11/poor-mans-mysql-api-for-gambit

https://webmail.iro.umontreal.ca/pipermail/gambit-list/2006-August/000782.ht... and http://blog.jonnay.net/archives/779-Version-0.3-of-the-MySQL-FFI-for-Gambit-...

(and I think I saw someone reimplementing the MySQL protocol in Gambit? Or maybe it was Postgres, by Dominique Boucher. Maybe I saw someone writing the MySQL protocol in another Scheme or language.)

I am also interested in expanding Gambit's libraries, so there is no problem for me to write libraries/bindings for various things..

(Honestly, I'm a bit hesitant when it comes to writing libraries. I've got lying around 150 "chjmodule" modules of my own writing right now. I really want to get module parametrization (and probably also lazy-by-default and other niceties like proper cross-module inlining declarations, stacked language layers and more) before I seriously continue writing modules, because if I do it the other way round, I'll have to rewrite more and more code once parametrization and lazyness is in place. At the same time that I think I need parametrized modules so soon, I hope R6RS doesn't catch on; at the same time that I'm tempted to say we should use a more powerful module system (and probably a common one, not one I'm just writing for my own use..) I hope that I (and maybe we all) get the chance of writing the right module system "ourselfes" (in the community). Or maybe rather, have a multitude of module styles but have them interoperate. Anyway it's time for me to make a more powerful modulesystem before I invest too much into writing more actual modules. I guess what I want to say is, go ahead, but be prepared that the way you write them now might not the final way (well, in any case Snow is probably a good starting point).)

Cheers, Christian.

On 1-Nov-07, at 12:18 PM, Rouan van Dalen wrote:

I am looking to build a web framework on the same level as Rails, only I want to be programming in Scheme.

May I suggest the name "Scheme on skis"?

So I need 2 questions answered:

1. Is there any interest in something like this?

For sure. Many people have rolled their own incomplete solutions (aka hacks). Perhaps you can distill something more complete by examining those solutions. I can contribute my Scheme to Javascript compiler. There is also the Mobit interpreter which could prove useful here.

2. Does MySQL5 bindings exist for Gambit? (if not I will gladly

write them)

I seem to remember someone contributed MySQL bindings to the mailing list at some point.

Marc