[gambit-list] OpenSSL SSL channels module pre-release and HTTP client early pre-release, attached

Mikael mikael.rcv at gmail.com
Sun Mar 24 13:42:40 EDT 2013


Update: HTTP client now has HTTPS support, and some minor fix.

(import openssl-sack) enables the HTTPS support in http-client.

SOCKS support essentially implemented and to be released someday.

When the IO error handling currently discussed is resolved, multithreaded
HTTP connection reuse can be implemented in a robust, holistic, clean way.
Will be great as by that Gambit finally has a complete HTTP client.

Mikael

2013/3/20 Mikael <mikael.rcv at gmail.com>

> Hi guys,
>
> For anyone interested here's the pre-release version of the OpenSSL SSL
> channels ("openssl") module and an early pre-release of the HTTP client.
>
> (If you read this after May 2013 the attachment is most probably outdated
> and you better look at https://github.com/gambit-community or
> https://github.com/m-i-k-a-e-l/ .)
>
>
> *openssl module*
> The openssl module works extremely well and is mature.
>
> Relative to the amount of functionality provided, extreme effort has been
> put into its making. This is because OpenSSL's documentation on nonblocking
> use is incorrect in several parts, so study of other user implementations
> and practical testing has been required to make it work. Over the course of
> a couple of years, in the range three man weeks has gone into it this way.
>
> Due to that this is the nature of the problem addressed by this module, I
> hope that this openssl module will be the main one for Gambit and that any
> more effort into OpenSSL SSL channel modules in Gambit will go into making
> this one even better.
>
> The current version however relies on a version-unsafe hack for extracting
> the respective TCP port's socket, which possibly would be a feature to be
> included in Gambit, please see the "Next steps for release" section below.
> OpenSSL is let to write and read from the OS socket directly. This saves
> lots of roundtrips and buffering.
>
>
> *http-client module*
> The HTTP client supports the passing of custom method, headers and request
> body, and supports retrieval of status code, headers and request body, and
> it does both these with support for all of HTTP 1.0 and 1.1, so this is all
> there is to the HTTP protocol in itself.
>
> Exports:
>
> (simple-raw-http-request uri #!optional request-body (headers '())) => req
>
> (define-macro (with-http-client-all-response all-response . code)
>   `(apply (lambda (reached-uri result stage status-code headers get-header
> response) , at code) ,all-response))
>
> (define (simple-raw-http-request-resp uri #!optional request-body)
>   (with-http-client-all-response
>    (simple-raw-http-request uri request-body)
>    response))
>
> Use:
>
>
> > (simple-raw-http-request "http://www.iro.umontreal.ca/~gambit/")
> (#<uri #13 scheme: "http" userinfo: #f host: "www.iro.umontreal.ca" port:
> #f path: "/~gambit/" query: () fragment: #f>
>  #t
>  done
>  200
>  (("connection" . "close")
>   ("content-type" . "text/html")
>   ("content-length" . "115")
>   ("accept-ranges" . "bytes")
>   ("etag" . "\"1c93d75-73-4443e1da730c0\"")
>   ("last-modified" . "Mon, 21 Jan 2008 16:38:03 GMT")
>   ("server" . "Apache/2.0.54 (Fedora)")
>   ("date" . "Wed, 20 Mar 2013 19:20:12 GMT"))
>  #<procedure #14 26#get-header>
>  "<HTML>\n<HEAD>\n<META HTTP-EQUIV=REFRESH CONTENT=0;URL=
> http://dynamo.iro.umontreal.ca/~gambit/wiki/>\n</HEAD>\n</HTML>\n")
>
> > (simple-raw-http-request-resp "http://www.iro.umontreal.ca/~gambit/")
> "<HTML>\n<HEAD>\n<META HTTP-EQUIV=REFRESH CONTENT=0;URL=
> http://dynamo.iro.umontreal.ca/~gambit/wiki/>\n</HEAD>\n</HTML>\n"
>
>
> Currently it supports single threaded execution i.e. it opens HTTP
> connection, makes request, retrieves response and closes the connection in
> a sequence and there's no connection reuse - reuse and multiplexing across
> threads as well as following redirects will be implemented. Also invocation
> of several declared callbacks is not done currently, and will be trivial to
> implement.
>
> The DSL uses Sack's idea of providing all protocol functionality in a
> 'core' form. It can extremely easily be wrapped to a XMLHTTPRequest object.
>
> This http-client module replaces Sack's previous http-client module, which
> unlike Sack's web server module which works great, had terrible problems,
> for more info see the header section in http-client.scm .
>
>
>
>
> *Next steps for release*
> I'll make a "real" release of these two modules when the IO exception
> handling question discussed on the list now has got an answer and maybe
> extraction of TCP port socket can be done with a Gambit-builtin procedure.
> These two things currently bring version-unsafe hacks into the code, more
> specifically into tcpip.scm and sack/src/gamgambit-noexceptions-io.scm ,
> so this is why I call this a pre-release.
>
> (Note that port-wait-for-input and port-wait-for-output of the tcpipmodule are not needed anymore as the firstmentioned has been in Gambit all
> the time and the latter was recently included in Gambit, ##-prefixed. Will
> remove them when I doublechecked they're in the newest Gambit version's
> namespace. As it looks the tcpip module will be removed.)
>
>
> *Future development*
> I'll be happy to see when current Black Hole will have package management
> functionality so that installation of these packages will be easy.
>
>
> Currently the OpenSSL module exports its functionality as a set of IO
> primitive closures, and, the HTTP server and client uses such IO primitive
> closures for interaction with its underlying IO layer.
>
> When there will be good support for application-layer port implementations
> in Gambit - I hope that will happen the next 1.5 years or so, we'll see -
> the HTTP server and client will use Gambit's IO primitives directly instead
> of these custom primitives, which will provide for a cleaner and more
> general code, and,
>
> The OpenSSL module and I suppose the HTTP-client (and probably HTTP server
> too) will export Gambit ports for use something like
>
> (open-ssl-channel-port settings) => port
> (open-http-client-port settings (lambda (request-body-port) (display ..
> request-body-port))) => response-body-port
>
>
>
>
> Thoughts, feedback, patches much welcome.
>
>
> Best regards,
> Mikael
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://webmail.iro.umontreal.ca/pipermail/gambit-list/attachments/20130324/7e29da2a/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2013-03-24-openssl_and_http_client_in_sack.tar.gz
Type: application/x-gzip
Size: 125324 bytes
Desc: not available
Url : http://webmail.iro.umontreal.ca/pipermail/gambit-list/attachments/20130324/7e29da2a/attachment-0001.gz 


More information about the Gambit-list mailing list