[gambit-list] OpenSSL SSL channels module pre-release and HTTP client early pre-release, attached

Mikael mikael.rcv at gmail.com
Wed Mar 20 15:46:26 EDT 2013


Hi guys,

For anyone interested here's the pre-release version of the OpenSSL SSL
channels ("openssl") module and an early pre-release of the HTTP client.

(If you read this after May 2013 the attachment is most probably outdated
and you better look at https://github.com/gambit-community or
https://github.com/m-i-k-a-e-l/ .)


*openssl module*
The openssl module works extremely well and is mature.

Relative to the amount of functionality provided, extreme effort has been
put into its making. This is because OpenSSL's documentation on nonblocking
use is incorrect in several parts, so study of other user implementations
and practical testing has been required to make it work. Over the course of
a couple of years, in the range three man weeks has gone into it this way.

Due to that this is the nature of the problem addressed by this module, I
hope that this openssl module will be the main one for Gambit and that any
more effort into OpenSSL SSL channel modules in Gambit will go into making
this one even better.

The current version however relies on a version-unsafe hack for extracting
the respective TCP port's socket, which possibly would be a feature to be
included in Gambit, please see the "Next steps for release" section below.
OpenSSL is let to write and read from the OS socket directly. This saves
lots of roundtrips and buffering.


*http-client module*
The HTTP client supports the passing of custom method, headers and request
body, and supports retrieval of status code, headers and request body, and
it does both these with support for all of HTTP 1.0 and 1.1, so this is all
there is to the HTTP protocol in itself.

Exports:

(simple-raw-http-request uri #!optional request-body (headers '())) => req

(define-macro (with-http-client-all-response all-response . code)
  `(apply (lambda (reached-uri result stage status-code headers get-header
response) , at code) ,all-response))

(define (simple-raw-http-request-resp uri #!optional request-body)
  (with-http-client-all-response
   (simple-raw-http-request uri request-body)
   response))

Use:


> (simple-raw-http-request "http://www.iro.umontreal.ca/~gambit/")
(#<uri #13 scheme: "http" userinfo: #f host: "www.iro.umontreal.ca" port:
#f path: "/~gambit/" query: () fragment: #f>
 #t
 done
 200
 (("connection" . "close")
  ("content-type" . "text/html")
  ("content-length" . "115")
  ("accept-ranges" . "bytes")
  ("etag" . "\"1c93d75-73-4443e1da730c0\"")
  ("last-modified" . "Mon, 21 Jan 2008 16:38:03 GMT")
  ("server" . "Apache/2.0.54 (Fedora)")
  ("date" . "Wed, 20 Mar 2013 19:20:12 GMT"))
 #<procedure #14 26#get-header>
 "<HTML>\n<HEAD>\n<META HTTP-EQUIV=REFRESH CONTENT=0;URL=
http://dynamo.iro.umontreal.ca/~gambit/wiki/>\n</HEAD>\n</HTML>\n")

> (simple-raw-http-request-resp "http://www.iro.umontreal.ca/~gambit/")
"<HTML>\n<HEAD>\n<META HTTP-EQUIV=REFRESH CONTENT=0;URL=
http://dynamo.iro.umontreal.ca/~gambit/wiki/>\n</HEAD>\n</HTML>\n"


Currently it supports single threaded execution i.e. it opens HTTP
connection, makes request, retrieves response and closes the connection in
a sequence and there's no connection reuse - reuse and multiplexing across
threads as well as following redirects will be implemented. Also invocation
of several declared callbacks is not done currently, and will be trivial to
implement.

The DSL uses Sack's idea of providing all protocol functionality in a
'core' form. It can extremely easily be wrapped to a XMLHTTPRequest object.

This http-client module replaces Sack's previous http-client module, which
unlike Sack's web server module which works great, had terrible problems,
for more info see the header section in http-client.scm .




*Next steps for release*
I'll make a "real" release of these two modules when the IO exception
handling question discussed on the list now has got an answer and maybe
extraction of TCP port socket can be done with a Gambit-builtin procedure.
These two things currently bring version-unsafe hacks into the code, more
specifically into tcpip.scm and sack/src/gamgambit-noexceptions-io.scm , so
this is why I call this a pre-release.

(Note that port-wait-for-input and port-wait-for-output of the tcpip module
are not needed anymore as the firstmentioned has been in Gambit all the
time and the latter was recently included in Gambit, ##-prefixed. Will
remove them when I doublechecked they're in the newest Gambit version's
namespace. As it looks the tcpip module will be removed.)


*Future development*
I'll be happy to see when current Black Hole will have package management
functionality so that installation of these packages will be easy.


Currently the OpenSSL module exports its functionality as a set of IO
primitive closures, and, the HTTP server and client uses such IO primitive
closures for interaction with its underlying IO layer.

When there will be good support for application-layer port implementations
in Gambit - I hope that will happen the next 1.5 years or so, we'll see -
the HTTP server and client will use Gambit's IO primitives directly instead
of these custom primitives, which will provide for a cleaner and more
general code, and,

The OpenSSL module and I suppose the HTTP-client (and probably HTTP server
too) will export Gambit ports for use something like

(open-ssl-channel-port settings) => port
(open-http-client-port settings (lambda (request-body-port) (display ..
request-body-port))) => response-body-port




Thoughts, feedback, patches much welcome.


Best regards,
Mikael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20130320/bf074e39/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: openssl_and_http_client_in_sack.tar.gz
Type: application/x-gzip
Size: 118450 bytes
Desc: not available
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20130320/bf074e39/attachment.bin>


More information about the Gambit-list mailing list