[gambit-list] 0mq with Gambit

Jason E. Aten j.e.aten at gmail.com
Wed Apr 6 12:17:16 EDT 2011


On Wed, Apr 6, 2011 at 8:13 AM, Marc Feeley <feeley at iro.umontreal.ca> wrote:

>
> On 2011-04-03, at 11:04 PM, Feng Hou wrote:
>
> > Hello everyone,
> >
> > Gambit has great concurrency support by green-thread and no-blocking I/O
> scheduling. However, they are only able to use one native thread on
> multicore hardware. I have noticed some wish list on wiki for
> multicore/multiprocessing concurrency support. I'm wondering whether 0MQ (
> http://www.zeromq.org/) can be used as substance for Gambit to build such
> capabilities.
> >
> > My thoughts are,
> >
> > -- Not just a FFI binding.
> > -- Use it as message passing broker for multicore and distributed
> concurrency.
> > -- Share nothing between the GVM thread and other in-process native
> threads (doing long CPU bound computation or blocking I/O).
> > -- How to integrate 0MQ inter-thread transport to gambit green-thread
> scheduler without blocking it?
> > -- Would it be possible to bind 0MQ inter-thread socket to gambit
> mailbox?
> > -- How to integrate 0MQ I/O event poller with gambit I/O loop?
> > -- Would it be better to expose 0MQ IPC/TCP/PGM socket types as Gambit
> Port objects?
> >
> > I realized they were way beyond my knowledge and skill level to implement
> (some may not even make sense, please correct me). Nevertheless, just want
> to see if others have similar thoughts or needs.
> >
> > Cheers,
> > Feng Hou
>
> I quickly read the 0MQ docs and it seems interesting.  I can give pointers
> to whoever wants to implement them into Gambit.
>
> Marc


I'd be up for contributing on this.  ZMQ and Gambit together would be quite
sweet on multicore and as the basis for very scalable distributed systems.

As an aside, the last missing piece would be using Google's Protocol Buffers
(PB) for very efficient serialization.  I say this in reference to the
comments in the Termite paper that the new serialization in Gambit was the
only bottleneck that kept Termite from beating Erlang outright in all
categories.  By the way, has the serialization situation improved? If so
then perhaps PB is not necessary.  But PB would also give one really nice
inter-language interoperability, since PB bindings are available from just
about any language (sadly except Scheme; but perhaps the Common Lisp
bindings could be ported without much trouble) at this point.

But back to ZMQ.   It would seem to be more general, as Feng suggests, to be
able to use ZMQ from Gambit rather than just Termite.  I say this because if
one could use ZMQ from Gambit, then there would be no need play elaborate
games to get mutable state, (or variables that actually vary =) ; i.e.
without needing to do the gen_server trick/(section 4.6 of the Termite paper
workaround) when you actually need mutable variables. Unfortunately, I work
with big enough data, that I really do need mutable variables.

So assuming that one wants to be able to use ZMQ in Gambit (which I would!),
I'd be very glad to hear any advice on how to do that.

Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20110406/c4777bf2/attachment.htm>


More information about the Gambit-list mailing list