[gambit-list] 0mq with Gambit

Feng Hou houfen at gmail.com
Thu Apr 7 17:50:18 EDT 2011


On Wed, Apr 6, 2011 at 12:17 PM, Jason E. Aten <j.e.aten at gmail.com> wrote:

> On Wed, Apr 6, 2011 at 8:13 AM, Marc Feeley <feeley at iro.umontreal.ca>wrote:
>
>>
>> On 2011-04-03, at 11:04 PM, Feng Hou wrote:
>>
>> > Hello everyone,
>> >
>> > Gambit has great concurrency support by green-thread and no-blocking I/O
>> scheduling. However, they are only able to use one native thread on
>> multicore hardware. I have noticed some wish list on wiki for
>> multicore/multiprocessing concurrency support. I'm wondering whether 0MQ (
>> http://www.zeromq.org/) can be used as substance for Gambit to build such
>> capabilities.
>> >
>> > My thoughts are,
>> >
>> > -- Not just a FFI binding.
>> > -- Use it as message passing broker for multicore and distributed
>> concurrency.
>> > -- Share nothing between the GVM thread and other in-process native
>> threads (doing long CPU bound computation or blocking I/O).
>> > -- How to integrate 0MQ inter-thread transport to gambit green-thread
>> scheduler without blocking it?
>> > -- Would it be possible to bind 0MQ inter-thread socket to gambit
>> mailbox?
>> > -- How to integrate 0MQ I/O event poller with gambit I/O loop?
>> > -- Would it be better to expose 0MQ IPC/TCP/PGM socket types as Gambit
>> Port objects?
>> >
>> > I realized they were way beyond my knowledge and skill level to
>> implement (some may not even make sense, please correct me). Nevertheless,
>> just want to see if others have similar thoughts or needs.
>> >
>> > Cheers,
>> > Feng Hou
>>
>> I quickly read the 0MQ docs and it seems interesting.  I can give pointers
>> to whoever wants to implement them into Gambit.
>>
>> Marc
>
>
> I'd be up for contributing on this.  ZMQ and Gambit together would be quite
> sweet on multicore and as the basis for very scalable distributed systems.
>
> As an aside, the last missing piece would be using Google's Protocol
> Buffers (PB) for very efficient serialization.  I say this in reference to
> the comments in the Termite paper that the new serialization in Gambit was
> the only bottleneck that kept Termite from beating Erlang outright in all
> categories.  By the way, has the serialization situation improved? If so
> then perhaps PB is not necessary.  But PB would also give one really nice
> inter-language interoperability, since PB bindings are available from just
> about any language (sadly except Scheme; but perhaps the Common Lisp
> bindings could be ported without much trouble) at this point.
>
> But back to ZMQ.   It would seem to be more general, as Feng suggests, to
> be able to use ZMQ from Gambit rather than just Termite.  I say this because
> if one could use ZMQ from Gambit, then there would be no need play elaborate
> games to get mutable state, (or variables that actually vary =) ; i.e.
> without needing to do the gen_server trick/(section 4.6 of the Termite paper
> workaround) when you actually need mutable variables. Unfortunately, I work
> with big enough data, that I really do need mutable variables.
>
> So assuming that one wants to be able to use ZMQ in Gambit (which I
> would!), I'd be very glad to hear any advice on how to do that.
>
> Jason
>

I'm glad to hear so much interest and help offered. I've been playing around
a toy binding, and reading 0mq source. I'd like to share a few more concrete
ideas.

0mq socket is a concurrent device, rather than an I/O device. It's very
lightweight, essentially a concurrent in-memory queue. Hundreds of thousands
to the same tcp end-point would only consume just one tcp connection. This
seems a perfect fit to gambit green-thread model. Pulling socket send/recv
in noblock mode loop is equivalent to spinning on memory CAS+fance of a
pointer (this appears to be how zmq implements it). 0mq also provides a
timer facility, but I believe it'd block gambit thread scheduler. Instead,
gambit thread-yield!/thread-sleep! can be used for scheduling noblock
send/recv pulling.  We can have thousands of green-threads pulling on
thousands of 0mq sockets concurrently. This can be done fairly
straightforward in user land. Ideally, it'd be nice to just write,

(write u8vector a-zmq-port)

(read u8vector a-zmq-port timeout: 10)

All looping/retry/yield/sleep can happen in gambit scheduler (need
continuation magic?).  Even nicer would be able to bind sockets to thread
mailboxes so messages are pumped into mailboxes on receiving automatically.
So we can use Erlang/Termite style receive loop, which gambit already
adopted for local thread message passing (Though I'm not too keen on sending
messages to remote threads, i.e. erlang Pid model. IMO, It's too tightly
coupled in distributed  environment, where 0mq subject-oriented message
passing is better architecturally).

I'll try to prototype above ideas over the weekend, but I'm sorry to say, my
skill is limited in gambit user land (still learning FFI, another post for
some memory and GC questions), definitely not up to gambit
scheduler/continuation part(but I'm willing to try and learn).

A final point, 0mq has a fairly scalable I/O polling infrastructure using
epoll/kqueue etc. It exposes it to a user land api as well. However, after
thought about it more. It seems to me to be used for native thread doing
concurrent I/O, which is a problem that gambit green-thread already solved
in much better way. I don't believe it'll provide any additional benefits by
converging or integrating gambit multiplexing I/O system to that. With above
approach, these two I/O event loops can peacefully live in their own
(thread) world. The only connection between them are 0mq sockets.

Comments and ideas are welcome. I'll try to write some code and put it up on
github, but *please* free feel to do your own if anyone can beat me on time
and skill :-)

Cheers,

- Feng
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20110407/4d80b134/attachment.htm>


More information about the Gambit-list mailing list