[gambit-list] C stack frames need to be returned in exactly the same order they were created Re: FFI and threads: abrupt exit 71

Mikael mikael.rcv at gmail.com
Sun Feb 24 11:23:21 EST 2013


Hi Dmitris,

Indeed it is a kind of limitation, though limitation is an aspect of the
nature of any technology solution.

Find below some reflections about the design area your example code regards
and a suggestion for best practice.



I checked your code closer now; basically the c-lambda is a part of the
application logics that could as well have been implemented in Scheme.

Whenever you can implement application logics in Scheme, do it, because the
C stack model is as you have found out limited to "direct style" i.e. each
step up must be balanced with one step down and there are *no* shortcuts.
This is the C stack, and how it's delivered to you in the FFI.

Of course it follows that if you attempt to do maneuvers on the C stack
that are not possible from Scheme, something breaks.

Using extraordinarily fancy stack manipulation techniques, this could be
worked around, though that would not be based on completely standard
features found in any computing environment and thus not end up as
crossplatform as Gambit is today, and, it'd take a bit of performance.

When Gambit gets SMP support in some months from now, you'll have one C
stack per OS thread you launched, so you could do a bit more of this then.

Now back to the problem -


So your c-lambda is:


(define do-read
  (c-lambda (scheme-object) int
       #<<END-C
char buf[4096];
___SCMOBJ ctx = ___EXT(___make_vector)(2, ___FAL, ___STILL);
___VECTORSET (ctx, ___FIX(0), ___arg1);

for (;;) {
  if (do_read ((void*)ctx, buf, sizeof(buf)) < 0) break;
}
END-C
))


To solve your issue, rewrite it to Scheme i.e.

 (lambda (v)
  (let ((buf (make-u8vector 4096))
        (ctx (make-still-vector 2 #f)))
    (vector-set! ctx 0 1)
    (let loop ()
      (if (< 0 (do-read ctx buf (u8vector-length buf)) (loop)))


Again, what you are wanting to get is an extremely high level
functionality, namely using a IO multiplexer, in this case the one of
Gambit's scheduler, to return to different places in code in essentially
different stacks - this would generally never be done in C, and this
remains the case also here.


Generally, never make a Scheme-to-C call with a C-to-Scheme call that may
block in any way, except for if such use is done once in parallell within
the OS thread. C's stack model does not deliver for other than exactly
this. Instead, make your C call return back to Scheme with some kind of
status value that instructs your Scheme code how to proceed. Such an
approach will always end up being 'good code'.

If you need to pass complex result data back from the C call to Scheme, you
can do this as C-to-Scheme calls though then keep the thread system off
during those calls as to keep your code safe from the possibility of other
code compromising the C stack in relation with the task you're performing
and thus crashing your program.

If you would really need multiple C stacks, then, presuming they do C
logics only and do not directly invoke any C code beyond your control (such
as OS routines), you could use a C cooperate multithreading library to get
the functionality you look for. That would be sufficiently crossplatform.


What are you trying to do?

Brgds,
Mikael

2013/2/24 Dimitris Vyzovitis <vyzo at hackzen.org>

> Switching threads off is not an option because of I/O.
>
> So basically you are saying that ffi scheme upcalls (through
> c-define'd lambdas) are not thread safe (scheme threads always).
> I would expect each thread to maintain its own c stack, but apparently
> it is not so and C-stacks cannot be safely interleaved with multiple
> threads.
>
> If this is the case, it is a pretty serious limitation in the ffi.
>
> PS: wiki seems down again
>
> On Sat, Feb 23, 2013 at 1:14 AM, Mikael <mikael.rcv at gmail.com> wrote:
> > Dimitris,
> >
> > (From only looking very generally at your code,)
> >
> > May this be a stack unwinding problem?
> >
> > If you unwind a stack other than in the same order it was wound in the
> first
> > place, your Gambit process will terminate or at least crash.
> >
> > Example:
> >
> > You have a lambda A that invokes C-lambda B that invokes lambda C.
> >
> > Also, a lambda D that invokes C-lambda E that invokes lambda F.
> >
> > If you invoke A and A invokes B and B invokes C,
> >
> > and then say, Gambit switches thread, and that Gambit thread invokes D
> which
> > invokes E which invokes F,
> >
> > and then, say, Gambit switches thread again, and that Gambit thread
> returns
> > from C to B.
> >
> > Then your Gambit process is toast.
> >
> > To handle this you can switch off Gambit's threading system when you're
> in B
> > or E for the course of your C and F calls respectively. Note that
> switching
> > off Gambit's threading system means that some or several of Gambit's
> > built-in IO primitives would block forever.
> >
> > See the "How to work with external libraries" Wiki page for how to
> switch it
> > off.
> >
> > Was this of help?
> >
> > Mikael
> >
> > 2013/2/23 Dimitris Vyzovitis <vyzo at hackzen.org>
> >>
> >> I am running into an abrupt exit with error code 71 (OSError) when
> >> multiple (scheme) threads are making ffi calls that upcall to scheme.
> >>
> >> A simplified scenario (see attached file) is as follows:
> >> 2 or more threads make an ffi call.
> >> The ffi call is a longer running procedure, that makes upcalls into
> >> scheme procedures that do I/O.
> >> This results in the program exiting abruptly with error code 71, and
> >> no context information regarding the error (even when compiled and
> >> running with full debugging).
> >>
> >> For an illustration of the problem, see the attached file.
> >>
> >> The following should print dots indefinitely, but instead quickly
> >> terminates with 71.
> >>
> >> $ gsc exit71.scm
> >> $  gsi -e "(begin (load \"exit71\") (main))"
> >> ....<plenty of dots>.... $ echo $?
> >> 71
> >>
> >> This is with gambit-4.6.6, system is GNU/Linux x86-64.
> >>
> >> -- vyzo
> >>
> >> _______________________________________________
> >> Gambit-list mailing list
> >> Gambit-list at iro.umontreal.ca
> >> https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20130224/1e93be0e/attachment.htm>


More information about the Gambit-list mailing list