On Sun, Feb 24, 2013 at 2:25 PM, Patrick Li patrickli.2001@gmail.com wrote:
I am writing a compiler for a coroutine-based language and have the exact same limitation with regards to the FFI.
I am curious, Dimitris, whether you know of any other language with first-class coroutines or coroutines that have your desired feature. I myself do not. Perhaps it's still an open research problem.
I am not aware of any strictly coroutine-based solution, but it is not such a severe limitation if you are maintaining separate stacks for each thread.
The main problem there is dealing with stack growth, since C stacks are not generally movable -- if your allocated stack is too small you will overflow and if your stack is too large you are wasting too much memory. Still, the kernel doesn't back it by physical memory until it gets used, so you can jumbo-size them with the real limitation being your address space size (not too big of an issue in 64bit architectures).
It is also possible in some situations to grow the stacks as needed. When allocating with MAP_GROWSDOWN, the kernel may leave some space for growth, so when you catch the segfault in the redzone you can remap it. But it is not guaranteed how much room you will have (if any), and the address space quickly gets tight in 32bit.
Perhaps the best way around it is to use segmented (split) stacks. With recent gcc you can get compiler support at C-level (still not trivial to integrate with your language kernel, but doable). You may have to compile your entire C-library with segmented stack support for this to work transparently however.
You can also try to implement dynamic segmented stacks by stepping at your redzone handler until you catch a call instruction, at which point you can make a new stack and stitch the execution contexts. Not exactly a walk in the park though.
-- vyzo