Hi all,
once I wrote a piece of code implementing "software transactional memory". The sugar part of it is in the tests[1]: handler can be attached to the transaction. The latter feature was used to implement a small data flow language embedded in Scheme. I miss it under gambit.
If there was a STM implementation for gambit, which I could start from, please tell me. It could be quite a short cut.
Otherwise I need help to understand three things:
1. Atomicity in GVM
To start with I could always just have a global lock instead of lock-free updates. Optimizations might be done later. (Likely crazy, but let's see.)
But I wonder: how are signal handlers actually handled? Can I declare a `for-each` statement in such a way that the GVM sees no signal handler running during execution time? Also, there appears to come some multi threaded GVM our way: will this be the case in the future too?
Background: the heart of the implementation is a list of in-transaction-value records, which need to be written back to the global scope atomically.
2. Defining yet another record type. Much like SRFI-9 just two actual slots per record field, the value and a version tag. Likely doable, but the tricky part: how do I hide the version tag accessors from user code?
3. For *each slot* of a record X of type Y defined using the definition from step 2, I need to be able to create another record ITV[2]. The `for-each` from #1 traverses a ist of ITV checks the version tag from ITV against X's tag and if all goes well, copies the value field from ITV to X.
In the original code, ITV holds a pointer to X and an integer offset into X representation. (A low level chicken feature `##sys#slot`.) Is there something alike in gambit?
Thank you so much
Jörg
[1] https://github.com/0-8-15/hopefully/blob/03210411161e0df057cabf9fae734688200... [2] ITV: in-transaction-value
Afficher les réponses par date
Hello Jorg,
On Dec 17, 2019, at 8:30 AM, Jörg F. Wittenberger Joerg.Wittenberger@softeyes.net wrote:
Hi all,
once I wrote a piece of code implementing "software transactional memory". The sugar part of it is in the tests[1]: handler can be attached to the transaction. The latter feature was used to implement a small data flow language embedded in Scheme. I miss it under gambit.
If there was a STM implementation for gambit, which I could start from, please tell me. It could be quite a short cut.
Otherwise I need help to understand three things:
- Atomicity in GVM
To start with I could always just have a global lock instead of lock-free updates. Optimizations might be done later. (Likely crazy, but let's see.)
But I wonder: how are signal handlers actually handled? Can I declare a `for-each` statement in such a way that the GVM sees no signal handler running during execution time? Also, there appears to come some multi threaded GVM our way: will this be the case in the future too?
Background: the heart of the implementation is a list of in-transaction-value records, which need to be written back to the global scope atomically.
There are two thread schedulers implemented by Gambit, the default scheduler multiplexes the Scheme threads on one “processor” and the SMP thread scheduler which uses multiple processors. To get the SMP scheduler you need to: ./configure;make;make bootstrap;make bootclean;./configure --enable-multiple-threaded-vms --enable-smp;make
The approach that will work with both schedulers is to use mutexes to implement critical sections. That way you can guarantee that a thread’s operations in the critical section will not be interleaved with operations from other threads that aquire the same mutex.
With the default thread scheduler, or the SMP thread scheduler when there is a single processor (runtime option -:p1) you can disable the scheduler’s thread preemption interrupt by using the (declare (not interrupts-enabled)) declaration. When that declaration is in effect, the compiler will no longer add interrupt checking code in the generated code. Note however that if there is a call to a procedure that wasn’t compiled with this declaration then the thread scheduler may interrupt its execution. Moreover disabling interrupt checking is tricky because stack overflows are detected using the same mechanism, so you have to make sure the code does not add more than a few stack frames.
- Defining yet another record type. Much like SRFI-9 just two actual
slots per record field, the value and a version tag. Likely doable, but the tricky part: how do I hide the version tag accessors from user code?
Gambit’s define-type can assign various properties to the fields, including whether the field is printed or not, whether the field is read-only, whether its content is checked by the equal? procedure, what the initial value of the field is, etc
A plain definition of a 2d point record can be done like this:
(define-type point x y)
It implicitly defines the procedures make-point, point-copy, point?, point-x, point-x-set!, point-x-set, point-y, point-y-set!, point-y-set. Both fields will be shown when a point is printed:
(make-point 1 2)
#<point #2 x: 1 y: 2>
If you want two other hidden fields for the version tags of x and y, you could define the record like this:
(define-type point x y (x-version unprintable: equality-skip: no-functional-setter: init: #f) (y-version unprintable: equality-skip: no-functional-setter: init: #f))
Then the version fields will not be printed:
(make-point 1 2)
#<point #3 x: 1 y: 2>
However, this implicitly defines the procedures point-x-version, point-x-version-set!, point-y-version, point-y-version-set!.
There are two ways to hide these procedures from users. The first is to use the module system to explicitly indicate the names that should be exported from the module that contains the record definition.
The other way is to use the macros: option to define-type which defines all operations as macros, and the prefix: option that allows specifying a prefix for the implicitly generated names.
(define-type point macros: prefix: macro- x y (x-version unprintable: equality-skip: no-functional-setter: init: #f) (y-version unprintable: equality-skip: no-functional-setter: init: #f))
(define (make-point x y) (macro-make-point x y)) (define (point? x) (macro-point? x)) (define (point-x p) (macro-point-x p)) (define (point-x-set! p x) (macro-point-x-set! p x)) (define (point-y p) (macro-point-y p)) (define (point-y-set! p y) (macro-point-y-set! p y))
Then your module can still use macro-point-x-version, etc to access these fields. Because the macros are not exported from the module these fields cannot be accessed by user code.
- For *each slot* of a record X of type Y defined using the definition
from step 2, I need to be able to create another record ITV[2]. The `for-each` from #1 traverses a ist of ITV checks the version tag from ITV against X's tag and if all goes well, copies the value field from ITV to X.
In the original code, ITV holds a pointer to X and an integer offset into X representation. (A low level chicken feature `##sys#slot`.) Is there something alike in gambit?
Gambit’s define-type expands into definitions that call a few primitive procedures to access records. Here are the unsafe primitives that don’t check the type of the first parameter:
(##unchecked-structure-ref obj i type proc) ;; fetch a field of obj (##unchecked-structure-set! obj val i type proc) ;; store in a field of obj (##unchecked-structure-cas! obj val oldval i type proc) ;; compare-and-swap a field of obj
These primitive procedures ignore the type and proc parameters. Note that the compare-and-swap operation may be useful for your purposes as it is an atomic operation in both thread schedulers.
Here’s a sample use:
(define-type point x y) (define p (make-point 11 22)) (##unchecked-structure-ref p 1 #f #f) ;; field x is at index 1
11
(##unchecked-structure-set! p 77 1 #f #f) ;; set field x
#<point #2 x: 77 y: 22>
p
#<point #2 x: 77 y: 22>
(##unchecked-structure-cas! p 88 77 1 #f #f) ;; will change field x because it contains 77
77
p
#<point #2 x: 88 y: 22>
(##unchecked-structure-cas! p 99 77 1 #f #f) ;; won’t change field x because it no longer contains 77
88
p
#<point #2 x: 88 y: 22>
(##structure-type p) ;; get type descriptor of record p
#<type #3 point>
(##type-fields (##structure-type p)) ;; get vector with field attributes (3 per field)
#(x 0 #f y 0 #f)
Marc
On Sat, Dec 21, 2019 at 07:20:32AM -0500, Marc Feeley wrote:
Hello Jorg,
On Dec 17, 2019, at 8:30 AM, Jörg F. Wittenberger Joerg.Wittenberger@softeyes.net wrote:
Hi all,
With the default thread scheduler, or the SMP thread scheduler when there is a single processor (runtime option -:p1) you can disable the scheduler’s thread preemption interrupt by using the (declare (not interrupts-enabled)) declaration. When that declaration is in effect, the compiler will no longer add interrupt checking code in the generated code. Note however that if there is a call to a procedure that wasn’t compiled with this declaration then the thread scheduler may interrupt its execution. Moreover disabling interrupt checking is tricky because stack overflows are detected using the same mechanism, so you have to make sure the code does not add more than a few stack frames.
I conclude that tail-calls will not add stack frames, not even temporary ones that can get garbage-collected when the stack is full. Otherwise this won't work.
- Defining yet another record type. Much like SRFI-9 just two actual
slots per record field, the value and a version tag. Likely doable, but the tricky part: how do I hide the version tag accessors from user code?
Gambit’s define-type can assign various properties to the fields, including whether the field is printed or not, whether the field is read-only, whether its content is checked by the equal? procedure, what the initial value of the field is, etc
A plain definition of a 2d point record can be done like this:
(define-type point x y)
It implicitly defines the procedures make-point, point-copy, point?, point-x, point-x-set!, point-x-set, point-y, point-y-set!, point-y-set. Both fields will be shown when a point is printed:
(make-point 1 2)
#<point #2 x: 1 y: 2>
If you want two other hidden fields for the version tags of x and y, you could define the record like this:
(define-type point x y (x-version unprintable: equality-skip: no-functional-setter: init: #f) (y-version unprintable: equality-skip: no-functional-setter: init: #f))
As I always wonder with Racket, I find myself wondering how much of this stuff is standard Scheme and how much is Gambit-specific. It would be lovely to program in a common subset fo the two, because I like Gambit, but Drracket is a very convenient development environment.
-- hendrik
On Dec 21, 2019, at 8:48 AM, Hendrik Boom hendrik@topoi.pooq.com wrote:
On Sat, Dec 21, 2019 at 07:20:32AM -0500, Marc Feeley wrote:
Hello Jorg,
On Dec 17, 2019, at 8:30 AM, Jörg F. Wittenberger Joerg.Wittenberger@softeyes.net wrote:
Hi all,
With the default thread scheduler, or the SMP thread scheduler when there is a single processor (runtime option -:p1) you can disable the scheduler’s thread preemption interrupt by using the (declare (not interrupts-enabled)) declaration. When that declaration is in effect, the compiler will no longer add interrupt checking code in the generated code. Note however that if there is a call to a procedure that wasn’t compiled with this declaration then the thread scheduler may interrupt its execution. Moreover disabling interrupt checking is tricky because stack overflows are detected using the same mechanism, so you have to make sure the code does not add more than a few stack frames.
I conclude that tail-calls will not add stack frames, not even temporary ones that can get garbage-collected when the stack is full. Otherwise this won't work.
Your conclusion is correct. Tail call = no additional stack frame (or to put it another way, the current stack frame is immediately garbage collected at the moment of the control transfer).
- Defining yet another record type. Much like SRFI-9 just two actual
slots per record field, the value and a version tag. Likely doable, but the tricky part: how do I hide the version tag accessors from user code?
Gambit’s define-type can assign various properties to the fields, including whether the field is printed or not, whether the field is read-only, whether its content is checked by the equal? procedure, what the initial value of the field is, etc
A plain definition of a 2d point record can be done like this:
(define-type point x y)
It implicitly defines the procedures make-point, point-copy, point?, point-x, point-x-set!, point-x-set, point-y, point-y-set!, point-y-set. Both fields will be shown when a point is printed:
(make-point 1 2)
#<point #2 x: 1 y: 2>
If you want two other hidden fields for the version tags of x and y, you could define the record like this:
(define-type point x y (x-version unprintable: equality-skip: no-functional-setter: init: #f) (y-version unprintable: equality-skip: no-functional-setter: init: #f))
As I always wonder with Racket, I find myself wondering how much of this stuff is standard Scheme and how much is Gambit-specific. It would be lovely to program in a common subset fo the two, because I like Gambit, but Drracket is a very convenient development environment.
Gambit/Racket unification is not (yet) on my TODO. Gerbil has gone part of the way there, so maybe some of the Gerbil goodness should be ported back to Gambit so that all users can benefit.
Marc
As I always wonder with Racket, I find myself wondering how much of this stuff is standard Scheme and how much is Gambit-specific. It would be lovely to program in a common subset fo the two, because I like Gambit, but Drracket is a very convenient development environment.
Gambit/Racket unification is not (yet) on my TODO. Gerbil has gone part of the way there, so maybe some of the Gerbil goodness should be ported back to Gambit so that all users can benefit.
Guile also very recently added a R7RS command line switch, so there is movement in the direction of greater standardization. The switch is in the master branch of Guile but AFAIK not yet in any published release.
The R7RS Large Edition standard is actively worked on, but I got the impression Racket is based on R6RS with plenty of custom extensions.
On Sat, Dec 21, 2019 at 9:55 AM Lassi Kortela lassi@lassi.io wrote:
Guile also very recently added a R7RS command line switch, so there is movement in the direction of greater standardization. The switch is in the master branch of Guile but AFAIK not yet in any published release.
Good to know. I've updated Guile's status in the R7RS-small implementations page from "partial" to "not yet released".
The R7RS Large Edition standard is actively worked on, but I got the impression Racket is based on R6RS with plenty of custom extensions.
I think it would be historically more accurate to say that R6RS is based on Racket and Chez (and to some extent Larceny) as they stood in 2007. There are basically four kinds of R6RS implementations on the list at < http://www.r6rs.org/implementations.html%3E:
a) Those whose implementations before R6RS contributed substantially to the standard: Racket, Chez, Larceny. Authors of the first two have their names on the standard, and Will Clinger's name was on all drafts except the final version.
b) Those which retrofitted R6RS into an existing implementation but did not contribute: Guile.
c) Those written after R6RS that (I think) were concerned to implement the latest standard: Ikarus/Vicare, Iron, Loko, Mosh, Sagittarius, Ypsilon. In addition, Biwa implements only the base library and a handful of others.
From which I think we may conclude that retrofitting R6RS is not a popular
thing to do.
John Cowan http://vrici.lojban.org/~cowan cowan@ccil.org Heckler: "Go on, Al, tell 'em all you know. It won't take long." Al Smith: "I'll tell 'em all we *both* know. It won't take any longer."
b) Those which retrofitted R6RS into an existing implementation but did not contribute: Guile.
c) Those written after R6RS that (I think) were concerned to implement the latest standard: Ikarus/Vicare, Iron, Loko, Mosh, Sagittarius, Ypsilon. In addition, Biwa implements only the base library and a handful of others.
From which I think we may conclude that retrofitting R6RS is not a popular thing to do.
R6RS is quite a big language and there are already so many good implementations of it that new ones might not have much to add. Same situation as with Common Lisp. R7RS is such a small language that all kinds of niche implementations are interesting and useful to explore.
Curiously, I can't think of a single R6RS->C compiler. There are interpreters and a few native-code compilers, but no compile-via-C.
Petit Larceny is a version of Larceny that compiles to C.
On Sat, Dec 21, 2019 at 4:03 PM Lassi Kortela lassi@lassi.io wrote:
b) Those which retrofitted R6RS into an existing implementation but did not contribute: Guile.
c) Those written after R6RS that (I think) were concerned to implement the latest standard: Ikarus/Vicare, Iron, Loko, Mosh, Sagittarius, Ypsilon. In addition, Biwa implements only the base library and a handful of others.
From which I think we may conclude that retrofitting R6RS is not a popular thing to do.
R6RS is quite a big language and there are already so many good implementations of it that new ones might not have much to add. Same situation as with Common Lisp. R7RS is such a small language that all kinds of niche implementations are interesting and useful to explore.
Curiously, I can't think of a single R6RS->C compiler. There are interpreters and a few native-code compilers, but no compile-via-C.
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list
On Sat, Dec 21, 2019 at 11:02:55PM +0200, Lassi Kortela wrote:
R6RS is quite a big language and there are already so many good implementations of it that new ones might not have much to add. Same situation as with Common Lisp. R7RS is such a small language that all kinds of niche implementations are interesting and useful to explore.
Which is why they split R7RS into a small language and a large language that I home can be implemented entirely on top of the small langauge.
-- hendrik
Not entirely, no. For example, syntax-case and other macro systems can't be layered over syntax-rules. By the same token, the Posix API (SRFI 170) can't be layered over R7RS-small. It could be layered over an FFI, but there are just too many different kinds of FFIs out there.
Here's the current list of non-portable SRFIs and pre-SRFIs that I have proposed to be voted on:
Conditions: ConditionsCowan https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/ConditionsCowan.md
File I/O: FilesAdvancedCowan https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/FilesAdvancedCowan.md plus SettingsListsCowan https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/SettingsListsCowan.md
Threads: SRFI 18 http://srfi.schemers.org/srfi-18/srfi-18.html plus optional SRFI 21 http://srfi.schemers.org/srfi-21/srfi-21.html or FuturesCowan https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/FuturesCowan.md (simplified with monad)
Sockets: SRFI 106 http://srfi.schemers.org/srfi-106/srfi-106.html or NetworkPortsCowan https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/NetworkPortsCowan.md with NetworkEndpointsCowan https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/NetworkEndpointsCowan.md
Datagram channels (UDP sockets): DatagramChannelsCowan https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/DatagramChannelsCowan.md
Timers: SRFI 120 http://srfi.schemers.org/srfi-120/srfi-120.html
Mutable environments: EnvironmentsMIT https://htmlpreview.github.io/?https://bitbucket.org/cowan/r7rs-wg1-infra/raw/default/EnvironmentsMIT.html
Host environment (Posix): SRFI 170 https://htmlpreview.github.io/?https://bitbucket.org/cowan/r7rs-wg1-infra/raw/default/srfi-170.html
Access to the REPL: ReplCowan https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/ReplCowan.md
Library declarations: LibraryDeclarationsCowan https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/LibraryDeclarationsCowan.md
Interfaces: InterfacesCowan https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/InterfacesCowan.md
Process control: ProcessesCowan https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/ProcessesCowan.md
System commands: SystemCommandCowan https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/SystemCommandCowan.md
Pure delay/force: PureDelayedGloria https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/PureDelayedGloria.md
Delimited continuations: Racket https://docs.racket-lang.org/reference/cont.html, Guile https://www.gnu.org/software/guile/manual/html_node/Prompt-Primitives.html , Scheme48/Kali https://github.com/tonyg/kali-scheme/blob/master/scheme/misc/shift-reset.scm , Gauche https://practical-scheme.net/gauche/man/gauche-refe/Partial-continuations.html , Chicken http://wiki.call-cc.org/eggref/4/F-operator
Continuation marks: SRFI 157 http://srfi.schemers.org/srfi-157/srfi-157.html
Extended exact numbers: SRFI 73 http://srfi.schemers.org/srfi-73/srfi-73.html or ExtendedRationalsCowan https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/ExtendedRationalsCowan.md
Adjustable strings: SRFI 118 http://srfi.schemers.org/srfi-118/srfi-118.html (basic) or SRFI 140 http://srfi.schemers.org/srfi-140/srfi-140.html (mutable/immutable)
Mutexes, condition variables: SRFI 18 http://srfi.schemers.org/srfi-18/srfi-18.html
Port type detector: see ticket #177 https://bitbucket.org/cowan/r7rs-wg1-infra/issues/177/distinguish-file-and-string-ports
Internationalization of strings: GettextCowan https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/GettextCowan.md
Chronometers: Chronometer https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/Chronometer.md
Character-cell terminals: TerminalsCowan https://bitbucket.org/cowan/r7rs-wg1-infra/src/4a3907a8b871518b0cf81e08084da6850278175b/TerminalsCowan.md
Graphics canvas: CanvasCowan http://smallbasic.com/doc/?id=8&language=
Comments are extremely welcome.
On Sat, Dec 21, 2019 at 7:24 PM Hendrik Boom hendrik@topoi.pooq.com wrote:
On Sat, Dec 21, 2019 at 11:02:55PM +0200, Lassi Kortela wrote:
R6RS is quite a big language and there are already so many good implementations of it that new ones might not have much to add. Same situation as with Common Lisp. R7RS is such a small language that all
kinds
of niche implementations are interesting and useful to explore.
Which is why they split R7RS into a small language and a large language that I home can be implemented entirely on top of the small langauge.
-- hendrik
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list
Le dim. 22 déc. 2019 à 00:57, John Cowan cowan@ccil.org a écrit :
Not entirely, no. For example, syntax-case and other macro systems can't be layered over syntax-rules. By the same token, the Posix API (SRFI 170) can't be layered over R7RS-small. It could be layered over an FFI, but there are just too many different kinds of FFIs out there.
Here's the current list of non-portable SRFIs and pre-SRFIs that I have proposed to be voted on:
[...]
Graphics canvas: CanvasCowan
The above links to http://smallbasic.com/doc/?id=8&language=
Comments are extremely welcome.
NB: Maybe it is better to move to scheme.lang.comp? or WG2?
Le dim. 22 déc. 2019 à 00:57, John Cowan cowan@ccil.org a écrit :
Not entirely, no. For example, syntax-case and other macro systems can't be layered over syntax-rules. By the same token, the Posix API (SRFI 170) can't be layered over R7RS-small. It could be layered over an FFI, but there are just too many different kinds of FFIs out there.
Here's the current list of non-portable SRFIs and pre-SRFIs that I have proposed to be voted on:
Character-cell terminals: TerminalsCowan
I made another pre-SRFI where I remove the procedure that requires selection to be supported:
-
Amirouche Boubekki -
Hendrik Boom -
John Cowan -
Jörg F. Wittenberger -
Lassi Kortela -
Marc Feeley