With all the changes to Gambit recently it is time to think about Gambit v5.0.0 which will probably be released in the next 6-12 months and include some backward incompatible changes due to the evolution of the linking model for the new module system.
Another backward incompatible change would be switching to multithreaded VMs by default, and the SMP scheduler. But this will require community involvement to weed out the issues and make sure it is rock solid.
One backward incompatible change that could also happen at that point is a change of the default character encoding to UTF-8 (currently Gambit defaults to ISO-8859-1, aka latin-1, for all I/O unless configured otherwise with --enable-default-runtime-options=...). The nice thing about the ISO-8859-1 encoding is the 1-to-1 mapping between characters and bytes, so a program can do character I/O to read/write binary files, which was the only way to do binary I/O in the old days of Scheme (so some programs might depend on that). Nowadays many environments support UTF-8 well.
Please share your thoughts...
Marc
Afficher les réponses par date
If Gambit 5 has full support for bytevector I/O primitives (which, being R7RS-compatible, presumably it does), then I would totally get behind UTF-8 as the default charset. I wince a little every time I'm forced to use 'read-char' or similar to ingest bytes, anyway.
On Wed, Nov 20, 2019, 8:55 AM Marc Feeley feeley@iro.umontreal.ca wrote:
With all the changes to Gambit recently it is time to think about Gambit v5.0.0 which will probably be released in the next 6-12 months and include some backward incompatible changes due to the evolution of the linking model for the new module system.
Another backward incompatible change would be switching to multithreaded VMs by default, and the SMP scheduler. But this will require community involvement to weed out the issues and make sure it is rock solid.
One backward incompatible change that could also happen at that point is a change of the default character encoding to UTF-8 (currently Gambit defaults to ISO-8859-1, aka latin-1, for all I/O unless configured otherwise with --enable-default-runtime-options=...). The nice thing about the ISO-8859-1 encoding is the 1-to-1 mapping between characters and bytes, so a program can do character I/O to read/write binary files, which was the only way to do binary I/O in the old days of Scheme (so some programs might depend on that). Nowadays many environments support UTF-8 well.
Please share your thoughts...
Marc
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list
+1 to all.
I'm not clear if you want to change just the external default for text files, or if you are considering changing the internal representation of strings as well. UTF-8-based storage comes with extra costs for random string access, so you might want to implement SRFI 130, which allows fast sequential cursor-based as well as index-based operations. (There is no standard string library for R7RS-large yet.) The sample implementation does cursors on top of indexes and so provides no speed advantage, but a tuned-up variety of the Chibi implementation would be much better for Gambit. In a UTF-8 internal representation, you of course make string mutation costly, but it's a rare thing to do, at least when strings are not being (mis)used as bytevectors.
SRFI 135, immutable texts, is part of R7RS-large. It provides disjoint O(1) read-only string-like objects called texts. Its operations always return texts, but most of them work on either texts or strings. There are a variety of available implementations: one based on UTF-8 stored in bytevectors, one based on UTF-16 stored in bytevectors, and one based on strings (for use when built-in string operations are much more efficient than operations written in Scheme). All permit a bounded amount of storage-sharing. So when writing programs that manipulate a lot of textual data, the idea is to use strings only at the edges of the program (literals and I/O) and to do all textual manipulation using texts. It's flawed, but that's all we can do.
On Wed, Nov 20, 2019 at 11:21 AM Jeff Read bitwize@gmail.com wrote:
If Gambit 5 has full support for bytevector I/O primitives (which, being R7RS-compatible, presumably it does), then I would totally get behind UTF-8 as the default charset. I wince a little every time I'm forced to use 'read-char' or similar to ingest bytes, anyway.
On Wed, Nov 20, 2019, 8:55 AM Marc Feeley feeley@iro.umontreal.ca wrote:
With all the changes to Gambit recently it is time to think about Gambit v5.0.0 which will probably be released in the next 6-12 months and include some backward incompatible changes due to the evolution of the linking model for the new module system.
Another backward incompatible change would be switching to multithreaded VMs by default, and the SMP scheduler. But this will require community involvement to weed out the issues and make sure it is rock solid.
One backward incompatible change that could also happen at that point is a change of the default character encoding to UTF-8 (currently Gambit defaults to ISO-8859-1, aka latin-1, for all I/O unless configured otherwise with --enable-default-runtime-options=...). The nice thing about the ISO-8859-1 encoding is the 1-to-1 mapping between characters and bytes, so a program can do character I/O to read/write binary files, which was the only way to do binary I/O in the old days of Scheme (so some programs might depend on that). Nowadays many environments support UTF-8 well.
Please share your thoughts...
Marc
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list
On Wed, Nov 20, 2019 at 3:55 PM Marc Feeley feeley@iro.umontreal.ca wrote:
With all the changes to Gambit recently it is time to think about Gambit v5.0.0 which will probably be released in the next 6-12 months and include some backward incompatible changes due to the evolution of the linking model for the new module system.
Can you elaborate on these changes and how they will affect existing code?
Another backward incompatible change would be switching to multithreaded VMs by default, and the SMP scheduler. But this will require community involvement to weed out the issues and make sure it is rock solid.
This will need quite a bit more testing and debugging! But I am dreaming about the day when this becomes rock solid :)
One backward incompatible change that could also happen at that point is a change of the default character encoding to UTF-8 (currently Gambit defaults to ISO-8859-1, aka latin-1, for all I/O unless configured otherwise with --enable-default-runtime-options=...). The nice thing about the ISO-8859-1 encoding is the 1-to-1 mapping between characters and bytes, so a program can do character I/O to read/write binary files, which was the only way to do binary I/O in the old days of Scheme (so some programs might depend on that). Nowadays many environments support UTF-8 well.
+1 for this.
-- vyzo
On Nov 29, 2019, at 7:46 AM, Sven Hartrumpf hartrumpf@gmx.net wrote:
Hi Marc.
Marc wrote, 2019-11-20 08:55:
With all the changes to Gambit recently it is time to think about Gambit v5.0.0 which will probably be released in the next 6-12 months
Very good news!
What are your plans for the native backend?
Gambit’s development is advancing on several fronts including the module system, SMP scheduler, universal backend, native backends, and various side projects.
Thanks to Laurent Huberdeau’s work in 2018 and Abdelhakim Qbaich’s work in 2019, the native backend for x86 has advanced nicely and is currently in a usable state (it can compile most Scheme programs you throw at it). However it is not fully integrated with the rest of the system and some primitives still rely on the C backend. I would like the native backend to be able to compile all of the Gambit runtime system without needing the C backend as a fallback. This goal is not high-priority currently, but if you or others are interested in helping out I can guide you on how you can contribute.
If you are interested in trying it out you need to use the --enable-lowlevel-exec configure flag, and compile/execute programs like this:
% gsc -target x86-64 prog.scm % gsi proc
A higher priority goal is implementing a better linker that takes advantage of the new module system to simplify the linking of programs (for the programmer) and generate much smaller compiled programs. This will have a positive impact on the C, native and universal backends.
Marc
On Dec 3, 2019, at 7:47 AM, Marc Feeley feeley@iro.umontreal.ca wrote:
Thanks to Laurent Huberdeau’s work in 2018 and Abdelhakim Qbaich’s work in 2019, the native backend for x86 has advanced nicely and is currently in a usable state (it can compile most Scheme programs you throw at it). However it is not fully integrated with the rest of the system and some primitives still rely on the C backend. I would like the native backend to be able to compile all of the Gambit runtime system without needing the C backend as a fallback. This goal is not high-priority currently, but if you or others are interested in helping out I can guide you on how you can contribute.
I'm interested in the advancement of the native and js backends, and would be glad to help if I can. I may have some issues scheduling enough time to contribute, but I'd certainly be interested in seeing what could be done to help out.
-
Dimitris Vyzovitis -
Jeff Read -
John Cowan -
Marc Feeley -
mikel evins -
Sven Hartrumpf