+1 to all.
I'm not clear if you want to change just the external default for text files, or if you are considering changing the internal representation of strings as well. UTF-8-based storage comes with extra costs for random string access, so you might want to implement SRFI 130, which allows fast sequential cursor-based as well as index-based operations. (There is no standard string library for R7RS-large yet.) The sample implementation does cursors on top of indexes and so provides no speed advantage, but a tuned-up variety of the Chibi implementation would be much better for Gambit. In a UTF-8 internal representation, you of course make string mutation costly, but it's a rare thing to do, at least when strings are not being (mis)used as bytevectors.
SRFI 135, immutable texts, is part of R7RS-large. It provides disjoint O(1) read-only string-like objects called texts. Its operations always return texts, but most of them work on either texts or strings. There are a variety of available implementations: one based on UTF-8 stored in bytevectors, one based on UTF-16 stored in bytevectors, and one based on strings (for use when built-in string operations are much more efficient than operations written in Scheme). All permit a bounded amount of storage-sharing. So when writing programs that manipulate a lot of textual data, the idea is to use strings only at the edges of the program (literals and I/O) and to do all textual manipulation using texts. It's flawed, but that's all we can do.
On Wed, Nov 20, 2019 at 11:21 AM Jeff Read bitwize@gmail.com wrote:
If Gambit 5 has full support for bytevector I/O primitives (which, being R7RS-compatible, presumably it does), then I would totally get behind UTF-8 as the default charset. I wince a little every time I'm forced to use 'read-char' or similar to ingest bytes, anyway.
On Wed, Nov 20, 2019, 8:55 AM Marc Feeley feeley@iro.umontreal.ca wrote:
With all the changes to Gambit recently it is time to think about Gambit v5.0.0 which will probably be released in the next 6-12 months and include some backward incompatible changes due to the evolution of the linking model for the new module system.
Another backward incompatible change would be switching to multithreaded VMs by default, and the SMP scheduler. But this will require community involvement to weed out the issues and make sure it is rock solid.
One backward incompatible change that could also happen at that point is a change of the default character encoding to UTF-8 (currently Gambit defaults to ISO-8859-1, aka latin-1, for all I/O unless configured otherwise with --enable-default-runtime-options=...). The nice thing about the ISO-8859-1 encoding is the 1-to-1 mapping between characters and bytes, so a program can do character I/O to read/write binary files, which was the only way to do binary I/O in the old days of Scheme (so some programs might depend on that). Nowadays many environments support UTF-8 well.
Please share your thoughts...
Marc
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list