[gambit-list] Character encoding and the repl

Adam adam.mlmb at gmail.com
Sat May 30 15:23:10 EDT 2015


2015-05-30 14:15 GMT+05:30 Atticus <atticus0 at posteo.org>:

>
> > Like, -:eN  (e as in "escape" and "N" as in "nothing", meaning
> #\U0010ffff).
>
> Would be really convenient but I can also put the configuration into my
> gambcini file.
>
> > Atticus, you're forgetting that the "external representation" discussed
> > here regards only the escaping applied in the serialization of objects.
>
> I dont know if I understand that correctly. I assumed we are discussing
> this change:
>
> with t8 or f8, before:
> (write "ü") -> "\374"
> (read port) -> "ü"
>
> with t8 or f8 after change:
> (write "ü") -> "ü"
> (read port) -> "ü"
>

Right - what I meant was, remember that, if you desire to do output to the
terminal in your application, you should *never* use |write|.

And, if you would write an application where a user inputs content from the
terminal, then you should *never* use |read|.

|read| and |write| are not IO routines specifically, but object
/de/serialization + IO combo routines.

I.e., |write| is not used for presentation, but to give you an absolute
serialized representation in text form of an object's contents, and there
human readability is secondary to technical correctness.

For instance, (write "ü") printing "\374" shows you that the string's
character coding is correct (vs. for instance "\303\274" which would show
you it's broken). If it prints out "ü", you have no idea.

Also, a good reason for escaping is that there would even be need of
security considerations for not escaping special characters like
https://en.wikipedia.org/wiki/Unicode_control_characters and
https://en.wikipedia.org/wiki/Bi-directional_text#Unicode_bidi_support . If
there exists any sensible across-the-board way to handle those except for
escaping them, and if there's anything more of the sort than these in
particular in Unicode, is beyond my present knowledge.



The pure IO output routines are |display|,|print|, |read-substring|,
|read-char|, and the pure IO intput routines are |read-char|, |read-line|,
|read-subu8vector|.


I.e., with t8 or f8, (display "ü") -> "ü" already and it always did.


And in the light of this, probably if you want (write "ü") to specifically
ouptut "ü" rather than "\374", then maybe/probably you are doing something
wrong, as reflected by Marc through what he said in the previous email


2015-05-30 0:38 GMT+05:30 Marc Feeley <feeley at iro.umontreal.ca>:
>
> This [escaping of all >=128 chars] allows Scheme strings that are printed
> to be processed by external tools which only support ASCII.  For example,
> if you pretty-print to a file a function that contains the string "ü" you
> will end up with a file containing the sequence of 6 characters "\374"
> which any editor can edit. In other words, it makes minimal requirements on
> the features of external tools (not just editors, but email clients if you
> want to email some Scheme code, shell utilities, C compilers, other Scheme
> systems, etc).



The only strict usecase I can see of having >=128 characters not escaped by
default, is if you have a class full of elementary school kids with another
language than English as mother tongue, who find it so confusing during
basic programming classes that them typing in 'über is object-serialized as
|\374ber|, that you need to do something about it.


This question of escaping we're discussing, is very similar to the
discussion of if it's good practice to use unicode characters in variable
names.

Until now I never saw any project do that.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20150531/9e18cfd0/attachment.htm>


More information about the Gambit-list mailing list