[gambit-list] Some UTF-8 support issues.
Marc Feeley
feeley at iro.umontreal.ca
Mon Sep 7 11:13:07 EDT 2009
On 7-Sep-09, at 9:13 AM, Ramil Farkhshatov wrote:
> Hello.
>
> 1. WRITE and PP procedures output unicode (UTF-8 with nonlatin
> characters) strings in such form:
> "\uXXXX\uXXXX" ; when run with t8 option
> "\XXX\XXX" ; when run without t8 option
> Also procedures returning strings output in REPL acts in same manner.
I don't get this behavior with v4.5.1. Which version are you using?
> Is there a way to change that behaviour?
You can change the readtable's "max-unescaped-char" attribute:
(define (set-no-escapes-for-unicode! port)
(output-port-readtable-set!
port
(readtable-max-unescaped-char-set
(output-port-readtable port)
(integer->char #x10ffff))))
(define (go)
(pretty-print
(list->string (map integer->char
(list #x80 #x100 #x10000)))))
(go)
(set-no-escapes-for-unicode! (current-output-port))
(go)
> 2. Have troubles operating files with unicode names when gsi runned
> with t8 option.
>
> With t8 option:
>> (file-exists? "~/develop/каталог")
> #f
>
> Without t8 option:
>> (file-exists? "~/develop/каталог")
> #f
>
> OPEN-DIRECTORY cannot open directories with unicode names:
>> (define fd (open-directory "~/develop/каталог/"))
> *** ERROR IN (console)@20.12 -- (Argument 1) Can't convert to C char-
> string
> (open-directory "~/develop/\u043a\u0430\u0442\u0430\u043b\u043e
> \u0433/")
>
> Additional info:
> % locale
> LANG=ru_RU.UTF-8
> LC_CTYPE="ru_RU.UTF-8"
> LC_NUMERIC="ru_RU.UTF-8"
> LC_TIME="ru_RU.UTF-8"
> LC_COLLATE=C
> LC_MONETARY="ru_RU.UTF-8"
> LC_MESSAGES="ru_RU.UTF-8"
> LC_PAPER="ru_RU.UTF-8"
> LC_NAME="ru_RU.UTF-8"
> LC_ADDRESS="ru_RU.UTF-8"
> LC_TELEPHONE="ru_RU.UTF-8"
> LC_MEASUREMENT="ru_RU.UTF-8"
> LC_IDENTIFICATION="ru_RU.UTF-8"
> LC_ALL=
>
> % echo $GAMBCOPT
> dar,t8E,f8,-8
I'm not sure what should be the right behavior. Currently, a latin-1
encoding will be used for the filenames. There is very little code to
change in the runtime system to use UTF-8 encoding for the filenames,
but this means that filesystems using plain character (latin-1)
encoding for filenames will stop working. I could add a configure
option, such as --enable-path-encoding=UTF-8 . But that begs the
question: what's the right default? Is the default OS dependent?
Marc
More information about the Gambit-list
mailing list