[gambit-list] SRFI 14 and data structures in gambit/lib/gambit/char/char#.scm
Bradley Lucier
lucier at math.purdue.edu
Fri May 19 18:25:56 EDT 2023
On 5/18/23 9:04 AM, Marc Feeley wrote:
> So for implementing SRFI 14 efficiently, I think the above encoding should be used when possible for the predefined character sets, i.e. char-set:lower-case, char-set:upper-case, char-set:letter, char-set:digit, etc. It will allow quickly testing membership. The encoding mentionned by John (which requires a binary search) is a good implementation for general character sets which typically have groups of contiguous characters in or out of the set. It would be wasteful in time and space to implement the predefined character sets with the general representation (for example each of char-set:lower-case, char-set:upper-case, char-set:letter occupy about 5 KB and require a binary search to test membership).
Marc: Thanks for your comments.
I have studied SRFI 14 a bit more in light of your comments.
It appears to me that it would suffice to use John's representation for
all character sets, but write
char-set-contains?
in terms of
char-alphabetic?
char-lower-case?
char-numeric?
char-upper-case?
char-whitespace?
for some of the standard character sets.
Maybe use a structure for character sets with a slot for a
membership-testing procedure, which would be a predefined, specialized,
fast procedure for many if not all the standard data sets:
char-set:lower-case Lower-case letters
char-set:upper-case Upper-case letters
char-set:title-case Title-case letters
char-set:letter Letters
char-set:digit Digits
char-set:letter+digit Letters and digits
char-set:graphic Printing characters except spaces
char-set:printing Printing characters including spaces
char-set:whitespace Whitespace characters
char-set:iso-control The ISO control characters
char-set:punctuation Punctuation characters
char-set:symbol Symbol characters
char-set:hex-digit A hexadecimal digit: 0-9, A-F, a-f
char-set:blank Blank characters -- horizontal whitespace
char-set:ascii All characters in the ASCII set.
char-set:empty Empty set
char-set:full All characters
but would use binary search for any user-defined character sets.
John gave the following Unicode character set definitions for the SRFI
14 standard character sets:
char-set:lower-case = property Lowercase
char-set:upper-case = property Uppercase
char-set:title-case = category Lt
char-set:letter = property Alphabetic
char-set:digit = category Nd
char-set:letter+digit = property Alphabetic + category Nd
char-set:graphic = category L* + category N* + category M*
category S* + category P*
char-set:printing = char-set:graphic + char-set:whitespace
char-set:whitespace = property White_Space
char-set:iso-control = 0000..001F + 007F..009F
char-set:punctuation = category P*
char-set:symbol = category S*
char-set:hex-digit = 0030..0039 + 0041..0046 + 0061..0066
char-set:blank = category Zs + 0009
char-set:ascii = 0000..007F
Your process-data routine in char#.scm does not seem to keep enough of
the property information to fill out the primitives of this table; the
missing parts seem to be
char-set:punctuation = category P*
char-set:symbol = category S*
char-set:graphic = category L* + category N* + category M*
category S* + category P* ;; M* and N*
Maybe a few more codes could be added for P*, S*, M* and N* above 199 in
your encoding in char#.scm to have fast membership testing for all the
"standard" classes.
Brad
PS: I still don't understand the table in char#.scm"
;; Encoding in unicode-class table:
;;
;; | UPPER | OTHER | LOWER |
;; |FLT: : : LF T=0 | LF U=0 :FLU: : | UT F=0 :FUT: : |
;; | F : L : T : LF T=0 | LF U=0 : F : L : U | UT F=0 : F : U : T |
;;
;; F | F : : : F | F : F : : | 0 : F : : |
;; L | : L : : L | L : : L : | 0 : 0 : : |
;; T | : : T : 0 | 0 : 0 : : | T : : : T |
;; U | 0 : : : 0 | 0 : : : U | U : : U : |
More information about the Gambit-list
mailing list