[gambit-list] SRFI 14 and data structures in gambit/lib/gambit/char/char#.scm

Bradley Lucier lucier at math.purdue.edu
Fri May 19 18:25:56 EDT 2023


On 5/18/23 9:04 AM, Marc Feeley wrote:
> So for implementing SRFI 14 efficiently, I think the above encoding should be used when possible for the predefined character sets, i.e. char-set:lower-case, char-set:upper-case, char-set:letter, char-set:digit, etc. It will allow quickly testing membership. The encoding mentionned by John (which requires a binary search) is a good implementation for general character sets which typically have groups of contiguous characters in or out of the set.  It would be wasteful in time and space to implement the predefined character sets with the general representation (for example each of char-set:lower-case, char-set:upper-case, char-set:letter occupy about 5 KB and require a binary search to test membership).


Marc:  Thanks for your comments.

I have studied SRFI 14 a bit more in light of your comments.

It appears to me that it would suffice to use John's representation for 
all character sets, but write

char-set-contains?

in terms of

char-alphabetic?
char-lower-case?
char-numeric?
char-upper-case?
char-whitespace?

for some of the standard character sets.

Maybe use a structure for character sets with a slot for a 
membership-testing procedure, which would be a predefined, specialized, 
fast procedure for many if not all the standard data sets:

char-set:lower-case 	Lower-case letters
char-set:upper-case 	Upper-case letters
char-set:title-case 	Title-case letters
char-set:letter 	Letters
char-set:digit 	Digits
char-set:letter+digit 	Letters and digits
char-set:graphic 	Printing characters except spaces
char-set:printing 	Printing characters including spaces
char-set:whitespace 	Whitespace characters
char-set:iso-control 	The ISO control characters
char-set:punctuation 	Punctuation characters
char-set:symbol 	Symbol characters
char-set:hex-digit 	A hexadecimal digit: 0-9, A-F, a-f
char-set:blank 	Blank characters -- horizontal whitespace
char-set:ascii 	All characters in the ASCII set.
char-set:empty 	Empty set
char-set:full 	All characters

but would use binary search for any user-defined character sets.

John gave the following Unicode character set definitions for the SRFI 
14 standard character sets:

char-set:lower-case   = property Lowercase
char-set:upper-case   = property Uppercase
char-set:title-case   = category Lt
char-set:letter       = property Alphabetic
char-set:digit        = category Nd
char-set:letter+digit = property Alphabetic + category Nd
char-set:graphic      = category L* + category N* + category M*
                         category S* + category P*
char-set:printing     = char-set:graphic + char-set:whitespace
char-set:whitespace   = property White_Space
char-set:iso-control  = 0000..001F + 007F..009F
char-set:punctuation  = category P*
char-set:symbol       = category S*
char-set:hex-digit    = 0030..0039 + 0041..0046 + 0061..0066
char-set:blank        = category Zs + 0009
char-set:ascii        = 0000..007F

Your process-data routine in char#.scm does not seem to keep enough of 
the property information to fill out the primitives of this table; the 
missing parts seem to be

char-set:punctuation  = category P*
char-set:symbol       = category S*
char-set:graphic      = category L* + category N* + category M*
                         category S* + category P*  ;; M* and N*

Maybe a few more codes could be added for P*, S*, M* and N* above 199 in 
your encoding in char#.scm to have fast membership testing for all the 
"standard" classes.

Brad

PS:  I still don't understand the table in char#.scm"

;; Encoding in unicode-class table:
;;
;;   |       UPPER        |       OTHER        |       LOWER        |
;;   |FLT:   :   : LF T=0 | LF U=0 :FLU:   :   | UT F=0 :FUT:   :   |
;;   | F : L : T : LF T=0 | LF U=0 : F : L : U | UT F=0 : F : U : T |
;;
;; F | F :   :   :    F   |    F   : F :   :   |   0    : F :   :   |
;; L |   : L :   :    L   |    L   :   : L :   |   0    : 0 :   :   |
;; T |   :   : T :    0   |    0   : 0 :   :   |   T    :   :   : T |
;; U | 0 :   :   :    0   |    0   :   :   : U |   U    :   : U :   |




More information about the Gambit-list mailing list