On 5/18/23 9:04 AM, Marc Feeley wrote:
So for implementing SRFI 14 efficiently, I think the above encoding should be used when possible for the predefined character sets, i.e. char-set:lower-case, char-set:upper-case, char-set:letter, char-set:digit, etc. It will allow quickly testing membership. The encoding mentionned by John (which requires a binary search) is a good implementation for general character sets which typically have groups of contiguous characters in or out of the set. It would be wasteful in time and space to implement the predefined character sets with the general representation (for example each of char-set:lower-case, char-set:upper-case, char-set:letter occupy about 5 KB and require a binary search to test membership).
Marc: Thanks for your comments.
I have studied SRFI 14 a bit more in light of your comments.
It appears to me that it would suffice to use John's representation for all character sets, but write
char-set-contains?
in terms of
char-alphabetic? char-lower-case? char-numeric? char-upper-case? char-whitespace?
for some of the standard character sets.
Maybe use a structure for character sets with a slot for a membership-testing procedure, which would be a predefined, specialized, fast procedure for many if not all the standard data sets:
char-set:lower-case Lower-case letters char-set:upper-case Upper-case letters char-set:title-case Title-case letters char-set:letter Letters char-set:digit Digits char-set:letter+digit Letters and digits char-set:graphic Printing characters except spaces char-set:printing Printing characters including spaces char-set:whitespace Whitespace characters char-set:iso-control The ISO control characters char-set:punctuation Punctuation characters char-set:symbol Symbol characters char-set:hex-digit A hexadecimal digit: 0-9, A-F, a-f char-set:blank Blank characters -- horizontal whitespace char-set:ascii All characters in the ASCII set. char-set:empty Empty set char-set:full All characters
but would use binary search for any user-defined character sets.
John gave the following Unicode character set definitions for the SRFI 14 standard character sets:
char-set:lower-case = property Lowercase char-set:upper-case = property Uppercase char-set:title-case = category Lt char-set:letter = property Alphabetic char-set:digit = category Nd char-set:letter+digit = property Alphabetic + category Nd char-set:graphic = category L* + category N* + category M* category S* + category P* char-set:printing = char-set:graphic + char-set:whitespace char-set:whitespace = property White_Space char-set:iso-control = 0000..001F + 007F..009F char-set:punctuation = category P* char-set:symbol = category S* char-set:hex-digit = 0030..0039 + 0041..0046 + 0061..0066 char-set:blank = category Zs + 0009 char-set:ascii = 0000..007F
Your process-data routine in char#.scm does not seem to keep enough of the property information to fill out the primitives of this table; the missing parts seem to be
char-set:punctuation = category P* char-set:symbol = category S* char-set:graphic = category L* + category N* + category M* category S* + category P* ;; M* and N*
Maybe a few more codes could be added for P*, S*, M* and N* above 199 in your encoding in char#.scm to have fast membership testing for all the "standard" classes.
Brad
PS: I still don't understand the table in char#.scm"
;; Encoding in unicode-class table: ;; ;; | UPPER | OTHER | LOWER | ;; |FLT: : : LF T=0 | LF U=0 :FLU: : | UT F=0 :FUT: : | ;; | F : L : T : LF T=0 | LF U=0 : F : L : U | UT F=0 : F : U : T | ;; ;; F | F : : : F | F : F : : | 0 : F : : | ;; L | : L : : L | L : : L : | 0 : 0 : : | ;; T | : : T : 0 | 0 : 0 : : | T : : : T | ;; U | 0 : : : 0 | 0 : : : U | U : : U : |