On Fri, May 19, 2023 at 6:27 PM Bradley Lucier <lucier@math.purdue.edu> wrote:

Unicode has stretches of code-points that are invalid, doesn't it?  In
which case complement would always be complement against the full valid
Unicode set (which isn't a single contiguous range).
The ranges of valid Unicode characters are #x0000 to #xD7FF and #xE000 to #x10FFFF.  As far as char-set-contains? is concerned, it doesn't matter if the non-range #xD7FF to #xDFFF are included or excluded, as there is no way to create a character whose char->int value is in this range.  However, when it comes to enumeration (the cursor functions, -fold, -for-each, -map, ->list, ->string) you do need to special-case this non-range to avoid traversing it.  Alternatively, the special-casing can be done in the set operations, which is probably better because they are less used than the enumerators.
 
It appears to me that it would suffice to use John's representation for
all character sets, but write

char-set-contains?

in terms of

char-alphabetic?
char-lower-case?
char-numeric?
char-upper-case?
char-whitespace?

for some of the standard character sets.

The difficulty for a portable implementation of SRFI 14 is that the above functions often cover only ASCII or only Latin-1 in the native implementation, or only cover a subset of the assigned characters corresponding to an older version of Unicode (after the long-obsolete version 1.1, the set of all assigned characters only grows with the version, it never shrinks).  If you know that they are up-to-date, that's a reasonable approach, but if they are incomplete, it's better to do it the other way round: reimplement char-alphabetic? in terms of char-set-contains? and char-set-letter?, etc.

Note that IMO this statement in the definition of ucs-range->charset
  • If the requested range includes unassigned UCS values, these are silently ignored (the current UCS specification has "holes" in the space of assigned codes).
should be disregarded, because it makes these functions unnecessarily dependent on a specific version of Unicode.  However, attempts to specify the surrogate codes #\xD800 to #\xDFFF should indeed be excluded, as they cannot ever be assigned.

Maybe a few more codes could be added for P*, S*, M* and N* above 199 in
your encoding in char#.scm to have fast membership testing for all the
"standard" classes.'

That makes sense to me.