API to parse floats in Scheme
We are planning a simple SRFI to mae floats from integers (or from strings), since that's useful when parsing any kind of text syntax that has real numbers. The details are implementation-dependent and tricky to get right. Here's the spec we have so far: https://github.com/pre-srfi/make-float The API is: (inexact-real-from-strings int frac precision exp) -> real (inexact-real-from-integers int frac frac-length precision exp) -> real Example usage: (inexact-real-from-strings "-123" "45" 'e "6") ; => -123.45e6 (inexact-real-from-integers -123 45 2 'e 6) ; => -123.45e6 The 2 in the last example means two decimal places. It's needed to disambiguate between .6 .06 .006 .0006 etc. There are both math and Scheme wizards on this list. Could someone verify whether this is a sane design before we submit the SRFI draft? It's been ok'd by one mathematician already so I'm hopeful :) There's also an inverse API to serve as a backend for string formatting packages: (strings-from-inexact-real real frac-length) -> int; frac; exp (integers-from-inexact-real real frac-length) -> int; frac; exp
On May 12, 2021, at 3:43 PM, Lassi Kortela <lassi@lassi.io> wrote:
We are planning a simple SRFI to mae floats from integers (or from strings), since that's useful when parsing any kind of text syntax that has real numbers. The details are implementation-dependent and tricky to get right.
Here's the spec we have so far: https://github.com/pre-srfi/make-float
The API is:
(inexact-real-from-strings int frac precision exp) -> real (inexact-real-from-integers int frac frac-length precision exp) -> real
Example usage:
(inexact-real-from-strings "-123" "45" 'e "6") ; => -123.45e6 (inexact-real-from-integers -123 45 2 'e 6) ; => -123.45e6
The 2 in the last example means two decimal places. It's needed to disambiguate between .6 .06 .006 .0006 etc.
There are both math and Scheme wizards on this list. Could someone verify whether this is a sane design before we submit the SRFI draft? It's been ok'd by one mathematician already so I'm hopeful :)
There's also an inverse API to serve as a backend for string formatting packages:
(strings-from-inexact-real real frac-length) -> int; frac; exp (integers-from-inexact-real real frac-length) -> int; frac; exp
A few things that come to mind: 1) The procedure inexact-real-from-integers can’t create negative zero. Perhaps the sign should be separate from the “int” part. 2) In the code I have written in the past to convert strings to numbers all of the digits are accumulated in a single integer and the number of decimals is kept separately (your fac-length). The parameters “int” and “frac” could be combined into one. By doing this the frac-length and exp can also be combined. 3) I don’t see provisions to parse +/- infinity and nan. 4) Many languages now allow “_” in the digits, so it is not clear how this fits into your API. Maybe an extra parameter indicating the allowed separator (with a default of #f meaning no separator allowed). 5) The API for the procedure inexact-real-from-strings is not ideal to implement something like string->number, because there will be some calls to “substring” to extract each part. This will be slow and generate garbage. It would be better to have an API where the number of calls to “substring” can be minimized. For example the API could be a single procedure where the sign, int and exp parts can be a string, symbol or integer. Trailing parameters could be optional with a default of 0. Something like this: (make-inexact-real sign int exp precision) (make-inexact-real '- 0) ;; => -0.0 (make-inexact-real '+ 123 -2) ;; => 1.23 (make-inexact-real "+" "123" 10 'e) ;; => 1.23e12 You can always do (make-inexact-real "-" (string-append "123" "45") (- 6 2) 'e) instead of (inexact-real-from-strings "-123" "45" 'e "6") Marc
A few things that come to mind:
1) The procedure inexact-real-from-integers can’t create negative zero. Perhaps the sign should be separate from the “int” part.
2) In the code I have written in the past to convert strings to numbers all of the digits are accumulated in a single integer and the number of decimals is kept separately (your fac-length). The parameters “int” and “frac” could be combined into one. By doing this the frac-length and exp can also be combined.
3) I don’t see provisions to parse +/- infinity and nan.
4) Many languages now allow “_” in the digits, so it is not clear how this fits into your API. Maybe an extra parameter indicating the allowed separator (with a default of #f meaning no separator allowed).
5) The API for the procedure inexact-real-from-strings is not ideal to implement something like string->number, because there will be some calls to “substring” to extract each part. This will be slow and generate garbage. It would be better to have an API where the number of calls to “substring” can be minimized. For example the API could be a single procedure where the sign, int and exp parts can be a string, symbol or integer. Trailing parameters could be optional with a default of 0. Something like this:
(make-inexact-real sign int exp precision)
(make-inexact-real '- 0) ;; => -0.0 (make-inexact-real '+ 123 -2) ;; => 1.23 (make-inexact-real "+" "123" 10 'e) ;; => 1.23e12
You can always do
(make-inexact-real "-" (string-append "123" "45") (- 6 2) 'e)
instead of (inexact-real-from-strings "-123" "45" 'e "6")
Excellent comments! I'm glad I asked for your advice. The interface you suggest is indeed better than what we came up with. Maybe the sign could be a fixnum or boolean? This would simplify the procedure even further. Gambit's lib/_num.scm has the string->number implementation with the following subroutine, which takes all the arguments you suggest except for precision: (define (make-inexact-real sign uinteger exponent) (let ((n (if (and (##fixnum? uinteger) (##fixnum->flonum-exact? uinteger) (##fixnum? exponent) (##fx< (##fx- exponent) (##f64vector-length exact-10^n-table)) (##fx< exponent (##f64vector-length exact-10^n-table))) (if (##fx< exponent 0) (##fl/ (##fixnum->flonum uinteger) (##f64vector-ref exact-10^n-table (##fx- exponent))) (##fl* (##fixnum->flonum uinteger) (##f64vector-ref exact-10^n-table exponent))) (##exact->inexact (##* uinteger (##expt 10 exponent)))))) (if (##char=? sign #\-) (##flcopysign n (macro-inexact--1)) n))) Two other subroutines in the file: (substring->uinteger string radix i j) (float-substring->uinteger string i j) At least substring->uinteger should probably be standardized. number->string is too much powerful when one only wants to convert a span of digits. R7RS says the infinity and not-a-number syntax is: +inf.0 -inf.0 +nan.0 -nan.0 (same as +nan.0) Given that these are only three or four choices, do we need to support them in a subroutine, or is it enough if the user can check whether the text input is inf or nan and build one of these by hand? There is now SRFI 208 (NaN procedures) which can work with NaNs on the bit level. Supporting underscores (and space, commas, etc.) for separators is certainly useful, and it simplifies a parser if it doesn't have remove the separators itself. Should we add a char or string argument for which separators to skip (#f to deny all separators), or add another procedure?
On Thu, May 13, 2021 at 01:24:17AM +0300, Lassi Kortela wrote:
Supporting underscores (and space, commas, etc.) for separators is certainly useful, and it simplifies a parser if it doesn't have remove the separators itself. Should we add a char or string argument for which separators to skip (#f to deny all separators), or add another procedure?
Commas are a minefield. Some languages use a comma as a decimal point and a period as a mere separator. And I seem to demember one programming language (I forget which one) that used underscore as the 'e' for an exponent. -- hendrik
_______________________________________________ Gambit-list mailing list Gambit-list@iro.umontreal.ca https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list
I’m sorry, but I don’t know what the problem is that will be solved here. The inherently difficult part of inputting or outputting floating-point numbers is converting between a human-readable base 10 representation and a machine-usable base two representation while maintaining maximal accuracy (with some side conditions for special values, overflow, underflow, etc.) Are you proposing that this is what this pre-SRFI will do? Brad
Thanks for replying.
I’m sorry, but I don’t know what the problem is that will be solved here.
The inherently difficult part of inputting or outputting floating-point numbers is converting between a human-readable base 10 representation and a machine-usable base two representation while maintaining maximal accuracy (with some side conditions for special values, overflow, underflow, etc.)
Are you proposing that this is what this pre-SRFI will do?
Yes, exactly! Each file format and programming language has a slightly different number syntax, so parsers for those are probably not fruitful to standardize. (RnRS string->number is serviceable for languages other than Scheme, but it supports such a wide range of syntax that one must be very careful to give it pre-validated input only. This is easy to get wrong.) A fundamental `make-inexact-real` procedure would be simple and useful. In the other reply, Marc proposes a better interface for it than what we came up with.
On Wed, May 12, 2021 at 10:43:16PM +0300, Lassi Kortela wrote:
We are planning a simple SRFI to mae floats from integers (or from strings), since that's useful when parsing any kind of text syntax that has real numbers. The details are implementation-dependent and tricky to get right.
Here's the spec we have so far: https://github.com/pre-srfi/make-float
The API is:
(inexact-real-from-strings int frac precision exp) -> real (inexact-real-from-integers int frac frac-length precision exp) -> real
Example usage:
(inexact-real-from-strings "-123" "45" 'e "6") ; => -123.45e6 (inexact-real-from-integers -123 45 2 'e 6) ; => -123.45e6
The 2 in the last example means two decimal places. It's needed to disambiguate between .6 .06 .006 .0006 etc.
There are both math and Scheme wizards on this list. Could someone verify whether this is a sane design before we submit the SRFI draft? It's been ok'd by one mathematician already so I'm hopeful :)
There's also an inverse API to serve as a backend for string formatting packages:
(strings-from-inexact-real real frac-length) -> int; frac; exp (integers-from-inexact-real real frac-length) -> int; frac; exp
Conversion on a round trip between float and string form should converge rapidly. At the least, string to float to string to float should give the same value as just string to float and similarly for float to string to float to string. And there's the question of what to do with he various not-a-number floating point values. And what if you do not want an exponent in the resulting string? And finally, there are ties you want it read and written *exactly*, for example, using hexadecimal instead of decimal so you get every bit. (for this you'd want a symbol for the exponent 'e' that's different from the digit 'e'. -- hendrik
Conversion on a round trip between float and string form should converge rapidly. At the least, string to float to string to float should give the same value as just string to float and similarly for float to string to float to string.
The above is above my paygrade :) Are there established methods to do it? R7RS gives the following references which may or may not do it: Robert G. Burger and R. Kent Dybvig. Printing floating-point numbers quickly and accurately. In Proceedings of the ACM SIGPLAN ’96 Conference on Programming Language Design and Implementation, pages 108–116. William Clinger. How to read floating point numbers accurately. In Proceedings of the ACM SIGPLAN ’90 Conference on Programming Language Design and Implementation, pages 92–101. Proceedings published as SIGPLAN Notices 25(6), June 1990.
And what if you do not want an exponent in the resulting string?
As in the difference between printf("%f", x) and printf("%e", x). Indeed, both cases should be covered.
And finally, there are ties you want it read and written *exactly*, for example, using hexadecimal instead of decimal so you get every bit. (for this you'd want a symbol for the exponent 'e' that's different from the digit 'e'.
Scheme may not have an established notation for this. For a low-level SRFI, would float<->bytevector be enough? R6RS has the following: (bytevector-ieee-single-native-ref bytevector k) (bytevector-ieee-single-ref bytevector k endianness) (bytevector-ieee-double-native-ref bytevector k) (bytevector-ieee-double-ref bytevector k endianness) (bytevector-ieee-single-native-set! bytevector k x) (bytevector-ieee-single-set! bytevector k x endianness) (bytevector-ieee-double-native-set! bytevector k x) (bytevector-ieee-double-set! bytevector k x endianness)
participants (4)
-
Hendrik Boom -
Lassi Kortela -
Lucier, Bradley J -
Marc Feeley