WOW! I m truly impressed by your perseverance and patience in using work-in-progress quality code. I hope you can turn your experience into an article for others to benefit. If there was a “Gambit badge of merit” you would have earned it.
Bignums… they *are* implemented in the Gambit library. But println is rather simplistic and doesn’t know how to print them. You would need to use write, display or pretty-print… if they were working properly (apparently something broke recently with threads and I/O… looking into this is on my TODO list). So for now you need to combine println and number->string or object->string, for example:
(println (object->string (expt 2 100000)))
FYI the bignums are implemented in Scheme in lib/_num.scm .
For syntax-case, you could try using Gambit’s unhygienic syntax-case like this:
(##define-syntax define-syntax (lambda (src) (##include "~~lib/_syntax.scm") (syntax-case src () ((_ id fn) #'(##define-syntax id (##lambda (##src) (##include "~~lib/_syntax.scm") (fn ##src)))))))
(define-syntax while (lambda (src) (syntax-case src () ((_ expr body ...) #'(let while-loop () (if expr (begin body ... (while-loop))))))))
(define i 0)
(while (fx< i 10) (println (fx* i i)) (set! i (fx+ i 1)))
I hope that gets you closer to your goal.
Marc
On Jul 20, 2019, at 8:23 PM, Paulo Silva Filho paulosfilho@gmail.com wrote:
Sires,
I got the Gambit Javascript Backend working suiting my needs yesterday. It was somewhat difficult and a little overworking, but I got my needed results.
Since the job to do so is not trivial, I'm posting my steps, here, to let it documented. Maybe, as my work advances, I can create an article on Medium, to show everybody how to create small, efficient and compiled scheme applications in the web.
My objectives were:
1 - To compile scheme scripts in Javascript, to import them from Javascript projects or from any language running on Javascript or compiling to Javascript. So, interpreters, like BiwaScheme, or Chibi-Scheme on WebASM wouldn't suit. If I couldn't achieve this objective, I would try a workaround with some Scheme interpreter in Javascript.
2 - These compiled scripts be small, or very small, since the algorithms I need to compile to Javascript are simple, only to make some numeric analysis, and User Interface manipulation in real time. Javascript has a very good performance to do these tasks, but the language is cumbersome to program these kinds of programs. I know this is a matter of opinion, but, in my opinion, scheme is better than Javascript to do this due scheme consistency and clarity.
3 - I would need to call to and from javascript native calls, to enable interoperability. Usually, scheme environments and native environment are orthogonal, due Scheme specificities (like call/cc and required TCO), but this orthogonality is no problem to me, since I can wrap the compiled scheme code in a consistent Javascript interface.
4 - The language to be compiled needed to be TRUE SCHEME. No Common-Lisp, nor Scheme-Like, nor Clojure. Clojure has a very competent Javascript Compiler, but, as a language, it's not suitable to be, due its complexity and compromises, regarding TCO and continuations.
Considering these objectives, I started to look for scheme->javascript COMPILERS, not interpreters. As a matter of fact, I've been using BiwaScheme as a Scheme interpreter in Javascript, and the only thing I'm really missing is syntax-case and syntax-rules macros, since BiwaScheme doesn't support define-syntax clauses. I've had some performance problems with BiwaScheme, but I've realized, during the experiments I was perfoming with Biwa and other compiled Schemes, that these performance issues are due my misuse of the interpreter, not due BiwaScheme properties. BiwaScheme has an excellent call/cc and TCO implementation, and if the algorithms use these, avoiding populating the stack (which access in BiwaScheme is slow), this Scheme implementation performs really well.
So, I decided to try these Scheme->Javascript compilers:
1 - Spock, from Chicken scheme. 2 - Gambit-C Javascript Compiler. 3 - Whalesong or RacketScript, from the Racket Project. 4 - Scheme2JS
I thought Whalesong and RacketScript somewhat low documented, although belonging to a huge project like Racket. So I decided to pass it. And I didn't want to deal with the huge complexity of Racket - I needed a more focused Scheme project to deal, since these smaller projects, by the rule, have more approachable people to deal with. My choice on Gambit has just proved this point. Other point, is that I would need to hack any Scheme->Javascript compiler, and in a smaller, more focused, project, it's easier to do.
Scheme2JS uses the GPL license, and, since I'll need to distribute my software in a proprietary format, although I think I could (and I will) open-source the entire Javascript Scheme Stack I'm developing, I discarded Scheme2JS as as an option.
The other two options were Spock and Gambit-C. After reading this article: http://www.schemeworkshop.org/2012/papers/thivierge-feeley-paper-sfp12.pdf, I decided to give Gambit-C a try.
Since I'm using windows as operating system, I installed Gambit from Chocolatey and tried to use the information I've gotten from the Gambit mail group to compile some Javascript code. No success with default Gambit installation from Chocolatey (version 4.6.6).
Then I tried a more up to date version, from Gambit web site (version 4.9.3). No success at all. So, I contacted the newsgroup by myself, with my doubts, and I realized that Javascript Backend is an experimental one, so I decided to use a very up to date version of Gambit, from source code repository, in Github.
No success too. So I decided to hack the Javascript created by Gambit, and I realized that this javascript, although complex, is very, very readable - and I discovered that the generated code has some undeclared objects (mainly the Gambit object), with procedural functions replacing their previous usage. So I decided to hack this generated code and, voilà, the script worked suitably in Node.js and in Web browser (Firefox and Chrome).
This process has been a little cumbersome and overworking, but the results were beautiful.
I started the process installing MSYS2 from Chocolatey:
;; Windows CMD Shell:
choco install -y msys2
Then, after installing msys2, I loaded the bash shell, and installed the development environment:
;; MSYS2 Bash Shell: $ pacman -Syu $ pacman -S base-devel gcc git
Then, my computer was ready to build Gambit-C. The next steps can be done in a Linux and MacOS X machines, as well, since they have the correct tools installed.
I created a directory to receive the Gambit code, then I download its source from Github, compiled it and installed it in a specific directory on my computer, to avoid clashes with the default installed Gambit distribution:
;; MSYS2 Bash Shell: $ git clone git@github.com:gambit/gambit.git $ cd gambit $ git log # To show the EXACT version of the repository, if you want to repeat this process I'm presenting here, without any surprise: commit 55a0b75ed9391c6e27475b251bd2bb9babfd9aab (HEAD -> master, origin/master, origin/HEAD) Author: Marc Feeley feeley@iro.umontreal.ca Date: Tue Jul 16 08:46:48 2019 -0400
Don't ignore the settings parameter of ##open-predefined
... Other commit information... ... ... $ ./configure --prefix=/c/tools/gambit-c --enable-cplusplus --enable-c-opt --enable-gcc-opts --enable-shared --enable-single-host configure: loading site script /etc/config.site checking build system type... x86_64-pc-msys checking host system type... x86_64-pc-msys checking target system type... x86_64-pc-msys checking for gcc... gcc checking whether the C compiler works... yes ... Lots of output!!! ... ... $ make making all in include make[1]: Entering directory '/c/Users/.../gambit/include' make[1]: Leaving directory '/c/Users/.../gambit/include' making all in lib make[1]: Entering directory '/c/Users/.../gambit/lib' making deselect-gen-for-commit in gambit ... Lots of output!!! ... ... $ make install ... ... Lots of output!!! ... ...
On you need to update suitably the next environment variables:
- LIBRARY_PATH: add "/tools/gambit-c/lib directory or the correspondent directory you installed gambit."
- INCLUDE: add "/tools/gambit-c/include directory or the correspondent directory you installed gambit."
- LIB: add "/tools/gambit-c/lib directory or the correspondent directory you installed gambit."
- CPATH: add "/tools/gambit-c/lib directory or the correspondent directory you installed gambit."
- PATH: : add "/tools/gambit-c/lib and /tools/gambit/bin directories or the correspondent directories you installed gambit."
After that, you need to return to your gambit source code folder and build the Gambit javascript library:
;; MSYS2 Bash Shell, going to the gambit source, downloaded from Git: $ cd gambit
Then, go to the lib directory, and build the Javascript runtime:
;; MSYS2 Bash Shell: $ cd lib $ make _gambit.js # This command finally build the needed Gambit Javascript runtime LD_LIBRARY_PATH=..: ../gsc/gsc -:~~bin=../bin,~~lib=../lib,~~include=../include -f -target js -prelude "(define-cond-expand-feature|enable-type-checking|)(define-cond-expand-feature|disable-auto-forcing|)(define-cond-expand-feature|enable-sharp-dot|)(define-cond-expand-feature|enable-bignum|)(define-cond-expand-feature|enable-ratnum|)(define-cond-expand-feature|enable-cpxnum|)(define-cond-expand-feature|disable-smp|)(##include"../lib/header.scm")" -o _gambit.js ../lib/_univlib.scm $ cp _gambit.js /c/tools/gambit-c/lib/_gambit.js # This command publish the Javascript runtime to the installed Gambit-C. $
At this point, the Gambit-C Javascript Backend is fully installed. Maintain Gambit-C source code in this state, in the case of any future updates, which can be common, since the Javascript backend is experimental.
Now, we need to build and execute a sample. You can build such sample with the next instructions.
First, in a suitable place, create a directory:
;; MSYS2 bash (assuming the new installed Gambit is in PATH, and all other environment variables are set) $ mkdir test $ cd test
Initialize the folder as an NPM project, to use Uglify-JS to minify a little the generated files. The minification isn't very efficient, but improve the file sizes. You need to install npm and put it on path:
;; Windows CMD Shell, with Administrative roles:
choco install -y npm
Then, in the created test directory, perform the next proceedings: ;; MSYS2 bash $ npm init ... Accept all given options, or customize them in according to your needs. $ npm i uglify-js
Then, now, create the files below:
;; FILE app.scm (declare (standard-bindings) (extended-bindings) (not safe) (not run-time-bindings))
(println ">>> app.scm")
(define (fib n) (fib-iter 1 0 n))
(define (fib-iter a b count) (if (= count 0) b (fib-iter (+ a b) a (- count 1))))
(for-each (lambda (x) (println (fib x))) (append '(1 2 3 4 5 6 7 8 9 10 11 12 13 14 15) '(16 17 18 19 20 21 22 23 24 25 26 27 28 29 30)))
(let* ((start (current-milliseconds)) (result (fib 35)) (end (current-milliseconds))) (js-alert (fx- end start)) (js-alert result))
(console.log 123456) (console.log "This is a scheme string!!!") (console.log 22.22) (console.log '(1 2 3 4)) (console.log '(1 . 2))
(js-global "teste" (lambda () (println "Function test")))
(js-global "fib" ; Javascript exported calls can't be recursive!!!!! (lambda (n) (fib n)))
;; FILE lib.scm (declare (standard-bindings) (extended-bindings) (not safe) (not run-time-bindings))
(println ">>> lib.scm")
(define (for-each f lst) (if (pair? lst) (begin (f (car lst)) (for-each f (cdr lst)))))
(define (append lst1 lst2) (if (pair? lst1) (cons (car lst1) (append (cdr lst1) lst2)) lst2))
(define (current-milliseconds) (##inline-host-expression "Date.now()"))
(define (js-alert obj) (##inline-host-statement "console.log(g_scm2js(@1@));" obj))
(define (console.log obj) (##inline-host-statement "console.log(g_scm2js(@1@));" obj))
(define (js-global name obj) (##inline-host-statement "window[g_scm2js(@1@)]=g_scm2js(@2@);" name obj))
(app#)
;; FILE makefile GSC=gsc -target js
all: run
lib.js: lib.scm $(GSC) -c lib.scm
app.js: app.scm $(GSC) -c app.scm
app_.js: lib.js app.js $(GSC) -link -l lib app.js
linked_app.js: lib.js app.js app_.js $(GSC) -c lib.scm cat app_.js lib.js app.js > linked_app.js wc *.js
run: linked_app.js node linked_app.js
clean: rm -f lib.js app.js app_.js linked_app.js
;; FILE patch.sh #!/usr/bin/env bash sed -i 's/Gambit.sp/g_sp/g' linked_app.js sed -i 's/Gambit.stack/g_stack/g' linked_app.js sed -i 's/Gambit.void_obj/void 0/g' linked_app.js sed -i 's/Gambit.nargs/g_nargs/g' linked_app.js sed -i 's/Gambit.stack/g_stack/g' linked_app.js sed -i 's/Gambit.underflow/g_underflow/g' linked_app.js sed -i 's/Gambit.trampoline/g_trampoline/g' linked_app.js sed -i 's/Gambit.scm2js/g_scm2js/g' linked_app.js sed -i 's/Gambit.r/g_r/g' linked_app.js sed -i 's/Gambit.js2scm_call/g_js2scm_call/g' linked_app.js sed -i 's/Gambit.scm2js/g_scm2js/g' linked_app.js sed -i 's/Gambit.String/G_ScmString/g' linked_app.js sed -i 's/Gambit.Flonum/G_Flonum/g' linked_app.js sed -i 's/Gambit.Pair/G_Pair/g' linked_app.js sed -i 's/Gambit.Structure/G_Structure/g' linked_app.js sed -i 's/Gambit./g_/g' linked_app.js sed -i 's/var fs = require(.*);//g' linked_app.js sed -i 's/var g_argv;/var g_argv = ["web-browser"];/*/g' linked_app.js sed -i 's/g_argv = arguments;/g_argv = arguments;*//g' linked_app.js npx uglifyjs --compress --mangle --output linked_app.min.js -- linked_app.js
;; FILE index.html
<!DOCTYPE html>
<html> <head> <title>fib</title> <script type="text/javascript"> function print(x) { console.log(x); } </script> <script type="text/javascript" src="linked_app.min.js"></script> </head> <body></body> </html>
After creating all these files, you can create the project with the next proceedings:
;; MSYS2 bash shell: $ chmod ugo+x patch.sh # Turn the patch script executable $ make linked_app.js gsc -target js -link -l lib app.js gsc -target js -c lib.scm cat app_.js lib.js app.js > linked_app.js wc *.js 889 2224 20534 app.js 722 2048 16061 app_.js 427 1025 10221 lib.js 2038 5297 46816 linked_app.js 4076 10594 93632 total
If you only run Make, the scripts will be compiled, but the patches aren't applied, so the process will fail.
After you created the linked_app.js file, patch it:
;; MSYS2 bash shell: $ ./patch.sh
After that, the script, compatible with web browser, is ready.
You can load it using your default browser:
$ explorer index.html
And, in the browser (Firefox or Chrome), you can hit <F12>, and select the console. And you'll see the output:
lib.scm app.scm
1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765 10946 17711 28657 46368 75025 121393 196418 317811 514229 832040 2 9227465 123456 This is a scheme string!!! 22.22 Object(4) [ 1, 2, 3, 4 ] Object [ 1 ]
The generated script isn't compatible with node.js, because the function js-global assume the existence of the "window" object, which is absent in Node.js. But, if in this function you replace "window" by "global", the script will work.
So:
;; MSYS2 bash shell: $ rm *.js $ sed -i 's/window/global/g' lib.scm $ make linked_app.js $ ./patch.sh $ node linked_app.min.js 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765 10946 17711 28657 46368 75025 121393 196418 317811 514229 832040 0 9227465 123456 This is a scheme string!!! 22.22 { '0': 1, '1': 2, '2': 3, '3': 4 } { '0': 1 }
But one of the most intersting things is to call Scheme code from javascript:
;; MSYS2 bash shell: $ node
.load linked_app.min.js fib(42)
267914296
Strangely, fib(42) is the biggest value I can load using the fib function. Some problem load the numerical tower. But, the entire process is very, very interesting and well suitable to my needs to generate reusable javascript libraries using Scheme. The performance is very good.
It's possible to load the entire Gambit runtime in a javascript file, which can be suitable to create Scheme Node.js programs. First, create the next files:
;; NodeApp.scm (declare (standard-bindings) (extended-bindings) (not safe) (fixnum) (block) )
;; All function, variables and other definitions before any execution:
(define (console.log obj) (##inline-host-statement "console.log(g_scm2js(@1@));" obj))
(define (js-global name obj) (##inline-host-statement "global[g_scm2js(@1@)]=g_scm2js(@2@);" name obj))
(define (fib n) (fib-iter 1 0 n))
(define (fib-iter a b count) (if (= count 0) b (fib-iter (+ a b) a (- count 1))))
(js-global "teste" (lambda () (console.log "Function test")))
(js-global "fib" ; Javascript exported calls can't be recursive!!!!! (lambda (n) (fib n)))
; Then some commands executing things:
(for-each (lambda (x) (console.log (fib x))) (append '(1 2 3 4 5 6 7 8 9 10 11 12 13 14 15) '(16 17 18 19 20 21 22 23 24 25 26 27 28 29 30) '(31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 300)))
(console.log 123456) (console.log "This is a scheme string!!!") (console.log 22.22) (console.log '(1 2 3 4)) (console.log '(1 . 2))
;; patch-app.sh #!/usr/bin/env bash sed -i 's/Gambit.sp/g_sp/g' NodeApp sed -i 's/Gambit.stack/g_stack/g' NodeApp sed -i 's/Gambit.void_obj/void 0/g' NodeApp sed -i 's/Gambit.nargs/g_nargs/g' NodeApp sed -i 's/Gambit.stack/g_stack/g' NodeApp sed -i 's/Gambit.underflow/g_underflow/g' NodeApp sed -i 's/Gambit.trampoline/g_trampoline/g' NodeApp sed -i 's/Gambit.scm2js/g_scm2js/g' NodeApp sed -i 's/Gambit.r/g_r/g' NodeApp sed -i 's/Gambit.js2scm_call/g_js2scm_call/g' NodeApp sed -i 's/Gambit.scm2js/g_scm2js/g' NodeApp sed -i 's/Gambit.String/G_ScmString/g' NodeApp sed -i 's/Gambit.Flonum/G_Flonum/g' NodeApp sed -i 's/Gambit.Pair/G_Pair/g' NodeApp sed -i 's/Gambit.Structure/G_Structure/g' NodeApp sed -i 's/Gambit./g_/g' NodeApp sed -i 's/var fs = require(.*);//g' NodeApp sed -i 's/var g_argv;/var g_argv = ["web-browser"];/*/g' NodeApp sed -i 's/g_argv = arguments;/g_argv = arguments;*//g' NodeApp npx uglifyjs --compress --mangle --output Min-NodeApp -- NodeApp mv Min-NodeApp NodeApp
Then make the executable:
;; MSYS2 bash shell: $ chmod ugo+x patch-app.sh $ gsc -target js -exe NodeApp.scm $ ./patch-app.sh # This will take a while!
The NodeApp executable is 15MB heavy. It is a very big file. But it contains the ENTIRE Gambit-C runtime - it is a COMPLETE SCHEME, which can be ran in Node.JS platform! And, although very large, NodeApp load and run very fast:
;; MSYS2 bash shell: $ node NodeApp 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765 10946 17711 28657 46368 75025 121393 196418 317811 514229 832040 1346269 2178309 3524578 5702887 9227465 14930352 24157817 39088169 63245986 102334155 165580141 267914296 433494437 701408733 1134903170 2.2223224462942035e+62 123456 This is a scheme string!!! 22.22 { '0': 1, '1': 2, '2': 3, '3': 4 } { '0': 1 } ##thread-end-with-uncaught-exception! not implemented yet
I don't know what is this error ##thread-end-with-uncaught-exception! not implemented yet, but it didn't bothered me, at this moment. In the future I can ask why this problem is happening and how to solve it, and I'm happy, because it seems the numerical tower is accessible from Javascript. Unfortunately, infinite precision integers aren't available - but this is no problem, since a BigNum library can be implemented.
The error ##thread-end-with-uncaught-exception thrown by this app hindered me to turn the generated NodeApp application into a Node.js library, which can be reused from Javascript; so you can help me to solve this problem, and we can turn these libraries accessible from Javascript. Loading the app from the Web Browser is possible, and I can use this entire file as a library, but from Node.JS I couldn't. I'll look for the samples sent to me to know if this problem is already solved, but, if no, can you help me to solve this exception, or to know how to avoid it?
After all this work, I thank you very much by the help.
The next steps are discovering how to solve the ##thread-end-with-uncaught-exception error, to learn how to use define-syntax/syntax-rules macros on Javascript Backend, and how to use define-macro in Javascript backend (if these macros can be used, at all), solve the numerical tower problem in the small scripts generated using makefiles, and how to import and define the runtime library for small javascript libraries, created using makefiles. I'll need to hack into the Gambit libraries, to discover how to extract from them the default library functions and definitions for my web scheme runtime.
After I got the suitable gambit-scheme libraries working, without exceptions, and modularized, I can write a Medium article, to spread that Clojure isn't the only advanced Lisp usable to create Javascript libraries and programs, in Node.js and in Web. Any help in my endeavor is welcome.
Thank you very much, best regards and bye,
Paulo Silva-Filho.
Em sex, 19 de jul de 2019 às 15:23, Paulo Silva Filho paulosfilho@gmail.com escreveu: Sires,
I've realized that the examples seemed to be built using a development version of Gambit-C. I tried on a very stable version (4.6.6), which is from six years ago, and I've tried from a more recent version, 4.9.3, from the default package installers, in Windows and Linux.
I've looked for the some hints about which version of Gambit is being used on all examples, and the features and files shown (like _univlib.scm, present in the development version of Gambit - directly from Git, on https://github.com/gambit/gambit/blob/master/lib/_univlib.scm) and I realized that the compiler version used need to be an experimental one.
So I'll try all the samples, again, but from a version of gambit compiled directly from the source code, downloaded from the main repository.
Thank you from all answers. I'll keep you in touch.
Em sex, 19 de jul de 2019 às 12:37, Marc Feeley feeley@iro.umontreal.ca escreveu: Like so:
% cd gambit % cd lib % make _gambit.js ../gsc/gsc -:~~bin=../bin,~~lib=../lib,~~include=../include -f -target js -prelude "(define-cond-expand-feature|enable-type-checking|)(define-cond-expand-feature|disable-auto-forcing|)(define-cond-expand-feature|enable-sharp-dot|)(define-cond-expand-feature|enable-bignum|)(define-cond-expand-feature|enable-ratnum|)(define-cond-expand-feature|enable-cpxnum|)(define-cond-expand-feature|disable-smp|)(##include"../lib/header.scm")" -o _gambit.js ../lib/_univlib.scm % cd .. % gsc/gsc -:=. -target js -exe app.scm % ./app 1 256 65536 16777216 4294967296 1099511627776 281474976710656 72057594037927936 18446744073709551616 4722366482869645213696 1208925819614629174706176 309485009821345068724781056 79228162514264337593543950336 20282409603651670423947251286016 5192296858534827628530496329220096 1329227995784915872903807060280344576 340282366920938463463374607431768211456 87112285931760246646623899502532662132736 22300745198530623141535718272648361505980416 5708990770823839524233143877797980545530986496 % cat app.scm (define n 20) (for-each (lambda (i) (println (object->string (expt 2 (* i 8))))) (iota n)) (exit)
This should work for the other backends too. Note that there’s some bit rot of lib/univlib.scm, so there are issues with some parts of the Gambit library (for example write and pretty-print are broken, (exit) is needed to avoid an exception on exit, etc). You can revert to previous recent releases of Gambit to get these working correctly.
Sorry but lots of things are in flux right now…
Marc
On Jul 19, 2019, at 11:08 AM, Amirouche Boubekki amirouche.boubekki@gmail.com wrote:
Sorry for the multiple emails!
Le ven. 19 juil. 2019 à 13:38, Marc Feeley feeley@iro.umontreal.ca a écrit : You should look at this recent thread: https://mailman.iro.umontreal.ca/pipermail/gambit-list/2019-July/009103.html
In the above mail thread, it is described how to compile a scheme file into javascript WITHOUT "Gambit runtime".
Q: How can I bundle the Gambit runtime into the compiled JavaScript file?
Thanks in advance!
-- Paulo Silva Filho
-- Paulo Silva Filho
Marc
Marc