Sires,

I got the Gambit Javascript Backend working suiting my needs yesterday. It was somewhat difficult and a little overworking, but I got my needed results.

Since the job to do so is not trivial, I'm posting my steps, here, to let it documented. Maybe, as my work advances, I can create an article on Medium, to show everybody how to create small, efficient and compiled scheme applications in the web.

My objectives were:

1 - To compile scheme scripts in Javascript, to import them from Javascript projects or from any language running on Javascript or compiling to Javascript. So, interpreters, like BiwaScheme, or Chibi-Scheme on WebASM wouldn't suit. If I couldn't achieve this objective, I would try a workaround with some Scheme interpreter in Javascript.

2 - These compiled scripts be small, or very small, since the algorithms I need to compile to Javascript are simple, only to make some numeric analysis, and User Interface manipulation in real time. Javascript has a very good performance to do these tasks, but the language is cumbersome to program these kinds of programs. I know this is a matter of opinion, but, in my opinion, scheme is better than Javascript to do this due scheme consistency and clarity.

3 - I would need to call to and from javascript native calls, to enable interoperability. Usually, scheme environments and native environment are orthogonal, due Scheme specificities (like call/cc and required TCO), but this orthogonality is no problem to me, since I can wrap the compiled scheme code in a consistent Javascript interface.

4 - The language to be compiled needed to be TRUE SCHEME. No Common-Lisp, nor Scheme-Like, nor Clojure. Clojure has a very competent Javascript Compiler, but, as a language, it's not suitable to be, due its complexity and compromises, regarding TCO and continuations.

Considering these objectives, I started to look for scheme->javascript COMPILERS, not interpreters. As a matter of fact, I've been using BiwaScheme as a Scheme interpreter in Javascript, and the only thing I'm really missing is syntax-case and syntax-rules macros, since BiwaScheme doesn't support define-syntax clauses. I've had some performance problems with BiwaScheme, but I've realized, during the experiments I was perfoming with Biwa and other compiled Schemes, that these performance issues are due my misuse of the interpreter, not due BiwaScheme properties. BiwaScheme has an excellent call/cc and TCO implementation, and if the algorithms use these, avoiding populating the stack (which access in BiwaScheme is slow), this Scheme implementation performs really well.

So, I decided to try these Scheme->Javascript compilers:

1 - Spock, from Chicken scheme.
2 - Gambit-C Javascript Compiler.
3 - Whalesong or RacketScript, from the Racket Project.
4 - Scheme2JS

I thought Whalesong and RacketScript somewhat low documented, although belonging to a huge project like Racket. So I decided to pass it. And I didn't want to deal with the huge complexity of Racket - I needed a more focused Scheme project to deal, since these smaller projects, by the rule, have more approachable people to deal with. My choice on Gambit has just proved this point. Other point, is that I would need to hack any Scheme->Javascript compiler, and in a smaller, more focused, project, it's easier to do.

Scheme2JS uses the GPL license, and, since I'll need to distribute my software in a proprietary format, although I think I could (and I will) open-source the entire Javascript Scheme Stack I'm developing, I discarded Scheme2JS as as an option.

The other two options were Spock and Gambit-C. After reading this article: http://www.schemeworkshop.org/2012/papers/thivierge-feeley-paper-sfp12.pdf, I decided to give Gambit-C a try.

Since I'm using windows as operating system, I installed Gambit from Chocolatey and tried to use the information I've gotten from the Gambit mail group to compile some Javascript code. No success with default Gambit installation from Chocolatey (version 4.6.6).

Then I tried a more up to date version, from Gambit web site (version 4.9.3). No success at all. So, I contacted the newsgroup by myself, with my doubts, and I realized that Javascript Backend is an experimental one, so I decided to use a very up to date version of Gambit, from source code repository, in Github.

No success too. So I decided to hack the Javascript created by Gambit, and I realized that this javascript, although complex, is very, very readable - and I discovered that the generated code has some undeclared objects (mainly the Gambit object), with procedural functions replacing their previous usage. So I decided to hack this generated code and, voilà, the script worked suitably in Node.js and in Web browser (Firefox and Chrome).

This process has been a little cumbersome and overworking, but the results were beautiful.

I started the process installing MSYS2 from Chocolatey:

;; Windows CMD Shell:
>  choco install -y msys2

Then, after installing msys2, I loaded the bash shell, and installed the development environment:

;; MSYS2 Bash Shell:
$ pacman -Syu
$ pacman -S base-devel gcc git

Then, my computer was ready to build Gambit-C. The next steps can be done in a Linux and MacOS X machines, as well, since they have the correct tools installed.

I created a directory to receive the Gambit code, then I download its source from Github, compiled it and installed it in a specific directory on my computer, to avoid clashes with the default installed Gambit distribution:

;; MSYS2 Bash Shell:
$ git clone git@github.com:gambit/gambit.git
$ cd gambit
$ git log # To show the EXACT version of the repository, if you want to repeat this process I'm presenting here, without any surprise:
commit 55a0b75ed9391c6e27475b251bd2bb9babfd9aab (HEAD -> master, origin/master, origin/HEAD)
Author: Marc Feeley <feeley@iro.umontreal.ca>
Date:   Tue Jul 16 08:46:48 2019 -0400

    Don't ignore the settings parameter of ##open-predefined

...
Other commit information...
...
...
$ ./configure --prefix=/c/tools/gambit-c --enable-cplusplus --enable-c-opt --enable-gcc-opts --enable-shared --enable-single-host
configure: loading site script /etc/config.site
checking build system type... x86_64-pc-msys
checking host system type... x86_64-pc-msys
checking target system type... x86_64-pc-msys
checking for gcc... gcc
checking whether the C compiler works... yes
...
Lots of output!!!
...
...
$ make
making all in include
make[1]: Entering directory '/c/Users/.../gambit/include'
make[1]: Leaving directory '/c/Users/.../gambit/include'
making all in lib
make[1]: Entering directory '/c/Users/.../gambit/lib'
making deselect-gen-for-commit in gambit
...
Lots of output!!!
...
...
$ make install
...
...
Lots of output!!!
...
...

On you need to update suitably the next environment variables:
* LIBRARY_PATH: add "/tools/gambit-c/lib directory or the correspondent directory you installed gambit."
* INCLUDE: add "/tools/gambit-c/include directory or the correspondent directory you installed gambit."
* LIB: add "/tools/gambit-c/lib directory or the correspondent directory you installed gambit."
* CPATH: add "/tools/gambit-c/lib directory or the correspondent directory you installed gambit."
* PATH: : add "/tools/gambit-c/lib and /tools/gambit/bin directories or the correspondent directories you installed gambit."

After that, you need to return to your gambit source code folder and build the Gambit javascript library:

;; MSYS2 Bash Shell, going to the gambit source, downloaded from Git:
$ cd gambit

Then, go to the lib directory, and build the Javascript runtime:

;; MSYS2 Bash Shell:
$ cd lib
$ make _gambit.js # This command finally build the needed Gambit Javascript runtime
LD_LIBRARY_PATH=..: ../gsc/gsc -:~~bin=../bin,~~lib=../lib,~~include=../include -f -target js  -prelude "(define-cond-expand-feature|enable-type-checking|)(define-cond-expand-feature|disable-auto-forcing|)(define-cond-expand-feature|enable-sharp-dot|)(define-cond-expand-feature|enable-bignum|)(define-cond-expand-feature|enable-ratnum|)(define-cond-expand-feature|enable-cpxnum|)(define-cond-expand-feature|disable-smp|)(##include\"../lib/header.scm\")" -o _gambit.js ../lib/_univlib.scm
$ cp _gambit.js /c/tools/gambit-c/lib/_gambit.js # This command publish the Javascript runtime to the installed Gambit-C.
$

At this point, the Gambit-C Javascript Backend is fully installed. Maintain Gambit-C source code in this state, in the case of any future updates, which can be common, since the Javascript backend is experimental.

Now, we need to build and execute a sample. You can build such sample with the next instructions.

First, in a suitable place, create a directory:

;; MSYS2 bash (assuming the new installed Gambit is in PATH, and all other environment variables are set)
$ mkdir test
$ cd test

Initialize the folder as an NPM project, to use Uglify-JS to minify a little the generated files. The minification isn't very efficient, but improve the file sizes. You need to install npm and put it on path:

;; Windows CMD Shell, with Administrative roles:
> choco install -y npm

Then, in the created test directory, perform the next proceedings:
;; MSYS2 bash
$ npm init
... Accept all given options, or customize them in according to your needs.
$ npm i uglify-js

Then, now, create the files below:

;; FILE app.scm
(declare (standard-bindings) (extended-bindings) (not safe) (not run-time-bindings))

(println ">>> app.scm")

(define (fib n)
  (fib-iter 1 0 n))

(define (fib-iter a b count)
  (if (= count 0)
    b
    (fib-iter (+ a b) a (- count 1))))

(for-each
 (lambda (x)
   (println (fib x)))
 (append '(1 2 3 4 5 6 7 8 9 10 11 12 13 14 15)
         '(16 17 18 19 20 21 22 23 24 25 26 27 28 29 30)))

(let* ((start (current-milliseconds))
       (result (fib 35))
       (end (current-milliseconds)))
  (js-alert (fx- end start))
  (js-alert result))

(console.log 123456)
(console.log "This is a scheme string!!!")
(console.log 22.22)
(console.log '(1 2 3 4))
(console.log '(1 . 2))

(js-global "teste"
  (lambda () (println "Function test")))

(js-global "fib" ; Javascript exported calls can't be recursive!!!!!
  (lambda (n) (fib n)))

;; FILE lib.scm
(declare (standard-bindings) (extended-bindings) (not safe) (not run-time-bindings))

(println ">>> lib.scm")

(define (for-each f lst)
  (if (pair? lst)
      (begin
        (f (car lst))
        (for-each f (cdr lst)))))

(define (append lst1 lst2)
  (if (pair? lst1)
      (cons (car lst1) (append (cdr lst1) lst2))
      lst2))

(define (current-milliseconds)
  (##inline-host-expression "Date.now()"))

(define (js-alert obj)
  (##inline-host-statement "console.log(g_scm2js(@1@));" obj))


(define (console.log obj)
  (##inline-host-statement "console.log(g_scm2js(@1@));" obj))

(define (js-global name obj)
  (##inline-host-statement "window[g_scm2js(@1@)]=g_scm2js(@2@);" name obj))
 
(app#)

;; FILE makefile
GSC=gsc -target js

all: run

lib.js: lib.scm
$(GSC) -c lib.scm

app.js: app.scm
$(GSC) -c app.scm

app_.js: lib.js app.js
$(GSC) -link -l lib app.js

linked_app.js: lib.js app.js app_.js
$(GSC) -c lib.scm
cat app_.js lib.js app.js > linked_app.js
wc *.js

run: linked_app.js
node linked_app.js

clean:
rm -f lib.js app.js app_.js linked_app.js

;; FILE patch.sh
#!/usr/bin/env bash
sed -i 's/Gambit\.sp/g_sp/g' linked_app.js
sed -i 's/Gambit\.stack/g_stack/g' linked_app.js
sed -i 's/Gambit\.void_obj/void 0/g' linked_app.js
sed -i 's/Gambit\.nargs/g_nargs/g' linked_app.js
sed -i 's/Gambit\.stack/g_stack/g' linked_app.js
sed -i 's/Gambit\.underflow/g_underflow/g' linked_app.js
sed -i 's/Gambit\.trampoline/g_trampoline/g' linked_app.js
sed -i 's/Gambit\.scm2js/g_scm2js/g' linked_app.js
sed -i 's/Gambit\.r/g_r/g' linked_app.js
sed -i 's/Gambit\.js2scm_call/g_js2scm_call/g' linked_app.js
sed -i 's/Gambit\.scm2js/g_scm2js/g' linked_app.js
sed -i 's/Gambit\.String/G_ScmString/g' linked_app.js
sed -i 's/Gambit\.Flonum/G_Flonum/g' linked_app.js
sed -i 's/Gambit\.Pair/G_Pair/g' linked_app.js
sed -i 's/Gambit\.Structure/G_Structure/g' linked_app.js
sed -i 's/Gambit\./g_/g' linked_app.js
sed -i 's/var fs = require\(.*\);//g' linked_app.js
sed -i 's/var g_argv;/var g_argv = ["web-browser"];\/\*/g' linked_app.js
sed -i 's/g_argv = arguments;/g_argv = arguments;*\//g' linked_app.js
npx uglifyjs --compress --mangle --output linked_app.min.js -- linked_app.js

;; FILE index.html
<!DOCTYPE html>
<html>
<head>
    <title>fib</title>
    <script type="text/javascript">
        function print(x) { console.log(x); }
    </script>
    <script type="text/javascript" src="linked_app.min.js"></script>
</head>
<body></body>
</html>

After creating all these files, you can create the project with the next proceedings:

;; MSYS2 bash shell:
$ chmod ugo+x patch.sh # Turn the patch script executable
$ make linked_app.js
gsc -target js -link -l lib app.js
gsc -target js -c lib.scm
cat app_.js lib.js app.js > linked_app.js
wc *.js
  889  2224 20534 app.js
  722  2048 16061 app_.js
  427  1025 10221 lib.js
 2038  5297 46816 linked_app.js
 4076 10594 93632 total
 
 If you only run Make, the scripts will be compiled, but the patches aren't applied, so the process will fail.
 
 After you created the linked_app.js file, patch it:
 
;; MSYS2 bash shell:
$ ./patch.sh

After that, the script, compatible with web browser, is ready.

You can load it using your default browser:

$ explorer index.html

And, in the browser (Firefox or Chrome), you can hit <F12>, and select the console. And you'll see the output:

>>> lib.scm
>>> app.scm
1
1
2
3
5
8
13
21
34
55
89
144
233
377
610
987
1597
2584
4181
6765
10946
17711
28657
46368
75025
121393
196418
317811
514229
832040
2
9227465
123456
This is a scheme string!!!
22.22
Object(4) [ 1, 2, 3, 4 ]
Object [ 1 ]

The generated script isn't compatible with node.js, because the function js-global assume the existence of the "window" object, which is absent in Node.js. But, if in this function you replace "window" by "global", the script will work.

So:

;; MSYS2 bash shell:
$ rm *.js
$ sed -i 's/window/global/g' lib.scm
$ make linked_app.js
$ ./patch.sh
$ node linked_app.min.js
1                                    
1                                    
2                                    
3                                    
5                                    
8                                    
13                                  
21                                  
34                                  
55                                  
89                                  
144                                  
233                                  
377                                  
610                                  
987                                  
1597                                
2584                                
4181                                
6765                                
10946                                
17711                                
28657                                
46368                                
75025                                
121393                              
196418                              
317811                              
514229                              
832040                              
0                                    
9227465                              
123456                              
This is a scheme string!!!          
22.22                                
{ '0': 1, '1': 2, '2': 3, '3': 4 }  
{ '0': 1 }                          

But one of the most intersting things is to call Scheme code from javascript:

;; MSYS2 bash shell:
$ node
> .load linked_app.min.js
> fib(42)
267914296

Strangely, fib(42) is the biggest value I can load using the fib function. Some problem load the numerical tower. But, the entire process is very, very interesting and well suitable to my needs to generate reusable javascript libraries using Scheme. The performance is very good.

It's possible to load the entire Gambit runtime in a javascript file, which can be suitable to create Scheme Node.js programs. First, create the next files:

;; NodeApp.scm
(declare
  (standard-bindings)
  (extended-bindings)
  (not safe)
  (fixnum)
  (block)
)

;; All function, variables and other definitions before any execution:

(define (console.log obj)
  (##inline-host-statement "console.log(g_scm2js(@1@));" obj))

(define (js-global name obj)
  (##inline-host-statement "global[g_scm2js(@1@)]=g_scm2js(@2@);" name obj))
 
(define (fib n)
  (fib-iter 1 0 n))

(define (fib-iter a b count)
  (if (= count 0)
    b
    (fib-iter (+ a b) a (- count 1))))

(js-global "teste"
  (lambda () (console.log "Function test")))

(js-global "fib" ; Javascript exported calls can't be recursive!!!!!
  (lambda (n) (fib n)))

; Then some commands executing things:

(for-each
 (lambda (x)
   (console.log (fib x)))
 (append '(1 2 3 4 5 6 7 8 9 10 11 12 13 14 15)
         '(16 17 18 19 20 21 22 23 24 25 26 27 28 29 30)
         '(31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 300)))

(console.log 123456)
(console.log "This is a scheme string!!!")
(console.log 22.22)
(console.log '(1 2 3 4))
(console.log '(1 . 2))

;; patch-app.sh
#!/usr/bin/env bash
sed -i 's/Gambit\.sp/g_sp/g' NodeApp
sed -i 's/Gambit\.stack/g_stack/g' NodeApp
sed -i 's/Gambit\.void_obj/void 0/g' NodeApp
sed -i 's/Gambit\.nargs/g_nargs/g' NodeApp
sed -i 's/Gambit\.stack/g_stack/g' NodeApp
sed -i 's/Gambit\.underflow/g_underflow/g' NodeApp
sed -i 's/Gambit\.trampoline/g_trampoline/g' NodeApp
sed -i 's/Gambit\.scm2js/g_scm2js/g' NodeApp
sed -i 's/Gambit\.r/g_r/g' NodeApp
sed -i 's/Gambit\.js2scm_call/g_js2scm_call/g' NodeApp
sed -i 's/Gambit\.scm2js/g_scm2js/g' NodeApp
sed -i 's/Gambit\.String/G_ScmString/g' NodeApp
sed -i 's/Gambit\.Flonum/G_Flonum/g' NodeApp
sed -i 's/Gambit\.Pair/G_Pair/g' NodeApp
sed -i 's/Gambit\.Structure/G_Structure/g' NodeApp
sed -i 's/Gambit\./g_/g' NodeApp
sed -i 's/var fs = require\(.*\);//g' NodeApp
sed -i 's/var g_argv;/var g_argv = ["web-browser"];\/\*/g' NodeApp
sed -i 's/g_argv = arguments;/g_argv = arguments;*\//g' NodeApp
npx uglifyjs --compress --mangle --output Min-NodeApp -- NodeApp
mv Min-NodeApp NodeApp

Then make the executable:

;; MSYS2 bash shell:
$ chmod ugo+x patch-app.sh
$ gsc -target js -exe NodeApp.scm
$ ./patch-app.sh # This will take a while!

The NodeApp executable is 15MB heavy. It is a very big file. But it contains the ENTIRE Gambit-C runtime - it is a COMPLETE SCHEME, which can be ran in Node.JS platform! And, although very large, NodeApp load and run very fast:

;; MSYS2 bash shell:
$ node NodeApp
1
1
2
3
5
8
13
21
34
55
89
144
233
377
610
987
1597
2584
4181
6765
10946
17711
28657
46368
75025
121393
196418
317811
514229
832040
1346269
2178309
3524578
5702887
9227465
14930352
24157817
39088169
63245986
102334155
165580141
267914296
433494437
701408733
1134903170
2.2223224462942035e+62
123456
This is a scheme string!!!
22.22
{ '0': 1, '1': 2, '2': 3, '3': 4 }
{ '0': 1 }
##thread-end-with-uncaught-exception! not implemented yet

I don't know what is this error ##thread-end-with-uncaught-exception! not implemented yet, but it didn't bothered me, at this moment. In the future I can ask why this problem is happening and how to solve it, and I'm happy, because it seems the numerical tower is accessible from Javascript. Unfortunately, infinite precision integers aren't available - but this is no problem, since a BigNum library can be implemented.

The error ##thread-end-with-uncaught-exception thrown by this app hindered me to turn the generated NodeApp application into a Node.js library, which can be reused from Javascript; so you can help me to solve this problem, and we can turn these libraries accessible from Javascript. Loading the app from the Web Browser is possible, and I can use this entire file as a library, but from Node.JS I couldn't. I'll look for the samples sent to me to know if this problem is already solved, but, if no, can you help me to solve this exception, or to know how to avoid it?

After all this work, I thank you very much by the help.

The next steps are discovering how to solve the ##thread-end-with-uncaught-exception error, to learn how to use define-syntax/syntax-rules macros on Javascript Backend, and how to use define-macro in Javascript backend (if these macros can be used, at all), solve the numerical tower problem in the small scripts generated using makefiles, and how to import and define the runtime library for small javascript libraries, created using makefiles. I'll need to hack into the Gambit libraries, to discover how to extract from them the default library functions and definitions for my web scheme runtime.
Thank you very much, best regards and bye,

Paulo Silva-Filho.