[gambit-list] JavaScript backend

Marc Feeley feeley at iro.umontreal.ca
Tue Jan 12 22:38:49 EST 2016


I also tried using the standard Gambit library by replacing

(include "mylib.scm")

with

(include "~~lib/header.scm")
(include "~~lib/_univlib.scm")

This generates a 4.3M file with the optimize-dead-definitions rather than a 7M file without, so a 40% savings.  I think the standard Gambit library could be better structured to have fewer dependencies so as to yield more dead definitions.  It would be nice also to have a way to disable features that cause bloat, such as the I/O, eval and the REPL (these are probably not “dead” in my example because the primordial exception handler is written so that it pops up a REPL when there is an error, and the REPL requires eval, read, write, etc).

Marc



> On Jan 12, 2016, at 9:41 PM, Marc Feeley <feeley at iro.umontreal.ca> wrote:
> 
> I have added a compiler optimization which removes useless definitions in a program.  This can be useful to reduce the generated code size of a program by only including the procedures of a custom library that are needed.
> 
> Let me make an example to show what would be required to create a custom library for the universal backend, specifically with JavaScript as a target.  But the same optimization applies for the other targets.
> 
> Let's start with an empty custom library and a trivial program that prints "hello!":
> 
> % cat mylib.scm
> ;; File: "mylib.scm"
> % cat myprog.scm
> ;; File: "myprog.scm"
> (include "mylib.scm")
> (println "hello!")
> 
> These can be compiled to JavaScript and linked, to produce a single "p.js" file of code like this:
> 
> % gsc -target js -link -flat myprog.scm ; cat myprog_.js myprog.js > p.js
> 
> This can be run with node.js (on OS X) like this:
> 
> % node p.js
> hello!
> % wc p.js
>     322     893    7231 p.js
> 
> So the JavaScript code for this trivial program is about 7K of code.
> 
> Let's extend the program so that it finds the number of solutions to the 8-queens problem.  The Scheme code added to myprog.scm is:
> 
> (define trace? #f)
> 
> (define (nqueens n)
> 
>   (define (one-to n)
>     (let loop ((i n) (l '()))
>       (if (= i 0) l (loop (- i 1) (cons i l)))))
> 
>   (define (attempt x y z)
>     (if (null? x)
>         (if (null? y)
>             (begin (if trace? (println z)) 1)
>             0)
>         (+ (if (ok? (car x) 1 z)
>                (attempt (append (cdr x) y) '() (cons (car x) z))
>                0)
>            (attempt (cdr x) (cons (car x) y) z))))
> 
>   (define (ok? row dist placed)
>     (if (null? placed)
>         #t
>         (and (not (= (car placed) (+ row dist)))
>              (not (= (car placed) (- row dist)))
>              (ok? row (+ dist 1) (cdr placed)))))
> 
>   (attempt (one-to n) '() '()))
> 
> (println (nqueens 8))
> 
> Now let's compile that, and use the "-warnings" flag to see which predefined procedures are missing from our library:
> 
> % gsc -target js -warnings -link -flat myprog.scm
> *** WARNING -- "+" is not defined,
> ***            referenced in: ("/Users/feeley/gambit/work/myprog.js")
> *** WARNING -- "=" is not defined,
> ***            referenced in: ("/Users/feeley/gambit/work/myprog.js")
> *** WARNING -- "car" is not defined,
> ***            referenced in: ("/Users/feeley/gambit/work/myprog.js")
> *** WARNING -- "cons" is not defined,
> ***            referenced in: ("/Users/feeley/gambit/work/myprog.js")
> *** WARNING -- "append" is not defined,
> ***            referenced in: ("/Users/feeley/gambit/work/myprog.js")
> *** WARNING -- "null?" is not defined,
> ***            referenced in: ("/Users/feeley/gambit/work/myprog.js")
> *** WARNING -- "cdr" is not defined,
> ***            referenced in: ("/Users/feeley/gambit/work/myprog.js")
> *** WARNING -- "-" is not defined,
> ***            referenced in: ("/Users/feeley/gambit/work/myprog.js")
> *** WARNING -- "not" is not defined,
> ***            referenced in: ("/Users/feeley/gambit/work/myprog.js")
> 
> Except for println, all global variables that should be bound to a predefined procedure are undefined.
> 
> So we need to implement these procedures in mylib.scm .  We need to use an "extended-bindings" declaration, and a "not safe", for the compiler to inline the primitives used in the definition of the predefined procedures:
> 
> (declare (extended-bindings) (not safe))
> 
> (define (not x) (##not x))
> 
> (define (+ x y) (##fx+ x y))
> (define (- x y) (##fx- x y))
> (define (= x y) (##fx= x y))
> 
> (define (cons x y) (##cons x y))
> (define (car x) (##car x))
> (define (cdr x) (##car x))
> 
> (define (null? x) (##null? x))
> 
> (define (append x y)
>   (if (##pair? x)
>       (##cons (##car x) (append (##cdr x) y))
>       y))
> 
> Now the compilation and link will complete with no warnings:
> 
> % gsc -target js -warnings -link -flat myprog.scm ; cat myprog_.js myprog.js > p.js
> % node p.js 
> hello!
> 92
> % wc p.js
>    3136    8253   67284 p.js
> 
> So the program compiles to about 67K of JavaScript code.
> 
> In general mylib.scm will contain definitions for predefined Scheme procedures that are not required by myprog.scm .  For example:
> 
> (define (map f lst)
>   (if (##pair? lst)
>       (##cons (f (##car lst)) (map f (##cdr lst)))
>       '()))
> 
> (define (length lst)
>   (let loop ((lst lst) (i 0))
>     (if (##pair? lst)
>         (loop (##cdr lst) (##fx+ i 1))
>         i)))
> 
> (define (list-ref lst i)
>   (let loop ((lst lst) (i i))
>     (if (##fx= i 0)
>         (##car lst)
>         (loop (##cdr lst) (##fx- i 1)))))
> 
> (define (reverse lst)
>   (let loop ((lst lst) (result '()))
>     (if (##pair? lst)
>         (loop (##cdr lst) (##cons (##car lst) result))
>         result)))
> 
> With these definitions in mylib.scm the program compiles to about 75K of JavaScript code:
> 
> % gsc -target js -warnings -link -flat myprog.scm ; cat myprog_.js myprog.js > p.js
> % wc p.js
>    3538    9318   75872 p.js
> 
> With my latest commit, the compiler is now able to remove the procedure definitions that are not needed by the program when the "optimize-dead-definitions" declaration is used.  This line must be added to myprog.scm :
> 
> (declare (optimize-dead-definitions))
> 
> Now the program compiles to about 67K of JavaScript code.  By adding the "block" declaration:
> 
> (declare (block))
> 
> this can be lowered to 52K:
> 
> % gsc -target js -warnings -link -flat myprog.scm ; cat myprog_.js myprog.js > p.js
> % wc p.js
>    2144    5870   52490 p.js
> 
> The procedure definitions for not, +, -, =, cons, car, cdr, and null? are still being compiled to JavaScript, even though the compiler could inline those primitives.  This is achieved by adding this line to myprog.scm :
> 
> (declare (standard-bindings) (fixnum))
> 
> With this declaration the program compiles to about 15K of JavaScript code:
> 
> % gsc -target js -warnings -link -flat myprog.scm ; cat myprog_.js myprog.js > p.js
> % wc p.js
>     690    1845   14774 p.js
> 
> Finally, with the declaration:
> 
> (declare (inlining-limit 0))
> 
> some code expansion due to inlining can be avoided, resulting in 14K of JavaScript code:
> 
> % gsc -target js -warnings -link -flat myprog.scm ; cat myprog_.js myprog.js > p.js
> % wc p.js
>     667    1768   14198 p.js
> 
> This code size will stay the same regardless of additional procedure definitions in mylib.scm .
> 
> So with the new optimization I think much more compact target code can be generated when using a custom library that is not separately compiled.
> 
> Marc
> 
>> On Jan 2, 2016, at 8:06 PM, Marc Feeley <feeley at iro.umontreal.ca> wrote:
>> 
>> asmjs is cool and Gambit has already been compiled to it thanks to emscripten (see “Gambit in the browser”: http://feeley.github.io/gambit-in-the-browser/).  The performance is not bad, perhaps 10x to 20x slower than when Gambit is compiled by gcc to native code.  The main problem is the size of the JS that is generated when compiling Gambit-C with emscripten to JS.  The gsi interpreter yields roughly 10 MB of asmjs code.
>> 
>> As far as code size is concerned, a better solution is to use Gambit's JS backend with the Gambit library.  The code generated is roughly half the size when minimized (and it could be even less if some thought went into how identifiers are encoded).
>> 
>> Even more savings can be had by doing away with the default Gambit library and writing a custom library specialized for the application.  Gambit’s library has lots of functionality that is not normally needed by typical applications.  For example, the predefined map procedure can handle an arbitrary number of list parameters.  If the application only uses the single list version, then this definition would be sufficient:
>> 
>> (define (map f lst)
>>  (if (pair? lst)
>>      (cons (f (car lst)) (map f (cdr lst)))
>>      ‘()))
>> 
>> That 4 line version is 1/20th the size of the map defined in the Gambit library (which handles multiple lists, has type checking, precise error messages, same list length checking and is safe for multithreading).
>> 
>> So perhaps what’s needed for Gambit to be more successful for web dev is the creation of such a “slim” library for Gambit to replace the default feature-full library.  Gambit’s “tree shaker” would also help to eliminate the unused procedures of the slim library (unfortunately this only works for “whole program” compilation, so separate compilation would only be used for the development phase).
>> 
>> Anyway, if this interests you please let me know.
>> 
>> Marc
>> 
>>> On Jan 1, 2016, at 11:44 AM, Blake McBride <blake at mcbride.name> wrote:
>>> 
>>> Just some opinions.
>>> 
>>> asmjs.org defines a portable subset that allows JS platforms to compile into very fast code.  Targeting that subset, and within their spec, is probably a good idea.
>>> 
>>> JS has, and is becoming increasingly, a very, very important platform.  With ajax and rest services, code increasingly independent from the back-end is being developed.  So, in a very important sense, JS has become its own platform, just like X86, and X86_64, along with Linux, Windows, Mac, etc.
>>> 
>>> Many apps consist of two major parts:  the back-end processing, and the front-end human interface.  While one can write the back-end processing in any of dozens of languages targeting X86 or a VM, and the OS, there is really only one target for the other half - the human interface - and that is JS.
>>> 
>>> While many languages are now targeting the JS platform (I am using the word "platform" to mean X86, X86_64, either, plus the OS) including, believe it or not, Smalltalk, there are few that can, IMO, be used in a real world application for several reasons.  But, these issues will likely be resolved soon.
>>> 
>>> With respect, Gambit, at least in the past, was in many ways another toy solution - one with a functioning "Hello world" application but missing important features that make it usable in a real-world situation.  I would love to be a part of that solution, but alas, life doesn't offer me that level of freedom.  I am only capable of using a system that is reportedly working, reporting bugs, making small adjustments, and providing feedback.  Regardless of my attitude, preferences, and intentions, I cannot provide more.
>>> 
>>> Look at node (JS for the back-end) and its vastly increasing popularity.  Since developers are being forced to use JS on the front-end, it's bleeding over to the back-end.  I think solutions that take JS seriously at this stage will prosper greatly.
>>> 
>>> Now, having said all that, I would love to see Gambit target JS as a high-priority, first-class target.  If that were the case, I would be happy to contribute what I stated above. 
>>> 
>>> Thanks!
>>> 
>>> Blake McBride
>>> 
>>> _______________________________________________
>>> Gambit-list mailing list
>>> Gambit-list at iro.umontreal.ca
>>> https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
>> 
> 
> _______________________________________________
> Gambit-list mailing list
> Gambit-list at iro.umontreal.ca
> https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list




More information about the Gambit-list mailing list