From lucier at purdue.edu Wed Apr 1 13:15:01 2020 From: lucier at purdue.edu (Bradley Lucier) Date: Wed, 1 Apr 2020 13:15:01 -0400 Subject: [gambit-list] newbie: bitwise and arithmetic runtime speed for u64 In-Reply-To: References: Message-ID: <0927f324-632e-9148-3e54-b3d77023e76b@purdue.edu> On 3/31/20 3:50 PM, Paolo Montrasi wrote: > I am asking here for an advice?in improving runtime speed crunching > bitwise operations on u64 integers since I have no experience using > scheme and Gambit. > Looking at the documentation I see that fixnum is providing some help > but I apparently u64 integers are not "compatible" with fixnum operations > > > (fixnum? 18446744073709551615) > #f > > Can you share suggestions for my next exercise That may be possible using some of Gambit's low-level, internal, unsafe, very-bad-indeed routines for manipulating bignum "adigits" (for "addition digits", although they're the size used for many other things), which are 64 bits, unsigned, on 64-bit machines. (They're only 14 bits wide in the universal backend, though.) You'll find these routines in _num.scm: ;;; Sets x[i] to x[i] & y[j] (accessing x and y as adigits) (define-prim (##bignum.adigit-bitwise-and! x i y j)) ;;; Sets x[i] to ~x[i] & y[j] (accessing x and y as adigits) (define-prim (##bignum.adigit-bitwise-andc1! x i y j)) ;;; Sets x[i] to x[i] & ~y[j] (accessing x and y as adigits) (define-prim (##bignum.adigit-bitwise-andc2! x i y j)) ;;; Sets x[i] to ~(x[i] ^ y[j]) (accessing x and y as adigits) (define-prim (##bignum.adigit-bitwise-eqv! x i y j)) ;;; Sets x[i] to x[i] | y[j] (accessing x and y as adigits) (define-prim (##bignum.adigit-bitwise-ior! x i y j)) ;;; Sets x[i] to ~(x[i] & y[j]) (accessing x and y as adigits) (define-prim (##bignum.adigit-bitwise-nand! x i y j)) ;;; Sets x[i] to ~(x[i] | y[j]) (accessing x and y as adigits) (define-prim (##bignum.adigit-bitwise-nor! x i y j)) ;;; Sets x[i] to !x[i] (accessing x as adigits) (define-prim (##bignum.adigit-bitwise-not! x i)) ;;; Sets x[i] to ~x[i] | y[j] (accessing x and y as adigits) (define-prim (##bignum.adigit-bitwise-orc1! x i y j)) ;;; Sets x[i] to x[i] | ~y[j] (accessing x and y as adigits) (define-prim (##bignum.adigit-bitwise-orc2! x i y j)) ;;; Sets x[i] to x[i] ^ y[j] (accessing x and y as adigits) (define-prim (##bignum.adigit-bitwise-xor! x i y j)) There's also (define-prim (##bignum.make k x complement?) which makes a new bignum with k adigits, copying x if it's given, and complementing things if the last argument is #t. It may be easier to just call (define-prim (##bignum.copy x) (##bignum.make (##bignum.adigit-length x) x #f)) Using one of (define ##bignum.adigit-ones (##fixnum->bignum -1)) ;; the 0th adigit is all ones (define ##bignum.adigit-zeros (##fixnum->bignum 0)) ;; the 0th adigit is all zeros as a starter. Now, ##bignum.adigit-ones and ##bignum.adigit-zeros are unnormalized bignums, as could be many other results one can get by using these routines that manipulate adigits, so I wouldn't try to print them using the usual library routines, but this could get you started. Brad From feeley at iro.umontreal.ca Thu Apr 2 00:21:49 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Thu, 2 Apr 2020 00:21:49 -0400 Subject: [gambit-list] Universal backend progress Message-ID: <93BD37E4-E542-4181-9CD4-D8D01156D79C@iro.umontreal.ca> The universal backend has reached a new milestone. There is now a --enable-targets=? configure option to enable individual target languages in addition to C. Currently JavaScript and Python are supported. The universal backend is sufficiently feature-full to compile the gsi interpreter and builtin modules and support the module system and dynamically loading compiled modules: % ./configure --enable-targets=js,python --enable-single-host % make % make modules % gsc/gsc -:= -target js -exe -o gsi/gsi-js -prelude '(include"~~lib/header.scm")' gsi/_gsi.scm % gsc/gsc -:= -target python -exe -o gsi/gsi-python -prelude '(include"~~lib/header.scm")' gsi/_gsi.scm % gsi/gsi lib/ _test/test *** all tests passed out of a total of 40 tests % gsi/gsi-js lib/ _test/test *** all tests passed out of a total of 40 tests % gsi/gsi-python lib/ _test/test *** all tests passed out of a total of 40 tests % gsc/gsc -:= -target js fib.scm % gsi/gsi-js fib.o1 (time (fib 30)) 0.078000 secs real time 0.078000 secs cpu time (0.078000 user, 0.000000 system) no collections no bytes allocated no minor faults no major faults 832040 % gsi/gsi-js fib.scm (time (fib 30)) 1.877000 secs real time 1.877000 secs cpu time (1.877000 user, 0.000000 system) no collections no bytes allocated no minor faults no major faults 832040 % gsi/gsi fib.scm (time (fib 30)) 0.243592 secs real time 0.243565 secs cpu time (0.241976 user, 0.001589 system) 20 collections accounting for 0.001477 secs real time (0.001463 user, 0.000023 system) 172322368 bytes allocated 2028 minor faults no major faults 832040 So the JavaScript backend generates code that is about 8x slower than the code generated by the C backend. Marc From santana at mailbox.org Sat Apr 4 22:49:21 2020 From: santana at mailbox.org (santana at mailbox.org) Date: Sat, 4 Apr 2020 19:49:21 -0700 (PDT) Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected Message-ID: <1593972579.28769.1586054961632@office.mailbox.org> Was fiddling with gambit's c-ffi and noticed that wills I attached to ports made with ##open-predefined weren't executing so I wrote some test code, if there's something wrong with my code please let me know. # cat fdes.scm (c-declare #< #include #include c-declare-end ) (define-macro (c-macro var) `(define ,var ((c-lambda () int ,(string-append "___return(" (symbol->string var) ");"))))) ;; posix open flags (c-macro O_RDONLY) (c-macro O_WRONLY) (c-macro O_RDWR) (c-macro O_CREAT) ;; filename flags mode -> fdes | -1 (define posix-open (c-lambda (nonnull-char-string int int) int "open")) ;; fdes -> 0 | -1 (define posix-close (c-lambda (int) int "close")) (define in 1) (define out 2) (define inout 3) ;;; ;;; The will only executes if direction is in (define (test-posix-open filename direction) (let* ((fd (case direction ((1) (posix-open filename (bitwise-ior O_RDONLY O_CREAT) #o755)) ((2) (posix-open filename (bitwise-ior O_WRONLY O_CREAT) #o755)) ((3) (posix-open filename (bitwise-ior O_RDWR O_CREAT) #o755)))) (port (##open-predefined direction filename fd))) (println "fd: " fd " port: " port) (make-will port (lambda (p) (println "[WILL] closing port " p) (posix-close fd)))) (##gc)) ;;; when run with direction other than in this eats up all my ram (define (test-loop filename direction) (let ((fd (case direction ((1) (posix-open filename (bitwise-ior O_RDONLY O_CREAT) #o755)) ((2) (posix-open filename (bitwise-ior O_WRONLY O_CREAT) #o755)) ((3) (posix-open filename (bitwise-ior O_RDWR O_CREAT) #o755))))) (let loop () (##open-predefined direction filename fd) (loop)))) From feeley at iro.umontreal.ca Sun Apr 5 00:38:56 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Sun, 5 Apr 2020 00:38:56 -0400 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: <1593972579.28769.1586054961632@office.mailbox.org> References: <1593972579.28769.1586054961632@office.mailbox.org> Message-ID: <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> The ##open-predefined procedure is meant for internal use by the runtime system. It is intended to open one of the standard input/output file descriptors (i.e. with the ?index? parameter is equal to -1, -2, -3, or -4). For output file descriptors it registers an ?exit job? that forces output on the file descriptor when the program terminates. So the output ports created by ##open-predefined will always be reachable until the end of the program?s execution, and the will never detects that the port is unreachable, so it is never executed. Note that the fact that ##open-predefined allows passing an ?index? that is really a file descriptor is not something that you can rely upon, and in particular it will not work on Windows and when Gambit is configured with --enable-ansi-c. Perhaps the ##open-predefined procedure could be augmented with an optional parameter to enable/disable the automatic forcing of output ports (a will can?t be used for this functionality because the termination of the program does not cause a ?final gc? to determine which wills need to trigger). BTW, why do you need this? Marc > On Apr 4, 2020, at 10:49 PM, santana at mailbox.org wrote: > > Was fiddling with gambit's c-ffi and noticed that wills I attached to ports made with ##open-predefined weren't executing so I wrote some test code, if there's something wrong with my code please let me know. > > # cat fdes.scm > > (c-declare #< > #include > #include > #include > > c-declare-end > ) > > (define-macro (c-macro var) > `(define ,var > ((c-lambda () int ,(string-append "___return(" (symbol->string var) ");"))))) > > ;; posix open flags > (c-macro O_RDONLY) > (c-macro O_WRONLY) > (c-macro O_RDWR) > (c-macro O_CREAT) > > ;; filename flags mode -> fdes | -1 > (define posix-open (c-lambda (nonnull-char-string int int) int "open")) > ;; fdes -> 0 | -1 > (define posix-close (c-lambda (int) int "close")) > > (define in 1) > (define out 2) > (define inout 3) > > ;;; > ;;; The will only executes if direction is in > > (define (test-posix-open filename direction) > (let* ((fd (case direction > ((1) (posix-open filename (bitwise-ior O_RDONLY O_CREAT) #o755)) > ((2) (posix-open filename (bitwise-ior O_WRONLY O_CREAT) #o755)) > ((3) (posix-open filename (bitwise-ior O_RDWR O_CREAT) #o755)))) > (port (##open-predefined direction filename fd))) > (println "fd: " fd " port: " port) > (make-will port (lambda (p) (println "[WILL] closing port " p) (posix-close fd)))) > (##gc)) > > ;;; when run with direction other than in this eats up all my ram > > (define (test-loop filename direction) > (let ((fd (case direction > ((1) (posix-open filename (bitwise-ior O_RDONLY O_CREAT) #o755)) > ((2) (posix-open filename (bitwise-ior O_WRONLY O_CREAT) #o755)) > ((3) (posix-open filename (bitwise-ior O_RDWR O_CREAT) #o755))))) > (let loop () > (##open-predefined direction filename fd) > (loop)))) > > _______________________________________________ > Gambit-list mailing list > Gambit-list at iro.umontreal.ca > https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list > From santana at mailbox.org Sun Apr 5 01:59:42 2020 From: santana at mailbox.org (Alejandro Santana) Date: Sat, 4 Apr 2020 22:59:42 -0700 (PDT) Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> Message-ID: <650610244.28852.1586066382725@office.mailbox.org> Thanks for the explanation. > BTW, why do you need this? Just learning scheme and c. I wanted to have gambit receive s-exps over a unix domain socket. I figured it would be simplest if I could turn the file descriptor returned from accept into a port and call scheme's read on it. I can do without this though and I'd rather not have ports piling up. I'm curious though if there's a better way to turn a file descriptor into a port. ##open-predefined is what I saw mentioned when I searched the mailing list. From feeley at iro.umontreal.ca Sun Apr 5 07:47:00 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Sun, 5 Apr 2020 07:47:00 -0400 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: <650610244.28852.1586066382725@office.mailbox.org> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> Message-ID: <311F9F95-ACD8-4CBD-935D-E3880A7A4419@iro.umontreal.ca> > On Apr 5, 2020, at 1:59 AM, Alejandro Santana wrote: > > Thanks for the explanation. > >> BTW, why do you need this? > > Just learning scheme and c. I wanted to have gambit receive s-exps over a unix domain socket. I figured > it would be simplest if I could turn the file descriptor returned from accept into a port and call scheme's > read on it. I can do without this though and I'd rather not have ports piling up. > No need to muck around with file descriptors to do this. The following program is better as it is portable to linux, macOS and Windows: (define (start-server local-address handler) (let ((listen-port (open-tcp-server local-address))) (let loop () (handler (read listen-port)) (loop)))) (start-server "*:12345" (lambda (conn) (pretty-print (list 'msg= (read conn))) (close-port conn))) In another terminal you can send a message to the server with: % echo "(hello world)" | nc localhost 12345 A server for handling incoming connections can be created even more simply with tcp-service-register! : (thread-join! (tcp-service-register! "*:12345" (lambda () (pp (list 'msg= (read)))))) > I'm curious though if there's a better way to turn a file descriptor into a port. ##open-predefined is what > I saw mentioned when I searched the mailing list. I have added a commit (4a9c3f8dbbbb13d587e6ffec51c5bc87bc9dc666) that fixes this issue. Now the automatic forcing on program termination will no longer happen on file descriptors passed to ##open-predefined. Your fdes.scm program no longer ?leaks? memory. Marc From lassi at lassi.io Mon Apr 6 09:53:33 2020 From: lassi at lassi.io (Lassi Kortela) Date: Mon, 6 Apr 2020 16:53:33 +0300 Subject: [gambit-list] Steps to bootstrap Git master from Gambit 4.9.3 In-Reply-To: References: <1459896f-1a98-5e23-fff4-b0ed859b478b@lassi.io> Message-ID: > I?m not sure I understand the problem you are having. I just tried to build from a fresh clone (on linux) and everything built with no issues and the ?make check? passed. No matter what kind of gsc-boot I tried, it didn't work. But finally managed to solve the problem: I had used a shallow clone (`git clone --depth 1`) which caused a mysterious build failure. An ordinary `git clone` with the full history works fine, even with no 4.9.3 gsc on the system. Anyway, there is now finally a ready-to-run Docker container of Gambit's latest commit! docker run -it schemers/gambit:head Should probably add Git to that container so that remote modules work (`gsi github.com/gambit/hello`). From feeley at iro.umontreal.ca Mon Apr 6 11:49:29 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Mon, 6 Apr 2020 11:49:29 -0400 Subject: [gambit-list] Steps to bootstrap Git master from Gambit 4.9.3 In-Reply-To: References: <1459896f-1a98-5e23-fff4-b0ed859b478b@lassi.io> Message-ID: Nice! I tried it out to check performance, and I?m quite surprized that the docker Gambit is over 3 times slower than the ?native? Gambit: % gsi -e "(define (f x) (if (< x 2) x (+ (f (- x 1)) (f (- x 2)))))(pp (time (f 30)))" (time (f 30)) 0.248934 secs real time 0.248852 secs cpu time (0.247554 user, 0.001298 system) 25 collections accounting for 0.001492 secs real time (0.001458 user, 0.000035 system) 172322368 bytes allocated 1580 minor faults no major faults 832040 % docker run -it schemers/gambit:head gsi -e "(define (f x) (if (< x 2) x (+ (f (- x 1)) (f (- x 2)))))(pp (time (f 30)))" (time (f 30)) 0.823593 secs real time 0.830000 secs cpu time (0.830000 user, 0.000000 system) 26 collections accounting for 0.003341 secs real time (0.010000 user, 0.000000 system) 172322368 bytes allocated 1589 minor faults no major faults 832040 % docker run -it schemers/gambit:head gsi -v v4.9.3 20200101213000 x86_64-pc-linux-gnu "./configure '--prefix=/usr/local' '--enable-single-host' '--enable-multiple-threaded-vms'" 3x is a ?big deal?! Do you have an explanation for this? Is this typical of docker apps? Marc > On Apr 6, 2020, at 9:53 AM, Lassi Kortela wrote: > >> I?m not sure I understand the problem you are having. I just tried to build from a fresh clone (on linux) and everything built with no issues and the ?make check? passed. > > No matter what kind of gsc-boot I tried, it didn't work. > > But finally managed to solve the problem: I had used a shallow clone (`git clone --depth 1`) which caused a mysterious build failure. An ordinary `git clone` with the full history works fine, even with no 4.9.3 gsc on the system. > > Anyway, there is now finally a ready-to-run Docker container of Gambit's latest commit! > > docker run -it schemers/gambit:head > > Should probably add Git to that container so that remote modules work (`gsi github.com/gambit/hello`). > > _______________________________________________ > Gambit-list mailing list > Gambit-list at iro.umontreal.ca > https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list From lassi at lassi.io Mon Apr 6 12:20:03 2020 From: lassi at lassi.io (Lassi Kortela) Date: Mon, 6 Apr 2020 19:20:03 +0300 Subject: [gambit-list] Steps to bootstrap Git master from Gambit 4.9.3 In-Reply-To: References: <1459896f-1a98-5e23-fff4-b0ed859b478b@lassi.io> Message-ID: > I tried it out to check performance, and I?m quite surprized that the docker Gambit is over 3 times slower than the ?native? Gambit: > 3x is a ?big deal?! Do you have an explanation for this? Is this typical of docker apps? Interesting. No obvious explanation. Docker processes should share the same Linux kernel as host processes, they just have a different view of the process tree and file system than host processes. On that account, speed ought to be roughly on par with host processes. The container is built with: ./configure --enable-single-host --enable-multiple-threaded-vms Are you running Docker on Linux or Mac? I think Mac docker internally runs a virtualized Linux instance in the background and runs containers on that. It's easy to imagine that would be somewhat slower than native MacOS programs. From ben at srctxt.com Mon Apr 6 13:37:50 2020 From: ben at srctxt.com (Ben) Date: Mon, 06 Apr 2020 19:37:50 +0200 Subject: [gambit-list] Universal backend progress Message-ID: <369ace94-935e-4ef4-9eed-d3961d55d2fd@www.fastmail.com> Hi > The universal backend is sufficiently feature-full to compile the gsi interpreter and builtin modules and support the module system and dynamically loading compiled modules: Wow thats great, I very much looked forward to this! The js target runs fine on my machine, the python target not: % gsi/gsi-python lib/ _test/test gsi/gsi-python lib/ _test/test Traceback (most recent call last): File "gsi/gsi-python", line 983046, in g_module_register([[g_make_interned_symbol("_gambit")],[],g_null_obj,1,g_bb1___gambit_23_,False]) File "gsi/gsi-python", line 744, in g_module_register g_trampoline(g_module_table[0][4]) File "gsi/gsi-python", line 549, in g_trampoline pc = pc() File "gsi/gsi-python", line 832658, in g_bb79__23__23_default_2d_path_2d_expand return g_bb80__23__23_default_2d_path_2d_expand() File "gsi/gsi-python", line 832628, in g_bb80__23__23_default_2d_path_2d_expand return g_stack[g_sp+1]() File "gsi/gsi-python", line 825016, in g_bb3__23__23_file_2d_info_2d_aux return g_bb6__23__23_file_2d_info_2d_aux() File "gsi/gsi-python", line 825041, in g_bb6__23__23_file_2d_info_2d_aux return g_bb1__23__23_os_2d_file_2d_info() File "gsi/gsi-python", line 49501, in g_bb1__23__23_os_2d_file_2d_info g_r1 = g_file_info(g_r1, g_scm2host(g_r2), g_scm2host(g_r3)) File "gsi/gsi-python", line 9445, in g_file_info fi.slots[ 3] = g_host2scm(st.st_ino) File "gsi/gsi-python", line 443, in g_host2scm raise Exception("host2scm error") Exception: host2scm error python -V Python 2.7.16 python3 -V Python 3.7.7 sw_vers ProductName: Mac OS X ProductVersion: 10.15.4 From lassi at lassi.io Mon Apr 6 15:13:59 2020 From: lassi at lassi.io (Lassi Kortela) Date: Mon, 6 Apr 2020 22:13:59 +0300 Subject: [gambit-list] Troubleshooting build problem with shallow git clone Message-ID: <8ba8e66e-06a3-f969-87d8-f49b276bdf52@lassi.io> Can anyone else replicate the build failure using a shallow clone? When I do: git clone https://github.com/gambit/gambit.git --depth 1 cd gambit ./configure --prefix=$HOME/tmp/gambit-shallow-clone make The build fails in about a minute with errors like the ones below. These are from clang; configuring with CC=gcc-9 to ensure GNU GCC is being used, gives the same errors with slightly different messages. I wonder what the key difference between full and shallow clones is. Could it be about git tags being absent from the shallow clone? _kernel.c:23183:52: error: too few arguments to function call, expected 2, have 1 ___CFUN_ASSIGN(___result,___os_network_info(___arg1)) ~~~~~~~~~~~~~~~~~~ ^ ../include/gambit.h:7894:34: note: expanded from macro '___CFUN_ASSIGN' #define ___CFUN_ASSIGN(r,val)r = val; ^~~ ./os_setup.h:221:1: note: '___os_network_info' declared here extern ___SCMOBJ ___os_network_info ^ _kernel.c:23660:59: error: too few arguments to function call, expected 3, have 2 ___CFUN_ASSIGN(___result,___os_rename_file(___arg1,___arg2)) ~~~~~~~~~~~~~~~~~ ^ ../include/gambit.h:7894:34: note: expanded from macro '___CFUN_ASSIGN' #define ___CFUN_ASSIGN(r,val)r = val; ^~~ ./os_files.h:321:1: note: '___os_rename_file' declared here extern ___SCMOBJ ___os_rename_file ^ 11 errors generated. From feeley at iro.umontreal.ca Mon Apr 6 22:10:00 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Mon, 6 Apr 2020 22:10:00 -0400 Subject: [gambit-list] Universal backend progress In-Reply-To: <369ace94-935e-4ef4-9eed-d3961d55d2fd@www.fastmail.com> References: <369ace94-935e-4ef4-9eed-d3961d55d2fd@www.fastmail.com> Message-ID: <62680AD8-2A54-446A-8185-3ECE752D7DA6@iro.umontreal.ca> > On Apr 6, 2020, at 1:37 PM, Ben wrote: > > Hi > >> The universal backend is sufficiently feature-full to compile the gsi interpreter and builtin modules and support the module system and dynamically loading compiled modules: > > Wow thats great, I very much looked forward to this! > > The js target runs fine on my machine, the python target not: > > % gsi/gsi-python lib/ _test/test > > gsi/gsi-python lib/ _test/test > Traceback (most recent call last): > File "gsi/gsi-python", line 983046, in > g_module_register([[g_make_interned_symbol("_gambit")],[],g_null_obj,1,g_bb1___gambit_23_,False]) > File "gsi/gsi-python", line 744, in g_module_register > g_trampoline(g_module_table[0][4]) > File "gsi/gsi-python", line 549, in g_trampoline > pc = pc() > File "gsi/gsi-python", line 832658, in g_bb79__23__23_default_2d_path_2d_expand > return g_bb80__23__23_default_2d_path_2d_expand() > File "gsi/gsi-python", line 832628, in g_bb80__23__23_default_2d_path_2d_expand > return g_stack[g_sp+1]() > File "gsi/gsi-python", line 825016, in g_bb3__23__23_file_2d_info_2d_aux > return g_bb6__23__23_file_2d_info_2d_aux() > File "gsi/gsi-python", line 825041, in g_bb6__23__23_file_2d_info_2d_aux > return g_bb1__23__23_os_2d_file_2d_info() > File "gsi/gsi-python", line 49501, in g_bb1__23__23_os_2d_file_2d_info > g_r1 = g_file_info(g_r1, g_scm2host(g_r2), g_scm2host(g_r3)) > File "gsi/gsi-python", line 9445, in g_file_info > fi.slots[ 3] = g_host2scm(st.st_ino) > File "gsi/gsi-python", line 443, in g_host2scm > raise Exception("host2scm error") > Exception: host2scm error > > python -V > Python 2.7.16 > > python3 -V > Python 3.7.7 > > sw_vers > ProductName: Mac OS X > ProductVersion: 10.15.4 > It works for me on macOS: % gsi/gsi-python lib/ _test/test *** all tests passed out of a total of 40 tests I suspect the problem is that the ?inode? number of some file on your filesystem is bigger than a fixnum so the host to scheme conversion of the number does not work (conversion to bignums is not currently implemented). That?s probably something easy to fix... Marc From Joerg.Wittenberger at softeyes.net Fri Apr 10 13:59:35 2020 From: Joerg.Wittenberger at softeyes.net (=?UTF-8?B?SsO2cmc=?= F. Wittenberger) Date: Fri, 10 Apr 2020 19:59:35 +0200 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: <650610244.28852.1586066382725@office.mailbox.org> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> Message-ID: <20200410195935.47718fb3@softeyes.net> Hi Alejandro & Marc, this is exactly the use case I had when I asked the list and likely Alejandro referred to Marcs answer to my question. So I'm afraid this is really something we need [Q]: How would I turn a file descriptor, as exported from some library and ready/intented to be used with poll(2)/select(2) into a port? Thanks sooo much. /J?rg Background: Since when I'm using this ##open-predefined and have on my list to ask why this seems not to integrate into gambit's threading as good as I had hoped for. Nevertheless Marcs suggestion to use open-tcp-server does not work for me. I really need a unix domain socket. Actually an abstract socket on Linux. No way around. :-/ But I'd like it to block the calling thread on input. Which did not work out for me. (There is lambdanative and Android in the mix too, but my current _guess_ is that neither is to blame here.) Which leaves me with the question above. On Sat, 4 Apr 2020 22:59:42 -0700 (PDT) Alejandro Santana wrote: > Thanks for the explanation. > > > BTW, why do you need this? > Just learning scheme and c. I wanted to have gambit receive s-exps > over a unix domain socket. I figured it would be simplest if I could > turn the file descriptor returned from accept into a port and call > scheme's read on it. I can do without this though and I'd rather not > have ports piling up. > > I'm curious though if there's a better way to turn a file descriptor > into a port. ##open-predefined is what I saw mentioned when I > searched the mailing list. From vyzo at hackzen.org Fri Apr 10 14:19:07 2020 From: vyzo at hackzen.org (Dimitris Vyzovitis) Date: Fri, 10 Apr 2020 21:19:07 +0300 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: <20200410195935.47718fb3@softeyes.net> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> Message-ID: You might be interested in the gerbil socket library a spin; there is support for UNIX domain sockets through raw devices -- vyzo On Fri, Apr 10, 2020 at 9:06 PM J?rg F. Wittenberger < Joerg.Wittenberger at softeyes.net> wrote: > Hi Alejandro & Marc, > > this is exactly the use case I had when I asked the list and likely > Alejandro referred to Marcs answer to my question. > > So I'm afraid this is really something we need [Q]: > > How would I turn a file descriptor, as exported from some library > and ready/intented to be used with poll(2)/select(2) into a port? > > Thanks sooo much. > > /J?rg > > Background: > > Since when I'm using this ##open-predefined and have on my list to ask > why this seems not to integrate into gambit's threading as good as I > had hoped for. > > Nevertheless Marcs suggestion to use open-tcp-server does not work for > me. I really need a unix domain socket. Actually an abstract socket > on Linux. No way around. :-/ > > But I'd like it to block the calling thread on input. Which did not > work out for me. (There is lambdanative and Android in the mix too, but > my current _guess_ is that neither is to blame here.) > > Which leaves me with the question above. > > On Sat, 4 Apr 2020 22:59:42 -0700 (PDT) > Alejandro Santana wrote: > > > Thanks for the explanation. > > > > > BTW, why do you need this? > > Just learning scheme and c. I wanted to have gambit receive s-exps > > over a unix domain socket. I figured it would be simplest if I could > > turn the file descriptor returned from accept into a port and call > > scheme's read on it. I can do without this though and I'd rather not > > have ports piling up. > > > > I'm curious though if there's a better way to turn a file descriptor > > into a port. ##open-predefined is what I saw mentioned when I > > searched the mailing list. > > _______________________________________________ > Gambit-list mailing list > Gambit-list at iro.umontreal.ca > https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Joerg.Wittenberger at softeyes.net Fri Apr 10 15:12:15 2020 From: Joerg.Wittenberger at softeyes.net (=?UTF-8?B?SsO2cmc=?= F. Wittenberger) Date: Fri, 10 Apr 2020 21:12:15 +0200 Subject: [gambit-list] How to properly allocate foreign object? In-Reply-To: <20200410195935.47718fb3@softeyes.net> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> Message-ID: <20200410211215.21bb0025@softeyes.net> Hi, I'm afraid I don't grock the documentation wrt. memory management of foreign bjects. > I really need a unix domain socket. Actually an abstract socket on > Linux. The example at at hand: a `struct sockaddr_storage`. That is, in general, any kind of C struct to allocate. Likely I'm doing it all wrong. My great excuse(TM): I learned gambit's FFI from `gamsock` while adding domain socket support. No, Mam, it's not been me. ;-) # Part I, the easy thing (c-define-type socket-address (pointer (struct "sockaddr_storage") socket-address)) (c-lambda (...) socket-address "... ___result_voidstar = malloc(sizeof(struct sockaddr_storage))); ...") That seems to work. But I'd bet this leaks memory as gamsock never frees this memory. Q: Does it leak? Or is there some gambit magic default? # Part II, the crazy thing If I got that right, than what I'd need to fix the leak was to attach a /will/ to the socket-address, which would free the storage and return ___NO_ERR. Q: Correct? If yes, is this a good idea? I'd assume that allocating short lived objects from the Scheme heap should be much less expesive. Isn't it? # Part III, how? 1. How would I allocate a short lived object on the scheme heap? 2. Still have it properly tagges as foreign object for type checking. (I.e., I know I could allocate a u8vector of the size I need. But this is an u8vector for Scheme, which I don't want.) Thanks /J?rg From feeley at iro.umontreal.ca Fri Apr 10 15:49:15 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Fri, 10 Apr 2020 15:49:15 -0400 Subject: [gambit-list] How to properly allocate foreign object? In-Reply-To: <20200410211215.21bb0025@softeyes.net> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> <20200410211215.21bb0025@softeyes.net> Message-ID: There are a few issues with your code. First of all, you should use ?___return(?);? to return a result (the assignment to ___result_voidstar is deprecated). When Scheme code receives a foreign pointer, there is no ?automatic freeing? of the memory done by Gambit. You need to program that explicitly, for example using a will. An alternative would be to create your own type with a release function, but that is probably overkill in this situation. Here?s how I would write the code: (c-declare "#include ") (c-declare "#include ") (c-define-type sockaddr_storage (struct "sockaddr_storage")) (c-define-type socket-address (pointer sockaddr_storage socket-address)) (define alloc-socket-address (c-lambda () socket-address "___return(malloc(sizeof(struct sockaddr_storage)));")) (define free-socket-address (c-lambda (socket-address) void "free(___arg1);")) (define (make-socket-address) (let ((sa (alloc-socket-address))) (if sa (make-will sa (lambda (sa) (pp (list 'freeing sa)) (free-socket-address sa)))) sa)) (println "-----------") (pp (make-socket-address)) (define foo (make-socket-address)) (pp foo) (println "-----------") (##gc) (println "-----------") (pp (make-socket-address)) (println "-----------") (##gc) (println "-----------") ;; prints: ;; ;; ----------- ;; # ;; # ;; ----------- ;; (freeing #) ;; ----------- ;; # ;; ----------- ;; (freeing #) ;; ----------- Marc > On Apr 10, 2020, at 3:12 PM, J?rg F. Wittenberger wrote: > > Hi, > > I'm afraid I don't grock the documentation wrt. memory management of > foreign bjects. > >> I really need a unix domain socket. Actually an abstract socket on >> Linux. > > The example at at hand: a `struct sockaddr_storage`. That is, in > general, any kind of C struct to allocate. > > Likely I'm doing it all wrong. My great excuse(TM): I learned gambit's > FFI from `gamsock` while adding domain socket support. No, Mam, it's > not been me. ;-) > > # Part I, the easy thing > > (c-define-type socket-address > (pointer (struct "sockaddr_storage") socket-address)) > > (c-lambda > (...) socket-address > "... > ___result_voidstar = malloc(sizeof(struct sockaddr_storage))); > ...") > > That seems to work. But I'd bet this leaks memory as gamsock never > frees this memory. > > Q: Does it leak? Or is there some gambit magic default? > > # Part II, the crazy thing > > If I got that right, than what I'd need to fix the leak was to attach a > /will/ to the socket-address, which would free the storage and return > ___NO_ERR. > > Q: Correct? > > If yes, is this a good idea? I'd assume that allocating short lived > objects from the Scheme heap should be much less expesive. Isn't it? > > > # Part III, how? > > 1. How would I allocate a short lived object on the scheme heap? > 2. Still have it properly tagges as foreign object for type checking. > (I.e., I know I could allocate a u8vector of the size I need. > But this is an u8vector for Scheme, which I don't want.) > > Thanks > > /J?rg > > > _______________________________________________ > Gambit-list mailing list > Gambit-list at iro.umontreal.ca > https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list From feeley at iro.umontreal.ca Fri Apr 10 16:08:36 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Fri, 10 Apr 2020 16:08:36 -0400 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> Message-ID: <4997D8D8-4D49-4244-A3F9-DF5084A67D6E@iro.umontreal.ca> > On Apr 10, 2020, at 2:19 PM, Dimitris Vyzovitis wrote: > > You might be interested in the gerbil socket library a spin; there is support for UNIX domain sockets through raw devices > > -- vyzo It would be nice if that socket library was a Gambit module so that other Gambit users can use it! :-) In fact it would be great to have a way for (most) Gerbil modules to be directly accessible from pure Gambit code with an (import (gerbil xxx)) or something like that. That would lessen the community split between Gerbil and pure Gambit users. Something to consider is to turn Gerbil into a Gambit module so it can be installed directly using Gambit?s package manager. This is already the model for Termite Scheme which was initially designed as a separate language but is now a Gambit module. I?ve also approached Guillaume Cartier to turn JazzScheme into a Gambit module. Note that I?m not talking about a builtin module, but rather a module hosted on its own git repository and installable with ?gsi -install github.com/user/my-great-lang?. If anyone is interested in contributing towards this goal, please let me know! Marc From feeley at iro.umontreal.ca Fri Apr 10 16:16:23 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Fri, 10 Apr 2020 16:16:23 -0400 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: <20200410195935.47718fb3@softeyes.net> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> Message-ID: <81BED373-C316-4F5A-AF52-FB7FE34542E8@iro.umontreal.ca> > On Apr 10, 2020, at 1:59 PM, J?rg F. Wittenberger wrote: > > Background: > > Since when I'm using this ##open-predefined and have on my list to ask > why this seems not to integrate into gambit's threading as good as I > had hoped for. > > Nevertheless Marcs suggestion to use open-tcp-server does not work for > me. I really need a unix domain socket. Actually an abstract socket > on Linux. No way around. :-/ > > But I'd like it to block the calling thread on input. Which did not > work out for me. (There is lambdanative and Android in the mix too, but > my current _guess_ is that neither is to blame here.) On POSIX systems, you can use ##open-predefined to create a port from a file descriptor. Then, when you want to wait for input to be available you can call ##wait-input-port with that port. This is the definition in lib/_io.scm: (define-prim (##wait-input-port port) ;; The thread will wait until there is data available to read on the ;; port's device or the port's timeout is reached. The value #f is ;; returned when the timeout is reached. The value #t is returned ;; when there is data available to read on the port's device or the ;; thread was interrupted (for example with thread-interrupt!). ?) This is fully integrated with the thread system, so it only blocks the calling thread. Note also that you can use input-port-timeout-set! if you want to block for a limited time. Marc From vyzo at hackzen.org Fri Apr 10 16:19:30 2020 From: vyzo at hackzen.org (Dimitris Vyzovitis) Date: Fri, 10 Apr 2020 23:19:30 +0300 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: <4997D8D8-4D49-4244-A3F9-DF5084A67D6E@iro.umontreal.ca> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> <4997D8D8-4D49-4244-A3F9-DF5084A67D6E@iro.umontreal.ca> Message-ID: I would wholeheartedly welcome such an initiative, and I would gladly help out anyone interested to take it on. I am currently swamped with work (and will be for the forseeable future), so I can't do it on my own. -- vyzo On Fri, Apr 10, 2020 at 11:08 PM Marc Feeley wrote: > > > On Apr 10, 2020, at 2:19 PM, Dimitris Vyzovitis > wrote: > > > > You might be interested in the gerbil socket library a spin; there is > support for UNIX domain sockets through raw devices > > > > -- vyzo > > It would be nice if that socket library was a Gambit module so that other > Gambit users can use it! :-) > > In fact it would be great to have a way for (most) Gerbil modules to be > directly accessible from pure Gambit code with an (import (gerbil xxx)) or > something like that. That would lessen the community split between Gerbil > and pure Gambit users. > > Something to consider is to turn Gerbil into a Gambit module so it can be > installed directly using Gambit?s package manager. This is already the > model for Termite Scheme which was initially designed as a separate > language but is now a Gambit module. I?ve also approached Guillaume > Cartier to turn JazzScheme into a Gambit module. Note that I?m not talking > about a builtin module, but rather a module hosted on its own git > repository and installable with ?gsi -install > github.com/user/my-great-lang?. > > If anyone is interested in contributing towards this goal, please let me > know! > > Marc > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vyzo at hackzen.org Fri Apr 10 16:21:45 2020 From: vyzo at hackzen.org (Dimitris Vyzovitis) Date: Fri, 10 Apr 2020 23:21:45 +0300 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> <4997D8D8-4D49-4244-A3F9-DF5084A67D6E@iro.umontreal.ca> Message-ID: As a first step it would be sufficient to find a way to automatically export the Gerbil standard library as a gambit module. This would bring all the libraries to the wider gambit ecosystem, sans the macrology (which is a little harder to do). -- vyzo On Fri, Apr 10, 2020 at 11:19 PM Dimitris Vyzovitis wrote: > I would wholeheartedly welcome such an initiative, and I would gladly help > out anyone interested to take it on. > I am currently swamped with work (and will be for the forseeable future), > so I can't do it on my own. > > -- vyzo > > On Fri, Apr 10, 2020 at 11:08 PM Marc Feeley > wrote: > >> >> > On Apr 10, 2020, at 2:19 PM, Dimitris Vyzovitis >> wrote: >> > >> > You might be interested in the gerbil socket library a spin; there is >> support for UNIX domain sockets through raw devices >> > >> > -- vyzo >> >> It would be nice if that socket library was a Gambit module so that other >> Gambit users can use it! :-) >> >> In fact it would be great to have a way for (most) Gerbil modules to be >> directly accessible from pure Gambit code with an (import (gerbil xxx)) or >> something like that. That would lessen the community split between Gerbil >> and pure Gambit users. >> >> Something to consider is to turn Gerbil into a Gambit module so it can be >> installed directly using Gambit?s package manager. This is already the >> model for Termite Scheme which was initially designed as a separate >> language but is now a Gambit module. I?ve also approached Guillaume >> Cartier to turn JazzScheme into a Gambit module. Note that I?m not talking >> about a builtin module, but rather a module hosted on its own git >> repository and installable with ?gsi -install >> github.com/user/my-great-lang?. >> >> If anyone is interested in contributing towards this goal, please let me >> know! >> >> Marc >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From feeley at iro.umontreal.ca Fri Apr 10 21:37:01 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Fri, 10 Apr 2020 21:37:01 -0400 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> <4997D8D8-4D49-4244-A3F9-DF5084A67D6E@iro.umontreal.ca> Message-ID: <3E8BCBBB-9D97-4184-83AD-F2F3B6725954@iro.umontreal.ca> Great! Let?s make it happen. Marc > On Apr 10, 2020, at 4:19 PM, Dimitris Vyzovitis wrote: > > I would wholeheartedly welcome such an initiative, and I would gladly help out anyone interested to take it on. > I am currently swamped with work (and will be for the forseeable future), so I can't do it on my own. > > -- vyzo > > On Fri, Apr 10, 2020 at 11:08 PM Marc Feeley wrote: > > > On Apr 10, 2020, at 2:19 PM, Dimitris Vyzovitis wrote: > > > > You might be interested in the gerbil socket library a spin; there is support for UNIX domain sockets through raw devices > > > > -- vyzo > > It would be nice if that socket library was a Gambit module so that other Gambit users can use it! :-) > > In fact it would be great to have a way for (most) Gerbil modules to be directly accessible from pure Gambit code with an (import (gerbil xxx)) or something like that. That would lessen the community split between Gerbil and pure Gambit users. > > Something to consider is to turn Gerbil into a Gambit module so it can be installed directly using Gambit?s package manager. This is already the model for Termite Scheme which was initially designed as a separate language but is now a Gambit module. I?ve also approached Guillaume Cartier to turn JazzScheme into a Gambit module. Note that I?m not talking about a builtin module, but rather a module hosted on its own git repository and installable with ?gsi -install github.com/user/my-great-lang?. > > If anyone is interested in contributing towards this goal, please let me know! > > Marc > > From Joerg.Wittenberger at softeyes.net Sat Apr 11 06:05:37 2020 From: Joerg.Wittenberger at softeyes.net (=?UTF-8?B?SsO2cmc=?= F. Wittenberger) Date: Sat, 11 Apr 2020 12:05:37 +0200 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: <81BED373-C316-4F5A-AF52-FB7FE34542E8@iro.umontreal.ca> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> <81BED373-C316-4F5A-AF52-FB7FE34542E8@iro.umontreal.ca> Message-ID: <20200411120537.5c7961b8@softeyes.net> On Fri, 10 Apr 2020 16:16:23 -0400 Marc Feeley wrote: > > On Apr 10, 2020, at 1:59 PM, J?rg F. Wittenberger > > wrote: > > > > Background: > > > > Since when I'm using this ##open-predefined and have on my list to > > ask why this seems not to integrate into gambit's threading as good > > as I had hoped for. >... > Then, when you want to wait for input to be available you can call > ##wait-input-port with that port. This is the definition in > lib/_io.scm: > > (define-prim (##wait-input-port > port) This is what I'm using so far. > This is fully integrated with the thread system, so it only blocks > the calling thread. Note also that you can use > input-port-timeout-set! if you want to block for a limited time. My observation (on Linux) was that ##wait-input-port did not block the current thread at all. Thus gambit was sitting in a tight loop until data arrives. J?rg From Joerg.Wittenberger at softeyes.net Sat Apr 11 06:16:01 2020 From: Joerg.Wittenberger at softeyes.net (=?UTF-8?B?SsO2cmc=?= F. Wittenberger) Date: Sat, 11 Apr 2020 12:16:01 +0200 Subject: [gambit-list] How to properly allocate foreign object? In-Reply-To: References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> <20200410211215.21bb0025@softeyes.net> Message-ID: <20200411121601.35797cde@softeyes.net> Thanks for your reply. On Fri, 10 Apr 2020 15:49:15 -0400 Marc Feeley wrote: > There are a few issues with your code. First of all, you should use > ?___return(?);? to return a result (the assignment to > ___result_voidstar is deprecated). OK. Is there some documentation wrt. these things? > An alternative would be to create your own type with a release > function, but that is probably overkill in this situation. Interesting: you consider a type with release function to be the overkill? I had the impression that the use of wills would be the overly complex way. Nevertheless: In your suggestion you still use malloc to allocate. I do see situations where I'd tend to do this too. Still I wonder if it was cheaper/faster to allocate on the Scheme heap. And if so, how? J?rg From feeley at iro.umontreal.ca Sat Apr 11 09:30:30 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Sat, 11 Apr 2020 09:30:30 -0400 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: <20200411120537.5c7961b8@softeyes.net> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> <81BED373-C316-4F5A-AF52-FB7FE34542E8@iro.umontreal.ca> <20200411120537.5c7961b8@softeyes.net> Message-ID: <6F66324D-F132-41E0-A2DA-7841B616F91C@iro.umontreal.ca> > On Apr 11, 2020, at 6:05 AM, J?rg F. Wittenberger wrote: > > On Fri, 10 Apr 2020 16:16:23 -0400 > Marc Feeley wrote: > >>> On Apr 10, 2020, at 1:59 PM, J?rg F. Wittenberger >>> wrote: >>> >>> Background: >>> >>> Since when I'm using this ##open-predefined and have on my list to >>> ask why this seems not to integrate into gambit's threading as good >>> as I had hoped for. >> ... >> Then, when you want to wait for input to be available you can call >> ##wait-input-port with that port. This is the definition in >> lib/_io.scm: >> >> (define-prim (##wait-input-port >> port) > > This is what I'm using so far. > >> This is fully integrated with the thread system, so it only blocks >> the calling thread. Note also that you can use >> input-port-timeout-set! if you want to block for a limited time. > > My observation (on Linux) was that ##wait-input-port did not block the > current thread at all. Thus gambit was sitting in a tight loop until > data arrives. > > J?rg > It should work. Can you show me a minimal version of your code? Marc From feeley at iro.umontreal.ca Sat Apr 11 09:53:58 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Sat, 11 Apr 2020 09:53:58 -0400 Subject: [gambit-list] How to properly allocate foreign object? In-Reply-To: <20200411121601.35797cde@softeyes.net> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> <20200410211215.21bb0025@softeyes.net> <20200411121601.35797cde@softeyes.net> Message-ID: <84E7188F-77DF-4DFD-A900-FEB8401A4595@iro.umontreal.ca> > On Apr 11, 2020, at 6:16 AM, J?rg F. Wittenberger wrote: > > Thanks for your reply. > > On Fri, 10 Apr 2020 15:49:15 -0400 > Marc Feeley wrote: > >> There are a few issues with your code. First of all, you should use >> ?___return(?);? to return a result (the assignment to >> ___result_voidstar is deprecated). > > OK. Is there some documentation wrt. these things? The Gambit manual (doc/gambit.pdf) has this to say: When c-name-or-code is not a valid C identifier, it is treated as an arbitrary piece of C code. Within the C code the variables ?___arg1?, ?___arg2?, etc. can be referenced to access the converted arguments. Note that the C return statement can?t be used to return from the procedure. Instead, the ___return macro must be used. A procedure whose result-type is not void must pass the procedure?s result as the single argument to the ___return macro, for example ?___return(123);? to return the value 123. When result-type is void, the ___return macro must be called without a parameter list, for example ?___return;?. > >> An alternative would be to create your own type with a release >> function, but that is probably overkill in this situation. > > Interesting: you consider a type with release function to be the > overkill? I had the impression that the use of wills would be the > overly complex way. > > Nevertheless: In your suggestion you still use malloc to allocate. I > do see situations where I'd tend to do this too. Still I wonder if it > was cheaper/faster to allocate on the Scheme heap. And if so, how? In your C code you could allocate a ?still? u8vector object using: ___SCMOBJ obj = ___EXT(___alloc_scmobj) (___PSTATE, ___sU8VECTOR, length_in_bytes); /* * '___alloc_scmobj (___ps, subtype, bytes)' allocates a permanent or * still Scheme object (depending on '___ps') of subtype ?subtype' * with a body containing 'bytes' bytes, and returns it as an encoded * Scheme object. When '___ps' is NULL, a permanent object is * allocated, and when '___ps' is not NULL, a still object is * allocated in the heap of that processor's VM. The initialization * of the object's body must be done by the caller. In the case of * still objects this initialization must be done before the next * allocation is requested. The 'refcount' field of still objects is * initially 1. A fixnum error code is returned when there is an * error. */ and then decrement the reference count so that in the future the Gambit garbage collector will be solely responsible for determining if the object can be reclaimed: ___EXT(___release_scmobj) (obj); To access the content of the object you can use the ___BODY(obj) macro to get a pointer to the content of the object: char *body = ___CAST(char*,___BODY(obj)); body[0] = 11; body[1] = 22; I don?t know if there is a significant performance difference between this and the other method I explained (but it does a single call to malloc rather than two). You?ll have to benchmark it to see which one is faster. Marc From Joerg.Wittenberger at softeyes.net Sat Apr 11 13:01:10 2020 From: Joerg.Wittenberger at softeyes.net (=?UTF-8?B?SsO2cmc=?= F. Wittenberger) Date: Sat, 11 Apr 2020 19:01:10 +0200 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: <6F66324D-F132-41E0-A2DA-7841B616F91C@iro.umontreal.ca> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> <81BED373-C316-4F5A-AF52-FB7FE34542E8@iro.umontreal.ca> <20200411120537.5c7961b8@softeyes.net> <6F66324D-F132-41E0-A2DA-7841B616F91C@iro.umontreal.ca> Message-ID: <20200411190110.51543e5a@softeyes.net> On Sat, 11 Apr 2020 09:30:30 -0400 Marc Feeley wrote: > > My observation (on Linux) was that ##wait-input-port did not block > > the current thread at all. Thus gambit was sitting in a tight loop > > until data arrives. > ... > It should work. Can you show me a minimal version of your code? I shall destill. (Too much praise actually: gamsock is not really "my code".) Though while walking the dog it occured to me that the gamsock code reads the data directly from the file descriptor using recvfrom(2). It could have easily escaped me that the first attempt to ##wait-input-port did actually block. If there was the need to reset the port state (or go to the trouble to figure out how to change the heritage to read via gambit -- whatever is simpler) in order to inform the threading system that it should actually check the file descriptor again, then maybe we can shortcut the session. Best /J?rg From feeley at iro.umontreal.ca Sat Apr 11 13:08:23 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Sat, 11 Apr 2020 13:08:23 -0400 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: <20200411190110.51543e5a@softeyes.net> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> <81BED373-C316-4F5A-AF52-FB7FE34542E8@iro.umontreal.ca> <20200411120537.5c7961b8@softeyes.net> <6F66324D-F132-41E0-A2DA-7841B616F91C@iro.umontreal.ca> <20200411190110.51543e5a@softeyes.net> Message-ID: <36866D80-4664-4DD7-B40E-17C2795B8917@iro.umontreal.ca> > On Apr 11, 2020, at 1:01 PM, J?rg F. Wittenberger wrote: > > On Sat, 11 Apr 2020 09:30:30 -0400 > Marc Feeley wrote: > >>> My observation (on Linux) was that ##wait-input-port did not block >>> the current thread at all. Thus gambit was sitting in a tight loop >>> until data arrives. >> ... >> It should work. Can you show me a minimal version of your code? > > I shall destill. (Too much praise actually: gamsock is not really "my > code".) > > Though while walking the dog it occured to me that the gamsock code > reads the data directly from the file descriptor using recvfrom(2). > > It could have easily escaped me that the first attempt to > ##wait-input-port did actually block. If there was the need to reset > the port state (or go to the trouble to figure out how to change the > heritage to read via gambit -- whatever is simpler) in order to inform > the threading system that it should actually check the file descriptor > again, then maybe we can shortcut the session. > > Best > > /J?rg > Are you communicating with UDP? If that is the case then maybe you can use Gambit?s UDP ports directly: (open-udp port-number-or-address-or-settings) This procedure opens a socket for doing network communication with the UDP protocol. The default value of the direction: setting is input-output, i.e. the Scheme program can send information and receive information on the socket. The sending direction can be closed using the close-output-port procedure and the receiving direction can be closed using the close-input-port procedure. The close-port procedure closes both directions. The resulting port designates a UDP socket. Each call to read and udp-read-subu8vector causes the reception of a single datagram on the designated UDP socket, and each call to write and udp-write-subu8vector sends a single datagram. UDP ports are a direct subtype of object-ports (i.e. they are not character-ports) and read and write transfer u8vectors. If read is called and a timeout occurs before a datagram is transferred and the timeout thunk returns #f (see the procedure input-port-timeout-set!) then the end-of-file object is returned. From Joerg.Wittenberger at softeyes.net Sat Apr 11 13:56:16 2020 From: Joerg.Wittenberger at softeyes.net (=?UTF-8?B?SsO2cmc=?= F. Wittenberger) Date: Sat, 11 Apr 2020 19:56:16 +0200 Subject: [gambit-list] repl and macros Message-ID: <20200411195616.3e98a168@softeyes.net> Hi, yet another gambit newbee question. From lambdanative console template I learned how to start a `repl` in a statically linked binary. (N.B. "statically linked" MAY be boring; maybe not.) The pattern is: (define (repl-loop) (with-exception-catcher (lambda (exn) (DISPLAY-EXCEPTION)) ##repl-debug) (repl-loop)) (repl-loop) Use of `define-macro` as input yields "Ill-placed define-macro" exception. Thanks /J?rg From Joerg.Wittenberger at softeyes.net Sat Apr 11 14:09:12 2020 From: Joerg.Wittenberger at softeyes.net (=?UTF-8?B?SsO2cmc=?= F. Wittenberger) Date: Sat, 11 Apr 2020 20:09:12 +0200 Subject: [gambit-list] Is there a low level hack to define global macros? Message-ID: <20200411200912.4f915421@softeyes.net> Hi Marc, (guess this goes only for you ;-) While I wonder how I'd teach gambit to understand `define-values` in the repl (see other message)... There is this low level hack (helpful so far while too lazy to ask) to define new global variables at runtime: (define (doDEFINE! x v) (##make-global-var x) (##global-var-set! x v)) (doDEFINE! 'result 42)) result 42 How would I define a macro (not evaluation arguments) at runtime instead of a procedure? Possible? Thanks /J?rg From Joerg.Wittenberger at softeyes.net Sat Apr 11 14:16:55 2020 From: Joerg.Wittenberger at softeyes.net (=?UTF-8?B?SsO2cmc=?= F. Wittenberger) Date: Sat, 11 Apr 2020 20:16:55 +0200 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: <36866D80-4664-4DD7-B40E-17C2795B8917@iro.umontreal.ca> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> <81BED373-C316-4F5A-AF52-FB7FE34542E8@iro.umontreal.ca> <20200411120537.5c7961b8@softeyes.net> <6F66324D-F132-41E0-A2DA-7841B616F91C@iro.umontreal.ca> <20200411190110.51543e5a@softeyes.net> <36866D80-4664-4DD7-B40E-17C2795B8917@iro.umontreal.ca> Message-ID: <20200411201655.0b172f71@softeyes.net> On Sat, 11 Apr 2020 13:08:23 -0400 Marc Feeley wrote: > > On Apr 11, 2020, at 1:01 PM, J?rg F. Wittenberger > > wrote: >... > Are you communicating with UDP? If that is the case then maybe you > can use Gambit?s UDP ports directly: No, it's abstract unix domain sockets in the case at hand. But I'm more interested to learn how to get the low level things straight in principle. > (open-udp port-number-or-address-or-settings) At another currently questionable code spot (after all, may code fails currenlty with confused malloc in short order:-/ ) I actually tried to used gambit's UDP. But I failed to find out how to bind the outgoing port to the one I'm listening on for the reply. So I went back to use gamsock again. /J?rg > This procedure opens a socket for doing network communication with > the UDP protocol. The default value of the direction: setting is > input-output, i.e. the Scheme program can send information and > receive information on the socket. The sending direction can be > closed using the close-output-port procedure and the receiving > direction can be closed using the close-input-port procedure. The > close-port procedure closes both directions. > > The resulting port designates a UDP socket. Each call to read and > udp-read-subu8vector causes the reception of a single datagram on the > designated UDP socket, and each call to write and > udp-write-subu8vector sends a single datagram. UDP ports are a direct > subtype of object-ports (i.e. they are not character-ports) and read > and write transfer u8vectors. If read is called and a timeout occurs > before a datagram is transferred and the timeout thunk returns #f > (see the procedure input-port-timeout-set!) then the end-of-file > object is returned. Just re-read this. Still no clue. From feeley at iro.umontreal.ca Sat Apr 11 14:54:31 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Sat, 11 Apr 2020 14:54:31 -0400 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: <20200411201655.0b172f71@softeyes.net> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> <81BED373-C316-4F5A-AF52-FB7FE34542E8@iro.umontreal.ca> <20200411120537.5c7961b8@softeyes.net> <6F66324D-F132-41E0-A2DA-7841B616F91C@iro.umontreal.ca> <20200411190110.51543e5a@softeyes.net> <36866D80-4664-4DD7-B40E-17C2795B8917@iro.umontreal.ca> <20200411201655.0b172f71@softeyes.net> Message-ID: <30DA3D2B-D673-4614-B132-5F94EABF02DC@iro.umontreal.ca> > On Apr 11, 2020, at 2:16 PM, J?rg F. Wittenberger wrote: > > On Sat, 11 Apr 2020 13:08:23 -0400 > Marc Feeley wrote: > >>> On Apr 11, 2020, at 1:01 PM, J?rg F. Wittenberger >>> wrote: >> ... >> Are you communicating with UDP? If that is the case then maybe you >> can use Gambit?s UDP ports directly: > > No, it's abstract unix domain sockets in the case at hand. > > But I'm more interested to learn how to get the low level things > straight in principle. > >> (open-udp port-number-or-address-or-settings) > > At another currently questionable code spot (after all, may code fails > currenlty with confused malloc in short order:-/ ) I actually tried to > used gambit's UDP. > > But I failed to find out how to bind the outgoing port to the one I'm > listening on for the reply. So I went back to use gamsock again. > > /J?rg > >> This procedure opens a socket for doing network communication with >> the UDP protocol. The default value of the direction: setting is >> input-output, i.e. the Scheme program can send information and >> receive information on the socket. The sending direction can be >> closed using the close-output-port procedure and the receiving >> direction can be closed using the close-input-port procedure. The >> close-port procedure closes both directions. >> >> The resulting port designates a UDP socket. Each call to read and >> udp-read-subu8vector causes the reception of a single datagram on the >> designated UDP socket, and each call to write and >> udp-write-subu8vector sends a single datagram. UDP ports are a direct >> subtype of object-ports (i.e. they are not character-ports) and read >> and write transfer u8vectors. If read is called and a timeout occurs >> before a datagram is transferred and the timeout thunk returns #f >> (see the procedure input-port-timeout-set!) then the end-of-file >> object is returned. > > Just re-read this. Still no clue. Maybe you should read that section of the Gambit manual which has examples, such as: > (define p (open-udp (list local-address: "*" address: "time.nist.gov:37"))) > (write '#u8() p) > (read p) #u8(222 27 158 226) Marc From feeley at iro.umontreal.ca Sat Apr 11 15:00:49 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Sat, 11 Apr 2020 15:00:49 -0400 Subject: [gambit-list] output ports created with ##open-predefined not being garbage collected In-Reply-To: <20200411190110.51543e5a@softeyes.net> References: <1593972579.28769.1586054961632@office.mailbox.org> <208C6336-5F8E-4961-829B-3DE0331B0BB7@iro.umontreal.ca> <650610244.28852.1586066382725@office.mailbox.org> <20200410195935.47718fb3@softeyes.net> <81BED373-C316-4F5A-AF52-FB7FE34542E8@iro.umontreal.ca> <20200411120537.5c7961b8@softeyes.net> <6F66324D-F132-41E0-A2DA-7841B616F91C@iro.umontreal.ca> <20200411190110.51543e5a@softeyes.net> Message-ID: > On Apr 11, 2020, at 1:01 PM, J?rg F. Wittenberger wrote: > > It could have easily escaped me that the first attempt to > ##wait-input-port did actually block. If there was the need to reset > the port state (or go to the trouble to figure out how to change the > heritage to read via gambit -- whatever is simpler) in order to inform > the threading system that it should actually check the file descriptor > again, then maybe we can shortcut the session. > If ##wait-input-port returns, it means (it is likely) there is something to read from the file descriptor. You?ll have to read the file descriptor (through C or Scheme code) if you want ##wait-input-port to block again. Otherwise it will return immediately because there is still something available on the file descriptor. Marc From feeley at iro.umontreal.ca Sat Apr 11 15:07:58 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Sat, 11 Apr 2020 15:07:58 -0400 Subject: [gambit-list] repl and macros In-Reply-To: <20200411195616.3e98a168@softeyes.net> References: <20200411195616.3e98a168@softeyes.net> Message-ID: > On Apr 11, 2020, at 1:56 PM, J?rg F. Wittenberger wrote: > > Hi, > > yet another gambit newbee question. > > From lambdanative console template I learned how to start a `repl` in a > statically linked binary. (N.B. "statically linked" MAY be boring; > maybe not.) > > The pattern is: > > (define (repl-loop) > (with-exception-catcher > (lambda (exn) (DISPLAY-EXCEPTION)) > ##repl-debug) > (repl-loop)) > > (repl-loop) > > Use of `define-macro` as input yields "Ill-placed define-macro" > exception. > > Thanks > > /J?rg > Use: (##repl-debug #f #t) Marc From feeley at iro.umontreal.ca Sat Apr 11 15:25:26 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Sat, 11 Apr 2020 15:25:26 -0400 Subject: [gambit-list] Is there a low level hack to define global macros? In-Reply-To: <20200411200912.4f915421@softeyes.net> References: <20200411200912.4f915421@softeyes.net> Message-ID: <751FF8A1-2893-45F5-84B8-34B887ABFC5A@iro.umontreal.ca> > On Apr 11, 2020, at 2:09 PM, J?rg F. Wittenberger wrote: > > Hi Marc, > > (guess this goes only for you ;-) > > While I wonder how I'd teach gambit to understand `define-values` in > the repl (see other message)... > > There is this low level hack (helpful so far while too lazy to ask) to > define new global variables at runtime: > > (define (doDEFINE! x v) > (##make-global-var x) > (##global-var-set! x v)) > > (doDEFINE! 'result 42)) > > result > 42 > The code is not 100% correct. It is better to do: (define (doDEFINE! sym val) (##global-var-set! (##make-global-var sym) val)) > How would I define a macro (not evaluation arguments) at runtime instead > of a procedure? Possible? You can of course do: > (eval '(define-macro (mymac x) `(begin ,x ,x))) > (mymac (pp 'hello)) hello hello Otherwise, there?s a way with: (##top-cte-add-macro! ##interaction-cte 'mymac ...) but that uses internal procedures that are best left for internal use (they may soon change). Marc From iainduncanlists at gmail.com Sun Apr 12 12:07:18 2020 From: iainduncanlists at gmail.com (Iain Duncan) Date: Sun, 12 Apr 2020 09:07:18 -0700 Subject: [gambit-list] Can one use define-macro in Gerbil? Message-ID: Sorry if this is a dumb question. I know that Gambit supports CL style macros with define-macro, and see it's not mentioned on the Gerbil site, but I'm wondering if that's means it's not possible or just not officially supported. Reason being that I'm writing code using it in S7 scheme and in a perfect world, my project would have a second scheme implementation other than the default (S7), but for that to make sense I should probably be able to run define-macro code on it. thanks iain -------------- next part -------------- An HTML attachment was scrubbed... URL: From feeley at iro.umontreal.ca Sun Apr 12 12:26:11 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Sun, 12 Apr 2020 12:26:11 -0400 Subject: [gambit-list] Can one use define-macro in Gerbil? In-Reply-To: References: Message-ID: <5EBAE609-03B7-4AFC-A9B5-6CE70192BDC3@iro.umontreal.ca> > On Apr 12, 2020, at 12:07 PM, Iain Duncan wrote: > > Sorry if this is a dumb question. I know that Gambit supports CL style macros with define-macro, and see it's not mentioned on the Gerbil site, but I'm wondering if that's means it's not possible or just not officially supported. Reason being that I'm writing code using it in S7 scheme and in a perfect world, my project would have a second scheme implementation other than the default (S7), but for that to make sense I should probably be able to run define-macro code on it. > > thanks > iain Why do you need to support Gerbil as a ?second Scheme implementation?? You could use plain Gambit for that given that S7 is a rather minimal (not quite R7RS) Scheme implementation. Marc From iainduncanlists at gmail.com Sun Apr 12 12:36:16 2020 From: iainduncanlists at gmail.com (Iain Duncan) Date: Sun, 12 Apr 2020 09:36:16 -0700 Subject: [gambit-list] Can one use define-macro in Gerbil? In-Reply-To: <5EBAE609-03B7-4AFC-A9B5-6CE70192BDC3@iro.umontreal.ca> References: <5EBAE609-03B7-4AFC-A9B5-6CE70192BDC3@iro.umontreal.ca> Message-ID: Thanks Mark, That is also a possibility, but I just didn't have that question as I already know define-macro is supported in Gambit. These are very much hypothetical questions at this point, but I was curious about the macro issue. I don't really know (and won't for some time) what potential users might want to see that is not in S7, hence the looking around. thanks On Sun, Apr 12, 2020 at 9:26 AM Marc Feeley wrote: > > > On Apr 12, 2020, at 12:07 PM, Iain Duncan > wrote: > > > > Sorry if this is a dumb question. I know that Gambit supports CL style > macros with define-macro, and see it's not mentioned on the Gerbil site, > but I'm wondering if that's means it's not possible or just not officially > supported. Reason being that I'm writing code using it in S7 scheme and in > a perfect world, my project would have a second scheme implementation other > than the default (S7), but for that to make sense I should probably be able > to run define-macro code on it. > > > > thanks > > iain > > Why do you need to support Gerbil as a ?second Scheme implementation?? > You could use plain Gambit for that given that S7 is a rather minimal (not > quite R7RS) Scheme implementation. > > Marc > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lassi at lassi.io Wed Apr 15 04:52:34 2020 From: lassi at lassi.io (Lassi Kortela) Date: Wed, 15 Apr 2020 11:52:34 +0300 Subject: [gambit-list] Can one use define-macro in Gerbil? In-Reply-To: References: <5EBAE609-03B7-4AFC-A9B5-6CE70192BDC3@iro.umontreal.ca> Message-ID: <71a86532-ee29-3de1-3569-7b392c5c1876@lassi.io> Thanks for sticking around, Iain. It's not a dumb question. Scheme macros are unusually difficult to understand especially since the macro systems are many and varied. Even experts have trouble with the finer points now and then. Re: Gambit and Gerbil, Gambit follows Scheme standards (R5RS and R7RS) faithfully and adds only a minimal number of its own concepts to the language, focusing more on efficiency. The main ones are: - threads - versatile ports (not just byte and character ports) - weak references - keyword arguments and keyword objects - various introspection facilities These are useful augmentations but do not change the essential programming style or "flair" of classic Scheme. By contrast, Gerbil is much more adventurous with the core language, notably adding actors and methods which are a big conceptual leap. (As an amusing coincidence, Wikipedia tells us that Scheme was originally invented as a tool to study the Actor model way back in the 1970s, so I guess Gerbil brings it full circle :) As for define-macro, it seems to exist in Gerbil but doesn't work the same way it does in pure Gambit: Gerbil v0.15.1 on Gambit v4.9.3 > (define-macro (foo x y) `(+ ,x ,y)) > (foo 2 3) *** ERROR IN (console)@2.1 -- Ill-formed expression From vyzo at hackzen.org Wed Apr 15 04:59:34 2020 From: vyzo at hackzen.org (Dimitris Vyzovitis) Date: Wed, 15 Apr 2020 11:59:34 +0300 Subject: [gambit-list] Can one use define-macro in Gerbil? In-Reply-To: <71a86532-ee29-3de1-3569-7b392c5c1876@lassi.io> References: <5EBAE609-03B7-4AFC-A9B5-6CE70192BDC3@iro.umontreal.ca> <71a86532-ee29-3de1-3569-7b392c5c1876@lassi.io> Message-ID: define-macro is supported only within begin-foreign (or begin-ffi) blocks, where you drop to pure gambit. -- vyzo On Wed, Apr 15, 2020 at 11:52 AM Lassi Kortela wrote: > Thanks for sticking around, Iain. It's not a dumb question. Scheme > macros are unusually difficult to understand especially since the macro > systems are many and varied. Even experts have trouble with the finer > points now and then. > > Re: Gambit and Gerbil, Gambit follows Scheme standards (R5RS and R7RS) > faithfully and adds only a minimal number of its own concepts to the > language, focusing more on efficiency. The main ones are: > > - threads > - versatile ports (not just byte and character ports) > - weak references > - keyword arguments and keyword objects > - various introspection facilities > > These are useful augmentations but do not change the essential > programming style or "flair" of classic Scheme. > > By contrast, Gerbil is much more adventurous with the core language, > notably adding actors and methods which are a big conceptual leap. (As > an amusing coincidence, Wikipedia tells us that Scheme was originally > invented as a tool to study the Actor model way back in the 1970s, so I > guess Gerbil brings it full circle :) > > As for define-macro, it seems to exist in Gerbil but doesn't work the > same way it does in pure Gambit: > > Gerbil v0.15.1 on Gambit v4.9.3 > > (define-macro (foo x y) `(+ ,x ,y)) > > (foo 2 3) > *** ERROR IN (console)@2.1 -- Ill-formed expression > > _______________________________________________ > Gambit-list mailing list > Gambit-list at iro.umontreal.ca > https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lassi at lassi.io Wed Apr 15 05:01:46 2020 From: lassi at lassi.io (Lassi Kortela) Date: Wed, 15 Apr 2020 12:01:46 +0300 Subject: [gambit-list] Can one use define-macro in Gerbil? In-Reply-To: <71a86532-ee29-3de1-3569-7b392c5c1876@lassi.io> References: <5EBAE609-03B7-4AFC-A9B5-6CE70192BDC3@iro.umontreal.ca> <71a86532-ee29-3de1-3569-7b392c5c1876@lassi.io> Message-ID: Re: concurrency and actors, Gambit's extension Termite Scheme should be mentioned as well. From iainduncanlists at gmail.com Wed Apr 15 13:52:57 2020 From: iainduncanlists at gmail.com (Iain Duncan) Date: Wed, 15 Apr 2020 10:52:57 -0700 Subject: [gambit-list] Can one use define-macro in Gerbil? In-Reply-To: References: <5EBAE609-03B7-4AFC-A9B5-6CE70192BDC3@iro.umontreal.ca> <71a86532-ee29-3de1-3569-7b392c5c1876@lassi.io> Message-ID: Thanks everyone, answers much appreciated. iain On Wed, Apr 15, 2020 at 2:01 AM Lassi Kortela wrote: > Re: concurrency and actors, Gambit's extension Termite Scheme should be > mentioned as well. > < > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.125.1527&rep=rep1&type=pdf > > > > _______________________________________________ > Gambit-list mailing list > Gambit-list at iro.umontreal.ca > https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lassi at lassi.io Sun Apr 19 11:57:15 2020 From: lassi at lassi.io (Lassi Kortela) Date: Sun, 19 Apr 2020 18:57:15 +0300 Subject: [gambit-list] Steps to bootstrap Git master from Gambit 4.9.3 In-Reply-To: References: <1459896f-1a98-5e23-fff4-b0ed859b478b@lassi.io> Message-ID: It seems the current HEAD still can't be bootstrapped using 4.9.3 release. The Docker container (docker run -it schemers/gambit:head) is now built with the following commands to first try release gsc, and if that fails, bootstrap from scratch: ln -s /usr/local/bin/gsc gsc-boot make || (rm -rf boot/ gsc-boot && make) That approach would seem to work fine, and saves a lot of time in case the build works with the release compiler (Docker Hub builds are really slow - you get what you pay for...) From lassi at lassi.io Sun Apr 26 09:28:56 2020 From: lassi at lassi.io (Lassi Kortela) Date: Sun, 26 Apr 2020 16:28:56 +0300 Subject: [gambit-list] Faster docker builds of git master Message-ID: <24038421-6826-5be1-2028-3fd0eb4836c0@lassi.io> I've been building Gambit's git master for Docker hub a few times now: . The builds consistently take 60-70 minutes. I assume that means the full bootstrap is done each time. For comparison, 4.9.3 release is built from its source archive in about 20 minutes on the same infrastructure. The build steps are: WORKDIR /build RUN git clone https://github.com/gambit/gambit.git WORKDIR /build/gambit RUN ./configure --prefix=/usr/local \ --enable-single-host \ --enable-multiple-threaded-vms RUN ln -s /usr/local/bin/gsc gsc-boot RUN make || (rm -rf boot/ gsc-boot && make) RUN make modules RUN make check RUN make install When the build starts, /usr/local/bin/gsc is the one from Gambit 4.9.3. Are those build steps right, and is there something simple that could be done to reduce the bootstrap time? IIRC a Docker Hub build is limited to one CPU core, so `make -j` isn't used here. From pmontrasi at gmail.com Sun Apr 26 09:48:30 2020 From: pmontrasi at gmail.com (Paolo) Date: Sun, 26 Apr 2020 15:48:30 +0200 Subject: [gambit-list] newbie: bitwise and arithmetic runtime speed for u64 In-Reply-To: <0927f324-632e-9148-3e54-b3d77023e76b@purdue.edu> References: <0927f324-632e-9148-3e54-b3d77023e76b@purdue.edu> Message-ID: Hi Brad, thank you for your suggestion. I ended up in testing something similar to the following example (define (u64-xor . args-list) (##bignum.normalize! (fold (lambda (x big) (let ((x-big (if (fixnum? x) (##fixnum->bignum x) x))) (##bignum.adigit-bitwise-xor! big 0 x-big 0) big)) (##bignum.make 2 ##bignum.adigit-zeros #f) args-list))) it worked but with no noticeable improvements and this fact helped me in looking elsewhere to find speed problems ? I found a lot of them in my code of course ;-) Well I tried to do my best to fix most of the performance issues and I am now pretty happy with what I have come to, therefore I point you to my code in case you are looking for a fun "chess scheme" challenge. https://github.com/pmon/coronachess Thank you for your help, my best Paolo > Il giorno 01 apr 2020, alle ore 19:15, Bradley Lucier ha scritto: > > On 3/31/20 3:50 PM, Paolo Montrasi wrote: >> I am asking here for an advice in improving runtime speed crunching bitwise operations on u64 integers since I have no experience using scheme and Gambit. >> Looking at the documentation I see that fixnum is providing some help but I apparently u64 integers are not "compatible" with fixnum operations >> > (fixnum? 18446744073709551615) >> #f >> Can you share suggestions for my next exercise > That may be possible using some of Gambit's low-level, internal, unsafe, very-bad-indeed routines for manipulating bignum "adigits" (for "addition digits", although they're the size used for many other things), which are 64 bits, unsigned, on 64-bit machines. (They're only 14 bits wide in the universal backend, though.) > > You'll find these routines in _num.scm: > > ;;; Sets x[i] to x[i] & y[j] (accessing x and y as adigits) > (define-prim (##bignum.adigit-bitwise-and! x i y j)) > > ;;; Sets x[i] to ~x[i] & y[j] (accessing x and y as adigits) > (define-prim (##bignum.adigit-bitwise-andc1! x i y j)) > > ;;; Sets x[i] to x[i] & ~y[j] (accessing x and y as adigits) > (define-prim (##bignum.adigit-bitwise-andc2! x i y j)) > > ;;; Sets x[i] to ~(x[i] ^ y[j]) (accessing x and y as adigits) > (define-prim (##bignum.adigit-bitwise-eqv! x i y j)) > > ;;; Sets x[i] to x[i] | y[j] (accessing x and y as adigits) > (define-prim (##bignum.adigit-bitwise-ior! x i y j)) > > ;;; Sets x[i] to ~(x[i] & y[j]) (accessing x and y as adigits) > (define-prim (##bignum.adigit-bitwise-nand! x i y j)) > > ;;; Sets x[i] to ~(x[i] | y[j]) (accessing x and y as adigits) > (define-prim (##bignum.adigit-bitwise-nor! x i y j)) > > ;;; Sets x[i] to !x[i] (accessing x as adigits) > (define-prim (##bignum.adigit-bitwise-not! x i)) > > ;;; Sets x[i] to ~x[i] | y[j] (accessing x and y as adigits) > (define-prim (##bignum.adigit-bitwise-orc1! x i y j)) > > ;;; Sets x[i] to x[i] | ~y[j] (accessing x and y as adigits) > (define-prim (##bignum.adigit-bitwise-orc2! x i y j)) > > ;;; Sets x[i] to x[i] ^ y[j] (accessing x and y as adigits) > (define-prim (##bignum.adigit-bitwise-xor! x i y j)) > > There's also > > (define-prim (##bignum.make k x complement?) > > which makes a new bignum with k adigits, copying x if it's given, and complementing things if the last argument is #t. It may be easier to just call > > (define-prim (##bignum.copy x) > (##bignum.make (##bignum.adigit-length x) x #f)) > > Using one of > > (define ##bignum.adigit-ones (##fixnum->bignum -1)) ;; the 0th adigit is all ones > (define ##bignum.adigit-zeros (##fixnum->bignum 0)) ;; the 0th adigit is all zeros > > as a starter. > > Now, ##bignum.adigit-ones and ##bignum.adigit-zeros are unnormalized bignums, as could be many other results one can get by using these routines that manipulate adigits, so I wouldn't try to print them using the usual library routines, but this could get you started. > > Brad -------------- next part -------------- An HTML attachment was scrubbed... URL: From lucier at purdue.edu Sun Apr 26 17:31:41 2020 From: lucier at purdue.edu (Bradley Lucier) Date: Sun, 26 Apr 2020 17:31:41 -0400 Subject: [gambit-list] newbie: bitwise and arithmetic runtime speed for u64 In-Reply-To: References: <0927f324-632e-9148-3e54-b3d77023e76b@purdue.edu> Message-ID: <183748af-6e32-5081-5763-cbe11f751438@purdue.edu> On 4/26/20 9:48 AM, Paolo wrote: > Well I tried to do my best to fix most of the performance issues and I > am now pretty happy with what I have come to, therefore I point you to > my code in case you are looking for a fun "chess scheme" challenge. Cool! It looks like a nice project. I looked at your code a bit. I usually add the following declarations when working with code that generally does not work with flonums: (declare (standard-bindings) + (extended-bindings) + (mostly-fixnum) (not safe) (block)) And, it seems that if you rename the "bitwise" functions then the compiler doesn't inline checks for fixnum arguments and use the inlined fixnum operations. I filed an issue: https://github.com/gambit/gambit/issues/535 If you redefine u64-and, etc., as macros, like @@ -214,16 +216,24 @@ ; int "___return(__builtin_ffsl(___arg1) - 1);")) ; (define (u64-and . args-list) (fold u64c-and #xFFFFFFFFFFFFFFFF args-list)) -(define u64-and bitwise-and) +; (define u64-and bitwise-and) +(define-macro (u64-and . rest) + `(bitwise-and , at rest)) ; (define (u64-ior . args-list) (fold u64c-ior 0 args-list)) -(define u64-ior bitwise-ior) +; (define u64-ior bitwise-ior) +(define-macro (u64-ior . rest) + `(bitwise-ior , at rest)) ; (define (u64-xor . args-list) (fold u64c-xor 0 args-list)) -(define u64-xor bitwise-xor) +; (define u64-xor bitwise-xor) +(define-macro (u64-xor . rest) + `(bitwise-xor , at rest)) ; (define u64-not u64c-not) -(define u64-not bitwise-not) +; (define u64-not bitwise-not) +(define-macro (u64-not . rest) + `(bitwise-not , at rest)) ; (define u64-shift u64c-shift) (define (u64-shift n i) (u64-and #xFFFFFFFFFFFFFFFF (arithmetic-shift n i))) @@ -232,10 +242,14 @@ (define (u64-mult a b) (u64-and #xFFFFFFFFFFFFFFFF (* a b))) ; (define bitwise-bit-count c-bitcount) -(define bitwise-bit-count bit-count) +; (define bitwise-bit-count bit-count) +(define-macro (bitwise-bit-count . rest) + `(bit-count , at rest)) ; (define bitscan-fwd c-bitscan-fwd) -(define bitscan-fwd first-bit-set) +; (define bitscan-fwd first-bit-set) +(define-macro (bitscan-fwd . rest) + `(first-bit-set , at rest)) ; (define reset-ls1b c-reset-ls1b) (define (reset-ls1b bits) (u64-and bits (- bits 1))) then you get a small but noticeable speedup. Brad From feeley at iro.umontreal.ca Sun Apr 26 18:20:56 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Sun, 26 Apr 2020 18:20:56 -0400 Subject: [gambit-list] newbie: bitwise and arithmetic runtime speed for u64 In-Reply-To: References: <0927f324-632e-9148-3e54-b3d77023e76b@purdue.edu> Message-ID: Marc > On Apr 26, 2020, at 9:48 AM, Paolo wrote: > > Hi Brad, thank you for your suggestion. > I ended up in testing something similar to the following example > > (define (u64-xor . args-list) > (##bignum.normalize! > (fold > (lambda (x big) > (let ((x-big (if (fixnum? x) (##fixnum->bignum x) x))) > (##bignum.adigit-bitwise-xor! big 0 x-big 0) > big)) > (##bignum.make 2 ##bignum.adigit-zeros #f) > args-list))) > > it worked but with no noticeable improvements and this fact helped me in looking elsewhere to find speed problems ? I found a lot of them in my code of course ;-) > > Well I tried to do my best to fix most of the performance issues and I am now pretty happy with what I have come to, therefore I point you to my code in case you are looking for a fun "chess scheme" challenge. > > https://github.com/pmon/coronachess > > Thank you for your help, my best > Paolo Nice! I?ll have to try it out? As you noticed the name Gambit has its origins in chess? I used to play regularly. A gambit is a kind of scheme? and it is a calculated risk (which felt quite appropriate for my PhD work which was also risky). To do high speed calculations on raw 64 bit integers, I would tend to use u64vectors (or even u8vectors) to store the 64 bit integers and to drop down to C when some operation on these integers must be done without creating bignums. Something along these lines: (declare (standard-bindings) (extended-bindings) (not safe) ) (c-declare "#define ELEM0(u64vect) ___BODY_AS(u64vect,___tSUBTYPED)[0]") (define-macro (u64-xor! v1 v2) ;; v1[0] = v1[0] ^ v2[0] `(##c-code "ELEM0(___ARG1) ^= ELEM0(___ARG2);" v1 v2)) (define v1 (u64vector #x0123456789ABCDEF)) (define v2 (u64vector #x00FF00FF00FF00FF)) (println (number->string (u64vector-ref v1 0) 16)) (println (number->string (u64vector-ref v2 0) 16)) (u64-xor! v1 v2) (println (number->string (u64vector-ref v1 0) 16)) You could create a small library of such macros for the various 64 bit operations, and try to avoid as much as possible conversions to and from bignums which probably incurs a high overhead in your application. Marc From feeley at iro.umontreal.ca Sun Apr 26 22:23:03 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Sun, 26 Apr 2020 22:23:03 -0400 Subject: [gambit-list] newbie: bitwise and arithmetic runtime speed for u64 In-Reply-To: <4C79E815-24CE-44D4-BD5C-2B2D094E6C1C@purdue.edu> References: <0927f324-632e-9148-3e54-b3d77023e76b@purdue.edu> <4C79E815-24CE-44D4-BD5C-2B2D094E6C1C@purdue.edu> Message-ID: With a c-lambda the procedure itself and the conversion functions have to be called, and this adds considerable overhead on such a tiny piece of code. A ##c-code has no overhead. Marc > On Apr 26, 2020, at 7:14 PM, Lucier, Bradley J wrote: > >> On Apr 26, 2020, at 6:20 PM, Marc Feeley wrote: >> >> (c-declare "#define ELEM0(u64vect) ___BODY_AS(u64vect,___tSUBTYPED)[0]") >> >> (define-macro (u64-xor! v1 v2) ;; v1[0] = v1[0] ^ v2[0] >> `(##c-code "ELEM0(___ARG1) ^= ELEM0(___ARG2);" v1 v2)) > > He originally had similar code: > > ; (define u64c-and > ; (c-lambda (unsigned-int64 unsigned-int64) > ; unsigned-int64 "___return(___arg1 & ___arg2);?)) > > Would inlining the code as you did give better performance instead of using a c-lambda? > > Brad From feeley at iro.umontreal.ca Mon Apr 27 08:36:49 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Mon, 27 Apr 2020 08:36:49 -0400 Subject: [gambit-list] Module system ELS20 presentation Message-ID: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> Fr?d?ric Hamel just presented our work on Gambit's new module system at the European Lisp Symposium. You can watch the presentation here: https://www.youtube.com/watch?v=8hoJBNHHg5I (including the ?termite clock? demo at the end). The slides are here: https://www.european-lisp-symposium.org/static/2020/hamel-feeley-slides.pdf The paper is here: http://doi.org/10.5281/zenodo.3742443 Marc From cowan at ccil.org Mon Apr 27 09:26:04 2020 From: cowan at ccil.org (John Cowan) Date: Mon, 27 Apr 2020 09:26:04 -0400 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> Message-ID: Very interesting indeed! I will look at the paper. I'm particularly glad to see that the module system is R7RS-compatible. A Termite-specific question: why send bytecode rather than C (perhaps minified by removing whitespace)? Presumably that would cost nothing at the source node (it's already been done) and be faster and cheaper at the destination node? Is the bytecode so much more compact as to make transmitting C untenable? John Cowan http://vrici.lojban.org/~cowan cowan at ccil.org Heckler: "Go on, Al, tell 'em all you know. It won't take long." Al Smith: "I'll tell 'em all we *both* know. It won't take any longer." On Mon, Apr 27, 2020 at 8:37 AM Marc Feeley wrote: > Fr?d?ric Hamel just presented our work on Gambit's new module system at > the European Lisp Symposium. You can watch the presentation here: > https://www.youtube.com/watch?v=8hoJBNHHg5I (including the ?termite > clock? demo at the end). > > The slides are here: > https://www.european-lisp-symposium.org/static/2020/hamel-feeley-slides.pdf > > The paper is here: http://doi.org/10.5281/zenodo.3742443 > > Marc > > > > > _______________________________________________ > Gambit-list mailing list > Gambit-list at iro.umontreal.ca > https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From feeley at iro.umontreal.ca Mon Apr 27 10:24:03 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Mon, 27 Apr 2020 10:24:03 -0400 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> Message-ID: > On Apr 27, 2020, at 9:26 AM, John Cowan wrote: > > Very interesting indeed! I will look at the paper. > > I'm particularly glad to see that the module system is R7RS-compatible. > Well? one of the supported module systems is R7RS compatible. There?s also a ?primitive? module system that is much more Gambit specific, but has some features not found in the higher-level R7RS compatible module system. The intent is to promote the R7RS compatible module system for typical use and the primitive module system mostly for internal use. One of the extensions motivated by Termite was the use of library names that include the URL of the VCS repository and version, for example: (import (github.com/gambit hello demo @2.1)) It allows running that code with: gsi github.com/gambit/hello/demo at 2.1 Gambit also has ?builtin? modules that are named like this: (import (gambit list)) (import (_test)) It would be good if the Scheme community came to a set of rules for managing the ?namespace? of library names, beyond reserving (scheme ?) and (srfi ?) and ( ?). Specifically, I think a library name starting with a URL should be automatically reserved to the person or organisation that controls that URL. Also, a library name starting with an underscore should be reserved for the Scheme implementation, and have a meaning that depends on the Scheme implementation. > A Termite-specific question: why send bytecode rather than C (perhaps minified by removing whitespace)? Presumably that would cost nothing at the source node (it's already been done) and be faster and cheaper at the destination node? Is the bytecode so much more compact as to make transmitting C untenable? Termite does not send bytecode. If you are referring to the slides, we were just saying ?one way to implement portable code is with bytecode? but Termite does not do that. It can either send what is essentially an AST (in the ?interpreted? code case), or a symbolic reference to the compiled code control point (in the ?compiled? code case). A symbolic reference is basically the pair . Marc From cowan at ccil.org Mon Apr 27 11:38:10 2020 From: cowan at ccil.org (John Cowan) Date: Mon, 27 Apr 2020 11:38:10 -0400 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> Message-ID: [+srfi-discuss] On Mon, Apr 27, 2020 at 10:24 AM Marc Feeley wrote: > It would be good if the Scheme community came to a set of rules for > managing the ?namespace? of library names, beyond reserving (scheme ?) and > (srfi ?) and ( ?). Specifically, I think a library name > starting with a URL should be automatically reserved to the person or > organisation that controls that URL. An excellent idea for a no-code SRFI that would be easy to write for people who haven't written one before. It should specify both the petname convention and the URL convention. Attention should be drawn to mailto: URLs, which allow people who don't control any conventional URL to have their own library namespace. I wonder if it's better to recommend that all URLs be enclosed in vertical bars, which is safe and simple as vertical bars are not valid in URLs unless %-escaped, or to specify that only URLs containing characters from "#[]'(),;" be enclosed, which minimizes the use of vertical bars. I am juggling too many SRFIs as it is, so I can't do this myself. Also, a library name starting with an underscore should be reserved for the > Scheme implementation, and have a meaning that depends on the Scheme > implementation. > By reserved, you mean that it is not to be imported from user code? If so, I am fine with it. If not, my objection is that if you get some code that imports (chibi ...) and you try to run it on some other implementation, you will probably get an unknown module error. But if it imports (_foo ...) and your implementation is using that name you don't know what will happen. In any case, I think putting such modules under the implementation petname is probably good enough. John Cowan http://vrici.lojban.org/~cowan cowan at ccil.org The man that wanders far from the walking tree --first line of a non-existent poem by me -------------- next part -------------- An HTML attachment was scrubbed... URL: From lassi at lassi.io Mon Apr 27 12:03:18 2020 From: lassi at lassi.io (Lassi Kortela) Date: Mon, 27 Apr 2020 19:03:18 +0300 Subject: [gambit-list] Scheme library namespace and library name mapping In-Reply-To: References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> Message-ID: > I agree it is a good idea to have some management of namespaces. I am > not so convinced about the use of URLs for that purpose. (Apart from > aesthetics, will `https://www.nieper-wisskirchen.de' and > `http://www.nieper-wisskirchen.de' be different URLs for that > purpose?) > > If the library names become long (like with URIs), it would be nice > each Scheme system allows to use or install a mapper (a simple > association) from long names to short names so that I can access > libraries as `(nieper foo)' instead of > `(|https://www.nieper-wisskirchen.de| foo)'. If we go this route, we > can drop URLs altogether and replace them with UUIDs, which everyone > can generate (version 4 UUIDs). In fact, they are shorter than most > URLs and no less pretty in library names. A standard way to rewrite library names (in the same way that individual identifiers exported from libraries can be renamed in R7RS) would be useful. Re: URIs and UUIDs, we just worked out this problem a few weeks ago with John in a different context: . Our solution was to use either UUIDs, or URIs of the form "example.com/myname#fragment". I'd recommend that same form for Scheme libraries as well. Maybe drop the fragment part; we should think about it some more. Gambit is tentatively using "@v" as the last part of the library name for versioning, where "v" is a Git ref (tag or commit). >> I wonder if it's better to recommend that all URLs be enclosed in vertical bars, which is safe and simple as vertical bars are not valid in URLs unless %-escaped, or to specify that only URLs containing characters from "#[]'(),;" be enclosed, which minimizes the use of vertical bars. > > Many R7RS systems don't define the mapping from library names to file > names for library names that include characters like the slash or the > dot. There may be other limitations. It might make sense to have a convention to treat any library name with a dot as a URI by default. With the possibility for each user to manually add their own rewrite rules to override that default where needed. From bradley.j.lucier.1 at purdue.edu Sun Apr 26 19:14:09 2020 From: bradley.j.lucier.1 at purdue.edu (Lucier, Bradley J) Date: Sun, 26 Apr 2020 23:14:09 +0000 Subject: [gambit-list] newbie: bitwise and arithmetic runtime speed for u64 In-Reply-To: References: <0927f324-632e-9148-3e54-b3d77023e76b@purdue.edu> Message-ID: <4C79E815-24CE-44D4-BD5C-2B2D094E6C1C@purdue.edu> > On Apr 26, 2020, at 6:20 PM, Marc Feeley wrote: > > (c-declare "#define ELEM0(u64vect) ___BODY_AS(u64vect,___tSUBTYPED)[0]") > > (define-macro (u64-xor! v1 v2) ;; v1[0] = v1[0] ^ v2[0] > `(##c-code "ELEM0(___ARG1) ^= ELEM0(___ARG2);" v1 v2)) He originally had similar code: ; (define u64c-and ; (c-lambda (unsigned-int64 unsigned-int64) ; unsigned-int64 "___return(___arg1 & ___arg2);?)) Would inlining the code as you did give better performance instead of using a c-lambda? Brad From marc at nieper-wisskirchen.de Mon Apr 27 11:54:19 2020 From: marc at nieper-wisskirchen.de (=?UTF-8?Q?Marc_Nieper=2DWi=C3=9Fkirchen?=) Date: Mon, 27 Apr 2020 17:54:19 +0200 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> Message-ID: Am Mo., 27. Apr. 2020 um 17:38 Uhr schrieb John Cowan : [...] >> It would be good if the Scheme community came to a set of rules for managing the ?namespace? of library names, beyond reserving (scheme ?) and (srfi ?) and ( ?). Specifically, I think a library name starting with a URL should be automatically reserved to the person or organisation that controls that URL. I agree it is a good idea to have some management of namespaces. I am not so convinced about the use of URLs for that purpose. (Apart from aesthetics, will `https://www.nieper-wisskirchen.de' and `http://www.nieper-wisskirchen.de' be different URLs for that purpose?) If the library names become long (like with URIs), it would be nice each Scheme system allows to use or install a mapper (a simple association) from long names to short names so that I can access libraries as `(nieper foo)' instead of `(|https://www.nieper-wisskirchen.de| foo)'. If we go this route, we can drop URLs altogether and replace them with UUIDs, which everyone can generate (version 4 UUIDs). In fact, they are shorter than most URLs and no less pretty in library names. > I wonder if it's better to recommend that all URLs be enclosed in vertical bars, which is safe and simple as vertical bars are not valid in URLs unless %-escaped, or to specify that only URLs containing characters from "#[]'(),;" be enclosed, which minimizes the use of vertical bars. Many R7RS systems don't define the mapping from library names to file names for library names that include characters like the slash or the dot. There may be other limitations. >> Also, a library name starting with an underscore should be reserved for the Scheme implementation, and have a meaning that depends on the Scheme implementation. > > > By reserved, you mean that it is not to be imported from user code? If so, I am fine with it. If not, my objection is that if you get some code that imports (chibi ...) and you try to run it on some other implementation, you will probably get an unknown module error. But if it imports (_foo ...) and your implementation is using that name you don't know what will happen. Do we really need this? Instead of `(_foo)', where `_foo' is supposed to be internal to Chibi-Scheme, it is already enough to put the code in `(chibi foo)' to make it private. The further conventions about the library namings inside the implementation namespace should be up to the implementation. psyntax, for example, uses `$' to mark internal libraries. > In any case, I think putting such modules under the implementation petname is probably good enough. Indeed. Marc From lassi at lassi.io Mon Apr 27 12:15:07 2020 From: lassi at lassi.io (Lassi Kortela) Date: Mon, 27 Apr 2020 19:15:07 +0300 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> Message-ID: <43dbcdd2-013f-148f-6392-a09efce67f81@lassi.io> > [+srfi-discuss] Thanks for broadening the audience. Good call. > An excellent idea for a no-code SRFI that would be easy to write for > people who haven't written one before.? It should specify both the > petname?convention and the URL convention. +1 > Attention should be drawn to > mailto: URLs, which allow people who don't control any conventional URL > to have their own library namespace. I'd advocate focusing on web URLs and only supporting other types if that falls out naturally from the overall design. There are now tens of thousands of hosting sites where anyone can get their own web URLs for free, both conventional GeoCities-style webhosts as well as social sites like GitHub that give everyone a github.com/username. Of course, if a library name part can be a URN (Uniform Resource Name, https://en.wikipedia.org/wiki/Uniform_Resource_Name) that's one way to solve the problem. We can do the same thing we did for Unisig (https://github.com/unisig/unisig) where the default URI scheme is whatever protocol is the most popular way to transfer web pages at any given time (previously http, now https, in the future http/2 or something else). > I wonder if it's better to recommend that all URLs be enclosed in > vertical bars, which is safe and simple as vertical bars are not valid > in URLs unless %-escaped, or to specify that only URLs containing > characters from "#[]'(),;" be enclosed, which minimizes the use of > vertical bars. I'd recommend simply not using any fancy URLs. A simple "foo.com/bar/baz" with lowercase [a-z0-9], dash, slash, and dot ought to be enough for almost everything. I guess we can permit arbitrary URIs (or URNs), and people can them use vertical bars for portability. > Also, a library name starting with an underscore should be reserved > for the Scheme implementation, and have a meaning that depends on > the Scheme implementation. > > By reserved, you mean that it is not to be imported from user code? Gambit has library names beginning with an underscore already. Most of those can be imported from user code, but only if the user knows the code doesn't need to run in other Scheme implementations. > If so, I am fine with it.? If not, my objection is that if you get some > code that imports (chibi ...) and you try to run it on some other > implementation, you will probably get an unknown module error.? But if > it imports (_foo ...) and your implementation is using that name you > don't know what will happen. Good point. It might make sense to have the "_" alias in the interaction environment only, meaning that source files need to import using the name of the implementation. From lassi at lassi.io Mon Apr 27 12:27:20 2020 From: lassi at lassi.io (Lassi Kortela) Date: Mon, 27 Apr 2020 19:27:20 +0300 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: <20200427161438.GB15813@scully.more-magic.net> References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> <20200427161438.GB15813@scully.more-magic.net> Message-ID: <8d1b0481-49aa-571d-ce8a-4c9b864c55b6@lassi.io> > Assuming/hoping there will at some point be a registry where one can > download r7rs libraries (maybe there already is? snowfort?), I think it > can be as simple as suggesting people use their library name as it occurs > in the registry as a prefix. Yes, snow-fort.org already has a nice collection of R7RS libraries. It doesn't enforce any particular library name conventions. We should set up something like lib.scheme.org that aggregates everything. Scheme.org news coming soon. Nevertheless, experience with golang shows that arbitrary URI imports are still a good idea. There are so many Go packages that they cannot hope to gather all of them in one registry, no matter how nice that would be. > This is lightweight, easy to understand and remember, and not as ugly or > fraught with issues like the URI proposal. Do you have specific issues in mind with the URIs? >> I wonder if it's better to recommend that all URLs be enclosed in vertical >> bars, which is safe and simple as vertical bars are not valid in URLs >> unless %-escaped, or to specify that only URLs containing characters from >> "#[]'(),;" be enclosed, which minimizes the use of vertical bars. > > That would be a problem because then you'd need an additional mapping of > URL to filesystem location for the module. A colon is not allowed in > Windows file names, more than one dot might be a problem in some OSes as > well, and slashes can't occur in file names either. There's many more > specifal characters: > > https://en.wikipedia.org/wiki/Filename#Comparison_of_filename_limitations > > So an e-mail address might not work either. Good call. I have a generic solution in mind to the filename mapping mess that would involve having a special metadata file in the lib directory. That file could have instructions about library lookup and compatibility, which would then drive a small library name rewriting engine in the Scheme implementation. By rewriting, I mean things like "assume all library names of the form (srfi-NNN) are aliases to equivalent names of the form (srfi NNN), and those can be found at filenames of the form srfi/NNN.sld". From lassi at lassi.io Mon Apr 27 12:35:06 2020 From: lassi at lassi.io (Lassi Kortela) Date: Mon, 27 Apr 2020 19:35:06 +0300 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> <43dbcdd2-013f-148f-6392-a09efce67f81@lassi.io> Message-ID: > Do we want advertising for commercial services in library names in Scheme? I'd leave that up to users. Is there a particular way we can restrict it? If we have a lib.scheme.org it will hopefully be neat enough that people will voluntarily want their libraries to have scheme.org instead of github.com. More generally, any large-scale distributed namespace is going to end up with lots of messy names. I don't think there's a general way to avoid that problem (other than not being popular enough to have mutually unknown people coining names :) > To encode a protocol in library names is crazy if you ask me, > especially because the most popular protocol changes over time as you > write. Fully agreed. > But without the protocol, we don't have URLs anymore. From Python 3: >>> import urllib.parse >>> urllib.parse.urlparse("example.com/foo") ParseResult(scheme='', netloc='', path='example.com/foo', params='', query='', fragment='') So if you give it a URL without a scheme, it returns scheme='' which we can fill in with the default. I'm not sure what the RFCs say about such URIs, but that can be remedied with a specification like "parse as a URI; if invalid, try prepending https:// and parsing again". It shouldn't be hard to come up with a practical rule. > I guess that most R7RS systems will map the library > `(foo.com/bar/baz)' to the same file system location as `(foo.com bar > baz)'. To me, this shows some flaws in the approach. > > To use the example from above, GitHub may distribute a library > `(github.com hacker foo)' under this proposal. Mr. Hacker owning a > GitHub account wants to distribute `(github.com/hacker foo)'. This > won't work without a clash. Good point. I'd advocate for standardizing a simple rewriting and filename mapping engine to work around problems like this (which I believe will come up no matter what kind of practical naming scheme we choose). From feeley at iro.umontreal.ca Mon Apr 27 12:38:07 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Mon, 27 Apr 2020 12:38:07 -0400 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> Message-ID: <647DB28F-B5AC-4AA1-AF6B-671C5DA82834@iro.umontreal.ca> > On Apr 27, 2020, at 11:54 AM, Marc Nieper-Wi?kirchen wrote: > > Am Mo., 27. Apr. 2020 um 17:38 Uhr schrieb John Cowan : > > [...] > >>> It would be good if the Scheme community came to a set of rules for managing the ?namespace? of library names, beyond reserving (scheme ?) and (srfi ?) and ( ?). Specifically, I think a library name starting with a URL should be automatically reserved to the person or organisation that controls that URL. > > I agree it is a good idea to have some management of namespaces. I am > not so convinced about the use of URLs for that purpose. (Apart from > aesthetics, will `https://www.nieper-wisskirchen.de' and > `http://www.nieper-wisskirchen.de' be different URLs for that > purpose?) That?s to be debated, but frankly I think the protocol part ?http[s]://? should not be part of the library name in general. > > If the library names become long (like with URIs), it would be nice > each Scheme system allows to use or install a mapper (a simple > association) from long names to short names so that I can access > libraries as `(nieper foo)' instead of > `(|https://www.nieper-wisskirchen.de| foo)'. If we go this route, we > can drop URLs altogether and replace them with UUIDs, which everyone > can generate (version 4 UUIDs). In fact, they are shorter than most > URLs and no less pretty in library names. Adding vertical bars should only be needed if? it is needed. There?s no point in forcing it if the name is a correct symbol. > >> I wonder if it's better to recommend that all URLs be enclosed in vertical bars, which is safe and simple as vertical bars are not valid in URLs unless %-escaped, or to specify that only URLs containing characters from "#[]'(),;" be enclosed, which minimizes the use of vertical bars. > > Many R7RS systems don't define the mapping from library names to file > names for library names that include characters like the slash or the > dot. There may be other limitations. > >>> Also, a library name starting with an underscore should be reserved for the Scheme implementation, and have a meaning that depends on the Scheme implementation. >> >> >> By reserved, you mean that it is not to be imported from user code? If so, I am fine with it. If not, my objection is that if you get some code that imports (chibi ...) and you try to run it on some other implementation, you will probably get an unknown module error. But if it imports (_foo ...) and your implementation is using that name you don't know what will happen. > > Do we really need this? Instead of `(_foo)', where `_foo' is supposed > to be internal to Chibi-Scheme, it is already enough to put the code > in `(chibi foo)' to make it private. The further conventions about the > library namings inside the implementation namespace should be up to > the implementation. psyntax, for example, uses `$' to mark internal > libraries. It is something that became useful as a way to easily specify builtin names on the command line for Gambit, i.e. to do gsi _hamt/test rather than gsi gambit/hamt/test It doesn?t seem like much difference at first sight, but with this idea of permitting library names on the command line you can start using them as a kind of flag to enable certain modes of execution, like quiet testing: gsi _test/quiet _hamt/test rather than gsi gambit/test/quiet gambit/hamt/test The shorter ?_? prefix makes a difference and almost looks like the ?-? in front of normal options. Marc From feeley at iro.umontreal.ca Mon Apr 27 12:52:05 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Mon, 27 Apr 2020 12:52:05 -0400 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: <20200427161438.GB15813@scully.more-magic.net> References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> <20200427161438.GB15813@scully.more-magic.net> Message-ID: > On Apr 27, 2020, at 12:14 PM, Peter Bex wrote: > > On Mon, Apr 27, 2020 at 11:38:10AM -0400, John Cowan wrote: >> An excellent idea for a no-code SRFI that would be easy to write for people >> who haven't written one before. It should specify both the >> petname convention and the URL convention. Attention should be drawn to >> mailto: URLs, which allow people who don't control any conventional URL to >> have their own library namespace. > > Assuming/hoping there will at some point be a registry where one can > download r7rs libraries (maybe there already is? snowfort?), I think it > can be as simple as suggesting people use their library name as it occurs > in the registry as a prefix. A registry would be useful but it should not be the only way to name libraries. A registry forces authors to register their libraries for every new library they want others to use. But that can become a rather tedious process, especially if you are into rapid development and sharing a new idea ?right now?. I want to put the library up on my github account and share the link? a 60 second process at most. > > This is lightweight, easy to understand and remember, and not as ugly or > fraught with issues like the URI proposal. > > For programs, one could use the main binary's name as a prefix. If the > binary is going to be installed into PATH under the scheme system's bin > directory, that has to be unique already, anyway. And if it's installed > somewhere else, it shouldn't cause any problem as long as the search path > for binaries matches the search path for libraries (e.g.., first the > current working directory, then system paths). > >> I wonder if it's better to recommend that all URLs be enclosed in vertical >> bars, which is safe and simple as vertical bars are not valid in URLs >> unless %-escaped, or to specify that only URLs containing characters from >> "#[]'(),;" be enclosed, which minimizes the use of vertical bars. > > That would be a problem because then you'd need an additional mapping of > URL to filesystem location for the module. A colon is not allowed in > Windows file names, more than one dot might be a problem in some OSes as > well, and slashes can't occur in file names either. There's many more > specifal characters: > > https://en.wikipedia.org/wiki/Filename#Comparison_of_filename_limitations > > So an e-mail address might not work either. My main point is for library names that include a URI (at the head) to be under the control of the person or organisation that controls that URI (or mail address). This avoids the need for a registration step and is lightweight (no administrativia involved for the author). Marc From marc at nieper-wisskirchen.de Mon Apr 27 12:25:28 2020 From: marc at nieper-wisskirchen.de (=?UTF-8?Q?Marc_Nieper=2DWi=C3=9Fkirchen?=) Date: Mon, 27 Apr 2020 18:25:28 +0200 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: <43dbcdd2-013f-148f-6392-a09efce67f81@lassi.io> References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> <43dbcdd2-013f-148f-6392-a09efce67f81@lassi.io> Message-ID: Am Mo., 27. Apr. 2020 um 18:15 Uhr schrieb Lassi Kortela : [...] > I'd advocate focusing on web URLs and only supporting other types if > that falls out naturally from the overall design. There are now tens of > thousands of hosting sites where anyone can get their own web URLs for > free, both conventional GeoCities-style webhosts as well as social sites > like GitHub that give everyone a github.com/username. Do we want advertising for commercial services in library names in Scheme? > Of course, if a library name part can be a URN (Uniform Resource Name, > https://en.wikipedia.org/wiki/Uniform_Resource_Name) that's one way to > solve the problem. We can do the same thing we did for Unisig > (https://github.com/unisig/unisig) where the default URI scheme is > whatever protocol is the most popular way to transfer web pages at any > given time (previously http, now https, in the future http/2 or > something else). To encode a protocol in library names is crazy if you ask me, especially because the most popular protocol changes over time as you write. But without the protocol, we don't have URLs anymore. > I'd recommend simply not using any fancy URLs. A simple > "foo.com/bar/baz" with lowercase [a-z0-9], dash, slash, and dot ought to > be enough for almost everything. I guess that most R7RS systems will map the library `(foo.com/bar/baz)' to the same file system location as `(foo.com bar baz)'. To me, this shows some flaws in the approach. To use the example from above, GitHub may distribute a library `(github.com hacker foo)' under this proposal. Mr. Hacker owning a GitHub account wants to distribute `(github.com/hacker foo)'. This won't work without a clash. From peter at more-magic.net Mon Apr 27 12:14:38 2020 From: peter at more-magic.net (Peter Bex) Date: Mon, 27 Apr 2020 18:14:38 +0200 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> Message-ID: <20200427161438.GB15813@scully.more-magic.net> On Mon, Apr 27, 2020 at 11:38:10AM -0400, John Cowan wrote: > An excellent idea for a no-code SRFI that would be easy to write for people > who haven't written one before. It should specify both the > petname convention and the URL convention. Attention should be drawn to > mailto: URLs, which allow people who don't control any conventional URL to > have their own library namespace. Assuming/hoping there will at some point be a registry where one can download r7rs libraries (maybe there already is? snowfort?), I think it can be as simple as suggesting people use their library name as it occurs in the registry as a prefix. This is lightweight, easy to understand and remember, and not as ugly or fraught with issues like the URI proposal. For programs, one could use the main binary's name as a prefix. If the binary is going to be installed into PATH under the scheme system's bin directory, that has to be unique already, anyway. And if it's installed somewhere else, it shouldn't cause any problem as long as the search path for binaries matches the search path for libraries (e.g.., first the current working directory, then system paths). > I wonder if it's better to recommend that all URLs be enclosed in vertical > bars, which is safe and simple as vertical bars are not valid in URLs > unless %-escaped, or to specify that only URLs containing characters from > "#[]'(),;" be enclosed, which minimizes the use of vertical bars. That would be a problem because then you'd need an additional mapping of URL to filesystem location for the module. A colon is not allowed in Windows file names, more than one dot might be a problem in some OSes as well, and slashes can't occur in file names either. There's many more specifal characters: https://en.wikipedia.org/wiki/Filename#Comparison_of_filename_limitations So an e-mail address might not work either. Cheers, Peter -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From lassi at lassi.io Mon Apr 27 13:01:59 2020 From: lassi at lassi.io (Lassi Kortela) Date: Mon, 27 Apr 2020 20:01:59 +0300 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> <20200427161438.GB15813@scully.more-magic.net> Message-ID: <462db98e-3deb-c715-b190-d45f3338bbe7@lassi.io> > A registry would be useful but it should not be the only way to name libraries. A registry forces authors to register their libraries for every new library they want others to use. But that can become a rather tedious process, especially if you are into rapid development and sharing a new idea ?right now?. I want to put the library up on my github account and share the link? a 60 second process at most. +1 Last I checked, Racket requires new packages to be registered in order to download them using their "raco pkg" tool. The registration process is completely automated, and it's very painless and nicely done, but it'd still be nice to be able to skip it if a geographically distributed group of people is working by themselves and doesn't (yet) need to share their work. npm (node.js package manager) arguably shows that it may not even be desirable to publish all libraries people write in a central registry. With too many of them it becomes difficult for users to figure out which ones are polished enough for serious use. Collections like snow-fort, Akku, Chicken eggs and Emacs MELPA strike a good balance between quality and comprehensiveness IMHO. From mflatt at cs.utah.edu Mon Apr 27 13:07:51 2020 From: mflatt at cs.utah.edu (Matthew Flatt) Date: Mon, 27 Apr 2020 11:07:51 -0600 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: <462db98e-3deb-c715-b190-d45f3338bbe7@lassi.io> References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> <20200427161438.GB15813@scully.more-magic.net> <462db98e-3deb-c715-b190-d45f3338bbe7@lassi.io> Message-ID: <20200427170757.F22EA450628@pmg3.iro.umontreal.ca> At Mon, 27 Apr 2020 20:01:59 +0300, Lassi Kortela wrote: > Last I checked, Racket requires new packages to be registered in order > to download them using their "raco pkg" tool. No, `raco pkg` supports package sources in the form of URLs, Git repo references, local paths, etc. Racketeers sometimes use those alternatives to avoid delays in the catalog refresh. https://docs.racket-lang.org/pkg/Package_Concepts.html#%28part._concept~3asource%29 Matthew From feeley at iro.umontreal.ca Mon Apr 27 13:14:52 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Mon, 27 Apr 2020 13:14:52 -0400 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: <462db98e-3deb-c715-b190-d45f3338bbe7@lassi.io> References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> <20200427161438.GB15813@scully.more-magic.net> <462db98e-3deb-c715-b190-d45f3338bbe7@lassi.io> Message-ID: > On Apr 27, 2020, at 1:01 PM, Lassi Kortela wrote: > >> A registry would be useful but it should not be the only way to name libraries. A registry forces authors to register their libraries for every new library they want others to use. But that can become a rather tedious process, especially if you are into rapid development and sharing a new idea ?right now?. I want to put the library up on my github account and share the link? a 60 second process at most. > > +1 > > Last I checked, Racket requires new packages to be registered in order to download them using their "raco pkg" tool. The registration process is completely automated, and it's very painless and nicely done, but it'd still be nice to be able to skip it if a geographically distributed group of people is working by themselves and doesn't (yet) need to share their work. The problem with registering a name is that it becomes reserved. So the first one to reserve ?list?, ?sort?, ?gui?, etc will prevent others to use those names even if they have a ?better? library. With URLs you don?t have that problem. Yes URLs are longer, but they point unambiguously to a specific library. > > npm (node.js package manager) arguably shows that it may not even be desirable to publish all libraries people write in a central registry. With too many of them it becomes difficult for users to figure out which ones are polished enough for serious use. > > Collections like snow-fort, Akku, Chicken eggs and Emacs MELPA strike a good balance between quality and comprehensiveness IMHO. > Perhaps and I?m not at all against it. It seems there are two needs? what you could call ?community agreed portable libraries? and ?libraries in the wild?. Marc From hendrik at topoi.pooq.com Mon Apr 27 14:56:46 2020 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 27 Apr 2020 14:56:46 -0400 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> <43dbcdd2-013f-148f-6392-a09efce67f81@lassi.io> Message-ID: <20200427185646.cgcwtnctp5kbhsyn@topoi.pooq.com> On Mon, Apr 27, 2020 at 07:35:06PM +0300, Lassi Kortela wrote: > > Good point. I'd advocate for standardizing a simple rewriting and filename > mapping engine to work around problems like this (which I believe will come > up no matter what kind of practical naming scheme we choose). The progrmming language Eiffel has a module linking notation used to direct its compiler. It recognised that modules get aggregated into packages, that packages get aggregated into larger packages and so forth. Each level of this hierarchy can get name clashes. So this notation has a mechanism to rename anything on import or export The idea is that each complete program comes with such a file telling the compiler in what hierarchy all these modules are to be combined and be renamed and have their imports and exports renamed. Presumably such a file can contain information where these modules and such can be found. Presumably it can also be wrapped with instructions about finding these modules, perhaps even rewriting the URLs or module names. I found this notation described near the end of one of the books describing Eiffel. I no longer have access to this book, or I'd give yo a detailed reference. But I suspect this mechanism shouls be considered when making similar decisions about Scheme. -- hendrik From lassi at lassi.io Mon Apr 27 15:11:36 2020 From: lassi at lassi.io (Lassi Kortela) Date: Mon, 27 Apr 2020 22:11:36 +0300 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: <20200427185646.cgcwtnctp5kbhsyn@topoi.pooq.com> References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> <43dbcdd2-013f-148f-6392-a09efce67f81@lassi.io> <20200427185646.cgcwtnctp5kbhsyn@topoi.pooq.com> Message-ID: On 27.04.2020 21.56, Hendrik Boom wrote: >> Good point. I'd advocate for standardizing a simple rewriting and filename >> mapping engine to work around problems like this (which I believe will come >> up no matter what kind of practical naming scheme we choose). > > The progrmming language Eiffel has a module linking notation used to > direct its compiler. > > It recognised that modules get aggregated into packages, that packages > get aggregated into larger packages and so forth. > > Each level of this hierarchy can get name clashes. So this notation > has a mechanism to rename anything on import or export > > The idea is that each complete program comes with such a file telling > the compiler in what hierarchy all these modules are to be combined and > be renamed and have their imports and exports renamed. > > Presumably such a file can contain information where these modules and > such can be found. > > Presumably it can also be wrapped with instructions about finding these > modules, perhaps even rewriting the URLs or module names. > > I found this notation described near the end of one of the books > describing Eiffel. I no longer have access to this book, or I'd give yo > a detailed reference. > > But I suspect this mechanism shouls be considered when making similar > decisions about Scheme. Very valuable tip, thank you for writing it up! The Eiffel site uses the term "cluster": "A group of related classes or, recursively, of related clusters." It says Eiffel projects have .ecf files that describe their configuration and dependencies. Here's one example: . Does this seem familiar to you? From hendrik at topoi.pooq.com Mon Apr 27 15:34:48 2020 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 27 Apr 2020 15:34:48 -0400 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> <43dbcdd2-013f-148f-6392-a09efce67f81@lassi.io> <20200427185646.cgcwtnctp5kbhsyn@topoi.pooq.com> Message-ID: <20200427193447.bzgdapp62vtveorc@topoi.pooq.com> On Mon, Apr 27, 2020 at 10:11:36PM +0300, Lassi Kortela wrote: > On 27.04.2020 21.56, Hendrik Boom wrote: > > > Good point. I'd advocate for standardizing a simple rewriting and filename > > > mapping engine to work around problems like this (which I believe will come > > > up no matter what kind of practical naming scheme we choose). > > > > The progrmming language Eiffel has a module linking notation used to > > direct its compiler. > > > > It recognised that modules get aggregated into packages, that packages > > get aggregated into larger packages and so forth. > > > > Each level of this hierarchy can get name clashes. So this notation > > has a mechanism to rename anything on import or export > > > > The idea is that each complete program comes with such a file telling > > the compiler in what hierarchy all these modules are to be combined and > > be renamed and have their imports and exports renamed. > > > > Presumably such a file can contain information where these modules and > > such can be found. > > > > Presumably it can also be wrapped with instructions about finding these > > modules, perhaps even rewriting the URLs or module names. > > > > I found this notation described near the end of one of the books > > describing Eiffel. I no longer have access to this book, or I'd give yo > > a detailed reference. > > > > But I suspect this mechanism shouls be considered when making similar > > decisions about Scheme. > > Very valuable tip, thank you for writing it up! > > The Eiffel site uses the term "cluster": "A group of related classes or, > recursively, of related clusters." It says Eiffel projects have .ecf files > that describe their configuration and dependencies. Here's one example: > . Does this seem > familiar to you? No. not familiar at all. The example in the book was not written in XML. I can't tell whether this is an XMLized version of that they had back then, or something completely different. I wish I still had the book. -- hendrik > > _______________________________________________ > Gambit-list mailing list > Gambit-list at iro.umontreal.ca > https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list > From lassi at lassi.io Mon Apr 27 15:43:13 2020 From: lassi at lassi.io (Lassi Kortela) Date: Mon, 27 Apr 2020 22:43:13 +0300 Subject: [gambit-list] Eiffel In-Reply-To: <20200427193447.bzgdapp62vtveorc@topoi.pooq.com> References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> <43dbcdd2-013f-148f-6392-a09efce67f81@lassi.io> <20200427185646.cgcwtnctp5kbhsyn@topoi.pooq.com> <20200427193447.bzgdapp62vtveorc@topoi.pooq.com> Message-ID: >>> The progrmming language Eiffel has a module linking notation used to >>> direct its compiler. >> Eiffel projects have .ecf files >> that describe their configuration and dependencies. Here's one example: >> . Does this seem >> familiar to you? > > No. not familiar at all. The example in the book was not written in XML. > I can't tell whether this is an XMLized version of that they had back then, or something completely different. What about this? From : PART IV: THE LACE CONTROL LANGUAGE ........................................ 1005 B Specifying systems in Lace (in progress) 1007 B.1 OVERVIEW 1007 B.2 A SIMPLE EXAMPLE 1008 B.3 ON THE ROLE OF LACE 1009 B.4 A COMPLETE EXAMPLE 1010 B.5 BASIC CONVENTIONS 1012 B.6 BASICS OF CLUSTER CLAUSES 1014 B.7 STORING PROPERTIES WITH A CLUSTER 1016 B.8 EXCLUDING AND INCLUDING SOURCE FILES 1016 B.9 SPECIFYING OPTIONS 1018 B.10 SPECIFYING EXTERNAL ELEMENTS 1022 B.11 ONCE CONTROL 1023 B.12 GENERATION 1023 B.13 VISIBLE FEATURES 1024 B.14 COMPLETE LACE GRAMMAR 1027 B.15 LACE VALIDITY RULES 1027 From hendrik at topoi.pooq.com Mon Apr 27 15:56:04 2020 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 27 Apr 2020 15:56:04 -0400 Subject: [gambit-list] Eiffel In-Reply-To: References: <43dbcdd2-013f-148f-6392-a09efce67f81@lassi.io> <20200427185646.cgcwtnctp5kbhsyn@topoi.pooq.com> <20200427193447.bzgdapp62vtveorc@topoi.pooq.com> Message-ID: <20200427195604.uszdms5mn5awem26@topoi.pooq.com> On Mon, Apr 27, 2020 at 10:43:13PM +0300, Lassi Kortela wrote: > > > > The progrmming language Eiffel has a module linking notation used to > > > > direct its compiler. > > > > Eiffel projects have .ecf files > > > that describe their configuration and dependencies. Here's one example: > > > . Does this seem > > > familiar to you? > > > > No. not familiar at all. The example in the book was not written in XML. > > I can't tell whether this is an XMLized version of that they had back then, or something completely different. > > What about this? From > : > > PART IV: THE LACE CONTROL LANGUAGE ........................................ > 1005 > B Specifying systems in Lace (in progress) 1007 > B.1 OVERVIEW 1007 > B.2 A SIMPLE EXAMPLE 1008 > B.3 ON THE ROLE OF LACE 1009 > B.4 A COMPLETE EXAMPLE 1010 > B.5 BASIC CONVENTIONS 1012 > B.6 BASICS OF CLUSTER CLAUSES 1014 > B.7 STORING PROPERTIES WITH A CLUSTER 1016 > B.8 EXCLUDING AND INCLUDING SOURCE FILES 1016 > B.9 SPECIFYING OPTIONS 1018 > B.10 SPECIFYING EXTERNAL ELEMENTS 1022 > B.11 ONCE CONTROL 1023 > B.12 GENERATION 1023 > B.13 VISIBLE FEATURES 1024 > B.14 COMPLETE LACE GRAMMAR 1027 > B.15 LACE VALIDITY RULES 1027 Can't tell for sure because the pdf doesn't contain anything beyond the table of contents, but this seems to be the right set of concepts. And from the number of pages, it may be more extensive than what I remember. Maybe I had an earlier edition. Or maybe I misremember the size. -- hendrik From lassi at lassi.io Mon Apr 27 16:02:44 2020 From: lassi at lassi.io (Lassi Kortela) Date: Mon, 27 Apr 2020 23:02:44 +0300 Subject: [gambit-list] Eiffel In-Reply-To: <20200427195604.uszdms5mn5awem26@topoi.pooq.com> References: <43dbcdd2-013f-148f-6392-a09efce67f81@lassi.io> <20200427185646.cgcwtnctp5kbhsyn@topoi.pooq.com> <20200427193447.bzgdapp62vtveorc@topoi.pooq.com> <20200427195604.uszdms5mn5awem26@topoi.pooq.com> Message-ID: >> What about this? From >> : >> > Can't tell for sure because the pdf doesn't contain anything beyond the > table of contents, but this seems to be the right set of concepts. > > And from the number of pages, it may be more extensive than what I > remember. Maybe I had an earlier edition. Or maybe I misremember the > size. LACE stands for Language for Assembly Classes in Eiffel. The files in which it is written are called "Ace files". More links: http://se.inf.ethz.ch/courses/2013b_fall/eprog/additional_materials/eiffel_the_essentials.pdf https://archive.eiffel.com/doc/online/eiffel50/intro/studio/index-05.html https://wiki.liberty-eiffel.org/index.php/ACE https://dev.eiffel.com/ACE_to_ECF:_The_Transition_Explained ("ACE to ECF: The Transition Explained. This page addresses the rationale behind EiffelStudio's new ECF configuration format, and clarifies which needs were covered by the new system. ECF stands for Eiffel Configuration File.") From hendrik at topoi.pooq.com Mon Apr 27 18:15:28 2020 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 27 Apr 2020 18:15:28 -0400 Subject: [gambit-list] Eiffel In-Reply-To: References: <43dbcdd2-013f-148f-6392-a09efce67f81@lassi.io> <20200427185646.cgcwtnctp5kbhsyn@topoi.pooq.com> <20200427193447.bzgdapp62vtveorc@topoi.pooq.com> <20200427195604.uszdms5mn5awem26@topoi.pooq.com> Message-ID: <20200427221528.64z7cxivnq46kxpn@topoi.pooq.com> On Mon, Apr 27, 2020 at 11:02:44PM +0300, Lassi Kortela wrote: > > > What about this? From > > > : > > > > > Can't tell for sure because the pdf doesn't contain anything beyond the > > table of contents, but this seems to be the right set of concepts. > > > > And from the number of pages, it may be more extensive than what I > > remember. Maybe I had an earlier edition. Or maybe I misremember the > > size. > > LACE stands for Language for Assembly Classes in Eiffel. The files in which > it is written are called "Ace files". More links: > > http://se.inf.ethz.ch/courses/2013b_fall/eprog/additional_materials/eiffel_the_essentials.pdf > > https://archive.eiffel.com/doc/online/eiffel50/intro/studio/index-05.html > > https://wiki.liberty-eiffel.org/index.php/ACE > > https://dev.eiffel.com/ACE_to_ECF:_The_Transition_Explained > ("ACE to ECF: The Transition Explained. This page addresses the rationale > behind EiffelStudio's new ECF configuration format, and clarifies which > needs were covered by the new system. ECF stands for Eiffel Configuration > File.") Yes, it now looks like its tie up-to-date successor to what I saw a decade or more ago and what I was talking about. -- hendrik From feeley at iro.umontreal.ca Mon Apr 27 22:30:10 2020 From: feeley at iro.umontreal.ca (Marc Feeley) Date: Mon, 27 Apr 2020 22:30:10 -0400 Subject: [gambit-list] Module system ELS20 presentation In-Reply-To: References: <16B4B124-7E51-47E5-BDDF-701D15F4E81A@iro.umontreal.ca> Message-ID: > On Apr 27, 2020, at 10:24 AM, Marc Feeley wrote: > >> >> A Termite-specific question: why send bytecode rather than C (perhaps minified by removing whitespace)? Presumably that would cost nothing at the source node (it's already been done) and be faster and cheaper at the destination node? Is the bytecode so much more compact as to make transmitting C untenable? > > Termite does not send bytecode. If you are referring to the slides, we were just saying ?one way to implement portable code is with bytecode? but Termite does not do that. It can either send what is essentially an AST (in the ?interpreted? code case), or a symbolic reference to the compiled code control point (in the ?compiled? code case). A symbolic reference is basically the pair . I notice my answer doesn?t cover all your questions. Even though the C code generated by the Gambit compiler is portable and could be compiled on all the platforms, it is not quite sufficient to do it that way because macros can observe properties of the system where the library was built. So it is conceivable that a macro could affect the generated C code. This has the potential to ruin deserialization of procedures if the macro behaves badly, but if the ?programmer knows what he?s doing? (like have a macro that simply put?s the build machine?s hostname in the binary, or a random number stamp) then all is well. Apart from that problem, there are lots of ways to represent the code, with various levels of space usage (tested with lib/_num.scm): 3396 KB _num.c (C code generated by gsc from _num.scm) 446 KB _num.scm 296 KB gzip -9 _num.c 276 KB _num.scm (read-all of _num.scm followed by write) 153 KB binary serialization format (read-all of _num.scm followed by object->u8vector) 88 KB gzip -9 _num.scm 60 KB gzip -9 of (read-all of _num.scm followed by write) 55 KB gzip -9 of binary serialization format So there?s a factor of 62x between the most compact and least compact representation. The various forms of the C code are not on the best side of that scale? Marc From lassi at lassi.io Thu Apr 30 07:54:43 2020 From: lassi at lassi.io (Lassi Kortela) Date: Thu, 30 Apr 2020 14:54:43 +0300 Subject: [gambit-list] Crowdsourcing Gambit documentation Message-ID: <2ccf7115-e48e-1655-3c8d-0b762f42d88c@lassi.io> Currently Gambit documentation exists in: - the wiki at gambitscheme.org - the manual in doc/gambit.txi in the gambit repo - various research papers and presentations - email messages and github issues Email and issues are great for getting work done, but document the (often messy) development process rather than its result. In papers and presentations the perspective can be too broad or too deep to serve as an effective manual. Lots of comparisons are made to theory as well as other systems in the field. That's great for research but is usually not actionable information for users. The wiki is in principle a good middle ground between email and the manual with a low barrier to entry, but in practice an organized documentation set has not emerged from it. Clojure offers a similar example: clojuredocs.org allows user comments on every page, but while they are useful they never seem to result in a cohesive documentation set even as the site has existed for years. The official documentation shown at the top of the page is quite rudimentary and users are left on their own to search the comments in case they need more details (as they commonly do). EmacsWiki.org is another wiki with a lot of effort put into it. While it is (or used to be) very comprehensive and useful, it's never quite clear what information is up to date and whether something is "official" or simply something that happened to solve particular people's problem at the time it was written. Wikipedia doesn't have much trouble developing wiki pages into readable articles, so I wonder whether the lack of convergence in programming wikis is idiosyncratic to their target audience. Programmers are one of the only populations who are more comfortable working around structured systems like Git repos instead of more free-form ones like wikis and email. The standard programmer workflow nowadays revolves around Git, so anything that is quickly done via Git is easy to add to the daily loop with low friction. To that end, would it make sense if we tried to centralize all Gambit documentation into its texinfo manual and try to make that manual really comprehensive? It would take some up-front effort: breaking down the monolithic texinfo file into smaller portions that are easier to edit; integrating existing knowledge from the gambitscheme.org wiki, mailing lists and research papers. But it would enable a Git-based workflow based around GitHub issues and pull requests for discussing and reviewing changes. I don't know about others, but for me that workflow has become so second nature that it's often easier to clone a repo and send them a PR than make a mental to-do note to return to a task later. Thoughts?