I generate a list of numbers, write them to file, and test the time it takes to read it back. Attached is my file bench.scm

For most of them, plain_text appears to be faster; and only at 10 million entries, does the binary format become slightly faster. What am I doing wrong?

running with: gsi bench.scm

(number-of-entries: 1 (binary: 1.5497207641601562e-4) (plain: 9.679794311523438e-5)) (number-of-entries: 10 (binary: 8.487701416015625e-5) (plain: 6.699562072753906e-5)) (number-of-entries: 100 (binary: 1.399517059326172e-4) (plain: 1.6808509826660156e-4)) (number-of-entries: 1000 (binary: .0024831295013427734) (plain: .0011830329895019531)) (number-of-entries: 10000 (binary: .004712104797363281) (plain: .005532979965209961)) (number-of-entries: 100000 (binary: .06526494026184082) (plain: .04858684539794922)) (number-of-entries: 1000000 (binary: .8762490749359131) (plain: .55377197265625)) (number-of-entries: 10000000 (binary: 7.88455605506897) (plain: 9.257179021835327))

On Mon, Feb 16, 2009 at 11:37 AM, Jeremie Lasalle Ratelle < pouexmachinax@gmail.com> wrote:

object->u8vector and u8vector->object write-subu8vector and read-subu8vector on the file port

What I currently do on tcp/ip port is to first write a u8vector of size 4 representing the size of the object following and then write the u8vector representing the object.

I guess you could do the same in files. You would read 4 u8 representing the size, read-subu8vector of the size read and repeat until eof.

I'm really glad you responded -- I wasn't aware gambit-list blocked attachments

$ cat bench.scm: (define (gen-lst min max) (define (helper min max cur) (if (< min max) (helper min (- max 1) (cons max cur)) cur)) (helper (- min 1) (- max 1) '()))

(define (pad-with value amount lst) (define (helper amount lst) (if (<= amount 0) lst (helper (- amount 1) (cons value lst)))) (helper amount lst))

(define (pad-to value ttl-length lst) (pad-with value (- ttl-length (length lst)) lst))

(define (convert-to-base value base) (define (helper value cur) (if (= value 0) cur (helper (quotient value base) (cons (remainder value base) cur)))) (helper value '()))

(define (base-to-int lst base) (define (helper lst cur) (if (null? lst) cur (helper (cdr lst) (+ (* cur base) (car lst))))) (helper lst 0))

(define (int-as-length8-uvector value) (list->u8vector (pad-to 0 8 (convert-to-base value 256))))

(define (length8-uvector-as-int vec) (base-to-int (u8vector->list vec) 256))

;(pp (base-to-int (convert-to-base 123456 10) 10))

;(pp (list->u8vector '(1 2 3 4 5 255)))

(define (make-timer) (let ((start-time 0)) (define (dispatch cmd) (cond ((equal? cmd 'start) (set! start-time (time->seconds (current-time)))) ((equal? cmd 'stop) (- (time->seconds (current-time)) start-time)))) dispatch))

(define my-timer (make-timer))

;(pp (gen-lst 0 10))

(define (serialize-text object filename) (call-with-output-file filename (lambda (file) (pp object file))))

(define (serialize-binary object filename) (call-with-output-file filename (lambda (file) (let ((as-vec (object->u8vector object))) (write-subu8vector (int-as-length8-uvector (u8vector-length as-vec)) 0 8 file) (write-subu8vector as-vec 0 (u8vector-length as-vec) file)))))

(define (unserialize-binary filename) (call-with-input-file filename (lambda (file) (let* ((vec (make-u8vector 8))) (read-subu8vector vec 0 8 file) (let* ((len (length8-uvector-as-int vec)) (obj-vec (make-u8vector len))) (read-subu8vector obj-vec 0 len file) (u8vector->object obj-vec))))))

(define (bench number-of-entries)

(define object (gen-lst 0 number-of-entries))

(serialize-text object "plain_text") (serialize-binary object "binary")

(let ((plain-time 0) (binary-time 0))

(my-timer 'start) (unserialize-binary "binary") (set! binary-time (my-timer 'stop))

(my-timer 'start) (read-all (open-input-file "plain_text")) (set! plain-time (my-timer 'stop))

(pp `(number-of-entries: ,number-of-entries (binary: ,binary-time) (plain: ,plain-time)))))

(bench 1) (bench 10) (bench 100) (bench 1000) (bench 10000) (bench 100000) (bench 1000000) (bench 10000000)

$ gsi bench.scm (number-of-entries: 1 (binary: 5.1021575927734375e-5) (plain: 3.0040740966796875e-5)) (number-of-entries: 10 (binary: 3.0040740966796875e-5) (plain: 2.4080276489257812e-5)) (number-of-entries: 100 (binary: 4.792213439941406e-5) (plain: 5.698204040527344e-5)) (number-of-entries: 1000 (binary: 2.8395652770996094e-4) (plain: 4.1794776916503906e-4)) (number-of-entries: 10000 (binary: .004720926284790039) (plain: .009029150009155273)) (number-of-entries: 100000 (binary: .07220315933227539) (plain: .052423954010009766)) (number-of-entries: 1000000 (binary: 1.0091660022735596) (plain: .6231379508972168)) (number-of-entries: 10000000 (binary: 9.397350072860718) (plain: 17.698192834854126))

On Mon, Feb 16, 2009 at 6:22 PM, Adrien Piérard pierarda@iro.umontreal.cawrote:

For most of them, plain_text appears to be faster; and only at 10 million

...

entries, does the binary format become slightly faster. What am I doing wrong?

You are not showing us the code. That is just *wrong*.

Are you converting each number independently, or the whole list at once?

P!

-- Français, English, 日本語, 한국어

Only first response necessary. Can you guys read the attached bench.scm file?

If so, please tell me why is it that binary is barely faster than plain text.

On Wed, Feb 18, 2009 at 7:42 AM, Marijn Schouten (hkBst) hkBst@gentoo.orgwrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

lowly coder wrote:

...

I'm really glad you responded -- I wasn't aware gambit-list blocked attachments

It doesn't.

Marijn

---

Sarcasm puts the iron in irony, cynicism the steel.

Marijn Schouten (hkBst), Gentoo Lisp project, Gentoo ML http://www.gentoo.org/proj/en/lisp/, #gentoo-{lisp,ml} on FreeNode -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkmcLGEACgkQp/VmCx0OL2wjqQCaAwGY3SU0pXmZ1GKnI0FP/7CE QRMAoMcep74UNOep6kr9G6QSNKPY/7U2 =uiVH -----END PGP SIGNATURE----- _______________________________________________ Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list