I need to descend a directory structure with ~5000 files in it and run an external process on each to gether information on each file. However, after running over about 250 files, I get a "Too many open files" error. 250 is probably OSX default max open file limit - but I don't understand why the 'with-input-from-process' doesn't appear to be closing its file handle? I also tried with 'open-input-process' and 'open-process' and manually calling 'close-input-port' but didn't help. Here's a small script that shows the exact same prob:
(define (test-fn)
(define (do-process) (with-input-from-process (list path: "pwd") read-all))
(let loop [(count 2000)] (if (> count 0) (begin (do-process) (loop (- count 1))) (println "Done"))))
(test-fn)
And the output:
bigmac:iosTexConverter aking$ gsc -v v4.6.1 20110519175028 i386-apple-darwin10.8.0 "./configure /usr/local/Gambit-C/share/config.site /usr/local/Gambit-C/etc/config.site"
bigmac:iosTexConverter aking$ gsi ./test.scm *** ERROR IN loop, "test.scm"@9.11 -- Too many open files (with-input-from-process '(path: "pwd") '#<procedure #2 read-all>) bigmac:iosTexConverter aking$
Running latest from the Gambit git repo and OSX 10.6.8. As far as I can tell, calling 'pwd' shouldn't be leaving around open file handles. I also checked the process-status, and it was 0. Any suggestions?
Afficher les réponses par date
(While making a real solution to the problem, and as a part of debugging, run ##gc right at the end of each do-process . Does it have any effect?)
2011/9/9 Adam King akingcoder@gmail.com
I need to descend a directory structure with ~5000 files in it and run an external process on each to gether information on each file. However, after running over about 250 files, I get a "Too many open files" error. 250 is probably OSX default max open file limit - but I don't understand why the 'with-input-from-process' doesn't appear to be closing its file handle? I also tried with 'open-input-process' and 'open-process' and manually calling 'close-input-port' but didn't help. Here's a small script that shows the exact same prob:
(define (test-fn)
(define (do-process) (with-input-from-process (list path: "pwd") read-all))
(let loop [(count 2000)] (if (> count 0) (begin (do-process) (loop (- count 1))) (println "Done"))))
(test-fn)
And the output:
bigmac:iosTexConverter aking$ gsc -v v4.6.1 20110519175028 i386-apple-darwin10.8.0 "./configure /usr/local/Gambit-C/share/config.site /usr/local/Gambit-C/etc/config.site"
bigmac:iosTexConverter aking$ gsi ./test.scm *** ERROR IN loop, "test.scm"@9.11 -- Too many open files (with-input-from-process '(path: "pwd") '#<procedure #2 read-all>) bigmac:iosTexConverter aking$
Running latest from the Gambit git repo and OSX 10.6.8. As far as I can tell, calling 'pwd' shouldn't be leaving around open file handles. I also checked the process-status, and it was 0. Any suggestions?
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
I placed a #gc in two different locations (before and after each call to (do-process)), but still the same error message. I also tried with a compiled version, same issue. Thanks for the suggestion though.
Adam
On Fri, Sep 9, 2011 at 5:16 PM, Mikael mikael.rcv@gmail.com wrote:
(While making a real solution to the problem, and as a part of debugging, run ##gc right at the end of each do-process . Does it have any effect?)
2011/9/9 Adam King akingcoder@gmail.com
I need to descend a directory structure with ~5000 files in it and run an external process on each to gether information on each file. However, after running over about 250 files, I get a "Too many open files" error. 250 is probably OSX default max open file limit - but I don't understand why the 'with-input-from-process' doesn't appear to be closing its file handle? I also tried with 'open-input-process' and 'open-process' and manually calling 'close-input-port' but didn't help. Here's a small script that shows the exact same prob:
(define (test-fn)
(define (do-process) (with-input-from-process (list path: "pwd") read-all))
(let loop [(count 2000)] (if (> count 0) (begin (do-process) (loop (- count 1))) (println "Done"))))
(test-fn)
And the output:
bigmac:iosTexConverter aking$ gsc -v v4.6.1 20110519175028 i386-apple-darwin10.8.0 "./configure /usr/local/Gambit-C/share/config.site /usr/local/Gambit-C/etc/config.site"
bigmac:iosTexConverter aking$ gsi ./test.scm *** ERROR IN loop, "test.scm"@9.11 -- Too many open files (with-input-from-process '(path: "pwd") '#<procedure #2 read-all>) bigmac:iosTexConverter aking$
Running latest from the Gambit git repo and OSX 10.6.8. As far as I can tell, calling 'pwd' shouldn't be leaving around open file handles. I also checked the process-status, and it was 0. Any suggestions?
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
-
Adam King -
Mikael