Still seems to hold on to a file handle somewhere :(
Marc - is there any chance you could have a look - or try to replicate under OSX? I've looked through _io.scm and at the ##open-process-generic prim, but it's still over my head. I really would like to use Gambit for some scripting duties over all our game data (thousands of media files). Thanks,
Adam
On Fri, Sep 9, 2011 at 5:36 PM, Mikael mikael.rcv@gmail.com wrote:
Huh. Just for more debug, what about, for each process port created, make a copy of it but with a weak GC reference, then after with-input-from-process is done do ##gc, and check if the port still exists. Does it? Mikael
2011/9/10 Adam King akingcoder@gmail.com
I placed a #gc in two different locations (before and after each call to (do-process)), but still the same error message. I also tried with a compiled version, same issue. Thanks for the suggestion though.
Adam
On Fri, Sep 9, 2011 at 5:16 PM, Mikael mikael.rcv@gmail.com wrote:
(While making a real solution to the problem, and as a part of debugging, run ##gc right at the end of each do-process . Does it have any effect?)
2011/9/9 Adam King akingcoder@gmail.com
I need to descend a directory structure with ~5000 files in it and run an external process on each to gether information on each file. However, after running over about 250 files, I get a "Too many open files" error. 250 is probably OSX default max open file limit - but I don't understand why the 'with-input-from-process' doesn't appear to be closing its file handle? I also tried with 'open-input-process' and 'open-process' and manually calling 'close-input-port' but didn't help. Here's a small script that shows the exact same prob:
(define (test-fn)
(define (do-process) (with-input-from-process (list path: "pwd") read-all))
(let loop [(count 2000)] (if (> count 0) (begin (do-process) (loop (- count 1))) (println "Done"))))
(test-fn)
And the output:
bigmac:iosTexConverter aking$ gsc -v v4.6.1 20110519175028 i386-apple-darwin10.8.0 "./configure /usr/local/Gambit-C/share/config.site /usr/local/Gambit-C/etc/config.site"
bigmac:iosTexConverter aking$ gsi ./test.scm *** ERROR IN loop, "test.scm"@9.11 -- Too many open files (with-input-from-process '(path: "pwd") '#<procedure #2 read-all>) bigmac:iosTexConverter aking$
Running latest from the Gambit git repo and OSX 10.6.8. As far as I can tell, calling 'pwd' shouldn't be leaving around open file handles. I also checked the process-status, and it was 0. Any suggestions?
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
Afficher les réponses par date
On 2011-09-11, at 4:55 PM, Adam King wrote:
Still seems to hold on to a file handle somewhere :(
Marc - is there any chance you could have a look - or try to replicate under OSX? I've looked through _io.scm and at the ##open-process-generic prim, but it's still over my head. I really would like to use Gambit for some scripting duties over all our game data (thousands of media files). Thanks,
Adam
I have reproduced the bug. I haven't had time to find a fix, but my guess is that there are some calls to "close" missing in the function ___device_process_cleanup in lib/os_io.c .
Marc
On 2011-09-11, at 4:55 PM, Adam King wrote:
Still seems to hold on to a file handle somewhere :(
Marc - is there any chance you could have a look - or try to replicate under OSX? I've looked through _io.scm and at the ##open-process-generic prim, but it's still over my head. I really would like to use Gambit for some scripting duties over all our game data (thousands of media files). Thanks,
The bug has now been fixed on the Gambit repo. It was a file descriptor leak. The responsibility for closing the file descriptors (2 per process created) was not being transferred to the runtime system.
The fix has been tested to work on Mac OS X (I assume the bug existed on other Unices, and the fix will work there too).
Thanks for reporting the bug!
Marc