Hi, I am a newbie of Gambit Scheme coming from Clojure. Currently I am trying to build an iOS app in Gambit. I have successfully done the following: 1. Built 4.6.6-devel git version (snapshot of two weeks ago) of Gambit using llvm-gcc 2. Built Gambit REPL using iOS SDK 6.1 by using both clang and llvm-gcc from Xcode. 3. Minimized the remote REPL part of Gambit REPL and created a simple app 3.1. Created a scheme file, init.scm, to register and change the text of the UILabel.
;; The whole init.scm (declare (standard-bindings) (extended-bindings) (block) (fixnum) ;(not safe) )
(c-define-type UILabel* (pointer "UILabel"))
(c-declare "#import <UIKit/UIKit.h>") (define current-label #f)
(c-define (register-label label) (UILabel*) void "register_label" "" (set! current-label label))
(define change-label-text (c-lambda ((pointer "UILabel")) void #<<c-code ___arg1.text = @"Text changed."; c-code ))
3.2. In Xcode, I defined a viewDidLoad method as below, which prints "Hello Wolrd" and then passes the label to the scheme function register_label().
// Part of VIewController.m - (void)viewDidLoad { [super viewDidLoad]; // Do any additional setup after loading the view, typically from a nib.
_label = [[UILabel alloc] init]; _label.text = @"Hello World"; [_label setLineBreakMode:UILineBreakModeWordWrap]; [_label setNumberOfLines:0]; _label.frame = CGRectMake(0, 225, 320, 30); _label.textAlignment = UITextAlignmentCenter; [self.view addSubview:_label];
_timer = nil; _queuedActions = [[NSMutableArray alloc] init];
gambit_setup(); register_label(_label); repl_server_start(); [self heartbeat_tick]; }
So far, I have successfully built this simple app. I can connect to the remote REPL of iOS Simulator from the Emacs Inferior Lisp, and even able to change the text of the label dynamically by doing like (change-label-text current-label).
Now it seems that I can develop iOS apps, but I want to go further with the interactive development using the REPL. What I want to do is like: 1. Redefine the change-label-text function by changing the text like "New text!"
2. Create a loadable library gsc -cc-options "-arch i386 -x objective-c -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator6.1.sdk -D__IPHONE_OS_VERSION_MIN_REQUIRED=40300" -ld-options "-framework CoreFoundation -framework Foundation -framework UIKit" lib/init.scm
3. Do below in REPL (define old-label current-label)
4. Load the loadable library init.o1 from the REPL
5. Do below in REPL (change-label-text old-label)
6. Then an error in Xcode and the app terminates. 2013-02-27 17:38:31.305 gambit-test[61410:11303] *** NSForwarding: warning: selector (0x87a6f44) for message 'setText:' does not match selector known to Objective C runtime (0xabb96d)-- abort 2013-02-27 17:38:31.306 gambit-test[61410:11303] -[UILabel setText:]: unrecognized selector sent to instance 0x8284a50 2013-02-27 17:38:31.306 gambit-test[61410:11303] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[UILabel setText:]: unrecognized selector sent to instance 0x8284a50' *** First throw call stack: (0x21a4012 0x15e1e7e 0x222f4bd 0x2193bbc 0x219394e 0x87a6e07 0x44b0c0) libc++abi.dylib: terminate called throwing an exception
I can create and load a loadable library with no errors. It seems that something is missing. Does anyone know how to solve this kind of problem?
Thanks, Tatsuya Tsuda
Afficher les réponses par date
On 2013-02-27, at 4:03 AM, Tatsuya Tsuda otabat@gmail.com wrote:
I can create and load a loadable library with no errors. It seems that something is missing. Does anyone know how to solve this kind of problem?
Given that you are compiling with -arch i386, I'm assuming you are doing these experiments on the iPhone simulator. If I remember correctly, on the actual device dynamic loading is not available. So your strategy is fine for debugging with the iPhone simulator but not on a real iPhone. What you'd need is a completely dynamic bridge to Objective-C. That way you wouldn't have to compile the files you load. Simply loading a source file (on the simulator or real iPhone) that uses the bridge would work. I've done some work on a dynamic Objective-C bridge (based on objc_msgSend and friends) but it is not quite finished yet.
Now back to your problem. I believe it is a linking problem at the Objective-C level. It seems that the selector setText: in your app and the setText: in the dynamically loaded init.o1 are not the same selector. There must be some linking flag that you didn't use when building your app which caused the executable to be stripped of its symbol table (selector table?). I'm not an expert with the linking options for iOS but I hope this gives you an idea for what to look for. Note that this problem would go away with a dynamic Objective-C bridge.
Marc
Hello,
I would recommend you this strategy that I tried:
1) build a static bridge to the classes of Objective-C that you need (using macros is a good idea here to generate all the FFI from lists) 2) compile that code in a static library XCode project 3) leave the Objective-C behind forever and start coding in pure Scheme, using the remote-REPL. That way you rarely need to recompile "change-label-text".
(define change-label-text (c-lambda ((pointer "UILabel")) void #<<c-code ___arg1.text = @"Text changed."; c-code ))
This will give you quick results and a overall idea on how you can build a dynamic bridge. The dynamic bridge is a Rolls Royce in this space. Start with the Lada.
François
On 2013-02-27, at 08:53, Marc Feeley feeley@iro.umontreal.ca wrote:
On 2013-02-27, at 4:03 AM, Tatsuya Tsuda otabat@gmail.com wrote:
I can create and load a loadable library with no errors. It seems that something is missing. Does anyone know how to solve this kind of problem?
Given that you are compiling with -arch i386, I'm assuming you are doing these experiments on the iPhone simulator. If I remember correctly, on the actual device dynamic loading is not available. So your strategy is fine for debugging with the iPhone simulator but not on a real iPhone. What you'd need is a completely dynamic bridge to Objective-C. That way you wouldn't have to compile the files you load. Simply loading a source file (on the simulator or real iPhone) that uses the bridge would work. I've done some work on a dynamic Objective-C bridge (based on objc_msgSend and friends) but it is not quite finished yet.
Now back to your problem. I believe it is a linking problem at the Objective-C level. It seems that the selector setText: in your app and the setText: in the dynamically loaded init.o1 are not the same selector. There must be some linking flag that you didn't use when building your app which caused the executable to be stripped of its symbol table (selector table?). I'm not an expert with the linking options for iOS but I hope this gives you an idea for what to look for. Note that this problem would go away with a dynamic Objective-C bridge.
Marc
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
*sigh* If only I had time to work on my dynamic ObjC bridge.... next week.
On Wed, Feb 27, 2013 at 8:06 AM, Francois Magnan < magnan@categoricaldesign.com> wrote:
Hello,
I would recommend you this strategy that I tried:
- build a static bridge to the classes of Objective-C that you need
(using macros is a good idea here to generate all the FFI from lists) 2) compile that code in a static library XCode project 3) leave the Objective-C behind forever and start coding in pure Scheme, using the remote-REPL. That way you rarely need to recompile "change-label-text".
(define change-label-text (c-lambda ((pointer "UILabel")) void #<<c-code ___arg1.text = @"Text changed."; c-code ))
This will give you quick results and a overall idea on how you can build a dynamic bridge. The dynamic bridge is a Rolls Royce in this space. Start with the Lada.
François
On 2013-02-27, at 08:53, Marc Feeley feeley@iro.umontreal.ca wrote:
On 2013-02-27, at 4:03 AM, Tatsuya Tsuda otabat@gmail.com wrote:
I can create and load a loadable library with no errors. It seems that
something is missing.
Does anyone know how to solve this kind of problem?
Given that you are compiling with -arch i386, I'm assuming you are doing
these experiments on the iPhone simulator. If I remember correctly, on the actual device dynamic loading is not available. So your strategy is fine for debugging with the iPhone simulator but not on a real iPhone. What you'd need is a completely dynamic bridge to Objective-C. That way you wouldn't have to compile the files you load. Simply loading a source file (on the simulator or real iPhone) that uses the bridge would work. I've done some work on a dynamic Objective-C bridge (based on objc_msgSend and friends) but it is not quite finished yet.
Now back to your problem. I believe it is a linking problem at the
Objective-C level. It seems that the selector setText: in your app and the setText: in the dynamically loaded init.o1 are not the same selector. There must be some linking flag that you didn't use when building your app which caused the executable to be stripped of its symbol table (selector table?). I'm not an expert with the linking options for iOS but I hope this gives you an idea for what to look for. Note that this problem would go away with a dynamic Objective-C bridge.
Marc
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
On 2013-02-27, at 9:40 AM, Jason Felice jason.m.felice@gmail.com wrote:
*sigh* If only I had time to work on my dynamic ObjC bridge.... next week.
Perhaps we should join our efforts... I propose the creation of a github repo specifically for this to share our codes. I know that Jason has his own repo (https://github.com/maitria/gambit-objc), but it seems to be based on blackhole and is x86-64 specific, and thus not usable on iOS.
I've created the (currently empty) repo
https://github.com/gambit-community/objc-bridge
for the purpose of putting together a portable and dynamic Objective-C bridge in pure Gambit Scheme. Who is interested in contributing?
Marc
I'd be happy to unblackhole mine if it gets more people involved.
The x86_64 part is because ObjC doesn't provide a completely machine-abstract way to invoke dynamically. Each way has some icky flaws. My first goal was actually iOS, but I went down the x86_64 path because Mikael had a need for that first. I'd like the package to have x86_64/i386/arm.
I did discover that a LOT more machinery is needed for x86_64 than for i386 or arm. Calling conventions on 64-bit are crazy insane (though I now have the worst of it solved).
On Wed, Feb 27, 2013 at 9:11 AM, Marc Feeley feeley@iro.umontreal.cawrote:
On 2013-02-27, at 9:40 AM, Jason Felice jason.m.felice@gmail.com wrote:
*sigh* If only I had time to work on my dynamic ObjC bridge.... next
week.
Perhaps we should join our efforts... I propose the creation of a github repo specifically for this to share our codes. I know that Jason has his own repo (https://github.com/maitria/gambit-objc), but it seems to be based on blackhole and is x86-64 specific, and thus not usable on iOS.
I've created the (currently empty) repo
https://github.com/gambit-community/objc-bridge
for the purpose of putting together a portable and dynamic Objective-C bridge in pure Gambit Scheme. Who is interested in contributing?
Marc
On 2013-02-27, at 10:19 AM, Jason Felice jason.m.felice@gmail.com wrote:
I'd be happy to unblackhole mine if it gets more people involved.
The x86_64 part is because ObjC doesn't provide a completely machine-abstract way to invoke dynamically. Each way has some icky flaws. My first goal was actually iOS, but I went down the x86_64 path because Mikael had a need for that first. I'd like the package to have x86_64/i386/arm.
I did discover that a LOT more machinery is needed for x86_64 than for i386 or arm. Calling conventions on 64-bit are crazy insane (though I now have the worst of it solved).
I agree that there are issues with all approaches. If possible I'd like to avoid using assembly language which is compiler and operating system dependent (Objective-C runs on other platforms than iOS and OS X).
I wonder if there are some ideas we can steal from the Python Objective-C bridge:
http://pythonhosted.org/pyobjc/
Marc
A quick look shows that it uses libffi, which doesn't work on iOS because of the way the stack is protected.
-Jason
On Wed, Feb 27, 2013 at 9:29 AM, Marc Feeley feeley@iro.umontreal.cawrote:
On 2013-02-27, at 10:19 AM, Jason Felice jason.m.felice@gmail.com wrote:
I'd be happy to unblackhole mine if it gets more people involved.
The x86_64 part is because ObjC doesn't provide a completely
machine-abstract way to invoke dynamically. Each way has some icky flaws. My first goal was actually iOS, but I went down the x86_64 path because Mikael had a need for that first. I'd like the package to have x86_64/i386/arm.
I did discover that a LOT more machinery is needed for x86_64 than for
i386 or arm. Calling conventions on 64-bit are crazy insane (though I now have the worst of it solved).
I agree that there are issues with all approaches. If possible I'd like to avoid using assembly language which is compiler and operating system dependent (Objective-C runs on other platforms than iOS and OS X).
I wonder if there are some ideas we can steal from the Python Objective-C bridge:
http://pythonhosted.org/pyobjc/
Marc
On Feb 27, 2013, at 9:29 AM, Marc Feeley feeley@iro.umontreal.ca wrote:
On 2013-02-27, at 10:19 AM, Jason Felice jason.m.felice@gmail.com wrote:
I'd be happy to unblackhole mine if it gets more people involved.
The x86_64 part is because ObjC doesn't provide a completely machine-abstract way to invoke dynamically. Each way has some icky flaws. My first goal was actually iOS, but I went down the x86_64 path because Mikael had a need for that first. I'd like the package to have x86_64/i386/arm.
I did discover that a LOT more machinery is needed for x86_64 than for i386 or arm. Calling conventions on 64-bit are crazy insane (though I now have the worst of it solved).
I agree that there are issues with all approaches. If possible I'd like to avoid using assembly language which is compiler and operating system dependent (Objective-C runs on other platforms than iOS and OS X).
I wonder if there are some ideas we can steal from the Python Objective-C bridge:
For a Lispier version, you could look at CCL's FFI, which is quite comprehensive, and which provides a lot of very nice conveniences for writing code that talks to Cocoa. For example:
(defclass converter (ns:ns-object) () (:metaclass ns:+ns-object))
(objc:defmethod (#/convertCurrency:atRate: :float) ((self converter) (currency :float) (rate :float)) (* currency rate))
Building CCL's FFI is largely automated using Gary Byers' ffigen, which is more or less a customized version of the gcc frontend. ffigen is freely available. It takes some googling to find it and discover how to use it, but I've done that before a few times; it's not too bad.
Alternatively, you could always make a tool to parse Apple's BridgeSupport files. Automating the generation of these interfaces is probably a good idea. You can pretty easily make a bridge that talks directly to the C API of the Objective-C runtime, and that gets you the power to do pretty much anything, but you're working at a very low level and have to do a lot of gritty stuff by hand. Building reasonable APIs at that level can be pretty time-consuming, and Cocoa is huge and growing.
When working with OSX and iOS from Gambit I've generally hand-built the interfaces I needed as I needed them. The disadvantage of that approach is that every nontrivial project requires me to hand-build some new Cocoa interfaces. The advantage is that the cost of doing that is no more than it has to be. Making a general Cocoa interface is a large task.
On Feb 27, 2013, at 9:50 AM, mikel evins mevins@me.com wrote:
For a Lispier version, you could look at CCL's FFI, which is quite comprehensive, and which provides a lot of very nice conveniences for writing code that talks to Cocoa. For example:
(defclass converter (ns:ns-object) () (:metaclass ns:+ns-object))
(objc:defmethod (#/convertCurrency:atRate: :float) ((self converter) (currency :float) (rate :float)) (* currency rate))
I think it's worth mentioning a couple of things that are made possible by the CCL FFI, and that Gambit users might want to consider.
With CCL's FFI you can make Lisp classes whose superclasses are Objective-C classes. You can write Lisp methods for Objective-C functions (or, rather, "messages"). You can create instances of Objective-C classes some of whose instance variables are Lisp values, and vice-versa: instances of Lisp classes, some of whose slots are Objective-C values. As shown in the example, you can call Objective-C methods directly, and you can refer to Objective-C variables and constants directly.
Again, almost all of the infrastructure is generated automatically, using ffigen together with parsers written in Lisp.
On Mar 1, 2013, at 2:33 AM, mikel evins mevins@me.com wrote:
I think it's worth mentioning a couple of things that are made possible by the CCL FFI, and that Gambit users might want to consider.
With CCL's FFI you can make Lisp classes whose superclasses are Objective-C classes. You can write Lisp methods for Objective-C functions (or, rather, "messages"). You can create instances of Objective-C classes some of whose instance variables are Lisp values, and vice-versa: instances of Lisp classes, some of whose slots are Objective-C values. As shown in the example, you can call Objective-C methods directly, and you can refer to Objective-C variables and constants directly.
Again, almost all of the infrastructure is generated automatically, using ffigen together with parsers written in Lisp.
I thought I'd just see how hard it was to use the current version of ffigen to generate Cocoa headers, as I haven't done it in a while. It was pretty easy; it took less than 15 minutes from start to finish. The longest phase was downloading the large piece of ffigen.
I've done it before several times, though.
Actually generating ffi interfaces to Cocoa took three quarters of a second.
Here are a few randomly-chosen samples of the output of ffigen, for illustration:
(function ("/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include/objc/message.h" 223) "objc_msgSendv_stret" (function ((pointer (void ())) (typedef "id") (typedef "SEL") (typedef "size_t") (typedef "marg_list") ) (void ())) (extern))
(macro ("/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include/objc/runtime.h" 364) "_C_ID" "'@'")
(enum-ident ("" 0) "OBJC_ASSOCIATION_ASSIGN" 0) (enum-ident ("" 0) "OBJC_ASSOCIATION_RETAIN_NONATOMIC" 1) (enum-ident ("" 0) "OBJC_ASSOCIATION_COPY_NONATOMIC" 3) (enum-ident ("" 0) "OBJC_ASSOCIATION_RETAIN" 769) (enum-ident ("" 0) "OBJC_ASSOCIATION_COPY" 771)
(struct ("" 0) "Protocol" (("isa" (field (typedef "Class") 0 8))))
(objc-instance-method ("/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObject.h" 158) "NSObject" ("NSDiscardableContentProxy") "autoContentAccessingProxy" () (typedef "id"))
(var ("/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include/objc/runtime.h" 593) "_zoneAlloc" (pointer (function ((typedef "Class") (typedef "size_t") (pointer (void ())) ) (typedef "id"))) (extern))
Interesting! Now the question is... can the forms generated by ffigen be transformed easily to Gambit's FFI (and if not, what can we change in the Gambit FFI to make it easy).
Marc
On 2013-03-06, at 9:16 PM, mikel evins mevins@me.com wrote:
On Mar 1, 2013, at 2:33 AM, mikel evins mevins@me.com wrote:
I think it's worth mentioning a couple of things that are made possible by the CCL FFI, and that Gambit users might want to consider.
With CCL's FFI you can make Lisp classes whose superclasses are Objective-C classes. You can write Lisp methods for Objective-C functions (or, rather, "messages"). You can create instances of Objective-C classes some of whose instance variables are Lisp values, and vice-versa: instances of Lisp classes, some of whose slots are Objective-C values. As shown in the example, you can call Objective-C methods directly, and you can refer to Objective-C variables and constants directly.
Again, almost all of the infrastructure is generated automatically, using ffigen together with parsers written in Lisp.
I thought I'd just see how hard it was to use the current version of ffigen to generate Cocoa headers, as I haven't done it in a while. It was pretty easy; it took less than 15 minutes from start to finish. The longest phase was downloading the large piece of ffigen.
I've done it before several times, though.
Actually generating ffi interfaces to Cocoa took three quarters of a second.
Here are a few randomly-chosen samples of the output of ffigen, for illustration:
(function ("/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include/objc/message.h" 223) "objc_msgSendv_stret" (function ((pointer (void ())) (typedef "id") (typedef "SEL") (typedef "size_t") (typedef "marg_list") ) (void ())) (extern))
(macro ("/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include/objc/runtime.h" 364) "_C_ID" "'@'")
(enum-ident ("" 0) "OBJC_ASSOCIATION_ASSIGN" 0) (enum-ident ("" 0) "OBJC_ASSOCIATION_RETAIN_NONATOMIC" 1) (enum-ident ("" 0) "OBJC_ASSOCIATION_COPY_NONATOMIC" 3) (enum-ident ("" 0) "OBJC_ASSOCIATION_RETAIN" 769) (enum-ident ("" 0) "OBJC_ASSOCIATION_COPY" 771)
(struct ("" 0) "Protocol" (("isa" (field (typedef "Class") 0 8))))
(objc-instance-method ("/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObject.h" 158) "NSObject" ("NSDiscardableContentProxy") "autoContentAccessingProxy" () (typedef "id"))
(var ("/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include/objc/runtime.h" 593) "_zoneAlloc" (pointer (function ((typedef "Class") (typedef "size_t") (pointer (void ())) ) (typedef "id"))) (extern))
On Mar 6, 2013, at 9:57 PM, Marc Feeley feeley@iro.umontreal.ca wrote:
Interesting! Now the question is... can the forms generated by ffigen be transformed easily to Gambit's FFI (and if not, what can we change in the Gambit FFI to make it easy).
In case you want to look at how CCL does it, the sources are in ccl/library/parse-ffi.lisp.
On Mar 6, 2013, at 9:57 PM, Marc Feeley feeley@iro.umontreal.ca wrote:
Interesting!
Since it's interesting, I figure it would be helpful if I told you the exact steps I used. Here they are:
1. Get ffigen itself.
This page tells you how to get it and build it:
http://trac.clozure.com/ccl/wiki/BuildFFIGEN
I used the version listed under the heading "(Experimental) ObjC blocks support".
2. Build ffigen
I simply followed the instructions under the abovementioned heading to obtain a working build. Please note that I had GCC 4.7.2 installed on my machine before starting; I use it to build Gambit, so I always have it installed.
3. Install ffigen
I followed the instructions on the ffigen page. That process installed ffigen in /usr/local, putting h-to-ffi.sh in /usr/local/bin, and some other files in /usr/local/ffigen. I added /usr/local/ffigen/bin to my PATH, though that may not have been necessary; /usr/local/bin was already on my PATH.
4. Make a directory for ffi files
ffigen grovels over C, C++, and Objective-C header files, generating many lines of s-expressions as output. It wants to write an .ffi file for each header file, and it wants to put them in a directory structure that parallels the directory structure where it finds the header files.
Make a directory somewhere convenient. In that directory, create a directory named "Cocoa", with a subdirectory named "C". In the "C" directory, put a file called "populate.sh" and put these lines in it:
#!/bin/sh rm -rf ./Applications SDK=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk CFLAGS="-m64 -isysroot ${SDK} -mmacosx-version-min=10.6"; export CFLAGS h-to-ffi.sh ${SDK}/usr/include/objc/objc-runtime.h h-to-ffi.sh ${SDK}/usr/include/objc/objc-exception.h h-to-ffi.sh ${SDK}/usr/include/objc/Object.h h-to-ffi.sh ${SDK}/usr/include/objc/Protocol.h h-to-ffi.sh ${SDK}/System/Library/Frameworks/Cocoa.framework/Headers/Cocoa.h
cd to that directory, and from there run the script. If all goes well, then less than a second later you should wind up with a new Application subdirectory. Buried inside it, in a deep directory hierarchy that parallels that of Apple's SDKs, you should find newly-generated .ffi files containing output similar to the examples I sent previously.
This script handles Cocoa.h and the Objective-C runtime APIs. There are lots more frameworks you might want to process. If you install the source release of CCL (not the prebuilt version that is on the App Store), you'll find that it includes header directories with "populate.sh" scripts for processing a bunch of useful frameworks--addressbook, audtiotoolbox, carbon, cocoa, coreaudiokit, gl, jni, libc, and so on.
If you succeed in following the steps I gave above, then you should see how you can write your own "populate.sh" scripts to process other frameworks, whether from Apple or from other sources.
Occasionally you may find some obscure corner case that ffigen doesn't quite handle correctly. In those cases, you can just write an FFI by hand. Even when such gaps do arise, ffigen saves a ton of time.
Now the question is... can the forms generated by ffigen be transformed easily to Gambit's FFI (and if not, what can we change in the Gambit FFI to make it easy).
I figure that, since they're just s-expression versions of the C declarations, processing them should be pretty straightforward. The Lisp source file I identified in my previous message shows how CCL turns the .ffi files into foreign definitions using CCL's. Naturally, Gambit's FFI is different, but I'm guessing that adapting ffigen's output will not be onerous. One of several advantages to using ffigen is that it's already seen many years of duty serving this same purpose for CCL, which has a very comprehensive and featureful Cocoa bridge as a result. With luck, maybe it can help Gambit do the same.
--me
Marc/Jason
I haven't yet looked at Jason's work, BUT... I did scratch a bit at Python<->ObjC.
The general approach there is to use a dynamic call library. libffi seems to be the one of (current) choice, although my direct experience is with libffcall (from Bruno Hable's CLISP). libffi supports everything from 386 and embedded through to S390, I've started playing around with the library; the ObjC stuff is basically C call convention, and is directly supported. I'll try to wire in libffi in a reasonable way (always, depends on my time and availability). Right now, all I can say is that I like ffcall a bit more -- but it appears "crusty",
If wired in correctly, this should provide not only an ObjC bridge, but a way to build dynamic links to Linux/Unix shared objects (dlopen/dlsym and then build a trampoline to the C function through libffi). Of course if binding C libraries directly, it is just as easy to generate the glue code statically from a description, but using libffi should allow lifting of the "hard bits" from Python.
Still scratching.
Fred Weigel
On Wed, 2013-02-27 at 10:29 -0500, Marc Feeley wrote:> I'd be happy to unblackhole mine if it gets more people involved.
The x86_64 part is because ObjC doesn't provide a completely machine-abstract way to invoke dynamically. Each way has some icky flaws. My first goal was actually iOS, but I went down the x86_64 path because Mikael had a need for that first. I'd like the package to have x86_64/i386/arm.
I did discover that a LOT more machinery is needed for x86_64 than for i386 or arm. Calling conventions on 64-bit are crazy insane (though I now have the worst of it solved).
I agree that there are issues with all approaches. If possible I'd like to avoid using assembly language which is compiler and operating system dependent (Objective-C runs on other platforms than iOS and OS X).
I wonder if there are some ideas we can steal from the Python Objective-C bridge:
http://pythonhosted.org/pyobjc/
Marc
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
I guess this is such a fundamental thing that relating to it primarily as a vanilla Gambit project. Then, adding cond-expand:s for it to hybridly support BH is piece of cake. Brgds
2013/2/27 Jason Felice jason.m.felice@gmail.com
I'd be happy to unblackhole mine if it gets more people involved.
I've been thinking about this thread and my project, and it has re-energized me to put some time in. I'm hoping this evening I will have that time to put in.
Some things which I've concluded:
1. The dependency on libffi is fine for all platforms I know of except for iOS on the native device. I'll investigate using libffi for most platforms and special-case ARM support. If feasible (I expect it will be), that should remove a boat load of code. 2. I'll make it not require black hole.
-Jason
On Wed, Feb 27, 2013 at 9:58 AM, Mikael mikael.rcv@gmail.com wrote:
I guess this is such a fundamental thing that relating to it primarily as a vanilla Gambit project. Then, adding cond-expand:s for it to hybridly support BH is piece of cake. Brgds
2013/2/27 Jason Felice jason.m.felice@gmail.com
I'd be happy to unblackhole mine if it gets more people involved.
On 2013-02-28, at 10:59 AM, Jason Felice jason.m.felice@gmail.com wrote:
I've been thinking about this thread and my project, and it has re-energized me to put some time in. I'm hoping this evening I will have that time to put in.
Some things which I've concluded:
- The dependency on libffi is fine for all platforms I know of except for iOS on the native device. I'll investigate using libffi for most platforms and special-case ARM support. If feasible (I expect it will be), that should remove a boat load of code.
- I'll make it not require black hole.
Couldn't we use Mike Evins' idea of parsing the BridgeSupport files and generating specific FFI code which covers all the available methods? This approach would be completely portable.
To avoid code bloat, we could rely on an "import" declaration which indicates which of the methods we actually need (perhaps inferred from a static program analysis).
Marc
On Feb 28, 2013, at 1:36 PM, Marc Feeley feeley@iro.umontreal.ca wrote:
On 2013-02-28, at 10:59 AM, Jason Felice jason.m.felice@gmail.com wrote:
I've been thinking about this thread and my project, and it has re-energized me to put some time in. I'm hoping this evening I will have that time to put in.
Some things which I've concluded:
- The dependency on libffi is fine for all platforms I know of except for iOS on the native device. I'll investigate using libffi for most platforms and special-case ARM support. If feasible (I expect it will be), that should remove a boat load of code.
- I'll make it not require black hole.
Couldn't we use Mike Evins' idea of parsing the BridgeSupport files and generating specific FFI code which covers all the available methods? This approach would be completely portable.
To avoid code bloat, we could rely on an "import" declaration which indicates which of the methods we actually need (perhaps inferred from a static program analysis).
BridgeSupport files are XML. Parsing them is easy. Do `man BridgeSupport` to find out more.
The man page for BridgeSupport claims that it describes API symbols which cannot be introspected at runtime. It also says, "Only classes where additional metadata is needed are described."
It seems the additional metadata is what kind of sentinel is used for variable-argument methods and which classes are toll-free bridged to which other types.
Also, no bridgesupport files appear for the iOS or iOS simulator platforms, though there is a tool which might generate this information based on headers.
IMHO, this does not solve the method dispatch problem, but it can add a lot of value for Mac OS X.
For my purposes - which is iOS development - I want to be able to call third-party class methods and even undocumented methods (in the simulator), so I'd like to keep the dynamic dispatch.
As an optimization on iOS, it would be cool to run some stats on the most commonly called function signatures and fast path those.
-Jason
On Thu, Feb 28, 2013 at 1:46 PM, mikel evins mevins@me.com wrote:
On Feb 28, 2013, at 1:36 PM, Marc Feeley feeley@iro.umontreal.ca wrote:
On 2013-02-28, at 10:59 AM, Jason Felice jason.m.felice@gmail.com
wrote:
I've been thinking about this thread and my project, and it has
re-energized me to put some time in. I'm hoping this evening I will have that time to put in.
Some things which I've concluded:
- The dependency on libffi is fine for all platforms I know of except
for iOS on the native device. I'll investigate using libffi for most platforms and special-case ARM support. If feasible (I expect it will be), that should remove a boat load of code.
- I'll make it not require black hole.
Couldn't we use Mike Evins' idea of parsing the BridgeSupport files and
generating specific FFI code which covers all the available methods? This approach would be completely portable.
To avoid code bloat, we could rely on an "import" declaration which
indicates which of the methods we actually need (perhaps inferred from a static program analysis).
BridgeSupport files are XML. Parsing them is easy. Do `man BridgeSupport` to find out more.
On Feb 28, 2013, at 2:41 PM, Jason Felice jason.m.felice@gmail.com wrote:
The man page for BridgeSupport claims that it describes API symbols which cannot be introspected at runtime. It also says, "Only classes where additional metadata is needed are described."
It seems the additional metadata is what kind of sentinel is used for variable-argument methods and which classes are toll-free bridged to which other types.
Also, no bridgesupport files appear for the iOS or iOS simulator platforms, though there is a tool which might generate this information based on headers.
IMHO, this does not solve the method dispatch problem, but it can add a lot of value for Mac OS X.
For my purposes - which is iOS development - I want to be able to call third-party class methods and even undocumented methods (in the simulator), so I'd like to keep the dynamic dispatch.
As an optimization on iOS, it would be cool to run some stats on the most commonly called function signatures and fast path those.
It's been a while since the last time I parsed BridgeSupport files (in Common Lisp that time), but if I remember correctly, you can use gen_bridge_metadata --framework FooBar to generate the BridgeSupport files for the FooBar framework, whether it's an OSX framework, or an iOS framework, or even some random third-party framework like CHDataStructures or something. One advantage of doing that is that it's an automated tool that gives you the APIs that Apple is claiming are the canonical ones for any given release of its system.
I doesn't always give you everything you need, but most of Cocoa automatically plus a little bit by hand is still a lot less work than all of Cocoa by hand.
ffigen does much the same thing, with a couple of notable differences. One is that it generates interface information for everything you point it at, whether Apple thinks you should use it or not. Another difference is that it parses C, C++, and Objective-C headers into s-expressions, which might be more congenial to work with than XML. It's primarily used to generate the (very good) Objective-C bridge for CCL, but there's no reason you couldn't use it for the same purpose with Gambit.
If that interests you, you can find a page about how to build and install ffigen here:
http://trac.clozure.com/ccl/wiki/BuildFFIGEN
...and there's a page about how to use it with CCL here:
http://ccl.clozure.com/manual/chapter12.7.html
If you decided that you did want to try doing things the way CCL does them, you'd probably want to download CCL and rummage around in its sources. That's pretty easy to do from either a SLIME session or the Cocoa IDE supplied with CCL. For example, I just launched the Cocoa app, typed in (require :parse-ffi), then ccl::parse-standard-ffi-files (actually, ccl::parse-s and then Option-/ to complete), and then stuck the insertion point in the symbol name and hit Option-period. CCL obligingly popped open a window with this in it:
(defun parse-standard-ffi-files (dirname &optional target) (let* ((backend (if target (find-backend target) *target-backend*)) (ftd (backend-target-foreign-type-data backend)) (*parse-ffi-target-ftd* ftd) (*target-ftd* ftd) (*target-backend* backend) (d (use-interface-dir dirname ftd)) (interface-dir (merge-pathnames (interface-dir-subdir d) (ftd-interface-db-directory ftd))) (*prepend-underscores-to-ffi-function-names* (getf (ftd-attributes ftd) :prepend-underscores)) (*ffi-global-typedefs* (make-hash-table :test 'string= :hash-function 'sxhash)) (*ffi-global-unions* (make-hash-table :test 'string= :hash-function 'sxhash)) (*ffi-global-transparent-unions* (make-hash-table :test 'string= :hash-function 'sxhash)) (*ffi-global-structs* (make-hash-table :test 'string= :hash-function 'sxhash)) (*ffi-global-objc-classes* (make-hash-table :test 'string= :hash-function 'sxhash)) (*ffi-global-objc-messages* (make-hash-table :test 'string= :hash-function 'sxhash)) (*ffi-global-functions* (make-hash-table :test 'string= :hash-function 'sxhash)) (*ffi-global-constants* (make-hash-table :test 'string= :hash-function 'sxhash)) (*ffi-global-vars* (make-hash-table :test 'string= :hash-function 'sxhash)))
(But for this purpose, you would want to download it from http://ccl.clozure.com/download.html, rather than getting the free App Store version, so that you have the sources handy.)
--me