Hi Alvaro,

First, before making BH work on a particular platform, doublecheck that Gambit works on it in itself. As you're aware, this is a pre-requirement.

Does Gambit work on Android, for you? If so, what configuration did you use?

What's your current goal, to get a complete environment with compilation and all running on the Android, or solely to make your Linux machine produce binaries runnable on the Android (including a REPL with interpreter)?

For resolving any compilation-related issues, you want Gambit to be verbose to you about exactly what C compiler and linker invocations it does, including all parameters passed.

Gambit invokes the C compiler and linker from the gambc-cc shellscript, and this script is verbose - outputs any C compiler and linker calls to the console - if it's run with the environment variable GAMBC_CC_VERBOSE set.

Thus, when starting the gsc you'll use for producing Android binaries, run it like this from bash: GAMBC_CC_VERBOSE=yes gsc

Now zooming in a bit on BH:

I'm generating all 5 files like BH usually does.

These 4 files are the part of Blackhole I understand least, but they are all needed for the module to be defined in C. According to Per:

"When compiling modules for use in development (as opposed to "bh exe"), Black Hole works internally by compiling every module into *4* Gambit modules (c files if you will):

1. runtime code
2. compiletime code (this is essentially the same code as the runtime

code, but does not use the global namespace and is thus slower but it's required to be able to create an unbound number of instances of the same module at any given time) 3) visit-time code (these are the macros) 4) module metadata (this is pretty much a compiled a-list)

If you want to retain the development information, you will need 4 .c files per Black Hole module. If you want to strip development information, you will only need one (the runtime code)."

I'm trying both with the 4 files and the only one option.

This is a good time to clarify, what's your goal within the following scope:

Basically, for code execution, only the runtime code C file is needed. (view this as the 'executable code' file.)

The total set of four files is needed to get a complete BH dev environment running. The reason for more than one file, is because BH needs to have several instances of each module "at hand", in order to get the macro expansion spinning the right way. The syntactic tower version of BH implements (exactly that;) the syntactic tower, appropriately and correctly, and this requires more than one instance of the module to be done the right way.

Thus, if your objective is to generate code that should be executable on the target environment and that's it, go with one file, otherwise go with all four.

Which of these two - exec only or complete dev environment - is your current usecase?

Now into details about your issue:

(A)

The multiple definition of symbols happens in both situations, but

(B)

an undefined reference happens for each one of the 3 files (I think that is due to the link file referencing them, I could take care of that). But the core of the problem is with the multiple definition of ___G_list, ___S_list, ___G_void and ___S_void How can this happen?

Is your point that (A) happens for either of the four files, while for the last three files (B) happens as well?

Producing and compiling with link files is indeed a bit tricky. To the best of my knowledge, the procedure to follow is something like this:

Make BH output C code:

(module-compile! 'yourmod to-c: t)

Now, for each of the C files you want to compile:

(link-flat '("yourmod.exact-c-file-filename") output: " yourmod.exact-c-file-filename.o1.c")

Then compile:

(include the other arguments your gsc passes to gcc, as per the advice at the top) gcc yourmod.exact-c-file-filename yourmod.exact-c-file-filename.o1.c -o " yourmod.exact-c-file-filename.o1"

you could give a shot with:

g++ -Wall -W -Wno-unused -O1 -fno-math-errno -fschedule-insns2 -O1 -fno-math-errno -fschedule-insns2 -fno-trapping-math -fno-strict-aliasing -fwrapv -fomit-frame-pointer -fPIC -fno-common -mieee-fp -rdynamic -shared -I"/usr/include/" -D___DYNAMIC yourmod.exact-c-file-filename yourmod.exact-c-file-filename.o1.c -o "yourmod.exact-c-file-filename.o1"

I attach the generated files. In order to try it, you can just use: g++ test.o1.c test-ct.c test-rt.c test-vt.c test-mi.c -lm -lutil -ldl -lgambc -o test.o ...and you'll get an (expected) undefined reference to main and all the multiple definitions I talk about.

Check with the suggested g++ use above, any improvement?

Best regards

Mikael

On Sun, Jan 8, 2012 at 8:00 PM, Mikael mikael.rcv@gmail.com wrote:

...

As to make BH compile to C rather than object files, pass to-c: #t to module-compile! .

What's the first 4 of the 5 files you mention below? (Format, content, when needed / scope of use, and so on)

The link file is produced by BH at some point (presuming by link file you mean link file as per Gambit's definition). To make a C link file out of Gambit, I don't remember, either there's a special BH procedure for it, or you do it the general way it's done in Gambit, please let the ml know what you got to, and if it's not obvious please query again.

Mikael

2012/1/8 Álvaro Castro-Castilla alvaro.castro.castilla@gmail.com

...

Hi!

I'm trying (again) to make Blackhole work with Android. For that purpose I need to make it compile to C, instead of its normal *.o files. BH needs to compile 5 files for each module:

• a compile-time file
• a real-time file
• a visit-time file
• a module metadata file
• the link file

Based on https://github.com/seoushi/gambit-android-example but using the newer "Native Activity" I'm trying to compile an example. First, I'm trying to compile an example with just one file, importing no modules.

I'm getting: [exec] ./jni/ext/libgambc.a(_gambc.o):(.data+0x4854c): multiple definition of `___G_list' [exec] ./obj/local/armeabi/objs/native-activity/main.o1.o:(.data+0x3c): first defined here [exec] ./jni/ext/libgambc.a(_gambc.o):(.data.rel+0x13708): multiple definition of `___S_list' [exec] ./obj/local/armeabi/objs/native-activity/main.o1.o:(.data.rel+0xb0): first defined here [exec] ./jni/ext/libgambc.a(_gambc.o):(.data+0x5ae40): multiple definition of `___G_void' [exec] ./obj/local/armeabi/objs/native-activity/main.o1.o:(.data+0x54): first defined here [exec] ./jni/ext/libgambc.a(_gambc.o):(.data.rel+0x19138): multiple definition of `___S_void' [exec] ./obj/local/armeabi/objs/native-activity/main.o1.o:(.data.rel+0xe0): first defined here

I would like to know if any of you had similar experiences, and why I'm getting this multiple definitions? The link file (called main.o1.c) seems to redefine these symbols, but they are also defined in libgambc.a (which is compiled for ARM architecture) Why could this be happening?

I'd appreciate any pointers. Thanks a lot for your help,

Álvaro Castro-Castilla

---

Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list

Hi Alvaro,

Ok, I got your usecase; you have a Scheme app, that you want to compile on your Linux box, to binaries for and runnable on Android.

You want debug symbols to be bundled so you can do meaningful debugging on it, but you don't want to bundle BH itself, and thus you're ok with that macro expansion for macros you defined in your code, will not work on the Android (within eval, and from the REPL).

Furthermore, you got Gambit 'raw' successfully produce binaries that run well on Android, so your issue is making BH do the same. This ought to be pretty straight forward, as the complexity that BH introduces at the level of C compilation and linking as to get your app running is pretty much zero.

Within your use of the syntactic tower branch of BH and given the use above, bundle only the RT files. This saves you any complexity involved in getting the additional three files per module compiled and running also; given that you won't use any macro expansion or other BH functionality in target environment, you don't need it anyhow.

The BH commands for producing executables straight off are to be documented soon, there should be ways to do this based on module-compile! only though, this is what I address here.

Also, just to ensure that your use of c-define does not cause any problem; while it's still on Per's desk to fix this bug, minor as it is indeed, could you switch to the non-syntactic tower branch where c-define works?

I'd advise you to do the following:

Extend your test of using 'raw' Gambit to compile Android binaries, to include the use of any external C libraries that you use. I.e., include a c-define to each of those libraries, in the code. By this test, you ensure that you have all functionality regarding the production of Android binaries functioning at the level of Gambit, thus isolating any errors to BH only/in itself. Also, during this test, dump all C compiler and linker invocations made by Gambit by setting GAMBC_CC_VERBOSE , as to get a reference behavior that you want BH to adapt.

When this is done, making BH produce the desired executables should be very straightforward; the most basic approach would be to make BH compile your app's modules to C files using (module-compile! module-name to-c: #t), then produce any link file needed using link-flat, and then do the C compilation and linking yourself.

I hope this was sufficient for you to get a working solution to production of Android binaries. Please keep the ML posted on your progress; if you got it working, please tell how, and if something still needs to be made work, please include a minimal Scheme code example, dumps of any Gambit and BH use you did, including full paths and arguments with any C compiler and linker invocations.

Find below some notes from while reading through your email.

Mikael

2012/1/10 Álvaro Castro-Castilla alvaro.castro.castilla@gmail.com

Hi,

First, before making BH work on a particular platform, doublecheck that

...

Gambit works on it in itself. As you're aware, this is a pre-requirement. Does Gambit work on Android, for you? If so, what configuration did you use?

Yes, it works in Gambit itself.

Super.

I'm using ant. You can check the project here: https://github.com/alvatarc/bh-android-template/blob/master/jni/main.scm

...

What's your current goal, to get a complete environment with compilation and all running on the Android, or solely to make your Linux machine produce binaries runnable on the Android (including a REPL with interpreter)?

I want to produce binaries to execute in Android. Cross-platform development.

...

For resolving any compilation-related issues, you want Gambit to be verbose to you about exactly what C compiler and linker invocations it does, including all parameters passed.

Gambit invokes the C compiler and linker from the gambc-cc shellscript, and this script is verbose - outputs any C compiler and linker calls to the console - if it's run with the environment variable GAMBC_CC_VERBOSE set.

Thus, when starting the gsc you'll use for producing Android binaries, run it like this from bash: GAMBC_CC_VERBOSE=yes gsc

Now zooming in a bit on BH:

I'm generating all 5 files like BH usually does.

...

These 4 files are the part of Blackhole I understand least, but they are all needed for the module to be defined in C. According to Per:

"When compiling modules for use in development (as opposed to "bh exe"), Black Hole works internally by compiling every module into *4* Gambit modules (c files if you will):

1. runtime code
2. compiletime code (this is essentially the same code as the runtime

code, but does not use the global namespace and is thus slower but it's required to be able to create an unbound number of instances of the same module at any given time) 3) visit-time code (these are the macros) 4) module metadata (this is pretty much a compiled a-list)

If you want to retain the development information, you will need 4 .c files per Black Hole module. If you want to strip development information, you will only need one (the runtime code)."

I'm trying both with the 4 files and the only one option.

This is a good time to clarify, what's your goal within the following scope:

Basically, for code execution, only the runtime code C file is needed. (view this as the 'executable code' file.)

Yes, I'm aware of that, but if only this one is included, then you get one symbol undefined per each one of the others, plus the same multiple definitions.

...

The total set of four files is needed to get a complete BH dev environment running. The reason for more than one file, is because BH needs to have several instances of each module "at hand", in order to get the macro expansion spinning the right way. The syntactic tower version of BH implements (exactly that;) the syntactic tower, appropriately and correctly, and this requires more than one instance of the module to be done the right way.

Thus, if your objective is to generate code that should be executable on the target environment and that's it, go with one file, otherwise go with all four.

Ideally, I only need the rt version, but I still need to check if there is interest in including the others (when debugging code for android maybe? I can't tell).

Debugging as in getting debug info kept in the binary and having a good debugging environment, you'll get with only the rt version.

The other three files are there to make BH do macro expansion correctly.

So, if you skip the combo of loading the three latter files + don't load BH, you won't get any new macro expansion on the target platform.

The ('older') non-syntactic tower version of BH generates only one .o/.c file per module btw.

(A)

...

...

The multiple definition of symbols happens in both situations, but

(B)

...

an undefined reference happens for each one of the 3 files (I think that is due to the link file referencing them, I could take care of that). But the core of the problem is with the multiple definition of ___G_list, ___S_list, ___G_void and ___S_void How can this happen?

Is your point that (A) happens for either of the four files, while for the last three files (B) happens as well?

Yes, always.

For instance, there are a couple of needed changes for

(c-define (c-draw) () void "draw_scheme_hook" "" (~#draw-hook)) -this one is due to the bug in blackhole when expanding c-define- and:

#define LINKER ____20_main_2e_o instead of #define LINKER ____20_main__

but the rest of main.scm is exactly the same code, only that Gambit vanilla generates two files that work well (main.c and the link file main_.c).

Producing and compiling with link files is indeed a bit tricky. To the

...

best of my knowledge, the procedure to follow is something like this:

Make BH output C code:

(module-compile! 'yourmod to-c: t)

Now, for each of the C files you want to compile:

(link-flat '("yourmod.exact-c-file-filename") output: " yourmod.exact-c-file-filename.o1.c")

That's how BH is doing it, and how I'm doing it. You can check my fork of Blackhole, that allows to compile to C via the command-line: bh compile -C module.scm (it uses the modern syntactic tower version, but it has a few bugs) Currently it doesn't work if you use forms that force compilation, such as any FFI c-define, c-declare etc. But for testing purposes a module with (define (double x) (* 2 x)) suffices.

...

Then compile:

(include the other arguments your gsc passes to gcc, as per the advice at the top) gcc yourmod.exact-c-file-filename yourmod.exact-c-file-filename.o1.c -o " yourmod.exact-c-file-filename.o1"

you could give a shot with:

g++ -Wall -W -Wno-unused -O1 -fno-math-errno -fschedule-insns2 -O1 -fno-math-errno -fschedule-insns2 -fno-trapping-math -fno-strict-aliasing -fwrapv -fomit-frame-pointer -fPIC -fno-common -mieee-fp -rdynamic -shared -I"/usr/include/" -D___DYNAMIC yourmod.exact-c-file-filename yourmod.exact-c-file-filename.o1.c -o "yourmod.exact-c-file-filename.o1"

That works, but that's expected: the issue with multiple symbol definition happens against libgambc.a, which you don't link here. But it is needed for any standalone executable, and that's when it fails.

Can you please paste here (A) a complete list of all the C compiler & linker invocations Gambit does/needed to produce a working binary on your machine, and, (B) a complete list of all the environment variables set, BH commands, and C compilier & linker invocations you do when trying to get your BH app compiled to Android?

Also paste their complete error output, if any.

If I get the manual right, (link-flat) takes a _Scheme input file_ and makes a C link file for that Scheme module.

In BH, I think for the three latter files, BH does not save a copy of the scheme code anywhere but just throws it away, and without those files you won't be able to make correct link files.

I must admit that I'm not up to date on how to make binaries with BH. I think there's a procedure in BH that is supposed to do this for you, compile-to-binary or sth like this. Do you know of it, did you try to use it?

In all cases, if you have the C code, you should easily enough be able to produce a binary, based on the RT files only.

I've made a quick README, in case you want to test the Android template I'm working on.

I'll further check any possible differences between BH-generated files and Gambit/vanilla-generated files. Thanks a lot for your help and interest.

Best regards,

Álvaro