[gambit-list] trying to make Blackhole work with Android

Mikael mikael.rcv at gmail.com
Thu Jan 12 18:25:13 EST 2012


Hi Alvaro,

Ok, I got your usecase; you have a Scheme app, that you want to compile on
your Linux box, to binaries for and runnable on Android.

You want debug symbols to be bundled so you can do meaningful debugging on
it, but you don't want to bundle BH itself, and thus you're ok with that
macro expansion for macros you defined in your code, will not work on the
Android (within eval, and from the REPL).

Furthermore, you got Gambit 'raw' successfully produce binaries that run
well on Android, so your issue is making BH do the same. This ought to be
pretty straight forward, as the complexity that BH introduces at the level
of C compilation and linking as to get your app running is pretty much zero.

Within your use of the syntactic tower branch of BH and given the use
above, bundle only the RT files. This saves you any complexity involved in
getting the additional three files per module compiled and running also;
given that you won't use any macro expansion or other BH functionality in
target environment, you don't need it anyhow.

The BH commands for producing executables straight off are to be documented
soon, there should be ways to do this based on module-compile! only though,
this is what I address here.

Also, just to ensure that your use of c-define does not cause any problem;
while it's still on Per's desk to fix this bug, minor as it is indeed,
could you switch to the non-syntactic tower branch where c-define works?

I'd advise you to do the following:

Extend your test of using 'raw' Gambit to compile Android binaries, to
include the use of any external C libraries that you use. I.e., include a
c-define to each of those libraries, in the code.
By this test, you ensure that you have all functionality regarding the
production of Android binaries functioning at the level of Gambit, thus
isolating any errors to BH only/in itself.
Also, during this test, dump all C compiler and linker invocations made by
Gambit by setting GAMBC_CC_VERBOSE , as to get a reference behavior that
you want BH to adapt.


When this is done, making BH produce the desired executables should be very
straightforward; the most basic approach would be to make BH compile your
app's modules to C files using (module-compile! module-name to-c: #t), then
produce any link file needed using link-flat, and then do the C compilation
and linking yourself.


I hope this was sufficient for you to get a working solution to production
of Android binaries. Please keep the ML posted on your progress; if you got
it working, please tell how, and if something still needs to be made work,
please include a minimal Scheme code example, dumps of any Gambit and BH
use you did, including full paths and arguments with any C compiler and
linker invocations.

Find below some notes from while reading through your email.

Mikael



2012/1/10 Álvaro Castro-Castilla <alvaro.castro.castilla at gmail.com>

> Hi,
>
> First, before making BH work on a particular platform, doublecheck that
>> Gambit works on it in itself. As you're aware, this is a pre-requirement.
>> Does Gambit work on Android, for you? If so, what configuration did you
>> use?
>>
>
> Yes, it works in Gambit itself.
>

Super.


> I'm using ant. You can check the project here:
> https://github.com/alvatarc/bh-android-template/blob/master/jni/main.scm
>
>
>
>> What's your current goal, to get a complete environment with compilation
>> and all running on the Android, or solely to make your Linux machine
>> produce binaries runnable on the Android (including a REPL with
>> interpreter)?
>>
>>
> I want to produce binaries to execute in Android. Cross-platform
> development.
>
>
>
>>
>> For resolving any compilation-related issues, you want Gambit to be
>> verbose to you about exactly what C compiler and linker invocations it
>> does, including all parameters passed.
>>
>> Gambit invokes the C compiler and linker from the gambc-cc shellscript,
>> and this script is verbose - outputs any C compiler and linker calls to the
>> console - if it's run with the environment variable GAMBC_CC_VERBOSE set.
>>
>> Thus, when starting the gsc you'll use for producing Android binaries,
>> run it like this from bash: GAMBC_CC_VERBOSE=yes gsc
>>
>>
>> Now zooming in a bit on BH:
>>
>> I'm generating all 5 files like BH usually does.
>>> These 4 files are the part of Blackhole I understand least, but they are
>>> all needed for the module to be defined in C. According to Per:
>>>
>>> "When compiling modules for use in development (as opposed to "bh exe"),
>>> Black Hole works internally by compiling every module into *4* Gambit
>>> modules (c files if you will):
>>>
>>> 1) runtime code
>>> 2) compiletime code (this is essentially the same code as the runtime
>>> code, but does not use the global namespace and is thus slower but it's
>>> required to be able to create an unbound number of instances of the same
>>> module at any given time)
>>> 3) visit-time code (these are the macros)
>>> 4) module metadata (this is pretty much a compiled a-list)
>>>
>>> If you want to retain the development information, you will need 4 .c
>>> files per Black Hole module. If you want to strip development information,
>>> you will only need one (the runtime code)."
>>>
>>> I'm trying both with the 4 files and the only one option.
>>>
>>
>> This is a good time to clarify, what's your goal within the following
>> scope:
>>
>> Basically, for code execution, only the runtime code C file is needed.
>> (view this as the 'executable code' file.)
>>
>>
> Yes, I'm aware of that, but if only this one is included, then you get one
> symbol undefined per each one of the others, plus the same multiple
> definitions.
>
>
>
>> The total set of four files is needed to get a complete BH dev
>> environment running. The reason for more than one file, is because BH needs
>> to have several instances of each module "at hand", in order to get the
>> macro expansion spinning the right way. The syntactic tower version of BH
>> implements (exactly that;) the syntactic tower, appropriately and
>> correctly, and this requires more than one instance of the module to be
>> done the right way.
>>
>> Thus, if your objective is to generate code that should be executable on
>> the target environment and that's it, go with one file, otherwise go with
>> all four.
>>
>>
> Ideally, I only need the rt version, but I still need to check if there is
> interest in including the others (when debugging code for android maybe? I
> can't tell).
>

Debugging as in getting debug info kept in the binary and having a good
debugging environment, you'll get with only the rt version.

The other three files are there to make BH do macro expansion correctly.

So, if you skip the combo of loading the three latter files + don't load
BH, you won't get any new macro expansion on the target platform.

The ('older') non-syntactic tower version of BH generates only one .o/.c
file per module btw.


>   (A)
>>
>>> The multiple definition of symbols happens in both situations, but
>>>
>> (B)
>>
>>> an undefined reference happens for each one of the 3 files (I think that
>>> is due to the link file referencing them, I could take care of that). But
>>> the core of the problem is with the multiple definition
>>> of ___G_list, ___S_list, ___G_void and ___S_void
>>> How can this happen?
>>>
>>
>> Is your point that (A) happens for either of the four files, while for
>> the last three files (B) happens as well?
>>
>>
>  Yes, always.
>
> For instance, there are a couple of needed changes for
>
> (c-define (c-draw) () void "draw_scheme_hook" "" (~#draw-hook))
> -this one is due to the bug in blackhole when expanding c-define-
> and:
>
> #define LINKER ____20_main_2e_o
> instead of
> #define LINKER ____20_main__
>
> but the rest of main.scm is exactly the same code, only that Gambit
> vanilla generates two files that work well (main.c and the link file
> main_.c).
>
>
> Producing and compiling with link files is indeed a bit tricky. To the
>> best of my knowledge, the procedure to follow is something like this:
>>
>> Make BH output C code:
>>
>> (module-compile! 'yourmod to-c: t)
>>
>> Now, for each of the C files you want to compile:
>>
>> (link-flat '("yourmod.exact-c-file-filename") output: "
>> yourmod.exact-c-file-filename.o1.c")
>>
>>
> That's how BH is doing it, and how I'm doing it.
> You can check my fork of Blackhole, that allows to compile to C via the
> command-line:
> bh compile -C module.scm
> (it uses the modern syntactic tower version, but it has a few bugs)
> Currently it doesn't work if you use forms that force compilation, such as
> any FFI c-define, c-declare etc. But for testing purposes a module with
> (define (double x) (* 2 x)) suffices.
>
>
>
>> Then compile:
>>
>> (include the other arguments your gsc passes to gcc, as per the advice at
>> the top)
>> gcc yourmod.exact-c-file-filename yourmod.exact-c-file-filename.o1.c -o "
>> yourmod.exact-c-file-filename.o1"
>>
>> you could give a shot with:
>>
>> g++    -Wall -W -Wno-unused -O1 -fno-math-errno -fschedule-insns2 -O1
>> -fno-math-errno -fschedule-insns2 -fno-trapping-math -fno-strict-aliasing
>> -fwrapv -fomit-frame-pointer -fPIC -fno-common -mieee-fp -rdynamic -shared
>> -I"/usr/include/"  -D___DYNAMIC yourmod.exact-c-file-filename
>> yourmod.exact-c-file-filename.o1.c -o "yourmod.exact-c-file-filename.o1"
>>
>>
> That works, but that's expected: the issue with multiple symbol definition
> happens against libgambc.a, which you don't link here. But it is needed for
> any standalone executable, and that's when it fails.
>

Can you please paste here
 (A) a complete list of all the C compiler & linker invocations Gambit
does/needed to produce a working binary on your machine, and,
 (B) a complete list of all the environment variables set, BH commands, and
C compilier & linker invocations you do when trying to get your BH app
compiled to Android?

Also paste their complete error output, if any.


If I get the manual right, (link-flat) takes a _Scheme input file_ and
makes a C link file for that Scheme module.

In BH, I think for the three latter files, BH does not save a copy of the
scheme code anywhere but just throws it away, and without those files you
won't be able to make correct link files.


I must admit that I'm not up to date on how to make binaries with BH. I
think there's a procedure in BH that is supposed to do this for you,
compile-to-binary or sth like this. Do you know of it, did you try to use
it?

In all cases, if you have the C code, you should easily enough be able to
produce a binary, based on the RT files only.



>  I've made a quick README, in case you want to test the Android template
> I'm working on.
>
> I'll further check any possible differences between BH-generated files and
> Gambit/vanilla-generated files.
> Thanks a lot for your help and interest.
>
> Best regards,
>
> Álvaro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20120113/a24830d1/attachment.htm>


More information about the Gambit-list mailing list