Bug on OS X, due to linker's incomplete support of thread local storage class
I tried to use Gambit-C with the 'multiple-vms’ option enabled, to harness full cpu power by SMP. But ‘compile-file’ on REPL failed with the message: ld: illegal thread local variable reference to regular symbol ____tls_ptr for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) After some experiments, it seems that the OS X linker does not properly support dynamic linking of externally defined TLS variables. This can be verified easily with the following code. // test.c extern __thread int tls_id; int get_tls_id() { return tls_id; } On OS X, both gcc and clang failed to produce a dynamically loadable object file. (OS X 10.10.1, Xcode 6.2, clang-600.0.57; gcc-4.8 intalled from homebrew.) $ gcc-4.8 -shared test.c -o libfoo.so $ gcc-4.9 -shared test.c -o libfoo.so $ clang -bundle test.c -o libfoo.so The three give the same error: Undefined symbols for architecture x86_64: "_tls_num", referenced from: _get_tls_num in test-9d6352.o (maybe you meant: _get_tls_num) ld: symbol(s) not found for architecture x86_64 While on Linux (ArchLinux 3.18.6 X86_64, gcc 4.9.2), it worked. The produced library linked fine with main.c (as follows), execution gives 1. // main.c #include <stdio.h> __thread int tls_id; extern int get_tls_id(); int main () { int id = 0; tls_id = 1; id = get_tls_id(); printf(“tls id is %d\n”, id); return 0; } I also tried to fallback on the pthread_get/setspecific() approach, by manually change CONF_THREAD_LOCAL_STORAGE_CLASS in configure/configure.ac. Well, gsc/gsi crashes on running, due to a memory related bug. I am fairly new to Gambit, hope someone more familiar with its source would help to pin this bug. If configures as enable-multiple-threaded-vm, then it crashes in alloc_scmobj_still(): (lldb) target create "gsc" Current executable set to 'gsc' (x86_64). (lldb) r Process 96941 launched: '/Users/whimse/Work/MobileGames/Rabbit/gambit/gsc/gsc' (x86_64) Process 96941 stopped * thread #1: tid = 0x6176f, 0x0000000100014399 gsc`alloc_scmobj_still(___ps=0x00007fff7678a300, subtype=18, bytes=24) + 105 at mem.c:1209, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x198) frame #0: 0x0000000100014399 gsc`alloc_scmobj_still(___ps=0x00007fff7678a300, subtype=18, bytes=24) + 105 at mem.c:1209 1206 * Account for words allocated only for non-permanent objects. 1207 */ 1208 -> 1209 words_nonmovable += words; 1210 1211 if (WORDS_OCCUPIED > heap_size 1212 #ifdef CALL_GC_FREQUENTLY (lldb) bt * thread #1: tid = 0x66281, 0x0000000100014399 gsc`alloc_scmobj_still(___ps=0x00007fff7678a300, subtype=18, bytes=24) + 105 at mem.c:1209, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x198) * frame #0: 0x0000000100014399 gsc`alloc_scmobj_still(___ps=0x00007fff7678a300, subtype=18, bytes=24) + 105 at mem.c:1209 frame #1: 0x0000000100014246 gsc`___alloc_scmobj(___ps=0x00007fff7678a300, subtype=18, bytes=24) + 70 at mem.c:1284 frame #2: 0x0000000100024686 gsc`___POINTER_to_SCMOBJ(___ps=0x00007fff7678a300, x=0x0000000102800000, tags=-2, release_fn=0x000000010003a200, obj=0x00007fff5fbff200, arg_num=127) + 86 at c_intf.c:5152 frame #3: 0x000000010002478f gsc`___NONNULLPOINTER_to_SCMOBJ(___ps=0x00007fff7678a300, x=0x0000000102800000, tags=-2, release_fn=0x000000010003a200, obj=0x00007fff5fbff200, arg_num=127) + 111 at c_intf.c:5194 frame #4: 0x000000010003c8ed gsc`___os_device_stream_open_predefined(index=-4, flags=64) + 365 at os_io.c:8032 frame #5: 0x0000000100082517 gsc`___H__20___kernel_23_36(___ps=0x0000000100e6f640) + 391 at _kernel.c:17902 frame #6: 0x0000000100010346 gsc`trampoline(___ps=0x0000000100e6f640) + 38 at setup.c:1939 frame #7: 0x0000000100010298 gsc`___call(___ps=0x0000000100e6f640, nargs=0, proc=4305759265, stack_marker=4331667505) + 552 at setup.c:2029 frame #8: 0x000000010001087a gsc`___setup_pstate(___ps=0x0000000100e6f640, ___vms=0x0000000100e6f640) + 330 at setup.c:2406 frame #9: 0x0000000100010966 gsc`___setup_vmstate(___vms=0x0000000100e6f640) + 38 at setup.c:2439 frame #10: 0x0000000100010b04 gsc`___setup(setup_params=0x00007fff5fbff6a8) + 404 at setup.c:3380 frame #11: 0x000000010000eaa6 gsc`___main(linker=0x000000010000d4e0) + 5478 at main.c:715 frame #12: 0x00000001000299e6 gsc`___main_char(argc=1, argv=0x00007fff5fbff8a8, linker=0x000000010000d4e0, script_line=0x0000000000000000) + 150 at os_base.c:413 frame #13: 0x000000010000d533 gsc`main(argc=1, argv=0x00007fff5fbff8a8) + 51 at _gsc_.c:14993 frame #14: 0x00007fff912225c9 libdyld.dylib`start + 1 frame #15: 0x00007fff912225c9 libdyld.dylib`start + 1 If configures as single-threaded-vms, then it crashes in next_msection(): (lldb) target create "gsc" Current executable set to 'gsc' (x86_64). (lldb) r Process 2727 launched: '/Users/whimse/Work/MobileGames/Rabbit/gambit/gsc/gsc' (x86_64) Process 2727 stopped * thread #1: tid = 0x630ab, 0x000000010001c16f gsc`next_msection(___ps=0x00007fff7678a300, ms=0x0000000000000000) + 47 at mem.c:1788, stop reason = EXC_BAD_ACCESS (code=1, address=0x8) frame #0: 0x000000010001c16f gsc`next_msection(___ps=0x00007fff7678a300, ms=0x0000000000000000) + 47 at mem.c:1788 1785 ___msection *result; 1786 1787 if (nb_msections_used == 0) -> 1788 result = the_msections->head; 1789 else 1790 result = alloc_msection->next; 1791 (lldb) bt * thread #1: tid = 0x630ab, 0x000000010001c16f gsc`next_msection(___ps=0x00007fff7678a300, ms=0x0000000000000000) + 47 at mem.c:1788, stop reason = EXC_BAD_ACCESS (code=1, address=0x8) * frame #0: 0x000000010001c16f gsc`next_msection(___ps=0x00007fff7678a300, ms=0x0000000000000000) + 47 at mem.c:1788 frame #1: 0x0000000100015d21 gsc`next_heap_msection(___ps=0x00007fff7678a300) + 129 at mem.c:1834 frame #2: 0x000000010001778b gsc`___garbage_collect(___ps=0x00007fff7678a300, nonmovable_words_needed=9) + 443 at mem.c:4584 frame #3: 0x0000000100013dd7 gsc`alloc_scmobj_still(___ps=0x00007fff7678a300, subtype=18, bytes=24) + 263 at mem.c:1221 frame #4: 0x0000000100013be6 gsc`___alloc_scmobj(___ps=0x00007fff7678a300, subtype=18, bytes=24) + 70 at mem.c:1284 frame #5: 0x0000000100023b66 gsc`___POINTER_to_SCMOBJ(___ps=0x00007fff7678a300, x=0x0000000102800000, tags=-2, release_fn=0x00000001000396e0, obj=0x00007fff5fbff200, arg_num=127) + 86 at c_intf.c:5152 frame #6: 0x0000000100023c6f gsc`___NONNULLPOINTER_to_SCMOBJ(___ps=0x00007fff7678a300, x=0x0000000102800000, tags=-2, release_fn=0x00000001000396e0, obj=0x00007fff5fbff200, arg_num=127) + 111 at c_intf.c:5194 frame #7: 0x000000010003bdcd gsc`___os_device_stream_open_predefined(index=-4, flags=64) + 365 at os_io.c:8032 frame #8: 0x0000000100080d07 gsc`___H__20___kernel_23_36(___ps=0x0000000100e4c640) + 391 at _kernel.c:17902 frame #9: 0x000000010000fd16 gsc`trampoline(___ps=0x0000000100e4c640) + 38 at setup.c:1939 frame #10: 0x000000010000fc68 gsc`___call(___ps=0x0000000100e4c640, nargs=0, proc=4305615905, stack_marker=4331667505) + 552 at setup.c:2029 frame #11: 0x000000010001024a gsc`___setup_pstate(___ps=0x0000000100e4c640, ___vms=0x0000000100e4c640) + 330 at setup.c:2406 frame #12: 0x0000000100010336 gsc`___setup_vmstate(___vms=0x0000000100e4c640) + 38 at setup.c:2439 frame #13: 0x00000001000104d4 gsc`___setup(setup_params=0x00007fff5fbff6a8) + 404 at setup.c:3380 frame #14: 0x000000010000e476 gsc`___main(linker=0x000000010000ceb0) + 5478 at main.c:715 frame #15: 0x0000000100028ec6 gsc`___main_char(argc=1, argv=0x00007fff5fbff8a8, linker=0x000000010000ceb0, script_line=0x0000000000000000) + 150 at os_base.c:413 frame #16: 0x000000010000cf03 gsc`main(argc=1, argv=0x00007fff5fbff8a8) + 51 at _gsc_.c:14993 frame #17: 0x00007fff912225c9 libdyld.dylib`start + 1 frame #18: 0x00007fff912225c9 libdyld.dylib`start + 1
Afficher les réponses par date
After a second thought, that there were probably not many people using the ‘enable-multiple-vms’ option, so I decided to take some time to track it down. The OS X linker does not support linking an "extern __thread var”, I can do nothing about that. But the crash bug in pthread_get/setspecific fallback approach, turned out to be a minor issue: the pthread_key_t has to be created before used. Patch follows, but i am not sure that is the right position to fix it. diff --git a/lib/setup.c b/lib/setup.c index 1dc009d..c81de87 100644 --- a/lib/setup.c +++ b/lib/setup.c @@ -16,6 +16,7 @@ #include "setup.h" #include "mem.h" #include "c_intf.h" +#include "os_thread.h" /*---------------------------------------------------------------------------*/ @@ -2430,6 +2431,9 @@ ___virtual_machine_state ___vms;) * Setup virtual machine's memory management. */ +#ifndef ___THREAD_LOCAL_STORAGE_CLASS + ___setup_thread_module (); +#endif ___setup_mem_vmstate (___vms); /*
Strangely, a bounce-back email complained me being not a subscriber, but i subscribed and comfirmed already. If my message is malformed or something else, please let me know. :-) On Mon, Mar 16, 2015 at 12:11 AM, <thewhimer@gmail.com> wrote:
After a second thought, that there were probably not many people using the ‘enable-multiple-vms’ option, so I decided to take some time to track it down.
The OS X linker does not support linking an "extern __thread var”, I can do nothing about that. But the crash bug in pthread_get/setspecific fallback approach, turned out to be a minor issue: the pthread_key_t has to be created before used.
Patch follows, but i am not sure that is the right position to fix it.
diff --git a/lib/setup.c b/lib/setup.c index 1dc009d..c81de87 100644 --- a/lib/setup.c +++ b/lib/setup.c @@ -16,6 +16,7 @@ #include "setup.h" #include "mem.h" #include "c_intf.h" +#include "os_thread.h"
/*---------------------------------------------------------------------------*/ @@ -2430,6 +2431,9 @@ ___virtual_machine_state ___vms;) * Setup virtual machine's memory management. */
+#ifndef ___THREAD_LOCAL_STORAGE_CLASS + ___setup_thread_module (); +#endif ___setup_mem_vmstate (___vms);
/*
Thanks for reporting the problem. Indeed the os_thread module was not initialized (this only mattered when the C compiler did not support a thread local storage qualifier, which I guess is unusual). Marc
On Mar 15, 2015, at 9:07 PM, NTJ <thewhimer@gmail.com> wrote:
Strangely, a bounce-back email complained me being not a subscriber, but i subscribed and comfirmed already. If my message is malformed or something else, please let me know. :-)
On Mon, Mar 16, 2015 at 12:11 AM, <thewhimer@gmail.com> wrote: After a second thought, that there were probably not many people using the ‘enable-multiple-vms’ option, so I decided to take some time to track it down.
The OS X linker does not support linking an "extern __thread var”, I can do nothing about that. But the crash bug in pthread_get/setspecific fallback approach, turned out to be a minor issue: the pthread_key_t has to be created before used.
Patch follows, but i am not sure that is the right position to fix it.
diff --git a/lib/setup.c b/lib/setup.c index 1dc009d..c81de87 100644 --- a/lib/setup.c +++ b/lib/setup.c @@ -16,6 +16,7 @@ #include "setup.h" #include "mem.h" #include "c_intf.h" +#include "os_thread.h"
/*---------------------------------------------------------------------------*/ @@ -2430,6 +2431,9 @@ ___virtual_machine_state ___vms;) * Setup virtual machine's memory management. */
+#ifndef ___THREAD_LOCAL_STORAGE_CLASS + ___setup_thread_module (); +#endif ___setup_mem_vmstate (___vms);
/*
_______________________________________________ Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
Could you post this on Gambit’s github issue tracker. I’ll have to look into this at a later time. It is strange that this does not work on OS X. Marc
On Mar 15, 2015, at 7:24 AM, Gmail <thewhimer@gmail.com> wrote:
I tried to use Gambit-C with the 'multiple-vms’ option enabled, to harness full cpu power by SMP. But ‘compile-file’ on REPL failed with the message:
ld: illegal thread local variable reference to regular symbol ____tls_ptr for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation)
After some experiments, it seems that the OS X linker does not properly support dynamic linking of externally defined TLS variables. This can be verified easily with the following code.
// test.c extern __thread int tls_id; int get_tls_id() { return tls_id; }
On OS X, both gcc and clang failed to produce a dynamically loadable object file. (OS X 10.10.1, Xcode 6.2, clang-600.0.57; gcc-4.8 intalled from homebrew.) $ gcc-4.8 -shared test.c -o libfoo.so $ gcc-4.9 -shared test.c -o libfoo.so $ clang -bundle test.c -o libfoo.so
The three give the same error:
Undefined symbols for architecture x86_64: "_tls_num", referenced from: _get_tls_num in test-9d6352.o (maybe you meant: _get_tls_num) ld: symbol(s) not found for architecture x86_64
While on Linux (ArchLinux 3.18.6 X86_64, gcc 4.9.2), it worked. The produced library linked fine with main.c (as follows), execution gives 1.
// main.c #include <stdio.h>
__thread int tls_id;
extern int get_tls_id();
int main () { int id = 0; tls_id = 1; id = get_tls_id(); printf(“tls id is %d\n”, id); return 0; }
I also tried to fallback on the pthread_get/setspecific() approach, by manually change CONF_THREAD_LOCAL_STORAGE_CLASS in configure/configure.ac. Well, gsc/gsi crashes on running, due to a memory related bug. I am fairly new to Gambit, hope someone more familiar with its source would help to pin this bug.
If configures as enable-multiple-threaded-vm, then it crashes in alloc_scmobj_still():
(lldb) target create "gsc" Current executable set to 'gsc' (x86_64). (lldb) r Process 96941 launched: '/Users/whimse/Work/MobileGames/Rabbit/gambit/gsc/gsc' (x86_64) Process 96941 stopped * thread #1: tid = 0x6176f, 0x0000000100014399 gsc`alloc_scmobj_still(___ps=0x00007fff7678a300, subtype=18, bytes=24) + 105 at mem.c:1209, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x198) frame #0: 0x0000000100014399 gsc`alloc_scmobj_still(___ps=0x00007fff7678a300, subtype=18, bytes=24) + 105 at mem.c:1209 1206 * Account for words allocated only for non-permanent objects. 1207 */ 1208 -> 1209 words_nonmovable += words; 1210 1211 if (WORDS_OCCUPIED > heap_size 1212 #ifdef CALL_GC_FREQUENTLY (lldb) bt * thread #1: tid = 0x66281, 0x0000000100014399 gsc`alloc_scmobj_still(___ps=0x00007fff7678a300, subtype=18, bytes=24) + 105 at mem.c:1209, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x198) * frame #0: 0x0000000100014399 gsc`alloc_scmobj_still(___ps=0x00007fff7678a300, subtype=18, bytes=24) + 105 at mem.c:1209 frame #1: 0x0000000100014246 gsc`___alloc_scmobj(___ps=0x00007fff7678a300, subtype=18, bytes=24) + 70 at mem.c:1284 frame #2: 0x0000000100024686 gsc`___POINTER_to_SCMOBJ(___ps=0x00007fff7678a300, x=0x0000000102800000, tags=-2, release_fn=0x000000010003a200, obj=0x00007fff5fbff200, arg_num=127) + 86 at c_intf.c:5152 frame #3: 0x000000010002478f gsc`___NONNULLPOINTER_to_SCMOBJ(___ps=0x00007fff7678a300, x=0x0000000102800000, tags=-2, release_fn=0x000000010003a200, obj=0x00007fff5fbff200, arg_num=127) + 111 at c_intf.c:5194 frame #4: 0x000000010003c8ed gsc`___os_device_stream_open_predefined(index=-4, flags=64) + 365 at os_io.c:8032 frame #5: 0x0000000100082517 gsc`___H__20___kernel_23_36(___ps=0x0000000100e6f640) + 391 at _kernel.c:17902 frame #6: 0x0000000100010346 gsc`trampoline(___ps=0x0000000100e6f640) + 38 at setup.c:1939 frame #7: 0x0000000100010298 gsc`___call(___ps=0x0000000100e6f640, nargs=0, proc=4305759265, stack_marker=4331667505) + 552 at setup.c:2029 frame #8: 0x000000010001087a gsc`___setup_pstate(___ps=0x0000000100e6f640, ___vms=0x0000000100e6f640) + 330 at setup.c:2406 frame #9: 0x0000000100010966 gsc`___setup_vmstate(___vms=0x0000000100e6f640) + 38 at setup.c:2439 frame #10: 0x0000000100010b04 gsc`___setup(setup_params=0x00007fff5fbff6a8) + 404 at setup.c:3380 frame #11: 0x000000010000eaa6 gsc`___main(linker=0x000000010000d4e0) + 5478 at main.c:715 frame #12: 0x00000001000299e6 gsc`___main_char(argc=1, argv=0x00007fff5fbff8a8, linker=0x000000010000d4e0, script_line=0x0000000000000000) + 150 at os_base.c:413 frame #13: 0x000000010000d533 gsc`main(argc=1, argv=0x00007fff5fbff8a8) + 51 at _gsc_.c:14993 frame #14: 0x00007fff912225c9 libdyld.dylib`start + 1 frame #15: 0x00007fff912225c9 libdyld.dylib`start + 1
If configures as single-threaded-vms, then it crashes in next_msection():
(lldb) target create "gsc" Current executable set to 'gsc' (x86_64). (lldb) r Process 2727 launched: '/Users/whimse/Work/MobileGames/Rabbit/gambit/gsc/gsc' (x86_64) Process 2727 stopped * thread #1: tid = 0x630ab, 0x000000010001c16f gsc`next_msection(___ps=0x00007fff7678a300, ms=0x0000000000000000) + 47 at mem.c:1788, stop reason = EXC_BAD_ACCESS (code=1, address=0x8) frame #0: 0x000000010001c16f gsc`next_msection(___ps=0x00007fff7678a300, ms=0x0000000000000000) + 47 at mem.c:1788 1785 ___msection *result; 1786 1787 if (nb_msections_used == 0) -> 1788 result = the_msections->head; 1789 else 1790 result = alloc_msection->next; 1791 (lldb) bt * thread #1: tid = 0x630ab, 0x000000010001c16f gsc`next_msection(___ps=0x00007fff7678a300, ms=0x0000000000000000) + 47 at mem.c:1788, stop reason = EXC_BAD_ACCESS (code=1, address=0x8) * frame #0: 0x000000010001c16f gsc`next_msection(___ps=0x00007fff7678a300, ms=0x0000000000000000) + 47 at mem.c:1788 frame #1: 0x0000000100015d21 gsc`next_heap_msection(___ps=0x00007fff7678a300) + 129 at mem.c:1834 frame #2: 0x000000010001778b gsc`___garbage_collect(___ps=0x00007fff7678a300, nonmovable_words_needed=9) + 443 at mem.c:4584 frame #3: 0x0000000100013dd7 gsc`alloc_scmobj_still(___ps=0x00007fff7678a300, subtype=18, bytes=24) + 263 at mem.c:1221 frame #4: 0x0000000100013be6 gsc`___alloc_scmobj(___ps=0x00007fff7678a300, subtype=18, bytes=24) + 70 at mem.c:1284 frame #5: 0x0000000100023b66 gsc`___POINTER_to_SCMOBJ(___ps=0x00007fff7678a300, x=0x0000000102800000, tags=-2, release_fn=0x00000001000396e0, obj=0x00007fff5fbff200, arg_num=127) + 86 at c_intf.c:5152 frame #6: 0x0000000100023c6f gsc`___NONNULLPOINTER_to_SCMOBJ(___ps=0x00007fff7678a300, x=0x0000000102800000, tags=-2, release_fn=0x00000001000396e0, obj=0x00007fff5fbff200, arg_num=127) + 111 at c_intf.c:5194 frame #7: 0x000000010003bdcd gsc`___os_device_stream_open_predefined(index=-4, flags=64) + 365 at os_io.c:8032 frame #8: 0x0000000100080d07 gsc`___H__20___kernel_23_36(___ps=0x0000000100e4c640) + 391 at _kernel.c:17902 frame #9: 0x000000010000fd16 gsc`trampoline(___ps=0x0000000100e4c640) + 38 at setup.c:1939 frame #10: 0x000000010000fc68 gsc`___call(___ps=0x0000000100e4c640, nargs=0, proc=4305615905, stack_marker=4331667505) + 552 at setup.c:2029 frame #11: 0x000000010001024a gsc`___setup_pstate(___ps=0x0000000100e4c640, ___vms=0x0000000100e4c640) + 330 at setup.c:2406 frame #12: 0x0000000100010336 gsc`___setup_vmstate(___vms=0x0000000100e4c640) + 38 at setup.c:2439 frame #13: 0x00000001000104d4 gsc`___setup(setup_params=0x00007fff5fbff6a8) + 404 at setup.c:3380 frame #14: 0x000000010000e476 gsc`___main(linker=0x000000010000ceb0) + 5478 at main.c:715 frame #15: 0x0000000100028ec6 gsc`___main_char(argc=1, argv=0x00007fff5fbff8a8, linker=0x000000010000ceb0, script_line=0x0000000000000000) + 150 at os_base.c:413 frame #16: 0x000000010000cf03 gsc`main(argc=1, argv=0x00007fff5fbff8a8) + 51 at _gsc_.c:14993 frame #17: 0x00007fff912225c9 libdyld.dylib`start + 1 frame #18: 0x00007fff912225c9 libdyld.dylib`start + 1
_______________________________________________ Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
participants (4)
-
Gmail -
Marc Feeley -
NTJ
-
thewhimer＠gmail.com