[threads] clear small_id_key TLS when unregistering a thread #16973
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixes ``` * thread mono#12, name = 'tid_a507', stop reason = EXC_BREAKPOINT (code=1, subcode=0x1be66144) * frame #0: 0x1be66144 libsystem_c.dylib`__abort + 184 frame #1: 0x1be6608c libsystem_c.dylib`abort + 152 frame #2: 0x003e1fa0 monotouchtest`log_callback(log_domain=0x00000000, log_level="error", message="* Assertion at ../../../../../mono/utils/hazard-pointer.c:158, condition `mono_bitset_test_fast (small_id_table, id)' not met\n", fatal=4, user_data=0x00000000) at runtime.m:1251:3 frame mono#3: 0x003abf44 monotouchtest`monoeg_g_logv_nofree(log_domain=0x00000000, log_level=G_LOG_LEVEL_ERROR, format=<unavailable>, args=<unavailable>) at goutput.c:149:2 [opt] frame mono#4: 0x003abfb4 monotouchtest`monoeg_assertion_message(format=<unavailable>) at goutput.c:184:22 [opt] frame mono#5: 0x003904dc monotouchtest`mono_thread_small_id_free(id=<unavailable>) at hazard-pointer.c:0:2 [opt] frame mono#6: 0x003a0a74 monotouchtest`unregister_thread(arg=0x15c88400) at mono-threads.c:588:2 [opt] frame mono#7: 0x00336110 monotouchtest`mono_thread_detach_if_exiting at threads.c:1571:4 [opt] frame mono#8: 0x003e7a14 monotouchtest`::xamarin_release_trampoline(self=0x166452f0, sel="release") at trampolines.m:644:3 frame mono#9: 0x001cdc40 monotouchtest`::-[__Xamarin_NSTimerActionDispatcher release](self=0x166452f0, _cmd="release") at registrar.m:83445:3 frame mono#10: 0x1ce2ae68 Foundation`_timerRelease + 80 frame mono#11: 0x1c31b56c CoreFoundation`CFRunLoopTimerInvalidate + 612 frame mono#12: 0x1c31a554 CoreFoundation`__CFRunLoopTimerDeallocate + 32 frame mono#13: 0x1c31dde4 CoreFoundation`_CFRelease + 220 frame mono#14: 0x1c2be6e8 CoreFoundation`__CFArrayReleaseValues + 500 frame mono#15: 0x1c2be4c4 CoreFoundation`CFArrayRemoveAllValues + 104 frame mono#16: 0x1c31ff64 CoreFoundation`__CFSetApplyFunction_block_invoke + 24 frame mono#17: 0x1c3b2e18 CoreFoundation`CFBasicHashApply + 116 frame mono#18: 0x1c31ff10 CoreFoundation`CFSetApplyFunction + 160 frame mono#19: 0x1c3152cc CoreFoundation`__CFRunLoopDeallocate + 204 frame mono#20: 0x1c31dde4 CoreFoundation`_CFRelease + 220 frame mono#21: 0x1c304a80 CoreFoundation`__CFTSDFinalize + 144 frame mono#22: 0x1bfa324c libsystem_pthread.dylib`_pthread_tsd_cleanup + 644 frame mono#23: 0x1bf9cc08 libsystem_pthread.dylib`_pthread_exit + 80 frame mono#24: 0x1bf9af24 libsystem_pthread.dylib`pthread_exit + 36 frame mono#25: 0x0039df84 monotouchtest`mono_threads_platform_exit(exit_code=<unavailable>) at mono-threads-posix.c:145:2 [opt] frame mono#26: 0x0033bb84 monotouchtest`start_wrapper(data=0x1609e1c0) at threads.c:1296:2 [opt] frame mono#27: 0x1bf9b914 libsystem_pthread.dylib`_pthread_body + 128 frame mono#28: 0x1bf9b874 libsystem_pthread.dylib`_pthread_start + 44 frame mono#29: 0x1bfa3b94 libsystem_pthread.dylib`thread_start + 4 ```
Member
|
We think what happens is:
|
lambdageek
previously approved these changes
Sep 20, 2019
lambdageek
reviewed
Sep 20, 2019
Maybe we should get someone else to look at this, since i'm a co-conspirator here.
Co-Authored-By: Aleksey Kliger (λgeek) <[email protected]>
Member
|
@vargaz could you please review |
BrzVlad
approved these changes
Sep 24, 2019
Contributor
Author
|
@monojenkins backport 2019-08 |
monojenkins
added a commit
that referenced
this pull request
Sep 24, 2019
#17015) [2019-08] [threads] clear small_id_key TLS when unregistering a thread Fixes ``` * thread #12, name = 'tid_a507', stop reason = EXC_BREAKPOINT (code=1, subcode=0x1be66144) * frame #0: 0x1be66144 libsystem_c.dylib`__abort + 184 frame #1: 0x1be6608c libsystem_c.dylib`abort + 152 frame #2: 0x003e1fa0 monotouchtest`log_callback(log_domain=0x00000000, log_level="error", message="* Assertion at ../../../../../mono/utils/hazard-pointer.c:158, condition `mono_bitset_test_fast (small_id_table, id)' not met\n", fatal=4, user_data=0x00000000) at runtime.m:1251:3 frame #3: 0x003abf44 monotouchtest`monoeg_g_logv_nofree(log_domain=0x00000000, log_level=G_LOG_LEVEL_ERROR, format=<unavailable>, args=<unavailable>) at goutput.c:149:2 [opt] frame #4: 0x003abfb4 monotouchtest`monoeg_assertion_message(format=<unavailable>) at goutput.c:184:22 [opt] frame #5: 0x003904dc monotouchtest`mono_thread_small_id_free(id=<unavailable>) at hazard-pointer.c:0:2 [opt] frame #6: 0x003a0a74 monotouchtest`unregister_thread(arg=0x15c88400) at mono-threads.c:588:2 [opt] frame #7: 0x00336110 monotouchtest`mono_thread_detach_if_exiting at threads.c:1571:4 [opt] frame #8: 0x003e7a14 monotouchtest`::xamarin_release_trampoline(self=0x166452f0, sel="release") at trampolines.m:644:3 frame #9: 0x001cdc40 monotouchtest`::-[__Xamarin_NSTimerActionDispatcher release](self=0x166452f0, _cmd="release") at registrar.m:83445:3 frame #10: 0x1ce2ae68 Foundation`_timerRelease + 80 frame #11: 0x1c31b56c CoreFoundation`CFRunLoopTimerInvalidate + 612 frame #12: 0x1c31a554 CoreFoundation`__CFRunLoopTimerDeallocate + 32 frame #13: 0x1c31dde4 CoreFoundation`_CFRelease + 220 frame #14: 0x1c2be6e8 CoreFoundation`__CFArrayReleaseValues + 500 frame #15: 0x1c2be4c4 CoreFoundation`CFArrayRemoveAllValues + 104 frame #16: 0x1c31ff64 CoreFoundation`__CFSetApplyFunction_block_invoke + 24 frame #17: 0x1c3b2e18 CoreFoundation`CFBasicHashApply + 116 frame #18: 0x1c31ff10 CoreFoundation`CFSetApplyFunction + 160 frame #19: 0x1c3152cc CoreFoundation`__CFRunLoopDeallocate + 204 frame #20: 0x1c31dde4 CoreFoundation`_CFRelease + 220 frame #21: 0x1c304a80 CoreFoundation`__CFTSDFinalize + 144 frame #22: 0x1bfa324c libsystem_pthread.dylib`_pthread_tsd_cleanup + 644 frame #23: 0x1bf9cc08 libsystem_pthread.dylib`_pthread_exit + 80 frame #24: 0x1bf9af24 libsystem_pthread.dylib`pthread_exit + 36 frame #25: 0x0039df84 monotouchtest`mono_threads_platform_exit(exit_code=<unavailable>) at mono-threads-posix.c:145:2 [opt] frame #26: 0x0033bb84 monotouchtest`start_wrapper(data=0x1609e1c0) at threads.c:1296:2 [opt] frame #27: 0x1bf9b914 libsystem_pthread.dylib`_pthread_body + 128 frame #28: 0x1bf9b874 libsystem_pthread.dylib`_pthread_start + 44 frame #29: 0x1bfa3b94 libsystem_pthread.dylib`thread_start + 4 ``` Debugged by @lambdageek. Contributes to #10641 Edit: C/p into PR description so it's in the commit message: We think what happens is: 1. `start_wrapper` runs `unregister_thread` when the thread is exiting. At that point the `small_id` for the thread is cleared from the hazard pointer bitset by `mono_thread_small_id_free (small_id)` 2. Some other TLS destructor runs and attaches the thread again (which runs `mono_thread_info_register_small_id` which first calls `mono_thread_info_get_small_id` which tries to get the small id from the `small_id_key` TLS key and so the new `MonoThreadInfo` has the same `small_id` as the previously destroyed `MonoThreadInfo` - but the hazard pointer bitset is **not** updated). 3. This other TLS destructor runs and calls `mono_thread_detach_if_exiting` which calls `unregister_thread` again. 4. `unregister_thread` calls ` mono_thread_small_id_free (small_id)` a second time which asserts because we already cleared that id from the hazard pointer bitset. Backport of #16973. /cc @lewurm
ManickaP
pushed a commit
to ManickaP/runtime
that referenced
this pull request
Jan 20, 2020
…no#16973) * [threads] clear small_id_key TLS when unregistering a thread Fixes ``` * thread mono/mono#12, name = 'tid_a507', stop reason = EXC_BREAKPOINT (code=1, subcode=0x1be66144) * frame mono/mono#0: 0x1be66144 libsystem_c.dylib`__abort + 184 frame mono/mono#1: 0x1be6608c libsystem_c.dylib`abort + 152 frame mono/mono#2: 0x003e1fa0 monotouchtest`log_callback(log_domain=0x00000000, log_level="error", message="* Assertion at ../../../../../mono/utils/hazard-pointer.c:158, condition `mono_bitset_test_fast (small_id_table, id)' not met\n", fatal=4, user_data=0x00000000) at runtime.m:1251:3 frame mono/mono#3: 0x003abf44 monotouchtest`monoeg_g_logv_nofree(log_domain=0x00000000, log_level=G_LOG_LEVEL_ERROR, format=<unavailable>, args=<unavailable>) at goutput.c:149:2 [opt] frame mono/mono#4: 0x003abfb4 monotouchtest`monoeg_assertion_message(format=<unavailable>) at goutput.c:184:22 [opt] frame mono/mono#5: 0x003904dc monotouchtest`mono_thread_small_id_free(id=<unavailable>) at hazard-pointer.c:0:2 [opt] frame mono/mono#6: 0x003a0a74 monotouchtest`unregister_thread(arg=0x15c88400) at mono-threads.c:588:2 [opt] frame mono/mono#7: 0x00336110 monotouchtest`mono_thread_detach_if_exiting at threads.c:1571:4 [opt] frame mono/mono#8: 0x003e7a14 monotouchtest`::xamarin_release_trampoline(self=0x166452f0, sel="release") at trampolines.m:644:3 frame mono/mono#9: 0x001cdc40 monotouchtest`::-[__Xamarin_NSTimerActionDispatcher release](self=0x166452f0, _cmd="release") at registrar.m:83445:3 frame mono/mono#10: 0x1ce2ae68 Foundation`_timerRelease + 80 frame mono/mono#11: 0x1c31b56c CoreFoundation`CFRunLoopTimerInvalidate + 612 frame mono/mono#12: 0x1c31a554 CoreFoundation`__CFRunLoopTimerDeallocate + 32 frame mono/mono#13: 0x1c31dde4 CoreFoundation`_CFRelease + 220 frame mono/mono#14: 0x1c2be6e8 CoreFoundation`__CFArrayReleaseValues + 500 frame mono/mono#15: 0x1c2be4c4 CoreFoundation`CFArrayRemoveAllValues + 104 frame mono/mono#16: 0x1c31ff64 CoreFoundation`__CFSetApplyFunction_block_invoke + 24 frame mono/mono#17: 0x1c3b2e18 CoreFoundation`CFBasicHashApply + 116 frame mono/mono#18: 0x1c31ff10 CoreFoundation`CFSetApplyFunction + 160 frame mono/mono#19: 0x1c3152cc CoreFoundation`__CFRunLoopDeallocate + 204 frame mono/mono#20: 0x1c31dde4 CoreFoundation`_CFRelease + 220 frame mono/mono#21: 0x1c304a80 CoreFoundation`__CFTSDFinalize + 144 frame mono/mono#22: 0x1bfa324c libsystem_pthread.dylib`_pthread_tsd_cleanup + 644 frame mono/mono#23: 0x1bf9cc08 libsystem_pthread.dylib`_pthread_exit + 80 frame mono/mono#24: 0x1bf9af24 libsystem_pthread.dylib`pthread_exit + 36 frame mono/mono#25: 0x0039df84 monotouchtest`mono_threads_platform_exit(exit_code=<unavailable>) at mono-threads-posix.c:145:2 [opt] frame mono/mono#26: 0x0033bb84 monotouchtest`start_wrapper(data=0x1609e1c0) at threads.c:1296:2 [opt] frame mono/mono#27: 0x1bf9b914 libsystem_pthread.dylib`_pthread_body + 128 frame mono/mono#28: 0x1bf9b874 libsystem_pthread.dylib`_pthread_start + 44 frame mono/mono#29: 0x1bfa3b94 libsystem_pthread.dylib`thread_start + 4 ``` * Update mono/utils/mono-threads.c Co-Authored-By: Aleksey Kliger (λgeek) <[email protected]> Commit migrated from mono/mono@749493d
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes
Debugged by @lambdageek. Contributes to #10641
Edit: C/p into PR description so it's in the commit message:
We think what happens is:
start_wrapperrunsunregister_threadwhen the thread is exiting. At that point thesmall_idfor the thread is cleared from the hazard pointer bitset bymono_thread_small_id_free (small_id)Some other TLS destructor runs and attaches the thread again (which runs
mono_thread_info_register_small_idwhich first callsmono_thread_info_get_small_idwhich tries to get the small id from thesmall_id_keyTLS key and so the newMonoThreadInfohas the samesmall_idas the previously destroyedMonoThreadInfo- but the hazard pointer bitset is not updated).This other TLS destructor runs and calls
mono_thread_detach_if_exitingwhich callsunregister_threadagain.unregister_threadcallsmono_thread_small_id_free (small_id)a second time which asserts because we already cleared that id from the hazard pointer bitset.