[runtime] Don't try (slow) dlopen on android if it doesn't exist #12074
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vargaz
approved these changes
Dec 14, 2018
Contributor
|
looks good. did you test this change on a device? |
Contributor
Author
|
@lewurm I haven't, I was hoping to see our CI test it on sim. I don't have any of the XA caches on my machine currently. |
Contributor
|
@alexanderkyte CI will not give you any result because of #12094 |
Contributor
Author
|
@lewurm @luhenry That indicates that this works on android (or passes those tests). |
Contributor
Author
The one outstanding failure seen on CI: ^ |
Contributor
Author
|
@monojenkins backport 2018-08 As requested by @jonpryor |
This was referenced Jan 18, 2019
monojenkins
added a commit
that referenced
this pull request
Jan 21, 2019
) [2018-12] [runtime] Don\u0027t try (slow) dlopen on android if it doesn\u0027t exist On android, the dlopen call will take around 1ms per AOT module we try to load. By not using dlopen with the existence check, we rely on a much slower dlopen check that happens later. I think they support dlopen of non-resolved names. This might break a few things if we don't use the full path to dlopen. If it does, we should probably add existence checks at the caller, but this will be an optimization that works everywhere. @jonpryor @grendello Backport of #12074. /cc @alexanderkyte
Member
|
Going to need to revert this one. It's been identified as the cause of dotnet/android#3388 |
lambdageek
added a commit
to lambdageek/mono
that referenced
this pull request
Aug 21, 2019
The intention of calling `g_file_test (file, G_FILE_TEST_EXISTS)` where file is the name of a shared library we want to open is to speed up probing for non-existent libraries. See mono#12074 The problem is that if file is just a simple "libdl.so" then `dlopen (file)` doesn't just look for it in the current working directory, it will probe some other paths too. (For example on desktop linux you'd also look in all the directories in LD_LIBRARY_PATH). So the g_file_test() call is not a robust way to avoid calling dlopen if the filename is relative. But it actually broke more things: dotnet/android#3388 When probing for "libdl.so" on Android mono_lookup_pinvoke_call will first try prepending some paths that it knows about and we end up calling `dlopen ("/system/lib/libdl.so")` which will fail because Bionic has security restrictions on what code can dlopen something from /system/lib with an absolute path. Eventually mono_lookup_pinvoke_call will go back to trying the bare "libdl.so" which hits `g_file_test` and returns NULL. The new code only does the file test if we pass it an absolute path, which gives Bionic's dlopen a chance to deal with relative paths however it needs to.
monojenkins
pushed a commit
to monojenkins/mono
that referenced
this pull request
Aug 21, 2019
The intention of calling `g_file_test (file, G_FILE_TEST_EXISTS)` where file is the name of a shared library we want to open is to speed up probing for non-existent libraries. See mono#12074 The problem is that if file is just a simple "libdl.so" then `dlopen (file)` doesn't just look for it in the current working directory, it will probe some other paths too. (For example on desktop linux you'd also look in all the directories in LD_LIBRARY_PATH). So the g_file_test() call is not a robust way to avoid calling dlopen if the filename is relative. But it actually broke more things: dotnet/android#3388 When probing for "libdl.so" on Android mono_lookup_pinvoke_call will first try prepending some paths that it knows about and we end up calling `dlopen ("/system/lib/libdl.so")` which will fail because Bionic has security restrictions on what code can dlopen something from /system/lib with an absolute path. Eventually mono_lookup_pinvoke_call will go back to trying the bare "libdl.so" which hits `g_file_test` and returns NULL. The new code only does the file test if we pass it an absolute path, which gives Bionic's dlopen a chance to deal with relative paths however it needs to.
monojenkins
pushed a commit
to monojenkins/mono
that referenced
this pull request
Aug 21, 2019
The intention of calling `g_file_test (file, G_FILE_TEST_EXISTS)` where file is the name of a shared library we want to open is to speed up probing for non-existent libraries. See mono#12074 The problem is that if file is just a simple "libdl.so" then `dlopen (file)` doesn't just look for it in the current working directory, it will probe some other paths too. (For example on desktop linux you'd also look in all the directories in LD_LIBRARY_PATH). So the g_file_test() call is not a robust way to avoid calling dlopen if the filename is relative. But it actually broke more things: dotnet/android#3388 When probing for "libdl.so" on Android mono_lookup_pinvoke_call will first try prepending some paths that it knows about and we end up calling `dlopen ("/system/lib/libdl.so")` which will fail because Bionic has security restrictions on what code can dlopen something from /system/lib with an absolute path. Eventually mono_lookup_pinvoke_call will go back to trying the bare "libdl.so" which hits `g_file_test` and returns NULL. The new code only does the file test if we pass it an absolute path, which gives Bionic's dlopen a chance to deal with relative paths however it needs to.
This was referenced Aug 21, 2019
lambdageek
added a commit
that referenced
this pull request
Aug 22, 2019
…16387) The intention of calling `g_file_test (file, G_FILE_TEST_EXISTS)` where file is the name of a shared library we want to open is to speed up probing for non-existent libraries. See #12074 The problem is that if file is just a simple "libdl.so" then `dlopen (file)` doesn't just look for it in the current working directory, it will probe some other paths too. (For example on desktop linux you'd also look in all the directories in LD_LIBRARY_PATH). So the g_file_test() call is not a robust way to avoid calling dlopen if the filename is relative. But it actually broke more things: dotnet/android#3388 When probing for "libdl.so" on Android mono_lookup_pinvoke_call will first try prepending some paths that it knows about and we end up calling `dlopen ("/system/lib/libdl.so")` which will fail because Bionic has security restrictions on what code can dlopen something from /system/lib with an absolute path. Eventually mono_lookup_pinvoke_call will go back to trying the bare "libdl.so" which hits `g_file_test` and returns NULL. The new code only does the file test if we pass it an absolute path, which gives Bionic's dlopen a chance to deal with relative paths however it needs to.
lambdageek
pushed a commit
that referenced
this pull request
Aug 22, 2019
…16392) The intention of calling `g_file_test (file, G_FILE_TEST_EXISTS)` where file is the name of a shared library we want to open is to speed up probing for non-existent libraries. See #12074 The problem is that if file is just a simple "libdl.so" then `dlopen (file)` doesn't just look for it in the current working directory, it will probe some other paths too. (For example on desktop linux you'd also look in all the directories in LD_LIBRARY_PATH). So the g_file_test() call is not a robust way to avoid calling dlopen if the filename is relative. But it actually broke more things: dotnet/android#3388 When probing for "libdl.so" on Android mono_lookup_pinvoke_call will first try prepending some paths that it knows about and we end up calling `dlopen ("/system/lib/libdl.so")` which will fail because Bionic has security restrictions on what code can dlopen something from /system/lib with an absolute path. Eventually mono_lookup_pinvoke_call will go back to trying the bare "libdl.so" which hits `g_file_test` and returns NULL. The new code only does the file test if we pass it an absolute path, which gives Bionic's dlopen a chance to deal with relative paths however it needs to.
lambdageek
pushed a commit
that referenced
this pull request
Aug 22, 2019
…16393) The intention of calling `g_file_test (file, G_FILE_TEST_EXISTS)` where file is the name of a shared library we want to open is to speed up probing for non-existent libraries. See #12074 The problem is that if file is just a simple "libdl.so" then `dlopen (file)` doesn't just look for it in the current working directory, it will probe some other paths too. (For example on desktop linux you'd also look in all the directories in LD_LIBRARY_PATH). So the g_file_test() call is not a robust way to avoid calling dlopen if the filename is relative. But it actually broke more things: dotnet/android#3388 When probing for "libdl.so" on Android mono_lookup_pinvoke_call will first try prepending some paths that it knows about and we end up calling `dlopen ("/system/lib/libdl.so")` which will fail because Bionic has security restrictions on what code can dlopen something from /system/lib with an absolute path. Eventually mono_lookup_pinvoke_call will go back to trying the bare "libdl.so" which hits `g_file_test` and returns NULL. The new code only does the file test if we pass it an absolute path, which gives Bionic's dlopen a chance to deal with relative paths however it needs to.
ManickaP
pushed a commit
to ManickaP/runtime
that referenced
this pull request
Jan 20, 2020
…ono/mono#16387) The intention of calling `g_file_test (file, G_FILE_TEST_EXISTS)` where file is the name of a shared library we want to open is to speed up probing for non-existent libraries. See mono/mono#12074 The problem is that if file is just a simple "libdl.so" then `dlopen (file)` doesn't just look for it in the current working directory, it will probe some other paths too. (For example on desktop linux you'd also look in all the directories in LD_LIBRARY_PATH). So the g_file_test() call is not a robust way to avoid calling dlopen if the filename is relative. But it actually broke more things: dotnet/android#3388 When probing for "libdl.so" on Android mono_lookup_pinvoke_call will first try prepending some paths that it knows about and we end up calling `dlopen ("/system/lib/libdl.so")` which will fail because Bionic has security restrictions on what code can dlopen something from /system/lib with an absolute path. Eventually mono_lookup_pinvoke_call will go back to trying the bare "libdl.so" which hits `g_file_test` and returns NULL. The new code only does the file test if we pass it an absolute path, which gives Bionic's dlopen a chance to deal with relative paths however it needs to. Commit migrated from mono/mono@e74736a
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
On android, the dlopen call will take around 1ms per AOT module we try to load. By not using dlopen with the existence check, we rely on a much slower dlopen check that happens later. I think they support dlopen of non-resolved names.
This might break a few things if we don't use the full path to dlopen. If it does, we should probably add existence checks at the caller, but this will be an optimization that works everywhere.
@jonpryor @grendello