Add --exclude-libs=ALL to libIREECompiler.so shared library#23574
Merged
AaronStGeorge merged 3 commits intoFeb 27, 2026
Merged
Conversation
libIREECompiler.so statically links LLVM/MLIR but exports ~175K internal
symbols via its version script (api_version.ld uses `global: *`). The
visibility design assumes LLVM is compiled with -fvisibility=hidden, but
LLVM's own CMake build only applies -fvisibility-inlines-hidden — not
-fvisibility=hidden — to the vast majority of its compilation units
(2250/2720 LLVM files, 1049/1145 MLIR files lack the flag). This means
all non-inline LLVM symbols default to visibility("default") and leak
through the version script.
When libIREECompiler.so is loaded via dlopen(RTLD_LOCAL) into a process
that already has libLLVM.so in the global scope (e.g. via HIP runtime's
libamd_comgr.so → libLLVM.so.22.0git), the ELF dynamic linker searches
the global scope first when resolving relocations. LLVM function calls
in libIREECompiler.so that go through the PLT/GOT resolve to
libLLVM.so's copies rather than the compiler's own static LLVM. This
causes LLVM cl::Option static initializers to register duplicate options
into libLLVM.so's global registry, crashing with:
LLVM ERROR: Option 'pbqp' already exists!
The fix adds --exclude-libs=ALL to the SharedImpl linker options, which
forces all symbols from static archives (.a files) to hidden visibility
at link time. Only symbols from object files (.o) compiled with explicit
__attribute__((visibility("default"))) annotations — i.e. the IREE and
MLIR C API functions marked with IREE_EMBED_EXPORTED / MLIR_CAPI_EXPORTED
— remain visible. This reduces exported dynamic symbols from ~274K to
~2.9K.
This matches the pattern already used by MLIR's own MLIR-C shared
library (mlir/lib/CAPI/CMakeLists.txt):
target_link_options(MLIR-C PRIVATE "-Wl,-exclude-libs,ALL")
Co-Authored-By: Claude Opus 4.6 <[email protected]>
bfc5f56 to
020b875
Compare
Contributor
|
I posted about the linker script initially, but I see iree is using it already but in some weird way |
…tension Signed-off-by: Bangtian Liu <[email protected]>
MaheshRavishankar
approved these changes
Feb 26, 2026
Collaborator
|
I think all the failures here are unrelated. The clang test failure seems scary. It might be a compiler flake. If this is persists, please file an issue and disable the test. |
Add `noaarch64` label support to `iree_is_bytecode_module_test_excluded_by_labels` and apply it to the `check_regression_dynamic_gather_attention_llvm-cpu` test suite. Co-Authored-By: Claude Opus 4.6 <[email protected]>
d5f4c73 to
2875264
Compare
kimm240
pushed a commit
to kimm240/iree
that referenced
this pull request
May 8, 2026
…#23574) When building `libIREECompiler.so` in `TheRock` I hit the following error when trying to load the library: ``` LLVM ERROR: Option 'pbqp' already exists! ``` I'll put the full claude generated description of what happened below but the long and the short of it is `libIREECompiler.so` seems to be exporting quite a few symbols when linking LLVM's static `.a`'s. In my case, `TheRock` has another library that loads `libLLVM.so`, and when loaded it [interposes](https://maskray.me/blog/2021-05-16-elf-interposition-and-bsymbolic) some of the global command line registration machinery. Because of the interposition, `libIREECompiler.so`'s command line registration got routed to `libLLVM.so`'s; luckily, rather than some incredibly hard to diagnose error, the double command line registration crashes on startup. `--exclude-libs,ALL` hides symbols from static libraries when linking the final `libIREECompiler.so`, preventing LLVM's symbols from being exported and therefore being interposable. --- Claude generated description ``` Add --exclude-libs=ALL to libIREECompiler.so shared library libIREECompiler.so statically links LLVM/MLIR but exports ~175K internal symbols via its version script (api_version.ld uses `global: *`). The visibility design assumes LLVM is compiled with -fvisibility=hidden, but LLVM's own CMake build only applies -fvisibility-inlines-hidden — not -fvisibility=hidden — to the vast majority of its compilation units (2250/2720 LLVM files, 1049/1145 MLIR files lack the flag). This means all non-inline LLVM symbols default to visibility("default") and leak through the version script. When libIREECompiler.so is loaded via dlopen(RTLD_LOCAL) into a process that already has libLLVM.so in the global scope (e.g. via HIP runtime's libamd_comgr.so → libLLVM.so.22.0git), the ELF dynamic linker searches the global scope first when resolving relocations. LLVM function calls in libIREECompiler.so that go through the PLT/GOT resolve to libLLVM.so's copies rather than the compiler's own static LLVM. This causes LLVM cl::Option static initializers to register duplicate options into libLLVM.so's global registry, crashing with: LLVM ERROR: Option 'pbqp' already exists! The fix adds --exclude-libs=ALL to the SharedImpl linker options, which forces all symbols from static archives (.a files) to hidden visibility at link time. Only symbols from object files (.o) compiled with explicit __attribute__((visibility("default"))) annotations — i.e. the IREE and MLIR C API functions marked with IREE_EMBED_EXPORTED / MLIR_CAPI_EXPORTED — remain visible. This reduces exported dynamic symbols from ~274K to ~2.9K. This matches the pattern already used by MLIR's own MLIR-C shared library (mlir/lib/CAPI/CMakeLists.txt): target_link_options(MLIR-C PRIVATE "-Wl,-exclude-libs,ALL") ``` ci-extra: all --------- Signed-off-by: Bangtian Liu <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]> Co-authored-by: Bangtian Liu <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When building
libIREECompiler.soinTheRockI hit the following error when trying to load the library:I'll put the full claude generated description of what happened below but the long and the short of it is
libIREECompiler.soseems to be exporting quite a few symbols when linking LLVM's static.a's. In my case,TheRockhas another library that loadslibLLVM.so, and when loaded it interposes some of the global command line registration machinery. Because of the interposition,libIREECompiler.so's command line registration got routed tolibLLVM.so's; luckily, rather than some incredibly hard to diagnose error, the double command line registration crashes on startup.--exclude-libs,ALLhides symbols from static libraries when linking the finallibIREECompiler.so, preventing LLVM's symbols from being exported and therefore being interposable.Claude generated description
ci-extra: all