Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add --exclude-libs=ALL to libIREECompiler.so shared library#23574

Merged
AaronStGeorge merged 3 commits into
iree-org:mainfrom
AaronStGeorge:p031-iree-compiler-exclude-libs-fix
Feb 27, 2026
Merged

Add --exclude-libs=ALL to libIREECompiler.so shared library#23574
AaronStGeorge merged 3 commits into
iree-org:mainfrom
AaronStGeorge:p031-iree-compiler-exclude-libs-fix

Conversation

@AaronStGeorge
Copy link
Copy Markdown
Contributor

@AaronStGeorge AaronStGeorge commented Feb 24, 2026

When building libIREECompiler.so in TheRock I hit the following error when trying to load the library:

LLVM ERROR: Option 'pbqp' already exists!

I'll put the full claude generated description of what happened below but the long and the short of it is libIREECompiler.so seems to be exporting quite a few symbols when linking LLVM's static .a's. In my case, TheRock has another library that loads libLLVM.so, and when loaded it interposes some of the global command line registration machinery. Because of the interposition, libIREECompiler.so's command line registration got routed to libLLVM.so's; luckily, rather than some incredibly hard to diagnose error, the double command line registration crashes on startup.

--exclude-libs,ALL hides symbols from static libraries when linking the final libIREECompiler.so, preventing LLVM's symbols from being exported and therefore being interposable.


Claude generated description

Add --exclude-libs=ALL to libIREECompiler.so shared library
libIREECompiler.so statically links LLVM/MLIR but exports ~175K internal
symbols via its version script (api_version.ld uses `global: *`). The
visibility design assumes LLVM is compiled with -fvisibility=hidden, but
LLVM's own CMake build only applies -fvisibility-inlines-hidden — not
-fvisibility=hidden — to the vast majority of its compilation units
(2250/2720 LLVM files, 1049/1145 MLIR files lack the flag). This means
all non-inline LLVM symbols default to visibility("default") and leak
through the version script.

When libIREECompiler.so is loaded via dlopen(RTLD_LOCAL) into a process
that already has libLLVM.so in the global scope (e.g. via HIP runtime's
libamd_comgr.so → libLLVM.so.22.0git), the ELF dynamic linker searches
the global scope first when resolving relocations. LLVM function calls
in libIREECompiler.so that go through the PLT/GOT resolve to
libLLVM.so's copies rather than the compiler's own static LLVM. This
causes LLVM cl::Option static initializers to register duplicate options
into libLLVM.so's global registry, crashing with:

  LLVM ERROR: Option 'pbqp' already exists!

The fix adds --exclude-libs=ALL to the SharedImpl linker options, which
forces all symbols from static archives (.a files) to hidden visibility
at link time. Only symbols from object files (.o) compiled with explicit
__attribute__((visibility("default"))) annotations — i.e. the IREE and
MLIR C API functions marked with IREE_EMBED_EXPORTED / MLIR_CAPI_EXPORTED
— remain visible. This reduces exported dynamic symbols from ~274K to
~2.9K.

This matches the pattern already used by MLIR's own MLIR-C shared
library (mlir/lib/CAPI/CMakeLists.txt):

  target_link_options(MLIR-C PRIVATE "-Wl,-exclude-libs,ALL")

ci-extra: all

@AaronStGeorge AaronStGeorge marked this pull request as ready for review February 25, 2026 15:10
libIREECompiler.so statically links LLVM/MLIR but exports ~175K internal
symbols via its version script (api_version.ld uses `global: *`). The
visibility design assumes LLVM is compiled with -fvisibility=hidden, but
LLVM's own CMake build only applies -fvisibility-inlines-hidden — not
-fvisibility=hidden — to the vast majority of its compilation units
(2250/2720 LLVM files, 1049/1145 MLIR files lack the flag). This means
all non-inline LLVM symbols default to visibility("default") and leak
through the version script.

When libIREECompiler.so is loaded via dlopen(RTLD_LOCAL) into a process
that already has libLLVM.so in the global scope (e.g. via HIP runtime's
libamd_comgr.so → libLLVM.so.22.0git), the ELF dynamic linker searches
the global scope first when resolving relocations. LLVM function calls
in libIREECompiler.so that go through the PLT/GOT resolve to
libLLVM.so's copies rather than the compiler's own static LLVM. This
causes LLVM cl::Option static initializers to register duplicate options
into libLLVM.so's global registry, crashing with:

  LLVM ERROR: Option 'pbqp' already exists!

The fix adds --exclude-libs=ALL to the SharedImpl linker options, which
forces all symbols from static archives (.a files) to hidden visibility
at link time. Only symbols from object files (.o) compiled with explicit
__attribute__((visibility("default"))) annotations — i.e. the IREE and
MLIR C API functions marked with IREE_EMBED_EXPORTED / MLIR_CAPI_EXPORTED
— remain visible. This reduces exported dynamic symbols from ~274K to
~2.9K.

This matches the pattern already used by MLIR's own MLIR-C shared
library (mlir/lib/CAPI/CMakeLists.txt):

  target_link_options(MLIR-C PRIVATE "-Wl,-exclude-libs,ALL")

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@AaronStGeorge AaronStGeorge force-pushed the p031-iree-compiler-exclude-libs-fix branch from bfc5f56 to 020b875 Compare February 25, 2026 22:11
@Hardcode84
Copy link
Copy Markdown
Contributor

I posted about the linker script initially, but I see iree is using it already but in some weird way

@MaheshRavishankar
Copy link
Copy Markdown
Collaborator

I think all the failures here are unrelated. The clang test failure seems scary. It might be a compiler flake. If this is persists, please file an issue and disable the test.

Add `noaarch64` label support to `iree_is_bytecode_module_test_excluded_by_labels`
and apply it to the `check_regression_dynamic_gather_attention_llvm-cpu` test suite.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@AaronStGeorge AaronStGeorge force-pushed the p031-iree-compiler-exclude-libs-fix branch from d5f4c73 to 2875264 Compare February 27, 2026 03:36
@AaronStGeorge AaronStGeorge merged commit 1fe030b into iree-org:main Feb 27, 2026
51 of 55 checks passed
kimm240 pushed a commit to kimm240/iree that referenced this pull request May 8, 2026
…#23574)

When building `libIREECompiler.so` in `TheRock` I hit the following
error when trying to load the library:

```
LLVM ERROR: Option 'pbqp' already exists!
```

I'll put the full claude generated description of what happened below
but the long and the short of it is `libIREECompiler.so` seems to be
exporting quite a few symbols when linking LLVM's static `.a`'s. In my
case, `TheRock` has another library that loads `libLLVM.so`, and when
loaded it
[interposes](https://maskray.me/blog/2021-05-16-elf-interposition-and-bsymbolic)
some of the global command line registration machinery. Because of the
interposition, `libIREECompiler.so`'s command line registration got
routed to `libLLVM.so`'s; luckily, rather than some incredibly hard to
diagnose error, the double command line registration crashes on startup.

`--exclude-libs,ALL` hides symbols from static libraries when linking
the final `libIREECompiler.so`, preventing LLVM's symbols from being
exported and therefore being interposable.

---

Claude generated description

```
Add --exclude-libs=ALL to libIREECompiler.so shared library
libIREECompiler.so statically links LLVM/MLIR but exports ~175K internal
symbols via its version script (api_version.ld uses `global: *`). The
visibility design assumes LLVM is compiled with -fvisibility=hidden, but
LLVM's own CMake build only applies -fvisibility-inlines-hidden — not
-fvisibility=hidden — to the vast majority of its compilation units
(2250/2720 LLVM files, 1049/1145 MLIR files lack the flag). This means
all non-inline LLVM symbols default to visibility("default") and leak
through the version script.

When libIREECompiler.so is loaded via dlopen(RTLD_LOCAL) into a process
that already has libLLVM.so in the global scope (e.g. via HIP runtime's
libamd_comgr.so → libLLVM.so.22.0git), the ELF dynamic linker searches
the global scope first when resolving relocations. LLVM function calls
in libIREECompiler.so that go through the PLT/GOT resolve to
libLLVM.so's copies rather than the compiler's own static LLVM. This
causes LLVM cl::Option static initializers to register duplicate options
into libLLVM.so's global registry, crashing with:

  LLVM ERROR: Option 'pbqp' already exists!

The fix adds --exclude-libs=ALL to the SharedImpl linker options, which
forces all symbols from static archives (.a files) to hidden visibility
at link time. Only symbols from object files (.o) compiled with explicit
__attribute__((visibility("default"))) annotations — i.e. the IREE and
MLIR C API functions marked with IREE_EMBED_EXPORTED / MLIR_CAPI_EXPORTED
— remain visible. This reduces exported dynamic symbols from ~274K to
~2.9K.

This matches the pattern already used by MLIR's own MLIR-C shared
library (mlir/lib/CAPI/CMakeLists.txt):

  target_link_options(MLIR-C PRIVATE "-Wl,-exclude-libs,ALL")
  ```
  
ci-extra: all

---------

Signed-off-by: Bangtian Liu <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Bangtian Liu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants