@llvm/pr-subscribers-bolt

Author: Amir Ayupov (aaupov)

diff --git a/bolt/test/X86/callcont-fallthru.s b/bolt/test/X86/callcont-fallthru.s
index ee72d8f62e032..6b5caa08d3128 100644
--- a/bolt/test/X86/callcont-fallthru.s
+++ b/bolt/test/X86/callcont-fallthru.s
@@ -9,7 +9,7 @@
 # RUN: link_fdata %s %t %t.pa3 PREAGG3
 # RUN: link_fdata %s %t %t.pat PREAGGT1
 # RUN: link_fdata %s %t %t.pat2 PREAGGT2
-# RUN: link_fdata %s %t %t.patplt PREAGGPLT
+# RUN: link_fdata %s %t %t.patplt PREAGGPLT --nmtool llvm-nm
 
 ## Check normal case: fallthrough is not LP or secondary entry.
 # RUN: llvm-strip --strip-unneeded %t -o %t.strip

@MaskRay – can you please advise how to force a PLT entry if linking with a DSO hack doesn't work?

Hey Amir,

Thanks for the PR. Unfortunately, it is still failing. The trick below doesn't seem to work on my buildbot machine:

Link against a DSO to ensure PLT entries.

So doing:

nm --synthetic callcont-fallthru.s.tmp

won't list a puts@plt symbol, which is what causes an link_fdata.py assertion:

AssertionError: ERROR: symbol puts@plt is not defined in binary

On my dev AArch64 instance --synthetic does the trick. BTW run lines 4 and 6 appear identical when inspected (-###)

You need a libc.so that defines puts, and then creates an executable that references puts and links against libc.so. Then the executable will have a PLT entry, and you do not need the --unresolved-symbols=ignore-all hack.

Thanks a lot both! In case there's some delay in resolving this edge case, may I suggest temporarily disabling this test on AArch64 until a more consistent workaround is in place?

Hey folks, any updates on this?

I spent some time experimenting with @MaskRay's suggestion. I used a mock libc shared object that had a puts symbol.
Indeed there won't be unresolved symbols now, however, still GNU nm doesn't show a PLT entry when using --synthetic .

Maybe I'm missing something but why use /dev/null library hack in the first place here? There is stub.c available with puts symbol already, just use it to compile so and link against it, plt entry should appear normally.

Hey @yota9, thanks for the input. I tried something similar.
Even when I use stub.c and link it with:

-# RUN: %clang %cflags -fpic -shared -xc /dev/null -o %t.so
-## Link against a DSO to ensure PLT entries.
+# RUN: %clang %cflags %p/../Inputs/stub.c -fPIC -shared -o %t.so

then running GNU nm:

nm %t --synthetic

would emit only

                 U puts

which link_fdata rejects. On some other machines though, GNU nm emits:

                 U puts
0000000000001234 T puts@plt

which works well. In both cases it was the same nm driver version.
TMU this inconsistency was reported on x86 machines too.

I might've missed something on my end. I briefly discussed this with Amir (see discord) as I'm trying to unblock our AArch64 buildbot. We figured it's fine to disable this test on AArch64 until the issue gets resolved. Could you mind taking a look at #137831, and consider accepting it?

@paschalis-mpeis Could you please check if the binaries are identical and it is indeed nm problem? E.g. with objdump, is plt entry is there? Maybe there is problem related to the plt section type, e.g. one of the binaries has .plt.sec or .plt.got section and there is some kind of but in nm that not lists symbols from these sections. Then we can use custom linker script with .plt section only.
My next suggestion would be just using llvm-nm here. Pass llvm-nm with --nmtool arg to link_fdata.py, since the lit.cfg.py has it in the list of mandatory tools for bolt testing, so we won't have environment dependencies here. Maybe even add nmtool as an link_fdata_cmd arg in lit.cfg.py , so all tests would use it by default...

Hey @yota9, thanks for the suggestions!

Indeed, the PLT entries exist in both binaries. For example running:

build/bin/llvm-objdump -d -j .plt build/tools/bolt/test/X86/Output/callcont-fallthru.s.tmp

shows:

build/tools/bolt/test/X86/Output/callcont-fallthru.s.tmp:       file format elf64-x86-64

Disassembly of section .plt:
0000000000001430 <.plt>:
    1430: ff 35 f2 20 00 00             pushq   0x20f2(%rip)            # 0x3528 <puts+0x3528>
    1436: ff 25 f4 20 00 00             jmpq    *0x20f4(%rip)           # 0x3530 <puts+0x3530>
    143c: 0f 1f 40 00                   nopl    (%rax)

0000000000001440 <puts@plt>:
    1440: ff 25 f2 20 00 00             jmpq    *0x20f2(%rip)           # 0x3538 <puts+0x3538>
    1446: 68 00 00 00 00                pushq   $0x0
    144b: e9 e0 ff ff ff                jmp     0x1430 <.plt>

I noticed some code differences in the binaries but I haven't looked deeper into it.

It looks like it's differences in GNU nm though:

On my AArch64 dev-machine, nm --synthetic lists puts@plt, but when I copy that same binary over to our upcoming AArch64 buildbot, it's missing.

Conversely, nm --synthetic on the buildbot does not list puts@plt, but when if I copy that binary to the dev-machine it does appear.

I too agree that relying on GNU is not ideal. Essentially using any binary tool that does not come from the built LLVM revision. However, llvm-nm does not seem support --synthetic.

BTW, thanks for all the help! I'm focused on AArch64, so while I may be involved to some extent with this, I'll let Amir drive the fix. That's why I'm looking for a code owner to get #137831 stamped. :)
(also cc'ing: @aaupov, @maksfb)

Thanks for tracking it down, looks like it's an issue with GNU nm. However llvm-nm has no functionality equivalent to nm --synthetic which prints the address of the PLT entry, and the test relies on that.

Let me try to decouple this test from GNU nm.

Great. A quick way to use an llvm tool could be:

llvm-objdump -d -j .plt %t | grep @plt

This produces output similar to what nm --synthetic produces (when it works):

0000000000001430 <puts@plt>:

You'll need ofc to tweak link_fdata to properly parse symbol+address:

Not sure of any cleaner approach? (@yota9, @MaskRay)

I've decided to add synthetic option to llvm-nm here #138232 . Unfortunately it would take some time, as main maintainer won't be able to review it soon, so probably for now we might just mark the test as XFAIL until then.. Not forgetting to replace nm with llvm-nm

That is perfect and the way we should go forward with this – thanks @yota9.

The problem is that the test is flaky: it passes on most systems but fails on a few.
UsingXFAIL would make my AArch64 buildbot happy but it will cause failures (Unexpectedly Passed) on other AArch64 machines I've tested . 🤷‍♂️

That's why I propose restricting this to X86 for now, as a way to unblock us in the meantime:

• [BOLT][test] Disable callcont-fallthru.s on AArch64 #137831

@paschalis-mpeis Indeed, you're right. Let's wait about @aaupov decision then, it LGTM

@paschalis-mpeis I realised that if would change the nm to llvm-nm that we can just mark test as xfail, as it would fail until the patch above would be submitted. This way we would guarantee to have proper changes it test.

Yeap, good idea. I could add XFAIL and modify runline like:

# RUN: link_fdata %s %t %t.patplt PREAGGPLT --synthetic --nmtool=llvm-nm

The differences would be :

• with REQUIRES we won't cross-run this x86 lit test on AArch64 (as I do currently in #137831)
• with XFAIL + llvm-nm the test would be expected to fail on both architectures. But once your work is merged, it would unexpectedly pass, which would break the test and prompt us to update it

I'm happy to proceed with this as well.

Yeah, that's right. Although maybe we need to replace nm to llvm-nm in link_fdata to be default... Up to you and @aaupov to decide..

Yes, and that'd actually be better so we don't depend on whatever host GNU nm the machine has.
Based on this, I'd say @aaupov intends to make this change too. I think he's away – let's see what he says once back.