Quarkslab's blog - Clang

Obfuscation vs the Optimizer: An LLVM Middle-End Arms Race

2026-04-16T00:00:00+02:00

Introduction

Obfuscation is security through obscurity; its purpose is to transform a piece of code into a much more complex representation, whilst preserving the original semantics of the code. A compiler's job is to transform source code into binary code and produce the simplest and most optimized representation it can for a given architecture. These are contrary goals, yet this contradiction is where obfuscators find their greatest leverage.

In this blog post, we will explore the relationship between compilers, obfuscation, and de-obfuscation. We will first learn about LLVM, but I will frame the information so it's a little deeper and more relevant to this topic. Finally, we will walk through an example of obfuscation and watch the tug-of-war between our code and the optimization passes and see how a single commit in LLVM breaks our obfuscation. Hopefully, by the end, we will have a better understanding of how this tug-of-war is, in fact, more of a yin-yang.

Meet the mystery function

The star of the blog will be the following function. We will watch how the compiler removes the obfuscation, and we will try to fight back.

#include <stdint.h>

uint8_t mystery(void) {
    return (uint8_t)(
        ((((40u ^ 0xFFu) | 0x9Bu) & 65u) +
         (((0u - (40u & 110u)) - 1u) | 81u) +
         (40u & 110u) - 65u) ^
        0xFFu
    );
}

Before we watch LLVM tear this down, here’s the minimal background you need.

A quick LLVM primer

LLVM is a framework for building compilers. A collection of reusable components helps the author build up their compiler stages.

A compiler is often described in 3 stages: Front-End / Middle-End / Back-End. The so-called middle-end is the stage of compilation where transformations and analyses take place to support optimizations.

Before that, it is the front-end's responsibility to parse the natural source code language into an abstract syntax tree AST. It is then lowered into an intermediate representation IR. In the reverse engineering world, it is sometimes referred to as an intermediate language, but both IL and IR are used.

The IR is an important state because its aim is to represent the semantics of the source language in a way that enables code to reason about its behaviour and perform optimizations. IR is target independent and therefore, in theory, generic and simple.

The IR is eventually passed to the back end; it's here that further lowering occurs into more target-selected architectures, and eventual instructions are selected to generate binary code, such as X86. The beauty in this architecture is that you can have many input languages and many output architectures. Still, the middle-end works to optimize the same IR using a large collection of complex analysis and transformation passes that don't break the semantics of the code, helping the back-end produce fast and/or small code.

Try it yourself

In this blog, we will be working with IR snippets, and knowing how to generate and work with these files would be useful.

We can generate IR from C or C++ code using clang:

clang hello.c -S -emit-llvm -o hello.ll

or, to disable all optimizations:

clang -O0 -Xclang -disable-O0-optnone hello.c -S -emit-llvm -o hello.ll

To run optimization pipelines or specific passes:

opt hello.ll -O2 -S
opt hello.ll -passes=sroa,mem2reg -S

-O0 -O1 -O2 -O3 are optimization levels, and these options trigger a ready-to-use arrangement of passes in a pipeline.

To generate object files:

llc -filetype=obj hello.ll -o hello.o

You can also use these tools with Compiler Explorer

Why the middle-end matters for both sides

LLVM’s middle-end is the product of decades of compiler research made concrete: Theory turned into analyses, algorithms into passes, and ideas refined through real implementation work. That makes it a rich source of knowledge for both reverse engineers and obfuscator authors. If we can produce code that remains difficult for these passes to simplify or reason about, that suggests the obfuscation is doing its job. On the other hand, if we can bring similar algorithms to RE tooling, then we have the beginnings of a capable de-obfuscator. The same machinery can help either hide intent or recover it.

As you saw earlier, we can run passes on the LLVM IR from the command line. LLVM has several passes, although that's a bit of an understatement. The LLVM pass list is split into analysis, transformation, and utility passes. They aim to eliminate unnecessary computation through methods such as dead code elimination, redundancy removal, control-flow simplification, memory optimizations, and much more.

On the flip side we could also write our own passes to introduce the exact opposite.

The optimizer's toolkit

In the context of reverse engineering, obfuscation, and de-obfuscation, I would categorise them by their effect on simplification. These categories help understand how compiler optimizations reduce code complexity, the same mechanisms that make optimization/de-obfuscation possible. Here is an extremely brief look at a few passes and my own groupings. (Inter-procedural analysis is purposely left out)

Dead Code/Store Elimination -> DSEPass DCEPass BDCEPass Removes code that does not affect program output. Obfuscators often insert junk code, opaque predicates, or unreachable paths. DCE passes eliminate these.
Constant Propagation & Folding -> SCCPPass CorrelatedValuePropagationPass Evaluates expressions at compile time and propagates known values. Defeats obfuscation that relies on dynamic computation of constants (opaque predicates, encoded values).
Control Flow Simplification -> SimplifyCFGPass JumpThreadingPass Simplifies the control flow graph by merging blocks, removing redundant branches, and threading jumps. Critical for defeating control flow flattening and bogus control flow.
Redundancy Elimination -> GVNPass EarlyCSEPass Removes redundant computations. Removes duplicate expressions or equivalent computations inserted by obfuscators across different code paths.
Instruction Simplification & Combining -> InstCombinePass ReassociatePass Simplifies and canonicalises instructions. Defeats arithmetic obfuscation (MBA expressions, substitution patterns, identity operations). A bit of a swiss army knife pass, I highly recommended looking through the code of this one.
Memory Optimization -> SROAPass MemCpyOptPass Optimizes memory access patterns. Simplifies obfuscation that on purpose routes values through stack and memory rather than direct access in registers.

The arms race

Now that we know some of the tools, let's watch some of them in action.

Since the middle-end is designed to be generic, it's a great place to optimize code; we could, in fact, de-optimize it, or in more familiar terms, we could obfuscate it. Our obfuscation should be resistant to LLVM's optimization pipelines at a bare minimum. Let's take a piece of already obfuscated code that could have been generated by a beginners pass and see how we fare against the optimization pipeline.

Round 1 — all constants, no contest

Back to our mystery function, let's work with it in LLVM IR form:

define i8 @mystery() {
  %notx = xor i8 40, -1
  %a = or i8 %notx, -101
  %b = and i8 %a, 65
  %c = and i8 40, 110
  %neg = sub i8 0, %c
  %comp = sub i8 %neg, 1
  %d = or i8 %comp, 81
  %sum1 = add i8 %b, %d
  %sum2 = add i8 %sum1, %c
  %sum3 = add i8 %sum2, -65
  %r = xor i8 %sum3, -1
  ret i8 %r
}

The syntax of LLVM IR is quite assembly like. Here is a more 1:1 C version to help with understanding how to read the IR.

#include <stdint.h>

uint8_t mystery(void) {
    uint8_t notx = (uint8_t)(40u ^ 0xFFu);
    uint8_t a    = (uint8_t)(notx | (uint8_t)-101);
    uint8_t b    = (uint8_t)(a & 65u);
    uint8_t c    = (uint8_t)(40u & 110u);
    uint8_t neg  = (uint8_t)(0u - c);
    uint8_t comp = (uint8_t)(neg - 1u);
    uint8_t d    = (uint8_t)(comp | 81u);
    uint8_t sum1 = (uint8_t)(b + d);
    uint8_t sum2 = (uint8_t)(sum1 + c);
    uint8_t sum3 = (uint8_t)(sum2 + (uint8_t)-65);
    uint8_t r    = (uint8_t)(sum3 ^ 0xFFu);
    return r;
}

The code appears to be a function that returns an 8-bit integer. We need to understand the contents of this function; it's very opaque and difficult to reason about what the result should be, and it's successfully obfuscated.

Let's see what happens when we run an O2 optimization pipeline on this. We shall use LLVM 18 and its tool opt, which allows us to run pipelines and passes.

opt sample01.ll -O2 -S

; ModuleID = 'sample01.ll'
source_filename = "sample01.ll"

; Function Attrs: mustprogress nofree norecurse nosync nounwind willreturn memory(none)
define noundef i8 @mystery() local_unnamed_addr #0 {
  ret i8 0
}

attributes #0 = { mustprogress nofree norecurse nosync nounwind willreturn memory(none) }

Round 2 — why it collapsed instantly

The code was complex, but the optimization process quickly discovered the result, revealing the mystery. The function returns 0. The function is simplified because the code can be seen as a constant expression, and the optimization pipeline fully folded it.

We don't even need to run an entire O2 pipeline on it because the pass responsible for this is only EarlyCSEPass. We can achieve the same result with: opt sample01.ll -passes=early-cse -S

If you wish to follow along, then you can use compiler explorer On the left side, choose LLVM IR and on the right side, choose opt 18.1.0 and add the compiler options -O2. Also, click Add New and Opt Pipeline

The pass instcombine could also achieve this, but "Early Common Subexpression Elimination" was run first and easily saw through the code, evaluating it as a constant expression. The pass knows that the first instruction %notx = xor i8 40, -1 is the same as a not, so %notx could be replaced with %notx = 0xD7. Therefore %a = or i8 %notx, -101 is %a = 0xDF, and so on so forth until the whole thing folds down to our 0.

Modern RE tools will also easily see through this; they lift assembly code into their own IR in order to optimize and reason about it for the final decompiler layer.

For example, take this Binary Ninja snippet. It shows data flow tracking in its disassembly view within the {}, and the folding happens line by line:

0x00400000  b0ff               mov     al, 0xff
0x00400002  3428               xor     al, 0x28  {0xd7}
0x00400004  0c9b               or      al, 0x9b  {0xdf}
0x00400006  2441               and     al, 0x41
0x00400008  b16e               mov     cl, 0x6e
0x0040000a  80e128             and     cl, 0x28
0x0040000d  31d2               xor     edx, edx  {0x0}
0x0040000f  28ca               sub     dl, cl  {0xd8}
0x00400011  80ea01             sub     dl, 0x1  {0xd7}
0x00400014  80ca51             or      dl, 0x51  {0xd7}
0x00400017  00d0               add     al, dl  {0x18}
0x00400019  00c8               add     al, cl  {0x40}
0x0040001b  04bf               add     al, 0xbf  {0xff}
0x0040001d  34ff               xor     al, 0xff  {0x0}    <-----
0x0040001f  c3                 retn     {__return_addr}

1) As an author of obfuscation, we have learnt that a linear set of simple instructions that use constant values can easily be broken. 2) As a reverse engineer trying to de-obfuscate code, we have learnt that proving that something is constant is very important. When something is constant, it will have a cascading impact on analysis.

If you are a C++ coder, you might remember being taught that you should be setting variables and class members as const wherever possible. Marking things const informs the compiler what cannot change, thereby enabling stronger optimizations. The same principle applies to RE tools, where asserting immutability improves analysis and de-obfuscation.

Round 3 — hiding behind a variable

Let's improve upon our example to make it stronger. We need to somehow prevent the compiler from knowing something is constant. In our example, we have the value 40 twice; we could replace this with an instance an unknown value.

Our first instinct might be:

define i8 @mystery() {
  %unknown = call i8 asm "", "=r"()
  %notx = xor i8 %unknown, -1
  %a = or i8 %notx, -101
  %b = and i8 %a, 65
  %c = and i8 %unknown, 110
  %neg = sub i8 0, %c
  %comp = sub i8 %neg, 1
  %d = or i8 %comp, 81
  %sum1 = add i8 %b, %d
  %sum2 = add i8 %sum1, %c
  %sum3 = add i8 %sum2, -65
  %r = xor i8 %sum3, -1
  ret i8 %r
}

The new version of the code now uses a random register and breaks the optimizer, and also our reversing tools. The random use of a register does stick out, since it appears out of nowhere and looks like uninitialised use; we can do better.

Such as interweaving our expression into the existing code, for instance using an existing variable in the program. Since this contrived example doesn't have one, I will add a parameter to the function and use that instead.

define i8 @mystery(i8 %arg1) {
  %notx = xor i8 %arg1, -1
  %a = or i8 %notx, -101
  %b = and i8 %a, 65
  %c = and i8 %arg1, 110
  %neg = sub i8 0, %c
  %comp = sub i8 %neg, 1
  %d = or i8 %comp, 81
  %sum1 = add i8 %b, %d
  %sum2 = add i8 %sum1, %c
  %sum3 = add i8 %sum2, -65
  %r = xor i8 %sum3, -1
  ret i8 %r
}

The new code is now a mix of variables, arithmetic, and bitwise operations; this is known as Mixed Boolean Arithmetic (MBA), and our example is, in fact, a semi-linear MBA used for constant obfuscation.

Now the optimizer can't figure out that this is a constant expression (even if it simplifies it a bit):

opt sample02.ll -O2 -S

define i8 @mystery(i8 %arg1) local_unnamed_addr #0 {
  %c = and i8 %arg1, 110
  %comp = xor i8 %c, -1
  %d = or i8 %comp, 81
  %1 = or i8 %arg1, -65
  %sub.neg = add nsw i8 %1, 64
  %2 = add nsw i8 %c, %d
  %r = sub nsw i8 %sub.neg, %2
  ret i8 %r
}

When viewed inside a decompiler the new complex expression now looks part of the functionality of the program. The interweaving with an existing value makes it hard for the decompiler to reason about the code. This is where using features in your reverse engineering tools to inform the decompiler about the state of certain values will help you to de-obfuscate this.

For now, the mystery value is once again secure; we require more work to figure it out before we can know the answer again.

Round 4 — version shock LLVM 18 vs LLVM 19

Up until now, we have been testing with LLVM version 18.1.8, but some time has passed in our contrived scenario, and we now have access to llvm 19 LLVM version 19.1.7. Let's rerun our command opt sample02.ll -O2 -S

; ModuleID = 'sample02.ll'
source_filename = "sample02.ll"

; Function Attrs: mustprogress nofree norecurse nosync nounwind willreturn memory(none)
define noundef range(i8 -110, 112) i8 @mystery(i8 %arg1) local_unnamed_addr #0 {
  ret i8 0
}

attributes #0 = { mustprogress nofree norecurse nosync nounwind willreturn memory(none) }

Wait ...what happened? The upgraded LLVM version can now reverse our encoded secret. If we run once more opt sample02.ll -passes=early-cse -S we get:

define i8 @mystery(i8 %arg1) {
  %notx = xor i8 %arg1, -1
  %a = or i8 %notx, -101
  %b = and i8 %a, 65
  %c = and i8 %arg1, 110
  %neg = sub i8 0, %c
  %comp = sub i8 %neg, 1
  %d = or i8 %comp, 81
  %sum1 = add i8 %b, %d
  %sum2 = add i8 %sum1, %c
  %sum3 = add i8 %sum2, -65
  %r = xor i8 %sum3, -1
  ret i8 %r
}

From the pipeline, we can see it's not the early-cse pass reverting our changes, but something new! We can figure out the exact cause for the optimization through the compiler explorer opt pipeline viewer.

opt sample02.ll -passes=instcombine,reassociate,instcombine,gvn,bdce -S

; ModuleID = 'sample02.ll'
source_filename = "sample02.ll"

define i8 @mystery(i8 %arg1) {
  ret i8 0
}

A chain of just 5 passes: InstCombine, Reassociate, InstCombine again, GVN, and BDCE, is all it takes to unravel the expression down to zero. The culprit that triggers this is a single commit that landed in LLVM 19, adding several lines of code to InstCombine's getFreelyInvertedImpl function.

The new change is an example of the middle-end evolving and finding ways to augment optimization. The change teaches the pass to apply De Morgan's Law, ~(A | B) → (~A & ~B), allowing it to push a bitwise NOT recursively through OR and AND operations. Our obfuscation relied on exactly this: a final NOT tangled through nested ORs that the compiler couldn't see through. With DeMorgan inversion, the NOT layers peel away, and the expression flattens into a form where both sides of a subtraction are visibly identical. The compiler folds x - x to zero. A single rule of boolean algebra that LLVM 19 learned to apply collapsed our obfuscated expression. A beautiful example of how a seemingly small algebraic rule can unlock a much larger simplification

Round 5 — one constant away from survival

This is the arms race; obfuscation techniques that exploit gaps in compiler reasoning have an expiry date. The middle-end only gets smarter with each release.

One last fun remark, if we change both 65s in our expression to 66 (and -65 to -66) like so:

define i8 @mystery(i8 %arg1) {
  %notx = xor i8 %arg1, -1
  %a = or i8 %notx, -101
  %b = and i8 %a, 66
  %c = and i8 %arg1, 110
  %neg = sub i8 0, %c
  %comp = sub i8 %neg, 1
  %d = or i8 %comp, 81
  %sum1 = add i8 %b, %d
  %sum2 = add i8 %sum1, %c
  %sum3 = add i8 %sum2, -66
  %r = xor i8 %sum3, -1
  ret i8 %r
}

then it's enough to defeat the llvm 19 change. The expression still returns zero for every input, but the altered constants misalign the bit masks that InstCombine needs for its algebraic cancellation; the XOR residue left behind poisons the entire simplification chain. Not even the opt version 22.1.0 can fold it, even more intriguing.

The yin-yang

This post was a very simple primer on the topic, but it demonstrates that staying ahead means understanding not just what the compiler can do today, but what it will be able to do tomorrow. Whether you are building obfuscation or building tools to break it, the knowledge is the same: understanding how optimization passes reason about code is the foundation for both sides of the game.

That's the yin-yang at the heart of this whole story: the same machinery that helps hide intent can also reveal it, and each side sharpens the other over time. Better obfuscation pressures optimizers and analysis tools to evolve, while better optimization and de-obfuscation force obfuscation to become more thoughtful and less fragile. They are not opposites moving apart; they are complementary forces in the same cycle, and understanding that cycle is what makes you dangerous on either side.

If you made it this far, then I thank you for your time and hope you enjoyed the post :)

Acknowledgments

Thanks to Béatrice Creusillet for her thorough review of my post. To Jean François for his encouragement, support and general jolliness :)

Clang Hardening Cheat Sheet - Ten Years Later

2026-01-08T00:00:00+01:00

Introduction

Ten years ago, we published on this blog a Clang Hardening Cheat Sheet. The original post walked through essential hardening techniques available at the time, such as FORTIFY_SOURCE checks, ASLR via position-independent code, stack protection (canaries and safe stack), Control Flow Integrity (CFI), GOT protection with RELRO/now, but also options to activate warnings about string formatting that could lead to potential attacks.

Since that article was published in early 2016, both the threat landscape and the Clang toolchain have evolved significantly.

To celebrate the 10th anniversary of the initial article, here is a new cheat sheet with some new hardening flags to improve security.

TL;DR

The OpenSSF Best Practices Working Group maintains a Compiler Options Hardening Guide for C and C++. They recommend using the following set of options:

-O2 -Wall -Wformat -Wformat=2 -Wconversion -Wimplicit-fallthrough \
-Werror=format-security \
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 \
-D_GLIBCXX_ASSERTIONS \
-fstrict-flex-arrays=3 \
-fstack-clash-protection -fstack-protector-strong \
-Wl,-z,nodlopen -Wl,-z,noexecstack \
-Wl,-z,relro -Wl,-z,now \
-Wl,--as-needed -Wl,--no-copy-dt-needed-entries

The options we recommended in our original blog post are still present, even if some have evolved (-Wformat=2, -D_FORTIFY_SOURCE=3). Others have been added:

to enable some compiler warnings: -Wconversion, -Wimplicit-fallthrough;
to limit the attack surface by only linking against necessary libraries: -Wl,--as-needed -Wl,--no-copy-dt-needed-entries;
to harden the code against attacks: -D_GLIBCXX_ASSERTIONS, -fstrict-flex-arrays=3, -fstack-clash-protection, -Wl,-z,nodlopen, -Wl,-z,noexecstack.

In this post, we present the additional hardening options recommended by the OpenSSF, as well as more specialized options that mitigate newer classes of exploits. We first go through several general protections when using the standard C/C++ libraries or loading libraries. Then we present mitigations against stack-based memory corruption, and the use of Return-Oriented Programming (ROP) or Jump-Oriented Programming (JOP) attacks. And finally, we talk about the defenses against speculative execution attacks.

General Protections

Fortify

Since last time, the -D_FORTIFY_SOURCE flag has evolved. It now provides a level 3¹, which includes all the security features of levels 1 and 2, along with additional checks for potentially dangerous code patterns. These additional checks are designed to detect a wider range of security issues, including:

Unsafe usage of memcpy and memmove
Risky use of snprintf, vsnprintf, and related functions
Potentially dangerous string manipulation functions such as strtok, strncat, and strpbrk

C++ Library Hardening

In addition to -D_FORTIFY_SOURCE, C++ developers can enable extra runtime checks in the standard library by defining -D_GLIBCXX_ASSERTIONS.

This option is one of several supported by the libstdc++ library and it is used to enable various NULL pointer and bounds checking security features.

nodlopen

The -Wl,-z,nodlopen linker flag is used to prevent a shared object from being dynamically loaded at runtime using dlopen().

This can help in reducing an attacker’s ability to load and manipulate shared objects.

To test this flag, let's compile a simple shared object:

#include <stdio.h>

void example_function(void)
{
    printf("Hello from the dynamically loaded library!\n");
}

We compile it with the nodlopen linker flag:

clang -fPIC -c src/libexample.c -o libexample.o
clang -Wl,-z,nodlopen -shared libexample.o -o libexample.so

Now we can attempt to load it at runtime using dlopen:

#include <stdio.h>
#include <dlfcn.h>

int main(void)
{
    void *handle;
    void (*func)(void);

    /* Load the shared library */
    handle = dlopen("./libexample.so", RTLD_NOW);
    if (!handle) {
        fprintf(stderr, "dlopen failed: %s\n", dlerror());
        return 1;
    }

    /* Resolve the symbol */
    func = (void (*)(void))dlsym(handle, "example_function");
    if (!func) {
        fprintf(stderr, "dlsym failed: %s\n", dlerror());
        dlclose(handle);
        return 1;
    }

    /* Call the function */
    func();

    /* Close the library */
    dlclose(handle);

    return 0;
}

When we run this program, we get the following error:

╭─trikkss@archlinux ~/work/clang-hardening/tests/nodlopen  ‹main*›
╰─➤  ./main
dlopen failed: ./libexample.so: shared object cannot be dlopen()ed

This demonstrates that the nodlopen flag effectively prevents the shared library from being loaded dynamically at runtime. Note that this does not prevent the library from being linked at build time:

╭─trikkss@archlinux ~/work/clang-hardening/tests/nodlopen  ‹main*›
╰─➤  cat src/main2.c
#include <stdio.h>

/* declaration of the library function */
void example_function(void);

int main(void)
{
    example_function();
    return 0;
}

╭─trikkss@archlinux ~/work/clang-hardening/tests/nodlopen  ‹main*›
╰─➤  clang -o main2 src/main2.c $PWD/libexample.so
╭─trikkss@archlinux ~/work/clang-hardening/tests/nodlopen  ‹main*›
╰─➤  ldd main2
    linux-vdso.so.1 (0x00007fec50fe3000)
    /home/trikkss/work/clang-hardening/tests/nodlopen/libexample.so (0x00007fec50fd1000)
    libc.so.6 => /usr/lib/libc.so.6 (0x00007fec50c00000)
    /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fec50fe5000)
╭─trikkss@archlinux ~/work/clang-hardening/tests/nodlopen  ‹main*›
╰─➤  ./main2
Hello from the dynamically loaded library!

If we inspect the dynamic section of the library, and compare it to the dynamic section of the same library generated without nodlopen, we can see that the main difference is the presence of the flag NOOPEN:

2c2
< Dynamic section at offset 0x2e08 contains 24 entries:
---
> Dynamic section at offset 0x2df8 contains 25 entries:
7c7
<  0x0000000000000019 (INIT_ARRAY)         0x3df8
---
>  0x0000000000000019 (INIT_ARRAY)         0x3de8
9c9
<  0x000000000000001a (FINI_ARRAY)         0x3e00
---
>  0x000000000000001a (FINI_ARRAY)         0x3df0
22a23
>  0x000000006ffffffb (FLAGS_1)            Flags: NOOPEN

The protection may therefore be easy to bypass: if an attacker has write access to the library in question, they can easily patch it.

Defenses Against Stack-Based Memory Corruption

Non-Executable Stack (NX)

One of the oldest and simplest exploit mitigation mechanisms is the non-executable stack. The idea is to prevent code execution directly from the stack, which was a common exploitation technique in early stack-based buffer overflow attacks.

When a binary is compiled or linked with the noexecstack flag, the stack memory region is marked as non-executable. As a result, even if an attacker manages to inject shellcode onto the stack and redirect execution to it, the CPU will refuse to execute it and will raise a fault.

Today, this protection is enabled by default on most modern systems and toolchains.

Stack clash vulnerabilities

During its execution, a program uses the stack memory region to store data. This memory region is special because it grows automatically as the program needs more stack memory. To prevent uncontrolled growth, modern operating systems place guard pages below the stack, which trigger a fault when the stack grows too far.

A stack clash occurs when the stack grows by a very large amount at once and skips over these guard pages. As a result, the stack can collide with another memory region, such as the heap or a memory-mapped area, without triggering an immediate fault. This can cause stack writes to corrupt adjacent memory regions, or allow other memory regions to overlap with the stack, leading to memory corruption and potential exploitation.

The Stack Clash vulnerability was publicly disclosed on June 19, 2017 by the Qualys research team².

Stack Clash Protection

LLVM's solution to this problem is to divide large allocations into smaller ones of size PAGE_SIZE as described in this blog post. This can be done by adding the -fstack-clash-protection compilation flag.

To observe this behavior, we can compile a simple C program that allocates a large buffer on the stack:

#include <stdio.h>
#include <sys/user.h>

int main(void)
{
    char buffer[PAGE_SIZE * 10];
    printf("hello world\n");

    return 0;
}

Without stack clash protection enabled, the generated assembly looks like this:

0000000000401130 <main>:
  401130:   55                      push   rbp
  401131:   48 89 e5                mov    rbp,rsp
  401134:   48 81 ec 10 a0 00 00    sub    rsp,0xa010
  40113b:   c7 45 fc 00 00 00 00    mov    DWORD PTR [rbp-0x4],0x0
  401142:   48 8d 3d bb 0e 00 00    lea    rdi,[rip+0xebb]        # 402004 <_IO_stdin_used+0x4>
  401149:   b0 00                   mov    al,0x0
  40114b:   e8 e0 fe ff ff          call   401030 <printf@plt>
  401150:   31 c0                   xor    eax,eax
  401152:   48 81 c4 10 a0 00 00    add    rsp,0xa010
  401159:   5d                      pop    rbp
  40115a:   c3                      ret

Here, we can clearly identify a single large stack allocation performed by the instruction sub rsp, 0xa010, which decreases the stack pointer by more than ten pages at once.

Now, we can take a look at the hardened version of this program:

0000000000401140 <main>:
  401140:   55                      push   rbp
  401141:   48 89 e5                mov    rbp,rsp
  401144:   49 89 e3                mov    r11,rsp
  401147:   49 81 eb 00 a0 00 00    sub    r11,0xa000
  40114e:   48 81 ec 00 10 00 00    sub    rsp,0x1000
  401155:   48 c7 04 24 00 00 00    mov    QWORD PTR [rsp],0x0
  40115c:   00
  40115d:   4c 39 dc                cmp    rsp,r11
  401160:   75 ec                   jne    40114e <main+0xe>
  401162:   48 83 ec 20             sub    rsp,0x20
  401166:   64 48 8b 04 25 28 00    mov    rax,QWORD PTR fs:0x28
  40116d:   00 00
  40116f:   48 89 45 f8             mov    QWORD PTR [rbp-0x8],rax
  401173:   c7 85 ec 5f ff ff 00    mov    DWORD PTR [rbp-0xa014],0x0
  40117a:   00 00 00
  40117d:   48 8d 3d 80 0e 00 00    lea    rdi,[rip+0xe80]        # 402004 <_IO_stdin_used+0x4>
  401184:   31 c0                   xor    eax,eax
  401186:   e8 b5 fe ff ff          call   401040 <printf@plt>
  40118b:   64 48 8b 04 25 28 00    mov    rax,QWORD PTR fs:0x28
  401192:   00 00
  401194:   48 8b 4d f8             mov    rcx,QWORD PTR [rbp-0x8]
  401198:   48 39 c8                cmp    rax,rcx
  40119b:   75 0b                   jne    4011a8 <main+0x68>
  40119d:   31 c0                   xor    eax,eax
  40119f:   48 81 c4 20 a0 00 00    add    rsp,0xa020
  4011a6:   5d                      pop    rbp
  4011a7:   c3                      ret
  4011a8:   e8 83 fe ff ff          call   401030 <__stack_chk_fail@plt>

We can see that the single stack allocation (sub rsp, 0xa10) has been replaced by a loop (address 40114e) that grows the stack by 0x1000 bytes (size of a memory page) per iteration until the stack pointer reaches the target address, touching each page along the way (mov QWORD PTR [rsp],0x0) to ensure that all guard pages are accessed and no stack guard is skipped.

Defenses Against Code-Reuse Attacks (ROP/JOP)

Code-Reuse Attacks

Modern exploitation techniques rarely rely on injecting new code. Instead, attackers increasingly reuse existing code already present in the binary or in linked libraries. This class of attacks, commonly referred to as code-reuse attacks, bypasses traditional defenses such as non-executable stack memory (NX) by chaining together short instruction sequences called gadgets that end with indirect control-flow transfers.

The most well-known form is Return-Oriented Programming (ROP), where execution is redirected through a sequence of RET instructions by overwriting the return address of a function. Variants such as Jump-Oriented Programming (JOP) and Call-Oriented Programming (COP) rely on indirect jumps or calls instead.

To mitigate these issues, modern defenses focus on controlling where indirect branches are allowed to land, protecting return addresses, or reducing the usefulness of available gadgets. Rather than preventing memory corruption itself, these mechanisms aim to constrain how corrupted control flow can be exploited.

In the following sections, we will examine how Clang compilation flags implement these strategies through hardware-assisted features and compiler-driven transformations, and how they can be combined to harden binaries against code-reuse attacks in practice.

If you want to learn more about how ROP works, I recommend the following blog post: ROP - Return-Oriented Programming by Pixis.

Control-Flow Protection (x86)

Modern x86_64 CPUs from Intel support Control-flow Enforcement Technology (CET), a hardware-assisted mechanism to protect against code-reuse attacks. CET combines two complementary features:

IBT (Indirect Branch Tracking): protects against Jump/Call-Oriented Programming (JOP) attacks.
Shadow Stack (SHSTK): protects against Return-Oriented Programming (ROP) attacks.

Clang exposes CET through the -fcf-protection flag, which can take several values:

-fcf-protection=return: enables shadow stack protection (SHSTK);
-fcf-protection=branch: enables endbranch (EB) generation (IBT);
-fcf-protection=full: enables shadow stack protection and endbranch (EB) generation; this is the same as specifying this compiler option with no keyword;
-fcf-protection=none: disables Intel CET protection.

To study this security mechanism, consider the following trivial function:

void hello_world()
{
    printf("hello world\n");
}

When compiled without CET, the assembly code looks like:

0000000000401130 <hello_world>:
  401130:   55                      push   rbp
  401131:   48 89 e5                mov    rbp,rsp
  401134:   48 8d 3d c9 0e 00 00    lea    rdi,[rip+0xec9]        # 402004 <_IO_stdin_used+0x4>
  40113b:   b0 00                   mov    al,0x0
  40113d:   e8 ee fe ff ff          call   401030 <printf@plt>
  401142:   5d                      pop    rbp
  401143:   c3                      ret

and compiled with -fcf-protection=full, it looks like:

0000000000001150 <hello_world>:
    1150:   f3 0f 1e fa             endbr64
    1154:   55                      push   rbp
    1155:   48 89 e5                mov    rbp,rsp
    1158:   48 8d 3d a5 0e 00 00    lea    rdi,[rip+0xea5]        # 2004 <_IO_stdin_used+0x4>
    115f:   b0 00                   mov    al,0x0
    1161:   e8 da fe ff ff          call   1040 <printf@plt>
    1166:   5d                      pop    rbp
    1167:   c3                      ret

The only difference is the ENDBR64 instruction at the beginning, which acts as a landing pad for indirect branches under IBT.

If we look at the properties of our ELFs, the hardened binary declares support for IBT and SHSTK:

╭─trikkss@archlinux ~/work/clang-hardening/tests/fcf-protection  ‹main*›
╰─➤  readelf -n main-hardened | grep Properties
      Properties: x86 feature: IBT, SHSTK
╭─trikkss@archlinux ~/work/clang-hardening/tests/fcf-protection  ‹main*›
╰─➤  readelf -n main | grep Properties
      Properties: x86 ISA needed: x86-64-baseline

⚠️ Today in the 64-bit linux kernel, only userspace shadow stack and kernel IBT are supported (kernel.org - Control-flow Enforcement Technology (CET) Shadow Stack).

Windows doesn't support Indirect Branch Tracking

Indirect Branch Tracking

When IBT is enabled, the compiler inserts landing pads at the beginning of functions that may be the target of an indirect branch. These landing pads are represented by the end branch instructions (ENDBR32 and ENDBR64). On Intel processors, if this property is enabled, the program can only jump to an end branch. This prevents attackers from using JOP gadgets because they are only allowed to jump to addresses marked by ENDBR instructions.

It is still possible to hijack the program’s control flow, but the options available to an attacker are limited. Additionally, this protection does not prevent ROP attacks.

Shadow Stack

Shadow stack, also referred to as SHSTK, is a backward-edge code flow integrity protection feature available in both Intel and AMD processors. Its main purpose is to protect the integrity of return addresses on the call stack to prevent return-oriented programming (ROP) attacks.

The way it works is conceptually straightforward: every time a CALL instruction is executed, the CPU pushes the return address not only onto the regular stack but also onto a separate, protected “shadow” stack. Later, when a RET instruction is executed, the CPU checks that the return address on the normal stack matches the one on the shadow stack. If the two addresses differ, a fault is triggered, preventing the program from returning to an attacker-controlled location.

Control-Flow Protection (ARM)

Just as x86 systems can be protected using CET, ARM provides equivalent hardware-assisted mechanisms through PAC (Pointer Authentication Codes) and BTI (Branch Target Identification). The Clang flag -mbranch-protection enables these protections, which are designed to prevent code-reuse attacks such as ROP, JOP, and COP on ARM systems.

Clang exposes PAC and BIT through the -mbranch-protection flag, which can take several values:

none : Disables all types of branch protection.
standard : Enables Branch Target Identification (BTI) and Pointer Authentication Code (PAC) branch protection
bti : Enables branch protection using BTI.
pac-ret: Enables branch protection using PAC.

For more information on these flags, you can consult the ARM Developer Manual.

Branch Target Identification (BTI)

Branch Target Identification (BTI) is an architectural hardware security feature that restricts the set of valid destination addresses for indirect branch instructions. By enforcing where indirect calls and jumps are allowed to land, BTI helps mitigate Jump-Oriented Programming (JOP) and Call-Oriented Programming (COP) attacks.

To examine how this mechanism works in practice, we can once again compile a trivial function and inspect the generated assembly code.

Here is the assembly code of a simple hello world compiled without BTI :

STP             X29, X30, [SP,#-0x10+var_s0]!
MOV             X29, SP
ADRL            X0, aHelloWorld ; "hello world\n"
BL              .printf
LDP             X29, X30, [SP+var_s0],#0x10
RET

And here is the hardened version compiled with BTI enabled:

BTI             c
STP             X29, X30, [SP,#-0x10+var_s0]!
MOV             X29, SP
ADRL            X0, aHelloWorld ; "hello world\n"
BL              .printf
LDP             X29, X30, [SP+var_s0],#0x10
RET

The only visible difference is the addition of a BTI instruction at the beginning of the function. This instruction plays a role similar to the ENDBR instruction on x86: every indirect branch must land on a valid BTI instruction, otherwise execution is aborted.

Unlike ENDBR, the BTI instruction takes an operand that specifies which types of indirect branches are allowed to target this location.

The operand controls the value of the PSTATE.BTYPE register, a 2-bit field that encodes the type of indirect control flow:

00 - none
01 - CALLS (c)
10 - JUMPS (j)
11 - JUMPS and CALLS (jc)

In this example, the BTI c instruction indicates that only indirect calls are allowed to branch to this function entry point. Indirect jumps targeting this address would result in a fault.

By enforcing these constraints, BTI significantly reduces the set of valid gadget entry points available to an attacker. While it does not completely prevent control-flow hijacking, it limits exploitation to a restricted set of legitimate targets, making JOP and COP chains much harder to construct reliably.

As with similar mechanisms on x86 (IBT), BTI doesn't protect against return-oriented programming. To achieve full control-flow integrity on ARM systems, BTI can be combined with PAC.

Pointer Authentication Codes (PAC)

Pointer Authentication Codes (PAC) is a hardware-assisted mechanism on ARM64 that protects against ROP by signing return addresses.

When a function is called, PAC generates a cryptographic signature using:

A key register (A or B) provided by the CPU.
A modifier, which can include context such as the stack pointer (SP) to prevent reuse of signed pointers across different functions.

The return address is then signed and stored on the stack. When the function returns, the CPU verifies the signature using the same key and modifier. If the signature is invalid, the program triggers a fault, effectively preventing the execution of an attacker-controlled return address.

Pointer authentication takes advantage of the fact that pointers are stored in a 64-bit format, but not all those bits are needed to represent the address. The virtual address space layout is the following:

kernel space : 0xFFF0_0000_0000_0000 to 0xFFFF_FFFF_FFFF_FFFF
user space : 0x0000_0000_0000_0000 to 0x00FF_FFFF_FFFF_FFFF

Any address that falls outside of both ranges is always invalid and results in a fault if accessed.

You can see that any valid virtual address has its top 12 bits as 0x000 or 0xFFF. When pointer authentication is enabled, the upper bits are used to store a signature and are not treated as part of the address. This signature is referred to as a Pointer Authentication Code (PAC). Its size can change depending on the architecture.

Here is an example taken from the ARM developer Manual - Return-Oriented Programming:

SIGNED POINTER = PAC | POINTER

e.g:

PAC = 0x123
POINTER = 0x0007FFFF5678
SIGNED POINTER = 0x1237FFFF5678

To analyse this security mechanism, we can use our previous hello world example:

STP             X29, X30, [SP,#-0x10+var_s0]!
MOV             X29, SP
ADRL            X0, aHelloWorld ; "hello world\n"
BL              .printf
LDP             X29, X30, [SP+var_s0],#0x10
RET

and compile it with PAC enabled:

PACIASP
STP             X29, X30, [SP,#-0x10+var_s0]!
MOV             X29, SP
ADRL            X0, aHelloWorld ; "hello world\n"
BL              .printf
LDP             X29, X30, [SP+var_s0],#0x10
RETAA

As we can see, two things have changed between the two versions. A PACIASP instruction has been added at the beginning, and the RET instruction has been changed to a RETAA instruction.

The PACIASP instruction signs the return address with key register A and uses SP as a modifier. The RETAA instruction means that the return address must be checked using the key register A.

Several PAC bypasses exist, depending on how pointer authentication is implemented. In some cases, PAC is signed with a null modifier, allowing an attacker to reuse an old authenticated pointer since the signature is not bound to any execution context. Additionally, because the number of bits allocated to the PAC signature varies across architectures, it may sometimes be possible to brute-force the authentication code, among other implementation-specific weaknesses.

If you want to go further, here are some very interesting slides by Brandon Azad on PAC bypasses on iOS: iOS Kernel PAC, One Year Later.

Register Zeroing

Register zeroing is a hardening technique designed to reduce the amount of useful gadgets an attacker can reuse after hijacking the control flow of a program. By explicitly clearing registers before a function returns, register zeroing limits what an attacker can carry from one gadget to the next. Instead of inheriting a rich execution context, each return leaves the program in a mostly clean state, making exploit construction significantly more complex. This helps mitigate Return-Oriented Programming exploits.

Clang provides this mitigation through the -fzero-call-used-regs compilation flag which tells the compiler to zero out certain registers before the function returns.

The two upper categories are:

used: Zero out used registers.
all: Zero out all registers, whether used or not.

The individual options are:

skip: Don't zero out any registers. This is the default.
used: Zero out all used registers.
used-arg: Zero out used registers that are used for arguments.
used-gpr: Zero out used registers that are GPRs.
used-gpr-arg: Zero out used GPRs that are used as arguments.
all: Zero out all registers.
all-arg: Zero out all registers used for arguments.
all-gpr: Zero out all GPRs.
all-gpr-arg: Zero out all GPRs used for arguments.

General-Purpose Registers (GPR) are CPU registers that programs can use freely to hold temporary values, addresses, or intermediate results during execution; for example, on x86-64, registers like RAX or RCX can be set to zero or reused without breaking the program, while RSP is not freely usable because it must always point to the top of the stack.

Let's take a look at a simple hello world program:

0000000000001140 <hello_world>:
    1140:   55                      push   rbp
    1141:   48 89 e5                mov    rbp,rsp
    1144:   48 8d 3d b9 0e 00 00    lea    rdi,[rip+0xeb9]        # 2004 <_IO_stdin_used+0x4>
    114b:   b0 00                   mov    al,0x0
    114d:   e8 de fe ff ff          call   1030 <printf@plt>
    1152:   5d                      pop    rbp
    1153:   c3                      ret

If we compile it with -fzero-call-used-regs=all:

0000000000001140 <hello_world>:
    1140:   55                      push   rbp
    1141:   48 89 e5                mov    rbp,rsp
    1144:   48 8d 3d b9 0e 00 00    lea    rdi,[rip+0xeb9]        # 2004 <_IO_stdin_used+0x4>
    114b:   b0 00                   mov    al,0x0
    114d:   e8 de fe ff ff          call   1030 <printf@plt>
    1152:   5d                      pop    rbp
    1153:   d9 ee                   fldz
    1155:   d9 ee                   fldz
    1157:   d9 ee                   fldz
    1159:   d9 ee                   fldz
    115b:   d9 ee                   fldz
    115d:   d9 ee                   fldz
    115f:   d9 ee                   fldz
    1161:   d9 ee                   fldz
    1163:   dd d8                   fstp   st(0)
    1165:   dd d8                   fstp   st(0)
    1167:   dd d8                   fstp   st(0)
    1169:   dd d8                   fstp   st(0)
    116b:   dd d8                   fstp   st(0)
    116d:   dd d8                   fstp   st(0)
    116f:   dd d8                   fstp   st(0)
    1171:   dd d8                   fstp   st(0)
    1173:   31 c0                   xor    eax,eax
    1175:   31 c9                   xor    ecx,ecx
    1177:   31 ff                   xor    edi,edi
    1179:   31 d2                   xor    edx,edx
    117b:   31 f6                   xor    esi,esi
    117d:   45 31 c0                xor    r8d,r8d
    1180:   45 31 c9                xor    r9d,r9d
    1183:   45 31 d2                xor    r10d,r10d
    1186:   45 31 db                xor    r11d,r11d
    1189:   0f 57 c0                xorps  xmm0,xmm0
    118c:   0f 57 c9                xorps  xmm1,xmm1
    118f:   0f 57 d2                xorps  xmm2,xmm2
    1192:   0f 57 db                xorps  xmm3,xmm3
    1195:   0f 57 e4                xorps  xmm4,xmm4
    1198:   0f 57 ed                xorps  xmm5,xmm5
    119b:   0f 57 f6                xorps  xmm6,xmm6
    119e:   0f 57 ff                xorps  xmm7,xmm7
    11a1:   45 0f 57 c0             xorps  xmm8,xmm8
    11a5:   45 0f 57 c9             xorps  xmm9,xmm9
    11a9:   45 0f 57 d2             xorps  xmm10,xmm10
    11ad:   45 0f 57 db             xorps  xmm11,xmm11
    11b1:   45 0f 57 e4             xorps  xmm12,xmm12
    11b5:   45 0f 57 ed             xorps  xmm13,xmm13
    11b9:   45 0f 57 f6             xorps  xmm14,xmm14
    11bd:   45 0f 57 ff             xorps  xmm15,xmm15
    11c1:   c3                      ret

Here we can clearly see the downside of using the -fzero-call-used-regs=all variant. In order to guarantee that no potentially useful state remains after the function returns, the compiler aggressively clears all classes of registers: general-purpose registers, SIMD registers (XMM), and even the x87 floating‑point stack (fldz, fstp st(0)). While this significantly reduces the availability of useful ROP gadgets, it comes at a high cost.

In practice, such an aggressive mode is often overkill. A more balanced approach is to use -fzero-call-used-regs=used-gpr, which only clears the general‑purpose registers that were actually used by the function.

Here is the code hardened with -fzero-call-used-regs=used-gpr:

0000000000001140 <hello_world>:
    1140:   55                      push   rbp
    1141:   48 89 e5                mov    rbp,rsp
    1144:   48 8d 3d b9 0e 00 00    lea    rdi,[rip+0xeb9]        # 2004 <_IO_stdin_used+0x4>
    114b:   b0 00                   mov    al,0x0
    114d:   e8 de fe ff ff          call   1030 <printf@plt>
    1152:   5d                      pop    rbp
    1153:   31 c0                   xor    eax,eax
    1155:   31 ff                   xor    edi,edi
    1157:   c3                      ret

As you can see, the difference is much more subtle. Only the general‑purpose registers that were actually used by the function (RAX and RDI in this case) are explicitly cleared before returning. This drastically reduces the amount of additional instructions compared to the all variant, while still removing valuable attacker‑controlled state that could otherwise be reused as part of a ROP chain.

If a binary relies on any library (libc, etc.) not compiled with this compilation flag, an attacker can still find a lot of gadgets. That's why this mitigation is most effective for kernel-sized or static binaries, as it does not depend on external libraries.

Defenses Against Speculative Execution Attacks

Speculative Attacks

Modern CPUs use speculative execution to improve performance. When the processor encounters a conditional branch, it may predict the outcome and execute instructions from the predicted path before the branch condition is fully resolved. If the prediction is correct, the results are committed; if it is wrong, the architectural effects are discarded.

However, even when speculative execution is rolled back architecturally, microarchitectural side effects remain, such as changes in cache state. Speculation attacks exploit this behavior by deliberately causing mispredicted branches so that the CPU speculatively accesses sensitive data. An attacker can then infer this data through side-channel measurements, leading to vulnerabilities.

The first public traces of this class of attacks appeared in early 2018, with the coordinated disclosure of Spectre and Meltdown in January 2018 by Google Project Zero.

Speculative Load Hardening (SLH)

To mitigate speculative execution attacks, Clang provides the compilation flag -mspeculative-load-hardening. For X86 targets, this flag can be configured with two main strategies, indirect masking and fencing, but in practice, only indirect masking is commonly used for production code. Other options are also available, which we do not present here.

To analyse these mitigations, we will use the following simple C program:

#include <stdio.h>

int main(int argc, char **argv)
{
    if (argc == 2)
        printf("%s", argv[1]);
    return 0;
}

This code is not vulnerable to any speculative execution attack. It is only used as a minimal example that contains a conditional branch.

Here is the corresponding assembly code:

0000000000001140 <main>:
    1140:   55                      push   rbp
    1141:   48 89 e5                mov    rbp,rsp
    1144:   48 83 ec 10             sub    rsp,0x10
    1148:   c7 45 fc 00 00 00 00    mov    DWORD PTR [rbp-0x4],0x0
    114f:   89 7d f8                mov    DWORD PTR [rbp-0x8],edi
    1152:   48 89 75 f0             mov    QWORD PTR [rbp-0x10],rsi
    1156:   83 7d f8 02             cmp    DWORD PTR [rbp-0x8],0x2
    115a:   75 16                   jne    1172 <main+0x32>
    115c:   48 8b 45 f0             mov    rax,QWORD PTR [rbp-0x10]
    1160:   48 8b 70 10             mov    rsi,QWORD PTR [rax+0x10]
    1164:   48 8d 3d 99 0e 00 00    lea    rdi,[rip+0xe99]        # 2004 <_IO_stdin_used+0x4>
    116b:   b0 00                   mov    al,0x0
    116d:   e8 be fe ff ff          call   1030 <printf@plt>
    1172:   31 c0                   xor    eax,eax
    1174:   48 83 c4 10             add    rsp,0x10
    1178:   5d                      pop    rbp
    1179:   c3                      ret

Indirect Masking

As described in the High Level Mitigation Approach section of the LLVM documentation on Speculative Load Hardening, one way to mitigate these attacks is to cause loads to be checked using branchless code to ensure that they are executing along a valid control flow path. In order to do that, Clang provides the SLH -mllvm -x86-slh-indirect option (default option).

The LLVM documentation illustrates this approach with the following example:

void leak(int data);
void example(int* pointer1, int* pointer2) {
  if (condition) {
    // ... lots of code ...
    leak(*pointer1);
  } else {
    // ... more code ...
    leak(*pointer2);
  }
}

This code can be transformed into a hardened version such as:

uintptr_t all_ones_mask = std::numerical_limits<uintptr_t>::max();
uintptr_t all_zeros_mask = 0;
void leak(int data);
void example(int* pointer1, int* pointer2) {
  uintptr_t predicate_state = all_ones_mask;
  if (condition) {
    // Assuming ?: is implemented using branchless logic...
    predicate_state = !condition ? all_zeros_mask : predicate_state;
    // ... lots of code ...
    //
    // Harden the pointer so it can't be loaded
    pointer1 &= predicate_state;
    leak(*pointer1);
  } else {
    predicate_state = condition ? all_zeros_mask : predicate_state;
    // ... more code ...
    //
    // Alternative: Harden the loaded value
    int value2 = *pointer2 & predicate_state;
    leak(value2);
  }
}

The key aspect here is that the update of predicate_state is performed in a branchless manner. This forces the processor to wait for the branch condition to be resolved before the mask can be correctly computed. The resulting mask is then applied to either the pointer or the loaded value, ensuring that speculative execution cannot access or propagate sensitive data when the control flow is mispredicted.

If we compile our example using this hardening method, the resulting assembly code becomes significantly more verbose. We can see that pointers inside conditional branches are masked using conditional move instructions (CMOV). The key point is that the CPU does not speculatively execute the outcome of conditional moves. This ensures that the pointer values are only applied once the branch condition is fully resolved, preventing speculative execution from using or leaking sensitive data along mispredicted paths.

0000000000001140 <main>:
    1140:   55                      push   rbp
    1141:   48 89 e5                mov    rbp,rsp
    1144:   48 83 ec 30             sub    rsp,0x30
    1148:   48 c7 c0 ff ff ff ff    mov    rax,0xffffffffffffffff
    114f:   48 89 45 e0             mov    QWORD PTR [rbp-0x20],rax
    1153:   48 89 e0                mov    rax,rsp
    1156:   48 c1 f8 3f             sar    rax,0x3f
    115a:   48 89 45 e8             mov    QWORD PTR [rbp-0x18],rax
    115e:   c7 45 fc 00 00 00 00    mov    DWORD PTR [rbp-0x4],0x0
    1165:   89 7d f8                mov    DWORD PTR [rbp-0x8],edi
    1168:   48 89 75 f0             mov    QWORD PTR [rbp-0x10],rsi
    116c:   83 7d f8 02             cmp    DWORD PTR [rbp-0x8],0x2
    1170:   75 02                   jne    1174 <main+0x34>
    1172:   eb 12                   jmp    1186 <main+0x46>
    1174:   48 8b 4d e0             mov    rcx,QWORD PTR [rbp-0x20]
    1178:   48 8b 45 e8             mov    rax,QWORD PTR [rbp-0x18]
    117c:   48 0f 44 c1             cmove  rax,rcx
    1180:   48 89 45 d8             mov    QWORD PTR [rbp-0x28],rax
    1184:   eb 59                   jmp    11df <main+0x9f>
    1186:   48 8b 45 e0             mov    rax,QWORD PTR [rbp-0x20]
    118a:   48 8b 4d e8             mov    rcx,QWORD PTR [rbp-0x18]
    118e:   48 0f 45 c8             cmovne rcx,rax
    1192:   48 8b 45 f0             mov    rax,QWORD PTR [rbp-0x10]
    1196:   48 8b 40 10             mov    rax,QWORD PTR [rax+0x10]
    119a:   48 89 ce                mov    rsi,rcx
    119d:   48 09 c6                or     rsi,rax
    11a0:   48 8d 3d 5d 0e 00 00    lea    rdi,[rip+0xe5d]        # 2004 <_IO_stdin_used+0x4>
    11a7:   31 c0                   xor    eax,eax
    11a9:   48 c1 e1 2f             shl    rcx,0x2f
    11ad:   48 09 cc                or     rsp,rcx
    11b0:   e8 7b fe ff ff          call   1030 <printf@plt>
    11b5:   48 8b 55 e0             mov    rdx,QWORD PTR [rbp-0x20]
    11b9:   48 8b 74 24 f8          mov    rsi,QWORD PTR [rsp-0x8]
    11be:   48 89 e1                mov    rcx,rsp
    11c1:   48 c1 f9 3f             sar    rcx,0x3f
    11c5:   48 8d 3d e9 ff ff ff    lea    rdi,[rip+0xffffffffffffffe9]        # 11b5 <main+0x75>
    11cc:   48 39 fe                cmp    rsi,rdi
    11cf:   48 0f 45 ca             cmovne rcx,rdx
    11d3:   48 89 4d d0             mov    QWORD PTR [rbp-0x30],rcx
    11d7:   48 8b 45 d0             mov    rax,QWORD PTR [rbp-0x30]
    11db:   48 89 45 d8             mov    QWORD PTR [rbp-0x28],rax
    11df:   48 8b 4d d8             mov    rcx,QWORD PTR [rbp-0x28]
    11e3:   31 c0                   xor    eax,eax
    11e5:   48 c1 e1 2f             shl    rcx,0x2f
    11e9:   48 09 cc                or     rsp,rcx
    11ec:   48 83 c4 30             add    rsp,0x30
    11f0:   5d                      pop    rbp
    11f1:   c3                      ret
    11f2:   66 2e 0f 1f 84 00 00    cs nop WORD PTR [rax+rax*1+0x0]
    11f9:   00 00 00
    11fc:   0f 1f 40 00             nop    DWORD PTR [rax+0x0]

To better understand this mechanism, let us take a closer look at what this code does in practice.

1. Mask initialization

mov    rax,0xffffffffffffffff
mov    QWORD PTR [rbp-0x20],rax

The compiler initializes a mask with all bits set (0xFFFFFFFFFFFFFFFF). This value represents a speculation-dependent mask that is later used to either preserve or deliberately corrupt pointers, ensuring that mis-speculated execution paths cannot access meaningful data.

2. Extracting the most significant bit of RSP

mov    rax,rsp
sar    rax,0x3f
mov    QWORD PTR [rbp-0x18],rax

This sequence extracts the most significant bit of the stack pointer by performing an arithmetic right shift. Since RSP is expected to hold a canonical user-space address, this bit should normally be zero.

3. First conditional branch

After the if (argc == 2) check, the CPU may speculatively execute one of the branches.

mov    rax,QWORD PTR [rbp-0x20]
mov    rcx,QWORD PTR [rbp-0x18]
cmovne rcx,rax

Here, a conditional move (CMOVNE) is used to update the mask stored in RCX.

The important property is that the CPU does not speculate on the result of a conditional move. As a consequence, it must wait for the condition flags to be resolved before applying the move. It creates a data dependency.

Once the condition is resolved, speculative execution may continue, but the mask will now correctly reflect the control-flow outcome. Depending on the condition, the mask will be either 0xFFFF_FFFF_FFFF_FFFF or 0x0000_0000_0000_0000.

4. Applying the mask to the pointer

At this point, RCX holds the computed mask.

mov    rax,QWORD PTR [rbp-0x10]
mov    rax,QWORD PTR [rax+0x10]
mov    rsi,rcx                         # copie du masque dans RSI
or     rsi,rax                         # OR du pointeur vers argv[1] avec notre masque.
lea    rdi,[rip+0xe5d]        # 2004 <_IO_stdin_used+0x4>
xor    eax,eax
shl    rcx,0x2f
or     rsp,rcx
call   1030 <printf@plt>

Here, the pointer to argv[1] is combined with the mask using a bitwise OR. If the execution path is valid, the mask preserves the pointer value. If the path is invalid due to mis-speculation, the mask corrupts the pointer, preventing it from being used to access meaningful data.

⚠️ **Enabling this feature has a notable performance impact, but it is still less costly than inserting lfence barriers everywhere (see next section). An attribute³ can be used to restrict hardening to specific functions, avoiding the need to slow down the entire program⁴.

`LFENCE` barriers

Another way to avoid speculative execution on branches is to insert LFENCE instructions after all conditional jumps to act as speculation barriers. To select this strategy, Clang provides the SLH option -mllvm -x86-slh-lfence. The LFENCE instruction does not execute until all prior instructions have completed locally, and no later instruction begins execution until LFENCE completes. As such, it acts as a barrier because the branch will never be executed speculatively.

push   rbp
mov    rbp,rsp
sub    rsp,0x10
mov    DWORD PTR [rbp-0x4],0x0
mov    DWORD PTR [rbp-0x8],edi
mov    QWORD PTR [rbp-0x10],rsi
cmp    DWORD PTR [rbp-0x8],0x2
jne    1175 <main+0x35>
lfence
mov    rax,QWORD PTR [rbp-0x10]
mov    rsi,QWORD PTR [rax+0x10]
lea    rdi,[rip+0xe96]        # 2004 <_IO_stdin_used+0x4>
xor    eax,eax
call   1030 <printf@plt>
lfence
xor    eax,eax
add    rsp,0x10
pop    rbp
ret

⚠️ We do not recommend using this compilation flag as its impact on program performance is very significant. It is mentioned here for completeness only.

Conclusion

Over the years, the list of recommended compiler hardening options has significantly evolved, driven by the continuous discovery of new classes of vulnerabilities and exploitation techniques. New compilation flags have been introduced to mitigate threats such as control-flow hijacking, speculative execution attacks, and information leaks at both the architectural and micro-architectural levels.

While these mitigations greatly improve the security posture of compiled binaries, they often come with a non-negligible performance cost. As a result, enabling them requires careful consideration and an informed trade-off between security and performance, depending on the threat model and deployment context.

The growing awareness that security is a first-class concern in modern software development suggests that these compiler options will continue to evolve. New mitigations will be added, existing ones will be refined, and defaults may change over time. Staying up to date with compiler developments and security recommendations is therefore essential.

Finally, compiler hardening options, while powerful, are not sufficient on their own. They must be combined with good programming practices, regular code reviews and audits. In addition, software protection techniques such as obfuscation, integrity checks or Runtime Application Self Protection (RASP) make the identification and exploitation of vulnerabilities more difficult and slow down attackers.

Acknowledgments

We would like to thank our colleagues Laurent Laubin and Rémy Salim for their thorough review of this article as well as their suggestions.

A developer’s guide to secure coding with FORTIFY_SOURCE, Sandipan Roy, July 2023. ↩
The Stack Clash, Qualys Research Team, December 2022. ↩
Attributes in Clang. ↩
Comparing GCC and Clang security features, Jonathan Corbet, September 2019. ↩

SCAF - Source Code Analysis Framework based on Clang - Pre-alpha preview

2014-08-25T00:00:00+02:00

1 - Introduction

There are many goals of source code analysis:

Security review
Bug finding
Program verification / understanding
Type / Style / Property checking

But, for each of these goals, when we work on source code analysis with colleagues, the main issue is how we can manage to work together and centralize information. There are a lot of libraries for source code analysis but none of them shares nor stores data and related meta-data.

That is why, we created SCAF, a framework based on Clang which helps us to deal with the source code manipulation and which stores information remotely and shares these information with others colleagues. Currently, SCAF is a pre-alpha and will be released in few days.

2 - SCAF architecture

SCAF is written in Python and offers two modules, an API and a GUI. The API is used to parse source code, to save parsing information in a database and do some analyzes. The GUI is mainly used to browse source code and to display API information but we can plug any others GUI (like a web app based on Apache/mod_python).

The API uses the libClang [1] to parse the source code and saves the AST in the database. When the AST (or part of it) is already stored, you don't have to re-parse the source code. All information can be directly requested via the API, which will extract data and metadata from the database, providing helpers to use them.

3 - Hook the "make" process

In source code analysis, one of the main issues is dealing with the context of compilation: include directories, preprocessor tricks and other obscure voodoo things that can be part of a build chain. When you have to parse a file, you must set up a context as close as possible the one used during compilation, otherwise you will get a lot of parsing issues. For that, you have two options:

The hard one: Parse the "Makefile"
The lazy one: Hook the make process

Actually, the best way (I guess) is to hook the make process (Yes, you're right, French guys are lazy, it's well known). Via this hook, you can get all information in a compilation context and forward everything to your parsing process.

SCAF offers an "agent" which hooks the make process and forwards all flags to both a compiler and the SCAF API. You just have to overload the CC variable like that:

$ make CC=/usr/bin/scaf_agent SCAF_HOST=localhost SCAF_LOGIN=user SCAF_PASSW=pwd SCAF_DB=project_1

In this case, the SCAF agent will saves all information in the "project_1" database and you don't care about flags and others compiler stuff. Then, after the compilation is completed, you can get any source code related information directly from the database.

4 - API under the hood

4.1 - API classes

There are still lot of classes in progress or things to do like aliasing, taint, symbolic... But currently these following classes are reliable:

SCAFDatabase: Database communication.
SCAFParsing: Parsing via the libClang.
SCAFAst: AST reconstruction.
SCAFAnalysis: This class contains all analysis.
SCAFCallsTraceAnalysis: Calls trace and xrefs.

4.2 - Parse a file and save information

The following example will create a new database, parse a file (with a default compilation context) and save information.

>>> from SCAF.API.Analysis import *
>>>
>>> analysis = SCAFAnalysis('localhost', 'login', 'pwd')
>>> analysis.createNewDatabase('SCAF_test1')
>>> analysis.useDatabase('SCAF_test1')
>>> analysis.parseFile('/tmp/test.c')
>>> analysis.closeDatabase()

Then, when you want to get information back, you just have to use the "SCAF_test1" database.

>>> from SCAF.API.Analysis import *
>>>
>>> analysis = SCAFAnalysis('localhost', 'login', 'pwd')
>>> analysis.getDatabaseListing()
['SCAF_driver', 'SCAF_driver2', 'SCAF_driver_w83793', 'SCAF_test1', 'SCAF_vmndh']

>>> analysis.useDatabase('SCAF_test1')

>>> analysis.getNumberOfNodesSaved()
16L

>>> analysis.getFilesList()
['/tmp/test.c']

>>> print analysis.getFileContent('/tmp/test.c')
int main(int ac, const char **av)
{
  int a = 1;
  int b = 2;

  return a + b;
}
>>>

When you parse files, these files will be stocked in the database allowing colleagues to work on the same project without any source on their disk.

4.3 - Run through the AST

The AST reconstruction is fully transparent, at each request in the API, a node is returned and can be run through like the original AST.

Each node returned by the getNodes() methods is a SCAF.API.AST.SCAFAst which contains these following attributes and methods:

self.idNode
self.idNodeParent
self.spelling
self.kind
self.type
self.value
self.result
self.fileName
self.startLine
self.endLine
self.startColumn
self.endColumn
self.getParent()
self.getChildren()
self.getContent()

The getParent() and getChildren() methods return also a SCAF.API.AST.SCAFAst which can be used to run through the AST recursively.

>>> nodes = analysis.getNodes(spelling='main')
>>> nodes
[<SCAF.API.AST.SCAFAst instance at 0x7fb94bda9170>]

>>> main = nodes[0]
>>> main
<SCAF.API.AST.SCAFAst instance at 0x7fb94bda9170>

>>> main.fileName
'/tmp/test.c'

>>> main.startLine
1L

>>> main.endLine
7L

>>> main.spelling
'main'

>>> main.kind
'FUNCTION_DECL'

>>> main.type
'FunctionProto'

Get children from a node:

>>> main
<SCAF.API.AST.SCAFAst instance at 0x7fb94bda9170>

>>> children = main.getChildren()
>>> children
[<SCAF.API.AST.SCAFAst instance at 0x7fb94bda9200>, <SCAF.API.AST.SCAFAst instance at 0x7fb94bda9248>, <SCAF.API.AST.SCAFAst instance at 0x7fb94bda9290>]

>>> for child in children:
...     print child.kind, child.spelling
...
PARM_DECL ac
PARM_DECL av
COMPOUND_STMT None

The COMPOUND_STMT is the statement node of the main function. Below, the complete AST of the main function.

The getNodes() method can take all SCAst attributes as filters in its arguments.

getNodes(kind='...', startLine='...', fileName='...', spelling='...', ...)

Each node has a kind (FUNCTION_DECL, COMPOUND_STMT, etc) and the possible list of kind is:

['ADDR_LABEL_EXPR', 'ANNOTATE_ATTR', 'ARRAY_SUBSCRIPT_EXPR', 'ASM_LABEL_ATTR', 'ASM_STMT', 'BINARY_OPERATOR',
'BLOCK_EXPR', 'BREAK_STMT', 'CALL_EXPR', 'CASE_STMT', 'CHARACTER_LITERAL', 'CLASS_DECL', 'CLASS_TEMPLATE',
'CLASS_TEMPLATE_PARTIAL_SPECIALIZATION', 'COMPOUND_ASSIGNMENT_OPERATOR', 'COMPOUND_LITERAL_EXPR', 'COMPOUND_STMT',
'CONDITIONAL_OPERATOR', 'CONSTRUCTOR', 'CONTINUE_STMT', 'CONVERSION_FUNCTION', 'CSTYLE_CAST_EXPR',
'CXX_ACCESS_SPEC_DECL', 'CXX_BASE_SPECIFIER', 'CXX_BOOL_LITERAL_EXPR', 'CXX_CATCH_STMT', 'CXX_CONST_CAST_EXPR',
'CXX_DELETE_EXPR', 'CXX_DYNAMIC_CAST_EXPR', 'CXX_FINAL_ATTR', 'CXX_FOR_RANGE_STMT', 'CXX_FUNCTIONAL_CAST_EXPR',
'CXX_METHOD', 'CXX_NEW_EXPR', 'CXX_NULL_PTR_LITERAL_EXPR', 'CXX_OVERRIDE_ATTR', 'CXX_REINTERPRET_CAST_EXPR',
'CXX_STATIC_CAST_EXPR', 'CXX_THIS_EXPR', 'CXX_THROW_EXPR', 'CXX_TRY_STMT', 'CXX_TYPEID_EXPR', 'CXX_UNARY_EXPR',
'DECL_REF_EXPR', 'DECL_STMT', 'DEFAULT_STMT', 'DESTRUCTOR', 'DO_STMT', 'ENUM_CONSTANT_DECL', 'ENUM_DECL',
'FIELD_DECL', 'FLOATING_LITERAL', 'FOR_STMT', 'FUNCTION_DECL', 'FUNCTION_TEMPLATE', 'GENERIC_SELECTION_EXPR',
'GNU_NULL_EXPR', 'GOTO_STMT', 'IB_ACTION_ATTR', 'IB_OUTLET_ATTR', 'IB_OUTLET_COLLECTION_ATTR', 'IF_STMT',
'IMAGINARY_LITERAL', 'INCLUSION_DIRECTIVE', 'INDIRECT_GOTO_STMT', 'INIT_LIST_EXPR', 'INTEGER_LITERAL',
'INVALID_CODE', 'INVALID_FILE', 'LABEL_REF', 'LABEL_STMT', 'LINKAGE_SPEC', 'MACRO_DEFINITION', 'MACRO_INSTANTIATION',
'MEMBER_REF', 'MEMBER_REF_EXPR', 'NAMESPACE', 'NAMESPACE_ALIAS', 'NAMESPACE_REF', 'NOT_IMPLEMENTED',
'NO_DECL_FOUND', 'NULL_STMT', 'OBJC_AT_CATCH_STMT', 'OBJC_AT_FINALLY_STMT', 'OBJC_AT_SYNCHRONIZED_STMT',
'OBJC_AT_THROW_STMT', 'OBJC_AT_TRY_STMT', 'OBJC_AUTORELEASE_POOL_STMT', 'OBJC_BRIDGE_CAST_EXPR', 'OBJC_CATEGORY_DECL',
'OBJC_CATEGORY_IMPL_DECL', 'OBJC_CLASS_METHOD_DECL', 'OBJC_CLASS_REF', 'OBJC_DYNAMIC_DECL', 'OBJC_ENCODE_EXPR',
'OBJC_FOR_COLLECTION_STMT', 'OBJC_IMPLEMENTATION_DECL', 'OBJC_INSTANCE_METHOD_DECL', 'OBJC_INTERFACE_DECL',
'OBJC_IVAR_DECL', 'OBJC_MESSAGE_EXPR', 'OBJC_PROPERTY_DECL', 'OBJC_PROTOCOL_DECL', 'OBJC_PROTOCOL_EXPR',
'OBJC_PROTOCOL_REF', 'OBJC_SELECTOR_EXPR', 'OBJC_STRING_LITERAL', 'OBJC_SUPER_CLASS_REF', 'OBJC_SYNTHESIZE_DECL',
'OVERLOADED_DECL_REF', 'PACK_EXPANSION_EXPR', 'PAREN_EXPR', 'PARM_DECL', 'PREPROCESSING_DIRECTIVE', 'RETURN_STMT',
'SEH_EXCEPT_STMT', 'SEH_FINALLY_STMT', 'SEH_TRY_STMT', 'SIZE_OF_PACK_EXPR', 'STRING_LITERAL', 'STRUCT_DECL',
'SWITCH_STMT', 'StmtExpr', 'TEMPLATE_NON_TYPE_PARAMETER', 'TEMPLATE_REF', 'TEMPLATE_TEMPLATE_PARAMETER',
'TEMPLATE_TYPE_PARAMETER', 'TRANSLATION_UNIT', 'TYPEDEF_DECL', 'TYPE_ALIAS_DECL', 'TYPE_REF', 'UNARY_OPERATOR',
'UNEXPOSED_ATTR', 'UNEXPOSED_DECL', 'UNEXPOSED_EXPR', 'UNEXPOSED_STMT', 'UNION_DECL', 'USING_DECLARATION',
'USING_DIRECTIVE', 'VAR_DECL', 'WHILE_STMT']

So, for example, if you want all functions, you just have to do something like that:

>>> analysis = SCAFAnalysis('localhost', 'login', 'pwd')
>>> analysis.useDatabase('SCAF_vmndh')
>>> funcs = analysis.getNodes(kind='FUNCTION_DECL')
>>> len(funcs)
275
>>>

Or, if you are looking for attributes of a specific structure:

>>> analysis = SCAFAnalysis('localhost', 'login', 'pwd')
>>> analysis.useDatabase('SCAF_driver_w83793')
>>> len(analysis.getNodes(kind='STRUCT_DECL'))
948

>>> analysis.getNodes(kind='STRUCT_DECL', spelling='watchdog_info')
[<SCAF.API.AST.SCAFAst instance at 0x7fb9487dad40>]

>>> watchdog_info = analysis.getNodes(kind='STRUCT_DECL', spelling='watchdog_info')[0]
>>> watchdog_info.fileName
'/usr/src/linux-3.14.14-gentoo/include/uapi/linux/watchdog.h'

>>> watchdog_info.startLine
17L

>>> watchdog_info.endLine
21L

>>> attributes = watchdog_info.getChildren()
>>> len(attributes)
3

>>> for attr in attributes:
...     print attr.kind, attr.spelling, attr.getContent()
...
FIELD_DECL options __u32 options
FIELD_DECL firmware_version __u32 firmware_version
FIELD_DECL identity __u8  identity[32]
>>>

If we do some speed benchmark about the search engine on a big source project like a Linux driver with all kernel includes.

#!/usr/bin/env python2
## -*- coding: utf-8 -*-

from SCAF.API.Analysis import *

analysis = SCAFAnalysis('localhost', 'login', 'pwd')
analysis.useDatabase('SCAF_driver_w83793')
print 'Looking into %d files' %(len(analysis.getFilesList()))
for node in analysis.getNodes(spelling='i'):
    print node.kind, node.fileName, node.startLine, node.startColumn, node.getContent()

We have something like that - Tested on a Lenovo x230:

$ time python2 ./test.py
Looking into 274 files
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/linux/bitmap.h 115 44 int i
PARM_DECL /usr/src/linux-3.14.14-gentoo/arch/x86/include/asm/atomic.h 36 44 int i
PARM_DECL /usr/src/linux-3.14.14-gentoo/arch/x86/include/asm/atomic.h 48 31 int i
PARM_DECL /usr/src/linux-3.14.14-gentoo/arch/x86/include/asm/atomic.h 62 31 int i
PARM_DECL /usr/src/linux-3.14.14-gentoo/arch/x86/include/asm/atomic.h 78 39 int i
PARM_DECL /usr/src/linux-3.14.14-gentoo/arch/x86/include/asm/atomic.h 142 39 int i
PARM_DECL /usr/src/linux-3.14.14-gentoo/arch/x86/include/asm/atomic.h 154 37 int i
PARM_DECL /usr/src/linux-3.14.14-gentoo/arch/x86/include/asm/atomic.h 166 37 int i
PARM_DECL /usr/src/linux-3.14.14-gentoo/arch/x86/include/asm/atomic64_64.h 31 48 long i
PARM_DECL /usr/src/linux-3.14.14-gentoo/arch/x86/include/asm/atomic64_64.h 43 33 long i
PARM_DECL /usr/src/linux-3.14.14-gentoo/arch/x86/include/asm/atomic64_64.h 57 33 long i
PARM_DECL /usr/src/linux-3.14.14-gentoo/arch/x86/include/asm/atomic64_64.h 73 41 long i
PARM_DECL /usr/src/linux-3.14.14-gentoo/arch/x86/include/asm/atomic64_64.h 139 41 long i
PARM_DECL /usr/src/linux-3.14.14-gentoo/arch/x86/include/asm/atomic64_64.h 151 40 long i
PARM_DECL /usr/src/linux-3.14.14-gentoo/arch/x86/include/asm/atomic64_64.h 156 40 long i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/linux/atomic.h 115 30 int i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/asm-generic/atomic-long.h 34 54 long i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/asm-generic/atomic-long.h 55 36 long i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/asm-generic/atomic-long.h 62 36 long i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/asm-generic/atomic-long.h 69 44 long i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/asm-generic/atomic-long.h 90 44 long i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/asm-generic/atomic-long.h 97 43 long i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/asm-generic/atomic-long.h 104 43 long i
PARM_DECL /usr/src/linux-3.14.14-gentoo/arch/x86/include/asm/pvclock.h 103 34 struct pvclock_vsyscall_time_info *i
VAR_DECL /usr/src/linux-3.14.14-gentoo/include/linux/slab.h 486 3 int i = kmalloc_index(size)
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/linux/klist.h 62 46 struct klist_iter *i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/linux/klist.h 63 51 struct klist_iter *i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/linux/klist.h 65 29 struct klist_iter *i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/linux/klist.h 66 38 struct klist_iter *i
VAR_DECL /usr/src/linux-3.14.14-gentoo/include/linux/dqblk_qtree.h 49 2 int i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/linux/fs.h 304 3 struct iov_iter *i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/linux/fs.h 306 3 struct iov_iter *i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/linux/fs.h 307 23 struct iov_iter *i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/linux/fs.h 308 32 struct iov_iter *i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/linux/fs.h 309 34 const struct iov_iter *i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/linux/fs.h 311 34 struct iov_iter *i
PARM_DECL /usr/src/linux-3.14.14-gentoo/include/linux/fs.h 323 37 struct iov_iter *i
VAR_DECL /secure/QuarksLab/Research/SCAF/samples/driver/w83793/w83793.c 867 3 int i = index >> 1
VAR_DECL /secure/QuarksLab/Research/SCAF/samples/driver/w83793/w83793.c 1375 4 size_t i
VAR_DECL /secure/QuarksLab/Research/SCAF/samples/driver/w83793/w83793.c 1513 2 int i
VAR_DECL /secure/QuarksLab/Research/SCAF/samples/driver/w83793/w83793.c 1580 2 int i
python2 ./test.py  0.09s user 0.01s system 70% cpu 0.145 total
$

As you can see it's pretty fast and we got all nodes which are named "i" - It's different of a simple grep search which will give us all occurrences of "i". This is the first advantage of centralized information. The second advantage is that you don't need the project sources on your disk and you can share everything remotely without that your colleagues have to re-parse a second time the source code.

5 - Calls trace and xrefs

The calls trace is also available as a tree. Each node is a SCAFCallsTraceAnalysisAst and this class contains only one attribute and two methods:

self.spelling : function name
self.getNode() : FUNCTION_DECL AST node
self.getChildren() : Called functions

So, you can run through the calls trace recursively via the getChildren() method and jump in the function's AST via the getNode() method.

5.1 - How to get a SCAFCallsTraceAnalysisAst from a SCAFAst node

The getAstFromNode() from the SCAFCallsTraceAnalysis class allows to make a SCAFCallsTraceAnalysisAst from a SCAFAst.

>>> from SCAF.API.Analysis import *
>>>
>>> analysis = SCAFAnalysis('localhost', 'login', 'pwd')
>>> analysis.useDatabase('SCAF_vmndh')

>>> main = analysis.getNodes(spelling='main')[0]
>>> main
<SCAF.API.AST.SCAFAst instance at 0x7f72a2b45560>

>>> ctNode = analysis.callsTraceAnalysis.getAstFromNode(main)
>>> ctNode
<SCAF.API.CallsTraceAnalysis.SCAFCallsTraceAnalysisAst instance at 0x7f72a2b45638>

5.2 - Display the called functions

As I already said above, you can run through the calls trace recursively via the getChildren() method and display the function name.

>>> ctNode = analysis.callsTraceAnalysis.getAstFromNode(main)
>>> ctNode
<SCAF.API.CallsTraceAnalysis.SCAFCallsTraceAnalysisAst instance at 0x7f72a2b45638>

>>> for child in ctNode.getChildren():
...     print child
...
check_aslr_mode
check_nx_mode
check_pie_mode
check_debug_mode
check_core_mode
check_file_mode
syntax

Each child is also a SCAFCallsTraceAnalysisAst class. Below, a little code which run through the calls trace recursively:

$ cat a.py
#!/usr/bin/env python2
## -*- coding: utf-8 -*-

from SCAF.API.Analysis import *

def throughAST(ctNode, depth):
    print ("...." * depth), ctNode
    for child in ctNode.getChildren():
        throughAST(child, depth + 1)

if __name__ == '__main__':
    analysis = SCAFAnalysis('localhost', 'login', 'pwd')
    analysis.useDatabase('SCAF_vmndh')
    func = analysis.getNodes(spelling='init_vmem')[1]
    ctNode = analysis.callsTraceAnalysis.getAstFromNode(func)
    throughAST(ctNode, 0)

$ python2 a.py
 init_vmem
.... xmalloc
........ malloc
........ perror
........ exit
.... memset
.... rand_aslr
.... rand_pie
.... set_arg_in_memory
........ strlen
........ fprintf
........ exit
........ strcpy
$

The visualization of this calls trace with Dot, is:

5.3 - Check if two functions are linked

The isLinked() method allows to know if two functions are linked. Example:

>>> init_vmem = analysis.getNodes(spelling='init_vmem')[1]
>>> xmalloc = analysis.getNodes(spelling='xmalloc')[1]
>>> set_arg = analysis.getNodes(spelling='set_arg_in_memory')[0]

>>> analysis.callsTraceAnalysis.isLinked(init_vmem, set_arg)
True

>>> analysis.callsTraceAnalysis.isLinked(xmalloc, set_arg)
False

5.4 - Know the depth between two functions

In order to make some specific analyzis, we sometimes have to know the depth between two functions. The getDepths() method can give us this information:

>>> analysis.callsTraceAnalysis.getDepths(init_vmem, set_arg)
[1]

>>> analysis.callsTraceAnalysis.getDepths(init_vmem, 'exit')
[2, 2]

>>> analysis.callsTraceAnalysis.getDepths(main, 'exit')
[3, 4, 4, 4, 4, 4, 4, 4, 5, 4, 2, 2]
>>>

In the first case, we have only one possible path with a 1 depth. In the second case, we have two possibles paths with both a depth of 2. In the last case, we have 12 possibles paths with a least depth of 2.

5.5 - Get all calls trace between two functions

Like the getDepths(), we can get the trace with the getTraces() method.

>>> analysis.callsTraceAnalysis.getTraces(init_vmem, 'exit')
[
  [<SCAFCallsTraceAnalysisAst>, <SCAFCallsTraceAnalysisAst>, <SCAFCallsTraceAnalysisAst>],
  [<SCAFCallsTraceAnalysisAst>, <SCAFCallsTraceAnalysisAst>, <SCAFCallsTraceAnalysisAst>]
]

>>> len(analysis.callsTraceAnalysis.getTraces(init_vmem, 'exit'))
2

>>> for x in analysis.callsTraceAnalysis.getTraces(init_vmem, 'exit'):
...     for c in x:
...             sys.stdout.write(' -> ' + c.spelling)
...     sys.stdout.write('\n')
...
 -> init_vmem -> xmalloc -> exit
 -> init_vmem -> set_arg_in_memory -> exit
>>>

Another example with all calls trace between main and exit:

>>> for x in analysis.callsTraceAnalysis.getTraces(main, 'exit'):
...     for c in x:
...             sys.stdout.write(' -> ' + c.spelling)
...     sys.stdout.write('\n')
...
 -> main -> check_file_mode -> check_arg_mode -> exit
 -> main -> check_file_mode -> init_vmem -> xmalloc -> exit
 -> main -> check_file_mode -> init_vmem -> set_arg_in_memory -> exit
 -> main -> check_file_mode -> save_binary -> xopen -> exit
 -> main -> check_file_mode -> save_binary -> xmalloc -> exit
 -> main -> check_file_mode -> save_binary -> xread -> exit
 -> main -> check_file_mode -> parse_binary -> error_ndh_format -> exit
 -> main -> check_file_mode -> parse_binary -> xmalloc -> exit
 -> main -> check_file_mode -> execute -> check_pc -> segfault -> exit
 -> main -> check_file_mode -> execute -> segfault -> exit
 -> main -> check_file_mode -> exit
 -> main -> syntax -> exit
>>>

5.6 - Jump from SCAFCallsTraceAnalysisAst to SCAFAst

We can also jump from a SCAFCallsTraceAnalysisAst to a SCAFAst for analyze a specific function in your calls trace. For that, you have to use the getNode() method which returns a SCAFAst node:

5.7 - Xrefs

The getXRefs() returns a list of the caller functions.

>>> xrefs = analysis.callsTraceAnalysis.getXRefs('exit')
>>> len(xrefs)
12

>>> for x in xrefs:
...     print x.spelling
...
set_arg_in_memory
syntax
segfault
error_ndh_format
xmalloc
xopen
xmmap
xread
console_quit
check_arg_mode
syscall_exit
check_file_mode
>>>

6 - GUI Screenshots

Some screenshots of the pre-alpha GUI are available here [2].

7 - Conclusion

The main objectives of SCAF are :

Allow you to work on projects which contains a lot of sources code.
Offers an other abstraction which allows you to use and search some specific nodes really easily.
Allow you to work on an analysis with others colleagues and share notes, favorites, information and analysis results.

This project is developed in python as two distinct module (API and GUI) and the API can be plugged/used on any others projects.

SCAF is still in pre-alpha and will be released in few days. Before the release, I have to make more helpers/examples scripts and write some documentations - All the boring parts :( but the helpful parts if you plan to use/contribute to this project.

8 - Acknowledgments

Thanks to:

Kévin Szkudlapski, Serge Guelton, Adrien Guinet and Cédric Tessier for some source code analysis tricks and conception.
Fabian 'fabs' Yamaguchi for some ideas related to his work [3].

9 - References

[1]	http://clang.llvm.org/doxygen/modules.html

[2]	http://shell-storm.org/repo/IMG/SCAF/v0.1/

[3]	http://codeexploration.blogspot.fr

Quarkslab's blog - Clang

Obfuscation vs the Optimizer: An LLVM Middle-End Arms Race

Introduction

Meet the mystery function

A quick LLVM primer

Try it yourself

Why the middle-end matters for both sides

The optimizer's toolkit

The arms race

Round 1 — all constants, no contest

Round 2 — why it collapsed instantly

Round 3 — hiding behind a variable

Round 4 — version shock LLVM 18 vs LLVM 19

Round 5 — one constant away from survival

The yin-yang

Acknowledgments

Clang Hardening Cheat Sheet - Ten Years Later

Introduction

TL;DR

General Protections

Fortify

C++ Library Hardening

nodlopen

Defenses Against Stack-Based Memory Corruption

Non-Executable Stack (NX)

Stack clash vulnerabilities

Stack Clash Protection

Defenses Against Code-Reuse Attacks (ROP/JOP)

Code-Reuse Attacks

Control-Flow Protection (x86)

Indirect Branch Tracking

Shadow Stack

Control-Flow Protection (ARM)

Branch Target Identification (BTI)

Pointer Authentication Codes (PAC)

Register Zeroing

Defenses Against Speculative Execution Attacks

Speculative Attacks

Speculative Load Hardening (SLH)

Indirect Masking

1. Mask initialization

2. Extracting the most significant bit of RSP

3. First conditional branch

4. Applying the mask to the pointer

LFENCE barriers

Conclusion

Acknowledgments

SCAF - Source Code Analysis Framework based on Clang - Pre-alpha preview

1 - Introduction

2 - SCAF architecture

3 - Hook the "make" process

4 - API under the hood

4.1 - API classes

4.2 - Parse a file and save information

4.3 - Run through the AST

5 - Calls trace and xrefs

5.1 - How to get a SCAFCallsTraceAnalysisAst from a SCAFAst node

5.2 - Display the called functions

5.3 - Check if two functions are linked

5.4 - Know the depth between two functions

5.5 - Get all calls trace between two functions

5.6 - Jump from SCAFCallsTraceAnalysisAst to SCAFAst

5.7 - Xrefs

6 - GUI Screenshots

7 - Conclusion

8 - Acknowledgments

9 - References

`LFENCE` barriers