Summary

QBDI debug builds consistently crash with a SmallVector bounds checking assertion when using QBDIPreload, while release builds work perfectly with identical code. This makes debugging QBDI-based applications nearly impossible.

Environment

• Platform: macOS 14.4.0 (Darwin 24.4.0)
• Architecture: Apple Silicon (AARCH64)
• QBDI Version: Git commit 6826871
• Compiler: Apple clang version 17.0.0
• Build Type: Debug vs Release comparison

Bug Description

When running any QBDIPreload-based instrumentation in debug builds, the program crashes with:

Assertion failed: (idx < size()), function operator[], file SmallVector.h, line 309.

The same code works flawlessly in release builds. This appears to be a bounds checking issue in LLVM's SmallVector container where QBDI attempts to access an invalid array index.

Steps to Reproduce

I've created a minimal reproduction case that demonstrates this issue consistently.

1. Test Program

File: minimal_test.c

#include <stdio.h>

int simple_add(int a, int b) {
    return a + b;
}

int main() {
    int result = simple_add(5, 3);
    printf("Result: %d\n", result);
    return 0;
}

2. QBDIPreload Tracer

File: minimal_tracer.c

#include <QBDIPreload.h>
#include <stdio.h>

QBDIPRELOAD_INIT;

static VMAction coverage_callback(VMInstanceRef vm, GPRState *gprState, FPRState *fprState, void *data) {
    static unsigned long instruction_count = 0;
    instruction_count++;
    
    if (instruction_count % 1000 == 0) {
        printf("[tracer] Executed %lu instructions\n", instruction_count);
    }
    
    return QBDI_CONTINUE;
}

int qbdipreload_on_start(void *main) {
    printf("[tracer] Starting QBDI instrumentation\n");
    return QBDIPRELOAD_NOT_HANDLED;
}

int qbdipreload_on_premain(void *gprCtx, void *fpuCtx) {
    printf("[tracer] Pre-main called\n");
    return QBDIPRELOAD_NOT_HANDLED;
}

int qbdipreload_on_main(int argc, char** argv) {
    printf("[tracer] Main called with %d arguments\n", argc);
    return QBDIPRELOAD_NOT_HANDLED;
}

int qbdipreload_on_run(VMInstanceRef vm, rword start, rword stop) {
    printf("[tracer] VM run called from 0x%llx to 0x%llx\n", start, stop);
    
    qbdi_addCodeCB(vm, QBDI_PREINST, coverage_callback, NULL, 0);
    printf("[tracer] Registered coverage callback\n");
    
    qbdi_run(vm, start, stop);
    printf("[tracer] VM run completed\n");
    
    return QBDIPRELOAD_NO_ERROR;
}

int qbdipreload_on_exit(int status) {
    printf("[tracer] Exit called with status %d\n", status);
    return QBDIPRELOAD_NO_ERROR;
}

3. Build Configuration

File: CMakeLists.txt

cmake_minimum_required(VERSION 3.20)
project(QBDIBugRepro)

if(NOT DEFINED QBDI_BUILD_DIR)
    message(FATAL_ERROR "Please specify QBDI_BUILD_DIR")
endif()

if(NOT DEFINED QBDI_SOURCE_DIR)
    message(FATAL_ERROR "Please specify QBDI_SOURCE_DIR")
endif()

set(QBDI_INCLUDE_DIR 
    "${QBDI_BUILD_DIR}/include"
    "${QBDI_BUILD_DIR}/include-shared" 
    "${QBDI_SOURCE_DIR}/include" 
    "${QBDI_SOURCE_DIR}/tools/QBDIPreload/include")

add_executable(minimal_test minimal_test.c)

add_library(minimal_tracer SHARED minimal_tracer.c)
target_include_directories(minimal_tracer PRIVATE ${QBDI_INCLUDE_DIR})
target_link_libraries(minimal_tracer 
    "${QBDI_BUILD_DIR}/libQBDI.dylib"
    "${QBDI_BUILD_DIR}/tools/QBDIPreload/libQBDIPreload.a")

set_target_properties(minimal_tracer PROPERTIES
    BUNDLE FALSE
    MACOSX_BUNDLE FALSE)

4. Build and Test Commands

# Build QBDI in debug mode
cmake -B qbdi-build-debug -DCMAKE_BUILD_TYPE=Debug /path/to/qbdi
cmake --build qbdi-build-debug --parallel

# Build QBDI in release mode
cmake -B qbdi-build-release -DCMAKE_BUILD_TYPE=Release /path/to/qbdi
cmake --build qbdi-build-release --parallel

# Build test case for debug
cmake -B build-debug -DCMAKE_BUILD_TYPE=Debug \
  -DQBDI_BUILD_DIR=/path/to/qbdi-build-debug \
  -DQBDI_SOURCE_DIR=/path/to/qbdi
cmake --build build-debug

# Build test case for release
cmake -B build-release -DCMAKE_BUILD_TYPE=Release \
  -DQBDI_BUILD_DIR=/path/to/qbdi-build-release \
  -DQBDI_SOURCE_DIR=/path/to/qbdi
cmake --build build-release

# Test release (works)
DYLD_INSERT_LIBRARIES=./build-release/libminimal_tracer.dylib ./build-release/minimal_test

# Test debug (crashes)
DYLD_INSERT_LIBRARIES=./build-debug/libminimal_tracer.dylib ./build-debug/minimal_test

Expected vs Actual Behavior

Expected (Release Build Output)

[tracer] Starting QBDI instrumentation
[tracer] Pre-main called
[tracer] Main called with 1 arguments
[tracer] VM run called from 0x1040cc848 to 0x0
[tracer] Registered coverage callback
Result: 8
[tracer] Exit called with status 0

Actual (Debug Build Output)

[tracer] Starting QBDI instrumentation
[tracer] Pre-main called
[tracer] Main called with 1 arguments
[tracer] VM run called from 0x1026b8860 to 0x0
[tracer] Registered coverage callback
Assertion failed: (idx < size()), function operator[], file SmallVector.h, line 309.
Result: 8
Process terminated with exit code 134 (SIGABRT)

Notes

Several observations about this crash:

1. The target program prints "Result: 8" before crashing, indicating the instrumentation works correctly
2. The crash happens after program completion, possibly during VM cleanup or in a background thread
3. Release builds work perfectly, suggesting this is a bounds checking assertion that only fires when LLVM debug assertions are enabled
4. The assertion (idx < size()) in SmallVector.h:309 says QBDI is accessing a container with an invalid index

Workaround

Currently, the only workaround is to use release builds for all development and testing, which is suboptimal for debugging and development workflows.

Additional Information

This issue has been reproduced consistently across multiple test cases and appears to affect any QBDIPreload-based instrumentation when built in debug mode. The fact that the program executes correctly before crashing suggests this is likely a cleanup or concurrent access issue rather than a fundamental logic error.

The SmallVector assertion failure indicates this is probably a relatively straightforward bounds checking bug that could be fixed by identifying where QBDI accesses container elements without proper bounds validation.

I'm happy to provide additional testing, logs, or debugging information if it would help track down the root cause.