This project contains tools (evt2json, dt2json, trace_merger, extract_metrics.py) to manipulate traces. The different tools implement the core tool logic + user interface. All common code is implemented in trace-utils lib. This project has been parametrized to be able to build (all/some) parts depending on the trace types required to manipulate.

option(ENABLE_UEVENT "Enable uevent support (TODO)" OFF)
option(ENABLE_CEREAL "Enable cereal support" ON)
option(ENABLE_ET_TRACE "Enable et-trace support" ON)
option(ENABLE_TESTS "Enable building tests" ON)
option(ENABLE_BENCHMARKS "Enable building benchmarks" OFF)
option(ENABLE_WARNINGS_AS_ERRORS "Enable -Werror" OFF)
option(ENABLE_COVERAGE "Enable coverage flags" OFF)

Make sure to build/download the needed dependencies.

In case of using dock.py, all system dependencies will be provided by the docker image. sw-platform build flow will build some other dependencies from sources.

Compile using the standard CMake flow:

mkdir build
cd build
cmake ..
make

The evt2json will be built inside of the build/tools directory. You can install it using make install DESTDIR=<INSTALL_PATH> or configuring cmake with cmake -DCMAKE_INSTALL_PREFIX=<INSTALL_PATH>.

When using conan, all dependencies will be provided by conan (instead of via the docker image)

Run conan install to pull all dependencies and generate custom FindXXX.cmake modules:

mkdir build
cd build
conan install .  -pr:b default -pr:h linux-ubuntu18.04-x86_64-gcc7-debug   --remote conan-develop --build missing -if=build
cmake --preset debug
cmake --build --preset debug

It's possible to build trace-utils (and any conan-ready) project with CLion IDE passing the appropiate CMAKE_TOOLCHAIN_FILE.

After building the project, execute cmake --install (this is done automatically in sw-platform builds).

CMake strips rpath(s) from binaries at install time. You'll need to set correct LD_LIBRARY_PATH pointing to the installation location.

Convert an event trace to a JSON file compatible with perfetto.

Perfetto is a tool to visualize traces, usually for Android/Linux. We can also use it to visualize recorded execution traces of our simulated tests. At the moment this only supports the old CoSim traces, but in the future this will work with the uevent library.

The following three files need to be provided as input:

This is a record of all events seen from the arch_state_monitor. You can enable this trace with --uarg record. This will usually create a file called cosim_evt.trace in the test directory.

We track the retire-like (i.e.: Retire, Exception, Interrupt) events in the trace. These events have at least the following fields:

• Cycle Time
• Hart ID
• PC

The following trace formats are supported:

• uevent, the default.
• cosim, enabled with --cosim.
• json, enabled with --json (see examples/retires.json).

The tool expects a JSON-file with a list of the symbols (i.e.: functions) you wish to track throughout the execution. Each symbol is an object with the following fields:

[
    // ...
    {
        "name": "foo",              // The name of the symbol
        "low_pc": "0x8000800000",   // Entry PC
        "high_pc": "0x1000"         // Offset to exit PC
    },
    // ...
]

A default symbol list can be generated using the evt2json-symbols.sh script. This calls nm and converts the output into valid JSON.

Alternatively, you can specify the ELFs used during the execution. The tool with then extract the symbols (via nm -S). This can be enabled by listing the ELFs on the command line (--elf <path>). Multiple ELFs are supported.

List of threads you want to track during the execution and their respective labels. As with the symbols list, this is also a JSON-list, this time of thread groups, each of which has a list of its threads.

[
    {"name": "Master Minion Shire"
     "id":    32,
     "threads": [
        {"name":"Dispatcher", "id":2048},
        {"name":"Submission Queue Worker", "id":2050},
        {"name":"Kernel Worker", "id":2054},
        {"name":"DMA Worker",    "id":2058},
        {"name":"DMA Worker",    "id":2059}
    ]}
]

The id field of the thread group can be arbitrary, but it maps naturally to the shire id. The id field of the thread itself maps to the hart ID as we see it in the input event trace.

A tool to parse device traces into json/txt files.

A tool to merge ML stack traces into a single perfetto trace.

1. If you make modifications, make sure your changes don't break tests/.
2. This project has many contributors, use git blame to find previous authors if you seek support.
3. As of 2022-12-16 the integration repository is develop/system-sw that means integration will happen there first.

As of 2022-12-16 only *.elf, *.png files are tracked. If you need to add tests with binary files, please track them with git lfs and not as a regular file.