Neural network inference in pure Odin

Anvil runs neural networks without the typical ML stack. Load a safetensors or ONNX file, run inference, get results. Currently implements enough operations to run vision models such as ResNet, YOLOv8, SAM (TinyViT). Your deployment is just a binary. No Python, no gigabytes of dependencies. Just compiled code running models.

Download mobile_sam-tiny-vitt.safetensors from here. Drop it in weights/ and run:

$ odin run examples/sam_raylib -o:speed -no-bounds-check

See examples/yolo_simple

$ odin build examples/yolo_simple -o:speed -no-bounds-check
$ ./yolo_simple weights/yolov8m.safetensors /path/to/image.jpg m

$ odin build examples/resnet50_onnx -o:speed -no-bounds-check
$ ./resnet50_onnx /path/to/image.jpg

This package uses context.temp_allocator for intermediate computations and uses the passed allocator argument for the function return values. If you're doing operations in a loop, just reset the temp allocator periodically:

for item in items {
    // Your tensor ops / model inference here
    // ...

    free_all(context.temp_allocator)  // Clear intermediates
}

Early development, APIs are unstable. More models coming as operations get implemented.

• Odin compiler
• A safetensors or ONNX file: skip this if you only care about using the tensor data structure.
• OpenBLAS (libopenblas-dev on Ubuntu, precompiled .lib for Windows)
• That's it