nekko is a command-line program to simplify pulling models and datasets for running inference, and then launching them in a Docker container.

nekko does not rely on the docker compose examples. It is a separate method of launching.

You can compile it from source by running:

go build -o dist/nekko ./cmd/nekko
# or just
make build

A pipeline for automatic compilation is being built. When it is ready, you will be able to download compiled binaries from the releases page.

$ nekko run -r <runtime> -m <model_name> [-d <dataset_name>] [-i <image>] [-c <command>]

It currently supports the following runtimes:

• onnx-eis - launches the model in the container ghcr.io/nekkoai/onnx-eis
• onnx-runtime - launches the model in the container ghcr.io/nekkoai/onnxruntime

The specific runtime not only determines the container image but also the command used to run the model. You can override both, leaving the runtime irrelevant, by using the -i and -c flags.

To show the full set of options and defaults, run nekko -h for help.

Set the env var DEBUG=1 to enable debug output, e.g. DEBUG=1 nekko.

The model and optional dataset are pulled from the following locations:

• hf.co/* or huggingface.co/* - from HuggingFace, using the huggingface CLI
• any other path - as OCI images, following the usual OCI rules

For example:

• ghcr.io/nekkoai/models/resnet50:2025.08.07 would be pulled via Docker from ghcr.io
• hf.co/rvs/llama3_awq_int4_complete would be pulled via HuggingFace CLI from huggingface.co

The default images to use for the container depend on the selected runtime:

• onnx-eis uses ghcr.io/nekkoai/onnx-eis
• onnx-runtime uses ghcr.io/nekkoai/onnxruntime

You can override it with the -i flag.

nekko supports a variety of model formats. At this time, it includes:

• ONNX
• .bundle - the unique compiled format for running on nekko chips; see bundle docs

1. Determine the image and command based on the selected runtime or explicit override.
2. Download the runtime container image.
3. For each of the model and optional dataset:
   • If the source is HuggingFace, download it to the usual local HuggingFace cache via the hf CLI.
   • If the source is anything else, download it via Docker to the docker model cache.
4. Construct the docker run CLI:
   • Mount to /model from the model file directly in either the docker model or the HuggingFace cache directory for the specific model.
   • Mount to /dataset, if any provided, from the dataset file directly in either the docker model or the HuggingFace cache directory for the specific dataset.
   • Mount any other required caching directories, e.g. for .bundle files.
   • If the ET devices are available, mount them in as devices.
   • Select the image based on the default for the runtime or the provided override.
   • Select the command based on the default for the runtime or the provided override.
5. Run the docker command.