NNtrainer is a Software Framework for training/Inference of Neural Network models on devices.
NNtrainer is an Open Source Project. The aim of the NNtrainer is to develop a Software Framework to train and run inference of neural network models on embedded devices which have relatively limited resources.
Training: Rather than training whole layers of a network from the scratch, NNtrainer finetunes the neural network model on device with user data for the personalization.
Inference: NNtrainer supports efficient LLM inference on memory-constrained devices through advanced memory optimization techniques:
- FSU (Flash Storage Utilization): Dynamically loads model weights from flash storage during inference, significantly reducing peak memory usage.
- MoE Cache: Intelligent caching mechanism for Mixture of Experts models, keeping frequently used experts in memory while swapping others to storage.
- Proactive Loading: Predicts and pre-loads required weights before they are needed, minimizing latency during inference.
Even if NNtrainer runs on device, it provides full functionalities to train and infer models and also utilizes limited device resources efficiently. NNTrainer supports various machine learning algorithms such as k-Nearest Neighbor (k-NN), Neural Networks, Logistic Regression, Reinforcement Learning algorithms, Recurrent network, Transformers (LLM) and more. We also provide examples for various tasks such as Few-shot learning, ResNet, VGG, Product Rating, LLM Inference and more will be added. All of these were tested on Samsung Galaxy smart phone with Android and PC (Ubuntu).
Memory-Efficient LLM Inference on Edge Devices With NNTrainer , Open Source Summit 2025 Seoul
A New Frontier of AI: On-Device AI Training and Personalization , ICSE-SEIP, 2024
NNTrainer: Light-Weight On-Device Training Framework , arXiv, 2022
Open Source On-Device AI SW Platform , Samsung Developer Conference 2023 (Korean)
NNTrainer: Personalize neural networks on devices! , Samsung Developer Conference 2021
NNTrainer: "On-device learning" , Samsung AI Forum 2021
- 📱 Run Locally, Fully Offline: Perform training and inference directly on your edge devices without any internet connection. Ensure data privacy and low latency.
- 🛠️ On-Device Training & Personalization: Fine-tune models on-device with private user data. Supports Transfer Learning, Few-Shot Learning, and Continuous Learning scenarios.
- 🚀 Efficient LLM Inference: Run Large Language Models on memory-constrained devices. Features Flash Storage Utilization (FSU) to offload weights and MoE Cache for efficient Mixture-of-Experts execution.
- 🤖 Broad Model Support: Built-in support for various architectures including CNNs (ResNet, VGG), RNNs (LSTM, GRU), Transformers (LLaMA, Qwen, DeepSeek), and Reinforcement Learning.
- ⚡ High Performance & Lightweight: Optimized for embedded systems. Functionalities are designed to minimize memory footprint and maximize execution speed on ARM, x86, and NPU targets.
- 🔌 Cross-Platform: Deploy seamlessly across Tizen, Android, Linux, and Windows environments with consistent C/C++ APIs.
NNTrainer supports Large Language Model (LLM) inference! Please refer to Applications/CausalLM for details. We currently support various LLM architectures, including:
- Qwen3
- Qwen3-MoE
- GPT-OSS
- And more to come!
NNTrainer enables the execution of large-scale Mixture of Experts (MoE) models directly on-device. We support on-the-fly expert loading using flash storage, allowing MoE models to run with significantly reduced peak memory usage. Below are demonstrations of MoE LLMs running on mobile devices:
| GPT-OSS 20B | Qwen3 MoE 30B-A3B |
|---|---|
NNTrainer's FSU feature allows for dynamic expert loading during inference. This minimizes initialization time and ensures efficiency in memory-constrained environments. To test this, try the models located in Applications/CausalLM/models/*-slim.
| Load Whole model (Qwen3-30B-A3B) | Load Expert On-The-Fly (Qwen3-30B-A3B-Slim) |
|---|---|
| Memory: 16.5 GB | Memory: 1.3 GB |
| Tizen | Ubuntu | Android/NDK Build | Windows | |
|---|---|---|---|---|
| 7.0M2 and later | 22.04/24.04 | 9/P | windows-2022 and later | |
| arm64 | ||||
| x86_64 | N/A | |||
| Publish | Tizen Repo | PPA | NA | NA |
| API | C (Official) | C/C++ | C/C++ | C/C++ |
- Ready: CI system ensures build-ability and unit-testing. Users may easily build and execute. However, we do not have automated release & deployment system for this instance.
- Available: binary packages are released and deployed automatically and periodically along with CI tests.
- SDK Support: Tizen Studio (7.0 and later)
Instructions for installing NNTrainer.
Introductions for creating your own model.
Instructions for preparing NNTrainer for execution
NNTrainer examples for a variety of networks
- Supported ptimizer
- Supported Loss Functions
- Supported Activation Functions
- Supported Tensor
- Others
Currently, we provide C APIs for Tizen. C++ APIs are also provided for other platform. Java & C# APIs will be provided soon.
The NNtrainer is an open source project released under the terms of the Apache License version 2.0.
Contributions are welcome! Please see our Contributing Guide for more details.
If you find this NNTrainer project useful or relevant to your research, please consider citing our paper:
@inproceedings{10.1145/3639477.3639716,
author = {Moon, Jijoong and Lee, Hyeonseok and Chu, Jiho and Park, Donghak and Hong, Seungbaek and Seo, Hyungjun and Jeong, Donghyeon and Kong, Sungsik and Ham, Myungjoo},
title = {A New Frontier of AI: On-Device AI Training and Personalization},
year = {2024},
isbn = {9798400705014},
publisher = {Association for Computing Machinery},
url = {https://doi.org/10.1145/3639477.3639716},
doi = {10.1145/3639477.3639716},
booktitle = {Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice},
pages = {323–333},
numpages = {11},
keywords = {on-device AI, neural network, personalization, training, software framework},
series = {ICSE-SEIP '24}
}