This repository is part of the work presented in:

XR-NPE: High-Throughput Mixed-Precision SIMD Neural Processing Engine for Extended Reality Perception Workloads

Augmented-Reality

Virtual-Reality

Mixed-Reality

This repository provides Python implementations of multi-precision quantization for various computer vision and sensor fusion workloads.
It has three workloads:

• Gaze-LLE – Eye gaze extraction
• ResNet – Image classification
• EfficientNet – Image classification
• UL-VIO – Visual-Inertial Odometry

The code supports the following quantization formats:

• FP4
• FP8
• Posit4
• Posit8
• Posit16
• BF16
• Mixed-Precision

This facilitates researchers and practitioners to explore the trade-offs across accuracy, latency, and resource usage.

• 42% area and 38% power reduction compared to SoTA MAC engines.
• 23% higher energy efficiency for VIO workloads.
• 4% better compute density at CMOS 28nm.

• Layer-Adaptive Mixed-Precision quantization (FP4/Posit/Mixed Precision).
• Reduced memory bandwidth with extreme compression (up to 4.2× smaller models).
• Reconfigurable Mantissa Multiplication & Exponent Circuitry (RMMEC) for dark-silicon reduction.
• 2.85× improved arithmetic intensity compared to state-of-the-art MAC engines.
• Hardware + Algorithm co-design (FPGA/ASIC friendly).

XR-NPE/
│
├── Gaze-LLE/   # Eye gaze estimation quantization — BF16, FP4, FP8, Posit4, Posit8
│
├── Resnet/     # ResNet image classification quantization — BF16, FP4, FP8, Posit8
│
├── Efficientnet/     # ResNet image classification quantization — BF16, FP4, FP8, Posit8
│
├── UL-VIO/     # Visual–Inertial Odometry quantization — BF16+INT8, FP4, FP8, Mixed Precision (Posit8+FP4), Posit4/8/16
│
└── README.md   # Generic README

Clone the repository:

git clone https://github.com/XXXXXXXXXXXXXXXX/XR-NPE.git
cd XR-NPE
pip install -r requirements.txt
pip install -r requirements_extra.txt  #for some additional library might be used

Gaze-LLE

ResNet

UL-VIO

• Gaze-LLE
• ResNet
• UL-VIO