ultralytics 8.3.220 Add ExecuTorch export (.pte) with XNNPACK#22244
Conversation
|
👋 Hello @ambitious-octopus, thank you for submitting an
Additional notes for this PR:
For more guidance, please refer to our Contributing Guide. Don’t hesitate to leave a comment if you have any questions. Thank you for contributing to Ultralytics! ⚡📱 |
|
The reason we added setuptools because until 0.7.0 there is a usage of "pkg_resources" and it is about to be removed in setuptools and there is a deprecation warning in code as well. But good news is in recent PR it is already fixed. : pytorch/executorch#13513 (it should be available in next release) When new package release we can remove setuptools as well. |
|
I also found one left over pkg_resource import in samsung backend : pytorch/executorch#14654 |
There was a problem hiding this comment.
@onuralpszr, nice! I'm strarting working on quantization.
Let's try to have XNNPACK Backend functional for this PR!
…and inference capabilities
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
from ultralytics import YOLO
model = YOLO("yolo11n.pt")
executorch_path = model.export(format="executorch")
model = YOLO(executorch_path)
model("https://ultralytics.com/images/bus.jpg",task='detect',imgsz=640, conf=0.25, save=True)@ambitious-octopus , I made some changes on better export and also autobackend part. I also had so make some changes on suffix part to avoid ".pt" vs ".pte" cases. Plus I had to add "yaml" file for metadata for proper "class names" in inference |
…nd suffix checks Signed-off-by: Onuralp SEZER <[email protected]>
There was a problem hiding this comment.
YOLO11n + XNNPACK on Raspberry Pi 5
Initial performance tests show significant improvement using ExecuTorch with XNNPACK optimization:
Results on COCO128:
- Baseline PyTorch: 354.5ms inference
- ExecuTorch + XNNPACK: 173.6ms inference
- Speedup: ~2x faster
Setup Instructions:
- Compile ExecuTorch from source (warning: takes 40+ minutes on Pi):
# Follow build instructions:
# https://docs.pytorch.org/executorch/stable/using-executorch-building-from-source.html- Export model:
yolo export model=yolo11n.pt format=executorch- Run validation:
yolo val model=yolo11n_executorch_model data=coco128.yamlTested on Raspberry Pi 5. XNNPACK provides ~2x speedup for edge deployment.
cc: @onuralpszr @lakshanthad
|
Great new export addition guys! @ambitious-octopus @onuralpszr Here is performance testing on Raspberry Pi 5 16GB: Results on COCO128 with YOLO11n:
Note: It takes about 20 minutes to compile ExecuTorch from source on a Raspberry Pi 5 16GB. I have also investigated about So I have gone ahead and tried the latest nighly release from below with Python3.11 support which comes with Raspberry Pi OS Bookworm (Debian 12): Installation of the above wheel resulted in a missing module: ModuleNotFoundError: No module named 'torchao'But I was able to resolve it by So I think we have 2 options here:
|
|
@lakshanthad personally, I wanna see "pypi" push first instead of assets, secondly we need to think about cpp examples because better inference speed and most of the example in their docs also revolving around cpp as well. |
|
I understand @onuralpszr. However PyPi push has been discussed in multiple issues and seems the wheels never made it there.
Yeah definitely we can experiment around CPP as well! |
…sions to >=2.9.0 Signed-off-by: Onuralp SEZER <[email protected]>
⚡ Actions TriggerMade with ❤️ by Ultralytics Actions GitHub Actions below triggered via workflow dispatch on this PR branch
|
|
@ambitious-octopus @lakshanthad must torch 2.9 because of TorchAO 0.14 needs 2.9 (pytorch/ao#2919)
|
|
@glenn-jocher had to make some extra adjustment because of torch 2.9.0 but it is ready for initial merge, tests are should passed, slow CI last minute changes because of docker side was using torch 2.8 make me re-check dep and I make sure executorch using correct packages and make sure match with supported with TorchAO as well. |
|
Thanks @onuralpszr, looking through this now |
Signed-off-by: Glenn Jocher <[email protected]>
ultralytics 8.3.220 Add ExecuTorch export (.pte) with XNNPACK
UltralyticsAssistant
left a comment
There was a problem hiding this comment.
🔍 PR Review
Made with ❤️ by Ultralytics Actions
Thanks for the ExecuTorch integration! A few of the new tests gate on Torch versions using string comparisons, which will fail once the minor version reaches double digits (e.g., 2.10). Please switch these skip conditions to use the boolean helpers from torch_utils (like TORCH_2_9) so the tests continue to run on newer Torch releases.
💬 Posted 3 inline comments
Signed-off-by: Glenn Jocher <[email protected]>
Signed-off-by: Glenn Jocher <[email protected]>
Signed-off-by: Glenn Jocher <[email protected]>
UltralyticsAssistant
left a comment
There was a problem hiding this comment.
🔍 PR Review 2
Made with ❤️ by Ultralytics Actions
ExecuTorch export currently hard-blocks on torch>=2.9, which makes the new feature unusable; please relax the version check to the real minimum requirement.
💬 Posted 1 inline comment
Signed-off-by: Onuralp SEZER <[email protected]>
|
🎉 Merged! Huge thanks to @ambitious-octopus for leading PR #22244, with stellar contributions from @larryliu0820, @lakshanthad, @Y-T-G, @onuralpszr, @mergennachin, and @glenn-jocher. This brings first-class ExecuTorch support to YOLO11—unlocking seamless mobile and edge deployment with a unified CLI/Python experience, solid docs, and reliable tests. “Innovation is the ability to see change as an opportunity — not a threat.” — Steve Jobs Quickstart:
from ultralytics import YOLO
model = YOLO("yolo11n.pt")
model.export(format="executorch")
YOLO("yolo11n_executorch_model/model.pte")(source="test.jpg")Appreciate the craftsmanship, polish, and care—this meaningfully grows the Ultralytics ecosystem and empowers developers from Ultralytics HUB to embedded devices. Thank you all! 🙌 |
…alytics#22244) Signed-off-by: Onuralp SEZER <[email protected]> Signed-off-by: Glenn Jocher <[email protected]> Signed-off-by: Lakshantha Dissanayake <[email protected]> Co-authored-by: UltralyticsAssistant <[email protected]> Co-authored-by: Onuralp SEZER <[email protected]> Co-authored-by: Glenn Jocher <[email protected]> Co-authored-by: Lakshantha Dissanayake <[email protected]>

🛠️ PR Summary
Made with ❤️ by Ultralytics Actions
🌟 Summary
Adds first-class ExecuTorch support to export and run YOLO11 models on mobile and edge devices, with docs, tests, and CLI/Python integration. 🚀
📊 Key Changes
format=executorchoption creates a<model>_executorch_model/folder with.pteandmetadata.yaml.executorch==1.0.0, and FlatBuffers..ptefiles and run inference via the ExecuTorch runtime.predict.8.3.220.🎯 Purpose & Impact
.pteformat optimized for on-device inference on iOS, Android, and embedded Linux. 📱🛠️