Thanks to visit codestin.com
Credit goes to github.com

Skip to content

ultralytics 8.3.220 Add ExecuTorch export (.pte) with XNNPACK#22244

Merged
glenn-jocher merged 84 commits into
mainfrom
executorch-export
Oct 22, 2025
Merged

ultralytics 8.3.220 Add ExecuTorch export (.pte) with XNNPACK#22244
glenn-jocher merged 84 commits into
mainfrom
executorch-export

Conversation

@ambitious-octopus

@ambitious-octopus ambitious-octopus commented Sep 29, 2025

Copy link
Copy Markdown
Contributor

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Adds first-class ExecuTorch support to export and run YOLO11 models on mobile and edge devices, with docs, tests, and CLI/Python integration. 🚀

📊 Key Changes

  • ExecuTorch export added:
    • New format=executorch option creates a <model>_executorch_model/ folder with .pte and metadata.yaml.
    • Export implemented with XNNPACK partitioner; requires Python 3.10+, Torch ≥ 2.9.0, executorch==1.0.0, and FlatBuffers.
  • Inference support:
    • AutoBackend can now load .pte files and run inference via the ExecuTorch runtime.
    • CLI/Python flows updated to accept ExecuTorch models for predict.
  • Documentation and examples:
    • New ExecuTorch integration guide with install, export, mobile/embedded integration, and benchmarks.
    • Export tables and tutorial updated to include ExecuTorch; added to Integrations index and site nav.
  • Testing and compatibility:
    • New tests for ExecuTorch export (skipped on Windows; gated by Python/Torch versions).
    • Version bump to 8.3.220.
  • Misc docs polish:
    • Minor copy tweaks (e.g., Seeed Studio reCamera description).

🎯 Purpose & Impact

  • Seamless edge deployment: One-command export to a portable .pte format optimized for on-device inference on iOS, Android, and embedded Linux. 📱🛠️
  • Faster CPU performance: Default XNNPACK backend provides strong mobile CPU inference without specialized hardware.
  • Better developer experience: Unified CLI/Python workflow for export and predict, structured output with metadata, and clear docs and examples.
  • Reliability and CI coverage: New tests ensure export correctness across tasks, improving stability.
  • Broader ecosystem support: Enables YOLO11 to fit natively into PyTorch’s edge stack, complementing existing formats like ONNX, TFLite, and TensorRT.

@ambitious-octopus ambitious-octopus self-assigned this Sep 29, 2025
@ambitious-octopus ambitious-octopus marked this pull request as draft September 29, 2025 10:58
@onuralpszr onuralpszr self-assigned this Sep 29, 2025
@UltralyticsAssistant UltralyticsAssistant added enhancement New feature or request exports Model exports (ONNX, TensorRT, TFLite, etc.) python Pull requests that update python code labels Sep 29, 2025
@UltralyticsAssistant

Copy link
Copy Markdown
Member

👋 Hello @ambitious-octopus, thank you for submitting an ultralytics/ultralytics 🚀 PR! This is an automated response — an Ultralytics engineer will review and assist you shortly. To ensure a seamless integration of your work, please review the following checklist:

  • Define a Purpose: Clearly explain the purpose of your fix or feature in your PR description, and link to any relevant issues. Ensure your commit messages are clear, concise, and adhere to the project's conventions.
  • Synchronize with Source: Confirm your PR is synchronized with the ultralytics/ultralytics main branch. If it's behind, update it by clicking the 'Update branch' button or by running git pull and git merge main locally.
  • Ensure CI Checks Pass: Verify all Ultralytics Continuous Integration (CI) checks are passing. If any checks fail, please address the issues.
  • Update Documentation: Update the relevant documentation for any new or modified features.
  • Add Tests: If applicable, include or update tests to cover your changes, and confirm that all tests are passing.
  • Sign the CLA: Please ensure you have signed our Contributor License Agreement if this is your first Ultralytics PR by writing "I have read the CLA Document and I sign the CLA" in a new message.
  • Minimize Changes: Limit your changes to the minimum necessary for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." — Bruce Lee

Additional notes for this PR:

  • Please confirm the new executorch export target is consistently listed in export_formats() and correctly wired in the exporter flow end-to-end.
  • Double-check naming consistency (format string, suffix, and user-facing messages) and that the new method is covered by unit tests, including edge cases like dynamic shapes or invalid inputs.
  • Ensure dependency checks (executorch>=0.7.0) are properly guarded and error messages are clear if the requirement isn’t met.
  • If there are known limitations (e.g., fixed input shape), kindly document them in code comments and in the user docs update.
  • Consider adding a brief usage snippet to the export docs to align with existing exporter UX, matching the style of other formats.

For more guidance, please refer to our Contributing Guide. Don’t hesitate to leave a comment if you have any questions. Thank you for contributing to Ultralytics! ⚡📱

@onuralpszr

Copy link
Copy Markdown
Member

The reason we added setuptools because until 0.7.0 there is a usage of "pkg_resources" and it is about to be removed in setuptools and there is a deprecation warning in code as well. But good news is in recent PR it is already fixed. : pytorch/executorch#13513 (it should be available in next release)

When new package release we can remove setuptools as well.

cc @ambitious-octopus

@onuralpszr

Copy link
Copy Markdown
Member

I also found one left over pkg_resource import in samsung backend : pytorch/executorch#14654
Hopefully it will get merged as well.

@ambitious-octopus ambitious-octopus left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@onuralpszr, nice! I'm strarting working on quantization.
Let's try to have XNNPACK Backend functional for this PR!

@codecov

codecov Bot commented Sep 29, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 64.10256% with 14 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
ultralytics/nn/autobackend.py 6.66% 14 Missing ⚠️

📢 Thoughts on this report? Let us know!

@onuralpszr

Copy link
Copy Markdown
Member
from ultralytics import YOLO

model = YOLO("yolo11n.pt")

executorch_path = model.export(format="executorch")
model = YOLO(executorch_path)

model("https://ultralytics.com/images/bus.jpg",task='detect',imgsz=640, conf=0.25, save=True)

@ambitious-octopus , I made some changes on better export and also autobackend part. I also had so make some changes on suffix part to avoid ".pt" vs ".pte" cases. Plus I had to add "yaml" file for metadata for proper "class names" in inference

@ambitious-octopus ambitious-octopus left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

YOLO11n + XNNPACK on Raspberry Pi 5

Initial performance tests show significant improvement using ExecuTorch with XNNPACK optimization:

Results on COCO128:

  • Baseline PyTorch: 354.5ms inference
  • ExecuTorch + XNNPACK: 173.6ms inference
  • Speedup: ~2x faster

Setup Instructions:

  1. Compile ExecuTorch from source (warning: takes 40+ minutes on Pi):
   # Follow build instructions:
   # https://docs.pytorch.org/executorch/stable/using-executorch-building-from-source.html
  1. Export model:
yolo export model=yolo11n.pt format=executorch
  1. Run validation:
yolo val model=yolo11n_executorch_model data=coco128.yaml

Tested on Raspberry Pi 5. XNNPACK provides ~2x speedup for edge deployment.
cc: @onuralpszr @lakshanthad

@lakshanthad

lakshanthad commented Sep 30, 2025

Copy link
Copy Markdown
Collaborator

Great new export addition guys! @ambitious-octopus @onuralpszr

Here is performance testing on Raspberry Pi 5 16GB:

Results on COCO128 with YOLO11n:

  • Baseline PyTorch: 373.1ms inference
  • ExecuTorch + XNNPACK: 184.2ms inference
  • Speedup: ~2x faster

Note: It takes about 20 minutes to compile ExecuTorch from source on a Raspberry Pi 5 16GB.

I have also investigated about aarch64 Linux wheels for Executorch on Executorch GitHub and have come across the following PR which has added the wheel building support on a nightly release cadence but not uploading to PyPi.
pytorch/executorch#13946

So I have gone ahead and tried the latest nighly release from below with Python3.11 support which comes with Raspberry Pi OS Bookworm (Debian 12):
https://github.com/pytorch/executorch/actions/runs/18128543623

Installation of the above wheel resulted in a missing module:

ModuleNotFoundError: No module named 'torchao'

But I was able to resolve it by pip install torchao (torchao has PyPi aarch64 wheel) and then able to successfully export a model with ExecuTorch.

So I think we have 2 options here:

  1. Host a compiled Executorch aarch64 Linux wheel on our GitHub assets.
  2. Discuss with ExecuTorch guys and maybe Raspberry Pi guys as well to have the official aarch64 Linux wheel support on PyPi.

@onuralpszr

Copy link
Copy Markdown
Member

@lakshanthad personally, I wanna see "pypi" push first instead of assets, secondly we need to think about cpp examples because better inference speed and most of the example in their docs also revolving around cpp as well.

@lakshanthad

Copy link
Copy Markdown
Collaborator

I understand @onuralpszr. However PyPi push has been discussed in multiple issues and seems the wheels never made it there.

Yeah definitely we can experiment around CPP as well!

@onuralpszr onuralpszr self-requested a review October 21, 2025 20:33
@onuralpszr

onuralpszr commented Oct 21, 2025

Copy link
Copy Markdown
Member

⚡ Actions Trigger

Made with ❤️ by Ultralytics Actions

GitHub Actions below triggered via workflow dispatch on this PR branch executorch-export at 2025-10-21 20:44:17 UTC with @ultralytics/run-ci command:

@onuralpszr

Copy link
Copy Markdown
Member

@ambitious-octopus @lakshanthad must torch 2.9 because of TorchAO 0.14 needs 2.9 (pytorch/ao#2919)

image

@onuralpszr

onuralpszr commented Oct 21, 2025

Copy link
Copy Markdown
Member

@glenn-jocher had to make some extra adjustment because of torch 2.9.0 but it is ready for initial merge, tests are should passed, slow CI last minute changes because of docker side was using torch 2.8 make me re-check dep and I make sure executorch using correct packages and make sure match with supported with TorchAO as well.

cc @ambitious-octopus

@glenn-jocher

Copy link
Copy Markdown
Member

Thanks @onuralpszr, looking through this now

Signed-off-by: Glenn Jocher <[email protected]>
@glenn-jocher glenn-jocher changed the title New export format: executorch xnnpack ultralytics 8.3.220 Add ExecuTorch export (.pte) with XNNPACK Oct 22, 2025

@UltralyticsAssistant UltralyticsAssistant left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 PR Review

Made with ❤️ by Ultralytics Actions

Thanks for the ExecuTorch integration! A few of the new tests gate on Torch versions using string comparisons, which will fail once the minor version reaches double digits (e.g., 2.10). Please switch these skip conditions to use the boolean helpers from torch_utils (like TORCH_2_9) so the tests continue to run on newer Torch releases.

💬 Posted 3 inline comments

Comment thread tests/test_cli.py Outdated
Comment thread tests/test_cli.py Outdated

@UltralyticsAssistant UltralyticsAssistant left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 PR Review 2

Made with ❤️ by Ultralytics Actions

ExecuTorch export currently hard-blocks on torch>=2.9, which makes the new feature unusable; please relax the version check to the real minimum requirement.

💬 Posted 1 inline comment

Comment thread ultralytics/engine/exporter.py
@glenn-jocher glenn-jocher merged commit 464202f into main Oct 22, 2025
19 checks passed
@glenn-jocher glenn-jocher deleted the executorch-export branch October 22, 2025 13:19
@UltralyticsAssistant UltralyticsAssistant removed the TODO High priority items label Oct 22, 2025
@UltralyticsAssistant

Copy link
Copy Markdown
Member

🎉 Merged! Huge thanks to @ambitious-octopus for leading PR #22244, with stellar contributions from @larryliu0820, @lakshanthad, @Y-T-G, @onuralpszr, @mergennachin, and @glenn-jocher. This brings first-class ExecuTorch support to YOLO11—unlocking seamless mobile and edge deployment with a unified CLI/Python experience, solid docs, and reliable tests.

“Innovation is the ability to see change as an opportunity — not a threat.” — Steve Jobs
Your work turns opportunity into reality: export once, run anywhere, and accelerate CPU inference on-device. 🚀

Quickstart:

  • CLI: yolo export model=yolo11n.pt format=executorch && yolo predict model=yolo11n_executorch_model/model.pte
  • Python:
from ultralytics import YOLO

model = YOLO("yolo11n.pt")
model.export(format="executorch")
YOLO("yolo11n_executorch_model/model.pte")(source="test.jpg")

Appreciate the craftsmanship, polish, and care—this meaningfully grows the Ultralytics ecosystem and empowers developers from Ultralytics HUB to embedded devices. Thank you all! 🙌

khatami-mehrdad pushed a commit to DeGirum/compiler_ultralytics that referenced this pull request Apr 24, 2026
…alytics#22244)

Signed-off-by: Onuralp SEZER <[email protected]>
Signed-off-by: Glenn Jocher <[email protected]>
Signed-off-by: Lakshantha Dissanayake <[email protected]>
Co-authored-by: UltralyticsAssistant <[email protected]>
Co-authored-by: Onuralp SEZER <[email protected]>
Co-authored-by: Glenn Jocher <[email protected]>
Co-authored-by: Lakshantha Dissanayake <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request exports Model exports (ONNX, TensorRT, TFLite, etc.) python Pull requests that update python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants