esp32: Fix USB Zero Length Packet issue via patched TinyUSB #18407

projectgus · 2025-11-12T06:07:00Z

Summary

This PR updates the TinyUSB component to a custom version that cherry-picks this patch: hathach/tinyusb#3293

This is necessary to avoid corrupt transfers when DMA is disabled for esp32 port (see #18332) and the host sends an OUT zero length packet (ZLP). This is particularly reproducible in serial transfers from macOS host (see #18402 for test program).

See #18406 for a TinyUSB update that includes the same fix for other ports.

Two other esp32 port changes were required as dependencies for this change:

Remove esp_tinyusb component

Espressif has an esp_tinyusb component which contains glue code for using TinyUSB with ESP-IDF. esp_tinyusb pulls in the "real" tinyusb component as a dependency. The IDF Component Manager doesn't allow overriding components that are added as dependencies this way (have requested it), so esp_tinyusb has been removed and replaced with a direct dependency on tinyusb.

While looking into this I noticed that MicroPython should actually not be building esp_tinyusb anyhow:

esp_tinyusb introduces a number of duplicate symbols that are also defined in MicroPython (for example, in descriptors_control.c). These don't currently fail the build as the linker doesn't examine these files, but it could.
esp_tinyusb includes its own tusb_config.h file and the tinyusb component source files are built with this config header, but MicroPython files are built with shared/tinyusb/tusb_config.h. This introduces the possibility for weird ABI incompatibility if the configs don't match.

The only user-facing changes are:

It's no longer possible to define a custom USB VID/PID/strings in an sdkconfig, the board must set MICROPY_HW_USB_VID, MICROPY_HW_USB_PID, etc. macros in mpconfigboard.h instead. (Existing boards are changed over in this PR.)
USB serial number is now generated the same as other MicroPython boards, based on the MAC address.

This change also makes #18332 redundant, as the config option it sets is set via esp_tinyusb (and the default is to disable DMA).

Remove component dependencies from mpconfigport.h

In order to build the TinyUSB component with MicroPython's tusb_config.h, the esp32 mpconfigport.h header needed to have some external dependencies (wifi, i2s driver) pulled out. This turned out to be pretty straightforward, and IMO it's a good change anyway not to have py/mpconfig.h pull in too many other things.

Testing

Ran the serial_test.py program from tests/serial_test.py: add a serial echo test #18402 on Linux and MacOS hosts with an ESP32_GENERIC_S3 board and one with octal SPIRAM, verified nothing failed.
I don't have an ESP32-S2 board with a native USB port, so was unable to test this.
Ran a USB keyboard HID program on an ESP32-S3 with Octal SPIRAM config, to double check that DMA is disabled.

Trade-offs and Alternatives

I tried creating an in-tree dcd_dwc2_patch.c file which contained the patched dcd_int_handler() function instead and then applying a linker wrap, but this didn't work (I think due to linker order). Decided that this approach was less hacky, out of the two. Working with the patch file was also how I noticed the other issues with esp_tinyusb component, though.

github-actions · 2025-11-12T06:18:26Z

Code size report:

Reference:  tests/serial_test.py: Allow up to 2 seconds between bytes. [2762fe6]
Comparison: esp32: Fix USB Zero Length Packet issue with patched tinyusb. [merge of f6b6e2c]
  mpy-cross:    +0 +0.000% 
   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:   +84 +0.022% TEENSY40[incl +4(data)]
        rp2:  +144 +0.016% RPI_PICO_W
       samd:   +80 +0.029% ADAFRUIT_ITSYBITSY_M4_EXPRESS[incl +4(data) -4(bss)]
  qemu rv32:    +0 +0.000% VIRT_RV32

This is necessary so the ESP-IDF TinyUSB component can include py/mpconfig.h, but is also a good design goal (less creep of symbols into unrelated parts of the code). This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>

codecov · 2025-11-12T06:19:55Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.38%. Comparing base (6773051) to head (21cc6f5).
⚠️ Report is 20 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #18407      +/-   ##
==========================================
- Coverage   98.38%   98.38%   -0.01%     
==========================================
  Files         171      171              
  Lines       22297    22294       -3     
==========================================
- Hits        21936    21933       -3     
  Misses        361      361

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

projectgus · 2025-11-12T06:36:01Z

I've reported the root cause of this to Espressif as well: espressif/esp-usb#315

roma-jam · 2025-11-12T17:12:52Z

Hi @projectgus,

The IDF Component Manager doesn't allow overriding components that are added as dependencies this way (espressif/idf-component-manager#99), so esp_tinyusb has been removed and replaced with a direct dependency on tinyusb.

may I clarify that after this change you will need the espressif/tinyusb as a IDF Component in the future?

projectgus · 2025-11-12T22:43:33Z

may I clarify that after this change you will need the espressif/tinyusb as a IDF Component in the future?

Yes, I'm very much hoping we can switch back to the Espressif TinyUSB component once the fix for espressif/esp-usb#315 is available in a component release (but this may not be soon.)

(In a perfect world we'd use the same lib/tinyusb submodule for all ports including esp32, and we did that for a while. However it's clear that some ESP-related changes won't make it into upstream TinyUSB for a while after we need them, so we need the esp32 port to use Espressif's TinyUSB fork. The recommended way to do that is via an IDF Component, so that's probably how we'll do it...)

projectgus · 2025-11-12T23:22:43Z

Have updated the sdkconfig entries to no longer use esp_tinyusb. Build UM_NANOS3 and verified it enumerated with the correct details.

(The UnexpectedMaker boards had hard-coded serial numbers before this PR, but now their serial numbers will be based on the ESP32 MAC address - same as other ports.)

dpgeorge · 2025-11-12T23:48:23Z

Quick comment: UM_FEATHERS3/sdkconfig.board needs TINYUSB_DESC_xxx entries removed.

Instead, depend directly on espressif/tinyusb component (which is otherwise transitively included via esp_tinyusb). Turns out esp_tinyusb builds a bunch of source files with symbols that conflict with our tinyusb symbols (i.e. descriptors_control.c). This only works because nothing in MicroPython causes the linker to include the esp_tinyusb.a library, however in order to override the dcd_int_handler (in following commit) this caused the linker to pull this library in and break the build. There's also a problematic header skew - TinyUSB component was building with the tusb_config.h file from esp_tinyusb component, but we have our own tusb_config.h file in shared/tinyusb. The changes in parent commit allow us to build the TinyUSB component with our tusb_config.h header. User-facing impacts are: - Can no longer override USB VID & PID via sdkconfig, have to set MICROPY_HW_USB_VID/PID instead (changes applied in this commit). - esp32 boards will have the same USB serial number as other ports (i.e. based on the hardware MAC address, not hard-coded). Side effects include: - CFG_TUD_DWC2_SLAVE_ENABLE is now set, DMA mode is disabled. Signed-off-by: Angus Gratton <[email protected]>

Temporarily switch from the espressif TinyUSB component to a MicroPython fork where this fix has been cherry-picked: hathach/tinyusb#3293 This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>

projectgus · 2025-11-12T23:52:00Z

Quick comment: UM_FEATHERS3/sdkconfig.board needs TINYUSB_DESC_xxx entries removed.

Fixed, thanks! I did these with a script, now idea how that snuck back in...

dpgeorge · 2025-11-13T00:01:30Z

I noticed that the default manufacturer/product string is now different. Eg ESP32_GENERIC_S3 used to be:

$ lsusb
...
Bus 001 Device 026: ID 303a:4001 Espressif Systems Espressif Device
...

and it's now changed to

$ lsusb
...
Bus 001 Device 032: ID 303a:4001 MicroPython Board in FS mode
...

I guess that's OK, it matches all other ports and keeps the configuration simple.

Although one could argue that the Espressif VID should go with the Espressif manufacturer name... so either we change the manufacturer or change the VID to the one all other ports use.

dpgeorge

Thanks @projectgus for fixing this. It was definitely more than I expected to get this working.

But, at the same time, the changes here are really good for the long term:

removing dependencies from mpconfigport.h is great
removing the esp_tinyusb component altogether is excellent! that's a really nice simplification and fixes potential issues with incorrect tusb_config.h being used
updating to use our fork of tinyusb-espressif is pretty simple and clean

Eventually we won't need to fork tinyusb-espressif. But it's also now very easy for us to switch esp32 to use upstream tinyusb (like we used to), it's really just a matter of swapping source files, no configuration would need to change.

I've tested this PR on the following hardware/boards:

ESP32_GENERIC_S3 (DevKit without PSRAM)
UM_FEATHERS2 (with PSRAM)
UM_TINYS3 (with PSRAM)

Tested using the updated serial_test.py with echo test, on Arch Linux and macOS. The test passes.

Also tested HID (simple keyboard example) on Arch Linux and macOS. It works.

roma-jam · 2025-11-13T13:19:44Z

Hi @projectgus,

thanks for the clarification! I am afraid that I still didn't get the answer to my original question. Let me rephrase it a bit.

Yes, I'm very much hoping we can switch back to the Espressif TinyUSB component once

Just to be clear, that there are three different repos:

upstream version of tinyusb
IDF component: espressif/tinyusb. Which is the upstream version + some ESP-related changes + component configuration to use it in ESP-IDF environment
IDF component: espressif/esp_tinyusb. This is a "wrapper" around the 2. and provides the tusb_config.h file and other class-related features, that should simplify the device stack usage.

esp_tinyusb includes its own tusb_config.h file and the tinyusb component source files are built with this config header, but MicroPython files are built with shared/tinyusb/tusb_config.h

Based on this comment, it seems that in this case the better option would be to use the component under number 2.
Or there is something else, that was not mentioned?

projectgus added the port-esp32 label Nov 12, 2025

This comment was marked as outdated.

Sign in to view

projectgus force-pushed the bugfix/esp32_usb_zlp branch from 21cc6f5 to f6b6e2c Compare November 12, 2025 06:24

This comment was marked as outdated.

Sign in to view

projectgus force-pushed the bugfix/esp32_usb_zlp branch from f6b6e2c to 3debe81 Compare November 12, 2025 06:28

projectgus force-pushed the bugfix/esp32_usb_zlp branch from 3debe81 to 49af741 Compare November 12, 2025 23:20

projectgus marked this pull request as ready for review November 12, 2025 23:21

projectgus requested a review from dpgeorge November 12, 2025 23:24

projectgus added 2 commits November 13, 2025 10:51

projectgus force-pushed the bugfix/esp32_usb_zlp branch from 49af741 to de51611 Compare November 12, 2025 23:51

dpgeorge approved these changes Nov 13, 2025

View reviewed changes

dpgeorge mentioned this pull request Nov 17, 2025

ESP32-S3 bulk transfer issues hathach/tinyusb#3154

Closed

1 task

Uh oh!

esp32: Fix USB Zero Length Packet issue via patched TinyUSB #18407

Are you sure you want to change the base?

esp32: Fix USB Zero Length Packet issue via patched TinyUSB #18407

Conversation

projectgus commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Remove esp_tinyusb component

Remove component dependencies from mpconfigport.h

Testing

Trade-offs and Alternatives

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

github-actions bot commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Nov 12, 2025

Codecov Report

Uh oh!

This comment was marked as outdated.

projectgus commented Nov 12, 2025

Uh oh!

roma-jam commented Nov 12, 2025

Uh oh!

projectgus commented Nov 12, 2025

Uh oh!

projectgus commented Nov 12, 2025

Uh oh!

dpgeorge commented Nov 12, 2025

Uh oh!

projectgus commented Nov 12, 2025

Uh oh!

dpgeorge commented Nov 13, 2025

Uh oh!

dpgeorge left a comment

Choose a reason for hiding this comment

Uh oh!

roma-jam commented Nov 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

projectgus commented Nov 12, 2025 •

edited

Loading

github-actions bot commented Nov 12, 2025 •

edited

Loading