tests/serial_test.py: add a serial echo test #18402
Conversation
Signed-off-by: Damien George <[email protected]>
Signed-off-by: Damien George <[email protected]>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #18402 +/- ##
=======================================
Coverage 98.38% 98.38%
=======================================
Files 171 171
Lines 22294 22294
=======================================
Hits 21933 21933
Misses 361 361 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
@andrewleech @iabdalkader this test uncovers a bug in TinyUSB's DWC2 backend (my current working theory), which is used by a lot of chips. Eg STM32 F4, F7 and N6 (but not WB!), as well as ESP32-S3. But it's only apparent when a macOS is the host PC. Would you be able to see if you can reproduce the issue that I'm seeing? To do that, build an STM32 board using TinyUSB that uses DWC2, and then run the script from this PR on macOS (with the board connected). I'm seeing it fail the echo test at a packet size of 65 (it repeats the 64 byte packet). You can also try with an ESP32-S3 if you like, you should see the same error. |
|
I don't have any macos to test with.
It sounds like the extra byte is buffered, maybe call |
|
It's actually a bug in TinyUSB, which was fixed very recently, see: hathach/tinyusb#3293 |
|
Oh nice find, I was about to test but looks like don't need to rush! |
Summary
The existing
serial_test.pyscript tests data in/out throughput and reliability. But it only tests data sizes that are a power of two. It's also important to test reliability of data in/out that is not a power of two.This PR adds a new echo sub-test to the
serial_test.pyscript. It sends out data to the board and gets the board to echo it back, and then compares the result (the echo'd data should be equal to the sent data). It does this for data packets between 1 and 520 (inclusive) bytes. That covers USB FS and HS packet sizes (64 and 512 respectively).It uses random data for the test, but seeded by a constant seed so that it's deterministic. If there's an error then it prints out all the sent and echo'd data to make it easier to see where it went wrong (eg if the previous packet was repeated).
Testing
Tested on PYBV11 with the stm32 and TinyUSB stacks:
Testing ESP32-S3 (uses TinyUSB), again it passes on Linux but fails on macOS.
So, this is an important test, it found a bug.
Trade-offs and Alternatives
It takes around 10 seconds to run on a board with UART REPL, but I'm not sure how to make that faster without decreasing the coverage of the test.