Thanks to visit codestin.com
Credit goes to github.com

Skip to content

py,esp32,rp2: Add new cstack API, check margin macro, fix esp32 stack check. #15605

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Aug 14, 2024

Conversation

projectgus
Copy link
Contributor

@projectgus projectgus commented Aug 6, 2024

Summary

  • Motivation for this was testing the ESP32-C3 bug fix in esp32: Use configured stack-limit-margin when creating threads. #15583 and realising there is a subtle race condition on esp32 port where the stack check limit may not be set reliably, depending on function timing.
  • On looking at the whole situation, every port adds some "stack limit margin" but it's usually hard-coded in the call to mp_stack_set_limit(). Threaded ports have the extra challenge of adding a margin in the thread module. However only the value passed to mp_stack_set_limit() needs to have the margin applied (i.e. if the user sets a thread stack size of N bytes, the tasks will have a real stack size of N bytes and the stack limit checker sets its value to N - margin bytes.)
  • After discussing with Damien this PR has expanded to add both a MICROPY_STACK_CHECK_MARGIN macro and a new cstack API which sets the stack origin and the stack limit (based off the stack size) in one call.
  • To offset the code size impact on ports without threads, new cstack API can be inlined in some cases.
  • Old stackctrl.h API is kept unchanged, apart from a note about deprecation and macros to disable when building MICROPY_PREVIEW_VERSION_2.
  • esp32, unix and rp2 ports are converted in this PR as initial examples.

Testing

  • Ran stress and thread tests on unix port, esp32 port (S3 and C3), and on rp2 port.
  • A commit in this PR enables the stress tests for the esp32 port.

Trade-offs and Alternatives

Could have made a much smaller fix for the esp32 port race condition by adjusting the stack size before creating the thread, or by applying a mutex. However this approach seems easier to understand overall, and implements a single source of truth for the stack limit margin.

Follow-up work

  • Refactor other ports to use the new API.
  • Suspect the stack limitmargin is set quite conservatively (especially for ESP32-C3 where it is 2KB). Could experiment with reducing this to save some RAM.

Copy link

codecov bot commented Aug 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.43%. Comparing base (288a036) to head (1342367).
Report is 3 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #15605      +/-   ##
==========================================
- Coverage   98.43%   98.43%   -0.01%     
==========================================
  Files         161      163       +2     
  Lines       21281    21293      +12     
==========================================
+ Hits        20948    20959      +11     
- Misses        333      334       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

github-actions bot commented Aug 6, 2024

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:  -104 -0.012% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:   -48 -0.005% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS

@dpgeorge
Copy link
Member

dpgeorge commented Aug 7, 2024

An alternative to this would be to still have the MICROPY_STACK_CHECK_MARGIN macro but use it explicitly at the call site for mp_stack_set_limit(). Eg in mp_thread_init_state() it would do:

    mp_stack_set_limit(stack_size - MICROPY_STACK_CHECK_MARGIN);

That makes it more obvious that the limit is the stack size minus an explicit margin. Otherwise the margin is kind of hidden in the stackctrl.h implementation.

But, this alternative approach is more verbose (more duplicated code with the subtraction of the margin everywhere) so probably not any better than what this PR does.

@projectgus projectgus force-pushed the bugfix/esp32_c3_stack_check branch 9 times, most recently from 44b6a19 to ec2e85a Compare August 13, 2024 07:05
@projectgus projectgus changed the title py,esp32,rp2: Add new stack check margin macro, fix esp32 stack check. py,esp32,rp2: Add new cstack API, check margin macro, fix esp32 stack check. Aug 13, 2024
@projectgus
Copy link
Contributor Author

@dpgeorge Have updated according to our discussion earlier, and updated the PR description.

The project coverage check is failing because stackctrl.c is no longer covered. What's the best way to deal with that?

@dpgeorge
Copy link
Member

The project coverage check is failing because stackctrl.c is no longer covered. What's the best way to deal with that?

Hmmmm... three options I can think of:

  1. Add code to ports/unix/coverage.c to call the old functions to get coverage.
  2. Add a #if MICROPY_STACK_CHECK_LEGACY around everything in py/stackctrl.c and disable this on unix coverage builds, to exclude that code from coverage checks.
  3. Implement the new functions in terms of the old ones.

Option 3 makes it harder to easily remove the old functions when the time comes. Probably option 1 is the best.

@projectgus projectgus force-pushed the bugfix/esp32_c3_stack_check branch from ec2e85a to 5a0038f Compare August 13, 2024 23:23
@projectgus
Copy link
Contributor Author

projectgus commented Aug 13, 2024

Option 3 makes it harder to easily remove the old functions when the time comes. Probably option 1 is the best.

Thanks, have pushed a commit to try that approach (and coverage seems happy, at least.)

@dpgeorge
Copy link
Member

This is looking really good now! Just a few minor comments.

@projectgus projectgus force-pushed the bugfix/esp32_c3_stack_check branch from 5a0038f to 1342367 Compare August 14, 2024 01:29
@projectgus projectgus force-pushed the bugfix/esp32_c3_stack_check branch from 1342367 to 15f53b5 Compare August 14, 2024 02:15
Currently the stack limit margin is hard-coded in each port's call to
`mp_stack_set_limit()`, but on threaded ports it's fiddlier and can lead to
bugs (such as incorrect thread stack margin on esp32).

This commit provides a new API to initialise the C Stack in one function
call, with a config macro to set the margin.  Where possible the new call
is inlined to reduce code size in thread-free ports.

Intended replacement for `MP_TASK_STACK_LIMIT_MARGIN` on esp32.

The previous `stackctrl.h` API is still present and unmodified apart from a
deprecation comment.  However it's not available when the
`MICROPY_PREVIEW_VERSION_2` macro is set.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
This change moves that complexity out into the stack checker and fixes the
bug where stack margin wasn't set correctly by ESP32-C3 threads.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
Now passing on ESP32-S3 and ESP32-C3.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
Necessary to pass CI when testing the V2 preview APIs.

Also adds an extra coverage test for the legacy stackctrl API, to maintain
coverage and check for any regression.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
@dpgeorge dpgeorge force-pushed the bugfix/esp32_c3_stack_check branch from 15f53b5 to a6fa85d Compare August 14, 2024 02:58
@dpgeorge dpgeorge merged commit a6fa85d into micropython:master Aug 14, 2024
63 checks passed
@dpgeorge
Copy link
Member

Thanks for updating, now merged.

@dpgeorge
Copy link
Member

@projectgus looks like this may have broken multi_bluetooth/ble_irq_calls.py on ESP32-C3. That test tests for recursion within a BLE IRQ handler, and now it seems that ESP32-C3 has less stack for Python calls because of the increased margin, so the test fails.

The BLE IRQ handler on esp32 has CONFIG_BT_NIMBLE_TASK_STACK_SIZE=6144 minus 2048 minus MICROPY_STACK_CHECK_MARGIN bytes. So that's very small on ESP32-C3/RISC-V.

To fix it I suggest decreasing MICROPY_STACK_CHECK_MARGIN on RISC-V to a less conservative value. If that still doesn't fix multi_bluetooth/ble_irq_calls.py then we should probably increase CONFIG_BT_NIMBLE_TASK_STACK_SIZE on RISC-V.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug port-esp32 port-rp2 py-core Relates to py/ directory in source
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants