core,rp2,esp8266,windows, unix: Add new cross-port functions for event waiting and handling. #13096

projectgus · 2023-11-30T06:07:22Z

This PR fixes #12925, which is a regression on rp2 introduced when rp2 became "tickless" in #12837.

It implements one of the suggestions from the linked Issue: to add new functions for "wait for event" and "handle events". The new functions are port-agnostic:

mp_event_handle_nowait() - Handle any pending MicroPython events without waiting for an interrupt or event.
mp_event_wait_indefinite() - Handle any pending MicroPython events and then suspend execution until the next interrupt or event. This function is deliberately named to attract attention when developers use it, as on tickless ports it will cause issues if the developer expects it to return without a corresponding interrupt having occurred.
mp_event_wait_ms(mp_uint_t timeout_ms) - Handle any pending MicroPython events and then suspend execution until the next interrupt or event, or until timeout_ms milliseconds have elapsed.

Each port only needs to provide a MICROPY_INTERNAL_WFE(OPTIONAL_TIMEOUT) macro to implement the "wait for event" logic. More port-specific macros may be needed when adding other ports.

These functions are intended to eventually replace MICROPY_EVENT_POLL_HOOK and MICROPY_EVENT_POLL_HOOK_FAST, although on all ports except rp2 they are currently implemented using these macros.

This PR switches over extmod, rp2 port and the built-in drivers supported by rp2 to use the new functions, fixing the issue of indefinite sleep under some conditions.

It also switches over esp8266, to avoid problems where i2c.scan() doesn't call ets_event_poll() and can trigger a WDT.

Finally it switches over Windows, as this was necessary to get the same sleep behaviour with the new functions, and unix so that the coverage tests pass on the new functions.

rp2 port also enables SEVONPEND to ensure SEV bit is set on each interrupt, removing potential race conditions between the interrupt and the main thread checking a hardware condition. It may be possible to make this more fine-grained in future by manually calling SEV from individual ISRs, but this approach ensures the firmware can't become stuck if a SEV is missing.

This work was funded through GitHub Sponsors.

github-actions · 2023-11-30T06:16:15Z

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:   +80 +0.010% standard
      stm32:   +40 +0.010% PYBV10
     mimxrt:   +40 +0.011% TEENSY40
        rp2:   +32 +0.010% RPI_PICO
       samd:   +40 +0.015% ADAFRUIT_ITSYBITSY_M4_EXPRESS

projectgus · 2023-11-30T06:53:53Z

Hmm, so the challenge from changing macros to functions here is ports with threading enabled.

A lot of the ports with threading call MP_THREAD_GIL_EXIT() & MP_THREAD_GIL_ENTER() as part of their wait for event hook. These macros are defined in py/mpthread.h and they depend on some other headers as well.

This creates a layering violation with mphal.h as mphal.h is a pretty low level header and needs to be included before many others. The layering violation doesn't matter for macro expansion, provided all of the relevant parts have been included when the macro expands.

Options would seem to be:

Make these back into macros.
Put these functions somewhere else, a new header like py/event.h or perhaps py/runtime.h (the existing header which contains the closest related functions, from what I can see, plus the macros currently extern a function that's defined in runtime.h.)

A version of the second option seems better from the point of view of having headers that "include what they use" and layer nicely, but the first option will be quick and easy...

projectgus · 2023-11-30T06:59:09Z

Oh, I thought of a third option: Convert mp_wait_for_event_indefinite() and mp_wait_for_event(mp_uint_t timeout_ms) to normal functions instead of inline. Then they can include whatever they need, and there will be a small code size win.

(Unlike the "nowait" version, these generally get called in places where I don't expect the overhead of one more function call will be very noticeable.)

py/mphal.h

dpgeorge · 2023-12-06T04:30:03Z

py/runtime.h

@@ -26,6 +26,7 @@
 #ifndef MICROPY_INCLUDED_PY_RUNTIME_H
 #define MICROPY_INCLUDED_PY_RUNTIME_H

+#include "py/mphal.h"


This breaks the example/natmod builds... I think just remove it from here. This header should only be included from .c files.

This is tricky. Without a definition of MICROPY_INTERNAL_WFE, including runtime.h will fail (the difference between macros and inline functions is they force you to structure your headers with a more layered "include what you use"!)

Conceptually these "event" functions are a bit different, in as much as runtime.h is "pure Python runtime", mphal.h is "hardware abstractions", and then event functions are a bridge between the two.

I've updated this PR to put them into a new py/event.h header, which solves these kind of conflicts (and allows removing all the messing about with esp8266 headers, too.)

Let me know what you think.

I'm pretty sure I tried just removing this include and the build worked.... but maybe I didn't test enough.

How about putting these inline functions in py/mphal.h? As you say, they are half hardware abstraction functions.

How about putting these inline functions in py/mphal.h? As you say, they are half hardware abstraction functions.

Hmm, maybe that's a bad idea, because then py/mphal.h will need to include py/runtime.h, which is no good.

Yeah, that's it. My very first version of this added the inlines tomphal.h and ran into that problem.

I think conceptually:

mphal.h is "lower" and relates to the hardware.

runtime.h is "higher" and relates to the pure Python layer.

event.h is "in between" the two, as it needs to refer to both.

Made the functions non-inline, declared them in py/runtime.h, and defined them in py/scheduler.c.

ports/rp2/cyw43_configport.h

dpgeorge · 2023-12-06T04:50:38Z

I've tested this on a Pico and Pico W and they both now pass all the asyncio tests (again). Also networking and BLE works on Pico W.

py/runtime.h

dpgeorge · 2023-12-06T05:11:45Z

py/runtime.h

+// next interrupt or event.
+//
+// Note: on "tickless" ports this can suspend execution for a long time,
+// don't call unless you know an interrupt is coming to continue execution.


Maybe add "and may also return early".

Reworded both these comments to explicitly explain that it's the tick macro that can cause frequent wakeups.

py/runtime.h

ports/windows/mpconfigport.h

ports/rp2/mpconfigport.h

ports/esp8266/esp_mphal.h

dpgeorge · 2023-12-06T05:38:29Z

I think esp8266 needs to be updated to use the new scheme, because it does things differently to the other ports:

#define MICROPY_EVENT_POLL_HOOK {ets_event_poll();}

In particular the i2c.scan() may cause a watchdog timeout if there are no pull-up resistors on the bus.

dpgeorge · 2023-12-06T05:42:09Z

py/runtime.h

+    // For ports still using the old macros
+    MICROPY_EVENT_POLL_HOOK_FAST
+    #else
+    mp_handle_pending(true);


This is not general enough. Some ports like esp8266 must do work in addition to mp_handle_pending(). So we need a way to hook into this function with a macro.

I guess for now we could use MICROPY_EVENT_POLL_HOOK_FAST for this, but eventually we will need a way to hook.

I had anticipated there would be some port-specific tweaks needed here, but I figured it was best to deal with them case-by-case as we move the ports over.

~~esp8266 port should still work the same with this PR, as it still defines MICROPY_EVENT_POLL_HOOK and the implementation of the mp_event_wait_* functions use that macro if it's defined.~~

I think esp8266 can probably be ported by defining #define MICROPY_INTERNAL_WFE ets_event_poll(), as this is the only way to "wait for event" on the esp8266 non-OS SDK. I'd suggest doing this in a follow-up PR, though, the esp8266 changes here were just juggling headers so it would compile.

~~In the specific case of MICROPY_EVENT_POLL_HOOK_FAST and mp_event_handle_no_wait(), esp8266 port doesn't define this hook at the moment so I think it will keep working much the same?~~

Fully agree that some other ports might need an additional macro added here though, when we move these over.

In the specific case of MICROPY_EVENT_POLL_HOOK_FAST and mp_event_handle_no_wait(), esp8266 port doesn't define this hook at the moment so I think it will keep working much the same?

Ah, I'm with you now that the i2c scan specifically failed over to the "slow" hook if the fast hook wasn't defined.

OK, will check all the ports for this behaviour.

Updated esp8266 with a new hook MICROPY_INTERNAL_EVENT_HOOK and verified i2c.scan() completes. Do you know if any other ports relied on that fallback logic in i2c? It looks like probably not.

Compared to before, there is a small behaviour change - esp8266 port will call ets_event_poll() anywhere else that mp_event_handle_nowait() is called. I think this is only modselect.c though, and it seems like a good idea to be processing wifi events in that instance.

Ah, I'm with you now that the i2c scan specifically failed over to the "slow" hook if the fast hook wasn't defined.

Indeed. The existing behaviour is quite subtle, actually too subtle. I'm pretty sure only esp8266 relied on it.

Compared to before, there is a small behaviour change - esp8266 port will call ets_event_poll() anywhere else that mp_event_handle_nowait() is called. I think this is only modselect.c though, and it seems like a good idea to be processing wifi events in that instance.

That sounds like a good improvement.

projectgus · 2023-12-06T23:19:37Z

ports/windows/mpconfigport.h

-    } while (0);
+#define MICROPY_INTERNAL_WFE(TIMEOUT_MS) msec_sleep(MAX(1.0, (double)(TIMEOUT_MS)))
+#else
+#define MICROPY_INTERNAL_WFE(TIMEOUT_MS) /* No-op */


@stinos Do these changes look alright to you? See py/event.h for context. I haven't had a chance to test the Windows port with this PR, yet.

Not completely sure what MICROPY_INTERNAL_WFE is supposed to do, but scheduler.c has MICROPY_INTERNAL_WFE(-1) so that does not seem to work as intended?

MICROPY_INTERNAL_WFE is "wait for event". But it can be conservative and return immediately, but in that case it'll use more power/CPU.

If the argument is -1 it means "wait indefinitely for the next event". Here that would just sleep 1ms.

When MICROPY_ENABLE_SCHEDULER is disabled, I don't think this macro is ever used, so doesn't need to do anything in that case.

projectgus · 2023-12-06T23:41:10Z

There's a problem with the AppVeyor CI:

Yeah, I think the order of inline and MP_ALWAYSINLINE might matter to VC++. Have swapped them over.

py/mphal.h

ports/esp8266/machine_uart.c

ports/esp8266/mpconfigport.h

py/event.h

codecov · 2023-12-06T23:57:49Z

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (66be82d) 98.36% compared to head (2c828a8) 98.36%.

Files	Patch %	Lines
extmod/modssl_mbedtls.c	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #13096      +/-   ##
==========================================
- Coverage   98.36%   98.36%   -0.01%     
==========================================
  Files         159      159              
  Lines       20978    20989      +11     
==========================================
+ Hits        20636    20646      +10     
- Misses        342      343       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

dpgeorge · 2023-12-07T01:13:09Z

The coverage CI check is complaining that mp_event_wait_ms() is never called... do you think you can update the unix port to use the new event scheme as well? That should get coverage passing.

projectgus · 2023-12-07T03:02:09Z

@dpgeorge Coverage is still failing. I think because the only code which calls mp_event_wait_ms() on the unix port is mp_hal_delay_ms. On unix+float, time.sleep() is implemented by calling sleep_select() not mp_hal_delay_ms().

It looks like select() has been used for sleep with floating point arguments for a long time, since f766264 and refined in 0bb57bf.

It seems like it might be reasonable to refactor this to call mp_hal_delay_ms() in all cases, though? It'll use a modicum more CPU time, but not a lot.

(The other option might be to implement MICROPY_INTERNAL_WFE on unix by calling select, as EINTR causes select to return early, but that seems a lot fiddlier.)

dpgeorge · 2023-12-07T04:26:21Z

I think because the only code which calls mp_event_wait_ms() on the unix port is mp_hal_delay_ms. On unix+float, time.sleep() is implemented by calling sleep_select() not mp_hal_delay_ms().

OK, that's a bit tricky.

How about this then:

--- a/ports/unix/coverage.c
+++ b/ports/unix/coverage.c
@@ -578,9 +578,9 @@ STATIC mp_obj_t extra_coverage(void) {
         mp_sched_unlock();
         mp_printf(&mp_plat_print, "unlocked\n");
 
-        // drain pending callbacks
+        // drain pending callbacks (and test mp_event_wait_ms())
         while (mp_sched_num_pending()) {
-            mp_handle_pending(true);
+            mp_event_wait_ms(1);
         }
 
         // setting the keyboard interrupt and raising it during mp_handle_pending

These are intended to replace MICROPY_EVENT_POLL_HOOK and MICROPY_EVENT_POLL_HOOK_FAST, which are insufficient for tickless ports. This implementation is along the lines suggested here: micropython#12925 (comment) Currently any usage of these functions expands to use the existing hook macros, but this can be switched over port by port. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>

See previous commit for details of these functions. As of this commit, these still call the old hook macros on all ports. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>

This commit changes all uses in the rp2 port, and drivers that are optionally supported by that port. The old MICROPY_EVENT_POLL_HOOK and MICROPY_EVENT_POLL_HOOK_FAST macros are no longer used for rp2 builds and are removed (C user code will need to be changed to suit). Also take the opportunity to change some timeouts that used 64-bit arithmetic to 32-bit, to hopefully claw back a little code size. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>

Previously this was not set, so potential for race conditions in interrupt handlers this didn't issue SEV. (Which is currently all of them, as far as I can see.) Eventually we might be able to augment the interrupt handlers that wake the main thread to call SEV, and leave the others as-is to suspend the CPU slightly faster, but this will solve the issue for now. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>

This should be the equivalent of the previous event poll hook macro. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>

This is necessary to avoid watchdog timeout in long i2c.scan(), as previously machine_i2c.c would call MICROPY_EVENT_POLL_HOOK if MICROPY_EVENT_POLL_HOOK_FAST was not available. Compared to previous implementation, this implementation removes the ets_event_poll() function and calls the SDK function ets_loop_iter() from MICROPY_INTERNAL_EVENT_HOOK instead. This allows using the port-agnostic functions in more places. There is a small behaviour change, which is that the event loop gets iterated in a few more places (i.e. anywhere that mp_event_handle_nowait() is called). However, this looks like maybe only modselect.c - and is probably good to process Wi-Fi events in that polling loop. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>

dpgeorge · 2023-12-08T01:58:03Z

Oh, I thought of a third option: Convert mp_wait_for_event_indefinite() and mp_wait_for_event(mp_uint_t timeout_ms) to normal functions instead of inline. Then they can include whatever they need, and there will be a small code size win.

I've changed it so all three of these event functions are normal functions (not inline).

This saves code size, and makes the abstraction a bit stronger. These functions really don't need to be inline. The original macros were because that was a bit of a hack that just allowed a port to insert arbitrary code for the polling. But now that it has been formalised in this PR these event functions are better not inlined. If they are inline then the more they are used the more code size will grow, and we want to be able to use them on all ports, even ones that are space constrained.

I checked all uses of these event functions and none of them are critical that they warrant (or need, eg to be in IRAM) an inline function. Better to spend the bytes elsewhere to improve performance. It also means we don't need a new py/event.h header file.

Necessary to get coverage of the new event functions. Deletes the case that called usleep(delay) for mp_hal_delay_ms(), it seems like this wouldn't have ever happened anyhow (MICROPY_EVENT_POOL_HOOK is always defined for the unix port). This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>

projectgus added py-core Relates to py/ directory in source port-rp2 labels Nov 30, 2023

projectgus force-pushed the bugfix/tickless_event_hook branch 2 times, most recently from 1ca5e66 to b750adc Compare November 30, 2023 06:11

projectgus force-pushed the bugfix/tickless_event_hook branch 4 times, most recently from f1aab4a to 7793250 Compare November 30, 2023 06:50

dpgeorge added this to the release-1.22.0 milestone Dec 5, 2023

projectgus force-pushed the bugfix/tickless_event_hook branch 3 times, most recently from 2674222 to 881214b Compare December 5, 2023 06:30

dpgeorge reviewed Dec 6, 2023

View reviewed changes

ports/rp2/cyw43_configport.h Show resolved Hide resolved

dpgeorge reviewed Dec 6, 2023

View reviewed changes

projectgus force-pushed the bugfix/tickless_event_hook branch 4 times, most recently from 0e72fd8 to da98c6f Compare December 6, 2023 23:07

projectgus added port-esp8266 port-windows labels Dec 6, 2023

projectgus changed the title ~~core,rp2: Add new cross-port functions for event waiting and handling.~~ core,rp2,esp8266,windows: Add new cross-port functions for event waiting and handling. Dec 6, 2023

projectgus force-pushed the bugfix/tickless_event_hook branch from da98c6f to deb930b Compare December 6, 2023 23:15

projectgus commented Dec 6, 2023

View reviewed changes

projectgus force-pushed the bugfix/tickless_event_hook branch from deb930b to c05313b Compare December 6, 2023 23:40

dpgeorge reviewed Dec 6, 2023

View reviewed changes

py/mphal.h Outdated Show resolved Hide resolved

dpgeorge reviewed Dec 6, 2023

View reviewed changes

ports/esp8266/machine_uart.c Outdated Show resolved Hide resolved

dpgeorge reviewed Dec 6, 2023

View reviewed changes

ports/esp8266/mpconfigport.h Outdated Show resolved Hide resolved

dpgeorge reviewed Dec 6, 2023

View reviewed changes

py/event.h Outdated Show resolved Hide resolved

projectgus force-pushed the bugfix/tickless_event_hook branch from c05313b to adf61cd Compare December 7, 2023 00:11

projectgus force-pushed the bugfix/tickless_event_hook branch from 49b4c9f to ee53c81 Compare December 7, 2023 05:15

projectgus added the port-unix label Dec 7, 2023

projectgus changed the title ~~core,rp2,esp8266,windows: Add new cross-port functions for event waiting and handling.~~ core,rp2,esp8266,windows, unix: Add new cross-port functions for event waiting and handling. Dec 7, 2023

projectgus force-pushed the bugfix/tickless_event_hook branch 2 times, most recently from fc19e73 to a023d2d Compare December 7, 2023 09:20

dpgeorge marked this pull request as ready for review December 8, 2023 00:39

dpgeorge force-pushed the bugfix/tickless_event_hook branch from a023d2d to 8d29027 Compare December 8, 2023 01:14

projectgus added 6 commits December 8, 2023 12:47

extmod: Switch to use new event functions.

df3948d

See previous commit for details of these functions. As of this commit, these still call the old hook macros on all ports. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>

windows: Implement MICROPY_INTERNAL_WFE() macro.

c393f5c

This should be the equivalent of the previous event poll hook macro. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>

dpgeorge force-pushed the bugfix/tickless_event_hook branch from 8d29027 to b30e80d Compare December 8, 2023 01:53

dpgeorge force-pushed the bugfix/tickless_event_hook branch from b30e80d to 2c828a8 Compare December 8, 2023 02:17

dpgeorge merged commit 2c828a8 into micropython:master Dec 8, 2023

projectgus deleted the bugfix/tickless_event_hook branch January 1, 2024 22:36

Uh oh!

core,rp2,esp8266,windows, unix: Add new cross-port functions for event waiting and handling. #13096

core,rp2,esp8266,windows, unix: Add new cross-port functions for event waiting and handling. #13096

Uh oh!

Conversation

projectgus commented Nov 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Nov 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

projectgus commented Nov 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

projectgus commented Nov 30, 2023

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

projectgus Dec 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dpgeorge commented Dec 6, 2023

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dpgeorge commented Dec 6, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

projectgus Dec 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

projectgus commented Dec 6, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Dec 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

dpgeorge commented Dec 7, 2023

Uh oh!

projectgus commented Dec 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dpgeorge commented Dec 7, 2023

projectgus commented Nov 30, 2023 •

edited

Loading

github-actions bot commented Nov 30, 2023 •

edited

Loading

projectgus commented Nov 30, 2023 •

edited

Loading

projectgus Dec 6, 2023 •

edited

Loading

projectgus Dec 6, 2023 •

edited

Loading

codecov bot commented Dec 6, 2023 •

edited

Loading

projectgus commented Dec 7, 2023 •

edited

Loading