REVIEW ONLY: first steps making GC reentrant #913
Conversation
No actually. The GC can run just fine in an irq (so long as it's not already running, but that's checked for). The only issue is that it might take some time to complete, so you should really know what you are doing if you let the GC run on an irq. You would need to explicitly disable auto-gc (using gc.disable) if you wanted to prevent it running in an irq. We would need to re-organise the IRQ priorities to help with latency. Eg, if the GC is running in an IRQ, then you still want other IRQs (like USB, UART buffering) to be able to preempt it. Also, would be nice for the user if they can specify priority for callbacks. Something like switch.callback(f, priority=3). The priority scheme would be something like (note that a large IRQ level number is lower priority):
This gives the user 9 levels (top level and 8 IRQs) to play with. There are no restrictions on the code that you can run in these levels (eg you can run gc.collect, or have it run automatically). We could additionally define levels 8-11 to disable automatic GC when they are entered. |
Yes. It is also true for non Python objects. Any "chunk" of memory allocated on the GC heap has HEAD plus 0-n TAILs.
Yes. The AT (allocation table) tells 4 states for each block: FREE, HEAD, TAIL, MARK. MARK means marked head for tracing. |
|
I implemented @dhylands' idea about setting |
|
This PR is stale. Some new ideas have surfaced (in my mind at least!) on how to tackle the issue of memory allocation in an interrupt. I think first we should concentrate on improving uPy core so that it doesn't need to allocate heap memory as far as is possible. |
Itsybitsy name fixes
This is totally not ready to merge, but I did a fair bit of work on it and want to see what others think before continuing (or abandoning).
The idea is that you want to be able to allocate memory during an interrupt. If we allow this, then you should be able to rely on allocating memory during an interrupt, so long as there is memory available. You can arrange for memory to be available by calling gc.collect() explicitly and often enough (outside the interrupt).
Allowing gc_alloc/gc_realloc/gc_free to work in an interrupt means that these 3 functions, and the garbage collector routine itself, must be reentrant (can be suspended, executed again at a higher priority interrupt, then resumed).
This means making all global state operations atomic, and always leaving the heap in a consistent state.
It is difficult because you can't simply disable IRQs for the whole gc_alloc: it could take some time to find a free piece of the heap, and in this time you might get another IRQ (eg UART character comes in). So you can only disable IRQs for a very short time.
I've tested this patch by creating 3 timers running at 1Hz, 10Hz and 1000Hz, all allocating large chunks of memory. Then I do gc.collect() in the foreground in an infinite loop. It seems to work, but needs a lot more testing.
This really opens a whole can of worms, since other issues start to become apparent. Eg, we need to atomically store to the global dictionary (mp_store_global). And, ctrl-C handler probably needs to be completely redesigned and rewritten so that you can't ctrl-C while in the middle of a gc collection/alloc/realloc/free (in fact, you shouldn't be able to ctrl-C in any HAL function either since they modify global state of the peripherals).