-
-
Notifications
You must be signed in to change notification settings - Fork 8.3k
py/ringbuf.h: Avoid wasted byte in ringbuf and don't inline get/put. #4920
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
When the size of the ringbuf is divisible into 2^16, we can make the code smaller and avoid a wasted byte. Inlining is only a saving if each of get/put are called once. This reduces code size on ESP8266 (-124) and NRF (-84) ports, and marginal increase on ESP32 (+24).
Excuse me, but do you have an idea what you're talking about? If so, you could probably start with explaining what does "divisible into 2^16" means. |
I have updated the description to be clearer. And just a reminder that the attitude is unwelcome. |
@jimmo: The attitude of copyright violation (per your own words, the code is due to @aykevl, while instead your name is sprinkled around), and impersonation (you put my name for the code I haven't written) is just unacceptable. And if you started to talk about that, that I have a suspicion that you don't have a clue why the existing ringbuf code is written like it is, and just eager to change things around just for the purpose of changing something around. That's, well, unwelcome. Please do more homework first. Thanks. |
To make a couple of things clear:
Probably not! Feel free to educate me and I would be more than willing to replace this PR with one that adds a comment so that future travellers can avoid trying to dismantle Chesterton's Fence.
I believe I've already made the case for this fairly clear (a new use for ringbuf in multiple ports, and the demonstrated code size reduction on two of the most ROM-constrained ports). |
Thanks for clarification, I stand corrected.
You added a new file, with completely different code content then was in the original implementation, so please remove my name from it. (Please leave my name on the header file, as the API stays the same).
Ok, let's try to do it together. You claim that the purpose of this patch is to save a single wasted byte in ringbuf data buffer. To do that, you replace implementation with one which rounds buf size up to next power of 2. Now just consider what happens if user requests buffer of size, say, 16385. You will round that up to 32768. So, for the purpose of saving one byte, you put up with wasting 16383. That doesn't make much sense. And excuse my initial brevity, but logic and math here is very simple, I'd say, midschool-level. It might be too time-consuming to point/explain each and every such case. It's very nice that people try to improve MicroPython codebase, but it may be a good idea to start with the assumption that there were good reasons why something was implemented like it is. Then, if desired, challenge it (using https://en.wikipedia.org/wiki/Rubber_duck_debugging or similar technique), then make up an alternative, then challenge it even more thoroughly, because any simple alternatives were likely already thought out or tried. |
It's true that's what's in the title, but as explained, there are other reasons.
Yes I agree this is unfortunate, but you'll see I was well aware of this, hence the comments suggesting that the buffer needs to be sized correctly. And all of the static buffers are fine as they are.
Are there other cases, or is the issue just the requirement that the size is a power-of-two? It's worth clarifying that this particular concern currently only affects the ESP8266 UART rxbuf kwarg, which might be suitably addressed by documentation? So is there a specific concern that the buffer must be exactly sized to the user's value and therefore this would be a breaking change? I also considered rounding down, but I think that makes less sense. |
MicroPython is a general-purpose high-level language. A user should be able to specify arbitrary buffer size (useful capacity), and get buffer of exactly that size. They should be ready however that there may be small, whenever possible O(1), memory overhead above that size (but definitely not O(n) overhead). The current ringbuf.h implementation is a generic (universal) ring buffer implementation which complies with these requirements. There's not much to add, take, or change there, I'm afraid. |
I did some work with circular buffers a few months back for #4617 which went through a few iterations for efficiency. At the time I had some reason/excuse not to merge the code there into More importantly, I found this to be an excellent reference for different circular buffer algorithms and their respective pros/cons: https://www.snellman.net/blog/archive/2016-12-13-ring-buffers/ |
For discussion... I'd like to consolidate
py/ringbuf.h
with the ringbuf implementation in modbluetooth.c in #4893 written by @aykevl, so this PR merges the two implementations into py/ringbuf.h.The benefit of @aykevl's implementation is that when the size of the ringbuf is divisible into 2^16 (i.e.
2**16 % size == 0
), we can make the code smaller and avoid a wasted byte in the buffer.I'm unsure whether the requirement that the buffer is divisible into 2^16 is a problem. It means that if the user sets the
rxbuf
kwarg on ESP8266, they may get a slightly larger buffer than expected (the code rounds up to the nearest power of two).I've also changed ringbuf to no longer inline get/put -- this is only a saving if each of get/put are called once (on ESP8266/ESP32 this is not the case). On NRF, LTO does this for us anyway. (And NRF will benefit further when this gets used in modbluetooth).
This reduces code size on ESP8266 (-124) and NRF (-84) ports, and marginal increase on ESP32 (+24).