Thanks to visit codestin.com
Credit goes to github.com

Skip to content

py/ringbuf.h: Avoid wasted byte in ringbuf and don't inline get/put. #4920

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

jimmo
Copy link
Member

@jimmo jimmo commented Jul 11, 2019

For discussion... I'd like to consolidate py/ringbuf.h with the ringbuf implementation in modbluetooth.c in #4893 written by @aykevl, so this PR merges the two implementations into py/ringbuf.h.
The benefit of @aykevl's implementation is that when the size of the ringbuf is divisible into 2^16 (i.e. 2**16 % size == 0), we can make the code smaller and avoid a wasted byte in the buffer.

I'm unsure whether the requirement that the buffer is divisible into 2^16 is a problem. It means that if the user sets the rxbuf kwarg on ESP8266, they may get a slightly larger buffer than expected (the code rounds up to the nearest power of two).

I've also changed ringbuf to no longer inline get/put -- this is only a saving if each of get/put are called once (on ESP8266/ESP32 this is not the case). On NRF, LTO does this for us anyway. (And NRF will benefit further when this gets used in modbluetooth).

This reduces code size on ESP8266 (-124) and NRF (-84) ports, and marginal increase on ESP32 (+24).

When the size of the ringbuf is divisible into 2^16, we can make the code smaller and avoid a wasted byte.

Inlining is only a saving if each of get/put are called once.

This reduces code size on ESP8266 (-124) and NRF (-84) ports, and marginal increase on ESP32 (+24).
@pfalcon
Copy link
Contributor

pfalcon commented Jul 11, 2019

the size of the ringbuf is divisible into 2^16

Excuse me, but do you have an idea what you're talking about? If so, you could probably start with explaining what does "divisible into 2^16" means.

@jimmo
Copy link
Member Author

jimmo commented Jul 11, 2019

Excuse me, but do you have an idea what you're talking about? If so, you could probably start with explaining what does "divisible into 2^16" means.

I have updated the description to be clearer. And just a reminder that the attitude is unwelcome.

@pfalcon
Copy link
Contributor

pfalcon commented Jul 11, 2019

@jimmo: The attitude of copyright violation (per your own words, the code is due to @aykevl, while instead your name is sprinkled around), and impersonation (you put my name for the code I haven't written) is just unacceptable.

And if you started to talk about that, that I have a suspicion that you don't have a clue why the existing ringbuf code is written like it is, and just eager to change things around just for the purpose of changing something around. That's, well, unwelcome. Please do more homework first. Thanks.

@jimmo
Copy link
Member Author

jimmo commented Jul 11, 2019

The attitude of copyright violation (per your own words, the code is due to @aykevl, while instead your name is sprinkled around), and impersonation (you put my name for the code I haven't written) is just unacceptable.

To make a couple of things clear:

  • @aykevl didn't write this code, I did. @aykevl 's PR just happens to include a ringbuffer that uses this style of indexing, but implemented differently. I mean, as differently as 10 lines of code can be, but sure.
  • I generally wouldn't remove people's name's from copyright when editing/moving things around, even if the edits are substantial. If you would like to remove your name, then that's fine.

And if you started to talk about that, that I have a suspicion that you don't have a clue why the existing ringbuf code is written like it is

Probably not! Feel free to educate me and I would be more than willing to replace this PR with one that adds a comment so that future travellers can avoid trying to dismantle Chesterton's Fence.

and just eager to change things around just for the purpose of changing something around.

I believe I've already made the case for this fairly clear (a new use for ringbuf in multiple ports, and the demonstrated code size reduction on two of the most ROM-constrained ports).

@pfalcon
Copy link
Contributor

pfalcon commented Jul 11, 2019

@aykevl didn't write this code, I did.

Thanks for clarification, I stand corrected.

I generally wouldn't remove people's name's from copyright when editing/moving things around

You added a new file, with completely different code content then was in the original implementation, so please remove my name from it. (Please leave my name on the header file, as the API stays the same).

Probably not!

Ok, let's try to do it together. You claim that the purpose of this patch is to save a single wasted byte in ringbuf data buffer. To do that, you replace implementation with one which rounds buf size up to next power of 2. Now just consider what happens if user requests buffer of size, say, 16385. You will round that up to 32768. So, for the purpose of saving one byte, you put up with wasting 16383. That doesn't make much sense.

And excuse my initial brevity, but logic and math here is very simple, I'd say, midschool-level. It might be too time-consuming to point/explain each and every such case.

It's very nice that people try to improve MicroPython codebase, but it may be a good idea to start with the assumption that there were good reasons why something was implemented like it is. Then, if desired, challenge it (using https://en.wikipedia.org/wiki/Rubber_duck_debugging or similar technique), then make up an alternative, then challenge it even more thoroughly, because any simple alternatives were likely already thought out or tried.

@jimmo
Copy link
Member Author

jimmo commented Jul 11, 2019

Ok, let's try to do it together. You claim that the purpose of this patch is to save a single wasted byte in ringbuf data buffer.

It's true that's what's in the title, but as explained, there are other reasons.

To do that, you replace implementation with one which rounds buf size up to next power of 2. Now just consider what happens if user requests buffer of size, say, 16385. You will round that up to 32768. So, for the purpose of saving one byte, you put up with wasting 16383. That doesn't make much sense.

Yes I agree this is unfortunate, but you'll see I was well aware of this, hence the comments suggesting that the buffer needs to be sized correctly. And all of the static buffers are fine as they are.

And excuse my initial brevity, but logic and math here is very simple, I'd say, midschool-level. It might be too time-consuming to point/explain each and every such case.

Are there other cases, or is the issue just the requirement that the size is a power-of-two?

It's worth clarifying that this particular concern currently only affects the ESP8266 UART rxbuf kwarg, which might be suitably addressed by documentation? So is there a specific concern that the buffer must be exactly sized to the user's value and therefore this would be a breaking change?

I also considered rounding down, but I think that makes less sense.

@pfalcon
Copy link
Contributor

pfalcon commented Jul 11, 2019

MicroPython is a general-purpose high-level language. A user should be able to specify arbitrary buffer size (useful capacity), and get buffer of exactly that size. They should be ready however that there may be small, whenever possible O(1), memory overhead above that size (but definitely not O(n) overhead). The current ringbuf.h implementation is a generic (universal) ring buffer implementation which complies with these requirements. There's not much to add, take, or change there, I'm afraid.

@andrewleech
Copy link
Contributor

I did some work with circular buffers a few months back for #4617 which went through a few iterations for efficiency.

At the time I had some reason/excuse not to merge the code there into py/ringbuf.h, I think the use case and/or restrictions didn't quite fit initially, not sure if that's still relevant for the final implementation.
Either way, the discussion there may be of interest.

More importantly, I found this to be an excellent reference for different circular buffer algorithms and their respective pros/cons: https://www.snellman.net/blog/archive/2016-12-13-ring-buffers/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants