Create semaphores dynamically #53
Conversation
|
@h2zero as I said before, I am not very experienced with semaphores. Currently my own test case is using PR #49 but PR #40 not yet, 3 peripherals, endlessly scanned, connect (once) to one and maintain that connection. Next, every 12s: reading service data from the first, reading manufacturer data from the second, reading one characteristic from the third (connected) device. Will it give you insights if I start this scenario as a test for a longer period of time, to see if it is stable? |
|
@h2zero I think I have to compliment you on this achievement, but frankly I have got no idea of what's going on, lol! |
|
@h2zero I am thinking a bit about this PR. I do not know for sure, but if we merge PR #40, then most of the class instances for the characteristics and most of the descriptors will never be created. In this scenario the savings of this PR are far less than you report now. If we find a solution for creating services as well dynamically then also the service class instances are sparsely created. IMHO dynamic population of the class instances is very important! If we go on with the dynamic object creation, is the added complexity (and the, I think, non-standard use of semaphores which creates an extra threshold for other programmers that want to add to this library) of the dynamic semaphores worth the effort? |
|
Ideally, we could achieve both. Creating semaphores dynamically is quite common actually, they just sit there taking up resources most of the time, especially in this library. The maintenance isn't that bad, just like any other resource you allocate, you need to release it properly, the macros help with this. The complexity here comes from the use of a single static pointer to the allocated semaphore, which is a binary type wrapped in a class to make it somewhat custom and keeps track of the number of tasks trying to take it. This has been hidden in the class/macros and could probably be a bit simpler but I can see it being confusing for those who are unfamiliar with the subject. Perhaps I should explain the operation of them somewhere. #40 Is the next step, it's also a bit of a complex problem due to how BLE spec uses "handles" and how NimBLE accesses the peripheral database. I need to do some experiments with it and see what's reasonably possible. The question I need to answer there first is "does NimBLE request the whole database when calling get_service/characteristic_uuid?". If that is what it does we will have to implement some other way to get "only" the attributes we want. |
Do you mean the stack or the library? For the stack I don't know, but I can't imagine it retrieving more than needed. For the library I am sure: all services are retrieved, and next every characteristic for every service and every descriptor for each characteristic. Actually it is done with the function call here defined here. PR #40 retrieves all services on connection, but the characteristics dynamically only on first request. It does not retrieve the descriptors (yet). |
|
I was referring to the stack, I'm not sure how it is done there. BLE was meant that you would connect and bond with the device so you would only ever get the database once and store a copy for future reconnection. So that is (somewhat) what the library does. Because of the way handles work you need to ask the device for the whole thing to find what you're looking for, it could be stopped once you found it but if the attribute you want is at the end of the database then you've just discovered the whole thing anyway. Also if you didn't store all those results and wanted a second attribute handle you would have to start the whole process over, wasting time and battery. |
|
Moving the discussion to #40 ... see there |
|
Thinking about this PR, even though it works well to my own surprise I believe there is a better option that can eliminate the semaphores completely. It’ll require a bit of work and testing but the results could be worth it. |
|
Closing this as there is a better solution almost complete. |
This introduces dynamic creation of the semaphores used internally by the library, most noticeably in the client code (server/scan to be done). The reason for this change is each semaphore used in each attribute that uses them consumes ~180 bytes of memory.
On the client especially, this effect is dramatic when considering there are 2 semaphores per remote service, 3 per remote characteristic and 2 per remote descriptor. If the remote device has 1 service with 1 characteristic and 1 descriptor, when we connect and obtain the database it will use ~1260 bytes of heap memory before beginning any other operations.
This PR will remove that and create the semaphores on an as required basis using new macros to help implement this procedure. Using this method has reduced heap use in my testing by ~700% per connection, depending on the number of attributes on the remote device.
This is not without a cost however, it requires careful and strict adherence to properly creating and making sure the semaphores are deleted when no longer needed. In other words increased maintenance and increased risk of memory leaks and bugs.
So far I have been able to run this iteration for days with 3 threads accessing the semaphore protected resources with potentially conflicting timings in both debug and release configuration without issue.
I believe this to be overwhelmingly worth the cost, @Jeroen88 has seen a ~27k heap use reduction connecting to a peripheral with 72 attributes, that is incredible and worth the effort IMHO.
If you can find a way to break this please comment, so far so good for me!