@Jeroen88 Funny that we were both working on the same idea. I've been playing with this locally as well. For an even more dramatic effect checkout the rework-FreeRTOS branch 😄

I'll get into this one a little later, after I get done with some other things and #35

@h2zero Nice! The std::string key duplication in the std::map annoyed me lot!!

I love your library, but I need to get it smaller, because I want to combine BLE devices with other (Serial) devices.

I will checkout the rework-FreeRTOS branch soon!

My thoughts are we could maintain backward compatible API while using a vector internally and dynamically create the map(s) if the user wants them, just created in the function call from the vector instead.

My thoughts are we could maintain backward compatible API while using a vector internally and dynamically create the map(s) if the user wants them, just created in the function call from the vector instead.

That is possible for sure, but is it worth the effort? A novice user will not need it, and a more experienced user can live with it, I think.

For an even more dramatic effect checkout the rework-FreeRTOS branch

I took a quick look at the differences here at github. I am not very familiar with semaphores (but I am very charmed about the use in asynchronous calls that I saw in this library), so I do not really grasp the benefits. What are he benefits? Are these changes already in master? If not: any plans to integrate these?

The changes there reduce the memory usage by ~200 or so bytes per service/characteristic/descriptor, yes this will be integrated once I'm satisfied with it.

So that is ~200 bytes on top of the map to vector change? Because I just checked and in the oter branch still a map is used. That is another huge improvement!

Yes, the old FreeRTOS::Semaphore class was storing 2 std:string and some other stuff that's not needed. More work to do there still, but overall there will be much memory saved.

Updated NimBLEScan to use a std::vector instead of a std::map too.

Good stuff, I'll add in the peripheral related changes. I think I will wait on breaking compatibility until there is a release version that maintains it for those who just want to keep changes minimal. After that, this change as well as many others will be done. Sound good?

Thnx @h2zero!

I would like the changes to be released soon, if possible. Breaking changes are in a release usually, but these are the reasons why I think it can be done now:

• I think the number of users for NimBLE is still limited. Better make changes now, for a few people.
• Most users will need the library call to read services and characteristics already known bij UUID, so most users will not need the maps. The only use case I can think of is displaying the structure of an unknown device. The toString() functions can be used for that, or the vector!
• The maps are an enormous waste of resources: each UUID is copied (not referenced!) into the map as keys. For my simple device with 72 UUIDs this wastes 72x the size of the std::string UUID! The vector on the other side doesn't use keys at all, just plain indices.

On top of that, I think the functions to get the maps shouldn't be there at all. The maps (and for that purpose also the vectors) are there to set up an internal structure, and are nicely encapsulated by the get-by-UUID functions. Instead an iterator could be defined, returning const pointers of course!

Last but not least, I need the changes and rather want them in the official library than in a fork or a branch.

Another thing, if anybody really wants the map, it can be easily created by the user from the vector.

@Jeroen88 Sorry for the delay in response, I wanted to take some time to consider your argument. Reason being, I know of a couple projects that use this code, but on the other hand changing the app code won't be that big of a chore for those.

It was always my vision to remain compatible as much as possible for a proper comparison of the benefits as a nearly plug and play library (this all started as an experiment after all). However there are quite a few shortcomings in the original code and I need to change my thoughts for the betterment of all users.

On that note I will start on the incorporation of this PR and add the server related changes to it.

Thank you again for all your efforts, they are very much appreciated 😄.

@Jeroen88 I was just testing this with a peripheral that has 10 uuids, saved 1080 bytes of memory. Then I merged the freeRTOS-rework branch and saved an additional 1630 bytes. I can see if you're connecting to a device with 72 uuid's that would be an incredible difference. I have to wonder what they were thinking when making such a device 🤯 .

Thnx @h2zero for making me happy!

In my opinion the original library did a very good job in designing a framework with the device, client, server, (remote) service, (remote) characteristic and (remote) descriptor. I think the designer may have had neat code in mind at first, like service['... uuid ... '], but it didn't work out that way. Now the map is a real waste of memory! Still, I like the original design. Your work made the library fit in my use case, thnx for that again!

I'll add a small commit to this PR, possibly today, I think it would be nice to find an advertised device by it's address.

@Jeroen88 If you're just looking to connect to a device directly by address you can do this already with connect(), it does a scan with address filtering in the stack.

@h2zero Thanks, but no, I am looking for the service data of a scanned device

See PR #45, I want to do a scan and next a pBLEScan->getResults()->getDevice(NimBLEAddress("58:2d:34:39:22:58")) to access the service data. Took me quite some headaches to find out why it wasn't working, but the error descibed in PR #45 was the cause.

I'm confused, getDevice() does not take a NimBLEAddress argument, the description says:

/**
 * @brief Return the specified device at the given index.
 * The index should be between 0 and getCount()-1.
 * @param [in] i The index of the device.
 * @return The device at the specified index.
 */
NimBLEAdvertisedDevice NimBLEScanResults::getDevice(uint32_t i) {

I'm surprised that it would even compile?

No I was not clear, I am working on a getDevice() taking an address as parameter, but ran into the bug described in PR #45, so I had to fix that first.

@Jeroen88 that makes more sense.

I just pushed a new branch single-semaphore-test, if you have a minute to test it with your 72 uuid peripheral I'd be interested in seeing the increase in free heap. With 10 uuid's I saved 5k, I think with that one it will save 30k lol.

@h2zero I did the test, your guessing was close :D

Sketch used for testing:

#include <Arduino.h>
#include <string>

#include <NimBLEDevice.h>


void setup() {
  // put your setup code here, to run once:
  Serial.begin(230400);
  Serial.println("\n\nStarted");
  Serial.printf("Free heap before connection is %u\n", ESP.getFreeHeap());

  BLEDevice::init("ESP32");

  BLEClient *pClient = BLEDevice::createClient();

  BLEAddress address = BLEAddress("a4:c1:38:5d:ef:16");

  Serial.println("Connecting...");
  pClient->connect(address);
  if(pClient->isConnected()) {
    Serial.println("Connected");

    Serial.printf("Free heap after connection is %u\n", ESP.getFreeHeap());

    pClient->disconnect();
  }
}

void loop() {
  // put your main code here, to run repeatedly:

}

Results on current master

Sketch uses 524684 bytes (40%) of program storage space. Maximum is 1310720 bytes.
Global variables use 25336 bytes (7%) of dynamic memory, leaving 302344 bytes for local variables. Maximum is 327680 bytes.

Started
Free heap before connection is 291832
Connecting...
Connected
Free heap after connection is 235628

Results on branch single-semaphore-test

Sketch uses 521940 bytes (39%) of program storage space. Maximum is 1310720 bytes.
Global variables use 25336 bytes (7%) of dynamic memory, leaving 302344 bytes for local variables. Maximum is 327680 bytes.

Started
Free heap before connection is 291832
Connecting...
Connected
Free heap after connection is 262596

The device is a MJ-HT-V1 temperature / humidity sensor. This is it's string representation:

peer address: a4:c1:38:5d:ef:16
Services:
Service: uuid: 00000100-0065-6c62-2e74-6f696d2e696d, start_handle: 96 0x0060, end_handle: 104 0x0068
Characteristic: uuid: 00000101-0065-6c62-2e74-6f696d2e696d, handle: 98 0x0062, props:  0x04
Descriptor: uuid: 0x2901, handle: 99
Descriptor: uuid: 0x2902, handle: 100
Characteristic: uuid: 00000102-0065-6c62-2e74-6f696d2e696d, handle: 102 0x0066, props:  0x10
Descriptor: uuid: 0x2901, handle: 103
Descriptor: uuid: 0x2902, handle: 104
Service: uuid: 00010203-0405-0607-0809-0a0b0c0d1912, start_handle: 29 0x001d, end_handle: 32 0x0020
Characteristic: uuid: 00010203-0405-0607-0809-0a0b0c0d2b12, handle: 31 0x001f, props:  0x06
Descriptor: uuid: 0x2901, handle: 32
Service: uuid: 0x1800, start_handle: 1 0x0001, end_handle: 7 0x0007
Characteristic: uuid: 0x2a00, handle: 3 0x0003, props:  0x12
Characteristic: uuid: 0x2a01, handle: 5 0x0005, props:  0x02
Characteristic: uuid: 0x2a04, handle: 7 0x0007, props:  0x02
Service: uuid: 0x1801, start_handle: 8 0x0008, end_handle: 11 0x000b
Characteristic: uuid: 0x2a05, handle: 10 0x000a, props:  0x20
Descriptor: uuid: 0x2902, handle: 11
Service: uuid: 0x180a, start_handle: 12 0x000c, end_handle: 24 0x0018
Characteristic: uuid: 0x2a24, handle: 14 0x000e, props:  0x02
Characteristic: uuid: 0x2a25, handle: 16 0x0010, props:  0x02
Characteristic: uuid: 0x2a26, handle: 18 0x0012, props:  0x02
Characteristic: uuid: 0x2a27, handle: 20 0x0014, props:  0x02
Characteristic: uuid: 0x2a28, handle: 22 0x0016, props:  0x02
Characteristic: uuid: 0x2a29, handle: 24 0x0018, props:  0x02
Service: uuid: 0x180f, start_handle: 25 0x0019, end_handle: 28 0x001c
Characteristic: uuid: 0x2a19, handle: 27 0x001b, props:  0x12
Descriptor: uuid: 0x2902, handle: 28
Service: uuid: 0xfe95, start_handle: 76 0x004c, end_handle: 95 0x005f
Characteristic: uuid: 0x0004, handle: 78 0x004e, props:  0x02
Descriptor: uuid: 0x2901, handle: 79
Characteristic: uuid: 0x0010, handle: 81 0x0051, props:  0x14
Descriptor: uuid: 0x2901, handle: 82
Descriptor: uuid: 0x2902, handle: 83
Characteristic: uuid: 0x0017, handle: 85 0x0055, props:  0x18
Descriptor: uuid: 0x2901, handle: 86
Descriptor: uuid: 0x2902, handle: 87
Characteristic: uuid: 0x0018, handle: 89 0x0059, props:  0x14
Descriptor: uuid: 0x2901, handle: 90
Descriptor: uuid: 0x2902, handle: 91
Characteristic: uuid: 0x0019, handle: 93 0x005d, props:  0x14
Descriptor: uuid: 0x2901, handle: 94
Descriptor: uuid: 0x2902, handle: 95
Service: uuid: ebe0ccb0-7a0a-4b0c-8a1a-6ff2997da3a6, start_handle: 33 0x0021, end_handle: 75 0x004b
Characteristic: uuid: ebe0ccb7-7a0a-4b0c-8a1a-6ff2997da3a6, handle: 35 0x0023, props:  0x0a
Descriptor: uuid: 0x2901, handle: 36
Characteristic: uuid: ebe0ccb9-7a0a-4b0c-8a1a-6ff2997da3a6, handle: 38 0x0026, props:  0x02
Descriptor: uuid: 0x2901, handle: 39
Characteristic: uuid: ebe0ccba-7a0a-4b0c-8a1a-6ff2997da3a6, handle: 41 0x0029, props:  0x0a
Descriptor: uuid: 0x2901, handle: 42
Characteristic: uuid: ebe0ccbb-7a0a-4b0c-8a1a-6ff2997da3a6, handle: 44 0x002c, props:  0x02
Descriptor: uuid: 0x2901, handle: 45
Characteristic: uuid: ebe0ccbc-7a0a-4b0c-8a1a-6ff2997da3a6, handle: 47 0x002f, props:  0x10
Descriptor: uuid: 0x2901, handle: 48
Descriptor: uuid: 0x2902, handle: 49
Characteristic: uuid: ebe0ccbe-7a0a-4b0c-8a1a-6ff2997da3a6, handle: 51 0x0033, props:  0x0a
Descriptor: uuid: 0x2901, handle: 52
Characteristic: uuid: ebe0ccc1-7a0a-4b0c-8a1a-6ff2997da3a6, handle: 54 0x0036, props:  0x12
Descriptor: uuid: 0x2901, handle: 55
Descriptor: uuid: 0x2902, handle: 56
Characteristic: uuid: ebe0ccc4-7a0a-4b0c-8a1a-6ff2997da3a6, handle: 58 0x003a, props:  0x02
Descriptor: uuid: 0x2901, handle: 59
Characteristic: uuid: ebe0ccc8-7a0a-4b0c-8a1a-6ff2997da3a6, handle: 61 0x003d, props:  0x08
Descriptor: uuid: 0x2901, handle: 62
Characteristic: uuid: ebe0ccd1-7a0a-4b0c-8a1a-6ff2997da3a6, handle: 64 0x0040, props:  0x08
Descriptor: uuid: 0x2901, handle: 65
Characteristic: uuid: ebe0ccd7-7a0a-4b0c-8a1a-6ff2997da3a6, handle: 67 0x0043, props:  0x0a
Descriptor: uuid: 0x2901, handle: 68
Characteristic: uuid: ebe0ccd8-7a0a-4b0c-8a1a-6ff2997da3a6, handle: 70 0x0046, props:  0x08
Descriptor: uuid: 0x2901, handle: 71
Characteristic: uuid: ebe0ccd9-7a0a-4b0c-8a1a-6ff2997da3a6, handle: 73 0x0049, props:  0x18
Descriptor: uuid: 0x2901, handle: 74
Descriptor: uuid: 0x2902, handle: 75

Conclusion:
This branch saves 2,744 bytes of program memory and 26,968 bytes of heap after connection compared to the current master. Although the MJ-HT-V1 has a lot of UUIDs, luckily not every UUID is 128 bits. A lot of them are 16 bits (descriptors 0x2901 and 0x2902) otherwise the heap savings would be even bigger.

And the savings from map to vector still to be added on top of this!

Nice! Don't get too comfortable with that branch, it was for experimental purposes and won't work in a few scenarios (semaphore lockups). Good to see the heap savings, I'm going to implement the semaphores dynamically instead, might add a bit of overhead and program size but totally worth it to save that heap space.

Very nice indeed!

Closing this as #46 was merged. Thanks again!