Couchbase sdk brpc #3

giriraj-singh-couchbase · 2025-08-13T18:52:47Z

What problem does this PR solve?

Currently, brpc supports various protocols like memcached, Redis, and HTTP, but lacks native support for Couchbase, a popular NoSQL document database. Users who want to integrate Couchbase with brpc applications can now directly use the builtin wrapper functions, this would help in writing less code and build the applications even faster.

What is changed and the side effects?

Changed:

Coucbase Wrapper files in src/brpc/couchbase.h && src/brpc/couchbase.cpp
Added Examples to use the couchbase wrapper functions in example/couchbase_example/client.cpp, multi_threaded_client.cpp
Changes in Makefile of the bRPC so that it can search for necessary libraries to help compile the couchbase SDK.
Added a comprehensive couchbase_client.md file to get the better understanding of using the couchbase c++ SDK wrapper functions and the features provided by the couchbase c++ sdk wrapper.

Side effects:

Performance effects:

-     Positive: Thread-safe connection reuse across multiple threads reduces connection overhead
-     Positive: Efficient JSON handling with tao::json library
-     Positive: Connection pooling handled automatically by underlying Couchbase SDK
-     Minimal overhead: Thin wrapper around official Couchbase C++ SDK

Breaking backward compatibility:

-     this is a new feature addition
-     However, building brpc now requires some additional dependencies such as couchbase-cxx-library and its dependencies
-     Existing brpc functionality remains unchanged

…wing to have multiple cluster connections

…README for couchbase

…/cb_brpc into couchbase_sdk_brpc

giriraj-singh-couchbase · 2025-08-13T18:55:36Z

example/couchbase_example/multi_threaded_client.cpp

+  std::vector<bthread_t> bids;
+  for (int i = 0; i < 5; i++) {
+    bthread_t bid;
+    if (bthread_start_background(&bid, nullptr, threaded_example, &params[i]) !=


All the threads sharing a single cluster connection, all the threads are communicating to different buckets and performing CRUD operations.

giriraj-singh-couchbase · 2025-08-13T18:58:03Z

example/couchbase_example/client.cpp

+      .adhoc(false);
+
+  // add positional parameters, you can also use named parameters and other
+  // query options that might be required.


query functions is overloaded with query_options so that end-user can have full control over the query. Here, we have used placeholders '$1' to test the working of the query_options.

giriraj-singh-couchbase · 2025-08-13T18:59:23Z

src/brpc/couchbase.h

+  void CloseCouchbase();
+
+  // query helper functions
+  std::pair<bool, std::vector<std::string>> Query(std::string statement);


cluster level query without any query_options. Simple string type queries, without any parameters.

giriraj-singh-couchbase · 2025-08-13T19:00:16Z

src/brpc/couchbase.h

+  // query helper functions
+  std::pair<bool, std::vector<std::string>> Query(std::string statement);
+  std::pair<bool, std::vector<std::string>> Query(
+      std::string statement, couchbase::query_options& q_opts);


cluster level query supporting query_options.

Great this is here.

giriraj-singh-couchbase · 2025-08-13T19:01:15Z

src/brpc/couchbase.h

+      std::string statement, couchbase::query_options& q_opts);
+  std::pair<bool, std::vector<std::string>> Query(
+      std::string statement, const std::string& bucket_name,
+      const std::string& scope = "_default");


Scope level query. However in the query(statement) the collection is needed to specified i.e. inside the string itself.

giriraj-singh-couchbase · 2025-08-13T19:01:35Z

src/brpc/couchbase.h

+      const std::string& scope = "_default");
+  std::pair<bool, std::vector<std::string>> Query(
+      std::string statement, const std::string& bucket_name,
+      const std::string& scope, couchbase::query_options& q_opts);


Scope level query with query_options.

giriraj-singh-couchbase · 2025-08-13T19:03:32Z

example/couchbase_example/client.cpp

+            << std::endl;
+  start = std::chrono::high_resolution_clock::now();
+  std::string scoped_query =
+      "SELECT META().id, email FROM _default WHERE email LIKE '%@%'";  // Here


When running a scoped query then we need to give a collection in the FROM field. So the _default is needed inside the query.

ingenthr

This isn't a completely thorough review, but I think there are a couple places to examine:

the initialization might need to consider a case where the connection won't work right now, but might work in the future. it's a bit binary at the moment, but a couchbase SDK instance isn't binary with respect to a 'connection'. I'm not sure the best way to reconcile this. Options might be to expose the ping/health_check or to put an optional waitUntilReady() in the initialization.
I'm not sure about the error handling. If this aligns to how other projects in bRPC do it, then I think it's fine, but we might need to populate some other closed over or thread local variable with error details. Can you have a thought on this?

ingenthr · 2025-08-22T13:56:41Z

docs/en/couchbase_client.md

+
+All operations return boolean values or pairs indicating success/failure:
+
+- **CRUD operations**: Return `true` on success, `false` on failure


so, in the 'false' case, wouldn't a user need more information? If it's a CAS mismatch, I'll want to backoff/retry, but if it's an add() case and the doc is already there, no point in retrying.

It's not boolean for all the CRUD operations. I will fix the doc. So, currently get is having a (bool and the error) as the return types so that client can know what error has actually come. Other operations(add,upsert,delete) are having the boolean return type which I should change as well following your later comment on the functions.

ingenthr · 2025-08-22T13:57:44Z

example/couchbase_example/multi_threaded_client.cpp

@@ -0,0 +1,141 @@
+#include <bthread/bthread.h>


nice that we have a multithread sample!

ingenthr · 2025-08-22T14:00:23Z

example/couchbase_example/multi_threaded_client.cpp

+    std::string key = "item::" + std::to_string(i);
+    std::string value =
+        R"({"name": "John Doe", "age": 31, "email": "[email protected]", "updated": true})";
+    if (couchbase_client->CouchbaseAdd(key, value, bucket_name)) {


I've not looked, but is this how the memcached handles errors @giriraj-singh-couchbase ? Is there an easy way to have this if() branching but still have details on failures? This one is related to my other feedback that we may need to know when the add() fails, is it because of something unusual (in which case I'll want to retry) or because of something that will never work (because some other client added the doc first)

Oh, okay. I was printing the errors inside the functions itself and then simply returning false. Client might want to keep different logic based on the errors. So over here, the changes that I am thinking is to return the boolean true or false and the error with it too if any. and based upon the error, user can handle the client code logic.

ingenthr · 2025-08-22T14:02:24Z

src/brpc/couchbase.cpp

+}
+
+// Query operation at cluster level without query options
+std::pair<bool, vector<std::string>> CouchbaseWrapper::Query(


Nice that you got queries in here!

ingenthr · 2025-08-22T14:02:52Z

src/brpc/couchbase.h

+  void CloseCouchbase();
+
+  // query helper functions
+  std::pair<bool, std::vector<std::string>> Query(std::string statement);


ingenthr · 2025-08-22T14:07:09Z

src/brpc/couchbase.cpp

+    auto [connect_err, cluster] =
+        couchbase::cluster::connect(connection_string, options).get();
+
+    if (connect_err) {


This is a section that probably requires some thought. Generally with the SDKs there is no such thing as a "connection failure". once initialized, it tries to connect forever. There are pros and cons to this, but it's where we are at the moment.

The code here reads as if it's looking for "a connection" which isn't really how it works. Maybe it should be a waituntilready(). I've added @DemetrisChr as a reviewer. He uses this SDK from multiple places, so he may be able to offer some advice.

The C++ SDK behaves a little differently compared to other SDKs here. cluster::connect blocks until a memcached connection is established and a config is seen on that connection, up until the bootstrap timeout.

So a wait_until_ready is not as necessary, and why it it hasn't been added to the SDK yet (although wait_until_ready can also be used for checking the readiness of services, so we probably need to add it eventually).

Hii @DemetrisChr, Can you tell me how is bootstrap timeout different from connect timeout? As far as I can think of bootstrap timeout means that the total time that can be taken for full initialisation and setup of clusters whereas connect timeout is for establishing of sockets only. Is it correct?

Yes, that's right.

The connect timeout is the timeout for establishing each individual memcached connection, used here. This includes connections made both during the initial bootstrap phase & any connections made later on (for example, connections made when opening buckets).

The bootstrap timeout (used here) covers the entire bootstrap phase, until a single connection is established successfully and a config received, which can involve a number of connection retries.

ingenthr · 2025-08-22T14:10:49Z

src/brpc/couchbase.h

+
+  // Upsert (insert/update) document
+  bool CouchbaseUpsert(const std::string& key, const std::string& value,
+                       const std::string& bucket_name,


We may want to file an improvement issue for the future to use fully qualified buckets/scopes/collections. This offers protection from a delete-recreate of a collection being logically different. I think this is fine for now though.

I do mention it as it may change the API in the future. I guess we could add something that isn't a string.

Can you please tell me more about fully qualified buckets/scopes/collection? I was not able to get the context of what is the meaning of that. However, we can do operation at any level: buckets, scopes and collections all are supported in this very function.

ingenthr · 2025-08-22T14:11:44Z

src/brpc/couchbase.h

+  // query helper functions
+  std::pair<bool, std::vector<std::string>> Query(std::string statement);
+  std::pair<bool, std::vector<std::string>> Query(
+      std::string statement, couchbase::query_options& q_opts);


Great this is here.

…ased upon the errors the code can be driven, tested and did changes to work on linux.

github-actions

license-eye has totally checked 1730 files.

Valid	Invalid	Ignored	Fixed
865	4	861	0

Click to see the invalid file list

example/couchbase_example/client.cpp
example/couchbase_example/multi_threaded_client.cpp
src/brpc/couchbase.cpp
src/brpc/couchbase.h

github-actions · 2025-08-31T16:27:45Z

example/couchbase_example/client.cpp

@@ -0,0 +1,394 @@
+#include <gflags/gflags.h>


Suggested change

#include <gflags/gflags.h>

/*

* Licensed to the Apache Software Foundation (ASF) under one

* or more contributor license agreements. See the NOTICE file

* distributed with this work for additional information

* regarding copyright ownership. The ASF licenses this file

* to you under the Apache License, Version 2.0 (the

* "License"); you may not use this file except in compliance

* with the License. You may obtain a copy of the License at

*

* http://www.apache.org/licenses/LICENSE-2.0

*

* Unless required by applicable law or agreed to in writing,

* software distributed under the License is distributed on an

* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY

* KIND, either express or implied. See the License for the

* specific language governing permissions and limitations

* under the License.

*/

#include <gflags/gflags.h>

github-actions · 2025-08-31T16:27:45Z

example/couchbase_example/multi_threaded_client.cpp

@@ -0,0 +1,161 @@
+#include <bthread/bthread.h>


Suggested change

#include <bthread/bthread.h>

/*

* Licensed to the Apache Software Foundation (ASF) under one

* or more contributor license agreements. See the NOTICE file

* distributed with this work for additional information

* regarding copyright ownership. The ASF licenses this file

* to you under the Apache License, Version 2.0 (the

* "License"); you may not use this file except in compliance

* with the License. You may obtain a copy of the License at

*

* http://www.apache.org/licenses/LICENSE-2.0

*

* Unless required by applicable law or agreed to in writing,

* software distributed under the License is distributed on an

* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY

* KIND, either express or implied. See the License for the

* specific language governing permissions and limitations

* under the License.

*/

#include <bthread/bthread.h>

github-actions · 2025-08-31T16:27:45Z

src/brpc/couchbase.cpp

@@ -0,0 +1,299 @@
+#include "brpc/couchbase.h"


Suggested change

#include "brpc/couchbase.h"

/*

* Licensed to the Apache Software Foundation (ASF) under one

* or more contributor license agreements. See the NOTICE file

* distributed with this work for additional information

* regarding copyright ownership. The ASF licenses this file

* to you under the Apache License, Version 2.0 (the

* "License"); you may not use this file except in compliance

* with the License. You may obtain a copy of the License at

*

* http://www.apache.org/licenses/LICENSE-2.0

*

* Unless required by applicable law or agreed to in writing,

* software distributed under the License is distributed on an

* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY

* KIND, either express or implied. See the License for the

* specific language governing permissions and limitations

* under the License.

*/

#include "brpc/couchbase.h"

github-actions · 2025-08-31T16:27:45Z

src/brpc/couchbase.h

@@ -0,0 +1,82 @@
+#pragma once


Suggested change

#pragma once

/*

* Licensed to the Apache Software Foundation (ASF) under one

* or more contributor license agreements. See the NOTICE file

* distributed with this work for additional information

* regarding copyright ownership. The ASF licenses this file

* to you under the Apache License, Version 2.0 (the

* "License"); you may not use this file except in compliance

* with the License. You may obtain a copy of the License at

*

* http://www.apache.org/licenses/LICENSE-2.0

*

* Unless required by applicable law or agreed to in writing,

* software distributed under the License is distributed on an

* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY

* KIND, either express or implied. See the License for the

* specific language governing permissions and limitations

* under the License.

*/

#pragma once

…UD and query

ingenthr · 2025-10-01T03:44:18Z

docs/en/couchbase_client.md

+  } else {
+    //based on the returned error do actual error handling
+    if(add_response.err.ec() == couchbase::errc::key_value::document_exists){
+        //handle the error


Much better-- it's great when these kinds of examples show at least rudimentary error handling.

ingenthr · 2025-10-01T03:45:10Z

docs/en/couchbase_client.md

 ```

-**Note**: If scope and collection are not specified, they default to `"_default"`.
+**Note**: If scope and collection are not specified, they set to `"_default"`.


Kind of funny English here to my en_US eyes. Instead "are set" might be better.

ingenthr · 2025-10-01T03:50:22Z

example/couchbase_example/client.cpp

-DEFINE_string(username, "selfdb", "Couchbase username");
-DEFINE_string(password, "Selfdb@1", "Couchbase password");
+DEFINE_string(couchbase_host, "couchbase://localhost", "Couchbase server host");
+DEFINE_string(username, "Administrator", "Couchbase username");


Nitpick: I tend to not like to use Administrator/password everywhere, as it encourages bad security behavior. They should have a normal user. Not necessary to change though unless you think about it.

ingenthr · 2025-10-01T03:52:43Z

example/couchbase_example/client.cpp

-              "Couchbase server host");
-DEFINE_string(username, "selfdb", "Couchbase username");
-DEFINE_string(password, "Selfdb@1", "Couchbase password");
+DEFINE_string(couchbase_host, "couchbase://localhost", "Couchbase server host");


Nitpick: examples like this are often great places to drop in a few commented out examples of common things a user might want to do. For example, switch on TLS. Or add the wan development profile.

See:
https://docs.couchbase.com/cxx-sdk/current/howtos/managing-connections.html#connecting-to-a-cluster

No need to change unless you want to though.

giriraj-singh-couchbase added 16 commits August 6, 2025 11:05

added support for couchbase

f2b6706

Added support for buckets selection during operations

0676294

Removed Global couchbase connection with object based connection allo…

bff85ac

…wing to have multiple cluster connections

Merge branch 'apache:master' into couchbase_sdk_brpc

9151264

Changes in Makefile for cross-platform library detection and updated …

0ebaad5

…README for couchbase

Merge branch 'couchbase_sdk_brpc' of https://github.com/couchbaselabs…

cf6365d

…/cb_brpc into couchbase_sdk_brpc

fixed some error formats

fa9e6de

minor formatting

2774f22

added support for scopes, collections and query operations

0cf550a

Added example of query_operations in scope level query execution

d618a49

Fixed minor bugs and added couchbase_client.md

5168e0e

Formatted the code according to Google C++ coding style

e4e0b88

fixed bad variable names

8cbe182

Fixed the confusing class name

8942027

Merge branch 'couchbase_sdk_brpc' of https://github.com/couchbaselabs…

c87036b

…/cb_brpc into couchbase_sdk_brpc

Updated Documentation

a7b5e93

giriraj-singh-couchbase commented Aug 13, 2025

View reviewed changes

giriraj-singh-couchbase requested a review from ingenthr August 13, 2025 19:05

giriraj-singh-couchbase self-assigned this Aug 13, 2025

giriraj-singh-couchbase added the enhancement New feature or request label Aug 13, 2025

giriraj-singh-couchbase marked this pull request as ready for review August 13, 2025 19:06

giriraj-singh-couchbase requested review from deniswsrosa and shyam-cb August 14, 2025 08:28

ingenthr requested a review from DemetrisChr August 22, 2025 13:54

ingenthr requested changes Aug 22, 2025

View reviewed changes

Now the wrapper will return a response object upon an operation and b…

093ae0c

…ased upon the errors the code can be driven, tested and did changes to work on linux.

github-actions bot reviewed Aug 31, 2025

View reviewed changes

giriraj-singh-couchbase added 19 commits September 4, 2025 07:48

fixed license action and some build actions

199db04

checking actions

89a63fa

Merge branch 'master' into couchbase_sdk_brpc

26b59d1

fixing make on macos && linux

a93c234

resolved conflicts

a7a6f9e

Fixing the issue of building with bazel

db431f3

adding python dep for bazel

f687dfa

fixing bazel

e55a24d

fixing bazel

123fa25

fixing bazel

e87b842

fixing bazel

743e61e

fixing bazel

be184b4

reverting bazel changes

1cf357e

fixed cmake

9dbf1ff

cmake fixing

4f2c925

fixing bazel

cc2135f

fixing bazel

604649b

fixing bazel

1f96cca

fixing bazel

34e62f3

giriraj-singh-couchbase requested a review from ingenthr September 10, 2025 08:34

Improved the response type, made a single common response type for CR…

b5e7c7e

…UD and query

ingenthr approved these changes Oct 1, 2025

View reviewed changes


		All operations return boolean values or pairs indicating success/failure:

		- CRUD operations: Return `true` on success, `false` on failure

-#include <gflags/gflags.h>
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <gflags/gflags.h>

Couchbase sdk brpc #3

Are you sure you want to change the base?

Couchbase sdk brpc #3

Uh oh!

Conversation

giriraj-singh-couchbase commented Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

What is changed and the side effects?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ingenthr left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

giriraj-singh-couchbase Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

giriraj-singh-couchbase Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 31, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 31, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 31, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 31, 2025

Choose a reason for hiding this comment

Uh oh!

giriraj-singh-couchbase commented Aug 13, 2025 •

edited

Loading

giriraj-singh-couchbase Aug 22, 2025 •

edited

Loading

giriraj-singh-couchbase Aug 27, 2025 •

edited

Loading