videoio: add Orbbec Gemini 330 camera support #27230

sirudoi · 2025-04-14T15:19:25Z

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
The feature is well documented and sample code can be built with the project CMake

Description of Changes

motivated：

Orbbec has launched a new RGB-D camera — the Gemini 330. To fully leverage the capabilities of the Gemini 330, Orbbec simultaneously released version 2 of the open-source OrbbecSDK. This PR adapts the support for the Gemini 330 series cameras to better meet and respond to users’ application requirements.

change：

Add support for the Orbbec Gemini330 camera.
Fixed an issue with Femto Mega on Windows 10/11; for details, see issue.
When enabling HAVE_OBSENSOR_ORBBEC_SDK, the build now fetches version 2 of the OrbbecSDK, and the sample API calls have been updated to the v2 format.

Testing

OS	Compiler	Camera	Result
Windows 11	(VS2022) MSVC runtime library version 14.40	Gemini 335/336L	Pass
Windows 11	(VS2022) MSVC runtime library version 14.19	Gemini 335/336L	Pass
Ubuntu22.04	GCC 11.4	Gemini 335/336L	Pass
Ubuntu18.04	GCC 7.5	Gemini 335/336L	Pass

Acknowledgements

Thank you to the OpenCV team for the continuous support and for creating such a robust open source project. I appreciate the valuable feedback from the community and reviewers, which has helped improve the quality of this contribution!

3rdparty/orbbecsdk/orbbecsdk.cmake

asmorkalov · 2025-04-16T07:46:48Z

@fengyuentau Could you handle the pr?

fengyuentau · 2025-04-16T08:02:25Z

@fengyuentau Could you handle the pr?

@kaingwade will help to test this PR once we get the camera.

kaingwade

Tested the latest Gemini 330 series on Windows 10 and Ubuntu 22.04 (6.8.0-57)，335/335L/336 work well, but 336L only outputs the RGB stream.

kaingwade · 2025-04-18T08:40:58Z

modules/videoio/cmake/detect_obsensor.cmake

+        if(WIN32)
+          # copy orbbecsdk dll to output floader
+          if(MSVC_IDE)
+            file(COPY ${OrbbecSDK_DLL_FILES} DESTINATION ${EXECUTABLE_OUTPUT_PATH}/Release)
+            file(COPY ${OrbbecSDK_DLL_FILES} DESTINATION ${EXECUTABLE_OUTPUT_PATH}/Debug)
+            file(COPY ${OrbbecSDK_RUNTIME_RESOURCE_FILES} DESTINATION ${EXECUTABLE_OUTPUT_PATH}/Release)
+            file(COPY ${OrbbecSDK_RUNTIME_RESOURCE_FILES} DESTINATION ${EXECUTABLE_OUTPUT_PATH}/Debug)
+
+            file(COPY ${OrbbecSDK_LIBRARY} DESTINATION ${LIBRARY_OUTPUT_PATH}/Release)
+            file(COPY ${OrbbecSDK_LIBRARY} DESTINATION ${LIBRARY_OUTPUT_PATH}/Debug)
+          elseif(MSVC AND (CMAKE_GENERATOR MATCHES "Visual"))
+            file(COPY ${OrbbecSDK_DLL_FILES} DESTINATION ${EXECUTABLE_OUTPUT_PATH}/${CMAKE_BUILD_TYPE})
+            file(COPY ${OrbbecSDK_RUNTIME_RESOURCE_FILES} DESTINATION ${EXECUTABLE_OUTPUT_PATH}/${CMAKE_BUILD_TYPE})
+            file(COPY ${OrbbecSDK_LIBRARY} DESTINATION ${LIBRARY_OUTPUT_PATH}/${CMAKE_BUILD_TYPE})
+          else()
+            file(COPY ${OrbbecSDK_DLL_FILES} DESTINATION ${EXECUTABLE_OUTPUT_PATH})
+            file(COPY ${OrbbecSDK_RUNTIME_RESOURCE_FILES} DESTINATION ${EXECUTABLE_OUTPUT_PATH})
+            file(COPY ${OrbbecSDK_LIBRARY} DESTINATION ${LIBRARY_OUTPUT_PATH})
+          endif()
+        else()
+          file(COPY ${OrbbecSDK_RUNTIME_RESOURCE_FILES} DESTINATION ${EXECUTABLE_OUTPUT_PATH})
+          file(COPY ${OrbbecSDK_DLL_FILES} DESTINATION ${LIBRARY_OUTPUT_PATH})


Unlike on MacOS, on Windows and Linux functionality of Orbbec SDK is integrated into OpenCV other than calling the SDK directly. These lines are not involved.

modules/videoio/src/cap_obsensor/obsensor_uvc_stream_channel.cpp

3rdparty/orbbecsdk/orbbecsdk.cmake

kaingwade · 2025-04-22T09:45:54Z

LGTM.

asmorkalov · 2025-04-22T14:04:11Z

modules/videoio/src/cap_obsensor/obsensor_uvc_stream_channel.cpp

+    cv::Mat srcMat(frame->height, frame->width, CV_16UC1, frame->data);
+    cv::resize(srcMat, srcMat, cv::Size(), scale, scale, cv::INTER_LINEAR);
+
+    // left/right crop
+    const int valid_width = srcMat.cols - crop_left - crop_right;
+    if (valid_width <= 0 || srcMat.rows <= 0) {
+        CV_LOG_ERROR(NULL, "Invalid horizontal crop parameters");
+        return;
+    }
+    srcMat = srcMat(cv::Rect(crop_left, 0, valid_width, srcMat.rows));
+
+    // top padding
+    if (pad_top > 0) {
+        cv::copyMakeBorder(srcMat, srcMat, pad_top, 0, 0, 0, cv::BORDER_CONSTANT, 0);
+    }
+
+    // bottom crop
+    const int valid_height = srcMat.rows - crop_bottom;
+    if (valid_height <= 0) {
+        CV_LOG_ERROR(NULL, "Invalid vertical crop parameters");
+        return;
+    }
+    srcMat = srcMat(cv::Rect(0, 0, srcMat.cols, valid_height));


Why do you need resize and crop for the camera frames? I propose to return them in original size.
Also resized does not support in-place processing. The code always triggers srcMat realloc.

Why do you need resize and crop for the camera frames? I propose to return them in original size. Also resized does not support in-place processing. The code always triggers srcMat realloc.

Hi, the resize/crop operations ensured pixel alignment between depth and color streams with mismatched resolutions. Without them, users would receive misaligned data. I now enforce matching resolutions to eliminate these steps. Thank you for your insight!

…the cropping operation.

videoio: add Orbbec Gemini 330 camera support opencv#27230 ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] The feature is well documented and sample code can be built with the project CMake ### Description of Changes #### motivated： - Orbbec has launched a new RGB-D camera — the Gemini 330. To fully leverage the capabilities of the Gemini 330, Orbbec simultaneously released version 2 of the open-source OrbbecSDK. This PR adapts the support for the Gemini 330 series cameras to better meet and respond to users’ application requirements. #### change： - Add support for the Orbbec Gemini330 camera. - Fixed an issue with Femto Mega on Windows 10/11; for details, see [issue](opencv#23237 (comment)). - When enabling `HAVE_OBSENSOR_ORBBEC_SDK`, the build now fetches version 2 of the OrbbecSDK, and the sample API calls have been updated to the v2 format. ### Testing | OS | Compiler | Camera | Result | |:----------:|:---------------------------------------:|:-----------------:|:------:| | Windows 11 | (VS2022) MSVC runtime library version 14.40 | Gemini 335/336L | Pass | | Windows 11 | (VS2022) MSVC runtime library version 14.19 | Gemini 335/336L | Pass | | Ubuntu22.04| GCC 11.4 | Gemini 335/336L | Pass | | Ubuntu18.04| GCC 7.5 | Gemini 335/336L | Pass | ### Acknowledgements Thank you to the OpenCV team for the continuous support and for creating such a robust open source project. I appreciate the valuable feedback from the community and reviewers, which has helped improve the quality of this contribution!

sirudoi added 2 commits April 14, 2025 21:50

obsensor: add Orbbec Gemini330 camera support

1b24e5d

obsensor: pull OrbbecSDK v2 during build and update API calls

2ef9880

asmorkalov reviewed Apr 16, 2025

View reviewed changes

3rdparty/orbbecsdk/orbbecsdk.cmake Outdated Show resolved Hide resolved

asmorkalov requested a review from fengyuentau April 16, 2025 07:46

asmorkalov added category: videoio(camera) feature category: 3rdparty labels Apr 16, 2025

fengyuentau requested a review from kaingwade April 16, 2025 08:02

sirudoi added 2 commits April 17, 2025 01:07

obsensor: fix compiler warnings

49a1437

obsensor: replace OrbbecSDK Git clone with archive download

9ce46bf

kaingwade reviewed Apr 18, 2025

View reviewed changes

sirudoi added 5 commits April 21, 2025 16:49

obsensor: add firmware reminder to Note section

c45874c

obsensor: revert OrbbecSDK invocation code on macOS

e9a444f

obsensor: fix API usage to v1 version

e145f1f

obsensor: revert all API calls back to the v1 SDK.

2f1eccf

obsensor: move ObExtensionUnit definitions from header to source file

9edb7d6

kaingwade approved these changes Apr 22, 2025

View reviewed changes

asmorkalov reviewed Apr 22, 2025

View reviewed changes

obsensor: modify the stream-start resolution configuration to remove …

bf3f4e3

…the cropping operation.

asmorkalov added this to the 4.12.0 milestone Apr 23, 2025

asmorkalov approved these changes Apr 25, 2025

View reviewed changes

asmorkalov assigned kaingwade Apr 25, 2025

asmorkalov merged commit 485c7d5 into opencv:4.x Apr 25, 2025
51 of 55 checks passed

asmorkalov mentioned this pull request Apr 29, 2025

5.x merge 4.x #27265

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

videoio: add Orbbec Gemini 330 camera support #27230

videoio: add Orbbec Gemini 330 camera support #27230

sirudoi commented Apr 14, 2025 •

edited

Loading

Uh oh!

Uh oh!

asmorkalov commented Apr 16, 2025

Uh oh!

fengyuentau commented Apr 16, 2025

Uh oh!

kaingwade left a comment

Uh oh!

kaingwade Apr 18, 2025

Uh oh!

Uh oh!

Uh oh!

kaingwade commented Apr 22, 2025

Uh oh!

asmorkalov Apr 22, 2025

Uh oh!

sirudoi Apr 23, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

videoio: add Orbbec Gemini 330 camera support #27230

videoio: add Orbbec Gemini 330 camera support #27230

Conversation

sirudoi commented Apr 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Readiness Checklist

Description of Changes

motivated：

change：

Testing

Acknowledgements

Uh oh!

Uh oh!

asmorkalov commented Apr 16, 2025

Uh oh!

fengyuentau commented Apr 16, 2025

Uh oh!

kaingwade left a comment

Choose a reason for hiding this comment

Uh oh!

kaingwade Apr 18, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

kaingwade commented Apr 22, 2025

Uh oh!

asmorkalov Apr 22, 2025

Choose a reason for hiding this comment

Uh oh!

sirudoi Apr 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sirudoi commented Apr 14, 2025 •

edited

Loading