G-API: VPL Add VAAPI into tests & VPL sample #22212

SergeyIvanov87 · 2022-07-08T08:49:50Z

Modified oneVPL sample:
-added device support matrix
-added user-friend hints about support configuration
-added flexible selection of decode, preproc & infer devices according to support matrix

Modified CfgDeviceSelector
-added VAAPI device selection

Modified UT
-added vaapi device selection test cases

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

Build Configuration

Xforce_builders=Custom,Custom Win,Custom Mac
build_gapi_standalone:Linux x64=ade-0.1.1f
build_gapi_standalone:Win64=ade-0.1.1f
build_gapi_standalone:Mac=ade-0.1.1f
build_gapi_standalone:Linux x64 Debug=ade-0.1.1f

buildworker:Custom=linux-1
build_gapi_standalone:Custom=ade-0.1.1f
build_image:Custom=onevpl-2021.6.0
test_opencl:Custom=OFF

Xbuild_image:Custom=ubuntu-openvino-2021.3.0:20.04
Xbuild_image:Custom Win=openvino-2021.2.0
build_image:Custom Mac=openvino-2021.2.0

test_modules:Custom=gapi,python2,python3,java
test_modules:Custom Win=gapi,python2,python3,java
test_modules:Custom Mac=gapi,python2,python3,java

build_image:Custom Win=gapi-onevpl-2021.6.0
buildworker:Custom Win=windows-3
build_contrib:Custom Win=OFF

dmatveev · 2022-07-09T16:20:19Z

modules/gapi/CMakeLists.txt

-    endif()
-    ocv_target_link_libraries(${the_module} PRIVATE ${PKG_LIBVA_LIBRARIES} ${PKG_THREAD_LIBRARIES})
+    else(WITH_VA)
+      message(FATAL_ERROR "libva not found: building HAVE_GAPI_ONEVPL without libVA support is impossible on UNIX systems")


Is this really a fatal error? Shouldn't the functionality be disabled instead?

good question.
VPL on linux doesn't work on CPU. So without VA_API it is useless
What do you think - should we allow to configure VPL without VAAPI for now?

let's put it as is for now. Later we added dynamic plugin feature which will rewrite this Cmake part anyway
What do you think?

ok, will see

modules/gapi/include/opencv2/gapi/streaming/onevpl/accel_types.hpp

dmatveev · 2022-07-09T16:24:56Z

modules/gapi/samples/onevpl_infer_single_roi.cpp

+        gpu_accel_ctx = cv::util::make_optional(
+                            cv::gapi::wip::onevpl::create_vaapi_context(nullptr));
+#endif // defined(HAVE_VA) || defined(HAVE_VA_INTEL)
+#endif // #ifdef __linux__


We should absorb all this complexity someday. User apps shouldn't be that complicated to use a VA path..

I put comment above:
https://github.com/opencv/opencv/pull/22212/files#diff-b4563bcdbe86eb5c8f1f2cce227809b02980777b25f5f4be2bb717826bedbebbR414

The idead behind the current sample is demonstration of the most complex way how to use VPL with GPU. This complexity represent a way HOW-TO-CHOOSE-SUITABLE-GPU from multi-acceleration hardware setup. When you have two or more GPUs then it should provide stable and clearest way how to enumerate them and use as pipeline accelerator.

Additionally, user might got their own framework/pipeline which uses PREdefined GPU device from own framework primitives AND i do not see reasons why we should not REuse this straight device handle with native API instead of using VPL/GAPI "default" one (simplest but not worth way). IN more complicated case use-predefined GPU might be differen when VPL/GAPI/IE recreate it by itself by using "default" selection. S0, for REAL scenarious (not synthetic one) we must implement accepting user-defiend device like these

So, I had intention to explain the most customizable scenarion by this sample (which is trickly and maybe complicated for newcomers), because the default GPU device selection is pretty simple: just invoke default onevpl::GSource ctor with like this:

auto s = onevpl::GSource( CfgParam {"mfxImplDescription.AccelerationMode","MFX_ACCEL_MODE_VIA_D3D11"});

modules/gapi/src/streaming/onevpl/cfg_param_device_selector.cpp

dmatveev · 2022-07-09T16:27:22Z

modules/gapi/src/streaming/onevpl/cfg_param_device_selector.cpp

-            GAPI_LOG_WARNING(nullptr, "TODO MFX_IMPL_VIA_VAAPI falls back to CPU case")
+#ifdef __linux__
+#if defined(HAVE_VA) || defined(HAVE_VA_INTEL)
+            static const char *predefined_vaapi_devices_list[] {"/dev/dri/renderD128",


If we have all this code here, why do we have it in the sample app as well?

this is "default device selection" part - for use-cases when we do not want to choose suitable GPU device for Source and just select the first one. It's according to the "simplest sample" configuration, which is not interesting case as sample, as for me :)

Please, check out comment in sample: https://github.com/opencv/opencv/pull/22212/files#diff-b4563bcdbe86eb5c8f1f2cce227809b02980777b25f5f4be2bb717826bedbebbR414

The current "complicated" sample demonstrate how to assing user-selected device for GAPI-pipelien: source, preproc, ie. The sample demonstrates this extended API usage using first-device in sample as "custom device". Therefore this "device selection part" is mostly the same in here and sample

In other words:
it is possible for using simple sample with auto selection GPU device by oneVPL source (you commented the right line for default device selection part)
But this simple demonstration sample for GPU will cause questions from intermediate users whos have got a mulltiple GPUs - like How-To-Use-My-Specifc device.

So, It is possible to anticipate such questions and provide complete example - "complicated sample" which uncover these questions. And because this samples demonstrates an interface with idea of fine-grained device selection - it doesn't matter ( from sample point of view) how specific device will be selected: it's assumed that user have one. Thus default device selection part is the same: in sample and in default-branch of cfg_param_device_selector

dmatveev · 2022-07-09T16:27:56Z

modules/gapi/src/streaming/onevpl/cfg_param_device_selector.hpp

 private:
    Device suggested_device;
    Context suggested_context;
+    std::unique_ptr<Aux> platform_specific_data;


Abstraction leak, isnt it?

?
Would you be kind to clarify your point?

…arams

dmatveev

👍

SergeyIvanov87 · 2022-08-01T09:17:57Z

@asmorkalov, @alalek Wouldn't you mind to merge it please?

Add VAAPI into tests & VPL sample

f562264

SergeyIvanov87 added the category: g-api / gapi label Jul 8, 2022

SergeyIvanov87 requested a review from dmatveev July 8, 2022 08:50

Apply WITHA_VA option in cmake

b3b235e

dmatveev self-assigned this Jul 9, 2022

dmatveev added this to the 4.7.0 milestone Jul 9, 2022

dmatveev reviewed Jul 9, 2022

View reviewed changes

SergeyIvanov87 added 5 commits July 12, 2022 10:54

Improve UX: autofill cfg param with proper accel type, printout cfg p…

3635b3d

…arams

Forbid Rctx in IE for VAAPI temporary, Add preproc VAAPI

2b2ba53

Fix sample WIN compilation

ff31f90

Remove clnn header from sample

1dee848

Address several comments

9ffb674

SergeyIvanov87 requested a review from dmatveev July 20, 2022 11:09

dmatveev approved these changes Jul 25, 2022

View reviewed changes

asmorkalov merged commit 885a446 into opencv:4.x Aug 3, 2022

SergeyIvanov87 deleted the gapi_vpl_multiple_devices branch August 4, 2022 19:39

sivanov-work mentioned this pull request Aug 16, 2022

G-API: Use different devices processing in streaming pipeline #21716

Closed

6 tasks

alalek mentioned this pull request Aug 21, 2022

(5.x) Merge 4.x #22408

Merged

Uh oh!

G-API: VPL Add VAAPI into tests & VPL sample #22212

G-API: VPL Add VAAPI into tests & VPL sample #22212

Uh oh!

Conversation

SergeyIvanov87 commented Jul 8, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Readiness Checklist

Build Configuration

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SergeyIvanov87 Jul 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SergeyIvanov87 Jul 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmatveev left a comment

Choose a reason for hiding this comment

Uh oh!

SergeyIvanov87 commented Aug 1, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

SergeyIvanov87 commented Jul 8, 2022 •

edited

Loading

SergeyIvanov87 Jul 13, 2022 •

edited

Loading

SergeyIvanov87 Jul 12, 2022 •

edited

Loading