Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Tuned threshold for FastNeuralStyle_eccv16 test #24359

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Oct 13, 2023
Merged

Tuned threshold for FastNeuralStyle_eccv16 test #24359

merged 1 commit into from
Oct 13, 2023

Conversation

@asmorkalov
Copy link
Contributor

@asmorkalov asmorkalov commented Oct 4, 2023 •
edited
Loading

In case of OpenCL failure the pipeline fails back to CPU and the test compares CPU vs CPU. Looks like CPU implementation produce slightly different result and it does not fit into threshold:

[==========] Running 2 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 2 tests from DNNTestNetwork
[ RUN      ] DNNTestNetwork.FastNeuralStyle_eccv16/0, where GetParam() = OCV/OCL
[ WARN:[email protected]] global ocl4dnn_conv_spatial.cpp:1923 loadTunedConfig OpenCV(ocl4dnn): consider to specify kernel configuration cache directory through OPENCV_OCL4DNN_CONFIG_PATH parameter.
OpenCL program build log: dnn/dummy
Status -11: CL_BUILD_PROGRAM_FAILURE
-cl-no-subgroup-ifp
Error in processing command line: Don't understand command line argument "-cl-no-subgroup-ifp"!
/home/alexander/Projects/OpenCV/opencv-master/modules/dnn/test/test_common.impl.hpp:79: Failure
Expected: (normInf) <= (lInf), actual: 0.00235748 vs 0.002
Second run  |ref| = 148.02413940429688
[  FAILED  ] DNNTestNetwork.FastNeuralStyle_eccv16/0, where GetParam() = OCV/OCL (1934 ms)
[ RUN      ] DNNTestNetwork.FastNeuralStyle_eccv16/1, where GetParam() = OCV/OCL_FP16
[       OK ] DNNTestNetwork.FastNeuralStyle_eccv16/1 (1120 ms)
[----------] 2 tests from DNNTestNetwork (3054 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test case ran. (3055 ms total)
[  PASSED  ] 1 test.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] DNNTestNetwork.FastNeuralStyle_eccv16/0, where GetParam() = OCV/OCL

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@asmorkalov asmorkalov requested a review from dkurt October 4, 2023 13:19
@opencv-alalek
Copy link
Contributor

opencv-alalek commented Oct 4, 2023

I don't observe problem with AVX optimizations: http://pullrequest.opencv.org/buildbot/builders/4_x_opt-avx_noICV-skl-lin64
SSE.4.2 baseline works fine too: http://pullrequest.opencv.org/buildbot/builders/4_x_opt-sse42_noICV-skl-lin32

What is the error message from test run?

@asmorkalov
Copy link
Contributor Author

asmorkalov commented Oct 4, 2023 

[==========] Running 2 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 2 tests from DNNTestNetwork
[ RUN      ] DNNTestNetwork.FastNeuralStyle_eccv16/0, where GetParam() = OCV/OCL
[ WARN:[email protected]] global ocl4dnn_conv_spatial.cpp:1923 loadTunedConfig OpenCV(ocl4dnn): consider to specify kernel configuration cache directory through OPENCV_OCL4DNN_CONFIG_PATH parameter.
OpenCL program build log: dnn/dummy
Status -11: CL_BUILD_PROGRAM_FAILURE
-cl-no-subgroup-ifp
Error in processing command line: Don't understand command line argument "-cl-no-subgroup-ifp"!
/home/alexander/Projects/OpenCV/opencv-master/modules/dnn/test/test_common.impl.hpp:79: Failure
Expected: (normInf) <= (lInf), actual: 0.00235748 vs 0.002
Second run  |ref| = 148.02413940429688
[  FAILED  ] DNNTestNetwork.FastNeuralStyle_eccv16/0, where GetParam() = OCV/OCL (1934 ms)
[ RUN      ] DNNTestNetwork.FastNeuralStyle_eccv16/1, where GetParam() = OCV/OCL_FP16
[       OK ] DNNTestNetwork.FastNeuralStyle_eccv16/1 (1120 ms)
[----------] 2 tests from DNNTestNetwork (3054 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test case ran. (3055 ms total)
[  PASSED  ] 1 test.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] DNNTestNetwork.FastNeuralStyle_eccv16/0, where GetParam() = OCV/OCL

@opencv-alalek
Copy link
Contributor

opencv-alalek commented Oct 4, 2023

OCV/OCL

Looks like change should go with if (target == DNN_TARGET_OPENCL) condition, but

CPU_BASELINE=AVX (no dispatch) with OpenCL=Intel iGPU:

[ RUN      ] DNNTestNetwork.FastNeuralStyle_eccv16/0, where GetParam() = OCV/OCL
[ WARN:[email protected]] global ocl4dnn_conv_spatial.cpp:1921 loadTunedConfig OpenCV(ocl4dnn): consider to specify kernel configuration cache directory through OPENCV_OCL4DNN_CONFIG_PATH parameter.
/home/alalek/projects/opencv/dev/modules/dnn/test/test_common.impl.hpp:79: Failure
Expected: (normInf) <= (lInf), actual: 0.00235748 vs 0.002
Second run  |ref| = 148.02413940429688
[  FAILED  ] DNNTestNetwork.FastNeuralStyle_eccv16/0, where GetParam() = OCV/OCL (1259 ms)

AVX2 (CPU_BASELINE=AVX2, OpenCL=Intel iGPU):

[ RUN      ] DNNTestNetwork.FastNeuralStyle_eccv16/0, where GetParam() = OCV/OCL
[ WARN:[email protected]] global ocl4dnn_conv_spatial.cpp:1921 loadTunedConfig OpenCV(ocl4dnn): consider to specify kernel configuration cache directory through OPENCV_OCL4DNN_CONFIG_PATH parameter.
/home/alalek/projects/opencv/dev/modules/dnn/test/test_common.impl.hpp:79: Failure
Expected: (normInf) <= (lInf), actual: 0.0021019 vs 0.002
First run  |ref| = 148.53861999511719
[  FAILED  ] DNNTestNetwork.FastNeuralStyle_eccv16/0, where GetParam() = OCV/OCL (1094 ms)

SSE4.2 + AVX (-DCPU_BASELINE=SSE4_2 -DCPU_DISPATCH=AVX)

[ RUN      ] DNNTestNetwork.FastNeuralStyle_eccv16/0, where GetParam() = OCV/OCL
[ WARN:[email protected]] global ocl4dnn_conv_spatial.cpp:1921 loadTunedConfig OpenCV(ocl4dnn): consider to specify kernel configuration cache directory through OPENCV_OCL4DNN_CONFIG_PATH parameter.
/home/alalek/projects/opencv/dev/modules/dnn/test/test_common.impl.hpp:79: Failure
Expected: (normInf) <= (lInf), actual: 0.00235748 vs 0.002
Second run  |ref| = 148.02413940429688
[  FAILED  ] DNNTestNetwork.FastNeuralStyle_eccv16/0, where GetParam() = OCV/OCL (1109 ms)

Test computes "reference" through OCV/CPU DNN target and these values are different between target instruction sets - perhaps there is an issue in CPU optimization code.

@asmorkalov
Copy link
Contributor Author

asmorkalov commented Oct 6, 2023

@opencv-alalek What are the next steps here? I propose to merge the PR.

@asmorkalov
Copy link
Contributor Author

asmorkalov commented Oct 12, 2023

@dkurt @opencv-alalek Friendly reminder.

@asmorkalov asmorkalov changed the title Tuned threshold for FastNeuralStyle_eccv16 test for systems without AVX2 Tuned threshold for FastNeuralStyle_eccv16 test Oct 13, 2023
@asmorkalov
Copy link
Contributor Author

asmorkalov commented Oct 13, 2023

@opencv-alalek I updated the PR description.

@asmorkalov asmorkalov added this to the 4.9.0 milestone Oct 13, 2023
@asmorkalov asmorkalov merged commit 58285e5 into opencv:4.x Oct 13, 2023
@asmorkalov asmorkalov mentioned this pull request Oct 17, 2023
Merged
@asmorkalov asmorkalov mentioned this pull request Nov 8, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dkurt dkurt dkurt approved these changes

@opencv-alalek opencv-alalek opencv-alalek approved these changes

Assignees

@opencv-alalek opencv-alalek

Labels

category: dnn

Projects

None yet

Milestone

4.9.0

Development

Successfully merging this pull request may close these issues.

3 participants

@asmorkalov @opencv-alalek @dkurt