Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Use std::priority_queue in inpaint function for performance improvement #25122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Mar 2, 2024
Merged

Use std::priority_queue in inpaint function for performance improvement #25122

merged 4 commits into from
Mar 2, 2024

Conversation

@unnonouno
Copy link
Contributor

@unnonouno unnonouno commented Feb 29, 2024

In cv::inpaint implementation, it uses a priority queue with O(n) time linear search. For large images it is very slow.
I replaced it with C++'s standard library std::priority_queue, that uses O(log(n)) algorithm.
In my use case, it is x10 faster than the original.

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@asmorkalov asmorkalov changed the base branch from master to 4.x February 29, 2024 05:44
@asmorkalov
Copy link
Contributor

asmorkalov commented Feb 29, 2024 •
edited
Loading

Perf results (11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz):

Geometric mean (ms)

                        Name of Test                           4.x-3  patched-3 patched-3 
                                                                                    vs    
                                                                                  4.x-3   
                                                                                (x-factor)
Debevec::HDR                                                  97.038   94.884      1.02   
DenoisingColored::OCL_Photo                                   166.807  163.219     1.02   
DenoisingGrayscale::OCL_Photo                                 85.176   82.939      1.03   
Mertens::HDR                                                  165.032  162.349     1.02   
Robertson::HDR                                                86.545   83.813      1.03   
inpaint::InpaintArea_InpaintingMethod::(24x24, INPAINT_NS)     7.700    7.276      1.06   
inpaint::InpaintArea_InpaintingMethod::(24x24, INPAINT_TELEA)  7.684    7.184      1.07   
inpaint::InpaintArea_InpaintingMethod::(32x32, INPAINT_NS)    11.586   11.204      1.03   
inpaint::InpaintArea_InpaintingMethod::(32x32, INPAINT_TELEA) 10.596   10.177      1.04   
inpaint::InpaintArea_InpaintingMethod::(64x64, INPAINT_NS)    37.057   36.585      1.01   
inpaint::InpaintArea_InpaintingMethod::(64x64, INPAINT_TELEA) 29.687   29.543      1.00

@asmorkalov
Copy link
Contributor

asmorkalov commented Feb 29, 2024

@unnonouno Thanks a lot for the patch. Could you provide performance test case, where the improvement is visible. For now I see ~5% speedup.

asmorkalov
asmorkalov requested changes Feb 29, 2024
View reviewed changes
modules/photo/src/inpaint.cpp
CvPriorityQueueFloat& operator=(const CvPriorityQueueFloat &); // assign disabled

protected:
CvHeapElem *mem,*empty,*head,*tail;
Copy link
Contributor

@asmorkalov asmorkalov Feb 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose to remove CvPriorityQueueFloat as it is not public API and replace its usage with std object calls. cv::Ptr is not needed any more and you can use std::priority_queue& as parameter.

Copy link
Contributor Author

@unnonouno unnonouno Feb 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. That is a good idea. I'll fix it.

Copy link
Contributor Author

@unnonouno unnonouno Mar 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I need to add next_order variable in the priority queue class to keep insertion order for complete compatibility. It is because std::priority_queue cannot keep insertion order when two elements have the same score.
So I think it is simpler to keep this class. If it is OK to break compatibility, i can remove this variable. Even in that case, the result is almost same. How do you think?

@unnonouno
Copy link
Contributor Author

unnonouno commented Feb 29, 2024

Thanks a lot. It depends on the size of the image. I'll make another performance test with a large image.

@asmorkalov
Copy link
Contributor

asmorkalov commented Feb 29, 2024

Please use existing test images from opencv_extra + resize, if it's required. I do not think, that we need extra files for it.

@unnonouno
Copy link
Contributor Author

unnonouno commented Mar 1, 2024

I made a dot pattern mask, and the PR is significantly faster.

4.x branch:

[----------] 2 tests from Perf_InpaintingMethod_inpaintDots
[ RUN      ] Perf_InpaintingMethod_inpaintDots.inpaintDots/0, where GetParam() = INPAINT_NS
[ PERFSTAT ]    (samples=10   mean=14.11   median=14.01   min=13.95   stddev=0.25 (1.8%))
[       OK ] Perf_InpaintingMethod_inpaintDots.inpaintDots/0 (151 ms)
[ RUN      ] Perf_InpaintingMethod_inpaintDots.inpaintDots/1, where GetParam() = INPAINT_TELEA
.
.
[ PERFSTAT ]    (samples=10   mean=2716.57   median=2715.88   min=2712.82   stddev=3.40 (0.1%))
[       OK ] Perf_InpaintingMethod_inpaintDots.inpaintDots/1 (27175 ms)
[----------] 2 tests from Perf_InpaintingMethod_inpaintDots (27326 ms total)

This PR:

[----------] 2 tests from Perf_InpaintingMethod_inpaintDots
[ RUN      ] Perf_InpaintingMethod_inpaintDots.inpaintDots/0, where GetParam() = INPAINT_NS
[ PERFSTAT ]    (samples=10   mean=14.58   median=14.50   min=14.43   stddev=0.22 (1.5%))
[       OK ] Perf_InpaintingMethod_inpaintDots.inpaintDots/0 (155 ms)
[ RUN      ] Perf_InpaintingMethod_inpaintDots.inpaintDots/1, where GetParam() = INPAINT_TELEA
[ PERFSTAT ]    (samples=10   mean=63.58   median=63.48   min=63.03   stddev=0.45 (0.7%))
[       OK ] Perf_InpaintingMethod_inpaintDots.inpaintDots/1 (646 ms)
[----------] 2 tests from Perf_InpaintingMethod_inpaintDots (801 ms total)

@asmorkalov
Copy link
Contributor

asmorkalov commented Mar 2, 2024 

Geometric mean (ms)

                        Name of Test                           4.x-1   patch-1  patch-1  
                                                                                   vs    
                                                                                 4.x-1   
                                                                               (x-factor)
Debevec::HDR                                                   98.356  95.311     1.03   
DenoisingColored::OCL_Photo                                   164.261  164.247    1.00   
DenoisingGrayscale::OCL_Photo                                  85.242  85.831     0.99   
Mertens::HDR                                                  164.607  164.901    1.00   
Robertson::HDR                                                 88.857  88.337     1.01   
inpaint::InpaintArea_InpaintingMethod::(24x24, INPAINT_NS)     7.660    7.304     1.05   
inpaint::InpaintArea_InpaintingMethod::(24x24, INPAINT_TELEA)  7.647    7.178     1.07   
inpaint::InpaintArea_InpaintingMethod::(32x32, INPAINT_NS)     11.609  11.180     1.04   
inpaint::InpaintArea_InpaintingMethod::(32x32, INPAINT_TELEA)  10.573  10.165     1.04   
inpaint::InpaintArea_InpaintingMethod::(64x64, INPAINT_NS)     37.113  36.428     1.02   
inpaint::InpaintArea_InpaintingMethod::(64x64, INPAINT_TELEA)  29.731  29.485     1.01   
inpaintDots::Perf_InpaintingMethod::INPAINT_NS                 26.154  26.213     1.00   
inpaintDots::Perf_InpaintingMethod::INPAINT_TELEA             3841.238 88.963    43.18

asmorkalov
asmorkalov approved these changes Mar 2, 2024
View reviewed changes
Copy link
Contributor

@asmorkalov asmorkalov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@asmorkalov asmorkalov merged commit 500c55a into opencv:4.x Mar 2, 2024
@unnonouno unnonouno deleted the pqueue branch March 4, 2024 00:09
@unnonouno
Copy link
Contributor Author

unnonouno commented Mar 4, 2024

Thank you for quick review!

@unnonouno unnonouno restored the pqueue branch March 5, 2024 06:30
@asmorkalov asmorkalov mentioned this pull request Mar 6, 2024
Closed
@asmorkalov asmorkalov mentioned this pull request Apr 1, 2024
Merged
@dkurt dkurt added this to the 4.10.0 milestone Apr 8, 2024
klatism pushed a commit to klatism/opencv that referenced this pull request May 17, 2024
@unnonouno @klatism
Merge pull request opencv#25122 from unnonouno:pqueue
1d7cca0 
Use std::priority_queue in inpaint function for performance improvement opencv#25122

In `cv::inpaint` implementation, it uses a priority queue with O(n) time linear search. For large images it is very slow.
I replaced it with C++'s standard library `std::priority_queue`, that uses O(log(n)) algorithm.
In my use case, it is x10 faster than the original.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@asmorkalov asmorkalov asmorkalov approved these changes

Assignees

No one assigned

Labels

category: photo optimization

Projects

None yet

Milestone

4.10.0

Development

Successfully merging this pull request may close these issues.

3 participants

@unnonouno @asmorkalov @dkurt