Extracting the bounding area of connected components #4699
Replies: 2 comments 6 replies
-
|
Hello @4lch,
You can get the bounding box by making an XY image, then finding an indexed histogram with min and then max. But I think the optimisation in https://www.libvips.org/API/current/ctor.Image.xyz.html There's also https://www.libvips.org/API/current/method.Image.fill_nearest.html It infills zero pixels with the nearest non-zero pixel, it might do what you want. There are some edge cases where it fails, unfortunately, but it works OK for small voids. I ought to try to fix it :(
|
Beta Was this translation helpful? Give feedback.
-
|
Hi John, Thanks a lot for replying so quickly with this ! It is really interesting, and one of the many reasons I am glad we are using Vips because this has already been figured out :) Sounds like this is already the most optimized this code can be then, and performance is frankly already fantastic compared to our previous option ! The algo from the paper we used is very simple, but seems quite effective as it only requires to apply roughly as many convolutions as the maximum thickness of the defect to inpaint. It is pretty naive diffusion, but works great for small defects on regular images :) Thanks again ! |
Beta Was this translation helpful? Give feedback.
-
|
I tried a python version: #!/usr/bin/env python3
import sys
import pyvips
image = pyvips.Image.new_from_file(sys.argv[1])
A = 0.073235
B = 0.176765
diffusion = pyvips.Image.new_from_array([
[A, B, A],
[B, 0, B],
[A, B, A]
])
iterations = int(sys.argv[3]);
# defects are all-zero pixels ... we reuse this result many times, so make a memory copy
mask = (image == 0).bandand().copy_memory()
# infill with the image average
image = mask.ifthenelse(image.avg(), image)
for _ in range(0, iterations):
image = mask.ifthenelse(image.conv(diffusion, precision="integer"), image)
image.write_to_file(sys.argv[2])
So about 0.8s for everything. I used this sample image (your mask over a PNG of one of my kids): 
|
Beta Was this translation helpful? Give feedback.
-
|
The result image is: 
Which is pretty nice. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for digging deeper with this, it looks great indeed ! This seems like a really nice and performant recipe to inpaint relatively small defect areas. I am sure it would not do too great in stress tests on checkerboards etc, but for natural images this seems nice enough. |
Beta Was this translation helpful? Give feedback.
-
|
For reference, on my test image that is ~1024x9500px and monochrome, OpenCvSharp's Telea runs in ~2050ms with a 8px radius where this more naive inpainting algorithm runs in 4-30ms depending on the number of iterations requested, indeed quite an improvement :) |
Beta Was this translation helpful? Give feedback.
-
|
I'm glad it's useful. Try |
Beta Was this translation helpful? Give feedback.
-
This is a great tip thanks, somehow missed it in your previous reply ! This will definitely help to make it a little more concise as well. I will give it a try as well as the memory copy trick for the mask. Thanks again ! |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Hello !
I am currently implementing the Fast Digital Image Inpainting paper to perform high-performance image inpainting with Vips. The basic premise is pretty simple, initialize areas to be inpainted with the average value of the image, then run multiple convolutions with a diffusion kernel to bleed values outside of the areas to inpaint inwards. This works remarkably well for small areas to inpaint, even if it has a smoothing effect. The performance of my implementation that runs the convolutions over the entire image is already 2 orders of magnitude faster than OpenCV's Telea !!
Using NetVips:
I am now trying to see whether this could be optimized further, by limiting the convolutions to areas surrounding each spot to inpaint. The connected-components-labelled mask with false colors looks like this :
My question is then, how to properly get the bounding box around each connected element ? I have seen
vips_hist_find_indexed()mentioned for this but couldn't seem to wrap my head around actually using it for bounding box extraction of connected components ? I am not sure that an approach where the convolution would run on smaller portions of images would actually be faster, but this feels worth trying if it can squeeze extra performance :)Thanks a lot in advance !
Beta Was this translation helpful? Give feedback.
All reactions