Home Page of Helio Perroni Filho http://machineawakening.blogspot.com.tr/2015/12/fft-based-cosine-sim...
HOME PAGE OF
HELIO PERRONI
FILHO
"THE FUTURE IS ALREADY HERE... IT'S
J U S T N O T E V E N LY D I S T R I B U T E D . "
(WILLIAM GIBSON)
M O N D A Y, D E C E M B E R 2 1 , 2 0 1 5
FFT-based cosine similarity for fast PA G E S
template matching Home
A common task in computer vision applications is Projects
template matching, the search for a comparatively Blog
small image patch – or more likely, its closest
approximation – in a larger image. In its simplest form,
this implies solving the following optimization: ARCHIVES
Where the similarity metric is defined
between a template and all patches
such that:
For an image with and (i.e.
the image is much larger than the template). A
common choice of similarity metric is cosine similarity,
which is defined as:
Cosine similarity is a reliable metric, but as defined
above its evaluation is rather costly. Combining
formulas , and gives:
1/7 7.2.2017 22:25
Home Page of Helio Perroni Filho http://machineawakening.blogspot.com.tr/2015/12/fft-based-cosine-sim...
Tweets by @xperroni
Helio Perroni Fi
@xperroni
Use .kateconfig for setting
per-project indentation and
Which is of time complexity – i.e. the tab settings in Kate.
size of the template times the size of the image. nhaehnle.blogspot.com.br/20
15/01/use-ka…
Depending on image and template sizes, the intended
use case (e.g. batch vs. real-time) or scale of the
template matching task (e.g. a couple dozen vs.
thousands of images), this may incur in prohibitive Helio Perroni Fi
processing costs. @xperroni
IBM Quantum Computing
platform open for general
Cosine similarity template matching can be sped up
access.
with application of the convolution theorem. First, let's research.ibm.com/quantum/
redefine formula as:
Helio Perroni Fi
@xperroni
Where: A repository of freely-licensed
art for FLOSS games.
opengameart.org
Embed View on Twitter
Looking into the three terms above, it's clear that
is constant for any given and can therefore be left
out of the computation. On the other hand, is just
the cross-correlation between and , which by the
convolution theorem can also be computed as:
Where is the fourier transform operator, the
inverse transform, the asterisk denotes the complex
conjugate, and is the Hadamard product
(element-wise multiplication) of the two transforms.
2/7 7.2.2017 22:25
Home Page of Helio Perroni Filho http://machineawakening.blogspot.com.tr/2015/12/fft-based-cosine-sim...
Likewise, can be computed as the cross-
correlation between and a window filter
:
The advantage of this approach is that algorithms
such as the Fast Fourier Transform (FFT) are of time
complexity , which depending on the
relative sizes of and may be faster than the
of the direct computation. Also the
Fourier transforms of and (if the same template will
be applied to several images) can be cached,
further saving up computation time.
Implementation
The C++ code below provides a basic implementation
of the method outlined above. It uses the popular
OpenCV library, with some further optimizations
particular to its implementation of the Fourier
transform, explained in the comments.
#include <opencv2/opencv.hpp>
static const cv::Scalar ONE(1);
static const cv::Scalar ZERO(0);
static const cv::Scalar WHITE(255, 255, 255);
// Fourier transform performance is not a monotonic functi
// size - matrices whose dimensions are powers of two are
// process, and multiples of 2, 3 and 5 (for example, 300
// also processed quite efficiently. Therefore it makes se
// data with zeros to get a bit larger matrix that can be
// faster than the original one.
cv::Size fit(const cv::Size &size)
{
return cv::Size(cv::getOptimalDFTSize(size.width),
cv::getOptimalDFTSize(size.height));
}
cv::Mat F_fwd(const cv::Mat &I, const cv::Size &size)
{
// Pad input matrix to given size.
3/7 7.2.2017 22:25
Home Page of Helio Perroni Filho http://machineawakening.blogspot.com.tr/2015/12/fft-based-cosine-sim...
cv::Mat P;
int m = size.height - I.rows;
int n = size.width - I.cols;
cv::copyMakeBorder(I, P, 0, m, 0, n, cv::BORDER_CONSTANT
// Compute Fourier transform for input data. The last ar
// informs the dft() function of how many non-zero rows
// so it can handle the rest of the rows more efficientl
// some time.
cv::Mat F;
cv::dft(P, F, 0, I.rows);
return F;
}
cv::Mat F_inv(const cv::Mat &F, const cv::Size &size)
{
// Compute inverse Fourier transform for input data. The
// argument informs the dft() function of how many non-z
// rows are expected in the output, so it can handle the
// of the rows more efficiently and save some time.
cv::Mat I;
cv::dft(F, I, cv::DFT_INVERSE + cv::DFT_SCALE, size.heig
return I(cv::Rect(0, 0, size.width, size.height));
}
cv::Mat C(const cv::Mat &T, const cv::Mat &I, const cv::Si
{
// Compute the Fourier transforms of template and image.
cv::Mat F_T = F_fwd(T, size);
cv::Mat F_I = F_fwd(I, size);
// Compute the cross correlation in the frequency domain
cv::Mat F_TI;
cv::mulSpectrums(F_I, F_T, F_TI, 0, true);
// Compute the inverse Fourier transform of the cross-co
// dismissing those rows and columns of the cross-correl
// matrix that would require the template to "roll over"
cv::Size clipped;
clipped.width = I.cols - T.cols;
clipped.height = I.rows - T.rows;
return F_inv(F_TI, clipped);
}
4/7 7.2.2017 22:25
Home Page of Helio Perroni Filho http://machineawakening.blogspot.com.tr/2015/12/fft-based-cosine-sim...
cv::Mat W(const cv::Mat &T)
{
return cv::Mat(T.size(), CV_64F, ONE);
}
cv::Point3f matchTemplate(const cv::Mat &T, const cv::Mat
{
// Compute the optimal size for DFT computing.
cv::Size size = fit(I.size());
//Compute the cross-correlation and normalizing matrix.
cv::Mat C_TI = C(T, I, size);
cv::Mat M_I = C(W(T), I.mul(I), size);
int i_s, j_s;
float r = 0;
int rows = C_TI.rows;
int cols = C_TI.cols;
for (int i = 0; i < rows; i++)
{
for (int j = 0; j < cols; j++)
{
float v = C_TI.at(i, j) / sqrt(M_I.at
if (r < v)
{
r = v;
i_s = i;
j_s = j;
}
}
}
return cv::Point3f(j_s, i_s, r);
}
cv::Mat L(const cv::Mat &I)
{
cv::Mat L_I, L_F;
cv::cvtColor(I, L_I, CV_BGR2GRAY);
L_I.convertTo(L_F, CV_64F);
return L_F;
}
int main(int argc, char *argv[])
5/7 7.2.2017 22:25
Home Page of Helio Perroni Filho http://machineawakening.blogspot.com.tr/2015/12/fft-based-cosine-sim...
{
cv::Mat T = cv::imread(argv[1]);
cv::Mat I = cv::imread(argv[2]);
cv::Point3f match = matchTemplate(L(T), L(I));
cv::rectangle(I, cv::Rect(match.x, match.y, T.cols, T.ro
cv::imshow("Template", T);
cv::imshow("Image", I);
cv::waitKey();
return 0;
}
P O S T E D BY HE LI O P E RR ON I F I L H O AT 1 0: 5 1 P M
Home Older Post
6/7 7.2.2017 22:25
Home Page of Helio Perroni Filho http://machineawakening.blogspot.com.tr/2015/12/fft-based-cosine-sim...
Subscribe to: Post Comments (Atom)
7/7 7.2.2017 22:25
