Image Filtering
Readings: Ch 5: 5.4, 5.5, 5.6,5.7.3, 5.8
(This lecture does not follow the book.)
Images by Pawan Sinha
• formal terminology
• filtering with masks
• mean filter
• Gaussian filter
• general cross-correlation
• convolution
• median filter
1
What is an image?
We can think of an image as a function, f, from R2 to R:
f( x, y ) gives the intensity at position ( x, y ).
It is a continuous function, usually over a rectangle.
A color image is just three functions pasted together.
We can write this as a “vector-valued” function:
r ( x, y )
f ( x, y ) = g ( x, y )
b( x, y )
Courtesy of NASA
2
Images as functions
These are 4 different ways that people use to
show gray tones, just for illustration.
The 3D shows the “height” of each pixel.
3
Digital images
In computer vision we usually operate on digital
(discrete) images:
Sample the continuous 2D space on a
regular grid.
Quantize each sample by rounding to
nearest integer.
The image can now be represented as a matrix of
integer values.
4
Filtering Operations Use Masks
• Masks operate on a neighborhood of pixels.
• A mask of coefficients is centered on a pixel.
• The mask coefficients are multiplied by the
pixel values in its neighborhood and the
products are summed.
• The result goes into the corresponding pixel
position in the output image.
36 36 36 36 36 ** ** ** ** **
36 36 45 45 45 1/9 1/9 1/9
** 39 ** ** **
1/9 1/9 1/9
36 45 45 45 54 1/9 1/9 1/9 ** ** ** ** **
36 45 54 54 54 ** ** ** ** **
45 45 54 54 54 3x3 Mask
** ** ** ** **
Input Image Output Image
5
Application: Noise Filtering
Image processing is useful for noise reduction...
Common types of noise:
Salt and pepper noise: contains random
occurrences of black and white pixels
Impulse noise: contains random occurrences
of white pixels
Gaussian noise: variations in intensity drawn
from a Gaussian normal distribution
6
Practical noise reduction
How can we “smooth” away noise in a single
image?
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 100 130 110 120 110 0 0
0 0 0 110 90 100 90 100 0 0
0 0 0 130 100 90 130 110 0 0
0 0 0 120 100 130 110 120 0 0
0 0 0 90 110 80 120 100 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
7
Mean filtering
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
8
Mean filtering
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0 Input
0 0 0 0 0 0 0 0 0 0 Image
0 0 90 0 0 0 0 0 0 0
Sum the
0 0 0 0 0 0 0 0 0 0
values in
a 3x3 nbd.
Divide by 9.
Replace
center.
0 10 20 30 30 30 20 10
0 20 40 60 60 60 40 20
0 30 60 90 90 90 60 30
0 30 50 80 80 90 60 30
0 30 50 80 80 90 60 30
0 20 30 50 50 60 40 20
10 20 30 30 30 30 20 10 Output
10 10 10 0 0 0 0 0
Image
9
Effect of mean filters
10
Generalization:
Cross-Correlation Filtering
Let’s write this down as an equation. Assume the
averaging window is (2k+1)x(2k+1):
We can generalize this idea by allowing different
weights for different neighboring pixels:
This is called a cross-correlation operation and
written:
H is called the “filter,” “kernel,” or “mask.”
11
Mean Kernel
What’s the kernel for a 3x3 mean filter?
In other words, what do you multiply each pixel by
when you find the mean?
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
12
Mean Kernel
What’s the kernel for a 3x3 mean filter?
In other words, what do you multiply each pixel by
when you find the mean?
0 0 0 0 0 0 0 0 0 0
mean filter
1 1 1
0 0 0 0 0 0 0 0 0 0
1 1 1
0 0 0 90 90 90 90 90 0 0 1/9
1 1 1
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
13
Gaussian Filtering
A Gaussian kernel gives less weight to pixels
further from the center of the window
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0 1 2 1
0 0 0 90 90 90 90 90 0 0
2 4 2
0 0 0 90 0 90 90 90 0 0
1 2 1
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
This kernel is an approximation of a Gaussian
function:
14
Mean vs. Gaussian filtering
15
Convolution
A convolution operation is a cross-correlation
where the filter is flipped both horizontally and
vertically before being applied to the image:
It is written:
Suppose H is a Gaussian or mean kernel. How
does convolution differ from cross-correlation?
In computer vision, we tend to use symmetric
kernels most of the time, and we tend to call them
convolution kernels.
In EE, convolution is useful for solving linear
systems problems.
16
Convolution vs. Correlation
When do they differ?
1D Example from Signal Processing
Correlation Convolution
Mask: w = 1 2 3 2 0 Mask: w' = 0 2 3 2 1
Signal: 0 0 0 1 0 0 0 0 Signal: 0 0 0 1 0 0 0 0
Zero padding Zero padding
0000000100000000 0000000100000000
Apply mask to first position Apply mask to first position
0000000100000000 0000000100000000
12320 02321
Produces a 0 for first out Produces a 0 for first out
Apply mask to 4th position Apply mask to 4th position
0000000100000000 0000000100000000
12320 02321
Produces a 0 Produces a 1
Final result: Final results:
000023210000 000123200000
Remove padding: Remove padding:
00232100 01232000
17
Convolution Examples
http://www.aishack.in/2010/08/image-convolution-examples/
18
Convolution Examples
19
Convolution Examples
20
Convolution Examples
21
Median filters
A Median Filter operates over a window by
selecting the median intensity in the window.
50 45 49
51 57 43 43 45 49 50 51 52 53 57 84
52 53 84
What advantage does a median filter have over a
mean filter?
Is a median filter a kind of convolution?
No, it’s called a nonlinear filter.
22
Comparison: salt and pepper noise
23
Comparison: Gaussian noise
24
More Comparisons:
Mean vs. Median using Matlab
http://www.mathworks.com/help/toolbox/images/f11-12251.html
25
Edge Detection
Basic idea: look for a neighborhood with
strong signs of change.
Problems: 81 82 26 24
82 33 25 25
• neighborhood size 81 82 26 24
• how to detect change
26
Differential Operators
Differential operators
• attempt to approximate the gradient at a
pixel via masks
• threshold the gradient to select the edge pixels
What’s a gradient?
Def: the gradient of a scalar function
f(x1, x2, ... xn) is denoted by ∇f (del f)
and is defined by:
∇f = (∂f/∂x1, ∂f/∂x2, . . . ∂f/∂xn)
What’s a derivative?
27
Example: Sobel Operator
-1 0 1 1 2 1
Sx = -2 0 2 Sy = 0 0 0
-1 0 1 -1 -2 -1
On a pixel of the image I
• let gx be the response to mask Sx
• let gy be the response to mask Sy
Then the gradient is
∇I = [gx gy]T
g = (gx2 + gy2 )1/2 is the gradient magnitude.
θ = atan2(gy,gx) is the gradient direction.
28
28
Sobel Operator on Blocks Image
original image gradient thresholded
magnitude gradient
magnitude
29
Some Well-Known Masks for
Computing Gradients
Sx Sy
-1 0 1 1 2 1
Sobel: -2 0 2 0 0 0
-1 0 1 -1 -2 -1
-1 0 1 1 1 1
Prewitt: -1 0 1 0 0 0
-1 0 1 -1 -1 -1
Roberts 0 1 1 0
-1 0 0 -1
30
