IDC410

A course on Image Processing and

Machine Learning
(Lecture 05)

Shashikant Dugad,
IISER Mohali

Shashikant R Dugad, IISER Mohali 1

 ..)
td
on
(c
r s
li te
F

Shashikant R Dugad, IISER Mohali 2

 Properties of Convolution Filter
● Notation: b = c ★ a
● Convolution is a multiplication-like operation
○ Commutative: a ★ b = b ★ a
○ Associative: a ★ (b ★ c) = (a ★ b) ★ c
○ Distributes over addition: a ★ (b + c) = (a ★ b) + (a ★ c)
○ Scalars factor out: αa ★ b = a ★ αb = α(a ★ b)
○ Identity: unit impulse e = […, 0, 0, 1, 0, 0, …]: a ★ e = a

● Usefulness of associativity
○ Often apply several filters one after another: (((a ★ b1) ★ b2) ★ b3)
○ This is equivalent to applying one filter: a * (b1 ★ b2 ★ b3)

Shashikant R Dugad, IISER Mohali 3

 Padding and Strides
● In Convolutional Neural Networks (CNNs), stride refers to the step size by which the
filter/kernel moves across the input image during the convolution operation in
horizontal and vertical direction
○ Stride defines how big of steps filters should take (i.e., how many pixels our filters should
skip ) while sliding over the image
○ Minimum value of stride = 1
○ Smaller strides (1 or 2) offer detailed feature extraction, while larger strides (3+) helps in
down-sampling.

● CNN is like a collection of small, overlapping magnifying glasses

called filters. These filters scan over different parts of a image to find interesting
features, like edges, shapes, or colours. These filters slide or convolve over the
entire image as defined by the stride.

● The kernel size, stride value can substantially reduce the size of output im the
output size and computational efficiency of the network, influencing feature
extraction and spatial dimensions of image.
Shashikant R Dugad, IISER Mohali 4
Movement of Filter

Stride = 1

Stride = 2

Shashikant R Dugad, IISER Mohali 5

 Image Padding
Output Image (7 x 7) [Pad = 1]
Input Image (5 x 5)
0 0 0 0 0 0 0
22 145 23 167 67 0 22 145 23 167 67 0
● Padding means adding extra columns
45 110 45 119 29 Pad = 1 0 45 110 45 119 29 0 and rows with ZERO pixel intensity
78 99 78 88 112 0 78 99 78 88 112 0
before doing any operations.
145 100 99 38 164 0 145 100 99 38 164 0
223 110 23 45 29 0 223 110 23 45 29 0
0 0 0 0 0 0 0 ● Helps in keeping the spatial info
intact, particularly prevents data loss
Output Image (9 x 9) [Pad = 2] at the edges hence stabilises training
Input Image (5 x 5) 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
● It also helps to keep the output size
22 145 23 167 67 0 0 22 145 23 167 67 0 0
consistent with the input and makes
45 110 45 119 29 Pad = 2 0 0 45 110 45 119 29 0 0
78 99 78 88 112 0 0 78 99 78 88 112 0 0 training more stable.
145 100 99 38 164 0 0 145 100 99 38 164 0 0
223 110 23 45 29 0 0 223 110 23 45 29 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0

Shashikant R Dugad, IISER Mohali 6

 Padding and Strides
Padding & Stride in CNN
● Image Dimension: Nrow X Mcol
● Padding: P Strides: Srow, Scol. Dilation: Drow, Dcol

● Kernal Size: Krow, Kcol

𝑵𝒊𝒏
𝒓𝒐𝒘 ' 𝟐 𝑿 𝑷 , 𝑫𝒓𝒐𝒘 𝑿 (𝑲𝒓𝒐𝒘 ,𝟏) ,𝟏
𝑵𝒐𝒖𝒕
𝒓𝒐𝒘 = 𝑺𝒓𝒐𝒘
+1

𝑴𝒊𝒏
𝒄𝒐𝒍 ' 𝟐 𝑿 𝑷 , 𝑫𝒄𝒐𝒍 𝑿 (𝑲𝒄𝒐𝒍 ,𝟏) ,𝟏
𝑴𝒐𝒖𝒕
𝒄𝒐𝒍 = 𝑺𝒄𝒐𝒍
+1

padding=1, stride=2

Shashikant R Dugad, IISER Mohali 7

 e s
g
Ed

Shashikant R Dugad, IISER Mohali 8

 Edges
● Feature extraction is one of the crucial steps in image processing and machine learning

● Edges in the image is one such key feature.

○ Edges typically occur on the boundary between two different regions in an image
○ An edge in an image is usually associated with a discontinuity in the image intensity
resulting large difference or a large amplitude of the first derivative of the image intensity

● Discontinuities in the image intensity can be either

○ Step discontinuities: the intensity abruptly changes from one value on one side of
the discontinuity to a different value on the opposite side
○ Line discontinuities: the intensity abruptly changes value but then returns to the
starting value within some short distance.

● However, step and line edges are rare in real images due to the low-frequency components
or the smoothing of the image.

● Step edges become ramp edges and line edges become roof edges

Shashikant R Dugad, IISER Mohali 9

5. EDGE DETECTION 5. EDGE141 Step
DETECTION 141

Edge Profile and Gradient

5. EDGE DETECTION 141
Step Theoretical edges
Step
Ramp Real edges

Step Ramp
Line
Ramp

Line
Ramp Roof
Line

Line Roof
● An edge
Roofis
associated with the maxima in the first derivative (gradient) ofedge
Figure 5.1: One-dimensional intensity
profiles. in
local region of an image
Figure 5.1: One-dimensional edge profiles.
● The gradient
Roof is the two-dimensional equivalent of the first derivative and is defined as
Figure 5.1: One-dimensional edge profiles. It is also possible for an edge to have both step and line characteristi
the vector For example, a surface that changes orientation from one flat surface
anotherIt is will
alsoproduce
possible afor step
an edge;
edge tobuthave
if the surface
both has line
step and a specular compon
characteristics.
Figure 5.1: One-dimensional edge profiles. ofFor
reflectance
example, and 𝝏𝒇
a if the
surface surface
that corner
changes is rounded,
orientation there
from can
one be
flata highlight
surface to d
It is also possible for an edge to have both step and line characteristics.
For example, a surface that changes orientation from one flat
𝑮the
toanother specular
𝒙surface to 𝝏𝒙
component
will produce as the
a step edge; butsurface orientation
if the surface of the rounded
has a specular component cor
𝑮(𝒇 𝒙, 𝒚) = to component
another will produce a step edge; but if the surface has a specular
passes =
of reflectance
the and ifangle
precise
the specular
the surface
component
for corner is reflection.
specular
as thea surface
rounded, there
The
orientation
can be aprofile
edge highlight
of the rounded line
due
genera
cornered
by such a situation looks like step edge with a superimposed
It is also possible for an edge to have both step and line characteristics.
of reflectance and if the surface corner is rounded, there can be aThere 𝑮surface
highlight
passes
𝒚 are thedue 𝝏𝒇
precise angle for specularwithreflection.
changes The edge profile generated
For example,
speculara component
surface that aschanges orientation from ofone
the flat toalso edges associated in the first derivative of
to the
another willprecise
produce a stepfor
the surface
edge; but
orientation
if reflection.
the surface The
has aedge
specular
rounded
by such corner
a
image intensity.
component 𝝏𝒚For example, mutual reflection from the sides of a10conc
situation looks like a step edge with a superimposed line edge.
Shashikant
passes the R Dugad,
angle IISER Mohali
specular There
profile are
generated
corner generate also edges associated
roof edges. Edgeswithare
changes in theimage
important first derivative of theth
features since
of reflectance and if the surface corner is rounded, there can be a highlight due
image intensity. For example, mutual reflection from the sides of a concave
 Gradient
● The vector G[f(x, y)] points in the direction of the maximum rate of
increase of the function f(x, y)

● The magnitude of the gradient, given by

|𝑮(𝒇(𝒙, 𝒚)| = 𝑮𝟐𝒙 + 𝑮𝟐𝒚

● The direction of the gradient is defined as,

𝑮𝒚
𝜶 𝒙, 𝒚 = tan$𝟏
𝑮𝒙

11
Shashikant R Dugad, IISER Mohali
 Gradient for digital image
● Gradient for a digital image can be defined as,

𝑮𝒙 𝒊, 𝒋 ≅ 𝒇 𝒊, 𝒋 + 𝟏 − 𝒇 𝒊, 𝒋 . 𝑮𝒚 𝒊, 𝒋 ≅ 𝒇 𝒊 + 𝟏, 𝒋 − 𝒇 𝒊, 𝒋
● These can be implemented with simple convolution masks as shown
below:
+1
Gx = -1 +1 Gy =
-1

● Gradient has to be computed at exactly the same position in space.

○ However, gradients Gx and Gy are calculated at different points, [i, j+ 1/2] and
[i+1/2, j] which are NOT the same
○ 3x3 convolution mask is preferred to maintain this criteria

12
Shashikant R Dugad, IISER Mohali
 Edge Detection Operators

● Roberts Operator: Provides a convolution mask for following simple gradient

operation: G[f(i, j)] = Gx + Gy = |f(i, j) - f(i+1, j+1)| + |f(i+1, j) - f(i, j+1)|

1 01 0 0 -10 -1
● Gx = Gy =
0 -10 -1 1 01 0
● The Roberts operator is NOT located at the desired point [i,j].
● In order to have an edge operation at fixed point [i,j] in both the directions, we
should have a mask of 3x3 size
● Sobel operator is the magnitude of the gradient computed by,

𝑴= 𝑺𝟐𝒙 + 𝑺𝟐𝒚

Shashikant R Dugad, IISER Mohali 13

 Edge Detection Operators
● Partial derivative are defined as per following matrix:

a0 a1 a2 Sx = (a2-a0) + c(a3-a7) + (a4-a6) Higher weightage can be given to closer

a7 [i, j] a3 pixels by appropriately choosing c
Sy = (a0-a6) + c(a1-a5) + (a2-a6)
a6 a5 a4
● For Sobel operator c = 2; Sx and Sy can be implemented using the following
convolution masks:
-1 0 1 1
-1 02 1 1 2 1
Sx = -2 0 2 0
-2 0 20Sy = 0 0 0
-1 0 1 -1 -2
0 -1
1 -1 -2 -1

● For Prewitt Operator c = 1: No emphasis is given to the closer pixels! Edge

operators obtained with c = 1

Shashikant R Dugad, IISER Mohali 14

 Application of Edge Detection Technique

● Direct application of edge operator on a image may result in fake edges

due to the noise in image
● First remove noise in the image
● Choose one of the edge detection operator appropriately and apply it on
the image
● Obtain gradient for each pixel and apply a threshold on the gradient value
to identify the pixel as a pixel representing edge

Image after Obtain gradient Gradient Image with

Original Noise noise Edge value for each Threshold Edges
Image Filter Operator
reduction pixel

Shashikant R Dugad, IISER Mohali 15

 Edge Detection with and without
152
Noise Filter
CHAPTER 5. EDGE DETECTION
5.3. SECOND DERIVATNE OPERATORS 153

(b)
(a) (b)

(d) (e)
(c) (d)

f'- - ~

~CJ~
\
'

,.
":~
\'.

(f) (g)
(e) (f)
Figure 5.6: A comparisonof various edge detectors on a noisy image. (a)
Figure 5.7: A comparison of various edge detectors on a noisy image without Noisy image. (b) Filtered image. (c) Simple gradient using 1 x 2 and 2 x 1
filtering. (a) Noisy image. (b) Simple gradient using 1 x 2 and 2 x 1 masks, masks, T = 32. (d) Gradient using 2 x 2 masks, T = 64. (e) Roberts
T = 64. (c) Gradient using 2 x 2 masks, T = 128. (d) Roberts cross operator, cross operator, T = 64. (f) Sobel operator, T = 225. (g) Prewitt operator,
T = 64. (e) Sobel operator, T = 225. (f) Prewitt operator, T = 225. T = 225.
https://cse.usf.edu/~r1k/MachineVisionBook/MachineVision.files/
Shashikant R Dugad, IISER Mohali 16
 Edge Detection with Second Derivative Operators

● A single derivate edge operators with a threshold provides too many edge points
depending on noise in the image
● A better approach would be to find only the points that have local maxima in
gradient values and consider them a edge points.
○ This means that at edge
CHAPTER points,
5. EDGE there will be a peak in the first derivative and,
DETECTION
equivalently, there will be a zero crossing in the second derivative.

If a threshold is used for detection of edges, all points

f (x, y)
x between a and b will be marked as edge pixels. However,
threshold by removing points that are not a local maximum in the
f'(x, y)
first derivative, edges can be detected more accurately.
x
a b Local maximum in the first derivative corresponds to a
"
f ( x,y) zero crossing in the second derivative.
x

Shashikant R Dugad, IISER Mohali 17

 Edge Detection with Second Derivative Operators
● There are two operators in two dimensions that correspond to the second
derivative: the Laplacian and second directional derivative.

𝝏𝒇(𝒙, 𝒚)
𝑮𝒙 = = 𝒇 𝒊, 𝒋 + 𝟏 − 𝒇 𝒊, 𝒋
𝝏𝒙
𝝏𝟐 𝒇 𝒙, 𝒚 𝝏𝑮𝒙 𝝏𝒇 𝒊, 𝒋 + 𝟏 𝝏𝒇 𝒊, 𝒋
∴ = = − = 𝒇 𝒊, 𝒋 + 𝟐 − 𝒇 𝒊, 𝒋 + 𝟏 − 𝒇 𝒊, 𝒋 + 𝟏 + 𝒇 𝒊, 𝒋
𝝏𝒙𝟐 𝝏𝒙 𝝏𝒙 𝒅𝒙

𝝏𝟐 𝒇 𝒙, 𝒚
= 𝒇 𝒊, 𝒋 + 𝟐 − 𝟐𝒇 𝒊, 𝒋 + 𝟏 + 𝒇(𝒊, 𝒋)
𝝏𝒙𝟐
● However, this approximation is centered about the pixel [i,j + 1]. Therefore, by
replacing j with j - 1,
𝝏𝟐 𝒇 𝒙, 𝒚 𝝏𝟐 𝒇 𝒙, 𝒚
= 𝒇 𝒊, 𝒋 + 𝟏 − 𝟐𝒇 𝒊, 𝒋 + 𝒇 𝒊, 𝒋 − 𝟏 . = 𝒇 𝒊 + 𝟏, 𝒋 − 𝟐𝒇 𝒊, 𝒋 + 𝒇(𝒊 − 𝟏, 𝒋)
𝝏𝒙𝟐 𝝏𝒚𝟐

Shashikant R Dugad, IISER Mohali 18

 Second Derivative Laplacian Operators

● By combining these two equations into a single operator, the following mask
can be used to approximate the Laplacian:
0 0 0 0 1 0 0 0 0 00 11 00
∇2 =1 ∇2-2x + 1∇2y = 0 -2 0 + 1 -2 1 = 01 -2-4 01
0 0 0 0 1 0 0 0 0 00 11 00

● It is desired to give weight to the corner pixels as below:

1 4 1
∇2 = 4 -20 4
1 4 1

Shashikant R Dugad, IISER Mohali 19

 Laplacian of Gaussian (LoG) Operators
● Very small local peaks in the first derivative will also result in zero
crossings the double derivative operators are quite sensitive to the noise

● The Laplacian operators has to be used in conjunction with powerful

filtering methods

● Laplacian operator combined Gaussian filter referred as LoG operator

● The detection criterion: presence of a zero crossing in the second derivative

with a corresponding large peak in the first derivative.

Shashikant R Dugad, IISER Mohali 20

 Laplacian of Gaussian (LoG) Operators

● The output of the LoG operator, h(x, y), is obtained by the convolution
operation

● h(x,y) = ∇2[g(x,y)★f(x,y)] = [∇2g(x,y)]★f(x,y)

𝒙𝟐 %𝒚𝟐
𝒙𝟐 + 𝒚𝟐 − 𝟐𝝈𝟐 $ 𝟐𝝈𝟐
160 𝜵𝟐 𝒈 𝒙, 𝒚 = 𝒆
CHAPTER 5. EDGE DETECTION
𝝈𝟒
5.4. LAPLACIAN OF GAUSSIAN 159
● 2 ∇ g(x,y) is offerent referred as Mexican Hat operator as shown in Figure
0.5
5 x 55x5
Laplacian of Gaussian mask
LoG Convolution Mask
04

0.3
0 0 -1 0 0
0.2
0 -1 -2 -1 0
16 -2 -1
0.1

0.1
-1 -2
.().1
10
0 -1 -2 -1 10
0
0 0 -1 0 0
-4>._\0 -8 -6 ... -2 10

Shashikant R Dugad, IISER Mohali 21

Figure 5.11: The inverted Laplacian of Gaussian function, (j = 2, in one and
 Laplacian of Gaussian (LoG) Operators
● Zero crossings may happen due to the noisy region of image

● The slope of the zero crossing depends on the contrast or sharpness of the
change in image intensity across the edge.

● To obtain real edges in an image, it may be necessary to combine

information from operators with several filter sizes or look at the amplitude
of variation (1st derivative)

● A larger σ results in better noise filtering but may lose important edge
information, which may affect the performance of an edge detector. If a
small filter is used, there is likely to be more noise due to insufficient
averaging.

Shashikant R Dugad, IISER Mohali 22

 Laplacian of Gaussian (LoG) Operators
● The LoG operator which is symmetric; can reduce noise by smoothing
the image, but it also dilutes the real edges resulting in uncertainty to the
accurate location of the edge

● The gradient may have greater sensitivity to the presence of edges, but it
has higher sensitivity to the noise.

● There is a trade-off between noise suppression, edge determination and

localization.

● The linear operator that provides the best compromise between noise
immunity and localization, is the first derivative of a Gaussian.

Shashikant R Dugad, IISER Mohali 23