Photogrammetry & Robotics Lab
Binary Images
and Commonly Used Operations
(connected components; distance transform; morphological operators)
Cyrill Stachniss
The slides have been created by Cyrill Stachniss.
1
Binary Image
§ So far, we considered grayscale
images where each pixel typically
takes values between 0 and 255
§ A binary image is
an image with a 1 bit
color depth
§ Each pixel is either
black or white
§ Either 0 or 1 (0 or 255)
2
Binary Image Examples
Handwritten digits
Scanned documents Image Courtesy: Leibe
Background Subtraction
- =
Image Courtesy: Jäggli
3
Connected Components
4
Connected Components
§ For several applications, we need to
identify which components are
connected
§ Example: Separation of characters
5
Connected Components
Two points A, B are connected if there
exists a path from A to B that goes only
through the same component
Image courtesy: Förstner 6
Neighborhoods on Grids
§ We need to define a neighborhood
relationship for pixels
§ Two popular neighborhood definitions
N4 neighborhood N8 neighborhood
7
N4 Neighborhood
§ The left, right, up, down pixels are
connected
§ Also called “city-block” neighborhood
or Manhattan neighborhood
8
N8 Neighborhood
§ The left, right, up, down and the
diagonal pixels
are connected
9
How to Determine Connected
Components?
Image courtesy: Förstner 10
Determining Connected
Components via a Graph
§ Build graphs of the foreground pixels
with edges according to N8 neighbors
§ Idea: Label the nodes in the graphs
Image courtesy: Förstner 11
Labeling Approach Informally
1. Select an unlabeled node and
assign a new label to it
2. For each unlabeled neighbor of a labeled
node, assign the same label
3. Repeat step 2 until all neighbors are
labeled
4. Repeat step 1 until all nodes are labeled
Also called: a “brushfire” approach
12
Connected Components
13
Example
2
1
1 1 1 1
1
14
Steps (outer loop, 1st iteration)
outer
loop 1
inner
loop 1
inner
loop 2
inner
loop 3
1
1 1 1 1
1
15
Steps (outer loop, 2nd iteration)
outer
loop 2
inner
loop 1
2
1
1 1 1 1
1
16
Properties of the Algorithm
§ Provides the connected components
§ Work in general graphs
§ Does not exploit the systematic
neighborhood of images
Is there any property we can exploit
to make the algorithm more efficient?
17
Labeling by Exploiting the
Grid Structure of the Image
Idea
§ Process the image in one pass
§ Generate a temporary label for a
foreground pixel based on the already
processed neighbors
§ In case of multiple labels for the same
component, use an equivalence table
18
Connected Components on Grids
§ Process the image from left to NOTE: Example uses
right, top to bottom: max to select the label
If the next pixel to process is 1
A) If none of its (top or left) neighbor
N4 is 1, assign new label.
B) If all neighboring (top or left) labels
are identical, copy the label
C) If the labels differ, copy the one
label (e.g., min or max) and update
the equivalence table.
Slide courtesy: J. Neira 19
Connected Components on Grids
§ Process the image from left to NOTE: Example uses
right, top to bottom: max to select the label
If the next pixel to process is 1
A) If none of its (top or left) neighbor
N4 is 1, assign new label.
B) If all neighboring (top or left) labels
are identical, copy the label
C) If the labels differ, copy the one
label (e.g., min or max) and update
the equivalence table.
§ Re-label with the smallest of equivalent
labels
§ Extends trivially for N8 neighborhoods
Slide courtesy: J. Neira 20
Connected Components for
Grids/Binary Images
21
Example (N8 Neighborhood)
Image courtesy: Förstner 22
Example (N8 Neighborhood)
Equivalence table: {1=3, 2=4=5}
Image courtesy: Förstner 23
Connected Components for
Grids/Binary Images
§ Exploits the grid neighborhood
§ Requires only one pass through the
image for labeling
§ Second pass to eliminate duplicate
labels
§ Linear complexity in the number of
foreground pixels
24
Distance Transform
25
Distance Transformation
Several problems require to compute
the distance from any pixel to the
border of the components
Examples
§ Nearest neighbor problems
§ Sensor models for range sensor
§ Map visualization
§ User interfaces
26
Distance Transformation
§ Distance transform on the foreground
pixels is defined as
border
§ with different distance functions
27
Examples
N4 N8
1 1 1 1 1 1
1 2 2 2 1 1 1 2 1 1
1 2 3 2 1 1 2 2 2 1
1 2 2 2 1 1 2 2 2 1
1 2 1 1 1 1 1 1 1 1
1 1 1 1
1 1
28
Distance Transformation
§ Distance transform can be computed
similar to the connected components
§ Two passes over the image
§ 1st: top-down, left-right
§ 2nd: down-up, right-left
§ Always store the minimum distance
29
First and Second Pass (N4)
First pass
1st pass dist. 2nd pass dist. 30
First and Second Pass (N4)
First pass Second pass
1 1 1
1 2 2 2 1
1 2 3 3 2
1 2 3 4 3
1 2 3 4 4
1 2
1
1st pass dist. 2nd pass dist. 31
First and Second Pass (N4)
First pass Second pass
1 1 1 1 1 1
1 2 2 2 1 1 2 2 2 1
1 2 3 3 2 1 2 3 2 1
1 2 3 4 3 1 2 2 2 1
1 2 3 4 4 1 2 1 1 1
1 2 1 1
1 1
1st pass dist. 2nd pass dist. 32
Analogous for N8 Neighborhood
First pass Second pass
1 1 1 1 1 1
1 1 2 1 1 1 1 2 1 1
1 2 2 2 1 1 2 2 2 1
1 2 3 2 1 1 2 2 2 1
1 2 3 2 1 1 1 1 1 1
1 2 1 1
1 1
1st pass dist. 2nd pass dist. 33
Examples for N4 and N8
Distances to the Center
N4 N8
Image courtesy: Förstner 34
N4 Distance Transformation
1. Initialization
2. 1st pass (top-left to bottom-right)
3. 2nd pass (bottom-right to top-left)
35
N8 Distance Transformation
1. Initialization
2. 1st pass (top-left to bottom-right)
3. 2nd pass (bottom-right to top-left)
36
N4 vs. N8 Neighborhood 1 1 1
1 2 2 2 1
§ The N4 neighborhood 1 2 3 2 1
overestimates the 1 2 2 2 1
Euclidean distance 1 2 1 1 1
§ The N8 neighborhood 1 1
N4
1
underestimates the
1 1 1
Euclidean distance 1 1 2 1 1
1 2 2 2 1
1 2 2 2 1
1 1 1 1 1
1 1
N8
1
37
N4 vs. N8 Neighborhood 1 1 1
1 2 2 2 1
§ The N4 neighborhood 1 2 3 2 1
overestimates the 1 2 2 2 1
Euclidean distance 1 2 1 1 1
§ The N8 neighborhood 1 1
N4
1
underestimates the
1 1 1
Euclidean distance 1 1 2 1 1
§ Can we efficiently 1 2 2 2 1
combine both? 1 2 2 2 1
1 1 1 1 1
1 1
N8
1
38
Combined Distance
§ The real cost of the diagonal is
§ If we approximate
§ The sum provides a better
approximation for twice the distance
§ Thus we can use the average distance
§ By using we can exploit
integer computations
39
Euclidian Distance Transform
§ Computing the real Euclidian distance
for every cell to the closest border is
more difficult
§ EDT in Python (scipy) available as
ndimage.morphology.distance_transform_edt
§ EDT in Matlab available as bwdist()
§ See, for example,: Breu et al., 1995
40
Morphological Operators:
Erosion and Dilation
41
Filtering
§ Binary images are often obtained
through thresholding (point operator)
§ Operator:
Image Courtesy: Whitaker 42
What We Have…
How to
Fix This?
Image Courtesy: Whitaker 43
What We Want…
Image Courtesy: Whitaker 44
Morphological Operators
§ Shrinking the foreground (“erosion”)
§ ︎Expanding the foreground (“dilation”)
§ ︎Removing holes in the foreground
(“closing”)
§ ︎Removing stray foreground pixels in
background (“opening”)
§ ︎…
45
Erosion
Change each foreground (here black)
pixel to a background (here white) pixel
if it has a background pixel as its N4
neighbor
46
Erosion Example
47
Erosion Example
48
Erosion Example
49
Dilation
Change each background (here while)
pixel to a foreground (here black) pixel
if it has a foreground pixel as its N4
neighbor
50
Dilation Example
51
Dilation Example
52
Dilation Example
53
Erosion and Dilation
§ Erosion shrinks the foreground
§ Erosion removes foreground outliers
§ Dilation expands the foreground
§ Dilation fills holes
Can we combine both to improve
segmentation and masking?
54
Opening and Closing
Opening
§ Removing stray foreground pixels in
background
§ Step 1: erosion; Step 2: dilation
Closing
§ Remove holes in the foreground
§ Step 1: dilation; Step 2: erosion
55
Original Image
Image Courtesy: Whitaker 56
Opening
Image Courtesy: Whitaker 57
Closing (After Opening)
Image Courtesy: Whitaker 58
Opening + Closing Result
Image Courtesy: Whitaker 59
Morphological Operators
§ Morphological Operators can be used
to eliminate noisy masks
§ Combination of Opening and Closing
removes stray pixels and fills holes
§ Erosion and dilation are local operators
60
Summary
§ Binary images are relevant tools in
several image processing applications
§ Connected components
§ Distance transforms
§ Erosion, dilation, opening, and closing
61
Literature
§ Szeliski, Computer Vision: Algorithms
and Applications, Chapter 3.3
§ Förstner, Scriptum Photogrammetrie I,
Chapter 6
62
Slide Information
§ The slides have been created by Cyrill Stachniss as part of the
photogrammetry and robotics courses.
§ I tried to acknowledge all people from whom I used
images or videos. In case I made a mistake or missed
someone, please let me know.
§ The photogrammetry material heavily relies on the very well
written lecture notes by Wolfgang Förstner and the
Photogrammetric Computer Vision book by Förstner & Wrobel.
§ Parts of the robotics material stems from the great
Probabilistic Robotics book by Thrun, Burgard and Fox.
§ If you are a university lecturer, feel free to use the course
material. If you adapt the course material, please make sure
that you keep the acknowledgements to others and please
acknowledge me as well. To satisfy my own curiosity, please
send me email notice if you use my slides.
