Unit: 3 Image Segmentation

(Similarity Based Techniques)

ET403:Principles of Image Processing

Image Segmentation

Group similar components (such as, pixels in an image, image frames in a video) to obtain a compact representation.

Applications: Finding tumors, veins, etc. in medical images, finding targets in satellite/aerial images, finding people in

surveillance images, summarizing video, etc.

Methods: Thresholding, K-means clustering, etc.

ET403:Principles of Image Processing

Image Segmentation

Segmentation algorithms for monochrome images generally are based on one of two basic properties of gray-scale values:

Discontinuity

The approach is to partition an image based on abrupt changes in

gray-scale levels.

The principal areas of interest within this category are detection of isolated points, lines, and edges in an image.

Similarity

The principal approaches in this category are based on

thresholding, region growing, and region splitting/merging.

ET403:Principles of Image Processing

Thresholding

Suppose that an image, f(x,y), is composed of light objects on a dark background, and the following figure is the histogram of the image.

image with dark background and a light object

Then, the objects can be extracted by comparing pixel values with a threshold T.

ET403:Principles of Image Processing

Thresholding
One way to extract the objects from the background is to select

a threshold T that separates object from background.

Any point (x,y) for which f(x,y) > T is called an object point; otherwise the point is called a background point.

g ( x, y )

1 if f ( x, y ) T 0 f f ( x, y ) T

When T is a constant applicable over an entire image, then the above process is called as Global thresholding.
ET403:Principles of Image Processing

Thresholding
When the value of T changes over an image

Then that process is referred as Variable thresholding.

Sometimes it is also termed as local or regional thresholding. Where, the value of T at any point (x,y) in an image depends on properties of a neighborhood of (x,y). If T depends on the spatial coordinates (x,y) themselves, then variable thresholding is often referred to as dynamic or adaptive thresholding.

ET403:Principles of Image Processing

Thresholding

ET403:Principles of Image Processing

Thresholding

ET403:Principles of Image Processing

Multilevel Thresholding

It is also possible to extract objects that have a specific intensity range using multiple thresholds.
image with dark background and two light objects

Extension to color images is straightforward: There are three color channels, in each one specify the intensity range of the object Even if objects are not separated in a single channel, they might be with all the channels Application example: Detecting/Tracking faces based on skin color
ET403:Principles of Image Processing

Multilevel thresholding
A point (x,y) belongs to an object class if T1 < f(x,y) T2

to another object class if f(x,y) > T2

to background if f(x,y)

T1

g ( x, y )

a b c

if f ( x, y ) T2 if T1 f ( x, y ) T2 if f ( x, y ) T1

ET403:Principles of Image Processing

Thresholding

ET403:Principles of Image Processing

Thresholding

ET403:Principles of Image Processing

Role of Noise in Image Thresholding

ET403:Principles of Image Processing

Role of Illumination in Image Thresholding

Non-uniform illumination may change the histogram in a way that it becomes impossible to segment the image using a single global threshold.

Choosing local threshold values may help.

ET403:Principles of Image Processing

Role of Illumination in Image Thresholding

ET403:Principles of Image Processing

Basic Global Thresholding

Based on visual inspection of histogram
1. 2. Select an initial estimate for T. Segment the image using T. This will produce two groups of pixels: G1 consisting of all pixels with gray level values > T and G2 consisting of pixels with gray level values 3. 4. 5. 6. regions G1 and G2 Compute a new threshold value T = 0.5 (
1+ 2)

T
1

Compute the average gray level values

and

for the pixels in

Repeat steps 2 through 4 until the difference between the values of T

in successive iterations is smaller than a predefined parameter T.

ET403:Principles of Image Processing

Basic Global Thresholding

Note: the clear valley of the histogram and the segmentation between object and background

Initially T= average intensity of image T0 = 0 3 iterations with result T = 125

ET403:Principles of Image Processing

Basic Global Thresholding

1. Works well in situations where there is a reasonably clear

valley between the modes of the histogram related to

objects and background. 2. 3. T is used to control the number of iterations. Initial threshold must be chosen greater than the minimum and less than the maximum intensity level in the image 4. The average intensity of the image is a good initial choice for T.

ET403:Principles of Image Processing

Basic Adaptive Thresholding

subdivide original image into small areas. utilize a different threshold to segment each subimages. since the threshold used for each pixel depends on the location of the pixel in terms of the subimages, this type of thresholding is adaptive.

ET403:Principles of Image Processing

Example : Adaptive Thresholding

ET403:Principles of Image Processing

Example : Adaptive Thresholding

How to solve this problem?

ET403:Principles of Image Processing

Further subdivision

ET403:Principles of Image Processing

Optimal Global and Adaptive Thresholding

This method treats pixel values as probability density functions. The goal of this method is to minimize the probability of misclassifying pixels as either object or background. There are two kinds of error: mislabeling an object pixel as background, and mislabeling a background pixel as object.

ET403:Principles of Image Processing

Optimal Global and Adaptive Thresholding

Method for estimating thresholds that produce the minimum average segmentation error. Let an image contains only two principal gray regions. Let z represents the gray-level values. These can be viewed as random quantities, and the histogram may be considered an estimate of their probability density function (PDF), p(z).

ET403:Principles of Image Processing

Optimal Global and Adaptive Thresholding

p( z )

P P2 p2 ( z ) 1 p1 ( z ) P P2 1 1

P1: probability that a random pixel with value z is an object pixel. P2: probability that a random pixel Is a background pixel

ET403:Principles of Image Processing

Probability of erroneously
T

E1 (T )
T

p2 ( z )dz p1 ( z )dz

E2 (T )

E(T )

P2 E1 (T ) P 1 E2 (T )

ET403:Principles of Image Processing

Minimum error
Differentiating E(T) with respect to T (using Leibnizs rule) and

equating the result to 0

dE (T ) dT
find T which makes

d ( P2 E1 (T ) P 1 E2 (T )) dT

P 1 p1 (T )

P2 p2 (T )

if P1 = P2 then the optimum threshold is where the curve p1(z) and p2(z) intersect

ET403:Principles of Image Processing

Gaussian density
Example: use PDF = Gaussian density : p1(z) and p2(z)

p1 ( z )
1

1 2 1

(z

1) 2 1

(z

p2 ( z )
2

2) 2 2

where 1 and 12 are the mean and variance of the Gaussian density of one object 2 and 22 are the mean and variance of the Gaussian density of the other object
ET403:Principles of Image Processing

Gaussian density
p1 (T )
1

1 2 1

(T

1) 2 1

(T

p2 (T )
2

2) 2 2

P 1 p1 (T )

P2 p2 (T )

ET403:Principles of Image Processing

Quadratic equation
AT B C
2

BT C A
1 2 2 2 2 2 2 2 1 2

0
2 2 2 1 2 1

where 2(
2 1

) 2
2 1

2 2

ln(

2 1

P/

1 2

P)

P2 ln P 1

if P1 = P2 or = 0 then the optimal threshold is the average of the means

ET403:Principles of Image Processing

Region-Based Segmentation

ET403:Principles of Image Processing

Region-Based Segmentation
BasicnFormulation (a) Ri R
i 1

(b) Ri is a connected region, i (c) Ri (e) P(Ri Rj Rj ) for all i and j, i TRUE for i FALSE for i (d ) P(Ri )

1, 2, ..., n j j 1, 2, ..., n

P(Ri) is a logical predicate property defined over the points in set Ri ex. P(Ri) = TRUE if all pixel in Ri have the same gray level
ET403:Principles of Image Processing

Region-Based Segmentation

Region Growing Region growing is a procedure that groups pixels or subregions into larger regions. The simplest of these approaches is pixel aggregation, which starts with a set of seed points and from these grows regions by appending to each seed points those neighboring pixels that have similar properties (such as gray level, texture, color, shape). Region growing based techniques are better than the edge-based techniques in noisy images where edges are difficult to detect.

ET403:Principles of Image Processing

Region-Based Segmentation

Region Growing Region growing is a procedure that groups pixels or subregions into larger regions. The simplest of these approaches is pixel aggregation, which starts with a set of seed points and from these grows regions by appending to each seed points those neighboring pixels that have similar properties (such as gray level, texture, color, shape). Region growing based techniques are better than the edge-based techniques in noisy images where edges are difficult to detect.

ET403:Principles of Image Processing

select all seed points with gray level 255

Region Growing
criteria: 1. the absolute graylevel difference between any pixel and the seed has to be less than 65 2. the pixel has to be 8connected to at least one pixel in that region (if more, the regions are merged)

ET403:Principles of Image Processing

Region Growing
criteria: 1. the absolute gray-level difference between any pixel and the seed has to be less than 65 2. the pixel has to be 8connected to at least one pixel in that region (if more, the regions are merged)

select all seed points with gray level 255

ET403:Principles of Image Processing

Histogram of fig 10.40 a)

used to find the criteria of the difference gray-level between each pixels and the seeds

ET403:Principles of Image Processing

Region-Based Segmentation

ET403:Principles of Image Processing

Region-Based Segmentation

ET403:Principles of Image Processing

Region-Based Segmentation

Region Splitting

Region growing starts from a set of seed points. An alternative is to start with the whole image as a single region and subdivide the regions that do not satisfy a condition of homogeneity. Region merging is the opposite of region splitting. Start with small regions (e.g. 2x2 or 4x4 regions) and merge the regions that have similar characteristics (such as gray level, variance). Typically, splitting and merging approaches are used iteratively.

Region Merging

ET403:Principles of Image Processing

Region splitting and merging

Quadtree
1. Split into 4 disjoint quadrants any region Ri for which P(Ri) = FALSE 2. Merge any adjacent region Rj and Rk for which P(Ri Rk ) = TRUE 3. Stop when no further merging or splitting is possible.
ET403:Principles of Image Processing

Example

P(Ri) = TRUE if at least 80% of the pixels in Ri have the property |zj-mi| where zj is the gray level of the jth pixel in Ri mi is the mean gray level of that region i is the standard deviation of the gray levels in Ri

2 i,

ET403:Principles of Image Processing

Example: Apply the split and merge technique to segment the image shown in fig. below.

Figure: Image R1 R2 R1 R3 R4 R2

ET403:Principles of Image Processing

Quadtree Representation

R1

R2

ET403:Principles of Image Processing

Example: Apply the split and merge technique to segment the image shown in fig. below.

Figure: Image

ET403:Principles of Image Processing

Solution:
R1

R2

R3

R4

Figure(a): Image
R21 R23 R31 R33 R32 R34 R22 R24

Figure(b): step-1

R1

R4

Figure(c): step-2

Figure(d): step-3

ET403:Principles of Image Processing

Quadtree Representation

R1
R21

R2
R24 R31

R3
R34

R4

R22

R23

R32

R33

ET403:Principles of Image Processing

Example: Apply the split and merge technique to segment the image shown in fig. below.

ET403:Principles of Image Processing

Example: Apply the split and merge technique to segment the image shown in fig. below.

R1

R2

R3

R4

ET403:Principles of Image Processing

More Complex Segmentation Methods

Snakes Level Sets Graph Cuts Generalized PCA

9/28/2013

Copyright G.D. Hager

ET403:Principles of Image Processing

Snakes

9/28/2013

Copyright G.D. Hager Images taken from ET403:Principles of Image http://www.cs.bris.ac.uk/home/xie/content.htm Processing

Level Sets

9/28/2013

Copyright G.D. Hager Images taken from ET403:Principles of Image http://www.cgl.uwaterloo.ca/~mmwasile/cs870/ Processing

Graph Cuts

9/28/2013

ET403:Principles of Image Processing

Copyright G.D. Hager Images taken from efficient graph-based segmentation paper

GPCA (Rene Vidal)

Human

GPCA

9/28/2013

Copyright G.D. Hager

ET403:Principles of Image Processing