0% found this document useful (0 votes)

93 views92 pages

Shape Recognition Techniques

This document discusses shape matching techniques for object recognition. It covers the Hausdorff distance method, which compares shapes using the distance between points. The document also introduces shape contexts, which solve for correspondences between points on shapes to estimate an aligning transform. Finally, it outlines hierarchical and machine learning approaches to shape matching.

Uploaded by

bunny

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

93 views92 pages

Shape Recognition Techniques

Uploaded by

bunny

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 92

Shape Matching

Intro Shape-Based Recognition

• Humans can recognize many objects

based on shape alone
• Fundamental cue for many object
categories
• Invariant to photometric variation.

Slide from Pedro Felzenszwalb

Intro Shapes vs. Intensity Values

Similar to a human in terms of shape,

but very different in terms of pixel
values.

Images from Belongie et al.

Intro Applications

• Shape retrieval
• Recognizing object categories
• Fingerprint identification
• Optical Character Recognition (OCR)
• Molecular-biology

Western 1909
Intro Geometric Transformations

• Often in matching images are allowed to

undergo some geometric transformation
• Related but not identical shapes can be
deformed into alignment using simple
coordinate transformations
• Find the transformations of one image that
produce good matches to the other image

Images from Belongie et al.

Intro Biological Shape
• D’Arcy Thompson: On Growth and Form, 1917
d • studied transformations between shapes of organisms

0 1 2 3 4 5
Fig. 177. Human skull Fig. 179. Skull of chimpanzee. Fig. 180. Skull of baboon.

Slide from Belongie et al.

Intro Related Problems

• Shape representation and decomposition

• Finding a set of correspondences between
shapes
• Transforming one shape into another
• Measuring the similarity between shapes
• Shape localization and model alignment
• Finding a shape similar to a model in a
cluttered image

Slide from Pedro Felzenszwalb

Intro References

• Shape Matching and Object Recognition Using Shape Contexts, by S. Belongie,

J. Malik, and J. Puzicha. Transactions on Pattern Analysis and Machine Intelligence
(PAMI), 2002.

• Recognizing Objects in Adversarial Clutter: Breaking a Visual CAPTCHA, by G.

Mori and J. Malik, in Proceedings IEEE Computer Vision and Pattern Recognition
(CVPR), 2003.

• Using the Inner-Distance for Classification of Articulated Shapes, by H. Ling and

D. Jacobs, Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), 2005.

• Comparing Images Using the Hausdorff Distance, by D. Huttenlocher, G.

Klanderman, and W. Rucklidge, Transactions on Pattern Analysis and Machine
Intelligence (PAMI), 1993.

• A Boundary-Fragment-Model for Object Detection, by A. Opelt, A. Pinz, and A.

Zisserman, Proceedings of the European Conference on Computer Vision (ECCV),
2006.

• Hierarchical Matching of Deformable Shapes, by P. Felzenszwalb and J. Schwartz,

in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
2007
Hausdorff
Distance
Outline

• Shape Distance and Correspondence

➢ Hausdorff Distance
• Shape Context
• Inner Distance
• Hierarchical Approach
• Hierarchical Matching
• Machine Learning Approach
• Boundary Fragment Model
Hausdorff
Distance

Comparing Images Using the Hausdorff

Distance
1993
D. Huttenlocher, G. Klanderman, and W. Rucklidge
Hausdorff
Distance
Overview

• Use Hausdorff distance to compare

images to a model
• Fast and simple approach
• Tolerant of small position errors
• Model is only allowed to translate with
respect to the image
• Can be extended to allow rotation and
scale
Hausdorff
Distance
Hausdorff Distance

• A means of determining the resemblance

of one point set to another

• Examines the fraction of points in one set

that lie near points in the other set

H (A; B) = max fh (A; B) ; h (B; A)g

½ ¾
h (A; B) = max min fd (a; b)g
a2A b2B
Hausdorff
Distance
Example

a2
a1
Given two sets of points
A and B, find h(A,B)
b3

b2
Hausdorff
Distance
Example

a2
a1
Compute the distance
between a1 and each bj
b3

b2
Hausdorff
Distance
Example

a2
a1
Keep the shortest

b2
Hausdorff
Distance
Example

a2
a1
Do the same for a2

b2
Hausdorff
Distance
Example

a2
a1
Find the largest of
these two distances
b3

b2
Hausdorff
Distance
Example

a2
a1
This is h(A,B)

b2
Hausdorff
Distance
Example

a2
a1
This is h(B,A)

b2
Hausdorff
Distance
Example

a2
a1
H(A,B) = max(h(A,B),h(B,A))

b2
Hausdorff
Distance
Example

a2
a1
This is H(A,B)

b2
Hausdorff
Distance
Generalization

• Hausdorff distance is very sensitive to

even one outlier in A or B
• Use kth ranked distance instead of the
maximal distance
• Match if hk (A; B) < ±
• k is how many points of the model need to be
near points of the image
• ± is how near these points need to be
½ ¾
hk (A; B) = kth min fd (a; b)g
a2A b2B
Hausdorff
Distance
Distance Transforms

• Processing can be sped up by probing a

precomputed Voronoi surface
• A Voronoi surface defines the distance
from any location in B to the nearest point
• Can be efficiently computed using dynamic
programming in linear time
Hausdorff
Distance
Example: Matching

Model

Edges

Match
Hausdorff
Distance
Example: Matching

Model Model

Image Edges

Match
Shape
Context Outline

• Shape Distance and Correspondence

• Hausdorff Distance
➢ Shape Context
• Inner Distance
• Hierarchical Approach
• Hierarchical Matching
• Machine Learning Approach
• Boundary Fragment Model
Shape
Context

Shape Matching and Object Recognition

Using Shape Contexts
2002
S. Belongie, J. Malik, and J. Puzicha
Shape
Context Overview

1) Solve for correspondences between points

on the two shapes
● Using shape contexts
2) Use the correspondences to estimate an
aligning transform
● Using regularized thin-plate splines
3) Compute the distance between the two
shapes
Shape
Context Related Work: Deformable Templates

• The Representation and Matching of Pictorial Structures, by

Fischler & Elschlager (1973)
• Structural image restoration through deformable templates,
by Grenander et al. (1991)
• Deformable Templates for Face Recognition, by Yuille (1991)
• Distortion invariant object recognition in the dynamic
linkarchitecture, by von der Malsburg (1993)

Slide from Belongie et al.

Shape
Context Sampling Points

• A shape is represented by a set of points

sampled from the edges of the object
Shape
Context Shape Context: Log-Polar Histograms

Count the number of points Count = 4

inside each bin.

Count = 12

Slide from Belongie et al.

Shape
Context Example: Shape Contexts

a) b)

c) d)

Images from Belongie et al.

Shape
Context Point Correspondences

• Compute matching costs C (pi ; pj ) using

Chi Squared distance:
K
1 X [hi (k) ¡ hj (k)]
2
C (pi ; pj ) =
2 hi (k) + hj (k)
k=1

• Minimize the total cost of matching, such

that matching is 1-to-1
X ¡ ¢
H (¼) = C pi ; q¼(i)
i
[Jonker & Volgenant, 1987]

Slide from Belongie et al.

Shape
Context Example: Point Correspondences

a) b)

c)
Shape
Context Thin Plate Spline Model

• The name “thin plate spline” refers to a

physical analogy involving the bending of a
thin sheet of metal
• The 2D generalization of the 1D cubic
spline
• Contains the affine model as a special
case
Shape
Context Minimizing Bend Energy

• The Thin Plate Spline interpolation has the

form: n
X
f (x; y) = a1 + ax x + ay y + wi U (jj (xi ; yi ) ¡ (x; y) jj)
i=1
| {z } | {z }
global affine transform local non-linear transformations

where, U (r) = r2 log r2

• Select a and w to minimize the bend

energy: Z Z µ ¶ µ ¶ µ ¶
2 2 2
@ 2f @2f @ 2f
I (f ) = +2 +2 dxdy
R2 @x2 @x@y @y 2
Shape
Context Example: Matching and Transformation

Images from Belongie et al.

Shape
Context Terms in Similarity Score

• Shape Context difference, Dsc

• Local Image appearance difference, Dac
• Orientation
• Gray-level correlation in Gaussian window
• … (many more possible)
• Bending energy, Dbe

Dsc + 1:6 ¤ Dac + 0:3 ¤ Dbe

Shape
Context Shape Context Results
Query Similarity Scores

0.086 0.108 0.109

0.066 0.073 0.077

0.046 0.107 0.114

0.117 0.121 0.129

0.096 0.147 0.153

Images from Belongie et al.

Inner
Distance Outline

• Shape Distance and Correspondence

• Hausdorff Distance
• Shape Context
➢ Inner Distance
• Hierarchical Approach
• Hierarchical Matching
• Machine Learning Approach
• Boundary Fragment Model
Inner
Distance

Using the Inner-Distance for Classification

of Articulated Shapes
2005
H. Ling and D. Jacobs
Inner
Distance Overview

• Its difficult to capture the part structure of

complex shapes with existing shape
matching methods
• Replace euclidean distance with the inner-
distance
• Insensitive to shape articulations
• Often more discriminative for complex shapes
• An extension to shape contexts
Inner
Distance Model of Articulated Objects

1) An object can be decomposed into a

number of parts
2) Junctions between parts are relatively
small with respect to the parts they
connect
3) Articulation on the object is rigid with
respect to any part, but can be non-rigid on
the junctions
4) An object that has been articulated can be
articulated back to its original form

Images from Ling and Jacobs

Inner
Distance The Inner-Distance

• The length of the shortest path between

landmark points within the shape
silhouette
• For convex shapes, the inner-distance
reduces to the Euclidean distance
• Inner-Distance changes only due do
deformations of the junctions

Images from Ling and Jacobs

Inner
Distance Inner-Distance vs Euclidean Distance

Images from Ling and Jacobs

Inner
Distance Computing the Inner-Distance

1) Build a graph on the sampled points

● For each pair of points x,y.
1. If line segment between them existed entirely
within the object
2. Build an edge connecting x and y with weight
w = jjx ¡ yjj