Photogrammetric Computer Vision

Geometric Principles of
Sensor Orientation and Object Reconstruction

Lectures WS 2018/19
(Course notes for internal use only!)
 Overview

• Multi-view reconstruction
– Metric & Euclidean reconstruction
– Auto-calibration

• Robust parameter estimation

– M-estimators (Huber, Tukey)
– Least-Median-of-Squares (LMedS)
– RANdom SAmple Consensus
(RANSAC)

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 2

Euclidean & metric reconstruction

Using additional information

 Calibration methods
• Factory calibration
– Calibration of photogrammetric cameras using special equipment (patented)
– Cons: No flexibility concerning focusing, lens change and aperture setting
• Reference field calibration
– Known reference coordinates with high precision, sufficient depth expansion
and large number of points
– Spatial resection for an image bundle using additional parameters
– Cons: No mobility, choice of the test field and the focusing must be related to
the object expansion
• Simultaneous calibration
– The camera parameters are determined together with the exterior orientation
and the object point coordinates
• Self-calibration
– Calibration without object space information, only from images
• Auto-calibration
– Abbreviation for automatic self-calibration
© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 4
 Additional information
• Direct reconstruction
Knowledge of at least 5 control points in general position for the
determination of the transformation from projective into Euclidean
space
• Model-based reconstruction
Utilization of well-known object characteristics, e.g. parallelism,
angles or distances.
• Restriction of the exterior orientation
For special camera configurations, e.g. turntable, or special
camera movements
• Restriction of the interior orientation
Assumptions over well-known or constant parameters of interior
orientation. Flexible method for most applications

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 5

 Example: Projective reconstruction

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 6

 Use of additional information
Projective reconstruction

Scene information
Affine
• Parallel 3D lines Camera information
• Length ratios
(Auto-)calibration
Control points of parallel segments
Restriction of the
• Known object space
Affine reconstruction orientation:
coordinates Xi, i ≥ 5
• Known parameters
in general position Metric
• Constant parameters
• 3D angles
• Length ratios
of 3D segments

Euclidean reconstruction
© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 7
 Example: Direct Euclidean reconstruction

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 8

 Use of additional information
Projective reconstruction

Scene information
Affine
• Parallel 3D lines Camera information
• Length ratios
(Auto-)calibration
Control points of parallel segments
Restriction of the
• Known object space
Affine reconstruction orientation:
coordinates Xi, i ≥ 5
• Known parameters
in general position Metric
• Constant parameters
• 3D angles
• Length ratios
of 3D segments

Metric reconstruction
© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 9
 Example: Affine reconstruction

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 10

 Use of Additional Information
Projective reconstruction

Scene information
Affine
• Parallel 3D lines Camera information
• Length ratios
(Auto-)calibration
Control points of parallel segments
Restriction of the
• Known object space
Affine reconstruction orientation:
coordinates Xi, i ≥ 5
• Known parameters
in general position Metric
• Constant parameters
• 3D angles
• Length ratios
of 3D segments

Metric reconstruction
© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 11
 Required camera information
• Projective transformation: 15 DOF
• Metric transformation: 7 DOF
(3 translations, 3 rotations and 1 scaling)
• Remaining 15-7=8 constraints

• General equation for n images and k known and/or

c constant parameters:
n ⋅ k + ( n − 1) ⋅ c ≥ 8

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 12

 Examples for model assumptions
Orientation parameter s Number
Assumption
αx αy x0 , y0 γ s of images

Skew and aspect ratio constant - - - c c 5*

All parameter s of K constant c c c c c 3

Skew known - - - - k 8*
Skew and aspect ratio known - - - k k 4*
Principle point known - - k - - 4 (5 linear)
Skew and principle point known - - k - k 3 linear
Only principle distance unknown - - k k k 2 (3 linear)

All parameter s of K known k k k k k 2

k = known, c = constant, *multiple solutions possible

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 13

 Automatic self-calibration

• Wanted: Altogether 8 parameters

• 3 parameters describes the exterior orientation
and are defined by the ideal plane.
(With their knowledge an affine reconstruction
can be achieved)
• The projection of the absolute conic
define the remaining 5 parameters of the interior orientation.
(They make a metric reconstruction possible)

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 14

 Exterior orientation with ideal plane Π∞
• Under perspective projection parallel lines intersect in vanishing points
• The ideal plane Π ∞ is defined by three vanishing points
• The transformation H, which take this plane to infinity Π ∞ = [ 0 0 0 1],
T

produce an affine reconstruction

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 15

 Interior orientation with absolute conic Ω∞

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 16

 Image of the absolute conic ω

• Image of the Absolute Conic (IAC)

General position Without skewing Known Known

principle point aspect ratio

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 17

Robust parameter estimation
 Projective transformations

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 19

 Challenge
• Automatically finding correspondences:
– At the beginning of an image matching process
only the correlation of local descriptors is available
– Mismatches (outlier) cannot be avoided and must be removed

• Assumptions:
– We have a data set with putative feature correspondences
– A subset is consistent with some geometric relation (model)

• Task:
– Search for subsets of matches consistent with the model (inliers)
– Estimate the transformation parameters

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 20

 Matches Consistent with the Model

Putative matches using local image Robust estimated matches satisfying

descriptors (contain 42% outliers) the epipolar constraint (F-Matrix)

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 21

 Types of measurement errors
• Small errors (Noise)
– The localization accuracy of the coordinates may not be perfect
– For small deviations, a Gaussian error distribution can be assumed
– A non-linear optimization can be used

• Blunders (Outlier)
– A serious problem are blunders (i.e. wrong correspondences)
– They arise particularly in automatic measurements
– The influence of errors can be limited using M-Estimators or LMedS
– Hypothesize-and-verify approaches are more successful
• Identify a minimal solution by random trials,
which is supported by as much data as possible (RANSAC)
• Particularly insensitive to outliers,
but significant in computational cost

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 22

 Lever points
• Points, which lie far away from the mass of the data, could have a
substantial influence on the parameter estimation
• Using robust statistics, such outliers can be identified and their
negative influence on the solution can be excluded

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 23

 Find data consistent with a model

Strong disturbed line fitting data Inlier (green) with best line model (red)

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 24

 Influence functions
• Over-determined homogeneous equation system
Ax e, e ≠ 0
=
Error of the i-th observation: ei
• Least-Squares-Method:
cost function
C = ∑ ei2
i

• Problem: The sum of squared

errors ei is a sensitive measure
• Objective: Find a suitable
influence function

C = ∑ ρ ( ei )
i
© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 25
 M(aximum-Likelihood)-estimators
• Min-max-function of Huber
the influence is limited to a constant value:

=ρ ( e ) min ( t , max(e, −t ) )

• Function of Tukey
the influence reduces again
after a certain value:

e (t 2 - e 2 ) 2 , e <t
ρ (e) = 
 0, otherwise

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 26

 Automatic threshold
• χ 2
Thresholds t of the distribution (confidence 95%)

• Standard deviation (Root Mean Square error, RMS)

1 n
Average deviation from the mean e = ∑ ei
n i =1
1 n
∑ ( ei − e )
2
=σ
n i =1
© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 27
 Median
• Median
The median can be found by picking the middle element of the sorted errors
n
e1 ≤ ei ≤ en for i=
2
Example: median 1,3,700,3,1,1,
= 2 1,1,1,
= 2,3,3, 700 2

• Least-Median-of-Squares method (LMedS)

- Tolerates up to 50% outliers
- No threshold must be defined
C = median ei2
i

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 28

 RANSAC

Random trying
 Monte-Carlo method: RANSAC
• Classical robust methods
– First estimation using as much as possible data (with smoothing effect)
– Then detect and eliminate outliers
– New estimation with only consistent data

• RANdom SAmple Consensus (FISCHLER & BOLLES, 1981)

– Use as few as possible data for an estimation
– Random selection of a minimal set
– Maximization of consistent data, which confirm the solution
– High computational cost for n data and k wanted parameters

( n
k )= n!
k ! ( n−k )!
(Binomial coefficient)

with the factorial n ! = 1 ⋅ 2 ⋅  ⋅ ( n − 1) ⋅ n

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 30

 Example: RANSAC

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 31

 Random trying
• In a dataset with n points a minimum set with s points
is randomly selected
– Degenerated configurations (i.e. collinear points) should be avoided
– A good spatial point distribution prevents extrapolation problems
• For these s points a model is determined
• Possible optimization criteria:
– The number of consistent data to the solution is maximized

 1, e <t
ρ (e) = 
 0, otherwise
– or, a robust error function (e.g. LMedS) concerning all data is minimized
• The random selection and model estimation is repeated
until a suitable termination criterion is reached

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 32

 RANSAC related work
• RANSAC by Fishler & Bolles, 1981
• Examples for various improvements
– MLESAC (Maximum Likelihood Estimation SAC)
by TORR & ZISSERMAN, 2000
– MAPSAC (Maximum A Posteriori SAC) by TORR, 2002
– Preemptive RANSAC by NISTÉR, 2003
– Guided-MLESAC by TORDOFF & MURRAY, 2005
– PROSAC (PROgressive SAC) by CHUM & MATAS, 2005
– R-RANSAC (Randomized RANSAC) with SPRT
(Sequential Probability Ratio Test) by MATAS & CHUM, 2005
– Bail-out Test for RANSAC by CAPEL, 2005
– GASAC (Genetic Algorithm SAC) by RODEHORST, 2006
– QDEGSAC (RANSAC for (quasi-)degenerate data )
by FRAHM & POLLEFEYS, 2006
© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 33
 RANSAC for (quasi-)degenerate data

All matches

Degenerate Additional
RANSAC QDEGSAC
matches matches

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 34

 Merry Christmas & happy new year!

© Volker Rodehorst Lecture Photogrammetric Computer Vision - WS 18/19 35