lOMoARcPSD|49979752

CV UNIT III - unit 3

Computer Vision (Anna University)

Scan to open on Studocu

Studocu is not sponsored or endorsed by any college or university

Downloaded by code breakers ([email protected])
 lOMoARcPSD|49979752

UNIT III
FEATURE-BASED ALIGNMENT & MOTION ESTIMATION
2D and 3D feature-based alignment - Pose estimation - Geometric intrinsic
calibration - Triangulation - Two-frame structure from motion - Factorization
- Bundle adjustment - Constrained structure and motion - Translational
alignment - Parametric motion - Spline-based motion - Optical flow -
Layered motion.

1. 2D and 3D feature-based alignment:

2D and 3D Feature-Based Alignment

Feature-based alignment is a technique used in computer vision and image processing

to align or match corresponding features in different images or scenes. The alignment
can be performed in either 2D or 3D space, depending on the nature of the data.

2D Feature-Based Alignment:
● Definition: In 2D feature-based alignment, the goal is to align and match
features in two or more 2D images.
● Features: Features can include points, corners, edges, or other distinctive
patterns.
● Applications: Commonly used in image stitching, panorama creation,
object recognition, and image registration.
3D Feature-Based Alignment:

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

● Definition: In 3D feature-based alignment, the goal is to align and match

features in three-dimensional space, typically in the context of 3D
reconstruction or scene understanding.
● Features: Features can include keypoints, landmarks, or other distinctive
3D points.
● Applications: Used in 3D reconstruction, simultaneous localization and
mapping (SLAM), object recognition in 3D scenes, and augmented reality.
Techniques for 2D and 3D Feature-Based Alignment:
● Correspondence Matching: Identifying corresponding features in different
images or 3D point clouds.
● RANSAC (Random Sample Consensus): Robust estimation technique to
find the best-fitting model despite the presence of outliers.
● Transformation Models: Applying transformation models (affine,
homography for 2D; rigid body, affine for 3D) to align features.
● Iterative Optimization: Refining the alignment through iterative
optimization methods such as Levenberg-Marquardt.
Challenges:
● Noise and Outliers: Real-world data often contains noise and outliers,
requiring robust techniques for feature matching.
● Scale and Viewpoint Changes: Features may undergo changes in scale or
viewpoint, requiring methods that are invariant to such variations.
Applications:
● Image Stitching: Aligning and stitching together multiple images to create
panoramic views.
● Robotics and SLAM: Aligning consecutive frames in the context of robotic
navigation and simultaneous localization and mapping.
● Medical Imaging: Aligning 2D slices or 3D volumes for accurate medical
image analysis.
Evaluation:
● Accuracy and Robustness: The accuracy and robustness of feature-based
alignment methods are crucial for their successful application in various
domains.

Feature-based alignment is a fundamental task in computer vision, enabling the

integration of information from multiple views or modalities for improved analysis and
understanding of the visual world.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

2. Pose estimation:

Pose estimation is a computer vision task that involves determining the position and
orientation of an object or camera relative to a coordinate system. It is a crucial aspect
of understanding the spatial relationships between objects in a scene. Pose estimation
can be applied to both 2D and 3D scenarios, and it finds applications in various fields,
including robotics, augmented reality, autonomous vehicles, and human-computer
interaction.

2D Pose Estimation:
● Definition: In 2D pose estimation, the goal is to estimate the position
(translation) and orientation (rotation) of an object in a two-dimensional
image.
● Methods: Techniques include keypoint-based approaches, where
distinctive points (such as corners or joints) are detected and used to
estimate pose. Common methods include PnP (Perspective-n-Point)
algorithms.
3D Pose Estimation:
● Definition: In 3D pose estimation, the goal is to estimate the position and
orientation of an object in three-dimensional space.
● Methods: Often involves associating 2D keypoints with corresponding 3D
points. PnP algorithms can be extended to 3D, and there are other
methods like Iterative Closest Point (ICP) for aligning a 3D model with a
point cloud.
Applications:
● Robotics: Pose estimation is crucial for robotic systems to navigate and
interact with the environment.
● Augmented Reality: Enables the alignment of virtual objects with the
real-world environment.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

● Autonomous Vehicles: Used for understanding the position and

orientation of the vehicle in its surroundings.
● Human Pose Estimation: Estimating the pose of a person, often used in
applications like gesture recognition and action recognition.
Camera Pose Estimation:
● Definition: Estimating the pose of a camera, which involves determining its
position and orientation in the scene.
● Methods: Camera pose can be estimated using visual odometry, SLAM
(Simultaneous Localization and Mapping), or using known reference
points in the environment.
Challenges:
● Ambiguity: Limited information or similar appearance of different poses
can introduce ambiguity.
● Occlusion: Partially or fully occluded objects can make pose estimation
challenging.
● Real-time Requirements: Many applications, especially in robotics and
augmented reality, require real-time pose estimation.
Evaluation Metrics:
● Common metrics include translation and rotation errors, which measure
the accuracy of the estimated pose compared to ground truth.
Deep Learning Approaches:
● Recent advances in deep learning have led to the development of neural
network-based methods for pose estimation, leveraging architectures like
convolutional neural networks (CNNs) for feature extraction.

Pose estimation is a fundamental task in computer vision with widespread applications.

It plays a crucial role in enabling machines to understand the spatial relationships
between objects and the environment.

3. Geometric intrinsic calibration:

Geometric intrinsic calibration is a process in computer vision and camera calibration

that involves determining the intrinsic parameters of a camera. Intrinsic parameters
describe the internal characteristics of a camera, such as its focal length, principal
point, and lens distortion coefficients. Accurate calibration is essential for applications

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

like 3D reconstruction, object tracking, and augmented reality, where knowing the
intrinsic properties of the camera is crucial for accurate scene interpretation.

Here are key points related to geometric intrinsic calibration:

Intrinsic Parameters:
● Focal Length (f): Represents the distance from the camera's optical center
to the image plane. It is a critical parameter for determining the scale of
objects in the scene.
● Principal Point (c): Denotes the coordinates of the image center. It
represents the offset from the top-left corner of the image to the center of
the image plane.
● Lens Distortion Coefficients: Describe imperfections in the lens, such as
radial and tangential distortions, that affect the mapping between 3D
world points and 2D image points.
Camera Model:
● The camera model, often used for intrinsic calibration, is the pinhole
camera model. This model assumes that light enters the camera through
a single point (pinhole) and projects onto the image plane.
Calibration Patterns:
● Intrinsic calibration is typically performed using calibration patterns with
known geometric features, such as chessboard patterns. These patterns
allow for the extraction of corresponding points in both 3D world
coordinates and 2D image coordinates.
Calibration Process:
● Image Capture: Multiple images of the calibration pattern are captured
from different viewpoints.
● Feature Extraction: Detected features (corners, intersections) in the
calibration pattern are identified in both image and world coordinates.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

● Parameter Estimation: The intrinsic parameters are estimated using

mathematical optimization techniques, such as nonlinear least squares
optimization.
● Evaluation: The accuracy of calibration is often assessed by reprojecting
3D points onto the images and comparing with the detected 2D points.
Radial and Tangential Distortions:
● Radial Distortion: Deviation from a perfect pinhole camera model due to
radial symmetry. Corrected using distortion coefficients.
● Tangential Distortion: Caused by the lens not being perfectly parallel to the
image plane. Corrected using tangential distortion coefficients.
Multiple Views:
● Calibration is often performed using multiple views to improve accuracy
and handle lens distortions more effectively.
Applications:
● Intrinsic calibration is essential for various computer vision applications,
including 3D reconstruction, camera pose estimation, and stereo vision.

Accurate geometric intrinsic calibration is a critical step in ensuring that the camera
model accurately represents the mapping between the 3D world and the 2D image,
facilitating precise computer vision tasks

4. Triangulation:

Triangulation, in the context of computer vision and 3D computer graphics, is a

technique used to determine the 3D coordinates of a point in space by computing its
position relative to multiple camera viewpoints. The process involves finding the
intersection point of lines or rays originating from corresponding 2D image points in
different camera views.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

Here are key points related to triangulation:

Basic Concept:
● Triangulation is based on the principle of finding the 3D location of a point
in space by measuring its projection onto two or more image planes.
Camera Setup:
● Triangulation requires at least two cameras (stereo vision) or more to
capture the same scene from different viewpoints. Each camera provides
a 2D projection of the 3D point.
Mathematical Representation:

Epipolar Geometry:
● Epipolar geometry is utilized to relate the 2D projections of a point in
different camera views. It defines the geometric relationship between the
two camera views and helps establish correspondences between points.
Triangulation Methods:
● Direct Linear Transform (DLT): An algorithmic approach that involves
solving a system of linear equations to find the 3D coordinates.
● Iterative Methods: Algorithms like the Gauss-Newton algorithm or the
Levenberg-Marquardt algorithm can be used for refining the initial
estimate obtained through DLT.
Accuracy and Precision:
● The accuracy of triangulation is influenced by factors such as the
calibration accuracy of the cameras, the quality of feature matching, and
the level of noise in the image data.
Bundle Adjustment:
● Triangulation is often used in conjunction with bundle adjustment, a
technique that optimizes the parameters of the cameras and the 3D points
simultaneously to minimize the reprojection error.
Applications:
● 3D Reconstruction: Triangulation is fundamental to creating 3D models of
scenes or objects from multiple camera views.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

● Structure from Motion (SfM): Used in SfM pipelines to estimate the 3D

structure of a scene from a sequence of images.
● Stereo Vision: Essential for depth estimation in stereo vision systems.
Challenges:
● Ambiguity: Ambiguities may arise when triangulating points from two
views if the views are not well-separated or if the point is near the baseline
connecting the cameras.
● Noise and Errors: Triangulation results can be sensitive to noise and errors
in feature matching and camera calibration.

Triangulation is a core technique in computer vision that enables the reconstruction of

3D geometry from multiple 2D images. It plays a crucial role in applications such as 3D
modeling, augmented reality, and structure-from-motion pipelines.

5. Two-frame structure from motion:

Two-Frame Structure from Motion

Structure from Motion (SfM) is a computer vision technique that aims to reconstruct the
three-dimensional structure of a scene from a sequence of two-dimensional images.
Two-frame Structure from Motion specifically refers to the reconstruction of scene
geometry using information from only two images (frames) taken from different
viewpoints. This process involves estimating both the 3D structure of the scene and the
camera motion between the two frames.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

Here are key points related to Two-Frame Structure from Motion:

Basic Concept:
● Two-frame Structure from Motion reconstructs the 3D structure of a scene
by analyzing the information from just two images taken from different
perspectives.
Correspondence Matching:
● Establishing correspondences between points or features in the two
images is a crucial step. This is often done by identifying key features
(such as keypoints) in both images and finding their correspondences.
Epipolar Geometry:
● Epipolar geometry describes the relationship between corresponding
points in two images taken by different cameras. It helps constrain the
possible 3D structures and camera motions.
Essential Matrix:
● The essential matrix is a fundamental matrix in epipolar geometry that
encapsulates the essential information about the relative pose of two
calibrated cameras.
Camera Pose Estimation:
● The camera poses (positions and orientations) are estimated for both
frames. This involves solving for the rotation and translation between the
two camera viewpoints.
Triangulation:
● Triangulation is applied to find the 3D coordinates of points in the scene.
By knowing the camera poses and corresponding points, the depth of
scene points can be estimated.
Bundle Adjustment:
● Bundle adjustment is often used to refine the estimates of camera poses
and 3D points. It is an optimization process that minimizes the error
between observed and predicted image points.
Depth Ambiguity:
● Two-frame SfM is susceptible to depth ambiguity, meaning that the
reconstructed scene could be scaled or mirrored without affecting the
projections onto the images.
Applications:
● Robotics: Two-frame SfM is used in robotics for environment mapping and
navigation.
● Augmented Reality: Reconstruction of the 3D structure for overlaying
virtual objects onto the real-world scene.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

● Computer Vision Research: Studying the principles of SfM and epipolar

geometry.
Challenges:
● Noise and Outliers: The accuracy of the reconstruction can be affected by noise
and outliers in the correspondence matching process.
● Limited Baseline: With only two frames, the baseline (distance between camera
viewpoints) may be limited, leading to potential depth ambiguities.

Two-frame Structure from Motion is a fundamental concept in computer vision,

providing a foundation for understanding 3D scene structure from a pair of images. It is
often extended to multi-frame SfM for more robust reconstructions in scenarios where
more images are available.

6. Factorization:

Factorization in the context of computer vision typically refers to the factorization of

matrices or tensors representing data in various computer vision tasks. One common
application is in the field of structure from motion (SfM) and multiple-view geometry.
Here are key points related to factorization in computer vision:

Matrix Factorization in SfM:

● Problem Statement: In structure from motion, the goal is to reconstruct
the 3D structure of a scene from a sequence of 2D images taken from
different viewpoints.
● Matrix Representation: The correspondence matrix, also known as the
measurement matrix, is constructed by stacking the image coordinates of
corresponding points from multiple views.
● Matrix Factorization: Factorizing the correspondence matrix into two
matrices representing camera parameters and 3D structure is a common
approach. This factorization is often achieved through techniques like
Singular Value Decomposition (SVD).
Singular Value Decomposition (SVD):
● Application: SVD is frequently used in matrix factorization problems in
computer vision.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

Applications:
● Structure from Motion (SfM): Factorization is used to recover camera
poses and 3D scene structure from 2D image correspondences.
● Background Subtraction: Matrix factorization techniques are employed in
background subtraction methods for video analysis.
● Face Recognition: Eigenface and Fisherface methods involve factorizing
covariance matrices for facial feature representation.
Non-Negative Matrix Factorization (NMF):
● Application: NMF is a variant of matrix factorization where the factors are
constrained to be non-negative.
● Use Cases: It is applied in areas such as topic modeling, image
segmentation, and feature extraction.
Tensor Factorization:
● Extension to Higher Dimensions: In some cases, data is represented as
tensors, and factorization techniques are extended to tensors for
applications like multi-way data analysis.
● Example: Canonical Polyadic Decomposition (CPD) is a tensor
factorization technique.
Robust Factorization:
● Challenges: Noise and outliers in the data can affect the accuracy of
factorization.
● Robust Methods: Robust factorization techniques are designed to handle
noisy data and outliers, providing more reliable results.
Deep Learning Approaches:
● Autoencoders and Neural Networks: Deep learning models, including
autoencoders, can be considered as a form of nonlinear factorization.
Factorization Machine (FM):
● Application: Factorization Machines are used in collaborative filtering and
recommendation systems to model interactions between features.

Factorization plays a crucial role in various computer vision and machine learning tasks,
providing a mathematical framework for extracting meaningful representations from

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

data and solving complex problems like 3D reconstruction and dimensionality

reduction.

7. Bundle adjustment:

Bundle Adjustment is a crucial optimization technique in computer vision and

photogrammetry. It is used to refine the parameters of a 3D scene, such as camera
poses and 3D points, by minimizing the reprojection error between the observed image
points and their corresponding projections from the 3D scene. Bundle Adjustment is
commonly employed in the context of structure from motion (SfM), simultaneous
localization and mapping (SLAM), and 3D reconstruction.

Here are key points related to Bundle Adjustment:

Optimization Objective:
● Minimization of Reprojection Error: Bundle Adjustment aims to find the
optimal set of parameters (camera poses, 3D points) that minimizes the
difference between the observed 2D image points and their projections
onto the image planes based on the estimated 3D scene.
Parameters to Optimize:
● Camera Parameters: Intrinsic parameters (focal length, principal point)
and extrinsic parameters (camera poses - rotation and translation).
● 3D Scene Structure: Coordinates of 3D points in the scene.
Reprojection Error:
● Definition: The reprojection error is the difference between the observed
2D image points and the projections of the corresponding 3D points onto
the image planes.
● Sum of Squared Differences: The objective is to minimize the sum of
squared differences between observed and projected points.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

Bundle Adjustment Process:

● Initialization: Start with initial estimates of camera poses and 3D points.
● Objective Function: Define an objective function that measures the
reprojection error.
● Optimization: Use optimization algorithms (such as Levenberg-Marquardt,
Gauss-Newton, or others) to iteratively refine the parameters, minimizing
the reprojection error.
Sparse and Dense Bundle Adjustment:
● Sparse BA: Considers a subset of 3D points and image points, making it
computationally more efficient.
● Dense BA: Involves all 3D points and image points, providing higher
accuracy but requiring more computational resources.
Sequential and Global Bundle Adjustment:
● Sequential BA: Optimizes camera poses and 3D points sequentially,
typically in a sliding window fashion.
● Global BA: Optimizes all camera poses and 3D points simultaneously.
Provides a more accurate solution but is computationally more
demanding.
Applications:
● Structure from Motion (SfM): Refines the reconstruction of 3D scenes
from a sequence of images.
● Simultaneous Localization and Mapping (SLAM): Improves the accuracy
of camera pose estimation and map reconstruction in real-time
environments.
● 3D Reconstruction: Enhances the accuracy of reconstructed 3D models
from images.
Challenges:
● Local Minima: The optimization problem may have multiple local minima,
making it essential to use robust optimization methods.
● Outliers and Noise: Bundle Adjustment needs to be robust to outliers and
noise in the input data.
Integration with Other Techniques:
● Feature Matching: Often used in conjunction with feature matching
techniques to establish correspondences between 2D and 3D points.
● Camera Calibration: Bundle Adjustment may be preceded by or integrated
with camera calibration to refine intrinsic parameters.

Bundle Adjustment is a fundamental optimization technique that significantly improves

the accuracy of 3D reconstructions and camera pose estimations in computer vision

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

applications. It has become a cornerstone in many systems dealing with 3D scene

understanding and reconstruction.

8. Constrained structure and motion:

Constrained Structure and Motion

Constrained Structure and Motion refers to a set of techniques and methods in

computer vision and photogrammetry that incorporate additional constraints into the
structure from motion (SfM) process. The goal is to improve the accuracy and reliability
of 3D reconstruction by imposing constraints on the estimated camera poses and 3D
scene points. These constraints may come from prior knowledge about the scene,
sensor characteristics, or additional information.

Here are key points related to Constrained Structure and Motion:

Introduction of Constraints:
● Prior Information: Constraints can be introduced based on prior
knowledge about the scene, such as known distances, planar structures,
or object shapes.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

● Sensor Constraints: Information about the camera system, such as focal

length or aspect ratio, can be incorporated as constraints.
Types of Constraints:
● Geometric Constraints: Constraints that enforce geometric relationships,
such as parallel lines, perpendicularity, or known distances between
points.
● Semantic Constraints: Incorporating semantic information about the
scene, such as the knowledge that certain points belong to a specific
object or structure.
Bundle Adjustment with Constraints:
● Objective Function: The bundle adjustment problem is formulated with an
objective function that includes the reprojection error, as well as additional
terms representing the constraints.
● Optimization: Optimization techniques, such as Levenberg-Marquardt or
Gauss-Newton, are used to minimize the combined cost function.
Advantages:
● Improved Accuracy: Incorporating constraints can lead to more accurate
and reliable reconstructions, especially in scenarios with limited or noisy
data.
● Handling Ambiguities: Constraints help in resolving ambiguities that may
arise in typical SfM scenarios.
Common Types of Constraints:
● Planar Constraints: Assuming that certain structures in the scene lie on
planes, which can be enforced during reconstruction.
● Scale Constraints: Fixing or constraining the scale of the scene to prevent
scale ambiguity in the reconstruction.
● Object Constraints: Incorporating constraints related to specific objects or
entities in the scene.
Applications:
● Architectural Reconstruction: Constraining the reconstruction based on
known architectural elements or planar surfaces.
● Robotics and Autonomous Systems: Utilizing constraints to enhance the
accuracy of pose estimation and mapping in robotic navigation.
● Augmented Reality: Incorporating semantic constraints for more accurate
alignment of virtual objects with the real world.
Challenges:
● Correctness of Constraints: The accuracy of the reconstruction depends
on the correctness of the imposed constraints.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

● Computational Complexity: Some constraint types may increase the

computational complexity of the optimization problem.
Integration with Semantic Technologies:
● Semantic 3D Reconstruction: Integrating semantic information into the
reconstruction process to improve the understanding of the scene.

Constrained Structure and Motion provides a way to incorporate additional information

and domain knowledge into the reconstruction process, making it a valuable approach
for scenarios where such information is available and reliable. It contributes to more
accurate and meaningful 3D reconstructions in computer vision applications.

9. Translational alignment

Translational alignment, in the context of computer vision and image processing, refers
to the process of aligning two or more images based on translational transformations.
Translational alignment involves adjusting the position of images along the x and y axes
to bring corresponding features or points into alignment. This type of alignment is often
a fundamental step in various computer vision tasks, such as image registration,
panorama stitching, and motion correction.

Here are key points related to translational alignment:

Objective:
● The primary goal of translational alignment is to align images by
minimizing the translation difference between corresponding points or
features in the images.
Translation Model:

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

Correspondence Matching:
● Correspondence matching involves identifying corresponding features or
points in the images that can be used as reference for alignment.
Common techniques include keypoint detection and matching.
Alignment Process:
● The translational alignment process typically involves the following steps:

Applications:
● Image Stitching: In panorama creation, translational alignment is used to
align images before merging them into a seamless panorama.
● Motion Correction: In video processing, translational alignment corrects
for translational motion between consecutive frames.
● Registration in Medical Imaging: Aligning medical images acquired from
different modalities or at different time points.
Evaluation:
● The success of translational alignment is often evaluated by measuring
the accuracy of the alignment, typically in terms of the distance between
corresponding points before and after alignment.
Robustness:
● Translational alignment is relatively straightforward and computationally
efficient. However, it may be sensitive to noise and outliers, particularly in
the presence of large rotations or distortions.
Integration with Other Transformations:
● Translational alignment is frequently used as an initial step in more
complex alignment processes that involve additional transformations,
such as rotational alignment or affine transformations.
Automated Alignment:
● In many applications, algorithms for translational alignment are designed
to operate automatically without requiring manual intervention.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

Translational alignment serves as a foundational step in various computer vision

applications, providing a simple and effective means to align images before further
processing or analysis.

10. Parametric motion

Parametric motion refers to the modeling and representation of motion in computer

vision and computer graphics using parametric functions or models. Instead of directly
capturing the motion with a set of discrete frames, parametric motion models describe
how the motion evolves over time using a set of parameters. These models are often
employed in various applications, such as video analysis, animation, and tracking.

Here are key points related to parametric motion:

Parametric Functions:
● Parametric motion models use mathematical functions with parameters
to represent the motion of objects or scenes over time. These functions
could be simple mathematical equations or more complex models.
Types of Parametric Motion Models:
● Linear Models: Simplest form of parametric motion, where motion is
represented by linear equations. For example, linear interpolation between
keyframes.
● Polynomial Models: Higher-order polynomial functions can be used to
model more complex motion. Cubic splines are commonly used for
smooth motion interpolation.
● Trigonometric Models: Sinusoidal functions can be employed to represent
periodic motion, such as oscillations or repetitive patterns.
● Exponential Models: Capture behaviors that exhibit exponential growth or
decay, suitable for certain types of motion.
Keyframe Animation:
● In parametric motion, keyframes are specified at certain points in time,
and the motion between keyframes is defined by the parametric motion
model. Interpolation is then used to generate frames between keyframes.
Control Points and Handles:
● Parametric models often involve control points and handles that influence
the shape and behavior of the motion curve. Adjusting these parameters
allows for creative control over the motion.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

Applications:
● Computer Animation: Used for animating characters, objects, or camera
movements in 3D computer graphics and animation.
● Video Compression: Parametric motion models can be used to describe
the motion between video frames, facilitating efficient compression
techniques.
● Video Synthesis: Generating realistic videos or predicting future frames in
a video sequence based on learned parametric models.
● Motion Tracking: Tracking the movement of objects in a video by fitting
parametric motion models to observed trajectories.
Smoothness and Continuity:
● One advantage of parametric motion models is their ability to provide
smooth and continuous motion, especially when using interpolation
techniques between keyframes.
Constraints and Constraints-Based Motion:
● Parametric models can be extended to include constraints, ensuring that
the motion adheres to specific rules or conditions. For example, enforcing
constant velocity or maintaining specific orientations.
Machine Learning Integration:
● Parametric motion models can be learned from data using machine
learning techniques. Machine learning algorithms can learn the
parameters of the motion model from observed examples.
Challenges:
● Designing appropriate parametric models that accurately capture the
desired motion can be challenging, especially for complex or non-linear
motions.
● Ensuring that the motion remains physically plausible and visually
appealing is crucial in animation and simulation.

Parametric motion provides a flexible framework for representing and controlling

motion in various visual computing applications. The choice of parametric model
depends on the specific characteristics of the motion to be represented and the desired
level of control and realism.

11. Spline-based motion

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

Spline-based motion refers to the use of spline curves to model and interpolate motion
in computer graphics, computer-aided design, and animation. Splines are mathematical
curves that provide a smooth and flexible way to represent motion paths and
trajectories. They are widely used in 3D computer graphics and animation for creating
natural and visually pleasing motion, particularly in scenarios where continuous and
smooth paths are desired.

Here are key points related to spline-based motion:

Spline Definition:
● Spline Curve: A spline is a piecewise-defined polynomial curve. It consists
of several polynomial segments (typically low-degree) that are smoothly
connected at specific points called knots or control points.
● Types of Splines: Common types of splines include B-splines, cubic
splines, and Bezier splines.
Spline Interpolation:
● Spline curves are often used to interpolate keyframes or control points in
animation. This means the curve passes through or follows the specified
keyframes, creating a smooth motion trajectory.
B-spline (Basis Spline):
● B-splines are widely used for spline-based motion. They are defined by a
set of control points, and their shape is influenced by a set of basis
functions.
● Local Control: Modifying the position of a control point affects only a local
portion of the curve, making B-splines versatile for animation.
Cubic Splines:
● Cubic splines are a specific type of spline where each polynomial segment
is a cubic (degree-3) polynomial.
● Natural Motion: Cubic splines are often used for creating natural motion
paths due to their smoothness and continuity.
Bezier Splines:
● Bezier splines are a type of spline that is defined by a set of control points.
They have intuitive control handles that influence the shape of the curve.
● Bezier Curves: Cubic Bezier curves, in particular, are frequently used for
creating motion paths in animation.
Spline Tangents and Curvature:
● Spline-based motion allows control over the tangents at control points,
influencing the direction of motion. Curvature continuity ensures smooth
transitions between segments.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

Applications:
● Computer Animation: Spline-based motion is extensively used for
animating characters, camera movements, and objects in 3D scenes.
● Path Generation: Designing smooth and visually appealing paths for
objects to follow in simulations or virtual environments.
● Motion Graphics: Creating dynamic and aesthetically pleasing visual
effects in motion graphics projects.
Parametric Representation:
● Spline-based motion is parametric, meaning the position of a point on the
spline is determined by a parameter. This allows for easy manipulation
and control over the motion.
Interpolation Techniques:
● Keyframe Interpolation: Spline curves interpolate smoothly between
keyframes, providing fluid motion transitions.
● Hermite Interpolation: Splines can be constructed using Hermite
interpolation, where both position and tangent information at control
points are considered.
Challenges:
● Overfitting: In some cases, spline curves can be overly flexible and lead to
overfitting if not properly controlled.
● Control Point Placement: Choosing the right placement for control points
is crucial for achieving the desired motion characteristics.

Spline-based motion provides animators and designers with a versatile tool for creating
smooth and controlled motion paths in computer-generated imagery. The ability to
adjust the shape of the spline through control points and handles makes it a popular
choice for a wide range of animation and graphics applications.

12. Optical flow

Optical flow is a computer vision technique that involves estimating the motion of
objects or surfaces in a visual scene based on the observed changes in brightness or
intensity over time. It is a fundamental concept used in various applications, including
motion analysis, video processing, object tracking, and scene understanding.

Here are key points related to optical flow:

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

Motion Estimation:
● Objective: The primary goal of optical flow is to estimate the velocity
vector (optical flow vector) for each pixel in an image, indicating the
apparent motion of that pixel in the scene.
● Pixel-level Motion: Optical flow provides a dense representation of motion
at the pixel level.
Brightness Constancy Assumption:
● Assumption: Optical flow is based on the assumption of brightness
constancy, which states that the brightness of a point in the scene
remains constant over time.

Optical Flow Equation:

● Derivation: The optical flow equation is derived from the brightness
constancy assumption using partial derivatives with respect to spatial
coordinates and time.

Dense and Sparse Optical Flow:

● Dense Optical Flow: Estimating optical flow for every pixel in the image,
providing a complete motion field.
● Sparse Optical Flow: Estimating optical flow only for selected key points or
features in the image.
Computational Methods:
● Correlation-based Methods: Match image patches or windows between
consecutive frames to estimate motion.
● Gradient-based Methods: Utilize image gradients to compute optical flow.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

● Variational Methods: Formulate energy minimization problems to estimate

optical flow.
Lucas-Kanade Method:
● A well-known differential method for estimating optical flow, particularly
suited for small motion and local analysis.
Horn-Schunck Method:
● A variational method that minimizes a global energy function, taking into
account smoothness constraints in addition to brightness constancy.
Applications:
● Video Compression: Optical flow is used in video compression algorithms
to predict motion between frames.
● Object Tracking: Tracking moving objects in a video sequence.
● Robotics: Providing visual feedback for navigation and obstacle
avoidance.
● Augmented Reality: Aligning virtual objects with the real-world scene.
Challenges:
● Illumination Changes: Optical flow may be sensitive to changes in
illumination.
● Occlusions: Occlusions and complex motion patterns can pose challenges
for accurate optical flow estimation.
● Large Displacements: Traditional methods may struggle with handling
large displacements.
Deep Learning for Optical Flow:
● Recent advances in deep learning have led to the development of neural
network-based methods for optical flow estimation, such as FlowNet and
PWC-Net.

Optical flow is a valuable tool for understanding and analyzing motion in visual data.
While traditional methods have been widely used, the integration of deep learning has
brought new perspectives and improved performance in optical flow estimation.

13. Layered motion

Layered motion, in the context of computer vision and motion analysis, refers to the
representation and analysis of a scene where different objects or layers move
independently of each other. It assumes that the motion in a scene can be decomposed
into multiple layers, each associated with a distinct object or surface. Layered motion

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

models are employed to better capture complex scenes with multiple moving entities,
handling occlusions and interactions between objects.

Here are key points related to layered motion:

Layered Motion Models:

● Objective: The goal of layered motion models is to represent the motion of
distinct objects or surfaces in a scene independently, allowing for a more
accurate description of complex motion scenarios.
● Assumption: It assumes that the observed motion in a scene can be
decomposed into the motion of different layers.
Key Concepts:
● Independence: Layers are assumed to move independently of each other,
simplifying the modeling of complex scenes.
● Occlusions: Layered motion models can handle occlusions more
effectively, as each layer represents a separate entity in the scene.
Motion Layer Segmentation:
● Segmentation Process: The process of identifying and separating the
different motion layers in a video sequence is referred to as motion layer
segmentation.
● Foreground and Background: Layers might represent the foreground and
background elements in a scene.
Challenges in Layered Motion:
● Interaction Handling: Representing the interaction between layers, such as
occlusions or overlapping motions.
● Dynamic Scene Changes: Adapting to changes in the scene, including the
appearance or disappearance of objects.
Optical Flow for Layered Motion:
● Optical flow techniques can be extended to estimate the motion of
individual layers in a scene.
● Layer-Specific Optical Flow: Applying optical flow independently to
different layers.
Multiple Object Tracking:
● Layered motion models are closely related to multiple object tracking, as
each layer can correspond to a tracked object.
Applications:
● Surveillance and Security: Tracking and analyzing the motion of multiple
objects in surveillance videos.

Downloaded by code breakers ([email protected])

 lOMoARcPSD|49979752

● Robotics: Layered motion models can aid robots in understanding and

navigating dynamic environments.
● Augmented Reality: Aligning virtual objects with the real-world scene by
understanding the layered motion.
Representation Formats:
● Layers can be represented in various formats, such as depth maps,
segmentation masks, or explicit motion models for each layer.
Integration with Scene Understanding:
● Layered motion models can be integrated with higher-level scene und

Downloaded by code breakers ([email protected])