0% found this document useful (0 votes)
345 views19 pagesImage Real World Coordinate Calculation
Image coordinate, camera coordinate, world coordinate basic concepts and mathematical calculation.
Uploaded by
Muhammad Zaka Ud DinCopyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
0% found this document useful (0 votes)
345 views19 pagesImage Real World Coordinate Calculation
Image coordinate, camera coordinate, world coordinate basic concepts and mathematical calculation.
Uploaded by
Muhammad Zaka Ud DinCopyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
/ 19
12472020 CCaloulate X, ¥,2 Real World Coordinates from Image Coordinates using OpenCV — 16x labs
Menu
# Home > Computer Vision > Calculate X, ¥,Z Real World Coordinates from Image Coordinates using OpenCv
Posted on April 10, 2019 by fdxlabs — 5 Comments
Calculate xX, Y, Z Real World Coordinates
from Image Coordinates using OpenCV
Projected Image
Camera
Real World Coordinates
As soon as I finished my Horizontal Travel Robot Arm prototype and was able to reliable make pick and
place motions using simple X, ¥, Z inputs, | decided to build a real use case that could show it’s potential
for real world applications.
2 Q
hips: flabs.comicaleuate-cy-z-real-worlé-coordinates fom-a-single-camerasing-opency! a9 1242020 CCaloulate X, ¥,2 Real World Coordinates from Image Coordinates using OpenCV — 16x labs
Note: I've taken the liberty of highlighting what | consider the most important parts. I'm counting you will
refer to the code repository as well as the multiple diagrams I reference below.
Coordinates and Frames of Reference
You can read through my Medium post on the overview of the robot and watch the video of it in
‘operation in Youtube.
Asa reminder, this is the setup of this robot:
Horizontal Travel Robot Arm - HTAO
As | dive deeper in this blog, it will be very important to keep in mind the frames of references I've used:
Q
htpsswwflabs.comicaleuate-cy-z-real-worlé-coordinales fom-a-single-camerasing-opency!
ane 1242020 CCaloulate X, ¥,2 Real World Coordinates from Image Coordinates using OpenCV — 16x labs
Robot, Camera and Plan coordinate frames of reference
Red is the X-axis, Green the Y-Axis and Blue the Z-axis, the arrows point in the direction of Positive
increases. This illustration will be crucial to understand the code and how you can use it on your own
projects (which may have different frames of reference based on your application).
OpenCVv Camera Calibration and 3D
Reconstruction
Formulas
The main reference I have used for this, is the OpenCV 2.4 and 3.0 documentation, using these three
pages:
* Calibration Tutorial
© Camers ration and 3D R nm
2 Q
htpsswwflabs.comicaleuate-cy-z-real-worlé-coordinales fom-a-single-camerasing-opency! ane 1242020 CCaloulate X, ¥,2 Real World Coordinates from Image Coordinates using OpenCV — 16x labs
journey was how to calculate real-world X ¥ Z coordinates from a given Image’s projection points. | was
looking for this, and I couldn't find any references that could easily explain how to do it:
Given this Find This
f 0 a) fm m me oy [y
w
s]v]=]0 fy cy] fru te ms te} [7
1 00 1) [rm te me tl) /4
where:
* (X, ¥,Z) are the coordinates of a 3D point in the world coordinate space
+ (u,v) ate the coordinates of the projection point in pixels
* Aisa camera matrix, of a matrix of intrinsic parameters
* (cx, cy) iS a principal point that is usuelly at the image center
# fx, fy are the focal lengths expressed in pixel units
The pinhole camera model
The main challenge | found with this pinhole model, is that if you want to solve for XY Z, it cannot be
done, because you cannot calculate the inverse of the R|t matrix as it is not square.
Before | dive into the solution, it is important to understand the pinhole camera model and coordinates:
optical
principal |
ein
Gen!
+y
htps:hww flabs.comicaleuate-cy-2-real-werlé-coordinales fom-a-single-camerauising-openci! ane sear2020 CCaloulate X, ¥,2 Real World Coordinates from Image Coordinates using OpenCV — 16x labs
to the following (when z # 0 }
x x
yl =R|¥] +t
Zz Z
Pinhole model when zis not equal to.0
To make it work, | added the scaling factor and camera matrix, to arrive at u, v which now enabled to
solve for X Y Z in the following manner:
e
I