-
-
Notifications
You must be signed in to change notification settings - Fork 56.4k
Closed
Description
System Information
OpenCV python version: 4.9.0-dev (I noticed in 4.5.4; bug is still in master)
Operating System / Platform: Ubuntu 22.04
Compiler & compiler version: GCC 11.4.0
Detailed description
In the script below, only the Park hand-eye calibration method works in all cases. Andreff also works when the transforms have non-zero translations (I originally ran into this while trying to do an orientation-only calibration).
Steps to reproduce
#!/usr/bin/env python3
import sys
sys.path.insert(0, '/home/john/opencv/build/python_loader')
import cv2 as cv
import numpy as np
import transformations as tf
from math import acos
print("OpenCV version", cv.__version__)
np.random.seed(42)
def small_random_T():
q = np.array([1, 0, 0, 0]) + 0.01*np.random.randn(4)
q /= np.linalg.norm(q)
return tf.translation_matrix(0.001 * np.random.randn(3)) @ tf.quaternion_matrix(q)
def random_T():
return tf.translation_matrix(np.random.randn(3)) @ tf.quaternion_matrix(np.random.randn(4))
num_samples = 25
T__imu__t265 = np.array([
[ 0, 0, -1, 0.1],
[ 1, 0, 0, 0.2],
[ 0, -1, 0, 0.3],
[ 0, 0, 0, 1],
])
T__wi__wt = random_T()
T__wt__wi = tf.inverse_matrix(T__wi__wt)
# Transforms between the IMU and its world frame (wi)
R__wi__imu = []
t__wi__imu = []
# Transforms between the T265 and its world frame (wt)
R__t265__wt = []
t__t265__wt = []
zeros = []
for use_translation, add_noise in [(True, True), (True, False), (False, True), (False, False)]:
print("Using translation." if use_translation else "Not using translation.",
"With noise." if add_noise else "No noise.")
for i in range(25):
T__wi__imu = random_T()
T__wt__t265 = T__wt__wi @ T__wi__imu @ T__imu__t265
if add_noise:
T__wi__imu @= small_random_T()
T__wt__t265 @= small_random_T()
T__imu__wi = tf.inverse_matrix(T__wi__imu)
T__t265__wt = tf.inverse_matrix(T__wt__t265)
R__wi__imu.append(T__wi__imu[:3,:3])
t__wi__imu.append(T__wi__imu[:3, 3])
R__t265__wt.append(T__t265__wt[:3,:3])
t__t265__wt.append(T__t265__wt[:3, 3])
zeros.append(np.zeros((3,)))
print("Method rotation error translation error")
for method_name, method in (
("None", None),
("TSAI", cv.CALIB_HAND_EYE_TSAI),
("PARK", cv.CALIB_HAND_EYE_PARK),
("HORAUD", cv.CALIB_HAND_EYE_HORAUD),
("ANDREFF", cv.CALIB_HAND_EYE_ANDREFF),
("DANIILIDIS", cv.CALIB_HAND_EYE_DANIILIDIS),
):
try: R_cal, t_cal = cv.calibrateHandEye(R__wi__imu, t__wi__imu if use_translation else zeros,
R__t265__wt, t__t265__wt if use_translation else zeros,
method=method)
except:
print(f" {method_name} threw an exception")
continue
rotation_error_radians = acos((np.trace(R_cal @ T__imu__t265[:3, :3].T) - 1) / 2)
translation_error_meters = np.linalg.norm(t_cal.squeeze() - use_translation*T__imu__t265[:3,3].squeeze())
print(f" {method_name:11s} {rotation_error_radians:1.6f} {translation_error_meters: 1.6f}")
print()Output:
OpenCV version 4.9.0-dev
Using translation. With noise.
Method rotation error translation error
None 2.098635 0.594940
TSAI 2.098635 0.594940
PARK 0.008859 0.006855
HORAUD 3.136278 0.790596
ANDREFF 0.008014 0.006409
DANIILIDIS 2.381483 14.875658
Using translation. No noise.
Method rotation error translation error
None 2.193811 0.536712
TSAI 2.193811 0.536712
PARK 0.003990 0.002868
HORAUD 3.141554 0.655092
ANDREFF 0.002626 0.002475
DANIILIDIS 3.140931 15.506926
Not using translation. With noise.
Method rotation error translation error
None 2.142475 0.000000
TSAI 2.142475 0.000000
PARK 0.005489 0.000000
HORAUD 3.139089 0.000000
ANDREFF threw an exception
DANIILIDIS nan nan
Not using translation. No noise.
Method rotation error translation error
None 2.185132 0.000000
TSAI 2.185132 0.000000
PARK 0.004144 0.000000
HORAUD 3.141151 0.000000
ANDREFF threw an exception
DANIILIDIS nan nan
Issue submission checklist
- I report the issue, it's not a question
- I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
- I updated to the latest OpenCV version and the issue is still there
- There is reproducer code and related data files (videos, images, onnx, etc)