Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Most hand-eye calibration methods don't work #24871

@JStech

Description

@JStech

System Information

OpenCV python version: 4.9.0-dev (I noticed in 4.5.4; bug is still in master)
Operating System / Platform: Ubuntu 22.04
Compiler & compiler version: GCC 11.4.0

Detailed description

In the script below, only the Park hand-eye calibration method works in all cases. Andreff also works when the transforms have non-zero translations (I originally ran into this while trying to do an orientation-only calibration).

Steps to reproduce

#!/usr/bin/env python3
import sys
sys.path.insert(0, '/home/john/opencv/build/python_loader')
import cv2 as cv
import numpy as np
import transformations as tf
from math import acos

print("OpenCV version", cv.__version__)

np.random.seed(42)
def small_random_T():
    q = np.array([1, 0, 0, 0]) + 0.01*np.random.randn(4)
    q /= np.linalg.norm(q)
    return tf.translation_matrix(0.001 * np.random.randn(3)) @ tf.quaternion_matrix(q)

def random_T():
    return tf.translation_matrix(np.random.randn(3)) @ tf.quaternion_matrix(np.random.randn(4))

num_samples = 25

T__imu__t265 = np.array([
    [  0,  0, -1,  0.1],
    [  1,  0,  0,  0.2],
    [  0, -1,  0,  0.3],
    [  0,  0,  0,  1],
    ])

T__wi__wt = random_T()
T__wt__wi = tf.inverse_matrix(T__wi__wt)

# Transforms between the IMU and its world frame (wi)
R__wi__imu = []
t__wi__imu = []

# Transforms between the T265 and its world frame (wt)
R__t265__wt = []
t__t265__wt = []

zeros = []

for use_translation, add_noise in [(True, True), (True, False), (False, True), (False, False)]:
    print("Using translation." if use_translation else "Not using translation.",
        "With noise." if add_noise else "No noise.")
    for i in range(25):
        T__wi__imu = random_T()
        T__wt__t265 = T__wt__wi @ T__wi__imu @ T__imu__t265

        if add_noise:
            T__wi__imu @= small_random_T()
            T__wt__t265 @= small_random_T()

        T__imu__wi = tf.inverse_matrix(T__wi__imu)
        T__t265__wt = tf.inverse_matrix(T__wt__t265)

        R__wi__imu.append(T__wi__imu[:3,:3])
        t__wi__imu.append(T__wi__imu[:3, 3])
        R__t265__wt.append(T__t265__wt[:3,:3])
        t__t265__wt.append(T__t265__wt[:3, 3])
        zeros.append(np.zeros((3,)))

    print("Method      rotation error   translation error")
    for method_name, method in (
            ("None", None),
            ("TSAI", cv.CALIB_HAND_EYE_TSAI),
            ("PARK", cv.CALIB_HAND_EYE_PARK),
            ("HORAUD", cv.CALIB_HAND_EYE_HORAUD),
            ("ANDREFF", cv.CALIB_HAND_EYE_ANDREFF),
            ("DANIILIDIS", cv.CALIB_HAND_EYE_DANIILIDIS),
            ):
        try: R_cal, t_cal = cv.calibrateHandEye(R__wi__imu, t__wi__imu if use_translation else zeros,
                                                R__t265__wt, t__t265__wt if use_translation else zeros,
                                                method=method)
        except:
            print(f"     {method_name} threw an exception")
            continue
        rotation_error_radians = acos((np.trace(R_cal @ T__imu__t265[:3, :3].T) - 1) / 2)
        translation_error_meters = np.linalg.norm(t_cal.squeeze() - use_translation*T__imu__t265[:3,3].squeeze())
        print(f"  {method_name:11s} {rotation_error_radians:1.6f}        {translation_error_meters: 1.6f}")
    print()

Output:

OpenCV version 4.9.0-dev
Using translation. With noise.
Method      rotation error   translation error
  None        2.098635         0.594940
  TSAI        2.098635         0.594940
  PARK        0.008859         0.006855
  HORAUD      3.136278         0.790596
  ANDREFF     0.008014         0.006409
  DANIILIDIS  2.381483         14.875658

Using translation. No noise.
Method      rotation error   translation error
  None        2.193811         0.536712
  TSAI        2.193811         0.536712
  PARK        0.003990         0.002868
  HORAUD      3.141554         0.655092
  ANDREFF     0.002626         0.002475
  DANIILIDIS  3.140931         15.506926

Not using translation. With noise.
Method      rotation error   translation error
  None        2.142475         0.000000
  TSAI        2.142475         0.000000
  PARK        0.005489         0.000000
  HORAUD      3.139089         0.000000
     ANDREFF threw an exception
  DANIILIDIS  nan         nan

Not using translation. No noise.
Method      rotation error   translation error
  None        2.185132         0.000000
  TSAI        2.185132         0.000000
  PARK        0.004144         0.000000
  HORAUD      3.141151         0.000000
     ANDREFF threw an exception
  DANIILIDIS  nan         nan

Issue submission checklist

  • I report the issue, it's not a question
  • I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
  • I updated to the latest OpenCV version and the issue is still there
  • There is reproducer code and related data files (videos, images, onnx, etc)

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions