System Information

OpenCV version: 4.7.0+

Detailed description

Issue origin: #23486 (comment)

In case if board width and height are swapped the method detects board, but result is inadequate. Marker IDs and corners coordinates looks realistic , but does not match actual board layout.

Steps to reproduce

Board: from PR: #23363

How to reproduce:

• With interactive calibration tool (copy interactive-calibration/defaultConfig.xml to current dir and replace charuco_dict value to 5): ./bin/opencv_interactive-calibration -w=6 -h=4 -t=charuco
• With calibration.cpp (PR Add charuco pattern into calibration.cpp #23486): ./example_cpp_calibration -w=6 -h=4 -pt=charucoboard -ad=5

Proposed behaviour:

• Return empty list for ids and coordinates
• Add return value and return false

Issue submission checklist

• I report the issue, it's not a question
• I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
• I updated to the latest OpenCV version and the issue is still there
• There is reproducer code and related data files (videos, images, onnx, etc)