|
| 1 | +Detection of ArUco boards {#tutorial_aruco_board_detection} |
| 2 | +========================= |
| 3 | + |
| 4 | +@prev_tutorial{tutorial_aruco_detection} |
| 5 | + |
| 6 | +| | | |
| 7 | +| -: | :- | |
| 8 | +| Original authors | Sergio Garrido, Alexander Panov | |
| 9 | +| Compatibility | OpenCV >= 4.7.0 | |
| 10 | + |
| 11 | +An ArUco board is a set of markers that acts like a single marker in the sense that it provides a |
| 12 | +single pose for the camera. |
| 13 | + |
| 14 | +The most popular board is the one with all the markers in the same plane, since it can be easily printed: |
| 15 | + |
| 16 | + |
| 17 | + |
| 18 | +However, boards are not limited to this arrangement and can represent any 2d or 3d layout. |
| 19 | + |
| 20 | +The difference between a board and a set of independent markers is that the relative position between |
| 21 | +the markers in the board is known a priori. This allows that the corners of all the markers can be used for |
| 22 | +estimating the pose of the camera respect to the whole board. |
| 23 | + |
| 24 | +When you use a set of independent markers, you can estimate the pose for each marker individually, |
| 25 | +since you dont know the relative position of the markers in the environment. |
| 26 | + |
| 27 | +The main benefits of using boards are: |
| 28 | + |
| 29 | +- The pose estimation is much more versatile. Only some markers are necessary to perform pose estimation. |
| 30 | +Thus, the pose can be calculated even in the presence of occlusions or partial views. |
| 31 | +- The obtained pose is usually more accurate since a higher amount of point correspondences (marker |
| 32 | +corners) are employed. |
| 33 | + |
| 34 | +Board Detection |
| 35 | +--------------- |
| 36 | + |
| 37 | +A board detection is similar to the standard marker detection. The only difference is in the pose estimation step. |
| 38 | +In fact, to use marker boards, a standard marker detection should be done before estimating the board pose. |
| 39 | + |
| 40 | +To perform pose estimation for boards, you should use `solvePnP()` function, as shown below |
| 41 | +in the `samples/cpp/tutorial_code/objectDetection/detect_board.cpp`. |
| 42 | + |
| 43 | +@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_detect_board_full_sample |
| 44 | + |
| 45 | + |
| 46 | +The parameters are: |
| 47 | + |
| 48 | +- `objPoints`, `imgPoints` object and image points, matched with `cv::aruco::GridBoard::matchImagePoints()` |
| 49 | + which, in turn, takes as input `markerCorners` and `markerIds` structures of detected markers from |
| 50 | + `cv::aruco::ArucoDetector::detectMarkers()` function. |
| 51 | +- `board` the `cv::aruco::Board` object that defines the board layout and its ids |
| 52 | +- `cameraMatrix` and `distCoeffs`: camera calibration parameters necessary for pose estimation. |
| 53 | +- `rvec` and `tvec`: estimated pose of the board. If not empty then treated as initial guess. |
| 54 | +- The function returns the total number of markers employed for estimating the board pose. |
| 55 | + |
| 56 | +The drawFrameAxes() function can be used to check the obtained pose. For instance: |
| 57 | + |
| 58 | + |
| 59 | + |
| 60 | +And this is another example with the board partially occluded: |
| 61 | + |
| 62 | + |
| 63 | + |
| 64 | +As it can be observed, although some markers have not been detected, the board pose can still be |
| 65 | +estimated from the rest of markers. |
| 66 | + |
| 67 | +Sample video: |
| 68 | + |
| 69 | +@youtube{Q1HlJEjW_j0} |
| 70 | + |
| 71 | +A full working example is included in the `detect_board.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`. |
| 72 | + |
| 73 | +The samples now take input via command line via the `cv::CommandLineParser`. For this file the example |
| 74 | +parameters will look like: |
| 75 | +@code{.cpp} |
| 76 | + -w=5 -h=7 -l=100 -s=10 |
| 77 | + -v=/path_to_opencv/opencv/doc/tutorials/objdetect/aruco_board_detection/gboriginal.jpg |
| 78 | + -c=/path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/tutorial_camera_params.yml |
| 79 | + -cd=/path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/tutorial_dict.yml |
| 80 | +@endcode |
| 81 | +Parameters for `detect_board.cpp`: |
| 82 | +@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_detect_board_keys |
| 83 | + |
| 84 | +Grid Board |
| 85 | +---------- |
| 86 | + |
| 87 | +Creating the `cv::aruco::Board` object requires specifying the corner positions for each marker in the environment. |
| 88 | +However, in many cases, the board will be just a set of markers in the same plane and in a grid layout, |
| 89 | +so it can be easily printed and used. |
| 90 | + |
| 91 | +Fortunately, the aruco module provides the basic functionality to create and print these types of markers |
| 92 | +easily. |
| 93 | + |
| 94 | +The `cv::aruco::GridBoard` class is a specialized class that inherits from the `cv::aruco::Board` |
| 95 | +class and which represents a Board with all the markers in the same plane and in a grid layout, |
| 96 | +as in the following image: |
| 97 | + |
| 98 | + |
| 99 | + |
| 100 | +Concretely, the coordinate system in a grid board is positioned in the board plane, centered in the bottom left |
| 101 | +corner of the board and with the Z pointing out, like in the following image (X:red, Y:green, Z:blue): |
| 102 | + |
| 103 | + |
| 104 | + |
| 105 | +A `cv::aruco::GridBoard` object can be defined using the following parameters: |
| 106 | + |
| 107 | +- Number of markers in the X direction. |
| 108 | +- Number of markers in the Y direction. |
| 109 | +- Length of the marker side. |
| 110 | +- Length of the marker separation. |
| 111 | +- The dictionary of the markers. |
| 112 | +- Ids of all the markers (X*Y markers). |
| 113 | + |
| 114 | +This object can be easily created from these parameters using the `cv::aruco::GridBoard` constructor: |
| 115 | + |
| 116 | +@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_create_board |
| 117 | + |
| 118 | +- The first and second parameters are the number of markers in the X and Y direction respectively. |
| 119 | +- The third and fourth parameters are the marker length and the marker separation respectively. |
| 120 | + They can be provided in any unit, having in mind that the estimated pose for this board will be |
| 121 | + measured in the same units (in general, meters are used). |
| 122 | +- Finally, the dictionary of the markers is provided. |
| 123 | + |
| 124 | +So, this board will be composed by 5x7=35 markers. The ids of each of the markers are assigned, by default, |
| 125 | +in ascending order starting on 0, so they will be 0, 1, 2, ..., 34. |
| 126 | + |
| 127 | +After creating a grid board, we probably want to print it and use it. |
| 128 | +There are two ways to do this: |
| 129 | +1. By using the script `doc/patter_tools/gen_pattern.py `, see @subpage tutorial_camera_calibration_pattern. |
| 130 | +2. By using the function `cv::aruco::GridBoard::generateImage()`. |
| 131 | + |
| 132 | +The function `cv::aruco::GridBoard::generateImage()` is provided in cv::aruco::GridBoard class and |
| 133 | +can be called by using the following code: |
| 134 | + |
| 135 | +@snippet samples/cpp/tutorial_code/objectDetection/create_board.cpp aruco_generate_board_image |
| 136 | + |
| 137 | +- The first parameter is the size of the output image in pixels. In this case 600x500 pixels. If this is not proportional |
| 138 | +to the board dimensions, it will be centered on the image. |
| 139 | +- `boardImage`: the output image with the board. |
| 140 | +- The third parameter is the (optional) margin in pixels, so none of the markers are touching the image border. |
| 141 | +In this case the margin is 10. |
| 142 | +- Finally, the size of the marker border, similarly to `generateImageMarker()` function. The default value is 1. |
| 143 | + |
| 144 | +A full working example of board creation is included in the `samples/cpp/tutorial_code/objectDetection/create_board.cpp` |
| 145 | + |
| 146 | +The output image will be something like this: |
| 147 | + |
| 148 | + |
| 149 | + |
| 150 | +The samples now take input via commandline via the `cv::CommandLineParser`. For this file the example |
| 151 | +parameters will look like: |
| 152 | +@code{.cpp} |
| 153 | + "_output_path_/aboard.png" -w=5 -h=7 -l=100 -s=10 -d=10 |
| 154 | +@endcode |
| 155 | + |
| 156 | +Refine marker detection |
| 157 | +----------------------- |
| 158 | + |
| 159 | +ArUco boards can also be used to improve the detection of markers. If we have detected a subset of the markers |
| 160 | +that belongs to the board, we can use these markers and the board layout information to try to find the |
| 161 | +markers that have not been previously detected. |
| 162 | + |
| 163 | +This can be done using the `cv::aruco::refineDetectedMarkers()` function, which should be called |
| 164 | +after calling `cv::aruco::ArucoDetector::detectMarkers()`. |
| 165 | + |
| 166 | +The main parameters of this function are the original image where markers were detected, the board object, |
| 167 | +the detected marker corners, the detected marker ids and the rejected marker corners. |
| 168 | + |
| 169 | +The rejected corners can be obtained from the `cv::aruco::ArucoDetector::detectMarkers()` function and |
| 170 | +are also known as marker candidates. This candidates are square shapes that have been found in the |
| 171 | +original image but have failed to pass the identification step (i.e. their inner codification presents |
| 172 | +too many errors) and thus they have not been recognized as markers. |
| 173 | + |
| 174 | +However, these candidates are sometimes actual markers that have not been correctly identified due to high |
| 175 | +noise in the image, very low resolution or other related problems that affect to the binary code extraction. |
| 176 | +The `cv::aruco::ArucoDetector::refineDetectedMarkers()` function finds correspondences between these |
| 177 | +candidates and the missing markers of the board. This search is based on two parameters: |
| 178 | + |
| 179 | +- Distance between the candidate and the projection of the missing marker. To obtain these projections, |
| 180 | +it is necessary to have detected at least one marker of the board. The projections are obtained using the |
| 181 | +camera parameters (camera matrix and distortion coefficients) if they are provided. If not, the projections |
| 182 | +are obtained from local homography and only planar board are allowed (i.e. the Z coordinate of all the |
| 183 | +marker corners should be the same). The `minRepDistance` parameter in `refineDetectedMarkers()` |
| 184 | +determines the minimum euclidean distance between the candidate corners and the projected marker corners |
| 185 | +(default value 10). |
| 186 | + |
| 187 | +- Binary codification. If a candidate surpasses the minimum distance condition, its internal bits |
| 188 | +are analyzed again to determine if it is actually the projected marker or not. However, in this case, |
| 189 | +the condition is not so strong and the number of allowed erroneous bits can be higher. This is indicated |
| 190 | +in the `errorCorrectionRate` parameter (default value 3.0). If a negative value is provided, the |
| 191 | +internal bits are not analyzed at all and only the corner distances are evaluated. |
| 192 | + |
| 193 | +This is an example of using the `cv::aruco::ArucoDetector::refineDetectedMarkers()` function: |
| 194 | + |
| 195 | +@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_detect_and_refine |
| 196 | + |
| 197 | +It must also be noted that, in some cases, if the number of detected markers in the first place is |
| 198 | +too low (for instance only 1 or 2 markers), the projections of the missing markers can be of bad |
| 199 | +quality, producing erroneous correspondences. |
| 200 | + |
| 201 | +See module samples for a more detailed implementation. |
0 commit comments