diff --git a/doc/tutorials/objdetect/aruco_board_detection/aruco_board_detection.markdown b/doc/tutorials/objdetect/aruco_board_detection/aruco_board_detection.markdown new file mode 100644 index 0000000000..9a2d39264d --- /dev/null +++ b/doc/tutorials/objdetect/aruco_board_detection/aruco_board_detection.markdown @@ -0,0 +1,201 @@ +Detection of ArUco boards {#tutorial_aruco_board_detection} +========================= + +@prev_tutorial{tutorial_aruco_detection} + +| | | +| -: | :- | +| Original authors | Sergio Garrido, Alexander Panov | +| Compatibility | OpenCV >= 4.7.0 | + +An ArUco board is a set of markers that acts like a single marker in the sense that it provides a +single pose for the camera. + +The most popular board is the one with all the markers in the same plane, since it can be easily printed: + +![](images/gboriginal.jpg) + +However, boards are not limited to this arrangement and can represent any 2d or 3d layout. + +The difference between a board and a set of independent markers is that the relative position between +the markers in the board is known a priori. This allows that the corners of all the markers can be used for +estimating the pose of the camera respect to the whole board. + +When you use a set of independent markers, you can estimate the pose for each marker individually, +since you dont know the relative position of the markers in the environment. + +The main benefits of using boards are: + +- The pose estimation is much more versatile. Only some markers are necessary to perform pose estimation. +Thus, the pose can be calculated even in the presence of occlusions or partial views. +- The obtained pose is usually more accurate since a higher amount of point correspondences (marker +corners) are employed. + +Board Detection +--------------- + +A board detection is similar to the standard marker detection. The only difference is in the pose estimation step. +In fact, to use marker boards, a standard marker detection should be done before estimating the board pose. + +To perform pose estimation for boards, you should use `solvePnP()` function, as shown below +in the `samples/cpp/tutorial_code/objectDetection/detect_board.cpp`. + +@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_detect_board_full_sample + + +The parameters are: + +- `objPoints`, `imgPoints` object and image points, matched with `cv::aruco::GridBoard::matchImagePoints()` + which, in turn, takes as input `markerCorners` and `markerIds` structures of detected markers from + `cv::aruco::ArucoDetector::detectMarkers()` function. +- `board` the `cv::aruco::Board` object that defines the board layout and its ids +- `cameraMatrix` and `distCoeffs`: camera calibration parameters necessary for pose estimation. +- `rvec` and `tvec`: estimated pose of the board. If not empty then treated as initial guess. +- The function returns the total number of markers employed for estimating the board pose. + +The drawFrameAxes() function can be used to check the obtained pose. For instance: + +![Board with axis](images/gbmarkersaxis.jpg) + +And this is another example with the board partially occluded: + +![Board with occlusions](images/gbocclusion.jpg) + +As it can be observed, although some markers have not been detected, the board pose can still be +estimated from the rest of markers. + +Sample video: + +@youtube{Q1HlJEjW_j0} + +A full working example is included in the `detect_board.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`. + +The samples now take input via command line via the `cv::CommandLineParser`. For this file the example +parameters will look like: +@code{.cpp} + -w=5 -h=7 -l=100 -s=10 + -v=/path_to_opencv/opencv/doc/tutorials/objdetect/aruco_board_detection/gboriginal.jpg + -c=/path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/tutorial_camera_params.yml + -cd=/path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/tutorial_dict.yml +@endcode +Parameters for `detect_board.cpp`: +@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_detect_board_keys + +Grid Board +---------- + +Creating the `cv::aruco::Board` object requires specifying the corner positions for each marker in the environment. +However, in many cases, the board will be just a set of markers in the same plane and in a grid layout, +so it can be easily printed and used. + +Fortunately, the aruco module provides the basic functionality to create and print these types of markers +easily. + +The `cv::aruco::GridBoard` class is a specialized class that inherits from the `cv::aruco::Board` +class and which represents a Board with all the markers in the same plane and in a grid layout, +as in the following image: + +![Image with aruco board](images/gboriginal.jpg) + +Concretely, the coordinate system in a grid board is positioned in the board plane, centered in the bottom left +corner of the board and with the Z pointing out, like in the following image (X:red, Y:green, Z:blue): + +![Board with axis](images/gbaxis.jpg) + +A `cv::aruco::GridBoard` object can be defined using the following parameters: + +- Number of markers in the X direction. +- Number of markers in the Y direction. +- Length of the marker side. +- Length of the marker separation. +- The dictionary of the markers. +- Ids of all the markers (X*Y markers). + +This object can be easily created from these parameters using the `cv::aruco::GridBoard` constructor: + +@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_create_board + +- The first and second parameters are the number of markers in the X and Y direction respectively. +- The third and fourth parameters are the marker length and the marker separation respectively. + They can be provided in any unit, having in mind that the estimated pose for this board will be + measured in the same units (in general, meters are used). +- Finally, the dictionary of the markers is provided. + +So, this board will be composed by 5x7=35 markers. The ids of each of the markers are assigned, by default, +in ascending order starting on 0, so they will be 0, 1, 2, ..., 34. + +After creating a grid board, we probably want to print it and use it. +There are two ways to do this: +1. By using the script `doc/patter_tools/gen_pattern.py `, see @subpage tutorial_camera_calibration_pattern. +2. By using the function `cv::aruco::GridBoard::generateImage()`. + +The function `cv::aruco::GridBoard::generateImage()` is provided in cv::aruco::GridBoard class and +can be called by using the following code: + +@snippet samples/cpp/tutorial_code/objectDetection/create_board.cpp aruco_generate_board_image + +- The first parameter is the size of the output image in pixels. In this case 600x500 pixels. If this is not proportional +to the board dimensions, it will be centered on the image. +- `boardImage`: the output image with the board. +- The third parameter is the (optional) margin in pixels, so none of the markers are touching the image border. +In this case the margin is 10. +- Finally, the size of the marker border, similarly to `generateImageMarker()` function. The default value is 1. + +A full working example of board creation is included in the `samples/cpp/tutorial_code/objectDetection/create_board.cpp` + +The output image will be something like this: + +![](images/board.png) + +The samples now take input via commandline via the `cv::CommandLineParser`. For this file the example +parameters will look like: +@code{.cpp} + "_output_path_/aboard.png" -w=5 -h=7 -l=100 -s=10 -d=10 +@endcode + +Refine marker detection +----------------------- + +ArUco boards can also be used to improve the detection of markers. If we have detected a subset of the markers +that belongs to the board, we can use these markers and the board layout information to try to find the +markers that have not been previously detected. + +This can be done using the `cv::aruco::refineDetectedMarkers()` function, which should be called +after calling `cv::aruco::ArucoDetector::detectMarkers()`. + +The main parameters of this function are the original image where markers were detected, the board object, +the detected marker corners, the detected marker ids and the rejected marker corners. + +The rejected corners can be obtained from the `cv::aruco::ArucoDetector::detectMarkers()` function and +are also known as marker candidates. This candidates are square shapes that have been found in the +original image but have failed to pass the identification step (i.e. their inner codification presents +too many errors) and thus they have not been recognized as markers. + +However, these candidates are sometimes actual markers that have not been correctly identified due to high +noise in the image, very low resolution or other related problems that affect to the binary code extraction. +The `cv::aruco::ArucoDetector::refineDetectedMarkers()` function finds correspondences between these +candidates and the missing markers of the board. This search is based on two parameters: + +- Distance between the candidate and the projection of the missing marker. To obtain these projections, +it is necessary to have detected at least one marker of the board. The projections are obtained using the +camera parameters (camera matrix and distortion coefficients) if they are provided. If not, the projections +are obtained from local homography and only planar board are allowed (i.e. the Z coordinate of all the +marker corners should be the same). The `minRepDistance` parameter in `refineDetectedMarkers()` +determines the minimum euclidean distance between the candidate corners and the projected marker corners +(default value 10). + +- Binary codification. If a candidate surpasses the minimum distance condition, its internal bits +are analyzed again to determine if it is actually the projected marker or not. However, in this case, +the condition is not so strong and the number of allowed erroneous bits can be higher. This is indicated +in the `errorCorrectionRate` parameter (default value 3.0). If a negative value is provided, the +internal bits are not analyzed at all and only the corner distances are evaluated. + +This is an example of using the `cv::aruco::ArucoDetector::refineDetectedMarkers()` function: + +@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_detect_and_refine + +It must also be noted that, in some cases, if the number of detected markers in the first place is +too low (for instance only 1 or 2 markers), the projections of the missing markers can be of bad +quality, producing erroneous correspondences. + +See module samples for a more detailed implementation. diff --git a/doc/tutorials/objdetect/aruco_board_detection/images/board.png b/doc/tutorials/objdetect/aruco_board_detection/images/board.png new file mode 100644 index 0000000000..0f2c25b559 Binary files /dev/null and b/doc/tutorials/objdetect/aruco_board_detection/images/board.png differ diff --git a/doc/tutorials/objdetect/aruco_board_detection/images/gbaxis.jpg b/doc/tutorials/objdetect/aruco_board_detection/images/gbaxis.jpg new file mode 100644 index 0000000000..ce14343784 Binary files /dev/null and b/doc/tutorials/objdetect/aruco_board_detection/images/gbaxis.jpg differ diff --git a/doc/tutorials/objdetect/aruco_board_detection/images/gbmarkersaxis.jpg b/doc/tutorials/objdetect/aruco_board_detection/images/gbmarkersaxis.jpg new file mode 100644 index 0000000000..d1d5fcba49 Binary files /dev/null and b/doc/tutorials/objdetect/aruco_board_detection/images/gbmarkersaxis.jpg differ diff --git a/doc/tutorials/objdetect/aruco_board_detection/images/gbocclusion.jpg b/doc/tutorials/objdetect/aruco_board_detection/images/gbocclusion.jpg new file mode 100644 index 0000000000..9222b7d2b9 Binary files /dev/null and b/doc/tutorials/objdetect/aruco_board_detection/images/gbocclusion.jpg differ diff --git a/doc/tutorials/objdetect/aruco_board_detection/images/gboriginal.jpg b/doc/tutorials/objdetect/aruco_board_detection/images/gboriginal.jpg new file mode 100644 index 0000000000..8343139aed Binary files /dev/null and b/doc/tutorials/objdetect/aruco_board_detection/images/gboriginal.jpg differ diff --git a/modules/objdetect/tutorials/images/singlemarkersaxes.jpg b/doc/tutorials/objdetect/aruco_board_detection/images/singlemarkersaxes.jpg similarity index 100% rename from modules/objdetect/tutorials/images/singlemarkersaxes.jpg rename to doc/tutorials/objdetect/aruco_board_detection/images/singlemarkersaxes.jpg diff --git a/doc/tutorials/objdetect/aruco_detection/aruco_detection.markdown b/doc/tutorials/objdetect/aruco_detection/aruco_detection.markdown new file mode 100644 index 0000000000..d3ac745160 --- /dev/null +++ b/doc/tutorials/objdetect/aruco_detection/aruco_detection.markdown @@ -0,0 +1,702 @@ +Detection of ArUco Markers {#tutorial_aruco_detection} +========================== + +@next_tutorial{tutorial_aruco_board_detection} + +| | | +| -: | :- | +| Original authors | Sergio Garrido, Alexander Panov | +| Compatibility | OpenCV >= 4.7.0 | + +Pose estimation is of great importance in many computer vision applications: robot navigation, +augmented reality, and many more. This process is based on finding correspondences between points in +the real environment and their 2d image projection. This is usually a difficult step, and thus it is +common to use synthetic or fiducial markers to make it easier. + +One of the most popular approaches is the use of binary square fiducial markers. The main benefit +of these markers is that a single marker provides enough correspondences (its four corners) +to obtain the camera pose. Also, the inner binary codification makes them specially robust, allowing +the possibility of applying error detection and correction techniques. + +The aruco module is based on the [ArUco library](http://www.uco.es/investiga/grupos/ava/node/26), +a popular library for detection of square fiducial markers developed by Rafael Muñoz and Sergio Garrido @cite Aruco2014. + +The aruco functionalities are included in: +@code{.cpp} +#include +@endcode + + +Markers and Dictionaries +------------------------ + +An ArUco marker is a synthetic square marker composed by a wide black border and an inner binary +matrix which determines its identifier (id). The black border facilitates its fast detection in the +image and the binary codification allows its identification and the application of error detection +and correction techniques. The marker size determines the size of the internal matrix. For instance +a marker size of 4x4 is composed by 16 bits. + +Some examples of ArUco markers: + +![Example of markers images](images/markers.jpg) + +It must be noted that a marker can be found rotated in the environment, however, the detection +process needs to be able to determine its original rotation, so that each corner is identified +unequivocally. This is also done based on the binary codification. + +A dictionary of markers is the set of markers that are considered in a specific application. It is +simply the list of binary codifications of each of its markers. + +The main properties of a dictionary are the dictionary size and the marker size. + +- The dictionary size is the number of markers that compose the dictionary. +- The marker size is the size of those markers (the number of bits/modules). + +The aruco module includes some predefined dictionaries covering a range of different dictionary +sizes and marker sizes. + +One may think that the marker id is the number obtained from converting the binary codification to +a decimal base number. However, this is not possible since for high marker sizes the number of bits +is too high and managing such huge numbers is not practical. Instead, a marker id is simply +the marker index within the dictionary it belongs to. For instance, the first 5 markers in a +dictionary have the ids: 0, 1, 2, 3 and 4. + +More information about dictionaries is provided in the "Selecting a dictionary" section. + + +Marker Creation +--------------- + +Before their detection, markers need to be printed in order to be placed in the environment. +Marker images can be generated using the `generateImageMarker()` function. + +For example, lets analyze the following call: + +@code{.cpp} +cv::Mat markerImage; +cv::aruco::Dictionary dictionary = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_250); +cv::aruco::generateImageMarker(dictionary, 23, 200, markerImage, 1); +cv::imwrite("marker23.png", markerImage); +@endcode + +First, the `cv::aruco::Dictionary` object is created by choosing one of the predefined dictionaries in the aruco module. +Concretely, this dictionary is composed of 250 markers and a marker size of 6x6 bits (`cv::aruco::DICT_6X6_250`). + +The parameters of `cv::aruco::generateImageMarker()` are: + +- The first parameter is the `cv::aruco::Dictionary` object previously created. +- The second parameter is the marker id, in this case the marker 23 of the dictionary `cv::aruco::DICT_6X6_250`. +Note that each dictionary is composed of a different number of markers. In this case, the valid ids +go from 0 to 249. Any specific id out of the valid range will produce an exception. +- The third parameter, 200, is the size of the output marker image. In this case, the output image +will have a size of 200x200 pixels. Note that this parameter should be large enough to store the +number of bits for the specific dictionary. So, for instance, you cannot generate an image of +5x5 pixels for a marker size of 6x6 bits (and that is without considering the marker border). +Furthermore, to avoid deformations, this parameter should be proportional to the number of bits + +border size, or at least much higher than the marker size (like 200 in the example), so that +deformations are insignificant. +- The fourth parameter is the output image. +- Finally, the last parameter is an optional parameter to specify the width of the marker black +border. The size is specified proportional to the number of bits. For instance a value of 2 means +that the border will have a width equivalent to the size of two internal bits. The default value +is 1. + +The generated image is: + +![Generated marker](images/marker23.png) + +A full working example is included in the `create_marker.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`. + +The samples now take input from the command line using cv::CommandLineParser. For this file the example +parameters will look like: +@code{.cpp} +"marker23.png" -d=10 -id=23 +@endcode +Parameters for `create_marker.cpp`: +@snippet samples/cpp/tutorial_code/objectDetection/create_marker.cpp aruco_create_markers_keys + +Marker Detection +---------------- + +Given an image containing ArUco markers, the detection process has to return a list of +detected markers. Each detected marker includes: + +- The position of its four corners in the image (in their original order). +- The id of the marker. + +The marker detection process is comprised of two main steps: + +1. Detection of marker candidates. In this step the image is analyzed in order to find square shapes +that are candidates to be markers. It begins with an adaptive thresholding to segment the markers, +then contours are extracted from the thresholded image and those that are not convex or do not +approximate to a square shape are discarded. Some extra filtering is also applied (removing contours +that are too small or too big, removing contours too close to each other, etc). + +2. After the candidate detection, it is necessary to determine if they are actually markers by +analyzing their inner codification. This step starts by extracting the marker bits of each marker. +To do so, a perspective transformation is first applied to obtain the marker in its canonical form. +Then, the canonical image is thresholded using Otsu to separate white and black bits. The image +is divided into different cells according to the marker size and the border size. Then the number +of black or white pixels in each cell is counted to determine if it is a white or a black bit. +Finally, the bits are analyzed to determine if the marker belongs to the specific dictionary. +Error correction techniques are employed when necessary. + + +Consider the following image: + +![Image with an assortment of markers](images/singlemarkerssource.jpg) + +And a printout of this image in a photo: + +![Original image with markers](images/singlemarkersoriginal.jpg) + +These are the detected markers (in green). Note that some markers are rotated. The small red square +indicates the marker’s top left corner: + +![Image with detected markers](images/singlemarkersdetection.jpg) + +And these are the marker candidates that have been rejected during the identification step (in pink): + +![Image with rejected candidates](images/singlemarkersrejected.jpg) + +In the aruco module, the detection is performed in the `cv::aruco::ArucoDetector::detectMarkers()` +function. This function is the most important in the module, since all the rest of the functionality +is based on the detected markers returned by `cv::aruco::ArucoDetector::detectMarkers()`. + +An example of marker detection: + +@code{.cpp} +cv::Mat inputImage; +// ... read inputImage ... +std::vector markerIds; +std::vector> markerCorners, rejectedCandidates; +cv::aruco::DetectorParameters detectorParams = cv::aruco::DetectorParameters(); +cv::aruco::Dictionary dictionary = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_250); +cv::aruco::ArucoDetector detector(dictionary, detectorParams); +detector.detectMarkers(inputImage, markerCorners, markerIds, rejectedCandidates); +@endcode + +When you create an `cv::aruco::ArucoDetector` object, you need to pass the following parameters to the constructor: + +- A dictionary object, in this case one of the predefined dictionaries (`cv::aruco::DICT_6X6_250`). +- Object of type `cv::aruco::DetectorParameters`. This object includes all parameters that can be customized during the detection process. +These parameters will be explained in the next section. + +The parameters of `cv::aruco::ArucoDetector::detectMarkers()` are: + +- The first parameter is the image containing the markers to be detected. +- The detected markers are stored in the `markerCorners` and `markerIds` structures: + - `markerCorners` is the list of corners of the detected markers. For each marker, its four + corners are returned in their original order (which is clockwise starting with top left). + So, the first corner is the top left corner, followed by the top right, bottom right and bottom left. + - `markerIds` is the list of ids of each of the detected markers in `markerCorners`. + Note that the returned `markerCorners` and `markerIds` vectors have the same size. +- The final optional parameter, `rejectedCandidates`, is a returned list of marker candidates, i.e. +shapes that were found and considered but did not contain a valid marker. Each candidate is also +defined by its four corners, and its format is the same as the `markerCorners` parameter. This +parameter can be omitted and is only useful for debugging purposes and for ‘refind’ strategies +(see `cv::aruco::ArucoDetector::refineDetectedMarkers()`). + + +The next thing you probably want to do after `cv::aruco::ArucoDetector::detectMarkers()` is check that +your markers have been correctly detected. Fortunately, the aruco module provides a function to draw +the detected markers in the input image, this function is `drawDetectedMarkers()`. For example: + +@code{.cpp} +cv::Mat outputImage = inputImage.clone(); +cv::aruco::drawDetectedMarkers(outputImage, markerCorners, markerIds); +@endcode + +- `outputImage ` is the input/output image where the markers will be drawn (it will normally be + the same as the image where the markers were detected). +- `markerCorners` and `markerIds` are the structures of the detected markers returned by the + `cv::aruco::ArucoDetector::detectMarkers()` function. + +![Image with detected markers](images/singlemarkersdetection.jpg) + +Note that this function is only provided for visualization and its use can be omitted. + +With these two functions we can create a basic marker detection loop to detect markers from our +camera: + +@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_detect_markers + +Note that some of the optional parameters have been omitted, like the detection parameter object and the +output vector of rejected candidates. + +A full working example is included in the `detect_markers.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`. + +The samples now take input from the command line using cv::CommandLineParser. For this file +the example parameters will look like: +@code{.cpp} +-v=/path_to_opencv/opencv/doc/tutorials/objdetect/aruco_detection/images/singlemarkersoriginal.jpg -d=10 +@endcode +Parameters for `detect_markers.cpp`: +@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_detect_markers_keys + + +Pose Estimation +--------------- + +The next thing you'll probably want to do after detecting the markers is to use them to get the camera pose. + +To perform camera pose estimation, you need to know your camera's calibration parameters. These are +the camera matrix and distortion coefficients. If you do not know how to calibrate your camera, +you can take a look at the `calibrateCamera()` function and the Calibration tutorial of OpenCV. +You can also calibrate your camera using the aruco module as explained in the **Calibration with ArUco and ChArUco** +tutorial. Note that this only needs to be done once unless the camera optics are modified +(for instance changing its focus). + +As a result of the calibration, you get a camera matrix: a matrix of 3x3 elements with the +focal distances and the camera center coordinates (a.k.a intrinsic parameters), and the distortion +coefficients: a vector of 5 or more elements that models the distortion produced by your camera. + +When you estimate the pose with ArUco markers, you can estimate the pose of each marker individually. +If you want to estimate one pose from a set of markers, use ArUco Boards (see the **Detection of ArUco +Boards** tutorial). Using ArUco boards instead of single markers allows some markers to be occluded. + +The camera pose relative to the marker is a 3d transformation from the marker coordinate system to the +camera coordinate system. It is specified by rotation and translation vectors. OpenCV provides +`cv::solvePnP()` function to do that. + +@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_pose_estimation1 +@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_pose_estimation2 +@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_pose_estimation3 + +- The `corners` parameter is the vector of marker corners returned by the `cv::aruco::ArucoDetector::detectMarkers()` function. +- The second parameter is the size of the marker side in meters or in any other unit. Note that the + translation vectors of the estimated poses will be in the same units. +- `camMatrix` and `distCoeffs` are the camera calibration parameters that were created during + the camera calibration process. +- The output parameters `rvecs` and `tvecs` are the rotation and translation vectors respectively, + for each of the detected markers in `corners`. + +The marker coordinate system that is assumed by this function is placed in the center (by default) or +in the top left corner of the marker with the Z axis pointing out, as in the following image. +Axis-color correspondences are X: red, Y: green, Z: blue. Note the axis directions of the rotated +markers in this image. + +![Image with axes drawn](images/singlemarkersaxes.jpg) + +OpenCV provides a function to draw the axis as in the image above, so pose estimation can be +checked: + +@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_draw_pose_estimation + +- `imageCopy` is the input/output image where the detected markers will be shown. +- `camMatrix` and `distCoeffs` are the camera calibration parameters. +- `rvecs[i]` and `tvecs[i]` are the rotation and translation vectors respectively, for each of the detected markers. +- The last parameter is the length of the axis, in the same unit as tvec (usually meters). + +Sample video: + +@youtube{IsXWrcB_Hvs} + +A full working example is included in the `detect_markers.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`. + +The samples now take input from the command line using cv::CommandLineParser. For this file +the example parameters will look like: +@code{.cpp} +-v=/path_to_opencv/opencv/doc/tutorials/objdetect/aruco_detection/images/singlemarkersoriginal.jpg -d=10 +-c=/path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/tutorial_camera_params.yml +@endcode +Parameters for `detect_markers.cpp`: +@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_detect_markers_keys + +Selecting a dictionary +---------------------- + +The aruco module provides the `Dictionary` class to represent a dictionary of markers. + +In addition to the marker size and the number of markers in the dictionary, there is another important +parameter of the dictionary - the inter-marker distance. The inter-marker distance is the minimum +Hamming distance between dictionary markers that determines the dictionary's ability to detect and +correct errors. + +In general, smaller dictionary sizes and larger marker sizes increase the inter-marker distance and +vice versa. However, the detection of markers with larger sizes is more difficult due to the higher +number of bits that need to be extracted from the image. + +For instance, if you need only 10 markers in your application, it is better to use a dictionary composed +only of those 10 markers than using a dictionary composed of 1000 markers. The reason is that +the dictionary composed of 10 markers will have a higher inter-marker distance and, thus, it will be +more robust to errors. + +As a consequence, the aruco module includes several ways to select your dictionary of markers, so that +you can increase your system robustness: + +### Predefined dictionaries + +This is the easiest way to select a dictionary. The aruco module includes a set of predefined +dictionaries in a variety of marker sizes and number of markers. For instance: + +@code{.cpp} +cv::aruco::Dictionary dictionary = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_250); +@endcode + +`cv::aruco::DICT_6X6_250` is an example of predefined dictionary of markers with 6x6 bits and a total of 250 +markers. + +From all the provided dictionaries, it is recommended to choose the smallest one that fits your application. +For instance, if you need 200 markers of 6x6 bits, it is better to use `cv::aruco::DICT_6X6_250` than `cv::aruco::DICT_6X6_1000`. +The smaller the dictionary, the higher the inter-marker distance. + +The list of available predefined dictionaries can be found in the documentation for the `PredefinedDictionaryType` enum. + +### Automatic dictionary generation + +A dictionary can be generated automatically to adjust the desired number of markers and bits +to optimize the inter-marker distance: + +@code{.cpp} +cv::aruco::Dictionary dictionary = cv::aruco::extendDictionary(36, 5); +@endcode + +This will generate a customized dictionary composed of 36 markers of 5x5 bits. The process can take several +seconds, depending on the parameters (it is slower for larger dictionaries and higher numbers of bits). + +Also you could use `aruco_dict_utils.cpp` sample inside the `opencv/samples/cpp`. This sample calculates +the minimum Hamming distance for the generated dictionary and also allows you to create markers that are +resistant to reflection. + +### Manual dictionary definition + +Finally, the dictionary can be configured manually, so that any encoding can be used. To do that, +the `cv::aruco::Dictionary` object parameters need to be assigned manually. It must be noted that, +unless you have a special reason to do this manually, it is preferable to use one of the previous alternatives. + +The `cv::aruco::Dictionary` parameters are: + +@code{.cpp} + class Dictionary { + public: + + cv::Mat bytesList; // marker code information + int markerSize; // number of bits per dimension + int maxCorrectionBits; // maximum number of bits that can be corrected + + ... + + } +@endcode + +`bytesList` is the array that contains all the information about the marker codes. `markerSize` is the size + of each marker dimension (for instance, 5 for markers with 5x5 bits). Finally, `maxCorrectionBits` is +the maximum number of erroneous bits that can be corrected during the marker detection. If this value is too +high, it can lead to a high number of false positives. + +Each row in `bytesList` represents one of the dictionary markers. However, the markers are not stored in their +binary form, instead they are stored in a special format to simplify their detection. + +Fortunately, a marker can be easily transformed to this form using the static method `Dictionary::getByteListFromBits()`. + +For example: + +@code{.cpp} + cv::aruco::Dictionary dictionary; + + // Markers of 6x6 bits + dictionary.markerSize = 6; + + // Maximum number of bit corrections + dictionary.maxCorrectionBits = 3; + + // Let's create a dictionary of 100 markers + for(int i = 0; i < 100; i++) + { + // Assume generateMarkerBits() generates a new marker in binary format, so that + // markerBits is a 6x6 matrix of CV_8UC1 type, only containing 0s and 1s + cv::Mat markerBits = generateMarkerBits(); + cv::Mat markerCompressed = cv::aruco::Dictionary::getByteListFromBits(markerBits); + + // Add the marker as a new row + dictionary.bytesList.push_back(markerCompressed); + } +@endcode + +Detector Parameters +------------------- + +One of the parameters of `cv::aruco::ArucoDetector` is a `cv::aruco::DetectorParameters` object. This object +includes all the options that can be customized during the marker detection process. + +This section describes each detector parameter. The parameters can be classified depending on +the process in which they’re involved: + +### Thresholding + +One of the first steps in the marker detection process is adaptive thresholding of the input image. + +For instance, the thresholded image for the sample image used above is: + +![Thresholded image](images/singlemarkersthresh.png) + +This thresholding can be customized with the following parameters: + +#### adaptiveThreshWinSizeMin, adaptiveThreshWinSizeMax, and adaptiveThreshWinSizeStep + +The `adaptiveThreshWinSizeMin` and `adaptiveThreshWinSizeMax` parameters represent the interval where the +thresholding window sizes (in pixels) are selected for the adaptive thresholding (see OpenCV +`threshold()` and `adaptiveThreshold()` functions for more details). + +The parameter `adaptiveThreshWinSizeStep` indicates the increments of the window size from +`adaptiveThreshWinSizeMin` to `adaptiveThreshWinSizeMax`. + +For instance, for the values `adaptiveThreshWinSizeMin` = 5 and `adaptiveThreshWinSizeMax` = 21 and +`adaptiveThreshWinSizeStep` = 4, there will be 5 thresholding steps with window sizes 5, 9, 13, 17 and 21. +On each thresholding image, marker candidates will be extracted. + +Low values of window size can "break" the marker border if the marker size is too large, causing it to not be detected, as in the following image: + +![Broken marker image](images/singlemarkersbrokenthresh.png) + +On the other hand, too large values can produce the same effect if the markers are too small, and can also +reduce the performance. Moreover the process will tend to global thresholding, resulting in a loss of adaptive benefits. + +The simplest case is using the same value for `adaptiveThreshWinSizeMin` and + `adaptiveThreshWinSizeMax`, which produces a single thresholding step. However, it is usually better to use a + range of values for the window size, although many thresholding steps can also reduce the performance considerably. + +@see cv::aruco::DetectorParameters::adaptiveThreshWinSizeMin, cv::aruco::DetectorParameters::adaptiveThreshWinSizeMax, +cv::aruco::DetectorParameters::adaptiveThreshWinSizeStep + +#### adaptiveThreshConstant + +The `adaptiveThreshConstant` parameter represents the constant value added in the thresholding operation (see OpenCV +`threshold()` and `adaptiveThreshold()` functions for more details). Its default value is a good option in most cases. + +@see cv::aruco::DetectorParameters::adaptiveThreshConstant + + +### Contour filtering + +After thresholding, contours are detected. However, not all contours +are considered as marker candidates. They are filtered out in different steps so that contours that are +very unlikely to be markers are discarded. The parameters in this section customize +this filtering process. + +It must be noted that in most cases it is a question of balance between detection capacity +and performance. All the considered contours will be processed in the following stages, which usually have +a higher computational cost. So, it is preferred to discard invalid candidates in this stage than in the later stages. + +On the other hand, if the filtering conditions are too strict, the real marker contours could be discarded and, +hence, not detected. + +#### minMarkerPerimeterRate and maxMarkerPerimeterRate + +These parameters determine the minimum and maximum size of a marker, specifically the minimum and +maximum marker perimeter. They are not specified in absolute pixel values, instead they are specified +relative to the maximum dimension of the input image. + +For instance, a image with size 640x480 and a minimum relative marker perimeter of 0.05 will lead +to a minimum marker perimeter of 640x0.05 = 32 pixels, since 640 is the maximum dimension of the +image. The same applies for the `maxMarkerPerimeterRate` parameter. + +If the `minMarkerPerimeterRate` is too low, detection performance can be significantly reduced, +as many more contours will be considered for future stages. +This penalization is not so noticeable for the `maxMarkerPerimeterRate` parameter, since there are +usually many more small contours than big contours. +A `minMarkerPerimeterRate` value of 0 and a `maxMarkerPerimeterRate` value of 4 (or more) will be +equivalent to consider all the contours in the image, however this is not recommended for +performance reasons. + +@see cv::aruco::DetectorParameters::minMarkerPerimeterRate, cv::aruco::DetectorParameters::maxMarkerPerimeterRate + +#### polygonalApproxAccuracyRate + +A polygonal approximation is applied to each candidate and only those that approximate to a square +shape are accepted. This value determines the maximum error that the polygonal approximation can +produce (see `approxPolyDP()` function for more information). + +This parameter is relative to the candidate length (in pixels). So if the candidate has +a perimeter of 100 pixels and the value of `polygonalApproxAccuracyRate` is 0.04, the maximum error +would be 100x0.04=5.4 pixels. + +In most cases, the default value works fine, but higher error values could be necessary for highly +distorted images. + +@see cv::aruco::DetectorParameters::polygonalApproxAccuracyRate + +#### minCornerDistanceRate + +Minimum distance between any pair of corners in the same marker. It is expressed relative to the marker +perimeter. Minimum distance in pixels is Perimeter * minCornerDistanceRate. + +@see cv::aruco::DetectorParameters::minCornerDistanceRate + +#### minMarkerDistanceRate + +Minimum distance between any pair of corners from two different markers. It is expressed relative to +the minimum marker perimeter of the two markers. If two candidates are too close, the smaller one is ignored. + +@see cv::aruco::DetectorParameters::minMarkerDistanceRate + +#### minDistanceToBorder + +Minimum distance to any of the marker corners to the image border (in pixels). Markers partially occluded +by the image border can be correctly detected if the occlusion is small. However, if one of the corners +is occluded, the returned corner is usually placed in a wrong position near the image border. + +If the position of marker corners is important, for instance if you want to do pose estimation, it is +better to discard any markers whose corners are too close to the image border. Elsewhere, it is not necessary. + +@see cv::aruco::DetectorParameters::minDistanceToBorder + +### Bits Extraction + +After candidate detection, the bits of each candidate are analyzed in order to determine if they +are markers or not. + +Before analyzing the binary code itself, the bits need to be extracted. To do this, perspective +distortion is corrected and the resulting image is thresholded using Otsu threshold to separate +black and white pixels. + +This is an example of the image obtained after removing the perspective distortion of a marker: + +![Perspective removing](images/removeperspective.jpg) + +Then, the image is divided into a grid with the same number of cells as the number of bits in the marker. +In each cell, the number of black and white pixels are counted to determine the bit value assigned +to the cell (from the majority value): + +![Marker cells](images/bitsextraction1.png) + +There are several parameters that can customize this process: + +#### markerBorderBits + +This parameter indicates the width of the marker border. It is relative to the size of each bit. So, a +value of 2 indicates the border has the width of two internal bits. + +This parameter needs to coincide with the border size of the markers you are using. The border size +can be configured in the marker drawing functions such as `generateImageMarker()`. + +@see cv::aruco::DetectorParameters::markerBorderBits + +#### minOtsuStdDev + +This value determines the minimum standard deviation of the pixel values to perform Otsu +thresholding. If the deviation is low, it probably means that all the square is black (or white) +and applying Otsu does not make sense. If this is the case, all the bits are set to 0 (or 1) +depending on whether the mean value is higher or lower than 128. + +@see cv::aruco::DetectorParameters::minOtsuStdDev + +#### perspectiveRemovePixelPerCell + +This parameter determines the number of pixels (per cell) in the obtained image after correcting perspective +distortion (including the border). This is the size of the red squares in the image above. + +For instance, let’s assume we are dealing with markers of 5x5 bits and border size of 1 bit +(see `markerBorderBits`). Then, the total number of cells/bits per dimension is 5 + 2*1 = 7 (the border +has to be counted twice). The total number of cells is 7x7. + +If the value of `perspectiveRemovePixelPerCell` is 10, then the size of the obtained image will be +10*7 = 70 -> 70x70 pixels. + +A higher value of this parameter can improve the bits extraction process (up to some degree), +however it can penalize the performance. + +@see cv::aruco::DetectorParameters::perspectiveRemovePixelPerCell + +#### perspectiveRemoveIgnoredMarginPerCell + +When extracting the bits of each cell, the numbers of black and white pixels are counted. In general, it is +not recommended to consider all the cell pixels. Instead it is better to ignore some pixels in the +margins of the cells. + +The reason for this is that, after removing the perspective distortion, the cells’ colors are, in general, not +perfectly separated and white cells can invade some pixels of black cells (and vice versa). Thus, it is +better to ignore some pixels just to avoid counting erroneous pixels. + +For instance, in the following image: + +![Marker cell margins](images/bitsextraction2.png) + +only the pixels inside the green squares are considered. It can be seen in the right image that +the resulting pixels contain a lower amount of noise from neighbor cells. +The `perspectiveRemoveIgnoredMarginPerCell` parameter indicates the difference between the red and +the green squares. + +This parameter is relative to the total size of the cell. For instance if the cell size is 40 pixels and the +value of this parameter is 0.1, a margin of 40*0.1=4 pixels is ignored in the cells. This means that the total +number of pixels that would be analyzed in each cell would actually be 32x32, instead of 40x40. + +@see cv::aruco::DetectorParameters::perspectiveRemoveIgnoredMarginPerCell + + +### Marker identification + +After the bits have been extracted, the next step is checking whether the extracted code belongs to the marker +dictionary and, if necessary, error correction can be performed. + +#### maxErroneousBitsInBorderRate + +The bits of the marker border should be black. This parameter specifies the allowed number of erroneous +bits in the border, i.e. the maximum number of white bits in the border. It is represented +relative to the total number of bits in the marker. + +@see cv::aruco::DetectorParameters::maxErroneousBitsInBorderRate + +#### errorCorrectionRate + +Each marker dictionary has a theoretical maximum number of bits that can be corrected (`Dictionary.maxCorrectionBits`). +However, this value can be modified by the `errorCorrectionRate` parameter. + +For instance, if the allowed number of bits that can be corrected (for the used dictionary) is 6 and the value of `errorCorrectionRate` is +0.5, the real maximum number of bits that can be corrected is 6*0.5=3 bits. + +This value is useful to reduce the error correction capabilities in order to avoid false positives. + +@see cv::aruco::DetectorParameters::errorCorrectionRate + + +### Corner Refinement + +After markers have been detected and identified, the last step is performing subpixel refinement +of the corner positions (see OpenCV `cornerSubPix()` and `cv::aruco::CornerRefineMethod`). + +Note that this step is optional and it only makes sense if the positions of the marker corners have to +be accurate, for instance for pose estimation. It is usually a time-consuming step and therefore is disabled by default. + +#### cornerRefinementMethod + +This parameter determines whether the corner subpixel process is performed or not and which method to use +if it is being performed. It can be disabled if accurate corners are not necessary. Possible values are +`CORNER_REFINE_NONE`, `CORNER_REFINE_SUBPIX`, `CORNER_REFINE_CONTOUR`, and `CORNER_REFINE_APRILTAG`. + +@see cv::aruco::DetectorParameters::cornerRefinementMethod + +#### cornerRefinementWinSize + +This parameter determines the maximum window size for the corner refinement process. + +High values can cause close corners of the image to be included in the window area, so that the corner +of the marker moves to a different and incorrect location during the process. Also, it may affect performance. +The window size may decrease if the ArUco marker is too small, check cv::aruco::DetectorParameters::relativeCornerRefinmentWinSize. +The final window size is calculated as: min(cornerRefinementWinSize, averageArucoModuleSize*relativeCornerRefinmentWinSize), +where averageArucoModuleSize is average module size of ArUco marker in pixels. + +@see cv::aruco::DetectorParameters::cornerRefinementWinSize + +#### relativeCornerRefinmentWinSize + +Dynamic window size for corner refinement relative to Aruco module size (default 0.3). + +The final window size is calculated as: min(cornerRefinementWinSize, averageArucoModuleSize*relativeCornerRefinmentWinSize), +where averageArucoModuleSize is average module size of ArUco marker in pixels. +In the case of markers located far from each other, it may be useful to increase the value of the parameter to 0.4-0.5. +In the case of markers located close to each other, it may be useful to decrease the parameter value to 0.1-0.2. + +@see cv::aruco::DetectorParameters::relativeCornerRefinmentWinSize + +#### cornerRefinementMaxIterations and cornerRefinementMinAccuracy + +These two parameters determine the stop criteria of the subpixel refinement process. The +`cornerRefinementMaxIterations` indicates the maximum number of iterations and +`cornerRefinementMinAccuracy` the minimum error value before stopping the process. + +If the number of iterations is too high, it may affect the performance. On the other hand, if it is +too low, it can result in poor subpixel refinement. + +@see cv::aruco::DetectorParameters::cornerRefinementMaxIterations, cv::aruco::DetectorParameters::cornerRefinementMinAccuracy diff --git a/doc/tutorials/objdetect/aruco_detection/images/bitsextraction1.png b/doc/tutorials/objdetect/aruco_detection/images/bitsextraction1.png new file mode 100644 index 0000000000..53c2d38c65 Binary files /dev/null and b/doc/tutorials/objdetect/aruco_detection/images/bitsextraction1.png differ diff --git a/doc/tutorials/objdetect/aruco_detection/images/bitsextraction2.png b/doc/tutorials/objdetect/aruco_detection/images/bitsextraction2.png new file mode 100644 index 0000000000..d3e8fb0507 Binary files /dev/null and b/doc/tutorials/objdetect/aruco_detection/images/bitsextraction2.png differ diff --git a/doc/tutorials/objdetect/aruco_detection/images/marker23.png b/doc/tutorials/objdetect/aruco_detection/images/marker23.png new file mode 100644 index 0000000000..f82555576e Binary files /dev/null and b/doc/tutorials/objdetect/aruco_detection/images/marker23.png differ diff --git a/doc/tutorials/objdetect/aruco_detection/images/markers.jpg b/doc/tutorials/objdetect/aruco_detection/images/markers.jpg new file mode 100644 index 0000000000..aa213f536e Binary files /dev/null and b/doc/tutorials/objdetect/aruco_detection/images/markers.jpg differ diff --git a/doc/tutorials/objdetect/aruco_detection/images/removeperspective.jpg b/doc/tutorials/objdetect/aruco_detection/images/removeperspective.jpg new file mode 100644 index 0000000000..8eeeb75578 Binary files /dev/null and b/doc/tutorials/objdetect/aruco_detection/images/removeperspective.jpg differ diff --git a/doc/tutorials/objdetect/aruco_detection/images/singlemarkersbrokenthresh.png b/doc/tutorials/objdetect/aruco_detection/images/singlemarkersbrokenthresh.png new file mode 100644 index 0000000000..e60f98b342 Binary files /dev/null and b/doc/tutorials/objdetect/aruco_detection/images/singlemarkersbrokenthresh.png differ diff --git a/doc/tutorials/objdetect/aruco_detection/images/singlemarkersdetection.jpg b/doc/tutorials/objdetect/aruco_detection/images/singlemarkersdetection.jpg new file mode 100644 index 0000000000..48995077d6 Binary files /dev/null and b/doc/tutorials/objdetect/aruco_detection/images/singlemarkersdetection.jpg differ diff --git a/doc/tutorials/objdetect/aruco_detection/images/singlemarkersoriginal.jpg b/doc/tutorials/objdetect/aruco_detection/images/singlemarkersoriginal.jpg new file mode 100644 index 0000000000..a0c9c43be4 Binary files /dev/null and b/doc/tutorials/objdetect/aruco_detection/images/singlemarkersoriginal.jpg differ diff --git a/doc/tutorials/objdetect/aruco_detection/images/singlemarkersrejected.jpg b/doc/tutorials/objdetect/aruco_detection/images/singlemarkersrejected.jpg new file mode 100644 index 0000000000..2b51e3913c Binary files /dev/null and b/doc/tutorials/objdetect/aruco_detection/images/singlemarkersrejected.jpg differ diff --git a/doc/tutorials/objdetect/aruco_detection/images/singlemarkerssource.jpg b/doc/tutorials/objdetect/aruco_detection/images/singlemarkerssource.jpg new file mode 100644 index 0000000000..a95b9e7dd9 Binary files /dev/null and b/doc/tutorials/objdetect/aruco_detection/images/singlemarkerssource.jpg differ diff --git a/doc/tutorials/objdetect/aruco_detection/images/singlemarkersthresh.png b/doc/tutorials/objdetect/aruco_detection/images/singlemarkersthresh.png new file mode 100644 index 0000000000..94738c952e Binary files /dev/null and b/doc/tutorials/objdetect/aruco_detection/images/singlemarkersthresh.png differ diff --git a/doc/tutorials/objdetect/table_of_content_objdetect.markdown b/doc/tutorials/objdetect/table_of_content_objdetect.markdown new file mode 100644 index 0000000000..2774d3ef91 --- /dev/null +++ b/doc/tutorials/objdetect/table_of_content_objdetect.markdown @@ -0,0 +1,5 @@ +Object Detection (objdetect module) {#tutorial_table_of_content_objdetect} +========================================================== + +- @subpage tutorial_aruco_detection +- @subpage tutorial_aruco_board_detection diff --git a/doc/tutorials/tutorials.markdown b/doc/tutorials/tutorials.markdown index 59aefc2b1f..cd83c241af 100644 --- a/doc/tutorials/tutorials.markdown +++ b/doc/tutorials/tutorials.markdown @@ -6,6 +6,7 @@ OpenCV Tutorials {#tutorial_root} - @subpage tutorial_table_of_content_imgproc - image processing functions - @subpage tutorial_table_of_content_app - application utils (GUI, image/video input/output) - @subpage tutorial_table_of_content_calib3d - extract 3D world information from 2D images +- @subpage tutorial_table_of_content_objdetect - INSERT OBJDETECT MODULE INFO - @subpage tutorial_table_of_content_features2d - feature detectors, descriptors and matching framework - @subpage tutorial_table_of_content_dnn - infer neural networks using built-in _dnn_ module - @subpage tutorial_table_of_content_gapi - graph-based approach to computer vision algorithms building diff --git a/modules/objdetect/include/opencv2/objdetect/charuco_detector.hpp b/modules/objdetect/include/opencv2/objdetect/charuco_detector.hpp index a23960d557..e10cb3f025 100644 --- a/modules/objdetect/include/opencv2/objdetect/charuco_detector.hpp +++ b/modules/objdetect/include/opencv2/objdetect/charuco_detector.hpp @@ -77,6 +77,9 @@ public: * If camera parameters are provided, the process is based in an approximated pose estimation, else it is based on local homography. * Only visible corners are returned. For each corner, its corresponding identifier is also returned in charucoIds. * @sa findChessboardCorners + * @note After OpenCV 4.6.0, there was an incompatible change in the ChArUco pattern generation algorithm for even row counts. + * Use cv::aruco::CharucoBoard::setLegacyPattern() to ensure compatibility with patterns created using OpenCV versions prior to 4.6.0. + * For more information, see the issue: https://github.com/opencv/opencv/issues/23152 */ CV_WRAP void detectBoard(InputArray image, OutputArray charucoCorners, OutputArray charucoIds, InputOutputArrayOfArrays markerCorners = noArray(), diff --git a/modules/objdetect/tutorials/images/singlemarkersaxes2.jpg b/modules/objdetect/tutorials/images/singlemarkersaxes2.jpg deleted file mode 100644 index dc8edee15d..0000000000 Binary files a/modules/objdetect/tutorials/images/singlemarkersaxes2.jpg and /dev/null differ diff --git a/samples/cpp/aruco_dict_utils.cpp b/samples/cpp/aruco_dict_utils.cpp index 4a33f15bbf..20fecd82e9 100644 --- a/samples/cpp/aruco_dict_utils.cpp +++ b/samples/cpp/aruco_dict_utils.cpp @@ -283,7 +283,7 @@ int main(int argc, char *argv[]) int markerSize = parser.get("markerSize"); bool checkFlippedMarkers = parser.get("r"); - aruco::Dictionary dictionary = aruco::getPredefinedDictionary(0); + aruco::Dictionary dictionary = aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_50); if (parser.has("d")) { string arucoDictName = parser.get("d"); diff --git a/samples/cpp/tutorial_code/calib3d/camera_calibration/camera_calibration.cpp b/samples/cpp/tutorial_code/calib3d/camera_calibration/camera_calibration.cpp index e46af18dcd..a80962b4c3 100644 --- a/samples/cpp/tutorial_code/calib3d/camera_calibration/camera_calibration.cpp +++ b/samples/cpp/tutorial_code/calib3d/camera_calibration/camera_calibration.cpp @@ -354,7 +354,7 @@ int main(int argc, char* argv[]) } else { // default dictionary - dictionary = cv::aruco::getPredefinedDictionary(0); + dictionary = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_50); } cv::aruco::CharucoBoard ch_board({s.boardSize.width, s.boardSize.height}, s.squareSize, s.markerSize, dictionary); cv::aruco::CharucoDetector ch_detector(ch_board); diff --git a/samples/cpp/tutorial_code/objectDetection/aruco_samples_utility.hpp b/samples/cpp/tutorial_code/objectDetection/aruco_samples_utility.hpp new file mode 100644 index 0000000000..3b28a91977 --- /dev/null +++ b/samples/cpp/tutorial_code/objectDetection/aruco_samples_utility.hpp @@ -0,0 +1,48 @@ +#include +#include +#include +#include + +namespace { +inline static bool readCameraParameters(const std::string& filename, cv::Mat &camMatrix, cv::Mat &distCoeffs) { + cv::FileStorage fs(filename, cv::FileStorage::READ); + if (!fs.isOpened()) + return false; + fs["camera_matrix"] >> camMatrix; + fs["distortion_coefficients"] >> distCoeffs; + return true; +} + +inline static bool saveCameraParams(const std::string &filename, cv::Size imageSize, float aspectRatio, int flags, + const cv::Mat &cameraMatrix, const cv::Mat &distCoeffs, double totalAvgErr) { + cv::FileStorage fs(filename, cv::FileStorage::WRITE); + if (!fs.isOpened()) + return false; + + time_t tt; + time(&tt); + struct tm *t2 = localtime(&tt); + char buf[1024]; + strftime(buf, sizeof(buf) - 1, "%c", t2); + + fs << "calibration_time" << buf; + fs << "image_width" << imageSize.width; + fs << "image_height" << imageSize.height; + + if (flags & cv::CALIB_FIX_ASPECT_RATIO) fs << "aspectRatio" << aspectRatio; + + if (flags != 0) { + sprintf(buf, "flags: %s%s%s%s", + flags & cv::CALIB_USE_INTRINSIC_GUESS ? "+use_intrinsic_guess" : "", + flags & cv::CALIB_FIX_ASPECT_RATIO ? "+fix_aspectRatio" : "", + flags & cv::CALIB_FIX_PRINCIPAL_POINT ? "+fix_principal_point" : "", + flags & cv::CALIB_ZERO_TANGENT_DIST ? "+zero_tangent_dist" : ""); + } + fs << "flags" << flags; + fs << "camera_matrix" << cameraMatrix; + fs << "distortion_coefficients" << distCoeffs; + fs << "avg_reprojection_error" << totalAvgErr; + return true; +} + +} diff --git a/samples/cpp/tutorial_code/objectDetection/create_board.cpp b/samples/cpp/tutorial_code/objectDetection/create_board.cpp new file mode 100644 index 0000000000..ead9f73857 --- /dev/null +++ b/samples/cpp/tutorial_code/objectDetection/create_board.cpp @@ -0,0 +1,95 @@ +#include +#include +#include +#include "aruco_samples_utility.hpp" + +using namespace cv; + +namespace { +const char* about = "Create an ArUco grid board image"; +const char* keys = + "{@outfile | | Output image }" + "{w | | Number of markers in X direction }" + "{h | | Number of markers in Y direction }" + "{l | | Marker side length (in pixels) }" + "{s | | Separation between two consecutive markers in the grid (in pixels)}" + "{d | | dictionary: DICT_4X4_50=0, DICT_4X4_100=1, DICT_4X4_250=2," + "DICT_4X4_1000=3, DICT_5X5_50=4, DICT_5X5_100=5, DICT_5X5_250=6, DICT_5X5_1000=7, " + "DICT_6X6_50=8, DICT_6X6_100=9, DICT_6X6_250=10, DICT_6X6_1000=11, DICT_7X7_50=12," + "DICT_7X7_100=13, DICT_7X7_250=14, DICT_7X7_1000=15, DICT_ARUCO_ORIGINAL = 16}" + "{cd | | Input file with custom dictionary }" + "{m | | Margins size (in pixels). Default is marker separation (-s) }" + "{bb | 1 | Number of bits in marker borders }" + "{si | false | show generated image }"; +} + + +int main(int argc, char *argv[]) { + CommandLineParser parser(argc, argv, keys); + parser.about(about); + + if(argc < 7) { + parser.printMessage(); + return 0; + } + + int markersX = parser.get("w"); + int markersY = parser.get("h"); + int markerLength = parser.get("l"); + int markerSeparation = parser.get("s"); + int margins = markerSeparation; + if(parser.has("m")) { + margins = parser.get("m"); + } + + int borderBits = parser.get("bb"); + bool showImage = parser.get("si"); + + String out = parser.get(0); + + if(!parser.check()) { + parser.printErrors(); + return 0; + } + + Size imageSize; + imageSize.width = markersX * (markerLength + markerSeparation) - markerSeparation + 2 * margins; + imageSize.height = + markersY * (markerLength + markerSeparation) - markerSeparation + 2 * margins; + + aruco::Dictionary dictionary = aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_50); + if (parser.has("d")) { + int dictionaryId = parser.get("d"); + dictionary = aruco::getPredefinedDictionary(aruco::PredefinedDictionaryType(dictionaryId)); + } + else if (parser.has("cd")) { + FileStorage fs(parser.get("cd"), FileStorage::READ); + bool readOk = dictionary.readDictionary(fs.root()); + if(!readOk) + { + std::cerr << "Invalid dictionary file" << std::endl; + return 0; + } + } + else { + std::cerr << "Dictionary not specified" << std::endl; + return 0; + } + + aruco::GridBoard board(Size(markersX, markersY), float(markerLength), float(markerSeparation), dictionary); + + // show created board + //! [aruco_generate_board_image] + Mat boardImage; + board.generateImage(imageSize, boardImage, margins, borderBits); + //! [aruco_generate_board_image] + + if(showImage) { + imshow("board", boardImage); + waitKey(0); + } + + imwrite(out, boardImage); + + return 0; +} diff --git a/samples/cpp/tutorial_code/objectDetection/create_marker.cpp b/samples/cpp/tutorial_code/objectDetection/create_marker.cpp new file mode 100644 index 0000000000..57b08b0ef7 --- /dev/null +++ b/samples/cpp/tutorial_code/objectDetection/create_marker.cpp @@ -0,0 +1,77 @@ +#include +#include +#include +#include "aruco_samples_utility.hpp" + +using namespace cv; + +namespace { +const char* about = "Create an ArUco marker image"; + +//! [aruco_create_markers_keys] +const char* keys = + "{@outfile | | Output image }" + "{d | | dictionary: DICT_4X4_50=0, DICT_4X4_100=1, DICT_4X4_250=2," + "DICT_4X4_1000=3, DICT_5X5_50=4, DICT_5X5_100=5, DICT_5X5_250=6, DICT_5X5_1000=7, " + "DICT_6X6_50=8, DICT_6X6_100=9, DICT_6X6_250=10, DICT_6X6_1000=11, DICT_7X7_50=12," + "DICT_7X7_100=13, DICT_7X7_250=14, DICT_7X7_1000=15, DICT_ARUCO_ORIGINAL = 16}" + "{cd | | Input file with custom dictionary }" + "{id | | Marker id in the dictionary }" + "{ms | 200 | Marker size in pixels }" + "{bb | 1 | Number of bits in marker borders }" + "{si | false | show generated image }"; +} +//! [aruco_create_markers_keys] + + +int main(int argc, char *argv[]) { + CommandLineParser parser(argc, argv, keys); + parser.about(about); + + if(argc < 4) { + parser.printMessage(); + return 0; + } + + int markerId = parser.get("id"); + int borderBits = parser.get("bb"); + int markerSize = parser.get("ms"); + bool showImage = parser.get("si"); + + String out = parser.get(0); + + if(!parser.check()) { + parser.printErrors(); + return 0; + } + + aruco::Dictionary dictionary = aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_50); + if (parser.has("d")) { + int dictionaryId = parser.get("d"); + dictionary = aruco::getPredefinedDictionary(aruco::PredefinedDictionaryType(dictionaryId)); + } + else if (parser.has("cd")) { + FileStorage fs(parser.get("cd"), FileStorage::READ); + bool readOk = dictionary.readDictionary(fs.root()); + if(!readOk) { + std::cerr << "Invalid dictionary file" << std::endl; + return 0; + } + } + else { + std::cerr << "Dictionary not specified" << std::endl; + return 0; + } + + Mat markerImg; + aruco::generateImageMarker(dictionary, markerId, markerSize, markerImg, borderBits); + + if(showImage) { + imshow("marker", markerImg); + waitKey(0); + } + + imwrite(out, markerImg); + + return 0; +} diff --git a/samples/cpp/tutorial_code/objectDetection/detect_board.cpp b/samples/cpp/tutorial_code/objectDetection/detect_board.cpp new file mode 100644 index 0000000000..ffea660e76 --- /dev/null +++ b/samples/cpp/tutorial_code/objectDetection/detect_board.cpp @@ -0,0 +1,201 @@ +#include +#include +#include +#include +#include "aruco_samples_utility.hpp" + +using namespace std; +using namespace cv; + +namespace { +const char* about = "Pose estimation using a ArUco Planar Grid board"; + +//! [aruco_detect_board_keys] +const char* keys = + "{w | | Number of squares in X direction }" + "{h | | Number of squares in Y direction }" + "{l | | Marker side length (in pixels) }" + "{s | | Separation between two consecutive markers in the grid (in pixels)}" + "{d | | dictionary: DICT_4X4_50=0, DICT_4X4_100=1, DICT_4X4_250=2," + "DICT_4X4_1000=3, DICT_5X5_50=4, DICT_5X5_100=5, DICT_5X5_250=6, DICT_5X5_1000=7, " + "DICT_6X6_50=8, DICT_6X6_100=9, DICT_6X6_250=10, DICT_6X6_1000=11, DICT_7X7_50=12," + "DICT_7X7_100=13, DICT_7X7_250=14, DICT_7X7_1000=15, DICT_ARUCO_ORIGINAL = 16}" + "{cd | | Input file with custom dictionary }" + "{c | | Output file with calibrated camera parameters }" + "{v | | Input from video or image file, if omitted, input comes from camera }" + "{ci | 0 | Camera id if input doesnt come from video (-v) }" + "{dp | | File of marker detector parameters }" + "{rs | | Apply refind strategy }" + "{r | | show rejected candidates too }"; +} +//! [aruco_detect_board_keys] + +static void readDetectorParamsFromCommandLine(CommandLineParser &parser, aruco::DetectorParameters& detectorParams) { + if(parser.has("dp")) { + FileStorage fs(parser.get("dp"), FileStorage::READ); + bool readOk = detectorParams.readDetectorParameters(fs.root()); + if(!readOk) { + cerr << "Invalid detector parameters file" << endl; + throw -1; + } + } +} + +static void readCameraParamsFromCommandLine(CommandLineParser &parser, Mat& camMatrix, Mat& distCoeffs) { + if(parser.has("c")) { + bool readOk = readCameraParameters(parser.get("c"), camMatrix, distCoeffs); + if(!readOk) { + cerr << "Invalid camera file" << endl; + throw -1; + } + } +} + +static void readDictionatyFromCommandLine(CommandLineParser &parser, aruco::Dictionary& dictionary) { + if (parser.has("d")) { + int dictionaryId = parser.get("d"); + dictionary = aruco::getPredefinedDictionary(aruco::PredefinedDictionaryType(dictionaryId)); + } + else if (parser.has("cd")) { + FileStorage fs(parser.get("cd"), FileStorage::READ); + bool readOk = dictionary.readDictionary(fs.root()); + if(!readOk) { + cerr << "Invalid dictionary file" << endl; + throw -1; + } + } + else { + cerr << "Dictionary not specified" << endl; + throw -1; + } +} + +int main(int argc, char *argv[]) { + CommandLineParser parser(argc, argv, keys); + parser.about(about); + + if(argc < 7) { + parser.printMessage(); + return 0; + } + + //! [aruco_detect_board_full_sample] + int markersX = parser.get("w"); + int markersY = parser.get("h"); + float markerLength = parser.get("l"); + float markerSeparation = parser.get("s"); + bool showRejected = parser.has("r"); + bool refindStrategy = parser.has("rs"); + int camId = parser.get("ci"); + + + Mat camMatrix, distCoeffs; + readCameraParamsFromCommandLine(parser, camMatrix, distCoeffs); + + aruco::DetectorParameters detectorParams; + detectorParams.cornerRefinementMethod = aruco::CORNER_REFINE_SUBPIX; // do corner refinement in markers + readDetectorParamsFromCommandLine(parser, detectorParams); + + String video; + if(parser.has("v")) { + video = parser.get("v"); + } + + if(!parser.check()) { + parser.printErrors(); + return 0; + } + + aruco::Dictionary dictionary = aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_50); + readDictionatyFromCommandLine(parser, dictionary); + + aruco::ArucoDetector detector(dictionary, detectorParams); + VideoCapture inputVideo; + int waitTime; + if(!video.empty()) { + inputVideo.open(video); + waitTime = 0; + } else { + inputVideo.open(camId); + waitTime = 10; + } + + float axisLength = 0.5f * ((float)min(markersX, markersY) * (markerLength + markerSeparation) + + markerSeparation); + + // Create GridBoard object + //! [aruco_create_board] + aruco::GridBoard board(Size(markersX, markersY), markerLength, markerSeparation, dictionary); + //! [aruco_create_board] + + // Also you could create Board object + //vector > objPoints; // array of object points of all the marker corners in the board + //vector ids; // vector of the identifiers of the markers in the board + //aruco::Board board(objPoints, dictionary, ids); + + double totalTime = 0; + int totalIterations = 0; + + while(inputVideo.grab()) { + Mat image, imageCopy; + inputVideo.retrieve(image); + + double tick = (double)getTickCount(); + + vector ids; + vector> corners, rejected; + Vec3d rvec, tvec; + + //! [aruco_detect_and_refine] + + // Detect markers + detector.detectMarkers(image, corners, ids, rejected); + + // Refind strategy to detect more markers + if(refindStrategy) + detector.refineDetectedMarkers(image, board, corners, ids, rejected, camMatrix, + distCoeffs); + + //! [aruco_detect_and_refine] + + // Estimate board pose + int markersOfBoardDetected = 0; + if(!ids.empty()) { + // Get object and image points for the solvePnP function + cv::Mat objPoints, imgPoints; + board.matchImagePoints(corners, ids, objPoints, imgPoints); + + // Find pose + cv::solvePnP(objPoints, imgPoints, camMatrix, distCoeffs, rvec, tvec); + + markersOfBoardDetected = (int)objPoints.total() / 4; + } + + double currentTime = ((double)getTickCount() - tick) / getTickFrequency(); + totalTime += currentTime; + totalIterations++; + if(totalIterations % 30 == 0) { + cout << "Detection Time = " << currentTime * 1000 << " ms " + << "(Mean = " << 1000 * totalTime / double(totalIterations) << " ms)" << endl; + } + + // Draw results + image.copyTo(imageCopy); + if(!ids.empty()) { + aruco::drawDetectedMarkers(imageCopy, corners, ids); + } + + if(showRejected && !rejected.empty()) + aruco::drawDetectedMarkers(imageCopy, rejected, noArray(), Scalar(100, 0, 255)); + + if(markersOfBoardDetected > 0) + cv::drawFrameAxes(imageCopy, camMatrix, distCoeffs, rvec, tvec, axisLength); + + imshow("out", imageCopy); + char key = (char)waitKey(waitTime); + if(key == 27) break; + //! [aruco_detect_board_full_sample] + } + + return 0; +} diff --git a/samples/cpp/tutorial_code/objectDetection/detect_markers.cpp b/samples/cpp/tutorial_code/objectDetection/detect_markers.cpp new file mode 100644 index 0000000000..720fb8ddae --- /dev/null +++ b/samples/cpp/tutorial_code/objectDetection/detect_markers.cpp @@ -0,0 +1,177 @@ +#include +#include +#include +#include "aruco_samples_utility.hpp" + +using namespace std; +using namespace cv; + +namespace { +const char* about = "Basic marker detection"; + +//! [aruco_detect_markers_keys] +const char* keys = + "{d | | dictionary: DICT_4X4_50=0, DICT_4X4_100=1, DICT_4X4_250=2," + "DICT_4X4_1000=3, DICT_5X5_50=4, DICT_5X5_100=5, DICT_5X5_250=6, DICT_5X5_1000=7, " + "DICT_6X6_50=8, DICT_6X6_100=9, DICT_6X6_250=10, DICT_6X6_1000=11, DICT_7X7_50=12," + "DICT_7X7_100=13, DICT_7X7_250=14, DICT_7X7_1000=15, DICT_ARUCO_ORIGINAL = 16," + "DICT_APRILTAG_16h5=17, DICT_APRILTAG_25h9=18, DICT_APRILTAG_36h10=19, DICT_APRILTAG_36h11=20}" + "{cd | | Input file with custom dictionary }" + "{v | | Input from video or image file, if ommited, input comes from camera }" + "{ci | 0 | Camera id if input doesnt come from video (-v) }" + "{c | | Camera intrinsic parameters. Needed for camera pose }" + "{l | 0.1 | Marker side length (in meters). Needed for correct scale in camera pose }" + "{dp | | File of marker detector parameters }" + "{r | | show rejected candidates too }" + "{refine | | Corner refinement: CORNER_REFINE_NONE=0, CORNER_REFINE_SUBPIX=1," + "CORNER_REFINE_CONTOUR=2, CORNER_REFINE_APRILTAG=3}"; +} +//! [aruco_detect_markers_keys] + +int main(int argc, char *argv[]) { + CommandLineParser parser(argc, argv, keys); + parser.about(about); + + if(argc < 2) { + parser.printMessage(); + return 0; + } + + bool showRejected = parser.has("r"); + bool estimatePose = parser.has("c"); + float markerLength = parser.get("l"); + + cv::aruco::DetectorParameters detectorParams; + if(parser.has("dp")) { + cv::FileStorage fs(parser.get("dp"), FileStorage::READ); + bool readOk = detectorParams.readDetectorParameters(fs.root()); + if(!readOk) { + cerr << "Invalid detector parameters file" << endl; + return 0; + } + } + + if (parser.has("refine")) { + // override cornerRefinementMethod read from config file + detectorParams.cornerRefinementMethod = parser.get("refine"); + } + std::cout << "Corner refinement method (0: None, 1: Subpixel, 2:contour, 3: AprilTag 2): " << (int)detectorParams.cornerRefinementMethod << std::endl; + + int camId = parser.get("ci"); + + String video; + if(parser.has("v")) { + video = parser.get("v"); + } + + if(!parser.check()) { + parser.printErrors(); + return 0; + } + + aruco::Dictionary dictionary = aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_50); + if (parser.has("d")) { + int dictionaryId = parser.get("d"); + dictionary = aruco::getPredefinedDictionary(aruco::PredefinedDictionaryType(dictionaryId)); + } + else if (parser.has("cd")) { + cv::FileStorage fs(parser.get("cd"), FileStorage::READ); + bool readOk = dictionary.readDictionary(fs.root()); + if(!readOk) { + std::cerr << "Invalid dictionary file" << std::endl; + return 0; + } + } + else { + std::cerr << "Dictionary not specified" << std::endl; + return 0; + } + + //! [aruco_pose_estimation1] + cv::Mat camMatrix, distCoeffs; + if(estimatePose) { + // You can read camera parameters from tutorial_camera_params.yml + bool readOk = readCameraParameters(parser.get("c"), camMatrix, distCoeffs); + if(!readOk) { + cerr << "Invalid camera file" << endl; + return 0; + } + } + //! [aruco_pose_estimation1] + //! [aruco_detect_markers] + cv::aruco::ArucoDetector detector(dictionary, detectorParams); + cv::VideoCapture inputVideo; + int waitTime; + if(!video.empty()) { + inputVideo.open(video); + waitTime = 0; + } else { + inputVideo.open(camId); + waitTime = 10; + } + + double totalTime = 0; + int totalIterations = 0; + + //! [aruco_pose_estimation2] + // set coordinate system + cv::Mat objPoints(4, 1, CV_32FC3); + objPoints.ptr(0)[0] = Vec3f(-markerLength/2.f, markerLength/2.f, 0); + objPoints.ptr(0)[1] = Vec3f(markerLength/2.f, markerLength/2.f, 0); + objPoints.ptr(0)[2] = Vec3f(markerLength/2.f, -markerLength/2.f, 0); + objPoints.ptr(0)[3] = Vec3f(-markerLength/2.f, -markerLength/2.f, 0); + //! [aruco_pose_estimation2] + + while(inputVideo.grab()) { + cv::Mat image, imageCopy; + inputVideo.retrieve(image); + + double tick = (double)getTickCount(); + + //! [aruco_pose_estimation3] + vector ids; + vector > corners, rejected; + + // detect markers and estimate pose + detector.detectMarkers(image, corners, ids, rejected); + + size_t nMarkers = corners.size(); + vector rvecs(nMarkers), tvecs(nMarkers); + + if(estimatePose && !ids.empty()) { + // Calculate pose for each marker + for (size_t i = 0; i < nMarkers; i++) { + solvePnP(objPoints, corners.at(i), camMatrix, distCoeffs, rvecs.at(i), tvecs.at(i)); + } + } + //! [aruco_pose_estimation3] + double currentTime = ((double)getTickCount() - tick) / getTickFrequency(); + totalTime += currentTime; + totalIterations++; + if(totalIterations % 30 == 0) { + cout << "Detection Time = " << currentTime * 1000 << " ms " + << "(Mean = " << 1000 * totalTime / double(totalIterations) << " ms)" << endl; + } + //! [aruco_draw_pose_estimation] + // draw results + image.copyTo(imageCopy); + if(!ids.empty()) { + cv::aruco::drawDetectedMarkers(imageCopy, corners, ids); + + if(estimatePose) { + for(unsigned int i = 0; i < ids.size(); i++) + cv::drawFrameAxes(imageCopy, camMatrix, distCoeffs, rvecs[i], tvecs[i], markerLength * 1.5f, 2); + } + } + //! [aruco_draw_pose_estimation] + + if(showRejected && !rejected.empty()) + cv::aruco::drawDetectedMarkers(imageCopy, rejected, noArray(), Scalar(100, 0, 255)); + + imshow("out", imageCopy); + char key = (char)waitKey(waitTime); + if(key == 27) break; + } + //! [aruco_detect_markers] + return 0; +} diff --git a/samples/cpp/tutorial_code/objectDetection/tutorial_camera_params.yml b/samples/cpp/tutorial_code/objectDetection/tutorial_camera_params.yml new file mode 100644 index 0000000000..69d2d6d22f --- /dev/null +++ b/samples/cpp/tutorial_code/objectDetection/tutorial_camera_params.yml @@ -0,0 +1,14 @@ +%YAML:1.0 +camera_matrix: !!opencv-matrix + rows: 3 + cols: 3 + dt: d + data: [ 628.158, 0., 324.099, + 0., 628.156, 260.908, + 0., 0., 1. ] +distortion_coefficients: !!opencv-matrix + rows: 5 + cols: 1 + dt: d + data: [ 0.0995485, -0.206384, + 0.00754589, 0.00336531, 0 ] diff --git a/samples/cpp/tutorial_code/objectDetection/tutorial_dict.yml b/samples/cpp/tutorial_code/objectDetection/tutorial_dict.yml new file mode 100644 index 0000000000..af5c87e4e0 --- /dev/null +++ b/samples/cpp/tutorial_code/objectDetection/tutorial_dict.yml @@ -0,0 +1,38 @@ +%YAML:1.0 +nmarkers: 35 +markersize: 6 +marker_0: "101011111011111001001001101100000000" +marker_1: "000000000010011001010011111010111000" +marker_2: "011001100000001010000101111101001101" +marker_3: "001000111111000111011001110000011111" +marker_4: "100110110100101111000000111101110011" +marker_5: "010101101110111000111010111100010111" +marker_6: "101001000110011110101001010100110100" +marker_7: "011010100100110000011101110110100010" +marker_8: "111110001000101000110001010010111101" +marker_9: "011101101100110111001100100001010100" +marker_10: "100001100001010001110001011000000111" +marker_11: "110010010010011100101111111000001111" +marker_12: "110101001001010110011111010110001101" +marker_13: "001111000001000100010001101001010001" +marker_14: "000000010010101010111110110011010011" +marker_15: "110001110111100101110011111100111010" +marker_16: "101011001110001010110011111011001110" +marker_17: "101110111101110100101101011001010111" +marker_18: "000100111000111101010011010101000101" +marker_19: "001110001110001101100101110100000011" +marker_20: "100101101100010110110110110001100011" +marker_21: "010110001001011010000100111000110110" +marker_22: "001000000000100100000000010100010010" +marker_23: "101001110010100110000111111010010000" +marker_24: "111001101010001100011010010001011100" +marker_25: "101000010001010000110100111101101001" +marker_26: "101010000001010011001010110110000001" +marker_27: "100101001000010101001000111101111110" +marker_28: "010010100110010011110001110101011100" +marker_29: "011001000101100001101111010001001111" +marker_30: "000111011100011110001101111011011001" +marker_31: "010100001011000100111101110001101010" +marker_32: "100101101001101010111111101101110100" +marker_33: "101101001010111000000100110111010101" +marker_34: "011111010000111011111110110101100101"