First, the ```Dictionary``` object is created by choosing one of the predefined dictionaries in the aruco module.
Concretely, this dictionary is composed by 250 markers and a marker size of 6x6 bits (```DICT_6X6_250```).
First, the `Dictionary` object is created by choosing one of the predefined dictionaries in the aruco module.
Concretely, this dictionary is composed of 250 markers and a marker size of 6x6 bits (`DICT_6X6_250`).
The parameters of ```drawMarker``` are:
The parameters of `drawMarker` are:
- The first parameter is the ```Dictionary``` object previously created.
- The second parameter is the marker id, in this case the marker 23 of the dictionary ```DICT_6X6_250```.
Note that each dictionary is composed by a different number of markers. In this case, the valid ids
- The first parameter is the `Dictionary` object previously created.
- The second parameter is the marker id, in this case the marker 23 of the dictionary `DICT_6X6_250`.
Note that each dictionary is composed of a different number of markers. In this case, the valid ids
go from 0 to 249. Any specific id out of the valid range will produce an exception.
- The third parameter, 200, is the size of the output marker image. In this case, the output image
will have a size of 200x200 pixels. Note that this parameter should be large enough to store the
@ -91,7 +92,7 @@ number of bits for the specific dictionary. So, for instance, you cannot generat
Furthermore, to avoid deformations, this parameter should be proportional to the number of bits +
border size, or at least much higher than the marker size (like 200 in the example), so that
deformations are insignificant.
- The forth parameter is the output image.
- The fourth parameter is the output image.
- Finally, the last parameter is an optional parameter to specify the width of the marker black
border. The size is specified proportional to the number of bits. For instance a value of 2 means
that the border will have a width equivalent to the size of two internal bits. The default value
@ -99,140 +100,142 @@ is 1.
The generated image is:
A full working example is included in the ```create_marker.cpp``` inside the module samples folder.
A full working example is included in the `create_marker.cpp` inside the module samples folder.
Note: The samples now take input via commandline via the [OpenCV Commandline Parser](http://docs.opencv.org/trunk/d0/d2e/classcv_1_1CommandLineParser.html#gsc.tab=0). For this file the example parameters will look like
Note: The samples now take input from the command line using cv::CommandLineParser. For this file the example parameters will look like
- The first parameter is the image where the markers are going to be detected.
- The second parameter is the dictionary object, in this case one of the predefined dictionaries (```DICT_6X6_250```).
- The detected markers are stored in the ```markerCorners``` and ```markerIds``` structures:
- ```markerCorners``` is the list of corners of the detected markers. For each marker, its four
- The first parameter is the image containing the markers to be detected.
- The second parameter is the dictionary object, in this case one of the predefined dictionaries (`DICT_6X6_250`).
- The detected markers are stored in the `markerCorners` and `markerIds` structures:
- `markerCorners` is the list of corners of the detected markers. For each marker, its four
corners are returned in their original order (which is clockwise starting with top left). So, the first corner is the top left corner, followed by the top right, bottom right and bottom left.
- ```markerIds``` is the list of ids of each of the detected markers in ```markerCorners```.
Note that the returned ```markerCorners``` and ```markerIds``` vectors have the same sizes.
- The fourth parameter is the object of type ```DetectionParameters```. This object includes all the
parameters that can be customized during the detection process. This parameters are commented in
detail in the next section.
- The final parameter, ```rejectedCandidates```, is a returned list of marker candidates, i.e. those
squares that have been found but they do not present a valid codification. Each candidate is also
defined by its four corners, and its format is the same than the ```markerCorners``` parameter. This
parameter can be omitted and is only useful for debugging purposes and for 'refind' strategies (see ```refineDetectedMarkers()``` ).
The next thing you probably want to do after ```detectMarkers()``` is checking that your markers have
- `markerIds` is the list of ids of each of the detected markers in `markerCorners`.
Note that the returned `markerCorners` and `markerIds` vectors have the same size.
- The fourth parameter is the object of type `DetectionParameters`. This object includes all the
parameters that can be customized during the detection process. These parameters are explained in the next section.
- The final parameter, `rejectedCandidates`, is a returned list of marker candidates, i.e.
shapes that were found and considered but did not contain a valid marker. Each candidate is also
defined by its four corners, and its format is the same as the `markerCorners` parameter. This
parameter can be omitted and is only useful for debugging purposes and for ‘refind’ strategies (see `refineDetectedMarkers()` ).
The next thing you probably want to do after `detectMarkers()` is check that your markers have
been correctly detected. Fortunately, the aruco module provides a function to draw the detected
markers in the input image, this function is ```drawDetectedMarkers()```. For example:
markers in the input image, this function is `drawDetectedMarkers()`. For example:
- ```image``` is the input/output image where the markers will be drawn (it will normally be the same image where the markers were detected).
- ```markerCorners``` and ```markerIds``` are the structures of the detected markers in the same format
provided by the ```detectMarkers()``` function.
- `outputImage ` is the input/output image where the markers will be drawn (it will normally be the same as the image where the markers were detected).
- `markerCorners` and `markerIds` are the structures of the detected markers returned by the `detectMarkers()` function.
Note that this function is only provided for visualization and its use can be perfectly omitted.
Note that this function is only provided for visualization and its use can be omitted.
With these two functions we can create a basic marker detection loop to detect markers from our
Note that some of the optional parameters have been omitted, like the detection parameter object or the
Note that some of the optional parameters have been omitted, like the detection parameter object and the
output vector of rejected candidates.
A full working example is included in the ```detect_markers.cpp``` inside the module samples folder.
A full working example is included in the `detect_markers.cpp` inside the module samples folder.
Note: The samples now take input via commandline via the [OpenCV Commandline Parser](http://docs.opencv.org/trunk/d0/d2e/classcv_1_1CommandLineParser.html#gsc.tab=0). For this file the example parameters will look like
Note: The samples now take input from the command line using cv::CommandLineParser. For this file the example parameters will look like
@code{.cpp}
-c="_path_/calib.txt" -d=10
-c="_path_/calib.txt" -d=10
@endcode
@ -242,94 +245,99 @@ Pose Estimation
The next thing you probably want to do after detecting the markers is to obtain the camera pose from them.
To perform camera pose estimation you need to know the calibration parameters of your camera. This is
To perform camera pose estimation you need to know the calibration parameters of your camera. These are
the camera matrix and distortion coefficients. If you do not know how to calibrate your camera, you can
take a look to the ```calibrateCamera()``` function and the Calibration tutorial of OpenCV. You can also calibrate your camera using the aruco module
as it is explained in the Calibration with aruco tutorial. Note that this only need to be done once unless the
take a look at the `calibrateCamera()` function and the Calibration tutorial of OpenCV. You can also calibrate your camera using the aruco module
as explained in the **Calibration with ArUco and ChArUco** tutorial. Note that this only needs to be done once unless the
camera optics are modified (for instance changing its focus).
At the end, what you get after the calibration is the camera matrix: a matrix of 3x3 elements with the
In the end, what you get after the calibration is the camera matrix: a matrix of 3x3 elements with the
focal distances and the camera center coordinates (a.k.a intrinsic parameters), and the distortion
coefficients: a vector of 5 elements or more that models the distortion produced by your camera.
coefficients: a vector of 5 or more elements that models the distortion produced by your camera.
When you estimate the pose with ArUco markers, you can estimate the pose of each marker individually.
If you want to estimate one pose from a set of markers, what you want to use is aruco Boards (see ArUco
Boards tutorial).
If you want to estimate one pose from a set of markers, use ArUco Boards (see the **Detection of ArUco
Boards** tutorial). Using ArUco boards instead of single markers allows some markers to be occluded.
The camera pose respect to a marker is the 3d transformation from the marker coordinate system to the
camera coordinate system. It is specified by a rotation and a translation vector (see ```solvePnP()``` function for more
The camera pose with respect to a marker is the 3d transformation from the marker coordinate system to the
camera coordinate system. It is specified by rotation and translation vectors (see `solvePnP()` function for more
information).
The aruco module provides a function to estimate the poses of all the detected markers:
- The ```corners``` parameter is the vector of marker corners returned by the ```detectMarkers()``` function.
- The `markerCorners` parameter is the vector of marker corners returned by the `detectMarkers()` function.
- The second parameter is the size of the marker side in meters or in any other unit. Note that the
translation vectors of the estimated poses will be in the same unit
- ```cameraMatrix``` and ```distCoeffs``` are the camera calibration parameters that need to be known a priori.
- ```rvecs``` and ```tvecs``` are the rotation and translation vectors respectively, for each of the markers
in corners.
- `cameraMatrix` and `distCoeffs` are the camera calibration parameters that were created during the camera calibration process.
- The output parameters `rvecs` and `tvecs` are the rotation and translation vectors respectively, for each of the markers
in `markerCorners`.
The marker coordinate system that is assumed by this function is placed at the center of the marker
with the Z axis pointing out, as in the following image. Axis-color correspondences are X:red, Y:green, Z:blue.
with the Z axis pointing out, as in the following image. Axis-color correspondences are X:red, Y:green, Z:blue. Note the axis directions of the rotated markers in this image.
The aruco module provides a function to draw the axis as in the image above, so pose estimation can be
A full working example is included in the ```detect_markers.cpp``` inside the module samples folder.
A full working example is included in the `detect_markers.cpp` inside the module samples folder.
Note: The samples now take input via commandline via the [OpenCV Commandline Parser](http://docs.opencv.org/trunk/d0/d2e/classcv_1_1CommandLineParser.html#gsc.tab=0). For this file the example parameters will look like
Note: The samples now take input from the command line using cv::CommandLineParser. For this file the example parameters will look like
@code{.cpp}
-c="_path_/calib.txt" -d=10
@endcode
@ -350,41 +358,42 @@ Note: The samples now take input via commandline via the [OpenCV Commandline Par
Selecting a dictionary
------
The aruco module provides the ```Dictionary``` class to represent a dictionary of markers.
The aruco module provides the `Dictionary` class to represent a dictionary of markers.
Apart of the marker size and the number of markers in the dictionary, there is another important dictionary
In addition to the marker size and the number of markers in the dictionary, there is another important dictionary
parameter, the inter-marker distance. The inter-marker distance is the minimum distance among its markers
and it determines the error detection and correction capabilities of the dictionary.
In general, lower dictionary sizes and higher marker sizes increase the inter-marker distance and
vice-versa. However, the detection of markers with higher sizes is more complex, due to the higher
amount of bits that need to be extracted from the image.
number of bits that need to be extracted from the image.
For instance, if you need only 10 markers in your application, it is better to use a dictionary only
composed by those 10 markers than using one dictionary composed by 1000 markers. The reason is that
the dictionary composed by 10 markers will have a higher inter-marker distance and, thus, it will be
For instance, if you need only 10 markers in your application, it is better to use a dictionary composed only of those 10 markers than using a dictionary composed of 1000 markers. The reason is that
the dictionary composed of 10 markers will have a higher inter-marker distance and, thus, it will be
more robust to errors.
As a consequence, the aruco module includes several ways to select your dictionary of markers, so that
you can increase your system robustness:
- Predefined dictionaries:
### Predefined dictionaries
This is the easiest way to select a dictionary. The aruco module includes a set of predefined dictionaries
of a variety of marker sizes and number of markers. For instance:
in a variety of marker sizes and number of markers. For instance:
#### minMarkerPerimeterRate and maxMarkerPerimeterRate
These parameters determine the minimum and maximum size of a marker, concretely the maximum and
minimum marker perimeter. They are not specified in absolute pixels values, instead they are
These parameters determine the minimum and maximum size of a marker, specifically the minimum and maximum marker perimeter. They are not specified in absolute pixel values, instead they are
specified relative to the maximum dimension of the input image.
For instance, a image with size 640x480 and a minimum relative marker perimeter of 0.05 will lead
to a minimum marker perimeter of 640x0.05 = 32 pixels, since 640 is the maximum dimension of the
image. The same applies for the ```maxMarkerPerimeterRate``` parameter.
image. The same applies for the `maxMarkerPerimeterRate` parameter.
If the ```minMarkerPerimeterRate``` is too low, it can penalize considerably the detection performance since
If the `minMarkerPerimeterRate` is too low, it can penalize considerably the detection performance since
many more contours would be considered for future stages.
This penalization is not so noticeable for the ```maxMarkerPerimeterRate``` parameter, since there are
This penalization is not so noticeable for the `maxMarkerPerimeterRate` parameter, since there are
usually many more small contours than big contours.
A ```minMarkerPerimeterRate``` value of 0 and a ```maxMarkerPerimeterRate``` value of 4 (or more) will be
A `minMarkerPerimeterRate` value of 0 and a `maxMarkerPerimeterRate` value of 4 (or more) will be
equivalent to consider all the contours in the image, however this is not recommended for
After the bits have been extracted, the next step is checking if the extracted code belongs to the marker
### Marker identification
After the bits have been extracted, the next step is checking whether the extracted code belongs to the marker
dictionary and, if necessary, error correction can be performed.
- ```double maxErroneousBitsInBorderRate```
#### maxErroneousBitsInBorderRate
The bits of the marker border should be black. This parameter specifies the allowed number of erroneous
bits in the border, i.e. the maximum number of white bits in the border. It is represented
relative to the total number of bits in the marker.
Default value: 0.35
Default value:
- `double maxErroneousBitsInBorderRate = 0.35`
- ```double errorCorrectionRate```
#### errorCorrectionRate
Each marker dictionary has a theoretical maximum number of bits that can be corrected (```Dictionary.maxCorrectionBits```).
However, this value can be modified by the ```errorCorrectionRate``` parameter.
Each marker dictionary has a theoretical maximum number of bits that can be corrected (`Dictionary.maxCorrectionBits`).
However, this value can be modified by the `errorCorrectionRate` parameter.
For instance, if the allowed number of bits that can be corrected (for the used dictionary) is 6 and the value of ```errorCorrectionRate``` is
For instance, if the allowed number of bits that can be corrected (for the used dictionary) is 6 and the value of `errorCorrectionRate` is
0.5, the real maximum number of bits that can be corrected is 6*0.5=3 bits.
This value is useful to reduce the error correction capabilities in order to avoid false positives.
Default value: 0.6
Default value:
- `double errorCorrectionRate = 0.6`
#### Corner Refinement
### Corner Refinement
After markers have been detected and identified, the last step is performing subpixel refinement
in the corner positions (see OpenCV ```cornerSubPix()``` and ```cv::aruco::CornerRefineMethod```)
of the corner positions (see OpenCV `cornerSubPix()` and `cv::aruco::CornerRefineMethod`).
Note that this step is optional and it only makes sense if the positions of the marker corners have to
be accurate, for instance for pose estimation. It is usually a time-consuming step and therefore is disabled by default.
Note that this step is optional and it only makes sense if the position of the marker corners have to
be accurate, for instance for pose estimation. It is usually a time consuming step and it is disabled by default.
#### cornerRefinementMethod
- ```int cornerRefinementMethod```
This parameter determines whether the corner subpixel process is performed or not and which method to use if it is being performed. It can be disabled
if accurate corners are not necessary. Possible values are `CORNER_REFINE_NONE`, `CORNER_REFINE_SUBPIX`, `CORNER_REFINE_CONTOUR`, and `CORNER_REFINE_APRILTAG`.
This parameter determines if the corner subpixel process is performed or not. It can be disabled
Alexander Alekhin
5 years ago
