@ -51,7 +51,7 @@ You can find the source code of this tutorial in the :file:`samples/cpp/tutorial
The tutorial consists of two main programs:
The tutorial consists of two main programs:
**1. Model registration**
1. **Model registration**
This applicaton is exclusive to whom don't have a 3D textured model of the object to be detected. You can use this program to create your own textured 3D model. This program only works for planar objects, then if you want to model an object with complex shape you should use a sophisticated software to create it.
This applicaton is exclusive to whom don't have a 3D textured model of the object to be detected. You can use this program to create your own textured 3D model. This program only works for planar objects, then if you want to model an object with complex shape you should use a sophisticated software to create it.
@ -64,7 +64,7 @@ The application starts up extracting the ORB features and descriptors from the i
:align:center
:align:center
**2. Model detection**
#. **Model detection**
The aim of this application is estimate in real time the object pose given its 3D textured model.
The aim of this application is estimate in real time the object pose given its 3D textured model.
@ -76,7 +76,7 @@ Explanation
Here is explained in detail the code for the real time application:
Here is explained in detail the code for the real time application:
**1. Read 3D textured object model and object mesh.**
1. **Read 3D textured object model and object mesh.**
In order to load the textured model I implemented the *class***Model** which has the function *load()* that opens a YAML file and take the stored 3D points with its corresponding descriptors. You can find an example of a 3D textured model in :file:`samples/cpp/tutorial_code/calib3d/real_time_pose_estimation/Data/cookies_ORB.yml`.
In order to load the textured model I implemented the *class***Model** which has the function *load()* that opens a YAML file and take the stored 3D points with its corresponding descriptors. You can find an example of a 3D textured model in :file:`samples/cpp/tutorial_code/calib3d/real_time_pose_estimation/Data/cookies_ORB.yml`.
@ -138,7 +138,7 @@ In the main program the mesh is loaded as follows:
mesh.load(ply_read_path); // load an object mesh
mesh.load(ply_read_path); // load an object mesh
**2. Take input from Camera or Video**
#. **Take input from Camera or Video**
To detect is necessary capture video. It's done loading a recorded video by passing the absolute path where it is located in your machine or using the default camera device. In order to test the application you can find a recorded video in :file:`samples/cpp/tutorial_code/calib3d/real_time_pose_estimation/Data/box.mp4`.
To detect is necessary capture video. It's done loading a recorded video by passing the absolute path where it is located in your machine or using the default camera device. In order to test the application you can find a recorded video in :file:`samples/cpp/tutorial_code/calib3d/real_time_pose_estimation/Data/box.mp4`.
@ -170,7 +170,7 @@ Then the algorithm is computed frame per frame:
}
}
**3. Extract ORB features and descriptors from the scene**
#. **Extract ORB features and descriptors from the scene**
The next step is to detect the scene features and extract it descriptors. For this task I implemented a *class***RobustMatcher** which has a function for keypoints detection and features extraction. You can find it in :file:`samples/cpp/tutorial_code/calib3d/real_time_pose_estimation/src/RobusMatcher.cpp`. In your *RobusMatch* object you can use any of the 2D features detectors of OpenCV. In this case I used :feature_detection_and_description:`ORB<orb>` features because is based on :feature_detection_and_description:`FAST<fast>` to detect the keypoints and :descriptor_extractor:`BRIEF<briefdescriptorextractor>` to extract the descriptors which means that is fast and robust to rotations. You can find more detailed information about *ORB* in the documentation.
The next step is to detect the scene features and extract it descriptors. For this task I implemented a *class***RobustMatcher** which has a function for keypoints detection and features extraction. You can find it in :file:`samples/cpp/tutorial_code/calib3d/real_time_pose_estimation/src/RobusMatcher.cpp`. In your *RobusMatch* object you can use any of the 2D features detectors of OpenCV. In this case I used :feature_detection_and_description:`ORB<orb>` features because is based on :feature_detection_and_description:`FAST<fast>` to detect the keypoints and :descriptor_extractor:`BRIEF<briefdescriptorextractor>` to extract the descriptors which means that is fast and robust to rotations. You can find more detailed information about *ORB* in the documentation.
@ -189,7 +189,7 @@ The following code is how to instantiate and set the features detector and the d
The features and descriptors will be computed by the *RobustMatcher* inside the matching function.
The features and descriptors will be computed by the *RobustMatcher* inside the matching function.
**4. Match scene descriptors with model descriptors using Flann matcher**
#. **Match scene descriptors with model descriptors using Flann matcher**
It is the first step in our detection algorithm. The main idea is to match the scene descriptors with our model descriptors in order to know the 3D coordinates of the found features into the current scene.
It is the first step in our detection algorithm. The main idea is to match the scene descriptors with our model descriptors in order to know the 3D coordinates of the found features into the current scene.
@ -287,7 +287,7 @@ After the matches filtering we have to subtract the 2D and 3D correspondences fr
}
}
**5. Pose estimation using PnP + Ransac**
#. **Pose estimation using PnP + Ransac**
Once with the 2D and 3D correspondences we have to apply a PnP algorithm in order to estimate the camera pose. The reason why we have to use :calib3d:`solvePnPRansac <solvepnpransac>` instead of :calib3d:`solvePnP <solvepnp>` is due to the fact that after the matching not all the found correspondences are correct and, as like as not, there are false correspondences or also called *outliers*. The `Random Sample Consensus <http://en.wikipedia.org/wiki/RANSAC>`_ or *Ransac* is a non-deterministic iterative method which estimate parameters of a mathematical model from observed data producing an aproximate result as the number of iterations increase. After appyling *Ransac* all the *outliers* will be eliminated to then estimate the camera pose with a certain probability to obtain a good solution.
Once with the 2D and 3D correspondences we have to apply a PnP algorithm in order to estimate the camera pose. The reason why we have to use :calib3d:`solvePnPRansac <solvepnpransac>` instead of :calib3d:`solvePnP <solvepnp>` is due to the fact that after the matching not all the found correspondences are correct and, as like as not, there are false correspondences or also called *outliers*. The `Random Sample Consensus <http://en.wikipedia.org/wiki/RANSAC>`_ or *Ransac* is a non-deterministic iterative method which estimate parameters of a mathematical model from observed data producing an aproximate result as the number of iterations increase. After appyling *Ransac* all the *outliers* will be eliminated to then estimate the camera pose with a certain probability to obtain a good solution.
@ -430,7 +430,7 @@ The following code corresponds to the *backproject3DPoint()* function which belo
The above function is used to compute all the 3D points of the object *Mesh* to show the pose of the object.
The above function is used to compute all the 3D points of the object *Mesh* to show the pose of the object.
**6. Linear Kalman Filter for bad poses rejection**
#. **Linear Kalman Filter for bad poses rejection**
Is it common in computer vision or robotics fields that after applying detection or tracking techniques, bad results are obtained due to some sensor errors. In order to avoid these bad detections in this tutorial is explained how to implement a Linear Kalman Filter. The Kalman Filter will be applied after detected a given number of inliers.
Is it common in computer vision or robotics fields that after applying detection or tracking techniques, bad results are obtained due to some sensor errors. In order to avoid these bad detections in this tutorial is explained how to implement a Linear Kalman Filter. The Kalman Filter will be applied after detected a given number of inliers.