f-morozov
11 years ago
parent
277802483f
commit
7f82960897
29 changed files with 2755 additions and 2767 deletions
@ -0,0 +1,161 @@ |
||||
.. _akazeMatching: |
||||
|
||||
|
||||
AKAZE local features matching |
||||
****************************** |
||||
|
||||
Introduction |
||||
------------------ |
||||
|
||||
In this tutorial we will learn how to use [AKAZE]_ local features to detect and match keypoints on two images. |
||||
|
||||
We will find keypoints on a pair of images with given homography matrix, |
||||
match them and count the number of inliers (i. e. matches that fit in the given homography). |
||||
|
||||
You can find expanded version of this example here: https://github.com/pablofdezalc/test_kaze_akaze_opencv |
||||
|
||||
.. [AKAZE] Fast Explicit Diffusion for Accelerated Features in Nonlinear Scale Spaces. Pablo F. Alcantarilla, Jesús Nuevo and Adrien Bartoli. In British Machine Vision Conference (BMVC), Bristol, UK, September 2013. |
||||
|
||||
Data |
||||
------------------ |
||||
We are going to use images 1 and 3 from *Graffity* sequence of Oxford dataset. |
||||
|
||||
.. image:: images/graf.png |
||||
:height: 200pt |
||||
:width: 320pt |
||||
:alt: Graffity |
||||
:align: center |
||||
|
||||
Homography is given by a 3 by 3 matrix: |
||||
|
||||
.. code-block:: none |
||||
|
||||
7.6285898e-01 -2.9922929e-01 2.2567123e+02 |
||||
3.3443473e-01 1.0143901e+00 -7.6999973e+01 |
||||
3.4663091e-04 -1.4364524e-05 1.0000000e+00 |
||||
|
||||
You can find the images (*graf1.png*, *graf3.png*) and homography (*H1to3p.xml*) in *opencv/samples/cpp*. |
||||
|
||||
Source Code |
||||
=========== |
||||
.. literalinclude:: ../../../../samples/cpp/tutorial_code/features2D/AKAZE_match.cpp |
||||
:language: cpp |
||||
:linenos: |
||||
:tab-width: 4 |
||||
|
||||
Explanation |
||||
=========== |
||||
|
||||
1. **Load images and homography** |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
Mat img1 = imread("graf1.png", IMREAD_GRAYSCALE); |
||||
Mat img2 = imread("graf3.png", IMREAD_GRAYSCALE); |
||||
|
||||
Mat homography; |
||||
FileStorage fs("H1to3p.xml", FileStorage::READ); |
||||
fs.getFirstTopLevelNode() >> homography; |
||||
|
||||
We are loading grayscale images here. Homography is stored in the xml created with FileStorage. |
||||
|
||||
2. **Detect keypoints and compute descriptors using AKAZE** |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
vector<KeyPoint> kpts1, kpts2; |
||||
Mat desc1, desc2; |
||||
|
||||
AKAZE akaze; |
||||
akaze(img1, noArray(), kpts1, desc1); |
||||
akaze(img2, noArray(), kpts2, desc2); |
||||
|
||||
We create AKAZE object and use it's *operator()* functionality. Since we don't need the *mask* parameter, *noArray()* is used. |
||||
|
||||
3. **Use brute-force matcher to find 2-nn matches** |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
BFMatcher matcher(NORM_HAMMING); |
||||
vector< vector<DMatch> > nn_matches; |
||||
matcher.knnMatch(desc1, desc2, nn_matches, 2); |
||||
|
||||
We use Hamming distance, because AKAZE uses binary descriptor by default. |
||||
|
||||
4. **Use 2-nn matches to find correct keypoint matches** |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
for(size_t i = 0; i < nn_matches.size(); i++) { |
||||
DMatch first = nn_matches[i][0]; |
||||
float dist1 = nn_matches[i][0].distance; |
||||
float dist2 = nn_matches[i][1].distance; |
||||
|
||||
if(dist1 < nn_match_ratio * dist2) { |
||||
matched1.push_back(kpts1[first.queryIdx]); |
||||
matched2.push_back(kpts2[first.trainIdx]); |
||||
} |
||||
} |
||||
|
||||
If the closest match is *ratio* closer than the second closest one, then the match is correct. |
||||
|
||||
5. **Check if our matches fit in the homography model** |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
for(int i = 0; i < matched1.size(); i++) { |
||||
Mat col = Mat::ones(3, 1, CV_64F); |
||||
col.at<double>(0) = matched1[i].pt.x; |
||||
col.at<double>(1) = matched1[i].pt.y; |
||||
|
||||
col = homography * col; |
||||
col /= col.at<double>(2); |
||||
float dist = sqrt( pow(col.at<double>(0) - matched2[i].pt.x, 2) + |
||||
pow(col.at<double>(1) - matched2[i].pt.y, 2)); |
||||
|
||||
if(dist < inlier_threshold) { |
||||
int new_i = inliers1.size(); |
||||
inliers1.push_back(matched1[i]); |
||||
inliers2.push_back(matched2[i]); |
||||
good_matches.push_back(DMatch(new_i, new_i, 0)); |
||||
} |
||||
} |
||||
|
||||
If the distance from first keypoint's projection to the second keypoint is less than threshold, then it it fits in the homography. |
||||
|
||||
We create a new set of matches for the inliers, because it is required by the drawing function. |
||||
|
||||
6. **Output results** |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
Mat res; |
||||
drawMatches(img1, inliers1, img2, inliers2, good_matches, res); |
||||
imwrite("res.png", res); |
||||
... |
||||
|
||||
Here we save the resulting image and print some statistics. |
||||
|
||||
Results |
||||
======= |
||||
|
||||
Found matches |
||||
-------------- |
||||
|
||||
.. image:: images/res.png |
||||
:height: 200pt |
||||
:width: 320pt |
||||
:alt: Matches |
||||
:align: center |
||||
|
||||
A-KAZE Matching Results |
||||
-------------------------- |
||||
Keypoints 1: 2943 |
||||
|
||||
Keypoints 2: 3511 |
||||
|
||||
Matches: 447 |
||||
|
||||
Inliers: 308 |
||||
|
||||
Inliers Ratio: 0.689038 |
||||
|
After Width: | Height: | Size: 2.0 MiB |
|
After Width: | Height: | Size: 1.8 MiB |
|
After Width: | Height: | Size: 63 KiB |
File diff suppressed because it is too large
Load Diff
@ -1,65 +0,0 @@ |
||||
/**
|
||||
* @file AKAZE.h |
||||
* @brief Main class for detecting and computing binary descriptors in an |
||||
* accelerated nonlinear scale space |
||||
* @date Mar 27, 2013 |
||||
* @author Pablo F. Alcantarilla, Jesus Nuevo |
||||
*/ |
||||
|
||||
#pragma once |
||||
|
||||
/* ************************************************************************* */ |
||||
// Includes
|
||||
#include "precomp.hpp" |
||||
#include "AKAZEConfig.h" |
||||
|
||||
/* ************************************************************************* */ |
||||
// AKAZE Class Declaration
|
||||
class AKAZEFeatures { |
||||
|
||||
private: |
||||
|
||||
AKAZEOptions options_; ///< Configuration options for AKAZE
|
||||
std::vector<TEvolution> evolution_; ///< Vector of nonlinear diffusion evolution
|
||||
|
||||
/// FED parameters
|
||||
int ncycles_; ///< Number of cycles
|
||||
bool reordering_; ///< Flag for reordering time steps
|
||||
std::vector<std::vector<float > > tsteps_; ///< Vector of FED dynamic time steps
|
||||
std::vector<int> nsteps_; ///< Vector of number of steps per cycle
|
||||
|
||||
/// Matrices for the M-LDB descriptor computation
|
||||
cv::Mat descriptorSamples_; // List of positions in the grids to sample LDB bits from.
|
||||
cv::Mat descriptorBits_; |
||||
cv::Mat bitMask_; |
||||
|
||||
public: |
||||
|
||||
/// Constructor with input arguments
|
||||
AKAZEFeatures(const AKAZEOptions& options); |
||||
|
||||
/// Scale Space methods
|
||||
void Allocate_Memory_Evolution(); |
||||
int Create_Nonlinear_Scale_Space(const cv::Mat& img); |
||||
void Feature_Detection(std::vector<cv::KeyPoint>& kpts); |
||||
void Compute_Determinant_Hessian_Response(void); |
||||
void Compute_Multiscale_Derivatives(void); |
||||
void Find_Scale_Space_Extrema(std::vector<cv::KeyPoint>& kpts); |
||||
void Do_Subpixel_Refinement(std::vector<cv::KeyPoint>& kpts); |
||||
|
||||
// Feature description methods
|
||||
void Compute_Descriptors(std::vector<cv::KeyPoint>& kpts, cv::Mat& desc); |
||||
|
||||
static void Compute_Main_Orientation(cv::KeyPoint& kpt, const std::vector<TEvolution>& evolution_); |
||||
}; |
||||
|
||||
/* ************************************************************************* */ |
||||
// Inline functions
|
||||
|
||||
// Inline functions
|
||||
void generateDescriptorSubsample(cv::Mat& sampleList, cv::Mat& comparisons, |
||||
int nbits, int pattern_size, int nchannels); |
||||
float get_angle(float x, float y); |
||||
float gaussian(float x, float y, float sigma); |
||||
void check_descriptor_limits(int& x, int& y, int width, int height); |
||||
int fRound(float flt); |
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,62 @@ |
||||
/**
|
||||
* @file AKAZE.h |
||||
* @brief Main class for detecting and computing binary descriptors in an |
||||
* accelerated nonlinear scale space |
||||
* @date Mar 27, 2013 |
||||
* @author Pablo F. Alcantarilla, Jesus Nuevo |
||||
*/ |
||||
|
||||
#ifndef __OPENCV_FEATURES_2D_AKAZE_FEATURES_H__ |
||||
#define __OPENCV_FEATURES_2D_AKAZE_FEATURES_H__ |
||||
|
||||
/* ************************************************************************* */ |
||||
// Includes
|
||||
#include "precomp.hpp" |
||||
#include "AKAZEConfig.h" |
||||
#include "TEvolution.h" |
||||
|
||||
/* ************************************************************************* */ |
||||
// AKAZE Class Declaration
|
||||
class AKAZEFeatures { |
||||
|
||||
private: |
||||
|
||||
AKAZEOptions options_; ///< Configuration options for AKAZE
|
||||
std::vector<TEvolution> evolution_; ///< Vector of nonlinear diffusion evolution
|
||||
|
||||
/// FED parameters
|
||||
int ncycles_; ///< Number of cycles
|
||||
bool reordering_; ///< Flag for reordering time steps
|
||||
std::vector<std::vector<float > > tsteps_; ///< Vector of FED dynamic time steps
|
||||
std::vector<int> nsteps_; ///< Vector of number of steps per cycle
|
||||
|
||||
/// Matrices for the M-LDB descriptor computation
|
||||
cv::Mat descriptorSamples_; // List of positions in the grids to sample LDB bits from.
|
||||
cv::Mat descriptorBits_; |
||||
cv::Mat bitMask_; |
||||
|
||||
public: |
||||
|
||||
/// Constructor with input arguments
|
||||
AKAZEFeatures(const AKAZEOptions& options); |
||||
|
||||
/// Scale Space methods
|
||||
void Allocate_Memory_Evolution(); |
||||
int Create_Nonlinear_Scale_Space(const cv::Mat& img); |
||||
void Feature_Detection(std::vector<cv::KeyPoint>& kpts); |
||||
void Compute_Determinant_Hessian_Response(void); |
||||
void Compute_Multiscale_Derivatives(void); |
||||
void Find_Scale_Space_Extrema(std::vector<cv::KeyPoint>& kpts); |
||||
void Do_Subpixel_Refinement(std::vector<cv::KeyPoint>& kpts); |
||||
|
||||
/// Feature description methods
|
||||
void Compute_Descriptors(std::vector<cv::KeyPoint>& kpts, cv::Mat& desc); |
||||
static void Compute_Main_Orientation(cv::KeyPoint& kpt, const std::vector<TEvolution>& evolution_); |
||||
}; |
||||
|
||||
/* ************************************************************************* */ |
||||
/// Inline functions
|
||||
void generateDescriptorSubsample(cv::Mat& sampleList, cv::Mat& comparisons, |
||||
int nbits, int pattern_size, int nchannels); |
||||
|
||||
#endif |
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,35 @@ |
||||
/**
|
||||
* @file TEvolution.h |
||||
* @brief Header file with the declaration of the TEvolution struct |
||||
* @date Jun 02, 2014 |
||||
* @author Pablo F. Alcantarilla |
||||
*/ |
||||
|
||||
#ifndef __OPENCV_FEATURES_2D_TEVOLUTION_H__ |
||||
#define __OPENCV_FEATURES_2D_TEVOLUTION_H__ |
||||
|
||||
/* ************************************************************************* */ |
||||
/// KAZE/A-KAZE nonlinear diffusion filtering evolution
|
||||
struct TEvolution { |
||||
|
||||
TEvolution() { |
||||
etime = 0.0f; |
||||
esigma = 0.0f; |
||||
octave = 0; |
||||
sublevel = 0; |
||||
sigma_size = 0; |
||||
} |
||||
|
||||
cv::Mat Lx, Ly; ///< First order spatial derivatives
|
||||
cv::Mat Lxx, Lxy, Lyy; ///< Second order spatial derivatives
|
||||
cv::Mat Lt; ///< Evolution image
|
||||
cv::Mat Lsmooth; ///< Smoothed image
|
||||
cv::Mat Ldet; ///< Detector response
|
||||
float etime; ///< Evolution time
|
||||
float esigma; ///< Evolution sigma. For linear diffusion t = sigma^2 / 2
|
||||
int octave; ///< Image octave
|
||||
int sublevel; ///< Image sublevel in each octave
|
||||
int sigma_size; ///< Integer esigma. For computing the feature detector responses
|
||||
}; |
||||
|
||||
#endif |
||||
@ -0,0 +1,77 @@ |
||||
#ifndef __OPENCV_FEATURES_2D_KAZE_UTILS_H__ |
||||
#define __OPENCV_FEATURES_2D_KAZE_UTILS_H__ |
||||
|
||||
/* ************************************************************************* */ |
||||
/**
|
||||
* @brief This function computes the angle from the vector given by (X Y). From 0 to 2*Pi |
||||
*/ |
||||
inline float getAngle(float x, float y) { |
||||
|
||||
if (x >= 0 && y >= 0) { |
||||
return atanf(y / x); |
||||
} |
||||
|
||||
if (x < 0 && y >= 0) { |
||||
return static_cast<float>(CV_PI)-atanf(-y / x); |
||||
} |
||||
|
||||
if (x < 0 && y < 0) { |
||||
return static_cast<float>(CV_PI)+atanf(y / x); |
||||
} |
||||
|
||||
if (x >= 0 && y < 0) { |
||||
return static_cast<float>(2.0 * CV_PI) - atanf(-y / x); |
||||
} |
||||
|
||||
return 0; |
||||
} |
||||
|
||||
/* ************************************************************************* */ |
||||
/**
|
||||
* @brief This function computes the value of a 2D Gaussian function |
||||
* @param x X Position |
||||
* @param y Y Position |
||||
* @param sig Standard Deviation |
||||
*/ |
||||
inline float gaussian(float x, float y, float sigma) { |
||||
return expf(-(x*x + y*y) / (2.0f*sigma*sigma)); |
||||
} |
||||
|
||||
/* ************************************************************************* */ |
||||
/**
|
||||
* @brief This function checks descriptor limits |
||||
* @param x X Position |
||||
* @param y Y Position |
||||
* @param width Image width |
||||
* @param height Image height |
||||
*/ |
||||
inline void checkDescriptorLimits(int &x, int &y, int width, int height) { |
||||
|
||||
if (x < 0) { |
||||
x = 0; |
||||
} |
||||
|
||||
if (y < 0) { |
||||
y = 0; |
||||
} |
||||
|
||||
if (x > width - 1) { |
||||
x = width - 1; |
||||
} |
||||
|
||||
if (y > height - 1) { |
||||
y = height - 1; |
||||
} |
||||
} |
||||
|
||||
/* ************************************************************************* */ |
||||
/**
|
||||
* @brief This funtion rounds float to nearest integer |
||||
* @param flt Input float |
||||
* @return dst Nearest integer |
||||
*/ |
||||
inline int fRound(float flt) { |
||||
return (int)(flt + 0.5f); |
||||
} |
||||
|
||||
#endif |
||||
@ -0,0 +1,11 @@ |
||||
<?xml version="1.0"?> |
||||
<opencv_storage> |
||||
<H13 type_id="opencv-matrix"> |
||||
<rows>3</rows> |
||||
<cols>3</cols> |
||||
<dt>d</dt> |
||||
<data> |
||||
7.6285898e-01 -2.9922929e-01 2.2567123e+02 |
||||
3.3443473e-01 1.0143901e+00 -7.6999973e+01 |
||||
3.4663091e-04 -1.4364524e-05 1.0000000e+00 </data></H13> |
||||
</opencv_storage> |
||||
|
After Width: | Height: | Size: 929 KiB |
|
After Width: | Height: | Size: 953 KiB |
@ -0,0 +1,79 @@ |
||||
#include <opencv2/features2d.hpp> |
||||
#include <opencv2/imgcodecs.hpp> |
||||
#include <opencv2/opencv.hpp> |
||||
#include <vector> |
||||
#include <iostream> |
||||
|
||||
using namespace std; |
||||
using namespace cv; |
||||
|
||||
const float inlier_threshold = 2.5f; // Distance threshold to identify inliers
|
||||
const float nn_match_ratio = 0.8f; // Nearest neighbor matching ratio
|
||||
|
||||
int main(void) |
||||
{ |
||||
Mat img1 = imread("graf1.png", IMREAD_GRAYSCALE); |
||||
Mat img2 = imread("graf3.png", IMREAD_GRAYSCALE); |
||||
|
||||
Mat homography; |
||||
FileStorage fs("H1to3p.xml", FileStorage::READ); |
||||
fs.getFirstTopLevelNode() >> homography; |
||||
|
||||
vector<KeyPoint> kpts1, kpts2; |
||||
Mat desc1, desc2; |
||||
|
||||
AKAZE akaze; |
||||
akaze(img1, noArray(), kpts1, desc1); |
||||
akaze(img2, noArray(), kpts2, desc2); |
||||
|
||||
BFMatcher matcher(NORM_HAMMING); |
||||
vector< vector<DMatch> > nn_matches; |
||||
matcher.knnMatch(desc1, desc2, nn_matches, 2); |
||||
|
||||
vector<KeyPoint> matched1, matched2, inliers1, inliers2; |
||||
vector<DMatch> good_matches; |
||||
for(size_t i = 0; i < nn_matches.size(); i++) { |
||||
DMatch first = nn_matches[i][0]; |
||||
float dist1 = nn_matches[i][0].distance; |
||||
float dist2 = nn_matches[i][1].distance; |
||||
|
||||
if(dist1 < nn_match_ratio * dist2) { |
||||
matched1.push_back(kpts1[first.queryIdx]); |
||||
matched2.push_back(kpts2[first.trainIdx]); |
||||
} |
||||
} |
||||
|
||||
for(unsigned i = 0; i < matched1.size(); i++) { |
||||
Mat col = Mat::ones(3, 1, CV_64F); |
||||
col.at<double>(0) = matched1[i].pt.x; |
||||
col.at<double>(1) = matched1[i].pt.y; |
||||
|
||||
col = homography * col; |
||||
col /= col.at<double>(2); |
||||
double dist = sqrt( pow(col.at<double>(0) - matched2[i].pt.x, 2) + |
||||
pow(col.at<double>(1) - matched2[i].pt.y, 2)); |
||||
|
||||
if(dist < inlier_threshold) { |
||||
int new_i = static_cast<int>(inliers1.size()); |
||||
inliers1.push_back(matched1[i]); |
||||
inliers2.push_back(matched2[i]); |
||||
good_matches.push_back(DMatch(new_i, new_i, 0)); |
||||
} |
||||
} |
||||
|
||||
Mat res; |
||||
drawMatches(img1, inliers1, img2, inliers2, good_matches, res); |
||||
imwrite("res.png", res); |
||||
|
||||
double inlier_ratio = inliers1.size() * 1.0 / matched1.size(); |
||||
cout << "A-KAZE Matching Results" << endl; |
||||
cout << "*******************************" << endl; |
||||
cout << "# Keypoints 1: \t" << kpts1.size() << endl; |
||||
cout << "# Keypoints 2: \t" << kpts2.size() << endl; |
||||
cout << "# Matches: \t" << matched1.size() << endl; |
||||
cout << "# Inliers: \t" << inliers1.size() << endl; |
||||
cout << "# Inliers Ratio: \t" << inlier_ratio << endl; |
||||
cout << endl; |
||||
|
||||
return 0; |
||||
} |
||||
Loading…
Reference in new issue