parent
277802483f
commit
7f82960897
29 changed files with 2755 additions and 2767 deletions
@ -0,0 +1,161 @@ |
|||||||
|
.. _akazeMatching: |
||||||
|
|
||||||
|
|
||||||
|
AKAZE local features matching |
||||||
|
****************************** |
||||||
|
|
||||||
|
Introduction |
||||||
|
------------------ |
||||||
|
|
||||||
|
In this tutorial we will learn how to use [AKAZE]_ local features to detect and match keypoints on two images. |
||||||
|
|
||||||
|
We will find keypoints on a pair of images with given homography matrix, |
||||||
|
match them and count the number of inliers (i. e. matches that fit in the given homography). |
||||||
|
|
||||||
|
You can find expanded version of this example here: https://github.com/pablofdezalc/test_kaze_akaze_opencv |
||||||
|
|
||||||
|
.. [AKAZE] Fast Explicit Diffusion for Accelerated Features in Nonlinear Scale Spaces. Pablo F. Alcantarilla, Jesús Nuevo and Adrien Bartoli. In British Machine Vision Conference (BMVC), Bristol, UK, September 2013. |
||||||
|
|
||||||
|
Data |
||||||
|
------------------ |
||||||
|
We are going to use images 1 and 3 from *Graffity* sequence of Oxford dataset. |
||||||
|
|
||||||
|
.. image:: images/graf.png |
||||||
|
:height: 200pt |
||||||
|
:width: 320pt |
||||||
|
:alt: Graffity |
||||||
|
:align: center |
||||||
|
|
||||||
|
Homography is given by a 3 by 3 matrix: |
||||||
|
|
||||||
|
.. code-block:: none |
||||||
|
|
||||||
|
7.6285898e-01 -2.9922929e-01 2.2567123e+02 |
||||||
|
3.3443473e-01 1.0143901e+00 -7.6999973e+01 |
||||||
|
3.4663091e-04 -1.4364524e-05 1.0000000e+00 |
||||||
|
|
||||||
|
You can find the images (*graf1.png*, *graf3.png*) and homography (*H1to3p.xml*) in *opencv/samples/cpp*. |
||||||
|
|
||||||
|
Source Code |
||||||
|
=========== |
||||||
|
.. literalinclude:: ../../../../samples/cpp/tutorial_code/features2D/AKAZE_match.cpp |
||||||
|
:language: cpp |
||||||
|
:linenos: |
||||||
|
:tab-width: 4 |
||||||
|
|
||||||
|
Explanation |
||||||
|
=========== |
||||||
|
|
||||||
|
1. **Load images and homography** |
||||||
|
|
||||||
|
.. code-block:: cpp |
||||||
|
|
||||||
|
Mat img1 = imread("graf1.png", IMREAD_GRAYSCALE); |
||||||
|
Mat img2 = imread("graf3.png", IMREAD_GRAYSCALE); |
||||||
|
|
||||||
|
Mat homography; |
||||||
|
FileStorage fs("H1to3p.xml", FileStorage::READ); |
||||||
|
fs.getFirstTopLevelNode() >> homography; |
||||||
|
|
||||||
|
We are loading grayscale images here. Homography is stored in the xml created with FileStorage. |
||||||
|
|
||||||
|
2. **Detect keypoints and compute descriptors using AKAZE** |
||||||
|
|
||||||
|
.. code-block:: cpp |
||||||
|
|
||||||
|
vector<KeyPoint> kpts1, kpts2; |
||||||
|
Mat desc1, desc2; |
||||||
|
|
||||||
|
AKAZE akaze; |
||||||
|
akaze(img1, noArray(), kpts1, desc1); |
||||||
|
akaze(img2, noArray(), kpts2, desc2); |
||||||
|
|
||||||
|
We create AKAZE object and use it's *operator()* functionality. Since we don't need the *mask* parameter, *noArray()* is used. |
||||||
|
|
||||||
|
3. **Use brute-force matcher to find 2-nn matches** |
||||||
|
|
||||||
|
.. code-block:: cpp |
||||||
|
|
||||||
|
BFMatcher matcher(NORM_HAMMING); |
||||||
|
vector< vector<DMatch> > nn_matches; |
||||||
|
matcher.knnMatch(desc1, desc2, nn_matches, 2); |
||||||
|
|
||||||
|
We use Hamming distance, because AKAZE uses binary descriptor by default. |
||||||
|
|
||||||
|
4. **Use 2-nn matches to find correct keypoint matches** |
||||||
|
|
||||||
|
.. code-block:: cpp |
||||||
|
|
||||||
|
for(size_t i = 0; i < nn_matches.size(); i++) { |
||||||
|
DMatch first = nn_matches[i][0]; |
||||||
|
float dist1 = nn_matches[i][0].distance; |
||||||
|
float dist2 = nn_matches[i][1].distance; |
||||||
|
|
||||||
|
if(dist1 < nn_match_ratio * dist2) { |
||||||
|
matched1.push_back(kpts1[first.queryIdx]); |
||||||
|
matched2.push_back(kpts2[first.trainIdx]); |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
If the closest match is *ratio* closer than the second closest one, then the match is correct. |
||||||
|
|
||||||
|
5. **Check if our matches fit in the homography model** |
||||||
|
|
||||||
|
.. code-block:: cpp |
||||||
|
|
||||||
|
for(int i = 0; i < matched1.size(); i++) { |
||||||
|
Mat col = Mat::ones(3, 1, CV_64F); |
||||||
|
col.at<double>(0) = matched1[i].pt.x; |
||||||
|
col.at<double>(1) = matched1[i].pt.y; |
||||||
|
|
||||||
|
col = homography * col; |
||||||
|
col /= col.at<double>(2); |
||||||
|
float dist = sqrt( pow(col.at<double>(0) - matched2[i].pt.x, 2) + |
||||||
|
pow(col.at<double>(1) - matched2[i].pt.y, 2)); |
||||||
|
|
||||||
|
if(dist < inlier_threshold) { |
||||||
|
int new_i = inliers1.size(); |
||||||
|
inliers1.push_back(matched1[i]); |
||||||
|
inliers2.push_back(matched2[i]); |
||||||
|
good_matches.push_back(DMatch(new_i, new_i, 0)); |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
If the distance from first keypoint's projection to the second keypoint is less than threshold, then it it fits in the homography. |
||||||
|
|
||||||
|
We create a new set of matches for the inliers, because it is required by the drawing function. |
||||||
|
|
||||||
|
6. **Output results** |
||||||
|
|
||||||
|
.. code-block:: cpp |
||||||
|
|
||||||
|
Mat res; |
||||||
|
drawMatches(img1, inliers1, img2, inliers2, good_matches, res); |
||||||
|
imwrite("res.png", res); |
||||||
|
... |
||||||
|
|
||||||
|
Here we save the resulting image and print some statistics. |
||||||
|
|
||||||
|
Results |
||||||
|
======= |
||||||
|
|
||||||
|
Found matches |
||||||
|
-------------- |
||||||
|
|
||||||
|
.. image:: images/res.png |
||||||
|
:height: 200pt |
||||||
|
:width: 320pt |
||||||
|
:alt: Matches |
||||||
|
:align: center |
||||||
|
|
||||||
|
A-KAZE Matching Results |
||||||
|
-------------------------- |
||||||
|
Keypoints 1: 2943 |
||||||
|
|
||||||
|
Keypoints 2: 3511 |
||||||
|
|
||||||
|
Matches: 447 |
||||||
|
|
||||||
|
Inliers: 308 |
||||||
|
|
||||||
|
Inliers Ratio: 0.689038 |
After Width: | Height: | Size: 2.0 MiB |
After Width: | Height: | Size: 1.8 MiB |
After Width: | Height: | Size: 63 KiB |
File diff suppressed because it is too large
Load Diff
@ -1,65 +0,0 @@ |
|||||||
/**
|
|
||||||
* @file AKAZE.h |
|
||||||
* @brief Main class for detecting and computing binary descriptors in an |
|
||||||
* accelerated nonlinear scale space |
|
||||||
* @date Mar 27, 2013 |
|
||||||
* @author Pablo F. Alcantarilla, Jesus Nuevo |
|
||||||
*/ |
|
||||||
|
|
||||||
#pragma once |
|
||||||
|
|
||||||
/* ************************************************************************* */ |
|
||||||
// Includes
|
|
||||||
#include "precomp.hpp" |
|
||||||
#include "AKAZEConfig.h" |
|
||||||
|
|
||||||
/* ************************************************************************* */ |
|
||||||
// AKAZE Class Declaration
|
|
||||||
class AKAZEFeatures { |
|
||||||
|
|
||||||
private: |
|
||||||
|
|
||||||
AKAZEOptions options_; ///< Configuration options for AKAZE
|
|
||||||
std::vector<TEvolution> evolution_; ///< Vector of nonlinear diffusion evolution
|
|
||||||
|
|
||||||
/// FED parameters
|
|
||||||
int ncycles_; ///< Number of cycles
|
|
||||||
bool reordering_; ///< Flag for reordering time steps
|
|
||||||
std::vector<std::vector<float > > tsteps_; ///< Vector of FED dynamic time steps
|
|
||||||
std::vector<int> nsteps_; ///< Vector of number of steps per cycle
|
|
||||||
|
|
||||||
/// Matrices for the M-LDB descriptor computation
|
|
||||||
cv::Mat descriptorSamples_; // List of positions in the grids to sample LDB bits from.
|
|
||||||
cv::Mat descriptorBits_; |
|
||||||
cv::Mat bitMask_; |
|
||||||
|
|
||||||
public: |
|
||||||
|
|
||||||
/// Constructor with input arguments
|
|
||||||
AKAZEFeatures(const AKAZEOptions& options); |
|
||||||
|
|
||||||
/// Scale Space methods
|
|
||||||
void Allocate_Memory_Evolution(); |
|
||||||
int Create_Nonlinear_Scale_Space(const cv::Mat& img); |
|
||||||
void Feature_Detection(std::vector<cv::KeyPoint>& kpts); |
|
||||||
void Compute_Determinant_Hessian_Response(void); |
|
||||||
void Compute_Multiscale_Derivatives(void); |
|
||||||
void Find_Scale_Space_Extrema(std::vector<cv::KeyPoint>& kpts); |
|
||||||
void Do_Subpixel_Refinement(std::vector<cv::KeyPoint>& kpts); |
|
||||||
|
|
||||||
// Feature description methods
|
|
||||||
void Compute_Descriptors(std::vector<cv::KeyPoint>& kpts, cv::Mat& desc); |
|
||||||
|
|
||||||
static void Compute_Main_Orientation(cv::KeyPoint& kpt, const std::vector<TEvolution>& evolution_); |
|
||||||
}; |
|
||||||
|
|
||||||
/* ************************************************************************* */ |
|
||||||
// Inline functions
|
|
||||||
|
|
||||||
// Inline functions
|
|
||||||
void generateDescriptorSubsample(cv::Mat& sampleList, cv::Mat& comparisons, |
|
||||||
int nbits, int pattern_size, int nchannels); |
|
||||||
float get_angle(float x, float y); |
|
||||||
float gaussian(float x, float y, float sigma); |
|
||||||
void check_descriptor_limits(int& x, int& y, int width, int height); |
|
||||||
int fRound(float flt); |
|
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,62 @@ |
|||||||
|
/**
|
||||||
|
* @file AKAZE.h |
||||||
|
* @brief Main class for detecting and computing binary descriptors in an |
||||||
|
* accelerated nonlinear scale space |
||||||
|
* @date Mar 27, 2013 |
||||||
|
* @author Pablo F. Alcantarilla, Jesus Nuevo |
||||||
|
*/ |
||||||
|
|
||||||
|
#ifndef __OPENCV_FEATURES_2D_AKAZE_FEATURES_H__ |
||||||
|
#define __OPENCV_FEATURES_2D_AKAZE_FEATURES_H__ |
||||||
|
|
||||||
|
/* ************************************************************************* */ |
||||||
|
// Includes
|
||||||
|
#include "precomp.hpp" |
||||||
|
#include "AKAZEConfig.h" |
||||||
|
#include "TEvolution.h" |
||||||
|
|
||||||
|
/* ************************************************************************* */ |
||||||
|
// AKAZE Class Declaration
|
||||||
|
class AKAZEFeatures { |
||||||
|
|
||||||
|
private: |
||||||
|
|
||||||
|
AKAZEOptions options_; ///< Configuration options for AKAZE
|
||||||
|
std::vector<TEvolution> evolution_; ///< Vector of nonlinear diffusion evolution
|
||||||
|
|
||||||
|
/// FED parameters
|
||||||
|
int ncycles_; ///< Number of cycles
|
||||||
|
bool reordering_; ///< Flag for reordering time steps
|
||||||
|
std::vector<std::vector<float > > tsteps_; ///< Vector of FED dynamic time steps
|
||||||
|
std::vector<int> nsteps_; ///< Vector of number of steps per cycle
|
||||||
|
|
||||||
|
/// Matrices for the M-LDB descriptor computation
|
||||||
|
cv::Mat descriptorSamples_; // List of positions in the grids to sample LDB bits from.
|
||||||
|
cv::Mat descriptorBits_; |
||||||
|
cv::Mat bitMask_; |
||||||
|
|
||||||
|
public: |
||||||
|
|
||||||
|
/// Constructor with input arguments
|
||||||
|
AKAZEFeatures(const AKAZEOptions& options); |
||||||
|
|
||||||
|
/// Scale Space methods
|
||||||
|
void Allocate_Memory_Evolution(); |
||||||
|
int Create_Nonlinear_Scale_Space(const cv::Mat& img); |
||||||
|
void Feature_Detection(std::vector<cv::KeyPoint>& kpts); |
||||||
|
void Compute_Determinant_Hessian_Response(void); |
||||||
|
void Compute_Multiscale_Derivatives(void); |
||||||
|
void Find_Scale_Space_Extrema(std::vector<cv::KeyPoint>& kpts); |
||||||
|
void Do_Subpixel_Refinement(std::vector<cv::KeyPoint>& kpts); |
||||||
|
|
||||||
|
/// Feature description methods
|
||||||
|
void Compute_Descriptors(std::vector<cv::KeyPoint>& kpts, cv::Mat& desc); |
||||||
|
static void Compute_Main_Orientation(cv::KeyPoint& kpt, const std::vector<TEvolution>& evolution_); |
||||||
|
}; |
||||||
|
|
||||||
|
/* ************************************************************************* */ |
||||||
|
/// Inline functions
|
||||||
|
void generateDescriptorSubsample(cv::Mat& sampleList, cv::Mat& comparisons, |
||||||
|
int nbits, int pattern_size, int nchannels); |
||||||
|
|
||||||
|
#endif |
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,35 @@ |
|||||||
|
/**
|
||||||
|
* @file TEvolution.h |
||||||
|
* @brief Header file with the declaration of the TEvolution struct |
||||||
|
* @date Jun 02, 2014 |
||||||
|
* @author Pablo F. Alcantarilla |
||||||
|
*/ |
||||||
|
|
||||||
|
#ifndef __OPENCV_FEATURES_2D_TEVOLUTION_H__ |
||||||
|
#define __OPENCV_FEATURES_2D_TEVOLUTION_H__ |
||||||
|
|
||||||
|
/* ************************************************************************* */ |
||||||
|
/// KAZE/A-KAZE nonlinear diffusion filtering evolution
|
||||||
|
struct TEvolution { |
||||||
|
|
||||||
|
TEvolution() { |
||||||
|
etime = 0.0f; |
||||||
|
esigma = 0.0f; |
||||||
|
octave = 0; |
||||||
|
sublevel = 0; |
||||||
|
sigma_size = 0; |
||||||
|
} |
||||||
|
|
||||||
|
cv::Mat Lx, Ly; ///< First order spatial derivatives
|
||||||
|
cv::Mat Lxx, Lxy, Lyy; ///< Second order spatial derivatives
|
||||||
|
cv::Mat Lt; ///< Evolution image
|
||||||
|
cv::Mat Lsmooth; ///< Smoothed image
|
||||||
|
cv::Mat Ldet; ///< Detector response
|
||||||
|
float etime; ///< Evolution time
|
||||||
|
float esigma; ///< Evolution sigma. For linear diffusion t = sigma^2 / 2
|
||||||
|
int octave; ///< Image octave
|
||||||
|
int sublevel; ///< Image sublevel in each octave
|
||||||
|
int sigma_size; ///< Integer esigma. For computing the feature detector responses
|
||||||
|
}; |
||||||
|
|
||||||
|
#endif |
@ -0,0 +1,77 @@ |
|||||||
|
#ifndef __OPENCV_FEATURES_2D_KAZE_UTILS_H__ |
||||||
|
#define __OPENCV_FEATURES_2D_KAZE_UTILS_H__ |
||||||
|
|
||||||
|
/* ************************************************************************* */ |
||||||
|
/**
|
||||||
|
* @brief This function computes the angle from the vector given by (X Y). From 0 to 2*Pi |
||||||
|
*/ |
||||||
|
inline float getAngle(float x, float y) { |
||||||
|
|
||||||
|
if (x >= 0 && y >= 0) { |
||||||
|
return atanf(y / x); |
||||||
|
} |
||||||
|
|
||||||
|
if (x < 0 && y >= 0) { |
||||||
|
return static_cast<float>(CV_PI)-atanf(-y / x); |
||||||
|
} |
||||||
|
|
||||||
|
if (x < 0 && y < 0) { |
||||||
|
return static_cast<float>(CV_PI)+atanf(y / x); |
||||||
|
} |
||||||
|
|
||||||
|
if (x >= 0 && y < 0) { |
||||||
|
return static_cast<float>(2.0 * CV_PI) - atanf(-y / x); |
||||||
|
} |
||||||
|
|
||||||
|
return 0; |
||||||
|
} |
||||||
|
|
||||||
|
/* ************************************************************************* */ |
||||||
|
/**
|
||||||
|
* @brief This function computes the value of a 2D Gaussian function |
||||||
|
* @param x X Position |
||||||
|
* @param y Y Position |
||||||
|
* @param sig Standard Deviation |
||||||
|
*/ |
||||||
|
inline float gaussian(float x, float y, float sigma) { |
||||||
|
return expf(-(x*x + y*y) / (2.0f*sigma*sigma)); |
||||||
|
} |
||||||
|
|
||||||
|
/* ************************************************************************* */ |
||||||
|
/**
|
||||||
|
* @brief This function checks descriptor limits |
||||||
|
* @param x X Position |
||||||
|
* @param y Y Position |
||||||
|
* @param width Image width |
||||||
|
* @param height Image height |
||||||
|
*/ |
||||||
|
inline void checkDescriptorLimits(int &x, int &y, int width, int height) { |
||||||
|
|
||||||
|
if (x < 0) { |
||||||
|
x = 0; |
||||||
|
} |
||||||
|
|
||||||
|
if (y < 0) { |
||||||
|
y = 0; |
||||||
|
} |
||||||
|
|
||||||
|
if (x > width - 1) { |
||||||
|
x = width - 1; |
||||||
|
} |
||||||
|
|
||||||
|
if (y > height - 1) { |
||||||
|
y = height - 1; |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
/* ************************************************************************* */ |
||||||
|
/**
|
||||||
|
* @brief This funtion rounds float to nearest integer |
||||||
|
* @param flt Input float |
||||||
|
* @return dst Nearest integer |
||||||
|
*/ |
||||||
|
inline int fRound(float flt) { |
||||||
|
return (int)(flt + 0.5f); |
||||||
|
} |
||||||
|
|
||||||
|
#endif |
@ -0,0 +1,11 @@ |
|||||||
|
<?xml version="1.0"?> |
||||||
|
<opencv_storage> |
||||||
|
<H13 type_id="opencv-matrix"> |
||||||
|
<rows>3</rows> |
||||||
|
<cols>3</cols> |
||||||
|
<dt>d</dt> |
||||||
|
<data> |
||||||
|
7.6285898e-01 -2.9922929e-01 2.2567123e+02 |
||||||
|
3.3443473e-01 1.0143901e+00 -7.6999973e+01 |
||||||
|
3.4663091e-04 -1.4364524e-05 1.0000000e+00 </data></H13> |
||||||
|
</opencv_storage> |
After Width: | Height: | Size: 929 KiB |
After Width: | Height: | Size: 953 KiB |
@ -0,0 +1,79 @@ |
|||||||
|
#include <opencv2/features2d.hpp> |
||||||
|
#include <opencv2/imgcodecs.hpp> |
||||||
|
#include <opencv2/opencv.hpp> |
||||||
|
#include <vector> |
||||||
|
#include <iostream> |
||||||
|
|
||||||
|
using namespace std; |
||||||
|
using namespace cv; |
||||||
|
|
||||||
|
const float inlier_threshold = 2.5f; // Distance threshold to identify inliers
|
||||||
|
const float nn_match_ratio = 0.8f; // Nearest neighbor matching ratio
|
||||||
|
|
||||||
|
int main(void) |
||||||
|
{ |
||||||
|
Mat img1 = imread("graf1.png", IMREAD_GRAYSCALE); |
||||||
|
Mat img2 = imread("graf3.png", IMREAD_GRAYSCALE); |
||||||
|
|
||||||
|
Mat homography; |
||||||
|
FileStorage fs("H1to3p.xml", FileStorage::READ); |
||||||
|
fs.getFirstTopLevelNode() >> homography; |
||||||
|
|
||||||
|
vector<KeyPoint> kpts1, kpts2; |
||||||
|
Mat desc1, desc2; |
||||||
|
|
||||||
|
AKAZE akaze; |
||||||
|
akaze(img1, noArray(), kpts1, desc1); |
||||||
|
akaze(img2, noArray(), kpts2, desc2); |
||||||
|
|
||||||
|
BFMatcher matcher(NORM_HAMMING); |
||||||
|
vector< vector<DMatch> > nn_matches; |
||||||
|
matcher.knnMatch(desc1, desc2, nn_matches, 2); |
||||||
|
|
||||||
|
vector<KeyPoint> matched1, matched2, inliers1, inliers2; |
||||||
|
vector<DMatch> good_matches; |
||||||
|
for(size_t i = 0; i < nn_matches.size(); i++) { |
||||||
|
DMatch first = nn_matches[i][0]; |
||||||
|
float dist1 = nn_matches[i][0].distance; |
||||||
|
float dist2 = nn_matches[i][1].distance; |
||||||
|
|
||||||
|
if(dist1 < nn_match_ratio * dist2) { |
||||||
|
matched1.push_back(kpts1[first.queryIdx]); |
||||||
|
matched2.push_back(kpts2[first.trainIdx]); |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
for(unsigned i = 0; i < matched1.size(); i++) { |
||||||
|
Mat col = Mat::ones(3, 1, CV_64F); |
||||||
|
col.at<double>(0) = matched1[i].pt.x; |
||||||
|
col.at<double>(1) = matched1[i].pt.y; |
||||||
|
|
||||||
|
col = homography * col; |
||||||
|
col /= col.at<double>(2); |
||||||
|
double dist = sqrt( pow(col.at<double>(0) - matched2[i].pt.x, 2) + |
||||||
|
pow(col.at<double>(1) - matched2[i].pt.y, 2)); |
||||||
|
|
||||||
|
if(dist < inlier_threshold) { |
||||||
|
int new_i = static_cast<int>(inliers1.size()); |
||||||
|
inliers1.push_back(matched1[i]); |
||||||
|
inliers2.push_back(matched2[i]); |
||||||
|
good_matches.push_back(DMatch(new_i, new_i, 0)); |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
Mat res; |
||||||
|
drawMatches(img1, inliers1, img2, inliers2, good_matches, res); |
||||||
|
imwrite("res.png", res); |
||||||
|
|
||||||
|
double inlier_ratio = inliers1.size() * 1.0 / matched1.size(); |
||||||
|
cout << "A-KAZE Matching Results" << endl; |
||||||
|
cout << "*******************************" << endl; |
||||||
|
cout << "# Keypoints 1: \t" << kpts1.size() << endl; |
||||||
|
cout << "# Keypoints 2: \t" << kpts2.size() << endl; |
||||||
|
cout << "# Matches: \t" << matched1.size() << endl; |
||||||
|
cout << "# Inliers: \t" << inliers1.size() << endl; |
||||||
|
cout << "# Inliers Ratio: \t" << inlier_ratio << endl; |
||||||
|
cout << endl; |
||||||
|
|
||||||
|
return 0; |
||||||
|
} |
Loading…
Reference in new issue