Maksim Shabunin
11 years ago
parent
812ce48c36
commit
4ccbd44559
8 changed files with 857 additions and 1 deletions
@ -0,0 +1,110 @@ |
|||||||
|
Features2d {#tutorial_ug_features2d} |
||||||
|
========== |
||||||
|
|
||||||
|
Detectors |
||||||
|
--------- |
||||||
|
|
||||||
|
Descriptors |
||||||
|
----------- |
||||||
|
|
||||||
|
Matching keypoints |
||||||
|
------------------ |
||||||
|
|
||||||
|
### The code |
||||||
|
|
||||||
|
We will start with a short sample \`opencv/samples/cpp/matcher_simple.cpp\`: |
||||||
|
|
||||||
|
@code{.cpp} |
||||||
|
Mat img1 = imread(argv[1], IMREAD_GRAYSCALE); |
||||||
|
Mat img2 = imread(argv[2], IMREAD_GRAYSCALE); |
||||||
|
if(img1.empty() || img2.empty()) |
||||||
|
{ |
||||||
|
printf("Can't read one of the images\n"); |
||||||
|
return -1; |
||||||
|
} |
||||||
|
|
||||||
|
// detecting keypoints |
||||||
|
SurfFeatureDetector detector(400); |
||||||
|
vector<KeyPoint> keypoints1, keypoints2; |
||||||
|
detector.detect(img1, keypoints1); |
||||||
|
detector.detect(img2, keypoints2); |
||||||
|
|
||||||
|
// computing descriptors |
||||||
|
SurfDescriptorExtractor extractor; |
||||||
|
Mat descriptors1, descriptors2; |
||||||
|
extractor.compute(img1, keypoints1, descriptors1); |
||||||
|
extractor.compute(img2, keypoints2, descriptors2); |
||||||
|
|
||||||
|
// matching descriptors |
||||||
|
BruteForceMatcher<L2<float> > matcher; |
||||||
|
vector<DMatch> matches; |
||||||
|
matcher.match(descriptors1, descriptors2, matches); |
||||||
|
|
||||||
|
// drawing the results |
||||||
|
namedWindow("matches", 1); |
||||||
|
Mat img_matches; |
||||||
|
drawMatches(img1, keypoints1, img2, keypoints2, matches, img_matches); |
||||||
|
imshow("matches", img_matches); |
||||||
|
waitKey(0); |
||||||
|
@endcode |
||||||
|
|
||||||
|
### The code explained |
||||||
|
|
||||||
|
Let us break the code down. |
||||||
|
@code{.cpp} |
||||||
|
Mat img1 = imread(argv[1], IMREAD_GRAYSCALE); |
||||||
|
Mat img2 = imread(argv[2], IMREAD_GRAYSCALE); |
||||||
|
if(img1.empty() || img2.empty()) |
||||||
|
{ |
||||||
|
printf("Can't read one of the images\n"); |
||||||
|
return -1; |
||||||
|
} |
||||||
|
@endcode |
||||||
|
We load two images and check if they are loaded correctly. |
||||||
|
@code{.cpp} |
||||||
|
// detecting keypoints |
||||||
|
Ptr<FeatureDetector> detector = FastFeatureDetector::create(15); |
||||||
|
vector<KeyPoint> keypoints1, keypoints2; |
||||||
|
detector->detect(img1, keypoints1); |
||||||
|
detector->detect(img2, keypoints2); |
||||||
|
@endcode |
||||||
|
First, we create an instance of a keypoint detector. All detectors inherit the abstract |
||||||
|
FeatureDetector interface, but the constructors are algorithm-dependent. The first argument to each |
||||||
|
detector usually controls the balance between the amount of keypoints and their stability. The range |
||||||
|
of values is different for different detectors (For instance, *FAST* threshold has the meaning of |
||||||
|
pixel intensity difference and usually varies in the region *[0,40]*. *SURF* threshold is applied to |
||||||
|
a Hessian of an image and usually takes on values larger than *100*), so use defaults in case of |
||||||
|
doubt. |
||||||
|
@code{.cpp} |
||||||
|
// computing descriptors |
||||||
|
Ptr<SURF> extractor = SURF::create(); |
||||||
|
Mat descriptors1, descriptors2; |
||||||
|
extractor->compute(img1, keypoints1, descriptors1); |
||||||
|
extractor->compute(img2, keypoints2, descriptors2); |
||||||
|
@endcode |
||||||
|
We create an instance of descriptor extractor. The most of OpenCV descriptors inherit |
||||||
|
DescriptorExtractor abstract interface. Then we compute descriptors for each of the keypoints. The |
||||||
|
output Mat of the DescriptorExtractor::compute method contains a descriptor in a row *i* for each |
||||||
|
*i*-th keypoint. Note that the method can modify the keypoints vector by removing the keypoints such |
||||||
|
that a descriptor for them is not defined (usually these are the keypoints near image border). The |
||||||
|
method makes sure that the ouptut keypoints and descriptors are consistent with each other (so that |
||||||
|
the number of keypoints is equal to the descriptors row count). : |
||||||
|
@code{.cpp} |
||||||
|
// matching descriptors |
||||||
|
BruteForceMatcher<L2<float> > matcher; |
||||||
|
vector<DMatch> matches; |
||||||
|
matcher.match(descriptors1, descriptors2, matches); |
||||||
|
@endcode |
||||||
|
Now that we have descriptors for both images, we can match them. First, we create a matcher that for |
||||||
|
each descriptor from image 2 does exhaustive search for the nearest descriptor in image 1 using |
||||||
|
Euclidean metric. Manhattan distance is also implemented as well as a Hamming distance for Brief |
||||||
|
descriptor. The output vector matches contains pairs of corresponding points indices. : |
||||||
|
@code{.cpp} |
||||||
|
// drawing the results |
||||||
|
namedWindow("matches", 1); |
||||||
|
Mat img_matches; |
||||||
|
drawMatches(img1, keypoints1, img2, keypoints2, matches, img_matches); |
||||||
|
imshow("matches", img_matches); |
||||||
|
waitKey(0); |
||||||
|
@endcode |
||||||
|
The final part of the sample is about visualizing the matching results. |
||||||
@ -0,0 +1,141 @@ |
|||||||
|
HighGUI {#tutorial_ug_highgui} |
||||||
|
======= |
||||||
|
|
||||||
|
Using Kinect and other OpenNI compatible depth sensors |
||||||
|
------------------------------------------------------ |
||||||
|
|
||||||
|
Depth sensors compatible with OpenNI (Kinect, XtionPRO, ...) are supported through VideoCapture |
||||||
|
class. Depth map, RGB image and some other formats of output can be retrieved by using familiar |
||||||
|
interface of VideoCapture. |
||||||
|
|
||||||
|
In order to use depth sensor with OpenCV you should do the following preliminary steps: |
||||||
|
|
||||||
|
-# Install OpenNI library (from here <http://www.openni.org/downloadfiles>) and PrimeSensor Module |
||||||
|
for OpenNI (from here <https://github.com/avin2/SensorKinect>). The installation should be done |
||||||
|
to default folders listed in the instructions of these products, e.g.: |
||||||
|
@code{.text} |
||||||
|
OpenNI: |
||||||
|
Linux & MacOSX: |
||||||
|
Libs into: /usr/lib |
||||||
|
Includes into: /usr/include/ni |
||||||
|
Windows: |
||||||
|
Libs into: c:/Program Files/OpenNI/Lib |
||||||
|
Includes into: c:/Program Files/OpenNI/Include |
||||||
|
PrimeSensor Module: |
||||||
|
Linux & MacOSX: |
||||||
|
Bins into: /usr/bin |
||||||
|
Windows: |
||||||
|
Bins into: c:/Program Files/Prime Sense/Sensor/Bin |
||||||
|
@endcode |
||||||
|
If one or both products were installed to the other folders, the user should change |
||||||
|
corresponding CMake variables OPENNI_LIB_DIR, OPENNI_INCLUDE_DIR or/and |
||||||
|
OPENNI_PRIME_SENSOR_MODULE_BIN_DIR. |
||||||
|
|
||||||
|
-# Configure OpenCV with OpenNI support by setting WITH_OPENNI flag in CMake. If OpenNI is found |
||||||
|
in install folders OpenCV will be built with OpenNI library (see a status OpenNI in CMake log) |
||||||
|
whereas PrimeSensor Modules can not be found (see a status OpenNI PrimeSensor Modules in CMake |
||||||
|
log). Without PrimeSensor module OpenCV will be successfully compiled with OpenNI library, but |
||||||
|
VideoCapture object will not grab data from Kinect sensor. |
||||||
|
|
||||||
|
-# Build OpenCV. |
||||||
|
|
||||||
|
VideoCapture can retrieve the following data: |
||||||
|
|
||||||
|
-# data given from depth generator: |
||||||
|
- CAP_OPENNI_DEPTH_MAP - depth values in mm (CV_16UC1) |
||||||
|
- CAP_OPENNI_POINT_CLOUD_MAP - XYZ in meters (CV_32FC3) |
||||||
|
- CAP_OPENNI_DISPARITY_MAP - disparity in pixels (CV_8UC1) |
||||||
|
- CAP_OPENNI_DISPARITY_MAP_32F - disparity in pixels (CV_32FC1) |
||||||
|
- CAP_OPENNI_VALID_DEPTH_MASK - mask of valid pixels (not ocluded, not shaded etc.) |
||||||
|
(CV_8UC1) |
||||||
|
|
||||||
|
-# data given from RGB image generator: |
||||||
|
- CAP_OPENNI_BGR_IMAGE - color image (CV_8UC3) |
||||||
|
- CAP_OPENNI_GRAY_IMAGE - gray image (CV_8UC1) |
||||||
|
|
||||||
|
In order to get depth map from depth sensor use VideoCapture::operator \>\>, e. g. : |
||||||
|
@code{.cpp} |
||||||
|
VideoCapture capture( CAP_OPENNI ); |
||||||
|
for(;;) |
||||||
|
{ |
||||||
|
Mat depthMap; |
||||||
|
capture >> depthMap; |
||||||
|
|
||||||
|
if( waitKey( 30 ) >= 0 ) |
||||||
|
break; |
||||||
|
} |
||||||
|
@endcode |
||||||
|
For getting several data maps use VideoCapture::grab and VideoCapture::retrieve, e.g. : |
||||||
|
@code{.cpp} |
||||||
|
VideoCapture capture(0); // or CAP_OPENNI |
||||||
|
for(;;) |
||||||
|
{ |
||||||
|
Mat depthMap; |
||||||
|
Mat bgrImage; |
||||||
|
|
||||||
|
capture.grab(); |
||||||
|
|
||||||
|
capture.retrieve( depthMap, CAP_OPENNI_DEPTH_MAP ); |
||||||
|
capture.retrieve( bgrImage, CAP_OPENNI_BGR_IMAGE ); |
||||||
|
|
||||||
|
if( waitKey( 30 ) >= 0 ) |
||||||
|
break; |
||||||
|
} |
||||||
|
@endcode |
||||||
|
For setting and getting some property of sensor\` data generators use VideoCapture::set and |
||||||
|
VideoCapture::get methods respectively, e.g. : |
||||||
|
@code{.cpp} |
||||||
|
VideoCapture capture( CAP_OPENNI ); |
||||||
|
capture.set( CAP_OPENNI_IMAGE_GENERATOR_OUTPUT_MODE, CAP_OPENNI_VGA_30HZ ); |
||||||
|
cout << "FPS " << capture.get( CAP_OPENNI_IMAGE_GENERATOR+CAP_PROP_FPS ) << endl; |
||||||
|
@endcode |
||||||
|
Since two types of sensor's data generators are supported (image generator and depth generator), |
||||||
|
there are two flags that should be used to set/get property of the needed generator: |
||||||
|
|
||||||
|
- CAP_OPENNI_IMAGE_GENERATOR -- A flag for access to the image generator properties. |
||||||
|
- CAP_OPENNI_DEPTH_GENERATOR -- A flag for access to the depth generator properties. This flag |
||||||
|
value is assumed by default if neither of the two possible values of the property is not set. |
||||||
|
|
||||||
|
Some depth sensors (for example XtionPRO) do not have image generator. In order to check it you can |
||||||
|
get CAP_OPENNI_IMAGE_GENERATOR_PRESENT property. |
||||||
|
@code{.cpp} |
||||||
|
bool isImageGeneratorPresent = capture.get( CAP_PROP_OPENNI_IMAGE_GENERATOR_PRESENT ) != 0; // or == 1 |
||||||
|
@endcode |
||||||
|
Flags specifing the needed generator type must be used in combination with particular generator |
||||||
|
property. The following properties of cameras available through OpenNI interfaces are supported: |
||||||
|
|
||||||
|
- For image generator: |
||||||
|
|
||||||
|
- CAP_PROP_OPENNI_OUTPUT_MODE -- Three output modes are supported: CAP_OPENNI_VGA_30HZ |
||||||
|
used by default (image generator returns images in VGA resolution with 30 FPS), |
||||||
|
CAP_OPENNI_SXGA_15HZ (image generator returns images in SXGA resolution with 15 FPS) and |
||||||
|
CAP_OPENNI_SXGA_30HZ (image generator returns images in SXGA resolution with 30 FPS, the |
||||||
|
mode is supported by XtionPRO Live); depth generator's maps are always in VGA resolution. |
||||||
|
|
||||||
|
- For depth generator: |
||||||
|
|
||||||
|
- CAP_PROP_OPENNI_REGISTRATION -- Flag that registers the remapping depth map to image map |
||||||
|
by changing depth generator's view point (if the flag is "on") or sets this view point to |
||||||
|
its normal one (if the flag is "off"). The registration process’s resulting images are |
||||||
|
pixel-aligned,which means that every pixel in the image is aligned to a pixel in the depth |
||||||
|
image. |
||||||
|
|
||||||
|
Next properties are available for getting only: |
||||||
|
|
||||||
|
- CAP_PROP_OPENNI_FRAME_MAX_DEPTH -- A maximum supported depth of Kinect in mm. |
||||||
|
- CAP_PROP_OPENNI_BASELINE -- Baseline value in mm. |
||||||
|
- CAP_PROP_OPENNI_FOCAL_LENGTH -- A focal length in pixels. |
||||||
|
- CAP_PROP_FRAME_WIDTH -- Frame width in pixels. |
||||||
|
- CAP_PROP_FRAME_HEIGHT -- Frame height in pixels. |
||||||
|
- CAP_PROP_FPS -- Frame rate in FPS. |
||||||
|
|
||||||
|
- Some typical flags combinations "generator type + property" are defined as single flags: |
||||||
|
|
||||||
|
- CAP_OPENNI_IMAGE_GENERATOR_OUTPUT_MODE = CAP_OPENNI_IMAGE_GENERATOR + CAP_PROP_OPENNI_OUTPUT_MODE |
||||||
|
- CAP_OPENNI_DEPTH_GENERATOR_BASELINE = CAP_OPENNI_DEPTH_GENERATOR + CAP_PROP_OPENNI_BASELINE |
||||||
|
- CAP_OPENNI_DEPTH_GENERATOR_FOCAL_LENGTH = CAP_OPENNI_DEPTH_GENERATOR + CAP_PROP_OPENNI_FOCAL_LENGTH |
||||||
|
- CAP_OPENNI_DEPTH_GENERATOR_REGISTRATION = CAP_OPENNI_DEPTH_GENERATOR + CAP_PROP_OPENNI_REGISTRATION |
||||||
|
|
||||||
|
For more information please refer to the example of usage |
||||||
|
[openniccaptureccpp](https://github.com/Itseez/opencv/tree/master/samples/cpp/openni_capture.cpp) in |
||||||
|
opencv/samples/cpp folder. |
||||||
@ -0,0 +1,85 @@ |
|||||||
|
HighGUI {#tutorial_ug_intelperc} |
||||||
|
======= |
||||||
|
|
||||||
|
Using Creative Senz3D and other Intel Perceptual Computing SDK compatible depth sensors |
||||||
|
--------------------------------------------------------------------------------------- |
||||||
|
|
||||||
|
Depth sensors compatible with Intel Perceptual Computing SDK are supported through VideoCapture |
||||||
|
class. Depth map, RGB image and some other formats of output can be retrieved by using familiar |
||||||
|
interface of VideoCapture. |
||||||
|
|
||||||
|
In order to use depth sensor with OpenCV you should do the following preliminary steps: |
||||||
|
|
||||||
|
-# Install Intel Perceptual Computing SDK (from here <http://www.intel.com/software/perceptual>). |
||||||
|
|
||||||
|
-# Configure OpenCV with Intel Perceptual Computing SDK support by setting WITH_INTELPERC flag in |
||||||
|
CMake. If Intel Perceptual Computing SDK is found in install folders OpenCV will be built with |
||||||
|
Intel Perceptual Computing SDK library (see a status INTELPERC in CMake log). If CMake process |
||||||
|
doesn't find Intel Perceptual Computing SDK installation folder automatically, the user should |
||||||
|
change corresponding CMake variables INTELPERC_LIB_DIR and INTELPERC_INCLUDE_DIR to the |
||||||
|
proper value. |
||||||
|
|
||||||
|
-# Build OpenCV. |
||||||
|
|
||||||
|
VideoCapture can retrieve the following data: |
||||||
|
|
||||||
|
-# data given from depth generator: |
||||||
|
- CAP_INTELPERC_DEPTH_MAP - each pixel is a 16-bit integer. The value indicates the |
||||||
|
distance from an object to the camera's XY plane or the Cartesian depth. (CV_16UC1) |
||||||
|
- CAP_INTELPERC_UVDEPTH_MAP - each pixel contains two 32-bit floating point values in |
||||||
|
the range of 0-1, representing the mapping of depth coordinates to the color |
||||||
|
coordinates. (CV_32FC2) |
||||||
|
- CAP_INTELPERC_IR_MAP - each pixel is a 16-bit integer. The value indicates the |
||||||
|
intensity of the reflected laser beam. (CV_16UC1) |
||||||
|
|
||||||
|
-# data given from RGB image generator: |
||||||
|
- CAP_INTELPERC_IMAGE - color image. (CV_8UC3) |
||||||
|
|
||||||
|
In order to get depth map from depth sensor use VideoCapture::operator \>\>, e. g. : |
||||||
|
@code{.cpp} |
||||||
|
VideoCapture capture( CAP_INTELPERC ); |
||||||
|
for(;;) |
||||||
|
{ |
||||||
|
Mat depthMap; |
||||||
|
capture >> depthMap; |
||||||
|
|
||||||
|
if( waitKey( 30 ) >= 0 ) |
||||||
|
break; |
||||||
|
} |
||||||
|
@endcode |
||||||
|
For getting several data maps use VideoCapture::grab and VideoCapture::retrieve, e.g. : |
||||||
|
@code{.cpp} |
||||||
|
VideoCapture capture(CAP_INTELPERC); |
||||||
|
for(;;) |
||||||
|
{ |
||||||
|
Mat depthMap; |
||||||
|
Mat image; |
||||||
|
Mat irImage; |
||||||
|
|
||||||
|
capture.grab(); |
||||||
|
|
||||||
|
capture.retrieve( depthMap, CAP_INTELPERC_DEPTH_MAP ); |
||||||
|
capture.retrieve( image, CAP_INTELPERC_IMAGE ); |
||||||
|
capture.retrieve( irImage, CAP_INTELPERC_IR_MAP); |
||||||
|
|
||||||
|
if( waitKey( 30 ) >= 0 ) |
||||||
|
break; |
||||||
|
} |
||||||
|
@endcode |
||||||
|
For setting and getting some property of sensor\` data generators use VideoCapture::set and |
||||||
|
VideoCapture::get methods respectively, e.g. : |
||||||
|
@code{.cpp} |
||||||
|
VideoCapture capture( CAP_INTELPERC ); |
||||||
|
capture.set( CAP_INTELPERC_DEPTH_GENERATOR | CAP_PROP_INTELPERC_PROFILE_IDX, 0 ); |
||||||
|
cout << "FPS " << capture.get( CAP_INTELPERC_DEPTH_GENERATOR+CAP_PROP_FPS ) << endl; |
||||||
|
@endcode |
||||||
|
Since two types of sensor's data generators are supported (image generator and depth generator), |
||||||
|
there are two flags that should be used to set/get property of the needed generator: |
||||||
|
|
||||||
|
- CAP_INTELPERC_IMAGE_GENERATOR -- a flag for access to the image generator properties. |
||||||
|
- CAP_INTELPERC_DEPTH_GENERATOR -- a flag for access to the depth generator properties. This |
||||||
|
flag value is assumed by default if neither of the two possible values of the property is set. |
||||||
|
|
||||||
|
For more information please refer to the example of usage |
||||||
|
[intelpercccaptureccpp](https://github.com/Itseez/opencv/tree/master/samples/cpp/intelperc_capture.cpp) |
||||||
|
in opencv/samples/cpp folder. |
||||||
@ -0,0 +1,180 @@ |
|||||||
|
Operations with images {#tutorial_ug_mat} |
||||||
|
====================== |
||||||
|
|
||||||
|
Input/Output |
||||||
|
------------ |
||||||
|
|
||||||
|
### Images |
||||||
|
|
||||||
|
Load an image from a file: |
||||||
|
@code{.cpp} |
||||||
|
Mat img = imread(filename) |
||||||
|
@endcode |
||||||
|
|
||||||
|
If you read a jpg file, a 3 channel image is created by default. If you need a grayscale image, use: |
||||||
|
|
||||||
|
@code{.cpp} |
||||||
|
Mat img = imread(filename, 0); |
||||||
|
@endcode |
||||||
|
|
||||||
|
@note format of the file is determined by its content (first few bytes) Save an image to a file: |
||||||
|
|
||||||
|
@code{.cpp} |
||||||
|
imwrite(filename, img); |
||||||
|
@endcode |
||||||
|
|
||||||
|
@note format of the file is determined by its extension. |
||||||
|
|
||||||
|
@note use imdecode and imencode to read and write image from/to memory rather than a file. |
||||||
|
|
||||||
|
XML/YAML |
||||||
|
-------- |
||||||
|
|
||||||
|
TBD |
||||||
|
|
||||||
|
Basic operations with images |
||||||
|
---------------------------- |
||||||
|
|
||||||
|
### Accessing pixel intensity values |
||||||
|
|
||||||
|
In order to get pixel intensity value, you have to know the type of an image and the number of |
||||||
|
channels. Here is an example for a single channel grey scale image (type 8UC1) and pixel coordinates |
||||||
|
x and y: |
||||||
|
@code{.cpp} |
||||||
|
Scalar intensity = img.at<uchar>(y, x); |
||||||
|
@endcode |
||||||
|
intensity.val[0] contains a value from 0 to 255. Note the ordering of x and y. Since in OpenCV |
||||||
|
images are represented by the same structure as matrices, we use the same convention for both |
||||||
|
cases - the 0-based row index (or y-coordinate) goes first and the 0-based column index (or |
||||||
|
x-coordinate) follows it. Alternatively, you can use the following notation: |
||||||
|
@code{.cpp} |
||||||
|
Scalar intensity = img.at<uchar>(Point(x, y)); |
||||||
|
@endcode |
||||||
|
Now let us consider a 3 channel image with BGR color ordering (the default format returned by |
||||||
|
imread): |
||||||
|
@code{.cpp} |
||||||
|
Vec3b intensity = img.at<Vec3b>(y, x); |
||||||
|
uchar blue = intensity.val[0]; |
||||||
|
uchar green = intensity.val[1]; |
||||||
|
uchar red = intensity.val[2]; |
||||||
|
@endcode |
||||||
|
You can use the same method for floating-point images (for example, you can get such an image by |
||||||
|
running Sobel on a 3 channel image): |
||||||
|
@code{.cpp} |
||||||
|
Vec3f intensity = img.at<Vec3f>(y, x); |
||||||
|
float blue = intensity.val[0]; |
||||||
|
float green = intensity.val[1]; |
||||||
|
float red = intensity.val[2]; |
||||||
|
@endcode |
||||||
|
The same method can be used to change pixel intensities: |
||||||
|
@code{.cpp} |
||||||
|
img.at<uchar>(y, x) = 128; |
||||||
|
@endcode |
||||||
|
There are functions in OpenCV, especially from calib3d module, such as projectPoints, that take an |
||||||
|
array of 2D or 3D points in the form of Mat. Matrix should contain exactly one column, each row |
||||||
|
corresponds to a point, matrix type should be 32FC2 or 32FC3 correspondingly. Such a matrix can be |
||||||
|
easily constructed from `std::vector`: |
||||||
|
@code{.cpp} |
||||||
|
vector<Point2f> points; |
||||||
|
//... fill the array |
||||||
|
Mat pointsMat = Mat(points); |
||||||
|
@endcode |
||||||
|
One can access a point in this matrix using the same method Mat::at : |
||||||
|
@code{.cpp} |
||||||
|
Point2f point = pointsMat.at<Point2f>(i, 0); |
||||||
|
@endcode |
||||||
|
|
||||||
|
### Memory management and reference counting |
||||||
|
|
||||||
|
Mat is a structure that keeps matrix/image characteristics (rows and columns number, data type etc) |
||||||
|
and a pointer to data. So nothing prevents us from having several instances of Mat corresponding to |
||||||
|
the same data. A Mat keeps a reference count that tells if data has to be deallocated when a |
||||||
|
particular instance of Mat is destroyed. Here is an example of creating two matrices without copying |
||||||
|
data: |
||||||
|
@code{.cpp} |
||||||
|
std::vector<Point3f> points; |
||||||
|
// .. fill the array |
||||||
|
Mat pointsMat = Mat(points).reshape(1); |
||||||
|
@endcode |
||||||
|
As a result we get a 32FC1 matrix with 3 columns instead of 32FC3 matrix with 1 column. pointsMat |
||||||
|
uses data from points and will not deallocate the memory when destroyed. In this particular |
||||||
|
instance, however, developer has to make sure that lifetime of points is longer than of pointsMat. |
||||||
|
If we need to copy the data, this is done using, for example, cv::Mat::copyTo or cv::Mat::clone: |
||||||
|
@code{.cpp} |
||||||
|
Mat img = imread("image.jpg"); |
||||||
|
Mat img1 = img.clone(); |
||||||
|
@endcode |
||||||
|
To the contrary with C API where an output image had to be created by developer, an empty output Mat |
||||||
|
can be supplied to each function. Each implementation calls Mat::create for a destination matrix. |
||||||
|
This method allocates data for a matrix if it is empty. If it is not empty and has the correct size |
||||||
|
and type, the method does nothing. If, however, size or type are different from input arguments, the |
||||||
|
data is deallocated (and lost) and a new data is allocated. For example: |
||||||
|
@code{.cpp} |
||||||
|
Mat img = imread("image.jpg"); |
||||||
|
Mat sobelx; |
||||||
|
Sobel(img, sobelx, CV_32F, 1, 0); |
||||||
|
@endcode |
||||||
|
|
||||||
|
### Primitive operations |
||||||
|
|
||||||
|
There is a number of convenient operators defined on a matrix. For example, here is how we can make |
||||||
|
a black image from an existing greyscale image \`img\`: |
||||||
|
@code{.cpp} |
||||||
|
img = Scalar(0); |
||||||
|
@endcode |
||||||
|
Selecting a region of interest: |
||||||
|
@code{.cpp} |
||||||
|
Rect r(10, 10, 100, 100); |
||||||
|
Mat smallImg = img(r); |
||||||
|
@endcode |
||||||
|
A convertion from Mat to C API data structures: |
||||||
|
@code{.cpp} |
||||||
|
Mat img = imread("image.jpg"); |
||||||
|
IplImage img1 = img; |
||||||
|
CvMat m = img; |
||||||
|
@endcode |
||||||
|
|
||||||
|
Note that there is no data copying here. |
||||||
|
|
||||||
|
Conversion from color to grey scale: |
||||||
|
@code{.cpp} |
||||||
|
Mat img = imread("image.jpg"); // loading a 8UC3 image |
||||||
|
Mat grey; |
||||||
|
cvtColor(img, grey, COLOR_BGR2GRAY); |
||||||
|
@endcode |
||||||
|
Change image type from 8UC1 to 32FC1: |
||||||
|
@code{.cpp} |
||||||
|
src.convertTo(dst, CV_32F); |
||||||
|
@endcode |
||||||
|
|
||||||
|
### Visualizing images |
||||||
|
|
||||||
|
It is very useful to see intermediate results of your algorithm during development process. OpenCV |
||||||
|
provides a convenient way of visualizing images. A 8U image can be shown using: |
||||||
|
@code{.cpp} |
||||||
|
Mat img = imread("image.jpg"); |
||||||
|
|
||||||
|
namedWindow("image", WINDOW_AUTOSIZE); |
||||||
|
imshow("image", img); |
||||||
|
waitKey(); |
||||||
|
@endcode |
||||||
|
|
||||||
|
A call to waitKey() starts a message passing cycle that waits for a key stroke in the "image" |
||||||
|
window. A 32F image needs to be converted to 8U type. For example: |
||||||
|
@code{.cpp} |
||||||
|
Mat img = imread("image.jpg"); |
||||||
|
Mat grey; |
||||||
|
cvtColor(img, grey, COLOR_BGR2GRAY); |
||||||
|
|
||||||
|
Mat sobelx; |
||||||
|
Sobel(grey, sobelx, CV_32F, 1, 0); |
||||||
|
|
||||||
|
double minVal, maxVal; |
||||||
|
minMaxLoc(sobelx, &minVal, &maxVal); //find minimum and maximum intensities |
||||||
|
Mat draw; |
||||||
|
sobelx.convertTo(draw, CV_8U, 255.0/(maxVal - minVal), -minVal * 255.0/(maxVal - minVal)); |
||||||
|
|
||||||
|
namedWindow("image", WINDOW_AUTOSIZE); |
||||||
|
imshow("image", draw); |
||||||
|
waitKey(); |
||||||
|
@endcode |
||||||
@ -0,0 +1,323 @@ |
|||||||
|
Cascade Classifier Training {#tutorial_ug_traincascade} |
||||||
|
=========================== |
||||||
|
|
||||||
|
Introduction |
||||||
|
------------ |
||||||
|
|
||||||
|
The work with a cascade classifier inlcudes two major stages: training and detection. Detection |
||||||
|
stage is described in a documentation of objdetect module of general OpenCV documentation. |
||||||
|
Documentation gives some basic information about cascade classifier. Current guide is describing how |
||||||
|
to train a cascade classifier: preparation of a training data and running the training application. |
||||||
|
|
||||||
|
### Important notes |
||||||
|
|
||||||
|
There are two applications in OpenCV to train cascade classifier: opencv_haartraining and |
||||||
|
opencv_traincascade. opencv_traincascade is a newer version, written in C++ in accordance to |
||||||
|
OpenCV 2.x API. But the main difference between this two applications is that opencv_traincascade |
||||||
|
supports both Haar @cite Viola01 and @cite Liao2007 (Local Binary Patterns) features. LBP features |
||||||
|
are integer in contrast to Haar features, so both training and detection with LBP are several times |
||||||
|
faster then with Haar features. Regarding the LBP and Haar detection quality, it depends on |
||||||
|
training: the quality of training dataset first of all and training parameters too. It's possible to |
||||||
|
train a LBP-based classifier that will provide almost the same quality as Haar-based one. |
||||||
|
|
||||||
|
opencv_traincascade and opencv_haartraining store the trained classifier in different file |
||||||
|
formats. Note, the newer cascade detection interface (see CascadeClassifier class in objdetect |
||||||
|
module) support both formats. opencv_traincascade can save (export) a trained cascade in the older |
||||||
|
format. But opencv_traincascade and opencv_haartraining can not load (import) a classifier in |
||||||
|
another format for the futher training after interruption. |
||||||
|
|
||||||
|
Note that opencv_traincascade application can use TBB for multi-threading. To use it in multicore |
||||||
|
mode OpenCV must be built with TBB. |
||||||
|
|
||||||
|
Also there are some auxilary utilities related to the training. |
||||||
|
|
||||||
|
- opencv_createsamples is used to prepare a training dataset of positive and test samples. |
||||||
|
opencv_createsamples produces dataset of positive samples in a format that is supported by |
||||||
|
both opencv_haartraining and opencv_traincascade applications. The output is a file |
||||||
|
with \*.vec extension, it is a binary format which contains images. |
||||||
|
- opencv_performance may be used to evaluate the quality of classifiers, but for trained by |
||||||
|
opencv_haartraining only. It takes a collection of marked up images, runs the classifier and |
||||||
|
reports the performance, i.e. number of found objects, number of missed objects, number of |
||||||
|
false alarms and other information. |
||||||
|
|
||||||
|
Since opencv_haartraining is an obsolete application, only opencv_traincascade will be described |
||||||
|
futher. opencv_createsamples utility is needed to prepare a training data for opencv_traincascade, |
||||||
|
so it will be described too. |
||||||
|
|
||||||
|
Training data preparation |
||||||
|
------------------------- |
||||||
|
|
||||||
|
For training we need a set of samples. There are two types of samples: negative and positive. |
||||||
|
Negative samples correspond to non-object images. Positive samples correspond to images with |
||||||
|
detected objects. Set of negative samples must be prepared manually, whereas set of positive samples |
||||||
|
is created using opencv_createsamples utility. |
||||||
|
|
||||||
|
### Negative Samples |
||||||
|
|
||||||
|
Negative samples are taken from arbitrary images. These images must not contain detected objects. |
||||||
|
Negative samples are enumerated in a special file. It is a text file in which each line contains an |
||||||
|
image filename (relative to the directory of the description file) of negative sample image. This |
||||||
|
file must be created manually. Note that negative samples and sample images are also called |
||||||
|
background samples or background samples images, and are used interchangeably in this document. |
||||||
|
Described images may be of different sizes. But each image should be (but not nessesarily) larger |
||||||
|
then a training window size, because these images are used to subsample negative image to the |
||||||
|
training size. |
||||||
|
|
||||||
|
An example of description file: |
||||||
|
|
||||||
|
Directory structure: |
||||||
|
@code{.text} |
||||||
|
/img |
||||||
|
img1.jpg |
||||||
|
img2.jpg |
||||||
|
bg.txt |
||||||
|
@endcode |
||||||
|
File bg.txt: |
||||||
|
@code{.text} |
||||||
|
img/img1.jpg |
||||||
|
img/img2.jpg |
||||||
|
@endcode |
||||||
|
### Positive Samples |
||||||
|
|
||||||
|
Positive samples are created by opencv_createsamples utility. They may be created from a single |
||||||
|
image with object or from a collection of previously marked up images. |
||||||
|
|
||||||
|
Please note that you need a large dataset of positive samples before you give it to the mentioned |
||||||
|
utility, because it only applies perspective transformation. For example you may need only one |
||||||
|
positive sample for absolutely rigid object like an OpenCV logo, but you definetely need hundreds |
||||||
|
and even thousands of positive samples for faces. In the case of faces you should consider all the |
||||||
|
race and age groups, emotions and perhaps beard styles. |
||||||
|
|
||||||
|
So, a single object image may contain a company logo. Then a large set of positive samples is |
||||||
|
created from the given object image by random rotating, changing the logo intensity as well as |
||||||
|
placing the logo on arbitrary background. The amount and range of randomness can be controlled by |
||||||
|
command line arguments of opencv_createsamples utility. |
||||||
|
|
||||||
|
Command line arguments: |
||||||
|
|
||||||
|
- -vec \<vec_file_name\> |
||||||
|
|
||||||
|
Name of the output file containing the positive samples for training. |
||||||
|
|
||||||
|
- -img \<image_file_name\> |
||||||
|
|
||||||
|
Source object image (e.g., a company logo). |
||||||
|
|
||||||
|
- -bg \<background_file_name\> |
||||||
|
|
||||||
|
Background description file; contains a list of images which are used as a background for |
||||||
|
randomly distorted versions of the object. |
||||||
|
|
||||||
|
- -num \<number_of_samples\> |
||||||
|
|
||||||
|
Number of positive samples to generate. |
||||||
|
|
||||||
|
- -bgcolor \<background_color\> |
||||||
|
|
||||||
|
Background color (currently grayscale images are assumed); the background color denotes the |
||||||
|
transparent color. Since there might be compression artifacts, the amount of color tolerance |
||||||
|
can be specified by -bgthresh. All pixels withing bgcolor-bgthresh and bgcolor+bgthresh range |
||||||
|
are interpreted as transparent. |
||||||
|
|
||||||
|
- -bgthresh \<background_color_threshold\> |
||||||
|
- -inv |
||||||
|
|
||||||
|
If specified, colors will be inverted. |
||||||
|
|
||||||
|
- -randinv |
||||||
|
|
||||||
|
If specified, colors will be inverted randomly. |
||||||
|
|
||||||
|
- -maxidev \<max_intensity_deviation\> |
||||||
|
|
||||||
|
Maximal intensity deviation of pixels in foreground samples. |
||||||
|
|
||||||
|
- -maxxangle \<max_x_rotation_angle\> |
||||||
|
- -maxyangle \<max_y_rotation_angle\> |
||||||
|
- -maxzangle \<max_z_rotation_angle\> |
||||||
|
|
||||||
|
Maximum rotation angles must be given in radians. |
||||||
|
|
||||||
|
- -show |
||||||
|
|
||||||
|
Useful debugging option. If specified, each sample will be shown. Pressing Esc will continue |
||||||
|
the samples creation process without. |
||||||
|
|
||||||
|
- -w \<sample_width\> |
||||||
|
|
||||||
|
Width (in pixels) of the output samples. |
||||||
|
|
||||||
|
- -h \<sample_height\> |
||||||
|
|
||||||
|
Height (in pixels) of the output samples. |
||||||
|
|
||||||
|
For following procedure is used to create a sample object instance: The source image is rotated |
||||||
|
randomly around all three axes. The chosen angle is limited my -max?angle. Then pixels having the |
||||||
|
intensity from [bg_color-bg_color_threshold; bg_color+bg_color_threshold] range are |
||||||
|
interpreted as transparent. White noise is added to the intensities of the foreground. If the -inv |
||||||
|
key is specified then foreground pixel intensities are inverted. If -randinv key is specified then |
||||||
|
algorithm randomly selects whether inversion should be applied to this sample. Finally, the obtained |
||||||
|
image is placed onto an arbitrary background from the background description file, resized to the |
||||||
|
desired size specified by -w and -h and stored to the vec-file, specified by the -vec command line |
||||||
|
option. |
||||||
|
|
||||||
|
Positive samples also may be obtained from a collection of previously marked up images. This |
||||||
|
collection is described by a text file similar to background description file. Each line of this |
||||||
|
file corresponds to an image. The first element of the line is the filename. It is followed by the |
||||||
|
number of object instances. The following numbers are the coordinates of objects bounding rectangles |
||||||
|
(x, y, width, height). |
||||||
|
|
||||||
|
An example of description file: |
||||||
|
|
||||||
|
Directory structure: |
||||||
|
@code{.text} |
||||||
|
/img |
||||||
|
img1.jpg |
||||||
|
img2.jpg |
||||||
|
info.dat |
||||||
|
@endcode |
||||||
|
File info.dat: |
||||||
|
@code{.text} |
||||||
|
img/img1.jpg 1 140 100 45 45 |
||||||
|
img/img2.jpg 2 100 200 50 50 50 30 25 25 |
||||||
|
@endcode |
||||||
|
Image img1.jpg contains single object instance with the following coordinates of bounding rectangle: |
||||||
|
(140, 100, 45, 45). Image img2.jpg contains two object instances. |
||||||
|
|
||||||
|
In order to create positive samples from such collection, -info argument should be specified instead |
||||||
|
of \`-img\`: |
||||||
|
|
||||||
|
- -info \<collection_file_name\> |
||||||
|
|
||||||
|
Description file of marked up images collection. |
||||||
|
|
||||||
|
The scheme of samples creation in this case is as follows. The object instances are taken from |
||||||
|
images. Then they are resized to target samples size and stored in output vec-file. No distortion is |
||||||
|
applied, so the only affecting arguments are -w, -h, -show and -num. |
||||||
|
|
||||||
|
opencv_createsamples utility may be used for examining samples stored in positive samples file. In |
||||||
|
order to do this only -vec, -w and -h parameters should be specified. |
||||||
|
|
||||||
|
Note that for training, it does not matter how vec-files with positive samples are generated. But |
||||||
|
opencv_createsamples utility is the only one way to collect/create a vector file of positive |
||||||
|
samples, provided by OpenCV. |
||||||
|
|
||||||
|
Example of vec-file is available here opencv/data/vec_files/trainingfaces_24-24.vec. It can be |
||||||
|
used to train a face detector with the following window size: -w 24 -h 24. |
||||||
|
|
||||||
|
Cascade Training |
||||||
|
---------------- |
||||||
|
|
||||||
|
The next step is the training of classifier. As mentioned above opencv_traincascade or |
||||||
|
opencv_haartraining may be used to train a cascade classifier, but only the newer |
||||||
|
opencv_traincascade will be described futher. |
||||||
|
|
||||||
|
Command line arguments of opencv_traincascade application grouped by purposes: |
||||||
|
|
||||||
|
-# Common arguments: |
||||||
|
|
||||||
|
- -data \<cascade_dir_name\> |
||||||
|
|
||||||
|
Where the trained classifier should be stored. |
||||||
|
|
||||||
|
- -vec \<vec_file_name\> |
||||||
|
|
||||||
|
vec-file with positive samples (created by opencv_createsamples utility). |
||||||
|
|
||||||
|
- -bg \<background_file_name\> |
||||||
|
|
||||||
|
Background description file. |
||||||
|
|
||||||
|
- -numPos \<number_of_positive_samples\> |
||||||
|
- -numNeg \<number_of_negative_samples\> |
||||||
|
|
||||||
|
Number of positive/negative samples used in training for every classifier stage. |
||||||
|
|
||||||
|
- -numStages \<number_of_stages\> |
||||||
|
|
||||||
|
Number of cascade stages to be trained. |
||||||
|
|
||||||
|
- -precalcValBufSize \<precalculated_vals_buffer_size_in_Mb\> |
||||||
|
|
||||||
|
Size of buffer for precalculated feature values (in Mb). |
||||||
|
|
||||||
|
- -precalcIdxBufSize \<precalculated_idxs_buffer_size_in_Mb\> |
||||||
|
|
||||||
|
Size of buffer for precalculated feature indices (in Mb). The more memory you have the |
||||||
|
faster the training process. |
||||||
|
|
||||||
|
- -baseFormatSave |
||||||
|
|
||||||
|
This argument is actual in case of Haar-like features. If it is specified, the cascade will |
||||||
|
be saved in the old format. |
||||||
|
|
||||||
|
- -numThreads \<max_number_of_threads\> |
||||||
|
|
||||||
|
Maximum number of threads to use during training. Notice that the actual number of used |
||||||
|
threads may be lower, depending on your machine and compilation options. |
||||||
|
|
||||||
|
-# Cascade parameters: |
||||||
|
|
||||||
|
- -stageType \<BOOST(default)\> |
||||||
|
|
||||||
|
Type of stages. Only boosted classifier are supported as a stage type at the moment. |
||||||
|
|
||||||
|
- -featureType\<{HAAR(default), LBP}\> |
||||||
|
|
||||||
|
Type of features: HAAR - Haar-like features, LBP - local binary patterns. |
||||||
|
|
||||||
|
- -w \<sampleWidth\> |
||||||
|
- -h \<sampleHeight\> |
||||||
|
|
||||||
|
Size of training samples (in pixels). Must have exactly the same values as used during |
||||||
|
training samples creation (opencv_createsamples utility). |
||||||
|
|
||||||
|
-# Boosted classifer parameters: |
||||||
|
|
||||||
|
- -bt \<{DAB, RAB, LB, GAB(default)}\> |
||||||
|
|
||||||
|
Type of boosted classifiers: DAB - Discrete AdaBoost, RAB - Real AdaBoost, LB - LogitBoost, |
||||||
|
GAB - Gentle AdaBoost. |
||||||
|
|
||||||
|
- -minHitRate \<min_hit_rate\> |
||||||
|
|
||||||
|
Minimal desired hit rate for each stage of the classifier. Overall hit rate may be estimated |
||||||
|
as (min_hit_rate\^number_of_stages). |
||||||
|
|
||||||
|
- -maxFalseAlarmRate \<max_false_alarm_rate\> |
||||||
|
|
||||||
|
Maximal desired false alarm rate for each stage of the classifier. Overall false alarm rate |
||||||
|
may be estimated as (max_false_alarm_rate\^number_of_stages). |
||||||
|
|
||||||
|
- -weightTrimRate \<weight_trim_rate\> |
||||||
|
|
||||||
|
Specifies whether trimming should be used and its weight. A decent choice is 0.95. |
||||||
|
|
||||||
|
- -maxDepth \<max_depth_of_weak_tree\> |
||||||
|
|
||||||
|
Maximal depth of a weak tree. A decent choice is 1, that is case of stumps. |
||||||
|
|
||||||
|
- -maxWeakCount \<max_weak_tree_count\> |
||||||
|
|
||||||
|
Maximal count of weak trees for every cascade stage. The boosted classifier (stage) will |
||||||
|
have so many weak trees (\<=maxWeakCount), as needed to achieve the |
||||||
|
given -maxFalseAlarmRate. |
||||||
|
|
||||||
|
-# Haar-like feature parameters: |
||||||
|
|
||||||
|
- -mode \<BASIC (default) | CORE | ALL\> |
||||||
|
|
||||||
|
Selects the type of Haar features set used in training. BASIC use only upright features, |
||||||
|
while ALL uses the full set of upright and 45 degree rotated feature set. See @cite Lienhart02 |
||||||
|
for more details. |
||||||
|
|
||||||
|
-# Local Binary Patterns parameters: |
||||||
|
|
||||||
|
Local Binary Patterns don't have parameters. |
||||||
|
|
||||||
|
After the opencv_traincascade application has finished its work, the trained cascade will be saved |
||||||
|
in cascade.xml file in the folder, which was passed as -data parameter. Other files in this folder |
||||||
|
are created for the case of interrupted training, so you may delete them after completion of |
||||||
|
training. |
||||||
|
|
||||||
|
Training is finished and you can test you cascade classifier! |
||||||
@ -0,0 +1,8 @@ |
|||||||
|
OpenCV User Guide {#tutorial_user_guide} |
||||||
|
================= |
||||||
|
|
||||||
|
- @subpage tutorial_ug_mat |
||||||
|
- @subpage tutorial_ug_features2d |
||||||
|
- @subpage tutorial_ug_highgui |
||||||
|
- @subpage tutorial_ug_traincascade |
||||||
|
- @subpage tutorial_ug_intelperc |
||||||
Loading…
Reference in new issue