Template "traits" class for other OpenCV primitive data types ::
Template "trait" class for other OpenCV primitive data types ::
template<typename _Tp> class DataType
{
// value_type is always a synonym for _Tp.
// value_type is always a synonym to _Tp.
typedef _Tp value_type;
// intermediate type used for operations on _Tp.
// it is int for uchar, signed char, unsigned short, signed short and int,
// it is int for uchar, signed char, unsigned short, signed short, and int,
// float for float, double for double, ...
typedef <...> work_type;
// in the case of multi-channel data it is the data type of each channel
// in the case of multi-channel data, it is the data type of each channel
typedef <...> channel_type;
enum
{
@ -34,11 +34,11 @@ Template "traits" class for other OpenCV primitive data types ::
};
};
The template class ``DataType`` is descriptive class for OpenCV primitive data types and other types that comply with the following definition. A primitive OpenCV data type is one of ``unsigned char``, ``bool``, ``signed char``, ``unsigned short``, ``signed short``, ``int``, ``float``, ``double`` or a tuple of values of one of these types, where all the values in the tuple have the same type. Any primitive type from the list can be defined by an identifier in a form ``CV_<bit-depth>{U|S|F}C<number_of_channels>``, for example,``uchar`` ~ ``CV_8UC1``, 3-element floating-point tuple ~ ``CV_32FC3`` etc. A universal OpenCV structure, which is able to store a single instance of such primitive data type is
:ref:`Vec`. Multiple instances of such a type can be stored to a ``std::vector``,``Mat``,``Mat_``,``SparseMat``,``SparseMat_`` or any other container that is able to store
The template class ``DataType`` is a descriptive class for OpenCV primitive data types and other types that comply with the following definition. A primitive OpenCV data type is one of ``unsigned char``, ``bool``, ``signed char``, ``unsigned short``, ``signed short``, ``int``, ``float``, ``double`` or a tuple of values of one of these types, where all the values in the tuple have the same type. Any primitive type from the list can be defined by an identifier in the form ``CV_<bit-depth>{U|S|F}C<number_of_channels>``, for example:``uchar`` ~ ``CV_8UC1``, 3-element floating-point tuple ~ ``CV_32FC3``, and so on. A universal OpenCV structure, which is able to store a single instance of such a primitive data type, is
:ref:`Vec`. Multiple instances of such a type can be stored in a ``std::vector``,``Mat``,``Mat_``,``SparseMat``,``SparseMat_``, or any other container that is able to store
:ref:`Vec` instances.
The class ``DataType``is basically used to provide some description of such primitive data types without adding any fields or methods to the corresponding classes (and it is actually impossible to add anything to primitive C/C++ data types). This technique is known in C++ as class traits. It's not ``DataType`` itself that is used, but its specialized versions, such as: ::
The ``DataType``class is basically used to provide a description of such primitive data types without adding any fields or methods to the corresponding classes (and it is actually impossible to add anything to primitive C/C++ data types). This technique is known in C++ as class traits. It is not ``DataType`` itself that is used, but its specialized versions, such as: ::
template<> class DataType<uchar>
{
@ -60,17 +60,17 @@ The class ``DataType`` is basically used to provide some description of such pri
};
...
The main purpose of the classes is to convert compile-time type information to OpenCV-compatible data type identifier, for example: ::
The main purpose of this class is to convert compilation-time type information to an OpenCV-compatible data type identifier, for example: ::
// allocates 30x40 floating-point matrix
// allocates a 30x40 floating-point matrix
Mat A(30, 40, DataType<float>::type);
Mat B = Mat_<std::complex<double> >(3, 3);
// the statement below will print 6, 2 /* i.e. depth == CV_64F, channels == 2 */
// the statement below will print 6, 2 /*, that is depth == CV_64F, channels == 2 */
that is, such traits are used to tell OpenCV which data type you are working with, even if such a type is not native to OpenCV (the matrix ``B`` intialization above compiles because OpenCV defines the proper specialized template class ``DataType<complex<_Tp> >`` ). Also, this mechanism is useful (and used in OpenCV this way) for generic algorithms implementations.
So, such traits are used to tell OpenCV which data type you are working with, even if such a type is not native to OpenCV (the matrix ``B`` intialization above is compiled because OpenCV defines the proper specialized template class ``DataType<complex<_Tp> >`` ). Also, this mechanism is useful (and used in OpenCV this way) for generic algorithms implementations.
Point\_
-------
@ -108,7 +108,7 @@ Template class for 2D points ::
The class represents a 2D point, specified by its coordinates
:math:`x` and
:math:`y` .
Instance of the class is interchangeable with C structures ``CvPoint`` and ``CvPoint2D32f`` . There is also cast operator to convert point coordinates to the specified type. The conversion from floating-point coordinates to integer coordinates is done by rounding; in general case the conversion uses
Instance of the class is interchangeable with C structures,``CvPoint`` and ``CvPoint2D32f`` . There is also a cast operator to convert point coordinates to the specified type. The conversion from floating-point coordinates to integer coordinates is done by rounding. Commonly, the conversion uses this
operation on each of the coordinates. Besides the class members listed in the declaration above, the following operations on points are implemented: ::
pt1 = pt2 + pt3;
@ -122,7 +122,7 @@ operation on each of the coordinates. Besides the class members listed in the de
pt1 == pt2;
pt1 != pt2;
For user convenience, the following type aliases are defined: ::
For your convenience, the following type aliases are defined: ::
typedef Point_<int> Point2i;
typedef Point2i Point;
@ -164,10 +164,10 @@ Template class for 3D points ::
};
The class represents a 3D point, specified by its coordinates
The class represents a 3D point specified by its coordinates
:math:`x`,:math:`y` and
:math:`z` .
Instance of the class is interchangeable with C structure ``CvPoint2D32f`` . Similarly to ``Point_`` , the 3D points' coordinates can be converted to another type, and the vector arithmetic and comparison operations are also supported.
Instance of the class is interchangeable with the C structure ``CvPoint2D32f`` . Similarly to ``Point_`` , the 3D points' coordinates can be converted to another type. The vector arithmetic and comparison operations are also supported.
The following type aliases are available: ::
@ -179,7 +179,7 @@ The following type aliases are available: ::
Size\_
------
Template class for specfying image or rectangle size. ::
Template class for specfying an image or rectangle size. ::
template<typename _Tp> class Size_
{
@ -205,7 +205,7 @@ Template class for specfying image or rectangle size. ::
};
The class ``Size_`` is similar to ``Point_``, except that the two members are called ``width`` and ``height`` instead of ``x`` and ``y`` . The structure can be converted to and from the old OpenCV structures
The class ``Size_`` is similar to ``Point_`` except that the two members are called ``width`` and ``height`` instead of ``x`` and ``y`` . The structure can be converted to and from the old OpenCV structures
``CvSize`` and ``CvSize2D32f`` . The same set of arithmetic and comparison operations as for ``Point_`` is available.
OpenCV defines the following type aliases: ::
@ -259,7 +259,7 @@ Template class for 2D rectangles ::
The rectangle is described by the coordinates of the top-left corner (which is the default interpretation of ``Rect_::x`` and ``Rect_::y`` in OpenCV; though, in your algorithms you may count ``x`` and ``y`` from the bottom-left corner), the rectangle width and height.
Another assumption OpenCV usually makes is that the top and left boundary of the rectangle are inclusive, while the right and bottom boundaries are not, for example, the method ``Rect_::contains`` returns true if
Another assumption OpenCV usually makes is that the top and left boundary of the rectangle are inclusive, while the right and bottom boundaries are not. For example, the method ``Rect_::contains`` returns ``true`` if
..math::
@ -279,10 +279,10 @@ ROI in OpenCV (where ROI is specified by ``Rect_<int>`` ) is implemented as: ::
In addition to the class members, the following operations on rectangles are implemented:
*
:math:`\texttt{rect} = \texttt{rect} \pm \texttt{point}` (shifting rectangle by a certain offset)
:math:`\texttt{rect} = \texttt{rect} \pm \texttt{point}` (shifting a rectangle by a certain offset)
*
:math:`\texttt{rect} = \texttt{rect} \pm \texttt{size}` (expanding or shrinking rectangle by a certain amount)
:math:`\texttt{rect} = \texttt{rect} \pm \texttt{size}` (expanding or shrinking a rectangle by a certain amount)
* ``rect += point, rect -= point, rect += size, rect -= size`` (augmenting operations)
@ -294,7 +294,7 @@ In addition to the class members, the following operations on rectangles are imp
* ``rect == rect1, rect != rect1`` (rectangle comparison)
Example. Here is how the partial ordering on rectangles can be established (rect1
This is an example how the partial ordering on rectangles can be established (rect1
:math:`\subseteq` rect2):::
template<typename _Tp> inline bool
@ -304,7 +304,7 @@ Example. Here is how the partial ordering on rectangles can be established (rect
}
For user convenience, the following type alias is available: ::
For your convenience, the following type alias is available: ::
typedef Rect_<int> Rect;
@ -314,7 +314,7 @@ For user convenience, the following type alias is available: ::
The class ``Vec`` is commonly used to describe pixel types of multi-channel arrays, see ``Mat_`` description.
The ``Vec``class is commonly used to describe pixel types of multi-channel arrays. See ``Mat_`` for details.
.._Scalar:
Scalar\_
--------
4-element vector ::
Template class for a 4-element vector ::
template<typename _Tp> class Scalar_ : public Vec<_Tp, 4>
{
@ -498,7 +498,7 @@ Scalar\_
typedef Scalar_<double> Scalar;
The template class ``Scalar_`` and it's double-precision instantiation ``Scalar`` represent 4-element vector. Being derived from ``Vec<_Tp, 4>`` , they can be used as typical 4-element vectors, but in addition they can be converted to/from ``CvScalar`` . The type ``Scalar`` is widely used in OpenCV for passing pixel values and it is a drop-in replacement for
The template class ``Scalar_`` and its double-precision instantiation ``Scalar`` represent a 4-element vector. Being derived from ``Vec<_Tp, 4>`` , they can be used as typical 4-element vectors, but in addition they can be converted to/from ``CvScalar`` . The type ``Scalar`` is widely used in OpenCV for passing pixel values and it is a drop-in replacement for
``CvScalar`` that was used for the same purpose in the earlier versions of OpenCV.
.._Range:
@ -506,7 +506,7 @@ The template class ``Scalar_`` and it's double-precision instantiation ``Scalar`
Range
-----
Specifies a continuous subsequence (a.k.a. slice) of a sequence. ::
Template class specifying a continuous subsequence (a.k.a. slice) of a sequence. ::
class Range
{
@ -524,10 +524,10 @@ Specifies a continuous subsequence (a.k.a. slice) of a sequence. ::
The class is used to specify a row or column span in a matrix (
:ref:`Mat` ), and for many other purposes. ``Range(a,b)`` is basically the same as ``a:b`` in Matlab or ``a..b`` in Python. As in Python, ``start`` is inclusive left boundary of the range, and ``end`` is exclusive right boundary of the range. Such a half-opened interval is usually denoted as
:ref:`Mat` ) and for many other purposes. ``Range(a,b)`` is basically the same as ``a:b`` in Matlab or ``a..b`` in Python. As in Python, ``start`` is an inclusive left boundary of the range and ``end`` is an exclusive right boundary of the range. Such a half-opened interval is usually denoted as
:math:`[start,end)` .
The static method ``Range::all()`` returns some special variable that means "the whole sequence" or "the whole range", just like " ``:`` " in Matlab or " ``...`` " in Python. All the methods and functions in OpenCV that take ``Range`` support this special ``Range::all()`` value, but of course, in the case of your own custom processing you will probably have to check and handle it explicitly: ::
The static method ``Range::all()`` returns a special variable that means "the whole sequence" or "the whole range", just like " ``:`` " in Matlab or " ``...`` " in Python. All the methods and functions in OpenCV that take ``Range`` support this special ``Range::all()`` value. But, of course, in case of your own custom processing, you will probably have to check and handle it explicitly: ::
void my_function(..., const Range& r, ....)
{
@ -545,7 +545,7 @@ The static method ``Range::all()`` returns some special variable that means "the
Ptr
---
A template class for smart reference-counting pointers ::
Template class for smart reference-counting pointers ::
template<typename _Tp> class Ptr
{
@ -589,27 +589,27 @@ A template class for smart reference-counting pointers ::
};
The class ``Ptr<_Tp>`` is a template class that wraps pointers of the corresponding type. It is similar to ``shared_ptr`` that is a part of Boost library (
The ``Ptr<_Tp>`` class is a template class that wraps pointers of the corresponding type. It is similar to ``shared_ptr`` that is part of the Boost library (
) and also a part of the `C++0x <http://en.wikipedia.org/wiki/C++0x>`_
) and also part of the `C++0x <http://en.wikipedia.org/wiki/C++0x>`_
standard.
By using this class you can get the following capabilities:
*
default constructor, copy constructor and assignment operator for an arbitrary C++ class or a C structure. For some objects, like files, windows, mutexes, sockets etc, copy constructor or assignment operator are difficult to define. For some other objects, like complex classifiers in OpenCV, copy constructors are absent and not easy to implement. Finally, some of complex OpenCV and your own data structures may have been written in C. However, copy constructors and default constructors can simplify programming a lot; besides, they are often required (e.g. by STL containers). By wrapping a pointer to such a complex object ``TObj`` to ``Ptr<TObj>`` you will automatically get all of the necessary constructors and the assignment operator.
Default constructor, copy constructor, and assignment operator for an arbitrary C++ class or a C structure. For some objects, like files, windows, mutexes, sockets, and others, copy constructor or assignment operator are difficult to define. For some other objects, like complex classifiers in OpenCV, copy constructors are absent and not easy to implement. Finally, some of complex OpenCV and your own data structures may have been written in C. However, copy constructors and default constructors can simplify programming a lot; besides, they are often required (for example, by STL containers). By wrapping a pointer to such a complex object ``TObj`` to ``Ptr<TObj>``, you will automatically get all of the necessary constructors and the assignment operator.
*
all the above-mentioned operations running very fast, regardless of the data size, i.e. as "O(1)" operations. Indeed, while some structures, like ``std::vector`` provide a copy constructor and an assignment operator, the operations may take considerable time if the data structures are big. But if the structures are put into ``Ptr<>`` , the overhead becomes small and independent of the data size.
All the above-mentioned operations running very fast, regardless of the data size, that is like "O(1)" operations. Indeed, while some structures, like ``std::vector``, provide a copy constructor and an assignment operator, the operations may take a considerable amount of time if the data structures are big. But if the structures are put into ``Ptr<>`` , the overhead becomes small and independent of the data size.
*
automatic destruction, even for C structures. See the example below with ``FILE*`` .
Automatic destruction, even for C structures. See the example below with ``FILE*`` .
*
heterogeneous collections of objects. The standard STL and most other C++ and OpenCV containers can only store objects of the same type and the same size. The classical solution to store objects of different types in the same container is to store pointers to the base class ``base_class_t*`` instead, but when you loose the automatic memory management. Again, by using ``Ptr<base_class_t>()`` instead of the raw pointers, you can solve the problem.
Heterogeneous collections of objects. The standard STL and most other C++ and OpenCV containers can only store objects of the same type and the same size. The classical solution to store objects of different types in the same container is to store pointers to the base class ``base_class_t*`` instead but when you loose the automatic memory management. Again, by using ``Ptr<base_class_t>()`` instead of the raw pointers, you can solve the problem.
The class ``Ptr``treats the wrapped object as a black box, the reference counter is allocated and managed separately. The only thing the pointer class needs to know about the object is how to deallocate it. This knowledge is incapsulated in ``Ptr::delete_obj()`` method, which is called when the reference counter becomes 0. If the object is a C++ class instance, no additional coding is needed, because the default implementation of this method calls ``delete obj;`` .
However, if the object is deallocated in a different way, then the specialized method should be created. For example, if you want to wrap ``FILE`` , the ``delete_obj`` may be implemented as following: ::
The ``Ptr``class treats the wrapped object as a black box. The reference counter is allocated and managed separately. The only thing the pointer class needs to know about the object is how to deallocate it. This knowledge is incapsulated in the ``Ptr::delete_obj()`` method that is called when the reference counter becomes 0. If the object is a C++ class instance, no additional coding is needed, because the default implementation of this method calls ``delete obj;`` .
However, if the object is deallocated in a different way, then the specialized method should be created. For example, if you want to wrap ``FILE`` , the ``delete_obj`` may be implemented as follows: ::
template<> inline void Ptr<FILE>::delete_obj()
{
@ -628,6 +628,7 @@ However, if the object is deallocated in a different way, then the specialized m
**Note**
: The reference increment/decrement operations are implemented as atomic operations, and therefore it is normally safe to use the classes in multi-threaded applications. The same is true for
:ref:`Mat` and other C++ OpenCV classes that operate on the reference counters.
@ -678,67 +679,67 @@ The class ``Mat`` represents an n-dimensional dense numerical single-channel or
Note that ``M.step[i] >= M.step[i+1]`` (in fact, ``M.step[i] >= M.step[i+1]*M.size[i+1]`` ), that is, 2-dimensional matrices are stored row-by-row, 3-dimensional matrices are stored plane-by-plane etc. ``M.step[M.dims-1]`` is minimal and always equal to the element size ``M.elemSize()`` .
Note that ``M.step[i] >= M.step[i+1]`` (in fact, ``M.step[i] >= M.step[i+1]*M.size[i+1]`` ), that is, 2-dimensional matrices are stored row-by-row, 3-dimensional matrices are stored plane-by-plane, and so on. ``M.step[M.dims-1]`` is minimal and always equal to the element size ``M.elemSize()`` .
That is, the data layout in ``Mat`` is fully compatible with ``CvMat``,``IplImage`` and ``CvMatND`` types from OpenCV 1.x, as well as with majority of dense array types from the standard toolkits and SDKs, such as Numpy (ndarray), Win32 (independent device bitmaps) etc, i.e. any other array that uses "steps", a.k.a. "strides", to compute position of a pixel. Because of such compatibility, it is possible to make a ``Mat`` header for user-allocated data and process it in-place using OpenCV functions.
So, the data layout in ``Mat`` is fully compatible with ``CvMat``,``IplImage`` and ``CvMatND`` types from OpenCV 1.x, as well as with the majority of dense array types from the standard toolkits and SDKs, such as Numpy (ndarray), Win32 (independent device bitmaps), and others, that is any other array that uses "steps", a.k.a. "strides", to compute the position of a pixel. Due to this compatibility, it is possible to make a ``Mat`` header for user-allocated data and process it in-place using OpenCV functions.
There are many different ways to create ``Mat`` object. Here are the some popular ones:
There are many different ways to create a ``Mat`` object. Here are some popular ones:
*
using``create(nrows, ncols, type)`` method or
the similar constructor ``Mat(nrows, ncols, type[, fillValue])`` constructor.
A new array of the specified size and specifed type will be allocated. ``type`` has the same meaning as in
:func:`cvCreateMat` method,
e.g.``CV_8UC1`` means 8-bit single-channel array, ``CV_32FC2`` means 2-channel (i.e. complex) floating-point array etc:
Use the``create(nrows, ncols, type)`` method or
the similar ``Mat(nrows, ncols, type[, fillValue])`` constructor.
A new array of the specified size and specifed type is allocated. ``type`` has the same meaning as in
:func:`cvCreateMat` method.
For example,``CV_8UC1`` means a 8-bit single-channel array, ``CV_32FC2`` means a 2-channel (complex) floating-point array, and so on:
::
// make 7x7 complex matrix filled with 1+3j.
// make a 7x7 complex matrix filled with 1+3j.
Mat M(7,7,CV_32FC2,Scalar(1,3));
// and now turn M to 100x60 15-channel 8-bit matrix.
// and now turn M to a 100x60 15-channel 8-bit matrix.
// The old content will be deallocated
M.create(100,60,CV_8UC(15));
..
As noted in the introduction of this chapter, ``create()``will only allocate a new array when the current array shape
or type are different from the specified.
As noted in the introduction to this chapter, ``create()`` allocates only a new array when the current array shape
or type are different from the specified ones.
*
similarly to above, you can create a multi-dimensional array:
Similarly to above, create a multi-dimensional array:
::
// create 100x100x100 8-bit array
// create a 100x100x100 8-bit array
int sz[] = {100, 100, 100};
Mat bigCube(3, sz, CV_8U, Scalar::all(0));
..
note that it is pass number of dimensions =1 to the ``Mat`` constructor, but the created array will be 2-dimensional, with the number of columns set to 1. That's why ``Mat::dims`` is always >= 2 (can also be 0 when the array is empty)
Note that it passes the number of dimensions =1 to the ``Mat`` constructor, but the created array will be 2-dimensional with the number of columns set to 1. That is why ``Mat::dims`` is always >= 2 (can also be 0 when the array is empty).
*
by using a copy constructor or assignment operator, where on the right side it can
be a array or expression, see below. Again, as noted in the introduction,
Use a copy constructor or assignment operator, where on the right side it can
be an array or expression (see below). Again, as noted in the introduction,
array assignment is O(1) operation because it only copies the header
and increases the reference counter. ``Mat::clone()`` method can be used to get a full
(a.k.a. deep) copy of the array when you need it.
*
by constructing a header for a part of another array. It can be a single row, single column,
Construct a header for a part of another array. It can be a single row, single column,
several rows, several columns, rectangular region in the array (called a minor in algebra) or
a diagonal. Such operations are also O(1), because the new header will reference the same data.
You can actually modify a part of the array using this feature, e.g.
You can actually modify a part of the array using this feature, for example:
::
@ -760,7 +761,7 @@ There are many different ways to create ``Mat`` object. Here are the some popula
..
Thanks to the additional ``datastart`` and ``dataend`` members, it is possible to compute the relative sub-array position in the main *"container"* array using ``locateROI()``:
Thanks to the additional ``datastart`` and ``dataend`` members, it is possible to compute a relative sub-array position in the main *"container"* array using ``locateROI()``:
::
@ -776,16 +777,16 @@ There are many different ways to create ``Mat`` object. Here are the some popula
..
As in the case of whole matrices, if you need a deep copy, use ``clone()`` method
As in the case of whole matrices, if you need a deep copy, use the ``clone()`` method
of the extracted sub-matrices.
*
by making a header for user-allocated-data. It can be useful for
Make a header for user-allocated data. It can be useful to do the following:
#.
processing "foreign" data using OpenCV (e.g. when you implement
a DirectShow filter or a processing module for gstreamer etc.), e.g.
Process "foreign" data using OpenCV (for example, when you implement
a DirectShow filter or a processing module for ``gstreamer``, and so on). For example:
::
@ -799,7 +800,7 @@ There are many different ways to create ``Mat`` object. Here are the some popula
..
#.
for quick initialization of small matrices and/or super-fast element access
Quickly initialize small matrices and/or get a super-fast element access.
OpenCV (Open Source Computer Vision Library: http://opencv.willowgarage.com/wiki/) is open-source BSD-licensed library that includes several hundreds computer vision algorithms. The document describes the so-called OpenCV 2.x API, which is essentially a C++ API, as opposite to the C-based OpenCV 1.x API. The latter is described in opencv1x.pdf.
OpenCV (Open Source Computer Vision Library: http://opencv.willowgarage.com/wiki/) is an open-source BSD-licensed library that includes several hundreds computer vision algorithms. The document describes the so-called OpenCV 2.x API, which is essentially a C++ API, as opposite to the C-based OpenCV 1.x API. The latter is described in opencv1x.pdf.
OpenCV has a modular structure (i.e. package includes several shared or static libraries). The modules are:
OpenCV has a modular structure, which means that the package includes several shared or static libraries. The modules are:
* **core** - the compact module defining basic data structures, including the dense multi-dimensional array ``Mat``, and basic functions, used by all other modules.
* **imgproc** - image processing module that includes linear and non-linear image filtering, geometrical image transformations (resize, affine and perspective warping, generic table-based remap), color space conversion, histograms etc.
* **video** - video analysis module that includes motion estimation, background subtraction and object tracking algorithms.
* **calib3d** - basic multiple-view geometry algorithms, single and stereo camera calibration, object pose estimation, stereo correspondence algorithms, elements of 3d reconstruction.
* **features2d** - salient feature detectors, descriptors and the descriptor matchers.
* **objdetect** - detection of objects, instances of the predefined classes (e.g faces, eyes, mugs, people, cars etc.)
* **highgui** - easy-to-use interface to video capturing, image and video codecs APIs, as well as simple UI capabilities.
* **core** - a compact module defining basic data structures, including the dense multi-dimensional array ``Mat``, and basic functions used by all other modules.
* **imgproc** - an image processing module that includes linear and non-linear image filtering, geometrical image transformations (resize, affine and perspective warping, generic table-based remap), color space conversion, histograms, and so on.
* **video** - a video analysis module that includes motion estimation, background subtraction, and object tracking algorithms.
* **calib3d** - basic multiple-view geometry algorithms, single and stereo camera calibration, object pose estimation, stereo correspondence algorithms, and elements of 3D reconstruction.
* **features2d** - salient feature detectors, descriptors, and descriptor matchers.
* **objdetect** - detection of objects and instances of the predefined classes (for example, faces, eyes, mugs, people, cars, and so on)
* **highgui** - an easy-to-use interface to video capturing, image and video codecs APIs, as well as simple UI capabilities.
* **gpu** - GPU-accelerated algorithms from different OpenCV modules.
* ... some other helper modules, such as FLANN and Google test wrappers, Python bindings etc.
* ... some other helper modules, such as FLANN and Google test wrappers, Python bindings, and others.
The further chapters of the document describe functionality of each module. But first, let's make an overview of the common API concepts, used thoroughly in the library.
The further chapters of the document describe functionality of each module. But first, make sure to get familiar with the common API concepts used thoroughly in the library.
The API Concepts
API Concepts
================
*"cv"* namespace
*``cv``* Namespace
----------------
All the OpenCV classes and functions are placed into *"cv"* namespace. Therefore, to access this functionality from your code, use ``cv::`` specifier or ``using namespace cv;`` directive:
All the OpenCV classes and functions are placed into the *``cv``* namespace. Therefore, to access this functionality from your code, use the ``cv::`` specifier or ``using namespace cv;`` directive:
..code-block:: c
@ -41,8 +41,8 @@ or ::
Mat H = findHomography(points1, points2, CV_RANSAC, 5 );
...
It is probable that some of the current or future OpenCV external names conflict with STL
or other libraries, in this case use explicit namespace specifiers to resolve the name conflicts: ::
It is possible that some of the current or future OpenCV external names conflict with STL
or other libraries. In this case, use explicit namespace specifiers to resolve the name conflicts: ::
First of all, ``std::vector``, ``Mat`` and other data structures used by the functions and methods have destructors that deallocate the underlying memory buffers when needed.
Secondly, in the case of ``Mat`` this *when needed* means that the destructors do not always deallocate the buffers, they take into account possible data sharing. That is, destructor decrements the reference counter, associated with the matrix data buffer, and the buffer is deallocated if and only if the reference counter reaches zero, that is, when no other structures refer to the same buffer. Similarly, when ``Mat`` instance is copied, not actual data is really copied; instead, the associated with it reference counter is incremented to memorize that there is another owner of the same data. There is also ``Mat::clone`` method that creates a full copy of the matrix data. Here is the example ::
First of all, ``std::vector``, ``Mat``, and other data structures used by the functions and methods have destructors that deallocate the underlying memory buffers when needed. This means that the destructors do not always deallocate the buffers, like in the case of ``Mat``. They take into account possible data sharing. A destructor decrements the reference counter, associated with the matrix data buffer, and the buffer is deallocated if and only if the reference counter reaches zero, which means - when no other structures refer to the same buffer. Similarly, when a ``Mat`` instance is copied, no actual data is really copied. Instead, the counter associated with its reference is incremented to memorize that there is another owner of the same data. There is also the ``Mat::clone`` method that creates a full copy of the matrix data. Here is the example ::
// create a big 8Mb matrix
Mat A(1000, 1000, CV_64F);
// create another header for the same matrix;
// this is instant operation, regardless of the matrix size.
// this is an instant operation, regardless of the matrix size.
Mat B = A;
// create another header for the 3-rd row of A; no data is copied either
Mat C = B.row(3);
@ -79,19 +77,19 @@ Secondly, in the case of ``Mat`` this *when needed* means that the destructors d
// despite that C is just a single row of the original A
B.release();
// finally, make a full copy of C. In result, the big modified
// matrix will be deallocated, since it's not referenced by anyone
// finally, make a full copy of C. As a result, the big modified
// matrix will be deallocated, since it is not referenced by anyone
C = C.clone();
Therefore, ``Mat`` and other basic structures use is simple. But what about high-level classes or even user data types that have been created without automatic memory management in mind? For them OpenCV offers ``Ptr<>`` template class, which is similar to the``std::shared_ptr`` from C++ TR1. That is, instead of using plain pointers::
Therefore, the use of ``Mat`` and other basic structures is simple. But what about high-level classes or even user data types created without automatic memory management in mind? For them OpenCV offers the``Ptr<>`` template class that is similar to``std::shared_ptr`` from C++ TR1. So, instead of using plain pointers::
T* ptr = new T(...);
one can use::
you can use::
Ptr<T> ptr = new T(...);
That is, ``Ptr<T> ptr`` incapsulates a pointer to ``T`` instance and a reference counter associated with the pointer. See ``Ptr`` description for details.
That is, ``Ptr<T> ptr`` incapsulates a pointer to a ``T`` instance and a reference counter associated with the pointer. See ``Ptr`` description for details.
..todo::
@ -100,7 +98,7 @@ That is, ``Ptr<T> ptr`` incapsulates a pointer to ``T`` instance and a reference
Automatic Allocation of the Output Data
---------------------------------------
OpenCV does not only deallocate the memory automatically, it can also allocate memory for the output function parameters automatically most of the time. That is, if a function has one or more input arrays (``cv::Mat`` instances) and some output arrays, the output arrays automatically allocated or reallocated. The size and type of the output arrays are determined from the input arrays' size and type. If needed, the functions take extra parameters that help to figure out the output array properties.
OpenCV does not only deallocate the memory automatically, it can also automatically allocate the memory for output function parameters most of the time. So, if a function has one or more input arrays (``cv::Mat`` instances) and some output arrays, the output arrays are automatically allocated or reallocated. The size and type of the output arrays are determined from the size and type of input arrays. If needed, the functions take extra parameters that help to figure out the output array properties.
Here is the example: ::
@ -128,33 +126,33 @@ Here is the example: ::
return 0;
}
The array ``frame`` is automatically allocated by ``>>`` operator, since the video frame resolution and bit-depth is known to the video capturing module. The array ``edges`` is automatically allocated by ``cvtColor`` function. It will have the same size and the bit-depth as the input array, and the number of channels will be 1, because we passed the color conversion code ``CV_BGR2GRAY`` (that means color to grayscale conversion). Note that ``frame`` and ``edges``will be allocated only once during the first execution of the loop body, since all the next video frames will have the same resolution (unless user somehow changes the video resolution, in this case the arrays will be automatically reallocated).
The array ``frame`` is automatically allocated by the ``>>`` operator, since the video frame resolution and bit-depth is known to the video capturing module. The array ``edges`` is automatically allocated by the ``cvtColor`` function. It has the same size and the bit-depth as the input array. The number of channels is 1 because the color conversion code ``CV_BGR2GRAY``is passed (that means color to grayscale conversion??). Note that ``frame`` and ``edges``are allocated only once during the first execution of the loop body, since all the next video frames have the same resolution. If you somehow change the video resolution, the arrays are automatically reallocated.
The key component of this technology is the method ``Mat::create``. It takes the desired array size and type. If the array already has the specified size and type, the method does nothing. Otherwise, it releases the previously allocated data, if any (this part involves decrementing the reference counter and comparing it with zero), and then allocates a new buffer of the required size. Most functions call this ``Mat::create`` method for each output array and so the automatic output data allocation is implemented.
The key component of this technology is the ``Mat::create`` method. It takes the desired array size and type. If the array already has the specified size and type, the method does nothing. Otherwise, it releases the previously allocated data, if any (this part involves decrementing the reference counter and comparing it with zero), and then allocates a new buffer of the required size. Most functions call this the``Mat::create`` method for each output array and so the automatic output data allocation is implemented.
Some notable exceptions from this scheme are ``cv::mixChannels``, ``cv::RNG::fill`` and a few others functions and methods. They are not able to allocate the output array, so the user has to do that in advance.
Some notable exceptions from this scheme are ``cv::mixChannels``, ``cv::RNG::fill``, and a few other functions and methods. They are not able to allocate the output array, so you have to do this in advance.
Saturation Arithmetics
----------------------
As computer vision library, OpenCV deals a lot with image pixels that are often encoded in a compact 8- or 16-bit per channel form and thus have a limited value range. Furthermore, certain operations on images, like color space conversions, brightness/contrast adjustments, sharpening, complex interpolation (bi-cubic, Lanczos) can produce values out of the available range. If we just store the lowest 8 (16) bit of the result, that will result in some visual artifacts and may affect the further image analysis. To solve this problem, we use so-called *saturation* arithmetics, e.g. to store ``r``, a result of some operation, to 8-bit image, we find the nearest value within 0..255 range:
As a computer vision library, OpenCV deals a lot with image pixels that are often encoded in a compact, 8- or 16-bit per channel, form and thus have a limited value range. Furthermore, certain operations on images, like color space conversions, brightness/contrast adjustments, sharpening, complex interpolation (bi-cubic, Lanczos) can produce values out of the available range. If you just store the lowest 8 (16) bit of the result, this will result in some visual artifacts and may affect the further image analysis. To solve this problem, the so-called *saturation* arithmetics is used. For example, to store ``r``, a result of an operation, to an 8-bit image, we find the nearest value within 0..255 range:
..math::
I(x,y)= \min ( \max (\textrm{round}(r), 0), 255)
The similar rules are applied to 8-bit signed and 16-bit signed and unsigned types. This semantics is used everywhere in the library. In C++ code it is done using ``saturate_cast<>`` functions that resembler the standard C++ cast operations. Here is the implementation of the above formula::
The similar rules are applied to 8-bit signed and 16-bit signed and unsigned types. This semantics is used everywhere in the library. In C++ code, it is done using ``saturate_cast<>`` functions that resemble standard C++ cast operations. Here is the implementation of the formula provided above::
I.at<uchar>(y, x) = saturate_cast<uchar>(r);
where ``cv::uchar`` is OpenCV's 8-bit unsigned integer type. In optimized SIMD code we use specialized instructions, like SSE2' ``paddusb``, ``packuswb`` etc. to achieve exactly the same behavior as in C++ code.
where ``cv::uchar`` is an OpenCV 8-bit unsigned integer type. In the optimized SIMD code, instructions like SSE2' ``paddusb``, ``packuswb``, and so on are used. They help achieve exactly the same behavior as in C++ code.
Fixed Pixel Types. Limited Use of Templates
-------------------------------------------
Templates is a great feature of C++ that enables implementation of very powerful, efficient and yet safe data structures and algorithms. However, the extensive use of templates may dramatically increase compile time and code size. Besides, it is difficult to separate interface and implementation when templates are used exclusively, which is fine for basic algorithms, but not good for computer vision libraries, where a single algorithm may span a thousands lines of code. Because of this, and also to simplify development of bindings for other languages, like Python, Java, Matlab, that do not have templates at all or have limited template capabilities, we prefer polymorphism and runtime dispatching over templates. In the places where runtime dispatching would be too slow (like pixel access operators), impossible (generic Ptr<> implementation) or just very inconvenient (saturate_cast<>()) we introduce small template classes, methods and functions. Everywhere else we prefer not to use templates.
Templates is a great feature of C++ that enables implementation of very powerful, efficient and yet safe data structures and algorithms. However, the extensive use of templates may dramatically increase compilation time and code size. Besides, it is difficult to separate an interface and implementation when templates are used exclusively. This could be fine for basic algorithms but not good for computer vision libraries where a single algorithm may span a thousand lines of code. Because of this and also to simplify development of bindings for other languages, like Python*, Java*, Matlab* that do not have templates at all or have limited template capabilities, the current OpenCV implementation is based on polymorphism and runtime dispatching over templates. In those places where runtime dispatching would be too slow (like pixel access operators), impossible (generic ``Ptr<>`` implementation), or just very inconvenient (``saturate_cast<>()``) the current implementation introduces small template classes, methods, and functions. Anywhere else in this implementation templates are not used.
Because of this, there is a limited fixed set of primitive data types that the library can operate on. That is, an array elements should have one of the following types:
There is a limited fixed set of primitive data types the library can operate on. That is, array elements should have one of the following types:
* 8-bit unsigned integer (uchar)
* 8-bit signed integer (schar)
@ -163,51 +161,51 @@ Because of this, there is a limited fixed set of primitive data types that the l
* 32-bit signed integer (int)
* 32-bit floating-point number (float)
* 64-bit floating-point number (double)
* a tuple of several elements, where all elements have the same type (one of the above). Array, which elements are such tuples, are called multi-channel arrays, as opposite to the single-channel arrays, which elements are scalar values. The maximum possible number of channels is defined by ``CV_CN_MAX`` constant (which is not smaller than 32).
* a tuple of several elements, where all elements have the same type (one of the above). An array, whose elements are such tuples, are called multi-channel arrays, as opposite to the single-channel arrays, whose elements are scalar values. The maximum possible number of channels is defined by the``CV_CN_MAX`` constant (which is not smaller than 32).
..todo::
Need we extend the above list? Shouldn't we throw away 8-bit signed (schar)?
For these basic types there is enumeration::
For these basic types, the following enumeration is applied::
Multi-channel (``n``-channel) types can be specified using ``CV_8UC1`` ... ``CV_64FC4`` constants (for number of channels from 1 to 4), or using ``CV_8UC(n)`` ... ``CV_64FC(n)`` or ``CV_MAKETYPE(CV_8U, n)`` ... ``CV_MAKETYPE(CV_64F, n)`` macros when the number of channels is more than 4 or unknown at compile time.
Multi-channel (``n``-channel) types can be specified using the ``CV_8UC1`` ... ``CV_64FC4`` constants (for number of channels from 1 to 4), or using the``CV_8UC(n)`` ... ``CV_64FC(n)`` or ``CV_MAKETYPE(CV_8U, n)`` ... ``CV_MAKETYPE(CV_64F, n)`` macros when the number of channels is more than 4 or unknown at compilation time.
..note::``CV_32FC1 == CV_32F``, ``CV_32FC2 == CV_32FC(2) == CV_MAKETYPE(CV_32F, 2)`` and ``CV_MAKETYPE(depth, n) == ((x&7)<<3) + (n-1)``, that is, the type constant is formed from the ``depth``, taking the lowest 3 bits, and the number of channels minus 1, taking the next ``log2(CV_CN_MAX)`` bits.
..note::``CV_32FC1 == CV_32F``, ``CV_32FC2 == CV_32FC(2) == CV_MAKETYPE(CV_32F, 2)``, and ``CV_MAKETYPE(depth, n) == ((x&7)<<3) + (n-1)``, that is, the constant type is formed from the ``depth``, taking the lowest 3 bits, and the number of channels minus 1, taking the next ``log2(CV_CN_MAX)`` bits.
Here are some examples::
Mat mtx(3, 3, CV_32F); // make 3x3 floating-point matrix
Mat cmtx(10, 1, CV_64FC2); // make 10x1 2-channel floating-point
// matrix (i.e. 10-element complex vector)
Mat img(Size(1920, 1080), CV_8UC3); // make 3-channel (color) image
Mat mtx(3, 3, CV_32F); // make a 3x3 floating-point matrix
Mat cmtx(10, 1, CV_64FC2); // make a 10x1 2-channel floating-point
// matrix (10-element complex vector)
Mat img(Size(1920, 1080), CV_8UC3); // make a 3-channel (color) image
// of 1920 columns and 1080 rows.
Mat grayscale(image.size(), CV_MAKETYPE(image.depth(), 1)); // make 1-channel image of
Mat grayscale(image.size(), CV_MAKETYPE(image.depth(), 1)); // make a 1-channel image of
// the same size and same
// channel type as img
Arrays, which elements are more complex, can not be constructed or processed using OpenCV. Furthermore, each function or method can handle only a subset of all possible array types. Usually, the more complex is the algorithm, the smaller is the supported subset of formats. Here are some typical examples of such limitations:
Arrays, whose elements are more complex, cannot be constructed or processed using OpenCV. Furthermore, each function or method can handle only a subset of all possible array types. Usually, the more complex is the algorithm, the smaller the supported subset of formats is. Here are some typical examples of such limitations:
* The face detection algorithm only works with 8-bit grayscale or color images.
* Linear algebra functions and most of the machine learning algorithms work with floating-point arrays only.
* Basic functions, such as ``cv::add``, support all types, except for ``CV_8SC(n)``.
* Color space conversion functions support 8-bit unsigned, 16-bit unsigned and 32-bit floating-point types.
* Color space conversion functions support 8-bit unsigned, 16-bit unsigned, and 32-bit floating-point types.
The subset of supported types for each functions has been defined from practical needs. All this information about supported types can be put together into a special table. In different implementations of the standard the tables may look differently, for example, on embedded platforms double-precision floating-point type (``CV_64F``) may be unavailable.
The subset of supported types for each functions has been defined from practical needs. All this information about supported types can be put together into a special table. In different implementations of the standard, the tables may look differently. For example, on embedded platforms the double-precision floating-point type (``CV_64F``) may be unavailable.
..todo::
Should we include such a table into the standard?
Should we specify minimum "must-have" set of supported formats for each functions?
Error handling
Error Handling
--------------
OpenCV uses exceptions to signal about the critical errors. When the input data has correct format and within the specified value range, but the algorithm can not succeed for some reason (e.g. the optimization algorithm did not converge), it returns a special error code (typically, just a boolean variable).
OpenCV uses exceptions to signal critical errors. When the input data has a correct format and within the specified value range, but the algorithm cannot succeed for some reason (for exampl, the optimization algorithm did not converge), it returns a special error code (typically, just a boolean variable).
The exceptions can be instances of ``cv::Exception`` class or its derivatives. In its turn, ``cv::Exception`` is a derivative of std::exception, so it can be gracefully handled in the code using other standard C++ library components.
The exceptions can be instances of the ``cv::Exception`` class or its derivatives. In its turn, ``cv::Exception`` is a derivative of ``std::exception``, so it can be gracefully handled in the code using other standard C++ library components.
The exception is typically thrown using ``CV_Error(errcode, description)`` macro, or its printf-like ``CV_Error_(errcode, printf-spec, (printf-args))`` variant, or using ``CV_Assert(condition)`` macro that checks the condition and throws exception when it is not satisfied. For performance-critical code there is ``CV_DbgAssert(condition)`` that is only retained in Debug configuration. Thanks to the automatic memory management, all the intermediate buffers are automatically deallocated in the case of sudden error; user only needs to put a try statement to catch the exceptions, if needed: ::
The exception is typically thrown using either the ``CV_Error(errcode, description)`` macro, or its printf-like ``CV_Error_(errcode, printf-spec, (printf-args))`` variant, or using the ``CV_Assert(condition)`` macro that checks the condition and throws an exception when it is not satisfied. For performance-critical code, there is ``CV_DbgAssert(condition)`` that is only retained in the Debug configuration. Thanks to the automatic memory management, all the intermediate buffers are automatically deallocated in the case of sudden error. You only need to add a try statement to catch the exceptions, if needed: ::
try
{
@ -219,7 +217,7 @@ The exception is typically thrown using ``CV_Error(errcode, description)`` macro
The current OpenCV implementation is fully reenterable, and so should be any alternative implementation targeted for multi-threaded environments. That is, the same function, the same *constant* method of a class instance, or the same *non-constant* method of different class instances can be called from different threads. Also, the same ``cv::Mat`` can be used in different threads, because the reference-counting operations use the architecture-specific atomic instructions.
The current OpenCV implementation is fully re-enterable as should be any alternative implementation targeted for multi-threaded environments. That is, the same function, the same *constant* method of a class instance, or the same *non-constant* method of different class instances can be called from different threads. Also, the same ``cv::Mat`` can be used in different threads because the reference-counting operations use the architecture-specific atomic instructions.
@ -146,7 +146,9 @@ This class computes stereo correspondence using the belief propagation algorithm
The class implements Pedro F. Felzenszwalb algorithm [Pedro F. Felzenszwalb and Daniel P. Huttenlocher. Efficient belief propagation for early vision. International Journal of Computer Vision, 70(1), October 2006]. It can compute own data cost (using a truncated linear model) or use a user-provided data cost.
**Note:**``StereoBeliefPropagation`` requires a lot of memory for message storage:
**Note:**
``StereoBeliefPropagation`` requires a lot of memory for message storage:
The OpenCV GPU module is a set of classes and functions to utilize GPU computational capabilities. It is implemented using NVIDIA* CUDA Runtime API and supports only NVIDIA GPUs. The OpenCV GPU module includes utility functions, low-level vision primitives, and high-level algorithms. The utility functions and low-level primitives provide a powerful infrastructure for developing fast vision algorithms taking advantage of GPU whereas the high-level functionality includes some state-of-the-art algorithms (such as stereo correspondence, face and people detectors, and others), ready to be used by the application developers.
The OpenCV GPU module is a set of classes and functions to utilize GPU computational capabilities. It is implemented using NVIDIA* CUDA* Runtime API and supports only NVIDIA GPUs. The OpenCV GPU module includes utility functions, low-level vision primitives, and high-level algorithms. The utility functions and low-level primitives provide a powerful infrastructure for developing fast vision algorithms taking advantage of GPU whereas the high-level functionality includes some state-of-the-art algorithms (such as stereo correspondence, face and people detectors, and others), ready to be used by the application developers.
The GPU module is designed as a host-level API. This means that if you have pre-compiled OpenCV GPU binaries, you are not required to have the CUDA Toolkit installed or write any extra code to make use of the GPU.
Elena Fedotova
14 years ago