Merge branch 'master' of https://github.com/Itseez/opencv_contrib into agast
cbalint13
10 years ago
commit
06cfc63f3a
354 changed files with 4578 additions and 12930 deletions
@ -1,418 +0,0 @@ |
||||
.. _Retina_Model: |
||||
|
||||
Discovering the human retina and its use for image processing |
||||
************************************************************* |
||||
|
||||
Goal |
||||
===== |
||||
|
||||
I present here a model of human retina that shows some interesting properties for image preprocessing and enhancement. |
||||
In this tutorial you will learn how to: |
||||
|
||||
.. container:: enumeratevisibleitemswithsquare |
||||
|
||||
+ discover the main two channels outing from your retina |
||||
|
||||
+ see the basics to use the retina model |
||||
|
||||
+ discover some parameters tweaks |
||||
|
||||
|
||||
General overview |
||||
================ |
||||
|
||||
The proposed model originates from Jeanny Herault's research [herault2010]_ at `Gipsa <http://www.gipsa-lab.inpg.fr>`_. It is involved in image processing applications with `Listic <http://www.listic.univ-savoie.fr>`_ (code maintainer and user) lab. This is not a complete model but it already present interesting properties that can be involved for enhanced image processing experience. The model allows the following human retina properties to be used : |
||||
|
||||
* spectral whitening that has 3 important effects: high spatio-temporal frequency signals canceling (noise), mid-frequencies details enhancement and low frequencies luminance energy reduction. This *all in one* property directly allows visual signals cleaning of classical undesired distortions introduced by image sensors and input luminance range. |
||||
|
||||
* local logarithmic luminance compression allows details to be enhanced even in low light conditions. |
||||
|
||||
* decorrelation of the details information (Parvocellular output channel) and transient information (events, motion made available at the Magnocellular output channel). |
||||
|
||||
The first two points are illustrated below : |
||||
|
||||
In the figure below, the OpenEXR image sample *CrissyField.exr*, a High Dynamic Range image is shown. In order to make it visible on this web-page, the original input image is linearly rescaled to the classical image luminance range [0-255] and is converted to 8bit/channel format. Such strong conversion hides many details because of too strong local contrasts. Furthermore, noise energy is also strong and pollutes visual information. |
||||
|
||||
.. image:: images/retina_TreeHdr_small.jpg |
||||
:alt: A High dynamic range image linearly rescaled within range [0-255]. |
||||
:align: center |
||||
|
||||
In the following image, applying the ideas proposed in [benoit2010]_, as your retina does, local luminance adaptation, spatial noise removal and spectral whitening work together and transmit accurate information on lower range 8bit data channels. On this picture, noise in significantly removed, local details hidden by strong luminance contrasts are enhanced. Output image keeps its naturalness and visual content is enhanced. Color processing is based on the color multiplexing/demultiplexing method proposed in [chaix2007]_. |
||||
|
||||
.. image:: images/retina_TreeHdr_retina.jpg |
||||
:alt: A High dynamic range image compressed within range [0-255] using the retina. |
||||
:align: center |
||||
|
||||
|
||||
*Note :* image sample can be downloaded from the `OpenEXR website <http://www.openexr.com>`_. Regarding this demonstration, before retina processing, input image has been linearly rescaled within 0-255 keeping its channels float format. 5% of its histogram ends has been cut (mostly removes wrong HDR pixels). Check out the sample *opencv/samples/cpp/OpenEXRimages_HighDynamicRange_Retina_toneMapping.cpp* for similar processing. The following demonstration will only consider classical 8bit/channel images. |
||||
|
||||
The retina model output channels |
||||
================================ |
||||
|
||||
The retina model presents two outputs that benefit from the above cited behaviors. |
||||
|
||||
* The first one is called the Parvocellular channel. It is mainly active in the foveal retina area (high resolution central vision with color sensitive photo-receptors), its aim is to provide accurate color vision for visual details remaining static on the retina. On the other hand objects moving on the retina projection are blurred. |
||||
|
||||
* The second well known channel is the Magnocellular channel. It is mainly active in the retina peripheral vision and send signals related to change events (motion, transient events, etc.). These outing signals also help visual system to focus/center retina on 'transient'/moving areas for more detailed analysis thus improving visual scene context and object classification. |
||||
|
||||
**NOTE :** regarding the proposed model, contrary to the real retina, we apply these two channels on the entire input images using the same resolution. This allows enhanced visual details and motion information to be extracted on all the considered images... but remember, that these two channels are complementary. For example, if Magnocellular channel gives strong energy in an area, then, the Parvocellular channel is certainly blurred there since there is a transient event. |
||||
|
||||
As an illustration, we apply in the following the retina model on a webcam video stream of a dark visual scene. In this visual scene, captured in an amphitheater of the university, some students are moving while talking to the teacher. |
||||
|
||||
In this video sequence, because of the dark ambiance, signal to noise ratio is low and color artifacts are present on visual features edges because of the low quality image capture tool-chain. |
||||
|
||||
.. image:: images/studentsSample_input.jpg |
||||
:alt: an input video stream extract sample |
||||
:align: center |
||||
|
||||
Below is shown the retina foveal vision applied on the entire image. In the used retina configuration, global luminance is preserved and local contrasts are enhanced. Also, signal to noise ratio is improved : since high frequency spatio-temporal noise is reduced, enhanced details are not corrupted by any enhanced noise. |
||||
|
||||
.. image:: images/studentsSample_parvo.jpg |
||||
:alt: the retina Parvocellular output. Enhanced details, luminance adaptation and noise removal. A processing tool for image analysis. |
||||
:align: center |
||||
|
||||
Below is the output of the Magnocellular output of the retina model. Its signals are strong where transient events occur. Here, a student is moving at the bottom of the image thus generating high energy. The remaining of the image is static however, it is corrupted by a strong noise. Here, the retina filters out most of the noise thus generating low false motion area 'alarms'. This channel can be used as a transient/moving areas detector : it would provide relevant information for a low cost segmentation tool that would highlight areas in which an event is occurring. |
||||
|
||||
.. image:: images/studentsSample_magno.jpg |
||||
:alt: the retina Magnocellular output. Enhanced transient signals (motion, etc.). A preprocessing tool for event detection. |
||||
:align: center |
||||
|
||||
Retina use case |
||||
=============== |
||||
|
||||
This model can be used basically for spatio-temporal video effects but also in the aim of : |
||||
|
||||
* performing texture analysis with enhanced signal to noise ratio and enhanced details robust against input images luminance ranges (check out the Parvocellular retina channel output) |
||||
|
||||
* performing motion analysis also taking benefit of the previously cited properties. |
||||
|
||||
Literature |
||||
========== |
||||
For more information, refer to the following papers : |
||||
|
||||
.. [benoit2010] Benoit A., Caplier A., Durette B., Herault, J., "Using Human Visual System Modeling For Bio-Inspired Low Level Image Processing", Elsevier, Computer Vision and Image Understanding 114 (2010), pp. 758-773. DOI <http://dx.doi.org/10.1016/j.cviu.2010.01.011> |
||||
|
||||
* Please have a look at the reference work of Jeanny Herault that you can read in his book : |
||||
|
||||
.. [herault2010] Vision: Images, Signals and Neural Networks: Models of Neural Processing in Visual Perception (Progress in Neural Processing),By: Jeanny Herault, ISBN: 9814273686. WAPI (Tower ID): 113266891. |
||||
|
||||
This retina filter code includes the research contributions of phd/research collegues from which code has been redrawn by the author : |
||||
|
||||
* take a look at the *retinacolor.hpp* module to discover Brice Chaix de Lavarene phD color mosaicing/demosaicing and his reference paper: |
||||
|
||||
.. [chaix2007] B. Chaix de Lavarene, D. Alleysson, B. Durette, J. Herault (2007). "Efficient demosaicing through recursive filtering", IEEE International Conference on Image Processing ICIP 2007 |
||||
|
||||
* take a look at *imagelogpolprojection.hpp* to discover retina spatial log sampling which originates from Barthelemy Durette phd with Jeanny Herault. A Retina / V1 cortex projection is also proposed and originates from Jeanny's discussions. More informations in the above cited Jeanny Heraults's book. |
||||
|
||||
Code tutorial |
||||
============= |
||||
|
||||
Please refer to the original tutorial source code in file *opencv_folder/samples/cpp/tutorial_code/bioinspired/retina_tutorial.cpp*. |
||||
|
||||
**Note :** do not forget that the retina model is included in the following namespace : *cv::bioinspired*. |
||||
|
||||
To compile it, assuming OpenCV is correctly installed, use the following command. It requires the opencv_core *(cv::Mat and friends objects management)*, opencv_highgui *(display and image/video read)* and opencv_bioinspired *(Retina description)* libraries to compile. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
// compile |
||||
gcc retina_tutorial.cpp -o Retina_tuto -lopencv_core -lopencv_highgui -lopencv_bioinspired |
||||
|
||||
// Run commands : add 'log' as a last parameter to apply a spatial log sampling (simulates retina sampling) |
||||
// run on webcam |
||||
./Retina_tuto -video |
||||
// run on video file |
||||
./Retina_tuto -video myVideo.avi |
||||
// run on an image |
||||
./Retina_tuto -image myPicture.jpg |
||||
// run on an image with log sampling |
||||
./Retina_tuto -image myPicture.jpg log |
||||
|
||||
Here is a code explanation : |
||||
|
||||
Retina definition is present in the bioinspired package and a simple include allows to use it. You can rather use the specific header : *opencv2/bioinspired.hpp* if you prefer but then include the other required openv modules : *opencv2/core.hpp* and *opencv2/highgui.hpp* |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
#include "opencv2/opencv.hpp" |
||||
|
||||
Provide user some hints to run the program with a help function |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
// the help procedure |
||||
static void help(std::string errorMessage) |
||||
{ |
||||
std::cout<<"Program init error : "<<errorMessage<<std::endl; |
||||
std::cout<<"\nProgram call procedure : retinaDemo [processing mode] [Optional : media target] [Optional LAST parameter: \"log\" to activate retina log sampling]"<<std::endl; |
||||
std::cout<<"\t[processing mode] :"<<std::endl; |
||||
std::cout<<"\t -image : for still image processing"<<std::endl; |
||||
std::cout<<"\t -video : for video stream processing"<<std::endl; |
||||
std::cout<<"\t[Optional : media target] :"<<std::endl; |
||||
std::cout<<"\t if processing an image or video file, then, specify the path and filename of the target to process"<<std::endl; |
||||
std::cout<<"\t leave empty if processing video stream coming from a connected video device"<<std::endl; |
||||
std::cout<<"\t[Optional : activate retina log sampling] : an optional last parameter can be specified for retina spatial log sampling"<<std::endl; |
||||
std::cout<<"\t set \"log\" without quotes to activate this sampling, output frame size will be divided by 4"<<std::endl; |
||||
std::cout<<"\nExamples:"<<std::endl; |
||||
std::cout<<"\t-Image processing : ./retinaDemo -image lena.jpg"<<std::endl; |
||||
std::cout<<"\t-Image processing with log sampling : ./retinaDemo -image lena.jpg log"<<std::endl; |
||||
std::cout<<"\t-Video processing : ./retinaDemo -video myMovie.mp4"<<std::endl; |
||||
std::cout<<"\t-Live video processing : ./retinaDemo -video"<<std::endl; |
||||
std::cout<<"\nPlease start again with new parameters"<<std::endl; |
||||
std::cout<<"****************************************************"<<std::endl; |
||||
std::cout<<" NOTE : this program generates the default retina parameters file 'RetinaDefaultParameters.xml'"<<std::endl; |
||||
std::cout<<" => you can use this to fine tune parameters and load them if you save to file 'RetinaSpecificParameters.xml'"<<std::endl; |
||||
} |
||||
|
||||
Then, start the main program and first declare a *cv::Mat* matrix in which input images will be loaded. Also allocate a *cv::VideoCapture* object ready to load video streams (if necessary) |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
int main(int argc, char* argv[]) { |
||||
// declare the retina input buffer... that will be fed differently in regard of the input media |
||||
cv::Mat inputFrame; |
||||
cv::VideoCapture videoCapture; // in case a video media is used, its manager is declared here |
||||
|
||||
|
||||
In the main program, before processing, first check input command parameters. Here it loads a first input image coming from a single loaded image (if user chose command *-image*) or from a video stream (if user chose command *-video*). Also, if the user added *log* command at the end of its program call, the spatial logarithmic image sampling performed by the retina is taken into account by the Boolean flag *useLogSampling*. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
// welcome message |
||||
std::cout<<"****************************************************"<<std::endl; |
||||
std::cout<<"* Retina demonstration : demonstrates the use of is a wrapper class of the Gipsa/Listic Labs retina model."<<std::endl; |
||||
std::cout<<"* This demo will try to load the file 'RetinaSpecificParameters.xml' (if exists).\nTo create it, copy the autogenerated template 'RetinaDefaultParameters.xml'.\nThen twaek it with your own retina parameters."<<std::endl; |
||||
// basic input arguments checking |
||||
if (argc<2) |
||||
{ |
||||
help("bad number of parameter"); |
||||
return -1; |
||||
} |
||||
|
||||
bool useLogSampling = !strcmp(argv[argc-1], "log"); // check if user wants retina log sampling processing |
||||
|
||||
std::string inputMediaType=argv[1]; |
||||
|
||||
////////////////////////////////////////////////////////////////////////////// |
||||
// checking input media type (still image, video file, live video acquisition) |
||||
if (!strcmp(inputMediaType.c_str(), "-image") && argc >= 3) |
||||
{ |
||||
std::cout<<"RetinaDemo: processing image "<<argv[2]<<std::endl; |
||||
// image processing case |
||||
inputFrame = cv::imread(std::string(argv[2]), 1); // load image in RGB mode |
||||
}else |
||||
if (!strcmp(inputMediaType.c_str(), "-video")) |
||||
{ |
||||
if (argc == 2 || (argc == 3 && useLogSampling)) // attempt to grab images from a video capture device |
||||
{ |
||||
videoCapture.open(0); |
||||
}else// attempt to grab images from a video filestream |
||||
{ |
||||
std::cout<<"RetinaDemo: processing video stream "<<argv[2]<<std::endl; |
||||
videoCapture.open(argv[2]); |
||||
} |
||||
|
||||
// grab a first frame to check if everything is ok |
||||
videoCapture>>inputFrame; |
||||
}else |
||||
{ |
||||
// bad command parameter |
||||
help("bad command parameter"); |
||||
return -1; |
||||
} |
||||
|
||||
Once all input parameters are processed, a first image should have been loaded, if not, display error and stop program : |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
if (inputFrame.empty()) |
||||
{ |
||||
help("Input media could not be loaded, aborting"); |
||||
return -1; |
||||
} |
||||
|
||||
Now, everything is ready to run the retina model. I propose here to allocate a retina instance and to manage the eventual log sampling option. The Retina constructor expects at least a cv::Size object that shows the input data size that will have to be managed. One can activate other options such as color and its related color multiplexing strategy (here Bayer multiplexing is chosen using *enum cv::bioinspired::RETINA_COLOR_BAYER*). If using log sampling, the image reduction factor (smaller output images) and log sampling strengh can be adjusted. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
// pointer to a retina object |
||||
cv::Ptr<cv::bioinspired::Retina> myRetina; |
||||
|
||||
// if the last parameter is 'log', then activate log sampling (favour foveal vision and subsamples peripheral vision) |
||||
if (useLogSampling) |
||||
{ |
||||
myRetina = cv::bioinspired::createRetina(inputFrame.size(), true, cv::bioinspired::RETINA_COLOR_BAYER, true, 2.0, 10.0); |
||||
} |
||||
else// -> else allocate "classical" retina : |
||||
myRetina = cv::bioinspired::createRetina(inputFrame.size()); |
||||
|
||||
Once done, the proposed code writes a default xml file that contains the default parameters of the retina. This is useful to make your own config using this template. Here generated template xml file is called *RetinaDefaultParameters.xml*. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
// save default retina parameters file in order to let you see this and maybe modify it and reload using method "setup" |
||||
myRetina->write("RetinaDefaultParameters.xml"); |
||||
|
||||
In the following line, the retina attempts to load another xml file called *RetinaSpecificParameters.xml*. If you created it and introduced your own setup, it will be loaded, in the other case, default retina parameters are used. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
// load parameters if file exists |
||||
myRetina->setup("RetinaSpecificParameters.xml"); |
||||
|
||||
It is not required here but just to show it is possible, you can reset the retina buffers to zero to force it to forget past events. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
// reset all retina buffers (imagine you close your eyes for a long time) |
||||
myRetina->clearBuffers(); |
||||
|
||||
Now, it is time to run the retina ! First create some output buffers ready to receive the two retina channels outputs |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
// declare retina output buffers |
||||
cv::Mat retinaOutput_parvo; |
||||
cv::Mat retinaOutput_magno; |
||||
|
||||
Then, run retina in a loop, load new frames from video sequence if necessary and get retina outputs back to dedicated buffers. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
// processing loop with no stop condition |
||||
while(true) |
||||
{ |
||||
// if using video stream, then, grabbing a new frame, else, input remains the same |
||||
if (videoCapture.isOpened()) |
||||
videoCapture>>inputFrame; |
||||
|
||||
// run retina filter on the loaded input frame |
||||
myRetina->run(inputFrame); |
||||
// Retrieve and display retina output |
||||
myRetina->getParvo(retinaOutput_parvo); |
||||
myRetina->getMagno(retinaOutput_magno); |
||||
cv::imshow("retina input", inputFrame); |
||||
cv::imshow("Retina Parvo", retinaOutput_parvo); |
||||
cv::imshow("Retina Magno", retinaOutput_magno); |
||||
cv::waitKey(10); |
||||
} |
||||
|
||||
That's done ! But if you want to secure the system, take care and manage Exceptions. The retina can throw some when it sees irrelevant data (no input frame, wrong setup, etc.). |
||||
Then, i recommend to surround all the retina code by a try/catch system like this : |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
try{ |
||||
// pointer to a retina object |
||||
cv::Ptr<cv::Retina> myRetina; |
||||
[---] |
||||
// processing loop with no stop condition |
||||
while(true) |
||||
{ |
||||
[---] |
||||
} |
||||
|
||||
}catch(cv::Exception e) |
||||
{ |
||||
std::cerr<<"Error using Retina : "<<e.what()<<std::endl; |
||||
} |
||||
|
||||
Retina parameters, what to do ? |
||||
=============================== |
||||
|
||||
First, it is recommended to read the reference paper : |
||||
|
||||
* Benoit A., Caplier A., Durette B., Herault, J., *"Using Human Visual System Modeling For Bio-Inspired Low Level Image Processing"*, Elsevier, Computer Vision and Image Understanding 114 (2010), pp. 758-773. DOI <http://dx.doi.org/10.1016/j.cviu.2010.01.011> |
||||
|
||||
Once done open the configuration file *RetinaDefaultParameters.xml* generated by the demo and let's have a look at it. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
<?xml version="1.0"?> |
||||
<opencv_storage> |
||||
<OPLandIPLparvo> |
||||
<colorMode>1</colorMode> |
||||
<normaliseOutput>1</normaliseOutput> |
||||
<photoreceptorsLocalAdaptationSensitivity>7.5e-01</photoreceptorsLocalAdaptationSensitivity> |
||||
<photoreceptorsTemporalConstant>9.0e-01</photoreceptorsTemporalConstant> |
||||
<photoreceptorsSpatialConstant>5.7e-01</photoreceptorsSpatialConstant> |
||||
<horizontalCellsGain>0.01</horizontalCellsGain> |
||||
<hcellsTemporalConstant>0.5</hcellsTemporalConstant> |
||||
<hcellsSpatialConstant>7.</hcellsSpatialConstant> |
||||
<ganglionCellsSensitivity>7.5e-01</ganglionCellsSensitivity></OPLandIPLparvo> |
||||
<IPLmagno> |
||||
<normaliseOutput>1</normaliseOutput> |
||||
<parasolCells_beta>0.</parasolCells_beta> |
||||
<parasolCells_tau>0.</parasolCells_tau> |
||||
<parasolCells_k>7.</parasolCells_k> |
||||
<amacrinCellsTemporalCutFrequency>2.0e+00</amacrinCellsTemporalCutFrequency> |
||||
<V0CompressionParameter>9.5e-01</V0CompressionParameter> |
||||
<localAdaptintegration_tau>0.</localAdaptintegration_tau> |
||||
<localAdaptintegration_k>7.</localAdaptintegration_k></IPLmagno> |
||||
</opencv_storage> |
||||
|
||||
Here are some hints but actually, the best parameter setup depends more on what you want to do with the retina rather than the images input that you give to retina. Apart from the more specific case of High Dynamic Range images (HDR) that require more specific setup for specific luminance compression objective, the retina behaviors should be rather stable from content to content. Note that OpenCV is able to manage such HDR format thanks to the OpenEXR images compatibility. |
||||
|
||||
Then, if the application target requires details enhancement prior to specific image processing, you need to know if mean luminance information is required or not. If not, the the retina can cancel or significantly reduce its energy thus giving more visibility to higher spatial frequency details. |
||||
|
||||
|
||||
Basic parameters |
||||
---------------- |
||||
|
||||
The most simple parameters are the following : |
||||
|
||||
* **colorMode** : let the retina process color information (if 1) or gray scale images (if 0). In this last case, only the first channel of the input will be processed. |
||||
|
||||
* **normaliseOutput** : each channel has this parameter, if value is 1, then the considered channel output is rescaled between 0 and 255. Take care in this case at the Magnocellular output level (motion/transient channel detection). Residual noise will also be rescaled ! |
||||
|
||||
**Note :** using color requires color channels multiplexing/demultipexing which requires more processing. You can expect much faster processing using gray levels : it would require around 30 product per pixel for all the retina processes and it has recently been parallelized for multicore architectures. |
||||
|
||||
Photo-receptors parameters |
||||
-------------------------- |
||||
|
||||
The following parameters act on the entry point of the retina - photo-receptors - and impact all the following processes. These sensors are low pass spatio-temporal filters that smooth temporal and spatial data and also adjust there sensitivity to local luminance thus improving details extraction and high frequency noise canceling. |
||||
|
||||
* **photoreceptorsLocalAdaptationSensitivity** between 0 and 1. Values close to 1 allow high luminance log compression effect at the photo-receptors level. Values closer to 0 give a more linear sensitivity. Increased alone, it can burn the *Parvo (details channel)* output image. If adjusted in collaboration with **ganglionCellsSensitivity** images can be very contrasted whatever the local luminance there is... at the price of a naturalness decrease. |
||||
|
||||
* **photoreceptorsTemporalConstant** this setups the temporal constant of the low pass filter effect at the entry of the retina. High value lead to strong temporal smoothing effect : moving objects are blurred and can disappear while static object are favored. But when starting the retina processing, stable state is reached lately. |
||||
|
||||
* **photoreceptorsSpatialConstant** specifies the spatial constant related to photo-receptors low pass filter effect. This parameters specify the minimum allowed spatial signal period allowed in the following. Typically, this filter should cut high frequency noise. Then a 0 value doesn't cut anything noise while higher values start to cut high spatial frequencies and more and more lower frequencies... Then, do not go to high if you wanna see some details of the input images ! A good compromise for color images is 0.53 since this won't affect too much the color spectrum. Higher values would lead to gray and blurred output images. |
||||
|
||||
Horizontal cells parameters |
||||
--------------------------- |
||||
|
||||
This parameter set tunes the neural network connected to the photo-receptors, the horizontal cells. It modulates photo-receptors sensitivity and completes the processing for final spectral whitening (part of the spatial band pass effect thus favoring visual details enhancement). |
||||
|
||||
* **horizontalCellsGain** here is a critical parameter ! If you are not interested by the mean luminance and focus on details enhancement, then, set to zero. But if you want to keep some environment luminance data, let some low spatial frequencies pass into the system and set a higher value (<1). |
||||
|
||||
* **hcellsTemporalConstant** similar to photo-receptors, this acts on the temporal constant of a low pass temporal filter that smooths input data. Here, a high value generates a high retina after effect while a lower value makes the retina more reactive. This value should be lower than **photoreceptorsTemporalConstant** to limit strong retina after effects. |
||||
|
||||
* **hcellsSpatialConstant** is the spatial constant of the low pass filter of these cells filter. It specifies the lowest spatial frequency allowed in the following. Visually, a high value leads to very low spatial frequencies processing and leads to salient halo effects. Lower values reduce this effect but the limit is : do not go lower than the value of **photoreceptorsSpatialConstant**. Those 2 parameters actually specify the spatial band-pass of the retina. |
||||
|
||||
**NOTE** after the processing managed by the previous parameters, input data is cleaned from noise and luminance in already partly enhanced. The following parameters act on the last processing stages of the two outing retina signals. |
||||
|
||||
Parvo (details channel) dedicated parameter |
||||
------------------------------------------- |
||||
|
||||
* **ganglionCellsSensitivity** specifies the strength of the final local adaptation occurring at the output of this details dedicated channel. Parameter values remain between 0 and 1. Low value tend to give a linear response while higher values enforces the remaining low contrasted areas. |
||||
|
||||
**Note :** this parameter can correct eventual burned images by favoring low energetic details of the visual scene, even in bright areas. |
||||
|
||||
IPL Magno (motion/transient channel) parameters |
||||
----------------------------------------------- |
||||
|
||||
Once image information is cleaned, this channel acts as a high pass temporal filter that only selects signals related to transient signals (events, motion, etc.). A low pass spatial filter smooths extracted transient data and a final logarithmic compression enhances low transient events thus enhancing event sensitivity. |
||||
|
||||
* **parasolCells_beta** generally set to zero, can be considered as an amplifier gain at the entry point of this processing stage. Generally set to 0. |
||||
|
||||
* **parasolCells_tau** the temporal smoothing effect that can be added |
||||
|
||||
* **parasolCells_k** the spatial constant of the spatial filtering effect, set it at a high value to favor low spatial frequency signals that are lower subject to residual noise. |
||||
|
||||
* **amacrinCellsTemporalCutFrequency** specifies the temporal constant of the high pass filter. High values let slow transient events to be selected. |
||||
|
||||
* **V0CompressionParameter** specifies the strength of the log compression. Similar behaviors to previous description but here it enforces sensitivity of transient events. |
||||
|
||||
* **localAdaptintegration_tau** generally set to 0, no real use here actually |
||||
|
||||
* **localAdaptintegration_k** specifies the size of the area on which local adaptation is performed. Low values lead to short range local adaptation (higher sensitivity to noise), high values secure log compression. |
||||
@ -1,36 +0,0 @@ |
||||
.. _Table-Of-Content-Bioinspired: |
||||
|
||||
*bioinspired* module. Algorithms inspired from biological models |
||||
---------------------------------------------------------------- |
||||
|
||||
Here you will learn how to use additional modules of OpenCV defined in the "bioinspired" module. |
||||
|
||||
.. include:: ../../definitions/tocDefinitions.rst |
||||
|
||||
+ |
||||
.. tabularcolumns:: m{100pt} m{300pt} |
||||
.. cssclass:: toctableopencv |
||||
|
||||
=============== ====================================================== |
||||
|RetinaDemoImg| **Title:** :ref:`Retina_Model` |
||||
|
||||
*Compatibility:* > OpenCV 2.4 |
||||
|
||||
*Author:* |Author_AlexB| |
||||
|
||||
You will learn how to process images and video streams with a model of retina filter for details enhancement, spatio-temporal noise removal, luminance correction and spatio-temporal events detection. |
||||
|
||||
=============== ====================================================== |
||||
|
||||
.. |RetinaDemoImg| image:: images/retina_TreeHdr_small.jpg |
||||
:height: 90pt |
||||
:width: 90pt |
||||
|
||||
.. raw:: latex |
||||
|
||||
\pagebreak |
||||
|
||||
.. toctree:: |
||||
:hidden: |
||||
|
||||
../retina_model/retina_model |
||||
@ -1,37 +0,0 @@ |
||||
.. _Table-Of-Content-CVV: |
||||
|
||||
*cvv* module. GUI for Interactive Visual Debugging |
||||
-------------------------------------------------- |
||||
|
||||
Here you will learn how to use the cvv module to ease programming computer vision software through visual debugging aids. |
||||
|
||||
.. include:: ../../definitions/tocDefinitions.rst |
||||
|
||||
+ |
||||
.. tabularcolumns:: m{100pt} m{300pt} |
||||
.. cssclass:: toctableopencv |
||||
|
||||
=============== ====================================================== |
||||
|cvvIntro| *Title:* :ref:`Visual_Debugging_Introduction` |
||||
|
||||
*Compatibility:* > OpenCV 2.4.8 |
||||
|
||||
*Author:* |Author_Bihlmaier| |
||||
|
||||
We will learn how to debug our applications in a visual and interactive way. |
||||
|
||||
=============== ====================================================== |
||||
|
||||
.. |cvvIntro| image:: images/Visual_Debugging_Introduction_Tutorial_Cover.jpg |
||||
:height: 90pt |
||||
:width: 90pt |
||||
|
||||
|
||||
.. raw:: latex |
||||
|
||||
\pagebreak |
||||
|
||||
.. toctree:: |
||||
:hidden: |
||||
|
||||
../visual_debugging_introduction/visual_debugging_introduction |
||||
@ -1,372 +0,0 @@ |
||||
.. _Visual_Debugging_Introduction: |
||||
|
||||
Interactive Visual Debugging of Computer Vision applications |
||||
************************************************************ |
||||
|
||||
What is the most common way to debug computer vision applications? |
||||
Usually the answer is temporary, hacked together, custom code that must be removed from the code for release compilation. |
||||
|
||||
In this tutorial we will show how to use the visual debugging features of the **cvv** module (*opencv2/cvv/cvv.hpp*) instead. |
||||
|
||||
|
||||
Goals |
||||
====== |
||||
|
||||
In this tutorial you will learn how to: |
||||
|
||||
* Add cvv debug calls to your application |
||||
* Use the visual debug GUI |
||||
* Enable and disable the visual debug features during compilation (with zero runtime overhead when disabled) |
||||
|
||||
|
||||
Code |
||||
===== |
||||
|
||||
The example code |
||||
|
||||
* captures images (*highgui*), e.g. from a webcam, |
||||
* applies some filters to each image (*imgproc*), |
||||
* detects image features and matches them to the previous image (*features2d*). |
||||
|
||||
If the program is compiled without visual debugging (see CMakeLists.txt below) the only result is some information printed to the command line. |
||||
We want to demonstrate how much debugging or development functionality is added by just a few lines of *cvv* commands. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
// system includes |
||||
#include <getopt.h> |
||||
#include <iostream> |
||||
|
||||
// library includes |
||||
#include <opencv2/highgui/highgui.hpp> |
||||
#include <opencv2/imgproc/imgproc.hpp> |
||||
#include <opencv2/features2d/features2d.hpp> |
||||
|
||||
// Visual debugging |
||||
#include <opencv2/cvv/cvv.hpp> |
||||
|
||||
|
||||
// helper function to convert objects that support operator<<() to std::string |
||||
template<class T> std::string toString(const T& p_arg) |
||||
{ |
||||
std::stringstream ss; |
||||
|
||||
ss << p_arg; |
||||
|
||||
return ss.str(); |
||||
} |
||||
|
||||
|
||||
void |
||||
usage() |
||||
{ |
||||
printf("usage: cvvt [-r WxH]\n"); |
||||
printf("-h print this help\n"); |
||||
printf("-r WxH change resolution to width W and height H\n"); |
||||
} |
||||
|
||||
|
||||
int |
||||
main(int argc, char** argv) |
||||
{ |
||||
#ifdef CVVISUAL_DEBUGMODE |
||||
std::cout << "Visual debugging is ENABLED" << std::endl; |
||||
#else |
||||
std::cout << "Visual debugging is DISABLED" << std::endl; |
||||
#endif |
||||
|
||||
cv::Size* resolution = nullptr; |
||||
|
||||
// parse options |
||||
const char* optstring = "hr:"; |
||||
int opt; |
||||
while ((opt = getopt(argc, argv, optstring)) != -1) { |
||||
switch (opt) { |
||||
case 'h': |
||||
usage(); |
||||
return 0; |
||||
break; |
||||
case 'r': |
||||
{ |
||||
char dummych; |
||||
resolution = new cv::Size(); |
||||
if (sscanf(optarg, "%d%c%d", &resolution->width, &dummych, &resolution->height) != 3) { |
||||
printf("%s not a valid resolution\n", optarg); |
||||
return 1; |
||||
} |
||||
} |
||||
break; |
||||
default: |
||||
usage(); |
||||
return 2; |
||||
} |
||||
} |
||||
|
||||
// setup video capture |
||||
cv::VideoCapture capture(0); |
||||
if (!capture.isOpened()) { |
||||
std::cout << "Could not open VideoCapture" << std::endl; |
||||
return 3; |
||||
} |
||||
|
||||
if (resolution) { |
||||
printf("Setting resolution to %dx%d\n", resolution->width, resolution->height); |
||||
capture.set(CV_CAP_PROP_FRAME_WIDTH, resolution->width); |
||||
capture.set(CV_CAP_PROP_FRAME_HEIGHT, resolution->height); |
||||
} |
||||
|
||||
|
||||
cv::Mat prevImgGray; |
||||
std::vector<cv::KeyPoint> prevKeypoints; |
||||
cv::Mat prevDescriptors; |
||||
|
||||
int maxFeatureCount = 500; |
||||
cv::ORB detector(maxFeatureCount); |
||||
|
||||
cv::BFMatcher matcher(cv::NORM_HAMMING); |
||||
|
||||
for (int imgId = 0; imgId < 10; imgId++) { |
||||
// capture a frame |
||||
cv::Mat imgRead; |
||||
capture >> imgRead; |
||||
printf("%d: image captured\n", imgId); |
||||
|
||||
std::string imgIdString{"imgRead"}; |
||||
imgIdString += toString(imgId); |
||||
cvv::showImage(imgRead, CVVISUAL_LOCATION, imgIdString.c_str()); |
||||
|
||||
// convert to grayscale |
||||
cv::Mat imgGray; |
||||
cv::cvtColor(imgRead, imgGray, CV_BGR2GRAY); |
||||
cvv::debugFilter(imgRead, imgGray, CVVISUAL_LOCATION, "to gray"); |
||||
|
||||
// filter edges using Canny on smoothed image |
||||
cv::Mat imgGraySmooth; |
||||
cv::GaussianBlur(imgGray, imgGraySmooth, cv::Size(9, 9), 2, 2); |
||||
cvv::debugFilter(imgGray, imgGraySmooth, CVVISUAL_LOCATION, "smoothed"); |
||||
cv::Mat imgEdges; |
||||
cv::Canny(imgGraySmooth, imgEdges, 50, 150); |
||||
cvv::showImage(imgEdges, CVVISUAL_LOCATION, "edges"); |
||||
|
||||
// dilate edges |
||||
cv::Mat imgEdgesDilated; |
||||
cv::dilate(imgEdges, imgEdgesDilated, cv::getStructuringElement(cv::MORPH_RECT, cv::Size(7, 7), cv::Point(3, 3))); |
||||
cvv::debugFilter(imgEdges, imgEdgesDilated, CVVISUAL_LOCATION, "dilated edges"); |
||||
|
||||
// detect ORB features |
||||
std::vector<cv::KeyPoint> keypoints; |
||||
cv::Mat descriptors; |
||||
detector(imgGray, cv::noArray(), keypoints, descriptors); |
||||
printf("%d: detected %zd keypoints\n", imgId, keypoints.size()); |
||||
|
||||
// match them to previous image (if available) |
||||
if (!prevImgGray.empty()) { |
||||
std::vector<cv::DMatch> matches; |
||||
matcher.match(prevDescriptors, descriptors, matches); |
||||
printf("%d: all matches size=%zd\n", imgId, matches.size()); |
||||
std::string allMatchIdString{"all matches "}; |
||||
allMatchIdString += toString(imgId-1) + "<->" + toString(imgId); |
||||
cvv::debugDMatch(prevImgGray, prevKeypoints, imgGray, keypoints, matches, CVVISUAL_LOCATION, allMatchIdString.c_str()); |
||||
|
||||
// remove worst (as defined by match distance) bestRatio quantile |
||||
double bestRatio = 0.8; |
||||
std::sort(matches.begin(), matches.end()); |
||||
matches.resize(int(bestRatio * matches.size())); |
||||
printf("%d: best matches size=%zd\n", imgId, matches.size()); |
||||
std::string bestMatchIdString{"best " + toString(bestRatio) + " matches "}; |
||||
bestMatchIdString += toString(imgId-1) + "<->" + toString(imgId); |
||||
cvv::debugDMatch(prevImgGray, prevKeypoints, imgGray, keypoints, matches, CVVISUAL_LOCATION, bestMatchIdString.c_str()); |
||||
} |
||||
|
||||
prevImgGray = imgGray; |
||||
prevKeypoints = keypoints; |
||||
prevDescriptors = descriptors; |
||||
} |
||||
|
||||
cvv::finalShow(); |
||||
|
||||
return 0; |
||||
} |
||||
|
||||
|
||||
.. code-block:: cmake |
||||
|
||||
cmake_minimum_required(VERSION 2.8) |
||||
|
||||
project(cvvisual_test) |
||||
|
||||
SET(CMAKE_PREFIX_PATH ~/software/opencv/install) |
||||
|
||||
SET(CMAKE_CXX_COMPILER "g++-4.8") |
||||
SET(CMAKE_CXX_FLAGS "-std=c++11 -O2 -pthread -Wall -Werror") |
||||
|
||||
# (un)set: cmake -DCVV_DEBUG_MODE=OFF .. |
||||
OPTION(CVV_DEBUG_MODE "cvvisual-debug-mode" ON) |
||||
if(CVV_DEBUG_MODE MATCHES ON) |
||||
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -DCVVISUAL_DEBUGMODE") |
||||
endif() |
||||
|
||||
|
||||
FIND_PACKAGE(OpenCV REQUIRED) |
||||
include_directories(${OpenCV_INCLUDE_DIRS}) |
||||
|
||||
add_executable(cvvt main.cpp) |
||||
target_link_libraries(cvvt |
||||
opencv_core opencv_highgui opencv_imgproc opencv_features2d |
||||
opencv_cvv |
||||
) |
||||
|
||||
|
||||
Explanation |
||||
============ |
||||
|
||||
#. We compile the program either using the above CmakeLists.txt with Option *CVV_DEBUG_MODE=ON* (*cmake -DCVV_DEBUG_MODE=ON*) or by adding the corresponding define *CVVISUAL_DEBUGMODE* to our compiler (e.g. *g++ -DCVVISUAL_DEBUGMODE*). |
||||
|
||||
#. The first cvv call simply shows the image (similar to *imshow*) with the imgIdString as comment. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
cvv::showImage(imgRead, CVVISUAL_LOCATION, imgIdString.c_str()); |
||||
|
||||
The image is added to the overview tab in the visual debug GUI and the cvv call blocks. |
||||
|
||||
|
||||
.. image:: images/01_overview_single.jpg |
||||
:alt: Overview with image of first cvv call |
||||
:align: center |
||||
|
||||
The image can then be selected and viewed |
||||
|
||||
.. image:: images/02_single_image_view.jpg |
||||
:alt: Display image added through cvv::showImage |
||||
:align: center |
||||
|
||||
Whenever you want to continue in the code, i.e. unblock the cvv call, you can |
||||
either continue until the next cvv call (*Step*), continue until the last cvv |
||||
call (*>>*) or run the application until it exists (*Close*). |
||||
|
||||
We decide to press the green *Step* button. |
||||
|
||||
|
||||
#. The next cvv calls are used to debug all kinds of filter operations, i.e. operations that take a picture as input and return a picture as output. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
cvv::debugFilter(imgRead, imgGray, CVVISUAL_LOCATION, "to gray"); |
||||
|
||||
As with every cvv call, you first end up in the overview. |
||||
|
||||
.. image:: images/03_overview_two.jpg |
||||
:alt: Overview with two cvv calls after pressing Step |
||||
:align: center |
||||
|
||||
We decide not to care about the conversion to gray scale and press *Step*. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
cvv::debugFilter(imgGray, imgGraySmooth, CVVISUAL_LOCATION, "smoothed"); |
||||
|
||||
If you open the filter call, you will end up in the so called "DefaultFilterView". |
||||
Both images are shown next to each other and you can (synchronized) zoom into them. |
||||
|
||||
.. image:: images/04_default_filter_view.jpg |
||||
:alt: Default filter view displaying a gray scale image and its corresponding GaussianBlur filtered one |
||||
:align: center |
||||
|
||||
When you go to very high zoom levels, each pixel is annotated with its numeric values. |
||||
|
||||
.. image:: images/05_default_filter_view_high_zoom.jpg |
||||
:alt: Default filter view at very high zoom levels |
||||
:align: center |
||||
|
||||
We press *Step* twice and have a look at the dilated image. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
cvv::debugFilter(imgEdges, imgEdgesDilated, CVVISUAL_LOCATION, "dilated edges"); |
||||
|
||||
The DefaultFilterView showing both images |
||||
|
||||
.. image:: images/06_default_filter_view_edges.jpg |
||||
:alt: Default filter view showing an edge image and the image after dilate() |
||||
:align: center |
||||
|
||||
Now we use the *View* selector in the top right and select the "DualFilterView". |
||||
We select "Changed Pixels" as filter and apply it (middle image). |
||||
|
||||
.. image:: images/07_dual_filter_view_edges.jpg |
||||
:alt: Dual filter view showing an edge image and the image after dilate() |
||||
:align: center |
||||
|
||||
After we had a close look at these images, perhaps using different views, filters or other GUI features, we decide to let the program run through. Therefore we press the yellow *>>* button. |
||||
|
||||
The program will block at |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
cvv::finalShow(); |
||||
|
||||
and display the overview with everything that was passed to cvv in the meantime. |
||||
|
||||
.. image:: images/08_overview_all.jpg |
||||
:alt: Overview displaying all cvv calls up to finalShow() |
||||
:align: center |
||||
|
||||
#. The cvv debugDMatch call is used in a situation where there are two images each with a set of descriptors that are matched to each other. |
||||
|
||||
We pass both images, both sets of keypoints and their matching to the visual debug module. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
cvv::debugDMatch(prevImgGray, prevKeypoints, imgGray, keypoints, matches, CVVISUAL_LOCATION, allMatchIdString.c_str()); |
||||
|
||||
Since we want to have a look at matches, we use the filter capabilities (*#type match*) in the overview to only show match calls. |
||||
|
||||
.. image:: images/09_overview_filtered_type_match.jpg |
||||
:alt: Overview displaying only match calls |
||||
:align: center |
||||
|
||||
We want to have a closer look at one of them, e.g. to tune our parameters that use the matching. |
||||
The view has various settings how to display keypoints and matches. |
||||
Furthermore, there is a mouseover tooltip. |
||||
|
||||
.. image:: images/10_line_match_view.jpg |
||||
:alt: Line match view |
||||
:align: center |
||||
|
||||
We see (visual debugging!) that there are many bad matches. |
||||
We decide that only 70% of the matches should be shown - those 70% with the lowest match distance. |
||||
|
||||
.. image:: images/11_line_match_view_portion_selector.jpg |
||||
:alt: Line match view showing the best 70% matches, i.e. lowest match distance |
||||
:align: center |
||||
|
||||
Having successfully reduced the visual distraction, we want to see more clearly what changed between the two images. |
||||
We select the "TranslationMatchView" that shows to where the keypoint was matched in a different way. |
||||
|
||||
.. image:: images/12_translation_match_view_portion_selector.jpg |
||||
:alt: Translation match view |
||||
:align: center |
||||
|
||||
It is easy to see that the cup was moved to the left during the two images. |
||||
|
||||
Although, cvv is all about interactively *seeing* the computer vision bugs, this is complemented by a "RawView" that allows to have a look at the underlying numeric data. |
||||
|
||||
.. image:: images/13_raw_view.jpg |
||||
:alt: Raw view of matches |
||||
:align: center |
||||
|
||||
#. There are many more useful features contained in the cvv GUI. For instance, one can group the overview tab. |
||||
|
||||
.. image:: images/14_overview_group_by_line.jpg |
||||
:alt: Overview grouped by call line |
||||
:align: center |
||||
|
||||
|
||||
Result |
||||
======= |
||||
|
||||
* By adding a view expressive lines to our computer vision program we can interactively debug it through different visualizations. |
||||
* Once we are done developing/debugging we do not have to remove those lines. We simply disable cvv debugging (*cmake -DCVV_DEBUG_MODE=OFF* or g++ without *-DCVVISUAL_DEBUGMODE*) and our programs runs without any debug overhead. |
||||
|
||||
Enjoy computer vision! |
||||
@ -1,151 +0,0 @@ |
||||
.. ximgproc: |
||||
|
||||
Structured forests for fast edge detection |
||||
****************************************** |
||||
|
||||
Introduction |
||||
------------ |
||||
In this tutorial you will learn how to use structured forests for the purpose of edge detection in an image. |
||||
|
||||
Examples |
||||
-------- |
||||
|
||||
.. image:: images/01.jpg |
||||
:height: 238pt |
||||
:width: 750pt |
||||
:alt: First example |
||||
:align: center |
||||
|
||||
.. image:: images/02.jpg |
||||
:height: 238pt |
||||
:width: 750pt |
||||
:alt: First example |
||||
:align: center |
||||
|
||||
.. image:: images/03.jpg |
||||
:height: 238pt |
||||
:width: 750pt |
||||
:alt: First example |
||||
:align: center |
||||
|
||||
.. image:: images/04.jpg |
||||
:height: 238pt |
||||
:width: 750pt |
||||
:alt: First example |
||||
:align: center |
||||
|
||||
.. image:: images/05.jpg |
||||
:height: 238pt |
||||
:width: 750pt |
||||
:alt: First example |
||||
:align: center |
||||
|
||||
.. image:: images/06.jpg |
||||
:height: 238pt |
||||
:width: 750pt |
||||
:alt: First example |
||||
:align: center |
||||
|
||||
.. image:: images/07.jpg |
||||
:height: 238pt |
||||
:width: 750pt |
||||
:alt: First example |
||||
:align: center |
||||
|
||||
.. image:: images/08.jpg |
||||
:height: 238pt |
||||
:width: 750pt |
||||
:alt: First example |
||||
:align: center |
||||
|
||||
.. image:: images/09.jpg |
||||
:height: 238pt |
||||
:width: 750pt |
||||
:alt: First example |
||||
:align: center |
||||
|
||||
.. image:: images/10.jpg |
||||
:height: 238pt |
||||
:width: 750pt |
||||
:alt: First example |
||||
:align: center |
||||
|
||||
.. image:: images/11.jpg |
||||
:height: 238pt |
||||
:width: 750pt |
||||
:alt: First example |
||||
:align: center |
||||
|
||||
.. image:: images/12.jpg |
||||
:height: 238pt |
||||
:width: 750pt |
||||
:alt: First example |
||||
:align: center |
||||
|
||||
**Note :** binarization techniques like Canny edge detector are applicable |
||||
to edges produced by both algorithms (``Sobel`` and ``StructuredEdgeDetection::detectEdges``). |
||||
|
||||
Source Code |
||||
----------- |
||||
|
||||
.. literalinclude:: ../../../../modules/ximpgroc/samples/cpp/structured_edge_detection.cpp |
||||
:language: cpp |
||||
:linenos: |
||||
:tab-width: 4 |
||||
|
||||
Explanation |
||||
----------- |
||||
|
||||
1. **Load source color image** |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
cv::Mat image = cv::imread(inFilename, 1); |
||||
if ( image.empty() ) |
||||
{ |
||||
printf("Cannot read image file: %s\n", inFilename.c_str()); |
||||
return -1; |
||||
} |
||||
|
||||
2. **Convert source image to [0;1] range** |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
image.convertTo(image, cv::DataType<float>::type, 1/255.0); |
||||
|
||||
3. **Run main algorithm** |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
cv::Mat edges(image.size(), image.type()); |
||||
|
||||
cv::Ptr<StructuredEdgeDetection> pDollar = |
||||
cv::createStructuredEdgeDetection(modelFilename); |
||||
pDollar->detectEdges(image, edges); |
||||
|
||||
4. **Show results** |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
if ( outFilename == "" ) |
||||
{ |
||||
cv::namedWindow("edges", 1); |
||||
cv::imshow("edges", edges); |
||||
|
||||
cv::waitKey(0); |
||||
} |
||||
else |
||||
cv::imwrite(outFilename, 255*edges); |
||||
|
||||
Literature |
||||
---------- |
||||
For more information, refer to the following papers : |
||||
|
||||
.. [Dollar2013] Dollar P., Zitnick C. L., "Structured forests for fast edge detection", |
||||
IEEE International Conference on Computer Vision (ICCV), 2013, |
||||
pp. 1841-1848. `DOI <http://dx.doi.org/10.1109/ICCV.2013.231>`_ |
||||
|
||||
.. [Lim2013] Lim J. J., Zitnick C. L., Dollar P., "Sketch Tokens: A Learned |
||||
Mid-level Representation for Contour and Object Detection", |
||||
Comoputer Vision and Pattern Recognition (CVPR), 2013, |
||||
pp. 3158-3165. `DOI <http://dx.doi.org/10.1109/CVPR.2013.406>`_ |
||||
@ -1,115 +0,0 @@ |
||||
.. ximgproc: |
||||
|
||||
Structured forest training |
||||
************************** |
||||
|
||||
Introduction |
||||
------------ |
||||
In this tutorial we show how to train your own structured forest using author's initial Matlab implementation. |
||||
|
||||
Training pipeline |
||||
----------------- |
||||
|
||||
1. Download "Piotr's Toolbox" from `link <http://vision.ucsd.edu/~pdollar/toolbox/doc/index.html>`_ |
||||
and put it into separate directory, e.g. PToolbox |
||||
|
||||
2. Download BSDS500 dataset from `link <http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/BSR/>` |
||||
and put it into separate directory named exactly BSR |
||||
|
||||
3. Add both directory and their subdirectories to Matlab path. |
||||
|
||||
4. Download detector code from `link <http://research.microsoft.com/en-us/downloads/389109f6-b4e8-404c-84bf-239f7cbf4e3d/>` |
||||
and put it into root directory. Now you should have :: |
||||
|
||||
. |
||||
BSR |
||||
PToolbox |
||||
models |
||||
private |
||||
Contents.m |
||||
edgesChns.m |
||||
edgesDemo.m |
||||
edgesDemoRgbd.m |
||||
edgesDetect.m |
||||
edgesEval.m |
||||
edgesEvalDir.m |
||||
edgesEvalImg.m |
||||
edgesEvalPlot.m |
||||
edgesSweeps.m |
||||
edgesTrain.m |
||||
license.txt |
||||
readme.txt |
||||
|
||||
5. Rename models/forest/modelFinal.mat to models/forest/modelFinal.mat.backup |
||||
|
||||
6. Open edgesChns.m and comment lines 26--41. Add after commented lines the following: :: |
||||
|
||||
shrink=opts.shrink; |
||||
chns = single(getFeatures( im2double(I) )); |
||||
|
||||
7. Now it is time to compile promised getFeatures. I do with the following code: |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
#include <cv.h> |
||||
#include <highgui.h> |
||||
|
||||
#include <mat.h> |
||||
#include <mex.h> |
||||
|
||||
#include "MxArray.hpp" // https://github.com/kyamagu/mexopencv |
||||
|
||||
class NewRFFeatureGetter : public cv::RFFeatureGetter |
||||
{ |
||||
public: |
||||
NewRFFeatureGetter() : name("NewRFFeatureGetter"){} |
||||
|
||||
virtual void getFeatures(const cv::Mat &src, NChannelsMat &features, |
||||
const int gnrmRad, const int gsmthRad, |
||||
const int shrink, const int outNum, const int gradNum) const |
||||
{ |
||||
// here your feature extraction code, the default one is: |
||||
// resulting features Mat should be n-channels, floating point matrix |
||||
} |
||||
|
||||
protected: |
||||
cv::String name; |
||||
}; |
||||
|
||||
MEXFUNCTION_LINKAGE void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) |
||||
{ |
||||
if (nlhs != 1) mexErrMsgTxt("nlhs != 1"); |
||||
if (nrhs != 1) mexErrMsgTxt("nrhs != 1"); |
||||
|
||||
cv::Mat src = MxArray(prhs[0]).toMat(); |
||||
src.convertTo(src, cv::DataType<float>::type); |
||||
|
||||
std::string modelFile = MxArray(prhs[1]).toString(); |
||||
NewRFFeatureGetter *pDollar = createNewRFFeatureGetter(); |
||||
|
||||
cv::Mat edges; |
||||
pDollar->getFeatures(src, edges, 4, 0, 2, 13, 4); |
||||
// you can use other numbers here |
||||
|
||||
edges.convertTo(edges, cv::DataType<double>::type); |
||||
|
||||
plhs[0] = MxArray(edges); |
||||
} |
||||
|
||||
8. Place compiled mex file into root dir and run edgesDemo. |
||||
You will need to wait a couple of hours after that the new model |
||||
will appear inside models/forest/. |
||||
|
||||
9. The final step is converting trained model from Matlab binary format |
||||
to YAML which you can use with our ocv::StructuredEdgeDetection. |
||||
For this purpose run opencv_contrib/doc/tutorials/ximpgroc/training/modelConvert(model, "model.yml") |
||||
|
||||
How to use your model |
||||
--------------------- |
||||
|
||||
Just use expanded constructor with above defined class NewRFFeatureGetter |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
cv::StructuredEdgeDetection pDollar |
||||
= cv::createStructuredEdgeDetection( modelName, makePtr<NewRFFeatureGetter>() ); |
||||
@ -1,2 +1,2 @@ |
||||
set(the_description "Background Segmentation Algorithms") |
||||
ocv_define_module(bgsegm opencv_core opencv_imgproc opencv_video opencv_highgui) |
||||
ocv_define_module(bgsegm opencv_core opencv_imgproc opencv_video opencv_highgui WRAP python) |
||||
|
||||
@ -1,3 +1,3 @@ |
||||
set(the_description "Biologically inspired algorithms") |
||||
ocv_warnings_disable(CMAKE_CXX_FLAGS -Wundef) |
||||
ocv_define_module(bioinspired opencv_core OPTIONAL opencv_highgui opencv_ocl) |
||||
ocv_define_module(bioinspired opencv_core OPTIONAL opencv_highgui opencv_ocl WRAP java) |
||||
|
||||
@ -1,10 +0,0 @@ |
||||
******************************************************************** |
||||
bioinspired. Biologically inspired vision models and derivated tools |
||||
******************************************************************** |
||||
|
||||
The module provides biological visual systems models (human visual system and others). It also provides derivated objects that take advantage of those bio-inspired models. |
||||
|
||||
.. toctree:: |
||||
:maxdepth: 2 |
||||
|
||||
Human retina documentation <retina> |
||||
@ -1,493 +0,0 @@ |
||||
Retina : a Bio mimetic human retina model |
||||
***************************************** |
||||
|
||||
.. highlight:: cpp |
||||
|
||||
Retina |
||||
====== |
||||
.. ocv:class:: Retina : public Algorithm |
||||
|
||||
**Note** : do not forget that the retina model is included in the following namespace : *cv::bioinspired*. |
||||
|
||||
Introduction |
||||
++++++++++++ |
||||
|
||||
Class which provides the main controls to the Gipsa/Listic labs human retina model. This is a non separable spatio-temporal filter modelling the two main retina information channels : |
||||
|
||||
* foveal vision for detailled color vision : the parvocellular pathway. |
||||
|
||||
* peripheral vision for sensitive transient signals detection (motion and events) : the magnocellular pathway. |
||||
|
||||
From a general point of view, this filter whitens the image spectrum and corrects luminance thanks to local adaptation. An other important property is its hability to filter out spatio-temporal noise while enhancing details. |
||||
This model originates from Jeanny Herault work [Herault2010]_. It has been involved in Alexandre Benoit phd and his current research [Benoit2010]_, [Strat2013]_ (he currently maintains this module within OpenCV). It includes the work of other Jeanny's phd student such as [Chaix2007]_ and the log polar transformations of Barthelemy Durette described in Jeanny's book. |
||||
|
||||
**NOTES :** |
||||
|
||||
* For ease of use in computer vision applications, the two retina channels are applied homogeneously on all the input images. This does not follow the real retina topology but this can still be done using the log sampling capabilities proposed within the class. |
||||
|
||||
* Extend the retina description and code use in the tutorial/contrib section for complementary explanations. |
||||
|
||||
Preliminary illustration |
||||
++++++++++++++++++++++++ |
||||
|
||||
As a preliminary presentation, let's start with a visual example. We propose to apply the filter on a low quality color jpeg image with backlight problems. Here is the considered input... *"Well, my eyes were able to see more that this strange black shadow..."* |
||||
|
||||
.. image:: images/retinaInput.jpg |
||||
:alt: a low quality color jpeg image with backlight problems. |
||||
:align: center |
||||
|
||||
Below, the retina foveal model applied on the entire image with default parameters. Here contours are enforced, halo effects are voluntary visible with this configuration. See parameters discussion below and increase horizontalCellsGain near 1 to remove them. |
||||
|
||||
.. image:: images/retinaOutput_default.jpg |
||||
:alt: the retina foveal model applied on the entire image with default parameters. Here contours are enforced, luminance is corrected and halo effects are voluntary visible with this configuration, increase horizontalCellsGain near 1 to remove them. |
||||
:align: center |
||||
|
||||
Below, a second retina foveal model output applied on the entire image with a parameters setup focused on naturalness perception. *"Hey, i now recognize my cat, looking at the mountains at the end of the day !"*. Here contours are enforced, luminance is corrected but halos are avoided with this configuration. The backlight effect is corrected and highlight details are still preserved. Then, even on a low quality jpeg image, if some luminance information remains, the retina is able to reconstruct a proper visual signal. Such configuration is also usefull for High Dynamic Range (*HDR*) images compression to 8bit images as discussed in [Benoit2010]_ and in the demonstration codes discussed below. |
||||
As shown at the end of the page, parameters change from defaults are : |
||||
|
||||
* horizontalCellsGain=0.3 |
||||
|
||||
* photoreceptorsLocalAdaptationSensitivity=ganglioncellsSensitivity=0.89. |
||||
|
||||
.. image:: images/retinaOutput_realistic.jpg |
||||
:alt: the retina foveal model applied on the entire image with 'naturalness' parameters. Here contours are enforced but are avoided with this configuration, horizontalCellsGain is 0.3 and photoreceptorsLocalAdaptationSensitivity=ganglioncellsSensitivity=0.89. |
||||
:align: center |
||||
|
||||
As observed in this preliminary demo, the retina can be settled up with various parameters, by default, as shown on the figure above, the retina strongly reduces mean luminance energy and enforces all details of the visual scene. Luminance energy and halo effects can be modulated (exagerated to cancelled as shown on the two examples). In order to use your own parameters, you can use at least one time the *write(String fs)* method which will write a proper XML file with all default parameters. Then, tweak it on your own and reload them at any time using method *setup(String fs)*. These methods update a *Retina::RetinaParameters* member structure that is described hereafter. XML parameters file samples are shown at the end of the page. |
||||
|
||||
Here is an overview of the abstract Retina interface, allocate one instance with the *createRetina* functions.:: |
||||
|
||||
namespace cv{namespace bioinspired{ |
||||
|
||||
class Retina : public Algorithm |
||||
{ |
||||
public: |
||||
// parameters setup instance |
||||
struct RetinaParameters; // this class is detailled later |
||||
|
||||
// main method for input frame processing (all use method, can also perform High Dynamic Range tone mapping) |
||||
void run (InputArray inputImage); |
||||
|
||||
// specific method aiming at correcting luminance only (faster High Dynamic Range tone mapping) |
||||
void applyFastToneMapping(InputArray inputImage, OutputArray outputToneMappedImage) |
||||
|
||||
// output buffers retreival methods |
||||
// -> foveal color vision details channel with luminance and noise correction |
||||
void getParvo (OutputArray retinaOutput_parvo); |
||||
void getParvoRAW (OutputArray retinaOutput_parvo);// retreive original output buffers without any normalisation |
||||
const Mat getParvoRAW () const;// retreive original output buffers without any normalisation |
||||
// -> peripheral monochrome motion and events (transient information) channel |
||||
void getMagno (OutputArray retinaOutput_magno); |
||||
void getMagnoRAW (OutputArray retinaOutput_magno); // retreive original output buffers without any normalisation |
||||
const Mat getMagnoRAW () const;// retreive original output buffers without any normalisation |
||||
|
||||
// reset retina buffers... equivalent to closing your eyes for some seconds |
||||
void clearBuffers (); |
||||
|
||||
// retreive input and output buffers sizes |
||||
Size getInputSize (); |
||||
Size getOutputSize (); |
||||
|
||||
// setup methods with specific parameters specification of global xml config file loading/write |
||||
void setup (String retinaParameterFile="", const bool applyDefaultSetupOnFailure=true); |
||||
void setup (FileStorage &fs, const bool applyDefaultSetupOnFailure=true); |
||||
void setup (RetinaParameters newParameters); |
||||
struct Retina::RetinaParameters getParameters (); |
||||
const String printSetup (); |
||||
virtual void write (String fs) const; |
||||
virtual void write (FileStorage &fs) const; |
||||
void setupOPLandIPLParvoChannel (const bool colorMode=true, const bool normaliseOutput=true, const float photoreceptorsLocalAdaptationSensitivity=0.7, const float photoreceptorsTemporalConstant=0.5, const float photoreceptorsSpatialConstant=0.53, const float horizontalCellsGain=0, const float HcellsTemporalConstant=1, const float HcellsSpatialConstant=7, const float ganglionCellsSensitivity=0.7); |
||||
void setupIPLMagnoChannel (const bool normaliseOutput=true, const float parasolCells_beta=0, const float parasolCells_tau=0, const float parasolCells_k=7, const float amacrinCellsTemporalCutFrequency=1.2, const float V0CompressionParameter=0.95, const float localAdaptintegration_tau=0, const float localAdaptintegration_k=7); |
||||
void setColorSaturation (const bool saturateColors=true, const float colorSaturationValue=4.0); |
||||
void activateMovingContoursProcessing (const bool activate); |
||||
void activateContoursProcessing (const bool activate); |
||||
}; |
||||
|
||||
// Allocators |
||||
cv::Ptr<Retina> createRetina (Size inputSize); |
||||
cv::Ptr<Retina> createRetina (Size inputSize, const bool colorMode, RETINA_COLORSAMPLINGMETHOD colorSamplingMethod=RETINA_COLOR_BAYER, const bool useRetinaLogSampling=false, const double reductionFactor=1.0, const double samplingStrenght=10.0); |
||||
}} // cv and bioinspired namespaces end |
||||
|
||||
.. Sample code:: |
||||
|
||||
* An example on retina tone mapping can be found at opencv_source_code/samples/cpp/OpenEXRimages_HDR_Retina_toneMapping.cpp |
||||
* An example on retina tone mapping on video input can be found at opencv_source_code/samples/cpp/OpenEXRimages_HDR_Retina_toneMapping.cpp |
||||
* A complete example illustrating the retina interface can be found at opencv_source_code/samples/cpp/retinaDemo.cpp |
||||
|
||||
Description |
||||
+++++++++++ |
||||
|
||||
Class which allows the `Gipsa <http://www.gipsa-lab.inpg.fr>`_ (preliminary work) / `Listic <http://www.listic.univ-savoie.fr>`_ (code maintainer and user) labs retina model to be used. This class allows human retina spatio-temporal image processing to be applied on still images, images sequences and video sequences. Briefly, here are the main human retina model properties: |
||||
|
||||
* spectral whithening (mid-frequency details enhancement) |
||||
|
||||
* high frequency spatio-temporal noise reduction (temporal noise and high frequency spatial noise are minimized) |
||||
|
||||
* low frequency luminance reduction (luminance range compression) : high luminance regions do not hide details in darker regions anymore |
||||
|
||||
* local logarithmic luminance compression allows details to be enhanced even in low light conditions |
||||
|
||||
Use : this model can be used basically for spatio-temporal video effects but also in the aim of : |
||||
|
||||
* performing texture analysis with enhanced signal to noise ratio and enhanced details robust against input images luminance ranges (check out the parvocellular retina channel output, by using the provided **getParvo** methods) |
||||
|
||||
* performing motion analysis also taking benefit of the previously cited properties (check out the magnocellular retina channel output, by using the provided **getMagno** methods) |
||||
|
||||
* general image/video sequence description using either one or both channels. An example of the use of Retina in a Bag of Words approach is given in [Strat2013]_. |
||||
|
||||
Literature |
||||
========== |
||||
For more information, refer to the following papers : |
||||
|
||||
* Model description : |
||||
|
||||
.. [Benoit2010] Benoit A., Caplier A., Durette B., Herault, J., "Using Human Visual System Modeling For Bio-Inspired Low Level Image Processing", Elsevier, Computer Vision and Image Understanding 114 (2010), pp. 758-773. DOI <http://dx.doi.org/10.1016/j.cviu.2010.01.011> |
||||
|
||||
* Model use in a Bag of Words approach : |
||||
|
||||
.. [Strat2013] Strat S., Benoit A., Lambert P., "Retina enhanced SIFT descriptors for video indexing", CBMI2013, Veszprém, Hungary, 2013. |
||||
|
||||
* Please have a look at the reference work of Jeanny Herault that you can read in his book : |
||||
|
||||
.. [Herault2010] Vision: Images, Signals and Neural Networks: Models of Neural Processing in Visual Perception (Progress in Neural Processing),By: Jeanny Herault, ISBN: 9814273686. WAPI (Tower ID): 113266891. |
||||
|
||||
This retina filter code includes the research contributions of phd/research collegues from which code has been redrawn by the author : |
||||
|
||||
* take a look at the *retinacolor.hpp* module to discover Brice Chaix de Lavarene phD color mosaicing/demosaicing and his reference paper: |
||||
|
||||
.. [Chaix2007] B. Chaix de Lavarene, D. Alleysson, B. Durette, J. Herault (2007). "Efficient demosaicing through recursive filtering", IEEE International Conference on Image Processing ICIP 2007 |
||||
|
||||
* take a look at *imagelogpolprojection.hpp* to discover retina spatial log sampling which originates from Barthelemy Durette phd with Jeanny Herault. A Retina / V1 cortex projection is also proposed and originates from Jeanny's discussions. More informations in the above cited Jeanny Heraults's book. |
||||
|
||||
* Meylan&al work on HDR tone mapping that is implemented as a specific method within the model : |
||||
|
||||
.. [Meylan2007] L. Meylan , D. Alleysson, S. Susstrunk, "A Model of Retinal Local Adaptation for the Tone Mapping of Color Filter Array Images", Journal of Optical Society of America, A, Vol. 24, N 9, September, 1st, 2007, pp. 2807-2816 |
||||
|
||||
Demos and experiments ! |
||||
======================= |
||||
|
||||
**NOTE : Complementary to the following examples, have a look at the Retina tutorial in the tutorial/contrib section for complementary explanations.** |
||||
|
||||
Take a look at the provided C++ examples provided with OpenCV : |
||||
|
||||
* **samples/cpp/retinademo.cpp** shows how to use the retina module for details enhancement (Parvo channel output) and transient maps observation (Magno channel output). You can play with images, video sequences and webcam video. |
||||
Typical uses are (provided your OpenCV installation is situated in folder *OpenCVReleaseFolder*) |
||||
|
||||
* image processing : **OpenCVReleaseFolder/bin/retinademo -image myPicture.jpg** |
||||
|
||||
* video processing : **OpenCVReleaseFolder/bin/retinademo -video myMovie.avi** |
||||
|
||||
* webcam processing: **OpenCVReleaseFolder/bin/retinademo -video** |
||||
|
||||
**Note :** This demo generates the file *RetinaDefaultParameters.xml* which contains the default parameters of the retina. Then, rename this as *RetinaSpecificParameters.xml*, adjust the parameters the way you want and reload the program to check the effect. |
||||
|
||||
|
||||
* **samples/cpp/OpenEXRimages_HDR_Retina_toneMapping.cpp** shows how to use the retina to perform High Dynamic Range (HDR) luminance compression |
||||
|
||||
Then, take a HDR image using bracketing with your camera and generate an OpenEXR image and then process it using the demo. |
||||
|
||||
Typical use, supposing that you have the OpenEXR image such as *memorial.exr* (present in the samples/cpp/ folder) |
||||
|
||||
**OpenCVReleaseFolder/bin/OpenEXRimages_HDR_Retina_toneMapping memorial.exr [optional: 'fast']** |
||||
|
||||
Note that some sliders are made available to allow you to play with luminance compression. |
||||
|
||||
If not using the 'fast' option, then, tone mapping is performed using the full retina model [Benoit2010]_. It includes spectral whitening that allows luminance energy to be reduced. When using the 'fast' option, then, a simpler method is used, it is an adaptation of the algorithm presented in [Meylan2007]_. This method gives also good results and is faster to process but it sometimes requires some more parameters adjustement. |
||||
|
||||
|
||||
Methods description |
||||
=================== |
||||
|
||||
Here are detailled the main methods to control the retina model |
||||
|
||||
Ptr<Retina>::createRetina |
||||
+++++++++++++++++++++++++ |
||||
|
||||
.. ocv:function:: Ptr<cv::bioinspired::Retina> createRetina(Size inputSize) |
||||
.. ocv:function:: Ptr<cv::bioinspired::Retina> createRetina(Size inputSize, const bool colorMode, cv::bioinspired::RETINA_COLORSAMPLINGMETHOD colorSamplingMethod = cv::bioinspired::RETINA_COLOR_BAYER, const bool useRetinaLogSampling = false, const double reductionFactor = 1.0, const double samplingStrenght = 10.0 ) |
||||
|
||||
Constructors from standardized interfaces : retreive a smart pointer to a Retina instance |
||||
|
||||
:param inputSize: the input frame size |
||||
:param colorMode: the chosen processing mode : with or without color processing |
||||
:param colorSamplingMethod: specifies which kind of color sampling will be used : |
||||
|
||||
* cv::bioinspired::RETINA_COLOR_RANDOM: each pixel position is either R, G or B in a random choice |
||||
|
||||
* cv::bioinspired::RETINA_COLOR_DIAGONAL: color sampling is RGBRGBRGB..., line 2 BRGBRGBRG..., line 3, GBRGBRGBR... |
||||
|
||||
* cv::bioinspired::RETINA_COLOR_BAYER: standard bayer sampling |
||||
|
||||
:param useRetinaLogSampling: activate retina log sampling, if true, the 2 following parameters can be used |
||||
:param reductionFactor: only usefull if param useRetinaLogSampling=true, specifies the reduction factor of the output frame (as the center (fovea) is high resolution and corners can be underscaled, then a reduction of the output is allowed without precision leak |
||||
:param samplingStrenght: only usefull if param useRetinaLogSampling=true, specifies the strenght of the log scale that is applied |
||||
|
||||
Retina::activateContoursProcessing |
||||
++++++++++++++++++++++++++++++++++ |
||||
|
||||
.. ocv:function:: void Retina::activateContoursProcessing(const bool activate) |
||||
|
||||
Activate/desactivate the Parvocellular pathway processing (contours information extraction), by default, it is activated |
||||
|
||||
:param activate: true if Parvocellular (contours information extraction) output should be activated, false if not... if activated, the Parvocellular output can be retrieved using the **getParvo** methods |
||||
|
||||
Retina::activateMovingContoursProcessing |
||||
++++++++++++++++++++++++++++++++++++++++ |
||||
|
||||
.. ocv:function:: void Retina::activateMovingContoursProcessing(const bool activate) |
||||
|
||||
Activate/desactivate the Magnocellular pathway processing (motion information extraction), by default, it is activated |
||||
|
||||
:param activate: true if Magnocellular output should be activated, false if not... if activated, the Magnocellular output can be retrieved using the **getMagno** methods |
||||
|
||||
Retina::clearBuffers |
||||
++++++++++++++++++++ |
||||
|
||||
.. ocv:function:: void Retina::clearBuffers() |
||||
|
||||
Clears all retina buffers (equivalent to opening the eyes after a long period of eye close ;o) whatchout the temporal transition occuring just after this method call. |
||||
|
||||
Retina::getParvo |
||||
++++++++++++++++ |
||||
|
||||
.. ocv:function:: void Retina::getParvo( OutputArray retinaOutput_parvo ) |
||||
.. ocv:function:: void Retina::getParvoRAW( OutputArray retinaOutput_parvo ) |
||||
.. ocv:function:: const Mat Retina::getParvoRAW() const |
||||
|
||||
Accessor of the details channel of the retina (models foveal vision). Warning, getParvoRAW methods return buffers that are not rescaled within range [0;255] while the non RAW method allows a normalized matrix to be retrieved. |
||||
|
||||
:param retinaOutput_parvo: the output buffer (reallocated if necessary), format can be : |
||||
|
||||
* a Mat, this output is rescaled for standard 8bits image processing use in OpenCV |
||||
|
||||
* RAW methods actually return a 1D matrix (encoding is R1, R2, ... Rn, G1, G2, ..., Gn, B1, B2, ...Bn), this output is the original retina filter model output, without any quantification or rescaling. |
||||
|
||||
Retina::getMagno |
||||
++++++++++++++++ |
||||
|
||||
.. ocv:function:: void Retina::getMagno( OutputArray retinaOutput_magno ) |
||||
.. ocv:function:: void Retina::getMagnoRAW( OutputArray retinaOutput_magno ) |
||||
.. ocv:function:: const Mat Retina::getMagnoRAW() const |
||||
|
||||
Accessor of the motion channel of the retina (models peripheral vision). Warning, getMagnoRAW methods return buffers that are not rescaled within range [0;255] while the non RAW method allows a normalized matrix to be retrieved. |
||||
|
||||
:param retinaOutput_magno: the output buffer (reallocated if necessary), format can be : |
||||
|
||||
* a Mat, this output is rescaled for standard 8bits image processing use in OpenCV |
||||
|
||||
* RAW methods actually return a 1D matrix (encoding is M1, M2,... Mn), this output is the original retina filter model output, without any quantification or rescaling. |
||||
|
||||
Retina::getInputSize |
||||
++++++++++++++++++++ |
||||
|
||||
.. ocv:function:: Size Retina::getInputSize() |
||||
|
||||
Retreive retina input buffer size |
||||
|
||||
:return: the retina input buffer size |
||||
|
||||
Retina::getOutputSize |
||||
+++++++++++++++++++++ |
||||
|
||||
.. ocv:function:: Size Retina::getOutputSize() |
||||
|
||||
Retreive retina output buffer size that can be different from the input if a spatial log transformation is applied |
||||
|
||||
:return: the retina output buffer size |
||||
|
||||
Retina::printSetup |
||||
++++++++++++++++++ |
||||
|
||||
.. ocv:function:: const String Retina::printSetup() |
||||
|
||||
Outputs a string showing the used parameters setup |
||||
|
||||
:return: a string which contains formated parameters information |
||||
|
||||
Retina::run |
||||
+++++++++++ |
||||
|
||||
.. ocv:function:: void Retina::run(InputArray inputImage) |
||||
|
||||
Method which allows retina to be applied on an input image, after run, encapsulated retina module is ready to deliver its outputs using dedicated acccessors, see getParvo and getMagno methods |
||||
|
||||
:param inputImage: the input Mat image to be processed, can be gray level or BGR coded in any format (from 8bit to 16bits) |
||||
|
||||
Retina::applyFastToneMapping |
||||
++++++++++++++++++++++++++++ |
||||
|
||||
.. ocv:function:: void Retina::applyFastToneMapping(InputArray inputImage, OutputArray outputToneMappedImage) |
||||
|
||||
Method which processes an image in the aim to correct its luminance : correct backlight problems, enhance details in shadows. This method is designed to perform High Dynamic Range image tone mapping (compress >8bit/pixel images to 8bit/pixel). This is a simplified version of the Retina Parvocellular model (simplified version of the run/getParvo methods call) since it does not include the spatio-temporal filter modelling the Outer Plexiform Layer of the retina that performs spectral whitening and many other stuff. However, it works great for tone mapping and in a faster way. |
||||
|
||||
Check the demos and experiments section to see examples and the way to perform tone mapping using the original retina model and the method. |
||||
|
||||
:param inputImage: the input image to process (should be coded in float format : CV_32F, CV_32FC1, CV_32F_C3, CV_32F_C4, the 4th channel won't be considered). |
||||
:param outputToneMappedImage: the output 8bit/channel tone mapped image (CV_8U or CV_8UC3 format). |
||||
|
||||
Retina::setColorSaturation |
||||
++++++++++++++++++++++++++ |
||||
|
||||
.. ocv:function:: void Retina::setColorSaturation(const bool saturateColors = true, const float colorSaturationValue = 4.0 ) |
||||
|
||||
Activate color saturation as the final step of the color demultiplexing process -> this saturation is a sigmoide function applied to each channel of the demultiplexed image. |
||||
|
||||
:param saturateColors: boolean that activates color saturation (if true) or desactivate (if false) |
||||
:param colorSaturationValue: the saturation factor : a simple factor applied on the chrominance buffers |
||||
|
||||
|
||||
Retina::setup |
||||
+++++++++++++ |
||||
|
||||
.. ocv:function:: void Retina::setup(String retinaParameterFile = "", const bool applyDefaultSetupOnFailure = true ) |
||||
.. ocv:function:: void Retina::setup(FileStorage & fs, const bool applyDefaultSetupOnFailure = true ) |
||||
.. ocv:function:: void Retina::setup(RetinaParameters newParameters) |
||||
|
||||
Try to open an XML retina parameters file to adjust current retina instance setup => if the xml file does not exist, then default setup is applied => warning, Exceptions are thrown if read XML file is not valid |
||||
|
||||
:param retinaParameterFile: the parameters filename |
||||
:param applyDefaultSetupOnFailure: set to true if an error must be thrown on error |
||||
:param fs: the open Filestorage which contains retina parameters |
||||
:param newParameters: a parameters structures updated with the new target configuration. You can retreive the current parameers structure using method *Retina::RetinaParameters Retina::getParameters()* and update it before running method *setup*. |
||||
|
||||
Retina::write |
||||
+++++++++++++ |
||||
|
||||
.. ocv:function:: void Retina::write( String fs ) const |
||||
.. ocv:function:: void Retina::write( FileStorage& fs ) const |
||||
|
||||
Write xml/yml formated parameters information |
||||
|
||||
:param fs: the filename of the xml file that will be open and writen with formatted parameters information |
||||
|
||||
Retina::setupIPLMagnoChannel |
||||
++++++++++++++++++++++++++++ |
||||
|
||||
.. ocv:function:: void Retina::setupIPLMagnoChannel(const bool normaliseOutput = true, const float parasolCells_beta = 0, const float parasolCells_tau = 0, const float parasolCells_k = 7, const float amacrinCellsTemporalCutFrequency = 1.2, const float V0CompressionParameter = 0.95, const float localAdaptintegration_tau = 0, const float localAdaptintegration_k = 7 ) |
||||
|
||||
Set parameters values for the Inner Plexiform Layer (IPL) magnocellular channel this channel processes signals output from OPL processing stage in peripheral vision, it allows motion information enhancement. It is decorrelated from the details channel. See reference papers for more details. |
||||
|
||||
:param normaliseOutput: specifies if (true) output is rescaled between 0 and 255 of not (false) |
||||
:param parasolCells_beta: the low pass filter gain used for local contrast adaptation at the IPL level of the retina (for ganglion cells local adaptation), typical value is 0 |
||||
:param parasolCells_tau: the low pass filter time constant used for local contrast adaptation at the IPL level of the retina (for ganglion cells local adaptation), unit is frame, typical value is 0 (immediate response) |
||||
:param parasolCells_k: the low pass filter spatial constant used for local contrast adaptation at the IPL level of the retina (for ganglion cells local adaptation), unit is pixels, typical value is 5 |
||||
:param amacrinCellsTemporalCutFrequency: the time constant of the first order high pass fiter of the magnocellular way (motion information channel), unit is frames, typical value is 1.2 |
||||
:param V0CompressionParameter: the compression strengh of the ganglion cells local adaptation output, set a value between 0.6 and 1 for best results, a high value increases more the low value sensitivity... and the output saturates faster, recommended value: 0.95 |
||||
:param localAdaptintegration_tau: specifies the temporal constant of the low pas filter involved in the computation of the local "motion mean" for the local adaptation computation |
||||
:param localAdaptintegration_k: specifies the spatial constant of the low pas filter involved in the computation of the local "motion mean" for the local adaptation computation |
||||
|
||||
Retina::setupOPLandIPLParvoChannel |
||||
++++++++++++++++++++++++++++++++++ |
||||
|
||||
.. ocv:function:: void Retina::setupOPLandIPLParvoChannel(const bool colorMode = true, const bool normaliseOutput = true, const float photoreceptorsLocalAdaptationSensitivity = 0.7, const float photoreceptorsTemporalConstant = 0.5, const float photoreceptorsSpatialConstant = 0.53, const float horizontalCellsGain = 0, const float HcellsTemporalConstant = 1, const float HcellsSpatialConstant = 7, const float ganglionCellsSensitivity = 0.7 ) |
||||
|
||||
Setup the OPL and IPL parvo channels (see biologocal model) OPL is referred as Outer Plexiform Layer of the retina, it allows the spatio-temporal filtering which withens the spectrum and reduces spatio-temporal noise while attenuating global luminance (low frequency energy) IPL parvo is the OPL next processing stage, it refers to a part of the Inner Plexiform layer of the retina, it allows high contours sensitivity in foveal vision. See reference papers for more informations. |
||||
|
||||
:param colorMode: specifies if (true) color is processed of not (false) to then processing gray level image |
||||
:param normaliseOutput: specifies if (true) output is rescaled between 0 and 255 of not (false) |
||||
:param photoreceptorsLocalAdaptationSensitivity: the photoreceptors sensitivity renage is 0-1 (more log compression effect when value increases) |
||||
:param photoreceptorsTemporalConstant: the time constant of the first order low pass filter of the photoreceptors, use it to cut high temporal frequencies (noise or fast motion), unit is frames, typical value is 1 frame |
||||
:param photoreceptorsSpatialConstant: the spatial constant of the first order low pass filter of the photoreceptors, use it to cut high spatial frequencies (noise or thick contours), unit is pixels, typical value is 1 pixel |
||||
:param horizontalCellsGain: gain of the horizontal cells network, if 0, then the mean value of the output is zero, if the parameter is near 1, then, the luminance is not filtered and is still reachable at the output, typicall value is 0 |
||||
:param HcellsTemporalConstant: the time constant of the first order low pass filter of the horizontal cells, use it to cut low temporal frequencies (local luminance variations), unit is frames, typical value is 1 frame, as the photoreceptors |
||||
:param HcellsSpatialConstant: the spatial constant of the first order low pass filter of the horizontal cells, use it to cut low spatial frequencies (local luminance), unit is pixels, typical value is 5 pixel, this value is also used for local contrast computing when computing the local contrast adaptation at the ganglion cells level (Inner Plexiform Layer parvocellular channel model) |
||||
:param ganglionCellsSensitivity: the compression strengh of the ganglion cells local adaptation output, set a value between 0.6 and 1 for best results, a high value increases more the low value sensitivity... and the output saturates faster, recommended value: 0.7 |
||||
|
||||
|
||||
Retina::RetinaParameters |
||||
======================== |
||||
|
||||
.. ocv:struct:: Retina::RetinaParameters |
||||
|
||||
This structure merges all the parameters that can be adjusted threw the **Retina::setup()**, **Retina::setupOPLandIPLParvoChannel** and **Retina::setupIPLMagnoChannel** setup methods |
||||
Parameters structure for better clarity, check explenations on the comments of methods : setupOPLandIPLParvoChannel and setupIPLMagnoChannel. :: |
||||
|
||||
class RetinaParameters{ |
||||
struct OPLandIplParvoParameters{ // Outer Plexiform Layer (OPL) and Inner Plexiform Layer Parvocellular (IplParvo) parameters |
||||
OPLandIplParvoParameters():colorMode(true), |
||||
normaliseOutput(true), // specifies if (true) output is rescaled between 0 and 255 of not (false) |
||||
photoreceptorsLocalAdaptationSensitivity(0.7f), // the photoreceptors sensitivity renage is 0-1 (more log compression effect when value increases) |
||||
photoreceptorsTemporalConstant(0.5f),// the time constant of the first order low pass filter of the photoreceptors, use it to cut high temporal frequencies (noise or fast motion), unit is frames, typical value is 1 frame |
||||
photoreceptorsSpatialConstant(0.53f),// the spatial constant of the first order low pass filter of the photoreceptors, use it to cut high spatial frequencies (noise or thick contours), unit is pixels, typical value is 1 pixel |
||||
horizontalCellsGain(0.0f),//gain of the horizontal cells network, if 0, then the mean value of the output is zero, if the parameter is near 1, then, the luminance is not filtered and is still reachable at the output, typicall value is 0 |
||||
hcellsTemporalConstant(1.f),// the time constant of the first order low pass filter of the horizontal cells, use it to cut low temporal frequencies (local luminance variations), unit is frames, typical value is 1 frame, as the photoreceptors. Reduce to 0.5 to limit retina after effects. |
||||
hcellsSpatialConstant(7.f),//the spatial constant of the first order low pass filter of the horizontal cells, use it to cut low spatial frequencies (local luminance), unit is pixels, typical value is 5 pixel, this value is also used for local contrast computing when computing the local contrast adaptation at the ganglion cells level (Inner Plexiform Layer parvocellular channel model) |
||||
ganglionCellsSensitivity(0.7f)//the compression strengh of the ganglion cells local adaptation output, set a value between 0.6 and 1 for best results, a high value increases more the low value sensitivity... and the output saturates faster, recommended value: 0.7 |
||||
{};// default setup |
||||
bool colorMode, normaliseOutput; |
||||
float photoreceptorsLocalAdaptationSensitivity, photoreceptorsTemporalConstant, photoreceptorsSpatialConstant, horizontalCellsGain, hcellsTemporalConstant, hcellsSpatialConstant, ganglionCellsSensitivity; |
||||
}; |
||||
struct IplMagnoParameters{ // Inner Plexiform Layer Magnocellular channel (IplMagno) |
||||
IplMagnoParameters(): |
||||
normaliseOutput(true), //specifies if (true) output is rescaled between 0 and 255 of not (false) |
||||
parasolCells_beta(0.f), // the low pass filter gain used for local contrast adaptation at the IPL level of the retina (for ganglion cells local adaptation), typical value is 0 |
||||
parasolCells_tau(0.f), //the low pass filter time constant used for local contrast adaptation at the IPL level of the retina (for ganglion cells local adaptation), unit is frame, typical value is 0 (immediate response) |
||||
parasolCells_k(7.f), //the low pass filter spatial constant used for local contrast adaptation at the IPL level of the retina (for ganglion cells local adaptation), unit is pixels, typical value is 5 |
||||
amacrinCellsTemporalCutFrequency(1.2f), //the time constant of the first order high pass fiter of the magnocellular way (motion information channel), unit is frames, typical value is 1.2 |
||||
V0CompressionParameter(0.95f), the compression strengh of the ganglion cells local adaptation output, set a value between 0.6 and 1 for best results, a high value increases more the low value sensitivity... and the output saturates faster, recommended value: 0.95 |
||||
localAdaptintegration_tau(0.f), // specifies the temporal constant of the low pas filter involved in the computation of the local "motion mean" for the local adaptation computation |
||||
localAdaptintegration_k(7.f) // specifies the spatial constant of the low pas filter involved in the computation of the local "motion mean" for the local adaptation computation |
||||
{};// default setup |
||||
bool normaliseOutput; |
||||
float parasolCells_beta, parasolCells_tau, parasolCells_k, amacrinCellsTemporalCutFrequency, V0CompressionParameter, localAdaptintegration_tau, localAdaptintegration_k; |
||||
}; |
||||
struct OPLandIplParvoParameters OPLandIplParvo; |
||||
struct IplMagnoParameters IplMagno; |
||||
}; |
||||
|
||||
Retina parameters files examples |
||||
++++++++++++++++++++++++++++++++ |
||||
|
||||
Here is the default configuration file of the retina module. It gives results such as the first retina output shown on the top of this page. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
<?xml version="1.0"?> |
||||
<opencv_storage> |
||||
<OPLandIPLparvo> |
||||
<colorMode>1</colorMode> |
||||
<normaliseOutput>1</normaliseOutput> |
||||
<photoreceptorsLocalAdaptationSensitivity>7.5e-01</photoreceptorsLocalAdaptationSensitivity> |
||||
<photoreceptorsTemporalConstant>9.0e-01</photoreceptorsTemporalConstant> |
||||
<photoreceptorsSpatialConstant>5.3e-01</photoreceptorsSpatialConstant> |
||||
<horizontalCellsGain>0.01</horizontalCellsGain> |
||||
<hcellsTemporalConstant>0.5</hcellsTemporalConstant> |
||||
<hcellsSpatialConstant>7.</hcellsSpatialConstant> |
||||
<ganglionCellsSensitivity>7.5e-01</ganglionCellsSensitivity></OPLandIPLparvo> |
||||
<IPLmagno> |
||||
<normaliseOutput>1</normaliseOutput> |
||||
<parasolCells_beta>0.</parasolCells_beta> |
||||
<parasolCells_tau>0.</parasolCells_tau> |
||||
<parasolCells_k>7.</parasolCells_k> |
||||
<amacrinCellsTemporalCutFrequency>2.0e+00</amacrinCellsTemporalCutFrequency> |
||||
<V0CompressionParameter>9.5e-01</V0CompressionParameter> |
||||
<localAdaptintegration_tau>0.</localAdaptintegration_tau> |
||||
<localAdaptintegration_k>7.</localAdaptintegration_k></IPLmagno> |
||||
</opencv_storage> |
||||
|
||||
Here is the 'realistic" setup used to obtain the second retina output shown on the top of this page. |
||||
|
||||
.. code-block:: cpp |
||||
|
||||
<?xml version="1.0"?> |
||||
<opencv_storage> |
||||
<OPLandIPLparvo> |
||||
<colorMode>1</colorMode> |
||||
<normaliseOutput>1</normaliseOutput> |
||||
<photoreceptorsLocalAdaptationSensitivity>8.9e-01</photoreceptorsLocalAdaptationSensitivity> |
||||
<photoreceptorsTemporalConstant>9.0e-01</photoreceptorsTemporalConstant> |
||||
<photoreceptorsSpatialConstant>5.3e-01</photoreceptorsSpatialConstant> |
||||
<horizontalCellsGain>0.3</horizontalCellsGain> |
||||
<hcellsTemporalConstant>0.5</hcellsTemporalConstant> |
||||
<hcellsSpatialConstant>7.</hcellsSpatialConstant> |
||||
<ganglionCellsSensitivity>8.9e-01</ganglionCellsSensitivity></OPLandIPLparvo> |
||||
<IPLmagno> |
||||
<normaliseOutput>1</normaliseOutput> |
||||
<parasolCells_beta>0.</parasolCells_beta> |
||||
<parasolCells_tau>0.</parasolCells_tau> |
||||
<parasolCells_k>7.</parasolCells_k> |
||||
<amacrinCellsTemporalCutFrequency>2.0e+00</amacrinCellsTemporalCutFrequency> |
||||
<V0CompressionParameter>9.5e-01</V0CompressionParameter> |
||||
<localAdaptintegration_tau>0.</localAdaptintegration_tau> |
||||
<localAdaptintegration_k>7.</localAdaptintegration_k></IPLmagno> |
||||
</opencv_storage> |
||||
|
After Width: | Height: | Size: 147 KiB |
|
After Width: | Height: | Size: 49 KiB |
|
After Width: | Height: | Size: 78 KiB |
|
After Width: | Height: | Size: 28 KiB |
|
After Width: | Height: | Size: 68 KiB |
@ -0,0 +1,476 @@ |
||||
Discovering the human retina and its use for image processing {#tutorial_bioinspired_retina_model} |
||||
============================================================= |
||||
|
||||
Goal |
||||
---- |
||||
|
||||
I present here a model of human retina that shows some interesting properties for image |
||||
preprocessing and enhancement. In this tutorial you will learn how to: |
||||
|
||||
- discover the main two channels outing from your retina |
||||
- see the basics to use the retina model |
||||
- discover some parameters tweaks |
||||
|
||||
General overview |
||||
---------------- |
||||
|
||||
The proposed model originates from Jeanny Herault's research @cite Herault2010 at |
||||
[Gipsa](http://www.gipsa-lab.inpg.fr). It is involved in image processing applications with |
||||
[Listic](http://www.listic.univ-savoie.fr) (code maintainer and user) lab. This is not a complete |
||||
model but it already present interesting properties that can be involved for enhanced image |
||||
processing experience. The model allows the following human retina properties to be used : |
||||
|
||||
- spectral whitening that has 3 important effects: high spatio-temporal frequency signals |
||||
canceling (noise), mid-frequencies details enhancement and low frequencies luminance energy |
||||
reduction. This *all in one* property directly allows visual signals cleaning of classical |
||||
undesired distortions introduced by image sensors and input luminance range. |
||||
- local logarithmic luminance compression allows details to be enhanced even in low light |
||||
conditions. |
||||
- decorrelation of the details information (Parvocellular output channel) and transient |
||||
information (events, motion made available at the Magnocellular output channel). |
||||
|
||||
The first two points are illustrated below : |
||||
|
||||
In the figure below, the OpenEXR image sample *CrissyField.exr*, a High Dynamic Range image is |
||||
shown. In order to make it visible on this web-page, the original input image is linearly rescaled |
||||
to the classical image luminance range [0-255] and is converted to 8bit/channel format. Such strong |
||||
conversion hides many details because of too strong local contrasts. Furthermore, noise energy is |
||||
also strong and pollutes visual information. |
||||
|
||||
 |
||||
|
||||
In the following image, applying the ideas proposed in @cite Benoit2010, as your retina does, local |
||||
luminance adaptation, spatial noise removal and spectral whitening work together and transmit |
||||
accurate information on lower range 8bit data channels. On this picture, noise in significantly |
||||
removed, local details hidden by strong luminance contrasts are enhanced. Output image keeps its |
||||
naturalness and visual content is enhanced. Color processing is based on the color |
||||
multiplexing/demultiplexing method proposed in @cite Chaix2007 . |
||||
|
||||
 |
||||
|
||||
*Note :* image sample can be downloaded from the [OpenEXR website](http://www.openexr.com). |
||||
Regarding this demonstration, before retina processing, input image has been linearly rescaled |
||||
within 0-255 keeping its channels float format. 5% of its histogram ends has been cut (mostly |
||||
removes wrong HDR pixels). Check out the sample |
||||
*opencv/samples/cpp/OpenEXRimages_HighDynamicRange_Retina_toneMapping.cpp* for similar |
||||
processing. The following demonstration will only consider classical 8bit/channel images. |
||||
|
||||
The retina model output channels |
||||
-------------------------------- |
||||
|
||||
The retina model presents two outputs that benefit from the above cited behaviors. |
||||
|
||||
- The first one is called the Parvocellular channel. It is mainly active in the foveal retina area |
||||
(high resolution central vision with color sensitive photo-receptors), its aim is to provide |
||||
accurate color vision for visual details remaining static on the retina. On the other hand |
||||
objects moving on the retina projection are blurred. |
||||
- The second well known channel is the Magnocellular channel. It is mainly active in the retina |
||||
peripheral vision and send signals related to change events (motion, transient events, etc.). |
||||
These outing signals also help visual system to focus/center retina on 'transient'/moving areas |
||||
for more detailed analysis thus improving visual scene context and object classification. |
||||
|
||||
**NOTE :** regarding the proposed model, contrary to the real retina, we apply these two channels on |
||||
the entire input images using the same resolution. This allows enhanced visual details and motion |
||||
information to be extracted on all the considered images... but remember, that these two channels |
||||
are complementary. For example, if Magnocellular channel gives strong energy in an area, then, the |
||||
Parvocellular channel is certainly blurred there since there is a transient event. |
||||
|
||||
As an illustration, we apply in the following the retina model on a webcam video stream of a dark |
||||
visual scene. In this visual scene, captured in an amphitheater of the university, some students are |
||||
moving while talking to the teacher. |
||||
|
||||
In this video sequence, because of the dark ambiance, signal to noise ratio is low and color |
||||
artifacts are present on visual features edges because of the low quality image capture tool-chain. |
||||
|
||||
 |
||||
|
||||
Below is shown the retina foveal vision applied on the entire image. In the used retina |
||||
configuration, global luminance is preserved and local contrasts are enhanced. Also, signal to noise |
||||
ratio is improved : since high frequency spatio-temporal noise is reduced, enhanced details are not |
||||
corrupted by any enhanced noise. |
||||
|
||||
 |
||||
|
||||
Below is the output of the Magnocellular output of the retina model. Its signals are strong where |
||||
transient events occur. Here, a student is moving at the bottom of the image thus generating high |
||||
energy. The remaining of the image is static however, it is corrupted by a strong noise. Here, the |
||||
retina filters out most of the noise thus generating low false motion area 'alarms'. This channel |
||||
can be used as a transient/moving areas detector : it would provide relevant information for a low |
||||
cost segmentation tool that would highlight areas in which an event is occurring. |
||||
|
||||
 |
||||
|
||||
Retina use case |
||||
--------------- |
||||
|
||||
This model can be used basically for spatio-temporal video effects but also in the aim of : |
||||
|
||||
- performing texture analysis with enhanced signal to noise ratio and enhanced details robust |
||||
against input images luminance ranges (check out the Parvocellular retina channel output) |
||||
- performing motion analysis also taking benefit of the previously cited properties. |
||||
|
||||
Literature |
||||
---------- |
||||
|
||||
For more information, refer to the following papers : @cite Benoit2010 |
||||
|
||||
- Please have a look at the reference work of Jeanny Herault that you can read in his book @cite Herault2010 |
||||
|
||||
This retina filter code includes the research contributions of phd/research collegues from which |
||||
code has been redrawn by the author : |
||||
|
||||
- take a look at the *retinacolor.hpp* module to discover Brice Chaix de Lavarene phD color |
||||
mosaicing/demosaicing and his reference paper @cite Chaix2007 |
||||
|
||||
- take a look at *imagelogpolprojection.hpp* to discover retina spatial log sampling which |
||||
originates from Barthelemy Durette phd with Jeanny Herault. A Retina / V1 cortex projection is |
||||
also proposed and originates from Jeanny's discussions. More informations in the above cited |
||||
Jeanny Heraults's book. |
||||
|
||||
Code tutorial |
||||
------------- |
||||
|
||||
Please refer to the original tutorial source code in file |
||||
*opencv_folder/samples/cpp/tutorial_code/bioinspired/retina_tutorial.cpp*. |
||||
|
||||
@note do not forget that the retina model is included in the following namespace: cv::bioinspired |
||||
|
||||
To compile it, assuming OpenCV is correctly installed, use the following command. It requires the |
||||
opencv_core *(cv::Mat and friends objects management)*, opencv_highgui *(display and image/video |
||||
read)* and opencv_bioinspired *(Retina description)* libraries to compile. |
||||
|
||||
@code{.sh} |
||||
// compile |
||||
gcc retina_tutorial.cpp -o Retina_tuto -lopencv_core -lopencv_highgui -lopencv_bioinspired |
||||
|
||||
// Run commands : add 'log' as a last parameter to apply a spatial log sampling (simulates retina sampling) |
||||
// run on webcam |
||||
./Retina_tuto -video |
||||
// run on video file |
||||
./Retina_tuto -video myVideo.avi |
||||
// run on an image |
||||
./Retina_tuto -image myPicture.jpg |
||||
// run on an image with log sampling |
||||
./Retina_tuto -image myPicture.jpg log |
||||
@endcode |
||||
|
||||
Here is a code explanation : |
||||
|
||||
Retina definition is present in the bioinspired package and a simple include allows to use it. You |
||||
can rather use the specific header : *opencv2/bioinspired.hpp* if you prefer but then include the |
||||
other required openv modules : *opencv2/core.hpp* and *opencv2/highgui.hpp* |
||||
@code{.cpp} |
||||
#include "opencv2/opencv.hpp" |
||||
@endcode |
||||
Provide user some hints to run the program with a help function |
||||
@code{.cpp} |
||||
// the help procedure |
||||
static void help(std::string errorMessage) |
||||
{ |
||||
std::cout<<"Program init error : "<<errorMessage<<std::endl; |
||||
std::cout<<"\nProgram call procedure : retinaDemo [processing mode] [Optional : media target] [Optional LAST parameter: \"log\" to activate retina log sampling]"<<std::endl; |
||||
std::cout<<"\t[processing mode] :"<<std::endl; |
||||
std::cout<<"\t -image : for still image processing"<<std::endl; |
||||
std::cout<<"\t -video : for video stream processing"<<std::endl; |
||||
std::cout<<"\t[Optional : media target] :"<<std::endl; |
||||
std::cout<<"\t if processing an image or video file, then, specify the path and filename of the target to process"<<std::endl; |
||||
std::cout<<"\t leave empty if processing video stream coming from a connected video device"<<std::endl; |
||||
std::cout<<"\t[Optional : activate retina log sampling] : an optional last parameter can be specified for retina spatial log sampling"<<std::endl; |
||||
std::cout<<"\t set \"log\" without quotes to activate this sampling, output frame size will be divided by 4"<<std::endl; |
||||
std::cout<<"\nExamples:"<<std::endl; |
||||
std::cout<<"\t-Image processing : ./retinaDemo -image lena.jpg"<<std::endl; |
||||
std::cout<<"\t-Image processing with log sampling : ./retinaDemo -image lena.jpg log"<<std::endl; |
||||
std::cout<<"\t-Video processing : ./retinaDemo -video myMovie.mp4"<<std::endl; |
||||
std::cout<<"\t-Live video processing : ./retinaDemo -video"<<std::endl; |
||||
std::cout<<"\nPlease start again with new parameters"<<std::endl; |
||||
std::cout<<"****************************************************"<<std::endl; |
||||
std::cout<<" NOTE : this program generates the default retina parameters file 'RetinaDefaultParameters.xml'"<<std::endl; |
||||
std::cout<<" => you can use this to fine tune parameters and load them if you save to file 'RetinaSpecificParameters.xml'"<<std::endl; |
||||
} |
||||
@endcode |
||||
Then, start the main program and first declare a *cv::Mat* matrix in which input images will be |
||||
loaded. Also allocate a *cv::VideoCapture* object ready to load video streams (if necessary) |
||||
@code{.cpp} |
||||
int main(int argc, char* argv[]) { |
||||
// declare the retina input buffer... that will be fed differently in regard of the input media |
||||
cv::Mat inputFrame; |
||||
cv::VideoCapture videoCapture; // in case a video media is used, its manager is declared here |
||||
@endcode |
||||
In the main program, before processing, first check input command parameters. Here it loads a first |
||||
input image coming from a single loaded image (if user chose command *-image*) or from a video |
||||
stream (if user chose command *-video*). Also, if the user added *log* command at the end of its |
||||
program call, the spatial logarithmic image sampling performed by the retina is taken into account |
||||
by the Boolean flag *useLogSampling*. |
||||
@code{.cpp} |
||||
// welcome message |
||||
std::cout<<"****************************************************"<<std::endl; |
||||
std::cout<<"* Retina demonstration : demonstrates the use of is a wrapper class of the Gipsa/Listic Labs retina model."<<std::endl; |
||||
std::cout<<"* This demo will try to load the file 'RetinaSpecificParameters.xml' (if exists).\nTo create it, copy the autogenerated template 'RetinaDefaultParameters.xml'.\nThen twaek it with your own retina parameters."<<std::endl; |
||||
// basic input arguments checking |
||||
if (argc<2) |
||||
{ |
||||
help("bad number of parameter"); |
||||
return -1; |
||||
} |
||||
|
||||
bool useLogSampling = !strcmp(argv[argc-1], "log"); // check if user wants retina log sampling processing |
||||
|
||||
std::string inputMediaType=argv[1]; |
||||
|
||||
////////////////////////////////////////////////////////////////////////////// |
||||
// checking input media type (still image, video file, live video acquisition) |
||||
if (!strcmp(inputMediaType.c_str(), "-image") && argc >= 3) |
||||
{ |
||||
std::cout<<"RetinaDemo: processing image "<<argv[2]<<std::endl; |
||||
// image processing case |
||||
inputFrame = cv::imread(std::string(argv[2]), 1); // load image in RGB mode |
||||
}else |
||||
if (!strcmp(inputMediaType.c_str(), "-video")) |
||||
{ |
||||
if (argc == 2 || (argc == 3 && useLogSampling)) // attempt to grab images from a video capture device |
||||
{ |
||||
videoCapture.open(0); |
||||
}else// attempt to grab images from a video filestream |
||||
{ |
||||
std::cout<<"RetinaDemo: processing video stream "<<argv[2]<<std::endl; |
||||
videoCapture.open(argv[2]); |
||||
} |
||||
|
||||
// grab a first frame to check if everything is ok |
||||
videoCapture>>inputFrame; |
||||
}else |
||||
{ |
||||
// bad command parameter |
||||
help("bad command parameter"); |
||||
return -1; |
||||
} |
||||
@endcode |
||||
Once all input parameters are processed, a first image should have been loaded, if not, display |
||||
error and stop program : |
||||
@code{.cpp} |
||||
if (inputFrame.empty()) |
||||
{ |
||||
help("Input media could not be loaded, aborting"); |
||||
return -1; |
||||
} |
||||
@endcode |
||||
Now, everything is ready to run the retina model. I propose here to allocate a retina instance and |
||||
to manage the eventual log sampling option. The Retina constructor expects at least a cv::Size |
||||
object that shows the input data size that will have to be managed. One can activate other options |
||||
such as color and its related color multiplexing strategy (here Bayer multiplexing is chosen using |
||||
*enum cv::bioinspired::RETINA_COLOR_BAYER*). If using log sampling, the image reduction factor |
||||
(smaller output images) and log sampling strengh can be adjusted. |
||||
@code{.cpp} |
||||
// pointer to a retina object |
||||
cv::Ptr<cv::bioinspired::Retina> myRetina; |
||||
|
||||
// if the last parameter is 'log', then activate log sampling (favour foveal vision and subsamples peripheral vision) |
||||
if (useLogSampling) |
||||
{ |
||||
myRetina = cv::bioinspired::createRetina(inputFrame.size(), true, cv::bioinspired::RETINA_COLOR_BAYER, true, 2.0, 10.0); |
||||
} |
||||
else// -> else allocate "classical" retina : |
||||
myRetina = cv::bioinspired::createRetina(inputFrame.size()); |
||||
@endcode |
||||
Once done, the proposed code writes a default xml file that contains the default parameters of the |
||||
retina. This is useful to make your own config using this template. Here generated template xml file |
||||
is called *RetinaDefaultParameters.xml*. |
||||
@code{.cpp} |
||||
// save default retina parameters file in order to let you see this and maybe modify it and reload using method "setup" |
||||
myRetina->write("RetinaDefaultParameters.xml"); |
||||
@endcode |
||||
In the following line, the retina attempts to load another xml file called |
||||
*RetinaSpecificParameters.xml*. If you created it and introduced your own setup, it will be loaded, |
||||
in the other case, default retina parameters are used. |
||||
@code{.cpp} |
||||
// load parameters if file exists |
||||
myRetina->setup("RetinaSpecificParameters.xml"); |
||||
@endcode |
||||
It is not required here but just to show it is possible, you can reset the retina buffers to zero to |
||||
force it to forget past events. |
||||
@code{.cpp} |
||||
// reset all retina buffers (imagine you close your eyes for a long time) |
||||
myRetina->clearBuffers(); |
||||
@endcode |
||||
Now, it is time to run the retina ! First create some output buffers ready to receive the two retina |
||||
channels outputs |
||||
@code{.cpp} |
||||
// declare retina output buffers |
||||
cv::Mat retinaOutput_parvo; |
||||
cv::Mat retinaOutput_magno; |
||||
@endcode |
||||
Then, run retina in a loop, load new frames from video sequence if necessary and get retina outputs |
||||
back to dedicated buffers. |
||||
@code{.cpp} |
||||
// processing loop with no stop condition |
||||
while(true) |
||||
{ |
||||
// if using video stream, then, grabbing a new frame, else, input remains the same |
||||
if (videoCapture.isOpened()) |
||||
videoCapture>>inputFrame; |
||||
|
||||
// run retina filter on the loaded input frame |
||||
myRetina->run(inputFrame); |
||||
// Retrieve and display retina output |
||||
myRetina->getParvo(retinaOutput_parvo); |
||||
myRetina->getMagno(retinaOutput_magno); |
||||
cv::imshow("retina input", inputFrame); |
||||
cv::imshow("Retina Parvo", retinaOutput_parvo); |
||||
cv::imshow("Retina Magno", retinaOutput_magno); |
||||
cv::waitKey(10); |
||||
} |
||||
@endcode |
||||
That's done ! But if you want to secure the system, take care and manage Exceptions. The retina can |
||||
throw some when it sees irrelevant data (no input frame, wrong setup, etc.). Then, i recommend to |
||||
surround all the retina code by a try/catch system like this : |
||||
@code{.cpp} |
||||
try{ |
||||
// pointer to a retina object |
||||
cv::Ptr<cv::Retina> myRetina; |
||||
[---] |
||||
// processing loop with no stop condition |
||||
while(true) |
||||
{ |
||||
[---] |
||||
} |
||||
|
||||
}catch(cv::Exception e) |
||||
{ |
||||
std::cerr<<"Error using Retina : "<<e.what()<<std::endl; |
||||
} |
||||
@endcode |
||||
|
||||
Retina parameters, what to do ? |
||||
------------------------------- |
||||
|
||||
First, it is recommended to read the reference paper @cite Benoit2010 |
||||
|
||||
Once done open the configuration file *RetinaDefaultParameters.xml* generated by the demo and let's |
||||
have a look at it. |
||||
@code{.cpp} |
||||
<?xml version="1.0"?> |
||||
<opencv_storage> |
||||
<OPLandIPLparvo> |
||||
<colorMode>1</colorMode> |
||||
<normaliseOutput>1</normaliseOutput> |
||||
<photoreceptorsLocalAdaptationSensitivity>7.5e-01</photoreceptorsLocalAdaptationSensitivity> |
||||
<photoreceptorsTemporalConstant>9.0e-01</photoreceptorsTemporalConstant> |
||||
<photoreceptorsSpatialConstant>5.7e-01</photoreceptorsSpatialConstant> |
||||
<horizontalCellsGain>0.01</horizontalCellsGain> |
||||
<hcellsTemporalConstant>0.5</hcellsTemporalConstant> |
||||
<hcellsSpatialConstant>7.</hcellsSpatialConstant> |
||||
<ganglionCellsSensitivity>7.5e-01</ganglionCellsSensitivity></OPLandIPLparvo> |
||||
<IPLmagno> |
||||
<normaliseOutput>1</normaliseOutput> |
||||
<parasolCells_beta>0.</parasolCells_beta> |
||||
<parasolCells_tau>0.</parasolCells_tau> |
||||
<parasolCells_k>7.</parasolCells_k> |
||||
<amacrinCellsTemporalCutFrequency>2.0e+00</amacrinCellsTemporalCutFrequency> |
||||
<V0CompressionParameter>9.5e-01</V0CompressionParameter> |
||||
<localAdaptintegration_tau>0.</localAdaptintegration_tau> |
||||
<localAdaptintegration_k>7.</localAdaptintegration_k></IPLmagno> |
||||
</opencv_storage> |
||||
@endcode |
||||
Here are some hints but actually, the best parameter setup depends more on what you want to do with |
||||
the retina rather than the images input that you give to retina. Apart from the more specific case |
||||
of High Dynamic Range images (HDR) that require more specific setup for specific luminance |
||||
compression objective, the retina behaviors should be rather stable from content to content. Note |
||||
that OpenCV is able to manage such HDR format thanks to the OpenEXR images compatibility. |
||||
|
||||
Then, if the application target requires details enhancement prior to specific image processing, you |
||||
need to know if mean luminance information is required or not. If not, the the retina can cancel or |
||||
significantly reduce its energy thus giving more visibility to higher spatial frequency details. |
||||
|
||||
### Basic parameters |
||||
|
||||
The most simple parameters are the following : |
||||
|
||||
- **colorMode** : let the retina process color information (if 1) or gray scale images (if 0). In |
||||
this last case, only the first channel of the input will be processed. |
||||
- **normaliseOutput** : each channel has this parameter, if value is 1, then the considered |
||||
channel output is rescaled between 0 and 255. Take care in this case at the Magnocellular output |
||||
level (motion/transient channel detection). Residual noise will also be rescaled ! |
||||
|
||||
**Note :** using color requires color channels multiplexing/demultipexing which requires more |
||||
processing. You can expect much faster processing using gray levels : it would require around 30 |
||||
product per pixel for all the retina processes and it has recently been parallelized for multicore |
||||
architectures. |
||||
|
||||
### Photo-receptors parameters |
||||
|
||||
The following parameters act on the entry point of the retina - photo-receptors - and impact all the |
||||
following processes. These sensors are low pass spatio-temporal filters that smooth temporal and |
||||
spatial data and also adjust there sensitivity to local luminance thus improving details extraction |
||||
and high frequency noise canceling. |
||||
|
||||
- **photoreceptorsLocalAdaptationSensitivity** between 0 and 1. Values close to 1 allow high |
||||
luminance log compression effect at the photo-receptors level. Values closer to 0 give a more |
||||
linear sensitivity. Increased alone, it can burn the *Parvo (details channel)* output image. If |
||||
adjusted in collaboration with **ganglionCellsSensitivity** images can be very contrasted |
||||
whatever the local luminance there is... at the price of a naturalness decrease. |
||||
- **photoreceptorsTemporalConstant** this setups the temporal constant of the low pass filter |
||||
effect at the entry of the retina. High value lead to strong temporal smoothing effect : moving |
||||
objects are blurred and can disappear while static object are favored. But when starting the |
||||
retina processing, stable state is reached lately. |
||||
- **photoreceptorsSpatialConstant** specifies the spatial constant related to photo-receptors low |
||||
pass filter effect. This parameters specify the minimum allowed spatial signal period allowed in |
||||
the following. Typically, this filter should cut high frequency noise. Then a 0 value doesn't |
||||
cut anything noise while higher values start to cut high spatial frequencies and more and more |
||||
lower frequencies... Then, do not go to high if you wanna see some details of the input images ! |
||||
A good compromise for color images is 0.53 since this won't affect too much the color spectrum. |
||||
Higher values would lead to gray and blurred output images. |
||||
|
||||
### Horizontal cells parameters |
||||
|
||||
This parameter set tunes the neural network connected to the photo-receptors, the horizontal cells. |
||||
It modulates photo-receptors sensitivity and completes the processing for final spectral whitening |
||||
(part of the spatial band pass effect thus favoring visual details enhancement). |
||||
|
||||
- **horizontalCellsGain** here is a critical parameter ! If you are not interested by the mean |
||||
luminance and focus on details enhancement, then, set to zero. But if you want to keep some |
||||
environment luminance data, let some low spatial frequencies pass into the system and set a |
||||
higher value (\<1). |
||||
- **hcellsTemporalConstant** similar to photo-receptors, this acts on the temporal constant of a |
||||
low pass temporal filter that smooths input data. Here, a high value generates a high retina |
||||
after effect while a lower value makes the retina more reactive. This value should be lower than |
||||
**photoreceptorsTemporalConstant** to limit strong retina after effects. |
||||
- **hcellsSpatialConstant** is the spatial constant of the low pass filter of these cells filter. |
||||
It specifies the lowest spatial frequency allowed in the following. Visually, a high value leads |
||||
to very low spatial frequencies processing and leads to salient halo effects. Lower values |
||||
reduce this effect but the limit is : do not go lower than the value of |
||||
**photoreceptorsSpatialConstant**. Those 2 parameters actually specify the spatial band-pass of |
||||
the retina. |
||||
|
||||
**NOTE** after the processing managed by the previous parameters, input data is cleaned from noise |
||||
and luminance in already partly enhanced. The following parameters act on the last processing stages |
||||
of the two outing retina signals. |
||||
|
||||
### Parvo (details channel) dedicated parameter |
||||
|
||||
- **ganglionCellsSensitivity** specifies the strength of the final local adaptation occurring at |
||||
the output of this details dedicated channel. Parameter values remain between 0 and 1. Low value |
||||
tend to give a linear response while higher values enforces the remaining low contrasted areas. |
||||
|
||||
**Note :** this parameter can correct eventual burned images by favoring low energetic details of |
||||
the visual scene, even in bright areas. |
||||
|
||||
### IPL Magno (motion/transient channel) parameters |
||||
|
||||
Once image information is cleaned, this channel acts as a high pass temporal filter that only |
||||
selects signals related to transient signals (events, motion, etc.). A low pass spatial filter |
||||
smooths extracted transient data and a final logarithmic compression enhances low transient events |
||||
thus enhancing event sensitivity. |
||||
|
||||
- **parasolCells_beta** generally set to zero, can be considered as an amplifier gain at the |
||||
entry point of this processing stage. Generally set to 0. |
||||
- **parasolCells_tau** the temporal smoothing effect that can be added |
||||
- **parasolCells_k** the spatial constant of the spatial filtering effect, set it at a high value |
||||
to favor low spatial frequency signals that are lower subject to residual noise. |
||||
- **amacrinCellsTemporalCutFrequency** specifies the temporal constant of the high pass filter. |
||||
High values let slow transient events to be selected. |
||||
- **V0CompressionParameter** specifies the strength of the log compression. Similar behaviors to |
||||
previous description but here it enforces sensitivity of transient events. |
||||
- **localAdaptintegration_tau** generally set to 0, no real use here actually |
||||
- **localAdaptintegration_k** specifies the size of the area on which local adaptation is |
||||
performed. Low values lead to short range local adaptation (higher sensitivity to noise), high |
||||
values secure log compression. |
||||
@ -1,2 +1,2 @@ |
||||
set(the_description "Custom Calibration Pattern") |
||||
ocv_define_module(ccalib opencv_core opencv_imgproc opencv_calib3d opencv_features2d) |
||||
ocv_define_module(ccalib opencv_core opencv_imgproc opencv_calib3d opencv_features2d WRAP python) |
||||
|
||||
@ -1,202 +0,0 @@ |
||||
Custom Calibration Pattern |
||||
========================== |
||||
|
||||
.. highlight:: cpp |
||||
|
||||
CustomPattern |
||||
------------- |
||||
A custom pattern class that can be used to calibrate a camera and to further track the translation and rotation of the pattern. Defaultly it uses an ``ORB`` feature detector and a ``BruteForce-Hamming(2)`` descriptor matcher to find the location of the pattern feature points that will subsequently be used for calibration. |
||||
|
||||
.. ocv:class:: CustomPattern : public Algorithm |
||||
|
||||
|
||||
CustomPattern::CustomPattern |
||||
---------------------------- |
||||
CustomPattern constructor. |
||||
|
||||
.. ocv:function:: CustomPattern() |
||||
|
||||
|
||||
CustomPattern::create |
||||
--------------------- |
||||
A method that initializes the class and generates the necessary detectors, extractors and matchers. |
||||
|
||||
.. ocv:function:: bool create(InputArray pattern, const Size2f boardSize, OutputArray output = noArray()) |
||||
|
||||
:param pattern: The image, which will be used as a pattern. If the desired pattern is part of a bigger image, you can crop it out using image(roi). |
||||
|
||||
:param boardSize: The size of the pattern in physical dimensions. These will be used to scale the points when the calibration occurs. |
||||
|
||||
:param output: A matrix that is the same as the input pattern image, but has all the feature points drawn on it. |
||||
|
||||
:return Returns whether the initialization was successful or not. Possible reason for failure may be that no feature points were detected. |
||||
|
||||
.. seealso:: |
||||
|
||||
:ocv:func:`getFeatureDetector`, |
||||
:ocv:func:`getDescriptorExtractor`, |
||||
:ocv:func:`getDescriptorMatcher` |
||||
|
||||
.. note:: |
||||
|
||||
* Determine the number of detected feature points can be done through :ocv:func:`getPatternPoints` method. |
||||
|
||||
* The feature detector, extractor and matcher cannot be changed after initialization. |
||||
|
||||
|
||||
|
||||
CustomPattern::findPattern |
||||
-------------------------- |
||||
Finds the pattern in the input image |
||||
|
||||
.. ocv:function:: bool findPattern(InputArray image, OutputArray matched_features, OutputArray pattern_points, const double ratio = 0.7, const double proj_error = 8.0, const bool refine_position = false, OutputArray out = noArray(), OutputArray H = noArray(), OutputArray pattern_corners = noArray()); |
||||
|
||||
:param image: The input image where the pattern is searched for. |
||||
|
||||
:param matched_features: A ``vector<Point2f>`` of the projections of calibration pattern points, matched in the image. The points correspond to the ``pattern_points``.``matched_features`` and ``pattern_points`` have the same size. |
||||
|
||||
:param pattern_points: A ``vector<Point3f>`` of calibration pattern points in the calibration pattern coordinate space. |
||||
|
||||
:param ratio: A ratio used to threshold matches based on D. Lowe's point ratio test. |
||||
|
||||
:param proj_error: The maximum projection error that is allowed when the found points are back projected. A lower projection error will be beneficial for eliminating mismatches. Higher values are recommended when the camera lens has greater distortions. |
||||
|
||||
:param refine_position: Whether to refine the position of the feature points with :ocv:func:`cornerSubPix`. |
||||
|
||||
:param out: An image showing the matched feature points and a contour around the estimated pattern. |
||||
|
||||
:param H: The homography transformation matrix between the pattern and the current image. |
||||
|
||||
:param pattern_corners: A ``vector<Point2f>`` containing the 4 corners of the found pattern. |
||||
|
||||
:return The method return whether the pattern was found or not. |
||||
|
||||
|
||||
CustomPattern::isInitialized |
||||
---------------------------- |
||||
|
||||
.. ocv:function:: bool isInitialized() |
||||
|
||||
:return If the class is initialized or not. |
||||
|
||||
|
||||
CustomPattern::getPatternPoints |
||||
------------------------------- |
||||
|
||||
.. ocv:function:: void getPatternPoints(OutputArray original_points) |
||||
|
||||
:param original_points: Fills the vector with the points found in the pattern. |
||||
|
||||
|
||||
CustomPattern::getPixelSize |
||||
--------------------------- |
||||
.. ocv:function:: double getPixelSize() |
||||
|
||||
:return Get the physical pixel size as initialized by the pattern. |
||||
|
||||
|
||||
CustomPattern::setFeatureDetector |
||||
--------------------------------- |
||||
.. ocv:function:: bool setFeatureDetector(Ptr<FeatureDetector> featureDetector) |
||||
|
||||
:param featureDetector: Set a new FeatureDetector. |
||||
|
||||
:return Is it successfully set? Will fail if the object is already initialized by :ocv:func:`create`. |
||||
|
||||
.. note:: |
||||
|
||||
* It is left to user discretion to select matching feature detector, extractor and matchers. Please consult the documentation for each to confirm coherence. |
||||
|
||||
|
||||
CustomPattern::setDescriptorExtractor |
||||
------------------------------------- |
||||
.. ocv:function:: bool setDescriptorExtractor(Ptr<DescriptorExtractor> extractor) |
||||
|
||||
:param extractor: Set a new DescriptorExtractor. |
||||
|
||||
:return Is it successfully set? Will fail if the object is already initialized by :ocv:func:`create`. |
||||
|
||||
|
||||
CustomPattern::setDescriptorMatcher |
||||
----------------------------------- |
||||
.. ocv:function:: bool setDescriptorMatcher(Ptr<DescriptorMatcher> matcher) |
||||
|
||||
:param matcher: Set a new DescriptorMatcher. |
||||
|
||||
:return Is it successfully set? Will fail if the object is already initialized by :ocv:func:`create`. |
||||
|
||||
|
||||
CustomPattern::getFeatureDetector |
||||
--------------------------------- |
||||
.. ocv:function:: Ptr<FeatureDetector> getFeatureDetector() |
||||
|
||||
:return The used FeatureDetector. |
||||
|
||||
|
||||
CustomPattern::getDescriptorExtractor |
||||
------------------------------------- |
||||
.. ocv:function:: Ptr<DescriptorExtractor> getDescriptorExtractor() |
||||
|
||||
:return The used DescriptorExtractor. |
||||
|
||||
|
||||
CustomPattern::getDescriptorMatcher |
||||
----------------------------------- |
||||
.. ocv:function:: Ptr<DescriptorMatcher> getDescriptorMatcher() |
||||
|
||||
:return The used DescriptorMatcher. |
||||
|
||||
|
||||
CustomPattern::calibrate |
||||
------------------------ |
||||
Calibrates the camera. |
||||
|
||||
.. ocv:function:: double calibrate(InputArrayOfArrays objectPoints, InputArrayOfArrays imagePoints, Size imageSize, InputOutputArray cameraMatrix, InputOutputArray distCoeffs, OutputArrayOfArrays rvecs, OutputArrayOfArrays tvecs, int flags = 0, TermCriteria criteria = TermCriteria(TermCriteria::COUNT + TermCriteria::EPS, 30, DBL_EPSILON)) |
||||
|
||||
See :ocv:func:`calibrateCamera` for parameter information. |
||||
|
||||
|
||||
CustomPattern::findRt |
||||
--------------------- |
||||
Finds the rotation and translation vectors of the pattern. |
||||
|
||||
.. ocv:function:: bool findRt(InputArray objectPoints, InputArray imagePoints, InputArray cameraMatrix, InputArray distCoeffs, OutputArray rvec, OutputArray tvec, bool useExtrinsicGuess = false, int flags = ITERATIVE) |
||||
.. ocv:function:: bool findRt(InputArray image, InputArray cameraMatrix, InputArray distCoeffs, OutputArray rvec, OutputArray tvec, bool useExtrinsicGuess = false, int flags = ITERATIVE) |
||||
|
||||
:param image: The image, in which the rotation and translation of the pattern will be found. |
||||
|
||||
See :ocv:func:`solvePnP` for parameter information. |
||||
|
||||
|
||||
CustomPattern::findRtRANSAC |
||||
--------------------------- |
||||
Finds the rotation and translation vectors of the pattern using RANSAC. |
||||
|
||||
.. ocv:function:: bool findRtRANSAC(InputArray objectPoints, InputArray imagePoints, InputArray cameraMatrix, InputArray distCoeffs, OutputArray rvec, OutputArray tvec, bool useExtrinsicGuess = false, int iterationsCount = 100, float reprojectionError = 8.0, int minInliersCount = 100, OutputArray inliers = noArray(), int flags = ITERATIVE) |
||||
.. ocv:function:: bool findRtRANSAC(InputArray image, InputArray cameraMatrix, InputArray distCoeffs, OutputArray rvec, OutputArray tvec, bool useExtrinsicGuess = false, int iterationsCount = 100, float reprojectionError = 8.0, int minInliersCount = 100, OutputArray inliers = noArray(), int flags = ITERATIVE) |
||||
|
||||
:param image: The image, in which the rotation and translation of the pattern will be found. |
||||
|
||||
See :ocv:func:`solvePnPRANSAC` for parameter information. |
||||
|
||||
|
||||
CustomPattern::drawOrientation |
||||
------------------------------ |
||||
Draws the ``(x,y,z)`` axis on the image, in the center of the pattern, showing the orientation of the pattern. |
||||
|
||||
.. ocv:function:: void drawOrientation(InputOutputArray image, InputArray tvec, InputArray rvec, InputArray cameraMatrix, InputArray distCoeffs, double axis_length = 3, int axis_width = 2) |
||||
|
||||
:param image: The image, based on which the rotation and translation was calculated. The axis will be drawn in color - ``x`` - in red, ``y`` - in green, ``z`` - in blue. |
||||
|
||||
:param tvec: Translation vector. |
||||
|
||||
:param rvec: Rotation vector. |
||||
|
||||
:param cameraMatrix: The camera matrix. |
||||
|
||||
:param distCoeffs: The distortion coefficients. |
||||
|
||||
:param axis_length: The length of the axis symbol. |
||||
|
||||
:param axis_width: The width of the axis symbol. |
||||
|
||||
@ -0,0 +1,43 @@ |
||||
set(the_description "Separate world module containing all contrib modules") |
||||
|
||||
set(OPENCV_MODULE_IS_PART_OF_WORLD FALSE) |
||||
set(BUILD_opencv_contrib_world_INIT OFF) # disabled by default |
||||
|
||||
# add new submodules to this list |
||||
set(OPENCV_MODULE_CHILDREN |
||||
bgsegm |
||||
bioinspired |
||||
ccalib |
||||
cvv |
||||
datasets |
||||
face |
||||
latentsvm |
||||
line_descriptor |
||||
optflow |
||||
reg |
||||
rgbd |
||||
saliency |
||||
surface_matching |
||||
text |
||||
tracking |
||||
xfeatures2d |
||||
ximgproc |
||||
xobjdetect |
||||
xphoto |
||||
) |
||||
ocv_list_add_prefix(OPENCV_MODULE_CHILDREN "opencv_") |
||||
|
||||
ocv_define_module(contrib_world) |
||||
|
||||
# ocv_add_module(contrib_world) |
||||
|
||||
# set(link_deps "") |
||||
# foreach(m ${OPENCV_MODULE_opencv_contrib_world_CHILDREN}) |
||||
# list(APPEND link_deps ${OPENCV_MODULE_${m}_LINK_DEPS}) |
||||
# endforeach() |
||||
|
||||
# |
||||
# ocv_glob_module_sources() |
||||
# ocv_module_include_directories() |
||||
# |
||||
# ocv_create_module(${link_deps}) |
||||
@ -0,0 +1,5 @@ |
||||
#ifndef __OPENCV_CONTRIB_WORLD_HPP__ |
||||
#define __OPENCV_CONTRIB_WORLD_HPP__ |
||||
|
||||
|
||||
#endif |
||||
@ -0,0 +1 @@ |
||||
#include "opencv2/contrib_world.hpp" |
||||
@ -1,11 +0,0 @@ |
||||
********************************************************************* |
||||
cvv. GUI for Interactive Visual Debugging of Computer Vision Programs |
||||
********************************************************************* |
||||
|
||||
The module provides an interactive GUI to debug and incrementally design computer vision algorithms. The debug statements can remain in the code after development and aid in further changes because they have neglectable overhead if the program is compiled in release mode. |
||||
|
||||
.. toctree:: |
||||
:maxdepth: 2 |
||||
|
||||
CVV API Documentation <cvv_api> |
||||
CVV GUI Documentation <cvv_gui> |
||||
@ -1,85 +0,0 @@ |
||||
CVV : the API |
||||
************* |
||||
|
||||
.. highlight:: cpp |
||||
|
||||
|
||||
Introduction |
||||
++++++++++++ |
||||
|
||||
Namespace for all functions is **cvv**, i.e. *cvv::showImage()*. |
||||
|
||||
Compilation: |
||||
|
||||
* For development, i.e. for cvv GUI to show up, compile your code using cvv with *g++ -DCVVISUAL_DEBUGMODE*. |
||||
* For release, i.e. cvv calls doing nothing, compile your code without above flag. |
||||
|
||||
See cvv tutorial for a commented example application using cvv. |
||||
|
||||
|
||||
|
||||
|
||||
API Functions |
||||
+++++++++++++ |
||||
|
||||
|
||||
showImage |
||||
--------- |
||||
Add a single image to debug GUI (similar to :imshow:`imshow <>`). |
||||
|
||||
.. ocv:function:: void showImage(InputArray img, const CallMetaData& metaData, const string& description, const string& view) |
||||
|
||||
:param img: Image to show in debug GUI. |
||||
:param metaData: Properly initialized CallMetaData struct, i.e. information about file, line and function name for GUI. Use CVVISUAL_LOCATION macro. |
||||
:param description: Human readable description to provide context to image. |
||||
:param view: Preselect view that will be used to visualize this image in GUI. Other views can still be selected in GUI later on. |
||||
|
||||
|
||||
|
||||
debugFilter |
||||
----------- |
||||
Add two images to debug GUI for comparison. Usually the input and output of some filter operation, whose result should be inspected. |
||||
|
||||
.. ocv:function:: void debugFilter(InputArray original, InputArray result, const CallMetaData& metaData, const string& description, const string& view) |
||||
|
||||
:param original: First image for comparison, e.g. filter input. |
||||
:param result: Second image for comparison, e.g. filter output. |
||||
:param metaData: See :ocv:func:`showImage` |
||||
:param description: See :ocv:func:`showImage` |
||||
:param view: See :ocv:func:`showImage` |
||||
|
||||
|
||||
|
||||
debugDMatch |
||||
----------- |
||||
Add a filled in :basicstructures:`DMatch <dmatch>` to debug GUI. The matches can are visualized for interactive inspection in different GUI views (one similar to an interactive :draw_matches:`drawMatches<>`). |
||||
|
||||
.. ocv:function:: void debugDMatch(InputArray img1, std::vector<cv::KeyPoint> keypoints1, InputArray img2, std::vector<cv::KeyPoint> keypoints2, std::vector<cv::DMatch> matches, const CallMetaData& metaData, const string& description, const string& view, bool useTrainDescriptor) |
||||
|
||||
:param img1: First image used in :basicstructures:`DMatch <dmatch>`. |
||||
:param keypoints1: Keypoints of first image. |
||||
:param img2: Second image used in DMatch. |
||||
:param keypoints2: Keypoints of second image. |
||||
:param metaData: See :ocv:func:`showImage` |
||||
:param description: See :ocv:func:`showImage` |
||||
:param view: See :ocv:func:`showImage` |
||||
:param useTrainDescriptor: Use :basicstructures:`DMatch <dmatch>`'s train descriptor index instead of query descriptor index. |
||||
|
||||
|
||||
|
||||
finalShow |
||||
--------- |
||||
This function **must** be called *once* *after* all cvv calls if any. |
||||
As an alternative create an instance of FinalShowCaller, which calls finalShow() in its destructor (RAII-style). |
||||
|
||||
.. ocv:function:: void finalShow() |
||||
|
||||
|
||||
|
||||
setDebugFlag |
||||
------------ |
||||
Enable or disable cvv for current translation unit and thread (disabled this way has higher - but still low - overhead compared to using the compile flags). |
||||
|
||||
.. ocv:function:: void setDebugFlag(bool active) |
||||
|
||||
:param active: See above |
||||
@ -1,24 +0,0 @@ |
||||
CVV : the GUI |
||||
************* |
||||
|
||||
.. highlight:: cpp |
||||
|
||||
|
||||
Introduction |
||||
++++++++++++ |
||||
|
||||
For now: See cvv tutorial. |
||||
|
||||
|
||||
|
||||
Overview |
||||
++++++++ |
||||
|
||||
|
||||
Filter |
||||
------ |
||||
|
||||
|
||||
|
||||
Views |
||||
++++++++ |
||||
@ -0,0 +1,26 @@ |
||||
#ifndef __OPENCV_CVV_HPP__ |
||||
#define __OPENCV_CVV_HPP__ |
||||
|
||||
/**
|
||||
@defgroup cvv GUI for Interactive Visual Debugging of Computer Vision Programs |
||||
|
||||
Namespace for all functions is **cvv**, i.e. *cvv::showImage()*. |
||||
|
||||
Compilation: |
||||
|
||||
- For development, i.e. for cvv GUI to show up, compile your code using cvv with |
||||
*g++ -DCVVISUAL_DEBUGMODE*. |
||||
- For release, i.e. cvv calls doing nothing, compile your code without above flag. |
||||
|
||||
See cvv tutorial for a commented example application using cvv. |
||||
|
||||
*/ |
||||
|
||||
#include <opencv2/cvv/call_meta_data.hpp> |
||||
#include <opencv2/cvv/debug_mode.hpp> |
||||
#include <opencv2/cvv/dmatch.hpp> |
||||
#include <opencv2/cvv/filter.hpp> |
||||
#include <opencv2/cvv/final_show.hpp> |
||||
#include <opencv2/cvv/show_image.hpp> |
||||
|
||||
#endif //__OPENCV_CVV_HPP__
|
||||
@ -1,22 +1,5 @@ |
||||
/**
|
||||
@defgroup cvv GUI for Interactive Visual Debugging of Computer Vision Programs |
||||
|
||||
Namespace for all functions is **cvv**, i.e. *cvv::showImage()*. |
||||
|
||||
Compilation: |
||||
|
||||
- For development, i.e. for cvv GUI to show up, compile your code using cvv with |
||||
*g++ -DCVVISUAL_DEBUGMODE*. |
||||
- For release, i.e. cvv calls doing nothing, compile your code without above flag. |
||||
|
||||
See cvv tutorial for a commented example application using cvv. |
||||
|
||||
*/ |
||||
|
||||
#include <opencv2/cvv/call_meta_data.hpp> |
||||
#include <opencv2/cvv/debug_mode.hpp> |
||||
#include <opencv2/cvv/dmatch.hpp> |
||||
#include <opencv2/cvv/filter.hpp> |
||||
#include <opencv2/cvv/final_show.hpp> |
||||
#include <opencv2/cvv/show_image.hpp> |
||||
#ifdef __OPENCV_BUILD |
||||
#error this is a compatibility header which should not be used inside the OpenCV library |
||||
#endif |
||||
|
||||
#include "opencv2/cvv.hpp" |
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in new issue