@note do not forget that the retina model is included in the following namespace : cv::bioinspired
@note do not forget that the retina model is included in the following namespace : cv::bioinspired with C++ and in cv2.bioinspired with Python
### Introduction
Class which provides the main controls to the Gipsa/Listic labs human retina model. This is a non
This class provides the main controls of the Gipsa/Listic labs human retina model. This is a non
separable spatio-temporal filter modelling the two main retina information channels :
- foveal vision for detailled color vision : the parvocellular pathway.
- peripheral vision for sensitive transient signals detection (motion and events) : the
magnocellular pathway.
- foveal vision for detailed color vision : the parvocellular pathway.
- peripheral vision for sensitive transient signals detection (motion and events) : the magnocellular pathway.
From a general point of view, this filter whitens the image spectrum and corrects luminance thanks
to local adaptation. An other important property is its hability to filter out spatio-temporal noise
while enhancing details. This model originates from Jeanny Herault work @cite Herault2010 . It has been
involved in Alexandre Benoit phd and his current research @cite Benoit2010, @cite Strat2013 (he
currently maintains this module within OpenCV). It includes the work of other Jeanny's phd student
This model originates from Jeanny Herault work @cite Herault2010 . It has been
involved in Alexandre Benoit phd and his current research @cite Benoit2010, @cite Benoit2014 . He
currently maintains this module within OpenCV. It includes the work of other Jeanny's phd student
such as @cite Chaix2007 and the log polar transformations of Barthelemy Durette described in Jeanny's
book.
More into details here is an overview of the retina properties that are implemented here :
- regarding luminance and details enhancement :
- local logarithmic luminance compression (at the entry point by photoreceptors and at the output by ganglion cells).
- spectral whitening at the Outer Plexiform Layer level (photoreceptors and horizontal cells spatio-temporal filtering).
The former behavior compresses luminance range and allows very bright areas and very dark ones to be visible on the same picture with lots of details. The latter reduces low frequency luminance energy (mean luminance) and enhances mid-frequencies (details). Applied all together, retina well prepares visual signals prior high level analysis. Those properties are really interesting with videos where light changes are dramatically reduced with an interesting temporal consistency.
- regarding noise filtering :
- high frequency spatial and temporal noise is filtered out. Both outputs Parvo and Magno pathways benefit from this. Noise reduction benefits from the non separable spatio-temporal filtering.
- at the Parvo output, static textures are enhanced and noise is filtered (on videos, temporal noise is nicely removed). However, as human behaviors, moving textures are smoothed. Then, moving object details can be only enhanced if the retina tracks it and keeps it static from its point of view.
- at Magno output, it allows a cleaner detection of events (motion, changes) with reduced noise errors even in difficult lighting conditions. As a compromise, the Magno output is a low spatial frequency signal and allows events' blobs to be reliably extracted (check the TransientAreasSegmentationModule module for that).
### Use
This model can be used as a preprocessing stage in the aim of :
- performing texture analysis with enhanced signal to noise ratio and enhanced details which are robust
against input images luminance ranges (check out the parvocellular retina channel output, by
using the provided **getParvo** methods)
- performing motion analysis that is also taking advantage of the previously cited properties (check out the
magnocellular retina channel output, by using the provided **getMagno** methods)
- general image/video sequence description using either one or both channels. An example of the
use of Retina in a Bag of Words approach is given in @cite Benoit2014 .
@note
- For ease of use in computer vision applications, the two retina channels are applied
homogeneously on all the input images. This does not follow the real retina topology but this
can still be done using the log sampling capabilities proposed within the class.
- Extend the retina description and code use in the tutorial/contrib section for complementary
explanations.
on all the input images. This does not follow the real retina topology but it is practical from an image processing point of view. If retina mapping (foveal and parafoveal vision) is required, use the log sampling capabilities proposed within the class.
- Please do not hesitate to contribute by extending the retina description, code, use cases for complementary explanations and demonstrations.
### Preliminary illustration
### Use case illustrations
#### Image preprocessing using the Parvocellular pathway (parvo retina output)
As a preliminary presentation, let's start with a visual example. We propose to apply the filter on
a low quality color jpeg image with backlight problems. Here is the considered input... *"Well, my
eyes were able to see more that this strange black shadow..."*
a low quality color jpeg image with backlight problems. Here is the considered input... *"Well,i could see more with my eyes than what i captured with my camera..."*

Below, the retina foveal model applied on the entire image with default parameters. Here contours
are enforced, halo effects are voluntary visible with this configuration. See parameters discussion
Below, the retina foveal model applied on the entire image with default parameters. Details are enforced whatever the local luminance is. Here there contours
are strongly enforced but the noise level is kept low. Halo effects are voluntary visible with this configuration. See parameters discussion
below and increase horizontalCellsGain near 1 to remove them.

@ -48,27 +68,96 @@ Below, a second retina foveal model output applied on the entire image with a pa
focused on naturalness perception. *"Hey, i now recognize my cat, looking at the mountains at the
end of the day !"*. Here contours are enforced, luminance is corrected but halos are avoided with
this configuration. The backlight effect is corrected and highlight details are still preserved.
Then, even on a low quality jpeg image, if some luminance information remains, the retina is able to
reconstruct a proper visual signal. Such configuration is also usefull for High Dynamic Range
Then, even on a low quality jpeg image, if some luminance's information remains, the retina is able to
reconstruct a proper visual signal. Such configuration is also useful for High Dynamic Range
(*HDR*) images compression to 8bit images as discussed in @cite Benoit2010 and in the demonstration
codes discussed below. As shown at the end of the page, parameters change from defaults are :
codes discussed below. As shown at the end of the page, parameter changes from defaults are :


As observed in this preliminary demo, the retina can be settled up with various parameters, by
default, as shown on the figure above, the retina strongly reduces mean luminance energy and
enforces all details of the visual scene. Luminance energy and halo effects can be modulated
(exagerated to cancelled as shown on the two examples). In order to use your own parameters, you can
(exaggerated to cancelled as shown on the two examples). In order to use your own parameters, you can
use at least one time the *write(String fs)* method which will write a proper XML file with all
default parameters. Then, tweak it on your own and reload them at any time using method
*setup(String fs)*. These methods update a *Retina::RetinaParameters* member structure that is
described hereafter. XML parameters file samples are shown at the end of the page.
Here is an overview of the abstract Retina interface, allocate one instance with the *createRetina*
functions.:
#### Tone mapping processing capability using the Parvocellular pathway (parvo retina output)
This retina model naturally handles luminance range compression. Local adaptation stages and spectral whitening contribute
to luminance range compression. In addition, high frequency noise that often corrupts tone mapped images is removed at early stages of the
process thus leading to natural perception and noise free tone mapping.
Compared to the demos shown above, setup differences are the following ones: (see bioinspired/samples/OpenEXRimages_HDR_Retina_toneMapping.cpp for more details)
* load HDR images (OpenEXR format is supported by OpenCV) and cut histogram borders at ~5% and 95% to eliminate salt&pepper like pixel's corruption.
* apply retina with default parameters along with the following changes (generic parameters used for the presented illustrations of the section) :
* retina Hcells gain =0.4 (the main change compared to the default configuration : it strongly reduces halo effects)
* localAdaptation_photoreceptors=0.99 (a little higher than default value to enforce local adaptation)
* localAdaptation_Gcells=0.95 (also slightly higher than default for local adaptation enforcement)
* get the parvo output using the *getParvo()* method.
Have a look at the end of this page to see how to specify these parameters in a configuration file.
The following two illustrations show the effect of such configuration on 2 image samples.


#### Motion and event detection using the Magnocellular pathway (magno retina output)
Spatio-temporal events can be easily detected using *magno* output of the retina (use the *getMagno()* method). Its energy linearly increases with motion speed.
An event blob detector is proposed with the TransientAreasSegmentationModule class also provided in the bioinspired module. The basic idea is to detect local energy drops with regard of the neighborhood and then to apply a threshold. Such process has been used in a bag of words description of videos on the TRECVid challenge @cite Benoit2014 and only allows video frames description on transient areas.
We present here some illustrations of the retina outputs on some examples taken from http://changedetection.net/ with RGB and thermal videos.
@note here, we use the default retina setup that generates halos around strong edges. Note that temporal constants allow a temporal effect to be visible on moting objects (useful for still image illustrations of a video). Halos can be removed by increasing retina Hcells gain while temporal effects can be reduced by decreasing temporal constant values.
Also take into account that the two retina outputs are rescaled in range [0:255] such that magno output can show a lot of "noise" when nothing moves while drawing it. However, its energy remains low if you retrieve it using *getMagnoRAW* getter instead.


### Literature
For more information, refer to the following papers :
- Model description : @cite Benoit2010
- Model use in a Bag of Words approach : @cite Benoit2014
- Please have a look at the reference work of Jeanny Herault that you can read in his book : @cite Herault2010
This retina filter code includes the research contributions of phd/research collegues from which
code has been redrawn by the author :
- take a look at the *retinacolor.hpp* module to discover Brice Chaix de Lavarene phD color
mosaicing/demosaicing and his reference paper: @cite Chaix2007
- take a look at *imagelogpolprojection.hpp* to discover retina spatial log sampling which
originates from Barthelemy Durette phd with Jeanny Herault. A Retina / V1 cortex projection is
also proposed and originates from Jeanny's discussions. More informations in the above cited
Jeanny Heraults's book.
- Meylan&al work on HDR tone mapping that is implemented as a specific method within the model : @cite Meylan2007
Retina programming interfaces
-----------------------------
The proposed class allows the [Gipsa](http://www.gipsa-lab.inpg.fr) (preliminary work) /
[Listic](http://www.listic.univ-savoie.fr) labs retina model to be used.
It can be applied on still images, images sequences and video sequences.
Here is an overview of the Retina interface, allocate one instance with the *createRetina*
functions (C++, Java, Python) :
@code{.cpp}
namespace cv{namespace bioinspired{
@ -76,7 +165,7 @@ functions.:
{
public:
// parameters setup instance
struct RetinaParameters; // this class is detailled later
struct RetinaParameters; // this class is detailed later
// main method for input frame processing (all use method, can also perform High Dynamic Range tone mapping)
void run (InputArray inputImage);
@ -84,20 +173,20 @@ functions.:
// specific method aiming at correcting luminance only (faster High Dynamic Range tone mapping)
Class which allows the [Gipsa](http://www.gipsa-lab.inpg.fr) (preliminary work) /
[Listic](http://www.listic.univ-savoie.fr) (code maintainer and user) labs retina model to be used.
This class allows human retina spatio-temporal image processing to be applied on still images,
images sequences and video sequences. Briefly, here are the main human retina model properties:
Here are some words about all those parameters, tweak them as you wish to amplify or moderate retina effects (contours enforcement, halos effects, motion sensitivity, motion blurring, etc.)
#### Basic parameters
The simplest parameters are as follows :
- **colorMode** : let the retina process color information (if 1) or gray scale images (if 0). In
that last case, only the first channels of the input will be processed.
- **normaliseOutput** : each channel has such parameter: if the value is set to 1, then the considered
channel's output is rescaled between 0 and 255. Be aware at this case of the Magnocellular output
level (motion/transient channel detection). Residual noise will also be rescaled !
**Note :** using color requires color channels multiplexing/demultipexing which also demands more
processing. You can expect much faster processing using gray levels : it would require around 30
product per pixel for all of the retina processes and it has recently been parallelized for multicore
architectures.
#### Photo-receptors parameters
The following parameters act on the entry point of the retina - photo-receptors - and has impact on all
of the following processes. These sensors are low pass spatio-temporal filters that smooth temporal and
spatial data and also adjust their sensitivity to local luminance,thus, leads to improving details extraction
and high frequency noise canceling.
- **photoreceptorsLocalAdaptationSensitivity** between 0 and 1. Values close to 1 allow high
luminance log compression's effect at the photo-receptors level. Values closer to 0 provide a more
linear sensitivity. Increased alone, it can burn the *Parvo (details channel)* output image. If
adjusted in collaboration with **ganglionCellsSensitivity**,images can be very contrasted
whatever the local luminance there is... at the cost of a naturalness decrease.
- **photoreceptorsTemporalConstant** this setups the temporal constant of the low pass filter
effect at the entry of the retina. High value leads to strong temporal smoothing effect : moving
objects are blurred and can disappear while static object are favored. But when starting the
retina processing, stable state is reached later.
- **photoreceptorsSpatialConstant** specifies the spatial constant related to photo-receptors' low
pass filter's effect. Those parameters specify the minimum value of the spatial signal period allowed
in what follows. Typically, this filter should cut high frequency noise. On the other hand, a 0 value
cuts none of the noise while higher values start to cut high spatial frequencies, and progressively
lower frequencies... Be aware to not go to high levels if you want to see some details of the input images !
A good compromise for color images is a 0.53 value since such choice won't affect too much the color spectrum.
Higher values would lead to gray and blurred output images.
#### Horizontal cells parameters
This parameter set tunes the neural network connected to the photo-receptors, the horizontal cells.
It modulates photo-receptors sensitivity and completes the processing for final spectral whitening
(part of the spatial band pass effect thus favoring visual details enhancement).
- **horizontalCellsGain** here is a critical parameter ! If you are not interested with the mean
luminance and want just to focus on details enhancement, then, set this parameterto zero. However, if
you want to keep some environment luminance's data, let some low spatial frequencies pass into the system and set a
higher value (\<1).
- **hcellsTemporalConstant** similar to photo-receptors, this parameter acts on the temporal constant of a
low pass temporal filter that smoothes input data. Here, a high value generates a high retina
after effect while a lower value makes the retina more reactive. This value should be lower than
**photoreceptorsTemporalConstant** to limit strong retina after effects.
- **hcellsSpatialConstant** is the spatial constant of these cells filter's low pass one.
It specifies the lowest spatial frequency allowed in what follows. Visually, a high value leads
to very low spatial frequencies processing and leads to salient halo effects. Lower values
reduce this effect but has the limit of not go lower than the value of
**photoreceptorsSpatialConstant**. Those 2 parameters actually specify the spatial band-pass of
the retina.
**NOTE** Once the processing managed by the previous parameters is done, input data is cleaned from noise
and luminance is already partly enhanced. The following parameters act on the last processing stages
of the two outing retina signals.
#### Parvo (details channel) dedicated parameter
- **ganglionCellsSensitivity** specifies the strength of the final local adaptation occurring at
the output of this details' dedicated channel. Parameter values remain between 0 and 1. Low value
tend to give a linear response while higher values enforce the remaining low contrasted areas.
**Note :** this parameter can correct eventual burned images by favoring low energetic details of
the visual scene, even in bright areas.
#### IPL Magno (motion/transient channel) parameters
Once image's information are cleaned, this channel acts as a high pass temporal filter that
selects only the signals related to transient signals (events, motion, etc.). A low pass spatial filter
smoothes extracted transient data while a final logarithmic compression enhances low transient events
thus enhancing event sensitivity.
- **parasolCells_beta** generally set to zero, can be considered as an amplifier gain at the
entry point of this processing stage. Generally set to 0.
- **parasolCells_tau** the temporal smoothing effect that can be added
- **parasolCells_k** the spatial constant of the spatial filtering effect, set it at a high value
to favor low spatial frequency signals that are lower subject for residual noise.
- **amacrinCellsTemporalCutFrequency** specifies the temporal constant of the high pass filter.
High values let slow transient events to be selected.
- **V0CompressionParameter** specifies the strength of the log compression. Similar behaviors to
previous description but here enforces sensitivity of transient events.
- **localAdaptintegration_tau** generally set to 0, has no real use actually in here.
- **localAdaptintegration_k** specifies the size of the area on which local adaptation is
performed. Low values lead to short range local adaptation (higher sensitivity to noise), high
values secure log compression.
### Demos and experiments !
#### First time experiments
Here are some code snippets to shortly show how to use Retina with default parameters (with halo effects). Next section redirects to more complete demos provided with the main retina class.
Here is presented how to process a webcam stream with the following steps :
- load a frist input image to get its size
- allocate a retina instance with appropriate input size
- loop over grabbed frames :
- grab a new frame
- run on a frame
- call the two output getters
- display retina outputs
C++ version (see bioinspired/samples/basicRetina.cpp) :
BenoitA.,CaplierA.,DuretteB.,Herault,J.,"USING HUMAN VISUAL SYSTEM MODELING FOR BIO-INSPIRED LOW LEVEL IMAGE PROCESSING",Elsevier,ComputerVisionandImageUnderstanding114(2010),pp.758-773,DOI:http://dx.doi.org/10.1016/j.cviu.2010.01.011
B.ChaixdeLavarene,D.Alleysson,B.Durette,J.Herault(2007)."Efficient demosaicing through recursive filtering",IEEEInternationalConferenceonImageProcessingICIP2007
BenoitA.,CaplierA.,DuretteB.,Herault,J.,"USING HUMAN VISUAL SYSTEM MODELING FOR BIO-INSPIRED LOW LEVEL IMAGE PROCESSING",Elsevier,ComputerVisionandImageUnderstanding114(2010),pp.758-773,DOI:http://dx.doi.org/10.1016/j.cviu.2010.01.011
B.ChaixdeLavarene,D.Alleysson,B.Durette,J.Herault(2007)."Efficient demosaicing through recursive filtering",IEEEInternationalConferenceonImageProcessingICIP2007
formoreinformations,pleasehavealookatthepaperBenoitA.,CaplierA.,DuretteB.,Herault,J.,"USING HUMAN VISUAL SYSTEM MODELING FOR BIO-INSPIRED LOW LEVEL IMAGE PROCESSING",Elsevier,ComputerVisionandImageUnderstanding114(2010),pp.758-773,DOI:http://dx.doi.org/10.1016/j.cviu.2010.01.011
formoreinformations,pleasehavealookatthepaperBenoitA.,CaplierA.,DuretteB.,Herault,J.,"USING HUMAN VISUAL SYSTEM MODELING FOR BIO-INSPIRED LOW LEVEL IMAGE PROCESSING",Elsevier,ComputerVisionandImageUnderstanding114(2010),pp.758-773,DOI:http://dx.doi.org/10.1016/j.cviu.2010.01.011
**StratS.T.,BenoitA.LambertP.,CaplierA.,"Retina Enhanced SURF Descriptors for Spatio-Temporal Concept Detection",MultimediaToolsandApplications,2012(DOI:10.1007/s11042-012-1280-0)
**Strat,S.T.;Benoit,A.;Lambert,P.,"Retina enhanced bag of words descriptors for video classification,"SignalProcessingConference(EUSIPCO),2014Proceedingsofthe22ndEuropean,vol.,no.,pp.1307,1311,1-5Sept.2014(http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6952461&isnumber=6951911)
**BenoitA.,CaplierA.,DuretteB.,Herault,J.,"USING HUMAN VISUAL SYSTEM MODELING FOR BIO-INSPIRED LOW LEVEL IMAGE PROCESSING",Elsevier,ComputerVisionandImageUnderstanding114(2010),pp.758-773,DOI:http://dx.doi.org/10.1016/j.cviu.2010.01.011
/** @brief parameter structure that stores the transient events detector setup parameters
*/
structSegmentationParameters{// CV_EXPORTS_W_MAP to export to python native dictionnaries
// default structure instance construction with default values
SegmentationParameters():
thresholdON(100),
thresholdOFF(100),
localEnergy_temporalConstant(0.5),
localEnergy_spatialConstant(5),
neighborhoodEnergy_temporalConstant(1),
neighborhoodEnergy_spatialConstant(15),
contextEnergy_temporalConstant(1),
contextEnergy_spatialConstant(75){};
// all properties list
floatthresholdON;
floatthresholdOFF;
//! the time constant of the first order low pass filter, use it to cut high temporal frequencies (noise or fast motion), unit is frames, typical value is 0.5 frame
floatlocalEnergy_temporalConstant;
//! the spatial constant of the first order low pass filter, use it to cut high spatial frequencies (noise or thick contours), unit is pixels, typical value is 5 pixel
floatlocalEnergy_spatialConstant;
//! local neighborhood energy filtering parameters : the aim is to get information about the energy neighborhood to perform a center surround energy analysis
floatneighborhoodEnergy_temporalConstant;
floatneighborhoodEnergy_spatialConstant;
//! context neighborhood energy filtering parameters : the aim is to get information about the energy on a wide neighborhood area to filtered out local effects
floatcontextEnergy_temporalConstant;
floatcontextEnergy_spatialConstant;
};
/** @brief class which provides a transient/moving areas segmentation module
//! the time constant of the first order low pass filter, use it to cut high temporal frequencies (noise or fast motion), unit is frames, typical value is 0.5 frame
CV_PROP_RWfloatlocalEnergy_temporalConstant;
//! the spatial constant of the first order low pass filter, use it to cut high spatial frequencies (noise or thick contours), unit is pixels, typical value is 5 pixel
CV_PROP_RWfloatlocalEnergy_spatialConstant;
//! local neighborhood energy filtering parameters : the aim is to get information about the energy neighborhood to perform a center surround energy analysis
//! context neighborhood energy filtering parameters : the aim is to get information about the energy on a wide neighborhood area to filtered out local effects
CV_PROP_RWfloatcontextEnergy_temporalConstant;
CV_PROP_RWfloatcontextEnergy_spatialConstant;
};
/** @brief return the sze of the manage input and output images
std::cout<<"* Retina demonstration : demonstrates the use of is a wrapper class of the Gipsa/Listic Labs retina model."<<std::endl;
std::cout<<"* This retina model allows spatio-temporal image processing (applied on a webcam sequences)."<<std::endl;
std::cout<<"* As a summary, these are the retina model properties:"<<std::endl;
std::cout<<"* => It applies a spectral whithening (mid-frequency details enhancement)"<<std::endl;
std::cout<<"* => high frequency spatio-temporal noise reduction"<<std::endl;
std::cout<<"* => low frequency luminance to be reduced (luminance range compression)"<<std::endl;
std::cout<<"* => local logarithmic luminance compression allows details to be enhanced in low light conditions\n"<<std::endl;
std::cout<<"* for more information, reer to the following papers :"<<std::endl;
std::cout<<"* Benoit A., Caplier A., Durette B., Herault, J., \"USING HUMAN VISUAL SYSTEM MODELING FOR BIO-INSPIRED LOW LEVEL IMAGE PROCESSING\", Elsevier, Computer Vision and Image Understanding 114 (2010), pp. 758-773, DOI: http://dx.doi.org/10.1016/j.cviu.2010.01.011"<<std::endl;
std::cout<<"* Vision: Images, Signals and Neural Networks: Models of Neural Processing in Visual Perception (Progress in Neural Processing),By: Jeanny Herault, ISBN: 9814273686. WAPI (Tower ID): 113266891."<<std::endl;
std::cout<<"* => reports comments/remarks at benoit.alexandre.vision@gmail.com"<<std::endl;
std::cout<<"* => more informations and papers at : http://sites.google.com/site/benoitalexandrevision/"<<std::endl;
**StratS.T.,BenoitA.LambertP.,CaplierA.,"Retina Enhanced SURF Descriptors for Spatio-Temporal Concept Detection",MultimediaToolsandApplications,2012(DOI:10.1007/s11042-012-1280-0)
**Strat,S.T.;Benoit,A.;Lambert,P.,"Retina enhanced bag of words descriptors for video classification,"SignalProcessingConference(EUSIPCO),2014Proceedingsofthe22ndEuropean,vol.,no.,pp.1307,1311,1-5Sept.2014(http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6952461&isnumber=6951911)
**BenoitA.,CaplierA.,DuretteB.,Herault,J.,"USING HUMAN VISUAL SYSTEM MODELING FOR BIO-INSPIRED LOW LEVEL IMAGE PROCESSING",Elsevier,ComputerVisionandImageUnderstanding114(2010),pp.758-773,DOI:http://dx.doi.org/10.1016/j.cviu.2010.01.011
std::cout<<"* Retina demonstration : demonstrates the use of is a wrapper class of the Gipsa/Listic Labs retina model."<<std::endl;
std::cout<<"* This demo will try to load the file 'RetinaSpecificParameters.xml' (if exists).\nTo create it, copy the autogenerated template 'RetinaDefaultParameters.xml'.\nThen twaek it with your own retina parameters."<<std::endl;
std::cout<<"* This demo will try to load the file 'RetinaSpecificParameters.xml' (if exists).\nTo create it, copy the autogenerated template 'RetinaDefaultParameters.xml'.\nThen tweak it with your own retina parameters."<<std::endl;
// basic input arguments checking
if (argc<2)
{
@ -259,7 +259,7 @@ to manage the eventual log sampling option. The Retina constructor expects at le
object that shows the input data size that will have to be managed. One can activate other options
such as color and its related color multiplexing strategy (here Bayer multiplexing is chosen using
*enum cv::bioinspired::RETINA_COLOR_BAYER*). If using log sampling, the image reduction factor
(smaller output images) and log sampling strengh can be adjusted.
(smaller output images) and log sampling strength can be adjusted.
@code{.cpp}
// pointer to a retina object
cv::Ptr<cv::bioinspired::Retina> myRetina;
@ -381,96 +381,98 @@ Then, if the application target requires details enhancement prior to specific i
need to know if mean luminance information is required or not. If not, the the retina can cancel or
significantly reduce its energy thus giving more visibility to higher spatial frequency details.
### Basic parameters
The most simple parameters are the following :
#### Basic parameters
The simplest parameters are as follows :
- **colorMode** : let the retina process color information (if 1) or gray scale images (if 0). In
this last case, only the first channel of the input will be processed.
- **normaliseOutput** : each channel has this parameter, if value is 1, then the considered
channel output is rescaled between 0 and 255. Take care in this case at the Magnocellular output
that last case, only the first channels of the input will be processed.
- **normaliseOutput** : each channel has such parameter: if the value is set to 1, then the considered
channel's output is rescaled between 0 and 255. Be aware at this case of the Magnocellular output
level (motion/transient channel detection). Residual noise will also be rescaled !
**Note :** using color requires color channels multiplexing/demultipexing which requires more
**Note :** using color requires color channels multiplexing/demultipexing which also demands more
processing. You can expect much faster processing using gray levels : it would require around 30
product per pixel for all the retina processes and it has recently been parallelized for multicore
product per pixel for all of the retina processes and it has recently been parallelized for multicore
architectures.
### Photo-receptors parameters
#### Photo-receptors parameters
The following parameters act on the entry point of the retina - photo-receptors - and impact all the
following processes. These sensors are low pass spatio-temporal filters that smooth temporal and
spatial data and also adjust there sensitivity to local luminance thus improving details extraction
The following parameters act on the entry point of the retina - photo-receptors - and has impact on all
of the following processes. These sensors are low pass spatio-temporal filters that smooth temporal and
spatial data and also adjust their sensitivity to local luminance,thus, leads to improving details extraction
and high frequency noise canceling.
- **photoreceptorsLocalAdaptationSensitivity** between 0 and 1. Values close to 1 allow high
luminance log compression effect at the photo-receptors level. Values closer to 0 give a more
luminance log compression's effect at the photo-receptors level. Values closer to 0 provide a more
linear sensitivity. Increased alone, it can burn the *Parvo (details channel)* output image. If
adjusted in collaboration with **ganglionCellsSensitivity**images can be very contrasted
whatever the local luminance there is... at the price of a naturalness decrease.
adjusted in collaboration with **ganglionCellsSensitivity**,images can be very contrasted
whatever the local luminance there is... at the cost of a naturalness decrease.
- **photoreceptorsTemporalConstant** this setups the temporal constant of the low pass filter
effect at the entry of the retina. High value lead to strong temporal smoothing effect : moving
effect at the entry of the retina. High value leads to strong temporal smoothing effect : moving
objects are blurred and can disappear while static object are favored. But when starting the
retina processing, stable state is reached lately.
- **photoreceptorsSpatialConstant** specifies the spatial constant related to photo-receptors low
pass filter effect. This parameters specify the minimum allowed spatial signal period allowed in
the following. Typically, this filter should cut high frequency noise. Then a 0 value doesn't
cut anything noise while higher values start to cut high spatial frequencies and more and more
lower frequencies... Then, do not go to high if you wanna see some details of the input images !
A good compromise for color images is 0.53 since this won't affect too much the color spectrum.
retina processing, stable state is reached later.
- **photoreceptorsSpatialConstant** specifies the spatial constant related to photo-receptors' low
pass filter's effect. Those parameters specify the minimum value of the spatial signal period allowed
in what follows. Typically, this filter should cut high frequency noise. On the other hand, a 0 value
cuts none of the noise while higher values start to cut high spatial frequencies, and progressively
lower frequencies... Be aware to not go to high levels if you want to see some details of the input images !
A good compromise for color images is a 0.53 value since such choice won't affect too much the color spectrum.
Higher values would lead to gray and blurred output images.
### Horizontal cells parameters
#### Horizontal cells parameters
This parameter set tunes the neural network connected to the photo-receptors, the horizontal cells.
It modulates photo-receptors sensitivity and completes the processing for final spectral whitening
(part of the spatial band pass effect thus favoring visual details enhancement).
- **horizontalCellsGain** here is a critical parameter ! If you are not interested by the mean
luminance and focus on details enhancement, then, set to zero. But if you want to keep some
environment luminance data, let some low spatial frequencies pass into the system and set a
- **horizontalCellsGain** here is a critical parameter ! If you are not interested with the mean
luminance and want just to focus on details enhancement, then, set this parameterto zero. However, if
you want to keep some environment luminance's data, let some low spatial frequencies pass into the system and set a
higher value (\<1).
- **hcellsTemporalConstant** similar to photo-receptors, this acts on the temporal constant of a
low pass temporal filter that smooths input data. Here, a high value generates a high retina
- **hcellsTemporalConstant** similar to photo-receptors, this parameter acts on the temporal constant of a
low pass temporal filter that smoothes input data. Here, a high value generates a high retina
after effect while a lower value makes the retina more reactive. This value should be lower than
**photoreceptorsTemporalConstant** to limit strong retina after effects.
- **hcellsSpatialConstant** is the spatial constant of the low pass filter of these cells filter.
It specifies the lowest spatial frequency allowed in the following. Visually, a high value leads
- **hcellsSpatialConstant** is the spatial constant of these cells filter's low pass one.
It specifies the lowest spatial frequency allowed in what follows. Visually, a high value leads
to very low spatial frequencies processing and leads to salient halo effects. Lower values
reduce this effect but the limit is : do not go lower than the value of
reduce this effect but has the limit of not go lower than the value of
**photoreceptorsSpatialConstant**. Those 2 parameters actually specify the spatial band-pass of
the retina.
**NOTE** after the processing managed by the previous parameters, input data is cleaned from noise
and luminance in already partly enhanced. The following parameters act on the last processing stages
**NOTE** Once the processing managed by the previous parameters is done, input data is cleaned from noise
and luminance is already partly enhanced. The following parameters act on the last processing stages
of the two outing retina signals.
### Parvo (details channel) dedicated parameter
#### Parvo (details channel) dedicated parameter
- **ganglionCellsSensitivity** specifies the strength of the final local adaptation occurring at
the output of this details dedicated channel. Parameter values remain between 0 and 1. Low value
tend to give a linear response while higher values enforces the remaining low contrasted areas.
the output of this details' dedicated channel. Parameter values remain between 0 and 1. Low value
tend to give a linear response while higher values enforce the remaining low contrasted areas.
**Note :** this parameter can correct eventual burned images by favoring low energetic details of
the visual scene, even in bright areas.
### IPL Magno (motion/transient channel) parameters
#### IPL Magno (motion/transient channel) parameters
Once image information is cleaned, this channel acts as a high pass temporal filter that only
selects signals related to transient signals (events, motion, etc.). A low pass spatial filter
smooths extracted transient data and a final logarithmic compression enhances low transient events
Once image's information are cleaned, this channel acts as a high pass temporal filter that
selects only the signals related to transient signals (events, motion, etc.). A low pass spatial filter
smoothes extracted transient data while a final logarithmic compression enhances low transient events
thus enhancing event sensitivity.
- **parasolCells_beta** generally set to zero, can be considered as an amplifier gain at the
entry point of this processing stage. Generally set to 0.
- **parasolCells_tau** the temporal smoothing effect that can be added
- **parasolCells_k** the spatial constant of the spatial filtering effect, set it at a high value
to favor low spatial frequency signals that are lower subject to residual noise.
to favor low spatial frequency signals that are lower subject for residual noise.
- **amacrinCellsTemporalCutFrequency** specifies the temporal constant of the high pass filter.
High values let slow transient events to be selected.
- **V0CompressionParameter** specifies the strength of the log compression. Similar behaviors to
previous description but here it enforces sensitivity of transient events.
- **localAdaptintegration_tau** generally set to 0, no real use here actually
previous description but here enforces sensitivity of transient events.
- **localAdaptintegration_tau** generally set to 0, has no real use actually in here.
- **localAdaptintegration_k** specifies the size of the area on which local adaptation is
performed. Low values lead to short range local adaptation (higher sensitivity to noise), high
The module contains algorithms to detect text, segment words and recognise the text.
It's mainly intended for the "text in the wild", i.e. short phrases and separate words that occur on navigation signs and such. It's not an OCR tool for scanned documents, do not treat it as such.
The detection part can in theory handle different languages, but will likely fail on hieroglyphic texts.
The recognition part currently uses open-source Tesseract OCR (https://code.google.com/p/tesseract-ocr/). If Tesseract OCR is not installed on your system, the corresponding part of the functionality will be unavailable.
Here are instructions on how to install Tesseract on your machine (Linux or Mac; Windows users should look for precompiled binaries or try to adopt the instructions below):
Tesseract installation instruction (Linux, Mac)
-----------------------------------------------
0. Linux users may try to install tesseract-3.03-rc1 (or later) and leptonica-1.70 (or later) with the corresponding developement packages using their package manager. Mac users may try brew. The instructions below are for those who wants to build tesseract from source.
1. download leptonica 1.70 tarball (helper image processing library, used by tesseract. Later versions might work too):
http://www.leptonica.com/download.html
unpack and build it:
cd leptonica-1.70
mkdir build && cd build && ../configure && make && sudo make install
leptonica will be installed to /usr/local.
2. download tesseract-3.03-rc1 tarball from https://drive.google.com/folderview?id=0B7l10Bj_LprhQnpSRkpGMGV2eE0&usp=sharing
unzip it (gzip -d eng.traineddata.gz) and copy to /usr/local/share/tessdata.
Notes
-----
1. Google announced that they close code.google.com, so at some moment in the future you may have to find Tesseract 3.03rc1 or later.
2. Tesseract configure script may fail to detect leptonica, so you may have to edit the configure script - comment off some if's around this message and retain only "then" branch.
3. You are encouraged to search the Net for some better pre-trained classifiers, as well as classifiers for other languages.
fprintf(cur_cfg_res_file,"Name,MSE,MSE after postfiltering,Percent bad,Percent bad after postfiltering,Matcher Execution Time(s),Filter Execution Time(s)\n");