In Logistic Regression, choosing the right parameters is of utmost importance for reducing the training error and ensuring high training accuracy.
``LogisticRegressionParams`` is the structure that defines parameters that are required to train a Logistic Regression classifier.
The learning rate is determined by ``LogisticRegressionParams.alpha``. It determines how faster we approach the solution.
``LogisticRegression::Params`` is the structure that defines parameters that are required to train a Logistic Regression classifier.
The learning rate is determined by ``LogisticRegression::Params.alpha``. It determines how faster we approach the solution.
It is a positive real number. Optimization algorithms like Batch Gradient Descent and Mini-Batch Gradient Descent are supported in ``LogisticRegression``.
It is important that we mention the number of iterations these optimization algorithms have to run.
The number of iterations are mentioned by ``LogisticRegressionParams.num_iters``.
The number of iterations are mentioned by ``LogisticRegression::Params.num_iters``.
The number of iterations can be thought as number of steps taken and learning rate specifies if it is a long step or a short step. These two parameters define how fast we arrive at a possible solution.
In order to compensate for overfitting regularization is performed, which can be enabled by setting ``LogisticRegressionParams.regularized`` to a positive integer (greater than zero).
One can specify what kind of regularization has to be performed by setting ``LogisticRegressionParams.norm`` to ``LogisticRegression::REG_L1`` or ``LogisticRegression::REG_L2`` values.
``LogisticRegression`` provides a choice of 2 training methods with Batch Gradient Descent or the Mini-Batch Gradient Descent. To specify this, set ``LogisticRegressionParams.train_method`` to either ``LogisticRegression::BATCH`` or ``LogisticRegression::MINI_BATCH``.
If ``LogisticRegressionParams`` is set to ``LogisticRegression::MINI_BATCH``, the size of the mini batch has to be to a postive integer using ``LogisticRegressionParams.mini_batch_size``.
In order to compensate for overfitting regularization is performed, which can be enabled by setting ``LogisticRegression::Params.regularized`` to a positive integer (greater than zero).
One can specify what kind of regularization has to be performed by setting ``LogisticRegression::Params.norm`` to ``LogisticRegression::REG_L1`` or ``LogisticRegression::REG_L2`` values.
``LogisticRegression`` provides a choice of 2 training methods with Batch Gradient Descent or the Mini-Batch Gradient Descent. To specify this, set ``LogisticRegression::Params.train_method`` to either ``LogisticRegression::BATCH`` or ``LogisticRegression::MINI_BATCH``.
If ``LogisticRegression::Params`` is set to ``LogisticRegression::MINI_BATCH``, the size of the mini batch has to be to a postive integer using ``LogisticRegression::Params.mini_batch_size``.
A sample set of training parameters for the Logistic Regression classifier can be initialized as follows:
::
LogisticRegressionParams params;
LogisticRegression::Params params;
params.alpha = 0.5;
params.num_iters = 10000;
params.norm = LogisticRegression::REG_L2;
@ -49,16 +51,19 @@ A sample set of training parameters for the Logistic Regression classifier can b
..[LogRegWiki] http://en.wikipedia.org/wiki/Logistic_regression. Wikipedia article about the Logistic Regression algorithm.
..[RenMalik2003] Learning a Classification Model for Segmentation. Proc. CVPR, Nice, France (2003).
..[LogRegTomMitch] http://www.cs.cmu.edu/~tom/NewChapters.html. "Generative and Discriminative Classifiers: Naive Bayes and Logistic Regression" in Machine Learning, Tom Mitchell.
..[BatchDesWiki] http://en.wikipedia.org/wiki/Gradient_descent_optimization. Wikipedia article about Gradient Descent based optimization.
LogisticRegressionParams
------------------------
..ocv:struct:: LogisticRegressionParams
LogisticRegression::Params
--------------------------
..ocv:struct:: LogisticRegression::Params
Parameters of the Logistic Regression training algorithm. You can initialize the structure using a constructor or declaring the variable and initializing the the individual parameters.
..ocv:function:: LogisticRegressionParams::LogisticRegressionParams(double learning_rate, int iters, int train_method, int normlization, int reg, int mini_batch_size)
..ocv:function:: LogisticRegression::Params::Params(double learning_rate = 0.001, int iters = 1000, int method = LogisticRegression::BATCH, int normlization = LogisticRegression::REG_L2, int reg = 1, int batch_size = 1)
:param learning_rate:Specifies the learning rate.
:param iters:Specifies the number of iterations.
:param:train_method: Specifies the kind of training method used. It should be set to either ``LogisticRegression::BATCH`` or ``LogisticRegression::MINI_BATCH``. If using ``LogisticRegression::MINI_BATCH``, set ``LogisticRegressionParams.mini_batch_size`` to a positive integer.
:param normalization:Specifies the kind of regularization to be applied. ``LogisticRegression::REG_L1`` or ``LogisticRegression::REG_L2`` (L1 norm or L2 norm). To use this, set ``LogisticRegressionParams.regularized`` to a integer greater than zero.
:param train_method:Specifies the kind of training method used. It should be set to either ``LogisticRegression::BATCH`` or ``LogisticRegression::MINI_BATCH``. If using ``LogisticRegression::MINI_BATCH``, set ``LogisticRegression::Params.mini_batch_size`` to a positive integer.
:param:reg: To enable or disable regularization. Set to positive integer (greater than zero) to enable and to 0 to disable.
:param normalization:Specifies the kind of regularization to be applied. ``LogisticRegression::REG_L1`` or ``LogisticRegression::REG_L2`` (L1 norm or L2 norm). To use this, set ``LogisticRegression::Params.regularized`` to a integer greater than zero.
:param:mini_batch_size: Specifies the number of training samples taken in each step of Mini-Batch Gradient Descent. Will only be used if using ``LogisticRegression::MINI_BATCH`` training algorithm. It has to take values less than the total number of training samples.
:param reg:To enable or disable regularization. Set to positive integer (greater than zero) to enable and to 0 to disable.
The full constructor initializes corresponding members. The default constructor creates an object with dummy parameters.
:param mini_batch_size:Specifies the number of training samples taken in each step of Mini-Batch Gradient Descent. Will only be used if using ``LogisticRegression::MINI_BATCH`` training algorithm. It has to take values less than the total number of training samples.
By initializing this structure, one can set all the parameters required for Logistic Regression classifier.
:param samples:The input data for the prediction algorithm. Matrix [m x n], where each row contains variables (features) of one object being classified. Should have data type ``CV_32F``.
:param data:The input data for the prediction algorithm. The ``data`` variable should be of type ``CV_32F``.
:param results:Predicted labels as a column matrix of type ``CV_32S``.
:param predicted_labels:Predicted labels as a column matrix and of type ``CV_32S``.
:param flags:Not used.
LogisticRegression::get_learnt_thetas
-------------------------------------
This function returns the trained paramters arranged across rows. For a two class classifcation problem, it returns a row matrix.
Maksim Shabunin
11 years ago