Some interface was refined (most notably, the method for returning
Hessian was removed and the method for getting gradient was added as
optional to base Solver::Function class) and basic code for
setters/getters was added. Now is the time for the real work on an
algorithm.
As the opencv's build-bot did not want to compile this revision, I had
to do some changes. In particular,
1) Removed unsigned int vs int comparisons, that were treated as errors
2) Removed unused variables and functions
3) Removed functions without previous declaration
4) Fixed whitespaces
This is an implementation of primal-dual algorithm, based on the C++
source code by Vadim Pisarevsky. It was extended to handle the denoising
based on multiple observations. It also contains documentation and
tests.
In response to the pull request comments by Vadim Pisarevsky. In
particular, the following was done:
*)cv::reduce use instead of custom code for calculating
per-coordinate sum
*) naming style of private methods is made consisted with overall --
mixed-case style
*) irrelevant create()'s were removed -- I did not know that copyTo()
method itself calls create
Request to comments on pull request for simplex method. In particular
*) while(1) is replaced with for(;;)
*) if(true){...} constructions in tests are replaced with #if 1 ...
#endif
This is done by keeping indexToRow vector, that keeps the information,
opposite to those kept by N and B. That is, while N and B help to
determine which variable corresponds to given column in column-vector c
or row in matrix b, indexToRow helps to determine the corresponding
row/column for a given variable.
At this point, I'm waiting for comments from pull request reviewer and
not working on any upgrades. Comments are appreciated, as usual.
Use opencv's print() procedure in place of my own procedures to output
matrices and std::vectors.
Interestingly enough, operator<< does not work for matrices, when called
from my .cpp files in src/ subfolder of the optim module, although it
works when called from tests and stand-alone programs, compiled with
opencv. I think, this requires investigation and, maybe, bug report.
Attempting to fix issues pointed out by Vadim Pisarevsky during the pull
request review. In particular, the following things are done:
*) The mechanism of debug info printing is changed and made more
procedure-style than the previous macro-style
*) z in solveLP() is now returned as a column-vector
*) Func parameter of solveLP() is now allowed to be column-vector, in
which case it is understood to be the transpose of what we need
*) Func and Constr now can contain floats, not only doubles (in the
former case the conversion is done via convertTo())
*)different constructor to allocate space for z in solveLP() is used,
making the size of z more explicit (this is just a notation change, not
functional, both constructors are achieving the same goal)
*) (big) mat.hpp and iostream headers are moved to precomp-headers from
optim.hpp
Fixed the code so to eliminate warnings related to shadowing and unused
parameters. In some settings, these warnings may be treated as an errors
and lead to failed build.
Suggested by Nikita Manovich.
Change qualifiers on auxiliary functions (for solveLP() procedure) from
const (that does not have much sense) to static (that makes them
invisible for outside world and hopefully exacerbates optimization).
In particular, the following things are done:
*) Consistent tabulation of 4 spaces is ensured
*) New function dprintf() is introduced, so now printing of the debug
information can be turned on/off via the ALEX_DEBUG macro
*) Removed solveLP_aux namespace
*) All auxiliary functions are declared as static
*) The return codes of solveLP() are encapsulated in enum.
This version is supposed to work on all problems (please, let me know if
this is not so), but is not optimized yet in terms of numerical
stability and performance. Bland's rule is implemented as well, so
algorithm is supposed to allow no cycling. Additional check for multiple
solutions is added (in case of multiple solutions algorithm returns an
appropriate return code of 1 and returns arbitrary optimal solution).
Finally, now we have 5 tests.
Before Thursday we have 4 directions that can be tackled in parallel:
*) Prepare the pull request!
*) Make the code more clear and readable (refactoring)
*) Wrap the core solveLP() procedure in OOP-style interface
*) Test solveLP on non-trivial tests (possibly test against
http://www.coin-or.org/Clp/)
What we have now corresponds to "formal simplex algorithm", described in
Cormen's "Intro to Algorithms". It will work *only* if the initial
problem has (0,0,0,...,0) as feasible solution (consequently, it will
work unpredictably if problem was unfeasible or did not have zero-vector as
feasible solution). Moreover, it might cycle.
TODO (first priority)
1. Implement initialize_simplex() procedure, that shall check for
feasibility and generate initial feasible solution. (in particular, code
should pass all 4 tests implemented at the moment)
2. Implement Bland's rule to avoid cycling.
3. Make the code more clear.
4. Implement several non-trivial tests (??) and check algorithm against
them. Debug if necessary.
TODO (second priority)
1. Concentrate on stability and speed (make difficult tests)
Added LPSolver class together with two nested classes: LPFunction and
LPConstraints. These represent function to be maximized and constraints
imposed respectively. They are implementations of interfaces Function
and Constraints respectively (latter ones are nested classes of Solver
interface, which is generic interface for all optimization algorithms to
be implemented within this project).
The next step is to implement the simplex algorithm! First, we shall
implement it for the case of constraints of the form Ax<=b and x>=0.
Then, we shall extend the sets of problems that can be handled by the
conversion to the one we've handled already. Finally, we shale
concentrate on numerical stability and efficiency.
At this point we have a skeleton of a new module (optim) which can
barely compile properly (unlike previous commit). Besides, there is a
first draft of solver and lpsolver (linear optimization solver) in this
commit.
Generic optimization package for openCV project, will be developed
between the June and September of 2013. This work is funded by Google
Summer of Code 2013 project. This project is about
implementing several algorithms, that will find global maxima/minima of a
given function on a given domain subject to a given constraints.
All comments/suggestions are warmly appreciated and to be sent to
alozz1991@gmail.com (please, mention the word "openCV" in topic of
message, for I'm using the spam-filters)
Ilya Lavrenov
Alex Leontiev
Alexander Alekhin
Roman Donchenko