Camera spatial velocity computation through interaction matrix #3641
### Feature
The code computed camera spatial velocity given two images, pixels depth, camera matrix and the timestamp between the images. This is enabled by, so called, interaction matrix (usually utilized in visual servoing applications) relating pixel plane velocities to the camera spatial velocity in 3D i.e., twist - velocity and angular rate of the camera. The inverse problem can be solved by sampling pixel & their velocities to solve least-squares for twist. The relationship can be seen below in the picture.
The code does not include a proper example and is not tested but if there is interest I could contribute more appealing example and use case for camera velocity computation. However, I attach below a dummy example with random data simply to make sure that it's running as is. I have used this before in aiding UAV localization and thought someone else might benefit from it being integrated into `opencv`.
```c++
#include "opencv2/rgbd/twist.hpp"
#include "opencv2/core.hpp"
#include "opencv2/imgproc.hpp"
int main()
{
using namespace cv::rgbd;
Twist t;
// create two random image
cv::Mat im0(480, 640, CV_8UC1);
cv::Mat im1(480, 640, CV_8UC1);
cv::Mat depths0(480, 640, CV_32F);
cv::Mat K = cv::Mat::eye(3, 3, CV_32F);
cv::theRNG().state = time(0);
cv::randn(im0, 0, 255);
cv::randn(im1, 0, 255);
cv::randn(depths0, 0, 100);
cv::Vec6d twist = t.compute(im0, im1, depths0, K, 0.1);
return 0;
}
```
References
1. Chaumette, François, and Seth Hutchinson. "Visual servo control. I. Basic approaches." IEEE Robotics & Automation Magazine 13.4 (2006): 82-90.
2. https://robotacademy.net.au/lesson/image-based-visual-servoing/
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Bounding box fixes for GOTURN tracker
* Trying to change branch from master to 3.4
* Removing unnecessary if statement
* Frame existence check is added
* Changes in GOTURN tracker for reading path to caffemodel and prototxt files in opencv_extra
* Replacing get_data_path with findDataFile and moving paths in parameters
* tracking: fix GOTURN model loading
* Fixing GOTURN memory test: paths to GOTURN files are in cv::FileNode now
* Fix
* Fix
* Fixing comments
* Deleting trailing space
* Trying to solve the problem with test
* Removing ground truth reading part in GOTURN test
* Removing trailing spaces
* Trying to fix win64/win32 issue
* Fixing problem with win32/win64
* Trying to solve issue with win32/win64
Implementation of CSR-DCF tracker (#1552)
* Initial commit for CSR-DCF tracker implementation
* Fixes for automatic build
* General code fixes
* Removed unused parameters. Added CSRT to automatic tests.
* Fixed VS build warnings. Fixed a bug with gray sequences.
* Fixed VS build errors for samples file.
* kcf use float data type rather than double.
In our practice, float is good enough and could get better performance.
With this patch, one of my benchmark could get about 20% performance gain.
Signed-off-by: Zhigang Gong <zhigang.gong@intel.com>
* Offload transpose matrix multiplication to ocl.
The matrix multiplication in updateProjectMatrix is one of the
hotspot. And because of the matrix shape is special, say the
m is very short but the n is very large. The GEMM implementation
in neither the clBLAS nor the in trunk implementation are very
inefficient, I implement an standalone transpose matrix mulplication
kernel here. It can get about 10% performance gain on Intel
desktop platform or 20% performance gain on a braswell platform.
And in the mean time, the CPU utilization will be lower.
Signed-off-by: Zhigang Gong <zhigang.gong@intel.com>
* Add verification code for kcf ocl transpose mm kernel.
Signed-off-by: Zhigang Gong <zhigang.gong@linux.intel.com>
* tracking: show FPS in traker sample
* tracking: fix MSVC warnings in KCF
* tracking: move OCL kernel initialization to constructor in KCF
- made some of dependencies explicit
- removed dependencies to highgui and some other modules where possible
- modified some samples to build without modules
* Fix several issues in TrackerMedianFlow implementation
Particularly, add possibility to tune optical flow parameters for a median
flow tracker.
* Improve code of TrackerMedianFlow
Replace a lot of calls of std::vector::erase by one call of
std::remove_if.
* Delete unused code, use norm from OpenCV
* medianFlow:turn getMedian method into function, small code cleanup
* TrackerMedianFlow:fixes in parameters I/O, add test for them
* TrackerMedianFlow:replace double with float in temp buffers
* Fix indentation
* TrackerMedianFlow:add absent parameter case handling in read()
* TrackerMedianFlow:use ROI instead of copy when getting a patch
* TrackerMedianFlow:don't calc image pyramids 2 times
* MedianFlowTracker: use cvIsNan()
* MedianFlow: refactor vector filtration code
* MedianFlow: change if statements layout in filterPointsInVectors
For some applications it is useful to have an estimate of how uncertain
the specific variable is estimated. This could help to act accordingly
e.g. increase the measurement zone if the current estimate is very
uncertain.
catree
ernie
Gary Allen
Elizarov Ilya
Brian Wignall
Suleyman TURKMEN
berak
Alexander Alekhin
Andrej Muhič
klchang
Vladislav Sovrasov
Fernando Martin
Maksim Shabunin
Vladimir
Gregory Kramida
baudenri
103yiran
Kurnianggoro
SvetlanaFilicheva