Camera spatial velocity computation through interaction matrix #3641
### Feature
The code computed camera spatial velocity given two images, pixels depth, camera matrix and the timestamp between the images. This is enabled by, so called, interaction matrix (usually utilized in visual servoing applications) relating pixel plane velocities to the camera spatial velocity in 3D i.e., twist - velocity and angular rate of the camera. The inverse problem can be solved by sampling pixel & their velocities to solve least-squares for twist. The relationship can be seen below in the picture.
The code does not include a proper example and is not tested but if there is interest I could contribute more appealing example and use case for camera velocity computation. However, I attach below a dummy example with random data simply to make sure that it's running as is. I have used this before in aiding UAV localization and thought someone else might benefit from it being integrated into `opencv`.
```c++
#include "opencv2/rgbd/twist.hpp"
#include "opencv2/core.hpp"
#include "opencv2/imgproc.hpp"
int main()
{
using namespace cv::rgbd;
Twist t;
// create two random image
cv::Mat im0(480, 640, CV_8UC1);
cv::Mat im1(480, 640, CV_8UC1);
cv::Mat depths0(480, 640, CV_32F);
cv::Mat K = cv::Mat::eye(3, 3, CV_32F);
cv::theRNG().state = time(0);
cv::randn(im0, 0, 255);
cv::randn(im1, 0, 255);
cv::randn(depths0, 0, 100);
cv::Vec6d twist = t.compute(im0, im1, depths0, K, 0.1);
return 0;
}
```
References
1. Chaumette, François, and Seth Hutchinson. "Visual servo control. I. Basic approaches." IEEE Robotics & Automation Magazine 13.4 (2006): 82-90.
2. https://robotacademy.net.au/lesson/image-based-visual-servoing/
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Bounding box fixes for GOTURN tracker
* Trying to change branch from master to 3.4
* Removing unnecessary if statement
* Frame existence check is added
* Changes in GOTURN tracker for reading path to caffemodel and prototxt files in opencv_extra
* Replacing get_data_path with findDataFile and moving paths in parameters
* tracking: fix GOTURN model loading
* Fixing GOTURN memory test: paths to GOTURN files are in cv::FileNode now
* Fix
* Fix
* Fixing comments
* Deleting trailing space
* Trying to solve the problem with test
* Removing ground truth reading part in GOTURN test
* Removing trailing spaces
* Trying to fix win64/win32 issue
* Fixing problem with win32/win64
* Trying to solve issue with win32/win64
Implementation of CSR-DCF tracker (#1552)
* Initial commit for CSR-DCF tracker implementation
* Fixes for automatic build
* General code fixes
* Removed unused parameters. Added CSRT to automatic tests.
* Fixed VS build warnings. Fixed a bug with gray sequences.
* Fixed VS build errors for samples file.
catree
ernie
Maksim Shabunin
Alexander Smorkalov
Alexander Alekhin
Tomoaki Teshima
Gary Allen
Vincent Rabaud
Elizarov Ilya
Kunal Tyagi
berak
Brian Wignall
Ahmed Ashour
Suleyman TURKMEN
Mohammad Haghighat
tompollok
Alexander Alekhin
Alireza Davoudi
Andrej Muhič
klchang
Vitaly Tuzov
klchang
Vladislav Sovrasov