dnn: try improving performance of Attention layer #25076
Checklist:
- [x] Use `Mat` over `Mat::zeros` for temporary buffer in forward
- [x] Use layer internal buffer over temporary Mat buffer
- [x] Try a single fastGemmBatch on the Q/K/V calculation
Performance:
Performance test case is `Layer_Attention.VisionTransformer/0`, which has input of shape {1, 197, 768}, weight of shape {768, 2304} and bias {2304}.
Data is in millisecond.
| | macOS 14.2.1, Apple M1 | Ubuntu 22.04.2, Intel i7 12700K |
| - | - | - |
| Current | 10.96 | 1.58 |
| w/ Mat | 6.27 | 1.41 |
| w/ Internals | 5.87 | 1.38 |
| w/ fastGemmBatch | 6.12 | 2.14 |
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Added and tested yolov8x model #25095
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [X] I agree to contribute to the project under Apache 2 License.
- [X] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [X] The PR is proposed to the proper branch
- [X] There is a reference to the original bug report and related work
- [X] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [X] The feature is well documented and sample code can be built with the project CMake
Below is evidence of the test:
Fix issue #25077#25100
Fixes https://github.com/opencv/opencv/issues/25077
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Fixes#25056 : Optimising postProcess(const std::vector<Mat>& output_blobs) #25091
Like mentioned in the issue #25056 , I think checking the condition with `scoreThreshold` and then assigning the bounding boxes can optimize the function pretty well. By doing this, we prevent allocating boxes to faces with scores below the threshold. It also reduces the amount of data that needs to be processed during the subsequent NMS step. Builds and passed locally.
- [X] I agree to contribute to the project under Apache 2 License.
- [X] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [X] The PR is proposed to the proper branch
- [X] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Co-authored-by: Dhanwanth1803 <dhanwanthvarala@gmail,com>
Handle degenerate cases in RQDecomp3x3 #25050
The point of the Givens rotations here is to iteratively set the lower left matrix entries to zero. If an element is zero already, we don't need to do anything. This resolves#24330.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Move Aruco tutorials and samples to main repo #23018
merge with https://github.com/opencv/opencv_contrib/pull/3401
merge with https://github.com/opencv/opencv_extra/pull/1143
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
---------
Co-authored-by: AleksandrPanov <alexander.panov@xperience.ai>
Co-authored-by: Alexander Smorkalov <alexander.smorkalov@xperience.ai>
Compensate edge length in ChessBoardDetector::generateQuads (attempt 2) #25090
### Pull Request Readiness Checklist
New attempt for #24833, which was reverted as #25036.
Locally I fixed `Calib3d_StereoCalibrate_CPP.regression` test by corners refinement using `cornerSubPix` function
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Add compatibility with latest (3.1.54) emsdk version #25084
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
### Details
I was following [this tutorial](https://docs.opencv.org/4.9.0/d4/da1/tutorial_js_setup.html) for building opencv with wasm target. The tutorial mentions that the last verified version of emscripten that is tested with opencv is 2.0.10, but I was curious if I could get it to work with more recent versions. I've run into a few issues with the latest version, for which fixes are included in this PR. I've found a few issues that have the same problems I encountered:
- https://github.com/opencv/opencv/issues/24620
- https://github.com/opencv/opencv/issues/20313
- https://stackoverflow.com/questions/77469603/custom-opencv-js-wasm-using-cv-matfromarray-results-in-cv-mat-is-not-a-co
- https://github.com/emscripten-core/emscripten/issues/14803
- https://github.com/opencv/opencv/issues/24572
- https://github.com/opencv/opencv/issues/19493#issuecomment-857167996
I used the docker image for building and comparing results with different emsdk versions. I tested by building with `--build_wasm` and `--build-test` flags and ran the tests in the browser. I addressed the following issues with newer versions of emscripten:
- In newer versions `EMSCRIPTEN` environemnt variable was stopped being set. I added support for deriving location based on the `EMSDK` environment variable, as suggested [here](https://github.com/emscripten-core/emscripten/issues/14803)
- In newer versions emcmake started passing `-DCMAKE...` arguments, however the opencv python script didn't know how to handle them. I added processing to the args that will forward all arguments to `cmake` that start with `-D`. I opted for this in hopes of being more futureproof, but another approach could be just ignoreing them, or explicitly forwarding them instead of matching anything starting with `-D`. These approches were suggested [here](https://github.com/opencv/opencv/issues/19493#issuecomment-855529448)
- With [version 3.1.31](https://github.com/emscripten-core/emscripten/blob/main/ChangeLog.md#3131---012623) some previously exported functions stopped being automatically exported. Because of this, `_free` and `_malloc` were no longer available and had to be explicitly exported because of breaking tests.
- With [version 3.1.42](https://github.com/emscripten-core/emscripten/compare/3.1.41...3.1.42#diff-e505aa80b2764c0197acfc9afd8179b3600f0ab5dd00ff77db01879a84515cdbL3875) the `post-js` code doesn't receive the module named as `EXPORT_NAME` anymore, but only as `moduleArg`/`Module`. This broke existing code in `helpers.js`, which was referencing exported functions through `cv.Mat`, etc. I changed all of these references to use `Module.Mat`, etc. If it is preferred, alternatively the `cv` variable could be reintroduced in `helper.js` as suggested [here](https://github.com/opencv/opencv/issues/24620)
With the above changes in place, I can successfully build and run tests with the latest emscripten/emsdk docker image (also with 2.0.10 and most of the other older tags, except for a few that contain transient issues like [this](https://github.com/emscripten-core/emscripten/issues/17700)).
This is my first time contributing to opencv, so I hope I got everything correct in this PR, but please let me know if I should change anything!
G-API: Make test execution lighter (first attempt) #25060
### Background
G-API tests look running longer than tests for the rest of modules (e.g., 5m), and the analysis show that there's several outliers in G-API test suite which take a lot of time but don't improve the testing quality much:
In this PR I will cut the execution time to something reasonable.
### Contents
- Marked some outliers as `verylong`:
- OneVPL barrier test - pure brute force
- Stateful test in stream - in fact BG Sub accuracy test clone
- Restructured parameters instantiation in Streaming tests to generate less test configurations (54 -> 36)
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Fix very slow compilation of five-point algorithm on some platforms (e.g. Qualcomm) #25064
Thanks to our big friend and long-term contributor for the patch!
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Added kotlin classes to AAR #24884
The resulting maven repo doesn't have kotlin-plugin dependency, and it works fine out of the box: Android Kotlin projects already have kotlin-plugin dependency, Android Java projects ignore kotlin classes.
Details on KGP versions: https://kotlinlang.org/docs/gradle-configure-project.html#apply-the-plugin
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Co-authored-by: Alexander Lyulkov <alexander.lyulkov@opencv.ai>
Co-authored-by: Alexander Smorkalov <alexander.smorkalov@xperience.ai>
G-API: Lower supported IE backend version #25054
Related to https://github.com/opencv/opencv/issues/25053
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Replace legacy __ARM_NEON__ by __ARM_NEON #25024
Even ACLE 1.1 referes to __ARM_NEON
https://developer.arm.com/documentation/ihi0053/b/?lang=en
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Fixes#24974 support HardSwishInt8 #24985
As given very clearly in the issue #24974 I made the required 2 changes to implement HardSwish Layer in INT8. Requesting comments.
resolves https://github.com/opencv/opencv/issues/24974
- [X] I agree to contribute to the project under Apache 2 License.
- [X] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [X] The PR is proposed to the proper branch
- [X] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Co-authored-by: Dhanwanth1803 <dhanwanthvarala@gmail,com>
solvePnP implementation for Fisheye camera model #25028
Credits to Linfei Pan
Extracted from https://github.com/opencv/opencv/pull/24052
**Warning:** The patch changes Obj-C generator behaviour and adds "fisheye_" prefix for all ObjC functions from namespace.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Co-authored-by: lpanaf <linpan@student.ethz.ch>
Co-authored-by: Vadim Levin <vadim.levin@xperience.ai>
Fix barcode detectAndDecode #25035
The method `detectAndDecode()` in the `BarcodeDetector` class doesn't return the barcode corners.
This PR fixes the and add test for `detectAndDecode`.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Fix qrcode bugs #25026
This PR fixes#22892, #24011 and #24450 and adds regression tests using the images provided. I've also verified with the [benchmark](https://github.com/opencv/opencv_benchmarks/tree/develop/python_benchmarks/qr_codes) that this doesn't break anything there.
resolves#22892resolves#24011resolves#24450
Replaces #23802
Requires extra: https://github.com/opencv/opencv_extra/pull/1148
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Yuantao Feng
Alexander Smorkalov
Alessandro de Oliveira Faria (A.K.A.CABELO)
Laurent Berger
Dhanwanth1803
Alexander Smorkalov
Adrian Kretz
catree
Sergei Shutov
Maxim Smolskiy
Dan Rosser
Székely Gábor
Misha Klatis
Ryo H
Markus Murschitz
刘雄康
Dmitry Matveev
Vadim Pisarevsky
alexlyulkov
Matthias Möller
Anatoliy Talamanov
Alexander Alekhin
Vincent Rabaud
Alex