G-API: Support CUDA & TensoRT Execution Providers for ONNXRT Backend #24059
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [ ] I agree to contribute to the project under Apache 2 License.
- [ ] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
G-API: Support DirectML Execution Provider for ONNXRT Backend #24045
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [ ] I agree to contribute to the project under Apache 2 License.
- [ ] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
G-API: Support OpenVINO Execution Provider for ONNXRT Backend #24024
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [ ] I agree to contribute to the project under Apache 2 License.
- [ ] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
G-API: Fix incorrect OpaqueKind for Kernel outputs #23843
### Pull Request Readiness Checklist
#### Overview
The PR is going to fix several problems:
1. Major: `GKernel` doesn't hold `kind` for its outputs. Since `GModelBuilder` traverse graph from outputs to inputs once it reaches any output of the operation it will use its `kind` to create `Data` meta for all operation outputs. Since it essential for `python` to know `GTypeInfo` (which is `shape` and `kind`) it will be confused.
Consider this operation:
```
@cv.gapi.op('custom.square_mean', in_types=[cv.GArray.Int], out_types=[cv.GOpaque.Float, cv.GArray.Int])
class GSquareMean:
@staticmethod
def outMeta(desc):
return cv.empty_gopaque_desc(), cv.empty_array_desc()
```
Even though `GOpaque` is `Float`, corresponding metadata might have `Int` kind because it might be taken from `cv.GArray.Int`
so it will be a problem if one of the outputs of these operation is graph output because python will cast it to the wrong type based on `Data` meta.
2. Minor: Some of the OpenVINO `IR`'s doesn't any layout information for input. It's usually true only for `IRv10` but since `OpenVINO 2.0` need this information to correctly configure resize we need to put default layout if there no such assigned in `ov::Model`.
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [ ] I agree to contribute to the project under Apache 2 License.
- [ ] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
G-API: Refine Semantic Segmentation Demo #23766
### Overview
* Supported demo working with camera id (e.g `--input=0`)
* Supported 3d output segmentation models (e.g `deeplabv3`)
* Supported `desync` execution
* Supported higher camera resolution
* Changed the color map to pascal voc (https://cloud.githubusercontent.com/assets/4503207/17803328/1006ca80-65f6-11e6-9ff6-36b7ef5b9ac6.png)
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [ ] I agree to contribute to the project under Apache 2 License.
- [ ] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
[G-API] Implement OpenVINO 2.0 backend #23595
### Pull Request Readiness Checklist
Implemented basic functionality for `OpenVINO` 2.0 G-API backend.
#### Overview
- [x] Implement `Infer` kernel with some of essential configurable parameters + IR/Blob models format support.
- [ ] Implement the rest of kernels: `InferList`, `InferROI`, `Infer2` + other configurable params (e.g reshape)
- [x] Asyncrhonous execution support
- [ ] Remote context support
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
G-API: Integration branch for ONNX & Python-related changes #23597
# Changes overview
## 1. Expose ONNX backend's Normalization and Mean-value parameters in Python
* Since Python G-API bindings rely on `Generic` infer to express Inference, the `Generic` specialization of `onnx::Params` was extended with new methods to control normalization (`/255`) and mean-value; these methods were exposed in the Python bindings
* Found some questionable parts in the existing API which I'd like to review/discuss (see comments)
UPD:
1. Thanks to @TolyaTalamanov normalization inconsistencies have been identified with `squeezenet1.0-9` ONNX model itself; tests using these model were updated to DISABLE normalization and NOT using mean/value.
2. Questionable parts were removed and tests still pass.
### Details (taken from @TolyaTalamanov's comment):
`squeezenet1.0.*onnx` - doesn't require scaling to [0,1] and mean/std because the weights of the first convolution already scaled. ONNX documentation is broken. So the correct approach to use this models is:
1. ONNX: apply preprocessing from the documentation: https://github.com/onnx/models/blob/main/vision/classification/imagenet_preprocess.py#L8-L44 but without normalization step:
```
# DON'T DO IT:
# mean_vec = np.array([0.485, 0.456, 0.406])
# stddev_vec = np.array([0.229, 0.224, 0.225])
# norm_img_data = np.zeros(img_data.shape).astype('float32')
# for i in range(img_data.shape[0]):
# norm_img_data[i,:,:] = (img_data[i,:,:]/255 - mean_vec[i]) / stddev_vec[i]
# # add batch channel
# norm_img_data = norm_img_data.reshape(1, 3, 224, 224).astype('float32')
# return norm_img_data
# INSTEAD
return img_data.reshape(1, 3, 224, 224)
```
2. G-API: Convert image from BGR to RGB and then pass to `apply` as-is with configuring parameters:
```
net = cv.gapi.onnx.params('squeezenet', model_filename)
net.cfgNormalize('data_0', False)
```
**Note**: Results might be difference because `G-API` doesn't apply central crop but just do resize to model resolution.
---
`squeezenet1.1.*onnx` - requires scaling to [0,1] and mean/std - onnx documentation is correct.
1. ONNX: apply preprocessing from the documentation: https://github.com/onnx/models/blob/main/vision/classification/imagenet_preprocess.py#L8-L44
2. G-API: Convert image from BGR to RGB and then pass to `apply` as-is with configuring parameters:
```
net = cv.gapi.onnx.params('squeezenet', model_filename)
net.cfgNormalize('data_0', True) // default
net.cfgMeanStd('data_0', [0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
```
**Note**: Results might be difference because `G-API` doesn't apply central crop but just do resize to model resolution.
## 2. Expose Fluid & kernel package-related functionality in Python
* `cv::gapi::combine()`
* `cv::GKernelPackage::size()` (mainly for testing purposes)
* `cv::gapi::imgproc::fluid::kernels()`
Added a test for the above.
## 3. Fixed issues with Python stateful kernel handling
Fixed error message when `outMeta()` of custom python operation fails.
## 4. Fixed various issues in Python tests
1. `test_gapi_streaming.py` - fixed behavior of Desync test to avoid sporadic issues
2. `test_gapi_infer_onnx.py` - fixed model lookup (it was still using the ONNX Zoo layout but was NOT using the proper env var we use to point to one).
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
G-API: replace GAPI_Assert() with 'false' and '0' to GAPI_Error()
* gapi: GAPI_Error() macro
* gapi: replace GAPI_Assert() with 'false' and '0' to GAPI_Error()
* build: eliminate 'unreachable code' after CV_Error() (MSVC 2015)
* build: eliminate 'unreachable code' warning for MSVS 2015/2017
- observed in constructors stubs with throwing exception
4.x: submodule or a class scope for exported classes
* feature: submodule or a class scope for exported classes
All classes are registered in the scope that corresponds to C++
namespace or exported class.
Example:
`cv::ml::Boost` is exported as `cv.ml.Boost`
`cv::SimpleBlobDetector::Params` is exported as
`cv.SimpleBlobDetector.Params`
For backward compatibility all classes are registered in the global
module with their mangling name containing scope information.
Example:
`cv::ml::Boost` has `cv.ml_Boost` alias to `cv.ml.Boost` type
* refactor: remove redundant GAPI aliases
* fix: use explicit string literals in CVPY_TYPE macro
* fix: add handling for class aliases
G-API: Wrap GStreamerSource
* Wrap GStreamerSource into python
* Fixed test skipping when can't make Gst-src
* Wrapped GStreamerPipeline class, added dummy test for it
* Fix no_gst testing
* Changed wrap for GStreamerPipeline::getStreamingSource() : now python-specific in-class method GStreamerPipeline::get_streaming_source()
* Added accuracy tests vs OCV:VideoCapture(Gstreamer)
* Add skipping when can't use VideoCapture(GSTREAMER);
Add better handling of GStreamer backend unavailable;
Changed video to avoid terminations
* Applying comments
* back to a separate get_streaming_source function, with comment
Co-authored-by: OrestChura <orest.chura@intel.com>
[G-API] Extend compileStreaming to support different overloads
* Make different overloads
* Order python compileStreaming overloads
* Fix compileStreaming bug
* Replace
gin -> descr_of
* Set error message
* Fix review comments
* Use macros for pyopencv_to GMetaArgs
* Use GAPI_PROP_RW
* Not split Prims python stuff
G-API: Wrap render functionality to python
* Wrap render Rect prim
* Add all primitives and tests
* Cover mosaic and image
* Handle error in pyopencv_to(Prim)
* Move Mosaic and Rect ctors wrappers to shadow file
* Use GAPI_PROP_RW
* Fix indent
G-API: Support vaargs for cv.compile_args
* Support cv.compile_args to work with variadic number of inputs
* Disable python2.x G-API
* Move compile_args to gapi pkg
G-API: New python operations API
* Reimplement test using decorators
* Custom python operation API
* Remove wip status
* python: support Python code in bindings (through loader only)
* cleanup, skip tests for Python 2.x (not supported)
* python 2.x can't skip unittest modules
* Clean up
* Clean up
* Fix segfault python3.9
Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
[G-API] Introduce cv.gin/cv.descr_of for python
* Implement cv.gin/cv.descr_of
* Fix macos build
* Fix gcomputation tests
* Add test
* Add using to a void exceeded length for windows build
* Add using to a void exceeded length for windows build
* Fix comments to review
* Fix comments to review
* Update from latest master
* Avoid graph compilation to obtain in/out info
* Fix indentation
* Fix comments to review
* Avoid using default in switches
* Post output meta for giebackend
[G-API] Introduce GOpaque and GArray for python
* Introduce GOpaque and GArray for python
* Fix ctor
* Avoid code duplication by using macros
* gapi: move Python-specific files to misc/python
* Fix windows build
Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
* Implement G-API python bindings
* Fix hdr_parser
* Drop initlization with brackets using regexp
* Handle bracket initilization another way
* Add test for core operations
* Declaration and definition of View constructor now in different files
* Refactor tests
* Remove combine decorator from tests
* Fix comment to review
* Fix test
* Fix comments to review
* Remove GCompilerArgs implementation from python
Co-authored-by: Pinaev <danil.pinaev@intel.com>
Dmitry Kurtaev
Anatoliy Talamanov
Dmitry Matveev
Maksim Shabunin
Alexander Alekhin
xiong-jie-y
luzpaz
Vadim Levin
Aleksei Trutnev
Maxim Pashchenkov
Anastasiya(Asya) Pronina
