Tag:
Branch:
Tree:
c96f48e7c9
2.4
3.4
4.x
5.x
master
next
2.2
2.3.0
2.3.1
2.4.0
2.4.1
2.4.10
2.4.10.1
2.4.10.2
2.4.10.3
2.4.10.4
2.4.11
2.4.12
2.4.12.1
2.4.12.2
2.4.12.3
2.4.13
2.4.13.1
2.4.13.2
2.4.13.3
2.4.13.4
2.4.13.5
2.4.13.6
2.4.13.7
2.4.2
2.4.3
2.4.3-rc
2.4.3.1
2.4.3.2
2.4.4
2.4.4-beta
2.4.5
2.4.6
2.4.6.1
2.4.6.2
2.4.6.2-rc1
2.4.6.2r2
2.4.6.2r3
2.4.7
2.4.7-rc1
2.4.7.1
2.4.7.2
2.4.8
2.4.8.1
2.4.8.2
2.4.8.3
2.4.9
2.4.9.1
3.0-ocl-tech-preview
3.0-ocl-tp2
3.0.0
3.0.0-alpha
3.0.0-beta
3.0.0-rc1
3.1.0
3.2.0
3.2.0-rc
3.3.0
3.3.0-cvsdk
3.3.0-rc
3.3.1
3.3.1-cvsdk
3.4.0
3.4.0-rc
3.4.1
3.4.1-cvsdk
3.4.10
3.4.11
3.4.12
3.4.13
3.4.14
3.4.15
3.4.16
3.4.17
3.4.18
3.4.19
3.4.2
3.4.2-openvino
3.4.20
3.4.3
3.4.3-openvino
3.4.4
3.4.5
3.4.6
3.4.7
3.4.8
3.4.9
4.0.0
4.0.0-alpha
4.0.0-beta
4.0.0-openvino
4.0.0-rc
4.0.1
4.0.1-openvino
4.1.0
4.1.0-openvino
4.1.1
4.1.1-openvino
4.1.2
4.1.2-openvino
4.10.0
4.10.0-kleidicv
4.11.0
4.2.0
4.2.0-openvino
4.3.0
4.3.0-openvino
4.3.0-openvino-2020.3.0
4.4.0
4.4.0-openvino
4.5.0
4.5.0-openvino
4.5.1
4.5.1-openvino
4.5.2
4.5.2-openvino
4.5.3
4.5.3-openvino
4.5.3-openvino-2021.4.1
4.5.3-openvino-2021.4.2
4.5.4
4.5.5
4.5.5-openvino-2022.1.0
4.6.0
4.7.0
4.8.0
4.8.1
4.9.0
5.0.0-alpha
${ noResults }
1 Commits (c96f48e7c91748f896e99f5bf943faf08cd98f0e)
| Author | SHA1 | Message | Date |
|---|---|---|---|
Dmitry Matveev
|
fc5d412ba7
|
Merge pull request #23597 from dmatveev:dm/gapi_onnx_py_integration
G-API: Integration branch for ONNX & Python-related changes #23597 # Changes overview ## 1. Expose ONNX backend's Normalization and Mean-value parameters in Python * Since Python G-API bindings rely on `Generic` infer to express Inference, the `Generic` specialization of `onnx::Params` was extended with new methods to control normalization (`/255`) and mean-value; these methods were exposed in the Python bindings * Found some questionable parts in the existing API which I'd like to review/discuss (see comments) UPD: 1. Thanks to @TolyaTalamanov normalization inconsistencies have been identified with `squeezenet1.0-9` ONNX model itself; tests using these model were updated to DISABLE normalization and NOT using mean/value. 2. Questionable parts were removed and tests still pass. ### Details (taken from @TolyaTalamanov's comment): `squeezenet1.0.*onnx` - doesn't require scaling to [0,1] and mean/std because the weights of the first convolution already scaled. ONNX documentation is broken. So the correct approach to use this models is: 1. ONNX: apply preprocessing from the documentation: https://github.com/onnx/models/blob/main/vision/classification/imagenet_preprocess.py#L8-L44 but without normalization step: ``` # DON'T DO IT: # mean_vec = np.array([0.485, 0.456, 0.406]) # stddev_vec = np.array([0.229, 0.224, 0.225]) # norm_img_data = np.zeros(img_data.shape).astype('float32') # for i in range(img_data.shape[0]): # norm_img_data[i,:,:] = (img_data[i,:,:]/255 - mean_vec[i]) / stddev_vec[i] # # add batch channel # norm_img_data = norm_img_data.reshape(1, 3, 224, 224).astype('float32') # return norm_img_data # INSTEAD return img_data.reshape(1, 3, 224, 224) ``` 2. G-API: Convert image from BGR to RGB and then pass to `apply` as-is with configuring parameters: ``` net = cv.gapi.onnx.params('squeezenet', model_filename) net.cfgNormalize('data_0', False) ``` **Note**: Results might be difference because `G-API` doesn't apply central crop but just do resize to model resolution. --- `squeezenet1.1.*onnx` - requires scaling to [0,1] and mean/std - onnx documentation is correct. 1. ONNX: apply preprocessing from the documentation: https://github.com/onnx/models/blob/main/vision/classification/imagenet_preprocess.py#L8-L44 2. G-API: Convert image from BGR to RGB and then pass to `apply` as-is with configuring parameters: ``` net = cv.gapi.onnx.params('squeezenet', model_filename) net.cfgNormalize('data_0', True) // default net.cfgMeanStd('data_0', [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]) ``` **Note**: Results might be difference because `G-API` doesn't apply central crop but just do resize to model resolution. ## 2. Expose Fluid & kernel package-related functionality in Python * `cv::gapi::combine()` * `cv::GKernelPackage::size()` (mainly for testing purposes) * `cv::gapi::imgproc::fluid::kernels()` Added a test for the above. ## 3. Fixed issues with Python stateful kernel handling Fixed error message when `outMeta()` of custom python operation fails. ## 4. Fixed various issues in Python tests 1. `test_gapi_streaming.py` - fixed behavior of Desync test to avoid sporadic issues 2. `test_gapi_infer_onnx.py` - fixed model lookup (it was still using the ONNX Zoo layout but was NOT using the proper env var we use to point to one). ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake |
2 years ago |