Chiebot-Mirror/opencv - opencv - Gitea: Git with a cup of tea

Author	SHA1	Message	Date
Yuantao Feng	0521a3a384	Merge pull request #24476 from fengyuentau:attention_layer dnn: add attention layer #24476 Resolves #24609 Merge with: https://github.com/opencv/opencv_extra/pull/1128. Attention operator spec from onnxruntime: https://github.com/microsoft/onnxruntime/blob/v1.16.1/docs/ContribOperators.md#com.microsoft.Attention. TODO: - [x] benchmark (before this PR vs. with this PR vs. ORT). - [x] Layer fusion: Take care Slice with end=INT64_MAX. - [x] Layer fusion: match more potential attention (VIT) patterns. - [x] Single-head attention is supported. - [x] Test AttentionSubgraph fusion. - [x] Add acc tests for VIT_B_32 and VitTrack - [x] Add perf tests for VIT_B_32 and VitTrack ## Benchmarks Platform: Macbook Air M1. ### Attention Subgraph Input scale: [1, 197, 768]. \| \| mean (ms) \| median (ms) \| min (ms) \| \| ---------------------- \| --------- \| ----------- \| -------- \| \| w/ Attention (this PR) \| 3.75 \| 3.68 \| 3.22 \| \| w/o Attention \| 9.06 \| 9.01 \| 8.24 \| \| ORT (python) \| 4.32 \| 2.63 \| 2.50 \| ### ViTs All data in millisecond (ms). \| ViTs \| With Attention \| Without Attention \| ORT \| \| -------- \| -------------- \| ----------------- \| ------ \| \| vit_b_16 \| 302.77 \| 365.35 \| 109.70 \| \| vit_b_32 \| 89.92 \| 116.22 \| 30.36 \| \| vit_l_16 \| 1593.32 \| 1730.74 \| 419.92 \| \| vit_l_32 \| 468.11 \| 577.41 \| 134.12 \| \| VitTrack \| 3.80 \| 3.87 \| 2.25 \| ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake	1 year ago
Dmitry Kurtaev	332748dd55	Merge pull request #24577 from dkurt:dnn_graph_match_stack Fix graph fusion with commutative ops #24577 ### Pull Request Readiness Checklist resolves https://github.com/opencv/opencv/issues/24568 Merge with extra: https://github.com/opencv/opencv_extra/pull/1125 TODO: - [x] replace recursive function to sequential See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake	1 year ago
Yuantao Feng	a478757483	Merge pull request #24544 from fengyuentau:layernorm_conformance dnn test: move layer norm tests into conformance tests #24544 Merge with https://github.com/opencv/opencv_extra/pull/1122 ## Motivation Some ONNX operators, such as `LayerNormalization`, `BatchNormalization` and so on, produce outputs for training (mean, stdev). So they have reference outputs of conformance tests for those training outputs as well. However, when it comes to inference, we do not need and produce those outputs for training here in dnn. Hence, output size does not match if we use dnn to infer those conformance models. This has become the barrier if we want to test these operators using their conformance tests. <!-- \| Operator \| Inference needed \| Outputs (required - total) \| Optional outputs for training? \| \| ----------------------- \| ----------------------------------- \| -------------------------- \| ------------------------------ \| \| BatchNormalization \| Yes \| 1 - 3 \| Yes \| \| Dropout \| Maybe, can be eliminated via fusion \| 1 - 2 \| Yes \| \| GRU \| Yes \| 0 - 2 \| No \| \| LSTM \| Yes \| 0 - 3 \| No \| \| LayerNormalization \| Yes \| 1 - 3 \| Yes \| \| MaxPool \| Yes \| 1 - 2 \| Yes \| \| RNN \| Yes \| 0 - 2 \| No \| \| SoftmaxCrossEntropyLoss \| No \| 1 - 2 \| -- \| --> I checked all ONNX operators with optional outputs. Turns out there are only `BatchNormalization`, `Dropout`, `LayerNormalization` and `MaxPool` has optional outputs for training. All except `LayerNormalization` have models set for training mode and eval mode. Blame ONNX for that. ## Solution In this pull request, we remove graph outputs if the graph looks like the following: ``` [X] [Scale] [Bias] [X] [Scale] [Bias] \ \| / this patch \ \| / LayerNormalization -----------> LayerNormalization / \| \ \| [Y] [Mean] [Stdev] [Y] ``` We can update conformance tests and turn on some cases as well if extending to more layers. Notes: 1. This workaround does not solve expanded function operators if they are fused into a single operator, such as `$onnx/onnx/backend/test/data/node/test_layer_normalization_2d_axis1_expanded`, but they can be run without fusion. Note that either dnn or onnxruntime does not fuse those expanded function operators. ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake	1 year ago
Yuantao Feng	6079e22523	Merge pull request #24500 from fengyuentau:test_layer_fusion dnn (onnx): add subgraph fusion tests #24500 ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake	1 year ago