G-API: Implement concurrent executor #24845
## Overview
This PR introduces the new G-API executor called `GThreadedExecutor` which can be selected when the `GComputation` is compiled in `serial` mode (a.k.a `GComputation::compile(...)`)
### ThreadPool
`cv::gapi::own::ThreadPool` has been introduced in order to abstract usage of threads in `GThreadedExecutor`.
`ThreadPool` is implemented by using `own::concurrent_bounded_queue`
`ThreadPool` has only as single method `schedule` that will push task into the queue for the further execution.
The **important** notice is that if `Task` executed in `ThreadPool` throws exception - this is `UB`.
### GThreadedExecutor
The `GThreadedExecutor` is mostly copy-paste of `GExecutor`, should we extend `GExecutor` instead?
#### Implementation details
1. Build the dependency graph for `Island` nodes.
2. Store the tasks that don't have dependencies into separate `vector` in order to run them first.
3. at the `GThreadedExecutor::run()` schedule the tasks that don't have dependencies that will schedule their dependents and wait for the completion.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [ ] I agree to contribute to the project under Apache 2 License.
- [ ] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
- desync() is a new (and for now, the only one) intrinsic
which splits the graph execution into asynchronous parts
when running in Streaming mode;
- desync() makes no effect when compiling in Traditional mode;
- Added tests on desync() working in various scenarios;
- Extended GStreamingExecutor to support desync(); also extended
GStreamingCompiled() with a new version of pull() returning a
vector of optional values;
- Fixed various issues with storing the type information & proper
construction callbacks for GArray<> and GOpaque;
- Introduced a new infer(Roi,GMat) overload with a sample;
- Introduced an internal API for Islands to control fusion
procedure (to fuse or not to fuse);
- Introduced handleStopStream() callback for island executables;
- Added GCompileArgs to metadata of the graph (required for other
features).
* G-API-NG/Streaming: Introduced a Streaming API
Now a GComputation can be compiled in a special "streaming" way
and then "played" on a video stream.
Currently only VideoCapture is supported as an input source.
* G-API-NG/Streaming: added threading & real streaming
* G-API-NG/Streaming: Added tests & docs on Copy kernel
- Added very simple pipeline tests, not all data types are covered yet
(in fact, only GMat is tested now);
- Started testing non-OCV backends in the streaming mode;
- Added required fixes to Fluid backend, likely it works OK now;
- Added required fixes to OCL backend, and now it is likely broken
- Also added a UMat-based (OCL) version of Copy kernel
* G-API-NG/Streaming: Added own concurrent queue class
- Used only if TBB is not available
* G-API-NG/Streaming: Fixing various issues
- Added missing header to CMakeLists.txt
- Fixed various CI issues and warnings
* G-API-NG/Streaming: Fixed a compile-time GScalar queue deadlock
- GStreamingExecutor blindly created island's input queues for
compile-time (value-initialized) GScalars which didn't have any
producers, making island actor threads wait there forever
* G-API-NG/Streaming: Dropped own version of Copy kernel
One was added into master already
* G-API-NG/Streaming: Addressed GArray<T> review comments
- Added tests on mov()
- Removed unnecessary changes in garray.hpp
* G-API-NG/Streaming: Added Doxygen comments to new public APIs
Also fixed some other comments in the code
* G-API-NG/Streaming: Removed debug info, added some comments & renamed vars
* G-API-NG/Streaming: Fixed own-vs-cv abstraction leak
- Now every island is triggered with own:: (instead of cv::)
data objects as inputs;
- Changes in Fluid backend required to support cv::Mat/Scalar were
reverted;
* G-API-NG/Streaming: use holds_alternative<> instead of index/index_of test
- Also fixed regression test comments
- Also added metadata check comments for GStreamingCompiled
* G-API-NG/Streaming: Made start()/stop() more robust
- Fixed various possible deadlocks
- Unified the shutdown code
- Added more tests covering different corner cases on start/stop
* G-API-NG/Streaming: Finally fixed Windows crashes
In fact the problem hasn't been Windows-only.
Island thread popped data from queues without preserving the Cmd
objects and without taking the ownership over data acquired so when
islands started to process the data, this data may be already freed.
Linux version worked only by occasion.
* G-API-NG/Streaming: Fixed (I hope so) Windows warnings
* G-API-NG/Streaming: fixed typos in internal comments
- Also added some more explanation on Streaming/OpenCL status
* G-API-NG/Streaming: Added more unit tests on streaming
- Various start()/stop()/setSource() call flow combinations
* G-API-NG/Streaming: Added tests on own concurrent bounded queue
* G-API-NG/Streaming: Added more tests on various data types, + more
- Vector/Scalar passed as input;
- Vector/Scalar passed in-between islands;
- Some more assertions;
- Also fixed a deadlock problem when inputs are mixed (1 constant, 1 stream)
* G-API-NG/Streaming: Added tests on output data types handling
- Vector
- Scalar
* G-API-NG/Streaming: Fixed test issues with IE + Windows warnings
* G-API-NG/Streaming: Decoupled G-API from videoio
- Now the core G-API doesn't use a cv::VideoCapture directly,
it comes in via an abstract interface;
- Polished a little bit the setSource()/start()/stop() semantics,
now setSource() is mandatory before ANY call to start().
* G-API-NG/Streaming: Fix STANDALONE build (errors brought by render)
* G-API-NG/API: Introduced inference API and IE-based backend
- Very quick-n-dirty implementation
- OpenCV's own DNN module is not used
- No tests so far
* G-API-NG/IE: Refined IE backend, added more tests
* G-API-NG/IE: Fixed various CI warnings & build issues + tests
- Added tests on multi-dimensional own::Mat
- Added tests on GMatDesc with dimensions
- Documentation on infer.hpp
- Fixed more warnings + added a ROI list test
- Fix descr_of clash for vector<Mat> & standalone mode
- Fix build issue with gcc-4.8x
- Addressed review comments
* G-API-NG/IE: Addressed review comments
- Pass `false` to findDataFile()
- Add deprecation warning suppression macros for IE
* Update G-API code base to 27-Sep-18
Changes mostly improve standalone build support
* G-API code base update 28-09-2018
* Windows/Documentation warnings should be fixed
* Fixed stability issues in Fluid backend
* Fixed precompiled headers issues in G-API source files
* G-API code base update 28-09-18 EOD
* Fixed several static analysis issues
* Fixed issues found when G-API is built in a standalone mode
* G-API Initial code upload
* Update G-API code base to Sep-24-2018
* The majority of OpenCV buildbot problems was addressed
* Update G-API code base to 24-Sep-18 EOD
* G-API code base update 25-Sep-2018
* Linux warnings should be resolved
* Documentation build should become green
* Number of Windows warnings should be reduced
* Update G-API code base to 25-Sep-18 EOD
* ARMv7 build issue should be resolved
* ADE is bumped to latest version and should fix Clang builds for macOS/iOS
* Remaining Windows warnings should be resolved
* New Linux32 / ARMv7 warnings should be resolved
* G-API code base update 25-Sep-2018-EOD2
* Final Windows warnings should be resolved now
* G-API code base update 26-Sep-2018
* Fixed issues with precompiled headers in module and its tests