Merge pull request #12857 from dmatveev:hld

* G-API Documentation: first submission

This PR introduces a number of new OpenCV documentation chapters for
Graph API module.

In particular, the following topics are covered:
- Introduction & background information;
- High-level design overview;
- Kernel API;
- Pipeline example.

All changes are done in Markdown files, no headers, etc modified.

Doxygen references for main API classes will be added later.

Also, a tutorial will be introduced soon (in the common Tutorials place)

* G-API Documentation - fix warnings & trailing whitespaces

* G-API Documentation: address review issues

* G-API Documentation: export code snippets to compileable files

* gapi: move documentation samples
pull/12922/head
Dmitry Matveev 6 years ago committed by Alexander Alekhin
parent ea31c09384
commit dbed39a931
  1. 1
      modules/gapi/CMakeLists.txt
  2. 109
      modules/gapi/doc/00-root.markdown
  3. 76
      modules/gapi/doc/01-background.markdown
  4. 159
      modules/gapi/doc/10-hld-overview.md
  5. 170
      modules/gapi/doc/20-kernel-api.markdown
  6. 27
      modules/gapi/doc/30-implementation.markdown
  7. 17
      modules/gapi/doc/dot/kernel_hierarchy.dot
  8. 3
      modules/gapi/doc/intro.markdown
  9. BIN
      modules/gapi/doc/pics/demo.jpg
  10. BIN
      modules/gapi/doc/pics/gapi_scheme.png
  11. BIN
      modules/gapi/doc/pics/kernel_hierarchy.png
  12. 4
      modules/gapi/include/opencv2/gapi/cpu/gcpukernel.hpp
  13. 6
      modules/gapi/include/opencv2/gapi/gkernel.hpp
  14. 34
      modules/gapi/samples/api_example.cpp
  15. 157
      modules/gapi/samples/kernel_api_snippets.cpp

@ -92,3 +92,4 @@ if(TARGET opencv_test_gapi)
endif()
ocv_add_perf_tests()
ocv_add_samples()

@ -0,0 +1,109 @@
# Graph API {#gapi}
# Introduction {#gapi_root_intro}
OpenCV Graph API (or G-API) is a new OpenCV module targeted to make
regular image processing fast and portable. These two goals are
achieved by introducing a new graph-based model of execution.
G-API is a special module in OpenCV -- in contrast with the majority
of other main modules, this one acts as a framework rather than some
specific CV algorithm. G-API provides means to define CV operations,
construct graphs (in form of expressions) using it, and finally
implement and run the operations for a particular backend.
# Contents
G-API documentation is organized into the following chapters:
- @subpage gapi_purposes
The motivation behind G-API and its goals.
- @subpage gapi_hld
General overview of G-API architecture and its major internal
components.
- @subpage gapi_kernel_api
Learn how to introduce new operations in G-API and implement it for
various backends.
- @subpage gapi_impl
Low-level implementation details of G-API, for those who want to
contribute.
- API Reference: functions and classes
- @subpage gapi_core
Core G-API operations - arithmetic, boolean, and other matrix
operations;
- @subpage gapi_imgproc
Image processing functions: color space conversions, various
filters, etc.
# API Example {#gapi_example}
A very basic example of G-API pipeline is shown below:
@include modules/gapi/samples/api_example.cpp
<!-- TODO align this code with text using marks and itemized list -->
G-API is a separate OpenCV module so its header files have to be
included explicitly. The first four lines of `main()` create and
initialize OpenCV's standard video capture object, which fetches
video frames from either an attached camera or a specified file.
G-API pipelie is constructed next. In fact, it is a series of G-API
operation calls on cv::GMat data. The important aspect of G-API is
that this code block is just a declaration of actions, but not the
actions themselves. No processing happens at this point, G-API only
tracks which operations form pipeline and how it is connected. G-API
_Data objects_ (here it is cv::GMat) are used to connect operations
each other. `in` is an _empty_ cv::GMat signalling that it is a
beginning of computation.
After G-API code is written, it is captured into a call graph with
instantiation of cv::GComputation object. This object takes
input/output data references (in this example, `in` and `out`
cv::GMat objects, respectively) as parameters and reconstructs the
call graph based on all the data flow between `in` and `out`.
cv::GComputation is a thin object in sense that it just captures which
operations form up a computation. However, it can be used to execute
computations -- in the following processing loop, every captured frame (a
cv::Mat `input_frame`) is passed to cv::GComputation::apply().
![Example pipeline running on sample video 'vtest.avi'](pics/demo.jpg)
cv::GComputation::apply() is a polimorphic method which accepts a
variadic number of arguments. Since this computation is defined on one
input, one output, a special overload of cv::GComputation::apply() is
used to pass input data and get output data.
Internally, cv::GComputation::apply() compiles the captured graph for
the given input parameters and executes the compiled graph on data
immediately.
There is a number important concepts can be outlines with this examle:
* Graph declaration and graph execution are distinct steps;
* Graph is built implicitly from a sequence of G-API expressions;
* G-API supports function-like calls -- e.g. cv::gapi::resize(), and
operators, e.g operator|() which is used to compute bitwise OR;
* G-API syntax aims to look pure: every operation call within a graph
yields a new result, thus forming a directed acyclic graph (DAG);
* Graph declaration is not bound to any data -- real data objects
(cv::Mat) come into picture after the graph is already declared.
<!-- FIXME: The above operator|() link links to MatExpr not GAPI -->
See Tutorial[TBD] and Porting examples[TBD] to learn more on various
G-API features and concepts.
<!-- TODO Add chapter on declaration, compilation, execution -->

@ -0,0 +1,76 @@
# Why Graph API? {#gapi_purposes}
# Motivation behind G-API {#gapi_intro_why}
G-API module brings graph-based model of execution to OpenCV. This
chapter briefly describes how this new model can help software
developers in two aspects: optimizing and porting image processing
algorithms.
## Optimizing with Graph API {#gapi_intro_opt}
Traditionally OpenCV provided a lot of stand-alone image processing
functions (see modules `core` and `imgproc`). Many of that functions
are well-optimized (e.g. vectorized for specific CPUs, parallel, etc)
but still the out-of-box optimization scope has been limited to a
single function only -- optimizing the whole algorithm built atop of that
functions was a responsibility of a programmer.
OpenCV 3.0 introduced _Transparent API_ (or _T-API_) which allowed to
offload OpenCV function calls transparently to OpenCL devices and save
on Host/Device data transfers with cv::UMat -- and it was a great step
forward. However, T-API is a dynamic API -- user code still remains
unconstrained and OpenCL kernels are enqueued in arbitrary order, thus
eliminating further pipeline-level optimization potential.
G-API brings implicit graph model to OpenCV 4.0. Graph model captures
all operations and its data dependencies in a pipeline and so provides
G-API framework with extra information to do pipeline-level
optimizations.
The cornerstone of graph-based optimizations is _Tiling_. Tiling
allows to break the processing into smaller parts and reorganize
operations to enable data parallelism, improve data locality, and save
memory footprint. Data locality is an especially important aspect of
software optimization due to diffent costs of memory access on modern
computer architectures -- the more data is reused in the first level
cache, the more efficient pipeline is.
Definitely the aforementioned techinques can be applied manually --
but it requires extra skills and knowledge of the target platform and
the algorithm implementation changes irrevocably -- becoming more
specific, less flexible, and harder to extend and maintain.
G-API takes this responsiblity and complexity from user and does the
majority of the work by itself, keeping the algorithm code clean from
device or optimization details. This approach has its own limitations,
though, as graph model is a _constrained_ model and not every
algorithm can be represented as a graph, so the G-API scope is limited
only to regular image processing -- various filters, arithmentic,
binary operations, and well-defined geometrical transformations.
## Porting with Graph API {#gapi_intro_port}
The essense of G-API is declaring a sequence of operations to run, and
then executing that sequence. G-API is a constrained API, so it puts a
number of limitations on which operations can form a pipeline and
which data these operations may exchange each other.
This formalization in fact helps to make an algorithm portable. G-API
clearly separates operation _interfaces_ from its _implementations_.
One operation (_kernel_) may have multiple implementations even for a
single device (e.g., OpenCV-based "reference" implementation and a
tiled optimized implementation, both running on CPU). Graphs (or
_Computations_ in G-API terms) are built only using operation
interfaces, not implementations -- thus the same graph can be executed
on different devices (and, of course, using different optimization
techniques) with little-to-no changes in the graph itself.
G-API supports plugins (_Backends_) which aggreate logic and
intelligence on what is the best way to execute on a particular
platform. Once a pipeline is built with G-API, it can be parametrized
to use either of the backends (or a combination of it) and so a graph
can be ported easily to a new platform.
@sa @ref gapi_hld

@ -0,0 +1,159 @@
# High-level design overview {#gapi_hld}
# G-API High-level design overview
[TOC]
G-API is a heterogeneous framework and provides single API to program
image processing pipelines with a number of supported backends.
The key design idea is to keep pipeline code itself platform-neutral
while specifying which kernels to use and which devices to utilize
using extra parameters at graph compile (configuration) time. This
requirement has led to the following architecture:
<!-- FIXME: Render from dot directly -->
![G-API framework architecture](pics/gapi_scheme.png)
There are three layers in this architecture:
* **API Layer** -- this is the top layer, which implements G-API
public interface, its building blocks and semantics.
When user constructs a pipeline with G-API, he interacts with this
layer directly, and the entities the user operates on (like cv::GMat
or cv::GComputation) are provided by this layer.
* **Graph Compiler Layer** -- this is the intermediate layer which
unrolls user computation into a graph and then applies a number of
transformations to it (e.g. optimizations). This layer is built atop
of [ADE Framework](@ref gapi_detail_ade).
* **Backends Layer** -- this is the lowest level layer, which lists a
number of _Backends_. In contrast with the above two layers,
backends are highly coupled with low-level platform details, with
every backend standing for every platform. A backend operates on a
processed graph (coming from the graph compiler) and executes this
graph optimally for a specific platform or device.
# API layer {#gapi_api_layer}
API layer is what user interacts with when defining and using a
pipeline (a Computation in G-API terms). API layer defines a set of
G-API _dynamic_ objects which can be used as inputs, outputs, and
intermediate data objects within a graph:
* cv::GMat
* cv::GScalar
* cv::GArray (template class)
API layer specifies a list of Operations which are defined on these
data objects -- so called kernels. See G-API [core](@ref gapi_core)
and [imgproc](@ref gapi_imgproc) namespaces for details on which
operations G-API provides by default.
G-API is not limited to these operations only -- users can define
their own kernels easily using a special macro G_TYPED_KERNEL().
API layer is also responsible for marshalling and storing operation
parameters on pipeline creation. In addition to the aforementioned
G-API dynamic objects, operations may also accept arbitrary
parameters (more on this [below](@ref gapi_detail_params)), so API
layer captures its values and stores internally upon the moment of
execution.
Finally, cv::GComputation and cv::GCompiled are the remaining
important components of API layer. The former wraps a series of G-API
expressions into an object (graph), and the latter is a product of
graph _compilation_ (see [this chapter](@ref gapi_detail_compiler) for
details).
# Graph compiler layer {#gapi_compiler}
Every G-API computation is compiled before it executes. Compilation
process is triggered in two ways:
* _implicitly_, when cv::GComputation::apply() is used. In this case,
graph compilation is then immediately followed by execution.
* _explicitly_, when cv::GComputation::compile() is used. In this case,
a cv::GCompiled object is returned which then can be invoked as a
C++ functor.
The first way is recommended for cases when input data format is not
known in advance -- e.g. when it comes from an arbitrary input file.
The second way is recommended for deployment (production) scenarios
where input data characteristics are usually predefined.
Graph compilation process is built atop of ADE Framework. Initially, a
bipartite graph is generated from expressions captured by API layer.
This graph contains nodes of two types: _Data_ and _Operations_. Graph
always starts and ends with a Data node(s), with Operations nodes
in-between. Every Operation node has inputs and outputs, both are Data
nodes.
After the initial graph is generated, it is actually processed by a
number of graph transformations, called _passes_. ADE Framework acts
as a compiler pass management engine, and passes are written
specifically for G-API.
There are different passes which check graph validity, refine details
on operations and data, organize nodes into clusters ("Islands") based
on affinity or user-specified regioning[TBD], and more. Backends also
are able to inject backend-specific passes into the compilation
process, see more on this in the [dedicated chapter](@ref gapi_detail_meta).
Result of graph compilation is a compiled object, represented by class
cv::GCompiled. A new cv::GCompiled object is always created regardless
if there was an explicit or implicit compilation request (see
above). Actual graph execution happens within cv::GCompiled and is
determined by backends which participated in the graph compilation.
@sa cv::GComputation::apply(), cv::GComputation::compile(), cv::GCompiled
# Backends layer {#gapi_backends}
The above diagram lists two backends, _OpenCV_ and _Fluid_. _OpenCV_
is so-called "reference backend", which implements G-API operations
using plain old OpenCV functions. This backend is useful for
prototyping on a familiar development system. _Fluid_ is a plugin for
cache-efficient execution on CPU -- it implements a different
execution policy and operates with its own, special kernels. Fluid
backend allows to achieve less memory footprint and better memory
locality when running on CPU.
There may be more backends available, e.g. Halide, OpenCL, etc. --
G-API provides an uniform internal API to develop backends so any
enthusiast or a company are free to scale G-API on a new platform or
accelerator. In terms of OpenCV infrastructure, every new backend is a
new distinct OpenCV module, which extends G-API when build as a part
of OpenCV.
# Graph execution {#gapi_compiled}
The way graph executed is defined by backends selected for
compilation. In fact, every backend builds its own execution script as
the final stage of graph compilation process, when an executable
(compiled) object is being generated. For example, in OpenCV backend,
this script is just a topologically-sorted sequence of OpenCV
functions to call; for Fluid backend, it is a similar thing -- a
topologically sorted list of _Agents_ processing lines of input on
every iteration.
Graph execution is triggered in two ways:
* via cv::GComputation::apply(), with graph compiled in-place exactly
for the given input data;
* via cv::GCompiled::operator()(), when the graph has been precompiled.
Both methods are polimorphic and take a variadic number of arguments,
with validity checks performed in runtime. If a number, shapes, and
formats of passed data objects differ from expected, a run-time
exception is thrown. G-API also provides _typed_ wrappers to move
these checks to the compile time -- see cv::GComputationT<>.
G-API graph execution is declared stateless -- it means that a
compiled functor (cv::GCompiled) acts like a pure C++ function and
provides the same result for the same set of input arguments.
Both execution methods take \f$N+M\f$ parameters, where \f$N\f$ is a
number of inputs, and \f$M\f$ is a number of outputs on which a
cv::GComputation is defined. Note that while G-API types (cv::GMat,
etc) are used in definition, the execution methods accept OpenCV's
traditional data types (like cv::Mat) which hold actual data -- see
table in [parameter marshalling](@#gapi_detail_params).
@sa @ref gapi_impl, @ref gapi_kernel_api

@ -0,0 +1,170 @@
# Kernel API {#gapi_kernel_api}
[TOC]
# G-API Kernel API
The core idea behind G-API is portability -- a pipeline built with
G-API must be portable (or at least able to be portable). It means
that either it works out-of-the box when compiled for new platform,
_or_ G-API provides necessary tools to make it running there, with
little-to-no changes in the algorithm itself.
This idea can be achieved by separating kernel interface from its
implementation. Once a pipeline is built using kernel interfaces, it
becomes implementation-neutral -- the implementation details
(i.e. which kernels to use) are passed on a separate stage (graph
compilation).
Kernel-implementation hierarchy may look like:
![Kernel API/implementation hierarchy example](pics/kernel_hierarchy.png)
A pipeline itself then can be expressed only in terms of `A`, `B`, and
so on, and choosing which implementation to use in execution becomes
an external parameter.
# Defining a kernel {#gapi_defining_kernel}
G-API provides a macro to define a new kernel interface --
G_TYPED_KERNEL():
@snippet modules/gapi/samples/kernel_api_snippets.cpp filter2d_api
This macro is a shortcut to a new type definition. It takes three
arguments to register a new type, and requires type body to be present
(see [below](@ref gapi_kernel_supp_info)). The macro arguments are:
1. Kernel interface name -- also serves as a name of new type defined
with this macro;
2. Kernel signature -- an `std::function<>`-like signature which defines
API of the kernel;
3. Kernel's unique name -- used to identify kernel when its type
informattion is stripped within the system.
Kernel declaration may be seen as function declaration -- in both cases
a new entity must be used then according to the way it was defined.
Kernel signature defines kernel's usage syntax -- which parameters
it takes during graph construction. Implementations can also use this
signature to derive it into backend-specific callback signatures (see
next chapter).
Kernel may accept values of any type, and G-API _dynamic_ types are
handled in a special way. All other types are opaque to G-API and
passed to kernel in `outMeta()` or in execution callbacks as-is.
Kernel's return value can _only_ be of G-API dynamic type -- cv::GMat,
cv::GScalar, or cv::GArray<T>. If an operation has more than one output,
it should be wrapped into an `std::tuple<>` (which can contain only
mentioned G-API types). Arbitrary-output-number operations are not
supported.
Once a kernel is defined, it can be used in pipelines with special,
G-API-supplied method "::on()". This method has the same signature as
defined in kernel, so this code:
@snippet modules/gapi/samples/kernel_api_snippets.cpp filter2d_on
is a perfectly legal construction. This example has some verbosity,
though, so usually a kernel declaration comes with a C++ function
wrapper ("factory method") which enables optional parameters, more
compact syntax, Doxygen comments, etc:
@snippet modules/gapi/samples/kernel_api_snippets.cpp filter2d_wrap
so now it can be used like:
@snippet modules/gapi/samples/kernel_api_snippets.cpp filter2d_wrap_call
# Extra information {#gapi_kernel_supp_info}
In the current version, kernel declaration body (everything within the
curly braces) must contain a static function `outMeta()`. This function
establishes a functional dependency between operation's input and
output metadata.
_Metadata_ is an information about data kernel operates on. Since
non-G-API types are opaque to G-API, G-API cares only about `G*` data
descriptors (i.e. dimensions and format of cv::GMat, etc).
`outMeta()` is also an example of how kernel's signature can be
transformed into a derived callback -- note that in this example,
`outMeta()` signature exactly follows the kernel signature (defined
within the macro) but is different -- where kernel expects cv::GMat,
`outMeta()` takes and returns cv::GMatDesc (a G-API structure metadata
for cv::GMat).
The point of `outMeta()` is to propagate metadata information within
computation from inputs to outputs and infer metadata of internal
(intermediate, temporary) data objects. This information is required
for further pipeline optimizations, memory allocation, and other
operations done by G-API framework during graph compilation.
<!-- TODO add examples -->
# Implementing a kernel {#gapi_kernel_implementing}
Once a kernel is declared, its interface can be used to implement
versions of this kernel in different backends. This concept is
naturally projected from object-oriented programming
"Interface/Implementation" idiom: an interface can be implemented
multiple times, and different implementations of a kernel should be
substitutable with each other without breaking the algorithm
(pipeline) logic (Liskov Substitution Principle).
Every backend defines its own way to implement a kernel interface.
This way is regular, though -- whatever plugin is, its kernel
implementation must be "derived" from a kernel interface type.
Kernel implementation are then organized into _kernel
packages_. Kernel packages are passed to cv::GComputation::compile()
as compile arguments, with some hints to G-API on how to select proper
kernels (see more on this in "Heterogeneity"[TBD]).
For example, the aforementioned `Filter2D` is implemented in
"reference" CPU (OpenCV) plugin this way (*NOTE* -- this is a
simplified form with improper border handling):
@snippet modules/gapi/samples/kernel_api_snippets.cpp filter2d_ocv
Note how CPU (OpenCV) plugin has transformed the original kernel
signature:
- Input cv::GMat has been substituted with cv::Mat, holding actual input
data for the underlying OpenCV function call;
- Output cv::GMat has been transformed into extra output parameter, thus
`GCPUFilter2D::run()` takes one argument more than the original
kernel signature.
The basic intuition for kernel developer here is _not to care_ where
that cv::Mat objects come from instead of the original cv::GMat -- and
just follow the signature conventions defined by the plugin. G-API
will call this method during execution and supply all the necessary
information (and forward the original opaque data as-is).
# Compound kernels
Sometimes kernel is a single thing only on API level. It is convenient
for users, but on a particular implementation side it would be better to
have multiple kernels (a subgraph) doing the thing instead. An example
is goodFeaturesToTrack() -- while in OpenCV backend it may remain a
single kernel, with Fluid it becomes compound -- Fluid can handle Harris
response calculation but can't do sparse non-maxima suppression and
point extraction to an STL vector:
<!-- PIC -->
A compound kernel _implementation_ can be defined using a generic
macro GAPI_COMPOUND_KERNEL():
@snippet modules/gapi/samples/kernel_api_snippets.cpp compound
<!-- TODO: ADD on how Compound kernels may simplify dispatching -->
<!-- TODO: Add details on when expand() is called! -->
It is important to distinguish a compound kernel from G-API high-order
function, i.e. a C++ function which looks like a kernel but in fact
generates a subgraph. The core difference is that a compound kernel is
an _implementation detail_ and a kernel implementation may be either
compound or not (depending on backend capabilities), while a
high-order function is a "macro" in terms of G-API and so cannot act as
an interface which then needs to be implemented by a backend.

@ -0,0 +1,27 @@
# Implementation details {#gapi_impl}
[TOC]
# G-API Implementation details {#gapi_impl_header}
## Api layer details {#gapi_detail_api}
### Expression unrolling {#gapi_detail_expr}
### Parameter marshalling {#gapi_detail_params}
### Operations representation {#gapi_detail_operations}
## Graph compiler details {#gapi_detail_compiler}
### ADE basics {#gapi_detail_ade}
### Graph model representation {#gapi_detail_gmodel}
### G-API metadata and passes {#gapi_detail_meta}
## Backends details {#gapi_detail_backends}
### Backend scope of work {#gapi_backend_scope}
### Graph transformation {#gapi_backend_pass}

@ -0,0 +1,17 @@
digraph {
rankdir=BT;
node [shape=record];
ki_a [label="{<f0> interface\nA}"];
ki_b [label="{<f0> interface\nB}"];
{rank=same; ki_a ki_b};
"CPU::A" -> ki_a [dir="forward"];
"OpenCL::A" -> ki_a [dir="forward"];
"Halide::A" -> ki_a [dir="forward"];
"CPU::B" -> ki_b [dir="forward"];
"OpenCL::B" -> ki_b [dir="forward"];
"Halide::B" -> ki_b [dir="forward"];
}

@ -1,3 +0,0 @@
# Graph API {#gapi}
Introduction to G-API (WIP).

Binary file not shown.

After

Width:  |  Height:  |  Size: 64 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.6 KiB

@ -8,10 +8,10 @@
#ifndef OPENCV_GAPI_GCPUKERNEL_HPP
#define OPENCV_GAPI_GCPUKERNEL_HPP
#include <vector>
#include <functional>
#include <map>
#include <unordered_map>
#include <utility>
#include <vector>
#include <opencv2/core/mat.hpp>
#include <opencv2/gapi/gcommon.hpp>

@ -10,11 +10,11 @@
#include <functional>
#include <iostream>
#include <unordered_map> // map (for GKernelPackage)
#include <vector> // lookup order
#include <string> // string
#include <utility> // tuple
#include <type_traits> // false_type, true_type
#include <unordered_map> // map (for GKernelPackage)
#include <utility> // tuple
#include <vector> // lookup order
#include <opencv2/gapi/gcommon.hpp> // CompileArgTag
#include <opencv2/gapi/util/util.hpp> // Seq

@ -0,0 +1,34 @@
#include <opencv2/videoio.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/gapi.hpp>
#include <opencv2/gapi/core.hpp>
#include <opencv2/gapi/imgproc.hpp>
int main(int argc, char *argv[])
{
cv::VideoCapture cap;
if (argc > 1) cap.open(argv[1]);
else cap.open(0);
CV_Assert(cap.isOpened());
cv::GMat in;
cv::GMat vga = cv::gapi::resize(in, cv::Size(), 0.5, 0.5);
cv::GMat gray = cv::gapi::BGR2Gray(vga);
cv::GMat blurred = cv::gapi::blur(gray, cv::Size(5,5));
cv::GMat edges = cv::gapi::Canny(blurred, 32, 128, 3);
cv::GMat b,g,r;
std::tie(b,g,r) = cv::gapi::split3(vga);
cv::GMat out = cv::gapi::merge3(b, g | edges, r);
cv::GComputation ac(in, out);
cv::Mat input_frame;
cv::Mat output_frame;
CV_Assert(cap.read(input_frame));
do
{
ac.apply(input_frame, output_frame);
cv::imshow("output", output_frame);
} while (cap.read(input_frame) && cv::waitKey(30) < 0);
return 0;
}

@ -0,0 +1,157 @@
// [filter2d_api]
#include <opencv2/gapi.hpp>
G_TYPED_KERNEL(GFilter2D,
<cv::GMat(cv::GMat,int,cv::Mat,cv::Point,double,int,cv::Scalar)>,
"org.opencv.imgproc.filters.filter2D")
{
static cv::GMatDesc // outMeta's return value type
outMeta(cv::GMatDesc in , // descriptor of input GMat
int ddepth , // depth parameter
cv::Mat /* coeffs */, // (unused)
cv::Point /* anchor */, // (unused)
double /* scale */, // (unused)
int /* border */, // (unused)
cv::Scalar /* bvalue */ ) // (unused)
{
return in.withDepth(ddepth);
}
};
// [filter2d_api]
cv::GMat filter2D(cv::GMat ,
int ,
cv::Mat ,
cv::Point ,
double ,
int ,
cv::Scalar);
// [filter2d_wrap]
cv::GMat filter2D(cv::GMat in,
int ddepth,
cv::Mat k,
cv::Point anchor = cv::Point(-1,-1),
double scale = 0.,
int border = cv::BORDER_DEFAULT,
cv::Scalar bval = cv::Scalar(0))
{
return GFilter2D::on(in, ddepth, k, anchor, scale, border, bval);
}
// [filter2d_wrap]
// [compound]
#include <opencv2/gapi/gcompoundkernel.hpp> // GAPI_COMPOUND_KERNEL()
using PointArray2f = cv::GArray<cv::Point2f>;
G_TYPED_KERNEL(HarrisCorners,
<PointArray2f(cv::GMat,int,double,double,int,double)>,
"org.opencv.imgproc.harris_corner")
{
static cv::GArrayDesc outMeta(const cv::GMatDesc &,
int,
double,
double,
int,
double)
{
// No special metadata for arrays in G-API (yet)
return cv::empty_array_desc();
}
};
// Define Fluid-backend-local kernels which form GoodFeatures
G_TYPED_KERNEL(HarrisResponse,
<cv::GMat(cv::GMat,double,int,double)>,
"org.opencv.fluid.harris_response")
{
static cv::GMatDesc outMeta(const cv::GMatDesc &in,
double,
int,
double)
{
return in.withType(CV_32F, 1);
}
};
G_TYPED_KERNEL(ArrayNMS,
<PointArray2f(cv::GMat,int,double)>,
"org.opencv.cpu.nms_array")
{
static cv::GArrayDesc outMeta(const cv::GMatDesc &,
int,
double)
{
return cv::empty_array_desc();
}
};
GAPI_COMPOUND_KERNEL(GFluidHarrisCorners, HarrisCorners)
{
static PointArray2f
expand(cv::GMat in,
int maxCorners,
double quality,
double minDist,
int blockSize,
double k)
{
cv::GMat response = HarrisResponse::on(in, quality, blockSize, k);
return ArrayNMS::on(response, maxCorners, minDist);
}
};
// Then implement HarrisResponse as Fluid kernel and NMSresponse
// as a generic (OpenCV) kernel
// [compound]
// [filter2d_ocv]
#include <opencv2/gapi/cpu/gcpukernel.hpp> // GAPI_OCV_KERNEL()
#include <opencv2/imgproc.hpp> // cv::filter2D()
GAPI_OCV_KERNEL(GCPUFilter2D, GFilter2D)
{
static void
run(const cv::Mat &in, // in - derived from GMat
const int ddepth, // opaque (passed as-is)
const cv::Mat &k, // opaque (passed as-is)
const cv::Point &anchor, // opaque (passed as-is)
const double delta, // opaque (passed as-is)
const int border, // opaque (passed as-is)
const cv::Scalar &, // opaque (passed as-is)
cv::Mat &out) // out - derived from GMat (retval)
{
cv::filter2D(in, out, ddepth, k, anchor, delta, border);
}
};
// [filter2d_ocv]
int main(int, char *[])
{
std::cout << "This sample is non-complete. It is used as code snippents in documentation." << std::endl;
cv::Mat conv_kernel_mat;
{
// [filter2d_on]
cv::GMat in;
cv::GMat out = GFilter2D::on(/* GMat */ in,
/* int */ -1,
/* Mat */ conv_kernel_mat,
/* Point */ cv::Point(-1,-1),
/* double */ 0.,
/* int */ cv::BORDER_DEFAULT,
/* Scalar */ cv::Scalar(0));
// [filter2d_on]
}
{
// [filter2d_wrap_call]
cv::GMat in;
cv::GMat out = filter2D(in, -1, conv_kernel_mat);
// [filter2d_wrap_call]
}
return 0;
}
Loading…
Cancel
Save