|
|
|
@ -3,29 +3,30 @@ |
|
|
|
|
@prev_tutorial{tutorial_dnn_javascript} |
|
|
|
|
|
|
|
|
|
## Introduction |
|
|
|
|
Deep learning is a fast growing area. The new approaches to build neural networks |
|
|
|
|
usually introduce new types of layers. They could be modifications of existing |
|
|
|
|
ones or implement outstanding researching ideas. |
|
|
|
|
Deep learning is a fast-growing area. New approaches to building neural networks |
|
|
|
|
usually introduce new types of layers. These could be modifications of existing |
|
|
|
|
ones or implementation of outstanding research ideas. |
|
|
|
|
|
|
|
|
|
OpenCV gives an opportunity to import and run networks from different deep learning |
|
|
|
|
frameworks. There are a number of the most popular layers. However you can face |
|
|
|
|
a problem that your network cannot be imported using OpenCV because of unimplemented layers. |
|
|
|
|
OpenCV allows importing and running networks from different deep learning frameworks. |
|
|
|
|
There is a number of the most popular layers. However, you can face a problem that |
|
|
|
|
your network cannot be imported using OpenCV because some layers of your network |
|
|
|
|
can be not implemented in the deep learning engine of OpenCV. |
|
|
|
|
|
|
|
|
|
The first solution is to create a feature request at https://github.com/opencv/opencv/issues |
|
|
|
|
mentioning details such a source of model and type of new layer. A new layer could |
|
|
|
|
be implemented if OpenCV community shares this need. |
|
|
|
|
mentioning details such as a source of a model and a type of new layer. |
|
|
|
|
The new layer could be implemented if the OpenCV community shares this need. |
|
|
|
|
|
|
|
|
|
The second way is to define a **custom layer** so OpenCV's deep learning engine |
|
|
|
|
The second way is to define a **custom layer** so that OpenCV's deep learning engine |
|
|
|
|
will know how to use it. This tutorial is dedicated to show you a process of deep |
|
|
|
|
learning models import customization. |
|
|
|
|
learning model's import customization. |
|
|
|
|
|
|
|
|
|
## Define a custom layer in C++ |
|
|
|
|
Deep learning layer is a building block of network's pipeline. |
|
|
|
|
It has connections to **input blobs** and produces results to **output blobs**. |
|
|
|
|
There are trained **weights** and **hyper-parameters**. |
|
|
|
|
Layers' names, types, weights and hyper-parameters are stored in files are generated by |
|
|
|
|
native frameworks during training. If OpenCV mets unknown layer type it throws an |
|
|
|
|
exception trying to read a model: |
|
|
|
|
Layers' names, types, weights and hyper-parameters are stored in files are |
|
|
|
|
generated by native frameworks during training. If OpenCV encounters unknown |
|
|
|
|
layer type it throws an exception while trying to read a model: |
|
|
|
|
|
|
|
|
|
``` |
|
|
|
|
Unspecified error: Can't create layer "layer_name" of type "MyType" in function getLayerInstance |
|
|
|
@ -61,7 +62,7 @@ This method should create an instance of you layer and return cv::Ptr with it. |
|
|
|
|
|
|
|
|
|
@snippet dnn/custom_layers.hpp MyLayer::getMemoryShapes |
|
|
|
|
|
|
|
|
|
Returns layer's output shapes depends on input shapes. You may request an extra |
|
|
|
|
Returns layer's output shapes depending on input shapes. You may request an extra |
|
|
|
|
memory using `internals`. |
|
|
|
|
|
|
|
|
|
- Run a layer |
|
|
|
@ -71,20 +72,20 @@ memory using `internals`. |
|
|
|
|
Implement a layer's logic here. Compute outputs for given inputs. |
|
|
|
|
|
|
|
|
|
@note OpenCV manages memory allocated for layers. In the most cases the same memory |
|
|
|
|
can be reused between layers. So your `forward` implementation should not rely that |
|
|
|
|
the second invocation of `forward` will has the same data at `outputs` and `internals`. |
|
|
|
|
can be reused between layers. So your `forward` implementation should not rely on that |
|
|
|
|
the second invocation of `forward` will have the same data at `outputs` and `internals`. |
|
|
|
|
|
|
|
|
|
- Optional `finalize` method |
|
|
|
|
|
|
|
|
|
@snippet dnn/custom_layers.hpp MyLayer::finalize |
|
|
|
|
|
|
|
|
|
The chain of methods are the following: OpenCV deep learning engine calls `create` |
|
|
|
|
method once then it calls `getMemoryShapes` for an every created layer then you |
|
|
|
|
can make some preparations depends on known input dimensions at cv::dnn::Layer::finalize. |
|
|
|
|
After network was initialized only `forward` method is called for an every network's input. |
|
|
|
|
The chain of methods is the following: OpenCV deep learning engine calls `create` |
|
|
|
|
method once, then it calls `getMemoryShapes` for every created layer, then you |
|
|
|
|
can make some preparations depend on known input dimensions at cv::dnn::Layer::finalize. |
|
|
|
|
After network was initialized only `forward` method is called for every network's input. |
|
|
|
|
|
|
|
|
|
@note Varying input blobs' sizes such height or width or batch size you make OpenCV |
|
|
|
|
reallocate all the internal memory. That leads efficiency gaps. Try to initialize |
|
|
|
|
@note Varying input blobs' sizes such height, width or batch size make OpenCV |
|
|
|
|
reallocate all the internal memory. That leads to efficiency gaps. Try to initialize |
|
|
|
|
and deploy models using a fixed batch size and image's dimensions. |
|
|
|
|
|
|
|
|
|
## Example: custom layer from Caffe |
|
|
|
@ -201,7 +202,7 @@ deep learning model. That was trained with one and only difference comparing to |
|
|
|
|
a current version of [Caffe framework](http://caffe.berkeleyvision.org/). `Crop` |
|
|
|
|
layers that receive two input blobs and crop the first one to match spatial dimensions |
|
|
|
|
of the second one used to crop from the center. Nowadays Caffe's layer does it |
|
|
|
|
from the top-left corner. So using the latest version of Caffe or OpenCV you'll |
|
|
|
|
from the top-left corner. So using the latest version of Caffe or OpenCV you will |
|
|
|
|
get shifted results with filled borders. |
|
|
|
|
|
|
|
|
|
Next we're going to replace OpenCV's `Crop` layer that makes top-left cropping by |
|
|
|
@ -217,7 +218,7 @@ a centric one. |
|
|
|
|
|
|
|
|
|
@snippet dnn/edge_detection.py Register |
|
|
|
|
|
|
|
|
|
That's it! We've replaced an implemented OpenCV's layer to a custom one. |
|
|
|
|
That's it! We have replaced an implemented OpenCV's layer to a custom one. |
|
|
|
|
You may find a full script in the [source code](https://github.com/opencv/opencv/tree/3.4/samples/dnn/edge_detection.py). |
|
|
|
|
|
|
|
|
|
<table border="0"> |
|
|
|
|