opencv/modules/matlab/README.md

OpenCV Matlab Code Generator

This module contains a code generator to automatically produce Matlab mex wrappers for other modules within the OpenCV library. Once compiled and added to the Matlab path, this gives users the ability to call OpenCV methods natively from within Matlab.

Build

The Matlab code generator is fully integrated into the OpenCV build system. If cmake finds a Matlab installation available on the host system while configuring OpenCV, it will attempt to generate Matlab wrappers for all OpenCV modules. If cmake is having trouble finding your Matlab installation, you can explicitly point it to the root by defining the MATLAB_ROOT_DIR variable. For example, on a Mac you could type:

cmake -DMATLAB_ROOT_DIR=/Applications/MATLAB_R2013a.app ..

Install

In order to use the bindings, you will need to add them to the Matlab path. The path to add is:

1. ${CMAKE_BUILD_DIR}/modules/matlab if you are working from the build tree, or
2. ${CMAKE_INSTALL_PREFIX}/matlab if you have installed OpenCV

In Matlab, simply run:

addpath('/path/to/opencv/matlab/');

Run

Once you've added the bindings directory to the Matlab path, you can start using them straight away! OpenCV calls need to be prefixed with a 'cv' qualifier, to disambiguate them from Matlab methods of the same name. For example, to compute the dft of a matrix, you might do the following:

% load an image (Matlab)
I = imread('cameraman.tif');

% compute the DFT (OpenCV)
If = cv.dft(I, cv.DFT_COMPLEX_OUTPUT);

As you can see, both OpenCV methods and constants can be used with 'cv' qualification. You can also call:

help cv.dft

to get help on the purpose and call signature of a particular method, or

help cv

to get general help regarding the OpenCV bindings. If you ever run into issues with the bindings

cv.buildInformation();

will produce a printout of diagnostic information pertaining to your particular build of OS, OpenCV and Matlab. It is useful to submit this information alongside a bug report to the OpenCV team.

Writing your own mex files

The Matlab bindings come with a set of utilities to help you quickly write your own mex files using OpenCV definitions. By doing so, you have all the speed and freedom of C++, with the power of OpenCV's math expressions and optimizations.

The first thing you need to learn how to do is write a mex-file with Matlab constructs. Following is a brief example:

// include useful constructs
// this automatically includes opencv core.hpp and mex.h)
#include <opencv2/matlab/bridge.hpp>
using namespace cv;
using namespace matlab;
using namespace bridge;

// define the mex gateway
void mexFunction(int nlhs, mxArray* plhs[],
                 int nrhs, const mxArray* prhs[]) {

  // claim the inputs into scoped management
  MxArrayVector raw(prhs, prhs+nrhs);

  // add an argument parser to automatically handle basic options
  ArgumentParser parser("my function");
  parser.addVariant(1, 1, "opt");
  MxArrayVector reordered = parser.parse(raw);

  // if we get here, we know the inputs are valid and reordered. Unpack...
  BridgeVector inputs(reordered.begin(), reordered.end());
  Mat required    = inputs[0].toMat();
  string optional = inputs[1].empty() ? "Default string" : inputs[1].toString();

  try {
    // Do stuff...
  } catch(Exception& e) {
    error(e.what());
  } catch(...) {
    error("Uncaught exception occurred");
  }

  // allocate an output
  Bridge out = required;
  plhs[0] = out.toMxArray().releaseOwnership();
}

There are a couple of important things going on in this example. Firstly, you need to include <opencv2/matlab/bridge.hpp> to enable the bridging capabilities. Once you've done this, you get some nice utilities for free. MxArray is a class that wraps Matlab's mxArray* class in an OOP-style interface. ArgumentParser is a class that handles default, optional and named arguments for you, along with multiple possible calling syntaxes. Finally, Bridge is a class that allows bidirectional conversions between OpenCV/std and Matlab types.

Once you have written your file, it can be compiled with the provided mex utility:

cv.mex('my_function.cpp');

This utility automatically links in all of the necessary OpenCV libraries to make your function work.

NOTE: OpenCV uses exceptions throughout the codebase. It is a very good idea to wrap your code in exception handlers to avoid crashing Matlab in the event of an exception being thrown.

---

Developer

The following sections contain information for developers seeking to use, understand or extend the Matlab bindings. The bindings are generated in python using a powerful templating engine called Jinja2. Because Matlab mex gateways have a common structure, they are well suited to templatization. There are separate templates for formatting C++ classes, Matlab classes, C++ functions, constants (enums) and documentation.

The task of the generator is two-fold:

1. To parse the OpenCV headers and build a semantic tree that can be fed to the template engine
2. To define type conversions between C++/OpenCV and Matlab types

Once a source file has been generated for each OpenCV definition, and type conversions have been established, the mex compiler is invoked to produce the mex gateway (shared object) and link in the OpenCV libraries.

File layout

opencv/modules/matlab (this module)

• CMakeLists.txt (main cmake configuration file)
• README.md (this file)
• compile.cmake (the cmake script for compiling generated source code)
• generator (the folder containing generator code)
  • filters.py (template filters)
  • gen_matlab.py (the binding generator control script)
  • parse_tree.py (python class to refactor the hdr_parser.py output)
  • templates (the raw templates for populating classes, constants, functions and docs)
• include (C++ headers for the bindings)
  • mxarray.hpp (C++ OOP-style interface for Matlab mxArray* class)
  • bridge.hpp (type conversions)
  • map.hpp (hash map interface for instance storage and method lookup)
• test (generator, compiler and binding test scripts)

Call Tree

The cmake call tree can be broken into 3 main components:

1. configure time
2. build time
3. install time

Find Matlab (configure) The first thing to do is discover a Matlab installation on the host system. This is handled by the OpenCVFindMatlab.cmake in opencv/cmake. On Windows machines it searches the registry and path, while on *NIX machines it searches a set of canonical install paths. Once Matlab has been found, a number of variables are defined, such as the path to the mex compiler, the mex libraries, the mex include paths, the architectural extension, etc.

Test the generator (configure) Attempt to produce a source file for a simple definition. This tests whether python and pythonlibs are correctly invoked on the host.

Test the mex compiler (configure) Attempt to compile a simple definition using the mex compiler. A mex file is actually just a shared object with a special exported symbol _mexFunction which serves as the entry-point to the function. As such, the mex compiler is just a set of scripts configuring the system compiler. In most cases this is the same as the OpenCV compiler, but could be different. The test checks whether the mex and generator includes can be found, the system libraries can be linked and the passed compiler flags are compatible.

If any of the configure time tests fail, the bindings will be disabled, but the main OpenCV configure will continue without error. The configuration summary will contain the block:

Matlab mex: /Applications/MATLAB_R2013a.app/bin/mex compiler/generator: Not working (bindings will not be generated)

Generate the sources (build) Given a set of modules (the intersection of the OpenCV modules being built and the matlab module optional dependencies), the CppHeaderParser() from opencv/modules/python/src2/hdr_parser.py will parse the module headers and produce a set of definitions.

The ParseTree() from opencv/modules/matlab/generator/parse_tree.py takes this set of definitions and refactors them into a semantic tree better suited to templatization. For example, a trivial definition from the header parser may look something like:

[fill, void, ['/S'], [cv::Mat&, mat, '', ['/I', '/O']]]

The equivalent refactored output will look like:

  Function
    name   = 'fill'
    rtype  = 'void'
    static = True
    req =
      Argument
        name    = 'mat'
        type    = 'cv::Mat'
        ref     = '&'
        I       = True
        O       = True
        default = ''

The added semantics (Namespace, Class, Function, Argument, name, etc) make it easier for the templating engine to parse, slice and populate definitions.

Once the definitions have been parsed, gen_matlab.py passes each definition to the template engine with the appropriate template (class, function, enum, doc) and the filled template gets written to the ${CMAKE_CURRENT_BUILD_DIR}/src directory.

The generator relies upon a proxy object called generate.proxy to determine when the sources are out of date and need to be re-generated.

Compile the sources (build) Once the sources have been generated, they are compiled by the mex compiler. The compile.cmake script in opencv/modules/matlab/ takes responsibility for iterating over each source file in ${CMAKE_CURRENT_BUILD_DIR}/src and compiling it with the passed includes and OpenCV libraries.

The flags used to compile the main OpenCV libraries are also forwarded to the mex compiler. So if, for example, you compiled OpenCV with SSE support, the mex bindings will also use SSE. Likewise, if you compile OpenCV in debug mode, the bindings will link to the debug version of the libraries.

Importantly, the mex compiler includes the mxarray.hpp, bridge.hpp and map.hpp files from the opencv/modules/matlab/include directory. mxarray.hpp defines a MxArray class which wraps Matlab's mxArray* type in a more friendly OOP-syle interface. bridge.hpp defines a Bridge class which is able to perform type conversions between Matlab types and std/OpenCV types. It can be extended with new definitions using the plugin interface described in that file.

The compiler relies upon a proxy object called compile.proxy to determine when the generated sources are out of date and need to be re-compiled.

Install the files (install) At install time, the mex files are put into place at ${CMAKE_INSTALL_PREFIX}/matlab and their linkages updated.

Jinja2

Jinja2 is a powerful templating engine, similar to python's builtin string.Template class but implementing the model-view-controller paradigm. For example, a trivial view could be populated as follows:

view.py

<title>{{ title }}</title>
<ul>
{% for user in users %}
  <li><a href="{{ user.url }}">{{ user.username | sanitize }}</a></li>
{% endfor %}
</ul>

model.py

class User(object):
  __init__(self):
    self.username = ''
    self.url = ''

def sanitize(text):
  """Filter for escaping html tags to prevent code injection"""

controller.py

def populate(users):
# initialize jinja
jtemplate = jinja2.Environment(loader=FileSystemLoader())

# add the filters to the engine
jtemplate['sanitize'] = sanitize

# get the view
template = jtemplate.get_template('view')

# populate the template with a list of User objects
populated = template.render(title='all users', users=users)

# write to file
with open('users.html', 'wb') as f:
  f.write(populated)

Thus the style and layout of the view is kept separate from the content (model). This modularity improves readability and maintainability of both the view and content and (for my own sanity) has helped significantly in debugging errors.

File Reference

gen_matlab.py gen_matlab has the following call signature:

gen_matlab.py --jinja2 path/to/jinja2/engine --hdrparser path/to/hdr_parser/dir --rstparser path/to/rst_parser/dir --moduleroot path/to/opencv/modules --modules [core imgproc highgui ...] --extra namespace=/additional/header/to/parse --outdir /path/to/place/generated/src

build_info.py build_info has the following call signature:

build_info.py --jinja2 path/to/jinja2/engine --os operating_system_string --arch [bitness processor] --compiler [id version] --mex_arch arch_string --mex_script /path/to/mex/script --cxx_flags [-list -of -flags -to -passthrough] --opencv_version version_string --commit commit_hash_if_using_git --modules core imgproc highgui etc --configuration Debug/Release --outdir path/to/place/build/info

cvmex.py cvmex.py, the custom compiler generator, has the following call signature:

cvmex.py --jinja2 path/to/jinja2/engine --opts [-list -of -opts] --include_dirs [-list -of -opencv_include_directories] --lib_dir opencv_lib_directory --libs [-lopencv_core -lopencv_imgproc ...] --flags [-Wall -opencv_build_flags ...] --outdir /path/to/generated/output

parse_tree.py To build a parse tree, first parse a set of headers, then invoke the parse tree to refactor the output:

# parse a set of definitions into a dictionary of namespaces
parser = CppHeaderParser()
ns['core'] = parser.parse('path/to/opencv/core.hpp')

# refactor into a semantic tree
parse_tree = ParseTree()
parse_tree.build(ns)

# iterate over the tree
for namespace in parse_tree.namespaces:
  for clss in namespace.classes:
    # do stuff
  for method in namespace.methods:
    # do stuff

mxarray.hpp mxarray.hpp defines a class called MxArray which provides an OOP-style interface for Matlab's homogeneous mxArray* type. To create an MxArray, you can either inherit an existing array

MxArray mat(prhs[0]);

or create a new array

MxArray mat(5, 5, Matlab::Traits<double>::ScalarType);
MxArray mat = MxArray::Matrix<double>(5, 5);

The default constructor allocates a 0 x 0 array. Once you have encapculated an mxArray* you can access its properties through member functions:

mat.rows();
mat.cols();
mat.size();
mat.channels();
mat.isComplex();
mat.isNumeric();
mat.isLogical();
mat.isClass();
mat.className();
mat.real();
mat.imag();

The MxArray object uses scoped memory management. If you wish to pass an MxArray back to Matlab (as a lhs pointer), you need to explicitly release ownership of the array so that it is not destroyed when it leaves scope:

plhs[0] = mat.releaseOwnership();

mxarray.hpp also includes a number of helper utilities that make working in mex-world a little easier. One such utility is the ArgumentParser. ArgumentParser automatically handles required and optional arguments to a method, and even enables named arguments as used in many core Matlab functions. For example, if you had a function with the following signature:

void f(Mat first, Mat second, Mat mask=Mat(), int dtype=-1);

then you can create an ArgumentParser as follows:

ArgumentParser parser("f");
parser.addVariant(2, 2, "mask", "dtype");
MxArrayVector inputs = parser.parse(prhs, prhs+nrhs);

and that will make available the following calling syntaxes:

f(first, second);
f(first, second, mask);
f(first, second, mask, dtype);
f(first, second, 'dtype', dtype, 'mask', mask); % optional ordering does not matter
f(first, second, 'dtype', dtype); % only second optional argument provided
f(first, second, mask, 'dtype', dtype); % mixture of ordered and named

Further, the output of the parser.parse() method will always contain the total number of required and optional arguments that the method can take, with unspecified arguments given by empty matrices. Thus, to check if an optional argument has been given, you can do:

int dtype = inputs[3].empty() ? -1 : inputs[3].scalar<double>();

bridge.hpp The bridge interface defines a Bridge class which provides type conversion between std/OpenCV and Matlab types. A type conversion must provide the following:

Bridge& operator=(const MyObject&);
MyObject toMyObject();
operator MyObject();

The binding generator will then automatically call the conversion operators (either explicitly or implicitly) if your MyObject class is encountered as an input or return from a parsed definition.