Merge pull request #14666 from ihsan314:file_io_xml_yml

Python code examples for file IO in xml and yml format

* Initial "Pythonization" of file_input_output.cpp

* Moved file_input_output.py to correct location

* Nearly done Pythonizing file_input_output.cpp

* Python equivalent of file_input_output.py created

* Started Pythonizing camera_calibration.cpp

* Completed Python tutorial/sample code for file_input_output

* Resolved whitespace issues

* Removed tabs in file_input_output.cpp

* Patched import order and wrapped code in main function

* Changed string to docstring format in help file

* Updated link to Python example code

doc/tutorials/core/file_input_output_with_xml_yml/file_input_output_with_xml_yml.markdown

@@ -17,7 +17,7 @@ You'll find answers for the following questions:
 
 Source code
 -----------
-
+@add_toggle_cpp
 You can [download this from here
 ](https://github.com/opencv/opencv/tree/3.4/samples/cpp/tutorial_code/core/file_input_output/file_input_output.cpp) or find it in the
 `samples/cpp/tutorial_code/core/file_input_output/file_input_output.cpp` of the OpenCV source code
@@ -26,13 +26,25 @@ library.
 Here's a sample code of how to achieve all the stuff enumerated at the goal list.
 
 @include cpp/tutorial_code/core/file_input_output/file_input_output.cpp
+@end_toggle
+
+@add_toggle_python
+You can [download this from here
+](https://github.com/opencv/opencv/tree/3.4/samples/python/tutorial_code/core/file_input_output/file_input_output.py) or find it in the
+`samples/python/tutorial_code/core/file_input_output/file_input_output.py` of the OpenCV source code
+library.
+
+Here's a sample code of how to achieve all the stuff enumerated at the goal list.
+
+@include python/tutorial_code/core/file_input_output/file_input_output.py
+@end_toggle
 
 Explanation
 -----------
 
 Here we talk only about XML and YAML file inputs. Your output (and its respective input) file may
 have only one of these extensions and the structure coming from this. They are two kinds of data
-structures you may serialize: *mappings* (like the STL map) and *element sequence* (like the STL
+structures you may serialize: *mappings* (like the STL map and the Python dictionary) and *element sequence* (like the STL
 vector). The difference between these is that in a map every element has a unique name through what
 you may access it. For sequences you need to go through them to query a specific item.
 
@@ -40,12 +52,12 @@ you may access it. For sequences you need to go through them to query a specific
     and at the end to close it. The XML/YAML data structure in OpenCV is @ref cv::FileStorage . To
     specify that this structure to which file binds on your hard drive you can use either its
     constructor or the *open()* function of this:
-    @code{.cpp}
+    @add_toggle_cpp
-    string filename = "I.xml";
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp open
-    FileStorage fs(filename, FileStorage::WRITE);
+    @end_toggle
-    //...
+    @add_toggle_python
-    fs.open(filename, FileStorage::READ);
+    @snippet python/tutorial_code/core/file_input_output/file_input_output.py open
-    @endcode
+    @end_toggle
     Either one of this you use the second argument is a constant specifying the type of operations
     you'll be able to on them: WRITE, READ or APPEND. The extension specified in the file name also
     determinates the output format that will be used. The output may be even compressed if you
@@ -53,75 +65,83 @@ you may access it. For sequences you need to go through them to query a specific
 
     The file automatically closes when the @ref cv::FileStorage objects is destroyed. However, you
     may explicitly call for this by using the *release* function:
-    @code{.cpp}
+    @add_toggle_cpp
-    fs.release();                                       // explicit close
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp close
-    @endcode
+    @end_toggle
--#  **Input and Output of text and numbers.** The data structure uses the same \<\< output operator
+    @add_toggle_python
-    that the STL library. For outputting any type of data structure we need first to specify its
+    @snippet python/tutorial_code/core/file_input_output/file_input_output.py close
-    name. We do this by just simply printing out the name of this. For basic types you may follow
+    @end_toggle
-    this with the print of the value :
+-#  **Input and Output of text and numbers.** In C++, the data structure uses the \<\< output
-    @code{.cpp}
+    operator in the STL library. In Python, @ref cv::FileStorage.write() is used instead. For
-    fs << "iterationNr" << 100;
+    outputting any type of data structure we need first to specify its name. We do this by just
-    @endcode
+    simply pushing the name of this to the stream in C++. In Python, the first parameter for the
+    write function is the name. For basic types you may follow this with the print of the value :
+    @add_toggle_cpp
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp writeNum
+    @end_toggle
+    @add_toggle_python
+    @snippet python/tutorial_code/core/file_input_output/file_input_output.py writeNum
+    @end_toggle
     Reading in is a simple addressing (via the [] operator) and casting operation or a read via
-    the \>\> operator :
+    the \>\> operator. In Python, we address with getNode() and use real() :
-    @code{.cpp}
+    @add_toggle_cpp
-    int itNr;
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp readNum
-    fs["iterationNr"] >> itNr;
+    @end_toggle
-    itNr = (int) fs["iterationNr"];
+    @add_toggle_python
-    @endcode
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp readNum
+    @end_toggle
 -#  **Input/Output of OpenCV Data structures.** Well these behave exactly just as the basic C++
-    types:
+    and Python types:
-    @code{.cpp}
+    @add_toggle_cpp
-    Mat R = Mat_<uchar >::eye  (3, 3),
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp iomati
-        T = Mat_<double>::zeros(3, 1);
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp iomatw
-
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp iomat
-    fs << "R" << R;                                      // Write cv::Mat
+    @end_toggle
-    fs << "T" << T;
+    @add_toggle_python
-
+    @snippet python/tutorial_code/core/file_input_output/file_input_output.py iomati
-    fs["R"] >> R;                                      // Read cv::Mat
+    @snippet python/tutorial_code/core/file_input_output/file_input_output.py iomatw
-    fs["T"] >> T;
+    @snippet python/tutorial_code/core/file_input_output/file_input_output.py iomat
-    @endcode
+    @end_toggle
 -#  **Input/Output of vectors (arrays) and associative maps.** As I mentioned beforehand, we can
     output maps and sequences (array, vector) too. Again we first print the name of the variable and
     then we have to specify if our output is either a sequence or map.
 
     For sequence before the first element print the "[" character and after the last one the "]"
-    character:
+    character. With Python, the "]" character could be written with the name of the sequence or
-    @code{.cpp}
+    the last element of the sequence depending on the number of elements:
-    fs << "strings" << "[";                              // text - string sequence
+    @add_toggle_cpp
-    fs << "image1.jpg" << "Awesomeness" << "baboon.jpg";
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp writeStr
-    fs << "]";                                           // close sequence
+    @end_toggle
-    @endcode
+    @add_toggle_python
+    @snippet python/tutorial_code/core/file_input_output/file_input_output.py writeStr
+    @end_toggle
     For maps the drill is the same however now we use the "{" and "}" delimiter characters:
-    @code{.cpp}
+    @add_toggle_cpp
-    fs << "Mapping";                              // text - mapping
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp writeMap
-    fs << "{" << "One" << 1;
+    @end_toggle
-    fs <<        "Two" << 2 << "}";
+    @add_toggle_python
-    @endcode
+    @snippet python/tutorial_code/core/file_input_output/file_input_output.py writeMap
+    @end_toggle
     To read from these we use the @ref cv::FileNode and the @ref cv::FileNodeIterator data
-    structures. The [] operator of the @ref cv::FileStorage class returns a @ref cv::FileNode data
+    structures. The [] operator of the @ref cv::FileStorage class (or the getNode() function in Python) returns a @ref cv::FileNode data
     type. If the node is sequential we can use the @ref cv::FileNodeIterator to iterate through the
-    items:
+    items. In Python, the at() function can be used to address elements of the sequence and the
-    @code{.cpp}
+    size() function returns the length of the sequence:
-    FileNode n = fs["strings"];                         // Read string sequence - Get node
+    @add_toggle_cpp
-    if (n.type() != FileNode::SEQ)
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp readStr
-    {
+    @end_toggle
-        cerr << "strings is not a sequence! FAIL" << endl;
+    @add_toggle_python
-        return 1;
+    @snippet python/tutorial_code/core/file_input_output/file_input_output.py readStr
-    }
+    @end_toggle
-
+    For maps you can use the [] operator (at() function in Python) again to access the given item (or the \>\> operator too):
-    FileNodeIterator it = n.begin(), it_end = n.end(); // Go through the node
+    @add_toggle_cpp
-    for (; it != it_end; ++it)
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp readMap
-        cout << (string)*it << endl;
+    @end_toggle
-    @endcode
+    @add_toggle_python
-    For maps you can use the [] operator again to access the given item (or the \>\> operator too):
+    @snippet python/tutorial_code/core/file_input_output/file_input_output.py readMap
-    @code{.cpp}
+    @end_toggle
-    n = fs["Mapping"];                                // Read mappings from a sequence
-    cout << "Two  " << (int)(n["Two"]) << "; ";
-    cout << "One  " << (int)(n["One"]) << endl << endl;
-    @endcode
 -#  **Read and write your own data structures.** Suppose you have a data structure such as:
+    @add_toggle_cpp
     @code{.cpp}
     class MyData
     {
@@ -133,53 +153,52 @@ you may access it. For sequences you need to go through them to query a specific
        string id;
     };
     @endcode
-    It's possible to serialize this through the OpenCV I/O XML/YAML interface (just as in case of
+    @end_toggle
-    the OpenCV data structures) by adding a read and a write function inside and outside of your
+    @add_toggle_python
-    class. For the inside part:
+    @code{.py}
-    @code{.cpp}
+    class MyData:
-    void write(FileStorage& fs) const                        //Write serialization for this class
+        def __init__(self):
-    {
+            self.A = self.X = 0
-      fs << "{" << "A" << A << "X" << X << "id" << id << "}";
+            self.name = ''
-    }
-
-    void read(const FileNode& node)                          //Read serialization for this class
-    {
-      A = (int)node["A"];
-      X = (double)node["X"];
-      id = (string)node["id"];
-    }
-    @endcode
-    Then you need to add the following functions definitions outside the class:
-    @code{.cpp}
-    void write(FileStorage& fs, const std::string&, const MyData& x)
-    {
-    x.write(fs);
-    }
-
-    void read(const FileNode& node, MyData& x, const MyData& default_value = MyData())
-    {
-    if(node.empty())
-        x = default_value;
-    else
-        x.read(node);
-    }
     @endcode
+    @end_toggle
+    In C++, it's possible to serialize this through the OpenCV I/O XML/YAML interface (just as
+    in case of the OpenCV data structures) by adding a read and a write function inside and outside of your
+    class. In Python, you can get close to this by implementing a read and write function inside
+    the class. For the inside part:
+    @add_toggle_cpp
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp inside
+    @end_toggle
+    @add_toggle_python
+    @snippet python/tutorial_code/core/file_input_output/file_input_output.py inside
+    @end_toggle
+    @add_toggle_cpp
+    In C++, you need to add the following functions definitions outside the class:
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp outside
+    @end_toggle
     Here you can observe that in the read section we defined what happens if the user tries to read
     a non-existing node. In this case we just return the default initialization value, however a
     more verbose solution would be to return for instance a minus one value for an object ID.
 
     Once you added these four functions use the \>\> operator for write and the \<\< operator for
-    read:
+    read (or the defined input/output functions for Python):
-    @code{.cpp}
+    @add_toggle_cpp
-    MyData m(1);
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp customIOi
-    fs << "MyData" << m;                                // your own data structures
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp customIOw
-    fs["MyData"] >> m;                                 // Read your own structure_
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp customIO
-    @endcode
+    @end_toggle
+    @add_toggle_python
+    @snippet python/tutorial_code/core/file_input_output/file_input_output.py customIOi
+    @snippet python/tutorial_code/core/file_input_output/file_input_output.py customIOw
+    @snippet python/tutorial_code/core/file_input_output/file_input_output.py customIO
+    @end_toggle
     Or to try out reading a non-existing read:
-    @code{.cpp}
+    @add_toggle_cpp
-    fs["NonExisting"] >> m;   // Do not add a fs << "NonExisting" << m command for this to work
+    @snippet cpp/tutorial_code/core/file_input_output/file_input_output.cpp nonexist
-    cout << endl << "NonExisting = " << endl << m << endl;
+    @end_toggle
-    @endcode
+    @add_toggle_python
+    @snippet python/tutorial_code/core/file_input_output/file_input_output.py nonexist
+    @end_toggle
 
 Result
 ------

samples/cpp/tutorial_code/core/file_input_output/file_input_output.cpp

@@ -25,6 +25,7 @@ public:
     {}
     explicit MyData(int) : A(97), X(CV_PI), id("mydata1234") // explicit to avoid implicit conversion
     {}
+    //! [inside]
     void write(FileStorage& fs) const                        //Write serialization for this class
     {
         fs << "{" << "A" << A << "X" << X << "id" << id << "}";
@@ -35,6 +36,7 @@ public:
         X = (double)node["X"];
         id = (string)node["id"];
     }
+    //! [inside]
 public:   // Data Members
     int A;
     double X;
@@ -42,6 +44,7 @@ public:   // Data Members
 };
 
 //These write and read functions must be defined for the serialization in FileStorage to work
+//! [outside]
 static void write(FileStorage& fs, const std::string&, const MyData& x)
 {
     x.write(fs);
@@ -52,6 +55,7 @@ static void read(const FileNode& node, MyData& x, const MyData& default_value =
     else
         x.read(node);
 }
+//! [outside]
 
 // This function will print our custom class to the console
 static ostream& operator<<(ostream& out, const MyData& m)
@@ -72,27 +76,48 @@ int main(int ac, char** av)
 
     string filename = av[1];
     { //write
+        //! [iomati]
         Mat R = Mat_<uchar>::eye(3, 3),
             T = Mat_<double>::zeros(3, 1);
+        //! [iomati]
+        //! [customIOi]
         MyData m(1);
+        //! [customIOi]
 
+        //! [open]
         FileStorage fs(filename, FileStorage::WRITE);
+        // or:
+        // FileStorage fs;
+        // fs.open(filename, FileStorage::WRITE);
+        //! [open]
 
+        //! [writeNum]
         fs << "iterationNr" << 100;
+        //! [writeNum]
+        //! [writeStr]
         fs << "strings" << "[";                              // text - string sequence
         fs << "image1.jpg" << "Awesomeness" << "../data/baboon.jpg";
         fs << "]";                                           // close sequence
+        //! [writeStr]
 
+        //! [writeMap]
         fs << "Mapping";                              // text - mapping
         fs << "{" << "One" << 1;
         fs <<        "Two" << 2 << "}";
+        //! [writeMap]
 
+        //! [iomatw]
         fs << "R" << R;                                      // cv::Mat
         fs << "T" << T;
+        //! [iomatw]
 
+        //! [customIOw]
         fs << "MyData" << m;                                // your own data structures
+        //! [customIOw]
 
+        //! [close]
         fs.release();                                       // explicit close
+        //! [close]
         cout << "Write Done." << endl;
     }
 
@@ -101,9 +126,11 @@ int main(int ac, char** av)
         FileStorage fs;
         fs.open(filename, FileStorage::READ);
 
+        //! [readNum]
         int itNr;
         //fs["iterationNr"] >> itNr;
         itNr = (int) fs["iterationNr"];
+        //! [readNum]
         cout << itNr;
         if (!fs.isOpened())
         {
@@ -112,6 +139,7 @@ int main(int ac, char** av)
             return 1;
         }
 
+        //! [readStr]
         FileNode n = fs["strings"];                         // Read string sequence - Get node
         if (n.type() != FileNode::SEQ)
         {
@@ -122,19 +150,26 @@ int main(int ac, char** av)
         FileNodeIterator it = n.begin(), it_end = n.end(); // Go through the node
         for (; it != it_end; ++it)
             cout << (string)*it << endl;
+        //! [readStr]
 
 
+        //! [readMap]
         n = fs["Mapping"];                                // Read mappings from a sequence
         cout << "Two  " << (int)(n["Two"]) << "; ";
         cout << "One  " << (int)(n["One"]) << endl << endl;
+        //! [readMap]
 
 
         MyData m;
         Mat R, T;
 
+        //! [iomat]
         fs["R"] >> R;                                      // Read cv::Mat
         fs["T"] >> T;
+        //! [iomat]
+        //! [customIO]
         fs["MyData"] >> m;                                 // Read your own structure_
+        //! [customIO]
 
         cout << endl
             << "R = " << R << endl;
@@ -142,9 +177,11 @@ int main(int ac, char** av)
         cout << "MyData = " << endl << m << endl << endl;
 
         //Show default behavior for non existing nodes
+        //! [nonexist]
         cout << "Attempt to read NonExisting (should initialize the data structure with its default).";
         fs["NonExisting"] >> m;
         cout << endl << "NonExisting = " << endl << m << endl;
+        //! [nonexist]
     }
 
     cout << endl

samples/python/tutorial_code/core/file_input_output/file_input_output.py

@@ -0,0 +1,156 @@
+from __future__ import print_function
+
+import numpy as np
+import cv2 as cv
+import sys
+
+def help(filename):
+    print (
+        '''
+        {0} shows the usage of the OpenCV serialization functionality. \n\n
+        usage:\n
+            python3 {0} outputfile.yml.gz\n\n
+        The output file may be either in XML, YAML or JSON. You can even compress it\n
+        by specifying this in its extension like xml.gz yaml.gz etc... With\n
+        FileStorage you can serialize objects in OpenCV.\n\n
+        For example: - create a class and have it serialized\n
+                     - use it to read and write matrices.\n
+        '''.format(filename)
+    )
+
+class MyData:
+    A = 97
+    X = np.pi
+    name = 'mydata1234'
+
+    def __repr__(self):
+        s = '{ name = ' + self.name + ', X = ' + str(self.X)
+        s = s + ', A = ' +  str(self.A) + '}'
+        return s
+
+    ## [inside]
+    def write(self, fs):
+        fs.write('MyData','{')
+        fs.write('A', self.A)
+        fs.write('X', self.X)
+        fs.write('name', self.name)
+        fs.write('MyData','}')
+
+    def read(self, node):
+        if (not node.empty()):
+            self.A = int(node.getNode('A').real())
+            self.X = node.getNode('X').real()
+            self.name = node.getNode('name').string()
+        else:
+            self.A = self.X = 0
+            self.name = ''
+    ## [inside]
+
+def main(argv):
+    if len(argv) != 2:
+        help(argv[0])
+        exit(1)
+
+    # write
+    ## [iomati]
+    R = np.eye(3,3)
+    T = np.zeros((3,1))
+    ## [iomati]
+    ## [customIOi]
+    m = MyData()
+    ## [customIOi]
+
+    filename = argv[1]
+
+    ## [open]
+    s = cv.FileStorage(filename, cv.FileStorage_WRITE)
+    # or:
+    # s = cv.FileStorage()
+    # s.open(filename, cv.FileStorage_WRITE)
+    ## [open]
+
+    ## [writeNum]
+    s.write('iterationNr', 100)
+    ## [writeNum]
+
+    ## [writeStr]
+    s.write('strings', '[')
+    s.write('image1.jpg','Awesomeness')
+    s.write('../data/baboon.jpg',']')
+    ## [writeStr]
+
+    ## [writeMap]
+    s.write ('Mapping', '{')
+    s.write ('One', 1)
+    s.write ('Two', 2)
+    s.write ('Mapping', '}')
+    ## [writeMap]
+
+    ## [iomatw]
+    s.write ('R_MAT', R)
+    s.write ('T_MAT', T)
+    ## [iomatw]
+
+    ## [customIOw]
+    m.write(s)
+    ## [customIOw]
+    ## [close]
+    s.release()
+    ## [close]
+    print ('Write Done.')
+
+    # read
+    print ('\nReading: ')
+    s = cv.FileStorage()
+    s.open(filename, cv.FileStorage_READ)
+
+    ## [readNum]
+    n = s.getNode('iterationNr')
+    itNr = int(n.real())
+    ## [readNum]
+    print (itNr)
+
+    if (not s.isOpened()):
+        print ('Failed to open ', filename, file=sys.stderr)
+        help(argv[0])
+        exit(1)
+
+    ## [readStr]
+    n = s.getNode('strings')
+    if (not n.isSeq()):
+        print ('strings is not a sequence! FAIL', file=sys.stderr)
+        exit(1)
+
+    for i in range(n.size()):
+        print (n.at(i).string())
+    ## [readStr]
+
+    ## [readMap]
+    n = s.getNode('Mapping')
+    print ('Two',int(n.getNode('Two').real()),'; ')
+    print ('One',int(n.getNode('One').real()),'\n')
+    ## [readMap]
+
+    ## [iomat]
+    R = s.getNode('R_MAT').mat()
+    T = s.getNode('T_MAT').mat()
+    ## [iomat]
+    ## [customIO]
+    m.read(s.getNode('MyData'))
+    ## [customIO]
+
+    print ('\nR =',R)
+    print ('T =',T,'\n')
+    print ('MyData =','\n',m,'\n')
+
+    ## [nonexist]
+    print ('Attempt to read NonExisting (should initialize the data structure',
+            'with its default).')
+    m.read(s.getNode('NonExisting'))
+    print ('\nNonExisting =','\n',m)
+    ## [nonexist]
+
+    print ('\nTip: Open up',filename,'with a text editor to see the serialized data.')
+
+if __name__ == '__main__':
+    main(sys.argv)