Merge pull request #20471 from ibvfteh:pointcloudio

GSoC module to save and load point cloud

* Add functionality to read point cloud data from files

* address issues found on review, add tests for mesh, refactor

* enable fail-safe execution and empty arrays as output

* Some improvements for point cloud io module

Co-authored-by: Julie Bareeva <julia.bareeva@xperience.ai>

doc/tutorials/3d/point_cloud/point_cloud.markdown

@@ -0,0 +1,75 @@
+Point cloud visualisation {#tutorial_point_cloud}
+==============================
+
+|    |    |
+| -: | :- |
+| Original author | Dmitrii Klepikov |
+| Compatibility | OpenCV >= 5.0 |
+
+Goal
+----
+
+In this tutorial you will:
+
+-   Load and save point cloud data
+-   Visualise your data
+
+Requirements
+------------
+
+For visualisations you need to compile OpenCV library with OpenGL support.
+For this you should set WITH_OPENGL flag ON in CMake while building OpenCV from source.
+
+Practice
+-------
+
+Loading and saving of point cloud can be done using `cv::loadPointCloud` and `cv::savePointCloud` accordingly.
+
+Currently supported formats are:
+- [.OBJ](https://en.wikipedia.org/wiki/Wavefront_.obj_file) (supported keys are v(which is responsible for point position), vn(normal coordinates) and f(faces of a mesh), other keys are ignored)
+- [.PLY](https://en.wikipedia.org/wiki/PLY_(file_format)) (all encoding types(ascii and byte) are supported with limitation to only float type for data)
+
+@code{.py}
+vertices, normals = cv2.loadPointCloud("teapot.obj")
+@endcode
+
+Function `cv::loadPointCloud` returns vector of points of float (`cv::Point3f`) and vector of their normals(if specified in source file).
+To visualize it you can use functions from viz3d module and it is needed to reinterpret data into another format
+
+@code{.py}
+vertices = np.squeeze(vertices, axis=1)
+
+color = [1.0, 1.0, 0.0]
+colors = np.tile(color, (vertices.shape[0], 1))
+obj_pts = np.concatenate((vertices, colors), axis=1).astype(np.float32)
+
+cv2.viz3d.showPoints("Window", "Points", obj_pts)
+
+cv2.waitKey(0)
+@endcode
+
+In presented code sample we add a colour attribute to every point
+Result will be:
+
+![](tutorial_point_cloud_teapot.jpg)
+
+For additional info grid can be added
+
+@code{.py}
+vertices, normals = cv2.loadPointCloud("teapot.obj")
+@endcode
+
+![](teapot_grid.jpg)
+
+Other possible way to draw 3d objects can be a mesh.
+For that we use special functions to load mesh data and display it.
+Here for now only .OBJ files are supported and they should be triangulated before processing (triangulation - process of breaking faces into triangles).
+
+@code{.py}
+vertices, _, indices = cv2.loadMesh("../data/teapot.obj")
+vertices = np.squeeze(vertices, axis=1)
+
+cv2.viz3d.showMesh("window", "mesh", vertices, indices)
+@endcode
+
+![](teapot_mesh.jpg)

doc/tutorials/3d/table_of_content_3d.markdown

@@ -0,0 +1,5 @@
+
+3d processing and visualisation (3d module) {#tutorial_table_of_content_3d}
+==========================================================
+
+-   @subpage tutorial_point_cloud

doc/tutorials/tutorials.markdown

@@ -11,6 +11,7 @@ OpenCV Tutorials {#tutorial_root}
 - @subpage tutorial_table_of_content_gapi - graph-based approach to computer vision algorithms building
 - @subpage tutorial_table_of_content_other - other modules (ml, objdetect, stitching, video, photo)
 - @subpage tutorial_table_of_content_ios - running OpenCV on an iDevice
+- @subpage tutorial_table_of_content_3d - 3d objects processing and visualisation
 @cond CUDA_MODULES
 - @subpage tutorial_table_of_content_gpu - utilizing power of video card to run CV algorithms
 @endcond

modules/3d/include/opencv2/3d.hpp

@@ -2608,6 +2608,60 @@ protected:
     Ptr<Impl> p;
 };
 
+
+/** @brief Loads a point cloud from a file.
+*
+* The function loads point cloud from the specified file and returns it.
+* If the cloud cannot be read, throws an error
+*
+* Currently, the following file formats are supported:
+* -  [Wavefront obj file *.obj](https://en.wikipedia.org/wiki/Wavefront_.obj_file)
+* -  [Polygon File Format *.ply](https://en.wikipedia.org/wiki/PLY_(file_format))
+*
+* @param filename Name of the file.
+* @param vertices (vector of Point3f) Point coordinates of a point cloud
+* @param normals (vector of Point3f) Point normals of a point cloud
+*/
+CV_EXPORTS_W void loadPointCloud(const String &filename, OutputArray vertices, OutputArray normals = noArray());
+
+/** @brief Saves a point cloud to a specified file.
+*
+* The function saves point cloud to the specified file.
+* File format is chosen based on the filename extension.
+*
+* @param filename Name of the file.
+* @param vertices (vector of Point3f) Point coordinates of a point cloud
+* @param normals (vector of Point3f) Point normals of a point cloud
+*/
+CV_EXPORTS_W void savePointCloud(const String &filename, InputArray vertices, InputArray normals = noArray());
+
+/** @brief Loads a mesh from a file.
+*
+* The function loads mesh from the specified file and returns it.
+* If the mesh cannot be read, throws an error
+*
+* Currently, the following file formats are supported:
+* -  [Wavefront obj file *.obj](https://en.wikipedia.org/wiki/Wavefront_.obj_file) (ONLY TRIANGULATED FACES)
+* @param filename Name of the file.
+* @param vertices (vector of Point3f) vertex coordinates of a mesh
+* @param normals (vector of Point3f) vertex normals of a mesh
+* @param indices (vector of vectors of int) vertex normals of a mesh
+*/
+CV_EXPORTS_W void loadMesh(const String &filename, OutputArray vertices, OutputArray normals, OutputArrayOfArrays indices);
+
+/** @brief Saves a mesh to a specified file.
+*
+* The function saves mesh to the specified file.
+* File format is chosen based on the filename extension.
+*
+* @param filename Name of the file.
+* @param vertices (vector of Point3f) vertex coordinates of a mesh
+* @param normals (vector of Point3f) vertex normals of a mesh
+* @param indices (vector of vectors of int) vertex normals of a mesh
+*/
+CV_EXPORTS_W void saveMesh(const String &filename, InputArray vertices, InputArray normals, InputArrayOfArrays indices);
+
+
 //! @} _3d
 } //end namespace cv
 

modules/3d/src/pointcloud/io_base.cpp

@@ -0,0 +1,32 @@
+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+
+#include "../precomp.hpp"
+#include "io_base.hpp"
+
+namespace cv {
+
+void BasePointCloudDecoder::setSource(const std::string &filename) noexcept
+{
+    m_filename = filename;
+}
+
+void BasePointCloudDecoder::readData(std::vector<Point3f> &points, std::vector<Point3f> &normals)
+{
+    std::vector<std::vector<int32_t>> indices;
+    readData(points, normals, indices);
+}
+
+void BasePointCloudEncoder::setDestination(const std::string &filename) noexcept
+{
+    m_filename = filename;
+}
+
+void BasePointCloudEncoder::writeData(const std::vector<Point3f> &points, const std::vector<Point3f> &normals)
+{
+    std::vector<std::vector<int32_t>> indices;
+    writeData(points, normals, indices);
+}
+
+} /* namespace cv */

modules/3d/src/pointcloud/io_base.hpp

@@ -0,0 +1,51 @@
+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+
+#ifndef _CODERS_BASE_H_
+#define _CODERS_BASE_H_
+
+#include <vector>
+#include <memory>
+#include <cstdint>
+
+#include <opencv2/core.hpp>
+
+namespace cv {
+
+class BasePointCloudDecoder;
+class BasePointCloudEncoder;
+using PointCloudDecoder = std::unique_ptr<BasePointCloudDecoder>;
+using PointCloudEncoder = std::unique_ptr<BasePointCloudEncoder>;
+
+///////////////////////////////// base class for decoders ////////////////////////
+class BasePointCloudDecoder
+{
+public:
+    virtual ~BasePointCloudDecoder() = default;
+
+    virtual void setSource(const String &filename) noexcept;
+    virtual void readData(std::vector<Point3f> &points, std::vector<Point3f> &normals);
+    virtual void readData(std::vector<Point3f> &points, std::vector<Point3f> &normals, std::vector<std::vector<int32_t>> &indices) = 0;
+
+protected:
+    String m_filename;
+};
+
+///////////////////////////////// base class for encoders ////////////////////////
+class BasePointCloudEncoder
+{
+public:
+    virtual ~BasePointCloudEncoder() = default;
+
+    virtual void setDestination(const String &filename) noexcept;
+    virtual void writeData(const std::vector<Point3f> &points, const std::vector<Point3f> &normals);
+    virtual void writeData(const std::vector<Point3f> &points, const std::vector<Point3f> &normals, const std::vector<std::vector<int32_t>> &indices) = 0;
+
+protected:
+    String m_filename;
+};
+
+} /* namespace cv */
+
+#endif

modules/3d/src/pointcloud/io_obj.cpp

@@ -0,0 +1,112 @@
+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+
+#include "../precomp.hpp"
+#include "io_obj.hpp"
+#include <fstream>
+#include <opencv2/core/utils/logger.hpp>
+#include "utils.hpp"
+
+namespace cv {
+
+std::unordered_set<std::string> ObjDecoder::m_unsupportedKeys;
+
+void ObjDecoder::readData(std::vector<Point3f> &points, std::vector<Point3f> &normals, std::vector<std::vector<int32_t>> &indices)
+{
+    points.clear();
+    normals.clear();
+    indices.clear();
+
+    std::ifstream file(m_filename, std::ios::binary);
+    if (!file)
+    {
+        CV_LOG_ERROR(NULL, "Impossible to open the file: " << m_filename);
+        return;
+    }
+    std::string s;
+
+    while (!file.eof())
+    {
+        std::getline(file, s);
+        if (s.empty())
+            continue;
+        std::stringstream ss(s);
+        std::string key;
+        ss >> key;
+
+        if (key == "#")
+            continue;
+        else if (key == "v")
+        {
+            Point3f vertex;
+            ss >> vertex.x >> vertex.y >> vertex.z;
+            points.push_back(vertex);
+        }
+        else if (key == "vn")
+        {
+            Point3f normal;
+            ss >> normal.x >> normal.y >> normal.z;
+            normals.push_back(normal);
+        }
+        else if (key == "f")
+        {
+            std::vector<int> vertexInd;
+            auto tokens = split(s, ' ');
+            for (size_t i = 1; i < tokens.size(); i++)
+            {
+                auto vertexinfo = split(tokens[i], '/');
+                int index;
+                std::stringstream vs(vertexinfo[0]);
+                vs >> index;
+                vertexInd.push_back(index - 1);
+            }
+            indices.push_back(vertexInd);
+        }
+        else
+        {
+            if (m_unsupportedKeys.find(key) == m_unsupportedKeys.end()) {
+                m_unsupportedKeys.insert(key);
+                CV_LOG_WARNING(NULL, "Key " << key << " not supported");
+            }
+        }
+    }
+
+    file.close();
+}
+
+void ObjEncoder::writeData(const std::vector<Point3f> &points, const std::vector<Point3f> &normals, const std::vector<std::vector<int32_t>> &indices)
+{
+    std::ofstream file(m_filename, std::ios::binary);
+    if (!file) {
+        CV_LOG_ERROR(NULL, "Impossible to open the file: " << m_filename);
+        return;
+    }
+
+    file << "# OBJ file writer" << std::endl;
+    file << "o Point_Cloud" << std::endl;
+
+    for (const auto& point : points)
+    {
+        file << "v " << point.x << " " << point.y << " " << point.z << std::endl;
+    }
+
+    for (const auto& normal : normals)
+    {
+        file << "vn " << normal.x << " " << normal.y << " " << normal.z << std::endl;
+    }
+
+    for (const auto& faceIndices : indices)
+    {
+        file << "f ";
+        for (const auto& index : faceIndices)
+        {
+            file << index + 1 << " ";
+        }
+        file << std::endl;
+    }
+
+    file.close();
+}
+
+} /* namespace cv */

modules/3d/src/pointcloud/io_obj.hpp

@@ -0,0 +1,31 @@
+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+
+#ifndef _CODERS_OBJ_H_
+#define _CODERS_OBJ_H_
+
+#include "io_base.hpp"
+#include <unordered_set>
+
+namespace cv {
+
+class ObjDecoder CV_FINAL : public BasePointCloudDecoder
+{
+public:
+    void readData(std::vector<Point3f> &points, std::vector<Point3f> &normals, std::vector<std::vector<int32_t>> &indices) CV_OVERRIDE;
+
+protected:
+    static std::unordered_set<std::string> m_unsupportedKeys;
+};
+
+class ObjEncoder CV_FINAL : public BasePointCloudEncoder
+{
+public:
+    void writeData(const std::vector<Point3f> &points, const std::vector<Point3f> &normals, const std::vector<std::vector<int32_t>> &indices) CV_OVERRIDE;
+
+};
+
+} /* namespace cv */
+
+#endif

modules/3d/src/pointcloud/io_ply.cpp

@@ -0,0 +1,205 @@
+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+
+#include "../precomp.hpp"
+#include "io_ply.hpp"
+#include "utils.hpp"
+#include <opencv2/core/utils/logger.hpp>
+#include <fstream>
+#include <string>
+
+namespace cv {
+
+void PlyDecoder::readData(std::vector<Point3f> &points, std::vector<Point3f> &normals, std::vector<std::vector<int32_t>> &indices)
+{
+    points.clear();
+    normals.clear();
+    CV_UNUSED(indices);
+
+    std::ifstream file(m_filename, std::ios::binary);
+    if (parseHeader(file))
+    {
+        parseBody(file, points, normals);
+    }
+}
+
+bool PlyDecoder::parseHeader(std::ifstream &file)
+{
+    std::string s;
+    std::getline(file, s);
+    if (trimSpaces(s) != "ply")
+    {
+        CV_LOG_ERROR(NULL, "Provided file is not in PLY format");
+        return false;
+    }
+    std::getline(file, s);
+    auto splitArr = split(s, ' ');
+    if (splitArr[0] != "format")
+    {
+        CV_LOG_ERROR(NULL, "Provided file doesn't have format");
+        return false;
+    }
+    if (splitArr[1] == "ascii")
+    {
+        m_inputDataFormat = DataFormat::ASCII;
+    }
+    else if (splitArr[1] == "binary_little_endian")
+    {
+        m_inputDataFormat = DataFormat::BinaryLittleEndian;
+    }
+    else if (splitArr[1] == "binary_big_endian")
+    {
+        m_inputDataFormat = DataFormat::BinaryBigEndian;
+    }
+    else
+    {
+        CV_LOG_ERROR(NULL, "Provided PLY file format is not supported");
+        return false;
+    }
+
+    bool onVertexRead = false;
+    while (std::getline(file, s))
+    {
+        if (startsWith(s, "element"))
+        {
+            auto splitArrElem = split(s, ' ');
+            if (splitArrElem[1] == "vertex")
+            {
+                onVertexRead = true;
+                std::istringstream iss(splitArrElem[2]);
+                iss >> m_vertexCount;
+            }
+            else
+            {
+                onVertexRead = false;
+            }
+            continue;
+        }
+        if (startsWith(s, "property"))
+        {
+            if (onVertexRead)
+            {
+                auto splitArrElem = split(s, ' ');
+                if (splitArrElem[2] == "x" || splitArrElem[2] == "red" || splitArrElem[2] == "nx")
+                {
+                    if (splitArrElem[1] != "float") {
+                        CV_LOG_ERROR(NULL, "Provided PLY file format '" << splitArrElem[1] << "' is not supported");
+                        return false;
+                    }
+                }
+                if (splitArrElem[2] == "red")
+                {
+                    m_hasColour = true;
+                }
+                if (splitArrElem[2] == "nx")
+                {
+                    m_hasNormal = true;
+                }
+            }
+            continue;
+        }
+        if (startsWith(s, "end_header"))
+            break;
+
+    }
+    return true;
+}
+
+template <typename T>
+T readNext(std::ifstream &file, DataFormat format)
+{
+    T val;
+    if (format == DataFormat::ASCII)
+    {
+        file >> val;
+        return val;
+    }
+    file.read((char *)&val, sizeof(T));
+#ifdef WORDS_BIGENDIAN
+    if (!(format == DataFormat::BinaryBigEndian) )
+    {
+        swapEndian<T>(val);
+    }
+#else
+    if (format == DataFormat::BinaryBigEndian)
+    {
+        swapEndian<T>(val);
+    }
+#endif
+    return val;
+}
+
+void PlyDecoder::parseBody(std::ifstream &file, std::vector<Point3f> &points, std::vector<Point3f> &normals)
+{
+    points.reserve(m_vertexCount);
+    if (m_hasNormal)
+    {
+        normals.reserve(m_vertexCount);
+    }
+    for (size_t i = 0; i < m_vertexCount; i++)
+    {
+        Point3f vertex;
+        vertex.x = readNext<float>(file, m_inputDataFormat);
+        vertex.y = readNext<float>(file, m_inputDataFormat);
+        vertex.z = readNext<float>(file, m_inputDataFormat);
+        points.push_back(vertex);
+        if (m_hasColour)
+        {
+            readNext<float>(file, m_inputDataFormat);
+            readNext<float>(file, m_inputDataFormat);
+            readNext<float>(file, m_inputDataFormat);
+        }
+        if (m_hasNormal)
+        {
+            Point3f normal;
+            normal.x = readNext<float>(file, m_inputDataFormat);
+            normal.y = readNext<float>(file, m_inputDataFormat);
+            normal.z = readNext<float>(file, m_inputDataFormat);
+            normals.push_back(normal);
+        }
+    }
+}
+
+void PlyEncoder::writeData(const std::vector<Point3f> &points, const std::vector<Point3f> &normals, const std::vector<std::vector<int32_t>> &indices)
+{
+    CV_UNUSED(indices);
+    std::ofstream file(m_filename, std::ios::binary);
+    if (!file) {
+        CV_LOG_ERROR(NULL, "Impossible to open the file: " << m_filename);
+        return;
+    }
+
+    file << "ply" << std::endl;
+    file << "format ascii 1.0" << std::endl;
+    file << "comment created by OpenCV" << std::endl;
+    file << "element vertex " << points.size() << std::endl;
+
+    file << "property float x" << std::endl;
+    file << "property float y" << std::endl;
+    file << "property float z" << std::endl;
+
+    file << "end_header" << std::endl;
+
+    if (normals.size() != 0)
+    {
+        file << "property float nx" << std::endl;
+        file << "property float ny" << std::endl;
+        file << "property float nz" << std::endl;
+    }
+
+    bool isNormals = (normals.size() != 0);
+
+    for (size_t i = 0; i < points.size(); i++)
+    {
+        file << points[i].x << " " << points[i].y << " " << points[i].z;
+        if (isNormals)
+        {
+            file << normals[i].x << " " << normals[i].y << " " << normals[i].z;
+        }
+        file << std::endl;
+    }
+    file.close();
+}
+
+} /* namespace cv */

modules/3d/src/pointcloud/io_ply.hpp

@@ -0,0 +1,45 @@
+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+
+#ifndef _CODERS_PLY_H_
+#define _CODERS_PLY_H_
+
+#include "io_base.hpp"
+#include <istream>
+#include <vector>
+
+namespace cv {
+
+enum class DataFormat
+{
+    ASCII,
+    BinaryLittleEndian,
+    BinaryBigEndian
+};
+
+class PlyDecoder CV_FINAL : public BasePointCloudDecoder
+{
+public:
+    void readData(std::vector<Point3f> &points, std::vector<Point3f> &normals, std::vector<std::vector<int32_t>> &indices) CV_OVERRIDE;
+
+protected:
+    bool parseHeader(std::ifstream &file);
+    void parseBody(std::ifstream &file, std::vector<Point3f> &points, std::vector<Point3f> &normals);
+
+    DataFormat m_inputDataFormat;
+    size_t m_vertexCount{0};
+    bool m_hasColour{false};
+    bool m_hasNormal{false};
+};
+
+class PlyEncoder CV_FINAL : public BasePointCloudEncoder
+{
+public:
+    void writeData(const std::vector<Point3f> &points, const std::vector<Point3f> &normals, const std::vector<std::vector<int32_t>> &indices) CV_OVERRIDE;
+
+};
+
+} /* namespace cv */
+
+#endif

modules/3d/src/pointcloud/load_point_cloud.cpp

@@ -0,0 +1,199 @@
+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+
+#include "../precomp.hpp"
+
+#include "io_base.hpp"
+#include "io_obj.hpp"
+#include "io_ply.hpp"
+#include "utils.hpp"
+#include "opencv2/core/utils/filesystem.private.hpp"
+#include <opencv2/core/utils/logger.hpp>
+
+#include <memory>
+
+namespace cv {
+
+#if OPENCV_HAVE_FILESYSTEM_SUPPORT
+
+static PointCloudDecoder findDecoder(const String &filename)
+{
+    auto file_ext = getExtension(filename);
+    if (file_ext == "obj" || file_ext == "OBJ")
+    {
+        return std::unique_ptr<ObjDecoder>(new ObjDecoder());
+    }
+    if (file_ext == "ply" || file_ext == "PLY")
+    {
+        return std::unique_ptr<PlyDecoder>(new PlyDecoder());
+    }
+
+    return nullptr;
+}
+
+static PointCloudEncoder findEncoder(const String &filename)
+{
+    auto file_ext = getExtension(filename);
+    if (file_ext == "obj" || file_ext == "OBJ")
+    {
+        return std::unique_ptr<ObjEncoder>(new ObjEncoder());
+    }
+    if (file_ext == "ply" || file_ext == "PLY")
+    {
+        return std::unique_ptr<PlyEncoder>(new PlyEncoder());
+    }
+
+    return nullptr;
+}
+
+#endif
+
+void loadPointCloud(const String &filename, OutputArray vertices, OutputArray normals)
+{
+#if OPENCV_HAVE_FILESYSTEM_SUPPORT
+    auto decoder = findDecoder(filename);
+    if (!decoder) {
+        String file_ext = getExtension(filename);
+        CV_LOG_ERROR(NULL, "File extension '" << file_ext << "' is not supported");
+        return;
+    }
+
+    decoder->setSource(filename);
+
+    std::vector<Point3f> vec_vertices;
+    std::vector<Point3f> vec_normals;
+
+    decoder->readData(vec_vertices, vec_normals);
+
+    if (!vec_vertices.empty())
+        Mat(static_cast<int>(vec_vertices.size()), 1, CV_32FC3, &vec_vertices[0]).copyTo(vertices);
+
+    if (!vec_normals.empty() && normals.needed())
+        Mat(static_cast<int>(vec_normals.size()), 1, CV_32FC3, &vec_normals[0]).copyTo(normals);
+#else // OPENCV_HAVE_FILESYSTEM_SUPPORT
+    CV_UNUSED(filename);
+    CV_UNUSED(vertices);
+    CV_UNUSED(normals);
+    CV_LOG_WARNING(NULL, "File system support is disabled in this OpenCV build!");
+#endif
+}
+
+void savePointCloud(const String &filename, InputArray vertices, InputArray normals)
+{
+#if OPENCV_HAVE_FILESYSTEM_SUPPORT
+    if (vertices.empty()) {
+        CV_LOG_WARNING(NULL, "Have no vertices to save");
+        return;
+    };
+
+    auto encoder = findEncoder(filename);
+    if (!encoder) {
+        String file_ext = getExtension(filename);
+        CV_LOG_ERROR(NULL, "File extension '" << file_ext << "' is not supported");
+        return;
+    }
+
+    encoder->setDestination(filename);
+
+    std::vector<Point3f> vec_vertices(vertices.getMat());
+    std::vector<Point3f> vec_normals;
+    if (!normals.empty()){
+        vec_normals = normals.getMat();
+    }
+
+    encoder->writeData(vec_vertices, vec_normals);
+
+#else // OPENCV_HAVE_FILESYSTEM_SUPPORT
+    CV_UNUSED(filename);
+    CV_UNUSED(vertices);
+    CV_UNUSED(normals);
+    CV_LOG_WARNING(NULL, "File system support is disabled in this OpenCV build!");
+#endif
+}
+
+void loadMesh(const String &filename, OutputArray vertices, OutputArray normals, OutputArrayOfArrays indices)
+{
+#if OPENCV_HAVE_FILESYSTEM_SUPPORT
+    PointCloudDecoder decoder = findDecoder(filename);
+    String file_ext = getExtension(filename);
+    if (!decoder || (file_ext != "obj" && file_ext != "OBJ")) {
+        CV_LOG_ERROR(NULL, "File extension '" << file_ext << "' is not supported");
+        return;
+    }
+
+    decoder->setSource(filename);
+
+    std::vector<Point3f> vec_vertices;
+    std::vector<Point3f> vec_normals;
+    std::vector<std::vector<int32_t>> vec_indices;
+
+    decoder->readData(vec_vertices, vec_normals, vec_indices);
+
+    if (!vec_vertices.empty()) {
+        Mat(1, static_cast<int>(vec_vertices.size()), CV_32FC3, vec_vertices.data()).copyTo(vertices);
+    }
+
+    if (!vec_normals.empty()) {
+        Mat(1, static_cast<int>(vec_normals.size()), CV_32FC3, vec_normals.data()).copyTo(normals);
+    }
+
+    if (!vec_indices.empty()) {
+        std::vector<std::vector<int32_t>>& vec = *(std::vector<std::vector<int32_t>>*)indices.getObj();
+        vec.resize(vec_indices.size());
+        for (size_t i = 0; i < vec_indices.size(); ++i) {
+            Mat(1, static_cast<int>(vec_indices[i].size()), CV_32SC1, vec_indices[i].data()).copyTo(vec[i]);
+        }
+    }
+
+#else // OPENCV_HAVE_FILESYSTEM_SUPPORT
+    CV_UNUSED(filename);
+    CV_UNUSED(vertices);
+    CV_UNUSED(normals);
+    CV_LOG_WARNING(NULL, "File system support is disabled in this OpenCV build!");
+#endif
+}
+
+void saveMesh(const String &filename, InputArray vertices, InputArray normals, InputArrayOfArrays indices)
+{
+#if OPENCV_HAVE_FILESYSTEM_SUPPORT
+    if (vertices.empty()) {
+        CV_LOG_WARNING(NULL, "Have no vertices to save");
+        return;
+    }
+
+    auto encoder = findEncoder(filename);
+    String file_ext = getExtension(filename);
+    if (!encoder || (file_ext != "obj" && file_ext != "OBJ")) {
+        CV_LOG_ERROR(NULL, "File extension '" << file_ext << "' is not supported");
+        return;
+    }
+
+    encoder->setDestination(filename);
+
+    std::vector<Point3f> vec_vertices(vertices.getMat());
+    std::vector<Point3f> vec_normals;
+    if (!normals.empty()){
+        vec_normals = normals.getMat();
+    }
+
+    std::vector<Mat> mat_indices;
+    indices.getMatVector(mat_indices);
+    std::vector<std::vector<int32_t>> vec_indices(mat_indices.size());
+
+    for (size_t i = 0; i < mat_indices.size(); ++i) {
+        mat_indices[i].copyTo(vec_indices[i]);
+    }
+
+    encoder->writeData(vec_vertices, vec_normals, vec_indices);
+
+#else // OPENCV_HAVE_FILESYSTEM_SUPPORT
+    CV_UNUSED(filename);
+    CV_UNUSED(vertices);
+    CV_UNUSED(normals);
+    CV_LOG_WARNING(NULL, "File system support is disabled in this OpenCV build!");
+#endif
+
+}
+
+}/* namespace cv */

modules/3d/src/pointcloud/utils.cpp

@@ -0,0 +1,21 @@
+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+
+#include "utils.hpp"
+
+namespace cv {
+
+std::vector<std::string> split(const std::string &s, char delimiter)
+{
+    std::vector<std::string> tokens;
+    std::string token;
+    std::istringstream tokenStream(s);
+    while (std::getline(tokenStream, token, delimiter))
+    {
+        tokens.push_back(token);
+    }
+    return tokens;
+}
+
+} /* namespace cv */

modules/3d/src/pointcloud/utils.hpp

@@ -0,0 +1,65 @@
+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+
+#ifndef _CODERS_UTILS_H_
+#define _CODERS_UTILS_H_
+
+#include <string>
+#include <vector>
+#include <sstream>
+#include <array>
+#include <algorithm>
+
+
+namespace cv {
+
+std::vector<std::string> split(const std::string &s, char delimiter);
+
+inline bool startsWith(const std::string &s1, const std::string &s2)
+{
+    return s1.compare(0, s2.length(), s2) == 0;
+}
+
+inline std::string trimSpaces(const std::string &input)
+{
+    size_t start = 0;
+    while (start < input.size() && input[start] == ' ')
+    {
+        start++;
+    }
+    size_t end = input.size();
+    while (end > start && (input[end - 1] == ' ' || input[end - 1] == '\n' || input[end - 1] == '\r'))
+    {
+        end--;
+    }
+    return input.substr(start, end - start);
+}
+
+inline std::string getExtension(const std::string& filename)
+{
+    auto pos = filename.find_last_of('.');
+    if (pos == std::string::npos)
+    {
+        return "";
+    }
+    return filename.substr( pos + 1);
+}
+
+template <typename T>
+void swapEndian(T &val)
+{
+    union U
+    {
+        T val;
+        std::array<std::uint8_t, sizeof(T)> raw;
+    } src, dst;
+
+    src.val = val;
+    std::reverse_copy(src.raw.begin(), src.raw.end(), dst.raw.begin());
+    val = dst.val;
+}
+
+} /* namespace cv */
+
+#endif

modules/3d/test/test_pointcloud.cpp

@@ -0,0 +1,186 @@
+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+
+#include <opencv2/core.hpp>
+#include <vector>
+#include <cstdio>
+
+#include "test_precomp.hpp"
+#include "opencv2/ts.hpp"
+
+namespace opencv_test { namespace {
+
+TEST(PointCloud, LoadObj)
+{
+    std::vector<cv::Point3f> points_gold = {
+        {-5.93915f, -0.13257f, 2.55837f},
+        {-5.93915f, 1.86743f, 2.55837f},
+        {-5.93915f, -0.13257f, -1.16339f},
+        {-5.93915f, 1.86743f, -1.16339f},
+        {0.399941f, -0.13257f, 2.55837f},
+        {0.399941f, 1.86743f, 2.55837f},
+        {0.399941f, -0.13257f, -1.16339f},
+        {0.399941f, 1.86743f, -1.16339f}};
+    std::vector<cv::Point3f> normals_gold = {
+        {-1.0000f, 0.0000f, 0.0000f},
+        {0.0000f, 0.0000f, -1.0000f},
+        {1.0000f, 0.0000f, 0.0000f},
+        {0.0000f, 0.0000f, 1.0000f},
+        {0.0000f, -1.0000f, 0.0000f},
+        {0.0000f, 1.0000f, 0.0000f}};
+
+    std::vector<cv::Point3f> points;
+    std::vector<cv::Point3f> normals;
+
+    auto folder = cvtest::TS::ptr()->get_data_path();
+    cv::loadPointCloud(folder + "pointcloudio/orig.obj", points, normals);
+
+    EXPECT_EQ(points_gold, points);
+    EXPECT_EQ(normals_gold, normals);
+}
+
+TEST(PointCloud, LoadObjNoNormals)
+{
+    std::vector<cv::Point3f> points_gold = {
+        {-5.93915f, -0.13257f, 2.55837f},
+        {-5.93915f, 1.86743f, 2.55837f},
+        {-5.93915f, -0.13257f, -1.16339f},
+        {-5.93915f, 1.86743f, -1.16339f},
+        {0.399941f, -0.13257f, 2.55837f},
+        {0.399941f, 1.86743f, 2.55837f},
+        {0.399941f, -0.13257f, -1.16339f},
+        {0.399941f, 1.86743f, -1.16339f}};
+
+    std::vector<cv::Point3f> points;
+    std::vector<cv::Point3f> normals;
+
+    auto folder = cvtest::TS::ptr()->get_data_path();
+    cv::loadPointCloud(folder + "pointcloudio/orig_no_norms.obj", points, normals);
+
+    EXPECT_EQ(points_gold, points);
+    EXPECT_TRUE(normals.empty());
+}
+
+TEST(PointCloud, SaveObj)
+{
+    std::vector<cv::Point3f> points_gold;
+    std::vector<cv::Point3f> normals_gold;
+
+    auto folder = cvtest::TS::ptr()->get_data_path();
+    auto new_path = tempfile("new.obj");
+
+    cv::loadPointCloud(folder + "pointcloudio/orig.obj", points_gold, normals_gold);
+    cv::savePointCloud(new_path, points_gold, normals_gold);
+
+    std::vector<cv::Point3f> points;
+    std::vector<cv::Point3f> normals;
+
+    cv::loadPointCloud(new_path, points, normals);
+
+    EXPECT_EQ(normals, normals_gold);
+    EXPECT_EQ(points, points_gold);
+    std::remove(new_path.c_str());
+}
+
+TEST(PointCloud, LoadSavePly)
+{
+    std::vector<cv::Point3f> points;
+    std::vector<cv::Point3f> normals;
+
+    auto folder = cvtest::TS::ptr()->get_data_path();
+    std::string new_path = tempfile("new.ply");
+
+    cv::loadPointCloud(folder + "pointcloudio/orig.ply", points, normals);
+    cv::savePointCloud(new_path, points, normals);
+
+    std::vector<cv::Point3f> points_gold;
+    std::vector<cv::Point3f> normals_gold;
+
+    cv::loadPointCloud(new_path, points_gold, normals_gold);
+
+    EXPECT_EQ(normals_gold, normals);
+    EXPECT_EQ(points_gold, points);
+    std::remove(new_path.c_str());
+}
+
+TEST(PointCloud, LoadSaveMeshObj)
+{
+    std::vector<cv::Point3f> points;
+    std::vector<cv::Point3f> normals;
+    std::vector<std::vector<int32_t>> indices;
+
+    auto folder = cvtest::TS::ptr()->get_data_path();
+    std::string new_path = tempfile("new_mesh.obj");
+
+    cv::loadMesh(folder + "pointcloudio/orig.obj", points, normals, indices);
+    cv::saveMesh(new_path, points, normals, indices);
+
+    std::vector<cv::Point3f> points_gold;
+    std::vector<cv::Point3f> normals_gold;
+    std::vector<std::vector<int32_t>> indices_gold;
+
+    cv::loadMesh(new_path, points_gold, normals_gold, indices_gold);
+
+    EXPECT_EQ(normals_gold, normals);
+    EXPECT_EQ(points_gold, points);
+    EXPECT_EQ(indices_gold, indices);
+    EXPECT_TRUE(!indices.empty());
+    std::remove(new_path.c_str());
+}
+
+TEST(PointCloud, LoadSaveMeshPly)
+{
+    std::vector<cv::Point3f> points;
+    std::vector<cv::Point3f> normals;
+    std::vector<std::vector<int32_t>> indices;
+
+    auto folder = cvtest::TS::ptr()->get_data_path();
+    std::string new_path = tempfile("new_mesh.ply");
+
+    // we don't support meshes in PLY format right now but it should exit silently
+    cv::loadMesh(folder + "pointcloudio/orig.ply", points, normals, indices);
+    EXPECT_TRUE(points.empty());
+    EXPECT_TRUE(normals.empty());
+    EXPECT_TRUE(indices.empty());
+
+    cv::saveMesh(new_path, points, normals, indices);
+    EXPECT_TRUE(points.empty());
+    EXPECT_TRUE(normals.empty());
+    EXPECT_TRUE(indices.empty());
+
+    std::remove(new_path.c_str());
+}
+
+TEST(PointCloud, NonexistentFile)
+{
+    std::vector<cv::Point3f> points;
+    std::vector<cv::Point3f> normals;
+
+    auto folder = cvtest::TS::ptr()->get_data_path();
+    cv::loadPointCloud(folder + "pointcloudio/fake.obj", points, normals);
+    EXPECT_TRUE(points.empty());
+    EXPECT_TRUE(normals.empty());
+}
+
+TEST(PointCloud, LoadBadExtension)
+{
+    std::vector<cv::Point3f> points;
+    std::vector<cv::Point3f> normals;
+
+    auto folder = cvtest::TS::ptr()->get_data_path();
+    cv::loadPointCloud(folder + "pointcloudio/fake.fake", points, normals);
+    EXPECT_TRUE(points.empty());
+    EXPECT_TRUE(normals.empty());
+}
+
+TEST(PointCloud, SaveBadExtension)
+{
+    std::vector<cv::Point3f> points;
+    std::vector<cv::Point3f> normals;
+
+    auto folder = cvtest::TS::ptr()->get_data_path();
+    cv::savePointCloud(folder + "pointcloudio/fake.fake", points, normals);
+}
+
+}} /* namespace opencv_test */

samples/python/essential_mat_reconstr.py

@@ -1,5 +1,4 @@
-import numpy as np, cv2 as cv, matplotlib.pyplot as plt, time, sys, os
-from mpl_toolkits.mplot3d import axes3d, Axes3D
+import numpy as np, cv2 as cv, time, sys, os
 
 def getEpipolarError(F, pts1_, pts2_, inliers):
     pts1 = np.concatenate((pts1_.T, np.ones((1, pts1_.shape[0]))))[:,inliers]
@@ -119,19 +118,16 @@ if __name__ == '__main__':
                                  int(np.sqrt (image1.shape[0] * new_img_size / image1.shape[1]))))
 
     # plot object points
-    fig = plt.figure(figsize=(13.0, 11.0))
-    ax = fig.add_subplot(111, projection='3d')
-    ax.set_aspect('equal')
-    ax.scatter(obj_pts[:,0], obj_pts[:,1], obj_pts[:,2], c='r', marker='o', s=3)
-    ax.set_xlabel('x')
-    ax.set_ylabel('y')
-    ax.set_zlabel('depth')
-    ax.view_init(azim=-80, elev=110)
+    color = [1.0, 0.0, 0.0]
+    colors = np.tile(color, (obj_pts.shape[0], 1))
+    obj_pts = np.concatenate((obj_pts, colors), axis=1)
+    obj_pts= np.float32(obj_pts)
+
+    cv.viz3d.showPoints("window", "points", obj_pts)
+    cv.viz3d.setGridVisible("window", True)
 
     # save figures
     cv.imshow("matches", image1)
     cv.imwrite('matches_E.png', image1)
-    plt.savefig('reconstruction_3D.png')
 
     cv.waitKey(0)
-    plt.show()

samples/python/point_cloud.py

@@ -0,0 +1,23 @@
+import numpy as np
+import cv2 as cv
+
+vertices, _ = cv.loadPointCloud("../data/teapot.obj")
+vertices = np.squeeze(vertices, axis=1)
+print(vertices)
+
+color = [1.0, 1.0, 0.0]
+colors = np.tile(color, (vertices.shape[0], 1))
+obj_pts = np.concatenate((vertices, colors), axis=1)
+obj_pts= np.float32(obj_pts)
+
+cv.viz3d.showPoints("window", "points", obj_pts)
+cv.viz3d.setGridVisible("window", True)
+
+cv.waitKey(0)
+
+vertices, _, indices = cv.loadMesh("../data/teapot.obj")
+vertices = np.squeeze(vertices, axis=1)
+
+cv.viz3d.showMesh("window", "mesh", vertices, indices)
+
+cv.waitKey(0)