videoio: fix cv::VideoWriter with FFmpeg encapsulation timestamps #25874
Fix https://github.com/opencv/opencv/issues/25873 by modifying `cv::VideoWriter` to use provided presentation indices (pts).
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
CAP_PROP_CODEC_EXTRADATA_INDEX=68,//!< Positive index indicates that returning extra data is supported by the video back end. This can be retrieved as cap.retrieve(data, <returned index>). E.g. When reading from a h264 encoded RTSP stream, the FFmpeg backend could return the SPS and/or PPS if available (if sent in reply to a DESCRIBE request), from calls to cap.retrieve(data, <returned index>).
CAP_PROP_FRAME_TYPE=69,//!< (read-only) FFmpeg back-end only - Frame type ascii code (73 = 'I', 80 = 'P', 66 = 'B' or 63 = '?' if unknown) of the most recently read frame.
CAP_PROP_N_THREADS=70,//!< (**open-only**) Set the maximum number of threads to use. Use 0 to use as many threads as CPU cores (applicable for FFmpeg back-end only).
CAP_PROP_PTS=71,//!< (read-only) FFmpeg back-end only - presentation timestamp of the most recently read frame using the FPS time base. e.g. fps = 25, VideoCapture::get(\ref CAP_PROP_PTS) = 3, presentation time = 3/25 seconds.
CAP_PROP_DTS_DELAY=72,//!< (read-only) FFmpeg back-end only - maximum difference between presentation (pts) and decompression timestamps (dts) using FPS time base. e.g. delay is maximum when frame_num = 0, if true, VideoCapture::get(\ref CAP_PROP_PTS) = 0 and VideoCapture::get(\ref CAP_PROP_DTS_DELAY) = 2, dts = -2. Non zero values usually imply the stream is encoded using B-frames which are not decoded in presentation order.
#ifndef CV_DOXYGEN
CV__CAP_PROP_LATEST
#endif
@ -230,8 +232,10 @@ enum VideoWriterProperties {
VIDEOWRITER_PROP_HW_DEVICE=7,//!< (**open-only**) Hardware device index (select GPU if multiple available). Device enumeration is acceleration type specific.
VIDEOWRITER_PROP_HW_ACCELERATION_USE_OPENCL=8,//!< (**open-only**) If non-zero, create new OpenCL context and bind it to current thread. The OpenCL context created with Video Acceleration context attached it (if not attached yet) for optimized GPU data copy between cv::UMat and HW accelerated encoder.
VIDEOWRITER_PROP_RAW_VIDEO=9,//!< (**open-only**) Set to non-zero to enable encapsulation of an encoded raw video stream. Each raw encoded video frame should be passed to VideoWriter::write() as single row or column of a \ref CV_8UC1 Mat. \note If the key frame interval is not 1 then it must be manually specified by the user. This can either be performed during initialization passing \ref VIDEOWRITER_PROP_KEY_INTERVAL as one of the extra encoder params to \ref VideoWriter::VideoWriter(const String &, int, double, const Size &, const std::vector< int > ¶ms) or afterwards by setting the \ref VIDEOWRITER_PROP_KEY_FLAG with \ref VideoWriter::set() before writing each frame. FFMpeg backend only.
VIDEOWRITER_PROP_KEY_INTERVAL=10,//!< (**open-only**) Set the key frame interval using raw video encapsulation (\ref VIDEOWRITER_PROP_RAW_VIDEO != 0). Defaults to 1 when not set. FFMpeg backend only.
VIDEOWRITER_PROP_KEY_FLAG=11,//!< Set to non-zero to signal that the following frames are key frames or zero if not, when encapsulating raw video (\ref VIDEOWRITER_PROP_RAW_VIDEO != 0). FFMpeg backend only.
VIDEOWRITER_PROP_KEY_INTERVAL=10,//!< (**open-only**) Set the key frame interval using raw video encapsulation (\ref VIDEOWRITER_PROP_RAW_VIDEO != 0). Defaults to 1 when not set. FFmpeg back-end only.
VIDEOWRITER_PROP_KEY_FLAG=11,//!< Set to non-zero to signal that the following frames are key frames or zero if not, when encapsulating raw video (\ref VIDEOWRITER_PROP_RAW_VIDEO != 0). FFmpeg back-end only.
VIDEOWRITER_PROP_PTS=12,//!< Specifies the frame presentation timestamp for each frame using the FPS time base. This property is **only** necessary when encapsulating **externally** encoded video where the decoding order differs from the presentation order, such as in GOP patterns with bi-directional B-frames. The value should be provided by your external encoder and for video sources with fixed frame rates it is equivalent to dividing the current frame's presentation time (\ref CAP_PROP_POS_MSEC) by the frame duration (1000.0 / VideoCapture::get(\ref CAP_PROP_FPS)). It can be queried from the resulting encapsulated video file using VideoCapture::get(\ref CAP_PROP_PTS). FFmpeg back-end only.
VIDEOWRITER_PROP_DTS_DELAY=13,//!< Specifies the maximum difference between presentation (pts) and decompression timestamps (dts) using the FPS time base. This property is necessary **only** when encapsulating **externally** encoded video where the decoding order differs from the presentation order, such as in GOP patterns with bi-directional B-frames. The value should be calculated based on the specific GOP pattern used during encoding. For example, in a GOP with presentation order IBP and decoding order IPB, this value would be 1, as the B-frame is the second frame presented but the third to be decoded. It can be queried from the resulting encapsulated video file using VideoCapture::get(\ref CAP_PROP_DTS_DELAY). Non-zero values usually imply the stream is encoded using B-frames. FFmpeg back-end only.
videoio_encapsulate_params_t("video/VID00003-20100701-2204.mpg","mp4",12,13,false,false),// tsWorking = false: PTS not available for all frames
videoio_encapsulate_params_t("video/VID00003-20100701-2204.avi","mp4",12,13,false,false),// tsWorking = false: Unable to correctly set PTS when writing
videoio_encapsulate_params_t("video/VID00003-20100701-2204.3GP","mp4",51,52,false,false),// tsWorking = false: Source with variable fps
cudawarped
8 months ago
committed by
GitHub