videoio: Add raw encoded video stream muxing to cv::VideoWriter with CAP_FFMPEG #24363
Allow raw encoded video streams (e.g. h264[5]) to be encapsulated by `cv::VideoWriter` to video containers (e.g. mp4/mkv).
Operates in a similar way to https://github.com/opencv/opencv/pull/15290 where encapsulation is enabled by setting the `VideoWriterProperties::VIDEOWRITER_PROP_RAW_VIDEO` flag when constructing `cv::VideoWriter` e.g.
```
VideoWriter container(fileNameOut, api, fourcc, fps, { width, height }, { VideoWriterProperties::VIDEOWRITER_PROP_RAW_VIDEO, 1 });
```
and each raw encoded frame is passed as single row of a `CV_8U` `cv::Mat`.
The main reason for this PR is to allow `cudacodec::VideoWriter` to output its encoded streams to a suitable container, see https://github.com/opencv/opencv_contrib/pull/3569.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Modify the outputVideoFormat after changing the output format in MSMF backend #24142
After changing the output format, need to modify the outputVideoFormat, otherwise the outputVideoFormat is always CV_CAP_MODE_BGR, and an error will occur when converting the format in retrieveVideoFrame(), and will always enter "case CV_CAP_MODE_BGR:" process.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [ ] I agree to contribute to the project under Apache 2 License.
- [ ] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Co-authored-by: 李龙 <lilong@sobey.com>
Add V4L2_PIX_FMT_Y16_BE pixel format #18498
Address #18495
relates to #23944
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
- [ ] The PR is proposed to proper branch
- [x] There is reference to original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
- [ ] Test using Melexis MLX90640
videoio/FFmpeg: increased packet read attempt limit, allow configuring it
resolves#9455
related #3225
* Use different counters for wrong packets recieved by demuxer and errors from decoder
* Allow modifying these counters via environment variables `OPENCV_FFMPEG_READ_ATTEMPTS`/`OPENCV_FFMPEG_DECODE_ATTEMPTS`
* Added logging when reading breaks at one of error limits
Notes:
* I've been able to reproduce original issue with a video file with 14 total streams (video + audio + subtitles), at some point in the video only packets from the last stream are being sent by the demuxer, thus exceeding our limit. For my specific video total number of packets from wrong stream was about 2700. I've chosen 4096 as default value.
* Default limit of decoding attempts is quite low, because I'm not sure in which cases it can be exceeded (network stream?). I tried to read 8k video from the disk, but it did not cause break at decode point.
Support VideoCapture CAP_PROP_AUTO_WB and CV_CAP_PROP_WHITE_BALANCE_BLUE_U for DShow
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [OK] I agree to contribute to the project under Apache 2 License.
- [OK] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [OK] The PR is proposed to the proper branch
- [OK] There is a reference to the original bug report and related work
https://github.com/opencv/opencv/issues/19621https://github.com/opencv/opencv/issues/21408
### Before apply this pull request console output.
before AWB setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: -1
after AWB disable setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: -1
after AWB enable setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: -1
after Manual WB(and Disable AWB) setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: -1
### After apply this pull request console output.
before AWB setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: 0
after AWB disable setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 4000
CAP_PROP_AUTO_WB: 0
after AWB enable setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 4000
CAP_PROP_AUTO_WB: 1
after Manual WB(and Disable AWB) setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: 0
### Test Code
[OpenCvVideoCapTest.zip](https://github.com/opencv/opencv/files/10825399/OpenCvVideoCapTest.zip)
Adding HEVC/H265 FourCC support to MSMF video writer
* Adding HEVC/H265 fourcc to MSMF video writer
Adding HEVC/H265 fourcc to MSMF video writer. I have verified it with my own video input stream, and it works well on my workstation.
* Update video io testing
* Adding macro fence to get rid of compiler error
H265/HEVC encoder is only available in Windows or later. https://learn.microsoft.com/en-us/windows/win32/medfound/h-265---hevc-video-encoder
* Update test_video_io.cpp
videoio: add Orbbec Gemini 2 and Astra 2 camera support
### Test Result
| OS | Compiler | Camera | Result |
|-----|-----------|---------|--------|
|Windows11| (VS2022)MSVC17.3|Orbbec Gemini 2|Pass|
|Windows11| (VS2022)MSVC17.3|Orbbec Astra 2|Pass|
|Ubuntu22.04|GCC9.2|Orbbec Gemini 2|Pass|
|Ubuntu22.04|GCC9.2|Orbbec Astra 2|Pass|
### Pull Request Readiness Checklist
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] The feature is well documented and sample code can be built with the project CMake
Address https://github.com/opencv/opencv/issues/22868
Used the same defaults as it's done for FFmpeg
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
```
force_builders=Custom
build_image:Custom=gstreamer:16.04
buildworker:Custom=linux-1
```
Introduce libavdevice to make v4l2 available to the ffmpeg backend
* introduce libavdevice to make v4l2 available to the ffmpeg backend
* downgrade the min required libavdevice version to 53.2.0
* make libavdevice optional
* create OCV_OPTION OPENCV_FFMPEG_ENABLE_LIBAVDEVICE and add definition through ocv_add_external_target
* move OCV_OPTION 'OPENCV_FFMPEG_ENABLE_LIBAVDEVICE' to detect_ffmpeg.cmake
Setting CAP_PROP_AUTO_EXPOSURE on VideoCapture with backend DSHOW does not change anything. Now with this implementation the property can be used with value 1 for availability.
* Allow the number of threads FFMpeg uses to be selected during VideoCapture::open().
Reset interupt timer in grab if
err = avformat_find_stream_info(ic, NULL);
is interupted but open is successful.
* Correct the returned number of threads and amend test cases.
* Update container test case.
* Reverse changes added to existing videoio_container test case and include test combining thread change and raw read in the newly added videoio_read test case.
Alexander Smorkalov
cudawarped
jason_w
beanjoy
Alexander Lyulkov
DeePingXian
Maksim Shabunin
firebladed
unknown
vovka643
kallaballa
gottagofaster236
ippei.i
hzcyf
Lilo Huang
Maxim Milashchenko
Alexander Alekhin
hzc
Alexander Smorkalov
Alexander Smorkalov
Vadim Levin
Stefan Dragnev
Hashem Zavvari
Giles Payne
Dan
Daisuke Mizobuchi
MENG Yu