alloc_frame_buffer in ff_alloc_picture asserts that the linesize
of planes 1 and 2 are the same. If the pixfmt has a single uv
plane, like NV12, this won't be true.
So, let's only do this check if there are more than 2 planes.
We never hit this with previous hw formats because they don't set
linesize to meaningful values, but the cuda hw format sets the
values based on the underlying data layout.
Use mask buffer.
Signed-off-by: Kaustubh Raste <kaustubh.raste@imgtec.com>
Reviewed-by: Manojkumar Bhosale <Manojkumar.Bhosale@imgtec.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This is a partial fix - to fix videotoolbox fully (with multithreaded
hwaccel decoding), much more work would be required.
Before this change, an assertion failure would trigger:
$ ffmpeg -hwaccel videotoolbox -hwaccel_output_format videotoolbox_vld -i h264.ts -f null -y /dev/null
Assertion (frame->private_ref && frame->private_ref->size == sizeof(FrameDecodeData)) || !(avctx->codec->capabilities & (1 << 1)) failed at libavcodec/decode.c:620
Signed-off-by: Aman Gupta <aman@tmm1.net>
This includes the SEI_PREFIX/SEI_SUFFIX NALUs, which can contain
updates like HEVC_SEI_TYPE_ACTIVE_PARAMETER_SETS. Previously, hevc
samples with this SEI present would not playback correctly.
See for example https://github.com/lhc70000/iina/issues/1123
Signed-off-by: Aman Gupta <aman@tmm1.net>
Removes the avctx->extradata_size requirement when creating avcC/hvcC, since
avctx->extradata is only used in the esds code path.
This fixes an issue where the VideoToolbox decoder would not work unless
avformat_find_stream_info() was called.
Signed-off-by: Aman Gupta <aman@tmm1.net>
This fixes decoding of H264 video samples with SPS and PPS changes.
See for example https://s3.amazonaws.com/tmm1/videotoolbox/spschange.ts,
which previously stalled the decoder and failed to produce any new frames
after the SPS change.
Also see https://s3.amazonaws.com/tmm1/videotoolbox/ppschange.ts, which
uses multiple PPS and would previously cause VT decode failures.
If the VideoToolbox session needs to be restarted, and
videotoolbox_start() fails for some reason (for instance, if the video
is interlaced and the decoder is running on iOS), avcodec will return
AVERROR_EXTERNAL. This can be used by the API user to switch to another
decoder.
Signed-off-by: Aman Gupta <aman@tmm1.net>
This callback will be used by the VideoToolbox H264 hwaccel so that it
can receive SPS and PPS NALUs. VideoToolbox requires PPS changes to be
fed into the decoder session, and for the session to be recreated when
the SPS changes.
Signed-off-by: Aman Gupta <aman@tmm1.net>
The only reason videotoolbox wouldn't produce frames is if the data fed
to it was invalid, so returning AVERROR_INVALIDDATA makes sense here.
Further, it means AVERROR_EXTERNAL can be used in further commits to signal
fatal VideoToolbox errors, letting the user know that they need to fallback to
another decoder.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
This allows decode_slice to be invoked multiple times before end_frame,
causing slices to accumulate before being fed into the VT decoder.
An upcoming commit will re-use decode_slice for parameter NALUs, so
they can be propagated into the VT decoder session along with slice
data.
Signed-off-by: Aman Gupta <aman@tmm1.net>
These helpers will be used in later commits to automatically restart
the decoder session when SPS changes are encountered.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Fixes: runtime error: signed integer overflow: 1939661764 - -454942263 cannot be represented in type 'int'
Fixes: 3191/clusterfuzz-testcase-minimized-5688798451073024
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: left shift of negative value -801112064
Fixes: 3492/clusterfuzz-testcase-minimized-5784775283441664
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: signed integer overflow: 1219998458 - -1469874012 cannot be represented in type 'int'
Fixes: 3443/clusterfuzz-testcase-minimized-5369987105554432
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 3984/clusterfuzz-testcase-minimized-5265759929368576
Fixes: runtime error: signed integer overflow: -1085585801 + -1094995529 cannot be represented in type 'int'
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Variables used in inline assembly need to be marked with attribute((used)).
Static constants already were, via the define of DECLARE_ASM_CONST.
But DECLARE_ALIGNED does not add this attribute, and some of the variables
defined with it are const only used in inline assembly, and therefore
appeared dead. This change adds a macro DECLARE_ASM_ALIGNED that marks
variables as used.
This change makes FFMPEG work with Clang's ThinLTO.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
With pre-2.0 Intel drivers in CBR mode, if an explicit SEI message with
the old (now deprecated) type is not included, the driver generates and
inserts some timing SEI which is almost certainly invlaid. Before
7a4fac5e91 we always inserted our own SEI
so this would not be visible, but since then it has been possible to
disable that. We would also like to avoid using the deprecated type,
and using the new type, while working in old drivers, does not suppress
the spurious message like the old type does.
Therefore, suppress the bad SEI insertion by providing a zero-length
buffer with the old type, which the driver can insert harmlessly.
Hendrik Leppkes
James Almer
Philip Langdale
Kaustubh Raste
Aman Gupta
Michael Niedermayer
wm4
Timo Rothenpieler
Thomas Köppe
Mark Thompson
Carl Eugen Hoyos