1. If user don't specify the profile, set it to main10 when pixel
format is 10 bits. Before the patch, videotoolbox output main
profile bitstream with 10 bit input, which can be confusing.
2. Warning when user set profile to main explicitly with 10 bit
input. It works, but not the best choice.
3. Reject main 10 profile with 8 bit input, it doesn't work.
Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
Will be used in the following patch. With hw_config we can get
avctx->hw_frames_ctx, and with avctx->hw_frames_ctx we get
sw_pix_fmt. Otherwise sw_pix_fmt is none. I need sw_pix_fmt
before get the first frame to set hevc encoder profile.
Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
In f7ac3512f5 the size of the dynamically
allocated buffer was shrunk, but it was made too small for very small
alphabet sizes. This patch restores the size to prevent an OOB read.
Reported-by: Cole Dilorenzo <coolkingcole@gmail.com>
Signed-off-by: Leo Izen <leo.izen@gmail.com>
EAGAIN causes an assertion failure when it is returned from the decoder
Fixes: Assertion consumed != (-(11)) failed at libavcodec/decode.c:462
Fixes: assertion_IOT_instruction_decode_c_462/poc
Found-by: Hardik Shah of Vehere (Dawn Treaders team)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 1496950099 + 728014168 cannot be represented in type 'int'
Fixes: 62667/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MJPEGB_fuzzer-6511785170305024
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: -2146469728 - 1488954 cannot be represented in type 'int'
Fixes: 62490/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_BONK_fuzzer-5612782399389696
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access
Fixes: 62678/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-4858264984354816
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Kieran Kunhya <kierank@obe.tv>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
They are generally set in ff_mpv_init_context_frame()
(mostly called by ff_mpv_common_init()); setting them
somewhere else should be avoided.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Forgotten in 0eb399ac39.
While just at it, also use a forward declaration.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
It is unused by ff_mpeg4_decode_picture_header() (unsurprisingly given
that when decoding this function is called before the context has been
initialized).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(The return value doesn't really matter: For video decoders
every return value >= 0 is treated as "consumed all of the input".)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This patch makes the libkvazaar encoder respect color settings that are
present on the codec context, including color range, primaries, transfer
function and colorspace.
Since at least commit c954cf1e1b
(adding ff_encode_alloc_frame()), a large part of ff_alloc_picture()
is completely separate for the two callers. Move the caller-specific
parts out to the callers.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
It is unnecessary in case of user-supplied frames, because
it happens directly after a av_frame_ref() with the same
src and dst.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
ff_alloc_picture() performs two tasks: a) In most instances,
it allocates frame buffers and b) it allocates certain
auxiliary buffers.
The exception to a) is the case when the encoder can reuse
user-supplied frames. And for these frames the auxiliary buffers
are unused, because this frame will never be used as current_picture
(and therefore also not as next_picture or last_picture);
see select_input_picture().
This means that we can simply avoid calling ff_alloc_picture()
with user-supplied frames at all.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
None of the mpegvideo encoders support anything but coded frames;
and if this were to change, it is unclear whether they would need
the adjustment here. So remove it.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Only entries 0..max_b_frames are ever used.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
In case "!direct" we are not reusing the input buffers
(due to e.g. insufficient alignment), but allocating
new ones. These of course do not alias with the ones
provided by the user, so these checks are always-false.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
mpegvideo_enc uses a fixed-size array of Pictures; a slot is
considered taken if the Picture's AVFrame is set.
When an error happens after a slot has been taken, this Picture
has typically not been reset and is therefore not usable for
future requests. The code aborts when one runs out of slots
and this can happen in case of allocation failures.
Fix this by always unreferencing a Picture in case of errors.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
There is no need to parse the header twice; doing so does nothing.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
And stop setting picture_number which was only done to not parse
extradata multiple times.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Fixes: signed integer overflow: 0 - -9223372036854775808 cannot be represented in type 'long'
Fixes: 51896/clusterfuzz-testcase-minimized-ffmpeg_IO_DEMUXER_fuzzer-6112289464123392
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Avoids allocations and error checks as well as the boilerplate
code for creating an AVBuffer with a custom free callback.
Also increases type safety.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Tested-by: Timo Rothenpieler <timo@rothenpieler.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Avoids allocations and error checks and allows to remove
cleanup code for earlier allocations. Also avoids casts
and indirections.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Given that the RefStruct API relies on the user to know
the size of the objects and does not provide a way to get it,
we need to store the number of elements allocated ourselves;
but this is actually better than deriving it from the size
in bytes.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Paul B Mahol
Zhao Zhili
Leo Izen
Michael Niedermayer
Andreas Rheinhardt
Timo Rothenpieler
John Mather
Rémi Denis-Courmont