mpeg_quant may only be set for MPEG-4 and MPEG-2, yet for the latter
it is no option as the code acts as if it were always set.
So deprecate the option for all codecs for which it makes no sense.
Furthermore, given that the code already errors out if the option is set
for a codec that doesn't support it we can restrict the range of
the option for all these codecs without breaking something. This means
that the checks for whether mpeg_quant is set for these codecs can be
removed as soon as AVCodecContext.mpeg_quant is removed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This has the advantage that one does not waste some allocations
if one errors out because of these checks.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Currently said list contains only the pixel formats that are always
supported irrespective of the range and the value of
strict_std_compliance. This makes the MJPEG encoder an outlier as all
other codecs put all potentially supported pixel formats into said list
and error out if the chosen pixel format is unsupported. This commit
brings it therefore in line with the other encoders.
The behaviour of fftools/ffmpeg_filter.c has been preserved. A more
informed decision would be possible if colour range were available
at this point, but it isn't.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The documentation for AV_PIX_FMT_YUVJ420P reads:
"planar YUV 4:2:0, 12bpp, full scale (JPEG), deprecated in favor of
AV_PIX_FMT_YUV420P and setting color_range"
Yet the LJPEG encoder only accepts full scale yuv420p when strictness is
set to unofficial or lower; with default strictness it emits a nonsense
error message that says that limit range YUV is unofficial. This has
been changed to allow full range yuv420p, yuv422p and yuv444p irrespective
of the level of strictness.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
All encoders using ff_mpv_encode_init() already have pix_fmts set
so that the pixel format is already checked in ff_encode_preinit().
The one exception to this is MJPEG whose check remains.
(Btw: The AVCodec.pix_fmts check for AMV is stricter than the check
in ff_mpv_encode_init().)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Using optimal Huffman tables is not supported for AMV and always
disabled by ff_mpv_encode_init(); therefore one can build
the AMV encoder without mjpegenc_huffman if one adds the necessary
compile-time checks.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Up until now the relevant checks all checked for the existence of the
MJPEG encoder only.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The ProRes encoder allocates huge worst-case buffers just to be safe;
and for huge resolutions (8k in this case) these can be so big that the
number of bits does no longer fit into a (signed 32-bit) int; this means
that one must no longer use the parts of the PutBits API that deal with
bit counters. Yet proresenc_kostya did it, namely for a check about
whether we are already beyond the end. Yet this check is unnecessary
nowadays, because the PutBits API comes with automatic checks (with
a log message and a av_assert2() in put_bits() and an av_assert0() in
flush_put_bits()), so this is unnecessary. So simply remove the check.
Fixes ticket #9173.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Calling av_frame_make_writable() from decoders is tricky, especially
when frame threading is used. It is much simpler and safer to just make
a private copy of the frame.
This is not expected to have a major performance impact, since
APNG_DISPOSE_OP_BACKGROUND is not used often and
av_frame_make_writable() would typically make a copy anyway.
Found-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array read
Fixes: 32968/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MSP2_fuzzer-5315296027082752
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The (deprecated) field AVCodecContext.mpeg_quant has no range
restriction; MpegEncContext.mpeg_quant is restricted to 0..1.
If the former is set, the latter is overwritten with it without
checking the range. This can trigger an av_assert2() with the MPEG-4
encoder when writing said field.
Fix this by just setting MpegEncContext.mpeg_quant to 1 if
AVCodecContext.mpeg_quant is set.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The pix_fmts of the LJPEG encoder already contain all supported pixel
formats (including the ones only supported when strictness is unofficial
or less); yet the check in ff_encode_preinit() ignored this list in case
strictness is unofficial or less. But the encoder presumed that it is
always applied and blacklists some of the entries in pix_fmts when
strictness is > unofficial. The result is that if one uses an entry not
on that list and sets strictness to unofficial, said entry passes both
checks and this can lead to segfaults lateron (e.g. when using gray).
Fix this by removing the exception for LJPEG in ff_encode_preinit().
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
For both the RealMedia as well as the IVR demuxer (which share the same
context) each AVStream's priv_data contains an AVPacket that might
contain data (even when reading the header) and therefore needs to be
unreferenced. Up until now, this has not always been done:
The RealMedia demuxer didn't do it when allocating a new stream's
priv_data failed although there might be other streams with packets to
unreference. (The reason for this was that until recently rm_read_close()
couldn't handle an AVStream without priv_data, so one had to choose
between a potential crash and a memleak.)
The IVR demuxer meanwhile never ever called read_close so that the data
already contained in packets leaks upon error.
This patch fixes both demuxers by adding the appropriate cleanup code.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This makes sure that reading a truncated chunk will never overflow into
the following chunk. It also allows to remove many repeated lines
skipping over the trailing crc checksum.
This data cannot be stored in PNGDecContext.picture, because the
corresponding chunks may be read after the call to
ff_thread_finish_setup(), at which point modifying shared context data
is a race.
Store intermediate state in the context and then write it directly to
the output frame.
Fixes exporting frame metadata after 5663301560Fixes#8972
Found-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Do not store the image buffer pointer/linesize in the context, just
access them directly from the frame.
Stop assuming that linesize is the same for the current and last frame.
ff_vc1_decode_init_alloc_tables() had one error path that forgot to free
already allocated buffers; these would then be overwritten on the next
allocation attempt (or they would just not be freed in case this
happened during init, as the decoders for which it is used do not have
the FF_CODEC_CAP_INIT_CLEANUP set).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Before 998c9f15d1, initializing an
MpegEncContext's IDCT parts occured in ff_mpv_common_init() and this
has been called in h261_decode_frame(), not h261_decode_init().
Yet said commit factored this out of ff_mpv_common_init() and therefore
there is no reason any more not to set this during init as this commit
does.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The RealVideo 3.0 and 4.0 decoders call ff_mpv_common_init() only during
their init function and not during decode_frame(); when the size of the
frame changes, they call ff_mpv_common_frame_size_change(). Yet upon
error, said function calls ff_mpv_common_end() which frees the whole
MpegEncContext and not only those parts that
ff_mpv_common_frame_size_change() reinits. As a result, the context will
never be usable again; worse, because decode_frame() contains no check
for whether the context is initialized or not, it is presumed that it is
initialized, leading to segfaults. Basically the same happens if
rv34_decoder_realloc() fails.
This commit fixes this by only resetting the parts that
ff_mpv_common_frame_size_change() changes upon error and by actually
checking whether the context is in need of reinitialization in
ff_rv34_decode_frame().
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
In case of resolution changes rv20_decode_picture_header() closes and
reopens its MpegEncContext; it checks the latter for errors, yet when
an error happens, it might happen that no new attempt at
reinitialization is performed when decoding the next frame; this leads
to crashes lateron.
This commit fixes this by making sure that initialization will always
be attempted if the context is currently not initialized.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
When slice-threading is used, ff_mpv_common_init() duplicates
the first MpegEncContext and allocates some buffers for each
MpegEncContext (the first as well as the copies). But the count of
allocated MpegEncContexts is not updated until after everything has
been allocated and if an error happens after the first one has been
allocated, only the first one is freed; the others leak.
This commit fixes this: The count is now set before the copies are
allocated. Furthermore, the copies are now created and initialized
before the first MpegEncContext, so that the buffers exclusively owned
by each MpegEncContext are still NULL in the src MpegEncContext so
that no double-free happens upon allocation failure.
Given that this effectively touches every line of the init code,
it has also been factored out in a function of its own in order to
remove code duplication with the same code in
ff_mpv_common_frame_size_change() (which was never called when using
more than one slice (and if it were, there would be potential
double-frees)).
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This mostly reverts commit 4b2863ff01.
Said commit removed the freeing code from ff_mpv_common_init(),
ff_mpv_common_frame_size_change() and ff_mpeg_framesize_alloc() and
instead added the FF_CODEC_CAP_INIT_CLEANUP to several codecs that use
ff_mpv_common_init(). This introduced several bugs:
a) Several decoders using ff_mpv_common_init() in their init function were
forgotten: This affected FLV, Intel H.263, RealVideo 3.0 and V4.0 as well as
VC-1/WMV3.
b) ff_mpv_common_init() is not only called from the init function of
codecs, it is also called from AVCodec.decode functions. If an error
happens after an allocation has succeeded, it can lead to memleaks;
furthermore, it is now possible for the MpegEncContext to be marked as
initialized even when ff_mpv_common_init() returns an error and this can
lead to segfaults because decoders that call ff_mpv_common_init() when
decoding a frame can mistakenly think that the MpegEncContext has been
properly initialized. This can e.g. happen with H.261 or MPEG-4.
c) Removing code for freeing from ff_mpeg_framesize_alloc() (which can't
be called from any init function) can lead to segfaults because the
check for whether it needs to allocate consists of checking whether the
first of the buffers allocated there has been allocated. This part has
already been fixed in 76cea1d2ce.
d) ff_mpv_common_frame_size_change() can also not be reached from any
AVCodec.init function; yet the changes can e.g. lead to segfaults with
decoders using ff_h263_decode_frame() upon allocation failure, because
the MpegEncContext will upon return be flagged as both initialized and
not in need of reinitialization (granted, the fact that
ff_h263_decode_frame() clears context_reinit before the context has been
reinited is a bug in itself). With the earlier version, the context
would be cleaned upon failure and it would be attempted to initialize
the context again in the next call to ff_h263_decode_frame().
While a) could be fixed by adding the missing FF_CODEC_CAP_INIT_CLEANUP,
keeping the current approach would entail adding cleanup code to several
other places because of b). Therefore ff_mpv_common_init() is again made
to clean up after itself; the changes to the wmv2 decoder and the SVQ1
encoder have not been reverted: The former fixed a memleak, the latter
allowed to remove cleanup code.
Fixes: double free
Fixes: ff_free_picture_tables.mp4
Fixes: ff_mpeg_update_thread_context.mp4
Fixes: decode_colskip.mp4
Fixes: memset.mp4
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Andreas Rheinhardt
Michael Niedermayer
Anton Khirnov
Guo, Yejun
Andreas Rheinhardt
Marton Balint
James Almer