Given that the AVCodec.next pointer has now been removed, most of the
AVCodecs are not modified at all any more and can therefore be made
const (as this patch does); the only exceptions are the very few codecs
for external libraries that have a init_static_data callback.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
In case of resolution changes rv20_decode_picture_header() closes and
reopens its MpegEncContext; it checks the latter for errors, yet when
an error happens, it might happen that no new attempt at
reinitialization is performed when decoding the next frame; this leads
to crashes lateron.
This commit fixes this by making sure that initialization will always
be attempted if the context is currently not initialized.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This mostly reverts commit 4b2863ff01.
Said commit removed the freeing code from ff_mpv_common_init(),
ff_mpv_common_frame_size_change() and ff_mpeg_framesize_alloc() and
instead added the FF_CODEC_CAP_INIT_CLEANUP to several codecs that use
ff_mpv_common_init(). This introduced several bugs:
a) Several decoders using ff_mpv_common_init() in their init function were
forgotten: This affected FLV, Intel H.263, RealVideo 3.0 and V4.0 as well as
VC-1/WMV3.
b) ff_mpv_common_init() is not only called from the init function of
codecs, it is also called from AVCodec.decode functions. If an error
happens after an allocation has succeeded, it can lead to memleaks;
furthermore, it is now possible for the MpegEncContext to be marked as
initialized even when ff_mpv_common_init() returns an error and this can
lead to segfaults because decoders that call ff_mpv_common_init() when
decoding a frame can mistakenly think that the MpegEncContext has been
properly initialized. This can e.g. happen with H.261 or MPEG-4.
c) Removing code for freeing from ff_mpeg_framesize_alloc() (which can't
be called from any init function) can lead to segfaults because the
check for whether it needs to allocate consists of checking whether the
first of the buffers allocated there has been allocated. This part has
already been fixed in 76cea1d2ce.
d) ff_mpv_common_frame_size_change() can also not be reached from any
AVCodec.init function; yet the changes can e.g. lead to segfaults with
decoders using ff_h263_decode_frame() upon allocation failure, because
the MpegEncContext will upon return be flagged as both initialized and
not in need of reinitialization (granted, the fact that
ff_h263_decode_frame() clears context_reinit before the context has been
reinited is a bug in itself). With the earlier version, the context
would be cleaned upon failure and it would be attempted to initialize
the context again in the next call to ff_h263_decode_frame().
While a) could be fixed by adding the missing FF_CODEC_CAP_INIT_CLEANUP,
keeping the current approach would entail adding cleanup code to several
other places because of b). Therefore ff_mpv_common_init() is again made
to clean up after itself; the changes to the wmv2 decoder and the SVQ1
encoder have not been reverted: The former fixed a memleak, the latter
allowed to remove cleanup code.
Fixes: double free
Fixes: ff_free_picture_tables.mp4
Fixes: ff_mpeg_update_thread_context.mp4
Fixes: decode_colskip.mp4
Fixes: memset.mp4
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
These two are always called directly after each other (with the
exception of the calls in mpeg_decode_init() where some irrelevant
modifications of the avctx (which could just as well be done before
ff_mpv_decode_defaults(), because it doesn't have a pointer to the
AVCodecContext at all and therefore can't see these modifications at
all) are performed in between), so merge ff_mpv_decode_defaults() in
ff_mpv_decode_init().
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The VLC tables to be used for parsing RealVideo 1.0 DC coefficients are
weird: The luma table contains a block of 2^11 codes beginning with the
same prefix and length that all have the same symbol (i.e. value only
depends upon the prefix); the same goes for the chroma block (except
it's only 2^9 codes). Up until now, these entries (which generally could
be parsed like ordinary entries with subtables) have been treated
specially: They have been treated like open ends of the tree, so that
get_vlc2() returned a value < 0 upon encountering them; afterwards it
was checked whether the right prefix was used and if so, the appropriate
number of bytes was skipped.
But there is actually an easy albeit slightly hacky way to support them
directly without pointless subtables: Just modify the VLC table so that
all the entries sharing the right prefix have a length that equals the
length of the whole entry.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
These tables were huge (14 bits) because one needed 14 bits in order to
find out whether a code is valid and in the VLC table or a valid code that
required hacky workarounds due to RealVideo 1.0 using multiple codes
for the same symbol and the code predating the introduction of symbols
tables for VLCs.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The RealVideo 1.0 decoder uses VLCs to parse DC coefficients. But the
values returned from get_vlc2() are not directly used; instead
-(val - 128) (which is in the range -127..128) is. This transformation
is unnecessary as it can effectively be done when initializing the VLC
by modifying the symbols table used. There is just one minor
complication: The chroma table is incomplete and in order to distinguish
an error from get_vlc2() (due to an invalid code) the ordinary return
range is modified to 0..255. This is possible because the only caller of
this function is (on success) only interested in the return value modulo
256.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
RealVideo 1.0 uses an insane way to encode DC coefficients: There are
several symbols that (for no good reason whatsoever) have multiple
encodings, leading to longer codes than necessary.
More specifically, the tree for the 256 luma symbols contains 255 codes
belonging to 255 different symbols on the left; going further right,
the tree consists of two blocks of 128 codes each of length 14 encoding
consecutive numbers (including two encodings for the symbol missing among
the 255 codes on the left); this is followed by two blocks of codes of
length 16 each containing 256 elements with consecutive symbols (i.e.
each of the blocks allows to encode all symbols). The rest of the tree
consists of 2^11 codes that all encode the same symbol.
The tree for the 256 chroma symbols is similar, but is missing the
blocks of length 256 and there are only 2^9 consecutive codes that
encode the same symbol; furthermore, the chroma tree is incomplete:
The right-most node has no right child.
All of this caused problems when parsing these codes; the reason is that
the code for this predates commit b613bacca9
which added support for explicit symbol tables and thereby removed the
requirement that different codes have different symbols. In order to
address this, the trees used for parsing were incomplete: They contained
the 255 codes on the left and one code for the remaining symbol. Whenever
a code not in these trees was encountered, it was dealt with in
special cases (one for each of the blocks mentioned above).
This commit reduces the number of special cases: Using a symbols table
allows to treat the blocks of consecutive symbols like ordinary codes;
only the blocks encoding a single symbol are still treated specially
(in order not to waste memory on tables for them).
In order to not increment the size of the tables used to initialize the
VLCs both the symbols as well as the lengths are now run-length encoded.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This can be achieved by switching to ff_init_vlc_from_lengths() which
allows to replace two uint16_t tables for codes with uint8_t tables for
the symbols by permuting the tables so that the codes are ordered from
left to right in the tree in which case they can be easily computed from
the lengths at runtime.
And after doing so, it became apparent that the tables for the symbols
are actually the same for luma and chroma, so that one can even omit one
of them.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Fixes: signed integer overflow: 2040 * 1187872 cannot be represented in type 'int'
Fixes: 15368/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RV20_fuzzer-5681657136283648
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: shift exponent 64 is too large for 32-bit type 'int'
Fixes: 15253/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RV10_fuzzer-5671114300194816
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The most interesting parts are initialization in ff_MPV_common_init() and
uninitialization in ff_MPV_common_end().
ff_mpeg_unref_picture and ff_thread_release_buffer have additional NULL
checks for Picture.f, because these functions can be called on
uninitialized or partially initialized Pictures.
NULL pointer checks are added to ff_thread_release_buffer() stub function.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Several decoders disable those anyway and they are not measurably faster
on x86. They might be somewhat faster on other platforms due to missing
emu edge SIMD, but the gain is not large enough (and those decoders
relevant enough) to justify the added complexity.
Andreas Rheinhardt
Michael Niedermayer
Andreas Rheinhardt
Limin Wang
Vittorio Giovara
Michael Niedermayer
Gabriel Dume
Diego Biurrun
John Stebbins
Nidhi Makhijani
wm4
Keiji Costantini
Anton Khirnov
Martin Storsjö