Both the MPEG-4 parser as well as the decoder initialized
several VLCs. There is a "static int done = 0;" in order to
guard against initializing these multiple times, but this does
not work when several threads try to initialize these VLCs
concurrently, which can happen when initializing several parsers
at the same time (they don't use the global lock that is used
for codecs without the FF_CODEC_CAP_INIT_THREADSAFE cap; actually,
they don't use any lock at all).
Since ff_mpeg4_decode_picture_header() now aborts early when called
from the parser, it no longer needs to have these VLCs initialized
at all. This commit therefore does exactly this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Namely, skip some elements that are only useful for a decoder
when calling ff_mpeg4_decode_picture_header() from the MPEG-4 parser.
In particular, this ensures that the VLCs need no longer be
initialized by the parser.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
It is supported only by a few decoders (h263, h263p, mpeg(1|2|)video
and mpeg4) and is entirely redundant with parsers. Furthermore, using
it leads to missing frames, as flushing the decoder at the end does not
work properly.
Co-authored-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Given that the AVCodec.next pointer has now been removed, most of the
AVCodecs are not modified at all any more and can therefore be made
const (as this patch does); the only exceptions are the very few codecs
for external libraries that have a init_static_data callback.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
Up until now the RLTable ff_mpeg4_rl_intra was initialized by both mpeg4
decoder and encoder (except the VLCs that are only used by the decoder).
This is an obstacle to making these codecs init-threadsafe, so move
initializing this to a single function that is guarded by a dedicated
AVOnce.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This mostly reverts commit 4b2863ff01.
Said commit removed the freeing code from ff_mpv_common_init(),
ff_mpv_common_frame_size_change() and ff_mpeg_framesize_alloc() and
instead added the FF_CODEC_CAP_INIT_CLEANUP to several codecs that use
ff_mpv_common_init(). This introduced several bugs:
a) Several decoders using ff_mpv_common_init() in their init function were
forgotten: This affected FLV, Intel H.263, RealVideo 3.0 and V4.0 as well as
VC-1/WMV3.
b) ff_mpv_common_init() is not only called from the init function of
codecs, it is also called from AVCodec.decode functions. If an error
happens after an allocation has succeeded, it can lead to memleaks;
furthermore, it is now possible for the MpegEncContext to be marked as
initialized even when ff_mpv_common_init() returns an error and this can
lead to segfaults because decoders that call ff_mpv_common_init() when
decoding a frame can mistakenly think that the MpegEncContext has been
properly initialized. This can e.g. happen with H.261 or MPEG-4.
c) Removing code for freeing from ff_mpeg_framesize_alloc() (which can't
be called from any init function) can lead to segfaults because the
check for whether it needs to allocate consists of checking whether the
first of the buffers allocated there has been allocated. This part has
already been fixed in 76cea1d2ce.
d) ff_mpv_common_frame_size_change() can also not be reached from any
AVCodec.init function; yet the changes can e.g. lead to segfaults with
decoders using ff_h263_decode_frame() upon allocation failure, because
the MpegEncContext will upon return be flagged as both initialized and
not in need of reinitialization (granted, the fact that
ff_h263_decode_frame() clears context_reinit before the context has been
reinited is a bug in itself). With the earlier version, the context
would be cleaned upon failure and it would be attempted to initialize
the context again in the next call to ff_h263_decode_frame().
While a) could be fixed by adding the missing FF_CODEC_CAP_INIT_CLEANUP,
keeping the current approach would entail adding cleanup code to several
other places because of b). Therefore ff_mpv_common_init() is again made
to clean up after itself; the changes to the wmv2 decoder and the SVQ1
encoder have not been reverted: The former fixed a memleak, the latter
allowed to remove cleanup code.
Fixes: double free
Fixes: ff_free_picture_tables.mp4
Fixes: ff_mpeg_update_thread_context.mp4
Fixes: decode_colskip.mp4
Fixes: memset.mp4
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This reverts commit 6ac0e78183.
The mpeg4video parser can reach code that presumes that a certain VLC
has been initialized; yet Libav did not ensure this and Libav bug #1012
[1] is about an ensuing crash.
Instead of fixing the root cause a simple check for whether said VLC
has already been initialized was added; said check is inherently racy.
The proper fix is of course to ensure that the VLC is initialized and
commit 7c76eaeca2 already ensured this,
so there was no need to merge 6ac0e78183
at all. This commit therefore reverts said commit.
[1]: https://bugzilla.libav.org/show_bug.cgi?id=1012
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Also fix the indentation of decode_studio_vol_header while at it;
it was wrong since 177133a0f4.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The RLTables whose VLC tables are only used for intra blocks only use
the very first VLC table; yet all 32 have been initialized. This commit
stops this by switching to INIT_FIRST_VLC_RL. This saves 201624B from
the .bss segment; in case the decoder is actually used, this translates
into less memory used.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Switching from ff_init_vlc_sparse() to ff_init_vlc_from_lengths()
allows to replace codes which are so long that they need to be stored
in an uint16_t by symbols which fit into an uint8_t; and even these can
be avoided in case of the sprite trajectory VLC.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The longest code here is 12 bits long and can be read in two attempts.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Fixes: signed integer overflow: -144876608 * 16 cannot be represented in type 'int'
Fixes: 22782/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-6039584977977344
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access
Fixes: 23327/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5134822992510976
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The current design, where
- proper init is called for the first per-thread context
- first thread's private data is copied into private data for all the
other threads
- a "fixup" function is called for all the other threads to e.g.
allocate dynamically allocated data
is very fragile and hard to follow, so it is abandoned. Instead, the
same init function is used to init each per-thread context. Where
necessary, AVCodecInternal.is_copy can be used to differentiate between
the first thread and the other ones (e.g. for decoding the extradata
just once).
Fixes: signed integer overflow: 24023040 * 112 cannot be represented in type 'int'
Fixes: 16570/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5173275211071488
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Kieran Kunhya <kierank@obe.tv>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: Out of array access
Fixes: 13500/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5769760178962432
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Kieran Kunhya <kierank@obe.tv>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Since db77230894 parsing of
mpeg4-extradata lead to a "Failed to parse extradata" warning, because
ff_mpeg4_decode_picture_header returns AVERROR_INVALIDDATA in case that
no VOP was found. This patch adds a parameter to signify whether a
header (where the absence of a VOP does not raise an error) or not is
parsed. The first mode is of course used for parsing headers.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: Out of array access
Fixes: 13090/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5408668986638336
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Kieran Kunhya <kierank@obe.tv>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
partitioned_frame is also set/cleared in decode_vop_header()
Fixes: out of array read
Fixes: 9789/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5638681627983872
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array read
Fixes: 10064/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5766801384800256
Fixes: 10225/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5071833448054784
Fixes: 10261/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5115048024866816
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array read
Fixes: asff-crash-0e53d0dc491dfdd507530b66562812fbd4c36678
Found-by: Paul Ch <paulcher@icloud.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The profile field is changed by code inside and outside the decoder,
its not a reliable indicator of the internal codec state.
Maintaining it consistency with studio_profile is messy.
Its easier to just avoid it and use only studio_profile
Fixes: assertion failure
Fixes: ffmpeg_crash_9.avi
Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: null pointer dereference
Fixes: ffmpeg_crash_7.avi
Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: assertion failure
Fixes: ffmpeg_crash_5.avi
Found-by: Thuan Pham <thuanpv@comp.nus.edu.sg>, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: shift exponent -1 is negative
Fixes: 7486/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-4977380939530240
Fixes: runtime error: index 36 out of bounds for type 'const uint8_t [32]'
Fixes: 7566/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-6536620682510336
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Andreas Rheinhardt
Anton Khirnov
Andreas Rheinhardt
James Almer
Paul B Mahol
Carl Eugen Hoyos
Michael Niedermayer
Limin Wang
Mark Thompson
Andreas Rheinhardt
Kieran Kunhya