MB_TYPE_L[01] is based upon H.264 terminology (it stands for
list); yet the mpegvideo based decoders don't have lists
of reference frames, they have at most one forward and one
backward reference. So use terminology based upon this.
This also has a second advantage: MB_TYPE_L[01] is actually
an OR of two flags (which are set independently for H.264,
but aren't for mpegvideo). Switching to different flags
makes the flags fit into an int16_t, which will be useful
in future commits.
The only downside to this is a very small amount of code
in error_resilience.c and mpegutils.c (the only code shared
between the H.264 decoder and mpegvideo).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The days in which an MPVPicture* is set with the corresponding frame
being blank are over; this allows to simplify some checks.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Given that MPVPictures are already directly shared between threads
in case of frame-threaded decoding, one can simply use it to
pass decoding progress information between threads. This allows
to avoid one level of indirection; it also means avoids allocations
(of the ThreadFrameProgress structure) in case of frame-threading
and indeed makes ff_thread_release_ext_buffer() decoder-only
(actually, H.264-decoder-only).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Up until now, an initialized MpegEncContext had an array of
MPVPictures (way more than were ever needed) and the MPVPicture*
contained in the MPVWorkPictures as well as the input_picture
and reordered_input_picture arrays (for the encoder) pointed
into this array. Several of the pointers could point to the
same slot and because there was no reference counting involved,
one had to check for aliasing before unreferencing.
Furthermore, given that these pointers were not ownership pointers
the pointers were often simply reset without unreferencing
the slot (happened e.g. for the RV30 and RV40 decoders) or
there were moved without resetting the src pointer (happened
for the encoders where the entries in the input_picture
and reordered_input_picture arrays were not reset).
Instead actually releasing these pictures was performed by looping
over the whole array and checking which one of the entries needed
to be kept. Given that the array had way too many slots (36),
this meant that more than 30 MPVPictures have been unnecessarily
unreferenced in every ff_mpv_frame_start(); something similar
happened for the encoder.
This commit changes this by making the MPVPictures refcounted
via the RefStruct API. The MPVPictures itself are part of a pool
so that this does not entail constant allocations; instead,
the amount of allocations actually goes down, because the
earlier code used such a large array of MPVPictures (36 entries) and
allocated an AVFrame for every one of these on every
ff_mpv_common_init(). In fact, the pool is only freed when closing
the codec, so that reinitializations don't lead to new allocations
(this avoids having to sync the pool in update_thread_context).
Making MPVPictures refcounted also has another key benefit:
It makes it possible to directly share them across threads
(when using frame-threaded decoding), eliminating ugly code
with underlying av_frame_ref()'s; sharing these pictures
can't fail any more.
The pool is allocated in ff_mpv_decode_init() for decoders,
which therefore can fail now. This and the fact that the pool
is not unreferenced in ff_mpv_common_end() also necessitated
to mark several mpegvideo-decoders with the FF_CODEC_CAP_INIT_CLEANUP
flag.
*: This also means that there is no good reason any more for
ff_mpv_common_frame_size_change() to exist.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Currently identical to the H.261 and H.263 close functions
(which it replaces). It will be extended in future commits.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
There are two types of MPVPictures: Three (cur_pic, last_pic, next_pic)
that are directly part of MpegEncContext and an array of MPVPictures
that are separately allocated and are mostly accessed via pointers
(cur|last|next)_pic_ptr; they are also used to store AVFrames in the
encoder (necessary due to B-frames). As the name implies, each of the
former is directly associated with one of the _ptr pointers:
They actually share the same underlying buffers, but the ones
that are part of the context can have their data pointers offset
and their linesize doubled for field pictures.
Up until now, each of these had their own references; in particular,
there was an underlying av_frame_ref() to sync cur_pic and cur_pic_ptr
etc. This is wasteful.
This commit changes this relationship: cur_pic, last_pic and next_pic
now become MPVWorkPictures; this structure does not have an AVFrame
at all any more, but only the cached values of data and linesize.
It also contains a pointer to the corresponding MPVPicture, establishing
a more natural relationsship between the two.
This already means that creating the context-pictures from the pointers
can no longer fail.
What has not been changed is the fact that the MPVPicture* pointers
are not ownership pointers and that the MPVPictures are part of an
array of MPVPictures that is owned by a single AVCodecContext.
Doing so will be done in a latter commit.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Specifically, add const to the pointed-to-type of pointers
that point to something static or that belong to last_pic
or next_pic (because modifying these might lead to data races).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Use the cached values from MpegEncContext.(cur|last|next)_pic
instead of the corresponding *_pic_ptr.
Also do the same in wmv2dec.c and mpegvideo_enc.c.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
current_picture->cur_pic, last_picture->last_pic, similarly
for new_picture and next_picture.
Also rename the corresponding *_ptr fields.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
There are lots of files that don't need it: The number of object
files that actually need it went down from 2011 to 884 here.
Keep it for external users in order to not cause breakages.
Also improve the other headers a bit while just at it.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Said object is only allowed to be modified during its
initialization and is immutable afterwards.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
For most VLCs here, the number of bits of the VLC is
write-only, because it is hardcoded at the call site.
Therefore one can replace these VLC structures with
the only thing that is actually used: The pointer
to the VLCElem table.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
It has currently not been done for H263, H263P and MPEG4.
Doing so avoids having to initialize the IDCT permutation
lateron when decoding packets in order to be able to parse
a quant matrix; it means that every mpegvideo decoder always
has an initialized IDCTDSPContext after init.
Initializing is done generically in ff_mpv_decode_init().
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Add a pointer parameter that if supplied will be used to return
the updated decode_error_flags. This will allow to fix several
races when using frame-threading; these resulted from AVFrame
that the earlier code updated concurrently being used as source
in an av_frame_ref() call in the decoder's update_thread_context.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Therefore use a proper prefix for this API, e.g.
ff_init_vlc_sparse -> ff_vlc_init_sparse
ff_free_vlc -> ff_vlc_free
INIT_VLC_LE -> VLC_INIT_LE
INIT_VLC_USE_NEW_STATIC -> VLC_INIT_USE_STATIC
(The ancient INIT_VLC_USE_STATIC has been removed
in 595324e143, so that
the NEW has been dropped.)
Finally, reorder the flags and change their values
accordingly.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Add the check for the return value of the av_malloc in order to avoid
NULL pointer deference.
Fixes: b86ab38137 ("Add weighted motion compensation for RV40 B-frames")
Signed-off-by: Jiasheng Jiang <jiasheng@iscas.ac.cn>
Decoder-only, as the dimensions are set by the user when encoding.
Also fixup the other headers a bit while removing unnecessary internal.h
inclusions.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This is possible for most of the callers, because e.g. only
the MPEG-4 decoder can have bits_per_raw_sample > 8.
Also most mpegvideo-based codecs are 420 only.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
In C, qualifiers for arrays are broken:
const VLC_TYPE (*foo)[2] is a pointer to an array of two const VLC_TYPE
elements and unfortunately this is not compatible with a pointer
to a const array of two VLC_TYPE, because the latter does not exist
as array types are never qualified (the qualifier applies to the base
type instead). This is the reason why get_vlc2() doesn't accept
a const VLC table despite not modifying the table at all, as
there is no automatic conversion from VLC_TYPE (*)[2] to
const VLC_TYPE (*)[2].
Fix this by using a structure VLCElem for the VLC table.
This also has the advantage of making it clear which
element is which.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This increases type-safety by avoiding conversions from/through void*.
It also avoids the boilerplate "AVFrame *frame = data;" line
for non-subtitle decoders.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The majority of frame-threaded decoders (mainly the intra-only)
need exactly one part of ThreadFrame: The AVFrame. They don't
need the owners nor the progress, yet they had to use it because
ff_thread_(get|release)_buffer() requires it.
This commit changes this and makes these functions work with ordinary
AVFrames; the decoders that need the extra fields for progress
use ff_thread_(get|release)_ext_buffer() which work exactly
as ff_thread_(get|release)_buffer() used to do.
This also avoids some unnecessary allocations of progress AVBuffers,
namely for H.264 and HEVC film grain frames: These frames are not
used for synchronization and therefore don't need a ThreadFrame.
Also move the ThreadFrame structure as well as ff_thread_ref_frame()
to threadframe.h, the header for frame-threaded decoders with
inter-frame dependencies.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This is in preparation for further commits that will stop
using ThreadFrame for frame-threaded codecs that don't use
ff_thread_(await|report)_progress(); the API for those codecs
having inter-frame depdendencies will live in threadframe.h.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The RealVideo 3.0 and 4.0 decoders call ff_mpv_common_init() only during
their init function and not during decode_frame(); when the size of the
frame changes, they call ff_mpv_common_frame_size_change(). Yet upon
error, said function calls ff_mpv_common_end() which frees the whole
MpegEncContext and not only those parts that
ff_mpv_common_frame_size_change() reinits. As a result, the context will
never be usable again; worse, because decode_frame() contains no check
for whether the context is initialized or not, it is presumed that it is
initialized, leading to segfaults. Basically the same happens if
rv34_decoder_realloc() fails.
This commit fixes this by only resetting the parts that
ff_mpv_common_frame_size_change() changes upon error and by actually
checking whether the context is in need of reinitialization in
ff_rv34_decode_frame().
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Do it only when requested with the AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS
flag.
Drop previous code using the long-deprecated AV_FRAME_DATA_QP_TABLE*
API. Temporarily disable fate-filter-pp, fate-filter-pp7,
fate-filter-spp. They will be reenabled once these filters are converted
in following commits.
These two are always called directly after each other (with the
exception of the calls in mpeg_decode_init() where some irrelevant
modifications of the avctx (which could just as well be done before
ff_mpv_decode_defaults(), because it doesn't have a pointer to the
AVCodecContext at all and therefore can't see these modifications at
all) are performed in between), so merge ff_mpv_decode_defaults() in
ff_mpv_decode_init().
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
For both RealVideo 3.0 as well as RealVideo 4.0 the VLC table to use
depends upon the slice's quantization parameter; these are coded on five
bits in the bitstream and are therefore in the range of 0..31; yet the
last element here is not valid and therefore the quantizer is clipped to
the range 0..30 to get the index. But this is unnecessary: One can just
add one element more to the relevant array to avoid the clipping.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Most of the VLCs used by RealVideo 3 and 4 obey three simple rules:
Shorter codes are on the left of the tree, for each length, the symbols
are ascending from left to right and the symbols either form a
permutation of 1..size or 0..(size - 1). For the latter case, one just
needs to store the length of each symbol and create the codes according
to the other rules; no explicit code or symbol array must be stored.
The former case is also treated in much the same way by artificially
assigning a length of zero to the symbol 0; when a length of zero was
encountered, the element was ignored except that the symbol counter was
still incremented. If the length was nonzero, the symbol would be
assigned via the symbol counter and the length copied over into a new
array.
Yet this is unnecessary, as ff_init_vlc_sparse() follows exactly the
same pattern: If a length of zero is encountered, the element is ignored
and only the symbol counter incremented. So one can directly forward the
length array and also need not create a symbol table oneself, because
ff_init_vlc_sparse() will infer the same symbol table in this case.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The current design, where
- proper init is called for the first per-thread context
- first thread's private data is copied into private data for all the
other threads
- a "fixup" function is called for all the other threads to e.g.
allocate dynamically allocated data
is very fragile and hard to follow, so it is abandoned. Instead, the
same init function is used to init each per-thread context. Where
necessary, AVCodecInternal.is_copy can be used to differentiate between
the first thread and the other ones (e.g. for decoding the extradata
just once).
This also fixes several integer overflows by checking each value before
use.
Fixes: 662/clusterfuzz-testcase-4898131432964096
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Failing earlier causes the context to be insufficiently initialized which
can break decoding future frames with threads
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Andreas Rheinhardt
Michael Niedermayer
Jiasheng Jiang
Anton Khirnov
Andreas Rheinhardt
Diego Biurrun
Vittorio Giovara