This fixes returning pictures with corrupted data pointers.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes out of array writes after resolution changes
No FFmpeg releases are known to be affected by this
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Also move the declaration to internal.h, and add restrict qualifiers
to the declaration (as in the implementation).
Signed-off-by: Martin Storsjö <martin@martin.st>
Not all hwaccels implement all codecs, so using one single list for
multiple such codecs means some codecs will be represented in the list,
even though they don't actually handle that codec. Copying specific
lists in each codec fixes that.
Signed-off-by: Martin Storsjö <martin@martin.st>
Not all hwaccels implement all codecs, so using one single list for
multiple such codecs means some codecs will be represented in the list,
even though they don't actually handle that codec. Copying specific
lists in each codec fixes that.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Most of the changes are just trivial are just trivial replacements of
fields from MpegEncContext with equivalent fields in H264Context.
Everything in h264* other than h264.c are those trivial changes.
The nontrivial parts are:
1) extracting a simplified version of the frame management code from
mpegvideo.c. We don't need last/next_picture anymore, since h264 uses
its own more complex system already and those were set only to appease
the mpegvideo parts.
2) some tables that need to be allocated/freed in appropriate places.
3) hwaccels -- mostly trivial replacements.
for dxva, the draw_horiz_band() call is moved from
ff_dxva2_common_end_frame() to per-codec end_frame() callbacks,
because it's now different for h264 and MpegEncContext-based
decoders.
4) svq3 -- it does not use h264 complex reference system, so I just
added some very simplistic frame management instead and dropped the
use of ff_h264_frame_start(). Because of this I also had to move some
initialization code to svq3.
Additional fixes for chroma format and bit depth changes by
Janne Grunau <janne-libav@jannau.net>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Always evaluate to NULL when the source Picture is not located in the
MpegEncContext.picture array. That will only happen for
next/last_picture_ptr when updating the thread context during h264 frame
threaded decoding, where they will point to elements of ref_list. Since
ref_list is not copied during updating the context and is invalid until
it is constructed for the current slice, there is no point in doing
anything complicated with next/last_picture_ptr, as they will get
updated when the ref_list is filled.
REBASE_PICTURE (more specifically, this half of it) takes a Picture
pointer that points into one larger struct, finds the offset of
that Picture within the struct and finds the corresponding field
within another instance of a similar struct.
The pointer difference "pic - (Picture*)old_ctx" is a value given
in sizeof(Picture) units, and when applied back on
(Picture*)new_ctx gets multiplied back with sizeof(Picture). Many
compilers seem to optimize out this division/multiplication, but
not all do.
GCC 4.2 on OS X doesn't seem to remove the division/multiplication,
therefore the new pointer didn't turn out to point to exactly
the right place in the new struct since it only had sizeof(Picture)
granularity (and the Picture is not aligned on a sizeof(Picture)
boundary within the encompassing struct). This bug has been present
before 47318953d as well - with H264, pointers to h->ref_list[0][0]
pointed to 88 bytes before h->ref_list[0][0] after the rebase. After
shrinking Picture, the difference ended up even larger, making
writes via such a Picture pointer overwrite other fields at random
in H264Context, ending up in crashes later.
This fixes H264 multithreaded decoding on OS X with GCC 4.2.
Signed-off-by: Martin Storsjö <martin@martin.st>
Since resolution change support this also was exploitable, which is
how it was found.
Fixes read after free and out of array reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Move some functions from dsputil. The idea is that videodsp contains
functions that are useful for a large and varied set of video decoders.
Currently, it contains emulated_edge_mc() and prefetch().
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Since we can't know which stride a custom get_buffer() implementation is
going to use we have to allocate this scratch buffers after the linesize
is known. It was pretty safe for 8 bit per pixel pixel formats since we
always allocated memory for up to 16 bits per pixel. It broke hoever
with cmdutis.c's alloc_buffer() and high pixel bit depth since it
allocated larger edges than mpegvideo expected.
Fixes fuzzed sample nasa-8s2.ts_s244342.
This requires to move the avcodec_default_free_buffers() call to
ff_MPV_common_end() since otherwise delayed pictures would get freed
during a size change.
In ISO/IEC 13818-2, bit rate is differently determined according to video type
1) MPEG1 Video
Bit_rate and vbv_delay are set to 3FFFF and FFFF respectively
to indicate variable bitrate. Other values are for constant bitrate.
VBV is only defined for constant bit rate operation.
Ths STD supersedes the VBV model for vbr.
2) MPEG2 Video
Even if the bitrate is constant, the value of bit_rate may not be the actual bitrate
since bit_rate need only be an upper bound to that actual bitrate.
VBV is only defined for variable bit rate operation.
Constant bit rate is viewed as a special case of vbr.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Adds a flag context_reinit to MpegEncContext to relieable keep track
of frame parameter changes which require a context reinitialization.
This is required for broken inputs which change the frame size but
error out before the context can be reinitialized.
This is mainly required for frame parameter changes during frame based
multithreading but single threaded usage profits too from avoiding
ff_MPV_common_end()/ff_MPV_common_init() cycles.
Michael Niedermayer
Diego Biurrun
Ronald S. Bultje
Martin Storsjö
Anton Khirnov
Janne Grunau
Paul B Mahol
Heesuk Jung