Since resolution change support this also was exploitable, which is
how it was found.
Fixes read after free and out of array reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Move some functions from dsputil. The idea is that videodsp contains
functions that are useful for a large and varied set of video decoders.
Currently, it contains emulated_edge_mc() and prefetch().
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
The VC1 decoder uses edge_emu_buffer simultaneously for luma and chroma
and needs more space. That was not a problem before f1d8763a02
since the size for edge_emu_buffer was always calculated with 2 byte per
pixel since the linesize was not known.
Fixes occasionally fate errors in vc1_sa10143.
Since we can't know which stride a custom get_buffer() implementation is
going to use we have to allocate this scratch buffers after the linesize
is known. It was pretty safe for 8 bit per pixel pixel formats since we
always allocated memory for up to 16 bits per pixel. It broke hoever
with cmdutis.c's alloc_buffer() and high pixel bit depth since it
allocated larger edges than mpegvideo expected.
Fixes fuzzed sample nasa-8s2.ts_s244342.
This requires to move the avcodec_default_free_buffers() call to
ff_MPV_common_end() since otherwise delayed pictures would get freed
during a size change.
This does not seem to have an effect currently. Fate-h264 passes with
THREADS=1..16 and both threading types as before. It fixes however a
segfault during error resilience with my adaptive-frame-mt patchset.
A picture in use during error resilience gets realloced in another
thread in the fuzzed sample sample_varPAR.avi_s226019.
AVFrame.extended_data has to reset to the AVFrame.data of the current
thread context after copying the frame contents.
Fixes crashes with frame-threading after 2bc0de3858.
Adds a flag context_reinit to MpegEncContext to relieable keep track
of frame parameter changes which require a context reinitialization.
This is required for broken inputs which change the frame size but
error out before the context can be reinitialized.
This is mainly required for frame parameter changes during frame based
multithreading but single threaded usage profits too from avoiding
ff_MPV_common_end()/ff_MPV_common_init() cycles.
Prevents dangling pointers and makes access after free more obvious.
Setting AVFrame.qscale_table to NULL is required for successfully
allocating a previously freed Picture with ff_alloc_picture().
this crashes otherwise, and can happen from try_decode_frame() in the case of decoding errors
Fixes Ticket1602
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This file defines a single, huge function, MPV_motion(), which
although being declared inline is not actually inlined by the
compiler (for good reason). There is thus no sense in defining
this function in a header file, resulting in multiple copies of
it in the final library.
Signed-off-by: Mans Rullgard <mans@mansr.com>
Defining restrict results - for some compilers - in changing other
uses of the restrict keyword also, e.g. __declspec(restrict) gets
changed to __declspec(__restrict) on MSVC. This causes compilation
failures. Therefore, using a private namespace macro instead is
more reliable and robust.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Michael Niedermayer
Rémi Denis-Courmont
Janne Grunau
Ronald S. Bultje
Diego Biurrun
Luca Barbato
Mans Rullgard
Anton Khirnov
Alberto Delmás
Lou Logan