Previously, s->context_initialized was left set to 1
if ff_vc1_decode_init_alloc_tables failed, skipping the
initialization completely on the next decode call.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
For codecs where decoding of a whole plane can simply
be skipped, we should offer applications to not decode
alpha for better performance (ca. 30% less CPU usage
and 40% reduced memory bandwidth).
It also means applications do not need to implement support
(even if it is rather simple) for YUVA formats in order to be
able to play these files.
Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Issues with the code:
1) The skip_bits_long breaks packed B-frames since we skip
of the packed frame, even for VDPAU.
2) Calling ff_h263_find_resync_marker_reverse is nonsense for MPEG-4,
and for H.263 the only code using this (vaapi_mpeg4) explicitly reverts
this change!
3) mb_x/mb_y are always 0 when vaapi_mpeg4_decode_slice, so doing
computations with them is just obfuscation
4) due to not updating mb_y the code would always go into the error
resilience case, causing nonsense error messages and maybe further
issues.
While tested to fix the data provided to the decoder in case of
VDPAU so it is the same as for the non-hwaccel code, the VA-API code
was not tested to still work, and adding regression testing even
as a quick hack is much more complicated for it.
Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Currently the code can in some cases draw tiles that hang outside the
allocated buffer. This patch increases the buffer size to avoid out
of array accesses. An alternative would be to fail if such tiles are
encountered.
I do not know if any valid files use such hanging tiles.
Fixes Ticket2971
Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Avoid overwriting the bitstream buffer data before we
have ended processing the frame.
This is necessary to fix hwaccels which might try to use
the buffer during the end_frame call.
I am not sure but it is possible this could even trigger
a use-after-free if the av_fast_malloc allocated a new buffer.
This would require that decode_slice did not wind the bitstream
forward all the way to the end, which does not currently happen in
normal streams.
Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
The specification does not allow multiple bits to be set,
but some encoders do it anyway.
Applying it only to the first seems to give better results.
Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
If the height is zero, the decompression will probably end up
failing due to not fitting into the allocated buffer later
anyway, so this doesn't need any more elaborate check.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
This is similar to the fix in 35cbc98b.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
The init function reads one byte of extradata.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
This is required, since invalid parameters actually could
pass the switch check below.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
A change in framesize caused a perpetual loss of synchronization.
So read (and use) the frame size from the frame header instead of
setting it only once.
Signed-off-by: Diego Biurrun <diego@biurrun.de>
It's not our responsibility to account for other projects'
terrible hacky usage.
Also, it is no longer needed by XMBC, since 78b71d4.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
This avoids crashes when initializing the range coder for
the first slice context.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
This is similar to an existing check for the second-last frame
from 062421e3.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
This avoids a potential division by zero.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
Also return a proper error code.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
Also pass on any returned error code.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
Previously the wrong buffer pointer was checked, when buf
instead of *buf was checked. But checking the return value
instead is even better.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
Michael Niedermayer
Martin Storsjö
Kieran Kunhya
Paul B Mahol
Reimar Döffinger
Hendrik Leppkes
Carl Eugen Hoyos
JULIAN GARDNER
Vittorio Giovara
Alex Converse
Justin Ruggles
John Stebbins
Derek Buitenhuis