Fixes out of array read
Fixes: d41d8cd98f00b204e9800998ecf8427e/signal_sigsegv_321165b_7641_077dfcd8cbc80b1c0b470c8554cd6ffb.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 2f95ddd996db8a6281d2e18c184595a7/asan_heap-oob_192fe91_3330_58e4441181e30a66c19f743dcb392347.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array access
Fixes: 08664a2a7921ef48172f26495c7455be/asan_heap-oob_23036c6_3301_523388ef84285a0270caf67a43247b59.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 007c4a36608ebdf27ee260ad60a81184/asan_heap-oob_32076b4_2243_116b1cb29d91cc4974d6680e3d10bd91.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Just realized my previous patch doesn't work quite right. I uploaded a better
sample file that actually has visible captions to /incoming/hevc_cc.ts. I
tested with that file doing hevc->x264 and it works.
This is basically an exact copy of the existing h264 logic.
Signed-off-by: Will Kelleher <wkelleher@gogoair.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
These casts are unnecessary, and may safely be removed.
Found by enabling -Wpedantic on clang 3.7.
Tested with FATE.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
There is not much reason to generate such a small table at runtime.
Signed-off-by: Derek Buitenhuis <derekb@vimeo.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
The intended meaning is "if this block is the first block in a slice then
its left boundary is a slice boundary". Silence a logical-not-parentheses
warning from gcc.
At least the new videotoolbox decoder does not actually set a frame if
end_frame fails. This causes the API to return success and signals that
a picture was decoded, even though AVFrame->data[0] is NULL.
Fix this by propagating end_frame errors.
Also, make hls_nal_unit() work only on the provided NAL unit, without
requiring a whole decoding context.
This will allow splitting this code for reuse by the parser.
It is used as get_bits argument and reading 0 bits doesn't make sense.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
It is used as get_bits argument and reading 0 bits isn't supported.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This change introduces basic support for HEVC decoding through vdpau.
Right now, there are problems with the nvidia driver/library implementation
that mean that frames are incorrectly laid out in memory when they are
returned from the decoder, and it is normally impossible to recover the
complete decoded frame due to loss of data from alignment inconsistencies.
I obviously hope that nvidia will be fixing it in due course - I've verified
the problems exist with their example application.
As such, this support is not useful for any real world application, but I
believe that it is correct (with the caveat that the mangled frames may hide
problems) and will work properly once the nvidia problem is fixed.
Right now it appears that any file encoded by x265 or nvenc is decoded
correctly, but that's because these files don't use a bunch of HEVC
features.
Quick summary:
Features that seem to work:
1) Short Term References
2) Scaling Lists
3) Tiling
Features with known problems:
1) Long Term References
It's hard to tell what's going on here. After I read the nvidia example
app that does not set the IsLongTerm flag on LTRs, and changed my code,
a bunch of frames using LTR started to display correctly, but there
are still samples with glitches that are related to LTRs.
In terms of real world files, both x265 and nvenc only use short term
refs from this list. The divx encoder seems similar.
Signed-off-by: Philip Langdale <philipl@overt.org>
Today, we track the short term RPS size for DXVA, but only if the
SliceHeader RPS is being used. Otherwise it's left uninitialized.
NVIDIA's VDPAU implementation requires that the size be accurately
tracked even if an SPS RPS is being used. In this case, it's really
counting the size of the RPS idx information, but you end up with
mangled output if the value is not accurate.
VDPAU also needs the size of the long term RPS.
Signed-off-by: Philip Langdale <philipl@overt.org>
Personally, I need the decoder to back out if get_format() returns no
usable pixel format. This didn't work because the error code was not
propagated down the call chain. This in turn happened because the
variable declaration removed in this patch shadowed the variable, whose
value is returned at the end of the function. Consequently, failures of
decode_nal_unit() were ignored in this place.
Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Andreas Cadhalpun
Anton Khirnov
Michael Niedermayer
Clément Bœsch
Will Kelleher
Ganesh Ajjanagadde
Vittorio Giovara
Ronald S. Bultje
Hendrik Leppkes
Timo Rothenpieler
wm4
Philip Langdale
Michael Niedermayer
Steve Lhomme