Michael Niedermayer
04da0d2586
avcodec/hevc_ps: Move log2_ctb_size checks up to prevent undefined shift operations
...
Fixes CID1292293
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
65e5032955
avcodec/hevc_ps: Explicitly check num_tile_* for negative values
...
This fixes nothing but maybe helps coverity which does not see that this is failing later
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
b195aa5d52
avcodec/hevc_ps: Check vps_num_hrd_parameters
...
Fix CID1239052 part2
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
16c95b1073
avcodec/hevc_ps: More completely check vps_num_layer_sets
...
Fixes CID1239052 part1
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
0e7444f6e4
avcodec/hevc: Remove current_sps
...
The variable should not be needed anymore
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
James Almer
ba625dd8a1
avcodec: use av_mod_uintp2() where useful
...
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: James Almer <jamrial@gmail.com>
10 years ago
Anton Khirnov
df528b11ac
hevc: make sure no dangling pointers remain around on VPS/SPS change
10 years ago
Anton Khirnov
ce0bc09ee2
hevc: do not change the VPS if its contents are the same
...
This is the same as is done for SPS.
10 years ago
Michael Niedermayer
06c70d4537
avcodec/hevc_ps: Check cropping parameters more correctly
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Anton Khirnov
5127c00b97
hevc: zero the correct variables on invalid crop parameters
...
It's the output_window that is applied to the output frame, not
pic_conf_win
10 years ago
Michael Niedermayer
701c965a76
avcodec/hevc_ps: Check that log2_ctb_size is not smaller than the bounds of all profiles
...
Fixes: unaligned memory access
Fixes: signal_sigsegv_3344165_576_cov_3406448105_DBLK_A_MAIN10_VIXS_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Suggested-by: Christophe Gisquet <christophe.gisquet@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
205b2ba3d6
avcodec/hevc_ps: Sanity checks for some log2_* values
...
log2 values which imply numeric overflow are not supported
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Mickaël Raulet
7cf6a67ef9
avcodec/hevc: adding support for monochrome sequences in hevc
...
cherry picked from commit 8e50557707d2ec11ccad657470b2e140f314348e
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
3281fa8925
avcodec/hevc_ps: Check diff_cu_qp_delta_depth
...
Fixes undefined behavior
Fixes: asan_static-oob_17aa046_582_cov_1577759978_DBLK_G_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
1adc1b97e1
avcodec/hevc_ps: Add context to av_log() call
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
ea38e5a6b7
avcodec/hevc_ps: Check num_long_term_ref_pics_sps
...
Fixes out of array access
Fixes: signal_sigsegv_35bd0f0_1182_cov_791726764_STRUCT_B_Samsung_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
57e5812198
avcodec/hevc_ps: More complete window reset
...
Fixes out of array read
Fixes: signal_sigsegv_35bcf26_471_cov_2806540268_CAINIT_A_SHARP_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
9f9440bd81
avcodec/hevc_ps: Check return code from pps_range_extensions()
...
Fixes out of array read
Fixes: asan_heap-oob_177e222_885_cov_1532528832_MERGE_D_TI_3.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
852aaead1f
avcodec/hevc_ps: Check default display window bitstream and skip if invalid
...
Fixes Ticket4035
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
cbb277988a
avcodec/hevc_ps: Always initialize backup in decode_vui()
...
Fixes CID1237283
Reviewed-by: Christophe Gisquet <christophe.gisquet@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
aaaf7261b7
avcodec/hevc_ps: fix 1 vs. 0 typo
...
Found-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Christophe Gisquet
0625a38066
hevc_ps: check overflow and test alternate syntax
...
Some streams were found to have what appeared to be truncated SPS.
Their syntax seem to be valid at least until the end of the VUI, so
try that syntax if the parsing would overflow the SPS in the
conforming syntax.
Fixes ticket #3872 .
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
7caacc50ae
avcodec/hevc_ps: do cleanup in case of unsupported bit depth
...
Fixes memleak
Fixes CID1231989
Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Reviewed-by: Mickaël Raulet <mraulet@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Christophe Gisquet
5ec85c9750
hevc: do generic validation of bitstream
...
After finishing parsing VPS/SPS/PPS/slice header, check remaining bits,
and if an overconsumption occurred, report invalid data.
Liked-by: BBB
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Christophe Gisquet
0d0d24af01
hevc_ps: verify P/T/L information
...
This makes the SPS parsing a little, but barely, safer.
Reviewed-by: Mickaël Raulet <mraulet@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Mickaël Raulet
37822a566f
hevc/rext: remove warning for cross component prediction
...
cherry picked from commit fb595102593c94179502bb1967f978a1d99cc7cb
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Mickaël Raulet
627c044f50
hevc/rext: fixing rdpcm and scaling list for Range Extension
...
-new rext bitstreams:
PERSIST_RPARAM_A_RExt_Sony_1.bit ok =
QMATRIX_A_RExt_Sony_1.bit ok =
SAO_A_RExt_MediaTek_1.bit ok =
(cherry picked from commit cdea029d452c521f8e5bcbe589f44b13a4011604)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
d13a731fc1
avcodec/hevc_ps: Check abs_delta_rps
...
Fixes integer overflow
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
ba80b8d29b
avcodec/hevc: check nb_cpb
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Mickaël Raulet
fdb20db642
hevc/rext: put a warning log message instead of an error log message
...
cherry picked from commit 243cb99cff727d6a14c32cdff2748f6c255dbaf4
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
0fc2045d5f
avcodec/hevc_ps: prevent stale pointer in malloc failure case
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Mickaël Raulet
5a41999d81
hevc/rext: basic infrastructure for supporting range extension
...
- support for 4:2:2 and 4:4:4 up to 12 bits
- add a new profile for range extension
(cherry picked from commit d3c067fa65bbc871758d28aa07f54123430ca346)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
ccd6911c18
avcodec/hevc_ps: do not loose all reference to pointers still in use
...
Fixes leaving a pointer to unreferenced memory
Fixes Ticket 3115
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Michael Niedermayer
66558fe715
avcodec/hevc_ps: Check layer_id_included_flags count
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Michael Niedermayer
6935ae22ed
avcodec/hevc_ps: more complete check for vps_max_dec_pic_buffering
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Anton Khirnov
17e9d52c8c
hevc_ps: remove a write-only variable
11 years ago
Gildas Cocherel
a0e1c3517a
hevc: remove unused array min_cb_addr_zs
...
Signed-off-by: Anton Khirnov <anton@khirnov.net>
11 years ago
Kieran Kunhya
9b60d91979
hevc: Allow out of bound values for num_reorder_pics
...
This fixes decoding for a sample that cannot be shared
Signed-off-by: Anton Khirnov <anton@khirnov.net>
11 years ago
gcocherel
ba70563d55
hevc/pps: optimized size of min_tb_addr_zs
...
reduce computation too
(cherry picked from commit 39c4d45c7788081c45c7fae51b7c5d0bcbaece9d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
gcocherel
f7f1f4c7ce
avcodec/hevc_ps: remove min_cb_addr_zs
...
(cherry picked from commit ff7926d5092f9d4158108963e977e8c992322ba4)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Kieran
bc21260e64
hevc: Fix 4K sample video
...
Reviewed-by: smarter
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Vittorio Giovara
4d33873c29
hevc: make pps/sps ids unsigned where necessary
...
Fixes integer overflow and out of array accesses.
Found-by: Mateusz j00ru Jurczyk and Gynvael Coldwind
11 years ago
Michael Niedermayer
920c01adce
hevc: Use get_bits_long() in decode_vui()
...
Fix assertion failure.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
11 years ago
Luca Barbato
175e506332
hevc: Mention the missing SPS in the error message
11 years ago
Michael Niedermayer
b818637b84
avcodec/hevc_ps: Use get_bits_long() in decode_vui()
...
Fix assertion failure
Fixes: a225222ef88a0f5b1e93e1d0432debc3-asan_static-oob_124a17d_1448_cov_77608227_DBLK_E_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Luca Barbato
838740e642
hevc: Prevent some integer overflows
...
get_ue_golomb_long() returns an unsigned.
Sample-Id: 00001541-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
11 years ago
Luca Barbato
b37e796082
hevc: Use uint64 to check for tile dimensions
...
And use unsigned datatypes.
Otherwise it would overflow.
Sample-Id: 00001315-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
11 years ago
Michael Niedermayer
a7a07cc98a
hevc: check that VPS referenced from SPS exists
...
This matches how its done for SPS/PPS.
Fixes null pointer dereference.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
11 years ago
Michael Niedermayer
ab296c7a9f
avcodec/hevc_ps: Override max_dec_pic_buffering when its inconsistent with num_reorder_pics
...
Fixes Ticket3304
Alternatively max_dec_pic_buffering could be removed completely as its not used.
Based on a patch by Jose Santiago <santiago@haivision.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Michael Niedermayer
d66bab0a69
avcodec/hevc_ps: check that VPS referenced from SPS exists
...
This matches how its done for SPS/PPS.
An alternative to this is to check it when its used.
Fixes null pointer dereference
Fixes: signal_sigsegv_e30a43_1437_CIP_A_Panasonic_3.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago