Michael Niedermayer
b789ebf681
avcodec/h264_cavlc: Fix integer overflows with motion vector residual addition
...
Fixes: signed integer overflow: 14 + 2147483647 cannot be represented in type 'int'
Fixes: 14794/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_H264_fuzzer-5677380695228416
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
5 years ago
Michael Niedermayer
f72b9904fe
avcodec/h264_cavlc: Check mb_skip_run
...
Fixes: 10300/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_H264_fuzzer-6292205497483264
Fixes: signed integer overflow: -2147483648 - 1 cannot be represented in type 'int'
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
6 years ago
Xiaohan Wang
71f39de2a5
avcodec/h264_cavlc: Set valid qscale value in ff_h264_decode_mb_cavlc()
...
When ff_h264_decode_mb_cavlc() failed due to wrong sl->qscale values,
e.g. dquant out of range, set the qscale to be a valid value before
returning -1 and exiting the function. The qscale value can be used
later e.g. in loop filter.
BUG=806122
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years ago
Anton Mitrofanov
70946e6059
h264dec: Fix mix of lossless and lossy MBs decoding
...
CC: libav-stable@libav.org
Signed-off-by: Anton Khirnov <anton@khirnov.net>
7 years ago
Wan-Teh Chang
99c5ac2039
avcodec/h264: Declare the local variable decode_chroma as const.
...
ff_h264_decode_mb_cabac() and ff_h264_decode_mb_cavlc() are very long
functions. Declaring decode_chroma as const makes it clear the variable
doesn't change after initialization.
Signed-off-by: Wan-Teh Chang <wtc@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
7 years ago
Anton Mitrofanov
cf231b68da
avcodec/h264: Fix mix of lossless and lossy MBs decoding
...
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
8 years ago
Michael Niedermayer
0e87c07d87
avcodec/h264_cavlc: Fix runtime error: index -1 out of bounds for type 'VLC [6]'
...
Fixes: 1639/clusterfuzz-testcase-minimized-5693801463021568
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
8 years ago
Michael Niedermayer
54e1b62ee2
avcodec/h264_cavlc: Fix runtime error: index -1 out of bounds for type 'VLC [15]
...
Fixes: 1513/clusterfuzz-testcase-minimized-6246484833992704
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
8 years ago
Michael Niedermayer
fc8cff96ed
avcodec/h264_cavlc: Fix undefined behavior on qscale overflow
...
Fixes: 1214/clusterfuzz-testcase-minimized-6130606599569408
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
8 years ago
Anton Khirnov
522d850e68
h264_cavlc: check the value of run_before
...
Section 9.2.3.2 of the spec implies that run_before must not be larger
than zeros_left.
Fixes invalid reads with corrupted files.
CC: libav-stable@libav.org
Bug-Id: 1000
Found-By: Kamil Frankowicz
8 years ago
Diego Biurrun
d4c2103bd3
golomb: Convert to the new bitstream reader
8 years ago
Anton Khirnov
9df889a5f1
h264: rename h264.[ch] to h264dec.[ch]
...
This is more consistent with the naming of other decoders.
9 years ago
James Almer
94011f994a
avodec/h264dec: fix compilation with -DTRACE
...
Signed-off-by: James Almer <jamrial@gmail.com>
9 years ago
Anton Khirnov
6e92181bf8
h264: pass just the PPS to get_chroma_qp()
...
It does not need the whole context. This will simplify the following
commit.
9 years ago
Diego Biurrun
52567e8198
get_bits: Drop some TRACE-level debug code
...
It will not be provided by the new bit reader anyway.
9 years ago
Vittorio Giovara
41ed7ab45f
cosmetics: Fix spelling mistakes
...
Signed-off-by: Diego Biurrun <diego@biurrun.de>
9 years ago
Diego Biurrun
c11c693acc
h264: Drop broken trace debug output
9 years ago
Anton Khirnov
3176217c60
h264: decouple h264_ps from the h264 decoder
...
Make the SPS/PPS parsing independent of the H264Context, to allow
decoupling the parser from the decoder. The change is modelled after the
one done earlier for HEVC.
Move the dequant buffers to the PPS to avoid complex checks whether they
changed and an expensive copy for frame threads.
9 years ago
Anton Khirnov
ecc31f6b08
h264: move ff_h264_check_intra[4x4]_pred_mode() to h264_parse
...
It is shared with svq3.
9 years ago
Diego Biurrun
a7da517f6a
h264data: Move all data tables from a header to a .c file
9 years ago
Diego Biurrun
6b96d2dcda
cosmetics: Drop particularly redundant silly comments
9 years ago
Vittorio Giovara
6a85dfc830
lavc: Replace av_dlog and tprintf with internal macros
10 years ago
Michael Niedermayer
797ba4d53b
avcodec/h264_cavlc: Drop local_ref_count
...
This basically switches to the implementation from f6f7d15041
which seems faster
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Anton Khirnov
7505c8dfba
h264_cavlc: constify all uses of H264Context
...
All the variables modified by this code are either per-MB arrays or have
been moved to the per-slice context
10 years ago
Anton Khirnov
e9b2383bf8
h264: move mb_mbaff into the per-slice context
10 years ago
Anton Khirnov
bc98e8c0e0
h264: move mb_field_decoding_flag into the per-slice context
10 years ago
Anton Khirnov
f42485dbce
h264: use a separate GetBitContext for slice data
10 years ago
Anton Khirnov
404a416d4b
h264: remove some remnants of data partitioning
10 years ago
Anton Khirnov
d4d9068cdf
h264: move mb_{x,y} into the per-slice context
10 years ago
Anton Khirnov
0edbe6faa7
h264: move mb_xy into the per-slice context
10 years ago
Anton Khirnov
47a0d39350
h264: move mb_skip_run into the per-slice context
10 years ago
Anton Khirnov
e7226984ac
h264: move [{top,left}_]cbp into the per-slice context
10 years ago
Anton Khirnov
bf03a878a7
h264: move mb[_{padding,luma_dc}] into the per-slice context
10 years ago
Anton Khirnov
b063582e0c
h264: move intra_pcm_ptr into the per-slice context
10 years ago
Anton Khirnov
95eb35f305
h264: move the ref lists variables into the per-slice context
10 years ago
Anton Khirnov
ed451a08a4
h264: move sub_mb_type into the per-slice context
10 years ago
Anton Khirnov
56febc993b
h264: move the slice type variables into the per-slice context
10 years ago
Anton Khirnov
e6287f077c
h264: move {mv,ref}_cache into the per-slice context
10 years ago
Anton Khirnov
f69574cf7a
h264: move non_zero_count_cache into the per-slice context
10 years ago
Anton Khirnov
64c81b2cd0
h264: move *_samples_available into the per-slice context
10 years ago
Anton Khirnov
7d8154edd5
h264: move intra4x4_pred_mode[_cache] into the per-slice context
10 years ago
Anton Khirnov
8b00f4df20
h264: move some neighbour information into the per-slice context
10 years ago
Anton Khirnov
4bd5ac200d
h264: move {chroma,intra16x16}_pred_mode into the per-slice context
10 years ago
Anton Khirnov
5355ed6b20
h264: move {prev,next}_mb_skipped into the per-slice context
10 years ago
Anton Khirnov
d231e84b06
h264: move the quantizers into the per-slice context
10 years ago
Vittorio Giovara
d24e9a99a4
h264 does not depend on mpegvideo any more
11 years ago
Vittorio Giovara
e0c16e4e32
mpegvideo: move mpegvideo formats-related defines to mpegutils.h
11 years ago
Anton Khirnov
074c769de9
h264_cavlc: check the size of the intra PCM data.
...
Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
11 years ago
Michael Niedermayer
4fb1221e66
h264: reduce whitespace differences to libav
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Diego Biurrun
2a61592573
avcodec: Remove some commented-out debug cruft
11 years ago