Michael Niedermayer
a3a4082599
avcodec/h264_cabac: Fix runtime error: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself
...
Fixes: 858/clusterfuzz-testcase-5168477042114560
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
8 years ago
Michael Niedermayer
258763ad0e
avcodec/h264_cabac: runtime error: signed integer overflow: 2147483647 + 14 cannot be represented in type 'int'
...
Fixes: 614/clusterfuzz-testcase-4931860079575040
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
8 years ago
Anton Khirnov
8281cd5cb8
h264_cabac: drop an always true condition
9 years ago
Anton Khirnov
9df889a5f1
h264: rename h264.[ch] to h264dec.[ch]
...
This is more consistent with the naming of other decoders.
9 years ago
James Almer
94011f994a
avodec/h264dec: fix compilation with -DTRACE
...
Signed-off-by: James Almer <jamrial@gmail.com>
9 years ago
Anton Khirnov
6e92181bf8
h264: pass just the PPS to get_chroma_qp()
...
It does not need the whole context. This will simplify the following
commit.
9 years ago
Diego Biurrun
4f81f8dba7
Drop unnecessary golomb.h #includes
9 years ago
Vittorio Giovara
41ed7ab45f
cosmetics: Fix spelling mistakes
...
Signed-off-by: Diego Biurrun <diego@biurrun.de>
9 years ago
Diego Biurrun
c11c693acc
h264: Drop broken trace debug output
9 years ago
Anton Khirnov
3176217c60
h264: decouple h264_ps from the h264 decoder
...
Make the SPS/PPS parsing independent of the H264Context, to allow
decoupling the parser from the decoder. The change is modelled after the
one done earlier for HEVC.
Move the dequant buffers to the PPS to avoid complex checks whether they
changed and an expensive copy for frame threads.
9 years ago
Anton Khirnov
ecc31f6b08
h264: move ff_h264_check_intra[4x4]_pred_mode() to h264_parse
...
It is shared with svq3.
9 years ago
Diego Biurrun
a7da517f6a
h264data: Move all data tables from a header to a .c file
9 years ago
Michael Niedermayer
e5655a32bc
avcodec/h264_cabac: Check decode_cabac_mb_mvd() for failure
...
Fixes harmless integer overflow
Fixes Ticket5150
No speedloss measured, actually its slightly faster, but please benchmark & double check this
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
9 years ago
Michael Niedermayer
8000d484b8
avcodec/cabac: Check initial cabac decoder state
...
Fixes integer overflows
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
9 years ago
Vittorio Giovara
10aa6f9db4
h264: Use the correct argument context in debug log
9 years ago
Michael Niedermayer
d52a8bf158
avcodec/h264_cabac: Use ff_tlog() for block level information
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Vittorio Giovara
6a85dfc830
lavc: Replace av_dlog and tprintf with internal macros
10 years ago
Michael Niedermayer
563a8b4aac
avcodec/h264_cabac: Drop local_ref_count
...
7966 -> 7897 decicycles
This basically switches to the implementation from f6f7d15041
which seems faster
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
20f5be1038
avcodec/h264_cabac: Fix intra4x4_pred_mode_cache argument to av_dlog()
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Anton Khirnov
b53569e068
h264_cabac: remove now unnecessary H264Context function parameters
10 years ago
Anton Khirnov
ba7e165b86
h264_cabac: constify all uses of H264Context
...
All the variables modified by this code are either per-MB arrays or have
been moved to the per-slice context
10 years ago
Anton Khirnov
e9b2383bf8
h264: move mb_mbaff into the per-slice context
10 years ago
Anton Khirnov
bc98e8c0e0
h264: move mb_field_decoding_flag into the per-slice context
10 years ago
Anton Khirnov
d4d9068cdf
h264: move mb_{x,y} into the per-slice context
10 years ago
Anton Khirnov
0edbe6faa7
h264: move mb_xy into the per-slice context
10 years ago
Anton Khirnov
a9b201cacf
h264: move cabac_init_idc into the per-slice context
10 years ago
Anton Khirnov
bd3e460b73
h264: move direct_cache into the per-slice context
10 years ago
Anton Khirnov
6479c79f55
h264: move mvd_cache into the per-slice context
10 years ago
Anton Khirnov
5c8280c307
h264: move last_qscale_diff into the per-slice context
10 years ago
Anton Khirnov
e7226984ac
h264: move [{top,left}_]cbp into the per-slice context
10 years ago
Anton Khirnov
30da98adbd
h264: move cabac[_state] into the per-slice context
10 years ago
Anton Khirnov
bf03a878a7
h264: move mb[_{padding,luma_dc}] into the per-slice context
10 years ago
Anton Khirnov
b063582e0c
h264: move intra_pcm_ptr into the per-slice context
10 years ago
Anton Khirnov
95eb35f305
h264: move the ref lists variables into the per-slice context
10 years ago
Anton Khirnov
ed451a08a4
h264: move sub_mb_type into the per-slice context
10 years ago
Anton Khirnov
56febc993b
h264: move the slice type variables into the per-slice context
10 years ago
Anton Khirnov
99a35d1ccb
h264: move neighbor_transform_size into the per-slice context
10 years ago
Anton Khirnov
e6287f077c
h264: move {mv,ref}_cache into the per-slice context
10 years ago
Anton Khirnov
f69574cf7a
h264: move non_zero_count_cache into the per-slice context
10 years ago
Anton Khirnov
64c81b2cd0
h264: move *_samples_available into the per-slice context
10 years ago
Anton Khirnov
7d8154edd5
h264: move intra4x4_pred_mode[_cache] into the per-slice context
10 years ago
Anton Khirnov
8b00f4df20
h264: move some neighbour information into the per-slice context
10 years ago
Anton Khirnov
4bd5ac200d
h264: move {chroma,intra16x16}_pred_mode into the per-slice context
10 years ago
Anton Khirnov
5355ed6b20
h264: move {prev,next}_mb_skipped into the per-slice context
10 years ago
Anton Khirnov
d231e84b06
h264: move the quantizers into the per-slice context
10 years ago
Michael Niedermayer
592ba6ec10
avcodec/h264_cabac: use int instead of long for mbb_xy
...
The mb address fits in int
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
def69553e7
h264_cabac: Break infinite loops
...
This fixes out of array reads and/or infinite loops.
30 is the maximum number of bits that can be read into
coeff_abs below.
CC: libav-stable@libav.org
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Martin Storsjö <martin@martin.st>
10 years ago
Vittorio Giovara
277ff7f5dc
lavu: move internal define to the only places where it is used
10 years ago
Michael Niedermayer
4cb9f1a774
h264_cabac: fix one fill_rectangle() indentation level
...
Bug-Id: CID 1087088
10 years ago
Diego Biurrun
2143948381
Drop unnecessary av_unused attributes.
10 years ago