Michael Niedermayer
cb5190bc9d
avcodec/diracdec: Move reference to DiracFrame, avoid use of the deprecated field from AVFrame
...
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
9 years ago
Michael Niedermayer
1c5b712c0a
avcodec/diracdec: Check for hpel_base allocation failure
...
Fixes null pointer dereference
Fixes: signal_sigsegv_b02a96_280_RL_420p_ffdirac.drc with memlimit of 67108864
Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
9 years ago
Michael Niedermayer
8f1afde11d
avcodec/diracdec: Make data_unit_size unsigned
...
Fixes CID1271788
with this change the value is more explicitly checked, it was fully checked
before though
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
9253cc496a
avcodec/diracdec: Replace default_bsep[] by multiplication
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Andreas Cadhalpun
75fc81c831
diracdec: check that block length is valid
...
In init_planes p->xblen and p->yblen are set to:
p->xblen = s->plane[0].xblen >> s->chroma_x_shift;
p->yblen = s->plane[0].yblen >> s->chroma_y_shift;
These are later used as block_w and block_h arguments of
s->vdsp.emulated_edge_mc. If one of them is 0 it triggers an av_assert2
in emulated_edge_mc:
av_assert2(start_x < end_x && block_w > 0);
av_assert2(start_y < end_y && block_h > 0);
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Andreas Cadhalpun
4b13a542a2
dirac: use and forward error codes
...
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
a2190de52d
avcodec/diracdec: Fix types and wraparounds in dirac_decode_picture_header()
...
previously various variables had a too small type to support the required 32bit unsigned
range allowed from the spec
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Andreas Cadhalpun
d93181ef3e
diracdec: check if reference could not be allocated
...
s->ref_pics[i] is later used as ref argument of interpolate_refplane,
where it is dereferenced.
If it is NULL, it causes a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Andreas Cadhalpun
9e66b39aa8
diracdec: avoid overflow of bytes*8 in decode_lowdelay
...
If bytes is large enough, bytes*8 can overflow and become negative.
In that case 'bufsize -= bytes*8' causes bufsize to increase instead of
decrease.
This leads to a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
5a455dd011
avcodec/diracdec: Move buf[] read after size check
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Andreas Cadhalpun
984f50deb2
diracdec: prevent overflow in data_unit_size check
...
buf_idx + data_unit_size can overflow, causing the '> buf_size' check to
wrongly fail.
This causes a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
a4d3cf10b2
avcodec/diracdec: Check slices malloc and propagate error code
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
5145d22b88
avcodec/diracdec: Tighter checks on CODEBLOCKS_X/Y
...
Fixes very long but finite loop
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
526886e606
avcodec/diracdec: Use 64bit in calculation of codeblock coordinates
...
Fixes integer overflow
Fixes out of array read
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
baee313aa9
avcodec/diracdec: Fix "warning: comparison of constant 4 with expression of type enum dirac_subband is always true [-Wtautological-constant-out-of-range-compare]"
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
5bedbc3da2
avcodec/diracdec: remove unneeded assignment
...
Found-by: CSA
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
James Almer
b67a0e99ee
diracdec: don't call ff_emulated_edge_mc_8 directly
...
Use the videodsp function pointer instead.
Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
James Almer
ad24256e7e
diracdec: remove unused dsputil context
...
Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Lou Logan
88f2586adb
fix various typos
...
Signed-off-by: Lou Logan <lou@lrcd.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Michael Niedermayer
4a30f08505
avcodec/diracdec: move mc buffer allocation to per frame
...
Fixes out of array accesses for non default buffers with large strides
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Michael Niedermayer
b8598f6ce6
avcodec/diracdec: fix undefined behavior with shifts
...
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Michael Niedermayer
46375e9a97
avcodec/diracdec: use av_malloc(z)_array
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Michael Niedermayer
7c6cf689d8
avcodec/diracdec: allocate edges internally instead of depending on CODEC_FLAG_EMU_EDGE being not set
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Nicolas George
a91394f4de
lavc/diracdec: check av_frame_alloc() failure.
11 years ago
Michael Niedermayer
fca7943850
avcodec/diracdec: avoid depending on sizeof(AVFrame)
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Clément Bœsch
b46f19100b
cosmetics: group remaining .name and .long_name.
...
See b2bed9325
.
11 years ago
Ronald S. Bultje
face578d56
Rewrite emu_edge functions to have separate src/dst_stride arguments.
...
This allows supporting files for which the image stride is smaller than
the max. block size + number of subpel mc taps, e.g. a 64x64 VP9 file
or a 16x16 VP8 file with -fflags +emu_edge.
11 years ago
Ronald S. Bultje
c341f734e5
Convert multiplier for MV from int to ptrdiff_t.
...
This prevents emulated_edge_mc from not undoing mvy*stride-related
integer overflows.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Michael Niedermayer
5dff269998
avcodec/diracdec: use init_get_bits8()
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Clément Bœsch
1ec94b0f06
lavc: factorize ff_{thread_,re,}get_buffer error messages.
...
Coccinelle profile used:
@@
expression r, ctx, f, loglevel, str, flags;
@@
-if ((r = ff_get_buffer(ctx, f, flags)) < 0) {
- av_log(ctx, loglevel, str);
- return r;
-}
+if ((r = ff_get_buffer(ctx, f, flags)) < 0)
+ return r;
@@
expression r, ctx, f, loglevel, str;
@@
-if ((r = ff_reget_buffer(ctx, f)) < 0) {
- av_log(ctx, loglevel, str);
- return r;
-}
+if ((r = ff_reget_buffer(ctx, f)) < 0)
+ return r;
@@
expression r, ctx, f, loglevel, str, flags;
@@
-if ((r = ff_thread_get_buffer(ctx, f, flags)) < 0) {
- av_log(ctx, loglevel, str);
- return r;
-}
+if ((r = ff_thread_get_buffer(ctx, f, flags)) < 0)
+ return r;
...along with some manual patches for the remaining ones.
12 years ago
Ronald S. Bultje
c7e3e55429
Move ff_emulated_edge_mc prototypes to videodsp.
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago
Michael Niedermayer
30981a966f
lavc: split snow and dirac DWTs
...
There is only about 4 lines of common code, so it alot cleaner
when seperated.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago
Michael Niedermayer
f5d6b0c9c2
diracdec: fix typo in mctmp allocation
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago
Michael Niedermayer
2f6ec9fdd7
diracdec: Test mctmp and mcscratch for malloc failure.
...
Fixes null pointer dereference
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago
Michael Niedermayer
b84d1bf193
diracdec: fix emulated_edge condition, fix out of array reads
...
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago
Paul B Mahol
4012cd6c4f
lavc: fix decode_frame() third parameter semantics for rest of video decoders
...
Signed-off-by: Paul B Mahol <onemda@gmail.com>
12 years ago
Michael Niedermayer
3ba58433e1
diracdec: check wavelet depth, prevent out of array read.
...
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago
Michael Niedermayer
0a4087b893
diracdec: Use only one frame per reference
...
Prevents null ptr dereference
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago
Michael Niedermayer
7dab48bb73
diracdec: allocate enough space for blocks
...
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago
Michael Niedermayer
dca2fa10d3
diracdec: fix edge emulation check, fixes out of array reads
...
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago
Lou Logan
6851130fd6
cosmetics: minor libavcodec spelling errors
...
Also update some common misspelled words in patcheck
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
13 years ago
Clément Bœsch
3c1d52d30b
Fix a few @file doxy inconsistencies.
13 years ago
Michael Niedermayer
601d072e68
diracdec: check xybsep
...
Fixes division by 0
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
13 years ago
Michael Niedermayer
efe1ba7201
diracdec: check lowdelay bytes.
...
Fixes division by zero
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
13 years ago
Michael Niedermayer
6a052e64b5
direcdec: fix infinite loop
...
Fixes Ticket1156
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
13 years ago
Michael Niedermayer
0f13cc732b
diracdec: Correct the bytestream end pointer.
...
This fixes some arith decoder overreads and a potential infinite loop.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
13 years ago
Michael Niedermayer
ad21c05172
diracdec: fix use of deprecated funnctions
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
13 years ago
Michael Niedermayer
5cd8afee99
diracdec: Check for negative quants which would cause out of array reads.
...
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
13 years ago
Michael Niedermayer
9729f140ae
diracdec: Fix integer overflow leading to out of global array read.
...
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
13 years ago
Michael Niedermayer
ddf0c1d86a
diracdec: Check num_refs.
...
Fixes: CVE-2011-3950
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
13 years ago