Andreas Cadhalpun
d637a58750
diracdec: fix idwt_stride calculation in bytes
...
The transformation to bytes must happen after alignment to get the same
resulting pointers as before.
This fixes segmentation faults in the assembler code.
The regression was introduced in commit 9553689
.
Reviewed-by: Kieran Kunhya <kierank@obe.tv>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
9 years ago
Michael Niedermayer
73840bbe4e
avcodec/diracdec: Check ff_set_dimensions() for failure
...
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
9 years ago
Michael Niedermayer
ffad6f6b89
avcodec/diracdec: fix aspect ratio (it was lost after efcc8fddd6
)
...
Reviewed-by: Hendrik Leppkes <h.leppkes@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
9 years ago
Kieran Kunhya
25f6ccccd6
diracdec: Fix codeblock parameters reading
9 years ago
Kieran Kunhya
a349a10edf
diracdec: Add support for HQ profile
9 years ago
Rostislav Pehlivanov
d8f13e783a
diracdec: remove duplicate codeblock decoding
...
Broken by commit 7424a6d0a5
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
9 years ago
Kieran Kunhya
3652dd5d0c
diracdec: Fix FPE on invalid low_delay data
9 years ago
Kieran Kunhya
cdf8c9038d
diracdec: Replace dirac parse codes with better ones
9 years ago
Kieran Kunhya
7424a6d0a5
diracdec: Read picture types by using parse_code
9 years ago
Kieran Kunhya
8eb6acef92
diracdec: Support new extended quantiser range
9 years ago
Kieran Kunhya
8dcc99dc68
diracdec: Extract version parameters
9 years ago
Kieran Kunhya
9f374c5906
diracdec: Make slice parameters common between lowdelay and future hq profile
9 years ago
Kieran Kunhya
3bb6ce1af9
diracdec: Rename lowdelay_subband to decode_subband because it is shared with HQ profile
9 years ago
Kieran Kunhya
3f07f12f65
diracdec: Template DSP functions adding 10-bit versions
9 years ago
Kieran Kunhya
9553689854
diracdec: Move strides to bytes, and pointer types to uint8_t.
...
Start templating functions for move to support 10-bit
Parts of this patch were written by Rostislav Pehlivanov
9 years ago
Michael Niedermayer
cb5190bc9d
avcodec/diracdec: Move reference to DiracFrame, avoid use of the deprecated field from AVFrame
...
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
10 years ago
Michael Niedermayer
1c5b712c0a
avcodec/diracdec: Check for hpel_base allocation failure
...
Fixes null pointer dereference
Fixes: signal_sigsegv_b02a96_280_RL_420p_ffdirac.drc with memlimit of 67108864
Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
10 years ago
Michael Niedermayer
8f1afde11d
avcodec/diracdec: Make data_unit_size unsigned
...
Fixes CID1271788
with this change the value is more explicitly checked, it was fully checked
before though
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
9253cc496a
avcodec/diracdec: Replace default_bsep[] by multiplication
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Andreas Cadhalpun
75fc81c831
diracdec: check that block length is valid
...
In init_planes p->xblen and p->yblen are set to:
p->xblen = s->plane[0].xblen >> s->chroma_x_shift;
p->yblen = s->plane[0].yblen >> s->chroma_y_shift;
These are later used as block_w and block_h arguments of
s->vdsp.emulated_edge_mc. If one of them is 0 it triggers an av_assert2
in emulated_edge_mc:
av_assert2(start_x < end_x && block_w > 0);
av_assert2(start_y < end_y && block_h > 0);
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Andreas Cadhalpun
4b13a542a2
dirac: use and forward error codes
...
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
a2190de52d
avcodec/diracdec: Fix types and wraparounds in dirac_decode_picture_header()
...
previously various variables had a too small type to support the required 32bit unsigned
range allowed from the spec
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Andreas Cadhalpun
d93181ef3e
diracdec: check if reference could not be allocated
...
s->ref_pics[i] is later used as ref argument of interpolate_refplane,
where it is dereferenced.
If it is NULL, it causes a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Andreas Cadhalpun
9e66b39aa8
diracdec: avoid overflow of bytes*8 in decode_lowdelay
...
If bytes is large enough, bytes*8 can overflow and become negative.
In that case 'bufsize -= bytes*8' causes bufsize to increase instead of
decrease.
This leads to a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
5a455dd011
avcodec/diracdec: Move buf[] read after size check
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Andreas Cadhalpun
984f50deb2
diracdec: prevent overflow in data_unit_size check
...
buf_idx + data_unit_size can overflow, causing the '> buf_size' check to
wrongly fail.
This causes a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
a4d3cf10b2
avcodec/diracdec: Check slices malloc and propagate error code
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
5145d22b88
avcodec/diracdec: Tighter checks on CODEBLOCKS_X/Y
...
Fixes very long but finite loop
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
526886e606
avcodec/diracdec: Use 64bit in calculation of codeblock coordinates
...
Fixes integer overflow
Fixes out of array read
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
10 years ago
Michael Niedermayer
baee313aa9
avcodec/diracdec: Fix "warning: comparison of constant 4 with expression of type enum dirac_subband is always true [-Wtautological-constant-out-of-range-compare]"
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Michael Niedermayer
5bedbc3da2
avcodec/diracdec: remove unneeded assignment
...
Found-by: CSA
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
James Almer
b67a0e99ee
diracdec: don't call ff_emulated_edge_mc_8 directly
...
Use the videodsp function pointer instead.
Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
James Almer
ad24256e7e
diracdec: remove unused dsputil context
...
Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Lou Logan
88f2586adb
fix various typos
...
Signed-off-by: Lou Logan <lou@lrcd.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Michael Niedermayer
4a30f08505
avcodec/diracdec: move mc buffer allocation to per frame
...
Fixes out of array accesses for non default buffers with large strides
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Michael Niedermayer
b8598f6ce6
avcodec/diracdec: fix undefined behavior with shifts
...
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Michael Niedermayer
46375e9a97
avcodec/diracdec: use av_malloc(z)_array
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Michael Niedermayer
7c6cf689d8
avcodec/diracdec: allocate edges internally instead of depending on CODEC_FLAG_EMU_EDGE being not set
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Nicolas George
a91394f4de
lavc/diracdec: check av_frame_alloc() failure.
11 years ago
Michael Niedermayer
fca7943850
avcodec/diracdec: avoid depending on sizeof(AVFrame)
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
11 years ago
Clément Bœsch
b46f19100b
cosmetics: group remaining .name and .long_name.
...
See b2bed9325
.
12 years ago
Ronald S. Bultje
face578d56
Rewrite emu_edge functions to have separate src/dst_stride arguments.
...
This allows supporting files for which the image stride is smaller than
the max. block size + number of subpel mc taps, e.g. a 64x64 VP9 file
or a 16x16 VP8 file with -fflags +emu_edge.
12 years ago
Ronald S. Bultje
c341f734e5
Convert multiplier for MV from int to ptrdiff_t.
...
This prevents emulated_edge_mc from not undoing mvy*stride-related
integer overflows.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago
Michael Niedermayer
5dff269998
avcodec/diracdec: use init_get_bits8()
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago
Clément Bœsch
1ec94b0f06
lavc: factorize ff_{thread_,re,}get_buffer error messages.
...
Coccinelle profile used:
@@
expression r, ctx, f, loglevel, str, flags;
@@
-if ((r = ff_get_buffer(ctx, f, flags)) < 0) {
- av_log(ctx, loglevel, str);
- return r;
-}
+if ((r = ff_get_buffer(ctx, f, flags)) < 0)
+ return r;
@@
expression r, ctx, f, loglevel, str;
@@
-if ((r = ff_reget_buffer(ctx, f)) < 0) {
- av_log(ctx, loglevel, str);
- return r;
-}
+if ((r = ff_reget_buffer(ctx, f)) < 0)
+ return r;
@@
expression r, ctx, f, loglevel, str, flags;
@@
-if ((r = ff_thread_get_buffer(ctx, f, flags)) < 0) {
- av_log(ctx, loglevel, str);
- return r;
-}
+if ((r = ff_thread_get_buffer(ctx, f, flags)) < 0)
+ return r;
...along with some manual patches for the remaining ones.
12 years ago
Ronald S. Bultje
c7e3e55429
Move ff_emulated_edge_mc prototypes to videodsp.
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago
Michael Niedermayer
30981a966f
lavc: split snow and dirac DWTs
...
There is only about 4 lines of common code, so it alot cleaner
when seperated.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago
Michael Niedermayer
f5d6b0c9c2
diracdec: fix typo in mctmp allocation
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago
Michael Niedermayer
2f6ec9fdd7
diracdec: Test mctmp and mcscratch for malloc failure.
...
Fixes null pointer dereference
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago
Michael Niedermayer
b84d1bf193
diracdec: fix emulated_edge condition, fix out of array reads
...
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
12 years ago