Affected the vsynth*-jpeg2000 and the vsynth*-jpeg2000-97 FATE tests
(where * ranges over { 1, 2, 3, _lena }) as well as ticket #7983.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: signed integer overflow: 2147483598 + 128 cannot be represented in type 'int'
Fixes: 12926/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-5705100733972480
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: signed integer overflow: 1157259380 + 1157259380 cannot be represented in type 'int'
Fixes: 2365/clusterfuzz-testcase-minimized-6020421927305216
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: signed integer overflow: -163654656 * 256 cannot be represented in type 'int'
Fixes: 2367/clusterfuzz-testcase-minimized-4648678897745920
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array access
Fixes: 01859c9a9ac6cd60a008274123275574/asan_heap-oob_1dff571_8250_50d3d1611e294c3519fd1fa82198b69b.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This should avoid problems on systems with little stack space and fix some crashes
in fate
crash found-by: jamrial
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This reduces the number of operations
Its not done for 9/7i as that would overflow thanks to JPEG2000 allowing
32 decomposition levels
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
thats how the specification defines it, this also improves numerical
accuracy of the integer wavelet implementation. It otherwise should
be equivalent, in case of overflows this can be reverted.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Even if the jpeg2000 spec uses a wrong value this does not
make mathematics work this way, also this has been corrected in the 2004
version AFAIK
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
We are missing the handling of some special cases for this.
These cases should be unused and there should be no reason
to ever use them unless some spec dictates their use
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Tha fate tests change because the edge mirroring was wrong before this commit
Reviewed-by: Nicolas BERTRAND <nicoinattendu@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Based on the 2007 GSoC project from Kamil Nowosad <k.nowosad@students.mimuw.edu.pl>
Updated to current programming standards, style and many more small
fixes by Diego Biurrun <diego@biurrun.de>.
Signed-off-by: Diego Biurrun <diego@biurrun.de>
Andreas Rheinhardt
Michael Niedermayer
Michael Niedermayer
Nicolas Bertrand