Prevent a division by zero down the codepath.
Sample-Id: 00001721-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Lanczos for general case, sinc for upscaling, Gaussian for
downscaling. According to current literature these scalers
should be the best quality-wise algorithms for each case.
Inspired from a patch by wm4 <nfxjfg@googlemail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
This reverts parts of d6d5ef5534, that didn't work right. (The
tests that were added failed on big endian, and the output looked
garbled on little endian as well.)
This is due to the fact that the intermediate scaling values (from
e.g. hScale8To19_c or hScale16To19_c) are stored as int32_t and
thus requires a separate output function, while yuv2gbrp_full_X_c
only interprets it as int16_t.
Signed-off-by: Martin Storsjö <martin@martin.st>
This code path is not implemented and makes not much sense to implement
either.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
sws_getCachedContext() and sws_getContext() expect sws_alloc_context()
to return NULL when out of memory, as follows.
if (!(context = sws_alloc_context()))
return NULL;
This patch fixes sws_alloc_context() to return NULL in that case.
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Some systems, e.g. Minix, have sys/mman.h defining MAP_ANONYMOUS without
providing (working) mmap and friends. The mmx filter generation code
checks only for MAP_ANONYMOUS, not for availability of mmap itself which
leads to build errors on aforementioned systems.
This changes the conditional compilation to use mmap only if all the
required functions are available.
Signed-off-by: Mans Rullgard <mans@mansr.com>
This reverts parts of e0c6cce447. There is external mmx asm that
requires this alignment.
This fixes crashes when using swscale in builds with external mmx,
without inline assembly.
Signed-off-by: Martin Storsjö <martin@martin.st>
Refactoring mmx2/mmxext YASM code with cpuflags will force renames.
So switching to a consistent naming scheme beforehand is sensible.
The name "mmxext" is more official and widespread and also the name
of the CPU flag, as reported e.g. by the Linux kernel.
At very small dimensions, this calculation could lead to zero-sized
filters, which leads to uninitialized output, zero-sized allocations,
loop overflows in SIMD that uses do{..}while(i++<filtersize); instead
of for(i=0;i<filtersize;i++){..} and several other similar failures.
Therefore, require a minimum filtersize of 1.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
This fixes integer multiplication overflows in RGB48 output
(vertical) scaling as detected by IOC. What happens is that for
certain types of filters (lanczos, spline, bicubic), the
intermediate sum of coefficients in the middle of a filter can
be larger than the fixed-point equivalent of 1.0, even if the
final sum is 1.0. This is fine and we support that.
However, at frame edges, initFilter() will merge the coefficients
for the off-screen pixels into the top or bottom pixel, such as
to emulate edge extension. This means that suddenly, a single
coefficient can be larger than the fixed-point equivalent of
1.0, which the vertical scaling routines do not support.
Therefore, remove the merging of coefficients for edges for
the vertical scaling filter, and instead add edge detection
to the scaler itself so that it copies the pointers (not data)
for the edges (i.e. it uses line[0] for line[-1] as well), so
that a single coefficient is never larger than the fixed-point
equivalent of 1.0.
This was removed erroneously in
046f081b46. This define still is
necessary for getting MAP_ANONYMOUS defined on linux/glibc,
despite the define reshuffling done in that commit.
Without MAP_ANONYMOUS defined, the mprotect calls for setting the
generated mmx2 scaler code pages executable are left out, causing
crashes if that codepath is chosen.
This patch fixes scaling from 192x144 to 320x240 with
-sws_flags fast_bilinear, which crashes on linux at the
moment.
Signed-off-by: Martin Storsjö <martin@martin.st>
Luca Barbato
Vittorio Giovara
Janne Grunau
Jean First
Diego Biurrun
Martin Storsjö
Michael Niedermayer
Xi Wang
Mans Rullgard
Anton Khirnov
Ronald S. Bultje
Alex Converse
Paul B Mahol
Reinhard Tartler