This fixes the same overflow as in the RGB48/16-bit YUV scaling;
some filters can overflow both negatively and positively (e.g.
spline/lanczos), so we bias a signed integer so it's "half signed"
and "half unsigned", and can cover overflows in both directions
while maintaining full 31-bit depth.
Signed-off-by: Mans Rullgard <mans@mansr.com>
We're shifting individual components (8-bit, unsigned) left by 24,
so making them unsigned should give the same results without the
overflow.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
For certain types of filters where the intermediate sum of coefficients
can go above the fixed-point equivalent of 1.0 in the middle of a filter,
the sum of a 31-bit calculation can overflow in both directions and can
thus not be represented in a 32-bit signed or unsigned integer. To work
around this, we subtract 0x40000000 from a signed integer base, so that
we're halfway signed/unsigned, which makes it fit even if it overflows.
After the filter finishes, we add the scaled bias back after a shift.
We use the same trick for 16-bit bpc YUV output routines.
Signed-off-by: Mans Rullgard <mans@mansr.com>
libswscale/swscale.c:2744:40: warning: to be safe all intermediate pointers in cast from ‘int16_t **’ to ‘const int16_t **’ must be ‘const’ qualified [-Wcast-qual]
libswscale/swscale.c:2745:41: warning: to be safe all intermediate pointers in cast from ‘int16_t **’ to ‘const int16_t **’ must be ‘const’ qualified [-Wcast-qual]
libswscale/swscale.c:2746:41: warning: to be safe all intermediate pointers in cast from ‘int16_t **’ to ‘const int16_t **’ must be ‘const’ qualified [-Wcast-qual]
libswscale/swscale.c:2747:78: warning: to be safe all intermediate pointers in cast from ‘int16_t **’ to ‘const int16_t **’ must be ‘const’ qualified [-Wcast-qual]
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
libswscale/x86/swscale_mmx.c:131:36: warning: to be safe all intermediate pointers in cast from ‘int16_t **’ to ‘const int16_t **’ must be ‘const’ qualified [-Wcast-qual]
libswscale/x86/swscale_mmx.c:132:37: warning: to be safe all intermediate pointers in cast from ‘int16_t **’ to ‘const int16_t **’ must be ‘const’ qualified [-Wcast-qual]
libswscale/x86/swscale_mmx.c:133:74: warning: to be safe all intermediate pointers in cast from ‘int16_t **’ to ‘const int16_t **’ must be ‘const’ qualified [-Wcast-qual]
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
As old bits are shifted out of the accumulator, they cause signed
overflows when they reach the end. Making the variable unsigned fixes
this.
Signed-off-by: Mans Rullgard <mans@mansr.com>
This was removed erroneously in
046f081b46. This define still is
necessary for getting MAP_ANONYMOUS defined on linux/glibc,
despite the define reshuffling done in that commit.
Without MAP_ANONYMOUS defined, the mprotect calls for setting the
generated mmx2 scaler code pages executable are left out, causing
crashes if that codepath is chosen.
This patch fixes scaling from 192x144 to 320x240 with
-sws_flags fast_bilinear, which crashes on linux at the
moment.
Signed-off-by: Martin Storsjö <martin@martin.st>
Although gcc guarantees 16 byte stack alignment, threads under WinXP
don't appear to be guaranteed to start stack aligned. So fix the
alignment.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
libswscale/swscale_unscaled.c:915:9: warning: new qualifiers in middle of multi-level non-const cast are unsafe
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
libswscale/swscale_unscaled.c:805:5: warning: passing argument 1 of ‘check_image_pointers’ from incompatible pointer type
libswscale/swscale_unscaled.c:774:12: note: expected ‘uint8_t **’ but argument is of type ‘const uint8_t * const*’
libswscale/swscale_unscaled.c:809:5: warning: passing argument 1 of ‘check_image_pointers’ discards qualifiers from pointer target type
libswscale/swscale_unscaled.c:774:12: note: expected ‘uint8_t **’ but argument is of type ‘uint8_t * const*’
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
libswscale/utils.c:941:13: warning: passing argument 5 of ‘initMMX2HScaler’ from incompatible pointer type
libswscale/utils.c:524:12: note: expected ‘int32_t *’ but argument is of type ‘int16_t *’
libswscale/utils.c:942:13: warning: passing argument 5 of ‘initMMX2HScaler’ from incompatible pointer type
libswscale/utils.c:524:12: note: expected ‘int32_t *’ but argument is of type ‘int16_t *’
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Altivec does unaligned reads from this buffer in
hscale_altivec_real(), and can thus read up to 16 bytes beyond
the end of the buffer. Therefore, add an extra 16 bytes of
padding at the end of the conversion buffer.
This fixes fate-lavfi-pixfmts_scale on AltiVec-enabled builds
under valgrind.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
Ronald S. Bultje
Mans Rullgard
Janne Grunau
Diego Biurrun
Michael Niedermayer
Reinhard Tartler
Martin Storsjö
John Stebbins
Sean McGovern
Themaister