swscale: prevent undefined behaviour in the PUTRGBA macro

For even small values of 'asrc[x]', shifting them by 24 bits or more
will cause arithmetic overflow and be caught by
GCC's undefined behaviour sanitizer.

Ensure the values do not overflow by up-casting the bracketed
expressions involving 'asrc' to uint32_t.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
release/7.1
Sean McGovern 4 months ago committed by Michael Niedermayer
parent e9e8bea2e7
commit 34b4ca8696
No known key found for this signature in database
GPG Key ID: B18E8928B3948D64
  1. 4
      libswscale/yuv2rgb.c

@ -100,9 +100,9 @@ const int *sws_getCoefficients(int colorspace)
#define PUTRGBA(dst, ysrc, asrc, i, abase) \
Y = ysrc[2 * i]; \
dst[2 * i] = r[Y] + g[Y] + b[Y] + (asrc[2 * i] << abase); \
dst[2 * i] = r[Y] + g[Y] + b[Y] + ((uint32_t)(asrc[2 * i]) << abase); \
Y = ysrc[2 * i + 1]; \
dst[2 * i + 1] = r[Y] + g[Y] + b[Y] + (asrc[2 * i + 1] << abase);
dst[2 * i + 1] = r[Y] + g[Y] + b[Y] + ((uint32_t)(asrc[2 * i + 1]) << abase);
#define PUTRGB48(dst, src, asrc, i, abase) \
Y = src[ 2 * i]; \

Loading…
Cancel
Save