From ce457bfccdd2558dc0e508f2ddf9925e1b741209 Mon Sep 17 00:00:00 2001 From: Niklas Haas Date: Wed, 11 Dec 2024 09:14:13 +0100 Subject: [PATCH] swscale/slice: fix init of 32 bpc planes In input.c and output.c and many other places, swscale follows the rule of using 15-bit intermediate if output bpc is <= 8, and 19-bit (inside int32_t) intermediate otherwise. See e.g. the comments on hyScale() on swscale_internal.h. These are also the coefficients that yuv2gbrpf32_full_X_c() is using. In contrast to this, the plane init code in slice.c (function fill_ones) is assuming that we use 35-bit intermediates (inside 64-bit integers) for this case, seemingly added by commit b4967fc71c63eae8cd96f9c46cd3e1fbd705bbf9 with no further justification. This causes a mismatch whenever the implicitly initialized plane contents leak out to the output, e.g. when converting from grayscale to RGB. Fixes: ticket #10716 Signed-off-by: Niklas Haas Sponsored-by: Sovereign Tech Fund --- libswscale/slice.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/libswscale/slice.c b/libswscale/slice.c index 2f660db4b6..44c3bd74b4 100644 --- a/libswscale/slice.c +++ b/libswscale/slice.c @@ -194,14 +194,10 @@ static void fill_ones(SwsSlice *s, int n, int bpc) for (i = 0; i < 4; ++i) { size = s->plane[i].available_lines; for (j = 0; j < size; ++j) { - if (bpc == 16) { + if (bpc >= 16) { end = (n>>1) + 1; for (k = 0; k < end; ++k) ((int32_t*)(s->plane[i].line[j]))[k] = 1<<18; - } else if (bpc == 32) { - end = (n>>2) + 1; - for (k = 0; k < end; ++k) - ((int64_t*)(s->plane[i].line[j]))[k] = 1LL<<34; } else { end = n + 1; for (k = 0; k < end; ++k)