avcodec/rka: Avoid undefined left shift

Fixes: left shift of 34136248 by 6 places cannot be represented in type 'int'
Fixes: 58429/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RKA_fuzzer-5692211592560640

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1ee303f1e1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
release/6.0
Michael Niedermayer 2 years ago
parent a66733a8c9
commit 819e9068af
No known key found for this signature in database
GPG Key ID: B18E8928B3948D64
  1. 2
      libavcodec/rka.c

@ -750,7 +750,7 @@ static int decode_filter(RKAContext *s, ChContext *ctx, ACoder *ac, int off, uns
}
if (ctx->cmode2 != 0) {
int sum = 0;
for (int i = (m << 6) / split; i > 0; i = i >> 1)
for (int i = (signed)((unsigned)m << 6) / split; i > 0; i = i >> 1)
sum++;
sum = sum - (ctx->cmode2 + 7);
ctx->cmode = FFMAX(sum, tab[ctx->cmode2]);

Loading…
Cancel
Save