avcodec/aacdec_fixed: Handle more extreem cases in noise_scale()

Its unclear if these cases have any relevance in real files

Fixes: shift exponent -2 is negative
Fixes: 14489/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AAC_FIXED_fuzzer-5681941631729664

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
pull/315/head
Michael Niedermayer 6 years ago
parent bc33c99d56
commit 3d14663f83
  1. 13
      libavcodec/aacdec_fixed.c

@ -221,10 +221,15 @@ static void noise_scale(int *coefs, int scale, int band_energy, int len)
}
else {
s = s + 32;
round = s ? 1 << (s-1) : 0;
for (i=0; i<len; i++) {
out = (int)((int64_t)((int64_t)coefs[i] * c + round) >> s);
coefs[i] = -out;
if (s > 0) {
round = 1 << (s-1);
for (i=0; i<len; i++) {
out = (int)((int64_t)((int64_t)coefs[i] * c + round) >> s);
coefs[i] = -out;
}
} else {
for (i=0; i<len; i++)
coefs[i] = -(int64_t)coefs[i] * c * (1 << -s);
}
}
}

Loading…
Cancel
Save