avcodec/mpegvideoenc: fix undefined negative left shift

This should fix the first undefined behavior reported in:
https://trac.ffmpeg.org/ticket/4727.

I can't reproduce the runtime behavior reported in the ticket, hence I
can't confirm that this actually fixes the exact issue reported in the
ticket.

Regardless, I can confirm that this is a genuine issue, and that
negative shifts can (and do) occur, fixed by this.

Tested with FATE.

Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
pull/154/head
Ganesh Ajjanagadde 9 years ago committed by Michael Niedermayer
parent 25d3013e36
commit 3be27e07d3
  1. 2
      libavcodec/mpegvideo_enc.c

@ -148,7 +148,7 @@ void ff_convert_matrix(MpegEncContext *s, int (*qmat)[64],
qmat16[qscale][0][i] == 128 * 256)
qmat16[qscale][0][i] = 128 * 256 - 1;
qmat16[qscale][1][i] =
ROUNDED_DIV(bias << (16 - QUANT_BIAS_SHIFT),
ROUNDED_DIV(bias * (1<<(16 - QUANT_BIAS_SHIFT)),
qmat16[qscale][0][i]);
}
}

Loading…
Cancel
Save