avcodec/proresenc_anatoliy: Fix invalid left shift of negative number

This fixes ticket #7997 as well as the vsynth*-prores_# FATE-tests
(where * ranges over { 1, 2, 3, _lena } and # over { , _int, _444,
_444_int }).

(Given that prev_dc is in the range -0xC000..0x3FFF, no overflow can
happen upon multiplication with 2.)

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
pull/331/head
Andreas Rheinhardt 5 years ago committed by Michael Niedermayer
parent 2dc2b11fba
commit 198081efb7
  1. 2
      libavcodec/proresenc_anatoliy.c

@ -224,7 +224,7 @@ static void encode_codeword(PutBitContext *pb, int val, int codebook)
}
#define QSCALE(qmat,ind,val) ((val) / ((qmat)[ind]))
#define TO_GOLOMB(val) (((val) << 1) ^ ((val) >> 31))
#define TO_GOLOMB(val) (((val) * 2) ^ ((val) >> 31))
#define DIFF_SIGN(val, sign) (((val) >> 31) ^ (sign))
#define IS_NEGATIVE(val) ((((val) >> 31) ^ -1) + 1)
#define TO_GOLOMB2(val,sign) ((val)==0 ? 0 : ((val) << 1) + (sign))

Loading…
Cancel
Save