avcodec/h263: Fix global-buffer-overflow with noout flag2 set

h263_get_motion_length() forgot to take an absolute value;
as a consequence, a negative index was used to access an array.
This leads to potential crashes, but mostly it just accesses what
is to the left of ff_mvtab (unless one uses ASAN), thereby defeating
the purpose of the AV_CODEC_FLAG2_NO_OUTPUT because the sizes of
the returned packets differ from the sizes the encoder would actually
have produced.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
pull/375/head
Andreas Rheinhardt 3 years ago
parent 27c9300027
commit 9207dc3b0d
  1. 7
      libavcodec/h263.h

@ -100,15 +100,16 @@ void ff_h263_encode_motion(PutBitContext *pb, int val, int f_code);
static inline int h263_get_motion_length(int val, int f_code){ static inline int h263_get_motion_length(int val, int f_code){
int l, bit_size, code; int bit_size, code, sign;
if (val == 0) { if (val == 0) {
return 1; /* ff_mvtab[0][1] */ return 1; /* ff_mvtab[0][1] */
} else { } else {
bit_size = f_code - 1; bit_size = f_code - 1;
/* modulo encoding */ /* modulo encoding */
l= INT_BIT - 6 - bit_size; val = sign_extend(val, 6 + bit_size);
val = (val<<l)>>l; sign = val >> 31;
val = (val ^ sign) - sign; /* val = FFABS(val) */
val--; val--;
code = (val >> bit_size) + 1; code = (val >> bit_size) + 1;

Loading…
Cancel
Save