avcodec/agm: Fix overflow of signed shift

Fixes: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 15328/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AGM_fuzzer-5637545171353600

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
pull/317/head
Michael Niedermayer 5 years ago
parent 7d3581e6bb
commit 6ebbfb377f
  1. 2
      libavcodec/agm.c

@ -918,7 +918,7 @@ static void get_tree_codes(uint32_t *codes, Node *nodes, int idx, uint32_t pfx,
codes[idx] = pfx;
} else if (idx >= 0) {
get_tree_codes(codes, nodes, nodes[idx].child[0], pfx + (0 << bitpos), bitpos + 1);
get_tree_codes(codes, nodes, nodes[idx].child[1], pfx + (1 << bitpos), bitpos + 1);
get_tree_codes(codes, nodes, nodes[idx].child[1], pfx + (1U << bitpos), bitpos + 1);
}
}

Loading…
Cancel
Save