Reported as https://trac.mplayerhq.hu/ticket/2264 but have
not been able to reproduce with FFmpeg-only.
I have no idea what coded_height is used for here exactly,
so this might not be the best fix.
Fixes the following chain of events:
ff_mss12_decode_init sets coded_height while not setting height.
ff_mpv_decode_init then copies coded_height into MpegEncContext height.
This is then used by init_context_frame to allocate the data structures.
However the wmv9rects are validated/initialized based on avctx->height, not
avctx->coded_height.
Thus the decode_wmv9 function will try to decode a larger video that we
allocated data structures for, causing out-of-bounds writes.
Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
This way it won't interfere with WMV9 initialisation inside MSS2 decoder and
avplay will play it fine.
Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
The pivot has to lie between 0 and base.
Check of ==base is insufficient.
Thus replace it by a proper check.
Fixes out of array write.
Fixes bug #1531.
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Paul B Mahol <onemda@gmail.com>