swresample/resample: fix invert_initial_buffer() after flush

Fixes: asan_heap-uaf_2071250_7_139.ogg
Fixes: assertion failure
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
pull/89/head
Michael Niedermayer 10 years ago
parent 080c846f59
commit f6bb2cd1b0
  1. 6
      libswresample/resample.c

@ -399,11 +399,11 @@ static int invert_initial_buffer(ResampleContext *c, AudioData *dst, const Audio
res = num - *out_sz;
*out_idx = c->filter_length + (c->index >> c->phase_shift);
*out_sz = 1 + c->filter_length * 2 - *out_idx;
*out_sz = FFMAX(*out_sz + c->filter_length,
1 + c->filter_length * 2) - *out_idx;
c->index &= c->phase_mask;
av_assert1(res > 0);
return res;
return FFMAX(res, 0);
}
struct Resampler const swri_resampler={

Loading…
Cancel
Save