avfilter/vf_psnr: Prevent integer overflow.

The 32-bit integer accumulator in MSE computation can overflow for 8-bit frame data.
(e.g., for 1080p white frame compared to a black frame can give sum of 255*255*1080*1920 > 2^32).

Signed-off-by: Neil Birkbeck <neil.birkbeck@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
pull/36/head
Neil Birkbeck 12 years ago committed by Michael Niedermayer
parent 15672e832f
commit a11c16a0b0
  1. 2
      libavfilter/vf_psnr.c

@ -91,7 +91,7 @@ void compute_images_mse(PSNRContext *s,
const uint8_t *ref_line = ref_data[c]; const uint8_t *ref_line = ref_data[c];
const int ref_linesize = ref_linesizes[c]; const int ref_linesize = ref_linesizes[c];
const int main_linesize = main_linesizes[c]; const int main_linesize = main_linesizes[c];
int m = 0; uint64_t m = 0;
for (i = 0; i < outh; i++) { for (i = 0; i < outh; i++) {
for (j = 0; j < outw; j++) for (j = 0; j < outw; j++)

Loading…
Cancel
Save