x86/tx_float: correctly load the transform length

The field is a standard field, yet we were loading it as if it was
a quadword. This worked for forward transforms by chance, but broke
when the transform was inverse.
checkasm couldn't catch that because we only test forward transforms,
which are identical to inverse transforms but with a different revtab.
pull/359/head
Lynne 3 years ago
parent 694545b6d5
commit 997f9bdb99
No known key found for this signature in database
GPG Key ID: A2FEA5F03F034464
  1. 2
      libavutil/x86/tx_float.asm

@ -884,7 +884,7 @@ ALIGN 16
%macro FFT_SPLIT_RADIX_FN 1
INIT_YMM %1
cglobal split_radix_fft_float, 4, 8, 16, 272, lut, out, in, len, tmp, itab, rtab, tgt
mov lenq, [lutq + AVTXContext.m]
movsxd lenq, dword [lutq + AVTXContext.m]
mov lutq, [lutq + AVTXContext.revtab]
mov tgtq, lenq

Loading…
Cancel
Save