avcodec: Fix reference data type for nvdec vc1 hwaccel

I took the reference lookup code from the vp9 hwaccel where the
type is unsigned char, but for vc1, the type is signed int.

This is particularly important because the value used when there's
no reference is different (255 vs -1).

It didn't seem to break anything, but for mpeg1/2/4, this mistake
caused decode errors.
pull/272/head
Philip Langdale 7 years ago
parent fb791d2876
commit 5a0f6b099f
  1. 4
      libavcodec/nvdec_vc1.c

@ -25,13 +25,13 @@
#include "decode.h" #include "decode.h"
#include "vc1.h" #include "vc1.h"
static unsigned char get_ref_idx(AVFrame *frame) static int get_ref_idx(AVFrame *frame)
{ {
FrameDecodeData *fdd; FrameDecodeData *fdd;
NVDECFrame *cf; NVDECFrame *cf;
if (!frame || !frame->private_ref) if (!frame || !frame->private_ref)
return 255; return -1;
fdd = (FrameDecodeData*)frame->private_ref->data; fdd = (FrameDecodeData*)frame->private_ref->data;
cf = (NVDECFrame*)fdd->hwaccel_priv; cf = (NVDECFrame*)fdd->hwaccel_priv;

Loading…
Cancel
Save