vulkan_h264: reject end_frame being called without valid session parameters

When seeking through MBAFF-coded H264, this can happen. Decoding calls end_frame
without calling start_frame. We are unable to decode this, as no frame
state has been set.

Happens for both VAAPI and Vulkan. Could be an issue elsewhere, hence
the individual commit.
pull/389/head
Lynne 2 years ago
parent 160a415e22
commit f701f088f4
No known key found for this signature in database
GPG Key ID: A2FEA5F03F034464
  1. 4
      libavcodec/vulkan_h264.c

@ -508,10 +508,14 @@ static int vk_h264_end_frame(AVCodecContext *avctx)
const H264Context *h = avctx->priv_data;
H264Picture *pic = h->cur_pic_ptr;
H264VulkanDecodePicture *hp = pic->hwaccel_picture_private;
FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data;
FFVulkanDecodePicture *vp = &hp->vp;
FFVulkanDecodePicture *rvp[H264_MAX_PICTURE_COUNT] = { 0 };
AVFrame *rav[H264_MAX_PICTURE_COUNT] = { 0 };
if (!dec->session_params)
return AVERROR(EINVAL);
for (int i = 0; i < vp->decode_info.referenceSlotCount; i++) {
H264Picture *rp = hp->ref_src[i];
H264VulkanDecodePicture *rhp = rp->hwaccel_picture_private;

Loading…
Cancel
Save