avcodec/omx: fix xFramerate calculation

Integer overflow in the Q16 framerate calculation was sending
invalid values to the OMX encoder.

On the RPI4, this manifested as bitrate controls being ignored
on video streams with 60000/1001 framerates. Video streams with
30000/1001 framerates were not affected.

Signed-off-by: Aman Gupta <aman@tmm1.net>
pull/321/head^2
Aman Gupta 5 years ago
parent f4eb7d84a7
commit b022d9ba28
  1. 4
      libavcodec/omx.c

@ -473,9 +473,9 @@ static av_cold int omx_component_init(AVCodecContext *avctx, const char *role)
in_port_params.format.video.nFrameWidth = avctx->width;
in_port_params.format.video.nFrameHeight = avctx->height;
if (avctx->framerate.den > 0 && avctx->framerate.num > 0)
in_port_params.format.video.xFramerate = (1 << 16) * avctx->framerate.num / avctx->framerate.den;
in_port_params.format.video.xFramerate = (1LL << 16) * avctx->framerate.num / avctx->framerate.den;
else
in_port_params.format.video.xFramerate = (1 << 16) * avctx->time_base.den / avctx->time_base.num;
in_port_params.format.video.xFramerate = (1LL << 16) * avctx->time_base.den / avctx->time_base.num;
err = OMX_SetParameter(s->handle, OMX_IndexParamPortDefinition, &in_port_params);
CHECK(err);

Loading…
Cancel
Save