It is only used inside libavcodec.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The AVCodecContext.get_format callback is not only used for pixel format
negotiation with the API user, but also for hwaccel init. For the
latter, it's required that some codec parameters, in particular the
codec profile, are set when the callback is invoked.
This patch removes a get_format invocation where this is not guaranteed.
The codec parameters, including the profile, are really set further
below. (The same code path that sets the profile also calls get_format
properly too.)
This just happened to work by coincidence in most cases. For example, if
the API user just copied or reused the AVStream's AVCodecContext when
decoding, the profile would be set properly. But in some cases it
fails., such as with the sample WolfensteinTwitch.mp4 on the samples
server.
Remove the redundant get_format call. Apparently it serves no purpose
anymore, although it is possible that this was different at the time it
was added in commit ffd77f94a2.
This fixes hwaccel usage for API users which do not set the profile
when setting up the AVCodecContext (which is allowed).
Deprecate older VA pixel formats (MOCO, IDCT) as it is now very unlikely
to ever be useful in the future. Only keep plain AV_PIX_FMT_VAAPI format
that is aliased to the older VLD variant.
This is an API change.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
At least the new videotoolbox decoder does not actually set a frame if
end_frame fails. This causes the API to return success and signals that
a picture was decoded, even though AVFrame->data[0] is NULL.
Fix this by propagating end_frame errors.
Fixes out of array read
Fixes: asan_heap-oob_394322e_138_cov_4265020547_CVPCMNL1_SVA_C.264
Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The container cropping is applied only when difference is within 16
pixels, and the smallest value between the two is chosen.
Bug-Id: 383
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Fixes out of array access
Fixes: asan_heap-oob_4d5bb0_682_cov_3124593265_Fraunhofer__a_driving_force_in_innovation__small.mp4
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Also use the frame pixel format instead of the one from the codec
context, which is more robust.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Also use the frame pixel format instead of the one from the codec
context, which is more robust.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Inconsistencies between the dimensions/pixel format of avctx and the
frame can confuse API users.
For example this can crash the demuxing_decoding example.
Back up the previous values and restore them, when decoding the next
frame. This is necessary, because these can be different between the
returned frame and the last decoded frame.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The AVFrame values are closer to the AVFrame bitmap changed instead of
the AVCodecContext values, so this should be more robust
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
It does not make sense to copy is_avc without copying this as well. This
patch should not change anything for now, but will have an effect in
later commits.
That function currently does two things -- reinitializing the DSP
contexts and setting low_delay based on the SPS values.
The former more appropriately belongs in h264_slice_header_init(), while
the latter only really makes sense in decode_slice_header().
The third call to ff_h264_set_parameter_from_sps(), done immediately
after parsing a new SPS, appears to serve no useful purpose, so it is
just dropped.
Also, drop now unneeded H264Context.cur_chroma_format_idc.
Currently, the DPB is initialized in alloc_tables() and uninitialized in
free_tables(), but those functions manage frame size-dependent
variables, so DPB management does not logically belong in there.
Since we want the init/uninit to happen exactly once per the context
lifetime, init_context()/free_context() are the proper place for this
code.
YCgCo decoding works just fine. It depends on the API user what is done
with the output. Some API users might support it, some not. Some users
might support it under certain circumstances only.
It is not the job of the decoder to print this message. If the API user
supports it, this message is extremely unhelpful.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Michael Niedermayer
Andreas Cadhalpun
wm4
Gwenole Beauchesne
Ronald S. Bultje
Sebastien Zwickert
Vittorio Giovara
Michael Niedermayer
Anton Khirnov
Luca Barbato
Steve Lhomme