At least the new videotoolbox decoder does not actually set a frame if
end_frame fails. This causes the API to return success and signals that
a picture was decoded, even though AVFrame->data[0] is NULL.
Fix this by propagating end_frame errors.
Fixes out of array read
Fixes: asan_heap-oob_394322e_138_cov_4265020547_CVPCMNL1_SVA_C.264
Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The container cropping is applied only when difference is within 16
pixels, and the smallest value between the two is chosen.
Bug-Id: 383
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Fixes out of array access
Fixes: asan_heap-oob_4d5bb0_682_cov_3124593265_Fraunhofer__a_driving_force_in_innovation__small.mp4
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Also use the frame pixel format instead of the one from the codec
context, which is more robust.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Also use the frame pixel format instead of the one from the codec
context, which is more robust.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Inconsistencies between the dimensions/pixel format of avctx and the
frame can confuse API users.
For example this can crash the demuxing_decoding example.
Back up the previous values and restore them, when decoding the next
frame. This is necessary, because these can be different between the
returned frame and the last decoded frame.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The AVFrame values are closer to the AVFrame bitmap changed instead of
the AVCodecContext values, so this should be more robust
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
It does not make sense to copy is_avc without copying this as well. This
patch should not change anything for now, but will have an effect in
later commits.
That function currently does two things -- reinitializing the DSP
contexts and setting low_delay based on the SPS values.
The former more appropriately belongs in h264_slice_header_init(), while
the latter only really makes sense in decode_slice_header().
The third call to ff_h264_set_parameter_from_sps(), done immediately
after parsing a new SPS, appears to serve no useful purpose, so it is
just dropped.
Also, drop now unneeded H264Context.cur_chroma_format_idc.
Currently, the DPB is initialized in alloc_tables() and uninitialized in
free_tables(), but those functions manage frame size-dependent
variables, so DPB management does not logically belong in there.
Since we want the init/uninit to happen exactly once per the context
lifetime, init_context()/free_context() are the proper place for this
code.
YCgCo decoding works just fine. It depends on the API user what is done
with the output. Some API users might support it, some not. Some users
might support it under certain circumstances only.
It is not the job of the decoder to print this message. If the API user
supports it, this message is extremely unhelpful.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
previously they where only updated when decoding started and would thus have
been inconsistent for a longer period of time leaving more chances for
problems
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
wm4
Sebastien Zwickert
Michael Niedermayer
Vittorio Giovara
Michael Niedermayer
Anton Khirnov
Andreas Cadhalpun
Luca Barbato
Steve Lhomme
Rainer Hochecker