The encoders such as libx264 support different QPs offset for different MBs,
it makes possible for ROI-based encoding. It makes sense to add support
within ffmpeg to generate/accept ROI infos and pass into encoders.
Typical usage: After AVFrame is decoded, a ffmpeg filter or user's code
generates ROI info for that frame, and the encoder finally does the
ROI-based encoding.
The ROI info is maintained as side data of AVFrame.
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
The dynamic metadata contains data for color volume transform -
application 4 of SMPTE 2094-40:2016 standard. The data comes from
HEVC in the SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Add error handle if av_image_fill_pointers fail.
Signed-off-by: Jun Zhao <mypopydev@gmail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Create SMPTE ST 12-1 timecodes based on H.264 SEI picture timing
info.
For framerates > 30 FPS, the field flag is used in conjunction with
pairs of frames which contain the same frame timestamp in S12M.
Ensure the field is properly set per the spec.
PSEUDOPAL pixel formats are not paletted, but carried a palette with the
intention of allowing code to treat unpaletted formats as paletted. The
palette simply mapped the byte values to the resulting RGB values,
making it some sort of LUT for RGB conversion.
It was used for 1 byte formats only: RGB4_BYTE, BGR4_BYTE, RGB8, BGR8,
GRAY8. The first 4 are awfully obscure, used only by some ancient bitmap
formats. The last one, GRAY8, is more common, but its treatment is
grossly incorrect. It considers full range GRAY8 only, so GRAY8 coming
from typical Y video planes was not mapped to the correct RGB values.
This cannot be fixed, because AVFrame.color_range can be freely changed
at runtime, and there is nothing to ensure the pseudo palette is
updated.
Also, nothing actually used the PSEUDOPAL palette data, except xwdenc
(trivially changed in the previous commit). All other code had to treat
it as a special case, just to ignore or to propagate palette data.
In conclusion, this was just a very strange old mechnaism that has no
real justification to exist anymore (although it may have been nice and
useful in the past). Now it's an artifact that makes the API harder to
use: API users who allocate their own pixel data have to be aware that
they need to allocate the palette, or FFmpeg will crash on them in
_some_ situations. On top of this, there was no API to allocate the
pseuo palette outside of av_frame_get_buffer().
This patch not only deprecates AV_PIX_FMT_FLAG_PSEUDOPAL, but also makes
the pseudo palette optional. Nothing accesses it anymore, though if it's
set, it's propagated. It's still allocated and initialized for
compatibility with API users that rely on this feature. But new API
users do not need to allocate it. This was an explicit goal of this
patch.
Most changes replace AV_PIX_FMT_FLAG_PSEUDOPAL with FF_PSEUDOPAL. I
first tried #ifdefing all code, but it was a mess. The FF_PSEUDOPAL
macro reduces the mess, and still allows defining FF_API_PSEUDOPAL to 0.
Passes FATE with FF_API_PSEUDOPAL enabled and disabled. In addition,
FATE passes with FF_API_PSEUDOPAL set to 1, but with allocation
functions manually changed to not allocating a palette.
This adds a way for an API user to transfer QP data and metadata without
having to keep the reference to AVFrame, and without having to
explicitly care about QP APIs. It might also provide a way to finally
remove the deprecated QP related fields. In the end, the QP table should
be handled in a very similar way to e.g. AV_FRAME_DATA_MOTION_VECTORS.
There are two side data types, because I didn't care about having to
repack the QP data so the table and the metadata are in a single
AVBufferRef. Otherwise it would have either required a copy on decoding
(extra slowdown for something as obscure as the QP data), or would have
required making intrusive changes to the codecs which support export of
this data.
The new side data types are added under deprecation guards, because I
don't intend to change the status of the QP export as being deprecated
(as it was before this patch too).
Everything related to the QP data is deprecated, with qp_table_buf being
an inconsistent exception. Some parts were under the deprecation guards,
some not. It probably didn't even compile.
This gives FFmpeg libs a field that they can freely and safely use.
Avoiding the need of wrapping of a users opaque_ref field and its issues.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The fields can be accessed directly, so these are not needed anymore.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
Found a case where we use size==0, the other related commits
remain needed, and should be sufficient to fix the original issue
This reverts commit 7e4f32f4e4.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The size 0 special case causes side data to be created which is
different and a special case if for any reasons size = 0 is passed
Fixes: multiple runtime error: null pointer passed as argument 1, which is declared to never be null
Fixes: 653/clusterfuzz-testcase-5773837415219200
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This is an extended version of the AVFrame.opaque field, which can be
used to attach arbitrary user information to an AVFrame.
The usefulness of the opaque field is rather limited, because it can
store only up to 32 bits of information (or 64 bit on 64 bit systems).
It's not possible to set this field to a memory allocation, because
there is no way to deallocate it correctly.
The opaque_ref field circumvents this by letting the user set an
AVBuffer, which makes the user data refcounted.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Merges Libav commit 04f3bd3496.
This is an extended version of the AVFrame.opaque field, which can be
used to attach arbitrary user information to an AVFrame.
The usefulness of the opaque field is rather limited, because it can
store only up to 32 bits of information (or 64 bit on 64 bit systems).
It's not possible to set this field to a memory allocation, because
there is no way to deallocate it correctly.
The opaque_ref field circumvents this by letting the user set an
AVBuffer, which makes the user data refcounted.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
I wonder how unknown layouts ever worked without this?
Reviewed-by: Nicolas George <george@nsup.org>
Reviewed-by: Hendrik Leppkes <h.leppkes@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Marton Balint <cus@passwd.hu>
The accessors are needed only from outside libavutil.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
Currently it's exported as AVFrame.pkt_pts, which is also the only use
for that field. The reason it is done like this is that lavc used to
export various codec-specific "timing" information in AVFrame.pts, which
is not done anymore.
Since it is confusing to the callers to have a separate field which is
used only for decoder timestamps and nothing else, deprecate pkt_pts and
use just AVFrame.pts everywhere.
Libav, for some reason, merged this as a public API function. This will
aid in future merges.
A define is left for backwards compat, just in case some person
used it, since it is in a public header.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Add names for recently added enums to av_frame_side_data_name.
Signed-off-by: Neil Birkbeck <neil.birkbeck@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
These field are difficult to interpret, and are provided by a single
encoder (mpegvideoenc). In general they do not belong to a structure
containing raw data only, so remove them from AVFrame.
Mpegvideoenc now uses a private field in Picture for its internal
computations.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Fixes: af94b3a3d26586c08f557cafe8246251_signal_sigsegv_7ffff713351a_343_XFMode.ASF with 2097152 alloc limit
Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The error path frees all side data, but forgets to reset the side data
count. This can blow up later in av_frame_unref() and free_side_data().
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The reasoning behind this addition is that various third party
applications are interested in getting some motion information out of a
video "for free" when it is available.
It was considered to export other information as well (such as the intra
information about the block, or the quantization) but the structure
might have ended up into a half full-generic, half full of codec
specific cruft. If more information is necessary, it should either be
added in the "flags" field of the AVMotionVector structure, or in
another side-data.
This commit also includes an example exporting them in a CSV stream.
Guo, Yejun
Mohammad Izadi
Jun Zhao
Devin Heitmueller
Luca Barbato
Vittorio Giovara
wm4
Rostislav Pehlivanov
Michael Niedermayer
James Almer
Steve Lhomme
Anton Khirnov
Marton Balint
Sasi Inguva
Diego Biurrun
Derek Buitenhuis
Neil Birkbeck
Clément Bœsch
Clément Bœsch
Hendrik Leppkes
Andreas Cadhalpun
Ronald S. Bultje
Reimar Döffinger