PSEUDOPAL pixel formats are not paletted, but carried a palette with the
intention of allowing code to treat unpaletted formats as paletted. The
palette simply mapped the byte values to the resulting RGB values,
making it some sort of LUT for RGB conversion.
It was used for 1 byte formats only: RGB4_BYTE, BGR4_BYTE, RGB8, BGR8,
GRAY8. The first 4 are awfully obscure, used only by some ancient bitmap
formats. The last one, GRAY8, is more common, but its treatment is
grossly incorrect. It considers full range GRAY8 only, so GRAY8 coming
from typical Y video planes was not mapped to the correct RGB values.
This cannot be fixed, because AVFrame.color_range can be freely changed
at runtime, and there is nothing to ensure the pseudo palette is
updated.
Also, nothing actually used the PSEUDOPAL palette data, except xwdenc
(trivially changed in the previous commit). All other code had to treat
it as a special case, just to ignore or to propagate palette data.
In conclusion, this was just a very strange old mechnaism that has no
real justification to exist anymore (although it may have been nice and
useful in the past). Now it's an artifact that makes the API harder to
use: API users who allocate their own pixel data have to be aware that
they need to allocate the palette, or FFmpeg will crash on them in
_some_ situations. On top of this, there was no API to allocate the
pseuo palette outside of av_frame_get_buffer().
This patch not only deprecates AV_PIX_FMT_FLAG_PSEUDOPAL, but also makes
the pseudo palette optional. Nothing accesses it anymore, though if it's
set, it's propagated. It's still allocated and initialized for
compatibility with API users that rely on this feature. But new API
users do not need to allocate it. This was an explicit goal of this
patch.
Most changes replace AV_PIX_FMT_FLAG_PSEUDOPAL with FF_PSEUDOPAL. I
first tried #ifdefing all code, but it was a mess. The FF_PSEUDOPAL
macro reduces the mess, and still allows defining FF_API_PSEUDOPAL to 0.
Passes FATE with FF_API_PSEUDOPAL enabled and disabled. In addition,
FATE passes with FF_API_PSEUDOPAL set to 1, but with allocation
functions manually changed to not allocating a palette.
Libav, for some reason, merged this as a public API function. This will
aid in future merges.
A define is left for backwards compat, just in case some person
used it, since it is in a public header.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Many of the functions from avfilter/formats can return errors, usually AVERROR(ENOMEM).
This propagates the return values.
All of these were found by using av_warn_unused_result, demonstrating its utility.
Tested with FATE. I am least sure of the changes to avfilter/filtergraph,
since I don't know what/how reduce_format is intended to behave and how it should
react to errors.
Fixes: CID 1325680, 1325679, 1325678.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Previous version Reviewed-by: Nicolas George <george@nsup.org>
Previous version Reviewed-by: Clément Bœsch <u@pkh.me>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
Theres currently no case where this could be triggered
Found-by: Daemon404
Reviewed-by: Nicolas George <george@nsup.org>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes that the GIF encoder crashes with it because
it has no palette.
And the arguments for the pseudopalette apply to gray8 as
much as to RGB8 etc.
In addition the changes required in lavfi should be needed anyway
when adding support for RGB8 etc.
Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
This new API stores useful data in a dedicated structure
and has clearly delimited init functions.
Hopefully, uses of the old API can be replaced quickly.
Add drawutils.h and drawutils.c, and use them in the pad filter.
The new functions are going to be shared by other filters.
Signed-off-by: Mans Rullgard <mans@mansr.com>
Add drawutils.h and drawutils.c, and use them in the pad filter.
The new functions are going to be shared by other filters.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Paul B Mahol
wm4
Carl Eugen Hoyos
Timo Rothenpieler
Michael Niedermayer
Derek Buitenhuis
Rodger Combs
Carl Eugen Hoyos
Ronald S. Bultje
Timothy Gu
Nicolas George
Ganesh Ajjanagadde
Hendrik Leppkes
Michael Niedermayer
Clément Bœsch
Matthieu Bouron
Anton Khirnov
Martin Storsjö
Stefano Sabatini
Reimar Döffinger
Nicolas George
Mans Rullgard
Stefano Sabatini