Fixes a warning with clang:
libavutil/imgutils.c:314:16: warning: absolute value function 'abs'
given an argument of type 'ptrdiff_t' (aka 'long') but has
parameter of type 'int' which may cause truncation of value
This is strongly based on code by Marton Balint, and depends on the previous commit
Fixes: Timeout
Fixes: 11502/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WCMV_fuzzer-5664893810769920
Before: Executed clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WCMV_fuzzer-5664893810769920 in 11209 ms
After: Executed clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WCMV_fuzzer-5664893810769920 in 4104 ms
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Marton Balint <cus@passwd.hu>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
PSEUDOPAL pixel formats are not paletted, but carried a palette with the
intention of allowing code to treat unpaletted formats as paletted. The
palette simply mapped the byte values to the resulting RGB values,
making it some sort of LUT for RGB conversion.
It was used for 1 byte formats only: RGB4_BYTE, BGR4_BYTE, RGB8, BGR8,
GRAY8. The first 4 are awfully obscure, used only by some ancient bitmap
formats. The last one, GRAY8, is more common, but its treatment is
grossly incorrect. It considers full range GRAY8 only, so GRAY8 coming
from typical Y video planes was not mapped to the correct RGB values.
This cannot be fixed, because AVFrame.color_range can be freely changed
at runtime, and there is nothing to ensure the pseudo palette is
updated.
Also, nothing actually used the PSEUDOPAL palette data, except xwdenc
(trivially changed in the previous commit). All other code had to treat
it as a special case, just to ignore or to propagate palette data.
In conclusion, this was just a very strange old mechnaism that has no
real justification to exist anymore (although it may have been nice and
useful in the past). Now it's an artifact that makes the API harder to
use: API users who allocate their own pixel data have to be aware that
they need to allocate the palette, or FFmpeg will crash on them in
_some_ situations. On top of this, there was no API to allocate the
pseuo palette outside of av_frame_get_buffer().
This patch not only deprecates AV_PIX_FMT_FLAG_PSEUDOPAL, but also makes
the pseudo palette optional. Nothing accesses it anymore, though if it's
set, it's propagated. It's still allocated and initialized for
compatibility with API users that rely on this feature. But new API
users do not need to allocate it. This was an explicit goal of this
patch.
Most changes replace AV_PIX_FMT_FLAG_PSEUDOPAL with FF_PSEUDOPAL. I
first tried #ifdefing all code, but it was a mess. The FF_PSEUDOPAL
macro reduces the mess, and still allows defining FF_API_PSEUDOPAL to 0.
Passes FATE with FF_API_PSEUDOPAL enabled and disabled. In addition,
FATE passes with FF_API_PSEUDOPAL set to 1, but with allocation
functions manually changed to not allocating a palette.
Black isn't always just memset(ptr, 0, size). Limited YUV in particular
requires relatively non-obvious values, and filling a frame with
repeating 0 bytes is disallowed in some contexts. With component sizes
larger than 8 or packed YUV, this can become relatively complicated. So
having a generic function for this seems helpful.
In order to handle the complex cases in a generic way without destroying
performance, this code attempts to compute a black pixel, and then uses
that value to clear the image data quickly by using a function like
memset.
Common cases like yuv410p10 or rgba can't be handled with a simple
memset, so there is some code to fill memory with 2/4/8 byte patterns.
For the remaining cases, a generic slow fallback is used.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Merged from Libav commit 45df7adc1d.
Black isn't always just memset(ptr, 0, size). Limited YUV in particular
requires relatively non-obvious values, and filling a frame with
repeating 0 bytes is disallowed in some contexts. With component sizes
larger than 8 or packed YUV, this can become relatively complicated. So
having a generic function for this seems helpful.
In order to handle the complex cases in a generic way without destroying
performance, this code attempts to compute a black pixel, and then uses
that value to clear the image data quickly by using a function like
memset.
Common cases like yuv410p10 or rgba can't be handled with a simple
memset, so there is some code to fill memory with 2/4/8 byte patterns.
For the remaining cases, a generic slow fallback is used.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
This was suggested by wm4 and stefano.
After this patch using align=1 the size used by various functions would not
contain padding, while the palette would be aligned at align>1
This patch makes it required to use align>=4 if the palette is to be accessed
as uint32
As a side-effect It fixes storing pal8 in nut with odd with&height
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This allows disabling the alignment by using a compact buffer
Reviewed-by: Stefano Sabatini <stefasab@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Libav, for some reason, merged this as a public API function. This will
aid in future merges.
A define is left for backwards compat, just in case some person
used it, since it is in a public header.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
av_image_fill_pointers always aligns the palette, but the padding
bytes don't (and can't) get initialized in av_image_copy.
Thus initialize them in av_image_alloc.
This fixes 'Syscall param write(buf) points to uninitialised byte(s)'
valgrind warnings.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
These could trigger assert failures previously
Found-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Move the lavc/imgconvert functions and rename them as follows:
avpicture_get_size -> av_image_get_buffer_size()
avpicture_fill -> av_image_fill_arrays()
avpicture_layout -> av_image_copy_to_buffer()
The new functions have an align parameter, which allows to define the
linesize alignment assumed in the buffer (which is set or read).
The names of the functions are consistent with the lavu/samples API
(av_samples_get_buffer_size(), av_samples_fill_arrays()).
A redundant check has been dropped from av_image_fill_arrays().
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Move the lavc/imgconvert functions and rename them as follows:
avpicture_get_size -> av_image_get_buffer_size()
avpicture_fill -> av_image_fill_arrays()
avpicture_layout -> av_image_copy_to_buffer()
The new functions have an align parameter, which allows to define the
linesize alignment assumed in the buffer (which is set or read).
The names of the functions are consistent with the lavu/samples API
(av_samples_get_buffer_size(), av_samples_fill_arrays()).
This allows simd optimized routines to work in steps of 8 pixels
without going over the linesize. (this matters for yuv->rgb24 for example)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Make it a subcase of the general algorithm used for the non-bitstream
case. Simplify, and make av_image_get_linesize() and
av_image_fill_linesizes() correctly return the right value when plane
!= 0.
In particular fix a crash occurring with:
-vf format=monow,showinfo,format=monow.
Marton Balint
Carl Eugen Hoyos
Michael Niedermayer
wm4
Diego Biurrun
Anton Khirnov
Derek Buitenhuis
Clément Bœsch
Timothy Gu
Vittorio Giovara
Michael Niedermayer
Andreas Cadhalpun
Stefano Sabatini
Justin Ruggles
Carl Eugen Hoyos
Paul B Mahol
Martin Storsjö
jamal
Thierry Foucu
Mans Rullgard
Stefano Sabatini