This fixes several inconsistencies with YUV >8bit formats caused
due to missing information in the PixFmtInfo table.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The old avcodec_find_best_pix_fmt() was insufficient due to 64 pix_fmt limit.
In ffmpeg this problem has been solved long ago through avcodec_find_best_pix_fmt2()
Today libav has added a incompatible modified version of avcodec_find_best_pix_fmt2()
under the same name, thus again breaking ABI/API ...
The avcodec_find_best_pix_fmt_of_list() added in this commit here makes the libav
API available to ffmpeg users too.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The number of pixel formats outgrew the number of available bits in
the bitmask used in avcodec_find_best_pix_fmt().
avcodec_find_best_pix_fmt2() uses a PIX_FMT_NONE terminated list
of pixel formats instead.
Move the lavc/imgconvert functions and rename them as follows:
avpicture_get_size -> av_image_get_buffer_size()
avpicture_fill -> av_image_fill_arrays()
avpicture_layout -> av_image_copy_to_buffer()
The new functions have an align parameter, which allows to define the
linesize alignment assumed in the buffer (which is set or read).
The names of the functions are consistent with the lavu/samples API
(av_samples_get_buffer_size(), av_samples_fill_arrays()).
The documentation states that av_image_check_size() will return a
negative value in case of error, while the check is done on ret != 0.
Also return a proper error code rather than -1 in case the check fails.
It's never been touched or used in any way since it was added in 2003
and only works for PIX_FMT_PAL8, which makes me conclude that nobody's
ever used it.
avpicture_get_size() returns the size of buffer required for avpicture_layout.
For pseudo-paletted formats (gray8...) this size does not include the palette.
However, avpicture_layout doesn't know this and still writes the palette. Consequently,
avpicture_layout writes passed the length of the buffer. This fixes it
by fixing avpicture_layout so that it doesn't write the palette for these formats.
Signed-off-by: Matthew Einhorn <moiein2000@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Similar to avcodec_find_best_pix_fmt(), but instead only compares two destination
pix fmts and selects one of these fmts as the best dest format when converting from
a given source format. Also, as opposed to avcodec_find_best_pix_fmt() which supports
only the first 64 defined pixel formats, this supports as input any of the defined
pixel formats.
Also replaces the old function to rely on the new function.
Signed-off-by: Matthew Einhorn <moiein2000@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>