This makes the code more testable as uninitialized fields are 0
and not random values from the last call
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
These entries do not correspond to VLC symbols that can be used
they do corrupt various variables like min/max bits
This also no longer assumes that there is a single non subtable
entry
Probably fixes some infinite loops too
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
For lots of static VLCs, the number of bits is not read from
VLC.bits, but rather a compile-constant that is hardcoded
at the callsite of get_vlc2(). Only VLC.table is ever used
and not using it directly is just an unnecessary indirection.
This commit adds helper functions and macros to avoid the VLC
structure when initializing VLC tables; there are 2x2 functions:
Two choices for init_sparse or from_lengths and two choices
for "overlong" initialization (as used when multiple VLCs are
initialized that share the same underlying table).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Fixes: use after free
Fixes: 62153/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MAGICYUV_fuzzer-4702814909366272
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Therefore use a proper prefix for this API, e.g.
ff_init_vlc_sparse -> ff_vlc_init_sparse
ff_free_vlc -> ff_vlc_free
INIT_VLC_LE -> VLC_INIT_LE
INIT_VLC_USE_NEW_STATIC -> VLC_INIT_USE_STATIC
(The ancient INIT_VLC_USE_STATIC has been removed
in 595324e143, so that
the NEW has been dropped.)
Finally, reorder the flags and change their values
accordingly.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Mostly taken from the documentation for ff_init_vlc_from_lengths();
also remove the documentation in vlc.c.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
In C, qualifiers for arrays are broken:
const VLC_TYPE (*foo)[2] is a pointer to an array of two const VLC_TYPE
elements and unfortunately this is not compatible with a pointer
to a const array of two VLC_TYPE, because the latter does not exist
as array types are never qualified (the qualifier applies to the base
type instead). This is the reason why get_vlc2() doesn't accept
a const VLC table despite not modifying the table at all, as
there is no automatic conversion from VLC_TYPE (*)[2] to
const VLC_TYPE (*)[2].
Fix this by using a structure VLCElem for the VLC table.
This also has the advantage of making it clear which
element is which.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
bitstream.c is currently the disjoint union of three parts:
The first part is ff_log2_run, the second part are some auxiliary
functions for the PutBits-API; and the third part is the code
for creating VLCs. This commit moves the latter into a file of its own.
This has the advantage of making one of the hacks in tableprint_vlc.h
redundant as vlc.c does not include config.h (whereas the PutBits-API
part does).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Effectively reverts eaff1aa09e
given that bitswap_32 is no longer used outside of bitstream.c
since 03008c2811.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Some of these were made possible by moving several common macros to
libavutil/macros.h.
While just at it, also improve the other headers a bit.
Reviewed-by: Martin Storsjö <martin@martin.st>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Since the MPEG-4 parser no longer initializes some MPEG-4 VLCs,
no VLC is initialized concurrently by multiple threads
(initializing static VLCs is guarded by locks and nonstatic VLCs
never posed an issue in this regard). So remove the code
in bitstream.c that only exists because of this possibility.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Scheduled for removal in 717503f716.
Also remove PutBitContext.size_in_bits which has been scheduled
for removal in e7cbbd9026.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
Clang infers from the existence of a default case that said case can be
taken. In case of libavcodec/bitstream.c said default case consisted of
an av_assert1 that evaluates to nothing in case of the ordinary assert
level. In this case (that doesn't happen) a variable wouldn't be
initialized, so Clang emitted Wsometimes-uninitialized warnings.
Solve this by making sure that the default path also initializes
the aforementioned variable.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Right now the allocated size of the VLC table of a static VLC has to
exactly match the size actually used for the VLC: If it is not enough,
abort is called; if it is more than enough, an error message is
emitted. This is no problem when one wants to initialize an individual
VLC via one of the INIT_VLC macros as one just hardcodes the needed
size. Yet it is an obstacle when one wants to initialize several VLCs
in a loop as one then needs to add an array for the sizes/offsets of
the VLC tables (unless max_depth of all arrays is one in which case
the sizes are derivable from the number of bits used).
Yet said size array is not necessary if one disables the warning for too
big buffers. The reason is that the amount of entries needed for the
table is of course generated as a byproduct of initializing the VLC.
To this end a flag that disables the warning has been added.
So one can proceed as follows:
static VLC vlcs[NUM];
static VLC_TYPE vlc_table[BUF_SIZE][2];
for (int i = 0, offset = 0; i < NUM; i++) {
vlcs[i].table = &vlc_table[offset];
vlcs[i].table_allocated = BUF_SIZE - offset;
init_vlc(); /* With INIT_VLC_STATIC_OVERLONG flag */
offset += vlcs[i].table_size;
}
Of course, BUF_SIZE should be equal to the number of entries actually
needed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
When using ff_init_vlc_sparse() to create a VLC, three input tables are
used: A table for lengths, one for codes and one for symbols; the latter
one can be omitted, then a default one will be used. These input tables
will be traversed twice, once to get the long codes (which will be put
into subtables) and once for the small codes. The long codes are then
sorted so that entries that should be in the same subtable are
contiguous.
This commit adds an alternative to ff_init_vlc_sparse():
ff_init_vlc_from_lengths(). It is based upon the observation that if
lengths, codes and symbols tables are permuted (in the same way) so that
the codes are ordered from left to right in the corresponding tree and
if said tree is complete (i.e. every non-leaf node has two children),
the codes can be easily computed from the lengths and are therefore
redundant. This means that if one initializes such a VLC with explicitly
coded lengths, codes and symbols, the codes can be avoided; and even if
one has no explicitly coded symbols, it might still be beneficial to
remove the codes even when one has to add a new symbol table, because
codes are typically longer than symbols so that the latter often fit
into a smaller type, saving space.
Furthermore, given that the codes here are by definition ordered from
left to right, it is unnecessary to sort them again; for the same
reason, one does not have to traverse the input twice. This function
proved to be faster than ff_init_vlc_sparse() whenever it has been
benchmarked.
This function is usable for static tables (they can simply be permuted
once) as well as in scenarios where the tables are naturally ordered
from left to right in the tree; the latter e.g. happens with Smacker,
Theora and several other formats.
In order to make it also usable for (static) tables with incomplete trees,
negative lengths are used to indicate that there is an open end of a
certain length.
Finally, ff_init_vlc_from_lengths() has one downside compared to
ff_init_vlc_sparse(): The latter uses tables that can be reused by
encoders. Of course, one could calculate the needed table at runtime
if one so wishes, but it is nevertheless an obstacle.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This function is so extremely simple that it is preferable to make it
inline rather than deal with all the complications arising from it being
an exported symbol.
Keep avpriv_align_put_bits() around until the next major bump to
preserve ABI compatibility.
Allocating one temporary entry more than needed was made necessary by
the COPY loop below writing an element before having checked that it
should be written at all. But given that this behaviour changed, the
need for overallocating is gone.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
If a static VLC table gets initialized a second time (or concurrently by
two threads) and if said VLC table uses symbols that have the sign bit
of VLC_TYPE (a typedef for int16_t) set, initializing the VLC fails. The
reason is that the type of the symbol in the temporary array is an
uint16_t and so comparing it to the symbol read from the VLC table will
fail, because only the lower 16bits coincide. Said failure triggers an
assert.
Reviewed-by: Lynne <dev@lynne.ee>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This is easily possible because ff_init_vlc_sparse() already transforms
both LE as well as BE codes to a normal form internally before
processing them further. This will be used in subsequent commits.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The comment referred to the INIT_VLC_USE_STATIC flag which has been
removed in 2009 in 595324e143b57a52e2329eb47b84395c70f93087; the
function it referred to was removed even earlier in commit
83422c1940 in 2008.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Fixes: out of array read
Fixes: 14563/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AGM_fuzzer-5646451545210880
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array read
Fixes: 14563/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AGM_fuzzer-5646451545210880
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Use the newly created vlc.h directly instead of including get_bits when needed.
The VLC and RL_VLC_ELEM structures are independent from the bitreader.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Commit 3a0a2f33a6 claims large performance
advantages for AV_QSORT over libc's qsort. The reason is that I suspect
that libc's qsort (at least on non LTO builds, like the typical FFmpeg config)
can't inline the comparison callback:
https://stackoverflow.com/questions/5290695/is-there-any-way-a-c-c-compiler-can-inline-a-c-callback-function.
AV_QSORT has two things going for it:
1. The guaranteed inlining of qsort itself. This yields a negligible
boost that may be ignored.
2. The more serious possibility of potentially allowing the comparison
function to be inlined - this is likely responsible for the large boosts
reported.
There is a comment explaining that this is a place that could use some
performance improvement. Thus AV_QSORT is used to achieve that.
Benchmarks deemed unnecessary due to existing claims about AV_QSORT.
Tested with FATE.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
Also fix typo found by Lou Logan:
Sacrifying -> Sacrificing
Reviewed-by: Lou Logan <lou@lrcd.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Michael Niedermayer
Andreas Rheinhardt
Paul B Mahol
Andreas Rheinhardt
Anton Khirnov
Clément Bœsch
Steinar H. Gunderson
Diego Biurrun
Alexandra Hájková
Vittorio Giovara
Lou Logan
Reimar Döffinger
Ganesh Ajjanagadde
Andreas Cadhalpun