The spec allows at least thirty-two zero bits followed by a one to mean
2^32-1, with no constraint on the number of zeroes. The libaom
reference decoder does not match this, instead reading thirty-two zeroes
but not the following one to mean 2^32-1. These two interpretations are
incompatible and other implementations may follow one or the other.
Therefore reject thirty-two zeroes because the intended behaviour is not
clear.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Unnecessary since acf63d5350adeae551d412db699f8ca03f7e76b9;
also avoids relocations.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This avoids allocations and error checks etc. as well
as duplicate pointer lists in the CodedBitstreamFooContexts.
It also avoids casting const away for use as opaque,
as the RefStruct API supports const opaques.
The fact that some of the units are not refcounted
(i.e. they are sometimes part of an encoding context
like VAAPIEncodeH264Context) meant that CodedBitstreamUnit
still contains two pointers, one to the content
and another ownership pointer, replacing the AVBufferRef* pointer.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Turn tracing into callbacks for each syntax element, with default
callbacks to match current trace_headers behaviour for debug. Move
the construction of bit strings into the trace callback, which
simplifies all of the read and write functions.
Signed-off-by: Fei Wang <fei.w.wang@intel.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
These defines are also used in other contexts than just AVCodecContext
ones, e.g. in libavformat. Furthermore, given that these defines are
public, the AV-prefix is the right one, so deprecate (and not just move)
the FF-macros.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
These functions allow not only to read and write unsigned values,
but also to check ranges and to emit trace output which can be
beautified when processing arrays (indices like "[i]" are replaced
by their actual numbers).
Yet lots of callers actually only need something simpler:
Their range is only implicitly restricted by the amount
of bits used and they are not part of arrays, hence don't
need this beautification.
This commit adds specializations for these callers;
this is very beneficial size-wise (it reduced the size
of .text by 23312 bytes here), as a call is now cheaper.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Accept it and pass it through unchanged.
The standard requires that decoders ignore unknown metadata, and indeed
this is tested by some of the Argon coverage streams.
The current pbc might be small for an obu frame, so a new pbc is
required then parse this obu frame again. Because
CodedBitstreamAV1Context has already been updated for this obu frame, we
need to restore CodedBitstreamAV1Context, otherwise
CodedBitstreamAV1Context doesn't match this obu frame when parsing obu
frame again, e.g. CodedBitstreamAV1Context.order_hint.
$ ffmpeg -i input.ivf -c:v copy -f null -
[...]
[av1_frame_merge @ 0x558bc3d6f880] ref_order_hint[i] does not match
inferred value: 20, but should be 22.
[av1_frame_merge @ 0x558bc3d6f880] Failed to write unit 1 (type 6).
[av1_frame_merge @ 0x558bc3d6f880] Failed to write packet.
[obu @ 0x558bc3d6e040] av1_frame_merge filter failed to send output
packet
Reviewed-by: James Almer <jamrial@gmail.com>
Reviewed-by: Wenbin Chen <wenbin.chen@intel.com>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
It is no longer used.
Also rename ff_cbs_alloc_unit_content2 to ff_cbs_alloc_unit_content.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This implements the function drop_obu() as defined in Setion 6.2.1 from the
spec.
In a reading only scenario, units that belong to an operating point the
caller doesn't want should not be parsed.
Signed-off-by: James Almer <jamrial@gmail.com>
Section 6.10.1 of the AV1 spec states:
It is a requirement of bitstream conformance that the value of tg_start is
equal to the value of TileNum at the point that tile_group_obu is invoked.
It is a requirement of bitstream conformance that the value of tg_end is
greater than or equal to tg_start.
Signed-off-by: James Almer <jamrial@gmail.com>
Several cbs-functions had an unused CodedBitstreamContext parameter.
This commit removes these.
Reviewed-by: Mark Thompson <sw@jkqxz.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
There is no reason to special-case writing a value of zero as uvlc
element as the generic code is perfectly capable of doing so.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Fixes: assertion failure
Fixes: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 23264/clusterfuzz-testcase-minimized-ffmpeg_BSF_AV1_METADATA_fuzzer-6308429248593920
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
If this happens, it's a sign of parsing issues earlier in the process, or
misuse by the calling module.
Prevents writing invalid bitstreams.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
According to C99, there has to be at least one argument for every ...
in a variadic function-like macro. In practice most (all?) compilers also
allow to leave it completely out, but it is nevertheless required: In a
variadic macro "there shall be more arguments in the invocation than there
are parameters in the macro definition (excluding the ...)." (C99,
6.10.3.4).
CBS (not the framework itself, but the macros used in the
cbs_*_syntax_template.c files) relies on the compiler allowing to leave
a variadic macro argument out. This leads to warnings when compiling in
-pedantic mode, e.g. "warning: must specify at least one argument for
'...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]"
from Clang.
Most of these warnings can be easily avoided: The syntax_templates
mostly contain helper macros that expand to more complex variadic macros
and these helper macros often omit an argument for the .... Modifying
them to always expand to complex macros with an empty argument for the
... at the end fixes most of these warnings: The number of warnings went
down from 400 to 0 for cbs_av1, from 1114 to 32 for cbs_h2645, from 38 to
0 for cbs_jpeg, from 166 to 0 for cbs_mpeg2 and from 110 to 8 for cbs_vp9.
These eight remaining warnings for cbs_vp9 have been fixed by switching
to another macro in cbs_vp9_syntax_template: The fixed values for the
sync bytes as well as the trailing bits for byte-alignment are now read
via the fixed() macro (this also adds a check to ensure that trailing
bits are indeed zero as they have to be).
Reviewed-by: Mark Thompson <sw@jkqxz.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
"It is a requirement of bitstream conformance that the value returned from the leb128 parsing process is less than or equal
to (1 << 32) - 1."
Fixes: assertion failure
Fixes: 19293/clusterfuzz-testcase-minimized-ffmpeg_BSF_TRACE_HEADERS_fuzzer-5749508361420800
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
In scearios where a Temporal Unit is written right after reading it using the same
CBS context (av1_metadata, av1_frame_merge, etc), the reference frame state used
by the writer must not be the state that's the result of the reader having already
parsed the current frame in question.
This fixes writing Switch frames, and frames using short ref signaling.
Signed-off-by: James Almer <jamrial@gmail.com>
All cbs-functions to write units share a common pattern:
1. They check whether they have a write buffer (that is used to store
the unit's data until the needed size becomes known after writing the
unit when a dedicated buffer will be allocated).
2. They use this buffer for a PutBitContext.
3. The (codec-specific) writing takes place through the PutBitContext.
4. The return value is checked. AVERROR(ENOSPC) here always indicates
that the buffer was too small and leads to a reallocation of said
buffer.
5. The final buffer will be allocated and the data copied.
This commit factors this common code out in a single function in cbs.c.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This will be needed by the next commit, where packets with a single Temporal
Delimiter OBU with no size will need to be parsed
Signed-off-by: James Almer <jamrial@gmail.com>
When performing a comparison of a signed int and an unsigned int, the
signed int is first converted to an unsigned int, so that negative
values are being treated as big, positive values. This can become a
problem in an overread check, namely when an overread already happened.
So change the type of the variable containing the amount of bits that
need to be left to signed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
cbs is currently inconsistent regarding the opaque field that can be
used as a special argument to av_buffer_create in order to be used
during freeing the buffer: ff_cbs_alloc_unit_content and all the free
functions used name this parameter as if it should contain a pointer to
the unit whose content is about to be created; but both
ff_cbs_alloc_unit_content as well as ff_cbs_h264_add_sei_message
actually use a pointer to the CodedBitstreamContext as opaque. It should
actually be neither, because it is unneeded (as is evidenced by the fact
that none of the free functions use this pointer at all) and because it
ties the unit's content to the lifetime of other objects, although a
refcounted buffer is supposed to have its own lifetime that only ends
when its reference count reaches zero. This problem manifests itself in
the pointer becoming dangling.
The pointer to the unit can become dangling if another unit is added to
the fragment later as happens in the bitstream filters; in this case,
the pointer can point to the wrong unit (if the fragment's unit array
needn't be relocated) or it can point to where the array was earlier.
It can also become dangling if the unit's content is meant to survive
the resetting of the fragment it was originally read with. This applies
to the extradata of H.264 and HEVC.
The pointer to the context can become dangling if the context is closed
before the content is freed. Although this doesn't seem to happen right
now, it could happen, in particular if one uses different
CodedBitstreamContexts for in- and output.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until now, a temporary variable was used and initialized every time a
value was read in CBS; if reading turned out to be successfull, this
value was overwritten (without having ever been looked at) with the
value read if reading was successfull; on failure the variable wasn't
touched either. Therefore these initializations can be and have been
removed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
READ has already been undefined at this point; it is obviously intended
to undef WRITE.
Furthermore, leb128 (in cbs_av1) was undefined too often and
inconsistently.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The specification allows 2^32-1 to be encoded as any number of zeroes
greater than 31, followed by a one. This previously failed because the
trace code would overflow the array containing the string representation
of the bits if there were more than 63 zeroes. Fix that by splitting the
trace output into batches, and at the same time move it out of the default
path.
(While this seems likely to be a specification error, libaom does support
it so we probably should as well.)
From a test case by keval shah <skeval65@gmail.com>.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
The ISOBMFF and Matroska specs allow the last OBU in a Sample/Block to have
obu_has_size_field equal to 0.
Reviewed-by: Mark Thompson <sw@jkqxz.net>
Signed-off-by: James Almer <jamrial@gmail.com>
Mark Thompson
Michael Niedermayer
Anton Khirnov
Andreas Rheinhardt
Fei Wang
Haihao Xiang
James Almer
Jan Ekström
Andreas Rheinhardt
Carl Eugen Hoyos