Before this commit, the parsing of clusters mixed EBML levels by
allowing elements from different levels in a EbmlSyntax (namely
matroska_cluster_parsing). This has been changed. And the level
is now explicitly used to determine how to parse.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This commit changes how levels are handled: If the level used for
ebml_parse ends directly after an element that has been consumed, then
ebml_parse ends the level itself (and any known-length levels that end
there as well) and informs the caller via the return value; if the
current level is of unknown-length, then the level is ended as soon as
an element that is not valid on the current level, but on a higher
level is encountered (or if EOF has been encountered).
This is designed for situations where one wants to parse master elements
incrementally, i.e. not in one go via ebml_parse_nest.
The (incremental) parsing of clusters still mixes levels by using a
syntax list that contains elements from different levels and the level
is still ended manually via a call to ebml_level_end if the last cluster
was an unknown-length cluster (known-length clusters are already ended
when their last element is read), but only if the next element is a
cluster, too. A different level 1 element following an unknown-length
cluster will currently simply be presumed to be part of the earlier
cluster. Fixing this will be done in a future patch. The modifications
to matroska_parse_cluster contained in this patch are only intended not
to cause regressions.
Nevertheless, the fact that known-length levels are automatically ended
in ebml_parse when their last element has been read already fixes a bogus
error message introduced in 9326117b that was emitted when a known-length
cluster is followed by another level 1 element other than a cluster in
which case the cluster's level was not ended (which only happened when
a new cluster has been encountered) so that the length check (introduced
in 9326117b) failed for the level 1 element as it is of course not
contained in the previous cluster. Most Matroska files were affected by
this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
By linking to the syntax of the parent (i.e. the containing master
element) one can check whether an element is actually part of a higher
level in the EBML hierarchy. Knowing this is important for
unknown-length levels, because they end when an element that doesn't
belong to this, but to a higher hierarchy level is encountered.
Sometimes there are different syntaxes dealing with the same elements.
In this case it is important to use a parent that contains all the
elements at the parent level; whether this is the syntax actually used
to enter the child's level is irrelevant. This affects the list of level
1 elements (which has been used as parent for matroska_cluster, too) and
it affects recursive elements (currently only the SimpleTag), where the
non-recursive parent has to be choosen.
This is in preparation for a patch that redoes level handling.
Finally, the segment id has been added to ebml_syntax. This will enable
handling of unknown-length EBML headers.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Currently, resyncing during reading packets works as follows:
The current position is recorded, then a call to matroska_parse_cluster
is made and if said call fails, the demuxer tries to resync from the
earlier position. If the call doesn't fail, but also doesn't deliver a
packet, then this is looped.
There are two problems with this approach:
1. The Matroska file format aims to be forward-compatible; to achieve
this, a demuxer should simply ignore and skip elements it doesn't
know about. But it is not possible to reliably distinguish unknown
elements from junk. If matroska_parse_cluster encounters an unknown
element, it can therefore not simply error out; instead it returns zero
and the loop is iterated which includes an update of the position that
is intended to be used in case of errors, i.e. the element that is
skipped is not searched for level 1 element ids to resync to at all if
later calls to matroska_parse_cluster return an error.
Notice that in case that sync has been lost there can be a chain of
several unknown/possibly junk elements before an error is detected.
2. Even if a call to matroska_parse_cluster delivers a packet, this does
not mean that everything is fine. E.g. it might be that some of the
block's data is missing and that the data that was presumed to be from
the block just read actually contains the beginning of the next element.
This will only be apparent at the next call of matroska_read_packet,
which uses the (false) end of the earlier block as resync position so
that in the (not unlikely) case that the call to matroska_parse_cluster
fails, the data believed to be part of the earlier block is not searched
for a level 1 element to resync to.
To counter this, a "last known good" position is introduced. When an
element id that is known to be allowed at this position in the hierarchy
(according to the syntax currently in use for parsing) is read and some
further checks (regarding the length of the element and its containing
master element) are passed, then the beginning of the current element is
treated as a "good" position and recorded as such in the
MatroskaDemuxContext. Because of 2., only the start of the element is
treated as a "good" position, not the whole element. If an error occurs
later during parsing of clusters, the resync process starts at the last
known good position.
Given that when the header is damaged the subsequent resync never skips over
data and is therefore unaffected by both issues, the "last known good"
concept is not used there.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Since the changes to the parsing of SimpleBlocks, both ebml_parse_id and
ebml_parse_elem are only called from one place, so that it is possible
to inline these two function calls. This is done, but not completely:
ebml_parse_id still exists in a modified form. This is done in
preparation for a further patch regarding the handling of
unknown-length elements.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The earlier code set the level to zero upon seeking and after a
discontinuity although in both cases parsing (re)starts at a level 1
element.
Also set the segment's length to unkown if an error occured in order not
to drop any valid data that happens to be beyond the designated end of
the segment.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This function will be useful later to reset the status (e.g. current
level and the already parsed id).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
When an error happens, the Matroska demuxer tries to resync to level 1
elements from an earlier position onwards. If the seek to said earlier
position fails, the demuxer currently treats this as an unrecoverable
error. And that behaviour is suboptimal as said failure is nothing
unrecoverable or unexpected (when the input isn't seekable).
It is preferable to simply resync from the earliest position available
(i.e. the start of the AVIOContext's buffer) onwards if the seek failed.
Here are some scenarios that might be treated as unrecoverable errors
by the current code if the input isn't seekable. They all have in
common that the current position is so far away from the desired
position that the seek can't be fulfilled from the AVIOContext's buffer:
1. Blocks (both SimpleBlocks as well as a Block in a BlockGroup) for
which reading them as binary EBML elements succeeds, but whose parsing
triggers an error (e.g. an invalid TrackNumber). In this case the
earlier position from which resyncing begins is at the start of the block
(or even earlier).
2. BlockGroups, whose parsing fails in one of the latter elements. Just
as in 1., the start of the BlockGroup (the target of the seek) might be
so far away from the current position that it is no longer in the
buffer.
3. At the beginning of parsing a cluster, the cluster is parsed until a
SimpleBlock or a BlockGroup is encountered. So if the input is damaged
between the beginning of the cluster and the first occurrence of a
SimpleBlock/BlockGroup and if said damage makes the demuxer read/skip so
much data that the beginning of the cluster is no longer in the buffer,
demuxing will currently fail completely.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Fixes: signed integer overflow: 32 * 538976288 cannot be represented in type 'int'
Fixes: 15633/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5752273981931520
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: left shift of 133 by 24 places cannot be represented in type 'int'
Fixes: 15365/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5716153105645568
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Suggested-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Reviewed-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The dump_extra bitstream filter currently simply adds the extradata to
the packets indicated by the user without checking whether said
extradata already exists in the packets. Besides wasting space
duplicated extradata in the same packet/access unit is also forbidden
for some codecs, e.g. MPEG-2.
This check has been added to be able to use the mpeg2_qsv encoder (which
only adds the sequence headers to the first packet) in broadcast
scenarios where repeating sequence headers are required.
The check used here is not perfect: E.g. dump_extra would add the
extradata to a H.264 access unit consisting of an access unit delimiter,
SPS, PPS and slices.
Fixes#8007.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
It's better to do it before the buffers are actually created. At least in VLC
we currently don't support changing some parameters dynamically easily so we
don't use the information if it comes after the buffer are created.
Co-authored-by: James Almer <jamrial@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
It's better to do it before the buffers are actually created. At least in VLC
we currently don't support changing some parameters dynamically easily so we
don't use the information if it comes after the buffer are created.
Co-authored-by: James Almer <jamrial@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
Current version of dvbsub encoder doesn't support HD DVB subtitles. The high
resolution bitmaps are muxed into the stream but without the DDS (display definition
segment) the players asume that the DVB subtitles are in SD (720x576) resolution
which causes them to either render the subtitles too large and misplaced or don't
render them at all. By including the DDS as defined in section 7.7.1 of ETSI EN 300
743 (V1.3.1) this problem is fixed.
7.2.1 Display definition segment The display definition for a subtitle service may
be defined by the display definition segment if present in the stream. Absence of a
DDS implies that the stream is coded in accordance with EN 300 743 (V1.2.1) [5] and
that a display width of 720 pixels and a display height of 576 lines may be assumed.
https://www.etsi.org/deliver/etsi_en/300700_300799/300743/01.03.01_60/en_300743v010301p.pdf
Signed-off-by: Jernej Fijacko <mikrohard@gmail.com>
Signed-off-by: Marton Balint <cus@passwd.hu>
Fixes: out of array access
Fixes: 15540/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APNG_fuzzer-5684905029140480
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access
Fixes: 15484/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HYMT_fuzzer-5765377054736384
Fixes: 15559/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HYMT_fuzzer-5710295743332352
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: null pointer dereference
Fixes: 15464/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HYMT_fuzzer-5681391150301184
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: left shift of negative value -456
Fixes: 15561/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPC8_fuzzer-5758130404720640
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Suggested-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
merge_pmt_versions was not usable if multiple programs were present because
when it was searching for candidate streams it did not make sure that the PMT was
of the same program. This caused the streams of all programs to get merged into
a single (garbled) program.
This patch makes sure that the program number (service ID) is also matching
with the old streams when parsing the PMT making the feature useful for multi
program streams.
This change might cause issues for single program streams if the program number
changes, but I think it is acceptable because the goal of the option is to make
the parsing resilient to PID changes, and that is still working as expected.
Signed-off-by: Marton Balint <cus@passwd.hu>
Up until now, it was simply presumed that the first packet had a pts of
zero; otherwise the duration of the first chunk was wrong.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The earlier version of the webm_chunk muxer had several bugs:
1. If the first packet of an audio stream didn't have a PTS of zero,
then no chunk will be started before a packet is delivered to the
underlying Matroska/WebM muxer, i.e. the AVFormatContext used to write
these packets had a NULL as AVIOContext for output. This is behind the
crash in ticket #5752.
2. If an error happens during writing a packet, the underlyimg
Matroska/WebM muxer context is freed. This leads to a use-after-free
coupled with a double-free in webm_chunk_write_trailer (which supposes
that the underlying AVFormatContext is still valid).
3. Even when no error occurs at all, webm_chunk_write_trailer is still
buggy: After the underlying Matroska/WebM muxer has written its trailer,
ending the chunk implicitly flushes it again which is illegal at this
point.
These bugs have been fixed.
Fixes#5752.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
after a successful init if the function fails for another reason close
the codec without requiring FF_CODEC_CAP_INIT_CLEANUP which is meant to
cover init failures themselves. fixes a memory leak in those cases.
BUG=oss-fuzz:15529
Signed-off-by: James Zern <jzern@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Andreas Rheinhardt
James Almer
Steven Liu
Jun Zhao
Michael Niedermayer
Paul B Mahol
Steve Lhomme
Jernej Fijacko
Marton Balint
James Zern