This brings a performance improvement when demuxing files, most of the
improvement comes from buffer pooling unbound packets.
time ffprobe -i samples/ffmpeg-bugs/trac/ticket6132/Samsung_HDR_-_Chasing_the_Light.ts -show_packets >/dev/null 2>&1
Before:
real 0m1.967s
user 0m1.471s
sys 0m0.493s
After:
real 0m1.497s
user 0m1.364s
sys 0m0.129s
Based on a patch of James Almer.
Signed-off-by: Marton Balint <cus@passwd.hu>
Put a buffer with a known fixed size into the demuxer's context instead
of allocating it separately. This also allows to remove the demuxer's
read_close()-function.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
8ffcc826 added support for muxing BlockAdditions with BlockAddID equal
to one. The restriction to BlockAddID == 1 probably resulted from
a limitation to what was needed; yet over time this led to three
occurences of "(side_data_size && additional_id == 1)". This commit
changes this by setting side_data_size to 0 if additional_id != 1.
It also stops hardcoding 1 for the value of BlockAddID to write;
but it still upholds the requirement that it is 1. See below.
Despite BlockAddId actually having a default value of 1, it is still
written, because until very recently (namely dbc50f8a) our demuxer
used a wrong default value of 0.
Furthermore, use put_ebml_binary() to write the BlockAdditional element.
(The Matroska specifications have evolved and now the BlockAddID 1 is
reserved for the codec (as described in the codec's codec mapping),
BlockMore elements with BlockAddID > 1 are now of a more
codec-independent nature and require a BlockAdditionalMapping in the
track's TrackEntry. Given that this muxer does not support writing said
BlockAdditionalMapping yet (actually, none have been defined yet), we
have to uphold the requirement that BlockAddID == 1.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
When updating the Tags at the end, the Matroska muxer would twice check
for whether (!mkv->is_live) is true, despite this code being only executed
if it is. Furthermore, a loop iterates over all the streams even when
there is no Tags element to update at all, because the check for whether
there are Tags is only performed later. This commit fixes this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avio_close_dyn_buf() has a bug: When the write pointer does not point to
the end of the written data when calling it (i.e. when one has performed
a seek back to update already written data), it would not add padding to
the end of the buffer, but to the current position, overwriting other
data; furthermore the reported size would be wrong (off by the amount of
data it has overwritten with padding).
In order not to run into this when updating already written elements or
elements for which size has only been reserved, the Matroska muxer would
first record the current position of the dynamic buffer, then seek to
the desired position, perform the update and seek back to the earlier
position.
But now that end_ebml_master_crc32() does not make use of
avio_close_dyn_buf() any more, this is no longer necessary.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The Matroska muxer uses a dynamic buffer to buffer the content of
Clusters before eventually writing them. Up until now, each time a
Cluster was written, the dynamic buffer was closed, i.e. freed; now it
is only reset, saving allocations of the AVIOContext itself, its opaque
as well as most of the reallocations of the buffer.
This is advantageous performance-wise, in particular on systems where
reallocations are slow (namely Windows). The following table shows the
decicyles for writing a frame on Linux (Ubuntu 19.10) and Windows (7)
on an x64 Haswell (to /dev/null on Linux, to stdout which is discarded
on Windows (the default values of the size and duration of clusters for
seekable output have been explicitly set in this case); in all tests,
writing CRC-32 values has been disabled in all tests; calls to the muxer's
write_packet function in write_packet() in libavformat/mux.c have been
timed; each of the following tests has been repeated 50 times):
| Windows before | Windows after | Linux before | Linux after
_________________________________________________________________
A | 979437 | 192304 | 259500 | 183320
B | 715936 | 155648 | 152786 | 130879
C | 265115 | 56034 | 78496 | 53243
D | 386224 | 80307 | 128894 | 75354
E | 21732 | 10695 | 11320 | 9801
(A is a 10.2 mb/s file with a GOP length of 2s, amounting to an average
Cluster size of about 2.5 MiB; the average Cluster size of B is 1.1 MiB;
for C it is 2.35 MiB, for D it is 0.46 MiB; for E - a file with just a
single audio track of 158kb/s resulting in a Cluster size of about 100
kB, the relative gains were the smallest, probably because of the small
Cluster size.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
If the buffer doesn't contain enough bytes when reading a stream,
fail rather than continuing on with initialized data. Caught by
Chromium fuzzeras (crbug.com/1065731).
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The hnm demuxer's context struct contained lots of fields that are
write-only variables or that are not used outside of parsing the header
and that can therefore be replaced by local variables of hnm_read_header().
This commit removes all of these from the context; the second type has
been replaced by local variables.
An AVPacket (that was initialized when reading the header and for which
dead code to unreference it existed in hnm_read_close()) is among the
removed things. Removing it allowed to remove hnm_read_close()
altogether and also removes another instance of usage of sizeof(AVPacket).
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Writing the language of WebVTT in WebM proceeded differently than the
language of all other tracks: In case no language was given, it does not
write anything instead of "und" (for undefined). Because the default
value of the Language element in WebM (that inherited it from Matroska)
is "eng" (for English), any such track will actually be flagged as
English.
Doing it this way goes back to commit 509642b4 (the commit adding
support for WebVTT) and no reason for this has been given in the commit
message or in the discussion about this patch on the mailing list; the
best I can think of is this: the WebM wiki contains "The srclang attribute
is stored as the Language sub-element." Someone unfamiliar with default
values in Matroska/WebM could interpret this as meaning that no Language
element should be written if the language is unknown. And this is wrong
and this commit changes it.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
mkv_write_track() currently has three places where it checks for whether
the current codec type is audio: One in a switch and two outside of it.
These checks can be combined by moving the code after the other two checks
inside the audio-related part of the switch.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reserving space in Matroska works by writing a Void element. And until
now this worked as follows: The current position was recorded and the
EBML ID as well as the length field written; then the new position was
recorded to know how much more to write. Afterwards the actual writing
has been performed via ffio_fill().
But it is unnecessary to explicitly use the positions (obtained via
avio_tell()) to find out how much still needs to be written, because the
length of the ID and the length field are known. So rewrite the function
to no longer use them.
Also, given that ffio_fill() uses an int parameter and given that no
current caller (and no sane future caller) will want to reserve several
GB of space, make the size parameter of put_ebml_void() itself an int.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
When the Cues are written in front of the Cluster, the muxer would seek
to the beginning (to where the Cues ought to be written) and write the
Cues; afterwards it would seek back to the end of the file only to seek
to the beginning once again to update several elements there. This
commit removes the seek to the end.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The Matroska muxer has the ability to write the Cues (the index) at the
beginning of the file (in front of the Cluster): The user inputs the
amount of space that should be reserved at the beginning of the file and
if this is sufficient, the Cues will be written there and the part of the
reserved space not used up by the Cues will be filled with a "Void"
element.
There is just one problem with this: One can not fill a single byte this
way, because said Void element is minimally two bytes long (one byte ID,
one byte length field). Up until now, if one reserved one byte more than
needed, one would run into an assert when writing the Void element.
There are two solutions for this: Error out if it happens. Or adjust the
length field of the Cues in order to ensure that the above situation
can't happen (i.e. write the length on one byte more than necessary).
The first solution is very unsatisfactory, as enough space has been
reserved. Therefore this commit implements the second solution.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
When the user opted to write the Cues at the beginning, the Cues were
simply written without checking in advance whether enough space has been
reserved for them. If it wasn't enough, the data following the space
reserved for the Cues was simply overwritten, corrupting the file.
This commit changes this by checking whether enough space has been
reserved for the Cues before outputting anything. If it isn't enough,
no Cues will be output at all and the file will be finalized normally,
yet writing the trailer will nevertheless return an error to notify
the user that his wish of having Cues at the front of the file hasn't
been fulfilled.
This change opens new usecases for this option: It is now safe to use
this option to e.g. record live streams or to use it when muxing the
output of an expensive encoding, because when the reserved space turns
out to be insufficient, one ends up with a file that just lacks Cues
but is otherwise fine.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The Matroska muxer currently assumed WavPack version 4.03 in case it was
not explicitly signalled via extradata; but following a recommendation
from David Bryant, the WavPack creator, this is changed to 4.10.
Reviewed-by: David Bryant <david@wavpack.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
It might be used by the Matroska muxer. This is also the reason why the
FATE-tests for muxing WavPack into Matroska needed to be updated: They
now write the correct version 4.07 and not 4.03 as before.
Reviewed-by: David Bryant <david@wavpack.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
mkvmerge versions 6.2 to 40.0 had a bug that made it not propagate the
WavPack extradata (containing the WavPack version) during remuxing from
a Matroska file; currently our demuxer would treat every WavPack block
encountered as invalid data (unless the WavPack stream is to be
discarded (i.e. the streams discard is >= AVDISCARD_ALL)) and try to
resync to the next level 1 element.
Luckily, the WavPack version is currently not really important; so we
fix this problem by assuming a version. David Bryant, the creator of
WavPack, recommended using version 0x410 (the most recent version) for
this. And this is what this commit does.
A FATE-test for this has been added.
Reviewed-by: David Bryant <david@wavpack.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
If the buffer doesn't contain enough bytes when reading a stream,
fail rather than continuing on with unitialized data. Caught by
Chromium fuzzers (crbug.com/1054229).
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The clli atom isn't in ISO/IEC 14496-12:2015 so the flag is marked as
experimental and the clli atom is not written by default.
The clli atom is already parsed by FFmpeg in mov.c.
Signed-off-by: Michael Bradshaw <mjbshaw@google.com>
1. When set_parameters was removed from AVOutputFormat in 2fb75019, it
was forgotten to remove the comment pertaining to it. Said comment now
appeared to apply to interleave_packet(); it is of course nonsense and
has been replaced by an accurate description.
2. The description of av_write_uncoded_frame() suggested
av_interleaved_write_frame() as a replacement if the input is not
already correctly interleaved; it also referred to said function for
details. Given that said function can't write AVFrames and that the
specifics of writing uncoded frames are explained in the description
of av_interleaved_write_uncoded_frame(), both references have been fixed.
3. Removed an outdated comment about avformat_seek_file().
Reviewed-by: Marton Balint <cus@passwd.hu>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
If ff_interleave_add_packet failed, the content of the newly created
packet new_pkt would leak.
Also, it is unnecessary to zero-initialize a packet that will be put
into av_new_packet lateron as the latter already initializes the packet.
Therefore this has been removed, too.
Reviewed-by: Marton Balint <cus@passwd.hu>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
It is unnecessary to call prepare_input_packet if there is no packet as
it doesn't do anything, except when the currently inactive code guarded
by !FF_API_COMPUTE_PKT_FIELDS2 || !FF_API_LAVF_AVCTX becomes active:
Then attempting to access pkt->stream_index will crash.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Don't use typedef struct MXFTrack {...} MXFTimecodeComponent, in
particular given the fact that MXFTrack is a type of its own.
Reviewed-by: Tomas Härdin <tjoppen@acc.umu.se>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
end_ebml_master_crc32_preliminary() has a MatroskaMuxContext as
parameter that isn't used at all. So remove it.
Furthermore it doesn't close its dynamic buffer; it just uses the
underlying buffer and therefore it only needs a pointer to the
dynamic buffer, not a pointer to a pointer.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until now, writing level 1 elements proceeded as follows: First, the
element id was written to the ordinary output AVIOContext and a dynamic
buffer was opened for the content of the level 1 element in
start_ebml_master_crc32(). Then this buffer was actually used and after it
was closed (in end_ebml_master_crc32()), the size field corresponding to
the buffer's size was written, after which the actual data was written.
This commit changes this: Nothing is written to the main AVIOContext any
more in start_ebml_master_crc32(). end_ebml_master_crc32() now writes
both the id, the length field as well as the data. This implies that
one can start a level 1 element in memory without outputting anything.
This is done to enable to test whether enough space has been reserved
for the Cues (if space has been reserved for them) before writing them.
A large duration between outputting the header and outputting the rest
could also break certain streaming usecases like the one from #8578
(which this commit fixes).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
When the Matroska muxer writes the Cues (the index), it groups index
entries with the same timestamp into the same CuePoint to save space.
But given Matroska's variable-length length fields, it either needs
to have an upper bound of the final size of the CuePoint before writing it
or the CuePoint has to be assembled in a different buffer, so that after
having assembled the CuePoint (when the real size is known), the CuePoint's
header can be written and its data copied after it.
The first of these approaches is the currently used one. This entails
finding out the number of entries in a CuePoint before starting the
CuePoint and therefore means that the list is read at least twice.
Furthermore, a worst-case upper-bound for the length of a single entry
was used, so that sometimes bytes are wasted on length fields.
This commit switches to the second approach. This is no longer more
expensive than the current approach if one only resets the dynamic
buffer used to write the CuePoint's content instead of opening a new
buffer for every CuePoint: Writing the trailer of a file with 540.000
CuePoints improved actually from 219054414 decicycles to 2164379394
decicycles (based upon 50 iterations).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Resetting a dynamic buffer means to keep the AVIOContext and the
internal buffer used by the dynamic buffer. This is done in order to
save (re)allocations when one has a workflow where one opens and closes
dynamic buffers in sequence.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until now, the Matroska muxer would allocate a structure containing
three members: The segment offset, a pointer to an array containing Cue
(index) entries and a counter for said array. It is unnecessary to
allocate it separately and it is unnecessary to contain the segment
offset in said structure, as it duplicates another field contained in
the MatroskaMuxContext. This commit implements the corresponding
changes.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
When writing the SeekHead (a form of index) at the end of the muxing
process, mkv_write_seekhead() would first seek to the position where the
SeekHead ought to be written, then write it there and seek back to the
original position afterwards. Which means: To the end of the file.
Afterwards, a seek to the beginning of the file is performed to update
further values. This of course means that the second seek in
mkv_write_seekhead() was unnecessary.
This has been changed: A new parameter was added to mkv_write_seekhead()
containing the destination for the second seek, effectively eliminating
the seek to the end of the file after writing the SeekHead.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
mkv_write_seekhead() would up until now try to seek to the position where
the SeekHead ought to be written, write the SeekHead and seek back. The
first of these seeks was checked as was writing, yet the seek back was
unchecked. Moreover the return value of mkv_write_seekhead() was unchecked
(the ordinary return value was the position where the SeekHead was written).
This commit changes this: Everything is checked. In the unseekable case
(where the first seek may nevertheless work when it happens in the buffer)
a failure at the first seek is not considered an error. In any case,
failure to seek back is an error.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
When the Matroska muxer writes an EBML ID, it calculates the length of
said ID before; and it does this as if this were a number that needs to
be encoded as EBML number: The formula used is (av_log2(id + 1) - 1) / 7
+ 1. But the constants used already contain the VINT_MARKER (the leading
bit indicating the length of the EBML number) and therefore the algorithm
used makes no sense. Instead the position of the most significant byte
set gives the desired length.
The algorithm used until now worked because EBML numbers are subject to
restrictions: If the EBML number takes up k bytes, then the bit 1 << (7
* k) is set and av_log2(id) is 7 * k. So the current algorithm produces
the correct result unless the EBML ID is of the form 7 * k - 1 because
of the "id + 1". But contrary to encoding lengths as EBML number (where
the + 1 exists to avoid the encodings reserved for unknown length),
such EBML numbers are simply forbidden as EBML IDs and as such none of
them were ever written.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This commit updates the documentation of av_read_frame() to match its
actual behaviour in several ways:
1. On success, av_read_frame() always returns refcounted packets.
2. It can handle uninitialized packets.
3. On error, it always returns blank packets.
This will allow callers to not initialize or unref unnecessarily.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Since commit e134c203 strdups of several elements of a manifest are kept
in the DASHContext; but said commit completely forgot to free these
strings again (with xmlFree()). Given that these strings are never used
at all, this commit closes this leak by reverting said commit.
This reverts commit e134c20374.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The specifications are de-facto frozen now as they've already been used in
production for years, the author has indicated reluctance on IRC to change
it further, and the only potential changes would, from what I understand,
be forward-compatible.
The classical ptr = av_realloc(ptr, size).
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Also return 0 after successfully reading a packet.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Also simply return 0 in case a packet has been successfully read.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Increasing it by 2048 entries per realloc is exessive.
Reduces memory usage, especially on long, non fragmented output.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: James Almer <jamrial@gmail.com>
Marton Balint
Michael Niedermayer
Steve Lhomme
Andreas Rheinhardt
John Rummell
Michael Bradshaw
phunkyfish
Lynne
James Almer