Up until now, the Matroska muxer did not use the dispositions it is
given as-is; instead it by default overrode the disposition of the first
track of a kind (audio, video, subtitles) if no track of this kind has
the default disposition set. And up until recently, it also enforced
by default that no more than one track of each kind be marked as
default.
The rationale for the former is that there are lots of containers which
lack the concept of default streams, so that it is not uncommon for no
stream to be marked as default at all; the rationale for the latter was
that up until recently, it was dubious whether the Matroska specification
allowed more than one default stream for track type (e.g. mkvmerge
disallowed it). It was this point which led to the implementation of
the above mentioned behaviour inspired by mkvmerge.
Yet the Matroska specifications have changed and now explicitly allow
to set more than one track of each type as default, so that the main
reason of not using the dispositions as-is was rendered moot. Therefore
this commit changes the default to pass the disposition through.
The matroska-mpegts-remux FATE-test has been updated to still use the
old "infer" mode so that it is still covered by FATE; the
matroska-zero-length-block test has also been updated to cover
the infer_no_subs mode. The references for lots of other FATE tests
needed to be updated because of a newly added FlagDefault element with
value zero (whereas a FlagDefault with value 1 needn't be coded at all,
as it coincided with the default value of said element).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
There is a minor bug in xbm encode which adds a trailing comma at the end
of data. This isn't a big problem, but it would be nicer to be more
technically true to an array of data (by not including the last comma).
This bug fixes the output from something like this (having 4 values):
static unsigned char image_bits[] = { 0x00, 0x11, 0x22, }
to C code that looks like this instead (having 3 values):
static unsigned char image_bits[] = { 0x00, 0x11, 0x22 }
which is the intended results.
Subject: [PATCH 1/3] avcodec/xbmenc: Do not add last comma into output array
xbm outputs c arrays of data.
Including a comma at the end means there is another value to be added.
This bug fix changes something like this:
static unsigned char image_bits[] = { 0x00, 0x11, 0x22, }
to C code like this:
static unsigned char image_bits[] = { 0x00, 0x11, 0x22 }
Signed-off-by: Joe Da Silva <digital@joescat.com>
The AC3 encoder used to be a separate library called "Aften", which
got merged into libavcodec (literally, SVN commits and all).
The merge preserved as much features from the library as possible.
The code had two versions - a fixed point version and a floating
point version. FFmpeg had floating point DSP code used by other
codecs, the AC3 decoder including, so the floating-point DSP was
simply replaced with FFmpeg's own functions.
However, FFmpeg had no fixed-point audio code at that point. So
the encoder brought along its own fixed-point DSP functions,
including a fixed-point MDCT.
The fixed-point MDCT itself is trivially just a float MDCT with a
different type and each multiply being a fixed-point multiply.
So over time, it got refactored, and the FFT used for all other codecs
was templated.
Due to design decisions at the time, the fixed-point version of the
encoder operates at 16-bits of precision. Although convenient, this,
even at the time, was inadequate and inefficient. The encoder is noisy,
does not produce output comparable to the float encoder, and even
rings at higher frequencies due to the badly approximated winow function.
Enter MIPS (owned by Imagination Technologies at the time). They wanted
quick fixed-point decoding on their FPUless cores. So they contributed
patches to template the AC3 decoder so it had both a fixed-point
and a floating-point version. They also did the same for the AAC decoder.
They however, used 32-bit samples. Not 16-bits. And we did not have
32-bit fixed-point DSP functions, including an MDCT. But instead of
templating our MDCT to output 3 versions (float, 32-bit fixed and 16-bit fixed),
they simply copy-pasted their own MDCT into ours, and completely
ifdeffed our own MDCT code out if a 32-bit fixed point MDCT was selected.
This is also the status quo nowadays - 2 separate MDCTs, one which
produces floating point and 16-bit fixed point versions, and one
sort-of integrated which produces 32-bit MDCT.
MIPS weren't all that interested in encoding, so they left the encoder
as-is, and they didn't care much about the ifdeffery, mess or quality - it's
not their problem.
So the MDCT/FFT code has always been a thorn in anyone looking to clean up
code's eye.
Backstory over. Internally AC3 operates on 25-bit fixed-point coefficients.
So for the floating point version, the encoder simply runs the float MDCT,
and converts the resulting coefficients to 25-bit fixed-point, as AC3 is inherently
a fixed-point codec. For the fixed-point version, the input is 16-bit samples,
so to maximize precision the frame samples are analyzed and the highest set
bit is detected via ac3_max_msb_abs_int16(), and the coefficients are then
scaled up via ac3_lshift_int16(), so the input for the FFT is always at least 14 bits,
computed in normalize_samples(). After FFT, the coefficients are scaled up to 25 bits.
This patch simply changes the encoder to accept 32-bit samples, reusing
the already well-optimized 32-bit MDCT code, allowing us to clean up and drop
a large part of a very messy code of ours, as well as prepare for the future lavu/tx
conversion. The coefficients are simply scaled down to 25 bits during windowing,
skipping 2 separate scalings, as the hacks to extend precision are simply no longer
necessary. There's no point in running the MDCT always at 32 bits when you're
going to drop 6 bits off anyway, the headroom is plenty, and the MDCT rounds
properly.
This also makes the encoder even slightly more accurate over the float version,
as there's no coefficient conversion step necessary.
SIZE SAVINGS:
ARM32:
HARDCODED TABLES:
BASE - 10709590
DROP DSP - 10702872 - diff: -6.56KiB
DROP MDCT - 10667932 - diff: -34.12KiB - both: -40.68KiB
DROP FFT - 10336652 - diff: -323.52KiB - all: -364.20KiB
SOFTCODED TABLES:
BASE - 9685096
DROP DSP - 9678378 - diff: -6.56KiB
DROP MDCT - 9643466 - diff: -34.09KiB - both: -40.65KiB
DROP FFT - 9573918 - diff: -67.92KiB - all: -108.57KiB
ARM64:
HARDCODED TABLES:
BASE - 14641112
DROP DSP - 14633806 - diff: -7.13KiB
DROP MDCT - 14604812 - diff: -28.31KiB - both: -35.45KiB
DROP FFT - 14286826 - diff: -310.53KiB - all: -345.98KiB
SOFTCODED TABLES:
BASE - 13636238
DROP DSP - 13628932 - diff: -7.13KiB
DROP MDCT - 13599866 - diff: -28.38KiB - both: -35.52KiB
DROP FFT - 13542080 - diff: -56.43KiB - all: -91.95KiB
x86:
HARDCODED TABLES:
BASE - 12367336
DROP DSP - 12354698 - diff: -12.34KiB
DROP MDCT - 12331024 - diff: -23.12KiB - both: -35.46KiB
DROP FFT - 12029788 - diff: -294.18KiB - all: -329.64KiB
SOFTCODED TABLES:
BASE - 11358094
DROP DSP - 11345456 - diff: -12.34KiB
DROP MDCT - 11321742 - diff: -23.16KiB - both: -35.50KiB
DROP FFT - 11276946 - diff: -43.75KiB - all: -79.25KiB
PERFORMANCE (10min random s32le):
ARM32 - before - 39.9x - 0m15.046s
ARM32 - after - 28.2x - 0m21.525s
Speed: -30%
ARM64 - before - 36.1x - 0m16.637s
ARM64 - after - 36.0x - 0m16.727s
Speed: -0.5%
x86 - before - 184x - 0m3.277s
x86 - after - 190x - 0m3.187s
Speed: +3%
We now have the possibility of getting AVFrames here, and we should
not touch the muxer's codecpar after writing the header.
Results of FATE tests change as the MXF and Matroska muxers actually
write down the field/frame coding type of a stream in their
respective headers. Before this change, these values in codecpar
would only be set after the muxer was initialized. Now, the
information is also available for encoder and muxer initialization.
Additionally, reap the first rewards by being able to set the
color related encoding values based on the passed AVFrame.
The only tests that seem to have changed their results with this
change seem to be the MXF tests. There, the muxer writes the
limited/full range flag to the output container if the encoder
is not set to "unspecified".
If the average bit rate cannot be calculated, such as in the case
of streamed fragmented mp4, utilize various available parameters
in priority order.
Tests are updated where the esds or btrt or ISML manifest boxes'
output changes.
This is utilized by various media ingests to figure out the bit
rate of the content you are pushing towards it, so write it for
video, audio and subtitle tracks in case at least one nonzero value
is available. It is only mentioned for timed metadata sample
descriptions in QTFF, so limit it only to ISOBMFF (MODE_MP4) mode.
Updates the FATE tests which have their results changed due to the
20 extra bytes being written per track.
The Matroska muxer writes the Chapters early when chapters were already
available when writing the header; in this case any tags pertaining to
these chapters get written, too.
Yet if no chapters had been supplied before writing the header, Chapters
can also be written when writing the trailer if any are supplied. Tags
belonging to these chapters were up until now completely ignored.
This commit changes this: Writing the tags belonging to chapters has
been moved to mkv_write_chapters(). If mkv_write_tags() has not been
called yet (i.e. when chapters are written when writing the header),
the AVIOContext for writing the ordinary Tags element is used, but not
output, as this is left to mkv_write_tags() in order to only write one
Tags element. Yet if mkv_write_tags() has already been called,
mkv_write_chapters() will output a Tags element of its own which only
contains the tags for chapters.
When chapters are available initially, the corresponding tags will now
be the first tags in the Tags element; but the ordering of tags in Tags
is irrelevant anyway.
This commit also makes chapter_id_offset local to mkv_write_chapters()
as it is used only there and not reused at all.
Potentially writing a second Tags element means that the maximum number
of SeekHead entries had to be incremented. All the changes to FATE
result from the ensuing increase in the amount of space reserved for the
SeekHead (21 bytes more).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Several EBML Master elements for which a good upper bound of the final
length was available were nevertheless written without giving an
upper bound of the final length to start_ebml_master(), so that their
length fields were eight bytes long. This has been changed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The Matroska muxer currently only adds CuePoints in three cases:
a) For video keyframes. b) For the first audio frame in a new Cluster if
in DASH-mode. c) For subtitles. This means that ordinary Matroska audio
files won't have any Cues which impedes seeking.
This commit changes this. For every track in a file without video track
it is checked and tracked whether a Cue entry has already been added
for said track for the current Cluster. This is used to add a Cue entry
for each first packet of each track in each Cluster.
Implements #3149.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Using random values for TrackUID and FileUID (as happens when the
AVFMT_FLAG_BITEXACT flag is not set) has the obvious downside of making
the output indeterministic. This commit mitigates this by writing the
potentially random values with a fixed size of eight byte, even if their
actual values would fit into less than eight bytes. This ensures that
even in non-bitexact mode, the differences between two files generated
with the same settings are restricted to a few bytes in the header.
(Namely the SegmentUID, the TrackUIDs (in Tracks as well as when
referencing them via TagTrackUID), the FileUIDs (in Attachments as
well as in TagAttachmentUID) as well as the CRC-32 checksums of the
Info, Tracks, Attachments and Tags level-1-elements.) Without this
patch, there might be an offset/a size difference between two such
files.
The FATE-tests had to be updated because the fixed-sized UIDs are also
used in bitexact mode.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
If there are Attachments to write, the Matroska muxer currently
allocates two objects: An array that contains an entry for each
AttachedFile containing just the stream index of the corresponding
stream and the FileUID used for this AttachedFile; and a structure with
a pointer to said array and a counter for said array. These uids are
generated via code special to Attachments: It uses an AVLFG in the
normal and a sha of the attachment data in the bitexact case. (Said sha
requires an allocation, too.)
But now that an uid is generated for each stream in mkv_init(), there is
no need any more to use special code for generating the FileUIDs of
AttachedFiles: One can simply use the uid already generated for the
corresponding stream. And this makes the whole allocations of the
structures for AttachedFiles as well as the structures itself superfluous.
They have been removed.
In case AVFMT_FLAG_BITEXACT is set, the uids will be different from the
old ones which is the reason why the FATE-test lavf-mkv_attachment
needed to be updated. The old method had the drawback that two
AttachedFiles with the same data would have the same FileUID.
The new one doesn't.
Also notice that the dynamic buffer used to write the Attachments leaks
if an error happens when writing the buffer. By removing the
allocations potential sources of errors have been removed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Tags in the Matroska file format can be summarized as follows: There is
a level 1-element called Tags containing one or many Tag elements each
of which in turn contain a Targets element and one or many SimpleTags.
Each SimpleTag roughly corresponds to a single key-value pair similar to
an AVDictionaryEntry. The Targets meanwhile contains information to what
the metadata contained in the SimpleTags contained in the containing Tag
applies (i.e. to the file as a whole or to an individual track).
The Matroska muxer writes such metadata. It puts the metadata of every
stream into a Tag whose Targets makes it point to the corresponding
track. And if the output is seekable, then it also adds another Tag for
each track whose Targets corresponds to the track and where it reserves
space in a SimpleTag to write the duration at the end of the muxing
process into.
Yet there is no reason to write two Tag elements for a track and a few
bytes (typically 24 bytes per track) can be saved by adding the duration
SimpleTag to the other Tag of the same track (if it exists).
FATE has been updated because the output files changed. (Tests that
write to unseekable output (pipes) needn't be updated (no duration tag
has ever been written for them) and the same applies to tests without
further metadata.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until e7ddafd5, the Matroska muxer wrote two SeekHeads: One at the
beginning referencing the main level 1 elements (i.e. not the Clusters)
and one at the end, referencing the Clusters. This second SeekHead was
useless and has therefore been removed. Yet the SeekHead-related
functions and structures are still geared towards this usecase: They
are built around an allocated array of variable size that gets
reallocated every time an element is added to it although the maximum
number of Seek entries is a small compile-time constant, so that one should
rather include the array in the SeekHead structure itself; and said
structure should be contained in the MatroskaMuxContext instead of being
allocated separately.
The earlier code reserved space for a SeekHead with 10 entries, although
we currently write at most 6. Reducing said number implied that every
Matroska/Webm file will be 84 bytes smaller and required to adapt
several FATE tests; furthermore, the reserved amount overestimated the
amount needed for for the SeekHead's length field and how many bytes
need to be reserved to write a EBML Void element, bringing the total
reduction to 89 bytes.
This also fixes a potential segfault: If !mkv->is_live and if the
AVIOContext is initially unseekable when writing the header, the
SeekHead is already written when writing the header and this used to
free the SeekHead-related structures that have been allocated. But if
the AVIOContext happens to be seekable when writing the trailer, it will
be attempted to write the SeekHead again which will lead to segfaults
because the corresponding structures have already been freed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The standard does not seem to require the counter to be zero based, but some
checker tools (MyriadBits MXFInspect, Interra Baton) have validations against 0
start...
Fixes ticket #6781.
Signed-off-by: Marton Balint <cus@passwd.hu>
Fixes: fate-fitsdec-bitpix-64
Possibly Fixes: -nan is outside the range of representable values of type 'unsigned short'
Possibly Fixes: 17769/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FITS_fuzzer-5678314672357376
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The packet counting based approach caused excessive sdt/pat/pmt for VBR, so
let's use a timestamp based approach instead similar to how we emit PCRs.
SDT/PAT/PMT period should be consistent for both VBR and CBR from now on.
Also change the type of sdt_period and pat_period to AV_OPT_TYPE_DURATION so no
floating point math is necessary.
Fixes ticket #3714.
Signed-off-by: Marton Balint <cus@passwd.hu>
Up until now, the length field of most level 1 elements has been written
using eight bytes, although it is known in advance how much space the
content of said elements will take up so that it would be possible to
determine the minimal amount of bytes for the length field. This
commit changes this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
Up until now the EBML Header length field has been written with eight
bytes, although the EBML Header is always so small that only one byte
is needed for it. This patch saves seven bytes for every Matroska/Webm
file.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
write_tmcd allows tmcd track to be created with any mode but in
mov_write_header, index for first tmcd track is only set for modes
MP4 or MOV, causing a crash if tmcd creation is attempted with other
modes.
ISMV lacks any sort of edit list support, as well as tfxd is
effectively the PTS of the fragment for most intents and purposes.
Thus, if b-frames are requested without negative CTS offsets you
end up with N frames' worth of delay (tfxd PTS plus the CTS offset
of the first sample). Negative CTS offsets enable the first sample
to have CTS=DTS, and thus a/v desync due to b-frame reorder delay
is avoided.
Andreas Rheinhardt
Paul B Mahol
Limin Wang
Jose Da Silva
Marton Balint
Lynne
Jan Ekström
Andreas Rheinhardt
Michael Niedermayer
James Almer
Gyan Doshi
Diego Biurrun
Baptiste Coudurier