Main use-case is proxying avio through a foreign I/O layer and a custom
AVIO context, without losing latency and performance characteristics.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Merged from Libav commit 173b56218f.
It must be freed using avio_context_free() starting with commit
b12e4d3bb8.
Found-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
Before this commit, AVIOContext is to be freed with a plain av_free(),
which prevents us from adding any deeper structure to it.
(cherry picked from commit 99684f3ae7)
Signed-off-by: James Almer <jamrial@gmail.com>
This patch makes aviobuf work more like traditinal file IO, which is how people
think about it.
For example, in the past, aviobuf only flushed buffers until the current buffer
position, even if more data was written to it previously, and a backward seek
was used to reposition the IO context.
From now, aviobuf will keep track of the written data, so no explicit seek will
be required till the end of the buffer, or till the end of file before flushing.
This fixes at least one regression, fate-vsynth3-flv was broken if
flush_packets option was set to false, an explicit seek was removed in
4e3cc4bdd8.
Also from now on, if a forward seek in the write buffer were to cause a gap
between the already written data and the new file position, a flush will
happen.
The must_flush varable is also removed, which might have caused needless
flushes with multiple seeks whithin the write buffer. Since we know the amount
of data written to it, we will know when to flush.
Signed-off-by: Marton Balint <cus@passwd.hu>
This commit optimizes HTTP performance by reducing forward seeks, instead
favoring a read-ahead and discard on the current connection (referred to
as a short seek) for seeks that are within a TCP window's worth of data.
This improves performance because with TCP flow control, a window's worth
of data will be in the local socket buffer already or in-flight from the
sender once congestion control on the sender is fully utilizing the window.
Note: this approach doesn't attempt to differentiate from a newly opened
connection which may not be fully utilizing the window due to congestion
control vs one that is. The receiver can't get at this information, so we
assume worst case; that full window is in use (we did advertise it after all)
and that data could be in-flight
The previous behavior of closing the connection, then opening a new
with a new HTTP range value results in a massive amounts of discarded
and re-sent data when large TCP windows are used. This has been observed
on MacOS/iOS which starts with an initial window of 256KB and grows up to
1MB depending on the bandwidth-product delay.
When seeking within a window's worth of data and we close the connection,
then open a new one within the same window's worth of data, we discard
from the current offset till the end of the window. Then on the new
connection the server ends up re-sending the previous data from new
offset till the end of old window.
Example (assumes full window utilization):
TCP window size: 64KB
Position: 32KB
Forward seek position: 40KB
* (Next window)
32KB |--------------| 96KB |---------------| 160KB
*
40KB |---------------| 104KB
Re-sent amount: 96KB - 40KB = 56KB
For a real world test example, I have MP4 file of ~25MB, which ffplay
only reads ~16MB and performs 177 seeks. With current ffmpeg, this results
in 177 HTTP GETs and ~73MB worth of TCP data communication. With this
patch, ffmpeg issues 4 HTTP GETs and 3 seeks for a total of ~22MB of TCP data
communication.
To support this feature, the short seek logic in avio_seek() has been
extended to call a function to get the short seek threshold value. This
callback has been plumbed to the URLProtocol structure, which now has
infrastructure in HTTP and TCP to get the underlying receiver window size
via SO_RCVBUF. If the underlying URL and protocol don't support returning
a short seek threshold, the default s->short_seek_threshold is used
This feature has been tested on Windows 7 and MacOS/iOS. Windows support
is slightly complicated by the fact that when TCP window auto-tuning is
enabled, SO_RCVBUF doesn't report the real window size, but it does if
SO_RCVBUF was manually set (disabling auto-tuning). So we can only use
this optimization on Windows in the later case
Signed-off-by: Joel Cunningham <joel.cunningham@me.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit adds the avio_get_dyn_buf function which allows accessing
the
content of a DynBuffer without destroying it.
This is required in matroskaenc for preliminary writing (correct) mkv
headers.
Context for this change is fixing regression bug #5977.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This allows callers with avio write callbacks to get the bytestream
positions that correspond to keyframes, suitable for live streaming.
In the simplest form, a caller could expect that a header is written
to the bytestream during the avformat_write_header, and the data
output to the avio context during e.g. av_write_frame corresponds
exactly to the current packet passed in.
When combined with av_interleaved_write_frame, and with muxers that
do buffering (most muxers that do some sort of fragmenting or
clustering), the mapping from input data to bytestream positions
is nontrivial.
This allows callers to get directly information about what part
of the bytestream is what, without having to resort to assumptions
about the muxer behaviour.
One keyframe/fragment/block can still be split into multiple (if
they are larger than the aviocontext buffer), which would call
the callback with e.g. AVIO_DATA_MARKER_SYNC_POINT, followed by
AVIO_DATA_MARKER_UNKNOWN for the second time it is called with
the following data.
Signed-off-by: Martin Storsjö <martin@martin.st>
The declarations that this comment referred to were removed
in 2439f2ca8 - there is no unbuffered IO in this header now.
Signed-off-by: Martin Storsjö <martin@martin.st>
Note to maintainers: update tools
Note to maintainers: set a default whitelist for your protocol
If that makes no sense then consider to set "none" and thus require the user to specify a white-list
for sub-protocols to be opened
Note, testing and checking for missing changes is needed
Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Previous "currently size is limited" didn't give away
much in terms of useful info.
Signed-off-by: Reynaldo H. Verdejo Pinochet <reynaldo@osg.samsung.com>
This was suggested in the discussion about these functions
With this change the functions are available internally but are not
part of the public API
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Neccessary -> Necessary
formated -> formatted
thee -> the
eventhough -> even though
seperately -> separately
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
API allows protocol implementations to provide API that
allows to list directory content.
API is similar to POSIX opendir/readdir/closedir.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
avio_flush() did nothing useful for read streams. Fix it to behave as
expected, and discard the currently read buffer properly.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
It's a public function and should use the avio_ namespace
Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
C++ chokes on the definition of AVBPrint.
Including avio.h from c++ code used to work.
Fix trac ticket #3800.
Signed-off-by: Nicolas George <george@nsup.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Define positive return values as non errors and leave further meaning undefined
This allows future extensions to use these values
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Anonymous typedef structs prevent forward declaration, this
change gives the AVIOContext and AVIOInterruptCB structures
a name. These structures are now in line with other common
structures such as AVFormatContext and AVCodecContext.
Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Diego Biurrun <diego@biurrun.de>
wm4
James Almer
Anton Khirnov
Marton Balint
Joel Cunningham
softworkz
Luca Barbato
Martin Storsjö
Vittorio Giovara
Derek Buitenhuis
Michael Niedermayer
Reynaldo H. Verdejo Pinochet
Bryan Huh
Stephan Holljes
Michael Niedermayer
Mariusz Szczepańczyk
Andreas Cadhalpun
Lukasz Marek
Mark Reid
Yu Xiaolei
Nicolas George
Andrey Utkin
Diego Biurrun
Alexander Strasser
Clément Bœsch
Dale Curtis
Stefano Sabatini
Paul B Mahol