It is undefined behaviour even in cases where it works
(it works because it is only a const uint8_t* vs. uint8_t* difference).
Instead add a cbuf parameter to pass a const buffer (for writing)
as well as a parameter indicating whether we are reading or writing;
retry_transfer_wrapper() itself then uses the correct function
based upon this information.
Reviewed-by: Marton Balint <cus@passwd.hu>
Reviewed-by: Tomas Härdin <git@haerdin.se>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
They are currently non-const for reasons unknown, although
avio_write() accepts a const buffer.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
It is undefined behaviour even in cases where it works
(it works because both are pointers). Instead change
the functions involved to use the type expected by the AVIO-API
and add inline wrappers for our internal callers.
Reviewed-by: Tomas Härdin <git@haerdin.se>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Users can't make anything with its content.
Making it opaque might allow us to avoid one level of indirection.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Also simplify it and make it always log the error.
This fixes for example the image2 muxer when used with an URL which also
contains the protocol:
ffmpeg -f lavfi -i testsrc -vframes 10 -atomic_writing 1 file:out%d.png
Signed-off-by: Marton Balint <cus@passwd.hu>
transfer_func variable passed to retry_transfer_wrapper
are h->prot->url_read and h->prot->url_write functions.
These need to return EOF or other error properly.
In case of returning >= 0, url_read/url_write is retried
until error is returned.
Signed-off-by: Daniel Kucera <daniel.kucera@gmail.com>
In url_find_protocol(), proto_str is either "file" or a string
consisting of only the characters in URL_SCHEME_CHARS, which does not
include ','. Therefore the strchr(proto_str, ',') call always returns
NULL.
Note: The code was added in commit
6161c41817.
Signed-off-by: Wan-Teh Chang <wtc@google.com>
Signed-off-by: Muhammad Faiz <mfcc64@gmail.com>
The current form of the messages indicating matches in the white
or black lists seems to be a bit too much relying on context.
Make the messages more explicit.
Signed-off-by: Alexander Strasser <eclipse7@gmx.net>
This commit optimizes HTTP performance by reducing forward seeks, instead
favoring a read-ahead and discard on the current connection (referred to
as a short seek) for seeks that are within a TCP window's worth of data.
This improves performance because with TCP flow control, a window's worth
of data will be in the local socket buffer already or in-flight from the
sender once congestion control on the sender is fully utilizing the window.
Note: this approach doesn't attempt to differentiate from a newly opened
connection which may not be fully utilizing the window due to congestion
control vs one that is. The receiver can't get at this information, so we
assume worst case; that full window is in use (we did advertise it after all)
and that data could be in-flight
The previous behavior of closing the connection, then opening a new
with a new HTTP range value results in a massive amounts of discarded
and re-sent data when large TCP windows are used. This has been observed
on MacOS/iOS which starts with an initial window of 256KB and grows up to
1MB depending on the bandwidth-product delay.
When seeking within a window's worth of data and we close the connection,
then open a new one within the same window's worth of data, we discard
from the current offset till the end of the window. Then on the new
connection the server ends up re-sending the previous data from new
offset till the end of old window.
Example (assumes full window utilization):
TCP window size: 64KB
Position: 32KB
Forward seek position: 40KB
* (Next window)
32KB |--------------| 96KB |---------------| 160KB
*
40KB |---------------| 104KB
Re-sent amount: 96KB - 40KB = 56KB
For a real world test example, I have MP4 file of ~25MB, which ffplay
only reads ~16MB and performs 177 seeks. With current ffmpeg, this results
in 177 HTTP GETs and ~73MB worth of TCP data communication. With this
patch, ffmpeg issues 4 HTTP GETs and 3 seeks for a total of ~22MB of TCP data
communication.
To support this feature, the short seek logic in avio_seek() has been
extended to call a function to get the short seek threshold value. This
callback has been plumbed to the URLProtocol structure, which now has
infrastructure in HTTP and TCP to get the underlying receiver window size
via SO_RCVBUF. If the underlying URL and protocol don't support returning
a short seek threshold, the default s->short_seek_threshold is used
This feature has been tested on Windows 7 and MacOS/iOS. Windows support
is slightly complicated by the fact that when TCP window auto-tuning is
enabled, SO_RCVBUF doesn't report the real window size, but it does if
SO_RCVBUF was manually set (disabling auto-tuning). So we can only use
this optimization on Windows in the later case
Signed-off-by: Joel Cunningham <joel.cunningham@me.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Since all URLContexts have the same AVOptions, such AVOptions
will be applied on the outermost context only and removed from the
dict, while they probably make sense on all contexts.
This makes sure that rw_timeout gets propagated to the innermost
URLContext (to make sure it gets passed to the tcp protocol, when
opening a http connection for instance).
Alternatively, such matching options would be kept in the dict
and only removed after the ffurl_connect call.
Signed-off-by: Martin Storsjö <martin@martin.st>
If set non-zero, this limits duration of the retry_transfer_wrapper()
loop, thus affecting ffurl_read*(), ffurl_write(). As soon as
one single byte is successfully received/transmitted, the timer
restarts.
This has further changes by Michael Niedermayer and Martin Storsjö.
Signed-off-by: Martin Storsjö <martin@martin.st>
Instead of a linked list constructed at av_register_all(), store them
in a constant array of pointers.
Since no registration is necessary now, this removes some global state
from lavf. This will also allow the urlprotocol layer caller to limit
the available protocols in a simple and flexible way in the following
commits.
Note to maintainers: update tools
Note to maintainers: set a default whitelist for your protocol
If that makes no sense then consider to set "none" and thus require the user to specify a white-list
for sub-protocols to be opened
Note, testing and checking for missing changes is needed
Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This feature is not know much or used much AFAIK, and it might be helpfull in
exploits.
No specific case is known where it can be used in an exploit though
subsequent commits depend on this commit though
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This was suggested in the discussion about these functions
With this change the functions are available internally but are not
part of the public API
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
API allows protocol implementations to provide API that
allows to list directory content.
API is similar to POSIX opendir/readdir/closedir.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Andreas Rheinhardt
Michael Niedermayer
Anton Khirnov
James Almer
Andreas Rheinhardt
Paul B Mahol
Marton Balint
Carl Eugen Hoyos
Jun Zhao
Steven Liu
Daniel Kucera
Wan-Teh Chang
Alexander Strasser
Joel Cunningham
Yong Lei
Carl Eugen Hoyos
Martin Storsjö
Andrey Utkin
Derek Buitenhuis
Stephan Holljes
Michael Niedermayer
Mariusz Szczepańczyk
wm4
Rodger Combs
Lukasz Marek
Diego Biurrun