Using Range allows for getting the full file size from the
Content-Range header in the response, even if the server sends
back the response using chunked Transfer-Encoding, which does not
allow using Content-Length.
When Transfer-Encoding:chunked is used, the client must ignore a
Content-Length header, if present. However, it should not ignore a
Content-Range header, which also includes the full size of the
entity.
av_dict_set() with AV_DICT_DONT_STRDUP_VAL takes ownership
of the string it is passed to as val; this includes freeing it
on error.
Fixes Coverity issue #1497468.
Reviewed-by: Eran Kornblau <eran.kornblau@kaltura.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
following 625ea2d, redirect caching is performed according to the http
response headers, there's no need to have it as an option -
always start from the original uri, and apply any redirects according
to the redirect_cache dictionary.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
add a dictionary that maps "src_url" -> "expiry;dst_url", the dictionary
is checked before issuing an http request, and updated after getting a
3xx redirect response.
the cache expiry is determined according to the following (in desc
priority) -
1. Expires header
2. Cache-Control containing no-cache/no-store (disables caching)
3. Cache-Control s-maxage/max-age
4. Http codes 301/308 are cached indefinitely, other codes are not
cached
added "cache_redirect" option to http.
when enabled, requests issued after seek will use the latest redirected
url.
when disabled, each call to seek will revert to the original url that
was sent to http_open.
currently, the default remains 'enabled', until the next major
libavformat bump, where it will change to 'disabled'.
path1 and sanitized_path are both MAX_URL_SIZE bytes long, yet the latter is
copied from the former with the addition of one extra character.
Should fix a -Wformat-truncation warning.
Signed-off-by: James Almer <jamrial@gmail.com>
In 45bfe8b838, short_seek_threshold was removed
from the public AVIO struct. Although this option was private and not intended
to be used by public API users, it was nonetheless, because it provided functionality
that could otherwise not be gained via public API.
This was especially important for networked I/O like HTTP, where the internal
size for lavf could be way to small depending on the specifics of a user's
usecase, such as reading interlavd media files from cloud storage.
Add an AVOption to make this functionality accessible to the HTTP client.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Add the "http_proxy" option and its handling to the "tls" protocol,
pass the option from the "https" protocol.
The "https" protocol already defines the "http_proxy" command line
option, like the "http" protocol does. The "http" protocol properly
honors that command line option in addition to the environment
variable. The "https" protocol doesn't, because the proxy is
evaluated in the underlying "tls" protocol, which doesn't have this
option, and thus only handles the environment variable, which it
has access to.
Fixes#7223.
Signed-off-by: Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Marton Balint <cus@passwd.hu>
When the deprecated option "user-agent" was set to something different
than its default value, said option would always precede and overwrite
the ordinary user_agent option (regardless of whether it was explicitly
set) which leads to a leak of the user_agent option (which has a default
value, so the leak happens always).
Fix this by setting the same destination for both options; the last
option applied wins then.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
FFmpeg does not support POST, so there is no difference between a
308 and 301 request (see [RFC7538] section 3).
Signed-off-by: Josh Dekker <josh@itanimul.li>
Add 2 new options:
- reconnect_on_http_error - a list of http status codes that should be
retried. the list can contain explicit status codes / the strings
4xx/5xx.
- reconnect_on_network_error - reconnects on arbitrary errors during
connect, e.g. ECONNRESET/ETIMEDOUT
the retry employs the same exponential backoff logic as the existing
reconnect/reconnect_at_eof flags.
related tickets:
https://trac.ffmpeg.org/ticket/6066https://trac.ffmpeg.org/ticket/7768
Signed-off-by: Marton Balint <cus@passwd.hu>
When the check was added (in 3668701f96, in 2015), some IO
functions returned 0 on EOF (in particular, the TCP protocol
did, but the TLS protocol returned AVERROR_EOF). Since
0e1f771d22 in 2017, the TCP protocol also returns AVERROR_EOF
instead of 0, making the check for premature end never have the
intended effect.
Signed-off-by: Martin Storsjö <martin@martin.st>
Some real-world sites use an authorization header with a bearer token; when
combined with lengthy request parameters to identify the video segment,
it's rather trivial these days to have a request body of more than 4k bytes.
MAX_URL_SIZE is hard-coded to 4k bytes in libavformat/internal.h, and
HTTP_HEADERS_SIZE is 4k as well in libavformat/http.h, so this patch increases
the buffer size to 8k, as that is the default request body limit in Apache, and
most other httpds seem to support at least as much, if not more.
Signed-off-by: Marton Balint <cus@passwd.hu>
v2: Use s->buffer for creating request (as the old code did) instead of
the AVBPrint internal buffer. Some minor cosmetics.
Signed-off-by: Marton Balint <cus@passwd.hu>
It is explicitly required by the HTTP RFC. Without this patch URLs like
http://example.com?query will not work.
Fixes ticket #8466.
Signed-off-by: Marton Balint <cus@passwd.hu>
RFC 3986 states that the fragment identifier is separated from the rest of the
URI prior to a dereference, and thus the identifying information within the
fragment itself is dereferenced solely by the user agent.
Signed-off-by: Marton Balint <cus@passwd.hu>
add ff_http_do_new_request2() which supports options to be applied to
HTTPContext after initialisation with the new uri
Signed-off-by: Steven Liu <lq@onvideo.cn>
Signed-off-by: vectronic <hello.vectronic@gmail.com>
this function is used to get the previous shutdown status
when reusing the old connection in block mode.
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
Fix ticket #7297
The current setting for send-expect-100 option is either
enabled if applicable or forced enabled, no option to force
disable the header. This change is to expand the option setting
to provide more flexibility, which is useful for rstp case.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This avoids making invalid HTTP Range requests for a byte range past the
known end of the file during a seek. Those requests generally return a HTTP
response of 416 Range Not Satisfiable, which causes an error response.
Reference: https://tools.ietf.org/html/rfc7233
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Previously, AVERROR(EIO) was returned on failure of
http_open_cnx_internal(). Now the value is passed to upper level, thus
it is possible to distinguish ECONNREFUSED, ETIMEDOUT, ENETUNREACH etc.
Signed-off-by: Martin Storsjö <martin@martin.st>
In write only mode, the TCP receive buffer's data keeps growing with
http response messages and the buffer eventually becomes full.
This results in zero tcp window size, which in turn causes unwanted
issues, like, terminated tcp connection. The issue is apparent when
http persistent connection is enabled in hls/dash live streaming use
cases. To overcome this issue, the logic here reads the buffer data
when a file transfer is completed, so that any accumulated data in
the recieve buffer gets flushed out.
This refactors get_cookies to simplify some code paths, specifically for
skipping logic in the while loop or exiting it. It also simplifies the logic
for appending additional values to *cookies by replacing strlen/malloc/snprintf
with one call av_asnprintf.
This refactor fixes a bug where the cookie_params AVDictionary would get leaked
if we failed to allocate a new buffer for writing to *cookies.
If the string consists entirely of whitespace, this could in theory
continue to write '\0' before the start of the memory allocation. In
practice, it didn't really happen: the generic HTTP header parsing code
already skips leading whitespaces, so the string is either empty, or
consists a non-whitespace. (The generic code and the cookie code
actually have different ideas about what bytes are whitespace: the
former uses av_isspace(), the latter uses WHITESPACES. Fortunately,
av_isspace() is a super set of the http.c specific WHITESPACES, so
there's probably no case where the above assumption could have been
broken.)
It's trivial to craft a HTTP response that will make the code for
skipping trailing whitespace access and possibly overwrite bytes outside
of the memory allocation. Why this can happen is blindingly obvious: it
accesses cstr[strlen(cstr)-1] without checking whether the string is
empty.
libavformat prints a warning that the cookie couldn't be parsed (see
callers of parse_cookie()). This is obviously not true - it could be
parsed, but was simply ignored. Don't return an error to avoid the
warning.
The condition was a bit too long, and most editors will break the line
and turn it into an unreadable mess. Move out some of the conditions.
This should not change the behavior.
If the stream was aborted using the libavformat interrupt callback, we
don't want it to log the reconnect warning. (Exiting after logging this
warning worked well, so this is only for avoiding the ugly warning.)
It makes no sense to return an error after the first reconnect, and then
somehow resume the next time it's called. Usually this will lead to
demuxer errors. Make reconnecting block instead, until it has either
successfully reconnected, or given up.
Also make the wait reasonably interruptible. Since there is no mechanism
for this in the API, polling is the best we can do. This behaves roughly
the same as other interruptible network functions in libavformat.
(The original code would work if it returned AVERROR(EAGAIN) or so,
which would make retry_transfer_wrapper() repeat the read call. But I
think having an explicit loop for this is better anyway.)
I also snuck in a fix for reconnect_at_eof. It has to check for
AVERROR_EOF, not 0.
Can be used by the api user to figure out what http features the server supports based on the response received.
Signed-off-by: Aman Gupta <aman@tmm1.net>
This makes do_new_request fail early when dealing with a http/1.0 server, avoiding unnecessary "reconnecting" warnings shown to the user.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Justin Ruggles
Andreas Rheinhardt
erankor
James Almer
Derek Buitenhuis
Robert Bengtsson-Ölund
Andreas Rheinhardt
Moritz Barsnick
Robin Cooksey
Martin Storsjö
Zhao Zhili
Joey Smith
Limin Wang
Marton Balint
Steven Liu
vectronic
Jun Li
Carl Eugen Hoyos
Wenxiang Qian
Gyan Doshi
kjeyapal@akamai.com
Andrey Utkin
Clément Bœsch
Vishwanath Dixit
Stephan Holljes
Richard Shaffer
wm4
Aman Gupta