As per Issue #883, an incorrect `libcares.pc` can be generated when
`CMAKE_THREAD_LIBS_INIT` contains a value like `-lpthread` because it
gets added to `libcares.pc` with another `-l` prefix. We can't control
the behavior of `CMAKE_THREAD_LIBS_INIT` since its set by `FIND_PACKAGE
(Threads)`.
Lets strip the `-l` prefix from the library before adding it to
`CARES_DEPENDENT_LIBS` which is used in the generation of `libcares.pc`.
Fixes#883
Reported-By: 前进，前进，进 (@leleliu008)
Fix By: Brad House (@bradh352)
The DNS server format is insufficient for future configurations, such as
supporting DNS over TLS (DoT) and DNS over HTTPS (DoH), as well as
additional functionality such as domain-specific servers. Already, in
the case where different UDP and TCP ports are used, it is impossible to
represent in the current format.
In order to try to use some standardized format, we are going to define
our own URI schemes that should be parse-able by any URI parser. The new
scheme will only be used when the configuration cannot otherwise be
expressed using the current `ipaddr%iface:port` format, which is the
format used as the nameserver configuration in `/etc/resolv.conf`.
However, the parser `ares_set_servers_csv()` shall accept the new URI
scheme format even when it is not necessary.
This PR implements a URI parser and writer and hooks the basic usage
into `ares_set_servers_csv()` and `ares_get_servers_csv()` as well as
provides updated documentation in the relevant manpages.
We will define these URI schemes:
* `dns://` - Normal DNS server (UDP + TCP). We need to be careful not to
conflict with query params defined in
https://datatracker.ietf.org/doc/html/rfc4501 since we'd technically be
extending this URI scheme. Port defaults to `53`.
* `dns+tls://` - DNS over TLS. Port defaults to `853`.
* `dns+https://` - DNS over HTTPS. Port defaults to `443`.
We initially will define these query parameters (additional arguments
may be required in the future to specify options such as TLS certificate
validation rules):
* `tcpport` - TCP port to use, only for `dns://` scheme. The `port`
specified as part of the `authority` component of the URI will be used
for both UDP and TCP by default, this option will override the TCP port.
* `ipaddr` - Only for `dns+tls://` and `dns+https://`. If the
`authority` component of the URI contains a hostname, this is used to
specify the ip address of the hostname. If not specified, will need to
use a non-secure server to perform a DNS lookup to retrieve this
information. It is always recommended to have both the ip address and
fully qualified domain name specified.
* `hostname` - Only for `dns+tls://` and `dns+https://`. If the
`authority` component of the URI contains an ip address, this is used to
specify the fully qualified domain name of the server. If not specified,
will need to use a non-secure server to perform a DNS reverse lookup to
retrieve this information. It is always recommended to have both the ip
address and fully qualified domain name specified.
* `domain` - If specified, this server is a domain-specific server. Any
queries for this domain will be routed to this server. Multiple servers
may be tagged with the same domain.
Examples:
```
dns://8.8.8.8
dns://[2001:4860:4860::8888]
dns://[fe80::b542:84df:1719:65e3%en0]
dns://192.168.1.1:55
dns://192.168.1.1?tcpport=1153
dns://10.0.1.1?domain=myvpn.com
dns+tls://8.8.8.8?hostname=dns.google
dns+tls://one.one.one.one?ipaddr=1.1.1.1
```
NOTE: While we are defining the scheme for things like domain-specific
servers, DNS over TLS and DNS over HTTPS, the underlying implementations
for those features do not yet exist and therefore will result in errors
if they are attempted to be used.
### Non-compliance in implementation
All these could be easily implemented/fixed if desired, however any such
changes would be of no use to the current c-ares usage of URIs:
* Does not currently support relative references
* Requires use of the authority section, blank is not allowed
* The query string is interpreted to be in
[application/x-www-form-urlencoded](https://en.wikipedia.org/wiki/Application/x-www-form-urlencoded)
format only and will result in parse errors if it is not. This is the
most common format used, however technically not valid to mandate this
format is used. We could add flags in the future to treat the query
string as opaque and leave it to the user to process. Or we could
internally have a list of schemes that use this format.
* [IDNA](https://en.wikipedia.org/wiki/Internationalized_domain_name) is
not supported.
* Does not support hex-encoded IPv4 addresses (this is compliant with RFC3986, but not WHATWG)
Authored-By: Brad House (@bradh352)
During compiling using MSVC and Ninja, its not uncommon to see errors
such as:
```
C:\projects\c-ares\test\ares_queryloop.c: fatal error C1041: cannot open program database 'C:\projects\c-ares\build\bin\vc140.pdb'; if multiple CL.EXE write to the same .PDB file, please use /FS
```
We see this as build failures in our AppVeyor runs.
The suggestion of using `/FS` is misleading as it already exists in the
compiler flags (and previous attempts to append them again were wrong).
The real issue I believe is that different output targets are being
compiled in parallel, but trying to use the same pdb file, but clearly
shouldn't be since they're different targets.
This CMake issue may be related:
https://gitlab.kitware.com/cmake/cmake/-/issues/20188
Authored-By: Brad House (@bradh352)
AppVeyor reports issues like "fatal error C1041: cannot open program database"
randomly. The /FS flag is supposed to correct this on highly parallel builds
like performed when using Ninja, but it looks like CMake already adds this.
Maybe appending it at the end will work as maybe some prior flag is
counteracting its effectiveness.
Fix By: Brad House (@bradh352)
pkg-config uses the .private suffix for static linking to c-ares,
set the libraries and cflags necessary for this.
Authored-By: Christoph Reiter (@lazka)
Approved-By: Brad House (@bradh352)
TCP Fast Open (TFO) allows TCP connection establishment in 0-RTT when a
client and server have previously communicated. The SYN packet will also
contain the initial data packet from the client to the server. This
means there should be virtually no slowdown over UDP when both sides
support TCP FastOpen, which is unfortunately not always the case. For
instance, `1.1.1.1` appears to support TFO, however `8.8.8.8` does not.
This implementation supports Linux, Android, FreeBSD, MacOS, and iOS.
While Windows does have support for TCP FastOpen it does so via
completion APIs only, and that can't be used with polling APIs like used
by every other OS. We could implement it in the future if desired for
those using `ARES_OPT_EVENT_THREAD`, but it would probably require
adopting IOCP completely on Windows.
Sysctls are required to be set appropriately:
- Linux: `net.ipv4.tcp_fastopen`:
- `1` = client only (typically default)
- `2` = server only
- `3` = client and server
- MacOS: `net.inet.tcp.fastopen`
- `1` = client only
- `2` = server only
- `3` = client and server (typically default)
- FreeBSD: `net.inet.tcp.fastopen.server_enable` (boolean) and
`net.inet.tcp.fastopen.client_enable` (boolean)
This feature is always-on, when running on an OS with the capability
enabled. Though some middleboxes have impacted end-to-end TFO and caused
connectivity errors, all modern OSs perform automatic blackholing of IPs
that have issues with TFO. It is not expected this to cause any issues
in the modern day implementations.
This will also help with improving latency for future DoT and DoH
implementations.
Authored-By: Brad House (@bradh352)
c-ares is getting larger these days and we keep adding source files to
the same directory so it can be hard to differentiate core c-ares
implementation from library/utility functions. Lets make some
subdirectories to help with that and shuffle files around.
Fix By: Brad House (@bradh352)
error C1041: cannot open program database '....'; if multiple CL.EXE write to the same .PDB file, please use /FS
might be output in some conditions, add /FS compiler flag to prevent it.
Fixes Issue: #796
Fix By: Brad House (@bradh352)
The DNS configuration for apple is stored in the system configuration
database. Apple does provide an emulated `/etc/resolv.conf` on MacOS
(but not iOS), it cannot, however, represent the entirety of the DNS
configuration. Alternatively, libresolv could be used to also retrieve
some system configuration, but it too is not capable of retrieving the
entirety of the DNS configuration.
Attempts to use the preferred public API of `SCDynamicStoreCreate()` and
friends yielded incomplete DNS information. Instead, that leaves some
apple "internal" symbols from `configd` that we need to access in order
to get the entire configuration. We can see that we're not the only ones
to do this as Google Chrome also does:
https://chromium.googlesource.com/chromium/src/+/HEAD/net/dns/dns_config_watcher_mac.cc
These internal functions are what what`libresolv` and `scutil` use to
retrieve the dns configuration. Since these symbols are not publicly
available, we will dynamically load the symbols from `libSystem` and
import the `dnsinfo.h` private header extracted from:
https://opensource.apple.com/source/configd/configd-1109.140.1/dnsinfo/dnsinfo.h
Fix By: Brad House (@bradh352)
CMake was unconditionally setting output locations globally, but
it should not do that if it is not the top-level project (e.g.
during chain building). Detect this fact and only set the output
location when top level.
Fixes Issue: #708
Fix By: Anthony Alayo (@anthonyalayo)
This PR implements an event thread to process all events on file descriptors registered by c-ares. Prior to this feature, integrators were required to understand the internals of c-ares and how to monitor file descriptors and timeouts and process events.
Implements OS-specific efficient polling such as epoll(), kqueue(), or IOCP, and falls back to poll() or select() if otherwise unsupported. At this point, it depends on basic threading primitives such as pthreads or windows threads.
If enabled via the ARES_OPT_EVENT_THREAD option passed to ares_init_options(), then socket callbacks cannot be used.
Fixes Bug: #611
Fix By: Brad House (@bradh352)
External integrations don't need sys/random.h in order to compile, remove the dependency. Try to fix building on legacy MacOS versions.
Fixes Issue: #682
Fix By: Brad House (@bradh352)
* cmake: avoid warning about non-existing include dir
In the Debian build logs I noticed the following warning:
cc1: warning: /build/c-ares-1.25.0/test/include: No such file or directory [-Wmissing-include-dirs]
This happened because ${CMAKE_INSTALL_INCLUDEDIR} had been added to
caresinternal. I believe it has been copied from the "real" lib
where it's used in the INSTALL_INTERFACE context. But because
caresinternal is never installed we don't need that include here.
* cmake: drop CARES_TOPLEVEL_DIR variable
The CARES_TOPLEVEL_DIR variable is the same as the automatically
created PROJECT_SOURCE_DIR variable. Let's stick to the official
one. Also because it is already used at places where CARES_TOPLEVEL_DIR
is used as well.
Fix By: Gregor Jasny (@gjasny)
Completely rework the autotools build system, issues have cropped up due to the complexity and could cause issues on even semi-modern Linux systems (Ubuntu 20.04 for example).
Changes include:
Remove all curl/xc/cares m4 helper files, they go overboard on detections of functions and datatypes. Go back to more plain autoconf macros as they've come a long way over the years.
Use known systems and heuristics to determine datatypes for functions like send() and recv(), rather than the error prone detection which required thousands of permutations and might still get it wrong.
Remove unneeded configure arguments like --enable-debug or --enable-optimize, its more common for people to simply pass their own CFLAGS on the command line.
Only require CARES_STATICLIB definition on Windows static builds, its not necessary ever for other systems, even when hiding non-public symbols.
Remove some function and definition detections that were never used in c-ares
The test framework is now embedded into the toplevel configure system, there was no need to chain build the test system as it is never built externally to c-ares.
As a side-effect of the changes, a configure run completes in about 25% of the original time.
This has been tested on various Linux distributions (of varying age), FreeBSD, MacOS, Windows (via MSYS2 with Mingw), and Solaris10/11 (by @dfandrich), AIX 7.3 (by @dfandrich). It is not unlikely that this may have broken more esoteric or legacy systems, and we'll likely need to be ready to accept bug reports and patches, but it has removed over 10k lines of build system code. It is very likely any issues that crop up will add far fewer lines of code to fix such systems.
Fixes Bug: #670
Fix By: Brad House (@bradh352)
Old MacOS SDKs (like 10.8) require you include `sys/socket.h` before you include `net/if.h` as reported by MacPorts. Using a new SDK but with setting the macos target version does not have the same issue.
Fixes Issue: #672
Fix By: Brad House (@bradh352)
There is a reported build issue where getrandom() is detected
but compile fails due to a missing prototype. This commit attempts
to resolve that issue.
Fixes Bug: #665
Fix By: Brad House (@bradh352)
GoogleTest should be unbundled. Google changed their guidance a few years back and modern versions of google test cannot build the bundling code file.
This PR also updates to use C++14 as is required by modern GoogleTest versions.
Fixes Bug: #506
Fix By: Brad House (@bradh352)
Some environments may send router advertisements on a link setting their link-local (fe80::/10) address as a valid DNS server to the remote system. This will cause a DNS entry to be created like `fe80::1%iface`, since all link-local network interfaces are technically part of the same /10 subnet, it must be told what interface to send packets through explicitly if there are multiple physical interfaces.
This PR adds support for the %iface modifier when setting DNS servers via `/etc/resolv.conf` as well as via `ares_set_servers_csv()`.
For MacOS and iOS it is assumed that libresolve will set the `sin6_scope_id` and should be supported, but my test systems don't seem to read the Router Advertisement for RDNSS link-local. Specifying the link-local dns server on MacOS via adig has been tested and confirmed working.
For Windows, this is similar to MacOS in that the system doesn't seem to honor the RDNSS RA, but specifying manually has been tested to work.
At this point, Android support does not exist.
Fixes Bug #462
Supersedes PR #463
Fix By: Brad House (@bradh352) and Serhii Purik (@sergvpurik)
c-ares does not have any concept of thread-safety. It has always been 100% up to the implementor to ensure they never call c-ares from more than one thread at a time. This patch adds basic thread-safety support, which can be disabled at compile time if not desired. It uses a single recursive mutex per channel, which should be extremely quick when uncontested so overhead should be minimal.
Fixes Bug: #610
Also sets the stage to implement #611
Fix By: Brad House (@bradh352)
When building for UWP (WindowsStore), additional headers are needed and some functions are not available. This also adds AppVeyor CI/CD support to catch these issues in the future.
Fix By: Deal (@halx99) and Brad House (@bradh352)
Brad House
Chenyu Yang
Faraz
Gregor Jasny
Christian Clauss