Aim is to ensure that code coverage information can escape the
container. To do this:
- Enter a new mount namespace too, so that we can...
- Bind mount the expected source directory into the container
- Share memory with the sub-process so coverage information is
shared too.
On Linux we can potentially use user and UTS namespaces to run a test
in a pseudo-container with:
- arbitrary filesystem (e.g. /etc/resolv.conf, /etc/nsswitch.conf, /etc/hosts)
- arbitrary hostname/domainname.
Include a first pass at the framework code to allow this, along with a
first test case that uses the container.
Different platforms will do different numbers of allocations
in the processing of a given API call; just check that the
return code is either success or ENOMEM, and free off any
returned state in the former case.
Also cope with ECONNREFUSED as well as ENOTFOUND.
- Initial nmake file based off library nmake file
- Cast socket call arguments to (char *)
- Use wrapper sclose() that maps to closesocket() or close()
- Build a config.h indicating presence of headers
- Conditionally include netdb.h
- Remove unnecessary include of sys/socket.h
- Force longer bitmask for allocation failure tracking
- Call WSAStartup() / WSACleanup() in main()
- Set TCP_NODELAY for mock server
- Turn on tests in AppVeyor build
- Update the MockServer to allow separate specification of
UDP and TCP ports
- Have an array of mock servers listening on consecutive
sets of ports.
- Rename Process(fd) to ProcessFD(fd) to avoid confusion.
- Initialize channel by using the new ares_set_servers_ports()
entrypoint, so multiple ports on the same loopback address
can be used.
- Make mock server listen on UDP + TCP in parallel.
- Test UDP->TCP fallback on truncation
- Test EDNS->no-EDNS fallback
- Test some environment init options
- Test nonsense reply
test: short response
Including:
- Split each parse function test set out into separate files.
- Add an allocation failure test for each parsing function.
- Add error check test for each parsing function.
Include tests of internal functions, based on the value of the
CARES_SYMBOL_HIDING macro; need to configure the library with
--disable-symbol-hiding to enable these tests.
Currently get error from Travis on this install step, and downgrading one
version appears to fix the problem.
"Could not find any downloads that satisfy the requirement pyOpenSSL>=0.13
(from requests[security])"
Cover Linux & OSX on the container infrastructure, but install
a later G++ to satisfy the tests' need for C++11.
Use a build matrix to include a variety of build variants:
- ASAN
- UBSAN
- LSAN
- Coverage via coveralls.io
test: invoke ASAN and coverage in Travis build
Also shift to use explicit build matrix
test: Use coveralls.io for coverage tracking
test: Add a build with UBSAN
Also expand and re-order the setting of environment variables
for easier modification.
test: Add LSAN build to Travis config
The tests are written in C++11, using the GoogleTest and GoogleMock
frameworks. They have their own independent autoconf setup, so that
users of the library need not have a C++ compiler just to get c-ares
working (however, the test/configure.ac file does assume the use of
a shared top-level m4/ directory). However, this autoconf setup has
only been tested on Linux and OSX so far.
Run with "./arestest", or "./arestest -v" to see extra debug info.
The GoogleTest options for running specific tests are also
available (e.g. "./arestest --gtest_filter=*Live*").
The tests are nowhere near complete yet (currently hitting around
60% coverage as reported by gcov), but they do include examples
of a few different styles of testing:
- There are live tests (ares-test-live.cc), which assume that the
current machine has a valid DNS setup and connection to the
internet; these tests issue queries for real domains but don't
particularly check what gets returned. The tests will fail on
an offline machine.
- There a few mock tests (ares-test-mock.cc) that set up a fake DNS
server and inject its port into the c-ares library configuration.
These tests allow specific response messages to be crafted and
injected, and so are likely to be used for many more tests in
future.
- To make this generation/injection easier, the dns-proto.h file
includes C++ helper classes for building DNS packets.
- Other library entrypoints that don't require network activity
(e.g. ares_parse_*_reply) are tested directly.
- There are few tests of library-internal functions that are not
normally visible to API users (in ares-test-internal.cc).
- A couple of the tests use a helper method of the test fixture to
inject memory allocation failures, using the earlier change to the
library to allow override of malloc/realloc/free.
- There is also an entrypoint to allow Clang's libfuzzer to drive
the packet parsing code in ares_parse_*_reply, together with a
standalone wrapper for it (./aresfuzz) to allow use of afl-fuzz
for further fuzz testing.
Pull in testing macros from the GNU autoconf archive to allow
configure scripts to test for and setup use of a C++11 compiler
(AX_CXX_COMPILE_STDCXX_11) and the pthreads library (AX_PTHREAD).
Note that these macros are not used by the main library autoconf,
just by the tests (which share the same m4/ directory).
Configure with:
./configure --enable-code-coverage
Show coverage output with:
make code-coverage-capture
Built on m4/ax_code_coverage.m4 from the GNU autoconf archive
to provide the macros to check for presence of gcov + lcov;
upstream macro modified to:
- Remove use of $(AM_DEFAULT_VERBOSITY) , as earlier versions of
autoconf (such as the one used by default on Travis) do not have this.
- Rather than automatically defining CODE_COVERAGE_RULES to be a set
of makefile rules that use ifeq/endif (which is GNU make-specific),
instead only define CODE_COVERAGE_RULES if coverages is turned on,
and in that case don't use conditionals in the makefile.
David Drysdale