[`MALLOC_PERTURB_`](http://man7.org/linux/man-pages/man3/mallopt.3.html) is
is set to a random value between 1..255. This can help find memory
set to a random value between 1..255. This can help find memory leaks on
leaks on configurations using glibc, including with non-GCC compilers.
configurations using glibc, including with non-GCC compilers. This feature
This feature can be disabled as discussed in [test()](Reference-manual.md#test).
can be disabled as discussed in [test()](Reference-manual.md#test).
## Coverage
## Coverage
If you enable coverage measurements by giving Meson the command line flag
If you enable coverage measurements by giving Meson the command line flag
`-Db_coverage=true`, you can generate coverage reports after running the tests
`-Db_coverage=true`, you can generate coverage reports after running the
(running the tests is required to gather the list of functions that get
tests (running the tests is required to gather the list of functions that get
called). Meson will autodetect what coverage generator tools you have installed
called). Meson will autodetect what coverage generator tools you have
and will generate the corresponding targets. These targets are `coverage-xml`
installed and will generate the corresponding targets. These targets are
and `coverage-text` which are both provided by [Gcovr](http://gcovr.com)
`coverage-xml` and `coverage-text` which are both provided by
(version 3.3 or higher) and `coverage-html`, which requires
[Gcovr](http://gcovr.com) (version 3.3 or higher) and `coverage-html`, which
[Lcov](https://ltp.sourceforge.io/coverage/lcov.php) and
requires [Lcov](https://ltp.sourceforge.io/coverage/lcov.php) and
[GenHTML](https://linux.die.net/man/1/genhtml) or
[GenHTML](https://linux.die.net/man/1/genhtml) or [Gcovr](http://gcovr.com).
[Gcovr](http://gcovr.com). As a convenience, a high-level `coverage` target is
As a convenience, a high-level `coverage` target is also generated which will
also generated which will produce all 3 coverage report types, if possible.
produce all 3 coverage report types, if possible.
The output of these commands is written to the log directory `meson-logs` in your build directory.
The output of these commands is written to the log directory `meson-logs` in
your build directory.
## Parallelism
## Parallelism
To reduce test times, Meson will by default run multiple unit tests in parallel. It is common to have some tests which can not be run in parallel because they require unique hold on some resource such as a file or a D-Bus name. You have to specify these tests with a keyword argument.
To reduce test times, Meson will by default run multiple unit tests in
parallel. It is common to have some tests which can not be run in parallel
because they require unique hold on some resource such as a file or a D-Bus
name. You have to specify these tests with a keyword argument.
```meson
```meson
test('unique test', t, is_parallel : false)
test('unique test', t, is_parallel : false)
```
```
Meson will then make sure that no other unit test is running at the same time. Non-parallel tests take longer to run so it is recommended that you write your unit tests to be parallel executable whenever possible.
Meson will then make sure that no other unit test is running at the same
time. Non-parallel tests take longer to run so it is recommended that you
write your unit tests to be parallel executable whenever possible.
By default Meson uses as many concurrent processes as there are cores on the test machine. You can override this with the environment variable `MESON_TESTTHREADS` like this.
By default Meson uses as many concurrent processes as there are cores on the
test machine. You can override this with the environment variable
`MESON_TESTTHREADS` like this.
```console
```console
$ MESON_TESTTHREADS=5 ninja test
$ MESON_TESTTHREADS=5 ninja test
@ -70,7 +82,10 @@ $ MESON_TESTTHREADS=5 ninja test
*(added in version 0.52.0)*
*(added in version 0.52.0)*
Tests can be assigned a priority that determines when a test is *started*. Tests with higher priority are started first, tests with lower priority started later. The default priority is 0, meson makes no guarantee on the ordering of tests with identical priority.
Tests can be assigned a priority that determines when a test is *started*.
Tests with higher priority are started first, tests with lower priority
started later. The default priority is 0, meson makes no guarantee on the
Note that the test priority only affects the starting order of tests and subsequent tests are affected by how long it takes previous tests to complete. It is thus possible that a higher-priority test is still running when lower-priority tests with a shorter runtime have completed.
Note that the test priority only affects the starting order of tests and
subsequent tests are affected by how long it takes previous tests to
complete. It is thus possible that a higher-priority test is still running
when lower-priority tests with a shorter runtime have completed.
## Skipped tests and hard errors
## Skipped tests and hard errors
Sometimes a test can only determine at runtime that it can not be run.
Sometimes a test can only determine at runtime that it can not be run.
For the default `exitcode` testing protocol, the GNU standard approach in this case is to exit the program with error code 77. Meson will detect this and report these tests as skipped rather than failed. This behavior was added in version 0.37.0.
For the default `exitcode` testing protocol, the GNU standard approach in
this case is to exit the program with error code 77. Meson will detect this
and report these tests as skipped rather than failed. This behavior was added
in version 0.37.0.
For TAP-based tests, skipped tests should print a single line starting with `1..0 # SKIP`.
For TAP-based tests, skipped tests should print a single line starting with
`1..0 # SKIP`.
In addition, sometimes a test fails set up so that it should fail even if it is marked as an expected failure. The GNU standard approach in this case is to exit the program with error code 99. Again, Meson will detect this and report these tests as `ERROR`, ignoring the setting of `should_fail`. This behavior was added in version 0.50.0.
In addition, sometimes a test fails set up so that it should fail even if it
is marked as an expected failure. The GNU standard approach in this case is
to exit the program with error code 99. Again, Meson will detect this and
report these tests as `ERROR`, ignoring the setting of `should_fail`. This
behavior was added in version 0.50.0.
## Testing tool
## Testing tool
The goal of the meson test tool is to provide a simple way to run tests in a variety of different ways. The tool is designed to be run in the build directory.
The goal of the meson test tool is to provide a simple way to run tests in a
variety of different ways. The tool is designed to be run in the build
directory.
The simplest thing to do is just to run all tests, which is equivalent to running `ninja test`.
The simplest thing to do is just to run all tests, which is equivalent to
running `ninja test`.
```console
```console
$ meson test
$ meson test
@ -125,7 +154,8 @@ Tests belonging to a suite `suite` can be run as follows
$ meson test --suite (sub)project_name:suite
$ meson test --suite (sub)project_name:suite
```
```
Since version *0.46*, `(sub)project_name` can be omitted if it is the top-level project.
Since version *0.46*, `(sub)project_name` can be omitted if it is the
top-level project.
Multiple suites are specified like:
Multiple suites are specified like:
@ -145,7 +175,8 @@ Sometimes you need to run the tests multiple times, which is done like this:
$ meson test --repeat=10
$ meson test --repeat=10
```
```
Invoking tests via a helper executable such as Valgrind can be done with the `--wrap` argument
Invoking tests via a helper executable such as Valgrind can be done with the
`--wrap` argument
```console
```console
$ meson test --wrap=valgrind testname
$ meson test --wrap=valgrind testname
@ -163,17 +194,25 @@ Meson also supports running the tests under GDB. Just doing this:
$ meson test --gdb testname
$ meson test --gdb testname
```
```
Meson will launch `gdb` all set up to run the test. Just type `run` in the GDB command prompt to start the program.
Meson will launch `gdb` all set up to run the test. Just type `run` in the
GDB command prompt to start the program.
The second use case is a test that segfaults only rarely. In this case you can invoke the following command:
The second use case is a test that segfaults only rarely. In this case you
can invoke the following command:
```console
```console
$ meson test --gdb --repeat=10000 testname
$ meson test --gdb --repeat=10000 testname
```
```
This runs the test up to 10 000 times under GDB automatically. If the program crashes, GDB will halt and the user can debug the application. Note that testing timeouts are disabled in this case so `meson test` will not kill `gdb` while the developer is still debugging it. The downside is that if the test binary freezes, the test runner will wait forever.
This runs the test up to 10 000 times under GDB automatically. If the program
crashes, GDB will halt and the user can debug the application. Note that
testing timeouts are disabled in this case so `meson test` will not kill
`gdb` while the developer is still debugging it. The downside is that if the
test binary freezes, the test runner will wait forever.
Sometimes, the GDB binary is not in the PATH variable or the user wants to use a GDB replacement. Therefore, the invoked GDB program can be specified *(added 0.52.0)*:
Sometimes, the GDB binary is not in the PATH variable or the user wants to
use a GDB replacement. Therefore, the invoked GDB program can be specified
*(added 0.52.0)*:
```console
```console
$ meson test --gdb --gdb-path /path/to/gdb testname
$ meson test --gdb --gdb-path /path/to/gdb testname
Meson will report the output produced by the failing tests along with other useful information as the environmental variables. This is useful, for example, when you run the tests on Travis-CI, Jenkins and the like.
Meson will report the output produced by the failing tests along with other
useful information as the environmental variables. This is useful, for
example, when you run the tests on Travis-CI, Jenkins and the like.
For further information see the command line help of Meson by running `meson test -h`.
For further information see the command line help of Meson by running `meson
test -h`.
## Legacy notes
## Legacy notes
If `meson test` does not work for you, you likely have a old version of Meson.
If `meson test` does not work for you, you likely have a old version of
In that case you should call `mesontest` instead. If `mesontest` doesn't work
Meson. In that case you should call `mesontest` instead. If `mesontest`
either you have a very old version prior to 0.37.0 and should upgrade.
doesn't work either you have a very old version prior to 0.37.0 and should
upgrade.
## Test outputs
Meson will write several different files with detailed results of running
tests. These will be written into $builddir/meson-logs/
### testlog.json
This is not a proper json file, but a file containing one valid json object
per line. This is file is designed so each line is streamed out as each test
is run, so it can be read as a stream while the test harness is running
### testlog.junit.xml
This is a valid JUnit XML description of all tests run. It is not streamed
out, and is written only once all tests complete running.
When tests use the `tap` protocol each test will be recorded as a testsuite
container, with each case named by the number of the result.
When tests use the `gtest` protocol meson will inject arguments to the test
to generate it's own JUnit XML, which meson will include as part of this XML
Jussi Pakkanen
5 years ago
committed by
GitHub