When running our integration tests in systemd we depend on each test
having a unique name. This is always the case unless --repeat is used,
in which case multiple tests with the same name run concurrently which
causes issues when allocating resources that use the test name as the
identifier.
Let's set MESON_TEST_ITERATION to the current iteration of the test so
we can use $TEST_NAME-$TEST_ITERATION as our test identifiers which will
avoid these issues.
This allows tests to check whether stdin is a tty to figure out if they're
running in interactive mode or not.
It also makes sure that tests that are not running in interactive mode
don't inadvertendly try to read from stdin.
This is very similar to --gdb, except it doesn't spawn GDB, but
connects stdin/stdout/stderr directly to the test itself. This allows
interacting with integration tests that spawn a shell in a container
or virtual machine when the test fails.
In systemd we're migrating our integration tests to run using the
meson test runner. We want to allow interactive debugging of failed
tests directly in the virtual machine or container that is spawned
to run the test. To make this possible, we need meson test to connect
stdin/stdout/stderr of the test directly to the user's terminal, just
like is done with the --gdb option.
Followup to 7b7d2e060b which handles ASAN and UBSAN.
It turns out that MSAN needs the same treatment. I've checked other sanitizers
like HWASAN and TSAN - it looks like they may both need it too, but Meson doesn't
currently suppose those anyway (see https://github.com/mesonbuild/meson/pull/12648).
Signed-off-by: Sam James <sam@gentoo.org>
Signed-off-by: Eli Schwartz <eschwartz93@gmail.com>
This replaces all of the Apache blurbs at the start of each file with an
`# SPDX-License-Identifier: Apache-2.0` string. It also fixes existing
uses to be consistent in capitalization, and to be placed above any
copyright notices.
This removes nearly 3000 lines of boilerplate from the project (only
python files), which no developer cares to look at.
SPDX is in common use, particularly in the Linux kernel, and is the
recommended format for Meson's own `project(license: )` field
Do as we do for MALLOC_PERTURB and set a sensible value for both ASAN_OPTIONS
and UBSAN_OPTIONS to abort on failure and give more helpful output at the
same time. We do not set these options if the user has exported a value
themselves to allow override.
In the last week alone, I've observed two cases where people were expecting
sanitizers to abort on failure and were surprised when it didn't:
1) 252d693797
2) c47df433f7
Correct this - which is in-line with meson's DWIM/DTRT philosophy.
Signed-off-by: Sam James <sam@gentoo.org>
Performed using https://github.com/ilevkivskyi/com2ann
This has no actual effect on the codebase as type checkers (still)
support both and negligible effect on runtime performance since
__future__ annotations ameliorates that. Technically, the bytecode would
be bigger for non function-local annotations, of which we have many
either way.
So if it doesn't really matter, why do a large-scale refactor? Simple:
because people keep wanting to, but it's getting nickle-and-dimed. If
we're going to do this we might as well do it consistently in one shot,
using tooling that guarantees repeatability and correctness.
Repeat with:
```
com2ann mesonbuild/
```
Make them into real type annotations. These are the only ones that if
automatically rewritten, would cause flake8 to error out with the
message: "E128 continuation line under-indented for visual indent".
This detects cases where module A imports a function from B, and C
imports that same function from A instead of B. It's not part of the API
contract of A, and causes innocent refactoring to break things.
The original point of specifying Optional was to default to None...
oops. The practical effect of this change is that the testsuite no
longer repeatedly logs "No tests defined." in between more meaningful
output.
Replace unencodable XML chars with their printable representation, so
that, xmllint can parse test outputs without error.
Closes#9894
Co-authored-by: Tristan Partin <tristan@partin.io>
It is not the primary purpose of mtest, and it ends up mingled with a
list of actual tests, which isn't nice if you're trying to capture and
parse this.
In commit 628effb369 we started verifying
the build.ninja file was up to date before even `--list`ing tests,
because we could end up with incorrect information. This meant that
ninja always runs at startup, and typically returns "no work to do",
which ended up listed as "one of" the tests.
Instead of unconditionally running ninja attached to the console, first
check it in dry-run mode with stdout intercepted, to see if ninja
considers itself up to date. If it is, continue. Only if an actual
refresh is needed, do we run it while attached to the console.
In the garden path, this avoids useless information. In cases where we'd
already print a full meson reconfigure log, we continue to do so.
Allow the use of wildcards (e.g. *) to match test names in `meson test`.
Raise an error is given test name does not match any test.
Optimize the search by looping through the list of tests only once.
Replace illegal characters when reading gtest generated xml file,
to prevent a ParseError and a stacktrace.
catch et.ParseError, just in case, to prevent stopping other tests
if the xml file was malformed.
We need to remember its value when reconfiguring, but the Build object
is not reused, only coredata is.
This also makes CLI more consistent by allowing `-Dvsenv=true` syntax.
Fixes: #11309
It can only be used for projects that don't have any rules at all, i.e.
they are purely using Meson to:
- configure files
- run (script?) tests
- install files that exist by the end of the setup stage
This can be useful e.g. for Meson itself, a pure python project.
What we are mainly doing here is checking that the options make sense,
or fixing that up if they don't. And in the next commit we will want to
do that by also checking the build object.
Given the construct `foo = (bar == baz)` some people like parentheses
and some do not. They're pointless and don't mean anything, though. I
don't feel this is particularly helpful to code clarity, tbh, and pylint
now notices this and warns about it in our current pylint config.
I think this is reasonable, so let's remove the odd parens.
Partial rollback of commit b7a5c384a1. The
rationale was based on a confusing wording of the TAP14 spec, which is
under discussion for clarification / amendment.
TAP14 doesn't (shouldn't) really say that missing a version line is
potentially an error. Rather, this is the correct way to denote TAP12,
which a TAP14 harness may not understand or try to parse. The intention
was never to suggest that harnesses "should" take exception to the
missing version line on the grounds that one should really add a version
line.
So, stop emitting an annoying warning for something that's valid usage.
Meson understands TAP12 and that's okay.
However, we do need to keep the part of that commit which set the
version to 12 if it was otherwise unspecified. But instead of
distinguishing between None and a version, just default to 12.
These functions constantly want the current asyncio loop, and we run a
function call each time to get it. And the function call is a deprecated
one. Python 3.7 brings the more explicit get_running_loop for use when
we know we're inside one, with the aim of getting rid of get_event_loop
once support for python <3.7 disappears. Meson no longer supports python
<3.7 either.
Switch to the new API, and save the reference for reuse instead of
constantly re-calculating it.
Fix a TODO comment about moving to asyncio.run, now that we use
sufficiently new python to do it.
Note that we create an event loop for Windows using the new python
defaults, but in a completely different part of the code from where we
need to use it. Since asyncio.run creates the loop on its own, we need
to set the default policy instead -- which we probably should have done
all along.
In https://bugs.python.org/issue42392 this stopped implicitly creating
an event loop if none exists. We created it before running _run_tests(),
so it would auto-create an event loop and set the default, which means
we cannot create one explicitly on our own schedule or we end up with
two of them.
Delay this until we actually start the logger. This happens inside the
actual testsuite loop, so it finds the running loop and doesn't create a
new one, even on python <3.10.
TAP 14 states:
> Harnesses may treat any TAP stream lacking a version as a failed test.
TAP 13 states:
> In the absence of any version line version 12 is assumed. It is an
> error to explicitly specify any version lower than 13.
So, modern TAP is saying that we should treat a missing version as a
test definition bug, it's no longer okay to use a missing version as
saying "let's use TAP 12". But, we can choose whether to treat it that
way or error out.
Let's do a diagnostic, as we do elsewhere. But allow TAP streams that
aren't well defined, if they used to be well defined (back in TAP 12).
In commit a7e458effa we stopped erroring
out on invalid TAP stream contents, with the rationale that "prove" has
become more lenient.
A close reading of the TAP spec indicates why, though:
> A TAP parser is required to not consider an unknown line as an error but
> may optionally choose to capture said line and hand it to the test
> harness, which may have custom behavior attached. This is to allow for
> forward compatability. Test::Harness silently ignores incorrect lines,
> but will become more stringent in the future. TAP::Harness reports TAP
> syntax errors at the end of a test run.
The goal of treating unknown lines as an error in the TAP parser is not
because unknown lines are fine and dandy. The goal is to allow
implementing future versions of TAP, and handling it via existing
parsers. Since Meson has both a parser and a harness, let's do exactly
that -- pass these lines as a distinctive status to the test harness,
then have the test harness complain.
Just like comment lines, blank lines do nothing. Before commit
a7e458effa we ended off the parser by
returning if the line was blank, because we needed to in order to catch
non-blank lines as errors. But really, we should have always returned
much earlier and not wasted time attempting to process anything.
This allows early exit of the project tests once a certain number of
failures are detected. For example `meson test --maxfail=1` will abort
as soon as a single test fails.
Currently running tests are marked as failed via INTERRUPT.
Resolves#9352
When the build definition has changed since the last ninja invocation meson
test operated on an outdated list of tests and their dependencies. That could
lead to some tests not being run / not all dependencies being built. This was
particularly confusing because the user would see the output of
reconfiguration and rebuilding, and the next mtest invocation would have the
updated configuration.
One issue with this is that that we will now output more useless ninja output
when nothing needs to be done (the "Entering directory" part is not repeated,
as we happen to be in the build directory already). It likely is worth
removing that output, perhaps by testing if anything needs to be done with
ninja -n, but that seems better addressed separately.
Fixes: #9852
A later commit will add a second invocation of ninja, no point in having the
detection code twice.
This changes the exit code in case ninja isn't found from 125 to 127, which
seems more appropriate given the justification for returning 125 (to make git
bisect run skip that commit). If we can't find ninja we'll not succeed in
other commits either.
A subsequent commit will do a bit more during test data loading, making a
dedicated function seem advisable.
The diff looks a bit odd, using git show --diff-algorithm=patience
will make it clearer.
googletest 1.12.1 generates new JUnit4 invalid attributes file and
line.
Maybe all gtest "invalid" attributes are actually valid JUnit5
attributes, and maybe schema should be upgraded to JUni5.
Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com>
Daan De Meyer
Christopher Dilks
Charles Brunet
Sam James
Dylan Baker
Xavier Claessens
Luke Elliott
Tristan Partin
Eli Schwartz
Nazir Bilal Yavuz
Josh Soref
HiPhish
Jussi Pakkanen
Andres Freund
Dimitri John Ledkov
Hemmo Nieminen