The test.json format currently has three keys related to skipping tests:
* `skip_on_jobname`
* `skip_on_os`
* `skip_on_env`
While `skip_on_env` marks the test itself as skipped, i.e. they don't get run when the
conditions are met, the other two skip options are just marking the test as "expected to be skipped"
if the conditions apply, i.e. they want to see `MESON_SKIP_TEST` in the output and things will
fail if that doesn't happen. They don't actually skip the tests as the names imply.
To make this clearer rename the keys:
* `skip_on_jobname` -> `expect_skip_on_jobname`
* `skip_on_os` -> `expect_skip_on_os`
`skip_on_env` stays the same, since that actually skips.
The docs were also confused about this, so adjust that too.
stdout line matching supports count since commit
66d62a224e to fail if certain output is
present (count: 0) but it was never documented in the contribution
guidelines.
Plan to replace the hard-coded list of 'may be skipped' framework tests in
skippable() with annotations in test.json which record 'will be skipped
in these specific CI jobs'.
If the value of the MESON_CI_JOBNAME env var (an arbitrary string
expected to be unique for each CI configuration) contains any of the
strings in the `skip_on_jobname` key in test.json, the test is expected
to output MESON_SKIP_TEST.
Unexpected skips or runs are treated as an error.
Future work: Maybe we should add additional count categories 'unexpected
skip' and 'unexpected not skipped', rather than counting those as 'skipped'
and 'failed', respectively.
Some changes:
* Set HOME to /root, since github mounts its own HOME and 'wine'
(because of permissions) and 'dub' (can't find packages) don't
like that.
* Remove the seccomp option, doesn't seem to be needed.
This is useful for automatically generated docs (doxygen, hotdoc)
with a lot of generated files that may differ with different
versions of the generator.
D lang compilers have an option -release (or similar) which turns off
asserts, contracts, and other runtime type checking. This patch wires
that up to the b_ndebug flag.
Fixes#7082
Update the test.json schema, adding the 'stdout' property.
Also amend the test.json schema so the presence of an unexpected
property on the root object causes a validation error.
v2:
Also add 'tools' property to json schema.
Amend the documentation not to use the word 'list' to describe a dict.
Expected stdout lines must match lines from the actual stdout, in the
same order. Lines with match type 're' are regex matched.
v2:
Ignore comment lines in expected_stdout
v3:
Automatically adjust path separators for location in expected output
v4:
Put expected stdout in test.json, rather than a separate file
Adds the `tools` section to `tests.json` to specify requirements
for the tools in the environment. All tests that fail at least
one tool requirements check are skipped.
This does a couple of nice things, one is that editors like vscode can
be configured to use this schema to provide auto completion and error
highlighting if invalid values are added or required values are missing.
It also allows us test that the format of the test matrix work in a unit
test, which I've added. It does require that the python jsonschema
package is installed.
This is needed when mixing D and C code, as it's possible to end up
witha combination of linkers and compilres such that C produces pdb
files but D does not.
This allows the harness to apply the version correctly, putting it in the right
place, dropping the right amount of numbers, etc.
pdb taking a version allows it to be more easily copied from the
shared_lib type.
* Fix flake8 whitespace reports
$ flake8 | grep -E '(E203|E221|E226|E303|W291|W293)'
./mesonbuild/coredata.py:337:5: E303 too many blank lines (2)
* Fix flake8 'variable assigned value but unused' reports
$ flake8 | grep -E F841
./mesonbuild/modules/gnome.py:922:9: F841 local variable 'target_name' is assigned to but never used
* Fix flake8 'imported but unused' reports
$ flake8 | grep F401
./mesonbuild/compilers/__init__.py:128:1: F401 '.c.ArmclangCCompiler' imported but unused
./mesonbuild/compilers/__init__.py:138:1: F401 '.cpp.ArmclangCPPCompiler' imported but unused
./mesonbuild/modules/__init__.py:4:1: F401 '..mlog' imported but unused
PR #3717 imports ARMCLANG compilers in __init__, but does not add them to
__all__, so they are not re-exported by the compilers package like
everything else.
* More details about flake8 in Contributing.md
Mention that Sider runs flake8
Suggest seting flake8 as a pre-commit hook
H. Vetinari
Christoph Reiter
Josh Soref
Andreas Deininger
Guilherme Janczak
Eli Schwartz
Alyssa Ross
andy5995
Jussi Pakkanen
Jon Turney
HRXN
Antonin Décimo
Daniel Mensinger
Dylan Baker
Michael Hirsch, Ph.D
Wolfgang Stöggl
Anton Leontiev
Nirbheek Chauhan
Aleksey Filippov
Edward Betts