The test.json format currently has three keys related to skipping tests:
* `skip_on_jobname`
* `skip_on_os`
* `skip_on_env`
While `skip_on_env` marks the test itself as skipped, i.e. they don't get run when the
conditions are met, the other two skip options are just marking the test as "expected to be skipped"
if the conditions apply, i.e. they want to see `MESON_SKIP_TEST` in the output and things will
fail if that doesn't happen. They don't actually skip the tests as the names imply.
To make this clearer rename the keys:
* `skip_on_jobname` -> `expect_skip_on_jobname`
* `skip_on_os` -> `expect_skip_on_os`
`skip_on_env` stays the same, since that actually skips.
The docs were also confused about this, so adjust that too.
Installing python sources causes the python module to call
create_install_data() before Ninja backends adds extra outputs to Vala
targets.
Target objects are supposed to be immutable, adding outputs that late is
totally wrong. Add extra vala outputs immediately, but be careful
because the main output is only added later in post_init(). Luckily
the base class already puts a placeholder item in self.outputs for the
main filename so we can just replace self.outputs[0] instead of
replacing the whole list which would contain vala outputs at that stage.
This is surprisingly what SharedLibrary was already doing.
This commit adds a new keyword arg to extension_module() that enables
a user to target the Python Limited API, declaring the version of the
limited API that they wish to target.
Two new unittests have been added to test this functionality.
Plan to replace the hard-coded list of 'may be skipped' framework tests in
skippable() with annotations in test.json which record 'will be skipped
in these specific CI jobs'.
If the value of the MESON_CI_JOBNAME env var (an arbitrary string
expected to be unique for each CI configuration) contains any of the
strings in the `skip_on_jobname` key in test.json, the test is expected
to output MESON_SKIP_TEST.
Unexpected skips or runs are treated as an error.
Future work: Maybe we should add additional count categories 'unexpected
skip' and 'unexpected not skipped', rather than counting those as 'skipped'
and 'failed', respectively.
I'm not sure what the current schema is describing, but it's not doing
anything useful to validate 'matrix' entries, as I discover when I come
to add something to that.
'matrix' is a dict with possible keys 'options' and 'exclude'.
'options' is a dict with arbitrary keys, whose's values are arrays
Once we describe those dicts correctly, also fix that 'val' keys may be
string, boolean or null, and the corresponding 'exclude' keys may be
string or boolean.
v2:
Also allow 'val' and 'exclude' to be an array of strings.
Improve test.json schema to disallow arbitrary keys in all dicts which
have a defined set of keys (the 'installed', matrix 'options' and
'stdout' dicts).
Add 'count' and 'comment' keys to 'stdout' dict.
This is useful for automatically generated docs (doxygen, hotdoc)
with a lot of generated files that may differ with different
versions of the generator.
D lang compilers have an option -release (or similar) which turns off
asserts, contracts, and other runtime type checking. This patch wires
that up to the b_ndebug flag.
Fixes#7082
Update the test.json schema, adding the 'stdout' property.
Also amend the test.json schema so the presence of an unexpected
property on the root object causes a validation error.
v2:
Also add 'tools' property to json schema.
Amend the documentation not to use the word 'list' to describe a dict.
This does a couple of nice things, one is that editors like vscode can
be configured to use this schema to provide auto completion and error
highlighting if invalid values are added or required values are missing.
It also allows us test that the format of the test matrix work in a unit
test, which I've added. It does require that the python jsonschema
package is installed.
Jussi Pakkanen
Christoph Reiter
Arsen Arsenović
Xavier Claessens
Andrew McNulty
Eli Schwartz
Jon Turney
Daniel Mensinger
Dylan Baker