Plan to replace the hard-coded list of 'may be skipped' framework tests in
skippable() with annotations in test.json which record 'will be skipped
in these specific CI jobs'.
If the value of the MESON_CI_JOBNAME env var (an arbitrary string
expected to be unique for each CI configuration) contains any of the
strings in the `skip_on_jobname` key in test.json, the test is expected
to output MESON_SKIP_TEST.
Unexpected skips or runs are treated as an error.
Future work: Maybe we should add additional count categories 'unexpected
skip' and 'unexpected not skipped', rather than counting those as 'skipped'
and 'failed', respectively.
Since 25df6e7d split the iteration over tests to start them from the
iteration to collect their results, the variable 'name' is only being
set in the first iteratiorn, so all tests are treated as being in the
last test category read (probably 'wasm') for skipppable() and in the
XML output.
Store the category name in the TestDef object
Use it in skippable()
Use it in classname attribute of XML test results
Split out checking of file format as a separate GitHub workflow, rather
than running it as part of the project tests for every platform and
toolchain combination in CI, so that this test is not effected by the
changed paths constraints which are applied to the project tests.
This now gives a clear error rather than silently passes for
unrecognized categories, like:
python run_project_tests.py --only nonexisting
or
python run_project_tests.py --only objc # should be 'objective c'
We don't always have qmake installed (and it's good to test failure
paths too!) so we can't expect this to succeed in all cases. With the
following commit we'll use a test.json to test both pkg-config and
qmake, so we need to be able to skip.
By default expected line must be matched in order. When an expected line
is matched it does not matter if it's matched again later or not.
When defining "count", it means that line must be matched exactly that
many times before matching the next expected line. Once all occurences
have been matched for an expected line, it not must appear any more in
all next lines.
All changes were created by running
"pyupgrade --py3-only --keep-percent-format"
and committing the results. I have not touched string formatting for
now.
- use set literals
- simplify .format() parameter naming
- remove __future__
- remove default "r" mode for open()
- use OSError rather than compatibility aliases
- remove stray parentheses in function(generator) scopes
Automatically colorize the text when printing the AnsiDecorator, based
on the result of mlog.colorize_console(). This is how AnsiDecorator
is used most of the time anyway.
* depenencies/llvm: Handle llvm-config --shared-mode failing
Fixes: #7371Fixes: #7878
* test cases/llvm: Refactor to use test.json
Instead of trying to cover everything internally
Jon Turney
Eli Schwartz
Daniel Mensinger
Dylan Baker
Jussi Pakkanen
Ralf Gommers
Xavier Claessens
Paolo Bonzini
Christoph Reiter
Jason Ekstrand
Sam Thursfield
Nirbheek Chauhan