There is never any reason to not do this since this script is supposed
to be run by developers and testers who are concerned with the details
of the problems.
It also helps with intermittent or hard-to-reproduce errors.
We have no test coverage for regeneration at all, which is why issues
like #1246 slide by without us noticing. With this, we will run a regen
on every test during `ninja test` after it has been compiled. This will
not affect test times too much since the regen will not rebuild anything
at all since there have been no source changes.
There is no way to do this in the .appveyor.yml file since it seems that
the appveyor environment is forcibly written after each cmd command that
is run.
Also add new tests for the platform-specific and compiler-specific
versioning scheme.
A rough summary is:
1. A bug in how run_tests.py:validate_install checked for files has been
fixed. Earlier it wasn't checking the install directory properly.
2. Shared libraries are no longer installed in common tests, and the
library name/path testing is now done in platform-specific tests.
3. Executables are now always called something?exe in the
installed_files.txt file, and the suffix automatically corrected
depending on the platform.
4. If a test installs a file called 'no-installed-files', the installed
files for that test are not validated. This is required to implement
compiler-specific tests for library names/paths such as MSVC vs MinGW
5. The platform-specific file renaming in run_tests.py has been mostly
removed since it is broken for shared libraries and isn't needed for
static libraries.
6. run_tests.py now reports all missing and extra files. The logic for
finding these has been reworked.
Print status on single line
only print a single line for succeeded tests and two lines for failed
tests. This makes it easier to scan the output for failed tests.
shutil.rmtree, which is used by tempfile.TemporaryDirectory, randomly fails
on Windows, because the directory is not empty although it should be,
because all files were deleted by shutil.rmtree internals before trying to
remove the directory.
A simple retry approach fixes the issue.
The _run_test method uses several global variables (unity_flags,
backend_flags, compile_commands, install_commands) which are
not set when the method is run by the executor (at least on Windows).
To resolve this, pass the variables as method parameters.
A 'test_args.txt' file in the same directory as the test case will be
parsed by the test framework and the content will be passed as arguments
to meson during configuration. The arguments are put before any
'extra_args' to make them overwritable from the command line.
Nirbheek Chauhan
Jussi Pakkanen
Igor Gnatenko
Elliott Sales de Andrade
Matthias Klumpp
Nirbheek Chauhan
Nicolas Schneider
Nicolas Schneider