This way we get some testing for the patches, and speed up our builds.
My server is hosted on a UK Linode, so it should have good uptimes.
However, we should likely move this into the Docker image at least
for Linux, and perhaps put it in a CI cache for the rest.
It is not feasible to test all failure modes by creating projects in
`test cases/failing` that would be an explosion of files, and that
mechanism is too coarse anyway. We have no way to ensure that the
expected error is being raised.
See FailureTests.test_dependency for an example.
This class now consolidates a lot of the logic that each external
dependency was duplicating in its class definition.
All external dependencies now set:
* self.version
* self.compile_args and self.link_args
* self.is_found (if found)
* self.sources
* etc
And the abstract ExternalDependency class defines the methods that
will fetch those properties. Some classes still override that for
various reasons, but those should also be migrated to properties as
far as possible.
Next step is to consolidate and standardize the way in which we call
'configuration binaries' such as sdl2-config, llvm-config, pkg-config,
etc. Currently each class has to duplicate code involved with that
even though the format is very similar.
Currently only pkg-config supports multiple version requirements, and
some classes don't even properly check the version requirement. That
will also become easier now.
This actually caught a cached-dependency related bug for me that the
test-time regen did not. I also increased the ninja wait time to
1 second because that's actually how long you need to sleep to be
guaranteed that a change will be detected.
Must poke upstream about https://github.com/ninja-build/ninja/issues/371
Also sets more groundwork for running unit tests with backends other
that Ninja.
Transferring global state to executors is totally broken in Python 3.4
so just serialize all the commands.
And use generic build/clean/test/install commands in the unit tests,
just like project tests. This sets the groundwork for running the unit
tests with all backends.
We also need to check whether the program found in PATH can be executed
directly by Windows or if we need to figure out what the interpreter is
and add it to the list.
Also add `msc` to the list of extensions that can be executed natively
Includes a project test and a unit test for this and all expected
behaviours on Windows.
Also add new tests for the platform-specific and compiler-specific
versioning scheme.
A rough summary is:
1. A bug in how run_tests.py:validate_install checked for files has been
fixed. Earlier it wasn't checking the install directory properly.
2. Shared libraries are no longer installed in common tests, and the
library name/path testing is now done in platform-specific tests.
3. Executables are now always called something?exe in the
installed_files.txt file, and the suffix automatically corrected
depending on the platform.
4. If a test installs a file called 'no-installed-files', the installed
files for that test are not validated. This is required to implement
compiler-specific tests for library names/paths such as MSVC vs MinGW
5. The platform-specific file renaming in run_tests.py has been mostly
removed since it is broken for shared libraries and isn't needed for
static libraries.
6. run_tests.py now reports all missing and extra files. The logic for
finding these has been reworked.
Print status on single line
only print a single line for succeeded tests and two lines for failed
tests. This makes it easier to scan the output for failed tests.
shutil.rmtree, which is used by tempfile.TemporaryDirectory, randomly fails
on Windows, because the directory is not empty although it should be,
because all files were deleted by shutil.rmtree internals before trying to
remove the directory.
A simple retry approach fixes the issue.
The _run_test method uses several global variables (unity_flags,
backend_flags, compile_commands, install_commands) which are
not set when the method is run by the executor (at least on Windows).
To resolve this, pass the variables as method parameters.
Nirbheek Chauhan
Elliott Sales de Andrade
Ole André Vadla Ravnås
Jussi Pakkanen
Igor Gnatenko
Matthias Klumpp
Nirbheek Chauhan
Nicolas Schneider
Nicolas Schneider