If the command fails for one subproject we should still continue with
others. Failed subprojects are reported at the end.
This issue became more problematic when doing parallel tasks because the
reason the command fails was completely hidden by other parallel tasks.
Logs are printed only when the task is done to not interleave logs while
running multiple tasks in parallel. That means that nothing is printed
until the task is done and that could take time.
This moves all the code into a class and call its run() method in a
thread. The class queues all logs to print them at the end to avoid
mixing output of multiple actions.
This will help facilitate cache busting in certain situations, and
replaces hand-rolled solutions of writing a length command to remove
various files/folders within the subprojects directory.
When using --reset we should guarantee that next reconfigure will pick
the latest code. For wrap-file we have no way to know if the revision
changed, so we have to delete the source tree and extract again.
It is unlikely that user has local changes in non-git subprojects, and
--reset is known to be dangerous.
Otherwise it's not possible to share git subprojects via worktrees
when creating a worktree of a git repository that uses meson
subprojects.
The downside is that the user needs to be careful while adding commits
to each tree's index.
wraps from subprojects are now merged into the list of wraps from main
project, so they can be used to download dependencies of dependencies
instead of having to promote wraps manually. If multiple projects
provides the same wrap file, the first one to be configured wins.
This also fix usage of sub-subproject that don't have wrap files. We can
now configure B when its source tree is at
`subprojects/A/subprojects/B/`. This has the implication that we cannot
assume that subproject "foo" is at `self.subproject_dir / 'foo'` any
more.
It was done to include them in `meson subprojects foreach` without
--types argument, but it's better to special case missing --types and
include wraps that have type=None too. It was a bad idea because that
was messing them in `meson subprojects update`, now they are ignored by
that command.
If revision is a tag that does not exist locally, `git fetch origin
<revision>` won't create it and checkout will fail. Using --refmap
ensures that references exists locally.
Besides refactoring code into smaller functions:
- Makes the --rebase behaviour the default for consistency: it was
already rebasing when current branch and revision are the same, it is
less confusing to rebase when they are different too.
- Add --reset mode that checkout the new branch and hard reset that
branch to remote commit. This new mode guarantees that every
subproject are exactly at the wrap's revision.
- Local changes are always stashed first to avoid any data loss. In the
worst case scenario the user can always check reflog and stash list to
rollback.
Fixes: #7526
If the command fails on some subprojects continue with the rest but
return non-0 code. This is useful for CI scripts to ensure it tests
latest code instead of old cached code in case of network error or
something.
D lang compilers have an option -release (or similar) which turns off
asserts, contracts, and other runtime type checking. This patch wires
that up to the b_ndebug flag.
Fixes#7082
Reuse the git helper for `meson wrap` and `meson subprojects` so we
don't need to maintain the same git-colors-on-windows workarounds in
multiple places.
The 'output' field of the subprocess.CalledProcessError exception is
valid only when subprocess.check_output() is called, trying to access it
after calling subprocess.check_call() results in an unwanted exception
when commands return non-zero exit code, e.g.:
-----------------------------------------------------------------------
$ meson subprojects foreach false
Executing command in ./subprojects/sqlite-amalgamation-3250100
-> Not downloaded yet
Executing command in ./subprojects/gstreamer
Traceback (most recent call last):
File "/home/ao2/meson/meson/mesonbuild/msubprojects.py", line 177, in foreach
subprocess.check_call([options.command] + options.args, cwd=repo_dir)
File "/usr/lib/python3.7/subprocess.py", line 363, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['false']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ao2/meson/meson/mesonbuild/mesonmain.py", line 129, in run
return options.run_func(options)
File "/home/ao2/meson/meson/mesonbuild/msubprojects.py", line 248, in run
options.subprojects_func(wrap, repo_dir, options)
File "/home/ao2/meson/meson/mesonbuild/msubprojects.py", line 180, in foreach
out = e.output.decode().strip()
AttributeError: 'NoneType' object has no attribute 'decode'
-----------------------------------------------------------------------
Use subprocess.check_output() instead and behave more like git commands
in handling stderr.
This makes it possible to actually run commands on all subprojects
allowing them to fail on some subprojects and succeed on others.
Also catch the case of missing commands and print an error message in
this case as well.
Sometimes it is convenient to run an arbitrary command (e.g. 'git diff')
on all subprojects.
Add a 'meson subprojects foreach' command to take care of that.
For this command the common argument 'subprojects' does not make sense,
so only add '--sourcedir' and cover the case of a missing
options.subprojects in run().
* If clone-recursive is specified, populate submodules only after
the specified revision/branch has been checked out. Fixes#4996
* Update submodules recursively on `meson subprojects update`
Andrei Alexeyev
Xavier Claessens
Tristan Partin
Eli Schwartz
Nirbheek Chauhan
Dylan Baker
Daniel Mensinger
Michael Brockus
Antonio Ospite
Markus Theil
Andrei Alexeyev