Merge pull request #2334 from mesonbuild/promotedep

Add functionality to promote nested dependencies to top level.

docs/markdown/Using-wraptool.md

@@ -1,12 +1,20 @@
 # Using wraptool
 
-Wraptool is a helper tool that allows you to manage your source dependencies using the WrapDB database. It gives you all things you would expect, such as installing and updating dependencies. The wrap tool works on all platforms, the only limitation is that the wrap definition works on your target platform. If you find some Wraps that don't work, please file bugs or, even better, patches.
+Wraptool is a helper tool that allows you to manage your source
+dependencies using the WrapDB database. It gives you all things you
+would expect, such as installing and updating dependencies. The wrap
+tool works on all platforms, the only limitation is that the wrap
+definition works on your target platform. If you find some Wraps that
+don't work, please file bugs or, even better, patches.
 
-All code examples here assume that you are running the commands in your top level source directory. Lines that start with the `$` mark are commands to type.
+All code examples here assume that you are running the commands in
+your top level source directory. Lines that start with the `$` mark
+are commands to type.
 
 ## Simple querying
 
-The simplest operation to do is to query the list of packages available. To list them all issue the following command:
+The simplest operation to do is to query the list of packages
+available. To list them all issue the following command:
 
     $ wraptool list
     box2d
@@ -22,27 +30,35 @@ The simplest operation to do is to query the list of packages available. To list
     vorbis
     zlib
 
-Usually you want to search for a specific package. This can be done with the `search` command:
+Usually you want to search for a specific package. This can be done
+with the `search` command:
 
     $ wraptool search jpeg
     libjpeg
 
-To determine which versions of libjpeg are available to install, issue the `info` command:
+To determine which versions of libjpeg are available to install, issue
+the `info` command:
 
     $ wraptool info libjpeg
     Available versions of libjpeg:
       9a 2
 
-The first number is the upstream release version, in this case `9a`. The second number is the Wrap revision number. They don't relate to anything in particular, but larger numbers imply newer releases. You should always use the newest available release.
+The first number is the upstream release version, in this case
+`9a`. The second number is the Wrap revision number. They don't relate
+to anything in particular, but larger numbers imply newer
+releases. You should always use the newest available release.
 
 ## Installing dependencies
 
-Installing dependencies is just as straightforward. First just create the `subprojects` directory at the top of your source tree and issue the install command.
+Installing dependencies is just as straightforward. First just create
+the `subprojects` directory at the top of your source tree and issue
+the install command.
 
     $ wraptool install libjpeg
     Installed libjpeg branch 9a revision 2
 
-Now you can issue a `subproject('libjpeg')` in your `meson.build` file to use it.
+Now you can issue a `subproject('libjpeg')` in your `meson.build` file
+to use it.
 
 To check if your projects are up to date you can issue the `status` command.
 
@@ -51,9 +67,33 @@ To check if your projects are up to date you can issue the `status` command.
      libjpeg up to date. Branch 9a, revision 2.
      zlib not up to date. Have 1.2.8 2, but 1.2.8 4 is available.
 
-In this case `zlib` has a newer release available. Updating it is straightforward:
+In this case `zlib` has a newer release available. Updating it is
+straightforward:
 
     $ wraptool update zlib
     Updated zlib to branch 1.2.8 revision 4
 
-Wraptool can do other things besides these. Documentation for these can be found in the command line help, which can be accessed by `wraptool --help`.
+Wraptool can do other things besides these. Documentation for these
+can be found in the command line help, which can be accessed by
+`wraptool --help`.
+
+## Promoting dependencies
+
+Meson will only search for subprojects from the top level
+`subprojects` directory. If you have subprojects that themselves have
+subprojects, you must transfer them to the top level. This can be done
+by going to your source root and issuing a promotion command.
+
+    meson wrap promote projname
+
+This will cause Meson to go through your entire project tree, find an
+embedded subproject and copy it to the top level.
+
+If there are multiple embedded copies of a subproject, Meson will not
+try to guess which one you want. Instead it will print all the
+possibilities. You can then manually select which one to promote by
+writing it out fully.
+
+    meson wrap promote subprojects/s1/subprojects/projname
+
+This functionality was added in Meson release 0.45.0.

docs/markdown/snippets/wrap_promote.md

@@ -0,0 +1,11 @@
+# Can promote dependencies with wrap command
+
+The `promote` command makes it easy to copy nested dependencies to the top level.
+
+    meson wrap promote scommon
+
+This will search the project tree for a subproject called `scommon` and copy it to the top level.
+
+If there are many embedded subprojects with the same name, you have to specify which one to promote manually like this:
+
+    meson wrap promote subprojects/s1/subprojects/scommon

mesonbuild/interpreter.py

@@ -2224,6 +2224,7 @@ to directly access options of other subprojects.''')
                         # we won't actually read all the build files.
                         return fallback_dep
                 if not dep:
+                    self.print_nested_info(name)
                     assert(exception is not None)
                     raise exception
 
@@ -2237,6 +2238,28 @@ to directly access options of other subprojects.''')
     def func_disabler(self, node, args, kwargs):
         return Disabler()
 
+    def print_nested_info(self, dependency_name):
+        message_templ = '''\nDependency %s not found but it is available in a sub-subproject.
+To use it in the current project, promote it by going in the project source
+root and issuing %s.
+
+'''
+        sprojs = mesonlib.detect_subprojects('subprojects', self.source_root)
+        if dependency_name not in sprojs:
+            return
+        found = sprojs[dependency_name]
+        if len(found) > 1:
+            suffix = 'one of the following commands'
+        else:
+            suffix = 'the following command'
+        message = message_templ % (dependency_name, suffix)
+        cmds = []
+        command_templ = 'meson wrap promote '
+        for l in found:
+            cmds.append(command_templ + l[len(self.source_root)+1:])
+        final_message = message + '\n'.join(cmds)
+        print(final_message)
+
     def get_subproject_infos(self, kwargs):
         fbinfo = kwargs['fallback']
         check_stringlist(fbinfo)

mesonbuild/mesonlib.py

@@ -881,6 +881,31 @@ def windows_proof_rmtree(f):
     # Try one last time and throw if it fails.
     shutil.rmtree(f)
 
+
+def detect_subprojects(spdir_name, current_dir='', result=None):
+    if result is None:
+        result = {}
+    spdir = os.path.join(current_dir, spdir_name)
+    if not os.path.exists(spdir):
+        return result
+    for trial in glob(os.path.join(spdir, '*')):
+        basename = os.path.split(trial)[1]
+        if trial == 'packagecache':
+            continue
+        append_this = True
+        if os.path.isdir(trial):
+            detect_subprojects(spdir_name, trial, result)
+        elif trial.endswith('.wrap') and os.path.isfile(trial):
+            basename = os.path.splitext(basename)[0]
+        else:
+            append_this = False
+        if append_this:
+            if basename in result:
+                result[basename].append(trial)
+            else:
+                result[basename] = [trial]
+    return result
+
 class OrderedSet(collections.MutableSet):
     """A set that preserves the order in which items are added, by first
     insertion.

mesonbuild/wrap/wraptool.py

@@ -21,6 +21,8 @@ from glob import glob
 
 from .wrap import API_ROOT, open_wrapdburl
 
+from .. import mesonlib
+
 help_templ = '''This program allows you to manage your Wrap dependencies
 using the online wrap database http://wrapdb.mesonbuild.com.
 
@@ -142,6 +144,36 @@ def info(name):
     for v in versions:
         print(' ', v['branch'], v['revision'])
 
+def do_promotion(from_path, spdir_name):
+    if os.path.isfile(from_path):
+        assert(from_path.endswith('.wrap'))
+        shutil.copy(from_path, spdir_name)
+    elif os.path.isdir(from_path):
+        sproj_name = os.path.split(from_path)[1]
+        outputdir = os.path.join(spdir_name, sproj_name)
+        if os.path.exists(outputdir):
+            sys.exit('Output dir %s already exists. Will not overwrite.' % outputdir)
+        shutil.copytree(from_path, outputdir, ignore=shutil.ignore_patterns('subprojects'))
+
+def promote(argument):
+    path_segment, subproject_name = os.path.split(argument)
+    spdir_name = 'subprojects'
+    sprojs = mesonlib.detect_subprojects(spdir_name)
+    if subproject_name not in sprojs:
+        sys.exit('Subproject %s not found in directory tree.' % subproject_name)
+    matches = sprojs[subproject_name]
+    if len(matches) == 1:
+        do_promotion(matches[0], spdir_name)
+        return
+    if path_segment == '':
+        print('There are many versions of %s in tree. Please specify which one to promote:\n' % subproject_name)
+        for s in matches:
+            print(s)
+        sys.exit(1)
+    system_native_path_argument = argument.replace('/', os.sep)
+    if system_native_path_argument in matches:
+        do_promotion(argument, spdir_name)
+
 def status():
     print('Subproject status')
     for w in glob('subprojects/*.wrap'):
@@ -189,6 +221,11 @@ def run(args):
             print('info requires exactly one argument.')
             return 1
         info(args[0])
+    elif command == 'promote':
+        if len(args) != 1:
+            print('promote requires exactly one argument.')
+            return 1
+        promote(args[0])
     elif command == 'status':
         status()
     else:

run_unittests.py

@@ -468,6 +468,7 @@ class BasePlatformTests(unittest.TestCase):
         self.mconf_command = meson_command + ['configure']
         self.mintro_command = meson_command + ['introspect']
         self.mtest_command = meson_command + ['test', '-C', self.builddir]
+        self.wrap_command = meson_command + ['wrap']
         # Backend-specific build commands
         self.build_command, self.clean_command, self.test_command, self.install_command, \
             self.uninstall_command = get_backend_commands(self.backend)
@@ -1682,6 +1683,30 @@ int main(int argc, char **argv) {
         self.setconf("-Dfree_array_opt=['a,b', 'c,d']", will_build=False)
         self.opt_has('free_array_opt', ['a,b', 'c,d'])
 
+    def test_subproject_promotion(self):
+        testdir = os.path.join(self.unit_test_dir, '13 promote')
+        workdir = os.path.join(self.builddir, 'work')
+        shutil.copytree(testdir, workdir)
+        spdir = os.path.join(workdir, 'subprojects')
+        s3dir = os.path.join(spdir, 's3')
+        scommondir = os.path.join(spdir, 'scommon')
+        self.assertFalse(os.path.isdir(s3dir))
+        subprocess.check_call(self.wrap_command + ['promote', 's3'], cwd=workdir)
+        self.assertTrue(os.path.isdir(s3dir))
+        self.assertFalse(os.path.isdir(scommondir))
+        self.assertNotEqual(subprocess.call(self.wrap_command + ['promote', 'scommon'],
+                                            cwd=workdir,
+                                            stdout=subprocess.DEVNULL), 0)
+        self.assertFalse(os.path.isdir(scommondir))
+        subprocess.check_call(self.wrap_command + ['promote', 'subprojects/s2/subprojects/scommon'], cwd=workdir)
+        self.assertTrue(os.path.isdir(scommondir))
+        promoted_wrap = os.path.join(spdir, 'athing.wrap')
+        self.assertFalse(os.path.isfile(promoted_wrap))
+        subprocess.check_call(self.wrap_command + ['promote', 'athing'], cwd=workdir)
+        self.assertTrue(os.path.isfile(promoted_wrap))
+        self.init(workdir)
+        self.build()
+
 
 class FailureTests(BasePlatformTests):
     '''

test cases/unit/13 promote/meson.build

@@ -0,0 +1,5 @@
+project('promotion test', 'c')
+
+subproject('s1')
+subproject('s2')
+

test cases/unit/13 promote/subprojects/s1/meson.build

@@ -0,0 +1,7 @@
+project('s1', 'c')
+
+sc = subproject('scommon')
+s3 = subproject('s3')
+
+executable('s1', 's1.c',
+  link_with : [sc.get_variable('clib'), s3.get_variable('l')])

test cases/unit/13 promote/subprojects/s1/s1.c

@@ -0,0 +1,6 @@
+int func();
+int func2();
+
+int main(int argc, char **argv) {
+    return func() + func2();
+}

test cases/unit/13 promote/subprojects/s1/subprojects/s3/meson.build

@@ -0,0 +1,4 @@
+project('s3', 'c')
+
+l = static_library('s3', 's3.c')
+

test cases/unit/13 promote/subprojects/s1/subprojects/s3/s3.c

@@ -0,0 +1,3 @@
+int func2() {
+    return -42;
+}

test cases/unit/13 promote/subprojects/s1/subprojects/scommon/meson.build

@@ -0,0 +1,4 @@
+project('scommon', 'c')
+
+clib = static_library('scommon', 'scommon_broken.c')
+

test cases/unit/13 promote/subprojects/s1/subprojects/scommon/scommon_broken.c

@@ -0,0 +1 @@
+#error This file must not be used. The other scommon one should be instead.

test cases/unit/13 promote/subprojects/s2/meson.build

@@ -0,0 +1,6 @@
+project('s2', 'c')
+
+sc = subproject('scommon')
+
+executable('s2', 's2.c', link_with : sc.get_variable('clib'))
+

test cases/unit/13 promote/subprojects/s2/s2.c

@@ -0,0 +1,6 @@
+int func();
+
+
+int main(int argc, char **argv) {
+    return func() != 42;
+}

test cases/unit/13 promote/subprojects/s2/subprojects/athing.wrap

@@ -0,0 +1,2 @@
+The contents of this wrap file are never evaluated so they
+can be anything.

test cases/unit/13 promote/subprojects/s2/subprojects/scommon/meson.build

@@ -0,0 +1,4 @@
+project('scommon', 'c')
+
+clib = static_library('scommon', 'scommon_ok.c')
+

test cases/unit/13 promote/subprojects/s2/subprojects/scommon/scommon_ok.c

@@ -0,0 +1,3 @@
+int func() {
+    return 42;
+}