From 1309b47cbbd75565ea647ba6a570f8db6f8a3c2a Mon Sep 17 00:00:00 2001 From: Kostiantyn Ponomarenko Date: Sun, 21 Oct 2018 20:05:33 +0300 Subject: [PATCH] doc: Improve wrap and subproject documentation [skip ci] Make docs more user friendly, and add missing information. Signed-off-by: Kostiantyn Ponomarenko --- .../markdown/Adding-new-projects-to-wrapdb.md | 146 +++++++++------ docs/markdown/Subprojects.md | 171 +++++++++++++----- .../markdown/Wrap-dependency-system-manual.md | 120 ++++-------- 3 files changed, 260 insertions(+), 177 deletions(-) diff --git a/docs/markdown/Adding-new-projects-to-wrapdb.md b/docs/markdown/Adding-new-projects-to-wrapdb.md index 832f80322..c79cfd393 100644 --- a/docs/markdown/Adding-new-projects-to-wrapdb.md +++ b/docs/markdown/Adding-new-projects-to-wrapdb.md @@ -1,84 +1,120 @@ -# Adding new projects to wrap +# Adding new projects to WrapDB -**If you don't have permissions to do something on this page, please - open issue against https://github.com/mesonbuild/wrapweb/issues to - let us know that you want to start new project.** -## Overview +## How it works -The wrap provider service is a simple web service that makes it easy -to download build definitions for projects. It works in much the same -way as Debian: we take the unaltered upstream source package and add a -new build system to it as a patch. These build systems are stored as -Git repositories on GitHub. They only contain build definition -files. You may also think of them as an overlay to upstream source. +Each wrap repository has a master branch with only one initial commit and *no* wrap files. +And that is the only commit ever made on that branch. -## Creator script +For every release of a project a new branch is created. The new branch is named after the +the upstream release number (e.g. `1.0.0`). This branch holds a wrap file for +this particular release. + +There are two types of wraps on WrapDB - regular wraps and wraps with Meson build +definition patches. A wrap file in a repository on WrapDB must have a name `upstream.wrap`. + +Wraps with Meson build definition patches work in much the same way as Debian: we take the unaltered upstream source package and add a new build system to it as a patch. These build systems are stored as Git repositories on GitHub. They only contain build definition files. You may also think of them as an overlay to upstream source. + +Whenever a new commit is pushed into GitHub's project branch, a new wrap is generated +with an incremented version number. All the old releases remain unaltered. +New commits are always done via GitHub merge requests and must be reviewed by +someone other than the submitter. -The WrapDB repository has a [helper -script](https://github.com/mesonbuild/wrapweb/blob/master/mesonwrap.py) -to generate new repositories, verify them and update them. The documentation below roughly explains -what it does to create a new wrap repository using plain shell commands. +Note that your Git repo with wrap must not contain the subdirectory of the source +release. That gets added automatically by the service. You also must not commit +any source code from the original tarball into the wrap repository. ## Choosing the repository name Wrapped subprojects are used much like external dependencies. Thus -they should have the same name as the upstream projects. If the -project provides a pkg-config file, then the repository name should be +they should have the same name as the upstream projects. + +If the project provides a pkg-config file, then the repository name should be the same as the pkg-config name. Usually this is the name of the project, such as `libpng`. Sometimes it is slightly different, however. As an example the libogg project's chosen pkg-config name is `ogg` instead of `libogg`, which is the reason why the repository is named plain `ogg`. -## Adding new project to the Wrap provider service +If there is no a pkg-config file, the name the project uses/promotes should be used, +lowercase only (Catch2 -> catch2). -Each project gets its own repo. It is initialized like this: +## How to contribute a new wrap - git init - git add readme.txt - git add LICENSE.build - git commit -a -m 'Create project foobar' - git remote add origin - git push -u origin master +If the project already uses Meson build system, then only a wrap file - `upstream.wrap` +should be provided. In other case a Meson build definition patch - a set of `meson.build` +files - should be also provided. -Note that this is the *only* commit that will ever be made to master branch. All other commits are done to branches. +### Request a new repository or branch -Repo names must fully match this regexp: `[a-z0-9._]+`. +Create an issue on the [wrapweb bug tracker](https://github.com/mesonbuild/wrapweb/issues) +using *Title* and *Description* below as a template. -## Adding a new branch to an existing project +*Title:* `new wrap: ` -Create a new branch whose name matches the upstream release number. +*Description:* +``` +upstream url: +version: +``` - git checkout master - git checkout -b 1.0.0 - git push origin 1.0.0 - (or from GitHub web page, remember to branch from master) +Wait until the new repository or branch is created. A link to the new repository or branch +will be posted in a comment to this issue. -Branch names must fully match this regexp: `[a-z0-9._]+`. +### Add a new wrap -## Adding a new release to an existing branch +First you need to fork the repository to your own page. +Then you can create the first Wrap commit that usually looks something like this. -Here is where the magic happens. Whenever a new commit is pushed into GitHub's project branch, a new wrap is generated with an incremented version number. All the old releases remain unaltered. New commits are always done via GitHub merge requests and must be reviewed by someone other than the submitter. +``` +tar xzf libfoo-1.0.0.tar.gz +git clone -b 1.0.0 git@github.com:yourusername/libfoo.git tmpdir +mv tmpdir/.git libfoo-1.0.0 +rm -rf tmpdir +cd libfoo-1.0.0 +git reset --hard +emacs upstream.wrap meson.build + +git add upstream.wrap meson.build +git commit -a -m 'Add wrap files for libfoo-1.0.0' +git push origin 1.0.0 +``` -Note that your Git repo must *not* contain the subdirectory of the source release. That gets added automatically by the service. You also must *not* commit any source code from the original tarball into the wrap repository. +Now you should create a pull request on GitHub. Remember to create it against the +correct branch rather than master (`1.0.0` branch in this example). GitHub should do +this automatically. -First you need to fork the repository to your own page. Then you can create the first Wrap commit that usually looks something like this. +## What is done by WrapDB maintainers - tar xzf libfoo_1.0.0.tar.gz - git clone -b 1.0.0 git@github.com:yourusername/libfoo.git tmpdir - mv tmpdir/.git libfoo-1.0.0 - rm -rf tmpdir - cd libfoo-1.0.0 - git reset --hard - emacs upstream.wrap meson.build - - git add upstream.wrap meson.build - git commit -a -m 'Create wrap files for libfoo-1.0.0.' - git push origin 1.0.0 +### Adding new project to the Wrap provider service -Now you can file a merge request. Remember to file it against branch -1.0.0 rather than master. GitHub should do this automatically. +Each project gets its own repo. It is initialized like this: + +``` +git init +git add readme.txt +git add LICENSE.build +git commit -a -m 'Create project foobar' +git remote add origin +git push -u origin master +``` + +Note that this is the *only* commit that will ever be made to master branch. All other commits are done to branches. + +Repo names must fully match this regexp: `[a-z0-9._]+`. + +### Adding a new branch to an existing project + +Create a new branch whose name matches the upstream release number. + +``` +git checkout master +git checkout -b 1.0.0 +git push origin 1.0.0 +(or from GitHub web page, remember to branch from master) +``` + +Branch names must fully match this regexp: `[a-z0-9._]+`. ## Changes to original source @@ -93,6 +129,12 @@ to functionality. All such changes must be submitted to upstream. You may also host your own Git repo with the changes if you wish. The Wrap system has native support for Git subprojects. +## Creator script + +The WrapDB repository has a +[helper script](https://github.com/mesonbuild/wrapweb/blob/master/mesonwrap.py) +to generate new repositories, verify them and update them. + ## Reviewing wraps See [Wrap review guidelines](Wrap-review-guidelines.md). diff --git a/docs/markdown/Subprojects.md b/docs/markdown/Subprojects.md index 80ed3e718..2e3e2ead5 100644 --- a/docs/markdown/Subprojects.md +++ b/docs/markdown/Subprojects.md @@ -14,45 +14,46 @@ Meson tries to solve this problem by making it extremely easy to provide both at the same time. The way this is done is that Meson allows you to take any other Meson project and make it a part of your build without (in the best case) any changes to its Meson setup. It -becomes a transparent part of the project. The basic idiom goes -something like this. +becomes a transparent part of the project. -```meson -dep = dependency('foo', fallback : [subproject_name, variable_name]) -``` +It should be noted that this only works for subprojects that are built +with Meson. It can not be used with any other build system. The reason +is the simple fact that there is no possible way to do this reliably +with mixed build systems. + +## A subproject example + +Usually dependencies consist of some header files plus a library to link against. +To declare this internal dependency use `declare_dependency` function. As an example, suppose we have a simple project that provides a shared -library. It would be set up like this. +library. It's `meson.build` would look like this. ```meson -project('simple', 'c') -i = include_directories('include') -l = shared_library('simple', 'simple.c', include_directories : i, install : true) -simple_dep = declare_dependency(include_directories : i, - link_with : l) -``` +project('libsimple', 'c') -Then we could use that from a master project. First we generate a -subdirectory called `subprojects` in the root of the master -directory. Then we create a subdirectory called `simple` and put the -subproject in that directory. Now the subproject can be used like -this. +inc = include_directories('include') +libsimple = shared_library('simple', + 'simple.c', + include_directories : inc, + install : true) -```meson -project('master', 'c') -dep = dependency('simple', fallback : ['simple', 'simple_dep']) -exe = executable('prog', 'prog.c', - dependencies : dep, install : true) +libsimple_dep = declare_dependency(include_directories : inc, + link_with : libsimple) ``` -With this setup the system dependency is used when it is available, -otherwise we fall back on the bundled version. If you wish to always -use the embedded version, then you would declare it like this: +### Naming convention for dependency variables -```meson -simple_sp = subproject('simple') -dep = simple_sp.get_variable('simple_dep') -``` +Ideally the dependency variable name should be of `_dep` form. +This way one can just use it without even looking inside build definitions of that subproject. + +In cases where there are multiple dependencies need to be declared, the default one +should be named as `_dep` (e.g. `gtest_dep`), and others can have +`___dep` form (e.g. `gtest_main_dep` - gtest with main function). + +There may be exceptions to these rules where common sense should be applied. + +### Build options in subproject All Meson features of the subproject, such as project options keep working and can be set in the master project. There are a few @@ -62,17 +63,111 @@ must not set global arguments because there is no way to do that reliably over multiple subprojects. To check whether you are running as a subproject, use the `is_subproject` function. -It should be noted that this only works for subprojects that are built -with Meson. It can not be used with any other build system. The reason -is the simple fact that there is no possible way to do this reliably -with mixed build systems. +## Using a subproject + +All subprojects must be inside `subprojects` directory. +The `subprojects` directory must be at the top level of your project. +Subproject declaration must be in your top level `meson.build`. + +### A simple example + +Let's use `libsimple` as a subproject. + +At the top level of your project create `subprojects` directory. +Then copy `libsimple` into `subprojects` directory. + +Your project's `meson.build` should look like this. + +```meson +project('my_project', 'cpp') + +libsimple_proj = subproject('libsimple') +libsimple_dep = libsimple_proj.get_variable('libsimple_dep') + +executable('my_project', + 'my_project.cpp', + dependencies : libsimple_dep, + install : true) +``` + +Note that the subproject object is *not* used as the dependency, but +rather you need to get the declared dependency from it with +`get_variable` because a subproject may have multiple declared +dependencies. + +### Toggling between system libraries and embedded sources + +When building distro packages it is very important that you do not +embed any sources. Some distros have a rule forbidding embedded +dependencies so your project must be buildable without them or +otherwise the packager will hate you. + +Here's how you would use system libraries and fall back to embedding sources +if the dependency is not available. + +```meson +project('my_project', 'cpp') + +libsimple_dep = dependency('libsimple', required : false) + +if not libsimple_dep.found() + libsimple_proj = subproject('libsimple') + libsimple_dep = libsimple_proj.get_variable('libsimple_dep') +endif + +executable('my_project', + 'my_project.cpp', + dependencies : libsimple_dep, + install : true) +``` + +Because this is such a common operation, Meson provides a shortcut for +this use case. + +```meson +dep = dependency('foo', fallback : [subproject_name, variable_name]) +``` + +The `fallback` keyword argument takes two items, the name of the +subproject and the name of the variable that holds the dependency. If +you need to do something more complicated, such as extract several +different variables, then you need to do it yourself with the manual +method described above. + +Using this shortcut the build definition would look like this. + +```meson +project('my_project', 'cpp') + +libsimple_dep = dependency('libsimple', fallback : ['libsimple', 'libsimple_dep']) + +executable('my_project', + 'my_project.cpp', + dependencies : libsimple_dep, + install : true) +``` + +With this setup when libsimple is provided by the system, we use it. When +that is not the case we use the embedded version (the one from subprojects). + +Note that `libsimple_dep` can point to an external or an internal dependency but +you don't have to worry about their differences. Meson will take care +of the details for you. + +### Subprojects depending on other subprojects Subprojects can use other subprojects, but all subprojects must reside in the top level `subprojects` directory. Recursive use of subprojects is not allowed, though, so you can't have subproject `a` that uses subproject `b` and have `b` also use `a`. -# Command-line options +## Obtaining subprojects + +Meson ships with a dependency system to automatically obtain +dependency subprojects. It is documented in the [Wrap dependency +system manual](Wrap-dependency-system-manual.md). + +## Command-line options The usage of subprojects can be controlled by users and distros with the following command-line options: @@ -101,13 +196,7 @@ the following command-line options: want to specifically build against the library sources provided by your subprojects. -# Obtaining subprojects - -Meson ships with a dependency system to automatically obtain -dependency subprojects. It is documented in the [Wrap dependency -system manual](Wrap-dependency-system-manual.md). - -# Why must all subprojects be inside a single directory? +## Why must all subprojects be inside a single directory? There are several reasons. diff --git a/docs/markdown/Wrap-dependency-system-manual.md b/docs/markdown/Wrap-dependency-system-manual.md index 7da4be541..619492aa7 100644 --- a/docs/markdown/Wrap-dependency-system-manual.md +++ b/docs/markdown/Wrap-dependency-system-manual.md @@ -27,10 +27,24 @@ itself in a way that makes it easy to use (usually this means as a static library). To use this kind of a project as a dependency you could just copy and -extract it inside your project's `subprojects` directory. However -there is a simpler way. You can specify a Wrap file that tells Meson -how to download it for you. An example wrap file would look like this -and should be put in `subprojects/foobar.wrap`: +extract it inside your project's `subprojects` directory. + +However there is a simpler way. You can specify a Wrap file that tells Meson +how to download it for you. If you then use this subproject in your build, +Meson will automatically download and extract it during build. This makes +subproject embedding extremely easy. + +All wrap files must have a name of `.wrap` form and be in `subprojects` dir. + +Currently Meson has three kinds of wraps: +- wrap-file +- wrap-file with Meson build patch +- wrap-git + +## wrap-file + +An example wrap file for `libfoobar` would have a name `libfoobar.wrap` +and would look like this: ```ini [wrap-file] @@ -41,21 +55,22 @@ source_filename = foobar-1.0.tar.gz source_hash = 5ebeea0dfb75d090ea0e7ff84799b2a7a1550db3fe61eb5f6f61c2e971e57663 ``` -If you then use this subproject in your build, Meson will -automatically download and extract it during build. This makes -subproject embedding extremely easy. +`source_hash` is *sha256sum* of `source_filename`. Since *0.49.0* if `source_filename` is found in project's `subprojects/packagecache` directory, it will be used instead of downloading the -source, even if `wrap-mode` option is set to `nodownload`. The file's hash will +source, even if `--wrap-mode` option is set to `nodownload`. The file's hash will be checked. +## wrap-file with Meson build patch + Unfortunately most software projects in the world do not build with Meson. Because of this Meson allows you to specify a patch URL. This works in much the same way as Debian's distro patches. That is, they are downloaded and automatically applied to the subproject. These -files contain a Meson build definition for the given subproject. A -wrap file with an additional patch URL would look like this. +files contain a Meson build definition for the given subproject. + +A wrap file with an additional patch URL would look like this: ```ini [wrap-file] @@ -83,10 +98,12 @@ put them somewhere where you can download them. Since *0.49.0* if `patch_filename` is found in project's `subprojects/packagecache` directory, it will be used instead of downloading the -patch, even if `wrap-mode` option is set to `nodownload`. The file's hash will +patch, even if `--wrap-mode` option is set to `nodownload`. The file's hash will be checked. -## Branching subprojects directly from git +## wrap-git + +This type of wrap allows branching subprojects directly from git. The above mentioned scheme assumes that your subproject is working off packaged files. Sometimes you want to check code out directly from @@ -128,79 +145,14 @@ clone-recursive = true ## Using wrapped projects -To use a subproject simply do this in your top level `meson.build`. - -```meson -foobar_proj = subproject('foobar') -``` - -Usually dependencies consist of some header files plus a library to -link against. To do this in a project so it can be used as a subproject you -would declare this internal dependency like this: - -```meson -foobar_dep = declare_dependency(link_with : mylib, - include_directories : myinc) -``` - -Then in your main project you would use them like this: +Wraps provide a convenient way of obtaining a project into your subproject directory. +Then you use it as a regular subproject (see [subprojects](Subprojects.md)). -```meson -executable('toplevel_exe', 'prog.c', - dependencies : foobar_proj.get_variable('foobar_dep')) -``` - -Note that the subproject object is *not* used as the dependency, but -rather you need to get the declared dependency from it with -`get_variable` because a subproject may have multiple declared -dependencies. - -## Toggling between distro packages and embedded source - -When building distro packages it is very important that you do not -embed any sources. Some distros have a rule forbidding embedded -dependencies so your project must be buildable without them or -otherwise the packager will hate you. - -Doing this with Meson and Wrap is simple. Here's how you would use -distro packages and fall back to embedding if the dependency is not -available. - -```meson -foobar_dep = dependency('foobar', required : false) - -if not foobar_dep.found() - foobar_proj = subproject('foobar') - # the subproject defines an internal dependency with - # the command declare_dependency(). - foobar_dep = foobar_proj.get_variable('foobar_dep') -endif - -executable('toplevel_exe', 'prog.c', - dependencies : foobar_dep) -``` - -Because this is such a common operation, Meson provides a shortcut for -this use case. - -```meson -foobar_dep = dependency('foobar', fallback : ['foobar', 'foobar_dep']) -``` - -The `fallback` keyword argument takes two items, the name of the -subproject and the name of the variable that holds the dependency. If -you need to do something more complicated, such as extract several -different variables, then you need to do it yourself with the manual -method described above. +## Getting wraps -With this setup when foobar is provided by the system, we use it. When -that is not the case we use the embedded version. Note that -`foobar_dep` can point to an external or an internal dependency but -you don't have to worry about their differences. Meson will take care -of the details for you. +Usually you don't want to write your wraps by hand. -## Getting wraps +There is an online repository called [WrapDB](https://wrapdb.mesonbuild.com) that provides +many dependencies ready to use. You can read more about WrapDB [here](Using-the-WrapDB.md). -Usually you don't want to write your wraps by hand. There is an online -repository called [WrapDB](Using-the-WrapDB.md) that provides many -dependencies ready to use. +There is also a Meson subcommand to get and manage wraps (see [using wraptool](Using-wraptool.md)).