11 KiB
| title |
|---|
| Design rationale |
At its core, building C and C++ code is not a terribly difficult task. In fact, writing a text editor is a lot more complicated and takes more effort. Yet we have lots of very high quality editors but only few build systems with questionable quality and usability.
So, in the grand tradition of own-itch-scratching, I decided to run a scientific experiment. The purpose of this experiment was to explore what would it take to build a "good" build system. What kind of syntax would suit this problem? What sort of problems would this application need to solve? What sort of solutions would be the most appropriate?
To get things started, here is a list of requirements any modern cross-platform build system needs to provide.
1. Must be simple to use
One of the great virtues of Python is the fact that it is very readable. It is easy to see what a given block of code does. It is concise, clear and easy to understand. The proposed build system must be syntactically and semantically clean. Side effects, global state and interrelations must be kept at a minimum or, if possible, eliminated entirely.
2. Must do the right thing by default
Most builds are done by developers working on the code. Therefore the defaults must be tailored towards that use case. As an example the system shall build objects without optimization and with debug information. It shall make binaries that can be run directly from the build directory without linker tricks, shell scripts or magic environment variables.
3. Must enforce established best practices
There really is no reason to compile source code without the
equivalent of -Wall. So enable it by default. A different kind of
best practice is the total separation of source and build
directories. All build artifacts must be stored in the build
directory. Writing stray files in the source directory is not
permitted under any circumstances.
4. Must have native support for platforms that are in common use
A lot of free software projects can be used on non-free platforms such as Windows or OSX. The system must provide native support for the tools of choice on those platforms. In practice this means native support for Visual Studio and XCode. Having said IDEs invoke external builder binaries does not count as native support.
5. Must not add complexity due to obsolete platforms
Work on this build system started during the Christmas holidays of 2012. This provides a natural hard cutoff line of 2012/12/24. Any platform, tool or library that was not in active use at that time is explicitly not supported. These include Unixes such as IRIX, SunOS, OSF-1, Ubuntu versions older than 12/10, GCC versions older than 4.7 and so on. If these old versions happen to work, great. If they don't, not a single line of code will be added to the system to work around their bugs.
6. Must be fast
Running the configuration step on a moderate sized project must not take more than five seconds. Running the compile command on a fully up to date tree of 1000 source files must not take more than 0.1 seconds.
7. Must provide easy to use support for modern sw development features
An example is precompiled headers. Currently no free software build system provides native support for them. Other examples could include easy integration of Valgrind and unit tests, test coverage reporting and so on.
8. Must allow override of default values
Sometimes you just have to compile files with only given compiler flags and no others, or install files in weird places. The system must allow the user to do this if he really wants to.
Overview of the solution
Going over these requirements it becomes quite apparent that the only viable approach is roughly the same as taken by CMake: having a domain specific language to declare the build system. Out of this declaration a configuration is generated for the backend build system. This can be a Makefile, Visual Studio or XCode project or anything else.
The difference between the proposed DSL and existing ones is that the new one is declarative. It also tries to work on a higher level of abstraction than existing systems. As an example, using external libraries in current build systems means manually extracting and passing around compiler flags and linker flags. In the proposed system the user just declares that a given build target uses a given external dependency. The build system then takes care of passing all flags and settings to their proper locations. This means that the user can focus on his own code rather than marshalling command line arguments from one place to another.
A DSL is more work than the approach taken by SCons, which is to provide the system as a Python library. However it allows us to make the syntax more expressive and prevent certain types of bugs by e.g. making certain objects truly immutable. The end result is again the same: less work for the user.
The backend for Unix requires a bit more thought. The default choice would be Make. However it is extremely slow. It is not uncommon on large code bases for Make to take several minutes just to determine that nothing needs to be done. Instead of Make we use Ninja, which is extremely fast. The backend code is abstracted away from the core, so other backends can be added with relatively little effort.
Sample code
Enough design talk, let's get to the code. Before looking at the examples we would like to emphasize that this is not in any way the final code. It is proof of concept code that works in the system as it currently exists (February 2013), but may change at any time.
Let's start simple. Here is the code to compile a single executable binary.
project('compile one', 'c')
executable('program', 'prog.c')
This is about as simple as one can get. First you declare the project name and the languages it uses. Then you specify the binary to build and its sources. The build system will do all the rest. It will add proper suffixes (e.g. '.exe' on Windows), set the default compiler flags and so on.
Usually programs have more than one source file. Listing them all in the function call can become unwieldy. That is why the system supports keyword arguments. They look like this.
project('compile several', 'c')
sourcelist = ['main.c', 'file1.c', 'file2.c', 'file3.c']
executable('program', sources : sourcelist)
External dependencies are simple to use.
project('external lib', 'c')
libdep = find_dep('extlibrary', required : true)
sourcelist = ['main.c', 'file1.c', 'file2.c', 'file3.c']
executable('program', sources : sourcelist, dep : libdep)
In other build systems you have to manually add the compile and link flags from external dependencies to targets. In this system you just declare that extlibrary is mandatory and that the generated program uses that. The build system does all the plumbing for you.
Here's a slightly more complicated definition. It should still be understandable.
project('build library', 'c')
foolib = shared_library('foobar', sources : 'foobar.c',\
install : true)
exe = executable('testfoobar', 'tester.c', link : foolib)
add_test('test library', exe)
First we build a shared library named foobar. It is marked
installable, so running ninja install installs it to the library
directory (the system knows which one so the user does not have to
care). Then we build a test executable which is linked against the
library. It will no tbe installed, but instead it is added to the list
of unit tests, which can be run with the command ninja test.
Above we mentioned precompiled headers as a feature not supported by other build systems. Here's how you would use them.
project('pch demo', 'cxx')
executable('myapp', 'myapp.cpp', pch : 'pch/myapp.hh')
The main reason other build systems can not provide pch support this easily is because they don't enforce certain best practices. Due to the way include paths work, it is impossible to provide pch support that always works with both in-source and out-of-source builds. Mandating separate build and source directories makes this and many other problems a lot easier.
Get the code
The code for this experiment can be found at the Meson repository. It should be noted that it is not a build system. It is only a proposal for one. It does not work reliably yet. You probably should not use it as the build system of your project.
All that said I hope that this experiment will eventually turn into a full blown build system. For that I need your help. Comments and especially patches are more than welcome.