Merge pull request #3893 from FFY00/master

Add dlang module (dub support)
pull/3927/head
Jussi Pakkanen 7 years ago committed by GitHub
commit 306fa07f62
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 4
      ciimage/Dockerfile
  2. 56
      docs/markdown/D.md
  3. 34
      docs/markdown/Dependencies.md
  4. 43
      docs/markdown/Dlang-module.md
  5. 1
      docs/sitemap.txt
  6. 166
      mesonbuild/dependencies/base.py
  7. 6
      mesonbuild/interpreter.py
  8. 141
      mesonbuild/modules/dlang.py
  9. 23
      test cases/d/11 dub/meson.build
  10. 14
      test cases/d/11 dub/test.d

@ -12,7 +12,9 @@ RUN apt-get -y update && apt-get -y upgrade \
&& apt-get -y install qt4-linguist-tools \
&& apt-get -y install python-dev \
&& apt-get -y install libomp-dev openssh-client \
&& python3 -m pip install hotdoc codecov
&& python3 -m pip install hotdoc codecov \
&& dub fetch urld \
&& dub build urld --compiler=gdc
# OpenSSH client is needed to run openmpi binaries.

@ -5,7 +5,8 @@ short-description: Compiling D sources
# Compiling D applications
Meson has support for compiling D programs. A minimal `meson.build` file for D looks like this:
Meson has support for compiling D programs. A minimal `meson.build`
file for D looks like this:
```meson
project('myapp', 'd')
@ -15,8 +16,8 @@ executable('myapp', 'app.d')
## Compiling different versions
If you are using the [version()](https://dlang.org/spec/version.html) feature for conditional compilation, you can use it using the `d_module_versions`
target property:
If you are using the [version()](https://dlang.org/spec/version.html) feature for conditional compilation,
you can use it using the `d_module_versions` target property:
```meson
project('myapp', 'd')
executable('myapp', 'app.d', d_module_versions: ['Demo', 'FeatureA'])
@ -24,10 +25,14 @@ executable('myapp', 'app.d', d_module_versions: ['Demo', 'FeatureA'])
## Using embedded unittests
If you are using embedded [unittest functions](https://dlang.org/spec/unittest.html), your source code needs to be compiled twice, once in regular
mode, and once with unittests active. This is done by setting the `d_unittest` target property to `true`.
Meson will only ever pass the respective compiler's `-unittest` flag, and never have the compiler generate an empty main function.
If you need that feature in a portable way, create an empty `main()` function for unittests yourself, since the GNU D compiler
If you are using embedded [unittest functions](https://dlang.org/spec/unittest.html), your source code needs
to be compiled twice, once in regular
mode, and once with unittests active. This is done by setting the
`d_unittest` target property to `true`.
Meson will only ever pass the respective compiler's `-unittest` flag,
and never have the compiler generate an empty main function.
If you need that feature in a portable way, create an empty `main()`
function for unittests yourself, since the GNU D compiler
does not have this feature.
This is an example for using D unittests with Meson:
@ -43,8 +48,10 @@ test('myapptest', test_exe)
# Compiling D libraries and installing them
Building D libraries is a straightforward process, not different from how C libraries are built in Meson. You should generate a pkg-config file
and install it, in order to make other software on the system find the dependency once it is installed.
Building D libraries is a straightforward process, not different from
how C libraries are built in Meson. You should generate a pkg-config
file and install it, in order to make other software on the system
find the dependency once it is installed.
This is an example on how to build a D shared library:
```meson
@ -71,12 +78,17 @@ pkgc.generate(name: 'mylib',
install_subdir('src/mylib/', install_dir: 'include/d/mylib/')
```
It is important to make the D sources install in a subdirectory in the include path, in this case `/usr/include/d/mylib/mylib`.
All D compilers include the `/usr/include/d` directory by default, and if your library would be installed into `/usr/include/d/mylib`, there
is a high chance that, when you compile your project again on a machine where you installed it, the compiler will prefer the old installed include over
the new version in the source tree, leading to very confusing errors.
This is an example of how to use the D library we just built and installed in an application:
It is important to make the D sources install in a subdirectory in the
include path, in this case `/usr/include/d/mylib/mylib`.
All D compilers include the `/usr/include/d` directory by default, and
if your library would be installed into `/usr/include/d/mylib`, there
is a high chance that, when you compile your project again on a
machine where you installed it, the compiler will prefer the old
installed include over the new version in the source tree, leading to
very confusing errors.
This is an example of how to use the D library we just built and
installed in an application:
```meson
project('myapp', 'd')
@ -85,5 +97,15 @@ myapp_src = ['app.d', 'alpha.d', 'beta.d']
executable('myapp', myapp_src, dependencies: [mylib_dep])
```
Please keep in mind that the library and executable would both need to be built with the exact same D compiler and D compiler version. The D ABI is not
stable across compilers and their versions, and mixing compilers will lead to problems.
Please keep in mind that the library and executable would both need to
be built with the exact same D compiler and D compiler version. The D
ABI is not stable across compilers and their versions, and mixing
compilers will lead to problems.
# Integrating with DUB
DUB is a fully integrated build system for D, but it is also a way to
provide dependencies. Adding dependencies from the [D package registry](https://code.dlang.org/)
is pretty straight forward. You can find how to do this in
[Dependencies](Dependencies.md#Dub). You can also automatically
generate a `dub.json` file as explained in [Dlang](Dlang-module.md#generatedubfile).

@ -109,6 +109,38 @@ object. Since they can be used interchangeably, the rest of the build
definitions do not need to care which one it is. Meson will take care
of all the work behind the scenes to make this work.
# Dependency method
You can use the keyword `method` to let meson know what method to use
when searching for the dependency. The default value is `auto`.
Aditional dependencies methods are `pkg-config`, `config-tool`,
`system`, `sysconfig`, `qmake`, `extraframework` and `dub`.
```meson
cups_dep = dependency('cups', method : 'pkg-config')
```
### Some notes on Dub
Please understand that meson is only able to find dependencies that
exist in the local Dub repository. You need to manually fetch and
build the target dependencies.
For `urld`.
```
dub fetch urld
dub build urld
```
Other thing you need to keep in mind is that both meson and Dub need
to be using the same compiler. This can be achieved using Dub's
`-compiler` argument and/or manually setting the `DC` environment
variable when running meson.
```
dub build urld --compiler=dmd
DC="dmd" meson builddir
```
# Dependencies with custom lookup functionality
Some dependencies have specific detection logic.
@ -280,7 +312,7 @@ The `language` keyword may used.
Python3 is handled specially by meson:
1. Meson tries to use `pkg-config`.
1. If `pkg-config` fails meson uses a fallback:
2. If `pkg-config` fails meson uses a fallback:
- On Windows the fallback is the current `python3` interpreter.
- On OSX the fallback is a framework dependency from `/Library/Frameworks`.

@ -0,0 +1,43 @@
# Dlang module
This module provides tools related to the D programming language.
## Usage
To use this module, just do: **`dlang = import('dlang')`**.
You can, of course, replace the name `dlang` with anything else.
The module only exposes one fucntion, `generate_dub_file`, used to
automatically generate Dub configuration files.
### generate_dub_file()
This method only has two required arguments, the project name and the
source folder. You can pass other arguments with additional keywords,
they will be automatically translated to json and added to the
`dub.json` file.
**Structure**
```meson
generate_dub_file("project name", "source/folder", key: "value" ...)
```
**Example**
```meson
dlang = import('dlang')
dlang.generate_dub_file(meson.project_name().to_lower(), meson.source_root(),
authors: 'Meson Team',
description: 'Test executable',
copyright: 'Copyright © 2018, Meson Team',
license: 'MIT',
sourceFiles: 'test.d',
targetType: 'executable',
dependencies: my_dep
)
```
You can manually edit a meson generated `dub.json` file or provide a
initial one. The module will only update the values specified in
`generate_dub_file()`.
Although not required, you will need to have a `description` and
`license` if you want to publish the package in the [D package registry](https://code.dlang.org/).

@ -40,6 +40,7 @@ index.md
RPM-module.md
Simd-module.md
Windows-module.md
Dlang-module.md
Java.md
Vala.md
D.md

@ -19,6 +19,7 @@ import copy
import os
import re
import stat
import json
import shlex
import shutil
import textwrap
@ -31,7 +32,6 @@ from ..compilers import clib_langs
from ..mesonlib import MesonException, OrderedSet
from ..mesonlib import Popen_safe, version_compare_many, version_compare, listify
# These must be defined in this file to avoid cyclical references.
packages = {}
_packages_accept_language = set()
@ -59,6 +59,8 @@ class DependencyMethods(Enum):
CUPSCONFIG = 'cups-config'
PCAPCONFIG = 'pcap-config'
LIBWMFCONFIG = 'libwmf-config'
# Misc
DUB = 'dub'
class Dependency:
@ -790,6 +792,162 @@ class PkgConfigDependency(ExternalDependency):
# a path rather than the raw dlname
return os.path.basename(dlname)
class DubDependency(ExternalDependency):
class_dubbin = None
def __init__(self, name, environment, kwargs):
super().__init__('dub', environment, 'd', kwargs)
self.name = name
self.compiler = super().get_compiler()
if 'required' in kwargs:
self.required = kwargs.get('required')
if DubDependency.class_dubbin is None:
self.dubbin = self.check_dub()
DubDependency.class_dubbin = self.dubbin
else:
self.dubbin = DubDependency.class_dubbin
if not self.dubbin:
if self.required:
raise DependencyException('DUB not found.')
self.is_found = False
mlog.log('Dependency', mlog.bold(name), 'found:', mlog.red('NO'))
return
mlog.debug('Determining dependency {!r} with DUB executable '
'{!r}'.format(name, self.dubbin.get_path()))
# Ask dub for the package
ret, res = self._call_dubbin(['describe', name])
if ret != 0:
if self.required:
raise DependencyException('Dependency {!r} not found'.format(name))
self.is_found = False
mlog.log('Dependency', mlog.bold(name), 'found:', mlog.red('NO'))
return
j = json.loads(res)
comp = self.compiler.get_id().replace('llvm', 'ldc').replace('gcc', 'gdc')
for package in j['packages']:
if package['name'] == name:
if j['compiler'] != comp:
msg = ['Dependency', mlog.bold(name), 'found but it was compiled with']
msg += [mlog.bold(j['compiler']), 'and we are using', mlog.bold(comp)]
mlog.error(*msg)
if self.required:
raise DependencyException('Dependency {!r} not found'.format(name))
self.is_found = False
mlog.log('Dependency', mlog.bold(name), 'found:', mlog.red('NO'))
return
self.version = package['version']
self.pkg = package
break
# Check if package version meets the requirements
found_msg = ['Dependency', mlog.bold(name), 'found:']
if self.version_reqs is None:
self.is_found = True
else:
if not isinstance(self.version_reqs, (str, list)):
raise DependencyException('Version argument must be string or list.')
if isinstance(self.version_reqs, str):
self.version_reqs = [self.version_reqs]
(self.is_found, not_found, found) = \
version_compare_many(self.version, self.version_reqs)
if not self.is_found:
found_msg += [mlog.red('NO'),
'found {!r} but need:'.format(self.version),
', '.join(["'{}'".format(e) for e in not_found])]
if found:
found_msg += ['; matched:',
', '.join(["'{}'".format(e) for e in found])]
if not self.silent:
mlog.log(*found_msg)
if self.required:
m = 'Invalid version of dependency, need {!r} {!r} found {!r}.'
raise DependencyException(m.format(name, not_found, self.version))
return
found_msg += [mlog.green('YES'), self.version]
if self.pkg['targetFileName'].endswith('.a'):
self.static = True
self.compile_args = []
for flag in self.pkg['dflags']:
self.link_args.append(flag)
for path in self.pkg['importPaths']:
self.compile_args.append('-I' + os.path.join(self.pkg['path'], path))
self.link_args = []
for flag in self.pkg['lflags']:
self.link_args.append(flag)
search_paths = []
search_paths.append(os.path.join(self.pkg['path'], self.pkg['targetPath']))
found, res = self.__search_paths(search_paths, self.pkg['targetFileName'])
for file in res:
self.link_args.append(file)
if not found:
if self.required:
raise DependencyException('Dependency {!r} not found'.format(name))
self.is_found = False
mlog.log('Dependency', mlog.bold(name), 'found:', mlog.red('NO'))
return
if not self.silent:
mlog.log(*found_msg)
def get_compiler(self):
return self.compiler
def __search_paths(self, search_paths, target_file):
found = False
res = []
if target_file == '':
return True, res
for path in search_paths:
if os.path.isdir(path):
for file in os.listdir(path):
if file == target_file:
res.append(os.path.join(path, file))
found = True
return found, res
def _call_dubbin(self, args, env=None):
p, out = Popen_safe(self.dubbin.get_command() + args, env=env)[0:2]
return p.returncode, out.strip()
def check_dub(self):
dubbin = ExternalProgram('dub', silent=True)
if dubbin.found():
try:
p, out = Popen_safe(dubbin.get_command() + ['--version'])[0:2]
if p.returncode != 0:
mlog.warning('Found dub {!r} but couldn\'t run it'
''.format(' '.join(dubbin.get_command())))
# Set to False instead of None to signify that we've already
# searched for it and not found it
dubbin = False
except (FileNotFoundError, PermissionError):
dubbin = False
else:
dubbin = False
if dubbin:
mlog.log('Found DUB:', mlog.bold(dubbin.get_path()),
'(%s)' % out.strip())
else:
mlog.log('Found DUB:', mlog.red('NO'))
return dubbin
@staticmethod
def get_methods():
return [DependencyMethods.PKGCONFIG, DependencyMethods.DUB]
class ExternalProgram:
windows_exts = ('exe', 'msc', 'com', 'bat', 'cmd')
@ -1097,6 +1255,7 @@ def find_external_dependency(name, env, kwargs):
raise DependencyException('Keyword "required" must be a boolean.')
if not isinstance(kwargs.get('method', ''), str):
raise DependencyException('Keyword "method" must be a string.')
method = kwargs.get('method', '')
lname = name.lower()
if lname in packages:
if lname not in _packages_accept_language and 'language' in kwargs:
@ -1113,6 +1272,11 @@ def find_external_dependency(name, env, kwargs):
if 'language' in kwargs:
# Remove check when PkgConfigDependency supports language.
raise DependencyException('%s dependency does not accept "language" keyword argument' % (lname, ))
if 'dub' == method:
dubdep = DubDependency(name, env, kwargs)
if required and not dubdep.found():
mlog.log('Dependency', mlog.bold(name), 'found:', mlog.red('NO'))
return dubdep
pkg_exc = None
pkgdep = None
try:

@ -376,6 +376,7 @@ class DependencyHolder(InterpreterObject, ObjectHolder):
self.methods.update({'found': self.found_method,
'type_name': self.type_name_method,
'version': self.version_method,
'name': self.name_method,
'get_pkgconfig_variable': self.pkgconfig_method,
'get_configtool_variable': self.configtool_method,
'partial_dependency': self.partial_dependency_method,
@ -398,6 +399,11 @@ class DependencyHolder(InterpreterObject, ObjectHolder):
def version_method(self, args, kwargs):
return self.held_object.get_version()
@noPosargs
@permittedKwargs({})
def name_method(self, args, kwargs):
return self.held_object.get_name()
@permittedKwargs({'define_variable'})
def pkgconfig_method(self, args, kwargs):
args = listify(args)

@ -0,0 +1,141 @@
# Copyright 2018 The Meson development team
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This file contains the detection logic for external dependencies that
# are UI-related.
import json
import os
from . import ExtensionModule
from .. import mlog
from ..mesonlib import (
Popen_safe, MesonException
)
from ..dependencies.base import (
ExternalProgram, DubDependency
)
from ..interpreter import DependencyHolder
class DlangModule(ExtensionModule):
class_dubbin = None
init_dub = False
def __init__(self, interpreter):
super().__init__(interpreter)
self.snippets.add('generate_dub_file')
def _init_dub(self):
if DlangModule.class_dubbin is None:
self.dubbin = DubDependency.class_dubbin
DlangModule.class_dubbin = self.dubbin
else:
self.dubbin = DlangModule.class_dubbin
if DlangModule.class_dubbin is None:
self.dubbin = self.check_dub()
DlangModule.class_dubbin = self.dubbin
else:
self.dubbin = DlangModule.class_dubbin
if not self.dubbin:
if not self.dubbin:
raise MesonException('DUB not found.')
def generate_dub_file(self, interpreter, state, args, kwargs):
if not DlangModule.init_dub:
self._init_dub()
if len(args) < 2:
raise MesonException('Missing arguments')
config = {
'name': args[0]
}
config_path = os.path.join(args[1], 'dub.json')
if os.path.exists(config_path):
with open(config_path, 'r', encoding='utf8') as ofile:
try:
config = json.load(ofile)
except ValueError:
mlog.warning('Failed to load the data in dub.json')
warn_publishing = ['description', 'license']
for arg in warn_publishing:
if arg not in kwargs and \
arg not in config:
mlog.warning('Without', mlog.bold(arg), 'the DUB package can\'t be published')
for key, value in kwargs.items():
if key == 'dependencies':
config[key] = {}
if isinstance(value, list):
for dep in value:
if isinstance(dep, DependencyHolder):
name = dep.method_call('name', [], [])
ret, res = self._call_dubbin(['describe', name])
if ret == 0:
version = dep.method_call('version', [], [])
if version is None:
config[key][name] = ''
else:
config[key][name] = version
elif isinstance(value, DependencyHolder):
name = value.method_call('name', [], [])
ret, res = self._call_dubbin(['describe', name])
if ret == 0:
version = value.method_call('version', [], [])
if version is None:
config[key][name] = ''
else:
config[key][name] = version
else:
config[key] = value
with open(config_path, 'w', encoding='utf8') as ofile:
ofile.write(json.dumps(config, indent=4, ensure_ascii=False))
def _call_dubbin(self, args, env=None):
p, out = Popen_safe(self.dubbin.get_command() + args, env=env)[0:2]
return p.returncode, out.strip()
def check_dub(self):
dubbin = ExternalProgram('dub', silent=True)
if dubbin.found():
try:
p, out = Popen_safe(dubbin.get_command() + ['--version'])[0:2]
if p.returncode != 0:
mlog.warning('Found dub {!r} but couldn\'t run it'
''.format(' '.join(dubbin.get_command())))
# Set to False instead of None to signify that we've already
# searched for it and not found it
dubbin = False
except (FileNotFoundError, PermissionError):
dubbin = False
else:
dubbin = False
if dubbin:
mlog.log('Found DUB:', mlog.bold(dubbin.get_path()),
'(%s)' % out.strip())
else:
mlog.log('Found DUB:', mlog.red('NO'))
return dubbin
def initialize(*args, **kwargs):
return DlangModule(*args, **kwargs)

@ -0,0 +1,23 @@
project('dub-example', 'd')
dub_exe = find_program('dub', required : false)
if not dub_exe.found()
error('MESON_SKIP_TEST: Dub not found')
endif
urld_dep = dependency('urld', method: 'dub')
test_exe = executable('test-urld', 'test.d', dependencies: urld_dep)
test('test urld', test_exe)
# If you want meson to generate/update a dub.json file
dlang = import('dlang')
dlang.generate_dub_file(meson.project_name().to_lower(), meson.source_root(),
authors: 'Meson Team',
description: 'Test executable',
copyright: 'Copyright © 2018, Meson Team',
license: 'MIT',
sourceFiles: 'test.d',
targetType: 'executable',
dependencies: urld_dep
)

@ -0,0 +1,14 @@
import std.stdio;
import url;
void main() {
URL url;
with (url) {
scheme = "soap.beep";
host = "beep.example.net";
port = 1772;
path = "/serverinfo/info";
queryParams.add("token", "my-api-token");
}
writeln(url);
}
Loading…
Cancel
Save