The old caching was a mess of spaghetti code layered over pasta code.
The new code is well-commented, is clear about what it's trying to do,
and uses a blacklist of keyword arguments instead of a whitelist while
generating identifiers for dep caching which makes it much more robust
for future changes.
The only side-effect of forgetting about a new keyword argument would
be that the dependency would not be cached unless the values of that
keyword arguments were the same in the cached and new dependency.
There are also more tests which identify scenarios that were broken
earlier.
All our cached_dep magic was totally useless since we ended up using
the same identifier for native and cross deps. Just nuke all this
cached_dep code since it is very error-prone and improve the
identifier generation instead.
For instance, this is broken *right now* with the `type_name` kwarg.
Add a bunch of tests to ensure that all this actually works...
Closes https://github.com/mesonbuild/meson/issues/1736
This adds a set of private flags that are removed from the output of
`llvm-config --cppflags` before storing them. This is unfortunately
necessary since some versions of LLVM include absolute garbage like
'-DNDEBUG' in their CPP flags, and we don't want to pass those.
Refactor to use ExternalProgram for the command instead of duplicating
that code (badly). Also improve messages to say "or not executable"
when a script/command is not found.
Also allow ExternalPrograms to be passed as arguments to
run_command(). The only thing we're doing by preventing that is
forcing people to use prog.path()
This adds a depdendncy wrapper for llvm-config based on the wxwidgets
dependency. IT handles libs, version, include dir, and the llvm unique
concept of components. These components are individual pieces of the
LLVM library that may or may not be available depending on the platform.
Meson has a common pattern of using 'if len(foo) == 0:' or
'if len(foo) != 0:', however, this is a common anti-pattern in python.
Instead tests for emptiness/non-emptiness should be done with a simple
'if foo:' or 'if not foo:'
Consider the following:
>>> import timeit
>>> timeit.timeit('if len([]) == 0: pass')
0.10730923599840025
>>> timeit.timeit('if not []: pass')
0.030033907998586074
>>> timeit.timeit('if len(['a', 'b', 'c', 'd']) == 0: pass')
0.1154778649979562
>>> timeit.timeit("if not ['a', 'b', 'c', 'd']: pass")
0.08259823200205574
>>> timeit.timeit('if len("") == 0: pass')
0.089759664999292
>>> timeit.timeit('if not "": pass')
0.02340641999762738
>>> timeit.timeit('if len("foo") == 0: pass')
0.08848102600313723
>>> timeit.timeit('if not "foo": pass')
0.04032287199879647
And for the one additional case of 'if len(foo.strip()) == 0', which can
be replaced with 'if not foo.isspace()'
>>> timeit.timeit('if len(" ".strip()) == 0: pass')
0.15294511600222904
>>> timeit.timeit('if " ".isspace(): pass')
0.09413968399894657
>>> timeit.timeit('if len(" abc".strip()) == 0: pass')
0.2023209120015963
>>> timeit.timeit('if " abc".isspace(): pass')
0.09571301700270851
In other words, it's always a win to not use len(), when you don't
actually want to check the length.
When cross compiling and looking for moc/uic/rcc you really want the
host binary.
Still fall back to QT_INSTALL_BINS as it appears that's the only
variable available with qt4.
Ideally, all dependency objects should support this, but it's a lot of
work and isn't supported by all dependency types (like frameworks and
pkg-config), so for now just enable it for external libraries.
... based on the compiler object
This addresses issue #1569
Need to be careful about using -isystem with the standard include
dirs (thanks to the unit tests for catching this). It may be
worthwhile trying to detect what the include dirs are.
Other dependencies (GTest) just avoid adding the include dir for those
system includes. Do the same here.
configure a detection method, for those types of dependencies that have
more than one means of detection.
The default detection methods are unchanged if 'method' is not
specified, and all dependencies support the method 'auto', which is the
same as not specifying a method.
The dependencies which do support multiple detection methods
additionally support other values, depending on the dependency.
Valgrind is a bit of a strange beast, in general use one isn't supposed
to link against valgrinds libs, they're non-PIC static libs, instead,
including the headers does magic to make valgrind work.
This patch implements a simple ValgrindDependency class subclassed from
PkgConfigDependency, that overwrites (effectively) only the
get_link_args method to always return an empty list. This solution may
seem strange, but I think that it follows the principle of least
surprise, and simplifies the most common use case for valgrind.
Essentially without this every valgrind consumer would be forced to
implement the following code to have a usable valgrind dependency
object:
_dep = dependency('valgrind', required : false)
if _dep.found()
valgrind_dep = declare_dependency(
compile_args : _dep.get_pkgconfig_variable('Cflags')
)
else
valgrind_dep = []
endif
While the above is workable, it's surprising behavior and the above code
snippet becomes boilerplate that everyone needs to copy into their meson
files.
Fixes#826
Ever since we changed how we do library searching, the full path to the
library has not been available under `.fullpath`. This has been broken
for at least a year...
And actually test that prog.path() works. The earlier test was just
running the command without checking if it succeeded.
Also make everything use prog.get_command() or get_path() instead of
accessing the internal member prog.fullpath directly.
We also need to check whether the program found in PATH can be executed
directly by Windows or if we need to figure out what the interpreter is
and add it to the list.
Also add `msc` to the list of extensions that can be executed natively
Includes a project test and a unit test for this and all expected
behaviours on Windows.
At the same time, also fix the order in which compile arguments are
added. Detailed comments have been added concerning the priority and
order of the arguments.
Also adds a unit test and an integration test for the same.
While reading shebangs, when we detect an attempt to run 'python3', use
sys.executable instead. For example:
#!/usr/bin/python3
#!python3
#!/usr/bin/env python3
Fixes:
Traceback (most recent call last):
File "mesonbuild/mesonmain.py", line 295, in run
app.generate()
File "mesonbuild/mesonmain.py", line 177, in generate
intr.run()
File "mesonbuild/interpreter.py", line 2444, in run
super().run()
File "mesonbuild/interpreterbase.py", line 124, in run
self.evaluate_codeblock(self.ast, start=1)
File "mesonbuild/interpreterbase.py", line 145, in evaluate_codeblock
raise e
File "mesonbuild/interpreterbase.py", line 139, in evaluate_codeblock
self.evaluate_statement(cur)
File "mesonbuild/interpreterbase.py", line 152, in evaluate_statement
return self.assignment(cur)
File "mesonbuild/interpreterbase.py", line 546, in assignment
value = self.evaluate_statement(node.value)
File "mesonbuild/interpreterbase.py", line 150, in evaluate_statement
return self.function_call(cur)
File "mesonbuild/interpreterbase.py", line 371, in function_call
return self.funcs[func_name](node, self.flatten(posargs), kwargs)
File "mesonbuild/interpreter.py", line 1876, in func_dependency
found = cached_dep.get_version()
File "mesonbuild/dependencies.py", line 369, in get_version
return self.modversion
AttributeError: 'WxDependency' object has no attribute 'modversion'
ninja: error: rebuilding 'build.ninja': subcommand failed
Fetching cflags and libs can also fail if, for instance, the pkg-config
file for a dependency needed by this package isn't found. Without this,
we will print "Found: YES" and then "Found: NO".
Without this, macOS users can't figure out why a particular dependency
wasn't found because the pkg-config error message gets masked by the
fresh and useless DependencyException.
Don't need to define __init__ and manually call the parent init. Doing
so messes up the error message you get by doing str(exception) because
it includes the current class name in it repeatedly.
We were storing the pkg-config path on the class as a static variable --
PkgConfigDependency.pkgbin. When we regenerate via Ninja, the found-deps
are all cached in a pickled coredata, so if you have a statement that
does dep.get_pkgconfig_variable(), it will raise the following exception
because the static class variables are per-meson-invocation.
To fix this, we store the pkg-config binary as both a class variable and
an instance variable.
Traceback (most recent call last):
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/mesonmain.py", line 289, in run
app.generate()
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/mesonmain.py", line 177, in generate
intr.run()
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreter.py", line 2215, in run
super().run()
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 124, in run
self.evaluate_codeblock(self.ast, start=1)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 145, in evaluate_codeblock
raise e
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 139, in evaluate_codeblock
self.evaluate_statement(cur)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 150, in evaluate_statement
return self.function_call(cur)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 371, in function_call
return self.funcs[func_name](node, self.flatten(posargs), kwargs)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 47, in wrapped
return f(self, node, args, kwargs)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreter.py", line 2035, in func_subdir
self.evaluate_codeblock(codeblock)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 145, in evaluate_codeblock
raise e
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 139, in evaluate_codeblock
self.evaluate_statement(cur)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 160, in evaluate_statement
return self.evaluate_if(cur)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 213, in evaluate_if
self.evaluate_codeblock(i.block)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 145, in evaluate_codeblock
raise e
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 139, in evaluate_codeblock
self.evaluate_statement(cur)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 160, in evaluate_statement
return self.evaluate_if(cur)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 213, in evaluate_if
self.evaluate_codeblock(i.block)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 145, in evaluate_codeblock
raise e
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 139, in evaluate_codeblock
self.evaluate_statement(cur)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 182, in evaluate_statement
return self.evaluate_plusassign(cur)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 333, in evaluate_plusassign
addition = self.evaluate_statement(node.value)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 166, in evaluate_statement
return self.evaluate_arraystatement(cur)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 193, in evaluate_arraystatement
(arguments, kwargs) = self.reduce_arguments(cur.args)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 515, in reduce_arguments
reduced_pos = [self.evaluate_statement(arg) for arg in args.arguments]
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 515, in <listcomp>
reduced_pos = [self.evaluate_statement(arg) for arg in args.arguments]
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 154, in evaluate_statement
return self.method_call(cur)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreterbase.py", line 399, in method_call
return obj.method_call(method_name, self.flatten(args), kwargs)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/interpreter.py", line 982, in method_call
value = fn(state, args, kwargs)
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/modules/gnome.py", line 499, in generate_gir
girdir = dep.get_pkgconfig_variable("girdir")
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/dependencies.py", line 219, in get_pkgconfig_variable
ret, out = self._call_pkgbin(['--variable=' + variable_name, self.name])
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/dependencies.py", line 184, in _call_pkgbin
p, out = Popen_safe([PkgConfigDependency.pkgbin] + args, env=os.environ)[0:2]
File "/home/nirbheek/projects/repositories/github/meson.git/mesonbuild/mesonlib.py", line 392, in Popen_safe
stderr=stderr, **kwargs)
File "/usr/lib64/python3.5/subprocess.py", line 947, in __init__
restore_signals, start_new_session)
File "/usr/lib64/python3.5/subprocess.py", line 1474, in _execute_child
executable = os.fsencode(executable)
File "/usr/lib64/python3.5/os.py", line 862, in fsencode
raise TypeError("expect bytes or str, not %s" % type(filename).__name__)
TypeError: expect bytes or str, not NoneType
FAILED: build.ninja
'/usr/bin/python3' '/home/nirbheek/projects/repositories/github/meson.git/meson.py' --internal regenerate '/home/nirbheek/projects/repositories/gst/gstreamer' '/home/nirbheek/projects/repositories/gst/gstreamer/build' --backend ninja
ninja: error: rebuilding 'build.ninja': subcommand failed
Elliott Sales de Andrade
Nirbheek Chauhan
Dylan Baker
Wade Berrier
Aaron Small
Thibault Saunier
Jussi Pakkanen
Mike Sinkovsky
Igor Gnatenko