Chiebot-Mirror
/
meson
mirror of https://github.com/mesonbuild/meson.git
8
0
Fork 0
Code Issues Projects Releases Wiki Activity

Add b_thinlto_cache for automatically configuring incremental ThinLTO

Browse Source
pull/10917/head
Tatsuyuki Ishi 3 years ago committed by Eli Schwartz
parent a0032480d6
commit 673dca5c07
7 changed files with 69 additions and 25 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 36
      docs/markdown/Builtin-options.md
  2. 8
      docs/markdown/snippets/thinlto_cache.md
  3. 3
      mesonbuild/backend/ninjabackend.py
  4. 16
      mesonbuild/compilers/compilers.py
  5. 10
      mesonbuild/compilers/mixins/clang.py
  6. 3
      mesonbuild/compilers/mixins/islinker.py
  7. 18
      mesonbuild/linkers/linkers.py

36
docs/markdown/Builtin-options.md
Unescape View File

@ -121,23 +121,25 @@ no options.
The following options are available. Note that they may not be The following options are available. Note that they may not be
available on all platforms or with all compilers: available on all platforms or with all compilers:
| Option | Default value | Possible values | Description | | Option | Default value | Possible values | Description |
|---------------|----------------|------------------------------------------------------------------|-------------------------------------------------------------------------------| |---------------------|----------------------|---------------------------------------------------------------|--------------------------------------------------------------------------------|
| b_asneeded | true | true, false | Use -Wl,--as-needed when linking | | b_asneeded | true | true, false | Use -Wl,--as-needed when linking |
| b_bitcode | false | true, false | Embed Apple bitcode, see below | | b_bitcode | false | true, false | Embed Apple bitcode, see below |
| b_colorout | always | auto, always, never | Use colored output | | b_colorout | always | auto, always, never | Use colored output |
| b_coverage | false | true, false | Enable coverage tracking | | b_coverage | false | true, false | Enable coverage tracking |
| b_lundef | true | true, false | Don't allow undefined symbols when linking | | b_lundef | true | true, false | Don't allow undefined symbols when linking |
| b_lto | false | true, false | Use link time optimization | | b_lto | false | true, false | Use link time optimization |
| b_lto_threads | 0 | Any integer* | Use multiple threads for lto. *(Added in 0.57.0)* | | b_lto_threads | 0 | Any integer* | Use multiple threads for lto. *(Added in 0.57.0)* |
| b_lto_mode | default | default, thin | Select between lto modes, thin and default. *(Added in 0.57.0)* | | b_lto_mode | default | default, thin | Select between lto modes, thin and default. *(Added in 0.57.0)* |
| b_ndebug | false | true, false, if-release | Disable asserts | | b_thinlto_cache | false | true, false | Enable LLVM's ThinLTO cache for faster incremental builds. *(Added in 0.64.0)* |
| b_pch | true | true, false | Use precompiled headers | | b_thinlto_cache_dir | (Internal build dir) | true, false | Specify where to store ThinLTO cache objects. *(Added in 0.64.0)* |
| b_pgo | off | off, generate, use | Use profile guided optimization | | b_ndebug | false | true, false, if-release | Disable asserts |
| b_sanitize | none | see below | Code sanitizer to use | | b_pch | true | true, false | Use precompiled headers |
| b_staticpic | true | true, false | Build static libraries as position independent | | b_pgo | off | off, generate, use | Use profile guided optimization |
| b_pie | false | true, false | Build position-independent executables (since 0.49.0) | | b_sanitize | none | see below | Code sanitizer to use |
| b_vscrt | from_buildtype | none, md, mdd, mt, mtd, from_buildtype, static_from_buildtype | VS runtime library to use (since 0.48.0) (static_from_buildtype since 0.56.0) | | b_staticpic | true | true, false | Build static libraries as position independent |
| b_pie | false | true, false | Build position-independent executables (since 0.49.0) |
| b_vscrt | from_buildtype | none, md, mdd, mt, mtd, from_buildtype, static_from_buildtype | VS runtime library to use (since 0.48.0) (static_from_buildtype since 0.56.0) |
The value of `b_sanitize` can be one of: `none`, `address`, `thread`, The value of `b_sanitize` can be one of: `none`, `address`, `thread`,
`undefined`, `memory`, `leak`, `address,undefined`, but note that some `undefined`, `memory`, `leak`, `address,undefined`, but note that some

8
docs/markdown/snippets/thinlto_cache.md
Unescape View File

@ -0,0 +1,8 @@
## Incremental ThinLTO with `b_thinlto_cache`
[Incremental ThinLTO](https://clang.llvm.org/docs/ThinLTO.html#incremental) can now be enabled by passing
`-Db_thinlto_cache=true` during setup. The use of caching speeds up incremental builds significantly while retaining all
the runtime performance benefits of ThinLTO.
The cache location defaults to a Meson-managed directory inside the build folder, but can be customized with
`b_thinlto_cache_dir`.

3
mesonbuild/backend/ninjabackend.py
Unescape View File

@ -3160,7 +3160,8 @@ https://gcc.gnu.org/bugzilla/show_bug.cgi?id=47485'''))
else: else:
commands += compilers.get_base_link_args(target.get_options(), commands += compilers.get_base_link_args(target.get_options(),
linker, linker,
isinstance(target, build.SharedModule)) isinstance(target, build.SharedModule),
self.environment.get_build_dir())
# Add -nostdlib if needed; can't be overridden # Add -nostdlib if needed; can't be overridden
commands += self.get_no_stdlib_link_args(target, linker) commands += self.get_no_stdlib_link_args(target, linker)
# Add things like /NOLOGO; usually can't be overridden # Add things like /NOLOGO; usually can't be overridden

16
mesonbuild/compilers/compilers.py
Unescape View File

@ -281,11 +281,12 @@ clike_debug_args = {False: [],
base_options: 'KeyedOptionDictType' = { base_options: 'KeyedOptionDictType' = {
OptionKey('b_pch'): coredata.UserBooleanOption('Use precompiled headers', True), OptionKey('b_pch'): coredata.UserBooleanOption('Use precompiled headers', True),
OptionKey('b_lto'): coredata.UserBooleanOption('Use link time optimization', False), OptionKey('b_lto'): coredata.UserBooleanOption('Use link time optimization', False),
OptionKey('b_lto'): coredata.UserBooleanOption('Use link time optimization', False),
OptionKey('b_lto_threads'): coredata.UserIntegerOption('Use multiple threads for Link Time Optimization', (None, None, 0)), OptionKey('b_lto_threads'): coredata.UserIntegerOption('Use multiple threads for Link Time Optimization', (None, None, 0)),
OptionKey('b_lto_mode'): coredata.UserComboOption('Select between different LTO modes.', OptionKey('b_lto_mode'): coredata.UserComboOption('Select between different LTO modes.',
['default', 'thin'], ['default', 'thin'],
'default'), 'default'),
OptionKey('b_thinlto_cache'): coredata.UserBooleanOption('Use LLVM ThinLTO caching for faster incremental builds', False),
OptionKey('b_thinlto_cache_dir'): coredata.UserStringOption('Directory to store ThinLTO cache objects', ''),
OptionKey('b_sanitize'): coredata.UserComboOption('Code sanitizer to use', OptionKey('b_sanitize'): coredata.UserComboOption('Code sanitizer to use',
['none', 'address', 'thread', 'undefined', 'memory', 'leak', 'address,undefined'], ['none', 'address', 'thread', 'undefined', 'memory', 'leak', 'address,undefined'],
'none'), 'none'),
@ -383,13 +384,19 @@ def get_base_compile_args(options: 'KeyedOptionDictType', compiler: 'Compiler')
return args return args
def get_base_link_args(options: 'KeyedOptionDictType', linker: 'Compiler', def get_base_link_args(options: 'KeyedOptionDictType', linker: 'Compiler',
is_shared_module: bool) -> T.List[str]: is_shared_module: bool, build_dir: str) -> T.List[str]:
args = [] # type: T.List[str] args = [] # type: T.List[str]
try: try:
if options[OptionKey('b_lto')].value: if options[OptionKey('b_lto')].value:
thinlto_cache_dir = None
if get_option_value(options, OptionKey('b_thinlto_cache'), False):
thinlto_cache_dir = get_option_value(options, OptionKey('b_thinlto_cache_dir'), '')
if thinlto_cache_dir == '':
thinlto_cache_dir = os.path.join(build_dir, 'meson-private', 'thinlto-cache')
args.extend(linker.get_lto_link_args( args.extend(linker.get_lto_link_args(
threads=get_option_value(options, OptionKey('b_lto_threads'), 0), threads=get_option_value(options, OptionKey('b_lto_threads'), 0),
mode=get_option_value(options, OptionKey('b_lto_mode'), 'default'))) mode=get_option_value(options, OptionKey('b_lto_mode'), 'default'),
thinlto_cache_dir=thinlto_cache_dir))
except KeyError: except KeyError:
pass pass
try: try:
@ -975,7 +982,8 @@ class Compiler(HoldableObject, metaclass=abc.ABCMeta):
def get_lto_compile_args(self, *, threads: int = 0, mode: str = 'default') -> T.List[str]: def get_lto_compile_args(self, *, threads: int = 0, mode: str = 'default') -> T.List[str]:
return [] return []
def get_lto_link_args(self, *, threads: int = 0, mode: str = 'default') -> T.List[str]: def get_lto_link_args(self, *, threads: int = 0, mode: str = 'default',
thinlto_cache_dir: T.Optional[str] = None) -> T.List[str]:
return self.linker.get_lto_args() return self.linker.get_lto_args()
def sanitizer_compile_args(self, value: str) -> T.List[str]: def sanitizer_compile_args(self, value: str) -> T.List[str]:

10
mesonbuild/compilers/mixins/clang.py
Unescape View File

@ -53,7 +53,8 @@ class ClangCompiler(GnuLikeCompiler):
super().__init__() super().__init__()
self.defines = defines or {} self.defines = defines or {}
self.base_options.update( self.base_options.update(
{OptionKey('b_colorout'), OptionKey('b_lto_threads'), OptionKey('b_lto_mode')}) {OptionKey('b_colorout'), OptionKey('b_lto_threads'), OptionKey('b_lto_mode'), OptionKey('b_thinlto_cache'),
OptionKey('b_thinlto_cache_dir')})
# TODO: this really should be part of the linker base_options, but # TODO: this really should be part of the linker base_options, but
# linkers don't have base_options. # linkers don't have base_options.
@ -163,8 +164,13 @@ class ClangCompiler(GnuLikeCompiler):
args.extend(super().get_lto_compile_args(threads=threads)) args.extend(super().get_lto_compile_args(threads=threads))
return args return args
def get_lto_link_args(self, *, threads: int = 0, mode: str = 'default') -> T.List[str]: def get_lto_link_args(self, *, threads: int = 0, mode: str = 'default',
thinlto_cache_dir: T.Optional[str] = None) -> T.List[str]:
args = self.get_lto_compile_args(threads=threads, mode=mode) args = self.get_lto_compile_args(threads=threads, mode=mode)
if mode == 'thin' and thinlto_cache_dir is not None:
# We check for ThinLTO linker support above in get_lto_compile_args, and all of them support
# get_thinlto_cache_args as well
args.extend(self.linker.get_thinlto_cache_args(thinlto_cache_dir))
# In clang -flto-jobs=0 means auto, and is the default if unspecified, just like in meson # In clang -flto-jobs=0 means auto, and is the default if unspecified, just like in meson
if threads > 0: if threads > 0:
if not mesonlib.version_compare(self.version, '>=4.0.0'): if not mesonlib.version_compare(self.version, '>=4.0.0'):

3
mesonbuild/compilers/mixins/islinker.py
Unescape View File

@ -48,7 +48,8 @@ class BasicLinkerIsCompilerMixin(Compiler):
def sanitizer_link_args(self, value: str) -> T.List[str]: def sanitizer_link_args(self, value: str) -> T.List[str]:
return [] return []
def get_lto_link_args(self, *, threads: int = 0, mode: str = 'default') -> T.List[str]: def get_lto_link_args(self, *, threads: int = 0, mode: str = 'default',
thinlto_cache_dir: T.Optional[str] = None) -> T.List[str]:
return [] return []
def can_linker_accept_rsp(self) -> bool: def can_linker_accept_rsp(self) -> bool:

18
mesonbuild/linkers/linkers.py
Unescape View File

@ -456,6 +456,9 @@ class DynamicLinker(metaclass=abc.ABCMeta):
def get_lto_args(self) -> T.List[str]: def get_lto_args(self) -> T.List[str]:
return [] return []
def get_thinlto_cache_args(self, path: str) -> T.List[str]:
return []
def sanitizer_args(self, value: str) -> T.List[str]: def sanitizer_args(self, value: str) -> T.List[str]:
return [] return []
@ -813,6 +816,9 @@ class AppleDynamicLinker(PosixDynamicLinkerMixin, DynamicLinker):
return (args, set()) return (args, set())
def get_thinlto_cache_args(self, path: str) -> T.List[str]:
return ["-Wl,-cache_path_lto," + path]
class GnuDynamicLinker(GnuLikeDynamicLinkerMixin, PosixDynamicLinkerMixin, DynamicLinker): class GnuDynamicLinker(GnuLikeDynamicLinkerMixin, PosixDynamicLinkerMixin, DynamicLinker):
@ -826,6 +832,9 @@ class GnuGoldDynamicLinker(GnuDynamicLinker):
id = 'ld.gold' id = 'ld.gold'
def get_thinlto_cache_args(self, path: str) -> T.List[str]:
return ['-Wl,-plugin-opt,cache-dir=' + path]
class GnuBFDDynamicLinker(GnuDynamicLinker): class GnuBFDDynamicLinker(GnuDynamicLinker):
@ -836,6 +845,9 @@ class MoldDynamicLinker(GnuDynamicLinker):
id = 'ld.mold' id = 'ld.mold'
def get_thinlto_cache_args(self, path: str) -> T.List[str]:
return ['-Wl,--thinlto-cache-dir=' + path]
class LLVMDynamicLinker(GnuLikeDynamicLinkerMixin, PosixDynamicLinkerMixin, DynamicLinker): class LLVMDynamicLinker(GnuLikeDynamicLinkerMixin, PosixDynamicLinkerMixin, DynamicLinker):
@ -862,6 +874,9 @@ class LLVMDynamicLinker(GnuLikeDynamicLinkerMixin, PosixDynamicLinkerMixin, Dyna
return self._apply_prefix('--allow-shlib-undefined') return self._apply_prefix('--allow-shlib-undefined')
return [] return []
def get_thinlto_cache_args(self, path: str) -> T.List[str]:
return ['-Wl,--thinlto-cache-dir=' + path]
class WASMDynamicLinker(GnuLikeDynamicLinkerMixin, PosixDynamicLinkerMixin, DynamicLinker): class WASMDynamicLinker(GnuLikeDynamicLinkerMixin, PosixDynamicLinkerMixin, DynamicLinker):
@ -1304,6 +1319,9 @@ class ClangClDynamicLinker(VisualStudioLikeLinkerMixin, DynamicLinker):
def get_win_subsystem_args(self, value: str) -> T.List[str]: def get_win_subsystem_args(self, value: str) -> T.List[str]:
return self._apply_prefix([f'/SUBSYSTEM:{value.upper()}']) return self._apply_prefix([f'/SUBSYSTEM:{value.upper()}'])
def get_thinlto_cache_args(self, path: str) -> T.List[str]:
return ["/lldltocache:" + path]
class XilinkDynamicLinker(VisualStudioLikeLinkerMixin, DynamicLinker): class XilinkDynamicLinker(VisualStudioLikeLinkerMixin, DynamicLinker):

Write Preview
Loading…
Cancel
Save