RFC: what happens when c_args/ld_args come from multiple locations? [skip ci]

Combinatorial explosion and... some documentation gaps?

Consider this is as a github issue with a lot of code.

I tried to find in the documentation all the places {cpp,c,ld}_args
could be set and the relationships between these places. However what I
found was scattered in the corresponding places - no single, high-level
list or overview - and seemed sometimes incomplete.

I searched github for the same topic but that didn't really help either
because most of it is or was of course "work in progress" (the
difference between "is" and "was" being not always immediately obvious)
For instance https://github.com/mesonbuild/meson/issues/4767 states "All
cross compilation arguments come from the cross file". Except for: -D,
project(default_options: c_args...), executable( c_args, native:false),...

Then I looked for some relevant test code that would be in a less
natural language but necessarily in sync with the current
implementation. I couldn't really find any, so I wrote this (long)
sample. While tedious, this helped me understand some of the situation
better.

I'm not sure exactly where to take this next. A new, "<lang>_args hub"
documentation node with an organized list of pointers would probably not
hurt. Another cool thing would be to convert some of this to fully
automated tests and it's probably not too far from it, except for the
important question of: what particular combinations should be tested?
Can't test everything due to the number of locations and the
combinatorial explosion.

test cases/linuxlike/exploreCargs/README.txt

@@ -0,0 +1,90 @@
+What happens when c_args come from multiple places?
+
+Combinatorial explosion and... some documentation gaps?
+
+I tried to find in the documentation all the places {cpp,c,ld}_args
+could be set and the relationships between these places. However what I
+found was scattered in the corresponding places - no single, high-level
+list or overview - and seemed sometimes incomplete.
+
+I searched github for the same topic but that didn't really help either
+because most of it is or was of course "work in progress" (the
+difference between "is" and "was" being not always immediately obvious)
+For instance https://github.com/mesonbuild/meson/issues/4767 states "All
+cross compilation arguments come from the cross file". Except for: -D,
+project(default_options: c_args...), executable( c_args, native:false),...
+
+Then I looked for some relevant test code that would be in a less
+natural language but necessarily in sync with the current
+implementation. I couldn't really find any, so I wrote this (long)
+sample. While tedious, this helped me understand some of the situation
+better.
+
+I'm not sure exactly where to take this next. A new, "<lang>_args hub"
+documentation node with an organized list of pointers would probably not
+hurt. Another cool thing would be to convert some of this to fully
+automated tests and it's probably not too far from it, except for the
+important question of: what particular combinations should be tested?
+Can't test everything due to the number of locations and the
+combinatorial explosion.
+
+
+Locations "under test":
+- environment C/CPP/LDFLAGS
+- meson setup -Dc*_args=-D...
+- meson setup --cross-file c*_args
+- project (default_options c*_args)
+- executable (c*_args)
+
+Any I forgot?
+- CFLAGS_FOR_BUILD ?
+- multiple --cross-file (https://github.com/mesonbuild/meson/issues/3878)
+
+
+Results below reproduced with this test directory + meson version 9c72d0fdb287.
+
+
+A. My top issue: while some c*_args of different locations append to
+each other (in which order?), other combinations of c*_args overwrite
+one another and it feels difficult to predict what will happen when.
+
+1 "meson setup -D[build.]c_args=..." overrides"
+2. "project(default_options: [build.]c_args)", which overrides:
+3. env LD/C/CPPFLAGS and --cross-file c_args.
+
+Either of the above _combines_ with "executable(c_args)". And with
+everything else? It could work differently, for instance one could
+assume "project(default_options: )" means "default when no
+executable(c_args)" instead.
+
+I would also like -Dbuild.c_args to combine with --cross-file c_args,
+because why not?
+
+
+B. environment CPP/C/LDFLAGS
+
+Granted: environment variables are discouraged. Still, it would be nice
+to have some clearer idea how they work right now, even if nothing's
+guaranteed in the future.
+
+When cross-compiling, the env CFLAGS and LDFLAGS affects only the build
+machine. Documented?
+
+project(build.c_args) overrides BOTH env CPPCFLAGS and CFLAGS!
+project(build.cpp_args) does... nothing ever?
+
+
+C. Some "unknown options: build.c_*_args, ... " are actually used by meson.
+
+ meson setup -Dbuild.cpp_args=... -Dbuild.c_args=... -Dbuild.c_link_args=... reports:
+
+  WARNING: Unknown options: "build.c_*_args, ...
+
+ It does drop build.cpp_args as claimed, however it uses the other two
+ anyway when native:true.
+
+
+D. --cross_file cpp_args are silently ignored. It's because everything
+   is free-form in a --cross-file as documented, however it's another
+   example of the "ugly duckling" status of cpp_args in general.
+

test cases/linuxlike/exploreCargs/c_args_manual_test.sh

@@ -0,0 +1,75 @@
+#!/bin/sh
+
+set -e
+set -x
+
+MESON=~/meson/meson.py
+
+
+# "-Wl,--undefined=_test_value" has for the linker useful test
+# properties similar to how D_test_value behaves for the preprocessor:
+# visible and testable yet "pass-through" and completely harmless. In a
+# way it's even better than -D because a dummy test program ends up with
+# a large noise of -D macros that the toolchain adds by default, whereas
+# "-Wl,--undefined=_test_value" doesn't even need to be filtered.
+
+do_bld()
+{
+    local bld="$1"; shift
+    if true; then
+        ${MESON} setup \
+             -Dcpp_args=-DCLIsetup_CPP_args=___YES_CLIsetup___ \
+             -Dbuild.cpp_args=-DCLIsetup_buildm_CPP_args=___YES_CLIsetup_buildm____ \
+             -Dc_args=-DCLIsetup_C_args=___YES_CLIsetup____ \
+             -Dbuild.c_args=-DCLIsetup_buildm_C_args=___YES_CLIsetup_buildm____ \
+             -Dc_link_args=-Wl,--undefined=CLIsetup_C_link_args \
+             -Dbuild.c_link_args=-Wl,--undefined=CLIsetup_buildm_C_link_args \
+             "$bld" "$@"
+    else
+        ${MESON} setup  "$bld" "$@"
+    fi
+
+    ${MESON} configure "$bld" | grep -C2 args
+    ninja -C "$bld" print_flags
+
+    if true; then
+        jq . "${bld}"/meson-info/intro-buildoptions.json | grep -C 5 args
+        jq . "${bld}"/meson-info/intro-targets.json | grep -C 8 _args
+    fi
+}
+
+main()
+{
+    local YES_SIR='___YES_ENV___'
+
+    # envvars considered harmful:
+    #   https://github.com/mesonbuild/meson/issues/4664
+    if true; then
+        export CPPFLAGS="-Denv_CPPFLAGS=${YES_SIR}"
+        export CFLAGS="-Denv_CFLAGS=${YES_SIR}"
+        export LDFLAGS='-Wl,--undefined=env_LDFLAGS'
+    fi
+
+    # --wipe fails when bld*/ is missing, so let's not use it.
+    rm -rf bldnat/ bldcross/
+
+    if false; then
+        printf '\n\n ----  native ---- \n\n'
+        do_bld bldnat
+        ninja -j1 -v -C bldnat print_link_nativetrue
+        ninja -j1 -v -C bldnat print_link_nativenone
+        ninja -j1 -v -C bldnat print_link_nativefalse
+    fi
+
+    # One --cross-file
+    if true; then
+        printf '\n\n ---- cross ----- \n\n'
+        do_bld bldcross --cross-file=cross-clang.ini
+#        ninja -j1 -v -C bldcross print_link_nativetrue
+#        ninja -j1 -v -C bldcross print_link_nativenone
+        ninja -j1 -v -C bldcross print_link_nativefalse
+    fi
+
+}
+
+main "$@"

test cases/linuxlike/exploreCargs/cross-clang.ini

@@ -0,0 +1,21 @@
+[binaries]
+# We assume gcc is the default. clang is conviently different yet
+# cloning most of gcc's user interface. Same for ld.gold.
+c = 'clang'
+ld = 'ld.gold'
+strip = 'strip'
+
+[properties]
+cpp_args = ['-Dcross_file_CPP_args=___YES_crossfile___']
+c_args = ['-Dcross_file_C_args=___YES_crossfile___']
+c_link_args = ['-Wl,--undefined=cross_file_C_link_args']
+
+[host_machine]
+# Use "windows" to avoid "fallback to
+# native" issue(s) like https://github.com/mesonbuild/meson/issues/5102
+# A "bare metal" option would be nice
+# https://github.com/mesonbuild/meson/issues/6063
+system = 'windows'
+cpu_family = 'x86'
+cpu = 'i586'
+endian = 'little'

test cases/linuxlike/exploreCargs/meson.build

@@ -0,0 +1,103 @@
+
+project('c_args test', 'c',
+       default_options: [
+
+          'build.cpp_args=-Dproject_default_options_buildm_CPP_args=___YES_project_defaults__',
+          'build.c_args=-Dproject_default_options_buildm_C_args=___YES_project_defaults__',
+          'build.c_link_args=-Wl,--undefined=project_default_options_buildm_C_link_args',
+
+         'cpp_args=-Dproject_default_options_CPP_args=___YES_project_defaults__',
+          'c_args=-Dproject_default_options_C_args=___YES_project_defaults__',
+          'c_link_args=-Wl,--undefined=project_default_options_C_link_args',
+
+        ]
+)
+
+
+# These also take native:true/false. The default value is: "!is_cross_build()"
+add_global_arguments('-Dadd_global_args=___YES_add_global_args____', language : 'c')
+
+add_project_arguments('-Dadd_project_args=___YES_add_project_args___', language : 'c')
+
+
+YES_SIR = '___YES_executable_nativetrue___'
+exe_true = executable('test_nativetrue.exe', 'test_params.c',     native:true,
+
+          # Never overriden?
+          cpp_args : '-Dexecutable_CPP_args=' + YES_SIR,
+          c_args : '-Dexecutable_C_args=' + YES_SIR,
+          link_args : '-Wl,--undefined=executable_nativetrue_C_link_args',
+
+)
+run_target('print_link_nativetrue',
+      command : [ 'nm', '--undefined-only', exe_true ],
+          )
+
+
+YES_SIR = '___YES_executable_nativenone___'
+exe_none = executable('test_nativenone.exe', 'test_params.c',    # native: none
+
+          cpp_args : '-Dexecutable_CPP_args=' + YES_SIR,
+          c_args : '-Dexecutable_C_args=' + YES_SIR,
+          link_args : '-Wl,--undefined=executable_nativenone_C_link_args',
+)
+run_target('print_link_nativenone',
+      command : [ 'nm', '--undefined-only', exe_none ]
+)
+
+
+YES_SIR = '___YES_executable_nativefalse___'
+exe_false = executable('test_nativefalse.exe', 'test_params.c',   native:false,
+
+          cpp_args : '-Dexecutable_CPP_args=' + YES_SIR,
+          c_args : '-Dexecutable_C_args=' + YES_SIR,
+          link_args : '-Wl,--undefined=executable_nativefalse_C_link_args',
+)
+run_target('print_link_nativefalse',
+      command : [ 'nm', '--undefined-only', exe_false ],
+          )
+
+
+
+
+# To not look ugly, \n and \t need this workaround:
+#   https://github.com/mesonbuild/meson/pull/6241
+run_target('print_flags',
+    command : [
+      'printf',
+
+#      '''\n\nget_option(cpp_args)\t=\t[%s]\n''' +
+      'get_option(c_args)\\t=\\t[%s]\\n' +
+      'get_option(c_link_args)\\t=\\t[%s]\\n' +
+
+#      '''get_option(build.cpp_args)\t=\t[%s]\n''' +
+      '''get_option(build.c_args)\t=\t[%s]\n''' +
+      'get_option(build.c_link_args)\\t=\\t[%s]\\n' +
+
+      'meson.get_cross_property(cpp_args, <empty>)\\t=\\t[%s]\\n' +
+      'meson.get_cross_property(c_args, <empty>)\\t=\\t[%s]\\n' +
+      'meson.get_cross_property(c_link_args, <empty>)\\t=\\t[%s]\\n' +
+      'meson.get_cross_property(fubar, <empty>)\\t=\\t[%s]\\n',
+
+#      ','.join(get_option('cpp_args')),
+      ','.join(get_option('c_args')),
+      ','.join(get_option('c_link_args')),
+
+#      ','.join(get_option('build.cpp_args')),
+      ','.join(get_option('build.c_args')),
+      ','.join(get_option('build.c_link_args')),
+
+# Unlike get_option(), .get_cross_property() is free-form.
+
+      ','.join(meson.get_cross_property('cpp_args',
+                                        ['<EMPTY>'])),
+      ','.join(meson.get_cross_property('c_args',
+                                        ['<EMPTY>'])),
+      ','.join(meson.get_cross_property('c_link_args',
+                                        ['<EMPTY>'])),
+      ','.join(meson.get_cross_property('fubar',
+                                        ['<EMPTY>'])),
+
+           ]
+)
+

test cases/linuxlike/exploreCargs/test_params.c

@@ -0,0 +1,53 @@
+
+// This code logs preprocessor values.
+
+// It would be trivial to change this and assert() them in some
+// automated test(s) instead.
+
+
+#define QUOTE(unquoted) #unquoted
+// QUOTE either "arg" iself, or its expansion if any.
+#define stringify_value(arg) QUOTE(arg)
+
+// As usual, undefined macros "expand to themselves".
+#define name_value(arg) #arg "\texpands to ->\t" stringify_value(arg)
+
+// clang makes #pragma message output look like a warning but -Werror
+// doesn't actually fail
+
+#if 1
+// executable( c_args: ) never overwrites, always appends?
+#pragma message name_value(executable_CPP_args)
+#pragma message name_value(executable_C_args)
+#endif
+
+// In tentative partial order: if defined in two places AND they are
+// mutually exclusive, then the first one should win.
+
+#if 1
+#pragma message name_value(CLIsetup_buildm_CPP_args)
+#pragma message name_value(CLIsetup_buildm_C_args)
+#pragma message name_value(CLIsetup_CPP_args)
+#pragma message name_value(CLIsetup_C_args)
+
+#pragma message name_value(project_default_options_buildm_CPP_args)
+#pragma message name_value(project_default_options_buildm_C_args)
+#pragma message name_value(project_default_options_CPP_args)
+#pragma message name_value(project_default_options_C_args)
+
+#pragma message name_value(env_CPPFLAGS)
+#pragma message name_value(env_CFLAGS)
+
+#pragma message name_value(cross_file_CPP_args)
+#pragma message name_value(cross_file_C_args)
+
+#pragma message name_value(add_global_args)
+#pragma message name_value(add_project_args)
+
+#endif
+
+
+int main(int argc, char *argv[])
+{
+  return 0;
+}