Ensure lld is used for external cmake with clang

[mathetake]

Resolves a comment #42397 (comment)

[repokitteh-read-only]

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #44075 was opened by mathetake.

see: more, trace.

[kralicky]

I ran into a similar issue when building envoy with compiler-rt + libunwind enabled. Looks like if we also configure CMAKE_C_FLAGS here in the same way, the cmake builds will also work correctly with compiler-rt. I used to have this in my bazelrc as a workaround:

common:libc++ --action_env=LDFLAGS="-stdlib=libc++ -fuse-ld=lld -rtlib=compiler-rt"
common:sanitizer --action_env=LDFLAGS="-stdlib=libc++ -fuse-ld=lld -rtlib=compiler-rt -l:libunwind.a"
common:coverage --action_env=LDFLAGS="-stdlib=libc++ -fuse-ld=lld -rtlib=compiler-rt -l:libunwind.a"

This worked for me:

diff --git a/bazel/envoy_build_system.bzl b/bazel/envoy_build_system.bzl
index c3d0995330..81a4690bcb 100644
--- a/bazel/envoy_build_system.bzl
+++ b/bazel/envoy_build_system.bzl
@@ -143,18 +143,25 @@ def envoy_cmake(
         "//conditions:default": {},
     })
 
+    _compiler_rt_entries = select({
+        "@toolchains_llvm//toolchain/config:use_compiler_rt": {
+            "CMAKE_C_FLAGS": "-rtlib=compiler-rt -l:libunwind.a",
+        },
+        "//conditions:default": {},
+    })
+
     # If cache_entries is a dict, merge defaults and wrap for debug builds.
     # If it's a select(), pass it through directly.
     if hasattr(cache_entries, "update"):
         cache_entries.update(default_cache_entries)
         cache_entries_debug = dict(cache_entries)
         cache_entries_debug.update(debug_cache_entries)
-        final_cache_entries = _lld_linker_entries | select({
+        final_cache_entries = _lld_linker_entries | _compiler_rt_entries | select({
             "@envoy//bazel:dbg_build": cache_entries_debug,
             "//conditions:default": cache_entries,
         })
     else:
-        final_cache_entries = _lld_linker_entries | cache_entries
+        final_cache_entries = _lld_linker_entries | _compiler_rt_entries | cache_entries
 
     pf = ""
     if copy_pdb:

There is a separate use_libunwind option which ideally should be used to conditionally enable the -l:libunwind.a flag. Couldn't figure out how to make that work in the select though but I'm sure there is a way.

[mathetake]

@kralicky cool - would it be possible to verify this works for your case? that would be very helpful

[phlax]

@kralicky you can use | operator to add to starlark dicts

[kralicky]

It works for normal builds but I think there are still issues when cross-compiling and for sanitizer builds. Maybe this is getting too off-topic but heres what I am seeing currently.

As far as I can tell, when cmake runs its compiler tests it will use CMAKE_C_FLAGS, but CMAKE_{SHARED,EXE,MODULE}_LINKER_FLAGS (and CMAKE_CXX_FLAGS for libevent etc.) are completely ignored.

If CMAKE_C_FLAGS is unset, then the compiler test will try to build with libgcc (which in my case will be missing from the sysroot).

If CMAKE_C_FLAGS is set to -rtlib=compiler-rt only, then:

• static non-cross-compile builds work, I think because clang will automatically add -l:libunwind.a for static builds here
• non-static non-cross-compile builds (sanitizers or coverage) fail when building the compiler tests, missing symbols from libunwind. Clang will report clang: warning: argument unused during compilation: '-rtlib=compiler-rt' (not sure why) and the -lunwind flag is never added to the actual ld.lld invocation.
• static cross-compile builds work, but I'm not sure why.

If CMAKE_C_FLAGS is set to -rtlib=compiler-rt -l:libunwind.a then:

• static non-cross-compile builds work
• non-static non-cross-compile builds work, since the extra -l:libunwind.a gets passed to ld.lld
• static cross-compile builds don't work when using the cxx_cross_lib toolchain option from [WIP] toolchain: Add cxx_lib attribute to inject libc++ in x-compile bazel-contrib/toolchains_llvm#696 to pull in libunwind.a for other archs. Pretty sure this is because these libs are located such that they cannot be found automatically (nearby the clang binary) and rely on an explicit -L flag which is not propagated to the cmake compiler tests.