- ensure ordering of `OnAccept` and `Shutdown` callbacks from thready event engine (previously these could be reordered and this caused spurious failures)
- enable thready_tsan for one C++ e2e test
- don't filter thready_tsan for local builds (only CI)
Closes#36886
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36886 from ctiller:erm e3b88e7d86
PiperOrigin-RevId: 642702724
Usage: `bazel build --config=clang-cl --build_tag_filters=-no_windows :all`
The highlight, for me anyway: thread safety annotation checking can now be done on Windows-only code.
clang-cl's interpretation of `-Wall` differs from that of clang on linux. This PR uses the set of warnings enabled on linux, and whittles down the list until all builds pass. I left comments in `copts.bzl` describing the warnings, many of which can be removed with targeted code cleanups.
Closes#36831
PiperOrigin-RevId: 641009783
https://github.com/grpc/grpc/pull/36753 has this ASAN test failure with the following error which doesn't seem to make sense. ([full log](https://btx.cloud.google.com/invocations/a587e5cc-ca1e-46ed-a3c3-199c581583db/targets))
```
Executing tests from //test/cpp/util:grpc_tool_test@poller=epoll1
-----------------------------------------------------------------------------
=================================================================
==15==ERROR: AddressSanitizer: odr-violation (0x7fcfa2961400):
[1] size=66 'typeinfo name for google::protobuf::compiler::java::ImmutableExtensionLiteGenerator' external/com_google_protobuf/src/google/protobuf/compiler/java/lite/extension.cc in /b/f/w/bazel-out/k8-fastbuild/bin/test/cpp/util/grpc_tool_test@poller=epoll1.runfiles/com_github_grpc_grpc/test/cpp/util/../../../_solib_k8/libexternal_Scom_Ugoogle_Uprotobuf_Ssrc_Sgoogle_Sprotobuf_Scompiler_Sjava_Slite_Sliblite.so
[2] size=66 'typeinfo name for google::protobuf::compiler::java::ImmutableExtensionLiteGenerator' external/com_google_protobuf/src/google/protobuf/compiler/java/lite/extension.cc in /b/f/w/bazel-out/k8-fastbuild/bin/test/cpp/util/grpc_tool_test@poller=epoll1.runfiles/com_github_grpc_grpc/test/cpp/util/../../../_solib_k8/libexternal_Scom_Ugoogle_Uprotobuf_Ssrc_Sgoogle_Sprotobuf_Scompiler_Sjava_Slite_Slibfield_Ugenerators.so
These globals were registered at these points:
[1]:
#0 0x5626b4ea1488 in __asan_register_globals /tmp/clang-build/src/compiler-rt/lib/asan/asan_globals.cpp:369:3
#1 0x5626b4ea2559 in __asan_register_elf_globals /tmp/clang-build/src/compiler-rt/lib/asan/asan_globals.cpp:352:3
#2 0x7fcfa3fa4b99 (/lib64/ld-linux-x86-64.so.2+0x11b99) (BuildId: 7ae2aaae1a0e5b262df913ee0885582d2e327982)
[2]:
#0 0x5626b4ea1488 in __asan_register_globals /tmp/clang-build/src/compiler-rt/lib/asan/asan_globals.cpp:369:3
#1 0x5626b4ea2559 in __asan_register_elf_globals /tmp/clang-build/src/compiler-rt/lib/asan/asan_globals.cpp:352:3
#2 0x7fcfa3fa4b99 (/lib64/ld-linux-x86-64.so.2+0x11b99) (BuildId: 7ae2aaae1a0e5b262df913ee0885582d2e327982)
==15==HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_odr_violation=0
SUMMARY: AddressSanitizer: odr-violation: global 'typeinfo name for google::protobuf::compiler::java::ImmutableExtensionLiteGenerator' at external/com_google_protobuf/src/google/protobuf/compiler/java/lite/extension.cc in /b/f/w/bazel-out/k8-fastbuild/bin/test/cpp/util/grpc_tool_test@poller=epoll1.runfiles/com_github_grpc_grpc/test/cpp/util/../../../_solib_k8/libexternal_Scom_Ugoogle_Uprotobuf_Ssrc_Sgoogle_Sprotobuf_Scompiler_Sjava_Slite_Sliblite.so
==15==ABORTING
```
This turned out to be a known issue described at https://github.com/google/sanitizers/issues/1017 and there is much to do other than disabling the ODR test. I gave a try to "-mllvm -asan-use-private-alias=1" option but it didn't change the result so I turned into this way.
Closes#36756
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36756 from veblush:asan-workaround 622581e8d2
PiperOrigin-RevId: 638421967
Reproduced with :
- bazel-bootstrap 3.5.1+ds-3 (pulling bazel 6.4.0)
- setting CC to gcc and CXX to g++ otherwise bazel does not detect it
- overall, using debian 11 using default gcc/g++ versions 10.2.1
When using --config=dbg, a warning name produces an error at the beginning of the compilation :
`cc1plus: error: '-Werror=return-stack-address':
no option '-Wreturn-stack-address';
did you mean '-Wreturn-local-addr'?`
Fixed the error following the compiler advice for GCC
PLEASE NOTE : I guess `return-stack-address` might be related to CLANG
and `return-local-addr` is the GCC equivalent. I do not know CLANG,
nor am i good enough with bazel to know how to make this option flexible
(so any help will be appreciated)
Closes#36040
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36040 from nipil:fix_compile_dbg_option 7ade99de76
PiperOrigin-RevId: 617281605
This is a prerequisite change to start supporting Bazel 7. Changes are
- Disabled bzlmod which Bazel 7 begins to enable by default. This eventually needs to be done to support bzlmod but not now.
- Upgraded some bazel rule dependencies which are required to support Bazel 7.
- Using Python 3 explcitly as Bazel 7 begins to reject Python 2.
Note that this isn't enough to enable Bazel 7 by default and another PR will follow for that.
Closes#35374
PiperOrigin-RevId: 592931675
This is to address too many warnings coming from upb 24.x
```
In file included from external/upb/upb/mini_table/field.h:32:
external/upb/upb/mini_table/internal/message.h:36:14:
warning: redefinition of typedef 'upb_Message' is a C11 feature [-Wtypedef-redefinition]
typedef void upb_Message;
^
external/upb/upb/message/tagged_ptr.h:40:14: note: previous definition is here
typedef void upb_Message;
^
1 warning generated.
```
Note that the plugin is still under `grpc::internal` namespace and not
under `experimental` intentionally.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
This test mode tries to create threads wherever it legally can to
maximize the chances of TSAN finding errors in our codebase.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
This maybe used to quickly verify the code coverage of a modified test
locally (e.g. fuzzer).
Example:
```
# Build and run target; the raw profile will be written to $LLVM_PROFILE_FILE when the program exits
$ bazel build --config=dbg --config=fuzzer_asan --config=coverage //test/core/end2end/fuzzers:api_fuzzer
$ LLVM_PROFILE_FILE="api_fuzzer.profraw" bazel-bin/test/core/end2end/fuzzers/api_fuzzer test/core/end2end/fuzzers/api_fuzzer_corpus/*
# Create coverage report
$ llvm-profdata-14 merge -sparse api_fuzzer.profraw -o api_fuzzer.profdata
$ llvm-cov-14 report ./bazel-bin/test/core/end2end/fuzzers/api_fuzzer --instr-profile=api_fuzzer.profdata
```
Sample report:
f94e444f25/gistfile1.txt
One trick is that the binary needs to be statically linked, e.g. specify
`linkstatic = 1` on the BUILD target.
See https://clang.llvm.org/docs/SourceBasedCodeCoverage.html for more
info.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Initial PR to establish a bazel dependency on
https://github.com/google/fuzztest, with which I'm planning on basing a
hardening program.
Casting a relatively wide net with reviewers: I'm genuinely interested
in feedback building up the docs, and general ergonomics of this change.
I've located relevant files in the `fuzztest/...` directory. The tests
only build with the `--config fuzztest` bazel argument for now (because
of needing C++17), so locating them separately keeps `bazel test
test/...` working as it does today. In a few years time, when we adopt
C++17, we'll be able to rationalize the test directories a little bit.
We'll need to add some kokoro jobs (maybe with this PR?) to execute the
relevant tests.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
With the `--copt="-std=c++14"` setting in the bazel.rc file as it is
today, MSVC builds have complained for every cc file:
```
cl : Command line warning D9002 : ignoring unknown option '-std=c++14'
```
This adds thousands of lines of noise to Windows builds, and hides
useful warnings. Using the `/std:c++14` flag on MSVC (and clang-cl) gets
us the desired result.
* WIP. A seemingly properly failing test
* WIP. Pre-fork handlers now work
* Roughly working.
* Clean up
* Clean up more
* Add to CI
* Format
* Ugh. Remove swap file
* And another
* clean up
* Add copyright
* Format
* Remove another debug line
* Add stub forkable methods
* Remove use of 3.9+ function
* Remove unintentional double copyright
* drfloob review comments
* Only hold lock during Close once
* Create separate job for fork test
* Bump up gdb timeout
* Format
* [cleanup] Remove profiling timers
- nobody has used this system in years
- if we needed it, we'd probably rewrite it at this point to be something more modern
- let's remove it until that need arises
* fix
* fixes
Craig Tiller
AJ Heller
Esun Kim
Mark D. Roth
nipil
Yash Tibrewal
Yijie Ma
Richard Belleville
Craig Tiller
Lidi Zheng
Jan Tattermusch
Nicolas Noble