Third try for #32466.
This adds an interop client / server for GCP Observability integration
testing.
Everything is new here with no refactor. Plan is to get this in first
before trying to refactor out the flags.
Avoids some compilation problems on older MSVC's, opens the door for
some future optimizations.
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
This test has been occasionally failing on CI with "Bus Error" crashes
while requiring the grpc shared library.
These crashes have been unreproducible locally. Let's continue debugging
(b/266212253) but skip this on CI.
Internal bug ref: b/271334230
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Note that the `StartRetryTimerLocked` code path is not covered by test.
And I'm having trouble adding a test case to cover it. Seems like we
need the health service (default_health_check_service) to shutdown and
end the server-side streaming RPC (and send the trailing metadata) but
keep the subchannel (and subchannel_stream_client) alive so that it
could retry a Health.Watch RPC.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Further Grpc.Tools tests
- TestCharactersInName - proto files with dots and numbers in their
names
- TestExtraOptions - test setting AdditionalProtocArguments,
OutputOptions and GrpcOutputOptions
- TestGrpcServices - test setting GrpcServices to "none", "client",
"server" and "both"
- TestSetOutputDirs - test setting OutputDir and GrpcOutputDir
Upgrade boringssl to the latest "master-with-bazel"
- use the `'USE_HEADERMAP' => 'NO'` fix for ObjC
- update the key for asm optimizations on mac/apple in python's setup.py
This PR depends on monterey fixes here:
https://github.com/grpc/grpc/pull/32493 and the boringssl's build
simplification
https://boringssl-review.googlesource.com/c/boringssl/+/56465.
---------
Co-authored-by: Hannah Shi <hannahshisfb@gmail.com>
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Refactor C++ interop test client flags into the common
`client_helper.h/cc`. This is needed by the observability testing PR
#32466
We need the `ABSL_DECLARE_FLAG` in the header file so that we can share
that across different implementation.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Make remaining objC jobs compatible with kokoro monterey workers and
prepare for boringssl upgrade.
The changes here are taken from https://github.com/grpc/grpc/pull/32357,
but they should be merged in a separate PR
(we need the changes to be able to upgrade to monterey anyway and
there's no reason to make the boringssl upgrade PR more complicated by
bundling more fixes into it).
I've checked that the grpc_basictests_objc_examples and
grpc_ios_binary_size are green if switched to monterey.
Unfortunately it's hard to make grpc_basictests_objc_examples pass on
both monterey and mojave, so I suggest merging this PR at the same time
as CL to upgrade the kokoro jobs to monterey.
- that way both PR and continuous runs will remain green
- older branches would need a backport anyway
---------
Co-authored-by: Hannah Shi <hannahshisfb@gmail.com>
AFAICT, travis hasn't been disabled for the grpc/grpc repository a long
time ago.
https://travis-ci.org/github/grpc/grpchttps://travis-ci.com/github/grpc/grpc
I'm happy to be proven wrong if travis is still being used, but if not,
we should delete the travis config to avoid confusing people (I've seen
some recently open PRs attempting to make modification to this file).
Alongside https://github.com/grpc/grpc/pull/32496, this makes this test
behave the same on all platforms.
FWIW, I verified this causes us to see the previous lock cycle problem
in https://github.com/grpc/grpc/pull/32491 on linux - originally that
lock cycle was only on mac, because of environmental differences between
mac and linux in CI.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
This compiles for //:grpc, but not for tests yet.
It's the right approach though - @veblush hoping this is something you
can pick up and finish off.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
1. All greeter servers now support flag `--port` to customize the listen
port.
2. All client implementation now have `--target` flag to specify the
server location.
3. ABSL is used to parse the flags to reduce the amount of boilerplate
code in the examples.
Fixes: #26989
The docs change is extracted from
https://github.com/grpc/grpc/pull/31869 and
https://github.com/grpc/grpc/pull/31938.
The actual upgrade of boringssl is in progress, but in the meantime we
can at least make sure the instructions are up-to-date.
I'll also update the internal counterpart (cl/501499368)
Co-authored-by: Hannah Shi <hannahshisfb@gmail.com>
This filter was originally written only for the C++ wrapped layer, but
we have plans to use this for Python (and maybe other wrapped languages
too in the future.)
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Looks like this was accidentally dropped from our build files in
https://github.com/grpc/grpc/pull/21929, which means that this test
hasn't actually been built or run in almost 3 years. Unsurprisingly
after all that time, I had to make some changes to the test to get it to
actually build.
I've replaced all use of `InternalError` here because none of these
scenarios would necessarily merit a bug or outage report.
Identified in the fuchsia test suite: calling the Listener's
`on_shutdown` method with anything other than `absl::OkStatus()` would
fail some assertions in the Posix-specialized client test suite if the
Oracle were implemented similarly. It _should_ fail the same way in the
listener test suite, but the statuses are ignored. I've fixed that.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
This fixes the problems identified while building with clang-cl on
Windows, with build arguments `/std:c++14 /W4`
Passes internal checks: cl/511562057
----
We can't yet enable a clang-cl build as part of our continuous
integration tests due to a few issues:
protobuf fails an `unused-parameter` warning check in v4.21 (the current
pinned version) on Windows. The upgrade to v4.22 is evidently painful
and in progress. Without maintaining a patch against protobuf, or
disabling warnings-as-errors somehow for the protobuf code alone, we'll
need to upgrade our dependency before we can automate the clang-cl build
for Windows.
Next, our Windows CI environment does not have clang installed. There
has been some work over the past year to create custom kokoro images,
but that work has apparently stalled after trading hands a few times.
Using our current images, installing clang every time we run the job may
be our best bet (likely from precompiled binaries that we host
ourselves), but it will eat up more CI resources.
Finally, some of the default build configurations are incorrect for
clang-cl. For example `-Wall` in clang-cl translates roughly to
`-Weverything` in clang linux, whereas `-W4` in clang-cl translates more
closely to `-Wall -Wextra`. This configuration in the gRPC bazel build
is not currently platform-specific, it will need to be updated.
Similarly, `-std=c++14` is an unknown argument on Windows (should be
`/std:c++14`), and should not be in the bazelrc. This will likely need
the same platform-specific support.
With the `--copt="-std=c++14"` setting in the bazel.rc file as it is
today, MSVC builds have complained for every cc file:
```
cl : Command line warning D9002 : ignoring unknown option '-std=c++14'
```
This adds thousands of lines of noise to Windows builds, and hides
useful warnings. Using the `/std:c++14` flag on MSVC (and clang-cl) gets
us the desired result.
I had some doubts about `Seq` debugging another problem, so expanded the
tests we have to try and isolate the problem (so far without success, so
I think the original problem was elsewhere).
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Internal Windows builds will catch issues that we cannot yet catch in
OSS. This will be mostly remedied once we have clang-cl in our CI (See a
rough roadmap in https://github.com/grpc/grpc/pull/32448). For now, this
PR identifies folders where most Windows-specific code is developed, and
requires cherrypicks for PRs that touch anything inside those folders.
This PR also refactors gpr and gprpp source files to better isolate all
platform-specific code ~the Windows-only code~. ~I will reorganize the
other platform-specific files using this structure if there are no
objections.~
This subset of folders covers about half of the `#ifdef GPR_WINDOWS`
usages in gRPC, but nearly all of the actively-developed Windows code
locations.
This reverts commit 0fc0384b5a.
Major changes: this code calls `GetDefaultEventEngine` once on Alarm
init instead of 7 times throughout.
I will run benchmarks to ensure b/237283941 is not reproduced.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: drfloob <drfloob@users.noreply.github.com>
For stats, the StackDriver/OpenCensus API allows setting the
MonitoredResource directly, so use that.
For tracing, there is no explicit MonitoredResource to use, so just
insert it into the attributes for a span.
This PR adds batching support for GCP Observability logging. So instead
of the naive creating a new RPC to cloud logging for each logging event,
we now batch the log events to meet one of the following requirements -
* Batch size of 1000
* Batch memory consumption of 1MB
* A timeout period of 1sec after which we flush the accumulated batch
irrespective of the size.
There can also be cases where for some reason the RPCs fail or the batch
just accumulates to a very large size(100000 entries or 10MB in size).
In such cases, we just log the events with gpr_log instead of just
continuing to accumulate.
Additionally, `GcpObservabilityClose()` has been added to gracefully
shut off logging where we block till all the currently logged events are
flushed. (We might be able to gracefully shut off stats and tracing in
the future too.)
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
core; ref #30979
1. avoid calling `grpc_dump_slice` if the log level is too low (and the
result will be ignored)
2. use `GRPC_TRACE_FLAG_ENABLED(x)` over `x.enabled()` in the touched
code
---------
Co-authored-by: Yash Tibrewal <yashkt@google.com>
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
AJ Heller
Mark D. Roth
Stanley Cheung
Craig Tiller
Richard Belleville
Vignesh Babu
apolcyn
Yijie Ma
tony
Jan Tattermusch
Mike Kruskal
Eugene Ostroukhov
Doug Fawley
Yash Tibrewal
Xuan Wang
Marc Gravell