From 118e69a48449b17a5c1fa0d660c119380528c190 Mon Sep 17 00:00:00 2001
From: David Hoover <karma@deadmoose.com>
Date: Wed, 12 Sep 2018 13:15:44 -0700
Subject: [PATCH 001/138] Fix backwards logic checking for --call_creds=none
release note: no
---
test/cpp/util/cli_credentials.cc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/test/cpp/util/cli_credentials.cc b/test/cpp/util/cli_credentials.cc
index 0a922617bb6..b5753a31854 100644
--- a/test/cpp/util/cli_credentials.cc
+++ b/test/cpp/util/cli_credentials.cc
@@ -118,7 +118,7 @@ std::shared_ptr<grpc::CallCredentials> CliCredentials::GetCallCredentials()
if (IsAccessToken(FLAGS_call_creds)) {
return grpc::AccessTokenCredentials(AccessToken(FLAGS_call_creds));
}
- if (FLAGS_call_creds.compare("none") != 0) {
+ if (FLAGS_call_creds.compare("none") == 0) {
// Nothing to do; creds, if any, are baked into the channel.
return std::shared_ptr<grpc::CallCredentials>();
}
From 1af046e7239a295d9c16823c98cfae52f936898b Mon Sep 17 00:00:00 2001
From: Jean de Klerk <deklerk@google.com>
Date: Mon, 11 Feb 2019 10:29:57 -0700
Subject: [PATCH 002/138] docs: enumerate status codes in statuscodes.md
statuscodes.md is one of the top search results for "grpc status codes",
right behind codes.proto. Since this is a landing page for many people*,
and since the title and description of this page purports to be the
generic status codes documentation page, it seems like a good idea to define
terms before using them.
* Many people who are new to grpc/protobuf, for example, will be more
accustomed to and choose markdown files over .proto files.
---
doc/statuscodes.md | 32 +++++++++++++++++++++++++++-----
1 file changed, 27 insertions(+), 5 deletions(-)
diff --git a/doc/statuscodes.md b/doc/statuscodes.md
index 547da054951..3d4d87e931a 100644
--- a/doc/statuscodes.md
+++ b/doc/statuscodes.md
@@ -1,13 +1,35 @@
# Status codes and their use in gRPC
-gRPC uses a set of well defined status codes as part of the RPC API. All
-RPCs started at a client return a `status` object composed of an integer
+gRPC uses a set of well defined status codes as part of the RPC API. These
+statuses are defined as such:
+
+| Code | Number | Description | Closest HTTP Mapping |
+|------------------|--------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------|
+| OK | 0 | Not an error; returned on success. | 200 OK |
+| CANCELLED | 1 | The operation was cancelled, typically by the caller. | 499 Client Closed Request |
+| UNKNOWN | 2 | Unknown error. For example, this error may be returned when a `Status` value received from another address space belongs to an error space that is not known in this address space. Also errors raised by APIs that do not return enough error information may be converted to this error. | 500 Internal Server Error |
+| INVALID_ARGUMENT | 3 | The client specified an invalid argument. Note that this differs from `FAILED_PRECONDITION`. `INVALID_ARGUMENT` indicates arguments that are problematic regardless of the state of the system (e.g., a malformed file name). | 400 Bad Request |
+| DEADLINE_EXCEEDED | 4 | The deadline expired before the operation could complete. For operations that change the state of the system, this error may be returned even if the operation has completed successfully. For example, a successful response from a server could have been delayed long | 504 Gateway Timeout |
+| NOT_FOUND | 5 | Some requested entity (e.g., file or directory) was not found. Note to server developers: if a request is denied for an entire class of users, such as gradual feature rollout or undocumented whitelist, `NOT_FOUND` may be used. If a request is denied for some users within a class of users, such as user-based access control, `PERMISSION_DENIED` must be used. | 404 Not Found |
+| ALREADY_EXISTS | 6 | The entity that a client attempted to create (e.g., file or directory) already exists. | 409 Conflict |
+| PERMISSION_DENIED | 7 | The caller does not have permission to execute the specified operation. `PERMISSION_DENIED` must not be used for rejections caused by exhausting some resource (use `RESOURCE_EXHAUSTED` instead for those errors). `PERMISSION_DENIED` must not be used if the caller can not be identified (use `UNAUTHENTICATED` instead for those errors). This error code does not imply the request is valid or the requested entity exists or satisfies other pre-conditions. | 403 Forbidden |
+| UNAUTHENTICATED | 16 | The request does not have valid authentication credentials for the operation. | 401 Unauthorized |
+| RESOURCE_EXHAUSTED | 8 | Some resource has been exhausted, perhaps a per-user quota, or perhaps the entire file system is out of space. | 429 Too Many Requests |
+| FAILED_PRECONDITION | 9 | The operation was rejected because the system is not in a state required for the operation's execution. For example, the directory to be deleted is non-empty, an rmdir operation is applied to a non-directory, etc. Service implementors can use the following guidelines to decide between `FAILED_PRECONDITION`, `ABORTED`, and `UNAVAILABLE`: (a) Use `UNAVAILABLE` if the client can retry just the failing call. (b) Use `ABORTED` if the client should retry at a higher level (e.g., when a client-specified test-and-set fails, indicating the client should restart a read-modify-write sequence). (c) Use `FAILED_PRECONDITION` if the client should not retry until the system state has been explicitly fixed. E.g., if an "rmdir" fails because the directory is non-empty, `FAILED_PRECONDITION` should be returned since the client should not retry unless the files are deleted from the directory. | 400 Bad Request |
+| ABORTED | 10 | The operation was aborted, typically due to a concurrency issue such as a sequencer check failure or transaction abort. See the guidelines above for deciding between `FAILED_PRECONDITION`, `ABORTED`, and `UNAVAILABLE`. | 409 Conflict |
+| OUT_OF_RANGE | 11 | The operation was attempted past the valid range. E.g., seeking or reading past end-of-file. Unlike `INVALID_ARGUMENT`, this error indicates a problem that may be fixed if the system state changes. For example, a 32-bit file system will generate `INVALID_ARGUMENT` if asked to read at an offset that is not in the range [0,2^32-1], but it will generate `OUT_OF_RANGE` if asked to read from an offset past the current file size. There is a fair bit of overlap between `FAILED_PRECONDITION` and `OUT_OF_RANGE`. We recommend using `OUT_OF_RANGE` (the more specific error) when it applies so that callers who are iterating through a space can easily look for an `OUT_OF_RANGE` error to detect when they are done. | 400 Bad Request |
+| UNIMPLEMENTED | 12 | The operation is not implemented or is not supported/enabled in this service. | 501 Not Implemented |
+| INTERNAL | 13 | Internal errors. This means that some invariants expected by the underlying system have been broken. This error code is reserved for serious errors. | 500 Internal Server Error |
+| UNAVAILABLE | 14 | The service is currently unavailable. This is most likely a transient condition, which can be corrected by retrying with a backoff. | 503 Service Unavailable |
+| DATA_LOSS | 15 | Unrecoverable data loss or corruption. | 500 Internal Server Error |
+
+All RPCs started at a client return a `status` object composed of an integer
`code` and a string `message`. The server-side can choose the status it
returns for a given RPC.
The gRPC client and server-side implementations may also generate and
-return `status` on their own when errors happen. Only a subset of
-the pre-defined status codes are generated by the gRPC libraries. This
+return `status` on their own when errors happen. Only a subset of
+the pre-defined status codes are generated by the gRPC libraries. This
allows applications to be sure that any other code it sees was actually
returned by the application (although it is also possible for the
server-side to return one of the codes generated by the gRPC libraries).
@@ -49,4 +71,4 @@ The following status codes are never generated by the library:
- OUT_OF_RANGE
- DATA_LOSS
-Applications that may wish to [retry](https://github.com/grpc/proposal/blob/master/A6-client-retries.md) failed RPCs must decide which status codes on which to retry. As shown in the table above, the gRPC library can generate the same status code for different cases. Server applications can also return those same status codes. Therefore, there is no fixed list of status codes on which it is appropriate to retry in all applications. As a result, individual applications must make their own determination as to which status codes should cause an RPC to be retried.
+Applications that may wish to [retry](https:github.com/grpc/proposal/blob/master/A6-client-retries.md) failed RPCs must decide which status codes on which to retry. As shown in the table above, the gRPC library can generate the same status code for different cases. Server applications can also return those same status codes. Therefore, there is no fixed list of status codes on which it is appropriate to retry in all applications. As a result, individual applications must make their own determination as to which status codes should cause an RPC to be retried.
From cfc2156665a262252eb01aa4e9508b6f9e54594e Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Thu, 14 Feb 2019 18:25:28 +0100
Subject: [PATCH 003/138] add AspNetCore interop server
---
.../Dockerfile.template | 20 ++++++++++++
.../build_interop.sh.template | 31 +++++++++++++++++++
.../grpc_interop_aspnetcore/Dockerfile | 18 +++++++++++
.../grpc_interop_aspnetcore/build_interop.sh | 29 +++++++++++++++++
.../dockerize/build_interop_image.sh | 8 +++++
5 files changed, 106 insertions(+)
create mode 100644 templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template
create mode 100644 templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh.template
create mode 100644 tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile
create mode 100644 tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh
diff --git a/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template b/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template
new file mode 100644
index 00000000000..e1b6da89a74
--- /dev/null
+++ b/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template
@@ -0,0 +1,20 @@
+%YAML 1.2
+--- |
+ # Copyright 2017 gRPC authors.
+ #
+ # Licensed under the Apache License, Version 2.0 (the "License");
+ # you may not use this file except in compliance with the License.
+ # You may obtain a copy of the License at
+ #
+ # http://www.apache.org/licenses/LICENSE-2.0
+ #
+ # Unless required by applicable law or agreed to in writing, software
+ # distributed under the License is distributed on an "AS IS" BASIS,
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ # See the License for the specific language governing permissions and
+ # limitations under the License.
+
+ FROM microsoft/dotnet:3.0.100-preview2-sdk-stretch
+
+ # Define the default command.
+ CMD ["bash"]
diff --git a/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh.template b/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh.template
new file mode 100644
index 00000000000..69e2ed387b2
--- /dev/null
+++ b/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh.template
@@ -0,0 +1,31 @@
+%YAML 1.2
+--- |
+ #!/bin/bash
+ # Copyright 2017 gRPC authors.
+ #
+ # Licensed under the Apache License, Version 2.0 (the "License");
+ # you may not use this file except in compliance with the License.
+ # You may obtain a copy of the License at
+ #
+ # http://www.apache.org/licenses/LICENSE-2.0
+ #
+ # Unless required by applicable law or agreed to in writing, software
+ # distributed under the License is distributed on an "AS IS" BASIS,
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ # See the License for the specific language governing permissions and
+ # limitations under the License.
+ #
+ # Builds Grpc.AspNetCore.Server interop server in a base image.
+ set -e
+
+ mkdir -p /var/local/git
+ git clone /var/local/jenkins/grpc-dotnet /var/local/git/grpc-dotnet
+
+ # copy service account keys if available
+ cp -r /var/local/jenkins/service_account $HOME || true
+
+ cd /var/local/git/grpc-dotnet
+ ./build/get-grpc.sh
+
+ cd testassets/InteropTestsWebsite
+ dotnet build --configuration Debug
diff --git a/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile b/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile
new file mode 100644
index 00000000000..2caa093ccc1
--- /dev/null
+++ b/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile
@@ -0,0 +1,18 @@
+# Copyright 2017 gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM microsoft/dotnet:3.0.100-preview2-sdk-stretch
+
+# Define the default command.
+CMD ["bash"]
diff --git a/tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh b/tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh
new file mode 100644
index 00000000000..38feae39623
--- /dev/null
+++ b/tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh
@@ -0,0 +1,29 @@
+#!/bin/bash
+# Copyright 2017 gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# Builds Grpc.AspNetCore.Server interop server in a base image.
+set -e
+
+mkdir -p /var/local/git
+git clone /var/local/jenkins/grpc-dotnet /var/local/git/grpc-dotnet
+
+# copy service account keys if available
+cp -r /var/local/jenkins/service_account $HOME || true
+
+cd /var/local/git/grpc-dotnet
+./build/get-grpc.sh
+
+cd testassets/InteropTestsWebsite
+dotnet build --configuration Debug
diff --git a/tools/run_tests/dockerize/build_interop_image.sh b/tools/run_tests/dockerize/build_interop_image.sh
index 025c532d976..fe37defd146 100755
--- a/tools/run_tests/dockerize/build_interop_image.sh
+++ b/tools/run_tests/dockerize/build_interop_image.sh
@@ -64,6 +64,14 @@ else
echo "WARNING: grpc-node not found, it won't be mounted to the docker container."
fi
+echo "GRPC_DOTNET_ROOT: ${GRPC_DOTNET_ROOT:=$(cd ../grpc-dotnet && pwd)}"
+if [ -n "$GRPC_DOTNET_ROOT" ]
+then
+ MOUNT_ARGS+=" -v $GRPC_DOTNET_ROOT:/var/local/jenkins/grpc-dotnet:ro"
+else
+ echo "WARNING: grpc-dotnet not found, it won't be mounted to the docker container."
+fi
+
# Mount service account dir if available.
# If service_directory does not contain the service account JSON file,
# some of the tests will fail.
From ecee06640e39b4c249a45da88a55da31a28cbe2d Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Fri, 15 Feb 2019 00:17:05 +0100
Subject: [PATCH 004/138] update prepare scripts
---
tools/internal_ci/helper_scripts/prepare_build_interop_rc | 1 +
tools/internal_ci/helper_scripts/prepare_build_macos_interop_rc | 1 +
2 files changed, 2 insertions(+)
diff --git a/tools/internal_ci/helper_scripts/prepare_build_interop_rc b/tools/internal_ci/helper_scripts/prepare_build_interop_rc
index fb0f4b8054e..e462a83e522 100644
--- a/tools/internal_ci/helper_scripts/prepare_build_interop_rc
+++ b/tools/internal_ci/helper_scripts/prepare_build_interop_rc
@@ -28,6 +28,7 @@ git clone --recursive https://github.com/grpc/grpc-go ./../grpc-go
git clone --recursive https://github.com/grpc/grpc-java ./../grpc-java
git clone --recursive https://github.com/grpc/grpc-node ./../grpc-node
git clone --recursive https://github.com/grpc/grpc-dart ./../grpc-dart
+git clone --recursive https://github.com/grpc/grpc-dotnet ./../grpc-dotnet
# Download json file.
mkdir ~/service_account
diff --git a/tools/internal_ci/helper_scripts/prepare_build_macos_interop_rc b/tools/internal_ci/helper_scripts/prepare_build_macos_interop_rc
index 43bc9609c7e..cbc3ef2d9a9 100644
--- a/tools/internal_ci/helper_scripts/prepare_build_macos_interop_rc
+++ b/tools/internal_ci/helper_scripts/prepare_build_macos_interop_rc
@@ -22,3 +22,4 @@ git clone --recursive https://github.com/grpc/grpc-go ./../grpc-go
git clone --recursive https://github.com/grpc/grpc-java ./../grpc-java
git clone --recursive https://github.com/grpc/grpc-node ./../grpc-node
git clone --recursive https://github.com/grpc/grpc-dart ./../grpc-dart
+git clone --recursive https://github.com/grpc/grpc-dotnet ./../grpc-dotnet
From 96f4454ce7f491974c1a6443cb9c38ddb73d9c82 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Fri, 15 Feb 2019 00:35:44 +0100
Subject: [PATCH 005/138] update run_interop_tests.py script
---
tools/run_tests/run_interop_tests.py | 43 ++++++++++++++++++++++++----
1 file changed, 38 insertions(+), 5 deletions(-)
diff --git a/tools/run_tests/run_interop_tests.py b/tools/run_tests/run_interop_tests.py
index 448b53a7207..33128c87320 100755
--- a/tools/run_tests/run_interop_tests.py
+++ b/tools/run_tests/run_interop_tests.py
@@ -58,9 +58,10 @@ _SKIP_SERVER_COMPRESSION = [
_SKIP_COMPRESSION = _SKIP_CLIENT_COMPRESSION + _SKIP_SERVER_COMPRESSION
-_SKIP_ADVANCED = [
- 'status_code_and_message', 'custom_metadata', 'unimplemented_method',
- 'unimplemented_service'
+_SKIP_UNIMPLEMENTED_HANDLERS = ['unimplemented_method', 'unimplemented_service']
+
+_SKIP_ADVANCED = _SKIP_UNIMPLEMENTED_HANDLERS + [
+ 'status_code_and_message', 'custom_metadata'
]
_SKIP_SPECIAL_STATUS_MESSAGE = ['special_status_message']
@@ -173,6 +174,37 @@ class CSharpCoreCLRLanguage:
return 'csharpcoreclr'
+class AspNetCoreLanguage:
+
+ def __init__(self):
+ self.client_cwd = '../grpc-dotnet'
+ self.server_cwd = '../grpc-dotnet/testassets/InteropTestsWebsite/bin/Debug/netcoreapp3.0'
+ self.safename = str(self)
+
+ def cloud_to_prod_env(self):
+ return {}
+
+ def client_cmd(self, args):
+ # attempt to run client should fail
+ return ['dotnet' 'exec', 'CLIENT_NOT_SUPPORTED'] + args
+
+ def server_cmd(self, args):
+ return ['dotnet', 'exec', 'InteropTestsWebsite.dll'] + args
+
+ def global_env(self):
+ return {}
+
+ def unimplemented_test_cases(self):
+ # aspnetcore doesn't have a client so ignore all test cases.
+ return _TEST_CASES + _AUTH_TEST_CASES
+
+ def unimplemented_test_cases_server(self):
+ return _SKIP_COMPRESSION + _SKIP_UNIMPLEMENTED_HANDLERS + _SKIP_SPECIAL_STATUS_MESSAGE
+
+ def __str__(self):
+ return 'aspnetcore'
+
+
class DartLanguage:
def __init__(self):
@@ -590,6 +622,7 @@ _LANGUAGES = {
'c++': CXXLanguage(),
'csharp': CSharpLanguage(),
'csharpcoreclr': CSharpCoreCLRLanguage(),
+ 'aspnetcore': AspNetCoreLanguage(),
'dart': DartLanguage(),
'go': GoLanguage(),
'java': JavaLanguage(),
@@ -605,8 +638,8 @@ _LANGUAGES = {
# languages supported as cloud_to_cloud servers
_SERVERS = [
- 'c++', 'node', 'csharp', 'csharpcoreclr', 'java', 'go', 'ruby', 'python',
- 'dart'
+ 'c++', 'node', 'csharp', 'csharpcoreclr', 'aspnetcore', 'java', 'go',
+ 'ruby', 'python', 'dart'
]
_TEST_CASES = [
From d0c42dec85b7255eeb76f94e25c1ad74390fe6f3 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Thu, 21 Feb 2019 09:15:22 +0100
Subject: [PATCH 006/138] hotfix: download unreleased preview3
---
.../grpc_interop_aspnetcore/Dockerfile.template | 7 +++++++
.../interoptest/grpc_interop_aspnetcore/Dockerfile | 7 +++++++
2 files changed, 14 insertions(+)
diff --git a/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template b/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template
index e1b6da89a74..a341592a2ab 100644
--- a/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template
+++ b/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template
@@ -16,5 +16,12 @@
FROM microsoft/dotnet:3.0.100-preview2-sdk-stretch
+ RUN rm /usr/bin/dotnet # remove symlink
+ RUN curl -sSL -o dotnet.tar.gz https://dotnetcli.azureedge.net/dotnet/Sdk/3.0.100-preview3-010313/dotnet-sdk-3.0.100-preview3-010313-linux-x64.tar.gz ${'\\'}
+ && mkdir -p /usr/share/dotnet ${'\\'}
+ && tar -zxf dotnet.tar.gz -C /usr/share/dotnet ${'\\'}
+ && rm dotnet.tar.gz ${'\\'}
+ && ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet
+
# Define the default command.
CMD ["bash"]
diff --git a/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile b/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile
index 2caa093ccc1..75a8a200ab4 100644
--- a/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile
+++ b/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile
@@ -14,5 +14,12 @@
FROM microsoft/dotnet:3.0.100-preview2-sdk-stretch
+RUN rm /usr/bin/dotnet # remove symlink
+RUN curl -sSL -o dotnet.tar.gz https://dotnetcli.azureedge.net/dotnet/Sdk/3.0.100-preview3-010313/dotnet-sdk-3.0.100-preview3-010313-linux-x64.tar.gz \
+ && mkdir -p /usr/share/dotnet \
+ && tar -zxf dotnet.tar.gz -C /usr/share/dotnet \
+ && rm dotnet.tar.gz \
+ && ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet
+
# Define the default command.
CMD ["bash"]
From 7c717838797438d2925c4e45e4d65ac1c83c352d Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Fri, 1 Mar 2019 09:30:18 +0100
Subject: [PATCH 007/138] grpc-dotnet now support unimplemented handlers
---
tools/run_tests/run_interop_tests.py | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/tools/run_tests/run_interop_tests.py b/tools/run_tests/run_interop_tests.py
index 33128c87320..fd5b7ea21d9 100755
--- a/tools/run_tests/run_interop_tests.py
+++ b/tools/run_tests/run_interop_tests.py
@@ -58,10 +58,9 @@ _SKIP_SERVER_COMPRESSION = [
_SKIP_COMPRESSION = _SKIP_CLIENT_COMPRESSION + _SKIP_SERVER_COMPRESSION
-_SKIP_UNIMPLEMENTED_HANDLERS = ['unimplemented_method', 'unimplemented_service']
-
-_SKIP_ADVANCED = _SKIP_UNIMPLEMENTED_HANDLERS + [
- 'status_code_and_message', 'custom_metadata'
+_SKIP_ADVANCED = [
+ 'status_code_and_message', 'custom_metadata', 'unimplemented_method',
+ 'unimplemented_service'
]
_SKIP_SPECIAL_STATUS_MESSAGE = ['special_status_message']
@@ -199,7 +198,7 @@ class AspNetCoreLanguage:
return _TEST_CASES + _AUTH_TEST_CASES
def unimplemented_test_cases_server(self):
- return _SKIP_COMPRESSION + _SKIP_UNIMPLEMENTED_HANDLERS + _SKIP_SPECIAL_STATUS_MESSAGE
+ return _SKIP_COMPRESSION + _SKIP_SPECIAL_STATUS_MESSAGE
def __str__(self):
return 'aspnetcore'
From 4a8a2e286ed10317673e46d0505311dc9f12d1ef Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 1 Mar 2019 11:14:40 -0800
Subject: [PATCH 008/138] Add basic multiprocessing-based server
---
examples/python/multiprocessing/BUILD | 0
examples/python/multiprocessing/README.md | 0
examples/python/multiprocessing/client.py | 9 ++
examples/python/multiprocessing/prime.proto | 35 +++++
examples/python/multiprocessing/prime_pb2.py | 132 ++++++++++++++++++
.../python/multiprocessing/prime_pb2_grpc.py | 46 ++++++
examples/python/multiprocessing/server.py | 98 +++++++++++++
.../test/_multiprocessing_test.py | 0
8 files changed, 320 insertions(+)
create mode 100644 examples/python/multiprocessing/BUILD
create mode 100644 examples/python/multiprocessing/README.md
create mode 100644 examples/python/multiprocessing/client.py
create mode 100644 examples/python/multiprocessing/prime.proto
create mode 100644 examples/python/multiprocessing/prime_pb2.py
create mode 100644 examples/python/multiprocessing/prime_pb2_grpc.py
create mode 100644 examples/python/multiprocessing/server.py
create mode 100644 examples/python/multiprocessing/test/_multiprocessing_test.py
diff --git a/examples/python/multiprocessing/BUILD b/examples/python/multiprocessing/BUILD
new file mode 100644
index 00000000000..e69de29bb2d
diff --git a/examples/python/multiprocessing/README.md b/examples/python/multiprocessing/README.md
new file mode 100644
index 00000000000..e69de29bb2d
diff --git a/examples/python/multiprocessing/client.py b/examples/python/multiprocessing/client.py
new file mode 100644
index 00000000000..4ab33374ce2
--- /dev/null
+++ b/examples/python/multiprocessing/client.py
@@ -0,0 +1,9 @@
+# spin up multiple concurrent clients
+
+import logging
+import multiprocessing
+import os
+import time
+
+import prime_pb2
+import prime_pb2_grpc
diff --git a/examples/python/multiprocessing/prime.proto b/examples/python/multiprocessing/prime.proto
new file mode 100644
index 00000000000..4ef232f86cb
--- /dev/null
+++ b/examples/python/multiprocessing/prime.proto
@@ -0,0 +1,35 @@
+// Copyright 2019 gRPC authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+syntax = "proto3";
+
+package prime;
+
+// A candidate integer for primality testing.
+message PrimeCandidate {
+ // The candidate.
+ int64 candidate = 1;
+}
+
+// The primality of the requested integer candidate.
+message Primality {
+ // Is the candidate prime?
+ bool isPrime = 1;
+}
+
+// Service to check primality.
+service PrimeChecker {
+ // Determines the primality of an integer.
+ rpc check (PrimeCandidate) returns (Primality) {}
+}
diff --git a/examples/python/multiprocessing/prime_pb2.py b/examples/python/multiprocessing/prime_pb2.py
new file mode 100644
index 00000000000..58e6e6a023a
--- /dev/null
+++ b/examples/python/multiprocessing/prime_pb2.py
@@ -0,0 +1,132 @@
+# Generated by the protocol buffer compiler. DO NOT EDIT!
+# source: prime.proto
+
+import sys
+_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
+from google.protobuf import descriptor as _descriptor
+from google.protobuf import message as _message
+from google.protobuf import reflection as _reflection
+from google.protobuf import symbol_database as _symbol_database
+# @@protoc_insertion_point(imports)
+
+_sym_db = _symbol_database.Default()
+
+
+
+
+DESCRIPTOR = _descriptor.FileDescriptor(
+ name='prime.proto',
+ package='prime',
+ syntax='proto3',
+ serialized_options=None,
+ serialized_pb=_b('\n\x0bprime.proto\x12\x05prime\"#\n\x0ePrimeCandidate\x12\x11\n\tcandidate\x18\x01 \x01(\x03\"\x1c\n\tPrimality\x12\x0f\n\x07isPrime\x18\x01 \x01(\x08\x32\x42\n\x0cPrimeChecker\x12\x32\n\x05\x63heck\x12\x15.prime.PrimeCandidate\x1a\x10.prime.Primality\"\x00\x62\x06proto3')
+)
+
+
+
+
+_PRIMECANDIDATE = _descriptor.Descriptor(
+ name='PrimeCandidate',
+ full_name='prime.PrimeCandidate',
+ filename=None,
+ file=DESCRIPTOR,
+ containing_type=None,
+ fields=[
+ _descriptor.FieldDescriptor(
+ name='candidate', full_name='prime.PrimeCandidate.candidate', index=0,
+ number=1, type=3, cpp_type=2, label=1,
+ has_default_value=False, default_value=0,
+ message_type=None, enum_type=None, containing_type=None,
+ is_extension=False, extension_scope=None,
+ serialized_options=None, file=DESCRIPTOR),
+ ],
+ extensions=[
+ ],
+ nested_types=[],
+ enum_types=[
+ ],
+ serialized_options=None,
+ is_extendable=False,
+ syntax='proto3',
+ extension_ranges=[],
+ oneofs=[
+ ],
+ serialized_start=22,
+ serialized_end=57,
+)
+
+
+_PRIMALITY = _descriptor.Descriptor(
+ name='Primality',
+ full_name='prime.Primality',
+ filename=None,
+ file=DESCRIPTOR,
+ containing_type=None,
+ fields=[
+ _descriptor.FieldDescriptor(
+ name='isPrime', full_name='prime.Primality.isPrime', index=0,
+ number=1, type=8, cpp_type=7, label=1,
+ has_default_value=False, default_value=False,
+ message_type=None, enum_type=None, containing_type=None,
+ is_extension=False, extension_scope=None,
+ serialized_options=None, file=DESCRIPTOR),
+ ],
+ extensions=[
+ ],
+ nested_types=[],
+ enum_types=[
+ ],
+ serialized_options=None,
+ is_extendable=False,
+ syntax='proto3',
+ extension_ranges=[],
+ oneofs=[
+ ],
+ serialized_start=59,
+ serialized_end=87,
+)
+
+DESCRIPTOR.message_types_by_name['PrimeCandidate'] = _PRIMECANDIDATE
+DESCRIPTOR.message_types_by_name['Primality'] = _PRIMALITY
+_sym_db.RegisterFileDescriptor(DESCRIPTOR)
+
+PrimeCandidate = _reflection.GeneratedProtocolMessageType('PrimeCandidate', (_message.Message,), dict(
+ DESCRIPTOR = _PRIMECANDIDATE,
+ __module__ = 'prime_pb2'
+ # @@protoc_insertion_point(class_scope:prime.PrimeCandidate)
+ ))
+_sym_db.RegisterMessage(PrimeCandidate)
+
+Primality = _reflection.GeneratedProtocolMessageType('Primality', (_message.Message,), dict(
+ DESCRIPTOR = _PRIMALITY,
+ __module__ = 'prime_pb2'
+ # @@protoc_insertion_point(class_scope:prime.Primality)
+ ))
+_sym_db.RegisterMessage(Primality)
+
+
+
+_PRIMECHECKER = _descriptor.ServiceDescriptor(
+ name='PrimeChecker',
+ full_name='prime.PrimeChecker',
+ file=DESCRIPTOR,
+ index=0,
+ serialized_options=None,
+ serialized_start=89,
+ serialized_end=155,
+ methods=[
+ _descriptor.MethodDescriptor(
+ name='check',
+ full_name='prime.PrimeChecker.check',
+ index=0,
+ containing_service=None,
+ input_type=_PRIMECANDIDATE,
+ output_type=_PRIMALITY,
+ serialized_options=None,
+ ),
+])
+_sym_db.RegisterServiceDescriptor(_PRIMECHECKER)
+
+DESCRIPTOR.services_by_name['PrimeChecker'] = _PRIMECHECKER
+
+# @@protoc_insertion_point(module_scope)
diff --git a/examples/python/multiprocessing/prime_pb2_grpc.py b/examples/python/multiprocessing/prime_pb2_grpc.py
new file mode 100644
index 00000000000..dcc3a35706d
--- /dev/null
+++ b/examples/python/multiprocessing/prime_pb2_grpc.py
@@ -0,0 +1,46 @@
+# Generated by the gRPC Python protocol compiler plugin. DO NOT EDIT!
+import grpc
+
+import prime_pb2 as prime__pb2
+
+
+class PrimeCheckerStub(object):
+ """Service to check primality.
+ """
+
+ def __init__(self, channel):
+ """Constructor.
+
+ Args:
+ channel: A grpc.Channel.
+ """
+ self.check = channel.unary_unary(
+ '/prime.PrimeChecker/check',
+ request_serializer=prime__pb2.PrimeCandidate.SerializeToString,
+ response_deserializer=prime__pb2.Primality.FromString,
+ )
+
+
+class PrimeCheckerServicer(object):
+ """Service to check primality.
+ """
+
+ def check(self, request, context):
+ """Determines the primality of an integer.
+ """
+ context.set_code(grpc.StatusCode.UNIMPLEMENTED)
+ context.set_details('Method not implemented!')
+ raise NotImplementedError('Method not implemented!')
+
+
+def add_PrimeCheckerServicer_to_server(servicer, server):
+ rpc_method_handlers = {
+ 'check': grpc.unary_unary_rpc_method_handler(
+ servicer.check,
+ request_deserializer=prime__pb2.PrimeCandidate.FromString,
+ response_serializer=prime__pb2.Primality.SerializeToString,
+ ),
+ }
+ generic_handler = grpc.method_handlers_generic_handler(
+ 'prime.PrimeChecker', rpc_method_handlers)
+ server.add_generic_rpc_handlers((generic_handler,))
diff --git a/examples/python/multiprocessing/server.py b/examples/python/multiprocessing/server.py
new file mode 100644
index 00000000000..d30f3b6734d
--- /dev/null
+++ b/examples/python/multiprocessing/server.py
@@ -0,0 +1,98 @@
+# Copyright 2019 gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""An example of multiprocess concurrency with gRPC."""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+from concurrent import futures
+import datetime
+import grpc
+import logging
+import math
+import multiprocessing
+import os
+import time
+
+import prime_pb2
+import prime_pb2_grpc
+
+_ONE_DAY = datetime.timedelta(days=1)
+_NUM_PROCESSES = 8
+_THREAD_CONCURRENCY = 10
+_BIND_ADDRESS = '[::]:50051'
+
+
+def is_prime(n):
+ for i in range(2, math.ceil(math.sqrt(n))):
+ if i % n == 0:
+ return False
+ else:
+ return True
+
+
+class PrimeChecker(prime_pb2_grpc.PrimeCheckerServicer):
+
+ def check(self, request, context):
+ logging.info(
+ '[PID {}] Determining primality of {}'.format(
+ os.getpid(), request.candidate))
+ return is_prime(request.candidate)
+
+
+def _wait_forever(server):
+ try:
+ while True:
+ time.sleep(_ONE_DAY.total_seconds())
+ except KeyboardInterrupt:
+ server.stop(None)
+
+
+def _run_server(bind_address):
+ logging.warning( '[PID {}] Starting new server.'.format( os.getpid()))
+ options = (('grpc.so_reuseport', 1),)
+
+ # WARNING: This example takes advantage of SO_REUSEPORT. Due to the
+ # limitations of manylinux1, none of our precompiled Linux wheels currently
+ # support this option. (https://github.com/grpc/grpc/issues/18210). To take
+ # advantage of this feature, install from source with
+ # `pip install grpcio --no-binary grpcio`.
+
+ server = grpc.server(
+ futures.ThreadPoolExecutor(
+ max_workers=_THREAD_CONCURRENCY,),
+ options=options)
+ prime_pb2_grpc.add_PrimeCheckerServicer_to_server(PrimeChecker(), server)
+ server.add_insecure_port(bind_address)
+ server.start()
+ _wait_forever(server)
+
+
+def main():
+ workers = []
+ for _ in range(_NUM_PROCESSES):
+ # NOTE: It is imperative that the worker subprocesses be forked before
+ # any gRPC servers start up. See
+ # https://github.com/grpc/grpc/issues/16001 for more details.
+ worker = multiprocessing.Process(target=_run_server, args=(_BIND_ADDRESS,))
+ worker.start()
+ workers.append(worker)
+ for worker in workers:
+ worker.join()
+
+
+if __name__ == "__main__":
+ logging.basicConfig()
+ main()
diff --git a/examples/python/multiprocessing/test/_multiprocessing_test.py b/examples/python/multiprocessing/test/_multiprocessing_test.py
new file mode 100644
index 00000000000..e69de29bb2d
From f2c7ffc9fb35b112a3c097df324a94727f1b95fe Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 1 Mar 2019 13:27:16 -0800
Subject: [PATCH 009/138] Add multiprocessed client
---
examples/python/multiprocessing/client.py | 69 ++++++++++++++++++++++-
examples/python/multiprocessing/server.py | 14 ++---
2 files changed, 75 insertions(+), 8 deletions(-)
diff --git a/examples/python/multiprocessing/client.py b/examples/python/multiprocessing/client.py
index 4ab33374ce2..788820916ea 100644
--- a/examples/python/multiprocessing/client.py
+++ b/examples/python/multiprocessing/client.py
@@ -1,9 +1,76 @@
-# spin up multiple concurrent clients
+# Copyright 2019 gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""An example of multiprocessing concurrency with gRPC."""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import atexit
+import grpc
import logging
import multiprocessing
+import operator
import os
import time
import prime_pb2
import prime_pb2_grpc
+
+_PROCESS_COUNT = 8
+_SERVER_ADDRESS = 'localhost:50051'
+_MAXIMUM_CANDIDATE = 10000
+
+# Each worker process initializes a single channel after forking.
+_worker_channel_singleton = None
+_worker_stub_singleton = None
+
+
+def _initialize_worker(server_address):
+ global _worker_channel_singleton
+ global _worker_stub_singleton
+ logging.warning('[PID {}] Initializing worker process.'.format(
+ os.getpid()))
+ _worker_channel_singleton = grpc.insecure_channel(server_address)
+ _worker_stub_singleton = prime_pb2_grpc.PrimeCheckerStub(
+ _worker_channel_singleton)
+ atexit.register(_shutdown_worker)
+
+
+def _shutdown_worker():
+ logging.warning('[PID {}] Shutting worker process down.'.format(
+ os.getpid()))
+ if _worker_channel_singleton is not None:
+ _worker_channel_singleton.stop()
+
+
+def _run_worker_query(primality_candidate):
+ logging.warning('[PID {}] Checking primality of {}.'.format(
+ os.getpid(), primality_candidate))
+ return _worker_stub_singleton.check(
+ prime_pb2.PrimeCandidate(candidate=primality_candidate))
+
+
+def main():
+ worker_pool = multiprocessing.Pool(processes=_PROCESS_COUNT,
+ initializer=_initialize_worker, initargs=(_SERVER_ADDRESS,))
+ check_range = range(2, _MAXIMUM_CANDIDATE)
+ primality = worker_pool.map(_run_worker_query, check_range)
+ primes = zip(check_range, map(operator.attrgetter('isPrime'), primality))
+ logging.warning(tuple(primes))
+
+
+if __name__ == '__main__':
+ logging.basicConfig()
+ main()
diff --git a/examples/python/multiprocessing/server.py b/examples/python/multiprocessing/server.py
index d30f3b6734d..d0ca6a0cdfb 100644
--- a/examples/python/multiprocessing/server.py
+++ b/examples/python/multiprocessing/server.py
@@ -30,14 +30,14 @@ import prime_pb2
import prime_pb2_grpc
_ONE_DAY = datetime.timedelta(days=1)
-_NUM_PROCESSES = 8
+_PROCESS_COUNT = 8
_THREAD_CONCURRENCY = 10
_BIND_ADDRESS = '[::]:50051'
def is_prime(n):
- for i in range(2, math.ceil(math.sqrt(n))):
- if i % n == 0:
+ for i in range(2, int(math.ceil(math.sqrt(n)))):
+ if n % i == 0:
return False
else:
return True
@@ -46,10 +46,10 @@ def is_prime(n):
class PrimeChecker(prime_pb2_grpc.PrimeCheckerServicer):
def check(self, request, context):
- logging.info(
+ logging.warning(
'[PID {}] Determining primality of {}'.format(
os.getpid(), request.candidate))
- return is_prime(request.candidate)
+ return prime_pb2.Primality(isPrime=is_prime(request.candidate))
def _wait_forever(server):
@@ -82,7 +82,7 @@ def _run_server(bind_address):
def main():
workers = []
- for _ in range(_NUM_PROCESSES):
+ for _ in range(_PROCESS_COUNT):
# NOTE: It is imperative that the worker subprocesses be forked before
# any gRPC servers start up. See
# https://github.com/grpc/grpc/issues/16001 for more details.
@@ -93,6 +93,6 @@ def main():
worker.join()
-if __name__ == "__main__":
+if __name__ == '__main__':
logging.basicConfig()
main()
From a6c33d351667bec864cc8c860ac94385f60d76cb Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 1 Mar 2019 17:35:35 -0800
Subject: [PATCH 010/138] Dynamically allocate port
---
examples/python/multiprocessing/server.py | 38 +++++++++++++++++------
1 file changed, 28 insertions(+), 10 deletions(-)
diff --git a/examples/python/multiprocessing/server.py b/examples/python/multiprocessing/server.py
index d0ca6a0cdfb..f2a2544b827 100644
--- a/examples/python/multiprocessing/server.py
+++ b/examples/python/multiprocessing/server.py
@@ -18,6 +18,7 @@ from __future__ import division
from __future__ import print_function
from concurrent import futures
+import contextlib
import datetime
import grpc
import logging
@@ -25,6 +26,7 @@ import math
import multiprocessing
import os
import time
+import socket
import prime_pb2
import prime_pb2_grpc
@@ -61,6 +63,7 @@ def _wait_forever(server):
def _run_server(bind_address):
+ """Start a server in a subprocess."""
logging.warning( '[PID {}] Starting new server.'.format( os.getpid()))
options = (('grpc.so_reuseport', 1),)
@@ -80,17 +83,32 @@ def _run_server(bind_address):
_wait_forever(server)
+@contextlib.contextmanager
+def _reserve_port():
+ """Find and reserve a port for all subprocesses to use."""
+ sock = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
+ sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
+ sock.bind(('', 0))
+ try:
+ yield sock.getsockname()[1]
+ finally:
+ sock.close()
+
+
def main():
- workers = []
- for _ in range(_PROCESS_COUNT):
- # NOTE: It is imperative that the worker subprocesses be forked before
- # any gRPC servers start up. See
- # https://github.com/grpc/grpc/issues/16001 for more details.
- worker = multiprocessing.Process(target=_run_server, args=(_BIND_ADDRESS,))
- worker.start()
- workers.append(worker)
- for worker in workers:
- worker.join()
+ with _reserve_port() as port:
+ bind_address = '[::]:{}'.format(port)
+ logging.warning("Binding to {}".format(bind_address))
+ workers = []
+ for _ in range(_PROCESS_COUNT):
+ # NOTE: It is imperative that the worker subprocesses be forked before
+ # any gRPC servers start up. See
+ # https://github.com/grpc/grpc/issues/16001 for more details.
+ worker = multiprocessing.Process(target=_run_server, args=(bind_address,))
+ worker.start()
+ workers.append(worker)
+ for worker in workers:
+ worker.join()
if __name__ == '__main__':
From 510beaaede6faf2aa1c18c9ec9c77bcbee6c2f4c Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 1 Mar 2019 17:41:56 -0800
Subject: [PATCH 011/138] Add a CLI parser to the client
---
examples/python/multiprocessing/client.py | 15 +++++++++++----
examples/python/multiprocessing/server.py | 1 -
2 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/examples/python/multiprocessing/client.py b/examples/python/multiprocessing/client.py
index 788820916ea..920b5285475 100644
--- a/examples/python/multiprocessing/client.py
+++ b/examples/python/multiprocessing/client.py
@@ -17,6 +17,7 @@ from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
+import argparse
import atexit
import grpc
import logging
@@ -29,7 +30,6 @@ import prime_pb2
import prime_pb2_grpc
_PROCESS_COUNT = 8
-_SERVER_ADDRESS = 'localhost:50051'
_MAXIMUM_CANDIDATE = 10000
# Each worker process initializes a single channel after forking.
@@ -61,16 +61,23 @@ def _run_worker_query(primality_candidate):
return _worker_stub_singleton.check(
prime_pb2.PrimeCandidate(candidate=primality_candidate))
-
-def main():
+def _calculate_primes(server_address):
worker_pool = multiprocessing.Pool(processes=_PROCESS_COUNT,
- initializer=_initialize_worker, initargs=(_SERVER_ADDRESS,))
+ initializer=_initialize_worker, initargs=(server_address,))
check_range = range(2, _MAXIMUM_CANDIDATE)
primality = worker_pool.map(_run_worker_query, check_range)
primes = zip(check_range, map(operator.attrgetter('isPrime'), primality))
logging.warning(tuple(primes))
+def main():
+ msg = 'Determine the primality of the first {} integers.'.format(
+ _MAXIMUM_CANDIDATE)
+ parser = argparse.ArgumentParser(description=msg)
+ parser.add_argument('server_address', help='The address of the server (e.g. localhost:50051)')
+ args = parser.parse_args()
+ _calculate_primes(args.server_address)
+
if __name__ == '__main__':
logging.basicConfig()
main()
diff --git a/examples/python/multiprocessing/server.py b/examples/python/multiprocessing/server.py
index f2a2544b827..6801f806126 100644
--- a/examples/python/multiprocessing/server.py
+++ b/examples/python/multiprocessing/server.py
@@ -34,7 +34,6 @@ import prime_pb2_grpc
_ONE_DAY = datetime.timedelta(days=1)
_PROCESS_COUNT = 8
_THREAD_CONCURRENCY = 10
-_BIND_ADDRESS = '[::]:50051'
def is_prime(n):
From 059e10447534eeb2e43b5d4cb1f1d29afdb591ee Mon Sep 17 00:00:00 2001
From: Soheil Hassas Yeganeh <soheil@google.com>
Date: Sat, 2 Mar 2019 23:25:18 -0500
Subject: [PATCH 012/138] Use grpc_core::RefCount for ServerContext.
---
src/cpp/server/server_context.cc | 30 ++++++------------------------
1 file changed, 6 insertions(+), 24 deletions(-)
diff --git a/src/cpp/server/server_context.cc b/src/cpp/server/server_context.cc
index e116cf6e7d2..d38b46822ae 100644
--- a/src/cpp/server/server_context.cc
+++ b/src/cpp/server/server_context.cc
@@ -32,6 +32,7 @@
#include <grpcpp/impl/call.h>
#include <grpcpp/support/time.h>
+#include "src/core/lib/gprpp/ref_counted.h"
#include "src/core/lib/surface/call.h"
namespace grpc {
@@ -116,13 +117,7 @@ class ServerContext::CompletionOp final : public internal::CallOpSetInterface {
done_intercepting_ = true;
if (!has_tag_) {
/* We don't have a tag to return. */
- std::unique_lock<std::mutex> lock(mu_);
- if (--refs_ == 0) {
- lock.unlock();
- grpc_call* call = call_.call();
- delete this;
- grpc_call_unref(call);
- }
+ Unref();
return;
}
/* Start a dummy op so that we can return the tag */
@@ -142,8 +137,8 @@ class ServerContext::CompletionOp final : public internal::CallOpSetInterface {
bool has_tag_;
void* tag_;
void* core_cq_tag_;
+ grpc_core::RefCount refs_;
std::mutex mu_;
- int refs_;
bool finalized_;
int cancelled_; // This is an int (not bool) because it is passed to core
bool done_intercepting_;
@@ -151,9 +146,7 @@ class ServerContext::CompletionOp final : public internal::CallOpSetInterface {
};
void ServerContext::CompletionOp::Unref() {
- std::unique_lock<std::mutex> lock(mu_);
- if (--refs_ == 0) {
- lock.unlock();
+ if (refs_.Unref()) {
grpc_call* call = call_.call();
delete this;
grpc_call_unref(call);
@@ -183,12 +176,7 @@ bool ServerContext::CompletionOp::FinalizeResult(void** tag, bool* status) {
*tag = tag_;
ret = true;
}
- if (--refs_ == 0) {
- lock.unlock();
- grpc_call* call = call_.call();
- delete this;
- grpc_call_unref(call);
- }
+ Unref();
return ret;
}
finalized_ = true;
@@ -220,13 +208,7 @@ bool ServerContext::CompletionOp::FinalizeResult(void** tag, bool* status) {
*tag = tag_;
ret = true;
}
- lock.lock();
- if (--refs_ == 0) {
- lock.unlock();
- grpc_call* call = call_.call();
- delete this;
- grpc_call_unref(call);
- }
+ Unref();
return ret;
}
/* There are interceptors to be run. Return false for now */
From 05d8ddfc6e106e8a960e0a9fa54874d9e9298835 Mon Sep 17 00:00:00 2001
From: Vijay Pai <vpai@google.com>
Date: Tue, 26 Feb 2019 22:29:21 -0800
Subject: [PATCH 013/138] Support callback-based generic service
---
.../impl/codegen/async_generic_service.h | 51 ++++++
include/grpcpp/impl/codegen/server_context.h | 10 +-
.../grpcpp/impl/codegen/server_interface.h | 23 +++
include/grpcpp/server.h | 42 ++++-
include/grpcpp/server_builder.h | 10 +-
src/cpp/server/server_builder.cc | 24 ++-
src/cpp/server/server_cc.cc | 160 +++++++++++++-----
test/cpp/end2end/hybrid_end2end_test.cc | 122 ++++++++++---
8 files changed, 357 insertions(+), 85 deletions(-)
diff --git a/include/grpcpp/impl/codegen/async_generic_service.h b/include/grpcpp/impl/codegen/async_generic_service.h
index 2a0e1b40881..46489b135d7 100644
--- a/include/grpcpp/impl/codegen/async_generic_service.h
+++ b/include/grpcpp/impl/codegen/async_generic_service.h
@@ -21,6 +21,7 @@
#include <grpcpp/impl/codegen/async_stream.h>
#include <grpcpp/impl/codegen/byte_buffer.h>
+#include <grpcpp/impl/codegen/server_callback.h>
struct grpc_server;
@@ -41,6 +42,12 @@ class GenericServerContext final : public ServerContext {
friend class Server;
friend class ServerInterface;
+ void Clear() {
+ method_.clear();
+ host_.clear();
+ ServerContext::Clear();
+ }
+
grpc::string method_;
grpc::string host_;
};
@@ -76,6 +83,50 @@ class AsyncGenericService final {
Server* server_;
};
+namespace experimental {
+
+class ServerGenericBidiReactor
+ : public ServerBidiReactor<ByteBuffer, ByteBuffer> {
+ public:
+ void OnStarted(ServerContext* ctx) final {
+ OnStarted(static_cast<GenericServerContext*>(ctx));
+ }
+ virtual void OnStarted(GenericServerContext* ctx) {}
+};
+
+} // namespace experimental
+
+namespace internal {
+class UnimplementedGenericBidiReactor
+ : public experimental::ServerGenericBidiReactor {
+ public:
+ void OnDone() override { delete this; }
+ void OnStarted(GenericServerContext*) override {
+ this->Finish(Status(StatusCode::UNIMPLEMENTED, ""));
+ }
+};
+} // namespace internal
+
+namespace experimental {
+class CallbackGenericService {
+ public:
+ CallbackGenericService() {}
+ virtual ~CallbackGenericService() {}
+ virtual ServerGenericBidiReactor* CreateReactor() {
+ return new internal::UnimplementedGenericBidiReactor;
+ }
+
+ private:
+ friend class ::grpc::Server;
+
+ internal::CallbackBidiHandler<ByteBuffer, ByteBuffer>* Handler() {
+ return new internal::CallbackBidiHandler<ByteBuffer, ByteBuffer>(
+ [this] { return CreateReactor(); });
+ }
+
+ Server* server_{nullptr};
+};
+} // namespace experimental
} // namespace grpc
#endif // GRPCPP_IMPL_CODEGEN_ASYNC_GENERIC_SERVICE_H
diff --git a/include/grpcpp/impl/codegen/server_context.h b/include/grpcpp/impl/codegen/server_context.h
index affe61b547b..fb82186d69e 100644
--- a/include/grpcpp/impl/codegen/server_context.h
+++ b/include/grpcpp/impl/codegen/server_context.h
@@ -43,6 +43,10 @@ struct census_context;
namespace grpc {
class ClientContext;
+class GenericServerContext;
+class CompletionQueue;
+class Server;
+class ServerInterface;
template <class W, class R>
class ServerAsyncReader;
template <class W>
@@ -55,6 +59,7 @@ template <class R>
class ServerReader;
template <class W>
class ServerWriter;
+
namespace internal {
template <class W, class R>
class ServerReaderWriterBody;
@@ -82,10 +87,6 @@ class Call;
class ServerReactor;
} // namespace internal
-class CompletionQueue;
-class Server;
-class ServerInterface;
-
namespace testing {
class InteropServerContextInspector;
class ServerContextTestSpouse;
@@ -302,6 +303,7 @@ class ServerContext {
template <StatusCode code>
friend class internal::ErrorMethodHandler;
friend class ::grpc::ClientContext;
+ friend class ::grpc::GenericServerContext;
/// Prevent copying.
ServerContext(const ServerContext&);
diff --git a/include/grpcpp/impl/codegen/server_interface.h b/include/grpcpp/impl/codegen/server_interface.h
index 890a5650d02..f599e037fd5 100644
--- a/include/grpcpp/impl/codegen/server_interface.h
+++ b/include/grpcpp/impl/codegen/server_interface.h
@@ -47,6 +47,10 @@ namespace internal {
class ServerAsyncStreamingInterface;
} // namespace internal
+namespace experimental {
+class CallbackGenericService;
+} // namespace experimental
+
class ServerInterface : public internal::CallHook {
public:
virtual ~ServerInterface() {}
@@ -115,6 +119,25 @@ class ServerInterface : public internal::CallHook {
/// service. The service must exist for the lifetime of the Server instance.
virtual void RegisterAsyncGenericService(AsyncGenericService* service) = 0;
+ /// NOTE: class experimental_registration_interface is not part of the public
+ /// API of this class
+ /// TODO(vjpai): Move these contents to public API when no longer experimental
+ class experimental_registration_interface {
+ public:
+ virtual ~experimental_registration_interface() {}
+ /// May not be abstract since this is a post-1.0 API addition
+ virtual void RegisterCallbackGenericService(
+ experimental::CallbackGenericService* service) {}
+ };
+
+ /// NOTE: The function experimental_registration() is not stable public API.
+ /// It is a view to the experimental components of this class. It may be
+ /// changed or removed at any time. May not be abstract since this is a
+ /// post-1.0 API addition
+ virtual experimental_registration_interface* experimental_registration() {
+ return nullptr;
+ }
+
/// Tries to bind \a server to the given \a addr.
///
/// It can be invoked multiple times.
diff --git a/include/grpcpp/server.h b/include/grpcpp/server.h
index 248f20452a5..21c908aebdb 100644
--- a/include/grpcpp/server.h
+++ b/include/grpcpp/server.h
@@ -202,6 +202,8 @@ class Server : public ServerInterface, private GrpcLibraryCodegen {
friend class ServerInitializer;
class SyncRequest;
+ class CallbackRequestBase;
+ template <class ServerContextType>
class CallbackRequest;
class UnimplementedAsyncRequest;
class UnimplementedAsyncResponse;
@@ -216,6 +218,34 @@ class Server : public ServerInterface, private GrpcLibraryCodegen {
/// service. The service must exist for the lifetime of the Server instance.
void RegisterAsyncGenericService(AsyncGenericService* service) override;
+ /// NOTE: class experimental_registration_type is not part of the public API
+ /// of this class
+ /// TODO(vjpai): Move these contents to the public API of Server when
+ /// they are no longer experimental
+ class experimental_registration_type final
+ : public experimental_registration_interface {
+ public:
+ explicit experimental_registration_type(Server* server) : server_(server) {}
+ void RegisterCallbackGenericService(
+ experimental::CallbackGenericService* service) override {
+ server_->RegisterCallbackGenericService(service);
+ }
+
+ private:
+ Server* server_;
+ };
+
+ /// TODO(vjpai): Mark this override when experimental type above is deleted
+ void RegisterCallbackGenericService(
+ experimental::CallbackGenericService* service);
+
+ /// NOTE: The function experimental_registration() is not stable public API.
+ /// It is a view to the experimental components of this class. It may be
+ /// changed or removed at any time.
+ experimental_registration_interface* experimental_registration() override {
+ return &experimental_registration_;
+ }
+
void PerformOpsOnCall(internal::CallOpSetInterface* ops,
internal::Call* call) override;
@@ -257,7 +287,11 @@ class Server : public ServerInterface, private GrpcLibraryCodegen {
std::vector<gpr_atm> callback_unmatched_reqs_count_;
// List of callback requests to start when server actually starts.
- std::list<CallbackRequest*> callback_reqs_to_start_;
+ std::list<CallbackRequestBase*> callback_reqs_to_start_;
+
+ // For registering experimental callback generic service; remove when that
+ // method longer experimental
+ experimental_registration_type experimental_registration_{this};
// Server status
std::mutex mu_;
@@ -281,7 +315,8 @@ class Server : public ServerInterface, private GrpcLibraryCodegen {
std::shared_ptr<GlobalCallbacks> global_callbacks_;
std::vector<grpc::string> services_;
- bool has_generic_service_;
+ bool has_async_generic_service_{false};
+ bool has_callback_generic_service_{false};
// Pointer to the wrapped grpc_server.
grpc_server* server_;
@@ -294,6 +329,9 @@ class Server : public ServerInterface, private GrpcLibraryCodegen {
// A special handler for resource exhausted in sync case
std::unique_ptr<internal::MethodHandler> resource_exhausted_handler_;
+ // Handler for callback generic service, if any
+ std::unique_ptr<internal::MethodHandler> generic_handler_;
+
// callback_cq_ references the callbackable completion queue associated
// with this server (if any). It is set on the first call to CallbackCQ().
// It is _not owned_ by the server; ownership belongs with its internal
diff --git a/include/grpcpp/server_builder.h b/include/grpcpp/server_builder.h
index 028b8cffaa7..498e5b7bb31 100644
--- a/include/grpcpp/server_builder.h
+++ b/include/grpcpp/server_builder.h
@@ -49,6 +49,10 @@ namespace testing {
class ServerBuilderPluginTest;
} // namespace testing
+namespace experimental {
+class CallbackGenericService;
+} // namespace experimental
+
/// A builder class for the creation and startup of \a grpc::Server instances.
class ServerBuilder {
public:
@@ -227,6 +231,9 @@ class ServerBuilder {
builder_->interceptor_creators_ = std::move(interceptor_creators);
}
+ ServerBuilder& RegisterCallbackGenericService(
+ experimental::CallbackGenericService* service);
+
private:
ServerBuilder* builder_;
};
@@ -311,7 +318,8 @@ class ServerBuilder {
std::shared_ptr<ServerCredentials> creds_;
std::vector<std::unique_ptr<ServerBuilderPlugin>> plugins_;
grpc_resource_quota* resource_quota_;
- AsyncGenericService* generic_service_;
+ AsyncGenericService* generic_service_{nullptr};
+ experimental::CallbackGenericService* callback_generic_service_{nullptr};
struct {
bool is_set;
grpc_compression_level level;
diff --git a/src/cpp/server/server_builder.cc b/src/cpp/server/server_builder.cc
index b7fad558abb..cd0e516d9a3 100644
--- a/src/cpp/server/server_builder.cc
+++ b/src/cpp/server/server_builder.cc
@@ -44,8 +44,7 @@ ServerBuilder::ServerBuilder()
: max_receive_message_size_(INT_MIN),
max_send_message_size_(INT_MIN),
sync_server_settings_(SyncServerSettings()),
- resource_quota_(nullptr),
- generic_service_(nullptr) {
+ resource_quota_(nullptr) {
gpr_once_init(&once_init_plugin_list, do_plugin_list_init);
for (auto it = g_plugin_factory_list->begin();
it != g_plugin_factory_list->end(); it++) {
@@ -91,9 +90,9 @@ ServerBuilder& ServerBuilder::RegisterService(const grpc::string& addr,
ServerBuilder& ServerBuilder::RegisterAsyncGenericService(
AsyncGenericService* service) {
- if (generic_service_) {
+ if (generic_service_ || callback_generic_service_) {
gpr_log(GPR_ERROR,
- "Adding multiple AsyncGenericService is unsupported for now. "
+ "Adding multiple generic services is unsupported for now. "
"Dropping the service %p",
(void*)service);
} else {
@@ -102,6 +101,19 @@ ServerBuilder& ServerBuilder::RegisterAsyncGenericService(
return *this;
}
+ServerBuilder& ServerBuilder::experimental_type::RegisterCallbackGenericService(
+ experimental::CallbackGenericService* service) {
+ if (builder_->generic_service_ || builder_->callback_generic_service_) {
+ gpr_log(GPR_ERROR,
+ "Adding multiple generic services is unsupported for now. "
+ "Dropping the service %p",
+ (void*)service);
+ } else {
+ builder_->callback_generic_service_ = service;
+ }
+ return *builder_;
+}
+
ServerBuilder& ServerBuilder::SetOption(
std::unique_ptr<ServerBuilderOption> option) {
options_.push_back(std::move(option));
@@ -310,7 +322,7 @@ std::unique_ptr<Server> ServerBuilder::BuildAndStart() {
has_frequently_polled_cqs = true;
}
- if (has_callback_methods) {
+ if (has_callback_methods || callback_generic_service_ != nullptr) {
auto* cq = server->CallbackCQ();
grpc_server_register_completion_queue(server->server_, cq->cq(), nullptr);
}
@@ -344,6 +356,8 @@ std::unique_ptr<Server> ServerBuilder::BuildAndStart() {
if (generic_service_) {
server->RegisterAsyncGenericService(generic_service_);
+ } else if (callback_generic_service_) {
+ server->RegisterCallbackGenericService(callback_generic_service_);
} else {
for (auto it = services_.begin(); it != services_.end(); ++it) {
if ((*it)->service->has_generic_methods()) {
diff --git a/src/cpp/server/server_cc.cc b/src/cpp/server/server_cc.cc
index 7eb0f2372b6..4d5c8179fce 100644
--- a/src/cpp/server/server_cc.cc
+++ b/src/cpp/server/server_cc.cc
@@ -19,6 +19,7 @@
#include <cstdlib>
#include <sstream>
+#include <type_traits>
#include <utility>
#include <grpc/grpc.h>
@@ -348,8 +349,18 @@ class Server::SyncRequest final : public internal::CompletionQueueTag {
grpc_completion_queue* cq_;
};
-class Server::CallbackRequest final : public internal::CompletionQueueTag {
+class Server::CallbackRequestBase : public internal::CompletionQueueTag {
public:
+ virtual ~CallbackRequestBase() {}
+ virtual bool Request() = 0;
+};
+
+template <class ServerContextType>
+class Server::CallbackRequest final : public Server::CallbackRequestBase {
+ public:
+ static_assert(std::is_base_of<ServerContext, ServerContextType>::value,
+ "ServerContextType must be derived from ServerContext");
+
CallbackRequest(Server* server, size_t method_idx,
internal::RpcServiceMethod* method, void* method_tag)
: server_(server),
@@ -357,8 +368,9 @@ class Server::CallbackRequest final : public internal::CompletionQueueTag {
method_(method),
method_tag_(method_tag),
has_request_payload_(
- method->method_type() == internal::RpcMethod::NORMAL_RPC ||
- method->method_type() == internal::RpcMethod::SERVER_STREAMING),
+ method_ != nullptr &&
+ (method->method_type() == internal::RpcMethod::NORMAL_RPC ||
+ method->method_type() == internal::RpcMethod::SERVER_STREAMING)),
cq_(server->CallbackCQ()),
tag_(this) {
server_->callback_reqs_outstanding_++;
@@ -376,7 +388,7 @@ class Server::CallbackRequest final : public internal::CompletionQueueTag {
}
}
- bool Request() {
+ bool Request() override {
if (method_tag_) {
if (GRPC_CALL_OK !=
grpc_server_request_registered_call(
@@ -400,12 +412,18 @@ class Server::CallbackRequest final : public internal::CompletionQueueTag {
return true;
}
- bool FinalizeResult(void** tag, bool* status) override { return false; }
+ // Needs specialization to account for different processing of metadata
+ // in generic API
+ bool FinalizeResult(void** tag, bool* status) override;
private:
+ // method_name needs to be specialized between named method and generic
+ const char* method_name() const;
+
class CallbackCallTag : public grpc_experimental_completion_queue_functor {
public:
- CallbackCallTag(Server::CallbackRequest* req) : req_(req) {
+ CallbackCallTag(Server::CallbackRequest<ServerContextType>* req)
+ : req_(req) {
functor_run = &CallbackCallTag::StaticRun;
}
@@ -415,7 +433,7 @@ class Server::CallbackRequest final : public internal::CompletionQueueTag {
void force_run(bool ok) { Run(ok); }
private:
- Server::CallbackRequest* req_;
+ Server::CallbackRequest<ServerContextType>* req_;
internal::Call* call_;
static void StaticRun(grpc_experimental_completion_queue_functor* cb,
@@ -446,8 +464,9 @@ class Server::CallbackRequest final : public internal::CompletionQueueTag {
if (count == 0 || (count < SOFT_MINIMUM_SPARE_CALLBACK_REQS_PER_METHOD &&
req_->server_->callback_reqs_outstanding_ <
SOFT_MAXIMUM_CALLBACK_REQS_OUTSTANDING)) {
- auto* new_req = new CallbackRequest(req_->server_, req_->method_index_,
- req_->method_, req_->method_tag_);
+ auto* new_req = new CallbackRequest<ServerContextType>(
+ req_->server_, req_->method_index_, req_->method_,
+ req_->method_tag_);
if (!new_req->Request()) {
// The server must have just decided to shutdown.
gpr_atm_no_barrier_fetch_add(
@@ -467,12 +486,14 @@ class Server::CallbackRequest final : public internal::CompletionQueueTag {
// Create a C++ Call to control the underlying core call
call_ = new (grpc_call_arena_alloc(req_->call_, sizeof(internal::Call)))
- internal::Call(
- req_->call_, req_->server_, req_->cq_,
- req_->server_->max_receive_message_size(),
- req_->ctx_.set_server_rpc_info(
- req_->method_->name(), req_->method_->method_type(),
- req_->server_->interceptor_creators_));
+ internal::Call(req_->call_, req_->server_, req_->cq_,
+ req_->server_->max_receive_message_size(),
+ req_->ctx_.set_server_rpc_info(
+ req_->method_name(),
+ (req_->method_ != nullptr)
+ ? req_->method_->method_type()
+ : internal::RpcMethod::BIDI_STREAMING,
+ req_->server_->interceptor_creators_));
req_->interceptor_methods_.SetCall(call_);
req_->interceptor_methods_.SetReverse();
@@ -501,31 +522,32 @@ class Server::CallbackRequest final : public internal::CompletionQueueTag {
}
}
void ContinueRunAfterInterception() {
- req_->method_->handler()->RunHandler(
- internal::MethodHandler::HandlerParameter(
- call_, &req_->ctx_, req_->request_, req_->request_status_,
- [this] {
- // Recycle this request if there aren't too many outstanding.
- // Note that we don't have to worry about a case where there
- // are no requests waiting to match for this method since that
- // is already taken care of when binding a request to a call.
- // TODO(vjpai): Also don't recycle this request if the dynamic
- // load no longer justifies it. Consider measuring
- // dynamic load and setting a target accordingly.
- if (req_->server_->callback_reqs_outstanding_ <
- SOFT_MAXIMUM_CALLBACK_REQS_OUTSTANDING) {
- req_->Clear();
- req_->Setup();
- } else {
- // We can free up this request because there are too many
- delete req_;
- return;
- }
- if (!req_->Request()) {
- // The server must have just decided to shutdown.
- delete req_;
- }
- }));
+ auto* handler = (req_->method_ != nullptr)
+ ? req_->method_->handler()
+ : req_->server_->generic_handler_.get();
+ handler->RunHandler(internal::MethodHandler::HandlerParameter(
+ call_, &req_->ctx_, req_->request_, req_->request_status_, [this] {
+ // Recycle this request if there aren't too many outstanding.
+ // Note that we don't have to worry about a case where there
+ // are no requests waiting to match for this method since that
+ // is already taken care of when binding a request to a call.
+ // TODO(vjpai): Also don't recycle this request if the dynamic
+ // load no longer justifies it. Consider measuring
+ // dynamic load and setting a target accordingly.
+ if (req_->server_->callback_reqs_outstanding_ <
+ SOFT_MAXIMUM_CALLBACK_REQS_OUTSTANDING) {
+ req_->Clear();
+ req_->Setup();
+ } else {
+ // We can free up this request because there are too many
+ delete req_;
+ return;
+ }
+ if (!req_->Request()) {
+ // The server must have just decided to shutdown.
+ delete req_;
+ }
+ }));
}
};
@@ -553,7 +575,7 @@ class Server::CallbackRequest final : public internal::CompletionQueueTag {
}
Server* const server_;
- size_t method_index_;
+ const size_t method_index_;
internal::RpcServiceMethod* const method_;
void* const method_tag_;
const bool has_request_payload_;
@@ -566,10 +588,39 @@ class Server::CallbackRequest final : public internal::CompletionQueueTag {
grpc_metadata_array request_metadata_;
CompletionQueue* cq_;
CallbackCallTag tag_;
- ServerContext ctx_;
+ ServerContextType ctx_;
internal::InterceptorBatchMethodsImpl interceptor_methods_;
};
+template <>
+bool Server::CallbackRequest<ServerContext>::FinalizeResult(void** tag,
+ bool* status) {
+ return false;
+}
+
+template <>
+bool Server::CallbackRequest<GenericServerContext>::FinalizeResult(
+ void** tag, bool* status) {
+ if (*status) {
+ // TODO(yangg) remove the copy here
+ ctx_.method_ = StringFromCopiedSlice(call_details_->method);
+ ctx_.host_ = StringFromCopiedSlice(call_details_->host);
+ }
+ grpc_slice_unref(call_details_->method);
+ grpc_slice_unref(call_details_->host);
+ return false;
+}
+
+template <>
+const char* Server::CallbackRequest<ServerContext>::method_name() const {
+ return method_->name();
+}
+
+template <>
+const char* Server::CallbackRequest<GenericServerContext>::method_name() const {
+ return ctx_.method().c_str();
+}
+
// Implementation of ThreadManager. Each instance of SyncRequestThreadManager
// manages a pool of threads that poll for incoming Sync RPCs and call the
// appropriate RPC handlers
@@ -708,7 +759,6 @@ Server::Server(
started_(false),
shutdown_(false),
shutdown_notified_(false),
- has_generic_service_(false),
server_(nullptr),
server_initializer_(new ServerInitializer(this)),
health_check_service_disabled_(false) {
@@ -865,7 +915,7 @@ bool Server::RegisterService(const grpc::string* host, Service* service) {
auto method_index = callback_unmatched_reqs_count_.size() - 1;
// TODO(vjpai): Register these dynamically based on need
for (int i = 0; i < DEFAULT_CALLBACK_REQS_PER_METHOD; i++) {
- callback_reqs_to_start_.push_back(new CallbackRequest(
+ callback_reqs_to_start_.push_back(new CallbackRequest<ServerContext>(
this, method_index, method, method_registration_tag));
}
// Enqueue it so that it will be Request'ed later after all request
@@ -891,7 +941,25 @@ void Server::RegisterAsyncGenericService(AsyncGenericService* service) {
GPR_ASSERT(service->server_ == nullptr &&
"Can only register an async generic service against one server.");
service->server_ = this;
- has_generic_service_ = true;
+ has_async_generic_service_ = true;
+}
+
+void Server::RegisterCallbackGenericService(
+ experimental::CallbackGenericService* service) {
+ GPR_ASSERT(
+ service->server_ == nullptr &&
+ "Can only register a callback generic service against one server.");
+ service->server_ = this;
+ has_callback_generic_service_ = true;
+ generic_handler_.reset(service->Handler());
+
+ callback_unmatched_reqs_count_.push_back(0);
+ auto method_index = callback_unmatched_reqs_count_.size() - 1;
+ // TODO(vjpai): Register these dynamically based on need
+ for (int i = 0; i < DEFAULT_CALLBACK_REQS_PER_METHOD; i++) {
+ callback_reqs_to_start_.push_back(new CallbackRequest<GenericServerContext>(
+ this, method_index, nullptr, nullptr));
+ }
}
int Server::AddListeningPort(const grpc::string& addr,
@@ -932,7 +1000,7 @@ void Server::Start(ServerCompletionQueue** cqs, size_t num_cqs) {
grpc_server_start(server_);
- if (!has_generic_service_) {
+ if (!has_async_generic_service_ && !has_callback_generic_service_) {
for (auto it = sync_req_mgrs_.begin(); it != sync_req_mgrs_.end(); it++) {
(*it)->AddUnknownSyncMethod();
}
diff --git a/test/cpp/end2end/hybrid_end2end_test.cc b/test/cpp/end2end/hybrid_end2end_test.cc
index 18bb1ff4b96..b0dd901cf11 100644
--- a/test/cpp/end2end/hybrid_end2end_test.cc
+++ b/test/cpp/end2end/hybrid_end2end_test.cc
@@ -28,6 +28,7 @@
#include <grpcpp/server_builder.h>
#include <grpcpp/server_context.h>
+#include "src/core/lib/iomgr/iomgr.h"
#include "src/proto/grpc/testing/duplicate/echo_duplicate.grpc.pb.h"
#include "src/proto/grpc/testing/echo.grpc.pb.h"
#include "test/core/util/port.h"
@@ -39,7 +40,6 @@
namespace grpc {
namespace testing {
-
namespace {
void* tag(int i) { return (void*)static_cast<intptr_t>(i); }
@@ -225,13 +225,23 @@ class TestServiceImplDupPkg
}
};
-class HybridEnd2endTest : public ::testing::Test {
+class HybridEnd2endTest : public ::testing::TestWithParam<bool> {
protected:
HybridEnd2endTest() {}
- void SetUpServer(::grpc::Service* service1, ::grpc::Service* service2,
- AsyncGenericService* generic_service,
- int max_message_size = 0) {
+ void SetUp() override {
+ inproc_ = (::testing::UnitTest::GetInstance()
+ ->current_test_info()
+ ->value_param() != nullptr)
+ ? GetParam()
+ : false;
+ }
+
+ bool SetUpServer(
+ ::grpc::Service* service1, ::grpc::Service* service2,
+ AsyncGenericService* generic_service,
+ experimental::CallbackGenericService* callback_generic_service,
+ int max_message_size = 0) {
int port = grpc_pick_unused_port_or_die();
server_address_ << "localhost:" << port;
@@ -249,6 +259,10 @@ class HybridEnd2endTest : public ::testing::Test {
if (generic_service) {
builder.RegisterAsyncGenericService(generic_service);
}
+ if (callback_generic_service) {
+ builder.experimental().RegisterCallbackGenericService(
+ callback_generic_service);
+ }
if (max_message_size != 0) {
builder.SetMaxMessageSize(max_message_size);
@@ -259,6 +273,11 @@ class HybridEnd2endTest : public ::testing::Test {
cqs_.push_back(builder.AddCompletionQueue(false));
}
server_ = builder.BuildAndStart();
+
+ // If there is a generic callback service, this setup is only successful if
+ // we have an iomgr that can run in the background or are inprocess
+ return !callback_generic_service || grpc_iomgr_run_in_background() ||
+ inproc_;
}
void TearDown() override {
@@ -276,7 +295,9 @@ class HybridEnd2endTest : public ::testing::Test {
void ResetStub() {
std::shared_ptr<Channel> channel =
- CreateChannel(server_address_.str(), InsecureChannelCredentials());
+ inproc_ ? server_->InProcessChannel(ChannelArguments())
+ : CreateChannel(server_address_.str(),
+ InsecureChannelCredentials());
stub_ = grpc::testing::EchoTestService::NewStub(channel);
}
@@ -411,12 +432,13 @@ class HybridEnd2endTest : public ::testing::Test {
std::unique_ptr<grpc::testing::EchoTestService::Stub> stub_;
std::unique_ptr<Server> server_;
std::ostringstream server_address_;
+ bool inproc_;
};
TEST_F(HybridEnd2endTest, AsyncEcho) {
typedef EchoTestService::WithAsyncMethod_Echo<TestServiceImpl> SType;
SType service;
- SetUpServer(&service, nullptr, nullptr);
+ SetUpServer(&service, nullptr, nullptr, nullptr);
ResetStub();
std::thread echo_handler_thread(HandleEcho<SType>, &service, cqs_[0].get(),
false);
@@ -427,7 +449,7 @@ TEST_F(HybridEnd2endTest, AsyncEcho) {
TEST_F(HybridEnd2endTest, RawEcho) {
typedef EchoTestService::WithRawMethod_Echo<TestServiceImpl> SType;
SType service;
- SetUpServer(&service, nullptr, nullptr);
+ SetUpServer(&service, nullptr, nullptr, nullptr);
ResetStub();
std::thread echo_handler_thread(HandleRawEcho<SType>, &service, cqs_[0].get(),
false);
@@ -438,7 +460,7 @@ TEST_F(HybridEnd2endTest, RawEcho) {
TEST_F(HybridEnd2endTest, RawRequestStream) {
typedef EchoTestService::WithRawMethod_RequestStream<TestServiceImpl> SType;
SType service;
- SetUpServer(&service, nullptr, nullptr);
+ SetUpServer(&service, nullptr, nullptr, nullptr);
ResetStub();
std::thread request_stream_handler_thread(HandleRawClientStreaming<SType>,
&service, cqs_[0].get());
@@ -451,7 +473,7 @@ TEST_F(HybridEnd2endTest, AsyncEchoRawRequestStream) {
EchoTestService::WithAsyncMethod_Echo<TestServiceImpl>>
SType;
SType service;
- SetUpServer(&service, nullptr, nullptr);
+ SetUpServer(&service, nullptr, nullptr, nullptr);
ResetStub();
std::thread echo_handler_thread(HandleEcho<SType>, &service, cqs_[0].get(),
false);
@@ -468,7 +490,7 @@ TEST_F(HybridEnd2endTest, GenericEchoRawRequestStream) {
SType;
SType service;
AsyncGenericService generic_service;
- SetUpServer(&service, nullptr, &generic_service);
+ SetUpServer(&service, nullptr, &generic_service, nullptr);
ResetStub();
std::thread generic_handler_thread(HandleGenericCall, &generic_service,
cqs_[0].get());
@@ -484,7 +506,7 @@ TEST_F(HybridEnd2endTest, AsyncEchoRequestStream) {
EchoTestService::WithAsyncMethod_Echo<TestServiceImpl>>
SType;
SType service;
- SetUpServer(&service, nullptr, nullptr);
+ SetUpServer(&service, nullptr, nullptr, nullptr);
ResetStub();
std::thread echo_handler_thread(HandleEcho<SType>, &service, cqs_[0].get(),
false);
@@ -500,7 +522,7 @@ TEST_F(HybridEnd2endTest, AsyncRequestStreamResponseStream) {
EchoTestService::WithAsyncMethod_ResponseStream<TestServiceImpl>>
SType;
SType service;
- SetUpServer(&service, nullptr, nullptr);
+ SetUpServer(&service, nullptr, nullptr, nullptr);
ResetStub();
std::thread response_stream_handler_thread(HandleServerStreaming<SType>,
&service, cqs_[0].get());
@@ -518,7 +540,7 @@ TEST_F(HybridEnd2endTest, AsyncRequestStreamResponseStream_SyncDupService) {
SType;
SType service;
TestServiceImplDupPkg dup_service;
- SetUpServer(&service, &dup_service, nullptr);
+ SetUpServer(&service, &dup_service, nullptr, nullptr);
ResetStub();
std::thread response_stream_handler_thread(HandleServerStreaming<SType>,
&service, cqs_[0].get());
@@ -557,7 +579,7 @@ TEST_F(HybridEnd2endTest,
SType;
SType service;
StreamedUnaryDupPkg dup_service;
- SetUpServer(&service, &dup_service, nullptr, 8192);
+ SetUpServer(&service, &dup_service, nullptr, nullptr, 8192);
ResetStub();
std::thread response_stream_handler_thread(HandleServerStreaming<SType>,
&service, cqs_[0].get());
@@ -595,7 +617,7 @@ TEST_F(HybridEnd2endTest,
SType;
SType service;
FullyStreamedUnaryDupPkg dup_service;
- SetUpServer(&service, &dup_service, nullptr, 8192);
+ SetUpServer(&service, &dup_service, nullptr, nullptr, 8192);
ResetStub();
std::thread response_stream_handler_thread(HandleServerStreaming<SType>,
&service, cqs_[0].get());
@@ -636,7 +658,7 @@ TEST_F(HybridEnd2endTest,
SType;
SType service;
SplitResponseStreamDupPkg dup_service;
- SetUpServer(&service, &dup_service, nullptr, 8192);
+ SetUpServer(&service, &dup_service, nullptr, nullptr, 8192);
ResetStub();
std::thread response_stream_handler_thread(HandleServerStreaming<SType>,
&service, cqs_[0].get());
@@ -676,7 +698,7 @@ TEST_F(HybridEnd2endTest,
SType;
SType service;
FullySplitStreamedDupPkg dup_service;
- SetUpServer(&service, &dup_service, nullptr, 8192);
+ SetUpServer(&service, &dup_service, nullptr, nullptr, 8192);
ResetStub();
std::thread response_stream_handler_thread(HandleServerStreaming<SType>,
&service, cqs_[0].get());
@@ -728,7 +750,7 @@ TEST_F(HybridEnd2endTest,
SType;
SType service;
FullyStreamedDupPkg dup_service;
- SetUpServer(&service, &dup_service, nullptr, 8192);
+ SetUpServer(&service, &dup_service, nullptr, nullptr, 8192);
ResetStub();
std::thread response_stream_handler_thread(HandleServerStreaming<SType>,
&service, cqs_[0].get());
@@ -748,7 +770,7 @@ TEST_F(HybridEnd2endTest, AsyncRequestStreamResponseStream_AsyncDupService) {
SType;
SType service;
duplicate::EchoTestService::AsyncService dup_service;
- SetUpServer(&service, &dup_service, nullptr);
+ SetUpServer(&service, &dup_service, nullptr, nullptr);
ResetStub();
std::thread response_stream_handler_thread(HandleServerStreaming<SType>,
&service, cqs_[0].get());
@@ -767,7 +789,7 @@ TEST_F(HybridEnd2endTest, AsyncRequestStreamResponseStream_AsyncDupService) {
TEST_F(HybridEnd2endTest, GenericEcho) {
EchoTestService::WithGenericMethod_Echo<TestServiceImpl> service;
AsyncGenericService generic_service;
- SetUpServer(&service, nullptr, &generic_service);
+ SetUpServer(&service, nullptr, &generic_service, nullptr);
ResetStub();
std::thread generic_handler_thread(HandleGenericCall, &generic_service,
cqs_[0].get());
@@ -775,13 +797,56 @@ TEST_F(HybridEnd2endTest, GenericEcho) {
generic_handler_thread.join();
}
+TEST_P(HybridEnd2endTest, CallbackGenericEcho) {
+ EchoTestService::WithGenericMethod_Echo<TestServiceImpl> service;
+ class GenericEchoService : public experimental::CallbackGenericService {
+ private:
+ experimental::ServerGenericBidiReactor* CreateReactor() override {
+ class Reactor : public experimental::ServerGenericBidiReactor {
+ private:
+ void OnStarted(GenericServerContext* ctx) override {
+ ctx_ = ctx;
+ EXPECT_EQ(ctx->method(), "/grpc.testing.EchoTestService/Echo");
+ StartRead(&request_);
+ }
+ void OnDone() override { delete this; }
+ void OnReadDone(bool ok) override {
+ if (!ok) {
+ EXPECT_EQ(reads_complete_, 1);
+ } else {
+ EXPECT_EQ(reads_complete_++, 0);
+ response_ = request_;
+ StartWrite(&response_);
+ StartRead(&request_);
+ }
+ }
+ void OnWriteDone(bool ok) override {
+ Finish(ok ? Status::OK
+ : Status(StatusCode::UNKNOWN, "Unexpected failure"));
+ }
+ GenericServerContext* ctx_;
+ ByteBuffer request_;
+ ByteBuffer response_;
+ std::atomic_int reads_complete_{0};
+ };
+ return new Reactor;
+ }
+ } generic_service;
+
+ if (!SetUpServer(&service, nullptr, nullptr, &generic_service)) {
+ return;
+ }
+ ResetStub();
+ TestAllMethods();
+}
+
TEST_F(HybridEnd2endTest, GenericEchoAsyncRequestStream) {
typedef EchoTestService::WithAsyncMethod_RequestStream<
EchoTestService::WithGenericMethod_Echo<TestServiceImpl>>
SType;
SType service;
AsyncGenericService generic_service;
- SetUpServer(&service, nullptr, &generic_service);
+ SetUpServer(&service, nullptr, &generic_service, nullptr);
ResetStub();
std::thread generic_handler_thread(HandleGenericCall, &generic_service,
cqs_[0].get());
@@ -800,7 +865,7 @@ TEST_F(HybridEnd2endTest, GenericEchoAsyncRequestStream_SyncDupService) {
SType service;
AsyncGenericService generic_service;
TestServiceImplDupPkg dup_service;
- SetUpServer(&service, &dup_service, &generic_service);
+ SetUpServer(&service, &dup_service, &generic_service, nullptr);
ResetStub();
std::thread generic_handler_thread(HandleGenericCall, &generic_service,
cqs_[0].get());
@@ -820,7 +885,7 @@ TEST_F(HybridEnd2endTest, GenericEchoAsyncRequestStream_AsyncDupService) {
SType service;
AsyncGenericService generic_service;
duplicate::EchoTestService::AsyncService dup_service;
- SetUpServer(&service, &dup_service, &generic_service);
+ SetUpServer(&service, &dup_service, &generic_service, nullptr);
ResetStub();
std::thread generic_handler_thread(HandleGenericCall, &generic_service,
cqs_[0].get());
@@ -843,7 +908,7 @@ TEST_F(HybridEnd2endTest, GenericEchoAsyncRequestStreamResponseStream) {
SType;
SType service;
AsyncGenericService generic_service;
- SetUpServer(&service, nullptr, &generic_service);
+ SetUpServer(&service, nullptr, &generic_service, nullptr);
ResetStub();
std::thread generic_handler_thread(HandleGenericCall, &generic_service,
cqs_[0].get());
@@ -864,7 +929,7 @@ TEST_F(HybridEnd2endTest, GenericEchoRequestStreamAsyncResponseStream) {
SType;
SType service;
AsyncGenericService generic_service;
- SetUpServer(&service, nullptr, &generic_service);
+ SetUpServer(&service, nullptr, &generic_service, nullptr);
ResetStub();
std::thread generic_handler_thread(HandleGenericCall, &generic_service,
cqs_[0].get());
@@ -885,10 +950,13 @@ TEST_F(HybridEnd2endTest, GenericMethodWithoutGenericService) {
EchoTestService::WithGenericMethod_Echo<
EchoTestService::WithAsyncMethod_ResponseStream<TestServiceImpl>>>
service;
- SetUpServer(&service, nullptr, nullptr);
+ SetUpServer(&service, nullptr, nullptr, nullptr);
EXPECT_EQ(nullptr, server_.get());
}
+INSTANTIATE_TEST_CASE_P(HybridEnd2endTest, HybridEnd2endTest,
+ ::testing::Bool());
+
} // namespace
} // namespace testing
} // namespace grpc
From ede3e61acb419057259e740141ad3d2ac606553a Mon Sep 17 00:00:00 2001
From: Prashant Jaikumar <pjaikumar@google.com>
Date: Mon, 4 Mar 2019 13:25:31 -0800
Subject: [PATCH 014/138] Try to fix
FlakyNetworkTest.ServerRestartKeepaliveDisabled flake
Set channel arg for max connect time.
---
test/cpp/end2end/flaky_network_test.cc | 8 +++++++-
tools/internal_ci/linux/grpc_flaky_network_in_docker.sh | 2 +-
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/test/cpp/end2end/flaky_network_test.cc b/test/cpp/end2end/flaky_network_test.cc
index d0c95740959..63a6897f931 100644
--- a/test/cpp/end2end/flaky_network_test.cc
+++ b/test/cpp/end2end/flaky_network_test.cc
@@ -345,9 +345,12 @@ TEST_F(FlakyNetworkTest, ServerUnreachableWithKeepalive) {
args.SetInt(GRPC_ARG_KEEPALIVE_TIMEOUT_MS, kKeepAliveTimeoutMs);
args.SetInt(GRPC_ARG_KEEPALIVE_PERMIT_WITHOUT_CALLS, 1);
args.SetInt(GRPC_ARG_HTTP2_MAX_PINGS_WITHOUT_DATA, 0);
- args.SetInt(GRPC_ARG_INITIAL_RECONNECT_BACKOFF_MS, kReconnectBackoffMs);
+ // max time for a connection attempt
+ args.SetInt(GRPC_ARG_MIN_RECONNECT_BACKOFF_MS, kReconnectBackoffMs);
+ // max time between reconnect attempts
args.SetInt(GRPC_ARG_MAX_RECONNECT_BACKOFF_MS, kReconnectBackoffMs);
+ gpr_log(GPR_DEBUG, "FlakyNetworkTest.ServerUnreachableWithKeepalive start");
auto channel = BuildChannel("pick_first", args);
auto stub = BuildStub(channel);
// Channel should be in READY state after we send an RPC
@@ -366,15 +369,18 @@ TEST_F(FlakyNetworkTest, ServerUnreachableWithKeepalive) {
});
// break network connectivity
+ gpr_log(GPR_DEBUG, "Adding iptables rule to drop packets");
DropPackets();
std::this_thread::sleep_for(std::chrono::milliseconds(10000));
EXPECT_TRUE(WaitForChannelNotReady(channel.get()));
// bring network interface back up
RestoreNetwork();
+ gpr_log(GPR_DEBUG, "Removed iptables rule to drop packets");
EXPECT_TRUE(WaitForChannelReady(channel.get()));
EXPECT_EQ(channel->GetState(false), GRPC_CHANNEL_READY);
shutdown.store(true);
sender.join();
+ gpr_log(GPR_DEBUG, "FlakyNetworkTest.ServerUnreachableWithKeepalive end");
}
//
diff --git a/tools/internal_ci/linux/grpc_flaky_network_in_docker.sh b/tools/internal_ci/linux/grpc_flaky_network_in_docker.sh
index eb6216c62c3..7fc8f146727 100755
--- a/tools/internal_ci/linux/grpc_flaky_network_in_docker.sh
+++ b/tools/internal_ci/linux/grpc_flaky_network_in_docker.sh
@@ -28,4 +28,4 @@ cd /var/local/git/grpc/test/cpp/end2end
# iptables is used to drop traffic between client and server
apt-get install -y iptables
-bazel test --spawn_strategy=standalone --genrule_strategy=standalone --test_output=all :flaky_network_test --test_env=GRPC_VERBOSITY=debug --test_env=GRPC_TRACE=channel,client_channel,call_error,connectivity_state
+bazel test --spawn_strategy=standalone --genrule_strategy=standalone --test_output=all :flaky_network_test --test_env=GRPC_VERBOSITY=debug --test_env=GRPC_TRACE=channel,client_channel,call_error,connectivity_state,tcp
From bc81010f10e31af51f2a790f1adf2500d4aba0ad Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Mon, 4 Mar 2019 12:12:37 -0800
Subject: [PATCH 015/138] Strip Python wheel binary
---
.../artifacts/build_package_python.sh | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/tools/run_tests/artifacts/build_package_python.sh b/tools/run_tests/artifacts/build_package_python.sh
index 29801a5b867..193d75db62a 100755
--- a/tools/run_tests/artifacts/build_package_python.sh
+++ b/tools/run_tests/artifacts/build_package_python.sh
@@ -23,6 +23,24 @@ mkdir -p artifacts/
# and we only collect them here to deliver them to the distribtest phase.
cp -r "${EXTERNAL_GIT_ROOT}"/input_artifacts/python_*/* artifacts/ || true
+strip_binary_wheel() {
+ WHEEL_PATH="$1"
+ TEMP_WHEEL_DIR=$(mktemp -d)
+ wheel unpack "$WHEEL_PATH" -d "$TEMP_WHEEL_DIR"
+ find "$TEMP_WHEEL_DIR" -name "_protoc_compiler*.so" -exec strip --strip-debug {} ";"
+ find "$TEMP_WHEEL_DIR" -name "cygrpc*.so" -exec strip --strip-debug {} ";"
+
+ WHEEL_FILE=$(basename "$WHEEL_PATH")
+ DISTRIBUTION_NAME=$(basename "$WHEEL_PATH" | cut -d '-' -f 1)
+ VERSION=$(basename "$WHEEL_PATH" | cut -d '-' -f 2)
+ wheel pack "$TEMP_WHEEL_DIR/$DISTRIBUTION_NAME-$VERSION" -d "$TEMP_WHEEL_DIR"
+ mv "$TEMP_WHEEL_DIR/$WHEEL_FILE" "$WHEEL_PATH"
+}
+
+for wheel in artifacts/*.whl; do
+ strip_binary_wheel "$wheel"
+done
+
# TODO: all the artifact builder configurations generate a grpcio-VERSION.tar.gz
# source distribution package, and only one of them will end up
# in the artifacts/ directory. They should be all equivalent though.
From 397bdd6b7bf96d0ecbb8642579df15c943b18fe0 Mon Sep 17 00:00:00 2001
From: Vijay Pai <vpai@google.com>
Date: Mon, 4 Mar 2019 13:55:12 -0800
Subject: [PATCH 016/138] Fix a bug that was exposed but unrelated...
---
test/cpp/end2end/test_service_impl.cc | 26 ++++++++++++++------------
1 file changed, 14 insertions(+), 12 deletions(-)
diff --git a/test/cpp/end2end/test_service_impl.cc b/test/cpp/end2end/test_service_impl.cc
index baebdbc8091..159ea33c2bc 100644
--- a/test/cpp/end2end/test_service_impl.cc
+++ b/test/cpp/end2end/test_service_impl.cc
@@ -125,6 +125,19 @@ void ServerTryCancelNonblocking(ServerContext* context) {
gpr_log(GPR_INFO, "Server called TryCancel() to cancel the request");
}
+void LoopUntilCancelled(Alarm* alarm, ServerContext* context,
+ experimental::ServerCallbackRpcController* controller) {
+ if (!context->IsCancelled()) {
+ alarm->experimental().Set(
+ gpr_time_add(gpr_now(GPR_CLOCK_REALTIME),
+ gpr_time_from_micros(1000, GPR_TIMESPAN)),
+ [alarm, context, controller](bool) {
+ LoopUntilCancelled(alarm, context, controller);
+ });
+ } else {
+ controller->Finish(Status::CANCELLED);
+ }
+}
} // namespace
Status TestServiceImpl::Echo(ServerContext* context, const EchoRequest* request,
@@ -290,18 +303,7 @@ void CallbackTestServiceImpl::EchoNonDelayed(
gpr_log(GPR_INFO, "Server called TryCancel() to cancel the request");
// Now wait until it's really canceled
- std::function<void(bool)> recurrence = [this, context, controller,
- &recurrence](bool) {
- if (!context->IsCancelled()) {
- alarm_.experimental().Set(
- gpr_time_add(gpr_now(GPR_CLOCK_REALTIME),
- gpr_time_from_micros(1000, GPR_TIMESPAN)),
- recurrence);
- } else {
- controller->Finish(Status::CANCELLED);
- }
- };
- recurrence(true);
+ LoopUntilCancelled(&alarm_, context, controller);
return;
}
From b606cad6ccfcc1785a181bcfcb0df603cb7d68f8 Mon Sep 17 00:00:00 2001
From: Karthik Ravi Shankar <karthikrs@google.com>
Date: Fri, 1 Mar 2019 16:14:24 -0800
Subject: [PATCH 017/138] Fold opencensus into grpc_impl namespace
Moving opencensus into grpc_impl namespace..
---
BUILD | 1 +
include/grpcpp/opencensus.h | 26 +-------
include/grpcpp/opencensus_impl.h | 51 +++++++++++++++
src/cpp/ext/filters/census/grpc_plugin.cc | 65 ++++++++++---------
src/cpp/ext/filters/census/grpc_plugin.h | 6 +-
src/cpp/ext/filters/census/views.cc | 31 +++++----
.../census/stats_plugin_end2end_test.cc | 2 +-
.../microbenchmarks/bm_opencensus_plugin.cc | 6 +-
8 files changed, 114 insertions(+), 74 deletions(-)
create mode 100644 include/grpcpp/opencensus_impl.h
diff --git a/BUILD b/BUILD
index 24c1fb31ced..f08a89fa5fc 100644
--- a/BUILD
+++ b/BUILD
@@ -2280,6 +2280,7 @@ grpc_cc_library(
],
hdrs = [
"include/grpcpp/opencensus.h",
+ "include/grpcpp/opencensus_impl.h",
"src/cpp/ext/filters/census/channel_filter.h",
"src/cpp/ext/filters/census/client_filter.h",
"src/cpp/ext/filters/census/context.h",
diff --git a/include/grpcpp/opencensus.h b/include/grpcpp/opencensus.h
index 29b221f7674..3b170336834 100644
--- a/include/grpcpp/opencensus.h
+++ b/include/grpcpp/opencensus.h
@@ -19,30 +19,6 @@
#ifndef GRPCPP_OPENCENSUS_H
#define GRPCPP_OPENCENSUS_H
-#include "opencensus/trace/span.h"
-
-namespace grpc {
-// These symbols in this file will not be included in the binary unless
-// grpc_opencensus_plugin build target was added as a dependency. At the moment
-// it is only setup to be built with Bazel.
-
-// Registers the OpenCensus plugin with gRPC, so that it will be used for future
-// RPCs. This must be called before any views are created.
-void RegisterOpenCensusPlugin();
-
-// RPC stats definitions, defined by
-// https://github.com/census-instrumentation/opencensus-specs/blob/master/stats/gRPC.md
-
-// Registers the cumulative gRPC views so that they will be exported by any
-// registered stats exporter. For on-task stats, construct a View using the
-// ViewDescriptors below.
-void RegisterOpenCensusViewsForExport();
-
-class ServerContext;
-
-// Returns the tracing Span for the current RPC.
-::opencensus::trace::Span GetSpanFromServerContext(ServerContext* context);
-
-} // namespace grpc
+#include "grpcpp/opencensus_impl.h"
#endif // GRPCPP_OPENCENSUS_H
diff --git a/include/grpcpp/opencensus_impl.h b/include/grpcpp/opencensus_impl.h
new file mode 100644
index 00000000000..631d2b861fd
--- /dev/null
+++ b/include/grpcpp/opencensus_impl.h
@@ -0,0 +1,51 @@
+/*
+ *
+ * Copyright 2019 gRPC authors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+#ifndef GRPCPP_OPENCENSUS_IMPL_H
+#define GRPCPP_OPENCENSUS_IMPL_H
+
+#include "opencensus/trace/span.h"
+
+namespace grpc {
+
+class ServerContext;
+}
+namespace grpc_impl {
+// These symbols in this file will not be included in the binary unless
+// grpc_opencensus_plugin build target was added as a dependency. At the moment
+// it is only setup to be built with Bazel.
+
+// Registers the OpenCensus plugin with gRPC, so that it will be used for future
+// RPCs. This must be called before any views are created.
+void RegisterOpenCensusPlugin();
+
+// RPC stats definitions, defined by
+// https://github.com/census-instrumentation/opencensus-specs/blob/master/stats/gRPC.md
+
+// Registers the cumulative gRPC views so that they will be exported by any
+// registered stats exporter. For on-task stats, construct a View using the
+// ViewDescriptors below.
+void RegisterOpenCensusViewsForExport();
+
+// Returns the tracing Span for the current RPC.
+::opencensus::trace::Span GetSpanFromServerContext(
+ grpc::ServerContext* context);
+
+} // namespace grpc_impl
+
+#endif // GRPCPP_OPENCENSUS_IMPL_H
diff --git a/src/cpp/ext/filters/census/grpc_plugin.cc b/src/cpp/ext/filters/census/grpc_plugin.cc
index f978ed3bf51..c5018f0673a 100644
--- a/src/cpp/ext/filters/census/grpc_plugin.cc
+++ b/src/cpp/ext/filters/census/grpc_plugin.cc
@@ -30,35 +30,6 @@
namespace grpc {
-void RegisterOpenCensusPlugin() {
- RegisterChannelFilter<CensusChannelData, CensusClientCallData>(
- "opencensus_client", GRPC_CLIENT_CHANNEL, INT_MAX /* priority */,
- nullptr /* condition function */);
- RegisterChannelFilter<CensusChannelData, CensusServerCallData>(
- "opencensus_server", GRPC_SERVER_CHANNEL, INT_MAX /* priority */,
- nullptr /* condition function */);
-
- // Access measures to ensure they are initialized. Otherwise, creating a view
- // before the first RPC would cause an error.
- RpcClientSentBytesPerRpc();
- RpcClientReceivedBytesPerRpc();
- RpcClientRoundtripLatency();
- RpcClientServerLatency();
- RpcClientSentMessagesPerRpc();
- RpcClientReceivedMessagesPerRpc();
-
- RpcServerSentBytesPerRpc();
- RpcServerReceivedBytesPerRpc();
- RpcServerServerLatency();
- RpcServerSentMessagesPerRpc();
- RpcServerReceivedMessagesPerRpc();
-}
-
-::opencensus::trace::Span GetSpanFromServerContext(ServerContext* context) {
- return reinterpret_cast<const CensusContext*>(context->census_context())
- ->Span();
-}
-
// These measure definitions should be kept in sync across opencensus
// implementations--see
// https://github.com/census-instrumentation/opencensus-java/blob/master/contrib/grpc_metrics/src/main/java/io/opencensus/contrib/grpc/metrics/RpcMeasureConstants.java.
@@ -126,5 +97,39 @@ ABSL_CONST_INIT const absl::string_view
ABSL_CONST_INIT const absl::string_view kRpcServerServerLatencyMeasureName =
"grpc.io/server/server_latency";
-
} // namespace grpc
+namespace grpc_impl {
+
+void RegisterOpenCensusPlugin() {
+ grpc::RegisterChannelFilter<grpc::CensusChannelData,
+ grpc::CensusClientCallData>(
+ "opencensus_client", GRPC_CLIENT_CHANNEL, INT_MAX /* priority */,
+ nullptr /* condition function */);
+ grpc::RegisterChannelFilter<grpc::CensusChannelData,
+ grpc::CensusServerCallData>(
+ "opencensus_server", GRPC_SERVER_CHANNEL, INT_MAX /* priority */,
+ nullptr /* condition function */);
+
+ // Access measures to ensure they are initialized. Otherwise, creating a view
+ // before the first RPC would cause an error.
+ grpc::RpcClientSentBytesPerRpc();
+ grpc::RpcClientReceivedBytesPerRpc();
+ grpc::RpcClientRoundtripLatency();
+ grpc::RpcClientServerLatency();
+ grpc::RpcClientSentMessagesPerRpc();
+ grpc::RpcClientReceivedMessagesPerRpc();
+
+ grpc::RpcServerSentBytesPerRpc();
+ grpc::RpcServerReceivedBytesPerRpc();
+ grpc::RpcServerServerLatency();
+ grpc::RpcServerSentMessagesPerRpc();
+ grpc::RpcServerReceivedMessagesPerRpc();
+}
+
+::opencensus::trace::Span GetSpanFromServerContext(
+ grpc::ServerContext* context) {
+ return reinterpret_cast<const grpc::CensusContext*>(context->census_context())
+ ->Span();
+}
+
+} // namespace grpc_impl
diff --git a/src/cpp/ext/filters/census/grpc_plugin.h b/src/cpp/ext/filters/census/grpc_plugin.h
index 9e319cb994e..209fad139ce 100644
--- a/src/cpp/ext/filters/census/grpc_plugin.h
+++ b/src/cpp/ext/filters/census/grpc_plugin.h
@@ -22,12 +22,14 @@
#include <grpc/support/port_platform.h>
#include "absl/strings/string_view.h"
-#include "include/grpcpp/opencensus.h"
+#include "include/grpcpp/opencensus_impl.h"
#include "opencensus/stats/stats.h"
-namespace grpc {
+namespace grpc_impl {
class ServerContext;
+}
+namespace grpc {
// The tag keys set when recording RPC stats.
::opencensus::stats::TagKey ClientMethodTagKey();
diff --git a/src/cpp/ext/filters/census/views.cc b/src/cpp/ext/filters/census/views.cc
index 2c0c5f72950..102745ab4c2 100644
--- a/src/cpp/ext/filters/census/views.cc
+++ b/src/cpp/ext/filters/census/views.cc
@@ -25,6 +25,23 @@
#include "opencensus/stats/internal/set_aggregation_window.h"
#include "opencensus/stats/stats.h"
+namespace grpc_impl {
+
+void RegisterOpenCensusViewsForExport() {
+ grpc::ClientSentMessagesPerRpcCumulative().RegisterForExport();
+ grpc::ClientSentBytesPerRpcCumulative().RegisterForExport();
+ grpc::ClientReceivedMessagesPerRpcCumulative().RegisterForExport();
+ grpc::ClientReceivedBytesPerRpcCumulative().RegisterForExport();
+ grpc::ClientRoundtripLatencyCumulative().RegisterForExport();
+ grpc::ClientServerLatencyCumulative().RegisterForExport();
+
+ grpc::ServerSentMessagesPerRpcCumulative().RegisterForExport();
+ grpc::ServerSentBytesPerRpcCumulative().RegisterForExport();
+ grpc::ServerReceivedMessagesPerRpcCumulative().RegisterForExport();
+ grpc::ServerReceivedBytesPerRpcCumulative().RegisterForExport();
+ grpc::ServerServerLatencyCumulative().RegisterForExport();
+}
+}
namespace grpc {
using ::opencensus::stats::Aggregation;
@@ -71,20 +88,6 @@ ViewDescriptor HourDescriptor() {
} // namespace
-void RegisterOpenCensusViewsForExport() {
- ClientSentMessagesPerRpcCumulative().RegisterForExport();
- ClientSentBytesPerRpcCumulative().RegisterForExport();
- ClientReceivedMessagesPerRpcCumulative().RegisterForExport();
- ClientReceivedBytesPerRpcCumulative().RegisterForExport();
- ClientRoundtripLatencyCumulative().RegisterForExport();
- ClientServerLatencyCumulative().RegisterForExport();
-
- ServerSentMessagesPerRpcCumulative().RegisterForExport();
- ServerSentBytesPerRpcCumulative().RegisterForExport();
- ServerReceivedMessagesPerRpcCumulative().RegisterForExport();
- ServerReceivedBytesPerRpcCumulative().RegisterForExport();
- ServerServerLatencyCumulative().RegisterForExport();
-}
// client cumulative
const ViewDescriptor& ClientSentBytesPerRpcCumulative() {
diff --git a/test/cpp/ext/filters/census/stats_plugin_end2end_test.cc b/test/cpp/ext/filters/census/stats_plugin_end2end_test.cc
index 73394028309..ad788a2dd68 100644
--- a/test/cpp/ext/filters/census/stats_plugin_end2end_test.cc
+++ b/test/cpp/ext/filters/census/stats_plugin_end2end_test.cc
@@ -58,7 +58,7 @@ class EchoServer final : public EchoTestService::Service {
class StatsPluginEnd2EndTest : public ::testing::Test {
protected:
- static void SetUpTestCase() { RegisterOpenCensusPlugin(); }
+ static void SetUpTestCase() { grpc_impl::RegisterOpenCensusPlugin(); }
void SetUp() {
// Set up a synchronous server on a different thread to avoid the asynch
diff --git a/test/cpp/microbenchmarks/bm_opencensus_plugin.cc b/test/cpp/microbenchmarks/bm_opencensus_plugin.cc
index 9d42eb891df..d23c4f0573f 100644
--- a/test/cpp/microbenchmarks/bm_opencensus_plugin.cc
+++ b/test/cpp/microbenchmarks/bm_opencensus_plugin.cc
@@ -29,7 +29,9 @@
#include "test/cpp/microbenchmarks/helpers.h"
absl::once_flag once;
-void RegisterOnce() { absl::call_once(once, grpc::RegisterOpenCensusPlugin); }
+void RegisterOnce() {
+ absl::call_once(once, grpc_impl::RegisterOpenCensusPlugin);
+}
class EchoServer final : public grpc::testing::EchoTestService::Service {
grpc::Status Echo(grpc::ServerContext* context,
@@ -99,7 +101,7 @@ static void BM_E2eLatencyCensusEnabled(benchmark::State& state) {
RegisterOnce();
// This we can safely repeat, and doing so clears accumulated data to avoid
// initialization costs varying between runs.
- grpc::RegisterOpenCensusViewsForExport();
+ grpc_impl::RegisterOpenCensusViewsForExport();
EchoServerThread server;
std::unique_ptr<grpc::testing::EchoTestService::Stub> stub =
From 168df1cb5fd49b9faaf12278bf029c87ad1b1ba8 Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Mon, 4 Mar 2019 16:01:45 -0800
Subject: [PATCH 018/138] Check hashes for pip wheels before installing
---
test/distrib/python/test_packages.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/test/distrib/python/test_packages.sh b/test/distrib/python/test_packages.sh
index 755daa10211..433148e6bd7 100755
--- a/test/distrib/python/test_packages.sh
+++ b/test/distrib/python/test_packages.sh
@@ -41,7 +41,7 @@ PYTHON=$VIRTUAL_ENV/bin/python
function at_least_one_installs() {
for file in "$@"; do
- if "$PYTHON" -m pip install "$file"; then
+ if "$PYTHON" -m pip install --require-hashes "$file"; then
return 0
fi
done
From 921df1c923858de38a4c282ec5236a57cb1ef1c1 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Thu, 28 Feb 2019 09:53:10 +0100
Subject: [PATCH 019/138] run_tests.py: increase timeout for pre-build step
---
tools/run_tests/run_tests.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py
index 986e7cd58c2..ed1c41e3256 100755
--- a/tools/run_tests/run_tests.py
+++ b/tools/run_tests/run_tests.py
@@ -106,6 +106,7 @@ def platform_string():
_DEFAULT_TIMEOUT_SECONDS = 5 * 60
+_PRE_BUILD_STEP_TIMEOUT_SECONDS = 10 * 60
def run_shell_command(cmd, env=None, cwd=None):
@@ -1634,7 +1635,10 @@ def build_step_environ(cfg):
build_steps = list(
set(
jobset.JobSpec(
- cmdline, environ=build_step_environ(build_config), flake_retries=2)
+ cmdline,
+ environ=build_step_environ(build_config),
+ timeout_seconds=_PRE_BUILD_STEP_TIMEOUT_SECONDS,
+ flake_retries=2)
for l in languages
for cmdline in l.pre_build_steps()))
if make_targets:
From f776bee37611d0dbdf73ac0e198d7024cb9716ee Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Tue, 5 Mar 2019 11:26:44 +0100
Subject: [PATCH 020/138] add retries for downloading interop matrix images
---
tools/interop_matrix/run_interop_matrix_tests.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tools/interop_matrix/run_interop_matrix_tests.py b/tools/interop_matrix/run_interop_matrix_tests.py
index de054e5d878..3f92c8e6641 100755
--- a/tools/interop_matrix/run_interop_matrix_tests.py
+++ b/tools/interop_matrix/run_interop_matrix_tests.py
@@ -224,7 +224,8 @@ def _pull_images_for_lang(lang, images):
cmdline=cmdline,
shortname='pull_image_%s' % (image),
timeout_seconds=_PULL_IMAGE_TIMEOUT_SECONDS,
- shell=True)
+ shell=True,
+ flake_retries=2)
download_specs.append(spec)
# too many image downloads at once tend to get stuck
max_pull_jobs = min(args.jobs, _MAX_PARALLEL_DOWNLOADS)
From 762676249ef1b316814ba8b1f6489206b36805d3 Mon Sep 17 00:00:00 2001
From: "Mark D. Roth" <roth@google.com>
Date: Tue, 5 Mar 2019 06:32:03 -0800
Subject: [PATCH 021/138] Make grpclb child policy configurable.
---
.../client_channel/lb_policy/grpclb/grpclb.cc | 344 +++++++++++++-----
test/cpp/end2end/grpclb_end2end_test.cc | 144 ++++++++
2 files changed, 402 insertions(+), 86 deletions(-)
diff --git a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
index c5d1ff22a9d..c1f2846f046 100644
--- a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
@@ -39,15 +39,14 @@
/// the balancer, we update the round_robin policy with the new list of
/// addresses. If we cannot communicate with the balancer on startup,
/// however, we may enter fallback mode, in which case we will populate
-/// the RR policy's addresses from the backend addresses returned by the
+/// the child policy's addresses from the backend addresses returned by the
/// resolver.
///
-/// Once an RR policy instance is in place (and getting updated as described),
+/// Once a child policy instance is in place (and getting updated as described),
/// calls for a pick, a ping, or a cancellation will be serviced right
-/// away by forwarding them to the RR instance. Any time there's no RR
-/// policy available (i.e., right after the creation of the gRPCLB policy),
-/// pick and ping requests are added to a list of pending picks and pings
-/// to be flushed and serviced when the RR policy instance becomes available.
+/// away by forwarding them to the child policy instance. Any time there's no
+/// child policy available (i.e., right after the creation of the gRPCLB
+/// policy), pick requests are queued.
///
/// \see https://github.com/grpc/grpc/blob/master/doc/load-balancing.md for the
/// high level design and details.
@@ -279,16 +278,23 @@ class GrpcLb : public LoadBalancingPolicy {
UniquePtr<SubchannelPicker> picker) override;
void RequestReresolution() override;
+ void set_child(LoadBalancingPolicy* child) { child_ = child; }
+
private:
+ bool CalledByPendingChild() const;
+ bool CalledByCurrentChild() const;
+
RefCountedPtr<GrpcLb> parent_;
+ LoadBalancingPolicy* child_ = nullptr;
};
~GrpcLb();
void ShutdownLocked() override;
- // Helper function used in UpdateLocked().
+ // Helper functions used in UpdateLocked().
void ProcessChannelArgsLocked(const grpc_channel_args& args);
+ void ParseLbConfig(Config* grpclb_config);
// Methods for dealing with the balancer channel and call.
void StartBalancerCallLocked();
@@ -296,10 +302,11 @@ class GrpcLb : public LoadBalancingPolicy {
void StartBalancerCallRetryTimerLocked();
static void OnBalancerCallRetryTimerLocked(void* arg, grpc_error* error);
- // Methods for dealing with the RR policy.
- grpc_channel_args* CreateRoundRobinPolicyArgsLocked();
- void CreateRoundRobinPolicyLocked(Args args);
- void CreateOrUpdateRoundRobinPolicyLocked();
+ // Methods for dealing with the child policy.
+ grpc_channel_args* CreateChildPolicyArgsLocked();
+ OrphanablePtr<LoadBalancingPolicy> CreateChildPolicyLocked(
+ const char* name, grpc_channel_args* args);
+ void CreateOrUpdateChildPolicyLocked();
// Who the client is trying to communicate with.
const char* server_name_ = nullptr;
@@ -345,8 +352,14 @@ class GrpcLb : public LoadBalancingPolicy {
grpc_timer lb_fallback_timer_;
grpc_closure lb_on_fallback_;
- // The RR policy to use for the backends.
- OrphanablePtr<LoadBalancingPolicy> rr_policy_;
+ // The child policy to use for the backends.
+ OrphanablePtr<LoadBalancingPolicy> child_policy_;
+ // When switching child policies, the new policy will be stored here
+ // until it reports READY, at which point it will be moved to child_policy_.
+ OrphanablePtr<LoadBalancingPolicy> pending_child_policy_;
+ // The child policy name and config.
+ UniquePtr<char> child_policy_name_;
+ RefCountedPtr<Config> child_policy_config_;
};
//
@@ -558,14 +571,30 @@ GrpcLb::Picker::PickResult GrpcLb::Picker::Pick(PickState* pick,
// GrpcLb::Helper
//
+bool GrpcLb::Helper::CalledByPendingChild() const {
+ GPR_ASSERT(child_ != nullptr);
+ return child_ == parent_->pending_child_policy_.get();
+}
+
+bool GrpcLb::Helper::CalledByCurrentChild() const {
+ GPR_ASSERT(child_ != nullptr);
+ return child_ == parent_->child_policy_.get();
+}
+
Subchannel* GrpcLb::Helper::CreateSubchannel(const grpc_channel_args& args) {
- if (parent_->shutting_down_) return nullptr;
+ if (parent_->shutting_down_ ||
+ (!CalledByPendingChild() && !CalledByCurrentChild())) {
+ return nullptr;
+ }
return parent_->channel_control_helper()->CreateSubchannel(args);
}
grpc_channel* GrpcLb::Helper::CreateChannel(const char* target,
const grpc_channel_args& args) {
- if (parent_->shutting_down_) return nullptr;
+ if (parent_->shutting_down_ ||
+ (!CalledByPendingChild() && !CalledByCurrentChild())) {
+ return nullptr;
+ }
return parent_->channel_control_helper()->CreateChannel(target, args);
}
@@ -576,31 +605,50 @@ void GrpcLb::Helper::UpdateState(grpc_connectivity_state state,
GRPC_ERROR_UNREF(state_error);
return;
}
+ // If this request is from the pending child policy, ignore it until
+ // it reports READY, at which point we swap it into place.
+ if (CalledByPendingChild()) {
+ if (grpc_lb_glb_trace.enabled()) {
+ gpr_log(GPR_INFO,
+ "[grpclb %p helper %p] pending child policy %p reports state=%s",
+ parent_.get(), this, parent_->pending_child_policy_.get(),
+ grpc_connectivity_state_name(state));
+ }
+ if (state != GRPC_CHANNEL_READY) {
+ GRPC_ERROR_UNREF(state_error);
+ return;
+ }
+ parent_->child_policy_ = std::move(parent_->pending_child_policy_);
+ } else if (!CalledByCurrentChild()) {
+ // This request is from an outdated child, so ignore it.
+ GRPC_ERROR_UNREF(state_error);
+ return;
+ }
// There are three cases to consider here:
// 1. We're in fallback mode. In this case, we're always going to use
- // RR's result, so we pass its picker through as-is.
+ // the child policy's result, so we pass its picker through as-is.
// 2. The serverlist contains only drop entries. In this case, we
// want to use our own picker so that we can return the drops.
// 3. Not in fallback mode and serverlist is not all drops (i.e., it
// may be empty or contain at least one backend address). There are
// two sub-cases:
- // a. RR is reporting state READY. In this case, we wrap RR's
- // picker in our own, so that we can handle drops and LB token
- // metadata for each pick.
- // b. RR is reporting a state other than READY. In this case, we
- // don't want to use our own picker, because we don't want to
- // process drops for picks that yield a QUEUE result; this would
+ // a. The child policy is reporting state READY. In this case, we wrap
+ // the child's picker in our own, so that we can handle drops and LB
+ // token metadata for each pick.
+ // b. The child policy is reporting a state other than READY. In this
+ // case, we don't want to use our own picker, because we don't want
+ // to process drops for picks that yield a QUEUE result; this would
// result in dropping too many calls, since we will see the
// queued picks multiple times, and we'd consider each one a
// separate call for the drop calculation.
//
- // Cases 1 and 3b: return picker from RR as-is.
+ // Cases 1 and 3b: return picker from the child policy as-is.
if (parent_->serverlist_ == nullptr ||
(!parent_->serverlist_->ContainsAllDropEntries() &&
state != GRPC_CHANNEL_READY)) {
if (grpc_lb_glb_trace.enabled()) {
gpr_log(GPR_INFO,
- "[grpclb %p helper %p] state=%s passing RR picker %p as-is",
+ "[grpclb %p helper %p] state=%s passing child picker %p as-is",
parent_.get(), this, grpc_connectivity_state_name(state),
picker.get());
}
@@ -608,9 +656,9 @@ void GrpcLb::Helper::UpdateState(grpc_connectivity_state state,
std::move(picker));
return;
}
- // Cases 2 and 3a: wrap picker from RR in our own picker.
+ // Cases 2 and 3a: wrap picker from the child in our own picker.
if (grpc_lb_glb_trace.enabled()) {
- gpr_log(GPR_INFO, "[grpclb %p helper %p] state=%s wrapping RR picker %p",
+ gpr_log(GPR_INFO, "[grpclb %p helper %p] state=%s wrapping child picker %p",
parent_.get(), this, grpc_connectivity_state_name(state),
picker.get());
}
@@ -628,15 +676,19 @@ void GrpcLb::Helper::UpdateState(grpc_connectivity_state state,
void GrpcLb::Helper::RequestReresolution() {
if (parent_->shutting_down_) return;
+ // If there is a pending child policy, ignore re-resolution requests
+ // from the current child policy (or any outdated pending child).
+ if (parent_->pending_child_policy_ != nullptr && !CalledByPendingChild()) {
+ return;
+ }
if (grpc_lb_glb_trace.enabled()) {
gpr_log(GPR_INFO,
- "[grpclb %p] Re-resolution requested from the internal RR policy "
- "(%p).",
- parent_.get(), parent_->rr_policy_.get());
+ "[grpclb %p] Re-resolution requested from child policy (%p).",
+ parent_.get(), child_);
}
// If we are talking to a balancer, we expect to get updated addresses
// from the balancer, so we can ignore the re-resolution request from
- // the RR policy. Otherwise, pass the re-resolution request up to the
+ // the child policy. Otherwise, pass the re-resolution request up to the
// channel.
if (parent_->lb_calld_ == nullptr ||
!parent_->lb_calld_->seen_initial_response()) {
@@ -984,7 +1036,7 @@ void GrpcLb::BalancerCallState::OnBalancerMessageReceivedLocked(
// instance will be destroyed either upon the next update or when the
// GrpcLb instance is destroyed.
grpclb_policy->serverlist_ = std::move(serverlist_wrapper);
- grpclb_policy->CreateOrUpdateRoundRobinPolicyLocked();
+ grpclb_policy->CreateOrUpdateChildPolicyLocked();
}
} else {
// No valid initial response or serverlist found.
@@ -1200,7 +1252,8 @@ void GrpcLb::ShutdownLocked() {
if (fallback_timer_callback_pending_) {
grpc_timer_cancel(&lb_fallback_timer_);
}
- rr_policy_.reset();
+ child_policy_.reset();
+ pending_child_policy_.reset();
// We destroy the LB channel here instead of in our destructor because
// destroying the channel triggers a last callback to
// OnBalancerChannelConnectivityChangedLocked(), and we need to be
@@ -1220,17 +1273,24 @@ void GrpcLb::ResetBackoffLocked() {
if (lb_channel_ != nullptr) {
grpc_channel_reset_connect_backoff(lb_channel_);
}
- if (rr_policy_ != nullptr) {
- rr_policy_->ResetBackoffLocked();
+ if (child_policy_ != nullptr) {
+ child_policy_->ResetBackoffLocked();
+ }
+ if (pending_child_policy_ != nullptr) {
+ pending_child_policy_->ResetBackoffLocked();
}
}
void GrpcLb::FillChildRefsForChannelz(
channelz::ChildRefsList* child_subchannels,
channelz::ChildRefsList* child_channels) {
- // delegate to the RoundRobin to fill the children subchannels.
- if (rr_policy_ != nullptr) {
- rr_policy_->FillChildRefsForChannelz(child_subchannels, child_channels);
+ // delegate to the child policy to fill the children subchannels.
+ if (child_policy_ != nullptr) {
+ child_policy_->FillChildRefsForChannelz(child_subchannels, child_channels);
+ }
+ if (pending_child_policy_ != nullptr) {
+ pending_child_policy_->FillChildRefsForChannelz(child_subchannels,
+ child_channels);
}
gpr_atm uuid = gpr_atm_no_barrier_load(&lb_channel_uuid_);
if (uuid != 0) {
@@ -1238,6 +1298,32 @@ void GrpcLb::FillChildRefsForChannelz(
}
}
+void GrpcLb::UpdateLocked(const grpc_channel_args& args,
+ RefCountedPtr<Config> lb_config) {
+ const bool is_initial_update = lb_channel_ == nullptr;
+ ParseLbConfig(lb_config.get());
+ ProcessChannelArgsLocked(args);
+ // Update the existing child policy.
+ if (child_policy_ != nullptr) CreateOrUpdateChildPolicyLocked();
+ // If this is the initial update, start the fallback timer.
+ if (is_initial_update) {
+ if (lb_fallback_timeout_ms_ > 0 && serverlist_ == nullptr &&
+ !fallback_timer_callback_pending_) {
+ grpc_millis deadline = ExecCtx::Get()->Now() + lb_fallback_timeout_ms_;
+ Ref(DEBUG_LOCATION, "on_fallback_timer").release(); // Ref for callback
+ GRPC_CLOSURE_INIT(&lb_on_fallback_, &GrpcLb::OnFallbackTimerLocked, this,
+ grpc_combiner_scheduler(combiner()));
+ fallback_timer_callback_pending_ = true;
+ grpc_timer_init(&lb_fallback_timer_, deadline, &lb_on_fallback_);
+ }
+ StartBalancerCallLocked();
+ }
+}
+
+//
+// helpers for UpdateLocked()
+//
+
// Returns the backend addresses extracted from the given addresses.
UniquePtr<ServerAddressList> ExtractBackendAddresses(
const ServerAddressList& addresses) {
@@ -1299,25 +1385,26 @@ void GrpcLb::ProcessChannelArgsLocked(const grpc_channel_args& args) {
grpc_channel_args_destroy(lb_channel_args);
}
-void GrpcLb::UpdateLocked(const grpc_channel_args& args,
- RefCountedPtr<Config> lb_config) {
- const bool is_initial_update = lb_channel_ == nullptr;
- ProcessChannelArgsLocked(args);
- // Update the existing RR policy.
- if (rr_policy_ != nullptr) CreateOrUpdateRoundRobinPolicyLocked();
- // If this is the initial update, start the fallback timer and the
- // balancer call.
- if (is_initial_update) {
- if (lb_fallback_timeout_ms_ > 0 && serverlist_ == nullptr &&
- !fallback_timer_callback_pending_) {
- grpc_millis deadline = ExecCtx::Get()->Now() + lb_fallback_timeout_ms_;
- Ref(DEBUG_LOCATION, "on_fallback_timer").release(); // Ref for callback
- GRPC_CLOSURE_INIT(&lb_on_fallback_, &GrpcLb::OnFallbackTimerLocked, this,
- grpc_combiner_scheduler(combiner()));
- fallback_timer_callback_pending_ = true;
- grpc_timer_init(&lb_fallback_timer_, deadline, &lb_on_fallback_);
+void GrpcLb::ParseLbConfig(Config* grpclb_config) {
+ const grpc_json* child_policy = nullptr;
+ if (grpclb_config != nullptr) {
+ const grpc_json* grpclb_config_json = grpclb_config->json();
+ for (const grpc_json* field = grpclb_config_json; field != nullptr;
+ field = field->next) {
+ if (field->key == nullptr) return;
+ if (strcmp(field->key, "childPolicy") == 0) {
+ if (child_policy != nullptr) return; // Duplicate.
+ child_policy = ParseLoadBalancingConfig(field);
+ }
}
- StartBalancerCallLocked();
+ }
+ if (child_policy != nullptr) {
+ child_policy_name_ = UniquePtr<char>(gpr_strdup(child_policy->key));
+ child_policy_config_ = MakeRefCounted<Config>(
+ child_policy->child, grpclb_config->service_config());
+ } else {
+ child_policy_name_.reset();
+ child_policy_config_.reset();
}
}
@@ -1352,7 +1439,7 @@ void GrpcLb::OnFallbackTimerLocked(void* arg, grpc_error* error) {
grpclb_policy);
}
GPR_ASSERT(grpclb_policy->fallback_backend_addresses_ != nullptr);
- grpclb_policy->CreateOrUpdateRoundRobinPolicyLocked();
+ grpclb_policy->CreateOrUpdateChildPolicyLocked();
}
grpclb_policy->Unref(DEBUG_LOCATION, "on_fallback_timer");
}
@@ -1396,10 +1483,10 @@ void GrpcLb::OnBalancerCallRetryTimerLocked(void* arg, grpc_error* error) {
}
//
-// code for interacting with the RR policy
+// code for interacting with the child policy
//
-grpc_channel_args* GrpcLb::CreateRoundRobinPolicyArgsLocked() {
+grpc_channel_args* GrpcLb::CreateChildPolicyArgsLocked() {
ServerAddressList tmp_addresses;
ServerAddressList* addresses = &tmp_addresses;
bool is_backend_from_grpclb_load_balancer = false;
@@ -1408,7 +1495,7 @@ grpc_channel_args* GrpcLb::CreateRoundRobinPolicyArgsLocked() {
lb_calld_ == nullptr ? nullptr : lb_calld_->client_stats());
is_backend_from_grpclb_load_balancer = true;
} else {
- // If CreateOrUpdateRoundRobinPolicyLocked() is invoked when we haven't
+ // If CreateOrUpdateChildPolicyLocked() is invoked when we haven't
// received any serverlist from the balancer, we use the fallback backends
// returned by the resolver. Note that the fallback backend list may be
// empty, in which case the new round_robin policy will keep the requested
@@ -1435,49 +1522,134 @@ grpc_channel_args* GrpcLb::CreateRoundRobinPolicyArgsLocked() {
const_cast<char*>(GRPC_ARG_INHIBIT_HEALTH_CHECKING), 1);
++num_args_to_add;
}
- grpc_channel_args* args = grpc_channel_args_copy_and_add_and_remove(
+ return grpc_channel_args_copy_and_add_and_remove(
args_, keys_to_remove, GPR_ARRAY_SIZE(keys_to_remove), args_to_add,
num_args_to_add);
- return args;
}
-void GrpcLb::CreateRoundRobinPolicyLocked(Args args) {
- GPR_ASSERT(rr_policy_ == nullptr);
- rr_policy_ = LoadBalancingPolicyRegistry::CreateLoadBalancingPolicy(
- "round_robin", std::move(args));
- if (GPR_UNLIKELY(rr_policy_ == nullptr)) {
- gpr_log(GPR_ERROR, "[grpclb %p] Failure creating a RoundRobin policy",
- this);
- return;
+OrphanablePtr<LoadBalancingPolicy> GrpcLb::CreateChildPolicyLocked(
+ const char* name, grpc_channel_args* args) {
+ Helper* helper = New<Helper>(Ref());
+ LoadBalancingPolicy::Args lb_policy_args;
+ lb_policy_args.combiner = combiner();
+ lb_policy_args.args = args;
+ lb_policy_args.channel_control_helper =
+ UniquePtr<ChannelControlHelper>(helper);
+ OrphanablePtr<LoadBalancingPolicy> lb_policy =
+ LoadBalancingPolicyRegistry::CreateLoadBalancingPolicy(
+ name, std::move(lb_policy_args));
+ if (GPR_UNLIKELY(lb_policy == nullptr)) {
+ gpr_log(GPR_ERROR, "[grpclb %p] Failure creating child policy %s", this,
+ name);
+ return nullptr;
}
+ helper->set_child(lb_policy.get());
if (grpc_lb_glb_trace.enabled()) {
- gpr_log(GPR_INFO, "[grpclb %p] Created new RR policy %p", this,
- rr_policy_.get());
+ gpr_log(GPR_INFO, "[grpclb %p] Created new child policy %s (%p)", this,
+ name, lb_policy.get());
}
// Add the gRPC LB's interested_parties pollset_set to that of the newly
- // created RR policy. This will make the RR policy progress upon activity on
- // gRPC LB, which in turn is tied to the application's call.
- grpc_pollset_set_add_pollset_set(rr_policy_->interested_parties(),
+ // created child policy. This will make the child policy progress upon
+ // activity on gRPC LB, which in turn is tied to the application's call.
+ grpc_pollset_set_add_pollset_set(lb_policy->interested_parties(),
interested_parties());
+ return lb_policy;
}
-void GrpcLb::CreateOrUpdateRoundRobinPolicyLocked() {
+void GrpcLb::CreateOrUpdateChildPolicyLocked() {
if (shutting_down_) return;
- grpc_channel_args* args = CreateRoundRobinPolicyArgsLocked();
+ grpc_channel_args* args = CreateChildPolicyArgsLocked();
GPR_ASSERT(args != nullptr);
- if (rr_policy_ == nullptr) {
- LoadBalancingPolicy::Args lb_policy_args;
- lb_policy_args.combiner = combiner();
- lb_policy_args.args = args;
- lb_policy_args.channel_control_helper =
- UniquePtr<ChannelControlHelper>(New<Helper>(Ref()));
- CreateRoundRobinPolicyLocked(std::move(lb_policy_args));
+ // If the child policy name changes, we need to create a new child
+ // policy. When this happens, we leave child_policy_ as-is and store
+ // the new child policy in pending_child_policy_. Once the new child
+ // policy transitions into state READY, we swap it into child_policy_,
+ // replacing the original child policy. So pending_child_policy_ is
+ // non-null only between when we apply an update that changes the child
+ // policy name and when the new child reports state READY.
+ //
+ // Updates can arrive at any point during this transition. We always
+ // apply updates relative to the most recently created child policy,
+ // even if the most recent one is still in pending_child_policy_. This
+ // is true both when applying the updates to an existing child policy
+ // and when determining whether we need to create a new policy.
+ //
+ // As a result of this, there are several cases to consider here:
+ //
+ // 1. We have no existing child policy (i.e., we have started up but
+ // have not yet received a serverlist from the balancer or gone
+ // into fallback mode; in this case, both child_policy_ and
+ // pending_child_policy_ are null). In this case, we create a
+ // new child policy and store it in child_policy_.
+ //
+ // 2. We have an existing child policy and have no pending child policy
+ // from a previous update (i.e., either there has not been a
+ // previous update that changed the policy name, or we have already
+ // finished swapping in the new policy; in this case, child_policy_
+ // is non-null but pending_child_policy_ is null). In this case:
+ // a. If child_policy_->name() equals child_policy_name, then we
+ // update the existing child policy.
+ // b. If child_policy_->name() does not equal child_policy_name,
+ // we create a new policy. The policy will be stored in
+ // pending_child_policy_ and will later be swapped into
+ // child_policy_ by the helper when the new child transitions
+ // into state READY.
+ //
+ // 3. We have an existing child policy and have a pending child policy
+ // from a previous update (i.e., a previous update set
+ // pending_child_policy_ as per case 2b above and that policy has
+ // not yet transitioned into state READY and been swapped into
+ // child_policy_; in this case, both child_policy_ and
+ // pending_child_policy_ are non-null). In this case:
+ // a. If pending_child_policy_->name() equals child_policy_name,
+ // then we update the existing pending child policy.
+ // b. If pending_child_policy->name() does not equal
+ // child_policy_name, then we create a new policy. The new
+ // policy is stored in pending_child_policy_ (replacing the one
+ // that was there before, which will be immediately shut down)
+ // and will later be swapped into child_policy_ by the helper
+ // when the new child transitions into state READY.
+ const char* child_policy_name =
+ child_policy_name_ == nullptr ? "round_robin" : child_policy_name_.get();
+ const bool create_policy =
+ // case 1
+ child_policy_ == nullptr ||
+ // case 2b
+ (pending_child_policy_ == nullptr &&
+ strcmp(child_policy_->name(), child_policy_name) != 0) ||
+ // case 3b
+ (pending_child_policy_ != nullptr &&
+ strcmp(pending_child_policy_->name(), child_policy_name) != 0);
+ LoadBalancingPolicy* policy_to_update = nullptr;
+ if (create_policy) {
+ // Cases 1, 2b, and 3b: create a new child policy.
+ // If child_policy_ is null, we set it (case 1), else we set
+ // pending_child_policy_ (cases 2b and 3b).
+ auto& lb_policy =
+ child_policy_ == nullptr ? child_policy_ : pending_child_policy_;
+ if (grpc_lb_glb_trace.enabled()) {
+ gpr_log(GPR_INFO, "[grpclb %p] Creating new %schild policy %s", this,
+ child_policy_ == nullptr ? "" : "pending ", child_policy_name);
+ }
+ lb_policy = CreateChildPolicyLocked(child_policy_name, args);
+ policy_to_update = lb_policy.get();
+ } else {
+ // Cases 2a and 3a: update an existing policy.
+ // If we have a pending child policy, send the update to the pending
+ // policy (case 3a), else send it to the current policy (case 2a).
+ policy_to_update = pending_child_policy_ != nullptr
+ ? pending_child_policy_.get()
+ : child_policy_.get();
}
+ GPR_ASSERT(policy_to_update != nullptr);
+ // Update the policy.
if (grpc_lb_glb_trace.enabled()) {
- gpr_log(GPR_INFO, "[grpclb %p] Updating RR policy %p", this,
- rr_policy_.get());
+ gpr_log(GPR_INFO, "[grpclb %p] Updating %schild policy %p", this,
+ policy_to_update == pending_child_policy_.get() ? "pending " : "",
+ policy_to_update);
}
- rr_policy_->UpdateLocked(*args, nullptr);
+ policy_to_update->UpdateLocked(*args, child_policy_config_);
+ // Clean up.
grpc_channel_args_destroy(args);
}
diff --git a/test/cpp/end2end/grpclb_end2end_test.cc b/test/cpp/end2end/grpclb_end2end_test.cc
index 2288b88b517..31353ba1304 100644
--- a/test/cpp/end2end/grpclb_end2end_test.cc
+++ b/test/cpp/end2end/grpclb_end2end_test.cc
@@ -723,6 +723,150 @@ TEST_F(SingleBalancerTest, SelectGrpclbWithMigrationServiceConfig) {
EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
}
+TEST_F(SingleBalancerTest, UsePickFirstChildPolicy) {
+ SetNextResolutionAllBalancers(
+ "{\n"
+ " \"loadBalancingConfig\":[\n"
+ " { \"grpclb\":{\n"
+ " \"childPolicy\":[\n"
+ " { \"pick_first\":{} }\n"
+ " ]\n"
+ " } }\n"
+ " ]\n"
+ "}");
+ ScheduleResponseForBalancer(
+ 0, BalancerServiceImpl::BuildResponseForBackends(GetBackendPorts(), {}),
+ 0);
+ const size_t kNumRpcs = num_backends_ * 2;
+ CheckRpcSendOk(kNumRpcs, 1000 /* timeout_ms */, true /* wait_for_ready */);
+ balancers_[0]->NotifyDoneWithServerlists();
+ // Check that all requests went to the first backend. This verifies
+ // that we used pick_first instead of round_robin as the child policy.
+ EXPECT_EQ(backend_servers_[0].service_->request_count(), kNumRpcs);
+ for (size_t i = 1; i < backends_.size(); ++i) {
+ EXPECT_EQ(backend_servers_[i].service_->request_count(), 0UL);
+ }
+ // The balancer got a single request.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ // and sent a single response.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ // Check LB policy name for the channel.
+ EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
+}
+
+TEST_F(SingleBalancerTest, SwapChildPolicy) {
+ SetNextResolutionAllBalancers(
+ "{\n"
+ " \"loadBalancingConfig\":[\n"
+ " { \"grpclb\":{\n"
+ " \"childPolicy\":[\n"
+ " { \"pick_first\":{} }\n"
+ " ]\n"
+ " } }\n"
+ " ]\n"
+ "}");
+ ScheduleResponseForBalancer(
+ 0, BalancerServiceImpl::BuildResponseForBackends(GetBackendPorts(), {}),
+ 0);
+ const size_t kNumRpcs = num_backends_ * 2;
+ CheckRpcSendOk(kNumRpcs, 1000 /* timeout_ms */, true /* wait_for_ready */);
+ // Check that all requests went to the first backend. This verifies
+ // that we used pick_first instead of round_robin as the child policy.
+ EXPECT_EQ(backend_servers_[0].service_->request_count(), kNumRpcs);
+ for (size_t i = 1; i < backends_.size(); ++i) {
+ EXPECT_EQ(backend_servers_[i].service_->request_count(), 0UL);
+ }
+ // Send new resolution that removes child policy from service config.
+ SetNextResolutionAllBalancers("{}");
+ WaitForAllBackends();
+ CheckRpcSendOk(kNumRpcs, 1000 /* timeout_ms */, true /* wait_for_ready */);
+ // Check that every backend saw the same number of requests. This verifies
+ // that we used round_robin.
+ for (size_t i = 0; i < backends_.size(); ++i) {
+ EXPECT_EQ(backend_servers_[i].service_->request_count(), 2UL);
+ }
+ // Done.
+ balancers_[0]->NotifyDoneWithServerlists();
+ // The balancer got a single request.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ // and sent a single response.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ // Check LB policy name for the channel.
+ EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
+}
+
+TEST_F(SingleBalancerTest, UpdatesGoToMostRecentChildPolicy) {
+ const int kFallbackTimeoutMs = 200 * grpc_test_slowdown_factor();
+ ResetStub(kFallbackTimeoutMs);
+ int unreachable_balancer_port = grpc_pick_unused_port_or_die();
+ int unreachable_backend_port = grpc_pick_unused_port_or_die();
+ // Phase 1: Start with RR pointing to first backend.
+ gpr_log(GPR_INFO, "PHASE 1: Initial setup with RR with first backend");
+ SetNextResolution(
+ {
+ // Unreachable balancer.
+ {unreachable_balancer_port, true, ""},
+ // Fallback address: first backend.
+ {backend_servers_[0].port_, false, ""},
+ },
+ "{\n"
+ " \"loadBalancingConfig\":[\n"
+ " { \"grpclb\":{\n"
+ " \"childPolicy\":[\n"
+ " { \"round_robin\":{} }\n"
+ " ]\n"
+ " } }\n"
+ " ]\n"
+ "}");
+ // RPCs should go to first backend.
+ WaitForBackend(0);
+ // Phase 2: Switch to PF pointing to unreachable backend.
+ gpr_log(GPR_INFO, "PHASE 2: Update to use PF with unreachable backend");
+ SetNextResolution(
+ {
+ // Unreachable balancer.
+ {unreachable_balancer_port, true, ""},
+ // Fallback address: unreachable backend.
+ {unreachable_backend_port, false, ""},
+ },
+ "{\n"
+ " \"loadBalancingConfig\":[\n"
+ " { \"grpclb\":{\n"
+ " \"childPolicy\":[\n"
+ " { \"pick_first\":{} }\n"
+ " ]\n"
+ " } }\n"
+ " ]\n"
+ "}");
+ // RPCs should continue to go to the first backend, because the new
+ // PF child policy will never go into state READY.
+ WaitForBackend(0);
+ // Phase 3: Switch back to RR pointing to second and third backends.
+ // This ensures that we create a new policy rather than updating the
+ // pending PF policy.
+ gpr_log(GPR_INFO, "PHASE 3: Update to use RR again with two backends");
+ SetNextResolution(
+ {
+ // Unreachable balancer.
+ {unreachable_balancer_port, true, ""},
+ // Fallback address: second and third backends.
+ {backend_servers_[1].port_, false, ""},
+ {backend_servers_[2].port_, false, ""},
+ },
+ "{\n"
+ " \"loadBalancingConfig\":[\n"
+ " { \"grpclb\":{\n"
+ " \"childPolicy\":[\n"
+ " { \"round_robin\":{} }\n"
+ " ]\n"
+ " } }\n"
+ " ]\n"
+ "}");
+ // RPCs should go to the second and third backends.
+ WaitForBackend(1);
+ WaitForBackend(2);
+}
+
TEST_F(SingleBalancerTest, SameBackendListedMultipleTimes) {
SetNextResolutionAllBalancers();
// Same backend listed twice.
From 840e7a2861d5221074dca0e745afa0f8fbc820b0 Mon Sep 17 00:00:00 2001
From: "Mark D. Roth" <roth@google.com>
Date: Tue, 5 Mar 2019 07:28:32 -0800
Subject: [PATCH 022/138] Store LB policy name in Config object.
---
.../ext/filters/client_channel/lb_policy.h | 3 ++-
.../client_channel/lb_policy/grpclb/grpclb.cc | 16 ++++++--------
.../client_channel/lb_policy/xds/xds.cc | 22 ++++++++-----------
.../client_channel/resolver_result_parsing.cc | 4 ++--
4 files changed, 20 insertions(+), 25 deletions(-)
diff --git a/src/core/ext/filters/client_channel/lb_policy.h b/src/core/ext/filters/client_channel/lb_policy.h
index 7a876966524..ee5d860b378 100644
--- a/src/core/ext/filters/client_channel/lb_policy.h
+++ b/src/core/ext/filters/client_channel/lb_policy.h
@@ -210,7 +210,8 @@ class LoadBalancingPolicy : public InternallyRefCounted<LoadBalancingPolicy> {
RefCountedPtr<ServiceConfig> service_config)
: json_(lb_config), service_config_(std::move(service_config)) {}
- const grpc_json* json() const { return json_; }
+ const char* name() const { return json_->key; }
+ const grpc_json* config() const { return json_->child; }
RefCountedPtr<ServiceConfig> service_config() const {
return service_config_;
}
diff --git a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
index c1f2846f046..d6c8a60fbfe 100644
--- a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
@@ -357,8 +357,7 @@ class GrpcLb : public LoadBalancingPolicy {
// When switching child policies, the new policy will be stored here
// until it reports READY, at which point it will be moved to child_policy_.
OrphanablePtr<LoadBalancingPolicy> pending_child_policy_;
- // The child policy name and config.
- UniquePtr<char> child_policy_name_;
+ // The child policy config.
RefCountedPtr<Config> child_policy_config_;
};
@@ -1388,7 +1387,7 @@ void GrpcLb::ProcessChannelArgsLocked(const grpc_channel_args& args) {
void GrpcLb::ParseLbConfig(Config* grpclb_config) {
const grpc_json* child_policy = nullptr;
if (grpclb_config != nullptr) {
- const grpc_json* grpclb_config_json = grpclb_config->json();
+ const grpc_json* grpclb_config_json = grpclb_config->config();
for (const grpc_json* field = grpclb_config_json; field != nullptr;
field = field->next) {
if (field->key == nullptr) return;
@@ -1399,11 +1398,9 @@ void GrpcLb::ParseLbConfig(Config* grpclb_config) {
}
}
if (child_policy != nullptr) {
- child_policy_name_ = UniquePtr<char>(gpr_strdup(child_policy->key));
- child_policy_config_ = MakeRefCounted<Config>(
- child_policy->child, grpclb_config->service_config());
+ child_policy_config_ =
+ MakeRefCounted<Config>(child_policy, grpclb_config->service_config());
} else {
- child_policy_name_.reset();
child_policy_config_.reset();
}
}
@@ -1609,8 +1606,9 @@ void GrpcLb::CreateOrUpdateChildPolicyLocked() {
// that was there before, which will be immediately shut down)
// and will later be swapped into child_policy_ by the helper
// when the new child transitions into state READY.
- const char* child_policy_name =
- child_policy_name_ == nullptr ? "round_robin" : child_policy_name_.get();
+ const char* child_policy_name = child_policy_config_ == nullptr
+ ? "round_robin"
+ : child_policy_config_->name();
const bool create_policy =
// case 1
child_policy_ == nullptr ||
diff --git a/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc b/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc
index 6c10d876af7..e6ce4fef782 100644
--- a/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc
@@ -302,7 +302,6 @@ class XdsLb : public LoadBalancingPolicy {
// Timeout in milliseconds for before using fallback backend addresses.
// 0 means not using fallback.
- UniquePtr<char> fallback_policy_name_;
RefCountedPtr<Config> fallback_policy_config_;
int lb_fallback_timeout_ms_ = 0;
// The backend addresses from the resolver.
@@ -313,7 +312,6 @@ class XdsLb : public LoadBalancingPolicy {
grpc_closure lb_on_fallback_;
// The policy to use for the backends.
- UniquePtr<char> child_policy_name_;
RefCountedPtr<Config> child_policy_config_;
OrphanablePtr<LoadBalancingPolicy> child_policy_;
};
@@ -1079,7 +1077,7 @@ void XdsLb::ProcessChannelArgsLocked(const grpc_channel_args& args) {
}
void XdsLb::ParseLbConfig(Config* xds_config) {
- const grpc_json* xds_config_json = xds_config->json();
+ const grpc_json* xds_config_json = xds_config->config();
const char* balancer_name = nullptr;
grpc_json* child_policy = nullptr;
grpc_json* fallback_policy = nullptr;
@@ -1099,17 +1097,15 @@ void XdsLb::ParseLbConfig(Config* xds_config) {
}
}
if (balancer_name == nullptr) return; // Required field.
+ balancer_name_ = UniquePtr<char>(gpr_strdup(balancer_name));
if (child_policy != nullptr) {
- child_policy_name_ = UniquePtr<char>(gpr_strdup(child_policy->key));
- child_policy_config_ = MakeRefCounted<Config>(child_policy->child,
- xds_config->service_config());
+ child_policy_config_ =
+ MakeRefCounted<Config>(child_policy, xds_config->service_config());
}
if (fallback_policy != nullptr) {
- fallback_policy_name_ = UniquePtr<char>(gpr_strdup(fallback_policy->key));
- fallback_policy_config_ = MakeRefCounted<Config>(
- fallback_policy->child, xds_config->service_config());
+ fallback_policy_config_ =
+ MakeRefCounted<Config>(fallback_policy, xds_config->service_config());
}
- balancer_name_ = UniquePtr<char>(gpr_strdup(balancer_name));
}
void XdsLb::UpdateLocked(const grpc_channel_args& args,
@@ -1328,16 +1324,16 @@ void XdsLb::CreateOrUpdateChildPolicyLocked() {
GPR_ASSERT(args != nullptr);
// TODO(juanlishen): If the child policy is not configured via service config,
// use whatever algorithm is specified by the balancer.
- // TODO(juanlishen): Switch policy according to child_policy_config->key.
+ // TODO(juanlishen): Switch policy according to child_policy_config_->name().
if (child_policy_ == nullptr) {
LoadBalancingPolicy::Args lb_policy_args;
lb_policy_args.combiner = combiner();
lb_policy_args.args = args;
lb_policy_args.channel_control_helper =
UniquePtr<ChannelControlHelper>(New<Helper>(Ref()));
- CreateChildPolicyLocked(child_policy_name_ == nullptr
+ CreateChildPolicyLocked(child_policy_config_ == nullptr
? "round_robin"
- : child_policy_name_.get(),
+ : child_policy_config_->name(),
std::move(lb_policy_args));
if (grpc_lb_xds_trace.enabled()) {
gpr_log(GPR_INFO, "[xdslb %p] Created a new child policy %p", this,
diff --git a/src/core/ext/filters/client_channel/resolver_result_parsing.cc b/src/core/ext/filters/client_channel/resolver_result_parsing.cc
index aaa63178793..ad23c735b51 100644
--- a/src/core/ext/filters/client_channel/resolver_result_parsing.cc
+++ b/src/core/ext/filters/client_channel/resolver_result_parsing.cc
@@ -148,8 +148,8 @@ void ProcessedResolverResult::ParseLbConfigFromServiceConfig(
LoadBalancingPolicy::ParseLoadBalancingConfig(field);
if (policy != nullptr) {
lb_policy_name_.reset(gpr_strdup(policy->key));
- lb_policy_config_ = MakeRefCounted<LoadBalancingPolicy::Config>(
- policy->child, service_config_);
+ lb_policy_config_ =
+ MakeRefCounted<LoadBalancingPolicy::Config>(policy, service_config_);
}
}
From c060d55cc7b9dae35cc9cec50756405e69e9b1f3 Mon Sep 17 00:00:00 2001
From: Soheil Hassas Yeganeh <soheil@google.com>
Date: Tue, 5 Mar 2019 14:01:01 -0500
Subject: [PATCH 023/138] Revert "Introduce grpc_byte_buffer_reader_peek and
use it for Protobuf parsing."
---
grpc.def | 1 -
include/grpc/impl/codegen/byte_buffer.h | 13 ----
include/grpcpp/impl/codegen/core_codegen.h | 2 -
.../impl/codegen/core_codegen_interface.h | 2 -
.../grpcpp/impl/codegen/proto_buffer_reader.h | 18 ++---
src/core/lib/surface/byte_buffer_reader.cc | 17 -----
src/cpp/common/core_codegen.cc | 5 --
src/ruby/ext/grpc/rb_grpc_imports.generated.c | 2 -
src/ruby/ext/grpc/rb_grpc_imports.generated.h | 3 -
test/core/surface/byte_buffer_reader_test.cc | 70 ------------------
.../core/surface/public_headers_must_be_c89.c | 1 -
test/cpp/microbenchmarks/bm_byte_buffer.cc | 71 +------------------
12 files changed, 11 insertions(+), 194 deletions(-)
diff --git a/grpc.def b/grpc.def
index 922f95383a3..e0a08d22c19 100644
--- a/grpc.def
+++ b/grpc.def
@@ -149,7 +149,6 @@ EXPORTS
grpc_byte_buffer_reader_init
grpc_byte_buffer_reader_destroy
grpc_byte_buffer_reader_next
- grpc_byte_buffer_reader_peek
grpc_byte_buffer_reader_readall
grpc_raw_byte_buffer_from_reader
gpr_log_severity_string
diff --git a/include/grpc/impl/codegen/byte_buffer.h b/include/grpc/impl/codegen/byte_buffer.h
index 12479068155..774655ed66f 100644
--- a/include/grpc/impl/codegen/byte_buffer.h
+++ b/include/grpc/impl/codegen/byte_buffer.h
@@ -73,19 +73,6 @@ GRPCAPI void grpc_byte_buffer_reader_destroy(grpc_byte_buffer_reader* reader);
GRPCAPI int grpc_byte_buffer_reader_next(grpc_byte_buffer_reader* reader,
grpc_slice* slice);
-/** EXPERIMENTAL API - This function may be removed and changed, in the future.
- *
- * Updates \a slice with the next piece of data from from \a reader and returns
- * 1. Returns 0 at the end of the stream. Caller is responsible for making sure
- * the slice pointer remains valid when accessed.
- *
- * NOTE: Do not use this function unless the caller can guarantee that the
- * underlying grpc_byte_buffer outlasts the use of the slice. This is only
- * safe when the underlying grpc_byte_buffer remains immutable while slice
- * is being accessed. */
-GRPCAPI int grpc_byte_buffer_reader_peek(grpc_byte_buffer_reader* reader,
- grpc_slice** slice);
-
/** Merge all data from \a reader into single slice */
GRPCAPI grpc_slice
grpc_byte_buffer_reader_readall(grpc_byte_buffer_reader* reader);
diff --git a/include/grpcpp/impl/codegen/core_codegen.h b/include/grpcpp/impl/codegen/core_codegen.h
index 27729e0d5db..b7ddb0c791c 100644
--- a/include/grpcpp/impl/codegen/core_codegen.h
+++ b/include/grpcpp/impl/codegen/core_codegen.h
@@ -85,8 +85,6 @@ class CoreCodegen final : public CoreCodegenInterface {
grpc_byte_buffer_reader* reader) override;
int grpc_byte_buffer_reader_next(grpc_byte_buffer_reader* reader,
grpc_slice* slice) override;
- int grpc_byte_buffer_reader_peek(grpc_byte_buffer_reader* reader,
- grpc_slice** slice) override;
grpc_byte_buffer* grpc_raw_byte_buffer_create(grpc_slice* slice,
size_t nslices) override;
diff --git a/include/grpcpp/impl/codegen/core_codegen_interface.h b/include/grpcpp/impl/codegen/core_codegen_interface.h
index 3792c3d4693..1d92b4f0dff 100644
--- a/include/grpcpp/impl/codegen/core_codegen_interface.h
+++ b/include/grpcpp/impl/codegen/core_codegen_interface.h
@@ -92,8 +92,6 @@ class CoreCodegenInterface {
grpc_byte_buffer_reader* reader) = 0;
virtual int grpc_byte_buffer_reader_next(grpc_byte_buffer_reader* reader,
grpc_slice* slice) = 0;
- virtual int grpc_byte_buffer_reader_peek(grpc_byte_buffer_reader* reader,
- grpc_slice** slice) = 0;
virtual grpc_byte_buffer* grpc_raw_byte_buffer_create(grpc_slice* slice,
size_t nslices) = 0;
diff --git a/include/grpcpp/impl/codegen/proto_buffer_reader.h b/include/grpcpp/impl/codegen/proto_buffer_reader.h
index 734da366f3a..9acae476b11 100644
--- a/include/grpcpp/impl/codegen/proto_buffer_reader.h
+++ b/include/grpcpp/impl/codegen/proto_buffer_reader.h
@@ -73,7 +73,7 @@ class ProtoBufferReader : public ::grpc::protobuf::io::ZeroCopyInputStream {
}
/// If we have backed up previously, we need to return the backed-up slice
if (backup_count_ > 0) {
- *data = GRPC_SLICE_START_PTR(*slice_) + GRPC_SLICE_LENGTH(*slice_) -
+ *data = GRPC_SLICE_START_PTR(slice_) + GRPC_SLICE_LENGTH(slice_) -
backup_count_;
GPR_CODEGEN_ASSERT(backup_count_ <= INT_MAX);
*size = (int)backup_count_;
@@ -81,14 +81,15 @@ class ProtoBufferReader : public ::grpc::protobuf::io::ZeroCopyInputStream {
return true;
}
/// Otherwise get the next slice from the byte buffer reader
- if (!g_core_codegen_interface->grpc_byte_buffer_reader_peek(&reader_,
+ if (!g_core_codegen_interface->grpc_byte_buffer_reader_next(&reader_,
&slice_)) {
return false;
}
- *data = GRPC_SLICE_START_PTR(*slice_);
+ g_core_codegen_interface->grpc_slice_unref(slice_);
+ *data = GRPC_SLICE_START_PTR(slice_);
// On win x64, int is only 32bit
- GPR_CODEGEN_ASSERT(GRPC_SLICE_LENGTH(*slice_) <= INT_MAX);
- byte_count_ += * size = (int)GRPC_SLICE_LENGTH(*slice_);
+ GPR_CODEGEN_ASSERT(GRPC_SLICE_LENGTH(slice_) <= INT_MAX);
+ byte_count_ += * size = (int)GRPC_SLICE_LENGTH(slice_);
return true;
}
@@ -99,7 +100,7 @@ class ProtoBufferReader : public ::grpc::protobuf::io::ZeroCopyInputStream {
/// bytes that have already been returned by the last call of Next.
/// So do the backup and have that ready for a later Next.
void BackUp(int count) override {
- GPR_CODEGEN_ASSERT(count <= static_cast<int>(GRPC_SLICE_LENGTH(*slice_)));
+ GPR_CODEGEN_ASSERT(count <= static_cast<int>(GRPC_SLICE_LENGTH(slice_)));
backup_count_ = count;
}
@@ -134,15 +135,14 @@ class ProtoBufferReader : public ::grpc::protobuf::io::ZeroCopyInputStream {
int64_t backup_count() { return backup_count_; }
void set_backup_count(int64_t backup_count) { backup_count_ = backup_count; }
grpc_byte_buffer_reader* reader() { return &reader_; }
- grpc_slice* slice() { return slice_; }
- grpc_slice** mutable_slice_ptr() { return &slice_; }
+ grpc_slice* slice() { return &slice_; }
private:
int64_t byte_count_; ///< total bytes read since object creation
int64_t backup_count_; ///< how far backed up in the stream we are
grpc_byte_buffer_reader reader_; ///< internal object to read \a grpc_slice
///< from the \a grpc_byte_buffer
- grpc_slice* slice_; ///< current slice passed back to the caller
+ grpc_slice slice_; ///< current slice passed back to the caller
Status status_; ///< status of the entire object
};
diff --git a/src/core/lib/surface/byte_buffer_reader.cc b/src/core/lib/surface/byte_buffer_reader.cc
index ed8ecc49590..1debc98ea0c 100644
--- a/src/core/lib/surface/byte_buffer_reader.cc
+++ b/src/core/lib/surface/byte_buffer_reader.cc
@@ -91,23 +91,6 @@ void grpc_byte_buffer_reader_destroy(grpc_byte_buffer_reader* reader) {
}
}
-int grpc_byte_buffer_reader_peek(grpc_byte_buffer_reader* reader,
- grpc_slice** slice) {
- switch (reader->buffer_in->type) {
- case GRPC_BB_RAW: {
- grpc_slice_buffer* slice_buffer;
- slice_buffer = &reader->buffer_out->data.raw.slice_buffer;
- if (reader->current.index < slice_buffer->count) {
- *slice = &slice_buffer->slices[reader->current.index];
- reader->current.index += 1;
- return 1;
- }
- break;
- }
- }
- return 0;
-}
-
int grpc_byte_buffer_reader_next(grpc_byte_buffer_reader* reader,
grpc_slice* slice) {
switch (reader->buffer_in->type) {
diff --git a/src/cpp/common/core_codegen.cc b/src/cpp/common/core_codegen.cc
index 665305ca0a5..ab5f601fdd4 100644
--- a/src/cpp/common/core_codegen.cc
+++ b/src/cpp/common/core_codegen.cc
@@ -139,11 +139,6 @@ int CoreCodegen::grpc_byte_buffer_reader_next(grpc_byte_buffer_reader* reader,
return ::grpc_byte_buffer_reader_next(reader, slice);
}
-int CoreCodegen::grpc_byte_buffer_reader_peek(grpc_byte_buffer_reader* reader,
- grpc_slice** slice) {
- return ::grpc_byte_buffer_reader_peek(reader, slice);
-}
-
grpc_byte_buffer* CoreCodegen::grpc_raw_byte_buffer_create(grpc_slice* slice,
size_t nslices) {
return ::grpc_raw_byte_buffer_create(slice, nslices);
diff --git a/src/ruby/ext/grpc/rb_grpc_imports.generated.c b/src/ruby/ext/grpc/rb_grpc_imports.generated.c
index f8a31286115..fdbe0df4e52 100644
--- a/src/ruby/ext/grpc/rb_grpc_imports.generated.c
+++ b/src/ruby/ext/grpc/rb_grpc_imports.generated.c
@@ -172,7 +172,6 @@ grpc_byte_buffer_destroy_type grpc_byte_buffer_destroy_import;
grpc_byte_buffer_reader_init_type grpc_byte_buffer_reader_init_import;
grpc_byte_buffer_reader_destroy_type grpc_byte_buffer_reader_destroy_import;
grpc_byte_buffer_reader_next_type grpc_byte_buffer_reader_next_import;
-grpc_byte_buffer_reader_peek_type grpc_byte_buffer_reader_peek_import;
grpc_byte_buffer_reader_readall_type grpc_byte_buffer_reader_readall_import;
grpc_raw_byte_buffer_from_reader_type grpc_raw_byte_buffer_from_reader_import;
gpr_log_severity_string_type gpr_log_severity_string_import;
@@ -441,7 +440,6 @@ void grpc_rb_load_imports(HMODULE library) {
grpc_byte_buffer_reader_init_import = (grpc_byte_buffer_reader_init_type) GetProcAddress(library, "grpc_byte_buffer_reader_init");
grpc_byte_buffer_reader_destroy_import = (grpc_byte_buffer_reader_destroy_type) GetProcAddress(library, "grpc_byte_buffer_reader_destroy");
grpc_byte_buffer_reader_next_import = (grpc_byte_buffer_reader_next_type) GetProcAddress(library, "grpc_byte_buffer_reader_next");
- grpc_byte_buffer_reader_peek_import = (grpc_byte_buffer_reader_peek_type) GetProcAddress(library, "grpc_byte_buffer_reader_peek");
grpc_byte_buffer_reader_readall_import = (grpc_byte_buffer_reader_readall_type) GetProcAddress(library, "grpc_byte_buffer_reader_readall");
grpc_raw_byte_buffer_from_reader_import = (grpc_raw_byte_buffer_from_reader_type) GetProcAddress(library, "grpc_raw_byte_buffer_from_reader");
gpr_log_severity_string_import = (gpr_log_severity_string_type) GetProcAddress(library, "gpr_log_severity_string");
diff --git a/src/ruby/ext/grpc/rb_grpc_imports.generated.h b/src/ruby/ext/grpc/rb_grpc_imports.generated.h
index 275ca6e9cbf..cf16f0ca33b 100644
--- a/src/ruby/ext/grpc/rb_grpc_imports.generated.h
+++ b/src/ruby/ext/grpc/rb_grpc_imports.generated.h
@@ -491,9 +491,6 @@ extern grpc_byte_buffer_reader_destroy_type grpc_byte_buffer_reader_destroy_impo
typedef int(*grpc_byte_buffer_reader_next_type)(grpc_byte_buffer_reader* reader, grpc_slice* slice);
extern grpc_byte_buffer_reader_next_type grpc_byte_buffer_reader_next_import;
#define grpc_byte_buffer_reader_next grpc_byte_buffer_reader_next_import
-typedef int(*grpc_byte_buffer_reader_peek_type)(grpc_byte_buffer_reader* reader, grpc_slice** slice);
-extern grpc_byte_buffer_reader_peek_type grpc_byte_buffer_reader_peek_import;
-#define grpc_byte_buffer_reader_peek grpc_byte_buffer_reader_peek_import
typedef grpc_slice(*grpc_byte_buffer_reader_readall_type)(grpc_byte_buffer_reader* reader);
extern grpc_byte_buffer_reader_readall_type grpc_byte_buffer_reader_readall_import;
#define grpc_byte_buffer_reader_readall grpc_byte_buffer_reader_readall_import
diff --git a/test/core/surface/byte_buffer_reader_test.cc b/test/core/surface/byte_buffer_reader_test.cc
index bc368c49657..301a1e283ba 100644
--- a/test/core/surface/byte_buffer_reader_test.cc
+++ b/test/core/surface/byte_buffer_reader_test.cc
@@ -101,73 +101,6 @@ static void test_read_none_compressed_slice(void) {
grpc_byte_buffer_destroy(buffer);
}
-static void test_peek_one_slice(void) {
- grpc_slice slice;
- grpc_byte_buffer* buffer;
- grpc_byte_buffer_reader reader;
- grpc_slice* first_slice;
- grpc_slice* second_slice;
- int first_code, second_code;
-
- LOG_TEST("test_peek_one_slice");
- slice = grpc_slice_from_copied_string("test");
- buffer = grpc_raw_byte_buffer_create(&slice, 1);
- grpc_slice_unref(slice);
- GPR_ASSERT(grpc_byte_buffer_reader_init(&reader, buffer) &&
- "Couldn't init byte buffer reader");
- first_code = grpc_byte_buffer_reader_peek(&reader, &first_slice);
- GPR_ASSERT(first_code != 0);
- GPR_ASSERT(memcmp(GRPC_SLICE_START_PTR(*first_slice), "test", 4) == 0);
- second_code = grpc_byte_buffer_reader_peek(&reader, &second_slice);
- GPR_ASSERT(second_code == 0);
- grpc_byte_buffer_destroy(buffer);
-}
-
-static void test_peek_one_slice_malloc(void) {
- grpc_slice slice;
- grpc_byte_buffer* buffer;
- grpc_byte_buffer_reader reader;
- grpc_slice* first_slice;
- grpc_slice* second_slice;
- int first_code, second_code;
-
- LOG_TEST("test_peek_one_slice_malloc");
- slice = grpc_slice_malloc(4);
- memcpy(GRPC_SLICE_START_PTR(slice), "test", 4);
- buffer = grpc_raw_byte_buffer_create(&slice, 1);
- grpc_slice_unref(slice);
- GPR_ASSERT(grpc_byte_buffer_reader_init(&reader, buffer) &&
- "Couldn't init byte buffer reader");
- first_code = grpc_byte_buffer_reader_peek(&reader, &first_slice);
- GPR_ASSERT(first_code != 0);
- GPR_ASSERT(memcmp(GRPC_SLICE_START_PTR(*first_slice), "test", 4) == 0);
- second_code = grpc_byte_buffer_reader_peek(&reader, &second_slice);
- GPR_ASSERT(second_code == 0);
- grpc_byte_buffer_destroy(buffer);
-}
-
-static void test_peek_none_compressed_slice(void) {
- grpc_slice slice;
- grpc_byte_buffer* buffer;
- grpc_byte_buffer_reader reader;
- grpc_slice* first_slice;
- grpc_slice* second_slice;
- int first_code, second_code;
-
- LOG_TEST("test_peek_none_compressed_slice");
- slice = grpc_slice_from_copied_string("test");
- buffer = grpc_raw_byte_buffer_create(&slice, 1);
- grpc_slice_unref(slice);
- GPR_ASSERT(grpc_byte_buffer_reader_init(&reader, buffer) &&
- "Couldn't init byte buffer reader");
- first_code = grpc_byte_buffer_reader_peek(&reader, &first_slice);
- GPR_ASSERT(first_code != 0);
- GPR_ASSERT(memcmp(GRPC_SLICE_START_PTR(*first_slice), "test", 4) == 0);
- second_code = grpc_byte_buffer_reader_peek(&reader, &second_slice);
- GPR_ASSERT(second_code == 0);
- grpc_byte_buffer_destroy(buffer);
-}
-
static void test_read_corrupted_slice(void) {
grpc_slice slice;
grpc_byte_buffer* buffer;
@@ -338,9 +271,6 @@ int main(int argc, char** argv) {
test_read_one_slice();
test_read_one_slice_malloc();
test_read_none_compressed_slice();
- test_peek_one_slice();
- test_peek_one_slice_malloc();
- test_peek_none_compressed_slice();
test_read_gzip_compressed_slice();
test_read_deflate_compressed_slice();
test_read_corrupted_slice();
diff --git a/test/core/surface/public_headers_must_be_c89.c b/test/core/surface/public_headers_must_be_c89.c
index fa02e76ec92..04d0506b3c2 100644
--- a/test/core/surface/public_headers_must_be_c89.c
+++ b/test/core/surface/public_headers_must_be_c89.c
@@ -209,7 +209,6 @@ int main(int argc, char **argv) {
printf("%lx", (unsigned long) grpc_byte_buffer_reader_init);
printf("%lx", (unsigned long) grpc_byte_buffer_reader_destroy);
printf("%lx", (unsigned long) grpc_byte_buffer_reader_next);
- printf("%lx", (unsigned long) grpc_byte_buffer_reader_peek);
printf("%lx", (unsigned long) grpc_byte_buffer_reader_readall);
printf("%lx", (unsigned long) grpc_raw_byte_buffer_from_reader);
printf("%lx", (unsigned long) gpr_log_severity_string);
diff --git a/test/cpp/microbenchmarks/bm_byte_buffer.cc b/test/cpp/microbenchmarks/bm_byte_buffer.cc
index 644c27c4873..a359e6f6212 100644
--- a/test/cpp/microbenchmarks/bm_byte_buffer.cc
+++ b/test/cpp/microbenchmarks/bm_byte_buffer.cc
@@ -29,8 +29,9 @@
namespace grpc {
namespace testing {
+auto& force_library_initialization = Library::get();
+
static void BM_ByteBuffer_Copy(benchmark::State& state) {
- Library::get();
int num_slices = state.range(0);
size_t slice_size = state.range(1);
std::vector<grpc::Slice> slices;
@@ -47,74 +48,6 @@ static void BM_ByteBuffer_Copy(benchmark::State& state) {
}
BENCHMARK(BM_ByteBuffer_Copy)->Ranges({{1, 64}, {1, 1024 * 1024}});
-static void BM_ByteBufferReader_Next(benchmark::State& state) {
- Library::get();
- const int num_slices = state.range(0);
- constexpr size_t kSliceSize = 16;
- std::vector<grpc_slice> slices;
- for (int i = 0; i < num_slices; ++i) {
- std::unique_ptr<char[]> buf(new char[kSliceSize]);
- slices.emplace_back(g_core_codegen_interface->grpc_slice_from_copied_buffer(
- buf.get(), kSliceSize));
- }
- grpc_byte_buffer* bb = g_core_codegen_interface->grpc_raw_byte_buffer_create(
- slices.data(), num_slices);
- grpc_byte_buffer_reader reader;
- GPR_ASSERT(
- g_core_codegen_interface->grpc_byte_buffer_reader_init(&reader, bb));
- while (state.KeepRunning()) {
- grpc_slice* slice;
- if (GPR_UNLIKELY(!g_core_codegen_interface->grpc_byte_buffer_reader_peek(
- &reader, &slice))) {
- g_core_codegen_interface->grpc_byte_buffer_reader_destroy(&reader);
- GPR_ASSERT(
- g_core_codegen_interface->grpc_byte_buffer_reader_init(&reader, bb));
- continue;
- }
- }
-
- g_core_codegen_interface->grpc_byte_buffer_reader_destroy(&reader);
- g_core_codegen_interface->grpc_byte_buffer_destroy(bb);
- for (auto& slice : slices) {
- g_core_codegen_interface->grpc_slice_unref(slice);
- }
-}
-BENCHMARK(BM_ByteBufferReader_Next)->Ranges({{64 * 1024, 1024 * 1024}});
-
-static void BM_ByteBufferReader_Peek(benchmark::State& state) {
- Library::get();
- const int num_slices = state.range(0);
- constexpr size_t kSliceSize = 16;
- std::vector<grpc_slice> slices;
- for (int i = 0; i < num_slices; ++i) {
- std::unique_ptr<char[]> buf(new char[kSliceSize]);
- slices.emplace_back(g_core_codegen_interface->grpc_slice_from_copied_buffer(
- buf.get(), kSliceSize));
- }
- grpc_byte_buffer* bb = g_core_codegen_interface->grpc_raw_byte_buffer_create(
- slices.data(), num_slices);
- grpc_byte_buffer_reader reader;
- GPR_ASSERT(
- g_core_codegen_interface->grpc_byte_buffer_reader_init(&reader, bb));
- while (state.KeepRunning()) {
- grpc_slice* slice;
- if (GPR_UNLIKELY(!g_core_codegen_interface->grpc_byte_buffer_reader_peek(
- &reader, &slice))) {
- g_core_codegen_interface->grpc_byte_buffer_reader_destroy(&reader);
- GPR_ASSERT(
- g_core_codegen_interface->grpc_byte_buffer_reader_init(&reader, bb));
- continue;
- }
- }
-
- g_core_codegen_interface->grpc_byte_buffer_reader_destroy(&reader);
- g_core_codegen_interface->grpc_byte_buffer_destroy(bb);
- for (auto& slice : slices) {
- g_core_codegen_interface->grpc_slice_unref(slice);
- }
-}
-BENCHMARK(BM_ByteBufferReader_Peek)->Ranges({{64 * 1024, 1024 * 1024}});
-
} // namespace testing
} // namespace grpc
From 9febbf2d9281225f25d7138e877c285d52239362 Mon Sep 17 00:00:00 2001
From: "Mark D. Roth" <roth@google.com>
Date: Tue, 5 Mar 2019 11:32:00 -0800
Subject: [PATCH 024/138] Revert "Make grpclb child policy configurable"
---
.../client_channel/lb_policy/grpclb/grpclb.cc | 344 +++++-------------
test/cpp/end2end/grpclb_end2end_test.cc | 144 --------
2 files changed, 86 insertions(+), 402 deletions(-)
diff --git a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
index c1f2846f046..c5d1ff22a9d 100644
--- a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
@@ -39,14 +39,15 @@
/// the balancer, we update the round_robin policy with the new list of
/// addresses. If we cannot communicate with the balancer on startup,
/// however, we may enter fallback mode, in which case we will populate
-/// the child policy's addresses from the backend addresses returned by the
+/// the RR policy's addresses from the backend addresses returned by the
/// resolver.
///
-/// Once a child policy instance is in place (and getting updated as described),
+/// Once an RR policy instance is in place (and getting updated as described),
/// calls for a pick, a ping, or a cancellation will be serviced right
-/// away by forwarding them to the child policy instance. Any time there's no
-/// child policy available (i.e., right after the creation of the gRPCLB
-/// policy), pick requests are queued.
+/// away by forwarding them to the RR instance. Any time there's no RR
+/// policy available (i.e., right after the creation of the gRPCLB policy),
+/// pick and ping requests are added to a list of pending picks and pings
+/// to be flushed and serviced when the RR policy instance becomes available.
///
/// \see https://github.com/grpc/grpc/blob/master/doc/load-balancing.md for the
/// high level design and details.
@@ -278,23 +279,16 @@ class GrpcLb : public LoadBalancingPolicy {
UniquePtr<SubchannelPicker> picker) override;
void RequestReresolution() override;
- void set_child(LoadBalancingPolicy* child) { child_ = child; }
-
private:
- bool CalledByPendingChild() const;
- bool CalledByCurrentChild() const;
-
RefCountedPtr<GrpcLb> parent_;
- LoadBalancingPolicy* child_ = nullptr;
};
~GrpcLb();
void ShutdownLocked() override;
- // Helper functions used in UpdateLocked().
+ // Helper function used in UpdateLocked().
void ProcessChannelArgsLocked(const grpc_channel_args& args);
- void ParseLbConfig(Config* grpclb_config);
// Methods for dealing with the balancer channel and call.
void StartBalancerCallLocked();
@@ -302,11 +296,10 @@ class GrpcLb : public LoadBalancingPolicy {
void StartBalancerCallRetryTimerLocked();
static void OnBalancerCallRetryTimerLocked(void* arg, grpc_error* error);
- // Methods for dealing with the child policy.
- grpc_channel_args* CreateChildPolicyArgsLocked();
- OrphanablePtr<LoadBalancingPolicy> CreateChildPolicyLocked(
- const char* name, grpc_channel_args* args);
- void CreateOrUpdateChildPolicyLocked();
+ // Methods for dealing with the RR policy.
+ grpc_channel_args* CreateRoundRobinPolicyArgsLocked();
+ void CreateRoundRobinPolicyLocked(Args args);
+ void CreateOrUpdateRoundRobinPolicyLocked();
// Who the client is trying to communicate with.
const char* server_name_ = nullptr;
@@ -352,14 +345,8 @@ class GrpcLb : public LoadBalancingPolicy {
grpc_timer lb_fallback_timer_;
grpc_closure lb_on_fallback_;
- // The child policy to use for the backends.
- OrphanablePtr<LoadBalancingPolicy> child_policy_;
- // When switching child policies, the new policy will be stored here
- // until it reports READY, at which point it will be moved to child_policy_.
- OrphanablePtr<LoadBalancingPolicy> pending_child_policy_;
- // The child policy name and config.
- UniquePtr<char> child_policy_name_;
- RefCountedPtr<Config> child_policy_config_;
+ // The RR policy to use for the backends.
+ OrphanablePtr<LoadBalancingPolicy> rr_policy_;
};
//
@@ -571,30 +558,14 @@ GrpcLb::Picker::PickResult GrpcLb::Picker::Pick(PickState* pick,
// GrpcLb::Helper
//
-bool GrpcLb::Helper::CalledByPendingChild() const {
- GPR_ASSERT(child_ != nullptr);
- return child_ == parent_->pending_child_policy_.get();
-}
-
-bool GrpcLb::Helper::CalledByCurrentChild() const {
- GPR_ASSERT(child_ != nullptr);
- return child_ == parent_->child_policy_.get();
-}
-
Subchannel* GrpcLb::Helper::CreateSubchannel(const grpc_channel_args& args) {
- if (parent_->shutting_down_ ||
- (!CalledByPendingChild() && !CalledByCurrentChild())) {
- return nullptr;
- }
+ if (parent_->shutting_down_) return nullptr;
return parent_->channel_control_helper()->CreateSubchannel(args);
}
grpc_channel* GrpcLb::Helper::CreateChannel(const char* target,
const grpc_channel_args& args) {
- if (parent_->shutting_down_ ||
- (!CalledByPendingChild() && !CalledByCurrentChild())) {
- return nullptr;
- }
+ if (parent_->shutting_down_) return nullptr;
return parent_->channel_control_helper()->CreateChannel(target, args);
}
@@ -605,50 +576,31 @@ void GrpcLb::Helper::UpdateState(grpc_connectivity_state state,
GRPC_ERROR_UNREF(state_error);
return;
}
- // If this request is from the pending child policy, ignore it until
- // it reports READY, at which point we swap it into place.
- if (CalledByPendingChild()) {
- if (grpc_lb_glb_trace.enabled()) {
- gpr_log(GPR_INFO,
- "[grpclb %p helper %p] pending child policy %p reports state=%s",
- parent_.get(), this, parent_->pending_child_policy_.get(),
- grpc_connectivity_state_name(state));
- }
- if (state != GRPC_CHANNEL_READY) {
- GRPC_ERROR_UNREF(state_error);
- return;
- }
- parent_->child_policy_ = std::move(parent_->pending_child_policy_);
- } else if (!CalledByCurrentChild()) {
- // This request is from an outdated child, so ignore it.
- GRPC_ERROR_UNREF(state_error);
- return;
- }
// There are three cases to consider here:
// 1. We're in fallback mode. In this case, we're always going to use
- // the child policy's result, so we pass its picker through as-is.
+ // RR's result, so we pass its picker through as-is.
// 2. The serverlist contains only drop entries. In this case, we
// want to use our own picker so that we can return the drops.
// 3. Not in fallback mode and serverlist is not all drops (i.e., it
// may be empty or contain at least one backend address). There are
// two sub-cases:
- // a. The child policy is reporting state READY. In this case, we wrap
- // the child's picker in our own, so that we can handle drops and LB
- // token metadata for each pick.
- // b. The child policy is reporting a state other than READY. In this
- // case, we don't want to use our own picker, because we don't want
- // to process drops for picks that yield a QUEUE result; this would
+ // a. RR is reporting state READY. In this case, we wrap RR's
+ // picker in our own, so that we can handle drops and LB token
+ // metadata for each pick.
+ // b. RR is reporting a state other than READY. In this case, we
+ // don't want to use our own picker, because we don't want to
+ // process drops for picks that yield a QUEUE result; this would
// result in dropping too many calls, since we will see the
// queued picks multiple times, and we'd consider each one a
// separate call for the drop calculation.
//
- // Cases 1 and 3b: return picker from the child policy as-is.
+ // Cases 1 and 3b: return picker from RR as-is.
if (parent_->serverlist_ == nullptr ||
(!parent_->serverlist_->ContainsAllDropEntries() &&
state != GRPC_CHANNEL_READY)) {
if (grpc_lb_glb_trace.enabled()) {
gpr_log(GPR_INFO,
- "[grpclb %p helper %p] state=%s passing child picker %p as-is",
+ "[grpclb %p helper %p] state=%s passing RR picker %p as-is",
parent_.get(), this, grpc_connectivity_state_name(state),
picker.get());
}
@@ -656,9 +608,9 @@ void GrpcLb::Helper::UpdateState(grpc_connectivity_state state,
std::move(picker));
return;
}
- // Cases 2 and 3a: wrap picker from the child in our own picker.
+ // Cases 2 and 3a: wrap picker from RR in our own picker.
if (grpc_lb_glb_trace.enabled()) {
- gpr_log(GPR_INFO, "[grpclb %p helper %p] state=%s wrapping child picker %p",
+ gpr_log(GPR_INFO, "[grpclb %p helper %p] state=%s wrapping RR picker %p",
parent_.get(), this, grpc_connectivity_state_name(state),
picker.get());
}
@@ -676,19 +628,15 @@ void GrpcLb::Helper::UpdateState(grpc_connectivity_state state,
void GrpcLb::Helper::RequestReresolution() {
if (parent_->shutting_down_) return;
- // If there is a pending child policy, ignore re-resolution requests
- // from the current child policy (or any outdated pending child).
- if (parent_->pending_child_policy_ != nullptr && !CalledByPendingChild()) {
- return;
- }
if (grpc_lb_glb_trace.enabled()) {
gpr_log(GPR_INFO,
- "[grpclb %p] Re-resolution requested from child policy (%p).",
- parent_.get(), child_);
+ "[grpclb %p] Re-resolution requested from the internal RR policy "
+ "(%p).",
+ parent_.get(), parent_->rr_policy_.get());
}
// If we are talking to a balancer, we expect to get updated addresses
// from the balancer, so we can ignore the re-resolution request from
- // the child policy. Otherwise, pass the re-resolution request up to the
+ // the RR policy. Otherwise, pass the re-resolution request up to the
// channel.
if (parent_->lb_calld_ == nullptr ||
!parent_->lb_calld_->seen_initial_response()) {
@@ -1036,7 +984,7 @@ void GrpcLb::BalancerCallState::OnBalancerMessageReceivedLocked(
// instance will be destroyed either upon the next update or when the
// GrpcLb instance is destroyed.
grpclb_policy->serverlist_ = std::move(serverlist_wrapper);
- grpclb_policy->CreateOrUpdateChildPolicyLocked();
+ grpclb_policy->CreateOrUpdateRoundRobinPolicyLocked();
}
} else {
// No valid initial response or serverlist found.
@@ -1252,8 +1200,7 @@ void GrpcLb::ShutdownLocked() {
if (fallback_timer_callback_pending_) {
grpc_timer_cancel(&lb_fallback_timer_);
}
- child_policy_.reset();
- pending_child_policy_.reset();
+ rr_policy_.reset();
// We destroy the LB channel here instead of in our destructor because
// destroying the channel triggers a last callback to
// OnBalancerChannelConnectivityChangedLocked(), and we need to be
@@ -1273,24 +1220,17 @@ void GrpcLb::ResetBackoffLocked() {
if (lb_channel_ != nullptr) {
grpc_channel_reset_connect_backoff(lb_channel_);
}
- if (child_policy_ != nullptr) {
- child_policy_->ResetBackoffLocked();
- }
- if (pending_child_policy_ != nullptr) {
- pending_child_policy_->ResetBackoffLocked();
+ if (rr_policy_ != nullptr) {
+ rr_policy_->ResetBackoffLocked();
}
}
void GrpcLb::FillChildRefsForChannelz(
channelz::ChildRefsList* child_subchannels,
channelz::ChildRefsList* child_channels) {
- // delegate to the child policy to fill the children subchannels.
- if (child_policy_ != nullptr) {
- child_policy_->FillChildRefsForChannelz(child_subchannels, child_channels);
- }
- if (pending_child_policy_ != nullptr) {
- pending_child_policy_->FillChildRefsForChannelz(child_subchannels,
- child_channels);
+ // delegate to the RoundRobin to fill the children subchannels.
+ if (rr_policy_ != nullptr) {
+ rr_policy_->FillChildRefsForChannelz(child_subchannels, child_channels);
}
gpr_atm uuid = gpr_atm_no_barrier_load(&lb_channel_uuid_);
if (uuid != 0) {
@@ -1298,32 +1238,6 @@ void GrpcLb::FillChildRefsForChannelz(
}
}
-void GrpcLb::UpdateLocked(const grpc_channel_args& args,
- RefCountedPtr<Config> lb_config) {
- const bool is_initial_update = lb_channel_ == nullptr;
- ParseLbConfig(lb_config.get());
- ProcessChannelArgsLocked(args);
- // Update the existing child policy.
- if (child_policy_ != nullptr) CreateOrUpdateChildPolicyLocked();
- // If this is the initial update, start the fallback timer.
- if (is_initial_update) {
- if (lb_fallback_timeout_ms_ > 0 && serverlist_ == nullptr &&
- !fallback_timer_callback_pending_) {
- grpc_millis deadline = ExecCtx::Get()->Now() + lb_fallback_timeout_ms_;
- Ref(DEBUG_LOCATION, "on_fallback_timer").release(); // Ref for callback
- GRPC_CLOSURE_INIT(&lb_on_fallback_, &GrpcLb::OnFallbackTimerLocked, this,
- grpc_combiner_scheduler(combiner()));
- fallback_timer_callback_pending_ = true;
- grpc_timer_init(&lb_fallback_timer_, deadline, &lb_on_fallback_);
- }
- StartBalancerCallLocked();
- }
-}
-
-//
-// helpers for UpdateLocked()
-//
-
// Returns the backend addresses extracted from the given addresses.
UniquePtr<ServerAddressList> ExtractBackendAddresses(
const ServerAddressList& addresses) {
@@ -1385,26 +1299,25 @@ void GrpcLb::ProcessChannelArgsLocked(const grpc_channel_args& args) {
grpc_channel_args_destroy(lb_channel_args);
}
-void GrpcLb::ParseLbConfig(Config* grpclb_config) {
- const grpc_json* child_policy = nullptr;
- if (grpclb_config != nullptr) {
- const grpc_json* grpclb_config_json = grpclb_config->json();
- for (const grpc_json* field = grpclb_config_json; field != nullptr;
- field = field->next) {
- if (field->key == nullptr) return;
- if (strcmp(field->key, "childPolicy") == 0) {
- if (child_policy != nullptr) return; // Duplicate.
- child_policy = ParseLoadBalancingConfig(field);
- }
+void GrpcLb::UpdateLocked(const grpc_channel_args& args,
+ RefCountedPtr<Config> lb_config) {
+ const bool is_initial_update = lb_channel_ == nullptr;
+ ProcessChannelArgsLocked(args);
+ // Update the existing RR policy.
+ if (rr_policy_ != nullptr) CreateOrUpdateRoundRobinPolicyLocked();
+ // If this is the initial update, start the fallback timer and the
+ // balancer call.
+ if (is_initial_update) {
+ if (lb_fallback_timeout_ms_ > 0 && serverlist_ == nullptr &&
+ !fallback_timer_callback_pending_) {
+ grpc_millis deadline = ExecCtx::Get()->Now() + lb_fallback_timeout_ms_;
+ Ref(DEBUG_LOCATION, "on_fallback_timer").release(); // Ref for callback
+ GRPC_CLOSURE_INIT(&lb_on_fallback_, &GrpcLb::OnFallbackTimerLocked, this,
+ grpc_combiner_scheduler(combiner()));
+ fallback_timer_callback_pending_ = true;
+ grpc_timer_init(&lb_fallback_timer_, deadline, &lb_on_fallback_);
}
- }
- if (child_policy != nullptr) {
- child_policy_name_ = UniquePtr<char>(gpr_strdup(child_policy->key));
- child_policy_config_ = MakeRefCounted<Config>(
- child_policy->child, grpclb_config->service_config());
- } else {
- child_policy_name_.reset();
- child_policy_config_.reset();
+ StartBalancerCallLocked();
}
}
@@ -1439,7 +1352,7 @@ void GrpcLb::OnFallbackTimerLocked(void* arg, grpc_error* error) {
grpclb_policy);
}
GPR_ASSERT(grpclb_policy->fallback_backend_addresses_ != nullptr);
- grpclb_policy->CreateOrUpdateChildPolicyLocked();
+ grpclb_policy->CreateOrUpdateRoundRobinPolicyLocked();
}
grpclb_policy->Unref(DEBUG_LOCATION, "on_fallback_timer");
}
@@ -1483,10 +1396,10 @@ void GrpcLb::OnBalancerCallRetryTimerLocked(void* arg, grpc_error* error) {
}
//
-// code for interacting with the child policy
+// code for interacting with the RR policy
//
-grpc_channel_args* GrpcLb::CreateChildPolicyArgsLocked() {
+grpc_channel_args* GrpcLb::CreateRoundRobinPolicyArgsLocked() {
ServerAddressList tmp_addresses;
ServerAddressList* addresses = &tmp_addresses;
bool is_backend_from_grpclb_load_balancer = false;
@@ -1495,7 +1408,7 @@ grpc_channel_args* GrpcLb::CreateChildPolicyArgsLocked() {
lb_calld_ == nullptr ? nullptr : lb_calld_->client_stats());
is_backend_from_grpclb_load_balancer = true;
} else {
- // If CreateOrUpdateChildPolicyLocked() is invoked when we haven't
+ // If CreateOrUpdateRoundRobinPolicyLocked() is invoked when we haven't
// received any serverlist from the balancer, we use the fallback backends
// returned by the resolver. Note that the fallback backend list may be
// empty, in which case the new round_robin policy will keep the requested
@@ -1522,134 +1435,49 @@ grpc_channel_args* GrpcLb::CreateChildPolicyArgsLocked() {
const_cast<char*>(GRPC_ARG_INHIBIT_HEALTH_CHECKING), 1);
++num_args_to_add;
}
- return grpc_channel_args_copy_and_add_and_remove(
+ grpc_channel_args* args = grpc_channel_args_copy_and_add_and_remove(
args_, keys_to_remove, GPR_ARRAY_SIZE(keys_to_remove), args_to_add,
num_args_to_add);
+ return args;
}
-OrphanablePtr<LoadBalancingPolicy> GrpcLb::CreateChildPolicyLocked(
- const char* name, grpc_channel_args* args) {
- Helper* helper = New<Helper>(Ref());
- LoadBalancingPolicy::Args lb_policy_args;
- lb_policy_args.combiner = combiner();
- lb_policy_args.args = args;
- lb_policy_args.channel_control_helper =
- UniquePtr<ChannelControlHelper>(helper);
- OrphanablePtr<LoadBalancingPolicy> lb_policy =
- LoadBalancingPolicyRegistry::CreateLoadBalancingPolicy(
- name, std::move(lb_policy_args));
- if (GPR_UNLIKELY(lb_policy == nullptr)) {
- gpr_log(GPR_ERROR, "[grpclb %p] Failure creating child policy %s", this,
- name);
- return nullptr;
+void GrpcLb::CreateRoundRobinPolicyLocked(Args args) {
+ GPR_ASSERT(rr_policy_ == nullptr);
+ rr_policy_ = LoadBalancingPolicyRegistry::CreateLoadBalancingPolicy(
+ "round_robin", std::move(args));
+ if (GPR_UNLIKELY(rr_policy_ == nullptr)) {
+ gpr_log(GPR_ERROR, "[grpclb %p] Failure creating a RoundRobin policy",
+ this);
+ return;
}
- helper->set_child(lb_policy.get());
if (grpc_lb_glb_trace.enabled()) {
- gpr_log(GPR_INFO, "[grpclb %p] Created new child policy %s (%p)", this,
- name, lb_policy.get());
+ gpr_log(GPR_INFO, "[grpclb %p] Created new RR policy %p", this,
+ rr_policy_.get());
}
// Add the gRPC LB's interested_parties pollset_set to that of the newly
- // created child policy. This will make the child policy progress upon
- // activity on gRPC LB, which in turn is tied to the application's call.
- grpc_pollset_set_add_pollset_set(lb_policy->interested_parties(),
+ // created RR policy. This will make the RR policy progress upon activity on
+ // gRPC LB, which in turn is tied to the application's call.
+ grpc_pollset_set_add_pollset_set(rr_policy_->interested_parties(),
interested_parties());
- return lb_policy;
}
-void GrpcLb::CreateOrUpdateChildPolicyLocked() {
+void GrpcLb::CreateOrUpdateRoundRobinPolicyLocked() {
if (shutting_down_) return;
- grpc_channel_args* args = CreateChildPolicyArgsLocked();
+ grpc_channel_args* args = CreateRoundRobinPolicyArgsLocked();
GPR_ASSERT(args != nullptr);
- // If the child policy name changes, we need to create a new child
- // policy. When this happens, we leave child_policy_ as-is and store
- // the new child policy in pending_child_policy_. Once the new child
- // policy transitions into state READY, we swap it into child_policy_,
- // replacing the original child policy. So pending_child_policy_ is
- // non-null only between when we apply an update that changes the child
- // policy name and when the new child reports state READY.
- //
- // Updates can arrive at any point during this transition. We always
- // apply updates relative to the most recently created child policy,
- // even if the most recent one is still in pending_child_policy_. This
- // is true both when applying the updates to an existing child policy
- // and when determining whether we need to create a new policy.
- //
- // As a result of this, there are several cases to consider here:
- //
- // 1. We have no existing child policy (i.e., we have started up but
- // have not yet received a serverlist from the balancer or gone
- // into fallback mode; in this case, both child_policy_ and
- // pending_child_policy_ are null). In this case, we create a
- // new child policy and store it in child_policy_.
- //
- // 2. We have an existing child policy and have no pending child policy
- // from a previous update (i.e., either there has not been a
- // previous update that changed the policy name, or we have already
- // finished swapping in the new policy; in this case, child_policy_
- // is non-null but pending_child_policy_ is null). In this case:
- // a. If child_policy_->name() equals child_policy_name, then we
- // update the existing child policy.
- // b. If child_policy_->name() does not equal child_policy_name,
- // we create a new policy. The policy will be stored in
- // pending_child_policy_ and will later be swapped into
- // child_policy_ by the helper when the new child transitions
- // into state READY.
- //
- // 3. We have an existing child policy and have a pending child policy
- // from a previous update (i.e., a previous update set
- // pending_child_policy_ as per case 2b above and that policy has
- // not yet transitioned into state READY and been swapped into
- // child_policy_; in this case, both child_policy_ and
- // pending_child_policy_ are non-null). In this case:
- // a. If pending_child_policy_->name() equals child_policy_name,
- // then we update the existing pending child policy.
- // b. If pending_child_policy->name() does not equal
- // child_policy_name, then we create a new policy. The new
- // policy is stored in pending_child_policy_ (replacing the one
- // that was there before, which will be immediately shut down)
- // and will later be swapped into child_policy_ by the helper
- // when the new child transitions into state READY.
- const char* child_policy_name =
- child_policy_name_ == nullptr ? "round_robin" : child_policy_name_.get();
- const bool create_policy =
- // case 1
- child_policy_ == nullptr ||
- // case 2b
- (pending_child_policy_ == nullptr &&
- strcmp(child_policy_->name(), child_policy_name) != 0) ||
- // case 3b
- (pending_child_policy_ != nullptr &&
- strcmp(pending_child_policy_->name(), child_policy_name) != 0);
- LoadBalancingPolicy* policy_to_update = nullptr;
- if (create_policy) {
- // Cases 1, 2b, and 3b: create a new child policy.
- // If child_policy_ is null, we set it (case 1), else we set
- // pending_child_policy_ (cases 2b and 3b).
- auto& lb_policy =
- child_policy_ == nullptr ? child_policy_ : pending_child_policy_;
- if (grpc_lb_glb_trace.enabled()) {
- gpr_log(GPR_INFO, "[grpclb %p] Creating new %schild policy %s", this,
- child_policy_ == nullptr ? "" : "pending ", child_policy_name);
- }
- lb_policy = CreateChildPolicyLocked(child_policy_name, args);
- policy_to_update = lb_policy.get();
- } else {
- // Cases 2a and 3a: update an existing policy.
- // If we have a pending child policy, send the update to the pending
- // policy (case 3a), else send it to the current policy (case 2a).
- policy_to_update = pending_child_policy_ != nullptr
- ? pending_child_policy_.get()
- : child_policy_.get();
+ if (rr_policy_ == nullptr) {
+ LoadBalancingPolicy::Args lb_policy_args;
+ lb_policy_args.combiner = combiner();
+ lb_policy_args.args = args;
+ lb_policy_args.channel_control_helper =
+ UniquePtr<ChannelControlHelper>(New<Helper>(Ref()));
+ CreateRoundRobinPolicyLocked(std::move(lb_policy_args));
}
- GPR_ASSERT(policy_to_update != nullptr);
- // Update the policy.
if (grpc_lb_glb_trace.enabled()) {
- gpr_log(GPR_INFO, "[grpclb %p] Updating %schild policy %p", this,
- policy_to_update == pending_child_policy_.get() ? "pending " : "",
- policy_to_update);
+ gpr_log(GPR_INFO, "[grpclb %p] Updating RR policy %p", this,
+ rr_policy_.get());
}
- policy_to_update->UpdateLocked(*args, child_policy_config_);
- // Clean up.
+ rr_policy_->UpdateLocked(*args, nullptr);
grpc_channel_args_destroy(args);
}
diff --git a/test/cpp/end2end/grpclb_end2end_test.cc b/test/cpp/end2end/grpclb_end2end_test.cc
index 31353ba1304..2288b88b517 100644
--- a/test/cpp/end2end/grpclb_end2end_test.cc
+++ b/test/cpp/end2end/grpclb_end2end_test.cc
@@ -723,150 +723,6 @@ TEST_F(SingleBalancerTest, SelectGrpclbWithMigrationServiceConfig) {
EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
}
-TEST_F(SingleBalancerTest, UsePickFirstChildPolicy) {
- SetNextResolutionAllBalancers(
- "{\n"
- " \"loadBalancingConfig\":[\n"
- " { \"grpclb\":{\n"
- " \"childPolicy\":[\n"
- " { \"pick_first\":{} }\n"
- " ]\n"
- " } }\n"
- " ]\n"
- "}");
- ScheduleResponseForBalancer(
- 0, BalancerServiceImpl::BuildResponseForBackends(GetBackendPorts(), {}),
- 0);
- const size_t kNumRpcs = num_backends_ * 2;
- CheckRpcSendOk(kNumRpcs, 1000 /* timeout_ms */, true /* wait_for_ready */);
- balancers_[0]->NotifyDoneWithServerlists();
- // Check that all requests went to the first backend. This verifies
- // that we used pick_first instead of round_robin as the child policy.
- EXPECT_EQ(backend_servers_[0].service_->request_count(), kNumRpcs);
- for (size_t i = 1; i < backends_.size(); ++i) {
- EXPECT_EQ(backend_servers_[i].service_->request_count(), 0UL);
- }
- // The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
- // and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
- // Check LB policy name for the channel.
- EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
-}
-
-TEST_F(SingleBalancerTest, SwapChildPolicy) {
- SetNextResolutionAllBalancers(
- "{\n"
- " \"loadBalancingConfig\":[\n"
- " { \"grpclb\":{\n"
- " \"childPolicy\":[\n"
- " { \"pick_first\":{} }\n"
- " ]\n"
- " } }\n"
- " ]\n"
- "}");
- ScheduleResponseForBalancer(
- 0, BalancerServiceImpl::BuildResponseForBackends(GetBackendPorts(), {}),
- 0);
- const size_t kNumRpcs = num_backends_ * 2;
- CheckRpcSendOk(kNumRpcs, 1000 /* timeout_ms */, true /* wait_for_ready */);
- // Check that all requests went to the first backend. This verifies
- // that we used pick_first instead of round_robin as the child policy.
- EXPECT_EQ(backend_servers_[0].service_->request_count(), kNumRpcs);
- for (size_t i = 1; i < backends_.size(); ++i) {
- EXPECT_EQ(backend_servers_[i].service_->request_count(), 0UL);
- }
- // Send new resolution that removes child policy from service config.
- SetNextResolutionAllBalancers("{}");
- WaitForAllBackends();
- CheckRpcSendOk(kNumRpcs, 1000 /* timeout_ms */, true /* wait_for_ready */);
- // Check that every backend saw the same number of requests. This verifies
- // that we used round_robin.
- for (size_t i = 0; i < backends_.size(); ++i) {
- EXPECT_EQ(backend_servers_[i].service_->request_count(), 2UL);
- }
- // Done.
- balancers_[0]->NotifyDoneWithServerlists();
- // The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
- // and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
- // Check LB policy name for the channel.
- EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
-}
-
-TEST_F(SingleBalancerTest, UpdatesGoToMostRecentChildPolicy) {
- const int kFallbackTimeoutMs = 200 * grpc_test_slowdown_factor();
- ResetStub(kFallbackTimeoutMs);
- int unreachable_balancer_port = grpc_pick_unused_port_or_die();
- int unreachable_backend_port = grpc_pick_unused_port_or_die();
- // Phase 1: Start with RR pointing to first backend.
- gpr_log(GPR_INFO, "PHASE 1: Initial setup with RR with first backend");
- SetNextResolution(
- {
- // Unreachable balancer.
- {unreachable_balancer_port, true, ""},
- // Fallback address: first backend.
- {backend_servers_[0].port_, false, ""},
- },
- "{\n"
- " \"loadBalancingConfig\":[\n"
- " { \"grpclb\":{\n"
- " \"childPolicy\":[\n"
- " { \"round_robin\":{} }\n"
- " ]\n"
- " } }\n"
- " ]\n"
- "}");
- // RPCs should go to first backend.
- WaitForBackend(0);
- // Phase 2: Switch to PF pointing to unreachable backend.
- gpr_log(GPR_INFO, "PHASE 2: Update to use PF with unreachable backend");
- SetNextResolution(
- {
- // Unreachable balancer.
- {unreachable_balancer_port, true, ""},
- // Fallback address: unreachable backend.
- {unreachable_backend_port, false, ""},
- },
- "{\n"
- " \"loadBalancingConfig\":[\n"
- " { \"grpclb\":{\n"
- " \"childPolicy\":[\n"
- " { \"pick_first\":{} }\n"
- " ]\n"
- " } }\n"
- " ]\n"
- "}");
- // RPCs should continue to go to the first backend, because the new
- // PF child policy will never go into state READY.
- WaitForBackend(0);
- // Phase 3: Switch back to RR pointing to second and third backends.
- // This ensures that we create a new policy rather than updating the
- // pending PF policy.
- gpr_log(GPR_INFO, "PHASE 3: Update to use RR again with two backends");
- SetNextResolution(
- {
- // Unreachable balancer.
- {unreachable_balancer_port, true, ""},
- // Fallback address: second and third backends.
- {backend_servers_[1].port_, false, ""},
- {backend_servers_[2].port_, false, ""},
- },
- "{\n"
- " \"loadBalancingConfig\":[\n"
- " { \"grpclb\":{\n"
- " \"childPolicy\":[\n"
- " { \"round_robin\":{} }\n"
- " ]\n"
- " } }\n"
- " ]\n"
- "}");
- // RPCs should go to the second and third backends.
- WaitForBackend(1);
- WaitForBackend(2);
-}
-
TEST_F(SingleBalancerTest, SameBackendListedMultipleTimes) {
SetNextResolutionAllBalancers();
// Same backend listed twice.
From e889fda4828872915229af82c286ae2be613f731 Mon Sep 17 00:00:00 2001
From: Juanli Shen <juanlishen@google.com>
Date: Tue, 5 Mar 2019 13:29:55 -0800
Subject: [PATCH 025/138] Use real resolver in xds lb channel
---
CMakeLists.txt | 48 +
Makefile | 52 +
build.yaml | 13 +
include/grpc/impl/codegen/grpc_types.h | 4 +
.../ext/filters/client_channel/lb_policy.h | 4 +-
.../client_channel/lb_policy/xds/xds.cc | 716 +++++-----
.../lb_policy/xds/xds_channel_secure.cc | 43 -
.../resolver/fake/fake_resolver.cc | 34 +-
.../resolver/fake/fake_resolver.h | 5 +-
.../fake/fake_security_connector.cc | 9 +-
test/cpp/end2end/BUILD | 22 +
test/cpp/end2end/xds_end2end_test.cc | 1214 +++++++++++++++++
.../generated/sources_and_headers.json | 22 +
tools/run_tests/generated/tests.json | 24 +
14 files changed, 1791 insertions(+), 419 deletions(-)
create mode 100644 test/cpp/end2end/xds_end2end_test.cc
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 939e83c481f..bccead24f28 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -720,6 +720,7 @@ add_dependencies(buildtests_cxx transport_security_common_api_test)
if(_gRPC_PLATFORM_LINUX OR _gRPC_PLATFORM_MAC OR _gRPC_PLATFORM_POSIX)
add_dependencies(buildtests_cxx writes_per_rpc_test)
endif()
+add_dependencies(buildtests_cxx xds_end2end_test)
add_dependencies(buildtests_cxx resolver_component_test_unsecure)
add_dependencies(buildtests_cxx resolver_component_test)
if(_gRPC_PLATFORM_LINUX OR _gRPC_PLATFORM_MAC OR _gRPC_PLATFORM_POSIX)
@@ -16232,6 +16233,53 @@ endif()
endif (gRPC_BUILD_TESTS)
if (gRPC_BUILD_TESTS)
+add_executable(xds_end2end_test
+ ${_gRPC_PROTO_GENS_DIR}/src/proto/grpc/lb/v1/load_balancer.pb.cc
+ ${_gRPC_PROTO_GENS_DIR}/src/proto/grpc/lb/v1/load_balancer.grpc.pb.cc
+ ${_gRPC_PROTO_GENS_DIR}/src/proto/grpc/lb/v1/load_balancer.pb.h
+ ${_gRPC_PROTO_GENS_DIR}/src/proto/grpc/lb/v1/load_balancer.grpc.pb.h
+ test/cpp/end2end/xds_end2end_test.cc
+ third_party/googletest/googletest/src/gtest-all.cc
+ third_party/googletest/googlemock/src/gmock-all.cc
+)
+
+protobuf_generate_grpc_cpp(
+ src/proto/grpc/lb/v1/load_balancer.proto
+)
+
+target_include_directories(xds_end2end_test
+ PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}
+ PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/include
+ PRIVATE ${_gRPC_SSL_INCLUDE_DIR}
+ PRIVATE ${_gRPC_PROTOBUF_INCLUDE_DIR}
+ PRIVATE ${_gRPC_ZLIB_INCLUDE_DIR}
+ PRIVATE ${_gRPC_BENCHMARK_INCLUDE_DIR}
+ PRIVATE ${_gRPC_CARES_INCLUDE_DIR}
+ PRIVATE ${_gRPC_GFLAGS_INCLUDE_DIR}
+ PRIVATE ${_gRPC_ADDRESS_SORTING_INCLUDE_DIR}
+ PRIVATE ${_gRPC_NANOPB_INCLUDE_DIR}
+ PRIVATE third_party/googletest/googletest/include
+ PRIVATE third_party/googletest/googletest
+ PRIVATE third_party/googletest/googlemock/include
+ PRIVATE third_party/googletest/googlemock
+ PRIVATE ${_gRPC_PROTO_GENS_DIR}
+)
+
+target_link_libraries(xds_end2end_test
+ ${_gRPC_PROTOBUF_LIBRARIES}
+ ${_gRPC_ALLTARGETS_LIBRARIES}
+ grpc++_test_util
+ grpc_test_util
+ grpc++
+ grpc
+ gpr
+ ${_gRPC_GFLAGS_LIBRARIES}
+)
+
+
+endif (gRPC_BUILD_TESTS)
+if (gRPC_BUILD_TESTS)
+
add_executable(public_headers_must_be_c89
test/core/surface/public_headers_must_be_c89.c
)
diff --git a/Makefile b/Makefile
index 3c890797431..94944265d61 100644
--- a/Makefile
+++ b/Makefile
@@ -1276,6 +1276,7 @@ time_change_test: $(BINDIR)/$(CONFIG)/time_change_test
transport_pid_controller_test: $(BINDIR)/$(CONFIG)/transport_pid_controller_test
transport_security_common_api_test: $(BINDIR)/$(CONFIG)/transport_security_common_api_test
writes_per_rpc_test: $(BINDIR)/$(CONFIG)/writes_per_rpc_test
+xds_end2end_test: $(BINDIR)/$(CONFIG)/xds_end2end_test
public_headers_must_be_c89: $(BINDIR)/$(CONFIG)/public_headers_must_be_c89
gen_hpack_tables: $(BINDIR)/$(CONFIG)/gen_hpack_tables
gen_legal_metadata_characters: $(BINDIR)/$(CONFIG)/gen_legal_metadata_characters
@@ -1787,6 +1788,7 @@ buildtests_cxx: privatelibs_cxx \
$(BINDIR)/$(CONFIG)/transport_pid_controller_test \
$(BINDIR)/$(CONFIG)/transport_security_common_api_test \
$(BINDIR)/$(CONFIG)/writes_per_rpc_test \
+ $(BINDIR)/$(CONFIG)/xds_end2end_test \
$(BINDIR)/$(CONFIG)/boringssl_crypto_test_data \
$(BINDIR)/$(CONFIG)/boringssl_asn1_test \
$(BINDIR)/$(CONFIG)/boringssl_base64_test \
@@ -1976,6 +1978,7 @@ buildtests_cxx: privatelibs_cxx \
$(BINDIR)/$(CONFIG)/transport_pid_controller_test \
$(BINDIR)/$(CONFIG)/transport_security_common_api_test \
$(BINDIR)/$(CONFIG)/writes_per_rpc_test \
+ $(BINDIR)/$(CONFIG)/xds_end2end_test \
$(BINDIR)/$(CONFIG)/resolver_component_test_unsecure \
$(BINDIR)/$(CONFIG)/resolver_component_test \
$(BINDIR)/$(CONFIG)/resolver_component_tests_runner_invoker_unsecure \
@@ -2500,6 +2503,8 @@ test_cxx: buildtests_cxx
$(Q) $(BINDIR)/$(CONFIG)/transport_security_common_api_test || ( echo test transport_security_common_api_test failed ; exit 1 )
$(E) "[RUN] Testing writes_per_rpc_test"
$(Q) $(BINDIR)/$(CONFIG)/writes_per_rpc_test || ( echo test writes_per_rpc_test failed ; exit 1 )
+ $(E) "[RUN] Testing xds_end2end_test"
+ $(Q) $(BINDIR)/$(CONFIG)/xds_end2end_test || ( echo test xds_end2end_test failed ; exit 1 )
$(E) "[RUN] Testing resolver_component_tests_runner_invoker_unsecure"
$(Q) $(BINDIR)/$(CONFIG)/resolver_component_tests_runner_invoker_unsecure || ( echo test resolver_component_tests_runner_invoker_unsecure failed ; exit 1 )
$(E) "[RUN] Testing resolver_component_tests_runner_invoker"
@@ -21308,6 +21313,53 @@ endif
endif
+XDS_END2END_TEST_SRC = \
+ $(GENDIR)/src/proto/grpc/lb/v1/load_balancer.pb.cc $(GENDIR)/src/proto/grpc/lb/v1/load_balancer.grpc.pb.cc \
+ test/cpp/end2end/xds_end2end_test.cc \
+
+XDS_END2END_TEST_OBJS = $(addprefix $(OBJDIR)/$(CONFIG)/, $(addsuffix .o, $(basename $(XDS_END2END_TEST_SRC))))
+ifeq ($(NO_SECURE),true)
+
+# You can't build secure targets if you don't have OpenSSL.
+
+$(BINDIR)/$(CONFIG)/xds_end2end_test: openssl_dep_error
+
+else
+
+
+
+
+ifeq ($(NO_PROTOBUF),true)
+
+# You can't build the protoc plugins or protobuf-enabled targets if you don't have protobuf 3.5.0+.
+
+$(BINDIR)/$(CONFIG)/xds_end2end_test: protobuf_dep_error
+
+else
+
+$(BINDIR)/$(CONFIG)/xds_end2end_test: $(PROTOBUF_DEP) $(XDS_END2END_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc++_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc++.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr.a
+ $(E) "[LD] Linking $@"
+ $(Q) mkdir -p `dirname $@`
+ $(Q) $(LDXX) $(LDFLAGS) $(XDS_END2END_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc++_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc++.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr.a $(LDLIBSXX) $(LDLIBS_PROTOBUF) $(LDLIBS) $(LDLIBS_SECURE) $(GTEST_LIB) -o $(BINDIR)/$(CONFIG)/xds_end2end_test
+
+endif
+
+endif
+
+$(OBJDIR)/$(CONFIG)/src/proto/grpc/lb/v1/load_balancer.o: $(LIBDIR)/$(CONFIG)/libgrpc++_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc++.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr.a
+
+$(OBJDIR)/$(CONFIG)/test/cpp/end2end/xds_end2end_test.o: $(LIBDIR)/$(CONFIG)/libgrpc++_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc++.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr.a
+
+deps_xds_end2end_test: $(XDS_END2END_TEST_OBJS:.o=.dep)
+
+ifneq ($(NO_SECURE),true)
+ifneq ($(NO_DEPS),true)
+-include $(XDS_END2END_TEST_OBJS:.o=.dep)
+endif
+endif
+$(OBJDIR)/$(CONFIG)/test/cpp/end2end/xds_end2end_test.o: $(GENDIR)/src/proto/grpc/lb/v1/load_balancer.pb.cc $(GENDIR)/src/proto/grpc/lb/v1/load_balancer.grpc.pb.cc
+
+
PUBLIC_HEADERS_MUST_BE_C89_SRC = \
test/core/surface/public_headers_must_be_c89.c \
diff --git a/build.yaml b/build.yaml
index c18630ecdd3..621b9a4de2f 100644
--- a/build.yaml
+++ b/build.yaml
@@ -5644,6 +5644,19 @@ targets:
- mac
- linux
- posix
+- name: xds_end2end_test
+ gtest: true
+ build: test
+ language: c++
+ src:
+ - src/proto/grpc/lb/v1/load_balancer.proto
+ - test/cpp/end2end/xds_end2end_test.cc
+ deps:
+ - grpc++_test_util
+ - grpc_test_util
+ - grpc++
+ - grpc
+ - gpr
- name: public_headers_must_be_c89
build: test
language: c89
diff --git a/include/grpc/impl/codegen/grpc_types.h b/include/grpc/impl/codegen/grpc_types.h
index 79b182c4515..078db2b90a8 100644
--- a/include/grpc/impl/codegen/grpc_types.h
+++ b/include/grpc/impl/codegen/grpc_types.h
@@ -317,6 +317,10 @@ typedef struct {
balancer before using fallback backend addresses from the resolver.
If 0, fallback will never be used. Default value is 10000. */
#define GRPC_ARG_GRPCLB_FALLBACK_TIMEOUT_MS "grpc.grpclb_fallback_timeout_ms"
+/* Timeout in milliseconds to wait for the serverlist from the xDS load
+ balancer before using fallback backend addresses from the resolver.
+ If 0, fallback will never be used. Default value is 10000. */
+#define GRPC_ARG_XDS_FALLBACK_TIMEOUT_MS "grpc.xds_fallback_timeout_ms"
/** If non-zero, grpc server's cronet compression workaround will be enabled */
#define GRPC_ARG_WORKAROUND_CRONET_COMPRESSION \
"grpc.workaround.cronet_compression"
diff --git a/src/core/ext/filters/client_channel/lb_policy.h b/src/core/ext/filters/client_channel/lb_policy.h
index 7a876966524..75dca52a615 100644
--- a/src/core/ext/filters/client_channel/lb_policy.h
+++ b/src/core/ext/filters/client_channel/lb_policy.h
@@ -297,8 +297,8 @@ class LoadBalancingPolicy : public InternallyRefCounted<LoadBalancingPolicy> {
grpc_combiner* combiner() const { return combiner_; }
- // Note: LB policies MUST NOT call any method on the helper from
- // their constructor.
+ // Note: LB policies MUST NOT call any method on the helper from their
+ // constructor.
// Note: This will return null after ShutdownLocked() has been called.
ChannelControlHelper* channel_control_helper() const {
return channel_control_helper_.get();
diff --git a/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc b/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc
index 6c10d876af7..5153330a84e 100644
--- a/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc
@@ -129,78 +129,128 @@ class XdsLb : public LoadBalancingPolicy {
channelz::ChildRefsList* child_channels) override;
private:
- /// Contains a call to the LB server and all the data related to the call.
- class BalancerCallState : public InternallyRefCounted<BalancerCallState> {
+ /// Contains a channel to the LB server and all the data related to the
+ /// channel.
+ class BalancerChannelState
+ : public InternallyRefCounted<BalancerChannelState> {
public:
- explicit BalancerCallState(
- RefCountedPtr<LoadBalancingPolicy> parent_xdslb_policy);
+ /// Contains a call to the LB server and all the data related to the call.
+ class BalancerCallState : public InternallyRefCounted<BalancerCallState> {
+ public:
+ explicit BalancerCallState(RefCountedPtr<BalancerChannelState> lb_chand);
- // It's the caller's responsibility to ensure that Orphan() is called from
- // inside the combiner.
- void Orphan() override;
-
- void StartQuery();
-
- XdsLbClientStats* client_stats() const { return client_stats_.get(); }
+ // It's the caller's responsibility to ensure that Orphan() is called from
+ // inside the combiner.
+ void Orphan() override;
- bool seen_initial_response() const { return seen_initial_response_; }
-
- private:
- // So Delete() can access our private dtor.
- template <typename T>
- friend void grpc_core::Delete(T*);
+ void StartQuery();
- ~BalancerCallState();
+ RefCountedPtr<XdsLbClientStats> client_stats() const {
+ return client_stats_;
+ }
- XdsLb* xdslb_policy() const {
- return static_cast<XdsLb*>(xdslb_policy_.get());
- }
+ bool seen_initial_response() const { return seen_initial_response_; }
- void ScheduleNextClientLoadReportLocked();
- void SendClientLoadReportLocked();
+ private:
+ // So Delete() can access our private dtor.
+ template <typename T>
+ friend void grpc_core::Delete(T*);
- static bool LoadReportCountersAreZero(xds_grpclb_request* request);
+ ~BalancerCallState();
- static void MaybeSendClientLoadReportLocked(void* arg, grpc_error* error);
- static void OnInitialRequestSentLocked(void* arg, grpc_error* error);
- static void OnBalancerMessageReceivedLocked(void* arg, grpc_error* error);
- static void OnBalancerStatusReceivedLocked(void* arg, grpc_error* error);
+ XdsLb* xdslb_policy() const { return lb_chand_->xdslb_policy_.get(); }
- // The owning LB policy.
- RefCountedPtr<LoadBalancingPolicy> xdslb_policy_;
+ bool IsCurrentCallOnChannel() const {
+ return this == lb_chand_->lb_calld_.get();
+ }
- // The streaming call to the LB server. Always non-NULL.
- grpc_call* lb_call_ = nullptr;
+ void ScheduleNextClientLoadReportLocked();
+ void SendClientLoadReportLocked();
+
+ static bool LoadReportCountersAreZero(xds_grpclb_request* request);
+
+ static void MaybeSendClientLoadReportLocked(void* arg, grpc_error* error);
+ static void OnInitialRequestSentLocked(void* arg, grpc_error* error);
+ static void OnBalancerMessageReceivedLocked(void* arg, grpc_error* error);
+ static void OnBalancerStatusReceivedLocked(void* arg, grpc_error* error);
+
+ // The owning LB channel.
+ RefCountedPtr<BalancerChannelState> lb_chand_;
+
+ // The streaming call to the LB server. Always non-NULL.
+ grpc_call* lb_call_ = nullptr;
+
+ // recv_initial_metadata
+ grpc_metadata_array lb_initial_metadata_recv_;
+
+ // send_message
+ grpc_byte_buffer* send_message_payload_ = nullptr;
+ grpc_closure lb_on_initial_request_sent_;
+
+ // recv_message
+ grpc_byte_buffer* recv_message_payload_ = nullptr;
+ grpc_closure lb_on_balancer_message_received_;
+ bool seen_initial_response_ = false;
+
+ // recv_trailing_metadata
+ grpc_closure lb_on_balancer_status_received_;
+ grpc_metadata_array lb_trailing_metadata_recv_;
+ grpc_status_code lb_call_status_;
+ grpc_slice lb_call_status_details_;
+
+ // The stats for client-side load reporting associated with this LB call.
+ // Created after the first serverlist is received.
+ RefCountedPtr<XdsLbClientStats> client_stats_;
+ grpc_millis client_stats_report_interval_ = 0;
+ grpc_timer client_load_report_timer_;
+ bool client_load_report_timer_callback_pending_ = false;
+ bool last_client_load_report_counters_were_zero_ = false;
+ bool client_load_report_is_due_ = false;
+ // The closure used for either the load report timer or the callback for
+ // completion of sending the load report.
+ grpc_closure client_load_report_closure_;
+ };
+
+ BalancerChannelState(const char* balancer_name,
+ const grpc_channel_args& args,
+ RefCountedPtr<XdsLb> parent_xdslb_policy);
+ ~BalancerChannelState();
- // recv_initial_metadata
- grpc_metadata_array lb_initial_metadata_recv_;
+ void Orphan() override;
- // send_message
- grpc_byte_buffer* send_message_payload_ = nullptr;
- grpc_closure lb_on_initial_request_sent_;
+ grpc_channel* channel() const { return channel_; }
+ BalancerCallState* lb_calld() const { return lb_calld_.get(); }
- // recv_message
- grpc_byte_buffer* recv_message_payload_ = nullptr;
- grpc_closure lb_on_balancer_message_received_;
- bool seen_initial_response_ = false;
+ bool IsCurrentChannel() const {
+ return this == xdslb_policy_->lb_chand_.get();
+ }
+ bool IsPendingChannel() const {
+ return this == xdslb_policy_->pending_lb_chand_.get();
+ }
+ bool HasActiveCall() const { return lb_calld_ != nullptr; }
- // recv_trailing_metadata
- grpc_closure lb_on_balancer_status_received_;
- grpc_metadata_array lb_trailing_metadata_recv_;
- grpc_status_code lb_call_status_;
- grpc_slice lb_call_status_details_;
+ void StartCallRetryTimerLocked();
+ static void OnCallRetryTimerLocked(void* arg, grpc_error* error);
+ void StartCallLocked();
- // The stats for client-side load reporting associated with this LB call.
- // Created after the first serverlist is received.
- RefCountedPtr<XdsLbClientStats> client_stats_;
- grpc_millis client_stats_report_interval_ = 0;
- grpc_timer client_load_report_timer_;
- bool client_load_report_timer_callback_pending_ = false;
- bool last_client_load_report_counters_were_zero_ = false;
- bool client_load_report_is_due_ = false;
- // The closure used for either the load report timer or the callback for
- // completion of sending the load report.
- grpc_closure client_load_report_closure_;
+ private:
+ // The owning LB policy.
+ RefCountedPtr<XdsLb> xdslb_policy_;
+
+ // The channel and its status.
+ grpc_channel* channel_;
+ bool shutting_down_ = false;
+
+ // The data associated with the current LB call. It holds a ref to this LB
+ // channel. It's instantiated every time we query for backends. It's reset
+ // whenever the current LB call is no longer needed (e.g., the LB policy is
+ // shutting down, or the LB call has ended). A non-NULL lb_calld_ always
+ // contains a non-NULL lb_call_.
+ OrphanablePtr<BalancerCallState> lb_calld_;
+ BackOff lb_call_backoff_;
+ grpc_timer lb_call_retry_timer_;
+ grpc_closure lb_on_call_retry_;
+ bool retry_timer_callback_pending_ = false;
};
class Picker : public SubchannelPicker {
@@ -245,13 +295,13 @@ class XdsLb : public LoadBalancingPolicy {
// found. Does nothing upon failure.
void ParseLbConfig(Config* xds_config);
- // Methods for dealing with the balancer channel and call.
- void StartBalancerCallLocked();
+ BalancerChannelState* LatestLbChannel() const {
+ return pending_lb_chand_ != nullptr ? pending_lb_chand_.get()
+ : lb_chand_.get();
+ }
+
+ // Callback to enter fallback mode.
static void OnFallbackTimerLocked(void* arg, grpc_error* error);
- void StartBalancerCallRetryTimerLocked();
- static void OnBalancerCallRetryTimerLocked(void* arg, grpc_error* error);
- static void OnBalancerChannelConnectivityChangedLocked(void* arg,
- grpc_error* error);
// Methods for dealing with the child policy.
void CreateOrUpdateChildPolicyLocked();
@@ -271,30 +321,15 @@ class XdsLb : public LoadBalancingPolicy {
bool shutting_down_ = false;
// The channel for communicating with the LB server.
- grpc_channel* lb_channel_ = nullptr;
+ OrphanablePtr<BalancerChannelState> lb_chand_;
+ OrphanablePtr<BalancerChannelState> pending_lb_chand_;
// Mutex to protect the channel to the LB server. This is used when
// processing a channelz request.
- gpr_mu lb_channel_mu_;
- grpc_connectivity_state lb_channel_connectivity_;
- grpc_closure lb_channel_on_connectivity_changed_;
- // Are we already watching the LB channel's connectivity?
- bool watching_lb_channel_ = false;
- // Response generator to inject address updates into lb_channel_.
- RefCountedPtr<FakeResolverResponseGenerator> response_generator_;
-
- // The data associated with the current LB call. It holds a ref to this LB
- // policy. It's initialized every time we query for backends. It's reset to
- // NULL whenever the current LB call is no longer needed (e.g., the LB policy
- // is shutting down, or the LB call has ended). A non-NULL lb_calld_ always
- // contains a non-NULL lb_call_.
- OrphanablePtr<BalancerCallState> lb_calld_;
+ // TODO(juanlishen): Replace this with atomic.
+ gpr_mu lb_chand_mu_;
+
// Timeout in milliseconds for the LB call. 0 means no deadline.
int lb_call_timeout_ms_ = 0;
- // Balancer call retry state.
- BackOff lb_call_backoff_;
- bool retry_timer_callback_pending_ = false;
- grpc_timer lb_call_retry_timer_;
- grpc_closure lb_on_call_retry_;
// The deserialized response from the balancer. May be nullptr until one
// such response has arrived.
@@ -360,11 +395,11 @@ void XdsLb::Helper::UpdateState(grpc_connectivity_state state,
// TODO(juanlishen): When in fallback mode, pass the child picker
// through without wrapping it. (Or maybe use a different helper for
// the fallback policy?)
- RefCountedPtr<XdsLbClientStats> client_stats;
- if (parent_->lb_calld_ != nullptr &&
- parent_->lb_calld_->client_stats() != nullptr) {
- client_stats = parent_->lb_calld_->client_stats()->Ref();
- }
+ GPR_ASSERT(parent_->lb_chand_ != nullptr);
+ RefCountedPtr<XdsLbClientStats> client_stats =
+ parent_->lb_chand_->lb_calld() == nullptr
+ ? nullptr
+ : parent_->lb_chand_->lb_calld()->client_stats();
parent_->channel_control_helper()->UpdateState(
state, state_error,
UniquePtr<SubchannelPicker>(
@@ -379,12 +414,13 @@ void XdsLb::Helper::RequestReresolution() {
"(%p).",
parent_.get(), parent_->child_policy_.get());
}
+ GPR_ASSERT(parent_->lb_chand_ != nullptr);
// If we are talking to a balancer, we expect to get updated addresses
// from the balancer, so we can ignore the re-resolution request from
- // the RR policy. Otherwise, pass the re-resolution request up to the
+ // the child policy. Otherwise, pass the re-resolution request up to the
// channel.
- if (parent_->lb_calld_ == nullptr ||
- !parent_->lb_calld_->seen_initial_response()) {
+ if (parent_->lb_chand_->lb_calld() == nullptr ||
+ !parent_->lb_chand_->lb_calld()->seen_initial_response()) {
parent_->channel_control_helper()->RequestReresolution();
}
}
@@ -465,14 +501,98 @@ UniquePtr<ServerAddressList> ProcessServerlist(
}
//
-// XdsLb::BalancerCallState
+// XdsLb::BalancerChannelState
//
-XdsLb::BalancerCallState::BalancerCallState(
- RefCountedPtr<LoadBalancingPolicy> parent_xdslb_policy)
+XdsLb::BalancerChannelState::BalancerChannelState(
+ const char* balancer_name, const grpc_channel_args& args,
+ grpc_core::RefCountedPtr<grpc_core::XdsLb> parent_xdslb_policy)
+ : InternallyRefCounted<BalancerChannelState>(&grpc_lb_xds_trace),
+ xdslb_policy_(std::move(parent_xdslb_policy)),
+ lb_call_backoff_(
+ BackOff::Options()
+ .set_initial_backoff(GRPC_XDS_INITIAL_CONNECT_BACKOFF_SECONDS *
+ 1000)
+ .set_multiplier(GRPC_XDS_RECONNECT_BACKOFF_MULTIPLIER)
+ .set_jitter(GRPC_XDS_RECONNECT_JITTER)
+ .set_max_backoff(GRPC_XDS_RECONNECT_MAX_BACKOFF_SECONDS * 1000)) {
+ channel_ = xdslb_policy_->channel_control_helper()->CreateChannel(
+ balancer_name, args);
+ GPR_ASSERT(channel_ != nullptr);
+ StartCallLocked();
+}
+
+XdsLb::BalancerChannelState::~BalancerChannelState() {
+ grpc_channel_destroy(channel_);
+}
+
+void XdsLb::BalancerChannelState::Orphan() {
+ shutting_down_ = true;
+ lb_calld_.reset();
+ if (retry_timer_callback_pending_) grpc_timer_cancel(&lb_call_retry_timer_);
+ Unref(DEBUG_LOCATION, "lb_channel_orphaned");
+}
+
+void XdsLb::BalancerChannelState::StartCallRetryTimerLocked() {
+ grpc_millis next_try = lb_call_backoff_.NextAttemptTime();
+ if (grpc_lb_xds_trace.enabled()) {
+ gpr_log(GPR_INFO,
+ "[xdslb %p] Failed to connect to LB server (lb_chand: %p)...",
+ xdslb_policy_.get(), this);
+ grpc_millis timeout = next_try - ExecCtx::Get()->Now();
+ if (timeout > 0) {
+ gpr_log(GPR_INFO, "[xdslb %p] ... retry_timer_active in %" PRId64 "ms.",
+ xdslb_policy_.get(), timeout);
+ } else {
+ gpr_log(GPR_INFO, "[xdslb %p] ... retry_timer_active immediately.",
+ xdslb_policy_.get());
+ }
+ }
+ Ref(DEBUG_LOCATION, "on_balancer_call_retry_timer").release();
+ GRPC_CLOSURE_INIT(&lb_on_call_retry_, &OnCallRetryTimerLocked, this,
+ grpc_combiner_scheduler(xdslb_policy_->combiner()));
+ grpc_timer_init(&lb_call_retry_timer_, next_try, &lb_on_call_retry_);
+ retry_timer_callback_pending_ = true;
+}
+
+void XdsLb::BalancerChannelState::OnCallRetryTimerLocked(void* arg,
+ grpc_error* error) {
+ BalancerChannelState* lb_chand = static_cast<BalancerChannelState*>(arg);
+ lb_chand->retry_timer_callback_pending_ = false;
+ if (!lb_chand->shutting_down_ && error == GRPC_ERROR_NONE &&
+ lb_chand->lb_calld_ == nullptr) {
+ if (grpc_lb_xds_trace.enabled()) {
+ gpr_log(GPR_INFO,
+ "[xdslb %p] Restarting call to LB server (lb_chand: %p)",
+ lb_chand->xdslb_policy_.get(), lb_chand);
+ }
+ lb_chand->StartCallLocked();
+ }
+ lb_chand->Unref(DEBUG_LOCATION, "on_balancer_call_retry_timer");
+}
+
+void XdsLb::BalancerChannelState::StartCallLocked() {
+ if (shutting_down_) return;
+ GPR_ASSERT(channel_ != nullptr);
+ GPR_ASSERT(lb_calld_ == nullptr);
+ lb_calld_ = MakeOrphanable<BalancerCallState>(Ref());
+ if (grpc_lb_xds_trace.enabled()) {
+ gpr_log(GPR_INFO,
+ "[xdslb %p] Query for backends (lb_chand: %p, lb_calld: %p)",
+ xdslb_policy_.get(), this, lb_calld_.get());
+ }
+ lb_calld_->StartQuery();
+}
+
+//
+// XdsLb::BalancerChannelState::BalancerCallState
+//
+
+XdsLb::BalancerChannelState::BalancerCallState::BalancerCallState(
+ RefCountedPtr<BalancerChannelState> lb_chand)
: InternallyRefCounted<BalancerCallState>(&grpc_lb_xds_trace),
- xdslb_policy_(std::move(parent_xdslb_policy)) {
- GPR_ASSERT(xdslb_policy_ != nullptr);
+ lb_chand_(std::move(lb_chand)) {
+ GPR_ASSERT(xdslb_policy() != nullptr);
GPR_ASSERT(!xdslb_policy()->shutting_down_);
// Init the LB call. Note that the LB call will progress every time there's
// activity in xdslb_policy_->interested_parties(), which is comprised of
@@ -484,8 +604,8 @@ XdsLb::BalancerCallState::BalancerCallState(
? GRPC_MILLIS_INF_FUTURE
: ExecCtx::Get()->Now() + xdslb_policy()->lb_call_timeout_ms_;
lb_call_ = grpc_channel_create_pollset_set_call(
- xdslb_policy()->lb_channel_, nullptr, GRPC_PROPAGATE_DEFAULTS,
- xdslb_policy_->interested_parties(),
+ lb_chand_->channel_, nullptr, GRPC_PROPAGATE_DEFAULTS,
+ xdslb_policy()->interested_parties(),
GRPC_MDSTR_SLASH_GRPC_DOT_LB_DOT_V1_DOT_LOADBALANCER_SLASH_BALANCELOAD,
nullptr, deadline, nullptr);
// Init the LB call request payload.
@@ -509,7 +629,7 @@ XdsLb::BalancerCallState::BalancerCallState(
grpc_combiner_scheduler(xdslb_policy()->combiner()));
}
-XdsLb::BalancerCallState::~BalancerCallState() {
+XdsLb::BalancerChannelState::BalancerCallState::~BalancerCallState() {
GPR_ASSERT(lb_call_ != nullptr);
grpc_call_unref(lb_call_);
grpc_metadata_array_destroy(&lb_initial_metadata_recv_);
@@ -519,7 +639,7 @@ XdsLb::BalancerCallState::~BalancerCallState() {
grpc_slice_unref_internal(lb_call_status_details_);
}
-void XdsLb::BalancerCallState::Orphan() {
+void XdsLb::BalancerChannelState::BalancerCallState::Orphan() {
GPR_ASSERT(lb_call_ != nullptr);
// If we are here because xdslb_policy wants to cancel the call,
// lb_on_balancer_status_received_ will complete the cancellation and clean
@@ -534,11 +654,11 @@ void XdsLb::BalancerCallState::Orphan() {
// in lb_on_balancer_status_received_ instead of here.
}
-void XdsLb::BalancerCallState::StartQuery() {
+void XdsLb::BalancerChannelState::BalancerCallState::StartQuery() {
GPR_ASSERT(lb_call_ != nullptr);
if (grpc_lb_xds_trace.enabled()) {
gpr_log(GPR_INFO, "[xdslb %p] Starting LB call (lb_calld: %p, lb_call: %p)",
- xdslb_policy_.get(), this, lb_call_);
+ xdslb_policy(), this, lb_call_);
}
// Create the ops.
grpc_call_error call_error;
@@ -606,7 +726,8 @@ void XdsLb::BalancerCallState::StartQuery() {
GPR_ASSERT(GRPC_CALL_OK == call_error);
}
-void XdsLb::BalancerCallState::ScheduleNextClientLoadReportLocked() {
+void XdsLb::BalancerChannelState::BalancerCallState::
+ ScheduleNextClientLoadReportLocked() {
const grpc_millis next_client_load_report_time =
ExecCtx::Get()->Now() + client_stats_report_interval_;
GRPC_CLOSURE_INIT(&client_load_report_closure_,
@@ -617,12 +738,11 @@ void XdsLb::BalancerCallState::ScheduleNextClientLoadReportLocked() {
client_load_report_timer_callback_pending_ = true;
}
-void XdsLb::BalancerCallState::MaybeSendClientLoadReportLocked(
- void* arg, grpc_error* error) {
+void XdsLb::BalancerChannelState::BalancerCallState::
+ MaybeSendClientLoadReportLocked(void* arg, grpc_error* error) {
BalancerCallState* lb_calld = static_cast<BalancerCallState*>(arg);
- XdsLb* xdslb_policy = lb_calld->xdslb_policy();
lb_calld->client_load_report_timer_callback_pending_ = false;
- if (error != GRPC_ERROR_NONE || lb_calld != xdslb_policy->lb_calld_.get()) {
+ if (error != GRPC_ERROR_NONE || !lb_calld->IsCurrentCallOnChannel()) {
lb_calld->Unref(DEBUG_LOCATION, "client_load_report");
return;
}
@@ -636,7 +756,7 @@ void XdsLb::BalancerCallState::MaybeSendClientLoadReportLocked(
}
}
-bool XdsLb::BalancerCallState::LoadReportCountersAreZero(
+bool XdsLb::BalancerChannelState::BalancerCallState::LoadReportCountersAreZero(
xds_grpclb_request* request) {
XdsLbClientStats::DroppedCallCounts* drop_entries =
static_cast<XdsLbClientStats::DroppedCallCounts*>(
@@ -650,7 +770,8 @@ bool XdsLb::BalancerCallState::LoadReportCountersAreZero(
}
// TODO(vpowar): Use LRS to send the client Load Report.
-void XdsLb::BalancerCallState::SendClientLoadReportLocked() {
+void XdsLb::BalancerChannelState::BalancerCallState::
+ SendClientLoadReportLocked() {
// Construct message payload.
GPR_ASSERT(send_message_payload_ == nullptr);
xds_grpclb_request* request =
@@ -671,27 +792,27 @@ void XdsLb::BalancerCallState::SendClientLoadReportLocked() {
xds_grpclb_request_destroy(request);
}
-void XdsLb::BalancerCallState::OnInitialRequestSentLocked(void* arg,
- grpc_error* error) {
+void XdsLb::BalancerChannelState::BalancerCallState::OnInitialRequestSentLocked(
+ void* arg, grpc_error* error) {
BalancerCallState* lb_calld = static_cast<BalancerCallState*>(arg);
grpc_byte_buffer_destroy(lb_calld->send_message_payload_);
lb_calld->send_message_payload_ = nullptr;
// If we attempted to send a client load report before the initial request was
// sent (and this lb_calld is still in use), send the load report now.
if (lb_calld->client_load_report_is_due_ &&
- lb_calld == lb_calld->xdslb_policy()->lb_calld_.get()) {
+ lb_calld->IsCurrentCallOnChannel()) {
lb_calld->SendClientLoadReportLocked();
lb_calld->client_load_report_is_due_ = false;
}
lb_calld->Unref(DEBUG_LOCATION, "on_initial_request_sent");
}
-void XdsLb::BalancerCallState::OnBalancerMessageReceivedLocked(
- void* arg, grpc_error* error) {
+void XdsLb::BalancerChannelState::BalancerCallState::
+ OnBalancerMessageReceivedLocked(void* arg, grpc_error* error) {
BalancerCallState* lb_calld = static_cast<BalancerCallState*>(arg);
XdsLb* xdslb_policy = lb_calld->xdslb_policy();
// Empty payload means the LB call was cancelled.
- if (lb_calld != xdslb_policy->lb_calld_.get() ||
+ if (!lb_calld->IsCurrentCallOnChannel() ||
lb_calld->recv_message_payload_ == nullptr) {
lb_calld->Unref(DEBUG_LOCATION, "on_message_received");
return;
@@ -709,20 +830,25 @@ void XdsLb::BalancerCallState::OnBalancerMessageReceivedLocked(
nullptr) {
// Have NOT seen initial response, look for initial response.
if (initial_response->has_client_stats_report_interval) {
- lb_calld->client_stats_report_interval_ = GPR_MAX(
- GPR_MS_PER_SEC, xds_grpclb_duration_to_millis(
- &initial_response->client_stats_report_interval));
- if (grpc_lb_xds_trace.enabled()) {
+ const grpc_millis interval = xds_grpclb_duration_to_millis(
+ &initial_response->client_stats_report_interval);
+ if (interval > 0) {
+ lb_calld->client_stats_report_interval_ =
+ GPR_MAX(GPR_MS_PER_SEC, interval);
+ }
+ }
+ if (grpc_lb_xds_trace.enabled()) {
+ if (lb_calld->client_stats_report_interval_ != 0) {
gpr_log(GPR_INFO,
"[xdslb %p] Received initial LB response message; "
"client load reporting interval = %" PRId64 " milliseconds",
xdslb_policy, lb_calld->client_stats_report_interval_);
+ } else {
+ gpr_log(GPR_INFO,
+ "[xdslb %p] Received initial LB response message; client load "
+ "reporting NOT enabled",
+ xdslb_policy);
}
- } else if (grpc_lb_xds_trace.enabled()) {
- gpr_log(GPR_INFO,
- "[xdslb %p] Received initial LB response message; client load "
- "reporting NOT enabled",
- xdslb_policy);
}
xds_grpclb_initial_response_destroy(initial_response);
lb_calld->seen_initial_response_ = true;
@@ -745,7 +871,23 @@ void XdsLb::BalancerCallState::OnBalancerMessageReceivedLocked(
}
}
/* update serverlist */
+ // TODO(juanlishen): Don't ingore empty serverlist.
if (serverlist->num_servers > 0) {
+ // Pending LB channel receives a serverlist; promote it.
+ // Note that this call can't be on a discarded pending channel, because
+ // such channels don't have any current call but we have checked this call
+ // is a current call.
+ if (!lb_calld->lb_chand_->IsCurrentChannel()) {
+ if (grpc_lb_xds_trace.enabled()) {
+ gpr_log(GPR_INFO,
+ "[xdslb %p] Promoting pending LB channel %p to replace "
+ "current LB channel %p",
+ xdslb_policy, lb_calld->lb_chand_.get(),
+ lb_calld->xdslb_policy()->lb_chand_.get());
+ }
+ lb_calld->xdslb_policy()->lb_chand_ =
+ std::move(lb_calld->xdslb_policy()->pending_lb_chand_);
+ }
// Start sending client load report only after we start using the
// serverlist returned from the current LB call.
if (lb_calld->client_stats_report_interval_ > 0 &&
@@ -818,37 +960,53 @@ void XdsLb::BalancerCallState::OnBalancerMessageReceivedLocked(
}
}
-void XdsLb::BalancerCallState::OnBalancerStatusReceivedLocked(
- void* arg, grpc_error* error) {
+void XdsLb::BalancerChannelState::BalancerCallState::
+ OnBalancerStatusReceivedLocked(void* arg, grpc_error* error) {
BalancerCallState* lb_calld = static_cast<BalancerCallState*>(arg);
XdsLb* xdslb_policy = lb_calld->xdslb_policy();
+ BalancerChannelState* lb_chand = lb_calld->lb_chand_.get();
GPR_ASSERT(lb_calld->lb_call_ != nullptr);
if (grpc_lb_xds_trace.enabled()) {
char* status_details =
grpc_slice_to_c_string(lb_calld->lb_call_status_details_);
gpr_log(GPR_INFO,
"[xdslb %p] Status from LB server received. Status = %d, details "
- "= '%s', (lb_calld: %p, lb_call: %p), error '%s'",
- xdslb_policy, lb_calld->lb_call_status_, status_details, lb_calld,
- lb_calld->lb_call_, grpc_error_string(error));
+ "= '%s', (lb_chand: %p, lb_calld: %p, lb_call: %p), error '%s'",
+ xdslb_policy, lb_calld->lb_call_status_, status_details, lb_chand,
+ lb_calld, lb_calld->lb_call_, grpc_error_string(error));
gpr_free(status_details);
}
- // If this lb_calld is still in use, this call ended because of a failure so
- // we want to retry connecting. Otherwise, we have deliberately ended this
- // call and no further action is required.
- if (lb_calld == xdslb_policy->lb_calld_.get()) {
- xdslb_policy->lb_calld_.reset();
+ // Ignore status from a stale call.
+ if (lb_calld->IsCurrentCallOnChannel()) {
+ // Because this call is the current one on the channel, the channel can't
+ // have been swapped out; otherwise, the call should have been reset.
+ GPR_ASSERT(lb_chand->IsCurrentChannel() || lb_chand->IsPendingChannel());
GPR_ASSERT(!xdslb_policy->shutting_down_);
- xdslb_policy->channel_control_helper()->RequestReresolution();
- if (lb_calld->seen_initial_response_) {
- // If we lose connection to the LB server, reset the backoff and restart
- // the LB call immediately.
- xdslb_policy->lb_call_backoff_.Reset();
- xdslb_policy->StartBalancerCallLocked();
+ if (lb_chand != xdslb_policy->LatestLbChannel()) {
+ // This channel must be the current one and there is a pending one. Swap
+ // in the pending one and we are done.
+ if (grpc_lb_xds_trace.enabled()) {
+ gpr_log(GPR_INFO,
+ "[xdslb %p] Promoting pending LB channel %p to replace "
+ "current LB channel %p",
+ xdslb_policy, lb_calld->lb_chand_.get(),
+ lb_calld->xdslb_policy()->lb_chand_.get());
+ }
+ xdslb_policy->lb_chand_ = std::move(xdslb_policy->pending_lb_chand_);
} else {
- // If this LB call fails establishing any connection to the LB server,
- // retry later.
- xdslb_policy->StartBalancerCallRetryTimerLocked();
+ // This channel is the most recently created one. Try to restart the call
+ // and reresolve.
+ lb_chand->lb_calld_.reset();
+ if (lb_calld->seen_initial_response_) {
+ // If we lost connection to the LB server, reset the backoff and restart
+ // the LB call immediately.
+ lb_chand->lb_call_backoff_.Reset();
+ lb_chand->StartCallLocked();
+ } else {
+ // If we failed to connect to the LB server, retry later.
+ lb_chand->StartCallRetryTimerLocked();
+ }
+ xdslb_policy->channel_control_helper()->RequestReresolution();
}
}
lb_calld->Unref(DEBUG_LOCATION, "lb_call_ended");
@@ -858,53 +1016,23 @@ void XdsLb::BalancerCallState::OnBalancerStatusReceivedLocked(
// helper code for creating balancer channel
//
-UniquePtr<ServerAddressList> ExtractBalancerAddresses(
- const ServerAddressList& addresses) {
- auto balancer_addresses = MakeUnique<ServerAddressList>();
- for (size_t i = 0; i < addresses.size(); ++i) {
- if (addresses[i].IsBalancer()) {
- balancer_addresses->emplace_back(addresses[i]);
- }
- }
- return balancer_addresses;
-}
-
-/* Returns the channel args for the LB channel, used to create a bidirectional
- * stream for the reception of load balancing updates.
- *
- * Inputs:
- * - \a addresses: corresponding to the balancers.
- * - \a response_generator: in order to propagate updates from the resolver
- * above the grpclb policy.
- * - \a args: other args inherited from the xds policy. */
-grpc_channel_args* BuildBalancerChannelArgs(
- const ServerAddressList& addresses,
- FakeResolverResponseGenerator* response_generator,
- const grpc_channel_args* args) {
- UniquePtr<ServerAddressList> balancer_addresses =
- ExtractBalancerAddresses(addresses);
- // Channel args to remove.
+// Returns the channel args for the LB channel, used to create a bidirectional
+// stream for the reception of load balancing updates.
+grpc_channel_args* BuildBalancerChannelArgs(const grpc_channel_args* args) {
static const char* args_to_remove[] = {
// LB policy name, since we want to use the default (pick_first) in
// the LB channel.
GRPC_ARG_LB_POLICY_NAME,
+ // The service config that contains the LB config. We don't want to
+ // recursively use xds in the LB channel.
+ GRPC_ARG_SERVICE_CONFIG,
// The channel arg for the server URI, since that will be different for
// the LB channel than for the parent channel. The client channel
// factory will re-add this arg with the right value.
GRPC_ARG_SERVER_URI,
// The resolved addresses, which will be generated by the name resolver
- // used in the LB channel. Note that the LB channel will use the fake
- // resolver, so this won't actually generate a query to DNS (or some
- // other name service). However, the addresses returned by the fake
- // resolver will have is_balancer=false, whereas our own addresses have
- // is_balancer=true. We need the LB channel to return addresses with
- // is_balancer=false so that it does not wind up recursively using the
- // xds LB policy, as per the special case logic in client_channel.c.
+ // used in the LB channel.
GRPC_ARG_SERVER_ADDRESS_LIST,
- // The fake resolver response generator, because we are replacing it
- // with the one from the xds policy, used to propagate updates to
- // the LB channel.
- GRPC_ARG_FAKE_RESOLVER_RESPONSE_GENERATOR,
// The LB channel should use the authority indicated by the target
// authority table (see \a grpc_lb_policy_xds_modify_lb_channel_args),
// as opposed to the authority from the parent channel.
@@ -916,14 +1044,6 @@ grpc_channel_args* BuildBalancerChannelArgs(
};
// Channel args to add.
const grpc_arg args_to_add[] = {
- // New server address list.
- // Note that we pass these in both when creating the LB channel
- // and via the fake resolver. The latter is what actually gets used.
- CreateServerAddressListChannelArg(balancer_addresses.get()),
- // The fake resolver response generator, which we use to inject
- // address updates into the LB channel.
- grpc_core::FakeResolverResponseGenerator::MakeChannelArg(
- response_generator),
// A channel arg indicating the target is a xds load balancer.
grpc_channel_arg_integer_create(
const_cast<char*>(GRPC_ARG_ADDRESS_IS_XDS_LOAD_BALANCER), 1),
@@ -944,21 +1064,8 @@ grpc_channel_args* BuildBalancerChannelArgs(
// ctor and dtor
//
-XdsLb::XdsLb(Args args)
- : LoadBalancingPolicy(std::move(args)),
- response_generator_(MakeRefCounted<FakeResolverResponseGenerator>()),
- lb_call_backoff_(
- BackOff::Options()
- .set_initial_backoff(GRPC_XDS_INITIAL_CONNECT_BACKOFF_SECONDS *
- 1000)
- .set_multiplier(GRPC_XDS_RECONNECT_BACKOFF_MULTIPLIER)
- .set_jitter(GRPC_XDS_RECONNECT_JITTER)
- .set_max_backoff(GRPC_XDS_RECONNECT_MAX_BACKOFF_SECONDS * 1000)) {
- // Initialization.
- gpr_mu_init(&lb_channel_mu_);
- GRPC_CLOSURE_INIT(&lb_channel_on_connectivity_changed_,
- &XdsLb::OnBalancerChannelConnectivityChangedLocked, this,
- grpc_combiner_scheduler(args.combiner));
+XdsLb::XdsLb(Args args) : LoadBalancingPolicy(std::move(args)) {
+ gpr_mu_init(&lb_chand_mu_);
// Record server name.
const grpc_arg* arg = grpc_channel_args_find(args.args, GRPC_ARG_SERVER_URI);
const char* server_uri = grpc_channel_arg_get_string(arg);
@@ -982,7 +1089,7 @@ XdsLb::XdsLb(Args args)
}
XdsLb::~XdsLb() {
- gpr_mu_destroy(&lb_channel_mu_);
+ gpr_mu_destroy(&lb_chand_mu_);
gpr_free((void*)server_name_);
grpc_channel_args_destroy(args_);
if (serverlist_ != nullptr) {
@@ -992,10 +1099,6 @@ XdsLb::~XdsLb() {
void XdsLb::ShutdownLocked() {
shutting_down_ = true;
- lb_calld_.reset();
- if (retry_timer_callback_pending_) {
- grpc_timer_cancel(&lb_call_retry_timer_);
- }
if (fallback_timer_callback_pending_) {
grpc_timer_cancel(&lb_fallback_timer_);
}
@@ -1004,11 +1107,10 @@ void XdsLb::ShutdownLocked() {
// destroying the channel triggers a last callback to
// OnBalancerChannelConnectivityChangedLocked(), and we need to be
// alive when that callback is invoked.
- if (lb_channel_ != nullptr) {
- gpr_mu_lock(&lb_channel_mu_);
- grpc_channel_destroy(lb_channel_);
- lb_channel_ = nullptr;
- gpr_mu_unlock(&lb_channel_mu_);
+ {
+ MutexLock lock(&lb_chand_mu_);
+ lb_chand_.reset();
+ pending_lb_chand_.reset();
}
}
@@ -1017,8 +1119,11 @@ void XdsLb::ShutdownLocked() {
//
void XdsLb::ResetBackoffLocked() {
- if (lb_channel_ != nullptr) {
- grpc_channel_reset_connect_backoff(lb_channel_);
+ if (lb_chand_ != nullptr) {
+ grpc_channel_reset_connect_backoff(lb_chand_->channel());
+ }
+ if (pending_lb_chand_ != nullptr) {
+ grpc_channel_reset_connect_backoff(pending_lb_chand_->channel());
}
if (child_policy_ != nullptr) {
child_policy_->ResetBackoffLocked();
@@ -1027,12 +1132,19 @@ void XdsLb::ResetBackoffLocked() {
void XdsLb::FillChildRefsForChannelz(channelz::ChildRefsList* child_subchannels,
channelz::ChildRefsList* child_channels) {
- // delegate to the child_policy_ to fill the children subchannels.
+ // Delegate to the child_policy_ to fill the children subchannels.
child_policy_->FillChildRefsForChannelz(child_subchannels, child_channels);
- MutexLock lock(&lb_channel_mu_);
- if (lb_channel_ != nullptr) {
+ MutexLock lock(&lb_chand_mu_);
+ if (lb_chand_ != nullptr) {
grpc_core::channelz::ChannelNode* channel_node =
- grpc_channel_get_channelz_node(lb_channel_);
+ grpc_channel_get_channelz_node(lb_chand_->channel());
+ if (channel_node != nullptr) {
+ child_channels->push_back(channel_node->uuid());
+ }
+ }
+ if (pending_lb_chand_ != nullptr) {
+ grpc_core::channelz::ChannelNode* channel_node =
+ grpc_channel_get_channelz_node(pending_lb_chand_->channel());
if (channel_node != nullptr) {
child_channels->push_back(channel_node->uuid());
}
@@ -1059,22 +1171,29 @@ void XdsLb::ProcessChannelArgsLocked(const grpc_channel_args& args) {
args_ = grpc_channel_args_copy_and_add_and_remove(
&args, args_to_remove, GPR_ARRAY_SIZE(args_to_remove), &new_arg, 1);
// Construct args for balancer channel.
- grpc_channel_args* lb_channel_args =
- BuildBalancerChannelArgs(*addresses, response_generator_.get(), &args);
- // Create balancer channel if needed.
- if (lb_channel_ == nullptr) {
- char* uri_str;
- gpr_asprintf(&uri_str, "fake:///%s", server_name_);
- gpr_mu_lock(&lb_channel_mu_);
- lb_channel_ =
- channel_control_helper()->CreateChannel(uri_str, *lb_channel_args);
- gpr_mu_unlock(&lb_channel_mu_);
- GPR_ASSERT(lb_channel_ != nullptr);
- gpr_free(uri_str);
+ grpc_channel_args* lb_channel_args = BuildBalancerChannelArgs(&args);
+ // Create an LB channel if we don't have one yet or the balancer name has
+ // changed from the last received one.
+ bool create_lb_channel = lb_chand_ == nullptr;
+ if (lb_chand_ != nullptr) {
+ UniquePtr<char> last_balancer_name(
+ grpc_channel_get_target(LatestLbChannel()->channel()));
+ create_lb_channel =
+ strcmp(last_balancer_name.get(), balancer_name_.get()) != 0;
+ }
+ if (create_lb_channel) {
+ OrphanablePtr<BalancerChannelState> lb_chand =
+ MakeOrphanable<BalancerChannelState>(balancer_name_.get(),
+ *lb_channel_args, Ref());
+ if (lb_chand_ == nullptr || !lb_chand_->HasActiveCall()) {
+ GPR_ASSERT(pending_lb_chand_ == nullptr);
+ // If we do not have a working LB channel yet, use the newly created one.
+ lb_chand_ = std::move(lb_chand);
+ } else {
+ // Otherwise, wait until the new LB channel to be ready to swap it in.
+ pending_lb_chand_ = std::move(lb_chand);
+ }
}
- // Propagate updates to the LB channel (pick_first) through the fake
- // resolver.
- response_generator_->SetResponse(lb_channel_args);
grpc_channel_args_destroy(lb_channel_args);
}
@@ -1114,12 +1233,13 @@ void XdsLb::ParseLbConfig(Config* xds_config) {
void XdsLb::UpdateLocked(const grpc_channel_args& args,
RefCountedPtr<Config> lb_config) {
- const bool is_initial_update = lb_channel_ == nullptr;
+ const bool is_initial_update = lb_chand_ == nullptr;
ParseLbConfig(lb_config.get());
// TODO(juanlishen): Pass fallback policy config update after fallback policy
// is added.
if (balancer_name_ == nullptr) {
gpr_log(GPR_ERROR, "[xdslb %p] LB config parsing fails.", this);
+ return;
}
ProcessChannelArgsLocked(args);
// Update the existing child policy.
@@ -1139,24 +1259,6 @@ void XdsLb::UpdateLocked(const grpc_channel_args& args,
fallback_timer_callback_pending_ = true;
grpc_timer_init(&lb_fallback_timer_, deadline, &lb_on_fallback_);
}
- StartBalancerCallLocked();
- } else if (!watching_lb_channel_) {
- // If this is not the initial update and we're not already watching
- // the LB channel's connectivity state, start a watch now. This
- // ensures that we'll know when to switch to a new balancer call.
- lb_channel_connectivity_ = grpc_channel_check_connectivity_state(
- lb_channel_, true /* try to connect */);
- grpc_channel_element* client_channel_elem = grpc_channel_stack_last_element(
- grpc_channel_get_channel_stack(lb_channel_));
- GPR_ASSERT(client_channel_elem->filter == &grpc_client_channel_filter);
- watching_lb_channel_ = true;
- // Ref held by closure.
- Ref(DEBUG_LOCATION, "watch_lb_channel_connectivity").release();
- grpc_client_channel_watch_connectivity_state(
- client_channel_elem,
- grpc_polling_entity_create_from_pollset_set(interested_parties()),
- &lb_channel_connectivity_, &lb_channel_on_connectivity_changed_,
- nullptr);
}
}
@@ -1164,20 +1266,6 @@ void XdsLb::UpdateLocked(const grpc_channel_args& args,
// code for balancer channel and call
//
-void XdsLb::StartBalancerCallLocked() {
- GPR_ASSERT(lb_channel_ != nullptr);
- if (shutting_down_) return;
- // Init the LB call data.
- GPR_ASSERT(lb_calld_ == nullptr);
- lb_calld_ = MakeOrphanable<BalancerCallState>(Ref());
- if (grpc_lb_xds_trace.enabled()) {
- gpr_log(GPR_INFO,
- "[xdslb %p] Query for backends (lb_channel: %p, lb_calld: %p)",
- this, lb_channel_, lb_calld_.get());
- }
- lb_calld_->StartQuery();
-}
-
void XdsLb::OnFallbackTimerLocked(void* arg, grpc_error* error) {
XdsLb* xdslb_policy = static_cast<XdsLb*>(arg);
xdslb_policy->fallback_timer_callback_pending_ = false;
@@ -1194,88 +1282,6 @@ void XdsLb::OnFallbackTimerLocked(void* arg, grpc_error* error) {
xdslb_policy->Unref(DEBUG_LOCATION, "on_fallback_timer");
}
-void XdsLb::StartBalancerCallRetryTimerLocked() {
- grpc_millis next_try = lb_call_backoff_.NextAttemptTime();
- if (grpc_lb_xds_trace.enabled()) {
- gpr_log(GPR_INFO, "[xdslb %p] Connection to LB server lost...", this);
- grpc_millis timeout = next_try - ExecCtx::Get()->Now();
- if (timeout > 0) {
- gpr_log(GPR_INFO, "[xdslb %p] ... retry_timer_active in %" PRId64 "ms.",
- this, timeout);
- } else {
- gpr_log(GPR_INFO, "[xdslb %p] ... retry_timer_active immediately.", this);
- }
- }
- // TODO(roth): We currently track this ref manually. Once the
- // ClosureRef API is ready, we should pass the RefCountedPtr<> along
- // with the callback.
- auto self = Ref(DEBUG_LOCATION, "on_balancer_call_retry_timer");
- self.release();
- GRPC_CLOSURE_INIT(&lb_on_call_retry_, &XdsLb::OnBalancerCallRetryTimerLocked,
- this, grpc_combiner_scheduler(combiner()));
- retry_timer_callback_pending_ = true;
- grpc_timer_init(&lb_call_retry_timer_, next_try, &lb_on_call_retry_);
-}
-
-void XdsLb::OnBalancerCallRetryTimerLocked(void* arg, grpc_error* error) {
- XdsLb* xdslb_policy = static_cast<XdsLb*>(arg);
- xdslb_policy->retry_timer_callback_pending_ = false;
- if (!xdslb_policy->shutting_down_ && error == GRPC_ERROR_NONE &&
- xdslb_policy->lb_calld_ == nullptr) {
- if (grpc_lb_xds_trace.enabled()) {
- gpr_log(GPR_INFO, "[xdslb %p] Restarting call to LB server",
- xdslb_policy);
- }
- xdslb_policy->StartBalancerCallLocked();
- }
- xdslb_policy->Unref(DEBUG_LOCATION, "on_balancer_call_retry_timer");
-}
-
-// Invoked as part of the update process. It continues watching the LB channel
-// until it shuts down or becomes READY. It's invoked even if the LB channel
-// stayed READY throughout the update (for example if the update is identical).
-void XdsLb::OnBalancerChannelConnectivityChangedLocked(void* arg,
- grpc_error* error) {
- XdsLb* xdslb_policy = static_cast<XdsLb*>(arg);
- if (xdslb_policy->shutting_down_) goto done;
- // Re-initialize the lb_call. This should also take care of updating the
- // child policy. Note that the current child policy, if any, will
- // stay in effect until an update from the new lb_call is received.
- switch (xdslb_policy->lb_channel_connectivity_) {
- case GRPC_CHANNEL_CONNECTING:
- case GRPC_CHANNEL_TRANSIENT_FAILURE: {
- // Keep watching the LB channel.
- grpc_channel_element* client_channel_elem =
- grpc_channel_stack_last_element(
- grpc_channel_get_channel_stack(xdslb_policy->lb_channel_));
- GPR_ASSERT(client_channel_elem->filter == &grpc_client_channel_filter);
- grpc_client_channel_watch_connectivity_state(
- client_channel_elem,
- grpc_polling_entity_create_from_pollset_set(
- xdslb_policy->interested_parties()),
- &xdslb_policy->lb_channel_connectivity_,
- &xdslb_policy->lb_channel_on_connectivity_changed_, nullptr);
- break;
- }
- // The LB channel may be IDLE because it's shut down before the update.
- // Restart the LB call to kick the LB channel into gear.
- case GRPC_CHANNEL_IDLE:
- case GRPC_CHANNEL_READY:
- xdslb_policy->lb_calld_.reset();
- if (xdslb_policy->retry_timer_callback_pending_) {
- grpc_timer_cancel(&xdslb_policy->lb_call_retry_timer_);
- }
- xdslb_policy->lb_call_backoff_.Reset();
- xdslb_policy->StartBalancerCallLocked();
- // Fall through.
- case GRPC_CHANNEL_SHUTDOWN:
- done:
- xdslb_policy->watching_lb_channel_ = false;
- xdslb_policy->Unref(DEBUG_LOCATION,
- "watch_lb_channel_connectivity_cb_shutdown");
- }
-}
-
//
// code for interacting with the child policy
//
@@ -1360,18 +1366,6 @@ class XdsFactory : public LoadBalancingPolicyFactory {
public:
OrphanablePtr<LoadBalancingPolicy> CreateLoadBalancingPolicy(
LoadBalancingPolicy::Args args) const override {
- /* Count the number of gRPC-LB addresses. There must be at least one. */
- const ServerAddressList* addresses =
- FindServerAddressListChannelArg(args.args);
- if (addresses == nullptr) return nullptr;
- bool found_balancer_address = false;
- for (size_t i = 0; i < addresses->size(); ++i) {
- if ((*addresses)[i].IsBalancer()) {
- found_balancer_address = true;
- break;
- }
- }
- if (!found_balancer_address) return nullptr;
return OrphanablePtr<LoadBalancingPolicy>(New<XdsLb>(std::move(args)));
}
diff --git a/src/core/ext/filters/client_channel/lb_policy/xds/xds_channel_secure.cc b/src/core/ext/filters/client_channel/lb_policy/xds/xds_channel_secure.cc
index 55c646e6eed..7f8c232d6d0 100644
--- a/src/core/ext/filters/client_channel/lb_policy/xds/xds_channel_secure.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/xds/xds_channel_secure.cc
@@ -33,55 +33,12 @@
#include "src/core/lib/security/transport/target_authority_table.h"
#include "src/core/lib/slice/slice_internal.h"
-namespace grpc_core {
-namespace {
-
-int BalancerNameCmp(const grpc_core::UniquePtr<char>& a,
- const grpc_core::UniquePtr<char>& b) {
- return strcmp(a.get(), b.get());
-}
-
-RefCountedPtr<TargetAuthorityTable> CreateTargetAuthorityTable(
- const ServerAddressList& addresses) {
- TargetAuthorityTable::Entry* target_authority_entries =
- static_cast<TargetAuthorityTable::Entry*>(
- gpr_zalloc(sizeof(*target_authority_entries) * addresses.size()));
- for (size_t i = 0; i < addresses.size(); ++i) {
- char* addr_str;
- GPR_ASSERT(
- grpc_sockaddr_to_string(&addr_str, &addresses[i].address(), true) > 0);
- target_authority_entries[i].key = grpc_slice_from_copied_string(addr_str);
- gpr_free(addr_str);
- char* balancer_name = grpc_channel_arg_get_string(grpc_channel_args_find(
- addresses[i].args(), GRPC_ARG_ADDRESS_BALANCER_NAME));
- target_authority_entries[i].value.reset(gpr_strdup(balancer_name));
- }
- RefCountedPtr<TargetAuthorityTable> target_authority_table =
- TargetAuthorityTable::Create(addresses.size(), target_authority_entries,
- BalancerNameCmp);
- gpr_free(target_authority_entries);
- return target_authority_table;
-}
-
-} // namespace
-} // namespace grpc_core
-
grpc_channel_args* grpc_lb_policy_xds_modify_lb_channel_args(
grpc_channel_args* args) {
const char* args_to_remove[1];
size_t num_args_to_remove = 0;
grpc_arg args_to_add[2];
size_t num_args_to_add = 0;
- // Add arg for targets info table.
- grpc_core::ServerAddressList* addresses =
- grpc_core::FindServerAddressListChannelArg(args);
- GPR_ASSERT(addresses != nullptr);
- grpc_core::RefCountedPtr<grpc_core::TargetAuthorityTable>
- target_authority_table =
- grpc_core::CreateTargetAuthorityTable(*addresses);
- args_to_add[num_args_to_add++] =
- grpc_core::CreateTargetAuthorityTableChannelArg(
- target_authority_table.get());
// Substitute the channel credentials with a version without call
// credentials: the load balancer is not necessarily trusted to handle
// bearer token credentials.
diff --git a/src/core/ext/filters/client_channel/resolver/fake/fake_resolver.cc b/src/core/ext/filters/client_channel/resolver/fake/fake_resolver.cc
index 258339491c1..3489f3d491b 100644
--- a/src/core/ext/filters/client_channel/resolver/fake/fake_resolver.cc
+++ b/src/core/ext/filters/client_channel/resolver/fake/fake_resolver.cc
@@ -86,7 +86,14 @@ FakeResolver::FakeResolver(const ResolverArgs& args) : Resolver(args.combiner) {
channel_args_ = grpc_channel_args_copy(args.args);
FakeResolverResponseGenerator* response_generator =
FakeResolverResponseGenerator::GetFromArgs(args.args);
- if (response_generator != nullptr) response_generator->resolver_ = this;
+ if (response_generator != nullptr) {
+ response_generator->resolver_ = this;
+ if (response_generator->response_ != nullptr) {
+ response_generator->SetResponse(response_generator->response_);
+ grpc_channel_args_destroy(response_generator->response_);
+ response_generator->response_ = nullptr;
+ }
+ }
}
FakeResolver::~FakeResolver() {
@@ -114,6 +121,9 @@ void FakeResolver::RequestReresolutionLocked() {
void FakeResolver::MaybeFinishNextLocked() {
if (next_completion_ != nullptr &&
(next_results_ != nullptr || return_failure_)) {
+ // When both next_results_ and channel_args_ contain an arg with the same
+ // name, only the one in next_results_ will be kept since next_results_ is
+ // before channel_args_.
*target_result_ =
return_failure_ ? nullptr
: grpc_channel_args_union(next_results_, channel_args_);
@@ -157,15 +167,19 @@ void FakeResolverResponseGenerator::SetResponseLocked(void* arg,
void FakeResolverResponseGenerator::SetResponse(grpc_channel_args* response) {
GPR_ASSERT(response != nullptr);
- GPR_ASSERT(resolver_ != nullptr);
- SetResponseClosureArg* closure_arg = New<SetResponseClosureArg>();
- closure_arg->generator = this;
- closure_arg->response = grpc_channel_args_copy(response);
- GRPC_CLOSURE_SCHED(
- GRPC_CLOSURE_INIT(&closure_arg->set_response_closure, SetResponseLocked,
- closure_arg,
- grpc_combiner_scheduler(resolver_->combiner())),
- GRPC_ERROR_NONE);
+ if (resolver_ != nullptr) {
+ SetResponseClosureArg* closure_arg = New<SetResponseClosureArg>();
+ closure_arg->generator = this;
+ closure_arg->response = grpc_channel_args_copy(response);
+ GRPC_CLOSURE_SCHED(
+ GRPC_CLOSURE_INIT(&closure_arg->set_response_closure, SetResponseLocked,
+ closure_arg,
+ grpc_combiner_scheduler(resolver_->combiner())),
+ GRPC_ERROR_NONE);
+ } else {
+ GPR_ASSERT(response_ == nullptr);
+ response_ = grpc_channel_args_copy(response);
+ }
}
void FakeResolverResponseGenerator::SetReresolutionResponseLocked(
diff --git a/src/core/ext/filters/client_channel/resolver/fake/fake_resolver.h b/src/core/ext/filters/client_channel/resolver/fake/fake_resolver.h
index d86111c3829..f423e6d46db 100644
--- a/src/core/ext/filters/client_channel/resolver/fake/fake_resolver.h
+++ b/src/core/ext/filters/client_channel/resolver/fake/fake_resolver.h
@@ -44,7 +44,9 @@ class FakeResolverResponseGenerator
FakeResolverResponseGenerator() {}
// Instructs the fake resolver associated with the response generator
- // instance to trigger a new resolution with the specified response.
+ // instance to trigger a new resolution with the specified response. If the
+ // resolver is not available yet, delays response setting until it is. This
+ // can be called at most once before the resolver is available.
void SetResponse(grpc_channel_args* next_response);
// Sets the re-resolution response, which is returned by the fake resolver
@@ -79,6 +81,7 @@ class FakeResolverResponseGenerator
static void SetFailureLocked(void* arg, grpc_error* error);
FakeResolver* resolver_ = nullptr; // Do not own.
+ grpc_channel_args* response_ = nullptr;
};
} // namespace grpc_core
diff --git a/src/core/lib/security/security_connector/fake/fake_security_connector.cc b/src/core/lib/security/security_connector/fake/fake_security_connector.cc
index a0e2e6f030b..c55fd34d0e2 100644
--- a/src/core/lib/security/security_connector/fake/fake_security_connector.cc
+++ b/src/core/lib/security/security_connector/fake/fake_security_connector.cc
@@ -26,6 +26,8 @@
#include <grpc/support/log.h>
#include <grpc/support/string_util.h>
+#include "src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.h"
+#include "src/core/ext/filters/client_channel/lb_policy/xds/xds.h"
#include "src/core/ext/transport/chttp2/alpn/alpn.h"
#include "src/core/lib/channel/channel_args.h"
#include "src/core/lib/channel/handshaker.h"
@@ -53,8 +55,11 @@ class grpc_fake_channel_security_connector final
target_(gpr_strdup(target)),
expected_targets_(
gpr_strdup(grpc_fake_transport_get_expected_targets(args))),
- is_lb_channel_(grpc_core::FindTargetAuthorityTableInArgs(args) !=
- nullptr) {
+ is_lb_channel_(
+ grpc_channel_args_find(
+ args, GRPC_ARG_ADDRESS_IS_XDS_LOAD_BALANCER) != nullptr ||
+ grpc_channel_args_find(
+ args, GRPC_ARG_ADDRESS_IS_GRPCLB_LOAD_BALANCER) != nullptr) {
const grpc_arg* target_name_override_arg =
grpc_channel_args_find(args, GRPC_SSL_TARGET_NAME_OVERRIDE_ARG);
if (target_name_override_arg != nullptr) {
diff --git a/test/cpp/end2end/BUILD b/test/cpp/end2end/BUILD
index d80fa33a83a..43dee177e7a 100644
--- a/test/cpp/end2end/BUILD
+++ b/test/cpp/end2end/BUILD
@@ -439,6 +439,28 @@ grpc_cc_test(
],
)
+grpc_cc_test(
+ name = "xds_end2end_test",
+ srcs = ["xds_end2end_test.cc"],
+ external_deps = [
+ "gmock",
+ "gtest",
+ ],
+ deps = [
+ ":test_service_impl",
+ "//:gpr",
+ "//:grpc",
+ "//:grpc++",
+ "//:grpc_resolver_fake",
+ "//src/proto/grpc/lb/v1:load_balancer_proto",
+ "//src/proto/grpc/testing:echo_messages_proto",
+ "//src/proto/grpc/testing:echo_proto",
+ "//src/proto/grpc/testing/duplicate:echo_duplicate_proto",
+ "//test/core/util:grpc_test_util",
+ "//test/cpp/util:test_util",
+ ],
+)
+
grpc_cc_test(
name = "proto_server_reflection_test",
srcs = ["proto_server_reflection_test.cc"],
diff --git a/test/cpp/end2end/xds_end2end_test.cc b/test/cpp/end2end/xds_end2end_test.cc
new file mode 100644
index 00000000000..09556675d43
--- /dev/null
+++ b/test/cpp/end2end/xds_end2end_test.cc
@@ -0,0 +1,1214 @@
+/*
+ *
+ * Copyright 2017 gRPC authors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+#include <memory>
+#include <mutex>
+#include <set>
+#include <sstream>
+#include <thread>
+
+#include <grpc/grpc.h>
+#include <grpc/support/alloc.h>
+#include <grpc/support/log.h>
+#include <grpc/support/string_util.h>
+#include <grpc/support/time.h>
+#include <grpcpp/channel.h>
+#include <grpcpp/client_context.h>
+#include <grpcpp/create_channel.h>
+#include <grpcpp/server.h>
+#include <grpcpp/server_builder.h>
+
+#include "src/core/ext/filters/client_channel/parse_address.h"
+#include "src/core/ext/filters/client_channel/resolver/fake/fake_resolver.h"
+#include "src/core/ext/filters/client_channel/server_address.h"
+#include "src/core/lib/gpr/env.h"
+#include "src/core/lib/gprpp/ref_counted_ptr.h"
+#include "src/core/lib/iomgr/sockaddr.h"
+#include "src/core/lib/security/credentials/fake/fake_credentials.h"
+#include "src/cpp/client/secure_credentials.h"
+#include "src/cpp/server/secure_server_credentials.h"
+
+#include "test/core/util/port.h"
+#include "test/core/util/test_config.h"
+#include "test/cpp/end2end/test_service_impl.h"
+
+#include "src/proto/grpc/lb/v1/load_balancer.grpc.pb.h"
+#include "src/proto/grpc/testing/echo.grpc.pb.h"
+
+#include <gmock/gmock.h>
+#include <gtest/gtest.h>
+
+// TODO(dgq): Other scenarios in need of testing:
+// - Send a serverlist with faulty ip:port addresses (port > 2^16, etc).
+// - Test reception of invalid serverlist
+// - Test pinging
+// - Test against a non-LB server.
+// - Random LB server closing the stream unexpectedly.
+// - Test using DNS-resolvable names (localhost?)
+// - Test handling of creation of faulty RR instance by having the LB return a
+// serverlist with non-existent backends after having initially returned a
+// valid one.
+//
+// Findings from end to end testing to be covered here:
+// - Handling of LB servers restart, including reconnection after backing-off
+// retries.
+// - Destruction of load balanced channel (and therefore of xds instance)
+// while:
+// 1) the internal LB call is still active. This should work by virtue
+// of the weak reference the LB call holds. The call should be terminated as
+// part of the xds shutdown process.
+// 2) the retry timer is active. Again, the weak reference it holds should
+// prevent a premature call to \a glb_destroy.
+// - Restart of backend servers with no changes to serverlist. This exercises
+// the RR handover mechanism.
+
+using std::chrono::system_clock;
+
+using grpc::lb::v1::LoadBalanceRequest;
+using grpc::lb::v1::LoadBalanceResponse;
+using grpc::lb::v1::LoadBalancer;
+
+namespace grpc {
+namespace testing {
+namespace {
+
+template <typename ServiceType>
+class CountedService : public ServiceType {
+ public:
+ size_t request_count() {
+ std::unique_lock<std::mutex> lock(mu_);
+ return request_count_;
+ }
+
+ size_t response_count() {
+ std::unique_lock<std::mutex> lock(mu_);
+ return response_count_;
+ }
+
+ void IncreaseResponseCount() {
+ std::unique_lock<std::mutex> lock(mu_);
+ ++response_count_;
+ }
+ void IncreaseRequestCount() {
+ std::unique_lock<std::mutex> lock(mu_);
+ ++request_count_;
+ }
+
+ void ResetCounters() {
+ std::unique_lock<std::mutex> lock(mu_);
+ request_count_ = 0;
+ response_count_ = 0;
+ }
+
+ protected:
+ std::mutex mu_;
+
+ private:
+ size_t request_count_ = 0;
+ size_t response_count_ = 0;
+};
+
+using BackendService = CountedService<TestServiceImpl>;
+using BalancerService = CountedService<LoadBalancer::Service>;
+
+const char g_kCallCredsMdKey[] = "Balancer should not ...";
+const char g_kCallCredsMdValue[] = "... receive me";
+
+class BackendServiceImpl : public BackendService {
+ public:
+ BackendServiceImpl() {}
+
+ Status Echo(ServerContext* context, const EchoRequest* request,
+ EchoResponse* response) override {
+ // Backend should receive the call credentials metadata.
+ auto call_credentials_entry =
+ context->client_metadata().find(g_kCallCredsMdKey);
+ EXPECT_NE(call_credentials_entry, context->client_metadata().end());
+ if (call_credentials_entry != context->client_metadata().end()) {
+ EXPECT_EQ(call_credentials_entry->second, g_kCallCredsMdValue);
+ }
+ IncreaseRequestCount();
+ const auto status = TestServiceImpl::Echo(context, request, response);
+ IncreaseResponseCount();
+ AddClient(context->peer());
+ return status;
+ }
+
+ // Returns true on its first invocation, false otherwise.
+ bool Shutdown() {
+ std::unique_lock<std::mutex> lock(mu_);
+ const bool prev = !shutdown_;
+ shutdown_ = true;
+ gpr_log(GPR_INFO, "Backend: shut down");
+ return prev;
+ }
+
+ std::set<grpc::string> clients() {
+ std::unique_lock<std::mutex> lock(clients_mu_);
+ return clients_;
+ }
+
+ private:
+ void AddClient(const grpc::string& client) {
+ std::unique_lock<std::mutex> lock(clients_mu_);
+ clients_.insert(client);
+ }
+
+ std::mutex mu_;
+ bool shutdown_ = false;
+ std::mutex clients_mu_;
+ std::set<grpc::string> clients_;
+};
+
+grpc::string Ip4ToPackedString(const char* ip_str) {
+ struct in_addr ip4;
+ GPR_ASSERT(inet_pton(AF_INET, ip_str, &ip4) == 1);
+ return grpc::string(reinterpret_cast<const char*>(&ip4), sizeof(ip4));
+}
+
+struct ClientStats {
+ size_t num_calls_started = 0;
+ size_t num_calls_finished = 0;
+ size_t num_calls_finished_with_client_failed_to_send = 0;
+ size_t num_calls_finished_known_received = 0;
+ std::map<grpc::string, size_t> drop_token_counts;
+
+ ClientStats& operator+=(const ClientStats& other) {
+ num_calls_started += other.num_calls_started;
+ num_calls_finished += other.num_calls_finished;
+ num_calls_finished_with_client_failed_to_send +=
+ other.num_calls_finished_with_client_failed_to_send;
+ num_calls_finished_known_received +=
+ other.num_calls_finished_known_received;
+ for (const auto& p : other.drop_token_counts) {
+ drop_token_counts[p.first] += p.second;
+ }
+ return *this;
+ }
+};
+
+class BalancerServiceImpl : public BalancerService {
+ public:
+ using Stream = ServerReaderWriter<LoadBalanceResponse, LoadBalanceRequest>;
+ using ResponseDelayPair = std::pair<LoadBalanceResponse, int>;
+
+ explicit BalancerServiceImpl(int client_load_reporting_interval_seconds)
+ : client_load_reporting_interval_seconds_(
+ client_load_reporting_interval_seconds),
+ shutdown_(false) {}
+
+ Status BalanceLoad(ServerContext* context, Stream* stream) override {
+ // TODO(juanlishen): Clean up the scoping.
+ gpr_log(GPR_INFO, "LB[%p]: BalanceLoad", this);
+ {
+ std::unique_lock<std::mutex> lock(mu_);
+ if (shutdown_) goto done;
+ }
+
+ {
+ // Balancer shouldn't receive the call credentials metadata.
+ EXPECT_EQ(context->client_metadata().find(g_kCallCredsMdKey),
+ context->client_metadata().end());
+ LoadBalanceRequest request;
+ std::vector<ResponseDelayPair> responses_and_delays;
+
+ if (!stream->Read(&request)) {
+ goto done;
+ }
+ IncreaseRequestCount();
+ gpr_log(GPR_INFO, "LB[%p]: received initial message '%s'", this,
+ request.DebugString().c_str());
+
+ {
+ LoadBalanceResponse initial_response;
+ initial_response.mutable_initial_response()
+ ->mutable_client_stats_report_interval()
+ ->set_seconds(client_load_reporting_interval_seconds_);
+ stream->Write(initial_response);
+ }
+
+ {
+ std::unique_lock<std::mutex> lock(mu_);
+ responses_and_delays = responses_and_delays_;
+ }
+ for (const auto& response_and_delay : responses_and_delays) {
+ {
+ std::unique_lock<std::mutex> lock(mu_);
+ if (shutdown_) goto done;
+ }
+ SendResponse(stream, response_and_delay.first,
+ response_and_delay.second);
+ }
+ {
+ std::unique_lock<std::mutex> lock(mu_);
+ if (shutdown_) goto done;
+ serverlist_cond_.wait(lock, [this] { return serverlist_ready_; });
+ }
+
+ if (client_load_reporting_interval_seconds_ > 0) {
+ request.Clear();
+ if (stream->Read(&request)) {
+ gpr_log(GPR_INFO, "LB[%p]: received client load report message '%s'",
+ this, request.DebugString().c_str());
+ GPR_ASSERT(request.has_client_stats());
+ // We need to acquire the lock here in order to prevent the notify_one
+ // below from firing before its corresponding wait is executed.
+ std::lock_guard<std::mutex> lock(mu_);
+ client_stats_.num_calls_started +=
+ request.client_stats().num_calls_started();
+ client_stats_.num_calls_finished +=
+ request.client_stats().num_calls_finished();
+ client_stats_.num_calls_finished_with_client_failed_to_send +=
+ request.client_stats()
+ .num_calls_finished_with_client_failed_to_send();
+ client_stats_.num_calls_finished_known_received +=
+ request.client_stats().num_calls_finished_known_received();
+ for (const auto& drop_token_count :
+ request.client_stats().calls_finished_with_drop()) {
+ client_stats_
+ .drop_token_counts[drop_token_count.load_balance_token()] +=
+ drop_token_count.num_calls();
+ }
+ load_report_ready_ = true;
+ load_report_cond_.notify_one();
+ }
+ }
+ }
+ done:
+ gpr_log(GPR_INFO, "LB[%p]: done", this);
+ return Status::OK;
+ }
+
+ void add_response(const LoadBalanceResponse& response, int send_after_ms) {
+ std::unique_lock<std::mutex> lock(mu_);
+ responses_and_delays_.push_back(std::make_pair(response, send_after_ms));
+ }
+
+ // Returns true on its first invocation, false otherwise.
+ bool Shutdown() {
+ bool prev;
+ {
+ std::unique_lock<std::mutex> lock(mu_);
+ prev = !shutdown_;
+ shutdown_ = true;
+ }
+ NotifyDoneWithServerlists();
+ gpr_log(GPR_INFO, "LB[%p]: shut down", this);
+ return prev;
+ }
+
+ static LoadBalanceResponse BuildResponseForBackends(
+ const std::vector<int>& backend_ports,
+ const std::map<grpc::string, size_t>& drop_token_counts) {
+ LoadBalanceResponse response;
+ for (const auto& drop_token_count : drop_token_counts) {
+ for (size_t i = 0; i < drop_token_count.second; ++i) {
+ auto* server = response.mutable_server_list()->add_servers();
+ server->set_drop(true);
+ server->set_load_balance_token(drop_token_count.first);
+ }
+ }
+ for (const int& backend_port : backend_ports) {
+ auto* server = response.mutable_server_list()->add_servers();
+ server->set_ip_address(Ip4ToPackedString("127.0.0.1"));
+ server->set_port(backend_port);
+ static int token_count = 0;
+ char* token;
+ gpr_asprintf(&token, "token%03d", ++token_count);
+ server->set_load_balance_token(token);
+ gpr_free(token);
+ }
+ return response;
+ }
+
+ const ClientStats& WaitForLoadReport() {
+ std::unique_lock<std::mutex> lock(mu_);
+ load_report_cond_.wait(lock, [this] { return load_report_ready_; });
+ load_report_ready_ = false;
+ return client_stats_;
+ }
+
+ void NotifyDoneWithServerlists() {
+ std::lock_guard<std::mutex> lock(mu_);
+ serverlist_ready_ = true;
+ serverlist_cond_.notify_all();
+ }
+
+ private:
+ void SendResponse(Stream* stream, const LoadBalanceResponse& response,
+ int delay_ms) {
+ gpr_log(GPR_INFO, "LB[%p]: sleeping for %d ms...", this, delay_ms);
+ if (delay_ms > 0) {
+ gpr_sleep_until(grpc_timeout_milliseconds_to_deadline(delay_ms));
+ }
+ gpr_log(GPR_INFO, "LB[%p]: Woke up! Sending response '%s'", this,
+ response.DebugString().c_str());
+ IncreaseResponseCount();
+ stream->Write(response);
+ }
+
+ const int client_load_reporting_interval_seconds_;
+ std::vector<ResponseDelayPair> responses_and_delays_;
+ std::mutex mu_;
+ std::condition_variable load_report_cond_;
+ bool load_report_ready_ = false;
+ std::condition_variable serverlist_cond_;
+ bool serverlist_ready_ = false;
+ ClientStats client_stats_;
+ bool shutdown_;
+};
+
+class XdsEnd2endTest : public ::testing::Test {
+ protected:
+ XdsEnd2endTest(int num_backends, int num_balancers,
+ int client_load_reporting_interval_seconds)
+ : server_host_("localhost"),
+ num_backends_(num_backends),
+ num_balancers_(num_balancers),
+ client_load_reporting_interval_seconds_(
+ client_load_reporting_interval_seconds) {
+ // Make the backup poller poll very frequently in order to pick up
+ // updates from all the subchannels's FDs.
+ gpr_setenv("GRPC_CLIENT_CHANNEL_BACKUP_POLL_INTERVAL_MS", "1");
+ }
+
+ void SetUp() override {
+ response_generator_ =
+ grpc_core::MakeRefCounted<grpc_core::FakeResolverResponseGenerator>();
+ lb_channel_response_generator_ =
+ grpc_core::MakeRefCounted<grpc_core::FakeResolverResponseGenerator>();
+ // Start the backends.
+ for (size_t i = 0; i < num_backends_; ++i) {
+ backends_.emplace_back(new BackendServiceImpl());
+ backend_servers_.emplace_back(ServerThread<BackendService>(
+ "backend", server_host_, backends_.back().get()));
+ }
+ // Start the load balancers.
+ for (size_t i = 0; i < num_balancers_; ++i) {
+ balancers_.emplace_back(
+ new BalancerServiceImpl(client_load_reporting_interval_seconds_));
+ balancer_servers_.emplace_back(ServerThread<BalancerService>(
+ "balancer", server_host_, balancers_.back().get()));
+ }
+ ResetStub();
+ }
+
+ void TearDown() override {
+ for (size_t i = 0; i < backends_.size(); ++i) {
+ if (backends_[i]->Shutdown()) backend_servers_[i].Shutdown();
+ }
+ for (size_t i = 0; i < balancers_.size(); ++i) {
+ if (balancers_[i]->Shutdown()) balancer_servers_[i].Shutdown();
+ }
+ }
+
+ void ResetStub(int fallback_timeout = 0,
+ const grpc::string& expected_targets = "") {
+ ChannelArguments args;
+ // TODO(juanlishen): Add setter to ChannelArguments.
+ args.SetInt(GRPC_ARG_XDS_FALLBACK_TIMEOUT_MS, fallback_timeout);
+ args.SetPointer(GRPC_ARG_FAKE_RESOLVER_RESPONSE_GENERATOR,
+ response_generator_.get());
+ if (!expected_targets.empty()) {
+ args.SetString(GRPC_ARG_FAKE_SECURITY_EXPECTED_TARGETS, expected_targets);
+ }
+ std::ostringstream uri;
+ uri << "fake:///" << kApplicationTargetName_;
+ // TODO(dgq): templatize tests to run everything using both secure and
+ // insecure channel credentials.
+ grpc_channel_credentials* channel_creds =
+ grpc_fake_transport_security_credentials_create();
+ grpc_call_credentials* call_creds = grpc_md_only_test_credentials_create(
+ g_kCallCredsMdKey, g_kCallCredsMdValue, false);
+ std::shared_ptr<ChannelCredentials> creds(
+ new SecureChannelCredentials(grpc_composite_channel_credentials_create(
+ channel_creds, call_creds, nullptr)));
+ call_creds->Unref();
+ channel_creds->Unref();
+ channel_ = CreateCustomChannel(uri.str(), creds, args);
+ stub_ = grpc::testing::EchoTestService::NewStub(channel_);
+ }
+
+ void ResetBackendCounters() {
+ for (const auto& backend : backends_) backend->ResetCounters();
+ }
+
+ ClientStats WaitForLoadReports() {
+ ClientStats client_stats;
+ for (const auto& balancer : balancers_) {
+ client_stats += balancer->WaitForLoadReport();
+ }
+ return client_stats;
+ }
+
+ bool SeenAllBackends() {
+ for (const auto& backend : backends_) {
+ if (backend->request_count() == 0) return false;
+ }
+ return true;
+ }
+
+ void SendRpcAndCount(int* num_total, int* num_ok, int* num_failure,
+ int* num_drops) {
+ const Status status = SendRpc();
+ if (status.ok()) {
+ ++*num_ok;
+ } else {
+ if (status.error_message() == "Call dropped by load balancing policy") {
+ ++*num_drops;
+ } else {
+ ++*num_failure;
+ }
+ }
+ ++*num_total;
+ }
+
+ std::tuple<int, int, int> WaitForAllBackends(
+ int num_requests_multiple_of = 1) {
+ int num_ok = 0;
+ int num_failure = 0;
+ int num_drops = 0;
+ int num_total = 0;
+ while (!SeenAllBackends()) {
+ SendRpcAndCount(&num_total, &num_ok, &num_failure, &num_drops);
+ }
+ while (num_total % num_requests_multiple_of != 0) {
+ SendRpcAndCount(&num_total, &num_ok, &num_failure, &num_drops);
+ }
+ ResetBackendCounters();
+ gpr_log(GPR_INFO,
+ "Performed %d warm up requests (a multiple of %d) against the "
+ "backends. %d succeeded, %d failed, %d dropped.",
+ num_total, num_requests_multiple_of, num_ok, num_failure,
+ num_drops);
+ return std::make_tuple(num_ok, num_failure, num_drops);
+ }
+
+ void WaitForBackend(size_t backend_idx) {
+ do {
+ (void)SendRpc();
+ } while (backends_[backend_idx]->request_count() == 0);
+ ResetBackendCounters();
+ }
+
+ grpc_core::ServerAddressList CreateLbAddressesFromPortList(
+ const std::vector<int>& ports) {
+ grpc_core::ServerAddressList addresses;
+ for (int port : ports) {
+ char* lb_uri_str;
+ gpr_asprintf(&lb_uri_str, "ipv4:127.0.0.1:%d", port);
+ grpc_uri* lb_uri = grpc_uri_parse(lb_uri_str, true);
+ GPR_ASSERT(lb_uri != nullptr);
+ grpc_resolved_address address;
+ GPR_ASSERT(grpc_parse_uri(lb_uri, &address));
+ std::vector<grpc_arg> args_to_add;
+ grpc_channel_args* args = grpc_channel_args_copy_and_add(
+ nullptr, args_to_add.data(), args_to_add.size());
+ addresses.emplace_back(address.addr, address.len, args);
+ grpc_uri_destroy(lb_uri);
+ gpr_free(lb_uri_str);
+ }
+ return addresses;
+ }
+
+ void SetNextResolution(const std::vector<int>& ports,
+ const char* service_config_json = nullptr,
+ grpc_core::FakeResolverResponseGenerator*
+ lb_channel_response_generator = nullptr) {
+ grpc_core::ExecCtx exec_ctx;
+ grpc_core::ServerAddressList addresses =
+ CreateLbAddressesFromPortList(ports);
+ std::vector<grpc_arg> args = {
+ CreateServerAddressListChannelArg(&addresses),
+ grpc_core::FakeResolverResponseGenerator::MakeChannelArg(
+ lb_channel_response_generator == nullptr
+ ? lb_channel_response_generator_.get()
+ : lb_channel_response_generator)};
+ if (service_config_json != nullptr) {
+ args.push_back(grpc_channel_arg_string_create(
+ const_cast<char*>(GRPC_ARG_SERVICE_CONFIG),
+ const_cast<char*>(service_config_json)));
+ }
+ grpc_channel_args fake_result = {args.size(), args.data()};
+ response_generator_->SetResponse(&fake_result);
+ }
+
+ void SetNextResolutionForLbChannelAllBalancers(
+ const char* service_config_json = nullptr,
+ grpc_core::FakeResolverResponseGenerator* lb_channel_response_generator =
+ nullptr) {
+ std::vector<int> ports;
+ for (size_t i = 0; i < balancer_servers_.size(); ++i) {
+ ports.emplace_back(balancer_servers_[i].port_);
+ }
+ SetNextResolutionForLbChannel(ports, service_config_json,
+ lb_channel_response_generator);
+ }
+
+ void SetNextResolutionForLbChannel(
+ const std::vector<int>& ports, const char* service_config_json = nullptr,
+ grpc_core::FakeResolverResponseGenerator* lb_channel_response_generator =
+ nullptr) {
+ grpc_core::ExecCtx exec_ctx;
+ grpc_core::ServerAddressList addresses =
+ CreateLbAddressesFromPortList(ports);
+ std::vector<grpc_arg> args = {
+ CreateServerAddressListChannelArg(&addresses),
+ };
+ if (service_config_json != nullptr) {
+ args.push_back(grpc_channel_arg_string_create(
+ const_cast<char*>(GRPC_ARG_SERVICE_CONFIG),
+ const_cast<char*>(service_config_json)));
+ }
+ grpc_channel_args fake_result = {args.size(), args.data()};
+ if (lb_channel_response_generator == nullptr) {
+ lb_channel_response_generator = lb_channel_response_generator_.get();
+ }
+ lb_channel_response_generator->SetResponse(&fake_result);
+ }
+
+ void SetNextReresolutionResponse(const std::vector<int>& ports) {
+ grpc_core::ExecCtx exec_ctx;
+ grpc_core::ServerAddressList addresses =
+ CreateLbAddressesFromPortList(ports);
+ grpc_arg fake_addresses = CreateServerAddressListChannelArg(&addresses);
+ grpc_channel_args fake_result = {1, &fake_addresses};
+ response_generator_->SetReresolutionResponse(&fake_result);
+ }
+
+ const std::vector<int> GetBackendPorts(const size_t start_index = 0) const {
+ std::vector<int> backend_ports;
+ for (size_t i = start_index; i < backend_servers_.size(); ++i) {
+ backend_ports.push_back(backend_servers_[i].port_);
+ }
+ return backend_ports;
+ }
+
+ void ScheduleResponseForBalancer(size_t i,
+ const LoadBalanceResponse& response,
+ int delay_ms) {
+ balancers_.at(i)->add_response(response, delay_ms);
+ }
+
+ Status SendRpc(EchoResponse* response = nullptr, int timeout_ms = 1000,
+ bool wait_for_ready = false) {
+ const bool local_response = (response == nullptr);
+ if (local_response) response = new EchoResponse;
+ EchoRequest request;
+ request.set_message(kRequestMessage_);
+ ClientContext context;
+ context.set_deadline(grpc_timeout_milliseconds_to_deadline(timeout_ms));
+ if (wait_for_ready) context.set_wait_for_ready(true);
+ Status status = stub_->Echo(&context, request, response);
+ if (local_response) delete response;
+ return status;
+ }
+
+ void CheckRpcSendOk(const size_t times = 1, const int timeout_ms = 1000,
+ bool wait_for_ready = false) {
+ for (size_t i = 0; i < times; ++i) {
+ EchoResponse response;
+ const Status status = SendRpc(&response, timeout_ms, wait_for_ready);
+ EXPECT_TRUE(status.ok()) << "code=" << status.error_code()
+ << " message=" << status.error_message();
+ EXPECT_EQ(response.message(), kRequestMessage_);
+ }
+ }
+
+ void CheckRpcSendFailure() {
+ const Status status = SendRpc();
+ EXPECT_FALSE(status.ok());
+ }
+
+ template <typename T>
+ struct ServerThread {
+ explicit ServerThread(const grpc::string& type,
+ const grpc::string& server_host, T* service)
+ : type_(type), service_(service) {
+ std::mutex mu;
+ // We need to acquire the lock here in order to prevent the notify_one
+ // by ServerThread::Start from firing before the wait below is hit.
+ std::unique_lock<std::mutex> lock(mu);
+ port_ = grpc_pick_unused_port_or_die();
+ gpr_log(GPR_INFO, "starting %s server on port %d", type_.c_str(), port_);
+ std::condition_variable cond;
+ thread_.reset(new std::thread(
+ std::bind(&ServerThread::Start, this, server_host, &mu, &cond)));
+ cond.wait(lock);
+ gpr_log(GPR_INFO, "%s server startup complete", type_.c_str());
+ }
+
+ void Start(const grpc::string& server_host, std::mutex* mu,
+ std::condition_variable* cond) {
+ // We need to acquire the lock here in order to prevent the notify_one
+ // below from firing before its corresponding wait is executed.
+ std::lock_guard<std::mutex> lock(*mu);
+ std::ostringstream server_address;
+ server_address << server_host << ":" << port_;
+ ServerBuilder builder;
+ std::shared_ptr<ServerCredentials> creds(new SecureServerCredentials(
+ grpc_fake_transport_security_server_credentials_create()));
+ builder.AddListeningPort(server_address.str(), creds);
+ builder.RegisterService(service_);
+ server_ = builder.BuildAndStart();
+ cond->notify_one();
+ }
+
+ void Shutdown() {
+ gpr_log(GPR_INFO, "%s about to shutdown", type_.c_str());
+ server_->Shutdown(grpc_timeout_milliseconds_to_deadline(0));
+ thread_->join();
+ gpr_log(GPR_INFO, "%s shutdown completed", type_.c_str());
+ }
+
+ int port_;
+ grpc::string type_;
+ std::unique_ptr<Server> server_;
+ T* service_;
+ std::unique_ptr<std::thread> thread_;
+ };
+
+ const grpc::string server_host_;
+ const size_t num_backends_;
+ const size_t num_balancers_;
+ const int client_load_reporting_interval_seconds_;
+ std::shared_ptr<Channel> channel_;
+ std::unique_ptr<grpc::testing::EchoTestService::Stub> stub_;
+ std::vector<std::unique_ptr<BackendServiceImpl>> backends_;
+ std::vector<std::unique_ptr<BalancerServiceImpl>> balancers_;
+ std::vector<ServerThread<BackendService>> backend_servers_;
+ std::vector<ServerThread<BalancerService>> balancer_servers_;
+ grpc_core::RefCountedPtr<grpc_core::FakeResolverResponseGenerator>
+ response_generator_;
+ grpc_core::RefCountedPtr<grpc_core::FakeResolverResponseGenerator>
+ lb_channel_response_generator_;
+ const grpc::string kRequestMessage_ = "Live long and prosper.";
+ const grpc::string kApplicationTargetName_ = "application_target_name";
+ const grpc::string kDefaultServiceConfig_ =
+ "{\n"
+ " \"loadBalancingConfig\":[\n"
+ " { \"does_not_exist\":{} },\n"
+ " { \"xds_experimental\":{ \"balancerName\": \"fake:///lb\" } }\n"
+ " ]\n"
+ "}";
+};
+
+class SingleBalancerTest : public XdsEnd2endTest {
+ public:
+ SingleBalancerTest() : XdsEnd2endTest(4, 1, 0) {}
+};
+
+TEST_F(SingleBalancerTest, Vanilla) {
+ SetNextResolution({}, kDefaultServiceConfig_.c_str());
+ SetNextResolutionForLbChannelAllBalancers();
+ const size_t kNumRpcsPerAddress = 100;
+ ScheduleResponseForBalancer(
+ 0, BalancerServiceImpl::BuildResponseForBackends(GetBackendPorts(), {}),
+ 0);
+ // Make sure that trying to connect works without a call.
+ channel_->GetState(true /* try_to_connect */);
+ // We need to wait for all backends to come online.
+ WaitForAllBackends();
+ // Send kNumRpcsPerAddress RPCs per server.
+ CheckRpcSendOk(kNumRpcsPerAddress * num_backends_);
+ // Each backend should have gotten 100 requests.
+ for (size_t i = 0; i < backends_.size(); ++i) {
+ EXPECT_EQ(kNumRpcsPerAddress,
+ backend_servers_[i].service_->request_count());
+ }
+ balancers_[0]->NotifyDoneWithServerlists();
+ // The balancer got a single request.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ // and sent a single response.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+
+ // Check LB policy name for the channel.
+ EXPECT_EQ("xds_experimental", channel_->GetLoadBalancingPolicyName());
+}
+
+TEST_F(SingleBalancerTest, SameBackendListedMultipleTimes) {
+ SetNextResolution({}, kDefaultServiceConfig_.c_str());
+ SetNextResolutionForLbChannelAllBalancers();
+ // Same backend listed twice.
+ std::vector<int> ports;
+ ports.push_back(backend_servers_[0].port_);
+ ports.push_back(backend_servers_[0].port_);
+ const size_t kNumRpcsPerAddress = 10;
+ ScheduleResponseForBalancer(
+ 0, BalancerServiceImpl::BuildResponseForBackends(ports, {}), 0);
+ // We need to wait for the backend to come online.
+ WaitForBackend(0);
+ // Send kNumRpcsPerAddress RPCs per server.
+ CheckRpcSendOk(kNumRpcsPerAddress * ports.size());
+ // Backend should have gotten 20 requests.
+ EXPECT_EQ(kNumRpcsPerAddress * 2,
+ backend_servers_[0].service_->request_count());
+ // And they should have come from a single client port, because of
+ // subchannel sharing.
+ EXPECT_EQ(1UL, backends_[0]->clients().size());
+ balancers_[0]->NotifyDoneWithServerlists();
+}
+
+TEST_F(SingleBalancerTest, SecureNaming) {
+ // TODO(juanlishen): Use separate fake creds for the balancer channel.
+ ResetStub(0, kApplicationTargetName_ + ";lb");
+ SetNextResolution({}, kDefaultServiceConfig_.c_str());
+ SetNextResolutionForLbChannel({balancer_servers_[0].port_});
+ const size_t kNumRpcsPerAddress = 100;
+ ScheduleResponseForBalancer(
+ 0, BalancerServiceImpl::BuildResponseForBackends(GetBackendPorts(), {}),
+ 0);
+ // Make sure that trying to connect works without a call.
+ channel_->GetState(true /* try_to_connect */);
+ // We need to wait for all backends to come online.
+ WaitForAllBackends();
+ // Send kNumRpcsPerAddress RPCs per server.
+ CheckRpcSendOk(kNumRpcsPerAddress * num_backends_);
+
+ // Each backend should have gotten 100 requests.
+ for (size_t i = 0; i < backends_.size(); ++i) {
+ EXPECT_EQ(kNumRpcsPerAddress,
+ backend_servers_[i].service_->request_count());
+ }
+ // The balancer got a single request.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ // and sent a single response.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+}
+
+TEST_F(SingleBalancerTest, SecureNamingDeathTest) {
+ ::testing::FLAGS_gtest_death_test_style = "threadsafe";
+ // Make sure that we blow up (via abort() from the security connector) when
+ // the name from the balancer doesn't match expectations.
+ ASSERT_DEATH(
+ {
+ ResetStub(0, kApplicationTargetName_ + ";lb");
+ SetNextResolution({},
+ "{\n"
+ " \"loadBalancingConfig\":[\n"
+ " { \"does_not_exist\":{} },\n"
+ " { \"xds_experimental\":{ \"balancerName\": "
+ "\"fake:///wrong_lb\" } }\n"
+ " ]\n"
+ "}");
+ SetNextResolutionForLbChannel({balancer_servers_[0].port_});
+ channel_->WaitForConnected(grpc_timeout_seconds_to_deadline(1));
+ },
+ "");
+}
+
+TEST_F(SingleBalancerTest, InitiallyEmptyServerlist) {
+ SetNextResolution({}, kDefaultServiceConfig_.c_str());
+ SetNextResolutionForLbChannelAllBalancers();
+ const int kServerlistDelayMs = 500 * grpc_test_slowdown_factor();
+ const int kCallDeadlineMs = kServerlistDelayMs * 2;
+ // First response is an empty serverlist, sent right away.
+ ScheduleResponseForBalancer(0, LoadBalanceResponse(), 0);
+ // Send non-empty serverlist only after kServerlistDelayMs
+ ScheduleResponseForBalancer(
+ 0, BalancerServiceImpl::BuildResponseForBackends(GetBackendPorts(), {}),
+ kServerlistDelayMs);
+ const auto t0 = system_clock::now();
+ // Client will block: LB will initially send empty serverlist.
+ CheckRpcSendOk(1, kCallDeadlineMs, true /* wait_for_ready */);
+ const auto ellapsed_ms =
+ std::chrono::duration_cast<std::chrono::milliseconds>(
+ system_clock::now() - t0);
+ // but eventually, the LB sends a serverlist update that allows the call to
+ // proceed. The call delay must be larger than the delay in sending the
+ // populated serverlist but under the call's deadline (which is enforced by
+ // the call's deadline).
+ EXPECT_GT(ellapsed_ms.count(), kServerlistDelayMs);
+ balancers_[0]->NotifyDoneWithServerlists();
+ // The balancer got a single request.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ // and sent two responses.
+ EXPECT_EQ(2U, balancer_servers_[0].service_->response_count());
+}
+
+TEST_F(SingleBalancerTest, AllServersUnreachableFailFast) {
+ SetNextResolution({}, kDefaultServiceConfig_.c_str());
+ SetNextResolutionForLbChannelAllBalancers();
+ const size_t kNumUnreachableServers = 5;
+ std::vector<int> ports;
+ for (size_t i = 0; i < kNumUnreachableServers; ++i) {
+ ports.push_back(grpc_pick_unused_port_or_die());
+ }
+ ScheduleResponseForBalancer(
+ 0, BalancerServiceImpl::BuildResponseForBackends(ports, {}), 0);
+ const Status status = SendRpc();
+ // The error shouldn't be DEADLINE_EXCEEDED.
+ EXPECT_EQ(StatusCode::UNAVAILABLE, status.error_code());
+ balancers_[0]->NotifyDoneWithServerlists();
+ // The balancer got a single request.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ // and sent a single response.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+}
+
+// The fallback tests are deferred because the fallback mode hasn't been
+// supported yet.
+
+// TODO(juanlishen): Add TEST_F(SingleBalancerTest, Fallback)
+
+// TODO(juanlishen): Add TEST_F(SingleBalancerTest, FallbackUpdate)
+
+TEST_F(SingleBalancerTest, BackendsRestart) {
+ SetNextResolution({}, kDefaultServiceConfig_.c_str());
+ SetNextResolutionForLbChannelAllBalancers();
+ const size_t kNumRpcsPerAddress = 100;
+ ScheduleResponseForBalancer(
+ 0, BalancerServiceImpl::BuildResponseForBackends(GetBackendPorts(), {}),
+ 0);
+ // Make sure that trying to connect works without a call.
+ channel_->GetState(true /* try_to_connect */);
+ // Send kNumRpcsPerAddress RPCs per server.
+ CheckRpcSendOk(kNumRpcsPerAddress * num_backends_);
+ balancers_[0]->NotifyDoneWithServerlists();
+ // The balancer got a single request.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ // and sent a single response.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ for (size_t i = 0; i < backends_.size(); ++i) {
+ if (backends_[i]->Shutdown()) backend_servers_[i].Shutdown();
+ }
+ CheckRpcSendFailure();
+ for (size_t i = 0; i < num_backends_; ++i) {
+ backends_.emplace_back(new BackendServiceImpl());
+ backend_servers_.emplace_back(ServerThread<BackendService>(
+ "backend", server_host_, backends_.back().get()));
+ }
+ // The following RPC will fail due to the backend ports having changed. It
+ // will nonetheless exercise the xds-roundrobin handling of the RR policy
+ // having gone into shutdown.
+ // TODO(dgq): implement the "backend restart" component as well. We need extra
+ // machinery to either update the LB responses "on the fly" or instruct
+ // backends which ports to restart on.
+ CheckRpcSendFailure();
+}
+
+class UpdatesTest : public XdsEnd2endTest {
+ public:
+ UpdatesTest() : XdsEnd2endTest(4, 3, 0) {}
+};
+
+TEST_F(UpdatesTest, UpdateBalancersButKeepUsingOriginalBalancer) {
+ SetNextResolution({}, kDefaultServiceConfig_.c_str());
+ SetNextResolutionForLbChannelAllBalancers();
+ const std::vector<int> first_backend{GetBackendPorts()[0]};
+ const std::vector<int> second_backend{GetBackendPorts()[1]};
+ ScheduleResponseForBalancer(
+ 0, BalancerServiceImpl::BuildResponseForBackends(first_backend, {}), 0);
+ ScheduleResponseForBalancer(
+ 1, BalancerServiceImpl::BuildResponseForBackends(second_backend, {}), 0);
+
+ // Wait until the first backend is ready.
+ WaitForBackend(0);
+
+ // Send 10 requests.
+ gpr_log(GPR_INFO, "========= BEFORE FIRST BATCH ==========");
+ CheckRpcSendOk(10);
+ gpr_log(GPR_INFO, "========= DONE WITH FIRST BATCH ==========");
+
+ // All 10 requests should have gone to the first backend.
+ EXPECT_EQ(10U, backend_servers_[0].service_->request_count());
+
+ // Balancer 0 got a single request.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ // and sent a single response.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(0U, balancer_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, balancer_servers_[1].service_->response_count());
+ EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
+ EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+
+ gpr_log(GPR_INFO, "========= ABOUT TO UPDATE 1 ==========");
+ SetNextResolutionForLbChannel({balancer_servers_[1].port_});
+ gpr_log(GPR_INFO, "========= UPDATE 1 DONE ==========");
+
+ EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ gpr_timespec deadline = gpr_time_add(
+ gpr_now(GPR_CLOCK_REALTIME), gpr_time_from_millis(10000, GPR_TIMESPAN));
+ // Send 10 seconds worth of RPCs
+ do {
+ CheckRpcSendOk();
+ } while (gpr_time_cmp(gpr_now(GPR_CLOCK_REALTIME), deadline) < 0);
+ // The current LB call is still working, so xds continued using it to the
+ // first balancer, which doesn't assign the second backend.
+ EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(0U, balancer_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, balancer_servers_[1].service_->response_count());
+ EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
+ EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+}
+
+TEST_F(UpdatesTest, UpdateBalancerName) {
+ SetNextResolution({}, kDefaultServiceConfig_.c_str());
+ SetNextResolutionForLbChannelAllBalancers();
+ const std::vector<int> first_backend{GetBackendPorts()[0]};
+ const std::vector<int> second_backend{GetBackendPorts()[1]};
+ ScheduleResponseForBalancer(
+ 0, BalancerServiceImpl::BuildResponseForBackends(first_backend, {}), 0);
+ ScheduleResponseForBalancer(
+ 1, BalancerServiceImpl::BuildResponseForBackends(second_backend, {}), 0);
+
+ // Wait until the first backend is ready.
+ WaitForBackend(0);
+
+ // Send 10 requests.
+ gpr_log(GPR_INFO, "========= BEFORE FIRST BATCH ==========");
+ CheckRpcSendOk(10);
+ gpr_log(GPR_INFO, "========= DONE WITH FIRST BATCH ==========");
+
+ // All 10 requests should have gone to the first backend.
+ EXPECT_EQ(10U, backend_servers_[0].service_->request_count());
+
+ // Balancer 0 got a single request.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ // and sent a single response.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(0U, balancer_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, balancer_servers_[1].service_->response_count());
+ EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
+ EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+
+ std::vector<int> ports;
+ ports.emplace_back(balancer_servers_[1].port_);
+ auto new_lb_channel_response_generator =
+ grpc_core::MakeRefCounted<grpc_core::FakeResolverResponseGenerator>();
+ SetNextResolutionForLbChannel(ports, nullptr,
+ new_lb_channel_response_generator.get());
+ gpr_log(GPR_INFO, "========= ABOUT TO UPDATE BALANCER NAME ==========");
+ SetNextResolution({},
+ "{\n"
+ " \"loadBalancingConfig\":[\n"
+ " { \"does_not_exist\":{} },\n"
+ " { \"xds_experimental\":{ \"balancerName\": "
+ "\"fake:///updated_lb\" } }\n"
+ " ]\n"
+ "}",
+ new_lb_channel_response_generator.get());
+ gpr_log(GPR_INFO, "========= UPDATED BALANCER NAME ==========");
+
+ // Wait until update has been processed, as signaled by the second backend
+ // receiving a request.
+ EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ WaitForBackend(1);
+
+ backend_servers_[1].service_->ResetCounters();
+ gpr_log(GPR_INFO, "========= BEFORE SECOND BATCH ==========");
+ CheckRpcSendOk(10);
+ gpr_log(GPR_INFO, "========= DONE WITH SECOND BATCH ==========");
+ // All 10 requests should have gone to the second backend.
+ EXPECT_EQ(10U, backend_servers_[1].service_->request_count());
+
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancer_servers_[1].service_->request_count());
+ EXPECT_EQ(1U, balancer_servers_[1].service_->response_count());
+ EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
+ EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+}
+
+// Send an update with the same set of LBs as the one in SetUp() in order to
+// verify that the LB channel inside xds keeps the initial connection (which
+// by definition is also present in the update).
+TEST_F(UpdatesTest, UpdateBalancersRepeated) {
+ SetNextResolution({}, kDefaultServiceConfig_.c_str());
+ SetNextResolutionForLbChannelAllBalancers();
+ const std::vector<int> first_backend{GetBackendPorts()[0]};
+ const std::vector<int> second_backend{GetBackendPorts()[0]};
+
+ ScheduleResponseForBalancer(
+ 0, BalancerServiceImpl::BuildResponseForBackends(first_backend, {}), 0);
+ ScheduleResponseForBalancer(
+ 1, BalancerServiceImpl::BuildResponseForBackends(second_backend, {}), 0);
+
+ // Wait until the first backend is ready.
+ WaitForBackend(0);
+
+ // Send 10 requests.
+ gpr_log(GPR_INFO, "========= BEFORE FIRST BATCH ==========");
+ CheckRpcSendOk(10);
+ gpr_log(GPR_INFO, "========= DONE WITH FIRST BATCH ==========");
+
+ // All 10 requests should have gone to the first backend.
+ EXPECT_EQ(10U, backend_servers_[0].service_->request_count());
+
+ // Balancer 0 got a single request.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ // and sent a single response.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(0U, balancer_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, balancer_servers_[1].service_->response_count());
+ EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
+ EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+
+ std::vector<int> ports;
+ ports.emplace_back(balancer_servers_[0].port_);
+ ports.emplace_back(balancer_servers_[1].port_);
+ ports.emplace_back(balancer_servers_[2].port_);
+ gpr_log(GPR_INFO, "========= ABOUT TO UPDATE 1 ==========");
+ SetNextResolutionForLbChannel(ports);
+ gpr_log(GPR_INFO, "========= UPDATE 1 DONE ==========");
+
+ EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ gpr_timespec deadline = gpr_time_add(
+ gpr_now(GPR_CLOCK_REALTIME), gpr_time_from_millis(10000, GPR_TIMESPAN));
+ // Send 10 seconds worth of RPCs
+ do {
+ CheckRpcSendOk();
+ } while (gpr_time_cmp(gpr_now(GPR_CLOCK_REALTIME), deadline) < 0);
+ // xds continued using the original LB call to the first balancer, which
+ // doesn't assign the second backend.
+ EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+
+ ports.clear();
+ ports.emplace_back(balancer_servers_[0].port_);
+ ports.emplace_back(balancer_servers_[1].port_);
+ gpr_log(GPR_INFO, "========= ABOUT TO UPDATE 2 ==========");
+ SetNextResolutionForLbChannel(ports);
+ gpr_log(GPR_INFO, "========= UPDATE 2 DONE ==========");
+
+ EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ deadline = gpr_time_add(gpr_now(GPR_CLOCK_REALTIME),
+ gpr_time_from_millis(10000, GPR_TIMESPAN));
+ // Send 10 seconds worth of RPCs
+ do {
+ CheckRpcSendOk();
+ } while (gpr_time_cmp(gpr_now(GPR_CLOCK_REALTIME), deadline) < 0);
+ // xds continued using the original LB call to the first balancer, which
+ // doesn't assign the second backend.
+ EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+}
+
+TEST_F(UpdatesTest, UpdateBalancersDeadUpdate) {
+ SetNextResolution({}, kDefaultServiceConfig_.c_str());
+ SetNextResolutionForLbChannel({balancer_servers_[0].port_});
+ const std::vector<int> first_backend{GetBackendPorts()[0]};
+ const std::vector<int> second_backend{GetBackendPorts()[1]};
+
+ ScheduleResponseForBalancer(
+ 0, BalancerServiceImpl::BuildResponseForBackends(first_backend, {}), 0);
+ ScheduleResponseForBalancer(
+ 1, BalancerServiceImpl::BuildResponseForBackends(second_backend, {}), 0);
+
+ // Start servers and send 10 RPCs per server.
+ gpr_log(GPR_INFO, "========= BEFORE FIRST BATCH ==========");
+ CheckRpcSendOk(10);
+ gpr_log(GPR_INFO, "========= DONE WITH FIRST BATCH ==========");
+ // All 10 requests should have gone to the first backend.
+ EXPECT_EQ(10U, backend_servers_[0].service_->request_count());
+
+ // Kill balancer 0
+ gpr_log(GPR_INFO, "********** ABOUT TO KILL BALANCER 0 *************");
+ if (balancers_[0]->Shutdown()) balancer_servers_[0].Shutdown();
+ gpr_log(GPR_INFO, "********** KILLED BALANCER 0 *************");
+
+ // This is serviced by the existing child policy.
+ gpr_log(GPR_INFO, "========= BEFORE SECOND BATCH ==========");
+ CheckRpcSendOk(10);
+ gpr_log(GPR_INFO, "========= DONE WITH SECOND BATCH ==========");
+ // All 10 requests should again have gone to the first backend.
+ EXPECT_EQ(20U, backend_servers_[0].service_->request_count());
+ EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+
+ // Balancer 0 got a single request.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ // and sent a single response.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(0U, balancer_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, balancer_servers_[1].service_->response_count());
+ EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
+ EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+
+ gpr_log(GPR_INFO, "========= ABOUT TO UPDATE 1 ==========");
+ SetNextResolutionForLbChannel({balancer_servers_[1].port_});
+ gpr_log(GPR_INFO, "========= UPDATE 1 DONE ==========");
+
+ // Wait until update has been processed, as signaled by the second backend
+ // receiving a request. In the meantime, the client continues to be serviced
+ // (by the first backend) without interruption.
+ EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ WaitForBackend(1);
+
+ // This is serviced by the updated RR policy
+ backend_servers_[1].service_->ResetCounters();
+ gpr_log(GPR_INFO, "========= BEFORE THIRD BATCH ==========");
+ CheckRpcSendOk(10);
+ gpr_log(GPR_INFO, "========= DONE WITH THIRD BATCH ==========");
+ // All 10 requests should have gone to the second backend.
+ EXPECT_EQ(10U, backend_servers_[1].service_->request_count());
+
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ // The second balancer, published as part of the first update, may end up
+ // getting two requests (that is, 1 <= #req <= 2) if the LB call retry timer
+ // firing races with the arrival of the update containing the second
+ // balancer.
+ EXPECT_GE(balancer_servers_[1].service_->request_count(), 1U);
+ EXPECT_GE(balancer_servers_[1].service_->response_count(), 1U);
+ EXPECT_LE(balancer_servers_[1].service_->request_count(), 2U);
+ EXPECT_LE(balancer_servers_[1].service_->response_count(), 2U);
+ EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
+ EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+}
+
+// The re-resolution tests are deferred because they rely on the fallback mode,
+// which hasn't been supported.
+
+// TODO(juanlishen): Add TEST_F(UpdatesTest, ReresolveDeadBackend).
+
+// TODO(juanlishen): Add TEST_F(UpdatesWithClientLoadReportingTest,
+// ReresolveDeadBalancer)
+
+// The drop tests are deferred because the drop handling hasn't been added yet.
+
+// TODO(roth): Add TEST_F(SingleBalancerTest, Drop)
+
+// TODO(roth): Add TEST_F(SingleBalancerTest, DropAllFirst)
+
+// TODO(roth): Add TEST_F(SingleBalancerTest, DropAll)
+
+class SingleBalancerWithClientLoadReportingTest : public XdsEnd2endTest {
+ public:
+ SingleBalancerWithClientLoadReportingTest() : XdsEnd2endTest(4, 1, 3) {}
+};
+
+// The client load reporting tests are deferred because the client load
+// reporting hasn't been supported yet.
+
+// TODO(vpowar): Add TEST_F(SingleBalancerWithClientLoadReportingTest, Vanilla)
+
+// TODO(roth): Add TEST_F(SingleBalancerWithClientLoadReportingTest, Drop)
+
+} // namespace
+} // namespace testing
+} // namespace grpc
+
+int main(int argc, char** argv) {
+ grpc_init();
+ grpc::testing::TestEnvironment env(argc, argv);
+ ::testing::InitGoogleTest(&argc, argv);
+ const auto result = RUN_ALL_TESTS();
+ grpc_shutdown();
+ return result;
+}
diff --git a/tools/run_tests/generated/sources_and_headers.json b/tools/run_tests/generated/sources_and_headers.json
index 0b84b8a4b95..5a1eafda6c2 100644
--- a/tools/run_tests/generated/sources_and_headers.json
+++ b/tools/run_tests/generated/sources_and_headers.json
@@ -5002,6 +5002,28 @@
"third_party": false,
"type": "target"
},
+ {
+ "deps": [
+ "gpr",
+ "grpc",
+ "grpc++",
+ "grpc++_test_util",
+ "grpc_test_util"
+ ],
+ "headers": [
+ "src/proto/grpc/lb/v1/load_balancer.grpc.pb.h",
+ "src/proto/grpc/lb/v1/load_balancer.pb.h",
+ "src/proto/grpc/lb/v1/load_balancer_mock.grpc.pb.h"
+ ],
+ "is_filegroup": false,
+ "language": "c++",
+ "name": "xds_end2end_test",
+ "src": [
+ "test/cpp/end2end/xds_end2end_test.cc"
+ ],
+ "third_party": false,
+ "type": "target"
+ },
{
"deps": [
"gpr",
diff --git a/tools/run_tests/generated/tests.json b/tools/run_tests/generated/tests.json
index 6dc9eb7f0d1..9399b9f6b9d 100644
--- a/tools/run_tests/generated/tests.json
+++ b/tools/run_tests/generated/tests.json
@@ -5686,6 +5686,30 @@
],
"uses_polling": true
},
+ {
+ "args": [],
+ "benchmark": false,
+ "ci_platforms": [
+ "linux",
+ "mac",
+ "posix",
+ "windows"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "gtest": true,
+ "language": "c++",
+ "name": "xds_end2end_test",
+ "platforms": [
+ "linux",
+ "mac",
+ "posix",
+ "windows"
+ ],
+ "uses_polling": true
+ },
{
"args": [],
"benchmark": false,
From 6d75cfe426c4acaf5c5d7a35e2b8dd9277e71348 Mon Sep 17 00:00:00 2001
From: "Mark D. Roth" <roth@google.com>
Date: Tue, 5 Mar 2019 14:18:01 -0800
Subject: [PATCH 026/138] Revert "Merge pull request #18254 from
grpc/revert-18078-grpclb_child_policy_configurable"
This reverts commit 6dcf6d164510a66efa831f2620e6511ffda0eba1, reversing
changes made to a2f1e924defce11bdbf97375a7410d34b5661d6b.
---
.../client_channel/lb_policy/grpclb/grpclb.cc | 344 +++++++++++++-----
test/cpp/end2end/grpclb_end2end_test.cc | 144 ++++++++
2 files changed, 402 insertions(+), 86 deletions(-)
diff --git a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
index c5d1ff22a9d..c1f2846f046 100644
--- a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
@@ -39,15 +39,14 @@
/// the balancer, we update the round_robin policy with the new list of
/// addresses. If we cannot communicate with the balancer on startup,
/// however, we may enter fallback mode, in which case we will populate
-/// the RR policy's addresses from the backend addresses returned by the
+/// the child policy's addresses from the backend addresses returned by the
/// resolver.
///
-/// Once an RR policy instance is in place (and getting updated as described),
+/// Once a child policy instance is in place (and getting updated as described),
/// calls for a pick, a ping, or a cancellation will be serviced right
-/// away by forwarding them to the RR instance. Any time there's no RR
-/// policy available (i.e., right after the creation of the gRPCLB policy),
-/// pick and ping requests are added to a list of pending picks and pings
-/// to be flushed and serviced when the RR policy instance becomes available.
+/// away by forwarding them to the child policy instance. Any time there's no
+/// child policy available (i.e., right after the creation of the gRPCLB
+/// policy), pick requests are queued.
///
/// \see https://github.com/grpc/grpc/blob/master/doc/load-balancing.md for the
/// high level design and details.
@@ -279,16 +278,23 @@ class GrpcLb : public LoadBalancingPolicy {
UniquePtr<SubchannelPicker> picker) override;
void RequestReresolution() override;
+ void set_child(LoadBalancingPolicy* child) { child_ = child; }
+
private:
+ bool CalledByPendingChild() const;
+ bool CalledByCurrentChild() const;
+
RefCountedPtr<GrpcLb> parent_;
+ LoadBalancingPolicy* child_ = nullptr;
};
~GrpcLb();
void ShutdownLocked() override;
- // Helper function used in UpdateLocked().
+ // Helper functions used in UpdateLocked().
void ProcessChannelArgsLocked(const grpc_channel_args& args);
+ void ParseLbConfig(Config* grpclb_config);
// Methods for dealing with the balancer channel and call.
void StartBalancerCallLocked();
@@ -296,10 +302,11 @@ class GrpcLb : public LoadBalancingPolicy {
void StartBalancerCallRetryTimerLocked();
static void OnBalancerCallRetryTimerLocked(void* arg, grpc_error* error);
- // Methods for dealing with the RR policy.
- grpc_channel_args* CreateRoundRobinPolicyArgsLocked();
- void CreateRoundRobinPolicyLocked(Args args);
- void CreateOrUpdateRoundRobinPolicyLocked();
+ // Methods for dealing with the child policy.
+ grpc_channel_args* CreateChildPolicyArgsLocked();
+ OrphanablePtr<LoadBalancingPolicy> CreateChildPolicyLocked(
+ const char* name, grpc_channel_args* args);
+ void CreateOrUpdateChildPolicyLocked();
// Who the client is trying to communicate with.
const char* server_name_ = nullptr;
@@ -345,8 +352,14 @@ class GrpcLb : public LoadBalancingPolicy {
grpc_timer lb_fallback_timer_;
grpc_closure lb_on_fallback_;
- // The RR policy to use for the backends.
- OrphanablePtr<LoadBalancingPolicy> rr_policy_;
+ // The child policy to use for the backends.
+ OrphanablePtr<LoadBalancingPolicy> child_policy_;
+ // When switching child policies, the new policy will be stored here
+ // until it reports READY, at which point it will be moved to child_policy_.
+ OrphanablePtr<LoadBalancingPolicy> pending_child_policy_;
+ // The child policy name and config.
+ UniquePtr<char> child_policy_name_;
+ RefCountedPtr<Config> child_policy_config_;
};
//
@@ -558,14 +571,30 @@ GrpcLb::Picker::PickResult GrpcLb::Picker::Pick(PickState* pick,
// GrpcLb::Helper
//
+bool GrpcLb::Helper::CalledByPendingChild() const {
+ GPR_ASSERT(child_ != nullptr);
+ return child_ == parent_->pending_child_policy_.get();
+}
+
+bool GrpcLb::Helper::CalledByCurrentChild() const {
+ GPR_ASSERT(child_ != nullptr);
+ return child_ == parent_->child_policy_.get();
+}
+
Subchannel* GrpcLb::Helper::CreateSubchannel(const grpc_channel_args& args) {
- if (parent_->shutting_down_) return nullptr;
+ if (parent_->shutting_down_ ||
+ (!CalledByPendingChild() && !CalledByCurrentChild())) {
+ return nullptr;
+ }
return parent_->channel_control_helper()->CreateSubchannel(args);
}
grpc_channel* GrpcLb::Helper::CreateChannel(const char* target,
const grpc_channel_args& args) {
- if (parent_->shutting_down_) return nullptr;
+ if (parent_->shutting_down_ ||
+ (!CalledByPendingChild() && !CalledByCurrentChild())) {
+ return nullptr;
+ }
return parent_->channel_control_helper()->CreateChannel(target, args);
}
@@ -576,31 +605,50 @@ void GrpcLb::Helper::UpdateState(grpc_connectivity_state state,
GRPC_ERROR_UNREF(state_error);
return;
}
+ // If this request is from the pending child policy, ignore it until
+ // it reports READY, at which point we swap it into place.
+ if (CalledByPendingChild()) {
+ if (grpc_lb_glb_trace.enabled()) {
+ gpr_log(GPR_INFO,
+ "[grpclb %p helper %p] pending child policy %p reports state=%s",
+ parent_.get(), this, parent_->pending_child_policy_.get(),
+ grpc_connectivity_state_name(state));
+ }
+ if (state != GRPC_CHANNEL_READY) {
+ GRPC_ERROR_UNREF(state_error);
+ return;
+ }
+ parent_->child_policy_ = std::move(parent_->pending_child_policy_);
+ } else if (!CalledByCurrentChild()) {
+ // This request is from an outdated child, so ignore it.
+ GRPC_ERROR_UNREF(state_error);
+ return;
+ }
// There are three cases to consider here:
// 1. We're in fallback mode. In this case, we're always going to use
- // RR's result, so we pass its picker through as-is.
+ // the child policy's result, so we pass its picker through as-is.
// 2. The serverlist contains only drop entries. In this case, we
// want to use our own picker so that we can return the drops.
// 3. Not in fallback mode and serverlist is not all drops (i.e., it
// may be empty or contain at least one backend address). There are
// two sub-cases:
- // a. RR is reporting state READY. In this case, we wrap RR's
- // picker in our own, so that we can handle drops and LB token
- // metadata for each pick.
- // b. RR is reporting a state other than READY. In this case, we
- // don't want to use our own picker, because we don't want to
- // process drops for picks that yield a QUEUE result; this would
+ // a. The child policy is reporting state READY. In this case, we wrap
+ // the child's picker in our own, so that we can handle drops and LB
+ // token metadata for each pick.
+ // b. The child policy is reporting a state other than READY. In this
+ // case, we don't want to use our own picker, because we don't want
+ // to process drops for picks that yield a QUEUE result; this would
// result in dropping too many calls, since we will see the
// queued picks multiple times, and we'd consider each one a
// separate call for the drop calculation.
//
- // Cases 1 and 3b: return picker from RR as-is.
+ // Cases 1 and 3b: return picker from the child policy as-is.
if (parent_->serverlist_ == nullptr ||
(!parent_->serverlist_->ContainsAllDropEntries() &&
state != GRPC_CHANNEL_READY)) {
if (grpc_lb_glb_trace.enabled()) {
gpr_log(GPR_INFO,
- "[grpclb %p helper %p] state=%s passing RR picker %p as-is",
+ "[grpclb %p helper %p] state=%s passing child picker %p as-is",
parent_.get(), this, grpc_connectivity_state_name(state),
picker.get());
}
@@ -608,9 +656,9 @@ void GrpcLb::Helper::UpdateState(grpc_connectivity_state state,
std::move(picker));
return;
}
- // Cases 2 and 3a: wrap picker from RR in our own picker.
+ // Cases 2 and 3a: wrap picker from the child in our own picker.
if (grpc_lb_glb_trace.enabled()) {
- gpr_log(GPR_INFO, "[grpclb %p helper %p] state=%s wrapping RR picker %p",
+ gpr_log(GPR_INFO, "[grpclb %p helper %p] state=%s wrapping child picker %p",
parent_.get(), this, grpc_connectivity_state_name(state),
picker.get());
}
@@ -628,15 +676,19 @@ void GrpcLb::Helper::UpdateState(grpc_connectivity_state state,
void GrpcLb::Helper::RequestReresolution() {
if (parent_->shutting_down_) return;
+ // If there is a pending child policy, ignore re-resolution requests
+ // from the current child policy (or any outdated pending child).
+ if (parent_->pending_child_policy_ != nullptr && !CalledByPendingChild()) {
+ return;
+ }
if (grpc_lb_glb_trace.enabled()) {
gpr_log(GPR_INFO,
- "[grpclb %p] Re-resolution requested from the internal RR policy "
- "(%p).",
- parent_.get(), parent_->rr_policy_.get());
+ "[grpclb %p] Re-resolution requested from child policy (%p).",
+ parent_.get(), child_);
}
// If we are talking to a balancer, we expect to get updated addresses
// from the balancer, so we can ignore the re-resolution request from
- // the RR policy. Otherwise, pass the re-resolution request up to the
+ // the child policy. Otherwise, pass the re-resolution request up to the
// channel.
if (parent_->lb_calld_ == nullptr ||
!parent_->lb_calld_->seen_initial_response()) {
@@ -984,7 +1036,7 @@ void GrpcLb::BalancerCallState::OnBalancerMessageReceivedLocked(
// instance will be destroyed either upon the next update or when the
// GrpcLb instance is destroyed.
grpclb_policy->serverlist_ = std::move(serverlist_wrapper);
- grpclb_policy->CreateOrUpdateRoundRobinPolicyLocked();
+ grpclb_policy->CreateOrUpdateChildPolicyLocked();
}
} else {
// No valid initial response or serverlist found.
@@ -1200,7 +1252,8 @@ void GrpcLb::ShutdownLocked() {
if (fallback_timer_callback_pending_) {
grpc_timer_cancel(&lb_fallback_timer_);
}
- rr_policy_.reset();
+ child_policy_.reset();
+ pending_child_policy_.reset();
// We destroy the LB channel here instead of in our destructor because
// destroying the channel triggers a last callback to
// OnBalancerChannelConnectivityChangedLocked(), and we need to be
@@ -1220,17 +1273,24 @@ void GrpcLb::ResetBackoffLocked() {
if (lb_channel_ != nullptr) {
grpc_channel_reset_connect_backoff(lb_channel_);
}
- if (rr_policy_ != nullptr) {
- rr_policy_->ResetBackoffLocked();
+ if (child_policy_ != nullptr) {
+ child_policy_->ResetBackoffLocked();
+ }
+ if (pending_child_policy_ != nullptr) {
+ pending_child_policy_->ResetBackoffLocked();
}
}
void GrpcLb::FillChildRefsForChannelz(
channelz::ChildRefsList* child_subchannels,
channelz::ChildRefsList* child_channels) {
- // delegate to the RoundRobin to fill the children subchannels.
- if (rr_policy_ != nullptr) {
- rr_policy_->FillChildRefsForChannelz(child_subchannels, child_channels);
+ // delegate to the child policy to fill the children subchannels.
+ if (child_policy_ != nullptr) {
+ child_policy_->FillChildRefsForChannelz(child_subchannels, child_channels);
+ }
+ if (pending_child_policy_ != nullptr) {
+ pending_child_policy_->FillChildRefsForChannelz(child_subchannels,
+ child_channels);
}
gpr_atm uuid = gpr_atm_no_barrier_load(&lb_channel_uuid_);
if (uuid != 0) {
@@ -1238,6 +1298,32 @@ void GrpcLb::FillChildRefsForChannelz(
}
}
+void GrpcLb::UpdateLocked(const grpc_channel_args& args,
+ RefCountedPtr<Config> lb_config) {
+ const bool is_initial_update = lb_channel_ == nullptr;
+ ParseLbConfig(lb_config.get());
+ ProcessChannelArgsLocked(args);
+ // Update the existing child policy.
+ if (child_policy_ != nullptr) CreateOrUpdateChildPolicyLocked();
+ // If this is the initial update, start the fallback timer.
+ if (is_initial_update) {
+ if (lb_fallback_timeout_ms_ > 0 && serverlist_ == nullptr &&
+ !fallback_timer_callback_pending_) {
+ grpc_millis deadline = ExecCtx::Get()->Now() + lb_fallback_timeout_ms_;
+ Ref(DEBUG_LOCATION, "on_fallback_timer").release(); // Ref for callback
+ GRPC_CLOSURE_INIT(&lb_on_fallback_, &GrpcLb::OnFallbackTimerLocked, this,
+ grpc_combiner_scheduler(combiner()));
+ fallback_timer_callback_pending_ = true;
+ grpc_timer_init(&lb_fallback_timer_, deadline, &lb_on_fallback_);
+ }
+ StartBalancerCallLocked();
+ }
+}
+
+//
+// helpers for UpdateLocked()
+//
+
// Returns the backend addresses extracted from the given addresses.
UniquePtr<ServerAddressList> ExtractBackendAddresses(
const ServerAddressList& addresses) {
@@ -1299,25 +1385,26 @@ void GrpcLb::ProcessChannelArgsLocked(const grpc_channel_args& args) {
grpc_channel_args_destroy(lb_channel_args);
}
-void GrpcLb::UpdateLocked(const grpc_channel_args& args,
- RefCountedPtr<Config> lb_config) {
- const bool is_initial_update = lb_channel_ == nullptr;
- ProcessChannelArgsLocked(args);
- // Update the existing RR policy.
- if (rr_policy_ != nullptr) CreateOrUpdateRoundRobinPolicyLocked();
- // If this is the initial update, start the fallback timer and the
- // balancer call.
- if (is_initial_update) {
- if (lb_fallback_timeout_ms_ > 0 && serverlist_ == nullptr &&
- !fallback_timer_callback_pending_) {
- grpc_millis deadline = ExecCtx::Get()->Now() + lb_fallback_timeout_ms_;
- Ref(DEBUG_LOCATION, "on_fallback_timer").release(); // Ref for callback
- GRPC_CLOSURE_INIT(&lb_on_fallback_, &GrpcLb::OnFallbackTimerLocked, this,
- grpc_combiner_scheduler(combiner()));
- fallback_timer_callback_pending_ = true;
- grpc_timer_init(&lb_fallback_timer_, deadline, &lb_on_fallback_);
+void GrpcLb::ParseLbConfig(Config* grpclb_config) {
+ const grpc_json* child_policy = nullptr;
+ if (grpclb_config != nullptr) {
+ const grpc_json* grpclb_config_json = grpclb_config->json();
+ for (const grpc_json* field = grpclb_config_json; field != nullptr;
+ field = field->next) {
+ if (field->key == nullptr) return;
+ if (strcmp(field->key, "childPolicy") == 0) {
+ if (child_policy != nullptr) return; // Duplicate.
+ child_policy = ParseLoadBalancingConfig(field);
+ }
}
- StartBalancerCallLocked();
+ }
+ if (child_policy != nullptr) {
+ child_policy_name_ = UniquePtr<char>(gpr_strdup(child_policy->key));
+ child_policy_config_ = MakeRefCounted<Config>(
+ child_policy->child, grpclb_config->service_config());
+ } else {
+ child_policy_name_.reset();
+ child_policy_config_.reset();
}
}
@@ -1352,7 +1439,7 @@ void GrpcLb::OnFallbackTimerLocked(void* arg, grpc_error* error) {
grpclb_policy);
}
GPR_ASSERT(grpclb_policy->fallback_backend_addresses_ != nullptr);
- grpclb_policy->CreateOrUpdateRoundRobinPolicyLocked();
+ grpclb_policy->CreateOrUpdateChildPolicyLocked();
}
grpclb_policy->Unref(DEBUG_LOCATION, "on_fallback_timer");
}
@@ -1396,10 +1483,10 @@ void GrpcLb::OnBalancerCallRetryTimerLocked(void* arg, grpc_error* error) {
}
//
-// code for interacting with the RR policy
+// code for interacting with the child policy
//
-grpc_channel_args* GrpcLb::CreateRoundRobinPolicyArgsLocked() {
+grpc_channel_args* GrpcLb::CreateChildPolicyArgsLocked() {
ServerAddressList tmp_addresses;
ServerAddressList* addresses = &tmp_addresses;
bool is_backend_from_grpclb_load_balancer = false;
@@ -1408,7 +1495,7 @@ grpc_channel_args* GrpcLb::CreateRoundRobinPolicyArgsLocked() {
lb_calld_ == nullptr ? nullptr : lb_calld_->client_stats());
is_backend_from_grpclb_load_balancer = true;
} else {
- // If CreateOrUpdateRoundRobinPolicyLocked() is invoked when we haven't
+ // If CreateOrUpdateChildPolicyLocked() is invoked when we haven't
// received any serverlist from the balancer, we use the fallback backends
// returned by the resolver. Note that the fallback backend list may be
// empty, in which case the new round_robin policy will keep the requested
@@ -1435,49 +1522,134 @@ grpc_channel_args* GrpcLb::CreateRoundRobinPolicyArgsLocked() {
const_cast<char*>(GRPC_ARG_INHIBIT_HEALTH_CHECKING), 1);
++num_args_to_add;
}
- grpc_channel_args* args = grpc_channel_args_copy_and_add_and_remove(
+ return grpc_channel_args_copy_and_add_and_remove(
args_, keys_to_remove, GPR_ARRAY_SIZE(keys_to_remove), args_to_add,
num_args_to_add);
- return args;
}
-void GrpcLb::CreateRoundRobinPolicyLocked(Args args) {
- GPR_ASSERT(rr_policy_ == nullptr);
- rr_policy_ = LoadBalancingPolicyRegistry::CreateLoadBalancingPolicy(
- "round_robin", std::move(args));
- if (GPR_UNLIKELY(rr_policy_ == nullptr)) {
- gpr_log(GPR_ERROR, "[grpclb %p] Failure creating a RoundRobin policy",
- this);
- return;
+OrphanablePtr<LoadBalancingPolicy> GrpcLb::CreateChildPolicyLocked(
+ const char* name, grpc_channel_args* args) {
+ Helper* helper = New<Helper>(Ref());
+ LoadBalancingPolicy::Args lb_policy_args;
+ lb_policy_args.combiner = combiner();
+ lb_policy_args.args = args;
+ lb_policy_args.channel_control_helper =
+ UniquePtr<ChannelControlHelper>(helper);
+ OrphanablePtr<LoadBalancingPolicy> lb_policy =
+ LoadBalancingPolicyRegistry::CreateLoadBalancingPolicy(
+ name, std::move(lb_policy_args));
+ if (GPR_UNLIKELY(lb_policy == nullptr)) {
+ gpr_log(GPR_ERROR, "[grpclb %p] Failure creating child policy %s", this,
+ name);
+ return nullptr;
}
+ helper->set_child(lb_policy.get());
if (grpc_lb_glb_trace.enabled()) {
- gpr_log(GPR_INFO, "[grpclb %p] Created new RR policy %p", this,
- rr_policy_.get());
+ gpr_log(GPR_INFO, "[grpclb %p] Created new child policy %s (%p)", this,
+ name, lb_policy.get());
}
// Add the gRPC LB's interested_parties pollset_set to that of the newly
- // created RR policy. This will make the RR policy progress upon activity on
- // gRPC LB, which in turn is tied to the application's call.
- grpc_pollset_set_add_pollset_set(rr_policy_->interested_parties(),
+ // created child policy. This will make the child policy progress upon
+ // activity on gRPC LB, which in turn is tied to the application's call.
+ grpc_pollset_set_add_pollset_set(lb_policy->interested_parties(),
interested_parties());
+ return lb_policy;
}
-void GrpcLb::CreateOrUpdateRoundRobinPolicyLocked() {
+void GrpcLb::CreateOrUpdateChildPolicyLocked() {
if (shutting_down_) return;
- grpc_channel_args* args = CreateRoundRobinPolicyArgsLocked();
+ grpc_channel_args* args = CreateChildPolicyArgsLocked();
GPR_ASSERT(args != nullptr);
- if (rr_policy_ == nullptr) {
- LoadBalancingPolicy::Args lb_policy_args;
- lb_policy_args.combiner = combiner();
- lb_policy_args.args = args;
- lb_policy_args.channel_control_helper =
- UniquePtr<ChannelControlHelper>(New<Helper>(Ref()));
- CreateRoundRobinPolicyLocked(std::move(lb_policy_args));
+ // If the child policy name changes, we need to create a new child
+ // policy. When this happens, we leave child_policy_ as-is and store
+ // the new child policy in pending_child_policy_. Once the new child
+ // policy transitions into state READY, we swap it into child_policy_,
+ // replacing the original child policy. So pending_child_policy_ is
+ // non-null only between when we apply an update that changes the child
+ // policy name and when the new child reports state READY.
+ //
+ // Updates can arrive at any point during this transition. We always
+ // apply updates relative to the most recently created child policy,
+ // even if the most recent one is still in pending_child_policy_. This
+ // is true both when applying the updates to an existing child policy
+ // and when determining whether we need to create a new policy.
+ //
+ // As a result of this, there are several cases to consider here:
+ //
+ // 1. We have no existing child policy (i.e., we have started up but
+ // have not yet received a serverlist from the balancer or gone
+ // into fallback mode; in this case, both child_policy_ and
+ // pending_child_policy_ are null). In this case, we create a
+ // new child policy and store it in child_policy_.
+ //
+ // 2. We have an existing child policy and have no pending child policy
+ // from a previous update (i.e., either there has not been a
+ // previous update that changed the policy name, or we have already
+ // finished swapping in the new policy; in this case, child_policy_
+ // is non-null but pending_child_policy_ is null). In this case:
+ // a. If child_policy_->name() equals child_policy_name, then we
+ // update the existing child policy.
+ // b. If child_policy_->name() does not equal child_policy_name,
+ // we create a new policy. The policy will be stored in
+ // pending_child_policy_ and will later be swapped into
+ // child_policy_ by the helper when the new child transitions
+ // into state READY.
+ //
+ // 3. We have an existing child policy and have a pending child policy
+ // from a previous update (i.e., a previous update set
+ // pending_child_policy_ as per case 2b above and that policy has
+ // not yet transitioned into state READY and been swapped into
+ // child_policy_; in this case, both child_policy_ and
+ // pending_child_policy_ are non-null). In this case:
+ // a. If pending_child_policy_->name() equals child_policy_name,
+ // then we update the existing pending child policy.
+ // b. If pending_child_policy->name() does not equal
+ // child_policy_name, then we create a new policy. The new
+ // policy is stored in pending_child_policy_ (replacing the one
+ // that was there before, which will be immediately shut down)
+ // and will later be swapped into child_policy_ by the helper
+ // when the new child transitions into state READY.
+ const char* child_policy_name =
+ child_policy_name_ == nullptr ? "round_robin" : child_policy_name_.get();
+ const bool create_policy =
+ // case 1
+ child_policy_ == nullptr ||
+ // case 2b
+ (pending_child_policy_ == nullptr &&
+ strcmp(child_policy_->name(), child_policy_name) != 0) ||
+ // case 3b
+ (pending_child_policy_ != nullptr &&
+ strcmp(pending_child_policy_->name(), child_policy_name) != 0);
+ LoadBalancingPolicy* policy_to_update = nullptr;
+ if (create_policy) {
+ // Cases 1, 2b, and 3b: create a new child policy.
+ // If child_policy_ is null, we set it (case 1), else we set
+ // pending_child_policy_ (cases 2b and 3b).
+ auto& lb_policy =
+ child_policy_ == nullptr ? child_policy_ : pending_child_policy_;
+ if (grpc_lb_glb_trace.enabled()) {
+ gpr_log(GPR_INFO, "[grpclb %p] Creating new %schild policy %s", this,
+ child_policy_ == nullptr ? "" : "pending ", child_policy_name);
+ }
+ lb_policy = CreateChildPolicyLocked(child_policy_name, args);
+ policy_to_update = lb_policy.get();
+ } else {
+ // Cases 2a and 3a: update an existing policy.
+ // If we have a pending child policy, send the update to the pending
+ // policy (case 3a), else send it to the current policy (case 2a).
+ policy_to_update = pending_child_policy_ != nullptr
+ ? pending_child_policy_.get()
+ : child_policy_.get();
}
+ GPR_ASSERT(policy_to_update != nullptr);
+ // Update the policy.
if (grpc_lb_glb_trace.enabled()) {
- gpr_log(GPR_INFO, "[grpclb %p] Updating RR policy %p", this,
- rr_policy_.get());
+ gpr_log(GPR_INFO, "[grpclb %p] Updating %schild policy %p", this,
+ policy_to_update == pending_child_policy_.get() ? "pending " : "",
+ policy_to_update);
}
- rr_policy_->UpdateLocked(*args, nullptr);
+ policy_to_update->UpdateLocked(*args, child_policy_config_);
+ // Clean up.
grpc_channel_args_destroy(args);
}
diff --git a/test/cpp/end2end/grpclb_end2end_test.cc b/test/cpp/end2end/grpclb_end2end_test.cc
index 2288b88b517..31353ba1304 100644
--- a/test/cpp/end2end/grpclb_end2end_test.cc
+++ b/test/cpp/end2end/grpclb_end2end_test.cc
@@ -723,6 +723,150 @@ TEST_F(SingleBalancerTest, SelectGrpclbWithMigrationServiceConfig) {
EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
}
+TEST_F(SingleBalancerTest, UsePickFirstChildPolicy) {
+ SetNextResolutionAllBalancers(
+ "{\n"
+ " \"loadBalancingConfig\":[\n"
+ " { \"grpclb\":{\n"
+ " \"childPolicy\":[\n"
+ " { \"pick_first\":{} }\n"
+ " ]\n"
+ " } }\n"
+ " ]\n"
+ "}");
+ ScheduleResponseForBalancer(
+ 0, BalancerServiceImpl::BuildResponseForBackends(GetBackendPorts(), {}),
+ 0);
+ const size_t kNumRpcs = num_backends_ * 2;
+ CheckRpcSendOk(kNumRpcs, 1000 /* timeout_ms */, true /* wait_for_ready */);
+ balancers_[0]->NotifyDoneWithServerlists();
+ // Check that all requests went to the first backend. This verifies
+ // that we used pick_first instead of round_robin as the child policy.
+ EXPECT_EQ(backend_servers_[0].service_->request_count(), kNumRpcs);
+ for (size_t i = 1; i < backends_.size(); ++i) {
+ EXPECT_EQ(backend_servers_[i].service_->request_count(), 0UL);
+ }
+ // The balancer got a single request.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ // and sent a single response.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ // Check LB policy name for the channel.
+ EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
+}
+
+TEST_F(SingleBalancerTest, SwapChildPolicy) {
+ SetNextResolutionAllBalancers(
+ "{\n"
+ " \"loadBalancingConfig\":[\n"
+ " { \"grpclb\":{\n"
+ " \"childPolicy\":[\n"
+ " { \"pick_first\":{} }\n"
+ " ]\n"
+ " } }\n"
+ " ]\n"
+ "}");
+ ScheduleResponseForBalancer(
+ 0, BalancerServiceImpl::BuildResponseForBackends(GetBackendPorts(), {}),
+ 0);
+ const size_t kNumRpcs = num_backends_ * 2;
+ CheckRpcSendOk(kNumRpcs, 1000 /* timeout_ms */, true /* wait_for_ready */);
+ // Check that all requests went to the first backend. This verifies
+ // that we used pick_first instead of round_robin as the child policy.
+ EXPECT_EQ(backend_servers_[0].service_->request_count(), kNumRpcs);
+ for (size_t i = 1; i < backends_.size(); ++i) {
+ EXPECT_EQ(backend_servers_[i].service_->request_count(), 0UL);
+ }
+ // Send new resolution that removes child policy from service config.
+ SetNextResolutionAllBalancers("{}");
+ WaitForAllBackends();
+ CheckRpcSendOk(kNumRpcs, 1000 /* timeout_ms */, true /* wait_for_ready */);
+ // Check that every backend saw the same number of requests. This verifies
+ // that we used round_robin.
+ for (size_t i = 0; i < backends_.size(); ++i) {
+ EXPECT_EQ(backend_servers_[i].service_->request_count(), 2UL);
+ }
+ // Done.
+ balancers_[0]->NotifyDoneWithServerlists();
+ // The balancer got a single request.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ // and sent a single response.
+ EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ // Check LB policy name for the channel.
+ EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
+}
+
+TEST_F(SingleBalancerTest, UpdatesGoToMostRecentChildPolicy) {
+ const int kFallbackTimeoutMs = 200 * grpc_test_slowdown_factor();
+ ResetStub(kFallbackTimeoutMs);
+ int unreachable_balancer_port = grpc_pick_unused_port_or_die();
+ int unreachable_backend_port = grpc_pick_unused_port_or_die();
+ // Phase 1: Start with RR pointing to first backend.
+ gpr_log(GPR_INFO, "PHASE 1: Initial setup with RR with first backend");
+ SetNextResolution(
+ {
+ // Unreachable balancer.
+ {unreachable_balancer_port, true, ""},
+ // Fallback address: first backend.
+ {backend_servers_[0].port_, false, ""},
+ },
+ "{\n"
+ " \"loadBalancingConfig\":[\n"
+ " { \"grpclb\":{\n"
+ " \"childPolicy\":[\n"
+ " { \"round_robin\":{} }\n"
+ " ]\n"
+ " } }\n"
+ " ]\n"
+ "}");
+ // RPCs should go to first backend.
+ WaitForBackend(0);
+ // Phase 2: Switch to PF pointing to unreachable backend.
+ gpr_log(GPR_INFO, "PHASE 2: Update to use PF with unreachable backend");
+ SetNextResolution(
+ {
+ // Unreachable balancer.
+ {unreachable_balancer_port, true, ""},
+ // Fallback address: unreachable backend.
+ {unreachable_backend_port, false, ""},
+ },
+ "{\n"
+ " \"loadBalancingConfig\":[\n"
+ " { \"grpclb\":{\n"
+ " \"childPolicy\":[\n"
+ " { \"pick_first\":{} }\n"
+ " ]\n"
+ " } }\n"
+ " ]\n"
+ "}");
+ // RPCs should continue to go to the first backend, because the new
+ // PF child policy will never go into state READY.
+ WaitForBackend(0);
+ // Phase 3: Switch back to RR pointing to second and third backends.
+ // This ensures that we create a new policy rather than updating the
+ // pending PF policy.
+ gpr_log(GPR_INFO, "PHASE 3: Update to use RR again with two backends");
+ SetNextResolution(
+ {
+ // Unreachable balancer.
+ {unreachable_balancer_port, true, ""},
+ // Fallback address: second and third backends.
+ {backend_servers_[1].port_, false, ""},
+ {backend_servers_[2].port_, false, ""},
+ },
+ "{\n"
+ " \"loadBalancingConfig\":[\n"
+ " { \"grpclb\":{\n"
+ " \"childPolicy\":[\n"
+ " { \"round_robin\":{} }\n"
+ " ]\n"
+ " } }\n"
+ " ]\n"
+ "}");
+ // RPCs should go to the second and third backends.
+ WaitForBackend(1);
+ WaitForBackend(2);
+}
+
TEST_F(SingleBalancerTest, SameBackendListedMultipleTimes) {
SetNextResolutionAllBalancers();
// Same backend listed twice.
From 8369b0552986dcdbaccda924357a1e92295a230b Mon Sep 17 00:00:00 2001
From: yang-g <yangg@google.com>
Date: Tue, 5 Mar 2019 14:31:55 -0800
Subject: [PATCH 027/138] Fix a memory leak
---
src/core/lib/compression/stream_compression_gzip.cc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/core/lib/compression/stream_compression_gzip.cc b/src/core/lib/compression/stream_compression_gzip.cc
index 682f712843a..bffdb1fd17d 100644
--- a/src/core/lib/compression/stream_compression_gzip.cc
+++ b/src/core/lib/compression/stream_compression_gzip.cc
@@ -60,7 +60,7 @@ static bool gzip_flate(grpc_stream_compression_context_gzip* ctx,
if (r < 0 && r != Z_BUF_ERROR) {
gpr_log(GPR_ERROR, "zlib error (%d)", r);
grpc_slice_unref_internal(slice_out);
-
+ grpc_slice_unref_internal(slice);
return false;
} else if (r == Z_STREAM_END && ctx->flate == inflate) {
eoc = true;
From 94c38ee0a13d0e2fc0931c56db7dfcede6200da1 Mon Sep 17 00:00:00 2001
From: yang-g <yangg@google.com>
Date: Tue, 5 Mar 2019 14:34:24 -0800
Subject: [PATCH 028/138] Add fuzzer test case
---
...imized-grpc_client_fuzzer-5765697914404864 | Bin 0 -> 1066 bytes
tools/run_tests/generated/tests.json | 23 ++++++++++++++++++
2 files changed, 23 insertions(+)
create mode 100644 test/core/end2end/fuzzers/client_fuzzer_corpus/clusterfuzz-testcase-minimized-grpc_client_fuzzer-5765697914404864
diff --git a/test/core/end2end/fuzzers/client_fuzzer_corpus/clusterfuzz-testcase-minimized-grpc_client_fuzzer-5765697914404864 b/test/core/end2end/fuzzers/client_fuzzer_corpus/clusterfuzz-testcase-minimized-grpc_client_fuzzer-5765697914404864
new file mode 100644
index 0000000000000000000000000000000000000000..e8a60f5a9b532e916ab42651c02a16dcf0c56aed
GIT binary patch
literal 1066
zcmZQzU|?YY0~QWOIUs|P!H|`KDWjyMfDtS#*bkx@C;ItmPK@=N=r?g9kg0C(r#?~B
zPiy7Gi3|Pw{FoRR9C*@;7`Pc2_c82)0Y-)}>m@aAKKo>WpiWkn>(nf3*@7}NSy`xx
zc|h99(a{m60UOOAl3rAhtecpeoLW$#o0^xLp8|H)fDm+aEHBvGVE<<3rL&}0Wfl<h
za$06iDnUggDiBD{&nrpIg9bg(P9f12nkW~9n98V6ZPSC=zVZ`m<}eArwyz8TaNv7?
literal 0
HcmV?d00001
diff --git a/tools/run_tests/generated/tests.json b/tools/run_tests/generated/tests.json
index 6dc9eb7f0d1..fc875b74023 100644
--- a/tools/run_tests/generated/tests.json
+++ b/tools/run_tests/generated/tests.json
@@ -137454,6 +137454,29 @@
],
"uses_polling": false
},
+ {
+ "args": [
+ "test/core/end2end/fuzzers/client_fuzzer_corpus/clusterfuzz-testcase-minimized-grpc_client_fuzzer-5765697914404864"
+ ],
+ "ci_platforms": [
+ "linux"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [
+ "tsan"
+ ],
+ "exclude_iomgrs": [
+ "uv"
+ ],
+ "flaky": false,
+ "language": "c",
+ "name": "client_fuzzer_one_entry",
+ "platforms": [
+ "mac",
+ "linux"
+ ],
+ "uses_polling": false
+ },
{
"args": [
"test/core/end2end/fuzzers/client_fuzzer_corpus/crash-12b69708d452b3cefe2da4a708a1030a661d37fc"
From 49c6d5044b0fbd6799972ba7ef2534bb9955bd28 Mon Sep 17 00:00:00 2001
From: "Mark D. Roth" <roth@google.com>
Date: Tue, 5 Mar 2019 14:46:16 -0800
Subject: [PATCH 029/138] Add lock for channelz access outside of the combiner.
---
.../client_channel/lb_policy/grpclb/grpclb.cc | 44 ++++++++++++++-----
1 file changed, 32 insertions(+), 12 deletions(-)
diff --git a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
index c1f2846f046..e21b1789172 100644
--- a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
@@ -352,6 +352,9 @@ class GrpcLb : public LoadBalancingPolicy {
grpc_timer lb_fallback_timer_;
grpc_closure lb_on_fallback_;
+ // Lock held when modifying the value of child_policy_ or
+ // pending_child_policy_.
+ gpr_mu child_policy_mu_;
// The child policy to use for the backends.
OrphanablePtr<LoadBalancingPolicy> child_policy_;
// When switching child policies, the new policy will be stored here
@@ -618,6 +621,7 @@ void GrpcLb::Helper::UpdateState(grpc_connectivity_state state,
GRPC_ERROR_UNREF(state_error);
return;
}
+ MutexLock lock(&parent_->child_policy_mu_);
parent_->child_policy_ = std::move(parent_->pending_child_policy_);
} else if (!CalledByCurrentChild()) {
// This request is from an outdated child, so ignore it.
@@ -1216,6 +1220,7 @@ GrpcLb::GrpcLb(Args args)
.set_jitter(GRPC_GRPCLB_RECONNECT_JITTER)
.set_max_backoff(GRPC_GRPCLB_RECONNECT_MAX_BACKOFF_SECONDS *
1000)) {
+ gpr_mu_init(&child_policy_mu_);
// Record server name.
const grpc_arg* arg = grpc_channel_args_find(args.args, GRPC_ARG_SERVER_URI);
const char* server_uri = grpc_channel_arg_get_string(arg);
@@ -1241,6 +1246,7 @@ GrpcLb::GrpcLb(Args args)
GrpcLb::~GrpcLb() {
gpr_free((void*)server_name_);
grpc_channel_args_destroy(args_);
+ gpr_mu_destroy(&child_policy_mu_);
}
void GrpcLb::ShutdownLocked() {
@@ -1252,8 +1258,11 @@ void GrpcLb::ShutdownLocked() {
if (fallback_timer_callback_pending_) {
grpc_timer_cancel(&lb_fallback_timer_);
}
- child_policy_.reset();
- pending_child_policy_.reset();
+ {
+ MutexLock lock(&child_policy_mu_);
+ child_policy_.reset();
+ pending_child_policy_.reset();
+ }
// We destroy the LB channel here instead of in our destructor because
// destroying the channel triggers a last callback to
// OnBalancerChannelConnectivityChangedLocked(), and we need to be
@@ -1284,13 +1293,19 @@ void GrpcLb::ResetBackoffLocked() {
void GrpcLb::FillChildRefsForChannelz(
channelz::ChildRefsList* child_subchannels,
channelz::ChildRefsList* child_channels) {
- // delegate to the child policy to fill the children subchannels.
- if (child_policy_ != nullptr) {
- child_policy_->FillChildRefsForChannelz(child_subchannels, child_channels);
- }
- if (pending_child_policy_ != nullptr) {
- pending_child_policy_->FillChildRefsForChannelz(child_subchannels,
- child_channels);
+ {
+ // Delegate to the child policy to fill the children subchannels.
+ // This must be done holding child_policy_mu_, since this method
+ // does not run in the combiner.
+ MutexLock lock(&child_policy_mu_);
+ if (child_policy_ != nullptr) {
+ child_policy_->FillChildRefsForChannelz(child_subchannels,
+ child_channels);
+ }
+ if (pending_child_policy_ != nullptr) {
+ pending_child_policy_->FillChildRefsForChannelz(child_subchannels,
+ child_channels);
+ }
}
gpr_atm uuid = gpr_atm_no_barrier_load(&lb_channel_uuid_);
if (uuid != 0) {
@@ -1625,13 +1640,18 @@ void GrpcLb::CreateOrUpdateChildPolicyLocked() {
// Cases 1, 2b, and 3b: create a new child policy.
// If child_policy_ is null, we set it (case 1), else we set
// pending_child_policy_ (cases 2b and 3b).
- auto& lb_policy =
- child_policy_ == nullptr ? child_policy_ : pending_child_policy_;
if (grpc_lb_glb_trace.enabled()) {
gpr_log(GPR_INFO, "[grpclb %p] Creating new %schild policy %s", this,
child_policy_ == nullptr ? "" : "pending ", child_policy_name);
}
- lb_policy = CreateChildPolicyLocked(child_policy_name, args);
+ auto new_policy = CreateChildPolicyLocked(child_policy_name, args);
+ // Swap the policy into place.
+ auto& lb_policy =
+ child_policy_ == nullptr ? child_policy_ : pending_child_policy_;
+ {
+ MutexLock lock(&child_policy_mu_);
+ lb_policy = std::move(new_policy);
+ }
policy_to_update = lb_policy.get();
} else {
// Cases 2a and 3a: update an existing policy.
From 0c8418c4bfb831cb1f7781fc36410d3908429ac9 Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Tue, 5 Mar 2019 15:08:23 -0800
Subject: [PATCH 030/138] WIP. Start BUILD file
---
examples/python/multiprocessing/BUILD | 28 +++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/examples/python/multiprocessing/BUILD b/examples/python/multiprocessing/BUILD
index e69de29bb2d..e47fb654aac 100644
--- a/examples/python/multiprocessing/BUILD
+++ b/examples/python/multiprocessing/BUILD
@@ -0,0 +1,28 @@
+load("@grpc_python_dependencies//:requirements.bzl", "requirement")
+
+py_binary(
+ name = "client",
+ testonly = 1,
+ srcs = ["client.py"],
+ deps = [
+ "//src/python/grpcio/grpc:grpcio"
+ ],
+)
+
+py_binary(
+ name = "server",
+ testonly = 1,
+ srcs = ["server.py"],
+ deps = [
+ "//src/python/grpcio/grpc:grpcio"
+ ],
+)
+
+py_test(
+ name = "_multiprocessing_example_test",
+ srcs = ["test/_multiprocessing_example_test.py"],
+ data = [
+ ":client",
+ ":server"
+ ]
+)
From 57759a3525b9c708933ec86da2d907342cc09077 Mon Sep 17 00:00:00 2001
From: Prashant Jaikumar <pjaikumar@google.com>
Date: Tue, 5 Mar 2019 15:25:37 -0800
Subject: [PATCH 031/138] Added automated test for network transitions on iOS
devices
---
.../GrpcIosTest.xcodeproj/project.pbxproj | 135 +++++++++++++-
.../GrpcIosTestUITests/GrpcIosTestUITests.m | 174 ++++++++++++++++++
.../GrpcIosTestUITests/Info.plist | 22 +++
src/objective-c/manual_tests/Main.storyboard | 1 +
src/objective-c/manual_tests/Podfile | 4 +-
src/objective-c/manual_tests/ViewController.m | 22 ++-
src/objective-c/manual_tests/main.m | 2 +
7 files changed, 352 insertions(+), 8 deletions(-)
create mode 100644 src/objective-c/manual_tests/GrpcIosTestUITests/GrpcIosTestUITests.m
create mode 100644 src/objective-c/manual_tests/GrpcIosTestUITests/Info.plist
diff --git a/src/objective-c/manual_tests/GrpcIosTest.xcodeproj/project.pbxproj b/src/objective-c/manual_tests/GrpcIosTest.xcodeproj/project.pbxproj
index 9063719aa2e..3ad16c3af6e 100644
--- a/src/objective-c/manual_tests/GrpcIosTest.xcodeproj/project.pbxproj
+++ b/src/objective-c/manual_tests/GrpcIosTest.xcodeproj/project.pbxproj
@@ -12,8 +12,19 @@
5EDA909C220DF1B00046D27A /* main.m in Sources */ = {isa = PBXBuildFile; fileRef = 5EDA9096220DF1B00046D27A /* main.m */; };
5EDA909E220DF1B00046D27A /* Main.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 5EDA9098220DF1B00046D27A /* Main.storyboard */; };
5EDA909F220DF1B00046D27A /* AppDelegate.m in Sources */ = {isa = PBXBuildFile; fileRef = 5EDA9099220DF1B00046D27A /* AppDelegate.m */; };
+ B0C18CA7222DEF140002B502 /* GrpcIosTestUITests.m in Sources */ = {isa = PBXBuildFile; fileRef = B0C18CA6222DEF140002B502 /* GrpcIosTestUITests.m */; };
/* End PBXBuildFile section */
+/* Begin PBXContainerItemProxy section */
+ B0C18CA9222DEF140002B502 /* PBXContainerItemProxy */ = {
+ isa = PBXContainerItemProxy;
+ containerPortal = 5EDA9073220DF0BC0046D27A /* Project object */;
+ proxyType = 1;
+ remoteGlobalIDString = 5EDA907A220DF0BC0046D27A;
+ remoteInfo = GrpcIosTest;
+ };
+/* End PBXContainerItemProxy section */
+
/* Begin PBXFileReference section */
1D22EC48A487B02F76135EA3 /* libPods-GrpcIosTest.a */ = {isa = PBXFileReference; explicitFileType = archive.ar; includeInIndex = 0; path = "libPods-GrpcIosTest.a"; sourceTree = BUILT_PRODUCTS_DIR; };
5EDA907B220DF0BC0046D27A /* GrpcIosTest.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = GrpcIosTest.app; sourceTree = BUILT_PRODUCTS_DIR; };
@@ -24,6 +35,9 @@
5EDA9099220DF1B00046D27A /* AppDelegate.m */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.objc; path = AppDelegate.m; sourceTree = SOURCE_ROOT; };
7C9FAFB11727DCA50888C1B8 /* Pods-GrpcIosTest.debug.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-GrpcIosTest.debug.xcconfig"; path = "Pods/Target Support Files/Pods-GrpcIosTest/Pods-GrpcIosTest.debug.xcconfig"; sourceTree = "<group>"; };
A4E7CA72304A7B43FE8A5BC7 /* Pods-GrpcIosTest.release.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-GrpcIosTest.release.xcconfig"; path = "Pods/Target Support Files/Pods-GrpcIosTest/Pods-GrpcIosTest.release.xcconfig"; sourceTree = "<group>"; };
+ B0C18CA4222DEF140002B502 /* GrpcIosTestUITests.xctest */ = {isa = PBXFileReference; explicitFileType = wrapper.cfbundle; includeInIndex = 0; path = GrpcIosTestUITests.xctest; sourceTree = BUILT_PRODUCTS_DIR; };
+ B0C18CA6222DEF140002B502 /* GrpcIosTestUITests.m */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.objc; path = GrpcIosTestUITests.m; sourceTree = "<group>"; };
+ B0C18CA8222DEF140002B502 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = "<group>"; };
/* End PBXFileReference section */
/* Begin PBXFrameworksBuildPhase section */
@@ -35,6 +49,13 @@
);
runOnlyForDeploymentPostprocessing = 0;
};
+ B0C18CA1222DEF140002B502 /* Frameworks */ = {
+ isa = PBXFrameworksBuildPhase;
+ buildActionMask = 2147483647;
+ files = (
+ );
+ runOnlyForDeploymentPostprocessing = 0;
+ };
/* End PBXFrameworksBuildPhase section */
/* Begin PBXGroup section */
@@ -55,6 +76,7 @@
5EDA9096220DF1B00046D27A /* main.m */,
5EDA9098220DF1B00046D27A /* Main.storyboard */,
5EDA9094220DF1B00046D27A /* ViewController.m */,
+ B0C18CA5222DEF140002B502 /* GrpcIosTestUITests */,
5EDA907C220DF0BC0046D27A /* Products */,
2B8131AC634883AFEC02557C /* Pods */,
E73D92116C1C328622A8C77F /* Frameworks */,
@@ -65,10 +87,20 @@
isa = PBXGroup;
children = (
5EDA907B220DF0BC0046D27A /* GrpcIosTest.app */,
+ B0C18CA4222DEF140002B502 /* GrpcIosTestUITests.xctest */,
);
name = Products;
sourceTree = "<group>";
};
+ B0C18CA5222DEF140002B502 /* GrpcIosTestUITests */ = {
+ isa = PBXGroup;
+ children = (
+ B0C18CA6222DEF140002B502 /* GrpcIosTestUITests.m */,
+ B0C18CA8222DEF140002B502 /* Info.plist */,
+ );
+ path = GrpcIosTestUITests;
+ sourceTree = "<group>";
+ };
E73D92116C1C328622A8C77F /* Frameworks */ = {
isa = PBXGroup;
children = (
@@ -99,6 +131,24 @@
productReference = 5EDA907B220DF0BC0046D27A /* GrpcIosTest.app */;
productType = "com.apple.product-type.application";
};
+ B0C18CA3222DEF140002B502 /* GrpcIosTestUITests */ = {
+ isa = PBXNativeTarget;
+ buildConfigurationList = B0C18CAD222DEF140002B502 /* Build configuration list for PBXNativeTarget "GrpcIosTestUITests" */;
+ buildPhases = (
+ B0C18CA0222DEF140002B502 /* Sources */,
+ B0C18CA1222DEF140002B502 /* Frameworks */,
+ B0C18CA2222DEF140002B502 /* Resources */,
+ );
+ buildRules = (
+ );
+ dependencies = (
+ B0C18CAA222DEF140002B502 /* PBXTargetDependency */,
+ );
+ name = GrpcIosTestUITests;
+ productName = GrpcIosTestUITests;
+ productReference = B0C18CA4222DEF140002B502 /* GrpcIosTestUITests.xctest */;
+ productType = "com.apple.product-type.bundle.ui-testing";
+ };
/* End PBXNativeTarget section */
/* Begin PBXProject section */
@@ -111,6 +161,10 @@
5EDA907A220DF0BC0046D27A = {
CreatedOnToolsVersion = 10.0;
};
+ B0C18CA3222DEF140002B502 = {
+ CreatedOnToolsVersion = 10.0;
+ TestTargetID = 5EDA907A220DF0BC0046D27A;
+ };
};
};
buildConfigurationList = 5EDA9076220DF0BC0046D27A /* Build configuration list for PBXProject "GrpcIosTest" */;
@@ -127,6 +181,7 @@
projectRoot = "";
targets = (
5EDA907A220DF0BC0046D27A /* GrpcIosTest */,
+ B0C18CA3222DEF140002B502 /* GrpcIosTestUITests */,
);
};
/* End PBXProject section */
@@ -140,6 +195,13 @@
);
runOnlyForDeploymentPostprocessing = 0;
};
+ B0C18CA2222DEF140002B502 /* Resources */ = {
+ isa = PBXResourcesBuildPhase;
+ buildActionMask = 2147483647;
+ files = (
+ );
+ runOnlyForDeploymentPostprocessing = 0;
+ };
/* End PBXResourcesBuildPhase section */
/* Begin PBXShellScriptBuildPhase section */
@@ -200,8 +262,24 @@
);
runOnlyForDeploymentPostprocessing = 0;
};
+ B0C18CA0222DEF140002B502 /* Sources */ = {
+ isa = PBXSourcesBuildPhase;
+ buildActionMask = 2147483647;
+ files = (
+ B0C18CA7222DEF140002B502 /* GrpcIosTestUITests.m in Sources */,
+ );
+ runOnlyForDeploymentPostprocessing = 0;
+ };
/* End PBXSourcesBuildPhase section */
+/* Begin PBXTargetDependency section */
+ B0C18CAA222DEF140002B502 /* PBXTargetDependency */ = {
+ isa = PBXTargetDependency;
+ target = 5EDA907A220DF0BC0046D27A /* GrpcIosTest */;
+ targetProxy = B0C18CA9222DEF140002B502 /* PBXContainerItemProxy */;
+ };
+/* End PBXTargetDependency section */
+
/* Begin XCBuildConfiguration section */
5EDA908F220DF0BD0046D27A /* Debug */ = {
isa = XCBuildConfiguration;
@@ -321,7 +399,7 @@
buildSettings = {
ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon;
CODE_SIGN_STYLE = Manual;
- DEVELOPMENT_TEAM = "";
+ DEVELOPMENT_TEAM = EQHXZ8M8AV;
INFOPLIST_FILE = Info.plist;
IPHONEOS_DEPLOYMENT_TARGET = 11.0;
LD_RUNPATH_SEARCH_PATHS = (
@@ -330,7 +408,7 @@
);
PRODUCT_BUNDLE_IDENTIFIER = io.grpc.GrpcIosTest;
PRODUCT_NAME = "$(TARGET_NAME)";
- PROVISIONING_PROFILE_SPECIFIER = "";
+ PROVISIONING_PROFILE_SPECIFIER = "Google Development";
TARGETED_DEVICE_FAMILY = "1,2";
};
name = Debug;
@@ -341,7 +419,7 @@
buildSettings = {
ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon;
CODE_SIGN_STYLE = Manual;
- DEVELOPMENT_TEAM = "";
+ DEVELOPMENT_TEAM = EQHXZ8M8AV;
INFOPLIST_FILE = Info.plist;
IPHONEOS_DEPLOYMENT_TARGET = 11.0;
LD_RUNPATH_SEARCH_PATHS = (
@@ -350,11 +428,51 @@
);
PRODUCT_BUNDLE_IDENTIFIER = io.grpc.GrpcIosTest;
PRODUCT_NAME = "$(TARGET_NAME)";
- PROVISIONING_PROFILE_SPECIFIER = "";
+ PROVISIONING_PROFILE_SPECIFIER = "Google Development";
TARGETED_DEVICE_FAMILY = "1,2";
};
name = Release;
};
+ B0C18CAB222DEF140002B502 /* Debug */ = {
+ isa = XCBuildConfiguration;
+ buildSettings = {
+ CODE_SIGN_STYLE = Manual;
+ DEVELOPMENT_TEAM = EQHXZ8M8AV;
+ INFOPLIST_FILE = GrpcIosTestUITests/Info.plist;
+ LD_RUNPATH_SEARCH_PATHS = (
+ "$(inherited)",
+ "@executable_path/Frameworks",
+ "@loader_path/Frameworks",
+ );
+ PRODUCT_BUNDLE_IDENTIFIER = com.google.GrpcIosTestUITests;
+ PRODUCT_NAME = "$(TARGET_NAME)";
+ PROVISIONING_PROFILE = "";
+ PROVISIONING_PROFILE_SPECIFIER = "Google Development";
+ TARGETED_DEVICE_FAMILY = "1,2";
+ TEST_TARGET_NAME = GrpcIosTest;
+ };
+ name = Debug;
+ };
+ B0C18CAC222DEF140002B502 /* Release */ = {
+ isa = XCBuildConfiguration;
+ buildSettings = {
+ CODE_SIGN_STYLE = Manual;
+ DEVELOPMENT_TEAM = EQHXZ8M8AV;
+ INFOPLIST_FILE = GrpcIosTestUITests/Info.plist;
+ LD_RUNPATH_SEARCH_PATHS = (
+ "$(inherited)",
+ "@executable_path/Frameworks",
+ "@loader_path/Frameworks",
+ );
+ PRODUCT_BUNDLE_IDENTIFIER = com.google.GrpcIosTestUITests;
+ PRODUCT_NAME = "$(TARGET_NAME)";
+ PROVISIONING_PROFILE = "";
+ PROVISIONING_PROFILE_SPECIFIER = "Google Development";
+ TARGETED_DEVICE_FAMILY = "1,2";
+ TEST_TARGET_NAME = GrpcIosTest;
+ };
+ name = Release;
+ };
/* End XCBuildConfiguration section */
/* Begin XCConfigurationList section */
@@ -376,6 +494,15 @@
defaultConfigurationIsVisible = 0;
defaultConfigurationName = Release;
};
+ B0C18CAD222DEF140002B502 /* Build configuration list for PBXNativeTarget "GrpcIosTestUITests" */ = {
+ isa = XCConfigurationList;
+ buildConfigurations = (
+ B0C18CAB222DEF140002B502 /* Debug */,
+ B0C18CAC222DEF140002B502 /* Release */,
+ );
+ defaultConfigurationIsVisible = 0;
+ defaultConfigurationName = Release;
+ };
/* End XCConfigurationList section */
};
rootObject = 5EDA9073220DF0BC0046D27A /* Project object */;
diff --git a/src/objective-c/manual_tests/GrpcIosTestUITests/GrpcIosTestUITests.m b/src/objective-c/manual_tests/GrpcIosTestUITests/GrpcIosTestUITests.m
new file mode 100644
index 00000000000..b0a929e689d
--- /dev/null
+++ b/src/objective-c/manual_tests/GrpcIosTestUITests/GrpcIosTestUITests.m
@@ -0,0 +1,174 @@
+/*
+ *
+ * Copyright 2019 gRPC authors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+#import <XCTest/XCTest.h>
+
+NSTimeInterval const kWaitTime = 30;
+
+@interface GrpcIosTestUITests : XCTestCase
+@end
+
+@implementation GrpcIosTestUITests {
+ XCUIApplication *testApp;
+ XCUIApplication *settingsApp;
+}
+
+- (void)setUp {
+ self.continueAfterFailure = NO;
+ [[[XCUIApplication alloc] init] launch];
+ testApp = [[XCUIApplication alloc] initWithBundleIdentifier:@"io.grpc.GrpcIosTest"];
+ settingsApp = [[XCUIApplication alloc] initWithBundleIdentifier:@"com.apple.Preferences"];
+ [settingsApp activate];
+ // Go back to the first page of Settings.
+ XCUIElement *backButton = settingsApp.navigationBars.buttons.firstMatch;
+ while (backButton.exists) {
+ [backButton tap];
+ }
+ XCTAssert([settingsApp.navigationBars[@"Settings"] waitForExistenceWithTimeout:kWaitTime]);
+ // Turn off airplane mode
+ [self setAirplaneMode:NO];
+}
+
+- (void)tearDown {
+}
+
+- (void)doUnaryCall {
+ [testApp activate];
+ [testApp.buttons[@"Unary call"] tap];
+}
+
+- (void)doStreamingCall {
+ [testApp activate];
+ [testApp.buttons[@"Start streaming call"] tap];
+ [testApp.buttons[@"Send Message"] tap];
+ [testApp.buttons[@"Stop streaming call"] tap];
+}
+
+- (void)expectCallSuccess {
+ XCTAssert([testApp.staticTexts[@"Call done"] waitForExistenceWithTimeout:kWaitTime]);
+}
+
+- (void)expectCallFailed {
+ XCTAssert([testApp.staticTexts[@"Call failed"] waitForExistenceWithTimeout:kWaitTime]);
+}
+
+- (void)setAirplaneMode:(BOOL)to {
+ [settingsApp activate];
+ XCUIElement *mySwitch = settingsApp.tables.element.cells.switches[@"Airplane Mode"];
+ BOOL from = [(NSString *)mySwitch.value boolValue];
+ if (from != to) {
+ [mySwitch tap];
+ // wait for gRPC to detect the change
+ sleep(10);
+ }
+ XCTAssert([(NSString *)mySwitch.value boolValue] == to);
+}
+
+- (void)testBackgroundBeforeUnaryCall {
+ // Open test app
+ [testApp activate];
+
+ // Send test app to background
+ [XCUIDevice.sharedDevice pressButton:XCUIDeviceButtonHome];
+ sleep(5);
+
+ // Bring test app to foreground and make a unary call. Call should succeed
+ [self doUnaryCall];
+ [self expectCallSuccess];
+}
+
+- (void)testBackgroundBeforeStreamingCall {
+ // Open test app
+ [testApp activate];
+
+ // Send test app to background
+ [XCUIDevice.sharedDevice pressButton:XCUIDeviceButtonHome];
+ sleep(5);
+
+ // Bring test app to foreground and make a streaming call. Call should succeed.
+ [self doStreamingCall];
+ [self expectCallSuccess];
+}
+
+- (void)testUnaryCallAfterNetworkFlap {
+ // Open test app and make a unary call. Channel to server should be open after this.
+ [self doUnaryCall];
+ [self expectCallSuccess];
+
+ // Toggle airplane mode on and off
+ [self setAirplaneMode:YES];
+ [self setAirplaneMode:NO];
+
+ // Bring test app to foreground and make a unary call. The call should succeed
+ [self doUnaryCall];
+ [self expectCallSuccess];
+}
+
+- (void)testStreamingCallAfterNetworkFlap {
+ // Open test app and make a unary call. Channel to server should be open after this.
+ [self doUnaryCall];
+ [self expectCallSuccess];
+
+ // Toggle airplane mode on and off
+ [self setAirplaneMode:YES];
+ [self setAirplaneMode:NO];
+
+ [self doStreamingCall];
+ [self expectCallSuccess];
+}
+
+- (void)testUnaryCallWhileNetworkDown {
+ // Open test app and make a unary call. Channel to server should be open after this.
+ [self doUnaryCall];
+ [self expectCallSuccess];
+
+ // Turn on airplane mode
+ [self setAirplaneMode:YES];
+
+ // Unary call should fail
+ [self doUnaryCall];
+ [self expectCallFailed];
+
+ // Turn off airplane mode
+ [self setAirplaneMode:NO];
+
+ // Unary call should succeed
+ [self doUnaryCall];
+ [self expectCallSuccess];
+}
+
+- (void)testStreamingCallWhileNetworkDown {
+ // Open test app and make a unary call. Channel to server should be open after this.
+ [self doUnaryCall];
+ [self expectCallSuccess];
+
+ // Turn on airplane mode
+ [self setAirplaneMode:YES];
+
+ // Streaming call should fail
+ [self doStreamingCall];
+ [self expectCallFailed];
+
+ // Turn off airplane mode
+ [self setAirplaneMode:NO];
+
+ // Unary call should succeed
+ [self doStreamingCall];
+ [self expectCallSuccess];
+}
+@end
diff --git a/src/objective-c/manual_tests/GrpcIosTestUITests/Info.plist b/src/objective-c/manual_tests/GrpcIosTestUITests/Info.plist
new file mode 100644
index 00000000000..6c40a6cd0c4
--- /dev/null
+++ b/src/objective-c/manual_tests/GrpcIosTestUITests/Info.plist
@@ -0,0 +1,22 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+ <key>CFBundleDevelopmentRegion</key>
+ <string>$(DEVELOPMENT_LANGUAGE)</string>
+ <key>CFBundleExecutable</key>
+ <string>$(EXECUTABLE_NAME)</string>
+ <key>CFBundleIdentifier</key>
+ <string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
+ <key>CFBundleInfoDictionaryVersion</key>
+ <string>6.0</string>
+ <key>CFBundleName</key>
+ <string>$(PRODUCT_NAME)</string>
+ <key>CFBundlePackageType</key>
+ <string>BNDL</string>
+ <key>CFBundleShortVersionString</key>
+ <string>1.0</string>
+ <key>CFBundleVersion</key>
+ <string>1</string>
+</dict>
+</plist>
diff --git a/src/objective-c/manual_tests/Main.storyboard b/src/objective-c/manual_tests/Main.storyboard
index e88f30e324b..e7e0530efcd 100644
--- a/src/objective-c/manual_tests/Main.storyboard
+++ b/src/objective-c/manual_tests/Main.storyboard
@@ -4,6 +4,7 @@
<adaptation id="fullscreen"/>
</device>
<dependencies>
+ <deployment identifier="iOS"/>
<plugIn identifier="com.apple.InterfaceBuilder.IBCocoaTouchPlugin" version="14283.14"/>
<capability name="Safe area layout guides" minToolsVersion="9.0"/>
<capability name="documents saved in the Xcode 8 format" minToolsVersion="8.0"/>
diff --git a/src/objective-c/manual_tests/Podfile b/src/objective-c/manual_tests/Podfile
index 7cb650a3412..919e649be71 100644
--- a/src/objective-c/manual_tests/Podfile
+++ b/src/objective-c/manual_tests/Podfile
@@ -18,8 +18,8 @@ GrpcIosTest
pod 'BoringSSL-GRPC', :podspec => "#{GRPC_LOCAL_SRC}/src/objective-c", :inhibit_warnings => true
- pod 'gRPC', :path => GRPC_LOCAL_SRC
- pod 'gRPC-Core', :path => GRPC_LOCAL_SRC
+ pod 'gRPC/CFStream', :path => GRPC_LOCAL_SRC
+ pod 'gRPC-Core/CFStream-Implementation', :path => GRPC_LOCAL_SRC
pod 'gRPC-RxLibrary', :path => GRPC_LOCAL_SRC
pod 'gRPC-ProtoRPC', :path => GRPC_LOCAL_SRC, :inhibit_warnings => true
pod 'RemoteTest', :path => "../tests/RemoteTestClient", :inhibit_warnings => true
diff --git a/src/objective-c/manual_tests/ViewController.m b/src/objective-c/manual_tests/ViewController.m
index 00bb516bdfc..813b176f3e8 100644
--- a/src/objective-c/manual_tests/ViewController.m
+++ b/src/objective-c/manual_tests/ViewController.m
@@ -27,7 +27,7 @@ NSString *const kRemoteHost = @"grpc-test.sandbox.googleapis.com";
const int32_t kMessageSize = 100;
@interface ViewController : UIViewController<GRPCProtoResponseHandler>
-
+@property(strong, nonatomic) UILabel *fromLabel;
@end
@implementation ViewController {
@@ -35,16 +35,25 @@ const int32_t kMessageSize = 100;
dispatch_queue_t _dispatchQueue;
GRPCStreamingProtoCall *_call;
}
+- (instancetype)init {
+ self = [super init];
+ return self;
+}
- (void)viewDidLoad {
[super viewDidLoad];
_dispatchQueue = dispatch_queue_create(NULL, DISPATCH_QUEUE_SERIAL);
+ _fromLabel = [[UILabel alloc] initWithFrame:CGRectMake(100, 500, 200, 20)];
+ _fromLabel.textColor = [UIColor blueColor];
+ _fromLabel.backgroundColor = [UIColor whiteColor];
+ [self.view addSubview:_fromLabel];
}
- (IBAction)tapUnaryCall:(id)sender {
if (_service == nil) {
_service = [RMTTestService serviceWithHost:kRemoteHost];
}
+ self->_fromLabel.text = @"";
// Set up request proto message
RMTSimpleRequest *request = [RMTSimpleRequest message];
@@ -61,6 +70,7 @@ const int32_t kMessageSize = 100;
if (_service == nil) {
_service = [RMTTestService serviceWithHost:kRemoteHost];
}
+ self->_fromLabel.text = @"";
// Set up request proto message
RMTStreamingOutputCallRequest *request = RMTStreamingOutputCallRequest.message;
@@ -92,7 +102,6 @@ const int32_t kMessageSize = 100;
if (_call == nil) return;
[_call finish];
-
_call = nil;
}
@@ -107,6 +116,15 @@ const int32_t kMessageSize = 100;
- (void)didCloseWithTrailingMetadata:(NSDictionary *)trailingMetadata
error:(nullable NSError *)error {
NSLog(@"Recv trailing metadata: %@, error: %@", trailingMetadata, error);
+ if (error == nil) {
+ dispatch_async(dispatch_get_main_queue(), ^{
+ self->_fromLabel.text = @"Call done";
+ });
+ } else {
+ dispatch_async(dispatch_get_main_queue(), ^{
+ self->_fromLabel.text = @"Call failed";
+ });
+ }
}
- (dispatch_queue_t)dispatchQueue {
diff --git a/src/objective-c/manual_tests/main.m b/src/objective-c/manual_tests/main.m
index 2797c6f17f2..451b50cc0e2 100644
--- a/src/objective-c/manual_tests/main.m
+++ b/src/objective-c/manual_tests/main.m
@@ -21,6 +21,8 @@
int main(int argc, char* argv[]) {
@autoreleasepool {
+ // enable CFStream API
+ setenv("grpc_cfstream", "1", 1);
return UIApplicationMain(argc, argv, nil, NSStringFromClass([AppDelegate class]));
}
}
From 07bfbec8f9c55961bc797f6ad25f04a612ca1bb5 Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Tue, 5 Mar 2019 16:09:31 -0800
Subject: [PATCH 032/138] Fix hanging build
---
src/python/grpcio/commands.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/src/python/grpcio/commands.py b/src/python/grpcio/commands.py
index 27b98362c11..d6353fc5a07 100644
--- a/src/python/grpcio/commands.py
+++ b/src/python/grpcio/commands.py
@@ -217,11 +217,15 @@ class BuildExt(build_ext.build_ext):
"""Test if default compiler is okay with specifying c++ version
when invoked in C mode. GCC is okay with this, while clang is not.
"""
+ print("Checking if compiler okay")
cc_test = subprocess.Popen(
['cc', '-x', 'c', '-std=c++11', '-'],
+ stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
- _, cc_err = cc_test.communicate(input='int main(){return 0;}')
+ print("Attempting to communicate")
+ _, cc_err = cc_test.communicate(input=b'int main(){return 0;}')
+ print("Completed with compiler")
return not 'invalid argument' in str(cc_err)
# This special conditioning is here due to difference of compiler
From efa1f8b993eaf6061e8b54a8f3dcd7089b709f48 Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Tue, 5 Mar 2019 16:11:05 -0800
Subject: [PATCH 033/138] Remove debug prints
---
src/python/grpcio/commands.py | 3 ---
1 file changed, 3 deletions(-)
diff --git a/src/python/grpcio/commands.py b/src/python/grpcio/commands.py
index d6353fc5a07..d189c2869d0 100644
--- a/src/python/grpcio/commands.py
+++ b/src/python/grpcio/commands.py
@@ -217,15 +217,12 @@ class BuildExt(build_ext.build_ext):
"""Test if default compiler is okay with specifying c++ version
when invoked in C mode. GCC is okay with this, while clang is not.
"""
- print("Checking if compiler okay")
cc_test = subprocess.Popen(
['cc', '-x', 'c', '-std=c++11', '-'],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
- print("Attempting to communicate")
_, cc_err = cc_test.communicate(input=b'int main(){return 0;}')
- print("Completed with compiler")
return not 'invalid argument' in str(cc_err)
# This special conditioning is here due to difference of compiler
From 04609b1ea5aa743182f0d910d86d054a113f60e6 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@users.noreply.github.com>
Date: Wed, 6 Mar 2019 11:00:13 +0100
Subject: [PATCH 034/138] Revert "Strip Python wheel binary"
---
test/distrib/python/test_packages.sh | 2 +-
.../artifacts/build_package_python.sh | 18 ------------------
2 files changed, 1 insertion(+), 19 deletions(-)
diff --git a/test/distrib/python/test_packages.sh b/test/distrib/python/test_packages.sh
index 433148e6bd7..755daa10211 100755
--- a/test/distrib/python/test_packages.sh
+++ b/test/distrib/python/test_packages.sh
@@ -41,7 +41,7 @@ PYTHON=$VIRTUAL_ENV/bin/python
function at_least_one_installs() {
for file in "$@"; do
- if "$PYTHON" -m pip install --require-hashes "$file"; then
+ if "$PYTHON" -m pip install "$file"; then
return 0
fi
done
diff --git a/tools/run_tests/artifacts/build_package_python.sh b/tools/run_tests/artifacts/build_package_python.sh
index 193d75db62a..29801a5b867 100755
--- a/tools/run_tests/artifacts/build_package_python.sh
+++ b/tools/run_tests/artifacts/build_package_python.sh
@@ -23,24 +23,6 @@ mkdir -p artifacts/
# and we only collect them here to deliver them to the distribtest phase.
cp -r "${EXTERNAL_GIT_ROOT}"/input_artifacts/python_*/* artifacts/ || true
-strip_binary_wheel() {
- WHEEL_PATH="$1"
- TEMP_WHEEL_DIR=$(mktemp -d)
- wheel unpack "$WHEEL_PATH" -d "$TEMP_WHEEL_DIR"
- find "$TEMP_WHEEL_DIR" -name "_protoc_compiler*.so" -exec strip --strip-debug {} ";"
- find "$TEMP_WHEEL_DIR" -name "cygrpc*.so" -exec strip --strip-debug {} ";"
-
- WHEEL_FILE=$(basename "$WHEEL_PATH")
- DISTRIBUTION_NAME=$(basename "$WHEEL_PATH" | cut -d '-' -f 1)
- VERSION=$(basename "$WHEEL_PATH" | cut -d '-' -f 2)
- wheel pack "$TEMP_WHEEL_DIR/$DISTRIBUTION_NAME-$VERSION" -d "$TEMP_WHEEL_DIR"
- mv "$TEMP_WHEEL_DIR/$WHEEL_FILE" "$WHEEL_PATH"
-}
-
-for wheel in artifacts/*.whl; do
- strip_binary_wheel "$wheel"
-done
-
# TODO: all the artifact builder configurations generate a grpcio-VERSION.tar.gz
# source distribution package, and only one of them will end up
# in the artifacts/ directory. They should be all equivalent though.
From 50a1ddab5c6126f8f91c16e78e9595c3388f7f13 Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Wed, 6 Mar 2019 07:57:21 -0800
Subject: [PATCH 035/138] Revert "Revert "Strip Python wheel binary""
This reverts commit 04609b1ea5aa743182f0d910d86d054a113f60e6.
---
test/distrib/python/test_packages.sh | 2 +-
.../artifacts/build_package_python.sh | 18 ++++++++++++++++++
2 files changed, 19 insertions(+), 1 deletion(-)
diff --git a/test/distrib/python/test_packages.sh b/test/distrib/python/test_packages.sh
index 755daa10211..433148e6bd7 100755
--- a/test/distrib/python/test_packages.sh
+++ b/test/distrib/python/test_packages.sh
@@ -41,7 +41,7 @@ PYTHON=$VIRTUAL_ENV/bin/python
function at_least_one_installs() {
for file in "$@"; do
- if "$PYTHON" -m pip install "$file"; then
+ if "$PYTHON" -m pip install --require-hashes "$file"; then
return 0
fi
done
diff --git a/tools/run_tests/artifacts/build_package_python.sh b/tools/run_tests/artifacts/build_package_python.sh
index 29801a5b867..193d75db62a 100755
--- a/tools/run_tests/artifacts/build_package_python.sh
+++ b/tools/run_tests/artifacts/build_package_python.sh
@@ -23,6 +23,24 @@ mkdir -p artifacts/
# and we only collect them here to deliver them to the distribtest phase.
cp -r "${EXTERNAL_GIT_ROOT}"/input_artifacts/python_*/* artifacts/ || true
+strip_binary_wheel() {
+ WHEEL_PATH="$1"
+ TEMP_WHEEL_DIR=$(mktemp -d)
+ wheel unpack "$WHEEL_PATH" -d "$TEMP_WHEEL_DIR"
+ find "$TEMP_WHEEL_DIR" -name "_protoc_compiler*.so" -exec strip --strip-debug {} ";"
+ find "$TEMP_WHEEL_DIR" -name "cygrpc*.so" -exec strip --strip-debug {} ";"
+
+ WHEEL_FILE=$(basename "$WHEEL_PATH")
+ DISTRIBUTION_NAME=$(basename "$WHEEL_PATH" | cut -d '-' -f 1)
+ VERSION=$(basename "$WHEEL_PATH" | cut -d '-' -f 2)
+ wheel pack "$TEMP_WHEEL_DIR/$DISTRIBUTION_NAME-$VERSION" -d "$TEMP_WHEEL_DIR"
+ mv "$TEMP_WHEEL_DIR/$WHEEL_FILE" "$WHEEL_PATH"
+}
+
+for wheel in artifacts/*.whl; do
+ strip_binary_wheel "$wheel"
+done
+
# TODO: all the artifact builder configurations generate a grpcio-VERSION.tar.gz
# source distribution package, and only one of them will end up
# in the artifacts/ directory. They should be all equivalent though.
From 8326ac64e4035ad93458bdcbfd861dfd569fecd3 Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Wed, 6 Mar 2019 08:11:46 -0800
Subject: [PATCH 036/138] Remove GIL for grpc_call_unref
---
src/python/grpcio/grpc/_cython/_cygrpc/call.pyx.pxi | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/call.pyx.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/call.pyx.pxi
index 0a31d9c52ff..6e4574af8d5 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/call.pyx.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/call.pyx.pxi
@@ -85,9 +85,10 @@ cdef class Call:
return result
def __dealloc__(self):
- if self.c_call != NULL:
- grpc_call_unref(self.c_call)
- grpc_shutdown_blocking()
+ with nogil:
+ if self.c_call != NULL:
+ grpc_call_unref(self.c_call)
+ grpc_shutdown_blocking()
# The object *should* always be valid from Python. Used for debugging.
@property
From f7dd48b2b67a28973577e0e6ba206059c5edc645 Mon Sep 17 00:00:00 2001
From: Karthik Ravi Shankar <karthikrs@google.com>
Date: Fri, 1 Mar 2019 13:32:36 -0800
Subject: [PATCH 037/138] Moving ::grpc::ResourceQuota to
::grpc_impl::ResouceQuota
This change moves ResourceQuota class fron grpc namespace to grpc_impl
namespace.
---
BUILD | 3 +-
CMakeLists.txt | 3 +
Makefile | 3 +
build.yaml | 1 +
gRPC-C++.podspec | 1 +
include/grpcpp/resource_quota.h | 45 +-----------
include/grpcpp/resource_quota_impl.h | 68 +++++++++++++++++++
include/grpcpp/server_builder.h | 9 ++-
include/grpcpp/support/channel_arguments.h | 9 ++-
src/cpp/common/channel_arguments.cc | 2 +-
src/cpp/common/resource_quota_cc.cc | 4 +-
src/cpp/server/server_builder.cc | 7 +-
test/cpp/end2end/end2end_test.cc | 5 ++
test/cpp/end2end/thread_stress_test.cc | 5 ++
test/cpp/qps/server.h | 5 ++
test/cpp/qps/server_async.cc | 1 -
tools/doxygen/Doxyfile.c++ | 1 +
tools/doxygen/Doxyfile.c++.internal | 1 +
.../generated/sources_and_headers.json | 2 +
19 files changed, 122 insertions(+), 53 deletions(-)
create mode 100644 include/grpcpp/resource_quota_impl.h
diff --git a/BUILD b/BUILD
index 24c1fb31ced..2b90f82aa00 100644
--- a/BUILD
+++ b/BUILD
@@ -192,8 +192,8 @@ GRPCXX_PUBLIC_HDRS = [
"include/grpc++/impl/service_type.h",
"include/grpc++/impl/sync_cxx11.h",
"include/grpc++/impl/sync_no_cxx11.h",
- "include/grpc++/resource_quota.h",
"include/grpc++/security/auth_context.h",
+ "include/grpc++/resource_quota.h",
"include/grpc++/security/auth_metadata_processor.h",
"include/grpc++/security/credentials.h",
"include/grpc++/security/server_credentials.h",
@@ -241,6 +241,7 @@ GRPCXX_PUBLIC_HDRS = [
"include/grpcpp/impl/sync_cxx11.h",
"include/grpcpp/impl/sync_no_cxx11.h",
"include/grpcpp/resource_quota.h",
+ "include/grpcpp/resource_quota_impl.h",
"include/grpcpp/security/auth_context.h",
"include/grpcpp/security/auth_metadata_processor.h",
"include/grpcpp/security/credentials.h",
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 1bab5e6cba2..14a1bdee404 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -3021,6 +3021,7 @@ foreach(_hdr
include/grpcpp/impl/server_initializer.h
include/grpcpp/impl/service_type.h
include/grpcpp/resource_quota.h
+ include/grpcpp/resource_quota_impl.h
include/grpcpp/security/auth_context.h
include/grpcpp/security/auth_metadata_processor.h
include/grpcpp/security/credentials.h
@@ -3612,6 +3613,7 @@ foreach(_hdr
include/grpcpp/impl/server_initializer.h
include/grpcpp/impl/service_type.h
include/grpcpp/resource_quota.h
+ include/grpcpp/resource_quota_impl.h
include/grpcpp/security/auth_context.h
include/grpcpp/security/auth_metadata_processor.h
include/grpcpp/security/credentials.h
@@ -4567,6 +4569,7 @@ foreach(_hdr
include/grpcpp/impl/server_initializer.h
include/grpcpp/impl/service_type.h
include/grpcpp/resource_quota.h
+ include/grpcpp/resource_quota_impl.h
include/grpcpp/security/auth_context.h
include/grpcpp/security/auth_metadata_processor.h
include/grpcpp/security/credentials.h
diff --git a/Makefile b/Makefile
index a2789e40431..4a95766d398 100644
--- a/Makefile
+++ b/Makefile
@@ -5442,6 +5442,7 @@ PUBLIC_HEADERS_CXX += \
include/grpcpp/impl/server_initializer.h \
include/grpcpp/impl/service_type.h \
include/grpcpp/resource_quota.h \
+ include/grpcpp/resource_quota_impl.h \
include/grpcpp/security/auth_context.h \
include/grpcpp/security/auth_metadata_processor.h \
include/grpcpp/security/credentials.h \
@@ -6042,6 +6043,7 @@ PUBLIC_HEADERS_CXX += \
include/grpcpp/impl/server_initializer.h \
include/grpcpp/impl/service_type.h \
include/grpcpp/resource_quota.h \
+ include/grpcpp/resource_quota_impl.h \
include/grpcpp/security/auth_context.h \
include/grpcpp/security/auth_metadata_processor.h \
include/grpcpp/security/credentials.h \
@@ -6954,6 +6956,7 @@ PUBLIC_HEADERS_CXX += \
include/grpcpp/impl/server_initializer.h \
include/grpcpp/impl/service_type.h \
include/grpcpp/resource_quota.h \
+ include/grpcpp/resource_quota_impl.h \
include/grpcpp/security/auth_context.h \
include/grpcpp/security/auth_metadata_processor.h \
include/grpcpp/security/credentials.h \
diff --git a/build.yaml b/build.yaml
index c18630ecdd3..8e5fb57801e 100644
--- a/build.yaml
+++ b/build.yaml
@@ -1362,6 +1362,7 @@ filegroups:
- include/grpcpp/impl/server_initializer.h
- include/grpcpp/impl/service_type.h
- include/grpcpp/resource_quota.h
+ - include/grpcpp/resource_quota_impl.h
- include/grpcpp/security/auth_context.h
- include/grpcpp/security/auth_metadata_processor.h
- include/grpcpp/security/credentials.h
diff --git a/gRPC-C++.podspec b/gRPC-C++.podspec
index 0f3888975c9..a4cc2b2b26d 100644
--- a/gRPC-C++.podspec
+++ b/gRPC-C++.podspec
@@ -105,6 +105,7 @@ Pod::Spec.new do |s|
'include/grpcpp/impl/server_initializer.h',
'include/grpcpp/impl/service_type.h',
'include/grpcpp/resource_quota.h',
+ 'include/grpcpp/resource_quota_impl.h',
'include/grpcpp/security/auth_context.h',
'include/grpcpp/security/auth_metadata_processor.h',
'include/grpcpp/security/credentials.h',
diff --git a/include/grpcpp/resource_quota.h b/include/grpcpp/resource_quota.h
index 50bd1cb849a..333767b95c5 100644
--- a/include/grpcpp/resource_quota.h
+++ b/include/grpcpp/resource_quota.h
@@ -1,6 +1,6 @@
/*
*
- * Copyright 2016 gRPC authors.
+ * Copyright 2019 gRPC authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -19,50 +19,11 @@
#ifndef GRPCPP_RESOURCE_QUOTA_H
#define GRPCPP_RESOURCE_QUOTA_H
-struct grpc_resource_quota;
-
-#include <grpcpp/impl/codegen/config.h>
-#include <grpcpp/impl/codegen/grpc_library.h>
+#include <grpcpp/resource_quota_impl.h>
namespace grpc {
-/// ResourceQuota represents a bound on memory and thread usage by the gRPC
-/// library. A ResourceQuota can be attached to a server (via \a ServerBuilder),
-/// or a client channel (via \a ChannelArguments).
-/// gRPC will attempt to keep memory and threads used by all attached entities
-/// below the ResourceQuota bound.
-class ResourceQuota final : private GrpcLibraryCodegen {
- public:
- /// \param name - a unique name for this ResourceQuota.
- explicit ResourceQuota(const grpc::string& name);
- ResourceQuota();
- ~ResourceQuota();
-
- /// Resize this \a ResourceQuota to a new size. If \a new_size is smaller
- /// than the current size of the pool, memory usage will be monotonically
- /// decreased until it falls under \a new_size.
- /// No time bound is given for this to occur however.
- ResourceQuota& Resize(size_t new_size);
-
- /// Set the max number of threads that can be allocated from this
- /// ResourceQuota object.
- ///
- /// If the new_max_threads value is smaller than the current value, no new
- /// threads are allocated until the number of active threads fall below
- /// new_max_threads. There is no time bound on when this may happen i.e none
- /// of the current threads are forcefully destroyed and all threads run their
- /// normal course.
- ResourceQuota& SetMaxThreads(int new_max_threads);
-
- grpc_resource_quota* c_resource_quota() const { return impl_; }
-
- private:
- ResourceQuota(const ResourceQuota& rhs);
- ResourceQuota& operator=(const ResourceQuota& rhs);
-
- grpc_resource_quota* const impl_;
-};
-
+typedef ::grpc_impl::ResourceQuota ResourceQuota;
} // namespace grpc
#endif // GRPCPP_RESOURCE_QUOTA_H
diff --git a/include/grpcpp/resource_quota_impl.h b/include/grpcpp/resource_quota_impl.h
new file mode 100644
index 00000000000..16c0e35385b
--- /dev/null
+++ b/include/grpcpp/resource_quota_impl.h
@@ -0,0 +1,68 @@
+/*
+ *
+ * Copyright 2016 gRPC authors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+#ifndef GRPCPP_RESOURCE_QUOTA_IMPL_H
+#define GRPCPP_RESOURCE_QUOTA_IMPL_H
+
+struct grpc_resource_quota;
+
+#include <grpcpp/impl/codegen/config.h>
+#include <grpcpp/impl/codegen/grpc_library.h>
+
+namespace grpc_impl {
+
+/// ResourceQuota represents a bound on memory and thread usage by the gRPC
+/// library. A ResourceQuota can be attached to a server (via \a ServerBuilder),
+/// or a client channel (via \a ChannelArguments).
+/// gRPC will attempt to keep memory and threads used by all attached entities
+/// below the ResourceQuota bound.
+class ResourceQuota final : private ::grpc::GrpcLibraryCodegen {
+ public:
+ /// \param name - a unique name for this ResourceQuota.
+ explicit ResourceQuota(const grpc::string& name);
+ ResourceQuota();
+ ~ResourceQuota();
+
+ /// Resize this \a ResourceQuota to a new size. If \a new_size is smaller
+ /// than the current size of the pool, memory usage will be monotonically
+ /// decreased until it falls under \a new_size.
+ /// No time bound is given for this to occur however.
+ ResourceQuota& Resize(size_t new_size);
+
+ /// Set the max number of threads that can be allocated from this
+ /// ResourceQuota object.
+ ///
+ /// If the new_max_threads value is smaller than the current value, no new
+ /// threads are allocated until the number of active threads fall below
+ /// new_max_threads. There is no time bound on when this may happen i.e none
+ /// of the current threads are forcefully destroyed and all threads run their
+ /// normal course.
+ ResourceQuota& SetMaxThreads(int new_max_threads);
+
+ grpc_resource_quota* c_resource_quota() const { return impl_; }
+
+ private:
+ ResourceQuota(const ResourceQuota& rhs);
+ ResourceQuota& operator=(const ResourceQuota& rhs);
+
+ grpc_resource_quota* const impl_;
+};
+
+} // namespace grpc_impl
+
+#endif // GRPCPP_RESOURCE_QUOTA_IMPL_H
diff --git a/include/grpcpp/server_builder.h b/include/grpcpp/server_builder.h
index 028b8cffaa7..ce5f3ef89c9 100644
--- a/include/grpcpp/server_builder.h
+++ b/include/grpcpp/server_builder.h
@@ -35,10 +35,14 @@
struct grpc_resource_quota;
+namespace grpc_impl {
+
+class ResourceQuota;
+}
+
namespace grpc {
class AsyncGenericService;
-class ResourceQuota;
class CompletionQueue;
class Server;
class ServerCompletionQueue;
@@ -182,7 +186,8 @@ class ServerBuilder {
grpc_compression_algorithm algorithm);
/// Set the attached buffer pool for this server
- ServerBuilder& SetResourceQuota(const ResourceQuota& resource_quota);
+ ServerBuilder& SetResourceQuota(
+ const ::grpc_impl::ResourceQuota& resource_quota);
ServerBuilder& SetOption(std::unique_ptr<ServerBuilderOption> option);
diff --git a/include/grpcpp/support/channel_arguments.h b/include/grpcpp/support/channel_arguments.h
index 217929d4aca..48ae4246462 100644
--- a/include/grpcpp/support/channel_arguments.h
+++ b/include/grpcpp/support/channel_arguments.h
@@ -26,13 +26,16 @@
#include <grpc/grpc.h>
#include <grpcpp/support/config.h>
+namespace grpc_impl {
+
+class ResourceQuota;
+}
+
namespace grpc {
namespace testing {
class ChannelArgumentsTest;
} // namespace testing
-class ResourceQuota;
-
/// Options for channel creation. The user can use generic setters to pass
/// key value pairs down to C channel creation code. For gRPC related options,
/// concrete setters are provided.
@@ -83,7 +86,7 @@ class ChannelArguments {
void SetUserAgentPrefix(const grpc::string& user_agent_prefix);
/// Set the buffer pool to be attached to the constructed channel.
- void SetResourceQuota(const ResourceQuota& resource_quota);
+ void SetResourceQuota(const ::grpc_impl::ResourceQuota& resource_quota);
/// Set the max receive and send message sizes.
void SetMaxReceiveMessageSize(int size);
diff --git a/src/cpp/common/channel_arguments.cc b/src/cpp/common/channel_arguments.cc
index 214d72f853f..c3d75054b9b 100644
--- a/src/cpp/common/channel_arguments.cc
+++ b/src/cpp/common/channel_arguments.cc
@@ -143,7 +143,7 @@ void ChannelArguments::SetUserAgentPrefix(
}
void ChannelArguments::SetResourceQuota(
- const grpc::ResourceQuota& resource_quota) {
+ const grpc_impl::ResourceQuota& resource_quota) {
SetPointerWithVtable(GRPC_ARG_RESOURCE_QUOTA,
resource_quota.c_resource_quota(),
grpc_resource_quota_arg_vtable());
diff --git a/src/cpp/common/resource_quota_cc.cc b/src/cpp/common/resource_quota_cc.cc
index 276e5f79548..4fab2975d89 100644
--- a/src/cpp/common/resource_quota_cc.cc
+++ b/src/cpp/common/resource_quota_cc.cc
@@ -19,7 +19,7 @@
#include <grpc/grpc.h>
#include <grpcpp/resource_quota.h>
-namespace grpc {
+namespace grpc_impl {
ResourceQuota::ResourceQuota() : impl_(grpc_resource_quota_create(nullptr)) {}
@@ -37,4 +37,4 @@ ResourceQuota& ResourceQuota::SetMaxThreads(int new_max_threads) {
grpc_resource_quota_set_max_threads(impl_, new_max_threads);
return *this;
}
-} // namespace grpc
+} // namespace grpc_impl
diff --git a/src/cpp/server/server_builder.cc b/src/cpp/server/server_builder.cc
index b7fad558abb..a8ea514d785 100644
--- a/src/cpp/server/server_builder.cc
+++ b/src/cpp/server/server_builder.cc
@@ -29,6 +29,11 @@
#include "src/core/lib/gpr/useful.h"
#include "src/cpp/server/thread_pool_interface.h"
+namespace grpc_impl {
+
+class ResourceQuota;
+}
+
namespace grpc {
static std::vector<std::unique_ptr<ServerBuilderPlugin> (*)()>*
@@ -152,7 +157,7 @@ ServerBuilder& ServerBuilder::SetDefaultCompressionAlgorithm(
}
ServerBuilder& ServerBuilder::SetResourceQuota(
- const grpc::ResourceQuota& resource_quota) {
+ const grpc_impl::ResourceQuota& resource_quota) {
if (resource_quota_ != nullptr) {
grpc_resource_quota_unref(resource_quota_);
}
diff --git a/test/cpp/end2end/end2end_test.cc b/test/cpp/end2end/end2end_test.cc
index f58a472bfaf..f7b9ee4b0b0 100644
--- a/test/cpp/end2end/end2end_test.cc
+++ b/test/cpp/end2end/end2end_test.cc
@@ -64,6 +64,11 @@ using std::chrono::system_clock;
} \
} while (0)
+namespace grpc_impl {
+
+class ResourceQuota;
+}
+
namespace grpc {
namespace testing {
namespace {
diff --git a/test/cpp/end2end/thread_stress_test.cc b/test/cpp/end2end/thread_stress_test.cc
index e30ce0dbcbf..e308e591d1a 100644
--- a/test/cpp/end2end/thread_stress_test.cc
+++ b/test/cpp/end2end/thread_stress_test.cc
@@ -48,6 +48,11 @@ const int kNumAsyncReceiveThreads = 50;
const int kNumAsyncServerThreads = 50;
const int kNumRpcs = 1000; // Number of RPCs per thread
+namespace grpc_impl {
+
+class ResourceQuota;
+}
+
namespace grpc {
namespace testing {
diff --git a/test/cpp/qps/server.h b/test/cpp/qps/server.h
index 89b0e3af4b2..3aec8644a94 100644
--- a/test/cpp/qps/server.h
+++ b/test/cpp/qps/server.h
@@ -34,6 +34,11 @@
#include "test/cpp/qps/usage_timer.h"
#include "test/cpp/util/test_credentials_provider.h"
+namespace grpc_impl {
+
+class ResourceQuota;
+}
+
namespace grpc {
namespace testing {
diff --git a/test/cpp/qps/server_async.cc b/test/cpp/qps/server_async.cc
index a5f8347c269..9343fd311e1 100644
--- a/test/cpp/qps/server_async.cc
+++ b/test/cpp/qps/server_async.cc
@@ -27,7 +27,6 @@
#include <grpc/support/alloc.h>
#include <grpc/support/log.h>
#include <grpcpp/generic/async_generic_service.h>
-#include <grpcpp/resource_quota.h>
#include <grpcpp/security/server_credentials.h>
#include <grpcpp/server.h>
#include <grpcpp/server_builder.h>
diff --git a/tools/doxygen/Doxyfile.c++ b/tools/doxygen/Doxyfile.c++
index 9f17a25298a..367160a0ca9 100644
--- a/tools/doxygen/Doxyfile.c++
+++ b/tools/doxygen/Doxyfile.c++
@@ -995,6 +995,7 @@ include/grpcpp/impl/server_builder_plugin.h \
include/grpcpp/impl/server_initializer.h \
include/grpcpp/impl/service_type.h \
include/grpcpp/resource_quota.h \
+include/grpcpp/resource_quota_impl.h \
include/grpcpp/security/auth_context.h \
include/grpcpp/security/auth_metadata_processor.h \
include/grpcpp/security/credentials.h \
diff --git a/tools/doxygen/Doxyfile.c++.internal b/tools/doxygen/Doxyfile.c++.internal
index 664a6b3acfe..62be71a0f72 100644
--- a/tools/doxygen/Doxyfile.c++.internal
+++ b/tools/doxygen/Doxyfile.c++.internal
@@ -997,6 +997,7 @@ include/grpcpp/impl/server_builder_plugin.h \
include/grpcpp/impl/server_initializer.h \
include/grpcpp/impl/service_type.h \
include/grpcpp/resource_quota.h \
+include/grpcpp/resource_quota_impl.h \
include/grpcpp/security/auth_context.h \
include/grpcpp/security/auth_metadata_processor.h \
include/grpcpp/security/credentials.h \
diff --git a/tools/run_tests/generated/sources_and_headers.json b/tools/run_tests/generated/sources_and_headers.json
index 95b6ae65008..682ecf9bc3e 100644
--- a/tools/run_tests/generated/sources_and_headers.json
+++ b/tools/run_tests/generated/sources_and_headers.json
@@ -11386,6 +11386,7 @@
"include/grpcpp/impl/server_initializer.h",
"include/grpcpp/impl/service_type.h",
"include/grpcpp/resource_quota.h",
+ "include/grpcpp/resource_quota_impl.h",
"include/grpcpp/security/auth_context.h",
"include/grpcpp/security/auth_metadata_processor.h",
"include/grpcpp/security/credentials.h",
@@ -11495,6 +11496,7 @@
"include/grpcpp/impl/server_initializer.h",
"include/grpcpp/impl/service_type.h",
"include/grpcpp/resource_quota.h",
+ "include/grpcpp/resource_quota_impl.h",
"include/grpcpp/security/auth_context.h",
"include/grpcpp/security/auth_metadata_processor.h",
"include/grpcpp/security/credentials.h",
From 5e5e337b033baf097b3edefd351695e8e6d34763 Mon Sep 17 00:00:00 2001
From: Karthik Ravi Shankar <karthikrs@google.com>
Date: Mon, 4 Mar 2019 17:56:03 -0800
Subject: [PATCH 038/138] fix from clang format code
Add fixes from clang_format_code.sh.
---
src/cpp/ext/filters/census/views.cc | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/src/cpp/ext/filters/census/views.cc b/src/cpp/ext/filters/census/views.cc
index 102745ab4c2..d7e3c81a955 100644
--- a/src/cpp/ext/filters/census/views.cc
+++ b/src/cpp/ext/filters/census/views.cc
@@ -41,7 +41,7 @@ void RegisterOpenCensusViewsForExport() {
grpc::ServerReceivedBytesPerRpcCumulative().RegisterForExport();
grpc::ServerServerLatencyCumulative().RegisterForExport();
}
-}
+} // namespace grpc_impl
namespace grpc {
using ::opencensus::stats::Aggregation;
@@ -88,7 +88,6 @@ ViewDescriptor HourDescriptor() {
} // namespace
-
// client cumulative
const ViewDescriptor& ClientSentBytesPerRpcCumulative() {
const static ViewDescriptor descriptor =
From 548beec563be0c822a62f8d23173d79a112e05df Mon Sep 17 00:00:00 2001
From: Stanley Cheung <stanleycheung@google.com>
Date: Wed, 6 Mar 2019 10:35:38 -0800
Subject: [PATCH 039/138] Fix qps composer.json
---
src/php/tests/qps/composer.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/php/tests/qps/composer.json b/src/php/tests/qps/composer.json
index 0f4b0c04ba4..c2d73f00ed9 100644
--- a/src/php/tests/qps/composer.json
+++ b/src/php/tests/qps/composer.json
@@ -1,7 +1,7 @@
{
"require": {
"grpc/grpc": "dev-master",
- "google/protobuf": "v3.5.1.1"
+ "google/protobuf": "^v3.3.0"
},
"autoload": {
"psr-4": {
From d85e6f4e94f14acd9dc3b8ead09165812324f500 Mon Sep 17 00:00:00 2001
From: "Mark D. Roth" <roth@google.com>
Date: Wed, 6 Mar 2019 08:13:22 -0800
Subject: [PATCH 040/138] Make grpclb work when selected via service config
with no balancer addresses.
---
.../client_channel/lb_policy/grpclb/grpclb.cc | 12 ------
test/cpp/end2end/grpclb_end2end_test.cc | 41 +++++++++++++++++++
2 files changed, 41 insertions(+), 12 deletions(-)
diff --git a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
index e21b1789172..77cd1059398 100644
--- a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
@@ -1681,18 +1681,6 @@ class GrpcLbFactory : public LoadBalancingPolicyFactory {
public:
OrphanablePtr<LoadBalancingPolicy> CreateLoadBalancingPolicy(
LoadBalancingPolicy::Args args) const override {
- /* Count the number of gRPC-LB addresses. There must be at least one. */
- const ServerAddressList* addresses =
- FindServerAddressListChannelArg(args.args);
- if (addresses == nullptr) return nullptr;
- bool found_balancer = false;
- for (size_t i = 0; i < addresses->size(); ++i) {
- if ((*addresses)[i].IsBalancer()) {
- found_balancer = true;
- break;
- }
- }
- if (!found_balancer) return nullptr;
return OrphanablePtr<LoadBalancingPolicy>(New<GrpcLb>(std::move(args)));
}
diff --git a/test/cpp/end2end/grpclb_end2end_test.cc b/test/cpp/end2end/grpclb_end2end_test.cc
index 31353ba1304..fe865ed4b72 100644
--- a/test/cpp/end2end/grpclb_end2end_test.cc
+++ b/test/cpp/end2end/grpclb_end2end_test.cc
@@ -723,6 +723,47 @@ TEST_F(SingleBalancerTest, SelectGrpclbWithMigrationServiceConfig) {
EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
}
+TEST_F(SingleBalancerTest,
+ SelectGrpclbWithMigrationServiceConfigAndNoAddresses) {
+ const int kFallbackTimeoutMs = 200 * grpc_test_slowdown_factor();
+ ResetStub(kFallbackTimeoutMs);
+ SetNextResolution({},
+ "{\n"
+ " \"loadBalancingConfig\":[\n"
+ " { \"does_not_exist\":{} },\n"
+ " { \"grpclb\":{} }\n"
+ " ]\n"
+ "}");
+ // Try to connect.
+ EXPECT_EQ(GRPC_CHANNEL_IDLE, channel_->GetState(true));
+ // Should go into state TRANSIENT_FAILURE when we enter fallback mode.
+ const gpr_timespec deadline = grpc_timeout_seconds_to_deadline(1);
+ grpc_connectivity_state state;
+ while ((state = channel_->GetState(false)) !=
+ GRPC_CHANNEL_TRANSIENT_FAILURE) {
+ ASSERT_TRUE(channel_->WaitForStateChange(state, deadline));
+ }
+ // Check LB policy name for the channel.
+ EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
+}
+
+TEST_F(SingleBalancerTest,
+ SelectGrpclbWithMigrationServiceConfigAndNoBalancerAddresses) {
+ const int kFallbackTimeoutMs = 200 * grpc_test_slowdown_factor();
+ ResetStub(kFallbackTimeoutMs);
+ // Resolution includes fallback address but no balancers.
+ SetNextResolution({AddressData{backend_servers_[0].port_, false, ""}},
+ "{\n"
+ " \"loadBalancingConfig\":[\n"
+ " { \"does_not_exist\":{} },\n"
+ " { \"grpclb\":{} }\n"
+ " ]\n"
+ "}");
+ CheckRpcSendOk(1, 1000 /* timeout_ms */, true /* wait_for_ready */);
+ // Check LB policy name for the channel.
+ EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
+}
+
TEST_F(SingleBalancerTest, UsePickFirstChildPolicy) {
SetNextResolutionAllBalancers(
"{\n"
From cf70b744f1656b55a3db815685b1a2b195f36532 Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Wed, 6 Mar 2019 15:02:29 -0800
Subject: [PATCH 041/138] Silent the check_on_pr failure
---
tools/run_tests/python_utils/check_on_pr.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/run_tests/python_utils/check_on_pr.py b/tools/run_tests/python_utils/check_on_pr.py
index 8250dd76e02..fff455f7d5e 100644
--- a/tools/run_tests/python_utils/check_on_pr.py
+++ b/tools/run_tests/python_utils/check_on_pr.py
@@ -76,7 +76,7 @@ def _access_token():
time.sleep(_ACCESS_TOKEN_FETCH_RETRIES_INTERVAL_S)
else:
print("error: Unable to fetch access token, exiting...")
- sys.exit(1)
+ sys.exit(0)
return _ACCESS_TOKEN_CACHE['token']
From 6b437ca80faf527bea5e3176af517f9325dc4426 Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Wed, 6 Mar 2019 15:30:33 -0800
Subject: [PATCH 042/138] Increase the retry interval
---
tools/run_tests/python_utils/check_on_pr.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/run_tests/python_utils/check_on_pr.py b/tools/run_tests/python_utils/check_on_pr.py
index fff455f7d5e..f2b0f24e4fe 100644
--- a/tools/run_tests/python_utils/check_on_pr.py
+++ b/tools/run_tests/python_utils/check_on_pr.py
@@ -29,8 +29,8 @@ _GITHUB_APP_ID = 22338
_INSTALLATION_ID = 519109
_ACCESS_TOKEN_CACHE = None
-_ACCESS_TOKEN_FETCH_RETRIES = 5
-_ACCESS_TOKEN_FETCH_RETRIES_INTERVAL_S = 1
+_ACCESS_TOKEN_FETCH_RETRIES = 6
+_ACCESS_TOKEN_FETCH_RETRIES_INTERVAL_S = 15
def _jwt_token():
From a6596b2fd51993e9001af5c35512eca1efd07e38 Mon Sep 17 00:00:00 2001
From: yang-g <yangg@google.com>
Date: Wed, 6 Mar 2019 15:43:12 -0800
Subject: [PATCH 043/138] Fix fuzzer test
---
src/core/lib/gpr/cpu_posix.cc | 5 ++-
...rpc_percent_decode_fuzzer-5652313562808320 | 1 +
test/core/slice/percent_decode_fuzzer.cc | 27 +++++++--------
test/core/slice/percent_encode_fuzzer.cc | 34 +++++++++----------
tools/run_tests/generated/tests.json | 23 +++++++++++++
5 files changed, 54 insertions(+), 36 deletions(-)
create mode 100644 test/core/slice/percent_decode_corpus/clusterfuzz-testcase-minimized-grpc_percent_decode_fuzzer-5652313562808320
diff --git a/src/core/lib/gpr/cpu_posix.cc b/src/core/lib/gpr/cpu_posix.cc
index 915fd4976c2..59f583e4a0d 100644
--- a/src/core/lib/gpr/cpu_posix.cc
+++ b/src/core/lib/gpr/cpu_posix.cc
@@ -25,7 +25,6 @@
#include <string.h>
#include <unistd.h>
-#include <grpc/support/alloc.h>
#include <grpc/support/cpu.h>
#include <grpc/support/log.h>
#include <grpc/support/sync.h>
@@ -52,7 +51,7 @@ unsigned gpr_cpu_num_cores(void) {
static void delete_thread_id(void* value) {
if (value) {
- gpr_free(value);
+ free(value);
}
}
@@ -71,7 +70,7 @@ unsigned gpr_cpu_current_cpu(void) {
unsigned int* thread_id =
static_cast<unsigned int*>(pthread_getspecific(thread_id_key));
if (thread_id == nullptr) {
- thread_id = static_cast<unsigned int*>(gpr_malloc(sizeof(unsigned int)));
+ thread_id = static_cast<unsigned int*>(malloc(sizeof(unsigned int)));
pthread_setspecific(thread_id_key, thread_id);
}
diff --git a/test/core/slice/percent_decode_corpus/clusterfuzz-testcase-minimized-grpc_percent_decode_fuzzer-5652313562808320 b/test/core/slice/percent_decode_corpus/clusterfuzz-testcase-minimized-grpc_percent_decode_fuzzer-5652313562808320
new file mode 100644
index 00000000000..797993a54d0
--- /dev/null
+++ b/test/core/slice/percent_decode_corpus/clusterfuzz-testcase-minimized-grpc_percent_decode_fuzzer-5652313562808320
@@ -0,0 +1 @@
+%c4%cc%c4%cc%cc%ccccc%cccc%ccc%ccc%cc%ccc%ccc
\ No newline at end of file
diff --git a/test/core/slice/percent_decode_fuzzer.cc b/test/core/slice/percent_decode_fuzzer.cc
index 11f71d92c46..da8e03a4662 100644
--- a/test/core/slice/percent_decode_fuzzer.cc
+++ b/test/core/slice/percent_decode_fuzzer.cc
@@ -31,23 +31,20 @@ bool squelch = true;
bool leak_check = true;
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
+ grpc_core::testing::LeakDetector leak_detector(true);
grpc_init();
- {
- grpc_core::testing::LeakDetector leak_detector(true);
- grpc_slice input = grpc_slice_from_copied_buffer((const char*)data, size);
- grpc_slice output;
- if (grpc_strict_percent_decode_slice(
- input, grpc_url_percent_encoding_unreserved_bytes, &output)) {
- grpc_slice_unref(output);
- }
- if (grpc_strict_percent_decode_slice(
- input, grpc_compatible_percent_encoding_unreserved_bytes,
- &output)) {
- grpc_slice_unref(output);
- }
- grpc_slice_unref(grpc_permissive_percent_decode_slice(input));
- grpc_slice_unref(input);
+ grpc_slice input = grpc_slice_from_copied_buffer((const char*)data, size);
+ grpc_slice output;
+ if (grpc_strict_percent_decode_slice(
+ input, grpc_url_percent_encoding_unreserved_bytes, &output)) {
+ grpc_slice_unref(output);
}
+ if (grpc_strict_percent_decode_slice(
+ input, grpc_compatible_percent_encoding_unreserved_bytes, &output)) {
+ grpc_slice_unref(output);
+ }
+ grpc_slice_unref(grpc_permissive_percent_decode_slice(input));
+ grpc_slice_unref(input);
grpc_shutdown_blocking();
return 0;
}
diff --git a/test/core/slice/percent_encode_fuzzer.cc b/test/core/slice/percent_encode_fuzzer.cc
index 1da982bba28..4efa7a8d8e7 100644
--- a/test/core/slice/percent_encode_fuzzer.cc
+++ b/test/core/slice/percent_encode_fuzzer.cc
@@ -31,25 +31,23 @@ bool squelch = true;
bool leak_check = true;
static void test(const uint8_t* data, size_t size, const uint8_t* dict) {
+ grpc_core::testing::LeakDetector leak_detector(true);
grpc_init();
- {
- grpc_core::testing::LeakDetector leak_detector(true);
- grpc_slice input = grpc_slice_from_copied_buffer(
- reinterpret_cast<const char*>(data), size);
- grpc_slice output = grpc_percent_encode_slice(input, dict);
- grpc_slice decoded_output;
- // encoder must always produce decodable output
- GPR_ASSERT(grpc_strict_percent_decode_slice(output, dict, &decoded_output));
- grpc_slice permissive_decoded_output =
- grpc_permissive_percent_decode_slice(output);
- // and decoded output must always match the input
- GPR_ASSERT(grpc_slice_eq(input, decoded_output));
- GPR_ASSERT(grpc_slice_eq(input, permissive_decoded_output));
- grpc_slice_unref(input);
- grpc_slice_unref(output);
- grpc_slice_unref(decoded_output);
- grpc_slice_unref(permissive_decoded_output);
- }
+ grpc_slice input =
+ grpc_slice_from_copied_buffer(reinterpret_cast<const char*>(data), size);
+ grpc_slice output = grpc_percent_encode_slice(input, dict);
+ grpc_slice decoded_output;
+ // encoder must always produce decodable output
+ GPR_ASSERT(grpc_strict_percent_decode_slice(output, dict, &decoded_output));
+ grpc_slice permissive_decoded_output =
+ grpc_permissive_percent_decode_slice(output);
+ // and decoded output must always match the input
+ GPR_ASSERT(grpc_slice_eq(input, decoded_output));
+ GPR_ASSERT(grpc_slice_eq(input, permissive_decoded_output));
+ grpc_slice_unref(input);
+ grpc_slice_unref(output);
+ grpc_slice_unref(decoded_output);
+ grpc_slice_unref(permissive_decoded_output);
grpc_shutdown_blocking();
}
diff --git a/tools/run_tests/generated/tests.json b/tools/run_tests/generated/tests.json
index 9df57b5e151..2e82995bc8e 100644
--- a/tools/run_tests/generated/tests.json
+++ b/tools/run_tests/generated/tests.json
@@ -169172,6 +169172,29 @@
],
"uses_polling": false
},
+ {
+ "args": [
+ "test/core/slice/percent_decode_corpus/clusterfuzz-testcase-minimized-grpc_percent_decode_fuzzer-5652313562808320"
+ ],
+ "ci_platforms": [
+ "linux"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [
+ "tsan"
+ ],
+ "exclude_iomgrs": [
+ "uv"
+ ],
+ "flaky": false,
+ "language": "c",
+ "name": "percent_decode_fuzzer_one_entry",
+ "platforms": [
+ "mac",
+ "linux"
+ ],
+ "uses_polling": false
+ },
{
"args": [
"test/core/slice/percent_decode_corpus/d5b2a7177339ba2b7ce2f60e5f4459bef1e72758"
From ab06853fc9a4e1a7bc58c9b6cc4aeaad1f9e666a Mon Sep 17 00:00:00 2001
From: billfeng327 <yfen@google.com>
Date: Wed, 6 Mar 2019 15:47:49 -0800
Subject: [PATCH 044/138] C++ Windows test builds
---
bazel/grpc_build_system.bzl | 16 +++++++++++++---
bazel/grpc_deps.bzl | 10 +++++-----
test/core/bad_connection/BUILD | 1 +
test/core/client_channel/BUILD | 1 +
test/core/end2end/generate_tests.bzl | 15 +++++++++++----
test/core/iomgr/BUILD | 10 ++++++++++
test/cpp/common/BUILD | 1 +
test/cpp/end2end/BUILD | 2 ++
test/cpp/interop/BUILD | 1 +
test/cpp/microbenchmarks/BUILD | 17 +++++++++++++++++
.../generate_resolver_component_tests.bzl | 5 ++++-
test/cpp/performance/BUILD | 1 +
test/cpp/qps/qps_benchmark_script.bzl | 1 +
test/cpp/server/BUILD | 3 +++
test/cpp/server/load_reporter/BUILD | 1 +
tools/remote_build/windows.bazelrc | 2 ++
16 files changed, 74 insertions(+), 13 deletions(-)
create mode 100644 tools/remote_build/windows.bazelrc
diff --git a/bazel/grpc_build_system.bzl b/bazel/grpc_build_system.bzl
index 3ea8e305ca5..c4f133ed6e5 100644
--- a/bazel/grpc_build_system.bzl
+++ b/bazel/grpc_build_system.bzl
@@ -28,6 +28,12 @@ load("//bazel:cc_grpc_library.bzl", "cc_grpc_library")
# The set of pollers to test against if a test exercises polling
POLLERS = ["epollex", "epoll1", "poll", "poll-cv"]
+def is_msvc():
+ return select({
+ "//:windows_msvc": True,
+ "//conditions:default": False,
+ })
+
def if_not_windows(a):
return select({
"//:windows": [],
@@ -80,7 +86,8 @@ def grpc_cc_library(
visibility = None,
alwayslink = 0,
data = [],
- use_cfstream = False):
+ use_cfstream = False,
+ tags = []):
copts = []
if use_cfstream:
copts = if_mac(["-DGRPC_CFSTREAM"])
@@ -116,6 +123,7 @@ def grpc_cc_library(
],
alwayslink = alwayslink,
data = data,
+ tags = tags,
)
def grpc_proto_plugin(name, srcs = [], deps = []):
@@ -158,8 +166,9 @@ def grpc_cc_test(name, srcs = [], deps = [], external_deps = [], args = [], data
"size": size,
"timeout": timeout,
"exec_compatible_with": exec_compatible_with,
+ "tags": tags,
}
- if uses_polling:
+ if uses_polling and not is_msvc():
native.cc_test(testonly = True, tags = ["manual"], **args)
for poller in POLLERS:
native.sh_test(
@@ -180,7 +189,7 @@ def grpc_cc_test(name, srcs = [], deps = [], external_deps = [], args = [], data
else:
native.cc_test(**args)
-def grpc_cc_binary(name, srcs = [], deps = [], external_deps = [], args = [], data = [], language = "C++", testonly = False, linkshared = False, linkopts = []):
+def grpc_cc_binary(name, srcs = [], deps = [], external_deps = [], args = [], data = [], language = "C++", testonly = False, linkshared = False, linkopts = [], tags = []):
copts = []
if language.upper() == "C":
copts = ["-std=c99"]
@@ -194,6 +203,7 @@ def grpc_cc_binary(name, srcs = [], deps = [], external_deps = [], args = [], da
deps = deps + _get_external_deps(external_deps),
copts = copts,
linkopts = if_not_windows(["-pthread"]) + linkopts,
+ tags = tags,
)
def grpc_generate_one_off_targets():
diff --git a/bazel/grpc_deps.bzl b/bazel/grpc_deps.bzl
index e2e47292242..799e864484c 100644
--- a/bazel/grpc_deps.bzl
+++ b/bazel/grpc_deps.bzl
@@ -147,16 +147,16 @@ def grpc_deps():
if "com_github_gflags_gflags" not in native.existing_rules():
http_archive(
name = "com_github_gflags_gflags",
- strip_prefix = "gflags-30dbc81fb5ffdc98ea9b14b1918bfe4e8779b26e",
- url = "https://github.com/gflags/gflags/archive/30dbc81fb5ffdc98ea9b14b1918bfe4e8779b26e.tar.gz",
+ strip_prefix = "gflags-28f50e0fed19872e0fd50dd23ce2ee8cd759338e",
+ url = "https://github.com/gflags/gflags/archive/28f50e0fed19872e0fd50dd23ce2ee8cd759338e.tar.gz",
)
if "com_github_google_benchmark" not in native.existing_rules():
http_archive(
name = "com_github_google_benchmark",
- build_file = "@com_github_grpc_grpc//third_party:benchmark.BUILD",
- strip_prefix = "benchmark-9913418d323e64a0111ca0da81388260c2bbe1e9",
- url = "https://github.com/google/benchmark/archive/9913418d323e64a0111ca0da81388260c2bbe1e9.tar.gz",
+ #build_file = "@com_github_grpc_grpc//third_party:benchmark.BUILD",
+ strip_prefix = "benchmark-e776aa0275e293707b6a0901e0e8d8a8a3679508",
+ url = "https://github.com/google/benchmark/archive/e776aa0275e293707b6a0901e0e8d8a8a3679508.tar.gz",
)
if "com_github_cares_cares" not in native.existing_rules():
diff --git a/test/core/bad_connection/BUILD b/test/core/bad_connection/BUILD
index 8ada933e796..4de9c0eb2d8 100644
--- a/test/core/bad_connection/BUILD
+++ b/test/core/bad_connection/BUILD
@@ -29,4 +29,5 @@ grpc_cc_binary(
"//:grpc",
"//test/core/util:grpc_test_util",
],
+ tags = ["exclude_windows"],
)
diff --git a/test/core/client_channel/BUILD b/test/core/client_channel/BUILD
index 57e5191af4c..d67f326aa6d 100644
--- a/test/core/client_channel/BUILD
+++ b/test/core/client_channel/BUILD
@@ -52,6 +52,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
diff --git a/test/core/end2end/generate_tests.bzl b/test/core/end2end/generate_tests.bzl
index ec32aa5102c..cb1d88a3014 100755
--- a/test/core/end2end/generate_tests.bzl
+++ b/test/core/end2end/generate_tests.bzl
@@ -15,7 +15,7 @@
"""Generates the appropriate build.json data for all the end2end tests."""
-load("//bazel:grpc_build_system.bzl", "grpc_cc_binary", "grpc_cc_library")
+load("//bazel:grpc_build_system.bzl", "grpc_cc_binary", "grpc_cc_library", "is_msvc")
POLLERS = ["epollex", "epoll1", "poll", "poll-cv"]
@@ -31,7 +31,8 @@ def _fixture_options(
is_http2 = True,
supports_proxy_auth = False,
supports_write_buffering = True,
- client_channel = True):
+ client_channel = True,
+ supports_msvc = True,):
return struct(
fullstack = fullstack,
includes_proxy = includes_proxy,
@@ -44,6 +45,7 @@ def _fixture_options(
supports_proxy_auth = supports_proxy_auth,
supports_write_buffering = supports_write_buffering,
client_channel = client_channel,
+ supports_msvc = supports_msvc,
#_platforms=_platforms,
)
@@ -119,10 +121,11 @@ END2END_NOSEC_FIXTURES = {
client_channel = False,
secure = False,
_platforms = ["linux", "mac", "posix"],
+ supports_msvc = False,
),
"h2_full": _fixture_options(secure = False),
- "h2_full+pipe": _fixture_options(secure = False, _platforms = ["linux"]),
- "h2_full+trace": _fixture_options(secure = False, tracing = True),
+ "h2_full+pipe": _fixture_options(secure = False, _platforms = ["linux"], supports_msvc = False),
+ "h2_full+trace": _fixture_options(secure = False, tracing = True, supports_msvc = False),
"h2_full+workarounds": _fixture_options(secure = False),
"h2_http_proxy": _fixture_options(secure = False, supports_proxy_auth = True),
"h2_proxy": _fixture_options(secure = False, includes_proxy = True),
@@ -151,6 +154,7 @@ END2END_NOSEC_FIXTURES = {
dns_resolver = False,
_platforms = ["linux", "mac", "posix"],
secure = False,
+ supports_msvc = False,
),
}
@@ -328,6 +332,9 @@ END2END_TESTS = {
}
def _compatible(fopt, topt):
+ if is_msvc:
+ if not fopt.supports_msvc:
+ return False
if topt.needs_fullstack:
if not fopt.fullstack:
return False
diff --git a/test/core/iomgr/BUILD b/test/core/iomgr/BUILD
index 7daabd50527..f9da7f7ba73 100644
--- a/test/core/iomgr/BUILD
+++ b/test/core/iomgr/BUILD
@@ -81,6 +81,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
@@ -92,6 +93,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
@@ -103,6 +105,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
@@ -139,6 +142,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
@@ -153,6 +157,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
@@ -214,6 +219,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
@@ -225,6 +231,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
@@ -237,6 +244,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
@@ -259,6 +267,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
@@ -303,6 +312,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
diff --git a/test/cpp/common/BUILD b/test/cpp/common/BUILD
index 01699b26add..e4ed3bc5460 100644
--- a/test/cpp/common/BUILD
+++ b/test/cpp/common/BUILD
@@ -28,6 +28,7 @@ grpc_cc_test(
"//:grpc++_unsecure",
"//test/core/util:grpc_test_util_unsecure",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
diff --git a/test/cpp/end2end/BUILD b/test/cpp/end2end/BUILD
index d80fa33a83a..82ad1e9387a 100644
--- a/test/cpp/end2end/BUILD
+++ b/test/cpp/end2end/BUILD
@@ -99,6 +99,7 @@ grpc_cc_test(
"//test/core/util:grpc_test_util",
"//test/cpp/util:test_util",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
@@ -607,6 +608,7 @@ grpc_cc_test(
"//test/core/util:grpc_test_util",
"//test/cpp/util:test_util",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
diff --git a/test/cpp/interop/BUILD b/test/cpp/interop/BUILD
index f36494d98db..d74566f56a7 100644
--- a/test/cpp/interop/BUILD
+++ b/test/cpp/interop/BUILD
@@ -161,4 +161,5 @@ grpc_cc_test(
"//test/cpp/util:test_config",
"//test/cpp/util:test_util",
],
+ tags = ["exclude_windows"],
)
diff --git a/test/cpp/microbenchmarks/BUILD b/test/cpp/microbenchmarks/BUILD
index 70b4000780c..db37d37af6c 100644
--- a/test/cpp/microbenchmarks/BUILD
+++ b/test/cpp/microbenchmarks/BUILD
@@ -45,6 +45,7 @@ grpc_cc_library(
"//test/core/util:grpc_test_util_unsecure",
"//test/cpp/util:test_config",
],
+ tags = ["exclude_windows"],
)
grpc_cc_binary(
@@ -52,6 +53,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_closure.cc"],
deps = [":helpers"],
+ tags = ["exclude_windows"],
)
grpc_cc_binary(
@@ -59,6 +61,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_alarm.cc"],
deps = [":helpers"],
+ tags = ["exclude_windows"],
)
grpc_cc_binary(
@@ -73,6 +76,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_byte_buffer.cc"],
deps = [":helpers"],
+ tags = ["exclude_windows"],
)
grpc_cc_binary(
@@ -80,6 +84,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_channel.cc"],
deps = [":helpers"],
+ tags = ["exclude_windows"],
)
grpc_cc_binary(
@@ -87,6 +92,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_call_create.cc"],
deps = [":helpers"],
+ tags = ["exclude_windows"],
)
grpc_cc_binary(
@@ -94,6 +100,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_cq.cc"],
deps = [":helpers"],
+ tags = ["exclude_windows"],
)
grpc_cc_binary(
@@ -101,6 +108,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_cq_multiple_threads.cc"],
deps = [":helpers"],
+ tags = ["exclude_windows"],
)
grpc_cc_binary(
@@ -108,6 +116,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_error.cc"],
deps = [":helpers"],
+ tags = ["exclude_windows"],
)
grpc_cc_library(
@@ -117,6 +126,7 @@ grpc_cc_library(
"fullstack_streaming_ping_pong.h",
],
deps = [":helpers"],
+ tags = ["exclude_windows"],
)
grpc_cc_binary(
@@ -126,6 +136,7 @@ grpc_cc_binary(
"bm_fullstack_streaming_ping_pong.cc",
],
deps = [":fullstack_streaming_ping_pong_h"],
+ tags = ["exclude_windows"],
)
grpc_cc_library(
@@ -144,6 +155,7 @@ grpc_cc_binary(
"bm_fullstack_streaming_pump.cc",
],
deps = [":fullstack_streaming_pump_h"],
+ tags = ["exclude_windows"],
)
grpc_cc_binary(
@@ -151,6 +163,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_fullstack_trickle.cc"],
deps = [":helpers"],
+ tags = ["exclude_windows"],
)
grpc_cc_library(
@@ -169,6 +182,7 @@ grpc_cc_binary(
"bm_fullstack_unary_ping_pong.cc",
],
deps = [":fullstack_unary_ping_pong_h"],
+ tags = ["exclude_windows"],
)
grpc_cc_binary(
@@ -176,6 +190,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_metadata.cc"],
deps = [":helpers"],
+ tags = ["exclude_windows"],
)
grpc_cc_binary(
@@ -183,6 +198,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_chttp2_hpack.cc"],
deps = [":helpers"],
+ tags = ["exclude_windows"],
)
grpc_cc_binary(
@@ -202,4 +218,5 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_timer.cc"],
deps = [":helpers"],
+ tags = ["exclude_windows"],
)
diff --git a/test/cpp/naming/generate_resolver_component_tests.bzl b/test/cpp/naming/generate_resolver_component_tests.bzl
index f36021560c1..8e584289628 100755
--- a/test/cpp/naming/generate_resolver_component_tests.bzl
+++ b/test/cpp/naming/generate_resolver_component_tests.bzl
@@ -33,6 +33,7 @@ def generate_resolver_component_tests():
"//:gpr",
"//test/cpp/util:test_config",
],
+ tags = ["exclude_windows"],
)
# meant to be invoked only through the top-level shell script driver
grpc_cc_binary(
@@ -52,6 +53,7 @@ def generate_resolver_component_tests():
"//:gpr",
"//test/cpp/util:test_config",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
name = "resolver_component_tests_runner_invoker%s" % unsecure_build_config_suffix,
@@ -77,5 +79,6 @@ def generate_resolver_component_tests():
args = [
"--test_bin_name=resolver_component_test%s" % unsecure_build_config_suffix,
"--running_under_bazel=true",
- ]
+ ],
+ tags = ["exclude_windows"],
)
diff --git a/test/cpp/performance/BUILD b/test/cpp/performance/BUILD
index 4fe95d5905e..ddc41e75102 100644
--- a/test/cpp/performance/BUILD
+++ b/test/cpp/performance/BUILD
@@ -31,4 +31,5 @@ grpc_cc_test(
"//src/proto/grpc/testing:echo_proto",
"//test/core/util:grpc_test_util_base",
],
+ tags = ["exclude_windows"],
)
diff --git a/test/cpp/qps/qps_benchmark_script.bzl b/test/cpp/qps/qps_benchmark_script.bzl
index 855caa0d37c..23b42c02b64 100644
--- a/test/cpp/qps/qps_benchmark_script.bzl
+++ b/test/cpp/qps/qps_benchmark_script.bzl
@@ -75,5 +75,6 @@ def json_run_localhost_batch():
],
tags = [
"json_run_localhost",
+ "exclude_windows",
],
)
diff --git a/test/cpp/server/BUILD b/test/cpp/server/BUILD
index 050b83f5c4f..3c4b35af709 100644
--- a/test/cpp/server/BUILD
+++ b/test/cpp/server/BUILD
@@ -29,6 +29,7 @@ grpc_cc_test(
"//src/proto/grpc/testing:echo_proto",
"//test/core/util:grpc_test_util_unsecure",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
@@ -42,6 +43,7 @@ grpc_cc_test(
"//src/proto/grpc/testing:echo_proto",
"//test/core/util:grpc_test_util_unsecure",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
@@ -55,4 +57,5 @@ grpc_cc_test(
"//src/proto/grpc/testing:echo_proto",
"//test/core/util:grpc_test_util_unsecure",
],
+ tags = ["exclude_windows"],
)
diff --git a/test/cpp/server/load_reporter/BUILD b/test/cpp/server/load_reporter/BUILD
index 8d876c56d29..2286119324b 100644
--- a/test/cpp/server/load_reporter/BUILD
+++ b/test/cpp/server/load_reporter/BUILD
@@ -45,6 +45,7 @@ grpc_cc_test(
"//:lb_server_load_reporting_filter",
"//test/core/util:grpc_test_util",
],
+ tags = ["exclude_windows"],
)
grpc_cc_test(
diff --git a/tools/remote_build/windows.bazelrc b/tools/remote_build/windows.bazelrc
new file mode 100644
index 00000000000..a74f23b8048
--- /dev/null
+++ b/tools/remote_build/windows.bazelrc
@@ -0,0 +1,2 @@
+build --test_tag_filters=-exclude_windows
+build --build_tag_filters=-exclude_windows
\ No newline at end of file
From 4814972080ea8490085e6fb60e31c6e96cb50771 Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Wed, 6 Mar 2019 15:00:02 -0800
Subject: [PATCH 045/138] Install `wheel` right before use it
---
tools/run_tests/artifacts/build_package_python.sh | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/tools/run_tests/artifacts/build_package_python.sh b/tools/run_tests/artifacts/build_package_python.sh
index 193d75db62a..29a26bc081c 100755
--- a/tools/run_tests/artifacts/build_package_python.sh
+++ b/tools/run_tests/artifacts/build_package_python.sh
@@ -23,17 +23,20 @@ mkdir -p artifacts/
# and we only collect them here to deliver them to the distribtest phase.
cp -r "${EXTERNAL_GIT_ROOT}"/input_artifacts/python_*/* artifacts/ || true
+apt-get install -y python-pip
+python -m pip install wheel --user
+
strip_binary_wheel() {
WHEEL_PATH="$1"
TEMP_WHEEL_DIR=$(mktemp -d)
- wheel unpack "$WHEEL_PATH" -d "$TEMP_WHEEL_DIR"
+ python -m wheel unpack "$WHEEL_PATH" -d "$TEMP_WHEEL_DIR"
find "$TEMP_WHEEL_DIR" -name "_protoc_compiler*.so" -exec strip --strip-debug {} ";"
find "$TEMP_WHEEL_DIR" -name "cygrpc*.so" -exec strip --strip-debug {} ";"
WHEEL_FILE=$(basename "$WHEEL_PATH")
DISTRIBUTION_NAME=$(basename "$WHEEL_PATH" | cut -d '-' -f 1)
VERSION=$(basename "$WHEEL_PATH" | cut -d '-' -f 2)
- wheel pack "$TEMP_WHEEL_DIR/$DISTRIBUTION_NAME-$VERSION" -d "$TEMP_WHEEL_DIR"
+ python -m wheel pack "$TEMP_WHEEL_DIR/$DISTRIBUTION_NAME-$VERSION" -d "$TEMP_WHEEL_DIR"
mv "$TEMP_WHEEL_DIR/$WHEEL_FILE" "$WHEEL_PATH"
}
From e0059af33b69806cb35e2cf125e301d500095b50 Mon Sep 17 00:00:00 2001
From: Nicolas Noble <nicolasnoble@users.noreply.github.com>
Date: Wed, 6 Mar 2019 16:44:37 -0800
Subject: [PATCH 046/138] Adding a few potential breaking changes
---
doc/core/pending_api_cleanups.md | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/doc/core/pending_api_cleanups.md b/doc/core/pending_api_cleanups.md
index 67d587deadc..f330240cadf 100644
--- a/doc/core/pending_api_cleanups.md
+++ b/doc/core/pending_api_cleanups.md
@@ -15,3 +15,7 @@ number:
`include/grpc/impl/codegen/grpc_types.h` (commit `af00d8b`)
(cannot be done until after next grpc release, so that TensorFlow can
use the same code both internally and externally)
+- get rid of all of the grpc++ headers that are currently deprecated
+ in favor of their grpcpp counterpart due to iOS' parsing issue on the
+ + sign.
+- require a C++ runtime for all languages.
From 0a36746ebe1dafd58b44620294ca571d592c3b06 Mon Sep 17 00:00:00 2001
From: Nicolas Noble <nicolasnoble@users.noreply.github.com>
Date: Wed, 6 Mar 2019 16:45:25 -0800
Subject: [PATCH 047/138] Changing + to plus.
---
doc/core/pending_api_cleanups.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/doc/core/pending_api_cleanups.md b/doc/core/pending_api_cleanups.md
index f330240cadf..eb44659c9e0 100644
--- a/doc/core/pending_api_cleanups.md
+++ b/doc/core/pending_api_cleanups.md
@@ -17,5 +17,5 @@ number:
use the same code both internally and externally)
- get rid of all of the grpc++ headers that are currently deprecated
in favor of their grpcpp counterpart due to iOS' parsing issue on the
- + sign.
+ plus sign.
- require a C++ runtime for all languages.
From fc56889a5cef2f43df22748a813cc5ed91df91f6 Mon Sep 17 00:00:00 2001
From: Nicolas Noble <nicolasnoble@users.noreply.github.com>
Date: Wed, 6 Mar 2019 16:47:30 -0800
Subject: [PATCH 048/138] Removing the change about headers, since it's the
wrong file.
---
doc/core/pending_api_cleanups.md | 3 ---
1 file changed, 3 deletions(-)
diff --git a/doc/core/pending_api_cleanups.md b/doc/core/pending_api_cleanups.md
index eb44659c9e0..5a8270349a4 100644
--- a/doc/core/pending_api_cleanups.md
+++ b/doc/core/pending_api_cleanups.md
@@ -15,7 +15,4 @@ number:
`include/grpc/impl/codegen/grpc_types.h` (commit `af00d8b`)
(cannot be done until after next grpc release, so that TensorFlow can
use the same code both internally and externally)
-- get rid of all of the grpc++ headers that are currently deprecated
- in favor of their grpcpp counterpart due to iOS' parsing issue on the
- plus sign.
- require a C++ runtime for all languages.
From 5030177c5c2276b4f5e283a212d1c74741138e3b Mon Sep 17 00:00:00 2001
From: Vijay Pai <vpai@google.com>
Date: Wed, 6 Mar 2019 18:17:11 -0800
Subject: [PATCH 049/138] Add comment to address reviewer feedback
---
src/cpp/server/server_cc.cc | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/src/cpp/server/server_cc.cc b/src/cpp/server/server_cc.cc
index 4d5c8179fce..6e78e93b835 100644
--- a/src/cpp/server/server_cc.cc
+++ b/src/cpp/server/server_cc.cc
@@ -361,6 +361,12 @@ class Server::CallbackRequest final : public Server::CallbackRequestBase {
static_assert(std::is_base_of<ServerContext, ServerContextType>::value,
"ServerContextType must be derived from ServerContext");
+ // The constructor needs to know the server for this callback request and its
+ // index in the server's request count array to allow for proper dynamic
+ // requesting of incoming RPCs. For codegen services, the values of method and
+ // method_tag represent the defined characteristics of the method being
+ // requested. For generic services, method and method_tag are nullptr since
+ // these services don't have pre-defined methods or method registration tags.
CallbackRequest(Server* server, size_t method_idx,
internal::RpcServiceMethod* method, void* method_tag)
: server_(server),
From 18b19105f2d97f9418d9a1e4fb4f66e4e4740028 Mon Sep 17 00:00:00 2001
From: Soheil Hassas Yeganeh <soheil@google.com>
Date: Mon, 4 Mar 2019 14:19:43 -0500
Subject: [PATCH 050/138] Implement TCP_INQ for gRPC in Linux
TCP_INQ is a socket option we added to Linux to report pending bytes
on the socket as a control message.
Using TCP_INQ we can accurately decide whether to continue read or not.
Add an urgent parameter, when we do not want to wait for EPOLLIN.
This commit improves the latency of 1 RPC unary (minimal benchmark)
significantly:
Before:
l_50: 61.3584984733
l_90: 94.8328711277
l_99: 126.211351174
l_999: 158.722406029
After:
l_50: 51.3546011488 (-16%)
l_90: 72.3420731581 (-23%)
l_99: 103.280218974 (-18%)
l_999: 130.905689996 (-17%)
---
.../client_channel/http_connect_handshaker.cc | 4 +-
.../chttp2/transport/chttp2_transport.cc | 3 +-
src/core/lib/http/httpcli.cc | 2 +-
src/core/lib/iomgr/endpoint.cc | 4 +-
src/core/lib/iomgr/endpoint.h | 5 +-
src/core/lib/iomgr/endpoint_cfstream.cc | 2 +-
src/core/lib/iomgr/port.h | 3 +
src/core/lib/iomgr/tcp_custom.cc | 2 +-
src/core/lib/iomgr/tcp_posix.cc | 206 +++++++++++++-----
src/core/lib/iomgr/tcp_windows.cc | 2 +-
.../lib/security/transport/secure_endpoint.cc | 4 +-
.../security/transport/security_handshaker.cc | 7 +-
test/core/bad_client/bad_client.cc | 3 +-
test/core/end2end/bad_server_response_test.cc | 6 +-
.../end2end/fixtures/http_proxy_fixture.cc | 12 +-
.../readahead_handshaker_server_ssl.cc | 3 +-
test/core/iomgr/endpoint_tests.cc | 13 +-
test/core/iomgr/tcp_posix_test.cc | 9 +-
test/core/security/secure_endpoint_test.cc | 2 +-
.../transport/chttp2/settings_timeout_test.cc | 3 +-
test/core/util/mock_endpoint.cc | 2 +-
test/core/util/passthru_endpoint.cc | 2 +-
test/core/util/trickle_endpoint.cc | 4 +-
.../microbenchmarks/bm_chttp2_transport.cc | 2 +-
24 files changed, 212 insertions(+), 93 deletions(-)
diff --git a/src/core/ext/filters/client_channel/http_connect_handshaker.cc b/src/core/ext/filters/client_channel/http_connect_handshaker.cc
index fa5aaa9e7ce..2b1eb92bbd4 100644
--- a/src/core/ext/filters/client_channel/http_connect_handshaker.cc
+++ b/src/core/ext/filters/client_channel/http_connect_handshaker.cc
@@ -144,7 +144,7 @@ void HttpConnectHandshaker::OnWriteDone(void* arg, grpc_error* error) {
// The read callback inherits our ref to the handshaker.
grpc_endpoint_read(handshaker->args_->endpoint,
handshaker->args_->read_buffer,
- &handshaker->response_read_closure_);
+ &handshaker->response_read_closure_, /*urgent=*/true);
gpr_mu_unlock(&handshaker->mu_);
}
}
@@ -207,7 +207,7 @@ void HttpConnectHandshaker::OnReadDone(void* arg, grpc_error* error) {
grpc_slice_buffer_reset_and_unref_internal(handshaker->args_->read_buffer);
grpc_endpoint_read(handshaker->args_->endpoint,
handshaker->args_->read_buffer,
- &handshaker->response_read_closure_);
+ &handshaker->response_read_closure_, /*urgent=*/true);
gpr_mu_unlock(&handshaker->mu_);
return;
}
diff --git a/src/core/ext/transport/chttp2/transport/chttp2_transport.cc b/src/core/ext/transport/chttp2/transport/chttp2_transport.cc
index 306349b7910..888c1757be1 100644
--- a/src/core/ext/transport/chttp2/transport/chttp2_transport.cc
+++ b/src/core/ext/transport/chttp2/transport/chttp2_transport.cc
@@ -2577,7 +2577,8 @@ static void read_action_locked(void* tp, grpc_error* error) {
grpc_slice_buffer_reset_and_unref_internal(&t->read_buffer);
if (keep_reading) {
- grpc_endpoint_read(t->ep, &t->read_buffer, &t->read_action_locked);
+ const bool urgent = t->goaway_error != GRPC_ERROR_NONE;
+ grpc_endpoint_read(t->ep, &t->read_buffer, &t->read_action_locked, urgent);
grpc_chttp2_act_on_flowctl_action(t->flow_control->MakeAction(), t,
nullptr);
GRPC_CHTTP2_UNREF_TRANSPORT(t, "keep_reading");
diff --git a/src/core/lib/http/httpcli.cc b/src/core/lib/http/httpcli.cc
index 8c9ce4da0d3..8a8da8b1604 100644
--- a/src/core/lib/http/httpcli.cc
+++ b/src/core/lib/http/httpcli.cc
@@ -121,7 +121,7 @@ static void append_error(internal_request* req, grpc_error* error) {
}
static void do_read(internal_request* req) {
- grpc_endpoint_read(req->ep, &req->incoming, &req->on_read);
+ grpc_endpoint_read(req->ep, &req->incoming, &req->on_read, /*urgent=*/true);
}
static void on_read(void* user_data, grpc_error* error) {
diff --git a/src/core/lib/iomgr/endpoint.cc b/src/core/lib/iomgr/endpoint.cc
index 06316c60315..bb07fe79608 100644
--- a/src/core/lib/iomgr/endpoint.cc
+++ b/src/core/lib/iomgr/endpoint.cc
@@ -23,8 +23,8 @@
grpc_core::TraceFlag grpc_tcp_trace(false, "tcp");
void grpc_endpoint_read(grpc_endpoint* ep, grpc_slice_buffer* slices,
- grpc_closure* cb) {
- ep->vtable->read(ep, slices, cb);
+ grpc_closure* cb, bool urgent) {
+ ep->vtable->read(ep, slices, cb, urgent);
}
void grpc_endpoint_write(grpc_endpoint* ep, grpc_slice_buffer* slices,
diff --git a/src/core/lib/iomgr/endpoint.h b/src/core/lib/iomgr/endpoint.h
index 79c8ece263a..932e7e15b9a 100644
--- a/src/core/lib/iomgr/endpoint.h
+++ b/src/core/lib/iomgr/endpoint.h
@@ -36,7 +36,8 @@ typedef struct grpc_endpoint_vtable grpc_endpoint_vtable;
class Timestamps;
struct grpc_endpoint_vtable {
- void (*read)(grpc_endpoint* ep, grpc_slice_buffer* slices, grpc_closure* cb);
+ void (*read)(grpc_endpoint* ep, grpc_slice_buffer* slices, grpc_closure* cb,
+ bool urgent);
void (*write)(grpc_endpoint* ep, grpc_slice_buffer* slices, grpc_closure* cb,
void* arg);
void (*add_to_pollset)(grpc_endpoint* ep, grpc_pollset* pollset);
@@ -56,7 +57,7 @@ struct grpc_endpoint_vtable {
Valid slices may be placed into \a slices even when the callback is
invoked with error != GRPC_ERROR_NONE. */
void grpc_endpoint_read(grpc_endpoint* ep, grpc_slice_buffer* slices,
- grpc_closure* cb);
+ grpc_closure* cb, bool urgent);
char* grpc_endpoint_get_peer(grpc_endpoint* ep);
diff --git a/src/core/lib/iomgr/endpoint_cfstream.cc b/src/core/lib/iomgr/endpoint_cfstream.cc
index 25146e7861c..6de22972dbf 100644
--- a/src/core/lib/iomgr/endpoint_cfstream.cc
+++ b/src/core/lib/iomgr/endpoint_cfstream.cc
@@ -251,7 +251,7 @@ static void CFStreamReadAllocationDone(void* arg, grpc_error* error) {
}
static void CFStreamRead(grpc_endpoint* ep, grpc_slice_buffer* slices,
- grpc_closure* cb) {
+ grpc_closure* cb, bool urgent) {
CFStreamEndpoint* ep_impl = reinterpret_cast<CFStreamEndpoint*>(ep);
if (grpc_tcp_trace.enabled()) {
gpr_log(GPR_DEBUG, "CFStream endpoint:%p read (%p, %p) length:%zu", ep_impl,
diff --git a/src/core/lib/iomgr/port.h b/src/core/lib/iomgr/port.h
index 7b6ca1bc0e1..3248343e27c 100644
--- a/src/core/lib/iomgr/port.h
+++ b/src/core/lib/iomgr/port.h
@@ -60,6 +60,9 @@
#define GRPC_HAVE_IP_PKTINFO 1
#define GRPC_HAVE_MSG_NOSIGNAL 1
#define GRPC_HAVE_UNIX_SOCKET 1
+/* Linux has TCP_INQ support since 4.18, but it is safe to set
+ the socket option on older kernels. */
+#define GRPC_HAVE_TCP_INQ 1
#ifdef LINUX_VERSION_CODE
#if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 0, 0)
#define GRPC_LINUX_ERRQUEUE 1
diff --git a/src/core/lib/iomgr/tcp_custom.cc b/src/core/lib/iomgr/tcp_custom.cc
index 1e5696e1279..f7ad120b026 100644
--- a/src/core/lib/iomgr/tcp_custom.cc
+++ b/src/core/lib/iomgr/tcp_custom.cc
@@ -192,7 +192,7 @@ static void tcp_read_allocation_done(void* tcpp, grpc_error* error) {
}
static void endpoint_read(grpc_endpoint* ep, grpc_slice_buffer* read_slices,
- grpc_closure* cb) {
+ grpc_closure* cb, bool urgent) {
custom_tcp_endpoint* tcp = (custom_tcp_endpoint*)ep;
GRPC_CUSTOM_IOMGR_ASSERT_SAME_THREAD();
GPR_ASSERT(tcp->read_cb == nullptr);
diff --git a/src/core/lib/iomgr/tcp_posix.cc b/src/core/lib/iomgr/tcp_posix.cc
index 525288a77ae..960a45b7b26 100644
--- a/src/core/lib/iomgr/tcp_posix.cc
+++ b/src/core/lib/iomgr/tcp_posix.cc
@@ -27,6 +27,7 @@
#include <errno.h>
#include <limits.h>
#include <netinet/in.h>
+#include <netinet/tcp.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
@@ -34,6 +35,7 @@
#include <sys/socket.h>
#include <sys/types.h>
#include <unistd.h>
+#include <algorithm>
#include <grpc/slice.h>
#include <grpc/support/alloc.h>
@@ -54,6 +56,15 @@
#include "src/core/lib/slice/slice_internal.h"
#include "src/core/lib/slice/slice_string_helpers.h"
+#ifndef SOL_TCP
+#define SOL_TCP IPPROTO_TCP
+#endif
+
+#ifndef TCP_INQ
+#define TCP_INQ 36
+#define TCP_CM_INQ TCP_INQ
+#endif
+
#ifdef GRPC_HAVE_MSG_NOSIGNAL
#define SENDMSG_FLAGS MSG_NOSIGNAL
#else
@@ -88,8 +99,11 @@ struct grpc_tcp {
grpc_slice_buffer last_read_buffer;
grpc_slice_buffer* incoming_buffer;
+ int inq; /* bytes pending on the socket from the last read. */
+ bool inq_capable; /* cache whether kernel supports inq */
+
grpc_slice_buffer* outgoing_buffer;
- /** byte within outgoing_buffer->slices[0] to write next */
+ /* byte within outgoing_buffer->slices[0] to write next */
size_t outgoing_byte_idx;
grpc_closure* read_cb;
@@ -429,69 +443,140 @@ static void tcp_do_read(grpc_tcp* tcp) {
GPR_TIMER_SCOPE("tcp_do_read", 0);
struct msghdr msg;
struct iovec iov[MAX_READ_IOVEC];
+ char cmsgbuf[24 /*CMSG_SPACE(sizeof(int))*/];
ssize_t read_bytes;
- size_t i;
-
- GPR_ASSERT(tcp->incoming_buffer->count <= MAX_READ_IOVEC);
+ size_t total_read_bytes = 0;
- for (i = 0; i < tcp->incoming_buffer->count; i++) {
+ size_t iov_len =
+ std::min<size_t>(MAX_READ_IOVEC, tcp->incoming_buffer->count);
+ for (size_t i = 0; i < iov_len; i++) {
iov[i].iov_base = GRPC_SLICE_START_PTR(tcp->incoming_buffer->slices[i]);
iov[i].iov_len = GRPC_SLICE_LENGTH(tcp->incoming_buffer->slices[i]);
}
- msg.msg_name = nullptr;
- msg.msg_namelen = 0;
- msg.msg_iov = iov;
- msg.msg_iovlen = static_cast<msg_iovlen_type>(tcp->incoming_buffer->count);
- msg.msg_control = nullptr;
- msg.msg_controllen = 0;
- msg.msg_flags = 0;
-
- GRPC_STATS_INC_TCP_READ_OFFER(tcp->incoming_buffer->length);
- GRPC_STATS_INC_TCP_READ_OFFER_IOV_SIZE(tcp->incoming_buffer->count);
-
do {
- GPR_TIMER_SCOPE("recvmsg", 0);
- GRPC_STATS_INC_SYSCALL_READ();
- read_bytes = recvmsg(tcp->fd, &msg, 0);
- } while (read_bytes < 0 && errno == EINTR);
-
- if (read_bytes < 0) {
- /* NB: After calling call_read_cb a parallel call of the read handler may
- * be running. */
- if (errno == EAGAIN) {
- finish_estimate(tcp);
- /* We've consumed the edge, request a new one */
- notify_on_read(tcp);
+ /* Assume there is something on the queue. If we receive TCP_INQ from
+ * kernel, we will update this value, otherwise, we have to assume there is
+ * always something to read until we get EAGAIN. */
+ tcp->inq = 1;
+
+ msg.msg_name = nullptr;
+ msg.msg_namelen = 0;
+ msg.msg_iov = iov;
+ msg.msg_iovlen = static_cast<msg_iovlen_type>(iov_len);
+ if (tcp->inq_capable) {
+ msg.msg_control = cmsgbuf;
+ msg.msg_controllen = sizeof(cmsgbuf);
} else {
+ msg.msg_control = nullptr;
+ msg.msg_controllen = 0;
+ }
+ msg.msg_flags = 0;
+
+ GRPC_STATS_INC_TCP_READ_OFFER(tcp->incoming_buffer->length);
+ GRPC_STATS_INC_TCP_READ_OFFER_IOV_SIZE(tcp->incoming_buffer->count);
+
+ do {
+ GPR_TIMER_SCOPE("recvmsg", 0);
+ GRPC_STATS_INC_SYSCALL_READ();
+ read_bytes = recvmsg(tcp->fd, &msg, 0);
+ } while (read_bytes < 0 && errno == EINTR);
+
+ /* We have read something in previous reads. We need to deliver those
+ * bytes to the upper layer. */
+ if (read_bytes <= 0 && total_read_bytes > 0) {
+ tcp->inq = 1;
+ break;
+ }
+
+ if (read_bytes < 0) {
+ /* NB: After calling call_read_cb a parallel call of the read handler may
+ * be running. */
+ if (errno == EAGAIN) {
+ finish_estimate(tcp);
+ tcp->inq = 0;
+ /* We've consumed the edge, request a new one */
+ notify_on_read(tcp);
+ } else {
+ grpc_slice_buffer_reset_and_unref_internal(tcp->incoming_buffer);
+ call_read_cb(tcp,
+ tcp_annotate_error(GRPC_OS_ERROR(errno, "recvmsg"), tcp));
+ TCP_UNREF(tcp, "read");
+ }
+ return;
+ }
+ if (read_bytes == 0) {
+ /* 0 read size ==> end of stream
+ *
+ * We may have read something, i.e., total_read_bytes > 0, but
+ * since the connection is closed we will drop the data here, because we
+ * can't call the callback multiple times. */
grpc_slice_buffer_reset_and_unref_internal(tcp->incoming_buffer);
- call_read_cb(tcp,
- tcp_annotate_error(GRPC_OS_ERROR(errno, "recvmsg"), tcp));
+ call_read_cb(
+ tcp, tcp_annotate_error(
+ GRPC_ERROR_CREATE_FROM_STATIC_STRING("Socket closed"), tcp));
TCP_UNREF(tcp, "read");
+ return;
}
- } else if (read_bytes == 0) {
- /* 0 read size ==> end of stream */
- grpc_slice_buffer_reset_and_unref_internal(tcp->incoming_buffer);
- call_read_cb(
- tcp, tcp_annotate_error(
- GRPC_ERROR_CREATE_FROM_STATIC_STRING("Socket closed"), tcp));
- TCP_UNREF(tcp, "read");
- } else {
+
GRPC_STATS_INC_TCP_READ_SIZE(read_bytes);
add_to_estimate(tcp, static_cast<size_t>(read_bytes));
- GPR_ASSERT((size_t)read_bytes <= tcp->incoming_buffer->length);
- if (static_cast<size_t>(read_bytes) == tcp->incoming_buffer->length) {
- finish_estimate(tcp);
- } else if (static_cast<size_t>(read_bytes) < tcp->incoming_buffer->length) {
- grpc_slice_buffer_trim_end(
- tcp->incoming_buffer,
- tcp->incoming_buffer->length - static_cast<size_t>(read_bytes),
- &tcp->last_read_buffer);
+ GPR_DEBUG_ASSERT((size_t)read_bytes <=
+ tcp->incoming_buffer->length - total_read_bytes);
+
+#ifdef GRPC_HAVE_TCP_INQ
+ if (tcp->inq_capable) {
+ GPR_DEBUG_ASSERT(!(msg.msg_flags & MSG_CTRUNC));
+ struct cmsghdr* cmsg = CMSG_FIRSTHDR(&msg);
+ for (; cmsg != nullptr; cmsg = CMSG_NXTHDR(&msg, cmsg)) {
+ if (cmsg->cmsg_level == SOL_TCP && cmsg->cmsg_type == TCP_CM_INQ &&
+ cmsg->cmsg_len == CMSG_LEN(sizeof(int))) {
+ tcp->inq = *reinterpret_cast<int*>(CMSG_DATA(cmsg));
+ }
+ }
}
- GPR_ASSERT((size_t)read_bytes == tcp->incoming_buffer->length);
- call_read_cb(tcp, GRPC_ERROR_NONE);
- TCP_UNREF(tcp, "read");
+#endif /* GRPC_HAVE_TCP_INQ */
+
+ total_read_bytes += read_bytes;
+ if (tcp->inq == 0 || total_read_bytes == tcp->incoming_buffer->length) {
+ /* We have filled incoming_buffer, and we cannot read any more. */
+ break;
+ }
+
+ /* We had a partial read, and still have space to read more data.
+ * So, adjust IOVs and try to read more. */
+ size_t remaining = read_bytes;
+ size_t j = 0;
+ for (size_t i = 0; i < iov_len; i++) {
+ if (remaining >= iov[i].iov_len) {
+ remaining -= iov[i].iov_len;
+ continue;
+ }
+ if (remaining > 0) {
+ iov[j].iov_base = static_cast<char*>(iov[i].iov_base) + remaining;
+ iov[j].iov_len = iov[i].iov_len - remaining;
+ remaining = 0;
+ } else {
+ iov[j].iov_base = iov[i].iov_base;
+ iov[j].iov_len = iov[i].iov_len;
+ }
+ ++j;
+ }
+ iov_len = j;
+ } while (true);
+
+ if (tcp->inq == 0) {
+ finish_estimate(tcp);
}
+
+ GPR_DEBUG_ASSERT(total_read_bytes > 0);
+ if (total_read_bytes < tcp->incoming_buffer->length) {
+ grpc_slice_buffer_trim_end(tcp->incoming_buffer,
+ tcp->incoming_buffer->length - total_read_bytes,
+ &tcp->last_read_buffer);
+ }
+ call_read_cb(tcp, GRPC_ERROR_NONE);
+ TCP_UNREF(tcp, "read");
}
static void tcp_read_allocation_done(void* tcpp, grpc_error* error) {
@@ -512,7 +597,8 @@ static void tcp_read_allocation_done(void* tcpp, grpc_error* error) {
static void tcp_continue_read(grpc_tcp* tcp) {
size_t target_read_size = get_target_read_size(tcp);
- if (tcp->incoming_buffer->length < target_read_size / 2 &&
+ /* Wait for allocation only when there is no buffer left. */
+ if (tcp->incoming_buffer->length == 0 &&
tcp->incoming_buffer->count < MAX_READ_IOVEC) {
if (grpc_tcp_trace.enabled()) {
gpr_log(GPR_INFO, "TCP:%p alloc_slices", tcp);
@@ -544,7 +630,7 @@ static void tcp_handle_read(void* arg /* grpc_tcp */, grpc_error* error) {
}
static void tcp_read(grpc_endpoint* ep, grpc_slice_buffer* incoming_buffer,
- grpc_closure* cb) {
+ grpc_closure* cb, bool urgent) {
grpc_tcp* tcp = reinterpret_cast<grpc_tcp*>(ep);
GPR_ASSERT(tcp->read_cb == nullptr);
tcp->read_cb = cb;
@@ -557,6 +643,11 @@ static void tcp_read(grpc_endpoint* ep, grpc_slice_buffer* incoming_buffer,
* the polling engine */
tcp->is_first_read = false;
notify_on_read(tcp);
+ } else if (!urgent && tcp->inq == 0) {
+ /* Upper layer asked to read more but we know there is no pending data
+ * to read from previous reads. So, wait for POLLIN.
+ */
+ notify_on_read(tcp);
} else {
/* Not the first time. We may or may not have more bytes available. In any
* case call tcp->read_done_closure (i.e tcp_handle_read()) which does the
@@ -1157,6 +1248,19 @@ grpc_endpoint* grpc_tcp_create(grpc_fd* em_fd,
tcp->tb_head = nullptr;
GRPC_CLOSURE_INIT(&tcp->read_done_closure, tcp_handle_read, tcp,
grpc_schedule_on_exec_ctx);
+ /* Always assume there is something on the queue to read. */
+ tcp->inq = 1;
+#ifdef GRPC_HAVE_TCP_INQ
+ int one = 1;
+ if (setsockopt(tcp->fd, SOL_TCP, TCP_INQ, &one, sizeof(one)) == 0) {
+ tcp->inq_capable = true;
+ } else {
+ gpr_log(GPR_INFO, "cannot set inq fd=%d errno=%d", tcp->fd, errno);
+ tcp->inq_capable = false;
+ }
+#else
+ tcp->inq_capable = false;
+#endif /* GRPC_HAVE_TCP_INQ */
/* Start being notified on errors if event engine can track errors. */
if (grpc_event_engine_can_track_errors()) {
/* Grab a ref to tcp so that we can safely access the tcp struct when
diff --git a/src/core/lib/iomgr/tcp_windows.cc b/src/core/lib/iomgr/tcp_windows.cc
index 43817c5a024..7b464651ea1 100644
--- a/src/core/lib/iomgr/tcp_windows.cc
+++ b/src/core/lib/iomgr/tcp_windows.cc
@@ -241,7 +241,7 @@ static void on_read(void* tcpp, grpc_error* error) {
#define DEFAULT_TARGET_READ_SIZE 8192
#define MAX_WSABUF_COUNT 16
static void win_read(grpc_endpoint* ep, grpc_slice_buffer* read_slices,
- grpc_closure* cb) {
+ grpc_closure* cb, bool urgent) {
grpc_tcp* tcp = (grpc_tcp*)ep;
grpc_winsocket* handle = tcp->socket;
grpc_winsocket_callback_info* info = &handle->read_info;
diff --git a/src/core/lib/security/transport/secure_endpoint.cc b/src/core/lib/security/transport/secure_endpoint.cc
index 14fb55884f1..2a862492bd7 100644
--- a/src/core/lib/security/transport/secure_endpoint.cc
+++ b/src/core/lib/security/transport/secure_endpoint.cc
@@ -255,7 +255,7 @@ static void on_read(void* user_data, grpc_error* error) {
}
static void endpoint_read(grpc_endpoint* secure_ep, grpc_slice_buffer* slices,
- grpc_closure* cb) {
+ grpc_closure* cb, bool urgent) {
secure_endpoint* ep = reinterpret_cast<secure_endpoint*>(secure_ep);
ep->read_cb = cb;
ep->read_buffer = slices;
@@ -269,7 +269,7 @@ static void endpoint_read(grpc_endpoint* secure_ep, grpc_slice_buffer* slices,
return;
}
- grpc_endpoint_read(ep->wrapped_ep, &ep->source_buffer, &ep->on_read);
+ grpc_endpoint_read(ep->wrapped_ep, &ep->source_buffer, &ep->on_read, urgent);
}
static void flush_write_staging_buffer(secure_endpoint* ep, uint8_t** cur,
diff --git a/src/core/lib/security/transport/security_handshaker.cc b/src/core/lib/security/transport/security_handshaker.cc
index 5369574b854..3605bbe5974 100644
--- a/src/core/lib/security/transport/security_handshaker.cc
+++ b/src/core/lib/security/transport/security_handshaker.cc
@@ -283,7 +283,7 @@ grpc_error* SecurityHandshaker::OnHandshakeNextDoneLocked(
if (result == TSI_INCOMPLETE_DATA) {
GPR_ASSERT(bytes_to_send_size == 0);
grpc_endpoint_read(args_->endpoint, args_->read_buffer,
- &on_handshake_data_received_from_peer_);
+ &on_handshake_data_received_from_peer_, /*urgent=*/true);
return error;
}
if (result != TSI_OK) {
@@ -306,7 +306,7 @@ grpc_error* SecurityHandshaker::OnHandshakeNextDoneLocked(
} else if (handshaker_result == nullptr) {
// There is nothing to send, but need to read from peer.
grpc_endpoint_read(args_->endpoint, args_->read_buffer,
- &on_handshake_data_received_from_peer_);
+ &on_handshake_data_received_from_peer_, /*urgent=*/true);
} else {
// Handshake has finished, check peer and so on.
error = CheckPeerLocked();
@@ -382,7 +382,8 @@ void SecurityHandshaker::OnHandshakeDataSentToPeerFn(void* arg,
// We may be done.
if (h->handshaker_result_ == nullptr) {
grpc_endpoint_read(h->args_->endpoint, h->args_->read_buffer,
- &h->on_handshake_data_received_from_peer_);
+ &h->on_handshake_data_received_from_peer_,
+ /*urgent=*/true);
} else {
error = h->CheckPeerLocked();
if (error != GRPC_ERROR_NONE) {
diff --git a/test/core/bad_client/bad_client.cc b/test/core/bad_client/bad_client.cc
index ae1e42a4e0d..6b492523219 100644
--- a/test/core/bad_client/bad_client.cc
+++ b/test/core/bad_client/bad_client.cc
@@ -143,7 +143,8 @@ void grpc_run_client_side_validator(grpc_bad_client_arg* arg, uint32_t flags,
grpc_closure read_done_closure;
GRPC_CLOSURE_INIT(&read_done_closure, set_read_done, &read_done_event,
grpc_schedule_on_exec_ctx);
- grpc_endpoint_read(sfd->client, &incoming, &read_done_closure);
+ grpc_endpoint_read(sfd->client, &incoming, &read_done_closure,
+ /*urgent=*/true);
grpc_core::ExecCtx::Get()->Flush();
do {
GPR_ASSERT(gpr_time_cmp(deadline, gpr_now(deadline.clock_type)) > 0);
diff --git a/test/core/end2end/bad_server_response_test.cc b/test/core/end2end/bad_server_response_test.cc
index 99cfec7adf6..3701a938a3d 100644
--- a/test/core/end2end/bad_server_response_test.cc
+++ b/test/core/end2end/bad_server_response_test.cc
@@ -126,7 +126,8 @@ static void handle_read(void* arg, grpc_error* error) {
SERVER_INCOMING_DATA_LENGTH_LOWER_THRESHOLD) {
handle_write();
} else {
- grpc_endpoint_read(state.tcp, &state.temp_incoming_buffer, &on_read);
+ grpc_endpoint_read(state.tcp, &state.temp_incoming_buffer, &on_read,
+ /*urgent=*/false);
}
}
@@ -142,7 +143,8 @@ static void on_connect(void* arg, grpc_endpoint* tcp,
state.tcp = tcp;
state.incoming_data_length = 0;
grpc_endpoint_add_to_pollset(tcp, server->pollset);
- grpc_endpoint_read(tcp, &state.temp_incoming_buffer, &on_read);
+ grpc_endpoint_read(tcp, &state.temp_incoming_buffer, &on_read,
+ /*urgent=*/false);
}
static gpr_timespec n_sec_deadline(int seconds) {
diff --git a/test/core/end2end/fixtures/http_proxy_fixture.cc b/test/core/end2end/fixtures/http_proxy_fixture.cc
index e6fc5dfcfca..6b5513f160e 100644
--- a/test/core/end2end/fixtures/http_proxy_fixture.cc
+++ b/test/core/end2end/fixtures/http_proxy_fixture.cc
@@ -271,7 +271,7 @@ static void on_client_read_done(void* arg, grpc_error* error) {
}
// Read more data.
grpc_endpoint_read(conn->client_endpoint, &conn->client_read_buffer,
- &conn->on_client_read_done);
+ &conn->on_client_read_done, /*urgent=*/false);
}
// Callback for reading data from the backend server, which will be
@@ -302,7 +302,7 @@ static void on_server_read_done(void* arg, grpc_error* error) {
}
// Read more data.
grpc_endpoint_read(conn->server_endpoint, &conn->server_read_buffer,
- &conn->on_server_read_done);
+ &conn->on_server_read_done, /*urgent=*/false);
}
// Callback to write the HTTP response for the CONNECT request.
@@ -323,9 +323,9 @@ static void on_write_response_done(void* arg, grpc_error* error) {
proxy_connection_ref(conn, "server_read");
proxy_connection_unref(conn, "write_response");
grpc_endpoint_read(conn->client_endpoint, &conn->client_read_buffer,
- &conn->on_client_read_done);
+ &conn->on_client_read_done, /*urgent=*/false);
grpc_endpoint_read(conn->server_endpoint, &conn->server_read_buffer,
- &conn->on_server_read_done);
+ &conn->on_server_read_done, /*urgent=*/false);
}
// Callback to connect to the backend server specified by the HTTP
@@ -405,7 +405,7 @@ static void on_read_request_done(void* arg, grpc_error* error) {
// If we're not done reading the request, read more data.
if (conn->http_parser.state != GRPC_HTTP_BODY) {
grpc_endpoint_read(conn->client_endpoint, &conn->client_read_buffer,
- &conn->on_read_request_done);
+ &conn->on_read_request_done, /*urgent=*/false);
return;
}
// Make sure we got a CONNECT request.
@@ -503,7 +503,7 @@ static void on_accept(void* arg, grpc_endpoint* endpoint,
grpc_http_parser_init(&conn->http_parser, GRPC_HTTP_REQUEST,
&conn->http_request);
grpc_endpoint_read(conn->client_endpoint, &conn->client_read_buffer,
- &conn->on_read_request_done);
+ &conn->on_read_request_done, /*urgent=*/false);
}
//
diff --git a/test/core/handshake/readahead_handshaker_server_ssl.cc b/test/core/handshake/readahead_handshaker_server_ssl.cc
index d91f2d2fe63..c0ab61136cb 100644
--- a/test/core/handshake/readahead_handshaker_server_ssl.cc
+++ b/test/core/handshake/readahead_handshaker_server_ssl.cc
@@ -59,7 +59,8 @@ class ReadAheadHandshaker : public Handshaker {
void DoHandshake(grpc_tcp_server_acceptor* acceptor,
grpc_closure* on_handshake_done,
HandshakerArgs* args) override {
- grpc_endpoint_read(args->endpoint, args->read_buffer, on_handshake_done);
+ grpc_endpoint_read(args->endpoint, args->read_buffer, on_handshake_done,
+ /*urgent=*/false);
}
};
diff --git a/test/core/iomgr/endpoint_tests.cc b/test/core/iomgr/endpoint_tests.cc
index a9e8ba86c5d..beae24769f6 100644
--- a/test/core/iomgr/endpoint_tests.cc
+++ b/test/core/iomgr/endpoint_tests.cc
@@ -129,7 +129,8 @@ static void read_and_write_test_read_handler(void* data, grpc_error* error) {
GRPC_LOG_IF_ERROR("pollset_kick", grpc_pollset_kick(g_pollset, nullptr));
gpr_mu_unlock(g_mu);
} else if (error == GRPC_ERROR_NONE) {
- grpc_endpoint_read(state->read_ep, &state->incoming, &state->done_read);
+ grpc_endpoint_read(state->read_ep, &state->incoming, &state->done_read,
+ /*urgent=*/false);
}
}
@@ -216,8 +217,8 @@ static void read_and_write_test(grpc_endpoint_test_config config,
read_and_write_test_write_handler(&state, GRPC_ERROR_NONE);
grpc_core::ExecCtx::Get()->Flush();
- grpc_endpoint_read(state.read_ep, &state.incoming, &state.done_read);
-
+ grpc_endpoint_read(state.read_ep, &state.incoming, &state.done_read,
+ /*urgent=*/false);
if (shutdown) {
gpr_log(GPR_DEBUG, "shutdown read");
grpc_endpoint_shutdown(
@@ -282,14 +283,16 @@ static void multiple_shutdown_test(grpc_endpoint_test_config config) {
grpc_endpoint_add_to_pollset(f.client_ep, g_pollset);
grpc_endpoint_read(f.client_ep, &slice_buffer,
GRPC_CLOSURE_CREATE(inc_on_failure, &fail_count,
- grpc_schedule_on_exec_ctx));
+ grpc_schedule_on_exec_ctx),
+ /*urgent=*/false);
wait_for_fail_count(&fail_count, 0);
grpc_endpoint_shutdown(f.client_ep,
GRPC_ERROR_CREATE_FROM_STATIC_STRING("Test Shutdown"));
wait_for_fail_count(&fail_count, 1);
grpc_endpoint_read(f.client_ep, &slice_buffer,
GRPC_CLOSURE_CREATE(inc_on_failure, &fail_count,
- grpc_schedule_on_exec_ctx));
+ grpc_schedule_on_exec_ctx),
+ /*urgent=*/false);
wait_for_fail_count(&fail_count, 2);
grpc_slice_buffer_add(&slice_buffer, grpc_slice_from_copied_string("a"));
grpc_endpoint_write(f.client_ep, &slice_buffer,
diff --git a/test/core/iomgr/tcp_posix_test.cc b/test/core/iomgr/tcp_posix_test.cc
index 5b601b1ae5f..33a4d973ed3 100644
--- a/test/core/iomgr/tcp_posix_test.cc
+++ b/test/core/iomgr/tcp_posix_test.cc
@@ -191,7 +191,8 @@ static void read_cb(void* user_data, grpc_error* error) {
GRPC_LOG_IF_ERROR("kick", grpc_pollset_kick(g_pollset, nullptr)));
gpr_mu_unlock(g_mu);
} else {
- grpc_endpoint_read(state->ep, &state->incoming, &state->read_cb);
+ grpc_endpoint_read(state->ep, &state->incoming, &state->read_cb,
+ /*urgent=*/false);
gpr_mu_unlock(g_mu);
}
}
@@ -229,7 +230,7 @@ static void read_test(size_t num_bytes, size_t slice_size) {
grpc_slice_buffer_init(&state.incoming);
GRPC_CLOSURE_INIT(&state.read_cb, read_cb, &state, grpc_schedule_on_exec_ctx);
- grpc_endpoint_read(ep, &state.incoming, &state.read_cb);
+ grpc_endpoint_read(ep, &state.incoming, &state.read_cb, /*urgent=*/false);
gpr_mu_lock(g_mu);
while (state.read_bytes < state.target_read_bytes) {
@@ -280,7 +281,7 @@ static void large_read_test(size_t slice_size) {
grpc_slice_buffer_init(&state.incoming);
GRPC_CLOSURE_INIT(&state.read_cb, read_cb, &state, grpc_schedule_on_exec_ctx);
- grpc_endpoint_read(ep, &state.incoming, &state.read_cb);
+ grpc_endpoint_read(ep, &state.incoming, &state.read_cb, /*urgent=*/false);
gpr_mu_lock(g_mu);
while (state.read_bytes < state.target_read_bytes) {
@@ -519,7 +520,7 @@ static void release_fd_test(size_t num_bytes, size_t slice_size) {
grpc_slice_buffer_init(&state.incoming);
GRPC_CLOSURE_INIT(&state.read_cb, read_cb, &state, grpc_schedule_on_exec_ctx);
- grpc_endpoint_read(ep, &state.incoming, &state.read_cb);
+ grpc_endpoint_read(ep, &state.incoming, &state.read_cb, /*urgent=*/false);
gpr_mu_lock(g_mu);
while (state.read_bytes < state.target_read_bytes) {
diff --git a/test/core/security/secure_endpoint_test.cc b/test/core/security/secure_endpoint_test.cc
index f6d02895b5f..3a2d599767a 100644
--- a/test/core/security/secure_endpoint_test.cc
+++ b/test/core/security/secure_endpoint_test.cc
@@ -182,7 +182,7 @@ static void test_leftover(grpc_endpoint_test_config config, size_t slice_size) {
grpc_slice_buffer_init(&incoming);
GRPC_CLOSURE_INIT(&done_closure, inc_call_ctr, &n, grpc_schedule_on_exec_ctx);
- grpc_endpoint_read(f.client_ep, &incoming, &done_closure);
+ grpc_endpoint_read(f.client_ep, &incoming, &done_closure, /*urgent=*/false);
grpc_core::ExecCtx::Get()->Flush();
GPR_ASSERT(n == 1);
diff --git a/test/core/transport/chttp2/settings_timeout_test.cc b/test/core/transport/chttp2/settings_timeout_test.cc
index a9789edbf2b..32a268ed521 100644
--- a/test/core/transport/chttp2/settings_timeout_test.cc
+++ b/test/core/transport/chttp2/settings_timeout_test.cc
@@ -133,7 +133,8 @@ class Client {
grpc_millis deadline = grpc_core::ExecCtx::Get()->Now() + 3000;
while (true) {
EventState state;
- grpc_endpoint_read(endpoint_, &read_buffer, state.closure());
+ grpc_endpoint_read(endpoint_, &read_buffer, state.closure(),
+ /*urgent=*/true);
if (!PollUntilDone(&state, deadline)) {
retval = false;
break;
diff --git a/test/core/util/mock_endpoint.cc b/test/core/util/mock_endpoint.cc
index df2ee7aedfd..2f78a7f8a97 100644
--- a/test/core/util/mock_endpoint.cc
+++ b/test/core/util/mock_endpoint.cc
@@ -41,7 +41,7 @@ typedef struct mock_endpoint {
} mock_endpoint;
static void me_read(grpc_endpoint* ep, grpc_slice_buffer* slices,
- grpc_closure* cb) {
+ grpc_closure* cb, bool urgent) {
mock_endpoint* m = reinterpret_cast<mock_endpoint*>(ep);
gpr_mu_lock(&m->mu);
if (m->read_buffer.count > 0) {
diff --git a/test/core/util/passthru_endpoint.cc b/test/core/util/passthru_endpoint.cc
index 51b6de46951..2d26902fc44 100644
--- a/test/core/util/passthru_endpoint.cc
+++ b/test/core/util/passthru_endpoint.cc
@@ -54,7 +54,7 @@ struct passthru_endpoint {
};
static void me_read(grpc_endpoint* ep, grpc_slice_buffer* slices,
- grpc_closure* cb) {
+ grpc_closure* cb, bool urgent) {
half* m = reinterpret_cast<half*>(ep);
gpr_mu_lock(&m->parent->mu);
if (m->parent->shutdown) {
diff --git a/test/core/util/trickle_endpoint.cc b/test/core/util/trickle_endpoint.cc
index b0da735e57f..bdac1334f48 100644
--- a/test/core/util/trickle_endpoint.cc
+++ b/test/core/util/trickle_endpoint.cc
@@ -47,9 +47,9 @@ typedef struct {
} trickle_endpoint;
static void te_read(grpc_endpoint* ep, grpc_slice_buffer* slices,
- grpc_closure* cb) {
+ grpc_closure* cb, bool urgent) {
trickle_endpoint* te = reinterpret_cast<trickle_endpoint*>(ep);
- grpc_endpoint_read(te->wrapped, slices, cb);
+ grpc_endpoint_read(te->wrapped, slices, cb, urgent);
}
static void maybe_call_write_cb_locked(trickle_endpoint* te) {
diff --git a/test/cpp/microbenchmarks/bm_chttp2_transport.cc b/test/cpp/microbenchmarks/bm_chttp2_transport.cc
index dcfaa684773..baa6da3fbcf 100644
--- a/test/cpp/microbenchmarks/bm_chttp2_transport.cc
+++ b/test/cpp/microbenchmarks/bm_chttp2_transport.cc
@@ -92,7 +92,7 @@ class DummyEndpoint : public grpc_endpoint {
}
static void read(grpc_endpoint* ep, grpc_slice_buffer* slices,
- grpc_closure* cb) {
+ grpc_closure* cb, bool urgent) {
static_cast<DummyEndpoint*>(ep)->QueueRead(slices, cb);
}
From 240bf8676093d52107be1fffed5ed61ec06b61b4 Mon Sep 17 00:00:00 2001
From: Vijay Pai <vpai@google.com>
Date: Wed, 6 Mar 2019 23:01:05 -0800
Subject: [PATCH 051/138] Add unimplemented RPC test
---
.../end2end/client_callback_end2end_test.cc | 33 +++++++++++++++++++
1 file changed, 33 insertions(+)
diff --git a/test/cpp/end2end/client_callback_end2end_test.cc b/test/cpp/end2end/client_callback_end2end_test.cc
index 893d009392d..3845c4c0b2a 100644
--- a/test/cpp/end2end/client_callback_end2end_test.cc
+++ b/test/cpp/end2end/client_callback_end2end_test.cc
@@ -1084,6 +1084,39 @@ TEST_P(ClientCallbackEnd2endTest, SimultaneousReadAndWritesDone) {
test.Await();
}
+TEST_P(ClientCallbackEnd2endTest, UnimplementedRpc) {
+ MAYBE_SKIP_TEST;
+ ChannelArguments args;
+ const auto& channel_creds = GetCredentialsProvider()->GetChannelCredentials(
+ GetParam().credentials_type, &args);
+ std::shared_ptr<Channel> channel =
+ (GetParam().protocol == Protocol::TCP)
+ ? CreateCustomChannel(server_address_.str(), channel_creds, args)
+ : server_->InProcessChannel(args);
+ std::unique_ptr<grpc::testing::UnimplementedEchoService::Stub> stub;
+ stub = grpc::testing::UnimplementedEchoService::NewStub(channel);
+ EchoRequest request;
+ EchoResponse response;
+ ClientContext cli_ctx;
+ request.set_message("Hello world.");
+ std::mutex mu;
+ std::condition_variable cv;
+ bool done = false;
+ stub->experimental_async()->Unimplemented(
+ &cli_ctx, &request, &response, [&done, &mu, &cv](Status s) {
+ EXPECT_EQ(StatusCode::UNIMPLEMENTED, s.error_code());
+ EXPECT_EQ("", s.error_message());
+
+ std::lock_guard<std::mutex> l(mu);
+ done = true;
+ cv.notify_one();
+ });
+ std::unique_lock<std::mutex> l(mu);
+ while (!done) {
+ cv.wait(l);
+ }
+}
+
std::vector<TestScenario> CreateTestScenarios(bool test_insecure) {
std::vector<TestScenario> scenarios;
std::vector<grpc::string> credentials_types{
From b7f14fdab8c6da31b8009a4b4dc395a913660493 Mon Sep 17 00:00:00 2001
From: Vijay Pai <vpai@google.com>
Date: Wed, 6 Mar 2019 23:43:34 -0800
Subject: [PATCH 052/138] Properly implement unimplemented RPCs at
callback-only server
---
include/grpcpp/server.h | 4 ++++
src/cpp/server/server_cc.cc | 8 ++++++++
2 files changed, 12 insertions(+)
diff --git a/include/grpcpp/server.h b/include/grpcpp/server.h
index 21c908aebdb..f5c99f22df2 100644
--- a/include/grpcpp/server.h
+++ b/include/grpcpp/server.h
@@ -326,6 +326,10 @@ class Server : public ServerInterface, private GrpcLibraryCodegen {
std::unique_ptr<HealthCheckServiceInterface> health_check_service_;
bool health_check_service_disabled_;
+ // When appropriate, use a default callback generic service to handle
+ // unimplemented methods
+ std::unique_ptr<experimental::CallbackGenericService> unimplemented_service_;
+
// A special handler for resource exhausted in sync case
std::unique_ptr<internal::MethodHandler> resource_exhausted_handler_;
diff --git a/src/cpp/server/server_cc.cc b/src/cpp/server/server_cc.cc
index 6e78e93b835..26e84f1aed4 100644
--- a/src/cpp/server/server_cc.cc
+++ b/src/cpp/server/server_cc.cc
@@ -1004,6 +1004,14 @@ void Server::Start(ServerCompletionQueue** cqs, size_t num_cqs) {
RegisterService(nullptr, default_health_check_service_impl);
}
+ // If this server uses callback methods, then create a callback generic
+ // service to handle any unimplemented methods using the default reactor
+ // creator
+ if (!callback_reqs_to_start_.empty() && !has_callback_generic_service_) {
+ unimplemented_service_.reset(new experimental::CallbackGenericService);
+ RegisterCallbackGenericService(unimplemented_service_.get());
+ }
+
grpc_server_start(server_);
if (!has_async_generic_service_ && !has_callback_generic_service_) {
From b1dbf6837358b87766564b7ab307b7e18b6fc5f4 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Thu, 7 Mar 2019 03:15:08 -0500
Subject: [PATCH 053/138] update the docker image to netcore3 preview3
---
.../grpc_interop_aspnetcore/Dockerfile.template | 11 ++---------
.../interoptest/grpc_interop_aspnetcore/Dockerfile | 11 ++---------
2 files changed, 4 insertions(+), 18 deletions(-)
diff --git a/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template b/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template
index a341592a2ab..e8f962403f7 100644
--- a/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template
+++ b/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template
@@ -1,6 +1,6 @@
%YAML 1.2
--- |
- # Copyright 2017 gRPC authors.
+ # Copyright 2019 The gRPC Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,14 +14,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
- FROM microsoft/dotnet:3.0.100-preview2-sdk-stretch
-
- RUN rm /usr/bin/dotnet # remove symlink
- RUN curl -sSL -o dotnet.tar.gz https://dotnetcli.azureedge.net/dotnet/Sdk/3.0.100-preview3-010313/dotnet-sdk-3.0.100-preview3-010313-linux-x64.tar.gz ${'\\'}
- && mkdir -p /usr/share/dotnet ${'\\'}
- && tar -zxf dotnet.tar.gz -C /usr/share/dotnet ${'\\'}
- && rm dotnet.tar.gz ${'\\'}
- && ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet
+ FROM mcr.microsoft.com/dotnet/core/sdk:3.0.100-preview3-stretch
# Define the default command.
CMD ["bash"]
diff --git a/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile b/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile
index 75a8a200ab4..26a21384911 100644
--- a/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile
+++ b/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile
@@ -1,4 +1,4 @@
-# Copyright 2017 gRPC authors.
+# Copyright 2019 The gRPC Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -12,14 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-FROM microsoft/dotnet:3.0.100-preview2-sdk-stretch
-
-RUN rm /usr/bin/dotnet # remove symlink
-RUN curl -sSL -o dotnet.tar.gz https://dotnetcli.azureedge.net/dotnet/Sdk/3.0.100-preview3-010313/dotnet-sdk-3.0.100-preview3-010313-linux-x64.tar.gz \
- && mkdir -p /usr/share/dotnet \
- && tar -zxf dotnet.tar.gz -C /usr/share/dotnet \
- && rm dotnet.tar.gz \
- && ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet
+FROM mcr.microsoft.com/dotnet/core/sdk:3.0.100-preview3-stretch
# Define the default command.
CMD ["bash"]
From 827c77bd240f5d6dbc9876e4c2db2b492953c862 Mon Sep 17 00:00:00 2001
From: "Mark D. Roth" <roth@google.com>
Date: Thu, 7 Mar 2019 07:57:29 -0800
Subject: [PATCH 054/138] Use fallback before timeout if balancer channel
reports TRANSIENT_FAILURE.
---
.../client_channel/lb_policy/grpclb/grpclb.cc | 80 +++++++++++++++++--
.../client_channel/lb_policy/xds/xds.cc | 3 +
test/cpp/end2end/grpclb_end2end_test.cc | 14 ++++
test/cpp/end2end/xds_end2end_test.cc | 3 +
4 files changed, 94 insertions(+), 6 deletions(-)
diff --git a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
index 5b3bb40f8cd..d6dde0d7a79 100644
--- a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
@@ -295,8 +295,10 @@ class GrpcLb : public LoadBalancingPolicy {
// Helper functions used in UpdateLocked().
void ProcessChannelArgsLocked(const grpc_channel_args& args);
void ParseLbConfig(Config* grpclb_config);
+ static void OnBalancerChannelConnectivityChangedLocked(void* arg,
+ grpc_error* error);
- // Methods for dealing with the balancer channel and call.
+ // Methods for dealing with the balancer call.
void StartBalancerCallLocked();
static void OnFallbackTimerLocked(void* arg, grpc_error* error);
void StartBalancerCallRetryTimerLocked();
@@ -323,6 +325,9 @@ class GrpcLb : public LoadBalancingPolicy {
gpr_atm lb_channel_uuid_ = 0;
// Response generator to inject address updates into lb_channel_.
RefCountedPtr<FakeResolverResponseGenerator> response_generator_;
+ // Connectivity state notification.
+ grpc_connectivity_state lb_channel_connectivity_ = GRPC_CHANNEL_IDLE;
+ grpc_closure lb_channel_on_connectivity_changed_;
// The data associated with the current LB call. It holds a ref to this LB
// policy. It's initialized every time we query for backends. It's reset to
@@ -1030,6 +1035,12 @@ void GrpcLb::BalancerCallState::OnBalancerMessageReceivedLocked(
} else { // New serverlist.
if (grpclb_policy->serverlist_ == nullptr) {
// Dispose of the fallback.
+ if (grpclb_policy->child_policy_ != nullptr) {
+ gpr_log(GPR_INFO,
+ "[grpclb %p] Received response from balancer; exiting "
+ "fallback mode",
+ grpclb_policy);
+ }
grpclb_policy->fallback_backend_addresses_.reset();
if (grpclb_policy->fallback_timer_callback_pending_) {
grpc_timer_cancel(&grpclb_policy->lb_fallback_timer_);
@@ -1219,6 +1230,10 @@ GrpcLb::GrpcLb(Args args)
.set_jitter(GRPC_GRPCLB_RECONNECT_JITTER)
.set_max_backoff(GRPC_GRPCLB_RECONNECT_MAX_BACKOFF_SECONDS *
1000)) {
+ // Initialization.
+ GRPC_CLOSURE_INIT(&lb_channel_on_connectivity_changed_,
+ &GrpcLb::OnBalancerChannelConnectivityChangedLocked, this,
+ grpc_combiner_scheduler(args.combiner));
gpr_mu_init(&child_policy_mu_);
// Record server name.
const grpc_arg* arg = grpc_channel_args_find(args.args, GRPC_ARG_SERVER_URI);
@@ -1329,6 +1344,20 @@ void GrpcLb::UpdateLocked(const grpc_channel_args& args,
grpc_combiner_scheduler(combiner()));
fallback_timer_callback_pending_ = true;
grpc_timer_init(&lb_fallback_timer_, deadline, &lb_on_fallback_);
+ // Start watching the channel's connectivity state. If the channel
+ // goes into state TRANSIENT_FAILURE, we go into fallback mode even if
+ // the fallback timeout has not elapsed.
+ grpc_channel_element* client_channel_elem =
+ grpc_channel_stack_last_element(
+ grpc_channel_get_channel_stack(lb_channel_));
+ GPR_ASSERT(client_channel_elem->filter == &grpc_client_channel_filter);
+ // Ref held by callback.
+ Ref(DEBUG_LOCATION, "watch_lb_channel_connectivity").release();
+ grpc_client_channel_watch_connectivity_state(
+ client_channel_elem,
+ grpc_polling_entity_create_from_pollset_set(interested_parties()),
+ &lb_channel_connectivity_, &lb_channel_on_connectivity_changed_,
+ nullptr);
}
StartBalancerCallLocked();
}
@@ -1420,6 +1449,37 @@ void GrpcLb::ParseLbConfig(Config* grpclb_config) {
}
}
+void GrpcLb::OnBalancerChannelConnectivityChangedLocked(void* arg,
+ grpc_error* error) {
+ GrpcLb* self = static_cast<GrpcLb*>(arg);
+ if (!self->shutting_down_ && self->fallback_timer_callback_pending_) {
+ if (self->lb_channel_connectivity_ != GRPC_CHANNEL_TRANSIENT_FAILURE) {
+ // Not in TRANSIENT_FAILURE. Renew connectivity watch.
+ grpc_channel_element* client_channel_elem =
+ grpc_channel_stack_last_element(
+ grpc_channel_get_channel_stack(self->lb_channel_));
+ GPR_ASSERT(client_channel_elem->filter == &grpc_client_channel_filter);
+ grpc_client_channel_watch_connectivity_state(
+ client_channel_elem,
+ grpc_polling_entity_create_from_pollset_set(
+ self->interested_parties()),
+ &self->lb_channel_connectivity_,
+ &self->lb_channel_on_connectivity_changed_, nullptr);
+ return; // Early out so we don't drop the ref below.
+ }
+ // In TRANSIENT_FAILURE. Cancel the fallback timer and go into
+ // fallback mode immediately.
+ gpr_log(GPR_INFO,
+ "[grpclb %p] balancer channel in state TRANSIENT_FAILURE; "
+ "entering fallback mode",
+ self);
+ grpc_timer_cancel(&self->lb_fallback_timer_);
+ self->CreateOrUpdateChildPolicyLocked();
+ }
+ // Done watching connectivity state, so drop ref.
+ self->Unref(DEBUG_LOCATION, "watch_lb_channel_connectivity");
+}
+
//
// code for balancer channel and call
//
@@ -1445,13 +1505,21 @@ void GrpcLb::OnFallbackTimerLocked(void* arg, grpc_error* error) {
// actually runs, don't fall back.
if (grpclb_policy->serverlist_ == nullptr && !grpclb_policy->shutting_down_ &&
error == GRPC_ERROR_NONE) {
- if (grpc_lb_glb_trace.enabled()) {
- gpr_log(GPR_INFO,
- "[grpclb %p] Falling back to use backends from resolver",
- grpclb_policy);
- }
+ gpr_log(GPR_INFO,
+ "[grpclb %p] No response from balancer after fallback timeout; "
+ "entering fallback mode",
+ grpclb_policy);
GPR_ASSERT(grpclb_policy->fallback_backend_addresses_ != nullptr);
grpclb_policy->CreateOrUpdateChildPolicyLocked();
+ // Cancel connectivity watch, since we no longer need it.
+ grpc_channel_element* client_channel_elem = grpc_channel_stack_last_element(
+ grpc_channel_get_channel_stack(grpclb_policy->lb_channel_));
+ GPR_ASSERT(client_channel_elem->filter == &grpc_client_channel_filter);
+ grpc_client_channel_watch_connectivity_state(
+ client_channel_elem,
+ grpc_polling_entity_create_from_pollset_set(
+ grpclb_policy->interested_parties()),
+ nullptr, &grpclb_policy->lb_channel_on_connectivity_changed_, nullptr);
}
grpclb_policy->Unref(DEBUG_LOCATION, "on_fallback_timer");
}
diff --git a/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc b/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc
index dbe68190ff5..a9ca34e5a52 100644
--- a/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc
@@ -1254,6 +1254,9 @@ void XdsLb::UpdateLocked(const grpc_channel_args& args,
grpc_combiner_scheduler(combiner()));
fallback_timer_callback_pending_ = true;
grpc_timer_init(&lb_fallback_timer_, deadline, &lb_on_fallback_);
+ // TODO(juanlishen): Monitor the connectivity state of the balancer
+ // channel. If the channel reports TRANSIENT_FAILURE before the
+ // fallback timeout expires, go into fallback mode early.
}
}
}
diff --git a/test/cpp/end2end/grpclb_end2end_test.cc b/test/cpp/end2end/grpclb_end2end_test.cc
index fe865ed4b72..abce031c539 100644
--- a/test/cpp/end2end/grpclb_end2end_test.cc
+++ b/test/cpp/end2end/grpclb_end2end_test.cc
@@ -1194,6 +1194,20 @@ TEST_F(SingleBalancerTest, FallbackUpdate) {
EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
}
+TEST_F(SingleBalancerTest, FallbackEarlyWhenBalancerChannelFails) {
+ const int kFallbackTimeoutMs = 10000 * grpc_test_slowdown_factor();
+ ResetStub(kFallbackTimeoutMs);
+ // Return an unreachable balancer and one fallback backend.
+ std::vector<AddressData> addresses;
+ addresses.emplace_back(AddressData{grpc_pick_unused_port_or_die(), true, ""});
+ addresses.emplace_back(AddressData{backend_servers_[0].port_, false, ""});
+ SetNextResolution(addresses);
+ // Send RPC with deadline less than the fallback timeout and make sure it
+ // succeeds.
+ CheckRpcSendOk(/* times */ 1, /* timeout_ms */ 1000,
+ /* wait_for_ready */ false);
+}
+
TEST_F(SingleBalancerTest, BackendsRestart) {
SetNextResolutionAllBalancers();
const size_t kNumRpcsPerAddress = 100;
diff --git a/test/cpp/end2end/xds_end2end_test.cc b/test/cpp/end2end/xds_end2end_test.cc
index 09556675d43..667481bafd2 100644
--- a/test/cpp/end2end/xds_end2end_test.cc
+++ b/test/cpp/end2end/xds_end2end_test.cc
@@ -868,6 +868,9 @@ TEST_F(SingleBalancerTest, AllServersUnreachableFailFast) {
// TODO(juanlishen): Add TEST_F(SingleBalancerTest, FallbackUpdate)
+// TODO(juanlishen): Add TEST_F(SingleBalancerTest,
+// FallbackEarlyWhenBalancerChannelFails)
+
TEST_F(SingleBalancerTest, BackendsRestart) {
SetNextResolution({}, kDefaultServiceConfig_.c_str());
SetNextResolutionForLbChannelAllBalancers();
From 65ef4f5cef300fd63c127d8dcf485261e0e1420c Mon Sep 17 00:00:00 2001
From: billfeng327 <yfen@google.com>
Date: Thu, 7 Mar 2019 11:00:06 -0800
Subject: [PATCH 055/138] added TODO and updated documentation for manual local
windows build
---
tools/remote_build/README.md | 6 ++++++
tools/remote_build/windows.bazelrc | 1 +
2 files changed, 7 insertions(+)
diff --git a/tools/remote_build/README.md b/tools/remote_build/README.md
index 19739e9ee12..8a236973946 100644
--- a/tools/remote_build/README.md
+++ b/tools/remote_build/README.md
@@ -29,5 +29,11 @@ Sanitizer runs (asan, msan, tsan, ubsan):
bazel --bazelrc=tools/remote_build/manual.bazelrc test --config=asan //test/...
```
+Run on Windows MSVC:
+```
+# local manual run only for C++ targets (RBE to be supported)
+bazel --bazelrc=tools/remote_build/windows.bazelrc test //test/cpp/...
+```
+
Available command line options can be found in
[Bazel command line reference](https://docs.bazel.build/versions/master/command-line-reference.html)
diff --git a/tools/remote_build/windows.bazelrc b/tools/remote_build/windows.bazelrc
index a74f23b8048..86025006e3d 100644
--- a/tools/remote_build/windows.bazelrc
+++ b/tools/remote_build/windows.bazelrc
@@ -1,2 +1,3 @@
+# TODO(yfen): Merge with rbe_common.bazelrc and enable Windows RBE
build --test_tag_filters=-exclude_windows
build --build_tag_filters=-exclude_windows
\ No newline at end of file
From d15605c0e5813d25e2dd1a65805496488064a919 Mon Sep 17 00:00:00 2001
From: Juanli Shen <juanlishen@google.com>
Date: Thu, 7 Mar 2019 12:42:51 -0800
Subject: [PATCH 056/138] Swap in new LB policy when it's ready
---
.../client_channel/lb_policy/grpclb/grpclb.cc | 6 +-
.../client_channel/lb_policy/xds/xds.cc | 225 ++++++++++++---
.../client_channel/resolving_lb_policy.cc | 270 ++++++++++++++----
.../client_channel/resolving_lb_policy.h | 13 +-
4 files changed, 417 insertions(+), 97 deletions(-)
diff --git a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
index d6dde0d7a79..184215a3da9 100644
--- a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
@@ -307,7 +307,7 @@ class GrpcLb : public LoadBalancingPolicy {
// Methods for dealing with the child policy.
grpc_channel_args* CreateChildPolicyArgsLocked();
OrphanablePtr<LoadBalancingPolicy> CreateChildPolicyLocked(
- const char* name, grpc_channel_args* args);
+ const char* name, const grpc_channel_args* args);
void CreateOrUpdateChildPolicyLocked();
// Who the client is trying to communicate with.
@@ -685,7 +685,7 @@ void GrpcLb::Helper::UpdateState(grpc_connectivity_state state,
void GrpcLb::Helper::RequestReresolution() {
if (parent_->shutting_down_) return;
// If there is a pending child policy, ignore re-resolution requests
- // from the current child policy (or any outdated pending child).
+ // from the current child policy (or any outdated child).
if (parent_->pending_child_policy_ != nullptr && !CalledByPendingChild()) {
return;
}
@@ -1608,7 +1608,7 @@ grpc_channel_args* GrpcLb::CreateChildPolicyArgsLocked() {
}
OrphanablePtr<LoadBalancingPolicy> GrpcLb::CreateChildPolicyLocked(
- const char* name, grpc_channel_args* args) {
+ const char* name, const grpc_channel_args* args) {
Helper* helper = New<Helper>(Ref());
LoadBalancingPolicy::Args lb_policy_args;
lb_policy_args.combiner = combiner();
diff --git a/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc b/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc
index a9ca34e5a52..4b386d37797 100644
--- a/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc
@@ -278,8 +278,14 @@ class XdsLb : public LoadBalancingPolicy {
UniquePtr<SubchannelPicker> picker) override;
void RequestReresolution() override;
+ void set_child(LoadBalancingPolicy* child) { child_ = child; }
+
private:
+ bool CalledByPendingChild() const;
+ bool CalledByCurrentChild() const;
+
RefCountedPtr<XdsLb> parent_;
+ LoadBalancingPolicy* child_ = nullptr;
};
~XdsLb();
@@ -306,7 +312,8 @@ class XdsLb : public LoadBalancingPolicy {
// Methods for dealing with the child policy.
void CreateOrUpdateChildPolicyLocked();
grpc_channel_args* CreateChildPolicyArgsLocked();
- void CreateChildPolicyLocked(const char* name, Args args);
+ OrphanablePtr<LoadBalancingPolicy> CreateChildPolicyLocked(
+ const char* name, const grpc_channel_args* args);
// Who the client is trying to communicate with.
const char* server_name_ = nullptr;
@@ -349,6 +356,10 @@ class XdsLb : public LoadBalancingPolicy {
// The policy to use for the backends.
RefCountedPtr<Config> child_policy_config_;
OrphanablePtr<LoadBalancingPolicy> child_policy_;
+ OrphanablePtr<LoadBalancingPolicy> pending_child_policy_;
+ // Lock held when modifying the value of child_policy_ or
+ // pending_child_policy_.
+ gpr_mu child_policy_mu_;
};
//
@@ -372,14 +383,30 @@ XdsLb::Picker::PickResult XdsLb::Picker::Pick(PickState* pick,
// XdsLb::Helper
//
+bool XdsLb::Helper::CalledByPendingChild() const {
+ GPR_ASSERT(child_ != nullptr);
+ return child_ == parent_->pending_child_policy_.get();
+}
+
+bool XdsLb::Helper::CalledByCurrentChild() const {
+ GPR_ASSERT(child_ != nullptr);
+ return child_ == parent_->child_policy_.get();
+}
+
Subchannel* XdsLb::Helper::CreateSubchannel(const grpc_channel_args& args) {
- if (parent_->shutting_down_) return nullptr;
+ if (parent_->shutting_down_ ||
+ (!CalledByPendingChild() && !CalledByCurrentChild())) {
+ return nullptr;
+ }
return parent_->channel_control_helper()->CreateSubchannel(args);
}
grpc_channel* XdsLb::Helper::CreateChannel(const char* target,
const grpc_channel_args& args) {
- if (parent_->shutting_down_) return nullptr;
+ if (parent_->shutting_down_ ||
+ (!CalledByPendingChild() && !CalledByCurrentChild())) {
+ return nullptr;
+ }
return parent_->channel_control_helper()->CreateChannel(target, args);
}
@@ -390,6 +417,26 @@ void XdsLb::Helper::UpdateState(grpc_connectivity_state state,
GRPC_ERROR_UNREF(state_error);
return;
}
+ // If this request is from the pending child policy, ignore it until
+ // it reports READY, at which point we swap it into place.
+ if (CalledByPendingChild()) {
+ if (grpc_lb_xds_trace.enabled()) {
+ gpr_log(GPR_INFO,
+ "[xdslb %p helper %p] pending child policy %p reports state=%s",
+ parent_.get(), this, parent_->pending_child_policy_.get(),
+ grpc_connectivity_state_name(state));
+ }
+ if (state != GRPC_CHANNEL_READY) {
+ GRPC_ERROR_UNREF(state_error);
+ return;
+ }
+ MutexLock lock(&parent_->child_policy_mu_);
+ parent_->child_policy_ = std::move(parent_->pending_child_policy_);
+ } else if (!CalledByCurrentChild()) {
+ // This request is from an outdated child, so ignore it.
+ GRPC_ERROR_UNREF(state_error);
+ return;
+ }
// TODO(juanlishen): When in fallback mode, pass the child picker
// through without wrapping it. (Or maybe use a different helper for
// the fallback policy?)
@@ -406,6 +453,11 @@ void XdsLb::Helper::UpdateState(grpc_connectivity_state state,
void XdsLb::Helper::RequestReresolution() {
if (parent_->shutting_down_) return;
+ // If there is a pending child policy, ignore re-resolution requests
+ // from the current child policy (or any outdated child).
+ if (parent_->pending_child_policy_ != nullptr && !CalledByPendingChild()) {
+ return;
+ }
if (grpc_lb_xds_trace.enabled()) {
gpr_log(GPR_INFO,
"[xdslb %p] Re-resolution requested from the internal RR policy "
@@ -1064,6 +1116,7 @@ grpc_channel_args* BuildBalancerChannelArgs(const grpc_channel_args* args) {
XdsLb::XdsLb(Args args) : LoadBalancingPolicy(std::move(args)) {
gpr_mu_init(&lb_chand_mu_);
+ gpr_mu_init(&child_policy_mu_);
// Record server name.
const grpc_arg* arg = grpc_channel_args_find(args.args, GRPC_ARG_SERVER_URI);
const char* server_uri = grpc_channel_arg_get_string(arg);
@@ -1093,6 +1146,7 @@ XdsLb::~XdsLb() {
if (serverlist_ != nullptr) {
xds_grpclb_destroy_serverlist(serverlist_);
}
+ gpr_mu_destroy(&child_policy_mu_);
}
void XdsLb::ShutdownLocked() {
@@ -1100,7 +1154,11 @@ void XdsLb::ShutdownLocked() {
if (fallback_timer_callback_pending_) {
grpc_timer_cancel(&lb_fallback_timer_);
}
- child_policy_.reset();
+ {
+ MutexLock lock(&child_policy_mu_);
+ child_policy_.reset();
+ pending_child_policy_.reset();
+ }
// We destroy the LB channel here instead of in our destructor because
// destroying the channel triggers a last callback to
// OnBalancerChannelConnectivityChangedLocked(), and we need to be
@@ -1126,12 +1184,27 @@ void XdsLb::ResetBackoffLocked() {
if (child_policy_ != nullptr) {
child_policy_->ResetBackoffLocked();
}
+ if (pending_child_policy_ != nullptr) {
+ pending_child_policy_->ResetBackoffLocked();
+ }
}
void XdsLb::FillChildRefsForChannelz(channelz::ChildRefsList* child_subchannels,
channelz::ChildRefsList* child_channels) {
- // Delegate to the child_policy_ to fill the children subchannels.
- child_policy_->FillChildRefsForChannelz(child_subchannels, child_channels);
+ {
+ // Delegate to the child_policy_ to fill the children subchannels.
+ // This must be done holding child_policy_mu_, since this method does not
+ // run in the combiner.
+ MutexLock lock(&child_policy_mu_);
+ if (child_policy_ != nullptr) {
+ child_policy_->FillChildRefsForChannelz(child_subchannels,
+ child_channels);
+ }
+ if (pending_child_policy_ != nullptr) {
+ pending_child_policy_->FillChildRefsForChannelz(child_subchannels,
+ child_channels);
+ }
+ }
MutexLock lock(&lb_chand_mu_);
if (lb_chand_ != nullptr) {
grpc_core::channelz::ChannelNode* channel_node =
@@ -1312,48 +1385,136 @@ grpc_channel_args* XdsLb::CreateChildPolicyArgsLocked() {
GPR_ARRAY_SIZE(args_to_add));
}
-void XdsLb::CreateChildPolicyLocked(const char* name, Args args) {
- GPR_ASSERT(child_policy_ == nullptr);
- child_policy_ = LoadBalancingPolicyRegistry::CreateLoadBalancingPolicy(
- name, std::move(args));
- if (GPR_UNLIKELY(child_policy_ == nullptr)) {
- gpr_log(GPR_ERROR, "[xdslb %p] Failure creating a child policy", this);
- return;
+OrphanablePtr<LoadBalancingPolicy> XdsLb::CreateChildPolicyLocked(
+ const char* name, const grpc_channel_args* args) {
+ Helper* helper = New<Helper>(Ref());
+ LoadBalancingPolicy::Args lb_policy_args;
+ lb_policy_args.combiner = combiner();
+ lb_policy_args.args = args;
+ lb_policy_args.channel_control_helper =
+ UniquePtr<ChannelControlHelper>(helper);
+ OrphanablePtr<LoadBalancingPolicy> lb_policy =
+ LoadBalancingPolicyRegistry::CreateLoadBalancingPolicy(
+ name, std::move(lb_policy_args));
+ if (GPR_UNLIKELY(lb_policy == nullptr)) {
+ gpr_log(GPR_ERROR, "[xdslb %p] Failure creating child policy %s", this,
+ name);
+ return nullptr;
+ }
+ helper->set_child(lb_policy.get());
+ if (grpc_lb_xds_trace.enabled()) {
+ gpr_log(GPR_INFO, "[xdslb %p] Created new child policy %s (%p)", this, name,
+ lb_policy.get());
}
// Add the xDS's interested_parties pollset_set to that of the newly created
- // child policy. This will make the child policy progress upon activity on
- // xDS LB, which in turn is tied to the application's call.
- grpc_pollset_set_add_pollset_set(child_policy_->interested_parties(),
+ // child policy. This will make the child policy progress upon activity on xDS
+ // LB, which in turn is tied to the application's call.
+ grpc_pollset_set_add_pollset_set(lb_policy->interested_parties(),
interested_parties());
+ return lb_policy;
}
void XdsLb::CreateOrUpdateChildPolicyLocked() {
if (shutting_down_) return;
grpc_channel_args* args = CreateChildPolicyArgsLocked();
GPR_ASSERT(args != nullptr);
+ // If the child policy name changes, we need to create a new child
+ // policy. When this happens, we leave child_policy_ as-is and store
+ // the new child policy in pending_child_policy_. Once the new child
+ // policy transitions into state READY, we swap it into child_policy_,
+ // replacing the original child policy. So pending_child_policy_ is
+ // non-null only between when we apply an update that changes the child
+ // policy name and when the new child reports state READY.
+ //
+ // Updates can arrive at any point during this transition. We always
+ // apply updates relative to the most recently created child policy,
+ // even if the most recent one is still in pending_child_policy_. This
+ // is true both when applying the updates to an existing child policy
+ // and when determining whether we need to create a new policy.
+ //
+ // As a result of this, there are several cases to consider here:
+ //
+ // 1. We have no existing child policy (i.e., we have started up but
+ // have not yet received a serverlist from the balancer or gone
+ // into fallback mode; in this case, both child_policy_ and
+ // pending_child_policy_ are null). In this case, we create a
+ // new child policy and store it in child_policy_.
+ //
+ // 2. We have an existing child policy and have no pending child policy
+ // from a previous update (i.e., either there has not been a
+ // previous update that changed the policy name, or we have already
+ // finished swapping in the new policy; in this case, child_policy_
+ // is non-null but pending_child_policy_ is null). In this case:
+ // a. If child_policy_->name() equals child_policy_name, then we
+ // update the existing child policy.
+ // b. If child_policy_->name() does not equal child_policy_name,
+ // we create a new policy. The policy will be stored in
+ // pending_child_policy_ and will later be swapped into
+ // child_policy_ by the helper when the new child transitions
+ // into state READY.
+ //
+ // 3. We have an existing child policy and have a pending child policy
+ // from a previous update (i.e., a previous update set
+ // pending_child_policy_ as per case 2b above and that policy has
+ // not yet transitioned into state READY and been swapped into
+ // child_policy_; in this case, both child_policy_ and
+ // pending_child_policy_ are non-null). In this case:
+ // a. If pending_child_policy_->name() equals child_policy_name,
+ // then we update the existing pending child policy.
+ // b. If pending_child_policy->name() does not equal
+ // child_policy_name, then we create a new policy. The new
+ // policy is stored in pending_child_policy_ (replacing the one
+ // that was there before, which will be immediately shut down)
+ // and will later be swapped into child_policy_ by the helper
+ // when the new child transitions into state READY.
// TODO(juanlishen): If the child policy is not configured via service config,
// use whatever algorithm is specified by the balancer.
- // TODO(juanlishen): Switch policy according to child_policy_config_->name().
- if (child_policy_ == nullptr) {
- LoadBalancingPolicy::Args lb_policy_args;
- lb_policy_args.combiner = combiner();
- lb_policy_args.args = args;
- lb_policy_args.channel_control_helper =
- UniquePtr<ChannelControlHelper>(New<Helper>(Ref()));
- CreateChildPolicyLocked(child_policy_config_ == nullptr
- ? "round_robin"
- : child_policy_config_->name(),
- std::move(lb_policy_args));
+ const char* child_policy_name = child_policy_config_ == nullptr
+ ? "round_robin"
+ : child_policy_config_->name();
+ const bool create_policy =
+ // case 1
+ child_policy_ == nullptr ||
+ // case 2b
+ (pending_child_policy_ == nullptr &&
+ strcmp(child_policy_->name(), child_policy_name) != 0) ||
+ // case 3b
+ (pending_child_policy_ != nullptr &&
+ strcmp(pending_child_policy_->name(), child_policy_name) != 0);
+ LoadBalancingPolicy* policy_to_update = nullptr;
+ if (create_policy) {
+ // Cases 1, 2b, and 3b: create a new child policy.
+ // If child_policy_ is null, we set it (case 1), else we set
+ // pending_child_policy_ (cases 2b and 3b).
if (grpc_lb_xds_trace.enabled()) {
- gpr_log(GPR_INFO, "[xdslb %p] Created a new child policy %p", this,
- child_policy_.get());
+ gpr_log(GPR_INFO, "[xdslb %p] Creating new %schild policy %s", this,
+ child_policy_ == nullptr ? "" : "pending ", child_policy_name);
+ }
+ auto new_policy = CreateChildPolicyLocked(child_policy_name, args);
+ auto& lb_policy =
+ child_policy_ == nullptr ? child_policy_ : pending_child_policy_;
+ {
+ MutexLock lock(&child_policy_mu_);
+ lb_policy = std::move(new_policy);
}
+ policy_to_update = lb_policy.get();
+ } else {
+ // Cases 2a and 3a: update an existing policy.
+ // If we have a pending child policy, send the update to the pending
+ // policy (case 3a), else send it to the current policy (case 2a).
+ policy_to_update = pending_child_policy_ != nullptr
+ ? pending_child_policy_.get()
+ : child_policy_.get();
}
+ GPR_ASSERT(policy_to_update != nullptr);
+ // Update the policy.
if (grpc_lb_xds_trace.enabled()) {
- gpr_log(GPR_INFO, "[xdslb %p] Updating child policy %p", this,
- child_policy_.get());
+ gpr_log(GPR_INFO, "[xdslb %p] Updating %schild policy %p", this,
+ policy_to_update == pending_child_policy_.get() ? "pending " : "",
+ policy_to_update);
}
- child_policy_->UpdateLocked(*args, child_policy_config_);
+ policy_to_update->UpdateLocked(*args, child_policy_config_);
+ // Clean up.
grpc_channel_args_destroy(args);
}
diff --git a/src/core/ext/filters/client_channel/resolving_lb_policy.cc b/src/core/ext/filters/client_channel/resolving_lb_policy.cc
index a02a7e8acdb..0dd51e8bc4c 100644
--- a/src/core/ext/filters/client_channel/resolving_lb_policy.cc
+++ b/src/core/ext/filters/client_channel/resolving_lb_policy.cc
@@ -47,6 +47,7 @@
#include "src/core/lib/gpr/string.h"
#include "src/core/lib/gprpp/inlined_vector.h"
#include "src/core/lib/gprpp/manual_constructor.h"
+#include "src/core/lib/gprpp/mutex_lock.h"
#include "src/core/lib/iomgr/combiner.h"
#include "src/core/lib/iomgr/iomgr.h"
#include "src/core/lib/iomgr/polling_entity.h"
@@ -77,12 +78,14 @@ class ResolvingLoadBalancingPolicy::ResolvingControlHelper
Subchannel* CreateSubchannel(const grpc_channel_args& args) override {
if (parent_->resolver_ == nullptr) return nullptr; // Shutting down.
+ if (!CalledByCurrentChild() && !CalledByPendingChild()) return nullptr;
return parent_->channel_control_helper()->CreateSubchannel(args);
}
grpc_channel* CreateChannel(const char* target,
const grpc_channel_args& args) override {
if (parent_->resolver_ == nullptr) return nullptr; // Shutting down.
+ if (!CalledByCurrentChild() && !CalledByPendingChild()) return nullptr;
return parent_->channel_control_helper()->CreateChannel(target, args);
}
@@ -93,11 +96,37 @@ class ResolvingLoadBalancingPolicy::ResolvingControlHelper
GRPC_ERROR_UNREF(state_error);
return;
}
+ // If this request is from the pending child policy, ignore it until
+ // it reports READY, at which point we swap it into place.
+ if (CalledByPendingChild()) {
+ if (parent_->tracer_->enabled()) {
+ gpr_log(GPR_INFO,
+ "resolving_lb=%p helper=%p: pending child policy %p reports "
+ "state=%s",
+ parent_.get(), this, child_,
+ grpc_connectivity_state_name(state));
+ }
+ if (state != GRPC_CHANNEL_READY) {
+ GRPC_ERROR_UNREF(state_error);
+ return;
+ }
+ MutexLock lock(&parent_->lb_policy_mu_);
+ parent_->lb_policy_ = std::move(parent_->pending_lb_policy_);
+ } else if (!CalledByCurrentChild()) {
+ // This request is from an outdated child, so ignore it.
+ GRPC_ERROR_UNREF(state_error);
+ return;
+ }
parent_->channel_control_helper()->UpdateState(state, state_error,
std::move(picker));
}
void RequestReresolution() override {
+ // If there is a pending child policy, ignore re-resolution requests
+ // from the current child policy (or any outdated child).
+ if (parent_->pending_lb_policy_ != nullptr && !CalledByPendingChild()) {
+ return;
+ }
if (parent_->tracer_->enabled()) {
gpr_log(GPR_INFO, "resolving_lb=%p: started name re-resolving",
parent_.get());
@@ -107,8 +136,21 @@ class ResolvingLoadBalancingPolicy::ResolvingControlHelper
}
}
+ void set_child(LoadBalancingPolicy* child) { child_ = child; }
+
private:
+ bool CalledByPendingChild() const {
+ GPR_ASSERT(child_ != nullptr);
+ return child_ == parent_->pending_lb_policy_.get();
+ }
+
+ bool CalledByCurrentChild() const {
+ GPR_ASSERT(child_ != nullptr);
+ return child_ == parent_->lb_policy_.get();
+ };
+
RefCountedPtr<ResolvingLoadBalancingPolicy> parent_;
+ LoadBalancingPolicy* child_ = nullptr;
};
//
@@ -146,6 +188,7 @@ ResolvingLoadBalancingPolicy::ResolvingLoadBalancingPolicy(
process_resolver_result_(process_resolver_result),
process_resolver_result_user_data_(process_resolver_result_user_data) {
GPR_ASSERT(process_resolver_result != nullptr);
+ gpr_mu_init(&lb_policy_mu_);
*error = Init(*args.args);
}
@@ -169,22 +212,38 @@ grpc_error* ResolvingLoadBalancingPolicy::Init(const grpc_channel_args& args) {
ResolvingLoadBalancingPolicy::~ResolvingLoadBalancingPolicy() {
GPR_ASSERT(resolver_ == nullptr);
GPR_ASSERT(lb_policy_ == nullptr);
+ gpr_mu_destroy(&lb_policy_mu_);
}
void ResolvingLoadBalancingPolicy::ShutdownLocked() {
if (resolver_ != nullptr) {
resolver_.reset();
+ MutexLock lock(&lb_policy_mu_);
if (lb_policy_ != nullptr) {
+ if (tracer_->enabled()) {
+ gpr_log(GPR_INFO, "resolving_lb=%p: shutting down lb_policy=%p", this,
+ lb_policy_.get());
+ }
grpc_pollset_set_del_pollset_set(lb_policy_->interested_parties(),
interested_parties());
lb_policy_.reset();
}
+ if (pending_lb_policy_ != nullptr) {
+ if (tracer_->enabled()) {
+ gpr_log(GPR_INFO, "resolving_lb=%p: shutting down pending lb_policy=%p",
+ this, pending_lb_policy_.get());
+ }
+ grpc_pollset_set_del_pollset_set(pending_lb_policy_->interested_parties(),
+ interested_parties());
+ pending_lb_policy_.reset();
+ }
}
}
void ResolvingLoadBalancingPolicy::ExitIdleLocked() {
if (lb_policy_ != nullptr) {
lb_policy_->ExitIdleLocked();
+ if (pending_lb_policy_ != nullptr) pending_lb_policy_->ExitIdleLocked();
} else {
if (!started_resolving_ && resolver_ != nullptr) {
StartResolvingLocked();
@@ -197,17 +256,24 @@ void ResolvingLoadBalancingPolicy::ResetBackoffLocked() {
resolver_->ResetBackoffLocked();
resolver_->RequestReresolutionLocked();
}
- if (lb_policy_ != nullptr) {
- lb_policy_->ResetBackoffLocked();
- }
+ if (lb_policy_ != nullptr) lb_policy_->ResetBackoffLocked();
+ if (pending_lb_policy_ != nullptr) pending_lb_policy_->ResetBackoffLocked();
}
void ResolvingLoadBalancingPolicy::FillChildRefsForChannelz(
channelz::ChildRefsList* child_subchannels,
channelz::ChildRefsList* child_channels) {
+ // Delegate to the lb_policy_ to fill the children subchannels.
+ // This must be done holding lb_policy_mu_, since this method does not
+ // run in the combiner.
+ MutexLock lock(&lb_policy_mu_);
if (lb_policy_ != nullptr) {
lb_policy_->FillChildRefsForChannelz(child_subchannels, child_channels);
}
+ if (pending_lb_policy_ != nullptr) {
+ pending_lb_policy_->FillChildRefsForChannelz(child_subchannels,
+ child_channels);
+ }
}
void ResolvingLoadBalancingPolicy::StartResolvingLocked() {
@@ -229,14 +295,26 @@ void ResolvingLoadBalancingPolicy::OnResolverShutdownLocked(grpc_error* error) {
if (tracer_->enabled()) {
gpr_log(GPR_INFO, "resolving_lb=%p: shutting down", this);
}
- if (lb_policy_ != nullptr) {
- if (tracer_->enabled()) {
- gpr_log(GPR_INFO, "resolving_lb=%p: shutting down lb_policy=%p", this,
- lb_policy_.get());
+ {
+ MutexLock lock(&lb_policy_mu_);
+ if (lb_policy_ != nullptr) {
+ if (tracer_->enabled()) {
+ gpr_log(GPR_INFO, "resolving_lb=%p: shutting down lb_policy=%p", this,
+ lb_policy_.get());
+ }
+ grpc_pollset_set_del_pollset_set(lb_policy_->interested_parties(),
+ interested_parties());
+ lb_policy_.reset();
+ }
+ if (pending_lb_policy_ != nullptr) {
+ if (tracer_->enabled()) {
+ gpr_log(GPR_INFO, "resolving_lb=%p: shutting down pending lb_policy=%p",
+ this, pending_lb_policy_.get());
+ }
+ grpc_pollset_set_del_pollset_set(pending_lb_policy_->interested_parties(),
+ interested_parties());
+ pending_lb_policy_.reset();
}
- grpc_pollset_set_del_pollset_set(lb_policy_->interested_parties(),
- interested_parties());
- lb_policy_.reset();
}
if (resolver_ != nullptr) {
// This should never happen; it can only be triggered by a resolver
@@ -260,53 +338,142 @@ void ResolvingLoadBalancingPolicy::OnResolverShutdownLocked(grpc_error* error) {
Unref();
}
-// Creates a new LB policy, replacing any previous one.
+void ResolvingLoadBalancingPolicy::CreateOrUpdateLbPolicyLocked(
+ const char* lb_policy_name, RefCountedPtr<Config> lb_policy_config,
+ TraceStringVector* trace_strings) {
+ // If the child policy name changes, we need to create a new child
+ // policy. When this happens, we leave child_policy_ as-is and store
+ // the new child policy in pending_child_policy_. Once the new child
+ // policy transitions into state READY, we swap it into child_policy_,
+ // replacing the original child policy. So pending_child_policy_ is
+ // non-null only between when we apply an update that changes the child
+ // policy name and when the new child reports state READY.
+ //
+ // Updates can arrive at any point during this transition. We always
+ // apply updates relative to the most recently created child policy,
+ // even if the most recent one is still in pending_child_policy_. This
+ // is true both when applying the updates to an existing child policy
+ // and when determining whether we need to create a new policy.
+ //
+ // As a result of this, there are several cases to consider here:
+ //
+ // 1. We have no existing child policy (i.e., we have started up but
+ // have not yet received a serverlist from the balancer or gone
+ // into fallback mode; in this case, both child_policy_ and
+ // pending_child_policy_ are null). In this case, we create a
+ // new child policy and store it in child_policy_.
+ //
+ // 2. We have an existing child policy and have no pending child policy
+ // from a previous update (i.e., either there has not been a
+ // previous update that changed the policy name, or we have already
+ // finished swapping in the new policy; in this case, child_policy_
+ // is non-null but pending_child_policy_ is null). In this case:
+ // a. If child_policy_->name() equals child_policy_name, then we
+ // update the existing child policy.
+ // b. If child_policy_->name() does not equal child_policy_name,
+ // we create a new policy. The policy will be stored in
+ // pending_child_policy_ and will later be swapped into
+ // child_policy_ by the helper when the new child transitions
+ // into state READY.
+ //
+ // 3. We have an existing child policy and have a pending child policy
+ // from a previous update (i.e., a previous update set
+ // pending_child_policy_ as per case 2b above and that policy has
+ // not yet transitioned into state READY and been swapped into
+ // child_policy_; in this case, both child_policy_ and
+ // pending_child_policy_ are non-null). In this case:
+ // a. If pending_child_policy_->name() equals child_policy_name,
+ // then we update the existing pending child policy.
+ // b. If pending_child_policy->name() does not equal
+ // child_policy_name, then we create a new policy. The new
+ // policy is stored in pending_child_policy_ (replacing the one
+ // that was there before, which will be immediately shut down)
+ // and will later be swapped into child_policy_ by the helper
+ // when the new child transitions into state READY.
+ const bool create_policy =
+ // case 1
+ lb_policy_ == nullptr ||
+ // case 2b
+ (pending_lb_policy_ == nullptr &&
+ strcmp(lb_policy_->name(), lb_policy_name) != 0) ||
+ // case 3b
+ (pending_lb_policy_ != nullptr &&
+ strcmp(pending_lb_policy_->name(), lb_policy_name) != 0);
+ LoadBalancingPolicy* policy_to_update = nullptr;
+ if (create_policy) {
+ // Cases 1, 2b, and 3b: create a new child policy.
+ // If lb_policy_ is null, we set it (case 1), else we set
+ // pending_lb_policy_ (cases 2b and 3b).
+ if (tracer_->enabled()) {
+ gpr_log(GPR_INFO, "resolving_lb=%p: Creating new %schild policy %s", this,
+ lb_policy_ == nullptr ? "" : "pending ", lb_policy_name);
+ }
+ auto new_policy = CreateLbPolicyLocked(lb_policy_name, trace_strings);
+ auto& lb_policy = lb_policy_ == nullptr ? lb_policy_ : pending_lb_policy_;
+ {
+ MutexLock lock(&lb_policy_mu_);
+ lb_policy = std::move(new_policy);
+ }
+ policy_to_update = lb_policy.get();
+ } else {
+ // Cases 2a and 3a: update an existing policy.
+ // If we have a pending child policy, send the update to the pending
+ // policy (case 3a), else send it to the current policy (case 2a).
+ policy_to_update = pending_lb_policy_ != nullptr ? pending_lb_policy_.get()
+ : lb_policy_.get();
+ }
+ GPR_ASSERT(policy_to_update != nullptr);
+ // Update the policy.
+ if (tracer_->enabled()) {
+ gpr_log(GPR_INFO, "resolving_lb=%p: Updating %schild policy %p", this,
+ policy_to_update == pending_lb_policy_.get() ? "pending " : "",
+ policy_to_update);
+ }
+ policy_to_update->UpdateLocked(*resolver_result_,
+ std::move(lb_policy_config));
+}
+
+// Creates a new LB policy.
// Updates trace_strings to indicate what was done.
-void ResolvingLoadBalancingPolicy::CreateNewLbPolicyLocked(
+OrphanablePtr<LoadBalancingPolicy>
+ResolvingLoadBalancingPolicy::CreateLbPolicyLocked(
const char* lb_policy_name, TraceStringVector* trace_strings) {
+ ResolvingControlHelper* helper = New<ResolvingControlHelper>(Ref());
LoadBalancingPolicy::Args lb_policy_args;
lb_policy_args.combiner = combiner();
lb_policy_args.channel_control_helper =
- UniquePtr<ChannelControlHelper>(New<ResolvingControlHelper>(Ref()));
+ UniquePtr<ChannelControlHelper>(helper);
lb_policy_args.args = resolver_result_;
- OrphanablePtr<LoadBalancingPolicy> new_lb_policy =
+ OrphanablePtr<LoadBalancingPolicy> lb_policy =
LoadBalancingPolicyRegistry::CreateLoadBalancingPolicy(
lb_policy_name, std::move(lb_policy_args));
- if (GPR_UNLIKELY(new_lb_policy == nullptr)) {
+ if (GPR_UNLIKELY(lb_policy == nullptr)) {
gpr_log(GPR_ERROR, "could not create LB policy \"%s\"", lb_policy_name);
if (channelz_node() != nullptr) {
char* str;
gpr_asprintf(&str, "Could not create LB policy \"%s\"", lb_policy_name);
trace_strings->push_back(str);
}
- } else {
- if (tracer_->enabled()) {
- gpr_log(GPR_INFO, "resolving_lb=%p: created new LB policy \"%s\" (%p)",
- this, lb_policy_name, new_lb_policy.get());
- }
- if (channelz_node() != nullptr) {
- char* str;
- gpr_asprintf(&str, "Created new LB policy \"%s\"", lb_policy_name);
- trace_strings->push_back(str);
- }
- // Propagate channelz node.
- auto* channelz = channelz_node();
- if (channelz != nullptr) {
- new_lb_policy->set_channelz_node(channelz->Ref());
- }
- // Swap out the LB policy and update the fds in interested_parties_.
- if (lb_policy_ != nullptr) {
- if (tracer_->enabled()) {
- gpr_log(GPR_INFO, "resolving_lb=%p: shutting down lb_policy=%p", this,
- lb_policy_.get());
- }
- grpc_pollset_set_del_pollset_set(lb_policy_->interested_parties(),
- interested_parties());
- }
- lb_policy_ = std::move(new_lb_policy);
- grpc_pollset_set_add_pollset_set(lb_policy_->interested_parties(),
- interested_parties());
+ return nullptr;
+ }
+ helper->set_child(lb_policy.get());
+ if (tracer_->enabled()) {
+ gpr_log(GPR_INFO, "resolving_lb=%p: created new LB policy \"%s\" (%p)",
+ this, lb_policy_name, lb_policy.get());
+ }
+ if (channelz_node() != nullptr) {
+ char* str;
+ gpr_asprintf(&str, "Created new LB policy \"%s\"", lb_policy_name);
+ trace_strings->push_back(str);
+ }
+ // Propagate channelz node.
+ auto* channelz = channelz_node();
+ if (channelz != nullptr) {
+ lb_policy->set_channelz_node(channelz->Ref());
}
+ grpc_pollset_set_add_pollset_set(lb_policy->interested_parties(),
+ interested_parties());
+ return lb_policy;
}
void ResolvingLoadBalancingPolicy::MaybeAddTraceMessagesForAddressChangesLocked(
@@ -415,23 +582,8 @@ void ResolvingLoadBalancingPolicy::OnResolverResultChangedLocked(
lb_policy_config = self->child_lb_config_;
}
GPR_ASSERT(lb_policy_name != nullptr);
- // If we're not already using the right LB policy name, instantiate
- // a new one.
- if (self->lb_policy_ == nullptr ||
- strcmp(self->lb_policy_->name(), lb_policy_name) != 0) {
- if (self->tracer_->enabled()) {
- gpr_log(GPR_INFO, "resolving_lb=%p: creating new LB policy \"%s\"",
- self, lb_policy_name);
- }
- self->CreateNewLbPolicyLocked(lb_policy_name, &trace_strings);
- }
- // Update the LB policy with the new addresses and config.
- if (self->tracer_->enabled()) {
- gpr_log(GPR_INFO, "resolving_lb=%p: updating LB policy \"%s\" (%p)", self,
- lb_policy_name, self->lb_policy_.get());
- }
- self->lb_policy_->UpdateLocked(*self->resolver_result_,
- std::move(lb_policy_config));
+ self->CreateOrUpdateLbPolicyLocked(
+ lb_policy_name, std::move(lb_policy_config), &trace_strings);
// Add channel trace event.
if (self->channelz_node() != nullptr) {
if (service_config_changed) {
diff --git a/src/core/ext/filters/client_channel/resolving_lb_policy.h b/src/core/ext/filters/client_channel/resolving_lb_policy.h
index d068a41f96f..b8f406da1b6 100644
--- a/src/core/ext/filters/client_channel/resolving_lb_policy.h
+++ b/src/core/ext/filters/client_channel/resolving_lb_policy.h
@@ -102,8 +102,11 @@ class ResolvingLoadBalancingPolicy : public LoadBalancingPolicy {
void StartResolvingLocked();
void OnResolverShutdownLocked(grpc_error* error);
- void CreateNewLbPolicyLocked(const char* lb_policy_name,
- TraceStringVector* trace_strings);
+ void CreateOrUpdateLbPolicyLocked(const char* lb_policy_name,
+ RefCountedPtr<Config>,
+ TraceStringVector* trace_strings);
+ OrphanablePtr<LoadBalancingPolicy> CreateLbPolicyLocked(
+ const char* lb_policy_name, TraceStringVector* trace_strings);
void MaybeAddTraceMessagesForAddressChangesLocked(
TraceStringVector* trace_strings);
void ConcatenateAndAddChannelTraceLocked(
@@ -125,8 +128,12 @@ class ResolvingLoadBalancingPolicy : public LoadBalancingPolicy {
bool previous_resolution_contained_addresses_ = false;
grpc_closure on_resolver_result_changed_;
- // Child LB policy and associated state.
+ // Child LB policy.
OrphanablePtr<LoadBalancingPolicy> lb_policy_;
+ OrphanablePtr<LoadBalancingPolicy> pending_lb_policy_;
+ // Lock held when modifying the value of child_policy_ or
+ // pending_child_policy_.
+ gpr_mu lb_policy_mu_;
};
} // namespace grpc_core
From 8878712aebf67d366112b1f80ace2a3015d73b45 Mon Sep 17 00:00:00 2001
From: "Mark D. Roth" <roth@google.com>
Date: Thu, 7 Mar 2019 13:02:04 -0800
Subject: [PATCH 057/138] Split client_channel tracer into two.
---
doc/environment_variables.md | 5 +-
.../filters/client_channel/client_channel.cc | 116 +++++++++---------
2 files changed, 63 insertions(+), 58 deletions(-)
diff --git a/doc/environment_variables.md b/doc/environment_variables.md
index 435edbcfdb4..635c5ee535f 100644
--- a/doc/environment_variables.md
+++ b/doc/environment_variables.md
@@ -45,8 +45,9 @@ some configuration as environment variables that can be set.
- cares_address_sorting - traces operations of the c-ares based DNS
resolver's resolved address sorter
- channel - traces operations on the C core channel stack
- - client_channel - traces client channel activity, including resolver
- and load balancing policy interaction
+ - client_channel_call - traces client channel call batch activity
+ - client_channel_routing - traces client channel call routing, including
+ resolver and load balancing policy interaction
- compression - traces compression operations
- connectivity_state - traces connectivity state changes to channels
- executor - traces grpc's internal thread pool ('the executor')
diff --git a/src/core/ext/filters/client_channel/client_channel.cc b/src/core/ext/filters/client_channel/client_channel.cc
index 3f87438b13b..ad00855be71 100644
--- a/src/core/ext/filters/client_channel/client_channel.cc
+++ b/src/core/ext/filters/client_channel/client_channel.cc
@@ -82,7 +82,10 @@ using grpc_core::LoadBalancingPolicy;
// any even moderately compelling reason to do so.
#define RETRY_BACKOFF_JITTER 0.2
-grpc_core::TraceFlag grpc_client_channel_trace(false, "client_channel");
+grpc_core::TraceFlag grpc_client_channel_call_trace(false,
+ "client_channel_call");
+grpc_core::TraceFlag grpc_client_channel_routing_trace(
+ false, "client_channel_routing");
/*************************************************************************
* CHANNEL-WIDE FUNCTIONS
@@ -219,7 +222,7 @@ class ClientChannelControlHelper
void UpdateState(
grpc_connectivity_state state, grpc_error* state_error,
UniquePtr<LoadBalancingPolicy::SubchannelPicker> picker) override {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_routing_trace.enabled()) {
const char* extra = chand_->disconnect_error == GRPC_ERROR_NONE
? ""
: " (ignoring -- channel shutting down)";
@@ -256,7 +259,7 @@ static bool process_resolver_result_locked(
ProcessedResolverResult resolver_result(args, chand->enable_retries);
grpc_core::UniquePtr<char> service_config_json =
resolver_result.service_config_json();
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_routing_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p: resolver returned service config: \"%s\"",
chand, service_config_json.get());
}
@@ -460,8 +463,9 @@ static grpc_error* cc_init_channel_elem(grpc_channel_element* elem,
grpc_error* error = GRPC_ERROR_NONE;
chand->resolving_lb_policy.reset(
grpc_core::New<grpc_core::ResolvingLoadBalancingPolicy>(
- std::move(lb_args), &grpc_client_channel_trace, std::move(target_uri),
- process_resolver_result_locked, chand, &error));
+ std::move(lb_args), &grpc_client_channel_routing_trace,
+ std::move(target_uri), process_resolver_result_locked, chand,
+ &error));
grpc_channel_args_destroy(new_args);
if (error != GRPC_ERROR_NONE) {
// Orphan the resolving LB policy and flush the exec_ctx to ensure
@@ -480,7 +484,7 @@ static grpc_error* cc_init_channel_elem(grpc_channel_element* elem,
grpc_pollset_set_add_pollset_set(
chand->resolving_lb_policy->interested_parties(),
chand->interested_parties);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_routing_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p: created resolving_lb_policy=%p", chand,
chand->resolving_lb_policy.get());
}
@@ -856,7 +860,7 @@ static void maybe_cache_send_ops_for_batch(call_data* calld,
// Frees cached send_initial_metadata.
static void free_cached_send_initial_metadata(channel_data* chand,
call_data* calld) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: destroying calld->send_initial_metadata", chand,
calld);
@@ -867,7 +871,7 @@ static void free_cached_send_initial_metadata(channel_data* chand,
// Frees cached send_message at index idx.
static void free_cached_send_message(channel_data* chand, call_data* calld,
size_t idx) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: destroying calld->send_messages[%" PRIuPTR "]",
chand, calld, idx);
@@ -878,7 +882,7 @@ static void free_cached_send_message(channel_data* chand, call_data* calld,
// Frees cached send_trailing_metadata.
static void free_cached_send_trailing_metadata(channel_data* chand,
call_data* calld) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: destroying calld->send_trailing_metadata",
chand, calld);
@@ -964,7 +968,7 @@ static void pending_batches_add(grpc_call_element* elem,
channel_data* chand = static_cast<channel_data*>(elem->channel_data);
call_data* calld = static_cast<call_data*>(elem->call_data);
const size_t idx = get_batch_index(batch);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: adding pending batch at index %" PRIuPTR, chand,
calld, idx);
@@ -993,7 +997,7 @@ static void pending_batches_add(grpc_call_element* elem,
}
if (GPR_UNLIKELY(calld->bytes_buffered_for_retry >
chand->per_rpc_retry_buffer_size)) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: exceeded retry buffer size, committing",
chand, calld);
@@ -1008,7 +1012,7 @@ static void pending_batches_add(grpc_call_element* elem,
// If we are not going to retry and have not yet started, pretend
// retries are disabled so that we don't bother with retry overhead.
if (calld->num_attempts_completed == 0) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: disabling retries before first attempt",
chand, calld);
@@ -1066,7 +1070,7 @@ static void pending_batches_fail(
YieldCallCombinerPredicate yield_call_combiner_predicate) {
GPR_ASSERT(error != GRPC_ERROR_NONE);
call_data* calld = static_cast<call_data*>(elem->call_data);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
size_t num_batches = 0;
for (size_t i = 0; i < GPR_ARRAY_SIZE(calld->pending_batches); ++i) {
if (calld->pending_batches[i].batch != nullptr) ++num_batches;
@@ -1121,7 +1125,7 @@ static void pending_batches_resume(grpc_call_element* elem) {
return;
}
// Retries not enabled; send down batches as-is.
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
size_t num_batches = 0;
for (size_t i = 0; i < GPR_ARRAY_SIZE(calld->pending_batches); ++i) {
if (calld->pending_batches[i].batch != nullptr) ++num_batches;
@@ -1169,7 +1173,7 @@ static void maybe_clear_pending_batch(grpc_call_element* elem,
(!batch->recv_trailing_metadata ||
batch->payload->recv_trailing_metadata.recv_trailing_metadata_ready ==
nullptr)) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: clearing pending batch", chand,
calld);
}
@@ -1189,7 +1193,7 @@ static pending_batch* pending_batch_find(grpc_call_element* elem,
pending_batch* pending = &calld->pending_batches[i];
grpc_transport_stream_op_batch* batch = pending->batch;
if (batch != nullptr && predicate(batch)) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: %s pending batch at index %" PRIuPTR, chand,
calld, log_message, i);
@@ -1211,7 +1215,7 @@ static void retry_commit(grpc_call_element* elem,
call_data* calld = static_cast<call_data*>(elem->call_data);
if (calld->retry_committed) return;
calld->retry_committed = true;
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: committing retries", chand, calld);
}
if (retry_state != nullptr) {
@@ -1250,7 +1254,7 @@ static void do_retry(grpc_call_element* elem,
}
next_attempt_time = calld->retry_backoff->NextAttemptTime();
}
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: retrying failed call in %" PRId64 " ms", chand,
calld, next_attempt_time - grpc_core::ExecCtx::Get()->Now());
@@ -1283,7 +1287,7 @@ static bool maybe_retry(grpc_call_element* elem,
retry_state = static_cast<subchannel_call_retry_state*>(
batch_data->subchannel_call->GetParentData());
if (retry_state->retry_dispatched) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: retry already dispatched", chand,
calld);
}
@@ -1295,14 +1299,14 @@ static bool maybe_retry(grpc_call_element* elem,
if (calld->retry_throttle_data != nullptr) {
calld->retry_throttle_data->RecordSuccess();
}
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: call succeeded", chand, calld);
}
return false;
}
// Status is not OK. Check whether the status is retryable.
if (!retry_policy->retryable_status_codes.Contains(status)) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: status %s not configured as retryable", chand,
calld, grpc_status_code_to_string(status));
@@ -1318,14 +1322,14 @@ static bool maybe_retry(grpc_call_element* elem,
// checks, so that we don't fail to record failures due to other factors.
if (calld->retry_throttle_data != nullptr &&
!calld->retry_throttle_data->RecordFailure()) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: retries throttled", chand, calld);
}
return false;
}
// Check whether the call is committed.
if (calld->retry_committed) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: retries already committed", chand,
calld);
}
@@ -1334,7 +1338,7 @@ static bool maybe_retry(grpc_call_element* elem,
// Check whether we have retries remaining.
++calld->num_attempts_completed;
if (calld->num_attempts_completed >= retry_policy->max_attempts) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: exceeded %d retry attempts", chand,
calld, retry_policy->max_attempts);
}
@@ -1342,7 +1346,7 @@ static bool maybe_retry(grpc_call_element* elem,
}
// If the call was cancelled from the surface, don't retry.
if (calld->cancel_error != GRPC_ERROR_NONE) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: call cancelled from surface, not retrying",
chand, calld);
@@ -1355,14 +1359,14 @@ static bool maybe_retry(grpc_call_element* elem,
// If the value is "-1" or any other unparseable string, we do not retry.
uint32_t ms;
if (!grpc_parse_slice_to_uint32(GRPC_MDVALUE(*server_pushback_md), &ms)) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: not retrying due to server push-back",
chand, calld);
}
return false;
} else {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: server push-back: retry in %u ms",
chand, calld, ms);
}
@@ -1484,7 +1488,7 @@ static void recv_initial_metadata_ready(void* arg, grpc_error* error) {
grpc_call_element* elem = batch_data->elem;
channel_data* chand = static_cast<channel_data*>(elem->channel_data);
call_data* calld = static_cast<call_data*>(elem->call_data);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: got recv_initial_metadata_ready, error=%s",
chand, calld, grpc_error_string(error));
@@ -1508,7 +1512,7 @@ static void recv_initial_metadata_ready(void* arg, grpc_error* error) {
if (GPR_UNLIKELY((retry_state->trailing_metadata_available ||
error != GRPC_ERROR_NONE) &&
!retry_state->completed_recv_trailing_metadata)) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: deferring recv_initial_metadata_ready "
"(Trailers-Only)",
@@ -1574,7 +1578,7 @@ static void recv_message_ready(void* arg, grpc_error* error) {
grpc_call_element* elem = batch_data->elem;
channel_data* chand = static_cast<channel_data*>(elem->channel_data);
call_data* calld = static_cast<call_data*>(elem->call_data);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: got recv_message_ready, error=%s",
chand, calld, grpc_error_string(error));
}
@@ -1596,7 +1600,7 @@ static void recv_message_ready(void* arg, grpc_error* error) {
if (GPR_UNLIKELY(
(retry_state->recv_message == nullptr || error != GRPC_ERROR_NONE) &&
!retry_state->completed_recv_trailing_metadata)) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: deferring recv_message_ready (nullptr "
"message and recv_trailing_metadata pending)",
@@ -1748,7 +1752,7 @@ static void add_closures_to_fail_unstarted_pending_batches(
for (size_t i = 0; i < GPR_ARRAY_SIZE(calld->pending_batches); ++i) {
pending_batch* pending = &calld->pending_batches[i];
if (pending_batch_is_unstarted(pending, calld, retry_state)) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: failing unstarted pending batch at index "
"%" PRIuPTR,
@@ -1797,7 +1801,7 @@ static void recv_trailing_metadata_ready(void* arg, grpc_error* error) {
grpc_call_element* elem = batch_data->elem;
channel_data* chand = static_cast<channel_data*>(elem->channel_data);
call_data* calld = static_cast<call_data*>(elem->call_data);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: got recv_trailing_metadata_ready, error=%s",
chand, calld, grpc_error_string(error));
@@ -1813,7 +1817,7 @@ static void recv_trailing_metadata_ready(void* arg, grpc_error* error) {
batch_data->batch.payload->recv_trailing_metadata.recv_trailing_metadata;
get_call_status(elem, md_batch, GRPC_ERROR_REF(error), &status,
&server_pushback_md);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: call finished, status=%s", chand,
calld, grpc_status_code_to_string(status));
}
@@ -1899,7 +1903,7 @@ static void add_closures_for_replay_or_pending_send_ops(
}
}
if (have_pending_send_message_ops || have_pending_send_trailing_metadata_op) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: starting next batch for pending send op(s)",
chand, calld);
@@ -1919,7 +1923,7 @@ static void on_complete(void* arg, grpc_error* error) {
grpc_call_element* elem = batch_data->elem;
channel_data* chand = static_cast<channel_data*>(elem->channel_data);
call_data* calld = static_cast<call_data*>(elem->call_data);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
char* batch_str = grpc_transport_stream_op_batch_string(&batch_data->batch);
gpr_log(GPR_INFO, "chand=%p calld=%p: got on_complete, error=%s, batch=%s",
chand, calld, grpc_error_string(error), batch_str);
@@ -1999,7 +2003,7 @@ static void add_closure_for_subchannel_batch(
GRPC_CLOSURE_INIT(&batch->handler_private.closure,
start_batch_in_call_combiner, batch,
grpc_schedule_on_exec_ctx);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
char* batch_str = grpc_transport_stream_op_batch_string(batch);
gpr_log(GPR_INFO, "chand=%p calld=%p: starting subchannel batch: %s", chand,
calld, batch_str);
@@ -2067,7 +2071,7 @@ static void add_retriable_send_message_op(
subchannel_batch_data* batch_data) {
channel_data* chand = static_cast<channel_data*>(elem->channel_data);
call_data* calld = static_cast<call_data*>(elem->call_data);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: starting calld->send_messages[%" PRIuPTR "]",
chand, calld, retry_state->started_send_message_count);
@@ -2161,7 +2165,7 @@ static void add_retriable_recv_trailing_metadata_op(
static void start_internal_recv_trailing_metadata(grpc_call_element* elem) {
channel_data* chand = static_cast<channel_data*>(elem->channel_data);
call_data* calld = static_cast<call_data*>(elem->call_data);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: call failed but recv_trailing_metadata not "
"started; starting it internally",
@@ -2194,7 +2198,7 @@ static subchannel_batch_data* maybe_create_subchannel_batch_for_replay(
if (calld->seen_send_initial_metadata &&
!retry_state->started_send_initial_metadata &&
!calld->pending_send_initial_metadata) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: replaying previously completed "
"send_initial_metadata op",
@@ -2210,7 +2214,7 @@ static subchannel_batch_data* maybe_create_subchannel_batch_for_replay(
retry_state->started_send_message_count ==
retry_state->completed_send_message_count &&
!calld->pending_send_message) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: replaying previously completed "
"send_message op",
@@ -2230,7 +2234,7 @@ static subchannel_batch_data* maybe_create_subchannel_batch_for_replay(
retry_state->started_send_message_count == calld->send_messages.size() &&
!retry_state->started_send_trailing_metadata &&
!calld->pending_send_trailing_metadata) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: replaying previously completed "
"send_trailing_metadata op",
@@ -2380,7 +2384,7 @@ static void start_retriable_subchannel_batches(void* arg, grpc_error* ignored) {
grpc_call_element* elem = static_cast<grpc_call_element*>(arg);
channel_data* chand = static_cast<channel_data*>(elem->channel_data);
call_data* calld = static_cast<call_data*>(elem->call_data);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: constructing retriable batches",
chand, calld);
}
@@ -2405,7 +2409,7 @@ static void start_retriable_subchannel_batches(void* arg, grpc_error* ignored) {
// Now add pending batches.
add_subchannel_batches_for_pending_batches(elem, retry_state, &closures);
// Start batches on subchannel call.
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: starting %" PRIuPTR
" retriable batches on subchannel_call=%p",
@@ -2439,7 +2443,7 @@ static void create_subchannel_call(grpc_call_element* elem) {
grpc_error* error = GRPC_ERROR_NONE;
calld->subchannel_call =
calld->pick.pick.connected_subchannel->CreateCall(call_args, &error);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_routing_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: create subchannel_call=%p: error=%s",
chand, calld, calld->subchannel_call.get(),
grpc_error_string(error));
@@ -2461,7 +2465,7 @@ static void pick_done(void* arg, grpc_error* error) {
channel_data* chand = static_cast<channel_data*>(elem->channel_data);
call_data* calld = static_cast<call_data*>(elem->call_data);
if (error != GRPC_ERROR_NONE) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_routing_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: failed to pick subchannel: error=%s", chand,
calld, grpc_error_string(error));
@@ -2493,7 +2497,7 @@ class QueuedPickCanceller {
auto* self = static_cast<QueuedPickCanceller*>(arg);
auto* chand = static_cast<channel_data*>(self->elem_->channel_data);
auto* calld = static_cast<call_data*>(self->elem_->call_data);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_routing_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: cancelling queued pick: "
"error=%s self=%p calld->pick_canceller=%p",
@@ -2525,7 +2529,7 @@ static void remove_call_from_queued_picks_locked(grpc_call_element* elem) {
for (QueuedPick** pick = &chand->queued_picks; *pick != nullptr;
pick = &(*pick)->next) {
if (*pick == &calld->pick) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_routing_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: removing from queued picks list",
chand, calld);
}
@@ -2545,7 +2549,7 @@ static void remove_call_from_queued_picks_locked(grpc_call_element* elem) {
static void add_call_to_queued_picks_locked(grpc_call_element* elem) {
auto* chand = static_cast<channel_data*>(elem->channel_data);
auto* calld = static_cast<call_data*>(elem->call_data);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_routing_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: adding to queued picks list", chand,
calld);
}
@@ -2567,7 +2571,7 @@ static void add_call_to_queued_picks_locked(grpc_call_element* elem) {
static void apply_service_config_to_call_locked(grpc_call_element* elem) {
channel_data* chand = static_cast<channel_data*>(elem->channel_data);
call_data* calld = static_cast<call_data*>(elem->call_data);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_routing_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: applying service config to call",
chand, calld);
}
@@ -2679,7 +2683,7 @@ static void start_pick_locked(void* arg, grpc_error* error) {
// Attempt pick.
error = GRPC_ERROR_NONE;
auto pick_result = chand->picker->Pick(&calld->pick.pick, &error);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_routing_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: LB pick returned %s (connected_subchannel=%p, "
"error=%s)",
@@ -2748,7 +2752,7 @@ static void cc_start_transport_stream_op_batch(
}
// If we've previously been cancelled, immediately fail any new batches.
if (GPR_UNLIKELY(calld->cancel_error != GRPC_ERROR_NONE)) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: failing batch with error: %s",
chand, calld, grpc_error_string(calld->cancel_error));
}
@@ -2767,7 +2771,7 @@ static void cc_start_transport_stream_op_batch(
GRPC_ERROR_UNREF(calld->cancel_error);
calld->cancel_error =
GRPC_ERROR_REF(batch->payload->cancel_stream.cancel_error);
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: recording cancel_error=%s", chand,
calld, grpc_error_string(calld->cancel_error));
}
@@ -2795,7 +2799,7 @@ static void cc_start_transport_stream_op_batch(
// the channel combiner, which is more efficient (especially for
// streaming calls).
if (calld->subchannel_call != nullptr) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: starting batch on subchannel_call=%p", chand,
calld, calld->subchannel_call.get());
@@ -2807,7 +2811,7 @@ static void cc_start_transport_stream_op_batch(
// For batches containing a send_initial_metadata op, enter the channel
// combiner to start a pick.
if (GPR_LIKELY(batch->send_initial_metadata)) {
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO, "chand=%p calld=%p: entering client_channel combiner",
chand, calld);
}
@@ -2817,7 +2821,7 @@ static void cc_start_transport_stream_op_batch(
GRPC_ERROR_NONE);
} else {
// For all other batches, release the call combiner.
- if (grpc_client_channel_trace.enabled()) {
+ if (grpc_client_channel_call_trace.enabled()) {
gpr_log(GPR_INFO,
"chand=%p calld=%p: saved batch, yielding call combiner", chand,
calld);
From bea84d54850f362727cba50916e64a097ceba1c9 Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Thu, 7 Mar 2019 14:12:14 -0800
Subject: [PATCH 058/138] Add the missing grpc_cfstream dependency
---
BUILD | 1 +
1 file changed, 1 insertion(+)
diff --git a/BUILD b/BUILD
index 24c1fb31ced..7d26a8f5c42 100644
--- a/BUILD
+++ b/BUILD
@@ -307,6 +307,7 @@ grpc_cc_library(
public_hdrs = GRPC_PUBLIC_HDRS + GRPC_SECURE_PUBLIC_HDRS,
standalone = True,
deps = [
+ "grpc_cfstream",
"grpc_common",
"grpc_lb_policy_grpclb_secure",
"grpc_lb_policy_xds_secure",
From abcd5861ebd4867516aea2e12567bb8f423c18d5 Mon Sep 17 00:00:00 2001
From: Yash Tibrewal <yashkt@google.com>
Date: Thu, 7 Mar 2019 14:49:24 -0800
Subject: [PATCH 059/138] Nuking the poll-cv polling engine
---
BUILD | 2 -
CMakeLists.txt | 45 --
Makefile | 42 --
build.yaml | 21 -
config.m4 | 1 -
config.w32 | 1 -
doc/core/grpc-polling-engines.md | 6 +-
gRPC-C++.podspec | 2 -
gRPC-Core.podspec | 3 -
grpc.gemspec | 2 -
grpc.gyp | 4 -
package.xml | 2 -
src/core/lib/iomgr/ev_poll_posix.cc | 484 ------------------
src/core/lib/iomgr/ev_posix.cc | 5 +-
src/core/lib/iomgr/wakeup_fd_cv.cc | 107 ----
src/core/lib/iomgr/wakeup_fd_cv.h | 69 ---
src/core/lib/iomgr/wakeup_fd_posix.cc | 20 +-
src/python/grpcio/grpc_core_dependencies.py | 1 -
test/core/end2end/generate_tests.bzl | 2 +-
test/core/end2end/tests/keepalive_timeout.cc | 3 +-
test/core/iomgr/BUILD | 11 -
test/core/iomgr/wakeup_fd_cv_test.cc | 243 ---------
test/core/util/test_config.cc | 7 -
tools/doxygen/Doxyfile.c++.internal | 1 -
tools/doxygen/Doxyfile.core.internal | 2 -
.../generated/sources_and_headers.json | 19 -
tools/run_tests/generated/tests.json | 96 +---
.../run_tests/performance/scenario_config.py | 6 +-
tools/run_tests/run_tests.py | 2 +-
29 files changed, 34 insertions(+), 1175 deletions(-)
delete mode 100644 src/core/lib/iomgr/wakeup_fd_cv.cc
delete mode 100644 src/core/lib/iomgr/wakeup_fd_cv.h
delete mode 100644 test/core/iomgr/wakeup_fd_cv_test.cc
diff --git a/BUILD b/BUILD
index 24c1fb31ced..53afe09f449 100644
--- a/BUILD
+++ b/BUILD
@@ -800,7 +800,6 @@ grpc_cc_library(
"src/core/lib/iomgr/udp_server.cc",
"src/core/lib/iomgr/unix_sockets_posix.cc",
"src/core/lib/iomgr/unix_sockets_posix_noop.cc",
- "src/core/lib/iomgr/wakeup_fd_cv.cc",
"src/core/lib/iomgr/wakeup_fd_eventfd.cc",
"src/core/lib/iomgr/wakeup_fd_nospecial.cc",
"src/core/lib/iomgr/wakeup_fd_pipe.cc",
@@ -941,7 +940,6 @@ grpc_cc_library(
"src/core/lib/iomgr/timer_manager.h",
"src/core/lib/iomgr/udp_server.h",
"src/core/lib/iomgr/unix_sockets_posix.h",
- "src/core/lib/iomgr/wakeup_fd_cv.h",
"src/core/lib/iomgr/wakeup_fd_pipe.h",
"src/core/lib/iomgr/wakeup_fd_posix.h",
"src/core/lib/json/json.h",
diff --git a/CMakeLists.txt b/CMakeLists.txt
index bccead24f28..7ccda85b125 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -437,9 +437,6 @@ if(_gRPC_PLATFORM_LINUX OR _gRPC_PLATFORM_MAC OR _gRPC_PLATFORM_POSIX)
add_dependencies(buildtests_c udp_server_test)
endif()
add_dependencies(buildtests_c uri_parser_test)
-if(_gRPC_PLATFORM_LINUX OR _gRPC_PLATFORM_MAC OR _gRPC_PLATFORM_POSIX)
-add_dependencies(buildtests_c wakeup_fd_cv_test)
-endif()
add_dependencies(buildtests_c public_headers_must_be_c89)
add_dependencies(buildtests_c badreq_bad_client_test)
add_dependencies(buildtests_c connection_prefix_bad_client_test)
@@ -1072,7 +1069,6 @@ add_library(grpc
src/core/lib/iomgr/udp_server.cc
src/core/lib/iomgr/unix_sockets_posix.cc
src/core/lib/iomgr/unix_sockets_posix_noop.cc
- src/core/lib/iomgr/wakeup_fd_cv.cc
src/core/lib/iomgr/wakeup_fd_eventfd.cc
src/core/lib/iomgr/wakeup_fd_nospecial.cc
src/core/lib/iomgr/wakeup_fd_pipe.cc
@@ -1497,7 +1493,6 @@ add_library(grpc_cronet
src/core/lib/iomgr/udp_server.cc
src/core/lib/iomgr/unix_sockets_posix.cc
src/core/lib/iomgr/unix_sockets_posix_noop.cc
- src/core/lib/iomgr/wakeup_fd_cv.cc
src/core/lib/iomgr/wakeup_fd_eventfd.cc
src/core/lib/iomgr/wakeup_fd_nospecial.cc
src/core/lib/iomgr/wakeup_fd_pipe.cc
@@ -1907,7 +1902,6 @@ add_library(grpc_test_util
src/core/lib/iomgr/udp_server.cc
src/core/lib/iomgr/unix_sockets_posix.cc
src/core/lib/iomgr/unix_sockets_posix_noop.cc
- src/core/lib/iomgr/wakeup_fd_cv.cc
src/core/lib/iomgr/wakeup_fd_eventfd.cc
src/core/lib/iomgr/wakeup_fd_nospecial.cc
src/core/lib/iomgr/wakeup_fd_pipe.cc
@@ -2232,7 +2226,6 @@ add_library(grpc_test_util_unsecure
src/core/lib/iomgr/udp_server.cc
src/core/lib/iomgr/unix_sockets_posix.cc
src/core/lib/iomgr/unix_sockets_posix_noop.cc
- src/core/lib/iomgr/wakeup_fd_cv.cc
src/core/lib/iomgr/wakeup_fd_eventfd.cc
src/core/lib/iomgr/wakeup_fd_nospecial.cc
src/core/lib/iomgr/wakeup_fd_pipe.cc
@@ -2533,7 +2526,6 @@ add_library(grpc_unsecure
src/core/lib/iomgr/udp_server.cc
src/core/lib/iomgr/unix_sockets_posix.cc
src/core/lib/iomgr/unix_sockets_posix_noop.cc
- src/core/lib/iomgr/wakeup_fd_cv.cc
src/core/lib/iomgr/wakeup_fd_eventfd.cc
src/core/lib/iomgr/wakeup_fd_nospecial.cc
src/core/lib/iomgr/wakeup_fd_pipe.cc
@@ -3420,7 +3412,6 @@ add_library(grpc++_cronet
src/core/lib/iomgr/udp_server.cc
src/core/lib/iomgr/unix_sockets_posix.cc
src/core/lib/iomgr/unix_sockets_posix_noop.cc
- src/core/lib/iomgr/wakeup_fd_cv.cc
src/core/lib/iomgr/wakeup_fd_eventfd.cc
src/core/lib/iomgr/wakeup_fd_nospecial.cc
src/core/lib/iomgr/wakeup_fd_pipe.cc
@@ -10453,42 +10444,6 @@ target_link_libraries(uri_parser_test
endif (gRPC_BUILD_TESTS)
if (gRPC_BUILD_TESTS)
-if(_gRPC_PLATFORM_LINUX OR _gRPC_PLATFORM_MAC OR _gRPC_PLATFORM_POSIX)
-
-add_executable(wakeup_fd_cv_test
- test/core/iomgr/wakeup_fd_cv_test.cc
-)
-
-
-target_include_directories(wakeup_fd_cv_test
- PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}
- PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/include
- PRIVATE ${_gRPC_SSL_INCLUDE_DIR}
- PRIVATE ${_gRPC_PROTOBUF_INCLUDE_DIR}
- PRIVATE ${_gRPC_ZLIB_INCLUDE_DIR}
- PRIVATE ${_gRPC_BENCHMARK_INCLUDE_DIR}
- PRIVATE ${_gRPC_CARES_INCLUDE_DIR}
- PRIVATE ${_gRPC_GFLAGS_INCLUDE_DIR}
- PRIVATE ${_gRPC_ADDRESS_SORTING_INCLUDE_DIR}
- PRIVATE ${_gRPC_NANOPB_INCLUDE_DIR}
-)
-
-target_link_libraries(wakeup_fd_cv_test
- ${_gRPC_ALLTARGETS_LIBRARIES}
- grpc_test_util
- grpc
- gpr
-)
-
- # avoid dependency on libstdc++
- if (_gRPC_CORE_NOSTDCXX_FLAGS)
- set_target_properties(wakeup_fd_cv_test PROPERTIES LINKER_LANGUAGE C)
- target_compile_options(wakeup_fd_cv_test PRIVATE $<$<COMPILE_LANGUAGE:CXX>:${_gRPC_CORE_NOSTDCXX_FLAGS}>)
- endif()
-
-endif()
-endif (gRPC_BUILD_TESTS)
-if (gRPC_BUILD_TESTS)
add_executable(alarm_test
test/cpp/common/alarm_test.cc
diff --git a/Makefile b/Makefile
index 94944265d61..91516de9a27 100644
--- a/Makefile
+++ b/Makefile
@@ -1140,7 +1140,6 @@ transport_security_test: $(BINDIR)/$(CONFIG)/transport_security_test
udp_server_test: $(BINDIR)/$(CONFIG)/udp_server_test
uri_fuzzer_test: $(BINDIR)/$(CONFIG)/uri_fuzzer_test
uri_parser_test: $(BINDIR)/$(CONFIG)/uri_parser_test
-wakeup_fd_cv_test: $(BINDIR)/$(CONFIG)/wakeup_fd_cv_test
alarm_test: $(BINDIR)/$(CONFIG)/alarm_test
alts_counter_test: $(BINDIR)/$(CONFIG)/alts_counter_test
alts_crypt_test: $(BINDIR)/$(CONFIG)/alts_crypt_test
@@ -1593,7 +1592,6 @@ buildtests_c: privatelibs_c \
$(BINDIR)/$(CONFIG)/transport_security_test \
$(BINDIR)/$(CONFIG)/udp_server_test \
$(BINDIR)/$(CONFIG)/uri_parser_test \
- $(BINDIR)/$(CONFIG)/wakeup_fd_cv_test \
$(BINDIR)/$(CONFIG)/public_headers_must_be_c89 \
$(BINDIR)/$(CONFIG)/badreq_bad_client_test \
$(BINDIR)/$(CONFIG)/connection_prefix_bad_client_test \
@@ -2243,8 +2241,6 @@ test_c: buildtests_c
$(Q) $(BINDIR)/$(CONFIG)/udp_server_test || ( echo test udp_server_test failed ; exit 1 )
$(E) "[RUN] Testing uri_parser_test"
$(Q) $(BINDIR)/$(CONFIG)/uri_parser_test || ( echo test uri_parser_test failed ; exit 1 )
- $(E) "[RUN] Testing wakeup_fd_cv_test"
- $(Q) $(BINDIR)/$(CONFIG)/wakeup_fd_cv_test || ( echo test wakeup_fd_cv_test failed ; exit 1 )
$(E) "[RUN] Testing public_headers_must_be_c89"
$(Q) $(BINDIR)/$(CONFIG)/public_headers_must_be_c89 || ( echo test public_headers_must_be_c89 failed ; exit 1 )
$(E) "[RUN] Testing badreq_bad_client_test"
@@ -3617,7 +3613,6 @@ LIBGRPC_SRC = \
src/core/lib/iomgr/udp_server.cc \
src/core/lib/iomgr/unix_sockets_posix.cc \
src/core/lib/iomgr/unix_sockets_posix_noop.cc \
- src/core/lib/iomgr/wakeup_fd_cv.cc \
src/core/lib/iomgr/wakeup_fd_eventfd.cc \
src/core/lib/iomgr/wakeup_fd_nospecial.cc \
src/core/lib/iomgr/wakeup_fd_pipe.cc \
@@ -4036,7 +4031,6 @@ LIBGRPC_CRONET_SRC = \
src/core/lib/iomgr/udp_server.cc \
src/core/lib/iomgr/unix_sockets_posix.cc \
src/core/lib/iomgr/unix_sockets_posix_noop.cc \
- src/core/lib/iomgr/wakeup_fd_cv.cc \
src/core/lib/iomgr/wakeup_fd_eventfd.cc \
src/core/lib/iomgr/wakeup_fd_nospecial.cc \
src/core/lib/iomgr/wakeup_fd_pipe.cc \
@@ -4439,7 +4433,6 @@ LIBGRPC_TEST_UTIL_SRC = \
src/core/lib/iomgr/udp_server.cc \
src/core/lib/iomgr/unix_sockets_posix.cc \
src/core/lib/iomgr/unix_sockets_posix_noop.cc \
- src/core/lib/iomgr/wakeup_fd_cv.cc \
src/core/lib/iomgr/wakeup_fd_eventfd.cc \
src/core/lib/iomgr/wakeup_fd_nospecial.cc \
src/core/lib/iomgr/wakeup_fd_pipe.cc \
@@ -4751,7 +4744,6 @@ LIBGRPC_TEST_UTIL_UNSECURE_SRC = \
src/core/lib/iomgr/udp_server.cc \
src/core/lib/iomgr/unix_sockets_posix.cc \
src/core/lib/iomgr/unix_sockets_posix_noop.cc \
- src/core/lib/iomgr/wakeup_fd_cv.cc \
src/core/lib/iomgr/wakeup_fd_eventfd.cc \
src/core/lib/iomgr/wakeup_fd_nospecial.cc \
src/core/lib/iomgr/wakeup_fd_pipe.cc \
@@ -5026,7 +5018,6 @@ LIBGRPC_UNSECURE_SRC = \
src/core/lib/iomgr/udp_server.cc \
src/core/lib/iomgr/unix_sockets_posix.cc \
src/core/lib/iomgr/unix_sockets_posix_noop.cc \
- src/core/lib/iomgr/wakeup_fd_cv.cc \
src/core/lib/iomgr/wakeup_fd_eventfd.cc \
src/core/lib/iomgr/wakeup_fd_nospecial.cc \
src/core/lib/iomgr/wakeup_fd_pipe.cc \
@@ -5890,7 +5881,6 @@ LIBGRPC++_CRONET_SRC = \
src/core/lib/iomgr/udp_server.cc \
src/core/lib/iomgr/unix_sockets_posix.cc \
src/core/lib/iomgr/unix_sockets_posix_noop.cc \
- src/core/lib/iomgr/wakeup_fd_cv.cc \
src/core/lib/iomgr/wakeup_fd_eventfd.cc \
src/core/lib/iomgr/wakeup_fd_nospecial.cc \
src/core/lib/iomgr/wakeup_fd_pipe.cc \
@@ -15405,38 +15395,6 @@ endif
endif
-WAKEUP_FD_CV_TEST_SRC = \
- test/core/iomgr/wakeup_fd_cv_test.cc \
-
-WAKEUP_FD_CV_TEST_OBJS = $(addprefix $(OBJDIR)/$(CONFIG)/, $(addsuffix .o, $(basename $(WAKEUP_FD_CV_TEST_SRC))))
-ifeq ($(NO_SECURE),true)
-
-# You can't build secure targets if you don't have OpenSSL.
-
-$(BINDIR)/$(CONFIG)/wakeup_fd_cv_test: openssl_dep_error
-
-else
-
-
-
-$(BINDIR)/$(CONFIG)/wakeup_fd_cv_test: $(WAKEUP_FD_CV_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr.a
- $(E) "[LD] Linking $@"
- $(Q) mkdir -p `dirname $@`
- $(Q) $(LD) $(LDFLAGS) $(WAKEUP_FD_CV_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr.a $(LDLIBS) $(LDLIBS_SECURE) -o $(BINDIR)/$(CONFIG)/wakeup_fd_cv_test
-
-endif
-
-$(OBJDIR)/$(CONFIG)/test/core/iomgr/wakeup_fd_cv_test.o: $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr.a
-
-deps_wakeup_fd_cv_test: $(WAKEUP_FD_CV_TEST_OBJS:.o=.dep)
-
-ifneq ($(NO_SECURE),true)
-ifneq ($(NO_DEPS),true)
--include $(WAKEUP_FD_CV_TEST_OBJS:.o=.dep)
-endif
-endif
-
-
ALARM_TEST_SRC = \
test/cpp/common/alarm_test.cc \
diff --git a/build.yaml b/build.yaml
index 621b9a4de2f..02ecaa221e6 100644
--- a/build.yaml
+++ b/build.yaml
@@ -333,7 +333,6 @@ filegroups:
- src/core/lib/iomgr/udp_server.cc
- src/core/lib/iomgr/unix_sockets_posix.cc
- src/core/lib/iomgr/unix_sockets_posix_noop.cc
- - src/core/lib/iomgr/wakeup_fd_cv.cc
- src/core/lib/iomgr/wakeup_fd_eventfd.cc
- src/core/lib/iomgr/wakeup_fd_nospecial.cc
- src/core/lib/iomgr/wakeup_fd_pipe.cc
@@ -498,7 +497,6 @@ filegroups:
- src/core/lib/iomgr/timer_manager.h
- src/core/lib/iomgr/udp_server.h
- src/core/lib/iomgr/unix_sockets_posix.h
- - src/core/lib/iomgr/wakeup_fd_cv.h
- src/core/lib/iomgr/wakeup_fd_pipe.h
- src/core/lib/iomgr/wakeup_fd_posix.h
- src/core/lib/json/json.h
@@ -3740,21 +3738,6 @@ targets:
- grpc_test_util
- grpc
- gpr
-- name: wakeup_fd_cv_test
- build: test
- language: c
- src:
- - test/core/iomgr/wakeup_fd_cv_test.cc
- deps:
- - grpc_test_util
- - grpc
- - gpr
- exclude_iomgrs:
- - uv
- platforms:
- - mac
- - linux
- - posix
- name: alarm_test
gtest: true
build: test
@@ -4180,7 +4163,6 @@ targets:
defaults: benchmark
excluded_poll_engines:
- poll
- - poll-cv
platforms:
- mac
- linux
@@ -4206,7 +4188,6 @@ targets:
defaults: benchmark
excluded_poll_engines:
- poll
- - poll-cv
platforms:
- mac
- linux
@@ -4232,7 +4213,6 @@ targets:
- tsan
excluded_poll_engines:
- poll
- - poll-cv
platforms:
- mac
- linux
@@ -4258,7 +4238,6 @@ targets:
defaults: benchmark
excluded_poll_engines:
- poll
- - poll-cv
platforms:
- mac
- linux
diff --git a/config.m4 b/config.m4
index 2616803d9b0..bb23c2ed956 100644
--- a/config.m4
+++ b/config.m4
@@ -187,7 +187,6 @@ if test "$PHP_GRPC" != "no"; then
src/core/lib/iomgr/udp_server.cc \
src/core/lib/iomgr/unix_sockets_posix.cc \
src/core/lib/iomgr/unix_sockets_posix_noop.cc \
- src/core/lib/iomgr/wakeup_fd_cv.cc \
src/core/lib/iomgr/wakeup_fd_eventfd.cc \
src/core/lib/iomgr/wakeup_fd_nospecial.cc \
src/core/lib/iomgr/wakeup_fd_pipe.cc \
diff --git a/config.w32 b/config.w32
index 64eca2a8472..35e52e15bd0 100644
--- a/config.w32
+++ b/config.w32
@@ -162,7 +162,6 @@ if (PHP_GRPC != "no") {
"src\\core\\lib\\iomgr\\udp_server.cc " +
"src\\core\\lib\\iomgr\\unix_sockets_posix.cc " +
"src\\core\\lib\\iomgr\\unix_sockets_posix_noop.cc " +
- "src\\core\\lib\\iomgr\\wakeup_fd_cv.cc " +
"src\\core\\lib\\iomgr\\wakeup_fd_eventfd.cc " +
"src\\core\\lib\\iomgr\\wakeup_fd_nospecial.cc " +
"src\\core\\lib\\iomgr\\wakeup_fd_pipe.cc " +
diff --git a/doc/core/grpc-polling-engines.md b/doc/core/grpc-polling-engines.md
index f273913b1e4..dd5a7654852 100644
--- a/doc/core/grpc-polling-engines.md
+++ b/doc/core/grpc-polling-engines.md
@@ -23,11 +23,9 @@ There are multiple polling engine implementations depending on the OS and the OS
- **`epollex`** (default but requires kernel version >= 4.5),
- `epoll1` (If `epollex` is not available and glibc version >= 2.9)
- `poll` (If kernel does not have epoll support)
- - `poll-cv` (If explicitly configured)
-- Mac: **`poll`** (default), `poll-cv` (If explicitly configured)
+- Mac: **`poll`** (default)
- Windows: (no name)
- One-off polling engines:
- - AppEngine platform: **`poll-cv`** (default)
- NodeJS : `libuv` polling engine implementation (requires different compile `#define`s)
## Polling Engine Interface
@@ -87,7 +85,7 @@ Add/Remove fd to the `grpc_pollset_set`
- **grpc\_pollset\_set_[add|del]\_pollset**
- Signature: `grpc_pollset_set_[add|del]_pollset(grpc_pollset_set* pss, grpc_pollset* ps)`
- What does adding a pollset to a pollset_set mean ?
- - It means that calling `grpc_pollset_work()` on the pollset will also poll all the fds in the pollset_set i.e semantically, it is similar to adding all the fds inside pollset_set to the pollset.
+ - It means that calling `grpc_pollset_work()` on the pollset will also poll all the fds in the pollset_set i.e semantically, it is similar to adding all the fds inside pollset_set to the pollset.
- This guarantee is no longer true once the pollset is removed from the pollset_set
- **grpc\_pollset\_set_[add|del]\_pollset\_set**
diff --git a/gRPC-C++.podspec b/gRPC-C++.podspec
index 0f3888975c9..e755b7aa602 100644
--- a/gRPC-C++.podspec
+++ b/gRPC-C++.podspec
@@ -474,7 +474,6 @@ Pod::Spec.new do |s|
'src/core/lib/iomgr/timer_manager.h',
'src/core/lib/iomgr/udp_server.h',
'src/core/lib/iomgr/unix_sockets_posix.h',
- 'src/core/lib/iomgr/wakeup_fd_cv.h',
'src/core/lib/iomgr/wakeup_fd_pipe.h',
'src/core/lib/iomgr/wakeup_fd_posix.h',
'src/core/lib/json/json.h',
@@ -666,7 +665,6 @@ Pod::Spec.new do |s|
'src/core/lib/iomgr/timer_manager.h',
'src/core/lib/iomgr/udp_server.h',
'src/core/lib/iomgr/unix_sockets_posix.h',
- 'src/core/lib/iomgr/wakeup_fd_cv.h',
'src/core/lib/iomgr/wakeup_fd_pipe.h',
'src/core/lib/iomgr/wakeup_fd_posix.h',
'src/core/lib/json/json.h',
diff --git a/gRPC-Core.podspec b/gRPC-Core.podspec
index 2e54b9d7847..7068f039870 100644
--- a/gRPC-Core.podspec
+++ b/gRPC-Core.podspec
@@ -468,7 +468,6 @@ Pod::Spec.new do |s|
'src/core/lib/iomgr/timer_manager.h',
'src/core/lib/iomgr/udp_server.h',
'src/core/lib/iomgr/unix_sockets_posix.h',
- 'src/core/lib/iomgr/wakeup_fd_cv.h',
'src/core/lib/iomgr/wakeup_fd_pipe.h',
'src/core/lib/iomgr/wakeup_fd_posix.h',
'src/core/lib/json/json.h',
@@ -634,7 +633,6 @@ Pod::Spec.new do |s|
'src/core/lib/iomgr/udp_server.cc',
'src/core/lib/iomgr/unix_sockets_posix.cc',
'src/core/lib/iomgr/unix_sockets_posix_noop.cc',
- 'src/core/lib/iomgr/wakeup_fd_cv.cc',
'src/core/lib/iomgr/wakeup_fd_eventfd.cc',
'src/core/lib/iomgr/wakeup_fd_nospecial.cc',
'src/core/lib/iomgr/wakeup_fd_pipe.cc',
@@ -1096,7 +1094,6 @@ Pod::Spec.new do |s|
'src/core/lib/iomgr/timer_manager.h',
'src/core/lib/iomgr/udp_server.h',
'src/core/lib/iomgr/unix_sockets_posix.h',
- 'src/core/lib/iomgr/wakeup_fd_cv.h',
'src/core/lib/iomgr/wakeup_fd_pipe.h',
'src/core/lib/iomgr/wakeup_fd_posix.h',
'src/core/lib/json/json.h',
diff --git a/grpc.gemspec b/grpc.gemspec
index 95de105d7af..a2a027a20d8 100644
--- a/grpc.gemspec
+++ b/grpc.gemspec
@@ -402,7 +402,6 @@ Gem::Specification.new do |s|
s.files += %w( src/core/lib/iomgr/timer_manager.h )
s.files += %w( src/core/lib/iomgr/udp_server.h )
s.files += %w( src/core/lib/iomgr/unix_sockets_posix.h )
- s.files += %w( src/core/lib/iomgr/wakeup_fd_cv.h )
s.files += %w( src/core/lib/iomgr/wakeup_fd_pipe.h )
s.files += %w( src/core/lib/iomgr/wakeup_fd_posix.h )
s.files += %w( src/core/lib/json/json.h )
@@ -568,7 +567,6 @@ Gem::Specification.new do |s|
s.files += %w( src/core/lib/iomgr/udp_server.cc )
s.files += %w( src/core/lib/iomgr/unix_sockets_posix.cc )
s.files += %w( src/core/lib/iomgr/unix_sockets_posix_noop.cc )
- s.files += %w( src/core/lib/iomgr/wakeup_fd_cv.cc )
s.files += %w( src/core/lib/iomgr/wakeup_fd_eventfd.cc )
s.files += %w( src/core/lib/iomgr/wakeup_fd_nospecial.cc )
s.files += %w( src/core/lib/iomgr/wakeup_fd_pipe.cc )
diff --git a/grpc.gyp b/grpc.gyp
index 53e891b28dc..94a9cd2cfb8 100644
--- a/grpc.gyp
+++ b/grpc.gyp
@@ -369,7 +369,6 @@
'src/core/lib/iomgr/udp_server.cc',
'src/core/lib/iomgr/unix_sockets_posix.cc',
'src/core/lib/iomgr/unix_sockets_posix_noop.cc',
- 'src/core/lib/iomgr/wakeup_fd_cv.cc',
'src/core/lib/iomgr/wakeup_fd_eventfd.cc',
'src/core/lib/iomgr/wakeup_fd_nospecial.cc',
'src/core/lib/iomgr/wakeup_fd_pipe.cc',
@@ -735,7 +734,6 @@
'src/core/lib/iomgr/udp_server.cc',
'src/core/lib/iomgr/unix_sockets_posix.cc',
'src/core/lib/iomgr/unix_sockets_posix_noop.cc',
- 'src/core/lib/iomgr/wakeup_fd_cv.cc',
'src/core/lib/iomgr/wakeup_fd_eventfd.cc',
'src/core/lib/iomgr/wakeup_fd_nospecial.cc',
'src/core/lib/iomgr/wakeup_fd_pipe.cc',
@@ -980,7 +978,6 @@
'src/core/lib/iomgr/udp_server.cc',
'src/core/lib/iomgr/unix_sockets_posix.cc',
'src/core/lib/iomgr/unix_sockets_posix_noop.cc',
- 'src/core/lib/iomgr/wakeup_fd_cv.cc',
'src/core/lib/iomgr/wakeup_fd_eventfd.cc',
'src/core/lib/iomgr/wakeup_fd_nospecial.cc',
'src/core/lib/iomgr/wakeup_fd_pipe.cc',
@@ -1201,7 +1198,6 @@
'src/core/lib/iomgr/udp_server.cc',
'src/core/lib/iomgr/unix_sockets_posix.cc',
'src/core/lib/iomgr/unix_sockets_posix_noop.cc',
- 'src/core/lib/iomgr/wakeup_fd_cv.cc',
'src/core/lib/iomgr/wakeup_fd_eventfd.cc',
'src/core/lib/iomgr/wakeup_fd_nospecial.cc',
'src/core/lib/iomgr/wakeup_fd_pipe.cc',
diff --git a/package.xml b/package.xml
index 34730a0c1d5..be7e258b98a 100644
--- a/package.xml
+++ b/package.xml
@@ -407,7 +407,6 @@
<file baseinstalldir="/" name="src/core/lib/iomgr/timer_manager.h" role="src" />
<file baseinstalldir="/" name="src/core/lib/iomgr/udp_server.h" role="src" />
<file baseinstalldir="/" name="src/core/lib/iomgr/unix_sockets_posix.h" role="src" />
- <file baseinstalldir="/" name="src/core/lib/iomgr/wakeup_fd_cv.h" role="src" />
<file baseinstalldir="/" name="src/core/lib/iomgr/wakeup_fd_pipe.h" role="src" />
<file baseinstalldir="/" name="src/core/lib/iomgr/wakeup_fd_posix.h" role="src" />
<file baseinstalldir="/" name="src/core/lib/json/json.h" role="src" />
@@ -573,7 +572,6 @@
<file baseinstalldir="/" name="src/core/lib/iomgr/udp_server.cc" role="src" />
<file baseinstalldir="/" name="src/core/lib/iomgr/unix_sockets_posix.cc" role="src" />
<file baseinstalldir="/" name="src/core/lib/iomgr/unix_sockets_posix_noop.cc" role="src" />
- <file baseinstalldir="/" name="src/core/lib/iomgr/wakeup_fd_cv.cc" role="src" />
<file baseinstalldir="/" name="src/core/lib/iomgr/wakeup_fd_eventfd.cc" role="src" />
<file baseinstalldir="/" name="src/core/lib/iomgr/wakeup_fd_nospecial.cc" role="src" />
<file baseinstalldir="/" name="src/core/lib/iomgr/wakeup_fd_pipe.cc" role="src" />
diff --git a/src/core/lib/iomgr/ev_poll_posix.cc b/src/core/lib/iomgr/ev_poll_posix.cc
index 9350ef5a2af..4c98ffa9448 100644
--- a/src/core/lib/iomgr/ev_poll_posix.cc
+++ b/src/core/lib/iomgr/ev_poll_posix.cc
@@ -43,7 +43,6 @@
#include "src/core/lib/gprpp/thd.h"
#include "src/core/lib/iomgr/block_annotate.h"
#include "src/core/lib/iomgr/iomgr_internal.h"
-#include "src/core/lib/iomgr/wakeup_fd_cv.h"
#include "src/core/lib/iomgr/wakeup_fd_posix.h"
#include "src/core/lib/profiling/timers.h"
@@ -256,56 +255,6 @@ struct grpc_pollset_set {
grpc_fd** fds;
};
-/*******************************************************************************
- * condition variable polling definitions
- */
-
-#define POLLCV_THREAD_GRACE_MS 1000
-#define CV_POLL_PERIOD_MS 1000
-#define CV_DEFAULT_TABLE_SIZE 16
-
-typedef struct poll_result {
- gpr_refcount refcount;
- grpc_cv_node* watchers;
- int watchcount;
- struct pollfd* fds;
- nfds_t nfds;
- int retval;
- int err;
- int completed;
-} poll_result;
-
-typedef struct poll_args {
- grpc_core::Thread poller_thd;
- gpr_cv trigger;
- int trigger_set;
- bool harvestable;
- gpr_cv harvest;
- bool joinable;
- gpr_cv join;
- struct pollfd* fds;
- nfds_t nfds;
- poll_result* result;
- struct poll_args* next;
- struct poll_args* prev;
-} poll_args;
-
-// This is a 2-tiered cache, we mantain a hash table
-// of active poll calls, so we can wait on the result
-// of that call. We also maintain freelists of inactive
-// poll args and of dead poller threads.
-typedef struct poll_hash_table {
- poll_args* free_pollers;
- poll_args** active_pollers;
- poll_args* dead_pollers;
- unsigned int size;
- unsigned int count;
-} poll_hash_table;
-
-// TODO(kpayson64): Eliminate use of global non-POD variables
-poll_hash_table poll_cache;
-grpc_cv_fd_table g_cvfds;
-
/*******************************************************************************
* functions to track opened fds. No-ops unless track_fds_for_fork is true.
*/
@@ -1363,425 +1312,6 @@ static void pollset_set_del_fd(grpc_pollset_set* pollset_set, grpc_fd* fd) {
gpr_mu_unlock(&pollset_set->mu);
}
-/*******************************************************************************
- * Condition Variable polling extensions
- */
-
-static void run_poll(void* args);
-static void cache_poller_locked(poll_args* args);
-static void cache_harvest_locked();
-
-static void cache_insert_locked(poll_args* args) {
- uint32_t key = gpr_murmur_hash3(args->fds, args->nfds * sizeof(struct pollfd),
- 0xDEADBEEF);
- key = key % poll_cache.size;
- if (poll_cache.active_pollers[key]) {
- poll_cache.active_pollers[key]->prev = args;
- }
- args->next = poll_cache.active_pollers[key];
- args->prev = nullptr;
- poll_cache.active_pollers[key] = args;
- poll_cache.count++;
-}
-
-static void init_result(poll_args* pargs) {
- pargs->result = static_cast<poll_result*>(gpr_malloc(sizeof(poll_result)));
- gpr_ref_init(&pargs->result->refcount, 1);
- pargs->result->watchers = nullptr;
- pargs->result->watchcount = 0;
- pargs->result->fds = static_cast<struct pollfd*>(
- gpr_malloc(sizeof(struct pollfd) * pargs->nfds));
- memcpy(pargs->result->fds, pargs->fds, sizeof(struct pollfd) * pargs->nfds);
- pargs->result->nfds = pargs->nfds;
- pargs->result->retval = 0;
- pargs->result->err = 0;
- pargs->result->completed = 0;
-}
-
-// Creates a poll_args object for a given arguments to poll().
-// This object may return a poll_args in the cache.
-static poll_args* get_poller_locked(struct pollfd* fds, nfds_t count) {
- uint32_t key =
- gpr_murmur_hash3(fds, count * sizeof(struct pollfd), 0xDEADBEEF);
- key = key % poll_cache.size;
- poll_args* curr = poll_cache.active_pollers[key];
- while (curr) {
- if (curr->nfds == count &&
- memcmp(curr->fds, fds, count * sizeof(struct pollfd)) == 0) {
- gpr_free(fds);
- return curr;
- }
- curr = curr->next;
- }
-
- if (poll_cache.free_pollers) {
- poll_args* pargs = poll_cache.free_pollers;
- poll_cache.free_pollers = pargs->next;
- if (poll_cache.free_pollers) {
- poll_cache.free_pollers->prev = nullptr;
- }
- pargs->fds = fds;
- pargs->nfds = count;
- pargs->next = nullptr;
- pargs->prev = nullptr;
- init_result(pargs);
- cache_poller_locked(pargs);
- return pargs;
- }
-
- poll_args* pargs =
- static_cast<poll_args*>(gpr_malloc(sizeof(struct poll_args)));
- gpr_cv_init(&pargs->trigger);
- gpr_cv_init(&pargs->harvest);
- gpr_cv_init(&pargs->join);
- pargs->harvestable = false;
- pargs->joinable = false;
- pargs->fds = fds;
- pargs->nfds = count;
- pargs->next = nullptr;
- pargs->prev = nullptr;
- pargs->trigger_set = 0;
- init_result(pargs);
- cache_poller_locked(pargs);
- gpr_ref(&g_cvfds.pollcount);
- pargs->poller_thd = grpc_core::Thread("grpc_poller", &run_poll, pargs);
- pargs->poller_thd.Start();
- return pargs;
-}
-
-static void cache_delete_locked(poll_args* args) {
- if (!args->prev) {
- uint32_t key = gpr_murmur_hash3(
- args->fds, args->nfds * sizeof(struct pollfd), 0xDEADBEEF);
- key = key % poll_cache.size;
- GPR_ASSERT(poll_cache.active_pollers[key] == args);
- poll_cache.active_pollers[key] = args->next;
- } else {
- args->prev->next = args->next;
- }
-
- if (args->next) {
- args->next->prev = args->prev;
- }
-
- poll_cache.count--;
- if (poll_cache.free_pollers) {
- poll_cache.free_pollers->prev = args;
- }
- args->prev = nullptr;
- args->next = poll_cache.free_pollers;
- gpr_free(args->fds);
- poll_cache.free_pollers = args;
-}
-
-static void cache_poller_locked(poll_args* args) {
- if (poll_cache.count + 1 > poll_cache.size / 2) {
- poll_args** old_active_pollers = poll_cache.active_pollers;
- poll_cache.size = poll_cache.size * 2;
- poll_cache.count = 0;
- poll_cache.active_pollers =
- static_cast<poll_args**>(gpr_malloc(sizeof(void*) * poll_cache.size));
- for (unsigned int i = 0; i < poll_cache.size; i++) {
- poll_cache.active_pollers[i] = nullptr;
- }
- for (unsigned int i = 0; i < poll_cache.size / 2; i++) {
- poll_args* curr = old_active_pollers[i];
- poll_args* next = nullptr;
- while (curr) {
- next = curr->next;
- cache_insert_locked(curr);
- curr = next;
- }
- }
- gpr_free(old_active_pollers);
- }
-
- cache_insert_locked(args);
-}
-
-static void cache_destroy_locked(poll_args* args) {
- if (args->next) {
- args->next->prev = args->prev;
- }
-
- if (args->prev) {
- args->prev->next = args->next;
- } else {
- poll_cache.free_pollers = args->next;
- }
-
- // Now move this args to the dead poller list for later join
- if (poll_cache.dead_pollers != nullptr) {
- poll_cache.dead_pollers->prev = args;
- }
- args->prev = nullptr;
- args->next = poll_cache.dead_pollers;
- poll_cache.dead_pollers = args;
-}
-
-static void cache_harvest_locked() {
- while (poll_cache.dead_pollers) {
- poll_args* args = poll_cache.dead_pollers;
- poll_cache.dead_pollers = poll_cache.dead_pollers->next;
- // Keep the list consistent in case new dead pollers get added when we
- // release the lock below to wait on joining
- if (poll_cache.dead_pollers) {
- poll_cache.dead_pollers->prev = nullptr;
- }
- args->harvestable = true;
- gpr_cv_signal(&args->harvest);
- while (!args->joinable) {
- gpr_cv_wait(&args->join, &g_cvfds.mu,
- gpr_inf_future(GPR_CLOCK_MONOTONIC));
- }
- args->poller_thd.Join();
- gpr_cv_destroy(&args->trigger);
- gpr_cv_destroy(&args->harvest);
- gpr_cv_destroy(&args->join);
- gpr_free(args);
- }
-}
-
-static void decref_poll_result(poll_result* res) {
- if (gpr_unref(&res->refcount)) {
- GPR_ASSERT(!res->watchers);
- gpr_free(res->fds);
- gpr_free(res);
- }
-}
-
-void remove_cvn(grpc_cv_node** head, grpc_cv_node* target) {
- if (target->next) {
- target->next->prev = target->prev;
- }
-
- if (target->prev) {
- target->prev->next = target->next;
- } else {
- *head = target->next;
- }
-}
-
-gpr_timespec thread_grace;
-
-// Poll in a background thread
-static void run_poll(void* args) {
- poll_args* pargs = static_cast<poll_args*>(args);
- while (1) {
- poll_result* result = pargs->result;
- int retval = g_cvfds.poll(result->fds, result->nfds, CV_POLL_PERIOD_MS);
- gpr_mu_lock(&g_cvfds.mu);
- cache_harvest_locked();
- if (retval != 0) {
- result->completed = 1;
- result->retval = retval;
- result->err = errno;
- grpc_cv_node* watcher = result->watchers;
- while (watcher) {
- gpr_cv_signal(watcher->cv);
- watcher = watcher->next;
- }
- }
- if (result->watchcount == 0 || result->completed) {
- cache_delete_locked(pargs);
- decref_poll_result(result);
- // Leave this polling thread alive for a grace period to do another poll()
- // op
- gpr_timespec deadline = gpr_now(GPR_CLOCK_MONOTONIC);
- deadline = gpr_time_add(deadline, thread_grace);
- pargs->trigger_set = 0;
- gpr_cv_wait(&pargs->trigger, &g_cvfds.mu, deadline);
- cache_harvest_locked();
- if (!pargs->trigger_set) {
- cache_destroy_locked(pargs);
- break;
- }
- }
- gpr_mu_unlock(&g_cvfds.mu);
- }
-
- if (gpr_unref(&g_cvfds.pollcount)) {
- gpr_cv_signal(&g_cvfds.shutdown_cv);
- }
- while (!pargs->harvestable) {
- gpr_cv_wait(&pargs->harvest, &g_cvfds.mu,
- gpr_inf_future(GPR_CLOCK_MONOTONIC));
- }
- pargs->joinable = true;
- gpr_cv_signal(&pargs->join);
- gpr_mu_unlock(&g_cvfds.mu);
-}
-
-// This function overrides poll() to handle condition variable wakeup fds
-static int cvfd_poll(struct pollfd* fds, nfds_t nfds, int timeout) {
- if (timeout == 0) {
- // Don't bother using background threads for polling if timeout is 0,
- // poll-cv might not wait for a poll to return otherwise.
- // https://github.com/grpc/grpc/issues/13298
- return poll(fds, nfds, 0);
- }
- unsigned int i;
- int res, idx;
- grpc_cv_node* pollcv;
- int skip_poll = 0;
- nfds_t nsockfds = 0;
- poll_result* result = nullptr;
- gpr_mu_lock(&g_cvfds.mu);
- cache_harvest_locked();
- pollcv = static_cast<grpc_cv_node*>(gpr_malloc(sizeof(grpc_cv_node)));
- pollcv->next = nullptr;
- gpr_cv pollcv_cv;
- gpr_cv_init(&pollcv_cv);
- pollcv->cv = &pollcv_cv;
- grpc_cv_node* fd_cvs =
- static_cast<grpc_cv_node*>(gpr_malloc(nfds * sizeof(grpc_cv_node)));
-
- for (i = 0; i < nfds; i++) {
- fds[i].revents = 0;
- if (fds[i].fd < 0 && (fds[i].events & POLLIN)) {
- idx = GRPC_FD_TO_IDX(fds[i].fd);
- fd_cvs[i].cv = &pollcv_cv;
- fd_cvs[i].prev = nullptr;
- fd_cvs[i].next = g_cvfds.cvfds[idx].cvs;
- if (g_cvfds.cvfds[idx].cvs) {
- g_cvfds.cvfds[idx].cvs->prev = &(fd_cvs[i]);
- }
- g_cvfds.cvfds[idx].cvs = &(fd_cvs[i]);
- // Don't bother polling if a wakeup fd is ready
- if (g_cvfds.cvfds[idx].is_set) {
- skip_poll = 1;
- }
- } else if (fds[i].fd >= 0) {
- nsockfds++;
- }
- }
-
- gpr_timespec deadline = gpr_now(GPR_CLOCK_MONOTONIC);
- if (timeout < 0) {
- deadline = gpr_inf_future(GPR_CLOCK_MONOTONIC);
- } else {
- deadline =
- gpr_time_add(deadline, gpr_time_from_millis(timeout, GPR_TIMESPAN));
- }
-
- res = 0;
- if (!skip_poll && nsockfds > 0) {
- struct pollfd* pollfds = static_cast<struct pollfd*>(
- gpr_malloc(sizeof(struct pollfd) * nsockfds));
- idx = 0;
- for (i = 0; i < nfds; i++) {
- if (fds[i].fd >= 0) {
- pollfds[idx].fd = fds[i].fd;
- pollfds[idx].events = fds[i].events;
- pollfds[idx].revents = 0;
- idx++;
- }
- }
- poll_args* pargs = get_poller_locked(pollfds, nsockfds);
- result = pargs->result;
- pollcv->next = result->watchers;
- pollcv->prev = nullptr;
- if (result->watchers) {
- result->watchers->prev = pollcv;
- }
- result->watchers = pollcv;
- result->watchcount++;
- gpr_ref(&result->refcount);
-
- pargs->trigger_set = 1;
- gpr_cv_signal(&pargs->trigger);
- gpr_cv_wait(&pollcv_cv, &g_cvfds.mu, deadline);
- cache_harvest_locked();
- res = result->retval;
- errno = result->err;
- result->watchcount--;
- remove_cvn(&result->watchers, pollcv);
- } else if (!skip_poll) {
- gpr_cv_wait(&pollcv_cv, &g_cvfds.mu, deadline);
- cache_harvest_locked();
- }
-
- idx = 0;
- for (i = 0; i < nfds; i++) {
- if (fds[i].fd < 0 && (fds[i].events & POLLIN)) {
- remove_cvn(&g_cvfds.cvfds[GRPC_FD_TO_IDX(fds[i].fd)].cvs, &(fd_cvs[i]));
- if (g_cvfds.cvfds[GRPC_FD_TO_IDX(fds[i].fd)].is_set) {
- fds[i].revents = POLLIN;
- if (res >= 0) res++;
- }
- } else if (!skip_poll && fds[i].fd >= 0 && result->completed) {
- fds[i].revents = result->fds[idx].revents;
- idx++;
- }
- }
-
- gpr_free(fd_cvs);
- gpr_cv_destroy(pollcv->cv);
- gpr_free(pollcv);
- if (result) {
- decref_poll_result(result);
- }
-
- gpr_mu_unlock(&g_cvfds.mu);
-
- return res;
-}
-
-static void global_cv_fd_table_init() {
- gpr_mu_init(&g_cvfds.mu);
- gpr_mu_lock(&g_cvfds.mu);
- gpr_cv_init(&g_cvfds.shutdown_cv);
- gpr_ref_init(&g_cvfds.pollcount, 1);
- g_cvfds.size = CV_DEFAULT_TABLE_SIZE;
- g_cvfds.cvfds = static_cast<grpc_fd_node*>(
- gpr_malloc(sizeof(grpc_fd_node) * CV_DEFAULT_TABLE_SIZE));
- g_cvfds.free_fds = nullptr;
- thread_grace = gpr_time_from_millis(POLLCV_THREAD_GRACE_MS, GPR_TIMESPAN);
- for (int i = 0; i < CV_DEFAULT_TABLE_SIZE; i++) {
- g_cvfds.cvfds[i].is_set = 0;
- g_cvfds.cvfds[i].cvs = nullptr;
- g_cvfds.cvfds[i].next_free = g_cvfds.free_fds;
- g_cvfds.free_fds = &g_cvfds.cvfds[i];
- }
- // Override the poll function with one that supports cvfds
- g_cvfds.poll = grpc_poll_function;
- grpc_poll_function = &cvfd_poll;
-
- // Initialize the cache
- poll_cache.size = 32;
- poll_cache.count = 0;
- poll_cache.free_pollers = nullptr;
- poll_cache.active_pollers =
- static_cast<poll_args**>(gpr_malloc(sizeof(void*) * 32));
- for (unsigned int i = 0; i < poll_cache.size; i++) {
- poll_cache.active_pollers[i] = nullptr;
- }
- poll_cache.dead_pollers = nullptr;
-
- gpr_mu_unlock(&g_cvfds.mu);
-}
-
-static void global_cv_fd_table_shutdown() {
- gpr_mu_lock(&g_cvfds.mu);
- // Attempt to wait for all abandoned poll() threads to terminate
- // Not doing so will result in reported memory leaks
- if (!gpr_unref(&g_cvfds.pollcount)) {
- int res = gpr_cv_wait(&g_cvfds.shutdown_cv, &g_cvfds.mu,
- gpr_time_add(gpr_now(GPR_CLOCK_MONOTONIC),
- gpr_time_from_seconds(3, GPR_TIMESPAN)));
- GPR_ASSERT(res == 0);
- }
- gpr_cv_destroy(&g_cvfds.shutdown_cv);
- grpc_poll_function = g_cvfds.poll;
- gpr_free(g_cvfds.cvfds);
-
- cache_harvest_locked();
- gpr_free(poll_cache.active_pollers);
-
- gpr_mu_unlock(&g_cvfds.mu);
- gpr_mu_destroy(&g_cvfds.mu);
-}
-
/*******************************************************************************
* event engine binding
*/
@@ -1792,9 +1322,6 @@ static void shutdown_background_closure(void) {}
static void shutdown_engine(void) {
pollset_global_shutdown();
- if (grpc_cv_wakeup_fds_enabled()) {
- global_cv_fd_table_shutdown();
- }
if (track_fds_for_fork) {
gpr_mu_destroy(&fork_fd_list_mu);
grpc_core::Fork::SetResetChildPollingEngineFunc(nullptr);
@@ -1876,15 +1403,4 @@ const grpc_event_engine_vtable* grpc_init_poll_posix(bool explicit_request) {
return &vtable;
}
-const grpc_event_engine_vtable* grpc_init_poll_cv_posix(bool explicit_request) {
- global_cv_fd_table_init();
- grpc_enable_cv_wakeup_fds(1);
- if (!GRPC_LOG_IF_ERROR("pollset_global_init", pollset_global_init())) {
- global_cv_fd_table_shutdown();
- grpc_enable_cv_wakeup_fds(0);
- return nullptr;
- }
- return &vtable;
-}
-
#endif /* GRPC_POSIX_SOCKET_EV_POLL */
diff --git a/src/core/lib/iomgr/ev_posix.cc b/src/core/lib/iomgr/ev_posix.cc
index fb2e70eee49..d7aeb81c69e 100644
--- a/src/core/lib/iomgr/ev_posix.cc
+++ b/src/core/lib/iomgr/ev_posix.cc
@@ -126,10 +126,9 @@ static event_engine_factory g_factories[] = {
{ENGINE_HEAD_CUSTOM, nullptr}, {ENGINE_HEAD_CUSTOM, nullptr},
{ENGINE_HEAD_CUSTOM, nullptr}, {ENGINE_HEAD_CUSTOM, nullptr},
{"epollex", grpc_init_epollex_linux}, {"epoll1", grpc_init_epoll1_linux},
- {"poll", grpc_init_poll_posix}, {"poll-cv", grpc_init_poll_cv_posix},
- {"none", init_non_polling}, {ENGINE_TAIL_CUSTOM, nullptr},
+ {"poll", grpc_init_poll_posix}, {"none", init_non_polling},
+ {ENGINE_TAIL_CUSTOM, nullptr}, {ENGINE_TAIL_CUSTOM, nullptr},
{ENGINE_TAIL_CUSTOM, nullptr}, {ENGINE_TAIL_CUSTOM, nullptr},
- {ENGINE_TAIL_CUSTOM, nullptr},
};
static void add(const char* beg, const char* end, char*** ss, size_t* ns) {
diff --git a/src/core/lib/iomgr/wakeup_fd_cv.cc b/src/core/lib/iomgr/wakeup_fd_cv.cc
deleted file mode 100644
index 74faa6379ef..00000000000
--- a/src/core/lib/iomgr/wakeup_fd_cv.cc
+++ /dev/null
@@ -1,107 +0,0 @@
-/*
- *
- * Copyright 2016 gRPC authors.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-#include <grpc/support/port_platform.h>
-
-#include "src/core/lib/iomgr/port.h"
-
-#ifdef GRPC_POSIX_WAKEUP_FD
-
-#include "src/core/lib/iomgr/wakeup_fd_cv.h"
-
-#include <errno.h>
-#include <string.h>
-
-#include <grpc/support/alloc.h>
-#include <grpc/support/log.h>
-#include <grpc/support/sync.h>
-#include <grpc/support/time.h>
-
-#include "src/core/lib/gpr/useful.h"
-#include "src/core/lib/gprpp/thd.h"
-
-#define MAX_TABLE_RESIZE 256
-
-extern grpc_cv_fd_table g_cvfds;
-
-static grpc_error* cv_fd_init(grpc_wakeup_fd* fd_info) {
- unsigned int i, newsize;
- int idx;
- gpr_mu_lock(&g_cvfds.mu);
- if (!g_cvfds.free_fds) {
- newsize = GPR_MIN(g_cvfds.size * 2, g_cvfds.size + MAX_TABLE_RESIZE);
- g_cvfds.cvfds = static_cast<grpc_fd_node*>(
- gpr_realloc(g_cvfds.cvfds, sizeof(grpc_fd_node) * newsize));
- for (i = g_cvfds.size; i < newsize; i++) {
- g_cvfds.cvfds[i].is_set = 0;
- g_cvfds.cvfds[i].cvs = nullptr;
- g_cvfds.cvfds[i].next_free = g_cvfds.free_fds;
- g_cvfds.free_fds = &g_cvfds.cvfds[i];
- }
- g_cvfds.size = newsize;
- }
-
- idx = static_cast<int>(g_cvfds.free_fds - g_cvfds.cvfds);
- g_cvfds.free_fds = g_cvfds.free_fds->next_free;
- g_cvfds.cvfds[idx].cvs = nullptr;
- g_cvfds.cvfds[idx].is_set = 0;
- fd_info->read_fd = GRPC_IDX_TO_FD(idx);
- fd_info->write_fd = -1;
- gpr_mu_unlock(&g_cvfds.mu);
- return GRPC_ERROR_NONE;
-}
-
-static grpc_error* cv_fd_wakeup(grpc_wakeup_fd* fd_info) {
- grpc_cv_node* cvn;
- gpr_mu_lock(&g_cvfds.mu);
- g_cvfds.cvfds[GRPC_FD_TO_IDX(fd_info->read_fd)].is_set = 1;
- cvn = g_cvfds.cvfds[GRPC_FD_TO_IDX(fd_info->read_fd)].cvs;
- while (cvn) {
- gpr_cv_signal(cvn->cv);
- cvn = cvn->next;
- }
- gpr_mu_unlock(&g_cvfds.mu);
- return GRPC_ERROR_NONE;
-}
-
-static grpc_error* cv_fd_consume(grpc_wakeup_fd* fd_info) {
- gpr_mu_lock(&g_cvfds.mu);
- g_cvfds.cvfds[GRPC_FD_TO_IDX(fd_info->read_fd)].is_set = 0;
- gpr_mu_unlock(&g_cvfds.mu);
- return GRPC_ERROR_NONE;
-}
-
-static void cv_fd_destroy(grpc_wakeup_fd* fd_info) {
- if (fd_info->read_fd == 0) {
- return;
- }
- gpr_mu_lock(&g_cvfds.mu);
- // Assert that there are no active pollers
- GPR_ASSERT(!g_cvfds.cvfds[GRPC_FD_TO_IDX(fd_info->read_fd)].cvs);
- g_cvfds.cvfds[GRPC_FD_TO_IDX(fd_info->read_fd)].next_free = g_cvfds.free_fds;
- g_cvfds.free_fds = &g_cvfds.cvfds[GRPC_FD_TO_IDX(fd_info->read_fd)];
- gpr_mu_unlock(&g_cvfds.mu);
-}
-
-static int cv_check_availability(void) { return 1; }
-
-const grpc_wakeup_fd_vtable grpc_cv_wakeup_fd_vtable = {
- cv_fd_init, cv_fd_consume, cv_fd_wakeup, cv_fd_destroy,
- cv_check_availability};
-
-#endif /* GRPC_POSIX_WAKUP_FD */
diff --git a/src/core/lib/iomgr/wakeup_fd_cv.h b/src/core/lib/iomgr/wakeup_fd_cv.h
deleted file mode 100644
index 86365f07e17..00000000000
--- a/src/core/lib/iomgr/wakeup_fd_cv.h
+++ /dev/null
@@ -1,69 +0,0 @@
-/*
- *
- * Copyright 2016 gRPC authors.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-/*
- * wakeup_fd_cv uses condition variables to implement wakeup fds.
- *
- * It is intended for use only in cases when eventfd() and pipe() are not
- * available. It can only be used with the "poll" engine.
- *
- * Implementation:
- * A global table of cv wakeup fds is mantained. A cv wakeup fd is a negative
- * file descriptor. poll() is then run in a background thread with only the
- * real socket fds while we wait on a condition variable trigged by either the
- * poll() completion or a wakeup_fd() call.
- *
- */
-
-#ifndef GRPC_CORE_LIB_IOMGR_WAKEUP_FD_CV_H
-#define GRPC_CORE_LIB_IOMGR_WAKEUP_FD_CV_H
-
-#include <grpc/support/port_platform.h>
-
-#include <grpc/support/sync.h>
-
-#include "src/core/lib/iomgr/ev_posix.h"
-
-#define GRPC_FD_TO_IDX(fd) (-(fd)-1)
-#define GRPC_IDX_TO_FD(idx) (-(idx)-1)
-
-typedef struct grpc_cv_node {
- gpr_cv* cv;
- struct grpc_cv_node* next;
- struct grpc_cv_node* prev;
-} grpc_cv_node;
-
-typedef struct grpc_fd_node {
- int is_set;
- grpc_cv_node* cvs;
- struct grpc_fd_node* next_free;
-} grpc_fd_node;
-
-typedef struct grpc_cv_fd_table {
- gpr_mu mu;
- gpr_refcount pollcount;
- gpr_cv shutdown_cv;
- grpc_fd_node* cvfds;
- grpc_fd_node* free_fds;
- unsigned int size;
- grpc_poll_function_type poll;
-} grpc_cv_fd_table;
-
-extern const grpc_wakeup_fd_vtable grpc_cv_wakeup_fd_vtable;
-
-#endif /* GRPC_CORE_LIB_IOMGR_WAKEUP_FD_CV_H */
diff --git a/src/core/lib/iomgr/wakeup_fd_posix.cc b/src/core/lib/iomgr/wakeup_fd_posix.cc
index b5b8b37a9af..3b66d6f34de 100644
--- a/src/core/lib/iomgr/wakeup_fd_posix.cc
+++ b/src/core/lib/iomgr/wakeup_fd_posix.cc
@@ -23,7 +23,6 @@
#ifdef GRPC_POSIX_WAKEUP_FD
#include <stddef.h>
-#include "src/core/lib/iomgr/wakeup_fd_cv.h"
#include "src/core/lib/iomgr/wakeup_fd_pipe.h"
#include "src/core/lib/iomgr/wakeup_fd_posix.h"
@@ -51,37 +50,20 @@ void grpc_wakeup_fd_global_destroy(void) { wakeup_fd_vtable = nullptr; }
int grpc_has_wakeup_fd(void) { return has_real_wakeup_fd; }
-int grpc_cv_wakeup_fds_enabled(void) { return cv_wakeup_fds_enabled; }
-
-void grpc_enable_cv_wakeup_fds(int enable) { cv_wakeup_fds_enabled = enable; }
-
grpc_error* grpc_wakeup_fd_init(grpc_wakeup_fd* fd_info) {
- if (cv_wakeup_fds_enabled) {
- return grpc_cv_wakeup_fd_vtable.init(fd_info);
- }
return wakeup_fd_vtable->init(fd_info);
}
grpc_error* grpc_wakeup_fd_consume_wakeup(grpc_wakeup_fd* fd_info) {
- if (cv_wakeup_fds_enabled) {
- return grpc_cv_wakeup_fd_vtable.consume(fd_info);
- }
return wakeup_fd_vtable->consume(fd_info);
}
grpc_error* grpc_wakeup_fd_wakeup(grpc_wakeup_fd* fd_info) {
- if (cv_wakeup_fds_enabled) {
- return grpc_cv_wakeup_fd_vtable.wakeup(fd_info);
- }
return wakeup_fd_vtable->wakeup(fd_info);
}
void grpc_wakeup_fd_destroy(grpc_wakeup_fd* fd_info) {
- if (cv_wakeup_fds_enabled) {
- grpc_cv_wakeup_fd_vtable.destroy(fd_info);
- } else {
- wakeup_fd_vtable->destroy(fd_info);
- }
+ wakeup_fd_vtable->destroy(fd_info);
}
#endif /* GRPC_POSIX_WAKEUP_FD */
diff --git a/src/python/grpcio/grpc_core_dependencies.py b/src/python/grpcio/grpc_core_dependencies.py
index a9d045281ec..8f7da3a5b04 100644
--- a/src/python/grpcio/grpc_core_dependencies.py
+++ b/src/python/grpcio/grpc_core_dependencies.py
@@ -161,7 +161,6 @@ CORE_SOURCE_FILES = [
'src/core/lib/iomgr/udp_server.cc',
'src/core/lib/iomgr/unix_sockets_posix.cc',
'src/core/lib/iomgr/unix_sockets_posix_noop.cc',
- 'src/core/lib/iomgr/wakeup_fd_cv.cc',
'src/core/lib/iomgr/wakeup_fd_eventfd.cc',
'src/core/lib/iomgr/wakeup_fd_nospecial.cc',
'src/core/lib/iomgr/wakeup_fd_pipe.cc',
diff --git a/test/core/end2end/generate_tests.bzl b/test/core/end2end/generate_tests.bzl
index 5174a7e5af5..7bb246b6067 100755
--- a/test/core/end2end/generate_tests.bzl
+++ b/test/core/end2end/generate_tests.bzl
@@ -17,7 +17,7 @@
load("//bazel:grpc_build_system.bzl", "grpc_cc_binary", "grpc_cc_library")
-POLLERS = ["epollex", "epoll1", "poll", "poll-cv"]
+POLLERS = ["epollex", "epoll1", "poll"]
def _fixture_options(
fullstack = True,
diff --git a/test/core/end2end/tests/keepalive_timeout.cc b/test/core/end2end/tests/keepalive_timeout.cc
index c4025e93b82..2c992848b95 100644
--- a/test/core/end2end/tests/keepalive_timeout.cc
+++ b/test/core/end2end/tests/keepalive_timeout.cc
@@ -228,8 +228,7 @@ static void test_read_delays_keepalive(grpc_end2end_test_config config) {
char* poller = gpr_getenv("GRPC_POLL_STRATEGY");
/* It is hard to get the timing right for the polling engines poll and poll-cv
*/
- if (poller != nullptr &&
- (0 == strcmp(poller, "poll-cv") || 0 == strcmp(poller, "poll"))) {
+ if (poller != nullptr && (0 == strcmp(poller, "poll"))) {
gpr_free(poller);
return;
}
diff --git a/test/core/iomgr/BUILD b/test/core/iomgr/BUILD
index 7daabd50527..5e4338aee37 100644
--- a/test/core/iomgr/BUILD
+++ b/test/core/iomgr/BUILD
@@ -304,14 +304,3 @@ grpc_cc_test(
"//test/core/util:grpc_test_util",
],
)
-
-grpc_cc_test(
- name = "wakeup_fd_cv_test",
- srcs = ["wakeup_fd_cv_test.cc"],
- language = "C++",
- deps = [
- "//:gpr",
- "//:grpc",
- "//test/core/util:grpc_test_util",
- ],
-)
diff --git a/test/core/iomgr/wakeup_fd_cv_test.cc b/test/core/iomgr/wakeup_fd_cv_test.cc
deleted file mode 100644
index f297a569d2d..00000000000
--- a/test/core/iomgr/wakeup_fd_cv_test.cc
+++ /dev/null
@@ -1,243 +0,0 @@
-/*
- *
- * Copyright 2016 gRPC authors.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-#include "src/core/lib/iomgr/port.h"
-
-#ifdef GRPC_POSIX_SOCKET
-
-#include <pthread.h>
-
-#include <grpc/support/log.h>
-#include <grpc/support/time.h>
-
-#include "src/core/lib/gpr/env.h"
-#include "src/core/lib/gprpp/thd.h"
-#include "src/core/lib/iomgr/ev_posix.h"
-#include "src/core/lib/iomgr/iomgr_posix.h"
-
-typedef struct poll_args {
- struct pollfd* fds;
- nfds_t nfds;
- int timeout;
- int result;
-} poll_args;
-
-gpr_cv poll_cv;
-gpr_mu poll_mu;
-static int socket_event = 0;
-
-// Trigger a "socket" POLLIN in mock_poll()
-void trigger_socket_event() {
- gpr_mu_lock(&poll_mu);
- socket_event = 1;
- gpr_cv_broadcast(&poll_cv);
- gpr_mu_unlock(&poll_mu);
-}
-
-void reset_socket_event() {
- gpr_mu_lock(&poll_mu);
- socket_event = 0;
- gpr_mu_unlock(&poll_mu);
-}
-
-// Mocks posix poll() function
-int mock_poll(struct pollfd* fds, nfds_t nfds, int timeout) {
- int res = 0;
- gpr_timespec poll_time;
- gpr_mu_lock(&poll_mu);
- GPR_ASSERT(nfds == 3);
- GPR_ASSERT(fds[0].fd == 20);
- GPR_ASSERT(fds[1].fd == 30);
- GPR_ASSERT(fds[2].fd == 50);
- GPR_ASSERT(fds[0].events == (POLLIN | POLLHUP));
- GPR_ASSERT(fds[1].events == (POLLIN | POLLHUP));
- GPR_ASSERT(fds[2].events == POLLIN);
-
- if (timeout < 0) {
- poll_time = gpr_inf_future(GPR_CLOCK_REALTIME);
- } else {
- poll_time = gpr_time_add(gpr_now(GPR_CLOCK_REALTIME),
- gpr_time_from_millis(timeout, GPR_TIMESPAN));
- }
-
- if (socket_event || !gpr_cv_wait(&poll_cv, &poll_mu, poll_time)) {
- fds[0].revents = POLLIN;
- res = 1;
- }
- gpr_mu_unlock(&poll_mu);
- return res;
-}
-
-void background_poll(void* args) {
- poll_args* pargs = static_cast<poll_args*>(args);
- pargs->result = grpc_poll_function(pargs->fds, pargs->nfds, pargs->timeout);
-}
-
-void test_many_fds(void) {
- int i;
- grpc_wakeup_fd fd[1000];
- for (i = 0; i < 1000; i++) {
- GPR_ASSERT(grpc_wakeup_fd_init(&fd[i]) == GRPC_ERROR_NONE);
- }
- for (i = 0; i < 1000; i++) {
- grpc_wakeup_fd_destroy(&fd[i]);
- }
-}
-
-void test_poll_cv_trigger(void) {
- grpc_wakeup_fd cvfd1, cvfd2, cvfd3;
- struct pollfd pfds[6];
- poll_args pargs;
-
- GPR_ASSERT(grpc_wakeup_fd_init(&cvfd1) == GRPC_ERROR_NONE);
- GPR_ASSERT(grpc_wakeup_fd_init(&cvfd2) == GRPC_ERROR_NONE);
- GPR_ASSERT(grpc_wakeup_fd_init(&cvfd3) == GRPC_ERROR_NONE);
- GPR_ASSERT(cvfd1.read_fd < 0);
- GPR_ASSERT(cvfd2.read_fd < 0);
- GPR_ASSERT(cvfd3.read_fd < 0);
- GPR_ASSERT(cvfd1.read_fd != cvfd2.read_fd);
- GPR_ASSERT(cvfd2.read_fd != cvfd3.read_fd);
- GPR_ASSERT(cvfd1.read_fd != cvfd3.read_fd);
-
- pfds[0].fd = cvfd1.read_fd;
- pfds[1].fd = cvfd2.read_fd;
- pfds[2].fd = 20;
- pfds[3].fd = 30;
- pfds[4].fd = cvfd3.read_fd;
- pfds[5].fd = 50;
-
- pfds[0].events = 0;
- pfds[1].events = POLLIN;
- pfds[2].events = POLLIN | POLLHUP;
- pfds[3].events = POLLIN | POLLHUP;
- pfds[4].events = POLLIN;
- pfds[5].events = POLLIN;
-
- pargs.fds = pfds;
- pargs.nfds = 6;
- pargs.timeout = 1000;
- pargs.result = -2;
-
- {
- grpc_core::Thread thd("grpc_background_poll", &background_poll, &pargs);
- thd.Start();
- // Wakeup wakeup_fd not listening for events
- GPR_ASSERT(grpc_wakeup_fd_wakeup(&cvfd1) == GRPC_ERROR_NONE);
- thd.Join();
- GPR_ASSERT(pargs.result == 0);
- GPR_ASSERT(pfds[0].revents == 0);
- GPR_ASSERT(pfds[1].revents == 0);
- GPR_ASSERT(pfds[2].revents == 0);
- GPR_ASSERT(pfds[3].revents == 0);
- GPR_ASSERT(pfds[4].revents == 0);
- GPR_ASSERT(pfds[5].revents == 0);
- }
-
- {
- // Pollin on socket fd
- pargs.timeout = -1;
- pargs.result = -2;
- grpc_core::Thread thd("grpc_background_poll", &background_poll, &pargs);
- thd.Start();
- trigger_socket_event();
- thd.Join();
- GPR_ASSERT(pargs.result == 1);
- GPR_ASSERT(pfds[0].revents == 0);
- GPR_ASSERT(pfds[1].revents == 0);
- GPR_ASSERT(pfds[2].revents == POLLIN);
- GPR_ASSERT(pfds[3].revents == 0);
- GPR_ASSERT(pfds[4].revents == 0);
- GPR_ASSERT(pfds[5].revents == 0);
- }
-
- {
- // Pollin on wakeup fd
- reset_socket_event();
- pargs.result = -2;
- grpc_core::Thread thd("grpc_background_poll", &background_poll, &pargs);
- thd.Start();
- GPR_ASSERT(grpc_wakeup_fd_wakeup(&cvfd2) == GRPC_ERROR_NONE);
- thd.Join();
-
- GPR_ASSERT(pargs.result == 1);
- GPR_ASSERT(pfds[0].revents == 0);
- GPR_ASSERT(pfds[1].revents == POLLIN);
- GPR_ASSERT(pfds[2].revents == 0);
- GPR_ASSERT(pfds[3].revents == 0);
- GPR_ASSERT(pfds[4].revents == 0);
- GPR_ASSERT(pfds[5].revents == 0);
- }
-
- {
- // Pollin on wakeupfd before poll()
- pargs.result = -2;
- grpc_core::Thread thd("grpc_background_poll", &background_poll, &pargs);
- thd.Start();
- thd.Join();
-
- GPR_ASSERT(pargs.result == 1);
- GPR_ASSERT(pfds[0].revents == 0);
- GPR_ASSERT(pfds[1].revents == POLLIN);
- GPR_ASSERT(pfds[2].revents == 0);
- GPR_ASSERT(pfds[3].revents == 0);
- GPR_ASSERT(pfds[4].revents == 0);
- GPR_ASSERT(pfds[5].revents == 0);
- }
-
- {
- // No Events
- pargs.result = -2;
- pargs.timeout = 1000;
- reset_socket_event();
- GPR_ASSERT(grpc_wakeup_fd_consume_wakeup(&cvfd1) == GRPC_ERROR_NONE);
- GPR_ASSERT(grpc_wakeup_fd_consume_wakeup(&cvfd2) == GRPC_ERROR_NONE);
- grpc_core::Thread thd("grpc_background_poll", &background_poll, &pargs);
- thd.Start();
- thd.Join();
-
- GPR_ASSERT(pargs.result == 0);
- GPR_ASSERT(pfds[0].revents == 0);
- GPR_ASSERT(pfds[1].revents == 0);
- GPR_ASSERT(pfds[2].revents == 0);
- GPR_ASSERT(pfds[3].revents == 0);
- GPR_ASSERT(pfds[4].revents == 0);
- GPR_ASSERT(pfds[5].revents == 0);
- }
-}
-
-int main(int argc, char** argv) {
- gpr_setenv("GRPC_POLL_STRATEGY", "poll-cv");
- grpc_poll_function = &mock_poll;
- gpr_mu_init(&poll_mu);
- gpr_cv_init(&poll_cv);
- grpc_determine_iomgr_platform();
- grpc_iomgr_platform_init();
- test_many_fds();
- grpc_iomgr_platform_shutdown();
-
- grpc_iomgr_platform_init();
- test_poll_cv_trigger();
- grpc_iomgr_platform_shutdown();
- return 0;
-}
-
-#else /* GRPC_POSIX_SOCKET */
-
-int main(int argc, char** argv) { return 1; }
-
-#endif /* GRPC_POSIX_SOCKET */
diff --git a/test/core/util/test_config.cc b/test/core/util/test_config.cc
index 0c0492fdbbd..476e424b1eb 100644
--- a/test/core/util/test_config.cc
+++ b/test/core/util/test_config.cc
@@ -382,13 +382,6 @@ gpr_timespec grpc_timeout_milliseconds_to_deadline(int64_t time_ms) {
void grpc_test_init(int argc, char** argv) {
install_crash_handler();
- { /* poll-cv poll strategy runs much more slowly than anything else */
- char* s = gpr_getenv("GRPC_POLL_STRATEGY");
- if (s != nullptr && 0 == strcmp(s, "poll-cv")) {
- g_poller_slowdown_factor = 5;
- }
- gpr_free(s);
- }
gpr_log(GPR_DEBUG,
"test slowdown factor: sanitizer=%" PRId64 ", fixture=%" PRId64
", poller=%" PRId64 ", total=%" PRId64,
diff --git a/tools/doxygen/Doxyfile.c++.internal b/tools/doxygen/Doxyfile.c++.internal
index 664a6b3acfe..c0078bf2764 100644
--- a/tools/doxygen/Doxyfile.c++.internal
+++ b/tools/doxygen/Doxyfile.c++.internal
@@ -1146,7 +1146,6 @@ src/core/lib/iomgr/timer_heap.h \
src/core/lib/iomgr/timer_manager.h \
src/core/lib/iomgr/udp_server.h \
src/core/lib/iomgr/unix_sockets_posix.h \
-src/core/lib/iomgr/wakeup_fd_cv.h \
src/core/lib/iomgr/wakeup_fd_pipe.h \
src/core/lib/iomgr/wakeup_fd_posix.h \
src/core/lib/json/json.h \
diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal
index f01531a0078..6e4a57ba00f 100644
--- a/tools/doxygen/Doxyfile.core.internal
+++ b/tools/doxygen/Doxyfile.core.internal
@@ -1323,8 +1323,6 @@ src/core/lib/iomgr/udp_server.h \
src/core/lib/iomgr/unix_sockets_posix.cc \
src/core/lib/iomgr/unix_sockets_posix.h \
src/core/lib/iomgr/unix_sockets_posix_noop.cc \
-src/core/lib/iomgr/wakeup_fd_cv.cc \
-src/core/lib/iomgr/wakeup_fd_cv.h \
src/core/lib/iomgr/wakeup_fd_eventfd.cc \
src/core/lib/iomgr/wakeup_fd_nospecial.cc \
src/core/lib/iomgr/wakeup_fd_pipe.cc \
diff --git a/tools/run_tests/generated/sources_and_headers.json b/tools/run_tests/generated/sources_and_headers.json
index 5a1eafda6c2..7a72a885336 100644
--- a/tools/run_tests/generated/sources_and_headers.json
+++ b/tools/run_tests/generated/sources_and_headers.json
@@ -2395,22 +2395,6 @@
"third_party": false,
"type": "target"
},
- {
- "deps": [
- "gpr",
- "grpc",
- "grpc_test_util"
- ],
- "headers": [],
- "is_filegroup": false,
- "language": "c",
- "name": "wakeup_fd_cv_test",
- "src": [
- "test/core/iomgr/wakeup_fd_cv_test.cc"
- ],
- "third_party": false,
- "type": "target"
- },
{
"deps": [
"gpr",
@@ -9554,7 +9538,6 @@
"src/core/lib/iomgr/udp_server.cc",
"src/core/lib/iomgr/unix_sockets_posix.cc",
"src/core/lib/iomgr/unix_sockets_posix_noop.cc",
- "src/core/lib/iomgr/wakeup_fd_cv.cc",
"src/core/lib/iomgr/wakeup_fd_eventfd.cc",
"src/core/lib/iomgr/wakeup_fd_nospecial.cc",
"src/core/lib/iomgr/wakeup_fd_pipe.cc",
@@ -9720,7 +9703,6 @@
"src/core/lib/iomgr/timer_manager.h",
"src/core/lib/iomgr/udp_server.h",
"src/core/lib/iomgr/unix_sockets_posix.h",
- "src/core/lib/iomgr/wakeup_fd_cv.h",
"src/core/lib/iomgr/wakeup_fd_pipe.h",
"src/core/lib/iomgr/wakeup_fd_posix.h",
"src/core/lib/json/json.h",
@@ -9874,7 +9856,6 @@
"src/core/lib/iomgr/timer_manager.h",
"src/core/lib/iomgr/udp_server.h",
"src/core/lib/iomgr/unix_sockets_posix.h",
- "src/core/lib/iomgr/wakeup_fd_cv.h",
"src/core/lib/iomgr/wakeup_fd_pipe.h",
"src/core/lib/iomgr/wakeup_fd_posix.h",
"src/core/lib/json/json.h",
diff --git a/tools/run_tests/generated/tests.json b/tools/run_tests/generated/tests.json
index 9df57b5e151..c86339398b5 100644
--- a/tools/run_tests/generated/tests.json
+++ b/tools/run_tests/generated/tests.json
@@ -2959,30 +2959,6 @@
],
"uses_polling": true
},
- {
- "args": [],
- "benchmark": false,
- "ci_platforms": [
- "linux",
- "mac",
- "posix"
- ],
- "cpu_cost": 1.0,
- "exclude_configs": [],
- "exclude_iomgrs": [
- "uv"
- ],
- "flaky": false,
- "gtest": false,
- "language": "c",
- "name": "wakeup_fd_cv_test",
- "platforms": [
- "linux",
- "mac",
- "posix"
- ],
- "uses_polling": true
- },
{
"args": [],
"benchmark": false,
@@ -3669,8 +3645,7 @@
"exclude_configs": [],
"exclude_iomgrs": [],
"excluded_poll_engines": [
- "poll",
- "poll-cv"
+ "poll"
],
"flaky": false,
"gtest": false,
@@ -3696,8 +3671,7 @@
"exclude_configs": [],
"exclude_iomgrs": [],
"excluded_poll_engines": [
- "poll",
- "poll-cv"
+ "poll"
],
"flaky": false,
"gtest": false,
@@ -3725,8 +3699,7 @@
],
"exclude_iomgrs": [],
"excluded_poll_engines": [
- "poll",
- "poll-cv"
+ "poll"
],
"flaky": false,
"gtest": false,
@@ -3752,8 +3725,7 @@
"exclude_configs": [],
"exclude_iomgrs": [],
"excluded_poll_engines": [
- "poll",
- "poll-cv"
+ "poll"
],
"flaky": false,
"gtest": false,
@@ -60416,7 +60388,7 @@
{
"args": [
"--scenarios_json",
- "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_unary_qps_unconstrained_secure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_servers\": 1, \"num_clients\": 0, \"client_config\": {\"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 100, \"rpc_type\": \"UNARY\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
+ "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_unary_qps_unconstrained_secure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"num_servers\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_clients\": 0, \"client_config\": {\"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 100, \"rpc_type\": \"UNARY\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
],
"auto_timeout_scaling": false,
"boringssl": true,
@@ -60429,9 +60401,7 @@
"tsan",
"asan"
],
- "excluded_poll_engines": [
- "poll-cv"
- ],
+ "excluded_poll_engines": [],
"flaky": false,
"language": "c++",
"name": "json_run_localhost",
@@ -60470,7 +60440,7 @@
{
"args": [
"--scenarios_json",
- "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_streaming_qps_unconstrained_secure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_servers\": 1, \"num_clients\": 0, \"client_config\": {\"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 100, \"rpc_type\": \"STREAMING\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
+ "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_streaming_qps_unconstrained_secure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"num_servers\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_clients\": 0, \"client_config\": {\"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 100, \"rpc_type\": \"STREAMING\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
],
"auto_timeout_scaling": false,
"boringssl": true,
@@ -60483,9 +60453,7 @@
"tsan",
"asan"
],
- "excluded_poll_engines": [
- "poll-cv"
- ],
+ "excluded_poll_engines": [],
"flaky": false,
"language": "c++",
"name": "json_run_localhost",
@@ -61382,7 +61350,7 @@
{
"args": [
"--scenarios_json",
- "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_unary_qps_unconstrained_insecure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"security_params\": null, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_servers\": 1, \"num_clients\": 0, \"client_config\": {\"security_params\": null, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 100, \"rpc_type\": \"UNARY\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
+ "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_unary_qps_unconstrained_insecure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"num_servers\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"security_params\": null, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_clients\": 0, \"client_config\": {\"security_params\": null, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 100, \"rpc_type\": \"UNARY\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
],
"auto_timeout_scaling": false,
"boringssl": true,
@@ -61395,9 +61363,7 @@
"tsan",
"asan"
],
- "excluded_poll_engines": [
- "poll-cv"
- ],
+ "excluded_poll_engines": [],
"flaky": false,
"language": "c++",
"name": "json_run_localhost",
@@ -61436,7 +61402,7 @@
{
"args": [
"--scenarios_json",
- "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_streaming_qps_unconstrained_insecure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"security_params\": null, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_servers\": 1, \"num_clients\": 0, \"client_config\": {\"security_params\": null, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 100, \"rpc_type\": \"STREAMING\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
+ "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_streaming_qps_unconstrained_insecure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"num_servers\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"security_params\": null, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_clients\": 0, \"client_config\": {\"security_params\": null, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 100, \"rpc_type\": \"STREAMING\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
],
"auto_timeout_scaling": false,
"boringssl": true,
@@ -61449,9 +61415,7 @@
"tsan",
"asan"
],
- "excluded_poll_engines": [
- "poll-cv"
- ],
+ "excluded_poll_engines": [],
"flaky": false,
"language": "c++",
"name": "json_run_localhost",
@@ -62375,7 +62339,7 @@
"args": [
"--run_inproc",
"--scenarios_json",
- "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_unary_qps_unconstrained_insecure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"security_params\": null, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_servers\": 1, \"num_clients\": 0, \"client_config\": {\"security_params\": null, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 100, \"rpc_type\": \"UNARY\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
+ "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_unary_qps_unconstrained_insecure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"num_servers\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"security_params\": null, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_clients\": 0, \"client_config\": {\"security_params\": null, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 100, \"rpc_type\": \"UNARY\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
],
"boringssl": true,
"ci_platforms": [
@@ -62387,9 +62351,7 @@
"tsan",
"asan"
],
- "excluded_poll_engines": [
- "poll-cv"
- ],
+ "excluded_poll_engines": [],
"flaky": false,
"language": "c++",
"name": "qps_json_driver",
@@ -62429,7 +62391,7 @@
"args": [
"--run_inproc",
"--scenarios_json",
- "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_streaming_qps_unconstrained_insecure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"security_params\": null, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_servers\": 1, \"num_clients\": 0, \"client_config\": {\"security_params\": null, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 100, \"rpc_type\": \"STREAMING\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
+ "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_streaming_qps_unconstrained_insecure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"num_servers\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"security_params\": null, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_clients\": 0, \"client_config\": {\"security_params\": null, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 100, \"rpc_type\": \"STREAMING\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
],
"boringssl": true,
"ci_platforms": [
@@ -62441,9 +62403,7 @@
"tsan",
"asan"
],
- "excluded_poll_engines": [
- "poll-cv"
- ],
+ "excluded_poll_engines": [],
"flaky": false,
"language": "c++",
"name": "qps_json_driver",
@@ -63434,7 +63394,7 @@
{
"args": [
"--scenarios_json",
- "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_unary_qps_unconstrained_secure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_servers\": 1, \"num_clients\": 0, \"client_config\": {\"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 10, \"rpc_type\": \"UNARY\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
+ "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_unary_qps_unconstrained_secure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"num_servers\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_clients\": 0, \"client_config\": {\"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 10, \"rpc_type\": \"UNARY\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
],
"auto_timeout_scaling": false,
"boringssl": true,
@@ -63461,9 +63421,7 @@
"stapprof",
"ubsan"
],
- "excluded_poll_engines": [
- "poll-cv"
- ],
+ "excluded_poll_engines": [],
"flaky": false,
"language": "c++",
"name": "json_run_localhost",
@@ -63516,7 +63474,7 @@
{
"args": [
"--scenarios_json",
- "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_streaming_qps_unconstrained_secure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_servers\": 1, \"num_clients\": 0, \"client_config\": {\"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 10, \"rpc_type\": \"STREAMING\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
+ "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_streaming_qps_unconstrained_secure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"num_servers\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_clients\": 0, \"client_config\": {\"security_params\": {\"use_test_ca\": true, \"server_host_override\": \"foo.test.google.fr\"}, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 10, \"rpc_type\": \"STREAMING\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
],
"auto_timeout_scaling": false,
"boringssl": true,
@@ -63543,9 +63501,7 @@
"stapprof",
"ubsan"
],
- "excluded_poll_engines": [
- "poll-cv"
- ],
+ "excluded_poll_engines": [],
"flaky": false,
"language": "c++",
"name": "json_run_localhost",
@@ -64918,7 +64874,7 @@
{
"args": [
"--scenarios_json",
- "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_unary_qps_unconstrained_insecure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"security_params\": null, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_servers\": 1, \"num_clients\": 0, \"client_config\": {\"security_params\": null, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 10, \"rpc_type\": \"UNARY\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
+ "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_unary_qps_unconstrained_insecure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"num_servers\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"security_params\": null, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_clients\": 0, \"client_config\": {\"security_params\": null, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 10, \"rpc_type\": \"UNARY\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
],
"auto_timeout_scaling": false,
"boringssl": true,
@@ -64945,9 +64901,7 @@
"stapprof",
"ubsan"
],
- "excluded_poll_engines": [
- "poll-cv"
- ],
+ "excluded_poll_engines": [],
"flaky": false,
"language": "c++",
"name": "json_run_localhost",
@@ -65000,7 +64954,7 @@
{
"args": [
"--scenarios_json",
- "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_streaming_qps_unconstrained_insecure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"security_params\": null, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_servers\": 1, \"num_clients\": 0, \"client_config\": {\"security_params\": null, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 10, \"rpc_type\": \"STREAMING\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
+ "{\"scenarios\": [{\"name\": \"cpp_protobuf_async_client_sync_server_streaming_qps_unconstrained_insecure\", \"warmup_seconds\": 0, \"benchmark_seconds\": 1, \"num_servers\": 1, \"server_config\": {\"async_server_threads\": 0, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"security_params\": null, \"threads_per_cq\": 0, \"server_type\": \"SYNC_SERVER\"}, \"num_clients\": 0, \"client_config\": {\"security_params\": null, \"channel_args\": [{\"str_value\": \"throughput\", \"name\": \"grpc.optimization_target\"}, {\"int_value\": 1, \"name\": \"grpc.minimal_stack\"}], \"async_client_threads\": 0, \"outstanding_rpcs_per_channel\": 10, \"rpc_type\": \"STREAMING\", \"payload_config\": {\"simple_params\": {\"resp_size\": 0, \"req_size\": 0}}, \"client_channels\": 64, \"threads_per_cq\": 0, \"load_params\": {\"closed_loop\": {}}, \"client_type\": \"ASYNC_CLIENT\", \"histogram_params\": {\"max_possible\": 60000000000.0, \"resolution\": 0.01}}}]}"
],
"auto_timeout_scaling": false,
"boringssl": true,
@@ -65027,9 +64981,7 @@
"stapprof",
"ubsan"
],
- "excluded_poll_engines": [
- "poll-cv"
- ],
+ "excluded_poll_engines": [],
"flaky": false,
"language": "c++",
"name": "json_run_localhost",
diff --git a/tools/run_tests/performance/scenario_config.py b/tools/run_tests/performance/scenario_config.py
index 481918c52e4..ac25b22d9e4 100644
--- a/tools/run_tests/performance/scenario_config.py
+++ b/tools/run_tests/performance/scenario_config.py
@@ -463,8 +463,7 @@ class CXXLanguage:
secure=secure,
minimal_stack=not secure,
categories=smoketest_categories + inproc_categories +
- [SCALABLE],
- excluded_poll_engines=['poll-cv'])
+ [SCALABLE])
yield _ping_pong_scenario(
'cpp_protobuf_async_client_unary_1channel_64wide_128Breq_8MBresp_%s'
@@ -490,8 +489,7 @@ class CXXLanguage:
secure=secure,
minimal_stack=not secure,
categories=smoketest_categories + inproc_categories +
- [SCALABLE],
- excluded_poll_engines=['poll-cv'])
+ [SCALABLE])
yield _ping_pong_scenario(
'cpp_protobuf_async_unary_ping_pong_%s_1MB' % secstr,
diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py
index ed1c41e3256..19a79ea41dd 100755
--- a/tools/run_tests/run_tests.py
+++ b/tools/run_tests/run_tests.py
@@ -61,7 +61,7 @@ _FORCE_ENVIRON_FOR_WRAPPERS = {
}
_POLLING_STRATEGIES = {
- 'linux': ['epollex', 'epoll1', 'poll', 'poll-cv'],
+ 'linux': ['epollex', 'epoll1', 'poll'],
'mac': ['poll'],
}
From 4241edeaa45972465e707d5a2d182a3c447e2610 Mon Sep 17 00:00:00 2001
From: billfeng327 <yfen@google.com>
Date: Thu, 7 Mar 2019 14:52:49 -0800
Subject: [PATCH 060/138] renamed tag to no_windows in conformation with Bazel
and TensorFlow
---
test/core/bad_connection/BUILD | 2 +-
test/core/client_channel/BUILD | 2 +-
test/core/iomgr/BUILD | 20 +++++------
test/cpp/common/BUILD | 2 +-
test/cpp/end2end/BUILD | 4 +--
test/cpp/interop/BUILD | 2 +-
test/cpp/microbenchmarks/BUILD | 34 +++++++++----------
.../generate_resolver_component_tests.bzl | 6 ++--
test/cpp/performance/BUILD | 2 +-
test/cpp/qps/qps_benchmark_script.bzl | 2 +-
test/cpp/server/BUILD | 6 ++--
test/cpp/server/load_reporter/BUILD | 2 +-
tools/remote_build/windows.bazelrc | 4 +--
13 files changed, 44 insertions(+), 44 deletions(-)
diff --git a/test/core/bad_connection/BUILD b/test/core/bad_connection/BUILD
index 4de9c0eb2d8..82b38ccc469 100644
--- a/test/core/bad_connection/BUILD
+++ b/test/core/bad_connection/BUILD
@@ -29,5 +29,5 @@ grpc_cc_binary(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
diff --git a/test/core/client_channel/BUILD b/test/core/client_channel/BUILD
index d67f326aa6d..68a71632daf 100644
--- a/test/core/client_channel/BUILD
+++ b/test/core/client_channel/BUILD
@@ -52,7 +52,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
diff --git a/test/core/iomgr/BUILD b/test/core/iomgr/BUILD
index f9da7f7ba73..57d8c70db1d 100644
--- a/test/core/iomgr/BUILD
+++ b/test/core/iomgr/BUILD
@@ -81,7 +81,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
@@ -93,7 +93,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
@@ -105,7 +105,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
@@ -142,7 +142,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
@@ -157,7 +157,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
@@ -219,7 +219,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
@@ -231,7 +231,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
@@ -244,7 +244,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
@@ -267,7 +267,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
@@ -312,7 +312,7 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
diff --git a/test/cpp/common/BUILD b/test/cpp/common/BUILD
index e4ed3bc5460..b67c1995ff7 100644
--- a/test/cpp/common/BUILD
+++ b/test/cpp/common/BUILD
@@ -28,7 +28,7 @@ grpc_cc_test(
"//:grpc++_unsecure",
"//test/core/util:grpc_test_util_unsecure",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
diff --git a/test/cpp/end2end/BUILD b/test/cpp/end2end/BUILD
index a0c7567c2b0..56b3219ae17 100644
--- a/test/cpp/end2end/BUILD
+++ b/test/cpp/end2end/BUILD
@@ -99,7 +99,7 @@ grpc_cc_test(
"//test/core/util:grpc_test_util",
"//test/cpp/util:test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
@@ -630,7 +630,7 @@ grpc_cc_test(
"//test/core/util:grpc_test_util",
"//test/cpp/util:test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
diff --git a/test/cpp/interop/BUILD b/test/cpp/interop/BUILD
index d74566f56a7..6cf4719c17b 100644
--- a/test/cpp/interop/BUILD
+++ b/test/cpp/interop/BUILD
@@ -161,5 +161,5 @@ grpc_cc_test(
"//test/cpp/util:test_config",
"//test/cpp/util:test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
diff --git a/test/cpp/microbenchmarks/BUILD b/test/cpp/microbenchmarks/BUILD
index db37d37af6c..1ce71b1bb99 100644
--- a/test/cpp/microbenchmarks/BUILD
+++ b/test/cpp/microbenchmarks/BUILD
@@ -45,7 +45,7 @@ grpc_cc_library(
"//test/core/util:grpc_test_util_unsecure",
"//test/cpp/util:test_config",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_binary(
@@ -53,7 +53,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_closure.cc"],
deps = [":helpers"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_binary(
@@ -61,7 +61,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_alarm.cc"],
deps = [":helpers"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_binary(
@@ -76,7 +76,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_byte_buffer.cc"],
deps = [":helpers"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_binary(
@@ -84,7 +84,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_channel.cc"],
deps = [":helpers"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_binary(
@@ -92,7 +92,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_call_create.cc"],
deps = [":helpers"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_binary(
@@ -100,7 +100,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_cq.cc"],
deps = [":helpers"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_binary(
@@ -108,7 +108,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_cq_multiple_threads.cc"],
deps = [":helpers"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_binary(
@@ -116,7 +116,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_error.cc"],
deps = [":helpers"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_library(
@@ -126,7 +126,7 @@ grpc_cc_library(
"fullstack_streaming_ping_pong.h",
],
deps = [":helpers"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_binary(
@@ -136,7 +136,7 @@ grpc_cc_binary(
"bm_fullstack_streaming_ping_pong.cc",
],
deps = [":fullstack_streaming_ping_pong_h"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_library(
@@ -155,7 +155,7 @@ grpc_cc_binary(
"bm_fullstack_streaming_pump.cc",
],
deps = [":fullstack_streaming_pump_h"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_binary(
@@ -163,7 +163,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_fullstack_trickle.cc"],
deps = [":helpers"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_library(
@@ -182,7 +182,7 @@ grpc_cc_binary(
"bm_fullstack_unary_ping_pong.cc",
],
deps = [":fullstack_unary_ping_pong_h"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_binary(
@@ -190,7 +190,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_metadata.cc"],
deps = [":helpers"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_binary(
@@ -198,7 +198,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_chttp2_hpack.cc"],
deps = [":helpers"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_binary(
@@ -218,5 +218,5 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_timer.cc"],
deps = [":helpers"],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
diff --git a/test/cpp/naming/generate_resolver_component_tests.bzl b/test/cpp/naming/generate_resolver_component_tests.bzl
index 8e584289628..589176762e6 100755
--- a/test/cpp/naming/generate_resolver_component_tests.bzl
+++ b/test/cpp/naming/generate_resolver_component_tests.bzl
@@ -33,7 +33,7 @@ def generate_resolver_component_tests():
"//:gpr",
"//test/cpp/util:test_config",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
# meant to be invoked only through the top-level shell script driver
grpc_cc_binary(
@@ -53,7 +53,7 @@ def generate_resolver_component_tests():
"//:gpr",
"//test/cpp/util:test_config",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
name = "resolver_component_tests_runner_invoker%s" % unsecure_build_config_suffix,
@@ -80,5 +80,5 @@ def generate_resolver_component_tests():
"--test_bin_name=resolver_component_test%s" % unsecure_build_config_suffix,
"--running_under_bazel=true",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
diff --git a/test/cpp/performance/BUILD b/test/cpp/performance/BUILD
index ddc41e75102..6068c33f95f 100644
--- a/test/cpp/performance/BUILD
+++ b/test/cpp/performance/BUILD
@@ -31,5 +31,5 @@ grpc_cc_test(
"//src/proto/grpc/testing:echo_proto",
"//test/core/util:grpc_test_util_base",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
diff --git a/test/cpp/qps/qps_benchmark_script.bzl b/test/cpp/qps/qps_benchmark_script.bzl
index 23b42c02b64..b4767ec8e09 100644
--- a/test/cpp/qps/qps_benchmark_script.bzl
+++ b/test/cpp/qps/qps_benchmark_script.bzl
@@ -75,6 +75,6 @@ def json_run_localhost_batch():
],
tags = [
"json_run_localhost",
- "exclude_windows",
+ "no_windows",
],
)
diff --git a/test/cpp/server/BUILD b/test/cpp/server/BUILD
index 3c4b35af709..a4811031691 100644
--- a/test/cpp/server/BUILD
+++ b/test/cpp/server/BUILD
@@ -29,7 +29,7 @@ grpc_cc_test(
"//src/proto/grpc/testing:echo_proto",
"//test/core/util:grpc_test_util_unsecure",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
@@ -43,7 +43,7 @@ grpc_cc_test(
"//src/proto/grpc/testing:echo_proto",
"//test/core/util:grpc_test_util_unsecure",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
@@ -57,5 +57,5 @@ grpc_cc_test(
"//src/proto/grpc/testing:echo_proto",
"//test/core/util:grpc_test_util_unsecure",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
diff --git a/test/cpp/server/load_reporter/BUILD b/test/cpp/server/load_reporter/BUILD
index 2286119324b..db5c93263ad 100644
--- a/test/cpp/server/load_reporter/BUILD
+++ b/test/cpp/server/load_reporter/BUILD
@@ -45,7 +45,7 @@ grpc_cc_test(
"//:lb_server_load_reporting_filter",
"//test/core/util:grpc_test_util",
],
- tags = ["exclude_windows"],
+ tags = ["no_windows"],
)
grpc_cc_test(
diff --git a/tools/remote_build/windows.bazelrc b/tools/remote_build/windows.bazelrc
index 86025006e3d..11f57cac32d 100644
--- a/tools/remote_build/windows.bazelrc
+++ b/tools/remote_build/windows.bazelrc
@@ -1,3 +1,3 @@
# TODO(yfen): Merge with rbe_common.bazelrc and enable Windows RBE
-build --test_tag_filters=-exclude_windows
-build --build_tag_filters=-exclude_windows
\ No newline at end of file
+build --test_tag_filters=-no_windows
+build --build_tag_filters=-no_windows
\ No newline at end of file
From 620d80d8172995706da6e9f7284008ffb56869ce Mon Sep 17 00:00:00 2001
From: Yash Tibrewal <yashkt@google.com>
Date: Thu, 7 Mar 2019 15:03:31 -0800
Subject: [PATCH 061/138] Remove poll-cv references from bazel and run_tests.py
---
bazel/grpc_build_system.bzl | 2 +-
tools/run_tests/run_tests.py | 9 ---------
2 files changed, 1 insertion(+), 10 deletions(-)
diff --git a/bazel/grpc_build_system.bzl b/bazel/grpc_build_system.bzl
index 3ea8e305ca5..dbcaece7a36 100644
--- a/bazel/grpc_build_system.bzl
+++ b/bazel/grpc_build_system.bzl
@@ -26,7 +26,7 @@
load("//bazel:cc_grpc_library.bzl", "cc_grpc_library")
# The set of pollers to test against if a test exercises polling
-POLLERS = ["epollex", "epoll1", "poll", "poll-cv"]
+POLLERS = ["epollex", "epoll1", "poll"]
def if_not_windows(a):
return select({
diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py
index 19a79ea41dd..f1e1f539ff9 100755
--- a/tools/run_tests/run_tests.py
+++ b/tools/run_tests/run_tests.py
@@ -345,15 +345,6 @@ class CLanguage(object):
# Scale overall test timeout if running under various sanitizers.
# scaling value is based on historical data analysis
timeout_scaling *= 3
- elif polling_strategy == 'poll-cv':
- # scale test timeout if running with poll-cv
- # sanitizer and poll-cv scaling is not cumulative to ensure
- # reasonable timeout values.
- # TODO(jtattermusch): based on historical data and 5min default
- # test timeout poll-cv scaling is currently not useful.
- # Leaving here so it can be reintroduced if the default test timeout
- # is decreased in the future.
- timeout_scaling *= 1
if self.config.build_config in target['exclude_configs']:
continue
From 9e102ea8b044813036d509551996f33d953e07b2 Mon Sep 17 00:00:00 2001
From: billfeng327 <yfen@google.com>
Date: Thu, 7 Mar 2019 16:14:02 -0800
Subject: [PATCH 062/138] excluded non-compatible test
---
test/cpp/microbenchmarks/BUILD | 1 +
1 file changed, 1 insertion(+)
diff --git a/test/cpp/microbenchmarks/BUILD b/test/cpp/microbenchmarks/BUILD
index 1ce71b1bb99..6e844a6dc62 100644
--- a/test/cpp/microbenchmarks/BUILD
+++ b/test/cpp/microbenchmarks/BUILD
@@ -69,6 +69,7 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_arena.cc"],
deps = [":helpers"],
+ tags = ["no_windows"],
)
grpc_cc_binary(
From 5c4823c17b302b92a456499e279370dcc378bada Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Thu, 7 Mar 2019 16:34:48 -0800
Subject: [PATCH 063/138] Build with bazel
---
examples/python/multiprocessing/BUILD | 7 ++++++-
examples/python/multiprocessing/server.py | 4 +++-
2 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/examples/python/multiprocessing/BUILD b/examples/python/multiprocessing/BUILD
index e47fb654aac..5fd32597355 100644
--- a/examples/python/multiprocessing/BUILD
+++ b/examples/python/multiprocessing/BUILD
@@ -7,6 +7,7 @@ py_binary(
deps = [
"//src/python/grpcio/grpc:grpcio"
],
+ default_python_version = "PY3",
)
py_binary(
@@ -15,7 +16,11 @@ py_binary(
srcs = ["server.py"],
deps = [
"//src/python/grpcio/grpc:grpcio"
- ],
+ ] + select({
+ "//conditions:default": [requirement("futures")],
+ "//:python3": [],
+ }),
+ default_python_version = "PY3",
)
py_test(
diff --git a/examples/python/multiprocessing/server.py b/examples/python/multiprocessing/server.py
index 6801f806126..5299589bb21 100644
--- a/examples/python/multiprocessing/server.py
+++ b/examples/python/multiprocessing/server.py
@@ -63,7 +63,7 @@ def _wait_forever(server):
def _run_server(bind_address):
"""Start a server in a subprocess."""
- logging.warning( '[PID {}] Starting new server.'.format( os.getpid()))
+ logging.warning( '[PID {}] Starting new server.'.format(os.getpid()))
options = (('grpc.so_reuseport', 1),)
# WARNING: This example takes advantage of SO_REUSEPORT. Due to the
@@ -87,6 +87,8 @@ def _reserve_port():
"""Find and reserve a port for all subprocesses to use."""
sock = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
+ if sock.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT) != 1:
+ raise RuntimeError("Failed to set SO_REUSEPORT.")
sock.bind(('', 0))
try:
yield sock.getsockname()[1]
From c619c49fab00c3348ee50da46f4a11d91ecee038 Mon Sep 17 00:00:00 2001
From: Jerry <jerrycuimh@gmail.com>
Date: Thu, 7 Mar 2019 17:31:34 -0800
Subject: [PATCH 064/138] fixed seg fault caused by access client after it is
closed
---
src/php/ext/grpc/call.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/src/php/ext/grpc/call.c b/src/php/ext/grpc/call.c
index a7942262987..668095cfbd2 100644
--- a/src/php/ext/grpc/call.c
+++ b/src/php/ext/grpc/call.c
@@ -217,6 +217,12 @@ PHP_METHOD(Call, __construct) {
}
wrapped_grpc_channel *channel =
PHP_GRPC_GET_WRAPPED_OBJECT(wrapped_grpc_channel, channel_obj);
+ if (channel->wrapper == NULL || channel->wrapper->wrapped == NULL) {
+ zend_throw_exception(spl_ce_InvalidArgumentException,
+ "Call cannot be constructed from a closed Channel",
+ 1 TSRMLS_CC);
+ return;
+ }
gpr_mu_lock(&channel->wrapper->mu);
if (channel->wrapper == NULL || channel->wrapper->wrapped == NULL) {
zend_throw_exception(spl_ce_InvalidArgumentException,
From 98fc9022005126897b809c5337944afadf27c619 Mon Sep 17 00:00:00 2001
From: Lidi Zheng <scallopsky@gmail.com>
Date: Thu, 7 Mar 2019 17:34:45 -0800
Subject: [PATCH 065/138] Revert "Roll foward "Strip Python wheel binary""
---
test/distrib/python/test_packages.sh | 2 +-
.../artifacts/build_package_python.sh | 21 -------------------
2 files changed, 1 insertion(+), 22 deletions(-)
diff --git a/test/distrib/python/test_packages.sh b/test/distrib/python/test_packages.sh
index 433148e6bd7..755daa10211 100755
--- a/test/distrib/python/test_packages.sh
+++ b/test/distrib/python/test_packages.sh
@@ -41,7 +41,7 @@ PYTHON=$VIRTUAL_ENV/bin/python
function at_least_one_installs() {
for file in "$@"; do
- if "$PYTHON" -m pip install --require-hashes "$file"; then
+ if "$PYTHON" -m pip install "$file"; then
return 0
fi
done
diff --git a/tools/run_tests/artifacts/build_package_python.sh b/tools/run_tests/artifacts/build_package_python.sh
index 29a26bc081c..29801a5b867 100755
--- a/tools/run_tests/artifacts/build_package_python.sh
+++ b/tools/run_tests/artifacts/build_package_python.sh
@@ -23,27 +23,6 @@ mkdir -p artifacts/
# and we only collect them here to deliver them to the distribtest phase.
cp -r "${EXTERNAL_GIT_ROOT}"/input_artifacts/python_*/* artifacts/ || true
-apt-get install -y python-pip
-python -m pip install wheel --user
-
-strip_binary_wheel() {
- WHEEL_PATH="$1"
- TEMP_WHEEL_DIR=$(mktemp -d)
- python -m wheel unpack "$WHEEL_PATH" -d "$TEMP_WHEEL_DIR"
- find "$TEMP_WHEEL_DIR" -name "_protoc_compiler*.so" -exec strip --strip-debug {} ";"
- find "$TEMP_WHEEL_DIR" -name "cygrpc*.so" -exec strip --strip-debug {} ";"
-
- WHEEL_FILE=$(basename "$WHEEL_PATH")
- DISTRIBUTION_NAME=$(basename "$WHEEL_PATH" | cut -d '-' -f 1)
- VERSION=$(basename "$WHEEL_PATH" | cut -d '-' -f 2)
- python -m wheel pack "$TEMP_WHEEL_DIR/$DISTRIBUTION_NAME-$VERSION" -d "$TEMP_WHEEL_DIR"
- mv "$TEMP_WHEEL_DIR/$WHEEL_FILE" "$WHEEL_PATH"
-}
-
-for wheel in artifacts/*.whl; do
- strip_binary_wheel "$wheel"
-done
-
# TODO: all the artifact builder configurations generate a grpcio-VERSION.tar.gz
# source distribution package, and only one of them will end up
# in the artifacts/ directory. They should be all equivalent though.
From 6b45cea2f0fa10ab44d5446105df4dff8be2d656 Mon Sep 17 00:00:00 2001
From: Yash Tibrewal <yashkt@google.com>
Date: Thu, 7 Mar 2019 17:56:49 -0800
Subject: [PATCH 066/138] Remove from poll-cv comments too
---
src/core/lib/iomgr/ev_poll_posix.cc | 2 +-
test/core/end2end/tests/keepalive_timeout.cc | 3 +--
test/cpp/end2end/BUILD | 2 +-
3 files changed, 3 insertions(+), 4 deletions(-)
diff --git a/src/core/lib/iomgr/ev_poll_posix.cc b/src/core/lib/iomgr/ev_poll_posix.cc
index 4c98ffa9448..29111dd44ed 100644
--- a/src/core/lib/iomgr/ev_poll_posix.cc
+++ b/src/core/lib/iomgr/ev_poll_posix.cc
@@ -125,7 +125,7 @@ struct grpc_fd {
grpc_fork_fd_list* fork_fd_list;
};
-/* True when GRPC_ENABLE_FORK_SUPPORT=1. We do not support fork with poll-cv */
+/* True when GRPC_ENABLE_FORK_SUPPORT=1. */
static bool track_fds_for_fork = false;
/* Only used when GRPC_ENABLE_FORK_SUPPORT=1 */
diff --git a/test/core/end2end/tests/keepalive_timeout.cc b/test/core/end2end/tests/keepalive_timeout.cc
index 2c992848b95..3c33f0419ad 100644
--- a/test/core/end2end/tests/keepalive_timeout.cc
+++ b/test/core/end2end/tests/keepalive_timeout.cc
@@ -226,8 +226,7 @@ static void test_keepalive_timeout(grpc_end2end_test_config config) {
* that the keepalive ping is never sent. */
static void test_read_delays_keepalive(grpc_end2end_test_config config) {
char* poller = gpr_getenv("GRPC_POLL_STRATEGY");
- /* It is hard to get the timing right for the polling engines poll and poll-cv
- */
+ /* It is hard to get the timing right for the polling engine poll. */
if (poller != nullptr && (0 == strcmp(poller, "poll"))) {
gpr_free(poller);
return;
diff --git a/test/cpp/end2end/BUILD b/test/cpp/end2end/BUILD
index 43dee177e7a..26095a41607 100644
--- a/test/cpp/end2end/BUILD
+++ b/test/cpp/end2end/BUILD
@@ -242,7 +242,7 @@ grpc_cc_test(
grpc_cc_test(
name = "end2end_test",
- size = "large", # with poll-cv this takes long, see #17493
+ size = "large",
deps = [
":end2end_test_lib",
],
From 37ba57ec48af127ad79efd2ccd26aa3cc0786b1a Mon Sep 17 00:00:00 2001
From: Jonas Vautherin <jonas.vautherin@gmail.com>
Date: Fri, 8 Mar 2019 13:44:30 +0100
Subject: [PATCH 067/138] Bugfix: cc_install following a cc_library must use
BUILD_CODEGEN, too
---
CMakeLists.txt | 6 ++++++
templates/CMakeLists.txt.template | 12 ++++++++++++
2 files changed, 18 insertions(+)
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 7ccda85b125..f504980e1ef 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -3828,6 +3828,7 @@ foreach(_hdr
endforeach()
endif (gRPC_BUILD_CODEGEN)
+if (gRPC_BUILD_CODEGEN)
if (gRPC_INSTALL)
install(TARGETS grpc++_error_details EXPORT gRPCTargets
@@ -3837,6 +3838,7 @@ if (gRPC_INSTALL)
)
endif()
+endif (gRPC_BUILD_CODEGEN)
if (gRPC_BUILD_TESTS)
if (gRPC_BUILD_CODEGEN)
@@ -3958,6 +3960,7 @@ foreach(_hdr
endforeach()
endif (gRPC_BUILD_CODEGEN)
+if (gRPC_BUILD_CODEGEN)
if (gRPC_INSTALL)
install(TARGETS grpc++_reflection EXPORT gRPCTargets
@@ -3967,6 +3970,7 @@ if (gRPC_INSTALL)
)
endif()
+endif (gRPC_BUILD_CODEGEN)
if (gRPC_BUILD_TESTS)
add_library(grpc++_test_config
@@ -4959,6 +4963,7 @@ foreach(_hdr
endforeach()
endif (gRPC_BUILD_CODEGEN)
+if (gRPC_BUILD_CODEGEN)
if (gRPC_INSTALL)
install(TARGETS grpcpp_channelz EXPORT gRPCTargets
@@ -4968,6 +4973,7 @@ if (gRPC_INSTALL)
)
endif()
+endif (gRPC_BUILD_CODEGEN)
if (gRPC_BUILD_TESTS)
if (gRPC_BUILD_CODEGEN)
diff --git a/templates/CMakeLists.txt.template b/templates/CMakeLists.txt.template
index e7fdfe5de55..dd996d736ac 100644
--- a/templates/CMakeLists.txt.template
+++ b/templates/CMakeLists.txt.template
@@ -331,12 +331,24 @@
% elif lib.name in ['grpc_csharp_ext']:
if (gRPC_BUILD_CSHARP_EXT)
${cc_library(lib)}
+ % if any(proto_re.match(src) for src in lib.src):
+ if (gRPC_BUILD_CODEGEN)
+ % endif
${cc_install(lib)}
+ % if any(proto_re.match(src) for src in lib.src):
+ endif (gRPC_BUILD_CODEGEN)
+ % endif
endif (gRPC_BUILD_CSHARP_EXT)
% else:
${cc_library(lib)}
% if not lib.build in ["tool"]:
+ % if any(proto_re.match(src) for src in lib.src):
+ if (gRPC_BUILD_CODEGEN)
+ % endif
${cc_install(lib)}
+ % if any(proto_re.match(src) for src in lib.src):
+ endif (gRPC_BUILD_CODEGEN)
+ % endif
% endif
% endif
% endif
From 919dc4cd2ce557308b372636307ccff8b1d9ab29 Mon Sep 17 00:00:00 2001
From: yang-g <yangg@google.com>
Date: Fri, 8 Mar 2019 10:04:59 -0800
Subject: [PATCH 068/138] Add comment
---
src/core/lib/gpr/cpu_posix.cc | 3 +++
1 file changed, 3 insertions(+)
diff --git a/src/core/lib/gpr/cpu_posix.cc b/src/core/lib/gpr/cpu_posix.cc
index 59f583e4a0d..982ccbd6ffe 100644
--- a/src/core/lib/gpr/cpu_posix.cc
+++ b/src/core/lib/gpr/cpu_posix.cc
@@ -70,6 +70,9 @@ unsigned gpr_cpu_current_cpu(void) {
unsigned int* thread_id =
static_cast<unsigned int*>(pthread_getspecific(thread_id_key));
if (thread_id == nullptr) {
+ // Note we cannot use gpr_malloc here because this allocation can happen in
+ // a main thread and will only be free'd when the main thread exits, which
+ // will cause our internal memory counters to believe it is a leak.
thread_id = static_cast<unsigned int*>(malloc(sizeof(unsigned int)));
pthread_setspecific(thread_id_key, thread_id);
}
From 127a6c1d9e3eb17b66929eedd1cfef73ebe72706 Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Fri, 8 Mar 2019 10:29:00 -0800
Subject: [PATCH 069/138] Revert "Revert "Roll foward "Strip Python wheel
binary"""
This reverts commit 98fc9022005126897b809c5337944afadf27c619.
---
test/distrib/python/test_packages.sh | 2 +-
.../artifacts/build_package_python.sh | 21 +++++++++++++++++++
2 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/test/distrib/python/test_packages.sh b/test/distrib/python/test_packages.sh
index 755daa10211..433148e6bd7 100755
--- a/test/distrib/python/test_packages.sh
+++ b/test/distrib/python/test_packages.sh
@@ -41,7 +41,7 @@ PYTHON=$VIRTUAL_ENV/bin/python
function at_least_one_installs() {
for file in "$@"; do
- if "$PYTHON" -m pip install "$file"; then
+ if "$PYTHON" -m pip install --require-hashes "$file"; then
return 0
fi
done
diff --git a/tools/run_tests/artifacts/build_package_python.sh b/tools/run_tests/artifacts/build_package_python.sh
index 29801a5b867..29a26bc081c 100755
--- a/tools/run_tests/artifacts/build_package_python.sh
+++ b/tools/run_tests/artifacts/build_package_python.sh
@@ -23,6 +23,27 @@ mkdir -p artifacts/
# and we only collect them here to deliver them to the distribtest phase.
cp -r "${EXTERNAL_GIT_ROOT}"/input_artifacts/python_*/* artifacts/ || true
+apt-get install -y python-pip
+python -m pip install wheel --user
+
+strip_binary_wheel() {
+ WHEEL_PATH="$1"
+ TEMP_WHEEL_DIR=$(mktemp -d)
+ python -m wheel unpack "$WHEEL_PATH" -d "$TEMP_WHEEL_DIR"
+ find "$TEMP_WHEEL_DIR" -name "_protoc_compiler*.so" -exec strip --strip-debug {} ";"
+ find "$TEMP_WHEEL_DIR" -name "cygrpc*.so" -exec strip --strip-debug {} ";"
+
+ WHEEL_FILE=$(basename "$WHEEL_PATH")
+ DISTRIBUTION_NAME=$(basename "$WHEEL_PATH" | cut -d '-' -f 1)
+ VERSION=$(basename "$WHEEL_PATH" | cut -d '-' -f 2)
+ python -m wheel pack "$TEMP_WHEEL_DIR/$DISTRIBUTION_NAME-$VERSION" -d "$TEMP_WHEEL_DIR"
+ mv "$TEMP_WHEEL_DIR/$WHEEL_FILE" "$WHEEL_PATH"
+}
+
+for wheel in artifacts/*.whl; do
+ strip_binary_wheel "$wheel"
+done
+
# TODO: all the artifact builder configurations generate a grpcio-VERSION.tar.gz
# source distribution package, and only one of them will end up
# in the artifacts/ directory. They should be all equivalent though.
From cb9cc5592e3259c60f8c7f7f3d30133a322ebaba Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Thu, 7 Mar 2019 17:59:39 -0800
Subject: [PATCH 070/138] Upgrade pip before using wheel
---
tools/run_tests/artifacts/build_package_python.sh | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tools/run_tests/artifacts/build_package_python.sh b/tools/run_tests/artifacts/build_package_python.sh
index 29a26bc081c..35c78e9a93e 100755
--- a/tools/run_tests/artifacts/build_package_python.sh
+++ b/tools/run_tests/artifacts/build_package_python.sh
@@ -24,7 +24,8 @@ mkdir -p artifacts/
cp -r "${EXTERNAL_GIT_ROOT}"/input_artifacts/python_*/* artifacts/ || true
apt-get install -y python-pip
-python -m pip install wheel --user
+python -m pip install -U pip
+python -m pip install -U wheel
strip_binary_wheel() {
WHEEL_PATH="$1"
From 5eb1e8d8c9e715e0b5fe30bfc80c2e2f5c3e3ed1 Mon Sep 17 00:00:00 2001
From: billfeng327 <yfen@google.com>
Date: Fri, 8 Mar 2019 10:39:03 -0800
Subject: [PATCH 071/138] code cleanup, removed old build file
---
bazel/grpc_deps.bzl | 1 -
third_party/BUILD | 1 -
third_party/benchmark.BUILD | 15 ---------------
tools/remote_build/windows.bazelrc | 2 +-
4 files changed, 1 insertion(+), 18 deletions(-)
delete mode 100644 third_party/benchmark.BUILD
diff --git a/bazel/grpc_deps.bzl b/bazel/grpc_deps.bzl
index 799e864484c..2795ce8e732 100644
--- a/bazel/grpc_deps.bzl
+++ b/bazel/grpc_deps.bzl
@@ -154,7 +154,6 @@ def grpc_deps():
if "com_github_google_benchmark" not in native.existing_rules():
http_archive(
name = "com_github_google_benchmark",
- #build_file = "@com_github_grpc_grpc//third_party:benchmark.BUILD",
strip_prefix = "benchmark-e776aa0275e293707b6a0901e0e8d8a8a3679508",
url = "https://github.com/google/benchmark/archive/e776aa0275e293707b6a0901e0e8d8a8a3679508.tar.gz",
)
diff --git a/third_party/BUILD b/third_party/BUILD
index 5ec919dc48d..8b43d6b8300 100644
--- a/third_party/BUILD
+++ b/third_party/BUILD
@@ -1,5 +1,4 @@
exports_files([
- "benchmark.BUILD",
"gtest.BUILD",
"objective_c/Cronet/bidirectional_stream_c.h",
"zlib.BUILD",
diff --git a/third_party/benchmark.BUILD b/third_party/benchmark.BUILD
deleted file mode 100644
index 4c622f32a84..00000000000
--- a/third_party/benchmark.BUILD
+++ /dev/null
@@ -1,15 +0,0 @@
-cc_library(
- name = "benchmark",
- srcs = glob(["src/*.cc"]),
- hdrs = glob(["include/**/*.h", "src/*.h"]),
- includes = [
- "include", "."
- ],
- copts = [
- "-DHAVE_POSIX_REGEX"
- ],
- linkstatic = 1,
- visibility = [
- "//visibility:public",
- ],
-)
diff --git a/tools/remote_build/windows.bazelrc b/tools/remote_build/windows.bazelrc
index 11f57cac32d..70575372d02 100644
--- a/tools/remote_build/windows.bazelrc
+++ b/tools/remote_build/windows.bazelrc
@@ -1,3 +1,3 @@
# TODO(yfen): Merge with rbe_common.bazelrc and enable Windows RBE
build --test_tag_filters=-no_windows
-build --build_tag_filters=-no_windows
\ No newline at end of file
+build --build_tag_filters=-no_windows
From fa7f01079517cf7075d3830eb8a207aacaa23ec6 Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Fri, 8 Mar 2019 10:47:04 -0800
Subject: [PATCH 072/138] Fix the hash checking mechanism
---
test/distrib/python/test_packages.sh | 19 ++++++++++++++++++-
1 file changed, 18 insertions(+), 1 deletion(-)
diff --git a/test/distrib/python/test_packages.sh b/test/distrib/python/test_packages.sh
index 433148e6bd7..4e1e6dbc94f 100755
--- a/test/distrib/python/test_packages.sh
+++ b/test/distrib/python/test_packages.sh
@@ -39,9 +39,16 @@ virtualenv "$VIRTUAL_ENV"
PYTHON=$VIRTUAL_ENV/bin/python
"$PYTHON" -m pip install --upgrade six pip
+function validate_wheel_hashes() {
+ for file in "$@"; do
+ "$PYTHON" -m wheel unpack "$file" --dest-dir /tmp || return 1
+ done
+ return 0
+}
+
function at_least_one_installs() {
for file in "$@"; do
- if "$PYTHON" -m pip install --require-hashes "$file"; then
+ if "$PYTHON" -m pip install "$file"; then
return 0
fi
done
@@ -49,6 +56,16 @@ function at_least_one_installs() {
}
+#
+# Validate the files in wheel matches their hashes and size in RECORD
+#
+
+if [[ "$1" == "binary" ]]; then
+ validate_wheel_hashes "${ARCHIVES[@]}"
+ validate_wheel_hashes "${TOOLS_ARCHIVES[@]}"
+fi
+
+
#
# Install our distributions in order of dependencies
#
From e2aef4d373089d0068666cc87502c18449358831 Mon Sep 17 00:00:00 2001
From: Jerry <jerrycuimh@gmail.com>
Date: Fri, 8 Mar 2019 10:48:56 -0800
Subject: [PATCH 073/138] fixed seg fault
---
src/php/ext/grpc/call.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/php/ext/grpc/call.c b/src/php/ext/grpc/call.c
index 668095cfbd2..46d1e22f1f8 100644
--- a/src/php/ext/grpc/call.c
+++ b/src/php/ext/grpc/call.c
@@ -217,7 +217,7 @@ PHP_METHOD(Call, __construct) {
}
wrapped_grpc_channel *channel =
PHP_GRPC_GET_WRAPPED_OBJECT(wrapped_grpc_channel, channel_obj);
- if (channel->wrapper == NULL || channel->wrapper->wrapped == NULL) {
+ if (channel->wrapper == NULL) {
zend_throw_exception(spl_ce_InvalidArgumentException,
"Call cannot be constructed from a closed Channel",
1 TSRMLS_CC);
From acbc095ab8f29957bec0c1da6d0a613f4378101d Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 8 Mar 2019 13:40:44 -0800
Subject: [PATCH 074/138] Implement test for example
---
examples/python/multiprocessing/BUILD | 5 +-
examples/python/multiprocessing/README.md | 3 +
examples/python/multiprocessing/client.py | 23 ++++--
examples/python/multiprocessing/server.py | 20 +++--
.../test/_multiprocessing_example_test.py | 74 +++++++++++++++++++
.../test/_multiprocessing_test.py | 0
6 files changed, 113 insertions(+), 12 deletions(-)
create mode 100644 examples/python/multiprocessing/test/_multiprocessing_example_test.py
delete mode 100644 examples/python/multiprocessing/test/_multiprocessing_test.py
diff --git a/examples/python/multiprocessing/BUILD b/examples/python/multiprocessing/BUILD
index 5fd32597355..48b98f3ad20 100644
--- a/examples/python/multiprocessing/BUILD
+++ b/examples/python/multiprocessing/BUILD
@@ -24,10 +24,11 @@ py_binary(
)
py_test(
- name = "_multiprocessing_example_test",
+ name = "test/_multiprocessing_example_test",
srcs = ["test/_multiprocessing_example_test.py"],
data = [
":client",
":server"
- ]
+ ],
+ size = "small",
)
diff --git a/examples/python/multiprocessing/README.md b/examples/python/multiprocessing/README.md
index e69de29bb2d..da0c411dc09 100644
--- a/examples/python/multiprocessing/README.md
+++ b/examples/python/multiprocessing/README.md
@@ -0,0 +1,3 @@
+TODO: Describe the example.
+TODO: Describe how to run the example.
+TODO: Describe how to run the test.
diff --git a/examples/python/multiprocessing/client.py b/examples/python/multiprocessing/client.py
index 920b5285475..10233e00bc0 100644
--- a/examples/python/multiprocessing/client.py
+++ b/examples/python/multiprocessing/client.py
@@ -25,6 +25,7 @@ import multiprocessing
import operator
import os
import time
+import sys
import prime_pb2
import prime_pb2_grpc
@@ -36,11 +37,13 @@ _MAXIMUM_CANDIDATE = 10000
_worker_channel_singleton = None
_worker_stub_singleton = None
+_LOGGER = logging.getLogger(__name__)
+
def _initialize_worker(server_address):
global _worker_channel_singleton
global _worker_stub_singleton
- logging.warning('[PID {}] Initializing worker process.'.format(
+ _LOGGER.info('[PID {}] Initializing worker process.'.format(
os.getpid()))
_worker_channel_singleton = grpc.insecure_channel(server_address)
_worker_stub_singleton = prime_pb2_grpc.PrimeCheckerStub(
@@ -49,25 +52,26 @@ def _initialize_worker(server_address):
def _shutdown_worker():
- logging.warning('[PID {}] Shutting worker process down.'.format(
+ _LOGGER.info('[PID {}] Shutting worker process down.'.format(
os.getpid()))
if _worker_channel_singleton is not None:
_worker_channel_singleton.stop()
def _run_worker_query(primality_candidate):
- logging.warning('[PID {}] Checking primality of {}.'.format(
+ _LOGGER.info('[PID {}] Checking primality of {}.'.format(
os.getpid(), primality_candidate))
return _worker_stub_singleton.check(
prime_pb2.PrimeCandidate(candidate=primality_candidate))
+
def _calculate_primes(server_address):
worker_pool = multiprocessing.Pool(processes=_PROCESS_COUNT,
initializer=_initialize_worker, initargs=(server_address,))
check_range = range(2, _MAXIMUM_CANDIDATE)
primality = worker_pool.map(_run_worker_query, check_range)
primes = zip(check_range, map(operator.attrgetter('isPrime'), primality))
- logging.warning(tuple(primes))
+ _LOGGER.info(tuple(primes))
def main():
@@ -77,7 +81,16 @@ def main():
parser.add_argument('server_address', help='The address of the server (e.g. localhost:50051)')
args = parser.parse_args()
_calculate_primes(args.server_address)
+ sys.stdout.flush()
+
if __name__ == '__main__':
- logging.basicConfig()
+ # TODO(rbellevi): Add PID to formatter
+ fh = logging.FileHandler('/tmp/client.log')
+ fh.setLevel(logging.INFO)
+ ch = logging.StreamHandler(sys.stdout)
+ ch.setLevel(logging.INFO)
+ _LOGGER.addHandler(fh)
+ _LOGGER.addHandler(ch)
+ _LOGGER.setLevel(logging.INFO)
main()
diff --git a/examples/python/multiprocessing/server.py b/examples/python/multiprocessing/server.py
index 5299589bb21..c45bbadecf2 100644
--- a/examples/python/multiprocessing/server.py
+++ b/examples/python/multiprocessing/server.py
@@ -27,10 +27,13 @@ import multiprocessing
import os
import time
import socket
+import sys
import prime_pb2
import prime_pb2_grpc
+_LOGGER = logging.getLogger(__name__)
+
_ONE_DAY = datetime.timedelta(days=1)
_PROCESS_COUNT = 8
_THREAD_CONCURRENCY = 10
@@ -47,7 +50,7 @@ def is_prime(n):
class PrimeChecker(prime_pb2_grpc.PrimeCheckerServicer):
def check(self, request, context):
- logging.warning(
+ _LOGGER.info(
'[PID {}] Determining primality of {}'.format(
os.getpid(), request.candidate))
return prime_pb2.Primality(isPrime=is_prime(request.candidate))
@@ -63,7 +66,7 @@ def _wait_forever(server):
def _run_server(bind_address):
"""Start a server in a subprocess."""
- logging.warning( '[PID {}] Starting new server.'.format(os.getpid()))
+ _LOGGER.info( '[PID {}] Starting new server.'.format(os.getpid()))
options = (('grpc.so_reuseport', 1),)
# WARNING: This example takes advantage of SO_REUSEPORT. Due to the
@@ -99,7 +102,8 @@ def _reserve_port():
def main():
with _reserve_port() as port:
bind_address = '[::]:{}'.format(port)
- logging.warning("Binding to {}".format(bind_address))
+ _LOGGER.info("Binding to '{}'".format(bind_address))
+ sys.stdout.flush()
workers = []
for _ in range(_PROCESS_COUNT):
# NOTE: It is imperative that the worker subprocesses be forked before
@@ -111,7 +115,13 @@ def main():
for worker in workers:
worker.join()
-
if __name__ == '__main__':
- logging.basicConfig()
+ # TODO(rbellevi): Add PID to formatter
+ fh = logging.FileHandler('/tmp/server.log')
+ fh.setLevel(logging.INFO)
+ ch = logging.StreamHandler(sys.stdout)
+ ch.setLevel(logging.INFO)
+ _LOGGER.addHandler(fh)
+ _LOGGER.addHandler(ch)
+ _LOGGER.setLevel(logging.INFO)
main()
diff --git a/examples/python/multiprocessing/test/_multiprocessing_example_test.py b/examples/python/multiprocessing/test/_multiprocessing_example_test.py
new file mode 100644
index 00000000000..5e8b31f19c0
--- /dev/null
+++ b/examples/python/multiprocessing/test/_multiprocessing_example_test.py
@@ -0,0 +1,74 @@
+# Copyright 2019 the gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Test for multiprocessing example."""
+
+import datetime
+import logging
+import math
+import os
+import re
+import subprocess
+import tempfile
+import time
+import unittest
+
+_BINARY_DIR = os.path.realpath(
+ os.path.join(
+ os.path.dirname(os.path.abspath(__file__)), '..'))
+_SERVER_PATH = os.path.join(_BINARY_DIR, 'server')
+_CLIENT_PATH = os.path.join(_BINARY_DIR, 'client')
+
+
+def is_prime(n):
+ for i in range(2, int(math.ceil(math.sqrt(n)))):
+ if n % i == 0:
+ return False
+ else:
+ return True
+
+
+def _get_server_address(server_stream):
+ while True:
+ server_stream.seek(0)
+ line = server_stream.readline()
+ while line:
+ matches = re.search('Binding to \'(.+)\'', line)
+ if matches is not None:
+ return matches.groups()[0]
+ line = server_stream.readline()
+
+
+class MultiprocessingExampleTest(unittest.TestCase):
+
+ def test_multiprocessing_example(self):
+ server_stdout = tempfile.TemporaryFile(mode='r')
+ server_process = subprocess.Popen((_SERVER_PATH,),
+ stdout=server_stdout)
+ server_address = _get_server_address(server_stdout)
+ client_stdout = tempfile.TemporaryFile(mode='r')
+ client_process = subprocess.Popen((_CLIENT_PATH, server_address,),
+ stdout=client_stdout)
+ client_process.wait()
+ server_process.terminate()
+ client_stdout.seek(0)
+ results = eval(client_stdout.read().strip().split('\n')[-1])
+ values = tuple(result[0] for result in results)
+ self.assertSequenceEqual(range(2, 10000), values)
+ for result in results:
+ self.assertEqual(is_prime(result[0]), result[1])
+
+
+if __name__ == '__main__':
+ logging.basicConfig()
+ unittest.main(verbosity=2)
diff --git a/examples/python/multiprocessing/test/_multiprocessing_test.py b/examples/python/multiprocessing/test/_multiprocessing_test.py
deleted file mode 100644
index e69de29bb2d..00000000000
From 0f6293e85e8a820fd3436716d41b2d13550a7c08 Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 8 Mar 2019 13:50:17 -0800
Subject: [PATCH 075/138] Improve logging handlers
---
examples/python/multiprocessing/client.py | 29 ++++++++++-------------
examples/python/multiprocessing/server.py | 17 ++++++-------
2 files changed, 20 insertions(+), 26 deletions(-)
diff --git a/examples/python/multiprocessing/client.py b/examples/python/multiprocessing/client.py
index 10233e00bc0..9c47080a1ff 100644
--- a/examples/python/multiprocessing/client.py
+++ b/examples/python/multiprocessing/client.py
@@ -43,8 +43,7 @@ _LOGGER = logging.getLogger(__name__)
def _initialize_worker(server_address):
global _worker_channel_singleton
global _worker_stub_singleton
- _LOGGER.info('[PID {}] Initializing worker process.'.format(
- os.getpid()))
+ _LOGGER.info('Initializing worker process.')
_worker_channel_singleton = grpc.insecure_channel(server_address)
_worker_stub_singleton = prime_pb2_grpc.PrimeCheckerStub(
_worker_channel_singleton)
@@ -52,15 +51,14 @@ def _initialize_worker(server_address):
def _shutdown_worker():
- _LOGGER.info('[PID {}] Shutting worker process down.'.format(
- os.getpid()))
+ _LOGGER.info('Shutting worker process down.')
if _worker_channel_singleton is not None:
_worker_channel_singleton.stop()
def _run_worker_query(primality_candidate):
- _LOGGER.info('[PID {}] Checking primality of {}.'.format(
- os.getpid(), primality_candidate))
+ _LOGGER.info('Checking primality of {}.'.format(
+ primality_candidate))
return _worker_stub_singleton.check(
prime_pb2.PrimeCandidate(candidate=primality_candidate))
@@ -71,26 +69,25 @@ def _calculate_primes(server_address):
check_range = range(2, _MAXIMUM_CANDIDATE)
primality = worker_pool.map(_run_worker_query, check_range)
primes = zip(check_range, map(operator.attrgetter('isPrime'), primality))
- _LOGGER.info(tuple(primes))
+ return tuple(primes)
def main():
msg = 'Determine the primality of the first {} integers.'.format(
_MAXIMUM_CANDIDATE)
parser = argparse.ArgumentParser(description=msg)
- parser.add_argument('server_address', help='The address of the server (e.g. localhost:50051)')
+ parser.add_argument('server_address',
+ help='The address of the server (e.g. localhost:50051)')
args = parser.parse_args()
- _calculate_primes(args.server_address)
+ primes = _calculate_primes(args.server_address)
+ print(primes)
sys.stdout.flush()
if __name__ == '__main__':
- # TODO(rbellevi): Add PID to formatter
- fh = logging.FileHandler('/tmp/client.log')
- fh.setLevel(logging.INFO)
- ch = logging.StreamHandler(sys.stdout)
- ch.setLevel(logging.INFO)
- _LOGGER.addHandler(fh)
- _LOGGER.addHandler(ch)
+ handler = logging.StreamHandler(sys.stdout)
+ formatter = logging.Formatter('[PID %(process)d] %(message)s')
+ handler.setFormatter(formatter)
+ _LOGGER.addHandler(handler)
_LOGGER.setLevel(logging.INFO)
main()
diff --git a/examples/python/multiprocessing/server.py b/examples/python/multiprocessing/server.py
index c45bbadecf2..da97d5d8252 100644
--- a/examples/python/multiprocessing/server.py
+++ b/examples/python/multiprocessing/server.py
@@ -51,8 +51,8 @@ class PrimeChecker(prime_pb2_grpc.PrimeCheckerServicer):
def check(self, request, context):
_LOGGER.info(
- '[PID {}] Determining primality of {}'.format(
- os.getpid(), request.candidate))
+ 'Determining primality of {}'.format(
+ request.candidate))
return prime_pb2.Primality(isPrime=is_prime(request.candidate))
@@ -66,7 +66,7 @@ def _wait_forever(server):
def _run_server(bind_address):
"""Start a server in a subprocess."""
- _LOGGER.info( '[PID {}] Starting new server.'.format(os.getpid()))
+ _LOGGER.info('Starting new server.')
options = (('grpc.so_reuseport', 1),)
# WARNING: This example takes advantage of SO_REUSEPORT. Due to the
@@ -116,12 +116,9 @@ def main():
worker.join()
if __name__ == '__main__':
- # TODO(rbellevi): Add PID to formatter
- fh = logging.FileHandler('/tmp/server.log')
- fh.setLevel(logging.INFO)
- ch = logging.StreamHandler(sys.stdout)
- ch.setLevel(logging.INFO)
- _LOGGER.addHandler(fh)
- _LOGGER.addHandler(ch)
+ handler = logging.StreamHandler(sys.stdout)
+ formatter = logging.Formatter('[PID %(process)d] %(message)s')
+ handler.setFormatter(formatter)
+ _LOGGER.addHandler(handler)
_LOGGER.setLevel(logging.INFO)
main()
From 67ca10b4f96ddab82544e128de55b664a56a527a Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 8 Mar 2019 15:06:02 -0800
Subject: [PATCH 076/138] Add README
---
examples/python/multiprocessing/README.md | 54 ++++++-
examples/python/multiprocessing/prime_pb2.py | 132 ------------------
.../python/multiprocessing/prime_pb2_grpc.py | 46 ------
3 files changed, 51 insertions(+), 181 deletions(-)
delete mode 100644 examples/python/multiprocessing/prime_pb2.py
delete mode 100644 examples/python/multiprocessing/prime_pb2_grpc.py
diff --git a/examples/python/multiprocessing/README.md b/examples/python/multiprocessing/README.md
index da0c411dc09..6dcec8b2132 100644
--- a/examples/python/multiprocessing/README.md
+++ b/examples/python/multiprocessing/README.md
@@ -1,3 +1,51 @@
-TODO: Describe the example.
-TODO: Describe how to run the example.
-TODO: Describe how to run the test.
+## Multiprocessing with gRPC Python
+
+Multiprocessing allows application developers to sidestep the Python global
+interpreter lock and achieve true concurrency on multicore systems.
+Unfortunately, using multiprocessing and gRPC Python is not yet as simple as
+instantiating your server with a `futures.ProcessPoolExecutor`.
+
+The library is implemented as a C extension, maintaining much of the state that
+drives the system in native code. As such, upon calling
+[`fork`](http://man7.org/linux/man-pages/man2/fork.2.html), much of the
+state copied into the child process is invalid, leading to hangs and crashes.
+
+However, calling `fork` without `exec` in your python process is supported
+*before* any gRPC servers have been instantiated. Application developers can
+take advantage of this to parallelize their CPU-intensive operations.
+
+## Running the Example
+
+This example calculates the first 10,000 prime numbers as an RPC. We instantiate
+one server per subprocess, balancing requests between the servers using the
+[`SO_REUSEPORT`](https://lwn.net/Articles/542629/) socket option.
+
+To run the server,
+[ensure `bazel` is installed](https://docs.bazel.build/versions/master/install.html)
+and run:
+
+```
+bazel run //examples/python/multiprocessing:server &
+```
+
+Note the address at which the server is running. For example,
+
+```
+...
+[PID 107153] Binding to '[::]:33915'
+[PID 107507] Starting new server.
+[PID 107508] Starting new server.
+...
+```
+
+Now, start the client by running
+
+```
+bazel run //examples/python/multiprocessing:client -- [SERVER_ADDRESS]
+```
+
+For example,
+
+```
+bazel run //examples/python/multiprocessing:client -- [::]:33915
+```
diff --git a/examples/python/multiprocessing/prime_pb2.py b/examples/python/multiprocessing/prime_pb2.py
deleted file mode 100644
index 58e6e6a023a..00000000000
--- a/examples/python/multiprocessing/prime_pb2.py
+++ /dev/null
@@ -1,132 +0,0 @@
-# Generated by the protocol buffer compiler. DO NOT EDIT!
-# source: prime.proto
-
-import sys
-_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
-from google.protobuf import descriptor as _descriptor
-from google.protobuf import message as _message
-from google.protobuf import reflection as _reflection
-from google.protobuf import symbol_database as _symbol_database
-# @@protoc_insertion_point(imports)
-
-_sym_db = _symbol_database.Default()
-
-
-
-
-DESCRIPTOR = _descriptor.FileDescriptor(
- name='prime.proto',
- package='prime',
- syntax='proto3',
- serialized_options=None,
- serialized_pb=_b('\n\x0bprime.proto\x12\x05prime\"#\n\x0ePrimeCandidate\x12\x11\n\tcandidate\x18\x01 \x01(\x03\"\x1c\n\tPrimality\x12\x0f\n\x07isPrime\x18\x01 \x01(\x08\x32\x42\n\x0cPrimeChecker\x12\x32\n\x05\x63heck\x12\x15.prime.PrimeCandidate\x1a\x10.prime.Primality\"\x00\x62\x06proto3')
-)
-
-
-
-
-_PRIMECANDIDATE = _descriptor.Descriptor(
- name='PrimeCandidate',
- full_name='prime.PrimeCandidate',
- filename=None,
- file=DESCRIPTOR,
- containing_type=None,
- fields=[
- _descriptor.FieldDescriptor(
- name='candidate', full_name='prime.PrimeCandidate.candidate', index=0,
- number=1, type=3, cpp_type=2, label=1,
- has_default_value=False, default_value=0,
- message_type=None, enum_type=None, containing_type=None,
- is_extension=False, extension_scope=None,
- serialized_options=None, file=DESCRIPTOR),
- ],
- extensions=[
- ],
- nested_types=[],
- enum_types=[
- ],
- serialized_options=None,
- is_extendable=False,
- syntax='proto3',
- extension_ranges=[],
- oneofs=[
- ],
- serialized_start=22,
- serialized_end=57,
-)
-
-
-_PRIMALITY = _descriptor.Descriptor(
- name='Primality',
- full_name='prime.Primality',
- filename=None,
- file=DESCRIPTOR,
- containing_type=None,
- fields=[
- _descriptor.FieldDescriptor(
- name='isPrime', full_name='prime.Primality.isPrime', index=0,
- number=1, type=8, cpp_type=7, label=1,
- has_default_value=False, default_value=False,
- message_type=None, enum_type=None, containing_type=None,
- is_extension=False, extension_scope=None,
- serialized_options=None, file=DESCRIPTOR),
- ],
- extensions=[
- ],
- nested_types=[],
- enum_types=[
- ],
- serialized_options=None,
- is_extendable=False,
- syntax='proto3',
- extension_ranges=[],
- oneofs=[
- ],
- serialized_start=59,
- serialized_end=87,
-)
-
-DESCRIPTOR.message_types_by_name['PrimeCandidate'] = _PRIMECANDIDATE
-DESCRIPTOR.message_types_by_name['Primality'] = _PRIMALITY
-_sym_db.RegisterFileDescriptor(DESCRIPTOR)
-
-PrimeCandidate = _reflection.GeneratedProtocolMessageType('PrimeCandidate', (_message.Message,), dict(
- DESCRIPTOR = _PRIMECANDIDATE,
- __module__ = 'prime_pb2'
- # @@protoc_insertion_point(class_scope:prime.PrimeCandidate)
- ))
-_sym_db.RegisterMessage(PrimeCandidate)
-
-Primality = _reflection.GeneratedProtocolMessageType('Primality', (_message.Message,), dict(
- DESCRIPTOR = _PRIMALITY,
- __module__ = 'prime_pb2'
- # @@protoc_insertion_point(class_scope:prime.Primality)
- ))
-_sym_db.RegisterMessage(Primality)
-
-
-
-_PRIMECHECKER = _descriptor.ServiceDescriptor(
- name='PrimeChecker',
- full_name='prime.PrimeChecker',
- file=DESCRIPTOR,
- index=0,
- serialized_options=None,
- serialized_start=89,
- serialized_end=155,
- methods=[
- _descriptor.MethodDescriptor(
- name='check',
- full_name='prime.PrimeChecker.check',
- index=0,
- containing_service=None,
- input_type=_PRIMECANDIDATE,
- output_type=_PRIMALITY,
- serialized_options=None,
- ),
-])
-_sym_db.RegisterServiceDescriptor(_PRIMECHECKER)
-
-DESCRIPTOR.services_by_name['PrimeChecker'] = _PRIMECHECKER
-
-# @@protoc_insertion_point(module_scope)
diff --git a/examples/python/multiprocessing/prime_pb2_grpc.py b/examples/python/multiprocessing/prime_pb2_grpc.py
deleted file mode 100644
index dcc3a35706d..00000000000
--- a/examples/python/multiprocessing/prime_pb2_grpc.py
+++ /dev/null
@@ -1,46 +0,0 @@
-# Generated by the gRPC Python protocol compiler plugin. DO NOT EDIT!
-import grpc
-
-import prime_pb2 as prime__pb2
-
-
-class PrimeCheckerStub(object):
- """Service to check primality.
- """
-
- def __init__(self, channel):
- """Constructor.
-
- Args:
- channel: A grpc.Channel.
- """
- self.check = channel.unary_unary(
- '/prime.PrimeChecker/check',
- request_serializer=prime__pb2.PrimeCandidate.SerializeToString,
- response_deserializer=prime__pb2.Primality.FromString,
- )
-
-
-class PrimeCheckerServicer(object):
- """Service to check primality.
- """
-
- def check(self, request, context):
- """Determines the primality of an integer.
- """
- context.set_code(grpc.StatusCode.UNIMPLEMENTED)
- context.set_details('Method not implemented!')
- raise NotImplementedError('Method not implemented!')
-
-
-def add_PrimeCheckerServicer_to_server(servicer, server):
- rpc_method_handlers = {
- 'check': grpc.unary_unary_rpc_method_handler(
- servicer.check,
- request_deserializer=prime__pb2.PrimeCandidate.FromString,
- response_serializer=prime__pb2.Primality.SerializeToString,
- ),
- }
- generic_handler = grpc.method_handlers_generic_handler(
- 'prime.PrimeChecker', rpc_method_handlers)
- server.add_generic_rpc_handlers((generic_handler,))
From 23c5fb8ca4d3830c6360cafb809f432d7f0c2b49 Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 8 Mar 2019 15:14:42 -0800
Subject: [PATCH 077/138] Add example tests to CI
---
tools/internal_ci/linux/grpc_python_bazel_test_in_docker.sh | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tools/internal_ci/linux/grpc_python_bazel_test_in_docker.sh b/tools/internal_ci/linux/grpc_python_bazel_test_in_docker.sh
index 14989648a2a..d844cff7f9a 100755
--- a/tools/internal_ci/linux/grpc_python_bazel_test_in_docker.sh
+++ b/tools/internal_ci/linux/grpc_python_bazel_test_in_docker.sh
@@ -25,5 +25,7 @@ git clone /var/local/jenkins/grpc /var/local/git/grpc
${name}')
cd /var/local/git/grpc/test
bazel test --spawn_strategy=standalone --genrule_strategy=standalone --test_output=errors //src/python/...
+bazel test --spawn_strategy=standalone --genrule_strategy=standalone --test_output=errors //examples/python/...
bazel clean --expunge
bazel test --config=python3 --spawn_strategy=standalone --genrule_strategy=standalone --test_output=errors //src/python/...
+bazel test --config=python3 --spawn_strategy=standalone --genrule_strategy=standalone --test_output=errors //examples/python/...
From d832738c08bb076ee27a91b7da61051e24d5e902 Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 8 Mar 2019 15:16:12 -0800
Subject: [PATCH 078/138] Yapf
---
examples/python/multiprocessing/client.py | 20 ++++++++++---------
examples/python/multiprocessing/server.py | 13 ++++++------
.../test/_multiprocessing_example_test.py | 13 ++++++------
3 files changed, 24 insertions(+), 22 deletions(-)
diff --git a/examples/python/multiprocessing/client.py b/examples/python/multiprocessing/client.py
index 9c47080a1ff..fa3b394f394 100644
--- a/examples/python/multiprocessing/client.py
+++ b/examples/python/multiprocessing/client.py
@@ -46,7 +46,7 @@ def _initialize_worker(server_address):
_LOGGER.info('Initializing worker process.')
_worker_channel_singleton = grpc.insecure_channel(server_address)
_worker_stub_singleton = prime_pb2_grpc.PrimeCheckerStub(
- _worker_channel_singleton)
+ _worker_channel_singleton)
atexit.register(_shutdown_worker)
@@ -57,15 +57,16 @@ def _shutdown_worker():
def _run_worker_query(primality_candidate):
- _LOGGER.info('Checking primality of {}.'.format(
- primality_candidate))
+ _LOGGER.info('Checking primality of {}.'.format(primality_candidate))
return _worker_stub_singleton.check(
- prime_pb2.PrimeCandidate(candidate=primality_candidate))
+ prime_pb2.PrimeCandidate(candidate=primality_candidate))
def _calculate_primes(server_address):
- worker_pool = multiprocessing.Pool(processes=_PROCESS_COUNT,
- initializer=_initialize_worker, initargs=(server_address,))
+ worker_pool = multiprocessing.Pool(
+ processes=_PROCESS_COUNT,
+ initializer=_initialize_worker,
+ initargs=(server_address,))
check_range = range(2, _MAXIMUM_CANDIDATE)
primality = worker_pool.map(_run_worker_query, check_range)
primes = zip(check_range, map(operator.attrgetter('isPrime'), primality))
@@ -74,10 +75,11 @@ def _calculate_primes(server_address):
def main():
msg = 'Determine the primality of the first {} integers.'.format(
- _MAXIMUM_CANDIDATE)
+ _MAXIMUM_CANDIDATE)
parser = argparse.ArgumentParser(description=msg)
- parser.add_argument('server_address',
- help='The address of the server (e.g. localhost:50051)')
+ parser.add_argument(
+ 'server_address',
+ help='The address of the server (e.g. localhost:50051)')
args = parser.parse_args()
primes = _calculate_primes(args.server_address)
print(primes)
diff --git a/examples/python/multiprocessing/server.py b/examples/python/multiprocessing/server.py
index da97d5d8252..588cd4734e9 100644
--- a/examples/python/multiprocessing/server.py
+++ b/examples/python/multiprocessing/server.py
@@ -50,9 +50,7 @@ def is_prime(n):
class PrimeChecker(prime_pb2_grpc.PrimeCheckerServicer):
def check(self, request, context):
- _LOGGER.info(
- 'Determining primality of {}'.format(
- request.candidate))
+ _LOGGER.info('Determining primality of {}'.format(request.candidate))
return prime_pb2.Primality(isPrime=is_prime(request.candidate))
@@ -76,9 +74,8 @@ def _run_server(bind_address):
# `pip install grpcio --no-binary grpcio`.
server = grpc.server(
- futures.ThreadPoolExecutor(
- max_workers=_THREAD_CONCURRENCY,),
- options=options)
+ futures.ThreadPoolExecutor(max_workers=_THREAD_CONCURRENCY,),
+ options=options)
prime_pb2_grpc.add_PrimeCheckerServicer_to_server(PrimeChecker(), server)
server.add_insecure_port(bind_address)
server.start()
@@ -109,12 +106,14 @@ def main():
# NOTE: It is imperative that the worker subprocesses be forked before
# any gRPC servers start up. See
# https://github.com/grpc/grpc/issues/16001 for more details.
- worker = multiprocessing.Process(target=_run_server, args=(bind_address,))
+ worker = multiprocessing.Process(
+ target=_run_server, args=(bind_address,))
worker.start()
workers.append(worker)
for worker in workers:
worker.join()
+
if __name__ == '__main__':
handler = logging.StreamHandler(sys.stdout)
formatter = logging.Formatter('[PID %(process)d] %(message)s')
diff --git a/examples/python/multiprocessing/test/_multiprocessing_example_test.py b/examples/python/multiprocessing/test/_multiprocessing_example_test.py
index 5e8b31f19c0..92e7c0a4b8d 100644
--- a/examples/python/multiprocessing/test/_multiprocessing_example_test.py
+++ b/examples/python/multiprocessing/test/_multiprocessing_example_test.py
@@ -24,8 +24,7 @@ import time
import unittest
_BINARY_DIR = os.path.realpath(
- os.path.join(
- os.path.dirname(os.path.abspath(__file__)), '..'))
+ os.path.join(os.path.dirname(os.path.abspath(__file__)), '..'))
_SERVER_PATH = os.path.join(_BINARY_DIR, 'server')
_CLIENT_PATH = os.path.join(_BINARY_DIR, 'client')
@@ -53,12 +52,14 @@ class MultiprocessingExampleTest(unittest.TestCase):
def test_multiprocessing_example(self):
server_stdout = tempfile.TemporaryFile(mode='r')
- server_process = subprocess.Popen((_SERVER_PATH,),
- stdout=server_stdout)
+ server_process = subprocess.Popen((_SERVER_PATH,), stdout=server_stdout)
server_address = _get_server_address(server_stdout)
client_stdout = tempfile.TemporaryFile(mode='r')
- client_process = subprocess.Popen((_CLIENT_PATH, server_address,),
- stdout=client_stdout)
+ client_process = subprocess.Popen(
+ (
+ _CLIENT_PATH,
+ server_address,
+ ), stdout=client_stdout)
client_process.wait()
server_process.terminate()
client_stdout.seek(0)
From 749c52de56e4d48c495404abb966274454ab36b3 Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 8 Mar 2019 15:31:37 -0800
Subject: [PATCH 079/138] Refcount vtables
---
src/python/grpcio/grpc/_cython/BUILD.bazel | 4 +-
.../grpc/_cython/_cygrpc/arguments.pxd.pxi | 15 +------
.../grpc/_cython/_cygrpc/arguments.pyx.pxi | 34 ++--------------
.../grpcio/grpc/_cython/_cygrpc/call.pyx.pxi | 4 +-
.../grpc/_cython/_cygrpc/channel.pxd.pxi | 2 +-
.../grpc/_cython/_cygrpc/channel.pyx.pxi | 18 ++++-----
.../grpc/_cython/_cygrpc/server.pxd.pxi | 4 +-
.../grpc/_cython/_cygrpc/server.pyx.pxi | 15 ++++---
.../grpcio/grpc/_cython/_cygrpc/tag.pxd.pxi | 1 +
.../grpcio/grpc/_cython/_cygrpc/tag.pyx.pxi | 5 ++-
.../grpc/_cython/_cygrpc/vtable.pxd.pxi | 26 +++++++++++++
.../grpc/_cython/_cygrpc/vtable.pyx.pxi | 39 +++++++++++++++++++
src/python/grpcio/grpc/_cython/cygrpc.pxd | 3 +-
src/python/grpcio/grpc/_cython/cygrpc.pyx | 5 ++-
14 files changed, 101 insertions(+), 74 deletions(-)
create mode 100644 src/python/grpcio/grpc/_cython/_cygrpc/vtable.pxd.pxi
create mode 100644 src/python/grpcio/grpc/_cython/_cygrpc/vtable.pyx.pxi
diff --git a/src/python/grpcio/grpc/_cython/BUILD.bazel b/src/python/grpcio/grpc/_cython/BUILD.bazel
index 42db7b87213..18b1c92b9a7 100644
--- a/src/python/grpcio/grpc/_cython/BUILD.bazel
+++ b/src/python/grpcio/grpc/_cython/BUILD.bazel
@@ -43,10 +43,12 @@ pyx_library(
"_cygrpc/tag.pyx.pxi",
"_cygrpc/time.pxd.pxi",
"_cygrpc/time.pyx.pxi",
+ "_cygrpc/vtable.pxd.pxi",
+ "_cygrpc/vtable.pyx.pxi",
"cygrpc.pxd",
"cygrpc.pyx",
],
deps = [
- "//:grpc",
+ "//:grpc",
],
)
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/arguments.pxd.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/arguments.pxd.pxi
index 01b82374845..9415b16344a 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/arguments.pxd.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/arguments.pxd.pxi
@@ -13,15 +13,6 @@
# limitations under the License.
-cdef void* _copy_pointer(void* pointer)
-
-
-cdef void _destroy_pointer(void* pointer)
-
-
-cdef int _compare_pointer(void* first_pointer, void* second_pointer)
-
-
cdef tuple _wrap_grpc_arg(grpc_arg arg)
@@ -32,7 +23,7 @@ cdef class _ChannelArg:
cdef grpc_arg c_argument
- cdef void c(self, argument, grpc_arg_pointer_vtable *vtable, references) except *
+ cdef void c(self, argument, _VTable vtable, references) except *
cdef class _ChannelArgs:
@@ -42,8 +33,4 @@ cdef class _ChannelArgs:
cdef readonly list _references
cdef grpc_channel_args _c_arguments
- cdef void _c(self, grpc_arg_pointer_vtable *vtable) except *
cdef grpc_channel_args *c_args(self) except *
-
- @staticmethod
- cdef _ChannelArgs from_args(object arguments, grpc_arg_pointer_vtable * vtable)
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/arguments.pyx.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/arguments.pyx.pxi
index bf12871015d..9211354b1ca 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/arguments.pyx.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/arguments.pyx.pxi
@@ -15,25 +15,6 @@
cimport cpython
-# TODO(https://github.com/grpc/grpc/issues/15662): Reform this.
-cdef void* _copy_pointer(void* pointer):
- return pointer
-
-
-# TODO(https://github.com/grpc/grpc/issues/15662): Reform this.
-cdef void _destroy_pointer(void* pointer):
- pass
-
-
-cdef int _compare_pointer(void* first_pointer, void* second_pointer):
- if first_pointer < second_pointer:
- return -1
- elif first_pointer > second_pointer:
- return 1
- else:
- return 0
-
-
cdef class _GrpcArgWrapper:
cdef grpc_arg arg
@@ -52,7 +33,7 @@ cdef grpc_arg _unwrap_grpc_arg(tuple wrapped_arg):
cdef class _ChannelArg:
- cdef void c(self, argument, grpc_arg_pointer_vtable *vtable, references) except *:
+ cdef void c(self, argument, _VTable vtable, references) except *:
key, value = argument
cdef bytes encoded_key = _encode(key)
if encoded_key is not key:
@@ -75,7 +56,7 @@ cdef class _ChannelArg:
# lifecycle of the pointer is fixed to the lifecycle of the
# python object wrapping it.
self.c_argument.type = GRPC_ARG_POINTER
- self.c_argument.value.pointer.vtable = vtable
+ self.c_argument.value.pointer.vtable = &vtable.c_vtable
self.c_argument.value.pointer.address = <void*>(<intptr_t>int(value))
else:
raise TypeError(
@@ -84,13 +65,10 @@ cdef class _ChannelArg:
cdef class _ChannelArgs:
- def __cinit__(self, arguments):
+ def __cinit__(self, arguments, _VTable vtable not None):
self._arguments = () if arguments is None else tuple(arguments)
self._channel_args = []
self._references = []
- self._c_arguments.arguments = NULL
-
- cdef void _c(self, grpc_arg_pointer_vtable *vtable) except *:
self._c_arguments.arguments_length = len(self._arguments)
if self._c_arguments.arguments_length != 0:
self._c_arguments.arguments = <grpc_arg *>gpr_malloc(
@@ -107,9 +85,3 @@ cdef class _ChannelArgs:
def __dealloc__(self):
if self._c_arguments.arguments != NULL:
gpr_free(self._c_arguments.arguments)
-
- @staticmethod
- cdef _ChannelArgs from_args(object arguments, grpc_arg_pointer_vtable * vtable):
- cdef _ChannelArgs channel_args = _ChannelArgs(arguments)
- channel_args._c(vtable)
- return channel_args
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/call.pyx.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/call.pyx.pxi
index 6e4574af8d5..84934db4d60 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/call.pyx.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/call.pyx.pxi
@@ -17,11 +17,11 @@ cimport cpython
cdef class Call:
- def __cinit__(self):
+ def __cinit__(self, _VTable vtable not None):
# Create an *empty* call
fork_handlers_and_grpc_init()
self.c_call = NULL
- self.references = []
+ self.references = [vtable]
def _start_batch(self, operations, tag, retain_self):
if not self.is_valid:
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/channel.pxd.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/channel.pxd.pxi
index ced32abba14..13c0c02ab21 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/channel.pxd.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/channel.pxd.pxi
@@ -68,8 +68,8 @@ cdef class SegregatedCall:
cdef class Channel:
- cdef grpc_arg_pointer_vtable _vtable
cdef _ChannelState _state
+ cdef _VTable _vtable
# TODO(https://github.com/grpc/grpc/issues/15662): Eliminate this.
cdef tuple _arguments
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi
index 24c11e63a6b..ca637094353 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi
@@ -420,11 +420,14 @@ cdef class Channel:
arguments = () if arguments is None else tuple(arguments)
fork_handlers_and_grpc_init()
self._state = _ChannelState()
- self._vtable.copy = &_copy_pointer
- self._vtable.destroy = &_destroy_pointer
- self._vtable.cmp = &_compare_pointer
- cdef _ChannelArgs channel_args = _ChannelArgs.from_args(
- arguments, &self._vtable)
+ self._state.c_call_completion_queue = (
+ grpc_completion_queue_create_for_next(NULL))
+ self._state.c_connectivity_completion_queue = (
+ grpc_completion_queue_create_for_next(NULL))
+ self._arguments = arguments
+ self._vtable = _VTable()
+ cdef _ChannelArgs channel_args = _ChannelArgs(
+ arguments, self._vtable)
if channel_credentials is None:
self._state.c_channel = grpc_insecure_channel_create(
<char *>target, channel_args.c_args(), NULL)
@@ -433,11 +436,6 @@ cdef class Channel:
self._state.c_channel = grpc_secure_channel_create(
c_channel_credentials, <char *>target, channel_args.c_args(), NULL)
grpc_channel_credentials_release(c_channel_credentials)
- self._state.c_call_completion_queue = (
- grpc_completion_queue_create_for_next(NULL))
- self._state.c_connectivity_completion_queue = (
- grpc_completion_queue_create_for_next(NULL))
- self._arguments = arguments
def target(self):
cdef char *c_target
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/server.pxd.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/server.pxd.pxi
index 4a6fbe0f96c..b3fadcdc62d 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/server.pxd.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/server.pxd.pxi
@@ -12,11 +12,11 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-
cdef class Server:
- cdef grpc_arg_pointer_vtable _vtable
cdef grpc_server *c_server
+
+ cdef _VTable _vtable
cdef bint is_started # start has been called
cdef bint is_shutting_down # shutdown has been called
cdef bint is_shutdown # notification of complete shutdown received
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/server.pyx.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/server.pyx.pxi
index fe55ea885e4..2369371cabe 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/server.pyx.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/server.pyx.pxi
@@ -20,22 +20,21 @@ import grpc
_LOGGER = logging.getLogger(__name__)
+
cdef class Server:
def __cinit__(self, object arguments):
fork_handlers_and_grpc_init()
self.references = []
self.registered_completion_queues = []
- self._vtable.copy = &_copy_pointer
- self._vtable.destroy = &_destroy_pointer
- self._vtable.cmp = &_compare_pointer
- cdef _ChannelArgs channel_args = _ChannelArgs.from_args(
- arguments, &self._vtable)
- self.c_server = grpc_server_create(channel_args.c_args(), NULL)
- self.references.append(arguments)
self.is_started = False
self.is_shutting_down = False
self.is_shutdown = False
+ self.c_server = NULL
+ self._vtable = _VTable()
+ cdef _ChannelArgs channel_args = _ChannelArgs(arguments, self._vtable)
+ self.c_server = grpc_server_create(channel_args.c_args(), NULL)
+ self.references.append(arguments)
def request_call(
self, CompletionQueue call_queue not None,
@@ -44,7 +43,7 @@ cdef class Server:
raise ValueError("server must be started and not shutting down")
if server_queue not in self.registered_completion_queues:
raise ValueError("server_queue must be a registered completion queue")
- cdef _RequestCallTag request_call_tag = _RequestCallTag(tag)
+ cdef _RequestCallTag request_call_tag = _RequestCallTag(tag, self._vtable)
request_call_tag.prepare()
cpython.Py_INCREF(request_call_tag)
return grpc_server_request_call(
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/tag.pxd.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/tag.pxd.pxi
index d8ba1ea9bd5..c77beb28194 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/tag.pxd.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/tag.pxd.pxi
@@ -29,6 +29,7 @@ cdef class _RequestCallTag(_Tag):
cdef readonly object _user_tag
cdef Call call
+ cdef _VTable _vtable
cdef CallDetails call_details
cdef grpc_metadata_array c_invocation_metadata
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/tag.pyx.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/tag.pyx.pxi
index e80dc88767e..d1280ef4948 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/tag.pyx.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/tag.pyx.pxi
@@ -30,13 +30,14 @@ cdef class _ConnectivityTag(_Tag):
cdef class _RequestCallTag(_Tag):
- def __cinit__(self, user_tag):
+ def __cinit__(self, user_tag, _VTable vtable not None):
self._user_tag = user_tag
self.call = None
self.call_details = None
+ self._vtable = vtable
cdef void prepare(self) except *:
- self.call = Call()
+ self.call = Call(self._vtable)
self.call_details = CallDetails()
grpc_metadata_array_init(&self.c_invocation_metadata)
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/vtable.pxd.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/vtable.pxd.pxi
new file mode 100644
index 00000000000..1799b6e1f14
--- /dev/null
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/vtable.pxd.pxi
@@ -0,0 +1,26 @@
+# Copyright 2019 gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+cdef void* _copy_pointer(void* pointer)
+
+
+cdef void _destroy_pointer(void* pointer)
+
+
+cdef int _compare_pointer(void* first_pointer, void* second_pointer)
+
+
+cdef class _VTable:
+ cdef grpc_arg_pointer_vtable c_vtable
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/vtable.pyx.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/vtable.pyx.pxi
new file mode 100644
index 00000000000..98cb60c10e3
--- /dev/null
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/vtable.pyx.pxi
@@ -0,0 +1,39 @@
+# Copyright 2019 gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# TODO(https://github.com/grpc/grpc/issues/15662): Reform this.
+cdef void* _copy_pointer(void* pointer):
+ return pointer
+
+
+# TODO(https://github.com/grpc/grpc/issues/15662): Reform this.
+cdef void _destroy_pointer(void* pointer):
+ pass
+
+
+cdef int _compare_pointer(void* first_pointer, void* second_pointer):
+ if first_pointer < second_pointer:
+ return -1
+ elif first_pointer > second_pointer:
+ return 1
+ else:
+ return 0
+
+
+cdef class _VTable:
+ def __cinit__(self):
+ self.c_vtable.copy = &_copy_pointer
+ self.c_vtable.destroy = &_destroy_pointer
+ self.c_vtable.cmp = &_compare_pointer
+
diff --git a/src/python/grpcio/grpc/_cython/cygrpc.pxd b/src/python/grpcio/grpc/_cython/cygrpc.pxd
index 64cae6b34d6..e29f7aee97a 100644
--- a/src/python/grpcio/grpc/_cython/cygrpc.pxd
+++ b/src/python/grpcio/grpc/_cython/cygrpc.pxd
@@ -23,13 +23,14 @@ include "_cygrpc/completion_queue.pxd.pxi"
include "_cygrpc/event.pxd.pxi"
include "_cygrpc/metadata.pxd.pxi"
include "_cygrpc/operation.pxd.pxi"
+include "_cygrpc/propagation_bits.pxd.pxi"
include "_cygrpc/records.pxd.pxi"
include "_cygrpc/security.pxd.pxi"
include "_cygrpc/server.pxd.pxi"
include "_cygrpc/tag.pxd.pxi"
include "_cygrpc/time.pxd.pxi"
+include "_cygrpc/vtable.pxd.pxi"
include "_cygrpc/_hooks.pxd.pxi"
-include "_cygrpc/propagation_bits.pxd.pxi"
include "_cygrpc/grpc_gevent.pxd.pxi"
diff --git a/src/python/grpcio/grpc/_cython/cygrpc.pyx b/src/python/grpcio/grpc/_cython/cygrpc.pyx
index ce98fa3a8e6..f2dd0df89d4 100644
--- a/src/python/grpcio/grpc/_cython/cygrpc.pyx
+++ b/src/python/grpcio/grpc/_cython/cygrpc.pyx
@@ -24,19 +24,20 @@ include "_cygrpc/grpc_string.pyx.pxi"
include "_cygrpc/arguments.pyx.pxi"
include "_cygrpc/call.pyx.pxi"
include "_cygrpc/channel.pyx.pxi"
+include "_cygrpc/channelz.pyx.pxi"
include "_cygrpc/credentials.pyx.pxi"
include "_cygrpc/completion_queue.pyx.pxi"
include "_cygrpc/event.pyx.pxi"
include "_cygrpc/metadata.pyx.pxi"
include "_cygrpc/operation.pyx.pxi"
+include "_cygrpc/propagation_bits.pyx.pxi"
include "_cygrpc/records.pyx.pxi"
include "_cygrpc/security.pyx.pxi"
include "_cygrpc/server.pyx.pxi"
include "_cygrpc/tag.pyx.pxi"
include "_cygrpc/time.pyx.pxi"
+include "_cygrpc/vtable.pyx.pxi"
include "_cygrpc/_hooks.pyx.pxi"
-include "_cygrpc/channelz.pyx.pxi"
-include "_cygrpc/propagation_bits.pyx.pxi"
include "_cygrpc/grpc_gevent.pyx.pxi"
From 34aa71464482b4b93b81ff430e942b9aa2517b49 Mon Sep 17 00:00:00 2001
From: Juanli Shen <juanlishen@google.com>
Date: Fri, 8 Mar 2019 15:37:09 -0800
Subject: [PATCH 080/138] Remove pollset_set when resetting LB policies
---
.../filters/client_channel/lb_policy/grpclb/grpclb.cc | 11 +++++++++++
.../ext/filters/client_channel/lb_policy/xds/xds.cc | 11 +++++++++++
.../ext/filters/client_channel/resolving_lb_policy.cc | 3 +++
3 files changed, 25 insertions(+)
diff --git a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
index 184215a3da9..34fe88215fe 100644
--- a/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc
@@ -625,6 +625,9 @@ void GrpcLb::Helper::UpdateState(grpc_connectivity_state state,
GRPC_ERROR_UNREF(state_error);
return;
}
+ grpc_pollset_set_del_pollset_set(
+ parent_->child_policy_->interested_parties(),
+ parent_->interested_parties());
MutexLock lock(&parent_->child_policy_mu_);
parent_->child_policy_ = std::move(parent_->pending_child_policy_);
} else if (!CalledByCurrentChild()) {
@@ -1272,6 +1275,14 @@ void GrpcLb::ShutdownLocked() {
if (fallback_timer_callback_pending_) {
grpc_timer_cancel(&lb_fallback_timer_);
}
+ if (child_policy_ != nullptr) {
+ grpc_pollset_set_del_pollset_set(child_policy_->interested_parties(),
+ interested_parties());
+ }
+ if (pending_child_policy_ != nullptr) {
+ grpc_pollset_set_del_pollset_set(
+ pending_child_policy_->interested_parties(), interested_parties());
+ }
{
MutexLock lock(&child_policy_mu_);
child_policy_.reset();
diff --git a/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc b/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc
index 4b386d37797..eca41bf3a2e 100644
--- a/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/xds/xds.cc
@@ -430,6 +430,9 @@ void XdsLb::Helper::UpdateState(grpc_connectivity_state state,
GRPC_ERROR_UNREF(state_error);
return;
}
+ grpc_pollset_set_del_pollset_set(
+ parent_->child_policy_->interested_parties(),
+ parent_->interested_parties());
MutexLock lock(&parent_->child_policy_mu_);
parent_->child_policy_ = std::move(parent_->pending_child_policy_);
} else if (!CalledByCurrentChild()) {
@@ -1154,6 +1157,14 @@ void XdsLb::ShutdownLocked() {
if (fallback_timer_callback_pending_) {
grpc_timer_cancel(&lb_fallback_timer_);
}
+ if (child_policy_ != nullptr) {
+ grpc_pollset_set_del_pollset_set(child_policy_->interested_parties(),
+ interested_parties());
+ }
+ if (pending_child_policy_ != nullptr) {
+ grpc_pollset_set_del_pollset_set(
+ pending_child_policy_->interested_parties(), interested_parties());
+ }
{
MutexLock lock(&child_policy_mu_);
child_policy_.reset();
diff --git a/src/core/ext/filters/client_channel/resolving_lb_policy.cc b/src/core/ext/filters/client_channel/resolving_lb_policy.cc
index 0dd51e8bc4c..52b14dcc7de 100644
--- a/src/core/ext/filters/client_channel/resolving_lb_policy.cc
+++ b/src/core/ext/filters/client_channel/resolving_lb_policy.cc
@@ -110,6 +110,9 @@ class ResolvingLoadBalancingPolicy::ResolvingControlHelper
GRPC_ERROR_UNREF(state_error);
return;
}
+ grpc_pollset_set_del_pollset_set(
+ parent_->lb_policy_->interested_parties(),
+ parent_->interested_parties());
MutexLock lock(&parent_->lb_policy_mu_);
parent_->lb_policy_ = std::move(parent_->pending_lb_policy_);
} else if (!CalledByCurrentChild()) {
From 307044c6aff8f51fc3496e34e7a72deb65f41658 Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 8 Mar 2019 16:18:45 -0800
Subject: [PATCH 081/138] Fix linting
---
tools/distrib/pylint_code.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/distrib/pylint_code.sh b/tools/distrib/pylint_code.sh
index abb37dde0ed..75aed1c4814 100755
--- a/tools/distrib/pylint_code.sh
+++ b/tools/distrib/pylint_code.sh
@@ -32,7 +32,7 @@ TEST_DIRS=(
)
VIRTUALENV=python_pylint_venv
-python3 -m virtualenv $VIRTUALENV
+python3 -m virtualenv $VIRTUALENV -p $(which python3)
PYTHON=$VIRTUALENV/bin/python
From 81c3b0bfb6ac4813ff96373eb8d9a0c702a6f0c2 Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 8 Mar 2019 16:18:57 -0800
Subject: [PATCH 082/138] Fix lint errors
---
examples/python/multiprocessing/client.py | 25 ++++++++++---------
examples/python/multiprocessing/server.py | 8 +++---
.../test/_multiprocessing_example_test.py | 5 ++--
3 files changed, 19 insertions(+), 19 deletions(-)
diff --git a/examples/python/multiprocessing/client.py b/examples/python/multiprocessing/client.py
index fa3b394f394..6c583964fde 100644
--- a/examples/python/multiprocessing/client.py
+++ b/examples/python/multiprocessing/client.py
@@ -19,14 +19,13 @@ from __future__ import print_function
import argparse
import atexit
-import grpc
import logging
import multiprocessing
import operator
-import os
-import time
import sys
+import grpc
+
import prime_pb2
import prime_pb2_grpc
@@ -34,15 +33,23 @@ _PROCESS_COUNT = 8
_MAXIMUM_CANDIDATE = 10000
# Each worker process initializes a single channel after forking.
+# It's regrettable, but to ensure that each subprocess only has to instantiate
+# a single channel to be reused across all RPCs, we use globals.
_worker_channel_singleton = None
_worker_stub_singleton = None
_LOGGER = logging.getLogger(__name__)
+def _shutdown_worker():
+ _LOGGER.info('Shutting worker process down.')
+ if _worker_channel_singleton is not None:
+ _worker_channel_singleton.stop()
+
+
def _initialize_worker(server_address):
- global _worker_channel_singleton
- global _worker_stub_singleton
+ global _worker_channel_singleton # pylint: disable=global-statement
+ global _worker_stub_singleton # pylint: disable=global-statement
_LOGGER.info('Initializing worker process.')
_worker_channel_singleton = grpc.insecure_channel(server_address)
_worker_stub_singleton = prime_pb2_grpc.PrimeCheckerStub(
@@ -50,14 +57,8 @@ def _initialize_worker(server_address):
atexit.register(_shutdown_worker)
-def _shutdown_worker():
- _LOGGER.info('Shutting worker process down.')
- if _worker_channel_singleton is not None:
- _worker_channel_singleton.stop()
-
-
def _run_worker_query(primality_candidate):
- _LOGGER.info('Checking primality of {}.'.format(primality_candidate))
+ _LOGGER.info('Checking primality of %s.', primality_candidate)
return _worker_stub_singleton.check(
prime_pb2.PrimeCandidate(candidate=primality_candidate))
diff --git a/examples/python/multiprocessing/server.py b/examples/python/multiprocessing/server.py
index 588cd4734e9..d686d90559a 100644
--- a/examples/python/multiprocessing/server.py
+++ b/examples/python/multiprocessing/server.py
@@ -20,15 +20,15 @@ from __future__ import print_function
from concurrent import futures
import contextlib
import datetime
-import grpc
import logging
import math
import multiprocessing
-import os
import time
import socket
import sys
+import grpc
+
import prime_pb2
import prime_pb2_grpc
@@ -50,7 +50,7 @@ def is_prime(n):
class PrimeChecker(prime_pb2_grpc.PrimeCheckerServicer):
def check(self, request, context):
- _LOGGER.info('Determining primality of {}'.format(request.candidate))
+ _LOGGER.info('Determining primality of %s', request.candidate)
return prime_pb2.Primality(isPrime=is_prime(request.candidate))
@@ -99,7 +99,7 @@ def _reserve_port():
def main():
with _reserve_port() as port:
bind_address = '[::]:{}'.format(port)
- _LOGGER.info("Binding to '{}'".format(bind_address))
+ _LOGGER.info("Binding to '%s'", bind_address)
sys.stdout.flush()
workers = []
for _ in range(_PROCESS_COUNT):
diff --git a/examples/python/multiprocessing/test/_multiprocessing_example_test.py b/examples/python/multiprocessing/test/_multiprocessing_example_test.py
index 92e7c0a4b8d..2d8f8d49db4 100644
--- a/examples/python/multiprocessing/test/_multiprocessing_example_test.py
+++ b/examples/python/multiprocessing/test/_multiprocessing_example_test.py
@@ -13,14 +13,13 @@
# limitations under the License.
"""Test for multiprocessing example."""
-import datetime
+import ast
import logging
import math
import os
import re
import subprocess
import tempfile
-import time
import unittest
_BINARY_DIR = os.path.realpath(
@@ -63,7 +62,7 @@ class MultiprocessingExampleTest(unittest.TestCase):
client_process.wait()
server_process.terminate()
client_stdout.seek(0)
- results = eval(client_stdout.read().strip().split('\n')[-1])
+ results = ast.literal_eval(client_stdout.read().strip().split('\n')[-1])
values = tuple(result[0] for result in results)
self.assertSequenceEqual(range(2, 10000), values)
for result in results:
From d359dbe44d24685d07a1e621404d6c497769b33f Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 8 Mar 2019 16:21:49 -0800
Subject: [PATCH 083/138] Remove unnecessary flush
---
examples/python/multiprocessing/client.py | 1 -
1 file changed, 1 deletion(-)
diff --git a/examples/python/multiprocessing/client.py b/examples/python/multiprocessing/client.py
index 6c583964fde..c3da4ba2b99 100644
--- a/examples/python/multiprocessing/client.py
+++ b/examples/python/multiprocessing/client.py
@@ -84,7 +84,6 @@ def main():
args = parser.parse_args()
primes = _calculate_primes(args.server_address)
print(primes)
- sys.stdout.flush()
if __name__ == '__main__':
From 69b5476429f883f15e6438f4de43da9504217434 Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 8 Mar 2019 16:38:58 -0800
Subject: [PATCH 084/138] Expand the readme
---
examples/python/multiprocessing/README.md | 21 +++++++++++++++++++--
examples/python/multiprocessing/server.py | 4 ++--
2 files changed, 21 insertions(+), 4 deletions(-)
diff --git a/examples/python/multiprocessing/README.md b/examples/python/multiprocessing/README.md
index 6dcec8b2132..c0e296924ee 100644
--- a/examples/python/multiprocessing/README.md
+++ b/examples/python/multiprocessing/README.md
@@ -14,11 +14,26 @@ However, calling `fork` without `exec` in your python process is supported
*before* any gRPC servers have been instantiated. Application developers can
take advantage of this to parallelize their CPU-intensive operations.
-## Running the Example
+## Calculating Prime Numbers with Multiple Processes
This example calculates the first 10,000 prime numbers as an RPC. We instantiate
one server per subprocess, balancing requests between the servers using the
-[`SO_REUSEPORT`](https://lwn.net/Articles/542629/) socket option.
+[`SO_REUSEPORT`](https://lwn.net/Articles/542629/) socket option. Note that this
+option is not available in `manylinux1` distributions, which are, as of the time
+of writing, the only gRPC Python wheels available on PyPi. To take advantage of this
+feature, you'll need to build from source, either using bazel (as we do for
+these examples) or via pip, using `pip install grpcio --no-binary grpcio`.
+
+```python
+_PROCESS_COUNT = multiprocessing.cpu_count()
+```
+
+On the server side, we detect the number of CPUs available on the system and
+spawn exactly that many child processes. If we spin up fewer, we won't be taking
+full advantage of the hardware resources available. If we spin up more, then the
+kernel will have to multiplex the processes on the available CPUs.
+
+## Running the Example
To run the server,
[ensure `bazel` is installed](https://docs.bazel.build/versions/master/install.html)
@@ -38,6 +53,8 @@ Note the address at which the server is running. For example,
...
```
+Note that several servers have been started, each with its own PID.
+
Now, start the client by running
```
diff --git a/examples/python/multiprocessing/server.py b/examples/python/multiprocessing/server.py
index d686d90559a..27a0758d224 100644
--- a/examples/python/multiprocessing/server.py
+++ b/examples/python/multiprocessing/server.py
@@ -35,8 +35,8 @@ import prime_pb2_grpc
_LOGGER = logging.getLogger(__name__)
_ONE_DAY = datetime.timedelta(days=1)
-_PROCESS_COUNT = 8
-_THREAD_CONCURRENCY = 10
+_PROCESS_COUNT = multiprocessing.cpu_count()
+_THREAD_CONCURRENCY = _PROCESS_COUNT
def is_prime(n):
From 4c8c8e36d241d6f526665619fa142c91fc8f9538 Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 8 Mar 2019 16:52:33 -0800
Subject: [PATCH 085/138] Show some respect
---
examples/python/multiprocessing/README.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/examples/python/multiprocessing/README.md b/examples/python/multiprocessing/README.md
index c0e296924ee..b752d85a9ae 100644
--- a/examples/python/multiprocessing/README.md
+++ b/examples/python/multiprocessing/README.md
@@ -20,7 +20,7 @@ This example calculates the first 10,000 prime numbers as an RPC. We instantiate
one server per subprocess, balancing requests between the servers using the
[`SO_REUSEPORT`](https://lwn.net/Articles/542629/) socket option. Note that this
option is not available in `manylinux1` distributions, which are, as of the time
-of writing, the only gRPC Python wheels available on PyPi. To take advantage of this
+of writing, the only gRPC Python wheels available on PyPI. To take advantage of this
feature, you'll need to build from source, either using bazel (as we do for
these examples) or via pip, using `pip install grpcio --no-binary grpcio`.
From 2640822c2d65adca9f15410e03f2e4be0b32e0f3 Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Fri, 8 Mar 2019 17:24:59 -0800
Subject: [PATCH 086/138] Remove a statement proven wrong by science
---
examples/python/multiprocessing/README.md | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/examples/python/multiprocessing/README.md b/examples/python/multiprocessing/README.md
index b752d85a9ae..709a815aca5 100644
--- a/examples/python/multiprocessing/README.md
+++ b/examples/python/multiprocessing/README.md
@@ -30,8 +30,7 @@ _PROCESS_COUNT = multiprocessing.cpu_count()
On the server side, we detect the number of CPUs available on the system and
spawn exactly that many child processes. If we spin up fewer, we won't be taking
-full advantage of the hardware resources available. If we spin up more, then the
-kernel will have to multiplex the processes on the available CPUs.
+full advantage of the hardware resources available.
## Running the Example
From 9f9c03d0f5023ff10058713de0dd4593676801c8 Mon Sep 17 00:00:00 2001
From: Vijay Pai <vpai@google.com>
Date: Sat, 9 Mar 2019 12:14:02 -0800
Subject: [PATCH 087/138] Prevent merging if PR marked DO NOT MERGE
---
.github/mergeable.yml | 24 ++++++++++++++----------
1 file changed, 14 insertions(+), 10 deletions(-)
diff --git a/.github/mergeable.yml b/.github/mergeable.yml
index 30692095c46..a10ae9b7589 100644
--- a/.github/mergeable.yml
+++ b/.github/mergeable.yml
@@ -1,14 +1,18 @@
mergeable:
pull_requests:
label:
- or:
- - and:
+ and:
+ - must_exclude:
+ regex: '^disposition/DO NOT MERGE'
+ message: 'Pull request marked not mergeable'
+ - or:
+ - and:
+ - must_include:
+ regex: 'release notes: yes'
+ message: 'Please include release note: yes'
+ - must_include:
+ regex: '^lang\/'
+ message: 'Please include a language label'
- must_include:
- regex: 'release notes: yes'
- message: 'Please include release note: yes'
- - must_include:
- regex: '^lang\/'
- message: 'Please include a language label'
- - must_include:
- regex: 'release notes: no'
- message: 'Please include release note: no'
+ regex: 'release notes: no'
+ message: 'Please include release note: no'
From 3291154db08d5f32de96991d3048a9e4954865e5 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Wed, 21 Nov 2018 13:28:09 +0100
Subject: [PATCH 088/138] update all executables to netcoreapp2.1
---
src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.csproj | 2 +-
.../Grpc.Examples.MathClient/Grpc.Examples.MathClient.csproj | 2 +-
.../Grpc.Examples.MathServer/Grpc.Examples.MathServer.csproj | 2 +-
src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.csproj | 2 +-
src/csharp/Grpc.Examples/Grpc.Examples.csproj | 2 +-
src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.csproj | 2 +-
.../Grpc.IntegrationTesting.Client.csproj | 2 +-
.../Grpc.IntegrationTesting.QpsWorker.csproj | 2 +-
.../Grpc.IntegrationTesting.Server.csproj | 2 +-
.../Grpc.IntegrationTesting.StressClient.csproj | 2 +-
.../Grpc.IntegrationTesting/Grpc.IntegrationTesting.csproj | 2 +-
src/csharp/Grpc.Microbenchmarks/Grpc.Microbenchmarks.csproj | 2 +-
src/csharp/Grpc.Reflection.Tests/Grpc.Reflection.Tests.csproj | 2 +-
src/csharp/Grpc.Tools.Tests/Grpc.Tools.Tests.csproj | 2 +-
14 files changed, 14 insertions(+), 14 deletions(-)
diff --git a/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.csproj b/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.csproj
index 2a3e30174c0..8de9b675a7d 100755
--- a/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.csproj
+++ b/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.csproj
@@ -4,7 +4,7 @@
<Import Project="..\Grpc.Core\Common.csproj.include" />
<PropertyGroup>
- <TargetFrameworks>net45;netcoreapp1.1</TargetFrameworks>
+ <TargetFrameworks>net45;netcoreapp2.1</TargetFrameworks>
<OutputType>Exe</OutputType>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.csproj b/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.csproj
index 557f4639bd5..1011ebce0f0 100755
--- a/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.csproj
+++ b/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.csproj
@@ -4,7 +4,7 @@
<Import Project="..\Grpc.Core\Common.csproj.include" />
<PropertyGroup>
- <TargetFrameworks>net45;netcoreapp1.1</TargetFrameworks>
+ <TargetFrameworks>net45;netcoreapp2.1</TargetFrameworks>
<OutputType>Exe</OutputType>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.csproj b/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.csproj
index 557f4639bd5..1011ebce0f0 100755
--- a/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.csproj
+++ b/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.csproj
@@ -4,7 +4,7 @@
<Import Project="..\Grpc.Core\Common.csproj.include" />
<PropertyGroup>
- <TargetFrameworks>net45;netcoreapp1.1</TargetFrameworks>
+ <TargetFrameworks>net45;netcoreapp2.1</TargetFrameworks>
<OutputType>Exe</OutputType>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.csproj b/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.csproj
index e2d988a8662..26ae2776446 100755
--- a/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.csproj
+++ b/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.csproj
@@ -4,7 +4,7 @@
<Import Project="..\Grpc.Core\Common.csproj.include" />
<PropertyGroup>
- <TargetFrameworks>net45;netcoreapp1.1</TargetFrameworks>
+ <TargetFrameworks>net45;netcoreapp2.1</TargetFrameworks>
<OutputType>Exe</OutputType>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.Examples/Grpc.Examples.csproj b/src/csharp/Grpc.Examples/Grpc.Examples.csproj
index 5e532b11982..65ca87ed121 100755
--- a/src/csharp/Grpc.Examples/Grpc.Examples.csproj
+++ b/src/csharp/Grpc.Examples/Grpc.Examples.csproj
@@ -4,7 +4,7 @@
<Import Project="..\Grpc.Core\Common.csproj.include" />
<PropertyGroup>
- <TargetFrameworks>net45;netcoreapp1.1</TargetFrameworks>
+ <TargetFrameworks>net45;netcoreapp2.1</TargetFrameworks>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.csproj b/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.csproj
index 6b9e37b3e6c..2c759124689 100755
--- a/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.csproj
+++ b/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.csproj
@@ -4,7 +4,7 @@
<Import Project="..\Grpc.Core\Common.csproj.include" />
<PropertyGroup>
- <TargetFrameworks>net45;netcoreapp1.1</TargetFrameworks>
+ <TargetFrameworks>net45;netcoreapp2.1</TargetFrameworks>
<OutputType>Exe</OutputType>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.csproj b/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.csproj
index 5b29bf0a72f..30991cd0b57 100755
--- a/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.csproj
+++ b/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.csproj
@@ -4,7 +4,7 @@
<Import Project="..\Grpc.Core\Common.csproj.include" />
<PropertyGroup>
- <TargetFrameworks>net45;netcoreapp1.1</TargetFrameworks>
+ <TargetFrameworks>net45;netcoreapp2.1</TargetFrameworks>
<OutputType>Exe</OutputType>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.csproj b/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.csproj
index c8bd3e3f186..8c682beb396 100755
--- a/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.csproj
+++ b/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.csproj
@@ -4,7 +4,7 @@
<Import Project="..\Grpc.Core\Common.csproj.include" />
<PropertyGroup>
- <TargetFrameworks>net45;netcoreapp1.1</TargetFrameworks>
+ <TargetFrameworks>net45;netcoreapp2.1</TargetFrameworks>
<OutputType>Exe</OutputType>
<ServerGarbageCollection>true</ServerGarbageCollection>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
diff --git a/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.csproj b/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.csproj
index 5b29bf0a72f..30991cd0b57 100755
--- a/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.csproj
+++ b/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.csproj
@@ -4,7 +4,7 @@
<Import Project="..\Grpc.Core\Common.csproj.include" />
<PropertyGroup>
- <TargetFrameworks>net45;netcoreapp1.1</TargetFrameworks>
+ <TargetFrameworks>net45;netcoreapp2.1</TargetFrameworks>
<OutputType>Exe</OutputType>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.csproj b/src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.csproj
index 5b29bf0a72f..30991cd0b57 100755
--- a/src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.csproj
+++ b/src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.csproj
@@ -4,7 +4,7 @@
<Import Project="..\Grpc.Core\Common.csproj.include" />
<PropertyGroup>
- <TargetFrameworks>net45;netcoreapp1.1</TargetFrameworks>
+ <TargetFrameworks>net45;netcoreapp2.1</TargetFrameworks>
<OutputType>Exe</OutputType>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.csproj b/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.csproj
index 6bf5d220e4b..fd90e19c843 100755
--- a/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.csproj
+++ b/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.csproj
@@ -4,7 +4,7 @@
<Import Project="..\Grpc.Core\Common.csproj.include" />
<PropertyGroup>
- <TargetFrameworks>net45;netcoreapp1.1</TargetFrameworks>
+ <TargetFrameworks>net45;netcoreapp2.1</TargetFrameworks>
<OutputType>Exe</OutputType>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.Microbenchmarks/Grpc.Microbenchmarks.csproj b/src/csharp/Grpc.Microbenchmarks/Grpc.Microbenchmarks.csproj
index f30b90b5130..71f970f09cd 100644
--- a/src/csharp/Grpc.Microbenchmarks/Grpc.Microbenchmarks.csproj
+++ b/src/csharp/Grpc.Microbenchmarks/Grpc.Microbenchmarks.csproj
@@ -4,7 +4,7 @@
<Import Project="..\Grpc.Core\Common.csproj.include" />
<PropertyGroup>
- <TargetFrameworks>net45;netcoreapp1.1</TargetFrameworks>
+ <TargetFrameworks>net45;netcoreapp2.1</TargetFrameworks>
<OutputType>Exe</OutputType>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.Reflection.Tests/Grpc.Reflection.Tests.csproj b/src/csharp/Grpc.Reflection.Tests/Grpc.Reflection.Tests.csproj
index 0fb0726d7aa..6436058d4e2 100755
--- a/src/csharp/Grpc.Reflection.Tests/Grpc.Reflection.Tests.csproj
+++ b/src/csharp/Grpc.Reflection.Tests/Grpc.Reflection.Tests.csproj
@@ -4,7 +4,7 @@
<Import Project="..\Grpc.Core\Common.csproj.include" />
<PropertyGroup>
- <TargetFrameworks>net45;netcoreapp1.1</TargetFrameworks>
+ <TargetFrameworks>net45;netcoreapp2.1</TargetFrameworks>
<OutputType>Exe</OutputType>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.Tools.Tests/Grpc.Tools.Tests.csproj b/src/csharp/Grpc.Tools.Tests/Grpc.Tools.Tests.csproj
index cfb40f44ae1..402d860e382 100644
--- a/src/csharp/Grpc.Tools.Tests/Grpc.Tools.Tests.csproj
+++ b/src/csharp/Grpc.Tools.Tests/Grpc.Tools.Tests.csproj
@@ -3,7 +3,7 @@
<Import Project="..\Grpc.Core\Version.csproj.include" />
<PropertyGroup>
- <TargetFrameworks>net45;netcoreapp1.1</TargetFrameworks>
+ <TargetFrameworks>net45;netcoreapp2.1</TargetFrameworks>
<OutputType>Exe</OutputType>
</PropertyGroup>
From c586eea27ef6118ede0a07cdf25512d7350529e7 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Wed, 21 Nov 2018 16:24:01 +0100
Subject: [PATCH 089/138] adjust run_*tests.py
---
tools/run_tests/performance/run_worker_csharp.sh | 2 +-
tools/run_tests/run_interop_tests.py | 4 ++--
tools/run_tests/run_tests.py | 2 +-
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/tools/run_tests/performance/run_worker_csharp.sh b/tools/run_tests/performance/run_worker_csharp.sh
index bfa59b5d9e6..af944f9fefa 100755
--- a/tools/run_tests/performance/run_worker_csharp.sh
+++ b/tools/run_tests/performance/run_worker_csharp.sh
@@ -18,6 +18,6 @@ set -ex
cd "$(dirname "$0")/../../.."
# needed to correctly locate testca
-cd src/csharp/Grpc.IntegrationTesting.QpsWorker/bin/Release/netcoreapp1.1
+cd src/csharp/Grpc.IntegrationTesting.QpsWorker/bin/Release/netcoreapp2.1
dotnet exec Grpc.IntegrationTesting.QpsWorker.dll "$@"
diff --git a/tools/run_tests/run_interop_tests.py b/tools/run_tests/run_interop_tests.py
index 448b53a7207..d35dccd8d03 100755
--- a/tools/run_tests/run_interop_tests.py
+++ b/tools/run_tests/run_interop_tests.py
@@ -147,8 +147,8 @@ class CSharpLanguage:
class CSharpCoreCLRLanguage:
def __init__(self):
- self.client_cwd = 'src/csharp/Grpc.IntegrationTesting.Client/bin/Debug/netcoreapp1.1'
- self.server_cwd = 'src/csharp/Grpc.IntegrationTesting.Server/bin/Debug/netcoreapp1.1'
+ self.client_cwd = 'src/csharp/Grpc.IntegrationTesting.Client/bin/Debug/netcoreapp2.1'
+ self.server_cwd = 'src/csharp/Grpc.IntegrationTesting.Server/bin/Debug/netcoreapp2.1'
self.safename = str(self)
def client_cmd(self, args):
diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py
index f1e1f539ff9..1c4d20ef6da 100755
--- a/tools/run_tests/run_tests.py
+++ b/tools/run_tests/run_tests.py
@@ -946,7 +946,7 @@ class CSharpLanguage(object):
assembly_extension = '.exe'
if self.args.compiler == 'coreclr':
- assembly_subdir += '/netcoreapp1.1'
+ assembly_subdir += '/netcoreapp2.1'
runtime_cmd = ['dotnet', 'exec']
assembly_extension = '.dll'
else:
From 38ecd3831b6b8355a673437bb42df5b30b6130fb Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Mon, 11 Mar 2019 07:58:58 -0400
Subject: [PATCH 090/138] install dotnet SDK before starting Windows build
---
src/csharp/install_dotnet_sdk.ps1 | 16 ++++++++++++++++
.../helper_scripts/prepare_build_windows.bat | 4 ++++
2 files changed, 20 insertions(+)
create mode 100644 src/csharp/install_dotnet_sdk.ps1
diff --git a/src/csharp/install_dotnet_sdk.ps1 b/src/csharp/install_dotnet_sdk.ps1
new file mode 100644
index 00000000000..57328fa9981
--- /dev/null
+++ b/src/csharp/install_dotnet_sdk.ps1
@@ -0,0 +1,16 @@
+#!/usr/bin/env powershell
+# Install dotnet SDK needed to build C# projects on Windows
+
+Set-StrictMode -Version 2
+$ErrorActionPreference = 'Stop'
+
+# avoid "Unknown error on a send" in Invoke-WebRequest
+[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
+
+$InstallScriptUrl = 'https://dot.net/v1/dotnet-install.ps1'
+$InstallScriptPath = Join-Path "$env:TEMP" 'dotnet-install.ps1'
+
+# Download install script
+Write-Host "Downloading install script: $InstallScriptUrl => $InstallScriptPath"
+Invoke-WebRequest -Uri $InstallScriptUrl -OutFile $InstallScriptPath
+&$InstallScriptPath -Version 2.1.504
diff --git a/tools/internal_ci/helper_scripts/prepare_build_windows.bat b/tools/internal_ci/helper_scripts/prepare_build_windows.bat
index f987f8a8cb5..bee59159331 100644
--- a/tools/internal_ci/helper_scripts/prepare_build_windows.bat
+++ b/tools/internal_ci/helper_scripts/prepare_build_windows.bat
@@ -34,6 +34,10 @@ netsh interface ip add dnsservers "Local Area Connection 8" 8.8.4.4 index=3
@rem Needed for big_query_utils
python -m pip install google-api-python-client
+@rem C# prerequisites: Install dotnet SDK
+powershell -File src\csharp\install_dotnet_sdk.ps1
+set PATH=%LOCALAPPDATA%\Microsoft\dotnet;%PATH%
+
@rem Disable some unwanted dotnet options
set NUGET_XMLDOC_MODE=skip
set DOTNET_SKIP_FIRST_TIME_EXPERIENCE=true
From 0c8c4c6dd50b57871e504377e0375207d5ace16d Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Mon, 11 Mar 2019 14:47:28 -0400
Subject: [PATCH 091/138] update third_party/protobuf to v3.7.0
---
third_party/protobuf | 2 +-
tools/run_tests/sanity/check_submodules.sh | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/third_party/protobuf b/third_party/protobuf
index 48cb18e5c41..582743bf40c 160000
--- a/third_party/protobuf
+++ b/third_party/protobuf
@@ -1 +1 @@
-Subproject commit 48cb18e5c419ddd23d9badcfe4e9df7bde1979b2
+Subproject commit 582743bf40c5d3639a70f98f183914a2c0cd0680
diff --git a/tools/run_tests/sanity/check_submodules.sh b/tools/run_tests/sanity/check_submodules.sh
index f1103596d51..2c447f887ee 100755
--- a/tools/run_tests/sanity/check_submodules.sh
+++ b/tools/run_tests/sanity/check_submodules.sh
@@ -38,7 +38,7 @@ cat << EOF | awk '{ print $1 }' | sort > "$want_submodules"
ec44c6c1675c25b9827aacd08c02433cccde7780 third_party/googletest (release-1.8.0)
6599cac0965be8e5a835ab7a5684bbef033d5ad0 third_party/libcxx (heads/release_60)
9245d481eb3e890f708ff2d7dadf2a10c04748ba third_party/libcxxabi (heads/release_60)
- 48cb18e5c419ddd23d9badcfe4e9df7bde1979b2 third_party/protobuf (v3.6.0.1-37-g48cb18e5)
+ 582743bf40c5d3639a70f98f183914a2c0cd0680 third_party/protobuf (v3.7.0-rc.2-20-g582743bf)
e143189bf6f37b3957fb31743df6a1bcf4a8c685 third_party/protoc-gen-validate (v0.0.10)
9ce4a77f61c134bbed28bfd5be5cd7dc0e80f5e3 third_party/upb (heads/upbc-cpp)
cacf7f1d4e3d44d871b605da3b647f07d718623f third_party/zlib (v1.2.11)
From e3c024591b8910d6239ef98c1213506bc04e6faa Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Mon, 11 Mar 2019 14:49:08 -0400
Subject: [PATCH 092/138] update bazel build to protobuf v3.7.0
---
bazel/grpc_deps.bzl | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/bazel/grpc_deps.bzl b/bazel/grpc_deps.bzl
index 2795ce8e732..d97e8368ed7 100644
--- a/bazel/grpc_deps.bzl
+++ b/bazel/grpc_deps.bzl
@@ -124,8 +124,8 @@ def grpc_deps():
if "com_google_protobuf" not in native.existing_rules():
http_archive(
name = "com_google_protobuf",
- strip_prefix = "protobuf-66dc42d891a4fc8e9190c524fd67961688a37bbe",
- url = "https://github.com/google/protobuf/archive/66dc42d891a4fc8e9190c524fd67961688a37bbe.tar.gz",
+ strip_prefix = "protobuf-582743bf40c5d3639a70f98f183914a2c0cd0680",
+ url = "https://github.com/google/protobuf/archive/582743bf40c5d3639a70f98f183914a2c0cd0680.tar.gz",
)
if "com_github_nanopb_nanopb" not in native.existing_rules():
From 1c6040162e55e264acb4f55747eeecbff05632b9 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Mon, 11 Mar 2019 14:59:45 -0400
Subject: [PATCH 093/138] regenerate C# protos
---
src/csharp/Grpc.Examples/MathGrpc.cs | 6 +-
src/csharp/Grpc.HealthCheck/Health.cs | 2 +-
src/csharp/Grpc.HealthCheck/HealthGrpc.cs | 6 +-
.../BenchmarkServiceGrpc.cs | 6 +-
src/csharp/Grpc.IntegrationTesting/Control.cs | 84 +++++++++----------
.../Grpc.IntegrationTesting/EchoMessages.cs | 24 +++---
src/csharp/Grpc.IntegrationTesting/Empty.cs | 1 -
.../EmptyServiceGrpc.cs | 6 +-
.../Grpc.IntegrationTesting/Messages.cs | 72 ++++++++--------
.../Grpc.IntegrationTesting/MetricsGrpc.cs | 8 +-
.../ReportQpsScenarioServiceGrpc.cs | 6 +-
src/csharp/Grpc.IntegrationTesting/Stats.cs | 18 ++--
.../Grpc.IntegrationTesting/TestGrpc.cs | 6 +-
.../WorkerServiceGrpc.cs | 6 +-
src/csharp/Grpc.Reflection/Reflection.cs | 6 +-
src/csharp/Grpc.Reflection/ReflectionGrpc.cs | 6 +-
16 files changed, 131 insertions(+), 132 deletions(-)
diff --git a/src/csharp/Grpc.Examples/MathGrpc.cs b/src/csharp/Grpc.Examples/MathGrpc.cs
index acd70b3714d..ba6824dd8e2 100644
--- a/src/csharp/Grpc.Examples/MathGrpc.cs
+++ b/src/csharp/Grpc.Examples/MathGrpc.cs
@@ -4,13 +4,13 @@
// </auto-generated>
// Original file comments:
// Copyright 2015 gRPC authors.
-//
+//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
-//
+//
// http://www.apache.org/licenses/LICENSE-2.0
-//
+//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
diff --git a/src/csharp/Grpc.HealthCheck/Health.cs b/src/csharp/Grpc.HealthCheck/Health.cs
index 2c3bb45c3cf..82e42febedd 100644
--- a/src/csharp/Grpc.HealthCheck/Health.cs
+++ b/src/csharp/Grpc.HealthCheck/Health.cs
@@ -296,7 +296,7 @@ namespace Grpc.Health.V1 {
_unknownFields = pb::UnknownFieldSet.MergeFieldFrom(_unknownFields, input);
break;
case 8: {
- status_ = (global::Grpc.Health.V1.HealthCheckResponse.Types.ServingStatus) input.ReadEnum();
+ Status = (global::Grpc.Health.V1.HealthCheckResponse.Types.ServingStatus) input.ReadEnum();
break;
}
}
diff --git a/src/csharp/Grpc.HealthCheck/HealthGrpc.cs b/src/csharp/Grpc.HealthCheck/HealthGrpc.cs
index e13b1147cf8..3edec5a37f8 100644
--- a/src/csharp/Grpc.HealthCheck/HealthGrpc.cs
+++ b/src/csharp/Grpc.HealthCheck/HealthGrpc.cs
@@ -4,13 +4,13 @@
// </auto-generated>
// Original file comments:
// Copyright 2015 The gRPC Authors
-//
+//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
-//
+//
// http://www.apache.org/licenses/LICENSE-2.0
-//
+//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
diff --git a/src/csharp/Grpc.IntegrationTesting/BenchmarkServiceGrpc.cs b/src/csharp/Grpc.IntegrationTesting/BenchmarkServiceGrpc.cs
index 5f18ba7accf..09691d28716 100644
--- a/src/csharp/Grpc.IntegrationTesting/BenchmarkServiceGrpc.cs
+++ b/src/csharp/Grpc.IntegrationTesting/BenchmarkServiceGrpc.cs
@@ -4,13 +4,13 @@
// </auto-generated>
// Original file comments:
// Copyright 2015 gRPC authors.
-//
+//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
-//
+//
// http://www.apache.org/licenses/LICENSE-2.0
-//
+//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
diff --git a/src/csharp/Grpc.IntegrationTesting/Control.cs b/src/csharp/Grpc.IntegrationTesting/Control.cs
index 2e80dac074c..3cac3b9d759 100644
--- a/src/csharp/Grpc.IntegrationTesting/Control.cs
+++ b/src/csharp/Grpc.IntegrationTesting/Control.cs
@@ -1502,7 +1502,7 @@ namespace Grpc.Testing {
}
if (other.securityParams_ != null) {
if (securityParams_ == null) {
- securityParams_ = new global::Grpc.Testing.SecurityParams();
+ SecurityParams = new global::Grpc.Testing.SecurityParams();
}
SecurityParams.MergeFrom(other.SecurityParams);
}
@@ -1520,19 +1520,19 @@ namespace Grpc.Testing {
}
if (other.loadParams_ != null) {
if (loadParams_ == null) {
- loadParams_ = new global::Grpc.Testing.LoadParams();
+ LoadParams = new global::Grpc.Testing.LoadParams();
}
LoadParams.MergeFrom(other.LoadParams);
}
if (other.payloadConfig_ != null) {
if (payloadConfig_ == null) {
- payloadConfig_ = new global::Grpc.Testing.PayloadConfig();
+ PayloadConfig = new global::Grpc.Testing.PayloadConfig();
}
PayloadConfig.MergeFrom(other.PayloadConfig);
}
if (other.histogramParams_ != null) {
if (histogramParams_ == null) {
- histogramParams_ = new global::Grpc.Testing.HistogramParams();
+ HistogramParams = new global::Grpc.Testing.HistogramParams();
}
HistogramParams.MergeFrom(other.HistogramParams);
}
@@ -1572,14 +1572,14 @@ namespace Grpc.Testing {
break;
}
case 16: {
- clientType_ = (global::Grpc.Testing.ClientType) input.ReadEnum();
+ ClientType = (global::Grpc.Testing.ClientType) input.ReadEnum();
break;
}
case 26: {
if (securityParams_ == null) {
- securityParams_ = new global::Grpc.Testing.SecurityParams();
+ SecurityParams = new global::Grpc.Testing.SecurityParams();
}
- input.ReadMessage(securityParams_);
+ input.ReadMessage(SecurityParams);
break;
}
case 32: {
@@ -1595,28 +1595,28 @@ namespace Grpc.Testing {
break;
}
case 64: {
- rpcType_ = (global::Grpc.Testing.RpcType) input.ReadEnum();
+ RpcType = (global::Grpc.Testing.RpcType) input.ReadEnum();
break;
}
case 82: {
if (loadParams_ == null) {
- loadParams_ = new global::Grpc.Testing.LoadParams();
+ LoadParams = new global::Grpc.Testing.LoadParams();
}
- input.ReadMessage(loadParams_);
+ input.ReadMessage(LoadParams);
break;
}
case 90: {
if (payloadConfig_ == null) {
- payloadConfig_ = new global::Grpc.Testing.PayloadConfig();
+ PayloadConfig = new global::Grpc.Testing.PayloadConfig();
}
- input.ReadMessage(payloadConfig_);
+ input.ReadMessage(PayloadConfig);
break;
}
case 98: {
if (histogramParams_ == null) {
- histogramParams_ = new global::Grpc.Testing.HistogramParams();
+ HistogramParams = new global::Grpc.Testing.HistogramParams();
}
- input.ReadMessage(histogramParams_);
+ input.ReadMessage(HistogramParams);
break;
}
case 106:
@@ -1765,7 +1765,7 @@ namespace Grpc.Testing {
}
if (other.stats_ != null) {
if (stats_ == null) {
- stats_ = new global::Grpc.Testing.ClientStats();
+ Stats = new global::Grpc.Testing.ClientStats();
}
Stats.MergeFrom(other.Stats);
}
@@ -1782,9 +1782,9 @@ namespace Grpc.Testing {
break;
case 10: {
if (stats_ == null) {
- stats_ = new global::Grpc.Testing.ClientStats();
+ Stats = new global::Grpc.Testing.ClientStats();
}
- input.ReadMessage(stats_);
+ input.ReadMessage(Stats);
break;
}
}
@@ -2467,7 +2467,7 @@ namespace Grpc.Testing {
}
if (other.securityParams_ != null) {
if (securityParams_ == null) {
- securityParams_ = new global::Grpc.Testing.SecurityParams();
+ SecurityParams = new global::Grpc.Testing.SecurityParams();
}
SecurityParams.MergeFrom(other.SecurityParams);
}
@@ -2482,7 +2482,7 @@ namespace Grpc.Testing {
}
if (other.payloadConfig_ != null) {
if (payloadConfig_ == null) {
- payloadConfig_ = new global::Grpc.Testing.PayloadConfig();
+ PayloadConfig = new global::Grpc.Testing.PayloadConfig();
}
PayloadConfig.MergeFrom(other.PayloadConfig);
}
@@ -2509,14 +2509,14 @@ namespace Grpc.Testing {
_unknownFields = pb::UnknownFieldSet.MergeFieldFrom(_unknownFields, input);
break;
case 8: {
- serverType_ = (global::Grpc.Testing.ServerType) input.ReadEnum();
+ ServerType = (global::Grpc.Testing.ServerType) input.ReadEnum();
break;
}
case 18: {
if (securityParams_ == null) {
- securityParams_ = new global::Grpc.Testing.SecurityParams();
+ SecurityParams = new global::Grpc.Testing.SecurityParams();
}
- input.ReadMessage(securityParams_);
+ input.ReadMessage(SecurityParams);
break;
}
case 32: {
@@ -2533,9 +2533,9 @@ namespace Grpc.Testing {
}
case 74: {
if (payloadConfig_ == null) {
- payloadConfig_ = new global::Grpc.Testing.PayloadConfig();
+ PayloadConfig = new global::Grpc.Testing.PayloadConfig();
}
- input.ReadMessage(payloadConfig_);
+ input.ReadMessage(PayloadConfig);
break;
}
case 82:
@@ -2924,7 +2924,7 @@ namespace Grpc.Testing {
}
if (other.stats_ != null) {
if (stats_ == null) {
- stats_ = new global::Grpc.Testing.ServerStats();
+ Stats = new global::Grpc.Testing.ServerStats();
}
Stats.MergeFrom(other.Stats);
}
@@ -2947,9 +2947,9 @@ namespace Grpc.Testing {
break;
case 10: {
if (stats_ == null) {
- stats_ = new global::Grpc.Testing.ServerStats();
+ Stats = new global::Grpc.Testing.ServerStats();
}
- input.ReadMessage(stats_);
+ input.ReadMessage(Stats);
break;
}
case 16: {
@@ -3584,7 +3584,7 @@ namespace Grpc.Testing {
}
if (other.clientConfig_ != null) {
if (clientConfig_ == null) {
- clientConfig_ = new global::Grpc.Testing.ClientConfig();
+ ClientConfig = new global::Grpc.Testing.ClientConfig();
}
ClientConfig.MergeFrom(other.ClientConfig);
}
@@ -3593,7 +3593,7 @@ namespace Grpc.Testing {
}
if (other.serverConfig_ != null) {
if (serverConfig_ == null) {
- serverConfig_ = new global::Grpc.Testing.ServerConfig();
+ ServerConfig = new global::Grpc.Testing.ServerConfig();
}
ServerConfig.MergeFrom(other.ServerConfig);
}
@@ -3626,9 +3626,9 @@ namespace Grpc.Testing {
}
case 18: {
if (clientConfig_ == null) {
- clientConfig_ = new global::Grpc.Testing.ClientConfig();
+ ClientConfig = new global::Grpc.Testing.ClientConfig();
}
- input.ReadMessage(clientConfig_);
+ input.ReadMessage(ClientConfig);
break;
}
case 24: {
@@ -3637,9 +3637,9 @@ namespace Grpc.Testing {
}
case 34: {
if (serverConfig_ == null) {
- serverConfig_ = new global::Grpc.Testing.ServerConfig();
+ ServerConfig = new global::Grpc.Testing.ServerConfig();
}
- input.ReadMessage(serverConfig_);
+ input.ReadMessage(ServerConfig);
break;
}
case 40: {
@@ -4696,13 +4696,13 @@ namespace Grpc.Testing {
}
if (other.scenario_ != null) {
if (scenario_ == null) {
- scenario_ = new global::Grpc.Testing.Scenario();
+ Scenario = new global::Grpc.Testing.Scenario();
}
Scenario.MergeFrom(other.Scenario);
}
if (other.latencies_ != null) {
if (latencies_ == null) {
- latencies_ = new global::Grpc.Testing.HistogramData();
+ Latencies = new global::Grpc.Testing.HistogramData();
}
Latencies.MergeFrom(other.Latencies);
}
@@ -4711,7 +4711,7 @@ namespace Grpc.Testing {
serverCores_.Add(other.serverCores_);
if (other.summary_ != null) {
if (summary_ == null) {
- summary_ = new global::Grpc.Testing.ScenarioResultSummary();
+ Summary = new global::Grpc.Testing.ScenarioResultSummary();
}
Summary.MergeFrom(other.Summary);
}
@@ -4731,16 +4731,16 @@ namespace Grpc.Testing {
break;
case 10: {
if (scenario_ == null) {
- scenario_ = new global::Grpc.Testing.Scenario();
+ Scenario = new global::Grpc.Testing.Scenario();
}
- input.ReadMessage(scenario_);
+ input.ReadMessage(Scenario);
break;
}
case 18: {
if (latencies_ == null) {
- latencies_ = new global::Grpc.Testing.HistogramData();
+ Latencies = new global::Grpc.Testing.HistogramData();
}
- input.ReadMessage(latencies_);
+ input.ReadMessage(Latencies);
break;
}
case 26: {
@@ -4758,9 +4758,9 @@ namespace Grpc.Testing {
}
case 50: {
if (summary_ == null) {
- summary_ = new global::Grpc.Testing.ScenarioResultSummary();
+ Summary = new global::Grpc.Testing.ScenarioResultSummary();
}
- input.ReadMessage(summary_);
+ input.ReadMessage(Summary);
break;
}
case 58:
diff --git a/src/csharp/Grpc.IntegrationTesting/EchoMessages.cs b/src/csharp/Grpc.IntegrationTesting/EchoMessages.cs
index 80a1007e9a5..e5af4a93e99 100644
--- a/src/csharp/Grpc.IntegrationTesting/EchoMessages.cs
+++ b/src/csharp/Grpc.IntegrationTesting/EchoMessages.cs
@@ -864,7 +864,7 @@ namespace Grpc.Testing {
}
if (other.debugInfo_ != null) {
if (debugInfo_ == null) {
- debugInfo_ = new global::Grpc.Testing.DebugInfo();
+ DebugInfo = new global::Grpc.Testing.DebugInfo();
}
DebugInfo.MergeFrom(other.DebugInfo);
}
@@ -876,7 +876,7 @@ namespace Grpc.Testing {
}
if (other.expectedError_ != null) {
if (expectedError_ == null) {
- expectedError_ = new global::Grpc.Testing.ErrorStatus();
+ ExpectedError = new global::Grpc.Testing.ErrorStatus();
}
ExpectedError.MergeFrom(other.ExpectedError);
}
@@ -939,9 +939,9 @@ namespace Grpc.Testing {
}
case 90: {
if (debugInfo_ == null) {
- debugInfo_ = new global::Grpc.Testing.DebugInfo();
+ DebugInfo = new global::Grpc.Testing.DebugInfo();
}
- input.ReadMessage(debugInfo_);
+ input.ReadMessage(DebugInfo);
break;
}
case 96: {
@@ -954,9 +954,9 @@ namespace Grpc.Testing {
}
case 114: {
if (expectedError_ == null) {
- expectedError_ = new global::Grpc.Testing.ErrorStatus();
+ ExpectedError = new global::Grpc.Testing.ErrorStatus();
}
- input.ReadMessage(expectedError_);
+ input.ReadMessage(ExpectedError);
break;
}
case 120: {
@@ -1104,7 +1104,7 @@ namespace Grpc.Testing {
}
if (other.param_ != null) {
if (param_ == null) {
- param_ = new global::Grpc.Testing.RequestParams();
+ Param = new global::Grpc.Testing.RequestParams();
}
Param.MergeFrom(other.Param);
}
@@ -1125,9 +1125,9 @@ namespace Grpc.Testing {
}
case 18: {
if (param_ == null) {
- param_ = new global::Grpc.Testing.RequestParams();
+ Param = new global::Grpc.Testing.RequestParams();
}
- input.ReadMessage(param_);
+ input.ReadMessage(Param);
break;
}
}
@@ -1452,7 +1452,7 @@ namespace Grpc.Testing {
}
if (other.param_ != null) {
if (param_ == null) {
- param_ = new global::Grpc.Testing.ResponseParams();
+ Param = new global::Grpc.Testing.ResponseParams();
}
Param.MergeFrom(other.Param);
}
@@ -1473,9 +1473,9 @@ namespace Grpc.Testing {
}
case 18: {
if (param_ == null) {
- param_ = new global::Grpc.Testing.ResponseParams();
+ Param = new global::Grpc.Testing.ResponseParams();
}
- input.ReadMessage(param_);
+ input.ReadMessage(Param);
break;
}
}
diff --git a/src/csharp/Grpc.IntegrationTesting/Empty.cs b/src/csharp/Grpc.IntegrationTesting/Empty.cs
index 389fe433755..0d4c28bf7fc 100644
--- a/src/csharp/Grpc.IntegrationTesting/Empty.cs
+++ b/src/csharp/Grpc.IntegrationTesting/Empty.cs
@@ -44,7 +44,6 @@ namespace Grpc.Testing {
/// service Foo {
/// rpc Bar (grpc.testing.Empty) returns (grpc.testing.Empty) { };
/// };
- ///
/// </summary>
public sealed partial class Empty : pb::IMessage<Empty> {
private static readonly pb::MessageParser<Empty> _parser = new pb::MessageParser<Empty>(() => new Empty());
diff --git a/src/csharp/Grpc.IntegrationTesting/EmptyServiceGrpc.cs b/src/csharp/Grpc.IntegrationTesting/EmptyServiceGrpc.cs
index 01af6c24f41..bfa3348f6a0 100644
--- a/src/csharp/Grpc.IntegrationTesting/EmptyServiceGrpc.cs
+++ b/src/csharp/Grpc.IntegrationTesting/EmptyServiceGrpc.cs
@@ -4,13 +4,13 @@
// </auto-generated>
// Original file comments:
// Copyright 2018 gRPC authors.
-//
+//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
-//
+//
// http://www.apache.org/licenses/LICENSE-2.0
-//
+//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
diff --git a/src/csharp/Grpc.IntegrationTesting/Messages.cs b/src/csharp/Grpc.IntegrationTesting/Messages.cs
index 35546f1b671..3b6c0010222 100644
--- a/src/csharp/Grpc.IntegrationTesting/Messages.cs
+++ b/src/csharp/Grpc.IntegrationTesting/Messages.cs
@@ -379,7 +379,7 @@ namespace Grpc.Testing {
_unknownFields = pb::UnknownFieldSet.MergeFieldFrom(_unknownFields, input);
break;
case 8: {
- type_ = (global::Grpc.Testing.PayloadType) input.ReadEnum();
+ Type = (global::Grpc.Testing.PayloadType) input.ReadEnum();
break;
}
case 18: {
@@ -844,7 +844,7 @@ namespace Grpc.Testing {
}
if (other.payload_ != null) {
if (payload_ == null) {
- payload_ = new global::Grpc.Testing.Payload();
+ Payload = new global::Grpc.Testing.Payload();
}
Payload.MergeFrom(other.Payload);
}
@@ -856,19 +856,19 @@ namespace Grpc.Testing {
}
if (other.responseCompressed_ != null) {
if (responseCompressed_ == null) {
- responseCompressed_ = new global::Grpc.Testing.BoolValue();
+ ResponseCompressed = new global::Grpc.Testing.BoolValue();
}
ResponseCompressed.MergeFrom(other.ResponseCompressed);
}
if (other.responseStatus_ != null) {
if (responseStatus_ == null) {
- responseStatus_ = new global::Grpc.Testing.EchoStatus();
+ ResponseStatus = new global::Grpc.Testing.EchoStatus();
}
ResponseStatus.MergeFrom(other.ResponseStatus);
}
if (other.expectCompressed_ != null) {
if (expectCompressed_ == null) {
- expectCompressed_ = new global::Grpc.Testing.BoolValue();
+ ExpectCompressed = new global::Grpc.Testing.BoolValue();
}
ExpectCompressed.MergeFrom(other.ExpectCompressed);
}
@@ -884,7 +884,7 @@ namespace Grpc.Testing {
_unknownFields = pb::UnknownFieldSet.MergeFieldFrom(_unknownFields, input);
break;
case 8: {
- responseType_ = (global::Grpc.Testing.PayloadType) input.ReadEnum();
+ ResponseType = (global::Grpc.Testing.PayloadType) input.ReadEnum();
break;
}
case 16: {
@@ -893,9 +893,9 @@ namespace Grpc.Testing {
}
case 26: {
if (payload_ == null) {
- payload_ = new global::Grpc.Testing.Payload();
+ Payload = new global::Grpc.Testing.Payload();
}
- input.ReadMessage(payload_);
+ input.ReadMessage(Payload);
break;
}
case 32: {
@@ -908,23 +908,23 @@ namespace Grpc.Testing {
}
case 50: {
if (responseCompressed_ == null) {
- responseCompressed_ = new global::Grpc.Testing.BoolValue();
+ ResponseCompressed = new global::Grpc.Testing.BoolValue();
}
- input.ReadMessage(responseCompressed_);
+ input.ReadMessage(ResponseCompressed);
break;
}
case 58: {
if (responseStatus_ == null) {
- responseStatus_ = new global::Grpc.Testing.EchoStatus();
+ ResponseStatus = new global::Grpc.Testing.EchoStatus();
}
- input.ReadMessage(responseStatus_);
+ input.ReadMessage(ResponseStatus);
break;
}
case 66: {
if (expectCompressed_ == null) {
- expectCompressed_ = new global::Grpc.Testing.BoolValue();
+ ExpectCompressed = new global::Grpc.Testing.BoolValue();
}
- input.ReadMessage(expectCompressed_);
+ input.ReadMessage(ExpectCompressed);
break;
}
}
@@ -1095,7 +1095,7 @@ namespace Grpc.Testing {
}
if (other.payload_ != null) {
if (payload_ == null) {
- payload_ = new global::Grpc.Testing.Payload();
+ Payload = new global::Grpc.Testing.Payload();
}
Payload.MergeFrom(other.Payload);
}
@@ -1118,9 +1118,9 @@ namespace Grpc.Testing {
break;
case 10: {
if (payload_ == null) {
- payload_ = new global::Grpc.Testing.Payload();
+ Payload = new global::Grpc.Testing.Payload();
}
- input.ReadMessage(payload_);
+ input.ReadMessage(Payload);
break;
}
case 18: {
@@ -1277,13 +1277,13 @@ namespace Grpc.Testing {
}
if (other.payload_ != null) {
if (payload_ == null) {
- payload_ = new global::Grpc.Testing.Payload();
+ Payload = new global::Grpc.Testing.Payload();
}
Payload.MergeFrom(other.Payload);
}
if (other.expectCompressed_ != null) {
if (expectCompressed_ == null) {
- expectCompressed_ = new global::Grpc.Testing.BoolValue();
+ ExpectCompressed = new global::Grpc.Testing.BoolValue();
}
ExpectCompressed.MergeFrom(other.ExpectCompressed);
}
@@ -1300,16 +1300,16 @@ namespace Grpc.Testing {
break;
case 10: {
if (payload_ == null) {
- payload_ = new global::Grpc.Testing.Payload();
+ Payload = new global::Grpc.Testing.Payload();
}
- input.ReadMessage(payload_);
+ input.ReadMessage(Payload);
break;
}
case 18: {
if (expectCompressed_ == null) {
- expectCompressed_ = new global::Grpc.Testing.BoolValue();
+ ExpectCompressed = new global::Grpc.Testing.BoolValue();
}
- input.ReadMessage(expectCompressed_);
+ input.ReadMessage(ExpectCompressed);
break;
}
}
@@ -1624,7 +1624,7 @@ namespace Grpc.Testing {
}
if (other.compressed_ != null) {
if (compressed_ == null) {
- compressed_ = new global::Grpc.Testing.BoolValue();
+ Compressed = new global::Grpc.Testing.BoolValue();
}
Compressed.MergeFrom(other.Compressed);
}
@@ -1649,9 +1649,9 @@ namespace Grpc.Testing {
}
case 26: {
if (compressed_ == null) {
- compressed_ = new global::Grpc.Testing.BoolValue();
+ Compressed = new global::Grpc.Testing.BoolValue();
}
- input.ReadMessage(compressed_);
+ input.ReadMessage(Compressed);
break;
}
}
@@ -1846,13 +1846,13 @@ namespace Grpc.Testing {
responseParameters_.Add(other.responseParameters_);
if (other.payload_ != null) {
if (payload_ == null) {
- payload_ = new global::Grpc.Testing.Payload();
+ Payload = new global::Grpc.Testing.Payload();
}
Payload.MergeFrom(other.Payload);
}
if (other.responseStatus_ != null) {
if (responseStatus_ == null) {
- responseStatus_ = new global::Grpc.Testing.EchoStatus();
+ ResponseStatus = new global::Grpc.Testing.EchoStatus();
}
ResponseStatus.MergeFrom(other.ResponseStatus);
}
@@ -1868,7 +1868,7 @@ namespace Grpc.Testing {
_unknownFields = pb::UnknownFieldSet.MergeFieldFrom(_unknownFields, input);
break;
case 8: {
- responseType_ = (global::Grpc.Testing.PayloadType) input.ReadEnum();
+ ResponseType = (global::Grpc.Testing.PayloadType) input.ReadEnum();
break;
}
case 18: {
@@ -1877,16 +1877,16 @@ namespace Grpc.Testing {
}
case 26: {
if (payload_ == null) {
- payload_ = new global::Grpc.Testing.Payload();
+ Payload = new global::Grpc.Testing.Payload();
}
- input.ReadMessage(payload_);
+ input.ReadMessage(Payload);
break;
}
case 58: {
if (responseStatus_ == null) {
- responseStatus_ = new global::Grpc.Testing.EchoStatus();
+ ResponseStatus = new global::Grpc.Testing.EchoStatus();
}
- input.ReadMessage(responseStatus_);
+ input.ReadMessage(ResponseStatus);
break;
}
}
@@ -2008,7 +2008,7 @@ namespace Grpc.Testing {
}
if (other.payload_ != null) {
if (payload_ == null) {
- payload_ = new global::Grpc.Testing.Payload();
+ Payload = new global::Grpc.Testing.Payload();
}
Payload.MergeFrom(other.Payload);
}
@@ -2025,9 +2025,9 @@ namespace Grpc.Testing {
break;
case 10: {
if (payload_ == null) {
- payload_ = new global::Grpc.Testing.Payload();
+ Payload = new global::Grpc.Testing.Payload();
}
- input.ReadMessage(payload_);
+ input.ReadMessage(Payload);
break;
}
}
diff --git a/src/csharp/Grpc.IntegrationTesting/MetricsGrpc.cs b/src/csharp/Grpc.IntegrationTesting/MetricsGrpc.cs
index 7b5b1a3aa7f..27746c07641 100644
--- a/src/csharp/Grpc.IntegrationTesting/MetricsGrpc.cs
+++ b/src/csharp/Grpc.IntegrationTesting/MetricsGrpc.cs
@@ -4,13 +4,13 @@
// </auto-generated>
// Original file comments:
// Copyright 2015-2016 gRPC authors.
-//
+//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
-//
+//
// http://www.apache.org/licenses/LICENSE-2.0
-//
+//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -19,7 +19,7 @@
//
// Contains the definitions for a metrics service and the type of metrics
// exposed by the service.
-//
+//
// Currently, 'Gauge' (i.e a metric that represents the measured value of
// something at an instant of time) is the only metric type supported by the
// service.
diff --git a/src/csharp/Grpc.IntegrationTesting/ReportQpsScenarioServiceGrpc.cs b/src/csharp/Grpc.IntegrationTesting/ReportQpsScenarioServiceGrpc.cs
index 04bb9c29d63..f92ae8e974b 100644
--- a/src/csharp/Grpc.IntegrationTesting/ReportQpsScenarioServiceGrpc.cs
+++ b/src/csharp/Grpc.IntegrationTesting/ReportQpsScenarioServiceGrpc.cs
@@ -4,13 +4,13 @@
// </auto-generated>
// Original file comments:
// Copyright 2015 gRPC authors.
-//
+//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
-//
+//
// http://www.apache.org/licenses/LICENSE-2.0
-//
+//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
diff --git a/src/csharp/Grpc.IntegrationTesting/Stats.cs b/src/csharp/Grpc.IntegrationTesting/Stats.cs
index af83eef7ba8..c3e5664e11a 100644
--- a/src/csharp/Grpc.IntegrationTesting/Stats.cs
+++ b/src/csharp/Grpc.IntegrationTesting/Stats.cs
@@ -328,7 +328,7 @@ namespace Grpc.Testing {
}
if (other.coreStats_ != null) {
if (coreStats_ == null) {
- coreStats_ = new global::Grpc.Core.Stats();
+ CoreStats = new global::Grpc.Core.Stats();
}
CoreStats.MergeFrom(other.CoreStats);
}
@@ -369,9 +369,9 @@ namespace Grpc.Testing {
}
case 58: {
if (coreStats_ == null) {
- coreStats_ = new global::Grpc.Core.Stats();
+ CoreStats = new global::Grpc.Core.Stats();
}
- input.ReadMessage(coreStats_);
+ input.ReadMessage(CoreStats);
break;
}
}
@@ -1210,7 +1210,7 @@ namespace Grpc.Testing {
}
if (other.latencies_ != null) {
if (latencies_ == null) {
- latencies_ = new global::Grpc.Testing.HistogramData();
+ Latencies = new global::Grpc.Testing.HistogramData();
}
Latencies.MergeFrom(other.Latencies);
}
@@ -1229,7 +1229,7 @@ namespace Grpc.Testing {
}
if (other.coreStats_ != null) {
if (coreStats_ == null) {
- coreStats_ = new global::Grpc.Core.Stats();
+ CoreStats = new global::Grpc.Core.Stats();
}
CoreStats.MergeFrom(other.CoreStats);
}
@@ -1246,9 +1246,9 @@ namespace Grpc.Testing {
break;
case 10: {
if (latencies_ == null) {
- latencies_ = new global::Grpc.Testing.HistogramData();
+ Latencies = new global::Grpc.Testing.HistogramData();
}
- input.ReadMessage(latencies_);
+ input.ReadMessage(Latencies);
break;
}
case 17: {
@@ -1273,9 +1273,9 @@ namespace Grpc.Testing {
}
case 58: {
if (coreStats_ == null) {
- coreStats_ = new global::Grpc.Core.Stats();
+ CoreStats = new global::Grpc.Core.Stats();
}
- input.ReadMessage(coreStats_);
+ input.ReadMessage(CoreStats);
break;
}
}
diff --git a/src/csharp/Grpc.IntegrationTesting/TestGrpc.cs b/src/csharp/Grpc.IntegrationTesting/TestGrpc.cs
index 05e1e3ccc7d..d47b5fe0d4b 100644
--- a/src/csharp/Grpc.IntegrationTesting/TestGrpc.cs
+++ b/src/csharp/Grpc.IntegrationTesting/TestGrpc.cs
@@ -4,13 +4,13 @@
// </auto-generated>
// Original file comments:
// Copyright 2015-2016 gRPC authors.
-//
+//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
-//
+//
// http://www.apache.org/licenses/LICENSE-2.0
-//
+//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
diff --git a/src/csharp/Grpc.IntegrationTesting/WorkerServiceGrpc.cs b/src/csharp/Grpc.IntegrationTesting/WorkerServiceGrpc.cs
index a36f1d7a356..f7dd2eecf2e 100644
--- a/src/csharp/Grpc.IntegrationTesting/WorkerServiceGrpc.cs
+++ b/src/csharp/Grpc.IntegrationTesting/WorkerServiceGrpc.cs
@@ -4,13 +4,13 @@
// </auto-generated>
// Original file comments:
// Copyright 2015 gRPC authors.
-//
+//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
-//
+//
// http://www.apache.org/licenses/LICENSE-2.0
-//
+//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
diff --git a/src/csharp/Grpc.Reflection/Reflection.cs b/src/csharp/Grpc.Reflection/Reflection.cs
index e319be5bff7..a1b99dff431 100644
--- a/src/csharp/Grpc.Reflection/Reflection.cs
+++ b/src/csharp/Grpc.Reflection/Reflection.cs
@@ -850,7 +850,7 @@ namespace Grpc.Reflection.V1Alpha {
}
if (other.originalRequest_ != null) {
if (originalRequest_ == null) {
- originalRequest_ = new global::Grpc.Reflection.V1Alpha.ServerReflectionRequest();
+ OriginalRequest = new global::Grpc.Reflection.V1Alpha.ServerReflectionRequest();
}
OriginalRequest.MergeFrom(other.OriginalRequest);
}
@@ -898,9 +898,9 @@ namespace Grpc.Reflection.V1Alpha {
}
case 18: {
if (originalRequest_ == null) {
- originalRequest_ = new global::Grpc.Reflection.V1Alpha.ServerReflectionRequest();
+ OriginalRequest = new global::Grpc.Reflection.V1Alpha.ServerReflectionRequest();
}
- input.ReadMessage(originalRequest_);
+ input.ReadMessage(OriginalRequest);
break;
}
case 34: {
diff --git a/src/csharp/Grpc.Reflection/ReflectionGrpc.cs b/src/csharp/Grpc.Reflection/ReflectionGrpc.cs
index 0b2bb2341b9..500738205a7 100644
--- a/src/csharp/Grpc.Reflection/ReflectionGrpc.cs
+++ b/src/csharp/Grpc.Reflection/ReflectionGrpc.cs
@@ -4,13 +4,13 @@
// </auto-generated>
// Original file comments:
// Copyright 2016 gRPC authors.
-//
+//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
-//
+//
// http://www.apache.org/licenses/LICENSE-2.0
-//
+//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
From b667c2f72ffecb1de672f5a7fb837737bf9ee405 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Mon, 11 Mar 2019 15:00:56 -0400
Subject: [PATCH 094/138] regenerate ruby protos
---
src/ruby/bin/math_pb.rb | 34 ++-
src/ruby/pb/grpc/health/v1/health_pb.rb | 23 +-
.../pb/grpc/health/v1/health_services_pb.rb | 18 ++
.../pb/src/proto/grpc/testing/empty_pb.rb | 4 +-
.../pb/src/proto/grpc/testing/messages_pb.rb | 114 +++----
src/ruby/pb/src/proto/grpc/testing/test_pb.rb | 2 +
src/ruby/qps/src/proto/grpc/core/stats_pb.rb | 32 +-
.../grpc/testing/benchmark_service_pb.rb | 2 +
.../qps/src/proto/grpc/testing/control_pb.rb | 289 +++++++++---------
.../qps/src/proto/grpc/testing/messages_pb.rb | 114 +++----
.../qps/src/proto/grpc/testing/payloads_pb.rb | 32 +-
.../testing/report_qps_scenario_service_pb.rb | 2 +
.../qps/src/proto/grpc/testing/stats_pb.rb | 68 +++--
.../proto/grpc/testing/worker_service_pb.rb | 2 +
14 files changed, 392 insertions(+), 344 deletions(-)
diff --git a/src/ruby/bin/math_pb.rb b/src/ruby/bin/math_pb.rb
index 60429a15052..ac287c81bcd 100644
--- a/src/ruby/bin/math_pb.rb
+++ b/src/ruby/bin/math_pb.rb
@@ -4,22 +4,24 @@
require 'google/protobuf'
Google::Protobuf::DescriptorPool.generated_pool.build do
- add_message "math.DivArgs" do
- optional :dividend, :int64, 1
- optional :divisor, :int64, 2
- end
- add_message "math.DivReply" do
- optional :quotient, :int64, 1
- optional :remainder, :int64, 2
- end
- add_message "math.FibArgs" do
- optional :limit, :int64, 1
- end
- add_message "math.Num" do
- optional :num, :int64, 1
- end
- add_message "math.FibReply" do
- optional :count, :int64, 1
+ add_file("math.proto", :syntax => :proto3) do
+ add_message "math.DivArgs" do
+ optional :dividend, :int64, 1
+ optional :divisor, :int64, 2
+ end
+ add_message "math.DivReply" do
+ optional :quotient, :int64, 1
+ optional :remainder, :int64, 2
+ end
+ add_message "math.FibArgs" do
+ optional :limit, :int64, 1
+ end
+ add_message "math.Num" do
+ optional :num, :int64, 1
+ end
+ add_message "math.FibReply" do
+ optional :count, :int64, 1
+ end
end
end
diff --git a/src/ruby/pb/grpc/health/v1/health_pb.rb b/src/ruby/pb/grpc/health/v1/health_pb.rb
index aa87a93918b..c11dbf48418 100644
--- a/src/ruby/pb/grpc/health/v1/health_pb.rb
+++ b/src/ruby/pb/grpc/health/v1/health_pb.rb
@@ -4,16 +4,19 @@
require 'google/protobuf'
Google::Protobuf::DescriptorPool.generated_pool.build do
- add_message "grpc.health.v1.HealthCheckRequest" do
- optional :service, :string, 1
- end
- add_message "grpc.health.v1.HealthCheckResponse" do
- optional :status, :enum, 1, "grpc.health.v1.HealthCheckResponse.ServingStatus"
- end
- add_enum "grpc.health.v1.HealthCheckResponse.ServingStatus" do
- value :UNKNOWN, 0
- value :SERVING, 1
- value :NOT_SERVING, 2
+ add_file("grpc/health/v1/health.proto", :syntax => :proto3) do
+ add_message "grpc.health.v1.HealthCheckRequest" do
+ optional :service, :string, 1
+ end
+ add_message "grpc.health.v1.HealthCheckResponse" do
+ optional :status, :enum, 1, "grpc.health.v1.HealthCheckResponse.ServingStatus"
+ end
+ add_enum "grpc.health.v1.HealthCheckResponse.ServingStatus" do
+ value :UNKNOWN, 0
+ value :SERVING, 1
+ value :NOT_SERVING, 2
+ value :SERVICE_UNKNOWN, 3
+ end
end
end
diff --git a/src/ruby/pb/grpc/health/v1/health_services_pb.rb b/src/ruby/pb/grpc/health/v1/health_services_pb.rb
index 169e160f90f..5992f1c403d 100644
--- a/src/ruby/pb/grpc/health/v1/health_services_pb.rb
+++ b/src/ruby/pb/grpc/health/v1/health_services_pb.rb
@@ -34,7 +34,25 @@ module Grpc
self.unmarshal_class_method = :decode
self.service_name = 'grpc.health.v1.Health'
+ # If the requested service is unknown, the call will fail with status
+ # NOT_FOUND.
rpc :Check, HealthCheckRequest, HealthCheckResponse
+ # Performs a watch for the serving status of the requested service.
+ # The server will immediately send back a message indicating the current
+ # serving status. It will then subsequently send a new message whenever
+ # the service's serving status changes.
+ #
+ # If the requested service is unknown when the call is received, the
+ # server will send a message setting the serving status to
+ # SERVICE_UNKNOWN but will *not* terminate the call. If at some
+ # future point, the serving status of the service becomes known, the
+ # server will send a new message with the service's serving status.
+ #
+ # If the call terminates with status UNIMPLEMENTED, then clients
+ # should assume this method is not supported and should not retry the
+ # call. If the call terminates with any other status (including OK),
+ # clients should retry the call with appropriate exponential backoff.
+ rpc :Watch, HealthCheckRequest, stream(HealthCheckResponse)
end
Stub = Service.rpc_stub_class
diff --git a/src/ruby/pb/src/proto/grpc/testing/empty_pb.rb b/src/ruby/pb/src/proto/grpc/testing/empty_pb.rb
index 9c2568d6053..3e46d8525ed 100644
--- a/src/ruby/pb/src/proto/grpc/testing/empty_pb.rb
+++ b/src/ruby/pb/src/proto/grpc/testing/empty_pb.rb
@@ -4,7 +4,9 @@
require 'google/protobuf'
Google::Protobuf::DescriptorPool.generated_pool.build do
- add_message "grpc.testing.Empty" do
+ add_file("src/proto/grpc/testing/empty.proto", :syntax => :proto3) do
+ add_message "grpc.testing.Empty" do
+ end
end
end
diff --git a/src/ruby/pb/src/proto/grpc/testing/messages_pb.rb b/src/ruby/pb/src/proto/grpc/testing/messages_pb.rb
index e27ccd0dc04..796d4bb9ae2 100644
--- a/src/ruby/pb/src/proto/grpc/testing/messages_pb.rb
+++ b/src/ruby/pb/src/proto/grpc/testing/messages_pb.rb
@@ -4,62 +4,64 @@
require 'google/protobuf'
Google::Protobuf::DescriptorPool.generated_pool.build do
- add_message "grpc.testing.BoolValue" do
- optional :value, :bool, 1
- end
- add_message "grpc.testing.Payload" do
- optional :type, :enum, 1, "grpc.testing.PayloadType"
- optional :body, :bytes, 2
- end
- add_message "grpc.testing.EchoStatus" do
- optional :code, :int32, 1
- optional :message, :string, 2
- end
- add_message "grpc.testing.SimpleRequest" do
- optional :response_type, :enum, 1, "grpc.testing.PayloadType"
- optional :response_size, :int32, 2
- optional :payload, :message, 3, "grpc.testing.Payload"
- optional :fill_username, :bool, 4
- optional :fill_oauth_scope, :bool, 5
- optional :response_compressed, :message, 6, "grpc.testing.BoolValue"
- optional :response_status, :message, 7, "grpc.testing.EchoStatus"
- optional :expect_compressed, :message, 8, "grpc.testing.BoolValue"
- end
- add_message "grpc.testing.SimpleResponse" do
- optional :payload, :message, 1, "grpc.testing.Payload"
- optional :username, :string, 2
- optional :oauth_scope, :string, 3
- end
- add_message "grpc.testing.StreamingInputCallRequest" do
- optional :payload, :message, 1, "grpc.testing.Payload"
- optional :expect_compressed, :message, 2, "grpc.testing.BoolValue"
- end
- add_message "grpc.testing.StreamingInputCallResponse" do
- optional :aggregated_payload_size, :int32, 1
- end
- add_message "grpc.testing.ResponseParameters" do
- optional :size, :int32, 1
- optional :interval_us, :int32, 2
- optional :compressed, :message, 3, "grpc.testing.BoolValue"
- end
- add_message "grpc.testing.StreamingOutputCallRequest" do
- optional :response_type, :enum, 1, "grpc.testing.PayloadType"
- repeated :response_parameters, :message, 2, "grpc.testing.ResponseParameters"
- optional :payload, :message, 3, "grpc.testing.Payload"
- optional :response_status, :message, 7, "grpc.testing.EchoStatus"
- end
- add_message "grpc.testing.StreamingOutputCallResponse" do
- optional :payload, :message, 1, "grpc.testing.Payload"
- end
- add_message "grpc.testing.ReconnectParams" do
- optional :max_reconnect_backoff_ms, :int32, 1
- end
- add_message "grpc.testing.ReconnectInfo" do
- optional :passed, :bool, 1
- repeated :backoff_ms, :int32, 2
- end
- add_enum "grpc.testing.PayloadType" do
- value :COMPRESSABLE, 0
+ add_file("src/proto/grpc/testing/messages.proto", :syntax => :proto3) do
+ add_message "grpc.testing.BoolValue" do
+ optional :value, :bool, 1
+ end
+ add_message "grpc.testing.Payload" do
+ optional :type, :enum, 1, "grpc.testing.PayloadType"
+ optional :body, :bytes, 2
+ end
+ add_message "grpc.testing.EchoStatus" do
+ optional :code, :int32, 1
+ optional :message, :string, 2
+ end
+ add_message "grpc.testing.SimpleRequest" do
+ optional :response_type, :enum, 1, "grpc.testing.PayloadType"
+ optional :response_size, :int32, 2
+ optional :payload, :message, 3, "grpc.testing.Payload"
+ optional :fill_username, :bool, 4
+ optional :fill_oauth_scope, :bool, 5
+ optional :response_compressed, :message, 6, "grpc.testing.BoolValue"
+ optional :response_status, :message, 7, "grpc.testing.EchoStatus"
+ optional :expect_compressed, :message, 8, "grpc.testing.BoolValue"
+ end
+ add_message "grpc.testing.SimpleResponse" do
+ optional :payload, :message, 1, "grpc.testing.Payload"
+ optional :username, :string, 2
+ optional :oauth_scope, :string, 3
+ end
+ add_message "grpc.testing.StreamingInputCallRequest" do
+ optional :payload, :message, 1, "grpc.testing.Payload"
+ optional :expect_compressed, :message, 2, "grpc.testing.BoolValue"
+ end
+ add_message "grpc.testing.StreamingInputCallResponse" do
+ optional :aggregated_payload_size, :int32, 1
+ end
+ add_message "grpc.testing.ResponseParameters" do
+ optional :size, :int32, 1
+ optional :interval_us, :int32, 2
+ optional :compressed, :message, 3, "grpc.testing.BoolValue"
+ end
+ add_message "grpc.testing.StreamingOutputCallRequest" do
+ optional :response_type, :enum, 1, "grpc.testing.PayloadType"
+ repeated :response_parameters, :message, 2, "grpc.testing.ResponseParameters"
+ optional :payload, :message, 3, "grpc.testing.Payload"
+ optional :response_status, :message, 7, "grpc.testing.EchoStatus"
+ end
+ add_message "grpc.testing.StreamingOutputCallResponse" do
+ optional :payload, :message, 1, "grpc.testing.Payload"
+ end
+ add_message "grpc.testing.ReconnectParams" do
+ optional :max_reconnect_backoff_ms, :int32, 1
+ end
+ add_message "grpc.testing.ReconnectInfo" do
+ optional :passed, :bool, 1
+ repeated :backoff_ms, :int32, 2
+ end
+ add_enum "grpc.testing.PayloadType" do
+ value :COMPRESSABLE, 0
+ end
end
end
diff --git a/src/ruby/pb/src/proto/grpc/testing/test_pb.rb b/src/ruby/pb/src/proto/grpc/testing/test_pb.rb
index 2cc98630314..ed4b5b5e1e7 100644
--- a/src/ruby/pb/src/proto/grpc/testing/test_pb.rb
+++ b/src/ruby/pb/src/proto/grpc/testing/test_pb.rb
@@ -6,6 +6,8 @@ require 'google/protobuf'
require 'src/proto/grpc/testing/empty_pb'
require 'src/proto/grpc/testing/messages_pb'
Google::Protobuf::DescriptorPool.generated_pool.build do
+ add_file("src/proto/grpc/testing/test.proto", :syntax => :proto3) do
+ end
end
module Grpc
diff --git a/src/ruby/qps/src/proto/grpc/core/stats_pb.rb b/src/ruby/qps/src/proto/grpc/core/stats_pb.rb
index 59c057820bf..b75ce043fbb 100644
--- a/src/ruby/qps/src/proto/grpc/core/stats_pb.rb
+++ b/src/ruby/qps/src/proto/grpc/core/stats_pb.rb
@@ -4,22 +4,24 @@
require 'google/protobuf'
Google::Protobuf::DescriptorPool.generated_pool.build do
- add_message "grpc.core.Bucket" do
- optional :start, :double, 1
- optional :count, :uint64, 2
- end
- add_message "grpc.core.Histogram" do
- repeated :buckets, :message, 1, "grpc.core.Bucket"
- end
- add_message "grpc.core.Metric" do
- optional :name, :string, 1
- oneof :value do
- optional :count, :uint64, 10
- optional :histogram, :message, 11, "grpc.core.Histogram"
+ add_file("src/proto/grpc/core/stats.proto", :syntax => :proto3) do
+ add_message "grpc.core.Bucket" do
+ optional :start, :double, 1
+ optional :count, :uint64, 2
+ end
+ add_message "grpc.core.Histogram" do
+ repeated :buckets, :message, 1, "grpc.core.Bucket"
+ end
+ add_message "grpc.core.Metric" do
+ optional :name, :string, 1
+ oneof :value do
+ optional :count, :uint64, 10
+ optional :histogram, :message, 11, "grpc.core.Histogram"
+ end
+ end
+ add_message "grpc.core.Stats" do
+ repeated :metrics, :message, 1, "grpc.core.Metric"
end
- end
- add_message "grpc.core.Stats" do
- repeated :metrics, :message, 1, "grpc.core.Metric"
end
end
diff --git a/src/ruby/qps/src/proto/grpc/testing/benchmark_service_pb.rb b/src/ruby/qps/src/proto/grpc/testing/benchmark_service_pb.rb
index 0bd3625f3d4..3f14f441730 100644
--- a/src/ruby/qps/src/proto/grpc/testing/benchmark_service_pb.rb
+++ b/src/ruby/qps/src/proto/grpc/testing/benchmark_service_pb.rb
@@ -5,6 +5,8 @@ require 'google/protobuf'
require 'src/proto/grpc/testing/messages_pb'
Google::Protobuf::DescriptorPool.generated_pool.build do
+ add_file("src/proto/grpc/testing/benchmark_service.proto", :syntax => :proto3) do
+ end
end
module Grpc
diff --git a/src/ruby/qps/src/proto/grpc/testing/control_pb.rb b/src/ruby/qps/src/proto/grpc/testing/control_pb.rb
index 5acc7fc0c6b..1053e504621 100644
--- a/src/ruby/qps/src/proto/grpc/testing/control_pb.rb
+++ b/src/ruby/qps/src/proto/grpc/testing/control_pb.rb
@@ -6,152 +6,157 @@ require 'google/protobuf'
require 'src/proto/grpc/testing/payloads_pb'
require 'src/proto/grpc/testing/stats_pb'
Google::Protobuf::DescriptorPool.generated_pool.build do
- add_message "grpc.testing.PoissonParams" do
- optional :offered_load, :double, 1
- end
- add_message "grpc.testing.ClosedLoopParams" do
- end
- add_message "grpc.testing.LoadParams" do
- oneof :load do
- optional :closed_loop, :message, 1, "grpc.testing.ClosedLoopParams"
- optional :poisson, :message, 2, "grpc.testing.PoissonParams"
+ add_file("src/proto/grpc/testing/control.proto", :syntax => :proto3) do
+ add_message "grpc.testing.PoissonParams" do
+ optional :offered_load, :double, 1
end
- end
- add_message "grpc.testing.SecurityParams" do
- optional :use_test_ca, :bool, 1
- optional :server_host_override, :string, 2
- optional :cred_type, :string, 3
- end
- add_message "grpc.testing.ChannelArg" do
- optional :name, :string, 1
- oneof :value do
- optional :str_value, :string, 2
- optional :int_value, :int32, 3
+ add_message "grpc.testing.ClosedLoopParams" do
end
- end
- add_message "grpc.testing.ClientConfig" do
- repeated :server_targets, :string, 1
- optional :client_type, :enum, 2, "grpc.testing.ClientType"
- optional :security_params, :message, 3, "grpc.testing.SecurityParams"
- optional :outstanding_rpcs_per_channel, :int32, 4
- optional :client_channels, :int32, 5
- optional :async_client_threads, :int32, 7
- optional :rpc_type, :enum, 8, "grpc.testing.RpcType"
- optional :load_params, :message, 10, "grpc.testing.LoadParams"
- optional :payload_config, :message, 11, "grpc.testing.PayloadConfig"
- optional :histogram_params, :message, 12, "grpc.testing.HistogramParams"
- repeated :core_list, :int32, 13
- optional :core_limit, :int32, 14
- optional :other_client_api, :string, 15
- repeated :channel_args, :message, 16, "grpc.testing.ChannelArg"
- optional :threads_per_cq, :int32, 17
- optional :messages_per_stream, :int32, 18
- optional :use_coalesce_api, :bool, 19
- end
- add_message "grpc.testing.ClientStatus" do
- optional :stats, :message, 1, "grpc.testing.ClientStats"
- end
- add_message "grpc.testing.Mark" do
- optional :reset, :bool, 1
- end
- add_message "grpc.testing.ClientArgs" do
- oneof :argtype do
- optional :setup, :message, 1, "grpc.testing.ClientConfig"
- optional :mark, :message, 2, "grpc.testing.Mark"
+ add_message "grpc.testing.LoadParams" do
+ oneof :load do
+ optional :closed_loop, :message, 1, "grpc.testing.ClosedLoopParams"
+ optional :poisson, :message, 2, "grpc.testing.PoissonParams"
+ end
end
- end
- add_message "grpc.testing.ServerConfig" do
- optional :server_type, :enum, 1, "grpc.testing.ServerType"
- optional :security_params, :message, 2, "grpc.testing.SecurityParams"
- optional :port, :int32, 4
- optional :async_server_threads, :int32, 7
- optional :core_limit, :int32, 8
- optional :payload_config, :message, 9, "grpc.testing.PayloadConfig"
- repeated :core_list, :int32, 10
- optional :other_server_api, :string, 11
- optional :threads_per_cq, :int32, 12
- optional :resource_quota_size, :int32, 1001
- repeated :channel_args, :message, 1002, "grpc.testing.ChannelArg"
- end
- add_message "grpc.testing.ServerArgs" do
- oneof :argtype do
- optional :setup, :message, 1, "grpc.testing.ServerConfig"
- optional :mark, :message, 2, "grpc.testing.Mark"
+ add_message "grpc.testing.SecurityParams" do
+ optional :use_test_ca, :bool, 1
+ optional :server_host_override, :string, 2
+ optional :cred_type, :string, 3
+ end
+ add_message "grpc.testing.ChannelArg" do
+ optional :name, :string, 1
+ oneof :value do
+ optional :str_value, :string, 2
+ optional :int_value, :int32, 3
+ end
+ end
+ add_message "grpc.testing.ClientConfig" do
+ repeated :server_targets, :string, 1
+ optional :client_type, :enum, 2, "grpc.testing.ClientType"
+ optional :security_params, :message, 3, "grpc.testing.SecurityParams"
+ optional :outstanding_rpcs_per_channel, :int32, 4
+ optional :client_channels, :int32, 5
+ optional :async_client_threads, :int32, 7
+ optional :rpc_type, :enum, 8, "grpc.testing.RpcType"
+ optional :load_params, :message, 10, "grpc.testing.LoadParams"
+ optional :payload_config, :message, 11, "grpc.testing.PayloadConfig"
+ optional :histogram_params, :message, 12, "grpc.testing.HistogramParams"
+ repeated :core_list, :int32, 13
+ optional :core_limit, :int32, 14
+ optional :other_client_api, :string, 15
+ repeated :channel_args, :message, 16, "grpc.testing.ChannelArg"
+ optional :threads_per_cq, :int32, 17
+ optional :messages_per_stream, :int32, 18
+ optional :use_coalesce_api, :bool, 19
+ optional :median_latency_collection_interval_millis, :int32, 20
+ end
+ add_message "grpc.testing.ClientStatus" do
+ optional :stats, :message, 1, "grpc.testing.ClientStats"
+ end
+ add_message "grpc.testing.Mark" do
+ optional :reset, :bool, 1
+ end
+ add_message "grpc.testing.ClientArgs" do
+ oneof :argtype do
+ optional :setup, :message, 1, "grpc.testing.ClientConfig"
+ optional :mark, :message, 2, "grpc.testing.Mark"
+ end
+ end
+ add_message "grpc.testing.ServerConfig" do
+ optional :server_type, :enum, 1, "grpc.testing.ServerType"
+ optional :security_params, :message, 2, "grpc.testing.SecurityParams"
+ optional :port, :int32, 4
+ optional :async_server_threads, :int32, 7
+ optional :core_limit, :int32, 8
+ optional :payload_config, :message, 9, "grpc.testing.PayloadConfig"
+ repeated :core_list, :int32, 10
+ optional :other_server_api, :string, 11
+ optional :threads_per_cq, :int32, 12
+ optional :resource_quota_size, :int32, 1001
+ repeated :channel_args, :message, 1002, "grpc.testing.ChannelArg"
+ end
+ add_message "grpc.testing.ServerArgs" do
+ oneof :argtype do
+ optional :setup, :message, 1, "grpc.testing.ServerConfig"
+ optional :mark, :message, 2, "grpc.testing.Mark"
+ end
+ end
+ add_message "grpc.testing.ServerStatus" do
+ optional :stats, :message, 1, "grpc.testing.ServerStats"
+ optional :port, :int32, 2
+ optional :cores, :int32, 3
+ end
+ add_message "grpc.testing.CoreRequest" do
+ end
+ add_message "grpc.testing.CoreResponse" do
+ optional :cores, :int32, 1
+ end
+ add_message "grpc.testing.Void" do
+ end
+ add_message "grpc.testing.Scenario" do
+ optional :name, :string, 1
+ optional :client_config, :message, 2, "grpc.testing.ClientConfig"
+ optional :num_clients, :int32, 3
+ optional :server_config, :message, 4, "grpc.testing.ServerConfig"
+ optional :num_servers, :int32, 5
+ optional :warmup_seconds, :int32, 6
+ optional :benchmark_seconds, :int32, 7
+ optional :spawn_local_worker_count, :int32, 8
+ end
+ add_message "grpc.testing.Scenarios" do
+ repeated :scenarios, :message, 1, "grpc.testing.Scenario"
+ end
+ add_message "grpc.testing.ScenarioResultSummary" do
+ optional :qps, :double, 1
+ optional :qps_per_server_core, :double, 2
+ optional :server_system_time, :double, 3
+ optional :server_user_time, :double, 4
+ optional :client_system_time, :double, 5
+ optional :client_user_time, :double, 6
+ optional :latency_50, :double, 7
+ optional :latency_90, :double, 8
+ optional :latency_95, :double, 9
+ optional :latency_99, :double, 10
+ optional :latency_999, :double, 11
+ optional :server_cpu_usage, :double, 12
+ optional :successful_requests_per_second, :double, 13
+ optional :failed_requests_per_second, :double, 14
+ optional :client_polls_per_request, :double, 15
+ optional :server_polls_per_request, :double, 16
+ optional :server_queries_per_cpu_sec, :double, 17
+ optional :client_queries_per_cpu_sec, :double, 18
+ end
+ add_message "grpc.testing.ScenarioResult" do
+ optional :scenario, :message, 1, "grpc.testing.Scenario"
+ optional :latencies, :message, 2, "grpc.testing.HistogramData"
+ repeated :client_stats, :message, 3, "grpc.testing.ClientStats"
+ repeated :server_stats, :message, 4, "grpc.testing.ServerStats"
+ repeated :server_cores, :int32, 5
+ optional :summary, :message, 6, "grpc.testing.ScenarioResultSummary"
+ repeated :client_success, :bool, 7
+ repeated :server_success, :bool, 8
+ repeated :request_results, :message, 9, "grpc.testing.RequestResultCount"
+ end
+ add_enum "grpc.testing.ClientType" do
+ value :SYNC_CLIENT, 0
+ value :ASYNC_CLIENT, 1
+ value :OTHER_CLIENT, 2
+ value :CALLBACK_CLIENT, 3
+ end
+ add_enum "grpc.testing.ServerType" do
+ value :SYNC_SERVER, 0
+ value :ASYNC_SERVER, 1
+ value :ASYNC_GENERIC_SERVER, 2
+ value :OTHER_SERVER, 3
+ value :CALLBACK_SERVER, 4
+ end
+ add_enum "grpc.testing.RpcType" do
+ value :UNARY, 0
+ value :STREAMING, 1
+ value :STREAMING_FROM_CLIENT, 2
+ value :STREAMING_FROM_SERVER, 3
+ value :STREAMING_BOTH_WAYS, 4
end
- end
- add_message "grpc.testing.ServerStatus" do
- optional :stats, :message, 1, "grpc.testing.ServerStats"
- optional :port, :int32, 2
- optional :cores, :int32, 3
- end
- add_message "grpc.testing.CoreRequest" do
- end
- add_message "grpc.testing.CoreResponse" do
- optional :cores, :int32, 1
- end
- add_message "grpc.testing.Void" do
- end
- add_message "grpc.testing.Scenario" do
- optional :name, :string, 1
- optional :client_config, :message, 2, "grpc.testing.ClientConfig"
- optional :num_clients, :int32, 3
- optional :server_config, :message, 4, "grpc.testing.ServerConfig"
- optional :num_servers, :int32, 5
- optional :warmup_seconds, :int32, 6
- optional :benchmark_seconds, :int32, 7
- optional :spawn_local_worker_count, :int32, 8
- end
- add_message "grpc.testing.Scenarios" do
- repeated :scenarios, :message, 1, "grpc.testing.Scenario"
- end
- add_message "grpc.testing.ScenarioResultSummary" do
- optional :qps, :double, 1
- optional :qps_per_server_core, :double, 2
- optional :server_system_time, :double, 3
- optional :server_user_time, :double, 4
- optional :client_system_time, :double, 5
- optional :client_user_time, :double, 6
- optional :latency_50, :double, 7
- optional :latency_90, :double, 8
- optional :latency_95, :double, 9
- optional :latency_99, :double, 10
- optional :latency_999, :double, 11
- optional :server_cpu_usage, :double, 12
- optional :successful_requests_per_second, :double, 13
- optional :failed_requests_per_second, :double, 14
- optional :client_polls_per_request, :double, 15
- optional :server_polls_per_request, :double, 16
- optional :server_queries_per_cpu_sec, :double, 17
- optional :client_queries_per_cpu_sec, :double, 18
- end
- add_message "grpc.testing.ScenarioResult" do
- optional :scenario, :message, 1, "grpc.testing.Scenario"
- optional :latencies, :message, 2, "grpc.testing.HistogramData"
- repeated :client_stats, :message, 3, "grpc.testing.ClientStats"
- repeated :server_stats, :message, 4, "grpc.testing.ServerStats"
- repeated :server_cores, :int32, 5
- optional :summary, :message, 6, "grpc.testing.ScenarioResultSummary"
- repeated :client_success, :bool, 7
- repeated :server_success, :bool, 8
- repeated :request_results, :message, 9, "grpc.testing.RequestResultCount"
- end
- add_enum "grpc.testing.ClientType" do
- value :SYNC_CLIENT, 0
- value :ASYNC_CLIENT, 1
- value :OTHER_CLIENT, 2
- end
- add_enum "grpc.testing.ServerType" do
- value :SYNC_SERVER, 0
- value :ASYNC_SERVER, 1
- value :ASYNC_GENERIC_SERVER, 2
- value :OTHER_SERVER, 3
- end
- add_enum "grpc.testing.RpcType" do
- value :UNARY, 0
- value :STREAMING, 1
- value :STREAMING_FROM_CLIENT, 2
- value :STREAMING_FROM_SERVER, 3
- value :STREAMING_BOTH_WAYS, 4
end
end
diff --git a/src/ruby/qps/src/proto/grpc/testing/messages_pb.rb b/src/ruby/qps/src/proto/grpc/testing/messages_pb.rb
index e27ccd0dc04..796d4bb9ae2 100644
--- a/src/ruby/qps/src/proto/grpc/testing/messages_pb.rb
+++ b/src/ruby/qps/src/proto/grpc/testing/messages_pb.rb
@@ -4,62 +4,64 @@
require 'google/protobuf'
Google::Protobuf::DescriptorPool.generated_pool.build do
- add_message "grpc.testing.BoolValue" do
- optional :value, :bool, 1
- end
- add_message "grpc.testing.Payload" do
- optional :type, :enum, 1, "grpc.testing.PayloadType"
- optional :body, :bytes, 2
- end
- add_message "grpc.testing.EchoStatus" do
- optional :code, :int32, 1
- optional :message, :string, 2
- end
- add_message "grpc.testing.SimpleRequest" do
- optional :response_type, :enum, 1, "grpc.testing.PayloadType"
- optional :response_size, :int32, 2
- optional :payload, :message, 3, "grpc.testing.Payload"
- optional :fill_username, :bool, 4
- optional :fill_oauth_scope, :bool, 5
- optional :response_compressed, :message, 6, "grpc.testing.BoolValue"
- optional :response_status, :message, 7, "grpc.testing.EchoStatus"
- optional :expect_compressed, :message, 8, "grpc.testing.BoolValue"
- end
- add_message "grpc.testing.SimpleResponse" do
- optional :payload, :message, 1, "grpc.testing.Payload"
- optional :username, :string, 2
- optional :oauth_scope, :string, 3
- end
- add_message "grpc.testing.StreamingInputCallRequest" do
- optional :payload, :message, 1, "grpc.testing.Payload"
- optional :expect_compressed, :message, 2, "grpc.testing.BoolValue"
- end
- add_message "grpc.testing.StreamingInputCallResponse" do
- optional :aggregated_payload_size, :int32, 1
- end
- add_message "grpc.testing.ResponseParameters" do
- optional :size, :int32, 1
- optional :interval_us, :int32, 2
- optional :compressed, :message, 3, "grpc.testing.BoolValue"
- end
- add_message "grpc.testing.StreamingOutputCallRequest" do
- optional :response_type, :enum, 1, "grpc.testing.PayloadType"
- repeated :response_parameters, :message, 2, "grpc.testing.ResponseParameters"
- optional :payload, :message, 3, "grpc.testing.Payload"
- optional :response_status, :message, 7, "grpc.testing.EchoStatus"
- end
- add_message "grpc.testing.StreamingOutputCallResponse" do
- optional :payload, :message, 1, "grpc.testing.Payload"
- end
- add_message "grpc.testing.ReconnectParams" do
- optional :max_reconnect_backoff_ms, :int32, 1
- end
- add_message "grpc.testing.ReconnectInfo" do
- optional :passed, :bool, 1
- repeated :backoff_ms, :int32, 2
- end
- add_enum "grpc.testing.PayloadType" do
- value :COMPRESSABLE, 0
+ add_file("src/proto/grpc/testing/messages.proto", :syntax => :proto3) do
+ add_message "grpc.testing.BoolValue" do
+ optional :value, :bool, 1
+ end
+ add_message "grpc.testing.Payload" do
+ optional :type, :enum, 1, "grpc.testing.PayloadType"
+ optional :body, :bytes, 2
+ end
+ add_message "grpc.testing.EchoStatus" do
+ optional :code, :int32, 1
+ optional :message, :string, 2
+ end
+ add_message "grpc.testing.SimpleRequest" do
+ optional :response_type, :enum, 1, "grpc.testing.PayloadType"
+ optional :response_size, :int32, 2
+ optional :payload, :message, 3, "grpc.testing.Payload"
+ optional :fill_username, :bool, 4
+ optional :fill_oauth_scope, :bool, 5
+ optional :response_compressed, :message, 6, "grpc.testing.BoolValue"
+ optional :response_status, :message, 7, "grpc.testing.EchoStatus"
+ optional :expect_compressed, :message, 8, "grpc.testing.BoolValue"
+ end
+ add_message "grpc.testing.SimpleResponse" do
+ optional :payload, :message, 1, "grpc.testing.Payload"
+ optional :username, :string, 2
+ optional :oauth_scope, :string, 3
+ end
+ add_message "grpc.testing.StreamingInputCallRequest" do
+ optional :payload, :message, 1, "grpc.testing.Payload"
+ optional :expect_compressed, :message, 2, "grpc.testing.BoolValue"
+ end
+ add_message "grpc.testing.StreamingInputCallResponse" do
+ optional :aggregated_payload_size, :int32, 1
+ end
+ add_message "grpc.testing.ResponseParameters" do
+ optional :size, :int32, 1
+ optional :interval_us, :int32, 2
+ optional :compressed, :message, 3, "grpc.testing.BoolValue"
+ end
+ add_message "grpc.testing.StreamingOutputCallRequest" do
+ optional :response_type, :enum, 1, "grpc.testing.PayloadType"
+ repeated :response_parameters, :message, 2, "grpc.testing.ResponseParameters"
+ optional :payload, :message, 3, "grpc.testing.Payload"
+ optional :response_status, :message, 7, "grpc.testing.EchoStatus"
+ end
+ add_message "grpc.testing.StreamingOutputCallResponse" do
+ optional :payload, :message, 1, "grpc.testing.Payload"
+ end
+ add_message "grpc.testing.ReconnectParams" do
+ optional :max_reconnect_backoff_ms, :int32, 1
+ end
+ add_message "grpc.testing.ReconnectInfo" do
+ optional :passed, :bool, 1
+ repeated :backoff_ms, :int32, 2
+ end
+ add_enum "grpc.testing.PayloadType" do
+ value :COMPRESSABLE, 0
+ end
end
end
diff --git a/src/ruby/qps/src/proto/grpc/testing/payloads_pb.rb b/src/ruby/qps/src/proto/grpc/testing/payloads_pb.rb
index ae8855f6850..6d55793fba4 100644
--- a/src/ruby/qps/src/proto/grpc/testing/payloads_pb.rb
+++ b/src/ruby/qps/src/proto/grpc/testing/payloads_pb.rb
@@ -4,21 +4,23 @@
require 'google/protobuf'
Google::Protobuf::DescriptorPool.generated_pool.build do
- add_message "grpc.testing.ByteBufferParams" do
- optional :req_size, :int32, 1
- optional :resp_size, :int32, 2
- end
- add_message "grpc.testing.SimpleProtoParams" do
- optional :req_size, :int32, 1
- optional :resp_size, :int32, 2
- end
- add_message "grpc.testing.ComplexProtoParams" do
- end
- add_message "grpc.testing.PayloadConfig" do
- oneof :payload do
- optional :bytebuf_params, :message, 1, "grpc.testing.ByteBufferParams"
- optional :simple_params, :message, 2, "grpc.testing.SimpleProtoParams"
- optional :complex_params, :message, 3, "grpc.testing.ComplexProtoParams"
+ add_file("src/proto/grpc/testing/payloads.proto", :syntax => :proto3) do
+ add_message "grpc.testing.ByteBufferParams" do
+ optional :req_size, :int32, 1
+ optional :resp_size, :int32, 2
+ end
+ add_message "grpc.testing.SimpleProtoParams" do
+ optional :req_size, :int32, 1
+ optional :resp_size, :int32, 2
+ end
+ add_message "grpc.testing.ComplexProtoParams" do
+ end
+ add_message "grpc.testing.PayloadConfig" do
+ oneof :payload do
+ optional :bytebuf_params, :message, 1, "grpc.testing.ByteBufferParams"
+ optional :simple_params, :message, 2, "grpc.testing.SimpleProtoParams"
+ optional :complex_params, :message, 3, "grpc.testing.ComplexProtoParams"
+ end
end
end
end
diff --git a/src/ruby/qps/src/proto/grpc/testing/report_qps_scenario_service_pb.rb b/src/ruby/qps/src/proto/grpc/testing/report_qps_scenario_service_pb.rb
index 1b43e372997..03461a4c556 100644
--- a/src/ruby/qps/src/proto/grpc/testing/report_qps_scenario_service_pb.rb
+++ b/src/ruby/qps/src/proto/grpc/testing/report_qps_scenario_service_pb.rb
@@ -5,6 +5,8 @@ require 'google/protobuf'
require 'src/proto/grpc/testing/control_pb'
Google::Protobuf::DescriptorPool.generated_pool.build do
+ add_file("src/proto/grpc/testing/report_qps_scenario_service.proto", :syntax => :proto3) do
+ end
end
module Grpc
diff --git a/src/ruby/qps/src/proto/grpc/testing/stats_pb.rb b/src/ruby/qps/src/proto/grpc/testing/stats_pb.rb
index 2069840168a..dd25d3159f3 100644
--- a/src/ruby/qps/src/proto/grpc/testing/stats_pb.rb
+++ b/src/ruby/qps/src/proto/grpc/testing/stats_pb.rb
@@ -5,39 +5,41 @@ require 'google/protobuf'
require 'src/proto/grpc/core/stats_pb'
Google::Protobuf::DescriptorPool.generated_pool.build do
- add_message "grpc.testing.ServerStats" do
- optional :time_elapsed, :double, 1
- optional :time_user, :double, 2
- optional :time_system, :double, 3
- optional :total_cpu_time, :uint64, 4
- optional :idle_cpu_time, :uint64, 5
- optional :cq_poll_count, :uint64, 6
- optional :core_stats, :message, 7, "grpc.core.Stats"
- end
- add_message "grpc.testing.HistogramParams" do
- optional :resolution, :double, 1
- optional :max_possible, :double, 2
- end
- add_message "grpc.testing.HistogramData" do
- repeated :bucket, :uint32, 1
- optional :min_seen, :double, 2
- optional :max_seen, :double, 3
- optional :sum, :double, 4
- optional :sum_of_squares, :double, 5
- optional :count, :double, 6
- end
- add_message "grpc.testing.RequestResultCount" do
- optional :status_code, :int32, 1
- optional :count, :int64, 2
- end
- add_message "grpc.testing.ClientStats" do
- optional :latencies, :message, 1, "grpc.testing.HistogramData"
- optional :time_elapsed, :double, 2
- optional :time_user, :double, 3
- optional :time_system, :double, 4
- repeated :request_results, :message, 5, "grpc.testing.RequestResultCount"
- optional :cq_poll_count, :uint64, 6
- optional :core_stats, :message, 7, "grpc.core.Stats"
+ add_file("src/proto/grpc/testing/stats.proto", :syntax => :proto3) do
+ add_message "grpc.testing.ServerStats" do
+ optional :time_elapsed, :double, 1
+ optional :time_user, :double, 2
+ optional :time_system, :double, 3
+ optional :total_cpu_time, :uint64, 4
+ optional :idle_cpu_time, :uint64, 5
+ optional :cq_poll_count, :uint64, 6
+ optional :core_stats, :message, 7, "grpc.core.Stats"
+ end
+ add_message "grpc.testing.HistogramParams" do
+ optional :resolution, :double, 1
+ optional :max_possible, :double, 2
+ end
+ add_message "grpc.testing.HistogramData" do
+ repeated :bucket, :uint32, 1
+ optional :min_seen, :double, 2
+ optional :max_seen, :double, 3
+ optional :sum, :double, 4
+ optional :sum_of_squares, :double, 5
+ optional :count, :double, 6
+ end
+ add_message "grpc.testing.RequestResultCount" do
+ optional :status_code, :int32, 1
+ optional :count, :int64, 2
+ end
+ add_message "grpc.testing.ClientStats" do
+ optional :latencies, :message, 1, "grpc.testing.HistogramData"
+ optional :time_elapsed, :double, 2
+ optional :time_user, :double, 3
+ optional :time_system, :double, 4
+ repeated :request_results, :message, 5, "grpc.testing.RequestResultCount"
+ optional :cq_poll_count, :uint64, 6
+ optional :core_stats, :message, 7, "grpc.core.Stats"
+ end
end
end
diff --git a/src/ruby/qps/src/proto/grpc/testing/worker_service_pb.rb b/src/ruby/qps/src/proto/grpc/testing/worker_service_pb.rb
index 18b63452b6e..2fdef48e933 100644
--- a/src/ruby/qps/src/proto/grpc/testing/worker_service_pb.rb
+++ b/src/ruby/qps/src/proto/grpc/testing/worker_service_pb.rb
@@ -5,6 +5,8 @@ require 'google/protobuf'
require 'src/proto/grpc/testing/control_pb'
Google::Protobuf::DescriptorPool.generated_pool.build do
+ add_file("src/proto/grpc/testing/worker_service.proto", :syntax => :proto3) do
+ end
end
module Grpc
From ce22e246d71cc13493a4396b7f9cd4f8cfccd721 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Mon, 11 Mar 2019 15:02:31 -0400
Subject: [PATCH 095/138] run tools/distrib/python/make_grpcio_tools.py
---
tools/distrib/python/grpcio_tools/protoc_lib_deps.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/distrib/python/grpcio_tools/protoc_lib_deps.py b/tools/distrib/python/grpcio_tools/protoc_lib_deps.py
index 7d10db0329c..e7e4ba307ec 100644
--- a/tools/distrib/python/grpcio_tools/protoc_lib_deps.py
+++ b/tools/distrib/python/grpcio_tools/protoc_lib_deps.py
@@ -14,10 +14,10 @@
# limitations under the License.
# AUTO-GENERATED BY make_grpcio_tools.py!
-CC_FILES=['google/protobuf/compiler/zip_writer.cc', 'google/protobuf/compiler/subprocess.cc', 'google/protobuf/compiler/ruby/ruby_generator.cc', 'google/protobuf/compiler/python/python_generator.cc', 'google/protobuf/compiler/plugin.pb.cc', 'google/protobuf/compiler/plugin.cc', 'google/protobuf/compiler/php/php_generator.cc', 'google/protobuf/compiler/objectivec/objectivec_primitive_field.cc', 'google/protobuf/compiler/objectivec/objectivec_oneof.cc', 'google/protobuf/compiler/objectivec/objectivec_message_field.cc', 'google/protobuf/compiler/objectivec/objectivec_message.cc', 'google/protobuf/compiler/objectivec/objectivec_map_field.cc', 'google/protobuf/compiler/objectivec/objectivec_helpers.cc', 'google/protobuf/compiler/objectivec/objectivec_generator.cc', 'google/protobuf/compiler/objectivec/objectivec_file.cc', 'google/protobuf/compiler/objectivec/objectivec_field.cc', 'google/protobuf/compiler/objectivec/objectivec_extension.cc', 'google/protobuf/compiler/objectivec/objectivec_enum_field.cc', 'google/protobuf/compiler/objectivec/objectivec_enum.cc', 'google/protobuf/compiler/js/well_known_types_embed.cc', 'google/protobuf/compiler/js/js_generator.cc', 'google/protobuf/compiler/java/java_string_field_lite.cc', 'google/protobuf/compiler/java/java_string_field.cc', 'google/protobuf/compiler/java/java_shared_code_generator.cc', 'google/protobuf/compiler/java/java_service.cc', 'google/protobuf/compiler/java/java_primitive_field_lite.cc', 'google/protobuf/compiler/java/java_primitive_field.cc', 'google/protobuf/compiler/java/java_name_resolver.cc', 'google/protobuf/compiler/java/java_message_lite.cc', 'google/protobuf/compiler/java/java_message_field_lite.cc', 'google/protobuf/compiler/java/java_message_field.cc', 'google/protobuf/compiler/java/java_message_builder_lite.cc', 'google/protobuf/compiler/java/java_message_builder.cc', 'google/protobuf/compiler/java/java_message.cc', 'google/protobuf/compiler/java/java_map_field_lite.cc', 'google/protobuf/compiler/java/java_map_field.cc', 'google/protobuf/compiler/java/java_lazy_message_field_lite.cc', 'google/protobuf/compiler/java/java_lazy_message_field.cc', 'google/protobuf/compiler/java/java_helpers.cc', 'google/protobuf/compiler/java/java_generator_factory.cc', 'google/protobuf/compiler/java/java_generator.cc', 'google/protobuf/compiler/java/java_file.cc', 'google/protobuf/compiler/java/java_field.cc', 'google/protobuf/compiler/java/java_extension_lite.cc', 'google/protobuf/compiler/java/java_extension.cc', 'google/protobuf/compiler/java/java_enum_lite.cc', 'google/protobuf/compiler/java/java_enum_field_lite.cc', 'google/protobuf/compiler/java/java_enum_field.cc', 'google/protobuf/compiler/java/java_enum.cc', 'google/protobuf/compiler/java/java_doc_comment.cc', 'google/protobuf/compiler/java/java_context.cc', 'google/protobuf/compiler/csharp/csharp_wrapper_field.cc', 'google/protobuf/compiler/csharp/csharp_source_generator_base.cc', 'google/protobuf/compiler/csharp/csharp_repeated_primitive_field.cc', 'google/protobuf/compiler/csharp/csharp_repeated_message_field.cc', 'google/protobuf/compiler/csharp/csharp_repeated_enum_field.cc', 'google/protobuf/compiler/csharp/csharp_reflection_class.cc', 'google/protobuf/compiler/csharp/csharp_primitive_field.cc', 'google/protobuf/compiler/csharp/csharp_message_field.cc', 'google/protobuf/compiler/csharp/csharp_message.cc', 'google/protobuf/compiler/csharp/csharp_map_field.cc', 'google/protobuf/compiler/csharp/csharp_helpers.cc', 'google/protobuf/compiler/csharp/csharp_generator.cc', 'google/protobuf/compiler/csharp/csharp_field_base.cc', 'google/protobuf/compiler/csharp/csharp_enum_field.cc', 'google/protobuf/compiler/csharp/csharp_enum.cc', 'google/protobuf/compiler/csharp/csharp_doc_comment.cc', 'google/protobuf/compiler/cpp/cpp_string_field.cc', 'google/protobuf/compiler/cpp/cpp_service.cc', 'google/protobuf/compiler/cpp/cpp_primitive_field.cc', 'google/protobuf/compiler/cpp/cpp_padding_optimizer.cc', 'google/protobuf/compiler/cpp/cpp_message_field.cc', 'google/protobuf/compiler/cpp/cpp_message.cc', 'google/protobuf/compiler/cpp/cpp_map_field.cc', 'google/protobuf/compiler/cpp/cpp_helpers.cc', 'google/protobuf/compiler/cpp/cpp_generator.cc', 'google/protobuf/compiler/cpp/cpp_file.cc', 'google/protobuf/compiler/cpp/cpp_field.cc', 'google/protobuf/compiler/cpp/cpp_extension.cc', 'google/protobuf/compiler/cpp/cpp_enum_field.cc', 'google/protobuf/compiler/cpp/cpp_enum.cc', 'google/protobuf/compiler/command_line_interface.cc', 'google/protobuf/compiler/code_generator.cc', 'google/protobuf/wrappers.pb.cc', 'google/protobuf/wire_format.cc', 'google/protobuf/util/type_resolver_util.cc', 'google/protobuf/util/time_util.cc', 'google/protobuf/util/message_differencer.cc', 'google/protobuf/util/json_util.cc', 'google/protobuf/util/internal/utility.cc', 'google/protobuf/util/internal/type_info_test_helper.cc', 'google/protobuf/util/internal/type_info.cc', 'google/protobuf/util/internal/protostream_objectwriter.cc', 'google/protobuf/util/internal/protostream_objectsource.cc', 'google/protobuf/util/internal/proto_writer.cc', 'google/protobuf/util/internal/object_writer.cc', 'google/protobuf/util/internal/json_stream_parser.cc', 'google/protobuf/util/internal/json_objectwriter.cc', 'google/protobuf/util/internal/json_escaping.cc', 'google/protobuf/util/internal/field_mask_utility.cc', 'google/protobuf/util/internal/error_listener.cc', 'google/protobuf/util/internal/default_value_objectwriter.cc', 'google/protobuf/util/internal/datapiece.cc', 'google/protobuf/util/field_mask_util.cc', 'google/protobuf/util/field_comparator.cc', 'google/protobuf/util/delimited_message_util.cc', 'google/protobuf/unknown_field_set.cc', 'google/protobuf/type.pb.cc', 'google/protobuf/timestamp.pb.cc', 'google/protobuf/text_format.cc', 'google/protobuf/stubs/substitute.cc', 'google/protobuf/stubs/mathlimits.cc', 'google/protobuf/struct.pb.cc', 'google/protobuf/source_context.pb.cc', 'google/protobuf/service.cc', 'google/protobuf/reflection_ops.cc', 'google/protobuf/message.cc', 'google/protobuf/map_field.cc', 'google/protobuf/io/zero_copy_stream_impl.cc', 'google/protobuf/io/tokenizer.cc', 'google/protobuf/io/strtod.cc', 'google/protobuf/io/printer.cc', 'google/protobuf/io/gzip_stream.cc', 'google/protobuf/generated_message_table_driven.cc', 'google/protobuf/generated_message_reflection.cc', 'google/protobuf/field_mask.pb.cc', 'google/protobuf/extension_set_heavy.cc', 'google/protobuf/empty.pb.cc', 'google/protobuf/dynamic_message.cc', 'google/protobuf/duration.pb.cc', 'google/protobuf/descriptor_database.cc', 'google/protobuf/descriptor.pb.cc', 'google/protobuf/descriptor.cc', 'google/protobuf/compiler/parser.cc', 'google/protobuf/compiler/importer.cc', 'google/protobuf/api.pb.cc', 'google/protobuf/any.pb.cc', 'google/protobuf/any.cc', 'google/protobuf/wire_format_lite.cc', 'google/protobuf/stubs/time.cc', 'google/protobuf/stubs/strutil.cc', 'google/protobuf/stubs/structurally_valid.cc', 'google/protobuf/stubs/stringprintf.cc', 'google/protobuf/stubs/stringpiece.cc', 'google/protobuf/stubs/statusor.cc', 'google/protobuf/stubs/status.cc', 'google/protobuf/stubs/io_win32.cc', 'google/protobuf/stubs/int128.cc', 'google/protobuf/stubs/common.cc', 'google/protobuf/stubs/bytestream.cc', 'google/protobuf/repeated_field.cc', 'google/protobuf/message_lite.cc', 'google/protobuf/io/zero_copy_stream_impl_lite.cc', 'google/protobuf/io/zero_copy_stream.cc', 'google/protobuf/io/coded_stream.cc', 'google/protobuf/implicit_weak_message.cc', 'google/protobuf/generated_message_util.cc', 'google/protobuf/generated_message_table_driven_lite.cc', 'google/protobuf/extension_set.cc', 'google/protobuf/arenastring.cc', 'google/protobuf/arena.cc']
+CC_FILES=['google/protobuf/compiler/zip_writer.cc', 'google/protobuf/compiler/subprocess.cc', 'google/protobuf/compiler/ruby/ruby_generator.cc', 'google/protobuf/compiler/python/python_generator.cc', 'google/protobuf/compiler/plugin.pb.cc', 'google/protobuf/compiler/plugin.cc', 'google/protobuf/compiler/php/php_generator.cc', 'google/protobuf/compiler/objectivec/objectivec_primitive_field.cc', 'google/protobuf/compiler/objectivec/objectivec_oneof.cc', 'google/protobuf/compiler/objectivec/objectivec_message_field.cc', 'google/protobuf/compiler/objectivec/objectivec_message.cc', 'google/protobuf/compiler/objectivec/objectivec_map_field.cc', 'google/protobuf/compiler/objectivec/objectivec_helpers.cc', 'google/protobuf/compiler/objectivec/objectivec_generator.cc', 'google/protobuf/compiler/objectivec/objectivec_file.cc', 'google/protobuf/compiler/objectivec/objectivec_field.cc', 'google/protobuf/compiler/objectivec/objectivec_extension.cc', 'google/protobuf/compiler/objectivec/objectivec_enum_field.cc', 'google/protobuf/compiler/objectivec/objectivec_enum.cc', 'google/protobuf/compiler/js/well_known_types_embed.cc', 'google/protobuf/compiler/js/js_generator.cc', 'google/protobuf/compiler/java/java_string_field_lite.cc', 'google/protobuf/compiler/java/java_string_field.cc', 'google/protobuf/compiler/java/java_shared_code_generator.cc', 'google/protobuf/compiler/java/java_service.cc', 'google/protobuf/compiler/java/java_primitive_field_lite.cc', 'google/protobuf/compiler/java/java_primitive_field.cc', 'google/protobuf/compiler/java/java_name_resolver.cc', 'google/protobuf/compiler/java/java_message_lite.cc', 'google/protobuf/compiler/java/java_message_field_lite.cc', 'google/protobuf/compiler/java/java_message_field.cc', 'google/protobuf/compiler/java/java_message_builder_lite.cc', 'google/protobuf/compiler/java/java_message_builder.cc', 'google/protobuf/compiler/java/java_message.cc', 'google/protobuf/compiler/java/java_map_field_lite.cc', 'google/protobuf/compiler/java/java_map_field.cc', 'google/protobuf/compiler/java/java_helpers.cc', 'google/protobuf/compiler/java/java_generator_factory.cc', 'google/protobuf/compiler/java/java_generator.cc', 'google/protobuf/compiler/java/java_file.cc', 'google/protobuf/compiler/java/java_field.cc', 'google/protobuf/compiler/java/java_extension_lite.cc', 'google/protobuf/compiler/java/java_extension.cc', 'google/protobuf/compiler/java/java_enum_lite.cc', 'google/protobuf/compiler/java/java_enum_field_lite.cc', 'google/protobuf/compiler/java/java_enum_field.cc', 'google/protobuf/compiler/java/java_enum.cc', 'google/protobuf/compiler/java/java_doc_comment.cc', 'google/protobuf/compiler/java/java_context.cc', 'google/protobuf/compiler/csharp/csharp_wrapper_field.cc', 'google/protobuf/compiler/csharp/csharp_source_generator_base.cc', 'google/protobuf/compiler/csharp/csharp_repeated_primitive_field.cc', 'google/protobuf/compiler/csharp/csharp_repeated_message_field.cc', 'google/protobuf/compiler/csharp/csharp_repeated_enum_field.cc', 'google/protobuf/compiler/csharp/csharp_reflection_class.cc', 'google/protobuf/compiler/csharp/csharp_primitive_field.cc', 'google/protobuf/compiler/csharp/csharp_message_field.cc', 'google/protobuf/compiler/csharp/csharp_message.cc', 'google/protobuf/compiler/csharp/csharp_map_field.cc', 'google/protobuf/compiler/csharp/csharp_helpers.cc', 'google/protobuf/compiler/csharp/csharp_generator.cc', 'google/protobuf/compiler/csharp/csharp_field_base.cc', 'google/protobuf/compiler/csharp/csharp_enum_field.cc', 'google/protobuf/compiler/csharp/csharp_enum.cc', 'google/protobuf/compiler/csharp/csharp_doc_comment.cc', 'google/protobuf/compiler/cpp/cpp_string_field.cc', 'google/protobuf/compiler/cpp/cpp_service.cc', 'google/protobuf/compiler/cpp/cpp_primitive_field.cc', 'google/protobuf/compiler/cpp/cpp_padding_optimizer.cc', 'google/protobuf/compiler/cpp/cpp_message_field.cc', 'google/protobuf/compiler/cpp/cpp_message.cc', 'google/protobuf/compiler/cpp/cpp_map_field.cc', 'google/protobuf/compiler/cpp/cpp_helpers.cc', 'google/protobuf/compiler/cpp/cpp_generator.cc', 'google/protobuf/compiler/cpp/cpp_file.cc', 'google/protobuf/compiler/cpp/cpp_field.cc', 'google/protobuf/compiler/cpp/cpp_extension.cc', 'google/protobuf/compiler/cpp/cpp_enum_field.cc', 'google/protobuf/compiler/cpp/cpp_enum.cc', 'google/protobuf/compiler/command_line_interface.cc', 'google/protobuf/compiler/code_generator.cc', 'google/protobuf/wrappers.pb.cc', 'google/protobuf/wire_format.cc', 'google/protobuf/util/type_resolver_util.cc', 'google/protobuf/util/time_util.cc', 'google/protobuf/util/message_differencer.cc', 'google/protobuf/util/json_util.cc', 'google/protobuf/util/internal/utility.cc', 'google/protobuf/util/internal/type_info_test_helper.cc', 'google/protobuf/util/internal/type_info.cc', 'google/protobuf/util/internal/protostream_objectwriter.cc', 'google/protobuf/util/internal/protostream_objectsource.cc', 'google/protobuf/util/internal/proto_writer.cc', 'google/protobuf/util/internal/object_writer.cc', 'google/protobuf/util/internal/json_stream_parser.cc', 'google/protobuf/util/internal/json_objectwriter.cc', 'google/protobuf/util/internal/json_escaping.cc', 'google/protobuf/util/internal/field_mask_utility.cc', 'google/protobuf/util/internal/error_listener.cc', 'google/protobuf/util/internal/default_value_objectwriter.cc', 'google/protobuf/util/internal/datapiece.cc', 'google/protobuf/util/field_mask_util.cc', 'google/protobuf/util/field_comparator.cc', 'google/protobuf/util/delimited_message_util.cc', 'google/protobuf/unknown_field_set.cc', 'google/protobuf/type.pb.cc', 'google/protobuf/timestamp.pb.cc', 'google/protobuf/text_format.cc', 'google/protobuf/stubs/substitute.cc', 'google/protobuf/stubs/mathlimits.cc', 'google/protobuf/struct.pb.cc', 'google/protobuf/source_context.pb.cc', 'google/protobuf/service.cc', 'google/protobuf/reflection_ops.cc', 'google/protobuf/message.cc', 'google/protobuf/map_field.cc', 'google/protobuf/io/zero_copy_stream_impl.cc', 'google/protobuf/io/tokenizer.cc', 'google/protobuf/io/strtod.cc', 'google/protobuf/io/printer.cc', 'google/protobuf/io/gzip_stream.cc', 'google/protobuf/generated_message_table_driven.cc', 'google/protobuf/generated_message_reflection.cc', 'google/protobuf/field_mask.pb.cc', 'google/protobuf/extension_set_heavy.cc', 'google/protobuf/empty.pb.cc', 'google/protobuf/dynamic_message.cc', 'google/protobuf/duration.pb.cc', 'google/protobuf/descriptor_database.cc', 'google/protobuf/descriptor.pb.cc', 'google/protobuf/descriptor.cc', 'google/protobuf/compiler/parser.cc', 'google/protobuf/compiler/importer.cc', 'google/protobuf/api.pb.cc', 'google/protobuf/any.pb.cc', 'google/protobuf/any.cc', 'google/protobuf/wire_format_lite.cc', 'google/protobuf/stubs/time.cc', 'google/protobuf/stubs/strutil.cc', 'google/protobuf/stubs/structurally_valid.cc', 'google/protobuf/stubs/stringprintf.cc', 'google/protobuf/stubs/stringpiece.cc', 'google/protobuf/stubs/statusor.cc', 'google/protobuf/stubs/status.cc', 'google/protobuf/stubs/io_win32.cc', 'google/protobuf/stubs/int128.cc', 'google/protobuf/stubs/common.cc', 'google/protobuf/stubs/bytestream.cc', 'google/protobuf/repeated_field.cc', 'google/protobuf/message_lite.cc', 'google/protobuf/io/zero_copy_stream_impl_lite.cc', 'google/protobuf/io/zero_copy_stream.cc', 'google/protobuf/io/coded_stream.cc', 'google/protobuf/implicit_weak_message.cc', 'google/protobuf/generated_message_util.cc', 'google/protobuf/generated_message_table_driven_lite.cc', 'google/protobuf/extension_set.cc', 'google/protobuf/arena.cc']
PROTO_FILES=['google/protobuf/wrappers.proto', 'google/protobuf/type.proto', 'google/protobuf/timestamp.proto', 'google/protobuf/struct.proto', 'google/protobuf/source_context.proto', 'google/protobuf/field_mask.proto', 'google/protobuf/empty.proto', 'google/protobuf/duration.proto', 'google/protobuf/descriptor.proto', 'google/protobuf/compiler/plugin.proto', 'google/protobuf/api.proto', 'google/protobuf/any.proto']
CC_INCLUDE='third_party/protobuf/src'
PROTO_INCLUDE='third_party/protobuf/src'
-PROTOBUF_SUBMODULE_VERSION="48cb18e5c419ddd23d9badcfe4e9df7bde1979b2"
+PROTOBUF_SUBMODULE_VERSION="582743bf40c5d3639a70f98f183914a2c0cd0680"
From f601afae144a66b040fce6149d2d154b2a5c6ba1 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Mon, 11 Mar 2019 15:05:03 -0400
Subject: [PATCH 096/138] bump ruby google-protobuf dependency
---
grpc.gemspec | 2 +-
templates/grpc.gemspec.template | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/grpc.gemspec b/grpc.gemspec
index a2a027a20d8..5c749dd285d 100644
--- a/grpc.gemspec
+++ b/grpc.gemspec
@@ -29,7 +29,7 @@ Gem::Specification.new do |s|
s.require_paths = %w( src/ruby/lib src/ruby/bin src/ruby/pb )
s.platform = Gem::Platform::RUBY
- s.add_dependency 'google-protobuf', '~> 3.1'
+ s.add_dependency 'google-protobuf', '~> 3.7'
s.add_dependency 'googleapis-common-protos-types', '~> 1.0.0'
s.add_development_dependency 'bundler', '~> 1.9'
diff --git a/templates/grpc.gemspec.template b/templates/grpc.gemspec.template
index 1498a280b0e..0e321717c99 100644
--- a/templates/grpc.gemspec.template
+++ b/templates/grpc.gemspec.template
@@ -31,7 +31,7 @@
s.require_paths = %w( src/ruby/lib src/ruby/bin src/ruby/pb )
s.platform = Gem::Platform::RUBY
- s.add_dependency 'google-protobuf', '~> 3.1'
+ s.add_dependency 'google-protobuf', '~> 3.7'
s.add_dependency 'googleapis-common-protos-types', '~> 1.0.0'
s.add_development_dependency 'bundler', '~> 1.9'
From 96fd87169476309cb0934617365101113abda8b9 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Mon, 11 Mar 2019 15:07:47 -0400
Subject: [PATCH 097/138] bump C# protobuf dependency
---
src/csharp/Grpc.Core/Version.csproj.include | 2 +-
templates/src/csharp/Grpc.Core/Version.csproj.include.template | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/csharp/Grpc.Core/Version.csproj.include b/src/csharp/Grpc.Core/Version.csproj.include
index de933448b96..f3d484643d7 100755
--- a/src/csharp/Grpc.Core/Version.csproj.include
+++ b/src/csharp/Grpc.Core/Version.csproj.include
@@ -2,6 +2,6 @@
<Project>
<PropertyGroup>
<GrpcCsharpVersion>1.20.0-dev</GrpcCsharpVersion>
- <GoogleProtobufVersion>3.6.1</GoogleProtobufVersion>
+ <GoogleProtobufVersion>3.7.0</GoogleProtobufVersion>
</PropertyGroup>
</Project>
diff --git a/templates/src/csharp/Grpc.Core/Version.csproj.include.template b/templates/src/csharp/Grpc.Core/Version.csproj.include.template
index 0ec0a08c499..0ed9018a49e 100755
--- a/templates/src/csharp/Grpc.Core/Version.csproj.include.template
+++ b/templates/src/csharp/Grpc.Core/Version.csproj.include.template
@@ -4,6 +4,6 @@
<Project>
<PropertyGroup>
<GrpcCsharpVersion>${settings.csharp_version}</GrpcCsharpVersion>
- <GoogleProtobufVersion>3.6.1</GoogleProtobufVersion>
+ <GoogleProtobufVersion>3.7.0</GoogleProtobufVersion>
</PropertyGroup>
</Project>
From c820cfc1589f8a7be148e9d6279ea3f2aa812a44 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Tue, 12 Mar 2019 06:19:37 -0400
Subject: [PATCH 098/138] activate sourcelink when building packages
---
src/csharp/build_packages_dotnetcli.bat | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/src/csharp/build_packages_dotnetcli.bat b/src/csharp/build_packages_dotnetcli.bat
index f500310865b..58520f2f497 100755
--- a/src/csharp/build_packages_dotnetcli.bat
+++ b/src/csharp/build_packages_dotnetcli.bat
@@ -32,13 +32,13 @@ expand_dev_version.sh
@rem To be able to build, we also need to put grpc_csharp_ext to its normal location
xcopy /Y /I nativelibs\csharp_ext_windows_x64\grpc_csharp_ext.dll ..\..\cmake\build\x64\Release\
-%DOTNET% pack --configuration Release Grpc.Core.Api --output ..\..\..\artifacts || goto :error
-%DOTNET% pack --configuration Release Grpc.Core --output ..\..\..\artifacts || goto :error
-%DOTNET% pack --configuration Release Grpc.Core.Testing --output ..\..\..\artifacts || goto :error
-%DOTNET% pack --configuration Release Grpc.Auth --output ..\..\..\artifacts || goto :error
-%DOTNET% pack --configuration Release Grpc.HealthCheck --output ..\..\..\artifacts || goto :error
-%DOTNET% pack --configuration Release Grpc.Reflection --output ..\..\..\artifacts || goto :error
-%DOTNET% pack --configuration Release Grpc.Tools --output ..\..\..\artifacts || goto :error
+%DOTNET% pack --configuration Release Grpc.Core.Api /p:SourceLinkCreate=true --output ..\..\..\artifacts || goto :error
+%DOTNET% pack --configuration Release Grpc.Core /p:SourceLinkCreate=true --output ..\..\..\artifacts || goto :error
+%DOTNET% pack --configuration Release Grpc.Core.Testing /p:SourceLinkCreate=true --output ..\..\..\artifacts || goto :error
+%DOTNET% pack --configuration Release Grpc.Auth /p:SourceLinkCreate=true --output ..\..\..\artifacts || goto :error
+%DOTNET% pack --configuration Release Grpc.HealthCheck /p:SourceLinkCreate=true --output ..\..\..\artifacts || goto :error
+%DOTNET% pack --configuration Release Grpc.Reflection /p:SourceLinkCreate=true --output ..\..\..\artifacts || goto :error
+%DOTNET% pack --configuration Release Grpc.Tools /p:SourceLinkCreate=true --output ..\..\..\artifacts || goto :error
@rem build auxiliary packages
%DOTNET% pack --configuration Release Grpc --output ..\..\..\artifacts || goto :error
%DOTNET% pack --configuration Release Grpc.Core.NativeDebug --output ..\..\..\artifacts || goto :error
From 478377ff70b771ef6084f31dbd464511987517e3 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Tue, 12 Mar 2019 08:42:47 -0400
Subject: [PATCH 099/138] PDBs will get included automatically
---
src/csharp/Grpc.Core/SourceLink.csproj.include | 11 -----------
1 file changed, 11 deletions(-)
diff --git a/src/csharp/Grpc.Core/SourceLink.csproj.include b/src/csharp/Grpc.Core/SourceLink.csproj.include
index 0ec273f57e6..526db954540 100755
--- a/src/csharp/Grpc.Core/SourceLink.csproj.include
+++ b/src/csharp/Grpc.Core/SourceLink.csproj.include
@@ -1,17 +1,6 @@
<!-- Ensure that debugging of the resulting NuGet packages work (we're using SourceLink). -->
<Project>
- <ItemGroup Label="dotnet pack instructions">
- <Content Include="$(OutputPath)netstandard1.5\$(PackageId).pdb">
- <Pack>true</Pack>
- <PackagePath>lib/netstandard1.5</PackagePath>
- </Content>
- <Content Include="$(OutputPath)net45\$(PackageId).pdb">
- <Pack>true</Pack>
- <PackagePath>lib/net45</PackagePath>
- </Content>
- </ItemGroup>
-
<ItemGroup>
<PackageReference Include="SourceLink.Create.CommandLine" Version="2.7.6" PrivateAssets="all" />
</ItemGroup>
From 2a50960b4c874e90d9d32c53e9a642c0869e1b80 Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Tue, 12 Mar 2019 08:37:07 -0700
Subject: [PATCH 100/138] Add copyright to BUILD file
---
examples/python/multiprocessing/BUILD | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/examples/python/multiprocessing/BUILD b/examples/python/multiprocessing/BUILD
index 48b98f3ad20..81bd67feeb6 100644
--- a/examples/python/multiprocessing/BUILD
+++ b/examples/python/multiprocessing/BUILD
@@ -1,3 +1,19 @@
+# gRPC Bazel BUILD file.
+#
+# Copyright 2019 The gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
load("@grpc_python_dependencies//:requirements.bzl", "requirement")
py_binary(
From b9659d58da915f64abf73eb146947d805476f4c5 Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Tue, 12 Mar 2019 09:16:30 -0700
Subject: [PATCH 101/138] Actually generate proto
---
examples/python/multiprocessing/BUILD | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/examples/python/multiprocessing/BUILD b/examples/python/multiprocessing/BUILD
index 81bd67feeb6..b72edfa6d04 100644
--- a/examples/python/multiprocessing/BUILD
+++ b/examples/python/multiprocessing/BUILD
@@ -15,13 +15,20 @@
# limitations under the License.
load("@grpc_python_dependencies//:requirements.bzl", "requirement")
+load("@org_pubref_rules_protobuf//python:rules.bzl", "py_proto_library")
+
+py_proto_library(
+ name = "prime_proto",
+ protos = ["prime.proto",],
+)
py_binary(
name = "client",
testonly = 1,
srcs = ["client.py"],
deps = [
- "//src/python/grpcio/grpc:grpcio"
+ "//src/python/grpcio/grpc:grpcio",
+ ":prime_proto",
],
default_python_version = "PY3",
)
@@ -31,7 +38,8 @@ py_binary(
testonly = 1,
srcs = ["server.py"],
deps = [
- "//src/python/grpcio/grpc:grpcio"
+ "//src/python/grpcio/grpc:grpcio",
+ ":prime_proto"
] + select({
"//conditions:default": [requirement("futures")],
"//:python3": [],
From f3b57e35ec127e2e19b72d27660402fbac7054ed Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Tue, 12 Mar 2019 10:15:19 -0700
Subject: [PATCH 102/138] Properly import protos
---
examples/python/multiprocessing/BUILD | 1 +
examples/python/multiprocessing/client.py | 4 ++--
examples/python/multiprocessing/server.py | 4 ++--
3 files changed, 5 insertions(+), 4 deletions(-)
diff --git a/examples/python/multiprocessing/BUILD b/examples/python/multiprocessing/BUILD
index b72edfa6d04..6de1e947d85 100644
--- a/examples/python/multiprocessing/BUILD
+++ b/examples/python/multiprocessing/BUILD
@@ -20,6 +20,7 @@ load("@org_pubref_rules_protobuf//python:rules.bzl", "py_proto_library")
py_proto_library(
name = "prime_proto",
protos = ["prime.proto",],
+ deps = [requirement("protobuf")],
)
py_binary(
diff --git a/examples/python/multiprocessing/client.py b/examples/python/multiprocessing/client.py
index c3da4ba2b99..6d3bf5d825a 100644
--- a/examples/python/multiprocessing/client.py
+++ b/examples/python/multiprocessing/client.py
@@ -26,8 +26,8 @@ import sys
import grpc
-import prime_pb2
-import prime_pb2_grpc
+from examples.python.multiprocessing import prime_pb2
+from examples.python.multiprocessing import prime_pb2_grpc
_PROCESS_COUNT = 8
_MAXIMUM_CANDIDATE = 10000
diff --git a/examples/python/multiprocessing/server.py b/examples/python/multiprocessing/server.py
index 27a0758d224..267019731f2 100644
--- a/examples/python/multiprocessing/server.py
+++ b/examples/python/multiprocessing/server.py
@@ -29,8 +29,8 @@ import sys
import grpc
-import prime_pb2
-import prime_pb2_grpc
+from examples.python.multiprocessing import prime_pb2
+from examples.python.multiprocessing import prime_pb2_grpc
_LOGGER = logging.getLogger(__name__)
From dde238cb5dbf3347bedccf73e3fadb59e5da5d47 Mon Sep 17 00:00:00 2001
From: Richard Belleville <rbellevi@google.com>
Date: Tue, 12 Mar 2019 10:44:05 -0700
Subject: [PATCH 103/138] Apparently no ipv6 on kokoro
---
examples/python/multiprocessing/server.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/examples/python/multiprocessing/server.py b/examples/python/multiprocessing/server.py
index 267019731f2..a05eb9edda0 100644
--- a/examples/python/multiprocessing/server.py
+++ b/examples/python/multiprocessing/server.py
@@ -98,7 +98,7 @@ def _reserve_port():
def main():
with _reserve_port() as port:
- bind_address = '[::]:{}'.format(port)
+ bind_address = 'localhost:{}'.format(port)
_LOGGER.info("Binding to '%s'", bind_address)
sys.stdout.flush()
workers = []
From b3889585a1f9f0a7600be92788d81ec73fca1941 Mon Sep 17 00:00:00 2001
From: Karthik Ravi Shankar <karthikrs@google.com>
Date: Tue, 12 Mar 2019 12:25:38 -0700
Subject: [PATCH 104/138] Revert "Moving ::grpc::ResourceQuota to
::grpc_impl::ResouceQuota"
---
BUILD | 3 +-
CMakeLists.txt | 3 -
Makefile | 3 -
build.yaml | 1 -
gRPC-C++.podspec | 1 -
include/grpcpp/resource_quota.h | 45 +++++++++++-
include/grpcpp/resource_quota_impl.h | 68 -------------------
include/grpcpp/server_builder.h | 9 +--
include/grpcpp/support/channel_arguments.h | 9 +--
src/cpp/common/channel_arguments.cc | 2 +-
src/cpp/common/resource_quota_cc.cc | 4 +-
src/cpp/server/server_builder.cc | 7 +-
test/cpp/end2end/end2end_test.cc | 5 --
test/cpp/end2end/thread_stress_test.cc | 5 --
test/cpp/qps/server.h | 5 --
test/cpp/qps/server_async.cc | 1 +
tools/doxygen/Doxyfile.c++ | 1 -
tools/doxygen/Doxyfile.c++.internal | 1 -
.../generated/sources_and_headers.json | 2 -
19 files changed, 53 insertions(+), 122 deletions(-)
delete mode 100644 include/grpcpp/resource_quota_impl.h
diff --git a/BUILD b/BUILD
index 191661fdd9a..4bf387cec3a 100644
--- a/BUILD
+++ b/BUILD
@@ -192,8 +192,8 @@ GRPCXX_PUBLIC_HDRS = [
"include/grpc++/impl/service_type.h",
"include/grpc++/impl/sync_cxx11.h",
"include/grpc++/impl/sync_no_cxx11.h",
- "include/grpc++/security/auth_context.h",
"include/grpc++/resource_quota.h",
+ "include/grpc++/security/auth_context.h",
"include/grpc++/security/auth_metadata_processor.h",
"include/grpc++/security/credentials.h",
"include/grpc++/security/server_credentials.h",
@@ -241,7 +241,6 @@ GRPCXX_PUBLIC_HDRS = [
"include/grpcpp/impl/sync_cxx11.h",
"include/grpcpp/impl/sync_no_cxx11.h",
"include/grpcpp/resource_quota.h",
- "include/grpcpp/resource_quota_impl.h",
"include/grpcpp/security/auth_context.h",
"include/grpcpp/security/auth_metadata_processor.h",
"include/grpcpp/security/credentials.h",
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 40719422aa8..7ccda85b125 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -3014,7 +3014,6 @@ foreach(_hdr
include/grpcpp/impl/server_initializer.h
include/grpcpp/impl/service_type.h
include/grpcpp/resource_quota.h
- include/grpcpp/resource_quota_impl.h
include/grpcpp/security/auth_context.h
include/grpcpp/security/auth_metadata_processor.h
include/grpcpp/security/credentials.h
@@ -3605,7 +3604,6 @@ foreach(_hdr
include/grpcpp/impl/server_initializer.h
include/grpcpp/impl/service_type.h
include/grpcpp/resource_quota.h
- include/grpcpp/resource_quota_impl.h
include/grpcpp/security/auth_context.h
include/grpcpp/security/auth_metadata_processor.h
include/grpcpp/security/credentials.h
@@ -4561,7 +4559,6 @@ foreach(_hdr
include/grpcpp/impl/server_initializer.h
include/grpcpp/impl/service_type.h
include/grpcpp/resource_quota.h
- include/grpcpp/resource_quota_impl.h
include/grpcpp/security/auth_context.h
include/grpcpp/security/auth_metadata_processor.h
include/grpcpp/security/credentials.h
diff --git a/Makefile b/Makefile
index f47184b3436..91516de9a27 100644
--- a/Makefile
+++ b/Makefile
@@ -5438,7 +5438,6 @@ PUBLIC_HEADERS_CXX += \
include/grpcpp/impl/server_initializer.h \
include/grpcpp/impl/service_type.h \
include/grpcpp/resource_quota.h \
- include/grpcpp/resource_quota_impl.h \
include/grpcpp/security/auth_context.h \
include/grpcpp/security/auth_metadata_processor.h \
include/grpcpp/security/credentials.h \
@@ -6038,7 +6037,6 @@ PUBLIC_HEADERS_CXX += \
include/grpcpp/impl/server_initializer.h \
include/grpcpp/impl/service_type.h \
include/grpcpp/resource_quota.h \
- include/grpcpp/resource_quota_impl.h \
include/grpcpp/security/auth_context.h \
include/grpcpp/security/auth_metadata_processor.h \
include/grpcpp/security/credentials.h \
@@ -6951,7 +6949,6 @@ PUBLIC_HEADERS_CXX += \
include/grpcpp/impl/server_initializer.h \
include/grpcpp/impl/service_type.h \
include/grpcpp/resource_quota.h \
- include/grpcpp/resource_quota_impl.h \
include/grpcpp/security/auth_context.h \
include/grpcpp/security/auth_metadata_processor.h \
include/grpcpp/security/credentials.h \
diff --git a/build.yaml b/build.yaml
index cfd6cdb18a4..02ecaa221e6 100644
--- a/build.yaml
+++ b/build.yaml
@@ -1360,7 +1360,6 @@ filegroups:
- include/grpcpp/impl/server_initializer.h
- include/grpcpp/impl/service_type.h
- include/grpcpp/resource_quota.h
- - include/grpcpp/resource_quota_impl.h
- include/grpcpp/security/auth_context.h
- include/grpcpp/security/auth_metadata_processor.h
- include/grpcpp/security/credentials.h
diff --git a/gRPC-C++.podspec b/gRPC-C++.podspec
index 2d50f28ff2a..e755b7aa602 100644
--- a/gRPC-C++.podspec
+++ b/gRPC-C++.podspec
@@ -105,7 +105,6 @@ Pod::Spec.new do |s|
'include/grpcpp/impl/server_initializer.h',
'include/grpcpp/impl/service_type.h',
'include/grpcpp/resource_quota.h',
- 'include/grpcpp/resource_quota_impl.h',
'include/grpcpp/security/auth_context.h',
'include/grpcpp/security/auth_metadata_processor.h',
'include/grpcpp/security/credentials.h',
diff --git a/include/grpcpp/resource_quota.h b/include/grpcpp/resource_quota.h
index 333767b95c5..50bd1cb849a 100644
--- a/include/grpcpp/resource_quota.h
+++ b/include/grpcpp/resource_quota.h
@@ -1,6 +1,6 @@
/*
*
- * Copyright 2019 gRPC authors.
+ * Copyright 2016 gRPC authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -19,11 +19,50 @@
#ifndef GRPCPP_RESOURCE_QUOTA_H
#define GRPCPP_RESOURCE_QUOTA_H
-#include <grpcpp/resource_quota_impl.h>
+struct grpc_resource_quota;
+
+#include <grpcpp/impl/codegen/config.h>
+#include <grpcpp/impl/codegen/grpc_library.h>
namespace grpc {
-typedef ::grpc_impl::ResourceQuota ResourceQuota;
+/// ResourceQuota represents a bound on memory and thread usage by the gRPC
+/// library. A ResourceQuota can be attached to a server (via \a ServerBuilder),
+/// or a client channel (via \a ChannelArguments).
+/// gRPC will attempt to keep memory and threads used by all attached entities
+/// below the ResourceQuota bound.
+class ResourceQuota final : private GrpcLibraryCodegen {
+ public:
+ /// \param name - a unique name for this ResourceQuota.
+ explicit ResourceQuota(const grpc::string& name);
+ ResourceQuota();
+ ~ResourceQuota();
+
+ /// Resize this \a ResourceQuota to a new size. If \a new_size is smaller
+ /// than the current size of the pool, memory usage will be monotonically
+ /// decreased until it falls under \a new_size.
+ /// No time bound is given for this to occur however.
+ ResourceQuota& Resize(size_t new_size);
+
+ /// Set the max number of threads that can be allocated from this
+ /// ResourceQuota object.
+ ///
+ /// If the new_max_threads value is smaller than the current value, no new
+ /// threads are allocated until the number of active threads fall below
+ /// new_max_threads. There is no time bound on when this may happen i.e none
+ /// of the current threads are forcefully destroyed and all threads run their
+ /// normal course.
+ ResourceQuota& SetMaxThreads(int new_max_threads);
+
+ grpc_resource_quota* c_resource_quota() const { return impl_; }
+
+ private:
+ ResourceQuota(const ResourceQuota& rhs);
+ ResourceQuota& operator=(const ResourceQuota& rhs);
+
+ grpc_resource_quota* const impl_;
+};
+
} // namespace grpc
#endif // GRPCPP_RESOURCE_QUOTA_H
diff --git a/include/grpcpp/resource_quota_impl.h b/include/grpcpp/resource_quota_impl.h
deleted file mode 100644
index 16c0e35385b..00000000000
--- a/include/grpcpp/resource_quota_impl.h
+++ /dev/null
@@ -1,68 +0,0 @@
-/*
- *
- * Copyright 2016 gRPC authors.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-#ifndef GRPCPP_RESOURCE_QUOTA_IMPL_H
-#define GRPCPP_RESOURCE_QUOTA_IMPL_H
-
-struct grpc_resource_quota;
-
-#include <grpcpp/impl/codegen/config.h>
-#include <grpcpp/impl/codegen/grpc_library.h>
-
-namespace grpc_impl {
-
-/// ResourceQuota represents a bound on memory and thread usage by the gRPC
-/// library. A ResourceQuota can be attached to a server (via \a ServerBuilder),
-/// or a client channel (via \a ChannelArguments).
-/// gRPC will attempt to keep memory and threads used by all attached entities
-/// below the ResourceQuota bound.
-class ResourceQuota final : private ::grpc::GrpcLibraryCodegen {
- public:
- /// \param name - a unique name for this ResourceQuota.
- explicit ResourceQuota(const grpc::string& name);
- ResourceQuota();
- ~ResourceQuota();
-
- /// Resize this \a ResourceQuota to a new size. If \a new_size is smaller
- /// than the current size of the pool, memory usage will be monotonically
- /// decreased until it falls under \a new_size.
- /// No time bound is given for this to occur however.
- ResourceQuota& Resize(size_t new_size);
-
- /// Set the max number of threads that can be allocated from this
- /// ResourceQuota object.
- ///
- /// If the new_max_threads value is smaller than the current value, no new
- /// threads are allocated until the number of active threads fall below
- /// new_max_threads. There is no time bound on when this may happen i.e none
- /// of the current threads are forcefully destroyed and all threads run their
- /// normal course.
- ResourceQuota& SetMaxThreads(int new_max_threads);
-
- grpc_resource_quota* c_resource_quota() const { return impl_; }
-
- private:
- ResourceQuota(const ResourceQuota& rhs);
- ResourceQuota& operator=(const ResourceQuota& rhs);
-
- grpc_resource_quota* const impl_;
-};
-
-} // namespace grpc_impl
-
-#endif // GRPCPP_RESOURCE_QUOTA_IMPL_H
diff --git a/include/grpcpp/server_builder.h b/include/grpcpp/server_builder.h
index 4c00f021d11..498e5b7bb31 100644
--- a/include/grpcpp/server_builder.h
+++ b/include/grpcpp/server_builder.h
@@ -35,14 +35,10 @@
struct grpc_resource_quota;
-namespace grpc_impl {
-
-class ResourceQuota;
-}
-
namespace grpc {
class AsyncGenericService;
+class ResourceQuota;
class CompletionQueue;
class Server;
class ServerCompletionQueue;
@@ -190,8 +186,7 @@ class ServerBuilder {
grpc_compression_algorithm algorithm);
/// Set the attached buffer pool for this server
- ServerBuilder& SetResourceQuota(
- const ::grpc_impl::ResourceQuota& resource_quota);
+ ServerBuilder& SetResourceQuota(const ResourceQuota& resource_quota);
ServerBuilder& SetOption(std::unique_ptr<ServerBuilderOption> option);
diff --git a/include/grpcpp/support/channel_arguments.h b/include/grpcpp/support/channel_arguments.h
index 48ae4246462..217929d4aca 100644
--- a/include/grpcpp/support/channel_arguments.h
+++ b/include/grpcpp/support/channel_arguments.h
@@ -26,16 +26,13 @@
#include <grpc/grpc.h>
#include <grpcpp/support/config.h>
-namespace grpc_impl {
-
-class ResourceQuota;
-}
-
namespace grpc {
namespace testing {
class ChannelArgumentsTest;
} // namespace testing
+class ResourceQuota;
+
/// Options for channel creation. The user can use generic setters to pass
/// key value pairs down to C channel creation code. For gRPC related options,
/// concrete setters are provided.
@@ -86,7 +83,7 @@ class ChannelArguments {
void SetUserAgentPrefix(const grpc::string& user_agent_prefix);
/// Set the buffer pool to be attached to the constructed channel.
- void SetResourceQuota(const ::grpc_impl::ResourceQuota& resource_quota);
+ void SetResourceQuota(const ResourceQuota& resource_quota);
/// Set the max receive and send message sizes.
void SetMaxReceiveMessageSize(int size);
diff --git a/src/cpp/common/channel_arguments.cc b/src/cpp/common/channel_arguments.cc
index c3d75054b9b..214d72f853f 100644
--- a/src/cpp/common/channel_arguments.cc
+++ b/src/cpp/common/channel_arguments.cc
@@ -143,7 +143,7 @@ void ChannelArguments::SetUserAgentPrefix(
}
void ChannelArguments::SetResourceQuota(
- const grpc_impl::ResourceQuota& resource_quota) {
+ const grpc::ResourceQuota& resource_quota) {
SetPointerWithVtable(GRPC_ARG_RESOURCE_QUOTA,
resource_quota.c_resource_quota(),
grpc_resource_quota_arg_vtable());
diff --git a/src/cpp/common/resource_quota_cc.cc b/src/cpp/common/resource_quota_cc.cc
index 4fab2975d89..276e5f79548 100644
--- a/src/cpp/common/resource_quota_cc.cc
+++ b/src/cpp/common/resource_quota_cc.cc
@@ -19,7 +19,7 @@
#include <grpc/grpc.h>
#include <grpcpp/resource_quota.h>
-namespace grpc_impl {
+namespace grpc {
ResourceQuota::ResourceQuota() : impl_(grpc_resource_quota_create(nullptr)) {}
@@ -37,4 +37,4 @@ ResourceQuota& ResourceQuota::SetMaxThreads(int new_max_threads) {
grpc_resource_quota_set_max_threads(impl_, new_max_threads);
return *this;
}
-} // namespace grpc_impl
+} // namespace grpc
diff --git a/src/cpp/server/server_builder.cc b/src/cpp/server/server_builder.cc
index f60c77dc8d6..cd0e516d9a3 100644
--- a/src/cpp/server/server_builder.cc
+++ b/src/cpp/server/server_builder.cc
@@ -29,11 +29,6 @@
#include "src/core/lib/gpr/useful.h"
#include "src/cpp/server/thread_pool_interface.h"
-namespace grpc_impl {
-
-class ResourceQuota;
-}
-
namespace grpc {
static std::vector<std::unique_ptr<ServerBuilderPlugin> (*)()>*
@@ -169,7 +164,7 @@ ServerBuilder& ServerBuilder::SetDefaultCompressionAlgorithm(
}
ServerBuilder& ServerBuilder::SetResourceQuota(
- const grpc_impl::ResourceQuota& resource_quota) {
+ const grpc::ResourceQuota& resource_quota) {
if (resource_quota_ != nullptr) {
grpc_resource_quota_unref(resource_quota_);
}
diff --git a/test/cpp/end2end/end2end_test.cc b/test/cpp/end2end/end2end_test.cc
index f7b9ee4b0b0..f58a472bfaf 100644
--- a/test/cpp/end2end/end2end_test.cc
+++ b/test/cpp/end2end/end2end_test.cc
@@ -64,11 +64,6 @@ using std::chrono::system_clock;
} \
} while (0)
-namespace grpc_impl {
-
-class ResourceQuota;
-}
-
namespace grpc {
namespace testing {
namespace {
diff --git a/test/cpp/end2end/thread_stress_test.cc b/test/cpp/end2end/thread_stress_test.cc
index e308e591d1a..e30ce0dbcbf 100644
--- a/test/cpp/end2end/thread_stress_test.cc
+++ b/test/cpp/end2end/thread_stress_test.cc
@@ -48,11 +48,6 @@ const int kNumAsyncReceiveThreads = 50;
const int kNumAsyncServerThreads = 50;
const int kNumRpcs = 1000; // Number of RPCs per thread
-namespace grpc_impl {
-
-class ResourceQuota;
-}
-
namespace grpc {
namespace testing {
diff --git a/test/cpp/qps/server.h b/test/cpp/qps/server.h
index 3aec8644a94..89b0e3af4b2 100644
--- a/test/cpp/qps/server.h
+++ b/test/cpp/qps/server.h
@@ -34,11 +34,6 @@
#include "test/cpp/qps/usage_timer.h"
#include "test/cpp/util/test_credentials_provider.h"
-namespace grpc_impl {
-
-class ResourceQuota;
-}
-
namespace grpc {
namespace testing {
diff --git a/test/cpp/qps/server_async.cc b/test/cpp/qps/server_async.cc
index 9343fd311e1..a5f8347c269 100644
--- a/test/cpp/qps/server_async.cc
+++ b/test/cpp/qps/server_async.cc
@@ -27,6 +27,7 @@
#include <grpc/support/alloc.h>
#include <grpc/support/log.h>
#include <grpcpp/generic/async_generic_service.h>
+#include <grpcpp/resource_quota.h>
#include <grpcpp/security/server_credentials.h>
#include <grpcpp/server.h>
#include <grpcpp/server_builder.h>
diff --git a/tools/doxygen/Doxyfile.c++ b/tools/doxygen/Doxyfile.c++
index 367160a0ca9..9f17a25298a 100644
--- a/tools/doxygen/Doxyfile.c++
+++ b/tools/doxygen/Doxyfile.c++
@@ -995,7 +995,6 @@ include/grpcpp/impl/server_builder_plugin.h \
include/grpcpp/impl/server_initializer.h \
include/grpcpp/impl/service_type.h \
include/grpcpp/resource_quota.h \
-include/grpcpp/resource_quota_impl.h \
include/grpcpp/security/auth_context.h \
include/grpcpp/security/auth_metadata_processor.h \
include/grpcpp/security/credentials.h \
diff --git a/tools/doxygen/Doxyfile.c++.internal b/tools/doxygen/Doxyfile.c++.internal
index 4c1ba9d4f9d..c0078bf2764 100644
--- a/tools/doxygen/Doxyfile.c++.internal
+++ b/tools/doxygen/Doxyfile.c++.internal
@@ -997,7 +997,6 @@ include/grpcpp/impl/server_builder_plugin.h \
include/grpcpp/impl/server_initializer.h \
include/grpcpp/impl/service_type.h \
include/grpcpp/resource_quota.h \
-include/grpcpp/resource_quota_impl.h \
include/grpcpp/security/auth_context.h \
include/grpcpp/security/auth_metadata_processor.h \
include/grpcpp/security/credentials.h \
diff --git a/tools/run_tests/generated/sources_and_headers.json b/tools/run_tests/generated/sources_and_headers.json
index 96fa00e387e..7a72a885336 100644
--- a/tools/run_tests/generated/sources_and_headers.json
+++ b/tools/run_tests/generated/sources_and_headers.json
@@ -11391,7 +11391,6 @@
"include/grpcpp/impl/server_initializer.h",
"include/grpcpp/impl/service_type.h",
"include/grpcpp/resource_quota.h",
- "include/grpcpp/resource_quota_impl.h",
"include/grpcpp/security/auth_context.h",
"include/grpcpp/security/auth_metadata_processor.h",
"include/grpcpp/security/credentials.h",
@@ -11501,7 +11500,6 @@
"include/grpcpp/impl/server_initializer.h",
"include/grpcpp/impl/service_type.h",
"include/grpcpp/resource_quota.h",
- "include/grpcpp/resource_quota_impl.h",
"include/grpcpp/security/auth_context.h",
"include/grpcpp/security/auth_metadata_processor.h",
"include/grpcpp/security/credentials.h",
From 33ebf719a55fd601f9b01f21c367fbc60c4d0af5 Mon Sep 17 00:00:00 2001
From: Muxi Yan <mxyan@google.com>
Date: Tue, 12 Mar 2019 08:43:57 -0700
Subject: [PATCH 105/138] use cached grpc ssl credential
---
include/grpc/grpc_security.h | 9 ++++++
.../private/GRPCSecureChannelFactory.h | 5 +++
.../private/GRPCSecureChannelFactory.m | 32 +++----------------
3 files changed, 18 insertions(+), 28 deletions(-)
diff --git a/include/grpc/grpc_security.h b/include/grpc/grpc_security.h
index f0323eb16a1..9b7822627e0 100644
--- a/include/grpc/grpc_security.h
+++ b/include/grpc/grpc_security.h
@@ -191,6 +191,15 @@ typedef struct {
try to get the roots set by grpc_override_ssl_default_roots. Eventually,
if all these fail, it will try to get the roots from a well-known place on
disk (in the grpc install directory).
+
+ gRPC has implemented root cache if the underlying OpenSSL library supports
+ it. The gRPC root certificates cache is only applicable on the default
+ root certificates, which is used when this parameter is nullptr. If user
+ provides their own pem_root_certs, when creating an SSL credential object,
+ gRPC would not be able to cache it, and each subchannel will generate a
+ copy of the root store. So it is recommended to avoid providing large room
+ pem with pem_root_certs parameter to avoid excessive memory consumption,
+ particularly on mobile platforms such as iOS.
- pem_key_cert_pair is a pointer on the object containing client's private
key and certificate chain. This parameter can be NULL if the client does
not have such a key/cert pair.
diff --git a/src/objective-c/GRPCClient/private/GRPCSecureChannelFactory.h b/src/objective-c/GRPCClient/private/GRPCSecureChannelFactory.h
index 588239b7064..572f20d341f 100644
--- a/src/objective-c/GRPCClient/private/GRPCSecureChannelFactory.h
+++ b/src/objective-c/GRPCClient/private/GRPCSecureChannelFactory.h
@@ -23,6 +23,11 @@ NS_ASSUME_NONNULL_BEGIN
@interface GRPCSecureChannelFactory : NSObject<GRPCChannelFactory>
+/**
+ * Creates a secure channel factory which uses provided root certificates and client authentication
+ * credentials. If rootCerts is nil, gRPC will use its default root certificates. If rootCerts is
+ * provided, it must only contain the server's CA to avoid memory issue.
+ */
+ (nullable instancetype)factoryWithPEMRootCertificates:(nullable NSString *)rootCerts
privateKey:(nullable NSString *)privateKey
certChain:(nullable NSString *)certChain
diff --git a/src/objective-c/GRPCClient/private/GRPCSecureChannelFactory.m b/src/objective-c/GRPCClient/private/GRPCSecureChannelFactory.m
index 96998895364..b1a6797b9e3 100644
--- a/src/objective-c/GRPCClient/private/GRPCSecureChannelFactory.m
+++ b/src/objective-c/GRPCClient/private/GRPCSecureChannelFactory.m
@@ -52,44 +52,20 @@
privateKey:(NSString *)privateKey
certChain:(NSString *)certChain
error:(NSError **)errorPtr {
- static NSData *defaultRootsASCII;
- static NSError *defaultRootsError;
static dispatch_once_t loading;
dispatch_once(&loading, ^{
NSString *defaultPath = @"gRPCCertificates.bundle/roots"; // .pem
// Do not use NSBundle.mainBundle, as it's nil for tests of library projects.
NSBundle *bundle = [NSBundle bundleForClass:[self class]];
NSString *path = [bundle pathForResource:defaultPath ofType:@"pem"];
- NSError *error;
- // Files in PEM format can have non-ASCII characters in their comments (e.g. for the name of the
- // issuer). Load them as UTF8 and produce an ASCII equivalent.
- NSString *contentInUTF8 =
- [NSString stringWithContentsOfFile:path encoding:NSUTF8StringEncoding error:&error];
- if (contentInUTF8 == nil) {
- defaultRootsError = error;
- return;
- }
- defaultRootsASCII = [self nullTerminatedDataWithString:contentInUTF8];
+ setenv(GRPC_DEFAULT_SSL_ROOTS_FILE_PATH_ENV_VAR,
+ [path cStringUsingEncoding:NSUTF8StringEncoding], 1);
});
- NSData *rootsASCII;
+ NSData *rootsASCII = nil;
+ // if rootCerts is not provided, gRPC will use its own default certs
if (rootCerts != nil) {
rootsASCII = [self nullTerminatedDataWithString:rootCerts];
- } else {
- if (defaultRootsASCII == nil) {
- if (errorPtr) {
- *errorPtr = defaultRootsError;
- }
- NSAssert(
- defaultRootsASCII, NSObjectNotAvailableException,
- @"Could not read gRPCCertificates.bundle/roots.pem. This file, "
- "with the root certificates, is needed to establish secure (TLS) connections. "
- "Because the file is distributed with the gRPC library, this error is usually a sign "
- "that the library wasn't configured correctly for your project. Error: %@",
- defaultRootsError);
- return nil;
- }
- rootsASCII = defaultRootsASCII;
}
grpc_channel_credentials *creds = NULL;
From c9421eeb8518cdf4abe2960a87a89f9f2f9f49b5 Mon Sep 17 00:00:00 2001
From: "Mark D. Roth" <roth@google.com>
Date: Tue, 12 Mar 2019 13:15:45 -0700
Subject: [PATCH 106/138] Fix state reported by pick_first when we receive a
GOAWAY with a pending subchannel list.
---
.../lb_policy/pick_first/pick_first.cc | 61 +++++++++++++------
.../chttp2/transport/chttp2_transport.cc | 6 +-
test/cpp/end2end/BUILD | 1 +
test/cpp/end2end/client_lb_end2end_test.cc | 54 +++++++++++++++-
4 files changed, 100 insertions(+), 22 deletions(-)
diff --git a/src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc b/src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc
index 0ac0f41d4ef..15d953cd92f 100644
--- a/src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc
+++ b/src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc
@@ -102,6 +102,14 @@ class PickFirst : public LoadBalancingPolicy {
PickFirst* p = static_cast<PickFirst*>(policy());
p->Unref(DEBUG_LOCATION, "subchannel_list");
}
+
+ bool in_transient_failure() const { return in_transient_failure_; }
+ void set_in_transient_failure(bool in_transient_failure) {
+ in_transient_failure_ = in_transient_failure;
+ }
+
+ private:
+ bool in_transient_failure_ = false;
};
class Picker : public SubchannelPicker {
@@ -368,12 +376,21 @@ void PickFirst::PickFirstSubchannelData::ProcessConnectivityChangeLocked(
p->selected_ = nullptr;
StopConnectivityWatchLocked();
p->subchannel_list_ = std::move(p->latest_pending_subchannel_list_);
- grpc_error* new_error = GRPC_ERROR_CREATE_REFERENCING_FROM_STATIC_STRING(
- "selected subchannel not ready; switching to pending update", &error,
- 1);
- p->channel_control_helper()->UpdateState(
- GRPC_CHANNEL_TRANSIENT_FAILURE, GRPC_ERROR_REF(new_error),
- UniquePtr<SubchannelPicker>(New<TransientFailurePicker>(new_error)));
+ // Set our state to that of the pending subchannel list.
+ if (p->subchannel_list_->in_transient_failure()) {
+ grpc_error* new_error =
+ GRPC_ERROR_CREATE_REFERENCING_FROM_STATIC_STRING(
+ "selected subchannel failed; switching to pending update",
+ &error, 1);
+ p->channel_control_helper()->UpdateState(
+ GRPC_CHANNEL_TRANSIENT_FAILURE, GRPC_ERROR_REF(new_error),
+ UniquePtr<SubchannelPicker>(
+ New<TransientFailurePicker>(new_error)));
+ } else {
+ p->channel_control_helper()->UpdateState(
+ GRPC_CHANNEL_CONNECTING, GRPC_ERROR_NONE,
+ UniquePtr<SubchannelPicker>(New<QueuePicker>(p->Ref())));
+ }
} else {
if (connectivity_state == GRPC_CHANNEL_TRANSIENT_FAILURE) {
// If the selected subchannel goes bad, request a re-resolution. We
@@ -382,7 +399,6 @@ void PickFirst::PickFirstSubchannelData::ProcessConnectivityChangeLocked(
// to connect to the re-resolved backends until we leave IDLE state.
p->idle_ = true;
p->channel_control_helper()->RequestReresolution();
- // In transient failure. Rely on re-resolution to recover.
p->selected_ = nullptr;
StopConnectivityWatchLocked();
p->channel_control_helper()->UpdateState(
@@ -418,6 +434,7 @@ void PickFirst::PickFirstSubchannelData::ProcessConnectivityChangeLocked(
// for a subchannel in p->latest_pending_subchannel_list_. The
// goal here is to find a subchannel from the update that we can
// select in place of the current one.
+ subchannel_list()->set_in_transient_failure(false);
switch (connectivity_state) {
case GRPC_CHANNEL_READY: {
// Renew notification.
@@ -431,17 +448,25 @@ void PickFirst::PickFirstSubchannelData::ProcessConnectivityChangeLocked(
size_t next_index =
(sd->Index() + 1) % subchannel_list()->num_subchannels();
sd = subchannel_list()->subchannel(next_index);
- // Case 1: Only set state to TRANSIENT_FAILURE if we've tried
- // all subchannels.
- if (sd->Index() == 0 && subchannel_list() == p->subchannel_list_.get()) {
- p->channel_control_helper()->RequestReresolution();
- grpc_error* new_error =
- GRPC_ERROR_CREATE_REFERENCING_FROM_STATIC_STRING(
- "failed to connect to all addresses", &error, 1);
- p->channel_control_helper()->UpdateState(
- GRPC_CHANNEL_TRANSIENT_FAILURE, GRPC_ERROR_REF(new_error),
- UniquePtr<SubchannelPicker>(
- New<TransientFailurePicker>(new_error)));
+ // If we're tried all subchannels, set state to TRANSIENT_FAILURE.
+ if (sd->Index() == 0) {
+ // Re-resolve if this is the most recent subchannel list.
+ if (subchannel_list() == (p->latest_pending_subchannel_list_ != nullptr
+ ? p->latest_pending_subchannel_list_.get()
+ : p->subchannel_list_.get())) {
+ p->channel_control_helper()->RequestReresolution();
+ }
+ subchannel_list()->set_in_transient_failure(true);
+ // Only report new state in case 1.
+ if (subchannel_list() == p->subchannel_list_.get()) {
+ grpc_error* new_error =
+ GRPC_ERROR_CREATE_REFERENCING_FROM_STATIC_STRING(
+ "failed to connect to all addresses", &error, 1);
+ p->channel_control_helper()->UpdateState(
+ GRPC_CHANNEL_TRANSIENT_FAILURE, GRPC_ERROR_REF(new_error),
+ UniquePtr<SubchannelPicker>(
+ New<TransientFailurePicker>(new_error)));
+ }
}
sd->CheckConnectivityStateAndStartWatchingLocked();
break;
diff --git a/src/core/ext/transport/chttp2/transport/chttp2_transport.cc b/src/core/ext/transport/chttp2/transport/chttp2_transport.cc
index 888c1757be1..829bee6bedd 100644
--- a/src/core/ext/transport/chttp2/transport/chttp2_transport.cc
+++ b/src/core/ext/transport/chttp2/transport/chttp2_transport.cc
@@ -1136,8 +1136,10 @@ void grpc_chttp2_add_incoming_goaway(grpc_chttp2_transport* t,
}
t->goaway_error = grpc_error_set_str(
grpc_error_set_int(
- GRPC_ERROR_CREATE_FROM_STATIC_STRING("GOAWAY received"),
- GRPC_ERROR_INT_HTTP2_ERROR, static_cast<intptr_t>(goaway_error)),
+ grpc_error_set_int(
+ GRPC_ERROR_CREATE_FROM_STATIC_STRING("GOAWAY received"),
+ GRPC_ERROR_INT_HTTP2_ERROR, static_cast<intptr_t>(goaway_error)),
+ GRPC_ERROR_INT_GRPC_STATUS, GRPC_STATUS_UNAVAILABLE),
GRPC_ERROR_STR_RAW_BYTES, goaway_text);
/* We want to log this irrespective of whether http tracing is enabled */
diff --git a/test/cpp/end2end/BUILD b/test/cpp/end2end/BUILD
index 12258a64e00..68e0ec3cef1 100644
--- a/test/cpp/end2end/BUILD
+++ b/test/cpp/end2end/BUILD
@@ -402,6 +402,7 @@ grpc_cc_test(
name = "client_lb_end2end_test",
srcs = ["client_lb_end2end_test.cc"],
external_deps = [
+ "gmock",
"gtest",
],
deps = [
diff --git a/test/cpp/end2end/client_lb_end2end_test.cc b/test/cpp/end2end/client_lb_end2end_test.cc
index 996ba0edbbe..3cd06e9e28c 100644
--- a/test/cpp/end2end/client_lb_end2end_test.cc
+++ b/test/cpp/end2end/client_lb_end2end_test.cc
@@ -56,6 +56,7 @@
#include "test/core/util/test_lb_policies.h"
#include "test/cpp/end2end/test_service_impl.h"
+#include <gmock/gmock.h>
#include <gtest/gtest.h>
using grpc::testing::EchoRequest;
@@ -221,9 +222,11 @@ class ClientLbEnd2endTest : public ::testing::Test {
response_generator_->SetFailureOnReresolution();
}
- std::vector<int> GetServersPorts() {
+ std::vector<int> GetServersPorts(size_t start_index = 0) {
std::vector<int> ports;
- for (const auto& server : servers_) ports.push_back(server->port_);
+ for (size_t i = start_index; i < servers_.size(); ++i) {
+ ports.push_back(servers_[i]->port_);
+ }
return ports;
}
@@ -897,6 +900,53 @@ TEST_F(ClientLbEnd2endTest, PickFirstIdleOnDisconnect) {
servers_.clear();
}
+TEST_F(ClientLbEnd2endTest, PickFirstPendingUpdateAndSelectedSubchannelFails) {
+ auto channel = BuildChannel(""); // pick_first is the default.
+ auto stub = BuildStub(channel);
+ // Create a number of servers, but only start 1 of them.
+ CreateServers(10);
+ StartServer(0);
+ // Initially resolve to first server and make sure it connects.
+ gpr_log(GPR_INFO, "Phase 1: Connect to first server.");
+ SetNextResolution({servers_[0]->port_});
+ CheckRpcSendOk(stub, DEBUG_LOCATION, true /* wait_for_ready */);
+ EXPECT_EQ(channel->GetState(false), GRPC_CHANNEL_READY);
+ // Send a resolution update with the remaining servers, none of which are
+ // running yet, so the update will stay pending. Note that it's important
+ // to have multiple servers here, or else the test will be flaky; with only
+ // one server, the pending subchannel list has already gone into
+ // TRANSIENT_FAILURE due to hitting the end of the list by the time we
+ // check the state.
+ gpr_log(GPR_INFO,
+ "Phase 2: Resolver update pointing to remaining "
+ "(not started) servers.");
+ SetNextResolution(GetServersPorts(1 /* start_index */));
+ // RPCs will continue to be sent to the first server.
+ CheckRpcSendOk(stub, DEBUG_LOCATION);
+ // Now stop the first server, so that the current subchannel list
+ // fails. This should cause us to immediately swap over to the
+ // pending list, even though it's not yet connected. The state should
+ // be set to CONNECTING, since that's what the pending subchannel list
+ // was doing when we swapped over.
+ gpr_log(GPR_INFO, "Phase 3: Stopping first server.");
+ servers_[0]->Shutdown();
+ WaitForChannelNotReady(channel.get());
+ // TODO(roth): This should always return CONNECTING, but it's flaky
+ // between that and TRANSIENT_FAILURE. I suspect that this problem
+ // will go away once we move the backoff code out of the subchannel
+ // and into the LB policies.
+ EXPECT_THAT(channel->GetState(false),
+ ::testing::AnyOf(GRPC_CHANNEL_CONNECTING,
+ GRPC_CHANNEL_TRANSIENT_FAILURE));
+ // Now start the second server.
+ gpr_log(GPR_INFO, "Phase 4: Starting second server.");
+ StartServer(1);
+ // The channel should go to READY state and RPCs should go to the
+ // second server.
+ WaitForChannelReady(channel.get());
+ WaitForServer(stub, 1, DEBUG_LOCATION, true /* ignore_failure */);
+}
+
TEST_F(ClientLbEnd2endTest, RoundRobin) {
// Start servers and send one RPC per server.
const int kNumServers = 3;
From fac3ec2563fc1174e9c9f3ade4c2d10a9878020d Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Wed, 13 Mar 2019 11:53:06 +0100
Subject: [PATCH 107/138] add netstandard2.0 target for Grpc.* packages
---
src/csharp/Grpc.Auth/Grpc.Auth.csproj | 2 +-
src/csharp/Grpc.Core.Api/Grpc.Core.Api.csproj | 2 +-
src/csharp/Grpc.Core.NativeDebug/Grpc.Core.NativeDebug.csproj | 2 +-
src/csharp/Grpc.Core.Testing/Grpc.Core.Testing.csproj | 2 +-
src/csharp/Grpc.Core/Grpc.Core.csproj | 2 +-
src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.csproj | 2 +-
src/csharp/Grpc.Reflection/Grpc.Reflection.csproj | 2 +-
src/csharp/Grpc/Grpc.csproj | 2 +-
8 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/src/csharp/Grpc.Auth/Grpc.Auth.csproj b/src/csharp/Grpc.Auth/Grpc.Auth.csproj
index e9a6d2cc198..e8974f7221f 100755
--- a/src/csharp/Grpc.Auth/Grpc.Auth.csproj
+++ b/src/csharp/Grpc.Auth/Grpc.Auth.csproj
@@ -15,7 +15,7 @@
</PropertyGroup>
<PropertyGroup>
- <TargetFrameworks>net45;netstandard1.5</TargetFrameworks>
+ <TargetFrameworks>net45;netstandard1.5;netstandard2.0</TargetFrameworks>
<DefineConstants>$(DefineConstants);SIGNED</DefineConstants>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
diff --git a/src/csharp/Grpc.Core.Api/Grpc.Core.Api.csproj b/src/csharp/Grpc.Core.Api/Grpc.Core.Api.csproj
index 4b772f6276a..556f65f4b32 100755
--- a/src/csharp/Grpc.Core.Api/Grpc.Core.Api.csproj
+++ b/src/csharp/Grpc.Core.Api/Grpc.Core.Api.csproj
@@ -15,7 +15,7 @@
</PropertyGroup>
<PropertyGroup>
- <TargetFrameworks>net45;netstandard1.5</TargetFrameworks>
+ <TargetFrameworks>net45;netstandard1.5;netstandard2.0</TargetFrameworks>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.Core.NativeDebug/Grpc.Core.NativeDebug.csproj b/src/csharp/Grpc.Core.NativeDebug/Grpc.Core.NativeDebug.csproj
index df4f31dc421..3cd5ba4fa6f 100644
--- a/src/csharp/Grpc.Core.NativeDebug/Grpc.Core.NativeDebug.csproj
+++ b/src/csharp/Grpc.Core.NativeDebug/Grpc.Core.NativeDebug.csproj
@@ -15,7 +15,7 @@
</PropertyGroup>
<PropertyGroup>
- <TargetFrameworks>net45;netstandard1.5</TargetFrameworks>
+ <TargetFrameworks>net45;netstandard1.5;netstandard2.0</TargetFrameworks>
<!-- This package only carries native debug symbols -->
<IncludeBuildOutput>false</IncludeBuildOutput>
<DisableImplicitFrameworkReferences>true</DisableImplicitFrameworkReferences>
diff --git a/src/csharp/Grpc.Core.Testing/Grpc.Core.Testing.csproj b/src/csharp/Grpc.Core.Testing/Grpc.Core.Testing.csproj
index 90ed88201d0..3727639f44a 100755
--- a/src/csharp/Grpc.Core.Testing/Grpc.Core.Testing.csproj
+++ b/src/csharp/Grpc.Core.Testing/Grpc.Core.Testing.csproj
@@ -15,7 +15,7 @@
</PropertyGroup>
<PropertyGroup>
- <TargetFrameworks>net45;netstandard1.5</TargetFrameworks>
+ <TargetFrameworks>net45;netstandard1.5;netstandard2.0</TargetFrameworks>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.Core/Grpc.Core.csproj b/src/csharp/Grpc.Core/Grpc.Core.csproj
index e6ccff823a4..5a73ac3deb8 100755
--- a/src/csharp/Grpc.Core/Grpc.Core.csproj
+++ b/src/csharp/Grpc.Core/Grpc.Core.csproj
@@ -15,7 +15,7 @@
</PropertyGroup>
<PropertyGroup>
- <TargetFrameworks>net45;netstandard1.5</TargetFrameworks>
+ <TargetFrameworks>net45;netstandard1.5;netstandard2.0</TargetFrameworks>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.csproj b/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.csproj
index 4f3862deeb7..338c9c2c0d8 100755
--- a/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.csproj
+++ b/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.csproj
@@ -15,7 +15,7 @@
</PropertyGroup>
<PropertyGroup>
- <TargetFrameworks>net45;netstandard1.5</TargetFrameworks>
+ <TargetFrameworks>net45;netstandard1.5;netstandard2.0</TargetFrameworks>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc.Reflection/Grpc.Reflection.csproj b/src/csharp/Grpc.Reflection/Grpc.Reflection.csproj
index c5362252d00..f080e2085dd 100755
--- a/src/csharp/Grpc.Reflection/Grpc.Reflection.csproj
+++ b/src/csharp/Grpc.Reflection/Grpc.Reflection.csproj
@@ -15,7 +15,7 @@
</PropertyGroup>
<PropertyGroup>
- <TargetFrameworks>net45;netstandard1.5</TargetFrameworks>
+ <TargetFrameworks>net45;netstandard1.5;netstandard2.0</TargetFrameworks>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
diff --git a/src/csharp/Grpc/Grpc.csproj b/src/csharp/Grpc/Grpc.csproj
index c529c38e989..5174b97086e 100644
--- a/src/csharp/Grpc/Grpc.csproj
+++ b/src/csharp/Grpc/Grpc.csproj
@@ -15,7 +15,7 @@
</PropertyGroup>
<PropertyGroup>
- <TargetFrameworks>net45;netstandard1.5</TargetFrameworks>
+ <TargetFrameworks>net45;netstandard1.5;netstandard2.0</TargetFrameworks>
<!-- This is only a metapackage -->
<IncludeBuildOutput>false</IncludeBuildOutput>
<DisableImplicitFrameworkReferences>true</DisableImplicitFrameworkReferences>
From cfe021a3b39072ea0fa58ef778fbd96a1705581f Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Wed, 13 Mar 2019 11:54:13 +0100
Subject: [PATCH 108/138] netstandard2.0 tweaks
---
src/csharp/Grpc.Core/GrpcEnvironment.cs | 2 +-
src/csharp/Grpc.Core/Internal/NativeExtension.cs | 4 ++--
src/csharp/Grpc.Core/Internal/PlatformApis.cs | 4 ++--
src/csharp/Grpc.Core/Internal/UnmanagedLibrary.cs | 2 +-
src/csharp/Grpc.Core/Utils/TaskUtils.cs | 2 +-
5 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/src/csharp/Grpc.Core/GrpcEnvironment.cs b/src/csharp/Grpc.Core/GrpcEnvironment.cs
index 6ca694e0e46..3c83b03cc1f 100644
--- a/src/csharp/Grpc.Core/GrpcEnvironment.cs
+++ b/src/csharp/Grpc.Core/GrpcEnvironment.cs
@@ -448,7 +448,7 @@ namespace Grpc.Core
// the gRPC channels and servers before the application exits. The following
// hooks provide some extra handling for cases when this is not the case,
// in the effort to achieve a reasonable behavior on shutdown.
-#if NETSTANDARD1_5
+#if NETSTANDARD1_5 || NETSTANDARD2_0
// No action required at shutdown on .NET Core
// - In-progress P/Invoke calls (such as grpc_completion_queue_next) don't seem
// to prevent a .NET core application from terminating, so no special handling
diff --git a/src/csharp/Grpc.Core/Internal/NativeExtension.cs b/src/csharp/Grpc.Core/Internal/NativeExtension.cs
index 5177b69fd90..9935ef109c8 100644
--- a/src/csharp/Grpc.Core/Internal/NativeExtension.cs
+++ b/src/csharp/Grpc.Core/Internal/NativeExtension.cs
@@ -153,7 +153,7 @@ namespace Grpc.Core.Internal
private static string GetAssemblyPath()
{
var assembly = typeof(NativeExtension).GetTypeInfo().Assembly;
-#if NETSTANDARD1_5
+#if NETSTANDARD1_5 || NETSTANDARD2_0
// Assembly.EscapedCodeBase does not exist under CoreCLR, but assemblies imported from a nuget package
// don't seem to be shadowed by DNX-based projects at all.
return assembly.Location;
@@ -172,7 +172,7 @@ namespace Grpc.Core.Internal
#endif
}
-#if !NETSTANDARD1_5
+#if !NETSTANDARD1_5 && !NETSTANDARD2_0
private static bool IsFileUri(string uri)
{
return uri.ToLowerInvariant().StartsWith(Uri.UriSchemeFile);
diff --git a/src/csharp/Grpc.Core/Internal/PlatformApis.cs b/src/csharp/Grpc.Core/Internal/PlatformApis.cs
index a8f147545b4..8d7e8c2acb5 100644
--- a/src/csharp/Grpc.Core/Internal/PlatformApis.cs
+++ b/src/csharp/Grpc.Core/Internal/PlatformApis.cs
@@ -49,7 +49,7 @@ namespace Grpc.Core.Internal
static PlatformApis()
{
-#if NETSTANDARD1_5
+#if NETSTANDARD1_5 || NETSTANDARD2_0
isLinux = RuntimeInformation.IsOSPlatform(OSPlatform.Linux);
isMacOSX = RuntimeInformation.IsOSPlatform(OSPlatform.OSX);
isWindows = RuntimeInformation.IsOSPlatform(OSPlatform.Windows);
@@ -171,7 +171,7 @@ namespace Grpc.Core.Internal
public static string GetUnityRuntimePlatform()
{
GrpcPreconditions.CheckState(IsUnity, "Not running on Unity.");
-#if NETSTANDARD1_5
+#if NETSTANDARD1_5 || NETSTANDARD2_0
return Type.GetType(UnityEngineApplicationClassName).GetTypeInfo().GetProperty("platform").GetValue(null).ToString();
#else
return Type.GetType(UnityEngineApplicationClassName).GetProperty("platform").GetValue(null).ToString();
diff --git a/src/csharp/Grpc.Core/Internal/UnmanagedLibrary.cs b/src/csharp/Grpc.Core/Internal/UnmanagedLibrary.cs
index 1786fc2e3f6..056758df8ca 100644
--- a/src/csharp/Grpc.Core/Internal/UnmanagedLibrary.cs
+++ b/src/csharp/Grpc.Core/Internal/UnmanagedLibrary.cs
@@ -120,7 +120,7 @@ namespace Grpc.Core.Internal
{
throw new MissingMethodException(string.Format("The native method \"{0}\" does not exist", methodName));
}
-#if NETSTANDARD1_5
+#if NETSTANDARD1_5 || NETSTANDARD2_0
return Marshal.GetDelegateForFunctionPointer<T>(ptr); // non-generic version is obsolete
#else
return Marshal.GetDelegateForFunctionPointer(ptr, typeof(T)) as T; // generic version not available in .NET45
diff --git a/src/csharp/Grpc.Core/Utils/TaskUtils.cs b/src/csharp/Grpc.Core/Utils/TaskUtils.cs
index f25106f8dd8..21cd63336dc 100644
--- a/src/csharp/Grpc.Core/Utils/TaskUtils.cs
+++ b/src/csharp/Grpc.Core/Utils/TaskUtils.cs
@@ -33,7 +33,7 @@ namespace Grpc.Core.Utils
{
get
{
-#if NETSTANDARD1_5
+#if NETSTANDARD1_5 || NETSTANDARD2_0
return Task.CompletedTask;
#else
return Task.FromResult<object>(null); // for .NET45, emulate the functionality
From 91da9380534121ff6b90dc0f2f7543ca3e5dc63c Mon Sep 17 00:00:00 2001
From: Soheil Hassas Yeganeh <soheil@google.com>
Date: Wed, 13 Mar 2019 09:29:52 -0400
Subject: [PATCH 109/138] Make the TCP_INQ log a debug entry.
Users reported they see a lot of these logs in their runs.
---
src/core/lib/iomgr/tcp_posix.cc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/core/lib/iomgr/tcp_posix.cc b/src/core/lib/iomgr/tcp_posix.cc
index 960a45b7b26..30305a94f37 100644
--- a/src/core/lib/iomgr/tcp_posix.cc
+++ b/src/core/lib/iomgr/tcp_posix.cc
@@ -1255,7 +1255,7 @@ grpc_endpoint* grpc_tcp_create(grpc_fd* em_fd,
if (setsockopt(tcp->fd, SOL_TCP, TCP_INQ, &one, sizeof(one)) == 0) {
tcp->inq_capable = true;
} else {
- gpr_log(GPR_INFO, "cannot set inq fd=%d errno=%d", tcp->fd, errno);
+ gpr_log(GPR_DEBUG, "cannot set inq fd=%d errno=%d", tcp->fd, errno);
tcp->inq_capable = false;
}
#else
From 64b29fba5de6cd49183e9b486174623744c5890d Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Wed, 13 Mar 2019 09:58:54 -0700
Subject: [PATCH 110/138] An attempt to fix distrib test
---
test/distrib/python/test_packages.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/test/distrib/python/test_packages.sh b/test/distrib/python/test_packages.sh
index 4e1e6dbc94f..0ed6a2965e6 100755
--- a/test/distrib/python/test_packages.sh
+++ b/test/distrib/python/test_packages.sh
@@ -37,7 +37,7 @@ TESTING_ARCHIVES=("$EXTERNAL_GIT_ROOT"/input_artifacts/grpcio-testing-[0-9]*.tar
VIRTUAL_ENV=$(mktemp -d)
virtualenv "$VIRTUAL_ENV"
PYTHON=$VIRTUAL_ENV/bin/python
-"$PYTHON" -m pip install --upgrade six pip
+"$PYTHON" -m pip install --upgrade six pip wheel
function validate_wheel_hashes() {
for file in "$@"; do
From 33dbbb98d8728b334326b2e4ea5bdb2b43eefd70 Mon Sep 17 00:00:00 2001
From: John Luo <johluo@microsoft.com>
Date: Wed, 13 Mar 2019 11:13:48 -0700
Subject: [PATCH 111/138] Update tools to ensure error details appear in VS
---
src/csharp/Grpc.Tools/ProtoCompile.cs | 1 +
.../Grpc.Tools/build/_protobuf/Google.Protobuf.Tools.targets | 1 +
2 files changed, 2 insertions(+)
diff --git a/src/csharp/Grpc.Tools/ProtoCompile.cs b/src/csharp/Grpc.Tools/ProtoCompile.cs
index 93608e1ac02..f6964205d2b 100644
--- a/src/csharp/Grpc.Tools/ProtoCompile.cs
+++ b/src/csharp/Grpc.Tools/ProtoCompile.cs
@@ -322,6 +322,7 @@ namespace Grpc.Tools
{
cmd.AddArg(proto.ItemSpec);
}
+ cmd.AddSwitchMaybe("error_format", "msvs");
return cmd.ToString();
}
diff --git a/src/csharp/Grpc.Tools/build/_protobuf/Google.Protobuf.Tools.targets b/src/csharp/Grpc.Tools/build/_protobuf/Google.Protobuf.Tools.targets
index 26f9efb5a84..1fa6ca1eb36 100644
--- a/src/csharp/Grpc.Tools/build/_protobuf/Google.Protobuf.Tools.targets
+++ b/src/csharp/Grpc.Tools/build/_protobuf/Google.Protobuf.Tools.targets
@@ -271,6 +271,7 @@
GrpcPluginExe="%(_Protobuf_OutOfDateProto.GrpcPluginExe)"
GrpcOutputDir="%(_Protobuf_OutOfDateProto.GrpcOutputDir)"
GrpcOutputOptions="%(_Protobuf_OutOfDateProto._GrpcOutputOptions)"
+ LogStandardErrorAsError="true"
>
<Output TaskParameter="GeneratedFiles" ItemName="_Protobuf_GeneratedFiles"/>
</ProtoCompile>
From f42ad52c44f5d06b3418676611d5758e5061f3d5 Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Wed, 13 Mar 2019 12:40:10 -0700
Subject: [PATCH 112/138] Update `--dest-dir` to `-d`
---
test/distrib/python/test_packages.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/test/distrib/python/test_packages.sh b/test/distrib/python/test_packages.sh
index 0ed6a2965e6..e9f494310d6 100755
--- a/test/distrib/python/test_packages.sh
+++ b/test/distrib/python/test_packages.sh
@@ -41,7 +41,7 @@ PYTHON=$VIRTUAL_ENV/bin/python
function validate_wheel_hashes() {
for file in "$@"; do
- "$PYTHON" -m wheel unpack "$file" --dest-dir /tmp || return 1
+ "$PYTHON" -m wheel unpack "$file" -d /tmp || return 1
done
return 0
}
From 319fcdf26ed46fd244521552645d33e17e65ef88 Mon Sep 17 00:00:00 2001
From: Vishal Powar <vishal.powar@gmail.com>
Date: Fri, 1 Mar 2019 13:30:14 -0800
Subject: [PATCH 113/138] Add a script for generating C code and build rule for
protobuf protos.
All these changes need to go together to make sense
- changes to use new version of upb in bazel
- allowing includes in build target option
- script for generating c code (upb) for protos
- generated code for example protos
- adding changes for non-bazel builds
- change sanity tests to ignore the generated files.
---
BUILD | 25 +
CMakeLists.txt | 52 +
Makefile | 37 +-
bazel/grpc_build_system.bzl | 1 +
bazel/grpc_deps.bzl | 6 +-
build.yaml | 3 +
grpc.gyp | 18 +
.../upb-generated/google/protobuf/any.upb.c | 27 +
.../upb-generated/google/protobuf/any.upb.h | 59 +
.../google/protobuf/descriptor.upb.c | 485 +++++
.../google/protobuf/descriptor.upb.h | 1691 +++++++++++++++++
.../google/protobuf/duration.upb.c | 27 +
.../google/protobuf/duration.upb.h | 59 +
.../google/protobuf/struct.upb.c | 79 +
.../google/protobuf/struct.upb.h | 216 +++
.../google/protobuf/timestamp.upb.c | 27 +
.../google/protobuf/timestamp.upb.h | 59 +
.../google/protobuf/wrappers.upb.c | 106 ++
.../google/protobuf/wrappers.upb.h | 239 +++
src/upb/gen_build_yaml.py | 66 +
third_party/upb | 2 +-
tools/buildgen/generate_build_additions.sh | 1 +
tools/codegen/core/gen_upb_api.sh | 38 +
tools/distrib/check_copyright.py | 14 +
tools/distrib/check_include_guards.py | 14 +
.../clang_format_all_the_things.sh | 2 +-
.../generated/sources_and_headers.json | 20 +
tools/run_tests/sanity/check_port_platform.py | 3 +
tools/run_tests/sanity/check_submodules.sh | 2 +-
29 files changed, 3371 insertions(+), 7 deletions(-)
create mode 100644 src/core/ext/upb-generated/google/protobuf/any.upb.c
create mode 100644 src/core/ext/upb-generated/google/protobuf/any.upb.h
create mode 100644 src/core/ext/upb-generated/google/protobuf/descriptor.upb.c
create mode 100644 src/core/ext/upb-generated/google/protobuf/descriptor.upb.h
create mode 100644 src/core/ext/upb-generated/google/protobuf/duration.upb.c
create mode 100644 src/core/ext/upb-generated/google/protobuf/duration.upb.h
create mode 100644 src/core/ext/upb-generated/google/protobuf/struct.upb.c
create mode 100644 src/core/ext/upb-generated/google/protobuf/struct.upb.h
create mode 100644 src/core/ext/upb-generated/google/protobuf/timestamp.upb.c
create mode 100644 src/core/ext/upb-generated/google/protobuf/timestamp.upb.h
create mode 100644 src/core/ext/upb-generated/google/protobuf/wrappers.upb.c
create mode 100644 src/core/ext/upb-generated/google/protobuf/wrappers.upb.h
create mode 100755 src/upb/gen_build_yaml.py
create mode 100755 tools/codegen/core/gen_upb_api.sh
diff --git a/BUILD b/BUILD
index 4bf387cec3a..0b6fff354f4 100644
--- a/BUILD
+++ b/BUILD
@@ -2308,4 +2308,29 @@ grpc_cc_library(
],
)
+#TODO: Get this into build.yaml once we start using it.
+grpc_cc_library(
+ name = "google_protobuf_upb",
+ srcs = [
+ "src/core/ext/upb-generated/google/protobuf/any.upb.c",
+ "src/core/ext/upb-generated/google/protobuf/descriptor.upb.c",
+ "src/core/ext/upb-generated/google/protobuf/duration.upb.c",
+ "src/core/ext/upb-generated/google/protobuf/struct.upb.c",
+ "src/core/ext/upb-generated/google/protobuf/timestamp.upb.c",
+ "src/core/ext/upb-generated/google/protobuf/wrappers.upb.c",
+ ],
+ hdrs = [
+ "src/core/ext/upb-generated/google/protobuf/any.upb.h",
+ "src/core/ext/upb-generated/google/protobuf/descriptor.upb.h",
+ "src/core/ext/upb-generated/google/protobuf/duration.upb.h",
+ "src/core/ext/upb-generated/google/protobuf/struct.upb.h",
+ "src/core/ext/upb-generated/google/protobuf/timestamp.upb.h",
+ "src/core/ext/upb-generated/google/protobuf/wrappers.upb.h",
+ ],
+ language = "c++",
+ external_deps = [
+ "upb_lib",
+ ],
+)
+
grpc_generate_one_off_targets()
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 7ccda85b125..b7c770ccd24 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -5494,6 +5494,58 @@ endif()
endif (gRPC_BUILD_CSHARP_EXT)
if (gRPC_BUILD_TESTS)
+add_library(upb
+ third_party/upb/google/protobuf/descriptor.upb.c
+ third_party/upb/upb/decode.c
+ third_party/upb/upb/def.c
+ third_party/upb/upb/encode.c
+ third_party/upb/upb/handlers.c
+ third_party/upb/upb/msg.c
+ third_party/upb/upb/msgfactory.c
+ third_party/upb/upb/sink.c
+ third_party/upb/upb/table.c
+ third_party/upb/upb/upb.c
+)
+
+if(WIN32 AND MSVC)
+ set_target_properties(upb PROPERTIES COMPILE_PDB_NAME "upb"
+ COMPILE_PDB_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}"
+ )
+ if (gRPC_INSTALL)
+ install(FILES ${CMAKE_CURRENT_BINARY_DIR}/upb.pdb
+ DESTINATION ${gRPC_INSTALL_LIBDIR} OPTIONAL
+ )
+ endif()
+endif()
+
+
+target_include_directories(upb
+ PUBLIC $<INSTALL_INTERFACE:${gRPC_INSTALL_INCLUDEDIR}> $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
+ PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}
+ PRIVATE ${_gRPC_SSL_INCLUDE_DIR}
+ PRIVATE ${_gRPC_PROTOBUF_INCLUDE_DIR}
+ PRIVATE ${_gRPC_ZLIB_INCLUDE_DIR}
+ PRIVATE ${_gRPC_BENCHMARK_INCLUDE_DIR}
+ PRIVATE ${_gRPC_CARES_INCLUDE_DIR}
+ PRIVATE ${_gRPC_GFLAGS_INCLUDE_DIR}
+ PRIVATE ${_gRPC_ADDRESS_SORTING_INCLUDE_DIR}
+ PRIVATE ${_gRPC_NANOPB_INCLUDE_DIR}
+)
+ # avoid dependency on libstdc++
+ if (_gRPC_CORE_NOSTDCXX_FLAGS)
+ set_target_properties(upb PROPERTIES LINKER_LANGUAGE C)
+ # only use the flags for C++ source files
+ target_compile_options(upb PRIVATE $<$<COMPILE_LANGUAGE:CXX>:${_gRPC_CORE_NOSTDCXX_FLAGS}>)
+ endif()
+target_link_libraries(upb
+ ${_gRPC_SSL_LIBRARIES}
+ ${_gRPC_ALLTARGETS_LIBRARIES}
+)
+
+
+endif (gRPC_BUILD_TESTS)
+if (gRPC_BUILD_TESTS)
+
add_library(bad_client_test
test/core/bad_client/bad_client.cc
)
diff --git a/Makefile b/Makefile
index 91516de9a27..5a31d648b32 100644
--- a/Makefile
+++ b/Makefile
@@ -1443,7 +1443,7 @@ plugins: $(PROTOC_PLUGINS)
privatelibs: privatelibs_c privatelibs_cxx
-privatelibs_c: $(LIBDIR)/$(CONFIG)/libalts_test_util.a $(LIBDIR)/$(CONFIG)/libcxxabi.a $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc_test_util_unsecure.a $(LIBDIR)/$(CONFIG)/libreconnect_server.a $(LIBDIR)/$(CONFIG)/libtest_tcp_server.a $(LIBDIR)/$(CONFIG)/libz.a $(LIBDIR)/$(CONFIG)/libares.a $(LIBDIR)/$(CONFIG)/libbad_client_test.a $(LIBDIR)/$(CONFIG)/libbad_ssl_test_server.a $(LIBDIR)/$(CONFIG)/libend2end_tests.a $(LIBDIR)/$(CONFIG)/libend2end_nosec_tests.a
+privatelibs_c: $(LIBDIR)/$(CONFIG)/libalts_test_util.a $(LIBDIR)/$(CONFIG)/libcxxabi.a $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc_test_util_unsecure.a $(LIBDIR)/$(CONFIG)/libreconnect_server.a $(LIBDIR)/$(CONFIG)/libtest_tcp_server.a $(LIBDIR)/$(CONFIG)/libupb.a $(LIBDIR)/$(CONFIG)/libz.a $(LIBDIR)/$(CONFIG)/libares.a $(LIBDIR)/$(CONFIG)/libbad_client_test.a $(LIBDIR)/$(CONFIG)/libbad_ssl_test_server.a $(LIBDIR)/$(CONFIG)/libend2end_tests.a $(LIBDIR)/$(CONFIG)/libend2end_nosec_tests.a
pc_c: $(LIBDIR)/$(CONFIG)/pkgconfig/grpc.pc $(LIBDIR)/$(CONFIG)/pkgconfig/gpr.pc
pc_c_unsecure: $(LIBDIR)/$(CONFIG)/pkgconfig/grpc_unsecure.pc $(LIBDIR)/$(CONFIG)/pkgconfig/gpr.pc
@@ -10188,6 +10188,41 @@ ifneq ($(NO_DEPS),true)
endif
+LIBUPB_SRC = \
+ third_party/upb/google/protobuf/descriptor.upb.c \
+ third_party/upb/upb/decode.c \
+ third_party/upb/upb/def.c \
+ third_party/upb/upb/encode.c \
+ third_party/upb/upb/handlers.c \
+ third_party/upb/upb/msg.c \
+ third_party/upb/upb/msgfactory.c \
+ third_party/upb/upb/sink.c \
+ third_party/upb/upb/table.c \
+ third_party/upb/upb/upb.c \
+
+PUBLIC_HEADERS_C += \
+
+LIBUPB_OBJS = $(addprefix $(OBJDIR)/$(CONFIG)/, $(addsuffix .o, $(basename $(LIBUPB_SRC))))
+
+$(LIBUPB_OBJS): CFLAGS += -Ithird_party/upb -Wno-sign-conversion -Wno-shadow -Wno-conversion -Wno-implicit-fallthrough -Wno-sign-compare -Wno-missing-field-initializers
+
+$(LIBDIR)/$(CONFIG)/libupb.a: $(ZLIB_DEP) $(CARES_DEP) $(ADDRESS_SORTING_DEP) $(LIBUPB_OBJS)
+ $(E) "[AR] Creating $@"
+ $(Q) mkdir -p `dirname $@`
+ $(Q) rm -f $(LIBDIR)/$(CONFIG)/libupb.a
+ $(Q) $(AR) $(AROPTS) $(LIBDIR)/$(CONFIG)/libupb.a $(LIBUPB_OBJS)
+ifeq ($(SYSTEM),Darwin)
+ $(Q) ranlib -no_warning_for_no_symbols $(LIBDIR)/$(CONFIG)/libupb.a
+endif
+
+
+
+
+ifneq ($(NO_DEPS),true)
+-include $(LIBUPB_OBJS:.o=.dep)
+endif
+
+
LIBZ_SRC = \
third_party/zlib/adler32.c \
third_party/zlib/compress.c \
diff --git a/bazel/grpc_build_system.bzl b/bazel/grpc_build_system.bzl
index 513efac7509..59e9c46e0a3 100644
--- a/bazel/grpc_build_system.bzl
+++ b/bazel/grpc_build_system.bzl
@@ -120,6 +120,7 @@ def grpc_cc_library(
linkopts = linkopts,
includes = [
"include",
+ "src/core/ext/upb-generated",
],
alwayslink = alwayslink,
data = data,
diff --git a/bazel/grpc_deps.bzl b/bazel/grpc_deps.bzl
index d97e8368ed7..9b6aaacbd58 100644
--- a/bazel/grpc_deps.bzl
+++ b/bazel/grpc_deps.bzl
@@ -12,7 +12,7 @@ def grpc_deps():
)
native.bind(
- name = "upblib",
+ name = "upb_lib",
actual = "@upb//:upb",
)
@@ -202,8 +202,8 @@ def grpc_deps():
if "upb" not in native.existing_rules():
http_archive(
name = "upb",
- strip_prefix = "upb-9ce4a77f61c134bbed28bfd5be5cd7dc0e80f5e3",
- url = "https://github.com/google/upb/archive/9ce4a77f61c134bbed28bfd5be5cd7dc0e80f5e3.tar.gz",
+ strip_prefix = "upb-ed9faae0993704b033c594b072d65e1bf19207fa",
+ url = "https://github.com/google/upb/archive/ed9faae0993704b033c594b072d65e1bf19207fa.tar.gz",
)
# TODO: move some dependencies from "grpc_deps" here?
diff --git a/build.yaml b/build.yaml
index 02ecaa221e6..d8322b176b7 100644
--- a/build.yaml
+++ b/build.yaml
@@ -5837,6 +5837,9 @@ defaults:
-Wno-deprecated-declarations -Ithird_party/nanopb -DPB_FIELD_32BIT
CXXFLAGS: -Wnon-virtual-dtor
LDFLAGS: -g
+ upb:
+ CFLAGS: -Ithird_party/upb -Wno-sign-conversion -Wno-shadow -Wno-conversion -Wno-implicit-fallthrough
+ -Wno-sign-compare -Wno-missing-field-initializers
zlib:
CFLAGS: -Wno-sign-conversion -Wno-conversion -Wno-unused-value -Wno-implicit-function-declaration
-Wno-implicit-fallthrough $(W_NO_SHIFT_NEGATIVE_VALUE) -fvisibility=hidden
diff --git a/grpc.gyp b/grpc.gyp
index 94a9cd2cfb8..cce9f738fbd 100644
--- a/grpc.gyp
+++ b/grpc.gyp
@@ -2639,6 +2639,24 @@
'third_party/benchmark/src/timers.cc',
],
},
+ {
+ 'target_name': 'upb',
+ 'type': 'static_library',
+ 'dependencies': [
+ ],
+ 'sources': [
+ 'third_party/upb/google/protobuf/descriptor.upb.c',
+ 'third_party/upb/upb/decode.c',
+ 'third_party/upb/upb/def.c',
+ 'third_party/upb/upb/encode.c',
+ 'third_party/upb/upb/handlers.c',
+ 'third_party/upb/upb/msg.c',
+ 'third_party/upb/upb/msgfactory.c',
+ 'third_party/upb/upb/sink.c',
+ 'third_party/upb/upb/table.c',
+ 'third_party/upb/upb/upb.c',
+ ],
+ },
{
'target_name': 'z',
'type': 'static_library',
diff --git a/src/core/ext/upb-generated/google/protobuf/any.upb.c b/src/core/ext/upb-generated/google/protobuf/any.upb.c
new file mode 100644
index 00000000000..14badf797c1
--- /dev/null
+++ b/src/core/ext/upb-generated/google/protobuf/any.upb.c
@@ -0,0 +1,27 @@
+/* This file was generated by upbc (the upb compiler) from the input
+ * file:
+ *
+ * google/protobuf/any.proto
+ *
+ * Do not edit -- your changes will be discarded when the file is
+ * regenerated. */
+
+#include <stddef.h>
+#include "upb/msg.h"
+#include "google/protobuf/any.upb.h"
+
+#include "upb/port_def.inc"
+
+static const upb_msglayout_field google_protobuf_Any__fields[2] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 9, 1},
+ {2, UPB_SIZE(8, 16), 0, 0, 12, 1},
+};
+
+const upb_msglayout google_protobuf_Any_msginit = {
+ NULL,
+ &google_protobuf_Any__fields[0],
+ UPB_SIZE(16, 32), 2, false,
+};
+
+#include "upb/port_undef.inc"
+
diff --git a/src/core/ext/upb-generated/google/protobuf/any.upb.h b/src/core/ext/upb-generated/google/protobuf/any.upb.h
new file mode 100644
index 00000000000..386916c7ca8
--- /dev/null
+++ b/src/core/ext/upb-generated/google/protobuf/any.upb.h
@@ -0,0 +1,59 @@
+/* This file was generated by upbc (the upb compiler) from the input
+ * file:
+ *
+ * google/protobuf/any.proto
+ *
+ * Do not edit -- your changes will be discarded when the file is
+ * regenerated. */
+
+#ifndef GOOGLE_PROTOBUF_ANY_PROTO_UPB_H_
+#define GOOGLE_PROTOBUF_ANY_PROTO_UPB_H_
+
+#include "upb/generated_util.h"
+
+#include "upb/msg.h"
+
+#include "upb/decode.h"
+#include "upb/encode.h"
+#include "upb/port_def.inc"
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct google_protobuf_Any;
+typedef struct google_protobuf_Any google_protobuf_Any;
+extern const upb_msglayout google_protobuf_Any_msginit;
+
+/* Enums */
+
+/* google.protobuf.Any */
+
+UPB_INLINE google_protobuf_Any *google_protobuf_Any_new(upb_arena *arena) {
+ return (google_protobuf_Any *)upb_msg_new(&google_protobuf_Any_msginit, arena);
+}
+UPB_INLINE google_protobuf_Any *google_protobuf_Any_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_Any *ret = google_protobuf_Any_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_Any_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_Any_serialize(const google_protobuf_Any *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_Any_msginit, arena, len);
+}
+
+UPB_INLINE upb_strview google_protobuf_Any_type_url(const google_protobuf_Any *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(0, 0)); }
+UPB_INLINE upb_strview google_protobuf_Any_value(const google_protobuf_Any *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(8, 16)); }
+
+UPB_INLINE void google_protobuf_Any_set_type_url(google_protobuf_Any *msg, upb_strview value) {
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(0, 0)) = value;
+}
+UPB_INLINE void google_protobuf_Any_set_value(google_protobuf_Any *msg, upb_strview value) {
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(8, 16)) = value;
+}
+
+
+#ifdef __cplusplus
+} /* extern "C" */
+#endif
+
+#include "upb/port_undef.inc"
+
+#endif /* GOOGLE_PROTOBUF_ANY_PROTO_UPB_H_ */
diff --git a/src/core/ext/upb-generated/google/protobuf/descriptor.upb.c b/src/core/ext/upb-generated/google/protobuf/descriptor.upb.c
new file mode 100644
index 00000000000..61b9299bb43
--- /dev/null
+++ b/src/core/ext/upb-generated/google/protobuf/descriptor.upb.c
@@ -0,0 +1,485 @@
+/* This file was generated by upbc (the upb compiler) from the input
+ * file:
+ *
+ * google/protobuf/descriptor.proto
+ *
+ * Do not edit -- your changes will be discarded when the file is
+ * regenerated. */
+
+#include <stddef.h>
+#include "upb/msg.h"
+#include "google/protobuf/descriptor.upb.h"
+
+#include "upb/port_def.inc"
+
+static const upb_msglayout *const google_protobuf_FileDescriptorSet_submsgs[1] = {
+ &google_protobuf_FileDescriptorProto_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_FileDescriptorSet__fields[1] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 11, 3},
+};
+
+const upb_msglayout google_protobuf_FileDescriptorSet_msginit = {
+ &google_protobuf_FileDescriptorSet_submsgs[0],
+ &google_protobuf_FileDescriptorSet__fields[0],
+ UPB_SIZE(4, 8), 1, false,
+};
+
+static const upb_msglayout *const google_protobuf_FileDescriptorProto_submsgs[6] = {
+ &google_protobuf_DescriptorProto_msginit,
+ &google_protobuf_EnumDescriptorProto_msginit,
+ &google_protobuf_FieldDescriptorProto_msginit,
+ &google_protobuf_FileOptions_msginit,
+ &google_protobuf_ServiceDescriptorProto_msginit,
+ &google_protobuf_SourceCodeInfo_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_FileDescriptorProto__fields[12] = {
+ {1, UPB_SIZE(4, 8), 1, 0, 9, 1},
+ {2, UPB_SIZE(12, 24), 2, 0, 9, 1},
+ {3, UPB_SIZE(36, 72), 0, 0, 9, 3},
+ {4, UPB_SIZE(40, 80), 0, 0, 11, 3},
+ {5, UPB_SIZE(44, 88), 0, 1, 11, 3},
+ {6, UPB_SIZE(48, 96), 0, 4, 11, 3},
+ {7, UPB_SIZE(52, 104), 0, 2, 11, 3},
+ {8, UPB_SIZE(28, 56), 4, 3, 11, 1},
+ {9, UPB_SIZE(32, 64), 5, 5, 11, 1},
+ {10, UPB_SIZE(56, 112), 0, 0, 5, 3},
+ {11, UPB_SIZE(60, 120), 0, 0, 5, 3},
+ {12, UPB_SIZE(20, 40), 3, 0, 9, 1},
+};
+
+const upb_msglayout google_protobuf_FileDescriptorProto_msginit = {
+ &google_protobuf_FileDescriptorProto_submsgs[0],
+ &google_protobuf_FileDescriptorProto__fields[0],
+ UPB_SIZE(64, 128), 12, false,
+};
+
+static const upb_msglayout *const google_protobuf_DescriptorProto_submsgs[8] = {
+ &google_protobuf_DescriptorProto_msginit,
+ &google_protobuf_DescriptorProto_ExtensionRange_msginit,
+ &google_protobuf_DescriptorProto_ReservedRange_msginit,
+ &google_protobuf_EnumDescriptorProto_msginit,
+ &google_protobuf_FieldDescriptorProto_msginit,
+ &google_protobuf_MessageOptions_msginit,
+ &google_protobuf_OneofDescriptorProto_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_DescriptorProto__fields[10] = {
+ {1, UPB_SIZE(4, 8), 1, 0, 9, 1},
+ {2, UPB_SIZE(16, 32), 0, 4, 11, 3},
+ {3, UPB_SIZE(20, 40), 0, 0, 11, 3},
+ {4, UPB_SIZE(24, 48), 0, 3, 11, 3},
+ {5, UPB_SIZE(28, 56), 0, 1, 11, 3},
+ {6, UPB_SIZE(32, 64), 0, 4, 11, 3},
+ {7, UPB_SIZE(12, 24), 2, 5, 11, 1},
+ {8, UPB_SIZE(36, 72), 0, 6, 11, 3},
+ {9, UPB_SIZE(40, 80), 0, 2, 11, 3},
+ {10, UPB_SIZE(44, 88), 0, 0, 9, 3},
+};
+
+const upb_msglayout google_protobuf_DescriptorProto_msginit = {
+ &google_protobuf_DescriptorProto_submsgs[0],
+ &google_protobuf_DescriptorProto__fields[0],
+ UPB_SIZE(48, 96), 10, false,
+};
+
+static const upb_msglayout *const google_protobuf_DescriptorProto_ExtensionRange_submsgs[1] = {
+ &google_protobuf_ExtensionRangeOptions_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_DescriptorProto_ExtensionRange__fields[3] = {
+ {1, UPB_SIZE(4, 4), 1, 0, 5, 1},
+ {2, UPB_SIZE(8, 8), 2, 0, 5, 1},
+ {3, UPB_SIZE(12, 16), 3, 0, 11, 1},
+};
+
+const upb_msglayout google_protobuf_DescriptorProto_ExtensionRange_msginit = {
+ &google_protobuf_DescriptorProto_ExtensionRange_submsgs[0],
+ &google_protobuf_DescriptorProto_ExtensionRange__fields[0],
+ UPB_SIZE(16, 24), 3, false,
+};
+
+static const upb_msglayout_field google_protobuf_DescriptorProto_ReservedRange__fields[2] = {
+ {1, UPB_SIZE(4, 4), 1, 0, 5, 1},
+ {2, UPB_SIZE(8, 8), 2, 0, 5, 1},
+};
+
+const upb_msglayout google_protobuf_DescriptorProto_ReservedRange_msginit = {
+ NULL,
+ &google_protobuf_DescriptorProto_ReservedRange__fields[0],
+ UPB_SIZE(12, 12), 2, false,
+};
+
+static const upb_msglayout *const google_protobuf_ExtensionRangeOptions_submsgs[1] = {
+ &google_protobuf_UninterpretedOption_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_ExtensionRangeOptions__fields[1] = {
+ {999, UPB_SIZE(0, 0), 0, 0, 11, 3},
+};
+
+const upb_msglayout google_protobuf_ExtensionRangeOptions_msginit = {
+ &google_protobuf_ExtensionRangeOptions_submsgs[0],
+ &google_protobuf_ExtensionRangeOptions__fields[0],
+ UPB_SIZE(4, 8), 1, false,
+};
+
+static const upb_msglayout *const google_protobuf_FieldDescriptorProto_submsgs[1] = {
+ &google_protobuf_FieldOptions_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_FieldDescriptorProto__fields[10] = {
+ {1, UPB_SIZE(32, 32), 5, 0, 9, 1},
+ {2, UPB_SIZE(40, 48), 6, 0, 9, 1},
+ {3, UPB_SIZE(24, 24), 3, 0, 5, 1},
+ {4, UPB_SIZE(8, 8), 1, 0, 14, 1},
+ {5, UPB_SIZE(16, 16), 2, 0, 14, 1},
+ {6, UPB_SIZE(48, 64), 7, 0, 9, 1},
+ {7, UPB_SIZE(56, 80), 8, 0, 9, 1},
+ {8, UPB_SIZE(72, 112), 10, 0, 11, 1},
+ {9, UPB_SIZE(28, 28), 4, 0, 5, 1},
+ {10, UPB_SIZE(64, 96), 9, 0, 9, 1},
+};
+
+const upb_msglayout google_protobuf_FieldDescriptorProto_msginit = {
+ &google_protobuf_FieldDescriptorProto_submsgs[0],
+ &google_protobuf_FieldDescriptorProto__fields[0],
+ UPB_SIZE(80, 128), 10, false,
+};
+
+static const upb_msglayout *const google_protobuf_OneofDescriptorProto_submsgs[1] = {
+ &google_protobuf_OneofOptions_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_OneofDescriptorProto__fields[2] = {
+ {1, UPB_SIZE(4, 8), 1, 0, 9, 1},
+ {2, UPB_SIZE(12, 24), 2, 0, 11, 1},
+};
+
+const upb_msglayout google_protobuf_OneofDescriptorProto_msginit = {
+ &google_protobuf_OneofDescriptorProto_submsgs[0],
+ &google_protobuf_OneofDescriptorProto__fields[0],
+ UPB_SIZE(16, 32), 2, false,
+};
+
+static const upb_msglayout *const google_protobuf_EnumDescriptorProto_submsgs[3] = {
+ &google_protobuf_EnumDescriptorProto_EnumReservedRange_msginit,
+ &google_protobuf_EnumOptions_msginit,
+ &google_protobuf_EnumValueDescriptorProto_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_EnumDescriptorProto__fields[5] = {
+ {1, UPB_SIZE(4, 8), 1, 0, 9, 1},
+ {2, UPB_SIZE(16, 32), 0, 2, 11, 3},
+ {3, UPB_SIZE(12, 24), 2, 1, 11, 1},
+ {4, UPB_SIZE(20, 40), 0, 0, 11, 3},
+ {5, UPB_SIZE(24, 48), 0, 0, 9, 3},
+};
+
+const upb_msglayout google_protobuf_EnumDescriptorProto_msginit = {
+ &google_protobuf_EnumDescriptorProto_submsgs[0],
+ &google_protobuf_EnumDescriptorProto__fields[0],
+ UPB_SIZE(32, 64), 5, false,
+};
+
+static const upb_msglayout_field google_protobuf_EnumDescriptorProto_EnumReservedRange__fields[2] = {
+ {1, UPB_SIZE(4, 4), 1, 0, 5, 1},
+ {2, UPB_SIZE(8, 8), 2, 0, 5, 1},
+};
+
+const upb_msglayout google_protobuf_EnumDescriptorProto_EnumReservedRange_msginit = {
+ NULL,
+ &google_protobuf_EnumDescriptorProto_EnumReservedRange__fields[0],
+ UPB_SIZE(12, 12), 2, false,
+};
+
+static const upb_msglayout *const google_protobuf_EnumValueDescriptorProto_submsgs[1] = {
+ &google_protobuf_EnumValueOptions_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_EnumValueDescriptorProto__fields[3] = {
+ {1, UPB_SIZE(8, 8), 2, 0, 9, 1},
+ {2, UPB_SIZE(4, 4), 1, 0, 5, 1},
+ {3, UPB_SIZE(16, 24), 3, 0, 11, 1},
+};
+
+const upb_msglayout google_protobuf_EnumValueDescriptorProto_msginit = {
+ &google_protobuf_EnumValueDescriptorProto_submsgs[0],
+ &google_protobuf_EnumValueDescriptorProto__fields[0],
+ UPB_SIZE(24, 32), 3, false,
+};
+
+static const upb_msglayout *const google_protobuf_ServiceDescriptorProto_submsgs[2] = {
+ &google_protobuf_MethodDescriptorProto_msginit,
+ &google_protobuf_ServiceOptions_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_ServiceDescriptorProto__fields[3] = {
+ {1, UPB_SIZE(4, 8), 1, 0, 9, 1},
+ {2, UPB_SIZE(16, 32), 0, 0, 11, 3},
+ {3, UPB_SIZE(12, 24), 2, 1, 11, 1},
+};
+
+const upb_msglayout google_protobuf_ServiceDescriptorProto_msginit = {
+ &google_protobuf_ServiceDescriptorProto_submsgs[0],
+ &google_protobuf_ServiceDescriptorProto__fields[0],
+ UPB_SIZE(24, 48), 3, false,
+};
+
+static const upb_msglayout *const google_protobuf_MethodDescriptorProto_submsgs[1] = {
+ &google_protobuf_MethodOptions_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_MethodDescriptorProto__fields[6] = {
+ {1, UPB_SIZE(4, 8), 3, 0, 9, 1},
+ {2, UPB_SIZE(12, 24), 4, 0, 9, 1},
+ {3, UPB_SIZE(20, 40), 5, 0, 9, 1},
+ {4, UPB_SIZE(28, 56), 6, 0, 11, 1},
+ {5, UPB_SIZE(1, 1), 1, 0, 8, 1},
+ {6, UPB_SIZE(2, 2), 2, 0, 8, 1},
+};
+
+const upb_msglayout google_protobuf_MethodDescriptorProto_msginit = {
+ &google_protobuf_MethodDescriptorProto_submsgs[0],
+ &google_protobuf_MethodDescriptorProto__fields[0],
+ UPB_SIZE(32, 64), 6, false,
+};
+
+static const upb_msglayout *const google_protobuf_FileOptions_submsgs[1] = {
+ &google_protobuf_UninterpretedOption_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_FileOptions__fields[21] = {
+ {1, UPB_SIZE(28, 32), 11, 0, 9, 1},
+ {8, UPB_SIZE(36, 48), 12, 0, 9, 1},
+ {9, UPB_SIZE(8, 8), 1, 0, 14, 1},
+ {10, UPB_SIZE(16, 16), 2, 0, 8, 1},
+ {11, UPB_SIZE(44, 64), 13, 0, 9, 1},
+ {16, UPB_SIZE(17, 17), 3, 0, 8, 1},
+ {17, UPB_SIZE(18, 18), 4, 0, 8, 1},
+ {18, UPB_SIZE(19, 19), 5, 0, 8, 1},
+ {20, UPB_SIZE(20, 20), 6, 0, 8, 1},
+ {23, UPB_SIZE(21, 21), 7, 0, 8, 1},
+ {27, UPB_SIZE(22, 22), 8, 0, 8, 1},
+ {31, UPB_SIZE(23, 23), 9, 0, 8, 1},
+ {36, UPB_SIZE(52, 80), 14, 0, 9, 1},
+ {37, UPB_SIZE(60, 96), 15, 0, 9, 1},
+ {39, UPB_SIZE(68, 112), 16, 0, 9, 1},
+ {40, UPB_SIZE(76, 128), 17, 0, 9, 1},
+ {41, UPB_SIZE(84, 144), 18, 0, 9, 1},
+ {42, UPB_SIZE(24, 24), 10, 0, 8, 1},
+ {44, UPB_SIZE(92, 160), 19, 0, 9, 1},
+ {45, UPB_SIZE(100, 176), 20, 0, 9, 1},
+ {999, UPB_SIZE(108, 192), 0, 0, 11, 3},
+};
+
+const upb_msglayout google_protobuf_FileOptions_msginit = {
+ &google_protobuf_FileOptions_submsgs[0],
+ &google_protobuf_FileOptions__fields[0],
+ UPB_SIZE(112, 208), 21, false,
+};
+
+static const upb_msglayout *const google_protobuf_MessageOptions_submsgs[1] = {
+ &google_protobuf_UninterpretedOption_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_MessageOptions__fields[5] = {
+ {1, UPB_SIZE(1, 1), 1, 0, 8, 1},
+ {2, UPB_SIZE(2, 2), 2, 0, 8, 1},
+ {3, UPB_SIZE(3, 3), 3, 0, 8, 1},
+ {7, UPB_SIZE(4, 4), 4, 0, 8, 1},
+ {999, UPB_SIZE(8, 8), 0, 0, 11, 3},
+};
+
+const upb_msglayout google_protobuf_MessageOptions_msginit = {
+ &google_protobuf_MessageOptions_submsgs[0],
+ &google_protobuf_MessageOptions__fields[0],
+ UPB_SIZE(12, 16), 5, false,
+};
+
+static const upb_msglayout *const google_protobuf_FieldOptions_submsgs[1] = {
+ &google_protobuf_UninterpretedOption_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_FieldOptions__fields[7] = {
+ {1, UPB_SIZE(8, 8), 1, 0, 14, 1},
+ {2, UPB_SIZE(24, 24), 3, 0, 8, 1},
+ {3, UPB_SIZE(25, 25), 4, 0, 8, 1},
+ {5, UPB_SIZE(26, 26), 5, 0, 8, 1},
+ {6, UPB_SIZE(16, 16), 2, 0, 14, 1},
+ {10, UPB_SIZE(27, 27), 6, 0, 8, 1},
+ {999, UPB_SIZE(28, 32), 0, 0, 11, 3},
+};
+
+const upb_msglayout google_protobuf_FieldOptions_msginit = {
+ &google_protobuf_FieldOptions_submsgs[0],
+ &google_protobuf_FieldOptions__fields[0],
+ UPB_SIZE(32, 40), 7, false,
+};
+
+static const upb_msglayout *const google_protobuf_OneofOptions_submsgs[1] = {
+ &google_protobuf_UninterpretedOption_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_OneofOptions__fields[1] = {
+ {999, UPB_SIZE(0, 0), 0, 0, 11, 3},
+};
+
+const upb_msglayout google_protobuf_OneofOptions_msginit = {
+ &google_protobuf_OneofOptions_submsgs[0],
+ &google_protobuf_OneofOptions__fields[0],
+ UPB_SIZE(4, 8), 1, false,
+};
+
+static const upb_msglayout *const google_protobuf_EnumOptions_submsgs[1] = {
+ &google_protobuf_UninterpretedOption_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_EnumOptions__fields[3] = {
+ {2, UPB_SIZE(1, 1), 1, 0, 8, 1},
+ {3, UPB_SIZE(2, 2), 2, 0, 8, 1},
+ {999, UPB_SIZE(4, 8), 0, 0, 11, 3},
+};
+
+const upb_msglayout google_protobuf_EnumOptions_msginit = {
+ &google_protobuf_EnumOptions_submsgs[0],
+ &google_protobuf_EnumOptions__fields[0],
+ UPB_SIZE(8, 16), 3, false,
+};
+
+static const upb_msglayout *const google_protobuf_EnumValueOptions_submsgs[1] = {
+ &google_protobuf_UninterpretedOption_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_EnumValueOptions__fields[2] = {
+ {1, UPB_SIZE(1, 1), 1, 0, 8, 1},
+ {999, UPB_SIZE(4, 8), 0, 0, 11, 3},
+};
+
+const upb_msglayout google_protobuf_EnumValueOptions_msginit = {
+ &google_protobuf_EnumValueOptions_submsgs[0],
+ &google_protobuf_EnumValueOptions__fields[0],
+ UPB_SIZE(8, 16), 2, false,
+};
+
+static const upb_msglayout *const google_protobuf_ServiceOptions_submsgs[1] = {
+ &google_protobuf_UninterpretedOption_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_ServiceOptions__fields[2] = {
+ {33, UPB_SIZE(1, 1), 1, 0, 8, 1},
+ {999, UPB_SIZE(4, 8), 0, 0, 11, 3},
+};
+
+const upb_msglayout google_protobuf_ServiceOptions_msginit = {
+ &google_protobuf_ServiceOptions_submsgs[0],
+ &google_protobuf_ServiceOptions__fields[0],
+ UPB_SIZE(8, 16), 2, false,
+};
+
+static const upb_msglayout *const google_protobuf_MethodOptions_submsgs[1] = {
+ &google_protobuf_UninterpretedOption_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_MethodOptions__fields[3] = {
+ {33, UPB_SIZE(16, 16), 2, 0, 8, 1},
+ {34, UPB_SIZE(8, 8), 1, 0, 14, 1},
+ {999, UPB_SIZE(20, 24), 0, 0, 11, 3},
+};
+
+const upb_msglayout google_protobuf_MethodOptions_msginit = {
+ &google_protobuf_MethodOptions_submsgs[0],
+ &google_protobuf_MethodOptions__fields[0],
+ UPB_SIZE(24, 32), 3, false,
+};
+
+static const upb_msglayout *const google_protobuf_UninterpretedOption_submsgs[1] = {
+ &google_protobuf_UninterpretedOption_NamePart_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_UninterpretedOption__fields[7] = {
+ {2, UPB_SIZE(56, 80), 0, 0, 11, 3},
+ {3, UPB_SIZE(32, 32), 4, 0, 9, 1},
+ {4, UPB_SIZE(8, 8), 1, 0, 4, 1},
+ {5, UPB_SIZE(16, 16), 2, 0, 3, 1},
+ {6, UPB_SIZE(24, 24), 3, 0, 1, 1},
+ {7, UPB_SIZE(40, 48), 5, 0, 12, 1},
+ {8, UPB_SIZE(48, 64), 6, 0, 9, 1},
+};
+
+const upb_msglayout google_protobuf_UninterpretedOption_msginit = {
+ &google_protobuf_UninterpretedOption_submsgs[0],
+ &google_protobuf_UninterpretedOption__fields[0],
+ UPB_SIZE(64, 96), 7, false,
+};
+
+static const upb_msglayout_field google_protobuf_UninterpretedOption_NamePart__fields[2] = {
+ {1, UPB_SIZE(4, 8), 2, 0, 9, 2},
+ {2, UPB_SIZE(1, 1), 1, 0, 8, 2},
+};
+
+const upb_msglayout google_protobuf_UninterpretedOption_NamePart_msginit = {
+ NULL,
+ &google_protobuf_UninterpretedOption_NamePart__fields[0],
+ UPB_SIZE(16, 32), 2, false,
+};
+
+static const upb_msglayout *const google_protobuf_SourceCodeInfo_submsgs[1] = {
+ &google_protobuf_SourceCodeInfo_Location_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_SourceCodeInfo__fields[1] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 11, 3},
+};
+
+const upb_msglayout google_protobuf_SourceCodeInfo_msginit = {
+ &google_protobuf_SourceCodeInfo_submsgs[0],
+ &google_protobuf_SourceCodeInfo__fields[0],
+ UPB_SIZE(4, 8), 1, false,
+};
+
+static const upb_msglayout_field google_protobuf_SourceCodeInfo_Location__fields[5] = {
+ {1, UPB_SIZE(20, 40), 0, 0, 5, 3},
+ {2, UPB_SIZE(24, 48), 0, 0, 5, 3},
+ {3, UPB_SIZE(4, 8), 1, 0, 9, 1},
+ {4, UPB_SIZE(12, 24), 2, 0, 9, 1},
+ {6, UPB_SIZE(28, 56), 0, 0, 9, 3},
+};
+
+const upb_msglayout google_protobuf_SourceCodeInfo_Location_msginit = {
+ NULL,
+ &google_protobuf_SourceCodeInfo_Location__fields[0],
+ UPB_SIZE(32, 64), 5, false,
+};
+
+static const upb_msglayout *const google_protobuf_GeneratedCodeInfo_submsgs[1] = {
+ &google_protobuf_GeneratedCodeInfo_Annotation_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_GeneratedCodeInfo__fields[1] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 11, 3},
+};
+
+const upb_msglayout google_protobuf_GeneratedCodeInfo_msginit = {
+ &google_protobuf_GeneratedCodeInfo_submsgs[0],
+ &google_protobuf_GeneratedCodeInfo__fields[0],
+ UPB_SIZE(4, 8), 1, false,
+};
+
+static const upb_msglayout_field google_protobuf_GeneratedCodeInfo_Annotation__fields[4] = {
+ {1, UPB_SIZE(20, 32), 0, 0, 5, 3},
+ {2, UPB_SIZE(12, 16), 3, 0, 9, 1},
+ {3, UPB_SIZE(4, 4), 1, 0, 5, 1},
+ {4, UPB_SIZE(8, 8), 2, 0, 5, 1},
+};
+
+const upb_msglayout google_protobuf_GeneratedCodeInfo_Annotation_msginit = {
+ NULL,
+ &google_protobuf_GeneratedCodeInfo_Annotation__fields[0],
+ UPB_SIZE(24, 48), 4, false,
+};
+
+#include "upb/port_undef.inc"
+
diff --git a/src/core/ext/upb-generated/google/protobuf/descriptor.upb.h b/src/core/ext/upb-generated/google/protobuf/descriptor.upb.h
new file mode 100644
index 00000000000..89e24a6c976
--- /dev/null
+++ b/src/core/ext/upb-generated/google/protobuf/descriptor.upb.h
@@ -0,0 +1,1691 @@
+/* This file was generated by upbc (the upb compiler) from the input
+ * file:
+ *
+ * google/protobuf/descriptor.proto
+ *
+ * Do not edit -- your changes will be discarded when the file is
+ * regenerated. */
+
+#ifndef GOOGLE_PROTOBUF_DESCRIPTOR_PROTO_UPB_H_
+#define GOOGLE_PROTOBUF_DESCRIPTOR_PROTO_UPB_H_
+
+#include "upb/generated_util.h"
+
+#include "upb/msg.h"
+
+#include "upb/decode.h"
+#include "upb/encode.h"
+#include "upb/port_def.inc"
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct google_protobuf_FileDescriptorSet;
+struct google_protobuf_FileDescriptorProto;
+struct google_protobuf_DescriptorProto;
+struct google_protobuf_DescriptorProto_ExtensionRange;
+struct google_protobuf_DescriptorProto_ReservedRange;
+struct google_protobuf_ExtensionRangeOptions;
+struct google_protobuf_FieldDescriptorProto;
+struct google_protobuf_OneofDescriptorProto;
+struct google_protobuf_EnumDescriptorProto;
+struct google_protobuf_EnumDescriptorProto_EnumReservedRange;
+struct google_protobuf_EnumValueDescriptorProto;
+struct google_protobuf_ServiceDescriptorProto;
+struct google_protobuf_MethodDescriptorProto;
+struct google_protobuf_FileOptions;
+struct google_protobuf_MessageOptions;
+struct google_protobuf_FieldOptions;
+struct google_protobuf_OneofOptions;
+struct google_protobuf_EnumOptions;
+struct google_protobuf_EnumValueOptions;
+struct google_protobuf_ServiceOptions;
+struct google_protobuf_MethodOptions;
+struct google_protobuf_UninterpretedOption;
+struct google_protobuf_UninterpretedOption_NamePart;
+struct google_protobuf_SourceCodeInfo;
+struct google_protobuf_SourceCodeInfo_Location;
+struct google_protobuf_GeneratedCodeInfo;
+struct google_protobuf_GeneratedCodeInfo_Annotation;
+typedef struct google_protobuf_FileDescriptorSet google_protobuf_FileDescriptorSet;
+typedef struct google_protobuf_FileDescriptorProto google_protobuf_FileDescriptorProto;
+typedef struct google_protobuf_DescriptorProto google_protobuf_DescriptorProto;
+typedef struct google_protobuf_DescriptorProto_ExtensionRange google_protobuf_DescriptorProto_ExtensionRange;
+typedef struct google_protobuf_DescriptorProto_ReservedRange google_protobuf_DescriptorProto_ReservedRange;
+typedef struct google_protobuf_ExtensionRangeOptions google_protobuf_ExtensionRangeOptions;
+typedef struct google_protobuf_FieldDescriptorProto google_protobuf_FieldDescriptorProto;
+typedef struct google_protobuf_OneofDescriptorProto google_protobuf_OneofDescriptorProto;
+typedef struct google_protobuf_EnumDescriptorProto google_protobuf_EnumDescriptorProto;
+typedef struct google_protobuf_EnumDescriptorProto_EnumReservedRange google_protobuf_EnumDescriptorProto_EnumReservedRange;
+typedef struct google_protobuf_EnumValueDescriptorProto google_protobuf_EnumValueDescriptorProto;
+typedef struct google_protobuf_ServiceDescriptorProto google_protobuf_ServiceDescriptorProto;
+typedef struct google_protobuf_MethodDescriptorProto google_protobuf_MethodDescriptorProto;
+typedef struct google_protobuf_FileOptions google_protobuf_FileOptions;
+typedef struct google_protobuf_MessageOptions google_protobuf_MessageOptions;
+typedef struct google_protobuf_FieldOptions google_protobuf_FieldOptions;
+typedef struct google_protobuf_OneofOptions google_protobuf_OneofOptions;
+typedef struct google_protobuf_EnumOptions google_protobuf_EnumOptions;
+typedef struct google_protobuf_EnumValueOptions google_protobuf_EnumValueOptions;
+typedef struct google_protobuf_ServiceOptions google_protobuf_ServiceOptions;
+typedef struct google_protobuf_MethodOptions google_protobuf_MethodOptions;
+typedef struct google_protobuf_UninterpretedOption google_protobuf_UninterpretedOption;
+typedef struct google_protobuf_UninterpretedOption_NamePart google_protobuf_UninterpretedOption_NamePart;
+typedef struct google_protobuf_SourceCodeInfo google_protobuf_SourceCodeInfo;
+typedef struct google_protobuf_SourceCodeInfo_Location google_protobuf_SourceCodeInfo_Location;
+typedef struct google_protobuf_GeneratedCodeInfo google_protobuf_GeneratedCodeInfo;
+typedef struct google_protobuf_GeneratedCodeInfo_Annotation google_protobuf_GeneratedCodeInfo_Annotation;
+extern const upb_msglayout google_protobuf_FileDescriptorSet_msginit;
+extern const upb_msglayout google_protobuf_FileDescriptorProto_msginit;
+extern const upb_msglayout google_protobuf_DescriptorProto_msginit;
+extern const upb_msglayout google_protobuf_DescriptorProto_ExtensionRange_msginit;
+extern const upb_msglayout google_protobuf_DescriptorProto_ReservedRange_msginit;
+extern const upb_msglayout google_protobuf_ExtensionRangeOptions_msginit;
+extern const upb_msglayout google_protobuf_FieldDescriptorProto_msginit;
+extern const upb_msglayout google_protobuf_OneofDescriptorProto_msginit;
+extern const upb_msglayout google_protobuf_EnumDescriptorProto_msginit;
+extern const upb_msglayout google_protobuf_EnumDescriptorProto_EnumReservedRange_msginit;
+extern const upb_msglayout google_protobuf_EnumValueDescriptorProto_msginit;
+extern const upb_msglayout google_protobuf_ServiceDescriptorProto_msginit;
+extern const upb_msglayout google_protobuf_MethodDescriptorProto_msginit;
+extern const upb_msglayout google_protobuf_FileOptions_msginit;
+extern const upb_msglayout google_protobuf_MessageOptions_msginit;
+extern const upb_msglayout google_protobuf_FieldOptions_msginit;
+extern const upb_msglayout google_protobuf_OneofOptions_msginit;
+extern const upb_msglayout google_protobuf_EnumOptions_msginit;
+extern const upb_msglayout google_protobuf_EnumValueOptions_msginit;
+extern const upb_msglayout google_protobuf_ServiceOptions_msginit;
+extern const upb_msglayout google_protobuf_MethodOptions_msginit;
+extern const upb_msglayout google_protobuf_UninterpretedOption_msginit;
+extern const upb_msglayout google_protobuf_UninterpretedOption_NamePart_msginit;
+extern const upb_msglayout google_protobuf_SourceCodeInfo_msginit;
+extern const upb_msglayout google_protobuf_SourceCodeInfo_Location_msginit;
+extern const upb_msglayout google_protobuf_GeneratedCodeInfo_msginit;
+extern const upb_msglayout google_protobuf_GeneratedCodeInfo_Annotation_msginit;
+
+/* Enums */
+
+typedef enum {
+ google_protobuf_FieldDescriptorProto_LABEL_OPTIONAL = 1,
+ google_protobuf_FieldDescriptorProto_LABEL_REQUIRED = 2,
+ google_protobuf_FieldDescriptorProto_LABEL_REPEATED = 3
+} google_protobuf_FieldDescriptorProto_Label;
+
+typedef enum {
+ google_protobuf_FieldDescriptorProto_TYPE_DOUBLE = 1,
+ google_protobuf_FieldDescriptorProto_TYPE_FLOAT = 2,
+ google_protobuf_FieldDescriptorProto_TYPE_INT64 = 3,
+ google_protobuf_FieldDescriptorProto_TYPE_UINT64 = 4,
+ google_protobuf_FieldDescriptorProto_TYPE_INT32 = 5,
+ google_protobuf_FieldDescriptorProto_TYPE_FIXED64 = 6,
+ google_protobuf_FieldDescriptorProto_TYPE_FIXED32 = 7,
+ google_protobuf_FieldDescriptorProto_TYPE_BOOL = 8,
+ google_protobuf_FieldDescriptorProto_TYPE_STRING = 9,
+ google_protobuf_FieldDescriptorProto_TYPE_GROUP = 10,
+ google_protobuf_FieldDescriptorProto_TYPE_MESSAGE = 11,
+ google_protobuf_FieldDescriptorProto_TYPE_BYTES = 12,
+ google_protobuf_FieldDescriptorProto_TYPE_UINT32 = 13,
+ google_protobuf_FieldDescriptorProto_TYPE_ENUM = 14,
+ google_protobuf_FieldDescriptorProto_TYPE_SFIXED32 = 15,
+ google_protobuf_FieldDescriptorProto_TYPE_SFIXED64 = 16,
+ google_protobuf_FieldDescriptorProto_TYPE_SINT32 = 17,
+ google_protobuf_FieldDescriptorProto_TYPE_SINT64 = 18
+} google_protobuf_FieldDescriptorProto_Type;
+
+typedef enum {
+ google_protobuf_FieldOptions_STRING = 0,
+ google_protobuf_FieldOptions_CORD = 1,
+ google_protobuf_FieldOptions_STRING_PIECE = 2
+} google_protobuf_FieldOptions_CType;
+
+typedef enum {
+ google_protobuf_FieldOptions_JS_NORMAL = 0,
+ google_protobuf_FieldOptions_JS_STRING = 1,
+ google_protobuf_FieldOptions_JS_NUMBER = 2
+} google_protobuf_FieldOptions_JSType;
+
+typedef enum {
+ google_protobuf_FileOptions_SPEED = 1,
+ google_protobuf_FileOptions_CODE_SIZE = 2,
+ google_protobuf_FileOptions_LITE_RUNTIME = 3
+} google_protobuf_FileOptions_OptimizeMode;
+
+typedef enum {
+ google_protobuf_MethodOptions_IDEMPOTENCY_UNKNOWN = 0,
+ google_protobuf_MethodOptions_NO_SIDE_EFFECTS = 1,
+ google_protobuf_MethodOptions_IDEMPOTENT = 2
+} google_protobuf_MethodOptions_IdempotencyLevel;
+
+/* google.protobuf.FileDescriptorSet */
+
+UPB_INLINE google_protobuf_FileDescriptorSet *google_protobuf_FileDescriptorSet_new(upb_arena *arena) {
+ return (google_protobuf_FileDescriptorSet *)upb_msg_new(&google_protobuf_FileDescriptorSet_msginit, arena);
+}
+UPB_INLINE google_protobuf_FileDescriptorSet *google_protobuf_FileDescriptorSet_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_FileDescriptorSet *ret = google_protobuf_FileDescriptorSet_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_FileDescriptorSet_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_FileDescriptorSet_serialize(const google_protobuf_FileDescriptorSet *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_FileDescriptorSet_msginit, arena, len);
+}
+
+UPB_INLINE const google_protobuf_FileDescriptorProto* const* google_protobuf_FileDescriptorSet_file(const google_protobuf_FileDescriptorSet *msg, size_t *len) { return (const google_protobuf_FileDescriptorProto* const*)_upb_array_accessor(msg, UPB_SIZE(0, 0), len); }
+
+UPB_INLINE google_protobuf_FileDescriptorProto** google_protobuf_FileDescriptorSet_mutable_file(google_protobuf_FileDescriptorSet *msg, size_t *len) {
+ return (google_protobuf_FileDescriptorProto**)_upb_array_mutable_accessor(msg, UPB_SIZE(0, 0), len);
+}
+UPB_INLINE google_protobuf_FileDescriptorProto** google_protobuf_FileDescriptorSet_resize_file(google_protobuf_FileDescriptorSet *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_FileDescriptorProto**)_upb_array_resize_accessor(msg, UPB_SIZE(0, 0), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_FileDescriptorProto* google_protobuf_FileDescriptorSet_add_file(google_protobuf_FileDescriptorSet *msg, upb_arena *arena) {
+ struct google_protobuf_FileDescriptorProto* sub = (struct google_protobuf_FileDescriptorProto*)upb_msg_new(&google_protobuf_FileDescriptorProto_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(0, 0), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+
+
+/* google.protobuf.FileDescriptorProto */
+
+UPB_INLINE google_protobuf_FileDescriptorProto *google_protobuf_FileDescriptorProto_new(upb_arena *arena) {
+ return (google_protobuf_FileDescriptorProto *)upb_msg_new(&google_protobuf_FileDescriptorProto_msginit, arena);
+}
+UPB_INLINE google_protobuf_FileDescriptorProto *google_protobuf_FileDescriptorProto_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_FileDescriptorProto *ret = google_protobuf_FileDescriptorProto_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_FileDescriptorProto_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_FileDescriptorProto_serialize(const google_protobuf_FileDescriptorProto *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_FileDescriptorProto_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_FileDescriptorProto_has_name(const google_protobuf_FileDescriptorProto *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE upb_strview google_protobuf_FileDescriptorProto_name(const google_protobuf_FileDescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)); }
+UPB_INLINE bool google_protobuf_FileDescriptorProto_has_package(const google_protobuf_FileDescriptorProto *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE upb_strview google_protobuf_FileDescriptorProto_package(const google_protobuf_FileDescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(12, 24)); }
+UPB_INLINE upb_strview const* google_protobuf_FileDescriptorProto_dependency(const google_protobuf_FileDescriptorProto *msg, size_t *len) { return (upb_strview const*)_upb_array_accessor(msg, UPB_SIZE(36, 72), len); }
+UPB_INLINE const google_protobuf_DescriptorProto* const* google_protobuf_FileDescriptorProto_message_type(const google_protobuf_FileDescriptorProto *msg, size_t *len) { return (const google_protobuf_DescriptorProto* const*)_upb_array_accessor(msg, UPB_SIZE(40, 80), len); }
+UPB_INLINE const google_protobuf_EnumDescriptorProto* const* google_protobuf_FileDescriptorProto_enum_type(const google_protobuf_FileDescriptorProto *msg, size_t *len) { return (const google_protobuf_EnumDescriptorProto* const*)_upb_array_accessor(msg, UPB_SIZE(44, 88), len); }
+UPB_INLINE const google_protobuf_ServiceDescriptorProto* const* google_protobuf_FileDescriptorProto_service(const google_protobuf_FileDescriptorProto *msg, size_t *len) { return (const google_protobuf_ServiceDescriptorProto* const*)_upb_array_accessor(msg, UPB_SIZE(48, 96), len); }
+UPB_INLINE const google_protobuf_FieldDescriptorProto* const* google_protobuf_FileDescriptorProto_extension(const google_protobuf_FileDescriptorProto *msg, size_t *len) { return (const google_protobuf_FieldDescriptorProto* const*)_upb_array_accessor(msg, UPB_SIZE(52, 104), len); }
+UPB_INLINE bool google_protobuf_FileDescriptorProto_has_options(const google_protobuf_FileDescriptorProto *msg) { return _upb_has_field(msg, 4); }
+UPB_INLINE const google_protobuf_FileOptions* google_protobuf_FileDescriptorProto_options(const google_protobuf_FileDescriptorProto *msg) { return UPB_FIELD_AT(msg, const google_protobuf_FileOptions*, UPB_SIZE(28, 56)); }
+UPB_INLINE bool google_protobuf_FileDescriptorProto_has_source_code_info(const google_protobuf_FileDescriptorProto *msg) { return _upb_has_field(msg, 5); }
+UPB_INLINE const google_protobuf_SourceCodeInfo* google_protobuf_FileDescriptorProto_source_code_info(const google_protobuf_FileDescriptorProto *msg) { return UPB_FIELD_AT(msg, const google_protobuf_SourceCodeInfo*, UPB_SIZE(32, 64)); }
+UPB_INLINE int32_t const* google_protobuf_FileDescriptorProto_public_dependency(const google_protobuf_FileDescriptorProto *msg, size_t *len) { return (int32_t const*)_upb_array_accessor(msg, UPB_SIZE(56, 112), len); }
+UPB_INLINE int32_t const* google_protobuf_FileDescriptorProto_weak_dependency(const google_protobuf_FileDescriptorProto *msg, size_t *len) { return (int32_t const*)_upb_array_accessor(msg, UPB_SIZE(60, 120), len); }
+UPB_INLINE bool google_protobuf_FileDescriptorProto_has_syntax(const google_protobuf_FileDescriptorProto *msg) { return _upb_has_field(msg, 3); }
+UPB_INLINE upb_strview google_protobuf_FileDescriptorProto_syntax(const google_protobuf_FileDescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(20, 40)); }
+
+UPB_INLINE void google_protobuf_FileDescriptorProto_set_name(google_protobuf_FileDescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)) = value;
+}
+UPB_INLINE void google_protobuf_FileDescriptorProto_set_package(google_protobuf_FileDescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(12, 24)) = value;
+}
+UPB_INLINE upb_strview* google_protobuf_FileDescriptorProto_mutable_dependency(google_protobuf_FileDescriptorProto *msg, size_t *len) {
+ return (upb_strview*)_upb_array_mutable_accessor(msg, UPB_SIZE(36, 72), len);
+}
+UPB_INLINE upb_strview* google_protobuf_FileDescriptorProto_resize_dependency(google_protobuf_FileDescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (upb_strview*)_upb_array_resize_accessor(msg, UPB_SIZE(36, 72), len, UPB_SIZE(8, 16), UPB_TYPE_STRING, arena);
+}
+UPB_INLINE bool google_protobuf_FileDescriptorProto_add_dependency(google_protobuf_FileDescriptorProto *msg, upb_strview val, upb_arena *arena) {
+ return _upb_array_append_accessor(
+ msg, UPB_SIZE(36, 72), UPB_SIZE(8, 16), UPB_TYPE_STRING, &val, arena);
+}
+UPB_INLINE google_protobuf_DescriptorProto** google_protobuf_FileDescriptorProto_mutable_message_type(google_protobuf_FileDescriptorProto *msg, size_t *len) {
+ return (google_protobuf_DescriptorProto**)_upb_array_mutable_accessor(msg, UPB_SIZE(40, 80), len);
+}
+UPB_INLINE google_protobuf_DescriptorProto** google_protobuf_FileDescriptorProto_resize_message_type(google_protobuf_FileDescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_DescriptorProto**)_upb_array_resize_accessor(msg, UPB_SIZE(40, 80), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_DescriptorProto* google_protobuf_FileDescriptorProto_add_message_type(google_protobuf_FileDescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_DescriptorProto* sub = (struct google_protobuf_DescriptorProto*)upb_msg_new(&google_protobuf_DescriptorProto_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(40, 80), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+UPB_INLINE google_protobuf_EnumDescriptorProto** google_protobuf_FileDescriptorProto_mutable_enum_type(google_protobuf_FileDescriptorProto *msg, size_t *len) {
+ return (google_protobuf_EnumDescriptorProto**)_upb_array_mutable_accessor(msg, UPB_SIZE(44, 88), len);
+}
+UPB_INLINE google_protobuf_EnumDescriptorProto** google_protobuf_FileDescriptorProto_resize_enum_type(google_protobuf_FileDescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_EnumDescriptorProto**)_upb_array_resize_accessor(msg, UPB_SIZE(44, 88), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_EnumDescriptorProto* google_protobuf_FileDescriptorProto_add_enum_type(google_protobuf_FileDescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_EnumDescriptorProto* sub = (struct google_protobuf_EnumDescriptorProto*)upb_msg_new(&google_protobuf_EnumDescriptorProto_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(44, 88), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+UPB_INLINE google_protobuf_ServiceDescriptorProto** google_protobuf_FileDescriptorProto_mutable_service(google_protobuf_FileDescriptorProto *msg, size_t *len) {
+ return (google_protobuf_ServiceDescriptorProto**)_upb_array_mutable_accessor(msg, UPB_SIZE(48, 96), len);
+}
+UPB_INLINE google_protobuf_ServiceDescriptorProto** google_protobuf_FileDescriptorProto_resize_service(google_protobuf_FileDescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_ServiceDescriptorProto**)_upb_array_resize_accessor(msg, UPB_SIZE(48, 96), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_ServiceDescriptorProto* google_protobuf_FileDescriptorProto_add_service(google_protobuf_FileDescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_ServiceDescriptorProto* sub = (struct google_protobuf_ServiceDescriptorProto*)upb_msg_new(&google_protobuf_ServiceDescriptorProto_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(48, 96), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+UPB_INLINE google_protobuf_FieldDescriptorProto** google_protobuf_FileDescriptorProto_mutable_extension(google_protobuf_FileDescriptorProto *msg, size_t *len) {
+ return (google_protobuf_FieldDescriptorProto**)_upb_array_mutable_accessor(msg, UPB_SIZE(52, 104), len);
+}
+UPB_INLINE google_protobuf_FieldDescriptorProto** google_protobuf_FileDescriptorProto_resize_extension(google_protobuf_FileDescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_FieldDescriptorProto**)_upb_array_resize_accessor(msg, UPB_SIZE(52, 104), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_FieldDescriptorProto* google_protobuf_FileDescriptorProto_add_extension(google_protobuf_FileDescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_FieldDescriptorProto* sub = (struct google_protobuf_FieldDescriptorProto*)upb_msg_new(&google_protobuf_FieldDescriptorProto_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(52, 104), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+UPB_INLINE void google_protobuf_FileDescriptorProto_set_options(google_protobuf_FileDescriptorProto *msg, google_protobuf_FileOptions* value) {
+ _upb_sethas(msg, 4);
+ UPB_FIELD_AT(msg, google_protobuf_FileOptions*, UPB_SIZE(28, 56)) = value;
+}
+UPB_INLINE struct google_protobuf_FileOptions* google_protobuf_FileDescriptorProto_mutable_options(google_protobuf_FileDescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_FileOptions* sub = (struct google_protobuf_FileOptions*)google_protobuf_FileDescriptorProto_options(msg);
+ if (sub == NULL) {
+ sub = (struct google_protobuf_FileOptions*)upb_msg_new(&google_protobuf_FileOptions_msginit, arena);
+ if (!sub) return NULL;
+ google_protobuf_FileDescriptorProto_set_options(msg, sub);
+ }
+ return sub;
+}
+UPB_INLINE void google_protobuf_FileDescriptorProto_set_source_code_info(google_protobuf_FileDescriptorProto *msg, google_protobuf_SourceCodeInfo* value) {
+ _upb_sethas(msg, 5);
+ UPB_FIELD_AT(msg, google_protobuf_SourceCodeInfo*, UPB_SIZE(32, 64)) = value;
+}
+UPB_INLINE struct google_protobuf_SourceCodeInfo* google_protobuf_FileDescriptorProto_mutable_source_code_info(google_protobuf_FileDescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_SourceCodeInfo* sub = (struct google_protobuf_SourceCodeInfo*)google_protobuf_FileDescriptorProto_source_code_info(msg);
+ if (sub == NULL) {
+ sub = (struct google_protobuf_SourceCodeInfo*)upb_msg_new(&google_protobuf_SourceCodeInfo_msginit, arena);
+ if (!sub) return NULL;
+ google_protobuf_FileDescriptorProto_set_source_code_info(msg, sub);
+ }
+ return sub;
+}
+UPB_INLINE int32_t* google_protobuf_FileDescriptorProto_mutable_public_dependency(google_protobuf_FileDescriptorProto *msg, size_t *len) {
+ return (int32_t*)_upb_array_mutable_accessor(msg, UPB_SIZE(56, 112), len);
+}
+UPB_INLINE int32_t* google_protobuf_FileDescriptorProto_resize_public_dependency(google_protobuf_FileDescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (int32_t*)_upb_array_resize_accessor(msg, UPB_SIZE(56, 112), len, UPB_SIZE(4, 4), UPB_TYPE_INT32, arena);
+}
+UPB_INLINE bool google_protobuf_FileDescriptorProto_add_public_dependency(google_protobuf_FileDescriptorProto *msg, int32_t val, upb_arena *arena) {
+ return _upb_array_append_accessor(
+ msg, UPB_SIZE(56, 112), UPB_SIZE(4, 4), UPB_TYPE_INT32, &val, arena);
+}
+UPB_INLINE int32_t* google_protobuf_FileDescriptorProto_mutable_weak_dependency(google_protobuf_FileDescriptorProto *msg, size_t *len) {
+ return (int32_t*)_upb_array_mutable_accessor(msg, UPB_SIZE(60, 120), len);
+}
+UPB_INLINE int32_t* google_protobuf_FileDescriptorProto_resize_weak_dependency(google_protobuf_FileDescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (int32_t*)_upb_array_resize_accessor(msg, UPB_SIZE(60, 120), len, UPB_SIZE(4, 4), UPB_TYPE_INT32, arena);
+}
+UPB_INLINE bool google_protobuf_FileDescriptorProto_add_weak_dependency(google_protobuf_FileDescriptorProto *msg, int32_t val, upb_arena *arena) {
+ return _upb_array_append_accessor(
+ msg, UPB_SIZE(60, 120), UPB_SIZE(4, 4), UPB_TYPE_INT32, &val, arena);
+}
+UPB_INLINE void google_protobuf_FileDescriptorProto_set_syntax(google_protobuf_FileDescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 3);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(20, 40)) = value;
+}
+
+
+/* google.protobuf.DescriptorProto */
+
+UPB_INLINE google_protobuf_DescriptorProto *google_protobuf_DescriptorProto_new(upb_arena *arena) {
+ return (google_protobuf_DescriptorProto *)upb_msg_new(&google_protobuf_DescriptorProto_msginit, arena);
+}
+UPB_INLINE google_protobuf_DescriptorProto *google_protobuf_DescriptorProto_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_DescriptorProto *ret = google_protobuf_DescriptorProto_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_DescriptorProto_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_DescriptorProto_serialize(const google_protobuf_DescriptorProto *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_DescriptorProto_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_DescriptorProto_has_name(const google_protobuf_DescriptorProto *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE upb_strview google_protobuf_DescriptorProto_name(const google_protobuf_DescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)); }
+UPB_INLINE const google_protobuf_FieldDescriptorProto* const* google_protobuf_DescriptorProto_field(const google_protobuf_DescriptorProto *msg, size_t *len) { return (const google_protobuf_FieldDescriptorProto* const*)_upb_array_accessor(msg, UPB_SIZE(16, 32), len); }
+UPB_INLINE const google_protobuf_DescriptorProto* const* google_protobuf_DescriptorProto_nested_type(const google_protobuf_DescriptorProto *msg, size_t *len) { return (const google_protobuf_DescriptorProto* const*)_upb_array_accessor(msg, UPB_SIZE(20, 40), len); }
+UPB_INLINE const google_protobuf_EnumDescriptorProto* const* google_protobuf_DescriptorProto_enum_type(const google_protobuf_DescriptorProto *msg, size_t *len) { return (const google_protobuf_EnumDescriptorProto* const*)_upb_array_accessor(msg, UPB_SIZE(24, 48), len); }
+UPB_INLINE const google_protobuf_DescriptorProto_ExtensionRange* const* google_protobuf_DescriptorProto_extension_range(const google_protobuf_DescriptorProto *msg, size_t *len) { return (const google_protobuf_DescriptorProto_ExtensionRange* const*)_upb_array_accessor(msg, UPB_SIZE(28, 56), len); }
+UPB_INLINE const google_protobuf_FieldDescriptorProto* const* google_protobuf_DescriptorProto_extension(const google_protobuf_DescriptorProto *msg, size_t *len) { return (const google_protobuf_FieldDescriptorProto* const*)_upb_array_accessor(msg, UPB_SIZE(32, 64), len); }
+UPB_INLINE bool google_protobuf_DescriptorProto_has_options(const google_protobuf_DescriptorProto *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE const google_protobuf_MessageOptions* google_protobuf_DescriptorProto_options(const google_protobuf_DescriptorProto *msg) { return UPB_FIELD_AT(msg, const google_protobuf_MessageOptions*, UPB_SIZE(12, 24)); }
+UPB_INLINE const google_protobuf_OneofDescriptorProto* const* google_protobuf_DescriptorProto_oneof_decl(const google_protobuf_DescriptorProto *msg, size_t *len) { return (const google_protobuf_OneofDescriptorProto* const*)_upb_array_accessor(msg, UPB_SIZE(36, 72), len); }
+UPB_INLINE const google_protobuf_DescriptorProto_ReservedRange* const* google_protobuf_DescriptorProto_reserved_range(const google_protobuf_DescriptorProto *msg, size_t *len) { return (const google_protobuf_DescriptorProto_ReservedRange* const*)_upb_array_accessor(msg, UPB_SIZE(40, 80), len); }
+UPB_INLINE upb_strview const* google_protobuf_DescriptorProto_reserved_name(const google_protobuf_DescriptorProto *msg, size_t *len) { return (upb_strview const*)_upb_array_accessor(msg, UPB_SIZE(44, 88), len); }
+
+UPB_INLINE void google_protobuf_DescriptorProto_set_name(google_protobuf_DescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)) = value;
+}
+UPB_INLINE google_protobuf_FieldDescriptorProto** google_protobuf_DescriptorProto_mutable_field(google_protobuf_DescriptorProto *msg, size_t *len) {
+ return (google_protobuf_FieldDescriptorProto**)_upb_array_mutable_accessor(msg, UPB_SIZE(16, 32), len);
+}
+UPB_INLINE google_protobuf_FieldDescriptorProto** google_protobuf_DescriptorProto_resize_field(google_protobuf_DescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_FieldDescriptorProto**)_upb_array_resize_accessor(msg, UPB_SIZE(16, 32), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_FieldDescriptorProto* google_protobuf_DescriptorProto_add_field(google_protobuf_DescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_FieldDescriptorProto* sub = (struct google_protobuf_FieldDescriptorProto*)upb_msg_new(&google_protobuf_FieldDescriptorProto_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(16, 32), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+UPB_INLINE google_protobuf_DescriptorProto** google_protobuf_DescriptorProto_mutable_nested_type(google_protobuf_DescriptorProto *msg, size_t *len) {
+ return (google_protobuf_DescriptorProto**)_upb_array_mutable_accessor(msg, UPB_SIZE(20, 40), len);
+}
+UPB_INLINE google_protobuf_DescriptorProto** google_protobuf_DescriptorProto_resize_nested_type(google_protobuf_DescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_DescriptorProto**)_upb_array_resize_accessor(msg, UPB_SIZE(20, 40), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_DescriptorProto* google_protobuf_DescriptorProto_add_nested_type(google_protobuf_DescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_DescriptorProto* sub = (struct google_protobuf_DescriptorProto*)upb_msg_new(&google_protobuf_DescriptorProto_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(20, 40), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+UPB_INLINE google_protobuf_EnumDescriptorProto** google_protobuf_DescriptorProto_mutable_enum_type(google_protobuf_DescriptorProto *msg, size_t *len) {
+ return (google_protobuf_EnumDescriptorProto**)_upb_array_mutable_accessor(msg, UPB_SIZE(24, 48), len);
+}
+UPB_INLINE google_protobuf_EnumDescriptorProto** google_protobuf_DescriptorProto_resize_enum_type(google_protobuf_DescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_EnumDescriptorProto**)_upb_array_resize_accessor(msg, UPB_SIZE(24, 48), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_EnumDescriptorProto* google_protobuf_DescriptorProto_add_enum_type(google_protobuf_DescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_EnumDescriptorProto* sub = (struct google_protobuf_EnumDescriptorProto*)upb_msg_new(&google_protobuf_EnumDescriptorProto_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(24, 48), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+UPB_INLINE google_protobuf_DescriptorProto_ExtensionRange** google_protobuf_DescriptorProto_mutable_extension_range(google_protobuf_DescriptorProto *msg, size_t *len) {
+ return (google_protobuf_DescriptorProto_ExtensionRange**)_upb_array_mutable_accessor(msg, UPB_SIZE(28, 56), len);
+}
+UPB_INLINE google_protobuf_DescriptorProto_ExtensionRange** google_protobuf_DescriptorProto_resize_extension_range(google_protobuf_DescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_DescriptorProto_ExtensionRange**)_upb_array_resize_accessor(msg, UPB_SIZE(28, 56), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_DescriptorProto_ExtensionRange* google_protobuf_DescriptorProto_add_extension_range(google_protobuf_DescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_DescriptorProto_ExtensionRange* sub = (struct google_protobuf_DescriptorProto_ExtensionRange*)upb_msg_new(&google_protobuf_DescriptorProto_ExtensionRange_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(28, 56), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+UPB_INLINE google_protobuf_FieldDescriptorProto** google_protobuf_DescriptorProto_mutable_extension(google_protobuf_DescriptorProto *msg, size_t *len) {
+ return (google_protobuf_FieldDescriptorProto**)_upb_array_mutable_accessor(msg, UPB_SIZE(32, 64), len);
+}
+UPB_INLINE google_protobuf_FieldDescriptorProto** google_protobuf_DescriptorProto_resize_extension(google_protobuf_DescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_FieldDescriptorProto**)_upb_array_resize_accessor(msg, UPB_SIZE(32, 64), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_FieldDescriptorProto* google_protobuf_DescriptorProto_add_extension(google_protobuf_DescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_FieldDescriptorProto* sub = (struct google_protobuf_FieldDescriptorProto*)upb_msg_new(&google_protobuf_FieldDescriptorProto_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(32, 64), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+UPB_INLINE void google_protobuf_DescriptorProto_set_options(google_protobuf_DescriptorProto *msg, google_protobuf_MessageOptions* value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, google_protobuf_MessageOptions*, UPB_SIZE(12, 24)) = value;
+}
+UPB_INLINE struct google_protobuf_MessageOptions* google_protobuf_DescriptorProto_mutable_options(google_protobuf_DescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_MessageOptions* sub = (struct google_protobuf_MessageOptions*)google_protobuf_DescriptorProto_options(msg);
+ if (sub == NULL) {
+ sub = (struct google_protobuf_MessageOptions*)upb_msg_new(&google_protobuf_MessageOptions_msginit, arena);
+ if (!sub) return NULL;
+ google_protobuf_DescriptorProto_set_options(msg, sub);
+ }
+ return sub;
+}
+UPB_INLINE google_protobuf_OneofDescriptorProto** google_protobuf_DescriptorProto_mutable_oneof_decl(google_protobuf_DescriptorProto *msg, size_t *len) {
+ return (google_protobuf_OneofDescriptorProto**)_upb_array_mutable_accessor(msg, UPB_SIZE(36, 72), len);
+}
+UPB_INLINE google_protobuf_OneofDescriptorProto** google_protobuf_DescriptorProto_resize_oneof_decl(google_protobuf_DescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_OneofDescriptorProto**)_upb_array_resize_accessor(msg, UPB_SIZE(36, 72), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_OneofDescriptorProto* google_protobuf_DescriptorProto_add_oneof_decl(google_protobuf_DescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_OneofDescriptorProto* sub = (struct google_protobuf_OneofDescriptorProto*)upb_msg_new(&google_protobuf_OneofDescriptorProto_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(36, 72), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+UPB_INLINE google_protobuf_DescriptorProto_ReservedRange** google_protobuf_DescriptorProto_mutable_reserved_range(google_protobuf_DescriptorProto *msg, size_t *len) {
+ return (google_protobuf_DescriptorProto_ReservedRange**)_upb_array_mutable_accessor(msg, UPB_SIZE(40, 80), len);
+}
+UPB_INLINE google_protobuf_DescriptorProto_ReservedRange** google_protobuf_DescriptorProto_resize_reserved_range(google_protobuf_DescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_DescriptorProto_ReservedRange**)_upb_array_resize_accessor(msg, UPB_SIZE(40, 80), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_DescriptorProto_ReservedRange* google_protobuf_DescriptorProto_add_reserved_range(google_protobuf_DescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_DescriptorProto_ReservedRange* sub = (struct google_protobuf_DescriptorProto_ReservedRange*)upb_msg_new(&google_protobuf_DescriptorProto_ReservedRange_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(40, 80), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+UPB_INLINE upb_strview* google_protobuf_DescriptorProto_mutable_reserved_name(google_protobuf_DescriptorProto *msg, size_t *len) {
+ return (upb_strview*)_upb_array_mutable_accessor(msg, UPB_SIZE(44, 88), len);
+}
+UPB_INLINE upb_strview* google_protobuf_DescriptorProto_resize_reserved_name(google_protobuf_DescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (upb_strview*)_upb_array_resize_accessor(msg, UPB_SIZE(44, 88), len, UPB_SIZE(8, 16), UPB_TYPE_STRING, arena);
+}
+UPB_INLINE bool google_protobuf_DescriptorProto_add_reserved_name(google_protobuf_DescriptorProto *msg, upb_strview val, upb_arena *arena) {
+ return _upb_array_append_accessor(
+ msg, UPB_SIZE(44, 88), UPB_SIZE(8, 16), UPB_TYPE_STRING, &val, arena);
+}
+
+
+/* google.protobuf.DescriptorProto.ExtensionRange */
+
+UPB_INLINE google_protobuf_DescriptorProto_ExtensionRange *google_protobuf_DescriptorProto_ExtensionRange_new(upb_arena *arena) {
+ return (google_protobuf_DescriptorProto_ExtensionRange *)upb_msg_new(&google_protobuf_DescriptorProto_ExtensionRange_msginit, arena);
+}
+UPB_INLINE google_protobuf_DescriptorProto_ExtensionRange *google_protobuf_DescriptorProto_ExtensionRange_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_DescriptorProto_ExtensionRange *ret = google_protobuf_DescriptorProto_ExtensionRange_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_DescriptorProto_ExtensionRange_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_DescriptorProto_ExtensionRange_serialize(const google_protobuf_DescriptorProto_ExtensionRange *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_DescriptorProto_ExtensionRange_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_DescriptorProto_ExtensionRange_has_start(const google_protobuf_DescriptorProto_ExtensionRange *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE int32_t google_protobuf_DescriptorProto_ExtensionRange_start(const google_protobuf_DescriptorProto_ExtensionRange *msg) { return UPB_FIELD_AT(msg, int32_t, UPB_SIZE(4, 4)); }
+UPB_INLINE bool google_protobuf_DescriptorProto_ExtensionRange_has_end(const google_protobuf_DescriptorProto_ExtensionRange *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE int32_t google_protobuf_DescriptorProto_ExtensionRange_end(const google_protobuf_DescriptorProto_ExtensionRange *msg) { return UPB_FIELD_AT(msg, int32_t, UPB_SIZE(8, 8)); }
+UPB_INLINE bool google_protobuf_DescriptorProto_ExtensionRange_has_options(const google_protobuf_DescriptorProto_ExtensionRange *msg) { return _upb_has_field(msg, 3); }
+UPB_INLINE const google_protobuf_ExtensionRangeOptions* google_protobuf_DescriptorProto_ExtensionRange_options(const google_protobuf_DescriptorProto_ExtensionRange *msg) { return UPB_FIELD_AT(msg, const google_protobuf_ExtensionRangeOptions*, UPB_SIZE(12, 16)); }
+
+UPB_INLINE void google_protobuf_DescriptorProto_ExtensionRange_set_start(google_protobuf_DescriptorProto_ExtensionRange *msg, int32_t value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, int32_t, UPB_SIZE(4, 4)) = value;
+}
+UPB_INLINE void google_protobuf_DescriptorProto_ExtensionRange_set_end(google_protobuf_DescriptorProto_ExtensionRange *msg, int32_t value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, int32_t, UPB_SIZE(8, 8)) = value;
+}
+UPB_INLINE void google_protobuf_DescriptorProto_ExtensionRange_set_options(google_protobuf_DescriptorProto_ExtensionRange *msg, google_protobuf_ExtensionRangeOptions* value) {
+ _upb_sethas(msg, 3);
+ UPB_FIELD_AT(msg, google_protobuf_ExtensionRangeOptions*, UPB_SIZE(12, 16)) = value;
+}
+UPB_INLINE struct google_protobuf_ExtensionRangeOptions* google_protobuf_DescriptorProto_ExtensionRange_mutable_options(google_protobuf_DescriptorProto_ExtensionRange *msg, upb_arena *arena) {
+ struct google_protobuf_ExtensionRangeOptions* sub = (struct google_protobuf_ExtensionRangeOptions*)google_protobuf_DescriptorProto_ExtensionRange_options(msg);
+ if (sub == NULL) {
+ sub = (struct google_protobuf_ExtensionRangeOptions*)upb_msg_new(&google_protobuf_ExtensionRangeOptions_msginit, arena);
+ if (!sub) return NULL;
+ google_protobuf_DescriptorProto_ExtensionRange_set_options(msg, sub);
+ }
+ return sub;
+}
+
+
+/* google.protobuf.DescriptorProto.ReservedRange */
+
+UPB_INLINE google_protobuf_DescriptorProto_ReservedRange *google_protobuf_DescriptorProto_ReservedRange_new(upb_arena *arena) {
+ return (google_protobuf_DescriptorProto_ReservedRange *)upb_msg_new(&google_protobuf_DescriptorProto_ReservedRange_msginit, arena);
+}
+UPB_INLINE google_protobuf_DescriptorProto_ReservedRange *google_protobuf_DescriptorProto_ReservedRange_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_DescriptorProto_ReservedRange *ret = google_protobuf_DescriptorProto_ReservedRange_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_DescriptorProto_ReservedRange_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_DescriptorProto_ReservedRange_serialize(const google_protobuf_DescriptorProto_ReservedRange *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_DescriptorProto_ReservedRange_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_DescriptorProto_ReservedRange_has_start(const google_protobuf_DescriptorProto_ReservedRange *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE int32_t google_protobuf_DescriptorProto_ReservedRange_start(const google_protobuf_DescriptorProto_ReservedRange *msg) { return UPB_FIELD_AT(msg, int32_t, UPB_SIZE(4, 4)); }
+UPB_INLINE bool google_protobuf_DescriptorProto_ReservedRange_has_end(const google_protobuf_DescriptorProto_ReservedRange *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE int32_t google_protobuf_DescriptorProto_ReservedRange_end(const google_protobuf_DescriptorProto_ReservedRange *msg) { return UPB_FIELD_AT(msg, int32_t, UPB_SIZE(8, 8)); }
+
+UPB_INLINE void google_protobuf_DescriptorProto_ReservedRange_set_start(google_protobuf_DescriptorProto_ReservedRange *msg, int32_t value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, int32_t, UPB_SIZE(4, 4)) = value;
+}
+UPB_INLINE void google_protobuf_DescriptorProto_ReservedRange_set_end(google_protobuf_DescriptorProto_ReservedRange *msg, int32_t value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, int32_t, UPB_SIZE(8, 8)) = value;
+}
+
+
+/* google.protobuf.ExtensionRangeOptions */
+
+UPB_INLINE google_protobuf_ExtensionRangeOptions *google_protobuf_ExtensionRangeOptions_new(upb_arena *arena) {
+ return (google_protobuf_ExtensionRangeOptions *)upb_msg_new(&google_protobuf_ExtensionRangeOptions_msginit, arena);
+}
+UPB_INLINE google_protobuf_ExtensionRangeOptions *google_protobuf_ExtensionRangeOptions_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_ExtensionRangeOptions *ret = google_protobuf_ExtensionRangeOptions_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_ExtensionRangeOptions_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_ExtensionRangeOptions_serialize(const google_protobuf_ExtensionRangeOptions *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_ExtensionRangeOptions_msginit, arena, len);
+}
+
+UPB_INLINE const google_protobuf_UninterpretedOption* const* google_protobuf_ExtensionRangeOptions_uninterpreted_option(const google_protobuf_ExtensionRangeOptions *msg, size_t *len) { return (const google_protobuf_UninterpretedOption* const*)_upb_array_accessor(msg, UPB_SIZE(0, 0), len); }
+
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_ExtensionRangeOptions_mutable_uninterpreted_option(google_protobuf_ExtensionRangeOptions *msg, size_t *len) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_mutable_accessor(msg, UPB_SIZE(0, 0), len);
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_ExtensionRangeOptions_resize_uninterpreted_option(google_protobuf_ExtensionRangeOptions *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_resize_accessor(msg, UPB_SIZE(0, 0), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_UninterpretedOption* google_protobuf_ExtensionRangeOptions_add_uninterpreted_option(google_protobuf_ExtensionRangeOptions *msg, upb_arena *arena) {
+ struct google_protobuf_UninterpretedOption* sub = (struct google_protobuf_UninterpretedOption*)upb_msg_new(&google_protobuf_UninterpretedOption_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(0, 0), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+
+
+/* google.protobuf.FieldDescriptorProto */
+
+UPB_INLINE google_protobuf_FieldDescriptorProto *google_protobuf_FieldDescriptorProto_new(upb_arena *arena) {
+ return (google_protobuf_FieldDescriptorProto *)upb_msg_new(&google_protobuf_FieldDescriptorProto_msginit, arena);
+}
+UPB_INLINE google_protobuf_FieldDescriptorProto *google_protobuf_FieldDescriptorProto_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_FieldDescriptorProto *ret = google_protobuf_FieldDescriptorProto_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_FieldDescriptorProto_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_FieldDescriptorProto_serialize(const google_protobuf_FieldDescriptorProto *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_FieldDescriptorProto_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_FieldDescriptorProto_has_name(const google_protobuf_FieldDescriptorProto *msg) { return _upb_has_field(msg, 5); }
+UPB_INLINE upb_strview google_protobuf_FieldDescriptorProto_name(const google_protobuf_FieldDescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(32, 32)); }
+UPB_INLINE bool google_protobuf_FieldDescriptorProto_has_extendee(const google_protobuf_FieldDescriptorProto *msg) { return _upb_has_field(msg, 6); }
+UPB_INLINE upb_strview google_protobuf_FieldDescriptorProto_extendee(const google_protobuf_FieldDescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(40, 48)); }
+UPB_INLINE bool google_protobuf_FieldDescriptorProto_has_number(const google_protobuf_FieldDescriptorProto *msg) { return _upb_has_field(msg, 3); }
+UPB_INLINE int32_t google_protobuf_FieldDescriptorProto_number(const google_protobuf_FieldDescriptorProto *msg) { return UPB_FIELD_AT(msg, int32_t, UPB_SIZE(24, 24)); }
+UPB_INLINE bool google_protobuf_FieldDescriptorProto_has_label(const google_protobuf_FieldDescriptorProto *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE google_protobuf_FieldDescriptorProto_Label google_protobuf_FieldDescriptorProto_label(const google_protobuf_FieldDescriptorProto *msg) { return UPB_FIELD_AT(msg, google_protobuf_FieldDescriptorProto_Label, UPB_SIZE(8, 8)); }
+UPB_INLINE bool google_protobuf_FieldDescriptorProto_has_type(const google_protobuf_FieldDescriptorProto *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE google_protobuf_FieldDescriptorProto_Type google_protobuf_FieldDescriptorProto_type(const google_protobuf_FieldDescriptorProto *msg) { return UPB_FIELD_AT(msg, google_protobuf_FieldDescriptorProto_Type, UPB_SIZE(16, 16)); }
+UPB_INLINE bool google_protobuf_FieldDescriptorProto_has_type_name(const google_protobuf_FieldDescriptorProto *msg) { return _upb_has_field(msg, 7); }
+UPB_INLINE upb_strview google_protobuf_FieldDescriptorProto_type_name(const google_protobuf_FieldDescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(48, 64)); }
+UPB_INLINE bool google_protobuf_FieldDescriptorProto_has_default_value(const google_protobuf_FieldDescriptorProto *msg) { return _upb_has_field(msg, 8); }
+UPB_INLINE upb_strview google_protobuf_FieldDescriptorProto_default_value(const google_protobuf_FieldDescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(56, 80)); }
+UPB_INLINE bool google_protobuf_FieldDescriptorProto_has_options(const google_protobuf_FieldDescriptorProto *msg) { return _upb_has_field(msg, 10); }
+UPB_INLINE const google_protobuf_FieldOptions* google_protobuf_FieldDescriptorProto_options(const google_protobuf_FieldDescriptorProto *msg) { return UPB_FIELD_AT(msg, const google_protobuf_FieldOptions*, UPB_SIZE(72, 112)); }
+UPB_INLINE bool google_protobuf_FieldDescriptorProto_has_oneof_index(const google_protobuf_FieldDescriptorProto *msg) { return _upb_has_field(msg, 4); }
+UPB_INLINE int32_t google_protobuf_FieldDescriptorProto_oneof_index(const google_protobuf_FieldDescriptorProto *msg) { return UPB_FIELD_AT(msg, int32_t, UPB_SIZE(28, 28)); }
+UPB_INLINE bool google_protobuf_FieldDescriptorProto_has_json_name(const google_protobuf_FieldDescriptorProto *msg) { return _upb_has_field(msg, 9); }
+UPB_INLINE upb_strview google_protobuf_FieldDescriptorProto_json_name(const google_protobuf_FieldDescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(64, 96)); }
+
+UPB_INLINE void google_protobuf_FieldDescriptorProto_set_name(google_protobuf_FieldDescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 5);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(32, 32)) = value;
+}
+UPB_INLINE void google_protobuf_FieldDescriptorProto_set_extendee(google_protobuf_FieldDescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 6);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(40, 48)) = value;
+}
+UPB_INLINE void google_protobuf_FieldDescriptorProto_set_number(google_protobuf_FieldDescriptorProto *msg, int32_t value) {
+ _upb_sethas(msg, 3);
+ UPB_FIELD_AT(msg, int32_t, UPB_SIZE(24, 24)) = value;
+}
+UPB_INLINE void google_protobuf_FieldDescriptorProto_set_label(google_protobuf_FieldDescriptorProto *msg, google_protobuf_FieldDescriptorProto_Label value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, google_protobuf_FieldDescriptorProto_Label, UPB_SIZE(8, 8)) = value;
+}
+UPB_INLINE void google_protobuf_FieldDescriptorProto_set_type(google_protobuf_FieldDescriptorProto *msg, google_protobuf_FieldDescriptorProto_Type value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, google_protobuf_FieldDescriptorProto_Type, UPB_SIZE(16, 16)) = value;
+}
+UPB_INLINE void google_protobuf_FieldDescriptorProto_set_type_name(google_protobuf_FieldDescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 7);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(48, 64)) = value;
+}
+UPB_INLINE void google_protobuf_FieldDescriptorProto_set_default_value(google_protobuf_FieldDescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 8);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(56, 80)) = value;
+}
+UPB_INLINE void google_protobuf_FieldDescriptorProto_set_options(google_protobuf_FieldDescriptorProto *msg, google_protobuf_FieldOptions* value) {
+ _upb_sethas(msg, 10);
+ UPB_FIELD_AT(msg, google_protobuf_FieldOptions*, UPB_SIZE(72, 112)) = value;
+}
+UPB_INLINE struct google_protobuf_FieldOptions* google_protobuf_FieldDescriptorProto_mutable_options(google_protobuf_FieldDescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_FieldOptions* sub = (struct google_protobuf_FieldOptions*)google_protobuf_FieldDescriptorProto_options(msg);
+ if (sub == NULL) {
+ sub = (struct google_protobuf_FieldOptions*)upb_msg_new(&google_protobuf_FieldOptions_msginit, arena);
+ if (!sub) return NULL;
+ google_protobuf_FieldDescriptorProto_set_options(msg, sub);
+ }
+ return sub;
+}
+UPB_INLINE void google_protobuf_FieldDescriptorProto_set_oneof_index(google_protobuf_FieldDescriptorProto *msg, int32_t value) {
+ _upb_sethas(msg, 4);
+ UPB_FIELD_AT(msg, int32_t, UPB_SIZE(28, 28)) = value;
+}
+UPB_INLINE void google_protobuf_FieldDescriptorProto_set_json_name(google_protobuf_FieldDescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 9);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(64, 96)) = value;
+}
+
+
+/* google.protobuf.OneofDescriptorProto */
+
+UPB_INLINE google_protobuf_OneofDescriptorProto *google_protobuf_OneofDescriptorProto_new(upb_arena *arena) {
+ return (google_protobuf_OneofDescriptorProto *)upb_msg_new(&google_protobuf_OneofDescriptorProto_msginit, arena);
+}
+UPB_INLINE google_protobuf_OneofDescriptorProto *google_protobuf_OneofDescriptorProto_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_OneofDescriptorProto *ret = google_protobuf_OneofDescriptorProto_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_OneofDescriptorProto_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_OneofDescriptorProto_serialize(const google_protobuf_OneofDescriptorProto *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_OneofDescriptorProto_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_OneofDescriptorProto_has_name(const google_protobuf_OneofDescriptorProto *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE upb_strview google_protobuf_OneofDescriptorProto_name(const google_protobuf_OneofDescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)); }
+UPB_INLINE bool google_protobuf_OneofDescriptorProto_has_options(const google_protobuf_OneofDescriptorProto *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE const google_protobuf_OneofOptions* google_protobuf_OneofDescriptorProto_options(const google_protobuf_OneofDescriptorProto *msg) { return UPB_FIELD_AT(msg, const google_protobuf_OneofOptions*, UPB_SIZE(12, 24)); }
+
+UPB_INLINE void google_protobuf_OneofDescriptorProto_set_name(google_protobuf_OneofDescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)) = value;
+}
+UPB_INLINE void google_protobuf_OneofDescriptorProto_set_options(google_protobuf_OneofDescriptorProto *msg, google_protobuf_OneofOptions* value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, google_protobuf_OneofOptions*, UPB_SIZE(12, 24)) = value;
+}
+UPB_INLINE struct google_protobuf_OneofOptions* google_protobuf_OneofDescriptorProto_mutable_options(google_protobuf_OneofDescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_OneofOptions* sub = (struct google_protobuf_OneofOptions*)google_protobuf_OneofDescriptorProto_options(msg);
+ if (sub == NULL) {
+ sub = (struct google_protobuf_OneofOptions*)upb_msg_new(&google_protobuf_OneofOptions_msginit, arena);
+ if (!sub) return NULL;
+ google_protobuf_OneofDescriptorProto_set_options(msg, sub);
+ }
+ return sub;
+}
+
+
+/* google.protobuf.EnumDescriptorProto */
+
+UPB_INLINE google_protobuf_EnumDescriptorProto *google_protobuf_EnumDescriptorProto_new(upb_arena *arena) {
+ return (google_protobuf_EnumDescriptorProto *)upb_msg_new(&google_protobuf_EnumDescriptorProto_msginit, arena);
+}
+UPB_INLINE google_protobuf_EnumDescriptorProto *google_protobuf_EnumDescriptorProto_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_EnumDescriptorProto *ret = google_protobuf_EnumDescriptorProto_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_EnumDescriptorProto_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_EnumDescriptorProto_serialize(const google_protobuf_EnumDescriptorProto *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_EnumDescriptorProto_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_EnumDescriptorProto_has_name(const google_protobuf_EnumDescriptorProto *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE upb_strview google_protobuf_EnumDescriptorProto_name(const google_protobuf_EnumDescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)); }
+UPB_INLINE const google_protobuf_EnumValueDescriptorProto* const* google_protobuf_EnumDescriptorProto_value(const google_protobuf_EnumDescriptorProto *msg, size_t *len) { return (const google_protobuf_EnumValueDescriptorProto* const*)_upb_array_accessor(msg, UPB_SIZE(16, 32), len); }
+UPB_INLINE bool google_protobuf_EnumDescriptorProto_has_options(const google_protobuf_EnumDescriptorProto *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE const google_protobuf_EnumOptions* google_protobuf_EnumDescriptorProto_options(const google_protobuf_EnumDescriptorProto *msg) { return UPB_FIELD_AT(msg, const google_protobuf_EnumOptions*, UPB_SIZE(12, 24)); }
+UPB_INLINE const google_protobuf_EnumDescriptorProto_EnumReservedRange* const* google_protobuf_EnumDescriptorProto_reserved_range(const google_protobuf_EnumDescriptorProto *msg, size_t *len) { return (const google_protobuf_EnumDescriptorProto_EnumReservedRange* const*)_upb_array_accessor(msg, UPB_SIZE(20, 40), len); }
+UPB_INLINE upb_strview const* google_protobuf_EnumDescriptorProto_reserved_name(const google_protobuf_EnumDescriptorProto *msg, size_t *len) { return (upb_strview const*)_upb_array_accessor(msg, UPB_SIZE(24, 48), len); }
+
+UPB_INLINE void google_protobuf_EnumDescriptorProto_set_name(google_protobuf_EnumDescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)) = value;
+}
+UPB_INLINE google_protobuf_EnumValueDescriptorProto** google_protobuf_EnumDescriptorProto_mutable_value(google_protobuf_EnumDescriptorProto *msg, size_t *len) {
+ return (google_protobuf_EnumValueDescriptorProto**)_upb_array_mutable_accessor(msg, UPB_SIZE(16, 32), len);
+}
+UPB_INLINE google_protobuf_EnumValueDescriptorProto** google_protobuf_EnumDescriptorProto_resize_value(google_protobuf_EnumDescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_EnumValueDescriptorProto**)_upb_array_resize_accessor(msg, UPB_SIZE(16, 32), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_EnumValueDescriptorProto* google_protobuf_EnumDescriptorProto_add_value(google_protobuf_EnumDescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_EnumValueDescriptorProto* sub = (struct google_protobuf_EnumValueDescriptorProto*)upb_msg_new(&google_protobuf_EnumValueDescriptorProto_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(16, 32), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+UPB_INLINE void google_protobuf_EnumDescriptorProto_set_options(google_protobuf_EnumDescriptorProto *msg, google_protobuf_EnumOptions* value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, google_protobuf_EnumOptions*, UPB_SIZE(12, 24)) = value;
+}
+UPB_INLINE struct google_protobuf_EnumOptions* google_protobuf_EnumDescriptorProto_mutable_options(google_protobuf_EnumDescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_EnumOptions* sub = (struct google_protobuf_EnumOptions*)google_protobuf_EnumDescriptorProto_options(msg);
+ if (sub == NULL) {
+ sub = (struct google_protobuf_EnumOptions*)upb_msg_new(&google_protobuf_EnumOptions_msginit, arena);
+ if (!sub) return NULL;
+ google_protobuf_EnumDescriptorProto_set_options(msg, sub);
+ }
+ return sub;
+}
+UPB_INLINE google_protobuf_EnumDescriptorProto_EnumReservedRange** google_protobuf_EnumDescriptorProto_mutable_reserved_range(google_protobuf_EnumDescriptorProto *msg, size_t *len) {
+ return (google_protobuf_EnumDescriptorProto_EnumReservedRange**)_upb_array_mutable_accessor(msg, UPB_SIZE(20, 40), len);
+}
+UPB_INLINE google_protobuf_EnumDescriptorProto_EnumReservedRange** google_protobuf_EnumDescriptorProto_resize_reserved_range(google_protobuf_EnumDescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_EnumDescriptorProto_EnumReservedRange**)_upb_array_resize_accessor(msg, UPB_SIZE(20, 40), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_EnumDescriptorProto_EnumReservedRange* google_protobuf_EnumDescriptorProto_add_reserved_range(google_protobuf_EnumDescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_EnumDescriptorProto_EnumReservedRange* sub = (struct google_protobuf_EnumDescriptorProto_EnumReservedRange*)upb_msg_new(&google_protobuf_EnumDescriptorProto_EnumReservedRange_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(20, 40), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+UPB_INLINE upb_strview* google_protobuf_EnumDescriptorProto_mutable_reserved_name(google_protobuf_EnumDescriptorProto *msg, size_t *len) {
+ return (upb_strview*)_upb_array_mutable_accessor(msg, UPB_SIZE(24, 48), len);
+}
+UPB_INLINE upb_strview* google_protobuf_EnumDescriptorProto_resize_reserved_name(google_protobuf_EnumDescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (upb_strview*)_upb_array_resize_accessor(msg, UPB_SIZE(24, 48), len, UPB_SIZE(8, 16), UPB_TYPE_STRING, arena);
+}
+UPB_INLINE bool google_protobuf_EnumDescriptorProto_add_reserved_name(google_protobuf_EnumDescriptorProto *msg, upb_strview val, upb_arena *arena) {
+ return _upb_array_append_accessor(
+ msg, UPB_SIZE(24, 48), UPB_SIZE(8, 16), UPB_TYPE_STRING, &val, arena);
+}
+
+
+/* google.protobuf.EnumDescriptorProto.EnumReservedRange */
+
+UPB_INLINE google_protobuf_EnumDescriptorProto_EnumReservedRange *google_protobuf_EnumDescriptorProto_EnumReservedRange_new(upb_arena *arena) {
+ return (google_protobuf_EnumDescriptorProto_EnumReservedRange *)upb_msg_new(&google_protobuf_EnumDescriptorProto_EnumReservedRange_msginit, arena);
+}
+UPB_INLINE google_protobuf_EnumDescriptorProto_EnumReservedRange *google_protobuf_EnumDescriptorProto_EnumReservedRange_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_EnumDescriptorProto_EnumReservedRange *ret = google_protobuf_EnumDescriptorProto_EnumReservedRange_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_EnumDescriptorProto_EnumReservedRange_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_EnumDescriptorProto_EnumReservedRange_serialize(const google_protobuf_EnumDescriptorProto_EnumReservedRange *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_EnumDescriptorProto_EnumReservedRange_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_EnumDescriptorProto_EnumReservedRange_has_start(const google_protobuf_EnumDescriptorProto_EnumReservedRange *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE int32_t google_protobuf_EnumDescriptorProto_EnumReservedRange_start(const google_protobuf_EnumDescriptorProto_EnumReservedRange *msg) { return UPB_FIELD_AT(msg, int32_t, UPB_SIZE(4, 4)); }
+UPB_INLINE bool google_protobuf_EnumDescriptorProto_EnumReservedRange_has_end(const google_protobuf_EnumDescriptorProto_EnumReservedRange *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE int32_t google_protobuf_EnumDescriptorProto_EnumReservedRange_end(const google_protobuf_EnumDescriptorProto_EnumReservedRange *msg) { return UPB_FIELD_AT(msg, int32_t, UPB_SIZE(8, 8)); }
+
+UPB_INLINE void google_protobuf_EnumDescriptorProto_EnumReservedRange_set_start(google_protobuf_EnumDescriptorProto_EnumReservedRange *msg, int32_t value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, int32_t, UPB_SIZE(4, 4)) = value;
+}
+UPB_INLINE void google_protobuf_EnumDescriptorProto_EnumReservedRange_set_end(google_protobuf_EnumDescriptorProto_EnumReservedRange *msg, int32_t value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, int32_t, UPB_SIZE(8, 8)) = value;
+}
+
+
+/* google.protobuf.EnumValueDescriptorProto */
+
+UPB_INLINE google_protobuf_EnumValueDescriptorProto *google_protobuf_EnumValueDescriptorProto_new(upb_arena *arena) {
+ return (google_protobuf_EnumValueDescriptorProto *)upb_msg_new(&google_protobuf_EnumValueDescriptorProto_msginit, arena);
+}
+UPB_INLINE google_protobuf_EnumValueDescriptorProto *google_protobuf_EnumValueDescriptorProto_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_EnumValueDescriptorProto *ret = google_protobuf_EnumValueDescriptorProto_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_EnumValueDescriptorProto_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_EnumValueDescriptorProto_serialize(const google_protobuf_EnumValueDescriptorProto *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_EnumValueDescriptorProto_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_EnumValueDescriptorProto_has_name(const google_protobuf_EnumValueDescriptorProto *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE upb_strview google_protobuf_EnumValueDescriptorProto_name(const google_protobuf_EnumValueDescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(8, 8)); }
+UPB_INLINE bool google_protobuf_EnumValueDescriptorProto_has_number(const google_protobuf_EnumValueDescriptorProto *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE int32_t google_protobuf_EnumValueDescriptorProto_number(const google_protobuf_EnumValueDescriptorProto *msg) { return UPB_FIELD_AT(msg, int32_t, UPB_SIZE(4, 4)); }
+UPB_INLINE bool google_protobuf_EnumValueDescriptorProto_has_options(const google_protobuf_EnumValueDescriptorProto *msg) { return _upb_has_field(msg, 3); }
+UPB_INLINE const google_protobuf_EnumValueOptions* google_protobuf_EnumValueDescriptorProto_options(const google_protobuf_EnumValueDescriptorProto *msg) { return UPB_FIELD_AT(msg, const google_protobuf_EnumValueOptions*, UPB_SIZE(16, 24)); }
+
+UPB_INLINE void google_protobuf_EnumValueDescriptorProto_set_name(google_protobuf_EnumValueDescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(8, 8)) = value;
+}
+UPB_INLINE void google_protobuf_EnumValueDescriptorProto_set_number(google_protobuf_EnumValueDescriptorProto *msg, int32_t value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, int32_t, UPB_SIZE(4, 4)) = value;
+}
+UPB_INLINE void google_protobuf_EnumValueDescriptorProto_set_options(google_protobuf_EnumValueDescriptorProto *msg, google_protobuf_EnumValueOptions* value) {
+ _upb_sethas(msg, 3);
+ UPB_FIELD_AT(msg, google_protobuf_EnumValueOptions*, UPB_SIZE(16, 24)) = value;
+}
+UPB_INLINE struct google_protobuf_EnumValueOptions* google_protobuf_EnumValueDescriptorProto_mutable_options(google_protobuf_EnumValueDescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_EnumValueOptions* sub = (struct google_protobuf_EnumValueOptions*)google_protobuf_EnumValueDescriptorProto_options(msg);
+ if (sub == NULL) {
+ sub = (struct google_protobuf_EnumValueOptions*)upb_msg_new(&google_protobuf_EnumValueOptions_msginit, arena);
+ if (!sub) return NULL;
+ google_protobuf_EnumValueDescriptorProto_set_options(msg, sub);
+ }
+ return sub;
+}
+
+
+/* google.protobuf.ServiceDescriptorProto */
+
+UPB_INLINE google_protobuf_ServiceDescriptorProto *google_protobuf_ServiceDescriptorProto_new(upb_arena *arena) {
+ return (google_protobuf_ServiceDescriptorProto *)upb_msg_new(&google_protobuf_ServiceDescriptorProto_msginit, arena);
+}
+UPB_INLINE google_protobuf_ServiceDescriptorProto *google_protobuf_ServiceDescriptorProto_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_ServiceDescriptorProto *ret = google_protobuf_ServiceDescriptorProto_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_ServiceDescriptorProto_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_ServiceDescriptorProto_serialize(const google_protobuf_ServiceDescriptorProto *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_ServiceDescriptorProto_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_ServiceDescriptorProto_has_name(const google_protobuf_ServiceDescriptorProto *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE upb_strview google_protobuf_ServiceDescriptorProto_name(const google_protobuf_ServiceDescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)); }
+UPB_INLINE const google_protobuf_MethodDescriptorProto* const* google_protobuf_ServiceDescriptorProto_method(const google_protobuf_ServiceDescriptorProto *msg, size_t *len) { return (const google_protobuf_MethodDescriptorProto* const*)_upb_array_accessor(msg, UPB_SIZE(16, 32), len); }
+UPB_INLINE bool google_protobuf_ServiceDescriptorProto_has_options(const google_protobuf_ServiceDescriptorProto *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE const google_protobuf_ServiceOptions* google_protobuf_ServiceDescriptorProto_options(const google_protobuf_ServiceDescriptorProto *msg) { return UPB_FIELD_AT(msg, const google_protobuf_ServiceOptions*, UPB_SIZE(12, 24)); }
+
+UPB_INLINE void google_protobuf_ServiceDescriptorProto_set_name(google_protobuf_ServiceDescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)) = value;
+}
+UPB_INLINE google_protobuf_MethodDescriptorProto** google_protobuf_ServiceDescriptorProto_mutable_method(google_protobuf_ServiceDescriptorProto *msg, size_t *len) {
+ return (google_protobuf_MethodDescriptorProto**)_upb_array_mutable_accessor(msg, UPB_SIZE(16, 32), len);
+}
+UPB_INLINE google_protobuf_MethodDescriptorProto** google_protobuf_ServiceDescriptorProto_resize_method(google_protobuf_ServiceDescriptorProto *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_MethodDescriptorProto**)_upb_array_resize_accessor(msg, UPB_SIZE(16, 32), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_MethodDescriptorProto* google_protobuf_ServiceDescriptorProto_add_method(google_protobuf_ServiceDescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_MethodDescriptorProto* sub = (struct google_protobuf_MethodDescriptorProto*)upb_msg_new(&google_protobuf_MethodDescriptorProto_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(16, 32), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+UPB_INLINE void google_protobuf_ServiceDescriptorProto_set_options(google_protobuf_ServiceDescriptorProto *msg, google_protobuf_ServiceOptions* value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, google_protobuf_ServiceOptions*, UPB_SIZE(12, 24)) = value;
+}
+UPB_INLINE struct google_protobuf_ServiceOptions* google_protobuf_ServiceDescriptorProto_mutable_options(google_protobuf_ServiceDescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_ServiceOptions* sub = (struct google_protobuf_ServiceOptions*)google_protobuf_ServiceDescriptorProto_options(msg);
+ if (sub == NULL) {
+ sub = (struct google_protobuf_ServiceOptions*)upb_msg_new(&google_protobuf_ServiceOptions_msginit, arena);
+ if (!sub) return NULL;
+ google_protobuf_ServiceDescriptorProto_set_options(msg, sub);
+ }
+ return sub;
+}
+
+
+/* google.protobuf.MethodDescriptorProto */
+
+UPB_INLINE google_protobuf_MethodDescriptorProto *google_protobuf_MethodDescriptorProto_new(upb_arena *arena) {
+ return (google_protobuf_MethodDescriptorProto *)upb_msg_new(&google_protobuf_MethodDescriptorProto_msginit, arena);
+}
+UPB_INLINE google_protobuf_MethodDescriptorProto *google_protobuf_MethodDescriptorProto_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_MethodDescriptorProto *ret = google_protobuf_MethodDescriptorProto_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_MethodDescriptorProto_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_MethodDescriptorProto_serialize(const google_protobuf_MethodDescriptorProto *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_MethodDescriptorProto_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_MethodDescriptorProto_has_name(const google_protobuf_MethodDescriptorProto *msg) { return _upb_has_field(msg, 3); }
+UPB_INLINE upb_strview google_protobuf_MethodDescriptorProto_name(const google_protobuf_MethodDescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)); }
+UPB_INLINE bool google_protobuf_MethodDescriptorProto_has_input_type(const google_protobuf_MethodDescriptorProto *msg) { return _upb_has_field(msg, 4); }
+UPB_INLINE upb_strview google_protobuf_MethodDescriptorProto_input_type(const google_protobuf_MethodDescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(12, 24)); }
+UPB_INLINE bool google_protobuf_MethodDescriptorProto_has_output_type(const google_protobuf_MethodDescriptorProto *msg) { return _upb_has_field(msg, 5); }
+UPB_INLINE upb_strview google_protobuf_MethodDescriptorProto_output_type(const google_protobuf_MethodDescriptorProto *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(20, 40)); }
+UPB_INLINE bool google_protobuf_MethodDescriptorProto_has_options(const google_protobuf_MethodDescriptorProto *msg) { return _upb_has_field(msg, 6); }
+UPB_INLINE const google_protobuf_MethodOptions* google_protobuf_MethodDescriptorProto_options(const google_protobuf_MethodDescriptorProto *msg) { return UPB_FIELD_AT(msg, const google_protobuf_MethodOptions*, UPB_SIZE(28, 56)); }
+UPB_INLINE bool google_protobuf_MethodDescriptorProto_has_client_streaming(const google_protobuf_MethodDescriptorProto *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE bool google_protobuf_MethodDescriptorProto_client_streaming(const google_protobuf_MethodDescriptorProto *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(1, 1)); }
+UPB_INLINE bool google_protobuf_MethodDescriptorProto_has_server_streaming(const google_protobuf_MethodDescriptorProto *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE bool google_protobuf_MethodDescriptorProto_server_streaming(const google_protobuf_MethodDescriptorProto *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(2, 2)); }
+
+UPB_INLINE void google_protobuf_MethodDescriptorProto_set_name(google_protobuf_MethodDescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 3);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)) = value;
+}
+UPB_INLINE void google_protobuf_MethodDescriptorProto_set_input_type(google_protobuf_MethodDescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 4);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(12, 24)) = value;
+}
+UPB_INLINE void google_protobuf_MethodDescriptorProto_set_output_type(google_protobuf_MethodDescriptorProto *msg, upb_strview value) {
+ _upb_sethas(msg, 5);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(20, 40)) = value;
+}
+UPB_INLINE void google_protobuf_MethodDescriptorProto_set_options(google_protobuf_MethodDescriptorProto *msg, google_protobuf_MethodOptions* value) {
+ _upb_sethas(msg, 6);
+ UPB_FIELD_AT(msg, google_protobuf_MethodOptions*, UPB_SIZE(28, 56)) = value;
+}
+UPB_INLINE struct google_protobuf_MethodOptions* google_protobuf_MethodDescriptorProto_mutable_options(google_protobuf_MethodDescriptorProto *msg, upb_arena *arena) {
+ struct google_protobuf_MethodOptions* sub = (struct google_protobuf_MethodOptions*)google_protobuf_MethodDescriptorProto_options(msg);
+ if (sub == NULL) {
+ sub = (struct google_protobuf_MethodOptions*)upb_msg_new(&google_protobuf_MethodOptions_msginit, arena);
+ if (!sub) return NULL;
+ google_protobuf_MethodDescriptorProto_set_options(msg, sub);
+ }
+ return sub;
+}
+UPB_INLINE void google_protobuf_MethodDescriptorProto_set_client_streaming(google_protobuf_MethodDescriptorProto *msg, bool value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(1, 1)) = value;
+}
+UPB_INLINE void google_protobuf_MethodDescriptorProto_set_server_streaming(google_protobuf_MethodDescriptorProto *msg, bool value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(2, 2)) = value;
+}
+
+
+/* google.protobuf.FileOptions */
+
+UPB_INLINE google_protobuf_FileOptions *google_protobuf_FileOptions_new(upb_arena *arena) {
+ return (google_protobuf_FileOptions *)upb_msg_new(&google_protobuf_FileOptions_msginit, arena);
+}
+UPB_INLINE google_protobuf_FileOptions *google_protobuf_FileOptions_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_FileOptions *ret = google_protobuf_FileOptions_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_FileOptions_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_FileOptions_serialize(const google_protobuf_FileOptions *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_FileOptions_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_FileOptions_has_java_package(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 11); }
+UPB_INLINE upb_strview google_protobuf_FileOptions_java_package(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(28, 32)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_java_outer_classname(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 12); }
+UPB_INLINE upb_strview google_protobuf_FileOptions_java_outer_classname(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(36, 48)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_optimize_for(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE google_protobuf_FileOptions_OptimizeMode google_protobuf_FileOptions_optimize_for(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, google_protobuf_FileOptions_OptimizeMode, UPB_SIZE(8, 8)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_java_multiple_files(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE bool google_protobuf_FileOptions_java_multiple_files(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(16, 16)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_go_package(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 13); }
+UPB_INLINE upb_strview google_protobuf_FileOptions_go_package(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(44, 64)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_cc_generic_services(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 3); }
+UPB_INLINE bool google_protobuf_FileOptions_cc_generic_services(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(17, 17)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_java_generic_services(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 4); }
+UPB_INLINE bool google_protobuf_FileOptions_java_generic_services(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(18, 18)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_py_generic_services(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 5); }
+UPB_INLINE bool google_protobuf_FileOptions_py_generic_services(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(19, 19)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_java_generate_equals_and_hash(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 6); }
+UPB_INLINE bool google_protobuf_FileOptions_java_generate_equals_and_hash(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(20, 20)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_deprecated(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 7); }
+UPB_INLINE bool google_protobuf_FileOptions_deprecated(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(21, 21)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_java_string_check_utf8(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 8); }
+UPB_INLINE bool google_protobuf_FileOptions_java_string_check_utf8(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(22, 22)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_cc_enable_arenas(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 9); }
+UPB_INLINE bool google_protobuf_FileOptions_cc_enable_arenas(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(23, 23)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_objc_class_prefix(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 14); }
+UPB_INLINE upb_strview google_protobuf_FileOptions_objc_class_prefix(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(52, 80)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_csharp_namespace(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 15); }
+UPB_INLINE upb_strview google_protobuf_FileOptions_csharp_namespace(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(60, 96)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_swift_prefix(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 16); }
+UPB_INLINE upb_strview google_protobuf_FileOptions_swift_prefix(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(68, 112)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_php_class_prefix(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 17); }
+UPB_INLINE upb_strview google_protobuf_FileOptions_php_class_prefix(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(76, 128)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_php_namespace(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 18); }
+UPB_INLINE upb_strview google_protobuf_FileOptions_php_namespace(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(84, 144)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_php_generic_services(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 10); }
+UPB_INLINE bool google_protobuf_FileOptions_php_generic_services(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(24, 24)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_php_metadata_namespace(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 19); }
+UPB_INLINE upb_strview google_protobuf_FileOptions_php_metadata_namespace(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(92, 160)); }
+UPB_INLINE bool google_protobuf_FileOptions_has_ruby_package(const google_protobuf_FileOptions *msg) { return _upb_has_field(msg, 20); }
+UPB_INLINE upb_strview google_protobuf_FileOptions_ruby_package(const google_protobuf_FileOptions *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(100, 176)); }
+UPB_INLINE const google_protobuf_UninterpretedOption* const* google_protobuf_FileOptions_uninterpreted_option(const google_protobuf_FileOptions *msg, size_t *len) { return (const google_protobuf_UninterpretedOption* const*)_upb_array_accessor(msg, UPB_SIZE(108, 192), len); }
+
+UPB_INLINE void google_protobuf_FileOptions_set_java_package(google_protobuf_FileOptions *msg, upb_strview value) {
+ _upb_sethas(msg, 11);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(28, 32)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_java_outer_classname(google_protobuf_FileOptions *msg, upb_strview value) {
+ _upb_sethas(msg, 12);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(36, 48)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_optimize_for(google_protobuf_FileOptions *msg, google_protobuf_FileOptions_OptimizeMode value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, google_protobuf_FileOptions_OptimizeMode, UPB_SIZE(8, 8)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_java_multiple_files(google_protobuf_FileOptions *msg, bool value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(16, 16)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_go_package(google_protobuf_FileOptions *msg, upb_strview value) {
+ _upb_sethas(msg, 13);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(44, 64)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_cc_generic_services(google_protobuf_FileOptions *msg, bool value) {
+ _upb_sethas(msg, 3);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(17, 17)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_java_generic_services(google_protobuf_FileOptions *msg, bool value) {
+ _upb_sethas(msg, 4);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(18, 18)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_py_generic_services(google_protobuf_FileOptions *msg, bool value) {
+ _upb_sethas(msg, 5);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(19, 19)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_java_generate_equals_and_hash(google_protobuf_FileOptions *msg, bool value) {
+ _upb_sethas(msg, 6);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(20, 20)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_deprecated(google_protobuf_FileOptions *msg, bool value) {
+ _upb_sethas(msg, 7);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(21, 21)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_java_string_check_utf8(google_protobuf_FileOptions *msg, bool value) {
+ _upb_sethas(msg, 8);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(22, 22)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_cc_enable_arenas(google_protobuf_FileOptions *msg, bool value) {
+ _upb_sethas(msg, 9);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(23, 23)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_objc_class_prefix(google_protobuf_FileOptions *msg, upb_strview value) {
+ _upb_sethas(msg, 14);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(52, 80)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_csharp_namespace(google_protobuf_FileOptions *msg, upb_strview value) {
+ _upb_sethas(msg, 15);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(60, 96)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_swift_prefix(google_protobuf_FileOptions *msg, upb_strview value) {
+ _upb_sethas(msg, 16);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(68, 112)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_php_class_prefix(google_protobuf_FileOptions *msg, upb_strview value) {
+ _upb_sethas(msg, 17);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(76, 128)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_php_namespace(google_protobuf_FileOptions *msg, upb_strview value) {
+ _upb_sethas(msg, 18);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(84, 144)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_php_generic_services(google_protobuf_FileOptions *msg, bool value) {
+ _upb_sethas(msg, 10);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(24, 24)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_php_metadata_namespace(google_protobuf_FileOptions *msg, upb_strview value) {
+ _upb_sethas(msg, 19);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(92, 160)) = value;
+}
+UPB_INLINE void google_protobuf_FileOptions_set_ruby_package(google_protobuf_FileOptions *msg, upb_strview value) {
+ _upb_sethas(msg, 20);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(100, 176)) = value;
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_FileOptions_mutable_uninterpreted_option(google_protobuf_FileOptions *msg, size_t *len) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_mutable_accessor(msg, UPB_SIZE(108, 192), len);
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_FileOptions_resize_uninterpreted_option(google_protobuf_FileOptions *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_resize_accessor(msg, UPB_SIZE(108, 192), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_UninterpretedOption* google_protobuf_FileOptions_add_uninterpreted_option(google_protobuf_FileOptions *msg, upb_arena *arena) {
+ struct google_protobuf_UninterpretedOption* sub = (struct google_protobuf_UninterpretedOption*)upb_msg_new(&google_protobuf_UninterpretedOption_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(108, 192), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+
+
+/* google.protobuf.MessageOptions */
+
+UPB_INLINE google_protobuf_MessageOptions *google_protobuf_MessageOptions_new(upb_arena *arena) {
+ return (google_protobuf_MessageOptions *)upb_msg_new(&google_protobuf_MessageOptions_msginit, arena);
+}
+UPB_INLINE google_protobuf_MessageOptions *google_protobuf_MessageOptions_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_MessageOptions *ret = google_protobuf_MessageOptions_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_MessageOptions_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_MessageOptions_serialize(const google_protobuf_MessageOptions *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_MessageOptions_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_MessageOptions_has_message_set_wire_format(const google_protobuf_MessageOptions *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE bool google_protobuf_MessageOptions_message_set_wire_format(const google_protobuf_MessageOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(1, 1)); }
+UPB_INLINE bool google_protobuf_MessageOptions_has_no_standard_descriptor_accessor(const google_protobuf_MessageOptions *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE bool google_protobuf_MessageOptions_no_standard_descriptor_accessor(const google_protobuf_MessageOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(2, 2)); }
+UPB_INLINE bool google_protobuf_MessageOptions_has_deprecated(const google_protobuf_MessageOptions *msg) { return _upb_has_field(msg, 3); }
+UPB_INLINE bool google_protobuf_MessageOptions_deprecated(const google_protobuf_MessageOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(3, 3)); }
+UPB_INLINE bool google_protobuf_MessageOptions_has_map_entry(const google_protobuf_MessageOptions *msg) { return _upb_has_field(msg, 4); }
+UPB_INLINE bool google_protobuf_MessageOptions_map_entry(const google_protobuf_MessageOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(4, 4)); }
+UPB_INLINE const google_protobuf_UninterpretedOption* const* google_protobuf_MessageOptions_uninterpreted_option(const google_protobuf_MessageOptions *msg, size_t *len) { return (const google_protobuf_UninterpretedOption* const*)_upb_array_accessor(msg, UPB_SIZE(8, 8), len); }
+
+UPB_INLINE void google_protobuf_MessageOptions_set_message_set_wire_format(google_protobuf_MessageOptions *msg, bool value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(1, 1)) = value;
+}
+UPB_INLINE void google_protobuf_MessageOptions_set_no_standard_descriptor_accessor(google_protobuf_MessageOptions *msg, bool value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(2, 2)) = value;
+}
+UPB_INLINE void google_protobuf_MessageOptions_set_deprecated(google_protobuf_MessageOptions *msg, bool value) {
+ _upb_sethas(msg, 3);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(3, 3)) = value;
+}
+UPB_INLINE void google_protobuf_MessageOptions_set_map_entry(google_protobuf_MessageOptions *msg, bool value) {
+ _upb_sethas(msg, 4);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(4, 4)) = value;
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_MessageOptions_mutable_uninterpreted_option(google_protobuf_MessageOptions *msg, size_t *len) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_mutable_accessor(msg, UPB_SIZE(8, 8), len);
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_MessageOptions_resize_uninterpreted_option(google_protobuf_MessageOptions *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_resize_accessor(msg, UPB_SIZE(8, 8), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_UninterpretedOption* google_protobuf_MessageOptions_add_uninterpreted_option(google_protobuf_MessageOptions *msg, upb_arena *arena) {
+ struct google_protobuf_UninterpretedOption* sub = (struct google_protobuf_UninterpretedOption*)upb_msg_new(&google_protobuf_UninterpretedOption_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(8, 8), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+
+
+/* google.protobuf.FieldOptions */
+
+UPB_INLINE google_protobuf_FieldOptions *google_protobuf_FieldOptions_new(upb_arena *arena) {
+ return (google_protobuf_FieldOptions *)upb_msg_new(&google_protobuf_FieldOptions_msginit, arena);
+}
+UPB_INLINE google_protobuf_FieldOptions *google_protobuf_FieldOptions_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_FieldOptions *ret = google_protobuf_FieldOptions_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_FieldOptions_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_FieldOptions_serialize(const google_protobuf_FieldOptions *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_FieldOptions_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_FieldOptions_has_ctype(const google_protobuf_FieldOptions *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE google_protobuf_FieldOptions_CType google_protobuf_FieldOptions_ctype(const google_protobuf_FieldOptions *msg) { return UPB_FIELD_AT(msg, google_protobuf_FieldOptions_CType, UPB_SIZE(8, 8)); }
+UPB_INLINE bool google_protobuf_FieldOptions_has_packed(const google_protobuf_FieldOptions *msg) { return _upb_has_field(msg, 3); }
+UPB_INLINE bool google_protobuf_FieldOptions_packed(const google_protobuf_FieldOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(24, 24)); }
+UPB_INLINE bool google_protobuf_FieldOptions_has_deprecated(const google_protobuf_FieldOptions *msg) { return _upb_has_field(msg, 4); }
+UPB_INLINE bool google_protobuf_FieldOptions_deprecated(const google_protobuf_FieldOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(25, 25)); }
+UPB_INLINE bool google_protobuf_FieldOptions_has_lazy(const google_protobuf_FieldOptions *msg) { return _upb_has_field(msg, 5); }
+UPB_INLINE bool google_protobuf_FieldOptions_lazy(const google_protobuf_FieldOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(26, 26)); }
+UPB_INLINE bool google_protobuf_FieldOptions_has_jstype(const google_protobuf_FieldOptions *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE google_protobuf_FieldOptions_JSType google_protobuf_FieldOptions_jstype(const google_protobuf_FieldOptions *msg) { return UPB_FIELD_AT(msg, google_protobuf_FieldOptions_JSType, UPB_SIZE(16, 16)); }
+UPB_INLINE bool google_protobuf_FieldOptions_has_weak(const google_protobuf_FieldOptions *msg) { return _upb_has_field(msg, 6); }
+UPB_INLINE bool google_protobuf_FieldOptions_weak(const google_protobuf_FieldOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(27, 27)); }
+UPB_INLINE const google_protobuf_UninterpretedOption* const* google_protobuf_FieldOptions_uninterpreted_option(const google_protobuf_FieldOptions *msg, size_t *len) { return (const google_protobuf_UninterpretedOption* const*)_upb_array_accessor(msg, UPB_SIZE(28, 32), len); }
+
+UPB_INLINE void google_protobuf_FieldOptions_set_ctype(google_protobuf_FieldOptions *msg, google_protobuf_FieldOptions_CType value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, google_protobuf_FieldOptions_CType, UPB_SIZE(8, 8)) = value;
+}
+UPB_INLINE void google_protobuf_FieldOptions_set_packed(google_protobuf_FieldOptions *msg, bool value) {
+ _upb_sethas(msg, 3);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(24, 24)) = value;
+}
+UPB_INLINE void google_protobuf_FieldOptions_set_deprecated(google_protobuf_FieldOptions *msg, bool value) {
+ _upb_sethas(msg, 4);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(25, 25)) = value;
+}
+UPB_INLINE void google_protobuf_FieldOptions_set_lazy(google_protobuf_FieldOptions *msg, bool value) {
+ _upb_sethas(msg, 5);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(26, 26)) = value;
+}
+UPB_INLINE void google_protobuf_FieldOptions_set_jstype(google_protobuf_FieldOptions *msg, google_protobuf_FieldOptions_JSType value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, google_protobuf_FieldOptions_JSType, UPB_SIZE(16, 16)) = value;
+}
+UPB_INLINE void google_protobuf_FieldOptions_set_weak(google_protobuf_FieldOptions *msg, bool value) {
+ _upb_sethas(msg, 6);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(27, 27)) = value;
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_FieldOptions_mutable_uninterpreted_option(google_protobuf_FieldOptions *msg, size_t *len) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_mutable_accessor(msg, UPB_SIZE(28, 32), len);
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_FieldOptions_resize_uninterpreted_option(google_protobuf_FieldOptions *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_resize_accessor(msg, UPB_SIZE(28, 32), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_UninterpretedOption* google_protobuf_FieldOptions_add_uninterpreted_option(google_protobuf_FieldOptions *msg, upb_arena *arena) {
+ struct google_protobuf_UninterpretedOption* sub = (struct google_protobuf_UninterpretedOption*)upb_msg_new(&google_protobuf_UninterpretedOption_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(28, 32), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+
+
+/* google.protobuf.OneofOptions */
+
+UPB_INLINE google_protobuf_OneofOptions *google_protobuf_OneofOptions_new(upb_arena *arena) {
+ return (google_protobuf_OneofOptions *)upb_msg_new(&google_protobuf_OneofOptions_msginit, arena);
+}
+UPB_INLINE google_protobuf_OneofOptions *google_protobuf_OneofOptions_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_OneofOptions *ret = google_protobuf_OneofOptions_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_OneofOptions_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_OneofOptions_serialize(const google_protobuf_OneofOptions *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_OneofOptions_msginit, arena, len);
+}
+
+UPB_INLINE const google_protobuf_UninterpretedOption* const* google_protobuf_OneofOptions_uninterpreted_option(const google_protobuf_OneofOptions *msg, size_t *len) { return (const google_protobuf_UninterpretedOption* const*)_upb_array_accessor(msg, UPB_SIZE(0, 0), len); }
+
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_OneofOptions_mutable_uninterpreted_option(google_protobuf_OneofOptions *msg, size_t *len) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_mutable_accessor(msg, UPB_SIZE(0, 0), len);
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_OneofOptions_resize_uninterpreted_option(google_protobuf_OneofOptions *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_resize_accessor(msg, UPB_SIZE(0, 0), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_UninterpretedOption* google_protobuf_OneofOptions_add_uninterpreted_option(google_protobuf_OneofOptions *msg, upb_arena *arena) {
+ struct google_protobuf_UninterpretedOption* sub = (struct google_protobuf_UninterpretedOption*)upb_msg_new(&google_protobuf_UninterpretedOption_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(0, 0), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+
+
+/* google.protobuf.EnumOptions */
+
+UPB_INLINE google_protobuf_EnumOptions *google_protobuf_EnumOptions_new(upb_arena *arena) {
+ return (google_protobuf_EnumOptions *)upb_msg_new(&google_protobuf_EnumOptions_msginit, arena);
+}
+UPB_INLINE google_protobuf_EnumOptions *google_protobuf_EnumOptions_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_EnumOptions *ret = google_protobuf_EnumOptions_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_EnumOptions_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_EnumOptions_serialize(const google_protobuf_EnumOptions *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_EnumOptions_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_EnumOptions_has_allow_alias(const google_protobuf_EnumOptions *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE bool google_protobuf_EnumOptions_allow_alias(const google_protobuf_EnumOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(1, 1)); }
+UPB_INLINE bool google_protobuf_EnumOptions_has_deprecated(const google_protobuf_EnumOptions *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE bool google_protobuf_EnumOptions_deprecated(const google_protobuf_EnumOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(2, 2)); }
+UPB_INLINE const google_protobuf_UninterpretedOption* const* google_protobuf_EnumOptions_uninterpreted_option(const google_protobuf_EnumOptions *msg, size_t *len) { return (const google_protobuf_UninterpretedOption* const*)_upb_array_accessor(msg, UPB_SIZE(4, 8), len); }
+
+UPB_INLINE void google_protobuf_EnumOptions_set_allow_alias(google_protobuf_EnumOptions *msg, bool value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(1, 1)) = value;
+}
+UPB_INLINE void google_protobuf_EnumOptions_set_deprecated(google_protobuf_EnumOptions *msg, bool value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(2, 2)) = value;
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_EnumOptions_mutable_uninterpreted_option(google_protobuf_EnumOptions *msg, size_t *len) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_mutable_accessor(msg, UPB_SIZE(4, 8), len);
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_EnumOptions_resize_uninterpreted_option(google_protobuf_EnumOptions *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_resize_accessor(msg, UPB_SIZE(4, 8), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_UninterpretedOption* google_protobuf_EnumOptions_add_uninterpreted_option(google_protobuf_EnumOptions *msg, upb_arena *arena) {
+ struct google_protobuf_UninterpretedOption* sub = (struct google_protobuf_UninterpretedOption*)upb_msg_new(&google_protobuf_UninterpretedOption_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(4, 8), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+
+
+/* google.protobuf.EnumValueOptions */
+
+UPB_INLINE google_protobuf_EnumValueOptions *google_protobuf_EnumValueOptions_new(upb_arena *arena) {
+ return (google_protobuf_EnumValueOptions *)upb_msg_new(&google_protobuf_EnumValueOptions_msginit, arena);
+}
+UPB_INLINE google_protobuf_EnumValueOptions *google_protobuf_EnumValueOptions_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_EnumValueOptions *ret = google_protobuf_EnumValueOptions_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_EnumValueOptions_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_EnumValueOptions_serialize(const google_protobuf_EnumValueOptions *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_EnumValueOptions_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_EnumValueOptions_has_deprecated(const google_protobuf_EnumValueOptions *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE bool google_protobuf_EnumValueOptions_deprecated(const google_protobuf_EnumValueOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(1, 1)); }
+UPB_INLINE const google_protobuf_UninterpretedOption* const* google_protobuf_EnumValueOptions_uninterpreted_option(const google_protobuf_EnumValueOptions *msg, size_t *len) { return (const google_protobuf_UninterpretedOption* const*)_upb_array_accessor(msg, UPB_SIZE(4, 8), len); }
+
+UPB_INLINE void google_protobuf_EnumValueOptions_set_deprecated(google_protobuf_EnumValueOptions *msg, bool value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(1, 1)) = value;
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_EnumValueOptions_mutable_uninterpreted_option(google_protobuf_EnumValueOptions *msg, size_t *len) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_mutable_accessor(msg, UPB_SIZE(4, 8), len);
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_EnumValueOptions_resize_uninterpreted_option(google_protobuf_EnumValueOptions *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_resize_accessor(msg, UPB_SIZE(4, 8), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_UninterpretedOption* google_protobuf_EnumValueOptions_add_uninterpreted_option(google_protobuf_EnumValueOptions *msg, upb_arena *arena) {
+ struct google_protobuf_UninterpretedOption* sub = (struct google_protobuf_UninterpretedOption*)upb_msg_new(&google_protobuf_UninterpretedOption_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(4, 8), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+
+
+/* google.protobuf.ServiceOptions */
+
+UPB_INLINE google_protobuf_ServiceOptions *google_protobuf_ServiceOptions_new(upb_arena *arena) {
+ return (google_protobuf_ServiceOptions *)upb_msg_new(&google_protobuf_ServiceOptions_msginit, arena);
+}
+UPB_INLINE google_protobuf_ServiceOptions *google_protobuf_ServiceOptions_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_ServiceOptions *ret = google_protobuf_ServiceOptions_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_ServiceOptions_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_ServiceOptions_serialize(const google_protobuf_ServiceOptions *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_ServiceOptions_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_ServiceOptions_has_deprecated(const google_protobuf_ServiceOptions *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE bool google_protobuf_ServiceOptions_deprecated(const google_protobuf_ServiceOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(1, 1)); }
+UPB_INLINE const google_protobuf_UninterpretedOption* const* google_protobuf_ServiceOptions_uninterpreted_option(const google_protobuf_ServiceOptions *msg, size_t *len) { return (const google_protobuf_UninterpretedOption* const*)_upb_array_accessor(msg, UPB_SIZE(4, 8), len); }
+
+UPB_INLINE void google_protobuf_ServiceOptions_set_deprecated(google_protobuf_ServiceOptions *msg, bool value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(1, 1)) = value;
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_ServiceOptions_mutable_uninterpreted_option(google_protobuf_ServiceOptions *msg, size_t *len) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_mutable_accessor(msg, UPB_SIZE(4, 8), len);
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_ServiceOptions_resize_uninterpreted_option(google_protobuf_ServiceOptions *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_resize_accessor(msg, UPB_SIZE(4, 8), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_UninterpretedOption* google_protobuf_ServiceOptions_add_uninterpreted_option(google_protobuf_ServiceOptions *msg, upb_arena *arena) {
+ struct google_protobuf_UninterpretedOption* sub = (struct google_protobuf_UninterpretedOption*)upb_msg_new(&google_protobuf_UninterpretedOption_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(4, 8), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+
+
+/* google.protobuf.MethodOptions */
+
+UPB_INLINE google_protobuf_MethodOptions *google_protobuf_MethodOptions_new(upb_arena *arena) {
+ return (google_protobuf_MethodOptions *)upb_msg_new(&google_protobuf_MethodOptions_msginit, arena);
+}
+UPB_INLINE google_protobuf_MethodOptions *google_protobuf_MethodOptions_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_MethodOptions *ret = google_protobuf_MethodOptions_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_MethodOptions_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_MethodOptions_serialize(const google_protobuf_MethodOptions *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_MethodOptions_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_MethodOptions_has_deprecated(const google_protobuf_MethodOptions *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE bool google_protobuf_MethodOptions_deprecated(const google_protobuf_MethodOptions *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(16, 16)); }
+UPB_INLINE bool google_protobuf_MethodOptions_has_idempotency_level(const google_protobuf_MethodOptions *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE google_protobuf_MethodOptions_IdempotencyLevel google_protobuf_MethodOptions_idempotency_level(const google_protobuf_MethodOptions *msg) { return UPB_FIELD_AT(msg, google_protobuf_MethodOptions_IdempotencyLevel, UPB_SIZE(8, 8)); }
+UPB_INLINE const google_protobuf_UninterpretedOption* const* google_protobuf_MethodOptions_uninterpreted_option(const google_protobuf_MethodOptions *msg, size_t *len) { return (const google_protobuf_UninterpretedOption* const*)_upb_array_accessor(msg, UPB_SIZE(20, 24), len); }
+
+UPB_INLINE void google_protobuf_MethodOptions_set_deprecated(google_protobuf_MethodOptions *msg, bool value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(16, 16)) = value;
+}
+UPB_INLINE void google_protobuf_MethodOptions_set_idempotency_level(google_protobuf_MethodOptions *msg, google_protobuf_MethodOptions_IdempotencyLevel value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, google_protobuf_MethodOptions_IdempotencyLevel, UPB_SIZE(8, 8)) = value;
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_MethodOptions_mutable_uninterpreted_option(google_protobuf_MethodOptions *msg, size_t *len) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_mutable_accessor(msg, UPB_SIZE(20, 24), len);
+}
+UPB_INLINE google_protobuf_UninterpretedOption** google_protobuf_MethodOptions_resize_uninterpreted_option(google_protobuf_MethodOptions *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_UninterpretedOption**)_upb_array_resize_accessor(msg, UPB_SIZE(20, 24), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_UninterpretedOption* google_protobuf_MethodOptions_add_uninterpreted_option(google_protobuf_MethodOptions *msg, upb_arena *arena) {
+ struct google_protobuf_UninterpretedOption* sub = (struct google_protobuf_UninterpretedOption*)upb_msg_new(&google_protobuf_UninterpretedOption_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(20, 24), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+
+
+/* google.protobuf.UninterpretedOption */
+
+UPB_INLINE google_protobuf_UninterpretedOption *google_protobuf_UninterpretedOption_new(upb_arena *arena) {
+ return (google_protobuf_UninterpretedOption *)upb_msg_new(&google_protobuf_UninterpretedOption_msginit, arena);
+}
+UPB_INLINE google_protobuf_UninterpretedOption *google_protobuf_UninterpretedOption_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_UninterpretedOption *ret = google_protobuf_UninterpretedOption_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_UninterpretedOption_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_UninterpretedOption_serialize(const google_protobuf_UninterpretedOption *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_UninterpretedOption_msginit, arena, len);
+}
+
+UPB_INLINE const google_protobuf_UninterpretedOption_NamePart* const* google_protobuf_UninterpretedOption_name(const google_protobuf_UninterpretedOption *msg, size_t *len) { return (const google_protobuf_UninterpretedOption_NamePart* const*)_upb_array_accessor(msg, UPB_SIZE(56, 80), len); }
+UPB_INLINE bool google_protobuf_UninterpretedOption_has_identifier_value(const google_protobuf_UninterpretedOption *msg) { return _upb_has_field(msg, 4); }
+UPB_INLINE upb_strview google_protobuf_UninterpretedOption_identifier_value(const google_protobuf_UninterpretedOption *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(32, 32)); }
+UPB_INLINE bool google_protobuf_UninterpretedOption_has_positive_int_value(const google_protobuf_UninterpretedOption *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE uint64_t google_protobuf_UninterpretedOption_positive_int_value(const google_protobuf_UninterpretedOption *msg) { return UPB_FIELD_AT(msg, uint64_t, UPB_SIZE(8, 8)); }
+UPB_INLINE bool google_protobuf_UninterpretedOption_has_negative_int_value(const google_protobuf_UninterpretedOption *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE int64_t google_protobuf_UninterpretedOption_negative_int_value(const google_protobuf_UninterpretedOption *msg) { return UPB_FIELD_AT(msg, int64_t, UPB_SIZE(16, 16)); }
+UPB_INLINE bool google_protobuf_UninterpretedOption_has_double_value(const google_protobuf_UninterpretedOption *msg) { return _upb_has_field(msg, 3); }
+UPB_INLINE double google_protobuf_UninterpretedOption_double_value(const google_protobuf_UninterpretedOption *msg) { return UPB_FIELD_AT(msg, double, UPB_SIZE(24, 24)); }
+UPB_INLINE bool google_protobuf_UninterpretedOption_has_string_value(const google_protobuf_UninterpretedOption *msg) { return _upb_has_field(msg, 5); }
+UPB_INLINE upb_strview google_protobuf_UninterpretedOption_string_value(const google_protobuf_UninterpretedOption *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(40, 48)); }
+UPB_INLINE bool google_protobuf_UninterpretedOption_has_aggregate_value(const google_protobuf_UninterpretedOption *msg) { return _upb_has_field(msg, 6); }
+UPB_INLINE upb_strview google_protobuf_UninterpretedOption_aggregate_value(const google_protobuf_UninterpretedOption *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(48, 64)); }
+
+UPB_INLINE google_protobuf_UninterpretedOption_NamePart** google_protobuf_UninterpretedOption_mutable_name(google_protobuf_UninterpretedOption *msg, size_t *len) {
+ return (google_protobuf_UninterpretedOption_NamePart**)_upb_array_mutable_accessor(msg, UPB_SIZE(56, 80), len);
+}
+UPB_INLINE google_protobuf_UninterpretedOption_NamePart** google_protobuf_UninterpretedOption_resize_name(google_protobuf_UninterpretedOption *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_UninterpretedOption_NamePart**)_upb_array_resize_accessor(msg, UPB_SIZE(56, 80), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_UninterpretedOption_NamePart* google_protobuf_UninterpretedOption_add_name(google_protobuf_UninterpretedOption *msg, upb_arena *arena) {
+ struct google_protobuf_UninterpretedOption_NamePart* sub = (struct google_protobuf_UninterpretedOption_NamePart*)upb_msg_new(&google_protobuf_UninterpretedOption_NamePart_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(56, 80), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+UPB_INLINE void google_protobuf_UninterpretedOption_set_identifier_value(google_protobuf_UninterpretedOption *msg, upb_strview value) {
+ _upb_sethas(msg, 4);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(32, 32)) = value;
+}
+UPB_INLINE void google_protobuf_UninterpretedOption_set_positive_int_value(google_protobuf_UninterpretedOption *msg, uint64_t value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, uint64_t, UPB_SIZE(8, 8)) = value;
+}
+UPB_INLINE void google_protobuf_UninterpretedOption_set_negative_int_value(google_protobuf_UninterpretedOption *msg, int64_t value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, int64_t, UPB_SIZE(16, 16)) = value;
+}
+UPB_INLINE void google_protobuf_UninterpretedOption_set_double_value(google_protobuf_UninterpretedOption *msg, double value) {
+ _upb_sethas(msg, 3);
+ UPB_FIELD_AT(msg, double, UPB_SIZE(24, 24)) = value;
+}
+UPB_INLINE void google_protobuf_UninterpretedOption_set_string_value(google_protobuf_UninterpretedOption *msg, upb_strview value) {
+ _upb_sethas(msg, 5);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(40, 48)) = value;
+}
+UPB_INLINE void google_protobuf_UninterpretedOption_set_aggregate_value(google_protobuf_UninterpretedOption *msg, upb_strview value) {
+ _upb_sethas(msg, 6);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(48, 64)) = value;
+}
+
+
+/* google.protobuf.UninterpretedOption.NamePart */
+
+UPB_INLINE google_protobuf_UninterpretedOption_NamePart *google_protobuf_UninterpretedOption_NamePart_new(upb_arena *arena) {
+ return (google_protobuf_UninterpretedOption_NamePart *)upb_msg_new(&google_protobuf_UninterpretedOption_NamePart_msginit, arena);
+}
+UPB_INLINE google_protobuf_UninterpretedOption_NamePart *google_protobuf_UninterpretedOption_NamePart_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_UninterpretedOption_NamePart *ret = google_protobuf_UninterpretedOption_NamePart_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_UninterpretedOption_NamePart_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_UninterpretedOption_NamePart_serialize(const google_protobuf_UninterpretedOption_NamePart *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_UninterpretedOption_NamePart_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_UninterpretedOption_NamePart_has_name_part(const google_protobuf_UninterpretedOption_NamePart *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE upb_strview google_protobuf_UninterpretedOption_NamePart_name_part(const google_protobuf_UninterpretedOption_NamePart *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)); }
+UPB_INLINE bool google_protobuf_UninterpretedOption_NamePart_has_is_extension(const google_protobuf_UninterpretedOption_NamePart *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE bool google_protobuf_UninterpretedOption_NamePart_is_extension(const google_protobuf_UninterpretedOption_NamePart *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(1, 1)); }
+
+UPB_INLINE void google_protobuf_UninterpretedOption_NamePart_set_name_part(google_protobuf_UninterpretedOption_NamePart *msg, upb_strview value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)) = value;
+}
+UPB_INLINE void google_protobuf_UninterpretedOption_NamePart_set_is_extension(google_protobuf_UninterpretedOption_NamePart *msg, bool value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(1, 1)) = value;
+}
+
+
+/* google.protobuf.SourceCodeInfo */
+
+UPB_INLINE google_protobuf_SourceCodeInfo *google_protobuf_SourceCodeInfo_new(upb_arena *arena) {
+ return (google_protobuf_SourceCodeInfo *)upb_msg_new(&google_protobuf_SourceCodeInfo_msginit, arena);
+}
+UPB_INLINE google_protobuf_SourceCodeInfo *google_protobuf_SourceCodeInfo_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_SourceCodeInfo *ret = google_protobuf_SourceCodeInfo_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_SourceCodeInfo_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_SourceCodeInfo_serialize(const google_protobuf_SourceCodeInfo *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_SourceCodeInfo_msginit, arena, len);
+}
+
+UPB_INLINE const google_protobuf_SourceCodeInfo_Location* const* google_protobuf_SourceCodeInfo_location(const google_protobuf_SourceCodeInfo *msg, size_t *len) { return (const google_protobuf_SourceCodeInfo_Location* const*)_upb_array_accessor(msg, UPB_SIZE(0, 0), len); }
+
+UPB_INLINE google_protobuf_SourceCodeInfo_Location** google_protobuf_SourceCodeInfo_mutable_location(google_protobuf_SourceCodeInfo *msg, size_t *len) {
+ return (google_protobuf_SourceCodeInfo_Location**)_upb_array_mutable_accessor(msg, UPB_SIZE(0, 0), len);
+}
+UPB_INLINE google_protobuf_SourceCodeInfo_Location** google_protobuf_SourceCodeInfo_resize_location(google_protobuf_SourceCodeInfo *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_SourceCodeInfo_Location**)_upb_array_resize_accessor(msg, UPB_SIZE(0, 0), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_SourceCodeInfo_Location* google_protobuf_SourceCodeInfo_add_location(google_protobuf_SourceCodeInfo *msg, upb_arena *arena) {
+ struct google_protobuf_SourceCodeInfo_Location* sub = (struct google_protobuf_SourceCodeInfo_Location*)upb_msg_new(&google_protobuf_SourceCodeInfo_Location_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(0, 0), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+
+
+/* google.protobuf.SourceCodeInfo.Location */
+
+UPB_INLINE google_protobuf_SourceCodeInfo_Location *google_protobuf_SourceCodeInfo_Location_new(upb_arena *arena) {
+ return (google_protobuf_SourceCodeInfo_Location *)upb_msg_new(&google_protobuf_SourceCodeInfo_Location_msginit, arena);
+}
+UPB_INLINE google_protobuf_SourceCodeInfo_Location *google_protobuf_SourceCodeInfo_Location_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_SourceCodeInfo_Location *ret = google_protobuf_SourceCodeInfo_Location_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_SourceCodeInfo_Location_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_SourceCodeInfo_Location_serialize(const google_protobuf_SourceCodeInfo_Location *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_SourceCodeInfo_Location_msginit, arena, len);
+}
+
+UPB_INLINE int32_t const* google_protobuf_SourceCodeInfo_Location_path(const google_protobuf_SourceCodeInfo_Location *msg, size_t *len) { return (int32_t const*)_upb_array_accessor(msg, UPB_SIZE(20, 40), len); }
+UPB_INLINE int32_t const* google_protobuf_SourceCodeInfo_Location_span(const google_protobuf_SourceCodeInfo_Location *msg, size_t *len) { return (int32_t const*)_upb_array_accessor(msg, UPB_SIZE(24, 48), len); }
+UPB_INLINE bool google_protobuf_SourceCodeInfo_Location_has_leading_comments(const google_protobuf_SourceCodeInfo_Location *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE upb_strview google_protobuf_SourceCodeInfo_Location_leading_comments(const google_protobuf_SourceCodeInfo_Location *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)); }
+UPB_INLINE bool google_protobuf_SourceCodeInfo_Location_has_trailing_comments(const google_protobuf_SourceCodeInfo_Location *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE upb_strview google_protobuf_SourceCodeInfo_Location_trailing_comments(const google_protobuf_SourceCodeInfo_Location *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(12, 24)); }
+UPB_INLINE upb_strview const* google_protobuf_SourceCodeInfo_Location_leading_detached_comments(const google_protobuf_SourceCodeInfo_Location *msg, size_t *len) { return (upb_strview const*)_upb_array_accessor(msg, UPB_SIZE(28, 56), len); }
+
+UPB_INLINE int32_t* google_protobuf_SourceCodeInfo_Location_mutable_path(google_protobuf_SourceCodeInfo_Location *msg, size_t *len) {
+ return (int32_t*)_upb_array_mutable_accessor(msg, UPB_SIZE(20, 40), len);
+}
+UPB_INLINE int32_t* google_protobuf_SourceCodeInfo_Location_resize_path(google_protobuf_SourceCodeInfo_Location *msg, size_t len, upb_arena *arena) {
+ return (int32_t*)_upb_array_resize_accessor(msg, UPB_SIZE(20, 40), len, UPB_SIZE(4, 4), UPB_TYPE_INT32, arena);
+}
+UPB_INLINE bool google_protobuf_SourceCodeInfo_Location_add_path(google_protobuf_SourceCodeInfo_Location *msg, int32_t val, upb_arena *arena) {
+ return _upb_array_append_accessor(
+ msg, UPB_SIZE(20, 40), UPB_SIZE(4, 4), UPB_TYPE_INT32, &val, arena);
+}
+UPB_INLINE int32_t* google_protobuf_SourceCodeInfo_Location_mutable_span(google_protobuf_SourceCodeInfo_Location *msg, size_t *len) {
+ return (int32_t*)_upb_array_mutable_accessor(msg, UPB_SIZE(24, 48), len);
+}
+UPB_INLINE int32_t* google_protobuf_SourceCodeInfo_Location_resize_span(google_protobuf_SourceCodeInfo_Location *msg, size_t len, upb_arena *arena) {
+ return (int32_t*)_upb_array_resize_accessor(msg, UPB_SIZE(24, 48), len, UPB_SIZE(4, 4), UPB_TYPE_INT32, arena);
+}
+UPB_INLINE bool google_protobuf_SourceCodeInfo_Location_add_span(google_protobuf_SourceCodeInfo_Location *msg, int32_t val, upb_arena *arena) {
+ return _upb_array_append_accessor(
+ msg, UPB_SIZE(24, 48), UPB_SIZE(4, 4), UPB_TYPE_INT32, &val, arena);
+}
+UPB_INLINE void google_protobuf_SourceCodeInfo_Location_set_leading_comments(google_protobuf_SourceCodeInfo_Location *msg, upb_strview value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(4, 8)) = value;
+}
+UPB_INLINE void google_protobuf_SourceCodeInfo_Location_set_trailing_comments(google_protobuf_SourceCodeInfo_Location *msg, upb_strview value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(12, 24)) = value;
+}
+UPB_INLINE upb_strview* google_protobuf_SourceCodeInfo_Location_mutable_leading_detached_comments(google_protobuf_SourceCodeInfo_Location *msg, size_t *len) {
+ return (upb_strview*)_upb_array_mutable_accessor(msg, UPB_SIZE(28, 56), len);
+}
+UPB_INLINE upb_strview* google_protobuf_SourceCodeInfo_Location_resize_leading_detached_comments(google_protobuf_SourceCodeInfo_Location *msg, size_t len, upb_arena *arena) {
+ return (upb_strview*)_upb_array_resize_accessor(msg, UPB_SIZE(28, 56), len, UPB_SIZE(8, 16), UPB_TYPE_STRING, arena);
+}
+UPB_INLINE bool google_protobuf_SourceCodeInfo_Location_add_leading_detached_comments(google_protobuf_SourceCodeInfo_Location *msg, upb_strview val, upb_arena *arena) {
+ return _upb_array_append_accessor(
+ msg, UPB_SIZE(28, 56), UPB_SIZE(8, 16), UPB_TYPE_STRING, &val, arena);
+}
+
+
+/* google.protobuf.GeneratedCodeInfo */
+
+UPB_INLINE google_protobuf_GeneratedCodeInfo *google_protobuf_GeneratedCodeInfo_new(upb_arena *arena) {
+ return (google_protobuf_GeneratedCodeInfo *)upb_msg_new(&google_protobuf_GeneratedCodeInfo_msginit, arena);
+}
+UPB_INLINE google_protobuf_GeneratedCodeInfo *google_protobuf_GeneratedCodeInfo_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_GeneratedCodeInfo *ret = google_protobuf_GeneratedCodeInfo_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_GeneratedCodeInfo_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_GeneratedCodeInfo_serialize(const google_protobuf_GeneratedCodeInfo *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_GeneratedCodeInfo_msginit, arena, len);
+}
+
+UPB_INLINE const google_protobuf_GeneratedCodeInfo_Annotation* const* google_protobuf_GeneratedCodeInfo_annotation(const google_protobuf_GeneratedCodeInfo *msg, size_t *len) { return (const google_protobuf_GeneratedCodeInfo_Annotation* const*)_upb_array_accessor(msg, UPB_SIZE(0, 0), len); }
+
+UPB_INLINE google_protobuf_GeneratedCodeInfo_Annotation** google_protobuf_GeneratedCodeInfo_mutable_annotation(google_protobuf_GeneratedCodeInfo *msg, size_t *len) {
+ return (google_protobuf_GeneratedCodeInfo_Annotation**)_upb_array_mutable_accessor(msg, UPB_SIZE(0, 0), len);
+}
+UPB_INLINE google_protobuf_GeneratedCodeInfo_Annotation** google_protobuf_GeneratedCodeInfo_resize_annotation(google_protobuf_GeneratedCodeInfo *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_GeneratedCodeInfo_Annotation**)_upb_array_resize_accessor(msg, UPB_SIZE(0, 0), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_GeneratedCodeInfo_Annotation* google_protobuf_GeneratedCodeInfo_add_annotation(google_protobuf_GeneratedCodeInfo *msg, upb_arena *arena) {
+ struct google_protobuf_GeneratedCodeInfo_Annotation* sub = (struct google_protobuf_GeneratedCodeInfo_Annotation*)upb_msg_new(&google_protobuf_GeneratedCodeInfo_Annotation_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(0, 0), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+
+
+/* google.protobuf.GeneratedCodeInfo.Annotation */
+
+UPB_INLINE google_protobuf_GeneratedCodeInfo_Annotation *google_protobuf_GeneratedCodeInfo_Annotation_new(upb_arena *arena) {
+ return (google_protobuf_GeneratedCodeInfo_Annotation *)upb_msg_new(&google_protobuf_GeneratedCodeInfo_Annotation_msginit, arena);
+}
+UPB_INLINE google_protobuf_GeneratedCodeInfo_Annotation *google_protobuf_GeneratedCodeInfo_Annotation_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_GeneratedCodeInfo_Annotation *ret = google_protobuf_GeneratedCodeInfo_Annotation_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_GeneratedCodeInfo_Annotation_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_GeneratedCodeInfo_Annotation_serialize(const google_protobuf_GeneratedCodeInfo_Annotation *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_GeneratedCodeInfo_Annotation_msginit, arena, len);
+}
+
+UPB_INLINE int32_t const* google_protobuf_GeneratedCodeInfo_Annotation_path(const google_protobuf_GeneratedCodeInfo_Annotation *msg, size_t *len) { return (int32_t const*)_upb_array_accessor(msg, UPB_SIZE(20, 32), len); }
+UPB_INLINE bool google_protobuf_GeneratedCodeInfo_Annotation_has_source_file(const google_protobuf_GeneratedCodeInfo_Annotation *msg) { return _upb_has_field(msg, 3); }
+UPB_INLINE upb_strview google_protobuf_GeneratedCodeInfo_Annotation_source_file(const google_protobuf_GeneratedCodeInfo_Annotation *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(12, 16)); }
+UPB_INLINE bool google_protobuf_GeneratedCodeInfo_Annotation_has_begin(const google_protobuf_GeneratedCodeInfo_Annotation *msg) { return _upb_has_field(msg, 1); }
+UPB_INLINE int32_t google_protobuf_GeneratedCodeInfo_Annotation_begin(const google_protobuf_GeneratedCodeInfo_Annotation *msg) { return UPB_FIELD_AT(msg, int32_t, UPB_SIZE(4, 4)); }
+UPB_INLINE bool google_protobuf_GeneratedCodeInfo_Annotation_has_end(const google_protobuf_GeneratedCodeInfo_Annotation *msg) { return _upb_has_field(msg, 2); }
+UPB_INLINE int32_t google_protobuf_GeneratedCodeInfo_Annotation_end(const google_protobuf_GeneratedCodeInfo_Annotation *msg) { return UPB_FIELD_AT(msg, int32_t, UPB_SIZE(8, 8)); }
+
+UPB_INLINE int32_t* google_protobuf_GeneratedCodeInfo_Annotation_mutable_path(google_protobuf_GeneratedCodeInfo_Annotation *msg, size_t *len) {
+ return (int32_t*)_upb_array_mutable_accessor(msg, UPB_SIZE(20, 32), len);
+}
+UPB_INLINE int32_t* google_protobuf_GeneratedCodeInfo_Annotation_resize_path(google_protobuf_GeneratedCodeInfo_Annotation *msg, size_t len, upb_arena *arena) {
+ return (int32_t*)_upb_array_resize_accessor(msg, UPB_SIZE(20, 32), len, UPB_SIZE(4, 4), UPB_TYPE_INT32, arena);
+}
+UPB_INLINE bool google_protobuf_GeneratedCodeInfo_Annotation_add_path(google_protobuf_GeneratedCodeInfo_Annotation *msg, int32_t val, upb_arena *arena) {
+ return _upb_array_append_accessor(
+ msg, UPB_SIZE(20, 32), UPB_SIZE(4, 4), UPB_TYPE_INT32, &val, arena);
+}
+UPB_INLINE void google_protobuf_GeneratedCodeInfo_Annotation_set_source_file(google_protobuf_GeneratedCodeInfo_Annotation *msg, upb_strview value) {
+ _upb_sethas(msg, 3);
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(12, 16)) = value;
+}
+UPB_INLINE void google_protobuf_GeneratedCodeInfo_Annotation_set_begin(google_protobuf_GeneratedCodeInfo_Annotation *msg, int32_t value) {
+ _upb_sethas(msg, 1);
+ UPB_FIELD_AT(msg, int32_t, UPB_SIZE(4, 4)) = value;
+}
+UPB_INLINE void google_protobuf_GeneratedCodeInfo_Annotation_set_end(google_protobuf_GeneratedCodeInfo_Annotation *msg, int32_t value) {
+ _upb_sethas(msg, 2);
+ UPB_FIELD_AT(msg, int32_t, UPB_SIZE(8, 8)) = value;
+}
+
+
+#ifdef __cplusplus
+} /* extern "C" */
+#endif
+
+#include "upb/port_undef.inc"
+
+#endif /* GOOGLE_PROTOBUF_DESCRIPTOR_PROTO_UPB_H_ */
diff --git a/src/core/ext/upb-generated/google/protobuf/duration.upb.c b/src/core/ext/upb-generated/google/protobuf/duration.upb.c
new file mode 100644
index 00000000000..7384c06cf27
--- /dev/null
+++ b/src/core/ext/upb-generated/google/protobuf/duration.upb.c
@@ -0,0 +1,27 @@
+/* This file was generated by upbc (the upb compiler) from the input
+ * file:
+ *
+ * google/protobuf/duration.proto
+ *
+ * Do not edit -- your changes will be discarded when the file is
+ * regenerated. */
+
+#include <stddef.h>
+#include "upb/msg.h"
+#include "google/protobuf/duration.upb.h"
+
+#include "upb/port_def.inc"
+
+static const upb_msglayout_field google_protobuf_Duration__fields[2] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 3, 1},
+ {2, UPB_SIZE(8, 8), 0, 0, 5, 1},
+};
+
+const upb_msglayout google_protobuf_Duration_msginit = {
+ NULL,
+ &google_protobuf_Duration__fields[0],
+ UPB_SIZE(16, 16), 2, false,
+};
+
+#include "upb/port_undef.inc"
+
diff --git a/src/core/ext/upb-generated/google/protobuf/duration.upb.h b/src/core/ext/upb-generated/google/protobuf/duration.upb.h
new file mode 100644
index 00000000000..871d67dcbb5
--- /dev/null
+++ b/src/core/ext/upb-generated/google/protobuf/duration.upb.h
@@ -0,0 +1,59 @@
+/* This file was generated by upbc (the upb compiler) from the input
+ * file:
+ *
+ * google/protobuf/duration.proto
+ *
+ * Do not edit -- your changes will be discarded when the file is
+ * regenerated. */
+
+#ifndef GOOGLE_PROTOBUF_DURATION_PROTO_UPB_H_
+#define GOOGLE_PROTOBUF_DURATION_PROTO_UPB_H_
+
+#include "upb/generated_util.h"
+
+#include "upb/msg.h"
+
+#include "upb/decode.h"
+#include "upb/encode.h"
+#include "upb/port_def.inc"
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct google_protobuf_Duration;
+typedef struct google_protobuf_Duration google_protobuf_Duration;
+extern const upb_msglayout google_protobuf_Duration_msginit;
+
+/* Enums */
+
+/* google.protobuf.Duration */
+
+UPB_INLINE google_protobuf_Duration *google_protobuf_Duration_new(upb_arena *arena) {
+ return (google_protobuf_Duration *)upb_msg_new(&google_protobuf_Duration_msginit, arena);
+}
+UPB_INLINE google_protobuf_Duration *google_protobuf_Duration_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_Duration *ret = google_protobuf_Duration_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_Duration_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_Duration_serialize(const google_protobuf_Duration *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_Duration_msginit, arena, len);
+}
+
+UPB_INLINE int64_t google_protobuf_Duration_seconds(const google_protobuf_Duration *msg) { return UPB_FIELD_AT(msg, int64_t, UPB_SIZE(0, 0)); }
+UPB_INLINE int32_t google_protobuf_Duration_nanos(const google_protobuf_Duration *msg) { return UPB_FIELD_AT(msg, int32_t, UPB_SIZE(8, 8)); }
+
+UPB_INLINE void google_protobuf_Duration_set_seconds(google_protobuf_Duration *msg, int64_t value) {
+ UPB_FIELD_AT(msg, int64_t, UPB_SIZE(0, 0)) = value;
+}
+UPB_INLINE void google_protobuf_Duration_set_nanos(google_protobuf_Duration *msg, int32_t value) {
+ UPB_FIELD_AT(msg, int32_t, UPB_SIZE(8, 8)) = value;
+}
+
+
+#ifdef __cplusplus
+} /* extern "C" */
+#endif
+
+#include "upb/port_undef.inc"
+
+#endif /* GOOGLE_PROTOBUF_DURATION_PROTO_UPB_H_ */
diff --git a/src/core/ext/upb-generated/google/protobuf/struct.upb.c b/src/core/ext/upb-generated/google/protobuf/struct.upb.c
new file mode 100644
index 00000000000..5c199fc5771
--- /dev/null
+++ b/src/core/ext/upb-generated/google/protobuf/struct.upb.c
@@ -0,0 +1,79 @@
+/* This file was generated by upbc (the upb compiler) from the input
+ * file:
+ *
+ * google/protobuf/struct.proto
+ *
+ * Do not edit -- your changes will be discarded when the file is
+ * regenerated. */
+
+#include <stddef.h>
+#include "upb/msg.h"
+#include "google/protobuf/struct.upb.h"
+
+#include "upb/port_def.inc"
+
+static const upb_msglayout *const google_protobuf_Struct_submsgs[1] = {
+ &google_protobuf_Struct_FieldsEntry_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_Struct__fields[1] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 11, 3},
+};
+
+const upb_msglayout google_protobuf_Struct_msginit = {
+ &google_protobuf_Struct_submsgs[0],
+ &google_protobuf_Struct__fields[0],
+ UPB_SIZE(4, 8), 1, false,
+};
+
+static const upb_msglayout *const google_protobuf_Struct_FieldsEntry_submsgs[1] = {
+ &google_protobuf_Value_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_Struct_FieldsEntry__fields[2] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 9, 1},
+ {2, UPB_SIZE(8, 16), 0, 0, 11, 1},
+};
+
+const upb_msglayout google_protobuf_Struct_FieldsEntry_msginit = {
+ &google_protobuf_Struct_FieldsEntry_submsgs[0],
+ &google_protobuf_Struct_FieldsEntry__fields[0],
+ UPB_SIZE(16, 32), 2, false,
+};
+
+static const upb_msglayout *const google_protobuf_Value_submsgs[2] = {
+ &google_protobuf_ListValue_msginit,
+ &google_protobuf_Struct_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_Value__fields[6] = {
+ {1, UPB_SIZE(0, 0), UPB_SIZE(-9, -17), 0, 14, 1},
+ {2, UPB_SIZE(0, 0), UPB_SIZE(-9, -17), 0, 1, 1},
+ {3, UPB_SIZE(0, 0), UPB_SIZE(-9, -17), 0, 9, 1},
+ {4, UPB_SIZE(0, 0), UPB_SIZE(-9, -17), 0, 8, 1},
+ {5, UPB_SIZE(0, 0), UPB_SIZE(-9, -17), 1, 11, 1},
+ {6, UPB_SIZE(0, 0), UPB_SIZE(-9, -17), 0, 11, 1},
+};
+
+const upb_msglayout google_protobuf_Value_msginit = {
+ &google_protobuf_Value_submsgs[0],
+ &google_protobuf_Value__fields[0],
+ UPB_SIZE(16, 32), 6, false,
+};
+
+static const upb_msglayout *const google_protobuf_ListValue_submsgs[1] = {
+ &google_protobuf_Value_msginit,
+};
+
+static const upb_msglayout_field google_protobuf_ListValue__fields[1] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 11, 3},
+};
+
+const upb_msglayout google_protobuf_ListValue_msginit = {
+ &google_protobuf_ListValue_submsgs[0],
+ &google_protobuf_ListValue__fields[0],
+ UPB_SIZE(4, 8), 1, false,
+};
+
+#include "upb/port_undef.inc"
+
diff --git a/src/core/ext/upb-generated/google/protobuf/struct.upb.h b/src/core/ext/upb-generated/google/protobuf/struct.upb.h
new file mode 100644
index 00000000000..2e28858fc7b
--- /dev/null
+++ b/src/core/ext/upb-generated/google/protobuf/struct.upb.h
@@ -0,0 +1,216 @@
+/* This file was generated by upbc (the upb compiler) from the input
+ * file:
+ *
+ * google/protobuf/struct.proto
+ *
+ * Do not edit -- your changes will be discarded when the file is
+ * regenerated. */
+
+#ifndef GOOGLE_PROTOBUF_STRUCT_PROTO_UPB_H_
+#define GOOGLE_PROTOBUF_STRUCT_PROTO_UPB_H_
+
+#include "upb/generated_util.h"
+
+#include "upb/msg.h"
+
+#include "upb/decode.h"
+#include "upb/encode.h"
+#include "upb/port_def.inc"
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct google_protobuf_Struct;
+struct google_protobuf_Struct_FieldsEntry;
+struct google_protobuf_Value;
+struct google_protobuf_ListValue;
+typedef struct google_protobuf_Struct google_protobuf_Struct;
+typedef struct google_protobuf_Struct_FieldsEntry google_protobuf_Struct_FieldsEntry;
+typedef struct google_protobuf_Value google_protobuf_Value;
+typedef struct google_protobuf_ListValue google_protobuf_ListValue;
+extern const upb_msglayout google_protobuf_Struct_msginit;
+extern const upb_msglayout google_protobuf_Struct_FieldsEntry_msginit;
+extern const upb_msglayout google_protobuf_Value_msginit;
+extern const upb_msglayout google_protobuf_ListValue_msginit;
+
+/* Enums */
+
+typedef enum {
+ google_protobuf_NULL_VALUE = 0
+} google_protobuf_NullValue;
+
+/* google.protobuf.Struct */
+
+UPB_INLINE google_protobuf_Struct *google_protobuf_Struct_new(upb_arena *arena) {
+ return (google_protobuf_Struct *)upb_msg_new(&google_protobuf_Struct_msginit, arena);
+}
+UPB_INLINE google_protobuf_Struct *google_protobuf_Struct_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_Struct *ret = google_protobuf_Struct_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_Struct_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_Struct_serialize(const google_protobuf_Struct *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_Struct_msginit, arena, len);
+}
+
+UPB_INLINE const google_protobuf_Struct_FieldsEntry* const* google_protobuf_Struct_fields(const google_protobuf_Struct *msg, size_t *len) { return (const google_protobuf_Struct_FieldsEntry* const*)_upb_array_accessor(msg, UPB_SIZE(0, 0), len); }
+
+UPB_INLINE google_protobuf_Struct_FieldsEntry** google_protobuf_Struct_mutable_fields(google_protobuf_Struct *msg, size_t *len) {
+ return (google_protobuf_Struct_FieldsEntry**)_upb_array_mutable_accessor(msg, UPB_SIZE(0, 0), len);
+}
+UPB_INLINE google_protobuf_Struct_FieldsEntry** google_protobuf_Struct_resize_fields(google_protobuf_Struct *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_Struct_FieldsEntry**)_upb_array_resize_accessor(msg, UPB_SIZE(0, 0), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_Struct_FieldsEntry* google_protobuf_Struct_add_fields(google_protobuf_Struct *msg, upb_arena *arena) {
+ struct google_protobuf_Struct_FieldsEntry* sub = (struct google_protobuf_Struct_FieldsEntry*)upb_msg_new(&google_protobuf_Struct_FieldsEntry_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(0, 0), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+
+
+/* google.protobuf.Struct.FieldsEntry */
+
+UPB_INLINE google_protobuf_Struct_FieldsEntry *google_protobuf_Struct_FieldsEntry_new(upb_arena *arena) {
+ return (google_protobuf_Struct_FieldsEntry *)upb_msg_new(&google_protobuf_Struct_FieldsEntry_msginit, arena);
+}
+UPB_INLINE google_protobuf_Struct_FieldsEntry *google_protobuf_Struct_FieldsEntry_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_Struct_FieldsEntry *ret = google_protobuf_Struct_FieldsEntry_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_Struct_FieldsEntry_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_Struct_FieldsEntry_serialize(const google_protobuf_Struct_FieldsEntry *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_Struct_FieldsEntry_msginit, arena, len);
+}
+
+UPB_INLINE upb_strview google_protobuf_Struct_FieldsEntry_key(const google_protobuf_Struct_FieldsEntry *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(0, 0)); }
+UPB_INLINE const google_protobuf_Value* google_protobuf_Struct_FieldsEntry_value(const google_protobuf_Struct_FieldsEntry *msg) { return UPB_FIELD_AT(msg, const google_protobuf_Value*, UPB_SIZE(8, 16)); }
+
+UPB_INLINE void google_protobuf_Struct_FieldsEntry_set_key(google_protobuf_Struct_FieldsEntry *msg, upb_strview value) {
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(0, 0)) = value;
+}
+UPB_INLINE void google_protobuf_Struct_FieldsEntry_set_value(google_protobuf_Struct_FieldsEntry *msg, google_protobuf_Value* value) {
+ UPB_FIELD_AT(msg, google_protobuf_Value*, UPB_SIZE(8, 16)) = value;
+}
+UPB_INLINE struct google_protobuf_Value* google_protobuf_Struct_FieldsEntry_mutable_value(google_protobuf_Struct_FieldsEntry *msg, upb_arena *arena) {
+ struct google_protobuf_Value* sub = (struct google_protobuf_Value*)google_protobuf_Struct_FieldsEntry_value(msg);
+ if (sub == NULL) {
+ sub = (struct google_protobuf_Value*)upb_msg_new(&google_protobuf_Value_msginit, arena);
+ if (!sub) return NULL;
+ google_protobuf_Struct_FieldsEntry_set_value(msg, sub);
+ }
+ return sub;
+}
+
+
+/* google.protobuf.Value */
+
+UPB_INLINE google_protobuf_Value *google_protobuf_Value_new(upb_arena *arena) {
+ return (google_protobuf_Value *)upb_msg_new(&google_protobuf_Value_msginit, arena);
+}
+UPB_INLINE google_protobuf_Value *google_protobuf_Value_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_Value *ret = google_protobuf_Value_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_Value_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_Value_serialize(const google_protobuf_Value *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_Value_msginit, arena, len);
+}
+
+typedef enum {
+ google_protobuf_Value_kind_null_value = 1,
+ google_protobuf_Value_kind_number_value = 2,
+ google_protobuf_Value_kind_string_value = 3,
+ google_protobuf_Value_kind_bool_value = 4,
+ google_protobuf_Value_kind_struct_value = 5,
+ google_protobuf_Value_kind_list_value = 6,
+ google_protobuf_Value_kind_NOT_SET = 0,
+} google_protobuf_Value_kind_oneofcases;
+UPB_INLINE google_protobuf_Value_kind_oneofcases google_protobuf_Value_kind_case(const google_protobuf_Value* msg) { return UPB_FIELD_AT(msg, int, UPB_SIZE(8, 16)); }
+
+UPB_INLINE bool google_protobuf_Value_has_null_value(const google_protobuf_Value *msg) { return _upb_has_oneof_field(msg, UPB_SIZE(8, 16), 1); }
+UPB_INLINE google_protobuf_NullValue google_protobuf_Value_null_value(const google_protobuf_Value *msg) { return UPB_READ_ONEOF(msg, google_protobuf_NullValue, UPB_SIZE(0, 0), UPB_SIZE(8, 16), 1, google_protobuf_NULL_VALUE); }
+UPB_INLINE bool google_protobuf_Value_has_number_value(const google_protobuf_Value *msg) { return _upb_has_oneof_field(msg, UPB_SIZE(8, 16), 2); }
+UPB_INLINE double google_protobuf_Value_number_value(const google_protobuf_Value *msg) { return UPB_READ_ONEOF(msg, double, UPB_SIZE(0, 0), UPB_SIZE(8, 16), 2, 0); }
+UPB_INLINE bool google_protobuf_Value_has_string_value(const google_protobuf_Value *msg) { return _upb_has_oneof_field(msg, UPB_SIZE(8, 16), 3); }
+UPB_INLINE upb_strview google_protobuf_Value_string_value(const google_protobuf_Value *msg) { return UPB_READ_ONEOF(msg, upb_strview, UPB_SIZE(0, 0), UPB_SIZE(8, 16), 3, upb_strview_make("", strlen(""))); }
+UPB_INLINE bool google_protobuf_Value_has_bool_value(const google_protobuf_Value *msg) { return _upb_has_oneof_field(msg, UPB_SIZE(8, 16), 4); }
+UPB_INLINE bool google_protobuf_Value_bool_value(const google_protobuf_Value *msg) { return UPB_READ_ONEOF(msg, bool, UPB_SIZE(0, 0), UPB_SIZE(8, 16), 4, false); }
+UPB_INLINE bool google_protobuf_Value_has_struct_value(const google_protobuf_Value *msg) { return _upb_has_oneof_field(msg, UPB_SIZE(8, 16), 5); }
+UPB_INLINE const google_protobuf_Struct* google_protobuf_Value_struct_value(const google_protobuf_Value *msg) { return UPB_READ_ONEOF(msg, const google_protobuf_Struct*, UPB_SIZE(0, 0), UPB_SIZE(8, 16), 5, NULL); }
+UPB_INLINE bool google_protobuf_Value_has_list_value(const google_protobuf_Value *msg) { return _upb_has_oneof_field(msg, UPB_SIZE(8, 16), 6); }
+UPB_INLINE const google_protobuf_ListValue* google_protobuf_Value_list_value(const google_protobuf_Value *msg) { return UPB_READ_ONEOF(msg, const google_protobuf_ListValue*, UPB_SIZE(0, 0), UPB_SIZE(8, 16), 6, NULL); }
+
+UPB_INLINE void google_protobuf_Value_set_null_value(google_protobuf_Value *msg, google_protobuf_NullValue value) {
+ UPB_WRITE_ONEOF(msg, google_protobuf_NullValue, UPB_SIZE(0, 0), value, UPB_SIZE(8, 16), 1);
+}
+UPB_INLINE void google_protobuf_Value_set_number_value(google_protobuf_Value *msg, double value) {
+ UPB_WRITE_ONEOF(msg, double, UPB_SIZE(0, 0), value, UPB_SIZE(8, 16), 2);
+}
+UPB_INLINE void google_protobuf_Value_set_string_value(google_protobuf_Value *msg, upb_strview value) {
+ UPB_WRITE_ONEOF(msg, upb_strview, UPB_SIZE(0, 0), value, UPB_SIZE(8, 16), 3);
+}
+UPB_INLINE void google_protobuf_Value_set_bool_value(google_protobuf_Value *msg, bool value) {
+ UPB_WRITE_ONEOF(msg, bool, UPB_SIZE(0, 0), value, UPB_SIZE(8, 16), 4);
+}
+UPB_INLINE void google_protobuf_Value_set_struct_value(google_protobuf_Value *msg, google_protobuf_Struct* value) {
+ UPB_WRITE_ONEOF(msg, google_protobuf_Struct*, UPB_SIZE(0, 0), value, UPB_SIZE(8, 16), 5);
+}
+UPB_INLINE struct google_protobuf_Struct* google_protobuf_Value_mutable_struct_value(google_protobuf_Value *msg, upb_arena *arena) {
+ struct google_protobuf_Struct* sub = (struct google_protobuf_Struct*)google_protobuf_Value_struct_value(msg);
+ if (sub == NULL) {
+ sub = (struct google_protobuf_Struct*)upb_msg_new(&google_protobuf_Struct_msginit, arena);
+ if (!sub) return NULL;
+ google_protobuf_Value_set_struct_value(msg, sub);
+ }
+ return sub;
+}
+UPB_INLINE void google_protobuf_Value_set_list_value(google_protobuf_Value *msg, google_protobuf_ListValue* value) {
+ UPB_WRITE_ONEOF(msg, google_protobuf_ListValue*, UPB_SIZE(0, 0), value, UPB_SIZE(8, 16), 6);
+}
+UPB_INLINE struct google_protobuf_ListValue* google_protobuf_Value_mutable_list_value(google_protobuf_Value *msg, upb_arena *arena) {
+ struct google_protobuf_ListValue* sub = (struct google_protobuf_ListValue*)google_protobuf_Value_list_value(msg);
+ if (sub == NULL) {
+ sub = (struct google_protobuf_ListValue*)upb_msg_new(&google_protobuf_ListValue_msginit, arena);
+ if (!sub) return NULL;
+ google_protobuf_Value_set_list_value(msg, sub);
+ }
+ return sub;
+}
+
+
+/* google.protobuf.ListValue */
+
+UPB_INLINE google_protobuf_ListValue *google_protobuf_ListValue_new(upb_arena *arena) {
+ return (google_protobuf_ListValue *)upb_msg_new(&google_protobuf_ListValue_msginit, arena);
+}
+UPB_INLINE google_protobuf_ListValue *google_protobuf_ListValue_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_ListValue *ret = google_protobuf_ListValue_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_ListValue_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_ListValue_serialize(const google_protobuf_ListValue *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_ListValue_msginit, arena, len);
+}
+
+UPB_INLINE const google_protobuf_Value* const* google_protobuf_ListValue_values(const google_protobuf_ListValue *msg, size_t *len) { return (const google_protobuf_Value* const*)_upb_array_accessor(msg, UPB_SIZE(0, 0), len); }
+
+UPB_INLINE google_protobuf_Value** google_protobuf_ListValue_mutable_values(google_protobuf_ListValue *msg, size_t *len) {
+ return (google_protobuf_Value**)_upb_array_mutable_accessor(msg, UPB_SIZE(0, 0), len);
+}
+UPB_INLINE google_protobuf_Value** google_protobuf_ListValue_resize_values(google_protobuf_ListValue *msg, size_t len, upb_arena *arena) {
+ return (google_protobuf_Value**)_upb_array_resize_accessor(msg, UPB_SIZE(0, 0), len, UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, arena);
+}
+UPB_INLINE struct google_protobuf_Value* google_protobuf_ListValue_add_values(google_protobuf_ListValue *msg, upb_arena *arena) {
+ struct google_protobuf_Value* sub = (struct google_protobuf_Value*)upb_msg_new(&google_protobuf_Value_msginit, arena);
+ bool ok = _upb_array_append_accessor(
+ msg, UPB_SIZE(0, 0), UPB_SIZE(4, 8), UPB_TYPE_MESSAGE, &sub, arena);
+ if (!ok) return NULL;
+ return sub;
+}
+
+
+#ifdef __cplusplus
+} /* extern "C" */
+#endif
+
+#include "upb/port_undef.inc"
+
+#endif /* GOOGLE_PROTOBUF_STRUCT_PROTO_UPB_H_ */
diff --git a/src/core/ext/upb-generated/google/protobuf/timestamp.upb.c b/src/core/ext/upb-generated/google/protobuf/timestamp.upb.c
new file mode 100644
index 00000000000..edc7af5f364
--- /dev/null
+++ b/src/core/ext/upb-generated/google/protobuf/timestamp.upb.c
@@ -0,0 +1,27 @@
+/* This file was generated by upbc (the upb compiler) from the input
+ * file:
+ *
+ * google/protobuf/timestamp.proto
+ *
+ * Do not edit -- your changes will be discarded when the file is
+ * regenerated. */
+
+#include <stddef.h>
+#include "upb/msg.h"
+#include "google/protobuf/timestamp.upb.h"
+
+#include "upb/port_def.inc"
+
+static const upb_msglayout_field google_protobuf_Timestamp__fields[2] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 3, 1},
+ {2, UPB_SIZE(8, 8), 0, 0, 5, 1},
+};
+
+const upb_msglayout google_protobuf_Timestamp_msginit = {
+ NULL,
+ &google_protobuf_Timestamp__fields[0],
+ UPB_SIZE(16, 16), 2, false,
+};
+
+#include "upb/port_undef.inc"
+
diff --git a/src/core/ext/upb-generated/google/protobuf/timestamp.upb.h b/src/core/ext/upb-generated/google/protobuf/timestamp.upb.h
new file mode 100644
index 00000000000..42413a43014
--- /dev/null
+++ b/src/core/ext/upb-generated/google/protobuf/timestamp.upb.h
@@ -0,0 +1,59 @@
+/* This file was generated by upbc (the upb compiler) from the input
+ * file:
+ *
+ * google/protobuf/timestamp.proto
+ *
+ * Do not edit -- your changes will be discarded when the file is
+ * regenerated. */
+
+#ifndef GOOGLE_PROTOBUF_TIMESTAMP_PROTO_UPB_H_
+#define GOOGLE_PROTOBUF_TIMESTAMP_PROTO_UPB_H_
+
+#include "upb/generated_util.h"
+
+#include "upb/msg.h"
+
+#include "upb/decode.h"
+#include "upb/encode.h"
+#include "upb/port_def.inc"
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct google_protobuf_Timestamp;
+typedef struct google_protobuf_Timestamp google_protobuf_Timestamp;
+extern const upb_msglayout google_protobuf_Timestamp_msginit;
+
+/* Enums */
+
+/* google.protobuf.Timestamp */
+
+UPB_INLINE google_protobuf_Timestamp *google_protobuf_Timestamp_new(upb_arena *arena) {
+ return (google_protobuf_Timestamp *)upb_msg_new(&google_protobuf_Timestamp_msginit, arena);
+}
+UPB_INLINE google_protobuf_Timestamp *google_protobuf_Timestamp_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_Timestamp *ret = google_protobuf_Timestamp_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_Timestamp_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_Timestamp_serialize(const google_protobuf_Timestamp *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_Timestamp_msginit, arena, len);
+}
+
+UPB_INLINE int64_t google_protobuf_Timestamp_seconds(const google_protobuf_Timestamp *msg) { return UPB_FIELD_AT(msg, int64_t, UPB_SIZE(0, 0)); }
+UPB_INLINE int32_t google_protobuf_Timestamp_nanos(const google_protobuf_Timestamp *msg) { return UPB_FIELD_AT(msg, int32_t, UPB_SIZE(8, 8)); }
+
+UPB_INLINE void google_protobuf_Timestamp_set_seconds(google_protobuf_Timestamp *msg, int64_t value) {
+ UPB_FIELD_AT(msg, int64_t, UPB_SIZE(0, 0)) = value;
+}
+UPB_INLINE void google_protobuf_Timestamp_set_nanos(google_protobuf_Timestamp *msg, int32_t value) {
+ UPB_FIELD_AT(msg, int32_t, UPB_SIZE(8, 8)) = value;
+}
+
+
+#ifdef __cplusplus
+} /* extern "C" */
+#endif
+
+#include "upb/port_undef.inc"
+
+#endif /* GOOGLE_PROTOBUF_TIMESTAMP_PROTO_UPB_H_ */
diff --git a/src/core/ext/upb-generated/google/protobuf/wrappers.upb.c b/src/core/ext/upb-generated/google/protobuf/wrappers.upb.c
new file mode 100644
index 00000000000..1b93ef437ac
--- /dev/null
+++ b/src/core/ext/upb-generated/google/protobuf/wrappers.upb.c
@@ -0,0 +1,106 @@
+/* This file was generated by upbc (the upb compiler) from the input
+ * file:
+ *
+ * google/protobuf/wrappers.proto
+ *
+ * Do not edit -- your changes will be discarded when the file is
+ * regenerated. */
+
+#include <stddef.h>
+#include "upb/msg.h"
+#include "google/protobuf/wrappers.upb.h"
+
+#include "upb/port_def.inc"
+
+static const upb_msglayout_field google_protobuf_DoubleValue__fields[1] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 1, 1},
+};
+
+const upb_msglayout google_protobuf_DoubleValue_msginit = {
+ NULL,
+ &google_protobuf_DoubleValue__fields[0],
+ UPB_SIZE(8, 8), 1, false,
+};
+
+static const upb_msglayout_field google_protobuf_FloatValue__fields[1] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 2, 1},
+};
+
+const upb_msglayout google_protobuf_FloatValue_msginit = {
+ NULL,
+ &google_protobuf_FloatValue__fields[0],
+ UPB_SIZE(4, 4), 1, false,
+};
+
+static const upb_msglayout_field google_protobuf_Int64Value__fields[1] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 3, 1},
+};
+
+const upb_msglayout google_protobuf_Int64Value_msginit = {
+ NULL,
+ &google_protobuf_Int64Value__fields[0],
+ UPB_SIZE(8, 8), 1, false,
+};
+
+static const upb_msglayout_field google_protobuf_UInt64Value__fields[1] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 4, 1},
+};
+
+const upb_msglayout google_protobuf_UInt64Value_msginit = {
+ NULL,
+ &google_protobuf_UInt64Value__fields[0],
+ UPB_SIZE(8, 8), 1, false,
+};
+
+static const upb_msglayout_field google_protobuf_Int32Value__fields[1] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 5, 1},
+};
+
+const upb_msglayout google_protobuf_Int32Value_msginit = {
+ NULL,
+ &google_protobuf_Int32Value__fields[0],
+ UPB_SIZE(4, 4), 1, false,
+};
+
+static const upb_msglayout_field google_protobuf_UInt32Value__fields[1] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 13, 1},
+};
+
+const upb_msglayout google_protobuf_UInt32Value_msginit = {
+ NULL,
+ &google_protobuf_UInt32Value__fields[0],
+ UPB_SIZE(4, 4), 1, false,
+};
+
+static const upb_msglayout_field google_protobuf_BoolValue__fields[1] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 8, 1},
+};
+
+const upb_msglayout google_protobuf_BoolValue_msginit = {
+ NULL,
+ &google_protobuf_BoolValue__fields[0],
+ UPB_SIZE(1, 1), 1, false,
+};
+
+static const upb_msglayout_field google_protobuf_StringValue__fields[1] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 9, 1},
+};
+
+const upb_msglayout google_protobuf_StringValue_msginit = {
+ NULL,
+ &google_protobuf_StringValue__fields[0],
+ UPB_SIZE(8, 16), 1, false,
+};
+
+static const upb_msglayout_field google_protobuf_BytesValue__fields[1] = {
+ {1, UPB_SIZE(0, 0), 0, 0, 12, 1},
+};
+
+const upb_msglayout google_protobuf_BytesValue_msginit = {
+ NULL,
+ &google_protobuf_BytesValue__fields[0],
+ UPB_SIZE(8, 16), 1, false,
+};
+
+#include "upb/port_undef.inc"
+
diff --git a/src/core/ext/upb-generated/google/protobuf/wrappers.upb.h b/src/core/ext/upb-generated/google/protobuf/wrappers.upb.h
new file mode 100644
index 00000000000..d08ee51780a
--- /dev/null
+++ b/src/core/ext/upb-generated/google/protobuf/wrappers.upb.h
@@ -0,0 +1,239 @@
+/* This file was generated by upbc (the upb compiler) from the input
+ * file:
+ *
+ * google/protobuf/wrappers.proto
+ *
+ * Do not edit -- your changes will be discarded when the file is
+ * regenerated. */
+
+#ifndef GOOGLE_PROTOBUF_WRAPPERS_PROTO_UPB_H_
+#define GOOGLE_PROTOBUF_WRAPPERS_PROTO_UPB_H_
+
+#include "upb/generated_util.h"
+
+#include "upb/msg.h"
+
+#include "upb/decode.h"
+#include "upb/encode.h"
+#include "upb/port_def.inc"
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct google_protobuf_DoubleValue;
+struct google_protobuf_FloatValue;
+struct google_protobuf_Int64Value;
+struct google_protobuf_UInt64Value;
+struct google_protobuf_Int32Value;
+struct google_protobuf_UInt32Value;
+struct google_protobuf_BoolValue;
+struct google_protobuf_StringValue;
+struct google_protobuf_BytesValue;
+typedef struct google_protobuf_DoubleValue google_protobuf_DoubleValue;
+typedef struct google_protobuf_FloatValue google_protobuf_FloatValue;
+typedef struct google_protobuf_Int64Value google_protobuf_Int64Value;
+typedef struct google_protobuf_UInt64Value google_protobuf_UInt64Value;
+typedef struct google_protobuf_Int32Value google_protobuf_Int32Value;
+typedef struct google_protobuf_UInt32Value google_protobuf_UInt32Value;
+typedef struct google_protobuf_BoolValue google_protobuf_BoolValue;
+typedef struct google_protobuf_StringValue google_protobuf_StringValue;
+typedef struct google_protobuf_BytesValue google_protobuf_BytesValue;
+extern const upb_msglayout google_protobuf_DoubleValue_msginit;
+extern const upb_msglayout google_protobuf_FloatValue_msginit;
+extern const upb_msglayout google_protobuf_Int64Value_msginit;
+extern const upb_msglayout google_protobuf_UInt64Value_msginit;
+extern const upb_msglayout google_protobuf_Int32Value_msginit;
+extern const upb_msglayout google_protobuf_UInt32Value_msginit;
+extern const upb_msglayout google_protobuf_BoolValue_msginit;
+extern const upb_msglayout google_protobuf_StringValue_msginit;
+extern const upb_msglayout google_protobuf_BytesValue_msginit;
+
+/* Enums */
+
+/* google.protobuf.DoubleValue */
+
+UPB_INLINE google_protobuf_DoubleValue *google_protobuf_DoubleValue_new(upb_arena *arena) {
+ return (google_protobuf_DoubleValue *)upb_msg_new(&google_protobuf_DoubleValue_msginit, arena);
+}
+UPB_INLINE google_protobuf_DoubleValue *google_protobuf_DoubleValue_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_DoubleValue *ret = google_protobuf_DoubleValue_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_DoubleValue_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_DoubleValue_serialize(const google_protobuf_DoubleValue *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_DoubleValue_msginit, arena, len);
+}
+
+UPB_INLINE double google_protobuf_DoubleValue_value(const google_protobuf_DoubleValue *msg) { return UPB_FIELD_AT(msg, double, UPB_SIZE(0, 0)); }
+
+UPB_INLINE void google_protobuf_DoubleValue_set_value(google_protobuf_DoubleValue *msg, double value) {
+ UPB_FIELD_AT(msg, double, UPB_SIZE(0, 0)) = value;
+}
+
+
+/* google.protobuf.FloatValue */
+
+UPB_INLINE google_protobuf_FloatValue *google_protobuf_FloatValue_new(upb_arena *arena) {
+ return (google_protobuf_FloatValue *)upb_msg_new(&google_protobuf_FloatValue_msginit, arena);
+}
+UPB_INLINE google_protobuf_FloatValue *google_protobuf_FloatValue_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_FloatValue *ret = google_protobuf_FloatValue_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_FloatValue_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_FloatValue_serialize(const google_protobuf_FloatValue *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_FloatValue_msginit, arena, len);
+}
+
+UPB_INLINE float google_protobuf_FloatValue_value(const google_protobuf_FloatValue *msg) { return UPB_FIELD_AT(msg, float, UPB_SIZE(0, 0)); }
+
+UPB_INLINE void google_protobuf_FloatValue_set_value(google_protobuf_FloatValue *msg, float value) {
+ UPB_FIELD_AT(msg, float, UPB_SIZE(0, 0)) = value;
+}
+
+
+/* google.protobuf.Int64Value */
+
+UPB_INLINE google_protobuf_Int64Value *google_protobuf_Int64Value_new(upb_arena *arena) {
+ return (google_protobuf_Int64Value *)upb_msg_new(&google_protobuf_Int64Value_msginit, arena);
+}
+UPB_INLINE google_protobuf_Int64Value *google_protobuf_Int64Value_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_Int64Value *ret = google_protobuf_Int64Value_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_Int64Value_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_Int64Value_serialize(const google_protobuf_Int64Value *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_Int64Value_msginit, arena, len);
+}
+
+UPB_INLINE int64_t google_protobuf_Int64Value_value(const google_protobuf_Int64Value *msg) { return UPB_FIELD_AT(msg, int64_t, UPB_SIZE(0, 0)); }
+
+UPB_INLINE void google_protobuf_Int64Value_set_value(google_protobuf_Int64Value *msg, int64_t value) {
+ UPB_FIELD_AT(msg, int64_t, UPB_SIZE(0, 0)) = value;
+}
+
+
+/* google.protobuf.UInt64Value */
+
+UPB_INLINE google_protobuf_UInt64Value *google_protobuf_UInt64Value_new(upb_arena *arena) {
+ return (google_protobuf_UInt64Value *)upb_msg_new(&google_protobuf_UInt64Value_msginit, arena);
+}
+UPB_INLINE google_protobuf_UInt64Value *google_protobuf_UInt64Value_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_UInt64Value *ret = google_protobuf_UInt64Value_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_UInt64Value_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_UInt64Value_serialize(const google_protobuf_UInt64Value *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_UInt64Value_msginit, arena, len);
+}
+
+UPB_INLINE uint64_t google_protobuf_UInt64Value_value(const google_protobuf_UInt64Value *msg) { return UPB_FIELD_AT(msg, uint64_t, UPB_SIZE(0, 0)); }
+
+UPB_INLINE void google_protobuf_UInt64Value_set_value(google_protobuf_UInt64Value *msg, uint64_t value) {
+ UPB_FIELD_AT(msg, uint64_t, UPB_SIZE(0, 0)) = value;
+}
+
+
+/* google.protobuf.Int32Value */
+
+UPB_INLINE google_protobuf_Int32Value *google_protobuf_Int32Value_new(upb_arena *arena) {
+ return (google_protobuf_Int32Value *)upb_msg_new(&google_protobuf_Int32Value_msginit, arena);
+}
+UPB_INLINE google_protobuf_Int32Value *google_protobuf_Int32Value_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_Int32Value *ret = google_protobuf_Int32Value_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_Int32Value_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_Int32Value_serialize(const google_protobuf_Int32Value *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_Int32Value_msginit, arena, len);
+}
+
+UPB_INLINE int32_t google_protobuf_Int32Value_value(const google_protobuf_Int32Value *msg) { return UPB_FIELD_AT(msg, int32_t, UPB_SIZE(0, 0)); }
+
+UPB_INLINE void google_protobuf_Int32Value_set_value(google_protobuf_Int32Value *msg, int32_t value) {
+ UPB_FIELD_AT(msg, int32_t, UPB_SIZE(0, 0)) = value;
+}
+
+
+/* google.protobuf.UInt32Value */
+
+UPB_INLINE google_protobuf_UInt32Value *google_protobuf_UInt32Value_new(upb_arena *arena) {
+ return (google_protobuf_UInt32Value *)upb_msg_new(&google_protobuf_UInt32Value_msginit, arena);
+}
+UPB_INLINE google_protobuf_UInt32Value *google_protobuf_UInt32Value_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_UInt32Value *ret = google_protobuf_UInt32Value_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_UInt32Value_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_UInt32Value_serialize(const google_protobuf_UInt32Value *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_UInt32Value_msginit, arena, len);
+}
+
+UPB_INLINE uint32_t google_protobuf_UInt32Value_value(const google_protobuf_UInt32Value *msg) { return UPB_FIELD_AT(msg, uint32_t, UPB_SIZE(0, 0)); }
+
+UPB_INLINE void google_protobuf_UInt32Value_set_value(google_protobuf_UInt32Value *msg, uint32_t value) {
+ UPB_FIELD_AT(msg, uint32_t, UPB_SIZE(0, 0)) = value;
+}
+
+
+/* google.protobuf.BoolValue */
+
+UPB_INLINE google_protobuf_BoolValue *google_protobuf_BoolValue_new(upb_arena *arena) {
+ return (google_protobuf_BoolValue *)upb_msg_new(&google_protobuf_BoolValue_msginit, arena);
+}
+UPB_INLINE google_protobuf_BoolValue *google_protobuf_BoolValue_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_BoolValue *ret = google_protobuf_BoolValue_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_BoolValue_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_BoolValue_serialize(const google_protobuf_BoolValue *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_BoolValue_msginit, arena, len);
+}
+
+UPB_INLINE bool google_protobuf_BoolValue_value(const google_protobuf_BoolValue *msg) { return UPB_FIELD_AT(msg, bool, UPB_SIZE(0, 0)); }
+
+UPB_INLINE void google_protobuf_BoolValue_set_value(google_protobuf_BoolValue *msg, bool value) {
+ UPB_FIELD_AT(msg, bool, UPB_SIZE(0, 0)) = value;
+}
+
+
+/* google.protobuf.StringValue */
+
+UPB_INLINE google_protobuf_StringValue *google_protobuf_StringValue_new(upb_arena *arena) {
+ return (google_protobuf_StringValue *)upb_msg_new(&google_protobuf_StringValue_msginit, arena);
+}
+UPB_INLINE google_protobuf_StringValue *google_protobuf_StringValue_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_StringValue *ret = google_protobuf_StringValue_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_StringValue_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_StringValue_serialize(const google_protobuf_StringValue *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_StringValue_msginit, arena, len);
+}
+
+UPB_INLINE upb_strview google_protobuf_StringValue_value(const google_protobuf_StringValue *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(0, 0)); }
+
+UPB_INLINE void google_protobuf_StringValue_set_value(google_protobuf_StringValue *msg, upb_strview value) {
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(0, 0)) = value;
+}
+
+
+/* google.protobuf.BytesValue */
+
+UPB_INLINE google_protobuf_BytesValue *google_protobuf_BytesValue_new(upb_arena *arena) {
+ return (google_protobuf_BytesValue *)upb_msg_new(&google_protobuf_BytesValue_msginit, arena);
+}
+UPB_INLINE google_protobuf_BytesValue *google_protobuf_BytesValue_parsenew(upb_strview buf, upb_arena *arena) {
+ google_protobuf_BytesValue *ret = google_protobuf_BytesValue_new(arena);
+ return (ret && upb_decode(buf, ret, &google_protobuf_BytesValue_msginit)) ? ret : NULL;
+}
+UPB_INLINE char *google_protobuf_BytesValue_serialize(const google_protobuf_BytesValue *msg, upb_arena *arena, size_t *len) {
+ return upb_encode(msg, &google_protobuf_BytesValue_msginit, arena, len);
+}
+
+UPB_INLINE upb_strview google_protobuf_BytesValue_value(const google_protobuf_BytesValue *msg) { return UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(0, 0)); }
+
+UPB_INLINE void google_protobuf_BytesValue_set_value(google_protobuf_BytesValue *msg, upb_strview value) {
+ UPB_FIELD_AT(msg, upb_strview, UPB_SIZE(0, 0)) = value;
+}
+
+
+#ifdef __cplusplus
+} /* extern "C" */
+#endif
+
+#include "upb/port_undef.inc"
+
+#endif /* GOOGLE_PROTOBUF_WRAPPERS_PROTO_UPB_H_ */
diff --git a/src/upb/gen_build_yaml.py b/src/upb/gen_build_yaml.py
new file mode 100755
index 00000000000..8b726d374a3
--- /dev/null
+++ b/src/upb/gen_build_yaml.py
@@ -0,0 +1,66 @@
+#!/usr/bin/env python2.7
+
+# Copyright 2015 gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# TODO: This should ideally be in upb submodule to avoid hardcoding this here.
+
+import re
+import os
+import sys
+import yaml
+
+srcs = [
+ "third_party/upb/google/protobuf/descriptor.upb.c",
+ "third_party/upb/upb/decode.c",
+ "third_party/upb/upb/def.c",
+ "third_party/upb/upb/encode.c",
+ "third_party/upb/upb/handlers.c",
+ "third_party/upb/upb/msg.c",
+ "third_party/upb/upb/msgfactory.c",
+ "third_party/upb/upb/sink.c",
+ "third_party/upb/upb/table.c",
+ "third_party/upb/upb/upb.c",
+]
+
+hdrs = [
+ "third_party/upb/google/protobuf/descriptor.upb.h",
+ "third_party/upb/upb/decode.h",
+ "third_party/upb/upb/def.h",
+ "third_party/upb/upb/encode.h",
+ "third_party/upb/upb/handlers.h",
+ "third_party/upb/upb/msg.h",
+ "third_party/upb/upb/msgfactory.h",
+ "third_party/upb/upb/sink.h",
+ "third_party/upb/upb/upb.h",
+]
+
+os.chdir(os.path.dirname(sys.argv[0])+'/../..')
+
+out = {}
+
+try:
+ out['libs'] = [{
+ 'name': 'upb',
+ 'defaults': 'upb',
+ 'build': 'private',
+ 'language': 'c',
+ 'secure': 'no',
+ 'src': srcs,
+ 'headers': hdrs,
+ }]
+except:
+ pass
+
+print yaml.dump(out)
diff --git a/third_party/upb b/third_party/upb
index 9ce4a77f61c..ed9faae0993 160000
--- a/third_party/upb
+++ b/third_party/upb
@@ -1 +1 @@
-Subproject commit 9ce4a77f61c134bbed28bfd5be5cd7dc0e80f5e3
+Subproject commit ed9faae0993704b033c594b072d65e1bf19207fa
diff --git a/tools/buildgen/generate_build_additions.sh b/tools/buildgen/generate_build_additions.sh
index 5a1f4a598a7..c99ad6ee552 100755
--- a/tools/buildgen/generate_build_additions.sh
+++ b/tools/buildgen/generate_build_additions.sh
@@ -19,6 +19,7 @@ gen_build_yaml_dirs=" \
src/boringssl \
src/benchmark \
src/proto \
+ src/upb \
src/zlib \
src/c-ares \
test/core/bad_client \
diff --git a/tools/codegen/core/gen_upb_api.sh b/tools/codegen/core/gen_upb_api.sh
new file mode 100755
index 00000000000..9457e06f124
--- /dev/null
+++ b/tools/codegen/core/gen_upb_api.sh
@@ -0,0 +1,38 @@
+#!/bin/bash
+
+# Copyright 2016 gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# REQUIRES: Bazel
+set -ex
+rm -rf src/core/ext/upb-generated
+mkdir src/core/ext/upb-generated
+cd third_party
+cd upb
+bazel build :protoc-gen-upb
+
+cd ../..
+
+proto_files=( \
+ "google/protobuf/any.proto" \
+ "google/protobuf/struct.proto" \
+ "google/protobuf/wrappers.proto" \
+ "google/protobuf/descriptor.proto" \
+ "google/protobuf/duration.proto" \
+ "google/protobuf/timestamp.proto" )
+
+for i in "${proto_files[@]}"
+do
+ protoc -I=$PWD/third_party/data-plane-api -I=$PWD/third_party/googleapis -I=$PWD/third_party/protobuf -I=$PWD/third_party/protoc-gen-validate $i --upb_out=./src/core/ext/upb-generated --plugin=protoc-gen-upb=third_party/upb/bazel-bin/protoc-gen-upb
+done
diff --git a/tools/distrib/check_copyright.py b/tools/distrib/check_copyright.py
index 787bef1778e..fd93cf31e05 100755
--- a/tools/distrib/check_copyright.py
+++ b/tools/distrib/check_copyright.py
@@ -104,6 +104,20 @@ _EXEMPT = frozenset((
# Designer-generated source
'examples/csharp/HelloworldXamarin/Droid/Resources/Resource.designer.cs',
'examples/csharp/HelloworldXamarin/iOS/ViewController.designer.cs',
+
+ # Upb generated source
+ 'src/core/ext/upb-generated/google/protobuf/any.upb.h',
+ 'src/core/ext/upb-generated/google/protobuf/any.upb.c',
+ 'src/core/ext/upb-generated/google/protobuf/descriptor.upb.h',
+ 'src/core/ext/upb-generated/google/protobuf/descriptor.upb.c',
+ 'src/core/ext/upb-generated/google/protobuf/duration.upb.h',
+ 'src/core/ext/upb-generated/google/protobuf/duration.upb.c',
+ 'src/core/ext/upb-generated/google/protobuf/struct.upb.h',
+ 'src/core/ext/upb-generated/google/protobuf/struct.upb.c',
+ 'src/core/ext/upb-generated/google/protobuf/timestamp.upb.h',
+ 'src/core/ext/upb-generated/google/protobuf/timestamp.upb.c',
+ 'src/core/ext/upb-generated/google/protobuf/wrappers.upb.h',
+ 'src/core/ext/upb-generated/google/protobuf/wrappers.upb.c',
))
RE_YEAR = r'Copyright (?P<first_year>[0-9]+\-)?(?P<last_year>[0-9]+) ([Tt]he )?gRPC [Aa]uthors(\.|)'
diff --git a/tools/distrib/check_include_guards.py b/tools/distrib/check_include_guards.py
index b8d530cce06..ac166ef3844 100755
--- a/tools/distrib/check_include_guards.py
+++ b/tools/distrib/check_include_guards.py
@@ -165,6 +165,20 @@ KNOWN_BAD = set([
'src/core/tsi/alts/handshaker/transport_security_common.pb.h',
'include/grpc++/ext/reflection.grpc.pb.h',
'include/grpc++/ext/reflection.pb.h',
+
+ # Upb generated code.
+ 'src/core/ext/upb-generated/google/protobuf/any.upb.h',
+ 'src/core/ext/upb-generated/google/protobuf/any.upb.c',
+ 'src/core/ext/upb-generated/google/protobuf/descriptor.upb.h',
+ 'src/core/ext/upb-generated/google/protobuf/descriptor.upb.c',
+ 'src/core/ext/upb-generated/google/protobuf/duration.upb.h',
+ 'src/core/ext/upb-generated/google/protobuf/duration.upb.c',
+ 'src/core/ext/upb-generated/google/protobuf/struct.upb.h',
+ 'src/core/ext/upb-generated/google/protobuf/struct.upb.c',
+ 'src/core/ext/upb-generated/google/protobuf/timestamp.upb.h',
+ 'src/core/ext/upb-generated/google/protobuf/timestamp.upb.c',
+ 'src/core/ext/upb-generated/google/protobuf/wrappers.upb.h',
+ 'src/core/ext/upb-generated/google/protobuf/wrappers.upb.c',
])
grep_filter = r"grep -E '^(include|src/core)/.*\.h$'"
diff --git a/tools/dockerfile/grpc_clang_format/clang_format_all_the_things.sh b/tools/dockerfile/grpc_clang_format/clang_format_all_the_things.sh
index 0c8ecc21a0c..ab37c0ae9bd 100755
--- a/tools/dockerfile/grpc_clang_format/clang_format_all_the_things.sh
+++ b/tools/dockerfile/grpc_clang_format/clang_format_all_the_things.sh
@@ -29,7 +29,7 @@ for dir in $DIRS
do
for glob in $GLOB
do
- files="$files `find ${CLANG_FORMAT_ROOT}/$dir -name $glob -and -not -name '*.generated.*' -and -not -name '*.pb.h' -and -not -name '*.pb.c' -and -not -name '*.pb.cc' -and -not -name '*.pbobjc.h' -and -not -name '*.pbobjc.m' -and -not -name '*.pbrpc.h' -and -not -name '*.pbrpc.m' -and -not -name end2end_tests.cc -and -not -name end2end_nosec_tests.cc -and -not -name public_headers_must_be_c89.c -and -not -name grpc_shadow_boringssl.h`"
+ files="$files `find ${CLANG_FORMAT_ROOT}/$dir -name $glob -and -not -name '*.generated.*' -and -not -name '*.upb.h' -and -not -name '*.upb.c' -and -not -name '*.pb.h' -and -not -name '*.pb.c' -and -not -name '*.pb.cc' -and -not -name '*.pbobjc.h' -and -not -name '*.pbobjc.m' -and -not -name '*.pbrpc.h' -and -not -name '*.pbrpc.m' -and -not -name end2end_tests.cc -and -not -name end2end_nosec_tests.cc -and -not -name public_headers_must_be_c89.c -and -not -name grpc_shadow_boringssl.h`"
done
done
diff --git a/tools/run_tests/generated/sources_and_headers.json b/tools/run_tests/generated/sources_and_headers.json
index 7a72a885336..8adde9ec602 100644
--- a/tools/run_tests/generated/sources_and_headers.json
+++ b/tools/run_tests/generated/sources_and_headers.json
@@ -8739,6 +8739,26 @@
"third_party": false,
"type": "lib"
},
+ {
+ "deps": [],
+ "headers": [
+ "third_party/upb/google/protobuf/descriptor.upb.h",
+ "third_party/upb/upb/decode.h",
+ "third_party/upb/upb/def.h",
+ "third_party/upb/upb/encode.h",
+ "third_party/upb/upb/handlers.h",
+ "third_party/upb/upb/msg.h",
+ "third_party/upb/upb/msgfactory.h",
+ "third_party/upb/upb/sink.h",
+ "third_party/upb/upb/upb.h"
+ ],
+ "is_filegroup": false,
+ "language": "c",
+ "name": "upb",
+ "src": [],
+ "third_party": false,
+ "type": "lib"
+ },
{
"deps": [],
"headers": [
diff --git a/tools/run_tests/sanity/check_port_platform.py b/tools/run_tests/sanity/check_port_platform.py
index fff828eaee8..79e7f9c4033 100755
--- a/tools/run_tests/sanity/check_port_platform.py
+++ b/tools/run_tests/sanity/check_port_platform.py
@@ -35,6 +35,9 @@ def check_port_platform_inclusion(directory_root):
continue
if filename.endswith('.pb.h') or filename.endswith('.pb.c'):
continue
+ # Skip check for upb generated code.
+ if filename.endswith('.upb.h') or filename.endswith('.upb.c'):
+ continue
with open(path) as f:
all_lines_in_file = f.readlines()
for index, l in enumerate(all_lines_in_file):
diff --git a/tools/run_tests/sanity/check_submodules.sh b/tools/run_tests/sanity/check_submodules.sh
index 2c447f887ee..12e4c157193 100755
--- a/tools/run_tests/sanity/check_submodules.sh
+++ b/tools/run_tests/sanity/check_submodules.sh
@@ -40,7 +40,7 @@ cat << EOF | awk '{ print $1 }' | sort > "$want_submodules"
9245d481eb3e890f708ff2d7dadf2a10c04748ba third_party/libcxxabi (heads/release_60)
582743bf40c5d3639a70f98f183914a2c0cd0680 third_party/protobuf (v3.7.0-rc.2-20-g582743bf)
e143189bf6f37b3957fb31743df6a1bcf4a8c685 third_party/protoc-gen-validate (v0.0.10)
- 9ce4a77f61c134bbed28bfd5be5cd7dc0e80f5e3 third_party/upb (heads/upbc-cpp)
+ ed9faae0993704b033c594b072d65e1bf19207fa third_party/upb (heads/master)
cacf7f1d4e3d44d871b605da3b647f07d718623f third_party/zlib (v1.2.11)
EOF
From 3d287610296cacb34bb7c10881033aa05cb658d3 Mon Sep 17 00:00:00 2001
From: Norman Link <norman.link@tomra.com>
Date: Fri, 18 Jan 2019 14:01:07 +0100
Subject: [PATCH 114/138] Fixing memory leak in interceptor by removing unsued
send_status_
---
include/grpcpp/impl/codegen/interceptor_common.h | 1 -
1 file changed, 1 deletion(-)
diff --git a/include/grpcpp/impl/codegen/interceptor_common.h b/include/grpcpp/impl/codegen/interceptor_common.h
index 8ed84230911..e7290aca838 100644
--- a/include/grpcpp/impl/codegen/interceptor_common.h
+++ b/include/grpcpp/impl/codegen/interceptor_common.h
@@ -403,7 +403,6 @@ class InterceptorBatchMethodsImpl
grpc_status_code* code_ = nullptr;
grpc::string* error_details_ = nullptr;
grpc::string* error_message_ = nullptr;
- Status send_status_;
std::multimap<grpc::string, grpc::string>* send_trailing_metadata_ = nullptr;
From 55897b9f69789420c581309bdea8fc6422d9ca67 Mon Sep 17 00:00:00 2001
From: John Luo <johluo@microsoft.com>
Date: Wed, 13 Mar 2019 16:56:16 -0700
Subject: [PATCH 115/138] WIP Fix tests
---
.../ProtoCompileCommandLineGeneratorTest.cs | 8 ++++----
src/csharp/Grpc.Tools/ProtoCompile.cs | 2 +-
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/src/csharp/Grpc.Tools.Tests/ProtoCompileCommandLineGeneratorTest.cs b/src/csharp/Grpc.Tools.Tests/ProtoCompileCommandLineGeneratorTest.cs
index cac71466345..1ed7ca67b42 100644
--- a/src/csharp/Grpc.Tools.Tests/ProtoCompileCommandLineGeneratorTest.cs
+++ b/src/csharp/Grpc.Tools.Tests/ProtoCompileCommandLineGeneratorTest.cs
@@ -49,7 +49,7 @@ namespace Grpc.Tools.Tests
ExecuteExpectSuccess();
Assert.That(_task.LastPathToTool, Does.Match(@"protoc(.exe)?$"));
Assert.That(_task.LastResponseFile, Is.EqualTo(new[] {
- "--csharp_out=outdir", "a.proto" }));
+ "--csharp_out=outdir", "--error_format=msvs", "a.proto" }));
}
[Test]
@@ -58,7 +58,7 @@ namespace Grpc.Tools.Tests
_task.ProtoBuf = Utils.MakeSimpleItems("a.proto", "foo/b.proto");
ExecuteExpectSuccess();
Assert.That(_task.LastResponseFile, Is.EqualTo(new[] {
- "--csharp_out=outdir", "a.proto", "foo/b.proto" }));
+ "--csharp_out=outdir", "--error_format=msvs", "a.proto", "foo/b.proto" }));
}
[Test]
@@ -68,7 +68,7 @@ namespace Grpc.Tools.Tests
ExecuteExpectSuccess();
Assert.That(_task.LastResponseFile, Is.EqualTo(new[] {
"--csharp_out=outdir", "--proto_path=/path1",
- "--proto_path=/path2", "a.proto" }));
+ "--proto_path=/path2", "--error_format=msvs", "a.proto" }));
}
[TestCase("Cpp")]
@@ -87,7 +87,7 @@ namespace Grpc.Tools.Tests
ExecuteExpectSuccess();
gen = gen.ToLowerInvariant();
Assert.That(_task.LastResponseFile, Is.EqualTo(new[] {
- $"--{gen}_out=outdir", $"--{gen}_opt=foo,bar", "a.proto" }));
+ $"--{gen}_out=outdir", $"--{gen}_opt=foo,bar", "--error_format=msvs", "a.proto" }));
}
[Test]
diff --git a/src/csharp/Grpc.Tools/ProtoCompile.cs b/src/csharp/Grpc.Tools/ProtoCompile.cs
index f6964205d2b..abff1ea016a 100644
--- a/src/csharp/Grpc.Tools/ProtoCompile.cs
+++ b/src/csharp/Grpc.Tools/ProtoCompile.cs
@@ -318,11 +318,11 @@ namespace Grpc.Tools
cmd.AddSwitchMaybe("proto_path", TrimEndSlash(path));
}
cmd.AddSwitchMaybe("dependency_out", DependencyOut);
+ cmd.AddSwitchMaybe("error_format", "msvs");
foreach (var proto in ProtoBuf)
{
cmd.AddArg(proto.ItemSpec);
}
- cmd.AddSwitchMaybe("error_format", "msvs");
return cmd.ToString();
}
From b089b21c6f62d9f324782a8903b757f699f5d10c Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Thu, 14 Mar 2019 14:44:30 +0100
Subject: [PATCH 116/138] Fix source stepping by upgrading sourcelink
---
src/csharp/Grpc.Core/SourceLink.csproj.include | 7 ++++++-
src/csharp/build_packages_dotnetcli.bat | 14 +++++++-------
2 files changed, 13 insertions(+), 8 deletions(-)
diff --git a/src/csharp/Grpc.Core/SourceLink.csproj.include b/src/csharp/Grpc.Core/SourceLink.csproj.include
index 526db954540..045dcb1a0d6 100755
--- a/src/csharp/Grpc.Core/SourceLink.csproj.include
+++ b/src/csharp/Grpc.Core/SourceLink.csproj.include
@@ -1,8 +1,13 @@
<!-- Ensure that debugging of the resulting NuGet packages work (we're using SourceLink). -->
<Project>
+ <PropertyGroup>
+ <!-- Include PDB in the built .nupkg -->
+ <AllowedOutputExtensionsInPackageBuildOutputFolder>$(AllowedOutputExtensionsInPackageBuildOutputFolder);.pdb</AllowedOutputExtensionsInPackageBuildOutputFolder>
+ </PropertyGroup>
+
<ItemGroup>
- <PackageReference Include="SourceLink.Create.CommandLine" Version="2.7.6" PrivateAssets="all" />
+ <PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.0.0-beta2-18618-05" PrivateAssets="All" />
</ItemGroup>
</Project>
diff --git a/src/csharp/build_packages_dotnetcli.bat b/src/csharp/build_packages_dotnetcli.bat
index 58520f2f497..f500310865b 100755
--- a/src/csharp/build_packages_dotnetcli.bat
+++ b/src/csharp/build_packages_dotnetcli.bat
@@ -32,13 +32,13 @@ expand_dev_version.sh
@rem To be able to build, we also need to put grpc_csharp_ext to its normal location
xcopy /Y /I nativelibs\csharp_ext_windows_x64\grpc_csharp_ext.dll ..\..\cmake\build\x64\Release\
-%DOTNET% pack --configuration Release Grpc.Core.Api /p:SourceLinkCreate=true --output ..\..\..\artifacts || goto :error
-%DOTNET% pack --configuration Release Grpc.Core /p:SourceLinkCreate=true --output ..\..\..\artifacts || goto :error
-%DOTNET% pack --configuration Release Grpc.Core.Testing /p:SourceLinkCreate=true --output ..\..\..\artifacts || goto :error
-%DOTNET% pack --configuration Release Grpc.Auth /p:SourceLinkCreate=true --output ..\..\..\artifacts || goto :error
-%DOTNET% pack --configuration Release Grpc.HealthCheck /p:SourceLinkCreate=true --output ..\..\..\artifacts || goto :error
-%DOTNET% pack --configuration Release Grpc.Reflection /p:SourceLinkCreate=true --output ..\..\..\artifacts || goto :error
-%DOTNET% pack --configuration Release Grpc.Tools /p:SourceLinkCreate=true --output ..\..\..\artifacts || goto :error
+%DOTNET% pack --configuration Release Grpc.Core.Api --output ..\..\..\artifacts || goto :error
+%DOTNET% pack --configuration Release Grpc.Core --output ..\..\..\artifacts || goto :error
+%DOTNET% pack --configuration Release Grpc.Core.Testing --output ..\..\..\artifacts || goto :error
+%DOTNET% pack --configuration Release Grpc.Auth --output ..\..\..\artifacts || goto :error
+%DOTNET% pack --configuration Release Grpc.HealthCheck --output ..\..\..\artifacts || goto :error
+%DOTNET% pack --configuration Release Grpc.Reflection --output ..\..\..\artifacts || goto :error
+%DOTNET% pack --configuration Release Grpc.Tools --output ..\..\..\artifacts || goto :error
@rem build auxiliary packages
%DOTNET% pack --configuration Release Grpc --output ..\..\..\artifacts || goto :error
%DOTNET% pack --configuration Release Grpc.Core.NativeDebug --output ..\..\..\artifacts || goto :error
From b302f912cea43176170fc40a25be070e91b9dd59 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Thu, 14 Mar 2019 15:31:05 +0100
Subject: [PATCH 117/138] publish repository url
---
src/csharp/Grpc.Core/SourceLink.csproj.include | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/csharp/Grpc.Core/SourceLink.csproj.include b/src/csharp/Grpc.Core/SourceLink.csproj.include
index 045dcb1a0d6..9c027deaa3e 100755
--- a/src/csharp/Grpc.Core/SourceLink.csproj.include
+++ b/src/csharp/Grpc.Core/SourceLink.csproj.include
@@ -4,6 +4,7 @@
<PropertyGroup>
<!-- Include PDB in the built .nupkg -->
<AllowedOutputExtensionsInPackageBuildOutputFolder>$(AllowedOutputExtensionsInPackageBuildOutputFolder);.pdb</AllowedOutputExtensionsInPackageBuildOutputFolder>
+ <PublishRepositoryUrl>true</PublishRepositoryUrl>
</PropertyGroup>
<ItemGroup>
From e8c8ca6d708c8c61c6516ca15dc42ad1c05fcf56 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Thu, 14 Mar 2019 18:29:42 +0100
Subject: [PATCH 118/138] fix layout of Grpc.Tools package
---
src/csharp/Grpc.Tools/Grpc.Tools.csproj | 24 ++++++++++++------------
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/src/csharp/Grpc.Tools/Grpc.Tools.csproj b/src/csharp/Grpc.Tools/Grpc.Tools.csproj
index 89307bfdd65..e1e3633220e 100644
--- a/src/csharp/Grpc.Tools/Grpc.Tools.csproj
+++ b/src/csharp/Grpc.Tools/Grpc.Tools.csproj
@@ -70,20 +70,20 @@ Linux and MacOS. Managed runtime is supplied separately in the Grpc.Core package
<!-- TODO(kkm): GPB builds assets into "macosx", GRPC into "macos". -->
<!-- TODO(kkm): Do not place non-tools under tools/, use build/native/bin/. -->
<!-- TODO(kkm): Do not package windows x64 builds (#13098). -->
- <_Asset PackagePath="tools/windows_x86/protoc.exe" Include="$(Assets_ProtoCompiler)windows_x86/protoc.exe" />
- <_Asset PackagePath="tools/windows_x64/protoc.exe" Include="$(Assets_ProtoCompiler)windows_x64/protoc.exe" />
- <_Asset PackagePath="tools/linux_x86/protoc" Include="$(Assets_ProtoCompiler)linux_x86/protoc" />
- <_Asset PackagePath="tools/linux_x64/protoc" Include="$(Assets_ProtoCompiler)linux_x64/protoc" />
- <_Asset PackagePath="tools/macosx_x86/protoc" Include="$(Assets_ProtoCompiler)macos_x86/protoc" /> <!-- GPB: macosx-->
- <_Asset PackagePath="tools/macosx_x64/protoc" Include="$(Assets_ProtoCompiler)macos_x64/protoc" /> <!-- GPB: macosx-->
+ <_Asset PackagePath="tools/windows_x86/" Include="$(Assets_ProtoCompiler)windows_x86/protoc.exe" />
+ <_Asset PackagePath="tools/windows_x64/" Include="$(Assets_ProtoCompiler)windows_x64/protoc.exe" />
+ <_Asset PackagePath="tools/linux_x86/" Include="$(Assets_ProtoCompiler)linux_x86/protoc" />
+ <_Asset PackagePath="tools/linux_x64/" Include="$(Assets_ProtoCompiler)linux_x64/protoc" />
+ <_Asset PackagePath="tools/macosx_x86/" Include="$(Assets_ProtoCompiler)macos_x86/protoc" /> <!-- GPB: macosx-->
+ <_Asset PackagePath="tools/macosx_x64/" Include="$(Assets_ProtoCompiler)macos_x64/protoc" /> <!-- GPB: macosx-->
<!-- gRPC assets (for Grpc.Tools) -->
- <_Asset PackagePath="tools/windows_x86/grpc_csharp_plugin.exe" Include="$(Assets_GrpcPlugins)protoc_windows_x86/grpc_csharp_plugin.exe" />
- <_Asset PackagePath="tools/windows_x64/grpc_csharp_plugin.exe" Include="$(Assets_GrpcPlugins)protoc_windows_x64/grpc_csharp_plugin.exe" />
- <_Asset PackagePath="tools/linux_x86/grpc_csharp_plugin" Include="$(Assets_GrpcPlugins)protoc_linux_x86/grpc_csharp_plugin" />
- <_Asset PackagePath="tools/linux_x64/grpc_csharp_plugin" Include="$(Assets_GrpcPlugins)protoc_linux_x64/grpc_csharp_plugin" />
- <_Asset PackagePath="tools/macosx_x86/grpc_csharp_plugin" Include="$(Assets_GrpcPlugins)protoc_macos_x86/grpc_csharp_plugin" />
- <_Asset PackagePath="tools/macosx_x64/grpc_csharp_plugin" Include="$(Assets_GrpcPlugins)protoc_macos_x64/grpc_csharp_plugin" />
+ <_Asset PackagePath="tools/windows_x86/" Include="$(Assets_GrpcPlugins)protoc_windows_x86/grpc_csharp_plugin.exe" />
+ <_Asset PackagePath="tools/windows_x64/" Include="$(Assets_GrpcPlugins)protoc_windows_x64/grpc_csharp_plugin.exe" />
+ <_Asset PackagePath="tools/linux_x86/" Include="$(Assets_GrpcPlugins)protoc_linux_x86/grpc_csharp_plugin" />
+ <_Asset PackagePath="tools/linux_x64/" Include="$(Assets_GrpcPlugins)protoc_linux_x64/grpc_csharp_plugin" />
+ <_Asset PackagePath="tools/macosx_x86/" Include="$(Assets_GrpcPlugins)protoc_macos_x86/grpc_csharp_plugin" />
+ <_Asset PackagePath="tools/macosx_x64/" Include="$(Assets_GrpcPlugins)protoc_macos_x64/grpc_csharp_plugin" />
<None Include="@(_Asset)" Pack="true" Visible="false" />
</ItemGroup>
From 7dbe321d767f2a28991ed1c57650620d5b1b4afa Mon Sep 17 00:00:00 2001
From: Nicolas Noble <nicolasnoble@users.noreply.github.com>
Date: Thu, 14 Mar 2019 10:54:02 -0700
Subject: [PATCH 119/138] Update pending_api_cleanups.md
---
doc/core/pending_api_cleanups.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/doc/core/pending_api_cleanups.md b/doc/core/pending_api_cleanups.md
index 5a8270349a4..970bcd08300 100644
--- a/doc/core/pending_api_cleanups.md
+++ b/doc/core/pending_api_cleanups.md
@@ -15,4 +15,4 @@ number:
`include/grpc/impl/codegen/grpc_types.h` (commit `af00d8b`)
(cannot be done until after next grpc release, so that TensorFlow can
use the same code both internally and externally)
-- require a C++ runtime for all languages.
+- require a C++ runtime for all languages wrapping core.
From efe0b4b2ddf53bcee99a1645035300bdc800b02a Mon Sep 17 00:00:00 2001
From: Yash Tibrewal <yashkt@google.com>
Date: Thu, 14 Mar 2019 12:04:05 -0700
Subject: [PATCH 120/138] Use GRPC_LINUX_ERRQUEUE
---
src/core/lib/iomgr/internal_errqueue.cc | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/src/core/lib/iomgr/internal_errqueue.cc b/src/core/lib/iomgr/internal_errqueue.cc
index 982d709f094..4e2bfe3ccd5 100644
--- a/src/core/lib/iomgr/internal_errqueue.cc
+++ b/src/core/lib/iomgr/internal_errqueue.cc
@@ -38,8 +38,7 @@ bool kernel_supports_errqueue() { return errqueue_supported; }
void grpc_errqueue_init() {
/* Both-compile time and run-time linux kernel versions should be atleast 4.0.0
*/
-#ifdef LINUX_VERSION_CODE
-#if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 0, 0)
+#ifdef GRPC_LINUX_ERRQUEUE
struct utsname buffer;
if (uname(&buffer) != 0) {
gpr_log(GPR_ERROR, "uname: %s", strerror(errno));
@@ -55,8 +54,7 @@ void grpc_errqueue_init() {
} else {
gpr_log(GPR_DEBUG, "ERRQUEUE support not enabled");
}
-#endif /* LINUX_VERSION_CODE <= KERNEL_VERSION(4, 0, 0) */
-#endif /* LINUX_VERSION_CODE */
+#endif /* GRPC_LINUX_ERRQUEUE */
}
} /* namespace grpc_core */
From 2b03154cf4cb28cedafecd34726e73baf54f0beb Mon Sep 17 00:00:00 2001
From: "Mark D. Roth" <roth@google.com>
Date: Thu, 14 Mar 2019 12:28:29 -0700
Subject: [PATCH 121/138] Update naming doc.
---
doc/naming.md | 12 ++++--------
1 file changed, 4 insertions(+), 8 deletions(-)
diff --git a/doc/naming.md b/doc/naming.md
index f7cda581f25..42045fd8337 100644
--- a/doc/naming.md
+++ b/doc/naming.md
@@ -67,14 +67,10 @@ Resolvers should be able to contact the authority and get a resolution
that they return back to the gRPC client library. The returned contents
include:
-- A list of resolved addresses, each of which has three attributes:
- - The address itself, including both IP address and port.
- - A boolean indicating whether the address is a backend address (i.e.,
- the address to use to contact the server directly) or a balancer
- address (for cases where [external load balancing](load-balancing.md)
- is in use).
- - The name of the balancer, if the address is a balancer address.
- This will be used to perform peer authorization.
+- A list of resolved addresses (both IP address and port). Each address
+ may have a set of arbitrary attributes (key/value pairs) associated with
+ it, which can be used to communicate information from the resolver to the
+ [load balancing](load-balancing.md) policy.
- A [service config](service_config.md).
The plugin API allows the resolvers to continuously watch an endpoint
From b017c801b683d47160bdb3f98c19124fcbd1fc0a Mon Sep 17 00:00:00 2001
From: Yihua Zhang <yihuaz@google.com>
Date: Thu, 14 Mar 2019 15:24:48 -0700
Subject: [PATCH 122/138] Add SPIFFE security stack to gRPC core
---
BUILD | 4 +
CMakeLists.txt | 40 +
Makefile | 40 +
build.yaml | 4 +
config.m4 | 3 +
config.w32 | 3 +
gRPC-C++.podspec | 2 +
gRPC-Core.podspec | 6 +
grpc.gemspec | 4 +
grpc.gyp | 2 +
include/grpc/grpc_security.h | 51 +-
package.xml | 4 +
.../tls/grpc_tls_credentials_options.h | 9 +-
.../credentials/tls/spiffe_credentials.cc | 129 ++
.../credentials/tls/spiffe_credentials.h | 62 +
.../ssl/ssl_security_connector.cc | 237 +--
.../security/security_connector/ssl_utils.cc | 134 ++
.../security/security_connector/ssl_utils.h | 32 +
.../tls/spiffe_security_connector.cc | 426 ++++
.../tls/spiffe_security_connector.h | 122 ++
src/core/tsi/ssl_transport_security.cc | 22 +-
src/python/grpcio/grpc_core_dependencies.py | 2 +
test/core/end2end/fixtures/h2_spiffe.cc | 290 +++
test/core/end2end/gen_build_yaml.py | 1 +
test/core/end2end/generate_tests.bzl | 1 +
tools/doxygen/Doxyfile.core.internal | 4 +
.../generated/sources_and_headers.json | 23 +
tools/run_tests/generated/tests.json | 1775 +++++++++++++++++
28 files changed, 3231 insertions(+), 201 deletions(-)
create mode 100644 src/core/lib/security/credentials/tls/spiffe_credentials.cc
create mode 100644 src/core/lib/security/credentials/tls/spiffe_credentials.h
create mode 100644 src/core/lib/security/security_connector/tls/spiffe_security_connector.cc
create mode 100644 src/core/lib/security/security_connector/tls/spiffe_security_connector.h
create mode 100644 test/core/end2end/fixtures/h2_spiffe.cc
diff --git a/BUILD b/BUILD
index 0b6fff354f4..afeacf0239a 100644
--- a/BUILD
+++ b/BUILD
@@ -1618,6 +1618,7 @@ grpc_cc_library(
"src/core/lib/security/credentials/plugin/plugin_credentials.cc",
"src/core/lib/security/credentials/ssl/ssl_credentials.cc",
"src/core/lib/security/credentials/tls/grpc_tls_credentials_options.cc",
+ "src/core/lib/security/credentials/tls/spiffe_credentials.cc",
"src/core/lib/security/security_connector/alts/alts_security_connector.cc",
"src/core/lib/security/security_connector/fake/fake_security_connector.cc",
"src/core/lib/security/security_connector/load_system_roots_fallback.cc",
@@ -1626,6 +1627,7 @@ grpc_cc_library(
"src/core/lib/security/security_connector/security_connector.cc",
"src/core/lib/security/security_connector/ssl/ssl_security_connector.cc",
"src/core/lib/security/security_connector/ssl_utils.cc",
+ "src/core/lib/security/security_connector/tls/spiffe_security_connector.cc",
"src/core/lib/security/transport/client_auth_filter.cc",
"src/core/lib/security/transport/secure_endpoint.cc",
"src/core/lib/security/transport/security_handshaker.cc",
@@ -1653,6 +1655,7 @@ grpc_cc_library(
"src/core/lib/security/credentials/plugin/plugin_credentials.h",
"src/core/lib/security/credentials/ssl/ssl_credentials.h",
"src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h",
+ "src/core/lib/security/credentials/tls/spiffe_credentials.h",
"src/core/lib/security/security_connector/alts/alts_security_connector.h",
"src/core/lib/security/security_connector/fake/fake_security_connector.h",
"src/core/lib/security/security_connector/load_system_roots.h",
@@ -1661,6 +1664,7 @@ grpc_cc_library(
"src/core/lib/security/security_connector/security_connector.h",
"src/core/lib/security/security_connector/ssl/ssl_security_connector.h",
"src/core/lib/security/security_connector/ssl_utils.h",
+ "src/core/lib/security/security_connector/tls/spiffe_security_connector.h",
"src/core/lib/security/transport/auth_filters.h",
"src/core/lib/security/transport/secure_endpoint.h",
"src/core/lib/security/transport/security_handshaker.h",
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 0030c9eb9af..2308582c2f4 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -482,6 +482,7 @@ add_dependencies(buildtests_c h2_proxy_test)
add_dependencies(buildtests_c h2_sockpair_test)
add_dependencies(buildtests_c h2_sockpair+trace_test)
add_dependencies(buildtests_c h2_sockpair_1byte_test)
+add_dependencies(buildtests_c h2_spiffe_test)
add_dependencies(buildtests_c h2_ssl_test)
add_dependencies(buildtests_c h2_ssl_proxy_test)
if(_gRPC_PLATFORM_LINUX OR _gRPC_PLATFORM_MAC OR _gRPC_PLATFORM_POSIX)
@@ -1164,6 +1165,7 @@ add_library(grpc
src/core/lib/security/credentials/plugin/plugin_credentials.cc
src/core/lib/security/credentials/ssl/ssl_credentials.cc
src/core/lib/security/credentials/tls/grpc_tls_credentials_options.cc
+ src/core/lib/security/credentials/tls/spiffe_credentials.cc
src/core/lib/security/security_connector/alts/alts_security_connector.cc
src/core/lib/security/security_connector/fake/fake_security_connector.cc
src/core/lib/security/security_connector/load_system_roots_fallback.cc
@@ -1172,6 +1174,7 @@ add_library(grpc
src/core/lib/security/security_connector/security_connector.cc
src/core/lib/security/security_connector/ssl/ssl_security_connector.cc
src/core/lib/security/security_connector/ssl_utils.cc
+ src/core/lib/security/security_connector/tls/spiffe_security_connector.cc
src/core/lib/security/transport/client_auth_filter.cc
src/core/lib/security/transport/secure_endpoint.cc
src/core/lib/security/transport/security_handshaker.cc
@@ -1621,6 +1624,7 @@ add_library(grpc_cronet
src/core/lib/security/credentials/plugin/plugin_credentials.cc
src/core/lib/security/credentials/ssl/ssl_credentials.cc
src/core/lib/security/credentials/tls/grpc_tls_credentials_options.cc
+ src/core/lib/security/credentials/tls/spiffe_credentials.cc
src/core/lib/security/security_connector/alts/alts_security_connector.cc
src/core/lib/security/security_connector/fake/fake_security_connector.cc
src/core/lib/security/security_connector/load_system_roots_fallback.cc
@@ -1629,6 +1633,7 @@ add_library(grpc_cronet
src/core/lib/security/security_connector/security_connector.cc
src/core/lib/security/security_connector/ssl/ssl_security_connector.cc
src/core/lib/security/security_connector/ssl_utils.cc
+ src/core/lib/security/security_connector/tls/spiffe_security_connector.cc
src/core/lib/security/transport/client_auth_filter.cc
src/core/lib/security/transport/secure_endpoint.cc
src/core/lib/security/transport/security_handshaker.cc
@@ -17469,6 +17474,41 @@ target_link_libraries(h2_sockpair_1byte_test
endif (gRPC_BUILD_TESTS)
if (gRPC_BUILD_TESTS)
+add_executable(h2_spiffe_test
+ test/core/end2end/fixtures/h2_spiffe.cc
+)
+
+
+target_include_directories(h2_spiffe_test
+ PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}
+ PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/include
+ PRIVATE ${_gRPC_SSL_INCLUDE_DIR}
+ PRIVATE ${_gRPC_PROTOBUF_INCLUDE_DIR}
+ PRIVATE ${_gRPC_ZLIB_INCLUDE_DIR}
+ PRIVATE ${_gRPC_BENCHMARK_INCLUDE_DIR}
+ PRIVATE ${_gRPC_CARES_INCLUDE_DIR}
+ PRIVATE ${_gRPC_GFLAGS_INCLUDE_DIR}
+ PRIVATE ${_gRPC_ADDRESS_SORTING_INCLUDE_DIR}
+ PRIVATE ${_gRPC_NANOPB_INCLUDE_DIR}
+)
+
+target_link_libraries(h2_spiffe_test
+ ${_gRPC_ALLTARGETS_LIBRARIES}
+ end2end_tests
+ grpc_test_util
+ grpc
+ gpr
+)
+
+ # avoid dependency on libstdc++
+ if (_gRPC_CORE_NOSTDCXX_FLAGS)
+ set_target_properties(h2_spiffe_test PROPERTIES LINKER_LANGUAGE C)
+ target_compile_options(h2_spiffe_test PRIVATE $<$<COMPILE_LANGUAGE:CXX>:${_gRPC_CORE_NOSTDCXX_FLAGS}>)
+ endif()
+
+endif (gRPC_BUILD_TESTS)
+if (gRPC_BUILD_TESTS)
+
add_executable(h2_ssl_test
test/core/end2end/fixtures/h2_ssl.cc
)
diff --git a/Makefile b/Makefile
index 5a31d648b32..69a2abbc8ca 100644
--- a/Makefile
+++ b/Makefile
@@ -1361,6 +1361,7 @@ h2_proxy_test: $(BINDIR)/$(CONFIG)/h2_proxy_test
h2_sockpair_test: $(BINDIR)/$(CONFIG)/h2_sockpair_test
h2_sockpair+trace_test: $(BINDIR)/$(CONFIG)/h2_sockpair+trace_test
h2_sockpair_1byte_test: $(BINDIR)/$(CONFIG)/h2_sockpair_1byte_test
+h2_spiffe_test: $(BINDIR)/$(CONFIG)/h2_spiffe_test
h2_ssl_test: $(BINDIR)/$(CONFIG)/h2_ssl_test
h2_ssl_proxy_test: $(BINDIR)/$(CONFIG)/h2_ssl_proxy_test
h2_uds_test: $(BINDIR)/$(CONFIG)/h2_uds_test
@@ -1623,6 +1624,7 @@ buildtests_c: privatelibs_c \
$(BINDIR)/$(CONFIG)/h2_sockpair_test \
$(BINDIR)/$(CONFIG)/h2_sockpair+trace_test \
$(BINDIR)/$(CONFIG)/h2_sockpair_1byte_test \
+ $(BINDIR)/$(CONFIG)/h2_spiffe_test \
$(BINDIR)/$(CONFIG)/h2_ssl_test \
$(BINDIR)/$(CONFIG)/h2_ssl_proxy_test \
$(BINDIR)/$(CONFIG)/h2_uds_test \
@@ -3708,6 +3710,7 @@ LIBGRPC_SRC = \
src/core/lib/security/credentials/plugin/plugin_credentials.cc \
src/core/lib/security/credentials/ssl/ssl_credentials.cc \
src/core/lib/security/credentials/tls/grpc_tls_credentials_options.cc \
+ src/core/lib/security/credentials/tls/spiffe_credentials.cc \
src/core/lib/security/security_connector/alts/alts_security_connector.cc \
src/core/lib/security/security_connector/fake/fake_security_connector.cc \
src/core/lib/security/security_connector/load_system_roots_fallback.cc \
@@ -3716,6 +3719,7 @@ LIBGRPC_SRC = \
src/core/lib/security/security_connector/security_connector.cc \
src/core/lib/security/security_connector/ssl/ssl_security_connector.cc \
src/core/lib/security/security_connector/ssl_utils.cc \
+ src/core/lib/security/security_connector/tls/spiffe_security_connector.cc \
src/core/lib/security/transport/client_auth_filter.cc \
src/core/lib/security/transport/secure_endpoint.cc \
src/core/lib/security/transport/security_handshaker.cc \
@@ -4159,6 +4163,7 @@ LIBGRPC_CRONET_SRC = \
src/core/lib/security/credentials/plugin/plugin_credentials.cc \
src/core/lib/security/credentials/ssl/ssl_credentials.cc \
src/core/lib/security/credentials/tls/grpc_tls_credentials_options.cc \
+ src/core/lib/security/credentials/tls/spiffe_credentials.cc \
src/core/lib/security/security_connector/alts/alts_security_connector.cc \
src/core/lib/security/security_connector/fake/fake_security_connector.cc \
src/core/lib/security/security_connector/load_system_roots_fallback.cc \
@@ -4167,6 +4172,7 @@ LIBGRPC_CRONET_SRC = \
src/core/lib/security/security_connector/security_connector.cc \
src/core/lib/security/security_connector/ssl/ssl_security_connector.cc \
src/core/lib/security/security_connector/ssl_utils.cc \
+ src/core/lib/security/security_connector/tls/spiffe_security_connector.cc \
src/core/lib/security/transport/client_auth_filter.cc \
src/core/lib/security/transport/secure_endpoint.cc \
src/core/lib/security/transport/security_handshaker.cc \
@@ -24353,6 +24359,38 @@ endif
endif
+H2_SPIFFE_TEST_SRC = \
+ test/core/end2end/fixtures/h2_spiffe.cc \
+
+H2_SPIFFE_TEST_OBJS = $(addprefix $(OBJDIR)/$(CONFIG)/, $(addsuffix .o, $(basename $(H2_SPIFFE_TEST_SRC))))
+ifeq ($(NO_SECURE),true)
+
+# You can't build secure targets if you don't have OpenSSL.
+
+$(BINDIR)/$(CONFIG)/h2_spiffe_test: openssl_dep_error
+
+else
+
+
+
+$(BINDIR)/$(CONFIG)/h2_spiffe_test: $(H2_SPIFFE_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libend2end_tests.a $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr.a
+ $(E) "[LD] Linking $@"
+ $(Q) mkdir -p `dirname $@`
+ $(Q) $(LD) $(LDFLAGS) $(H2_SPIFFE_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libend2end_tests.a $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr.a $(LDLIBS) $(LDLIBS_SECURE) -o $(BINDIR)/$(CONFIG)/h2_spiffe_test
+
+endif
+
+$(OBJDIR)/$(CONFIG)/test/core/end2end/fixtures/h2_spiffe.o: $(LIBDIR)/$(CONFIG)/libend2end_tests.a $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr.a
+
+deps_h2_spiffe_test: $(H2_SPIFFE_TEST_OBJS:.o=.dep)
+
+ifneq ($(NO_SECURE),true)
+ifneq ($(NO_DEPS),true)
+-include $(H2_SPIFFE_TEST_OBJS:.o=.dep)
+endif
+endif
+
+
H2_SSL_TEST_SRC = \
test/core/end2end/fixtures/h2_ssl.cc \
@@ -25572,6 +25610,7 @@ src/core/lib/security/credentials/oauth2/oauth2_credentials.cc: $(OPENSSL_DEP)
src/core/lib/security/credentials/plugin/plugin_credentials.cc: $(OPENSSL_DEP)
src/core/lib/security/credentials/ssl/ssl_credentials.cc: $(OPENSSL_DEP)
src/core/lib/security/credentials/tls/grpc_tls_credentials_options.cc: $(OPENSSL_DEP)
+src/core/lib/security/credentials/tls/spiffe_credentials.cc: $(OPENSSL_DEP)
src/core/lib/security/security_connector/alts/alts_security_connector.cc: $(OPENSSL_DEP)
src/core/lib/security/security_connector/fake/fake_security_connector.cc: $(OPENSSL_DEP)
src/core/lib/security/security_connector/load_system_roots_fallback.cc: $(OPENSSL_DEP)
@@ -25580,6 +25619,7 @@ src/core/lib/security/security_connector/local/local_security_connector.cc: $(OP
src/core/lib/security/security_connector/security_connector.cc: $(OPENSSL_DEP)
src/core/lib/security/security_connector/ssl/ssl_security_connector.cc: $(OPENSSL_DEP)
src/core/lib/security/security_connector/ssl_utils.cc: $(OPENSSL_DEP)
+src/core/lib/security/security_connector/tls/spiffe_security_connector.cc: $(OPENSSL_DEP)
src/core/lib/security/transport/client_auth_filter.cc: $(OPENSSL_DEP)
src/core/lib/security/transport/secure_endpoint.cc: $(OPENSSL_DEP)
src/core/lib/security/transport/security_handshaker.cc: $(OPENSSL_DEP)
diff --git a/build.yaml b/build.yaml
index d8322b176b7..34b271f58de 100644
--- a/build.yaml
+++ b/build.yaml
@@ -833,6 +833,7 @@ filegroups:
- src/core/lib/security/credentials/plugin/plugin_credentials.h
- src/core/lib/security/credentials/ssl/ssl_credentials.h
- src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h
+ - src/core/lib/security/credentials/tls/spiffe_credentials.h
- src/core/lib/security/security_connector/alts/alts_security_connector.h
- src/core/lib/security/security_connector/fake/fake_security_connector.h
- src/core/lib/security/security_connector/load_system_roots.h
@@ -841,6 +842,7 @@ filegroups:
- src/core/lib/security/security_connector/security_connector.h
- src/core/lib/security/security_connector/ssl/ssl_security_connector.h
- src/core/lib/security/security_connector/ssl_utils.h
+ - src/core/lib/security/security_connector/tls/spiffe_security_connector.h
- src/core/lib/security/transport/auth_filters.h
- src/core/lib/security/transport/secure_endpoint.h
- src/core/lib/security/transport/security_handshaker.h
@@ -866,6 +868,7 @@ filegroups:
- src/core/lib/security/credentials/plugin/plugin_credentials.cc
- src/core/lib/security/credentials/ssl/ssl_credentials.cc
- src/core/lib/security/credentials/tls/grpc_tls_credentials_options.cc
+ - src/core/lib/security/credentials/tls/spiffe_credentials.cc
- src/core/lib/security/security_connector/alts/alts_security_connector.cc
- src/core/lib/security/security_connector/fake/fake_security_connector.cc
- src/core/lib/security/security_connector/load_system_roots_fallback.cc
@@ -874,6 +877,7 @@ filegroups:
- src/core/lib/security/security_connector/security_connector.cc
- src/core/lib/security/security_connector/ssl/ssl_security_connector.cc
- src/core/lib/security/security_connector/ssl_utils.cc
+ - src/core/lib/security/security_connector/tls/spiffe_security_connector.cc
- src/core/lib/security/transport/client_auth_filter.cc
- src/core/lib/security/transport/secure_endpoint.cc
- src/core/lib/security/transport/security_handshaker.cc
diff --git a/config.m4 b/config.m4
index bb23c2ed956..b920799f49b 100644
--- a/config.m4
+++ b/config.m4
@@ -282,6 +282,7 @@ if test "$PHP_GRPC" != "no"; then
src/core/lib/security/credentials/plugin/plugin_credentials.cc \
src/core/lib/security/credentials/ssl/ssl_credentials.cc \
src/core/lib/security/credentials/tls/grpc_tls_credentials_options.cc \
+ src/core/lib/security/credentials/tls/spiffe_credentials.cc \
src/core/lib/security/security_connector/alts/alts_security_connector.cc \
src/core/lib/security/security_connector/fake/fake_security_connector.cc \
src/core/lib/security/security_connector/load_system_roots_fallback.cc \
@@ -290,6 +291,7 @@ if test "$PHP_GRPC" != "no"; then
src/core/lib/security/security_connector/security_connector.cc \
src/core/lib/security/security_connector/ssl/ssl_security_connector.cc \
src/core/lib/security/security_connector/ssl_utils.cc \
+ src/core/lib/security/security_connector/tls/spiffe_security_connector.cc \
src/core/lib/security/transport/client_auth_filter.cc \
src/core/lib/security/transport/secure_endpoint.cc \
src/core/lib/security/transport/security_handshaker.cc \
@@ -733,6 +735,7 @@ if test "$PHP_GRPC" != "no"; then
PHP_ADD_BUILD_DIR($ext_builddir/src/core/lib/security/security_connector/fake)
PHP_ADD_BUILD_DIR($ext_builddir/src/core/lib/security/security_connector/local)
PHP_ADD_BUILD_DIR($ext_builddir/src/core/lib/security/security_connector/ssl)
+ PHP_ADD_BUILD_DIR($ext_builddir/src/core/lib/security/security_connector/tls)
PHP_ADD_BUILD_DIR($ext_builddir/src/core/lib/security/transport)
PHP_ADD_BUILD_DIR($ext_builddir/src/core/lib/security/util)
PHP_ADD_BUILD_DIR($ext_builddir/src/core/lib/slice)
diff --git a/config.w32 b/config.w32
index 35e52e15bd0..49eab2dc109 100644
--- a/config.w32
+++ b/config.w32
@@ -257,6 +257,7 @@ if (PHP_GRPC != "no") {
"src\\core\\lib\\security\\credentials\\plugin\\plugin_credentials.cc " +
"src\\core\\lib\\security\\credentials\\ssl\\ssl_credentials.cc " +
"src\\core\\lib\\security\\credentials\\tls\\grpc_tls_credentials_options.cc " +
+ "src\\core\\lib\\security\\credentials\\tls\\spiffe_credentials.cc " +
"src\\core\\lib\\security\\security_connector\\alts\\alts_security_connector.cc " +
"src\\core\\lib\\security\\security_connector\\fake\\fake_security_connector.cc " +
"src\\core\\lib\\security\\security_connector\\load_system_roots_fallback.cc " +
@@ -265,6 +266,7 @@ if (PHP_GRPC != "no") {
"src\\core\\lib\\security\\security_connector\\security_connector.cc " +
"src\\core\\lib\\security\\security_connector\\ssl\\ssl_security_connector.cc " +
"src\\core\\lib\\security\\security_connector\\ssl_utils.cc " +
+ "src\\core\\lib\\security\\security_connector\\tls\\spiffe_security_connector.cc " +
"src\\core\\lib\\security\\transport\\client_auth_filter.cc " +
"src\\core\\lib\\security\\transport\\secure_endpoint.cc " +
"src\\core\\lib\\security\\transport\\security_handshaker.cc " +
@@ -748,6 +750,7 @@ if (PHP_GRPC != "no") {
FSO.CreateFolder(base_dir+"\\ext\\grpc\\src\\core\\lib\\security\\security_connector\\fake");
FSO.CreateFolder(base_dir+"\\ext\\grpc\\src\\core\\lib\\security\\security_connector\\local");
FSO.CreateFolder(base_dir+"\\ext\\grpc\\src\\core\\lib\\security\\security_connector\\ssl");
+ FSO.CreateFolder(base_dir+"\\ext\\grpc\\src\\core\\lib\\security\\security_connector\\tls");
FSO.CreateFolder(base_dir+"\\ext\\grpc\\src\\core\\lib\\security\\transport");
FSO.CreateFolder(base_dir+"\\ext\\grpc\\src\\core\\lib\\security\\util");
FSO.CreateFolder(base_dir+"\\ext\\grpc\\src\\core\\lib\\slice");
diff --git a/gRPC-C++.podspec b/gRPC-C++.podspec
index e755b7aa602..5a850bc8438 100644
--- a/gRPC-C++.podspec
+++ b/gRPC-C++.podspec
@@ -299,6 +299,7 @@ Pod::Spec.new do |s|
'src/core/lib/security/credentials/plugin/plugin_credentials.h',
'src/core/lib/security/credentials/ssl/ssl_credentials.h',
'src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h',
+ 'src/core/lib/security/credentials/tls/spiffe_credentials.h',
'src/core/lib/security/security_connector/alts/alts_security_connector.h',
'src/core/lib/security/security_connector/fake/fake_security_connector.h',
'src/core/lib/security/security_connector/load_system_roots.h',
@@ -307,6 +308,7 @@ Pod::Spec.new do |s|
'src/core/lib/security/security_connector/security_connector.h',
'src/core/lib/security/security_connector/ssl/ssl_security_connector.h',
'src/core/lib/security/security_connector/ssl_utils.h',
+ 'src/core/lib/security/security_connector/tls/spiffe_security_connector.h',
'src/core/lib/security/transport/auth_filters.h',
'src/core/lib/security/transport/secure_endpoint.h',
'src/core/lib/security/transport/security_handshaker.h',
diff --git a/gRPC-Core.podspec b/gRPC-Core.podspec
index 7068f039870..42633186ac3 100644
--- a/gRPC-Core.podspec
+++ b/gRPC-Core.podspec
@@ -293,6 +293,7 @@ Pod::Spec.new do |s|
'src/core/lib/security/credentials/plugin/plugin_credentials.h',
'src/core/lib/security/credentials/ssl/ssl_credentials.h',
'src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h',
+ 'src/core/lib/security/credentials/tls/spiffe_credentials.h',
'src/core/lib/security/security_connector/alts/alts_security_connector.h',
'src/core/lib/security/security_connector/fake/fake_security_connector.h',
'src/core/lib/security/security_connector/load_system_roots.h',
@@ -301,6 +302,7 @@ Pod::Spec.new do |s|
'src/core/lib/security/security_connector/security_connector.h',
'src/core/lib/security/security_connector/ssl/ssl_security_connector.h',
'src/core/lib/security/security_connector/ssl_utils.h',
+ 'src/core/lib/security/security_connector/tls/spiffe_security_connector.h',
'src/core/lib/security/transport/auth_filters.h',
'src/core/lib/security/transport/secure_endpoint.h',
'src/core/lib/security/transport/security_handshaker.h',
@@ -728,6 +730,7 @@ Pod::Spec.new do |s|
'src/core/lib/security/credentials/plugin/plugin_credentials.cc',
'src/core/lib/security/credentials/ssl/ssl_credentials.cc',
'src/core/lib/security/credentials/tls/grpc_tls_credentials_options.cc',
+ 'src/core/lib/security/credentials/tls/spiffe_credentials.cc',
'src/core/lib/security/security_connector/alts/alts_security_connector.cc',
'src/core/lib/security/security_connector/fake/fake_security_connector.cc',
'src/core/lib/security/security_connector/load_system_roots_fallback.cc',
@@ -736,6 +739,7 @@ Pod::Spec.new do |s|
'src/core/lib/security/security_connector/security_connector.cc',
'src/core/lib/security/security_connector/ssl/ssl_security_connector.cc',
'src/core/lib/security/security_connector/ssl_utils.cc',
+ 'src/core/lib/security/security_connector/tls/spiffe_security_connector.cc',
'src/core/lib/security/transport/client_auth_filter.cc',
'src/core/lib/security/transport/secure_endpoint.cc',
'src/core/lib/security/transport/security_handshaker.cc',
@@ -919,6 +923,7 @@ Pod::Spec.new do |s|
'src/core/lib/security/credentials/plugin/plugin_credentials.h',
'src/core/lib/security/credentials/ssl/ssl_credentials.h',
'src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h',
+ 'src/core/lib/security/credentials/tls/spiffe_credentials.h',
'src/core/lib/security/security_connector/alts/alts_security_connector.h',
'src/core/lib/security/security_connector/fake/fake_security_connector.h',
'src/core/lib/security/security_connector/load_system_roots.h',
@@ -927,6 +932,7 @@ Pod::Spec.new do |s|
'src/core/lib/security/security_connector/security_connector.h',
'src/core/lib/security/security_connector/ssl/ssl_security_connector.h',
'src/core/lib/security/security_connector/ssl_utils.h',
+ 'src/core/lib/security/security_connector/tls/spiffe_security_connector.h',
'src/core/lib/security/transport/auth_filters.h',
'src/core/lib/security/transport/secure_endpoint.h',
'src/core/lib/security/transport/security_handshaker.h',
diff --git a/grpc.gemspec b/grpc.gemspec
index 5c749dd285d..d9fc6ef0ebc 100644
--- a/grpc.gemspec
+++ b/grpc.gemspec
@@ -223,6 +223,7 @@ Gem::Specification.new do |s|
s.files += %w( src/core/lib/security/credentials/plugin/plugin_credentials.h )
s.files += %w( src/core/lib/security/credentials/ssl/ssl_credentials.h )
s.files += %w( src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h )
+ s.files += %w( src/core/lib/security/credentials/tls/spiffe_credentials.h )
s.files += %w( src/core/lib/security/security_connector/alts/alts_security_connector.h )
s.files += %w( src/core/lib/security/security_connector/fake/fake_security_connector.h )
s.files += %w( src/core/lib/security/security_connector/load_system_roots.h )
@@ -231,6 +232,7 @@ Gem::Specification.new do |s|
s.files += %w( src/core/lib/security/security_connector/security_connector.h )
s.files += %w( src/core/lib/security/security_connector/ssl/ssl_security_connector.h )
s.files += %w( src/core/lib/security/security_connector/ssl_utils.h )
+ s.files += %w( src/core/lib/security/security_connector/tls/spiffe_security_connector.h )
s.files += %w( src/core/lib/security/transport/auth_filters.h )
s.files += %w( src/core/lib/security/transport/secure_endpoint.h )
s.files += %w( src/core/lib/security/transport/security_handshaker.h )
@@ -662,6 +664,7 @@ Gem::Specification.new do |s|
s.files += %w( src/core/lib/security/credentials/plugin/plugin_credentials.cc )
s.files += %w( src/core/lib/security/credentials/ssl/ssl_credentials.cc )
s.files += %w( src/core/lib/security/credentials/tls/grpc_tls_credentials_options.cc )
+ s.files += %w( src/core/lib/security/credentials/tls/spiffe_credentials.cc )
s.files += %w( src/core/lib/security/security_connector/alts/alts_security_connector.cc )
s.files += %w( src/core/lib/security/security_connector/fake/fake_security_connector.cc )
s.files += %w( src/core/lib/security/security_connector/load_system_roots_fallback.cc )
@@ -670,6 +673,7 @@ Gem::Specification.new do |s|
s.files += %w( src/core/lib/security/security_connector/security_connector.cc )
s.files += %w( src/core/lib/security/security_connector/ssl/ssl_security_connector.cc )
s.files += %w( src/core/lib/security/security_connector/ssl_utils.cc )
+ s.files += %w( src/core/lib/security/security_connector/tls/spiffe_security_connector.cc )
s.files += %w( src/core/lib/security/transport/client_auth_filter.cc )
s.files += %w( src/core/lib/security/transport/secure_endpoint.cc )
s.files += %w( src/core/lib/security/transport/security_handshaker.cc )
diff --git a/grpc.gyp b/grpc.gyp
index cce9f738fbd..b3795cbfd06 100644
--- a/grpc.gyp
+++ b/grpc.gyp
@@ -464,6 +464,7 @@
'src/core/lib/security/credentials/plugin/plugin_credentials.cc',
'src/core/lib/security/credentials/ssl/ssl_credentials.cc',
'src/core/lib/security/credentials/tls/grpc_tls_credentials_options.cc',
+ 'src/core/lib/security/credentials/tls/spiffe_credentials.cc',
'src/core/lib/security/security_connector/alts/alts_security_connector.cc',
'src/core/lib/security/security_connector/fake/fake_security_connector.cc',
'src/core/lib/security/security_connector/load_system_roots_fallback.cc',
@@ -472,6 +473,7 @@
'src/core/lib/security/security_connector/security_connector.cc',
'src/core/lib/security/security_connector/ssl/ssl_security_connector.cc',
'src/core/lib/security/security_connector/ssl_utils.cc',
+ 'src/core/lib/security/security_connector/tls/spiffe_security_connector.cc',
'src/core/lib/security/transport/client_auth_filter.cc',
'src/core/lib/security/transport/secure_endpoint.cc',
'src/core/lib/security/transport/security_handshaker.cc',
diff --git a/include/grpc/grpc_security.h b/include/grpc/grpc_security.h
index 9b7822627e0..f1185d2b2a6 100644
--- a/include/grpc/grpc_security.h
+++ b/include/grpc/grpc_security.h
@@ -720,7 +720,7 @@ struct grpc_tls_credential_reload_arg {
grpc_tls_on_credential_reload_done_cb cb;
void* cb_user_data;
grpc_tls_key_materials_config* key_materials_config;
- grpc_status_code status;
+ grpc_ssl_certificate_config_reload_status status;
const char* error_details;
};
@@ -767,17 +767,19 @@ typedef void (*grpc_tls_on_server_authorization_check_done_cb)(
/** A struct containing all information necessary to schedule/cancel a server
authorization check request. cb and cb_user_data represent a gRPC-provided
- callback and an argument passed to it. result will store the result of
- server authorization check. target_name is the name of an endpoint the
- channel is connecting to and certificate represents a complete certificate
- chain including both signing and leaf certificates. status and error_details
- contain information about errors occurred when a server authorization check
- request is scheduled/cancelled. It is used for experimental purpose for now
- and subject to change.*/
+ callback and an argument passed to it. success will store the result of
+ server authorization check. That is, if success returns a non-zero value, it
+ means the authorization check passes and if returning zero, it means the
+ check fails. target_name is the name of an endpoint the channel is connecting
+ to and certificate represents a complete certificate chain including both
+ signing and leaf certificates. status and error_details contain information
+ about errors occurred when a server authorization check request is
+ scheduled/cancelled. It is used for experimental purpose for now and subject
+ to change.*/
struct grpc_tls_server_authorization_check_arg {
grpc_tls_on_server_authorization_check_done_cb cb;
void* cb_user_data;
- int result;
+ int success;
const char* target_name;
const char* peer_cert;
grpc_status_code status;
@@ -813,6 +815,37 @@ grpc_tls_server_authorization_check_config_create(
grpc_tls_server_authorization_check_arg* arg),
void (*destruct)(void* config_user_data));
+/** --- SPIFFE channel/server credentials --- **/
+
+/**
+ * This method creates a TLS SPIFFE channel credential object.
+ * It takes ownership of the options parameter.
+ *
+ * - options: grpc TLS credentials options instance.
+ *
+ * It returns the created credential object.
+ *
+ * It is used for experimental purpose for now and subject
+ * to change.
+ */
+
+grpc_channel_credentials* grpc_tls_spiffe_credentials_create(
+ grpc_tls_credentials_options* options);
+
+/**
+ * This method creates a TLS server credential object.
+ * It takes ownership of the options parameter.
+ *
+ * - options: grpc TLS credentials options instance.
+ *
+ * It returns the created credential object.
+ *
+ * It is used for experimental purpose for now and subject
+ * to change.
+ */
+grpc_server_credentials* grpc_tls_spiffe_server_credentials_create(
+ grpc_tls_credentials_options* options);
+
#ifdef __cplusplus
}
#endif
diff --git a/package.xml b/package.xml
index be7e258b98a..48ccdeff73b 100644
--- a/package.xml
+++ b/package.xml
@@ -228,6 +228,7 @@
<file baseinstalldir="/" name="src/core/lib/security/credentials/plugin/plugin_credentials.h" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/credentials/ssl/ssl_credentials.h" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h" role="src" />
+ <file baseinstalldir="/" name="src/core/lib/security/credentials/tls/spiffe_credentials.h" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/security_connector/alts/alts_security_connector.h" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/security_connector/fake/fake_security_connector.h" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/security_connector/load_system_roots.h" role="src" />
@@ -236,6 +237,7 @@
<file baseinstalldir="/" name="src/core/lib/security/security_connector/security_connector.h" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/security_connector/ssl/ssl_security_connector.h" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/security_connector/ssl_utils.h" role="src" />
+ <file baseinstalldir="/" name="src/core/lib/security/security_connector/tls/spiffe_security_connector.h" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/transport/auth_filters.h" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/transport/secure_endpoint.h" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/transport/security_handshaker.h" role="src" />
@@ -667,6 +669,7 @@
<file baseinstalldir="/" name="src/core/lib/security/credentials/plugin/plugin_credentials.cc" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/credentials/ssl/ssl_credentials.cc" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/credentials/tls/grpc_tls_credentials_options.cc" role="src" />
+ <file baseinstalldir="/" name="src/core/lib/security/credentials/tls/spiffe_credentials.cc" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/security_connector/alts/alts_security_connector.cc" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/security_connector/fake/fake_security_connector.cc" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/security_connector/load_system_roots_fallback.cc" role="src" />
@@ -675,6 +678,7 @@
<file baseinstalldir="/" name="src/core/lib/security/security_connector/security_connector.cc" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/security_connector/ssl/ssl_security_connector.cc" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/security_connector/ssl_utils.cc" role="src" />
+ <file baseinstalldir="/" name="src/core/lib/security/security_connector/tls/spiffe_security_connector.cc" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/transport/client_auth_filter.cc" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/transport/secure_endpoint.cc" role="src" />
<file baseinstalldir="/" name="src/core/lib/security/transport/security_handshaker.cc" role="src" />
diff --git a/src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h b/src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h
index 71410d20a8f..aee9292acb8 100644
--- a/src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h
+++ b/src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h
@@ -167,19 +167,16 @@ struct grpc_tls_credentials_options
grpc_ssl_client_certificate_request_type cert_request_type() const {
return cert_request_type_;
}
- const grpc_tls_key_materials_config* key_materials_config() const {
+ grpc_tls_key_materials_config* key_materials_config() const {
return key_materials_config_.get();
}
- const grpc_tls_credential_reload_config* credential_reload_config() const {
+ grpc_tls_credential_reload_config* credential_reload_config() const {
return credential_reload_config_.get();
}
- const grpc_tls_server_authorization_check_config*
+ grpc_tls_server_authorization_check_config*
server_authorization_check_config() const {
return server_authorization_check_config_.get();
}
- grpc_tls_key_materials_config* mutable_key_materials_config() {
- return key_materials_config_.get();
- }
/* Setters for member fields. */
void set_cert_request_type(
diff --git a/src/core/lib/security/credentials/tls/spiffe_credentials.cc b/src/core/lib/security/credentials/tls/spiffe_credentials.cc
new file mode 100644
index 00000000000..da764936c76
--- /dev/null
+++ b/src/core/lib/security/credentials/tls/spiffe_credentials.cc
@@ -0,0 +1,129 @@
+/*
+ *
+ * Copyright 2018 gRPC authors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+#include <grpc/support/port_platform.h>
+
+#include "src/core/lib/security/credentials/tls/spiffe_credentials.h"
+
+#include <cstring>
+
+#include <grpc/grpc.h>
+#include <grpc/support/alloc.h>
+#include <grpc/support/log.h>
+#include <grpc/support/string_util.h>
+
+#include "src/core/lib/channel/channel_args.h"
+#include "src/core/lib/security/security_connector/tls/spiffe_security_connector.h"
+
+#define GRPC_CREDENTIALS_TYPE_SPIFFE "Spiffe"
+
+namespace {
+
+bool CredentialOptionSanityCheck(const grpc_tls_credentials_options* options,
+ bool is_client) {
+ if (options == nullptr) {
+ gpr_log(GPR_ERROR, "SPIFFE TLS credentials options is nullptr.");
+ return false;
+ }
+ if (options->key_materials_config() == nullptr &&
+ options->credential_reload_config() == nullptr) {
+ gpr_log(
+ GPR_ERROR,
+ "SPIFFE TLS credentials options must specify either key materials or "
+ "credential reload config.");
+ return false;
+ }
+ if (!is_client && options->server_authorization_check_config() != nullptr) {
+ gpr_log(GPR_INFO,
+ "Server's credentials options should not contain server "
+ "authorization check config.");
+ }
+ return true;
+}
+
+} // namespace
+
+SpiffeCredentials::SpiffeCredentials(
+ grpc_core::RefCountedPtr<grpc_tls_credentials_options> options)
+ : grpc_channel_credentials(GRPC_CREDENTIALS_TYPE_SPIFFE),
+ options_(std::move(options)) {}
+
+SpiffeCredentials::~SpiffeCredentials() {}
+
+grpc_core::RefCountedPtr<grpc_channel_security_connector>
+SpiffeCredentials::create_security_connector(
+ grpc_core::RefCountedPtr<grpc_call_credentials> call_creds,
+ const char* target_name, const grpc_channel_args* args,
+ grpc_channel_args** new_args) {
+ const char* overridden_target_name = nullptr;
+ tsi_ssl_session_cache* ssl_session_cache = nullptr;
+ for (size_t i = 0; args != nullptr && i < args->num_args; i++) {
+ grpc_arg* arg = &args->args[i];
+ if (strcmp(arg->key, GRPC_SSL_TARGET_NAME_OVERRIDE_ARG) == 0 &&
+ arg->type == GRPC_ARG_STRING) {
+ overridden_target_name = arg->value.string;
+ }
+ if (strcmp(arg->key, GRPC_SSL_SESSION_CACHE_ARG) == 0 &&
+ arg->type == GRPC_ARG_POINTER) {
+ ssl_session_cache =
+ static_cast<tsi_ssl_session_cache*>(arg->value.pointer.p);
+ }
+ }
+ grpc_core::RefCountedPtr<grpc_channel_security_connector> sc =
+ SpiffeChannelSecurityConnector::CreateSpiffeChannelSecurityConnector(
+ this->Ref(), std::move(call_creds), target_name,
+ overridden_target_name, ssl_session_cache);
+ if (sc == nullptr) {
+ return nullptr;
+ }
+ grpc_arg new_arg = grpc_channel_arg_string_create(
+ (char*)GRPC_ARG_HTTP2_SCHEME, (char*)"https");
+ *new_args = grpc_channel_args_copy_and_add(args, &new_arg, 1);
+ return sc;
+}
+
+SpiffeServerCredentials::SpiffeServerCredentials(
+ grpc_core::RefCountedPtr<grpc_tls_credentials_options> options)
+ : grpc_server_credentials(GRPC_CREDENTIALS_TYPE_SPIFFE),
+ options_(std::move(options)) {}
+
+SpiffeServerCredentials::~SpiffeServerCredentials() {}
+
+grpc_core::RefCountedPtr<grpc_server_security_connector>
+SpiffeServerCredentials::create_security_connector() {
+ return SpiffeServerSecurityConnector::CreateSpiffeServerSecurityConnector(
+ this->Ref());
+}
+
+grpc_channel_credentials* grpc_tls_spiffe_credentials_create(
+ grpc_tls_credentials_options* options) {
+ if (!CredentialOptionSanityCheck(options, true /* is_client */)) {
+ return nullptr;
+ }
+ return grpc_core::New<SpiffeCredentials>(
+ grpc_core::RefCountedPtr<grpc_tls_credentials_options>(options));
+}
+
+grpc_server_credentials* grpc_tls_spiffe_server_credentials_create(
+ grpc_tls_credentials_options* options) {
+ if (!CredentialOptionSanityCheck(options, false /* is_client */)) {
+ return nullptr;
+ }
+ return grpc_core::New<SpiffeServerCredentials>(
+ grpc_core::RefCountedPtr<grpc_tls_credentials_options>(options));
+}
diff --git a/src/core/lib/security/credentials/tls/spiffe_credentials.h b/src/core/lib/security/credentials/tls/spiffe_credentials.h
new file mode 100644
index 00000000000..4985fda4a7e
--- /dev/null
+++ b/src/core/lib/security/credentials/tls/spiffe_credentials.h
@@ -0,0 +1,62 @@
+/*
+ *
+ * Copyright 2018 gRPC authors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+#ifndef GRPC_CORE_LIB_SECURITY_CREDENTIALS_TLS_SPIFFE_CREDENTIALS_H
+#define GRPC_CORE_LIB_SECURITY_CREDENTIALS_TLS_SPIFFE_CREDENTIALS_H
+
+#include <grpc/support/port_platform.h>
+
+#include <grpc/grpc_security.h>
+
+#include "src/core/lib/security/credentials/credentials.h"
+#include "src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h"
+
+class SpiffeCredentials final : public grpc_channel_credentials {
+ public:
+ explicit SpiffeCredentials(
+ grpc_core::RefCountedPtr<grpc_tls_credentials_options> options);
+ ~SpiffeCredentials() override;
+
+ grpc_core::RefCountedPtr<grpc_channel_security_connector>
+ create_security_connector(
+ grpc_core::RefCountedPtr<grpc_call_credentials> call_creds,
+ const char* target_name, const grpc_channel_args* args,
+ grpc_channel_args** new_args) override;
+
+ const grpc_tls_credentials_options& options() const { return *options_; }
+
+ private:
+ grpc_core::RefCountedPtr<grpc_tls_credentials_options> options_;
+};
+
+class SpiffeServerCredentials final : public grpc_server_credentials {
+ public:
+ explicit SpiffeServerCredentials(
+ grpc_core::RefCountedPtr<grpc_tls_credentials_options> options);
+ ~SpiffeServerCredentials() override;
+
+ grpc_core::RefCountedPtr<grpc_server_security_connector>
+ create_security_connector() override;
+
+ const grpc_tls_credentials_options& options() const { return *options_; }
+
+ private:
+ grpc_core::RefCountedPtr<grpc_tls_credentials_options> options_;
+};
+
+#endif /* GRPC_CORE_LIB_SECURITY_CREDENTIALS_TLS_SPIFFE_CREDENTIALS_H */
diff --git a/src/core/lib/security/security_connector/ssl/ssl_security_connector.cc b/src/core/lib/security/security_connector/ssl/ssl_security_connector.cc
index 8a00bbb82ed..39c5434208b 100644
--- a/src/core/lib/security/security_connector/ssl/ssl_security_connector.cc
+++ b/src/core/lib/security/security_connector/ssl/ssl_security_connector.cc
@@ -41,33 +41,6 @@
#include "src/core/tsi/transport_security.h"
namespace {
-grpc_error* ssl_check_peer(
- const char* peer_name, const tsi_peer* peer,
- grpc_core::RefCountedPtr<grpc_auth_context>* auth_context) {
-#if TSI_OPENSSL_ALPN_SUPPORT
- /* Check the ALPN if ALPN is supported. */
- const tsi_peer_property* p =
- tsi_peer_get_property_by_name(peer, TSI_SSL_ALPN_SELECTED_PROTOCOL);
- if (p == nullptr) {
- return GRPC_ERROR_CREATE_FROM_STATIC_STRING(
- "Cannot check peer: missing selected ALPN property.");
- }
- if (!grpc_chttp2_is_alpn_version_supported(p->value.data, p->value.length)) {
- return GRPC_ERROR_CREATE_FROM_STATIC_STRING(
- "Cannot check peer: invalid ALPN value.");
- }
-#endif /* TSI_OPENSSL_ALPN_SUPPORT */
- /* Check the peer name if specified. */
- if (peer_name != nullptr && !grpc_ssl_host_matches_name(peer, peer_name)) {
- char* msg;
- gpr_asprintf(&msg, "Peer name %s is not in peer certificate", peer_name);
- grpc_error* error = GRPC_ERROR_CREATE_FROM_COPIED_STRING(msg);
- gpr_free(msg);
- return error;
- }
- *auth_context = grpc_ssl_peer_to_auth_context(peer);
- return GRPC_ERROR_NONE;
-}
class grpc_ssl_channel_security_connector final
: public grpc_channel_security_connector {
@@ -96,34 +69,10 @@ class grpc_ssl_channel_security_connector final
}
grpc_security_status InitializeHandshakerFactory(
- const grpc_ssl_config* config, const char* pem_root_certs,
- const tsi_ssl_root_certs_store* root_store,
- tsi_ssl_session_cache* ssl_session_cache) {
- bool has_key_cert_pair =
- config->pem_key_cert_pair != nullptr &&
- config->pem_key_cert_pair->private_key != nullptr &&
- config->pem_key_cert_pair->cert_chain != nullptr;
- tsi_ssl_client_handshaker_options options;
- GPR_DEBUG_ASSERT(pem_root_certs != nullptr);
- options.pem_root_certs = pem_root_certs;
- options.root_store = root_store;
- options.alpn_protocols =
- grpc_fill_alpn_protocol_strings(&options.num_alpn_protocols);
- if (has_key_cert_pair) {
- options.pem_key_cert_pair = config->pem_key_cert_pair;
- }
- options.cipher_suites = grpc_get_ssl_cipher_suites();
- options.session_cache = ssl_session_cache;
- const tsi_result result =
- tsi_create_ssl_client_handshaker_factory_with_options(
- &options, &client_handshaker_factory_);
- gpr_free((void*)options.alpn_protocols);
- if (result != TSI_OK) {
- gpr_log(GPR_ERROR, "Handshaker factory creation failed with %s.",
- tsi_result_to_string(result));
- return GRPC_SECURITY_ERROR;
- }
- return GRPC_SECURITY_OK;
+ const grpc_ssl_config* config, tsi_ssl_session_cache* ssl_session_cache) {
+ return grpc_ssl_tsi_client_handshaker_factory_init(
+ config->pem_key_cert_pair, config->pem_root_certs, ssl_session_cache,
+ &client_handshaker_factory_);
}
void add_handshakers(grpc_pollset_set* interested_parties,
@@ -150,29 +99,35 @@ class grpc_ssl_channel_security_connector final
const char* target_name = overridden_target_name_ != nullptr
? overridden_target_name_
: target_name_;
- grpc_error* error = ssl_check_peer(target_name, &peer, auth_context);
- if (error == GRPC_ERROR_NONE &&
- verify_options_->verify_peer_callback != nullptr) {
- const tsi_peer_property* p =
- tsi_peer_get_property_by_name(&peer, TSI_X509_PEM_CERT_PROPERTY);
- if (p == nullptr) {
- error = GRPC_ERROR_CREATE_FROM_STATIC_STRING(
- "Cannot check peer: missing pem cert property.");
- } else {
- char* peer_pem = static_cast<char*>(gpr_malloc(p->value.length + 1));
- memcpy(peer_pem, p->value.data, p->value.length);
- peer_pem[p->value.length] = '\0';
- int callback_status = verify_options_->verify_peer_callback(
- target_name, peer_pem,
- verify_options_->verify_peer_callback_userdata);
- gpr_free(peer_pem);
- if (callback_status) {
- char* msg;
- gpr_asprintf(&msg, "Verify peer callback returned a failure (%d)",
- callback_status);
- error = GRPC_ERROR_CREATE_FROM_COPIED_STRING(msg);
- gpr_free(msg);
+ grpc_error* error = grpc_ssl_check_alpn(&peer);
+ if (error == GRPC_ERROR_NONE) {
+ error = grpc_ssl_check_peer_name(target_name, &peer);
+ if (error == GRPC_ERROR_NONE) {
+ if (verify_options_->verify_peer_callback != nullptr) {
+ const tsi_peer_property* p =
+ tsi_peer_get_property_by_name(&peer, TSI_X509_PEM_CERT_PROPERTY);
+ if (p == nullptr) {
+ error = GRPC_ERROR_CREATE_FROM_STATIC_STRING(
+ "Cannot check peer: missing pem cert property.");
+ } else {
+ char* peer_pem =
+ static_cast<char*>(gpr_malloc(p->value.length + 1));
+ memcpy(peer_pem, p->value.data, p->value.length);
+ peer_pem[p->value.length] = '\0';
+ int callback_status = verify_options_->verify_peer_callback(
+ target_name, peer_pem,
+ verify_options_->verify_peer_callback_userdata);
+ gpr_free(peer_pem);
+ if (callback_status) {
+ char* msg;
+ gpr_asprintf(&msg, "Verify peer callback returned a failure (%d)",
+ callback_status);
+ error = GRPC_ERROR_CREATE_FROM_COPIED_STRING(msg);
+ gpr_free(msg);
+ }
+ }
}
+ *auth_context = grpc_ssl_peer_to_auth_context(&peer);
}
}
GRPC_CLOSURE_SCHED(on_peer_checked, error);
@@ -184,34 +139,16 @@ class grpc_ssl_channel_security_connector final
reinterpret_cast<const grpc_ssl_channel_security_connector*>(other_sc);
int c = channel_security_connector_cmp(other);
if (c != 0) return c;
- c = strcmp(target_name_, other->target_name_);
- if (c != 0) return c;
- return (overridden_target_name_ == nullptr ||
- other->overridden_target_name_ == nullptr)
- ? GPR_ICMP(overridden_target_name_,
- other->overridden_target_name_)
- : strcmp(overridden_target_name_,
- other->overridden_target_name_);
+ return grpc_ssl_cmp_target_name(target_name_, other->target_name_,
+ overridden_target_name_,
+ other->overridden_target_name_);
}
bool check_call_host(const char* host, grpc_auth_context* auth_context,
grpc_closure* on_call_host_checked,
grpc_error** error) override {
- grpc_security_status status = GRPC_SECURITY_ERROR;
- tsi_peer peer = grpc_shallow_peer_from_ssl_auth_context(auth_context);
- if (grpc_ssl_host_matches_name(&peer, host)) status = GRPC_SECURITY_OK;
- /* If the target name was overridden, then the original target_name was
- 'checked' transitively during the previous peer check at the end of the
- handshake. */
- if (overridden_target_name_ != nullptr && strcmp(host, target_name_) == 0) {
- status = GRPC_SECURITY_OK;
- }
- if (status != GRPC_SECURITY_OK) {
- *error = GRPC_ERROR_CREATE_FROM_STATIC_STRING(
- "call host does not match SSL server name");
- }
- grpc_shallow_peer_destruct(&peer);
- return true;
+ return grpc_ssl_check_call_host(host, target_name_, overridden_target_name_,
+ auth_context, on_call_host_checked, error);
}
void cancel_check_call_host(grpc_closure* on_call_host_checked,
@@ -248,43 +185,25 @@ class grpc_ssl_server_security_connector
}
grpc_security_status InitializeHandshakerFactory() {
+ grpc_security_status retval = GRPC_SECURITY_OK;
if (has_cert_config_fetcher()) {
// Load initial credentials from certificate_config_fetcher:
if (!try_fetch_ssl_server_credentials()) {
gpr_log(GPR_ERROR,
"Failed loading SSL server credentials from fetcher.");
- return GRPC_SECURITY_ERROR;
+ retval = GRPC_SECURITY_ERROR;
}
} else {
auto* server_credentials =
static_cast<const grpc_ssl_server_credentials*>(server_creds());
- size_t num_alpn_protocols = 0;
- const char** alpn_protocol_strings =
- grpc_fill_alpn_protocol_strings(&num_alpn_protocols);
- tsi_ssl_server_handshaker_options options;
- options.pem_key_cert_pairs =
- server_credentials->config().pem_key_cert_pairs;
- options.num_key_cert_pairs =
- server_credentials->config().num_key_cert_pairs;
- options.pem_client_root_certs =
- server_credentials->config().pem_root_certs;
- options.client_certificate_request =
- grpc_get_tsi_client_certificate_request_type(
- server_credentials->config().client_certificate_request);
- options.cipher_suites = grpc_get_ssl_cipher_suites();
- options.alpn_protocols = alpn_protocol_strings;
- options.num_alpn_protocols = static_cast<uint16_t>(num_alpn_protocols);
- const tsi_result result =
- tsi_create_ssl_server_handshaker_factory_with_options(
- &options, &server_handshaker_factory_);
- gpr_free((void*)alpn_protocol_strings);
- if (result != TSI_OK) {
- gpr_log(GPR_ERROR, "Handshaker factory creation failed with %s.",
- tsi_result_to_string(result));
- return GRPC_SECURITY_ERROR;
- }
+ retval = grpc_ssl_tsi_server_handshaker_factory_init(
+ server_credentials->config().pem_key_cert_pairs,
+ server_credentials->config().num_key_cert_pairs,
+ server_credentials->config().pem_root_certs,
+ server_credentials->config().client_certificate_request,
+ &server_handshaker_factory_);
}
- return GRPC_SECURITY_OK;
+ return retval;
}
void add_handshakers(grpc_pollset_set* interested_parties,
@@ -306,7 +225,8 @@ class grpc_ssl_server_security_connector
void check_peer(tsi_peer peer, grpc_endpoint* ep,
grpc_core::RefCountedPtr<grpc_auth_context>* auth_context,
grpc_closure* on_peer_checked) override {
- grpc_error* error = ssl_check_peer(nullptr, &peer, auth_context);
+ grpc_error* error = grpc_ssl_check_alpn(&peer);
+ *auth_context = grpc_ssl_peer_to_auth_context(&peer);
tsi_peer_destruct(&peer);
GRPC_CLOSURE_SCHED(on_peer_checked, error);
}
@@ -323,9 +243,7 @@ class grpc_ssl_server_security_connector
bool try_fetch_ssl_server_credentials() {
grpc_ssl_server_certificate_config* certificate_config = nullptr;
bool status;
-
if (!has_cert_config_fetcher()) return false;
-
grpc_ssl_server_credentials* server_creds =
static_cast<grpc_ssl_server_credentials*>(this->mutable_server_creds());
grpc_ssl_certificate_config_reload_status cb_result =
@@ -342,7 +260,6 @@ class grpc_ssl_server_security_connector
"use previously-loaded credentials.");
status = false;
}
-
if (certificate_config != nullptr) {
grpc_ssl_server_certificate_config_destroy(certificate_config);
}
@@ -361,34 +278,18 @@ class grpc_ssl_server_security_connector
"config.");
return false;
}
- gpr_log(GPR_DEBUG, "Using new server certificate config (%p).", config);
-
- size_t num_alpn_protocols = 0;
- const char** alpn_protocol_strings =
- grpc_fill_alpn_protocol_strings(&num_alpn_protocols);
- tsi_ssl_server_handshaker_factory* new_handshaker_factory = nullptr;
- const grpc_ssl_server_credentials* server_creds =
+ tsi_ssl_pem_key_cert_pair* pem_key_cert_pairs =
+ grpc_convert_grpc_to_tsi_cert_pairs(config->pem_key_cert_pairs,
+ config->num_key_cert_pairs);
+ const grpc_ssl_server_credentials* server_credentials =
static_cast<const grpc_ssl_server_credentials*>(this->server_creds());
- GPR_DEBUG_ASSERT(config->pem_root_certs != nullptr);
- tsi_ssl_server_handshaker_options options;
- options.pem_key_cert_pairs = grpc_convert_grpc_to_tsi_cert_pairs(
- config->pem_key_cert_pairs, config->num_key_cert_pairs);
- options.num_key_cert_pairs = config->num_key_cert_pairs;
- options.pem_client_root_certs = config->pem_root_certs;
- options.client_certificate_request =
- grpc_get_tsi_client_certificate_request_type(
- server_creds->config().client_certificate_request);
- options.cipher_suites = grpc_get_ssl_cipher_suites();
- options.alpn_protocols = alpn_protocol_strings;
- options.num_alpn_protocols = static_cast<uint16_t>(num_alpn_protocols);
- tsi_result result = tsi_create_ssl_server_handshaker_factory_with_options(
- &options, &new_handshaker_factory);
- gpr_free((void*)options.pem_key_cert_pairs);
- gpr_free((void*)alpn_protocol_strings);
-
- if (result != TSI_OK) {
- gpr_log(GPR_ERROR, "Handshaker factory creation failed with %s.",
- tsi_result_to_string(result));
+ tsi_ssl_server_handshaker_factory* new_handshaker_factory = nullptr;
+ grpc_security_status retval = grpc_ssl_tsi_server_handshaker_factory_init(
+ pem_key_cert_pairs, config->num_key_cert_pairs, config->pem_root_certs,
+ server_credentials->config().client_certificate_request,
+ &new_handshaker_factory);
+ gpr_free(pem_key_cert_pairs);
+ if (retval != GRPC_SECURITY_OK) {
return false;
}
set_server_handshaker_factory(new_handshaker_factory);
@@ -418,28 +319,12 @@ grpc_ssl_channel_security_connector_create(
gpr_log(GPR_ERROR, "An ssl channel needs a config and a target name.");
return nullptr;
}
-
- const char* pem_root_certs;
- const tsi_ssl_root_certs_store* root_store;
- if (config->pem_root_certs == nullptr) {
- // Use default root certificates.
- pem_root_certs = grpc_core::DefaultSslRootStore::GetPemRootCerts();
- if (pem_root_certs == nullptr) {
- gpr_log(GPR_ERROR, "Could not get default pem root certs.");
- return nullptr;
- }
- root_store = grpc_core::DefaultSslRootStore::GetRootStore();
- } else {
- pem_root_certs = config->pem_root_certs;
- root_store = nullptr;
- }
-
grpc_core::RefCountedPtr<grpc_ssl_channel_security_connector> c =
grpc_core::MakeRefCounted<grpc_ssl_channel_security_connector>(
std::move(channel_creds), std::move(request_metadata_creds), config,
target_name, overridden_target_name);
- const grpc_security_status result = c->InitializeHandshakerFactory(
- config, pem_root_certs, root_store, ssl_session_cache);
+ const grpc_security_status result =
+ c->InitializeHandshakerFactory(config, ssl_session_cache);
if (result != GRPC_SECURITY_OK) {
return nullptr;
}
diff --git a/src/core/lib/security/security_connector/ssl_utils.cc b/src/core/lib/security/security_connector/ssl_utils.cc
index 29030f07ad6..c9af5ca6ad0 100644
--- a/src/core/lib/security/security_connector/ssl_utils.cc
+++ b/src/core/lib/security/security_connector/ssl_utils.cc
@@ -112,6 +112,55 @@ grpc_get_tsi_client_certificate_request_type(
}
}
+grpc_error* grpc_ssl_check_alpn(const tsi_peer* peer) {
+#if TSI_OPENSSL_ALPN_SUPPORT
+ /* Check the ALPN if ALPN is supported. */
+ const tsi_peer_property* p =
+ tsi_peer_get_property_by_name(peer, TSI_SSL_ALPN_SELECTED_PROTOCOL);
+ if (p == nullptr) {
+ return GRPC_ERROR_CREATE_FROM_STATIC_STRING(
+ "Cannot check peer: missing selected ALPN property.");
+ }
+ if (!grpc_chttp2_is_alpn_version_supported(p->value.data, p->value.length)) {
+ return GRPC_ERROR_CREATE_FROM_STATIC_STRING(
+ "Cannot check peer: invalid ALPN value.");
+ }
+#endif /* TSI_OPENSSL_ALPN_SUPPORT */
+ return GRPC_ERROR_NONE;
+}
+
+grpc_error* grpc_ssl_check_peer_name(const char* peer_name,
+ const tsi_peer* peer) {
+ /* Check the peer name if specified. */
+ if (peer_name != nullptr && !grpc_ssl_host_matches_name(peer, peer_name)) {
+ char* msg;
+ gpr_asprintf(&msg, "Peer name %s is not in peer certificate", peer_name);
+ grpc_error* error = GRPC_ERROR_CREATE_FROM_COPIED_STRING(msg);
+ gpr_free(msg);
+ return error;
+ }
+ return GRPC_ERROR_NONE;
+}
+
+bool grpc_ssl_check_call_host(const char* host, const char* target_name,
+ const char* overridden_target_name,
+ grpc_auth_context* auth_context,
+ grpc_closure* on_call_host_checked,
+ grpc_error** error) {
+ grpc_security_status status = GRPC_SECURITY_ERROR;
+ tsi_peer peer = grpc_shallow_peer_from_ssl_auth_context(auth_context);
+ if (grpc_ssl_host_matches_name(&peer, host)) status = GRPC_SECURITY_OK;
+ if (overridden_target_name != nullptr && strcmp(host, target_name) == 0) {
+ status = GRPC_SECURITY_OK;
+ }
+ if (status != GRPC_SECURITY_OK) {
+ *error = GRPC_ERROR_CREATE_FROM_STATIC_STRING(
+ "call host does not match SSL server name");
+ }
+ grpc_shallow_peer_destruct(&peer);
+ return true;
+}
+
const char** grpc_fill_alpn_protocol_strings(size_t* num_alpn_protocols) {
GPR_ASSERT(num_alpn_protocols != nullptr);
*num_alpn_protocols = grpc_chttp2_num_alpn_versions();
@@ -142,6 +191,18 @@ int grpc_ssl_host_matches_name(const tsi_peer* peer, const char* peer_name) {
return r;
}
+bool grpc_ssl_cmp_target_name(const char* target_name,
+ const char* other_target_name,
+ const char* overridden_target_name,
+ const char* other_overridden_target_name) {
+ int c = strcmp(target_name, other_target_name);
+ if (c != 0) return c;
+ return (overridden_target_name == nullptr ||
+ other_overridden_target_name == nullptr)
+ ? GPR_ICMP(overridden_target_name, other_overridden_target_name)
+ : strcmp(overridden_target_name, other_overridden_target_name);
+}
+
grpc_core::RefCountedPtr<grpc_auth_context> grpc_ssl_peer_to_auth_context(
const tsi_peer* peer) {
size_t i;
@@ -230,6 +291,79 @@ void grpc_shallow_peer_destruct(tsi_peer* peer) {
if (peer->properties != nullptr) gpr_free(peer->properties);
}
+grpc_security_status grpc_ssl_tsi_client_handshaker_factory_init(
+ tsi_ssl_pem_key_cert_pair* pem_key_cert_pair, const char* pem_root_certs,
+ tsi_ssl_session_cache* ssl_session_cache,
+ tsi_ssl_client_handshaker_factory** handshaker_factory) {
+ const char* root_certs;
+ const tsi_ssl_root_certs_store* root_store;
+ if (pem_root_certs == nullptr) {
+ // Use default root certificates.
+ root_certs = grpc_core::DefaultSslRootStore::GetPemRootCerts();
+ if (root_certs == nullptr) {
+ gpr_log(GPR_ERROR, "Could not get default pem root certs.");
+ return GRPC_SECURITY_ERROR;
+ }
+ root_store = grpc_core::DefaultSslRootStore::GetRootStore();
+ } else {
+ root_certs = pem_root_certs;
+ root_store = nullptr;
+ }
+ bool has_key_cert_pair = pem_key_cert_pair != nullptr &&
+ pem_key_cert_pair->private_key != nullptr &&
+ pem_key_cert_pair->cert_chain != nullptr;
+ tsi_ssl_client_handshaker_options options;
+ GPR_DEBUG_ASSERT(root_certs != nullptr);
+ options.pem_root_certs = root_certs;
+ options.root_store = root_store;
+ options.alpn_protocols =
+ grpc_fill_alpn_protocol_strings(&options.num_alpn_protocols);
+ if (has_key_cert_pair) {
+ options.pem_key_cert_pair = pem_key_cert_pair;
+ }
+ options.cipher_suites = grpc_get_ssl_cipher_suites();
+ options.session_cache = ssl_session_cache;
+ const tsi_result result =
+ tsi_create_ssl_client_handshaker_factory_with_options(&options,
+ handshaker_factory);
+ gpr_free((void*)options.alpn_protocols);
+ if (result != TSI_OK) {
+ gpr_log(GPR_ERROR, "Handshaker factory creation failed with %s.",
+ tsi_result_to_string(result));
+ return GRPC_SECURITY_ERROR;
+ }
+ return GRPC_SECURITY_OK;
+}
+
+grpc_security_status grpc_ssl_tsi_server_handshaker_factory_init(
+ tsi_ssl_pem_key_cert_pair* pem_key_cert_pairs, size_t num_key_cert_pairs,
+ const char* pem_root_certs,
+ grpc_ssl_client_certificate_request_type client_certificate_request,
+ tsi_ssl_server_handshaker_factory** handshaker_factory) {
+ size_t num_alpn_protocols = 0;
+ const char** alpn_protocol_strings =
+ grpc_fill_alpn_protocol_strings(&num_alpn_protocols);
+ tsi_ssl_server_handshaker_options options;
+ options.pem_key_cert_pairs = pem_key_cert_pairs;
+ options.num_key_cert_pairs = num_key_cert_pairs;
+ options.pem_client_root_certs = pem_root_certs;
+ options.client_certificate_request =
+ grpc_get_tsi_client_certificate_request_type(client_certificate_request);
+ options.cipher_suites = grpc_get_ssl_cipher_suites();
+ options.alpn_protocols = alpn_protocol_strings;
+ options.num_alpn_protocols = static_cast<uint16_t>(num_alpn_protocols);
+ const tsi_result result =
+ tsi_create_ssl_server_handshaker_factory_with_options(&options,
+ handshaker_factory);
+ gpr_free((void*)alpn_protocol_strings);
+ if (result != TSI_OK) {
+ gpr_log(GPR_ERROR, "Handshaker factory creation failed with %s.",
+ tsi_result_to_string(result));
+ return GRPC_SECURITY_ERROR;
+ }
+ return GRPC_SECURITY_OK;
+}
+
/* --- Ssl cache implementation. --- */
grpc_ssl_session_cache* grpc_ssl_session_cache_create_lru(size_t capacity) {
diff --git a/src/core/lib/security/security_connector/ssl_utils.h b/src/core/lib/security/security_connector/ssl_utils.h
index 972ca439dea..080e277f944 100644
--- a/src/core/lib/security/security_connector/ssl_utils.h
+++ b/src/core/lib/security/security_connector/ssl_utils.h
@@ -27,7 +27,10 @@
#include <grpc/slice_buffer.h>
#include "src/core/lib/gprpp/ref_counted_ptr.h"
+#include "src/core/lib/iomgr/error.h"
+#include "src/core/lib/security/security_connector/security_connector.h"
#include "src/core/tsi/ssl_transport_security.h"
+#include "src/core/tsi/transport_security.h"
#include "src/core/tsi/transport_security_interface.h"
/* --- Util. --- */
@@ -35,6 +38,23 @@
/* --- URL schemes. --- */
#define GRPC_SSL_URL_SCHEME "https"
+/* Check ALPN information returned from SSL handshakes. */
+grpc_error* grpc_ssl_check_alpn(const tsi_peer* peer);
+
+/* Check peer name information returned from SSL handshakes. */
+grpc_error* grpc_ssl_check_peer_name(const char* peer_name,
+ const tsi_peer* peer);
+/* Compare targer_name information extracted from SSL security connectors. */
+bool grpc_ssl_cmp_target_name(const char* target_name,
+ const char* other_target_name,
+ const char* overridden_target_name,
+ const char* other_overridden_target_name);
+/* Check the host that will be set for a call is acceptable.*/
+bool grpc_ssl_check_call_host(const char* host, const char* target_name,
+ const char* overridden_target_name,
+ grpc_auth_context* auth_context,
+ grpc_closure* on_call_host_checked,
+ grpc_error** error);
/* Return HTTP2-compliant cipher suites that gRPC accepts by default. */
const char* grpc_get_ssl_cipher_suites(void);
@@ -47,6 +67,18 @@ grpc_get_tsi_client_certificate_request_type(
/* Return an array of strings containing alpn protocols. */
const char** grpc_fill_alpn_protocol_strings(size_t* num_alpn_protocols);
+/* Initialize TSI SSL server/client handshaker factory. */
+grpc_security_status grpc_ssl_tsi_client_handshaker_factory_init(
+ tsi_ssl_pem_key_cert_pair* key_cert_pair, const char* pem_root_certs,
+ tsi_ssl_session_cache* ssl_session_cache,
+ tsi_ssl_client_handshaker_factory** handshaker_factory);
+
+grpc_security_status grpc_ssl_tsi_server_handshaker_factory_init(
+ tsi_ssl_pem_key_cert_pair* key_cert_pairs, size_t num_key_cert_pairs,
+ const char* pem_root_certs,
+ grpc_ssl_client_certificate_request_type client_certificate_request,
+ tsi_ssl_server_handshaker_factory** handshaker_factory);
+
/* Exposed for testing only. */
grpc_core::RefCountedPtr<grpc_auth_context> grpc_ssl_peer_to_auth_context(
const tsi_peer* peer);
diff --git a/src/core/lib/security/security_connector/tls/spiffe_security_connector.cc b/src/core/lib/security/security_connector/tls/spiffe_security_connector.cc
new file mode 100644
index 00000000000..075b1c9d53c
--- /dev/null
+++ b/src/core/lib/security/security_connector/tls/spiffe_security_connector.cc
@@ -0,0 +1,426 @@
+/*
+ *
+ * Copyright 2018 gRPC authors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+#include <grpc/support/port_platform.h>
+
+#include "src/core/lib/security/security_connector/tls/spiffe_security_connector.h"
+
+#include <stdbool.h>
+#include <string.h>
+
+#include <grpc/grpc.h>
+#include <grpc/support/alloc.h>
+#include <grpc/support/log.h>
+#include <grpc/support/string_util.h>
+
+#include "src/core/lib/gpr/host_port.h"
+#include "src/core/lib/security/credentials/ssl/ssl_credentials.h"
+#include "src/core/lib/security/credentials/tls/spiffe_credentials.h"
+#include "src/core/lib/security/security_connector/ssl_utils.h"
+#include "src/core/lib/security/transport/security_handshaker.h"
+#include "src/core/lib/slice/slice_internal.h"
+#include "src/core/lib/transport/transport.h"
+#include "src/core/tsi/ssl_transport_security.h"
+#include "src/core/tsi/transport_security.h"
+
+namespace {
+
+tsi_ssl_pem_key_cert_pair* ConvertToTsiPemKeyCertPair(
+ const grpc_tls_key_materials_config::PemKeyCertPairList& cert_pair_list) {
+ tsi_ssl_pem_key_cert_pair* tsi_pairs = nullptr;
+ size_t num_key_cert_pairs = cert_pair_list.size();
+ if (num_key_cert_pairs > 0) {
+ GPR_ASSERT(cert_pair_list.data() != nullptr);
+ tsi_pairs = static_cast<tsi_ssl_pem_key_cert_pair*>(
+ gpr_zalloc(num_key_cert_pairs * sizeof(tsi_ssl_pem_key_cert_pair)));
+ }
+ for (size_t i = 0; i < num_key_cert_pairs; i++) {
+ GPR_ASSERT(cert_pair_list[i].private_key() != nullptr);
+ GPR_ASSERT(cert_pair_list[i].cert_chain() != nullptr);
+ tsi_pairs[i].cert_chain = gpr_strdup(cert_pair_list[i].cert_chain());
+ tsi_pairs[i].private_key = gpr_strdup(cert_pair_list[i].private_key());
+ }
+ return tsi_pairs;
+}
+
+/** -- Util function to populate SPIFFE server/channel credentials. -- */
+grpc_core::RefCountedPtr<grpc_tls_key_materials_config>
+PopulateSpiffeCredentials(const grpc_tls_credentials_options& options) {
+ GPR_ASSERT(options.credential_reload_config() != nullptr ||
+ options.key_materials_config() != nullptr);
+ grpc_core::RefCountedPtr<grpc_tls_key_materials_config> key_materials_config;
+ /* Use credential reload config to fetch credentials. */
+ if (options.credential_reload_config() != nullptr) {
+ grpc_tls_credential_reload_arg* arg =
+ grpc_core::New<grpc_tls_credential_reload_arg>();
+ key_materials_config = grpc_tls_key_materials_config_create()->Ref();
+ arg->key_materials_config = key_materials_config.get();
+ int result = options.credential_reload_config()->Schedule(arg);
+ if (result) {
+ /* Do not support async credential reload. */
+ gpr_log(GPR_ERROR, "Async credential reload is unsupported now.");
+ } else {
+ grpc_ssl_certificate_config_reload_status status = arg->status;
+ if (status == GRPC_SSL_CERTIFICATE_CONFIG_RELOAD_UNCHANGED) {
+ gpr_log(GPR_DEBUG, "Credential does not change after reload.");
+ } else if (status == GRPC_SSL_CERTIFICATE_CONFIG_RELOAD_FAIL) {
+ gpr_log(GPR_ERROR, "Credential reload failed with an error: %s",
+ arg->error_details);
+ }
+ }
+ gpr_free((void*)arg->error_details);
+ grpc_core::Delete(arg);
+ /* Use existing key materials config. */
+ } else {
+ key_materials_config = options.key_materials_config()->Ref();
+ }
+ return key_materials_config;
+}
+
+} // namespace
+
+SpiffeChannelSecurityConnector::SpiffeChannelSecurityConnector(
+ grpc_core::RefCountedPtr<grpc_channel_credentials> channel_creds,
+ grpc_core::RefCountedPtr<grpc_call_credentials> request_metadata_creds,
+ const char* target_name, const char* overridden_target_name)
+ : grpc_channel_security_connector(GRPC_SSL_URL_SCHEME,
+ std::move(channel_creds),
+ std::move(request_metadata_creds)),
+ overridden_target_name_(overridden_target_name == nullptr
+ ? nullptr
+ : gpr_strdup(overridden_target_name)) {
+ check_arg_ = ServerAuthorizationCheckArgCreate(this);
+ char* port;
+ gpr_split_host_port(target_name, &target_name_, &port);
+ gpr_free(port);
+}
+
+SpiffeChannelSecurityConnector::~SpiffeChannelSecurityConnector() {
+ if (target_name_ != nullptr) {
+ gpr_free(target_name_);
+ }
+ if (overridden_target_name_ != nullptr) {
+ gpr_free(overridden_target_name_);
+ }
+ if (client_handshaker_factory_ != nullptr) {
+ tsi_ssl_client_handshaker_factory_unref(client_handshaker_factory_);
+ }
+ ServerAuthorizationCheckArgDestroy(check_arg_);
+}
+
+void SpiffeChannelSecurityConnector::add_handshakers(
+ grpc_pollset_set* interested_parties,
+ grpc_core::HandshakeManager* handshake_mgr) {
+ // Instantiate TSI handshaker.
+ tsi_handshaker* tsi_hs = nullptr;
+ tsi_result result = tsi_ssl_client_handshaker_factory_create_handshaker(
+ client_handshaker_factory_,
+ overridden_target_name_ != nullptr ? overridden_target_name_
+ : target_name_,
+ &tsi_hs);
+ if (result != TSI_OK) {
+ gpr_log(GPR_ERROR, "Handshaker creation failed with error %s.",
+ tsi_result_to_string(result));
+ return;
+ }
+ // Create handshakers.
+ handshake_mgr->Add(grpc_core::SecurityHandshakerCreate(tsi_hs, this));
+}
+
+void SpiffeChannelSecurityConnector::check_peer(
+ tsi_peer peer, grpc_endpoint* ep,
+ grpc_core::RefCountedPtr<grpc_auth_context>* auth_context,
+ grpc_closure* on_peer_checked) {
+ const char* target_name = overridden_target_name_ != nullptr
+ ? overridden_target_name_
+ : target_name_;
+ grpc_error* error = grpc_ssl_check_alpn(&peer);
+ if (error != GRPC_ERROR_NONE) {
+ GRPC_CLOSURE_SCHED(on_peer_checked, error);
+ tsi_peer_destruct(&peer);
+ return;
+ }
+ *auth_context = grpc_ssl_peer_to_auth_context(&peer);
+ const SpiffeCredentials* creds =
+ static_cast<const SpiffeCredentials*>(channel_creds());
+ const grpc_tls_server_authorization_check_config* config =
+ creds->options().server_authorization_check_config();
+ /* If server authorization config is not null, use it to perform
+ * server authorizaiton check. */
+ if (config != nullptr) {
+ const tsi_peer_property* p =
+ tsi_peer_get_property_by_name(&peer, TSI_X509_PEM_CERT_PROPERTY);
+ if (p == nullptr) {
+ error = GRPC_ERROR_CREATE_FROM_STATIC_STRING(
+ "Cannot check peer: missing pem cert property.");
+ } else {
+ char* peer_pem = static_cast<char*>(gpr_malloc(p->value.length + 1));
+ memcpy(peer_pem, p->value.data, p->value.length);
+ peer_pem[p->value.length] = '\0';
+ GPR_ASSERT(check_arg_ != nullptr);
+ check_arg_->peer_cert = check_arg_->peer_cert == nullptr
+ ? gpr_strdup(peer_pem)
+ : check_arg_->peer_cert;
+ check_arg_->target_name = check_arg_->target_name == nullptr
+ ? gpr_strdup(target_name)
+ : check_arg_->target_name;
+ on_peer_checked_ = on_peer_checked;
+ gpr_free(peer_pem);
+ int callback_status = config->Schedule(check_arg_);
+ /* Server authorization check is handled asynchronously. */
+ if (callback_status) {
+ tsi_peer_destruct(&peer);
+ return;
+ }
+ /* Server authorization check is handled synchronously. */
+ error = ProcessServerAuthorizationCheckResult(check_arg_);
+ }
+ }
+ GRPC_CLOSURE_SCHED(on_peer_checked, error);
+ tsi_peer_destruct(&peer);
+}
+
+int SpiffeChannelSecurityConnector::cmp(
+ const grpc_security_connector* other_sc) const {
+ auto* other =
+ reinterpret_cast<const SpiffeChannelSecurityConnector*>(other_sc);
+ int c = channel_security_connector_cmp(other);
+ if (c != 0) {
+ return c;
+ }
+ return grpc_ssl_cmp_target_name(target_name_, other->target_name_,
+ overridden_target_name_,
+ other->overridden_target_name_);
+}
+
+bool SpiffeChannelSecurityConnector::check_call_host(
+ const char* host, grpc_auth_context* auth_context,
+ grpc_closure* on_call_host_checked, grpc_error** error) {
+ return grpc_ssl_check_call_host(host, target_name_, overridden_target_name_,
+ auth_context, on_call_host_checked, error);
+}
+
+void SpiffeChannelSecurityConnector::cancel_check_call_host(
+ grpc_closure* on_call_host_checked, grpc_error* error) {
+ GRPC_ERROR_UNREF(error);
+}
+
+grpc_core::RefCountedPtr<grpc_channel_security_connector>
+SpiffeChannelSecurityConnector::CreateSpiffeChannelSecurityConnector(
+ grpc_core::RefCountedPtr<grpc_channel_credentials> channel_creds,
+ grpc_core::RefCountedPtr<grpc_call_credentials> request_metadata_creds,
+ const char* target_name, const char* overridden_target_name,
+ tsi_ssl_session_cache* ssl_session_cache) {
+ if (channel_creds == nullptr) {
+ gpr_log(GPR_ERROR,
+ "channel_creds is nullptr in "
+ "SpiffeChannelSecurityConnectorCreate()");
+ return nullptr;
+ }
+ if (target_name == nullptr) {
+ gpr_log(GPR_ERROR,
+ "target_name is nullptr in "
+ "SpiffeChannelSecurityConnectorCreate()");
+ return nullptr;
+ }
+ grpc_core::RefCountedPtr<SpiffeChannelSecurityConnector> c =
+ grpc_core::MakeRefCounted<SpiffeChannelSecurityConnector>(
+ std::move(channel_creds), std::move(request_metadata_creds),
+ target_name, overridden_target_name);
+ if (c->InitializeHandshakerFactory(ssl_session_cache) != GRPC_SECURITY_OK) {
+ return nullptr;
+ }
+ return c;
+}
+
+grpc_security_status
+SpiffeChannelSecurityConnector::InitializeHandshakerFactory(
+ tsi_ssl_session_cache* ssl_session_cache) {
+ const SpiffeCredentials* creds =
+ static_cast<const SpiffeCredentials*>(channel_creds());
+ auto key_materials_config = PopulateSpiffeCredentials(creds->options());
+ if (!key_materials_config.get()->pem_key_cert_pair_list().size()) {
+ key_materials_config.get()->Unref();
+ return GRPC_SECURITY_ERROR;
+ }
+ tsi_ssl_pem_key_cert_pair* pem_key_cert_pair = ConvertToTsiPemKeyCertPair(
+ key_materials_config.get()->pem_key_cert_pair_list());
+ grpc_security_status status = grpc_ssl_tsi_client_handshaker_factory_init(
+ pem_key_cert_pair, key_materials_config.get()->pem_root_certs(),
+ ssl_session_cache, &client_handshaker_factory_);
+ // Free memory.
+ key_materials_config.get()->Unref();
+ grpc_tsi_ssl_pem_key_cert_pairs_destroy(pem_key_cert_pair, 1);
+ return status;
+}
+
+void SpiffeChannelSecurityConnector::ServerAuthorizationCheckDone(
+ grpc_tls_server_authorization_check_arg* arg) {
+ GPR_ASSERT(arg != nullptr);
+ grpc_core::ExecCtx exec_ctx;
+ grpc_error* error = ProcessServerAuthorizationCheckResult(arg);
+ SpiffeChannelSecurityConnector* connector =
+ static_cast<SpiffeChannelSecurityConnector*>(arg->cb_user_data);
+ GRPC_CLOSURE_SCHED(connector->on_peer_checked_, error);
+}
+
+grpc_error*
+SpiffeChannelSecurityConnector::ProcessServerAuthorizationCheckResult(
+ grpc_tls_server_authorization_check_arg* arg) {
+ grpc_error* error = GRPC_ERROR_NONE;
+ char* msg = nullptr;
+ /* Server authorization check is cancelled by caller. */
+ if (arg->status == GRPC_STATUS_CANCELLED) {
+ gpr_asprintf(&msg,
+ "Server authorization check is cancelled by the caller with "
+ "error: %s",
+ arg->error_details);
+ error = GRPC_ERROR_CREATE_FROM_COPIED_STRING(msg);
+ } else if (arg->status == GRPC_STATUS_OK) {
+ /* Server authorization check completed successfully but returned check
+ * failure. */
+ if (!arg->success) {
+ gpr_asprintf(&msg, "Server authorization check failed with error: %s",
+ arg->error_details);
+ error = GRPC_ERROR_CREATE_FROM_COPIED_STRING(msg);
+ }
+ /* Server authorization check did not complete correctly. */
+ } else {
+ gpr_asprintf(
+ &msg,
+ "Server authorization check did not finish correctly with error: %s",
+ arg->error_details);
+ error = GRPC_ERROR_CREATE_FROM_COPIED_STRING(msg);
+ }
+ gpr_free(msg);
+ return error;
+}
+
+grpc_tls_server_authorization_check_arg*
+SpiffeChannelSecurityConnector::ServerAuthorizationCheckArgCreate(
+ void* user_data) {
+ grpc_tls_server_authorization_check_arg* arg =
+ grpc_core::New<grpc_tls_server_authorization_check_arg>();
+ arg->cb = ServerAuthorizationCheckDone;
+ arg->cb_user_data = user_data;
+ arg->status = GRPC_STATUS_OK;
+ return arg;
+}
+
+void SpiffeChannelSecurityConnector::ServerAuthorizationCheckArgDestroy(
+ grpc_tls_server_authorization_check_arg* arg) {
+ if (arg == nullptr) {
+ return;
+ }
+ gpr_free((void*)arg->target_name);
+ gpr_free((void*)arg->peer_cert);
+ gpr_free((void*)arg->error_details);
+ grpc_core::Delete(arg);
+}
+
+SpiffeServerSecurityConnector::SpiffeServerSecurityConnector(
+ grpc_core::RefCountedPtr<grpc_server_credentials> server_creds)
+ : grpc_server_security_connector(GRPC_SSL_URL_SCHEME,
+ std::move(server_creds)) {}
+
+SpiffeServerSecurityConnector::~SpiffeServerSecurityConnector() {
+ if (server_handshaker_factory_ != nullptr) {
+ tsi_ssl_server_handshaker_factory_unref(server_handshaker_factory_);
+ }
+}
+
+void SpiffeServerSecurityConnector::add_handshakers(
+ grpc_pollset_set* interested_parties,
+ grpc_core::HandshakeManager* handshake_mgr) {
+ /* Create a TLS SPIFFE TSI handshaker for server. */
+ RefreshServerHandshakerFactory();
+ tsi_handshaker* tsi_hs = nullptr;
+ tsi_result result = tsi_ssl_server_handshaker_factory_create_handshaker(
+ server_handshaker_factory_, &tsi_hs);
+ if (result != TSI_OK) {
+ gpr_log(GPR_ERROR, "Handshaker creation failed with error %s.",
+ tsi_result_to_string(result));
+ return;
+ }
+ handshake_mgr->Add(grpc_core::SecurityHandshakerCreate(tsi_hs, this));
+}
+
+void SpiffeServerSecurityConnector::check_peer(
+ tsi_peer peer, grpc_endpoint* ep,
+ grpc_core::RefCountedPtr<grpc_auth_context>* auth_context,
+ grpc_closure* on_peer_checked) {
+ grpc_error* error = grpc_ssl_check_alpn(&peer);
+ *auth_context = grpc_ssl_peer_to_auth_context(&peer);
+ tsi_peer_destruct(&peer);
+ GRPC_CLOSURE_SCHED(on_peer_checked, error);
+}
+
+int SpiffeServerSecurityConnector::cmp(
+ const grpc_security_connector* other) const {
+ return server_security_connector_cmp(
+ static_cast<const grpc_server_security_connector*>(other));
+}
+
+grpc_core::RefCountedPtr<grpc_server_security_connector>
+SpiffeServerSecurityConnector::CreateSpiffeServerSecurityConnector(
+ grpc_core::RefCountedPtr<grpc_server_credentials> server_creds) {
+ if (server_creds == nullptr) {
+ gpr_log(GPR_ERROR,
+ "server_creds is nullptr in "
+ "SpiffeServerSecurityConnectorCreate()");
+ return nullptr;
+ }
+ grpc_core::RefCountedPtr<SpiffeServerSecurityConnector> c =
+ grpc_core::MakeRefCounted<SpiffeServerSecurityConnector>(
+ std::move(server_creds));
+ if (c->RefreshServerHandshakerFactory() != GRPC_SECURITY_OK) {
+ return nullptr;
+ }
+ return c;
+}
+
+grpc_security_status
+SpiffeServerSecurityConnector::RefreshServerHandshakerFactory() {
+ const SpiffeServerCredentials* creds =
+ static_cast<const SpiffeServerCredentials*>(server_creds());
+ auto key_materials_config = PopulateSpiffeCredentials(creds->options());
+ /* Credential reload does NOT take effect and we need to keep using
+ * the existing handshaker factory. */
+ if (key_materials_config.get()->pem_key_cert_pair_list().empty()) {
+ key_materials_config.get()->Unref();
+ return GRPC_SECURITY_ERROR;
+ }
+ /* Credential reload takes effect and we need to free the existing
+ * handshaker library. */
+ if (server_handshaker_factory_) {
+ tsi_ssl_server_handshaker_factory_unref(server_handshaker_factory_);
+ }
+ tsi_ssl_pem_key_cert_pair* pem_key_cert_pairs = ConvertToTsiPemKeyCertPair(
+ key_materials_config.get()->pem_key_cert_pair_list());
+ size_t num_key_cert_pairs =
+ key_materials_config.get()->pem_key_cert_pair_list().size();
+ grpc_security_status status = grpc_ssl_tsi_server_handshaker_factory_init(
+ pem_key_cert_pairs, num_key_cert_pairs,
+ key_materials_config.get()->pem_root_certs(),
+ creds->options().cert_request_type(), &server_handshaker_factory_);
+ // Free memory.
+ key_materials_config.get()->Unref();
+ grpc_tsi_ssl_pem_key_cert_pairs_destroy(pem_key_cert_pairs,
+ num_key_cert_pairs);
+ return status;
+}
diff --git a/src/core/lib/security/security_connector/tls/spiffe_security_connector.h b/src/core/lib/security/security_connector/tls/spiffe_security_connector.h
new file mode 100644
index 00000000000..56972153e07
--- /dev/null
+++ b/src/core/lib/security/security_connector/tls/spiffe_security_connector.h
@@ -0,0 +1,122 @@
+/*
+ *
+ * Copyright 2018 gRPC authors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+#ifndef GRPC_CORE_LIB_SECURITY_SECURITY_CONNECTOR_TLS_SPIFFE_SECURITY_CONNECTOR_H
+#define GRPC_CORE_LIB_SECURITY_SECURITY_CONNECTOR_TLS_SPIFFE_SECURITY_CONNECTOR_H
+
+#include <grpc/support/port_platform.h>
+
+#include "src/core/lib/security/context/security_context.h"
+#include "src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h"
+
+#define GRPC_TLS_SPIFFE_TRANSPORT_SECURITY_TYPE "spiffe"
+
+// Spiffe channel security connector.
+class SpiffeChannelSecurityConnector final
+ : public grpc_channel_security_connector {
+ public:
+ // static factory method to create a SPIFFE channel security connector.
+ static grpc_core::RefCountedPtr<grpc_channel_security_connector>
+ CreateSpiffeChannelSecurityConnector(
+ grpc_core::RefCountedPtr<grpc_channel_credentials> channel_creds,
+ grpc_core::RefCountedPtr<grpc_call_credentials> request_metadata_creds,
+ const char* target_name, const char* overridden_target_name,
+ tsi_ssl_session_cache* ssl_session_cache);
+
+ SpiffeChannelSecurityConnector(
+ grpc_core::RefCountedPtr<grpc_channel_credentials> channel_creds,
+ grpc_core::RefCountedPtr<grpc_call_credentials> request_metadata_creds,
+ const char* target_name, const char* overridden_target_name);
+ ~SpiffeChannelSecurityConnector() override;
+
+ void add_handshakers(grpc_pollset_set* interested_parties,
+ grpc_core::HandshakeManager* handshake_mgr) override;
+
+ void check_peer(tsi_peer peer, grpc_endpoint* ep,
+ grpc_core::RefCountedPtr<grpc_auth_context>* auth_context,
+ grpc_closure* on_peer_checked) override;
+
+ int cmp(const grpc_security_connector* other_sc) const override;
+
+ bool check_call_host(const char* host, grpc_auth_context* auth_context,
+ grpc_closure* on_call_host_checked,
+ grpc_error** error) override;
+
+ void cancel_check_call_host(grpc_closure* on_call_host_checked,
+ grpc_error* error) override;
+
+ private:
+ // Initialize SSL TSI client handshaker factory.
+ grpc_security_status InitializeHandshakerFactory(
+ tsi_ssl_session_cache* ssl_session_cache);
+
+ // gRPC-provided callback executed by application, which servers to bring the
+ // control back to gRPC core.
+ static void ServerAuthorizationCheckDone(
+ grpc_tls_server_authorization_check_arg* arg);
+
+ // A util function to process server authorization check result.
+ static grpc_error* ProcessServerAuthorizationCheckResult(
+ grpc_tls_server_authorization_check_arg* arg);
+
+ // A util function to create a server authorization check arg instance.
+ static grpc_tls_server_authorization_check_arg*
+ ServerAuthorizationCheckArgCreate(void* user_data);
+
+ // A util function to destroy a server authorization check arg instance.
+ static void ServerAuthorizationCheckArgDestroy(
+ grpc_tls_server_authorization_check_arg* arg);
+
+ grpc_closure* on_peer_checked_;
+ char* target_name_;
+ char* overridden_target_name_;
+ tsi_ssl_client_handshaker_factory* client_handshaker_factory_ = nullptr;
+ grpc_tls_server_authorization_check_arg* check_arg_;
+};
+
+// Spiffe server security connector.
+class SpiffeServerSecurityConnector final
+ : public grpc_server_security_connector {
+ public:
+ // static factory method to create a SPIFFE server security connector.
+ static grpc_core::RefCountedPtr<grpc_server_security_connector>
+ CreateSpiffeServerSecurityConnector(
+ grpc_core::RefCountedPtr<grpc_server_credentials> server_creds);
+
+ explicit SpiffeServerSecurityConnector(
+ grpc_core::RefCountedPtr<grpc_server_credentials> server_creds);
+ ~SpiffeServerSecurityConnector() override;
+
+ void add_handshakers(grpc_pollset_set* interested_parties,
+ grpc_core::HandshakeManager* handshake_mgr) override;
+
+ void check_peer(tsi_peer peer, grpc_endpoint* ep,
+ grpc_core::RefCountedPtr<grpc_auth_context>* auth_context,
+ grpc_closure* on_peer_checked) override;
+
+ int cmp(const grpc_security_connector* other) const override;
+
+ private:
+ // A util function to refresh SSL TSI server handshaker factory with a valid
+ // credential.
+ grpc_security_status RefreshServerHandshakerFactory();
+ tsi_ssl_server_handshaker_factory* server_handshaker_factory_ = nullptr;
+};
+
+#endif /* GRPC_CORE_LIB_SECURITY_SECURITY_CONNECTOR_TLS_SPIFFE_SECURITY_CONNECTOR_H \
+ */
diff --git a/src/core/tsi/ssl_transport_security.cc b/src/core/tsi/ssl_transport_security.cc
index 2107bcaa748..9ab76d99c02 100644
--- a/src/core/tsi/ssl_transport_security.cc
+++ b/src/core/tsi/ssl_transport_security.cc
@@ -344,18 +344,24 @@ static tsi_result add_subject_alt_names_properties_to_peer(
size_t subject_alt_name_count) {
size_t i;
tsi_result result = TSI_OK;
-
/* Reset for DNS entries filtering. */
peer->property_count -= subject_alt_name_count;
-
for (i = 0; i < subject_alt_name_count; i++) {
GENERAL_NAME* subject_alt_name =
sk_GENERAL_NAME_value(subject_alt_names, TSI_SIZE_AS_SIZE(i));
- /* Filter out the non-dns entries names. */
- if (subject_alt_name->type == GEN_DNS) {
+ if (subject_alt_name->type == GEN_DNS ||
+ subject_alt_name->type == GEN_EMAIL ||
+ subject_alt_name->type == GEN_URI) {
unsigned char* name = nullptr;
int name_size;
- name_size = ASN1_STRING_to_UTF8(&name, subject_alt_name->d.dNSName);
+ if (subject_alt_name->type == GEN_DNS) {
+ name_size = ASN1_STRING_to_UTF8(&name, subject_alt_name->d.dNSName);
+ } else if (subject_alt_name->type == GEN_EMAIL) {
+ name_size = ASN1_STRING_to_UTF8(&name, subject_alt_name->d.rfc822Name);
+ } else {
+ name_size = ASN1_STRING_to_UTF8(
+ &name, subject_alt_name->d.uniformResourceIdentifier);
+ }
if (name_size < 0) {
gpr_log(GPR_ERROR, "Could not get utf8 from asn1 string.");
result = TSI_INTERNAL_ERROR;
@@ -369,7 +375,6 @@ static tsi_result add_subject_alt_names_properties_to_peer(
} else if (subject_alt_name->type == GEN_IPADD) {
char ntop_buf[INET6_ADDRSTRLEN];
int af;
-
if (subject_alt_name->d.iPAddress->length == 4) {
af = AF_INET;
} else if (subject_alt_name->d.iPAddress->length == 16) {
@@ -386,7 +391,6 @@ static tsi_result add_subject_alt_names_properties_to_peer(
result = TSI_INTERNAL_ERROR;
break;
}
-
result = tsi_construct_string_peer_property_from_cstring(
TSI_X509_SUBJECT_ALTERNATIVE_NAME_PEER_PROPERTY, name,
&peer->properties[peer->property_count++]);
@@ -1017,7 +1021,6 @@ static void tsi_ssl_handshaker_factory_init(
}
/* --- tsi_handshaker_result methods implementation. ---*/
-
static tsi_result ssl_handshaker_result_extract_peer(
const tsi_handshaker_result* self, tsi_peer* peer) {
tsi_result result = TSI_OK;
@@ -1025,6 +1028,7 @@ static tsi_result ssl_handshaker_result_extract_peer(
unsigned int alpn_selected_len;
const tsi_ssl_handshaker_result* impl =
reinterpret_cast<const tsi_ssl_handshaker_result*>(self);
+ // TODO(yihuazhang): Return a full certificate chain as a peer property.
X509* peer_cert = SSL_get_peer_certificate(impl->ssl);
if (peer_cert != nullptr) {
result = peer_from_x509(peer_cert, 1, peer);
@@ -1066,7 +1070,6 @@ static tsi_result ssl_handshaker_result_extract_peer(
&peer->properties[peer->property_count]);
if (result != TSI_OK) return result;
peer->property_count++;
-
return result;
}
@@ -1400,7 +1403,6 @@ static tsi_result create_tsi_ssl_handshaker(SSL_CTX* ctx, int is_client,
static_cast<unsigned char*>(gpr_zalloc(impl->outgoing_bytes_buffer_size));
impl->base.vtable = &handshaker_vtable;
impl->factory_ref = tsi_ssl_handshaker_factory_ref(factory);
-
*handshaker = &impl->base;
return TSI_OK;
}
diff --git a/src/python/grpcio/grpc_core_dependencies.py b/src/python/grpcio/grpc_core_dependencies.py
index 8f7da3a5b04..814f920eaf7 100644
--- a/src/python/grpcio/grpc_core_dependencies.py
+++ b/src/python/grpcio/grpc_core_dependencies.py
@@ -256,6 +256,7 @@ CORE_SOURCE_FILES = [
'src/core/lib/security/credentials/plugin/plugin_credentials.cc',
'src/core/lib/security/credentials/ssl/ssl_credentials.cc',
'src/core/lib/security/credentials/tls/grpc_tls_credentials_options.cc',
+ 'src/core/lib/security/credentials/tls/spiffe_credentials.cc',
'src/core/lib/security/security_connector/alts/alts_security_connector.cc',
'src/core/lib/security/security_connector/fake/fake_security_connector.cc',
'src/core/lib/security/security_connector/load_system_roots_fallback.cc',
@@ -264,6 +265,7 @@ CORE_SOURCE_FILES = [
'src/core/lib/security/security_connector/security_connector.cc',
'src/core/lib/security/security_connector/ssl/ssl_security_connector.cc',
'src/core/lib/security/security_connector/ssl_utils.cc',
+ 'src/core/lib/security/security_connector/tls/spiffe_security_connector.cc',
'src/core/lib/security/transport/client_auth_filter.cc',
'src/core/lib/security/transport/secure_endpoint.cc',
'src/core/lib/security/transport/security_handshaker.cc',
diff --git a/test/core/end2end/fixtures/h2_spiffe.cc b/test/core/end2end/fixtures/h2_spiffe.cc
new file mode 100644
index 00000000000..3cd7c362212
--- /dev/null
+++ b/test/core/end2end/fixtures/h2_spiffe.cc
@@ -0,0 +1,290 @@
+/*
+ *
+ * Copyright 2018 gRPC authors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+#include "test/core/end2end/end2end_tests.h"
+
+#include <stdio.h>
+#include <string.h>
+
+#include <grpc/grpc_security.h>
+#include <grpc/support/alloc.h>
+#include <grpc/support/log.h>
+
+#include <grpc/support/string_util.h>
+#include "src/core/lib/channel/channel_args.h"
+#include "src/core/lib/gpr/env.h"
+#include "src/core/lib/gpr/host_port.h"
+#include "src/core/lib/gpr/string.h"
+#include "src/core/lib/gpr/tmpfile.h"
+#include "src/core/lib/gprpp/inlined_vector.h"
+#include "src/core/lib/gprpp/thd.h"
+#include "src/core/lib/security/credentials/credentials.h"
+#include "src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h"
+#include "test/core/end2end/data/ssl_test_data.h"
+#include "test/core/util/port.h"
+#include "test/core/util/test_config.h"
+
+typedef grpc_core::InlinedVector<grpc_core::Thread, 1> ThreadList;
+
+typedef struct fullstack_secure_fixture_data {
+ char* localaddr;
+ ThreadList thd_list;
+} fullstack_secure_fixture_data;
+
+static grpc_end2end_test_fixture chttp2_create_fixture_secure_fullstack(
+ grpc_channel_args* client_args, grpc_channel_args* server_args) {
+ grpc_end2end_test_fixture f;
+ int port = grpc_pick_unused_port_or_die();
+ fullstack_secure_fixture_data* ffd =
+ grpc_core::New<fullstack_secure_fixture_data>();
+ memset(&f, 0, sizeof(f));
+ gpr_join_host_port(&ffd->localaddr, "localhost", port);
+ f.fixture_data = ffd;
+ f.cq = grpc_completion_queue_create_for_next(nullptr);
+ f.shutdown_cq = grpc_completion_queue_create_for_pluck(nullptr);
+ return f;
+}
+
+static void process_auth_failure(void* state, grpc_auth_context* ctx,
+ const grpc_metadata* md, size_t md_count,
+ grpc_process_auth_metadata_done_cb cb,
+ void* user_data) {
+ GPR_ASSERT(state == nullptr);
+ cb(user_data, nullptr, 0, nullptr, 0, GRPC_STATUS_UNAUTHENTICATED, nullptr);
+}
+
+static void chttp2_init_client_secure_fullstack(
+ grpc_end2end_test_fixture* f, grpc_channel_args* client_args,
+ grpc_channel_credentials* creds) {
+ fullstack_secure_fixture_data* ffd =
+ static_cast<fullstack_secure_fixture_data*>(f->fixture_data);
+ f->client =
+ grpc_secure_channel_create(creds, ffd->localaddr, client_args, nullptr);
+ GPR_ASSERT(f->client != nullptr);
+ grpc_channel_credentials_release(creds);
+}
+
+static void chttp2_init_server_secure_fullstack(
+ grpc_end2end_test_fixture* f, grpc_channel_args* server_args,
+ grpc_server_credentials* server_creds) {
+ fullstack_secure_fixture_data* ffd =
+ static_cast<fullstack_secure_fixture_data*>(f->fixture_data);
+ if (f->server) {
+ grpc_server_destroy(f->server);
+ }
+ f->server = grpc_server_create(server_args, nullptr);
+ grpc_server_register_completion_queue(f->server, f->cq, nullptr);
+ GPR_ASSERT(grpc_server_add_secure_http2_port(f->server, ffd->localaddr,
+ server_creds));
+ grpc_server_credentials_release(server_creds);
+ grpc_server_start(f->server);
+}
+
+void chttp2_tear_down_secure_fullstack(grpc_end2end_test_fixture* f) {
+ fullstack_secure_fixture_data* ffd =
+ static_cast<fullstack_secure_fixture_data*>(f->fixture_data);
+ for (size_t ind = 0; ind < ffd->thd_list.size(); ind++) {
+ ffd->thd_list[ind].Join();
+ }
+ gpr_free(ffd->localaddr);
+ grpc_core::Delete(ffd);
+}
+
+// Application-provided callback for server authorization check.
+static void server_authz_check_cb(void* user_data) {
+ grpc_tls_server_authorization_check_arg* check_arg =
+ static_cast<grpc_tls_server_authorization_check_arg*>(user_data);
+ GPR_ASSERT(check_arg != nullptr);
+ // result = 1 indicates the server authorization check passes.
+ // Normally, the applicaiton code should resort to mapping information
+ // between server identity and target name to derive the result.
+ // For this test, we directly return 1 for simplicity.
+ check_arg->success = 1;
+ check_arg->status = GRPC_STATUS_OK;
+ check_arg->cb(check_arg);
+}
+
+// Asynchronous implementation of schedule field in
+// grpc_server_authorization_check_config.
+static int server_authz_check_async(
+ void* config_user_data, grpc_tls_server_authorization_check_arg* arg) {
+ fullstack_secure_fixture_data* ffd =
+ static_cast<fullstack_secure_fixture_data*>(config_user_data);
+ ffd->thd_list.push_back(
+ grpc_core::Thread("h2_spiffe_test", &server_authz_check_cb, arg));
+ ffd->thd_list[ffd->thd_list.size() - 1].Start();
+ return 1;
+}
+
+// Synchronous implementation of schedule field in
+// grpc_tls_credential_reload_config instance that is a part of client-side
+// grpc_tls_credentials_options instance.
+static int client_cred_reload_sync(void* config_user_data,
+ grpc_tls_credential_reload_arg* arg) {
+ grpc_ssl_pem_key_cert_pair** key_cert_pair =
+ static_cast<grpc_ssl_pem_key_cert_pair**>(
+ gpr_zalloc(sizeof(grpc_ssl_pem_key_cert_pair*)));
+ key_cert_pair[0] = static_cast<grpc_ssl_pem_key_cert_pair*>(
+ gpr_zalloc(sizeof(grpc_ssl_pem_key_cert_pair)));
+ key_cert_pair[0]->private_key = gpr_strdup(test_server1_key);
+ key_cert_pair[0]->cert_chain = gpr_strdup(test_server1_cert);
+ if (!arg->key_materials_config->pem_key_cert_pair_list().size()) {
+ grpc_tls_key_materials_config_set_key_materials(
+ arg->key_materials_config, gpr_strdup(test_root_cert),
+ (const grpc_ssl_pem_key_cert_pair**)key_cert_pair, 1);
+ }
+ // new credential has been reloaded.
+ arg->status = GRPC_SSL_CERTIFICATE_CONFIG_RELOAD_NEW;
+ return 0;
+}
+
+// Synchronous implementation of schedule field in
+// grpc_tls_credential_reload_config instance that is a part of server-side
+// grpc_tls_credentials_options instance.
+static int server_cred_reload_sync(void* config_user_data,
+ grpc_tls_credential_reload_arg* arg) {
+ grpc_ssl_pem_key_cert_pair** key_cert_pair =
+ static_cast<grpc_ssl_pem_key_cert_pair**>(
+ gpr_zalloc(sizeof(grpc_ssl_pem_key_cert_pair*)));
+ key_cert_pair[0] = static_cast<grpc_ssl_pem_key_cert_pair*>(
+ gpr_zalloc(sizeof(grpc_ssl_pem_key_cert_pair)));
+ key_cert_pair[0]->private_key = gpr_strdup(test_server1_key);
+ key_cert_pair[0]->cert_chain = gpr_strdup(test_server1_cert);
+ GPR_ASSERT(arg != nullptr);
+ GPR_ASSERT(arg->key_materials_config != nullptr);
+ GPR_ASSERT(arg->key_materials_config->pem_key_cert_pair_list().data() !=
+ nullptr);
+ if (!arg->key_materials_config->pem_key_cert_pair_list().size()) {
+ grpc_tls_key_materials_config_set_key_materials(
+ arg->key_materials_config, gpr_strdup(test_root_cert),
+ (const grpc_ssl_pem_key_cert_pair**)key_cert_pair, 1);
+ }
+ // new credential has been reloaded.
+ arg->status = GRPC_SSL_CERTIFICATE_CONFIG_RELOAD_NEW;
+ return 0;
+}
+
+// Create a SPIFFE channel credential.
+static grpc_channel_credentials* create_spiffe_channel_credentials(
+ fullstack_secure_fixture_data* ffd) {
+ grpc_tls_credentials_options* options = grpc_tls_credentials_options_create();
+ /* Set credential reload config. */
+ grpc_tls_credential_reload_config* reload_config =
+ grpc_tls_credential_reload_config_create(nullptr, client_cred_reload_sync,
+ nullptr, nullptr);
+ grpc_tls_credentials_options_set_credential_reload_config(options,
+ reload_config);
+ /* Set server authorization check config. */
+ grpc_tls_server_authorization_check_config* check_config =
+ grpc_tls_server_authorization_check_config_create(
+ ffd, server_authz_check_async, nullptr, nullptr);
+ grpc_tls_credentials_options_set_server_authorization_check_config(
+ options, check_config);
+ /* Create SPIFFE channel credentials. */
+ grpc_channel_credentials* creds = grpc_tls_spiffe_credentials_create(options);
+ return creds;
+}
+
+// Create a SPIFFE server credential.
+static grpc_server_credentials* create_spiffe_server_credentials() {
+ grpc_tls_credentials_options* options = grpc_tls_credentials_options_create();
+ /* Set credential reload config. */
+ grpc_tls_credential_reload_config* reload_config =
+ grpc_tls_credential_reload_config_create(nullptr, server_cred_reload_sync,
+ nullptr, nullptr);
+ grpc_tls_credentials_options_set_credential_reload_config(options,
+ reload_config);
+ /* Set client certificate request type. */
+ grpc_tls_credentials_options_set_cert_request_type(
+ options, GRPC_SSL_REQUEST_AND_REQUIRE_CLIENT_CERTIFICATE_AND_VERIFY);
+ grpc_server_credentials* creds =
+ grpc_tls_spiffe_server_credentials_create(options);
+ return creds;
+}
+
+static void chttp2_init_client(grpc_end2end_test_fixture* f,
+ grpc_channel_args* client_args) {
+ grpc_channel_credentials* ssl_creds = create_spiffe_channel_credentials(
+ static_cast<fullstack_secure_fixture_data*>(f->fixture_data));
+ grpc_arg ssl_name_override = {
+ GRPC_ARG_STRING,
+ const_cast<char*>(GRPC_SSL_TARGET_NAME_OVERRIDE_ARG),
+ {const_cast<char*>("foo.test.google.fr")}};
+ grpc_channel_args* new_client_args =
+ grpc_channel_args_copy_and_add(client_args, &ssl_name_override, 1);
+ chttp2_init_client_secure_fullstack(f, new_client_args, ssl_creds);
+ grpc_channel_args_destroy(new_client_args);
+}
+
+static int fail_server_auth_check(grpc_channel_args* server_args) {
+ size_t i;
+ if (server_args == nullptr) return 0;
+ for (i = 0; i < server_args->num_args; i++) {
+ if (strcmp(server_args->args[i].key, FAIL_AUTH_CHECK_SERVER_ARG_NAME) ==
+ 0) {
+ return 1;
+ }
+ }
+ return 0;
+}
+
+static void chttp2_init_server(grpc_end2end_test_fixture* f,
+ grpc_channel_args* server_args) {
+ grpc_server_credentials* ssl_creds = create_spiffe_server_credentials();
+ if (fail_server_auth_check(server_args)) {
+ grpc_auth_metadata_processor processor = {process_auth_failure, nullptr,
+ nullptr};
+ grpc_server_credentials_set_auth_metadata_processor(ssl_creds, processor);
+ }
+ chttp2_init_server_secure_fullstack(f, server_args, ssl_creds);
+}
+
+static grpc_end2end_test_config configs[] = {
+ /* client sync reload async authz + server sync reload. */
+ {"chttp2/simple_ssl_fullstack",
+ FEATURE_MASK_SUPPORTS_DELAYED_CONNECTION |
+ FEATURE_MASK_SUPPORTS_PER_CALL_CREDENTIALS |
+ FEATURE_MASK_SUPPORTS_CLIENT_CHANNEL |
+ FEATURE_MASK_SUPPORTS_AUTHORITY_HEADER,
+ "foo.test.google.fr", chttp2_create_fixture_secure_fullstack,
+ chttp2_init_client, chttp2_init_server, chttp2_tear_down_secure_fullstack},
+};
+
+int main(int argc, char** argv) {
+ FILE* roots_file;
+ size_t roots_size = strlen(test_root_cert);
+ char* roots_filename;
+ grpc_test_init(argc, argv);
+ grpc_end2end_tests_pre_init();
+ /* Set the SSL roots env var. */
+ roots_file = gpr_tmpfile("chttp2_simple_ssl_fullstack_test", &roots_filename);
+ GPR_ASSERT(roots_filename != nullptr);
+ GPR_ASSERT(roots_file != nullptr);
+ GPR_ASSERT(fwrite(test_root_cert, 1, roots_size, roots_file) == roots_size);
+ fclose(roots_file);
+ gpr_setenv(GRPC_DEFAULT_SSL_ROOTS_FILE_PATH_ENV_VAR, roots_filename);
+ grpc_init();
+ for (size_t ind = 0; ind < sizeof(configs) / sizeof(*configs); ind++) {
+ grpc_end2end_tests(argc, argv, configs[ind]);
+ }
+ grpc_shutdown();
+ /* Cleanup. */
+ remove(roots_filename);
+ gpr_free(roots_filename);
+ return 0;
+}
diff --git a/test/core/end2end/gen_build_yaml.py b/test/core/end2end/gen_build_yaml.py
index f8ac7036530..231d2262a9d 100755
--- a/test/core/end2end/gen_build_yaml.py
+++ b/test/core/end2end/gen_build_yaml.py
@@ -74,6 +74,7 @@ END2END_FIXTURES = {
'h2_sockpair+trace': socketpair_unsecure_fixture_options._replace(
ci_mac=False, tracing=True, large_writes=False, exclude_iomgrs=['uv']),
'h2_ssl': default_secure_fixture_options,
+ 'h2_spiffe': default_secure_fixture_options,
'h2_local_uds': local_fixture_options,
'h2_local_ipv4': local_fixture_options,
'h2_local_ipv6': local_fixture_options,
diff --git a/test/core/end2end/generate_tests.bzl b/test/core/end2end/generate_tests.bzl
index a5fe091c2fc..d1ce92ebfba 100755
--- a/test/core/end2end/generate_tests.bzl
+++ b/test/core/end2end/generate_tests.bzl
@@ -87,6 +87,7 @@ END2END_FIXTURES = {
client_channel = False,
),
"h2_ssl": _fixture_options(secure = True),
+ "h2_spiffe": _fixture_options(secure = True),
"h2_local_uds": _fixture_options(secure = True, dns_resolver = False, _platforms = ["linux", "mac", "posix"]),
"h2_local_ipv4": _fixture_options(secure = True, dns_resolver = False, _platforms = ["linux", "mac", "posix"]),
"h2_local_ipv6": _fixture_options(secure = True, dns_resolver = False, _platforms = ["linux", "mac", "posix"]),
diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal
index 6e4a57ba00f..5ce5d5d3ce3 100644
--- a/tools/doxygen/Doxyfile.core.internal
+++ b/tools/doxygen/Doxyfile.core.internal
@@ -1381,6 +1381,8 @@ src/core/lib/security/credentials/ssl/ssl_credentials.cc \
src/core/lib/security/credentials/ssl/ssl_credentials.h \
src/core/lib/security/credentials/tls/grpc_tls_credentials_options.cc \
src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h \
+src/core/lib/security/credentials/tls/spiffe_credentials.cc \
+src/core/lib/security/credentials/tls/spiffe_credentials.h \
src/core/lib/security/security_connector/alts/alts_security_connector.cc \
src/core/lib/security/security_connector/alts/alts_security_connector.h \
src/core/lib/security/security_connector/fake/fake_security_connector.cc \
@@ -1397,6 +1399,8 @@ src/core/lib/security/security_connector/ssl/ssl_security_connector.cc \
src/core/lib/security/security_connector/ssl/ssl_security_connector.h \
src/core/lib/security/security_connector/ssl_utils.cc \
src/core/lib/security/security_connector/ssl_utils.h \
+src/core/lib/security/security_connector/tls/spiffe_security_connector.cc \
+src/core/lib/security/security_connector/tls/spiffe_security_connector.h \
src/core/lib/security/transport/auth_filters.h \
src/core/lib/security/transport/client_auth_filter.cc \
src/core/lib/security/transport/secure_endpoint.cc \
diff --git a/tools/run_tests/generated/sources_and_headers.json b/tools/run_tests/generated/sources_and_headers.json
index 8adde9ec602..2d427804d07 100644
--- a/tools/run_tests/generated/sources_and_headers.json
+++ b/tools/run_tests/generated/sources_and_headers.json
@@ -6285,6 +6285,23 @@
"third_party": false,
"type": "target"
},
+ {
+ "deps": [
+ "end2end_tests",
+ "gpr",
+ "grpc",
+ "grpc_test_util"
+ ],
+ "headers": [],
+ "is_filegroup": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "src": [
+ "test/core/end2end/fixtures/h2_spiffe.cc"
+ ],
+ "third_party": false,
+ "type": "target"
+ },
{
"deps": [
"end2end_tests",
@@ -10458,6 +10475,7 @@
"src/core/lib/security/credentials/plugin/plugin_credentials.h",
"src/core/lib/security/credentials/ssl/ssl_credentials.h",
"src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h",
+ "src/core/lib/security/credentials/tls/spiffe_credentials.h",
"src/core/lib/security/security_connector/alts/alts_security_connector.h",
"src/core/lib/security/security_connector/fake/fake_security_connector.h",
"src/core/lib/security/security_connector/load_system_roots.h",
@@ -10466,6 +10484,7 @@
"src/core/lib/security/security_connector/security_connector.h",
"src/core/lib/security/security_connector/ssl/ssl_security_connector.h",
"src/core/lib/security/security_connector/ssl_utils.h",
+ "src/core/lib/security/security_connector/tls/spiffe_security_connector.h",
"src/core/lib/security/transport/auth_filters.h",
"src/core/lib/security/transport/secure_endpoint.h",
"src/core/lib/security/transport/security_handshaker.h",
@@ -10513,6 +10532,8 @@
"src/core/lib/security/credentials/ssl/ssl_credentials.h",
"src/core/lib/security/credentials/tls/grpc_tls_credentials_options.cc",
"src/core/lib/security/credentials/tls/grpc_tls_credentials_options.h",
+ "src/core/lib/security/credentials/tls/spiffe_credentials.cc",
+ "src/core/lib/security/credentials/tls/spiffe_credentials.h",
"src/core/lib/security/security_connector/alts/alts_security_connector.cc",
"src/core/lib/security/security_connector/alts/alts_security_connector.h",
"src/core/lib/security/security_connector/fake/fake_security_connector.cc",
@@ -10529,6 +10550,8 @@
"src/core/lib/security/security_connector/ssl/ssl_security_connector.h",
"src/core/lib/security/security_connector/ssl_utils.cc",
"src/core/lib/security/security_connector/ssl_utils.h",
+ "src/core/lib/security/security_connector/tls/spiffe_security_connector.cc",
+ "src/core/lib/security/security_connector/tls/spiffe_security_connector.h",
"src/core/lib/security/transport/auth_filters.h",
"src/core/lib/security/transport/client_auth_filter.cc",
"src/core/lib/security/transport/secure_endpoint.cc",
diff --git a/tools/run_tests/generated/tests.json b/tools/run_tests/generated/tests.json
index 7c9fb8d6804..cffc8b98764 100644
--- a/tools/run_tests/generated/tests.json
+++ b/tools/run_tests/generated/tests.json
@@ -34543,6 +34543,1781 @@
"posix"
]
},
+ {
+ "args": [
+ "authority_not_supported"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "bad_hostname"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "bad_ping"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "binary_metadata"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "call_creds"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "call_host_override"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "cancel_after_accept"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "cancel_after_client_done"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "cancel_after_invoke"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "cancel_after_round_trip"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "cancel_before_invoke"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "cancel_in_a_vacuum"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "cancel_with_status"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "channelz"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "compressed_payload"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "connectivity"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [
+ "uv"
+ ],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "default_host"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "disappearing_server"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": true,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "empty_batch"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "filter_call_init_fails"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "filter_causes_close"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "filter_context"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "filter_latency"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "filter_status_code"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "graceful_server_shutdown"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "high_initial_seqno"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "hpack_size"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "idempotent_request"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "invoke_large_request"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "keepalive_timeout"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "large_metadata"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "max_concurrent_streams"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "max_connection_age"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "max_connection_idle"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [
+ "uv"
+ ],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "max_message_length"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "negative_deadline"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "no_error_on_hotpath"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "no_logging"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "no_op"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "payload"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "ping"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "ping_pong_streaming"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "registered_call"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "request_with_flags"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "request_with_payload"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "resource_quota_server"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry_cancellation"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry_disabled"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry_exceeds_buffer_size_in_initial_batch"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry_exceeds_buffer_size_in_subsequent_batch"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry_non_retriable_status"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry_non_retriable_status_before_recv_trailing_metadata_started"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry_recv_initial_metadata"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry_recv_message"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry_server_pushback_delay"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry_server_pushback_disabled"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry_streaming"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry_streaming_after_commit"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry_streaming_succeeds_before_replay_finished"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry_throttled"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "retry_too_many_attempts"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "server_finishes_request"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "shutdown_finishes_calls"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "shutdown_finishes_tags"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "simple_cacheable_request"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "simple_delayed_request"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "simple_metadata"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "simple_request"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "stream_compression_compressed_payload"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "stream_compression_payload"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "stream_compression_ping_pong_streaming"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "streaming_error_response"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "trailing_metadata"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "workaround_cronet_compression"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 1.0,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "write_buffering"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
+ {
+ "args": [
+ "write_buffering_at_end"
+ ],
+ "ci_platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ],
+ "cpu_cost": 0.1,
+ "exclude_configs": [],
+ "exclude_iomgrs": [],
+ "flaky": false,
+ "language": "c",
+ "name": "h2_spiffe_test",
+ "platforms": [
+ "windows",
+ "linux",
+ "mac",
+ "posix"
+ ]
+ },
{
"args": [
"authority_not_supported"
From 32b9e5245353eef7cf2f71ff115f82a9d3652a39 Mon Sep 17 00:00:00 2001
From: "Nicolas \"Pixel\" Noble" <pixel@nobis-crew.org>
Date: Fri, 15 Mar 2019 00:47:15 +0100
Subject: [PATCH 123/138] Fix a memset on a non-trivial object.
---
src/core/ext/transport/chttp2/transport/chttp2_transport.cc | 1 -
1 file changed, 1 deletion(-)
diff --git a/src/core/ext/transport/chttp2/transport/chttp2_transport.cc b/src/core/ext/transport/chttp2/transport/chttp2_transport.cc
index 829bee6bedd..49ec869d707 100644
--- a/src/core/ext/transport/chttp2/transport/chttp2_transport.cc
+++ b/src/core/ext/transport/chttp2/transport/chttp2_transport.cc
@@ -2476,7 +2476,6 @@ static grpc_error* try_http_parsing(grpc_chttp2_transport* t) {
size_t i = 0;
grpc_error* error = GRPC_ERROR_NONE;
grpc_http_response response;
- memset(&response, 0, sizeof(response));
grpc_http_parser_init(&parser, GRPC_HTTP_RESPONSE, &response);
From 8479bed4b5a240001a9a4f2813e7af8090cf9c57 Mon Sep 17 00:00:00 2001
From: "Nicolas \"Pixel\" Noble" <pixel@nobis-crew.org>
Date: Fri, 15 Mar 2019 01:01:29 +0100
Subject: [PATCH 124/138] Removing superfluous const in a static_cast.
This seems to be producing a warning on some compilers, and it's unnecessary anyway.
---
src/core/ext/transport/chttp2/transport/flow_control.cc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/core/ext/transport/chttp2/transport/flow_control.cc b/src/core/ext/transport/chttp2/transport/flow_control.cc
index ee2bb930802..d53475a1b61 100644
--- a/src/core/ext/transport/chttp2/transport/flow_control.cc
+++ b/src/core/ext/transport/chttp2/transport/flow_control.cc
@@ -190,7 +190,7 @@ TransportFlowControl::TransportFlowControl(const grpc_chttp2_transport* t,
uint32_t TransportFlowControl::MaybeSendUpdate(bool writing_anyway) {
FlowControlTrace trace("t updt sent", this, nullptr);
const uint32_t target_announced_window =
- static_cast<const uint32_t>(target_window());
+ static_cast<uint32_t>(target_window());
if ((writing_anyway || announced_window_ <= target_announced_window / 2) &&
announced_window_ != target_announced_window) {
const uint32_t announce = static_cast<uint32_t> GPR_CLAMP(
From ec78d0f56913fde14995b21515a525882e175602 Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Thu, 14 Mar 2019 19:51:09 -0700
Subject: [PATCH 125/138] Use correct C integer types in Cython
---
src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi | 10 +---------
1 file changed, 1 insertion(+), 9 deletions(-)
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi
index 759479089d4..88a6c0039c6 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi
@@ -13,15 +13,7 @@
# limitations under the License.
cimport libc.time
-from libc.stdint cimport intptr_t
-
-
-# Typedef types with approximately the same semantics to provide their names to
-# Cython
-ctypedef unsigned char uint8_t
-ctypedef int int32_t
-ctypedef unsigned uint32_t
-ctypedef long int64_t
+from libc.stdint cimport intptr_t, uint8_t, int32_t, uint32_t, int64_t
cdef extern from "grpc/support/alloc.h":
From cf6a31176137984d3335ad72273333491dc25153 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@users.noreply.github.com>
Date: Fri, 15 Mar 2019 17:15:20 +0100
Subject: [PATCH 126/138] Revert "Windows builds for gRPC C++ tests"
---
bazel/grpc_build_system.bzl | 16 +++-------------
bazel/grpc_deps.bzl | 9 +++++----
test/core/bad_connection/BUILD | 1 -
test/core/client_channel/BUILD | 1 -
test/core/end2end/generate_tests.bzl | 15 ++++-----------
test/core/iomgr/BUILD | 10 ----------
test/cpp/common/BUILD | 1 -
test/cpp/end2end/BUILD | 2 --
test/cpp/interop/BUILD | 1 -
test/cpp/microbenchmarks/BUILD | 18 ------------------
.../generate_resolver_component_tests.bzl | 5 +----
test/cpp/performance/BUILD | 1 -
test/cpp/qps/qps_benchmark_script.bzl | 1 -
test/cpp/server/BUILD | 3 ---
test/cpp/server/load_reporter/BUILD | 1 -
third_party/BUILD | 1 +
third_party/benchmark.BUILD | 15 +++++++++++++++
tools/remote_build/README.md | 6 ------
tools/remote_build/windows.bazelrc | 3 ---
19 files changed, 29 insertions(+), 81 deletions(-)
create mode 100644 third_party/benchmark.BUILD
delete mode 100644 tools/remote_build/windows.bazelrc
diff --git a/bazel/grpc_build_system.bzl b/bazel/grpc_build_system.bzl
index 59e9c46e0a3..2a09022c64c 100644
--- a/bazel/grpc_build_system.bzl
+++ b/bazel/grpc_build_system.bzl
@@ -28,12 +28,6 @@ load("//bazel:cc_grpc_library.bzl", "cc_grpc_library")
# The set of pollers to test against if a test exercises polling
POLLERS = ["epollex", "epoll1", "poll"]
-def is_msvc():
- return select({
- "//:windows_msvc": True,
- "//conditions:default": False,
- })
-
def if_not_windows(a):
return select({
"//:windows": [],
@@ -86,8 +80,7 @@ def grpc_cc_library(
visibility = None,
alwayslink = 0,
data = [],
- use_cfstream = False,
- tags = []):
+ use_cfstream = False):
copts = []
if use_cfstream:
copts = if_mac(["-DGRPC_CFSTREAM"])
@@ -124,7 +117,6 @@ def grpc_cc_library(
],
alwayslink = alwayslink,
data = data,
- tags = tags,
)
def grpc_proto_plugin(name, srcs = [], deps = []):
@@ -167,9 +159,8 @@ def grpc_cc_test(name, srcs = [], deps = [], external_deps = [], args = [], data
"size": size,
"timeout": timeout,
"exec_compatible_with": exec_compatible_with,
- "tags": tags,
}
- if uses_polling and not is_msvc():
+ if uses_polling:
native.cc_test(testonly = True, tags = ["manual"], **args)
for poller in POLLERS:
native.sh_test(
@@ -190,7 +181,7 @@ def grpc_cc_test(name, srcs = [], deps = [], external_deps = [], args = [], data
else:
native.cc_test(**args)
-def grpc_cc_binary(name, srcs = [], deps = [], external_deps = [], args = [], data = [], language = "C++", testonly = False, linkshared = False, linkopts = [], tags = []):
+def grpc_cc_binary(name, srcs = [], deps = [], external_deps = [], args = [], data = [], language = "C++", testonly = False, linkshared = False, linkopts = []):
copts = []
if language.upper() == "C":
copts = ["-std=c99"]
@@ -204,7 +195,6 @@ def grpc_cc_binary(name, srcs = [], deps = [], external_deps = [], args = [], da
deps = deps + _get_external_deps(external_deps),
copts = copts,
linkopts = if_not_windows(["-pthread"]) + linkopts,
- tags = tags,
)
def grpc_generate_one_off_targets():
diff --git a/bazel/grpc_deps.bzl b/bazel/grpc_deps.bzl
index 9b6aaacbd58..6b04c0b2c48 100644
--- a/bazel/grpc_deps.bzl
+++ b/bazel/grpc_deps.bzl
@@ -147,15 +147,16 @@ def grpc_deps():
if "com_github_gflags_gflags" not in native.existing_rules():
http_archive(
name = "com_github_gflags_gflags",
- strip_prefix = "gflags-28f50e0fed19872e0fd50dd23ce2ee8cd759338e",
- url = "https://github.com/gflags/gflags/archive/28f50e0fed19872e0fd50dd23ce2ee8cd759338e.tar.gz",
+ strip_prefix = "gflags-30dbc81fb5ffdc98ea9b14b1918bfe4e8779b26e",
+ url = "https://github.com/gflags/gflags/archive/30dbc81fb5ffdc98ea9b14b1918bfe4e8779b26e.tar.gz",
)
if "com_github_google_benchmark" not in native.existing_rules():
http_archive(
name = "com_github_google_benchmark",
- strip_prefix = "benchmark-e776aa0275e293707b6a0901e0e8d8a8a3679508",
- url = "https://github.com/google/benchmark/archive/e776aa0275e293707b6a0901e0e8d8a8a3679508.tar.gz",
+ build_file = "@com_github_grpc_grpc//third_party:benchmark.BUILD",
+ strip_prefix = "benchmark-9913418d323e64a0111ca0da81388260c2bbe1e9",
+ url = "https://github.com/google/benchmark/archive/9913418d323e64a0111ca0da81388260c2bbe1e9.tar.gz",
)
if "com_github_cares_cares" not in native.existing_rules():
diff --git a/test/core/bad_connection/BUILD b/test/core/bad_connection/BUILD
index 82b38ccc469..8ada933e796 100644
--- a/test/core/bad_connection/BUILD
+++ b/test/core/bad_connection/BUILD
@@ -29,5 +29,4 @@ grpc_cc_binary(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["no_windows"],
)
diff --git a/test/core/client_channel/BUILD b/test/core/client_channel/BUILD
index 68a71632daf..57e5191af4c 100644
--- a/test/core/client_channel/BUILD
+++ b/test/core/client_channel/BUILD
@@ -52,7 +52,6 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["no_windows"],
)
grpc_cc_test(
diff --git a/test/core/end2end/generate_tests.bzl b/test/core/end2end/generate_tests.bzl
index a5fe091c2fc..7bb246b6067 100755
--- a/test/core/end2end/generate_tests.bzl
+++ b/test/core/end2end/generate_tests.bzl
@@ -15,7 +15,7 @@
"""Generates the appropriate build.json data for all the end2end tests."""
-load("//bazel:grpc_build_system.bzl", "grpc_cc_binary", "grpc_cc_library", "is_msvc")
+load("//bazel:grpc_build_system.bzl", "grpc_cc_binary", "grpc_cc_library")
POLLERS = ["epollex", "epoll1", "poll"]
@@ -31,8 +31,7 @@ def _fixture_options(
is_http2 = True,
supports_proxy_auth = False,
supports_write_buffering = True,
- client_channel = True,
- supports_msvc = True,):
+ client_channel = True):
return struct(
fullstack = fullstack,
includes_proxy = includes_proxy,
@@ -45,7 +44,6 @@ def _fixture_options(
supports_proxy_auth = supports_proxy_auth,
supports_write_buffering = supports_write_buffering,
client_channel = client_channel,
- supports_msvc = supports_msvc,
#_platforms=_platforms,
)
@@ -121,11 +119,10 @@ END2END_NOSEC_FIXTURES = {
client_channel = False,
secure = False,
_platforms = ["linux", "mac", "posix"],
- supports_msvc = False,
),
"h2_full": _fixture_options(secure = False),
- "h2_full+pipe": _fixture_options(secure = False, _platforms = ["linux"], supports_msvc = False),
- "h2_full+trace": _fixture_options(secure = False, tracing = True, supports_msvc = False),
+ "h2_full+pipe": _fixture_options(secure = False, _platforms = ["linux"]),
+ "h2_full+trace": _fixture_options(secure = False, tracing = True),
"h2_full+workarounds": _fixture_options(secure = False),
"h2_http_proxy": _fixture_options(secure = False, supports_proxy_auth = True),
"h2_proxy": _fixture_options(secure = False, includes_proxy = True),
@@ -154,7 +151,6 @@ END2END_NOSEC_FIXTURES = {
dns_resolver = False,
_platforms = ["linux", "mac", "posix"],
secure = False,
- supports_msvc = False,
),
}
@@ -333,9 +329,6 @@ END2END_TESTS = {
}
def _compatible(fopt, topt):
- if is_msvc:
- if not fopt.supports_msvc:
- return False
if topt.needs_fullstack:
if not fopt.fullstack:
return False
diff --git a/test/core/iomgr/BUILD b/test/core/iomgr/BUILD
index 1aefa0ab224..5e4338aee37 100644
--- a/test/core/iomgr/BUILD
+++ b/test/core/iomgr/BUILD
@@ -81,7 +81,6 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["no_windows"],
)
grpc_cc_test(
@@ -93,7 +92,6 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["no_windows"],
)
grpc_cc_test(
@@ -105,7 +103,6 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["no_windows"],
)
grpc_cc_test(
@@ -142,7 +139,6 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["no_windows"],
)
grpc_cc_test(
@@ -157,7 +153,6 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["no_windows"],
)
grpc_cc_test(
@@ -219,7 +214,6 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["no_windows"],
)
grpc_cc_test(
@@ -231,7 +225,6 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["no_windows"],
)
grpc_cc_test(
@@ -244,7 +237,6 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["no_windows"],
)
grpc_cc_test(
@@ -267,7 +259,6 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["no_windows"],
)
grpc_cc_test(
@@ -312,5 +303,4 @@ grpc_cc_test(
"//:grpc",
"//test/core/util:grpc_test_util",
],
- tags = ["no_windows"],
)
diff --git a/test/cpp/common/BUILD b/test/cpp/common/BUILD
index b67c1995ff7..01699b26add 100644
--- a/test/cpp/common/BUILD
+++ b/test/cpp/common/BUILD
@@ -28,7 +28,6 @@ grpc_cc_test(
"//:grpc++_unsecure",
"//test/core/util:grpc_test_util_unsecure",
],
- tags = ["no_windows"],
)
grpc_cc_test(
diff --git a/test/cpp/end2end/BUILD b/test/cpp/end2end/BUILD
index 68e0ec3cef1..de7725d163d 100644
--- a/test/cpp/end2end/BUILD
+++ b/test/cpp/end2end/BUILD
@@ -99,7 +99,6 @@ grpc_cc_test(
"//test/core/util:grpc_test_util",
"//test/cpp/util:test_util",
],
- tags = ["no_windows"],
)
grpc_cc_test(
@@ -631,7 +630,6 @@ grpc_cc_test(
"//test/core/util:grpc_test_util",
"//test/cpp/util:test_util",
],
- tags = ["no_windows"],
)
grpc_cc_test(
diff --git a/test/cpp/interop/BUILD b/test/cpp/interop/BUILD
index 6cf4719c17b..f36494d98db 100644
--- a/test/cpp/interop/BUILD
+++ b/test/cpp/interop/BUILD
@@ -161,5 +161,4 @@ grpc_cc_test(
"//test/cpp/util:test_config",
"//test/cpp/util:test_util",
],
- tags = ["no_windows"],
)
diff --git a/test/cpp/microbenchmarks/BUILD b/test/cpp/microbenchmarks/BUILD
index 6e844a6dc62..70b4000780c 100644
--- a/test/cpp/microbenchmarks/BUILD
+++ b/test/cpp/microbenchmarks/BUILD
@@ -45,7 +45,6 @@ grpc_cc_library(
"//test/core/util:grpc_test_util_unsecure",
"//test/cpp/util:test_config",
],
- tags = ["no_windows"],
)
grpc_cc_binary(
@@ -53,7 +52,6 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_closure.cc"],
deps = [":helpers"],
- tags = ["no_windows"],
)
grpc_cc_binary(
@@ -61,7 +59,6 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_alarm.cc"],
deps = [":helpers"],
- tags = ["no_windows"],
)
grpc_cc_binary(
@@ -69,7 +66,6 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_arena.cc"],
deps = [":helpers"],
- tags = ["no_windows"],
)
grpc_cc_binary(
@@ -77,7 +73,6 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_byte_buffer.cc"],
deps = [":helpers"],
- tags = ["no_windows"],
)
grpc_cc_binary(
@@ -85,7 +80,6 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_channel.cc"],
deps = [":helpers"],
- tags = ["no_windows"],
)
grpc_cc_binary(
@@ -93,7 +87,6 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_call_create.cc"],
deps = [":helpers"],
- tags = ["no_windows"],
)
grpc_cc_binary(
@@ -101,7 +94,6 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_cq.cc"],
deps = [":helpers"],
- tags = ["no_windows"],
)
grpc_cc_binary(
@@ -109,7 +101,6 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_cq_multiple_threads.cc"],
deps = [":helpers"],
- tags = ["no_windows"],
)
grpc_cc_binary(
@@ -117,7 +108,6 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_error.cc"],
deps = [":helpers"],
- tags = ["no_windows"],
)
grpc_cc_library(
@@ -127,7 +117,6 @@ grpc_cc_library(
"fullstack_streaming_ping_pong.h",
],
deps = [":helpers"],
- tags = ["no_windows"],
)
grpc_cc_binary(
@@ -137,7 +126,6 @@ grpc_cc_binary(
"bm_fullstack_streaming_ping_pong.cc",
],
deps = [":fullstack_streaming_ping_pong_h"],
- tags = ["no_windows"],
)
grpc_cc_library(
@@ -156,7 +144,6 @@ grpc_cc_binary(
"bm_fullstack_streaming_pump.cc",
],
deps = [":fullstack_streaming_pump_h"],
- tags = ["no_windows"],
)
grpc_cc_binary(
@@ -164,7 +151,6 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_fullstack_trickle.cc"],
deps = [":helpers"],
- tags = ["no_windows"],
)
grpc_cc_library(
@@ -183,7 +169,6 @@ grpc_cc_binary(
"bm_fullstack_unary_ping_pong.cc",
],
deps = [":fullstack_unary_ping_pong_h"],
- tags = ["no_windows"],
)
grpc_cc_binary(
@@ -191,7 +176,6 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_metadata.cc"],
deps = [":helpers"],
- tags = ["no_windows"],
)
grpc_cc_binary(
@@ -199,7 +183,6 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_chttp2_hpack.cc"],
deps = [":helpers"],
- tags = ["no_windows"],
)
grpc_cc_binary(
@@ -219,5 +202,4 @@ grpc_cc_binary(
testonly = 1,
srcs = ["bm_timer.cc"],
deps = [":helpers"],
- tags = ["no_windows"],
)
diff --git a/test/cpp/naming/generate_resolver_component_tests.bzl b/test/cpp/naming/generate_resolver_component_tests.bzl
index 589176762e6..f36021560c1 100755
--- a/test/cpp/naming/generate_resolver_component_tests.bzl
+++ b/test/cpp/naming/generate_resolver_component_tests.bzl
@@ -33,7 +33,6 @@ def generate_resolver_component_tests():
"//:gpr",
"//test/cpp/util:test_config",
],
- tags = ["no_windows"],
)
# meant to be invoked only through the top-level shell script driver
grpc_cc_binary(
@@ -53,7 +52,6 @@ def generate_resolver_component_tests():
"//:gpr",
"//test/cpp/util:test_config",
],
- tags = ["no_windows"],
)
grpc_cc_test(
name = "resolver_component_tests_runner_invoker%s" % unsecure_build_config_suffix,
@@ -79,6 +77,5 @@ def generate_resolver_component_tests():
args = [
"--test_bin_name=resolver_component_test%s" % unsecure_build_config_suffix,
"--running_under_bazel=true",
- ],
- tags = ["no_windows"],
+ ]
)
diff --git a/test/cpp/performance/BUILD b/test/cpp/performance/BUILD
index 6068c33f95f..4fe95d5905e 100644
--- a/test/cpp/performance/BUILD
+++ b/test/cpp/performance/BUILD
@@ -31,5 +31,4 @@ grpc_cc_test(
"//src/proto/grpc/testing:echo_proto",
"//test/core/util:grpc_test_util_base",
],
- tags = ["no_windows"],
)
diff --git a/test/cpp/qps/qps_benchmark_script.bzl b/test/cpp/qps/qps_benchmark_script.bzl
index b4767ec8e09..855caa0d37c 100644
--- a/test/cpp/qps/qps_benchmark_script.bzl
+++ b/test/cpp/qps/qps_benchmark_script.bzl
@@ -75,6 +75,5 @@ def json_run_localhost_batch():
],
tags = [
"json_run_localhost",
- "no_windows",
],
)
diff --git a/test/cpp/server/BUILD b/test/cpp/server/BUILD
index a4811031691..050b83f5c4f 100644
--- a/test/cpp/server/BUILD
+++ b/test/cpp/server/BUILD
@@ -29,7 +29,6 @@ grpc_cc_test(
"//src/proto/grpc/testing:echo_proto",
"//test/core/util:grpc_test_util_unsecure",
],
- tags = ["no_windows"],
)
grpc_cc_test(
@@ -43,7 +42,6 @@ grpc_cc_test(
"//src/proto/grpc/testing:echo_proto",
"//test/core/util:grpc_test_util_unsecure",
],
- tags = ["no_windows"],
)
grpc_cc_test(
@@ -57,5 +55,4 @@ grpc_cc_test(
"//src/proto/grpc/testing:echo_proto",
"//test/core/util:grpc_test_util_unsecure",
],
- tags = ["no_windows"],
)
diff --git a/test/cpp/server/load_reporter/BUILD b/test/cpp/server/load_reporter/BUILD
index db5c93263ad..8d876c56d29 100644
--- a/test/cpp/server/load_reporter/BUILD
+++ b/test/cpp/server/load_reporter/BUILD
@@ -45,7 +45,6 @@ grpc_cc_test(
"//:lb_server_load_reporting_filter",
"//test/core/util:grpc_test_util",
],
- tags = ["no_windows"],
)
grpc_cc_test(
diff --git a/third_party/BUILD b/third_party/BUILD
index 8b43d6b8300..5ec919dc48d 100644
--- a/third_party/BUILD
+++ b/third_party/BUILD
@@ -1,4 +1,5 @@
exports_files([
+ "benchmark.BUILD",
"gtest.BUILD",
"objective_c/Cronet/bidirectional_stream_c.h",
"zlib.BUILD",
diff --git a/third_party/benchmark.BUILD b/third_party/benchmark.BUILD
new file mode 100644
index 00000000000..4c622f32a84
--- /dev/null
+++ b/third_party/benchmark.BUILD
@@ -0,0 +1,15 @@
+cc_library(
+ name = "benchmark",
+ srcs = glob(["src/*.cc"]),
+ hdrs = glob(["include/**/*.h", "src/*.h"]),
+ includes = [
+ "include", "."
+ ],
+ copts = [
+ "-DHAVE_POSIX_REGEX"
+ ],
+ linkstatic = 1,
+ visibility = [
+ "//visibility:public",
+ ],
+)
diff --git a/tools/remote_build/README.md b/tools/remote_build/README.md
index 8a236973946..19739e9ee12 100644
--- a/tools/remote_build/README.md
+++ b/tools/remote_build/README.md
@@ -29,11 +29,5 @@ Sanitizer runs (asan, msan, tsan, ubsan):
bazel --bazelrc=tools/remote_build/manual.bazelrc test --config=asan //test/...
```
-Run on Windows MSVC:
-```
-# local manual run only for C++ targets (RBE to be supported)
-bazel --bazelrc=tools/remote_build/windows.bazelrc test //test/cpp/...
-```
-
Available command line options can be found in
[Bazel command line reference](https://docs.bazel.build/versions/master/command-line-reference.html)
diff --git a/tools/remote_build/windows.bazelrc b/tools/remote_build/windows.bazelrc
deleted file mode 100644
index 70575372d02..00000000000
--- a/tools/remote_build/windows.bazelrc
+++ /dev/null
@@ -1,3 +0,0 @@
-# TODO(yfen): Merge with rbe_common.bazelrc and enable Windows RBE
-build --test_tag_filters=-no_windows
-build --build_tag_filters=-no_windows
From a5e0fe95e89061dcb23edd6d9d83941e7c74ed97 Mon Sep 17 00:00:00 2001
From: "Nicolas \"Pixel\" Noble" <pixel@nobis-crew.org>
Date: Fri, 15 Mar 2019 17:31:17 +0100
Subject: [PATCH 127/138] Removing a few more non-trivial struct memsets.
This structure (tsi_ssl_client_handshaker_options) has an explicit constructor that sets everything to zeroes.
---
src/core/tsi/ssl_transport_security.cc | 2 --
1 file changed, 2 deletions(-)
diff --git a/src/core/tsi/ssl_transport_security.cc b/src/core/tsi/ssl_transport_security.cc
index 2107bcaa748..2953fb53a57 100644
--- a/src/core/tsi/ssl_transport_security.cc
+++ b/src/core/tsi/ssl_transport_security.cc
@@ -1634,7 +1634,6 @@ tsi_result tsi_create_ssl_client_handshaker_factory(
const char** alpn_protocols, uint16_t num_alpn_protocols,
tsi_ssl_client_handshaker_factory** factory) {
tsi_ssl_client_handshaker_options options;
- memset(&options, 0, sizeof(options));
options.pem_key_cert_pair = pem_key_cert_pair;
options.pem_root_certs = pem_root_certs;
options.cipher_suites = cipher_suites;
@@ -1764,7 +1763,6 @@ tsi_result tsi_create_ssl_server_handshaker_factory_ex(
const char* cipher_suites, const char** alpn_protocols,
uint16_t num_alpn_protocols, tsi_ssl_server_handshaker_factory** factory) {
tsi_ssl_server_handshaker_options options;
- memset(&options, 0, sizeof(options));
options.pem_key_cert_pairs = pem_key_cert_pairs;
options.num_key_cert_pairs = num_key_cert_pairs;
options.pem_client_root_certs = pem_client_root_certs;
From 2bf934f97d33a076f87f52ed846bb8620f0dbae9 Mon Sep 17 00:00:00 2001
From: Karthik Ravi Shankar <karthikrs@google.com>
Date: Fri, 15 Mar 2019 10:20:06 -0700
Subject: [PATCH 128/138] Revert "Fold opencensus into grpc_impl namespace"
---
BUILD | 1 -
include/grpcpp/opencensus.h | 26 +++++++-
include/grpcpp/opencensus_impl.h | 51 ---------------
src/cpp/ext/filters/census/grpc_plugin.cc | 65 +++++++++----------
src/cpp/ext/filters/census/grpc_plugin.h | 6 +-
src/cpp/ext/filters/census/views.cc | 32 +++++----
.../census/stats_plugin_end2end_test.cc | 2 +-
.../microbenchmarks/bm_opencensus_plugin.cc | 6 +-
8 files changed, 75 insertions(+), 114 deletions(-)
delete mode 100644 include/grpcpp/opencensus_impl.h
diff --git a/BUILD b/BUILD
index 0b6fff354f4..e158059b30f 100644
--- a/BUILD
+++ b/BUILD
@@ -2279,7 +2279,6 @@ grpc_cc_library(
],
hdrs = [
"include/grpcpp/opencensus.h",
- "include/grpcpp/opencensus_impl.h",
"src/cpp/ext/filters/census/channel_filter.h",
"src/cpp/ext/filters/census/client_filter.h",
"src/cpp/ext/filters/census/context.h",
diff --git a/include/grpcpp/opencensus.h b/include/grpcpp/opencensus.h
index 3b170336834..29b221f7674 100644
--- a/include/grpcpp/opencensus.h
+++ b/include/grpcpp/opencensus.h
@@ -19,6 +19,30 @@
#ifndef GRPCPP_OPENCENSUS_H
#define GRPCPP_OPENCENSUS_H
-#include "grpcpp/opencensus_impl.h"
+#include "opencensus/trace/span.h"
+
+namespace grpc {
+// These symbols in this file will not be included in the binary unless
+// grpc_opencensus_plugin build target was added as a dependency. At the moment
+// it is only setup to be built with Bazel.
+
+// Registers the OpenCensus plugin with gRPC, so that it will be used for future
+// RPCs. This must be called before any views are created.
+void RegisterOpenCensusPlugin();
+
+// RPC stats definitions, defined by
+// https://github.com/census-instrumentation/opencensus-specs/blob/master/stats/gRPC.md
+
+// Registers the cumulative gRPC views so that they will be exported by any
+// registered stats exporter. For on-task stats, construct a View using the
+// ViewDescriptors below.
+void RegisterOpenCensusViewsForExport();
+
+class ServerContext;
+
+// Returns the tracing Span for the current RPC.
+::opencensus::trace::Span GetSpanFromServerContext(ServerContext* context);
+
+} // namespace grpc
#endif // GRPCPP_OPENCENSUS_H
diff --git a/include/grpcpp/opencensus_impl.h b/include/grpcpp/opencensus_impl.h
deleted file mode 100644
index 631d2b861fd..00000000000
--- a/include/grpcpp/opencensus_impl.h
+++ /dev/null
@@ -1,51 +0,0 @@
-/*
- *
- * Copyright 2019 gRPC authors.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-#ifndef GRPCPP_OPENCENSUS_IMPL_H
-#define GRPCPP_OPENCENSUS_IMPL_H
-
-#include "opencensus/trace/span.h"
-
-namespace grpc {
-
-class ServerContext;
-}
-namespace grpc_impl {
-// These symbols in this file will not be included in the binary unless
-// grpc_opencensus_plugin build target was added as a dependency. At the moment
-// it is only setup to be built with Bazel.
-
-// Registers the OpenCensus plugin with gRPC, so that it will be used for future
-// RPCs. This must be called before any views are created.
-void RegisterOpenCensusPlugin();
-
-// RPC stats definitions, defined by
-// https://github.com/census-instrumentation/opencensus-specs/blob/master/stats/gRPC.md
-
-// Registers the cumulative gRPC views so that they will be exported by any
-// registered stats exporter. For on-task stats, construct a View using the
-// ViewDescriptors below.
-void RegisterOpenCensusViewsForExport();
-
-// Returns the tracing Span for the current RPC.
-::opencensus::trace::Span GetSpanFromServerContext(
- grpc::ServerContext* context);
-
-} // namespace grpc_impl
-
-#endif // GRPCPP_OPENCENSUS_IMPL_H
diff --git a/src/cpp/ext/filters/census/grpc_plugin.cc b/src/cpp/ext/filters/census/grpc_plugin.cc
index c5018f0673a..f978ed3bf51 100644
--- a/src/cpp/ext/filters/census/grpc_plugin.cc
+++ b/src/cpp/ext/filters/census/grpc_plugin.cc
@@ -30,6 +30,35 @@
namespace grpc {
+void RegisterOpenCensusPlugin() {
+ RegisterChannelFilter<CensusChannelData, CensusClientCallData>(
+ "opencensus_client", GRPC_CLIENT_CHANNEL, INT_MAX /* priority */,
+ nullptr /* condition function */);
+ RegisterChannelFilter<CensusChannelData, CensusServerCallData>(
+ "opencensus_server", GRPC_SERVER_CHANNEL, INT_MAX /* priority */,
+ nullptr /* condition function */);
+
+ // Access measures to ensure they are initialized. Otherwise, creating a view
+ // before the first RPC would cause an error.
+ RpcClientSentBytesPerRpc();
+ RpcClientReceivedBytesPerRpc();
+ RpcClientRoundtripLatency();
+ RpcClientServerLatency();
+ RpcClientSentMessagesPerRpc();
+ RpcClientReceivedMessagesPerRpc();
+
+ RpcServerSentBytesPerRpc();
+ RpcServerReceivedBytesPerRpc();
+ RpcServerServerLatency();
+ RpcServerSentMessagesPerRpc();
+ RpcServerReceivedMessagesPerRpc();
+}
+
+::opencensus::trace::Span GetSpanFromServerContext(ServerContext* context) {
+ return reinterpret_cast<const CensusContext*>(context->census_context())
+ ->Span();
+}
+
// These measure definitions should be kept in sync across opencensus
// implementations--see
// https://github.com/census-instrumentation/opencensus-java/blob/master/contrib/grpc_metrics/src/main/java/io/opencensus/contrib/grpc/metrics/RpcMeasureConstants.java.
@@ -97,39 +126,5 @@ ABSL_CONST_INIT const absl::string_view
ABSL_CONST_INIT const absl::string_view kRpcServerServerLatencyMeasureName =
"grpc.io/server/server_latency";
-} // namespace grpc
-namespace grpc_impl {
-void RegisterOpenCensusPlugin() {
- grpc::RegisterChannelFilter<grpc::CensusChannelData,
- grpc::CensusClientCallData>(
- "opencensus_client", GRPC_CLIENT_CHANNEL, INT_MAX /* priority */,
- nullptr /* condition function */);
- grpc::RegisterChannelFilter<grpc::CensusChannelData,
- grpc::CensusServerCallData>(
- "opencensus_server", GRPC_SERVER_CHANNEL, INT_MAX /* priority */,
- nullptr /* condition function */);
-
- // Access measures to ensure they are initialized. Otherwise, creating a view
- // before the first RPC would cause an error.
- grpc::RpcClientSentBytesPerRpc();
- grpc::RpcClientReceivedBytesPerRpc();
- grpc::RpcClientRoundtripLatency();
- grpc::RpcClientServerLatency();
- grpc::RpcClientSentMessagesPerRpc();
- grpc::RpcClientReceivedMessagesPerRpc();
-
- grpc::RpcServerSentBytesPerRpc();
- grpc::RpcServerReceivedBytesPerRpc();
- grpc::RpcServerServerLatency();
- grpc::RpcServerSentMessagesPerRpc();
- grpc::RpcServerReceivedMessagesPerRpc();
-}
-
-::opencensus::trace::Span GetSpanFromServerContext(
- grpc::ServerContext* context) {
- return reinterpret_cast<const grpc::CensusContext*>(context->census_context())
- ->Span();
-}
-
-} // namespace grpc_impl
+} // namespace grpc
diff --git a/src/cpp/ext/filters/census/grpc_plugin.h b/src/cpp/ext/filters/census/grpc_plugin.h
index 209fad139ce..9e319cb994e 100644
--- a/src/cpp/ext/filters/census/grpc_plugin.h
+++ b/src/cpp/ext/filters/census/grpc_plugin.h
@@ -22,14 +22,12 @@
#include <grpc/support/port_platform.h>
#include "absl/strings/string_view.h"
-#include "include/grpcpp/opencensus_impl.h"
+#include "include/grpcpp/opencensus.h"
#include "opencensus/stats/stats.h"
-namespace grpc_impl {
+namespace grpc {
class ServerContext;
-}
-namespace grpc {
// The tag keys set when recording RPC stats.
::opencensus::stats::TagKey ClientMethodTagKey();
diff --git a/src/cpp/ext/filters/census/views.cc b/src/cpp/ext/filters/census/views.cc
index d7e3c81a955..2c0c5f72950 100644
--- a/src/cpp/ext/filters/census/views.cc
+++ b/src/cpp/ext/filters/census/views.cc
@@ -25,23 +25,6 @@
#include "opencensus/stats/internal/set_aggregation_window.h"
#include "opencensus/stats/stats.h"
-namespace grpc_impl {
-
-void RegisterOpenCensusViewsForExport() {
- grpc::ClientSentMessagesPerRpcCumulative().RegisterForExport();
- grpc::ClientSentBytesPerRpcCumulative().RegisterForExport();
- grpc::ClientReceivedMessagesPerRpcCumulative().RegisterForExport();
- grpc::ClientReceivedBytesPerRpcCumulative().RegisterForExport();
- grpc::ClientRoundtripLatencyCumulative().RegisterForExport();
- grpc::ClientServerLatencyCumulative().RegisterForExport();
-
- grpc::ServerSentMessagesPerRpcCumulative().RegisterForExport();
- grpc::ServerSentBytesPerRpcCumulative().RegisterForExport();
- grpc::ServerReceivedMessagesPerRpcCumulative().RegisterForExport();
- grpc::ServerReceivedBytesPerRpcCumulative().RegisterForExport();
- grpc::ServerServerLatencyCumulative().RegisterForExport();
-}
-} // namespace grpc_impl
namespace grpc {
using ::opencensus::stats::Aggregation;
@@ -88,6 +71,21 @@ ViewDescriptor HourDescriptor() {
} // namespace
+void RegisterOpenCensusViewsForExport() {
+ ClientSentMessagesPerRpcCumulative().RegisterForExport();
+ ClientSentBytesPerRpcCumulative().RegisterForExport();
+ ClientReceivedMessagesPerRpcCumulative().RegisterForExport();
+ ClientReceivedBytesPerRpcCumulative().RegisterForExport();
+ ClientRoundtripLatencyCumulative().RegisterForExport();
+ ClientServerLatencyCumulative().RegisterForExport();
+
+ ServerSentMessagesPerRpcCumulative().RegisterForExport();
+ ServerSentBytesPerRpcCumulative().RegisterForExport();
+ ServerReceivedMessagesPerRpcCumulative().RegisterForExport();
+ ServerReceivedBytesPerRpcCumulative().RegisterForExport();
+ ServerServerLatencyCumulative().RegisterForExport();
+}
+
// client cumulative
const ViewDescriptor& ClientSentBytesPerRpcCumulative() {
const static ViewDescriptor descriptor =
diff --git a/test/cpp/ext/filters/census/stats_plugin_end2end_test.cc b/test/cpp/ext/filters/census/stats_plugin_end2end_test.cc
index ad788a2dd68..73394028309 100644
--- a/test/cpp/ext/filters/census/stats_plugin_end2end_test.cc
+++ b/test/cpp/ext/filters/census/stats_plugin_end2end_test.cc
@@ -58,7 +58,7 @@ class EchoServer final : public EchoTestService::Service {
class StatsPluginEnd2EndTest : public ::testing::Test {
protected:
- static void SetUpTestCase() { grpc_impl::RegisterOpenCensusPlugin(); }
+ static void SetUpTestCase() { RegisterOpenCensusPlugin(); }
void SetUp() {
// Set up a synchronous server on a different thread to avoid the asynch
diff --git a/test/cpp/microbenchmarks/bm_opencensus_plugin.cc b/test/cpp/microbenchmarks/bm_opencensus_plugin.cc
index d23c4f0573f..9d42eb891df 100644
--- a/test/cpp/microbenchmarks/bm_opencensus_plugin.cc
+++ b/test/cpp/microbenchmarks/bm_opencensus_plugin.cc
@@ -29,9 +29,7 @@
#include "test/cpp/microbenchmarks/helpers.h"
absl::once_flag once;
-void RegisterOnce() {
- absl::call_once(once, grpc_impl::RegisterOpenCensusPlugin);
-}
+void RegisterOnce() { absl::call_once(once, grpc::RegisterOpenCensusPlugin); }
class EchoServer final : public grpc::testing::EchoTestService::Service {
grpc::Status Echo(grpc::ServerContext* context,
@@ -101,7 +99,7 @@ static void BM_E2eLatencyCensusEnabled(benchmark::State& state) {
RegisterOnce();
// This we can safely repeat, and doing so clears accumulated data to avoid
// initialization costs varying between runs.
- grpc_impl::RegisterOpenCensusViewsForExport();
+ grpc::RegisterOpenCensusViewsForExport();
EchoServerThread server;
std::unique_ptr<grpc::testing::EchoTestService::Stub> stub =
From c3ecc618672631c36db5a204b94fc6b259bda191 Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Fri, 15 Mar 2019 11:48:10 -0700
Subject: [PATCH 129/138] Use C-Core API to perform time conversion
---
src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi | 6 ++++++
src/python/grpcio/grpc/_cython/_cygrpc/time.pxd.pxi | 2 +-
src/python/grpcio/grpc/_cython/_cygrpc/time.pyx.pxi | 13 ++++++-------
.../tests/unit/_cython/_channel_test.py | 6 ++++++
4 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi
index 88a6c0039c6..7e20f7a4e2d 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi
@@ -53,6 +53,10 @@ cdef extern from "grpc/grpc.h":
void *grpc_slice_start_ptr "GRPC_SLICE_START_PTR" (grpc_slice s) nogil
size_t grpc_slice_length "GRPC_SLICE_LENGTH" (grpc_slice s) nogil
+ const int GPR_MS_PER_SEC
+ const int GPR_US_PER_SEC
+ const int GPR_NS_PER_SEC
+
ctypedef enum gpr_clock_type:
GPR_CLOCK_MONOTONIC
GPR_CLOCK_REALTIME
@@ -74,6 +78,8 @@ cdef extern from "grpc/grpc.h":
gpr_clock_type target_clock) nogil
gpr_timespec gpr_time_from_millis(int64_t ms, gpr_clock_type type) nogil
+ gpr_timespec gpr_time_from_nanos(int64_t ns, gpr_clock_type type) nogil
+ double gpr_timespec_to_micros(gpr_timespec t) nogil
gpr_timespec gpr_time_add(gpr_timespec a, gpr_timespec b) nogil
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/time.pxd.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/time.pxd.pxi
index 1319ac0481d..c46e8a98b04 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/time.pxd.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/time.pxd.pxi
@@ -13,7 +13,7 @@
# limitations under the License.
-cdef gpr_timespec _timespec_from_time(object time)
+cdef gpr_timespec _timespec_from_time(object time) except *
cdef double _time_from_timespec(gpr_timespec timespec) except *
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/time.pyx.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/time.pyx.pxi
index c452dd54f82..6d181bb1d60 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/time.pyx.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/time.pyx.pxi
@@ -13,18 +13,17 @@
# limitations under the License.
-cdef gpr_timespec _timespec_from_time(object time):
- cdef gpr_timespec timespec
+cdef gpr_timespec _timespec_from_time(object time) except *:
if time is None:
return gpr_inf_future(GPR_CLOCK_REALTIME)
else:
- timespec.seconds = time
- timespec.nanoseconds = (time - float(timespec.seconds)) * 1e9
- timespec.clock_type = GPR_CLOCK_REALTIME
- return timespec
+ return gpr_time_from_nanos(
+ <int64_t>(<double>time * GPR_NS_PER_SEC),
+ GPR_CLOCK_REALTIME,
+ )
cdef double _time_from_timespec(gpr_timespec timespec) except *:
cdef gpr_timespec real_timespec = gpr_convert_clock_type(
timespec, GPR_CLOCK_REALTIME)
- return <double>real_timespec.seconds + <double>real_timespec.nanoseconds / 1e9
+ return gpr_timespec_to_micros(real_timespec) / GPR_US_PER_SEC
diff --git a/src/python/grpcio_tests/tests/unit/_cython/_channel_test.py b/src/python/grpcio_tests/tests/unit/_cython/_channel_test.py
index d95286071d5..9fd9ede144d 100644
--- a/src/python/grpcio_tests/tests/unit/_cython/_channel_test.py
+++ b/src/python/grpcio_tests/tests/unit/_cython/_channel_test.py
@@ -57,6 +57,12 @@ class ChannelTest(unittest.TestCase):
def test_multiple_channels_lonely_connectivity(self):
_in_parallel(_create_loop_destroy, ())
+ def test_negative_deadline_connectivity(self):
+ channel = _channel()
+ connectivity = channel.check_connectivity_state(True)
+ channel.watch_connectivity_state(connectivity, -3.14)
+ channel.close(cygrpc.StatusCode.ok, 'Channel close!')
+
if __name__ == '__main__':
unittest.main(verbosity=2)
From 64caf4d3dfa1aea9e299d5715b7907a7157690ac Mon Sep 17 00:00:00 2001
From: Yihua Zhang <yihuaz@google.com>
Date: Fri, 15 Mar 2019 12:57:25 -0700
Subject: [PATCH 130/138] fix clang-tidy errors
---
.../tls/spiffe_security_connector.cc | 24 +++++++++----------
test/core/end2end/fixtures/h2_spiffe.cc | 6 ++---
2 files changed, 15 insertions(+), 15 deletions(-)
diff --git a/src/core/lib/security/security_connector/tls/spiffe_security_connector.cc b/src/core/lib/security/security_connector/tls/spiffe_security_connector.cc
index 075b1c9d53c..ebf9c905079 100644
--- a/src/core/lib/security/security_connector/tls/spiffe_security_connector.cc
+++ b/src/core/lib/security/security_connector/tls/spiffe_security_connector.cc
@@ -161,7 +161,7 @@ void SpiffeChannelSecurityConnector::check_peer(
const grpc_tls_server_authorization_check_config* config =
creds->options().server_authorization_check_config();
/* If server authorization config is not null, use it to perform
- * server authorizaiton check. */
+ * server authorization check. */
if (config != nullptr) {
const tsi_peer_property* p =
tsi_peer_get_property_by_name(&peer, TSI_X509_PEM_CERT_PROPERTY);
@@ -254,17 +254,17 @@ SpiffeChannelSecurityConnector::InitializeHandshakerFactory(
const SpiffeCredentials* creds =
static_cast<const SpiffeCredentials*>(channel_creds());
auto key_materials_config = PopulateSpiffeCredentials(creds->options());
- if (!key_materials_config.get()->pem_key_cert_pair_list().size()) {
- key_materials_config.get()->Unref();
+ if (key_materials_config->pem_key_cert_pair_list().empty()) {
+ key_materials_config->Unref();
return GRPC_SECURITY_ERROR;
}
tsi_ssl_pem_key_cert_pair* pem_key_cert_pair = ConvertToTsiPemKeyCertPair(
- key_materials_config.get()->pem_key_cert_pair_list());
+ key_materials_config->pem_key_cert_pair_list());
grpc_security_status status = grpc_ssl_tsi_client_handshaker_factory_init(
- pem_key_cert_pair, key_materials_config.get()->pem_root_certs(),
+ pem_key_cert_pair, key_materials_config->pem_root_certs(),
ssl_session_cache, &client_handshaker_factory_);
// Free memory.
- key_materials_config.get()->Unref();
+ key_materials_config->Unref();
grpc_tsi_ssl_pem_key_cert_pairs_destroy(pem_key_cert_pair, 1);
return status;
}
@@ -401,8 +401,8 @@ SpiffeServerSecurityConnector::RefreshServerHandshakerFactory() {
auto key_materials_config = PopulateSpiffeCredentials(creds->options());
/* Credential reload does NOT take effect and we need to keep using
* the existing handshaker factory. */
- if (key_materials_config.get()->pem_key_cert_pair_list().empty()) {
- key_materials_config.get()->Unref();
+ if (key_materials_config->pem_key_cert_pair_list().empty()) {
+ key_materials_config->Unref();
return GRPC_SECURITY_ERROR;
}
/* Credential reload takes effect and we need to free the existing
@@ -411,15 +411,15 @@ SpiffeServerSecurityConnector::RefreshServerHandshakerFactory() {
tsi_ssl_server_handshaker_factory_unref(server_handshaker_factory_);
}
tsi_ssl_pem_key_cert_pair* pem_key_cert_pairs = ConvertToTsiPemKeyCertPair(
- key_materials_config.get()->pem_key_cert_pair_list());
+ key_materials_config->pem_key_cert_pair_list());
size_t num_key_cert_pairs =
- key_materials_config.get()->pem_key_cert_pair_list().size();
+ key_materials_config->pem_key_cert_pair_list().size();
grpc_security_status status = grpc_ssl_tsi_server_handshaker_factory_init(
pem_key_cert_pairs, num_key_cert_pairs,
- key_materials_config.get()->pem_root_certs(),
+ key_materials_config->pem_root_certs(),
creds->options().cert_request_type(), &server_handshaker_factory_);
// Free memory.
- key_materials_config.get()->Unref();
+ key_materials_config->Unref();
grpc_tsi_ssl_pem_key_cert_pairs_destroy(pem_key_cert_pairs,
num_key_cert_pairs);
return status;
diff --git a/test/core/end2end/fixtures/h2_spiffe.cc b/test/core/end2end/fixtures/h2_spiffe.cc
index 3cd7c362212..9ab796ea429 100644
--- a/test/core/end2end/fixtures/h2_spiffe.cc
+++ b/test/core/end2end/fixtures/h2_spiffe.cc
@@ -111,7 +111,7 @@ static void server_authz_check_cb(void* user_data) {
static_cast<grpc_tls_server_authorization_check_arg*>(user_data);
GPR_ASSERT(check_arg != nullptr);
// result = 1 indicates the server authorization check passes.
- // Normally, the applicaiton code should resort to mapping information
+ // Normally, the application code should resort to mapping information
// between server identity and target name to derive the result.
// For this test, we directly return 1 for simplicity.
check_arg->success = 1;
@@ -143,7 +143,7 @@ static int client_cred_reload_sync(void* config_user_data,
gpr_zalloc(sizeof(grpc_ssl_pem_key_cert_pair)));
key_cert_pair[0]->private_key = gpr_strdup(test_server1_key);
key_cert_pair[0]->cert_chain = gpr_strdup(test_server1_cert);
- if (!arg->key_materials_config->pem_key_cert_pair_list().size()) {
+ if (arg->key_materials_config->pem_key_cert_pair_list().empty()) {
grpc_tls_key_materials_config_set_key_materials(
arg->key_materials_config, gpr_strdup(test_root_cert),
(const grpc_ssl_pem_key_cert_pair**)key_cert_pair, 1);
@@ -169,7 +169,7 @@ static int server_cred_reload_sync(void* config_user_data,
GPR_ASSERT(arg->key_materials_config != nullptr);
GPR_ASSERT(arg->key_materials_config->pem_key_cert_pair_list().data() !=
nullptr);
- if (!arg->key_materials_config->pem_key_cert_pair_list().size()) {
+ if (arg->key_materials_config->pem_key_cert_pair_list().empty()) {
grpc_tls_key_materials_config_set_key_materials(
arg->key_materials_config, gpr_strdup(test_root_cert),
(const grpc_ssl_pem_key_cert_pair**)key_cert_pair, 1);
From dfb5a2dbc67d7968c3c9218d003c9a3d97e452a5 Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Fri, 15 Mar 2019 13:01:28 -0700
Subject: [PATCH 131/138] Disable negative deadline test in gevent
---
src/python/grpcio_tests/commands.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/src/python/grpcio_tests/commands.py b/src/python/grpcio_tests/commands.py
index 866fb6de1f7..e6e5e3d4e93 100644
--- a/src/python/grpcio_tests/commands.py
+++ b/src/python/grpcio_tests/commands.py
@@ -147,7 +147,9 @@ class TestGevent(setuptools.Command):
# TODO(https://github.com/grpc/grpc/issues/17330) enable these three tests
'channelz._channelz_servicer_test.ChannelzServicerTest.test_many_subchannels',
'channelz._channelz_servicer_test.ChannelzServicerTest.test_many_subchannels_and_sockets',
- 'channelz._channelz_servicer_test.ChannelzServicerTest.test_streaming_rpc'
+ 'channelz._channelz_servicer_test.ChannelzServicerTest.test_streaming_rpc',
+ # TODO(https://github.com/grpc/grpc/issues/15411) enable this test
+ 'unit._cython._channel_test.ChannelTest.test_negative_deadline_connectivity'
)
description = 'run tests with gevent. Assumes grpc/gevent are installed'
user_options = []
From c02034eb96c29e27dccda249a6cc180e8f48668c Mon Sep 17 00:00:00 2001
From: Alexander Polcyn <apolcyn@google.com>
Date: Thu, 14 Mar 2019 19:17:14 -0700
Subject: [PATCH 132/138] Add more tracing around the status of each c-ares
query; refactor error handling
---
.../resolver/dns/c_ares/grpc_ares_wrapper.cc | 47 ++++++++-----------
1 file changed, 20 insertions(+), 27 deletions(-)
diff --git a/src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc b/src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc
index 986af89454f..37b0b365eed 100644
--- a/src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc
+++ b/src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc
@@ -67,9 +67,7 @@ struct grpc_ares_request {
/** number of ongoing queries */
size_t pending_queries;
- /** is there at least one successful query, set in on_done_cb */
- bool success;
- /** the errors explaining the request failure, set in on_done_cb */
+ /** the errors explaining query failures, appended to in query callbacks */
grpc_error* error;
};
@@ -145,6 +143,10 @@ void grpc_ares_complete_request_locked(grpc_ares_request* r) {
ServerAddressList* addresses = r->addresses_out->get();
if (addresses != nullptr) {
grpc_cares_wrapper_address_sorting_sort(addresses);
+ GRPC_ERROR_UNREF(r->error);
+ r->error = GRPC_ERROR_NONE;
+ // TODO(apolcyn): allow c-ares to return a service config
+ // with no addresses along side it
}
GRPC_CLOSURE_SCHED(r->on_done, r->error);
}
@@ -175,9 +177,9 @@ static void on_hostbyname_done_locked(void* arg, int status, int timeouts,
static_cast<grpc_ares_hostbyname_request*>(arg);
grpc_ares_request* r = hr->parent_request;
if (status == ARES_SUCCESS) {
- GRPC_ERROR_UNREF(r->error);
- r->error = GRPC_ERROR_NONE;
- r->success = true;
+ GRPC_CARES_TRACE_LOG(
+ "request:%p on_hostbyname_done_locked host=%s ARES_SUCCESS", r,
+ hr->host);
if (*r->addresses_out == nullptr) {
*r->addresses_out = grpc_core::MakeUnique<ServerAddressList>();
}
@@ -229,17 +231,15 @@ static void on_hostbyname_done_locked(void* arg, int status, int timeouts,
}
}
}
- } else if (!r->success) {
+ } else {
char* error_msg;
gpr_asprintf(&error_msg, "C-ares status is not ARES_SUCCESS: %s",
ares_strerror(status));
+ GRPC_CARES_TRACE_LOG("request:%p on_hostbyname_done_locked host=%s %s", r,
+ hr->host, error_msg);
grpc_error* error = GRPC_ERROR_CREATE_FROM_COPIED_STRING(error_msg);
gpr_free(error_msg);
- if (r->error == GRPC_ERROR_NONE) {
- r->error = error;
- } else {
- r->error = grpc_error_add_child(error, r->error);
- }
+ r->error = grpc_error_add_child(error, r->error);
}
destroy_hostbyname_request_locked(hr);
}
@@ -247,9 +247,8 @@ static void on_hostbyname_done_locked(void* arg, int status, int timeouts,
static void on_srv_query_done_locked(void* arg, int status, int timeouts,
unsigned char* abuf, int alen) {
grpc_ares_request* r = static_cast<grpc_ares_request*>(arg);
- GRPC_CARES_TRACE_LOG("request:%p on_query_srv_done_locked", r);
if (status == ARES_SUCCESS) {
- GRPC_CARES_TRACE_LOG("request:%p on_query_srv_done_locked ARES_SUCCESS", r);
+ GRPC_CARES_TRACE_LOG("request:%p on_srv_query_done_locked ARES_SUCCESS", r);
struct ares_srv_reply* reply;
const int parse_status = ares_parse_srv_reply(abuf, alen, &reply);
if (parse_status == ARES_SUCCESS) {
@@ -273,17 +272,15 @@ static void on_srv_query_done_locked(void* arg, int status, int timeouts,
if (reply != nullptr) {
ares_free_data(reply);
}
- } else if (!r->success) {
+ } else {
char* error_msg;
gpr_asprintf(&error_msg, "C-ares status is not ARES_SUCCESS: %s",
ares_strerror(status));
+ GRPC_CARES_TRACE_LOG("request:%p on_srv_query_done_locked %s", r,
+ error_msg);
grpc_error* error = GRPC_ERROR_CREATE_FROM_COPIED_STRING(error_msg);
gpr_free(error_msg);
- if (r->error == GRPC_ERROR_NONE) {
- r->error = error;
- } else {
- r->error = grpc_error_add_child(error, r->error);
- }
+ r->error = grpc_error_add_child(error, r->error);
}
grpc_ares_request_unref_locked(r);
}
@@ -294,12 +291,12 @@ static void on_txt_done_locked(void* arg, int status, int timeouts,
unsigned char* buf, int len) {
char* error_msg;
grpc_ares_request* r = static_cast<grpc_ares_request*>(arg);
- GRPC_CARES_TRACE_LOG("request:%p on_txt_done_locked", r);
const size_t prefix_len = sizeof(g_service_config_attribute_prefix) - 1;
struct ares_txt_ext* result = nullptr;
struct ares_txt_ext* reply = nullptr;
grpc_error* error = GRPC_ERROR_NONE;
if (status != ARES_SUCCESS) goto fail;
+ GRPC_CARES_TRACE_LOG("request:%p on_txt_done_locked ARES_SUCCESS", r);
status = ares_parse_txt_reply_ext(buf, len, &reply);
if (status != ARES_SUCCESS) goto fail;
// Find service config in TXT record.
@@ -337,12 +334,9 @@ fail:
gpr_asprintf(&error_msg, "C-ares TXT lookup status is not ARES_SUCCESS: %s",
ares_strerror(status));
error = GRPC_ERROR_CREATE_FROM_COPIED_STRING(error_msg);
+ GRPC_CARES_TRACE_LOG("request:%p on_txt_done_locked %s", r, error_msg);
gpr_free(error_msg);
- if (r->error == GRPC_ERROR_NONE) {
- r->error = error;
- } else {
- r->error = grpc_error_add_child(error, r->error);
- }
+ r->error = grpc_error_add_child(error, r->error);
done:
grpc_ares_request_unref_locked(r);
}
@@ -534,7 +528,6 @@ static grpc_ares_request* grpc_dns_lookup_ares_locked_impl(
r->on_done = on_done;
r->addresses_out = addrs;
r->service_config_json_out = service_config_json;
- r->success = false;
r->error = GRPC_ERROR_NONE;
r->pending_queries = 0;
GRPC_CARES_TRACE_LOG(
From b0ad6ac3ae04de924fd7c1331b413f7b230772d9 Mon Sep 17 00:00:00 2001
From: "Mark D. Roth" <roth@google.com>
Date: Fri, 15 Mar 2019 13:54:28 -0700
Subject: [PATCH 133/138] Clean up grpclb and xds end2end tests.
---
test/cpp/end2end/grpclb_end2end_test.cc | 633 +++++++++++++-----------
test/cpp/end2end/xds_end2end_test.cc | 396 ++++++++-------
2 files changed, 534 insertions(+), 495 deletions(-)
diff --git a/test/cpp/end2end/grpclb_end2end_test.cc b/test/cpp/end2end/grpclb_end2end_test.cc
index abce031c539..761b6ec39d3 100644
--- a/test/cpp/end2end/grpclb_end2end_test.cc
+++ b/test/cpp/end2end/grpclb_end2end_test.cc
@@ -56,13 +56,8 @@
// TODO(dgq): Other scenarios in need of testing:
// - Send a serverlist with faulty ip:port addresses (port > 2^16, etc).
// - Test reception of invalid serverlist
-// - Test pinging
// - Test against a non-LB server.
// - Random LB server closing the stream unexpectedly.
-// - Test using DNS-resolvable names (localhost?)
-// - Test handling of creation of faulty RR instance by having the LB return a
-// serverlist with non-existent backends after having initially returned a
-// valid one.
//
// Findings from end to end testing to be covered here:
// - Handling of LB servers restart, including reconnection after backing-off
@@ -74,8 +69,6 @@
// part of the grpclb shutdown process.
// 2) the retry timer is active. Again, the weak reference it holds should
// prevent a premature call to \a glb_destroy.
-// - Restart of backend servers with no changes to serverlist. This exercises
-// the RR handover mechanism.
using std::chrono::system_clock;
@@ -149,14 +142,7 @@ class BackendServiceImpl : public BackendService {
return status;
}
- // Returns true on its first invocation, false otherwise.
- bool Shutdown() {
- std::unique_lock<std::mutex> lock(mu_);
- const bool prev = !shutdown_;
- shutdown_ = true;
- gpr_log(GPR_INFO, "Backend: shut down");
- return prev;
- }
+ void Shutdown() {}
std::set<grpc::string> clients() {
std::unique_lock<std::mutex> lock(clients_mu_);
@@ -170,7 +156,6 @@ class BackendServiceImpl : public BackendService {
}
std::mutex mu_;
- bool shutdown_ = false;
std::mutex clients_mu_;
std::set<grpc::string> clients_;
};
@@ -200,6 +185,14 @@ struct ClientStats {
}
return *this;
}
+
+ void Reset() {
+ num_calls_started = 0;
+ num_calls_finished = 0;
+ num_calls_finished_with_client_failed_to_send = 0;
+ num_calls_finished_known_received = 0;
+ drop_token_counts.clear();
+ }
};
class BalancerServiceImpl : public BalancerService {
@@ -209,8 +202,7 @@ class BalancerServiceImpl : public BalancerService {
explicit BalancerServiceImpl(int client_load_reporting_interval_seconds)
: client_load_reporting_interval_seconds_(
- client_load_reporting_interval_seconds),
- shutdown_(false) {}
+ client_load_reporting_interval_seconds) {}
Status BalanceLoad(ServerContext* context, Stream* stream) override {
// Balancer shouldn't receive the call credentials metadata.
@@ -241,16 +233,11 @@ class BalancerServiceImpl : public BalancerService {
responses_and_delays = responses_and_delays_;
}
for (const auto& response_and_delay : responses_and_delays) {
- {
- std::unique_lock<std::mutex> lock(mu_);
- if (shutdown_) goto done;
- }
SendResponse(stream, response_and_delay.first, response_and_delay.second);
}
{
std::unique_lock<std::mutex> lock(mu_);
- if (shutdown_) goto done;
- serverlist_cond_.wait(lock, [this] { return serverlist_ready_; });
+ serverlist_cond_.wait(lock, [this] { return serverlist_done_; });
}
if (client_load_reporting_interval_seconds_ > 0) {
@@ -291,14 +278,12 @@ class BalancerServiceImpl : public BalancerService {
responses_and_delays_.push_back(std::make_pair(response, send_after_ms));
}
- // Returns true on its first invocation, false otherwise.
- bool Shutdown() {
- NotifyDoneWithServerlists();
+ void Shutdown() {
std::unique_lock<std::mutex> lock(mu_);
- const bool prev = !shutdown_;
- shutdown_ = true;
+ NotifyDoneWithServerlistsLocked();
+ responses_and_delays_.clear();
+ client_stats_.Reset();
gpr_log(GPR_INFO, "LB[%p]: shut down", this);
- return prev;
}
static LoadBalanceResponse BuildResponseForBackends(
@@ -334,8 +319,14 @@ class BalancerServiceImpl : public BalancerService {
void NotifyDoneWithServerlists() {
std::lock_guard<std::mutex> lock(mu_);
- serverlist_ready_ = true;
- serverlist_cond_.notify_all();
+ NotifyDoneWithServerlistsLocked();
+ }
+
+ void NotifyDoneWithServerlistsLocked() {
+ if (!serverlist_done_) {
+ serverlist_done_ = true;
+ serverlist_cond_.notify_all();
+ }
}
private:
@@ -357,14 +348,13 @@ class BalancerServiceImpl : public BalancerService {
std::condition_variable load_report_cond_;
bool load_report_ready_ = false;
std::condition_variable serverlist_cond_;
- bool serverlist_ready_ = false;
+ bool serverlist_done_ = false;
ClientStats client_stats_;
- bool shutdown_;
};
class GrpclbEnd2endTest : public ::testing::Test {
protected:
- GrpclbEnd2endTest(int num_backends, int num_balancers,
+ GrpclbEnd2endTest(size_t num_backends, size_t num_balancers,
int client_load_reporting_interval_seconds)
: server_host_("localhost"),
num_backends_(num_backends),
@@ -381,29 +371,35 @@ class GrpclbEnd2endTest : public ::testing::Test {
grpc_core::MakeRefCounted<grpc_core::FakeResolverResponseGenerator>();
// Start the backends.
for (size_t i = 0; i < num_backends_; ++i) {
- backends_.emplace_back(new BackendServiceImpl());
- backend_servers_.emplace_back(ServerThread<BackendService>(
- "backend", server_host_, backends_.back().get()));
+ backends_.emplace_back(new ServerThread<BackendServiceImpl>("backend"));
+ backends_.back()->Start(server_host_);
}
// Start the load balancers.
for (size_t i = 0; i < num_balancers_; ++i) {
- balancers_.emplace_back(
- new BalancerServiceImpl(client_load_reporting_interval_seconds_));
- balancer_servers_.emplace_back(ServerThread<BalancerService>(
- "balancer", server_host_, balancers_.back().get()));
+ balancers_.emplace_back(new ServerThread<BalancerServiceImpl>(
+ "balancer", client_load_reporting_interval_seconds_));
+ balancers_.back()->Start(server_host_);
}
ResetStub();
}
void TearDown() override {
- for (size_t i = 0; i < backends_.size(); ++i) {
- if (backends_[i]->Shutdown()) backend_servers_[i].Shutdown();
- }
- for (size_t i = 0; i < balancers_.size(); ++i) {
- if (balancers_[i]->Shutdown()) balancer_servers_[i].Shutdown();
- }
+ ShutdownAllBackends();
+ for (auto& balancer : balancers_) balancer->Shutdown();
}
+ void StartAllBackends() {
+ for (auto& backend : backends_) backend->Start(server_host_);
+ }
+
+ void StartBackend(size_t index) { backends_[index]->Start(server_host_); }
+
+ void ShutdownAllBackends() {
+ for (auto& backend : backends_) backend->Shutdown();
+ }
+
+ void ShutdownBackend(size_t index) { backends_[index]->Shutdown(); }
+
void ResetStub(int fallback_timeout = 0,
const grpc::string& expected_targets = "") {
ChannelArguments args;
@@ -431,20 +427,21 @@ class GrpclbEnd2endTest : public ::testing::Test {
}
void ResetBackendCounters() {
- for (const auto& backend : backends_) backend->ResetCounters();
+ for (auto& backend : backends_) backend->service_.ResetCounters();
}
ClientStats WaitForLoadReports() {
ClientStats client_stats;
- for (const auto& balancer : balancers_) {
- client_stats += balancer->WaitForLoadReport();
+ for (auto& balancer : balancers_) {
+ client_stats += balancer->service_.WaitForLoadReport();
}
return client_stats;
}
- bool SeenAllBackends() {
- for (const auto& backend : backends_) {
- if (backend->request_count() == 0) return false;
+ bool SeenAllBackends(size_t start_index = 0, size_t stop_index = 0) {
+ if (stop_index == 0) stop_index = backends_.size();
+ for (size_t i = start_index; i < stop_index; ++i) {
+ if (backends_[i]->service_.request_count() == 0) return false;
}
return true;
}
@@ -464,13 +461,14 @@ class GrpclbEnd2endTest : public ::testing::Test {
++*num_total;
}
- std::tuple<int, int, int> WaitForAllBackends(
- int num_requests_multiple_of = 1) {
+ std::tuple<int, int, int> WaitForAllBackends(int num_requests_multiple_of = 1,
+ size_t start_index = 0,
+ size_t stop_index = 0) {
int num_ok = 0;
int num_failure = 0;
int num_drops = 0;
int num_total = 0;
- while (!SeenAllBackends()) {
+ while (!SeenAllBackends(start_index, stop_index)) {
SendRpcAndCount(&num_total, &num_ok, &num_failure, &num_drops);
}
while (num_total % num_requests_multiple_of != 0) {
@@ -488,7 +486,7 @@ class GrpclbEnd2endTest : public ::testing::Test {
void WaitForBackend(size_t backend_idx) {
do {
(void)SendRpc();
- } while (backends_[backend_idx]->request_count() == 0);
+ } while (backends_[backend_idx]->service_.request_count() == 0);
ResetBackendCounters();
}
@@ -528,8 +526,8 @@ class GrpclbEnd2endTest : public ::testing::Test {
void SetNextResolutionAllBalancers(
const char* service_config_json = nullptr) {
std::vector<AddressData> addresses;
- for (size_t i = 0; i < balancer_servers_.size(); ++i) {
- addresses.emplace_back(AddressData{balancer_servers_[i].port_, true, ""});
+ for (size_t i = 0; i < balancers_.size(); ++i) {
+ addresses.emplace_back(AddressData{balancers_[i]->port_, true, ""});
}
SetNextResolution(addresses, service_config_json);
}
@@ -561,10 +559,12 @@ class GrpclbEnd2endTest : public ::testing::Test {
response_generator_->SetReresolutionResponse(&fake_result);
}
- const std::vector<int> GetBackendPorts(const size_t start_index = 0) const {
+ const std::vector<int> GetBackendPorts(size_t start_index = 0,
+ size_t stop_index = 0) const {
+ if (stop_index == 0) stop_index = backends_.size();
std::vector<int> backend_ports;
- for (size_t i = start_index; i < backend_servers_.size(); ++i) {
- backend_ports.push_back(backend_servers_[i].port_);
+ for (size_t i = start_index; i < stop_index; ++i) {
+ backend_ports.push_back(backends_[i]->port_);
}
return backend_ports;
}
@@ -572,7 +572,7 @@ class GrpclbEnd2endTest : public ::testing::Test {
void ScheduleResponseForBalancer(size_t i,
const LoadBalanceResponse& response,
int delay_ms) {
- balancers_.at(i)->add_response(response, delay_ms);
+ balancers_[i]->service_.add_response(response, delay_ms);
}
Status SendRpc(EchoResponse* response = nullptr, int timeout_ms = 1000,
@@ -607,23 +607,28 @@ class GrpclbEnd2endTest : public ::testing::Test {
template <typename T>
struct ServerThread {
- explicit ServerThread(const grpc::string& type,
- const grpc::string& server_host, T* service)
- : type_(type), service_(service) {
+ template <typename... Args>
+ explicit ServerThread(const grpc::string& type, Args&&... args)
+ : port_(grpc_pick_unused_port_or_die()),
+ type_(type),
+ service_(std::forward<Args>(args)...) {}
+
+ void Start(const grpc::string& server_host) {
+ gpr_log(GPR_INFO, "starting %s server on port %d", type_.c_str(), port_);
+ GPR_ASSERT(!running_);
+ running_ = true;
std::mutex mu;
// We need to acquire the lock here in order to prevent the notify_one
- // by ServerThread::Start from firing before the wait below is hit.
+ // by ServerThread::Serve from firing before the wait below is hit.
std::unique_lock<std::mutex> lock(mu);
- port_ = grpc_pick_unused_port_or_die();
- gpr_log(GPR_INFO, "starting %s server on port %d", type_.c_str(), port_);
std::condition_variable cond;
thread_.reset(new std::thread(
- std::bind(&ServerThread::Start, this, server_host, &mu, &cond)));
+ std::bind(&ServerThread::Serve, this, server_host, &mu, &cond)));
cond.wait(lock);
gpr_log(GPR_INFO, "%s server startup complete", type_.c_str());
}
- void Start(const grpc::string& server_host, std::mutex* mu,
+ void Serve(const grpc::string& server_host, std::mutex* mu,
std::condition_variable* cond) {
// We need to acquire the lock here in order to prevent the notify_one
// below from firing before its corresponding wait is executed.
@@ -634,23 +639,27 @@ class GrpclbEnd2endTest : public ::testing::Test {
std::shared_ptr<ServerCredentials> creds(new SecureServerCredentials(
grpc_fake_transport_security_server_credentials_create()));
builder.AddListeningPort(server_address.str(), creds);
- builder.RegisterService(service_);
+ builder.RegisterService(&service_);
server_ = builder.BuildAndStart();
cond->notify_one();
}
void Shutdown() {
+ if (!running_) return;
gpr_log(GPR_INFO, "%s about to shutdown", type_.c_str());
+ service_.Shutdown();
server_->Shutdown(grpc_timeout_milliseconds_to_deadline(0));
thread_->join();
gpr_log(GPR_INFO, "%s shutdown completed", type_.c_str());
+ running_ = false;
}
- int port_;
+ const int port_;
grpc::string type_;
+ T service_;
std::unique_ptr<Server> server_;
- T* service_;
std::unique_ptr<std::thread> thread_;
+ bool running_ = false;
};
const grpc::string server_host_;
@@ -659,10 +668,8 @@ class GrpclbEnd2endTest : public ::testing::Test {
const int client_load_reporting_interval_seconds_;
std::shared_ptr<Channel> channel_;
std::unique_ptr<grpc::testing::EchoTestService::Stub> stub_;
- std::vector<std::unique_ptr<BackendServiceImpl>> backends_;
- std::vector<std::unique_ptr<BalancerServiceImpl>> balancers_;
- std::vector<ServerThread<BackendService>> backend_servers_;
- std::vector<ServerThread<BalancerService>> balancer_servers_;
+ std::vector<std::unique_ptr<ServerThread<BackendServiceImpl>>> backends_;
+ std::vector<std::unique_ptr<ServerThread<BalancerServiceImpl>>> balancers_;
grpc_core::RefCountedPtr<grpc_core::FakeResolverResponseGenerator>
response_generator_;
const grpc::string kRequestMessage_ = "Live long and prosper.";
@@ -689,14 +696,13 @@ TEST_F(SingleBalancerTest, Vanilla) {
// Each backend should have gotten 100 requests.
for (size_t i = 0; i < backends_.size(); ++i) {
- EXPECT_EQ(kNumRpcsPerAddress,
- backend_servers_[i].service_->request_count());
+ EXPECT_EQ(kNumRpcsPerAddress, backends_[i]->service_.request_count());
}
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
// Check LB policy name for the channel.
EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
@@ -714,11 +720,11 @@ TEST_F(SingleBalancerTest, SelectGrpclbWithMigrationServiceConfig) {
0, BalancerServiceImpl::BuildResponseForBackends(GetBackendPorts(), {}),
0);
CheckRpcSendOk(1, 1000 /* timeout_ms */, true /* wait_for_ready */);
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
// Check LB policy name for the channel.
EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
}
@@ -752,7 +758,7 @@ TEST_F(SingleBalancerTest,
const int kFallbackTimeoutMs = 200 * grpc_test_slowdown_factor();
ResetStub(kFallbackTimeoutMs);
// Resolution includes fallback address but no balancers.
- SetNextResolution({AddressData{backend_servers_[0].port_, false, ""}},
+ SetNextResolution({AddressData{backends_[0]->port_, false, ""}},
"{\n"
" \"loadBalancingConfig\":[\n"
" { \"does_not_exist\":{} },\n"
@@ -780,17 +786,17 @@ TEST_F(SingleBalancerTest, UsePickFirstChildPolicy) {
0);
const size_t kNumRpcs = num_backends_ * 2;
CheckRpcSendOk(kNumRpcs, 1000 /* timeout_ms */, true /* wait_for_ready */);
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// Check that all requests went to the first backend. This verifies
// that we used pick_first instead of round_robin as the child policy.
- EXPECT_EQ(backend_servers_[0].service_->request_count(), kNumRpcs);
+ EXPECT_EQ(backends_[0]->service_.request_count(), kNumRpcs);
for (size_t i = 1; i < backends_.size(); ++i) {
- EXPECT_EQ(backend_servers_[i].service_->request_count(), 0UL);
+ EXPECT_EQ(backends_[i]->service_.request_count(), 0UL);
}
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
// Check LB policy name for the channel.
EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
}
@@ -813,9 +819,9 @@ TEST_F(SingleBalancerTest, SwapChildPolicy) {
CheckRpcSendOk(kNumRpcs, 1000 /* timeout_ms */, true /* wait_for_ready */);
// Check that all requests went to the first backend. This verifies
// that we used pick_first instead of round_robin as the child policy.
- EXPECT_EQ(backend_servers_[0].service_->request_count(), kNumRpcs);
+ EXPECT_EQ(backends_[0]->service_.request_count(), kNumRpcs);
for (size_t i = 1; i < backends_.size(); ++i) {
- EXPECT_EQ(backend_servers_[i].service_->request_count(), 0UL);
+ EXPECT_EQ(backends_[i]->service_.request_count(), 0UL);
}
// Send new resolution that removes child policy from service config.
SetNextResolutionAllBalancers("{}");
@@ -824,14 +830,14 @@ TEST_F(SingleBalancerTest, SwapChildPolicy) {
// Check that every backend saw the same number of requests. This verifies
// that we used round_robin.
for (size_t i = 0; i < backends_.size(); ++i) {
- EXPECT_EQ(backend_servers_[i].service_->request_count(), 2UL);
+ EXPECT_EQ(backends_[i]->service_.request_count(), 2UL);
}
// Done.
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
// Check LB policy name for the channel.
EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
}
@@ -848,7 +854,7 @@ TEST_F(SingleBalancerTest, UpdatesGoToMostRecentChildPolicy) {
// Unreachable balancer.
{unreachable_balancer_port, true, ""},
// Fallback address: first backend.
- {backend_servers_[0].port_, false, ""},
+ {backends_[0]->port_, false, ""},
},
"{\n"
" \"loadBalancingConfig\":[\n"
@@ -891,8 +897,8 @@ TEST_F(SingleBalancerTest, UpdatesGoToMostRecentChildPolicy) {
// Unreachable balancer.
{unreachable_balancer_port, true, ""},
// Fallback address: second and third backends.
- {backend_servers_[1].port_, false, ""},
- {backend_servers_[2].port_, false, ""},
+ {backends_[1]->port_, false, ""},
+ {backends_[2]->port_, false, ""},
},
"{\n"
" \"loadBalancingConfig\":[\n"
@@ -912,8 +918,8 @@ TEST_F(SingleBalancerTest, SameBackendListedMultipleTimes) {
SetNextResolutionAllBalancers();
// Same backend listed twice.
std::vector<int> ports;
- ports.push_back(backend_servers_[0].port_);
- ports.push_back(backend_servers_[0].port_);
+ ports.push_back(backends_[0]->port_);
+ ports.push_back(backends_[0]->port_);
const size_t kNumRpcsPerAddress = 10;
ScheduleResponseForBalancer(
0, BalancerServiceImpl::BuildResponseForBackends(ports, {}), 0);
@@ -922,17 +928,16 @@ TEST_F(SingleBalancerTest, SameBackendListedMultipleTimes) {
// Send kNumRpcsPerAddress RPCs per server.
CheckRpcSendOk(kNumRpcsPerAddress * ports.size());
// Backend should have gotten 20 requests.
- EXPECT_EQ(kNumRpcsPerAddress * 2,
- backend_servers_[0].service_->request_count());
+ EXPECT_EQ(kNumRpcsPerAddress * 2, backends_[0]->service_.request_count());
// And they should have come from a single client port, because of
// subchannel sharing.
- EXPECT_EQ(1UL, backends_[0]->clients().size());
- balancers_[0]->NotifyDoneWithServerlists();
+ EXPECT_EQ(1UL, backends_[0]->service_.clients().size());
+ balancers_[0]->service_.NotifyDoneWithServerlists();
}
TEST_F(SingleBalancerTest, SecureNaming) {
ResetStub(0, kApplicationTargetName_ + ";lb");
- SetNextResolution({AddressData{balancer_servers_[0].port_, true, "lb"}});
+ SetNextResolution({AddressData{balancers_[0]->port_, true, "lb"}});
const size_t kNumRpcsPerAddress = 100;
ScheduleResponseForBalancer(
0, BalancerServiceImpl::BuildResponseForBackends(GetBackendPorts(), {}),
@@ -946,14 +951,13 @@ TEST_F(SingleBalancerTest, SecureNaming) {
// Each backend should have gotten 100 requests.
for (size_t i = 0; i < backends_.size(); ++i) {
- EXPECT_EQ(kNumRpcsPerAddress,
- backend_servers_[i].service_->request_count());
+ EXPECT_EQ(kNumRpcsPerAddress, backends_[i]->service_.request_count());
}
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
// Check LB policy name for the channel.
EXPECT_EQ("grpclb", channel_->GetLoadBalancingPolicyName());
}
@@ -965,8 +969,7 @@ TEST_F(SingleBalancerTest, SecureNamingDeathTest) {
ASSERT_DEATH(
{
ResetStub(0, kApplicationTargetName_ + ";lb");
- SetNextResolution(
- {AddressData{balancer_servers_[0].port_, true, "woops"}});
+ SetNextResolution({AddressData{balancers_[0]->port_, true, "woops"}});
channel_->WaitForConnected(grpc_timeout_seconds_to_deadline(1));
},
"");
@@ -993,11 +996,11 @@ TEST_F(SingleBalancerTest, InitiallyEmptyServerlist) {
// populated serverlist but under the call's deadline (which is enforced by
// the call's deadline).
EXPECT_GT(ellapsed_ms.count(), kServerlistDelayMs);
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent two responses.
- EXPECT_EQ(2U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(2U, balancers_[0]->service_.response_count());
}
TEST_F(SingleBalancerTest, AllServersUnreachableFailFast) {
@@ -1012,11 +1015,11 @@ TEST_F(SingleBalancerTest, AllServersUnreachableFailFast) {
const Status status = SendRpc();
// The error shouldn't be DEADLINE_EXCEEDED.
EXPECT_EQ(StatusCode::UNAVAILABLE, status.error_code());
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
}
TEST_F(SingleBalancerTest, Fallback) {
@@ -1027,9 +1030,9 @@ TEST_F(SingleBalancerTest, Fallback) {
ResetStub(kFallbackTimeoutMs);
std::vector<AddressData> addresses;
- addresses.emplace_back(AddressData{balancer_servers_[0].port_, true, ""});
+ addresses.emplace_back(AddressData{balancers_[0]->port_, true, ""});
for (size_t i = 0; i < kNumBackendInResolution; ++i) {
- addresses.emplace_back(AddressData{backend_servers_[i].port_, false, ""});
+ addresses.emplace_back(AddressData{backends_[i]->port_, false, ""});
}
SetNextResolution(addresses);
@@ -1053,10 +1056,10 @@ TEST_F(SingleBalancerTest, Fallback) {
// Fallback is used: each backend returned by the resolver should have
// gotten one request.
for (size_t i = 0; i < kNumBackendInResolution; ++i) {
- EXPECT_EQ(1U, backend_servers_[i].service_->request_count());
+ EXPECT_EQ(1U, backends_[i]->service_.request_count());
}
for (size_t i = kNumBackendInResolution; i < backends_.size(); ++i) {
- EXPECT_EQ(0U, backend_servers_[i].service_->request_count());
+ EXPECT_EQ(0U, backends_[i]->service_.request_count());
}
// Wait until the serverlist reception has been processed and all backends
@@ -1073,17 +1076,17 @@ TEST_F(SingleBalancerTest, Fallback) {
// Serverlist is used: each backend returned by the balancer should
// have gotten one request.
for (size_t i = 0; i < kNumBackendInResolution; ++i) {
- EXPECT_EQ(0U, backend_servers_[i].service_->request_count());
+ EXPECT_EQ(0U, backends_[i]->service_.request_count());
}
for (size_t i = kNumBackendInResolution; i < backends_.size(); ++i) {
- EXPECT_EQ(1U, backend_servers_[i].service_->request_count());
+ EXPECT_EQ(1U, backends_[i]->service_.request_count());
}
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
}
TEST_F(SingleBalancerTest, FallbackUpdate) {
@@ -1095,9 +1098,9 @@ TEST_F(SingleBalancerTest, FallbackUpdate) {
ResetStub(kFallbackTimeoutMs);
std::vector<AddressData> addresses;
- addresses.emplace_back(AddressData{balancer_servers_[0].port_, true, ""});
+ addresses.emplace_back(AddressData{balancers_[0]->port_, true, ""});
for (size_t i = 0; i < kNumBackendInResolution; ++i) {
- addresses.emplace_back(AddressData{backend_servers_[i].port_, false, ""});
+ addresses.emplace_back(AddressData{backends_[i]->port_, false, ""});
}
SetNextResolution(addresses);
@@ -1123,17 +1126,17 @@ TEST_F(SingleBalancerTest, FallbackUpdate) {
// Fallback is used: each backend returned by the resolver should have
// gotten one request.
for (size_t i = 0; i < kNumBackendInResolution; ++i) {
- EXPECT_EQ(1U, backend_servers_[i].service_->request_count());
+ EXPECT_EQ(1U, backends_[i]->service_.request_count());
}
for (size_t i = kNumBackendInResolution; i < backends_.size(); ++i) {
- EXPECT_EQ(0U, backend_servers_[i].service_->request_count());
+ EXPECT_EQ(0U, backends_[i]->service_.request_count());
}
addresses.clear();
- addresses.emplace_back(AddressData{balancer_servers_[0].port_, true, ""});
+ addresses.emplace_back(AddressData{balancers_[0]->port_, true, ""});
for (size_t i = kNumBackendInResolution;
i < kNumBackendInResolution + kNumBackendInResolutionUpdate; ++i) {
- addresses.emplace_back(AddressData{backend_servers_[i].port_, false, ""});
+ addresses.emplace_back(AddressData{backends_[i]->port_, false, ""});
}
SetNextResolution(addresses);
@@ -1152,15 +1155,15 @@ TEST_F(SingleBalancerTest, FallbackUpdate) {
// The resolution update is used: each backend in the resolution update should
// have gotten one request.
for (size_t i = 0; i < kNumBackendInResolution; ++i) {
- EXPECT_EQ(0U, backend_servers_[i].service_->request_count());
+ EXPECT_EQ(0U, backends_[i]->service_.request_count());
}
for (size_t i = kNumBackendInResolution;
i < kNumBackendInResolution + kNumBackendInResolutionUpdate; ++i) {
- EXPECT_EQ(1U, backend_servers_[i].service_->request_count());
+ EXPECT_EQ(1U, backends_[i]->service_.request_count());
}
for (size_t i = kNumBackendInResolution + kNumBackendInResolutionUpdate;
i < backends_.size(); ++i) {
- EXPECT_EQ(0U, backend_servers_[i].service_->request_count());
+ EXPECT_EQ(0U, backends_[i]->service_.request_count());
}
// Wait until the serverlist reception has been processed and all backends
@@ -1180,18 +1183,18 @@ TEST_F(SingleBalancerTest, FallbackUpdate) {
// have gotten one request.
for (size_t i = 0;
i < kNumBackendInResolution + kNumBackendInResolutionUpdate; ++i) {
- EXPECT_EQ(0U, backend_servers_[i].service_->request_count());
+ EXPECT_EQ(0U, backends_[i]->service_.request_count());
}
for (size_t i = kNumBackendInResolution + kNumBackendInResolutionUpdate;
i < backends_.size(); ++i) {
- EXPECT_EQ(1U, backend_servers_[i].service_->request_count());
+ EXPECT_EQ(1U, backends_[i]->service_.request_count());
}
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
}
TEST_F(SingleBalancerTest, FallbackEarlyWhenBalancerChannelFails) {
@@ -1200,7 +1203,7 @@ TEST_F(SingleBalancerTest, FallbackEarlyWhenBalancerChannelFails) {
// Return an unreachable balancer and one fallback backend.
std::vector<AddressData> addresses;
addresses.emplace_back(AddressData{grpc_pick_unused_port_or_die(), true, ""});
- addresses.emplace_back(AddressData{backend_servers_[0].port_, false, ""});
+ addresses.emplace_back(AddressData{backends_[0]->port_, false, ""});
SetNextResolution(addresses);
// Send RPC with deadline less than the fallback timeout and make sure it
// succeeds.
@@ -1218,27 +1221,18 @@ TEST_F(SingleBalancerTest, BackendsRestart) {
channel_->GetState(true /* try_to_connect */);
// Send kNumRpcsPerAddress RPCs per server.
CheckRpcSendOk(kNumRpcsPerAddress * num_backends_);
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
- for (size_t i = 0; i < backends_.size(); ++i) {
- if (backends_[i]->Shutdown()) backend_servers_[i].Shutdown();
- }
- CheckRpcSendFailure();
- for (size_t i = 0; i < num_backends_; ++i) {
- backends_.emplace_back(new BackendServiceImpl());
- backend_servers_.emplace_back(ServerThread<BackendService>(
- "backend", server_host_, backends_.back().get()));
- }
- // The following RPC will fail due to the backend ports having changed. It
- // will nonetheless exercise the grpclb-roundrobin handling of the RR policy
- // having gone into shutdown.
- // TODO(dgq): implement the "backend restart" component as well. We need extra
- // machinery to either update the LB responses "on the fly" or instruct
- // backends which ports to restart on.
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
+ // Stop backends. RPCs should fail.
+ ShutdownAllBackends();
CheckRpcSendFailure();
+ // Restart backends. RPCs should start succeeding again.
+ StartAllBackends();
+ CheckRpcSendOk(1 /* times */, 1000 /* timeout_ms */,
+ true /* wait_for_ready */);
}
class UpdatesTest : public GrpclbEnd2endTest {
@@ -1264,47 +1258,47 @@ TEST_F(UpdatesTest, UpdateBalancers) {
gpr_log(GPR_INFO, "========= DONE WITH FIRST BATCH ==========");
// All 10 requests should have gone to the first backend.
- EXPECT_EQ(10U, backend_servers_[0].service_->request_count());
+ EXPECT_EQ(10U, backends_[0]->service_.request_count());
- balancers_[0]->NotifyDoneWithServerlists();
- balancers_[1]->NotifyDoneWithServerlists();
- balancers_[2]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
+ balancers_[1]->service_.NotifyDoneWithServerlists();
+ balancers_[2]->service_.NotifyDoneWithServerlists();
// Balancer 0 got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.response_count());
std::vector<AddressData> addresses;
- addresses.emplace_back(AddressData{balancer_servers_[1].port_, true, ""});
+ addresses.emplace_back(AddressData{balancers_[1]->port_, true, ""});
gpr_log(GPR_INFO, "========= ABOUT TO UPDATE 1 ==========");
SetNextResolution(addresses);
gpr_log(GPR_INFO, "========= UPDATE 1 DONE ==========");
// Wait until update has been processed, as signaled by the second backend
// receiving a request.
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
WaitForBackend(1);
- backend_servers_[1].service_->ResetCounters();
+ backends_[1]->service_.ResetCounters();
gpr_log(GPR_INFO, "========= BEFORE SECOND BATCH ==========");
CheckRpcSendOk(10);
gpr_log(GPR_INFO, "========= DONE WITH SECOND BATCH ==========");
// All 10 requests should have gone to the second backend.
- EXPECT_EQ(10U, backend_servers_[1].service_->request_count());
-
- balancers_[0]->NotifyDoneWithServerlists();
- balancers_[1]->NotifyDoneWithServerlists();
- balancers_[2]->NotifyDoneWithServerlists();
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
- EXPECT_EQ(1U, balancer_servers_[1].service_->request_count());
- EXPECT_EQ(1U, balancer_servers_[1].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+ EXPECT_EQ(10U, backends_[1]->service_.request_count());
+
+ balancers_[0]->service_.NotifyDoneWithServerlists();
+ balancers_[1]->service_.NotifyDoneWithServerlists();
+ balancers_[2]->service_.NotifyDoneWithServerlists();
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
+ EXPECT_EQ(1U, balancers_[1]->service_.request_count());
+ EXPECT_EQ(1U, balancers_[1]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.response_count());
}
// Send an update with the same set of LBs as the one in SetUp() in order to
@@ -1329,27 +1323,27 @@ TEST_F(UpdatesTest, UpdateBalancersRepeated) {
gpr_log(GPR_INFO, "========= DONE WITH FIRST BATCH ==========");
// All 10 requests should have gone to the first backend.
- EXPECT_EQ(10U, backend_servers_[0].service_->request_count());
+ EXPECT_EQ(10U, backends_[0]->service_.request_count());
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// Balancer 0 got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.response_count());
std::vector<AddressData> addresses;
- addresses.emplace_back(AddressData{balancer_servers_[0].port_, true, ""});
- addresses.emplace_back(AddressData{balancer_servers_[1].port_, true, ""});
- addresses.emplace_back(AddressData{balancer_servers_[2].port_, true, ""});
+ addresses.emplace_back(AddressData{balancers_[0]->port_, true, ""});
+ addresses.emplace_back(AddressData{balancers_[1]->port_, true, ""});
+ addresses.emplace_back(AddressData{balancers_[2]->port_, true, ""});
gpr_log(GPR_INFO, "========= ABOUT TO UPDATE 1 ==========");
SetNextResolution(addresses);
gpr_log(GPR_INFO, "========= UPDATE 1 DONE ==========");
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
gpr_timespec deadline = gpr_time_add(
gpr_now(GPR_CLOCK_REALTIME), gpr_time_from_millis(10000, GPR_TIMESPAN));
// Send 10 seconds worth of RPCs
@@ -1358,17 +1352,17 @@ TEST_F(UpdatesTest, UpdateBalancersRepeated) {
} while (gpr_time_cmp(gpr_now(GPR_CLOCK_REALTIME), deadline) < 0);
// grpclb continued using the original LB call to the first balancer, which
// doesn't assign the second backend.
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
- balancers_[0]->NotifyDoneWithServerlists();
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
+ balancers_[0]->service_.NotifyDoneWithServerlists();
addresses.clear();
- addresses.emplace_back(AddressData{balancer_servers_[0].port_, true, ""});
- addresses.emplace_back(AddressData{balancer_servers_[1].port_, true, ""});
+ addresses.emplace_back(AddressData{balancers_[0]->port_, true, ""});
+ addresses.emplace_back(AddressData{balancers_[1]->port_, true, ""});
gpr_log(GPR_INFO, "========= ABOUT TO UPDATE 2 ==========");
SetNextResolution(addresses);
gpr_log(GPR_INFO, "========= UPDATE 2 DONE ==========");
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
deadline = gpr_time_add(gpr_now(GPR_CLOCK_REALTIME),
gpr_time_from_millis(10000, GPR_TIMESPAN));
// Send 10 seconds worth of RPCs
@@ -1377,13 +1371,13 @@ TEST_F(UpdatesTest, UpdateBalancersRepeated) {
} while (gpr_time_cmp(gpr_now(GPR_CLOCK_REALTIME), deadline) < 0);
// grpclb continued using the original LB call to the first balancer, which
// doesn't assign the second backend.
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
- balancers_[0]->NotifyDoneWithServerlists();
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
+ balancers_[0]->service_.NotifyDoneWithServerlists();
}
TEST_F(UpdatesTest, UpdateBalancersDeadUpdate) {
std::vector<AddressData> addresses;
- addresses.emplace_back(AddressData{balancer_servers_[0].port_, true, ""});
+ addresses.emplace_back(AddressData{balancers_[0]->port_, true, ""});
SetNextResolution(addresses);
const std::vector<int> first_backend{GetBackendPorts()[0]};
const std::vector<int> second_backend{GetBackendPorts()[1]};
@@ -1398,12 +1392,11 @@ TEST_F(UpdatesTest, UpdateBalancersDeadUpdate) {
CheckRpcSendOk(10);
gpr_log(GPR_INFO, "========= DONE WITH FIRST BATCH ==========");
// All 10 requests should have gone to the first backend.
- EXPECT_EQ(10U, backend_servers_[0].service_->request_count());
+ EXPECT_EQ(10U, backends_[0]->service_.request_count());
// Kill balancer 0
gpr_log(GPR_INFO, "********** ABOUT TO KILL BALANCER 0 *************");
- balancers_[0]->NotifyDoneWithServerlists();
- if (balancers_[0]->Shutdown()) balancer_servers_[0].Shutdown();
+ balancers_[0]->Shutdown();
gpr_log(GPR_INFO, "********** KILLED BALANCER 0 *************");
// This is serviced by the existing RR policy
@@ -1411,23 +1404,23 @@ TEST_F(UpdatesTest, UpdateBalancersDeadUpdate) {
CheckRpcSendOk(10);
gpr_log(GPR_INFO, "========= DONE WITH SECOND BATCH ==========");
// All 10 requests should again have gone to the first backend.
- EXPECT_EQ(20U, backend_servers_[0].service_->request_count());
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(20U, backends_[0]->service_.request_count());
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
- balancers_[0]->NotifyDoneWithServerlists();
- balancers_[1]->NotifyDoneWithServerlists();
- balancers_[2]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
+ balancers_[1]->service_.NotifyDoneWithServerlists();
+ balancers_[2]->service_.NotifyDoneWithServerlists();
// Balancer 0 got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.response_count());
addresses.clear();
- addresses.emplace_back(AddressData{balancer_servers_[1].port_, true, ""});
+ addresses.emplace_back(AddressData{balancers_[1]->port_, true, ""});
gpr_log(GPR_INFO, "========= ABOUT TO UPDATE 1 ==========");
SetNextResolution(addresses);
gpr_log(GPR_INFO, "========= UPDATE 1 DONE ==========");
@@ -1435,32 +1428,32 @@ TEST_F(UpdatesTest, UpdateBalancersDeadUpdate) {
// Wait until update has been processed, as signaled by the second backend
// receiving a request. In the meantime, the client continues to be serviced
// (by the first backend) without interruption.
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
WaitForBackend(1);
// This is serviced by the updated RR policy
- backend_servers_[1].service_->ResetCounters();
+ backends_[1]->service_.ResetCounters();
gpr_log(GPR_INFO, "========= BEFORE THIRD BATCH ==========");
CheckRpcSendOk(10);
gpr_log(GPR_INFO, "========= DONE WITH THIRD BATCH ==========");
// All 10 requests should have gone to the second backend.
- EXPECT_EQ(10U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(10U, backends_[1]->service_.request_count());
- balancers_[0]->NotifyDoneWithServerlists();
- balancers_[1]->NotifyDoneWithServerlists();
- balancers_[2]->NotifyDoneWithServerlists();
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ balancers_[0]->service_.NotifyDoneWithServerlists();
+ balancers_[1]->service_.NotifyDoneWithServerlists();
+ balancers_[2]->service_.NotifyDoneWithServerlists();
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
// The second balancer, published as part of the first update, may end up
// getting two requests (that is, 1 <= #req <= 2) if the LB call retry timer
// firing races with the arrival of the update containing the second
// balancer.
- EXPECT_GE(balancer_servers_[1].service_->request_count(), 1U);
- EXPECT_GE(balancer_servers_[1].service_->response_count(), 1U);
- EXPECT_LE(balancer_servers_[1].service_->request_count(), 2U);
- EXPECT_LE(balancer_servers_[1].service_->response_count(), 2U);
- EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+ EXPECT_GE(balancers_[1]->service_.request_count(), 1U);
+ EXPECT_GE(balancers_[1]->service_.response_count(), 1U);
+ EXPECT_LE(balancers_[1]->service_.request_count(), 2U);
+ EXPECT_LE(balancers_[1]->service_.response_count(), 2U);
+ EXPECT_EQ(0U, balancers_[2]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.response_count());
}
TEST_F(UpdatesTest, ReresolveDeadBackend) {
@@ -1468,14 +1461,14 @@ TEST_F(UpdatesTest, ReresolveDeadBackend) {
// The first resolution contains the addresses of a balancer that never
// responds, and a fallback backend.
std::vector<AddressData> addresses;
- addresses.emplace_back(AddressData{balancer_servers_[0].port_, true, ""});
- addresses.emplace_back(AddressData{backend_servers_[0].port_, false, ""});
+ addresses.emplace_back(AddressData{balancers_[0]->port_, true, ""});
+ addresses.emplace_back(AddressData{backends_[0]->port_, false, ""});
SetNextResolution(addresses);
// The re-resolution result will contain the addresses of the same balancer
// and a new fallback backend.
addresses.clear();
- addresses.emplace_back(AddressData{balancer_servers_[0].port_, true, ""});
- addresses.emplace_back(AddressData{backend_servers_[1].port_, false, ""});
+ addresses.emplace_back(AddressData{balancers_[0]->port_, true, ""});
+ addresses.emplace_back(AddressData{backends_[1]->port_, false, ""});
SetNextReresolutionResponse(addresses);
// Start servers and send 10 RPCs per server.
@@ -1483,11 +1476,11 @@ TEST_F(UpdatesTest, ReresolveDeadBackend) {
CheckRpcSendOk(10);
gpr_log(GPR_INFO, "========= DONE WITH FIRST BATCH ==========");
// All 10 requests should have gone to the fallback backend.
- EXPECT_EQ(10U, backend_servers_[0].service_->request_count());
+ EXPECT_EQ(10U, backends_[0]->service_.request_count());
// Kill backend 0.
gpr_log(GPR_INFO, "********** ABOUT TO KILL BACKEND 0 *************");
- if (backends_[0]->Shutdown()) backend_servers_[0].Shutdown();
+ backends_[0]->Shutdown();
gpr_log(GPR_INFO, "********** KILLED BACKEND 0 *************");
// Wait until re-resolution has finished, as signaled by the second backend
@@ -1498,17 +1491,17 @@ TEST_F(UpdatesTest, ReresolveDeadBackend) {
CheckRpcSendOk(10);
gpr_log(GPR_INFO, "========= DONE WITH SECOND BATCH ==========");
// All 10 requests should have gone to the second backend.
- EXPECT_EQ(10U, backend_servers_[1].service_->request_count());
-
- balancers_[0]->NotifyDoneWithServerlists();
- balancers_[1]->NotifyDoneWithServerlists();
- balancers_[2]->NotifyDoneWithServerlists();
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[0].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+ EXPECT_EQ(10U, backends_[1]->service_.request_count());
+
+ balancers_[0]->service_.NotifyDoneWithServerlists();
+ balancers_[1]->service_.NotifyDoneWithServerlists();
+ balancers_[2]->service_.NotifyDoneWithServerlists();
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[0]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.response_count());
}
// TODO(juanlishen): Should be removed when the first response is always the
@@ -1523,10 +1516,10 @@ class UpdatesWithClientLoadReportingTest : public GrpclbEnd2endTest {
TEST_F(UpdatesWithClientLoadReportingTest, ReresolveDeadBalancer) {
std::vector<AddressData> addresses;
- addresses.emplace_back(AddressData{balancer_servers_[0].port_, true, ""});
+ addresses.emplace_back(AddressData{balancers_[0]->port_, true, ""});
SetNextResolution(addresses);
addresses.clear();
- addresses.emplace_back(AddressData{balancer_servers_[1].port_, true, ""});
+ addresses.emplace_back(AddressData{balancers_[1]->port_, true, ""});
SetNextReresolutionResponse(addresses);
const std::vector<int> first_backend{GetBackendPorts()[0]};
const std::vector<int> second_backend{GetBackendPorts()[1]};
@@ -1541,27 +1534,27 @@ TEST_F(UpdatesWithClientLoadReportingTest, ReresolveDeadBalancer) {
CheckRpcSendOk(10);
gpr_log(GPR_INFO, "========= DONE WITH FIRST BATCH ==========");
// All 10 requests should have gone to the first backend.
- EXPECT_EQ(10U, backend_servers_[0].service_->request_count());
+ EXPECT_EQ(10U, backends_[0]->service_.request_count());
// Kill backend 0.
gpr_log(GPR_INFO, "********** ABOUT TO KILL BACKEND 0 *************");
- if (backends_[0]->Shutdown()) backend_servers_[0].Shutdown();
+ backends_[0]->Shutdown();
gpr_log(GPR_INFO, "********** KILLED BACKEND 0 *************");
CheckRpcSendFailure();
// Balancer 0 got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.response_count());
// Kill balancer 0.
gpr_log(GPR_INFO, "********** ABOUT TO KILL BALANCER 0 *************");
- if (balancers_[0]->Shutdown()) balancer_servers_[0].Shutdown();
+ balancers_[0]->Shutdown();
gpr_log(GPR_INFO, "********** KILLED BALANCER 0 *************");
// Wait until re-resolution has finished, as signaled by the second backend
@@ -1573,22 +1566,22 @@ TEST_F(UpdatesWithClientLoadReportingTest, ReresolveDeadBalancer) {
CheckRpcSendOk(10);
gpr_log(GPR_INFO, "========= DONE WITH SECOND BATCH ==========");
// All 10 requests should have gone to the second backend.
- EXPECT_EQ(10U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(10U, backends_[1]->service_.request_count());
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
// After balancer 0 is killed, we restart an LB call immediately (because we
// disconnect to a previously connected balancer). Although we will cancel
// this call when the re-resolution update is done and another LB call restart
// is needed, this old call may still succeed reaching the LB server if
// re-resolution is slow. So balancer 1 may have received 2 requests and sent
// 2 responses.
- EXPECT_GE(balancer_servers_[1].service_->request_count(), 1U);
- EXPECT_GE(balancer_servers_[1].service_->response_count(), 1U);
- EXPECT_LE(balancer_servers_[1].service_->request_count(), 2U);
- EXPECT_LE(balancer_servers_[1].service_->response_count(), 2U);
- EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+ EXPECT_GE(balancers_[1]->service_.request_count(), 1U);
+ EXPECT_GE(balancers_[1]->service_.response_count(), 1U);
+ EXPECT_LE(balancers_[1]->service_.request_count(), 2U);
+ EXPECT_LE(balancers_[1]->service_.response_count(), 2U);
+ EXPECT_EQ(0U, balancers_[2]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.response_count());
}
TEST_F(SingleBalancerTest, Drop) {
@@ -1623,16 +1616,14 @@ TEST_F(SingleBalancerTest, Drop) {
}
}
EXPECT_EQ(kNumRpcsPerAddress * num_of_drop_addresses, num_drops);
-
// Each backend should have gotten 100 requests.
for (size_t i = 0; i < backends_.size(); ++i) {
- EXPECT_EQ(kNumRpcsPerAddress,
- backend_servers_[i].service_->request_count());
+ EXPECT_EQ(kNumRpcsPerAddress, backends_[i]->service_.request_count());
}
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
}
TEST_F(SingleBalancerTest, DropAllFirst) {
@@ -1682,9 +1673,6 @@ class SingleBalancerWithClientLoadReportingTest : public GrpclbEnd2endTest {
SingleBalancerWithClientLoadReportingTest() : GrpclbEnd2endTest(4, 1, 3) {}
};
-// TODO(roth): Add test that when switching balancers, we don't include
-// any calls that were sent prior to connecting to the new balancer.
-
TEST_F(SingleBalancerWithClientLoadReportingTest, Vanilla) {
SetNextResolutionAllBalancers();
const size_t kNumRpcsPerAddress = 100;
@@ -1700,14 +1688,13 @@ TEST_F(SingleBalancerWithClientLoadReportingTest, Vanilla) {
CheckRpcSendOk(kNumRpcsPerAddress * num_backends_);
// Each backend should have gotten 100 requests.
for (size_t i = 0; i < backends_.size(); ++i) {
- EXPECT_EQ(kNumRpcsPerAddress,
- backend_servers_[i].service_->request_count());
+ EXPECT_EQ(kNumRpcsPerAddress, backends_[i]->service_.request_count());
}
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
const ClientStats client_stats = WaitForLoadReports();
EXPECT_EQ(kNumRpcsPerAddress * num_backends_ + num_ok,
@@ -1720,6 +1707,69 @@ TEST_F(SingleBalancerWithClientLoadReportingTest, Vanilla) {
EXPECT_THAT(client_stats.drop_token_counts, ::testing::ElementsAre());
}
+TEST_F(SingleBalancerWithClientLoadReportingTest, BalancerRestart) {
+ SetNextResolutionAllBalancers();
+ const size_t kNumBackendsFirstPass = 2;
+ const size_t kNumBackendsSecondPass =
+ backends_.size() - kNumBackendsFirstPass;
+ // Balancer returns backends starting at index 1.
+ ScheduleResponseForBalancer(
+ 0,
+ BalancerServiceImpl::BuildResponseForBackends(
+ GetBackendPorts(0, kNumBackendsFirstPass), {}),
+ 0);
+ // Wait until all backends returned by the balancer are ready.
+ int num_ok = 0;
+ int num_failure = 0;
+ int num_drops = 0;
+ std::tie(num_ok, num_failure, num_drops) =
+ WaitForAllBackends(/* num_requests_multiple_of */ 1, /* start_index */ 0,
+ /* stop_index */ kNumBackendsFirstPass);
+ balancers_[0]->service_.NotifyDoneWithServerlists();
+ ClientStats client_stats = WaitForLoadReports();
+ EXPECT_EQ(static_cast<size_t>(num_ok), client_stats.num_calls_started);
+ EXPECT_EQ(static_cast<size_t>(num_ok), client_stats.num_calls_finished);
+ EXPECT_EQ(0U, client_stats.num_calls_finished_with_client_failed_to_send);
+ EXPECT_EQ(static_cast<size_t>(num_ok),
+ client_stats.num_calls_finished_known_received);
+ EXPECT_THAT(client_stats.drop_token_counts, ::testing::ElementsAre());
+ // Shut down the balancer.
+ balancers_[0]->Shutdown();
+ // Send 10 more requests per backend. This will continue using the
+ // last serverlist we received from the balancer before it was shut down.
+ ResetBackendCounters();
+ CheckRpcSendOk(kNumBackendsFirstPass);
+ // Each backend should have gotten 1 request.
+ for (size_t i = 0; i < kNumBackendsFirstPass; ++i) {
+ EXPECT_EQ(1UL, backends_[i]->service_.request_count());
+ }
+ // Now restart the balancer, this time pointing to all backends.
+ balancers_[0]->Start(server_host_);
+ ScheduleResponseForBalancer(0,
+ BalancerServiceImpl::BuildResponseForBackends(
+ GetBackendPorts(kNumBackendsFirstPass), {}),
+ 0);
+ // Wait for queries to start going to one of the new backends.
+ // This tells us that we're now using the new serverlist.
+ do {
+ CheckRpcSendOk();
+ } while (backends_[2]->service_.request_count() == 0 &&
+ backends_[3]->service_.request_count() == 0);
+ // Send one RPC per backend.
+ CheckRpcSendOk(kNumBackendsSecondPass);
+ balancers_[0]->service_.NotifyDoneWithServerlists();
+ EXPECT_EQ(2U, balancers_[0]->service_.request_count());
+ EXPECT_EQ(2U, balancers_[0]->service_.response_count());
+ // Check client stats.
+ client_stats = WaitForLoadReports();
+ EXPECT_EQ(kNumBackendsSecondPass + 1, client_stats.num_calls_started);
+ EXPECT_EQ(kNumBackendsSecondPass + 1, client_stats.num_calls_finished);
+ EXPECT_EQ(0U, client_stats.num_calls_finished_with_client_failed_to_send);
+ EXPECT_EQ(kNumBackendsSecondPass + 1,
+ client_stats.num_calls_finished_known_received);
+ EXPECT_THAT(client_stats.drop_token_counts, ::testing::ElementsAre());
+}
+
TEST_F(SingleBalancerWithClientLoadReportingTest, Drop) {
SetNextResolutionAllBalancers();
const size_t kNumRpcsPerAddress = 3;
@@ -1759,14 +1809,13 @@ TEST_F(SingleBalancerWithClientLoadReportingTest, Drop) {
EXPECT_EQ(kNumRpcsPerAddress * num_of_drop_addresses, num_drops);
// Each backend should have gotten 100 requests.
for (size_t i = 0; i < backends_.size(); ++i) {
- EXPECT_EQ(kNumRpcsPerAddress,
- backend_servers_[i].service_->request_count());
+ EXPECT_EQ(kNumRpcsPerAddress, backends_[i]->service_.request_count());
}
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
const ClientStats client_stats = WaitForLoadReports();
EXPECT_EQ(
diff --git a/test/cpp/end2end/xds_end2end_test.cc b/test/cpp/end2end/xds_end2end_test.cc
index 667481bafd2..8657ba78d90 100644
--- a/test/cpp/end2end/xds_end2end_test.cc
+++ b/test/cpp/end2end/xds_end2end_test.cc
@@ -56,13 +56,8 @@
// TODO(dgq): Other scenarios in need of testing:
// - Send a serverlist with faulty ip:port addresses (port > 2^16, etc).
// - Test reception of invalid serverlist
-// - Test pinging
// - Test against a non-LB server.
// - Random LB server closing the stream unexpectedly.
-// - Test using DNS-resolvable names (localhost?)
-// - Test handling of creation of faulty RR instance by having the LB return a
-// serverlist with non-existent backends after having initially returned a
-// valid one.
//
// Findings from end to end testing to be covered here:
// - Handling of LB servers restart, including reconnection after backing-off
@@ -74,8 +69,6 @@
// part of the xds shutdown process.
// 2) the retry timer is active. Again, the weak reference it holds should
// prevent a premature call to \a glb_destroy.
-// - Restart of backend servers with no changes to serverlist. This exercises
-// the RR handover mechanism.
using std::chrono::system_clock;
@@ -149,14 +142,7 @@ class BackendServiceImpl : public BackendService {
return status;
}
- // Returns true on its first invocation, false otherwise.
- bool Shutdown() {
- std::unique_lock<std::mutex> lock(mu_);
- const bool prev = !shutdown_;
- shutdown_ = true;
- gpr_log(GPR_INFO, "Backend: shut down");
- return prev;
- }
+ void Shutdown() {}
std::set<grpc::string> clients() {
std::unique_lock<std::mutex> lock(clients_mu_);
@@ -170,7 +156,6 @@ class BackendServiceImpl : public BackendService {
}
std::mutex mu_;
- bool shutdown_ = false;
std::mutex clients_mu_;
std::set<grpc::string> clients_;
};
@@ -200,6 +185,14 @@ struct ClientStats {
}
return *this;
}
+
+ void Reset() {
+ num_calls_started = 0;
+ num_calls_finished = 0;
+ num_calls_finished_with_client_failed_to_send = 0;
+ num_calls_finished_known_received = 0;
+ drop_token_counts.clear();
+ }
};
class BalancerServiceImpl : public BalancerService {
@@ -209,17 +202,11 @@ class BalancerServiceImpl : public BalancerService {
explicit BalancerServiceImpl(int client_load_reporting_interval_seconds)
: client_load_reporting_interval_seconds_(
- client_load_reporting_interval_seconds),
- shutdown_(false) {}
+ client_load_reporting_interval_seconds) {}
Status BalanceLoad(ServerContext* context, Stream* stream) override {
// TODO(juanlishen): Clean up the scoping.
gpr_log(GPR_INFO, "LB[%p]: BalanceLoad", this);
- {
- std::unique_lock<std::mutex> lock(mu_);
- if (shutdown_) goto done;
- }
-
{
// Balancer shouldn't receive the call credentials metadata.
EXPECT_EQ(context->client_metadata().find(g_kCallCredsMdKey),
@@ -247,17 +234,12 @@ class BalancerServiceImpl : public BalancerService {
responses_and_delays = responses_and_delays_;
}
for (const auto& response_and_delay : responses_and_delays) {
- {
- std::unique_lock<std::mutex> lock(mu_);
- if (shutdown_) goto done;
- }
SendResponse(stream, response_and_delay.first,
response_and_delay.second);
}
{
std::unique_lock<std::mutex> lock(mu_);
- if (shutdown_) goto done;
- serverlist_cond_.wait(lock, [this] { return serverlist_ready_; });
+ serverlist_cond_.wait(lock, [this] { return serverlist_done_; });
}
if (client_load_reporting_interval_seconds_ > 0) {
@@ -299,17 +281,12 @@ class BalancerServiceImpl : public BalancerService {
responses_and_delays_.push_back(std::make_pair(response, send_after_ms));
}
- // Returns true on its first invocation, false otherwise.
- bool Shutdown() {
- bool prev;
- {
- std::unique_lock<std::mutex> lock(mu_);
- prev = !shutdown_;
- shutdown_ = true;
- }
- NotifyDoneWithServerlists();
+ void Shutdown() {
+ std::unique_lock<std::mutex> lock(mu_);
+ NotifyDoneWithServerlistsLocked();
+ responses_and_delays_.clear();
+ client_stats_.Reset();
gpr_log(GPR_INFO, "LB[%p]: shut down", this);
- return prev;
}
static LoadBalanceResponse BuildResponseForBackends(
@@ -345,8 +322,14 @@ class BalancerServiceImpl : public BalancerService {
void NotifyDoneWithServerlists() {
std::lock_guard<std::mutex> lock(mu_);
- serverlist_ready_ = true;
- serverlist_cond_.notify_all();
+ NotifyDoneWithServerlistsLocked();
+ }
+
+ void NotifyDoneWithServerlistsLocked() {
+ if (!serverlist_done_) {
+ serverlist_done_ = true;
+ serverlist_cond_.notify_all();
+ }
}
private:
@@ -368,14 +351,13 @@ class BalancerServiceImpl : public BalancerService {
std::condition_variable load_report_cond_;
bool load_report_ready_ = false;
std::condition_variable serverlist_cond_;
- bool serverlist_ready_ = false;
+ bool serverlist_done_ = false;
ClientStats client_stats_;
- bool shutdown_;
};
class XdsEnd2endTest : public ::testing::Test {
protected:
- XdsEnd2endTest(int num_backends, int num_balancers,
+ XdsEnd2endTest(size_t num_backends, size_t num_balancers,
int client_load_reporting_interval_seconds)
: server_host_("localhost"),
num_backends_(num_backends),
@@ -394,29 +376,35 @@ class XdsEnd2endTest : public ::testing::Test {
grpc_core::MakeRefCounted<grpc_core::FakeResolverResponseGenerator>();
// Start the backends.
for (size_t i = 0; i < num_backends_; ++i) {
- backends_.emplace_back(new BackendServiceImpl());
- backend_servers_.emplace_back(ServerThread<BackendService>(
- "backend", server_host_, backends_.back().get()));
+ backends_.emplace_back(new ServerThread<BackendServiceImpl>("backend"));
+ backends_.back()->Start(server_host_);
}
// Start the load balancers.
for (size_t i = 0; i < num_balancers_; ++i) {
- balancers_.emplace_back(
- new BalancerServiceImpl(client_load_reporting_interval_seconds_));
- balancer_servers_.emplace_back(ServerThread<BalancerService>(
- "balancer", server_host_, balancers_.back().get()));
+ balancers_.emplace_back(new ServerThread<BalancerServiceImpl>(
+ "balancer", client_load_reporting_interval_seconds_));
+ balancers_.back()->Start(server_host_);
}
ResetStub();
}
void TearDown() override {
- for (size_t i = 0; i < backends_.size(); ++i) {
- if (backends_[i]->Shutdown()) backend_servers_[i].Shutdown();
- }
- for (size_t i = 0; i < balancers_.size(); ++i) {
- if (balancers_[i]->Shutdown()) balancer_servers_[i].Shutdown();
- }
+ ShutdownAllBackends();
+ for (auto& balancer : balancers_) balancer->Shutdown();
+ }
+
+ void StartAllBackends() {
+ for (auto& backend : backends_) backend->Start(server_host_);
+ }
+
+ void StartBackend(size_t index) { backends_[index]->Start(server_host_); }
+
+ void ShutdownAllBackends() {
+ for (auto& backend : backends_) backend->Shutdown();
}
+ void ShutdownBackend(size_t index) { backends_[index]->Shutdown(); }
+
void ResetStub(int fallback_timeout = 0,
const grpc::string& expected_targets = "") {
ChannelArguments args;
@@ -445,20 +433,21 @@ class XdsEnd2endTest : public ::testing::Test {
}
void ResetBackendCounters() {
- for (const auto& backend : backends_) backend->ResetCounters();
+ for (auto& backend : backends_) backend->service_.ResetCounters();
}
ClientStats WaitForLoadReports() {
ClientStats client_stats;
- for (const auto& balancer : balancers_) {
- client_stats += balancer->WaitForLoadReport();
+ for (auto& balancer : balancers_) {
+ client_stats += balancer->service_.WaitForLoadReport();
}
return client_stats;
}
- bool SeenAllBackends() {
- for (const auto& backend : backends_) {
- if (backend->request_count() == 0) return false;
+ bool SeenAllBackends(size_t start_index = 0, size_t stop_index = 0) {
+ if (stop_index == 0) stop_index = backends_.size();
+ for (size_t i = start_index; i < stop_index; ++i) {
+ if (backends_[i]->service_.request_count() == 0) return false;
}
return true;
}
@@ -478,13 +467,14 @@ class XdsEnd2endTest : public ::testing::Test {
++*num_total;
}
- std::tuple<int, int, int> WaitForAllBackends(
- int num_requests_multiple_of = 1) {
+ std::tuple<int, int, int> WaitForAllBackends(int num_requests_multiple_of = 1,
+ size_t start_index = 0,
+ size_t stop_index = 0) {
int num_ok = 0;
int num_failure = 0;
int num_drops = 0;
int num_total = 0;
- while (!SeenAllBackends()) {
+ while (!SeenAllBackends(start_index, stop_index)) {
SendRpcAndCount(&num_total, &num_ok, &num_failure, &num_drops);
}
while (num_total % num_requests_multiple_of != 0) {
@@ -502,7 +492,7 @@ class XdsEnd2endTest : public ::testing::Test {
void WaitForBackend(size_t backend_idx) {
do {
(void)SendRpc();
- } while (backends_[backend_idx]->request_count() == 0);
+ } while (backends_[backend_idx]->service_.request_count() == 0);
ResetBackendCounters();
}
@@ -553,8 +543,8 @@ class XdsEnd2endTest : public ::testing::Test {
grpc_core::FakeResolverResponseGenerator* lb_channel_response_generator =
nullptr) {
std::vector<int> ports;
- for (size_t i = 0; i < balancer_servers_.size(); ++i) {
- ports.emplace_back(balancer_servers_[i].port_);
+ for (size_t i = 0; i < balancers_.size(); ++i) {
+ ports.emplace_back(balancers_[i]->port_);
}
SetNextResolutionForLbChannel(ports, service_config_json,
lb_channel_response_generator);
@@ -591,10 +581,12 @@ class XdsEnd2endTest : public ::testing::Test {
response_generator_->SetReresolutionResponse(&fake_result);
}
- const std::vector<int> GetBackendPorts(const size_t start_index = 0) const {
+ const std::vector<int> GetBackendPorts(size_t start_index = 0,
+ size_t stop_index = 0) const {
+ if (stop_index == 0) stop_index = backends_.size();
std::vector<int> backend_ports;
- for (size_t i = start_index; i < backend_servers_.size(); ++i) {
- backend_ports.push_back(backend_servers_[i].port_);
+ for (size_t i = start_index; i < stop_index; ++i) {
+ backend_ports.push_back(backends_[i]->port_);
}
return backend_ports;
}
@@ -602,7 +594,7 @@ class XdsEnd2endTest : public ::testing::Test {
void ScheduleResponseForBalancer(size_t i,
const LoadBalanceResponse& response,
int delay_ms) {
- balancers_.at(i)->add_response(response, delay_ms);
+ balancers_[i]->service_.add_response(response, delay_ms);
}
Status SendRpc(EchoResponse* response = nullptr, int timeout_ms = 1000,
@@ -637,23 +629,28 @@ class XdsEnd2endTest : public ::testing::Test {
template <typename T>
struct ServerThread {
- explicit ServerThread(const grpc::string& type,
- const grpc::string& server_host, T* service)
- : type_(type), service_(service) {
+ template <typename... Args>
+ explicit ServerThread(const grpc::string& type, Args&&... args)
+ : port_(grpc_pick_unused_port_or_die()),
+ type_(type),
+ service_(std::forward<Args>(args)...) {}
+
+ void Start(const grpc::string& server_host) {
+ gpr_log(GPR_INFO, "starting %s server on port %d", type_.c_str(), port_);
+ GPR_ASSERT(!running_);
+ running_ = true;
std::mutex mu;
// We need to acquire the lock here in order to prevent the notify_one
- // by ServerThread::Start from firing before the wait below is hit.
+ // by ServerThread::Serve from firing before the wait below is hit.
std::unique_lock<std::mutex> lock(mu);
- port_ = grpc_pick_unused_port_or_die();
- gpr_log(GPR_INFO, "starting %s server on port %d", type_.c_str(), port_);
std::condition_variable cond;
thread_.reset(new std::thread(
- std::bind(&ServerThread::Start, this, server_host, &mu, &cond)));
+ std::bind(&ServerThread::Serve, this, server_host, &mu, &cond)));
cond.wait(lock);
gpr_log(GPR_INFO, "%s server startup complete", type_.c_str());
}
- void Start(const grpc::string& server_host, std::mutex* mu,
+ void Serve(const grpc::string& server_host, std::mutex* mu,
std::condition_variable* cond) {
// We need to acquire the lock here in order to prevent the notify_one
// below from firing before its corresponding wait is executed.
@@ -664,23 +661,27 @@ class XdsEnd2endTest : public ::testing::Test {
std::shared_ptr<ServerCredentials> creds(new SecureServerCredentials(
grpc_fake_transport_security_server_credentials_create()));
builder.AddListeningPort(server_address.str(), creds);
- builder.RegisterService(service_);
+ builder.RegisterService(&service_);
server_ = builder.BuildAndStart();
cond->notify_one();
}
void Shutdown() {
+ if (!running_) return;
gpr_log(GPR_INFO, "%s about to shutdown", type_.c_str());
+ service_.Shutdown();
server_->Shutdown(grpc_timeout_milliseconds_to_deadline(0));
thread_->join();
gpr_log(GPR_INFO, "%s shutdown completed", type_.c_str());
+ running_ = false;
}
- int port_;
+ const int port_;
grpc::string type_;
+ T service_;
std::unique_ptr<Server> server_;
- T* service_;
std::unique_ptr<std::thread> thread_;
+ bool running_ = false;
};
const grpc::string server_host_;
@@ -689,10 +690,8 @@ class XdsEnd2endTest : public ::testing::Test {
const int client_load_reporting_interval_seconds_;
std::shared_ptr<Channel> channel_;
std::unique_ptr<grpc::testing::EchoTestService::Stub> stub_;
- std::vector<std::unique_ptr<BackendServiceImpl>> backends_;
- std::vector<std::unique_ptr<BalancerServiceImpl>> balancers_;
- std::vector<ServerThread<BackendService>> backend_servers_;
- std::vector<ServerThread<BalancerService>> balancer_servers_;
+ std::vector<std::unique_ptr<ServerThread<BackendServiceImpl>>> backends_;
+ std::vector<std::unique_ptr<ServerThread<BalancerServiceImpl>>> balancers_;
grpc_core::RefCountedPtr<grpc_core::FakeResolverResponseGenerator>
response_generator_;
grpc_core::RefCountedPtr<grpc_core::FakeResolverResponseGenerator>
@@ -728,14 +727,13 @@ TEST_F(SingleBalancerTest, Vanilla) {
CheckRpcSendOk(kNumRpcsPerAddress * num_backends_);
// Each backend should have gotten 100 requests.
for (size_t i = 0; i < backends_.size(); ++i) {
- EXPECT_EQ(kNumRpcsPerAddress,
- backend_servers_[i].service_->request_count());
+ EXPECT_EQ(kNumRpcsPerAddress, backends_[i]->service_.request_count());
}
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
// Check LB policy name for the channel.
EXPECT_EQ("xds_experimental", channel_->GetLoadBalancingPolicyName());
@@ -746,8 +744,8 @@ TEST_F(SingleBalancerTest, SameBackendListedMultipleTimes) {
SetNextResolutionForLbChannelAllBalancers();
// Same backend listed twice.
std::vector<int> ports;
- ports.push_back(backend_servers_[0].port_);
- ports.push_back(backend_servers_[0].port_);
+ ports.push_back(backends_[0]->port_);
+ ports.push_back(backends_[0]->port_);
const size_t kNumRpcsPerAddress = 10;
ScheduleResponseForBalancer(
0, BalancerServiceImpl::BuildResponseForBackends(ports, {}), 0);
@@ -756,19 +754,18 @@ TEST_F(SingleBalancerTest, SameBackendListedMultipleTimes) {
// Send kNumRpcsPerAddress RPCs per server.
CheckRpcSendOk(kNumRpcsPerAddress * ports.size());
// Backend should have gotten 20 requests.
- EXPECT_EQ(kNumRpcsPerAddress * 2,
- backend_servers_[0].service_->request_count());
+ EXPECT_EQ(kNumRpcsPerAddress * 2, backends_[0]->service_.request_count());
// And they should have come from a single client port, because of
// subchannel sharing.
- EXPECT_EQ(1UL, backends_[0]->clients().size());
- balancers_[0]->NotifyDoneWithServerlists();
+ EXPECT_EQ(1UL, backends_[0]->service_.clients().size());
+ balancers_[0]->service_.NotifyDoneWithServerlists();
}
TEST_F(SingleBalancerTest, SecureNaming) {
// TODO(juanlishen): Use separate fake creds for the balancer channel.
ResetStub(0, kApplicationTargetName_ + ";lb");
SetNextResolution({}, kDefaultServiceConfig_.c_str());
- SetNextResolutionForLbChannel({balancer_servers_[0].port_});
+ SetNextResolutionForLbChannel({balancers_[0]->port_});
const size_t kNumRpcsPerAddress = 100;
ScheduleResponseForBalancer(
0, BalancerServiceImpl::BuildResponseForBackends(GetBackendPorts(), {}),
@@ -782,13 +779,12 @@ TEST_F(SingleBalancerTest, SecureNaming) {
// Each backend should have gotten 100 requests.
for (size_t i = 0; i < backends_.size(); ++i) {
- EXPECT_EQ(kNumRpcsPerAddress,
- backend_servers_[i].service_->request_count());
+ EXPECT_EQ(kNumRpcsPerAddress, backends_[i]->service_.request_count());
}
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
}
TEST_F(SingleBalancerTest, SecureNamingDeathTest) {
@@ -806,7 +802,7 @@ TEST_F(SingleBalancerTest, SecureNamingDeathTest) {
"\"fake:///wrong_lb\" } }\n"
" ]\n"
"}");
- SetNextResolutionForLbChannel({balancer_servers_[0].port_});
+ SetNextResolutionForLbChannel({balancers_[0]->port_});
channel_->WaitForConnected(grpc_timeout_seconds_to_deadline(1));
},
"");
@@ -834,11 +830,11 @@ TEST_F(SingleBalancerTest, InitiallyEmptyServerlist) {
// populated serverlist but under the call's deadline (which is enforced by
// the call's deadline).
EXPECT_GT(ellapsed_ms.count(), kServerlistDelayMs);
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent two responses.
- EXPECT_EQ(2U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(2U, balancers_[0]->service_.response_count());
}
TEST_F(SingleBalancerTest, AllServersUnreachableFailFast) {
@@ -854,11 +850,11 @@ TEST_F(SingleBalancerTest, AllServersUnreachableFailFast) {
const Status status = SendRpc();
// The error shouldn't be DEADLINE_EXCEEDED.
EXPECT_EQ(StatusCode::UNAVAILABLE, status.error_code());
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
}
// The fallback tests are deferred because the fallback mode hasn't been
@@ -882,27 +878,18 @@ TEST_F(SingleBalancerTest, BackendsRestart) {
channel_->GetState(true /* try_to_connect */);
// Send kNumRpcsPerAddress RPCs per server.
CheckRpcSendOk(kNumRpcsPerAddress * num_backends_);
- balancers_[0]->NotifyDoneWithServerlists();
+ balancers_[0]->service_.NotifyDoneWithServerlists();
// The balancer got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
- for (size_t i = 0; i < backends_.size(); ++i) {
- if (backends_[i]->Shutdown()) backend_servers_[i].Shutdown();
- }
- CheckRpcSendFailure();
- for (size_t i = 0; i < num_backends_; ++i) {
- backends_.emplace_back(new BackendServiceImpl());
- backend_servers_.emplace_back(ServerThread<BackendService>(
- "backend", server_host_, backends_.back().get()));
- }
- // The following RPC will fail due to the backend ports having changed. It
- // will nonetheless exercise the xds-roundrobin handling of the RR policy
- // having gone into shutdown.
- // TODO(dgq): implement the "backend restart" component as well. We need extra
- // machinery to either update the LB responses "on the fly" or instruct
- // backends which ports to restart on.
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
+ // Stop backends. RPCs should fail.
+ ShutdownAllBackends();
CheckRpcSendFailure();
+ // Restart all backends. RPCs should start succeeding again.
+ StartAllBackends();
+ CheckRpcSendOk(1 /* times */, 1000 /* timeout_ms */,
+ true /* wait_for_ready */);
}
class UpdatesTest : public XdsEnd2endTest {
@@ -929,22 +916,22 @@ TEST_F(UpdatesTest, UpdateBalancersButKeepUsingOriginalBalancer) {
gpr_log(GPR_INFO, "========= DONE WITH FIRST BATCH ==========");
// All 10 requests should have gone to the first backend.
- EXPECT_EQ(10U, backend_servers_[0].service_->request_count());
+ EXPECT_EQ(10U, backends_[0]->service_.request_count());
// Balancer 0 got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.response_count());
gpr_log(GPR_INFO, "========= ABOUT TO UPDATE 1 ==========");
- SetNextResolutionForLbChannel({balancer_servers_[1].port_});
+ SetNextResolutionForLbChannel({balancers_[1]->port_});
gpr_log(GPR_INFO, "========= UPDATE 1 DONE ==========");
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
gpr_timespec deadline = gpr_time_add(
gpr_now(GPR_CLOCK_REALTIME), gpr_time_from_millis(10000, GPR_TIMESPAN));
// Send 10 seconds worth of RPCs
@@ -953,14 +940,14 @@ TEST_F(UpdatesTest, UpdateBalancersButKeepUsingOriginalBalancer) {
} while (gpr_time_cmp(gpr_now(GPR_CLOCK_REALTIME), deadline) < 0);
// The current LB call is still working, so xds continued using it to the
// first balancer, which doesn't assign the second backend.
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
-
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
+
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.response_count());
}
TEST_F(UpdatesTest, UpdateBalancerName) {
@@ -982,19 +969,19 @@ TEST_F(UpdatesTest, UpdateBalancerName) {
gpr_log(GPR_INFO, "========= DONE WITH FIRST BATCH ==========");
// All 10 requests should have gone to the first backend.
- EXPECT_EQ(10U, backend_servers_[0].service_->request_count());
+ EXPECT_EQ(10U, backends_[0]->service_.request_count());
// Balancer 0 got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.response_count());
std::vector<int> ports;
- ports.emplace_back(balancer_servers_[1].port_);
+ ports.emplace_back(balancers_[1]->port_);
auto new_lb_channel_response_generator =
grpc_core::MakeRefCounted<grpc_core::FakeResolverResponseGenerator>();
SetNextResolutionForLbChannel(ports, nullptr,
@@ -1013,22 +1000,22 @@ TEST_F(UpdatesTest, UpdateBalancerName) {
// Wait until update has been processed, as signaled by the second backend
// receiving a request.
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
WaitForBackend(1);
- backend_servers_[1].service_->ResetCounters();
+ backends_[1]->service_.ResetCounters();
gpr_log(GPR_INFO, "========= BEFORE SECOND BATCH ==========");
CheckRpcSendOk(10);
gpr_log(GPR_INFO, "========= DONE WITH SECOND BATCH ==========");
// All 10 requests should have gone to the second backend.
- EXPECT_EQ(10U, backend_servers_[1].service_->request_count());
-
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
- EXPECT_EQ(1U, balancer_servers_[1].service_->request_count());
- EXPECT_EQ(1U, balancer_servers_[1].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+ EXPECT_EQ(10U, backends_[1]->service_.request_count());
+
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
+ EXPECT_EQ(1U, balancers_[1]->service_.request_count());
+ EXPECT_EQ(1U, balancers_[1]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.response_count());
}
// Send an update with the same set of LBs as the one in SetUp() in order to
@@ -1054,26 +1041,26 @@ TEST_F(UpdatesTest, UpdateBalancersRepeated) {
gpr_log(GPR_INFO, "========= DONE WITH FIRST BATCH ==========");
// All 10 requests should have gone to the first backend.
- EXPECT_EQ(10U, backend_servers_[0].service_->request_count());
+ EXPECT_EQ(10U, backends_[0]->service_.request_count());
// Balancer 0 got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.response_count());
std::vector<int> ports;
- ports.emplace_back(balancer_servers_[0].port_);
- ports.emplace_back(balancer_servers_[1].port_);
- ports.emplace_back(balancer_servers_[2].port_);
+ ports.emplace_back(balancers_[0]->port_);
+ ports.emplace_back(balancers_[1]->port_);
+ ports.emplace_back(balancers_[2]->port_);
gpr_log(GPR_INFO, "========= ABOUT TO UPDATE 1 ==========");
SetNextResolutionForLbChannel(ports);
gpr_log(GPR_INFO, "========= UPDATE 1 DONE ==========");
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
gpr_timespec deadline = gpr_time_add(
gpr_now(GPR_CLOCK_REALTIME), gpr_time_from_millis(10000, GPR_TIMESPAN));
// Send 10 seconds worth of RPCs
@@ -1082,16 +1069,16 @@ TEST_F(UpdatesTest, UpdateBalancersRepeated) {
} while (gpr_time_cmp(gpr_now(GPR_CLOCK_REALTIME), deadline) < 0);
// xds continued using the original LB call to the first balancer, which
// doesn't assign the second backend.
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
ports.clear();
- ports.emplace_back(balancer_servers_[0].port_);
- ports.emplace_back(balancer_servers_[1].port_);
+ ports.emplace_back(balancers_[0]->port_);
+ ports.emplace_back(balancers_[1]->port_);
gpr_log(GPR_INFO, "========= ABOUT TO UPDATE 2 ==========");
SetNextResolutionForLbChannel(ports);
gpr_log(GPR_INFO, "========= UPDATE 2 DONE ==========");
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
deadline = gpr_time_add(gpr_now(GPR_CLOCK_REALTIME),
gpr_time_from_millis(10000, GPR_TIMESPAN));
// Send 10 seconds worth of RPCs
@@ -1100,12 +1087,12 @@ TEST_F(UpdatesTest, UpdateBalancersRepeated) {
} while (gpr_time_cmp(gpr_now(GPR_CLOCK_REALTIME), deadline) < 0);
// xds continued using the original LB call to the first balancer, which
// doesn't assign the second backend.
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
}
TEST_F(UpdatesTest, UpdateBalancersDeadUpdate) {
SetNextResolution({}, kDefaultServiceConfig_.c_str());
- SetNextResolutionForLbChannel({balancer_servers_[0].port_});
+ SetNextResolutionForLbChannel({balancers_[0]->port_});
const std::vector<int> first_backend{GetBackendPorts()[0]};
const std::vector<int> second_backend{GetBackendPorts()[1]};
@@ -1119,11 +1106,11 @@ TEST_F(UpdatesTest, UpdateBalancersDeadUpdate) {
CheckRpcSendOk(10);
gpr_log(GPR_INFO, "========= DONE WITH FIRST BATCH ==========");
// All 10 requests should have gone to the first backend.
- EXPECT_EQ(10U, backend_servers_[0].service_->request_count());
+ EXPECT_EQ(10U, backends_[0]->service_.request_count());
// Kill balancer 0
gpr_log(GPR_INFO, "********** ABOUT TO KILL BALANCER 0 *************");
- if (balancers_[0]->Shutdown()) balancer_servers_[0].Shutdown();
+ balancers_[0]->Shutdown();
gpr_log(GPR_INFO, "********** KILLED BALANCER 0 *************");
// This is serviced by the existing child policy.
@@ -1131,48 +1118,48 @@ TEST_F(UpdatesTest, UpdateBalancersDeadUpdate) {
CheckRpcSendOk(10);
gpr_log(GPR_INFO, "========= DONE WITH SECOND BATCH ==========");
// All 10 requests should again have gone to the first backend.
- EXPECT_EQ(20U, backend_servers_[0].service_->request_count());
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(20U, backends_[0]->service_.request_count());
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
// Balancer 0 got a single request.
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
// and sent a single response.
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[1].service_->response_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[1]->service_.response_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.response_count());
gpr_log(GPR_INFO, "========= ABOUT TO UPDATE 1 ==========");
- SetNextResolutionForLbChannel({balancer_servers_[1].port_});
+ SetNextResolutionForLbChannel({balancers_[1]->port_});
gpr_log(GPR_INFO, "========= UPDATE 1 DONE ==========");
// Wait until update has been processed, as signaled by the second backend
// receiving a request. In the meantime, the client continues to be serviced
// (by the first backend) without interruption.
- EXPECT_EQ(0U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(0U, backends_[1]->service_.request_count());
WaitForBackend(1);
// This is serviced by the updated RR policy
- backend_servers_[1].service_->ResetCounters();
+ backends_[1]->service_.ResetCounters();
gpr_log(GPR_INFO, "========= BEFORE THIRD BATCH ==========");
CheckRpcSendOk(10);
gpr_log(GPR_INFO, "========= DONE WITH THIRD BATCH ==========");
// All 10 requests should have gone to the second backend.
- EXPECT_EQ(10U, backend_servers_[1].service_->request_count());
+ EXPECT_EQ(10U, backends_[1]->service_.request_count());
- EXPECT_EQ(1U, balancer_servers_[0].service_->request_count());
- EXPECT_EQ(1U, balancer_servers_[0].service_->response_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.request_count());
+ EXPECT_EQ(1U, balancers_[0]->service_.response_count());
// The second balancer, published as part of the first update, may end up
// getting two requests (that is, 1 <= #req <= 2) if the LB call retry timer
// firing races with the arrival of the update containing the second
// balancer.
- EXPECT_GE(balancer_servers_[1].service_->request_count(), 1U);
- EXPECT_GE(balancer_servers_[1].service_->response_count(), 1U);
- EXPECT_LE(balancer_servers_[1].service_->request_count(), 2U);
- EXPECT_LE(balancer_servers_[1].service_->response_count(), 2U);
- EXPECT_EQ(0U, balancer_servers_[2].service_->request_count());
- EXPECT_EQ(0U, balancer_servers_[2].service_->response_count());
+ EXPECT_GE(balancers_[1]->service_.request_count(), 1U);
+ EXPECT_GE(balancers_[1]->service_.response_count(), 1U);
+ EXPECT_LE(balancers_[1]->service_.request_count(), 2U);
+ EXPECT_LE(balancers_[1]->service_.response_count(), 2U);
+ EXPECT_EQ(0U, balancers_[2]->service_.request_count());
+ EXPECT_EQ(0U, balancers_[2]->service_.response_count());
}
// The re-resolution tests are deferred because they rely on the fallback mode,
@@ -1201,6 +1188,9 @@ class SingleBalancerWithClientLoadReportingTest : public XdsEnd2endTest {
// TODO(vpowar): Add TEST_F(SingleBalancerWithClientLoadReportingTest, Vanilla)
+// TODO(roth): Add TEST_F(SingleBalancerWithClientLoadReportingTest,
+// BalancerRestart)
+
// TODO(roth): Add TEST_F(SingleBalancerWithClientLoadReportingTest, Drop)
} // namespace
From 84cb531b72cd7f355a493b4eaa460afb7defea0f Mon Sep 17 00:00:00 2001
From: Moses Koledoye <moseskoledoye@gmail.com>
Date: Fri, 15 Mar 2019 22:23:52 +0000
Subject: [PATCH 134/138] Fix typo
---
examples/cpp/helloworld/README.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/examples/cpp/helloworld/README.md b/examples/cpp/helloworld/README.md
index c598658b1d7..813a80f288f 100644
--- a/examples/cpp/helloworld/README.md
+++ b/examples/cpp/helloworld/README.md
@@ -98,7 +98,7 @@ $ protoc -I ../../protos/ --cpp_out=. ../../protos/helloworld.proto
```
- Create a stub. A stub implements the rpc methods of a service and in the
- generated code, a method is provided to created a stub with a channel:
+ generated code, a method is provided to create a stub with a channel:
```cpp
auto stub = helloworld::Greeter::NewStub(channel);
From 48ce4ca939181b46e5893618711e391fe94828b9 Mon Sep 17 00:00:00 2001
From: Vijay Pai <vpai@google.com>
Date: Thu, 14 Feb 2019 16:52:15 -0800
Subject: [PATCH 135/138] Add support for extra-reaction operations via Holds
---
include/grpcpp/impl/codegen/client_callback.h | 47 ++++++++++++
.../end2end/client_callback_end2end_test.cc | 76 +++++++++++++++++++
2 files changed, 123 insertions(+)
diff --git a/include/grpcpp/impl/codegen/client_callback.h b/include/grpcpp/impl/codegen/client_callback.h
index 0b2631014a2..89629c079af 100644
--- a/include/grpcpp/impl/codegen/client_callback.h
+++ b/include/grpcpp/impl/codegen/client_callback.h
@@ -112,6 +112,8 @@ class ClientCallbackReaderWriter {
virtual void Write(const Request* req, WriteOptions options) = 0;
virtual void WritesDone() = 0;
virtual void Read(Response* resp) = 0;
+ virtual void AddHold(int holds) = 0;
+ virtual void RemoveHold() = 0;
protected:
void BindReactor(ClientBidiReactor<Request, Response>* reactor) {
@@ -125,6 +127,8 @@ class ClientCallbackReader {
virtual ~ClientCallbackReader() {}
virtual void StartCall() = 0;
virtual void Read(Response* resp) = 0;
+ virtual void AddHold(int holds) = 0;
+ virtual void RemoveHold() = 0;
protected:
void BindReactor(ClientReadReactor<Response>* reactor) {
@@ -144,6 +148,9 @@ class ClientCallbackWriter {
}
virtual void WritesDone() = 0;
+ virtual void AddHold(int holds) = 0;
+ virtual void RemoveHold() = 0;
+
protected:
void BindReactor(ClientWriteReactor<Request>* reactor) {
reactor->BindWriter(this);
@@ -174,6 +181,29 @@ class ClientBidiReactor {
}
void StartWritesDone() { stream_->WritesDone(); }
+ /// Holds are needed if (and only if) this stream has operations that take
+ /// place on it after StartCall but from outside one of the reactions
+ /// (OnReadDone, etc). This is _not_ a common use of the streaming API.
+ ///
+ /// Holds must be added before calling StartCall. If a stream still has a hold
+ /// in place, its resources will not be destroyed even if the status has
+ /// already come in from the wire and there are currently no active callbacks
+ /// outstanding. Similarly, the stream will not call OnDone if there are still
+ /// holds on it.
+ ///
+ /// For example, if a StartRead or StartWrite operation is going to be
+ /// initiated from elsewhere in the application, the application should call
+ /// AddHold or AddMultipleHolds before StartCall. If there is going to be,
+ /// for example, a read-flow and a write-flow taking place outside the
+ /// reactions, then call AddMultipleHolds(2) before StartCall. When the
+ /// application knows that it won't issue any more Read operations (such as
+ /// when a read comes back as not ok), it should issue a RemoveHold(). It
+ /// should also call RemoveHold() again after it does StartWriteLast or
+ /// StartWritesDone that indicates that there will be no more Write ops.
+ void AddHold() { AddMultipleHolds(1); }
+ void AddMultipleHolds(int holds) { stream_->AddHold(holds); }
+ void RemoveHold() { stream_->RemoveHold(); }
+
private:
friend class ClientCallbackReaderWriter<Request, Response>;
void BindStream(ClientCallbackReaderWriter<Request, Response>* stream) {
@@ -193,6 +223,10 @@ class ClientReadReactor {
void StartCall() { reader_->StartCall(); }
void StartRead(Response* resp) { reader_->Read(resp); }
+ void AddHold() { AddMultipleHolds(1); }
+ void AddMultipleHolds(int holds) { reader_->AddHold(holds); }
+ void RemoveHold() { reader_->RemoveHold(); }
+
private:
friend class ClientCallbackReader<Response>;
void BindReader(ClientCallbackReader<Response>* reader) { reader_ = reader; }
@@ -218,6 +252,10 @@ class ClientWriteReactor {
}
void StartWritesDone() { writer_->WritesDone(); }
+ void AddHold() { AddMultipleHolds(1); }
+ void AddMultipleHolds(int holds) { writer_->AddHold(holds); }
+ void RemoveHold() { writer_->RemoveHold(); }
+
private:
friend class ClientCallbackWriter<Request>;
void BindWriter(ClientCallbackWriter<Request>* writer) { writer_ = writer; }
@@ -374,6 +412,9 @@ class ClientCallbackReaderWriterImpl
}
}
+ virtual void AddHold(int holds) override { callbacks_outstanding_ += holds; }
+ virtual void RemoveHold() override { MaybeFinish(); }
+
private:
friend class ClientCallbackReaderWriterFactory<Request, Response>;
@@ -509,6 +550,9 @@ class ClientCallbackReaderImpl
}
}
+ virtual void AddHold(int holds) override { callbacks_outstanding_ += holds; }
+ virtual void RemoveHold() override { MaybeFinish(); }
+
private:
friend class ClientCallbackReaderFactory<Response>;
@@ -677,6 +721,9 @@ class ClientCallbackWriterImpl
}
}
+ virtual void AddHold(int holds) override { callbacks_outstanding_ += holds; }
+ virtual void RemoveHold() override { MaybeFinish(); }
+
private:
friend class ClientCallbackWriterFactory<Request>;
diff --git a/test/cpp/end2end/client_callback_end2end_test.cc b/test/cpp/end2end/client_callback_end2end_test.cc
index 3845c4c0b2a..e1e898275b3 100644
--- a/test/cpp/end2end/client_callback_end2end_test.cc
+++ b/test/cpp/end2end/client_callback_end2end_test.cc
@@ -1117,6 +1117,82 @@ TEST_P(ClientCallbackEnd2endTest, UnimplementedRpc) {
}
}
+TEST_P(ClientCallbackEnd2endTest,
+ ResponseStreamExtraReactionFlowReadsUntilDone) {
+ MAYBE_SKIP_TEST;
+ ResetStub();
+ class ReadAllIncomingDataClient
+ : public grpc::experimental::ClientReadReactor<EchoResponse> {
+ public:
+ ReadAllIncomingDataClient(grpc::testing::EchoTestService::Stub* stub) {
+ request_.set_message("Hello client ");
+ stub->experimental_async()->ResponseStream(&context_, &request_, this);
+ }
+ bool WaitForReadDone() {
+ std::unique_lock<std::mutex> l(mu_);
+ while (!read_done_) {
+ read_cv_.wait(l);
+ }
+ read_done_ = false;
+ return read_ok_;
+ }
+ void Await() {
+ std::unique_lock<std::mutex> l(mu_);
+ while (!done_) {
+ done_cv_.wait(l);
+ }
+ }
+ const Status& status() {
+ std::unique_lock<std::mutex> l(mu_);
+ return status_;
+ }
+
+ private:
+ void OnReadDone(bool ok) override {
+ std::unique_lock<std::mutex> l(mu_);
+ read_ok_ = ok;
+ read_done_ = true;
+ read_cv_.notify_one();
+ }
+ void OnDone(const Status& s) override {
+ std::unique_lock<std::mutex> l(mu_);
+ done_ = true;
+ status_ = s;
+ done_cv_.notify_one();
+ }
+
+ EchoRequest request_;
+ EchoResponse response_;
+ ClientContext context_;
+ bool read_ok_ = false;
+ bool read_done_ = false;
+ std::mutex mu_;
+ std::condition_variable read_cv_;
+ std::condition_variable done_cv_;
+ bool done_ = false;
+ Status status_;
+ } client{stub_.get()};
+
+ int reads_complete = 0;
+ client.AddHold();
+ client.StartCall();
+
+ EchoResponse response;
+ bool read_ok = true;
+ while (read_ok) {
+ client.StartRead(&response);
+ read_ok = client.WaitForReadDone();
+ if (read_ok) {
+ ++reads_complete;
+ }
+ }
+ client.RemoveHold();
+ client.Await();
+
+ EXPECT_EQ(kServerDefaultResponseStreamsToSend, reads_complete);
+ EXPECT_EQ(client.status().error_code(), grpc::StatusCode::OK);
+}
+
std::vector<TestScenario> CreateTestScenarios(bool test_insecure) {
std::vector<TestScenario> scenarios;
std::vector<grpc::string> credentials_types{
From 4c3b577650590afcdcfd57e72b9b5faa67c6f7a8 Mon Sep 17 00:00:00 2001
From: Lidi Zheng <lidiz@google.com>
Date: Fri, 15 Mar 2019 17:42:50 -0700
Subject: [PATCH 136/138] Add expectation to negative timeout test case
---
src/python/grpcio_tests/tests/unit/_cython/_channel_test.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/python/grpcio_tests/tests/unit/_cython/_channel_test.py b/src/python/grpcio_tests/tests/unit/_cython/_channel_test.py
index 9fd9ede144d..54f620523ea 100644
--- a/src/python/grpcio_tests/tests/unit/_cython/_channel_test.py
+++ b/src/python/grpcio_tests/tests/unit/_cython/_channel_test.py
@@ -62,6 +62,8 @@ class ChannelTest(unittest.TestCase):
connectivity = channel.check_connectivity_state(True)
channel.watch_connectivity_state(connectivity, -3.14)
channel.close(cygrpc.StatusCode.ok, 'Channel close!')
+ # NOTE(lidiz) The negative timeout should not trigger SIGABRT.
+ # Bug report: https://github.com/grpc/grpc/issues/18244
if __name__ == '__main__':
From 472613c3bc71dafb1afd489bc10bea81837dd3c8 Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Mon, 18 Mar 2019 14:20:31 +0100
Subject: [PATCH 137/138] check dotnet SDK version before building ASP.NET core
interop image
---
.../grpc_interop_aspnetcore/build_interop.sh.template | 1 +
.../interoptest/grpc_interop_aspnetcore/build_interop.sh | 1 +
2 files changed, 2 insertions(+)
diff --git a/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh.template b/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh.template
index 69e2ed387b2..449383c0d6f 100644
--- a/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh.template
+++ b/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh.template
@@ -25,6 +25,7 @@
cp -r /var/local/jenkins/service_account $HOME || true
cd /var/local/git/grpc-dotnet
+ ./build/get-dotnet.sh
./build/get-grpc.sh
cd testassets/InteropTestsWebsite
diff --git a/tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh b/tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh
index 38feae39623..ed82440ba3c 100644
--- a/tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh
+++ b/tools/dockerfile/interoptest/grpc_interop_aspnetcore/build_interop.sh
@@ -23,6 +23,7 @@ git clone /var/local/jenkins/grpc-dotnet /var/local/git/grpc-dotnet
cp -r /var/local/jenkins/service_account $HOME || true
cd /var/local/git/grpc-dotnet
+./build/get-dotnet.sh
./build/get-grpc.sh
cd testassets/InteropTestsWebsite
From 0f8a3aeeb7e171db1eeb4023f6e918aa6caf9d6f Mon Sep 17 00:00:00 2001
From: Jan Tattermusch <jtattermusch@google.com>
Date: Mon, 18 Mar 2019 14:30:23 +0100
Subject: [PATCH 138/138] install jq
---
.../interoptest/grpc_interop_aspnetcore/Dockerfile.template | 3 +++
.../dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile | 3 +++
2 files changed, 6 insertions(+)
diff --git a/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template b/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template
index e8f962403f7..4f6c52b19af 100644
--- a/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template
+++ b/templates/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile.template
@@ -16,5 +16,8 @@
FROM mcr.microsoft.com/dotnet/core/sdk:3.0.100-preview3-stretch
+ # needed by get-dotnet.sh script
+ RUN apt-get update && apt-get install -y jq && apt-get clean
+
# Define the default command.
CMD ["bash"]
diff --git a/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile b/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile
index 26a21384911..9ad6c1f28ea 100644
--- a/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile
+++ b/tools/dockerfile/interoptest/grpc_interop_aspnetcore/Dockerfile
@@ -14,5 +14,8 @@
FROM mcr.microsoft.com/dotnet/core/sdk:3.0.100-preview3-stretch
+# needed by get-dotnet.sh script
+RUN apt-get update && apt-get install -y jq && apt-get clean
+
# Define the default command.
CMD ["bash"]