mirror of https://github.com/grpc/grpc.git
Kerrick Staley
7 years ago
commit
f44f0bc592
70 changed files with 1263 additions and 184 deletions
@ -0,0 +1,28 @@ |
||||
/*
|
||||
* |
||||
* Copyright 2018 gRPC authors. |
||||
* |
||||
* Licensed under the Apache License, Version 2.0 (the "License"); |
||||
* you may not use this file except in compliance with the License. |
||||
* You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
* |
||||
*/ |
||||
|
||||
#ifndef GRPC_CORE_LIB_GPR_ALLOC_H |
||||
#define GRPC_CORE_LIB_GPR_ALLOC_H |
||||
|
||||
#include <grpc/support/port_platform.h> |
||||
|
||||
/// Given a size, round up to the next multiple of sizeof(void*).
|
||||
#define GPR_ROUND_UP_TO_ALIGNMENT_SIZE(x) \ |
||||
(((x) + GPR_MAX_ALIGNMENT - 1u) & ~(GPR_MAX_ALIGNMENT - 1u)) |
||||
|
||||
#endif /* GRPC_CORE_LIB_GPR_ALLOC_H */ |
||||
@ -0,0 +1,92 @@ |
||||
|
||||
# gRPC iOS Network Transition Behaviors |
||||
Network connectivity on an iOS device may transition between cellular, WIFI, or |
||||
no network connectivity. This document describes how these network changes |
||||
should be handled by gRPC and current known issues. |
||||
|
||||
## Expected Network Transition Behaviors |
||||
The expected gRPC iOS channel and network transition behaviors are: |
||||
* Channel connection to a particular host is established at the time of |
||||
starting the first call to the channel and remains connected for future calls |
||||
to the same host. |
||||
* If the underlying connection to the remote host is broken, the channel is |
||||
disconnected and enters TRANSIENT\_FAILURE state. |
||||
* A channel is broken if the channel connection is no longer viable. This |
||||
happens when |
||||
* The network interface is no longer available, e.g. WiFi or cellular |
||||
interface is turned off or goes offline, airplane mode turned on, etc; |
||||
* The underlying TCP connection is no longer valid, e.g. WiFi connects to |
||||
another hotspot, cellular data switched from LTE to 4G, etc; |
||||
* A network interface more preferable by the OS is valid, e.g. WiFi gets |
||||
connected when the channel is already connected via cellular. |
||||
* A channel in TRANSIENT\_FAILURE state attempts reconnection on start of the |
||||
next call to the same host, but only after a certain backoff period (see |
||||
corresponding |
||||
[doc](https://github.com/grpc/grpc/blob/master/doc/connection-backoff.md)). |
||||
During the backoff period, any call to the same host will wait until the |
||||
first of the following events occur: |
||||
* Connection succeeded; calls will be made using this channel; |
||||
* Conncetion failed; calls will be failed and return UNAVAILABLE status code; |
||||
* The call's deadline is reached; the call will fail and return |
||||
DEADLINE\_EXCEEDED status code. |
||||
The length of backoff period of a channel is reset whenever a connection |
||||
attempt is successful. |
||||
|
||||
## Implementations |
||||
### gRPC iOS with TCP Sockets |
||||
gRPC's default implementation is to use TCP sockets for networking. It turns |
||||
out that although Apple supports this type of usage, it is [not recommended by |
||||
Apple](https://developer.apple.com/library/archive/documentation/NetworkingInternetWeb/Conceptual/NetworkingOverview/SocketsAndStreams/SocketsAndStreams.html) |
||||
and has some issues described below. |
||||
|
||||
#### Issues with TCP Sockets |
||||
The TCP sockets on iOS is flawed in that it does not reflect the viability of |
||||
the channel connection. Particularly, we observed the following issues when |
||||
using TCP sockets: |
||||
* When a TCP socket connection is established on cellular data and WiFi |
||||
becomes available, the TCP socket neither return an error event nor continue |
||||
sending/receiving data on it, but still accepts write on it. |
||||
* A TCP socket does not report certain events that happen in the |
||||
background. When a TCP connection breaks in the background for the reason |
||||
like WiFi connects to another hotspot, the socket neither return an error nor |
||||
continue sending/receiving data on it, but still accepts write on it. |
||||
In both situations, the user will see the call hang for an extended period of |
||||
time before the TCP socket times out. |
||||
|
||||
#### gRPC iOS library's resolution to TCP socket issues |
||||
We introduced |
||||
[`ConnectivityMonitor`](https://developer.apple.com/library/archive/documentation/NetworkingInternetWeb/Conceptual/NetworkingOverview/SocketsAndStreams/SocketsAndStreams.html) |
||||
in gRPC iOS library v0.14.0 to alleviate these issues in TCP sockets, |
||||
which changes the network transition behaviors a bit. |
||||
|
||||
We classify network connectivity state of the device into three categories |
||||
based on flags obtained from `SCNetworkReachability` API: |
||||
|
||||
| Reachable | ConnectionRequired | IsWWAN | **Category** | |
||||
|:---------:|:------------------:|:------:|:------------:| |
||||
| 0 | X | X | None | |
||||
| X | 1 | X | None | |
||||
| 1 | 0 | 0 | WiFi | |
||||
| 1 | 0 | 1 | Cellular | |
||||
|
||||
Whenever there is a transition of network between two of these categories, all |
||||
previously existing channels are assumed to be broken and are actively |
||||
destroyed. If there is an unfinished call, the call should return with status |
||||
code `UNAVAILABLE`. |
||||
|
||||
`ConnectivityMonitor` is able to detect the scenario of the first issue above |
||||
and actively destroy the channels. However, the second issue is not resolvable. |
||||
To solve that issue the best solution is to switch to CFStream implementation |
||||
which eliminates all of them. |
||||
|
||||
### gRPC iOS with CFStream |
||||
gRPC iOS with CFStream implementation (introduced in v1.13.0) uses Apple's |
||||
networking API to make connections. It resolves the issues with TCP sockets |
||||
mentioned above. Users are recommended to use this implementation rather than |
||||
TCP socket implementation. The detailed behavior of streams in CFStream is not |
||||
documented by Apple, but our experiments show that it accords to the expected |
||||
behaviors. With CFStream implementation, an event is always received when the |
||||
underlying connection is no longer viable. For more detailed information and |
||||
usages of CFStream implementation, refer to the |
||||
[user guide](https://github.com/grpc/grpc/blob/master/src/objective-c/README-CFSTREAM.md). |
||||
|
||||
@ -0,0 +1,32 @@ |
||||
[](https://cocoapods.org/pods/gRPC) |
||||
# gRPC Objective-C with CFStream |
||||
|
||||
gRPC Objective-C library now provides the option to use Apple's CFStream API (rather than TCP |
||||
sockets) for networking. Using CFStream resolves a bunch of network connectivity transition issues |
||||
(see the [doc](https://github.com/grpc/grpc/blob/master/src/objective-c/NetworkTransitionBehavior.md) |
||||
for more information). |
||||
|
||||
CFStream integration is now in experimental state. You will need explicit opt-in to use it to get |
||||
the benefits of resolving the issues above. We expect to make CFStream the default networking |
||||
interface that gRPC uses when it is ready for production. |
||||
|
||||
## Usage |
||||
If you use gRPC following the instructions in |
||||
[README.md](https://github.com/grpc/grpc/blob/master/src/objective-c/README.md): |
||||
- Simply replace the |
||||
dependency on `gRPC-ProtoRPC` with `gRPC-ProtoRPC/CFStream`. The build system will take care of |
||||
everything else and switch networking to CFStream. |
||||
|
||||
If your project directly depends on podspecs other than `gRPC-ProtoRPC` (e.g. `gRPC` or |
||||
`gRPC-Core`): |
||||
|
||||
- Make your projects depend on subspecs corresponding to CFStream in each gRPC podspec. For |
||||
`gRPC-Core`, you will need to make sure that the completion queue you create is of type |
||||
`GRPC_CQ_NON_POLLING`. This is expected to be fixed soon so that you do not have to modify the |
||||
completion queue type. |
||||
|
||||
## Notes |
||||
|
||||
- Currently we do not support platforms other than iOS, although it is likely that this integration |
||||
can run on MacOS targets with Apple's compiler. |
||||
- Let us know if you meet any issue by filing issue and ping @muxi. |
||||
@ -0,0 +1,19 @@ |
||||
/*
|
||||
* |
||||
* Copyright 2015 gRPC authors. |
||||
* |
||||
* Licensed under the Apache License, Version 2.0 (the "License"); |
||||
* you may not use this file except in compliance with the License. |
||||
* You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
* |
||||
*/ |
||||
|
||||
int main(void) {} |
||||
@ -0,0 +1,19 @@ |
||||
/*
|
||||
* |
||||
* Copyright 2015 gRPC authors. |
||||
* |
||||
* Licensed under the Apache License, Version 2.0 (the "License"); |
||||
* you may not use this file except in compliance with the License. |
||||
* You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
* |
||||
*/ |
||||
|
||||
int main(void) {} |
||||
@ -0,0 +1,19 @@ |
||||
/*
|
||||
* |
||||
* Copyright 2015 gRPC authors. |
||||
* |
||||
* Licensed under the Apache License, Version 2.0 (the "License"); |
||||
* you may not use this file except in compliance with the License. |
||||
* You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
* |
||||
*/ |
||||
|
||||
int main(void) {} |
||||
@ -0,0 +1,115 @@ |
||||
/*
|
||||
* |
||||
* Copyright 2018 gRPC authors. |
||||
* |
||||
* Licensed under the Apache License, Version 2.0 (the "License"); |
||||
* you may not use this file except in compliance with the License. |
||||
* You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
* |
||||
*/ |
||||
#include "src/core/lib/iomgr/port.h" |
||||
|
||||
/* This test only relevant on linux systems where epoll() is available */ |
||||
#if defined(GRPC_LINUX_EPOLL_CREATE1) && defined(GRPC_LINUX_EVENTFD) |
||||
#include "src/core/lib/iomgr/ev_epollex_linux.h" |
||||
|
||||
#include <grpc/grpc.h> |
||||
#include <string.h> |
||||
#include <sys/eventfd.h> |
||||
|
||||
#include "test/core/util/test_config.h" |
||||
|
||||
static void pollset_destroy(void* ps, grpc_error* error) { |
||||
grpc_pollset_destroy(static_cast<grpc_pollset*>(ps)); |
||||
gpr_free(ps); |
||||
} |
||||
|
||||
// This test is added to cover the case found in bug:
|
||||
// https://github.com/grpc/grpc/issues/15760
|
||||
static void test_pollable_owner_fd() { |
||||
grpc_core::ExecCtx exec_ctx; |
||||
int ev_fd1; |
||||
int ev_fd2; |
||||
grpc_fd* grpc_fd1; |
||||
grpc_fd* grpc_fd2; |
||||
grpc_pollset* ps; |
||||
gpr_mu* mu; |
||||
|
||||
// == Create two grpc_fds ==
|
||||
// All we need is two file descriptors. Doesn't matter what type. We use
|
||||
// eventfd type here for the purpose of this test
|
||||
ev_fd1 = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); |
||||
ev_fd2 = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); |
||||
if (ev_fd1 < 0 || ev_fd2 < 0) { |
||||
gpr_log(GPR_ERROR, "Error in creating event fds for the test"); |
||||
return; |
||||
} |
||||
grpc_fd1 = grpc_fd_create(ev_fd1, "epollex-test-fd1", false); |
||||
grpc_fd2 = grpc_fd_create(ev_fd2, "epollex-test-fd2", false); |
||||
grpc_core::ExecCtx::Get()->Flush(); |
||||
|
||||
// == Create a pollset ==
|
||||
ps = static_cast<grpc_pollset*>(gpr_zalloc(grpc_pollset_size())); |
||||
grpc_pollset_init(ps, &mu); |
||||
grpc_core::ExecCtx::Get()->Flush(); |
||||
|
||||
// == Add fd1 to pollset ==
|
||||
grpc_pollset_add_fd(ps, grpc_fd1); |
||||
grpc_core::ExecCtx::Get()->Flush(); |
||||
|
||||
// == Destroy fd1 ==
|
||||
grpc_fd_orphan(grpc_fd1, nullptr, nullptr, "test fd1 orphan"); |
||||
grpc_core::ExecCtx::Get()->Flush(); |
||||
|
||||
// = Add fd2 to pollset ==
|
||||
//
|
||||
// Before https://github.com/grpc/grpc/issues/15760, the following line caused
|
||||
// unexpected behavior (The previous grpc_pollset_add_fd(ps, grpc_fd1) created
|
||||
// an underlying structure in epollex that held a reference to grpc_fd1 which
|
||||
// was being accessed here even after grpc_fd_orphan(grpc_fd1) was called
|
||||
grpc_pollset_add_fd(ps, grpc_fd2); |
||||
grpc_core::ExecCtx::Get()->Flush(); |
||||
|
||||
// == Destroy fd2 ==
|
||||
grpc_fd_orphan(grpc_fd2, nullptr, nullptr, "test fd2 orphan"); |
||||
grpc_core::ExecCtx::Get()->Flush(); |
||||
|
||||
// == Destroy pollset
|
||||
grpc_closure ps_destroy_closure; |
||||
GRPC_CLOSURE_INIT(&ps_destroy_closure, pollset_destroy, ps, |
||||
grpc_schedule_on_exec_ctx); |
||||
grpc_pollset_shutdown(ps, &ps_destroy_closure); |
||||
grpc_core::ExecCtx::Get()->Flush(); |
||||
} |
||||
|
||||
int main(int argc, char** argv) { |
||||
const char* poll_strategy = nullptr; |
||||
grpc_test_init(argc, argv); |
||||
grpc_init(); |
||||
{ |
||||
grpc_core::ExecCtx exec_ctx; |
||||
poll_strategy = grpc_get_poll_strategy_name(); |
||||
if (poll_strategy != nullptr && strcmp(poll_strategy, "epollex") == 0) { |
||||
test_pollable_owner_fd(); |
||||
} else { |
||||
gpr_log(GPR_INFO, |
||||
"Skipping the test. The test is only relevant for 'epollex' " |
||||
"strategy. and the current strategy is: '%s'", |
||||
poll_strategy); |
||||
} |
||||
} |
||||
|
||||
grpc_shutdown(); |
||||
return 0; |
||||
} |
||||
#else /* defined(GRPC_LINUX_EPOLL_CREATE1) && defined(GRPC_LINUX_EVENTFD) */ |
||||
int main(int argc, char** argv) { return 0; } |
||||
#endif |
||||
@ -0,0 +1 @@ |
||||
Subproject commit 6599cac0965be8e5a835ab7a5684bbef033d5ad0 |
||||
@ -0,0 +1 @@ |
||||
Subproject commit 9245d481eb3e890f708ff2d7dadf2a10c04748ba |
||||
@ -0,0 +1,19 @@ |
||||
#!/usr/bin/env bash |
||||
# Copyright 2017 gRPC authors. |
||||
# |
||||
# Licensed under the Apache License, Version 2.0 (the "License"); |
||||
# you may not use this file except in compliance with the License. |
||||
# You may obtain a copy of the License at |
||||
# |
||||
# http://www.apache.org/licenses/LICENSE-2.0 |
||||
# |
||||
# Unless required by applicable law or agreed to in writing, software |
||||
# distributed under the License is distributed on an "AS IS" BASIS, |
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
# See the License for the specific language governing permissions and |
||||
# limitations under the License. |
||||
|
||||
export UPLOAD_TEST_RESULTS=true |
||||
EXTRA_FLAGS="--copt=-gmlt --strip=never --copt=-fsanitize=address --linkopt=-fsanitize=address --test_timeout=3600" |
||||
github/grpc/tools/internal_ci/linux/grpc_bazel_on_foundry_base.sh "${EXTRA_FLAGS}" |
||||
|
||||
@ -0,0 +1,68 @@ |
||||
#!/usr/bin/env bash |
||||
# Copyright 2017 gRPC authors. |
||||
# |
||||
# Licensed under the Apache License, Version 2.0 (the "License"); |
||||
# you may not use this file except in compliance with the License. |
||||
# You may obtain a copy of the License at |
||||
# |
||||
# http://www.apache.org/licenses/LICENSE-2.0 |
||||
# |
||||
# Unless required by applicable law or agreed to in writing, software |
||||
# distributed under the License is distributed on an "AS IS" BASIS, |
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
# See the License for the specific language governing permissions and |
||||
# limitations under the License. |
||||
|
||||
set -ex |
||||
|
||||
# A temporary solution to give Kokoro credentials. |
||||
# The file name 4321_grpc-testing-service needs to match auth_credential in |
||||
# the build config. |
||||
# TODO: Use keystore. |
||||
mkdir -p ${KOKORO_KEYSTORE_DIR} |
||||
cp ${KOKORO_GFILE_DIR}/GrpcTesting-d0eeee2db331.json ${KOKORO_KEYSTORE_DIR}/4321_grpc-testing-service |
||||
|
||||
temp_dir=$(mktemp -d) |
||||
ln -f "${KOKORO_GFILE_DIR}/bazel-rc-0.14.0rc5" ${temp_dir}/bazel |
||||
chmod 755 "${KOKORO_GFILE_DIR}/bazel-rc-0.14.0rc5" |
||||
export PATH="${temp_dir}:${PATH}" |
||||
# This should show ${temp_dir}/bazel |
||||
which bazel |
||||
chmod +x "${KOKORO_GFILE_DIR}/bazel_wrapper.py" |
||||
|
||||
# change to grpc repo root |
||||
cd $(dirname $0)/../../../.. |
||||
|
||||
source tools/internal_ci/helper_scripts/prepare_build_linux_rc |
||||
|
||||
"${KOKORO_GFILE_DIR}/bazel_wrapper.py" \ |
||||
--host_jvm_args=-Dbazel.DigestFunction=SHA256 \ |
||||
test --jobs="200" \ |
||||
--test_timeout="3600,3600,3600,3600" \ |
||||
--test_output=errors \ |
||||
--verbose_failures=true \ |
||||
--keep_going \ |
||||
--remote_accept_cached=true \ |
||||
--spawn_strategy=remote \ |
||||
--remote_local_fallback=false \ |
||||
--remote_timeout=3600 \ |
||||
--strategy=Javac=remote \ |
||||
--strategy=Closure=remote \ |
||||
--genrule_strategy=remote \ |
||||
--experimental_strict_action_env=true \ |
||||
--experimental_remote_platform_override='properties:{name:"container-image" value:"docker://gcr.io/cloud-marketplace/google/rbe-ubuntu16-04@sha256:59bf0e191a6b5cc1ab62c2224c810681d1326bad5a27b1d36c9f40113e79da7f" }' \ |
||||
--define GRPC_PORT_ISOLATED_RUNTIME=1 \ |
||||
--copt=-gmlt \ |
||||
--strip=never \ |
||||
--copt=-fsanitize=undefined \ |
||||
--linkopt=-fsanitize=undefined \ |
||||
--crosstool_top=@com_github_bazelbuild_bazeltoolchains//configs/experimental/ubuntu16_04_clang/1.0/bazel_0.13.0/ubsan:toolchain \ |
||||
--action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 \ |
||||
--extra_toolchains=@com_github_bazelbuild_bazeltoolchains//configs/ubuntu16_04_clang/1.0/bazel_0.13.0/cpp:cc-toolchain-clang-x86_64-default \ |
||||
--extra_execution_platforms=@com_github_bazelbuild_bazeltoolchains//configs/ubuntu16_04_clang/1.0:rbe_ubuntu1604 \ |
||||
-- //test/... || FAILED="true" |
||||
|
||||
if [ "$FAILED" != "" ] |
||||
then |
||||
exit 1 |
||||
fi |
||||
Loading…
Reference in new issue