We've been trying to upgrade Cmake to 3.13 or later as OSS policy bumped it. But we couldn't as Android has a weird linker error with Cmake 3.18 (you can see the error from https://github.com/grpc/grpc/pull/34331) This PR instead upgrades it to use 3.22 for Android test.
Closes#35572
PiperOrigin-RevId: 599317285
The very non-trivial upgrade of third_party/protobuf to 22.x
This PR strives to be as small as possible and many changes that were
compatible with protobuf 21.x and didn't have to be merged atomically
with the upgrade were already merged.
Due to the complexity of the upgrade, this PR wasn't created
automatically by a tool, but manually. Subsequent upgraded of
third_party/protobuf with our OSS release script should work again once
this change is merged.
This is best reviewed commit-by-commit, I tried to group changes in
logical areas.
Notable changes:
- the upgrade of third_party/protobuf submodule, the bazel protobuf
dependency itself
- upgrade of UPB dependency to 22.x (in the past, we used to always
upgrade upb to "main", but upb now has release branch as well). UPB
needs to be upgraded atomically with protobuf since there's a de-facto
circular dependency (new protobuf depends on new upb, which depends on
new protobuf for codegen).
- some protobuf and upb bazel rules are now aliases, so `
extract_metadata_from_bazel_xml.py` and `gen_upb_api_from_bazel_xml.py`
had to be modified to be able to follow aliases and reach the actual
aliased targets.
- some protobuf public headers were renamed, so especially
`src/compiler` needed to be updated to use the new headers.
- protobuf and upb now both depend on utf8_range project, so since we
bundle upb with grpc in some languages, we now have to bundle utf8_range
as well (hence changes in build for python, PHP, objC, cmake etc).
- protoc now depends on absl and utf8_range (previously protobuf had
absl dependency, but not for the codegen part), so python's
make_grpcio_tools.py required partial rewrite to be able to handle those
dependencies in the grpcio_tools build.
- many updates and fixes required for C++ distribtests (currently they
all pass, but we'll probably need to follow up, make protobuf's and
grpc's handling of dependencies more aligned and revisit the
distribtests)
- bunch of other changes mostly due to overhaul of protobuf's and upb's
internal build layout.
TODOs:
- [DONE] make sure IWYU and clang_tidy_code pass
- create a list of followups (e.g. work to reenable the few tests I had
to disable and to remove workaround I had to use)
- [DONE in cl/523706129] figure out problem(s) with internal import
---------
Co-authored-by: Craig Tiller <ctiller@google.com>
* Assert Android API >= v21
This precedes a change that would otherwise break on older Android APIs,
but in a more obvious way.
* error if __ANDROID_API__ is not defined
* update all Andriod minSdkVersions to 21
* csharp experimental: android-19 to 21
Due to limitation of JVM, when user want to create binder channel in
threads created in unmanaged native code, they will need to call this
new API first to make sure Java helper classes can correctly be found.
Unused code in jni_utils.cc are also cleaned up in this commit
* [BinderTransport] Avoid depending on NdkBinder at compile time
We would like to make it possible to use BinderTransport in a APK that
has min sdk version lower than 29 (NdkBinder was introduced at 29)
We copies constants and type definitions from Ndk headers, creates a
same name wrapper for every NdkBinder API we use in
grpc_binder::ndk_util namespace.
We will try to load libbinder_ndk.so and resolve the symbol when the
NdkBinder API wrappers are invoked.
* regenerate projects
* Add GRPC_NO_BINDER guard
A config option is provided so user can pass --define=grpc_no_binder=true to bazel, or passing `-DGRPC_NO_BINDER` to compiler to disable the dependency.
* Expose experimental binder transport API
New headers are added
`grpcpp/create_channel_binder.h `: interfaces for creating client
channel
`grpcpp/security/binder_credentials.h`: interfaces for binder server
credentials
`grpcpp/security/binder_security_policy.h`: interfaces for binder
security policy, which is used by both server and client. Individual
security policies are merged into this single header.
Users can now depend on the `grpc++_binder` target to use the headers
listed above.
* Regenerate projects
* [BinderTransport] Create client channel instead of direct channel
In this commit we create a client channel instead of direct channel.
BinderConnector is added to connect subchannel when the user actually
make RPC call using the channel.
BindToOnDeviceServerService() is not required anymore since now the
actual connection is delay until the channel is used.
* Regenerate projects.
This commit
1. Implements the security policies that does not require JNI. This
includes a security policy that always allow connection and a security
policy that allows connection when remote UID is the same us local UID.
2. Add security policy as an argument in our interfaces. Old interfaces
are temporarily preserved so compilation will not suddenly break when we
import the code to internal repo.
3. Pumping the security policy from the public interfaces to the code
that handles SETUP_TRANSPORT transaction.
4. Abort the transport setup when the security policy is not satisfied.
Since meaningful tests will require to be run in real Android
environment, we will implement it later. For now, this change is
manually tested with example APKs.
For security policies that need to invoke Java to check authorization,
we will implement them in later.
Some code are commented out because the server interface has not been
merged yet.
After this, user should be able to install both apps on Android device
and test if basic unary call works or not
Also try use the API in example apk to make sure the binder transport
code and the rest of gRPC compiles with the Android toolchain
We will properly expose our interface later.
This commit adds an app that will be served as BinderTransport example
later.
Currently the app simply calls C++ function when button is pressed. In
the future the C++ function will run BinderTransport gRPC example
instead.
Tests not included in this commit, later we will add a build test to
CI to make sure this apk is always build-able. (This app will also be
used to make sure our BinderTransport implementation is compile-able
with Android toolchain, on GitHub) For now we exclude this target in
bazel build test.
Changes are made to WORKSPACE file in order to support android builds.
Build instructions are documented at
examples/android/binder/java/io/grpc/binder/cpp/example/README.md
Esun Kim
Jan Tattermusch
Ming-Chuan
AJ Heller
Craig Tiller
Ta-Wei Tu
Nicolas Noble
Ming-Chuan
Eric Gribkoff