The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#) https://grpc.io/
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 
Esun Kim f1254a78b3
[Doc] Update README.md to address upb issue (#34411)
11 months ago
..
abseil-cpp@29bf8085f3 [Deps] Update abseil to 20230802 (#33974) 1 year ago
address_sorting address_sorting BUILD: remove license comment breaking import (#31145) 2 years ago
android Make Compatible with Bazel 5.0 (#28683) 3 years ago
benchmark@344117638c [Build] Prepared for Bazel 7 (#35374) 12 months ago
bloaty@60209eb1cc Reland bloat labelling code. (#27926) 3 years ago
boringssl-with-bazel@2ff4b968a7 [Deps] Update boringssl 202308211007 (#34115) 1 year ago
cares [GPR] Removed GPR_BACKWARDS_COMPATIBILITY_MODE (#35602) 11 months ago
envoy-api@9d6ffa7067 [xDS protos] update to get new fields for dualstack endpoints (#34491) 1 year ago
googleapis@2f9af297c8 Update Envoy API to the latest version (2021-07-30) (#26848) 3 years ago
googletest@2dd1c13195 [test] Update gTest (#34629) 1 year ago
objective_c Revert "Import cronet net error list and modify grpc-cronet error message." 4 years ago
opencensus-proto@4aa53e15cb [xDS Proto] Update Bazel dependencies (#25511) 4 years ago
opentelemetry@60fa8754d8 Add opentelemetry as a submodule for latest xDS API (#26850) 3 years ago
protobuf@7f94235e55 [Deps] Update protobuf & upb to 3.25.1 (#35315) 1 year ago
protoc-gen-validate@fab737efbb [packaging] Publish xds-protos as part of the standard package pipeline (#33797) 1 year ago
py fix python detection on windows (#30302) 2 years ago
rake-compiler-dock [ruby] Build/test ruby 3.3 and build native gems with Ruby 3.3 support (#35399) 11 months ago
re2@0c5616df9c Upgrade re2 to 2022-04-01 (#31665) 2 years ago
toolchains [bazel rbe] Update RBE windows docker image (and move it to GAR) (#34229) 1 year ago
upb/upb [Deps] Update protobuf & upb to 3.25.1 (#35315) 1 year ago
utf8_range [deps] Upgrade protobuf version to v25.0 (#34513) 1 year ago
xds@e9ce68804c [WRR] Prefer application_utilization to cpu_utilization (#33355) 2 years ago
xxhash update third_party/xxhash (#28236) 3 years ago
zlib@09155eaa2f Updated zlib to 1.3 (#35147) 1 year ago
ABSEIL_MANUAL.md Update abseil-doc (#26450) 4 years ago
BUILD Fix breakage when built against protobuf head (#34945) 11 months ago
README.md [Doc] Update README.md to address upb issue (#34411) 11 months ago
constantly.BUILD Add python bazel deps needed to run resolver component tests 7 years ago
cython.BUILD Basic setup to build gRPC Python with Bazel 6 years ago
enum34.BUILD Separate py_grpc_library and py_proto_library. 5 years ago
futures.BUILD Separate py_grpc_library and py_proto_library. 5 years ago
incremental.BUILD Add python bazel deps needed to run resolver component tests 7 years ago
libprotobuf_mutator.BUILD Add structurally aware fuzzing support (#27405) 3 years ago
protobuf.patch [Deps] Update protobuf & upb to 3.25.1 (#35315) 1 year ago
protoc-gen-validate.patch [xDS Proto] Update Bazel build dependencies for Envoy API (#29219) 3 years ago
six.BUILD Upgrade six from 1.12 to 1.16 in Bazel (#26451) 4 years ago
twisted.BUILD Add python bazel deps needed to run resolver component tests 7 years ago
yaml.BUILD Add python bazel deps needed to run resolver component tests 7 years ago
zlib.BUILD Update zlib dependency 6 years ago
zope_interface.BUILD Add python bazel deps needed to run resolver component tests 7 years ago

README.md

Third-party libraries

gRPC depends on several third-party libraries, their source code is available (usually as a git submodule) in this directory.

Guidelines on updating submodules

  • IMPORTANT: whenever possible, try to only update to a stable release of a library (= not to master / random commit). Depending on unreleased revisions makes gRPC installation harder for users, as it forces them to always build the dependency from source and prevents them from using more convenient installation channels (linux packages, package managers etc.)

  • bazel BUILD uses a different dependency model - whenever updating a submodule, also update the revision in grpc_deps.bzl so that bazel and non-bazel builds stay in sync (this is actually enforced by a sanity check in some cases)

Considerations when adding a new third-party dependency

  • gRPC C++ needs to stay buildable/installable even if the submodules are not present (e.g. the tar.gz archive with gRPC doesn't contain the submodules), assuming that the dependencies are already installed. This is a requirement for being able to provide a reasonable install process (e.g. using cmake) and to support package managers for gRPC C++.

    • CAVEAT: upb is an exception here because of its lack of cmake support. Therefore, third_party/upb should be present to build gRPC until upb supports it.
  • Adding a new dependency is a lot of work (both for us and for the users). We currently support multiple build systems (BAZEL, cmake, make, ...) so adding a new dependency usually requires updates in multiple build systems (often not trivial). The installation process also needs to continue to work (we do have distrib tests to test many of the possible installation scenarios, but they are not perfect). Adding a new dependency also usually affects the installation instructions that need to be updated. Also keep in mind that adding a new dependency can be quite disruptive for the users and community - it means that all users will need to update their projects accordingly (for C++ projects often non-trivial) and the community-provided C++ packages (e.g. vcpkg) will need to be updated as well.

Checklist for adding a new third-party dependency

READ THIS BEFORE YOU ADD A NEW DEPENDENCY

  • Make sure you understand the hidden costs of adding a dependency (see section above) and that you understand the complexities of updating the build files. Maintenance of the build files isn't for free, so expect to be involved in maintenance tasks, cleanup and support (e.g resolving user bugs) of the build files in the future.

  • Once your change is ready, start an adhoc run of artifact - packages - distribtests flow and make sure everything passes (for technical reasons, not all the distribtests can run on each PR automatically).

  • Check the impact of the new dependency on the size of our distribution packages (compare BEFORE and AFTER) and post the comparison on your PR (it should not be approved without checking the impact sizes of packages first). The package sizes AFTER can be obtained from the adhoc package build from bullet point above.

Instructions for updating dependencies

Usually the process is

  1. update the submodule to selected commit (see guidance above)
  2. update the dependency in grpc_deps.bzl to the same commit
  3. update tools/run_tests/sanity/check_submodules.sh to make the sanity test pass
  4. (when needed) run tools/buildgen/generate_projects.sh to regenerate the generated files
  5. populate the bazel download mirror by running bazel/update_mirror.sh

Updating some dependencies requires extra care.

Updating third_party/abseil-cpp

  • Two additional steps should be done before running generate_projects.sh above.
    • Running src/abseil-cpp/preprocessed_builds.yaml.gen.py.
    • Updating abseil_version = scripts in templates/gRPC-C++.podspec.template and templates/gRPC-Core.podspec.template.
  • You can see an example of previous upgrade.

Updating third_party/boringssl-with-bazel

NOTE: updating the boringssl dependency is now part of the internal grpc release tooling (see go/grpc-release). Prefer using the release tooling when possible. The instructions below are provided as a reference and aren't guaranteed to be up-to-date.

  • Update the third_party/boringssl-with-bazel submodule to the latest master-with-bazel branch
git submodule update --init      # just to start in a clean state
cd third_party/boringssl-with-bazel
git fetch origin   # fetch what's new in the boringssl repository
git checkout origin/master-with-bazel  # checkout the current state of master-with-bazel branch in the boringssl repo
# Note the latest commit SHA on master-with-bazel branch
cd ../..   # go back to grpc repo root
git status   #  will show that there are new commits in third_party/boringssl-with-bazel
git add  third_party/boringssl-with-bazel     # we actually want to update the changes to the submodule
git commit -m "update submodule boringssl-with-bazel with origin/master-with-bazel"   # commit
  • Update boringssl dependency in bazel/grpc_deps.bzl to the same commit SHA as master-with-bazel branch

    • Update http_archive(name = "boringssl", section by updating the sha in strip_prefix and urls fields.
    • Also, set sha256 field to "" as the existing value is not valid. This will be added later once we know what that value is.
  • Update tools/run_tests/sanity/check_submodules.sh with the same commit

  • Commit these changes git commit -m "update boringssl dependency to master-with-bazel commit SHA"

  • Run tools/buildgen/generate_projects.sh to regenerate the generated files

    • Because sha256 in bazel/grpc_deps.bzl was left empty, you will get a DEBUG msg like this one:
      Rule 'boringssl' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "SHA value"
      
    • Commit the regenrated files git commit -m "regenerate files"
    • Update bazel/grpc_deps.bzl with the SHA value shown in the above debug msg. Commit again git commit -m "Updated sha256"
  • Run tools/distrib/generate_boringssl_prefix_header.sh

    • Commit again git commit -m "generate boringssl prefix headers"
  • Increment the boringssl podspec version number in templates/src/objective-c/BoringSSL-GRPC.podspec.template and templates/gRPC-Core.podspec.template. example

    • Commit again git commit -m "Increment podspec version"
  • Run tools/buildgen/generate_projects.sh (yes, again)

    • Commit again git commit -m "Second regeneration"
  • Create a PR with all the above commits.

  • Run bazel/update_mirror.sh to update GCS mirror.

Updating third_party/protobuf

Updating the protobuf dependency is now part of the internal release process (see go/grpc-release).

Updating third_party/envoy-api

Apart from the above steps, please run tools/codegen/core/gen_upb_api.sh to regenerate upb files.

In addition, please perform the following two steps to generate the Python xds-protos package:

  1. Bump the version in the tools/distrib/python/xds_protos/setup.py;
  2. Run tools/distrib/python/xds_protos/build_validate_upload.sh to upload the built wheel.

Updating third_party/upb

Since upb is vendored in the gRPC repo, you cannot use submodule to update it. Please follow the steps below;

  1. Update third_party/upb directory by running
    • export GRPC_ROOT=~/git/grpc
    • wget https://github.com/protocolbuffers/protobuf/releases/download/v25.1/protobuf-25.1.zip
    • rm -rf $GRPC_ROOT/third_party/upb
    • unzip protobuf-25.1.zip -d /tmp/protobuf
    • cp -r /tmp/protobuf/protobuf-25.1/upb $GRPC_ROOT/third_party/upb
  2. Update the dependency in grpc_deps.bzl to the same commit
  3. Populate the bazel download mirror by running bazel/update_mirror.sh
  4. Run tools/buildgen/generate_projects.sh to regenerate the generated files
  5. Run tools/codegen/core/gen_upb_api.sh to regenerate upb files.

Updating third_party/utf8_range

# set to wherever your grpc repo lives
export GRPC_ROOT=~/git/grpc
wget https://github.com/protocolbuffers/utf8_range/archive/refs/heads/main.zip
rm -rf $GRPC_ROOT/third_party/utf8_range
unzip main.zip -d $GRPC_ROOT/third_party
mv $GRPC_ROOT/third_party/utf8_range-main $GRPC_ROOT/third_party/utf8_range

Updating third_party/xxhash

TODO(https://github.com/Cyan4973/xxHash/issues/548): revisit LICENSE instructions if upstream splits library and CLI.

The upstream xxhash repository contains a bunch of files that we don't want, so we employ a rather manual update flow to pick up just the bits we care about:

git remote add xxhash https://github.com/Cyan4973/xxHash.git
git fetch xxhash
git show xxhash/dev:xxhash.h > third_party/xxhash/xxhash.h
git show xxhash/dev:LICENSE | sed -nE '/^-+$/q;p' > third_party/xxhash/LICENSE