Chiebot-Mirror/grpc - grpc - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Shane Loretz	385431ca4b	[Python Bazel] Allow overwriting grpc_library in python_rules.bzl (#35629 ) Continues #35412 addressing feedback in https://github.com/grpc/grpc/pull/35412#issuecomment-1875980755 I'm unable to test the workspace with the newly added py_test, so I'm hoping a CI run here will tell me if the test works. ```console ~/grpc/test/distrib/bazel/python$ bazel --nohome_rc --nosystem_rc test //... Loading: Loading: Loading: 0 packages loaded Analyzing: 37 targets (0 packages loaded, 0 targets configured) ERROR: /usr/local/foobar/home/sloretz/.cache/bazel/_bazel_sloretz/7f83b4f00f370e7c52a5cc586445673c/external/com_google_protobuf/upb_generator/BUILD:266:21: @com_google_protobuf//upb_generator:protoc-gen-upb_toolchain: no such attribute 'output_files' in 'proto_lang_toolchain' rule ERROR: /usr/local/foobar/home/sloretz/.cache/bazel/_bazel_sloretz/7f83b4f00f370e7c52a5cc586445673c/external/com_google_protobuf/upb_generator/BUILD:305:21: @com_google_protobuf//upb_generator:protoc-gen-upb_minitable_toolchain: no such attribute 'output_files' in 'proto_lang_toolchain' rule ERROR: /usr/local/foobar/home/sloretz/.cache/bazel/_bazel_sloretz/7f83b4f00f370e7c52a5cc586445673c/external/com_google_protobuf/upb_generator/BUILD:338:21: @com_google_protobuf//upb_generator:protoc-gen-upbdefs_toolchain: no such attribute 'output_files' in 'proto_lang_toolchain' rule ERROR: /usr/local/foobar/home/sloretz/.cache/bazel/_bazel_sloretz/7f83b4f00f370e7c52a5cc586445673c/external/com_google_protobuf/upb_generator/BUILD:305:21: Target '@com_google_protobuf//upb_generator:protoc-gen-upb_minitable_stage1' contains an error and its package is in error and referenced by '@com_google_protobuf//upb_generator:protoc-gen-upb_minitable_toolchain' ERROR: /usr/local/foobar/home/sloretz/.cache/bazel/_bazel_sloretz/7f83b4f00f370e7c52a5cc586445673c/external/com_github_grpc_grpc/src/proto/grpc/gcp/BUILD:19:14: every rule of type proto_library implicitly depends upon the target '@com_google_protobuf//upb_generator:protoc-gen-upb_minitable_toolchain', but this target could not be found because of: Target '@com_google_protobuf//upb_generator:protoc-gen-upb_minitable_toolchain' contains an error and its package is in error ERROR: Analysis failed ERROR: Analysis of target '//namespaced/upper/example:no_import_no_strip_py_pb2_grpc' failed; build aborted: INFO: Elapsed time: 0.171s INFO: 0 processes. FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured) ERROR: Couldn't start the build. Unable to run tests ``` Closes #35629 PiperOrigin-RevId: 606695865	10 months ago
Ming Lyu	c8f467aea9	[Bazel] Enable grpcio-reflection to be used via Bazel (#31013 ) The current `py_grpc_library` results in the wrong grpc proto python code path when grpc is a third-party source code in a Bazel project. This PR should fix it. fixes #31011 <!-- If you know who should review your pull request, please assign it to that person, otherwise the pull request would get assigned randomly. If your pull request is for a specific language, please add the appropriate lang label. --> --------- Co-authored-by: Richard Belleville <rbellevi@google.com>	1 year ago
Florian Ferstl	fa32eb36b3	[bazel] Auto-generate .pyi files in py_proto_library rule (#32872 ) With some delay, this is a PR for https://github.com/grpc/grpc/issues/32564 (and previously https://github.com/grpc/grpc/pull/31791). I looked into adding a regular `py_test` for this change [as suggested](https://github.com/grpc/grpc/pull/31791#issuecomment-1423245116) but I am not aware of any effect that the presence of a .pyi stub file would have at runtime and where some sort of type-checking in a .py script would be affected. Stub files are only for use by type checkers & IDE's. I mean, something like this would work: ``` import helloworld_pb2 py_file = helloworld_pb2.__file__ pyi_file = py_file + 'i’ self.assertTrue(os.path.exists(pyi_file)) ``` But that seems really hacky to me. Instead I created a simple rule test for `py_proto_library` with Bazel Skylib which tests the declared outputs for an example `py_proto_library` target. Indirectly, this also tests that the declared output files are actually generated. Please let me know if this is sufficient.	2 years ago
Yijie Ma	10001d16a9	[Release] Upmerge v1.55.x branch into master (#33277 ) Change was created by the release automation script. See go/grpc-release <!-- If you know who should review your pull request, please assign it to that person, otherwise the pull request would get assigned randomly. If your pull request is for a specific language, please add the appropriate lang label. --> --------- Co-authored-by: Yash Tibrewal <yashkt@google.com> Co-authored-by: Esun Kim <veblush@google.com> Co-authored-by: Mark D. Roth <roth@google.com> Co-authored-by: Yousuk Seung <ysseung@google.com> Co-authored-by: Craig Tiller <ctiller@google.com> Co-authored-by: Richard Belleville <rbellevi@google.com> Co-authored-by: gnossen <gnossen@users.noreply.github.com>	2 years ago
Jan Tattermusch	0f1afec5a8	[protobuf] Upgrade third_party/protobuf to 22.x (#32606 ) The very non-trivial upgrade of third_party/protobuf to 22.x This PR strives to be as small as possible and many changes that were compatible with protobuf 21.x and didn't have to be merged atomically with the upgrade were already merged. Due to the complexity of the upgrade, this PR wasn't created automatically by a tool, but manually. Subsequent upgraded of third_party/protobuf with our OSS release script should work again once this change is merged. This is best reviewed commit-by-commit, I tried to group changes in logical areas. Notable changes: - the upgrade of third_party/protobuf submodule, the bazel protobuf dependency itself - upgrade of UPB dependency to 22.x (in the past, we used to always upgrade upb to "main", but upb now has release branch as well). UPB needs to be upgraded atomically with protobuf since there's a de-facto circular dependency (new protobuf depends on new upb, which depends on new protobuf for codegen). - some protobuf and upb bazel rules are now aliases, so ` extract_metadata_from_bazel_xml.py` and `gen_upb_api_from_bazel_xml.py` had to be modified to be able to follow aliases and reach the actual aliased targets. - some protobuf public headers were renamed, so especially `src/compiler` needed to be updated to use the new headers. - protobuf and upb now both depend on utf8_range project, so since we bundle upb with grpc in some languages, we now have to bundle utf8_range as well (hence changes in build for python, PHP, objC, cmake etc). - protoc now depends on absl and utf8_range (previously protobuf had absl dependency, but not for the codegen part), so python's make_grpcio_tools.py required partial rewrite to be able to handle those dependencies in the grpcio_tools build. - many updates and fixes required for C++ distribtests (currently they all pass, but we'll probably need to follow up, make protobuf's and grpc's handling of dependencies more aligned and revisit the distribtests) - bunch of other changes mostly due to overhaul of protobuf's and upb's internal build layout. TODOs: - [DONE] make sure IWYU and clang_tidy_code pass - create a list of followups (e.g. work to reenable the few tests I had to disable and to remove workaround I had to use) - [DONE in cl/523706129] figure out problem(s) with internal import --------- Co-authored-by: Craig Tiller <ctiller@google.com>	2 years ago
Richard Belleville	f688f5fcf2	Rehome python bazel distribtest (#29275 ) * WIP * Test multiple languages * Yapf * isort * Shellcheck * comment not provided * Fix import ordering issue. * Turn up Bazel logging	3 years ago
Thomas Köppe	90a56a3d15	[bazel/protobuf.bzl] Improve py_proto_library path handling (#28040 ) This change replaces the simple `basename` logic with a more general "remove prefix" logic. This allows py_proto_library to work even when the input proto_library's sources are contained in strict subdirectories of their package. (This situation is not uncommon when using remote, non-Bazel repositories that only get a single BUILD file.)	3 years ago
Richard Belleville	eab375e439	Reland #27275 (#27806 ) * Revert "Revert "Add Aspects to Bazel py_proto_library and py_grpc_library Rules (#27275)" (#27805)" This reverts commit `763e5b40f3`. * Remove unused symbol	3 years ago
Craig Tiller	763e5b40f3	Revert "Add Aspects to Bazel py_proto_library and py_grpc_library Rules (#27275 )" (#27805 ) This reverts commit `e970b8fa03`.	3 years ago
Michael Beardsworth	e970b8fa03	Add Aspects to Bazel py_proto_library and py_grpc_library Rules (#27275 ) * Move most py_proto_library macro contents into rule. The _generate_pb2_src rule is modified to produce a valid PyInfo provider that can be depended on directly by py_.* rules. Keeping an intermediate macro in place increases complexity without adding any real value. * Move most py_grpc_library macro contents into rule. The _generate_pb2_grpc_src rule is modified to produce a valid PyInfo provider that can be depended on directly by py_.* rules. Keeping an intermediate macro in place increase complexity without adding any real value. * Remove support for additional plugins. It's easy enough for a plugin author to build their own rule-set, and it adds extra unused (and untested) complexity to gRPC's rules. * Rewrite py_proto_library to track dependencies correctly. * Propagate Python dependencies of py_grpc_library rules to consumers. * Add test coverage for aspect-based py_proto_library.	3 years ago
Richard Belleville	25d8458721	Migrate remaining tests with protobuf dep to Python 3 only (#27776 ) * Migrate remaining tests with protobuf dep to Python 3 only * Move Bazel Python repo tests off of Python 2 * Format	3 years ago
Matt Wilson	594e1f6a64	py_proto_library uses workspace and pkg paths All python imports currently are added to the runfiles under __main__ which is only the default when no workspace name is provided. This change supports both empty workspace names, and specified workspace names. This also supports python libs not anchored at root	5 years ago
Yannic Bonenberger	06ecc87290	[bazel] Add fixes for --incompatible_load_proto_rules_from_bzl	5 years ago
Lidi Zheng	c34bef4ca4	Format all Bazel files with buildifier	5 years ago
vam-google	91c0d093bd	Fix unnecessary dependency and add new line	5 years ago
vam-google	5d9eeb34fc	Address PR feedback. Also add a dummy plugin.	5 years ago
vam-google	dbe62a6e6c	Switch to `py2and3_test` rule from py_test	5 years ago
vam-google	564dc771dc	Return PyInfo provider with imports from _gen rules and pass that as deps in py_library. This allows hiding _virtual_imports include path from the surface.	5 years ago
vam	461c053c37	[bazel][python] Support _virtual_imports input for py_proto_library and py_grpc_library rules `proto_library` targets are used as deps for `py_proto_library` and `py_grpc_library` rules. The `proto_library` targets can be configured using `import_prefix` and/or `strip_import_prefix` (which essentially move original location of the proto file and, as result, affects import path within proto files themselves and in the generated language-specific stubs). The biggest question to answer when generating stubs from moved protos is where to put result (this decision affects all downstream rules as well, because the location of file affects its import path). This PR tries to follow same logic as the native `cc_proto_library` (created and maintained by Bazel team). For example, if we have `firestore.proto` file, which is located under `google/firestore/v1beta1` Bazel package and want to move it under `google/cloud/firestore_v1beta1/proto` (this is axactly what happens in googleapis repository), it can be configured the following way: ```bzl proto_library( name = "firestore_moved_proto", srcs = ["firestore.proto"], import_prefix = "google/cloud/firestore_v1beta1/proto", strip_import_prefix = "google/firestore/v1beta1", ) ``` The rule above will first generate virtual `.proto` files (under new location) and only after that generate a binary descriptor from them. Specifically it will generate the following "virtual" file under `_virtual_imports` subdirectory of same package: ``` bazel-bin/google/firestore/v1beta1/_virtual_imports/firestore_moved_proto/google/cloud/firestore_v1beta1/proto/common.proto ``` When supplied to `cc_proto_library`, like the following: ```bzl cc_proto_library( name = "firestore_moved_cc_proto", deps = ["firestore_moved_proto"], ) ``` The rule will generate .cc and .h files like the following: ``` bazel-bin/google/firestore/v1beta1/_virtual_imports/firestore_moved_proto/google/cloud/firestore_v1beta1/proto/firestore.pb.h bazel-bin/google/firestore/v1beta1/_virtual_imports/firestore_moved_proto/google/cloud/firestore_v1beta1/proto/firestore.pb.cc ``` Notice, that each generated `.cc` and `.h` file is prefixed with `_virtual_imports/<name_of_proto_library_target>`. The python rules try to do same thing, so for the following py_proto_library rule: ```bzl py_proto_library( name = "firestore_moved_py_proto", deps = [":firestore_moved_proto"], ) ``` It wil generate the following file: ``` bazel-bin/google/firestore/v1beta1/_virtual_imports/firestore_moved_proto/google/cloud/firestore_v1beta1/proto/firestore_pb2.py ``` I.e in same path as cc_proto_library. This all woks, but an annoying part is that to use the generated library in some other py_library, in needs to specify the "_virtual_imports/proto_target" path as its "includes" parameter (and I can't make it better than that). Another option would be to skeep the `_virtual_imports/<name_of_proto_library_target>` the suffix for the generated python stubs, which will make the path look like the following: ``` bazel-bin/google/firestore/v1beta1/google/cloud/firestore_v1beta1/proto/firestore_pb2.py ``` That will make using generated stubs simpler and cleaner (no need to specify imports argument), but it will make it inconsistent with the other rules (like cc_proto_library) and also more susseptible to naming conflicts (if there is already something under the generated path).	5 years ago
Richard Belleville	8ce740cfe0	Switch py_proto_library from using src to deps to conform with google3	5 years ago
Richard Belleville	835b3d3293	Copyright.	5 years ago
Richard Belleville	7b2c8c27b0	Separate py_grpc_library and py_proto_library. By popular demand, we'll now be offering separate py_grpc_library and py_proto_library targets sharing the same interface as within google3. This change necessitated some modifications to how we pull in our own Python-level dependencies and how we make those available to those pulling in our project via Bazel. There is now a grpc_python_deps() Bazel workspace rule that pulls in the appropriate dependencies, which should be called from the client project's WORKSPACE file. A test has been added to the bazel/test/ directory to verify that this behavior works as intended. It's worth noting that the protobuf repository's usage of Starlark bind() caused a great deal of trouble in ensuring that we could also pull in six. This change also required a change in the way generated proto code is imported in the channelz and health-check modules, as well as in their associated tests. We were importing them two different ways, each relative. This resulted in two different module objects being imported into the process, which were incompatible. I am not sure exactly what caused this behavior to begin, as this should have been possible before this PR. As a workaround, I am simply trying two different absolute imports and using the one that works. This should function both inside and outside of Bazel environments.	5 years ago

6 Commits (384437db450d16fa6844157909ff834eb487ff0e)