This PR is required in order to upgrade to the `master` branch version of protobuf. To do this, we upgrade to the latest version of `rules_python` by:
- Adding in explicit dependencies on all PyPi Bazel test dependencies
- Resolving the circular dependency this creates for `xds-protos` by giving it a Bazel build
- Generating the Python code for `xds-protos` as part of `generate-projects.sh` since doing so directly in Bazel would be highly fragily, dependent on many other projects' Bazel builds
Closes#34945
PiperOrigin-RevId: 599875020
This commit upgrades gRPC to protobuf v25.0 and makes some fixes to
account for upb changes. One major change is that upb has been merged
into the protobuf repo, so we can now drop the separate `@upb`
dependency. Another is that `.upb.c` files no longer exist and there are
new `.upb_minitable.h` and `.upb_minitable.c` files. The longer
filenames exceeded a Windows restriction, so to work around that I
renamed the `upb-generated` directory to just `upb-gen`, and likewise
for `upbdefs-generated`.
- Switched from yapf to black
- Reconfigure isort for black
- Resolve black/pylint idiosyncrasies
Note: I used `--experimental-string-processing` because black was
producing "implicit string concatenation", similar to what described
here: https://github.com/psf/black/issues/1837. While currently this
feature is experimental, it will be enabled by default:
https://github.com/psf/black/issues/2188. After running black with the
new string processing so that the generated code merges these `"hello" "
world"` strings concatenations, then I removed
`--experimental-string-processing` for stability, and regenerated the
code again.
To the reviewer: don't even try to open "Files Changed" tab 😄 It's
better to review commit-by-commit, and ignore `run black and isort`.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Initial PR to establish a bazel dependency on
https://github.com/google/fuzztest, with which I'm planning on basing a
hardening program.
Casting a relatively wide net with reviewers: I'm genuinely interested
in feedback building up the docs, and general ergonomics of this change.
I've located relevant files in the `fuzztest/...` directory. The tests
only build with the `--config fuzztest` bazel argument for now (because
of needing C++17), so locating them separately keeps `bazel test
test/...` working as it does today. In a few years time, when we adopt
C++17, we'll be able to rationalize the test directories a little bit.
We'll need to add some kokoro jobs (maybe with this PR?) to execute the
relevant tests.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
* Run 2to3 on tools directory
* Delete github_stats_tracking
* Re-run 2to3
* Remove unused script
* Remove unused script
* Remove unused line count utility
* Yapf. Isort
* Remove accidentally included file
* Migrate tools/distrib directory to python 3
* Remove unnecessary shebang
* Restore line_count directory
* Immediately convert subprocess.check_output output to string
* Take care of Python 2 shebangs
* Invoke scripts using a Python 3 interpreter
* Yapf. Isort
* Try installing Python 3 first
* See if we have any Python 3 versions installed
* Add Python 3.7 to Windows path
* Try adding a symlink
* Try to symlink differently
* Install six for Python 3
* Run run_interop_tests with python 3
* Try installing six in python3.7 explicitly
* Revert "Try installing six in python3.7 explicitly"
This reverts commit 2cf60d72f3.
* And debug some more
* Fix issue with jobset.py
* Add debug for CI failure
* Revert microbenchmark changes
* Add isort_code.sh to sanity tests
* Run tools/distrib/isort_code.sh
* Fine tune the import order for relative imports
* Make pylint and project generation happy
* Fix a few corner cases
* Use --check instead of --diff
* The import order impacts test result somehow
* Make isort print diff and check output at the same time
* Let tools/run_tests/python_utils be firstparty library
* Run isort against latest HEAD
There was a bug in #26722 where we added a whitespace between comment
prefix and each line of the license notice, regardless it's empty or not.
This creates trailing whitespaces if the license note contains blank
lines, which makes most of the formatters and linters unhappy.
This PR resolves this issue.
* Add a --fix flag to check_copyright.py
When enabled, automatically add copyright headers to files without one.
* Fix formatting issue.
* Address code review comments
All these changes need to go together to make sense
- changes to use new version of upb in bazel
- allowing includes in build target option
- script for generating c code (upb) for protos
- generated code for example protos
- adding changes for non-bazel builds
- change sanity tests to ignore the generated files.
protos
All these changes need to go together to make sense
- changes to use new version of upb in bazel
- allowing includes in build target option
- script for generating c code for protos
- generated code for example build
- adding changes for non-bazel builds
- change sanity tests to ignore the generated files.
Richard Belleville
Adam Cozzette
Sergii Tkachenko
Craig Tiller
Yijie Ma
Lidi Zheng
Ta-Wei Tu
Ta-Wei Tu
Esun Kim
Jan Tattermusch
Yash Tibrewal
Hannah Shi
Mark D. Roth
nanahpang
Muxi Yan
Vishal Powar
Alexander Polcyn
Vishal Powar
David Garcia Quintas
Carl Mastrangelo
Eric Gribkoff
Yihua Zhang
Sree Kuchibhotla
Matt Kwong
Muxi Yan
Mehrdad Afshari
ncteisen