~~NB: I haven't tested this at all and am hoping the CI will tell me
where I've (undoubtedly) messed something up.~~ Edit: looks like CI is
now clear!
BoringSSL's gas-compatible assembly files, like its C files, are now
wrapped with preprocessor ifdefs to capture which platforms each file
should be enabled on. This means that, provided the platform can process
.S files it all (i.e. not Windows), we no longer need to detect the
exact CPU architecture in the build.
Switch gRPC's build to take advantage of this. I've retained
BUILD_OVERRIDE_BORING_SSL_ASM_PLATFORM, on the off chance anyone is
using it to cross-compile between Windows and non-Windows, though I
doubt that works particularly well.
As part of this, restore assembly optimizations in a few places where
they were seemingly disabled for issues relating to this:
- https://github.com/grpc/grpc/pull/31747 had to disable the assembly,
because at the time assembly required the library be built differently
for each architecture and then stitched back together. This should now
work.
- tools/run_tests/run_tests.py disabled x86 assembly due to some issues
with CMAKE_SYSTEM_PROCESSOR in a Docker image. This too should now be
moot.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
This PR:
- Fixes the xds-protos Python package, which was broken when the `udpa`
submodule was removed
- This required re-adding the protoc-gen-validate submodule
- Adds non-Bazel tests for xds-protos and all of its dependent packages
- Versions xds-protos the same way as the rest of the Python packages
- Fixes Python 3.11 support in `run_tests.py`, which is necessary for
the testing mentioned above
CC @sergiitk You won't be able to consume this in the interop tests
until it makes it into a release. I'm thinking I'll want to backport
this to the 1.57.x branch to make that happen faster.
CC @drfloob to inform him about the likely backport.
PanCakes to the rescue!
We noticed that our 'sanity' test was going to fail, but we think we can
fix that automatically, so we put together this PR to do just that!
If you'd like to opt-out of these PR's, add yourself to NO_AUTOFIX_USERS
in .github/workflows/pr-auto-fix.yaml
Co-authored-by: HannahShiSFB <HannahShiSFB@users.noreply.github.com>
This adds pre-built library for aarch64 linux, will help improve the
install speed and avoid building environment issues at customer side.
@apolcyn@jtattermusch Can you help build and push the new rake compiler
image?
Will update the tag and hash after the image is available
Manually tested locally:
```
uname -a
Linux u20 5.15.49-linuxkit #1 SMP PREEMPT Tue Sep 13 07:51:32 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
```
```
time gem install /work/ruby/grpc/pkg/grpc-1.56.0.dev-aarch64-linux.gem
Successfully installed grpc-1.56.0.dev-aarch64-linux
Parsing documentation for grpc-1.56.0.dev-aarch64-linux
Installing ri documentation for grpc-1.56.0.dev-aarch64-linux
Done installing documentation for grpc after 0 seconds
1 gem installed
real 0m22.794s
user 0m17.268s
sys 0m5.156s
```
```
ruby greeter_server.rb &
[1] 319
ruby greeter_client.rb
"Greeting: Hello world"
```
Fixes:
https://github.com/grpc/grpc/issues/31855https://github.com/grpc/grpc/issues/29489
- Switched from yapf to black
- Reconfigure isort for black
- Resolve black/pylint idiosyncrasies
Note: I used `--experimental-string-processing` because black was
producing "implicit string concatenation", similar to what described
here: https://github.com/psf/black/issues/1837. While currently this
feature is experimental, it will be enabled by default:
https://github.com/psf/black/issues/2188. After running black with the
new string processing so that the generated code merges these `"hello" "
world"` strings concatenations, then I removed
`--experimental-string-processing` for stability, and regenerated the
code again.
To the reviewer: don't even try to open "Files Changed" tab 😄 It's
better to review commit-by-commit, and ignore `run black and isort`.
(part of removing support for VS2017)
Also see https://github.com/grpc/grpc/pull/32649
Also see https://github.com/grpc/grpc/pull/32615
The switch to grpc-win2019 windows workers has already happened:
(cl/517400022).
Once this PR lands, I'll backport to 1.53.x branch as well (since that
release removes the VS2017 support).
Fix incompatibilities identified when running adhoc runs on the new
custom win2019 image.
After merging this, it should be possible to switch to the new image
without breaking any tests.
- for most fixes I added a comment that explains why they're necessary.
- the new image won't have VS2015 installed, so I'm switching the protoc
artifact build to VS2017
This PR will need to be backported to older release branches to ensure
the windows tests continue working on those branches as well (IMHO I
haven't made any changes that would be difficult to backport and I tried
to keeps the diff as small as possible to avoid issues when
backporting).
After we switch to the new image (and all the windows tests are green),
we can incrementally move the builds that are still using VS2017 to
VS2019.
* Support python 3.11 on aarch64
Fixes https://github.com/grpc/grpc/issues/30927
* Change base tag to something more specific
* Update current version
---------
Co-authored-by: Richard Belleville <rbellevi@google.com>
* remove DOCKERHUB_ORGANIZATION from prepare_build_interop_rc
* switch rubys rake-compiler-dock to .current_version files
* remove DOCKERHUB_ORGANIZATION from prepare_build_linux_rc
* fix .current_version files for third_party/rake-compiler-dock
* fix: avoid .current_version files named */.current_version
* fix: avoid pushing in local only mode
* remove remains of grpc_csharp_ext artifact build completely
* add back part of C# distribtests
* redirect C# linux distribtests
* comment out some distribtests
* switch updateversion to Grpc.tools
* fix .proto
* fix distribtest complile
* temporarily comment out service in .proto
* cleanup
* improvements
* reenable mac distribtests
* fixes
* Support Python 3.11
* Update build images for 3.11
* Whoopsie
* The architecture of this thing is garbage
* Silence ownership warning
* Account for change in git behavior
* Fix directory
* I am in great pain
* Update Windows and arm linux
* Agh
* Clean up
* Update third_party/protobuf
* run tools/distrib/python/make_grpcio_tools.py
* regenerate protos for ruby, php
* update build_handwritten.yaml
* regenerate projects
* Build - Use :well_known_type_protos instead of :well_known_protos
* Fix target
* Update upb
* Update Python for Protobuf 4.21 (#140)
* Update protobuf dependency on grpcio-tools
* Off by one
* Drop python 3.6 support
* Try upgrading pip
* And in the other script
* Try to figure out if we're compatible with abi3
* See what we've already got installed
* Update the requirements.txt file I didn't know existed
* And here too
* See what's installed
* Let's try that again
* Remove
* Try to confirm version
* Let me see the generated code
* Fix non-Bazel test runner
* Work for all test directories
* Regenerate example protos
* Clean up
* Generate .pyi files
* Fix type checking and linting
* Exclude pyi files from isort
* Upgrade to 3.21.4
* Update iwyu to get around messy protobuf IWYU rules
Co-authored-by: Richard Belleville <gnossen@gmail.com>
* Fix Ruby Windows 3.1 (x64-mingw-ucrt) build:
- Pass-in CPPFLAG "GPR_WINDOWS_UCRT" to pickup correct shared object file.
- Rename intermediate legacy SO file from 'grpc_c.64.ruby' to 'grpc_c.64-msvcrt.ruby' for clarity.
- Add artifact target for Ruby x64-mingw-ucrt
* Change CPP flag name
* Attempt to set correct platform on Mac OS
* Add some debug
* Make it fail
* Print more
* Try again
* Maybe it's an ordering issue?
* Get logs back
* Try copying distutils to see exactly what is being used
* Actually export the variable
* I just love debugging with CI
* One directory higher this time
* Try with an upgraded Python install
* Fix version
* Rebreak
* Try setting it even earlier?
* Unbreak
* Try explicitly renaming the artifacts
* Fix
* I am about ready to start NAT hole punching for SSH
* Break things for logs
* Whoops
* Clean up
* Shellcheck
* unify DOCKER_TTY_ARGS in docker scripts
* improvements and cleanup in build_and_run_docker.sh
* fix shellcheck
* make sure python sdist artifact is readable
* Initial GCF distribtest
* Tenatively hook up to CI
* Try again
* Allow dev0 artifacts
* Fix invocation path
* Update gcloud
* Add a 3.8 artifact for presubmits
* And 3.9 too
* Put test files back to normal
* Formatting/linting
* Copyright
* That copyright script doesn't work with shebangs
* Review comments
* Try to create latest-manylinux label
* Accidentally a letter
* Add Python 3.7 manylinux 2014 to presubmit
* Revert CI config file used for test
* Review comments
* Yapf
* Re-add presubmit wheel
* Review comments
* cleanup C# win artifact build
* build C# basictests on win with ninja
* specify default parallelism for C# win artifact
* honor GRPC_PYTHON_BUILD_EXT_COMPILER_JOBS in build_artifact_python.bat
* set --inner_jobs for windows grpc_build_artifact job
* fixup C# build picking env variables from C core build
* get rid of no longer needed NativeDependenciesConfiguration C# setting
* Support musllinux binary wheels
* Skip aarach64 for now
* Consolidate the difference of mktemp
* Extend linux artifact building time && install bash for distribtest
* Stop using grpc.tools, use grpc_tools
* Update the README to use grpc_tools
* Force static link libc++ for alpine binaries
* Rebase recent build script changes
* Install ccache for musllinux distribtest images
* Revert timeout change to grpc_build_artifacts
* Revert "Revert "Cleanup run docker machinery in run_tests.py, task_runner.py and elsewhere (#28704)" (#28741)"
This reverts commit 9a79d44e9b.
* fix python distribtest failure on fedora34
* attempt
* slim down build_docker_and_run_tests.sh
* HOST_GIT_ROOT is unused
* config and arch variable no need to propagate to run_tests.py
* unify unused LOCAL_GIT_ROOT to EXTERNAL_GIT_ROOT
* mounting gcloud config seems useless
* reorder docker args
* cleanup
* allow passing args to build_docker_and_run_tests.sh
* converge build_docker_and_run_tests.sh and build_and_run_docker.sh
* more convergence
* convergence of docker runners
* finalize convergence of run_docker scripts
* GRPC_TEST_REPORT_BASE_DIR might not exist
* adjust report copying
* make report_dir and output_dir readable
* alpine linux does not support cp -t
* xDS: Rbac filter updates (#28568) (#28608)
* Bump version to v1.44.0-pre1 (on the release branch) (#28593)
* bump version to 1.44.0-pre1
* regenerate projects
* only apply "singleplatform" nuget suffix when actually needed (#28677)
* Bump version to 1.44.0-pre2 (#28681)
* bump version to 1.44.0-pre2
* regenerate projects
Co-authored-by: Yash Tibrewal <yashkt@google.com>
David Benjamin
Xuan Wang
Richard Belleville
github-actions[bot]
alto-ruby
Sergii Tkachenko
Jan Tattermusch
Mickaël Guérin
Esun Kim
apolcyn
Alisha Nanda
Yash Tibrewal
Johnny Shields
Jan Tattermusch
Lidi Zheng