~~NB: I haven't tested this at all and am hoping the CI will tell me
where I've (undoubtedly) messed something up.~~ Edit: looks like CI is
now clear!
BoringSSL's gas-compatible assembly files, like its C files, are now
wrapped with preprocessor ifdefs to capture which platforms each file
should be enabled on. This means that, provided the platform can process
.S files it all (i.e. not Windows), we no longer need to detect the
exact CPU architecture in the build.
Switch gRPC's build to take advantage of this. I've retained
BUILD_OVERRIDE_BORING_SSL_ASM_PLATFORM, on the off chance anyone is
using it to cross-compile between Windows and non-Windows, though I
doubt that works particularly well.
As part of this, restore assembly optimizations in a few places where
they were seemingly disabled for issues relating to this:
- https://github.com/grpc/grpc/pull/31747 had to disable the assembly,
because at the time assembly required the library be built differently
for each architecture and then stitched back together. This should now
work.
- tools/run_tests/run_tests.py disabled x86 assembly due to some issues
with CMAKE_SYSTEM_PROCESSOR in a Docker image. This too should now be
moot.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
PanCakes to the rescue!
We noticed that our 'sanity' test was going to fail, but we think we can
fix that automatically, so we put together this PR to do just that!
If you'd like to opt-out of these PR's, add yourself to NO_AUTOFIX_USERS
in .github/workflows/pr-auto-fix.yaml
Co-authored-by: HannahShiSFB <HannahShiSFB@users.noreply.github.com>
This adds pre-built library for aarch64 linux, will help improve the
install speed and avoid building environment issues at customer side.
@apolcyn@jtattermusch Can you help build and push the new rake compiler
image?
Will update the tag and hash after the image is available
Manually tested locally:
```
uname -a
Linux u20 5.15.49-linuxkit #1 SMP PREEMPT Tue Sep 13 07:51:32 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
```
```
time gem install /work/ruby/grpc/pkg/grpc-1.56.0.dev-aarch64-linux.gem
Successfully installed grpc-1.56.0.dev-aarch64-linux
Parsing documentation for grpc-1.56.0.dev-aarch64-linux
Installing ri documentation for grpc-1.56.0.dev-aarch64-linux
Done installing documentation for grpc after 0 seconds
1 gem installed
real 0m22.794s
user 0m17.268s
sys 0m5.156s
```
```
ruby greeter_server.rb &
[1] 319
ruby greeter_client.rb
"Greeting: Hello world"
```
Fixes:
https://github.com/grpc/grpc/issues/31855https://github.com/grpc/grpc/issues/29489
- Switched from yapf to black
- Reconfigure isort for black
- Resolve black/pylint idiosyncrasies
Note: I used `--experimental-string-processing` because black was
producing "implicit string concatenation", similar to what described
here: https://github.com/psf/black/issues/1837. While currently this
feature is experimental, it will be enabled by default:
https://github.com/psf/black/issues/2188. After running black with the
new string processing so that the generated code merges these `"hello" "
world"` strings concatenations, then I removed
`--experimental-string-processing` for stability, and regenerated the
code again.
To the reviewer: don't even try to open "Files Changed" tab 😄 It's
better to review commit-by-commit, and ignore `run black and isort`.
* Support python 3.11 on aarch64
Fixes https://github.com/grpc/grpc/issues/30927
* Change base tag to something more specific
* Update current version
---------
Co-authored-by: Richard Belleville <rbellevi@google.com>
* remove remains of grpc_csharp_ext artifact build completely
* add back part of C# distribtests
* redirect C# linux distribtests
* comment out some distribtests
* switch updateversion to Grpc.tools
* fix .proto
* fix distribtest complile
* temporarily comment out service in .proto
* cleanup
* improvements
* reenable mac distribtests
* fixes
* Support Python 3.11
* Update build images for 3.11
* Whoopsie
* The architecture of this thing is garbage
* Silence ownership warning
* Account for change in git behavior
* Fix directory
* I am in great pain
* Update Windows and arm linux
* Agh
* Clean up
* Update third_party/protobuf
* run tools/distrib/python/make_grpcio_tools.py
* regenerate protos for ruby, php
* update build_handwritten.yaml
* regenerate projects
* Build - Use :well_known_type_protos instead of :well_known_protos
* Fix target
* Update upb
* Update Python for Protobuf 4.21 (#140)
* Update protobuf dependency on grpcio-tools
* Off by one
* Drop python 3.6 support
* Try upgrading pip
* And in the other script
* Try to figure out if we're compatible with abi3
* See what we've already got installed
* Update the requirements.txt file I didn't know existed
* And here too
* See what's installed
* Let's try that again
* Remove
* Try to confirm version
* Let me see the generated code
* Fix non-Bazel test runner
* Work for all test directories
* Regenerate example protos
* Clean up
* Generate .pyi files
* Fix type checking and linting
* Exclude pyi files from isort
* Upgrade to 3.21.4
* Update iwyu to get around messy protobuf IWYU rules
Co-authored-by: Richard Belleville <gnossen@gmail.com>
* Fix Ruby Windows 3.1 (x64-mingw-ucrt) build:
- Pass-in CPPFLAG "GPR_WINDOWS_UCRT" to pickup correct shared object file.
- Rename intermediate legacy SO file from 'grpc_c.64.ruby' to 'grpc_c.64-msvcrt.ruby' for clarity.
- Add artifact target for Ruby x64-mingw-ucrt
* Change CPP flag name
* Attempt to set correct platform on Mac OS
* Add some debug
* Make it fail
* Print more
* Try again
* Maybe it's an ordering issue?
* Get logs back
* Try copying distutils to see exactly what is being used
* Actually export the variable
* I just love debugging with CI
* One directory higher this time
* Try with an upgraded Python install
* Fix version
* Rebreak
* Try setting it even earlier?
* Unbreak
* Try explicitly renaming the artifacts
* Fix
* I am about ready to start NAT hole punching for SSH
* Break things for logs
* Whoops
* Clean up
* Shellcheck
* Initial GCF distribtest
* Tenatively hook up to CI
* Try again
* Allow dev0 artifacts
* Fix invocation path
* Update gcloud
* Add a 3.8 artifact for presubmits
* And 3.9 too
* Put test files back to normal
* Formatting/linting
* Copyright
* That copyright script doesn't work with shebangs
* Review comments
* Try to create latest-manylinux label
* Accidentally a letter
* Add Python 3.7 manylinux 2014 to presubmit
* Revert CI config file used for test
* Review comments
* Yapf
* Re-add presubmit wheel
* Review comments
* Support musllinux binary wheels
* Skip aarach64 for now
* Consolidate the difference of mktemp
* Extend linux artifact building time && install bash for distribtest
* Stop using grpc.tools, use grpc_tools
* Update the README to use grpc_tools
* Force static link libc++ for alpine binaries
* Rebase recent build script changes
* Install ccache for musllinux distribtest images
* Revert timeout change to grpc_build_artifacts
* Revert "Revert "Cleanup run docker machinery in run_tests.py, task_runner.py and elsewhere (#28704)" (#28741)"
This reverts commit 9a79d44e9b.
* fix python distribtest failure on fedora34
* attempt
* slim down build_docker_and_run_tests.sh
* HOST_GIT_ROOT is unused
* config and arch variable no need to propagate to run_tests.py
* unify unused LOCAL_GIT_ROOT to EXTERNAL_GIT_ROOT
* mounting gcloud config seems useless
* reorder docker args
* cleanup
* allow passing args to build_docker_and_run_tests.sh
* converge build_docker_and_run_tests.sh and build_and_run_docker.sh
* more convergence
* convergence of docker runners
* finalize convergence of run_docker scripts
* GRPC_TEST_REPORT_BASE_DIR might not exist
* adjust report copying
* make report_dir and output_dir readable
* alpine linux does not support cp -t
* add --dry_run support for task_runner.py
* dont sort task_runner targets before running
* fixup dry run
* support task_runner.py --inner_jobs
* pass inner_jobs to build_jobspec for task_runner.py
* support inner_jobs for C# artifacts
* support inner_jobs for protoc artifacts
* inner jobs support for src/csharp/experimental/build_native_ext_for_android.sh
* address review feedback
* Run 2to3 on tools directory
* Delete github_stats_tracking
* Re-run 2to3
* Remove unused script
* Remove unused script
* Remove unused line count utility
* Yapf. Isort
* Remove accidentally included file
* Migrate tools/distrib directory to python 3
* Remove unnecessary shebang
* Restore line_count directory
* Immediately convert subprocess.check_output output to string
* Take care of Python 2 shebangs
* Invoke scripts using a Python 3 interpreter
* Yapf. Isort
* Try installing Python 3 first
* See if we have any Python 3 versions installed
* Add Python 3.7 to Windows path
* Try adding a symlink
* Try to symlink differently
* Install six for Python 3
* Run run_interop_tests with python 3
* Try installing six in python3.7 explicitly
* Revert "Try installing six in python3.7 explicitly"
This reverts commit 2cf60d72f3.
* And debug some more
* Fix issue with jobset.py
* Add debug for CI failure
* Revert microbenchmark changes
* Let's see if the aarch64 manylinux bug is fixed or not
* rename aarch64 image back to manylinux2014 for consistency
* fix C# aarch64 artifact build
Co-authored-by: Lidi Zheng <lidiz@google.com>
* Add Python 3.10.0rc1 binary wheels
* Drop Python 3.5 artifacts
* Document the drop of 3.5
* Fix the wrong pip pointer
* Update manylinux2014 to a newer version, remove 3.5 distribtest
* Update manylinux aarch64 to see if the absl error go away
* Use the preferred alias
* Allow different wheel library to produce different tag order
* Remove unused shell var and log produced wheels
* Use copy instead of move
* Make bash happy about the wildcard
* Upgrade the debian image to use 3.5+ Python
* Polish the comments for the Dockerfiles
* Remove Python 2.7 binary wheels
* Make sure pip won't pick-up newer versions
* Attempt to fix the distribtest
* Fallback to virtualenv==16.7.9 for 34 and 35
* Remove 34 from binary wheel distribtests
* build aarch64 version of protoc
* remove csharp artifact x86 build logic for unix systems
* build grpc_csharp_ext artifact for linux aarch64
* refactor platform detection
* add generated dllimports for arm64
* fix native library loading on arm64
* include arm64 artifacts in Grpc.Tools
* add Grpc.Tools codegen support for linux aarch64
* grpc.tools cleanup
David Benjamin
github-actions[bot]
alto-ruby
Sergii Tkachenko
Mickaël Guérin
Richard Belleville
Jan Tattermusch
Alisha Nanda
Yash Tibrewal
Johnny Shields
Esun Kim
apolcyn
Lidi Zheng
Jan Tattermusch
AJ Heller