apolcyn
3c368a5b2c
Don't gem install rake-compiler on macos setup scripts ( #28415 )
...
* Don't gem install rake-compiler on macos setup scripts
3 years ago
Jan Tattermusch
858eff5ab9
Introduce C# single-platform nuget build (and distribtests) ( #27180 )
...
* C# linux distribtests
* add distribtests C# linux job cfg
* support running a subset of C# artifact/distribtests on jobs
3 years ago
Lidi Zheng
7cfd399dd6
Reduce the workload of presubmit artifact builds and distribtests ( #28228 )
...
* Reduce the workload of presubmit artifact builds and distribtests
* Create and redirect Kokoro config
* Remove duplicated scripts and use env var config
* Clean-up Kokoro cfg
* [DO-NOT-SUBMIT] Add ad-hoc TASK_RUNNER_EXTRA_FILTERS
* Revert "[DO-NOT-SUBMIT] Add ad-hoc TASK_RUNNER_EXTRA_FILTERS"
This reverts commit 9436ac54ac
.
3 years ago
Jan Tattermusch
ae65bb8f7f
windows: speedup build of C# extension and protoc artifacts ( #28225 )
3 years ago
Richard Belleville
59693b7e72
Migrate Infrastructure Scripts to Python 3 ( #27135 )
...
* Run 2to3 on tools directory
* Delete github_stats_tracking
* Re-run 2to3
* Remove unused script
* Remove unused script
* Remove unused line count utility
* Yapf. Isort
* Remove accidentally included file
* Migrate tools/distrib directory to python 3
* Remove unnecessary shebang
* Restore line_count directory
* Immediately convert subprocess.check_output output to string
* Take care of Python 2 shebangs
* Invoke scripts using a Python 3 interpreter
* Yapf. Isort
* Try installing Python 3 first
* See if we have any Python 3 versions installed
* Add Python 3.7 to Windows path
* Try adding a symlink
* Try to symlink differently
* Install six for Python 3
* Run run_interop_tests with python 3
* Try installing six in python3.7 explicitly
* Revert "Try installing six in python3.7 explicitly"
This reverts commit 2cf60d72f3
.
* And debug some more
* Fix issue with jobset.py
* Add debug for CI failure
* Revert microbenchmark changes
3 years ago
Lidi Zheng
9978223a26
Add python_requires field to grpcio-* packages ( #27495 )
...
- Update the run_tests.py and distrib tests
- Update the Dockerfiles
3 years ago
Jan Tattermusch
e11dcbc3e4
Include "linux_extra" python artifacts in regular linux build ( #27279 )
3 years ago
Jan Tattermusch
82d8bb089d
Use manylinux_2_17 instead of manylinux_2_24 tag for manylinux2014 aarch64 wheels ( #27280 )
...
* Let's see if the aarch64 manylinux bug is fixed or not
* rename aarch64 image back to manylinux2014 for consistency
* fix C# aarch64 artifact build
Co-authored-by: Lidi Zheng <lidiz@google.com>
3 years ago
Jan Tattermusch
782734a82f
python: move native debug symbol stripping to build_artifact phase ( #26194 )
3 years ago
Jan Tattermusch
fd0a4624c9
increase timeout for building mac ruby artifact ( #27230 )
3 years ago
Lidi Zheng
4729f6fdcc
Add Python 3.10 drop 3.5 ( #26074 )
...
* Add Python 3.10.0rc1 binary wheels
* Drop Python 3.5 artifacts
* Document the drop of 3.5
* Fix the wrong pip pointer
* Update manylinux2014 to a newer version, remove 3.5 distribtest
* Update manylinux aarch64 to see if the absl error go away
* Use the preferred alias
* Allow different wheel library to produce different tag order
* Remove unused shell var and log produced wheels
* Use copy instead of move
* Make bash happy about the wildcard
* Upgrade the debian image to use 3.5+ Python
* Polish the comments for the Dockerfiles
3 years ago
Jan Tattermusch
ce5b4e949f
Faster and streamlined PHP distribtests (for linux and macos) ( #27101 )
...
* faster and streamlined PHP distribtests
* add kokoro job for PHP distribtests
* php distribtests job for macos
* add trailing newline
3 years ago
Richard Belleville
b6cc72f129
Revert "Run 2to3 on tools directory ( #26002 )" ( #27183 )
...
This reverts commit a388361c3d
.
3 years ago
Richard Belleville
a388361c3d
Run 2to3 on tools directory ( #26002 )
...
* Run 2to3 on tools directory
* Delete github_stats_tracking
* Re-run 2to3
* Remove unused script
* Remove unused script
* Remove unused line count utility
* Yapf. Isort
* Remove accidentally included file
* Restore line_count directory
3 years ago
Jan Tattermusch
7e532fd198
add streamlined and faster ruby distribtest ( #27124 )
3 years ago
Lidi Zheng
5820e152cd
Remove Python 2.7 binary wheel generations ( #26691 )
...
* Remove Python 2.7 binary wheels
* Make sure pip won't pick-up newer versions
* Attempt to fix the distribtest
* Fallback to virtualenv==16.7.9 for 34 and 35
* Remove 34 from binary wheel distribtests
3 years ago
Jan Tattermusch
288fe131b4
Reintroduce #22964 with extra fixes ( #26465 )
...
* Revert "Revert "Upgrade dotnet SDK 3 for C# build (#22964 )" (#26462 )"
This reverts commit 736d914cfd
.
* Revert "Exclude csharp from performance test continuous runs. (#26458 )"
This reverts commit 6744e8f84a
.
* switch remaining tests to C# buster image
* remove C#'s global.json (which requires dotnet SDK 3+)
* dotnet SDK3 currently breaks C# package build
* regenerate dockerfiles
4 years ago
Jan Tattermusch
156edd687e
set xds user agent name everywhere where grpc_csharp_ext is built ( #26337 )
...
* set xds user agent name everywhere where grpc_csharp_ext is built
* regenerate projects
4 years ago
Jan Tattermusch
3379d998b8
remove unnecessary comments left by #26430 ( #26435 )
4 years ago
Jan Tattermusch
4fb0ccecf1
Workaround #26279 by publishing manylinux_2_24 wheels instead of manylinux2014 on aarch64 ( #26430 )
...
* workaround #26279 at the expense of binary compatibility
* correctly tag aarch64 wheels as manylinux_2_24
4 years ago
Jan Tattermusch
e38e59320c
better way of building protoc aarch64 artifacts ( #26416 )
4 years ago
AJ Heller
24e9a6978b
Increase artifact build timeout for windows-C# ( #26179 )
...
The 30min timeout has caused the multiarch artifact build to fail twice
in the past week. See
http://sponge2/55f7f6f8-d3c0-4b6e-91c8-c0c316f4717b
4 years ago
Lidi Zheng
25f5399b41
Add grpcio-admin Python package ( #26166 )
...
* Add grpcio-admin Python package
* Polish package content
* Make Linux artifact build happy
* Improve documentation
4 years ago
Lidi Zheng
dc63d6a53e
Add CSDS API to Python ( #26114 )
...
* Add grpcio-csds pacakge
* Remove unused file
* Fix the proto import path issue
* Update the CSDS package and xds-protos for PY2
* Make tests happy
* Fix Bazel proto dependency
* Add Python2 tests for CSDS
4 years ago
Jan Tattermusch
d61db49b48
Add python3.8 aarch64 wheel distribtest ( #26136 )
...
* add python3.8 aarch64 distribtest
* register qemu emulator for linux distribtests
4 years ago
Jan Tattermusch
98339824b0
also build python3.6 aarch64 manylinux2014 wheel ( #25928 )
4 years ago
Jan Tattermusch
a737ff8612
Reduce Grpc.Core nuget size by generating separate .so with/without debug symbols for grpc_csharp_ext ( #25729 )
...
* generate separate .so with/without debug symbols for grpc_csharp_ext
* add linux .dbginfo.so libraries to Grpc.Core.NativeDebug package
* fix aarch64 grpc_csharp_ext stripping
* lots of linux artifact build jobs are currently timing out
4 years ago
Jan Tattermusch
0088dae36a
Use crosscompilation to build python armv7 wheels ( #25704 )
...
* removed unused DOCKER_BASE_IMAGE functionality
* remove legacy docker images for arm build
* build python armv7 wheel via crosscompilation
* add pyconfig.h hack
* improve the dockerfile
* yapf format code
4 years ago
Jan Tattermusch
21c83cbb08
make Grpc C# work on aarch64 linux ( #25717 )
...
* build aarch64 version of protoc
* remove csharp artifact x86 build logic for unix systems
* build grpc_csharp_ext artifact for linux aarch64
* refactor platform detection
* add generated dllimports for arm64
* fix native library loading on arm64
* include arm64 artifacts in Grpc.Tools
* add Grpc.Tools codegen support for linux aarch64
* grpc.tools cleanup
4 years ago
AJ Heller
b516dbfa21
Increase artifact build timeouts for problem targets ( #25695 )
...
Going back 2 weeks, nightly packages have failed to build due to
artifact build timeouts. This change increases timeouts for:
Python Linux Extra: 5hrs -> 7hrs
Python Manylinux: 1hr -> 2hrs
CSharp iOS: 45min -> 1hr
CSharp MacOS x64: 30m -> 45m
This is a stopgap measure to get nightly packages deployed more regularly;
March has seen 3 nightlies published in 11 days.
I've cherry picked some of the artifact build timeout errors below.
2021.03.10
grpc/core/master/linux/grpc_build_artifacts_extra_release
2021-03-10 05:03:40,560 TIMEOUT: build_artifact.python_linux_extra_armv7_3.6 [pid=7384, time=18000.4sec]
2021-03-10 05:03:40,646 TIMEOUT: build_artifact.python_linux_extra_armv7_2.7 [pid=7375, time=18000.5sec]
2021-03-10 05:03:40,728 TIMEOUT: build_artifact.python_linux_extra_armv7_3.5 [pid=7378, time=18000.6sec]
2021.03.08
grpc/core/master/linux/grpc_build_artifacts_extra_release
2021-03-08 05:03:18,542 TIMEOUT: build_artifact.python_linux_extra_armv7_3.6 [pid=6853, time=18000.5sec]
2021-03-08 05:03:18,639 TIMEOUT: build_artifact.python_linux_extra_armv7_2.7 [pid=6844, time=18000.6sec]
2021-03-08 05:03:18,741 TIMEOUT: build_artifact.python_linux_extra_armv7_3.5 [pid=6847, time=18000.7sec]
2021.03.07
grpc/core/master/macos/grpc_build_artifacts:
2021-03-07 00:58:02,492 TIMEOUT: build_artifact.csharp_ext_macos_x64 [pid=58673, time=1805.5sec]
2021-03-07 01:13:03,065 TIMEOUT: build_artifact.csharp_ext_macos_ios [pid=58672, time=2706.1sec]
Bumping iOS again from https://github.com/grpc/grpc/pull/25049
grpc/core/master/linux/grpc_build_artifacts_extra_release
2021-03-07 05:03:36,061 TIMEOUT: build_artifact.python_linux_extra_armv7_3.6 [pid=8867, time=18002.9sec]
2021.03.06
grpc/core/master/linux/grpc_build_artifacts_extra_release
2021-03-06 05:03:12,474 TIMEOUT: build_artifact.python_linux_extra_armv7_3.5 [pid=8307, time=18000.6sec]
2021.03.05
grpc/core/master/linux/grpc_build_artifacts_extra_release
2021-03-05 05:03:48,611 TIMEOUT: build_artifact.python_linux_extra_armv7_3.6 [pid=7075, time=18008.6sec]
2021.02.27
grpc/core/master/linux/grpc_build_artifacts
2021-02-27 01:09:57,943 TIMEOUT: build_artifact.python_manylinux2010_x64_cp27-cp27mu [pid=21983, time=3608.9sec]
2021.02.23
grpc/core/master/windows/grpc_build_artifacts
```
2021-02-23 01:48:23,729 SUCCESS: All targets built successfully.
T:\src\github\grpc>set RUNTESTS_EXITCODE=0
T:\src\github\grpc>bash tools/internal_ci/helper_scripts/delete_nonartifacts.sh
++ dirname tools/internal_ci/helper_scripts/delete_nonartifacts.sh
+ cd tools/internal_ci/helper_scripts/../../..
+ find . -type f -not -iname '*sponge_log.*' -not -path './reports/*' -not -path './artifacts/*' -not -path './tools/internal_ci/*' -exec rm -f '{}' +
ERROR: Aborting VM command due to timeout of 10800 seconds
```
4 years ago
Jan Tattermusch
1dce57f35f
Assorted python wheel build improvements ( #25602 )
...
* cleanup unnecessary deps from grpc_artifact_python_manylinux2014_aarch64
* cleanup in grpc_artifact_python dockerfiles
* enable boringssl assembly optimizations when crosscompiling aarch64 linux wheels
* pin manylinux2010 images to keep python27 build
* avoid pip install --upgrade cython when not necessary
4 years ago
Jan Tattermusch
cb8114d689
Delete no longer needed artifact_linux dockerfiles ( #25542 )
...
* delete no longer needed artifact_linux dockerfiles
* fix build_package_python.sh
4 years ago
Esun Kim
160cb1b215
Changed RPI build test to general ARM build test ( #25580 )
...
* Change rpi build test to arm test
* Having a separate docker and targetting aarch64
4 years ago
Jan Tattermusch
fcd43e9030
Crosscompile python aarch64 wheels with dockcross ( #25418 )
...
* build aarch64 python wheels via crosscompilation
* yapf format code
* fix shellcheck complaints
* fix python37 aarch64 wheel build
* build python wheels on linux aarch64 with static libstdc++
* yapf format code
4 years ago
Yash Tibrewal
7a5288d2a0
Replace disrespectful words
4 years ago
Lidi Zheng
b62c984eab
Pin pip to 19.3.1 to fix the build failure on master
4 years ago
Alexander Polcyn
b94f09aaf4
Drop support for ruby 2.3
4 years ago
Sergii Tkachenko
8db79e2e71
Python style: reformat code using updated yapf 0.30.0
4 years ago
Alexander Polcyn
4fa05591e3
skil ruby 3.0 distrib test until protobuf adds support
4 years ago
Alexander Polcyn
f5fae54331
Remove no longer supported centos6 distrib test
4 years ago
Alexander Polcyn
039936387e
Add a ruby 3.0.0 distrib test
4 years ago
Esun Kim
5383d368e0
Increase timeout of build_artifact_csharp_ios to 45 mins
4 years ago
Esun Kim
a0a11bfc28
Increase ruby build timeout to 60min
4 years ago
Jan Tattermusch
a35cd4f89d
build protoc artifacts with cmake
4 years ago
Esun Kim
76afc77ff6
Update PHP & Ruby Mac BUILD
4 years ago
Jan Tattermusch
db521f52f9
fixup shellcheck problem
4 years ago
Jan Tattermusch
94ccbef5ea
skip packaging macos x86 protoc in grpc-tools gem
4 years ago
Jan Tattermusch
f46bd7f80d
stop building protoc x86 artifacts on macos
4 years ago
Lidi Zheng
16157cebe9
Hopefully fixed 27 build on winserver2016
...
* Download the working msys64 and use it to compile 27
* Remove the cygwin detection override logic
* Only install Python interpreters in jobs running on winserver2016
* Fix the batch script substring check
* Increase the timeout for related jobs
* TIL that there is an allow list for Kokoro env vars
4 years ago
Lidi Zheng
511af22c78
Hopefully fixed 27 build on winserver2016
...
* Download the working msys64 and use it to compile 27
* Remove the cygwin detection override logic
* Only install Python interpreters in jobs running on winserver2016
* Fix the batch script substring check
* Increase the timeout for related jobs
* TIL that there is an allow list for Kokoro env vars
4 years ago