7cfd399dd6
Reduce the workload of presubmit artifact builds and distribtests ( #28228 )
...
* Reduce the workload of presubmit artifact builds and distribtests
* Create and redirect Kokoro config
* Remove duplicated scripts and use env var config
* Clean-up Kokoro cfg
* [DO-NOT-SUBMIT] Add ad-hoc TASK_RUNNER_EXTRA_FILTERS
* Revert "[DO-NOT-SUBMIT] Add ad-hoc TASK_RUNNER_EXTRA_FILTERS"
This reverts commit 9436ac54ac
3 years ago
59693b7e72
Migrate Infrastructure Scripts to Python 3 ( #27135 )
...
* Run 2to3 on tools directory
* Delete github_stats_tracking
* Re-run 2to3
* Remove unused script
* Remove unused script
* Remove unused line count utility
* Yapf. Isort
* Remove accidentally included file
* Migrate tools/distrib directory to python 3
* Remove unnecessary shebang
* Restore line_count directory
* Immediately convert subprocess.check_output output to string
* Take care of Python 2 shebangs
* Invoke scripts using a Python 3 interpreter
* Yapf. Isort
* Try installing Python 3 first
* See if we have any Python 3 versions installed
* Add Python 3.7 to Windows path
* Try adding a symlink
* Try to symlink differently
* Install six for Python 3
* Run run_interop_tests with python 3
* Try installing six in python3.7 explicitly
* Revert "Try installing six in python3.7 explicitly"
This reverts commit 2cf60d72f3
3 years ago
e11dcbc3e4
Include "linux_extra" python artifacts in regular linux build ( #27279 )
3 years ago
82d8bb089d
Use manylinux_2_17 instead of manylinux_2_24 tag for manylinux2014 aarch64 wheels ( #27280 )
...
* Let's see if the aarch64 manylinux bug is fixed or not
* rename aarch64 image back to manylinux2014 for consistency
* fix C# aarch64 artifact build
Co-authored-by: Lidi Zheng <lidiz@google.com>
3 years ago
fd0a4624c9
increase timeout for building mac ruby artifact ( #27230 )
3 years ago
4729f6fdcc
Add Python 3.10 drop 3.5 ( #26074 )
...
* Add Python 3.10.0rc1 binary wheels
* Drop Python 3.5 artifacts
* Document the drop of 3.5
* Fix the wrong pip pointer
* Update manylinux2014 to a newer version, remove 3.5 distribtest
* Update manylinux aarch64 to see if the absl error go away
* Use the preferred alias
* Allow different wheel library to produce different tag order
* Remove unused shell var and log produced wheels
* Use copy instead of move
* Make bash happy about the wildcard
* Upgrade the debian image to use 3.5+ Python
* Polish the comments for the Dockerfiles
3 years ago
ce5b4e949f
Faster and streamlined PHP distribtests (for linux and macos) ( #27101 )
...
* faster and streamlined PHP distribtests
* add kokoro job for PHP distribtests
* php distribtests job for macos
* add trailing newline
3 years ago
b6cc72f129
Revert "Run 2to3 on tools directory ( #26002 )" ( #27183 )
...
This reverts commit a388361c3d
3 years ago
a388361c3d
Run 2to3 on tools directory ( #26002 )
...
* Run 2to3 on tools directory
* Delete github_stats_tracking
* Re-run 2to3
* Remove unused script
* Remove unused script
* Remove unused line count utility
* Yapf. Isort
* Remove accidentally included file
* Restore line_count directory
3 years ago
5820e152cd
Remove Python 2.7 binary wheel generations ( #26691 )
...
* Remove Python 2.7 binary wheels
* Make sure pip won't pick-up newer versions
* Attempt to fix the distribtest
* Fallback to virtualenv==16.7.9 for 34 and 35
* Remove 34 from binary wheel distribtests
3 years ago
3379d998b8
remove unnecessary comments left by #26430 ( #26435 )
4 years ago
4fb0ccecf1
Workaround #26279 by publishing manylinux_2_24 wheels instead of manylinux2014 on aarch64 ( #26430 )
...
* workaround #26279 at the expense of binary compatibility
* correctly tag aarch64 wheels as manylinux_2_24
4 years ago
e38e59320c
better way of building protoc aarch64 artifacts ( #26416 )
4 years ago
24e9a6978b
Increase artifact build timeout for windows-C# ( #26179 )
...
The 30min timeout has caused the multiarch artifact build to fail twice
in the past week. See
http://sponge2/55f7f6f8-d3c0-4b6e-91c8-c0c316f4717b
4 years ago
98339824b0
also build python3.6 aarch64 manylinux2014 wheel ( #25928 )
4 years ago
0088dae36a
Use crosscompilation to build python armv7 wheels ( #25704 )
...
* removed unused DOCKER_BASE_IMAGE functionality
* remove legacy docker images for arm build
* build python armv7 wheel via crosscompilation
* add pyconfig.h hack
* improve the dockerfile
* yapf format code
4 years ago
21c83cbb08
make Grpc C# work on aarch64 linux ( #25717 )
...
* build aarch64 version of protoc
* remove csharp artifact x86 build logic for unix systems
* build grpc_csharp_ext artifact for linux aarch64
* refactor platform detection
* add generated dllimports for arm64
* fix native library loading on arm64
* include arm64 artifacts in Grpc.Tools
* add Grpc.Tools codegen support for linux aarch64
* grpc.tools cleanup
4 years ago
b516dbfa21
Increase artifact build timeouts for problem targets ( #25695 )
...
Going back 2 weeks, nightly packages have failed to build due to
artifact build timeouts. This change increases timeouts for:
Python Linux Extra: 5hrs -> 7hrs
Python Manylinux: 1hr -> 2hrs
CSharp iOS: 45min -> 1hr
CSharp MacOS x64: 30m -> 45m
This is a stopgap measure to get nightly packages deployed more regularly;
March has seen 3 nightlies published in 11 days.
I've cherry picked some of the artifact build timeout errors below.
2021.03.10
grpc/core/master/linux/grpc_build_artifacts_extra_release
2021-03-10 05:03:40,560 TIMEOUT: build_artifact.python_linux_extra_armv7_3.6 [pid=7384, time=18000.4sec]
2021-03-10 05:03:40,646 TIMEOUT: build_artifact.python_linux_extra_armv7_2.7 [pid=7375, time=18000.5sec]
2021-03-10 05:03:40,728 TIMEOUT: build_artifact.python_linux_extra_armv7_3.5 [pid=7378, time=18000.6sec]
2021.03.08
grpc/core/master/linux/grpc_build_artifacts_extra_release
2021-03-08 05:03:18,542 TIMEOUT: build_artifact.python_linux_extra_armv7_3.6 [pid=6853, time=18000.5sec]
2021-03-08 05:03:18,639 TIMEOUT: build_artifact.python_linux_extra_armv7_2.7 [pid=6844, time=18000.6sec]
2021-03-08 05:03:18,741 TIMEOUT: build_artifact.python_linux_extra_armv7_3.5 [pid=6847, time=18000.7sec]
2021.03.07
grpc/core/master/macos/grpc_build_artifacts:
2021-03-07 00:58:02,492 TIMEOUT: build_artifact.csharp_ext_macos_x64 [pid=58673, time=1805.5sec]
2021-03-07 01:13:03,065 TIMEOUT: build_artifact.csharp_ext_macos_ios [pid=58672, time=2706.1sec]
Bumping iOS again from https://github.com/grpc/grpc/pull/25049
grpc/core/master/linux/grpc_build_artifacts_extra_release
2021-03-07 05:03:36,061 TIMEOUT: build_artifact.python_linux_extra_armv7_3.6 [pid=8867, time=18002.9sec]
2021.03.06
grpc/core/master/linux/grpc_build_artifacts_extra_release
2021-03-06 05:03:12,474 TIMEOUT: build_artifact.python_linux_extra_armv7_3.5 [pid=8307, time=18000.6sec]
2021.03.05
grpc/core/master/linux/grpc_build_artifacts_extra_release
2021-03-05 05:03:48,611 TIMEOUT: build_artifact.python_linux_extra_armv7_3.6 [pid=7075, time=18008.6sec]
2021.02.27
grpc/core/master/linux/grpc_build_artifacts
2021-02-27 01:09:57,943 TIMEOUT: build_artifact.python_manylinux2010_x64_cp27-cp27mu [pid=21983, time=3608.9sec]
2021.02.23
grpc/core/master/windows/grpc_build_artifacts
```
2021-02-23 01:48:23,729 SUCCESS: All targets built successfully.
T:\src\github\grpc>set RUNTESTS_EXITCODE=0
T:\src\github\grpc>bash tools/internal_ci/helper_scripts/delete_nonartifacts.sh
++ dirname tools/internal_ci/helper_scripts/delete_nonartifacts.sh
+ cd tools/internal_ci/helper_scripts/../../..
+ find . -type f -not -iname '*sponge_log.*' -not -path './reports/*' -not -path './artifacts/*' -not -path './tools/internal_ci/*' -exec rm -f '{}' +
ERROR: Aborting VM command due to timeout of 10800 seconds
```
4 years ago
1dce57f35f
Assorted python wheel build improvements ( #25602 )
...
* cleanup unnecessary deps from grpc_artifact_python_manylinux2014_aarch64
* cleanup in grpc_artifact_python dockerfiles
* enable boringssl assembly optimizations when crosscompiling aarch64 linux wheels
* pin manylinux2010 images to keep python27 build
* avoid pip install --upgrade cython when not necessary
4 years ago
fcd43e9030
Crosscompile python aarch64 wheels with dockcross ( #25418 )
...
* build aarch64 python wheels via crosscompilation
* yapf format code
* fix shellcheck complaints
* fix python37 aarch64 wheel build
* build python wheels on linux aarch64 with static libstdc++
* yapf format code
4 years ago
7a5288d2a0
Replace disrespectful words
4 years ago
5383d368e0
Increase timeout of build_artifact_csharp_ios to 45 mins
4 years ago
a0a11bfc28
Increase ruby build timeout to 60min
4 years ago
a35cd4f89d
build protoc artifacts with cmake
4 years ago
76afc77ff6
Update PHP & Ruby Mac BUILD
4 years ago
f46bd7f80d
stop building protoc x86 artifacts on macos
4 years ago
16157cebe9
Hopefully fixed 27 build on winserver2016
...
* Download the working msys64 and use it to compile 27
* Remove the cygwin detection override logic
* Only install Python interpreters in jobs running on winserver2016
* Fix the batch script substring check
* Increase the timeout for related jobs
* TIL that there is an allow list for Kokoro env vars
4 years ago
511af22c78
Hopefully fixed 27 build on winserver2016
...
* Download the working msys64 and use it to compile 27
* Remove the cygwin detection override logic
* Only install Python interpreters in jobs running on winserver2016
* Fix the batch script substring check
* Increase the timeout for related jobs
* TIL that there is an allow list for Kokoro env vars
4 years ago
e7f5fe1605
dont include x86 grpc_csharp_ext for linux and mac
4 years ago
37e38b6448
Build Python 3.9 artifacts for Linux x64 platforms
4 years ago
63180d9cc8
Build Python 3.9 artifacts for Linux x64 platforms
4 years ago
b63f5177e8
Remove 3.9 Windows support
4 years ago
62e9821515
Support pre-compiled Python 3.9 wheels
4 years ago
a910dc1802
Remove 3.9 Windows support
4 years ago
4fa90f42ce
Support pre-compiled Python 3.9 wheels
4 years ago
cf9be63c49
Update PHP artifact test
4 years ago
40dbe0e28a
Removed grpc_artifact_protoc/Dockerfile
4 years ago
445f6814eb
Add manylinux2014 artifacts
5 years ago
be1bc912fd
Remove docker base image for manylinux
5 years ago
23e1d30327
Removes manylinux1-specific instructions
5 years ago
c4a8d65512
Remove yet more targets
5 years ago
87c7c2af3f
Stop building manylinux1 images
5 years ago
7be821ac47
Yapf all target python sources
5 years ago
f00bdb3d90
Change the linux for linux artifacts to CentOS 6
5 years ago
db4ce94ebb
Revert "Revert "Release Python3.8 wheels for Windows""
5 years ago
42039d09b7
Revert "Release Python3.8 wheels for Windows"
5 years ago
ed054d03a1
Fix the hack, hope the test still passes
5 years ago
70232fbd9c
Change the msvc back
5 years ago
6dc57b3b0c
Let's see CI is happy with additional changes
5 years ago
95528207ca
Add Python3.8 to Kokoro windows
5 years ago
Lidi Zheng
Richard Belleville
Jan Tattermusch
AJ Heller
Yash Tibrewal
Esun Kim
Jan Tattermusch
Stanley Cheung
Lidi Zheng