ad2ca50712
Simplify building release packages through per-language grpc_distribtests_* jobs. ( #29699 )
...
* add release version of grpc_distribtests_* jobs .cfg files
* make grpc_distribtests_ruby more aligned with other single-job distribtests
* dont hide packages from build_artifacts step for python an php
3 years ago
da1977f98c
Cleanup configuration of bazel RBE jobs ( #29301 )
...
* cleanup bazel_rbe .cfg and .sh files
* upload sponge_log.xml artifacts for selected bazel jobs
* use move_src_tree_and_respawn_itself_rc for bazel RBE tests on linux
* fix wrong config
3 years ago
07a75427bd
Simplify running bazel with structured test results on CI ( #29353 )
...
* add bazel_report_helper.py
* simplify bazel invocations in selected builds
* introduce __main__
* update gitignore
* introduce sleep constant
* add type annotations
* use f-strings
* Revert "use f-strings"
This reverts commit f970d6a40b
3 years ago
59d9749c71
fix csharp single-job distribtest on grpc-win2016 ( #29183 )
3 years ago
c8430e6ac4
add grpc_distribtests_python for windows ( #29153 )
3 years ago
409c732813
C# windows single-job distribtest ( #29152 )
...
* add windows grpc_distribtest_csharp
* add kokoro cfg files
3 years ago
24dfe21cad
activate ccache for selected win builds ( #28735 )
3 years ago
8024eed4c5
Print debug info about prepare_build_windows.bat timing ( #28872 )
...
* prepare build windows timing
* C# deps not need to be installed for C# native artifact on win
3 years ago
fea1f3de33
A more reliable way of installing bazel on windows workers ( #28835 )
...
* more reliable bazel installation for win bazel RBE builds
* win RBE build already using the same credentials
* also recognize MSYS_NT returned by kokoro workers
3 years ago
9d73b3e85a
Use ninja for all grpc_csharp_ext builds on windows ( #28841 )
...
* cleanup C# win artifact build
* build C# basictests on win with ninja
* specify default parallelism for C# win artifact
* honor GRPC_PYTHON_BUILD_EXT_COMPILER_JOBS in build_artifact_python.bat
* set --inner_jobs for windows grpc_build_artifact job
* fixup C# build picking env variables from C core build
* get rid of no longer needed NativeDependenciesConfiguration C# setting
3 years ago
e324bf5eee
Speed up windows C/C++ builds by using cmake Ninja generator ( #28833 )
...
* support building C/C++ with ninja
* make cmake_ninja_vs2015 build the default for C/C++
* fix vcvarsall location for vs2017
3 years ago
c0564aa21f
Speedup grpc_portability_build_only windows kokoro job ( #28460 )
...
* only run windows C++ portability build only tests on win
* remove .cfg files for no longer used kokoro windows jobs
3 years ago
7dfdf3d679
Delete no-longer-used kokoro jobs .cfg files. ( #28465 )
...
* remove .cfg files for non-existent kokoro jobs
* more cleanup
3 years ago
c796d5786b
Reintroduce #28396 (Cleanup of windows CI .bat scripts) ( #28457 )
...
* Revert "Revert "Cleanup of windows CI .bat scripts (#28396 )""
This reverts commit 530187f865
3 years ago
a7dcceb365
Revert "Cleanup of windows CI .bat scripts ( #28396 )" ( #28446 )
...
This reverts commit 44e7be448d
3 years ago
44e7be448d
Cleanup of windows CI .bat scripts ( #28396 )
...
* cleanup of windows CI .bat scripts
* fixup
* fixup
* fixup
3 years ago
769369ad3f
fix invocation of respawn scripts on windows ( #28366 )
3 years ago
8afb68238b
Fix windows RBE results uploading to bigquery ( #28365 )
3 years ago
7cfd399dd6
Reduce the workload of presubmit artifact builds and distribtests ( #28228 )
...
* Reduce the workload of presubmit artifact builds and distribtests
* Create and redirect Kokoro config
* Remove duplicated scripts and use env var config
* Clean-up Kokoro cfg
* [DO-NOT-SUBMIT] Add ad-hoc TASK_RUNNER_EXTRA_FILTERS
* Revert "[DO-NOT-SUBMIT] Add ad-hoc TASK_RUNNER_EXTRA_FILTERS"
This reverts commit 9436ac54ac
3 years ago
3a024ea40c
Avoid extremely slow kokoro job finalization by moving workspace to a non-synced directory on kokoro workers. ( #28259 )
...
* support "move and respawn" for kokoro jobs
* delete_nonartifacts.sh no longer needed
* bazel distribtest dont publish any artifacts
* try the altsrc respawn for selected jobs
* use altsrc for all windows CI jobs
* use altsrc for all macos CI jobs
* only move files under src/github
* fix uploading artifacts on windows
3 years ago
63d48ee8d8
Use more parallelism to windows portability tests ( #28179 )
...
* Use more parallelism to windows portability tests
* Added /MP
* Changed it to gRPC_BUILD_MSVC_MP_COUNT
3 years ago
8f50d574fb
Add note about starting port server out of band ( #28012 )
3 years ago
05b695c95a
Start port server separately ( #28005 )
3 years ago
59693b7e72
Migrate Infrastructure Scripts to Python 3 ( #27135 )
...
* Run 2to3 on tools directory
* Delete github_stats_tracking
* Re-run 2to3
* Remove unused script
* Remove unused script
* Remove unused line count utility
* Yapf. Isort
* Remove accidentally included file
* Migrate tools/distrib directory to python 3
* Remove unnecessary shebang
* Restore line_count directory
* Immediately convert subprocess.check_output output to string
* Take care of Python 2 shebangs
* Invoke scripts using a Python 3 interpreter
* Yapf. Isort
* Try installing Python 3 first
* See if we have any Python 3 versions installed
* Add Python 3.7 to Windows path
* Try adding a symlink
* Try to symlink differently
* Install six for Python 3
* Run run_interop_tests with python 3
* Try installing six in python3.7 explicitly
* Revert "Try installing six in python3.7 explicitly"
This reverts commit 2cf60d72f3
3 years ago
99af2bdf3f
Extend the Windows Python test timeout to 3hr ( #27843 )
3 years ago
9b2b9976c6
increase windows C# basictests job timeout ( #27483 )
3 years ago
0042c2b86f
Upgrade bazel to 4.2.1 (LTS), upgrade bazel toolchain to 4.1.0 ( #27410 )
...
* update bazel to 4.2.1
* add bazel 4.2.1 to update_mirror.sh
* regenerate dockerfiles
* upgrade bazel toolchains to 4.1.0
* use create_rbe_exec_properties_dict instead of the deprecated create_exec_properties_dict
* RBE ubsan doesnt require the crosstool_top hack anymore
* update rules_apple and build_bazel_apple_support
3 years ago
c4aa6af0a0
Increase timeout for windows portability test (on master) ( #27210 )
...
* remove no longer use kokoro job config
* increase timeout of windows grpc_portability job
3 years ago
f6bcb40b7f
Use T drive for windows bazel RBE build ( #26895 )
3 years ago
b516dbfa21
Increase artifact build timeouts for problem targets ( #25695 )
...
Going back 2 weeks, nightly packages have failed to build due to
artifact build timeouts. This change increases timeouts for:
Python Linux Extra: 5hrs -> 7hrs
Python Manylinux: 1hr -> 2hrs
CSharp iOS: 45min -> 1hr
CSharp MacOS x64: 30m -> 45m
This is a stopgap measure to get nightly packages deployed more regularly;
March has seen 3 nightlies published in 11 days.
I've cherry picked some of the artifact build timeout errors below.
2021.03.10
grpc/core/master/linux/grpc_build_artifacts_extra_release
2021-03-10 05:03:40,560 TIMEOUT: build_artifact.python_linux_extra_armv7_3.6 [pid=7384, time=18000.4sec]
2021-03-10 05:03:40,646 TIMEOUT: build_artifact.python_linux_extra_armv7_2.7 [pid=7375, time=18000.5sec]
2021-03-10 05:03:40,728 TIMEOUT: build_artifact.python_linux_extra_armv7_3.5 [pid=7378, time=18000.6sec]
2021.03.08
grpc/core/master/linux/grpc_build_artifacts_extra_release
2021-03-08 05:03:18,542 TIMEOUT: build_artifact.python_linux_extra_armv7_3.6 [pid=6853, time=18000.5sec]
2021-03-08 05:03:18,639 TIMEOUT: build_artifact.python_linux_extra_armv7_2.7 [pid=6844, time=18000.6sec]
2021-03-08 05:03:18,741 TIMEOUT: build_artifact.python_linux_extra_armv7_3.5 [pid=6847, time=18000.7sec]
2021.03.07
grpc/core/master/macos/grpc_build_artifacts:
2021-03-07 00:58:02,492 TIMEOUT: build_artifact.csharp_ext_macos_x64 [pid=58673, time=1805.5sec]
2021-03-07 01:13:03,065 TIMEOUT: build_artifact.csharp_ext_macos_ios [pid=58672, time=2706.1sec]
Bumping iOS again from https://github.com/grpc/grpc/pull/25049
grpc/core/master/linux/grpc_build_artifacts_extra_release
2021-03-07 05:03:36,061 TIMEOUT: build_artifact.python_linux_extra_armv7_3.6 [pid=8867, time=18002.9sec]
2021.03.06
grpc/core/master/linux/grpc_build_artifacts_extra_release
2021-03-06 05:03:12,474 TIMEOUT: build_artifact.python_linux_extra_armv7_3.5 [pid=8307, time=18000.6sec]
2021.03.05
grpc/core/master/linux/grpc_build_artifacts_extra_release
2021-03-05 05:03:48,611 TIMEOUT: build_artifact.python_linux_extra_armv7_3.6 [pid=7075, time=18008.6sec]
2021.02.27
grpc/core/master/linux/grpc_build_artifacts
2021-02-27 01:09:57,943 TIMEOUT: build_artifact.python_manylinux2010_x64_cp27-cp27mu [pid=21983, time=3608.9sec]
2021.02.23
grpc/core/master/windows/grpc_build_artifacts
```
2021-02-23 01:48:23,729 SUCCESS: All targets built successfully.
T:\src\github\grpc>set RUNTESTS_EXITCODE=0
T:\src\github\grpc>bash tools/internal_ci/helper_scripts/delete_nonartifacts.sh
++ dirname tools/internal_ci/helper_scripts/delete_nonartifacts.sh
+ cd tools/internal_ci/helper_scripts/../../..
+ find . -type f -not -iname '*sponge_log.*' -not -path './reports/*' -not -path './artifacts/*' -not -path './tools/internal_ci/*' -exec rm -f '{}' +
ERROR: Aborting VM command due to timeout of 10800 seconds
```
4 years ago
bf21b2b3a0
update bazel version in other places too
4 years ago
16157cebe9
Hopefully fixed 27 build on winserver2016
...
* Download the working msys64 and use it to compile 27
* Remove the cygwin detection override logic
* Only install Python interpreters in jobs running on winserver2016
* Fix the batch script substring check
* Increase the timeout for related jobs
* TIL that there is an allow list for Kokoro env vars
4 years ago
511af22c78
Hopefully fixed 27 build on winserver2016
...
* Download the working msys64 and use it to compile 27
* Remove the cygwin detection override logic
* Only install Python interpreters in jobs running on winserver2016
* Fix the batch script substring check
* Increase the timeout for related jobs
* TIL that there is an allow list for Kokoro env vars
4 years ago
e58f0129d2
Stop relying on rbe-windows-credentials.json
4 years ago
85d0d548c4
Increase the timeout of Python basic tests on Windows
4 years ago
858797ae77
disable grpc_build_packages.bat
4 years ago
a0a69de4df
Improve Win CI script with fail fast
4 years ago
424d33415c
upgrade bazel to 2.2.0
4 years ago
db4ce94ebb
Revert "Revert "Release Python3.8 wheels for Windows""
5 years ago
42039d09b7
Revert "Release Python3.8 wheels for Windows"
5 years ago
ed054d03a1
Fix the hack, hope the test still passes
5 years ago
6b92b4af05
Attempt to drop support for Python 3.4
5 years ago
e5e3f70a1a
Revert "Revert "Upgrade to bazel 1.0.0""
5 years ago
0765cf940b
Refactor the way AsyncIO is tested under run_tests.py
5 years ago
b231e7f88f
Add three more Kokoro job for Linux, macOS and Windows
5 years ago
79cfdb525c
increase timeout of bazel RBE jobs
...
if the kokoro job times out and the bazel command is not done, it ends up in
weird test results ("Unevaluated targets") that are hard to interpret.
Setting the timeout for kokoro jobs high enough to create extra reserve
should help.
5 years ago
04d7e0d11f
Revert "Upgrade to bazel 1.0.0"
5 years ago
27f1a94aa5
upgrade to bazel 1.0.0
5 years ago
d10fb3ba99
fix win RBE opt build
5 years ago
17fc384b1e
add job config for Windows RBE opt config, upload test results
5 years ago
Jan Tattermusch
Jan Tattermusch
Craig Tiller
Lidi Zheng
Esun Kim
Richard Belleville
AJ Heller
Lidi Zheng
Nicolas Noble