From 80fec20bc2787dd9554f4e652b17485422f1f692 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Thu, 19 May 2016 11:53:19 -0700 Subject: [PATCH 1/4] Update README.md --- tools/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/README.md b/tools/README.md index cb6c22dd836..d142d4aee28 100644 --- a/tools/README.md +++ b/tools/README.md @@ -1,6 +1,6 @@ buildgen: Template renderer for our build system. -distrib: Scripts to distribute language-specific packages. +distrib: Scripts to distribute language-specific packages and other distribution-related helper scripts. dockerfile: Docker files to test gRPC. From 94a0c92354915a6d9b46c36f2ff5a94a353b56e1 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Thu, 19 May 2016 11:59:14 -0700 Subject: [PATCH 2/4] Update README.md --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 3ee2b9f5caa..3283517df17 100644 --- a/README.md +++ b/README.md @@ -11,10 +11,12 @@ Copyright 2015 Google Inc. You can find more detailed documentation and examples in the [doc](doc) and [examples](examples) directories respectively. -#Installation +#Installation & Testing See [INSTALL](INSTALL.md) for installation instructions for various platforms. +See [tools/run_tests](tools/run_tests) for more guidance on how to run various test suites (e.g. unit tests, interop tests, benchmarks) + #Repository Structure & Status This repository contains source code for gRPC libraries for multiple languages written on top of shared C core library [src/core] (src/core). From 9d410be893d99a695b207e413f17878904334eb5 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Thu, 19 May 2016 12:49:19 -0700 Subject: [PATCH 3/4] provide readme for tools/run_tests/ --- tools/run_tests/README.md | 50 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 tools/run_tests/README.md diff --git a/tools/run_tests/README.md b/tools/run_tests/README.md new file mode 100644 index 00000000000..5589b359257 --- /dev/null +++ b/tools/run_tests/README.md @@ -0,0 +1,50 @@ +#Overview + +This directory contains scripts that facilitate building and running tests. We are using python scripts as entrypoint for our +tests because that gives us the opportunity to run tests using the same commandline regardless of the platform you are using. + +#Unit tests (run_tests.py) + +Builds gRPC in given language and runs unit tests. Use `tools/run_tests/run_tests.py --help` for more help. + +######Example +`tools/run_tests/run_tests.py -l csharp -c dbg` + +######Useful options (among many others) +- `--use_docker` Builds a docker container containing all the prerequisites for given language and runs the tests under that container. +- `--build_only` Only build, do not run the tests. + +#Interop tests (run_interop_tests.py) + +Runs tests for cross-platform/cross-language interoperability. For more details, see [Interop tests descriptions](/doc/interop-test-descriptions.md) +The script is also capable of running interop tests for grpc-java and grpc-go, using sources checked out alongside the ones of the grpc repository. + +######Example +`tools/run_tests/run_interop_tests.py -l csharp -s c++ --use_docker` (run interop tests with C# client and C++ server) + +#Performance benchmarks (run_performance_tests.py) + +Runs predefined benchmark scenarios for given languages. Besides the simple configuration of running all the scenarios locally, +the script also supports orchestrating test runs with client and server running on different machines and uploading the results +to BigQuery. + +######Example +`tools/run_tests/run_peformance_tests.py -l c++ node` + +######Useful options +- `--regex` use regex to select particular scenarios to run. + +#Stress tests (run_stress_tests.py) + +Runs modified interop tests clients and servers under heavy load for an extended period of time to discover potential stability issues. +The tests are internally using Kubernetes to run the client and server on GKE and upload statistics to BigQuery. + +Run `tools/run_tests/run_stress_tests.py --help` for more details. + +#Artifacts & Packages (task_runner.py) + +A generalized framework for running predefined tasks based on their labels. We use this to building binary artifacts & distrib packages and testing them) + +######Example +`tools/run_tests/task_runner.py -f python artifact linux x64` (build tasks with labels `python`, `artifact`, `linux`, and `x64`) + From bf682f46f6557777f5849fd3bb8b7f4aea58a892 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Thu, 19 May 2016 15:13:40 -0700 Subject: [PATCH 4/4] address comments --- tools/run_tests/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/run_tests/README.md b/tools/run_tests/README.md index 5589b359257..dd727f43091 100644 --- a/tools/run_tests/README.md +++ b/tools/run_tests/README.md @@ -39,7 +39,9 @@ to BigQuery. Runs modified interop tests clients and servers under heavy load for an extended period of time to discover potential stability issues. The tests are internally using Kubernetes to run the client and server on GKE and upload statistics to BigQuery. -Run `tools/run_tests/run_stress_tests.py --help` for more details. +`tools/run_tests/stress_test/run_on_gke.py --gcp_project_id= --config_file=` + +The directory `tools/run_tests/stress_test/configs/` contains the config files for several scenarios #Artifacts & Packages (task_runner.py)