grpc/tools/run_tests/README.md

# Overview

This directory contains scripts that facilitate building and running tests. We are using python scripts as entrypoint for our
tests because that gives us the opportunity to run tests using the same commandline regardless of the platform you are using.

# Unit tests (run_tests.py)

Builds gRPC in given language and runs unit tests. Use `tools/run_tests/run_tests.py --help` for more help.

###### Example
`tools/run_tests/run_tests.py -l python -c dbg`

###### Useful options (among many others)
- `--use_docker` Builds a docker container containing all the prerequisites for given language and runs the tests under that container.
- `--build_only` Only build, do not run the tests.

Note: If you get an error such as `ImportError: No module named httplib2`, then you may be missing some Python modules. Install the module listed in the error and try again. 

Note: some tests may be flaky. Check the "Issues" tab for known flakes and other issues.

The full suite of unit tests will take many minutes to run.

# Interop tests (run_interop_tests.py)

Runs tests for cross-platform/cross-language interoperability. For more details, see [Interop tests descriptions](/doc/interop-test-descriptions.md)
The script is also capable of running interop tests for grpc-java and grpc-go, using sources checked out alongside the ones of the grpc repository.

###### Example
`tools/run_tests/run_interop_tests.py -l python -s c++ --use_docker` (run interop tests with C# client and C++ server)

Note: if you see an error like `no space left on device` when running the
interop tests using Docker, make sure that Docker is building the image files in
a location with sufficient disk space.

# Performance benchmarks (run_performance_tests.py)

The `run_performance_tests.py` script is deprecated.
See [end-to-end benchmarking documentation](/tools/run_tests/peformance/README.md) for more info.

# Artifacts & Packages (task_runner.py)

A generalized framework for running predefined tasks based on their labels. We use this to building binary artifacts & distrib packages and testing them)

###### Example
`tools/run_tests/task_runner.py -f python artifact linux x64` (build tasks with labels `python`, `artifact`, `linux`, and `x64`)
fix markdown 8 years ago			`# Overview`
provide readme for tools/run_tests/ 9 years ago
			`This directory contains scripts that facilitate building and running tests. We are using python scripts as entrypoint for our`
			`tests because that gives us the opportunity to run tests using the same commandline regardless of the platform you are using.`

fix markdown 8 years ago			`# Unit tests (run_tests.py)`
provide readme for tools/run_tests/ 9 years ago
			Builds gRPC in given language and runs unit tests. Use `tools/run_tests/run_tests.py --help` for more help.

fix markdown 8 years ago			`###### Example`
Remove C# implementation (individual packages will continue to be maintained through v2.46.x patches or moved to grpc-dotnet) (#29225) * remove examples/csharp * remove reference to examples/csharp * remove C# references from doc/ * remove src/csharp * update test_creds locations * remove C# distribtests * remove buildgen templates for files under src/csharp * remove grpc_csharp_ext C build target * regenerate projects * update tools/run_tests/README.md * change many C# test/build scripts to noop * unbreak run_tests.py -l csharp * unbreak C# build_package targets * unbreak C# distribtests * remove C# pre-build steps from prepare_build_windows.bat * remove src/csharp from clang_format * remove csharp and csharpcoreclr from run_interop_tests.py * remove C# interop test docker images * Revert "remove src/csharp" This reverts commit 97f2b6d707cc1140788b3618f9ab3de36a3dc2fb. * remove src/csharp, but keep Grpc.Tools * add back templates/src/csharp/build/dependencies.props.template * remove everything but Grpc.Tools and Grpc.Tools.Tests from sln * Revert "unbreak run_tests.py -l csharp" This reverts commit 529b5e2ec19532038b41e935ea9f4f015ac60881. * revive Grpc.Tools.Tests through run_tests.py * Revert "unbreak C# build_package targets" This reverts commit 79d8d83c8bfacdca5c41fa4b33a97751167c7553. * experiment with build_nuget.sh fixes * simplify create_fake_native_artifacts.sh * building of Grpc.Tools in package_targets.py * move shared files from Grpc.Core to Grpc.Tools * yapf format: distribtest_targets.py * Revert "remove C# pre-build steps from prepare_build_windows.bat" This reverts commit bd38956a2b3a670fc51edb0de08aed48e9c885b6. * update C# README.md * disable C# in gke OSS benchmarks 3 years ago			`tools/run_tests/run_tests.py -l python -c dbg`
provide readme for tools/run_tests/ 9 years ago
fix markdown 8 years ago			`###### Useful options (among many others)`
provide readme for tools/run_tests/ 9 years ago			- `--use_docker` Builds a docker container containing all the prerequisites for given language and runs the tests under that container.
			- `--build_only` Only build, do not run the tests.

Update the run tests instructions to clarify a python module error 7 years ago			Note: If you get an error such as `ImportError: No module named httplib2`, then you may be missing some Python modules. Install the module listed in the error and try again.

Update unit testing documentation 7 years ago			`Note: some tests may be flaky. Check the "Issues" tab for known flakes and other issues.`

			`The full suite of unit tests will take many minutes to run.`

fix markdown 8 years ago			`# Interop tests (run_interop_tests.py)`
provide readme for tools/run_tests/ 9 years ago
			`Runs tests for cross-platform/cross-language interoperability. For more details, see [Interop tests descriptions](/doc/interop-test-descriptions.md)`
			`The script is also capable of running interop tests for grpc-java and grpc-go, using sources checked out alongside the ones of the grpc repository.`

fix markdown 8 years ago			`###### Example`
Remove C# implementation (individual packages will continue to be maintained through v2.46.x patches or moved to grpc-dotnet) (#29225) * remove examples/csharp * remove reference to examples/csharp * remove C# references from doc/ * remove src/csharp * update test_creds locations * remove C# distribtests * remove buildgen templates for files under src/csharp * remove grpc_csharp_ext C build target * regenerate projects * update tools/run_tests/README.md * change many C# test/build scripts to noop * unbreak run_tests.py -l csharp * unbreak C# build_package targets * unbreak C# distribtests * remove C# pre-build steps from prepare_build_windows.bat * remove src/csharp from clang_format * remove csharp and csharpcoreclr from run_interop_tests.py * remove C# interop test docker images * Revert "remove src/csharp" This reverts commit 97f2b6d707cc1140788b3618f9ab3de36a3dc2fb. * remove src/csharp, but keep Grpc.Tools * add back templates/src/csharp/build/dependencies.props.template * remove everything but Grpc.Tools and Grpc.Tools.Tests from sln * Revert "unbreak run_tests.py -l csharp" This reverts commit 529b5e2ec19532038b41e935ea9f4f015ac60881. * revive Grpc.Tools.Tests through run_tests.py * Revert "unbreak C# build_package targets" This reverts commit 79d8d83c8bfacdca5c41fa4b33a97751167c7553. * experiment with build_nuget.sh fixes * simplify create_fake_native_artifacts.sh * building of Grpc.Tools in package_targets.py * move shared files from Grpc.Core to Grpc.Tools * yapf format: distribtest_targets.py * Revert "remove C# pre-build steps from prepare_build_windows.bat" This reverts commit bd38956a2b3a670fc51edb0de08aed48e9c885b6. * update C# README.md * disable C# in gke OSS benchmarks 3 years ago			`tools/run_tests/run_interop_tests.py -l python -s c++ --use_docker` (run interop tests with C# client and C++ server)
provide readme for tools/run_tests/ 9 years ago
Note to help diagnose errors with docker's default location for building images. 8 years ago			Note: if you see an error like `no space left on device` when running the
			`interop tests using Docker, make sure that Docker is building the image files in`
			`a location with sufficient disk space.`

fix markdown 8 years ago			`# Performance benchmarks (run_performance_tests.py)`
provide readme for tools/run_tests/ 9 years ago
Remove C# implementation (individual packages will continue to be maintained through v2.46.x patches or moved to grpc-dotnet) (#29225) * remove examples/csharp * remove reference to examples/csharp * remove C# references from doc/ * remove src/csharp * update test_creds locations * remove C# distribtests * remove buildgen templates for files under src/csharp * remove grpc_csharp_ext C build target * regenerate projects * update tools/run_tests/README.md * change many C# test/build scripts to noop * unbreak run_tests.py -l csharp * unbreak C# build_package targets * unbreak C# distribtests * remove C# pre-build steps from prepare_build_windows.bat * remove src/csharp from clang_format * remove csharp and csharpcoreclr from run_interop_tests.py * remove C# interop test docker images * Revert "remove src/csharp" This reverts commit 97f2b6d707cc1140788b3618f9ab3de36a3dc2fb. * remove src/csharp, but keep Grpc.Tools * add back templates/src/csharp/build/dependencies.props.template * remove everything but Grpc.Tools and Grpc.Tools.Tests from sln * Revert "unbreak run_tests.py -l csharp" This reverts commit 529b5e2ec19532038b41e935ea9f4f015ac60881. * revive Grpc.Tools.Tests through run_tests.py * Revert "unbreak C# build_package targets" This reverts commit 79d8d83c8bfacdca5c41fa4b33a97751167c7553. * experiment with build_nuget.sh fixes * simplify create_fake_native_artifacts.sh * building of Grpc.Tools in package_targets.py * move shared files from Grpc.Core to Grpc.Tools * yapf format: distribtest_targets.py * Revert "remove C# pre-build steps from prepare_build_windows.bat" This reverts commit bd38956a2b3a670fc51edb0de08aed48e9c885b6. * update C# README.md * disable C# in gke OSS benchmarks 3 years ago			The `run_performance_tests.py` script is deprecated.
			`See [end-to-end benchmarking documentation](/tools/run_tests/peformance/README.md) for more info.`
provide readme for tools/run_tests/ 9 years ago
fix markdown 8 years ago			`# Artifacts & Packages (task_runner.py)`
provide readme for tools/run_tests/ 9 years ago
			`A generalized framework for running predefined tasks based on their labels. We use this to building binary artifacts & distrib packages and testing them)`

fix markdown 8 years ago			`###### Example`
provide readme for tools/run_tests/ 9 years ago			`tools/run_tests/task_runner.py -f python artifact linux x64` (build tasks with labels `python`, `artifact`, `linux`, and `x64`)