Merge github.com:grpc/grpc into monkey_man

pull/5560/head
Craig Tiller 9 years ago
commit b519af8b10
  1. 2
      CONTRIBUTING.md
  2. 217
      INSTALL
  3. 46
      INSTALL.md
  4. 2
      README.md
  5. 2
      examples/cpp/README.md
  6. 2
      examples/cpp/cpptutorial.md
  7. 2
      examples/cpp/helloworld/README.md
  8. 11
      src/python/grpcio/tests/_runner.py
  9. 62
      src/python/grpcio/tests/tests.json
  10. 30
      src/python/grpcio/tests/unit/_sanity/__init__.py
  11. 53
      src/python/grpcio/tests/unit/_sanity/_sanity_test.py
  12. 3
      tools/run_tests/build_python.sh
  13. 6
      tools/run_tests/run_interop_tests.py
  14. 5
      tools/run_tests/run_python.sh
  15. 18
      tools/run_tests/run_tests.py

@ -13,7 +13,7 @@ In order to protect both you and ourselves, you will need to sign the
### Technical requirements
You will need several tools to work with this repository. In addition to all of
the packages described in the [INSTALL](INSTALL) file, you will also need
the packages described in the [INSTALL](INSTALL.md) file, you will also need
python, and the mako template renderer. To install the latter, using pip, one
should simply be able to do `pip install mako`.

@ -1,217 +0,0 @@
These instructions only cover building grpc C and C++ libraries under
typical unix systems. If you need more information, please try grpc's
wiki pages:
https://github.com/google/grpc/wiki
*************************
* If you are in a hurry *
*************************
On Linux (Debian):
Note: you will need to add the Debian 'jessie-backports' distribution to your sources
file first.
Add the following line to your `/etc/apt/sources.list` file:
deb http://http.debian.net/debian jessie-backports main
Install the gRPC library:
$ [sudo] apt-get install libgrpc-dev
OR
$ git clone https://github.com/grpc/grpc.git
$ cd grpc
$ git submodule update --init
$ make
$ [sudo] make install
You don't need anything else than GNU Make, gcc and autotools. Under a Debian
or Ubuntu system, this should boil down to the following packages:
$ [sudo] apt-get install build-essential autoconf libtool
Building the python wrapper requires the following:
$ [sudo] apt-get install python-all-dev python-virtualenv
If you want to install in a different directory than the default /usr/lib, you can
override it on the command line:
$ [sudo] make install prefix=/opt
*******************************
* More detailled instructions *
*******************************
Setting up dependencies
=======================
Dependencies to compile the libraries
-------------------------------------
grpc libraries have few external dependencies. If you need to compile and
install them, they are present in the third_party directory if you have
cloned the github repository recursively. If you didn't clone recursively,
you can still get them later by running the following command:
$ git submodule update --init
Note that the Makefile makes it much easier for you to compile from sources
if you were to clone recursively our git repository: it will automatically
compile zlib and OpenSSL, which are core requirements for grpc. Note this
creates grpc libraries that will have zlib and OpenSSL built-in inside of them,
which significantly increases the libraries' size.
In order to decrease that size, you can manually install zlib and OpenSSL on
your system, so that the Makefile can use them instead.
Under a Debian or Ubuntu system, one can acquire the development package
for zlib this way:
# apt-get install zlib1g-dev
To the best of our knowledge, no distribution has an OpenSSL package that
supports ALPN yet, so you would still have to depend on installing from source
for that particular dependency if you want to reduce the libraries' size.
The recommended version of OpenSSL that provides ALPN support is available
at this URL:
https://www.openssl.org/source/openssl-1.0.2.tar.gz
Dependencies to compile and run the tests
-----------------------------------------
Compiling and running grpc plain-C tests dont't require any more dependency.
Compiling and running grpc C++ tests depend on protobuf 3.0.0, gtest and
gflags. Although gflags is provided in third_party, you will need to manually
install that dependency on your system to run these tests.
Under a Debian or Ubuntu system, you can install the gtests and gflags packages
using apt-get:
# apt-get install libgflags-dev libgtest-dev
However, protobuf 3.0.0 isn't in a debian package yet, but the Makefile will
automatically try and compile the one present in third_party if you cloned the
repository recursively, and that it detects your system is lacking it.
Compiling and installing protobuf 3.0.0 requires a few more dependencies in
itself, notably the autoconf suite. If you have apt-get, you can install
these dependencies this way:
# apt-get install autoconf libtool
If you want to run the tests using one of the sanitized configurations, you
will need clang and its instrumented libc++:
# apt-get install clang libc++-dev
Mac-specific notes:
-------------------
For a Mac system, git is not available by default. You will first need to
install Xcode from the Mac AppStore and then run the following command from a
terminal:
$ sudo xcode-select --install
You should also install "port" following the instructions at
https://www.macports.org . This will reside in /opt/local/bin/port for
most Mac installations. Do the "git submodule" command listed above.
Then execute the following for all the needed build dependencies
$ sudo /opt/local/bin/port install autoconf automake libtool gflags cmake
$ mkdir ~/gtest-svn
$ svn checkout http://googletest.googlecode.com/svn/trunk/ gtest-svn
$ mkdir mybuild
$ cd mybuild
$ cmake ../gtest-svn
$ make
$ make gtest.a gtest_main.a
$ sudo cp libgtest.a libgtest_main.a /opt/local/lib
$ sudo mkdir /opt/local/include/gtest
$ sudo cp -pr ../gtest-svn/include/gtest /opt/local/include/gtest
If you are going to make changes and need to regenerate the projects file,
you will need to install certain modules for python.
$ sudo easy_install simplejson mako
Mingw-specific notes:
---------------------
While gRPC compiles properly under mingw, some more preparation work is needed.
The recommendation is to use msys2. The installation instructions are available
at that address: http://msys2.github.io/
Once this is installed, make sure you are using the following: MinGW-w64 Win64.
You'll be required to install a few more packages:
$ pacman -S make mingw-w64-x86_64-gcc mingw-w64-x86_64-zlib autoconf automake libtool
Please also install OpenSSL from that website:
http://slproweb.com/products/Win32OpenSSL.html
The package Win64 OpenSSL v1.0.2a should do. At that point you should be able
to compile gRPC with the following:
$ export LDFLAGS="-L/mingw64/lib -L/c/OpenSSL-Win64"
$ export CPPFLAGS="-I/mingw64/include -I/c/OpenSSL-Win64/include"
$ make
A word on OpenSSL
-----------------
Secure HTTP2 requires the TLS extension ALPN (see rfc 7301 and
http://http2.github.io/http2-spec/ section 3.3). Our HTTP2 implementation
relies on OpenSSL's implementation. OpenSSL 1.0.2 is the first released version
of OpenSSL that has ALPN support, and this explains our dependency on it.
Note that the Makefile supports compiling only the unsecure elements of grpc,
and if you do not have OpenSSL and do not want it, you can still proceed
with installing only the elements you require. However, we strongly recommend
the use of encryption for all network traffic, and discourage the use of grpc
without TLS.
Compiling
=========
If you have all the dependencies mentioned above, you should simply be able
to go ahead and run "make" to compile grpc's C and C++ libraries:
$ make
Testing
=======
To build and run the tests, you can run the command:
$ make test
If you want to be able to run them in parallel, and get better output, you can
also use the python tool we have written:
$ ./tools/run_tests/run_tests.py
Installing
==========
Once everything is compiled, you should be able to install grpc C and C++
libraries and headers:
# make install

@ -0,0 +1,46 @@
#If you are in a hurry
For language-specific installation instructions for gRPC runtime, please
refer to these documents
* [C++](examples/cpp)
* [C#](src/csharp): NuGet package `Grpc`
* [Go](https://github.com/grpc/grpc-go): `go get google.golang.org/grpc`
* [Java](https://github.com/grpc/grpc-java)
* [Node](src/node): `npm install grpc`
* [Objective-C](src/objective-c)
* [PHP](src/php): `pecl install grpc-beta`
* [Python](src/python/grpcio): `pip install grpcio`
* [Ruby](src/ruby): `gem install grpc`
#Pre-requisites
##Linux
```sh
$ [sudo] apt-get install build-essential autoconf libtool
```
##Mac OSX
For a Mac system, git is not available by default. You will first need to
install Xcode from the Mac AppStore and then run the following command from a
terminal:
```sh
$ [sudo] xcode-select --install
```
#Build from Source
For developers who are interested to contribute, here is how to compile the
gRPC C Core library.
```sh
$ git clone https://github.com/grpc/grpc.git
$ cd grpc
$ git submodule update --init
$ make
$ [sudo] make install
```

@ -13,7 +13,7 @@ You can find more detailed documentation and examples in the [doc](doc) and [exa
#Installation
See [grpc/INSTALL](INSTALL) for installation instructions for various platforms.
See [INSTALL](INSTALL.md) for installation instructions for various platforms.
#Repository Structure & Status

@ -2,7 +2,7 @@
## Installation
To install gRPC on your system, follow the instructions [here](../../INSTALL).
To install gRPC on your system, follow the instructions [here](../../INSTALL.md).
## Hello C++ gRPC!

@ -91,7 +91,7 @@ message Point {
Next we need to generate the gRPC client and server interfaces from our .proto service definition. We do this using the protocol buffer compiler `protoc` with a special gRPC C++ plugin.
For simplicity, we've provided a [makefile](route_guide/Makefile) that runs `protoc` for you with the appropriate plugin, input, and output (if you want to run this yourself, make sure you've installed protoc and followed the gRPC code [installation instructions](../../INSTALL) first):
For simplicity, we've provided a [makefile](route_guide/Makefile) that runs `protoc` for you with the appropriate plugin, input, and output (if you want to run this yourself, make sure you've installed protoc and followed the gRPC code [installation instructions](../../INSTALL.md) first):
```shell
$ make route_guide.grpc.pb.cc route_guide.pb.cc

@ -2,7 +2,7 @@
### Install gRPC
Make sure you have installed gRPC on your system. Follow the instructions here:
[https://github.com/grpc/grpc/blob/master/INSTALL](../../../INSTALL).
[https://github.com/grpc/grpc/blob/master/INSTALL](../../../INSTALL.md).
### Get the tutorial source code

@ -1,4 +1,4 @@
# Copyright 2015, Google Inc.
# Copyright 2015-2016, Google Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
@ -143,10 +143,17 @@ class Runner(object):
def run(self, suite):
"""See setuptools' test_runner setup argument for information."""
# only run test cases with id starting with given prefix
testcase_filter = os.getenv('GPRC_PYTHON_TESTRUNNER_FILTER')
filtered_cases = []
for case in _loader.iterate_suite_cases(suite):
if not testcase_filter or case.id().startswith(testcase_filter):
filtered_cases.append(case)
# Ensure that every test case has no collision with any other test case in
# the augmented results.
augmented_cases = [AugmentedCase(case, uuid.uuid4())
for case in _loader.iterate_suite_cases(suite)]
for case in filtered_cases]
case_id_by_case = dict((augmented_case.case, augmented_case.id)
for augmented_case in augmented_cases)
result_out = StringIO.StringIO()

@ -0,0 +1,62 @@
[
"_base_interface_test.AsyncEasyTest",
"_base_interface_test.AsyncPeasyTest",
"_base_interface_test.SyncEasyTest",
"_base_interface_test.SyncPeasyTest",
"_beta_features_test.BetaFeaturesTest",
"_beta_features_test.ContextManagementAndLifecycleTest",
"_channel_test.ChannelTest",
"_connectivity_channel_test.ChannelConnectivityTest",
"_core_over_links_base_interface_test.AsyncEasyTest",
"_core_over_links_base_interface_test.AsyncPeasyTest",
"_core_over_links_base_interface_test.SyncEasyTest",
"_core_over_links_base_interface_test.SyncPeasyTest",
"_crust_over_core_face_interface_test.DynamicInvokerBlockingInvocationInlineServiceTest",
"_crust_over_core_face_interface_test.DynamicInvokerEventInvocationSynchronousEventServiceTest",
"_crust_over_core_face_interface_test.DynamicInvokerFutureInvocationAsynchronousEventServiceTest",
"_crust_over_core_face_interface_test.GenericInvokerBlockingInvocationInlineServiceTest",
"_crust_over_core_face_interface_test.GenericInvokerEventInvocationSynchronousEventServiceTest",
"_crust_over_core_face_interface_test.GenericInvokerFutureInvocationAsynchronousEventServiceTest",
"_crust_over_core_face_interface_test.MultiCallableInvokerBlockingInvocationInlineServiceTest",
"_crust_over_core_face_interface_test.MultiCallableInvokerEventInvocationSynchronousEventServiceTest",
"_crust_over_core_face_interface_test.MultiCallableInvokerFutureInvocationAsynchronousEventServiceTest",
"_crust_over_core_over_links_face_interface_test.DynamicInvokerBlockingInvocationInlineServiceTest",
"_crust_over_core_over_links_face_interface_test.DynamicInvokerEventInvocationSynchronousEventServiceTest",
"_crust_over_core_over_links_face_interface_test.DynamicInvokerFutureInvocationAsynchronousEventServiceTest",
"_crust_over_core_over_links_face_interface_test.GenericInvokerBlockingInvocationInlineServiceTest",
"_crust_over_core_over_links_face_interface_test.GenericInvokerEventInvocationSynchronousEventServiceTest",
"_crust_over_core_over_links_face_interface_test.GenericInvokerFutureInvocationAsynchronousEventServiceTest",
"_crust_over_core_over_links_face_interface_test.MultiCallableInvokerBlockingInvocationInlineServiceTest",
"_crust_over_core_over_links_face_interface_test.MultiCallableInvokerEventInvocationSynchronousEventServiceTest",
"_crust_over_core_over_links_face_interface_test.MultiCallableInvokerFutureInvocationAsynchronousEventServiceTest",
"_face_interface_test.DynamicInvokerBlockingInvocationInlineServiceTest",
"_face_interface_test.DynamicInvokerEventInvocationSynchronousEventServiceTest",
"_face_interface_test.DynamicInvokerFutureInvocationAsynchronousEventServiceTest",
"_face_interface_test.GenericInvokerBlockingInvocationInlineServiceTest",
"_face_interface_test.GenericInvokerEventInvocationSynchronousEventServiceTest",
"_face_interface_test.GenericInvokerFutureInvocationAsynchronousEventServiceTest",
"_face_interface_test.MultiCallableInvokerBlockingInvocationInlineServiceTest",
"_face_interface_test.MultiCallableInvokerEventInvocationSynchronousEventServiceTest",
"_face_interface_test.MultiCallableInvokerFutureInvocationAsynchronousEventServiceTest",
"_implementations_test.ChannelCredentialsTest",
"_insecure_interop_test.InsecureInteropTest",
"_intermediary_low_test.CancellationTest",
"_intermediary_low_test.EchoTest",
"_intermediary_low_test.ExpirationTest",
"_intermediary_low_test.LonelyClientTest",
"_later_test.LaterTest",
"_logging_pool_test.LoggingPoolTest",
"_lonely_invocation_link_test.LonelyInvocationLinkTest",
"_low_test.HangingServerShutdown",
"_low_test.InsecureServerInsecureClient",
"_not_found_test.NotFoundTest",
"_sanity_test.Sanity",
"_secure_interop_test.SecureInteropTest",
"_transmission_test.RoundTripTest",
"_transmission_test.TransmissionTest",
"_utilities_test.ChannelConnectivityTest",
"beta_python_plugin_test.PythonPluginTest",
"cygrpc_test.InsecureServerInsecureClient",
"cygrpc_test.SecureServerSecureClient",
"cygrpc_test.TypeSmokeTest"
]

@ -0,0 +1,30 @@
# Copyright 2016, Google Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following disclaimer
# in the documentation and/or other materials provided with the
# distribution.
# * Neither the name of Google Inc. nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

@ -0,0 +1,53 @@
# Copyright 2016, Google Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following disclaimer
# in the documentation and/or other materials provided with the
# distribution.
# * Neither the name of Google Inc. nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
import json
import unittest
import tests
class Sanity(unittest.TestCase):
def testTestsJsonUpToDate(self):
"""Autodiscovers all test suites and checks that tests.json is up to date"""
loader = tests.Loader()
loader.loadTestsFromNames(['tests'])
test_suite_names = [
test_case_class.id().rsplit('.', 1)[0]
for test_case_class in tests._loader.iterate_suite_cases(loader.suite)]
test_suite_names = sorted(set(test_suite_names))
with open('src/python/grpcio/tests/tests.json') as tests_json_file:
tests_json = json.load(tests_json_file)
self.assertListEqual(test_suite_names, tests_json)
if __name__ == '__main__':
unittest.main(verbosity=2)

@ -45,3 +45,6 @@ export GRPC_PYTHON_ENABLE_CYTHON_TRACING=1
tox --notest
$ROOT/.tox/py27/bin/python $ROOT/setup.py build
$ROOT/.tox/py27/bin/python $ROOT/setup.py build_py
$ROOT/.tox/py27/bin/python $ROOT/setup.py build_ext --inplace
$ROOT/.tox/py27/bin/python $ROOT/setup.py gather --test

@ -60,6 +60,8 @@ _SKIP_COMPRESSION = ['large_compressed_unary',
_SKIP_ADVANCED = ['custom_metadata', 'status_code_and_message',
'unimplemented_method']
_TEST_TIMEOUT = 3*60
class CXXLanguage:
def __init__(self):
@ -459,7 +461,7 @@ def cloud_to_prod_jobspec(language, test_case, server_host_name,
environ=environ,
shortname='%s:%s:%s:%s' % (suite_name, server_host_name, language,
test_case),
timeout_seconds=90,
timeout_seconds=_TEST_TIMEOUT,
flake_retries=5 if args.allow_flakes else 0,
timeout_retries=2 if args.allow_flakes else 0,
kill_handler=_job_kill_handler)
@ -495,7 +497,7 @@ def cloud_to_cloud_jobspec(language, test_case, server_name, server_host,
environ=environ,
shortname='cloud_to_cloud:%s:%s_server:%s' % (language, server_name,
test_case),
timeout_seconds=90,
timeout_seconds=_TEST_TIMEOUT,
flake_retries=5 if args.allow_flakes else 0,
timeout_retries=2 if args.allow_flakes else 0,
kill_handler=_job_kill_handler)

@ -42,7 +42,12 @@ export LDFLAGS="-L$ROOT/libs/$CONFIG"
export GRPC_PYTHON_BUILD_WITH_CYTHON=1
export GRPC_PYTHON_ENABLE_CYTHON_TRACING=1
if [ "$CONFIG" = "gcov" ]
then
tox
else
$ROOT/.tox/py27/bin/python $ROOT/setup.py test
fi
mkdir -p $ROOT/reports
rm -rf $ROOT/reports/python-coverage

@ -353,15 +353,27 @@ class PythonLanguage(object):
_check_compiler(self.args.compiler, ['default'])
def test_specs(self):
# load list of known test suites
with open('src/python/grpcio/tests/tests.json') as tests_json_file:
tests_json = json.load(tests_json_file)
environment = dict(_FORCE_ENVIRON_FOR_WRAPPERS)
environment['PYVER'] = '2.7'
if self.config.build_config != 'gcov':
return [self.config.job_spec(
['tools/run_tests/run_python.sh'],
None,
environ=dict(environment.items() +
[('GPRC_PYTHON_TESTRUNNER_FILTER', suite_name)]),
shortname='py.test.%s' % suite_name,
timeout_seconds=5*60)
for suite_name in tests_json]
else:
return [self.config.job_spec(['tools/run_tests/run_python.sh'],
None,
environ=environment,
shortname='py.test',
timeout_seconds=15*60
)]
shortname='py.test.coverage',
timeout_seconds=15*60)]
def pre_build_steps(self):
return []

Loading…
Cancel
Save