Merge branch 'master' into debug_qps_stream

pull/5444/head
vjpai 9 years ago
commit 2697f2141f
  1. 17
      CONTRIBUTING.md
  2. 13
      INSTALL.md
  3. 447
      examples/README.md
  4. 2
      examples/cpp/README.md
  5. 10
      examples/php/README.md
  6. 4
      examples/python/route_guide/route_guide_server.py
  7. 9
      setup.py
  8. 29
      src/core/channel/client_channel.c
  9. 18
      src/core/channel/subchannel_call_holder.c
  10. 22
      src/core/iomgr/timer_heap.c
  11. 74
      src/cpp/README.md
  12. 2
      src/node/README.md
  13. 2
      src/node/src/client.js
  14. 117
      src/php/README.md
  15. 12
      src/python/grpcio/README.rst
  16. 39
      src/python/grpcio/commands.py
  17. 5
      src/python/grpcio/grpc/_cython/_cygrpc/completion_queue.pyx.pxi
  18. 13
      src/python/grpcio/precompiled.py
  19. 2
      src/python/grpcio/tests/_runner.py
  20. 3
      src/ruby/.rubocop.yml
  21. 97
      src/ruby/lib/grpc/generic/rpc_server.rb
  22. 25
      src/ruby/spec/generic/rpc_server_spec.rb
  23. 4
      summerofcode/ideas.md
  24. 194
      test/core/iomgr/timer_heap_test.c
  25. 19
      test/cpp/qps/qps_worker.cc
  26. 14
      tools/distrib/python/docgen.py
  27. 20
      tools/run_tests/artifact_targets.py
  28. 6
      tools/run_tests/build_python.sh
  29. 4
      tools/run_tests/run_python.sh
  30. 2
      tools/run_tests/run_tests.py

@ -21,6 +21,15 @@ In order to run all of the tests we provide, you will need valgrind and clang.
More specifically, under debian, you will need the package libc++-dev to
properly run all the tests.
Compiling and running grpc C++ tests depend on protobuf 3.0.0, gtest and gflags.
Although gflags is provided in third_party, you will need to manually install
that dependency on your system to run these tests. Under a Debian or Ubuntu
system, you can install the gtests and gflags packages using apt-get:
```sh
$ [sudo] apt-get install libgflags-dev libgtest-dev
```
If you are planning to work on any of the languages other than C and C++, you
will also need their appropriate development environments.
@ -36,9 +45,13 @@ In order to run most of the available tests, one would need to run:
`./tools/run_tests/run_tests.py`
If you want to run all the possible tests for any of the languages {c, c++, node, php, python}, do this:
If you want to run tests for any of the languages {c, c++, csharp, node, objc, php, python, ruby}, do this:
`./tools/run_tests/run_tests.py -l <lang>`
To know about the list of available commands, do this:
`./tools/run_tests/run_tests.py -l <lang> -c all`
`./tools/run_tests/run_tests.py -h`
## Adding or removing source code

@ -3,7 +3,7 @@
For language-specific installation instructions for gRPC runtime, please
refer to these documents
* [C++](examples/cpp)
* [C++](examples/cpp): Currently to install gRPC for C++, you need to build from source as described below.
* [C#](src/csharp): NuGet package `Grpc`
* [Go](https://github.com/grpc/grpc-go): `go get google.golang.org/grpc`
* [Java](https://github.com/grpc/grpc-java)
@ -32,6 +32,17 @@ terminal:
$ [sudo] xcode-select --install
```
##Protoc
By default gRPC uses [protocol buffers](https://github.com/google/protobuf),
you will need the `protoc` compiler to generate stub server and client code.
If you compile gRPC from source, as described below, the Makefile will
automatically try and compile the `protoc` in third_party if you cloned the
repository recursively and it detects that you don't already have it
installed.
#Build from Source
For developers who are interested to contribute, here is how to compile the

@ -1,450 +1,27 @@
# Examples
# Getting started
This directory contains code examples for all the C-based gRPC implementations: C++, Node.js, Python, Ruby, Objective-C, PHP, and C#. You can find examples and instructions specific to your
favourite language in the relevant subdirectory.
Examples for Go and Java gRPC live in their own repositories:
Welcome to the developer documentation for gRPC, a language-neutral,
platform-neutral remote procedure call (RPC) system developed at Google.
* [Java](https://github.com/grpc/grpc-java/tree/master/examples)
* [Android Java](https://github.com/grpc/grpc-java/tree/master/examples/android)
* [Go](https://github.com/grpc/grpc-go/tree/master/examples)
This document introduces you to gRPC with a quick overview and a simple
Hello World example. You'll find more tutorials and reference docs in this repository - more documentation is coming soon!
For more comprehensive documentation, including an [overview](http://www.grpc.io/docs/) and tutorials that use this example code, visit [grpc.io](http://www.grpc.io/docs/).
<a name="quickstart"></a>
## Quick start
You can find quick start guides for each language, including installation instructions, examples, and tutorials here:
Each example directory has quick start instructions for the appropriate language, including installation instructions and how to run our simplest Hello World example:
* [C++](cpp)
* [Java](https://github.com/grpc/grpc-java/tree/master/examples)
* [Go](https://github.com/grpc/grpc-go/tree/master/examples)
* [Ruby](ruby)
* [Node.js](node)
* [Android Java](https://github.com/grpc/grpc-java/tree/master/examples/android)
* [Python](python/helloworld)
* [C#](csharp)
* [Objective-C](objective-c/helloworld)
* [PHP](php)
## What's in this repository?
The `examples` directory contains documentation, resources, and examples
for all gRPC users. You can find examples and instructions specific to your
favourite language in the relevant subdirectory.
You can find out about the gRPC source code repositories in
[`grpc`](https://github.com/grpc/grpc). Each repository provides instructions
for building the appropriate libraries for your language.
## What is gRPC?
In gRPC a *client* application can directly call
methods on a *server* application on a different machine as if it was a
local object, making it easier for you to create distributed applications and
services. As in many RPC systems, gRPC is based around the idea of defining
a *service*, specifying the methods that can be called remotely with their
parameters and return types. On the server side, the server implements this
interface and runs a gRPC server to handle client calls. On the client side,
the client has a *stub* that provides exactly the same methods as the server.
<!--TODO: diagram-->
gRPC clients and servers can run and talk to each other in a variety of
environments - from servers inside Google to your own desktop - and can
be written in any of gRPC's [supported languages](#quickstart). So, for
example, you can easily create a gRPC server in Java with clients in Go,
Python, or Ruby. In addition, the latest Google APIs will have gRPC versions
of their interfaces, letting you easily build Google functionality into
your applications.
<a name="protocolbuffers"></a>
### Working with protocol buffers
By default gRPC uses *protocol buffers*, Google’s
mature open source mechanism for serializing structured data (although it
can be used with other data formats such as JSON). As you'll
see in our example below, you define gRPC services using *proto files*,
with method parameters and return types specified as protocol buffer message
types. You
can find out lots more about protocol buffers in the [Protocol Buffers
documentation](https://developers.google.com/protocol-buffers/docs/overview).
#### Protocol buffer versions
While protocol buffers have been available for open source users for some
time, our examples use a new flavour of protocol buffers called proto3,
which has a slightly simplified syntax, some useful new features, and supports
lots more languages. This is currently available as an alpha release in
Java, C++, Java_nano (Android Java), Python, and Ruby from [the protocol buffers Github
repo](https://github.com/google/protobuf/releases), as well as a Go language
generator from [the golang/protobuf Github repo](https://github.com/golang/protobuf), with more languages in development. You can find out more in the [proto3 language guide](https://developers.google.com/protocol-buffers/docs/proto3), and see
the major differences from the current default version in the [release notes](https://github.com/google/protobuf/releases). More proto3 documentation is coming soon.
In general, while you *can* use proto2 (the current default protocol buffers version), we recommend that you use proto3 with gRPC as it lets you use the full range of gRPC-supported languages, as well as avoiding compatibility
issues with proto2 clients talking to proto3 servers and vice versa.
<a name="hello"></a>
## Hello gRPC!
Now that you know a bit more about gRPC, the easiest way to see how it
works is to look at a simple example. Our Hello World walks you through the
construction of a simple gRPC client-server application, showing you how to:
- Create a protocol buffers schema that defines a simple RPC service with
a single
Hello World method.
- Create a Java server that implements this interface.
- Create a Java client that accesses the Java server.
- Create a Go client that accesses
the same Java server.
The complete code for the example is available in the `examples`
directory. We use the Git versioning system for source code management:
however, you don't need to know anything about Git to follow along other
than how to install and run a few git commands.
This is an introductory example rather than a comprehensive tutorial, so
don't worry if you're not a Go or
Java developer - the concepts are similar for all languages, and you can
find more implementations of our Hello World example in other languages (and full tutorials where available) in
the [language-specific folders](#quickstart) in this repository. Complete tutorials and
reference documentation for all gRPC languages are coming soon.
<a name="setup"></a>
### Setup
This section explains how to set up your local machine to work with
the example code. If you just want to read the example, you can go straight
to the [next step](#servicedef).
#### Install Git
You can download and install Git from http://git-scm.com/download. Once
installed you should have access to the git command line tool. The main
commands that you will need to use are:
- git clone ... : clone a remote repository onto your local machine
- git checkout ... : check out a particular branch or a tagged version of
the code to hack on
#### Install gRPC
To build and install gRPC plugins and related tools:
- For Java, see the [Java quick start](https://github.com/grpc/grpc-java).
- For Go, see the [Go quick start](https://github.com/grpc/grpc-go).
#### Get the source code
The example code for our Java example lives in the `grpc-java`
GitHub repository. Clone this repository to your local machine by running the
following command:
```
git clone https://github.com/grpc/grpc-java.git
```
Change your current directory to grpc-java/examples
```
cd grpc-java/examples
```
<a name="servicedef"></a>
### Defining a service
The first step in creating our example is to define a *service*: an RPC
service specifies the methods that can be called remotely with their parameters
and return types. As you saw in the
[overview](#protocolbuffers) above, gRPC does this using [protocol
buffers](https://developers.google.com/protocol-buffers/docs/overview). We
use the protocol buffers interface definition language (IDL) to define our
service methods, and define the parameters and return
types as protocol buffer message types. Both the client and the
server use interface code generated from the service definition.
Here's our example service definition, defined using protocol buffers IDL in
[helloworld.proto](https://github.com/grpc/grpc-java/tree/master/examples/src/main/proto). The `Greeter`
service has one method, `SayHello`, that lets the server receive a single
`HelloRequest`
message from the remote client containing the user's name, then send back
a greeting in a single `HelloReply`. This is the simplest type of RPC you
can specify in gRPC - you can find out about other types in the tutorial for your chosen language.
```proto
syntax = "proto3";
option java_package = "io.grpc.examples";
package helloworld;
// The greeter service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
```
<a name="generating"></a>
### Generating gRPC code
Once we've defined our service, we use the protocol buffer compiler
`protoc` to generate the special client and server code we need to create
our application - right now we're going to generate Java code, though you
can generate gRPC code in any gRPC-supported language (as you'll see later
in this example). The generated code contains both stub code for clients to
use and an abstract interface for servers to implement, both with the method
defined in our `Greeter` service.
(If you didn't install the gRPC plugins and protoc on your system and are just reading along with
the example, you can skip this step and move
onto the next one where we examine the generated code.)
For simplicity, we've provided a [Gradle build file](https://github.com/grpc/grpc-java/blob/master/examples/build.gradle) with our Java examples that runs `protoc` for you with the appropriate plugin, input, and output:
```shell
../gradlew build
```
This generates the following classes from our .proto, which contain all the generated code
we need to create our example:
- `Helloworld.java`, which
has all the protocol buffer code to populate, serialize, and retrieve our
`HelloRequest` and `HelloReply` message types
- `GreeterGrpc.java`, which contains (along with some other useful code):
- an interface for `Greeter` servers to implement
```java
public static interface Greeter {
public void sayHello(io.grpc.examples.Helloworld.HelloRequest request,
io.grpc.stub.StreamObserver<io.grpc.examples.Helloworld.HelloReply> responseObserver);
}
```
- _stub_ classes that clients can use to talk to a `Greeter` server. As you can see, they also implement the `Greeter` interface.
```java
public static class GreeterStub extends
io.grpc.stub.AbstractStub<GreeterStub, GreeterServiceDescriptor>
implements Greeter {
...
}
```
<a name="server"></a>
### Writing a server
Now let's write some code! First we'll create a server application to implement
our service. Note that we're not going to go into a lot of detail about how
to create a server in this section. More detailed information will be in the
tutorial for your chosen language: check if there's one available yet in the relevant [quick start](#quickstart).
Our server application has two classes:
- a main server class that hosts the service implementation and allows access over the
network: [HelloWorldServer.java](https://github.com/grpc/grpc-java/blob/master/examples/src/main/java/io/grpc/examples/helloworld/HelloWorldServer.java).
- a simple service implementation class [GreeterImpl.java](https://github.com/grpc/grpc-java/blob/master/examples/src/main/java/io/grpc/examples/helloworld/HelloWorldServer.java#L51).
#### Service implementation
[GreeterImpl.java](https://github.com/grpc/grpc-java/blob/master/examples/src/main/java/io/grpc/examples/helloworld/HelloWorldServer.java#L51)
actually implements our `Greeter` service's required behaviour.
As you can see, the class `GreeterImpl` implements the interface
`GreeterGrpc.Greeter` that we [generated](#generating) from our proto
[IDL](https://github.com/grpc/grpc-java/tree/master/examples/src/main/proto) by implementing the method `sayHello`:
```java
@Override
public void sayHello(HelloRequest req, StreamObserver<HelloReply> responseObserver) {
HelloReply reply = HelloReply.newBuilder().setMessage("Hello " + req.getName()).build();
responseObserver.onValue(reply);
responseObserver.onCompleted();
}
```
- `sayHello` takes two parameters:
- `HelloRequest`: the request
- `StreamObserver<HelloReply>`: a response observer, which is
a special interface for the server to call with its response
To return our response to the client and complete the call:
1. We construct and populate a `HelloReply` response object with our exciting
message, as specified in our interface definition.
2. We return the `HelloReply` to the client and then specify that we've finished dealing with the RPC.
#### Server implementation
[HelloWorldServer.java](https://github.com/grpc/grpc-java/blob/master/examples/src/main/java/io/grpc/examples/helloworld/HelloWorldServer.java)
shows the other main feature required to provide a gRPC service; making the service
implementation available from the network.
```java
/* The port on which the server should run */
private int port = 50051;
private ServerImpl server;
private void start() throws Exception {
server = NettyServerBuilder.forPort(port)
.addService(GreeterGrpc.bindService(new GreeterImpl()))
.build().start();
logger.info("Server started, listening on " + port);
Runtime.getRuntime().addShutdownHook(new Thread() {
@Override
public void run() {
// Use stderr here since the logger may have been reset by its JVM shutdown hook.
System.err.println("*** shutting down gRPC server since JVM is shutting down");
HelloWorldServer.this.stop();
System.err.println("*** server shut down");
}
});
}
```
Here we create an appropriate gRPC server, binding the `Greeter` service
implementation that we created to a port. Then we start the server running: the server is now ready to receive
requests from `Greeter` service clients on our specified port. We'll cover
how all this works in a bit more detail in our language-specific documentation.
<a name="client"></a>
### Writing a client
Client-side gRPC is pretty simple. In this step, we'll use the generated code
to write a simple client that can access the `Greeter` server we created
in the [previous section](#server). You can see the complete client code in
[HelloWorldClient.java](https://github.com/grpc/grpc-java/blob/master/examples/src/main/java/io/grpc/examples/helloworld/HelloWorldClient.java).
Again, we're not going to go into much detail about how to implement a client;
we'll leave that for the tutorial.
#### Connecting to the service
First let's look at how we connect to the `Greeter` server. First we need
to create a gRPC channel, specifying the hostname and port of the server we
want to connect to. Then we use the channel to construct the stub instance.
```java
private final ChannelImpl channel;
private final GreeterGrpc.GreeterBlockingStub blockingStub;
public HelloWorldClient(String host, int port) {
channel =
NettyChannelBuilder.forAddress(host, port).negotiationType(NegotiationType.PLAINTEXT)
.build();
blockingStub = GreeterGrpc.newBlockingStub(channel);
}
```
In this case, we create a blocking stub. This means that the RPC call waits
for the server to respond, and will either return a response or raise an
exception. gRPC Java has other kinds of stubs that make non-blocking calls
to the server, where the response is returned asynchronously.
#### Calling an RPC
Now we can contact the service and obtain a greeting:
1. We construct and fill in a `HelloRequest` to send to the service.
2. We call the stub's `hello()` RPC with our request and get a `HelloReply`
back, from which we can get our greeting.
```java
HelloRequest req = HelloRequest.newBuilder().setName(name).build();
HelloReply reply = blockingStub.sayHello(req);
```
<a name="run"></a>
### Try it out!
Our [Gradle build file](https://github.com/grpc/grpc-java/blob/master/examples/build.gradle) simplifies building and running the examples.
You can build and run the server from the `grpc-java` root folder with:
```sh
$ ./gradlew :grpc-examples:helloWorldServer
```
and in another terminal window confirm that it receives a message.
```sh
$ ./gradlew :grpc-examples:helloWorldClient
```
### Adding another client
Finally, let's look at one of gRPC's most useful features - interoperability
between code in different languages. So far, we've just looked at Java code
generated from and implementing our `Greeter` service definition. However,
as you'll see if you look at the language-specific subdirectories
in this repository, we've also generated and implemented `Greeter`
in some of gRPC's other supported languages. Each service
and client uses interface code generated from the same proto
that we used for the Java example.
So, for example, if we visit the [`go` example
directory](https://github.com/grpc/grpc-go/tree/master/examples) and look at the
[`greeter_client`](https://github.com/grpc/grpc-go/blob/master/examples/greeter_client/main.go),
we can see that like the Java client, it connects to a `Greeter` service
at `localhost:50051` and uses a stub to call the `SayHello` method with a
`HelloRequest`:
```go
const (
address = "localhost:50051"
defaultName = "world"
)
func main() {
// Set up a connection to the server.
conn, err := grpc.Dial(address)
if err != nil {
log.Fatalf("did not connect: %v", err)
}
defer conn.Close()
c := pb.NewGreeterClient(conn)
// Contact the server and print out its response.
name := defaultName
if len(os.Args) > 1 {
name = os.Args[1]
}
r, err := c.SayHello(context.Background(), &pb.HelloRequest{Name:
name})
if err != nil {
log.Fatalf("could not greet: %v", err)
}
log.Printf("Greeting: %s", r.Message)
}
```
If we run the Java server from earlier in another terminal window, we can
run the Go client and connect to it just like the Java client, even though
it's written in a different language.
```
$ greeter_client
```
## Read more!
- You can find links to language-specific tutorials, examples, and other docs in each language's [quick start](#quickstart).
- [gRPC Authentication Support](http://www.grpc.io/docs/guides/auth.html) introduces authentication support in gRPC with supported mechanisms and examples.

@ -2,7 +2,7 @@
## Installation
To install gRPC on your system, follow the instructions [here](../../INSTALL.md).
To install gRPC on your system, follow the instructions to build from source [here](../../INSTALL.md). This also installs the protocol buffer compiler `protoc` (if you don't have it already), and the C++ gRPC plugin for `protoc`.
## Hello C++ gRPC!

@ -4,16 +4,15 @@ gRPC in 3 minutes (PHP)
PREREQUISITES
-------------
This requires PHP 5.5 or greater.
This requires `php` >=5.5, `phpize`, `pecl`, `phpunit`
INSTALL
-------
- On Mac OS X, install [homebrew][]. Run the following command to install gRPC.
- Install the gRPC PHP extension
```sh
$ curl -fsSL https://goo.gl/getgrpc | bash -s php
$ [sudo] pecl install grpc-beta
```
This will download and run the [gRPC install script][] and compile the gRPC PHP extension.
- Clone this repository
@ -37,6 +36,7 @@ TRY IT!
Please follow the instruction in [Node][] to run the server
```
$ cd examples/node
$ npm install
$ nodejs greeter_server.js
```
@ -58,7 +58,5 @@ TUTORIAL
You can find a more detailed tutorial in [gRPC Basics: PHP][]
[homebrew]:http://brew.sh
[gRPC install script]:https://raw.githubusercontent.com/grpc/homebrew-grpc/master/scripts/install
[Node]:https://github.com/grpc/grpc/tree/master/examples/node
[gRPC Basics: PHP]:http://www.grpc.io/docs/tutorials/basic/php.html

@ -1,4 +1,4 @@
# Copyright 2015, Google Inc.
# Copyright 2015-2016, Google Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
@ -128,7 +128,7 @@ def serve():
while True:
time.sleep(_ONE_DAY_IN_SECONDS)
except KeyboardInterrupt:
server.stop()
server.stop(0)
if __name__ == '__main__':
serve()

@ -108,8 +108,13 @@ if "linux" in sys.platform or "darwin" in sys.platform:
def cython_extensions(package_names, module_names, extra_sources, include_dirs,
libraries, define_macros, build_with_cython=False):
# Set compiler directives linetrace argument only if we care about tracing;
# this is due to Cython having different behavior between linetrace being
# False and linetrace being unset. See issue #5689.
cython_compiler_directives = {}
if ENABLE_CYTHON_TRACING:
define_macros = define_macros + [('CYTHON_TRACE_NOGIL', 1)]
cython_compiler_directives['linetrace'] = True
file_extension = 'pyx' if build_with_cython else 'c'
module_files = [os.path.join(PYTHON_STEM,
name.replace('.', '/') + '.' + file_extension)
@ -129,7 +134,7 @@ def cython_extensions(package_names, module_names, extra_sources, include_dirs,
return Cython.Build.cythonize(
extensions,
include_path=include_dirs,
compiler_directives={'linetrace': bool(ENABLE_CYTHON_TRACING)})
compiler_directives=cython_compiler_directives)
else:
return extensions
@ -154,6 +159,7 @@ INSTALL_REQUIRES = (
SETUP_REQUIRES = (
'sphinx>=1.3',
'sphinx_rtd_theme>=0.1.8'
) + INSTALL_REQUIRES
COMMAND_CLASS = {
@ -165,6 +171,7 @@ COMMAND_CLASS = {
'build_tagged_ext': precompiled.BuildTaggedExt,
'gather': commands.Gather,
'run_interop': commands.RunInterop,
'test_lite': commands.TestLite
}
# Ensure that package data is copied over before any commands have been run:

@ -165,7 +165,6 @@ static void cc_on_config_changed(grpc_exec_ctx *exec_ctx, void *arg,
channel_data *chand = arg;
grpc_lb_policy *lb_policy = NULL;
grpc_lb_policy *old_lb_policy;
grpc_resolver *old_resolver;
grpc_connectivity_state state = GRPC_CHANNEL_TRANSIENT_FAILURE;
int exit_idle = 0;
@ -201,28 +200,25 @@ static void cc_on_config_changed(grpc_exec_ctx *exec_ctx, void *arg,
}
if (iomgr_success && chand->resolver) {
grpc_resolver *resolver = chand->resolver;
GRPC_RESOLVER_REF(resolver, "channel-next");
grpc_connectivity_state_set(exec_ctx, &chand->state_tracker, state,
"new_lb+resolver");
if (lb_policy != NULL) {
watch_lb_policy(exec_ctx, chand, lb_policy, state);
}
gpr_mu_unlock(&chand->mu_config);
GRPC_CHANNEL_STACK_REF(chand->owning_stack, "resolver");
grpc_resolver_next(exec_ctx, resolver, &chand->incoming_configuration,
grpc_resolver_next(exec_ctx, chand->resolver,
&chand->incoming_configuration,
&chand->on_config_changed);
GRPC_RESOLVER_UNREF(exec_ctx, resolver, "channel-next");
gpr_mu_unlock(&chand->mu_config);
} else {
old_resolver = chand->resolver;
chand->resolver = NULL;
if (chand->resolver != NULL) {
grpc_resolver_shutdown(exec_ctx, chand->resolver);
GRPC_RESOLVER_UNREF(exec_ctx, chand->resolver, "channel");
chand->resolver = NULL;
}
grpc_connectivity_state_set(exec_ctx, &chand->state_tracker,
GRPC_CHANNEL_FATAL_FAILURE, "resolver_gone");
gpr_mu_unlock(&chand->mu_config);
if (old_resolver != NULL) {
grpc_resolver_shutdown(exec_ctx, old_resolver);
GRPC_RESOLVER_UNREF(exec_ctx, old_resolver, "channel");
}
}
if (exit_idle) {
@ -247,7 +243,6 @@ static void cc_start_transport_op(grpc_exec_ctx *exec_ctx,
grpc_channel_element *elem,
grpc_transport_op *op) {
channel_data *chand = elem->channel_data;
grpc_resolver *destroy_resolver = NULL;
grpc_exec_ctx_enqueue(exec_ctx, op->on_consumed, true, NULL);
@ -279,7 +274,8 @@ static void cc_start_transport_op(grpc_exec_ctx *exec_ctx,
if (op->disconnect && chand->resolver != NULL) {
grpc_connectivity_state_set(exec_ctx, &chand->state_tracker,
GRPC_CHANNEL_FATAL_FAILURE, "disconnect");
destroy_resolver = chand->resolver;
grpc_resolver_shutdown(exec_ctx, chand->resolver);
GRPC_RESOLVER_UNREF(exec_ctx, chand->resolver, "channel");
chand->resolver = NULL;
if (chand->lb_policy != NULL) {
grpc_pollset_set_del_pollset_set(exec_ctx,
@ -290,11 +286,6 @@ static void cc_start_transport_op(grpc_exec_ctx *exec_ctx,
}
}
gpr_mu_unlock(&chand->mu_config);
if (destroy_resolver) {
grpc_resolver_shutdown(exec_ctx, destroy_resolver);
GRPC_RESOLVER_UNREF(exec_ctx, destroy_resolver, "channel");
}
}
typedef struct {

@ -174,17 +174,15 @@ static void subchannel_ready(grpc_exec_ctx *exec_ctx, void *arg, bool success) {
holder->creation_phase = GRPC_SUBCHANNEL_CALL_HOLDER_NOT_CREATING;
if (holder->connected_subchannel == NULL) {
fail_locked(exec_ctx, holder);
} else if (1 == gpr_atm_acq_load(&holder->subchannel_call)) {
/* already cancelled before subchannel became ready */
fail_locked(exec_ctx, holder);
} else {
if (!gpr_atm_rel_cas(
&holder->subchannel_call, 0,
(gpr_atm)(uintptr_t)grpc_connected_subchannel_create_call(
exec_ctx, holder->connected_subchannel, holder->pollset))) {
GPR_ASSERT(gpr_atm_acq_load(&holder->subchannel_call) == 1);
/* if this cas fails, the call was cancelled before the pick completed */
fail_locked(exec_ctx, holder);
} else {
retry_waiting_locked(exec_ctx, holder);
}
gpr_atm_rel_store(
&holder->subchannel_call,
(gpr_atm)(uintptr_t)grpc_connected_subchannel_create_call(
exec_ctx, holder->connected_subchannel, holder->pollset));
retry_waiting_locked(exec_ctx, holder);
}
gpr_mu_unlock(&holder->mu);
GRPC_CALL_STACK_UNREF(exec_ctx, holder->owning_call, "pick_subchannel");

@ -1,6 +1,6 @@
/*
*
* Copyright 2015, Google Inc.
* Copyright 2015-2016, Google Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@ -46,7 +46,7 @@
static void adjust_upwards(grpc_timer **first, uint32_t i, grpc_timer *t) {
while (i > 0) {
uint32_t parent = (uint32_t)(((int)i - 1) / 2);
if (gpr_time_cmp(first[parent]->deadline, t->deadline) >= 0) break;
if (gpr_time_cmp(first[parent]->deadline, t->deadline) <= 0) break;
first[i] = first[parent];
first[i]->heap_index = i;
i = parent;
@ -62,16 +62,14 @@ static void adjust_downwards(grpc_timer **first, uint32_t i, uint32_t length,
grpc_timer *t) {
for (;;) {
uint32_t left_child = 1u + 2u * i;
uint32_t right_child;
uint32_t next_i;
if (left_child >= length) break;
right_child = left_child + 1;
next_i = right_child < length &&
gpr_time_cmp(first[left_child]->deadline,
first[right_child]->deadline) < 0
? right_child
: left_child;
if (gpr_time_cmp(t->deadline, first[next_i]->deadline) >= 0) break;
uint32_t right_child = left_child + 1;
uint32_t next_i = right_child < length &&
gpr_time_cmp(first[left_child]->deadline,
first[right_child]->deadline) > 0
? right_child
: left_child;
if (gpr_time_cmp(t->deadline, first[next_i]->deadline) <= 0) break;
first[i] = first[next_i];
first[i]->heap_index = i;
i = next_i;
@ -95,7 +93,7 @@ static void maybe_shrink(grpc_timer_heap *heap) {
static void note_changed_priority(grpc_timer_heap *heap, grpc_timer *timer) {
uint32_t i = timer->heap_index;
uint32_t parent = (uint32_t)(((int)i - 1) / 2);
if (gpr_time_cmp(heap->timers[parent]->deadline, timer->deadline) < 0) {
if (gpr_time_cmp(heap->timers[parent]->deadline, timer->deadline) > 0) {
adjust_upwards(heap->timers, i, timer);
} else {
adjust_downwards(heap->timers, i, heap->timer_count, timer);

@ -6,3 +6,77 @@ This directory contains source code for C++ implementation of gRPC.
#Status
Beta
#Pre-requisites
##Linux
```sh
$ [sudo] apt-get install build-essential autoconf libtool
```
##Mac OSX
For a Mac system, git is not available by default. You will first need to
install Xcode from the Mac AppStore and then run the following command from a
terminal:
```sh
$ [sudo] xcode-select --install
```
##Protoc
By default gRPC uses [protocol buffers](https://github.com/google/protobuf),
you will need the `protoc` compiler to generate stub server and client code.
If you compile gRPC from source, as described below, this also installs the
`protoc` compiler.
If it hasn't been installed, you can run the following commands to install it.
```sh
$ cd grpc/third_party/protobuf
$ sudo make install # 'make' should have been run by core grpc
```
Alternatively, you can download `protoc` binaries from
[the protocol buffers Github repository](https://github.com/google/protobuf/releases).
#Installation
Currently to install gRPC for C++, you need to build from source as described
below.
#Build from Source
```sh
$ git clone https://github.com/grpc/grpc.git
$ cd grpc
$ git submodule update --init
$ make
$ [sudo] make install
```
#Documentation
You can find out how to build and run our simplest gRPC C++ example in our
[C++ quick start](https://github.com/grpc/grpc/tree/{{ site.data.config.grpc_release_branch }}/examples/cpp).
For more detailed documentation on using gRPC in C++ , see our main
documentation site at [grpc.io](http://grpc.io), specifically:
* [Overview](http://www.grpc.io/docs/): An introduction to gRPC with a simple
Hello World example in all our supported languages, including C++.
* [gRPC Basics - C++](http://www.grpc.io/docs/tutorials/basic/c.html):
A tutorial that steps you through creating a simple gRPC C++ example
application.
* [Asynchronous Basics - C++](http://www.grpc.io/docs/tutorials/async/helloasync-cpp.html):
A tutorial that shows you how to use gRPC C++'s asynchronous/non-blocking
APIs.
# Examples
Code examples for gRPC C++ live in this repository's
[examples/cpp](https://github.com/grpc/grpc/tree/{{ site.data.config.grpc_release_branch }}/examples/cpp) directory.

@ -5,7 +5,7 @@
Beta
## PREREQUISITES
- `node`: This requires `node` to be installed. If you instead have the `nodejs` executable on Debian, you should install the [`nodejs-legacy`](https://packages.debian.org/sid/nodejs-legacy) package.
- `node`: This requires `node` to be installed, version `0.12` or above. If you instead have the `nodejs` executable on Debian, you should install the [`nodejs-legacy`](https://packages.debian.org/sid/nodejs-legacy) package.
## INSTALLATION

@ -408,7 +408,7 @@ function makeUnaryRequestFunction(method, serialize, deserialize) {
}
}
if (status.code !== grpc.status.OK) {
error = new Error(response.status.details);
error = new Error(status.details);
error.code = status.code;
error.metadata = status.metadata;
callback(error);

@ -9,14 +9,21 @@ Beta
## Environment
Prerequisite: PHP 5.5 or later, `phpunit`, `pecl`
Prerequisite: `php` >=5.5, `phpize`, `pecl`, `phpunit`
**Linux:**
**Linux (Debian):**
```sh
$ sudo apt-get install php5 php5-dev php-pear
```
**Linux (CentOS):**
```sh
$ yum install php55w
$ yum --enablerepo=remi,remi-php55 install php-devel php-pear
```
**Mac OS X:**
```sh
@ -24,11 +31,11 @@ $ curl -O http://pear.php.net/go-pear.phar
$ sudo php -d detect_unicode=0 go-pear.phar
```
**PHPUnit: (Both Linux and Mac OS X)**
**PHPUnit:**
```sh
$ curl https://phar.phpunit.de/phpunit.phar -o phpunit.phar
$ chmod +x phpunit.phar
$ sudo mv phpunit.phar /usr/local/bin/phpunit
$ wget https://phar.phpunit.de/phpunit-old.phar
$ chmod +x phpunit-old.phar
$ sudo mv phpunit-old.phar /usr/bin/phpunit
```
## Quick Install
@ -39,15 +46,22 @@ Install the gRPC PHP extension
sudo pecl install grpc-beta
```
This will compile and install the gRPC PHP extension into the standard PHP extension directory. You should be able to run the [unit tests](#unit-tests), with the PHP extension installed.
To run tests with generated stub code from `.proto` files, you will also need the `composer`, `protoc` and `protoc-gen-php` binaries. You can find out how to get these [below](#generated-code-tests).
## Build from Source
### gRPC C core library
Clone this repository
```sh
$ git clone https://github.com/grpc/grpc.git
```
Build and install the gRPC C core libraries
Build and install the gRPC C core library
```sh
$ cd grpc
@ -56,20 +70,15 @@ $ make
$ sudo make install
```
Note: you may encounter a warning about the Protobuf compiler `protoc` 3.0.0+ not being installed. The following might help, and will be useful later on when we need to compile the `protoc-gen-php` tool.
### gRPC PHP extension
```sh
$ cd grpc/third_party/protobuf
$ sudo make install # 'make' should have been run by core grpc
```
Install the gRPC PHP extension
Install the gRPC PHP extension from PECL
```sh
$ sudo pecl install grpc-beta
```
OR
Or, compile from source
```sh
$ cd grpc/src/php/ext/grpc
@ -79,58 +88,98 @@ $ make
$ sudo make install
```
### Update php.ini
Add this line to your `php.ini` file, e.g. `/etc/php5/cli/php.ini`
```sh
extension=grpc.so
```
Install Composer
## Unit Tests
You will need the source code to run tests
```sh
$ git clone https://github.com/grpc/grpc.git
$ cd grpc
$ git pull --recurse-submodules && git submodule update --init --recursive
```
Run unit tests
```sh
$ cd grpc/src/php
$ ./bin/run_tests.sh
```
## Generated Code Tests
This section specifies the prerequisites for running the generated code tests, as well as how to run the tests themselves.
### Composer
If you don't have it already, install `composer` to pull in some runtime dependencies based on the `composer.json` file.
```sh
$ curl -sS https://getcomposer.org/installer | php
$ sudo mv composer.phar /usr/local/bin/composer
$ cd grpc/src/php
$ composer install
```
## Unit Tests
### Protobuf compiler
Run unit tests
Again if you don't have it already, you need to install the protobuf compiler `protoc`, version 3.0.0+.
If you compiled the gRPC C core library from source above, the `protoc` binary should have been installed as well. If it hasn't been installed, you can run the following commands to install it.
```sh
$ cd grpc/src/php
$ ./bin/run_tests.sh
$ cd grpc/third_party/protobuf
$ sudo make install # 'make' should have been run by core grpc
```
## Generated Code Tests
Alternatively, you can download `protoc` binaries from [the protocol buffers Github repository](https://github.com/google/protobuf/releases).
Install `protoc-gen-php`
### PHP protobuf compiler
You need to install `protoc-gen-php` to generate stub class `.php` files from service definition `.proto` files.
```sh
$ cd grpc/src/php/vendor/datto/protobuf-php
$ cd grpc/src/php/vendor/datto/protobuf-php # if you had run `composer install` in the previous step
OR
$ git clone https://github.com/stanley-cheung/Protobuf-PHP # clone from github repo
$ gem install rake ronn
$ rake pear:package version=1.0
$ sudo pear install Protobuf-1.0.tgz
```
Generate client stub code
### Client Stub
Generate client stub classes from `.proto` files
```sh
$ cd grpc/src/php
$ ./bin/generate_proto_php.sh
```
Run a local server serving the math services
### Run test server
- Please see [Node][] on how to run an example server
Run a local server serving the math services. Please see [Node][] for how to run an example server.
```sh
$ cd grpc/src/node
$ cd grpc
$ npm install
$ nodejs examples/math_server.js
$ nodejs src/node/test/math/math_server.js
```
### Run test client
Run the generated code tests
```sh
@ -161,13 +210,15 @@ $ sudo service apache2 restart
Make sure the Node math server is still running, as above.
```sh
$ cd grpc/src/node
$ nodejs examples/math_server.js
$ cd grpc
$ npm install
$ nodejs src/node/test/math/math_server.js
```
Make sure you have run `composer install` to generate the `vendor/autoload.php` file
```sh
$ cd grpc/src/php
$ composer install
```
@ -229,13 +280,15 @@ $ sudo service php5-fpm restart
Make sure the Node math server is still running, as above.
```sh
$ cd grpc/src/node
$ nodejs examples/math_server.js
$ cd grpc
$ npm install
$ nodejs src/node/test/math/math_server.js
```
Make sure you have run `composer install` to generate the `vendor/autoload.php` file
```sh
$ cd grpc/src/php
$ composer install
```

@ -6,7 +6,7 @@ Package for gRPC Python.
Installation
------------
gRPC Python is available for Linux and Mac OS X running Python 2.7.
gRPC Python is available for Linux, Mac OS X, and Windows running Python 2.7.
From PyPI
~~~~~~~~~
@ -23,11 +23,15 @@ Else system wide (on Ubuntu)...
$ sudo pip install grpcio
n.b. On Windows and on Mac OS X one *must* have a recent release of :code:`pip`
to retrieve the proper wheel from PyPI. Be sure to upgrade to the latest
version!
From Source
~~~~~~~~~~~
Building from source requires that you have the Python headers (usually a
package named `python-dev`).
package named :code:`python-dev`).
::
@ -36,8 +40,8 @@ package named `python-dev`).
$ cd $REPO_ROOT
$ pip install .
Note that `$REPO_ROOT` can be assigned to whatever directory name floats your
fancy.
Note that :code:`$REPO_ROOT` can be assigned to whatever directory name floats
your fancy.
Troubleshooting
~~~~~~~~~~~~~~~

@ -119,8 +119,7 @@ class SphinxDocumentation(setuptools.Command):
import sphinx
import sphinx.apidoc
metadata = self.distribution.metadata
src_dir = os.path.join(
PYTHON_STEM, self.distribution.package_dir[''], 'grpc')
src_dir = os.path.join(PYTHON_STEM, 'grpc')
sys.path.append(src_dir)
sphinx.apidoc.main([
'', '--force', '--full', '-H', metadata.name, '-A', metadata.author,
@ -264,6 +263,42 @@ class Gather(setuptools.Command):
self.distribution.fetch_build_eggs(self.distribution.tests_require)
class TestLite(setuptools.Command):
"""Command to run tests without fetching or building anything."""
description = 'run tests without fetching or building anything.'
user_options = []
def initialize_options(self):
pass
def finalize_options(self):
# distutils requires this override.
pass
def run(self):
self._add_eggs_to_path()
import tests
loader = tests.Loader()
loader.loadTestsFromNames(['tests'])
runner = tests.Runner()
result = runner.run(loader.suite)
if not result.wasSuccessful():
sys.exit('Test failure')
def _add_eggs_to_path(self):
"""Adds all egg files under .eggs to sys.path"""
# TODO(jtattemusch): there has to be a cleaner way to do this
import pkg_resources
eggs_dir = os.path.join(PYTHON_STEM, '../../../.eggs')
eggs = [os.path.join(eggs_dir, filename)
for filename in os.listdir(eggs_dir)
if filename.endswith('.egg')]
for egg in eggs:
sys.path.insert(0, pkg_resources.normalize_path(egg))
class RunInterop(test.test):
description = 'run interop test client/server'

@ -140,7 +140,8 @@ cdef class CompletionQueue:
grpc_completion_queue_shutdown(self.c_completion_queue)
# Pump the queue
while not self.is_shutdown:
event = grpc_completion_queue_next(
self.c_completion_queue, c_deadline, NULL)
with nogil:
event = grpc_completion_queue_next(
self.c_completion_queue, c_deadline, NULL)
self._interpret_event(event)
grpc_completion_queue_destroy(self.c_completion_queue)

@ -31,6 +31,7 @@ import os
import platform
import shutil
import sys
import sysconfig
import setuptools
@ -51,9 +52,15 @@ USE_PRECOMPILED_BINARIES = bool(int(os.environ.get(
def _tagged_ext_name(base):
uname = platform.uname()
tags = '-'.join((grpc_version.VERSION, uname[0], uname[4]))
flavor = 'ucs2' if sys.maxunicode == 65535 else 'ucs4'
return '{base}-{tags}-{flavor}'.format(base=base, tags=tags, flavor=flavor)
tags = (
grpc_version.VERSION,
'py{}'.format(sysconfig.get_python_version()),
uname[0],
uname[4],
)
ucs = 'ucs{}'.format(sysconfig.get_config_var('Py_UNICODE_SIZE'))
return '{base}-{tags}-{ucs}'.format(
base=base, tags='-'.join(tags), ucs=ucs)
class BuildTaggedExt(setuptools.Command):

@ -144,7 +144,7 @@ class Runner(object):
def run(self, suite):
"""See setuptools' test_runner setup argument for information."""
# only run test cases with id starting with given prefix
testcase_filter = os.getenv('GPRC_PYTHON_TESTRUNNER_FILTER')
testcase_filter = os.getenv('GRPC_PYTHON_TESTRUNNER_FILTER')
filtered_cases = []
for case in _loader.iterate_suite_cases(suite):
if not testcase_filter or case.id().startswith(testcase_filter):

@ -15,3 +15,6 @@ Metrics/CyclomaticComplexity:
Metrics/PerceivedComplexity:
Max: 8
Metrics/ClassLength:
Max: 250

@ -107,7 +107,9 @@ module GRPC
# Starts running the jobs in the thread pool.
def start
fail 'already stopped' if @stopped
@stop_mutex.synchronize do
fail 'already stopped' if @stopped
end
until @workers.size == @size.to_i
next_thread = Thread.new do
catch(:exit) do # allows { throw :exit } to kill a thread
@ -264,10 +266,10 @@ module GRPC
@pool = Pool.new(@pool_size)
@run_cond = ConditionVariable.new
@run_mutex = Mutex.new
@running = false
# running_state can take 4 values: :not_started, :running, :stopping, and
# :stopped. State transitions can only proceed in that order.
@running_state = :not_started
@server = RpcServer.setup_srv(server_override, @cq, **kw)
@stopped = false
@stop_mutex = Mutex.new
end
# stops a running server
@ -275,27 +277,42 @@ module GRPC
# the call has no impact if the server is already stopped, otherwise
# server's current call loop is it's last.
def stop
return unless @running
@stop_mutex.synchronize do
@stopped = true
@run_mutex.synchronize do
fail 'Cannot stop before starting' if @running_state == :not_started
return if @running_state != :running
transition_running_state(:stopping)
end
deadline = from_relative_time(@poll_period)
return if @server.close(@cq, deadline)
deadline = from_relative_time(@poll_period)
@server.close(@cq, deadline)
@pool.stop
end
# determines if the server has been stopped
def stopped?
@stop_mutex.synchronize do
return @stopped
def running_state
@run_mutex.synchronize do
return @running_state
end
end
# Can only be called while holding @run_mutex
def transition_running_state(target_state)
state_transitions = {
not_started: :running,
running: :stopping,
stopping: :stopped
}
if state_transitions[@running_state] == target_state
@running_state = target_state
else
fail "Bad server state transition: #{@running_state}->#{target_state}"
end
end
# determines if the server is currently running
def running?
@running
running_state == :running
end
def stopped?
running_state == :stopped
end
# Is called from other threads to wait for #run to start up the server.
@ -304,13 +321,11 @@ module GRPC
#
# @param timeout [Numeric] number of seconds to wait
# @result [true, false] true if the server is running, false otherwise
def wait_till_running(timeout = 0.1)
end_time, sleep_period = Time.now + timeout, (1.0 * timeout) / 100
while Time.now < end_time
@run_mutex.synchronize { @run_cond.wait(@run_mutex) } unless running?
sleep(sleep_period)
def wait_till_running(timeout = nil)
@run_mutex.synchronize do
@run_cond.wait(@run_mutex, timeout) if @running_state == :not_started
return @running_state == :running
end
running?
end
# Runs the server in its own thread, then waits for signal INT or TERM on
@ -360,11 +375,14 @@ module GRPC
# @param service [Object|Class] a service class or object as described
# above
def handle(service)
fail 'cannot add services if the server is running' if running?
fail 'cannot add services if the server is stopped' if stopped?
cls = service.is_a?(Class) ? service : service.class
assert_valid_service_class(cls)
add_rpc_descs_for(service)
@run_mutex.synchronize do
unless @running_state == :not_started
fail 'cannot add services if the server has been started'
end
cls = service.is_a?(Class) ? service : service.class
assert_valid_service_class(cls)
add_rpc_descs_for(service)
end
end
# runs the server
@ -375,16 +393,13 @@ module GRPC
# - #running? returns true after this is called, until #stop cause the
# the server to stop.
def run
if rpc_descs.size.zero?
GRPC.logger.warn('did not run as no services were present')
return
end
@run_mutex.synchronize do
@running = true
@run_cond.signal
fail 'cannot run without registering services' if rpc_descs.size.zero?
@pool.start
@server.start
transition_running_state(:running)
@run_cond.broadcast
end
@pool.start
@server.start
loop_handle_server_calls
end
@ -413,9 +428,9 @@ module GRPC
# handles calls to the server
def loop_handle_server_calls
fail 'not running' unless @running
fail 'not started' if running_state == :not_started
loop_tag = Object.new
until stopped?
while running_state == :running
begin
an_rpc = @server.request_call(@cq, loop_tag, INFINITE_FUTURE)
break if (!an_rpc.nil?) && an_rpc.call.nil?
@ -430,11 +445,14 @@ module GRPC
rescue Core::CallError, RuntimeError => e
# these might happen for various reasonse. The correct behaviour of
# the server is to log them and continue, if it's not shutting down.
GRPC.logger.warn("server call failed: #{e}") unless stopped?
if running_state == :running
GRPC.logger.warn("server call failed: #{e}")
end
next
end
end
@running = false
# @running_state should be :stopping here
@run_mutex.synchronize { transition_running_state(:stopped) }
GRPC.logger.info("stopped: #{self}")
end
@ -484,9 +502,10 @@ module GRPC
cls.assert_rpc_descs_have_methods
end
# This should be called while holding @run_mutex
def add_rpc_descs_for(service)
cls = service.is_a?(Class) ? service : service.class
specs, handlers = rpc_descs, rpc_handlers
specs, handlers = (@rpc_descs ||= {}), (@rpc_handlers ||= {})
cls.rpc_descs.each_pair do |name, spec|
route = "/#{cls.service_name}/#{name}".to_sym
fail "already registered: rpc #{route} from #{spec}" if specs.key? route

@ -1,4 +1,4 @@
# Copyright 2015, Google Inc.
# Copyright 2015-2016, Google Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
@ -220,19 +220,10 @@ describe GRPC::RpcServer do
@srv = RpcServer.new(**opts)
end
after(:each) do
@srv.stop
end
it 'starts out false' do
expect(@srv.stopped?).to be(false)
end
it 'stays false after a #stop is called before #run' do
@srv.stop
expect(@srv.stopped?).to be(false)
end
it 'stays false after the server starts running', server: true do
@srv.handle(EchoService)
t = Thread.new { @srv.run }
@ -247,8 +238,8 @@ describe GRPC::RpcServer do
t = Thread.new { @srv.run }
@srv.wait_till_running
@srv.stop
expect(@srv.stopped?).to be(true)
t.join
expect(@srv.stopped?).to be(true)
end
end
@ -266,9 +257,7 @@ describe GRPC::RpcServer do
server_override: @server
}
r = RpcServer.new(**opts)
r.run
expect(r.running?).to be(false)
r.stop
expect { r.run }.to raise_error(RuntimeError)
end
it 'is true after run is called with a registered service' do
@ -293,10 +282,6 @@ describe GRPC::RpcServer do
@srv = RpcServer.new(**@opts)
end
after(:each) do
@srv.stop
end
it 'raises if #run has already been called' do
@srv.handle(EchoService)
t = Thread.new { @srv.run }
@ -528,10 +513,6 @@ describe GRPC::RpcServer do
@srv = RpcServer.new(**server_opts)
end
after(:each) do
@srv.stop
end
it 'should be added to BadStatus when requests fail', server: true do
service = FailingService.new
@srv.handle(service)

@ -19,7 +19,9 @@ gRPC C Core:
1. Port gRPC to one of the major BSD platforms ([FreeBSD](https://freebsd.org), [NetBSD](https://netbsd.org), and [OpenBSD](https://openbsd.org)) and create packages for them. Add [kqueue](https://www.freebsd.org/cgi/man.cgi?query=kqueue) support in the process.
* **Required skills:** C programming language, BSD operating system.
* **Likely mentors:** [Craig Tiller](https://github.com/ctiller), [Nicolas Noble](https://github.com/nicolasnoble).
* **Likely mentors:** [Craig Tiller](https://github.com/ctiller),
[Nicolas Noble](https://github.com/nicolasnoble),
[Vijay Pai](https://github.com/vjpai).
1. Fix gRPC C-core's URI parser. The current parser does not qualify as a standard parser according to [RFC3986]( https://tools.ietf.org/html/rfc3986). Write test suites to verify this and make changes necessary to make the URI parser compliant.
* **Required skills:** C programming language, HTTP standard compliance.
* **Likely mentors:** [Craig Tiller](https://github.com/ctiller).

@ -1,6 +1,6 @@
/*
*
* Copyright 2015, Google Inc.
* Copyright 2015-2016, Google Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@ -38,6 +38,8 @@
#include <grpc/support/alloc.h>
#include <grpc/support/log.h>
#include <grpc/support/useful.h>
#include "test/core/util/test_config.h"
static gpr_timespec random_deadline(void) {
@ -57,79 +59,6 @@ static grpc_timer *create_test_elements(size_t num_elements) {
return elems;
}
static int cmp_elem(const void *a, const void *b) {
int i = *(const int *)a;
int j = *(const int *)b;
return i - j;
}
static size_t *all_top(grpc_timer_heap *pq, size_t *n) {
size_t *vec = NULL;
size_t *need_to_check_children;
size_t num_need_to_check_children = 0;
*n = 0;
if (pq->timer_count == 0) return vec;
need_to_check_children =
gpr_malloc(pq->timer_count * sizeof(*need_to_check_children));
need_to_check_children[num_need_to_check_children++] = 0;
vec = gpr_malloc(pq->timer_count * sizeof(*vec));
while (num_need_to_check_children > 0) {
size_t ind = need_to_check_children[0];
size_t leftchild, rightchild;
num_need_to_check_children--;
memmove(need_to_check_children, need_to_check_children + 1,
num_need_to_check_children * sizeof(*need_to_check_children));
vec[(*n)++] = ind;
leftchild = 1u + 2u * ind;
if (leftchild < pq->timer_count) {
if (gpr_time_cmp(pq->timers[leftchild]->deadline,
pq->timers[ind]->deadline) >= 0) {
need_to_check_children[num_need_to_check_children++] = leftchild;
}
rightchild = leftchild + 1;
if (rightchild < pq->timer_count &&
gpr_time_cmp(pq->timers[rightchild]->deadline,
pq->timers[ind]->deadline) >= 0) {
need_to_check_children[num_need_to_check_children++] = rightchild;
}
}
}
gpr_free(need_to_check_children);
return vec;
}
static void check_pq_top(grpc_timer *elements, grpc_timer_heap *pq,
uint8_t *inpq, size_t num_elements) {
gpr_timespec max_deadline = gpr_inf_past(GPR_CLOCK_REALTIME);
size_t *max_deadline_indices =
gpr_malloc(num_elements * sizeof(*max_deadline_indices));
size_t *top_elements;
size_t num_max_deadline_indices = 0;
size_t num_top_elements;
size_t i;
for (i = 0; i < num_elements; ++i) {
if (inpq[i] && gpr_time_cmp(elements[i].deadline, max_deadline) >= 0) {
if (gpr_time_cmp(elements[i].deadline, max_deadline) > 0) {
num_max_deadline_indices = 0;
max_deadline = elements[i].deadline;
}
max_deadline_indices[num_max_deadline_indices++] = elements[i].heap_index;
}
}
qsort(max_deadline_indices, num_max_deadline_indices,
sizeof(*max_deadline_indices), cmp_elem);
top_elements = all_top(pq, &num_top_elements);
GPR_ASSERT(num_top_elements == num_max_deadline_indices);
for (i = 0; i < num_top_elements; i++) {
GPR_ASSERT(max_deadline_indices[i] == top_elements[i]);
}
gpr_free(max_deadline_indices);
gpr_free(top_elements);
}
static int contains(grpc_timer_heap *pq, grpc_timer *el) {
size_t i;
for (i = 0; i < pq->timer_count; i++) {
@ -145,15 +74,19 @@ static void check_valid(grpc_timer_heap *pq) {
size_t right_child = left_child + 1u;
if (left_child < pq->timer_count) {
GPR_ASSERT(gpr_time_cmp(pq->timers[i]->deadline,
pq->timers[left_child]->deadline) >= 0);
pq->timers[left_child]->deadline) <= 0);
}
if (right_child < pq->timer_count) {
GPR_ASSERT(gpr_time_cmp(pq->timers[i]->deadline,
pq->timers[right_child]->deadline) >= 0);
pq->timers[right_child]->deadline) <= 0);
}
}
}
/*******************************************************************************
* test1
*/
static void test1(void) {
grpc_timer_heap pq;
const size_t num_test_elements = 200;
@ -162,6 +95,8 @@ static void test1(void) {
grpc_timer *test_elements = create_test_elements(num_test_elements);
uint8_t *inpq = gpr_malloc(num_test_elements);
gpr_log(GPR_INFO, "test1");
grpc_timer_heap_init(&pq);
memset(inpq, 0, num_test_elements);
GPR_ASSERT(grpc_timer_heap_is_empty(&pq));
@ -172,7 +107,6 @@ static void test1(void) {
check_valid(&pq);
GPR_ASSERT(contains(&pq, &test_elements[i]));
inpq[i] = 1;
check_pq_top(test_elements, &pq, inpq, num_test_elements);
}
for (i = 0; i < num_test_elements; ++i) {
/* Test that check still succeeds even for element that wasn't just
@ -182,7 +116,7 @@ static void test1(void) {
GPR_ASSERT(pq.timer_count == num_test_elements);
check_pq_top(test_elements, &pq, inpq, num_test_elements);
check_valid(&pq);
for (i = 0; i < num_test_operations; ++i) {
size_t elem_num = (size_t)rand() % num_test_elements;
@ -193,14 +127,12 @@ static void test1(void) {
grpc_timer_heap_add(&pq, el);
GPR_ASSERT(contains(&pq, el));
inpq[elem_num] = 1;
check_pq_top(test_elements, &pq, inpq, num_test_elements);
check_valid(&pq);
} else {
GPR_ASSERT(contains(&pq, el));
grpc_timer_heap_remove(&pq, el);
GPR_ASSERT(!contains(&pq, el));
inpq[elem_num] = 0;
check_pq_top(test_elements, &pq, inpq, num_test_elements);
check_valid(&pq);
}
}
@ -210,7 +142,108 @@ static void test1(void) {
gpr_free(inpq);
}
/*******************************************************************************
* test2
*/
typedef struct {
grpc_timer elem;
bool inserted;
} elem_struct;
static elem_struct *search_elems(elem_struct *elems, size_t count,
bool inserted) {
size_t *search_order = gpr_malloc(count * sizeof(*search_order));
for (size_t i = 0; i < count; i++) {
search_order[i] = i;
}
for (size_t i = 0; i < count * 2; i++) {
size_t a = (size_t)rand() % count;
size_t b = (size_t)rand() % count;
GPR_SWAP(size_t, search_order[a], search_order[b]);
}
elem_struct *out = NULL;
for (size_t i = 0; out == NULL && i < count; i++) {
if (elems[search_order[i]].inserted == inserted) {
out = &elems[search_order[i]];
}
}
gpr_free(search_order);
return out;
}
static void test2(void) {
gpr_log(GPR_INFO, "test2");
grpc_timer_heap pq;
elem_struct elems[1000];
size_t num_inserted = 0;
grpc_timer_heap_init(&pq);
memset(elems, 0, sizeof(elems));
for (size_t round = 0; round < 10000; round++) {
int r = rand() % 1000;
if (r <= 550) {
/* 55% of the time we try to add something */
elem_struct *el = search_elems(elems, GPR_ARRAY_SIZE(elems), false);
if (el != NULL) {
el->elem.deadline = random_deadline();
grpc_timer_heap_add(&pq, &el->elem);
el->inserted = true;
num_inserted++;
check_valid(&pq);
}
} else if (r <= 650) {
/* 10% of the time we try to remove something */
elem_struct *el = search_elems(elems, GPR_ARRAY_SIZE(elems), true);
if (el != NULL) {
grpc_timer_heap_remove(&pq, &el->elem);
el->inserted = false;
num_inserted--;
check_valid(&pq);
}
} else {
/* the remaining times we pop */
if (num_inserted > 0) {
grpc_timer *top = grpc_timer_heap_top(&pq);
grpc_timer_heap_pop(&pq);
for (size_t i = 0; i < GPR_ARRAY_SIZE(elems); i++) {
if (top == &elems[i].elem) {
GPR_ASSERT(elems[i].inserted);
elems[i].inserted = false;
}
}
num_inserted--;
check_valid(&pq);
}
}
if (num_inserted) {
gpr_timespec *min_deadline = NULL;
for (size_t i = 0; i < GPR_ARRAY_SIZE(elems); i++) {
if (elems[i].inserted) {
if (min_deadline == NULL) {
min_deadline = &elems[i].elem.deadline;
} else {
if (gpr_time_cmp(elems[i].elem.deadline, *min_deadline) < 0) {
min_deadline = &elems[i].elem.deadline;
}
}
}
}
GPR_ASSERT(
0 == gpr_time_cmp(grpc_timer_heap_top(&pq)->deadline, *min_deadline));
}
}
grpc_timer_heap_destroy(&pq);
}
static void shrink_test(void) {
gpr_log(GPR_INFO, "shrink_test");
grpc_timer_heap pq;
size_t i;
size_t expected_size;
@ -274,6 +307,7 @@ int main(int argc, char **argv) {
for (i = 0; i < 5; i++) {
test1();
test2();
shrink_test();
}

@ -101,6 +101,19 @@ static std::unique_ptr<Server> CreateServer(const ServerConfig& config) {
abort();
}
class ScopedProfile GRPC_FINAL {
public:
ScopedProfile(const char* filename, bool enable) : enable_(enable) {
if (enable_) grpc_profiler_start(filename);
}
~ScopedProfile() {
if (enable_) grpc_profiler_stop();
}
private:
const bool enable_;
};
class WorkerServiceImpl GRPC_FINAL : public WorkerService::Service {
public:
WorkerServiceImpl(int server_port, QpsWorker* worker)
@ -114,9 +127,8 @@ class WorkerServiceImpl GRPC_FINAL : public WorkerService::Service {
return Status(StatusCode::RESOURCE_EXHAUSTED, "");
}
grpc_profiler_start("qps_client.prof");
ScopedProfile profile("qps_client.prof", false);
Status ret = RunClientBody(ctx, stream);
grpc_profiler_stop();
return ret;
}
@ -128,9 +140,8 @@ class WorkerServiceImpl GRPC_FINAL : public WorkerService::Service {
return Status(StatusCode::RESOURCE_EXHAUSTED, "");
}
grpc_profiler_start("qps_server.prof");
ScopedProfile profile("qps_server.prof", false);
Status ret = RunServerBody(ctx, stream);
grpc_profiler_stop();
return ret;
}

@ -1,5 +1,5 @@
#!/usr/bin/env python2.7
# Copyright 2015, Google Inc.
# Copyright 2015-2016, Google Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
@ -51,29 +51,35 @@ SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
PROJECT_ROOT = os.path.abspath(os.path.join(SCRIPT_DIR, '..', '..', '..'))
CONFIG = args.config
SETUP_PATH = os.path.join(PROJECT_ROOT, 'src/python/grpcio/setup.py')
DOC_PATH = os.path.join(PROJECT_ROOT, 'src/python/grpcio/doc/build')
SETUP_PATH = os.path.join(PROJECT_ROOT, 'setup.py')
REQUIREMENTS_PATH = os.path.join(PROJECT_ROOT, 'requirements.txt')
DOC_PATH = os.path.join(PROJECT_ROOT, 'doc/build')
INCLUDE_PATH = os.path.join(PROJECT_ROOT, 'include')
LIBRARY_PATH = os.path.join(PROJECT_ROOT, 'libs/{}'.format(CONFIG))
VIRTUALENV_DIR = os.path.join(SCRIPT_DIR, 'distrib_virtualenv')
VIRTUALENV_PYTHON_PATH = os.path.join(VIRTUALENV_DIR, 'bin', 'python')
VIRTUALENV_PIP_PATH = os.path.join(VIRTUALENV_DIR, 'bin', 'pip')
environment = os.environ.copy()
environment.update({
'CONFIG': CONFIG,
'CFLAGS': '-I{}'.format(INCLUDE_PATH),
'LDFLAGS': '-L{}'.format(LIBRARY_PATH),
'LD_LIBRARY_PATH': LIBRARY_PATH
'LD_LIBRARY_PATH': LIBRARY_PATH,
'GRPC_PYTHON_BUILD_WITH_CYTHON': '1',
})
subprocess_arguments_list = [
{'args': ['make'], 'cwd': PROJECT_ROOT},
{'args': ['virtualenv', VIRTUALENV_DIR], 'env': environment},
{'args': [VIRTUALENV_PIP_PATH, 'install', '-r', REQUIREMENTS_PATH],
'env': environment},
{'args': [VIRTUALENV_PYTHON_PATH, SETUP_PATH, 'build'], 'env': environment},
{'args': [VIRTUALENV_PYTHON_PATH, SETUP_PATH, 'doc'], 'env': environment},
]
for subprocess_arguments in subprocess_arguments_list:
print('Running command: {}'.format(subprocess_arguments['args']))
subprocess.check_call(**subprocess_arguments)
if args.submit:

@ -69,16 +69,6 @@ def create_jobspec(name, cmdline, environ=None, shell=False,
return jobspec
def macos_arch_env(arch):
"""Returns environ specifying -arch arguments for make."""
if arch == 'x86':
arch_arg = '-arch i386'
elif arch == 'x64':
arch_arg = '-arch x86_64'
else:
raise Exception('Unsupported arch')
return {'CFLAGS': arch_arg, 'LDFLAGS': arch_arg}
_MACOS_COMPAT_FLAG = '-mmacosx-version-min=10.7'
_ARCH_FLAG_MAP = {
@ -191,13 +181,17 @@ class CSharpExtArtifact:
environ = {'CONFIG': 'opt',
'EMBED_OPENSSL': 'true',
'EMBED_ZLIB': 'true',
'CFLAGS': '-DGPR_BACKWARDS_COMPATIBILITY_MODE'}
'CFLAGS': '-DGPR_BACKWARDS_COMPATIBILITY_MODE',
'LDFLAGS': ''}
if self.platform == 'linux':
return create_docker_jobspec(self.name,
'tools/dockerfile/grpc_artifact_linux_%s' % self.arch,
'tools/run_tests/build_artifact_csharp.sh')
'tools/run_tests/build_artifact_csharp.sh',
environ=environ)
else:
environ.update(macos_arch_env(self.arch))
archflag = _ARCH_FLAG_MAP[self.arch]
environ['CFLAGS'] += ' %s %s' % (archflag, _MACOS_COMPAT_FLAG)
environ['LDFLAGS'] += ' %s' % archflag
return create_jobspec(self.name,
['tools/run_tests/build_artifact_csharp.sh'],
environ=environ)

@ -40,7 +40,11 @@ export PATH=$ROOT/bins/$CONFIG:$ROOT/bins/$CONFIG/protobuf:$PATH
export CFLAGS="-I$ROOT/include -std=gnu99"
export LDFLAGS="-L$ROOT/libs/$CONFIG"
export GRPC_PYTHON_BUILD_WITH_CYTHON=1
export GRPC_PYTHON_ENABLE_CYTHON_TRACING=1
if [ "$CONFIG" = "gcov" ]
then
export GRPC_PYTHON_ENABLE_CYTHON_TRACING=1
fi
tox --notest

@ -40,13 +40,13 @@ export PATH=$ROOT/bins/$CONFIG:$ROOT/bins/$CONFIG/protobuf:$PATH
export CFLAGS="-I$ROOT/include -std=c89"
export LDFLAGS="-L$ROOT/libs/$CONFIG"
export GRPC_PYTHON_BUILD_WITH_CYTHON=1
export GRPC_PYTHON_ENABLE_CYTHON_TRACING=1
if [ "$CONFIG" = "gcov" ]
then
export GRPC_PYTHON_ENABLE_CYTHON_TRACING=1
tox
else
$ROOT/.tox/py27/bin/python $ROOT/setup.py test
$ROOT/.tox/py27/bin/python $ROOT/setup.py test_lite
fi
mkdir -p $ROOT/reports

@ -360,7 +360,7 @@ class PythonLanguage(object):
['tools/run_tests/run_python.sh'],
None,
environ=dict(environment.items() +
[('GPRC_PYTHON_TESTRUNNER_FILTER', suite_name)]),
[('GRPC_PYTHON_TESTRUNNER_FILTER', suite_name)]),
shortname='py.test.%s' % suite_name,
timeout_seconds=5*60)
for suite_name in tests_json]

Loading…
Cancel
Save