From 3165ff3d95d46e3e8f091bc64ad88e7e52dd8a65 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 8 Mar 2016 00:53:53 -0800 Subject: [PATCH 001/215] Added compression spec --- doc/PROTOCOL-HTTP2.md | 4 +- doc/compression.md | 105 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 107 insertions(+), 2 deletions(-) create mode 100644 doc/compression.md diff --git a/doc/PROTOCOL-HTTP2.md b/doc/PROTOCOL-HTTP2.md index 357ea72571b..31d694b803d 100644 --- a/doc/PROTOCOL-HTTP2.md +++ b/doc/PROTOCOL-HTTP2.md @@ -38,7 +38,7 @@ Request-Headers are delivered as HTTP2 headers in HEADERS + CONTINUATION frames. * **Nanosecond** → "n" * **Content-Type** → "content-type" "application/grpc" [("+proto" / "+json" / {_custom_})] * **Content-Coding** → "identity" / "gzip" / "deflate" / "snappy" / {_custom_} -* **Message-Encoding** → "grpc-encoding" Content-Coding +* **Message-Encoding** → "grpc-encoding" Content-Coding * **Message-Accept-Encoding** → "grpc-accept-encoding" Content-Coding \*("," Content-Coding) * **User-Agent** → "user-agent" {_structured user-agent string_} * **Message-Type** → "grpc-message-type" {_type name for message schema_} @@ -83,7 +83,7 @@ binary values' lengths being post-Base64. The repeated sequence of **Length-Prefixed-Message** items is delivered in DATA frames * **Length-Prefixed-Message** → Compressed-Flag Message-Length Message -* **Compressed-Flag** → 0 / 1 # encoded as 1 byte unsigned integer +* **Compressed-Flag** → 0 / 1 # encoded as 1 byte unsigned integer * **Message-Length** → {_length of Message_} # encoded as 4 byte unsigned integer * **Message** → \*{binary octet} diff --git a/doc/compression.md b/doc/compression.md new file mode 100644 index 00000000000..ddbbff1c541 --- /dev/null +++ b/doc/compression.md @@ -0,0 +1,105 @@ +## **gRPC Compression** + +The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", +"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be +interpreted as described in [RFC 2119](http://www.ietf.org/rfc/rfc2119.txt). + +### Intent + +Compression is used to reduce the amount of bandwidth used between peers. The +compression supported by gRPC acts _at the individual message level_, taking +_message_ [as defined in the wire format +document](PROTOCOL-HTTP2.md). + +The implementation supports different compression algorithms. A _default +compression level_, to be used in the absence of message-specific settings, MAY +be specified for during channel creation. + +The ability to control compression settings per call and to enable/disable +compression on a per message basis MAY be used to prevent CRIME/BEAST attacks. +It also allows for asymmetric compression communication, whereby a response MAY +be compressed differently, if at all. + +### Specification + +Compression MAY be configured by the Client Application by calling the +appropriate API method. There are two scenarios where compression MAY be +configured: + ++ At channel creation time, which sets the channel default compression and + therefore the compression that SHALL be used in the absence of per-RPC + compression configuration. ++ At response time, via: + + For unary RPCs, the {Client,Server}Context instance. + + For streaming RPCs, the {Client,Server}Writer instance. In this case, + configuration is reduced to disabling compression altogether. + +### Compression Method Asymmetry Between Peers + +A gRPC peer MAY choose to respond using a different compression method to that +of the request, including not performing any compression, regardless of channel +and RPC settings (for example, if compression would result in small or negative +gains). + +A compressed message from a client with an algorithm unsupported by a server, +WILL result in an INVALID\_ARGUMENT error, alongside the receiving peer's +`grpc-accept-encoding` header specifying the algorithms it accepts. If an +INTERNAL error is returned from the server despite having used one of the +algorithms from the `grpc-accept-encoding h`eader, the cause MUST NOT be related +to compression. Data sent from a server compressed with an algorithm not +supported by the client will also result in an INTERNAL error. + +Note that a peer MAY choose to not disclose all the encodings it supports. +However, if it receives a message compressed in an undisclosed but supported +encoding, it MUST include said encoding in the response's `grpc-accept-encoding +h`eader. + +For every message a server is requested to compress using an algorithm it knows +the client doesn't support (as indicated by the last `grpc-accept-encoding` +header received from the client), it SHALL send the message uncompressed. + +### Specific Disabling of Compression + +If the user (through the previously described mechanisms) requests to disable +compression the next message MUST be sent uncompressed. This is instrumental in +preventing BEAST/CRIME attacks. This applies to both the the unary and streaming +cases. + +### Compression Levels and Algorithms + +We currently (as of July 2015) support _gzip_ and _deflate_ as algorithms (with +the possible future addition of snappy). In order to simplify the public API, +it's intended to abstract the algorithms as _compression levels_ (such as "low", +"medium", "high") that'd map to concrete algorithms and/or their settings (such +as "low" mapping to "gzip -3" and "high" mapping to "gzip -9"). However, we +can't presently (July 2015) implement said compression levels at the client side +without either a initial negotiation of capabilities or an automatic retry +mechanism. Therefore, compression levels are only supported at the server side, +which is aware of the client's capabilities by virtue of the incoming +Message-Accept-Encoding header. + +### Propagation to child RPCs + +The inheritance of the compression configuration by child RPCs is left up to the +implementation. Note that in the absence of changes to the parent channel, its +configuration will be used. + +### Test cases + +1. When a compression level is not specified for either the channel or the +message, the default channel level _none_ is considered: data MUST NOT be +compressed. +1. When per-RPC compression configuration isn't present for a message, the +channel compression configuration MUST be used. +1. When a compression method (including no compression) is specified for an +outgoing message, the message MUST be compressed accordingly. +1. A message compressed in a way not supported by its endpoint MUST fail with +INVALID\_ARGUMENT status, its associated description indicating the unsupported +condition as well as the supported ones. The returned `grpc-accept-encoding` +header MUST NOT contain the compression method (encoding) used. +1. An ill-constructed message with its [Compressed-Flag +bit](PROTOCOL-HTTP2.md#compressed-flag) +set but lacking a +"[grpc-encoding](PROTOCOL-HTTP2.md#message-encoding)" +entry different from _identity_ in its metadata MUST fail with INTERNAL status, +its associated description indicating the invalid Compressed-Flag condition. From 9a4d0893881f6811646695cc73fe9752fed308ae Mon Sep 17 00:00:00 2001 From: Yosuke Ishikawa Date: Fri, 29 Apr 2016 01:05:39 +0900 Subject: [PATCH 002/215] Fix build error caused by immutability of value type --- src/objective-c/examples/SwiftSample/ViewController.swift | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/objective-c/examples/SwiftSample/ViewController.swift b/src/objective-c/examples/SwiftSample/ViewController.swift index a21ce07978d..80d7a47917f 100644 --- a/src/objective-c/examples/SwiftSample/ViewController.swift +++ b/src/objective-c/examples/SwiftSample/ViewController.swift @@ -71,7 +71,7 @@ class ViewController: UIViewController { NSLog("2. Response trailers: \(RPC.responseTrailers)") } - RPC.requestHeaders["My-Header"] = "My value" + RPC.requestHeaders.setObject("My value", forKey: "My-Header") RPC.start() @@ -84,7 +84,7 @@ class ViewController: UIViewController { let call = GRPCCall(host: RemoteHost, path: method.HTTPPath, requestsWriter: requestsWriter) - call.requestHeaders["My-Header"] = "My value" + call.requestHeaders.setObject("My value", forKey: "My-Header") call.startWithWriteable(GRXWriteable { response, error in if let response = response as? NSData { From 454432542a6cc6145621003ecd8303c82a4e2b7b Mon Sep 17 00:00:00 2001 From: Jorge Canizales Date: Fri, 29 Apr 2016 16:19:51 -0700 Subject: [PATCH 003/215] Add a maybe-temporary way for apps to clear the channel cache --- .../GRPCClient/GRPCCall+ChannelArg.h | 5 +++++ .../GRPCClient/GRPCCall+ChannelArg.m | 4 ++++ src/objective-c/GRPCClient/private/GRPCHost.h | 2 ++ src/objective-c/GRPCClient/private/GRPCHost.m | 21 ++++++++++++++----- 4 files changed, 27 insertions(+), 5 deletions(-) diff --git a/src/objective-c/GRPCClient/GRPCCall+ChannelArg.h b/src/objective-c/GRPCClient/GRPCCall+ChannelArg.h index bd6b064f166..646bf43b547 100644 --- a/src/objective-c/GRPCClient/GRPCCall+ChannelArg.h +++ b/src/objective-c/GRPCClient/GRPCCall+ChannelArg.h @@ -32,6 +32,8 @@ */ #import "GRPCCall.h" +#include + /** * Methods to configure GRPC channel options. */ @@ -43,4 +45,7 @@ */ + (void)setUserAgentPrefix:(NSString *)userAgentPrefix forHost:(NSString *)host; ++ (void)closeOpenConnections DEPRECATED_MSG_ATTRIBUTE("The API for this feature is experimental, " + "and might be removed or modified at any " + "time."); @end diff --git a/src/objective-c/GRPCClient/GRPCCall+ChannelArg.m b/src/objective-c/GRPCClient/GRPCCall+ChannelArg.m index 5f9932d86dc..bcc3b915075 100644 --- a/src/objective-c/GRPCClient/GRPCCall+ChannelArg.m +++ b/src/objective-c/GRPCClient/GRPCCall+ChannelArg.m @@ -46,4 +46,8 @@ hostConfig.userAgentPrefix = userAgentPrefix; } ++ (void)closeOpenConnections { + [GRPCHost flushChannelCache]; +} + @end diff --git a/src/objective-c/GRPCClient/private/GRPCHost.h b/src/objective-c/GRPCClient/private/GRPCHost.h index 9220e2a33db..350c69bf8e8 100644 --- a/src/objective-c/GRPCClient/private/GRPCHost.h +++ b/src/objective-c/GRPCClient/private/GRPCHost.h @@ -41,6 +41,8 @@ struct grpc_channel_credentials; @interface GRPCHost : NSObject ++ (void)flushChannelCache; + @property(nonatomic, readonly) NSString *address; @property(nonatomic, copy, nullable) NSString *userAgentPrefix; @property(nonatomic, nullable) struct grpc_channel_credentials *channelCreds; diff --git a/src/objective-c/GRPCClient/private/GRPCHost.m b/src/objective-c/GRPCClient/private/GRPCHost.m index 43166cbb527..0358cc62365 100644 --- a/src/objective-c/GRPCClient/private/GRPCHost.m +++ b/src/objective-c/GRPCClient/private/GRPCHost.m @@ -48,6 +48,8 @@ NS_ASSUME_NONNULL_BEGIN // templates/src/core/surface/version.c.template . #define GRPC_OBJC_VERSION_STRING @"0.13.0" +static NSMutableDictionary *kHostCache; + @implementation GRPCHost { // TODO(mlumish): Investigate whether caching channels with strong links is a good idea. GRPCChannel *_channel; @@ -79,13 +81,12 @@ NS_ASSUME_NONNULL_BEGIN } // Look up the GRPCHost in the cache. - static NSMutableDictionary *hostCache; static dispatch_once_t cacheInitialization; dispatch_once(&cacheInitialization, ^{ - hostCache = [NSMutableDictionary dictionary]; + kHostCache = [NSMutableDictionary dictionary]; }); - @synchronized(hostCache) { - GRPCHost *cachedHost = hostCache[address]; + @synchronized(kHostCache) { + GRPCHost *cachedHost = kHostCache[address]; if (cachedHost) { return cachedHost; } @@ -93,12 +94,22 @@ NS_ASSUME_NONNULL_BEGIN if ((self = [super init])) { _address = address; _secure = YES; - hostCache[address] = self; + kHostCache[address] = self; } } return self; } ++ (void)flushChannelCache { + @synchronized(kHostCache) { + [kHostCache enumerateKeysAndObjectsUsingBlock:^(id _Nonnull key, + GRPCHost * _Nonnull host, + BOOL * _Nonnull stop) { + [host disconnect]; + }]; + } +} + - (nullable grpc_call *)unmanagedCallWithPath:(NSString *)path completionQueue:(GRPCCompletionQueue *)queue { GRPCChannel *channel; From fa70dacf95b93486b7dfe0c21ada90d75a5d5bcd Mon Sep 17 00:00:00 2001 From: Jorge Canizales Date: Fri, 29 Apr 2016 19:58:52 -0700 Subject: [PATCH 004/215] =?UTF-8?q?Smoke=20test=20that=20things=20still=20?= =?UTF-8?q?work=20after=20=E2=80=9CcloseOpenConnections=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/objective-c/tests/InteropTests.m | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/src/objective-c/tests/InteropTests.m b/src/objective-c/tests/InteropTests.m index 26877b1ae84..379271a312d 100644 --- a/src/objective-c/tests/InteropTests.m +++ b/src/objective-c/tests/InteropTests.m @@ -35,6 +35,7 @@ #include +#import #import #import #import @@ -312,4 +313,25 @@ [self waitForExpectationsWithTimeout:8 handler:nil]; } +- (void)testRPCAfterClosingOpenConnections { + XCTAssertNotNil(self.class.host); + __weak XCTestExpectation *expectation = + [self expectationWithDescription:@"RPC after closing connection"]; + + RMTEmpty *request = [RMTEmpty message]; + + [_service emptyCallWithRequest:request handler:^(RMTEmpty *response, NSError *error) { + XCTAssertNil(error, @"First RPC finished with unexpected error: %@", error); + + [GRPCCall closeOpenConnections]; + + [_service emptyCallWithRequest:request handler:^(RMTEmpty *response, NSError *error) { + XCTAssertNil(error, @"Second RPC finished with unexpected error: %@", error); + [expectation fulfill]; + }]; + }]; + + [self waitForExpectationsWithTimeout:4 handler:nil]; +} + @end From 2e3c9ad6dd6fbf4f7532eedd9d5d2b52e7a3eb1f Mon Sep 17 00:00:00 2001 From: Julien Boeuf Date: Tue, 19 Jan 2016 17:14:38 -0800 Subject: [PATCH 005/215] Starting the work to fix #3803. - We still need a way to bubble up this error. --- src/core/lib/security/client_auth_filter.c | 3 +- src/core/lib/security/credentials.c | 35 +++++++------ src/core/lib/security/credentials.h | 4 +- test/core/security/credentials_test.c | 49 ++++++++++--------- test/core/security/oauth2_utils.c | 3 +- .../print_google_default_creds_token.c | 3 +- 6 files changed, 54 insertions(+), 43 deletions(-) diff --git a/src/core/lib/security/client_auth_filter.c b/src/core/lib/security/client_auth_filter.c index 8b58cb86bf9..f6de877021e 100644 --- a/src/core/lib/security/client_auth_filter.c +++ b/src/core/lib/security/client_auth_filter.c @@ -98,7 +98,8 @@ static void bubble_up_error(grpc_exec_ctx *exec_ctx, grpc_call_element *elem, static void on_credentials_metadata(grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, size_t num_md, - grpc_credentials_status status) { + grpc_credentials_status status, + const char *error_details) { grpc_call_element *elem = (grpc_call_element *)user_data; call_data *calld = elem->call_data; grpc_transport_stream_op *op = &calld->op; diff --git a/src/core/lib/security/credentials.c b/src/core/lib/security/credentials.c index fd5ad3589b7..1c9832333aa 100644 --- a/src/core/lib/security/credentials.c +++ b/src/core/lib/security/credentials.c @@ -122,7 +122,7 @@ void grpc_call_credentials_get_request_metadata( grpc_credentials_metadata_cb cb, void *user_data) { if (creds == NULL || creds->vtable->get_request_metadata == NULL) { if (cb != NULL) { - cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_OK); + cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_OK, NULL); } return; } @@ -497,10 +497,10 @@ static void jwt_get_request_metadata(grpc_exec_ctx *exec_ctx, if (jwt_md != NULL) { cb(exec_ctx, user_data, jwt_md->entries, jwt_md->num_entries, - GRPC_CREDENTIALS_OK); + GRPC_CREDENTIALS_OK, NULL); grpc_credentials_md_store_unref(jwt_md); } else { - cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_ERROR); + cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_ERROR, ""); } } @@ -660,10 +660,10 @@ static void on_oauth2_token_fetcher_http_response( c->token_expiration = gpr_time_add(gpr_now(GPR_CLOCK_REALTIME), token_lifetime); r->cb(exec_ctx, r->user_data, c->access_token_md->entries, - c->access_token_md->num_entries, status); + c->access_token_md->num_entries, status, NULL); } else { c->token_expiration = gpr_inf_past(GPR_CLOCK_REALTIME); - r->cb(exec_ctx, r->user_data, NULL, 0, status); + r->cb(exec_ctx, r->user_data, NULL, 0, status, ""); } gpr_mu_unlock(&c->mu); grpc_credentials_metadata_request_destroy(r); @@ -691,7 +691,7 @@ static void oauth2_token_fetcher_get_request_metadata( } if (cached_access_token_md != NULL) { cb(exec_ctx, user_data, cached_access_token_md->entries, - cached_access_token_md->num_entries, GRPC_CREDENTIALS_OK); + cached_access_token_md->num_entries, GRPC_CREDENTIALS_OK, NULL); grpc_credentials_md_store_unref(cached_access_token_md); } else { c->fetch_func( @@ -821,7 +821,7 @@ static void on_simulated_token_fetch_done(grpc_exec_ctx *exec_ctx, (grpc_credentials_metadata_request *)user_data; grpc_md_only_test_credentials *c = (grpc_md_only_test_credentials *)r->creds; r->cb(exec_ctx, r->user_data, c->md_store->entries, c->md_store->num_entries, - GRPC_CREDENTIALS_OK); + GRPC_CREDENTIALS_OK, NULL); grpc_credentials_metadata_request_destroy(r); } @@ -837,7 +837,7 @@ static void md_only_test_get_request_metadata( grpc_executor_enqueue( grpc_closure_create(on_simulated_token_fetch_done, cb_arg), true); } else { - cb(exec_ctx, user_data, c->md_store->entries, 1, GRPC_CREDENTIALS_OK); + cb(exec_ctx, user_data, c->md_store->entries, 1, GRPC_CREDENTIALS_OK, NULL); } } @@ -870,7 +870,8 @@ static void access_token_get_request_metadata( grpc_pollset *pollset, grpc_auth_metadata_context context, grpc_credentials_metadata_cb cb, void *user_data) { grpc_access_token_credentials *c = (grpc_access_token_credentials *)creds; - cb(exec_ctx, user_data, c->access_token_md->entries, 1, GRPC_CREDENTIALS_OK); + cb(exec_ctx, user_data, c->access_token_md->entries, 1, GRPC_CREDENTIALS_OK, + NULL); } static grpc_call_credentials_vtable access_token_vtable = { @@ -973,11 +974,12 @@ static void composite_call_md_context_destroy( static void composite_call_metadata_cb(grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, size_t num_md, - grpc_credentials_status status) { + grpc_credentials_status status, + const char *error_details) { grpc_composite_call_credentials_metadata_context *ctx = (grpc_composite_call_credentials_metadata_context *)user_data; if (status != GRPC_CREDENTIALS_OK) { - ctx->cb(exec_ctx, ctx->user_data, NULL, 0, status); + ctx->cb(exec_ctx, ctx->user_data, NULL, 0, status, NULL); return; } @@ -1002,7 +1004,7 @@ static void composite_call_metadata_cb(grpc_exec_ctx *exec_ctx, void *user_data, /* We're done!. */ ctx->cb(exec_ctx, ctx->user_data, ctx->md_elems->entries, - ctx->md_elems->num_entries, GRPC_CREDENTIALS_OK); + ctx->md_elems->num_entries, GRPC_CREDENTIALS_OK, NULL); composite_call_md_context_destroy(ctx); } @@ -1122,7 +1124,7 @@ static void iam_get_request_metadata(grpc_exec_ctx *exec_ctx, void *user_data) { grpc_google_iam_credentials *c = (grpc_google_iam_credentials *)creds; cb(exec_ctx, user_data, c->iam_md->entries, c->iam_md->num_entries, - GRPC_CREDENTIALS_OK); + GRPC_CREDENTIALS_OK, NULL); } static grpc_call_credentials_vtable iam_vtable = {iam_destruct, @@ -1178,7 +1180,8 @@ static void plugin_md_request_metadata_ready(void *request, gpr_log(GPR_ERROR, "Getting metadata from plugin failed with error: %s", error_details); } - r->cb(&exec_ctx, r->user_data, NULL, 0, GRPC_CREDENTIALS_ERROR); + r->cb(&exec_ctx, r->user_data, NULL, 0, GRPC_CREDENTIALS_ERROR, + error_details); } else { size_t i; grpc_credentials_md *md_array = NULL; @@ -1190,7 +1193,7 @@ static void plugin_md_request_metadata_ready(void *request, gpr_slice_from_copied_buffer(md[i].value, md[i].value_length); } } - r->cb(&exec_ctx, r->user_data, md_array, num_md, GRPC_CREDENTIALS_OK); + r->cb(&exec_ctx, r->user_data, md_array, num_md, GRPC_CREDENTIALS_OK, NULL); if (md_array != NULL) { for (i = 0; i < num_md; i++) { gpr_slice_unref(md_array[i].key); @@ -1218,7 +1221,7 @@ static void plugin_get_request_metadata(grpc_exec_ctx *exec_ctx, c->plugin.get_metadata(c->plugin.state, context, plugin_md_request_metadata_ready, request); } else { - cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_OK); + cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_OK, NULL); } } diff --git a/src/core/lib/security/credentials.h b/src/core/lib/security/credentials.h index 0373ceaa3fc..412b6e48fc9 100644 --- a/src/core/lib/security/credentials.h +++ b/src/core/lib/security/credentials.h @@ -160,11 +160,13 @@ void grpc_credentials_md_store_unref(grpc_credentials_md_store *store); /* --- grpc_call_credentials. --- */ +/* error_details must be NULL if status is GRPC_CREDENTIALS_OK. */ typedef void (*grpc_credentials_metadata_cb)(grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, size_t num_md, - grpc_credentials_status status); + grpc_credentials_status status, + const char *error_details); typedef struct { void (*destruct)(grpc_call_credentials *c); diff --git a/test/core/security/credentials_test.c b/test/core/security/credentials_test.c index 78672932787..36a4fdae14e 100644 --- a/test/core/security/credentials_test.c +++ b/test/core/security/credentials_test.c @@ -338,13 +338,15 @@ static void check_metadata(expected_md *expected, grpc_credentials_md *md_elems, static void check_google_iam_metadata(grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, size_t num_md, - grpc_credentials_status status) { + grpc_credentials_status status, + const char *error_details) { grpc_call_credentials *c = (grpc_call_credentials *)user_data; expected_md emd[] = {{GRPC_IAM_AUTHORIZATION_TOKEN_METADATA_KEY, test_google_iam_authorization_token}, {GRPC_IAM_AUTHORITY_SELECTOR_METADATA_KEY, test_google_iam_authority_selector}}; GPR_ASSERT(status == GRPC_CREDENTIALS_OK); + GPR_ASSERT(error_details == NULL); GPR_ASSERT(num_md == 2); check_metadata(emd, md_elems, num_md); grpc_call_credentials_unref(c); @@ -362,14 +364,13 @@ static void test_google_iam_creds(void) { grpc_exec_ctx_finish(&exec_ctx); } -static void check_access_token_metadata(grpc_exec_ctx *exec_ctx, - void *user_data, - grpc_credentials_md *md_elems, - size_t num_md, - grpc_credentials_status status) { +static void check_access_token_metadata( + grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, + size_t num_md, grpc_credentials_status status, const char *error_details) { grpc_call_credentials *c = (grpc_call_credentials *)user_data; expected_md emd[] = {{GRPC_AUTHORIZATION_METADATA_KEY, "Bearer blah"}}; GPR_ASSERT(status == GRPC_CREDENTIALS_OK); + GPR_ASSERT(error_details == NULL); GPR_ASSERT(num_md == 1); check_metadata(emd, md_elems, num_md); grpc_call_credentials_unref(c); @@ -418,7 +419,7 @@ static void test_channel_oauth2_composite_creds(void) { static void check_oauth2_google_iam_composite_metadata( grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, - size_t num_md, grpc_credentials_status status) { + size_t num_md, grpc_credentials_status status, const char *error_details) { grpc_call_credentials *c = (grpc_call_credentials *)user_data; expected_md emd[] = { {GRPC_AUTHORIZATION_METADATA_KEY, test_oauth2_bearer_token}, @@ -427,6 +428,7 @@ static void check_oauth2_google_iam_composite_metadata( {GRPC_IAM_AUTHORITY_SELECTOR_METADATA_KEY, test_google_iam_authority_selector}}; GPR_ASSERT(status == GRPC_CREDENTIALS_OK); + GPR_ASSERT(error_details == NULL); GPR_ASSERT(num_md == 3); check_metadata(emd, md_elems, num_md); grpc_call_credentials_unref(c); @@ -511,8 +513,9 @@ static void test_channel_oauth2_google_iam_composite_creds(void) { static void on_oauth2_creds_get_metadata_success( grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, - size_t num_md, grpc_credentials_status status) { + size_t num_md, grpc_credentials_status status, const char *error_details) { GPR_ASSERT(status == GRPC_CREDENTIALS_OK); + GPR_ASSERT(error_details == NULL); GPR_ASSERT(num_md == 1); GPR_ASSERT(gpr_slice_str_cmp(md_elems[0].key, "authorization") == 0); GPR_ASSERT(gpr_slice_str_cmp(md_elems[0].value, @@ -524,7 +527,7 @@ static void on_oauth2_creds_get_metadata_success( static void on_oauth2_creds_get_metadata_failure( grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, - size_t num_md, grpc_credentials_status status) { + size_t num_md, grpc_credentials_status status, const char *error_details) { GPR_ASSERT(status == GRPC_CREDENTIALS_ERROR); GPR_ASSERT(num_md == 0); GPR_ASSERT(user_data != NULL); @@ -760,14 +763,13 @@ static char *encode_and_sign_jwt_should_not_be_called( return NULL; } -static void on_jwt_creds_get_metadata_success(grpc_exec_ctx *exec_ctx, - void *user_data, - grpc_credentials_md *md_elems, - size_t num_md, - grpc_credentials_status status) { +static void on_jwt_creds_get_metadata_success( + grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, + size_t num_md, grpc_credentials_status status, const char *error_details) { char *expected_md_value; gpr_asprintf(&expected_md_value, "Bearer %s", test_signed_jwt); GPR_ASSERT(status == GRPC_CREDENTIALS_OK); + GPR_ASSERT(error_details == NULL); GPR_ASSERT(num_md == 1); GPR_ASSERT(gpr_slice_str_cmp(md_elems[0].key, "authorization") == 0); GPR_ASSERT(gpr_slice_str_cmp(md_elems[0].value, expected_md_value) == 0); @@ -776,11 +778,9 @@ static void on_jwt_creds_get_metadata_success(grpc_exec_ctx *exec_ctx, gpr_free(expected_md_value); } -static void on_jwt_creds_get_metadata_failure(grpc_exec_ctx *exec_ctx, - void *user_data, - grpc_credentials_md *md_elems, - size_t num_md, - grpc_credentials_status status) { +static void on_jwt_creds_get_metadata_failure( + grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, + size_t num_md, grpc_credentials_status status, const char *error_details) { GPR_ASSERT(status == GRPC_CREDENTIALS_ERROR); GPR_ASSERT(num_md == 0); GPR_ASSERT(user_data != NULL); @@ -1024,6 +1024,8 @@ static void plugin_get_metadata_success(void *state, cb(user_data, md, GPR_ARRAY_SIZE(md), GRPC_STATUS_OK, NULL); } +static const char *plugin_error_details = "Could not get metadata for plugin."; + static void plugin_get_metadata_failure(void *state, grpc_auth_metadata_context context, grpc_credentials_plugin_metadata_cb cb, @@ -1034,13 +1036,12 @@ static void plugin_get_metadata_failure(void *state, GPR_ASSERT(context.channel_auth_context == NULL); GPR_ASSERT(context.reserved == NULL); *s = PLUGIN_GET_METADATA_CALLED_STATE; - cb(user_data, NULL, 0, GRPC_STATUS_UNAUTHENTICATED, - "Could not get metadata for plugin."); + cb(user_data, NULL, 0, GRPC_STATUS_UNAUTHENTICATED, plugin_error_details); } static void on_plugin_metadata_received_success( grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, - size_t num_md, grpc_credentials_status status) { + size_t num_md, grpc_credentials_status status, const char *error_details) { size_t i = 0; GPR_ASSERT(user_data == NULL); GPR_ASSERT(md_elems != NULL); @@ -1053,11 +1054,13 @@ static void on_plugin_metadata_received_success( static void on_plugin_metadata_received_failure( grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, - size_t num_md, grpc_credentials_status status) { + size_t num_md, grpc_credentials_status status, const char *error_details) { GPR_ASSERT(user_data == NULL); GPR_ASSERT(md_elems == NULL); GPR_ASSERT(num_md == 0); GPR_ASSERT(status == GRPC_CREDENTIALS_ERROR); + GPR_ASSERT(error_details != NULL); + GPR_ASSERT(strcmp(error_details, plugin_error_details) == 0); } static void plugin_destroy(void *state) { diff --git a/test/core/security/oauth2_utils.c b/test/core/security/oauth2_utils.c index 20815d184cd..10155fccf59 100644 --- a/test/core/security/oauth2_utils.c +++ b/test/core/security/oauth2_utils.c @@ -53,7 +53,8 @@ typedef struct { static void on_oauth2_response(grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, size_t num_md, - grpc_credentials_status status) { + grpc_credentials_status status, + const char *error_details) { oauth2_request *request = user_data; char *token = NULL; gpr_slice token_slice; diff --git a/test/core/security/print_google_default_creds_token.c b/test/core/security/print_google_default_creds_token.c index 99bce4fbdfb..be900d84498 100644 --- a/test/core/security/print_google_default_creds_token.c +++ b/test/core/security/print_google_default_creds_token.c @@ -53,7 +53,8 @@ typedef struct { static void on_metadata_response(grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, size_t num_md, - grpc_credentials_status status) { + grpc_credentials_status status, + const char *error_details) { synchronizer *sync = user_data; if (status == GRPC_CREDENTIALS_ERROR) { fprintf(stderr, "Fetching token failed.\n"); From ef96264d8724a71f4f1fef5ebaf361d28b1a1752 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Wed, 18 May 2016 22:57:17 -0700 Subject: [PATCH 006/215] Support SO_REUSEPORT --- include/grpc/impl/codegen/grpc_types.h | 2 + .../chttp2/server/insecure/server_chttp2.c | 3 +- .../server/secure/server_secure_chttp2.c | 3 +- .../lib/iomgr/socket_utils_common_posix.c | 22 ++++ src/core/lib/iomgr/socket_utils_posix.h | 3 + src/core/lib/iomgr/tcp_server.h | 3 + src/core/lib/iomgr/tcp_server_posix.c | 122 ++++++++++++++++-- 7 files changed, 143 insertions(+), 15 deletions(-) diff --git a/include/grpc/impl/codegen/grpc_types.h b/include/grpc/impl/codegen/grpc_types.h index af3d0c21916..cd3cae71b6a 100644 --- a/include/grpc/impl/codegen/grpc_types.h +++ b/include/grpc/impl/codegen/grpc_types.h @@ -154,6 +154,8 @@ typedef struct { #define GRPC_SSL_TARGET_NAME_OVERRIDE_ARG "grpc.ssl_target_name_override" /* Maximum metadata size */ #define GRPC_ARG_MAX_METADATA_SIZE "grpc.max_metadata_size" +/** If non-zero, allow the use of SO_REUSEPORT if it's available (default 1) */ +#define GRPC_ARG_ALLOW_REUSEPORT "grpc.so_reuseport" /** Result of a grpc call. If the caller satisfies the prerequisites of a particular operation, the grpc_call_error returned will be GRPC_CALL_OK. diff --git a/src/core/ext/transport/chttp2/server/insecure/server_chttp2.c b/src/core/ext/transport/chttp2/server/insecure/server_chttp2.c index d1a58b66211..08a09a933a6 100644 --- a/src/core/ext/transport/chttp2/server/insecure/server_chttp2.c +++ b/src/core/ext/transport/chttp2/server/insecure/server_chttp2.c @@ -102,7 +102,8 @@ int grpc_server_add_insecure_http2_port(grpc_server *server, const char *addr) { goto error; } - err = grpc_tcp_server_create(NULL, &tcp); + err = + grpc_tcp_server_create(NULL, grpc_server_get_channel_args(server), &tcp); if (err != GRPC_ERROR_NONE) { goto error; } diff --git a/src/core/ext/transport/chttp2/server/secure/server_secure_chttp2.c b/src/core/ext/transport/chttp2/server/secure/server_secure_chttp2.c index 629959b7c03..64162cc7857 100644 --- a/src/core/ext/transport/chttp2/server/secure/server_secure_chttp2.c +++ b/src/core/ext/transport/chttp2/server/secure/server_secure_chttp2.c @@ -208,7 +208,8 @@ int grpc_server_add_secure_http2_port(grpc_server *server, const char *addr, state = gpr_malloc(sizeof(*state)); memset(state, 0, sizeof(*state)); grpc_closure_init(&state->destroy_closure, destroy_done, state); - err = grpc_tcp_server_create(&state->destroy_closure, &tcp); + err = grpc_tcp_server_create(&state->destroy_closure, + grpc_server_get_channel_args(server), &tcp); if (err != GRPC_ERROR_NONE) { goto error; } diff --git a/src/core/lib/iomgr/socket_utils_common_posix.c b/src/core/lib/iomgr/socket_utils_common_posix.c index d1721c910c4..bd8ef00c859 100644 --- a/src/core/lib/iomgr/socket_utils_common_posix.c +++ b/src/core/lib/iomgr/socket_utils_common_posix.c @@ -154,6 +154,28 @@ grpc_error *grpc_set_socket_reuse_addr(int fd, int reuse) { return GRPC_ERROR_NONE; } +/* set a socket to reuse old addresses */ +grpc_error *grpc_set_socket_reuse_port(int fd, int reuse) { +#ifndef SO_REUSEPORT + return GRPC_ERROR_CREATE("SO_REUSEPORT unavailable on compiling system"); +#else + int val = (reuse != 0); + int newval; + socklen_t intlen = sizeof(newval); + if (0 != setsockopt(fd, SOL_SOCKET, SO_REUSEPORT, &val, sizeof(val))) { + return GRPC_OS_ERROR(errno, "setsockopt(SO_REUSEPORT)"); + } + if (0 != getsockopt(fd, SOL_SOCKET, SO_REUSEPORT, &newval, &intlen)) { + return GRPC_OS_ERROR(errno, "getsockopt(SO_REUSEPORT)"); + } + if ((newval != 0) != val) { + return GRPC_ERROR_CREATE("Failed to set SO_REUSEPORT"); + } + + return GRPC_ERROR_NONE; +#endif +} + /* disable nagle */ grpc_error *grpc_set_socket_low_latency(int fd, int low_latency) { int val = (low_latency != 0); diff --git a/src/core/lib/iomgr/socket_utils_posix.h b/src/core/lib/iomgr/socket_utils_posix.h index 4e66cbc0c58..607fbc7a791 100644 --- a/src/core/lib/iomgr/socket_utils_posix.h +++ b/src/core/lib/iomgr/socket_utils_posix.h @@ -55,6 +55,9 @@ grpc_error *grpc_set_socket_reuse_addr(int fd, int reuse); /* disable nagle */ grpc_error *grpc_set_socket_low_latency(int fd, int low_latency); +/* set SO_REUSEPORT */ +grpc_error *grpc_set_socket_reuse_port(int fd, int reuse); + /* Returns true if this system can create AF_INET6 sockets bound to ::1. The value is probed once, and cached for the life of the process. diff --git a/src/core/lib/iomgr/tcp_server.h b/src/core/lib/iomgr/tcp_server.h index 924121f0c79..79f0242b26d 100644 --- a/src/core/lib/iomgr/tcp_server.h +++ b/src/core/lib/iomgr/tcp_server.h @@ -34,6 +34,8 @@ #ifndef GRPC_CORE_LIB_IOMGR_TCP_SERVER_H #define GRPC_CORE_LIB_IOMGR_TCP_SERVER_H +#include + #include "src/core/lib/iomgr/closure.h" #include "src/core/lib/iomgr/endpoint.h" @@ -58,6 +60,7 @@ typedef void (*grpc_tcp_server_cb)(grpc_exec_ctx *exec_ctx, void *arg, If shutdown_complete is not NULL, it will be used by grpc_tcp_server_unref() when the ref count reaches zero. */ grpc_error *grpc_tcp_server_create(grpc_closure *shutdown_complete, + const grpc_channel_args *args, grpc_tcp_server **server); /* Start listening to bound ports */ diff --git a/src/core/lib/iomgr/tcp_server_posix.c b/src/core/lib/iomgr/tcp_server_posix.c index db70fada96a..7047934780b 100644 --- a/src/core/lib/iomgr/tcp_server_posix.c +++ b/src/core/lib/iomgr/tcp_server_posix.c @@ -112,8 +112,10 @@ struct grpc_tcp_server { /* destroyed port count: how many ports are completely destroyed */ size_t destroyed_ports; - /* is this server shutting down? (boolean) */ - int shutdown; + /* is this server shutting down? */ + bool shutdown; + /* use SO_REUSEPORT */ + bool so_reuseport; /* linked list of server ports */ grpc_tcp_listener *head; @@ -132,14 +134,42 @@ struct grpc_tcp_server { size_t pollset_count; }; +static gpr_once check_init = GPR_ONCE_INIT; +static bool has_so_reuseport; + +static void init(void) { + int s = socket(AF_INET, SOCK_STREAM, 0); + if (s >= 0) { + has_so_reuseport = GRPC_LOG_IF_ERROR("check for SO_REUSEPORT", + grpc_set_socket_reuse_port(s, 1)); + close(s); + } +} + grpc_error *grpc_tcp_server_create(grpc_closure *shutdown_complete, + const grpc_channel_args *args, grpc_tcp_server **server) { + gpr_once_init(&check_init, init); + grpc_tcp_server *s = gpr_malloc(sizeof(grpc_tcp_server)); + s->so_reuseport = has_so_reuseport; + for (size_t i = 0; i < (args == NULL ? 0 : args->num_args); i++) { + if (0 == strcmp(GRPC_ARG_ALLOW_REUSEPORT, args->args[i].key)) { + if (args->args[i].type == GRPC_ARG_INTEGER) { + s->so_reuseport = + has_so_reuseport && (args->args[i].value.integer != 0); + } else { + gpr_free(s); + return GRPC_ERROR_CREATE(GRPC_ARG_ALLOW_REUSEPORT + " must be an integer"); + } + } + } gpr_ref_init(&s->refs, 1); gpr_mu_init(&s->mu); s->active_ports = 0; s->destroyed_ports = 0; - s->shutdown = 0; + s->shutdown = false; s->shutdown_starting.head = NULL; s->shutdown_starting.tail = NULL; s->shutdown_complete = shutdown_complete; @@ -214,7 +244,7 @@ static void tcp_server_destroy(grpc_exec_ctx *exec_ctx, grpc_tcp_server *s) { gpr_mu_lock(&s->mu); GPR_ASSERT(!s->shutdown); - s->shutdown = 1; + s->shutdown = true; /* shutdown all fd's */ if (s->active_ports) { @@ -264,13 +294,19 @@ static int get_max_accept_queue_size(void) { /* Prepare a recently-created socket for listening. */ static grpc_error *prepare_socket(int fd, const struct sockaddr *addr, - size_t addr_len, int *port) { + size_t addr_len, bool so_reuseport, + int *port) { struct sockaddr_storage sockname_temp; socklen_t sockname_len; grpc_error *err = GRPC_ERROR_NONE; GPR_ASSERT(fd >= 0); + if (so_reuseport) { + err = grpc_set_socket_reuse_port(fd, 1); + if (err != GRPC_ERROR_NONE) goto error; + } + err = grpc_set_socket_nonblocking(fd, 1); if (err != GRPC_ERROR_NONE) goto error; err = grpc_set_socket_cloexec(fd, 1); @@ -397,7 +433,7 @@ static grpc_error *add_socket_to_server(grpc_tcp_server *s, int fd, char *addr_str; char *name; - grpc_error *err = prepare_socket(fd, addr, addr_len, &port); + grpc_error *err = prepare_socket(fd, addr, addr_len, s->so_reuseport, &port); if (err == GRPC_ERROR_NONE) { GPR_ASSERT(port > 0); grpc_sockaddr_to_string(&addr_str, (struct sockaddr *)&addr, 1); @@ -433,6 +469,51 @@ static grpc_error *add_socket_to_server(grpc_tcp_server *s, int fd, return err; } +static grpc_error *clone_port(grpc_tcp_listener *listener, unsigned count) { + grpc_tcp_listener *sp = NULL; + char *addr_str; + char *name; + grpc_error *err; + + for (grpc_tcp_listener *l = listener->next; l && l->is_sibling; l = l->next) { + l->fd_index += count; + } + + for (unsigned i = 0; i < count; i++) { + int fd, port; + grpc_dualstack_mode dsmode; + err = grpc_create_dualstack_socket(&listener->addr.sockaddr, SOCK_STREAM, 0, + &dsmode, &fd); + if (err != GRPC_ERROR_NONE) return err; + err = prepare_socket(fd, &listener->addr.sockaddr, listener->addr_len, true, + &port); + if (err != GRPC_ERROR_NONE) return err; + grpc_sockaddr_to_string(&addr_str, &listener->addr.sockaddr, 1); + gpr_asprintf(&name, "tcp-server-listener:%s/clone-%d", addr_str, i); + sp = gpr_malloc(sizeof(grpc_tcp_listener)); + sp->next = listener->next; + listener->next = sp; + sp->server = listener->server; + sp->fd = fd; + sp->emfd = grpc_fd_create(fd, name); + memcpy(sp->addr.untyped, listener->addr.untyped, listener->addr_len); + sp->addr_len = listener->addr_len; + sp->port = port; + sp->port_index = listener->port_index; + sp->fd_index = listener->fd_index + count - i; + sp->is_sibling = 1; + sp->sibling = listener->is_sibling ? listener->sibling : listener; + GPR_ASSERT(sp->emfd); + while (listener->server->tail->next != NULL) { + listener->server->tail = listener->server->tail->next; + } + gpr_free(addr_str); + gpr_free(name); + } + + return GRPC_ERROR_NONE; +} + grpc_error *grpc_tcp_server_add_port(grpc_tcp_server *s, const void *addr, size_t addr_len, int *out_port) { grpc_tcp_listener *sp; @@ -589,14 +670,29 @@ void grpc_tcp_server_start(grpc_exec_ctx *exec_ctx, grpc_tcp_server *s, s->on_accept_cb_arg = on_accept_cb_arg; s->pollsets = pollsets; s->pollset_count = pollset_count; - for (sp = s->head; sp; sp = sp->next) { - for (i = 0; i < pollset_count; i++) { - grpc_pollset_add_fd(exec_ctx, pollsets[i], sp->emfd); + sp = s->head; + while (sp != NULL) { + if (s->so_reuseport && pollset_count > 1) { + GPR_ASSERT(GRPC_LOG_IF_ERROR( + "clone_port", clone_port(sp, (unsigned)(pollset_count - 1)))); + for (i = 0; i < pollset_count; i++) { + grpc_pollset_add_fd(exec_ctx, pollsets[i], sp->emfd); + sp->read_closure.cb = on_read; + sp->read_closure.cb_arg = sp; + grpc_fd_notify_on_read(exec_ctx, sp->emfd, &sp->read_closure); + s->active_ports++; + sp = sp->next; + } + } else { + for (i = 0; i < pollset_count; i++) { + grpc_pollset_add_fd(exec_ctx, pollsets[i], sp->emfd); + } + sp->read_closure.cb = on_read; + sp->read_closure.cb_arg = sp; + grpc_fd_notify_on_read(exec_ctx, sp->emfd, &sp->read_closure); + s->active_ports++; + sp = sp->next; } - sp->read_closure.cb = on_read; - sp->read_closure.cb_arg = sp; - grpc_fd_notify_on_read(exec_ctx, sp->emfd, &sp->read_closure); - s->active_ports++; } gpr_mu_unlock(&s->mu); } From c49464de3ed6108956561128b006c78777bd6db7 Mon Sep 17 00:00:00 2001 From: Julien Boeuf Date: Wed, 18 May 2016 23:11:50 -0700 Subject: [PATCH 007/215] replacing cancel op by close op in order to plumb error message. --- src/core/lib/security/client_auth_filter.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/src/core/lib/security/client_auth_filter.c b/src/core/lib/security/client_auth_filter.c index f6de877021e..3908b734a27 100644 --- a/src/core/lib/security/client_auth_filter.c +++ b/src/core/lib/security/client_auth_filter.c @@ -91,7 +91,8 @@ static void bubble_up_error(grpc_exec_ctx *exec_ctx, grpc_call_element *elem, grpc_status_code status, const char *error_msg) { call_data *calld = elem->call_data; gpr_log(GPR_ERROR, "Client side authentication failure: %s", error_msg); - grpc_transport_stream_op_add_cancellation(&calld->op, status); + gpr_slice error_slice = gpr_slice_from_copied_string(error_msg); + grpc_transport_stream_op_add_close(&calld->op, status, &error_slice); grpc_call_next_op(exec_ctx, elem, &calld->op); } @@ -108,7 +109,9 @@ static void on_credentials_metadata(grpc_exec_ctx *exec_ctx, void *user_data, reset_auth_metadata_context(&calld->auth_md_context); if (status != GRPC_CREDENTIALS_OK) { bubble_up_error(exec_ctx, elem, GRPC_STATUS_UNAUTHENTICATED, - "Credentials failed to get metadata."); + (error_details != NULL && strlen(error_details) > 0) + ? error_details + : "Credentials failed to get metadata."); return; } GPR_ASSERT(num_md <= MAX_CREDENTIALS_METADATA_COUNT); From 644da9857300151b27a1f297c40d971597a0845f Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 19 May 2016 09:19:28 -0700 Subject: [PATCH 008/215] Fixes --- src/core/lib/iomgr/tcp_server_posix.c | 1 + test/core/iomgr/tcp_server_posix_test.c | 10 +++++----- test/core/surface/concurrent_connectivity_test.c | 2 +- test/core/surface/server_chttp2_test.c | 8 +++++++- test/core/surface/server_test.c | 10 ++++++++-- test/core/util/test_tcp_server.c | 2 +- third_party/protobuf | 2 +- 7 files changed, 24 insertions(+), 11 deletions(-) diff --git a/src/core/lib/iomgr/tcp_server_posix.c b/src/core/lib/iomgr/tcp_server_posix.c index 7047934780b..83627e59d9d 100644 --- a/src/core/lib/iomgr/tcp_server_posix.c +++ b/src/core/lib/iomgr/tcp_server_posix.c @@ -488,6 +488,7 @@ static grpc_error *clone_port(grpc_tcp_listener *listener, unsigned count) { err = prepare_socket(fd, &listener->addr.sockaddr, listener->addr_len, true, &port); if (err != GRPC_ERROR_NONE) return err; + listener->server->nports++; grpc_sockaddr_to_string(&addr_str, &listener->addr.sockaddr, 1); gpr_asprintf(&name, "tcp-server-listener:%s/clone-%d", addr_str, i); sp = gpr_malloc(sizeof(grpc_tcp_listener)); diff --git a/test/core/iomgr/tcp_server_posix_test.c b/test/core/iomgr/tcp_server_posix_test.c index a37ff1773b2..54437f6a9bd 100644 --- a/test/core/iomgr/tcp_server_posix_test.c +++ b/test/core/iomgr/tcp_server_posix_test.c @@ -128,7 +128,7 @@ static void on_connect(grpc_exec_ctx *exec_ctx, void *arg, grpc_endpoint *tcp, static void test_no_op(void) { grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; grpc_tcp_server *s; - GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, &s)); + GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, NULL, &s)); grpc_tcp_server_unref(&exec_ctx, s); grpc_exec_ctx_finish(&exec_ctx); } @@ -136,7 +136,7 @@ static void test_no_op(void) { static void test_no_op_with_start(void) { grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; grpc_tcp_server *s; - GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, &s)); + GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, NULL, &s)); LOG_TEST("test_no_op_with_start"); grpc_tcp_server_start(&exec_ctx, s, NULL, 0, on_connect, NULL); grpc_tcp_server_unref(&exec_ctx, s); @@ -147,7 +147,7 @@ static void test_no_op_with_port(void) { grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; struct sockaddr_in addr; grpc_tcp_server *s; - GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, &s)); + GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, NULL, &s)); LOG_TEST("test_no_op_with_port"); memset(&addr, 0, sizeof(addr)); @@ -165,7 +165,7 @@ static void test_no_op_with_port_and_start(void) { grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; struct sockaddr_in addr; grpc_tcp_server *s; - GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, &s)); + GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, NULL, &s)); LOG_TEST("test_no_op_with_port_and_start"); int port; @@ -225,7 +225,7 @@ static void test_connect(unsigned n) { unsigned svr1_fd_count; int svr1_port; grpc_tcp_server *s; - GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, &s)); + GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, NULL, &s)); unsigned i; server_weak_ref weak_ref; server_weak_ref_init(&weak_ref); diff --git a/test/core/surface/concurrent_connectivity_test.c b/test/core/surface/concurrent_connectivity_test.c index 74e3183c26b..6d7d0413b28 100644 --- a/test/core/surface/concurrent_connectivity_test.c +++ b/test/core/surface/concurrent_connectivity_test.c @@ -112,7 +112,7 @@ void bad_server_thread(void *vargs) { socklen_t addr_len = sizeof(addr); int port; grpc_tcp_server *s; - grpc_error *error = grpc_tcp_server_create(NULL, &s); + grpc_error *error = grpc_tcp_server_create(NULL, NULL, &s); GPR_ASSERT(error == GRPC_ERROR_NONE); memset(&addr, 0, sizeof(addr)); addr.ss_family = AF_INET; diff --git a/test/core/surface/server_chttp2_test.c b/test/core/surface/server_chttp2_test.c index f42ca9f9cdf..40cfa6b5989 100644 --- a/test/core/surface/server_chttp2_test.c +++ b/test/core/surface/server_chttp2_test.c @@ -49,10 +49,16 @@ void test_unparsable_target(void) { } void test_add_same_port_twice() { + grpc_arg a; + a.type = GRPC_ARG_INTEGER; + a.key = GRPC_ARG_ALLOW_REUSEPORT; + a.value.integer = 0; + grpc_channel_args args = {1, &a}; + int port = grpc_pick_unused_port_or_die(); char *addr = NULL; grpc_completion_queue *cq = grpc_completion_queue_create(NULL); - grpc_server *server = grpc_server_create(NULL, NULL); + grpc_server *server = grpc_server_create(&args, NULL); grpc_server_credentials *fake_creds = grpc_fake_transport_security_server_credentials_create(); gpr_join_host_port(&addr, "localhost", port); diff --git a/test/core/surface/server_test.c b/test/core/surface/server_test.c index 3d2e25379a0..d1e646a9f15 100644 --- a/test/core/surface/server_test.c +++ b/test/core/surface/server_test.c @@ -76,9 +76,15 @@ void test_request_call_on_no_server_cq(void) { } void test_bind_server_twice(void) { + grpc_arg a; + a.type = GRPC_ARG_INTEGER; + a.key = GRPC_ARG_ALLOW_REUSEPORT; + a.value.integer = 0; + grpc_channel_args args = {1, &a}; + char *addr; - grpc_server *server1 = grpc_server_create(NULL, NULL); - grpc_server *server2 = grpc_server_create(NULL, NULL); + grpc_server *server1 = grpc_server_create(&args, NULL); + grpc_server *server2 = grpc_server_create(&args, NULL); grpc_completion_queue *cq = grpc_completion_queue_create(NULL); int port = grpc_pick_unused_port_or_die(); gpr_asprintf(&addr, "[::]:%d", port); diff --git a/test/core/util/test_tcp_server.c b/test/core/util/test_tcp_server.c index 27c16fc764a..c1d307fc779 100644 --- a/test/core/util/test_tcp_server.c +++ b/test/core/util/test_tcp_server.c @@ -73,7 +73,7 @@ void test_tcp_server_start(test_tcp_server *server, int port) { memset(&addr.sin_addr, 0, sizeof(addr.sin_addr)); grpc_error *error = - grpc_tcp_server_create(&server->shutdown_complete, &server->tcp_server); + grpc_tcp_server_create(&server->shutdown_complete, NULL, &server->tcp_server); GPR_ASSERT(error == GRPC_ERROR_NONE); error = grpc_tcp_server_add_port(server->tcp_server, &addr, sizeof(addr), &port_added); diff --git a/third_party/protobuf b/third_party/protobuf index a1938b2aa9c..d5fb408ddc2 160000 --- a/third_party/protobuf +++ b/third_party/protobuf @@ -1 +1 @@ -Subproject commit a1938b2aa9ca86ce7ce50c27ff9737c1008d2a03 +Subproject commit d5fb408ddc281ffcadeb08699e65bb694656d0bd From e48b1bc011e2b5783ce75f4122dfd5aba01f0d97 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 11 May 2016 15:17:22 -0700 Subject: [PATCH 009/215] Base changes. Create ev_epoll_posix.{c,h} files by making a copy of ev_poll_and_epoll.c file --- BUILD | 6 + Makefile | 2 + binding.gyp | 1 + build.yaml | 2 + config.m4 | 1 + gRPC.podspec | 3 + grpc.gemspec | 2 + package.xml | 2 + src/core/lib/iomgr/ev_epoll_posix.c | 1733 +++++++++++++++++ src/core/lib/iomgr/ev_epoll_posix.h | 41 + src/core/lib/iomgr/ev_posix.c | 3 +- src/python/grpcio/grpc_core_dependencies.py | 1 + tools/doxygen/Doxyfile.core.internal | 2 + tools/run_tests/sources_and_headers.json | 3 + vsprojects/vcxproj/grpc/grpc.vcxproj | 3 + vsprojects/vcxproj/grpc/grpc.vcxproj.filters | 6 + .../grpc_unsecure/grpc_unsecure.vcxproj | 3 + .../grpc_unsecure.vcxproj.filters | 6 + 18 files changed, 1819 insertions(+), 1 deletion(-) create mode 100644 src/core/lib/iomgr/ev_epoll_posix.c create mode 100644 src/core/lib/iomgr/ev_epoll_posix.h diff --git a/BUILD b/BUILD index fd03d52e3cb..0be8f27a01b 100644 --- a/BUILD +++ b/BUILD @@ -178,6 +178,7 @@ cc_library( "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", @@ -321,6 +322,7 @@ cc_library( "src/core/lib/iomgr/endpoint.c", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", "src/core/lib/iomgr/exec_ctx.c", @@ -546,6 +548,7 @@ cc_library( "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", @@ -666,6 +669,7 @@ cc_library( "src/core/lib/iomgr/endpoint.c", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", "src/core/lib/iomgr/exec_ctx.c", @@ -1358,6 +1362,7 @@ objc_library( "src/core/lib/iomgr/endpoint.c", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", "src/core/lib/iomgr/exec_ctx.c", @@ -1562,6 +1567,7 @@ objc_library( "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", diff --git a/Makefile b/Makefile index 817fcd072d7..29ebc0e5adf 100644 --- a/Makefile +++ b/Makefile @@ -2486,6 +2486,7 @@ LIBGRPC_SRC = \ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ + src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ @@ -2840,6 +2841,7 @@ LIBGRPC_UNSECURE_SRC = \ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ + src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ diff --git a/binding.gyp b/binding.gyp index 7b187070005..89774ead4da 100644 --- a/binding.gyp +++ b/binding.gyp @@ -581,6 +581,7 @@ 'src/core/lib/iomgr/endpoint.c', 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', + 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', 'src/core/lib/iomgr/exec_ctx.c', diff --git a/build.yaml b/build.yaml index 429dbb3351a..7ba65332972 100644 --- a/build.yaml +++ b/build.yaml @@ -165,6 +165,7 @@ filegroups: - src/core/lib/iomgr/closure.h - src/core/lib/iomgr/endpoint.h - src/core/lib/iomgr/endpoint_pair.h + - src/core/lib/iomgr/ev_epoll_posix.h - src/core/lib/iomgr/ev_poll_posix.h - src/core/lib/iomgr/ev_posix.h - src/core/lib/iomgr/exec_ctx.h @@ -239,6 +240,7 @@ filegroups: - src/core/lib/iomgr/endpoint.c - src/core/lib/iomgr/endpoint_pair_posix.c - src/core/lib/iomgr/endpoint_pair_windows.c + - src/core/lib/iomgr/ev_epoll_posix.c - src/core/lib/iomgr/ev_poll_posix.c - src/core/lib/iomgr/ev_posix.c - src/core/lib/iomgr/exec_ctx.c diff --git a/config.m4 b/config.m4 index 29df77ce1db..6987c741541 100644 --- a/config.m4 +++ b/config.m4 @@ -100,6 +100,7 @@ if test "$PHP_GRPC" != "no"; then src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ + src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ diff --git a/gRPC.podspec b/gRPC.podspec index cc8ba3de94e..3b4dd52380e 100644 --- a/gRPC.podspec +++ b/gRPC.podspec @@ -181,6 +181,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/closure.h', 'src/core/lib/iomgr/endpoint.h', 'src/core/lib/iomgr/endpoint_pair.h', + 'src/core/lib/iomgr/ev_epoll_posix.h', 'src/core/lib/iomgr/ev_poll_posix.h', 'src/core/lib/iomgr/ev_posix.h', 'src/core/lib/iomgr/exec_ctx.h', @@ -358,6 +359,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/endpoint.c', 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', + 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', 'src/core/lib/iomgr/exec_ctx.c', @@ -546,6 +548,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/closure.h', 'src/core/lib/iomgr/endpoint.h', 'src/core/lib/iomgr/endpoint_pair.h', + 'src/core/lib/iomgr/ev_epoll_posix.h', 'src/core/lib/iomgr/ev_poll_posix.h', 'src/core/lib/iomgr/ev_posix.h', 'src/core/lib/iomgr/exec_ctx.h', diff --git a/grpc.gemspec b/grpc.gemspec index ae7f9b7d2ee..71cccb6ca8d 100755 --- a/grpc.gemspec +++ b/grpc.gemspec @@ -190,6 +190,7 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/closure.h ) s.files += %w( src/core/lib/iomgr/endpoint.h ) s.files += %w( src/core/lib/iomgr/endpoint_pair.h ) + s.files += %w( src/core/lib/iomgr/ev_epoll_posix.h ) s.files += %w( src/core/lib/iomgr/ev_poll_posix.h ) s.files += %w( src/core/lib/iomgr/ev_posix.h ) s.files += %w( src/core/lib/iomgr/exec_ctx.h ) @@ -337,6 +338,7 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/endpoint.c ) s.files += %w( src/core/lib/iomgr/endpoint_pair_posix.c ) s.files += %w( src/core/lib/iomgr/endpoint_pair_windows.c ) + s.files += %w( src/core/lib/iomgr/ev_epoll_posix.c ) s.files += %w( src/core/lib/iomgr/ev_poll_posix.c ) s.files += %w( src/core/lib/iomgr/ev_posix.c ) s.files += %w( src/core/lib/iomgr/exec_ctx.c ) diff --git a/package.xml b/package.xml index 507a2a7ed6c..0fc5d0dee47 100644 --- a/package.xml +++ b/package.xml @@ -197,6 +197,7 @@ + @@ -344,6 +345,7 @@ + diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c new file mode 100644 index 00000000000..ce8d3981b3f --- /dev/null +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -0,0 +1,1733 @@ +/* + * + * Copyright 2016, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include + +#ifdef GPR_POSIX_SOCKET + +#include "src/core/lib/iomgr/ev_epoll_posix.h" + +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "src/core/lib/iomgr/iomgr_internal.h" +#include "src/core/lib/iomgr/wakeup_fd_posix.h" +#include "src/core/lib/profiling/timers.h" +#include "src/core/lib/support/block_annotate.h" + +/******************************************************************************* + * FD declarations + */ + +/* TODO(sreek) : Check if grpc_fd_watcher is needed (and if so, check if we can + * share this between ev_poll_posix.h and ev_epoll_posix versions */ + +typedef struct grpc_fd_watcher { + struct grpc_fd_watcher *next; + struct grpc_fd_watcher *prev; + grpc_pollset *pollset; + grpc_pollset_worker *worker; + grpc_fd *fd; +} grpc_fd_watcher; + +struct grpc_fd { + int fd; + /* refst format: + bit0: 1=active/0=orphaned + bit1-n: refcount + meaning that mostly we ref by two to avoid altering the orphaned bit, + and just unref by 1 when we're ready to flag the object as orphaned */ + gpr_atm refst; + + gpr_mu mu; + int shutdown; + int closed; + int released; + + /* The watcher list. + + The following watcher related fields are protected by watcher_mu. + + An fd_watcher is an ephemeral object created when an fd wants to + begin polling, and destroyed after the poll. + + It denotes the fd's interest in whether to read poll or write poll + or both or neither on this fd. + + If a watcher is asked to poll for reads or writes, the read_watcher + or write_watcher fields are set respectively. A watcher may be asked + to poll for both, in which case both fields will be set. + + read_watcher and write_watcher may be NULL if no watcher has been + asked to poll for reads or writes. + + If an fd_watcher is not asked to poll for reads or writes, it's added + to a linked list of inactive watchers, rooted at inactive_watcher_root. + If at a later time there becomes need of a poller to poll, one of + the inactive pollers may be kicked out of their poll loops to take + that responsibility. */ + grpc_fd_watcher inactive_watcher_root; + grpc_fd_watcher *read_watcher; + grpc_fd_watcher *write_watcher; + + grpc_closure *read_closure; + grpc_closure *write_closure; + + struct grpc_fd *freelist_next; + + grpc_closure *on_done_closure; + + grpc_iomgr_object iomgr_object; +}; + +/* Begin polling on an fd. + Registers that the given pollset is interested in this fd - so that if read + or writability interest changes, the pollset can be kicked to pick up that + new interest. + Return value is: + (fd_needs_read? read_mask : 0) | (fd_needs_write? write_mask : 0) + i.e. a combination of read_mask and write_mask determined by the fd's current + interest in said events. + Polling strategies that do not need to alter their behavior depending on the + fd's current interest (such as epoll) do not need to call this function. + MUST NOT be called with a pollset lock taken */ +static uint32_t fd_begin_poll(grpc_fd *fd, grpc_pollset *pollset, + grpc_pollset_worker *worker, uint32_t read_mask, + uint32_t write_mask, grpc_fd_watcher *rec); +/* Complete polling previously started with fd_begin_poll + MUST NOT be called with a pollset lock taken + if got_read or got_write are 1, also does the become_{readable,writable} as + appropriate. */ +static void fd_end_poll(grpc_exec_ctx *exec_ctx, grpc_fd_watcher *rec, + int got_read, int got_write); + +/* Return 1 if this fd is orphaned, 0 otherwise */ +static bool fd_is_orphaned(grpc_fd *fd); + +/* Reference counting for fds */ +/*#define GRPC_FD_REF_COUNT_DEBUG*/ +#ifdef GRPC_FD_REF_COUNT_DEBUG +static void fd_ref(grpc_fd *fd, const char *reason, const char *file, int line); +static void fd_unref(grpc_fd *fd, const char *reason, const char *file, + int line); +#define GRPC_FD_REF(fd, reason) fd_ref(fd, reason, __FILE__, __LINE__) +#define GRPC_FD_UNREF(fd, reason) fd_unref(fd, reason, __FILE__, __LINE__) +#else +static void fd_ref(grpc_fd *fd); +static void fd_unref(grpc_fd *fd); +#define GRPC_FD_REF(fd, reason) fd_ref(fd) +#define GRPC_FD_UNREF(fd, reason) fd_unref(fd) +#endif + +static void fd_global_init(void); +static void fd_global_shutdown(void); + +#define CLOSURE_NOT_READY ((grpc_closure *)0) +#define CLOSURE_READY ((grpc_closure *)1) + +/******************************************************************************* + * pollset declarations + */ + +typedef struct grpc_pollset_vtable grpc_pollset_vtable; + +typedef struct grpc_cached_wakeup_fd { + grpc_wakeup_fd fd; + struct grpc_cached_wakeup_fd *next; +} grpc_cached_wakeup_fd; + +struct grpc_pollset_worker { + grpc_cached_wakeup_fd *wakeup_fd; + int reevaluate_polling_on_wakeup; + int kicked_specifically; + struct grpc_pollset_worker *next; + struct grpc_pollset_worker *prev; +}; + +struct grpc_pollset { + /* pollsets under posix can mutate representation as fds are added and + removed. + For example, we may choose a poll() based implementation on linux for + few fds, and an epoll() based implementation for many fds */ + const grpc_pollset_vtable *vtable; + gpr_mu mu; + grpc_pollset_worker root_worker; + int in_flight_cbs; + int shutting_down; + int called_shutdown; + int kicked_without_pollers; + grpc_closure *shutdown_done; + grpc_closure_list idle_jobs; + union { + int fd; + void *ptr; + } data; + /* Local cache of eventfds for workers */ + grpc_cached_wakeup_fd *local_wakeup_cache; +}; + +struct grpc_pollset_vtable { + void (*add_fd)(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + struct grpc_fd *fd, int and_unlock_pollset); + void (*maybe_work_and_unlock)(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_pollset_worker *worker, + gpr_timespec deadline, gpr_timespec now); + void (*finish_shutdown)(grpc_pollset *pollset); + void (*destroy)(grpc_pollset *pollset); +}; + +/* Add an fd to a pollset */ +static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + struct grpc_fd *fd); + +static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd); + +/* Convert a timespec to milliseconds: + - very small or negative poll times are clamped to zero to do a + non-blocking poll (which becomes spin polling) + - other small values are rounded up to one millisecond + - longer than a millisecond polls are rounded up to the next nearest + millisecond to avoid spinning + - infinite timeouts are converted to -1 */ +static int poll_deadline_to_millis_timeout(gpr_timespec deadline, + gpr_timespec now); + +/* Allow kick to wakeup the currently polling worker */ +#define GRPC_POLLSET_CAN_KICK_SELF 1 +/* Force the wakee to repoll when awoken */ +#define GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP 2 +/* As per pollset_kick, with an extended set of flags (defined above) + -- mostly for fd_posix's use. */ +static void pollset_kick_ext(grpc_pollset *p, + grpc_pollset_worker *specific_worker, + uint32_t flags); + +/* turn a pollset into a multipoller: platform specific */ +typedef void (*platform_become_multipoller_type)(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, + struct grpc_fd **fds, + size_t fd_count); +static platform_become_multipoller_type platform_become_multipoller; + + +/* Return 1 if the pollset has active threads in pollset_work (pollset must + * be locked) */ +static int pollset_has_workers(grpc_pollset *pollset); + +static void remove_fd_from_all_epoll_sets(int fd); + +/******************************************************************************* + * pollset_set definitions + */ + +struct grpc_pollset_set { + gpr_mu mu; + + size_t pollset_count; + size_t pollset_capacity; + grpc_pollset **pollsets; + + size_t pollset_set_count; + size_t pollset_set_capacity; + struct grpc_pollset_set **pollset_sets; + + size_t fd_count; + size_t fd_capacity; + grpc_fd **fds; +}; + +/******************************************************************************* + * fd_posix.c + */ + +/* We need to keep a freelist not because of any concerns of malloc performance + * but instead so that implementations with multiple threads in (for example) + * epoll_wait deal with the race between pollset removal and incoming poll + * notifications. + * + * The problem is that the poller ultimately holds a reference to this + * object, so it is very difficult to know when is safe to free it, at least + * without some expensive synchronization. + * + * If we keep the object freelisted, in the worst case losing this race just + * becomes a spurious read notification on a reused fd. + */ +/* TODO(klempner): We could use some form of polling generation count to know + * when these are safe to free. */ +/* TODO(klempner): Consider disabling freelisting if we don't have multiple + * threads in poll on the same fd */ +/* TODO(klempner): Batch these allocations to reduce fragmentation */ +static grpc_fd *fd_freelist = NULL; +static gpr_mu fd_freelist_mu; + +static void freelist_fd(grpc_fd *fd) { + gpr_mu_lock(&fd_freelist_mu); + fd->freelist_next = fd_freelist; + fd_freelist = fd; + grpc_iomgr_unregister_object(&fd->iomgr_object); + gpr_mu_unlock(&fd_freelist_mu); +} + +static grpc_fd *alloc_fd(int fd) { + grpc_fd *r = NULL; + gpr_mu_lock(&fd_freelist_mu); + if (fd_freelist != NULL) { + r = fd_freelist; + fd_freelist = fd_freelist->freelist_next; + } + gpr_mu_unlock(&fd_freelist_mu); + if (r == NULL) { + r = gpr_malloc(sizeof(grpc_fd)); + gpr_mu_init(&r->mu); + } + + gpr_mu_lock(&r->mu); + gpr_atm_rel_store(&r->refst, 1); + r->shutdown = 0; + r->read_closure = CLOSURE_NOT_READY; + r->write_closure = CLOSURE_NOT_READY; + r->fd = fd; + r->inactive_watcher_root.next = r->inactive_watcher_root.prev = + &r->inactive_watcher_root; + r->freelist_next = NULL; + r->read_watcher = r->write_watcher = NULL; + r->on_done_closure = NULL; + r->closed = 0; + r->released = 0; + gpr_mu_unlock(&r->mu); + return r; +} + +static void destroy(grpc_fd *fd) { + gpr_mu_destroy(&fd->mu); + gpr_free(fd); +} + +#ifdef GRPC_FD_REF_COUNT_DEBUG +#define REF_BY(fd, n, reason) ref_by(fd, n, reason, __FILE__, __LINE__) +#define UNREF_BY(fd, n, reason) unref_by(fd, n, reason, __FILE__, __LINE__) +static void ref_by(grpc_fd *fd, int n, const char *reason, const char *file, + int line) { + gpr_log(GPR_DEBUG, "FD %d %p ref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, + gpr_atm_no_barrier_load(&fd->refst), + gpr_atm_no_barrier_load(&fd->refst) + n, reason, file, line); +#else +#define REF_BY(fd, n, reason) ref_by(fd, n) +#define UNREF_BY(fd, n, reason) unref_by(fd, n) +static void ref_by(grpc_fd *fd, int n) { +#endif + GPR_ASSERT(gpr_atm_no_barrier_fetch_add(&fd->refst, n) > 0); +} + +#ifdef GRPC_FD_REF_COUNT_DEBUG +static void unref_by(grpc_fd *fd, int n, const char *reason, const char *file, + int line) { + gpr_atm old; + gpr_log(GPR_DEBUG, "FD %d %p unref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, + gpr_atm_no_barrier_load(&fd->refst), + gpr_atm_no_barrier_load(&fd->refst) - n, reason, file, line); +#else +static void unref_by(grpc_fd *fd, int n) { + gpr_atm old; +#endif + old = gpr_atm_full_fetch_add(&fd->refst, -n); + if (old == n) { + freelist_fd(fd); + } else { + GPR_ASSERT(old > n); + } +} + +static void fd_global_init(void) { gpr_mu_init(&fd_freelist_mu); } + +static void fd_global_shutdown(void) { + gpr_mu_lock(&fd_freelist_mu); + gpr_mu_unlock(&fd_freelist_mu); + while (fd_freelist != NULL) { + grpc_fd *fd = fd_freelist; + fd_freelist = fd_freelist->freelist_next; + destroy(fd); + } + gpr_mu_destroy(&fd_freelist_mu); +} + +static grpc_fd *fd_create(int fd, const char *name) { + grpc_fd *r = alloc_fd(fd); + char *name2; + gpr_asprintf(&name2, "%s fd=%d", name, fd); + grpc_iomgr_register_object(&r->iomgr_object, name2); + gpr_free(name2); +#ifdef GRPC_FD_REF_COUNT_DEBUG + gpr_log(GPR_DEBUG, "FD %d %p create %s", fd, r, name); +#endif + return r; +} + +static bool fd_is_orphaned(grpc_fd *fd) { + return (gpr_atm_acq_load(&fd->refst) & 1) == 0; +} + +static void pollset_kick_locked(grpc_fd_watcher *watcher) { + gpr_mu_lock(&watcher->pollset->mu); + GPR_ASSERT(watcher->worker); + pollset_kick_ext(watcher->pollset, watcher->worker, + GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP); + gpr_mu_unlock(&watcher->pollset->mu); +} + +static void maybe_wake_one_watcher_locked(grpc_fd *fd) { + if (fd->inactive_watcher_root.next != &fd->inactive_watcher_root) { + pollset_kick_locked(fd->inactive_watcher_root.next); + } else if (fd->read_watcher) { + pollset_kick_locked(fd->read_watcher); + } else if (fd->write_watcher) { + pollset_kick_locked(fd->write_watcher); + } +} + +static void wake_all_watchers_locked(grpc_fd *fd) { + grpc_fd_watcher *watcher; + for (watcher = fd->inactive_watcher_root.next; + watcher != &fd->inactive_watcher_root; watcher = watcher->next) { + pollset_kick_locked(watcher); + } + if (fd->read_watcher) { + pollset_kick_locked(fd->read_watcher); + } + if (fd->write_watcher && fd->write_watcher != fd->read_watcher) { + pollset_kick_locked(fd->write_watcher); + } +} + +static int has_watchers(grpc_fd *fd) { + return fd->read_watcher != NULL || fd->write_watcher != NULL || + fd->inactive_watcher_root.next != &fd->inactive_watcher_root; +} + +static void close_fd_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + fd->closed = 1; + if (!fd->released) { + close(fd->fd); + } else { + remove_fd_from_all_epoll_sets(fd->fd); + } + grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); +} + +static int fd_wrapped_fd(grpc_fd *fd) { + if (fd->released || fd->closed) { + return -1; + } else { + return fd->fd; + } +} + +static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *on_done, int *release_fd, + const char *reason) { + fd->on_done_closure = on_done; + fd->released = release_fd != NULL; + if (!fd->released) { + shutdown(fd->fd, SHUT_RDWR); + } else { + *release_fd = fd->fd; + } + gpr_mu_lock(&fd->mu); + REF_BY(fd, 1, reason); /* remove active status, but keep referenced */ + if (!has_watchers(fd)) { + close_fd_locked(exec_ctx, fd); + } else { + wake_all_watchers_locked(fd); + } + gpr_mu_unlock(&fd->mu); + UNREF_BY(fd, 2, reason); /* drop the reference */ +} + +/* increment refcount by two to avoid changing the orphan bit */ +#ifdef GRPC_FD_REF_COUNT_DEBUG +static void fd_ref(grpc_fd *fd, const char *reason, const char *file, + int line) { + ref_by(fd, 2, reason, file, line); +} + +static void fd_unref(grpc_fd *fd, const char *reason, const char *file, + int line) { + unref_by(fd, 2, reason, file, line); +} +#else +static void fd_ref(grpc_fd *fd) { ref_by(fd, 2); } + +static void fd_unref(grpc_fd *fd) { unref_by(fd, 2); } +#endif + +static void notify_on_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure **st, grpc_closure *closure) { + if (*st == CLOSURE_NOT_READY) { + /* not ready ==> switch to a waiting state by setting the closure */ + *st = closure; + } else if (*st == CLOSURE_READY) { + /* already ready ==> queue the closure to run immediately */ + *st = CLOSURE_NOT_READY; + grpc_exec_ctx_enqueue(exec_ctx, closure, !fd->shutdown, NULL); + maybe_wake_one_watcher_locked(fd); + } else { + /* upcallptr was set to a different closure. This is an error! */ + gpr_log(GPR_ERROR, + "User called a notify_on function with a previous callback still " + "pending"); + abort(); + } +} + +/* returns 1 if state becomes not ready */ +static int set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure **st) { + if (*st == CLOSURE_READY) { + /* duplicate ready ==> ignore */ + return 0; + } else if (*st == CLOSURE_NOT_READY) { + /* not ready, and not waiting ==> flag ready */ + *st = CLOSURE_READY; + return 0; + } else { + /* waiting ==> queue closure */ + grpc_exec_ctx_enqueue(exec_ctx, *st, !fd->shutdown, NULL); + *st = CLOSURE_NOT_READY; + return 1; + } +} + +static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + gpr_mu_lock(&fd->mu); + GPR_ASSERT(!fd->shutdown); + fd->shutdown = 1; + set_ready_locked(exec_ctx, fd, &fd->read_closure); + set_ready_locked(exec_ctx, fd, &fd->write_closure); + gpr_mu_unlock(&fd->mu); +} + +static void fd_notify_on_read(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *closure) { + gpr_mu_lock(&fd->mu); + notify_on_locked(exec_ctx, fd, &fd->read_closure, closure); + gpr_mu_unlock(&fd->mu); +} + +static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *closure) { + gpr_mu_lock(&fd->mu); + notify_on_locked(exec_ctx, fd, &fd->write_closure, closure); + gpr_mu_unlock(&fd->mu); +} + +static uint32_t fd_begin_poll(grpc_fd *fd, grpc_pollset *pollset, + grpc_pollset_worker *worker, uint32_t read_mask, + uint32_t write_mask, grpc_fd_watcher *watcher) { + uint32_t mask = 0; + grpc_closure *cur; + int requested; + /* keep track of pollers that have requested our events, in case they change + */ + GRPC_FD_REF(fd, "poll"); + + gpr_mu_lock(&fd->mu); + + /* if we are shutdown, then don't add to the watcher set */ + if (fd->shutdown) { + watcher->fd = NULL; + watcher->pollset = NULL; + watcher->worker = NULL; + gpr_mu_unlock(&fd->mu); + GRPC_FD_UNREF(fd, "poll"); + return 0; + } + + /* if there is nobody polling for read, but we need to, then start doing so */ + cur = fd->read_closure; + requested = cur != CLOSURE_READY; + if (read_mask && fd->read_watcher == NULL && requested) { + fd->read_watcher = watcher; + mask |= read_mask; + } + /* if there is nobody polling for write, but we need to, then start doing so + */ + cur = fd->write_closure; + requested = cur != CLOSURE_READY; + if (write_mask && fd->write_watcher == NULL && requested) { + fd->write_watcher = watcher; + mask |= write_mask; + } + /* if not polling, remember this watcher in case we need someone to later */ + if (mask == 0 && worker != NULL) { + watcher->next = &fd->inactive_watcher_root; + watcher->prev = watcher->next->prev; + watcher->next->prev = watcher->prev->next = watcher; + } + watcher->pollset = pollset; + watcher->worker = worker; + watcher->fd = fd; + gpr_mu_unlock(&fd->mu); + + return mask; +} + +static void fd_end_poll(grpc_exec_ctx *exec_ctx, grpc_fd_watcher *watcher, + int got_read, int got_write) { + int was_polling = 0; + int kick = 0; + grpc_fd *fd = watcher->fd; + + if (fd == NULL) { + return; + } + + gpr_mu_lock(&fd->mu); + + if (watcher == fd->read_watcher) { + /* remove read watcher, kick if we still need a read */ + was_polling = 1; + if (!got_read) { + kick = 1; + } + fd->read_watcher = NULL; + } + if (watcher == fd->write_watcher) { + /* remove write watcher, kick if we still need a write */ + was_polling = 1; + if (!got_write) { + kick = 1; + } + fd->write_watcher = NULL; + } + if (!was_polling && watcher->worker != NULL) { + /* remove from inactive list */ + watcher->next->prev = watcher->prev; + watcher->prev->next = watcher->next; + } + if (got_read) { + if (set_ready_locked(exec_ctx, fd, &fd->read_closure)) { + kick = 1; + } + } + if (got_write) { + if (set_ready_locked(exec_ctx, fd, &fd->write_closure)) { + kick = 1; + } + } + if (kick) { + maybe_wake_one_watcher_locked(fd); + } + if (fd_is_orphaned(fd) && !has_watchers(fd) && !fd->closed) { + close_fd_locked(exec_ctx, fd); + } + gpr_mu_unlock(&fd->mu); + + GRPC_FD_UNREF(fd, "poll"); +} + +/******************************************************************************* + * pollset_posix.c + */ + +GPR_TLS_DECL(g_current_thread_poller); +GPR_TLS_DECL(g_current_thread_worker); + +/** The alarm system needs to be able to wakeup 'some poller' sometimes + * (specifically when a new alarm needs to be triggered earlier than the next + * alarm 'epoch'). + * This wakeup_fd gives us something to alert on when such a case occurs. */ +grpc_wakeup_fd grpc_global_wakeup_fd; + +static void remove_worker(grpc_pollset *p, grpc_pollset_worker *worker) { + worker->prev->next = worker->next; + worker->next->prev = worker->prev; +} + +static int pollset_has_workers(grpc_pollset *p) { + return p->root_worker.next != &p->root_worker; +} + +static grpc_pollset_worker *pop_front_worker(grpc_pollset *p) { + if (pollset_has_workers(p)) { + grpc_pollset_worker *w = p->root_worker.next; + remove_worker(p, w); + return w; + } else { + return NULL; + } +} + +static void push_back_worker(grpc_pollset *p, grpc_pollset_worker *worker) { + worker->next = &p->root_worker; + worker->prev = worker->next->prev; + worker->prev->next = worker->next->prev = worker; +} + +static void push_front_worker(grpc_pollset *p, grpc_pollset_worker *worker) { + worker->prev = &p->root_worker; + worker->next = worker->prev->next; + worker->prev->next = worker->next->prev = worker; +} + +static void pollset_kick_ext(grpc_pollset *p, + grpc_pollset_worker *specific_worker, + uint32_t flags) { + GPR_TIMER_BEGIN("pollset_kick_ext", 0); + + /* pollset->mu already held */ + if (specific_worker != NULL) { + if (specific_worker == GRPC_POLLSET_KICK_BROADCAST) { + GPR_TIMER_BEGIN("pollset_kick_ext.broadcast", 0); + GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); + for (specific_worker = p->root_worker.next; + specific_worker != &p->root_worker; + specific_worker = specific_worker->next) { + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + } + p->kicked_without_pollers = 1; + GPR_TIMER_END("pollset_kick_ext.broadcast", 0); + } else if (gpr_tls_get(&g_current_thread_worker) != + (intptr_t)specific_worker) { + GPR_TIMER_MARK("different_thread_worker", 0); + if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { + specific_worker->reevaluate_polling_on_wakeup = 1; + } + specific_worker->kicked_specifically = 1; + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + } else if ((flags & GRPC_POLLSET_CAN_KICK_SELF) != 0) { + GPR_TIMER_MARK("kick_yoself", 0); + if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { + specific_worker->reevaluate_polling_on_wakeup = 1; + } + specific_worker->kicked_specifically = 1; + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + } + } else if (gpr_tls_get(&g_current_thread_poller) != (intptr_t)p) { + GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); + GPR_TIMER_MARK("kick_anonymous", 0); + specific_worker = pop_front_worker(p); + if (specific_worker != NULL) { + if (gpr_tls_get(&g_current_thread_worker) == (intptr_t)specific_worker) { + GPR_TIMER_MARK("kick_anonymous_not_self", 0); + push_back_worker(p, specific_worker); + specific_worker = pop_front_worker(p); + if ((flags & GRPC_POLLSET_CAN_KICK_SELF) == 0 && + gpr_tls_get(&g_current_thread_worker) == + (intptr_t)specific_worker) { + push_back_worker(p, specific_worker); + specific_worker = NULL; + } + } + if (specific_worker != NULL) { + GPR_TIMER_MARK("finally_kick", 0); + push_back_worker(p, specific_worker); + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + } + } else { + GPR_TIMER_MARK("kicked_no_pollers", 0); + p->kicked_without_pollers = 1; + } + } + + GPR_TIMER_END("pollset_kick_ext", 0); +} + +static void pollset_kick(grpc_pollset *p, + grpc_pollset_worker *specific_worker) { + pollset_kick_ext(p, specific_worker, 0); +} + +/* global state management */ + +static void pollset_global_init(void) { + gpr_tls_init(&g_current_thread_poller); + gpr_tls_init(&g_current_thread_worker); + grpc_wakeup_fd_init(&grpc_global_wakeup_fd); +} + +static void pollset_global_shutdown(void) { + grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); + gpr_tls_destroy(&g_current_thread_poller); + gpr_tls_destroy(&g_current_thread_worker); +} + +static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } + +/* main interface */ + +static void become_basic_pollset(grpc_pollset *pollset, grpc_fd *fd_or_null); + +static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { + gpr_mu_init(&pollset->mu); + *mu = &pollset->mu; + pollset->root_worker.next = pollset->root_worker.prev = &pollset->root_worker; + pollset->in_flight_cbs = 0; + pollset->shutting_down = 0; + pollset->called_shutdown = 0; + pollset->kicked_without_pollers = 0; + pollset->idle_jobs.head = pollset->idle_jobs.tail = NULL; + pollset->local_wakeup_cache = NULL; + pollset->kicked_without_pollers = 0; + become_basic_pollset(pollset, NULL); +} + +static void pollset_destroy(grpc_pollset *pollset) { + GPR_ASSERT(pollset->in_flight_cbs == 0); + GPR_ASSERT(!pollset_has_workers(pollset)); + GPR_ASSERT(pollset->idle_jobs.head == pollset->idle_jobs.tail); + pollset->vtable->destroy(pollset); + while (pollset->local_wakeup_cache) { + grpc_cached_wakeup_fd *next = pollset->local_wakeup_cache->next; + grpc_wakeup_fd_destroy(&pollset->local_wakeup_cache->fd); + gpr_free(pollset->local_wakeup_cache); + pollset->local_wakeup_cache = next; + } + gpr_mu_destroy(&pollset->mu); +} + +static void pollset_reset(grpc_pollset *pollset) { + GPR_ASSERT(pollset->shutting_down); + GPR_ASSERT(pollset->in_flight_cbs == 0); + GPR_ASSERT(!pollset_has_workers(pollset)); + GPR_ASSERT(pollset->idle_jobs.head == pollset->idle_jobs.tail); + pollset->vtable->destroy(pollset); + pollset->shutting_down = 0; + pollset->called_shutdown = 0; + pollset->kicked_without_pollers = 0; + become_basic_pollset(pollset, NULL); +} + +static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_fd *fd) { + gpr_mu_lock(&pollset->mu); + pollset->vtable->add_fd(exec_ctx, pollset, fd, 1); +/* the following (enabled only in debug) will reacquire and then release + our lock - meaning that if the unlocking flag passed to add_fd above is + not respected, the code will deadlock (in a way that we have a chance of + debugging) */ +#ifndef NDEBUG + gpr_mu_lock(&pollset->mu); + gpr_mu_unlock(&pollset->mu); +#endif +} + +static void finish_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { + GPR_ASSERT(grpc_closure_list_empty(pollset->idle_jobs)); + pollset->vtable->finish_shutdown(pollset); + grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); +} + +static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_pollset_worker **worker_hdl, gpr_timespec now, + gpr_timespec deadline) { + grpc_pollset_worker worker; + *worker_hdl = &worker; + + /* pollset->mu already held */ + int added_worker = 0; + int locked = 1; + int queued_work = 0; + int keep_polling = 0; + GPR_TIMER_BEGIN("pollset_work", 0); + /* this must happen before we (potentially) drop pollset->mu */ + worker.next = worker.prev = NULL; + worker.reevaluate_polling_on_wakeup = 0; + if (pollset->local_wakeup_cache != NULL) { + worker.wakeup_fd = pollset->local_wakeup_cache; + pollset->local_wakeup_cache = worker.wakeup_fd->next; + } else { + worker.wakeup_fd = gpr_malloc(sizeof(*worker.wakeup_fd)); + grpc_wakeup_fd_init(&worker.wakeup_fd->fd); + } + worker.kicked_specifically = 0; + /* If there's work waiting for the pollset to be idle, and the + pollset is idle, then do that work */ + if (!pollset_has_workers(pollset) && + !grpc_closure_list_empty(pollset->idle_jobs)) { + GPR_TIMER_MARK("pollset_work.idle_jobs", 0); + grpc_exec_ctx_enqueue_list(exec_ctx, &pollset->idle_jobs, NULL); + goto done; + } + /* If we're shutting down then we don't execute any extended work */ + if (pollset->shutting_down) { + GPR_TIMER_MARK("pollset_work.shutting_down", 0); + goto done; + } + /* Give do_promote priority so we don't starve it out */ + if (pollset->in_flight_cbs) { + GPR_TIMER_MARK("pollset_work.in_flight_cbs", 0); + gpr_mu_unlock(&pollset->mu); + locked = 0; + goto done; + } + /* Start polling, and keep doing so while we're being asked to + re-evaluate our pollers (this allows poll() based pollers to + ensure they don't miss wakeups) */ + keep_polling = 1; + while (keep_polling) { + keep_polling = 0; + if (!pollset->kicked_without_pollers) { + if (!added_worker) { + push_front_worker(pollset, &worker); + added_worker = 1; + gpr_tls_set(&g_current_thread_worker, (intptr_t)&worker); + } + gpr_tls_set(&g_current_thread_poller, (intptr_t)pollset); + GPR_TIMER_BEGIN("maybe_work_and_unlock", 0); + pollset->vtable->maybe_work_and_unlock(exec_ctx, pollset, &worker, + deadline, now); + GPR_TIMER_END("maybe_work_and_unlock", 0); + locked = 0; + gpr_tls_set(&g_current_thread_poller, 0); + } else { + GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); + pollset->kicked_without_pollers = 0; + } + /* Finished execution - start cleaning up. + Note that we may arrive here from outside the enclosing while() loop. + In that case we won't loop though as we haven't added worker to the + worker list, which means nobody could ask us to re-evaluate polling). */ + done: + if (!locked) { + queued_work |= grpc_exec_ctx_flush(exec_ctx); + gpr_mu_lock(&pollset->mu); + locked = 1; + } + /* If we're forced to re-evaluate polling (via pollset_kick with + GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) then we land here and force + a loop */ + if (worker.reevaluate_polling_on_wakeup) { + worker.reevaluate_polling_on_wakeup = 0; + pollset->kicked_without_pollers = 0; + if (queued_work || worker.kicked_specifically) { + /* If there's queued work on the list, then set the deadline to be + immediate so we get back out of the polling loop quickly */ + deadline = gpr_inf_past(GPR_CLOCK_MONOTONIC); + } + keep_polling = 1; + } + } + if (added_worker) { + remove_worker(pollset, &worker); + gpr_tls_set(&g_current_thread_worker, 0); + } + /* release wakeup fd to the local pool */ + worker.wakeup_fd->next = pollset->local_wakeup_cache; + pollset->local_wakeup_cache = worker.wakeup_fd; + /* check shutdown conditions */ + if (pollset->shutting_down) { + if (pollset_has_workers(pollset)) { + pollset_kick(pollset, NULL); + } else if (!pollset->called_shutdown && pollset->in_flight_cbs == 0) { + pollset->called_shutdown = 1; + gpr_mu_unlock(&pollset->mu); + finish_shutdown(exec_ctx, pollset); + grpc_exec_ctx_flush(exec_ctx); + /* Continuing to access pollset here is safe -- it is the caller's + * responsibility to not destroy when it has outstanding calls to + * pollset_work. + * TODO(dklempner): Can we refactor the shutdown logic to avoid this? */ + gpr_mu_lock(&pollset->mu); + } else if (!grpc_closure_list_empty(pollset->idle_jobs)) { + grpc_exec_ctx_enqueue_list(exec_ctx, &pollset->idle_jobs, NULL); + gpr_mu_unlock(&pollset->mu); + grpc_exec_ctx_flush(exec_ctx); + gpr_mu_lock(&pollset->mu); + } + } + *worker_hdl = NULL; + GPR_TIMER_END("pollset_work", 0); +} + +static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_closure *closure) { + GPR_ASSERT(!pollset->shutting_down); + pollset->shutting_down = 1; + pollset->shutdown_done = closure; + pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); + if (!pollset_has_workers(pollset)) { + grpc_exec_ctx_enqueue_list(exec_ctx, &pollset->idle_jobs, NULL); + } + if (!pollset->called_shutdown && pollset->in_flight_cbs == 0 && + !pollset_has_workers(pollset)) { + pollset->called_shutdown = 1; + finish_shutdown(exec_ctx, pollset); + } +} + +static int poll_deadline_to_millis_timeout(gpr_timespec deadline, + gpr_timespec now) { + gpr_timespec timeout; + static const int64_t max_spin_polling_us = 10; + if (gpr_time_cmp(deadline, gpr_inf_future(deadline.clock_type)) == 0) { + return -1; + } + if (gpr_time_cmp(deadline, gpr_time_add(now, gpr_time_from_micros( + max_spin_polling_us, + GPR_TIMESPAN))) <= 0) { + return 0; + } + timeout = gpr_time_sub(deadline, now); + return gpr_time_to_millis(gpr_time_add( + timeout, gpr_time_from_nanos(GPR_NS_PER_MS - 1, GPR_TIMESPAN))); +} + +/* + * basic_pollset - a vtable that provides polling for zero or one file + * descriptor via poll() + */ + +typedef struct grpc_unary_promote_args { + const grpc_pollset_vtable *original_vtable; + grpc_pollset *pollset; + grpc_fd *fd; + grpc_closure promotion_closure; +} grpc_unary_promote_args; + +static void basic_do_promote(grpc_exec_ctx *exec_ctx, void *args, + bool success) { + grpc_unary_promote_args *up_args = args; + const grpc_pollset_vtable *original_vtable = up_args->original_vtable; + grpc_pollset *pollset = up_args->pollset; + grpc_fd *fd = up_args->fd; + + /* + * This is quite tricky. There are a number of cases to keep in mind here: + * 1. fd may have been orphaned + * 2. The pollset may no longer be a unary poller (and we can't let case #1 + * leak to other pollset types!) + * 3. pollset's fd (which may have changed) may have been orphaned + * 4. The pollset may be shutting down. + */ + + gpr_mu_lock(&pollset->mu); + /* First we need to ensure that nobody is polling concurrently */ + GPR_ASSERT(!pollset_has_workers(pollset)); + + gpr_free(up_args); + /* At this point the pollset may no longer be a unary poller. In that case + * we should just call the right add function and be done. */ + /* TODO(klempner): If we're not careful this could cause infinite recursion. + * That's not a problem for now because empty_pollset has a trivial poller + * and we don't have any mechanism to unbecome multipoller. */ + pollset->in_flight_cbs--; + if (pollset->shutting_down) { + /* We don't care about this pollset anymore. */ + if (pollset->in_flight_cbs == 0 && !pollset->called_shutdown) { + pollset->called_shutdown = 1; + finish_shutdown(exec_ctx, pollset); + } + } else if (fd_is_orphaned(fd)) { + /* Don't try to add it to anything, we'll drop our ref on it below */ + } else if (pollset->vtable != original_vtable) { + pollset->vtable->add_fd(exec_ctx, pollset, fd, 0); + } else if (fd != pollset->data.ptr) { + grpc_fd *fds[2]; + fds[0] = pollset->data.ptr; + fds[1] = fd; + + if (fds[0] && !fd_is_orphaned(fds[0])) { + platform_become_multipoller(exec_ctx, pollset, fds, GPR_ARRAY_SIZE(fds)); + GRPC_FD_UNREF(fds[0], "basicpoll"); + } else { + /* old fd is orphaned and we haven't cleaned it up until now, so remain a + * unary poller */ + /* Note that it is possible that fds[1] is also orphaned at this point. + * That's okay, we'll correct it at the next add or poll. */ + if (fds[0]) GRPC_FD_UNREF(fds[0], "basicpoll"); + pollset->data.ptr = fd; + GRPC_FD_REF(fd, "basicpoll"); + } + } + + gpr_mu_unlock(&pollset->mu); + + /* Matching ref in basic_pollset_add_fd */ + GRPC_FD_UNREF(fd, "basicpoll_add"); +} + +static void basic_pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_fd *fd, int and_unlock_pollset) { + grpc_unary_promote_args *up_args; + GPR_ASSERT(fd); + if (fd == pollset->data.ptr) goto exit; + + if (!pollset_has_workers(pollset)) { + /* Fast path -- no in flight cbs */ + /* TODO(klempner): Comment this out and fix any test failures or establish + * they are due to timing issues */ + grpc_fd *fds[2]; + fds[0] = pollset->data.ptr; + fds[1] = fd; + + if (fds[0] == NULL) { + pollset->data.ptr = fd; + GRPC_FD_REF(fd, "basicpoll"); + } else if (!fd_is_orphaned(fds[0])) { + platform_become_multipoller(exec_ctx, pollset, fds, GPR_ARRAY_SIZE(fds)); + GRPC_FD_UNREF(fds[0], "basicpoll"); + } else { + /* old fd is orphaned and we haven't cleaned it up until now, so remain a + * unary poller */ + GRPC_FD_UNREF(fds[0], "basicpoll"); + pollset->data.ptr = fd; + GRPC_FD_REF(fd, "basicpoll"); + } + goto exit; + } + + /* Now we need to promote. This needs to happen when we're not polling. Since + * this may be called from poll, the wait needs to happen asynchronously. */ + GRPC_FD_REF(fd, "basicpoll_add"); + pollset->in_flight_cbs++; + up_args = gpr_malloc(sizeof(*up_args)); + up_args->fd = fd; + up_args->original_vtable = pollset->vtable; + up_args->pollset = pollset; + up_args->promotion_closure.cb = basic_do_promote; + up_args->promotion_closure.cb_arg = up_args; + + grpc_closure_list_add(&pollset->idle_jobs, &up_args->promotion_closure, 1); + pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); + +exit: + if (and_unlock_pollset) { + gpr_mu_unlock(&pollset->mu); + } +} + +static void basic_pollset_maybe_work_and_unlock(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, + grpc_pollset_worker *worker, + gpr_timespec deadline, + gpr_timespec now) { +#define POLLOUT_CHECK (POLLOUT | POLLHUP | POLLERR) +#define POLLIN_CHECK (POLLIN | POLLHUP | POLLERR) + + struct pollfd pfd[3]; + grpc_fd *fd; + grpc_fd_watcher fd_watcher; + int timeout; + int r; + nfds_t nfds; + + fd = pollset->data.ptr; + if (fd && fd_is_orphaned(fd)) { + GRPC_FD_UNREF(fd, "basicpoll"); + fd = pollset->data.ptr = NULL; + } + timeout = poll_deadline_to_millis_timeout(deadline, now); + pfd[0].fd = GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd); + pfd[0].events = POLLIN; + pfd[0].revents = 0; + pfd[1].fd = GRPC_WAKEUP_FD_GET_READ_FD(&worker->wakeup_fd->fd); + pfd[1].events = POLLIN; + pfd[1].revents = 0; + nfds = 2; + if (fd) { + pfd[2].fd = fd->fd; + pfd[2].revents = 0; + GRPC_FD_REF(fd, "basicpoll_begin"); + gpr_mu_unlock(&pollset->mu); + pfd[2].events = + (short)fd_begin_poll(fd, pollset, worker, POLLIN, POLLOUT, &fd_watcher); + if (pfd[2].events != 0) { + nfds++; + } + } else { + gpr_mu_unlock(&pollset->mu); + } + + /* TODO(vpai): Consider first doing a 0 timeout poll here to avoid + even going into the blocking annotation if possible */ + /* poll fd count (argument 2) is shortened by one if we have no events + to poll on - such that it only includes the kicker */ + GPR_TIMER_BEGIN("poll", 0); + GRPC_SCHEDULING_START_BLOCKING_REGION; + r = grpc_poll_function(pfd, nfds, timeout); + GRPC_SCHEDULING_END_BLOCKING_REGION; + GPR_TIMER_END("poll", 0); + + if (r < 0) { + if (errno != EINTR) { + gpr_log(GPR_ERROR, "poll() failed: %s", strerror(errno)); + } + if (fd) { + fd_end_poll(exec_ctx, &fd_watcher, 0, 0); + } + } else if (r == 0) { + if (fd) { + fd_end_poll(exec_ctx, &fd_watcher, 0, 0); + } + } else { + if (pfd[0].revents & POLLIN_CHECK) { + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + } + if (pfd[1].revents & POLLIN_CHECK) { + grpc_wakeup_fd_consume_wakeup(&worker->wakeup_fd->fd); + } + if (nfds > 2) { + fd_end_poll(exec_ctx, &fd_watcher, pfd[2].revents & POLLIN_CHECK, + pfd[2].revents & POLLOUT_CHECK); + } else if (fd) { + fd_end_poll(exec_ctx, &fd_watcher, 0, 0); + } + } + + if (fd) { + GRPC_FD_UNREF(fd, "basicpoll_begin"); + } +} + +static void basic_pollset_destroy(grpc_pollset *pollset) { + if (pollset->data.ptr != NULL) { + GRPC_FD_UNREF(pollset->data.ptr, "basicpoll"); + pollset->data.ptr = NULL; + } +} + +static const grpc_pollset_vtable basic_pollset = { + basic_pollset_add_fd, basic_pollset_maybe_work_and_unlock, + basic_pollset_destroy, basic_pollset_destroy}; + +static void become_basic_pollset(grpc_pollset *pollset, grpc_fd *fd_or_null) { + pollset->vtable = &basic_pollset; + pollset->data.ptr = fd_or_null; + if (fd_or_null != NULL) { + GRPC_FD_REF(fd_or_null, "basicpoll"); + } +} + + +/******************************************************************************* + * pollset_multipoller_with_epoll_posix.c + */ + +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "src/core/lib/iomgr/ev_posix.h" +#include "src/core/lib/profiling/timers.h" +#include "src/core/lib/support/block_annotate.h" + +static void set_ready(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st) { + /* only one set_ready can be active at once (but there may be a racing + notify_on) */ + gpr_mu_lock(&fd->mu); + set_ready_locked(exec_ctx, fd, st); + gpr_mu_unlock(&fd->mu); +} + +static void fd_become_readable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + set_ready(exec_ctx, fd, &fd->read_closure); +} + +static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + set_ready(exec_ctx, fd, &fd->write_closure); +} + +struct epoll_fd_list { + int *epoll_fds; + size_t count; + size_t capacity; +}; + +static struct epoll_fd_list epoll_fd_global_list; +static gpr_once init_epoll_fd_list_mu = GPR_ONCE_INIT; +static gpr_mu epoll_fd_list_mu; + +static void init_mu(void) { gpr_mu_init(&epoll_fd_list_mu); } + +static void add_epoll_fd_to_global_list(int epoll_fd) { + gpr_once_init(&init_epoll_fd_list_mu, init_mu); + + gpr_mu_lock(&epoll_fd_list_mu); + if (epoll_fd_global_list.count == epoll_fd_global_list.capacity) { + epoll_fd_global_list.capacity = + GPR_MAX((size_t)8, epoll_fd_global_list.capacity * 2); + epoll_fd_global_list.epoll_fds = + gpr_realloc(epoll_fd_global_list.epoll_fds, + epoll_fd_global_list.capacity * sizeof(int)); + } + epoll_fd_global_list.epoll_fds[epoll_fd_global_list.count++] = epoll_fd; + gpr_mu_unlock(&epoll_fd_list_mu); +} + +static void remove_epoll_fd_from_global_list(int epoll_fd) { + gpr_mu_lock(&epoll_fd_list_mu); + GPR_ASSERT(epoll_fd_global_list.count > 0); + for (size_t i = 0; i < epoll_fd_global_list.count; i++) { + if (epoll_fd == epoll_fd_global_list.epoll_fds[i]) { + epoll_fd_global_list.epoll_fds[i] = + epoll_fd_global_list.epoll_fds[--(epoll_fd_global_list.count)]; + break; + } + } + gpr_mu_unlock(&epoll_fd_list_mu); +} + +static void remove_fd_from_all_epoll_sets(int fd) { + int err; + gpr_once_init(&init_epoll_fd_list_mu, init_mu); + gpr_mu_lock(&epoll_fd_list_mu); + if (epoll_fd_global_list.count == 0) { + gpr_mu_unlock(&epoll_fd_list_mu); + return; + } + for (size_t i = 0; i < epoll_fd_global_list.count; i++) { + err = epoll_ctl(epoll_fd_global_list.epoll_fds[i], EPOLL_CTL_DEL, fd, NULL); + if (err < 0 && errno != ENOENT) { + gpr_log(GPR_ERROR, "epoll_ctl del for %d failed: %s", fd, + strerror(errno)); + } + } + gpr_mu_unlock(&epoll_fd_list_mu); +} + +typedef struct { + grpc_pollset *pollset; + grpc_fd *fd; + grpc_closure closure; +} delayed_add; + +typedef struct { int epoll_fd; } epoll_hdr; + +static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_fd *fd) { + epoll_hdr *h = pollset->data.ptr; + struct epoll_event ev; + int err; + grpc_fd_watcher watcher; + + /* We pretend to be polling whilst adding an fd to keep the fd from being + closed during the add. This may result in a spurious wakeup being assigned + to this pollset whilst adding, but that should be benign. */ + GPR_ASSERT(fd_begin_poll(fd, pollset, NULL, 0, 0, &watcher) == 0); + if (watcher.fd != NULL) { + ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); + ev.data.ptr = fd; + err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); + if (err < 0) { + /* FDs may be added to a pollset multiple times, so EEXIST is normal. */ + if (errno != EEXIST) { + gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", fd->fd, + strerror(errno)); + } + } + } + fd_end_poll(exec_ctx, &watcher, 0, 0); +} + +static void perform_delayed_add(grpc_exec_ctx *exec_ctx, void *arg, + bool iomgr_status) { + delayed_add *da = arg; + + if (!fd_is_orphaned(da->fd)) { + finally_add_fd(exec_ctx, da->pollset, da->fd); + } + + gpr_mu_lock(&da->pollset->mu); + da->pollset->in_flight_cbs--; + if (da->pollset->shutting_down) { + /* We don't care about this pollset anymore. */ + if (da->pollset->in_flight_cbs == 0 && !da->pollset->called_shutdown) { + da->pollset->called_shutdown = 1; + grpc_exec_ctx_enqueue(exec_ctx, da->pollset->shutdown_done, true, NULL); + } + } + gpr_mu_unlock(&da->pollset->mu); + + GRPC_FD_UNREF(da->fd, "delayed_add"); + + gpr_free(da); +} + +static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, + grpc_fd *fd, + int and_unlock_pollset) { + if (and_unlock_pollset) { + gpr_mu_unlock(&pollset->mu); + finally_add_fd(exec_ctx, pollset, fd); + } else { + delayed_add *da = gpr_malloc(sizeof(*da)); + da->pollset = pollset; + da->fd = fd; + GRPC_FD_REF(fd, "delayed_add"); + grpc_closure_init(&da->closure, perform_delayed_add, da); + pollset->in_flight_cbs++; + grpc_exec_ctx_enqueue(exec_ctx, &da->closure, true, NULL); + } +} + +/* TODO(klempner): We probably want to turn this down a bit */ +#define GRPC_EPOLL_MAX_EVENTS 1000 + +static void multipoll_with_epoll_pollset_maybe_work_and_unlock( + grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, + gpr_timespec deadline, gpr_timespec now) { + struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; + int ep_rv; + int poll_rv; + epoll_hdr *h = pollset->data.ptr; + int timeout_ms; + struct pollfd pfds[2]; + + /* If you want to ignore epoll's ability to sanely handle parallel pollers, + * for a more apples-to-apples performance comparison with poll, add a + * if (pollset->counter != 0) { return 0; } + * here. + */ + + gpr_mu_unlock(&pollset->mu); + + timeout_ms = poll_deadline_to_millis_timeout(deadline, now); + + pfds[0].fd = GRPC_WAKEUP_FD_GET_READ_FD(&worker->wakeup_fd->fd); + pfds[0].events = POLLIN; + pfds[0].revents = 0; + pfds[1].fd = h->epoll_fd; + pfds[1].events = POLLIN; + pfds[1].revents = 0; + + /* TODO(vpai): Consider first doing a 0 timeout poll here to avoid + even going into the blocking annotation if possible */ + GPR_TIMER_BEGIN("poll", 0); + GRPC_SCHEDULING_START_BLOCKING_REGION; + poll_rv = grpc_poll_function(pfds, 2, timeout_ms); + GRPC_SCHEDULING_END_BLOCKING_REGION; + GPR_TIMER_END("poll", 0); + + if (poll_rv < 0) { + if (errno != EINTR) { + gpr_log(GPR_ERROR, "poll() failed: %s", strerror(errno)); + } + } else if (poll_rv == 0) { + /* do nothing */ + } else { + if (pfds[0].revents) { + grpc_wakeup_fd_consume_wakeup(&worker->wakeup_fd->fd); + } + if (pfds[1].revents) { + do { + /* The following epoll_wait never blocks; it has a timeout of 0 */ + ep_rv = epoll_wait(h->epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); + if (ep_rv < 0) { + if (errno != EINTR) { + gpr_log(GPR_ERROR, "epoll_wait() failed: %s", strerror(errno)); + } + } else { + int i; + for (i = 0; i < ep_rv; ++i) { + grpc_fd *fd = ep_ev[i].data.ptr; + /* TODO(klempner): We might want to consider making err and pri + * separate events */ + int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + int write_ev = ep_ev[i].events & EPOLLOUT; + if (fd == NULL) { + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + } else { + if (read_ev || cancel) { + fd_become_readable(exec_ctx, fd); + } + if (write_ev || cancel) { + fd_become_writable(exec_ctx, fd); + } + } + } + } + } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); + } + } +} + +static void multipoll_with_epoll_pollset_finish_shutdown( + grpc_pollset *pollset) {} + +static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset) { + epoll_hdr *h = pollset->data.ptr; + close(h->epoll_fd); + remove_epoll_fd_from_global_list(h->epoll_fd); + gpr_free(h); +} + +static const grpc_pollset_vtable multipoll_with_epoll_pollset = { + multipoll_with_epoll_pollset_add_fd, + multipoll_with_epoll_pollset_maybe_work_and_unlock, + multipoll_with_epoll_pollset_finish_shutdown, + multipoll_with_epoll_pollset_destroy}; + +static void epoll_become_multipoller(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, grpc_fd **fds, + size_t nfds) { + size_t i; + epoll_hdr *h = gpr_malloc(sizeof(epoll_hdr)); + struct epoll_event ev; + int err; + + pollset->vtable = &multipoll_with_epoll_pollset; + pollset->data.ptr = h; + h->epoll_fd = epoll_create1(EPOLL_CLOEXEC); + if (h->epoll_fd < 0) { + /* TODO(klempner): Fall back to poll here, especially on ENOSYS */ + gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); + abort(); + } + add_epoll_fd_to_global_list(h->epoll_fd); + + ev.events = (uint32_t)(EPOLLIN | EPOLLET); + ev.data.ptr = NULL; + err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); + if (err < 0) { + gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), + strerror(errno)); + } + + for (i = 0; i < nfds; i++) { + multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fds[i], 0); + } +} + +/******************************************************************************* + * pollset_set_posix.c + */ + +static grpc_pollset_set *pollset_set_create(void) { + grpc_pollset_set *pollset_set = gpr_malloc(sizeof(*pollset_set)); + memset(pollset_set, 0, sizeof(*pollset_set)); + gpr_mu_init(&pollset_set->mu); + return pollset_set; +} + +static void pollset_set_destroy(grpc_pollset_set *pollset_set) { + size_t i; + gpr_mu_destroy(&pollset_set->mu); + for (i = 0; i < pollset_set->fd_count; i++) { + GRPC_FD_UNREF(pollset_set->fds[i], "pollset_set"); + } + gpr_free(pollset_set->pollsets); + gpr_free(pollset_set->pollset_sets); + gpr_free(pollset_set->fds); + gpr_free(pollset_set); +} + +static void pollset_set_add_pollset(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, + grpc_pollset *pollset) { + size_t i, j; + gpr_mu_lock(&pollset_set->mu); + if (pollset_set->pollset_count == pollset_set->pollset_capacity) { + pollset_set->pollset_capacity = + GPR_MAX(8, 2 * pollset_set->pollset_capacity); + pollset_set->pollsets = + gpr_realloc(pollset_set->pollsets, pollset_set->pollset_capacity * + sizeof(*pollset_set->pollsets)); + } + pollset_set->pollsets[pollset_set->pollset_count++] = pollset; + for (i = 0, j = 0; i < pollset_set->fd_count; i++) { + if (fd_is_orphaned(pollset_set->fds[i])) { + GRPC_FD_UNREF(pollset_set->fds[i], "pollset_set"); + } else { + pollset_add_fd(exec_ctx, pollset, pollset_set->fds[i]); + pollset_set->fds[j++] = pollset_set->fds[i]; + } + } + pollset_set->fd_count = j; + gpr_mu_unlock(&pollset_set->mu); +} + +static void pollset_set_del_pollset(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, + grpc_pollset *pollset) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + for (i = 0; i < pollset_set->pollset_count; i++) { + if (pollset_set->pollsets[i] == pollset) { + pollset_set->pollset_count--; + GPR_SWAP(grpc_pollset *, pollset_set->pollsets[i], + pollset_set->pollsets[pollset_set->pollset_count]); + break; + } + } + gpr_mu_unlock(&pollset_set->mu); +} + +static void pollset_set_add_pollset_set(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *bag, + grpc_pollset_set *item) { + size_t i, j; + gpr_mu_lock(&bag->mu); + if (bag->pollset_set_count == bag->pollset_set_capacity) { + bag->pollset_set_capacity = GPR_MAX(8, 2 * bag->pollset_set_capacity); + bag->pollset_sets = + gpr_realloc(bag->pollset_sets, + bag->pollset_set_capacity * sizeof(*bag->pollset_sets)); + } + bag->pollset_sets[bag->pollset_set_count++] = item; + for (i = 0, j = 0; i < bag->fd_count; i++) { + if (fd_is_orphaned(bag->fds[i])) { + GRPC_FD_UNREF(bag->fds[i], "pollset_set"); + } else { + pollset_set_add_fd(exec_ctx, item, bag->fds[i]); + bag->fds[j++] = bag->fds[i]; + } + } + bag->fd_count = j; + gpr_mu_unlock(&bag->mu); +} + +static void pollset_set_del_pollset_set(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *bag, + grpc_pollset_set *item) { + size_t i; + gpr_mu_lock(&bag->mu); + for (i = 0; i < bag->pollset_set_count; i++) { + if (bag->pollset_sets[i] == item) { + bag->pollset_set_count--; + GPR_SWAP(grpc_pollset_set *, bag->pollset_sets[i], + bag->pollset_sets[bag->pollset_set_count]); + break; + } + } + gpr_mu_unlock(&bag->mu); +} + +static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + if (pollset_set->fd_count == pollset_set->fd_capacity) { + pollset_set->fd_capacity = GPR_MAX(8, 2 * pollset_set->fd_capacity); + pollset_set->fds = gpr_realloc( + pollset_set->fds, pollset_set->fd_capacity * sizeof(*pollset_set->fds)); + } + GRPC_FD_REF(fd, "pollset_set"); + pollset_set->fds[pollset_set->fd_count++] = fd; + for (i = 0; i < pollset_set->pollset_count; i++) { + pollset_add_fd(exec_ctx, pollset_set->pollsets[i], fd); + } + for (i = 0; i < pollset_set->pollset_set_count; i++) { + pollset_set_add_fd(exec_ctx, pollset_set->pollset_sets[i], fd); + } + gpr_mu_unlock(&pollset_set->mu); +} + +static void pollset_set_del_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + for (i = 0; i < pollset_set->fd_count; i++) { + if (pollset_set->fds[i] == fd) { + pollset_set->fd_count--; + GPR_SWAP(grpc_fd *, pollset_set->fds[i], + pollset_set->fds[pollset_set->fd_count]); + GRPC_FD_UNREF(fd, "pollset_set"); + break; + } + } + for (i = 0; i < pollset_set->pollset_set_count; i++) { + pollset_set_del_fd(exec_ctx, pollset_set->pollset_sets[i], fd); + } + gpr_mu_unlock(&pollset_set->mu); +} + +/******************************************************************************* + * event engine binding + */ + +static void shutdown_engine(void) { + fd_global_shutdown(); + pollset_global_shutdown(); +} + +static const grpc_event_engine_vtable vtable = { + .pollset_size = sizeof(grpc_pollset), + + .fd_create = fd_create, + .fd_wrapped_fd = fd_wrapped_fd, + .fd_orphan = fd_orphan, + .fd_shutdown = fd_shutdown, + .fd_notify_on_read = fd_notify_on_read, + .fd_notify_on_write = fd_notify_on_write, + + .pollset_init = pollset_init, + .pollset_shutdown = pollset_shutdown, + .pollset_reset = pollset_reset, + .pollset_destroy = pollset_destroy, + .pollset_work = pollset_work, + .pollset_kick = pollset_kick, + .pollset_add_fd = pollset_add_fd, + + .pollset_set_create = pollset_set_create, + .pollset_set_destroy = pollset_set_destroy, + .pollset_set_add_pollset = pollset_set_add_pollset, + .pollset_set_del_pollset = pollset_set_del_pollset, + .pollset_set_add_pollset_set = pollset_set_add_pollset_set, + .pollset_set_del_pollset_set = pollset_set_del_pollset_set, + .pollset_set_add_fd = pollset_set_add_fd, + .pollset_set_del_fd = pollset_set_del_fd, + + .kick_poller = kick_poller, + + .shutdown_engine = shutdown_engine, +}; + +const grpc_event_engine_vtable *grpc_init_epoll_posix(void) { + platform_become_multipoller = epoll_become_multipoller; + fd_global_init(); + pollset_global_init(); + return &vtable; +} + +#endif diff --git a/src/core/lib/iomgr/ev_epoll_posix.h b/src/core/lib/iomgr/ev_epoll_posix.h new file mode 100644 index 00000000000..35319b4fc5d --- /dev/null +++ b/src/core/lib/iomgr/ev_epoll_posix.h @@ -0,0 +1,41 @@ +/* + * + * Copyright 2015, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#ifndef GRPC_CORE_LIB_IOMGR_EV_EPOLL_POSIX_H +#define GRPC_CORE_LIB_IOMGR_EV_EPOLL_POSIX_H + +#include "src/core/lib/iomgr/ev_posix.h" + +const grpc_event_engine_vtable *grpc_init_epoll_posix(void); + +#endif /* GRPC_CORE_LIB_IOMGR_EV_EPOLL_POSIX_H */ diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index 95520b01d3a..baa3b9856ad 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -44,6 +44,7 @@ #include #include +#include "src/core/lib/iomgr/ev_epoll_posix.h" #include "src/core/lib/iomgr/ev_poll_posix.h" #include "src/core/lib/support/env.h" @@ -61,7 +62,7 @@ typedef struct { } event_engine_factory; static const event_engine_factory g_factories[] = { - {"poll", grpc_init_poll_posix}, + {"poll", grpc_init_poll_posix}, {"epoll", grpc_init_epoll_posix}, }; static void add(const char *beg, const char *end, char ***ss, size_t *ns) { diff --git a/src/python/grpcio/grpc_core_dependencies.py b/src/python/grpcio/grpc_core_dependencies.py index be24dc7cf02..49b4ddc457c 100644 --- a/src/python/grpcio/grpc_core_dependencies.py +++ b/src/python/grpcio/grpc_core_dependencies.py @@ -94,6 +94,7 @@ CORE_SOURCE_FILES = [ 'src/core/lib/iomgr/endpoint.c', 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', + 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', 'src/core/lib/iomgr/exec_ctx.c', diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal index c3fd59b3c23..36b25cca161 100644 --- a/tools/doxygen/Doxyfile.core.internal +++ b/tools/doxygen/Doxyfile.core.internal @@ -807,6 +807,7 @@ src/core/lib/http/parser.h \ src/core/lib/iomgr/closure.h \ src/core/lib/iomgr/endpoint.h \ src/core/lib/iomgr/endpoint_pair.h \ +src/core/lib/iomgr/ev_epoll_posix.h \ src/core/lib/iomgr/ev_poll_posix.h \ src/core/lib/iomgr/ev_posix.h \ src/core/lib/iomgr/exec_ctx.h \ @@ -954,6 +955,7 @@ src/core/lib/iomgr/closure.c \ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ +src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index 24d23fe28bd..43940495864 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -5530,6 +5530,7 @@ "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", @@ -5629,6 +5630,8 @@ "src/core/lib/iomgr/endpoint_pair.h", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_posix.c", + "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.c", diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj b/vsprojects/vcxproj/grpc/grpc.vcxproj index e5379dc6a49..55304af5869 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj @@ -316,6 +316,7 @@ + @@ -483,6 +484,8 @@ + + diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters index 95a5a73d268..7d1c90fda7c 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters @@ -55,6 +55,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr @@ -674,6 +677,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj index 90ad80f2fc2..3d0cdfc668b 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj @@ -304,6 +304,7 @@ + @@ -449,6 +450,8 @@ + + diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters index 2b19c0fb341..d2ff4c630fd 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters @@ -58,6 +58,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr @@ -572,6 +575,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr From a7786001a22f511130a4292893cc8e2ab0ccdf75 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 11 May 2016 18:13:31 -0700 Subject: [PATCH 010/215] Remove basic_pollset and the promotion related code --- src/core/lib/iomgr/ev_epoll_posix.c | 355 ++++++---------------------- 1 file changed, 71 insertions(+), 284 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index ce8d3981b3f..cb4a00a75c4 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -185,11 +185,6 @@ struct grpc_pollset_worker { }; struct grpc_pollset { - /* pollsets under posix can mutate representation as fds are added and - removed. - For example, we may choose a poll() based implementation on linux for - few fds, and an epoll() based implementation for many fds */ - const grpc_pollset_vtable *vtable; gpr_mu mu; grpc_pollset_worker root_worker; int in_flight_cbs; @@ -248,8 +243,6 @@ typedef void (*platform_become_multipoller_type)(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, struct grpc_fd **fds, size_t fd_count); -static platform_become_multipoller_type platform_become_multipoller; - /* Return 1 if the pollset has active threads in pollset_work (pollset must * be locked) */ @@ -796,8 +789,6 @@ static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } /* main interface */ -static void become_basic_pollset(grpc_pollset *pollset, grpc_fd *fd_or_null); - static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { gpr_mu_init(&pollset->mu); *mu = &pollset->mu; @@ -809,14 +800,20 @@ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { pollset->idle_jobs.head = pollset->idle_jobs.tail = NULL; pollset->local_wakeup_cache = NULL; pollset->kicked_without_pollers = 0; - become_basic_pollset(pollset, NULL); + pollset->data.ptr = NULL; } +/* TODO(sreek): Maybe merge multipoll_*_destroy() with pollset_destroy() + * function */ +static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset); + static void pollset_destroy(grpc_pollset *pollset) { GPR_ASSERT(pollset->in_flight_cbs == 0); GPR_ASSERT(!pollset_has_workers(pollset)); GPR_ASSERT(pollset->idle_jobs.head == pollset->idle_jobs.tail); - pollset->vtable->destroy(pollset); + + multipoll_with_epoll_pollset_destroy(pollset); + while (pollset->local_wakeup_cache) { grpc_cached_wakeup_fd *next = pollset->local_wakeup_cache->next; grpc_wakeup_fd_destroy(&pollset->local_wakeup_cache->fd); @@ -831,17 +828,24 @@ static void pollset_reset(grpc_pollset *pollset) { GPR_ASSERT(pollset->in_flight_cbs == 0); GPR_ASSERT(!pollset_has_workers(pollset)); GPR_ASSERT(pollset->idle_jobs.head == pollset->idle_jobs.tail); - pollset->vtable->destroy(pollset); + + multipoll_with_epoll_pollset_destroy(pollset); + pollset->shutting_down = 0; pollset->called_shutdown = 0; pollset->kicked_without_pollers = 0; - become_basic_pollset(pollset, NULL); } +/* TODO (sreek): Remove multipoll_with_epoll_add_fd declaration*/ +static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, + grpc_fd *fd, + int and_unlock_pollset); + static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { gpr_mu_lock(&pollset->mu); - pollset->vtable->add_fd(exec_ctx, pollset, fd, 1); + multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fd, 1); /* the following (enabled only in debug) will reacquire and then release our lock - meaning that if the unlocking flag passed to add_fd above is not respected, the code will deadlock (in a way that we have a chance of @@ -852,12 +856,21 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, #endif } +/* TODO (sreek): Remove multipoll_with_epoll_finish_shutdown() declaration */ +static void multipoll_with_epoll_pollset_finish_shutdown(grpc_pollset *pollset); + static void finish_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { GPR_ASSERT(grpc_closure_list_empty(pollset->idle_jobs)); - pollset->vtable->finish_shutdown(pollset); + multipoll_with_epoll_pollset_finish_shutdown(pollset); grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); } +/* TODO(sreek): Remove multipoll_with_epoll_*_maybe_work_and_unlock declaration + */ +static void multipoll_with_epoll_pollset_maybe_work_and_unlock( + grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, + gpr_timespec deadline, gpr_timespec now); + static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker **worker_hdl, gpr_timespec now, gpr_timespec deadline) { @@ -915,8 +928,10 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } gpr_tls_set(&g_current_thread_poller, (intptr_t)pollset); GPR_TIMER_BEGIN("maybe_work_and_unlock", 0); - pollset->vtable->maybe_work_and_unlock(exec_ctx, pollset, &worker, - deadline, now); + + multipoll_with_epoll_pollset_maybe_work_and_unlock( + exec_ctx, pollset, &worker, deadline, now); + GPR_TIMER_END("maybe_work_and_unlock", 0); locked = 0; gpr_tls_set(&g_current_thread_poller, 0); @@ -1013,233 +1028,6 @@ static int poll_deadline_to_millis_timeout(gpr_timespec deadline, timeout, gpr_time_from_nanos(GPR_NS_PER_MS - 1, GPR_TIMESPAN))); } -/* - * basic_pollset - a vtable that provides polling for zero or one file - * descriptor via poll() - */ - -typedef struct grpc_unary_promote_args { - const grpc_pollset_vtable *original_vtable; - grpc_pollset *pollset; - grpc_fd *fd; - grpc_closure promotion_closure; -} grpc_unary_promote_args; - -static void basic_do_promote(grpc_exec_ctx *exec_ctx, void *args, - bool success) { - grpc_unary_promote_args *up_args = args; - const grpc_pollset_vtable *original_vtable = up_args->original_vtable; - grpc_pollset *pollset = up_args->pollset; - grpc_fd *fd = up_args->fd; - - /* - * This is quite tricky. There are a number of cases to keep in mind here: - * 1. fd may have been orphaned - * 2. The pollset may no longer be a unary poller (and we can't let case #1 - * leak to other pollset types!) - * 3. pollset's fd (which may have changed) may have been orphaned - * 4. The pollset may be shutting down. - */ - - gpr_mu_lock(&pollset->mu); - /* First we need to ensure that nobody is polling concurrently */ - GPR_ASSERT(!pollset_has_workers(pollset)); - - gpr_free(up_args); - /* At this point the pollset may no longer be a unary poller. In that case - * we should just call the right add function and be done. */ - /* TODO(klempner): If we're not careful this could cause infinite recursion. - * That's not a problem for now because empty_pollset has a trivial poller - * and we don't have any mechanism to unbecome multipoller. */ - pollset->in_flight_cbs--; - if (pollset->shutting_down) { - /* We don't care about this pollset anymore. */ - if (pollset->in_flight_cbs == 0 && !pollset->called_shutdown) { - pollset->called_shutdown = 1; - finish_shutdown(exec_ctx, pollset); - } - } else if (fd_is_orphaned(fd)) { - /* Don't try to add it to anything, we'll drop our ref on it below */ - } else if (pollset->vtable != original_vtable) { - pollset->vtable->add_fd(exec_ctx, pollset, fd, 0); - } else if (fd != pollset->data.ptr) { - grpc_fd *fds[2]; - fds[0] = pollset->data.ptr; - fds[1] = fd; - - if (fds[0] && !fd_is_orphaned(fds[0])) { - platform_become_multipoller(exec_ctx, pollset, fds, GPR_ARRAY_SIZE(fds)); - GRPC_FD_UNREF(fds[0], "basicpoll"); - } else { - /* old fd is orphaned and we haven't cleaned it up until now, so remain a - * unary poller */ - /* Note that it is possible that fds[1] is also orphaned at this point. - * That's okay, we'll correct it at the next add or poll. */ - if (fds[0]) GRPC_FD_UNREF(fds[0], "basicpoll"); - pollset->data.ptr = fd; - GRPC_FD_REF(fd, "basicpoll"); - } - } - - gpr_mu_unlock(&pollset->mu); - - /* Matching ref in basic_pollset_add_fd */ - GRPC_FD_UNREF(fd, "basicpoll_add"); -} - -static void basic_pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_fd *fd, int and_unlock_pollset) { - grpc_unary_promote_args *up_args; - GPR_ASSERT(fd); - if (fd == pollset->data.ptr) goto exit; - - if (!pollset_has_workers(pollset)) { - /* Fast path -- no in flight cbs */ - /* TODO(klempner): Comment this out and fix any test failures or establish - * they are due to timing issues */ - grpc_fd *fds[2]; - fds[0] = pollset->data.ptr; - fds[1] = fd; - - if (fds[0] == NULL) { - pollset->data.ptr = fd; - GRPC_FD_REF(fd, "basicpoll"); - } else if (!fd_is_orphaned(fds[0])) { - platform_become_multipoller(exec_ctx, pollset, fds, GPR_ARRAY_SIZE(fds)); - GRPC_FD_UNREF(fds[0], "basicpoll"); - } else { - /* old fd is orphaned and we haven't cleaned it up until now, so remain a - * unary poller */ - GRPC_FD_UNREF(fds[0], "basicpoll"); - pollset->data.ptr = fd; - GRPC_FD_REF(fd, "basicpoll"); - } - goto exit; - } - - /* Now we need to promote. This needs to happen when we're not polling. Since - * this may be called from poll, the wait needs to happen asynchronously. */ - GRPC_FD_REF(fd, "basicpoll_add"); - pollset->in_flight_cbs++; - up_args = gpr_malloc(sizeof(*up_args)); - up_args->fd = fd; - up_args->original_vtable = pollset->vtable; - up_args->pollset = pollset; - up_args->promotion_closure.cb = basic_do_promote; - up_args->promotion_closure.cb_arg = up_args; - - grpc_closure_list_add(&pollset->idle_jobs, &up_args->promotion_closure, 1); - pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); - -exit: - if (and_unlock_pollset) { - gpr_mu_unlock(&pollset->mu); - } -} - -static void basic_pollset_maybe_work_and_unlock(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, - grpc_pollset_worker *worker, - gpr_timespec deadline, - gpr_timespec now) { -#define POLLOUT_CHECK (POLLOUT | POLLHUP | POLLERR) -#define POLLIN_CHECK (POLLIN | POLLHUP | POLLERR) - - struct pollfd pfd[3]; - grpc_fd *fd; - grpc_fd_watcher fd_watcher; - int timeout; - int r; - nfds_t nfds; - - fd = pollset->data.ptr; - if (fd && fd_is_orphaned(fd)) { - GRPC_FD_UNREF(fd, "basicpoll"); - fd = pollset->data.ptr = NULL; - } - timeout = poll_deadline_to_millis_timeout(deadline, now); - pfd[0].fd = GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd); - pfd[0].events = POLLIN; - pfd[0].revents = 0; - pfd[1].fd = GRPC_WAKEUP_FD_GET_READ_FD(&worker->wakeup_fd->fd); - pfd[1].events = POLLIN; - pfd[1].revents = 0; - nfds = 2; - if (fd) { - pfd[2].fd = fd->fd; - pfd[2].revents = 0; - GRPC_FD_REF(fd, "basicpoll_begin"); - gpr_mu_unlock(&pollset->mu); - pfd[2].events = - (short)fd_begin_poll(fd, pollset, worker, POLLIN, POLLOUT, &fd_watcher); - if (pfd[2].events != 0) { - nfds++; - } - } else { - gpr_mu_unlock(&pollset->mu); - } - - /* TODO(vpai): Consider first doing a 0 timeout poll here to avoid - even going into the blocking annotation if possible */ - /* poll fd count (argument 2) is shortened by one if we have no events - to poll on - such that it only includes the kicker */ - GPR_TIMER_BEGIN("poll", 0); - GRPC_SCHEDULING_START_BLOCKING_REGION; - r = grpc_poll_function(pfd, nfds, timeout); - GRPC_SCHEDULING_END_BLOCKING_REGION; - GPR_TIMER_END("poll", 0); - - if (r < 0) { - if (errno != EINTR) { - gpr_log(GPR_ERROR, "poll() failed: %s", strerror(errno)); - } - if (fd) { - fd_end_poll(exec_ctx, &fd_watcher, 0, 0); - } - } else if (r == 0) { - if (fd) { - fd_end_poll(exec_ctx, &fd_watcher, 0, 0); - } - } else { - if (pfd[0].revents & POLLIN_CHECK) { - grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); - } - if (pfd[1].revents & POLLIN_CHECK) { - grpc_wakeup_fd_consume_wakeup(&worker->wakeup_fd->fd); - } - if (nfds > 2) { - fd_end_poll(exec_ctx, &fd_watcher, pfd[2].revents & POLLIN_CHECK, - pfd[2].revents & POLLOUT_CHECK); - } else if (fd) { - fd_end_poll(exec_ctx, &fd_watcher, 0, 0); - } - } - - if (fd) { - GRPC_FD_UNREF(fd, "basicpoll_begin"); - } -} - -static void basic_pollset_destroy(grpc_pollset *pollset) { - if (pollset->data.ptr != NULL) { - GRPC_FD_UNREF(pollset->data.ptr, "basicpoll"); - pollset->data.ptr = NULL; - } -} - -static const grpc_pollset_vtable basic_pollset = { - basic_pollset_add_fd, basic_pollset_maybe_work_and_unlock, - basic_pollset_destroy, basic_pollset_destroy}; - -static void become_basic_pollset(grpc_pollset *pollset, grpc_fd *fd_or_null) { - pollset->vtable = &basic_pollset; - pollset->data.ptr = fd_or_null; - if (fd_or_null != NULL) { - GRPC_FD_REF(fd_or_null, "basicpoll"); - } -} - - /******************************************************************************* * pollset_multipoller_with_epoll_posix.c */ @@ -1274,6 +1062,7 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { set_ready(exec_ctx, fd, &fd->write_closure); } +/* TODO (sreek): Maybe this global list is not required. Double check*/ struct epoll_fd_list { int *epoll_fds; size_t count; @@ -1390,10 +1179,48 @@ static void perform_delayed_add(grpc_exec_ctx *exec_ctx, void *arg, gpr_free(da); } +/* Creates an epoll fd and initializes the pollset */ +static void multipoll_with_epoll_pollset_create_efd(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset) { + epoll_hdr *h = gpr_malloc(sizeof(epoll_hdr)); + struct epoll_event ev; + int err; + + /* Ensuring that the pollset is infact empty (with no epoll fd either) */ + GPR_ASSERT(pollset->data.ptr == NULL); + + pollset->data.ptr = h; + h->epoll_fd = epoll_create1(EPOLL_CLOEXEC); + if (h->epoll_fd < 0) { + gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); + abort(); + } + add_epoll_fd_to_global_list(h->epoll_fd); + + ev.events = (uint32_t)(EPOLLIN | EPOLLET); + ev.data.ptr = NULL; + + /* TODO (sreek): Double-check the use of grpc_global_wakeup_fd here (right now + * I do not know why this is used. I just copied this code from + * epoll_become_mutipoller() function in ev_poll_and_epoll_posix.c file */ + err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); + if (err < 0) { + gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), + strerror(errno)); + } +} + static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd, int and_unlock_pollset) { + /* If there is no epoll fd on the pollset, create one */ + if (pollset->data.ptr == NULL) { + multipoll_with_epoll_pollset_create_efd(exec_ctx, pollset); + } + if (and_unlock_pollset) { gpr_mu_unlock(&pollset->mu); finally_add_fd(exec_ctx, pollset, fd); @@ -1500,45 +1327,6 @@ static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset) { gpr_free(h); } -static const grpc_pollset_vtable multipoll_with_epoll_pollset = { - multipoll_with_epoll_pollset_add_fd, - multipoll_with_epoll_pollset_maybe_work_and_unlock, - multipoll_with_epoll_pollset_finish_shutdown, - multipoll_with_epoll_pollset_destroy}; - -static void epoll_become_multipoller(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, grpc_fd **fds, - size_t nfds) { - size_t i; - epoll_hdr *h = gpr_malloc(sizeof(epoll_hdr)); - struct epoll_event ev; - int err; - - pollset->vtable = &multipoll_with_epoll_pollset; - pollset->data.ptr = h; - h->epoll_fd = epoll_create1(EPOLL_CLOEXEC); - if (h->epoll_fd < 0) { - /* TODO(klempner): Fall back to poll here, especially on ENOSYS */ - gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); - abort(); - } - add_epoll_fd_to_global_list(h->epoll_fd); - - ev.events = (uint32_t)(EPOLLIN | EPOLLET); - ev.data.ptr = NULL; - err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); - if (err < 0) { - gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), - strerror(errno)); - } - - for (i = 0; i < nfds; i++) { - multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fds[i], 0); - } -} - /******************************************************************************* * pollset_set_posix.c */ @@ -1724,7 +1512,6 @@ static const grpc_event_engine_vtable vtable = { }; const grpc_event_engine_vtable *grpc_init_epoll_posix(void) { - platform_become_multipoller = epoll_become_multipoller; fd_global_init(); pollset_global_init(); return &vtable; From ab7f10ed61c7e0d104ea839206d2cd0340f5fed8 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 12 May 2016 20:21:37 -0700 Subject: [PATCH 011/215] Remove delayed_add --- src/core/lib/iomgr/ev_epoll_posix.c | 61 ++++++----------------------- 1 file changed, 13 insertions(+), 48 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index cb4a00a75c4..03c544b30c0 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -187,7 +187,7 @@ struct grpc_pollset_worker { struct grpc_pollset { gpr_mu mu; grpc_pollset_worker root_worker; - int in_flight_cbs; + int in_flight_cbs; /* TODO (sreek): Most likely this isn't needed anymore */ int shutting_down; int called_shutdown; int kicked_without_pollers; @@ -839,13 +839,12 @@ static void pollset_reset(grpc_pollset *pollset) { /* TODO (sreek): Remove multipoll_with_epoll_add_fd declaration*/ static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_fd *fd, - int and_unlock_pollset); + grpc_fd *fd); static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { gpr_mu_lock(&pollset->mu); - multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fd, 1); + multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fd); /* the following (enabled only in debug) will reacquire and then release our lock - meaning that if the unlocking flag passed to add_fd above is not respected, the code will deadlock (in a way that we have a chance of @@ -1121,12 +1120,6 @@ static void remove_fd_from_all_epoll_sets(int fd) { gpr_mu_unlock(&epoll_fd_list_mu); } -typedef struct { - grpc_pollset *pollset; - grpc_fd *fd; - grpc_closure closure; -} delayed_add; - typedef struct { int epoll_fd; } epoll_hdr; static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, @@ -1139,6 +1132,13 @@ static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, /* We pretend to be polling whilst adding an fd to keep the fd from being closed during the add. This may result in a spurious wakeup being assigned to this pollset whilst adding, but that should be benign. */ + /* TODO (sreek). This fd_begin_poll() really seem to accomplish adding + * GRPC_FD_REF() (i.e adding a refcount to the fd) and checking that the + * fd is not shutting down (in which case watcher.fd will be NULL and no + * refcount is added). The ref count is added only durng hte duration of + * adding it to the epoll set (after which fd_end_poll would be called and + * the fd's ref count is decremented by 1. So do we still need fd_begin_poll + * ??? */ GPR_ASSERT(fd_begin_poll(fd, pollset, NULL, 0, 0, &watcher) == 0); if (watcher.fd != NULL) { ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); @@ -1155,30 +1155,6 @@ static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, fd_end_poll(exec_ctx, &watcher, 0, 0); } -static void perform_delayed_add(grpc_exec_ctx *exec_ctx, void *arg, - bool iomgr_status) { - delayed_add *da = arg; - - if (!fd_is_orphaned(da->fd)) { - finally_add_fd(exec_ctx, da->pollset, da->fd); - } - - gpr_mu_lock(&da->pollset->mu); - da->pollset->in_flight_cbs--; - if (da->pollset->shutting_down) { - /* We don't care about this pollset anymore. */ - if (da->pollset->in_flight_cbs == 0 && !da->pollset->called_shutdown) { - da->pollset->called_shutdown = 1; - grpc_exec_ctx_enqueue(exec_ctx, da->pollset->shutdown_done, true, NULL); - } - } - gpr_mu_unlock(&da->pollset->mu); - - GRPC_FD_UNREF(da->fd, "delayed_add"); - - gpr_free(da); -} - /* Creates an epoll fd and initializes the pollset */ static void multipoll_with_epoll_pollset_create_efd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { @@ -1214,25 +1190,14 @@ static void multipoll_with_epoll_pollset_create_efd(grpc_exec_ctx *exec_ctx, static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_fd *fd, - int and_unlock_pollset) { + grpc_fd *fd) { /* If there is no epoll fd on the pollset, create one */ if (pollset->data.ptr == NULL) { multipoll_with_epoll_pollset_create_efd(exec_ctx, pollset); } - if (and_unlock_pollset) { - gpr_mu_unlock(&pollset->mu); - finally_add_fd(exec_ctx, pollset, fd); - } else { - delayed_add *da = gpr_malloc(sizeof(*da)); - da->pollset = pollset; - da->fd = fd; - GRPC_FD_REF(fd, "delayed_add"); - grpc_closure_init(&da->closure, perform_delayed_add, da); - pollset->in_flight_cbs++; - grpc_exec_ctx_enqueue(exec_ctx, &da->closure, true, NULL); - } + gpr_mu_unlock(&pollset->mu); + finally_add_fd(exec_ctx, pollset, fd); } /* TODO(klempner): We probably want to turn this down a bit */ From f6a2adf0cfa14e473e05b6cb2d6b8fdc7667945b Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 13 May 2016 16:00:20 -0700 Subject: [PATCH 012/215] Pollset_reset should not destroy the epoll_fd --- src/core/lib/iomgr/ev_epoll_posix.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index 03c544b30c0..19465280a1b 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -828,9 +828,6 @@ static void pollset_reset(grpc_pollset *pollset) { GPR_ASSERT(pollset->in_flight_cbs == 0); GPR_ASSERT(!pollset_has_workers(pollset)); GPR_ASSERT(pollset->idle_jobs.head == pollset->idle_jobs.tail); - - multipoll_with_epoll_pollset_destroy(pollset); - pollset->shutting_down = 0; pollset->called_shutdown = 0; pollset->kicked_without_pollers = 0; From 24f0f57ea16b77e710f7ff4ae8a048c888ab6b12 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 13 May 2016 19:22:44 -0700 Subject: [PATCH 013/215] Moving the creation of epoll_fd to pollset_init() instead of pollset_add_fd() [Verified stable. All tests pass] --- src/core/lib/iomgr/ev_epoll_posix.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index 19465280a1b..0ee0ee58a8b 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -787,6 +787,9 @@ static void pollset_global_shutdown(void) { static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } +/* TODO: sreek. Try to Remove this forward declaration*/ +static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset); + /* main interface */ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { @@ -800,7 +803,9 @@ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { pollset->idle_jobs.head = pollset->idle_jobs.tail = NULL; pollset->local_wakeup_cache = NULL; pollset->kicked_without_pollers = 0; + pollset->data.ptr = NULL; + multipoll_with_epoll_pollset_create_efd(pollset); } /* TODO(sreek): Maybe merge multipoll_*_destroy() with pollset_destroy() @@ -1153,13 +1158,15 @@ static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } /* Creates an epoll fd and initializes the pollset */ -static void multipoll_with_epoll_pollset_create_efd(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset) { +/* TODO: This has to be called ONLY from pollset_init function. and hence it + * does not acquire any lock */ +static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { epoll_hdr *h = gpr_malloc(sizeof(epoll_hdr)); struct epoll_event ev; int err; - /* Ensuring that the pollset is infact empty (with no epoll fd either) */ + /* TODO (sreek). remove this assert. Currently added this just to ensure that + * we do not overwrite h->epoll_fd without freeing the older one*/ GPR_ASSERT(pollset->data.ptr == NULL); pollset->data.ptr = h; @@ -1188,10 +1195,10 @@ static void multipoll_with_epoll_pollset_create_efd(grpc_exec_ctx *exec_ctx, static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { - /* If there is no epoll fd on the pollset, create one */ - if (pollset->data.ptr == NULL) { - multipoll_with_epoll_pollset_create_efd(exec_ctx, pollset); - } + GPR_ASSERT(pollset->data.ptr != NULL); + + /* TODO(sreek). Remove this unlock code (and also the code that acquires the + * lock before calling multipoll_with_epoll_add_fd() function */ gpr_mu_unlock(&pollset->mu); finally_add_fd(exec_ctx, pollset, fd); From 9ff57f67a0920e8e39baedfec5671fb6e662d257 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Sat, 14 May 2016 15:56:57 -0700 Subject: [PATCH 014/215] Remove idle_jobs and in_flight_cbs from pollset --- src/core/lib/iomgr/ev_epoll_posix.c | 50 ++--------------------------- 1 file changed, 3 insertions(+), 47 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index 0ee0ee58a8b..5776b6c1cf1 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -169,8 +169,6 @@ static void fd_global_shutdown(void); * pollset declarations */ -typedef struct grpc_pollset_vtable grpc_pollset_vtable; - typedef struct grpc_cached_wakeup_fd { grpc_wakeup_fd fd; struct grpc_cached_wakeup_fd *next; @@ -187,12 +185,10 @@ struct grpc_pollset_worker { struct grpc_pollset { gpr_mu mu; grpc_pollset_worker root_worker; - int in_flight_cbs; /* TODO (sreek): Most likely this isn't needed anymore */ int shutting_down; int called_shutdown; int kicked_without_pollers; grpc_closure *shutdown_done; - grpc_closure_list idle_jobs; union { int fd; void *ptr; @@ -201,16 +197,6 @@ struct grpc_pollset { grpc_cached_wakeup_fd *local_wakeup_cache; }; -struct grpc_pollset_vtable { - void (*add_fd)(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - struct grpc_fd *fd, int and_unlock_pollset); - void (*maybe_work_and_unlock)(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_pollset_worker *worker, - gpr_timespec deadline, gpr_timespec now); - void (*finish_shutdown)(grpc_pollset *pollset); - void (*destroy)(grpc_pollset *pollset); -}; - /* Add an fd to a pollset */ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, struct grpc_fd *fd); @@ -796,11 +782,9 @@ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { gpr_mu_init(&pollset->mu); *mu = &pollset->mu; pollset->root_worker.next = pollset->root_worker.prev = &pollset->root_worker; - pollset->in_flight_cbs = 0; pollset->shutting_down = 0; pollset->called_shutdown = 0; pollset->kicked_without_pollers = 0; - pollset->idle_jobs.head = pollset->idle_jobs.tail = NULL; pollset->local_wakeup_cache = NULL; pollset->kicked_without_pollers = 0; @@ -813,9 +797,7 @@ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset); static void pollset_destroy(grpc_pollset *pollset) { - GPR_ASSERT(pollset->in_flight_cbs == 0); GPR_ASSERT(!pollset_has_workers(pollset)); - GPR_ASSERT(pollset->idle_jobs.head == pollset->idle_jobs.tail); multipoll_with_epoll_pollset_destroy(pollset); @@ -830,9 +812,7 @@ static void pollset_destroy(grpc_pollset *pollset) { static void pollset_reset(grpc_pollset *pollset) { GPR_ASSERT(pollset->shutting_down); - GPR_ASSERT(pollset->in_flight_cbs == 0); GPR_ASSERT(!pollset_has_workers(pollset)); - GPR_ASSERT(pollset->idle_jobs.head == pollset->idle_jobs.tail); pollset->shutting_down = 0; pollset->called_shutdown = 0; pollset->kicked_without_pollers = 0; @@ -861,7 +841,6 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, static void multipoll_with_epoll_pollset_finish_shutdown(grpc_pollset *pollset); static void finish_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { - GPR_ASSERT(grpc_closure_list_empty(pollset->idle_jobs)); multipoll_with_epoll_pollset_finish_shutdown(pollset); grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); } @@ -895,26 +874,11 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_wakeup_fd_init(&worker.wakeup_fd->fd); } worker.kicked_specifically = 0; - /* If there's work waiting for the pollset to be idle, and the - pollset is idle, then do that work */ - if (!pollset_has_workers(pollset) && - !grpc_closure_list_empty(pollset->idle_jobs)) { - GPR_TIMER_MARK("pollset_work.idle_jobs", 0); - grpc_exec_ctx_enqueue_list(exec_ctx, &pollset->idle_jobs, NULL); - goto done; - } /* If we're shutting down then we don't execute any extended work */ if (pollset->shutting_down) { GPR_TIMER_MARK("pollset_work.shutting_down", 0); goto done; } - /* Give do_promote priority so we don't starve it out */ - if (pollset->in_flight_cbs) { - GPR_TIMER_MARK("pollset_work.in_flight_cbs", 0); - gpr_mu_unlock(&pollset->mu); - locked = 0; - goto done; - } /* Start polling, and keep doing so while we're being asked to re-evaluate our pollers (this allows poll() based pollers to ensure they don't miss wakeups) */ @@ -975,7 +939,7 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, if (pollset->shutting_down) { if (pollset_has_workers(pollset)) { pollset_kick(pollset, NULL); - } else if (!pollset->called_shutdown && pollset->in_flight_cbs == 0) { + } else if (!pollset->called_shutdown) { pollset->called_shutdown = 1; gpr_mu_unlock(&pollset->mu); finish_shutdown(exec_ctx, pollset); @@ -985,11 +949,6 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, * pollset_work. * TODO(dklempner): Can we refactor the shutdown logic to avoid this? */ gpr_mu_lock(&pollset->mu); - } else if (!grpc_closure_list_empty(pollset->idle_jobs)) { - grpc_exec_ctx_enqueue_list(exec_ctx, &pollset->idle_jobs, NULL); - gpr_mu_unlock(&pollset->mu); - grpc_exec_ctx_flush(exec_ctx); - gpr_mu_lock(&pollset->mu); } } *worker_hdl = NULL; @@ -1002,11 +961,8 @@ static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, pollset->shutting_down = 1; pollset->shutdown_done = closure; pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); - if (!pollset_has_workers(pollset)) { - grpc_exec_ctx_enqueue_list(exec_ctx, &pollset->idle_jobs, NULL); - } - if (!pollset->called_shutdown && pollset->in_flight_cbs == 0 && - !pollset_has_workers(pollset)) { + + if (!pollset->called_shutdown && !pollset_has_workers(pollset)) { pollset->called_shutdown = 1; finish_shutdown(exec_ctx, pollset); } From 97c2d6a269472a8a6e56d9e3d88f89bd27aff5ac Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Sat, 14 May 2016 16:33:16 -0700 Subject: [PATCH 015/215] Remove grpc_fd_watcher and related code from ev_epoll_posix.c --- src/core/lib/iomgr/ev_epoll_posix.c | 271 ++++------------------------ 1 file changed, 38 insertions(+), 233 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index 5776b6c1cf1..920be1951fa 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -59,17 +59,6 @@ * FD declarations */ -/* TODO(sreek) : Check if grpc_fd_watcher is needed (and if so, check if we can - * share this between ev_poll_posix.h and ev_epoll_posix versions */ - -typedef struct grpc_fd_watcher { - struct grpc_fd_watcher *next; - struct grpc_fd_watcher *prev; - grpc_pollset *pollset; - grpc_pollset_worker *worker; - grpc_fd *fd; -} grpc_fd_watcher; - struct grpc_fd { int fd; /* refst format: @@ -84,32 +73,6 @@ struct grpc_fd { int closed; int released; - /* The watcher list. - - The following watcher related fields are protected by watcher_mu. - - An fd_watcher is an ephemeral object created when an fd wants to - begin polling, and destroyed after the poll. - - It denotes the fd's interest in whether to read poll or write poll - or both or neither on this fd. - - If a watcher is asked to poll for reads or writes, the read_watcher - or write_watcher fields are set respectively. A watcher may be asked - to poll for both, in which case both fields will be set. - - read_watcher and write_watcher may be NULL if no watcher has been - asked to poll for reads or writes. - - If an fd_watcher is not asked to poll for reads or writes, it's added - to a linked list of inactive watchers, rooted at inactive_watcher_root. - If at a later time there becomes need of a poller to poll, one of - the inactive pollers may be kicked out of their poll loops to take - that responsibility. */ - grpc_fd_watcher inactive_watcher_root; - grpc_fd_watcher *read_watcher; - grpc_fd_watcher *write_watcher; - grpc_closure *read_closure; grpc_closure *write_closure; @@ -120,27 +83,6 @@ struct grpc_fd { grpc_iomgr_object iomgr_object; }; -/* Begin polling on an fd. - Registers that the given pollset is interested in this fd - so that if read - or writability interest changes, the pollset can be kicked to pick up that - new interest. - Return value is: - (fd_needs_read? read_mask : 0) | (fd_needs_write? write_mask : 0) - i.e. a combination of read_mask and write_mask determined by the fd's current - interest in said events. - Polling strategies that do not need to alter their behavior depending on the - fd's current interest (such as epoll) do not need to call this function. - MUST NOT be called with a pollset lock taken */ -static uint32_t fd_begin_poll(grpc_fd *fd, grpc_pollset *pollset, - grpc_pollset_worker *worker, uint32_t read_mask, - uint32_t write_mask, grpc_fd_watcher *rec); -/* Complete polling previously started with fd_begin_poll - MUST NOT be called with a pollset lock taken - if got_read or got_write are 1, also does the become_{readable,writable} as - appropriate. */ -static void fd_end_poll(grpc_exec_ctx *exec_ctx, grpc_fd_watcher *rec, - int got_read, int got_write); - /* Return 1 if this fd is orphaned, 0 otherwise */ static bool fd_is_orphaned(grpc_fd *fd); @@ -307,10 +249,7 @@ static grpc_fd *alloc_fd(int fd) { r->read_closure = CLOSURE_NOT_READY; r->write_closure = CLOSURE_NOT_READY; r->fd = fd; - r->inactive_watcher_root.next = r->inactive_watcher_root.prev = - &r->inactive_watcher_root; r->freelist_next = NULL; - r->read_watcher = r->write_watcher = NULL; r->on_done_closure = NULL; r->closed = 0; r->released = 0; @@ -387,43 +326,6 @@ static bool fd_is_orphaned(grpc_fd *fd) { return (gpr_atm_acq_load(&fd->refst) & 1) == 0; } -static void pollset_kick_locked(grpc_fd_watcher *watcher) { - gpr_mu_lock(&watcher->pollset->mu); - GPR_ASSERT(watcher->worker); - pollset_kick_ext(watcher->pollset, watcher->worker, - GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP); - gpr_mu_unlock(&watcher->pollset->mu); -} - -static void maybe_wake_one_watcher_locked(grpc_fd *fd) { - if (fd->inactive_watcher_root.next != &fd->inactive_watcher_root) { - pollset_kick_locked(fd->inactive_watcher_root.next); - } else if (fd->read_watcher) { - pollset_kick_locked(fd->read_watcher); - } else if (fd->write_watcher) { - pollset_kick_locked(fd->write_watcher); - } -} - -static void wake_all_watchers_locked(grpc_fd *fd) { - grpc_fd_watcher *watcher; - for (watcher = fd->inactive_watcher_root.next; - watcher != &fd->inactive_watcher_root; watcher = watcher->next) { - pollset_kick_locked(watcher); - } - if (fd->read_watcher) { - pollset_kick_locked(fd->read_watcher); - } - if (fd->write_watcher && fd->write_watcher != fd->read_watcher) { - pollset_kick_locked(fd->write_watcher); - } -} - -static int has_watchers(grpc_fd *fd) { - return fd->read_watcher != NULL || fd->write_watcher != NULL || - fd->inactive_watcher_root.next != &fd->inactive_watcher_root; -} - static void close_fd_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { fd->closed = 1; if (!fd->released) { @@ -454,11 +356,7 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, } gpr_mu_lock(&fd->mu); REF_BY(fd, 1, reason); /* remove active status, but keep referenced */ - if (!has_watchers(fd)) { - close_fd_locked(exec_ctx, fd); - } else { - wake_all_watchers_locked(fd); - } + close_fd_locked(exec_ctx, fd); gpr_mu_unlock(&fd->mu); UNREF_BY(fd, 2, reason); /* drop the reference */ } @@ -489,7 +387,6 @@ static void notify_on_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, /* already ready ==> queue the closure to run immediately */ *st = CLOSURE_NOT_READY; grpc_exec_ctx_enqueue(exec_ctx, closure, !fd->shutdown, NULL); - maybe_wake_one_watcher_locked(fd); } else { /* upcallptr was set to a different closure. This is an error! */ gpr_log(GPR_ERROR, @@ -540,111 +437,6 @@ static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, gpr_mu_unlock(&fd->mu); } -static uint32_t fd_begin_poll(grpc_fd *fd, grpc_pollset *pollset, - grpc_pollset_worker *worker, uint32_t read_mask, - uint32_t write_mask, grpc_fd_watcher *watcher) { - uint32_t mask = 0; - grpc_closure *cur; - int requested; - /* keep track of pollers that have requested our events, in case they change - */ - GRPC_FD_REF(fd, "poll"); - - gpr_mu_lock(&fd->mu); - - /* if we are shutdown, then don't add to the watcher set */ - if (fd->shutdown) { - watcher->fd = NULL; - watcher->pollset = NULL; - watcher->worker = NULL; - gpr_mu_unlock(&fd->mu); - GRPC_FD_UNREF(fd, "poll"); - return 0; - } - - /* if there is nobody polling for read, but we need to, then start doing so */ - cur = fd->read_closure; - requested = cur != CLOSURE_READY; - if (read_mask && fd->read_watcher == NULL && requested) { - fd->read_watcher = watcher; - mask |= read_mask; - } - /* if there is nobody polling for write, but we need to, then start doing so - */ - cur = fd->write_closure; - requested = cur != CLOSURE_READY; - if (write_mask && fd->write_watcher == NULL && requested) { - fd->write_watcher = watcher; - mask |= write_mask; - } - /* if not polling, remember this watcher in case we need someone to later */ - if (mask == 0 && worker != NULL) { - watcher->next = &fd->inactive_watcher_root; - watcher->prev = watcher->next->prev; - watcher->next->prev = watcher->prev->next = watcher; - } - watcher->pollset = pollset; - watcher->worker = worker; - watcher->fd = fd; - gpr_mu_unlock(&fd->mu); - - return mask; -} - -static void fd_end_poll(grpc_exec_ctx *exec_ctx, grpc_fd_watcher *watcher, - int got_read, int got_write) { - int was_polling = 0; - int kick = 0; - grpc_fd *fd = watcher->fd; - - if (fd == NULL) { - return; - } - - gpr_mu_lock(&fd->mu); - - if (watcher == fd->read_watcher) { - /* remove read watcher, kick if we still need a read */ - was_polling = 1; - if (!got_read) { - kick = 1; - } - fd->read_watcher = NULL; - } - if (watcher == fd->write_watcher) { - /* remove write watcher, kick if we still need a write */ - was_polling = 1; - if (!got_write) { - kick = 1; - } - fd->write_watcher = NULL; - } - if (!was_polling && watcher->worker != NULL) { - /* remove from inactive list */ - watcher->next->prev = watcher->prev; - watcher->prev->next = watcher->next; - } - if (got_read) { - if (set_ready_locked(exec_ctx, fd, &fd->read_closure)) { - kick = 1; - } - } - if (got_write) { - if (set_ready_locked(exec_ctx, fd, &fd->write_closure)) { - kick = 1; - } - } - if (kick) { - maybe_wake_one_watcher_locked(fd); - } - if (fd_is_orphaned(fd) && !has_watchers(fd) && !fd->closed) { - close_fd_locked(exec_ctx, fd); - } - gpr_mu_unlock(&fd->mu); - - GRPC_FD_UNREF(fd, "poll"); -} - /******************************************************************************* * pollset_posix.c */ @@ -1085,32 +877,45 @@ static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, epoll_hdr *h = pollset->data.ptr; struct epoll_event ev; int err; - grpc_fd_watcher watcher; - - /* We pretend to be polling whilst adding an fd to keep the fd from being - closed during the add. This may result in a spurious wakeup being assigned - to this pollset whilst adding, but that should be benign. */ - /* TODO (sreek). This fd_begin_poll() really seem to accomplish adding - * GRPC_FD_REF() (i.e adding a refcount to the fd) and checking that the - * fd is not shutting down (in which case watcher.fd will be NULL and no - * refcount is added). The ref count is added only durng hte duration of - * adding it to the epoll set (after which fd_end_poll would be called and - * the fd's ref count is decremented by 1. So do we still need fd_begin_poll - * ??? */ - GPR_ASSERT(fd_begin_poll(fd, pollset, NULL, 0, 0, &watcher) == 0); - if (watcher.fd != NULL) { - ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); - ev.data.ptr = fd; - err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); - if (err < 0) { - /* FDs may be added to a pollset multiple times, so EEXIST is normal. */ - if (errno != EEXIST) { - gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", fd->fd, - strerror(errno)); - } + + /* Hold a ref to the fd to keep it from being closed during the add. This may + result in a spurious wakeup being assigned to this pollset whilst adding, + but that should be benign. */ + /* TODO: (sreek): Understand how a spurious wake up migh be assinged to this + * pollset..and how holding a reference will prevent the fd from being closed + * (and perhaps more importantly, see how can an fd be closed while being + * added to the epollset */ + GRPC_FD_REF(fd, "add fd"); + + gpr_mu_lock(&fd->mu); + if (fd->shutdown) { + gpr_mu_unlock(&fd->mu); + GRPC_FD_UNREF(fd, "add fd"); + return; + } + gpr_mu_unlock(&fd->mu); + + ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); + ev.data.ptr = fd; + err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); + if (err < 0) { + /* FDs may be added to a pollset multiple times, so EEXIST is normal. */ + if (errno != EEXIST) { + gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", fd->fd, + strerror(errno)); } } - fd_end_poll(exec_ctx, &watcher, 0, 0); + + /* The fd might have been orphaned while we were adding it to the epoll set. + Close the fd in such a case (which will also take care of removing it from + the epoll set */ + gpr_mu_lock(&fd->mu); + if (fd_is_orphaned(fd) && !fd->closed) { + close_fd_locked(exec_ctx, fd); + } + gpr_mu_unlock(&fd->mu); + + GRPC_FD_UNREF(fd, "add fd"); } /* Creates an epoll fd and initializes the pollset */ From e9ee1f34b8efdc47ea12821351e5cc23125d62b2 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Sat, 14 May 2016 17:22:10 -0700 Subject: [PATCH 016/215] Minor refactor of add_fd path --- src/core/lib/iomgr/ev_epoll_posix.c | 24 +++++++----------------- 1 file changed, 7 insertions(+), 17 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index 920be1951fa..15126b3b62a 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -617,16 +617,13 @@ static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { + /* TODO (sreek) - Does reading pollset->data.ptr need pollset->mu lock ? + * because finally_add_fd() also reads it but without the lock! */ gpr_mu_lock(&pollset->mu); - multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fd); -/* the following (enabled only in debug) will reacquire and then release - our lock - meaning that if the unlocking flag passed to add_fd above is - not respected, the code will deadlock (in a way that we have a chance of - debugging) */ -#ifndef NDEBUG - gpr_mu_lock(&pollset->mu); + GPR_ASSERT(pollset->data.ptr != NULL); gpr_mu_unlock(&pollset->mu); -#endif + + multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fd); } /* TODO (sreek): Remove multipoll_with_epoll_finish_shutdown() declaration */ @@ -874,6 +871,8 @@ typedef struct { int epoll_fd; } epoll_hdr; static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { + /*TODO: (sree) Shouldn't this read (pollset->data.ptr) be done under a + pollset lock - i.e pollset->mu ? */ epoll_hdr *h = pollset->data.ptr; struct epoll_event ev; int err; @@ -941,9 +940,6 @@ static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { ev.events = (uint32_t)(EPOLLIN | EPOLLET); ev.data.ptr = NULL; - /* TODO (sreek): Double-check the use of grpc_global_wakeup_fd here (right now - * I do not know why this is used. I just copied this code from - * epoll_become_mutipoller() function in ev_poll_and_epoll_posix.c file */ err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); if (err < 0) { @@ -956,12 +952,6 @@ static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { - GPR_ASSERT(pollset->data.ptr != NULL); - - /* TODO(sreek). Remove this unlock code (and also the code that acquires the - * lock before calling multipoll_with_epoll_add_fd() function */ - - gpr_mu_unlock(&pollset->mu); finally_add_fd(exec_ctx, pollset, fd); } From c22eb5ac4dbf44209f0d431b6d1e9267210e0120 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Sat, 14 May 2016 19:22:36 -0700 Subject: [PATCH 017/215] Add epoll polling strategy to run_tests.py --- tools/run_tests/run_tests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 0538dce4198..ddaf96c345a 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -892,7 +892,7 @@ for l in languages: language_make_options=[] if any(language.make_options() for language in languages): - if not 'gcov' in args.config and len(languages) != 1: + if len(languages) != 1: print 'languages with custom make options cannot be built simultaneously with other languages' sys.exit(1) else: From 2ea165911b23099499da0af48088c47c47d659ba Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 17 May 2016 09:37:48 -0700 Subject: [PATCH 018/215] experiment with signals --- src/core/lib/iomgr/ev_epoll_posix.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index 15126b3b62a..4481bab4380 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include #include @@ -120,6 +121,7 @@ struct grpc_pollset_worker { grpc_cached_wakeup_fd *wakeup_fd; int reevaluate_polling_on_wakeup; int kicked_specifically; + pthread_t pt_id; struct grpc_pollset_worker *next; struct grpc_pollset_worker *prev; }; @@ -506,6 +508,8 @@ static void pollset_kick_ext(grpc_pollset *p, } specific_worker->kicked_specifically = 1; grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + /* TODO (sreek): Refactor this into a separate file*/ + pthread_kill(specific_worker->pt_id, SIGUSR1); } else if ((flags & GRPC_POLLSET_CAN_KICK_SELF) != 0) { GPR_TIMER_MARK("kick_yoself", 0); if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { @@ -551,10 +555,15 @@ static void pollset_kick(grpc_pollset *p, /* global state management */ +static void sig_handler(int sig_num) { + gpr_log(GPR_INFO, "Received signal %d", sig_num); +} + static void pollset_global_init(void) { gpr_tls_init(&g_current_thread_poller); gpr_tls_init(&g_current_thread_worker); grpc_wakeup_fd_init(&grpc_global_wakeup_fd); + signal(SIGUSR1, sig_handler); } static void pollset_global_shutdown(void) { @@ -663,6 +672,9 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_wakeup_fd_init(&worker.wakeup_fd->fd); } worker.kicked_specifically = 0; + + /* TODO(sreek): Abstract this thread id stuff out into a separate file */ + worker.pt_id = pthread_self(); /* If we're shutting down then we don't execute any extended work */ if (pollset->shutting_down) { GPR_TIMER_MARK("pollset_work.shutting_down", 0); From f448c34a6839f75476900a4a2b24b2160fe4d164 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 19 May 2016 10:51:24 -0700 Subject: [PATCH 019/215] Remove union { } data and epoll_hdr structures. Added ev_epoll_linux files --- BUILD | 6 + Makefile | 2 + binding.gyp | 1 + build.yaml | 2 + config.m4 | 1 + gRPC.podspec | 3 + grpc.gemspec | 2 + package.xml | 2 + src/core/lib/iomgr/ev_epoll_linux.c | 1335 +++++++++++++++++ src/core/lib/iomgr/ev_epoll_linux.h | 41 + src/core/lib/iomgr/ev_epoll_posix.c | 87 +- src/core/lib/iomgr/ev_posix.c | 7 +- src/python/grpcio/grpc_core_dependencies.py | 1 + tools/doxygen/Doxyfile.core.internal | 2 + tools/run_tests/sources_and_headers.json | 3 + vsprojects/vcxproj/grpc/grpc.vcxproj | 3 + vsprojects/vcxproj/grpc/grpc.vcxproj.filters | 6 + .../grpc_unsecure/grpc_unsecure.vcxproj | 3 + .../grpc_unsecure.vcxproj.filters | 6 + 19 files changed, 1442 insertions(+), 71 deletions(-) create mode 100644 src/core/lib/iomgr/ev_epoll_linux.c create mode 100644 src/core/lib/iomgr/ev_epoll_linux.h diff --git a/BUILD b/BUILD index 0be8f27a01b..a32352ebb30 100644 --- a/BUILD +++ b/BUILD @@ -178,6 +178,7 @@ cc_library( "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_linux.h", "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", @@ -322,6 +323,7 @@ cc_library( "src/core/lib/iomgr/endpoint.c", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_linux.c", "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", @@ -548,6 +550,7 @@ cc_library( "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_linux.h", "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", @@ -669,6 +672,7 @@ cc_library( "src/core/lib/iomgr/endpoint.c", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_linux.c", "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", @@ -1362,6 +1366,7 @@ objc_library( "src/core/lib/iomgr/endpoint.c", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_linux.c", "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", @@ -1567,6 +1572,7 @@ objc_library( "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_linux.h", "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", diff --git a/Makefile b/Makefile index 29ebc0e5adf..063698d943f 100644 --- a/Makefile +++ b/Makefile @@ -2486,6 +2486,7 @@ LIBGRPC_SRC = \ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ + src/core/lib/iomgr/ev_epoll_linux.c \ src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ @@ -2841,6 +2842,7 @@ LIBGRPC_UNSECURE_SRC = \ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ + src/core/lib/iomgr/ev_epoll_linux.c \ src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ diff --git a/binding.gyp b/binding.gyp index 89774ead4da..41e1b5bb416 100644 --- a/binding.gyp +++ b/binding.gyp @@ -581,6 +581,7 @@ 'src/core/lib/iomgr/endpoint.c', 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', + 'src/core/lib/iomgr/ev_epoll_linux.c', 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', diff --git a/build.yaml b/build.yaml index 7ba65332972..2f3d07071da 100644 --- a/build.yaml +++ b/build.yaml @@ -165,6 +165,7 @@ filegroups: - src/core/lib/iomgr/closure.h - src/core/lib/iomgr/endpoint.h - src/core/lib/iomgr/endpoint_pair.h + - src/core/lib/iomgr/ev_epoll_linux.h - src/core/lib/iomgr/ev_epoll_posix.h - src/core/lib/iomgr/ev_poll_posix.h - src/core/lib/iomgr/ev_posix.h @@ -240,6 +241,7 @@ filegroups: - src/core/lib/iomgr/endpoint.c - src/core/lib/iomgr/endpoint_pair_posix.c - src/core/lib/iomgr/endpoint_pair_windows.c + - src/core/lib/iomgr/ev_epoll_linux.c - src/core/lib/iomgr/ev_epoll_posix.c - src/core/lib/iomgr/ev_poll_posix.c - src/core/lib/iomgr/ev_posix.c diff --git a/config.m4 b/config.m4 index 6987c741541..4308295afda 100644 --- a/config.m4 +++ b/config.m4 @@ -100,6 +100,7 @@ if test "$PHP_GRPC" != "no"; then src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ + src/core/lib/iomgr/ev_epoll_linux.c \ src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ diff --git a/gRPC.podspec b/gRPC.podspec index 3b4dd52380e..de55880125a 100644 --- a/gRPC.podspec +++ b/gRPC.podspec @@ -181,6 +181,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/closure.h', 'src/core/lib/iomgr/endpoint.h', 'src/core/lib/iomgr/endpoint_pair.h', + 'src/core/lib/iomgr/ev_epoll_linux.h', 'src/core/lib/iomgr/ev_epoll_posix.h', 'src/core/lib/iomgr/ev_poll_posix.h', 'src/core/lib/iomgr/ev_posix.h', @@ -359,6 +360,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/endpoint.c', 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', + 'src/core/lib/iomgr/ev_epoll_linux.c', 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', @@ -548,6 +550,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/closure.h', 'src/core/lib/iomgr/endpoint.h', 'src/core/lib/iomgr/endpoint_pair.h', + 'src/core/lib/iomgr/ev_epoll_linux.h', 'src/core/lib/iomgr/ev_epoll_posix.h', 'src/core/lib/iomgr/ev_poll_posix.h', 'src/core/lib/iomgr/ev_posix.h', diff --git a/grpc.gemspec b/grpc.gemspec index 71cccb6ca8d..54ae2eb68dd 100755 --- a/grpc.gemspec +++ b/grpc.gemspec @@ -190,6 +190,7 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/closure.h ) s.files += %w( src/core/lib/iomgr/endpoint.h ) s.files += %w( src/core/lib/iomgr/endpoint_pair.h ) + s.files += %w( src/core/lib/iomgr/ev_epoll_linux.h ) s.files += %w( src/core/lib/iomgr/ev_epoll_posix.h ) s.files += %w( src/core/lib/iomgr/ev_poll_posix.h ) s.files += %w( src/core/lib/iomgr/ev_posix.h ) @@ -338,6 +339,7 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/endpoint.c ) s.files += %w( src/core/lib/iomgr/endpoint_pair_posix.c ) s.files += %w( src/core/lib/iomgr/endpoint_pair_windows.c ) + s.files += %w( src/core/lib/iomgr/ev_epoll_linux.c ) s.files += %w( src/core/lib/iomgr/ev_epoll_posix.c ) s.files += %w( src/core/lib/iomgr/ev_poll_posix.c ) s.files += %w( src/core/lib/iomgr/ev_posix.c ) diff --git a/package.xml b/package.xml index 0fc5d0dee47..d8e82a8bc37 100644 --- a/package.xml +++ b/package.xml @@ -197,6 +197,7 @@ + @@ -345,6 +346,7 @@ + diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c new file mode 100644 index 00000000000..f257ac8a1dd --- /dev/null +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -0,0 +1,1335 @@ +/* + * + * Copyright 2016, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include + +#ifdef GPR_POSIX_SOCKET + +#include "src/core/lib/iomgr/ev_epoll_posix.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "src/core/lib/iomgr/ev_posix.h" +#include "src/core/lib/iomgr/iomgr_internal.h" +#include "src/core/lib/iomgr/wakeup_fd_posix.h" +#include "src/core/lib/profiling/timers.h" +#include "src/core/lib/support/block_annotate.h" + +struct polling_island; + +/******************************************************************************* + * FD declarations + */ + +struct grpc_fd { + int fd; + /* refst format: + bit0: 1=active/0=orphaned + bit1-n: refcount + meaning that mostly we ref by two to avoid altering the orphaned bit, + and just unref by 1 when we're ready to flag the object as orphaned */ + gpr_atm refst; + + gpr_mu mu; + int shutdown; + int closed; + int released; + + grpc_closure *read_closure; + grpc_closure *write_closure; + + /* Mutex protecting the 'polling_island' field */ + gpr_mu pi_mu; + + /* The polling island to which this fd belongs to. An fd belongs to exactly + one polling island */ + struct polling_island *polling_island; + + struct grpc_fd *freelist_next; + + grpc_closure *on_done_closure; + + grpc_iomgr_object iomgr_object; +}; + +/* Return 1 if this fd is orphaned, 0 otherwise */ +static bool fd_is_orphaned(grpc_fd *fd); + +/* Reference counting for fds */ +/*#define GRPC_FD_REF_COUNT_DEBUG*/ +#ifdef GRPC_FD_REF_COUNT_DEBUG +static void fd_ref(grpc_fd *fd, const char *reason, const char *file, int line); +static void fd_unref(grpc_fd *fd, const char *reason, const char *file, + int line); +#define GRPC_FD_REF(fd, reason) fd_ref(fd, reason, __FILE__, __LINE__) +#define GRPC_FD_UNREF(fd, reason) fd_unref(fd, reason, __FILE__, __LINE__) +#else +static void fd_ref(grpc_fd *fd); +static void fd_unref(grpc_fd *fd); +#define GRPC_FD_REF(fd, reason) fd_ref(fd) +#define GRPC_FD_UNREF(fd, reason) fd_unref(fd) +#endif + +static void fd_global_init(void); +static void fd_global_shutdown(void); + +#define CLOSURE_NOT_READY ((grpc_closure *)0) +#define CLOSURE_READY ((grpc_closure *)1) + +/******************************************************************************* + * Polling Island + */ +typedef struct polling_island { + gpr_mu mu; + int ref_cnt; + + /* Pointer to the polling_island this merged into. If this is not NULL, all + the remaining fields in this pollset (i.e all fields except mu and ref_cnt) + are considered invalid and must be ignored */ + struct polling_island *merged_to; + + /* The fd of the underlying epoll set */ + int epoll_fd; + + /* The file descriptors in the epoll set */ + size_t fd_cnt; + size_t fd_capacity; + grpc_fd **fds; + + /* Polling islands that are no longer needed are kept in a freelist so that + they can be reused. This field points to the next polling island in the + free list. Note that this is only used if the polling island is in the + free list */ + struct polling_island *next_free; +} polling_island; + +/* Polling island freelist */ +static gpr_mu g_pi_freelist_mu; +static polling_island *g_pi_freelist = NULL; + +/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? */ +static void add_fd_to_polling_island_locked(polling_island *pi, grpc_fd *fd) { + int err; + struct epoll_event ev; + + ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); + ev.data.ptr = fd; + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); + + if (err < 0 && errno != EEXIST) { + gpr_log(GPR_ERROR, "epoll_ctl add for fd: %d failed with error: %s", fd->fd, + strerror(errno)); + return; + } + + pi->fd_capacity = GPR_MAX(pi->fd_capacity + 8, pi->fd_cnt * 3 / 2); + pi->fds = gpr_realloc(pi->fds, sizeof(grpc_fd *) * pi->fd_capacity); + pi->fds[pi->fd_cnt++] = fd; +} + +static polling_island *polling_island_create(int initial_ref_cnt, + grpc_fd *initial_fd) { + polling_island *pi = NULL; + gpr_mu_lock(&g_pi_freelist_mu); + if (g_pi_freelist != NULL) { + pi = g_pi_freelist; + g_pi_freelist = g_pi_freelist->next_free; + pi->next_free = NULL; + } + gpr_mu_unlock(&g_pi_freelist_mu); + + /* Create new polling island if we could not get one from the free list */ + if (pi == NULL) { + pi = gpr_malloc(sizeof(*pi)); + gpr_mu_init(&pi->mu); + pi->fd_cnt = 0; + pi->fd_capacity = 0; + pi->fds = NULL; + + pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); + if (pi->epoll_fd < 0) { + gpr_log(GPR_ERROR, "epoll_create1() failed with error: %s", + strerror(errno)); + } + GPR_ASSERT(pi->epoll_fd >= 0); + } + + pi->ref_cnt = initial_ref_cnt; + pi->merged_to = NULL; + pi->next_free = NULL; + + if (initial_fd != NULL) { + /* add_fd_to_polling_island_locked() expects the caller to hold a pi->mu + * lock. However, since this is a new polling island (and no one has a + * reference to it yet), it is okay to not acquire pi->mu here */ + add_fd_to_polling_island_locked(pi, initial_fd); + } + + return pi; +} + +static void polling_island_global_init() { + polling_island_create(0, NULL); /* TODO(sreek): Delete this line */ + gpr_mu_init(&g_pi_freelist_mu); + g_pi_freelist = NULL; +} + +/******************************************************************************* + * pollset declarations + */ + +typedef struct grpc_cached_wakeup_fd { + grpc_wakeup_fd fd; + struct grpc_cached_wakeup_fd *next; +} grpc_cached_wakeup_fd; + +struct grpc_pollset_worker { + grpc_cached_wakeup_fd *wakeup_fd; + int reevaluate_polling_on_wakeup; + int kicked_specifically; + pthread_t pt_id; + struct grpc_pollset_worker *next; + struct grpc_pollset_worker *prev; +}; + +struct grpc_pollset { + gpr_mu mu; + grpc_pollset_worker root_worker; + int shutting_down; + int called_shutdown; + int kicked_without_pollers; + grpc_closure *shutdown_done; + + int epoll_fd; + + /* Mutex protecting the 'polling_island' field */ + gpr_mu pi_mu; + + /* The polling island to which this fd belongs to. An fd belongs to exactly + one polling island */ + struct polling_island *polling_island; + + /* Local cache of eventfds for workers */ + grpc_cached_wakeup_fd *local_wakeup_cache; +}; + +/* Add an fd to a pollset */ +static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + struct grpc_fd *fd); + +static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd); + +/* Convert a timespec to milliseconds: + - very small or negative poll times are clamped to zero to do a + non-blocking poll (which becomes spin polling) + - other small values are rounded up to one millisecond + - longer than a millisecond polls are rounded up to the next nearest + millisecond to avoid spinning + - infinite timeouts are converted to -1 */ +static int poll_deadline_to_millis_timeout(gpr_timespec deadline, + gpr_timespec now); + +/* Allow kick to wakeup the currently polling worker */ +#define GRPC_POLLSET_CAN_KICK_SELF 1 +/* Force the wakee to repoll when awoken */ +#define GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP 2 +/* As per pollset_kick, with an extended set of flags (defined above) + -- mostly for fd_posix's use. */ +static void pollset_kick_ext(grpc_pollset *p, + grpc_pollset_worker *specific_worker, + uint32_t flags); + +/* turn a pollset into a multipoller: platform specific */ +typedef void (*platform_become_multipoller_type)(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, + struct grpc_fd **fds, + size_t fd_count); + +/* Return 1 if the pollset has active threads in pollset_work (pollset must + * be locked) */ +static int pollset_has_workers(grpc_pollset *pollset); + +static void remove_fd_from_all_epoll_sets(int fd); + +/******************************************************************************* + * pollset_set definitions + */ + +struct grpc_pollset_set { + gpr_mu mu; + + size_t pollset_count; + size_t pollset_capacity; + grpc_pollset **pollsets; + + size_t pollset_set_count; + size_t pollset_set_capacity; + struct grpc_pollset_set **pollset_sets; + + size_t fd_count; + size_t fd_capacity; + grpc_fd **fds; +}; + +/******************************************************************************* + * fd_posix.c + */ + +/* We need to keep a freelist not because of any concerns of malloc performance + * but instead so that implementations with multiple threads in (for example) + * epoll_wait deal with the race between pollset removal and incoming poll + * notifications. + * + * The problem is that the poller ultimately holds a reference to this + * object, so it is very difficult to know when is safe to free it, at least + * without some expensive synchronization. + * + * If we keep the object freelisted, in the worst case losing this race just + * becomes a spurious read notification on a reused fd. + */ +/* TODO(klempner): We could use some form of polling generation count to know + * when these are safe to free. */ +/* TODO(klempner): Consider disabling freelisting if we don't have multiple + * threads in poll on the same fd */ +/* TODO(klempner): Batch these allocations to reduce fragmentation */ +static grpc_fd *fd_freelist = NULL; +static gpr_mu fd_freelist_mu; + +static void freelist_fd(grpc_fd *fd) { + gpr_mu_lock(&fd_freelist_mu); + fd->freelist_next = fd_freelist; + fd_freelist = fd; + grpc_iomgr_unregister_object(&fd->iomgr_object); + gpr_mu_unlock(&fd_freelist_mu); +} + +static grpc_fd *alloc_fd(int fd) { + grpc_fd *r = NULL; + + gpr_mu_lock(&fd_freelist_mu); + if (fd_freelist != NULL) { + r = fd_freelist; + fd_freelist = fd_freelist->freelist_next; + } + gpr_mu_unlock(&fd_freelist_mu); + + if (r == NULL) { + r = gpr_malloc(sizeof(grpc_fd)); + gpr_mu_init(&r->mu); + gpr_mu_init(&r->pi_mu); + } + + /* TODO: sreek - check with ctiller on why we need to acquire a lock here */ + gpr_mu_lock(&r->mu); + gpr_atm_rel_store(&r->refst, 1); + r->shutdown = 0; + r->read_closure = CLOSURE_NOT_READY; + r->write_closure = CLOSURE_NOT_READY; + r->fd = fd; + r->polling_island = NULL; + r->freelist_next = NULL; + r->on_done_closure = NULL; + r->closed = 0; + r->released = 0; + gpr_mu_unlock(&r->mu); + return r; +} + +static void destroy(grpc_fd *fd) { + gpr_mu_destroy(&fd->mu); + gpr_free(fd); +} + +#ifdef GRPC_FD_REF_COUNT_DEBUG +#define REF_BY(fd, n, reason) ref_by(fd, n, reason, __FILE__, __LINE__) +#define UNREF_BY(fd, n, reason) unref_by(fd, n, reason, __FILE__, __LINE__) +static void ref_by(grpc_fd *fd, int n, const char *reason, const char *file, + int line) { + gpr_log(GPR_DEBUG, "FD %d %p ref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, + gpr_atm_no_barrier_load(&fd->refst), + gpr_atm_no_barrier_load(&fd->refst) + n, reason, file, line); +#else +#define REF_BY(fd, n, reason) ref_by(fd, n) +#define UNREF_BY(fd, n, reason) unref_by(fd, n) +static void ref_by(grpc_fd *fd, int n) { +#endif + GPR_ASSERT(gpr_atm_no_barrier_fetch_add(&fd->refst, n) > 0); +} + +#ifdef GRPC_FD_REF_COUNT_DEBUG +static void unref_by(grpc_fd *fd, int n, const char *reason, const char *file, + int line) { + gpr_atm old; + gpr_log(GPR_DEBUG, "FD %d %p unref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, + gpr_atm_no_barrier_load(&fd->refst), + gpr_atm_no_barrier_load(&fd->refst) - n, reason, file, line); +#else +static void unref_by(grpc_fd *fd, int n) { + gpr_atm old; +#endif + old = gpr_atm_full_fetch_add(&fd->refst, -n); + if (old == n) { + freelist_fd(fd); + } else { + GPR_ASSERT(old > n); + } +} + +static void fd_global_init(void) { gpr_mu_init(&fd_freelist_mu); } + +static void fd_global_shutdown(void) { + gpr_mu_lock(&fd_freelist_mu); + gpr_mu_unlock(&fd_freelist_mu); + while (fd_freelist != NULL) { + grpc_fd *fd = fd_freelist; + fd_freelist = fd_freelist->freelist_next; + destroy(fd); + } + gpr_mu_destroy(&fd_freelist_mu); +} + +static grpc_fd *fd_create(int fd, const char *name) { + grpc_fd *r = alloc_fd(fd); + char *name2; + gpr_asprintf(&name2, "%s fd=%d", name, fd); + grpc_iomgr_register_object(&r->iomgr_object, name2); + gpr_free(name2); +#ifdef GRPC_FD_REF_COUNT_DEBUG + gpr_log(GPR_DEBUG, "FD %d %p create %s", fd, r, name); +#endif + return r; +} + +static bool fd_is_orphaned(grpc_fd *fd) { + return (gpr_atm_acq_load(&fd->refst) & 1) == 0; +} + +static void close_fd_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + fd->closed = 1; + if (!fd->released) { + close(fd->fd); + } else { + remove_fd_from_all_epoll_sets(fd->fd); + } + grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); +} + +static int fd_wrapped_fd(grpc_fd *fd) { + if (fd->released || fd->closed) { + return -1; + } else { + return fd->fd; + } +} + +/* TODO: sreek - do something here with the pollset island link */ +static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *on_done, int *release_fd, + const char *reason) { + fd->on_done_closure = on_done; + fd->released = release_fd != NULL; + if (!fd->released) { + shutdown(fd->fd, SHUT_RDWR); + } else { + *release_fd = fd->fd; + } + gpr_mu_lock(&fd->mu); + REF_BY(fd, 1, reason); /* remove active status, but keep referenced */ + close_fd_locked(exec_ctx, fd); + gpr_mu_unlock(&fd->mu); + UNREF_BY(fd, 2, reason); /* drop the reference */ +} + +/* increment refcount by two to avoid changing the orphan bit */ +#ifdef GRPC_FD_REF_COUNT_DEBUG +static void fd_ref(grpc_fd *fd, const char *reason, const char *file, + int line) { + ref_by(fd, 2, reason, file, line); +} + +static void fd_unref(grpc_fd *fd, const char *reason, const char *file, + int line) { + unref_by(fd, 2, reason, file, line); +} +#else +static void fd_ref(grpc_fd *fd) { ref_by(fd, 2); } + +static void fd_unref(grpc_fd *fd) { unref_by(fd, 2); } +#endif + +static void notify_on_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure **st, grpc_closure *closure) { + if (*st == CLOSURE_NOT_READY) { + /* not ready ==> switch to a waiting state by setting the closure */ + *st = closure; + } else if (*st == CLOSURE_READY) { + /* already ready ==> queue the closure to run immediately */ + *st = CLOSURE_NOT_READY; + grpc_exec_ctx_enqueue(exec_ctx, closure, !fd->shutdown, NULL); + } else { + /* upcallptr was set to a different closure. This is an error! */ + gpr_log(GPR_ERROR, + "User called a notify_on function with a previous callback still " + "pending"); + abort(); + } +} + +/* returns 1 if state becomes not ready */ +static int set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure **st) { + if (*st == CLOSURE_READY) { + /* duplicate ready ==> ignore */ + return 0; + } else if (*st == CLOSURE_NOT_READY) { + /* not ready, and not waiting ==> flag ready */ + *st = CLOSURE_READY; + return 0; + } else { + /* waiting ==> queue closure */ + grpc_exec_ctx_enqueue(exec_ctx, *st, !fd->shutdown, NULL); + *st = CLOSURE_NOT_READY; + return 1; + } +} + +/* Do something here with the pollset island link (?) */ +static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + gpr_mu_lock(&fd->mu); + GPR_ASSERT(!fd->shutdown); + fd->shutdown = 1; + set_ready_locked(exec_ctx, fd, &fd->read_closure); + set_ready_locked(exec_ctx, fd, &fd->write_closure); + gpr_mu_unlock(&fd->mu); +} + +static void fd_notify_on_read(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *closure) { + gpr_mu_lock(&fd->mu); + notify_on_locked(exec_ctx, fd, &fd->read_closure, closure); + gpr_mu_unlock(&fd->mu); +} + +static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *closure) { + gpr_mu_lock(&fd->mu); + notify_on_locked(exec_ctx, fd, &fd->write_closure, closure); + gpr_mu_unlock(&fd->mu); +} + +/******************************************************************************* + * pollset_posix.c + */ + +GPR_TLS_DECL(g_current_thread_poller); +GPR_TLS_DECL(g_current_thread_worker); + +/** The alarm system needs to be able to wakeup 'some poller' sometimes + * (specifically when a new alarm needs to be triggered earlier than the next + * alarm 'epoch'). + * This wakeup_fd gives us something to alert on when such a case occurs. */ +grpc_wakeup_fd grpc_global_wakeup_fd; + +static void remove_worker(grpc_pollset *p, grpc_pollset_worker *worker) { + worker->prev->next = worker->next; + worker->next->prev = worker->prev; +} + +static int pollset_has_workers(grpc_pollset *p) { + return p->root_worker.next != &p->root_worker; +} + +static grpc_pollset_worker *pop_front_worker(grpc_pollset *p) { + if (pollset_has_workers(p)) { + grpc_pollset_worker *w = p->root_worker.next; + remove_worker(p, w); + return w; + } else { + return NULL; + } +} + +static void push_back_worker(grpc_pollset *p, grpc_pollset_worker *worker) { + worker->next = &p->root_worker; + worker->prev = worker->next->prev; + worker->prev->next = worker->next->prev = worker; +} + +static void push_front_worker(grpc_pollset *p, grpc_pollset_worker *worker) { + worker->prev = &p->root_worker; + worker->next = worker->prev->next; + worker->prev->next = worker->next->prev = worker; +} + +static void pollset_kick_ext(grpc_pollset *p, + grpc_pollset_worker *specific_worker, + uint32_t flags) { + GPR_TIMER_BEGIN("pollset_kick_ext", 0); + + /* pollset->mu already held */ + if (specific_worker != NULL) { + if (specific_worker == GRPC_POLLSET_KICK_BROADCAST) { + GPR_TIMER_BEGIN("pollset_kick_ext.broadcast", 0); + GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); + for (specific_worker = p->root_worker.next; + specific_worker != &p->root_worker; + specific_worker = specific_worker->next) { + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + } + p->kicked_without_pollers = 1; + GPR_TIMER_END("pollset_kick_ext.broadcast", 0); + } else if (gpr_tls_get(&g_current_thread_worker) != + (intptr_t)specific_worker) { + GPR_TIMER_MARK("different_thread_worker", 0); + if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { + specific_worker->reevaluate_polling_on_wakeup = 1; + } + specific_worker->kicked_specifically = 1; + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + /* TODO (sreek): Refactor this into a separate file*/ + pthread_kill(specific_worker->pt_id, SIGUSR1); + } else if ((flags & GRPC_POLLSET_CAN_KICK_SELF) != 0) { + GPR_TIMER_MARK("kick_yoself", 0); + if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { + specific_worker->reevaluate_polling_on_wakeup = 1; + } + specific_worker->kicked_specifically = 1; + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + } + } else if (gpr_tls_get(&g_current_thread_poller) != (intptr_t)p) { + GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); + GPR_TIMER_MARK("kick_anonymous", 0); + specific_worker = pop_front_worker(p); + if (specific_worker != NULL) { + if (gpr_tls_get(&g_current_thread_worker) == (intptr_t)specific_worker) { + GPR_TIMER_MARK("kick_anonymous_not_self", 0); + push_back_worker(p, specific_worker); + specific_worker = pop_front_worker(p); + if ((flags & GRPC_POLLSET_CAN_KICK_SELF) == 0 && + gpr_tls_get(&g_current_thread_worker) == + (intptr_t)specific_worker) { + push_back_worker(p, specific_worker); + specific_worker = NULL; + } + } + if (specific_worker != NULL) { + GPR_TIMER_MARK("finally_kick", 0); + push_back_worker(p, specific_worker); + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + } + } else { + GPR_TIMER_MARK("kicked_no_pollers", 0); + p->kicked_without_pollers = 1; + } + } + + GPR_TIMER_END("pollset_kick_ext", 0); +} + +static void pollset_kick(grpc_pollset *p, + grpc_pollset_worker *specific_worker) { + pollset_kick_ext(p, specific_worker, 0); +} + +/* global state management */ + +static void sig_handler(int sig_num) { + gpr_log(GPR_INFO, "Received signal %d", sig_num); +} + +static void pollset_global_init(void) { + gpr_tls_init(&g_current_thread_poller); + gpr_tls_init(&g_current_thread_worker); + grpc_wakeup_fd_init(&grpc_global_wakeup_fd); + signal(SIGUSR1, sig_handler); +} + +static void pollset_global_shutdown(void) { + grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); + gpr_tls_destroy(&g_current_thread_poller); + gpr_tls_destroy(&g_current_thread_worker); +} + +static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } + +/* TODO: sreek. Try to Remove this forward declaration*/ +static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset); + +/* main interface */ + +static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { + gpr_mu_init(&pollset->mu); + *mu = &pollset->mu; + pollset->root_worker.next = pollset->root_worker.prev = &pollset->root_worker; + gpr_mu_init(&pollset->pi_mu); + pollset->polling_island = NULL; + pollset->shutting_down = 0; + pollset->called_shutdown = 0; + pollset->kicked_without_pollers = 0; + pollset->local_wakeup_cache = NULL; + pollset->kicked_without_pollers = 0; + + multipoll_with_epoll_pollset_create_efd(pollset); +} + +/* TODO(sreek): Maybe merge multipoll_*_destroy() with pollset_destroy() + * function */ +static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset); + +static void pollset_destroy(grpc_pollset *pollset) { + GPR_ASSERT(!pollset_has_workers(pollset)); + + multipoll_with_epoll_pollset_destroy(pollset); + + while (pollset->local_wakeup_cache) { + grpc_cached_wakeup_fd *next = pollset->local_wakeup_cache->next; + grpc_wakeup_fd_destroy(&pollset->local_wakeup_cache->fd); + gpr_free(pollset->local_wakeup_cache); + pollset->local_wakeup_cache = next; + } + gpr_mu_destroy(&pollset->pi_mu); + gpr_mu_destroy(&pollset->mu); +} + +/* TODO(sreek) - Do something with the pollset island link (??) */ +static void pollset_reset(grpc_pollset *pollset) { + GPR_ASSERT(pollset->shutting_down); + GPR_ASSERT(!pollset_has_workers(pollset)); + pollset->shutting_down = 0; + pollset->called_shutdown = 0; + pollset->kicked_without_pollers = 0; +} + +/* TODO (sreek): Remove multipoll_with_epoll_finish_shutdown() declaration */ +static void multipoll_with_epoll_pollset_finish_shutdown(grpc_pollset *pollset); + +static void finish_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { + multipoll_with_epoll_pollset_finish_shutdown(pollset); + grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); +} + +/* TODO(sreek): Remove multipoll_with_epoll_*_maybe_work_and_unlock declaration + */ +static void multipoll_with_epoll_pollset_maybe_work_and_unlock( + grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, + gpr_timespec deadline, gpr_timespec now); + +static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_pollset_worker **worker_hdl, gpr_timespec now, + gpr_timespec deadline) { + grpc_pollset_worker worker; + *worker_hdl = &worker; + + /* pollset->mu already held */ + int added_worker = 0; + int locked = 1; + int queued_work = 0; + int keep_polling = 0; + GPR_TIMER_BEGIN("pollset_work", 0); + /* this must happen before we (potentially) drop pollset->mu */ + worker.next = worker.prev = NULL; + worker.reevaluate_polling_on_wakeup = 0; + if (pollset->local_wakeup_cache != NULL) { + worker.wakeup_fd = pollset->local_wakeup_cache; + pollset->local_wakeup_cache = worker.wakeup_fd->next; + } else { + worker.wakeup_fd = gpr_malloc(sizeof(*worker.wakeup_fd)); + grpc_wakeup_fd_init(&worker.wakeup_fd->fd); + } + worker.kicked_specifically = 0; + + /* TODO(sreek): Abstract this thread id stuff out into a separate file */ + worker.pt_id = pthread_self(); + /* If we're shutting down then we don't execute any extended work */ + if (pollset->shutting_down) { + GPR_TIMER_MARK("pollset_work.shutting_down", 0); + goto done; + } + /* Start polling, and keep doing so while we're being asked to + re-evaluate our pollers (this allows poll() based pollers to + ensure they don't miss wakeups) */ + keep_polling = 1; + while (keep_polling) { + keep_polling = 0; + if (!pollset->kicked_without_pollers) { + if (!added_worker) { + push_front_worker(pollset, &worker); + added_worker = 1; + gpr_tls_set(&g_current_thread_worker, (intptr_t)&worker); + } + gpr_tls_set(&g_current_thread_poller, (intptr_t)pollset); + GPR_TIMER_BEGIN("maybe_work_and_unlock", 0); + + multipoll_with_epoll_pollset_maybe_work_and_unlock( + exec_ctx, pollset, &worker, deadline, now); + + GPR_TIMER_END("maybe_work_and_unlock", 0); + locked = 0; + gpr_tls_set(&g_current_thread_poller, 0); + } else { + GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); + pollset->kicked_without_pollers = 0; + } + /* Finished execution - start cleaning up. + Note that we may arrive here from outside the enclosing while() loop. + In that case we won't loop though as we haven't added worker to the + worker list, which means nobody could ask us to re-evaluate polling). */ + done: + if (!locked) { + queued_work |= grpc_exec_ctx_flush(exec_ctx); + gpr_mu_lock(&pollset->mu); + locked = 1; + } + /* If we're forced to re-evaluate polling (via pollset_kick with + GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) then we land here and force + a loop */ + if (worker.reevaluate_polling_on_wakeup) { + worker.reevaluate_polling_on_wakeup = 0; + pollset->kicked_without_pollers = 0; + if (queued_work || worker.kicked_specifically) { + /* If there's queued work on the list, then set the deadline to be + immediate so we get back out of the polling loop quickly */ + deadline = gpr_inf_past(GPR_CLOCK_MONOTONIC); + } + keep_polling = 1; + } + } + if (added_worker) { + remove_worker(pollset, &worker); + gpr_tls_set(&g_current_thread_worker, 0); + } + /* release wakeup fd to the local pool */ + worker.wakeup_fd->next = pollset->local_wakeup_cache; + pollset->local_wakeup_cache = worker.wakeup_fd; + /* check shutdown conditions */ + if (pollset->shutting_down) { + if (pollset_has_workers(pollset)) { + pollset_kick(pollset, NULL); + } else if (!pollset->called_shutdown) { + pollset->called_shutdown = 1; + gpr_mu_unlock(&pollset->mu); + finish_shutdown(exec_ctx, pollset); + grpc_exec_ctx_flush(exec_ctx); + /* Continuing to access pollset here is safe -- it is the caller's + * responsibility to not destroy when it has outstanding calls to + * pollset_work. + * TODO(dklempner): Can we refactor the shutdown logic to avoid this? */ + gpr_mu_lock(&pollset->mu); + } + } + *worker_hdl = NULL; + GPR_TIMER_END("pollset_work", 0); +} + +/* TODO: (sreek) Do something with the pollset island link */ +static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_closure *closure) { + GPR_ASSERT(!pollset->shutting_down); + pollset->shutting_down = 1; + pollset->shutdown_done = closure; + pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); + + if (!pollset->called_shutdown && !pollset_has_workers(pollset)) { + pollset->called_shutdown = 1; + finish_shutdown(exec_ctx, pollset); + } +} + +static int poll_deadline_to_millis_timeout(gpr_timespec deadline, + gpr_timespec now) { + gpr_timespec timeout; + static const int64_t max_spin_polling_us = 10; + if (gpr_time_cmp(deadline, gpr_inf_future(deadline.clock_type)) == 0) { + return -1; + } + if (gpr_time_cmp(deadline, gpr_time_add(now, gpr_time_from_micros( + max_spin_polling_us, + GPR_TIMESPAN))) <= 0) { + return 0; + } + timeout = gpr_time_sub(deadline, now); + return gpr_time_to_millis(gpr_time_add( + timeout, gpr_time_from_nanos(GPR_NS_PER_MS - 1, GPR_TIMESPAN))); +} + +/******************************************************************************* + * pollset_multipoller_with_epoll_posix.c + */ + +static void set_ready(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st) { + /* only one set_ready can be active at once (but there may be a racing + notify_on) */ + gpr_mu_lock(&fd->mu); + set_ready_locked(exec_ctx, fd, st); + gpr_mu_unlock(&fd->mu); +} + +static void fd_become_readable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + set_ready(exec_ctx, fd, &fd->read_closure); +} + +static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + set_ready(exec_ctx, fd, &fd->write_closure); +} + +/* TODO (sreek): Maybe this global list is not required. Double check*/ +struct epoll_fd_list { + int *epoll_fds; + size_t count; + size_t capacity; +}; + +static struct epoll_fd_list epoll_fd_global_list; +static gpr_once init_epoll_fd_list_mu = GPR_ONCE_INIT; +static gpr_mu epoll_fd_list_mu; + +static void init_mu(void) { gpr_mu_init(&epoll_fd_list_mu); } + +static void add_epoll_fd_to_global_list(int epoll_fd) { + gpr_once_init(&init_epoll_fd_list_mu, init_mu); + + gpr_mu_lock(&epoll_fd_list_mu); + if (epoll_fd_global_list.count == epoll_fd_global_list.capacity) { + epoll_fd_global_list.capacity = + GPR_MAX((size_t)8, epoll_fd_global_list.capacity * 2); + epoll_fd_global_list.epoll_fds = + gpr_realloc(epoll_fd_global_list.epoll_fds, + epoll_fd_global_list.capacity * sizeof(int)); + } + epoll_fd_global_list.epoll_fds[epoll_fd_global_list.count++] = epoll_fd; + gpr_mu_unlock(&epoll_fd_list_mu); +} + +static void remove_epoll_fd_from_global_list(int epoll_fd) { + gpr_mu_lock(&epoll_fd_list_mu); + GPR_ASSERT(epoll_fd_global_list.count > 0); + for (size_t i = 0; i < epoll_fd_global_list.count; i++) { + if (epoll_fd == epoll_fd_global_list.epoll_fds[i]) { + epoll_fd_global_list.epoll_fds[i] = + epoll_fd_global_list.epoll_fds[--(epoll_fd_global_list.count)]; + break; + } + } + gpr_mu_unlock(&epoll_fd_list_mu); +} + +static void remove_fd_from_all_epoll_sets(int fd) { + int err; + gpr_once_init(&init_epoll_fd_list_mu, init_mu); + gpr_mu_lock(&epoll_fd_list_mu); + if (epoll_fd_global_list.count == 0) { + gpr_mu_unlock(&epoll_fd_list_mu); + return; + } + for (size_t i = 0; i < epoll_fd_global_list.count; i++) { + err = epoll_ctl(epoll_fd_global_list.epoll_fds[i], EPOLL_CTL_DEL, fd, NULL); + if (err < 0 && errno != ENOENT) { + gpr_log(GPR_ERROR, "epoll_ctl del for %d failed: %s", fd, + strerror(errno)); + } + } + gpr_mu_unlock(&epoll_fd_list_mu); +} + +/* TODO: sreek - This function multipoll_with_epoll_pollset_add_fd() and + * finally_add_fd() in ev_poll_and_epoll_posix.c */ +static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_fd *fd) { + + /* TODO sreek - Check if we need to get a pollset->mu lock here */ + + struct epoll_event ev; + int err; + + /* Hold a ref to the fd to keep it from being closed during the add. This may + result in a spurious wakeup being assigned to this pollset whilst adding, + but that should be benign. */ + /* TODO: (sreek): Understand how a spurious wake up migh be assinged to this + * pollset..and how holding a reference will prevent the fd from being closed + * (and perhaps more importantly, see how can an fd be closed while being + * added to the epollset */ + GRPC_FD_REF(fd, "add fd"); + + gpr_mu_lock(&fd->mu); + if (fd->shutdown) { + gpr_mu_unlock(&fd->mu); + GRPC_FD_UNREF(fd, "add fd"); + return; + } + gpr_mu_unlock(&fd->mu); + + ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); + ev.data.ptr = fd; + err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); + if (err < 0) { + /* FDs may be added to a pollset multiple times, so EEXIST is normal. */ + if (errno != EEXIST) { + gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", fd->fd, + strerror(errno)); + } + } + + /* The fd might have been orphaned while we were adding it to the epoll set. + Close the fd in such a case (which will also take care of removing it from + the epoll set */ + gpr_mu_lock(&fd->mu); + if (fd_is_orphaned(fd) && !fd->closed) { + close_fd_locked(exec_ctx, fd); + } + gpr_mu_unlock(&fd->mu); + + GRPC_FD_UNREF(fd, "add fd"); +} + +/* Creates an epoll fd and initializes the pollset */ +/* TODO: This has to be called ONLY from pollset_init function. and hence it + * does not acquire any lock */ +static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { + struct epoll_event ev; + int err; + + pollset->epoll_fd = epoll_create1(EPOLL_CLOEXEC); + if (pollset->epoll_fd < 0) { + gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); + abort(); + } + add_epoll_fd_to_global_list(pollset->epoll_fd); + + ev.events = (uint32_t)(EPOLLIN | EPOLLET); + ev.data.ptr = NULL; + + err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); + if (err < 0) { + gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), + strerror(errno)); + } +} + +/* TODO(klempner): We probably want to turn this down a bit */ +#define GRPC_EPOLL_MAX_EVENTS 1000 + +static void multipoll_with_epoll_pollset_maybe_work_and_unlock( + grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, + gpr_timespec deadline, gpr_timespec now) { + struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; + int epoll_fd = pollset->epoll_fd; + int ep_rv; + int poll_rv; + int timeout_ms; + struct pollfd pfds[2]; + + /* If you want to ignore epoll's ability to sanely handle parallel pollers, + * for a more apples-to-apples performance comparison with poll, add a + * if (pollset->counter != 0) { return 0; } + * here. + */ + + gpr_mu_unlock(&pollset->mu); + + timeout_ms = poll_deadline_to_millis_timeout(deadline, now); + + pfds[0].fd = GRPC_WAKEUP_FD_GET_READ_FD(&worker->wakeup_fd->fd); + pfds[0].events = POLLIN; + pfds[0].revents = 0; + pfds[1].fd = epoll_fd; + pfds[1].events = POLLIN; + pfds[1].revents = 0; + + /* TODO(vpai): Consider first doing a 0 timeout poll here to avoid + even going into the blocking annotation if possible */ + GPR_TIMER_BEGIN("poll", 0); + GRPC_SCHEDULING_START_BLOCKING_REGION; + poll_rv = grpc_poll_function(pfds, 2, timeout_ms); + GRPC_SCHEDULING_END_BLOCKING_REGION; + GPR_TIMER_END("poll", 0); + + if (poll_rv < 0) { + if (errno != EINTR) { + gpr_log(GPR_ERROR, "poll() failed: %s", strerror(errno)); + } + } else if (poll_rv == 0) { + /* do nothing */ + } else { + if (pfds[0].revents) { + grpc_wakeup_fd_consume_wakeup(&worker->wakeup_fd->fd); + } + if (pfds[1].revents) { + do { + /* The following epoll_wait never blocks; it has a timeout of 0 */ + ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); + if (ep_rv < 0) { + if (errno != EINTR) { + gpr_log(GPR_ERROR, "epoll_wait() failed: %s", strerror(errno)); + } + } else { + int i; + for (i = 0; i < ep_rv; ++i) { + grpc_fd *fd = ep_ev[i].data.ptr; + /* TODO(klempner): We might want to consider making err and pri + * separate events */ + int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + int write_ev = ep_ev[i].events & EPOLLOUT; + if (fd == NULL) { + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + } else { + if (read_ev || cancel) { + fd_become_readable(exec_ctx, fd); + } + if (write_ev || cancel) { + fd_become_writable(exec_ctx, fd); + } + } + } + } + } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); + } + } +} + +static void multipoll_with_epoll_pollset_finish_shutdown( + grpc_pollset *pollset) {} + +static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset) { + close(pollset->epoll_fd); + remove_epoll_fd_from_global_list(pollset->epoll_fd); +} + +/******************************************************************************* + * pollset_set_posix.c + */ + +static grpc_pollset_set *pollset_set_create(void) { + grpc_pollset_set *pollset_set = gpr_malloc(sizeof(*pollset_set)); + memset(pollset_set, 0, sizeof(*pollset_set)); + gpr_mu_init(&pollset_set->mu); + return pollset_set; +} + +static void pollset_set_destroy(grpc_pollset_set *pollset_set) { + size_t i; + gpr_mu_destroy(&pollset_set->mu); + for (i = 0; i < pollset_set->fd_count; i++) { + GRPC_FD_UNREF(pollset_set->fds[i], "pollset_set"); + } + gpr_free(pollset_set->pollsets); + gpr_free(pollset_set->pollset_sets); + gpr_free(pollset_set->fds); + gpr_free(pollset_set); +} + +static void pollset_set_add_pollset(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, + grpc_pollset *pollset) { + size_t i, j; + gpr_mu_lock(&pollset_set->mu); + if (pollset_set->pollset_count == pollset_set->pollset_capacity) { + pollset_set->pollset_capacity = + GPR_MAX(8, 2 * pollset_set->pollset_capacity); + pollset_set->pollsets = + gpr_realloc(pollset_set->pollsets, pollset_set->pollset_capacity * + sizeof(*pollset_set->pollsets)); + } + pollset_set->pollsets[pollset_set->pollset_count++] = pollset; + for (i = 0, j = 0; i < pollset_set->fd_count; i++) { + if (fd_is_orphaned(pollset_set->fds[i])) { + GRPC_FD_UNREF(pollset_set->fds[i], "pollset_set"); + } else { + pollset_add_fd(exec_ctx, pollset, pollset_set->fds[i]); + pollset_set->fds[j++] = pollset_set->fds[i]; + } + } + pollset_set->fd_count = j; + gpr_mu_unlock(&pollset_set->mu); +} + +static void pollset_set_del_pollset(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, + grpc_pollset *pollset) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + for (i = 0; i < pollset_set->pollset_count; i++) { + if (pollset_set->pollsets[i] == pollset) { + pollset_set->pollset_count--; + GPR_SWAP(grpc_pollset *, pollset_set->pollsets[i], + pollset_set->pollsets[pollset_set->pollset_count]); + break; + } + } + gpr_mu_unlock(&pollset_set->mu); +} + +static void pollset_set_add_pollset_set(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *bag, + grpc_pollset_set *item) { + size_t i, j; + gpr_mu_lock(&bag->mu); + if (bag->pollset_set_count == bag->pollset_set_capacity) { + bag->pollset_set_capacity = GPR_MAX(8, 2 * bag->pollset_set_capacity); + bag->pollset_sets = + gpr_realloc(bag->pollset_sets, + bag->pollset_set_capacity * sizeof(*bag->pollset_sets)); + } + bag->pollset_sets[bag->pollset_set_count++] = item; + for (i = 0, j = 0; i < bag->fd_count; i++) { + if (fd_is_orphaned(bag->fds[i])) { + GRPC_FD_UNREF(bag->fds[i], "pollset_set"); + } else { + pollset_set_add_fd(exec_ctx, item, bag->fds[i]); + bag->fds[j++] = bag->fds[i]; + } + } + bag->fd_count = j; + gpr_mu_unlock(&bag->mu); +} + +static void pollset_set_del_pollset_set(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *bag, + grpc_pollset_set *item) { + size_t i; + gpr_mu_lock(&bag->mu); + for (i = 0; i < bag->pollset_set_count; i++) { + if (bag->pollset_sets[i] == item) { + bag->pollset_set_count--; + GPR_SWAP(grpc_pollset_set *, bag->pollset_sets[i], + bag->pollset_sets[bag->pollset_set_count]); + break; + } + } + gpr_mu_unlock(&bag->mu); +} + +static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + if (pollset_set->fd_count == pollset_set->fd_capacity) { + pollset_set->fd_capacity = GPR_MAX(8, 2 * pollset_set->fd_capacity); + pollset_set->fds = gpr_realloc( + pollset_set->fds, pollset_set->fd_capacity * sizeof(*pollset_set->fds)); + } + GRPC_FD_REF(fd, "pollset_set"); + pollset_set->fds[pollset_set->fd_count++] = fd; + for (i = 0; i < pollset_set->pollset_count; i++) { + pollset_add_fd(exec_ctx, pollset_set->pollsets[i], fd); + } + for (i = 0; i < pollset_set->pollset_set_count; i++) { + pollset_set_add_fd(exec_ctx, pollset_set->pollset_sets[i], fd); + } + gpr_mu_unlock(&pollset_set->mu); +} + +static void pollset_set_del_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + for (i = 0; i < pollset_set->fd_count; i++) { + if (pollset_set->fds[i] == fd) { + pollset_set->fd_count--; + GPR_SWAP(grpc_fd *, pollset_set->fds[i], + pollset_set->fds[pollset_set->fd_count]); + GRPC_FD_UNREF(fd, "pollset_set"); + break; + } + } + for (i = 0; i < pollset_set->pollset_set_count; i++) { + pollset_set_del_fd(exec_ctx, pollset_set->pollset_sets[i], fd); + } + gpr_mu_unlock(&pollset_set->mu); +} + +/******************************************************************************* + * event engine binding + */ + +static void shutdown_engine(void) { + fd_global_shutdown(); + pollset_global_shutdown(); +} + +static const grpc_event_engine_vtable vtable = { + .pollset_size = sizeof(grpc_pollset), + + .fd_create = fd_create, + .fd_wrapped_fd = fd_wrapped_fd, + .fd_orphan = fd_orphan, + .fd_shutdown = fd_shutdown, + .fd_notify_on_read = fd_notify_on_read, + .fd_notify_on_write = fd_notify_on_write, + + .pollset_init = pollset_init, + .pollset_shutdown = pollset_shutdown, + .pollset_reset = pollset_reset, + .pollset_destroy = pollset_destroy, + .pollset_work = pollset_work, + .pollset_kick = pollset_kick, + .pollset_add_fd = pollset_add_fd, + + .pollset_set_create = pollset_set_create, + .pollset_set_destroy = pollset_set_destroy, + .pollset_set_add_pollset = pollset_set_add_pollset, + .pollset_set_del_pollset = pollset_set_del_pollset, + .pollset_set_add_pollset_set = pollset_set_add_pollset_set, + .pollset_set_del_pollset_set = pollset_set_del_pollset_set, + .pollset_set_add_fd = pollset_set_add_fd, + .pollset_set_del_fd = pollset_set_del_fd, + + .kick_poller = kick_poller, + + .shutdown_engine = shutdown_engine, +}; + +const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { + fd_global_init(); + pollset_global_init(); + polling_island_global_init(); + return &vtable; +} + +#endif diff --git a/src/core/lib/iomgr/ev_epoll_linux.h b/src/core/lib/iomgr/ev_epoll_linux.h new file mode 100644 index 00000000000..8c819975a4c --- /dev/null +++ b/src/core/lib/iomgr/ev_epoll_linux.h @@ -0,0 +1,41 @@ +/* + * + * Copyright 2015, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#ifndef GRPC_CORE_LIB_IOMGR_EV_EPOLL_LINUX_H +#define GRPC_CORE_LIB_IOMGR_EV_EPOLL_LINUX_H + +#include "src/core/lib/iomgr/ev_posix.h" + +const grpc_event_engine_vtable *grpc_init_epoll_linux(void); + +#endif /* GRPC_CORE_LIB_IOMGR_EV_EPOLL_LINUX_H */ diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index 4481bab4380..5abd5b2a94c 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -42,6 +42,7 @@ #include #include #include +#include #include #include @@ -51,11 +52,13 @@ #include #include +#include "src/core/lib/iomgr/ev_posix.h" #include "src/core/lib/iomgr/iomgr_internal.h" #include "src/core/lib/iomgr/wakeup_fd_posix.h" #include "src/core/lib/profiling/timers.h" #include "src/core/lib/support/block_annotate.h" + /******************************************************************************* * FD declarations */ @@ -133,10 +136,9 @@ struct grpc_pollset { int called_shutdown; int kicked_without_pollers; grpc_closure *shutdown_done; - union { - int fd; - void *ptr; - } data; + + int epoll_fd; + /* Local cache of eventfds for workers */ grpc_cached_wakeup_fd *local_wakeup_cache; }; @@ -589,7 +591,6 @@ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { pollset->local_wakeup_cache = NULL; pollset->kicked_without_pollers = 0; - pollset->data.ptr = NULL; multipoll_with_epoll_pollset_create_efd(pollset); } @@ -619,22 +620,6 @@ static void pollset_reset(grpc_pollset *pollset) { pollset->kicked_without_pollers = 0; } -/* TODO (sreek): Remove multipoll_with_epoll_add_fd declaration*/ -static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, - grpc_fd *fd); - -static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_fd *fd) { - /* TODO (sreek) - Does reading pollset->data.ptr need pollset->mu lock ? - * because finally_add_fd() also reads it but without the lock! */ - gpr_mu_lock(&pollset->mu); - GPR_ASSERT(pollset->data.ptr != NULL); - gpr_mu_unlock(&pollset->mu); - - multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fd); -} - /* TODO (sreek): Remove multipoll_with_epoll_finish_shutdown() declaration */ static void multipoll_with_epoll_pollset_finish_shutdown(grpc_pollset *pollset); @@ -790,20 +775,6 @@ static int poll_deadline_to_millis_timeout(gpr_timespec deadline, * pollset_multipoller_with_epoll_posix.c */ -#include -#include -#include -#include -#include - -#include -#include -#include - -#include "src/core/lib/iomgr/ev_posix.h" -#include "src/core/lib/profiling/timers.h" -#include "src/core/lib/support/block_annotate.h" - static void set_ready(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st) { /* only one set_ready can be active at once (but there may be a racing notify_on) */ @@ -879,13 +850,13 @@ static void remove_fd_from_all_epoll_sets(int fd) { gpr_mu_unlock(&epoll_fd_list_mu); } -typedef struct { int epoll_fd; } epoll_hdr; - -static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, +/* TODO: sreek - This function multipoll_with_epoll_pollset_add_fd() and + * finally_add_fd() in ev_poll_and_epoll_posix.c */ +static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { - /*TODO: (sree) Shouldn't this read (pollset->data.ptr) be done under a - pollset lock - i.e pollset->mu ? */ - epoll_hdr *h = pollset->data.ptr; + + /* TODO sreek - Check if we need to get a pollset->mu lock here */ + struct epoll_event ev; int err; @@ -908,7 +879,7 @@ static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); ev.data.ptr = fd; - err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); + err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); if (err < 0) { /* FDs may be added to a pollset multiple times, so EEXIST is normal. */ if (errno != EEXIST) { @@ -933,26 +904,20 @@ static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, /* TODO: This has to be called ONLY from pollset_init function. and hence it * does not acquire any lock */ static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { - epoll_hdr *h = gpr_malloc(sizeof(epoll_hdr)); struct epoll_event ev; int err; - /* TODO (sreek). remove this assert. Currently added this just to ensure that - * we do not overwrite h->epoll_fd without freeing the older one*/ - GPR_ASSERT(pollset->data.ptr == NULL); - - pollset->data.ptr = h; - h->epoll_fd = epoll_create1(EPOLL_CLOEXEC); - if (h->epoll_fd < 0) { + pollset->epoll_fd = epoll_create1(EPOLL_CLOEXEC); + if (pollset->epoll_fd < 0) { gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); abort(); } - add_epoll_fd_to_global_list(h->epoll_fd); + add_epoll_fd_to_global_list(pollset->epoll_fd); ev.events = (uint32_t)(EPOLLIN | EPOLLET); ev.data.ptr = NULL; - err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, + err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); if (err < 0) { gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", @@ -961,12 +926,6 @@ static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { } } -static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, - grpc_fd *fd) { - finally_add_fd(exec_ctx, pollset, fd); -} - /* TODO(klempner): We probably want to turn this down a bit */ #define GRPC_EPOLL_MAX_EVENTS 1000 @@ -974,9 +933,9 @@ static void multipoll_with_epoll_pollset_maybe_work_and_unlock( grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, gpr_timespec deadline, gpr_timespec now) { struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; + int epoll_fd = pollset->epoll_fd; int ep_rv; int poll_rv; - epoll_hdr *h = pollset->data.ptr; int timeout_ms; struct pollfd pfds[2]; @@ -993,7 +952,7 @@ static void multipoll_with_epoll_pollset_maybe_work_and_unlock( pfds[0].fd = GRPC_WAKEUP_FD_GET_READ_FD(&worker->wakeup_fd->fd); pfds[0].events = POLLIN; pfds[0].revents = 0; - pfds[1].fd = h->epoll_fd; + pfds[1].fd = epoll_fd; pfds[1].events = POLLIN; pfds[1].revents = 0; @@ -1018,7 +977,7 @@ static void multipoll_with_epoll_pollset_maybe_work_and_unlock( if (pfds[1].revents) { do { /* The following epoll_wait never blocks; it has a timeout of 0 */ - ep_rv = epoll_wait(h->epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); + ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); if (ep_rv < 0) { if (errno != EINTR) { gpr_log(GPR_ERROR, "epoll_wait() failed: %s", strerror(errno)); @@ -1053,10 +1012,8 @@ static void multipoll_with_epoll_pollset_finish_shutdown( grpc_pollset *pollset) {} static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset) { - epoll_hdr *h = pollset->data.ptr; - close(h->epoll_fd); - remove_epoll_fd_from_global_list(h->epoll_fd); - gpr_free(h); + close(pollset->epoll_fd); + remove_epoll_fd_from_global_list(pollset->epoll_fd); } /******************************************************************************* diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index baa3b9856ad..404ef2a64b9 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -44,7 +44,7 @@ #include #include -#include "src/core/lib/iomgr/ev_epoll_posix.h" +#include "src/core/lib/iomgr/ev_epoll_linux.h" #include "src/core/lib/iomgr/ev_poll_posix.h" #include "src/core/lib/support/env.h" @@ -163,11 +163,6 @@ void grpc_fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, g_event_engine->fd_notify_on_write(exec_ctx, fd, closure); } -grpc_pollset *grpc_fd_get_read_notifier_pollset(grpc_exec_ctx *exec_ctx, - grpc_fd *fd) { - return g_event_engine->fd_get_read_notifier_pollset(exec_ctx, fd); -} - size_t grpc_pollset_size(void) { return g_event_engine->pollset_size; } void grpc_pollset_init(grpc_pollset *pollset, gpr_mu **mu) { diff --git a/src/python/grpcio/grpc_core_dependencies.py b/src/python/grpcio/grpc_core_dependencies.py index 49b4ddc457c..13bc6888d66 100644 --- a/src/python/grpcio/grpc_core_dependencies.py +++ b/src/python/grpcio/grpc_core_dependencies.py @@ -94,6 +94,7 @@ CORE_SOURCE_FILES = [ 'src/core/lib/iomgr/endpoint.c', 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', + 'src/core/lib/iomgr/ev_epoll_linux.c', 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal index 36b25cca161..d968278f2a4 100644 --- a/tools/doxygen/Doxyfile.core.internal +++ b/tools/doxygen/Doxyfile.core.internal @@ -807,6 +807,7 @@ src/core/lib/http/parser.h \ src/core/lib/iomgr/closure.h \ src/core/lib/iomgr/endpoint.h \ src/core/lib/iomgr/endpoint_pair.h \ +src/core/lib/iomgr/ev_epoll_linux.h \ src/core/lib/iomgr/ev_epoll_posix.h \ src/core/lib/iomgr/ev_poll_posix.h \ src/core/lib/iomgr/ev_posix.h \ @@ -955,6 +956,7 @@ src/core/lib/iomgr/closure.c \ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ +src/core/lib/iomgr/ev_epoll_linux.c \ src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index 43940495864..97cc55db36e 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -5530,6 +5530,7 @@ "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_linux.h", "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", @@ -5630,6 +5631,8 @@ "src/core/lib/iomgr/endpoint_pair.h", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_linux.c", + "src/core/lib/iomgr/ev_epoll_linux.h", "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.c", diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj b/vsprojects/vcxproj/grpc/grpc.vcxproj index 55304af5869..a67e4d16dad 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj @@ -316,6 +316,7 @@ + @@ -484,6 +485,8 @@ + + diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters index 7d1c90fda7c..bf9b7dc7dcf 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters @@ -55,6 +55,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr @@ -677,6 +680,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj index 3d0cdfc668b..afc9a2ca1b2 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj @@ -304,6 +304,7 @@ + @@ -450,6 +451,8 @@ + + diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters index d2ff4c630fd..b7507f9a963 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters @@ -58,6 +58,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr @@ -575,6 +578,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr From 9442bab5d303d7bd33e9406129ad897588d07111 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 20 May 2016 17:54:06 -0700 Subject: [PATCH 020/215] Write most of the methods in the new epoll implementation --- src/core/lib/iomgr/ev_epoll_linux.c | 301 ++++++++++++++++++++++------ 1 file changed, 244 insertions(+), 57 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index f257ac8a1dd..0d30bb659b6 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -150,28 +150,84 @@ typedef struct polling_island { static gpr_mu g_pi_freelist_mu; static polling_island *g_pi_freelist = NULL; -/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? */ -static void add_fd_to_polling_island_locked(polling_island *pi, grpc_fd *fd) { +/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? + * TODO: sreek - Should this add a ref to the grpc_fd ? */ +/* The caller is expected to hold pi->mu lock before calling this function */ +static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, + size_t fd_count) { int err; + size_t i; struct epoll_event ev; - ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); - ev.data.ptr = fd; - err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); + for (i = 0; i < fd_count; i++) { + ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); + ev.data.ptr = fds[i]; + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, fds[i]->fd, &ev); + + if (err < 0 && errno != EEXIST) { + gpr_log(GPR_ERROR, "epoll_ctl add for fd: %d failed with error: %s", + fds[i]->fd, strerror(errno)); + /* TODO: sreek - Not sure if it is a good idea to continue here. We need a + * better way to bubble up this error instead of doing an abort() */ + continue; + } - if (err < 0 && errno != EEXIST) { - gpr_log(GPR_ERROR, "epoll_ctl add for fd: %d failed with error: %s", fd->fd, - strerror(errno)); - return; + if (pi->fd_cnt == pi->fd_capacity) { + pi->fd_capacity = GPR_MAX(pi->fd_capacity + 8, pi->fd_cnt * 3 / 2); + pi->fds = gpr_realloc(pi->fds, sizeof(grpc_fd *) * pi->fd_capacity); + } + + pi->fds[pi->fd_cnt++] = fds[i]; + } +} + +/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? + * TODO: sreek - Might have to unref the fds (assuming whether we add a ref to + * the fd when adding it to the epollset) */ +/* The caller is expected to hold pi->mu lock before calling this function */ +static void polling_island_clear_fds_locked(polling_island *pi) { + int err; + size_t i; + + for (i = 0; i < pi->fd_cnt; i++) { + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, pi->fds[i]->fd, NULL); + + if (err < 0 && errno != ENOENT) { + gpr_log(GPR_ERROR, + "epoll_ctl delete for fds[i]: %d failed with error: %s", i, + pi->fds[i]->fd, strerror(errno)); + /* TODO: sreek - Not sure if it is a good idea to continue here. We need a + * better way to bubble up this error instead of doing an abort() */ + continue; + } + } + + pi->fd_cnt = 0; +} + +/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? + * TODO: sreek - Might have to unref the fd (assuming whether we add a ref to + * the fd when adding it to the epollset) */ +/* The caller is expected to hold pi->mu lock before calling this function */ +static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd) { + int err; + size_t i; + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, fd->fd, NULL); + if (err < 0 && errno != ENOENT) { + gpr_log(GPR_ERROR, "epoll_ctl delete for fd: %d failed with error; %s", + fd->fd, strerror(errno)); } - pi->fd_capacity = GPR_MAX(pi->fd_capacity + 8, pi->fd_cnt * 3 / 2); - pi->fds = gpr_realloc(pi->fds, sizeof(grpc_fd *) * pi->fd_capacity); - pi->fds[pi->fd_cnt++] = fd; + for (i = 0; i < pi->fd_cnt; i++) { + if (pi->fds[i] == fd) { + pi->fds[i] = pi->fds[--pi->fd_cnt]; + break; + } + } } -static polling_island *polling_island_create(int initial_ref_cnt, - grpc_fd *initial_fd) { +static polling_island *polling_island_create(grpc_fd *initial_fd, + int initial_ref_cnt) { polling_island *pi = NULL; gpr_mu_lock(&g_pi_freelist_mu); if (g_pi_freelist != NULL) { @@ -202,17 +258,151 @@ static polling_island *polling_island_create(int initial_ref_cnt, pi->next_free = NULL; if (initial_fd != NULL) { - /* add_fd_to_polling_island_locked() expects the caller to hold a pi->mu + /* polling_island_add_fds_locked() expects the caller to hold a pi->mu * lock. However, since this is a new polling island (and no one has a * reference to it yet), it is okay to not acquire pi->mu here */ - add_fd_to_polling_island_locked(pi, initial_fd); + polling_island_add_fds_locked(pi, &initial_fd, 1); } return pi; } +static void polling_island_delete(polling_island *pi) { + GPR_ASSERT(pi->ref_cnt == 0); + GPR_ASSERT(pi->fd_cnt == 0); + + pi->merged_to = NULL; + + gpr_mu_lock(&g_pi_freelist_mu); + pi->next_free = g_pi_freelist; + g_pi_freelist = pi; + gpr_mu_unlock(&g_pi_freelist_mu); +} + +void polling_island_unref_and_unlock(polling_island *pi, int unref_by) { + pi->ref_cnt -= unref_by; + int ref_cnt = pi->ref_cnt; + GPR_ASSERT(ref_cnt >= 0); + + gpr_mu_unlock(&pi->mu); + + if (ref_cnt == 0) { + polling_island_delete(pi); + } +} + +polling_island *polling_island_update_and_lock(polling_island *pi, int unref_by, + int add_ref_by) { + polling_island *next = NULL; + gpr_mu_lock(&pi->mu); + while (pi->merged_to != NULL) { + next = pi->merged_to; + polling_island_unref_and_unlock(pi, unref_by); + pi = next; + gpr_mu_lock(&pi->mu); + } + + pi->ref_cnt += add_ref_by; + return pi; +} + +void polling_island_pair_update_and_lock(polling_island **p, + polling_island **q) { + polling_island *pi_1 = *p; + polling_island *pi_2 = *q; + polling_island *temp = NULL; + bool pi_1_locked = false; + bool pi_2_locked = false; + int num_swaps = 0; + + while (pi_1 != pi_2 && !(pi_1_locked && pi_2_locked)) { + // pi_1 is NOT equal to pi_2 + // pi_1 MAY be locked + + if (pi_1 > pi_2) { + if (pi_1_locked) { + gpr_mu_unlock(&pi_1->mu); + pi_1_locked = false; + } + + GPR_SWAP(polling_island *, pi_1, pi_2); + num_swaps++; + } + + // p1 < p2 + // p1 MAY BE locked + // p2 is NOT locked + + if (!pi_1_locked) { + gpr_mu_lock(&pi_1->mu); + pi_1_locked = true; + + if (pi_1->merged_to != NULL) { + temp = pi_1->merged_to; + polling_island_unref_and_unlock(pi_1, 1); + pi_1 = temp; + pi_1_locked = false; + + continue; + } + } + + // p1 is LOCKED + // p2 is UNLOCKED + // p1 != p2 + + gpr_mu_lock(&pi_2->mu); + pi_2_locked = true; + + if (pi_2->merged_to != NULL) { + temp = pi_2->merged_to; + polling_island_unref_and_unlock(pi_2, 1); + pi_2 = temp; + pi_2_locked = false; + } + } + + // Either pi_1 == pi_2 OR we got both locks! + if (pi_1 == pi_2) { + GPR_ASSERT(pi_1_locked || (!pi_1_locked && !pi_2_locked)); + if (!pi_1_locked) { + pi_1 = pi_2 = polling_island_update_and_lock(pi_1, 2, 0); + } + } else { + GPR_ASSERT(pi_1_locked && pi_2_locked); + if (num_swaps % 2 > 0) { + GPR_SWAP(polling_island *, pi_1, pi_2); + } + } + + *p = pi_1; + *q = pi_2; +} + +polling_island *polling_island_merge(polling_island *p, polling_island *q) { + polling_island *merged = NULL; + + polling_island_pair_update_and_lock(&p, &q); + + /* TODO: sreek: Think about this scenario some more. Is it possible ?. what + * does it mean, when would this happen */ + if (p == q) { + merged = p; + } + + // Move all the fds from polling_island p to polling_island q + polling_island_add_fds_locked(q, p->fds, p->fd_cnt); + polling_island_clear_fds_locked(p); + + q->ref_cnt += p->ref_cnt; + + gpr_mu_unlock(&p->mu); + gpr_mu_unlock(&q->mu); + + return merged; +} + static void polling_island_global_init() { - polling_island_create(0, NULL); /* TODO(sreek): Delete this line */ gpr_mu_init(&g_pi_freelist_mu); g_pi_freelist = NULL; } @@ -245,7 +435,7 @@ struct grpc_pollset { int epoll_fd; - /* Mutex protecting the 'polling_island' field */ + /* Mutex protecting the 'polling_island' field */ gpr_mu pi_mu; /* The polling island to which this fd belongs to. An fd belongs to exactly @@ -319,7 +509,8 @@ struct grpc_pollset_set { * fd_posix.c */ -/* We need to keep a freelist not because of any concerns of malloc performance +/* We need to keep a freelist not because of any concerns of malloc + * performance * but instead so that implementations with multiple threads in (for example) * epoll_wait deal with the race between pollset removal and incoming poll * notifications. @@ -434,6 +625,7 @@ static void fd_global_shutdown(void) { static grpc_fd *fd_create(int fd, const char *name) { grpc_fd *r = alloc_fd(fd); + char *name2; gpr_asprintf(&name2, "%s fd=%d", name, fd); grpc_iomgr_register_object(&r->iomgr_object, name2); @@ -453,6 +645,20 @@ static void close_fd_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { if (!fd->released) { close(fd->fd); } else { + /* TODO: sreek - Check for deadlocks */ + + gpr_mu_lock(&fd->pi_mu); + fd->polling_island = + polling_island_update_and_lock(fd->polling_island, 1, 0); + + polling_island_remove_fd_locked(fd->polling_island, fd); + polling_island_unref_and_unlock(fd->polling_island, 1); + + fd->polling_island = NULL; + gpr_mu_unlock(&fd->pi_mu); + + + /* TODO: sreek - This should be no longer needed */ remove_fd_from_all_epoll_sets(fd->fd); } grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); @@ -752,7 +958,8 @@ static void finish_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); } -/* TODO(sreek): Remove multipoll_with_epoll_*_maybe_work_and_unlock declaration +/* TODO(sreek): Remove multipoll_with_epoll_*_maybe_work_and_unlock + * declaration */ static void multipoll_with_epoll_pollset_maybe_work_and_unlock( grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, @@ -979,50 +1186,30 @@ static void remove_fd_from_all_epoll_sets(int fd) { * finally_add_fd() in ev_poll_and_epoll_posix.c */ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { - /* TODO sreek - Check if we need to get a pollset->mu lock here */ + gpr_mu_lock(&pollset->pi_mu); + gpr_mu_lock(&fd->pi_mu); - struct epoll_event ev; - int err; - - /* Hold a ref to the fd to keep it from being closed during the add. This may - result in a spurious wakeup being assigned to this pollset whilst adding, - but that should be benign. */ - /* TODO: (sreek): Understand how a spurious wake up migh be assinged to this - * pollset..and how holding a reference will prevent the fd from being closed - * (and perhaps more importantly, see how can an fd be closed while being - * added to the epollset */ - GRPC_FD_REF(fd, "add fd"); + polling_island *pi_new = NULL; - gpr_mu_lock(&fd->mu); - if (fd->shutdown) { - gpr_mu_unlock(&fd->mu); - GRPC_FD_UNREF(fd, "add fd"); - return; - } - gpr_mu_unlock(&fd->mu); - - ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); - ev.data.ptr = fd; - err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); - if (err < 0) { - /* FDs may be added to a pollset multiple times, so EEXIST is normal. */ - if (errno != EEXIST) { - gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", fd->fd, - strerror(errno)); + if (fd->polling_island == pollset->polling_island) { + pi_new = fd->polling_island; + if (pi_new == NULL) { + pi_new = polling_island_create(fd, 2); } - } + } else if (fd->polling_island == NULL) { + pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); - /* The fd might have been orphaned while we were adding it to the epoll set. - Close the fd in such a case (which will also take care of removing it from - the epoll set */ - gpr_mu_lock(&fd->mu); - if (fd_is_orphaned(fd) && !fd->closed) { - close_fd_locked(exec_ctx, fd); + } else if (pollset->polling_island == NULL) { + pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); + } else { // Non null and different + pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); } - gpr_mu_unlock(&fd->mu); - GRPC_FD_UNREF(fd, "add fd"); + fd->polling_island = pollset->polling_island = pi_new; + + gpr_mu_unlock(&fd->pi_mu); + gpr_mu_unlock(&pollset->pi_mu); } /* Creates an epoll fd and initializes the pollset */ From d806145573f8e78a52012b4b8ab94ff46b855d58 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 20 May 2016 18:12:30 -0700 Subject: [PATCH 021/215] Removed epoll_fd_global_list --- src/core/lib/iomgr/ev_epoll_linux.c | 68 +---------------------------- 1 file changed, 1 insertion(+), 67 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 0d30bb659b6..7793a952016 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -483,8 +483,6 @@ typedef void (*platform_become_multipoller_type)(grpc_exec_ctx *exec_ctx, * be locked) */ static int pollset_has_workers(grpc_pollset *pollset); -static void remove_fd_from_all_epoll_sets(int fd); - /******************************************************************************* * pollset_set definitions */ @@ -656,11 +654,8 @@ static void close_fd_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { fd->polling_island = NULL; gpr_mu_unlock(&fd->pi_mu); - - - /* TODO: sreek - This should be no longer needed */ - remove_fd_from_all_epoll_sets(fd->fd); } + grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); } @@ -1123,65 +1118,6 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { set_ready(exec_ctx, fd, &fd->write_closure); } -/* TODO (sreek): Maybe this global list is not required. Double check*/ -struct epoll_fd_list { - int *epoll_fds; - size_t count; - size_t capacity; -}; - -static struct epoll_fd_list epoll_fd_global_list; -static gpr_once init_epoll_fd_list_mu = GPR_ONCE_INIT; -static gpr_mu epoll_fd_list_mu; - -static void init_mu(void) { gpr_mu_init(&epoll_fd_list_mu); } - -static void add_epoll_fd_to_global_list(int epoll_fd) { - gpr_once_init(&init_epoll_fd_list_mu, init_mu); - - gpr_mu_lock(&epoll_fd_list_mu); - if (epoll_fd_global_list.count == epoll_fd_global_list.capacity) { - epoll_fd_global_list.capacity = - GPR_MAX((size_t)8, epoll_fd_global_list.capacity * 2); - epoll_fd_global_list.epoll_fds = - gpr_realloc(epoll_fd_global_list.epoll_fds, - epoll_fd_global_list.capacity * sizeof(int)); - } - epoll_fd_global_list.epoll_fds[epoll_fd_global_list.count++] = epoll_fd; - gpr_mu_unlock(&epoll_fd_list_mu); -} - -static void remove_epoll_fd_from_global_list(int epoll_fd) { - gpr_mu_lock(&epoll_fd_list_mu); - GPR_ASSERT(epoll_fd_global_list.count > 0); - for (size_t i = 0; i < epoll_fd_global_list.count; i++) { - if (epoll_fd == epoll_fd_global_list.epoll_fds[i]) { - epoll_fd_global_list.epoll_fds[i] = - epoll_fd_global_list.epoll_fds[--(epoll_fd_global_list.count)]; - break; - } - } - gpr_mu_unlock(&epoll_fd_list_mu); -} - -static void remove_fd_from_all_epoll_sets(int fd) { - int err; - gpr_once_init(&init_epoll_fd_list_mu, init_mu); - gpr_mu_lock(&epoll_fd_list_mu); - if (epoll_fd_global_list.count == 0) { - gpr_mu_unlock(&epoll_fd_list_mu); - return; - } - for (size_t i = 0; i < epoll_fd_global_list.count; i++) { - err = epoll_ctl(epoll_fd_global_list.epoll_fds[i], EPOLL_CTL_DEL, fd, NULL); - if (err < 0 && errno != ENOENT) { - gpr_log(GPR_ERROR, "epoll_ctl del for %d failed: %s", fd, - strerror(errno)); - } - } - gpr_mu_unlock(&epoll_fd_list_mu); -} - /* TODO: sreek - This function multipoll_with_epoll_pollset_add_fd() and * finally_add_fd() in ev_poll_and_epoll_posix.c */ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, @@ -1224,7 +1160,6 @@ static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); abort(); } - add_epoll_fd_to_global_list(pollset->epoll_fd); ev.events = (uint32_t)(EPOLLIN | EPOLLET); ev.data.ptr = NULL; @@ -1325,7 +1260,6 @@ static void multipoll_with_epoll_pollset_finish_shutdown( static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset) { close(pollset->epoll_fd); - remove_epoll_fd_from_global_list(pollset->epoll_fd); } /******************************************************************************* From 96b2554313120949dd26e3c0968e8aea9b8a650f Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 20 May 2016 18:14:48 -0700 Subject: [PATCH 022/215] ctiller's ev_epoll_linux.c file (for reference) --- src/core/lib/iomgr/ctiller_ev_epoll_linux.c | 461 ++++++++++++++++++++ 1 file changed, 461 insertions(+) create mode 100644 src/core/lib/iomgr/ctiller_ev_epoll_linux.c diff --git a/src/core/lib/iomgr/ctiller_ev_epoll_linux.c b/src/core/lib/iomgr/ctiller_ev_epoll_linux.c new file mode 100644 index 00000000000..23c20a77aae --- /dev/null +++ b/src/core/lib/iomgr/ctiller_ev_epoll_linux.c @@ -0,0 +1,461 @@ +/* + * + * Copyright 2015-2016, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include "src/core/lib/iomgr/ev_epoll_linux.h" + +#include +#include +#include +#include + +#include +#include +#include +#include + +#include "src/core/lib/iomgr/iomgr_internal.h" + +/* TODO(sreek) Remove this file */ + + +//////////////////////////////////////////////////////////////////////////////// +// Definitions + +#define STATE_NOT_READY ((gpr_atm)0) +#define STATE_READY ((gpr_atm)1) + +typedef enum { POLLABLE_FD, POLLABLE_EPOLL_SET } pollable_type; + +typedef struct { + pollable_type type; + int fd; + grpc_iomgr_object iomgr_object; +} pollable_object; + +typedef struct polling_island { + pollable_object pollable; + gpr_mu mu; + int refs; + grpc_fd *only_fd; + struct polling_island *became; + struct polling_island *next; +} polling_island; + +struct grpc_fd { + pollable_object pollable; + + // each event atomic is a tri state: + // STATE_NOT_READY - no event received, nobody waiting for it either + // STATE_READY - event received, nobody waiting for it + // closure pointer - no event received, upper layer is waiting for it + gpr_atm on_readable; + gpr_atm on_writable; + + // mutex guarding set_ready & shutdown state + gpr_mu set_ready_mu; + bool shutdown; + + // mutex protecting polling_island + gpr_mu polling_island_mu; + // current polling island + polling_island *polling_island; + + grpc_fd *next_free; +}; + +struct grpc_pollset_worker {}; + +struct grpc_pollset { + gpr_mu mu; + // current polling island + polling_island *polling_island; +}; + +//////////////////////////////////////////////////////////////////////////////// +// Polling island implementation + +static gpr_mu g_pi_freelist_mu; +static polling_island *g_first_free_pi; + +static void add_pollable_to_epoll_set(pollable_object *pollable, int epoll_set, + uint32_t events) { + struct epoll_event ev; + ev.events = events; + ev.data.ptr = pollable; + int err = epoll_ctl(epoll_set, EPOLL_CTL_ADD, pollable->fd, &ev); + if (err < 0) { + gpr_log(GPR_ERROR, "epoll_ctl add for %d faild: %s", pollable->fd, + strerror(errno)); + } +} + +static void add_fd_to_epoll_set(grpc_fd *fd, int epoll_set) { + add_pollable_to_epoll_set(&fd->pollable, epoll_set, + EPOLLIN | EPOLLOUT | EPOLLET); +} + +static void add_island_to_epoll_set(polling_island *pi, int epoll_set) { + add_pollable_to_epoll_set(&pi->pollable, epoll_set, EPOLLIN | EPOLLET); +} + +static polling_island *polling_island_create(grpc_fd *initial_fd) { + polling_island *r = NULL; + gpr_mu_lock(&g_pi_freelist_mu); + if (g_first_free_pi == NULL) { + r = gpr_malloc(sizeof(*r)); + r->pollable.type = POLLABLE_EPOLL_SET; + gpr_mu_init(&r->mu); + } else { + r = g_first_free_pi; + g_first_free_pi = r->next; + } + gpr_mu_unlock(&g_pi_freelist_mu); + + r->pollable.fd = epoll_create1(EPOLL_CLOEXEC); + GPR_ASSERT(r->pollable.fd >= 0); + + gpr_mu_lock(&r->mu); + r->only_fd = initial_fd; + r->refs = 2; // creation of a polling island => a referencing pollset & fd + gpr_mu_unlock(&r->mu); + + add_fd_to_epoll_set(initial_fd, r->pollable.fd); + return r; +} + +static void polling_island_delete(polling_island *p) { + gpr_mu_lock(&g_pi_freelist_mu); + p->next = g_first_free_pi; + g_first_free_pi = p; + gpr_mu_unlock(&g_pi_freelist_mu); +} + +static polling_island *polling_island_add(polling_island *p, grpc_fd *fd) { + gpr_mu_lock(&p->mu); + p->only_fd = NULL; + p->refs++; // new fd picks up a ref + gpr_mu_unlock(&p->mu); + + add_fd_to_epoll_set(fd, p->pollable.fd); + + return p; +} + +static void add_siblings_to(polling_island *siblings, polling_island *dest) { + polling_island *sibling_tail = dest; + while (sibling_tail->next != NULL) { + sibling_tail = sibling_tail->next; + } + sibling_tail->next = siblings; +} + +static polling_island *polling_island_merge(polling_island *a, + polling_island *b) { + GPR_ASSERT(a != b); + polling_island *out; + + gpr_mu_lock(&GPR_MIN(a, b)->mu); + gpr_mu_lock(&GPR_MAX(a, b)->mu); + + GPR_ASSERT(a->became == NULL); + GPR_ASSERT(b->became == NULL); + + if (a->only_fd == NULL && b->only_fd == NULL) { + b->became = a; + add_siblings_to(b, a); + add_island_to_epoll_set(b, a->pollable.fd); + out = a; + } else if (a->only_fd == NULL) { + GPR_ASSERT(b->only_fd != NULL); + add_fd_to_epoll_set(b->only_fd, a->pollable.fd); + b->became = a; + out = a; + } else if (b->only_fd == NULL) { + GPR_ASSERT(a->only_fd != NULL); + add_fd_to_epoll_set(a->only_fd, b->pollable.fd); + a->became = b; + out = b; + } else { + add_fd_to_epoll_set(b->only_fd, a->pollable.fd); + a->only_fd = NULL; + b->only_fd = NULL; + b->became = a; + out = a; + } + + gpr_mu_unlock(&a->mu); + gpr_mu_unlock(&b->mu); + + return out; +} + +static polling_island *polling_island_update_and_lock(polling_island *p) { + gpr_mu_lock(&p->mu); + if (p->became != NULL) { + do { + polling_island *from = p; + p = p->became; + gpr_mu_lock(&p->mu); + bool delete_from = 0 == --from->refs; + p->refs++; + gpr_mu_unlock(&from->mu); + if (delete_from) { + polling_island_delete(from); + } + } while (p->became != NULL); + } + return p; +} + +static polling_island *polling_island_ref(polling_island *p) { + gpr_mu_lock(&p->mu); + gpr_mu_unlock(&p->mu); + return p; +} + +static void polling_island_drop(polling_island *p) {} + +static polling_island *polling_island_update(polling_island *p, + int updating_owner_count) { + p = polling_island_update_and_lock(p); + GPR_ASSERT(p->refs != 0); + p->refs += updating_owner_count; + gpr_mu_unlock(&p->mu); + return p; +} + +//////////////////////////////////////////////////////////////////////////////// +// FD implementation + +static gpr_mu g_fd_freelist_mu; +static grpc_fd *g_first_free_fd; + +static grpc_fd *fd_create(int fd, const char *name) { + grpc_fd *r = NULL; + gpr_mu_lock(&g_fd_freelist_mu); + if (g_first_free_fd == NULL) { + r = gpr_malloc(sizeof(*r)); + r->pollable.type = POLLABLE_FD; + gpr_atm_rel_store(&r->on_readable, 0); + gpr_atm_rel_store(&r->on_writable, 0); + gpr_mu_init(&r->polling_island_mu); + gpr_mu_init(&r->set_ready_mu); + } else { + r = g_first_free_fd; + g_first_free_fd = r->next_free; + } + gpr_mu_unlock(&g_fd_freelist_mu); + + r->pollable.fd = fd; + grpc_iomgr_register_object(&r->pollable.iomgr_object, name); + r->next_free = NULL; + return r; +} + +static int fd_wrapped_fd(grpc_fd *fd) { return fd->pollable.fd; } + +static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *on_done, int *release_fd, + const char *reason) { + if (release_fd != NULL) { + *release_fd = fd->pollable.fd; + } else { + close(fd->pollable.fd); + } + + gpr_mu_lock(&fd->polling_island_mu); + if (fd->polling_island != NULL) { + polling_island_drop(fd->polling_island); + } + gpr_mu_unlock(&fd->polling_island_mu); + + gpr_mu_lock(&g_fd_freelist_mu); + fd->next_free = g_first_free_fd; + g_first_free_fd = fd; + grpc_iomgr_unregister_object(&fd->pollable.iomgr_object); + gpr_mu_unlock(&g_fd_freelist_mu); + + grpc_exec_ctx_enqueue(exec_ctx, on_done, true, NULL); +} + +static void notify_on(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *closure, gpr_atm *state) { + if (gpr_atm_acq_cas(state, STATE_NOT_READY, (gpr_atm)closure)) { + // state was not ready, and is now the closure - we're done */ + } else { + // cas failed - we MUST be in STATE_READY (can't request two notifications + // for the same event) + // flip back to not ready, enqueue the closure directly + GPR_ASSERT(gpr_atm_rel_cas(state, STATE_READY, STATE_NOT_READY)); + grpc_exec_ctx_enqueue(exec_ctx, closure, true, NULL); + } +} + +static void fd_notify_on_read(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *closure) { + notify_on(exec_ctx, fd, closure, &fd->on_readable); +} + +static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *closure) { + notify_on(exec_ctx, fd, closure, &fd->on_readable); +} + +static void destroy_fd_freelist(void) { + while (g_first_free_fd) { + grpc_fd *next = g_first_free_fd->next_free; + gpr_mu_destroy(&g_first_free_fd->polling_island_mu); + gpr_free(next); + g_first_free_fd = next; + } +} + +static void set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + gpr_atm *state) { + if (gpr_atm_acq_cas(state, STATE_NOT_READY, STATE_READY)) { + // state was not ready, and is now ready - we're done + } else { + // cas failed - either there's a closure queued which we should consume OR + // the state was already STATE_READY + gpr_atm cur_state = gpr_atm_acq_load(state); + if (cur_state != STATE_READY) { + // state wasn't STATE_READY - it *must* have been a closure + // since it's illegal to ask for notification twice, it's safe to assume + // that we'll resume being the closure + GPR_ASSERT(gpr_atm_rel_cas(state, cur_state, STATE_NOT_READY)); + grpc_exec_ctx_enqueue(exec_ctx, (grpc_closure *)cur_state, !fd->shutdown, + NULL); + } + } +} + +static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + gpr_mu_lock(&fd->set_ready_mu); + GPR_ASSERT(!fd->shutdown); + fd->shutdown = 1; + set_ready_locked(exec_ctx, fd, &fd->on_readable); + set_ready_locked(exec_ctx, fd, &fd->on_writable); + gpr_mu_unlock(&fd->set_ready_mu); +} + +//////////////////////////////////////////////////////////////////////////////// +// Pollset implementation + +static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { + gpr_mu_init(&pollset->mu); + *mu = &pollset->mu; + pollset->polling_island = NULL; +} + +static void pollset_destroy(grpc_pollset *pollset) { + gpr_mu_destroy(&pollset->mu); + if (pollset->polling_island) { + polling_island_drop(pollset->polling_island); + } +} + +static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + struct grpc_fd *fd) { + gpr_mu_lock(&pollset->mu); + gpr_mu_lock(&fd->polling_island_mu); + + polling_island *new; + + if (fd->polling_island == NULL) { + if (pollset->polling_island == NULL) { + new = polling_island_create(fd); + } else { + new = polling_island_add(pollset->polling_island, fd); + } + } else if (pollset->polling_island == NULL) { + new = polling_island_ref(fd->polling_island); + } else if (pollset->polling_island != fd->polling_island) { + new = polling_island_merge(pollset->polling_island, fd->polling_island); + } else { + new = polling_island_update(pollset->polling_island, 1); + } + + fd->polling_island = pollset->polling_island = new; + + gpr_mu_unlock(&fd->polling_island_mu); + gpr_mu_unlock(&pollset->mu); +} + +//////////////////////////////////////////////////////////////////////////////// +// Engine binding + +static void shutdown_engine(void) { destroy_fd_freelist(); } + +static const grpc_event_engine_vtable vtable = { + .pollset_size = sizeof(grpc_pollset), + + .fd_create = fd_create, + .fd_wrapped_fd = fd_wrapped_fd, + .fd_orphan = fd_orphan, + .fd_shutdown = fd_shutdown, + .fd_notify_on_read = fd_notify_on_read, + .fd_notify_on_write = fd_notify_on_write, + + .pollset_init = pollset_init, + .pollset_shutdown = pollset_shutdown, + .pollset_reset = pollset_reset, + .pollset_destroy = pollset_destroy, + .pollset_work = pollset_work, + .pollset_kick = pollset_kick, + .pollset_add_fd = pollset_add_fd, + + .pollset_set_create = pollset_set_create, + .pollset_set_destroy = pollset_set_destroy, + .pollset_set_add_pollset = pollset_set_add_pollset, + .pollset_set_del_pollset = pollset_set_del_pollset, + .pollset_set_add_pollset_set = pollset_set_add_pollset_set, + .pollset_set_del_pollset_set = pollset_set_del_pollset_set, + .pollset_set_add_fd = pollset_set_add_fd, + .pollset_set_del_fd = pollset_set_del_fd, + + .kick_poller = kick_poller, + + .shutdown_engine = shutdown_engine, +}; + +static bool is_epoll_available(void) { + abort(); + return false; +} + +const grpc_event_engine_vtable *grpc_init_poll_posix(void) { + if (!is_epoll_available()) { + return NULL; + } + return &vtable; +} From d7d6eed78822ed8de14fdf9b39255365946cf8c4 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 31 May 2016 09:44:08 -0700 Subject: [PATCH 023/215] Correct typo --- src/core/lib/iomgr/ev_posix.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index 404ef2a64b9..96399ef8379 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -44,7 +44,7 @@ #include #include -#include "src/core/lib/iomgr/ev_epoll_linux.h" +#include "src/core/lib/iomgr/ev_epoll_posix.h" #include "src/core/lib/iomgr/ev_poll_posix.h" #include "src/core/lib/support/env.h" From 8c6c9067bc001a7cd94c2689570a6ec27affee82 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 31 May 2016 10:57:20 -0700 Subject: [PATCH 024/215] run epoll tests too --- tools/run_tests/run_tests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index ddaf96c345a..f3b191feab7 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -157,7 +157,7 @@ class CLanguage(object): 'windows': ['all'], 'mac': ['all'], 'posix': ['all'], - 'linux': ['poll'], + 'linux': ['poll', 'epoll'], } for target in binaries: polling_strategies = (POLLING_STRATEGIES[self.platform] From 5098f91159f2a5c0494688b8cfaff4debef5686f Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 31 May 2016 10:58:17 -0700 Subject: [PATCH 025/215] Rewrite all the pollset and fd functions in ev_epoll_linux.c --- src/core/lib/iomgr/ev_epoll_linux.c | 161 +++++++--------------------- 1 file changed, 38 insertions(+), 123 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 7793a952016..1201c10a7e0 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -67,10 +67,10 @@ struct polling_island; struct grpc_fd { int fd; /* refst format: - bit0: 1=active/0=orphaned - bit1-n: refcount - meaning that mostly we ref by two to avoid altering the orphaned bit, - and just unref by 1 when we're ready to flag the object as orphaned */ + bit 0 : 1=Active / 0=Orphaned + bits 1-n : refcount + - ref/unref by two to avoid altering the orphaned bit + - To orphan, unref by 1 */ gpr_atm refst; gpr_mu mu; @@ -84,12 +84,11 @@ struct grpc_fd { /* Mutex protecting the 'polling_island' field */ gpr_mu pi_mu; - /* The polling island to which this fd belongs to. An fd belongs to exactly - one polling island */ + /* The polling island to which this fd belongs to. + * An fd belongs to exactly one polling island */ struct polling_island *polling_island; struct grpc_fd *freelist_next; - grpc_closure *on_done_closure; grpc_iomgr_object iomgr_object; @@ -141,7 +140,6 @@ typedef struct polling_island { /* Polling islands that are no longer needed are kept in a freelist so that they can be reused. This field points to the next polling island in the - free list. Note that this is only used if the polling island is in the free list */ struct polling_island *next_free; } polling_island; @@ -185,7 +183,7 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, * TODO: sreek - Might have to unref the fds (assuming whether we add a ref to * the fd when adding it to the epollset) */ /* The caller is expected to hold pi->mu lock before calling this function */ -static void polling_island_clear_fds_locked(polling_island *pi) { +static void polling_island_remove_all_fds_locked(polling_island *pi) { int err; size_t i; @@ -392,7 +390,7 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { // Move all the fds from polling_island p to polling_island q polling_island_add_fds_locked(q, p->fds, p->fd_cnt); - polling_island_clear_fds_locked(p); + polling_island_remove_all_fds_locked(p); q->ref_cnt += p->ref_cnt; @@ -411,14 +409,7 @@ static void polling_island_global_init() { * pollset declarations */ -typedef struct grpc_cached_wakeup_fd { - grpc_wakeup_fd fd; - struct grpc_cached_wakeup_fd *next; -} grpc_cached_wakeup_fd; - struct grpc_pollset_worker { - grpc_cached_wakeup_fd *wakeup_fd; - int reevaluate_polling_on_wakeup; int kicked_specifically; pthread_t pt_id; struct grpc_pollset_worker *next; @@ -441,9 +432,6 @@ struct grpc_pollset { /* The polling island to which this fd belongs to. An fd belongs to exactly one polling island */ struct polling_island *polling_island; - - /* Local cache of eventfds for workers */ - grpc_cached_wakeup_fd *local_wakeup_cache; }; /* Add an fd to a pollset */ @@ -465,8 +453,6 @@ static int poll_deadline_to_millis_timeout(gpr_timespec deadline, /* Allow kick to wakeup the currently polling worker */ #define GRPC_POLLSET_CAN_KICK_SELF 1 -/* Force the wakee to repoll when awoken */ -#define GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP 2 /* As per pollset_kick, with an extended set of flags (defined above) -- mostly for fd_posix's use. */ static void pollset_kick_ext(grpc_pollset *p, @@ -815,34 +801,25 @@ static void pollset_kick_ext(grpc_pollset *p, if (specific_worker != NULL) { if (specific_worker == GRPC_POLLSET_KICK_BROADCAST) { GPR_TIMER_BEGIN("pollset_kick_ext.broadcast", 0); - GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); for (specific_worker = p->root_worker.next; specific_worker != &p->root_worker; specific_worker = specific_worker->next) { - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + pthread_kill(specific_worker->pt_id, SIGUSR1); } p->kicked_without_pollers = 1; GPR_TIMER_END("pollset_kick_ext.broadcast", 0); } else if (gpr_tls_get(&g_current_thread_worker) != (intptr_t)specific_worker) { GPR_TIMER_MARK("different_thread_worker", 0); - if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { - specific_worker->reevaluate_polling_on_wakeup = 1; - } specific_worker->kicked_specifically = 1; - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); /* TODO (sreek): Refactor this into a separate file*/ pthread_kill(specific_worker->pt_id, SIGUSR1); } else if ((flags & GRPC_POLLSET_CAN_KICK_SELF) != 0) { GPR_TIMER_MARK("kick_yoself", 0); - if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { - specific_worker->reevaluate_polling_on_wakeup = 1; - } specific_worker->kicked_specifically = 1; - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + pthread_kill(specific_worker->pt_id, SIGUSR1); } } else if (gpr_tls_get(&g_current_thread_poller) != (intptr_t)p) { - GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); GPR_TIMER_MARK("kick_anonymous", 0); specific_worker = pop_front_worker(p); if (specific_worker != NULL) { @@ -860,7 +837,7 @@ static void pollset_kick_ext(grpc_pollset *p, if (specific_worker != NULL) { GPR_TIMER_MARK("finally_kick", 0); push_back_worker(p, specific_worker); - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + pthread_kill(specific_worker->pt_id, SIGUSR1); } } else { GPR_TIMER_MARK("kicked_no_pollers", 0); @@ -911,8 +888,6 @@ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { pollset->shutting_down = 0; pollset->called_shutdown = 0; pollset->kicked_without_pollers = 0; - pollset->local_wakeup_cache = NULL; - pollset->kicked_without_pollers = 0; multipoll_with_epoll_pollset_create_efd(pollset); } @@ -926,12 +901,6 @@ static void pollset_destroy(grpc_pollset *pollset) { multipoll_with_epoll_pollset_destroy(pollset); - while (pollset->local_wakeup_cache) { - grpc_cached_wakeup_fd *next = pollset->local_wakeup_cache->next; - grpc_wakeup_fd_destroy(&pollset->local_wakeup_cache->fd); - gpr_free(pollset->local_wakeup_cache); - pollset->local_wakeup_cache = next; - } gpr_mu_destroy(&pollset->pi_mu); gpr_mu_destroy(&pollset->mu); } @@ -974,14 +943,6 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, GPR_TIMER_BEGIN("pollset_work", 0); /* this must happen before we (potentially) drop pollset->mu */ worker.next = worker.prev = NULL; - worker.reevaluate_polling_on_wakeup = 0; - if (pollset->local_wakeup_cache != NULL) { - worker.wakeup_fd = pollset->local_wakeup_cache; - pollset->local_wakeup_cache = worker.wakeup_fd->next; - } else { - worker.wakeup_fd = gpr_malloc(sizeof(*worker.wakeup_fd)); - grpc_wakeup_fd_init(&worker.wakeup_fd->fd); - } worker.kicked_specifically = 0; /* TODO(sreek): Abstract this thread id stuff out into a separate file */ @@ -1026,27 +987,12 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, gpr_mu_lock(&pollset->mu); locked = 1; } - /* If we're forced to re-evaluate polling (via pollset_kick with - GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) then we land here and force - a loop */ - if (worker.reevaluate_polling_on_wakeup) { - worker.reevaluate_polling_on_wakeup = 0; - pollset->kicked_without_pollers = 0; - if (queued_work || worker.kicked_specifically) { - /* If there's queued work on the list, then set the deadline to be - immediate so we get back out of the polling loop quickly */ - deadline = gpr_inf_past(GPR_CLOCK_MONOTONIC); - } - keep_polling = 1; - } } if (added_worker) { remove_worker(pollset, &worker); gpr_tls_set(&g_current_thread_worker, 0); } - /* release wakeup fd to the local pool */ - worker.wakeup_fd->next = pollset->local_wakeup_cache; - pollset->local_wakeup_cache = worker.wakeup_fd; + /* check shutdown conditions */ if (pollset->shutting_down) { if (pollset_has_workers(pollset)) { @@ -1135,10 +1081,9 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } } else if (fd->polling_island == NULL) { pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); - } else if (pollset->polling_island == NULL) { pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); - } else { // Non null and different + } else { pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); } @@ -1182,9 +1127,7 @@ static void multipoll_with_epoll_pollset_maybe_work_and_unlock( struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; int epoll_fd = pollset->epoll_fd; int ep_rv; - int poll_rv; int timeout_ms; - struct pollfd pfds[2]; /* If you want to ignore epoll's ability to sanely handle parallel pollers, * for a more apples-to-apples performance comparison with poll, add a @@ -1196,63 +1139,35 @@ static void multipoll_with_epoll_pollset_maybe_work_and_unlock( timeout_ms = poll_deadline_to_millis_timeout(deadline, now); - pfds[0].fd = GRPC_WAKEUP_FD_GET_READ_FD(&worker->wakeup_fd->fd); - pfds[0].events = POLLIN; - pfds[0].revents = 0; - pfds[1].fd = epoll_fd; - pfds[1].events = POLLIN; - pfds[1].revents = 0; - - /* TODO(vpai): Consider first doing a 0 timeout poll here to avoid - even going into the blocking annotation if possible */ - GPR_TIMER_BEGIN("poll", 0); - GRPC_SCHEDULING_START_BLOCKING_REGION; - poll_rv = grpc_poll_function(pfds, 2, timeout_ms); - GRPC_SCHEDULING_END_BLOCKING_REGION; - GPR_TIMER_END("poll", 0); - - if (poll_rv < 0) { - if (errno != EINTR) { - gpr_log(GPR_ERROR, "poll() failed: %s", strerror(errno)); - } - } else if (poll_rv == 0) { - /* do nothing */ - } else { - if (pfds[0].revents) { - grpc_wakeup_fd_consume_wakeup(&worker->wakeup_fd->fd); - } - if (pfds[1].revents) { - do { - /* The following epoll_wait never blocks; it has a timeout of 0 */ - ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); - if (ep_rv < 0) { - if (errno != EINTR) { - gpr_log(GPR_ERROR, "epoll_wait() failed: %s", strerror(errno)); - } + do { + /* The following epoll_wait never blocks; it has a timeout of 0 */ + ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms); + if (ep_rv < 0) { + if (errno != EINTR) { + gpr_log(GPR_ERROR, "epoll_wait() failed: %s", strerror(errno)); + } + } else { + int i; + for (i = 0; i < ep_rv; ++i) { + grpc_fd *fd = ep_ev[i].data.ptr; + /* TODO(klempner): We might want to consider making err and pri + * separate events */ + int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + int write_ev = ep_ev[i].events & EPOLLOUT; + if (fd == NULL) { + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); } else { - int i; - for (i = 0; i < ep_rv; ++i) { - grpc_fd *fd = ep_ev[i].data.ptr; - /* TODO(klempner): We might want to consider making err and pri - * separate events */ - int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); - int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); - int write_ev = ep_ev[i].events & EPOLLOUT; - if (fd == NULL) { - grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); - } else { - if (read_ev || cancel) { - fd_become_readable(exec_ctx, fd); - } - if (write_ev || cancel) { - fd_become_writable(exec_ctx, fd); - } - } + if (read_ev || cancel) { + fd_become_readable(exec_ctx, fd); + } + if (write_ev || cancel) { + fd_become_writable(exec_ctx, fd); } } - } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); + } } - } + } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); } static void multipoll_with_epoll_pollset_finish_shutdown( From 894900734662b7012d7b858db716c3cc1e9f8179 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 1 Jun 2016 08:52:43 -0700 Subject: [PATCH 026/215] Add a epoll_test.c file to experiment. REMOVE this from the final commit --- Makefile | 36 +++ build.yaml | 12 + test/core/network_benchmarks/epoll_test.c | 263 ++++++++++++++++++ .../network_benchmarks/low_level_ping_pong.c | 2 + tools/run_tests/sources_and_headers.json | 16 ++ tools/run_tests/tests.json | 15 + 6 files changed, 344 insertions(+) create mode 100644 test/core/network_benchmarks/epoll_test.c diff --git a/Makefile b/Makefile index 063698d943f..235f32d9a30 100644 --- a/Makefile +++ b/Makefile @@ -903,6 +903,7 @@ dns_resolver_connectivity_test: $(BINDIR)/$(CONFIG)/dns_resolver_connectivity_te dns_resolver_test: $(BINDIR)/$(CONFIG)/dns_resolver_test dualstack_socket_test: $(BINDIR)/$(CONFIG)/dualstack_socket_test endpoint_pair_test: $(BINDIR)/$(CONFIG)/endpoint_pair_test +epoll_test: $(BINDIR)/$(CONFIG)/epoll_test fd_conservation_posix_test: $(BINDIR)/$(CONFIG)/fd_conservation_posix_test fd_posix_test: $(BINDIR)/$(CONFIG)/fd_posix_test fling_client: $(BINDIR)/$(CONFIG)/fling_client @@ -1234,6 +1235,7 @@ buildtests_c: privatelibs_c \ $(BINDIR)/$(CONFIG)/dns_resolver_test \ $(BINDIR)/$(CONFIG)/dualstack_socket_test \ $(BINDIR)/$(CONFIG)/endpoint_pair_test \ + $(BINDIR)/$(CONFIG)/epoll_test \ $(BINDIR)/$(CONFIG)/fd_conservation_posix_test \ $(BINDIR)/$(CONFIG)/fd_posix_test \ $(BINDIR)/$(CONFIG)/fling_client \ @@ -1497,6 +1499,8 @@ test_c: buildtests_c $(Q) $(BINDIR)/$(CONFIG)/dualstack_socket_test || ( echo test dualstack_socket_test failed ; exit 1 ) $(E) "[RUN] Testing endpoint_pair_test" $(Q) $(BINDIR)/$(CONFIG)/endpoint_pair_test || ( echo test endpoint_pair_test failed ; exit 1 ) + $(E) "[RUN] Testing epoll_test" + $(Q) $(BINDIR)/$(CONFIG)/epoll_test || ( echo test epoll_test failed ; exit 1 ) $(E) "[RUN] Testing fd_conservation_posix_test" $(Q) $(BINDIR)/$(CONFIG)/fd_conservation_posix_test || ( echo test fd_conservation_posix_test failed ; exit 1 ) $(E) "[RUN] Testing fd_posix_test" @@ -6577,6 +6581,38 @@ endif endif +EPOLL_TEST_SRC = \ + test/core/network_benchmarks/epoll_test.c \ + +EPOLL_TEST_OBJS = $(addprefix $(OBJDIR)/$(CONFIG)/, $(addsuffix .o, $(basename $(EPOLL_TEST_SRC)))) +ifeq ($(NO_SECURE),true) + +# You can't build secure targets if you don't have OpenSSL. + +$(BINDIR)/$(CONFIG)/epoll_test: openssl_dep_error + +else + + + +$(BINDIR)/$(CONFIG)/epoll_test: $(EPOLL_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a + $(E) "[LD] Linking $@" + $(Q) mkdir -p `dirname $@` + $(Q) $(LD) $(LDFLAGS) $(EPOLL_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a $(LDLIBS) $(LDLIBS_SECURE) -o $(BINDIR)/$(CONFIG)/epoll_test + +endif + +$(OBJDIR)/$(CONFIG)/test/core/network_benchmarks/epoll_test.o: $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a + +deps_epoll_test: $(EPOLL_TEST_OBJS:.o=.dep) + +ifneq ($(NO_SECURE),true) +ifneq ($(NO_DEPS),true) +-include $(EPOLL_TEST_OBJS:.o=.dep) +endif +endif + + FD_CONSERVATION_POSIX_TEST_SRC = \ test/core/iomgr/fd_conservation_posix_test.c \ diff --git a/build.yaml b/build.yaml index 2f3d07071da..db9787546ad 100644 --- a/build.yaml +++ b/build.yaml @@ -1321,6 +1321,18 @@ targets: - grpc - gpr_test_util - gpr +- name: epoll_test + build: test + language: c + src: + - test/core/network_benchmarks/epoll_test.c + deps: + - grpc_test_util + - grpc + - gpr_test_util + - gpr + platforms: + - linux - name: fd_conservation_posix_test build: test language: c diff --git a/test/core/network_benchmarks/epoll_test.c b/test/core/network_benchmarks/epoll_test.c new file mode 100644 index 00000000000..a918dd9bb94 --- /dev/null +++ b/test/core/network_benchmarks/epoll_test.c @@ -0,0 +1,263 @@ +/* + * + * Copyright 2015, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +/* TODO: sreek: REMOVE THIS FILE */ + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +int g_signal_num = SIGUSR1; + +int g_timeout_secs = 2; + +int g_eventfd_create = 1; +int g_eventfd_wakeup = 0; +int g_eventfd_teardown = 0; +int g_close_epoll_fd = 1; + +typedef struct thread_args { + gpr_thd_id id; + int epoll_fd; + int thread_num; +} thread_args; + +static int eventfd_create() { + if (!g_eventfd_create) { + return -1; + } + + int efd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + GPR_ASSERT(efd >= 0); + return efd; +} + +static void eventfd_wakeup(int efd) { + if (!g_eventfd_wakeup) { + return; + } + + int err; + do { + err = eventfd_write(efd, 1); + } while (err < 0 && errno == EINTR); +} + +static void epoll_teardown(int epoll_fd, int fd) { + if (!g_eventfd_teardown) { + return; + } + + if (epoll_ctl(epoll_fd, EPOLL_CTL_DEL, fd, NULL) < 0) { + if (errno != ENOENT) { + gpr_log(GPR_ERROR, "epoll_ctl: %s", strerror(errno)); + GPR_ASSERT(0); + } + } +} + +/* Special case for epoll, where we need to create the fd ahead of time. */ +static int epoll_setup(int fd) { + int epoll_fd; + struct epoll_event ev; + + epoll_fd = epoll_create(1); + if (epoll_fd < 0) { + gpr_log(GPR_ERROR, "epoll_create: %s", strerror(errno)); + return -1; + } + + ev.events = (uint32_t)EPOLLIN; + ev.data.fd = fd; + if (epoll_ctl(epoll_fd, EPOLL_CTL_ADD, fd, &ev) < 0) { + if (errno != EEXIST) { + gpr_log(GPR_ERROR, "epoll_ctl: %s", strerror(errno)); + return -1; + } + + gpr_log(GPR_ERROR, "epoll_ctl: The fd %d already exists", fd); + } + + return epoll_fd; +} + +#define GRPC_EPOLL_MAX_EVENTS 1000 +static void thread_main(void *args) { + int ep_rv; + struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; + int fd; + int i; + int cancel; + int read; + int write; + thread_args *thd_args = args; + sigset_t new_mask; + sigset_t orig_mask; + int keep_polling = 0; + + gpr_log(GPR_INFO, "Thread: %d Started", thd_args->thread_num); + + do { + keep_polling = 0; + + /* Mask the signal before getting the epoll_fd */ + gpr_log(GPR_INFO, "Thread: %d Blocking signal: %d", thd_args->thread_num, + g_signal_num); + sigemptyset(&new_mask); + sigaddset(&new_mask, g_signal_num); + pthread_sigmask(SIG_BLOCK, &new_mask, &orig_mask); + + gpr_log(GPR_INFO, "Thread: %d Waiting on epoll_wait()", + thd_args->thread_num); + ep_rv = epoll_pwait(thd_args->epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, + g_timeout_secs * 5000, &orig_mask); + gpr_log(GPR_INFO, "Thread: %d out of epoll_wait. ep_rv = %d", + thd_args->thread_num, ep_rv); + + if (ep_rv < 0) { + if (errno != EINTR) { + gpr_log(GPR_ERROR, "Thread: %d. epoll_wait failed with error: %d", + thd_args->thread_num, errno); + } else { + gpr_log(GPR_INFO, + "Thread: %d. epoll_wait was interrupted. Polling again >>>>>>>", + thd_args->thread_num); + keep_polling = 1; + } + } else { + if (ep_rv == 0) { + gpr_log(GPR_INFO, + "Thread: %d - epoll_wait returned 0. Most likely a timeout. " + "Polling again", + thd_args->thread_num); + keep_polling = 1; + } + + for (i = 0; i < ep_rv; i++) { + fd = ep_ev[i].data.fd; + cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + read = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + write = ep_ev[i].events & EPOLLOUT; + gpr_log(GPR_INFO, + "Thread: %d. epoll_wait returned that fd: %d has event of " + "interest. read: %d, write: %d, cancel: %d", + thd_args->thread_num, fd, read, write, cancel); + } + } + } while (keep_polling); +} + +static void close_fd(int fd) { + if (!g_close_epoll_fd) { + return; + } + + gpr_log(GPR_INFO, "*** Closing fd : %d ****", fd); + close(fd); + gpr_log(GPR_INFO, "*** Closed fd : %d ****", fd); +} + + +static void sig_handler(int sig_num) { + gpr_log(GPR_INFO, "<<<<< Received signal %d", sig_num); +} + +static void set_signal_handler() { + gpr_log(GPR_INFO, "Setting signal handler"); + signal(g_signal_num, sig_handler); +} + +#define NUM_THREADS 2 +int main(int argc, char **argv) { + int efd; + int epoll_fd; + int i; + thread_args thd_args[NUM_THREADS]; + gpr_thd_options options = gpr_thd_options_default(); + + set_signal_handler(); + + gpr_log(GPR_INFO, "Starting.."); + efd = eventfd_create(); + gpr_log(GPR_INFO, "Created event fd: %d", efd); + epoll_fd = epoll_setup(efd); + gpr_log(GPR_INFO, "Created epoll_fd: %d", epoll_fd); + + gpr_thd_options_set_joinable(&options); + for (i = 0; i < NUM_THREADS; i++) { + thd_args[i].thread_num = i; + thd_args[i].epoll_fd = epoll_fd; + gpr_log(GPR_INFO, "Starting thread: %d", i); + gpr_thd_new(&thd_args[i].id, thread_main, &thd_args[i], &options); + } + + sleep((unsigned)g_timeout_secs * 2); + + /* Send signals first */ + for (i = 0; i < NUM_THREADS; i++) { + gpr_log(GPR_INFO, "Sending signal to thread: %d", thd_args->thread_num); + pthread_kill(thd_args[i].id, g_signal_num); + gpr_log(GPR_INFO, "Sent signal to thread: %d >>>>>> ", + thd_args->thread_num); + } + + sleep((unsigned)g_timeout_secs * 2); + + close_fd(epoll_fd); + + sleep((unsigned)g_timeout_secs * 2); + + eventfd_wakeup(efd); + epoll_teardown(epoll_fd, efd); + + for (i = 0; i < NUM_THREADS; i++) { + gpr_thd_join(thd_args[i].id); + gpr_log(GPR_INFO, "Thread: %d joined", i); + } + + return 0; +} diff --git a/test/core/network_benchmarks/low_level_ping_pong.c b/test/core/network_benchmarks/low_level_ping_pong.c index 1b40895a719..b72a07778e0 100644 --- a/test/core/network_benchmarks/low_level_ping_pong.c +++ b/test/core/network_benchmarks/low_level_ping_pong.c @@ -44,6 +44,7 @@ #include #include #include +#include #ifdef __linux__ #include #endif @@ -84,6 +85,7 @@ typedef struct thread_args { static int read_bytes(int fd, char *buf, size_t read_size, int spin) { size_t bytes_read = 0; ssize_t err; + do { err = read(fd, buf + bytes_read, read_size - bytes_read); if (err < 0) { diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index 97cc55db36e..85b71a8255a 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -301,6 +301,22 @@ "third_party": false, "type": "target" }, + { + "deps": [ + "gpr", + "gpr_test_util", + "grpc", + "grpc_test_util" + ], + "headers": [], + "language": "c", + "name": "epoll_test", + "src": [ + "test/core/network_benchmarks/epoll_test.c" + ], + "third_party": false, + "type": "target" + }, { "deps": [ "gpr", diff --git a/tools/run_tests/tests.json b/tools/run_tests/tests.json index 850f9474aec..be7b72f61d2 100644 --- a/tools/run_tests/tests.json +++ b/tools/run_tests/tests.json @@ -356,6 +356,21 @@ "windows" ] }, + { + "args": [], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "gtest": false, + "language": "c", + "name": "epoll_test", + "platforms": [ + "linux" + ] + }, { "args": [], "ci_platforms": [ From 9b9708a9c484c02ad8ef27060370ae605a83942c Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Wed, 1 Jun 2016 11:42:20 -0700 Subject: [PATCH 027/215] Allow Node users to set a custom logger and log verbosity. Defaults to existing core logger --- src/node/ext/node_grpc.cc | 117 ++++++++++++++++++++++++++++++++++++ src/node/index.js | 33 ++++++++++ src/node/src/common.js | 21 +++++++ src/node/src/credentials.js | 4 +- 4 files changed, 174 insertions(+), 1 deletion(-) diff --git a/src/node/ext/node_grpc.cc b/src/node/ext/node_grpc.cc index 6b6e42737b5..45762cad2e6 100644 --- a/src/node/ext/node_grpc.cc +++ b/src/node/ext/node_grpc.cc @@ -31,12 +31,15 @@ * */ +#include + #include #include #include #include "grpc/grpc.h" #include "grpc/grpc_security.h" #include "grpc/support/alloc.h" +#include "grpc/support/log.h" #include "call.h" #include "call_credentials.h" @@ -49,10 +52,22 @@ using v8::FunctionTemplate; using v8::Local; using v8::Value; +using v8::Number; using v8::Object; using v8::Uint32; using v8::String; +typedef struct logger_state { + Nan::Callback *callback; + std::list *pending_args; + uv_mutex_t mutex; + uv_async_t async; + // Indicates that a logger has been set + bool logger_set; +} logger_state; + +logger_state grpc_logger_state; + static char *pem_root_certs = NULL; void InitStatusConstants(Local exports) { @@ -235,6 +250,18 @@ void InitWriteFlags(Local exports) { Nan::Set(write_flags, Nan::New("NO_COMPRESS").ToLocalChecked(), NO_COMPRESS); } +void InitLogConstants(Local exports) { + Nan::HandleScope scope; + Local log_verbosity = Nan::New(); + Nan::Set(exports, Nan::New("logVerbosity").ToLocalChecked(), log_verbosity); + Local DEBUG(Nan::New(GPR_LOG_SEVERITY_DEBUG)); + Nan::Set(log_verbosity, Nan::New("DEBUG").ToLocalChecked(), DEBUG); + Local INFO(Nan::New(GPR_LOG_SEVERITY_INFO)); + Nan::Set(log_verbosity, Nan::New("INFO").ToLocalChecked(), INFO); + Local ERROR(Nan::New(GPR_LOG_SEVERITY_ERROR)); + Nan::Set(log_verbosity, Nan::New("ERROR").ToLocalChecked(), ERROR); +} + NAN_METHOD(MetadataKeyIsLegal) { if (!info[0]->IsString()) { return Nan::ThrowTypeError( @@ -298,16 +325,98 @@ NAN_METHOD(SetDefaultRootsPem) { } } +NAUV_WORK_CB(LogMessagesCallback) { + Nan::HandleScope scope; + std::list args; + uv_mutex_lock(&grpc_logger_state.mutex); + args.splice(args.begin(), *grpc_logger_state.pending_args); + uv_mutex_unlock(&grpc_logger_state.mutex); + /* Call the callback with each log message */ + while (!args.empty()) { + gpr_log_func_args *arg = args.front(); + args.pop_front(); + Local file = Nan::New(arg->file).ToLocalChecked(); + Local line = Nan::New(arg->line); + Local severity = Nan::New( + gpr_log_severity_string(arg->severity)).ToLocalChecked(); + Local message = Nan::New(arg->message).ToLocalChecked(); + const int argc = 4; + Local argv[argc] = {file, line, severity, message}; + grpc_logger_state.callback->Call(argc, argv); + delete[] arg->message; + delete arg; + } +} + +void node_log_func(gpr_log_func_args *args) { + // TODO(mlumish): Use the core's log formatter when it becomes available + gpr_log_func_args *args_copy = new gpr_log_func_args; + size_t message_len = strlen(args->message) + 1; + char *message = new char[message_len]; + memcpy(message, args->message, message_len); + memcpy(args_copy, args, sizeof(gpr_log_func_args)); + args_copy->message = message; + + uv_mutex_lock(&grpc_logger_state.mutex); + grpc_logger_state.pending_args->push_back(args_copy); + uv_mutex_unlock(&grpc_logger_state.mutex); + + uv_async_send(&grpc_logger_state.async); +} + +void init_logger() { + memset(&grpc_logger_state, 0, sizeof(logger_state)); + grpc_logger_state.pending_args = new std::list(); + uv_mutex_init(&grpc_logger_state.mutex); + uv_async_init(uv_default_loop(), + &grpc_logger_state.async, + LogMessagesCallback); + uv_unref((uv_handle_t*)&grpc_logger_state.async); + grpc_logger_state.logger_set = false; + + gpr_log_verbosity_init(); +} + +/* This registers a JavaScript logger for messages from the gRPC core. Because + that handler has to be run in the context of the JavaScript event loop, it + will be run asynchronously. To minimize the problems that could cause for + debugging, we leave core to do its default synchronous logging until a + JavaScript logger is set */ +NAN_METHOD(SetDefaultLoggerCallback) { + if (!info[0]->IsFunction()) { + return Nan::ThrowTypeError( + "setDefaultLoggerCallback's argument must be a function"); + } + if (!grpc_logger_state.logger_set) { + gpr_set_log_function(node_log_func); + grpc_logger_state.logger_set = true; + } + grpc_logger_state.callback = new Nan::Callback(info[0].As()); +} + +NAN_METHOD(SetLogVerbosity) { + if (!info[0]->IsUint32()) { + return Nan::ThrowTypeError( + "setLogVerbosity's argument must be a number"); + } + gpr_log_severity severity = static_cast( + Nan::To(info[0]).FromJust()); + gpr_set_log_verbosity(severity); +} + void init(Local exports) { Nan::HandleScope scope; grpc_init(); grpc_set_ssl_roots_override_callback(get_ssl_roots_override); + init_logger(); + InitStatusConstants(exports); InitCallErrorConstants(exports); InitOpTypeConstants(exports); InitPropagateConstants(exports); InitConnectivityStateConstants(exports); InitWriteFlags(exports); + InitLogConstants(exports); grpc::node::Call::Init(exports); grpc::node::CallCredentials::Init(exports); @@ -333,6 +442,14 @@ void init(Local exports) { Nan::GetFunction( Nan::New(SetDefaultRootsPem) ).ToLocalChecked()); + Nan::Set(exports, Nan::New("setDefaultLoggerCallback").ToLocalChecked(), + Nan::GetFunction( + Nan::New(SetDefaultLoggerCallback) + ).ToLocalChecked()); + Nan::Set(exports, Nan::New("setLogVerbosity").ToLocalChecked(), + Nan::GetFunction( + Nan::New(SetLogVerbosity) + ).ToLocalChecked()); } NODE_MODULE(grpc_node, init) diff --git a/src/node/index.js b/src/node/index.js index 66664d94b5a..b85cec68414 100644 --- a/src/node/index.js +++ b/src/node/index.js @@ -46,6 +46,8 @@ var client = require('./src/client.js'); var server = require('./src/server.js'); +var common = require('./src/common.js'); + var Metadata = require('./src/metadata.js'); var grpc = require('./src/grpc_extension'); @@ -122,6 +124,32 @@ exports.load = function load(filename, format, options) { return loadObject(builder.ns, options); }; +/** + * Sets the logger function for the gRPC module. For debugging purposes, the C + * core will log synchronously directly to stdout unless this function is + * called. Note: the output format here is intended to be informational, and + * is not guaranteed to stay the same in the future. + * Logs will be directed to logger.error. + * @param {Console} logger A Console-like object. + */ +exports.setLogger = function setLogger(logger) { + common.logger = logger; + grpc.setDefaultLoggerCallback(function(file, line, severity, message) { + file = path.basename(file); + logger.error(severity + '\t' + file + ':' + line + ']\t' + message); + }); +}; + +/** + * Sets the logger verbosity for gRPC module logging. The options are members + * of the grpc.logVerbosity map. + * @param {Number} verbosity The minimum severity to log + */ +exports.setLogVerbosity = function setLogVerbosity(verbosity) { + common.logVerbosity = verbosity; + grpc.setLogVerbosity(verbosity); +}; + /** * @see module:src/server.Server */ @@ -152,6 +180,11 @@ exports.callError = grpc.callError; */ exports.writeFlags = grpc.writeFlags; +/** + * Log verbosity setting name to code number mapping + */ +exports.logVerbosity = grpc.logVerbosity; + /** * Credentials factories */ diff --git a/src/node/src/common.js b/src/node/src/common.js index 8cf43b7a84c..e2d1a50df4a 100644 --- a/src/node/src/common.js +++ b/src/node/src/common.js @@ -157,3 +157,24 @@ exports.getProtobufServiceAttrs = function getProtobufServiceAttrs(service, }]; })); }; + +/** + * The logger object for the gRPC module. Defaults to console. + */ +exports.logger = console; + +/** + * The current logging verbosity. UNSET corresponds to logging everything + */ +exports.logVerbosity = -1; + +/** + * Log a message if the severity is at least as high as the current verbosity + * @param {Number} severity A value of the grpc.logVerbosity map + * @param {String} message The message to log + */ +exports.log = function log(severity, message) { + if (severity >= exports.logVerbosity) { + exports.logger.error(message); + } +}; diff --git a/src/node/src/credentials.js b/src/node/src/credentials.js index a12eade4e1f..b746d0625df 100644 --- a/src/node/src/credentials.js +++ b/src/node/src/credentials.js @@ -69,6 +69,8 @@ var ChannelCredentials = grpc.ChannelCredentials; var Metadata = require('./metadata.js'); +var common = require('./common.js'); + /** * Create an SSL Credentials object. If using a client-side certificate, both * the second and third arguments must be passed. @@ -120,7 +122,7 @@ exports.createFromGoogleCredential = function(google_credential) { var service_url = auth_context.service_url; google_credential.getRequestMetadata(service_url, function(err, header) { if (err) { - console.log('Auth error:', err); + common.log(grpc.logVerbosity.INFO, 'Auth error:' + err); callback(err); return; } From 4a43fe4a2fc62ab179e6fff7ca10f2c9c13dd1fc Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Wed, 1 Jun 2016 16:05:48 -0700 Subject: [PATCH 028/215] Minor change to common.js --- src/node/src/common.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/node/src/common.js b/src/node/src/common.js index e2d1a50df4a..22159dd39f7 100644 --- a/src/node/src/common.js +++ b/src/node/src/common.js @@ -164,9 +164,9 @@ exports.getProtobufServiceAttrs = function getProtobufServiceAttrs(service, exports.logger = console; /** - * The current logging verbosity. UNSET corresponds to logging everything + * The current logging verbosity. 0 corresponds to logging everything */ -exports.logVerbosity = -1; +exports.logVerbosity = 0; /** * Log a message if the severity is at least as high as the current verbosity From 1d2f28913e93481cc4b425c608dc7c59ebe026e4 Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Thu, 2 Jun 2016 14:33:22 -0700 Subject: [PATCH 029/215] Add timestamps to custom log output --- src/node/ext/node_grpc.cc | 38 ++++++++++++++++++++++++-------------- src/node/index.js | 16 +++++++++++++--- 2 files changed, 37 insertions(+), 17 deletions(-) diff --git a/src/node/ext/node_grpc.cc b/src/node/ext/node_grpc.cc index 45762cad2e6..6daa1039342 100644 --- a/src/node/ext/node_grpc.cc +++ b/src/node/ext/node_grpc.cc @@ -40,6 +40,7 @@ #include "grpc/grpc_security.h" #include "grpc/support/alloc.h" #include "grpc/support/log.h" +#include "grpc/support/time.h" #include "call.h" #include "call_credentials.h" @@ -48,6 +49,7 @@ #include "server.h" #include "completion_queue_async_worker.h" #include "server_credentials.h" +#include "timeval.h" using v8::FunctionTemplate; using v8::Local; @@ -57,9 +59,14 @@ using v8::Object; using v8::Uint32; using v8::String; +typedef struct log_args { + gpr_log_func_args core_args; + gpr_timespec timestamp; +} log_args; + typedef struct logger_state { Nan::Callback *callback; - std::list *pending_args; + std::list *pending_args; uv_mutex_t mutex; uv_async_t async; // Indicates that a logger has been set @@ -327,35 +334,38 @@ NAN_METHOD(SetDefaultRootsPem) { NAUV_WORK_CB(LogMessagesCallback) { Nan::HandleScope scope; - std::list args; + std::list args; uv_mutex_lock(&grpc_logger_state.mutex); args.splice(args.begin(), *grpc_logger_state.pending_args); uv_mutex_unlock(&grpc_logger_state.mutex); /* Call the callback with each log message */ while (!args.empty()) { - gpr_log_func_args *arg = args.front(); + log_args *arg = args.front(); args.pop_front(); - Local file = Nan::New(arg->file).ToLocalChecked(); - Local line = Nan::New(arg->line); + Local file = Nan::New(arg->core_args.file).ToLocalChecked(); + Local line = Nan::New(arg->core_args.line); Local severity = Nan::New( - gpr_log_severity_string(arg->severity)).ToLocalChecked(); - Local message = Nan::New(arg->message).ToLocalChecked(); - const int argc = 4; - Local argv[argc] = {file, line, severity, message}; + gpr_log_severity_string(arg->core_args.severity)).ToLocalChecked(); + Local message = Nan::New(arg->core_args.message).ToLocalChecked(); + Local timestamp = Nan::New( + grpc::node::TimespecToMilliseconds(arg->timestamp)).ToLocalChecked(); + const int argc = 5; + Local argv[argc] = {file, line, severity, message, timestamp}; grpc_logger_state.callback->Call(argc, argv); - delete[] arg->message; + delete[] arg->core_args.message; delete arg; } } void node_log_func(gpr_log_func_args *args) { // TODO(mlumish): Use the core's log formatter when it becomes available - gpr_log_func_args *args_copy = new gpr_log_func_args; + log_args *args_copy = new log_args; size_t message_len = strlen(args->message) + 1; char *message = new char[message_len]; memcpy(message, args->message, message_len); - memcpy(args_copy, args, sizeof(gpr_log_func_args)); - args_copy->message = message; + memcpy(&args_copy->core_args, args, sizeof(gpr_log_func_args)); + args_copy->core_args.message = message; + args_copy->timestamp = gpr_now(GPR_CLOCK_REALTIME); uv_mutex_lock(&grpc_logger_state.mutex); grpc_logger_state.pending_args->push_back(args_copy); @@ -366,7 +376,7 @@ void node_log_func(gpr_log_func_args *args) { void init_logger() { memset(&grpc_logger_state, 0, sizeof(logger_state)); - grpc_logger_state.pending_args = new std::list(); + grpc_logger_state.pending_args = new std::list(); uv_mutex_init(&grpc_logger_state.mutex); uv_async_init(uv_default_loop(), &grpc_logger_state.async, diff --git a/src/node/index.js b/src/node/index.js index b85cec68414..9fb6faa5d7c 100644 --- a/src/node/index.js +++ b/src/node/index.js @@ -124,6 +124,10 @@ exports.load = function load(filename, format, options) { return loadObject(builder.ns, options); }; +var log_template = _.template( + '{severity} {timestamp}\t{file}:{line}]\t{message}', + {interpolate: /{([\s\S]+?)}/g}); + /** * Sets the logger function for the gRPC module. For debugging purposes, the C * core will log synchronously directly to stdout unless this function is @@ -134,9 +138,15 @@ exports.load = function load(filename, format, options) { */ exports.setLogger = function setLogger(logger) { common.logger = logger; - grpc.setDefaultLoggerCallback(function(file, line, severity, message) { - file = path.basename(file); - logger.error(severity + '\t' + file + ':' + line + ']\t' + message); + grpc.setDefaultLoggerCallback(function(file, line, severity, + message, timestamp) { + logger.error(log_template({ + file: path.basename(file), + line: line, + severity: severity, + message: message, + timestamp: timestamp.toISOString() + })); }); }; From 0bcbd79baa57c16d4ab64a070b5fbfc93293f543 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 1 Jun 2016 15:43:03 -0700 Subject: [PATCH 030/215] Functionality complete in ev_epoll_linux.c --- src/core/lib/iomgr/ev_epoll_linux.c | 1202 +++++++++++++-------------- 1 file changed, 591 insertions(+), 611 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 1201c10a7e0..ce42a9e7ce6 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -61,31 +61,27 @@ struct polling_island; /******************************************************************************* - * FD declarations + * Fd Declarations */ - struct grpc_fd { int fd; /* refst format: bit 0 : 1=Active / 0=Orphaned bits 1-n : refcount - - ref/unref by two to avoid altering the orphaned bit - - To orphan, unref by 1 */ + Ref/Unref by two to avoid altering the orphaned bit */ gpr_atm refst; gpr_mu mu; - int shutdown; + bool shutdown; int closed; - int released; + bool released; grpc_closure *read_closure; grpc_closure *write_closure; - /* Mutex protecting the 'polling_island' field */ + /* The polling island to which this fd belongs to and the mutex protecting the + the field */ gpr_mu pi_mu; - - /* The polling island to which this fd belongs to. - * An fd belongs to exactly one polling island */ struct polling_island *polling_island; struct grpc_fd *freelist_next; @@ -94,11 +90,7 @@ struct grpc_fd { grpc_iomgr_object iomgr_object; }; -/* Return 1 if this fd is orphaned, 0 otherwise */ -static bool fd_is_orphaned(grpc_fd *fd); - /* Reference counting for fds */ -/*#define GRPC_FD_REF_COUNT_DEBUG*/ #ifdef GRPC_FD_REF_COUNT_DEBUG static void fd_ref(grpc_fd *fd, const char *reason, const char *file, int line); static void fd_unref(grpc_fd *fd, const char *reason, const char *file, @@ -119,15 +111,15 @@ static void fd_global_shutdown(void); #define CLOSURE_READY ((grpc_closure *)1) /******************************************************************************* - * Polling Island + * Polling-island Declarations */ typedef struct polling_island { gpr_mu mu; int ref_cnt; - /* Pointer to the polling_island this merged into. If this is not NULL, all - the remaining fields in this pollset (i.e all fields except mu and ref_cnt) - are considered invalid and must be ignored */ + /* Points to the polling_island this merged into. + * If merged_to is not NULL, all the remaining fields (except mu and ref_cnt) + * are invalid and must be ignored */ struct polling_island *merged_to; /* The fd of the underlying epoll set */ @@ -144,15 +136,62 @@ typedef struct polling_island { struct polling_island *next_free; } polling_island; +/******************************************************************************* + * Pollset Declarations + */ + +struct grpc_pollset_worker { + int kicked_specifically; + pthread_t pt_id; /* TODO (sreek) - Add an abstraction here */ + struct grpc_pollset_worker *next; + struct grpc_pollset_worker *prev; +}; + +struct grpc_pollset { + gpr_mu mu; + grpc_pollset_worker root_worker; + bool kicked_without_pollers; + + bool shutting_down; /* Is the pollset shutting down ? */ + bool finish_shutdown_called; /* Is the 'finish_shutdown_locked()' called ? */ + grpc_closure *shutdown_done; /* Called after after shutdown is complete */ + + /* The polling island to which this pollset belongs to and the mutex + protecting the field */ + gpr_mu pi_mu; + struct polling_island *polling_island; +}; + +/******************************************************************************* + * Pollset-set Declarations + */ +struct grpc_pollset_set { + gpr_mu mu; + + size_t pollset_count; + size_t pollset_capacity; + grpc_pollset **pollsets; + + size_t pollset_set_count; + size_t pollset_set_capacity; + struct grpc_pollset_set **pollset_sets; + + size_t fd_count; + size_t fd_capacity; + grpc_fd **fds; +}; + +/******************************************************************************* + * Polling-island Definitions + */ + /* Polling island freelist */ static gpr_mu g_pi_freelist_mu; static polling_island *g_pi_freelist = NULL; -/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? - * TODO: sreek - Should this add a ref to the grpc_fd ? */ /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, - size_t fd_count) { + size_t fd_count, bool add_fd_refs) { int err; size_t i; struct epoll_event ev; @@ -162,11 +201,14 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, ev.data.ptr = fds[i]; err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, fds[i]->fd, &ev); - if (err < 0 && errno != EEXIST) { - gpr_log(GPR_ERROR, "epoll_ctl add for fd: %d failed with error: %s", - fds[i]->fd, strerror(errno)); - /* TODO: sreek - Not sure if it is a good idea to continue here. We need a - * better way to bubble up this error instead of doing an abort() */ + if (err < 0) { + if (errno != EEXIST) { + /* TODO: sreek - We need a better way to bubble up this error instead of + just logging a message */ + gpr_log(GPR_ERROR, "epoll_ctl add for fd: %d failed with error: %s", + fds[i]->fd, strerror(errno)); + } + continue; } @@ -176,26 +218,30 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, } pi->fds[pi->fd_cnt++] = fds[i]; + if (add_fd_refs) { + GRPC_FD_REF(fds[i], "polling_island"); + } } } -/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? - * TODO: sreek - Might have to unref the fds (assuming whether we add a ref to - * the fd when adding it to the epollset) */ /* The caller is expected to hold pi->mu lock before calling this function */ -static void polling_island_remove_all_fds_locked(polling_island *pi) { +static void polling_island_remove_all_fds_locked(polling_island *pi, + bool remove_fd_refs) { int err; size_t i; for (i = 0; i < pi->fd_cnt; i++) { - err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, pi->fds[i]->fd, NULL); + if (remove_fd_refs) { + GRPC_FD_UNREF(pi->fds[i], "polling_island"); + } + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, pi->fds[i]->fd, NULL); if (err < 0 && errno != ENOENT) { gpr_log(GPR_ERROR, "epoll_ctl delete for fds[i]: %d failed with error: %s", i, pi->fds[i]->fd, strerror(errno)); - /* TODO: sreek - Not sure if it is a good idea to continue here. We need a - * better way to bubble up this error instead of doing an abort() */ + /* TODO: sreek - We need a better way to bubble up this error instead of + * just logging a message */ continue; } } @@ -203,22 +249,31 @@ static void polling_island_remove_all_fds_locked(polling_island *pi) { pi->fd_cnt = 0; } -/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? - * TODO: sreek - Might have to unref the fd (assuming whether we add a ref to - * the fd when adding it to the epollset) */ /* The caller is expected to hold pi->mu lock before calling this function */ -static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd) { +static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, + bool close_fd, bool remove_fd_ref) { int err; size_t i; - err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, fd->fd, NULL); - if (err < 0 && errno != ENOENT) { - gpr_log(GPR_ERROR, "epoll_ctl delete for fd: %d failed with error; %s", - fd->fd, strerror(errno)); + + /* Calling close() on the fd will automatically remove it from the epoll set. + If not calling close(), the fd must be explicitly removed from the epoll + set */ + if (close_fd) { + close(fd->fd); + } else { + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, fd->fd, NULL); + if (err < 0 && errno != ENOENT) { + gpr_log(GPR_ERROR, "epoll_ctl delete for fd: %d failed with error; %s", + fd->fd, strerror(errno)); + } } for (i = 0; i < pi->fd_cnt; i++) { if (pi->fds[i] == fd) { pi->fds[i] = pi->fds[--pi->fd_cnt]; + if (remove_fd_ref) { + GRPC_FD_UNREF(fd, "polling_island"); + } break; } } @@ -227,6 +282,10 @@ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd) { static polling_island *polling_island_create(grpc_fd *initial_fd, int initial_ref_cnt) { polling_island *pi = NULL; + struct epoll_event ev; + int err; + + /* Try to get one from the polling island freelist */ gpr_mu_lock(&g_pi_freelist_mu); if (g_pi_freelist != NULL) { pi = g_pi_freelist; @@ -242,13 +301,25 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, pi->fd_cnt = 0; pi->fd_capacity = 0; pi->fds = NULL; + } - pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); - if (pi->epoll_fd < 0) { - gpr_log(GPR_ERROR, "epoll_create1() failed with error: %s", - strerror(errno)); - } - GPR_ASSERT(pi->epoll_fd >= 0); + pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); + if (pi->epoll_fd < 0) { + gpr_log(GPR_ERROR, "epoll_create1() failed with error: %s", + strerror(errno)); + } + GPR_ASSERT(pi->epoll_fd >= 0); + + ev.events = (uint32_t)(EPOLLIN | EPOLLET); + ev.data.ptr = NULL; + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); + if (err < 0) { + gpr_log(GPR_ERROR, + "Failed to add grpc_global_wake_up_fd (%d) to the epoll set " + "(epoll_fd: %d) with error: %s", + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), pi->epoll_fd, + strerror(errno)); } pi->ref_cnt = initial_ref_cnt; @@ -256,10 +327,12 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, pi->next_free = NULL; if (initial_fd != NULL) { - /* polling_island_add_fds_locked() expects the caller to hold a pi->mu - * lock. However, since this is a new polling island (and no one has a - * reference to it yet), it is okay to not acquire pi->mu here */ - polling_island_add_fds_locked(pi, &initial_fd, 1); + /* It is not really needed to get the pi->mu lock here. If this is a newly + created polling island (or one that we got from the freelist), no one + else would be holding a lock to it anyway */ + gpr_mu_lock(&pi->mu); + polling_island_add_fds_locked(pi, &initial_fd, 1, true); + gpr_mu_unlock(&pi->mu); } return pi; @@ -269,6 +342,9 @@ static void polling_island_delete(polling_island *pi) { GPR_ASSERT(pi->ref_cnt == 0); GPR_ASSERT(pi->fd_cnt == 0); + close(pi->epoll_fd); + pi->epoll_fd = -1; + pi->merged_to = NULL; gpr_mu_lock(&g_pi_freelist_mu); @@ -313,10 +389,20 @@ void polling_island_pair_update_and_lock(polling_island **p, bool pi_2_locked = false; int num_swaps = 0; + /* Loop until either pi_1 == pi_2 or until we acquired locks on both pi_1 + and pi_2 */ while (pi_1 != pi_2 && !(pi_1_locked && pi_2_locked)) { - // pi_1 is NOT equal to pi_2 - // pi_1 MAY be locked - + /* The following assertions are true at this point: + - pi_1 != pi_2 (else, the while loop would have exited) + - pi_1 MAY be locked + - pi_2 is NOT locked */ + + /* To maintain lock order consistency, always lock polling_island node with + lower address first. + First, make sure pi_1 < pi_2 before proceeding any further. If it turns + out that pi_1 > pi_2, unlock pi_1 if locked (because pi_2 is not locked + at this point and having pi_1 locked would violate the lock order) and + swap pi_1 and pi_2 so that pi_1 becomes less than pi_2 */ if (pi_1 > pi_2) { if (pi_1_locked) { gpr_mu_unlock(&pi_1->mu); @@ -327,14 +413,22 @@ void polling_island_pair_update_and_lock(polling_island **p, num_swaps++; } - // p1 < p2 - // p1 MAY BE locked - // p2 is NOT locked + /* The following assertions are true at this point: + - pi_1 != pi_2 + - pi_1 < pi_2 (address of pi_1 is less than that of pi_2) + - pi_1 MAYBE locked + - pi_2 is NOT locked */ + /* Lock pi_1 (if pi_1 is pointing to the terminal node in the list) */ if (!pi_1_locked) { gpr_mu_lock(&pi_1->mu); pi_1_locked = true; + /* If pi_1 is not terminal node (i.e pi_1->merged_to != NULL), we are not + done locking this polling_island yet. Release the lock on this node and + advance pi_1 to the next node in the list; and go to the beginning of + the loop (we can't proceed to locking pi_2 unless we locked pi_1 first) + */ if (pi_1->merged_to != NULL) { temp = pi_1->merged_to; polling_island_unref_and_unlock(pi_1, 1); @@ -345,13 +439,16 @@ void polling_island_pair_update_and_lock(polling_island **p, } } - // p1 is LOCKED - // p2 is UNLOCKED - // p1 != p2 + /* The following assertions are true at this point: + - pi_1 is locked + - pi_2 is unlocked + - pi_1 != pi_2 */ gpr_mu_lock(&pi_2->mu); pi_2_locked = true; + /* If pi_2 is not terminal node, we are not done locking this polling_island + yet. Release the lock and update pi_2 to the next node in the list */ if (pi_2->merged_to != NULL) { temp = pi_2->merged_to; polling_island_unref_and_unlock(pi_2, 1); @@ -360,14 +457,19 @@ void polling_island_pair_update_and_lock(polling_island **p, } } - // Either pi_1 == pi_2 OR we got both locks! + /* At this point, either pi_1 == pi_2 AND/OR we got both locks */ if (pi_1 == pi_2) { + /* We may or may not have gotten the lock. If we didn't, walk the rest of + the polling_island list and get the lock */ GPR_ASSERT(pi_1_locked || (!pi_1_locked && !pi_2_locked)); if (!pi_1_locked) { pi_1 = pi_2 = polling_island_update_and_lock(pi_1, 2, 0); } } else { GPR_ASSERT(pi_1_locked && pi_2_locked); + /* If we swapped pi_1 and pi_2 odd number of times, do one more swap so that + pi_1 and pi_2 point to the same polling_island lists they started off + with at the beginning of this function (i.e *p and *q respectively) */ if (num_swaps % 2 > 0) { GPR_SWAP(polling_island *, pi_1, pi_2); } @@ -378,26 +480,37 @@ void polling_island_pair_update_and_lock(polling_island **p, } polling_island *polling_island_merge(polling_island *p, polling_island *q) { - polling_island *merged = NULL; - + /* Get locks on both the polling islands */ polling_island_pair_update_and_lock(&p, &q); /* TODO: sreek: Think about this scenario some more. Is it possible ?. what * does it mean, when would this happen */ if (p == q) { - merged = p; + /* Nothing needs to be done here */ + gpr_mu_unlock(&p->mu); + return p; } - // Move all the fds from polling_island p to polling_island q - polling_island_add_fds_locked(q, p->fds, p->fd_cnt); - polling_island_remove_all_fds_locked(p); + /* Make sure that p points to the polling island with fewer fds than q */ + if (p->fd_cnt > q->fd_cnt) { + GPR_SWAP(polling_island *, p, q); + } + + /* "Merge" p with q i.e move all the fds from p (the polling_island with fewer + fds) to q. + Note: Not altering the ref counts on the affected fds here because they + would effectively remain unchanged */ + polling_island_add_fds_locked(q, p->fds, p->fd_cnt, false); + polling_island_remove_all_fds_locked(p, false); + /* The merged polling island inherits all the ref counts of the island merging + with it */ q->ref_cnt += p->ref_cnt; gpr_mu_unlock(&p->mu); gpr_mu_unlock(&q->mu); - return merged; + return q; } static void polling_island_global_init() { @@ -406,95 +519,10 @@ static void polling_island_global_init() { } /******************************************************************************* - * pollset declarations - */ - -struct grpc_pollset_worker { - int kicked_specifically; - pthread_t pt_id; - struct grpc_pollset_worker *next; - struct grpc_pollset_worker *prev; -}; - -struct grpc_pollset { - gpr_mu mu; - grpc_pollset_worker root_worker; - int shutting_down; - int called_shutdown; - int kicked_without_pollers; - grpc_closure *shutdown_done; - - int epoll_fd; - - /* Mutex protecting the 'polling_island' field */ - gpr_mu pi_mu; - - /* The polling island to which this fd belongs to. An fd belongs to exactly - one polling island */ - struct polling_island *polling_island; -}; - -/* Add an fd to a pollset */ -static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - struct grpc_fd *fd); - -static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, grpc_fd *fd); - -/* Convert a timespec to milliseconds: - - very small or negative poll times are clamped to zero to do a - non-blocking poll (which becomes spin polling) - - other small values are rounded up to one millisecond - - longer than a millisecond polls are rounded up to the next nearest - millisecond to avoid spinning - - infinite timeouts are converted to -1 */ -static int poll_deadline_to_millis_timeout(gpr_timespec deadline, - gpr_timespec now); - -/* Allow kick to wakeup the currently polling worker */ -#define GRPC_POLLSET_CAN_KICK_SELF 1 -/* As per pollset_kick, with an extended set of flags (defined above) - -- mostly for fd_posix's use. */ -static void pollset_kick_ext(grpc_pollset *p, - grpc_pollset_worker *specific_worker, - uint32_t flags); - -/* turn a pollset into a multipoller: platform specific */ -typedef void (*platform_become_multipoller_type)(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, - struct grpc_fd **fds, - size_t fd_count); - -/* Return 1 if the pollset has active threads in pollset_work (pollset must - * be locked) */ -static int pollset_has_workers(grpc_pollset *pollset); - -/******************************************************************************* - * pollset_set definitions - */ - -struct grpc_pollset_set { - gpr_mu mu; - - size_t pollset_count; - size_t pollset_capacity; - grpc_pollset **pollsets; - - size_t pollset_set_count; - size_t pollset_set_capacity; - struct grpc_pollset_set **pollset_sets; - - size_t fd_count; - size_t fd_capacity; - grpc_fd **fds; -}; - -/******************************************************************************* - * fd_posix.c + * Fd Definitions */ -/* We need to keep a freelist not because of any concerns of malloc - * performance +/* We need to keep a freelist not because of any concerns of malloc performance * but instead so that implementations with multiple threads in (for example) * epoll_wait deal with the race between pollset removal and incoming poll * notifications. @@ -506,58 +534,16 @@ struct grpc_pollset_set { * If we keep the object freelisted, in the worst case losing this race just * becomes a spurious read notification on a reused fd. */ -/* TODO(klempner): We could use some form of polling generation count to know - * when these are safe to free. */ -/* TODO(klempner): Consider disabling freelisting if we don't have multiple - * threads in poll on the same fd */ -/* TODO(klempner): Batch these allocations to reduce fragmentation */ -static grpc_fd *fd_freelist = NULL; -static gpr_mu fd_freelist_mu; - -static void freelist_fd(grpc_fd *fd) { - gpr_mu_lock(&fd_freelist_mu); - fd->freelist_next = fd_freelist; - fd_freelist = fd; - grpc_iomgr_unregister_object(&fd->iomgr_object); - gpr_mu_unlock(&fd_freelist_mu); -} - -static grpc_fd *alloc_fd(int fd) { - grpc_fd *r = NULL; - - gpr_mu_lock(&fd_freelist_mu); - if (fd_freelist != NULL) { - r = fd_freelist; - fd_freelist = fd_freelist->freelist_next; - } - gpr_mu_unlock(&fd_freelist_mu); - - if (r == NULL) { - r = gpr_malloc(sizeof(grpc_fd)); - gpr_mu_init(&r->mu); - gpr_mu_init(&r->pi_mu); - } - /* TODO: sreek - check with ctiller on why we need to acquire a lock here */ - gpr_mu_lock(&r->mu); - gpr_atm_rel_store(&r->refst, 1); - r->shutdown = 0; - r->read_closure = CLOSURE_NOT_READY; - r->write_closure = CLOSURE_NOT_READY; - r->fd = fd; - r->polling_island = NULL; - r->freelist_next = NULL; - r->on_done_closure = NULL; - r->closed = 0; - r->released = 0; - gpr_mu_unlock(&r->mu); - return r; -} +/* The alarm system needs to be able to wakeup 'some poller' sometimes + * (specifically when a new alarm needs to be triggered earlier than the next + * alarm 'epoch'). This wakeup_fd gives us something to alert on when such a + * case occurs. */ +/* TODO: sreek: Right now, this wakes up all pollers */ +grpc_wakeup_fd grpc_global_wakeup_fd; -static void destroy(grpc_fd *fd) { - gpr_mu_destroy(&fd->mu); - gpr_free(fd); -} +static grpc_fd *fd_freelist = NULL; +static gpr_mu fd_freelist_mu; #ifdef GRPC_FD_REF_COUNT_DEBUG #define REF_BY(fd, n, reason) ref_by(fd, n, reason, __FILE__, __LINE__) @@ -588,12 +574,33 @@ static void unref_by(grpc_fd *fd, int n) { #endif old = gpr_atm_full_fetch_add(&fd->refst, -n); if (old == n) { - freelist_fd(fd); + /* Add the fd to the freelist */ + gpr_mu_lock(&fd_freelist_mu); + fd->freelist_next = fd_freelist; + fd_freelist = fd; + grpc_iomgr_unregister_object(&fd->iomgr_object); + gpr_mu_unlock(&fd_freelist_mu); } else { GPR_ASSERT(old > n); } } +/* Increment refcount by two to avoid changing the orphan bit */ +#ifdef GRPC_FD_REF_COUNT_DEBUG +static void fd_ref(grpc_fd *fd, const char *reason, const char *file, + int line) { + ref_by(fd, 2, reason, file, line); +} + +static void fd_unref(grpc_fd *fd, const char *reason, const char *file, + int line) { + unref_by(fd, 2, reason, file, line); +} +#else +static void fd_ref(grpc_fd *fd) { ref_by(fd, 2); } +static void fd_unref(grpc_fd *fd) { unref_by(fd, 2); } +#endif + static void fd_global_init(void) { gpr_mu_init(&fd_freelist_mu); } static void fd_global_shutdown(void) { @@ -602,91 +609,111 @@ static void fd_global_shutdown(void) { while (fd_freelist != NULL) { grpc_fd *fd = fd_freelist; fd_freelist = fd_freelist->freelist_next; - destroy(fd); + gpr_mu_destroy(&fd->mu); + gpr_free(fd); } gpr_mu_destroy(&fd_freelist_mu); } static grpc_fd *fd_create(int fd, const char *name) { - grpc_fd *r = alloc_fd(fd); + grpc_fd *new_fd = NULL; + + gpr_mu_lock(&fd_freelist_mu); + if (fd_freelist != NULL) { + new_fd = fd_freelist; + fd_freelist = fd_freelist->freelist_next; + } + gpr_mu_unlock(&fd_freelist_mu); + + if (new_fd == NULL) { + new_fd = gpr_malloc(sizeof(grpc_fd)); + gpr_mu_init(&new_fd->mu); + gpr_mu_init(&new_fd->pi_mu); + } - char *name2; - gpr_asprintf(&name2, "%s fd=%d", name, fd); - grpc_iomgr_register_object(&r->iomgr_object, name2); - gpr_free(name2); + /* Note: It is not really needed to get the new_fd->mu lock here. If this is a + newly created fd (or an fd we got from the freelist), no one else would be + holding a lock to it anyway. */ + gpr_mu_lock(&new_fd->mu); + + gpr_atm_rel_store(&new_fd->refst, 1); + new_fd->shutdown = false; + new_fd->read_closure = CLOSURE_NOT_READY; + new_fd->write_closure = CLOSURE_NOT_READY; + new_fd->fd = fd; + new_fd->polling_island = NULL; + new_fd->freelist_next = NULL; + new_fd->on_done_closure = NULL; + new_fd->closed = 0; + new_fd->released = false; + + gpr_mu_unlock(&new_fd->mu); + + char *fd_name; + gpr_asprintf(&fd_name, "%s fd=%d", name, fd); + grpc_iomgr_register_object(&new_fd->iomgr_object, fd_name); + gpr_free(fd_name); #ifdef GRPC_FD_REF_COUNT_DEBUG - gpr_log(GPR_DEBUG, "FD %d %p create %s", fd, r, name); + gpr_log(GPR_DEBUG, "FD %d %p create %s", fd, r, fd_name); #endif - return r; + return new_fd; } static bool fd_is_orphaned(grpc_fd *fd) { return (gpr_atm_acq_load(&fd->refst) & 1) == 0; } -static void close_fd_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - fd->closed = 1; - if (!fd->released) { - close(fd->fd); - } else { - /* TODO: sreek - Check for deadlocks */ - - gpr_mu_lock(&fd->pi_mu); - fd->polling_island = - polling_island_update_and_lock(fd->polling_island, 1, 0); - - polling_island_remove_fd_locked(fd->polling_island, fd); - polling_island_unref_and_unlock(fd->polling_island, 1); - - fd->polling_island = NULL; - gpr_mu_unlock(&fd->pi_mu); - } - - grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); -} - static int fd_wrapped_fd(grpc_fd *fd) { - if (fd->released || fd->closed) { - return -1; - } else { - return fd->fd; + int ret_fd = -1; + gpr_mu_lock(&fd->mu); + if (!fd->released && !fd->closed) { + ret_fd = fd->fd; } + gpr_mu_unlock(&fd->mu); + + return ret_fd; } -/* TODO: sreek - do something here with the pollset island link */ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure *on_done, int *release_fd, const char *reason) { + /* TODO(sreek) In ev_poll_posix.c,the lock is acquired a little later. Why? */ + gpr_mu_lock(&fd->mu); fd->on_done_closure = on_done; + + /* If release_fd is not NULL, we should be relinquishing control of the file + descriptor fd->fd (but we still own the grpc_fd structure). */ fd->released = release_fd != NULL; if (!fd->released) { shutdown(fd->fd, SHUT_RDWR); } else { *release_fd = fd->fd; } - gpr_mu_lock(&fd->mu); - REF_BY(fd, 1, reason); /* remove active status, but keep referenced */ - close_fd_locked(exec_ctx, fd); - gpr_mu_unlock(&fd->mu); - UNREF_BY(fd, 2, reason); /* drop the reference */ -} -/* increment refcount by two to avoid changing the orphan bit */ -#ifdef GRPC_FD_REF_COUNT_DEBUG -static void fd_ref(grpc_fd *fd, const char *reason, const char *file, - int line) { - ref_by(fd, 2, reason, file, line); -} + REF_BY(fd, 1, reason); /* Remove active status, but keep referenced */ + fd->closed = 1; -static void fd_unref(grpc_fd *fd, const char *reason, const char *file, - int line) { - unref_by(fd, 2, reason, file, line); -} -#else -static void fd_ref(grpc_fd *fd) { ref_by(fd, 2); } + /* Remove the fd from the polling island: + - Update the fd->polling_island to point to the latest polling island + - Remove the fd from the polling island. Also, call close() on the file + descriptor fd->fd ONLY if we haven't relinquised control (i.e + fd->released is 'false') + - Decrement the ref count on the polling island and det fd->polling_island + to NULL */ + gpr_mu_lock(&fd->pi_mu); -static void fd_unref(grpc_fd *fd) { unref_by(fd, 2); } -#endif + fd->polling_island = polling_island_update_and_lock(fd->polling_island, 1, 0); + polling_island_remove_fd_locked(fd->polling_island, fd, !fd->released, true); + polling_island_unref_and_unlock(fd->polling_island, 1); + fd->polling_island = NULL; + + gpr_mu_unlock(&fd->pi_mu); + + grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); + + gpr_mu_unlock(&fd->mu); + UNREF_BY(fd, 2, reason); /* Drop the reference */ +} static void notify_on_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st, grpc_closure *closure) { @@ -724,11 +751,13 @@ static int set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, } } -/* Do something here with the pollset island link (?) */ static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { gpr_mu_lock(&fd->mu); GPR_ASSERT(!fd->shutdown); - fd->shutdown = 1; + fd->shutdown = true; + + /* Flush any pending read and write closures. Since fd->shutdown is 'true' at + this point, the closures would be called with 'success = false' */ set_ready_locked(exec_ctx, fd, &fd->read_closure); set_ready_locked(exec_ctx, fd, &fd->write_closure); gpr_mu_unlock(&fd->mu); @@ -749,27 +778,39 @@ static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, } /******************************************************************************* - * pollset_posix.c + * Pollset Definitions */ -GPR_TLS_DECL(g_current_thread_poller); -GPR_TLS_DECL(g_current_thread_worker); +static void sig_handler(int sig_num) { + /* TODO: sreek - Remove this expensive log line */ + gpr_log(GPR_INFO, "Received signal %d", sig_num); +} -/** The alarm system needs to be able to wakeup 'some poller' sometimes - * (specifically when a new alarm needs to be triggered earlier than the next - * alarm 'epoch'). - * This wakeup_fd gives us something to alert on when such a case occurs. */ -grpc_wakeup_fd grpc_global_wakeup_fd; +/* Global state management */ +static void pollset_global_init(void) { + gpr_tls_init(&g_current_thread_poller); + gpr_tls_init(&g_current_thread_worker); + grpc_wakeup_fd_init(&grpc_global_wakeup_fd); + signal(SIGUSR1, sig_handler); +} -static void remove_worker(grpc_pollset *p, grpc_pollset_worker *worker) { - worker->prev->next = worker->next; - worker->next->prev = worker->prev; +static void pollset_global_shutdown(void) { + grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); + gpr_tls_destroy(&g_current_thread_poller); + gpr_tls_destroy(&g_current_thread_worker); } +/* Return 1 if the pollset has active threads in pollset_work (pollset must + * be locked) */ static int pollset_has_workers(grpc_pollset *p) { return p->root_worker.next != &p->root_worker; } +static void remove_worker(grpc_pollset *p, grpc_pollset_worker *worker) { + worker->prev->next = worker->next; + worker->next->prev = worker->prev; +} + static grpc_pollset_worker *pop_front_worker(grpc_pollset *p) { if (pollset_has_workers(p)) { grpc_pollset_worker *w = p->root_worker.next; @@ -792,241 +833,69 @@ static void push_front_worker(grpc_pollset *p, grpc_pollset_worker *worker) { worker->prev->next = worker->next->prev = worker; } -static void pollset_kick_ext(grpc_pollset *p, - grpc_pollset_worker *specific_worker, - uint32_t flags) { - GPR_TIMER_BEGIN("pollset_kick_ext", 0); - - /* pollset->mu already held */ - if (specific_worker != NULL) { - if (specific_worker == GRPC_POLLSET_KICK_BROADCAST) { - GPR_TIMER_BEGIN("pollset_kick_ext.broadcast", 0); - for (specific_worker = p->root_worker.next; - specific_worker != &p->root_worker; - specific_worker = specific_worker->next) { - pthread_kill(specific_worker->pt_id, SIGUSR1); +/* p->mu must be held before calling this function */ +static void pollset_kick(grpc_pollset *p, + grpc_pollset_worker *specific_worker) { + GPR_TIMER_BEGIN("pollset_kick", 0); + + grpc_pollset_worker *worker = specific_worker; + if (worker != NULL) { + if (worker == GRPC_POLLSET_KICK_BROADCAST) { + GPR_TIMER_BEGIN("pollset_kick.broadcast", 0); + if (pollset_has_workers(p)) { + for (worker = p->root_worker.next; worker != &p->root_worker; + worker = worker->next) { + pthread_kill(worker->pt_id, SIGUSR1); + } + } else { + p->kicked_without_pollers = true; } - p->kicked_without_pollers = 1; - GPR_TIMER_END("pollset_kick_ext.broadcast", 0); - } else if (gpr_tls_get(&g_current_thread_worker) != - (intptr_t)specific_worker) { - GPR_TIMER_MARK("different_thread_worker", 0); - specific_worker->kicked_specifically = 1; - /* TODO (sreek): Refactor this into a separate file*/ - pthread_kill(specific_worker->pt_id, SIGUSR1); - } else if ((flags & GRPC_POLLSET_CAN_KICK_SELF) != 0) { - GPR_TIMER_MARK("kick_yoself", 0); - specific_worker->kicked_specifically = 1; - pthread_kill(specific_worker->pt_id, SIGUSR1); + GPR_TIMER_END("pollset_kick.broadcast", 0); + } else { + GPR_TIMER_MARK("kicked_specifically", 0); + worker->kicked_specifically = true; + pthread_kill(worker->pt_id, SIGUSR1); } - } else if (gpr_tls_get(&g_current_thread_poller) != (intptr_t)p) { + } else { GPR_TIMER_MARK("kick_anonymous", 0); - specific_worker = pop_front_worker(p); - if (specific_worker != NULL) { - if (gpr_tls_get(&g_current_thread_worker) == (intptr_t)specific_worker) { - GPR_TIMER_MARK("kick_anonymous_not_self", 0); - push_back_worker(p, specific_worker); - specific_worker = pop_front_worker(p); - if ((flags & GRPC_POLLSET_CAN_KICK_SELF) == 0 && - gpr_tls_get(&g_current_thread_worker) == - (intptr_t)specific_worker) { - push_back_worker(p, specific_worker); - specific_worker = NULL; - } - } - if (specific_worker != NULL) { - GPR_TIMER_MARK("finally_kick", 0); - push_back_worker(p, specific_worker); - pthread_kill(specific_worker->pt_id, SIGUSR1); - } + worker = pop_front_worker(p); + if (worker != NULL) { + GPR_TIMER_MARK("finally_kick", 0); + push_back_worker(p, worker); + pthread_kill(worker->pt_id, SIGUSR1); } else { GPR_TIMER_MARK("kicked_no_pollers", 0); - p->kicked_without_pollers = 1; + p->kicked_without_pollers = true; } } - GPR_TIMER_END("pollset_kick_ext", 0); + GPR_TIMER_END("pollset_kick", 0); } -static void pollset_kick(grpc_pollset *p, - grpc_pollset_worker *specific_worker) { - pollset_kick_ext(p, specific_worker, 0); -} +static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } -/* global state management */ +static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { + gpr_mu_init(&pollset->mu); + *mu = &pollset->mu; -static void sig_handler(int sig_num) { - gpr_log(GPR_INFO, "Received signal %d", sig_num); -} + pollset->root_worker.next = pollset->root_worker.prev = &pollset->root_worker; + pollset->kicked_without_pollers = false; -static void pollset_global_init(void) { - gpr_tls_init(&g_current_thread_poller); - gpr_tls_init(&g_current_thread_worker); - grpc_wakeup_fd_init(&grpc_global_wakeup_fd); - signal(SIGUSR1, sig_handler); -} + pollset->shutting_down = false; + pollset->finish_shutdown_called = false; + pollset->shutdown_done = NULL; -static void pollset_global_shutdown(void) { - grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); - gpr_tls_destroy(&g_current_thread_poller); - gpr_tls_destroy(&g_current_thread_worker); -} - -static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } - -/* TODO: sreek. Try to Remove this forward declaration*/ -static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset); - -/* main interface */ - -static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { - gpr_mu_init(&pollset->mu); - *mu = &pollset->mu; - pollset->root_worker.next = pollset->root_worker.prev = &pollset->root_worker; gpr_mu_init(&pollset->pi_mu); pollset->polling_island = NULL; - pollset->shutting_down = 0; - pollset->called_shutdown = 0; - pollset->kicked_without_pollers = 0; - - multipoll_with_epoll_pollset_create_efd(pollset); -} - -/* TODO(sreek): Maybe merge multipoll_*_destroy() with pollset_destroy() - * function */ -static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset); - -static void pollset_destroy(grpc_pollset *pollset) { - GPR_ASSERT(!pollset_has_workers(pollset)); - - multipoll_with_epoll_pollset_destroy(pollset); - - gpr_mu_destroy(&pollset->pi_mu); - gpr_mu_destroy(&pollset->mu); -} - -/* TODO(sreek) - Do something with the pollset island link (??) */ -static void pollset_reset(grpc_pollset *pollset) { - GPR_ASSERT(pollset->shutting_down); - GPR_ASSERT(!pollset_has_workers(pollset)); - pollset->shutting_down = 0; - pollset->called_shutdown = 0; - pollset->kicked_without_pollers = 0; -} - -/* TODO (sreek): Remove multipoll_with_epoll_finish_shutdown() declaration */ -static void multipoll_with_epoll_pollset_finish_shutdown(grpc_pollset *pollset); - -static void finish_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { - multipoll_with_epoll_pollset_finish_shutdown(pollset); - grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); -} - -/* TODO(sreek): Remove multipoll_with_epoll_*_maybe_work_and_unlock - * declaration - */ -static void multipoll_with_epoll_pollset_maybe_work_and_unlock( - grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, - gpr_timespec deadline, gpr_timespec now); - -static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_pollset_worker **worker_hdl, gpr_timespec now, - gpr_timespec deadline) { - grpc_pollset_worker worker; - *worker_hdl = &worker; - - /* pollset->mu already held */ - int added_worker = 0; - int locked = 1; - int queued_work = 0; - int keep_polling = 0; - GPR_TIMER_BEGIN("pollset_work", 0); - /* this must happen before we (potentially) drop pollset->mu */ - worker.next = worker.prev = NULL; - worker.kicked_specifically = 0; - - /* TODO(sreek): Abstract this thread id stuff out into a separate file */ - worker.pt_id = pthread_self(); - /* If we're shutting down then we don't execute any extended work */ - if (pollset->shutting_down) { - GPR_TIMER_MARK("pollset_work.shutting_down", 0); - goto done; - } - /* Start polling, and keep doing so while we're being asked to - re-evaluate our pollers (this allows poll() based pollers to - ensure they don't miss wakeups) */ - keep_polling = 1; - while (keep_polling) { - keep_polling = 0; - if (!pollset->kicked_without_pollers) { - if (!added_worker) { - push_front_worker(pollset, &worker); - added_worker = 1; - gpr_tls_set(&g_current_thread_worker, (intptr_t)&worker); - } - gpr_tls_set(&g_current_thread_poller, (intptr_t)pollset); - GPR_TIMER_BEGIN("maybe_work_and_unlock", 0); - - multipoll_with_epoll_pollset_maybe_work_and_unlock( - exec_ctx, pollset, &worker, deadline, now); - - GPR_TIMER_END("maybe_work_and_unlock", 0); - locked = 0; - gpr_tls_set(&g_current_thread_poller, 0); - } else { - GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); - pollset->kicked_without_pollers = 0; - } - /* Finished execution - start cleaning up. - Note that we may arrive here from outside the enclosing while() loop. - In that case we won't loop though as we haven't added worker to the - worker list, which means nobody could ask us to re-evaluate polling). */ - done: - if (!locked) { - queued_work |= grpc_exec_ctx_flush(exec_ctx); - gpr_mu_lock(&pollset->mu); - locked = 1; - } - } - if (added_worker) { - remove_worker(pollset, &worker); - gpr_tls_set(&g_current_thread_worker, 0); - } - - /* check shutdown conditions */ - if (pollset->shutting_down) { - if (pollset_has_workers(pollset)) { - pollset_kick(pollset, NULL); - } else if (!pollset->called_shutdown) { - pollset->called_shutdown = 1; - gpr_mu_unlock(&pollset->mu); - finish_shutdown(exec_ctx, pollset); - grpc_exec_ctx_flush(exec_ctx); - /* Continuing to access pollset here is safe -- it is the caller's - * responsibility to not destroy when it has outstanding calls to - * pollset_work. - * TODO(dklempner): Can we refactor the shutdown logic to avoid this? */ - gpr_mu_lock(&pollset->mu); - } - } - *worker_hdl = NULL; - GPR_TIMER_END("pollset_work", 0); -} - -/* TODO: (sreek) Do something with the pollset island link */ -static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_closure *closure) { - GPR_ASSERT(!pollset->shutting_down); - pollset->shutting_down = 1; - pollset->shutdown_done = closure; - pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); - - if (!pollset->called_shutdown && !pollset_has_workers(pollset)) { - pollset->called_shutdown = 1; - finish_shutdown(exec_ctx, pollset); - } } +/* Convert a timespec to milliseconds: + - Very small or negative poll times are clamped to zero to do a non-blocking + poll (which becomes spin polling) + - Other small values are rounded up to one millisecond + - Longer than a millisecond polls are rounded up to the next nearest + millisecond to avoid spinning + - Infinite timeouts are converted to -1 */ static int poll_deadline_to_millis_timeout(gpr_timespec deadline, gpr_timespec now) { gpr_timespec timeout; @@ -1034,6 +903,7 @@ static int poll_deadline_to_millis_timeout(gpr_timespec deadline, if (gpr_time_cmp(deadline, gpr_inf_future(deadline.clock_type)) == 0) { return -1; } + if (gpr_time_cmp(deadline, gpr_time_add(now, gpr_time_from_micros( max_spin_polling_us, GPR_TIMESPAN))) <= 0) { @@ -1044,10 +914,6 @@ static int poll_deadline_to_millis_timeout(gpr_timespec deadline, timeout, gpr_time_from_nanos(GPR_NS_PER_MS - 1, GPR_TIMESPAN))); } -/******************************************************************************* - * pollset_multipoller_with_epoll_posix.c - */ - static void set_ready(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st) { /* only one set_ready can be active at once (but there may be a racing notify_on) */ @@ -1064,94 +930,46 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { set_ready(exec_ctx, fd, &fd->write_closure); } -/* TODO: sreek - This function multipoll_with_epoll_pollset_add_fd() and - * finally_add_fd() in ev_poll_and_epoll_posix.c */ -static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_fd *fd) { - /* TODO sreek - Check if we need to get a pollset->mu lock here */ - gpr_mu_lock(&pollset->pi_mu); - gpr_mu_lock(&fd->pi_mu); - - polling_island *pi_new = NULL; - - if (fd->polling_island == pollset->polling_island) { - pi_new = fd->polling_island; - if (pi_new == NULL) { - pi_new = polling_island_create(fd, 2); - } - } else if (fd->polling_island == NULL) { - pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); - } else if (pollset->polling_island == NULL) { - pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); - } else { - pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); - } - - fd->polling_island = pollset->polling_island = pi_new; - - gpr_mu_unlock(&fd->pi_mu); - gpr_mu_unlock(&pollset->pi_mu); -} - -/* Creates an epoll fd and initializes the pollset */ -/* TODO: This has to be called ONLY from pollset_init function. and hence it - * does not acquire any lock */ -static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { - struct epoll_event ev; - int err; - - pollset->epoll_fd = epoll_create1(EPOLL_CLOEXEC); - if (pollset->epoll_fd < 0) { - gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); - abort(); - } - - ev.events = (uint32_t)(EPOLLIN | EPOLLET); - ev.data.ptr = NULL; - - err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); - if (err < 0) { - gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), - strerror(errno)); - } -} - /* TODO(klempner): We probably want to turn this down a bit */ #define GRPC_EPOLL_MAX_EVENTS 1000 - -static void multipoll_with_epoll_pollset_maybe_work_and_unlock( - grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, - gpr_timespec deadline, gpr_timespec now) { +static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, int timeout_ms, + sigset_t *sig_mask) { struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; - int epoll_fd = pollset->epoll_fd; + int epoll_fd; int ep_rv; - int timeout_ms; + GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); + + /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the + polling island pointed by pollset->polling_island. + Acquire the following locks: + - pollset->mu (which we already have) + - pollset->pi_mu + - pollset->polling_island->mu */ + gpr_mu_lock(&pollset->pi_mu); + pollset->polling_island = + polling_island_update_and_lock(pollset->polling_island, 1, 0); - /* If you want to ignore epoll's ability to sanely handle parallel pollers, - * for a more apples-to-apples performance comparison with poll, add a - * if (pollset->counter != 0) { return 0; } - * here. - */ + epoll_fd = pollset->polling_island->epoll_fd; + /* Release the locks */ + polling_island_unref_and_unlock(pollset->polling_island, 0); /* Keep the ref*/ + gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); - timeout_ms = poll_deadline_to_millis_timeout(deadline, now); - do { - /* The following epoll_wait never blocks; it has a timeout of 0 */ - ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms); + ep_rv = epoll_pwait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms, + sig_mask); + if (ep_rv < 0) { if (errno != EINTR) { - gpr_log(GPR_ERROR, "epoll_wait() failed: %s", strerror(errno)); + /* TODO (sreek) - Check for bad file descriptor error */ + gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); } } else { int i; for (i = 0; i < ep_rv; ++i) { grpc_fd *fd = ep_ev[i].data.ptr; - /* TODO(klempner): We might want to consider making err and pri - * separate events */ int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); int write_ev = ep_ev[i].events & EPOLLOUT; @@ -1168,17 +986,179 @@ static void multipoll_with_epoll_pollset_maybe_work_and_unlock( } } } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); + + GPR_TIMER_END("pollset_work_and_unlock", 0); +} + +/* Release the reference to pollset->polling_island and set it to NULL. + pollset->mu must be held */ +static void pollset_release_polling_island_locked(grpc_pollset *pollset) { + gpr_mu_lock(&pollset->pi_mu); + if (pollset->polling_island) { + pollset->polling_island = + polling_island_update_and_lock(pollset->polling_island, 1, 0); + polling_island_unref_and_unlock(pollset->polling_island, 1); + pollset->polling_island = NULL; + } + gpr_mu_unlock(&pollset->pi_mu); +} + +static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset) { + /* The pollset cannot have any workers if we are at this stage */ + GPR_ASSERT(!pollset_has_workers(pollset)); + + pollset->finish_shutdown_called = true; + pollset_release_polling_island_locked(pollset); + + grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); +} + +/* pollset->mu lock must be held by the caller before calling this */ +static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_closure *closure) { + GPR_TIMER_BEGIN("pollset_shutdown", 0); + GPR_ASSERT(!pollset->shutting_down); + pollset->shutting_down = true; + pollset->shutdown_done = closure; + pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); + + /* If the pollset has any workers, we cannot call finish_shutdown_locked() + because it would release the underlying polling island. In such a case, we + let the last worker call finish_shutdown_locked() from pollset_work() */ + if (!pollset_has_workers(pollset)) { + GPR_ASSERT(!pollset->finish_shutdown_called); + GPR_TIMER_MARK("pollset_shutdown.finish_shutdown_locked", 0); + finish_shutdown_locked(exec_ctx, pollset); + } + GPR_TIMER_END("pollset_shutdown", 0); } -static void multipoll_with_epoll_pollset_finish_shutdown( - grpc_pollset *pollset) {} +/* TODO(sreek) Is pollset_shutdown() guranteed to be called before this? */ +static void pollset_destroy(grpc_pollset *pollset) { + GPR_ASSERT(!pollset_has_workers(pollset)); + gpr_mu_destroy(&pollset->pi_mu); + gpr_mu_destroy(&pollset->mu); +} -static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset) { - close(pollset->epoll_fd); +static void pollset_reset(grpc_pollset *pollset) { + GPR_ASSERT(pollset->shutting_down); + GPR_ASSERT(!pollset_has_workers(pollset)); + pollset->shutting_down = false; + pollset->finish_shutdown_called = false; + pollset->kicked_without_pollers = false; + /* TODO(sreek) - Should pollset->shutdown closure be set to NULL here? */ + pollset_release_polling_island_locked(pollset); +} + +/* pollset->mu lock must be held by the caller before calling this. + The function pollset_work() may temporarily release the lock (pollset->mu) + during the course of its execution but it will always re-acquire the lock and + ensure that it is held by the time the function returns */ +static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_pollset_worker **worker_hdl, gpr_timespec now, + gpr_timespec deadline) { + GPR_TIMER_BEGIN("pollset_work", 0); + + int timeout_ms = poll_deadline_to_millis_timeout(deadline, now); + + sigset_t new_mask; + sigset_t orig_mask; + + grpc_pollset_worker worker; + worker.next = worker.prev = NULL; + worker.kicked_specifically = 0; + worker.pt_id = pthread_self(); + + *worker_hdl = &worker; + + if (pollset->kicked_without_pollers) { + /* If the pollset was kicked without pollers, pretend that the current + worker got the kick and skip polling. A kick indicates that there is some + work that needs attention like an event on the completion queue or an + alarm */ + GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); + pollset->kicked_without_pollers = 0; + } else if (!pollset->shutting_down) { + sigemptyset(&new_mask); + sigaddset(&new_mask, SIGUSR1); + pthread_sigmask(SIG_BLOCK, &new_mask, &orig_mask); + sigdelset(&orig_mask, SIGUSR1); + + push_front_worker(pollset, &worker); + + pollset_work_and_unlock(exec_ctx, pollset, timeout_ms, &orig_mask); + grpc_exec_ctx_flush(exec_ctx); + + gpr_mu_lock(&pollset->mu); + remove_worker(pollset, &worker); + } + + /* If we are the last worker on the pollset (i.e pollset_has_workers() is + false at this point) and the pollset is shutting down, we may have to + finish the shutdown process by calling finish_shutdown_locked(). + See pollset_shutdown() for more details. + + Note: Continuing to access pollset here is safe; it is the caller's + responsibility to not destroy a pollset when it has outstanding calls to + pollset_work() */ + if (pollset->shutting_down && !pollset_has_workers(pollset) && + !pollset->finish_shutdown_called) { + GPR_TIMER_MARK("pollset_work.finish_shutdown_locked", 0); + finish_shutdown_locked(exec_ctx, pollset); + + gpr_mu_unlock(&pollset->mu); + grpc_exec_ctx_flush(exec_ctx); + gpr_mu_lock(&pollset->mu); + } + + *worker_hdl = NULL; + GPR_TIMER_END("pollset_work", 0); +} + +static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_fd *fd) { + /* TODO sreek - Check if we need to get a pollset->mu lock here */ + gpr_mu_lock(&pollset->pi_mu); + gpr_mu_lock(&fd->pi_mu); + + polling_island *pi_new = NULL; + + /* 1) If fd->polling_island and pollset->polling_island are both non-NULL and + * equal, do nothing. + * 2) If fd->polling_island and pollset->polling_island are both NULL, create + * a new polling island (with a refcount of 2) and make the polling_island + * fields in both fd and pollset to point to the new island + * 3) If one of fd->polling_island or pollset->polling_island is NULL, update + * the NULL polling_island field to point to the non-NULL polling_island + * field (ensure that the refcount on the polling island is incremented by + * 1 to account for the newly added reference) + * 4) Finally, if fd->polling_island and pollset->polling_island are non-NULL + * and different, merge both the polling islands and update the + * polling_island fields in both fd and pollset to point to the merged + * polling island. + */ + if (fd->polling_island == pollset->polling_island) { + pi_new = fd->polling_island; + if (pi_new == NULL) { + pi_new = polling_island_create(fd, 2); + } + } else if (fd->polling_island == NULL) { + pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); + } else if (pollset->polling_island == NULL) { + pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); + } else { + pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); + } + + fd->polling_island = pollset->polling_island = pi_new; + + gpr_mu_unlock(&fd->pi_mu); + gpr_mu_unlock(&pollset->pi_mu); } /******************************************************************************* - * pollset_set_posix.c + * Pollset-set Definitions */ static grpc_pollset_set *pollset_set_create(void) { @@ -1200,6 +1180,45 @@ static void pollset_set_destroy(grpc_pollset_set *pollset_set) { gpr_free(pollset_set); } +static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + if (pollset_set->fd_count == pollset_set->fd_capacity) { + pollset_set->fd_capacity = GPR_MAX(8, 2 * pollset_set->fd_capacity); + pollset_set->fds = gpr_realloc( + pollset_set->fds, pollset_set->fd_capacity * sizeof(*pollset_set->fds)); + } + GRPC_FD_REF(fd, "pollset_set"); + pollset_set->fds[pollset_set->fd_count++] = fd; + for (i = 0; i < pollset_set->pollset_count; i++) { + pollset_add_fd(exec_ctx, pollset_set->pollsets[i], fd); + } + for (i = 0; i < pollset_set->pollset_set_count; i++) { + pollset_set_add_fd(exec_ctx, pollset_set->pollset_sets[i], fd); + } + gpr_mu_unlock(&pollset_set->mu); +} + +static void pollset_set_del_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + for (i = 0; i < pollset_set->fd_count; i++) { + if (pollset_set->fds[i] == fd) { + pollset_set->fd_count--; + GPR_SWAP(grpc_fd *, pollset_set->fds[i], + pollset_set->fds[pollset_set->fd_count]); + GRPC_FD_UNREF(fd, "pollset_set"); + break; + } + } + for (i = 0; i < pollset_set->pollset_set_count; i++) { + pollset_set_del_fd(exec_ctx, pollset_set->pollset_sets[i], fd); + } + gpr_mu_unlock(&pollset_set->mu); +} + static void pollset_set_add_pollset(grpc_exec_ctx *exec_ctx, grpc_pollset_set *pollset_set, grpc_pollset *pollset) { @@ -1281,47 +1300,8 @@ static void pollset_set_del_pollset_set(grpc_exec_ctx *exec_ctx, gpr_mu_unlock(&bag->mu); } -static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, grpc_fd *fd) { - size_t i; - gpr_mu_lock(&pollset_set->mu); - if (pollset_set->fd_count == pollset_set->fd_capacity) { - pollset_set->fd_capacity = GPR_MAX(8, 2 * pollset_set->fd_capacity); - pollset_set->fds = gpr_realloc( - pollset_set->fds, pollset_set->fd_capacity * sizeof(*pollset_set->fds)); - } - GRPC_FD_REF(fd, "pollset_set"); - pollset_set->fds[pollset_set->fd_count++] = fd; - for (i = 0; i < pollset_set->pollset_count; i++) { - pollset_add_fd(exec_ctx, pollset_set->pollsets[i], fd); - } - for (i = 0; i < pollset_set->pollset_set_count; i++) { - pollset_set_add_fd(exec_ctx, pollset_set->pollset_sets[i], fd); - } - gpr_mu_unlock(&pollset_set->mu); -} - -static void pollset_set_del_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, grpc_fd *fd) { - size_t i; - gpr_mu_lock(&pollset_set->mu); - for (i = 0; i < pollset_set->fd_count; i++) { - if (pollset_set->fds[i] == fd) { - pollset_set->fd_count--; - GPR_SWAP(grpc_fd *, pollset_set->fds[i], - pollset_set->fds[pollset_set->fd_count]); - GRPC_FD_UNREF(fd, "pollset_set"); - break; - } - } - for (i = 0; i < pollset_set->pollset_set_count; i++) { - pollset_set_del_fd(exec_ctx, pollset_set->pollset_sets[i], fd); - } - gpr_mu_unlock(&pollset_set->mu); -} - /******************************************************************************* - * event engine binding + * Event engine binding */ static void shutdown_engine(void) { From 73ef9154024290e86a3566a574c04992afc93d00 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 3 Jun 2016 15:25:05 -0700 Subject: [PATCH 031/215] epoll polling strategy now points to the new code --- src/core/lib/iomgr/ev_epoll_linux.c | 4 ---- src/core/lib/iomgr/ev_posix.c | 4 ++-- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index ce42a9e7ce6..ab4224b2d56 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -788,16 +788,12 @@ static void sig_handler(int sig_num) { /* Global state management */ static void pollset_global_init(void) { - gpr_tls_init(&g_current_thread_poller); - gpr_tls_init(&g_current_thread_worker); grpc_wakeup_fd_init(&grpc_global_wakeup_fd); signal(SIGUSR1, sig_handler); } static void pollset_global_shutdown(void) { grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); - gpr_tls_destroy(&g_current_thread_poller); - gpr_tls_destroy(&g_current_thread_worker); } /* Return 1 if the pollset has active threads in pollset_work (pollset must diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index 96399ef8379..b6b113aed3f 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -44,7 +44,7 @@ #include #include -#include "src/core/lib/iomgr/ev_epoll_posix.h" +#include "src/core/lib/iomgr/ev_epoll_linux.h" #include "src/core/lib/iomgr/ev_poll_posix.h" #include "src/core/lib/support/env.h" @@ -62,7 +62,7 @@ typedef struct { } event_engine_factory; static const event_engine_factory g_factories[] = { - {"poll", grpc_init_poll_posix}, {"epoll", grpc_init_epoll_posix}, + {"poll", grpc_init_poll_posix}, {"epoll", grpc_init_epoll_linux}, }; static void add(const char *beg, const char *end, char ***ss, size_t *ns) { From 88ee12fbe98685e736366d6a151a10ed103f8979 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 3 Jun 2016 19:26:48 -0700 Subject: [PATCH 032/215] Handle pollsets and fds witn no polling islands and fix locking bug in pollset_add_fd --- src/core/lib/iomgr/ev_epoll_linux.c | 83 ++++++++++++++++------------- 1 file changed, 45 insertions(+), 38 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index ab4224b2d56..0fb1ccfa0fe 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -701,12 +701,14 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - Decrement the ref count on the polling island and det fd->polling_island to NULL */ gpr_mu_lock(&fd->pi_mu); - - fd->polling_island = polling_island_update_and_lock(fd->polling_island, 1, 0); - polling_island_remove_fd_locked(fd->polling_island, fd, !fd->released, true); - polling_island_unref_and_unlock(fd->polling_island, 1); - fd->polling_island = NULL; - + if (fd->polling_island != NULL) { + fd->polling_island = + polling_island_update_and_lock(fd->polling_island, 1, 0); + polling_island_remove_fd_locked(fd->polling_island, fd, !fd->released, + true); + polling_island_unref_and_unlock(fd->polling_island, 1); + fd->polling_island = NULL; + } gpr_mu_unlock(&fd->pi_mu); grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); @@ -926,13 +928,12 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { set_ready(exec_ctx, fd, &fd->write_closure); } -/* TODO(klempner): We probably want to turn this down a bit */ #define GRPC_EPOLL_MAX_EVENTS 1000 static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, int timeout_ms, sigset_t *sig_mask) { struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; - int epoll_fd; + int epoll_fd = -1; int ep_rv; GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); @@ -943,45 +944,49 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, - pollset->pi_mu - pollset->polling_island->mu */ gpr_mu_lock(&pollset->pi_mu); - pollset->polling_island = - polling_island_update_and_lock(pollset->polling_island, 1, 0); - epoll_fd = pollset->polling_island->epoll_fd; + if (pollset->polling_island != NULL) { + pollset->polling_island = + polling_island_update_and_lock(pollset->polling_island, 1, 0); + epoll_fd = pollset->polling_island->epoll_fd; + gpr_mu_unlock(&pollset->polling_island->mu); + } - /* Release the locks */ - polling_island_unref_and_unlock(pollset->polling_island, 0); /* Keep the ref*/ gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); - do { - ep_rv = epoll_pwait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms, - sig_mask); + /* If epoll_fd == -1, this is a blank pollset and does not have any fds yet */ + if (epoll_fd != -1) { + do { + ep_rv = epoll_pwait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms, + sig_mask); - if (ep_rv < 0) { - if (errno != EINTR) { - /* TODO (sreek) - Check for bad file descriptor error */ - gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); - } - } else { - int i; - for (i = 0; i < ep_rv; ++i) { - grpc_fd *fd = ep_ev[i].data.ptr; - int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); - int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); - int write_ev = ep_ev[i].events & EPOLLOUT; - if (fd == NULL) { - grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); - } else { - if (read_ev || cancel) { - fd_become_readable(exec_ctx, fd); - } - if (write_ev || cancel) { - fd_become_writable(exec_ctx, fd); + if (ep_rv < 0) { + if (errno != EINTR) { + /* TODO (sreek) - Check for bad file descriptor error */ + gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); + } + } else { + int i; + for (i = 0; i < ep_rv; ++i) { + grpc_fd *fd = ep_ev[i].data.ptr; + int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + int write_ev = ep_ev[i].events & EPOLLOUT; + if (fd == NULL) { + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + } else { + if (read_ev || cancel) { + fd_become_readable(exec_ctx, fd); + } + if (write_ev || cancel) { + fd_become_writable(exec_ctx, fd); + } } } } - } - } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); + } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); + } GPR_TIMER_END("pollset_work_and_unlock", 0); } @@ -1141,8 +1146,10 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } } else if (fd->polling_island == NULL) { pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); + gpr_mu_unlock(&pi_new->mu); } else if (pollset->polling_island == NULL) { pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); + gpr_mu_unlock(&pi_new->mu); } else { pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); } From 79a6233bef501e1f51250351a55abc61cc024827 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Sat, 4 Jun 2016 14:01:03 -0700 Subject: [PATCH 033/215] Fix a few bugs in ev_epoll_linux.c 1. pollset_add_fd: Add fd to epoll set if fd->polling_island == NULL 2. close(fd) in fd_orphan instead of polling_island_remove_fd_locked() since fd->polling_island may be NULL 3. If pollset work() is interrupted, do a zero timeout epoll_wait(). pollset_work may be called without a polling island --- src/core/lib/iomgr/ev_epoll_linux.c | 125 +++++++++++++++++----------- 1 file changed, 78 insertions(+), 47 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 0fb1ccfa0fe..3aa26109f22 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -72,9 +72,14 @@ struct grpc_fd { gpr_atm refst; gpr_mu mu; + + /* Indicates that the fd is shutdown and that any pending read/write closures + should fail */ bool shutdown; - int closed; - bool released; + + /* The fd is either closed or we relinquished control of it. In either cases, + this indicates that the 'fd' on this structure is no longer valid */ + bool orphaned; grpc_closure *read_closure; grpc_closure *write_closure; @@ -251,16 +256,13 @@ static void polling_island_remove_all_fds_locked(polling_island *pi, /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, - bool close_fd, bool remove_fd_ref) { + bool is_fd_closed) { int err; size_t i; - /* Calling close() on the fd will automatically remove it from the epoll set. - If not calling close(), the fd must be explicitly removed from the epoll - set */ - if (close_fd) { - close(fd->fd); - } else { + /* If fd is already closed, then it would have been automatically been removed + from the epoll set */ + if (!is_fd_closed) { err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, fd->fd, NULL); if (err < 0 && errno != ENOENT) { gpr_log(GPR_ERROR, "epoll_ctl delete for fd: %d failed with error; %s", @@ -271,9 +273,7 @@ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, for (i = 0; i < pi->fd_cnt; i++) { if (pi->fds[i] == fd) { pi->fds[i] = pi->fds[--pi->fd_cnt]; - if (remove_fd_ref) { - GRPC_FD_UNREF(fd, "polling_island"); - } + GRPC_FD_UNREF(fd, "polling_island"); break; } } @@ -644,8 +644,7 @@ static grpc_fd *fd_create(int fd, const char *name) { new_fd->polling_island = NULL; new_fd->freelist_next = NULL; new_fd->on_done_closure = NULL; - new_fd->closed = 0; - new_fd->released = false; + new_fd->orphaned = false; gpr_mu_unlock(&new_fd->mu); @@ -666,7 +665,7 @@ static bool fd_is_orphaned(grpc_fd *fd) { static int fd_wrapped_fd(grpc_fd *fd) { int ret_fd = -1; gpr_mu_lock(&fd->mu); - if (!fd->released && !fd->closed) { + if (!fd->orphaned) { ret_fd = fd->fd; } gpr_mu_unlock(&fd->mu); @@ -678,34 +677,35 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure *on_done, int *release_fd, const char *reason) { /* TODO(sreek) In ev_poll_posix.c,the lock is acquired a little later. Why? */ + bool is_fd_closed = false; gpr_mu_lock(&fd->mu); fd->on_done_closure = on_done; /* If release_fd is not NULL, we should be relinquishing control of the file descriptor fd->fd (but we still own the grpc_fd structure). */ - fd->released = release_fd != NULL; - if (!fd->released) { - shutdown(fd->fd, SHUT_RDWR); - } else { + if (release_fd != NULL) { *release_fd = fd->fd; + } else { + close(fd->fd); + is_fd_closed = true; } - REF_BY(fd, 1, reason); /* Remove active status, but keep referenced */ - fd->closed = 1; + fd->orphaned = true; + + /* Remove the active status but keep referenced. We want this grpc_fd struct + to be alive (and not added to freelist) until the end of this function */ + REF_BY(fd, 1, reason); /* Remove the fd from the polling island: - Update the fd->polling_island to point to the latest polling island - - Remove the fd from the polling island. Also, call close() on the file - descriptor fd->fd ONLY if we haven't relinquised control (i.e - fd->released is 'false') - - Decrement the ref count on the polling island and det fd->polling_island - to NULL */ + - Remove the fd from the polling island. + - Remove a ref to the polling island and set fd->polling_island to NULL */ gpr_mu_lock(&fd->pi_mu); if (fd->polling_island != NULL) { fd->polling_island = polling_island_update_and_lock(fd->polling_island, 1, 0); - polling_island_remove_fd_locked(fd->polling_island, fd, !fd->released, - true); + polling_island_remove_fd_locked(fd->polling_island, fd, is_fd_closed); + polling_island_unref_and_unlock(fd->polling_island, 1); fd->polling_island = NULL; } @@ -839,17 +839,20 @@ static void pollset_kick(grpc_pollset *p, grpc_pollset_worker *worker = specific_worker; if (worker != NULL) { if (worker == GRPC_POLLSET_KICK_BROADCAST) { - GPR_TIMER_BEGIN("pollset_kick.broadcast", 0); + gpr_log(GPR_DEBUG, "pollset_kick: broadcast!"); if (pollset_has_workers(p)) { + GPR_TIMER_BEGIN("pollset_kick.broadcast", 0); for (worker = p->root_worker.next; worker != &p->root_worker; worker = worker->next) { pthread_kill(worker->pt_id, SIGUSR1); } } else { + gpr_log(GPR_DEBUG, "pollset_kick: (broadcast) Kicked without pollers"); p->kicked_without_pollers = true; } GPR_TIMER_END("pollset_kick.broadcast", 0); } else { + gpr_log(GPR_DEBUG, "pollset_kick: kicked kicked_specifically"); GPR_TIMER_MARK("kicked_specifically", 0); worker->kicked_specifically = true; pthread_kill(worker->pt_id, SIGUSR1); @@ -860,9 +863,11 @@ static void pollset_kick(grpc_pollset *p, if (worker != NULL) { GPR_TIMER_MARK("finally_kick", 0); push_back_worker(p, worker); + gpr_log(GPR_DEBUG, "pollset_kick: anonymous kick"); pthread_kill(worker->pt_id, SIGUSR1); } else { GPR_TIMER_MARK("kicked_no_pollers", 0); + gpr_log(GPR_DEBUG, "pollset_kick: kicked without pollers"); p->kicked_without_pollers = true; } } @@ -935,6 +940,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; int epoll_fd = -1; int ep_rv; + gpr_log(GPR_DEBUG, "pollset_work_and_unlock: Entering.."); GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the @@ -949,6 +955,16 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, pollset->polling_island = polling_island_update_and_lock(pollset->polling_island, 1, 0); epoll_fd = pollset->polling_island->epoll_fd; + if (pollset->polling_island->fd_cnt == 0) { + gpr_log(GPR_DEBUG, "pollset_work_and_unlock: epoll_fd: %d, No other fds", + epoll_fd); + } + for (size_t i = 0; i < pollset->polling_island->fd_cnt; i++) { + gpr_log(GPR_DEBUG, + "pollset_work_and_unlock: epoll_fd: %d, fd_count: %d, fd[%d]: %d", + epoll_fd, pollset->polling_island->fd_cnt, i, + pollset->polling_island->fds[i]->fd); + } gpr_mu_unlock(&pollset->polling_island->mu); } @@ -958,36 +974,47 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, /* If epoll_fd == -1, this is a blank pollset and does not have any fds yet */ if (epoll_fd != -1) { do { + gpr_timespec before_epoll = gpr_now(GPR_CLOCK_PRECISE); + gpr_log(GPR_DEBUG, "pollset_work_and_unlock: epoll_wait()...."); ep_rv = epoll_pwait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms, sig_mask); + gpr_timespec after_epoll = gpr_now(GPR_CLOCK_PRECISE); + int dur = gpr_time_to_millis(gpr_time_sub(after_epoll, before_epoll)); + gpr_log(GPR_DEBUG, + "pollset_work_and_unlock: DONE epoll_wait() : %d ms, ep_rv: %d", + dur, ep_rv); if (ep_rv < 0) { if (errno != EINTR) { /* TODO (sreek) - Check for bad file descriptor error */ gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); + } else { + gpr_log(GPR_DEBUG, "pollset_work_and_unlock: 0-timeout epoll_wait()"); + ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); + gpr_log(GPR_DEBUG, "pollset_work_and_unlock: ep_rv: %d", ep_rv); } - } else { - int i; - for (i = 0; i < ep_rv; ++i) { - grpc_fd *fd = ep_ev[i].data.ptr; - int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); - int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); - int write_ev = ep_ev[i].events & EPOLLOUT; - if (fd == NULL) { - grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); - } else { - if (read_ev || cancel) { - fd_become_readable(exec_ctx, fd); - } - if (write_ev || cancel) { - fd_become_writable(exec_ctx, fd); - } + } + + int i; + for (i = 0; i < ep_rv; ++i) { + grpc_fd *fd = ep_ev[i].data.ptr; + int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + int write_ev = ep_ev[i].events & EPOLLOUT; + if (fd == NULL) { + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + } else { + if (read_ev || cancel) { + fd_become_readable(exec_ctx, fd); + } + if (write_ev || cancel) { + fd_become_writable(exec_ctx, fd); } } } } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); } - + gpr_log(GPR_DEBUG, "pollset_work_and_unlock: Leaving.."); GPR_TIMER_END("pollset_work_and_unlock", 0); } @@ -1060,7 +1087,7 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker **worker_hdl, gpr_timespec now, gpr_timespec deadline) { GPR_TIMER_BEGIN("pollset_work", 0); - + gpr_log(GPR_DEBUG, "pollset_work: enter"); int timeout_ms = poll_deadline_to_millis_timeout(deadline, now); sigset_t new_mask; @@ -1079,6 +1106,7 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, work that needs attention like an event on the completion queue or an alarm */ GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); + gpr_log(GPR_INFO, "pollset_work: kicked without pollers.."); pollset->kicked_without_pollers = 0; } else if (!pollset->shutting_down) { sigemptyset(&new_mask); @@ -1113,12 +1141,14 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, gpr_mu_lock(&pollset->mu); } + gpr_log(GPR_DEBUG, "pollset_work(): leaving"); *worker_hdl = NULL; GPR_TIMER_END("pollset_work", 0); } static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { + gpr_log(GPR_DEBUG, "pollset_add_fd: pollset: %p, fd: %d", pollset, fd->fd); /* TODO sreek - Check if we need to get a pollset->mu lock here */ gpr_mu_lock(&pollset->pi_mu); gpr_mu_lock(&fd->pi_mu); @@ -1146,6 +1176,7 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } } else if (fd->polling_island == NULL) { pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); + polling_island_add_fds_locked(pollset->polling_island, &fd, 1, true); gpr_mu_unlock(&pi_new->mu); } else if (pollset->polling_island == NULL) { pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); From b5fe44ea0f7e5ef6f013d6c97725f93a74fed5ef Mon Sep 17 00:00:00 2001 From: Julien Boeuf Date: Mon, 6 Jun 2016 09:50:38 +0200 Subject: [PATCH 034/215] Adding error details for JWT and oauth2. --- src/core/lib/security/credentials/jwt/jwt_credentials.c | 3 ++- .../lib/security/credentials/oauth2/oauth2_credentials.c | 5 +++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/src/core/lib/security/credentials/jwt/jwt_credentials.c b/src/core/lib/security/credentials/jwt/jwt_credentials.c index 0d29729da17..7f96e5247b0 100644 --- a/src/core/lib/security/credentials/jwt/jwt_credentials.c +++ b/src/core/lib/security/credentials/jwt/jwt_credentials.c @@ -116,7 +116,8 @@ static void jwt_get_request_metadata(grpc_exec_ctx *exec_ctx, GRPC_CREDENTIALS_OK, NULL); grpc_credentials_md_store_unref(jwt_md); } else { - cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_ERROR, NULL); + cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_ERROR, + "Could not generate JWT."); } } diff --git a/src/core/lib/security/credentials/oauth2/oauth2_credentials.c b/src/core/lib/security/credentials/oauth2/oauth2_credentials.c index 81457183d2d..a0e4b6fe8e9 100644 --- a/src/core/lib/security/credentials/oauth2/oauth2_credentials.c +++ b/src/core/lib/security/credentials/oauth2/oauth2_credentials.c @@ -233,10 +233,11 @@ static void on_oauth2_token_fetcher_http_response( c->token_expiration = gpr_time_add(gpr_now(GPR_CLOCK_REALTIME), token_lifetime); r->cb(exec_ctx, r->user_data, c->access_token_md->entries, - c->access_token_md->num_entries, status, NULL); + c->access_token_md->num_entries, GRPC_CREDENTIALS_OK, NULL); } else { c->token_expiration = gpr_inf_past(GPR_CLOCK_REALTIME); - r->cb(exec_ctx, r->user_data, NULL, 0, status, NULL); + r->cb(exec_ctx, r->user_data, NULL, 0, status, + "Error occured when fetching oauth2 token."); } gpr_mu_unlock(&c->mu); grpc_credentials_metadata_request_destroy(r); From 38c0cdee34697bacb95f34d1a4d31131d0df3af5 Mon Sep 17 00:00:00 2001 From: Julien Boeuf Date: Mon, 6 Jun 2016 14:46:08 +0200 Subject: [PATCH 035/215] Fix double-free issue for optional_message. Also better end to end testing of plumbing of plugin error message. --- .../ext/transport/chttp2/transport/chttp2_transport.c | 8 +++----- test/cpp/end2end/end2end_test.cc | 6 +++++- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/src/core/ext/transport/chttp2/transport/chttp2_transport.c b/src/core/ext/transport/chttp2/transport/chttp2_transport.c index 1abc49f435e..b0da0578f5e 100644 --- a/src/core/ext/transport/chttp2/transport/chttp2_transport.c +++ b/src/core/ext/transport/chttp2/transport/chttp2_transport.c @@ -1504,17 +1504,15 @@ static void close_from_api(grpc_exec_ctx *exec_ctx, gpr_slice_buffer_add(&transport_global->qbuf, gpr_slice_ref(*optional_message)); } - gpr_slice_buffer_add( &transport_global->qbuf, grpc_chttp2_rst_stream_create(stream_global->id, GRPC_CHTTP2_NO_ERROR, &stream_global->stats.outgoing)); - - if (optional_message) { - gpr_slice_ref(*optional_message); - } } + if (optional_message) { + gpr_slice_ref(*optional_message); + } grpc_chttp2_fake_status(exec_ctx, transport_global, stream_global, status, optional_message); grpc_chttp2_mark_stream_closed(exec_ctx, transport_global, stream_global, 1, diff --git a/test/cpp/end2end/end2end_test.cc b/test/cpp/end2end/end2end_test.cc index f52aa52f39f..4602145f54b 100644 --- a/test/cpp/end2end/end2end_test.cc +++ b/test/cpp/end2end/end2end_test.cc @@ -75,6 +75,8 @@ bool CheckIsLocalhost(const grpc::string& addr) { addr.substr(0, kIpv6.size()) == kIpv6; } +const char kTestCredsPluginErrorMsg[] = "Could not find plugin metadata."; + class TestMetadataCredentialsPlugin : public MetadataCredentialsPlugin { public: static const char kMetadataKey[]; @@ -99,7 +101,7 @@ class TestMetadataCredentialsPlugin : public MetadataCredentialsPlugin { metadata->insert(std::make_pair(kMetadataKey, metadata_value_)); return Status::OK; } else { - return Status(StatusCode::NOT_FOUND, "Could not find plugin metadata."); + return Status(StatusCode::NOT_FOUND, kTestCredsPluginErrorMsg); } } @@ -1318,6 +1320,7 @@ TEST_P(SecureEnd2endTest, NonBlockingAuthMetadataPluginFailure) { Status s = stub_->Echo(&context, request, &response); EXPECT_FALSE(s.ok()); EXPECT_EQ(s.error_code(), StatusCode::UNAUTHENTICATED); + EXPECT_EQ(s.error_message(), kTestCredsPluginErrorMsg); } TEST_P(SecureEnd2endTest, NonBlockingAuthMetadataPluginAndProcessorSuccess) { @@ -1375,6 +1378,7 @@ TEST_P(SecureEnd2endTest, BlockingAuthMetadataPluginFailure) { Status s = stub_->Echo(&context, request, &response); EXPECT_FALSE(s.ok()); EXPECT_EQ(s.error_code(), StatusCode::UNAUTHENTICATED); + EXPECT_EQ(s.error_message(), kTestCredsPluginErrorMsg); } TEST_P(SecureEnd2endTest, ClientAuthContext) { From 7d3d4d48cd3a0b916d7f561f6fa525886fb34d8a Mon Sep 17 00:00:00 2001 From: Julien Boeuf Date: Mon, 6 Jun 2016 14:48:41 +0200 Subject: [PATCH 036/215] Fixing clang-format. --- src/core/lib/security/credentials/credentials.h | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/src/core/lib/security/credentials/credentials.h b/src/core/lib/security/credentials/credentials.h index c6df6cdf2f7..b097233bee4 100644 --- a/src/core/lib/security/credentials/credentials.h +++ b/src/core/lib/security/credentials/credentials.h @@ -156,12 +156,9 @@ void grpc_credentials_md_store_unref(grpc_credentials_md_store *store); /* --- grpc_call_credentials. --- */ /* error_details must be NULL if status is GRPC_CREDENTIALS_OK. */ -typedef void (*grpc_credentials_metadata_cb)(grpc_exec_ctx *exec_ctx, - void *user_data, - grpc_credentials_md *md_elems, - size_t num_md, - grpc_credentials_status status, - const char *error_details); +typedef void (*grpc_credentials_metadata_cb)( + grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, + size_t num_md, grpc_credentials_status status, const char *error_details); typedef struct { void (*destruct)(grpc_call_credentials *c); From 4c11a20bf0b6b1d64a2800bcb06f76404294aa84 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 6 Jun 2016 09:23:25 -0700 Subject: [PATCH 037/215] Remove unused files --- BUILD | 6 - Makefile | 38 - binding.gyp | 1 - build.yaml | 14 - config.m4 | 1 - gRPC.podspec | 3 - grpc.gemspec | 2 - package.xml | 2 - src/core/lib/iomgr/ctiller_ev_epoll_linux.c | 461 ------- src/core/lib/iomgr/ev_epoll_linux.c | 2 +- src/core/lib/iomgr/ev_epoll_posix.c | 1209 ----------------- src/core/lib/iomgr/ev_epoll_posix.h | 41 - src/python/grpcio/grpc_core_dependencies.py | 1 - test/core/network_benchmarks/epoll_test.c | 263 ---- .../network_benchmarks/low_level_ping_pong.c | 2 - tools/doxygen/Doxyfile.core.internal | 2 - tools/run_tests/sources_and_headers.json | 19 - tools/run_tests/tests.json | 15 - vsprojects/vcxproj/grpc/grpc.vcxproj | 3 - vsprojects/vcxproj/grpc/grpc.vcxproj.filters | 6 - .../grpc_unsecure/grpc_unsecure.vcxproj | 3 - .../grpc_unsecure.vcxproj.filters | 6 - 22 files changed, 1 insertion(+), 2099 deletions(-) delete mode 100644 src/core/lib/iomgr/ctiller_ev_epoll_linux.c delete mode 100644 src/core/lib/iomgr/ev_epoll_posix.c delete mode 100644 src/core/lib/iomgr/ev_epoll_posix.h delete mode 100644 test/core/network_benchmarks/epoll_test.c diff --git a/BUILD b/BUILD index a32352ebb30..70354a88101 100644 --- a/BUILD +++ b/BUILD @@ -179,7 +179,6 @@ cc_library( "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", "src/core/lib/iomgr/ev_epoll_linux.h", - "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", @@ -324,7 +323,6 @@ cc_library( "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", "src/core/lib/iomgr/ev_epoll_linux.c", - "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", "src/core/lib/iomgr/exec_ctx.c", @@ -551,7 +549,6 @@ cc_library( "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", "src/core/lib/iomgr/ev_epoll_linux.h", - "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", @@ -673,7 +670,6 @@ cc_library( "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", "src/core/lib/iomgr/ev_epoll_linux.c", - "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", "src/core/lib/iomgr/exec_ctx.c", @@ -1367,7 +1363,6 @@ objc_library( "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", "src/core/lib/iomgr/ev_epoll_linux.c", - "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", "src/core/lib/iomgr/exec_ctx.c", @@ -1573,7 +1568,6 @@ objc_library( "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", "src/core/lib/iomgr/ev_epoll_linux.h", - "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", diff --git a/Makefile b/Makefile index 235f32d9a30..1c83aec21e7 100644 --- a/Makefile +++ b/Makefile @@ -903,7 +903,6 @@ dns_resolver_connectivity_test: $(BINDIR)/$(CONFIG)/dns_resolver_connectivity_te dns_resolver_test: $(BINDIR)/$(CONFIG)/dns_resolver_test dualstack_socket_test: $(BINDIR)/$(CONFIG)/dualstack_socket_test endpoint_pair_test: $(BINDIR)/$(CONFIG)/endpoint_pair_test -epoll_test: $(BINDIR)/$(CONFIG)/epoll_test fd_conservation_posix_test: $(BINDIR)/$(CONFIG)/fd_conservation_posix_test fd_posix_test: $(BINDIR)/$(CONFIG)/fd_posix_test fling_client: $(BINDIR)/$(CONFIG)/fling_client @@ -1235,7 +1234,6 @@ buildtests_c: privatelibs_c \ $(BINDIR)/$(CONFIG)/dns_resolver_test \ $(BINDIR)/$(CONFIG)/dualstack_socket_test \ $(BINDIR)/$(CONFIG)/endpoint_pair_test \ - $(BINDIR)/$(CONFIG)/epoll_test \ $(BINDIR)/$(CONFIG)/fd_conservation_posix_test \ $(BINDIR)/$(CONFIG)/fd_posix_test \ $(BINDIR)/$(CONFIG)/fling_client \ @@ -1499,8 +1497,6 @@ test_c: buildtests_c $(Q) $(BINDIR)/$(CONFIG)/dualstack_socket_test || ( echo test dualstack_socket_test failed ; exit 1 ) $(E) "[RUN] Testing endpoint_pair_test" $(Q) $(BINDIR)/$(CONFIG)/endpoint_pair_test || ( echo test endpoint_pair_test failed ; exit 1 ) - $(E) "[RUN] Testing epoll_test" - $(Q) $(BINDIR)/$(CONFIG)/epoll_test || ( echo test epoll_test failed ; exit 1 ) $(E) "[RUN] Testing fd_conservation_posix_test" $(Q) $(BINDIR)/$(CONFIG)/fd_conservation_posix_test || ( echo test fd_conservation_posix_test failed ; exit 1 ) $(E) "[RUN] Testing fd_posix_test" @@ -2491,7 +2487,6 @@ LIBGRPC_SRC = \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ src/core/lib/iomgr/ev_epoll_linux.c \ - src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ @@ -2847,7 +2842,6 @@ LIBGRPC_UNSECURE_SRC = \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ src/core/lib/iomgr/ev_epoll_linux.c \ - src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ @@ -6581,38 +6575,6 @@ endif endif -EPOLL_TEST_SRC = \ - test/core/network_benchmarks/epoll_test.c \ - -EPOLL_TEST_OBJS = $(addprefix $(OBJDIR)/$(CONFIG)/, $(addsuffix .o, $(basename $(EPOLL_TEST_SRC)))) -ifeq ($(NO_SECURE),true) - -# You can't build secure targets if you don't have OpenSSL. - -$(BINDIR)/$(CONFIG)/epoll_test: openssl_dep_error - -else - - - -$(BINDIR)/$(CONFIG)/epoll_test: $(EPOLL_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a - $(E) "[LD] Linking $@" - $(Q) mkdir -p `dirname $@` - $(Q) $(LD) $(LDFLAGS) $(EPOLL_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a $(LDLIBS) $(LDLIBS_SECURE) -o $(BINDIR)/$(CONFIG)/epoll_test - -endif - -$(OBJDIR)/$(CONFIG)/test/core/network_benchmarks/epoll_test.o: $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a - -deps_epoll_test: $(EPOLL_TEST_OBJS:.o=.dep) - -ifneq ($(NO_SECURE),true) -ifneq ($(NO_DEPS),true) --include $(EPOLL_TEST_OBJS:.o=.dep) -endif -endif - - FD_CONSERVATION_POSIX_TEST_SRC = \ test/core/iomgr/fd_conservation_posix_test.c \ diff --git a/binding.gyp b/binding.gyp index 41e1b5bb416..255998aafd8 100644 --- a/binding.gyp +++ b/binding.gyp @@ -582,7 +582,6 @@ 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', 'src/core/lib/iomgr/ev_epoll_linux.c', - 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', 'src/core/lib/iomgr/exec_ctx.c', diff --git a/build.yaml b/build.yaml index db9787546ad..75c7a76bdb3 100644 --- a/build.yaml +++ b/build.yaml @@ -166,7 +166,6 @@ filegroups: - src/core/lib/iomgr/endpoint.h - src/core/lib/iomgr/endpoint_pair.h - src/core/lib/iomgr/ev_epoll_linux.h - - src/core/lib/iomgr/ev_epoll_posix.h - src/core/lib/iomgr/ev_poll_posix.h - src/core/lib/iomgr/ev_posix.h - src/core/lib/iomgr/exec_ctx.h @@ -242,7 +241,6 @@ filegroups: - src/core/lib/iomgr/endpoint_pair_posix.c - src/core/lib/iomgr/endpoint_pair_windows.c - src/core/lib/iomgr/ev_epoll_linux.c - - src/core/lib/iomgr/ev_epoll_posix.c - src/core/lib/iomgr/ev_poll_posix.c - src/core/lib/iomgr/ev_posix.c - src/core/lib/iomgr/exec_ctx.c @@ -1321,18 +1319,6 @@ targets: - grpc - gpr_test_util - gpr -- name: epoll_test - build: test - language: c - src: - - test/core/network_benchmarks/epoll_test.c - deps: - - grpc_test_util - - grpc - - gpr_test_util - - gpr - platforms: - - linux - name: fd_conservation_posix_test build: test language: c diff --git a/config.m4 b/config.m4 index 4308295afda..e2d1c00b6e4 100644 --- a/config.m4 +++ b/config.m4 @@ -101,7 +101,6 @@ if test "$PHP_GRPC" != "no"; then src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ src/core/lib/iomgr/ev_epoll_linux.c \ - src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ diff --git a/gRPC.podspec b/gRPC.podspec index de55880125a..736ae98b543 100644 --- a/gRPC.podspec +++ b/gRPC.podspec @@ -182,7 +182,6 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/endpoint.h', 'src/core/lib/iomgr/endpoint_pair.h', 'src/core/lib/iomgr/ev_epoll_linux.h', - 'src/core/lib/iomgr/ev_epoll_posix.h', 'src/core/lib/iomgr/ev_poll_posix.h', 'src/core/lib/iomgr/ev_posix.h', 'src/core/lib/iomgr/exec_ctx.h', @@ -361,7 +360,6 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', 'src/core/lib/iomgr/ev_epoll_linux.c', - 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', 'src/core/lib/iomgr/exec_ctx.c', @@ -551,7 +549,6 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/endpoint.h', 'src/core/lib/iomgr/endpoint_pair.h', 'src/core/lib/iomgr/ev_epoll_linux.h', - 'src/core/lib/iomgr/ev_epoll_posix.h', 'src/core/lib/iomgr/ev_poll_posix.h', 'src/core/lib/iomgr/ev_posix.h', 'src/core/lib/iomgr/exec_ctx.h', diff --git a/grpc.gemspec b/grpc.gemspec index 54ae2eb68dd..01b2890493f 100755 --- a/grpc.gemspec +++ b/grpc.gemspec @@ -191,7 +191,6 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/endpoint.h ) s.files += %w( src/core/lib/iomgr/endpoint_pair.h ) s.files += %w( src/core/lib/iomgr/ev_epoll_linux.h ) - s.files += %w( src/core/lib/iomgr/ev_epoll_posix.h ) s.files += %w( src/core/lib/iomgr/ev_poll_posix.h ) s.files += %w( src/core/lib/iomgr/ev_posix.h ) s.files += %w( src/core/lib/iomgr/exec_ctx.h ) @@ -340,7 +339,6 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/endpoint_pair_posix.c ) s.files += %w( src/core/lib/iomgr/endpoint_pair_windows.c ) s.files += %w( src/core/lib/iomgr/ev_epoll_linux.c ) - s.files += %w( src/core/lib/iomgr/ev_epoll_posix.c ) s.files += %w( src/core/lib/iomgr/ev_poll_posix.c ) s.files += %w( src/core/lib/iomgr/ev_posix.c ) s.files += %w( src/core/lib/iomgr/exec_ctx.c ) diff --git a/package.xml b/package.xml index d8e82a8bc37..ba6e11fadc7 100644 --- a/package.xml +++ b/package.xml @@ -198,7 +198,6 @@ - @@ -347,7 +346,6 @@ - diff --git a/src/core/lib/iomgr/ctiller_ev_epoll_linux.c b/src/core/lib/iomgr/ctiller_ev_epoll_linux.c deleted file mode 100644 index 23c20a77aae..00000000000 --- a/src/core/lib/iomgr/ctiller_ev_epoll_linux.c +++ /dev/null @@ -1,461 +0,0 @@ -/* - * - * Copyright 2015-2016, Google Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are - * met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following disclaimer - * in the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Google Inc. nor the names of its - * contributors may be used to endorse or promote products derived from - * this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - * - */ - -#include "src/core/lib/iomgr/ev_epoll_linux.h" - -#include -#include -#include -#include - -#include -#include -#include -#include - -#include "src/core/lib/iomgr/iomgr_internal.h" - -/* TODO(sreek) Remove this file */ - - -//////////////////////////////////////////////////////////////////////////////// -// Definitions - -#define STATE_NOT_READY ((gpr_atm)0) -#define STATE_READY ((gpr_atm)1) - -typedef enum { POLLABLE_FD, POLLABLE_EPOLL_SET } pollable_type; - -typedef struct { - pollable_type type; - int fd; - grpc_iomgr_object iomgr_object; -} pollable_object; - -typedef struct polling_island { - pollable_object pollable; - gpr_mu mu; - int refs; - grpc_fd *only_fd; - struct polling_island *became; - struct polling_island *next; -} polling_island; - -struct grpc_fd { - pollable_object pollable; - - // each event atomic is a tri state: - // STATE_NOT_READY - no event received, nobody waiting for it either - // STATE_READY - event received, nobody waiting for it - // closure pointer - no event received, upper layer is waiting for it - gpr_atm on_readable; - gpr_atm on_writable; - - // mutex guarding set_ready & shutdown state - gpr_mu set_ready_mu; - bool shutdown; - - // mutex protecting polling_island - gpr_mu polling_island_mu; - // current polling island - polling_island *polling_island; - - grpc_fd *next_free; -}; - -struct grpc_pollset_worker {}; - -struct grpc_pollset { - gpr_mu mu; - // current polling island - polling_island *polling_island; -}; - -//////////////////////////////////////////////////////////////////////////////// -// Polling island implementation - -static gpr_mu g_pi_freelist_mu; -static polling_island *g_first_free_pi; - -static void add_pollable_to_epoll_set(pollable_object *pollable, int epoll_set, - uint32_t events) { - struct epoll_event ev; - ev.events = events; - ev.data.ptr = pollable; - int err = epoll_ctl(epoll_set, EPOLL_CTL_ADD, pollable->fd, &ev); - if (err < 0) { - gpr_log(GPR_ERROR, "epoll_ctl add for %d faild: %s", pollable->fd, - strerror(errno)); - } -} - -static void add_fd_to_epoll_set(grpc_fd *fd, int epoll_set) { - add_pollable_to_epoll_set(&fd->pollable, epoll_set, - EPOLLIN | EPOLLOUT | EPOLLET); -} - -static void add_island_to_epoll_set(polling_island *pi, int epoll_set) { - add_pollable_to_epoll_set(&pi->pollable, epoll_set, EPOLLIN | EPOLLET); -} - -static polling_island *polling_island_create(grpc_fd *initial_fd) { - polling_island *r = NULL; - gpr_mu_lock(&g_pi_freelist_mu); - if (g_first_free_pi == NULL) { - r = gpr_malloc(sizeof(*r)); - r->pollable.type = POLLABLE_EPOLL_SET; - gpr_mu_init(&r->mu); - } else { - r = g_first_free_pi; - g_first_free_pi = r->next; - } - gpr_mu_unlock(&g_pi_freelist_mu); - - r->pollable.fd = epoll_create1(EPOLL_CLOEXEC); - GPR_ASSERT(r->pollable.fd >= 0); - - gpr_mu_lock(&r->mu); - r->only_fd = initial_fd; - r->refs = 2; // creation of a polling island => a referencing pollset & fd - gpr_mu_unlock(&r->mu); - - add_fd_to_epoll_set(initial_fd, r->pollable.fd); - return r; -} - -static void polling_island_delete(polling_island *p) { - gpr_mu_lock(&g_pi_freelist_mu); - p->next = g_first_free_pi; - g_first_free_pi = p; - gpr_mu_unlock(&g_pi_freelist_mu); -} - -static polling_island *polling_island_add(polling_island *p, grpc_fd *fd) { - gpr_mu_lock(&p->mu); - p->only_fd = NULL; - p->refs++; // new fd picks up a ref - gpr_mu_unlock(&p->mu); - - add_fd_to_epoll_set(fd, p->pollable.fd); - - return p; -} - -static void add_siblings_to(polling_island *siblings, polling_island *dest) { - polling_island *sibling_tail = dest; - while (sibling_tail->next != NULL) { - sibling_tail = sibling_tail->next; - } - sibling_tail->next = siblings; -} - -static polling_island *polling_island_merge(polling_island *a, - polling_island *b) { - GPR_ASSERT(a != b); - polling_island *out; - - gpr_mu_lock(&GPR_MIN(a, b)->mu); - gpr_mu_lock(&GPR_MAX(a, b)->mu); - - GPR_ASSERT(a->became == NULL); - GPR_ASSERT(b->became == NULL); - - if (a->only_fd == NULL && b->only_fd == NULL) { - b->became = a; - add_siblings_to(b, a); - add_island_to_epoll_set(b, a->pollable.fd); - out = a; - } else if (a->only_fd == NULL) { - GPR_ASSERT(b->only_fd != NULL); - add_fd_to_epoll_set(b->only_fd, a->pollable.fd); - b->became = a; - out = a; - } else if (b->only_fd == NULL) { - GPR_ASSERT(a->only_fd != NULL); - add_fd_to_epoll_set(a->only_fd, b->pollable.fd); - a->became = b; - out = b; - } else { - add_fd_to_epoll_set(b->only_fd, a->pollable.fd); - a->only_fd = NULL; - b->only_fd = NULL; - b->became = a; - out = a; - } - - gpr_mu_unlock(&a->mu); - gpr_mu_unlock(&b->mu); - - return out; -} - -static polling_island *polling_island_update_and_lock(polling_island *p) { - gpr_mu_lock(&p->mu); - if (p->became != NULL) { - do { - polling_island *from = p; - p = p->became; - gpr_mu_lock(&p->mu); - bool delete_from = 0 == --from->refs; - p->refs++; - gpr_mu_unlock(&from->mu); - if (delete_from) { - polling_island_delete(from); - } - } while (p->became != NULL); - } - return p; -} - -static polling_island *polling_island_ref(polling_island *p) { - gpr_mu_lock(&p->mu); - gpr_mu_unlock(&p->mu); - return p; -} - -static void polling_island_drop(polling_island *p) {} - -static polling_island *polling_island_update(polling_island *p, - int updating_owner_count) { - p = polling_island_update_and_lock(p); - GPR_ASSERT(p->refs != 0); - p->refs += updating_owner_count; - gpr_mu_unlock(&p->mu); - return p; -} - -//////////////////////////////////////////////////////////////////////////////// -// FD implementation - -static gpr_mu g_fd_freelist_mu; -static grpc_fd *g_first_free_fd; - -static grpc_fd *fd_create(int fd, const char *name) { - grpc_fd *r = NULL; - gpr_mu_lock(&g_fd_freelist_mu); - if (g_first_free_fd == NULL) { - r = gpr_malloc(sizeof(*r)); - r->pollable.type = POLLABLE_FD; - gpr_atm_rel_store(&r->on_readable, 0); - gpr_atm_rel_store(&r->on_writable, 0); - gpr_mu_init(&r->polling_island_mu); - gpr_mu_init(&r->set_ready_mu); - } else { - r = g_first_free_fd; - g_first_free_fd = r->next_free; - } - gpr_mu_unlock(&g_fd_freelist_mu); - - r->pollable.fd = fd; - grpc_iomgr_register_object(&r->pollable.iomgr_object, name); - r->next_free = NULL; - return r; -} - -static int fd_wrapped_fd(grpc_fd *fd) { return fd->pollable.fd; } - -static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure *on_done, int *release_fd, - const char *reason) { - if (release_fd != NULL) { - *release_fd = fd->pollable.fd; - } else { - close(fd->pollable.fd); - } - - gpr_mu_lock(&fd->polling_island_mu); - if (fd->polling_island != NULL) { - polling_island_drop(fd->polling_island); - } - gpr_mu_unlock(&fd->polling_island_mu); - - gpr_mu_lock(&g_fd_freelist_mu); - fd->next_free = g_first_free_fd; - g_first_free_fd = fd; - grpc_iomgr_unregister_object(&fd->pollable.iomgr_object); - gpr_mu_unlock(&g_fd_freelist_mu); - - grpc_exec_ctx_enqueue(exec_ctx, on_done, true, NULL); -} - -static void notify_on(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure *closure, gpr_atm *state) { - if (gpr_atm_acq_cas(state, STATE_NOT_READY, (gpr_atm)closure)) { - // state was not ready, and is now the closure - we're done */ - } else { - // cas failed - we MUST be in STATE_READY (can't request two notifications - // for the same event) - // flip back to not ready, enqueue the closure directly - GPR_ASSERT(gpr_atm_rel_cas(state, STATE_READY, STATE_NOT_READY)); - grpc_exec_ctx_enqueue(exec_ctx, closure, true, NULL); - } -} - -static void fd_notify_on_read(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure *closure) { - notify_on(exec_ctx, fd, closure, &fd->on_readable); -} - -static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure *closure) { - notify_on(exec_ctx, fd, closure, &fd->on_readable); -} - -static void destroy_fd_freelist(void) { - while (g_first_free_fd) { - grpc_fd *next = g_first_free_fd->next_free; - gpr_mu_destroy(&g_first_free_fd->polling_island_mu); - gpr_free(next); - g_first_free_fd = next; - } -} - -static void set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - gpr_atm *state) { - if (gpr_atm_acq_cas(state, STATE_NOT_READY, STATE_READY)) { - // state was not ready, and is now ready - we're done - } else { - // cas failed - either there's a closure queued which we should consume OR - // the state was already STATE_READY - gpr_atm cur_state = gpr_atm_acq_load(state); - if (cur_state != STATE_READY) { - // state wasn't STATE_READY - it *must* have been a closure - // since it's illegal to ask for notification twice, it's safe to assume - // that we'll resume being the closure - GPR_ASSERT(gpr_atm_rel_cas(state, cur_state, STATE_NOT_READY)); - grpc_exec_ctx_enqueue(exec_ctx, (grpc_closure *)cur_state, !fd->shutdown, - NULL); - } - } -} - -static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - gpr_mu_lock(&fd->set_ready_mu); - GPR_ASSERT(!fd->shutdown); - fd->shutdown = 1; - set_ready_locked(exec_ctx, fd, &fd->on_readable); - set_ready_locked(exec_ctx, fd, &fd->on_writable); - gpr_mu_unlock(&fd->set_ready_mu); -} - -//////////////////////////////////////////////////////////////////////////////// -// Pollset implementation - -static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { - gpr_mu_init(&pollset->mu); - *mu = &pollset->mu; - pollset->polling_island = NULL; -} - -static void pollset_destroy(grpc_pollset *pollset) { - gpr_mu_destroy(&pollset->mu); - if (pollset->polling_island) { - polling_island_drop(pollset->polling_island); - } -} - -static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - struct grpc_fd *fd) { - gpr_mu_lock(&pollset->mu); - gpr_mu_lock(&fd->polling_island_mu); - - polling_island *new; - - if (fd->polling_island == NULL) { - if (pollset->polling_island == NULL) { - new = polling_island_create(fd); - } else { - new = polling_island_add(pollset->polling_island, fd); - } - } else if (pollset->polling_island == NULL) { - new = polling_island_ref(fd->polling_island); - } else if (pollset->polling_island != fd->polling_island) { - new = polling_island_merge(pollset->polling_island, fd->polling_island); - } else { - new = polling_island_update(pollset->polling_island, 1); - } - - fd->polling_island = pollset->polling_island = new; - - gpr_mu_unlock(&fd->polling_island_mu); - gpr_mu_unlock(&pollset->mu); -} - -//////////////////////////////////////////////////////////////////////////////// -// Engine binding - -static void shutdown_engine(void) { destroy_fd_freelist(); } - -static const grpc_event_engine_vtable vtable = { - .pollset_size = sizeof(grpc_pollset), - - .fd_create = fd_create, - .fd_wrapped_fd = fd_wrapped_fd, - .fd_orphan = fd_orphan, - .fd_shutdown = fd_shutdown, - .fd_notify_on_read = fd_notify_on_read, - .fd_notify_on_write = fd_notify_on_write, - - .pollset_init = pollset_init, - .pollset_shutdown = pollset_shutdown, - .pollset_reset = pollset_reset, - .pollset_destroy = pollset_destroy, - .pollset_work = pollset_work, - .pollset_kick = pollset_kick, - .pollset_add_fd = pollset_add_fd, - - .pollset_set_create = pollset_set_create, - .pollset_set_destroy = pollset_set_destroy, - .pollset_set_add_pollset = pollset_set_add_pollset, - .pollset_set_del_pollset = pollset_set_del_pollset, - .pollset_set_add_pollset_set = pollset_set_add_pollset_set, - .pollset_set_del_pollset_set = pollset_set_del_pollset_set, - .pollset_set_add_fd = pollset_set_add_fd, - .pollset_set_del_fd = pollset_set_del_fd, - - .kick_poller = kick_poller, - - .shutdown_engine = shutdown_engine, -}; - -static bool is_epoll_available(void) { - abort(); - return false; -} - -const grpc_event_engine_vtable *grpc_init_poll_posix(void) { - if (!is_epoll_available()) { - return NULL; - } - return &vtable; -} diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 3aa26109f22..61106faef90 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -35,7 +35,7 @@ #ifdef GPR_POSIX_SOCKET -#include "src/core/lib/iomgr/ev_epoll_posix.h" +#include "src/core/lib/iomgr/ev_epoll_linux.h" #include #include diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c deleted file mode 100644 index 5abd5b2a94c..00000000000 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ /dev/null @@ -1,1209 +0,0 @@ -/* - * - * Copyright 2016, Google Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are - * met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following disclaimer - * in the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Google Inc. nor the names of its - * contributors may be used to endorse or promote products derived from - * this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - * - */ - -#include - -#ifdef GPR_POSIX_SOCKET - -#include "src/core/lib/iomgr/ev_epoll_posix.h" - -#include -#include -#include -#include -#include -#include -#include -#include - -#include -#include -#include -#include -#include - -#include "src/core/lib/iomgr/ev_posix.h" -#include "src/core/lib/iomgr/iomgr_internal.h" -#include "src/core/lib/iomgr/wakeup_fd_posix.h" -#include "src/core/lib/profiling/timers.h" -#include "src/core/lib/support/block_annotate.h" - - -/******************************************************************************* - * FD declarations - */ - -struct grpc_fd { - int fd; - /* refst format: - bit0: 1=active/0=orphaned - bit1-n: refcount - meaning that mostly we ref by two to avoid altering the orphaned bit, - and just unref by 1 when we're ready to flag the object as orphaned */ - gpr_atm refst; - - gpr_mu mu; - int shutdown; - int closed; - int released; - - grpc_closure *read_closure; - grpc_closure *write_closure; - - struct grpc_fd *freelist_next; - - grpc_closure *on_done_closure; - - grpc_iomgr_object iomgr_object; -}; - -/* Return 1 if this fd is orphaned, 0 otherwise */ -static bool fd_is_orphaned(grpc_fd *fd); - -/* Reference counting for fds */ -/*#define GRPC_FD_REF_COUNT_DEBUG*/ -#ifdef GRPC_FD_REF_COUNT_DEBUG -static void fd_ref(grpc_fd *fd, const char *reason, const char *file, int line); -static void fd_unref(grpc_fd *fd, const char *reason, const char *file, - int line); -#define GRPC_FD_REF(fd, reason) fd_ref(fd, reason, __FILE__, __LINE__) -#define GRPC_FD_UNREF(fd, reason) fd_unref(fd, reason, __FILE__, __LINE__) -#else -static void fd_ref(grpc_fd *fd); -static void fd_unref(grpc_fd *fd); -#define GRPC_FD_REF(fd, reason) fd_ref(fd) -#define GRPC_FD_UNREF(fd, reason) fd_unref(fd) -#endif - -static void fd_global_init(void); -static void fd_global_shutdown(void); - -#define CLOSURE_NOT_READY ((grpc_closure *)0) -#define CLOSURE_READY ((grpc_closure *)1) - -/******************************************************************************* - * pollset declarations - */ - -typedef struct grpc_cached_wakeup_fd { - grpc_wakeup_fd fd; - struct grpc_cached_wakeup_fd *next; -} grpc_cached_wakeup_fd; - -struct grpc_pollset_worker { - grpc_cached_wakeup_fd *wakeup_fd; - int reevaluate_polling_on_wakeup; - int kicked_specifically; - pthread_t pt_id; - struct grpc_pollset_worker *next; - struct grpc_pollset_worker *prev; -}; - -struct grpc_pollset { - gpr_mu mu; - grpc_pollset_worker root_worker; - int shutting_down; - int called_shutdown; - int kicked_without_pollers; - grpc_closure *shutdown_done; - - int epoll_fd; - - /* Local cache of eventfds for workers */ - grpc_cached_wakeup_fd *local_wakeup_cache; -}; - -/* Add an fd to a pollset */ -static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - struct grpc_fd *fd); - -static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, grpc_fd *fd); - -/* Convert a timespec to milliseconds: - - very small or negative poll times are clamped to zero to do a - non-blocking poll (which becomes spin polling) - - other small values are rounded up to one millisecond - - longer than a millisecond polls are rounded up to the next nearest - millisecond to avoid spinning - - infinite timeouts are converted to -1 */ -static int poll_deadline_to_millis_timeout(gpr_timespec deadline, - gpr_timespec now); - -/* Allow kick to wakeup the currently polling worker */ -#define GRPC_POLLSET_CAN_KICK_SELF 1 -/* Force the wakee to repoll when awoken */ -#define GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP 2 -/* As per pollset_kick, with an extended set of flags (defined above) - -- mostly for fd_posix's use. */ -static void pollset_kick_ext(grpc_pollset *p, - grpc_pollset_worker *specific_worker, - uint32_t flags); - -/* turn a pollset into a multipoller: platform specific */ -typedef void (*platform_become_multipoller_type)(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, - struct grpc_fd **fds, - size_t fd_count); - -/* Return 1 if the pollset has active threads in pollset_work (pollset must - * be locked) */ -static int pollset_has_workers(grpc_pollset *pollset); - -static void remove_fd_from_all_epoll_sets(int fd); - -/******************************************************************************* - * pollset_set definitions - */ - -struct grpc_pollset_set { - gpr_mu mu; - - size_t pollset_count; - size_t pollset_capacity; - grpc_pollset **pollsets; - - size_t pollset_set_count; - size_t pollset_set_capacity; - struct grpc_pollset_set **pollset_sets; - - size_t fd_count; - size_t fd_capacity; - grpc_fd **fds; -}; - -/******************************************************************************* - * fd_posix.c - */ - -/* We need to keep a freelist not because of any concerns of malloc performance - * but instead so that implementations with multiple threads in (for example) - * epoll_wait deal with the race between pollset removal and incoming poll - * notifications. - * - * The problem is that the poller ultimately holds a reference to this - * object, so it is very difficult to know when is safe to free it, at least - * without some expensive synchronization. - * - * If we keep the object freelisted, in the worst case losing this race just - * becomes a spurious read notification on a reused fd. - */ -/* TODO(klempner): We could use some form of polling generation count to know - * when these are safe to free. */ -/* TODO(klempner): Consider disabling freelisting if we don't have multiple - * threads in poll on the same fd */ -/* TODO(klempner): Batch these allocations to reduce fragmentation */ -static grpc_fd *fd_freelist = NULL; -static gpr_mu fd_freelist_mu; - -static void freelist_fd(grpc_fd *fd) { - gpr_mu_lock(&fd_freelist_mu); - fd->freelist_next = fd_freelist; - fd_freelist = fd; - grpc_iomgr_unregister_object(&fd->iomgr_object); - gpr_mu_unlock(&fd_freelist_mu); -} - -static grpc_fd *alloc_fd(int fd) { - grpc_fd *r = NULL; - gpr_mu_lock(&fd_freelist_mu); - if (fd_freelist != NULL) { - r = fd_freelist; - fd_freelist = fd_freelist->freelist_next; - } - gpr_mu_unlock(&fd_freelist_mu); - if (r == NULL) { - r = gpr_malloc(sizeof(grpc_fd)); - gpr_mu_init(&r->mu); - } - - gpr_mu_lock(&r->mu); - gpr_atm_rel_store(&r->refst, 1); - r->shutdown = 0; - r->read_closure = CLOSURE_NOT_READY; - r->write_closure = CLOSURE_NOT_READY; - r->fd = fd; - r->freelist_next = NULL; - r->on_done_closure = NULL; - r->closed = 0; - r->released = 0; - gpr_mu_unlock(&r->mu); - return r; -} - -static void destroy(grpc_fd *fd) { - gpr_mu_destroy(&fd->mu); - gpr_free(fd); -} - -#ifdef GRPC_FD_REF_COUNT_DEBUG -#define REF_BY(fd, n, reason) ref_by(fd, n, reason, __FILE__, __LINE__) -#define UNREF_BY(fd, n, reason) unref_by(fd, n, reason, __FILE__, __LINE__) -static void ref_by(grpc_fd *fd, int n, const char *reason, const char *file, - int line) { - gpr_log(GPR_DEBUG, "FD %d %p ref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, - gpr_atm_no_barrier_load(&fd->refst), - gpr_atm_no_barrier_load(&fd->refst) + n, reason, file, line); -#else -#define REF_BY(fd, n, reason) ref_by(fd, n) -#define UNREF_BY(fd, n, reason) unref_by(fd, n) -static void ref_by(grpc_fd *fd, int n) { -#endif - GPR_ASSERT(gpr_atm_no_barrier_fetch_add(&fd->refst, n) > 0); -} - -#ifdef GRPC_FD_REF_COUNT_DEBUG -static void unref_by(grpc_fd *fd, int n, const char *reason, const char *file, - int line) { - gpr_atm old; - gpr_log(GPR_DEBUG, "FD %d %p unref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, - gpr_atm_no_barrier_load(&fd->refst), - gpr_atm_no_barrier_load(&fd->refst) - n, reason, file, line); -#else -static void unref_by(grpc_fd *fd, int n) { - gpr_atm old; -#endif - old = gpr_atm_full_fetch_add(&fd->refst, -n); - if (old == n) { - freelist_fd(fd); - } else { - GPR_ASSERT(old > n); - } -} - -static void fd_global_init(void) { gpr_mu_init(&fd_freelist_mu); } - -static void fd_global_shutdown(void) { - gpr_mu_lock(&fd_freelist_mu); - gpr_mu_unlock(&fd_freelist_mu); - while (fd_freelist != NULL) { - grpc_fd *fd = fd_freelist; - fd_freelist = fd_freelist->freelist_next; - destroy(fd); - } - gpr_mu_destroy(&fd_freelist_mu); -} - -static grpc_fd *fd_create(int fd, const char *name) { - grpc_fd *r = alloc_fd(fd); - char *name2; - gpr_asprintf(&name2, "%s fd=%d", name, fd); - grpc_iomgr_register_object(&r->iomgr_object, name2); - gpr_free(name2); -#ifdef GRPC_FD_REF_COUNT_DEBUG - gpr_log(GPR_DEBUG, "FD %d %p create %s", fd, r, name); -#endif - return r; -} - -static bool fd_is_orphaned(grpc_fd *fd) { - return (gpr_atm_acq_load(&fd->refst) & 1) == 0; -} - -static void close_fd_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - fd->closed = 1; - if (!fd->released) { - close(fd->fd); - } else { - remove_fd_from_all_epoll_sets(fd->fd); - } - grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); -} - -static int fd_wrapped_fd(grpc_fd *fd) { - if (fd->released || fd->closed) { - return -1; - } else { - return fd->fd; - } -} - -static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure *on_done, int *release_fd, - const char *reason) { - fd->on_done_closure = on_done; - fd->released = release_fd != NULL; - if (!fd->released) { - shutdown(fd->fd, SHUT_RDWR); - } else { - *release_fd = fd->fd; - } - gpr_mu_lock(&fd->mu); - REF_BY(fd, 1, reason); /* remove active status, but keep referenced */ - close_fd_locked(exec_ctx, fd); - gpr_mu_unlock(&fd->mu); - UNREF_BY(fd, 2, reason); /* drop the reference */ -} - -/* increment refcount by two to avoid changing the orphan bit */ -#ifdef GRPC_FD_REF_COUNT_DEBUG -static void fd_ref(grpc_fd *fd, const char *reason, const char *file, - int line) { - ref_by(fd, 2, reason, file, line); -} - -static void fd_unref(grpc_fd *fd, const char *reason, const char *file, - int line) { - unref_by(fd, 2, reason, file, line); -} -#else -static void fd_ref(grpc_fd *fd) { ref_by(fd, 2); } - -static void fd_unref(grpc_fd *fd) { unref_by(fd, 2); } -#endif - -static void notify_on_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure **st, grpc_closure *closure) { - if (*st == CLOSURE_NOT_READY) { - /* not ready ==> switch to a waiting state by setting the closure */ - *st = closure; - } else if (*st == CLOSURE_READY) { - /* already ready ==> queue the closure to run immediately */ - *st = CLOSURE_NOT_READY; - grpc_exec_ctx_enqueue(exec_ctx, closure, !fd->shutdown, NULL); - } else { - /* upcallptr was set to a different closure. This is an error! */ - gpr_log(GPR_ERROR, - "User called a notify_on function with a previous callback still " - "pending"); - abort(); - } -} - -/* returns 1 if state becomes not ready */ -static int set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure **st) { - if (*st == CLOSURE_READY) { - /* duplicate ready ==> ignore */ - return 0; - } else if (*st == CLOSURE_NOT_READY) { - /* not ready, and not waiting ==> flag ready */ - *st = CLOSURE_READY; - return 0; - } else { - /* waiting ==> queue closure */ - grpc_exec_ctx_enqueue(exec_ctx, *st, !fd->shutdown, NULL); - *st = CLOSURE_NOT_READY; - return 1; - } -} - -static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - gpr_mu_lock(&fd->mu); - GPR_ASSERT(!fd->shutdown); - fd->shutdown = 1; - set_ready_locked(exec_ctx, fd, &fd->read_closure); - set_ready_locked(exec_ctx, fd, &fd->write_closure); - gpr_mu_unlock(&fd->mu); -} - -static void fd_notify_on_read(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure *closure) { - gpr_mu_lock(&fd->mu); - notify_on_locked(exec_ctx, fd, &fd->read_closure, closure); - gpr_mu_unlock(&fd->mu); -} - -static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure *closure) { - gpr_mu_lock(&fd->mu); - notify_on_locked(exec_ctx, fd, &fd->write_closure, closure); - gpr_mu_unlock(&fd->mu); -} - -/******************************************************************************* - * pollset_posix.c - */ - -GPR_TLS_DECL(g_current_thread_poller); -GPR_TLS_DECL(g_current_thread_worker); - -/** The alarm system needs to be able to wakeup 'some poller' sometimes - * (specifically when a new alarm needs to be triggered earlier than the next - * alarm 'epoch'). - * This wakeup_fd gives us something to alert on when such a case occurs. */ -grpc_wakeup_fd grpc_global_wakeup_fd; - -static void remove_worker(grpc_pollset *p, grpc_pollset_worker *worker) { - worker->prev->next = worker->next; - worker->next->prev = worker->prev; -} - -static int pollset_has_workers(grpc_pollset *p) { - return p->root_worker.next != &p->root_worker; -} - -static grpc_pollset_worker *pop_front_worker(grpc_pollset *p) { - if (pollset_has_workers(p)) { - grpc_pollset_worker *w = p->root_worker.next; - remove_worker(p, w); - return w; - } else { - return NULL; - } -} - -static void push_back_worker(grpc_pollset *p, grpc_pollset_worker *worker) { - worker->next = &p->root_worker; - worker->prev = worker->next->prev; - worker->prev->next = worker->next->prev = worker; -} - -static void push_front_worker(grpc_pollset *p, grpc_pollset_worker *worker) { - worker->prev = &p->root_worker; - worker->next = worker->prev->next; - worker->prev->next = worker->next->prev = worker; -} - -static void pollset_kick_ext(grpc_pollset *p, - grpc_pollset_worker *specific_worker, - uint32_t flags) { - GPR_TIMER_BEGIN("pollset_kick_ext", 0); - - /* pollset->mu already held */ - if (specific_worker != NULL) { - if (specific_worker == GRPC_POLLSET_KICK_BROADCAST) { - GPR_TIMER_BEGIN("pollset_kick_ext.broadcast", 0); - GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); - for (specific_worker = p->root_worker.next; - specific_worker != &p->root_worker; - specific_worker = specific_worker->next) { - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); - } - p->kicked_without_pollers = 1; - GPR_TIMER_END("pollset_kick_ext.broadcast", 0); - } else if (gpr_tls_get(&g_current_thread_worker) != - (intptr_t)specific_worker) { - GPR_TIMER_MARK("different_thread_worker", 0); - if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { - specific_worker->reevaluate_polling_on_wakeup = 1; - } - specific_worker->kicked_specifically = 1; - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); - /* TODO (sreek): Refactor this into a separate file*/ - pthread_kill(specific_worker->pt_id, SIGUSR1); - } else if ((flags & GRPC_POLLSET_CAN_KICK_SELF) != 0) { - GPR_TIMER_MARK("kick_yoself", 0); - if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { - specific_worker->reevaluate_polling_on_wakeup = 1; - } - specific_worker->kicked_specifically = 1; - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); - } - } else if (gpr_tls_get(&g_current_thread_poller) != (intptr_t)p) { - GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); - GPR_TIMER_MARK("kick_anonymous", 0); - specific_worker = pop_front_worker(p); - if (specific_worker != NULL) { - if (gpr_tls_get(&g_current_thread_worker) == (intptr_t)specific_worker) { - GPR_TIMER_MARK("kick_anonymous_not_self", 0); - push_back_worker(p, specific_worker); - specific_worker = pop_front_worker(p); - if ((flags & GRPC_POLLSET_CAN_KICK_SELF) == 0 && - gpr_tls_get(&g_current_thread_worker) == - (intptr_t)specific_worker) { - push_back_worker(p, specific_worker); - specific_worker = NULL; - } - } - if (specific_worker != NULL) { - GPR_TIMER_MARK("finally_kick", 0); - push_back_worker(p, specific_worker); - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); - } - } else { - GPR_TIMER_MARK("kicked_no_pollers", 0); - p->kicked_without_pollers = 1; - } - } - - GPR_TIMER_END("pollset_kick_ext", 0); -} - -static void pollset_kick(grpc_pollset *p, - grpc_pollset_worker *specific_worker) { - pollset_kick_ext(p, specific_worker, 0); -} - -/* global state management */ - -static void sig_handler(int sig_num) { - gpr_log(GPR_INFO, "Received signal %d", sig_num); -} - -static void pollset_global_init(void) { - gpr_tls_init(&g_current_thread_poller); - gpr_tls_init(&g_current_thread_worker); - grpc_wakeup_fd_init(&grpc_global_wakeup_fd); - signal(SIGUSR1, sig_handler); -} - -static void pollset_global_shutdown(void) { - grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); - gpr_tls_destroy(&g_current_thread_poller); - gpr_tls_destroy(&g_current_thread_worker); -} - -static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } - -/* TODO: sreek. Try to Remove this forward declaration*/ -static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset); - -/* main interface */ - -static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { - gpr_mu_init(&pollset->mu); - *mu = &pollset->mu; - pollset->root_worker.next = pollset->root_worker.prev = &pollset->root_worker; - pollset->shutting_down = 0; - pollset->called_shutdown = 0; - pollset->kicked_without_pollers = 0; - pollset->local_wakeup_cache = NULL; - pollset->kicked_without_pollers = 0; - - multipoll_with_epoll_pollset_create_efd(pollset); -} - -/* TODO(sreek): Maybe merge multipoll_*_destroy() with pollset_destroy() - * function */ -static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset); - -static void pollset_destroy(grpc_pollset *pollset) { - GPR_ASSERT(!pollset_has_workers(pollset)); - - multipoll_with_epoll_pollset_destroy(pollset); - - while (pollset->local_wakeup_cache) { - grpc_cached_wakeup_fd *next = pollset->local_wakeup_cache->next; - grpc_wakeup_fd_destroy(&pollset->local_wakeup_cache->fd); - gpr_free(pollset->local_wakeup_cache); - pollset->local_wakeup_cache = next; - } - gpr_mu_destroy(&pollset->mu); -} - -static void pollset_reset(grpc_pollset *pollset) { - GPR_ASSERT(pollset->shutting_down); - GPR_ASSERT(!pollset_has_workers(pollset)); - pollset->shutting_down = 0; - pollset->called_shutdown = 0; - pollset->kicked_without_pollers = 0; -} - -/* TODO (sreek): Remove multipoll_with_epoll_finish_shutdown() declaration */ -static void multipoll_with_epoll_pollset_finish_shutdown(grpc_pollset *pollset); - -static void finish_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { - multipoll_with_epoll_pollset_finish_shutdown(pollset); - grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); -} - -/* TODO(sreek): Remove multipoll_with_epoll_*_maybe_work_and_unlock declaration - */ -static void multipoll_with_epoll_pollset_maybe_work_and_unlock( - grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, - gpr_timespec deadline, gpr_timespec now); - -static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_pollset_worker **worker_hdl, gpr_timespec now, - gpr_timespec deadline) { - grpc_pollset_worker worker; - *worker_hdl = &worker; - - /* pollset->mu already held */ - int added_worker = 0; - int locked = 1; - int queued_work = 0; - int keep_polling = 0; - GPR_TIMER_BEGIN("pollset_work", 0); - /* this must happen before we (potentially) drop pollset->mu */ - worker.next = worker.prev = NULL; - worker.reevaluate_polling_on_wakeup = 0; - if (pollset->local_wakeup_cache != NULL) { - worker.wakeup_fd = pollset->local_wakeup_cache; - pollset->local_wakeup_cache = worker.wakeup_fd->next; - } else { - worker.wakeup_fd = gpr_malloc(sizeof(*worker.wakeup_fd)); - grpc_wakeup_fd_init(&worker.wakeup_fd->fd); - } - worker.kicked_specifically = 0; - - /* TODO(sreek): Abstract this thread id stuff out into a separate file */ - worker.pt_id = pthread_self(); - /* If we're shutting down then we don't execute any extended work */ - if (pollset->shutting_down) { - GPR_TIMER_MARK("pollset_work.shutting_down", 0); - goto done; - } - /* Start polling, and keep doing so while we're being asked to - re-evaluate our pollers (this allows poll() based pollers to - ensure they don't miss wakeups) */ - keep_polling = 1; - while (keep_polling) { - keep_polling = 0; - if (!pollset->kicked_without_pollers) { - if (!added_worker) { - push_front_worker(pollset, &worker); - added_worker = 1; - gpr_tls_set(&g_current_thread_worker, (intptr_t)&worker); - } - gpr_tls_set(&g_current_thread_poller, (intptr_t)pollset); - GPR_TIMER_BEGIN("maybe_work_and_unlock", 0); - - multipoll_with_epoll_pollset_maybe_work_and_unlock( - exec_ctx, pollset, &worker, deadline, now); - - GPR_TIMER_END("maybe_work_and_unlock", 0); - locked = 0; - gpr_tls_set(&g_current_thread_poller, 0); - } else { - GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); - pollset->kicked_without_pollers = 0; - } - /* Finished execution - start cleaning up. - Note that we may arrive here from outside the enclosing while() loop. - In that case we won't loop though as we haven't added worker to the - worker list, which means nobody could ask us to re-evaluate polling). */ - done: - if (!locked) { - queued_work |= grpc_exec_ctx_flush(exec_ctx); - gpr_mu_lock(&pollset->mu); - locked = 1; - } - /* If we're forced to re-evaluate polling (via pollset_kick with - GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) then we land here and force - a loop */ - if (worker.reevaluate_polling_on_wakeup) { - worker.reevaluate_polling_on_wakeup = 0; - pollset->kicked_without_pollers = 0; - if (queued_work || worker.kicked_specifically) { - /* If there's queued work on the list, then set the deadline to be - immediate so we get back out of the polling loop quickly */ - deadline = gpr_inf_past(GPR_CLOCK_MONOTONIC); - } - keep_polling = 1; - } - } - if (added_worker) { - remove_worker(pollset, &worker); - gpr_tls_set(&g_current_thread_worker, 0); - } - /* release wakeup fd to the local pool */ - worker.wakeup_fd->next = pollset->local_wakeup_cache; - pollset->local_wakeup_cache = worker.wakeup_fd; - /* check shutdown conditions */ - if (pollset->shutting_down) { - if (pollset_has_workers(pollset)) { - pollset_kick(pollset, NULL); - } else if (!pollset->called_shutdown) { - pollset->called_shutdown = 1; - gpr_mu_unlock(&pollset->mu); - finish_shutdown(exec_ctx, pollset); - grpc_exec_ctx_flush(exec_ctx); - /* Continuing to access pollset here is safe -- it is the caller's - * responsibility to not destroy when it has outstanding calls to - * pollset_work. - * TODO(dklempner): Can we refactor the shutdown logic to avoid this? */ - gpr_mu_lock(&pollset->mu); - } - } - *worker_hdl = NULL; - GPR_TIMER_END("pollset_work", 0); -} - -static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_closure *closure) { - GPR_ASSERT(!pollset->shutting_down); - pollset->shutting_down = 1; - pollset->shutdown_done = closure; - pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); - - if (!pollset->called_shutdown && !pollset_has_workers(pollset)) { - pollset->called_shutdown = 1; - finish_shutdown(exec_ctx, pollset); - } -} - -static int poll_deadline_to_millis_timeout(gpr_timespec deadline, - gpr_timespec now) { - gpr_timespec timeout; - static const int64_t max_spin_polling_us = 10; - if (gpr_time_cmp(deadline, gpr_inf_future(deadline.clock_type)) == 0) { - return -1; - } - if (gpr_time_cmp(deadline, gpr_time_add(now, gpr_time_from_micros( - max_spin_polling_us, - GPR_TIMESPAN))) <= 0) { - return 0; - } - timeout = gpr_time_sub(deadline, now); - return gpr_time_to_millis(gpr_time_add( - timeout, gpr_time_from_nanos(GPR_NS_PER_MS - 1, GPR_TIMESPAN))); -} - -/******************************************************************************* - * pollset_multipoller_with_epoll_posix.c - */ - -static void set_ready(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st) { - /* only one set_ready can be active at once (but there may be a racing - notify_on) */ - gpr_mu_lock(&fd->mu); - set_ready_locked(exec_ctx, fd, st); - gpr_mu_unlock(&fd->mu); -} - -static void fd_become_readable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - set_ready(exec_ctx, fd, &fd->read_closure); -} - -static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - set_ready(exec_ctx, fd, &fd->write_closure); -} - -/* TODO (sreek): Maybe this global list is not required. Double check*/ -struct epoll_fd_list { - int *epoll_fds; - size_t count; - size_t capacity; -}; - -static struct epoll_fd_list epoll_fd_global_list; -static gpr_once init_epoll_fd_list_mu = GPR_ONCE_INIT; -static gpr_mu epoll_fd_list_mu; - -static void init_mu(void) { gpr_mu_init(&epoll_fd_list_mu); } - -static void add_epoll_fd_to_global_list(int epoll_fd) { - gpr_once_init(&init_epoll_fd_list_mu, init_mu); - - gpr_mu_lock(&epoll_fd_list_mu); - if (epoll_fd_global_list.count == epoll_fd_global_list.capacity) { - epoll_fd_global_list.capacity = - GPR_MAX((size_t)8, epoll_fd_global_list.capacity * 2); - epoll_fd_global_list.epoll_fds = - gpr_realloc(epoll_fd_global_list.epoll_fds, - epoll_fd_global_list.capacity * sizeof(int)); - } - epoll_fd_global_list.epoll_fds[epoll_fd_global_list.count++] = epoll_fd; - gpr_mu_unlock(&epoll_fd_list_mu); -} - -static void remove_epoll_fd_from_global_list(int epoll_fd) { - gpr_mu_lock(&epoll_fd_list_mu); - GPR_ASSERT(epoll_fd_global_list.count > 0); - for (size_t i = 0; i < epoll_fd_global_list.count; i++) { - if (epoll_fd == epoll_fd_global_list.epoll_fds[i]) { - epoll_fd_global_list.epoll_fds[i] = - epoll_fd_global_list.epoll_fds[--(epoll_fd_global_list.count)]; - break; - } - } - gpr_mu_unlock(&epoll_fd_list_mu); -} - -static void remove_fd_from_all_epoll_sets(int fd) { - int err; - gpr_once_init(&init_epoll_fd_list_mu, init_mu); - gpr_mu_lock(&epoll_fd_list_mu); - if (epoll_fd_global_list.count == 0) { - gpr_mu_unlock(&epoll_fd_list_mu); - return; - } - for (size_t i = 0; i < epoll_fd_global_list.count; i++) { - err = epoll_ctl(epoll_fd_global_list.epoll_fds[i], EPOLL_CTL_DEL, fd, NULL); - if (err < 0 && errno != ENOENT) { - gpr_log(GPR_ERROR, "epoll_ctl del for %d failed: %s", fd, - strerror(errno)); - } - } - gpr_mu_unlock(&epoll_fd_list_mu); -} - -/* TODO: sreek - This function multipoll_with_epoll_pollset_add_fd() and - * finally_add_fd() in ev_poll_and_epoll_posix.c */ -static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_fd *fd) { - - /* TODO sreek - Check if we need to get a pollset->mu lock here */ - - struct epoll_event ev; - int err; - - /* Hold a ref to the fd to keep it from being closed during the add. This may - result in a spurious wakeup being assigned to this pollset whilst adding, - but that should be benign. */ - /* TODO: (sreek): Understand how a spurious wake up migh be assinged to this - * pollset..and how holding a reference will prevent the fd from being closed - * (and perhaps more importantly, see how can an fd be closed while being - * added to the epollset */ - GRPC_FD_REF(fd, "add fd"); - - gpr_mu_lock(&fd->mu); - if (fd->shutdown) { - gpr_mu_unlock(&fd->mu); - GRPC_FD_UNREF(fd, "add fd"); - return; - } - gpr_mu_unlock(&fd->mu); - - ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); - ev.data.ptr = fd; - err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); - if (err < 0) { - /* FDs may be added to a pollset multiple times, so EEXIST is normal. */ - if (errno != EEXIST) { - gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", fd->fd, - strerror(errno)); - } - } - - /* The fd might have been orphaned while we were adding it to the epoll set. - Close the fd in such a case (which will also take care of removing it from - the epoll set */ - gpr_mu_lock(&fd->mu); - if (fd_is_orphaned(fd) && !fd->closed) { - close_fd_locked(exec_ctx, fd); - } - gpr_mu_unlock(&fd->mu); - - GRPC_FD_UNREF(fd, "add fd"); -} - -/* Creates an epoll fd and initializes the pollset */ -/* TODO: This has to be called ONLY from pollset_init function. and hence it - * does not acquire any lock */ -static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { - struct epoll_event ev; - int err; - - pollset->epoll_fd = epoll_create1(EPOLL_CLOEXEC); - if (pollset->epoll_fd < 0) { - gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); - abort(); - } - add_epoll_fd_to_global_list(pollset->epoll_fd); - - ev.events = (uint32_t)(EPOLLIN | EPOLLET); - ev.data.ptr = NULL; - - err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); - if (err < 0) { - gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), - strerror(errno)); - } -} - -/* TODO(klempner): We probably want to turn this down a bit */ -#define GRPC_EPOLL_MAX_EVENTS 1000 - -static void multipoll_with_epoll_pollset_maybe_work_and_unlock( - grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, - gpr_timespec deadline, gpr_timespec now) { - struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; - int epoll_fd = pollset->epoll_fd; - int ep_rv; - int poll_rv; - int timeout_ms; - struct pollfd pfds[2]; - - /* If you want to ignore epoll's ability to sanely handle parallel pollers, - * for a more apples-to-apples performance comparison with poll, add a - * if (pollset->counter != 0) { return 0; } - * here. - */ - - gpr_mu_unlock(&pollset->mu); - - timeout_ms = poll_deadline_to_millis_timeout(deadline, now); - - pfds[0].fd = GRPC_WAKEUP_FD_GET_READ_FD(&worker->wakeup_fd->fd); - pfds[0].events = POLLIN; - pfds[0].revents = 0; - pfds[1].fd = epoll_fd; - pfds[1].events = POLLIN; - pfds[1].revents = 0; - - /* TODO(vpai): Consider first doing a 0 timeout poll here to avoid - even going into the blocking annotation if possible */ - GPR_TIMER_BEGIN("poll", 0); - GRPC_SCHEDULING_START_BLOCKING_REGION; - poll_rv = grpc_poll_function(pfds, 2, timeout_ms); - GRPC_SCHEDULING_END_BLOCKING_REGION; - GPR_TIMER_END("poll", 0); - - if (poll_rv < 0) { - if (errno != EINTR) { - gpr_log(GPR_ERROR, "poll() failed: %s", strerror(errno)); - } - } else if (poll_rv == 0) { - /* do nothing */ - } else { - if (pfds[0].revents) { - grpc_wakeup_fd_consume_wakeup(&worker->wakeup_fd->fd); - } - if (pfds[1].revents) { - do { - /* The following epoll_wait never blocks; it has a timeout of 0 */ - ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); - if (ep_rv < 0) { - if (errno != EINTR) { - gpr_log(GPR_ERROR, "epoll_wait() failed: %s", strerror(errno)); - } - } else { - int i; - for (i = 0; i < ep_rv; ++i) { - grpc_fd *fd = ep_ev[i].data.ptr; - /* TODO(klempner): We might want to consider making err and pri - * separate events */ - int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); - int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); - int write_ev = ep_ev[i].events & EPOLLOUT; - if (fd == NULL) { - grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); - } else { - if (read_ev || cancel) { - fd_become_readable(exec_ctx, fd); - } - if (write_ev || cancel) { - fd_become_writable(exec_ctx, fd); - } - } - } - } - } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); - } - } -} - -static void multipoll_with_epoll_pollset_finish_shutdown( - grpc_pollset *pollset) {} - -static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset) { - close(pollset->epoll_fd); - remove_epoll_fd_from_global_list(pollset->epoll_fd); -} - -/******************************************************************************* - * pollset_set_posix.c - */ - -static grpc_pollset_set *pollset_set_create(void) { - grpc_pollset_set *pollset_set = gpr_malloc(sizeof(*pollset_set)); - memset(pollset_set, 0, sizeof(*pollset_set)); - gpr_mu_init(&pollset_set->mu); - return pollset_set; -} - -static void pollset_set_destroy(grpc_pollset_set *pollset_set) { - size_t i; - gpr_mu_destroy(&pollset_set->mu); - for (i = 0; i < pollset_set->fd_count; i++) { - GRPC_FD_UNREF(pollset_set->fds[i], "pollset_set"); - } - gpr_free(pollset_set->pollsets); - gpr_free(pollset_set->pollset_sets); - gpr_free(pollset_set->fds); - gpr_free(pollset_set); -} - -static void pollset_set_add_pollset(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, - grpc_pollset *pollset) { - size_t i, j; - gpr_mu_lock(&pollset_set->mu); - if (pollset_set->pollset_count == pollset_set->pollset_capacity) { - pollset_set->pollset_capacity = - GPR_MAX(8, 2 * pollset_set->pollset_capacity); - pollset_set->pollsets = - gpr_realloc(pollset_set->pollsets, pollset_set->pollset_capacity * - sizeof(*pollset_set->pollsets)); - } - pollset_set->pollsets[pollset_set->pollset_count++] = pollset; - for (i = 0, j = 0; i < pollset_set->fd_count; i++) { - if (fd_is_orphaned(pollset_set->fds[i])) { - GRPC_FD_UNREF(pollset_set->fds[i], "pollset_set"); - } else { - pollset_add_fd(exec_ctx, pollset, pollset_set->fds[i]); - pollset_set->fds[j++] = pollset_set->fds[i]; - } - } - pollset_set->fd_count = j; - gpr_mu_unlock(&pollset_set->mu); -} - -static void pollset_set_del_pollset(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, - grpc_pollset *pollset) { - size_t i; - gpr_mu_lock(&pollset_set->mu); - for (i = 0; i < pollset_set->pollset_count; i++) { - if (pollset_set->pollsets[i] == pollset) { - pollset_set->pollset_count--; - GPR_SWAP(grpc_pollset *, pollset_set->pollsets[i], - pollset_set->pollsets[pollset_set->pollset_count]); - break; - } - } - gpr_mu_unlock(&pollset_set->mu); -} - -static void pollset_set_add_pollset_set(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *bag, - grpc_pollset_set *item) { - size_t i, j; - gpr_mu_lock(&bag->mu); - if (bag->pollset_set_count == bag->pollset_set_capacity) { - bag->pollset_set_capacity = GPR_MAX(8, 2 * bag->pollset_set_capacity); - bag->pollset_sets = - gpr_realloc(bag->pollset_sets, - bag->pollset_set_capacity * sizeof(*bag->pollset_sets)); - } - bag->pollset_sets[bag->pollset_set_count++] = item; - for (i = 0, j = 0; i < bag->fd_count; i++) { - if (fd_is_orphaned(bag->fds[i])) { - GRPC_FD_UNREF(bag->fds[i], "pollset_set"); - } else { - pollset_set_add_fd(exec_ctx, item, bag->fds[i]); - bag->fds[j++] = bag->fds[i]; - } - } - bag->fd_count = j; - gpr_mu_unlock(&bag->mu); -} - -static void pollset_set_del_pollset_set(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *bag, - grpc_pollset_set *item) { - size_t i; - gpr_mu_lock(&bag->mu); - for (i = 0; i < bag->pollset_set_count; i++) { - if (bag->pollset_sets[i] == item) { - bag->pollset_set_count--; - GPR_SWAP(grpc_pollset_set *, bag->pollset_sets[i], - bag->pollset_sets[bag->pollset_set_count]); - break; - } - } - gpr_mu_unlock(&bag->mu); -} - -static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, grpc_fd *fd) { - size_t i; - gpr_mu_lock(&pollset_set->mu); - if (pollset_set->fd_count == pollset_set->fd_capacity) { - pollset_set->fd_capacity = GPR_MAX(8, 2 * pollset_set->fd_capacity); - pollset_set->fds = gpr_realloc( - pollset_set->fds, pollset_set->fd_capacity * sizeof(*pollset_set->fds)); - } - GRPC_FD_REF(fd, "pollset_set"); - pollset_set->fds[pollset_set->fd_count++] = fd; - for (i = 0; i < pollset_set->pollset_count; i++) { - pollset_add_fd(exec_ctx, pollset_set->pollsets[i], fd); - } - for (i = 0; i < pollset_set->pollset_set_count; i++) { - pollset_set_add_fd(exec_ctx, pollset_set->pollset_sets[i], fd); - } - gpr_mu_unlock(&pollset_set->mu); -} - -static void pollset_set_del_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, grpc_fd *fd) { - size_t i; - gpr_mu_lock(&pollset_set->mu); - for (i = 0; i < pollset_set->fd_count; i++) { - if (pollset_set->fds[i] == fd) { - pollset_set->fd_count--; - GPR_SWAP(grpc_fd *, pollset_set->fds[i], - pollset_set->fds[pollset_set->fd_count]); - GRPC_FD_UNREF(fd, "pollset_set"); - break; - } - } - for (i = 0; i < pollset_set->pollset_set_count; i++) { - pollset_set_del_fd(exec_ctx, pollset_set->pollset_sets[i], fd); - } - gpr_mu_unlock(&pollset_set->mu); -} - -/******************************************************************************* - * event engine binding - */ - -static void shutdown_engine(void) { - fd_global_shutdown(); - pollset_global_shutdown(); -} - -static const grpc_event_engine_vtable vtable = { - .pollset_size = sizeof(grpc_pollset), - - .fd_create = fd_create, - .fd_wrapped_fd = fd_wrapped_fd, - .fd_orphan = fd_orphan, - .fd_shutdown = fd_shutdown, - .fd_notify_on_read = fd_notify_on_read, - .fd_notify_on_write = fd_notify_on_write, - - .pollset_init = pollset_init, - .pollset_shutdown = pollset_shutdown, - .pollset_reset = pollset_reset, - .pollset_destroy = pollset_destroy, - .pollset_work = pollset_work, - .pollset_kick = pollset_kick, - .pollset_add_fd = pollset_add_fd, - - .pollset_set_create = pollset_set_create, - .pollset_set_destroy = pollset_set_destroy, - .pollset_set_add_pollset = pollset_set_add_pollset, - .pollset_set_del_pollset = pollset_set_del_pollset, - .pollset_set_add_pollset_set = pollset_set_add_pollset_set, - .pollset_set_del_pollset_set = pollset_set_del_pollset_set, - .pollset_set_add_fd = pollset_set_add_fd, - .pollset_set_del_fd = pollset_set_del_fd, - - .kick_poller = kick_poller, - - .shutdown_engine = shutdown_engine, -}; - -const grpc_event_engine_vtable *grpc_init_epoll_posix(void) { - fd_global_init(); - pollset_global_init(); - return &vtable; -} - -#endif diff --git a/src/core/lib/iomgr/ev_epoll_posix.h b/src/core/lib/iomgr/ev_epoll_posix.h deleted file mode 100644 index 35319b4fc5d..00000000000 --- a/src/core/lib/iomgr/ev_epoll_posix.h +++ /dev/null @@ -1,41 +0,0 @@ -/* - * - * Copyright 2015, Google Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are - * met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following disclaimer - * in the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Google Inc. nor the names of its - * contributors may be used to endorse or promote products derived from - * this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - * - */ - -#ifndef GRPC_CORE_LIB_IOMGR_EV_EPOLL_POSIX_H -#define GRPC_CORE_LIB_IOMGR_EV_EPOLL_POSIX_H - -#include "src/core/lib/iomgr/ev_posix.h" - -const grpc_event_engine_vtable *grpc_init_epoll_posix(void); - -#endif /* GRPC_CORE_LIB_IOMGR_EV_EPOLL_POSIX_H */ diff --git a/src/python/grpcio/grpc_core_dependencies.py b/src/python/grpcio/grpc_core_dependencies.py index 13bc6888d66..50a6f196d8d 100644 --- a/src/python/grpcio/grpc_core_dependencies.py +++ b/src/python/grpcio/grpc_core_dependencies.py @@ -95,7 +95,6 @@ CORE_SOURCE_FILES = [ 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', 'src/core/lib/iomgr/ev_epoll_linux.c', - 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', 'src/core/lib/iomgr/exec_ctx.c', diff --git a/test/core/network_benchmarks/epoll_test.c b/test/core/network_benchmarks/epoll_test.c deleted file mode 100644 index a918dd9bb94..00000000000 --- a/test/core/network_benchmarks/epoll_test.c +++ /dev/null @@ -1,263 +0,0 @@ -/* - * - * Copyright 2015, Google Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are - * met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following disclaimer - * in the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Google Inc. nor the names of its - * contributors may be used to endorse or promote products derived from - * this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - * - */ - -/* TODO: sreek: REMOVE THIS FILE */ - -#include - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include -#include -#include -#include - -int g_signal_num = SIGUSR1; - -int g_timeout_secs = 2; - -int g_eventfd_create = 1; -int g_eventfd_wakeup = 0; -int g_eventfd_teardown = 0; -int g_close_epoll_fd = 1; - -typedef struct thread_args { - gpr_thd_id id; - int epoll_fd; - int thread_num; -} thread_args; - -static int eventfd_create() { - if (!g_eventfd_create) { - return -1; - } - - int efd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); - GPR_ASSERT(efd >= 0); - return efd; -} - -static void eventfd_wakeup(int efd) { - if (!g_eventfd_wakeup) { - return; - } - - int err; - do { - err = eventfd_write(efd, 1); - } while (err < 0 && errno == EINTR); -} - -static void epoll_teardown(int epoll_fd, int fd) { - if (!g_eventfd_teardown) { - return; - } - - if (epoll_ctl(epoll_fd, EPOLL_CTL_DEL, fd, NULL) < 0) { - if (errno != ENOENT) { - gpr_log(GPR_ERROR, "epoll_ctl: %s", strerror(errno)); - GPR_ASSERT(0); - } - } -} - -/* Special case for epoll, where we need to create the fd ahead of time. */ -static int epoll_setup(int fd) { - int epoll_fd; - struct epoll_event ev; - - epoll_fd = epoll_create(1); - if (epoll_fd < 0) { - gpr_log(GPR_ERROR, "epoll_create: %s", strerror(errno)); - return -1; - } - - ev.events = (uint32_t)EPOLLIN; - ev.data.fd = fd; - if (epoll_ctl(epoll_fd, EPOLL_CTL_ADD, fd, &ev) < 0) { - if (errno != EEXIST) { - gpr_log(GPR_ERROR, "epoll_ctl: %s", strerror(errno)); - return -1; - } - - gpr_log(GPR_ERROR, "epoll_ctl: The fd %d already exists", fd); - } - - return epoll_fd; -} - -#define GRPC_EPOLL_MAX_EVENTS 1000 -static void thread_main(void *args) { - int ep_rv; - struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; - int fd; - int i; - int cancel; - int read; - int write; - thread_args *thd_args = args; - sigset_t new_mask; - sigset_t orig_mask; - int keep_polling = 0; - - gpr_log(GPR_INFO, "Thread: %d Started", thd_args->thread_num); - - do { - keep_polling = 0; - - /* Mask the signal before getting the epoll_fd */ - gpr_log(GPR_INFO, "Thread: %d Blocking signal: %d", thd_args->thread_num, - g_signal_num); - sigemptyset(&new_mask); - sigaddset(&new_mask, g_signal_num); - pthread_sigmask(SIG_BLOCK, &new_mask, &orig_mask); - - gpr_log(GPR_INFO, "Thread: %d Waiting on epoll_wait()", - thd_args->thread_num); - ep_rv = epoll_pwait(thd_args->epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, - g_timeout_secs * 5000, &orig_mask); - gpr_log(GPR_INFO, "Thread: %d out of epoll_wait. ep_rv = %d", - thd_args->thread_num, ep_rv); - - if (ep_rv < 0) { - if (errno != EINTR) { - gpr_log(GPR_ERROR, "Thread: %d. epoll_wait failed with error: %d", - thd_args->thread_num, errno); - } else { - gpr_log(GPR_INFO, - "Thread: %d. epoll_wait was interrupted. Polling again >>>>>>>", - thd_args->thread_num); - keep_polling = 1; - } - } else { - if (ep_rv == 0) { - gpr_log(GPR_INFO, - "Thread: %d - epoll_wait returned 0. Most likely a timeout. " - "Polling again", - thd_args->thread_num); - keep_polling = 1; - } - - for (i = 0; i < ep_rv; i++) { - fd = ep_ev[i].data.fd; - cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); - read = ep_ev[i].events & (EPOLLIN | EPOLLPRI); - write = ep_ev[i].events & EPOLLOUT; - gpr_log(GPR_INFO, - "Thread: %d. epoll_wait returned that fd: %d has event of " - "interest. read: %d, write: %d, cancel: %d", - thd_args->thread_num, fd, read, write, cancel); - } - } - } while (keep_polling); -} - -static void close_fd(int fd) { - if (!g_close_epoll_fd) { - return; - } - - gpr_log(GPR_INFO, "*** Closing fd : %d ****", fd); - close(fd); - gpr_log(GPR_INFO, "*** Closed fd : %d ****", fd); -} - - -static void sig_handler(int sig_num) { - gpr_log(GPR_INFO, "<<<<< Received signal %d", sig_num); -} - -static void set_signal_handler() { - gpr_log(GPR_INFO, "Setting signal handler"); - signal(g_signal_num, sig_handler); -} - -#define NUM_THREADS 2 -int main(int argc, char **argv) { - int efd; - int epoll_fd; - int i; - thread_args thd_args[NUM_THREADS]; - gpr_thd_options options = gpr_thd_options_default(); - - set_signal_handler(); - - gpr_log(GPR_INFO, "Starting.."); - efd = eventfd_create(); - gpr_log(GPR_INFO, "Created event fd: %d", efd); - epoll_fd = epoll_setup(efd); - gpr_log(GPR_INFO, "Created epoll_fd: %d", epoll_fd); - - gpr_thd_options_set_joinable(&options); - for (i = 0; i < NUM_THREADS; i++) { - thd_args[i].thread_num = i; - thd_args[i].epoll_fd = epoll_fd; - gpr_log(GPR_INFO, "Starting thread: %d", i); - gpr_thd_new(&thd_args[i].id, thread_main, &thd_args[i], &options); - } - - sleep((unsigned)g_timeout_secs * 2); - - /* Send signals first */ - for (i = 0; i < NUM_THREADS; i++) { - gpr_log(GPR_INFO, "Sending signal to thread: %d", thd_args->thread_num); - pthread_kill(thd_args[i].id, g_signal_num); - gpr_log(GPR_INFO, "Sent signal to thread: %d >>>>>> ", - thd_args->thread_num); - } - - sleep((unsigned)g_timeout_secs * 2); - - close_fd(epoll_fd); - - sleep((unsigned)g_timeout_secs * 2); - - eventfd_wakeup(efd); - epoll_teardown(epoll_fd, efd); - - for (i = 0; i < NUM_THREADS; i++) { - gpr_thd_join(thd_args[i].id); - gpr_log(GPR_INFO, "Thread: %d joined", i); - } - - return 0; -} diff --git a/test/core/network_benchmarks/low_level_ping_pong.c b/test/core/network_benchmarks/low_level_ping_pong.c index b72a07778e0..1b40895a719 100644 --- a/test/core/network_benchmarks/low_level_ping_pong.c +++ b/test/core/network_benchmarks/low_level_ping_pong.c @@ -44,7 +44,6 @@ #include #include #include -#include #ifdef __linux__ #include #endif @@ -85,7 +84,6 @@ typedef struct thread_args { static int read_bytes(int fd, char *buf, size_t read_size, int spin) { size_t bytes_read = 0; ssize_t err; - do { err = read(fd, buf + bytes_read, read_size - bytes_read); if (err < 0) { diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal index d968278f2a4..21ee1e6ff8a 100644 --- a/tools/doxygen/Doxyfile.core.internal +++ b/tools/doxygen/Doxyfile.core.internal @@ -808,7 +808,6 @@ src/core/lib/iomgr/closure.h \ src/core/lib/iomgr/endpoint.h \ src/core/lib/iomgr/endpoint_pair.h \ src/core/lib/iomgr/ev_epoll_linux.h \ -src/core/lib/iomgr/ev_epoll_posix.h \ src/core/lib/iomgr/ev_poll_posix.h \ src/core/lib/iomgr/ev_posix.h \ src/core/lib/iomgr/exec_ctx.h \ @@ -957,7 +956,6 @@ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ src/core/lib/iomgr/ev_epoll_linux.c \ -src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index 85b71a8255a..304a0e1e3a9 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -301,22 +301,6 @@ "third_party": false, "type": "target" }, - { - "deps": [ - "gpr", - "gpr_test_util", - "grpc", - "grpc_test_util" - ], - "headers": [], - "language": "c", - "name": "epoll_test", - "src": [ - "test/core/network_benchmarks/epoll_test.c" - ], - "third_party": false, - "type": "target" - }, { "deps": [ "gpr", @@ -5547,7 +5531,6 @@ "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", "src/core/lib/iomgr/ev_epoll_linux.h", - "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", @@ -5649,8 +5632,6 @@ "src/core/lib/iomgr/endpoint_pair_windows.c", "src/core/lib/iomgr/ev_epoll_linux.c", "src/core/lib/iomgr/ev_epoll_linux.h", - "src/core/lib/iomgr/ev_epoll_posix.c", - "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.c", diff --git a/tools/run_tests/tests.json b/tools/run_tests/tests.json index be7b72f61d2..850f9474aec 100644 --- a/tools/run_tests/tests.json +++ b/tools/run_tests/tests.json @@ -356,21 +356,6 @@ "windows" ] }, - { - "args": [], - "ci_platforms": [ - "linux" - ], - "cpu_cost": 1.0, - "exclude_configs": [], - "flaky": false, - "gtest": false, - "language": "c", - "name": "epoll_test", - "platforms": [ - "linux" - ] - }, { "args": [], "ci_platforms": [ diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj b/vsprojects/vcxproj/grpc/grpc.vcxproj index a67e4d16dad..ce523725e8e 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj @@ -317,7 +317,6 @@ - @@ -487,8 +486,6 @@ - - diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters index bf9b7dc7dcf..d46676f2296 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters @@ -58,9 +58,6 @@ src\core\lib\iomgr - - src\core\lib\iomgr - src\core\lib\iomgr @@ -683,9 +680,6 @@ src\core\lib\iomgr - - src\core\lib\iomgr - src\core\lib\iomgr diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj index afc9a2ca1b2..d4dd428c2db 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj @@ -305,7 +305,6 @@ - @@ -453,8 +452,6 @@ - - diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters index b7507f9a963..d14e7e7ab41 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters @@ -61,9 +61,6 @@ src\core\lib\iomgr - - src\core\lib\iomgr - src\core\lib\iomgr @@ -581,9 +578,6 @@ src\core\lib\iomgr - - src\core\lib\iomgr - src\core\lib\iomgr From 9bc3d2d67f32f4dad8cd1319dd4f3fce48c1abee Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 6 Jun 2016 10:27:56 -0700 Subject: [PATCH 038/215] Minor comments --- src/core/lib/iomgr/ev_epoll_linux.c | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 61106faef90..d3abf3bd84f 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -144,10 +144,9 @@ typedef struct polling_island { /******************************************************************************* * Pollset Declarations */ - struct grpc_pollset_worker { int kicked_specifically; - pthread_t pt_id; /* TODO (sreek) - Add an abstraction here */ + pthread_t pt_id; /* Thread id of this worker */ struct grpc_pollset_worker *next; struct grpc_pollset_worker *prev; }; @@ -483,8 +482,7 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Get locks on both the polling islands */ polling_island_pair_update_and_lock(&p, &q); - /* TODO: sreek: Think about this scenario some more. Is it possible ?. what - * does it mean, when would this happen */ + /* TODO: sreek: Think about this scenario some more */ if (p == q) { /* Nothing needs to be done here */ gpr_mu_unlock(&p->mu); @@ -539,7 +537,10 @@ static void polling_island_global_init() { * (specifically when a new alarm needs to be triggered earlier than the next * alarm 'epoch'). This wakeup_fd gives us something to alert on when such a * case occurs. */ -/* TODO: sreek: Right now, this wakes up all pollers */ + +/* TODO: sreek: Right now, this wakes up all pollers. In future we should make + * sure to wake up one polling thread (which can wake up other threads if + * needed) */ grpc_wakeup_fd grpc_global_wakeup_fd; static grpc_fd *fd_freelist = NULL; @@ -676,7 +677,6 @@ static int fd_wrapped_fd(grpc_fd *fd) { static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure *on_done, int *release_fd, const char *reason) { - /* TODO(sreek) In ev_poll_posix.c,the lock is acquired a little later. Why? */ bool is_fd_closed = false; gpr_mu_lock(&fd->mu); fd->on_done_closure = on_done; @@ -784,8 +784,9 @@ static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, */ static void sig_handler(int sig_num) { - /* TODO: sreek - Remove this expensive log line */ +#ifdef GPRC_EPOLL_DEBUG gpr_log(GPR_INFO, "Received signal %d", sig_num); +#endif } /* Global state management */ @@ -986,7 +987,10 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, if (ep_rv < 0) { if (errno != EINTR) { - /* TODO (sreek) - Check for bad file descriptor error */ + /* TODO (sreek) - Do not log an error in case of bad file descriptor + * (A bad file descriptor here would just mean that the epoll set was + * merged with another epoll set and that the current epoll_fd is + * closed) */ gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); } else { gpr_log(GPR_DEBUG, "pollset_work_and_unlock: 0-timeout epoll_wait()"); @@ -1062,7 +1066,9 @@ static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, GPR_TIMER_END("pollset_shutdown", 0); } -/* TODO(sreek) Is pollset_shutdown() guranteed to be called before this? */ +/* pollset_shutdown is guaranteed to be called before pollset_destroy. So other + * than destroying the mutexes, there is nothing special that needs to be done + * here */ static void pollset_destroy(grpc_pollset *pollset) { GPR_ASSERT(!pollset_has_workers(pollset)); gpr_mu_destroy(&pollset->pi_mu); @@ -1075,7 +1081,7 @@ static void pollset_reset(grpc_pollset *pollset) { pollset->shutting_down = false; pollset->finish_shutdown_called = false; pollset->kicked_without_pollers = false; - /* TODO(sreek) - Should pollset->shutdown closure be set to NULL here? */ + pollset->shutdown_done = NULL; pollset_release_polling_island_locked(pollset); } @@ -1149,7 +1155,7 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { gpr_log(GPR_DEBUG, "pollset_add_fd: pollset: %p, fd: %d", pollset, fd->fd); - /* TODO sreek - Check if we need to get a pollset->mu lock here */ + /* TODO sreek - Double check if we need to get a pollset->mu lock here */ gpr_mu_lock(&pollset->pi_mu); gpr_mu_lock(&fd->pi_mu); From d627c105847f0d262ce3886fb5b463dc914ddbd7 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 6 Jun 2016 15:49:32 -0700 Subject: [PATCH 039/215] Fix asan failures (i.e add pollset_global_shutdown), remove debug log lines --- src/core/lib/iomgr/ev_epoll_linux.c | 76 ++++++++++++++--------------- 1 file changed, 38 insertions(+), 38 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index d3abf3bd84f..0e00d4d216a 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -516,6 +516,21 @@ static void polling_island_global_init() { g_pi_freelist = NULL; } +static void polling_island_global_shutdown() { + polling_island *next; + gpr_mu_lock(&g_pi_freelist_mu); + gpr_mu_unlock(&g_pi_freelist_mu); + while (g_pi_freelist != NULL) { + next = g_pi_freelist->next_free; + gpr_mu_destroy(&g_pi_freelist->mu); + gpr_free(g_pi_freelist->fds); + gpr_free(g_pi_freelist); + g_pi_freelist = next; + } + + gpr_mu_destroy(&g_pi_freelist_mu); +} + /******************************************************************************* * Fd Definitions */ @@ -784,7 +799,7 @@ static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, */ static void sig_handler(int sig_num) { -#ifdef GPRC_EPOLL_DEBUG +#ifdef GRPC_EPOLL_DEBUG gpr_log(GPR_INFO, "Received signal %d", sig_num); #endif } @@ -792,7 +807,7 @@ static void sig_handler(int sig_num) { /* Global state management */ static void pollset_global_init(void) { grpc_wakeup_fd_init(&grpc_global_wakeup_fd); - signal(SIGUSR1, sig_handler); + signal(SIGUSR1, sig_handler); /* TODO: sreek - Do not hardcode SIGUSR1 */ } static void pollset_global_shutdown(void) { @@ -840,7 +855,6 @@ static void pollset_kick(grpc_pollset *p, grpc_pollset_worker *worker = specific_worker; if (worker != NULL) { if (worker == GRPC_POLLSET_KICK_BROADCAST) { - gpr_log(GPR_DEBUG, "pollset_kick: broadcast!"); if (pollset_has_workers(p)) { GPR_TIMER_BEGIN("pollset_kick.broadcast", 0); for (worker = p->root_worker.next; worker != &p->root_worker; @@ -848,12 +862,10 @@ static void pollset_kick(grpc_pollset *p, pthread_kill(worker->pt_id, SIGUSR1); } } else { - gpr_log(GPR_DEBUG, "pollset_kick: (broadcast) Kicked without pollers"); p->kicked_without_pollers = true; } GPR_TIMER_END("pollset_kick.broadcast", 0); } else { - gpr_log(GPR_DEBUG, "pollset_kick: kicked kicked_specifically"); GPR_TIMER_MARK("kicked_specifically", 0); worker->kicked_specifically = true; pthread_kill(worker->pt_id, SIGUSR1); @@ -864,11 +876,9 @@ static void pollset_kick(grpc_pollset *p, if (worker != NULL) { GPR_TIMER_MARK("finally_kick", 0); push_back_worker(p, worker); - gpr_log(GPR_DEBUG, "pollset_kick: anonymous kick"); pthread_kill(worker->pt_id, SIGUSR1); } else { GPR_TIMER_MARK("kicked_no_pollers", 0); - gpr_log(GPR_DEBUG, "pollset_kick: kicked without pollers"); p->kicked_without_pollers = true; } } @@ -941,7 +951,6 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; int epoll_fd = -1; int ep_rv; - gpr_log(GPR_DEBUG, "pollset_work_and_unlock: Entering.."); GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the @@ -952,22 +961,27 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, - pollset->polling_island->mu */ gpr_mu_lock(&pollset->pi_mu); - if (pollset->polling_island != NULL) { - pollset->polling_island = - polling_island_update_and_lock(pollset->polling_island, 1, 0); - epoll_fd = pollset->polling_island->epoll_fd; - if (pollset->polling_island->fd_cnt == 0) { - gpr_log(GPR_DEBUG, "pollset_work_and_unlock: epoll_fd: %d, No other fds", - epoll_fd); - } - for (size_t i = 0; i < pollset->polling_island->fd_cnt; i++) { - gpr_log(GPR_DEBUG, - "pollset_work_and_unlock: epoll_fd: %d, fd_count: %d, fd[%d]: %d", - epoll_fd, pollset->polling_island->fd_cnt, i, - pollset->polling_island->fds[i]->fd); - } - gpr_mu_unlock(&pollset->polling_island->mu); + if (pollset->polling_island == NULL) { + pollset->polling_island = polling_island_create(NULL, 1); + } + + pollset->polling_island = + polling_island_update_and_lock(pollset->polling_island, 1, 0); + epoll_fd = pollset->polling_island->epoll_fd; + +#ifdef GRPC_EPOLL_DEBUG + if (pollset->polling_island->fd_cnt == 0) { + gpr_log(GPR_DEBUG, "pollset_work_and_unlock: epoll_fd: %d, No other fds", + epoll_fd); + } + for (size_t i = 0; i < pollset->polling_island->fd_cnt; i++) { + gpr_log(GPR_DEBUG, + "pollset_work_and_unlock: epoll_fd: %d, fd_count: %d, fd[%d]: %d", + epoll_fd, pollset->polling_island->fd_cnt, i, + pollset->polling_island->fds[i]->fd); } +#endif + gpr_mu_unlock(&pollset->polling_island->mu); gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); @@ -975,16 +989,8 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, /* If epoll_fd == -1, this is a blank pollset and does not have any fds yet */ if (epoll_fd != -1) { do { - gpr_timespec before_epoll = gpr_now(GPR_CLOCK_PRECISE); - gpr_log(GPR_DEBUG, "pollset_work_and_unlock: epoll_wait()...."); ep_rv = epoll_pwait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms, sig_mask); - gpr_timespec after_epoll = gpr_now(GPR_CLOCK_PRECISE); - int dur = gpr_time_to_millis(gpr_time_sub(after_epoll, before_epoll)); - gpr_log(GPR_DEBUG, - "pollset_work_and_unlock: DONE epoll_wait() : %d ms, ep_rv: %d", - dur, ep_rv); - if (ep_rv < 0) { if (errno != EINTR) { /* TODO (sreek) - Do not log an error in case of bad file descriptor @@ -993,9 +999,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, * closed) */ gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); } else { - gpr_log(GPR_DEBUG, "pollset_work_and_unlock: 0-timeout epoll_wait()"); ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); - gpr_log(GPR_DEBUG, "pollset_work_and_unlock: ep_rv: %d", ep_rv); } } @@ -1018,7 +1022,6 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, } } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); } - gpr_log(GPR_DEBUG, "pollset_work_and_unlock: Leaving.."); GPR_TIMER_END("pollset_work_and_unlock", 0); } @@ -1093,7 +1096,6 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker **worker_hdl, gpr_timespec now, gpr_timespec deadline) { GPR_TIMER_BEGIN("pollset_work", 0); - gpr_log(GPR_DEBUG, "pollset_work: enter"); int timeout_ms = poll_deadline_to_millis_timeout(deadline, now); sigset_t new_mask; @@ -1112,7 +1114,6 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, work that needs attention like an event on the completion queue or an alarm */ GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); - gpr_log(GPR_INFO, "pollset_work: kicked without pollers.."); pollset->kicked_without_pollers = 0; } else if (!pollset->shutting_down) { sigemptyset(&new_mask); @@ -1147,14 +1148,12 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, gpr_mu_lock(&pollset->mu); } - gpr_log(GPR_DEBUG, "pollset_work(): leaving"); *worker_hdl = NULL; GPR_TIMER_END("pollset_work", 0); } static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { - gpr_log(GPR_DEBUG, "pollset_add_fd: pollset: %p, fd: %d", pollset, fd->fd); /* TODO sreek - Double check if we need to get a pollset->mu lock here */ gpr_mu_lock(&pollset->pi_mu); gpr_mu_lock(&fd->pi_mu); @@ -1347,6 +1346,7 @@ static void pollset_set_del_pollset_set(grpc_exec_ctx *exec_ctx, static void shutdown_engine(void) { fd_global_shutdown(); pollset_global_shutdown(); + polling_island_global_shutdown(); } static const grpc_event_engine_vtable vtable = { From e5012bac7a57df6b1993a1eaa9b8b3c4d7671975 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 6 Jun 2016 16:01:45 -0700 Subject: [PATCH 040/215] Remove redundant code --- src/core/lib/iomgr/ev_epoll_linux.c | 61 ++++++++++++++--------------- 1 file changed, 29 insertions(+), 32 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 0e00d4d216a..f7ac4ae1ff4 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -986,42 +986,39 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); - /* If epoll_fd == -1, this is a blank pollset and does not have any fds yet */ - if (epoll_fd != -1) { - do { - ep_rv = epoll_pwait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms, - sig_mask); - if (ep_rv < 0) { - if (errno != EINTR) { - /* TODO (sreek) - Do not log an error in case of bad file descriptor - * (A bad file descriptor here would just mean that the epoll set was - * merged with another epoll set and that the current epoll_fd is - * closed) */ - gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); - } else { - ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); - } + do { + ep_rv = epoll_pwait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms, + sig_mask); + if (ep_rv < 0) { + if (errno != EINTR) { + /* TODO (sreek) - Do not log an error in case of bad file descriptor + * (A bad file descriptor here would just mean that the epoll set was + * merged with another epoll set and that the current epoll_fd is + * closed) */ + gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); + } else { + ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); } + } - int i; - for (i = 0; i < ep_rv; ++i) { - grpc_fd *fd = ep_ev[i].data.ptr; - int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); - int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); - int write_ev = ep_ev[i].events & EPOLLOUT; - if (fd == NULL) { - grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); - } else { - if (read_ev || cancel) { - fd_become_readable(exec_ctx, fd); - } - if (write_ev || cancel) { - fd_become_writable(exec_ctx, fd); - } + int i; + for (i = 0; i < ep_rv; ++i) { + grpc_fd *fd = ep_ev[i].data.ptr; + int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + int write_ev = ep_ev[i].events & EPOLLOUT; + if (fd == NULL) { + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + } else { + if (read_ev || cancel) { + fd_become_readable(exec_ctx, fd); + } + if (write_ev || cancel) { + fd_become_writable(exec_ctx, fd); } } - } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); - } + } + } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); GPR_TIMER_END("pollset_work_and_unlock", 0); } From ad162ba5a9f91481b71f233d1f4f8c15b01de644 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 6 Jun 2016 16:23:37 -0700 Subject: [PATCH 041/215] Core review comments and remove 'kicked_specifically' field as its not needed --- src/core/lib/iomgr/ev_epoll_linux.c | 21 ++++++++------------- 1 file changed, 8 insertions(+), 13 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index f7ac4ae1ff4..d5aac96fa46 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -145,7 +145,6 @@ typedef struct polling_island { * Pollset Declarations */ struct grpc_pollset_worker { - int kicked_specifically; pthread_t pt_id; /* Thread id of this worker */ struct grpc_pollset_worker *next; struct grpc_pollset_worker *prev; @@ -235,18 +234,16 @@ static void polling_island_remove_all_fds_locked(polling_island *pi, size_t i; for (i = 0; i < pi->fd_cnt; i++) { - if (remove_fd_refs) { - GRPC_FD_UNREF(pi->fds[i], "polling_island"); - } - err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, pi->fds[i]->fd, NULL); if (err < 0 && errno != ENOENT) { - gpr_log(GPR_ERROR, - "epoll_ctl delete for fds[i]: %d failed with error: %s", i, - pi->fds[i]->fd, strerror(errno)); /* TODO: sreek - We need a better way to bubble up this error instead of - * just logging a message */ - continue; + * just logging a message */ + gpr_log(GPR_ERROR, "epoll_ctl deleting fds[%d]: %d failed with error: %s", + i, pi->fds[i]->fd, strerror(errno)); + } + + if (remove_fd_refs) { + GRPC_FD_UNREF(pi->fds[i], "polling_island"); } } @@ -264,7 +261,7 @@ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, if (!is_fd_closed) { err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, fd->fd, NULL); if (err < 0 && errno != ENOENT) { - gpr_log(GPR_ERROR, "epoll_ctl delete for fd: %d failed with error; %s", + gpr_log(GPR_ERROR, "epoll_ctl deleting fd: %d failed with error; %s", fd->fd, strerror(errno)); } } @@ -867,7 +864,6 @@ static void pollset_kick(grpc_pollset *p, GPR_TIMER_END("pollset_kick.broadcast", 0); } else { GPR_TIMER_MARK("kicked_specifically", 0); - worker->kicked_specifically = true; pthread_kill(worker->pt_id, SIGUSR1); } } else { @@ -1100,7 +1096,6 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker worker; worker.next = worker.prev = NULL; - worker.kicked_specifically = 0; worker.pt_id = pthread_self(); *worker_hdl = &worker; From 5855c478c69508f000baa4878f515d72b5f5a1e9 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 8 Jun 2016 12:56:56 -0700 Subject: [PATCH 042/215] Use poll if not linux, add read notifier pollset support and some groundwork for adding API that allows users to register custom kick signal number --- include/grpc/impl/codegen/port_platform.h | 1 + src/core/lib/iomgr/ev_epoll_linux.c | 72 ++++++++++++++++------- src/core/lib/iomgr/ev_posix.c | 2 +- 3 files changed, 54 insertions(+), 21 deletions(-) diff --git a/include/grpc/impl/codegen/port_platform.h b/include/grpc/impl/codegen/port_platform.h index be4215a54bf..7a6ec53fb4c 100644 --- a/include/grpc/impl/codegen/port_platform.h +++ b/include/grpc/impl/codegen/port_platform.h @@ -189,6 +189,7 @@ #define GPR_GCC_ATOMIC 1 #define GPR_GCC_TLS 1 #define GPR_LINUX 1 +#define GPR_LINUX_EPOLL 1 #define GPR_LINUX_LOG #define GPR_LINUX_MULTIPOLL_WITH_EPOLL 1 #define GPR_POSIX_WAKEUP_FD 1 diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index d5aac96fa46..69ab665e155 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -33,7 +33,7 @@ #include -#ifdef GPR_POSIX_SOCKET +#ifdef GPR_LINUX_EPOLL #include "src/core/lib/iomgr/ev_epoll_linux.h" @@ -60,6 +60,8 @@ struct polling_island; +static int grpc_poller_kick_signum; + /******************************************************************************* * Fd Declarations */ @@ -92,6 +94,9 @@ struct grpc_fd { struct grpc_fd *freelist_next; grpc_closure *on_done_closure; + /* The pollset that last noticed that the fd is readable */ + grpc_pollset *read_notifier_pollset; + grpc_iomgr_object iomgr_object; }; @@ -650,14 +655,15 @@ static grpc_fd *fd_create(int fd, const char *name) { gpr_mu_lock(&new_fd->mu); gpr_atm_rel_store(&new_fd->refst, 1); + new_fd->fd = fd; new_fd->shutdown = false; + new_fd->orphaned = false; new_fd->read_closure = CLOSURE_NOT_READY; new_fd->write_closure = CLOSURE_NOT_READY; - new_fd->fd = fd; new_fd->polling_island = NULL; new_fd->freelist_next = NULL; new_fd->on_done_closure = NULL; - new_fd->orphaned = false; + new_fd->read_notifier_pollset = NULL; gpr_mu_unlock(&new_fd->mu); @@ -765,6 +771,17 @@ static int set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, } } +static grpc_pollset *fd_get_read_notifier_pollset(grpc_exec_ctx *exec_ctx, + grpc_fd *fd) { + grpc_pollset *notifier = NULL; + + gpr_mu_lock(&fd->mu); + notifier = fd->read_notifier_pollset; + gpr_mu_unlock(&fd->mu); + + return notifier; +} + static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { gpr_mu_lock(&fd->mu); GPR_ASSERT(!fd->shutdown); @@ -801,16 +818,25 @@ static void sig_handler(int sig_num) { #endif } +static void poller_kick_init() { + grpc_poller_kick_signum = SIGRTMIN + 2; + signal(grpc_poller_kick_signum, sig_handler); +} + /* Global state management */ static void pollset_global_init(void) { grpc_wakeup_fd_init(&grpc_global_wakeup_fd); - signal(SIGUSR1, sig_handler); /* TODO: sreek - Do not hardcode SIGUSR1 */ + poller_kick_init(); } static void pollset_global_shutdown(void) { grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); } +static void pollset_worker_kick(grpc_pollset_worker *worker) { + pthread_kill(worker->pt_id, grpc_poller_kick_signum); +} + /* Return 1 if the pollset has active threads in pollset_work (pollset must * be locked) */ static int pollset_has_workers(grpc_pollset *p) { @@ -856,7 +882,7 @@ static void pollset_kick(grpc_pollset *p, GPR_TIMER_BEGIN("pollset_kick.broadcast", 0); for (worker = p->root_worker.next; worker != &p->root_worker; worker = worker->next) { - pthread_kill(worker->pt_id, SIGUSR1); + pollset_worker_kick(worker); } } else { p->kicked_without_pollers = true; @@ -864,7 +890,7 @@ static void pollset_kick(grpc_pollset *p, GPR_TIMER_END("pollset_kick.broadcast", 0); } else { GPR_TIMER_MARK("kicked_specifically", 0); - pthread_kill(worker->pt_id, SIGUSR1); + pollset_worker_kick(worker); } } else { GPR_TIMER_MARK("kick_anonymous", 0); @@ -872,7 +898,7 @@ static void pollset_kick(grpc_pollset *p, if (worker != NULL) { GPR_TIMER_MARK("finally_kick", 0); push_back_worker(p, worker); - pthread_kill(worker->pt_id, SIGUSR1); + pollset_worker_kick(worker); } else { GPR_TIMER_MARK("kicked_no_pollers", 0); p->kicked_without_pollers = true; @@ -924,20 +950,20 @@ static int poll_deadline_to_millis_timeout(gpr_timespec deadline, timeout, gpr_time_from_nanos(GPR_NS_PER_MS - 1, GPR_TIMESPAN))); } -static void set_ready(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st) { - /* only one set_ready can be active at once (but there may be a racing - notify_on) */ +static void fd_become_readable(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_pollset *notifier) { + /* Need the fd->mu since we might be racing with fd_notify_on_read */ gpr_mu_lock(&fd->mu); - set_ready_locked(exec_ctx, fd, st); + set_ready_locked(exec_ctx, fd, &fd->read_closure); + fd->read_notifier_pollset = notifier; gpr_mu_unlock(&fd->mu); } -static void fd_become_readable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - set_ready(exec_ctx, fd, &fd->read_closure); -} - static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - set_ready(exec_ctx, fd, &fd->write_closure); + /* Need the fd->mu since we might be racing with fd_notify_on_write */ + gpr_mu_lock(&fd->mu); + set_ready_locked(exec_ctx, fd, &fd->write_closure); + gpr_mu_unlock(&fd->mu); } #define GRPC_EPOLL_MAX_EVENTS 1000 @@ -1007,7 +1033,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); } else { if (read_ev || cancel) { - fd_become_readable(exec_ctx, fd); + fd_become_readable(exec_ctx, fd, pollset); } if (write_ev || cancel) { fd_become_writable(exec_ctx, fd); @@ -1109,9 +1135,9 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, pollset->kicked_without_pollers = 0; } else if (!pollset->shutting_down) { sigemptyset(&new_mask); - sigaddset(&new_mask, SIGUSR1); + sigaddset(&new_mask, grpc_poller_kick_signum); pthread_sigmask(SIG_BLOCK, &new_mask, &orig_mask); - sigdelset(&orig_mask, SIGUSR1); + sigdelset(&orig_mask, grpc_poller_kick_signum); push_front_worker(pollset, &worker); @@ -1350,6 +1376,7 @@ static const grpc_event_engine_vtable vtable = { .fd_shutdown = fd_shutdown, .fd_notify_on_read = fd_notify_on_read, .fd_notify_on_write = fd_notify_on_write, + .fd_get_read_notifier_pollset = fd_get_read_notifier_pollset, .pollset_init = pollset_init, .pollset_shutdown = pollset_shutdown, @@ -1380,4 +1407,9 @@ const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { return &vtable; } -#endif +#else /* defined(GPR_LINUX_EPOLL) */ +/* If GPR_LINUX_EPOLL is not defined, it means epoll is not available. Return + * NULL */ +const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { return NULL; } + +#endif /* !defined(GPR_LINUX_EPOLL) */ diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index e0c3558a51e..2b15967adcc 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -63,8 +63,8 @@ typedef struct { } event_engine_factory; static const event_engine_factory g_factories[] = { - {"poll", grpc_init_poll_posix}, {"epoll", grpc_init_epoll_linux}, + {"poll", grpc_init_poll_posix}, {"legacy", grpc_init_poll_and_epoll_posix}, }; From 24b1062f42ef01bd47a458e94423f068ec1765f0 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 8 Jun 2016 15:20:17 -0700 Subject: [PATCH 043/215] Do not close epoll_fd while there are any pollers and add the ability to wake up all pollers when an island is merged --- src/core/lib/iomgr/ev_epoll_linux.c | 113 ++++++++++++++++++++-------- 1 file changed, 80 insertions(+), 33 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 69ab665e155..3a3c136a5aa 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -190,9 +190,18 @@ struct grpc_pollset_set { }; /******************************************************************************* - * Polling-island Definitions + * Polling island Definitions */ +/* The wakeup fd that is used to wake up all threads in a Polling island. This + is useful in the polling island merge operation where we need to wakeup all + the threads currently polling the smaller polling island (so that they can + start polling the new/merged polling island) + + NOTE: This fd is initialized to be readable and MUST NOT be consumed i.e the + threads that woke up MUST NOT call grpc_wakeup_fd_consume_wakeup() */ +static grpc_wakeup_fd polling_island_wakeup_fd; + /* Polling island freelist */ static gpr_mu g_pi_freelist_mu; static polling_island *g_pi_freelist = NULL; @@ -232,6 +241,25 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, } } +/* The caller is expected to hold pi->mu before calling this */ +static void polling_island_add_wakeup_fd_locked(polling_island *pi, + grpc_wakeup_fd *wakeup_fd) { + struct epoll_event ev; + int err; + + ev.events = (uint32_t)(EPOLLIN | EPOLLET); + ev.data.ptr = wakeup_fd; + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, + GRPC_WAKEUP_FD_GET_READ_FD(wakeup_fd), &ev); + if (err < 0) { + gpr_log(GPR_ERROR, + "Failed to add grpc_wake_up_fd (%d) to the epoll set (epoll_fd: %d)" + ". Error: %s", + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), pi->epoll_fd, + strerror(errno)); + } +} + /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_remove_all_fds_locked(polling_island *pi, bool remove_fd_refs) { @@ -283,8 +311,6 @@ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, static polling_island *polling_island_create(grpc_fd *initial_fd, int initial_ref_cnt) { polling_island *pi = NULL; - struct epoll_event ev; - int err; /* Try to get one from the polling island freelist */ gpr_mu_lock(&g_pi_freelist_mu); @@ -311,17 +337,7 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, } GPR_ASSERT(pi->epoll_fd >= 0); - ev.events = (uint32_t)(EPOLLIN | EPOLLET); - ev.data.ptr = NULL; - err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); - if (err < 0) { - gpr_log(GPR_ERROR, - "Failed to add grpc_global_wake_up_fd (%d) to the epoll set " - "(epoll_fd: %d) with error: %s", - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), pi->epoll_fd, - strerror(errno)); - } + polling_island_add_wakeup_fd_locked(pi, &grpc_global_wakeup_fd); pi->ref_cnt = initial_ref_cnt; pi->merged_to = NULL; @@ -496,13 +512,15 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { GPR_SWAP(polling_island *, p, q); } - /* "Merge" p with q i.e move all the fds from p (the polling_island with fewer - fds) to q. - Note: Not altering the ref counts on the affected fds here because they - would effectively remain unchanged */ + /* "Merge" p with q i.e move all the fds from p (The one with fewer fds) to q + )Note that the refcounts on the fds being moved will not change here. This + is why the last parameter in the following two functions is 'false') */ polling_island_add_fds_locked(q, p->fds, p->fd_cnt, false); polling_island_remove_all_fds_locked(p, false); + /* Wakeup all the pollers (if any) on p so that they can pickup this change */ + polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); + /* The merged polling island inherits all the ref counts of the island merging with it */ q->ref_cnt += p->ref_cnt; @@ -516,6 +534,8 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { static void polling_island_global_init() { gpr_mu_init(&g_pi_freelist_mu); g_pi_freelist = NULL; + grpc_wakeup_fd_init(&polling_island_wakeup_fd); + grpc_wakeup_fd_wakeup(&polling_island_wakeup_fd); } static void polling_island_global_shutdown() { @@ -529,8 +549,9 @@ static void polling_island_global_shutdown() { gpr_free(g_pi_freelist); g_pi_freelist = next; } - gpr_mu_destroy(&g_pi_freelist_mu); + + grpc_wakeup_fd_destroy(&polling_island_wakeup_fd); } /******************************************************************************* @@ -973,6 +994,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; int epoll_fd = -1; int ep_rv; + polling_island *pi = NULL; GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the @@ -983,13 +1005,19 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, - pollset->polling_island->mu */ gpr_mu_lock(&pollset->pi_mu); - if (pollset->polling_island == NULL) { - pollset->polling_island = polling_island_create(NULL, 1); + pi = pollset->polling_island; + if (pi == NULL) { + pi = polling_island_create(NULL, 1); } - pollset->polling_island = - polling_island_update_and_lock(pollset->polling_island, 1, 0); - epoll_fd = pollset->polling_island->epoll_fd; + /* In addition to locking the polling island, add a ref so that the island + does not get destroyed (which means the epoll_fd won't be closed) while + we are are doing an epoll_wait() on the epoll_fd */ + pi = polling_island_update_and_lock(pi, 1, 1); + epoll_fd = pi->epoll_fd; + + /* Update the pollset->polling_island */ + pollset->polling_island = pi; #ifdef GRPC_EPOLL_DEBUG if (pollset->polling_island->fd_cnt == 0) { @@ -1013,25 +1041,29 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, sig_mask); if (ep_rv < 0) { if (errno != EINTR) { - /* TODO (sreek) - Do not log an error in case of bad file descriptor - * (A bad file descriptor here would just mean that the epoll set was - * merged with another epoll set and that the current epoll_fd is - * closed) */ gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); } else { + /* We were interrupted. Save an interation by doing a zero timeout + epoll_wait to see if there are any other events of interest */ ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); } } int i; for (i = 0; i < ep_rv; ++i) { - grpc_fd *fd = ep_ev[i].data.ptr; - int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); - int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); - int write_ev = ep_ev[i].events & EPOLLOUT; - if (fd == NULL) { + void *data_ptr = ep_ev[i].data.ptr; + if (data_ptr == &grpc_global_wakeup_fd) { grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + } else if (data_ptr == &polling_island_wakeup_fd) { + /* This means that our polling island is merged with a different + island. We do not have to do anything here since the subsequent call + to the function pollset_work_and_unlock() will pick up the correct + epoll_fd */ } else { + grpc_fd *fd = data_ptr; + int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + int write_ev = ep_ev[i].events & EPOLLOUT; if (read_ev || cancel) { fd_become_readable(exec_ctx, fd, pollset); } @@ -1041,6 +1073,21 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, } } } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); + + GPR_ASSERT(pi != NULL); + + /* Before leaving, release the extra ref we added to the polling island */ + /* It is important to note that at this point 'pi' may not be the same as + * pollset->polling_island. This is because pollset->polling_island pointer + * gets updated whenever the underlying polling island is merged with another + * island and while we are doing epoll_wait() above, the polling island may + * have been merged */ + + /* TODO (sreek) - Change the ref count on polling island to gpr_atm so that + * we do not have to do this here */ + gpr_mu_lock(&pi->mu); + polling_island_unref_and_unlock(pi, 1); + GPR_TIMER_END("pollset_work_and_unlock", 0); } From e682e46a9e451a94cfcb1cf9c927185abd81fceb Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 8 Jun 2016 15:40:21 -0700 Subject: [PATCH 044/215] Add TODOs --- src/core/lib/iomgr/ev_epoll_linux.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 3a3c136a5aa..2e871a4f1ba 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -83,6 +83,7 @@ struct grpc_fd { this indicates that the 'fd' on this structure is no longer valid */ bool orphaned; + /* TODO: sreek - Move this a lockfree implementation */ grpc_closure *read_closure; grpc_closure *write_closure; @@ -166,6 +167,9 @@ struct grpc_pollset { /* The polling island to which this pollset belongs to and the mutex protecting the field */ + /* TODO: sreek: This lock might actually be adding more overhead to the + critical path (i.e pollset_work() function). Consider removing this lock + and just using the overall pollset lock */ gpr_mu pi_mu; struct polling_island *polling_island; }; From 3dbf4d61b26e2364a974e47f16f3a655d3eda908 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 8 Jun 2016 16:26:45 -0700 Subject: [PATCH 045/215] More TODOs --- src/core/lib/iomgr/ev_epoll_linux.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 2e871a4f1ba..046ec5e7407 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -83,7 +83,7 @@ struct grpc_fd { this indicates that the 'fd' on this structure is no longer valid */ bool orphaned; - /* TODO: sreek - Move this a lockfree implementation */ + /* TODO: sreek - Move this to a lockfree implementation */ grpc_closure *read_closure; grpc_closure *write_closure; @@ -124,6 +124,8 @@ static void fd_global_shutdown(void); /******************************************************************************* * Polling-island Declarations */ +/* TODO: sree: Consider making ref_cnt and merged_to to gpr_atm - This would + * significantly reduce the number of mutex acquisition calls. */ typedef struct polling_island { gpr_mu mu; int ref_cnt; @@ -177,6 +179,12 @@ struct grpc_pollset { /******************************************************************************* * Pollset-set Declarations */ +/* TODO: sreek - Change the pollset_set implementation such that a pollset_set + * directly points to a polling_island (and adding an fd/pollset/pollset_set to + * the current pollset_set would result in polling island merges. This would + * remove the need to maintain fd_count here. This will also significantly + * simplify the grpc_fd structure since we would no longer need to explicitly + * maintain the orphaned state */ struct grpc_pollset_set { gpr_mu mu; From 8e4926c0eeb0df9e5c8029136e39f6c8700f0814 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 8 Jun 2016 20:33:19 -0700 Subject: [PATCH 046/215] pollset_kick optimization (do not kick any other thread if the current thread can be kicked) --- src/core/lib/iomgr/ev_epoll_linux.c | 153 ++++++++++++++++------------ 1 file changed, 87 insertions(+), 66 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 046ec5e7407..d45f87c2f8e 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -844,6 +844,8 @@ static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, /******************************************************************************* * Pollset Definitions */ +GPR_TLS_DECL(g_current_thread_pollset); +GPR_TLS_DECL(g_current_thread_worker); static void sig_handler(int sig_num) { #ifdef GRPC_EPOLL_DEBUG @@ -859,11 +861,15 @@ static void poller_kick_init() { /* Global state management */ static void pollset_global_init(void) { grpc_wakeup_fd_init(&grpc_global_wakeup_fd); + gpr_tls_init(&g_current_thread_pollset); + gpr_tls_init(&g_current_thread_worker); poller_kick_init(); } static void pollset_global_shutdown(void) { grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); + gpr_tls_destroy(&g_current_thread_pollset); + gpr_tls_destroy(&g_current_thread_worker); } static void pollset_worker_kick(grpc_pollset_worker *worker) { @@ -915,7 +921,9 @@ static void pollset_kick(grpc_pollset *p, GPR_TIMER_BEGIN("pollset_kick.broadcast", 0); for (worker = p->root_worker.next; worker != &p->root_worker; worker = worker->next) { - pollset_worker_kick(worker); + if (gpr_tls_get(&g_current_thread_worker) != (intptr_t)worker) { + pollset_worker_kick(worker); + } } } else { p->kicked_without_pollers = true; @@ -923,9 +931,18 @@ static void pollset_kick(grpc_pollset *p, GPR_TIMER_END("pollset_kick.broadcast", 0); } else { GPR_TIMER_MARK("kicked_specifically", 0); - pollset_worker_kick(worker); + if (gpr_tls_get(&g_current_thread_worker) != (intptr_t)worker) { + pollset_worker_kick(worker); + } } - } else { + } else if (gpr_tls_get(&g_current_thread_pollset) != (intptr_t)p) { + /* Since worker == NULL, it means that we can kick "any" worker on this + pollset 'p'. If 'p' happens to be the same pollset this thread is + currently polling (i.e in pollset_work() function), then there is no need + to kick any other worker since the current thread can just absorb the + kick. This is the reason why we enter this case only when + g_current_thread_pollset is != p */ + GPR_TIMER_MARK("kick_anonymous", 0); worker = pop_front_worker(p); if (worker != NULL) { @@ -999,6 +1016,69 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { gpr_mu_unlock(&fd->mu); } +/* Release the reference to pollset->polling_island and set it to NULL. + pollset->mu must be held */ +static void pollset_release_polling_island_locked(grpc_pollset *pollset) { + gpr_mu_lock(&pollset->pi_mu); + if (pollset->polling_island) { + pollset->polling_island = + polling_island_update_and_lock(pollset->polling_island, 1, 0); + polling_island_unref_and_unlock(pollset->polling_island, 1); + pollset->polling_island = NULL; + } + gpr_mu_unlock(&pollset->pi_mu); +} + +static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset) { + /* The pollset cannot have any workers if we are at this stage */ + GPR_ASSERT(!pollset_has_workers(pollset)); + + pollset->finish_shutdown_called = true; + pollset_release_polling_island_locked(pollset); + + grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); +} + +/* pollset->mu lock must be held by the caller before calling this */ +static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_closure *closure) { + GPR_TIMER_BEGIN("pollset_shutdown", 0); + GPR_ASSERT(!pollset->shutting_down); + pollset->shutting_down = true; + pollset->shutdown_done = closure; + pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); + + /* If the pollset has any workers, we cannot call finish_shutdown_locked() + because it would release the underlying polling island. In such a case, we + let the last worker call finish_shutdown_locked() from pollset_work() */ + if (!pollset_has_workers(pollset)) { + GPR_ASSERT(!pollset->finish_shutdown_called); + GPR_TIMER_MARK("pollset_shutdown.finish_shutdown_locked", 0); + finish_shutdown_locked(exec_ctx, pollset); + } + GPR_TIMER_END("pollset_shutdown", 0); +} + +/* pollset_shutdown is guaranteed to be called before pollset_destroy. So other + * than destroying the mutexes, there is nothing special that needs to be done + * here */ +static void pollset_destroy(grpc_pollset *pollset) { + GPR_ASSERT(!pollset_has_workers(pollset)); + gpr_mu_destroy(&pollset->pi_mu); + gpr_mu_destroy(&pollset->mu); +} + +static void pollset_reset(grpc_pollset *pollset) { + GPR_ASSERT(pollset->shutting_down); + GPR_ASSERT(!pollset_has_workers(pollset)); + pollset->shutting_down = false; + pollset->finish_shutdown_called = false; + pollset->kicked_without_pollers = false; + pollset->shutdown_done = NULL; + pollset_release_polling_island_locked(pollset); +} + #define GRPC_EPOLL_MAX_EVENTS 1000 static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, int timeout_ms, @@ -1103,69 +1183,6 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, GPR_TIMER_END("pollset_work_and_unlock", 0); } -/* Release the reference to pollset->polling_island and set it to NULL. - pollset->mu must be held */ -static void pollset_release_polling_island_locked(grpc_pollset *pollset) { - gpr_mu_lock(&pollset->pi_mu); - if (pollset->polling_island) { - pollset->polling_island = - polling_island_update_and_lock(pollset->polling_island, 1, 0); - polling_island_unref_and_unlock(pollset->polling_island, 1); - pollset->polling_island = NULL; - } - gpr_mu_unlock(&pollset->pi_mu); -} - -static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset) { - /* The pollset cannot have any workers if we are at this stage */ - GPR_ASSERT(!pollset_has_workers(pollset)); - - pollset->finish_shutdown_called = true; - pollset_release_polling_island_locked(pollset); - - grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); -} - -/* pollset->mu lock must be held by the caller before calling this */ -static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_closure *closure) { - GPR_TIMER_BEGIN("pollset_shutdown", 0); - GPR_ASSERT(!pollset->shutting_down); - pollset->shutting_down = true; - pollset->shutdown_done = closure; - pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); - - /* If the pollset has any workers, we cannot call finish_shutdown_locked() - because it would release the underlying polling island. In such a case, we - let the last worker call finish_shutdown_locked() from pollset_work() */ - if (!pollset_has_workers(pollset)) { - GPR_ASSERT(!pollset->finish_shutdown_called); - GPR_TIMER_MARK("pollset_shutdown.finish_shutdown_locked", 0); - finish_shutdown_locked(exec_ctx, pollset); - } - GPR_TIMER_END("pollset_shutdown", 0); -} - -/* pollset_shutdown is guaranteed to be called before pollset_destroy. So other - * than destroying the mutexes, there is nothing special that needs to be done - * here */ -static void pollset_destroy(grpc_pollset *pollset) { - GPR_ASSERT(!pollset_has_workers(pollset)); - gpr_mu_destroy(&pollset->pi_mu); - gpr_mu_destroy(&pollset->mu); -} - -static void pollset_reset(grpc_pollset *pollset) { - GPR_ASSERT(pollset->shutting_down); - GPR_ASSERT(!pollset_has_workers(pollset)); - pollset->shutting_down = false; - pollset->finish_shutdown_called = false; - pollset->kicked_without_pollers = false; - pollset->shutdown_done = NULL; - pollset_release_polling_island_locked(pollset); -} - /* pollset->mu lock must be held by the caller before calling this. The function pollset_work() may temporarily release the lock (pollset->mu) during the course of its execution but it will always re-acquire the lock and @@ -1184,6 +1201,8 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, worker.pt_id = pthread_self(); *worker_hdl = &worker; + gpr_tls_set(&g_current_thread_pollset, (intptr_t)pollset); + gpr_tls_set(&g_current_thread_worker, (intptr_t)&worker); if (pollset->kicked_without_pollers) { /* If the pollset was kicked without pollers, pretend that the current @@ -1226,6 +1245,8 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } *worker_hdl = NULL; + gpr_tls_set(&g_current_thread_pollset, (intptr_t)0); + gpr_tls_set(&g_current_thread_worker, (intptr_t)0); GPR_TIMER_END("pollset_work", 0); } From 0553a436610201b252cfce0ed5d2cea69da15e85 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 9 Jun 2016 00:42:41 -0700 Subject: [PATCH 047/215] Fix refcounting bug in polling_island_merge --- src/core/lib/iomgr/ev_epoll_linux.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index d45f87c2f8e..66bbae52b28 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -512,7 +512,6 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Get locks on both the polling islands */ polling_island_pair_update_and_lock(&p, &q); - /* TODO: sreek: Think about this scenario some more */ if (p == q) { /* Nothing needs to be done here */ gpr_mu_unlock(&p->mu); @@ -525,7 +524,7 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { } /* "Merge" p with q i.e move all the fds from p (The one with fewer fds) to q - )Note that the refcounts on the fds being moved will not change here. This + Note that the refcounts on the fds being moved will not change here. This is why the last parameter in the following two functions is 'false') */ polling_island_add_fds_locked(q, p->fds, p->fd_cnt, false); polling_island_remove_all_fds_locked(p, false); @@ -533,9 +532,11 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Wakeup all the pollers (if any) on p so that they can pickup this change */ polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); - /* The merged polling island inherits all the ref counts of the island merging - with it */ + /* - The merged polling island (i.e q) inherits all the ref counts of the + island merging with it (i.e p) + - The island p will lose a ref count */ q->ref_cnt += p->ref_cnt; + p->ref_cnt--; gpr_mu_unlock(&p->mu); gpr_mu_unlock(&q->mu); From 727440216553c01925c0f5bc88293108bb3f051f Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 9 Jun 2016 09:42:06 -0700 Subject: [PATCH 048/215] Check epoll is actually available. set GPR_LINUX_EPOLL only in GLIBC ver 2.9 and above --- include/grpc/impl/codegen/port_platform.h | 2 +- src/core/lib/iomgr/ev_epoll_linux.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/include/grpc/impl/codegen/port_platform.h b/include/grpc/impl/codegen/port_platform.h index 7a6ec53fb4c..affef9e66b6 100644 --- a/include/grpc/impl/codegen/port_platform.h +++ b/include/grpc/impl/codegen/port_platform.h @@ -189,7 +189,6 @@ #define GPR_GCC_ATOMIC 1 #define GPR_GCC_TLS 1 #define GPR_LINUX 1 -#define GPR_LINUX_EPOLL 1 #define GPR_LINUX_LOG #define GPR_LINUX_MULTIPOLL_WITH_EPOLL 1 #define GPR_POSIX_WAKEUP_FD 1 @@ -201,6 +200,7 @@ #ifdef __GLIBC_PREREQ #if __GLIBC_PREREQ(2, 9) #define GPR_LINUX_EVENTFD 1 +#define GPR_LINUX_EPOLL 1 #endif #if __GLIBC_PREREQ(2, 10) #define GPR_LINUX_SOCKETUTILS 1 diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 66bbae52b28..d2d5d2852b9 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -1481,7 +1481,26 @@ static const grpc_event_engine_vtable vtable = { .shutdown_engine = shutdown_engine, }; +/* It is possible that GLIBC has epoll but the underlying kernel doesn't. + * Create a dummy epoll_fd to make sure epoll support is available */ +static bool is_epoll_available() { + int fd = epoll_create1(EPOLL_CLOEXEC); + if (fd < 0) { + gpr_log( + GPR_ERROR, + "epoll_create1 failed with error: %d. Not using epoll polling engine", + fd); + return false; + } + close(fd); + return true; +} + const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { + if (!is_epoll_available()) { + return NULL; + } + fd_global_init(); pollset_global_init(); polling_island_global_init(); From c7be7c688829281b543428ec22029d4d09bd2a9c Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 9 Jun 2016 17:08:50 -0700 Subject: [PATCH 049/215] Add an API at the core level to disable signals or use a different signal number --- include/grpc/grpc_posix.h | 8 +++++ src/core/lib/iomgr/ev_epoll_linux.c | 45 ++++++++++++++++++++++------- 2 files changed, 43 insertions(+), 10 deletions(-) diff --git a/include/grpc/grpc_posix.h b/include/grpc/grpc_posix.h index 9742b83374a..5e89ae3b1ee 100644 --- a/include/grpc/grpc_posix.h +++ b/include/grpc/grpc_posix.h @@ -63,6 +63,14 @@ GRPCAPI void grpc_server_add_insecure_channel_from_fd(grpc_server *server, grpc_completion_queue *cq, int fd); +/** GRPC Core POSIX library may internally use signals to optimize some work. + The library uses (SIGRTMIN + 2) signal by default. Use this API to instruct + the library to use a different signal i.e 'signum' instead. + Note: + - To prevent GRPC library from using any signals, pass a 'signum' of -1 + - This API is optional but if called, it MUST be called before grpc_init() */ +GRPCAPI void grpc_use_signal(int signum); + #ifdef __cplusplus } #endif diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index d2d5d2852b9..7e01ac144f2 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -31,6 +31,7 @@ * */ +#include #include #ifdef GPR_LINUX_EPOLL @@ -58,9 +59,26 @@ #include "src/core/lib/profiling/timers.h" #include "src/core/lib/support/block_annotate.h" -struct polling_island; +static int grpc_wakeup_signal = -1; +static bool is_grpc_wakeup_signal_initialized = false; + +/* Implements the function defined in grpc_posix.h. This function might be + * called before even calling grpc_init() to set either a different signal to + * use. If signum == -1, then the use of signals is disabled */ +void grpc_use_signal(int signum) { + grpc_wakeup_signal = signum; + is_grpc_wakeup_signal_initialized = true; -static int grpc_poller_kick_signum; + if (grpc_wakeup_signal < 0) { + gpr_log(GPR_INFO, + "Use of signals is disabled. Epoll engine will not be used"); + } else { + gpr_log(GPR_INFO, "epoll engine will be using signal: %d", + grpc_wakeup_signal); + } +} + +struct polling_island; /******************************************************************************* * Fd Declarations @@ -854,10 +872,7 @@ static void sig_handler(int sig_num) { #endif } -static void poller_kick_init() { - grpc_poller_kick_signum = SIGRTMIN + 2; - signal(grpc_poller_kick_signum, sig_handler); -} +static void poller_kick_init() { signal(grpc_wakeup_signal, sig_handler); } /* Global state management */ static void pollset_global_init(void) { @@ -874,7 +889,7 @@ static void pollset_global_shutdown(void) { } static void pollset_worker_kick(grpc_pollset_worker *worker) { - pthread_kill(worker->pt_id, grpc_poller_kick_signum); + pthread_kill(worker->pt_id, grpc_wakeup_signal); } /* Return 1 if the pollset has active threads in pollset_work (pollset must @@ -1214,9 +1229,9 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, pollset->kicked_without_pollers = 0; } else if (!pollset->shutting_down) { sigemptyset(&new_mask); - sigaddset(&new_mask, grpc_poller_kick_signum); + sigaddset(&new_mask, grpc_wakeup_signal); pthread_sigmask(SIG_BLOCK, &new_mask, &orig_mask); - sigdelset(&orig_mask, grpc_poller_kick_signum); + sigdelset(&orig_mask, grpc_wakeup_signal); push_front_worker(pollset, &worker); @@ -1497,19 +1512,29 @@ static bool is_epoll_available() { } const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { + /* If use of signals is disabled, we cannot use epoll engine*/ + if (is_grpc_wakeup_signal_initialized && grpc_wakeup_signal < 0) { + return NULL; + } + if (!is_epoll_available()) { return NULL; } + if (!is_grpc_wakeup_signal_initialized) { + grpc_use_signal(SIGRTMIN + 2); + } + fd_global_init(); pollset_global_init(); polling_island_global_init(); return &vtable; } -#else /* defined(GPR_LINUX_EPOLL) */ +#else /* defined(GPR_LINUX_EPOLL) */ /* If GPR_LINUX_EPOLL is not defined, it means epoll is not available. Return * NULL */ const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { return NULL; } +void grpc_use_signal(int signum) {} #endif /* !defined(GPR_LINUX_EPOLL) */ From 492fd961824447e7f12974893dd9bbf37291e5cc Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 10 Jun 2016 09:03:34 -0700 Subject: [PATCH 050/215] generate_projects.sh after adding grpc_use_signal() API --- BUILD | 2 ++ Makefile | 1 + grpc.def | 1 + src/python/grpcio/grpc/_cython/imports.generated.c | 2 ++ src/python/grpcio/grpc/_cython/imports.generated.h | 3 +++ src/ruby/ext/grpc/rb_grpc_imports.generated.c | 2 ++ src/ruby/ext/grpc/rb_grpc_imports.generated.h | 3 +++ 7 files changed, 14 insertions(+) diff --git a/BUILD b/BUILD index 68b066d5593..43855751cbc 100644 --- a/BUILD +++ b/BUILD @@ -556,6 +556,7 @@ cc_library( "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_linux.h", "src/core/lib/iomgr/ev_poll_and_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", @@ -693,6 +694,7 @@ cc_library( "src/core/lib/iomgr/endpoint.c", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_linux.c", "src/core/lib/iomgr/ev_poll_and_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", diff --git a/Makefile b/Makefile index 620299ed15b..4d8b060760a 100644 --- a/Makefile +++ b/Makefile @@ -2764,6 +2764,7 @@ LIBGRPC_CRONET_SRC = \ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ + src/core/lib/iomgr/ev_epoll_linux.c \ src/core/lib/iomgr/ev_poll_and_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ diff --git a/grpc.def b/grpc.def index 0046028949e..9f70ce61a30 100644 --- a/grpc.def +++ b/grpc.def @@ -90,6 +90,7 @@ EXPORTS grpc_call_error_to_string grpc_insecure_channel_create_from_fd grpc_server_add_insecure_channel_from_fd + grpc_use_signal grpc_auth_property_iterator_next grpc_auth_context_property_iterator grpc_auth_context_peer_identity diff --git a/src/python/grpcio/grpc/_cython/imports.generated.c b/src/python/grpcio/grpc/_cython/imports.generated.c index 5c49f6cf3e9..a025b99047d 100644 --- a/src/python/grpcio/grpc/_cython/imports.generated.c +++ b/src/python/grpcio/grpc/_cython/imports.generated.c @@ -128,6 +128,7 @@ grpc_is_binary_header_type grpc_is_binary_header_import; grpc_call_error_to_string_type grpc_call_error_to_string_import; grpc_insecure_channel_create_from_fd_type grpc_insecure_channel_create_from_fd_import; grpc_server_add_insecure_channel_from_fd_type grpc_server_add_insecure_channel_from_fd_import; +grpc_use_signal_type grpc_use_signal_import; grpc_auth_property_iterator_next_type grpc_auth_property_iterator_next_import; grpc_auth_context_property_iterator_type grpc_auth_context_property_iterator_import; grpc_auth_context_peer_identity_type grpc_auth_context_peer_identity_import; @@ -401,6 +402,7 @@ void pygrpc_load_imports(HMODULE library) { grpc_call_error_to_string_import = (grpc_call_error_to_string_type) GetProcAddress(library, "grpc_call_error_to_string"); grpc_insecure_channel_create_from_fd_import = (grpc_insecure_channel_create_from_fd_type) GetProcAddress(library, "grpc_insecure_channel_create_from_fd"); grpc_server_add_insecure_channel_from_fd_import = (grpc_server_add_insecure_channel_from_fd_type) GetProcAddress(library, "grpc_server_add_insecure_channel_from_fd"); + grpc_use_signal_import = (grpc_use_signal_type) GetProcAddress(library, "grpc_use_signal"); grpc_auth_property_iterator_next_import = (grpc_auth_property_iterator_next_type) GetProcAddress(library, "grpc_auth_property_iterator_next"); grpc_auth_context_property_iterator_import = (grpc_auth_context_property_iterator_type) GetProcAddress(library, "grpc_auth_context_property_iterator"); grpc_auth_context_peer_identity_import = (grpc_auth_context_peer_identity_type) GetProcAddress(library, "grpc_auth_context_peer_identity"); diff --git a/src/python/grpcio/grpc/_cython/imports.generated.h b/src/python/grpcio/grpc/_cython/imports.generated.h index 16bb5cdfab6..5bdbbce89b5 100644 --- a/src/python/grpcio/grpc/_cython/imports.generated.h +++ b/src/python/grpcio/grpc/_cython/imports.generated.h @@ -335,6 +335,9 @@ extern grpc_insecure_channel_create_from_fd_type grpc_insecure_channel_create_fr typedef void(*grpc_server_add_insecure_channel_from_fd_type)(grpc_server *server, grpc_completion_queue *cq, int fd); extern grpc_server_add_insecure_channel_from_fd_type grpc_server_add_insecure_channel_from_fd_import; #define grpc_server_add_insecure_channel_from_fd grpc_server_add_insecure_channel_from_fd_import +typedef void(*grpc_use_signal_type)(int signum); +extern grpc_use_signal_type grpc_use_signal_import; +#define grpc_use_signal grpc_use_signal_import typedef const grpc_auth_property *(*grpc_auth_property_iterator_next_type)(grpc_auth_property_iterator *it); extern grpc_auth_property_iterator_next_type grpc_auth_property_iterator_next_import; #define grpc_auth_property_iterator_next grpc_auth_property_iterator_next_import diff --git a/src/ruby/ext/grpc/rb_grpc_imports.generated.c b/src/ruby/ext/grpc/rb_grpc_imports.generated.c index c13d1a00d74..8e24ef38ed5 100644 --- a/src/ruby/ext/grpc/rb_grpc_imports.generated.c +++ b/src/ruby/ext/grpc/rb_grpc_imports.generated.c @@ -128,6 +128,7 @@ grpc_is_binary_header_type grpc_is_binary_header_import; grpc_call_error_to_string_type grpc_call_error_to_string_import; grpc_insecure_channel_create_from_fd_type grpc_insecure_channel_create_from_fd_import; grpc_server_add_insecure_channel_from_fd_type grpc_server_add_insecure_channel_from_fd_import; +grpc_use_signal_type grpc_use_signal_import; grpc_auth_property_iterator_next_type grpc_auth_property_iterator_next_import; grpc_auth_context_property_iterator_type grpc_auth_context_property_iterator_import; grpc_auth_context_peer_identity_type grpc_auth_context_peer_identity_import; @@ -397,6 +398,7 @@ void grpc_rb_load_imports(HMODULE library) { grpc_call_error_to_string_import = (grpc_call_error_to_string_type) GetProcAddress(library, "grpc_call_error_to_string"); grpc_insecure_channel_create_from_fd_import = (grpc_insecure_channel_create_from_fd_type) GetProcAddress(library, "grpc_insecure_channel_create_from_fd"); grpc_server_add_insecure_channel_from_fd_import = (grpc_server_add_insecure_channel_from_fd_type) GetProcAddress(library, "grpc_server_add_insecure_channel_from_fd"); + grpc_use_signal_import = (grpc_use_signal_type) GetProcAddress(library, "grpc_use_signal"); grpc_auth_property_iterator_next_import = (grpc_auth_property_iterator_next_type) GetProcAddress(library, "grpc_auth_property_iterator_next"); grpc_auth_context_property_iterator_import = (grpc_auth_context_property_iterator_type) GetProcAddress(library, "grpc_auth_context_property_iterator"); grpc_auth_context_peer_identity_import = (grpc_auth_context_peer_identity_type) GetProcAddress(library, "grpc_auth_context_peer_identity"); diff --git a/src/ruby/ext/grpc/rb_grpc_imports.generated.h b/src/ruby/ext/grpc/rb_grpc_imports.generated.h index 9c86a3690c5..d6e2a367999 100644 --- a/src/ruby/ext/grpc/rb_grpc_imports.generated.h +++ b/src/ruby/ext/grpc/rb_grpc_imports.generated.h @@ -335,6 +335,9 @@ extern grpc_insecure_channel_create_from_fd_type grpc_insecure_channel_create_fr typedef void(*grpc_server_add_insecure_channel_from_fd_type)(grpc_server *server, grpc_completion_queue *cq, int fd); extern grpc_server_add_insecure_channel_from_fd_type grpc_server_add_insecure_channel_from_fd_import; #define grpc_server_add_insecure_channel_from_fd grpc_server_add_insecure_channel_from_fd_import +typedef void(*grpc_use_signal_type)(int signum); +extern grpc_use_signal_type grpc_use_signal_import; +#define grpc_use_signal grpc_use_signal_import typedef const grpc_auth_property *(*grpc_auth_property_iterator_next_type)(grpc_auth_property_iterator *it); extern grpc_auth_property_iterator_next_type grpc_auth_property_iterator_next_import; #define grpc_auth_property_iterator_next grpc_auth_property_iterator_next_import From eb16b3dc3cd579931d730ba3fef1f7008f649003 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 10 Jun 2016 23:06:25 -0700 Subject: [PATCH 051/215] Fix ref counting bug --- src/core/lib/iomgr/ev_epoll_linux.c | 20 ++------------------ 1 file changed, 2 insertions(+), 18 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 7e01ac144f2..617afad1975 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -1127,20 +1127,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, /* Update the pollset->polling_island */ pollset->polling_island = pi; -#ifdef GRPC_EPOLL_DEBUG - if (pollset->polling_island->fd_cnt == 0) { - gpr_log(GPR_DEBUG, "pollset_work_and_unlock: epoll_fd: %d, No other fds", - epoll_fd); - } - for (size_t i = 0; i < pollset->polling_island->fd_cnt; i++) { - gpr_log(GPR_DEBUG, - "pollset_work_and_unlock: epoll_fd: %d, fd_count: %d, fd[%d]: %d", - epoll_fd, pollset->polling_island->fd_cnt, i, - pollset->polling_island->fds[i]->fd); - } -#endif - gpr_mu_unlock(&pollset->polling_island->mu); - + polling_island_unref_and_unlock(pollset->polling_island, 0); /* Keep the ref*/ gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); @@ -1190,10 +1177,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, * gets updated whenever the underlying polling island is merged with another * island and while we are doing epoll_wait() above, the polling island may * have been merged */ - - /* TODO (sreek) - Change the ref count on polling island to gpr_atm so that - * we do not have to do this here */ - gpr_mu_lock(&pi->mu); + polling_island_update_and_lock(pi, 1, 0); /* No new ref added */ polling_island_unref_and_unlock(pi, 1); GPR_TIMER_END("pollset_work_and_unlock", 0); From 58e589644403b10afb31ffd45befabe13b652db8 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 13 Jun 2016 00:52:56 -0700 Subject: [PATCH 052/215] Fix bad merge --- src/core/lib/iomgr/ev_epoll_linux.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 617afad1975..a8a874cd4b0 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -550,14 +550,14 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Wakeup all the pollers (if any) on p so that they can pickup this change */ polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); + p->merged_to = q; + /* - The merged polling island (i.e q) inherits all the ref counts of the island merging with it (i.e p) - The island p will lose a ref count */ q->ref_cnt += p->ref_cnt; - p->ref_cnt--; - - gpr_mu_unlock(&p->mu); - gpr_mu_unlock(&q->mu); + polling_island_unref_and_unlock(p, 1); /* Decrement refcount */ + polling_island_unref_and_unlock(q, 0); /* Just Unlock. Don't decrement ref */ return q; } @@ -1110,7 +1110,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, Acquire the following locks: - pollset->mu (which we already have) - pollset->pi_mu - - pollset->polling_island->mu */ + - pollset->polling_island->mu (call polling_island_update_and_lock())*/ gpr_mu_lock(&pollset->pi_mu); pi = pollset->polling_island; @@ -1144,8 +1144,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, } } - int i; - for (i = 0; i < ep_rv; ++i) { + for (int i = 0; i < ep_rv; ++i) { void *data_ptr = ep_ev[i].data.ptr; if (data_ptr == &grpc_global_wakeup_fd) { grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); @@ -1177,7 +1176,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, * gets updated whenever the underlying polling island is merged with another * island and while we are doing epoll_wait() above, the polling island may * have been merged */ - polling_island_update_and_lock(pi, 1, 0); /* No new ref added */ + pi = polling_island_update_and_lock(pi, 1, 0); /* No new ref added */ polling_island_unref_and_unlock(pi, 1); GPR_TIMER_END("pollset_work_and_unlock", 0); From ddc3ebb6ea694b252c27e149507bec4df427a592 Mon Sep 17 00:00:00 2001 From: yang-g Date: Mon, 13 Jun 2016 10:40:32 -0700 Subject: [PATCH 053/215] add missing language tag --- build.yaml | 11 +++++----- tools/run_tests/sources_and_headers.json | 26 ++++++++++++------------ 2 files changed, 19 insertions(+), 18 deletions(-) diff --git a/build.yaml b/build.yaml index a94fb052a5d..8589ba28f8e 100644 --- a/build.yaml +++ b/build.yaml @@ -143,11 +143,6 @@ filegroups: - include/grpc/impl/codegen/sync_posix.h - include/grpc/impl/codegen/sync_windows.h - include/grpc/impl/codegen/time.h -- name: grpc++_codegen_base_src - src: - - src/cpp/codegen/codegen_init.cc - uses: - - grpc++_codegen_base - name: grpc_base public_headers: - include/grpc/byte_buffer.h @@ -747,6 +742,12 @@ filegroups: - include/grpc++/impl/codegen/time.h uses: - grpc_codegen +- name: grpc++_codegen_base_src + language: c++ + src: + - src/cpp/codegen/codegen_init.cc + uses: + - grpc++_codegen_base - name: grpc++_codegen_proto language: c++ public_headers: diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index fd522ee1739..1d73434f3dc 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -5607,19 +5607,6 @@ "third_party": false, "type": "filegroup" }, - { - "deps": [ - "grpc++_codegen_base" - ], - "headers": [], - "language": "c", - "name": "grpc++_codegen_base_src", - "src": [ - "src/cpp/codegen/codegen_init.cc" - ], - "third_party": false, - "type": "filegroup" - }, { "deps": [ "gpr", @@ -6636,6 +6623,19 @@ "third_party": false, "type": "filegroup" }, + { + "deps": [ + "grpc++_codegen_base" + ], + "headers": [], + "language": "c++", + "name": "grpc++_codegen_base_src", + "src": [ + "src/cpp/codegen/codegen_init.cc" + ], + "third_party": false, + "type": "filegroup" + }, { "deps": [ "grpc++_codegen_base", From 773a8825bcb9cae06af18263b112b8f8fd96f10d Mon Sep 17 00:00:00 2001 From: yang-g Date: Mon, 13 Jun 2016 11:09:29 -0700 Subject: [PATCH 054/215] Minor fixes --- src/core/lib/iomgr/udp_server.c | 4 ++-- src/cpp/server/server_posix.cc | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/core/lib/iomgr/udp_server.c b/src/core/lib/iomgr/udp_server.c index 16150687d3d..1ebccf2ee2e 100644 --- a/src/core/lib/iomgr/udp_server.c +++ b/src/core/lib/iomgr/udp_server.c @@ -243,13 +243,13 @@ static int prepare_socket(int fd, const struct sockaddr *addr, if (!grpc_set_socket_sndbuf(fd, buffer_size_bytes)) { gpr_log(GPR_ERROR, "Failed to set send buffer size to %d bytes", - buf_size_bytes); + buffer_size_bytes); goto error; } if (!grpc_set_socket_rcvbuf(fd, buffer_size_bytes)) { gpr_log(GPR_ERROR, "Failed to set receive buffer size to %d bytes", - buf_size_bytes); + buffer_size_bytes); goto error; } diff --git a/src/cpp/server/server_posix.cc b/src/cpp/server/server_posix.cc index 8cb9753a125..c3aa2adc60e 100644 --- a/src/cpp/server/server_posix.cc +++ b/src/cpp/server/server_posix.cc @@ -42,8 +42,8 @@ namespace grpc { void AddInsecureChannelFromFd(Server* server, int fd) { grpc_server_add_insecure_channel_from_fd( server->c_server(), server->completion_queue()->cq(), fd); +} #endif // GPR_SUPPORT_CHANNELS_FROM_FD -} } // namespace grpc From 41622a8e389e8eda38d6d3bfbf34cbf35f437156 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 13 Jun 2016 16:43:14 -0700 Subject: [PATCH 055/215] Fix tsan failures --- Makefile | 1 + build.yaml | 1 + src/core/lib/iomgr/ev_epoll_linux.c | 27 ++++++++++++++++++++++++++- 3 files changed, 28 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 4d8b060760a..28d6842c76c 100644 --- a/Makefile +++ b/Makefile @@ -200,6 +200,7 @@ LD_tsan = clang LDXX_tsan = clang++ CPPFLAGS_tsan = -O0 -fsanitize=thread -fno-omit-frame-pointer -Wno-unused-command-line-argument -DGPR_NO_DIRECT_SYSCALLS LDFLAGS_tsan = -fsanitize=thread +DEFINES_tsan = _GRPC_TSAN DEFINES_tsan += GRPC_TEST_SLOWDOWN_BUILD_FACTOR=5 VALID_CONFIG_stapprof = 1 diff --git a/build.yaml b/build.yaml index 85b66d985bb..139ab3e8bca 100644 --- a/build.yaml +++ b/build.yaml @@ -3231,6 +3231,7 @@ configs: CPPFLAGS: -O0 -fsanitize=thread -fno-omit-frame-pointer -Wno-unused-command-line-argument -DGPR_NO_DIRECT_SYSCALLS CXX: clang++ + DEFINES: _GRPC_TSAN LD: clang LDFLAGS: -fsanitize=thread LDXX: clang++ diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index a8a874cd4b0..35a15e00c96 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -236,6 +236,17 @@ static grpc_wakeup_fd polling_island_wakeup_fd; static gpr_mu g_pi_freelist_mu; static polling_island *g_pi_freelist = NULL; +#ifdef _GRPC_TSAN +/* Currently TSAN may incorrectly flag data races between epoll_ctl and + epoll_wait for any grpc_fd structs that are added to the epoll set via + epoll_ctl and are returned (within a very short window) via epoll_wait(). + + To work-around this race, we establish a happens-before relation between + the code just-before epoll_ctl() and the code after epoll_wait() by using + this atomic */ +gpr_atm g_epoll_sync; +#endif + /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, size_t fd_count, bool add_fd_refs) { @@ -243,6 +254,11 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, size_t i; struct epoll_event ev; +#ifdef _GRPC_TSAN + /* See the definition of g_epoll_sync for more context */ + gpr_atm_rel_store(&g_epoll_sync, 0); +#endif + for (i = 0; i < fd_count; i++) { ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); ev.data.ptr = fds[i]; @@ -361,6 +377,7 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, } pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); + if (pi->epoll_fd < 0) { gpr_log(GPR_ERROR, "epoll_create1() failed with error: %s", strerror(errno)); @@ -1144,6 +1161,11 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, } } +#ifdef _GRPC_TSAN + /* See the definition of g_poll_sync for more details */ + gpr_atm_acq_load(&g_epoll_sync); +#endif + for (int i = 0; i < ep_rv; ++i) { void *data_ptr = ep_ev[i].data.ptr; if (data_ptr == &grpc_global_wakeup_fd) { @@ -1514,10 +1536,13 @@ const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { return &vtable; } -#else /* defined(GPR_LINUX_EPOLL) */ +#else /* defined(GPR_LINUX_EPOLL) */ +#if defined(GPR_POSIX_SOCKET) +#include "src/core/lib/iomgr/ev_posix.h" /* If GPR_LINUX_EPOLL is not defined, it means epoll is not available. Return * NULL */ const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { return NULL; } +#endif /* defined(GPR_POSIX_SOCKET) */ void grpc_use_signal(int signum) {} #endif /* !defined(GPR_LINUX_EPOLL) */ From ad2c4778fc560f10f38550428189c97c9e2bc5a1 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 13 Jun 2016 19:06:54 -0700 Subject: [PATCH 056/215] Rename _GRPC_TSAN to GRPC_TSAN --- Makefile | 2 +- build.yaml | 2 +- src/core/lib/iomgr/ev_epoll_linux.c | 12 ++++++------ 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Makefile b/Makefile index 040ebd4102b..e35e360785b 100644 --- a/Makefile +++ b/Makefile @@ -200,7 +200,7 @@ LD_tsan = clang LDXX_tsan = clang++ CPPFLAGS_tsan = -O0 -fsanitize=thread -fno-omit-frame-pointer -Wno-unused-command-line-argument -DGPR_NO_DIRECT_SYSCALLS LDFLAGS_tsan = -fsanitize=thread -DEFINES_tsan = _GRPC_TSAN +DEFINES_tsan = GRPC_TSAN DEFINES_tsan += GRPC_TEST_SLOWDOWN_BUILD_FACTOR=5 VALID_CONFIG_stapprof = 1 diff --git a/build.yaml b/build.yaml index 0847232b504..3d327f8bffd 100644 --- a/build.yaml +++ b/build.yaml @@ -3266,7 +3266,7 @@ configs: CPPFLAGS: -O0 -fsanitize=thread -fno-omit-frame-pointer -Wno-unused-command-line-argument -DGPR_NO_DIRECT_SYSCALLS CXX: clang++ - DEFINES: _GRPC_TSAN + DEFINES: GRPC_TSAN LD: clang LDFLAGS: -fsanitize=thread LDXX: clang++ diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 35a15e00c96..006c2a8ee7f 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -236,7 +236,7 @@ static grpc_wakeup_fd polling_island_wakeup_fd; static gpr_mu g_pi_freelist_mu; static polling_island *g_pi_freelist = NULL; -#ifdef _GRPC_TSAN +#ifdef GRPC_TSAN /* Currently TSAN may incorrectly flag data races between epoll_ctl and epoll_wait for any grpc_fd structs that are added to the epoll set via epoll_ctl and are returned (within a very short window) via epoll_wait(). @@ -245,7 +245,7 @@ static polling_island *g_pi_freelist = NULL; the code just-before epoll_ctl() and the code after epoll_wait() by using this atomic */ gpr_atm g_epoll_sync; -#endif +#endif /* defined(GRPC_TSAN) */ /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, @@ -254,10 +254,10 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, size_t i; struct epoll_event ev; -#ifdef _GRPC_TSAN +#ifdef GRPC_TSAN /* See the definition of g_epoll_sync for more context */ gpr_atm_rel_store(&g_epoll_sync, 0); -#endif +#endif /* defined(GRPC_TSAN) */ for (i = 0; i < fd_count; i++) { ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); @@ -1161,10 +1161,10 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, } } -#ifdef _GRPC_TSAN +#ifdef GRPC_TSAN /* See the definition of g_poll_sync for more details */ gpr_atm_acq_load(&g_epoll_sync); -#endif +#endif /* defined(GRPC_TSAN) */ for (int i = 0; i < ep_rv; ++i) { void *data_ptr = ep_ev[i].data.ptr; From 88651de8a713b068eaf499d36bf0b67c0cc9e8e3 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Tue, 14 Jun 2016 16:30:58 -0700 Subject: [PATCH 057/215] Fixes #2646 Pass NULL in the host parameter of grpc_channel_create_call --- src/objective-c/GRPCClient/private/GRPCChannel.m | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/src/objective-c/GRPCClient/private/GRPCChannel.m b/src/objective-c/GRPCClient/private/GRPCChannel.m index d3192c983d1..6cd4cb0ca30 100644 --- a/src/objective-c/GRPCClient/private/GRPCChannel.m +++ b/src/objective-c/GRPCClient/private/GRPCChannel.m @@ -199,9 +199,7 @@ grpc_channel_args * buildChannelArgs(NSDictionary *dictionary) { NULL, GRPC_PROPAGATE_DEFAULTS, queue.unmanagedQueue, path.UTF8String, - // Get "host" from "host:port" - // TODO(jcanizales): Use NSURLs throughout, to clarify these. - [_host componentsSeparatedByString:@":"][0].UTF8String, + NULL, gpr_inf_future(GPR_CLOCK_REALTIME), NULL); } From cf4205dff54d8e3920f98dac28049770b5f8f044 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 15 Jun 2016 11:22:20 -0700 Subject: [PATCH 058/215] Compilation error --- src/core/lib/iomgr/ev_epoll_linux.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 006c2a8ee7f..1fb59474640 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -317,7 +317,7 @@ static void polling_island_remove_all_fds_locked(polling_island *pi, if (err < 0 && errno != ENOENT) { /* TODO: sreek - We need a better way to bubble up this error instead of * just logging a message */ - gpr_log(GPR_ERROR, "epoll_ctl deleting fds[%d]: %d failed with error: %s", + gpr_log(GPR_ERROR, "epoll_ctl deleting fds[%zu]: %d failed with error: %s", i, pi->fds[i]->fd, strerror(errno)); } From 9f6a80597771eae74760b23a66737eb8960f8fd7 Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Wed, 15 Jun 2016 17:39:03 -0700 Subject: [PATCH 059/215] Allow disabling traces, add trace variables for pluck and timeout events --- BUILD | 4 -- build.yaml | 1 - gRPC.podspec | 2 - grpc.gemspec | 1 - package.xml | 1 - src/core/lib/debug/trace.c | 8 +++- src/core/lib/surface/call.h | 1 - src/core/lib/surface/completion_queue.c | 29 +++++++---- src/core/lib/surface/completion_queue.h | 3 ++ src/core/lib/surface/init.c | 7 ++- src/core/lib/surface/surface_trace.h | 48 ------------------- tools/doxygen/Doxyfile.core.internal | 1 - tools/run_tests/sources_and_headers.json | 2 - vsprojects/vcxproj/grpc/grpc.vcxproj | 1 - vsprojects/vcxproj/grpc/grpc.vcxproj.filters | 3 -- .../grpc_unsecure/grpc_unsecure.vcxproj | 1 - .../grpc_unsecure.vcxproj.filters | 3 -- 17 files changed, 36 insertions(+), 80 deletions(-) delete mode 100644 src/core/lib/surface/surface_trace.h diff --git a/BUILD b/BUILD index f049e3c4056..bf001ac7273 100644 --- a/BUILD +++ b/BUILD @@ -228,7 +228,6 @@ cc_library( "src/core/lib/surface/init.h", "src/core/lib/surface/lame_client.h", "src/core/lib/surface/server.h", - "src/core/lib/surface/surface_trace.h", "src/core/lib/transport/byte_stream.h", "src/core/lib/transport/connectivity_state.h", "src/core/lib/transport/metadata.h", @@ -606,7 +605,6 @@ cc_library( "src/core/lib/surface/init.h", "src/core/lib/surface/lame_client.h", "src/core/lib/surface/server.h", - "src/core/lib/surface/surface_trace.h", "src/core/lib/transport/byte_stream.h", "src/core/lib/transport/connectivity_state.h", "src/core/lib/transport/metadata.h", @@ -949,7 +947,6 @@ cc_library( "src/core/lib/surface/init.h", "src/core/lib/surface/lame_client.h", "src/core/lib/surface/server.h", - "src/core/lib/surface/surface_trace.h", "src/core/lib/transport/byte_stream.h", "src/core/lib/transport/connectivity_state.h", "src/core/lib/transport/metadata.h", @@ -2061,7 +2058,6 @@ objc_library( "src/core/lib/surface/init.h", "src/core/lib/surface/lame_client.h", "src/core/lib/surface/server.h", - "src/core/lib/surface/surface_trace.h", "src/core/lib/transport/byte_stream.h", "src/core/lib/transport/connectivity_state.h", "src/core/lib/transport/metadata.h", diff --git a/build.yaml b/build.yaml index a83ebc595ad..11cc9347cb0 100644 --- a/build.yaml +++ b/build.yaml @@ -224,7 +224,6 @@ filegroups: - src/core/lib/surface/init.h - src/core/lib/surface/lame_client.h - src/core/lib/surface/server.h - - src/core/lib/surface/surface_trace.h - src/core/lib/transport/byte_stream.h - src/core/lib/transport/connectivity_state.h - src/core/lib/transport/metadata.h diff --git a/gRPC.podspec b/gRPC.podspec index d3665d51744..622be89ca74 100644 --- a/gRPC.podspec +++ b/gRPC.podspec @@ -231,7 +231,6 @@ Pod::Spec.new do |s| 'src/core/lib/surface/init.h', 'src/core/lib/surface/lame_client.h', 'src/core/lib/surface/server.h', - 'src/core/lib/surface/surface_trace.h', 'src/core/lib/transport/byte_stream.h', 'src/core/lib/transport/connectivity_state.h', 'src/core/lib/transport/metadata.h', @@ -606,7 +605,6 @@ Pod::Spec.new do |s| 'src/core/lib/surface/init.h', 'src/core/lib/surface/lame_client.h', 'src/core/lib/surface/server.h', - 'src/core/lib/surface/surface_trace.h', 'src/core/lib/transport/byte_stream.h', 'src/core/lib/transport/connectivity_state.h', 'src/core/lib/transport/metadata.h', diff --git a/grpc.gemspec b/grpc.gemspec index 9f3ae048a70..e0a8ff00a42 100755 --- a/grpc.gemspec +++ b/grpc.gemspec @@ -240,7 +240,6 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/surface/init.h ) s.files += %w( src/core/lib/surface/lame_client.h ) s.files += %w( src/core/lib/surface/server.h ) - s.files += %w( src/core/lib/surface/surface_trace.h ) s.files += %w( src/core/lib/transport/byte_stream.h ) s.files += %w( src/core/lib/transport/connectivity_state.h ) s.files += %w( src/core/lib/transport/metadata.h ) diff --git a/package.xml b/package.xml index b25e42c1c01..fef06ef9eed 100644 --- a/package.xml +++ b/package.xml @@ -247,7 +247,6 @@ - diff --git a/src/core/lib/debug/trace.c b/src/core/lib/debug/trace.c index 555f497b784..c56046785b3 100644 --- a/src/core/lib/debug/trace.c +++ b/src/core/lib/debug/trace.c @@ -88,7 +88,11 @@ static void parse(const char *s) { split(s, &strings, &nstrings); for (i = 0; i < nstrings; i++) { - grpc_tracer_set_enabled(strings[i], 1); + if (strings[i][0] == '-') { + grpc_tracer_set_enabled(strings[i] + 1, 0); + } else { + grpc_tracer_set_enabled(strings[i], 1); + } } for (i = 0; i < nstrings; i++) { @@ -117,7 +121,7 @@ int grpc_tracer_set_enabled(const char *name, int enabled) { tracer *t; if (0 == strcmp(name, "all")) { for (t = tracers; t; t = t->next) { - *t->flag = 1; + *t->flag = enabled; } } else { int found = 0; diff --git a/src/core/lib/surface/call.h b/src/core/lib/surface/call.h index b640345c21d..3a78fe3aa36 100644 --- a/src/core/lib/surface/call.h +++ b/src/core/lib/surface/call.h @@ -37,7 +37,6 @@ #include "src/core/lib/channel/channel_stack.h" #include "src/core/lib/channel/context.h" #include "src/core/lib/surface/api_trace.h" -#include "src/core/lib/surface/surface_trace.h" #include #include diff --git a/src/core/lib/surface/completion_queue.c b/src/core/lib/surface/completion_queue.c index 5eb7cf1bf4f..de2b674d0e0 100644 --- a/src/core/lib/surface/completion_queue.c +++ b/src/core/lib/surface/completion_queue.c @@ -48,7 +48,6 @@ #include "src/core/lib/surface/api_trace.h" #include "src/core/lib/surface/call.h" #include "src/core/lib/surface/event_string.h" -#include "src/core/lib/surface/surface_trace.h" typedef struct { grpc_pollset_worker **worker; @@ -91,6 +90,18 @@ struct grpc_completion_queue { static gpr_mu g_freelist_mu; static grpc_completion_queue *g_freelist; +int grpc_cq_pluck_trace; +int grpc_cq_event_timeout_trace; + +#define GRPC_SURFACE_TRACE_RETURNED_EVENT(cq, event) \ + if (grpc_api_trace && \ + (grpc_cq_pluck_trace || (event)->type != GRPC_QUEUE_TIMEOUT)) { \ + char *_ev = grpc_event_string(event); \ + gpr_log(GPR_INFO, "RETURN_EVENT[%p]: %s", cq, _ev); \ + gpr_free(_ev); \ + } + + static void on_pollset_shutdown_done(grpc_exec_ctx *exec_ctx, void *cc, bool success); @@ -396,13 +407,15 @@ grpc_event grpc_completion_queue_pluck(grpc_completion_queue *cc, void *tag, GPR_TIMER_BEGIN("grpc_completion_queue_pluck", 0); - GRPC_API_TRACE( - "grpc_completion_queue_pluck(" - "cc=%p, tag=%p, " - "deadline=gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " - "reserved=%p)", - 6, (cc, tag, (long long)deadline.tv_sec, (int)deadline.tv_nsec, - (int)deadline.clock_type, reserved)); + if (grpc_cq_pluck_trace) { + GRPC_API_TRACE( + "grpc_completion_queue_pluck(" + "cc=%p, tag=%p, " + "deadline=gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " + "reserved=%p)", + 6, (cc, tag, (long long)deadline.tv_sec, (int)deadline.tv_nsec, + (int)deadline.clock_type, reserved)); + } GPR_ASSERT(!reserved); deadline = gpr_convert_clock_type(deadline, GPR_CLOCK_MONOTONIC); diff --git a/src/core/lib/surface/completion_queue.h b/src/core/lib/surface/completion_queue.h index 3d0dd13c53b..108445b28f1 100644 --- a/src/core/lib/surface/completion_queue.h +++ b/src/core/lib/surface/completion_queue.h @@ -39,6 +39,9 @@ #include #include "src/core/lib/iomgr/pollset.h" +extern int grpc_cq_pluck_trace; +extern int grpc_cq_event_timeout_trace; + typedef struct grpc_cq_completion { /** user supplied tag */ void *tag; diff --git a/src/core/lib/surface/init.c b/src/core/lib/surface/init.c index 1c8b7090156..499ecffc2ae 100644 --- a/src/core/lib/surface/init.c +++ b/src/core/lib/surface/init.c @@ -57,7 +57,6 @@ #include "src/core/lib/surface/init.h" #include "src/core/lib/surface/lame_client.h" #include "src/core/lib/surface/server.h" -#include "src/core/lib/surface/surface_trace.h" #include "src/core/lib/transport/connectivity_state.h" #include "src/core/lib/transport/transport_impl.h" @@ -165,6 +164,12 @@ void grpc_init(void) { &grpc_trace_channel_stack_builder); grpc_register_tracer("http1", &grpc_http1_trace); grpc_register_tracer("compression", &grpc_compression_trace); + grpc_register_tracer("queue_pluck", &grpc_cq_pluck_trace); + // Default pluck trace to 1 + grpc_cq_pluck_trace = 1; + grpc_register_tracer("queue_timeout", &grpc_cq_event_timeout_trace); + // Default timeout trace to 1 + grpc_cq_event_timeout_trace = 1; grpc_security_pre_init(); grpc_iomgr_init(); grpc_executor_init(); diff --git a/src/core/lib/surface/surface_trace.h b/src/core/lib/surface/surface_trace.h deleted file mode 100644 index a69a0fff577..00000000000 --- a/src/core/lib/surface/surface_trace.h +++ /dev/null @@ -1,48 +0,0 @@ -/* - * - * Copyright 2015, Google Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are - * met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following disclaimer - * in the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Google Inc. nor the names of its - * contributors may be used to endorse or promote products derived from - * this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - * - */ - -#ifndef GRPC_CORE_LIB_SURFACE_SURFACE_TRACE_H -#define GRPC_CORE_LIB_SURFACE_SURFACE_TRACE_H - -#include -#include "src/core/lib/debug/trace.h" -#include "src/core/lib/surface/api_trace.h" - -#define GRPC_SURFACE_TRACE_RETURNED_EVENT(cq, event) \ - if (grpc_api_trace) { \ - char *_ev = grpc_event_string(event); \ - gpr_log(GPR_INFO, "RETURN_EVENT[%p]: %s", cq, _ev); \ - gpr_free(_ev); \ - } - -#endif /* GRPC_CORE_LIB_SURFACE_SURFACE_TRACE_H */ diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal index aee866adf46..337ce16551a 100644 --- a/tools/doxygen/Doxyfile.core.internal +++ b/tools/doxygen/Doxyfile.core.internal @@ -857,7 +857,6 @@ src/core/lib/surface/event_string.h \ src/core/lib/surface/init.h \ src/core/lib/surface/lame_client.h \ src/core/lib/surface/server.h \ -src/core/lib/surface/surface_trace.h \ src/core/lib/transport/byte_stream.h \ src/core/lib/transport/connectivity_state.h \ src/core/lib/transport/metadata.h \ diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index 4aad52c69d0..f9c497afcdd 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -5762,7 +5762,6 @@ "src/core/lib/surface/init.h", "src/core/lib/surface/lame_client.h", "src/core/lib/surface/server.h", - "src/core/lib/surface/surface_trace.h", "src/core/lib/transport/byte_stream.h", "src/core/lib/transport/connectivity_state.h", "src/core/lib/transport/metadata.h", @@ -5921,7 +5920,6 @@ "src/core/lib/surface/metadata_array.c", "src/core/lib/surface/server.c", "src/core/lib/surface/server.h", - "src/core/lib/surface/surface_trace.h", "src/core/lib/surface/validate_metadata.c", "src/core/lib/surface/version.c", "src/core/lib/transport/byte_stream.c", diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj b/vsprojects/vcxproj/grpc/grpc.vcxproj index ec0c984f2e7..c69623bf736 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj @@ -366,7 +366,6 @@ - diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters index 273f73b2a30..593024993fa 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters @@ -839,9 +839,6 @@ src\core\lib\surface - - src\core\lib\surface - src\core\lib\transport diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj index bba099d803c..28dcd8269eb 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj @@ -355,7 +355,6 @@ - diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters index 78a8777f942..058ae653cad 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters @@ -749,9 +749,6 @@ src\core\lib\surface - - src\core\lib\surface - src\core\lib\transport From dbe2b9e976d50ce7971df32dc9a3122525dad3ce Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Wed, 15 Jun 2016 20:23:04 -0700 Subject: [PATCH 060/215] Enable treating warnings as errors in objc tests --- src/objective-c/tests/GRPCClientTests.m | 16 ++++++++-------- src/objective-c/tests/InteropTests.m | 10 ++++++---- .../tests/Tests.xcodeproj/project.pbxproj | 4 ++++ 3 files changed, 18 insertions(+), 12 deletions(-) diff --git a/src/objective-c/tests/GRPCClientTests.m b/src/objective-c/tests/GRPCClientTests.m index 9a8d4253241..2eca7bf5498 100644 --- a/src/objective-c/tests/GRPCClientTests.m +++ b/src/objective-c/tests/GRPCClientTests.m @@ -48,9 +48,9 @@ static NSString * const kPackage = @"grpc.testing"; static NSString * const kService = @"TestService"; static NSString * const kRemoteSSLHost = @"grpc-test.sandbox.googleapis.com"; -static ProtoMethod *kInexistentMethod; -static ProtoMethod *kEmptyCallMethod; -static ProtoMethod *kUnaryCallMethod; +static GRPCProtoMethod *kInexistentMethod; +static GRPCProtoMethod *kEmptyCallMethod; +static GRPCProtoMethod *kUnaryCallMethod; /** Observer class for testing that responseMetadata is KVO-compliant */ @interface PassthroughObserver : NSObject @@ -109,13 +109,13 @@ static ProtoMethod *kUnaryCallMethod; [GRPCCall useInsecureConnectionsForHost:kHostAddress]; // This method isn't implemented by the remote server. - kInexistentMethod = [[ProtoMethod alloc] initWithPackage:kPackage + kInexistentMethod = [[GRPCProtoMethod alloc] initWithPackage:kPackage service:kService method:@"Inexistent"]; - kEmptyCallMethod = [[ProtoMethod alloc] initWithPackage:kPackage + kEmptyCallMethod = [[GRPCProtoMethod alloc] initWithPackage:kPackage service:kService method:@"EmptyCall"]; - kUnaryCallMethod = [[ProtoMethod alloc] initWithPackage:kPackage + kUnaryCallMethod = [[GRPCProtoMethod alloc] initWithPackage:kPackage service:kService method:@"UnaryCall"]; } @@ -303,7 +303,7 @@ static ProtoMethod *kUnaryCallMethod; // Try to set parameters to nil for GRPCCall. This should cause an exception @try { - GRPCCall *call = [[GRPCCall alloc] initWithHost:nil + GRPCCall *call __unused = [[GRPCCall alloc] initWithHost:nil path:nil requestsWriter:nil]; XCTFail(@"Did not receive an exception when parameters are nil"); @@ -316,7 +316,7 @@ static ProtoMethod *kUnaryCallMethod; GRXWriter *requestsWriter = [GRXWriter emptyWriter]; [requestsWriter finishWithError:nil]; @try { - GRPCCall *call = [[GRPCCall alloc] initWithHost:kHostAddress + GRPCCall *call __unused = [[GRPCCall alloc] initWithHost:kHostAddress path:kUnaryCallMethod.HTTPPath requestsWriter:requestsWriter]; XCTFail(@"Did not receive an exception when GRXWriter has incorrect state."); diff --git a/src/objective-c/tests/InteropTests.m b/src/objective-c/tests/InteropTests.m index 781c500f73e..60a83259fa6 100644 --- a/src/objective-c/tests/InteropTests.m +++ b/src/objective-c/tests/InteropTests.m @@ -58,7 +58,7 @@ requestedResponseSize:(NSNumber *)responseSize { RMTStreamingOutputCallRequest *request = [self message]; RMTResponseParameters *parameters = [RMTResponseParameters message]; - parameters.size = responseSize.integerValue; + parameters.size = (int)responseSize.integerValue; [request.responseParametersArray addObject:parameters]; request.payload.body = [NSMutableData dataWithLength:payloadSize.unsignedIntegerValue]; return request; @@ -80,7 +80,9 @@ #pragma mark Tests +#ifdef GRPC_COMPILE_WITH_CRONET static cronet_engine *cronetEngine = NULL; +#endif @implementation InteropTests { RMTTestService *_service; @@ -186,7 +188,7 @@ static cronet_engine *cronetEngine = NULL; RMTStreamingOutputCallRequest *request = [RMTStreamingOutputCallRequest message]; for (NSNumber *size in expectedSizes) { RMTResponseParameters *parameters = [RMTResponseParameters message]; - parameters.size = [size integerValue]; + parameters.size = (int)[size integerValue]; [request.responseParametersArray addObject:parameters]; } @@ -282,7 +284,7 @@ static cronet_engine *cronetEngine = NULL; // A buffered pipe to which we never write any value acts as a writer that just hangs. GRXBufferedPipe *requestsBuffer = [[GRXBufferedPipe alloc] init]; - ProtoRPC *call = [_service RPCToStreamingInputCallWithRequestsWriter:requestsBuffer + GRPCProtoCall *call = [_service RPCToStreamingInputCallWithRequestsWriter:requestsBuffer handler:^(RMTStreamingInputCallResponse *response, NSError *error) { XCTAssertEqual(error.code, GRPC_STATUS_CANCELLED); @@ -313,7 +315,7 @@ static cronet_engine *cronetEngine = NULL; [requestsBuffer writeValue:request]; - __block ProtoRPC *call = + __block GRPCProtoCall *call = [_service RPCToFullDuplexCallWithRequestsWriter:requestsBuffer eventHandler:^(BOOL done, RMTStreamingOutputCallResponse *response, diff --git a/src/objective-c/tests/Tests.xcodeproj/project.pbxproj b/src/objective-c/tests/Tests.xcodeproj/project.pbxproj index b0429617c01..59a6af2e11e 100644 --- a/src/objective-c/tests/Tests.xcodeproj/project.pbxproj +++ b/src/objective-c/tests/Tests.xcodeproj/project.pbxproj @@ -823,6 +823,7 @@ "$(inherited)", ); GCC_SYMBOLS_PRIVATE_EXTERN = NO; + GCC_TREAT_WARNINGS_AS_ERRORS = YES; GCC_WARN_64_TO_32_BIT_CONVERSION = YES; GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; GCC_WARN_UNDECLARED_SELECTOR = YES; @@ -859,6 +860,7 @@ ENABLE_STRICT_OBJC_MSGSEND = YES; GCC_C_LANGUAGE_STANDARD = gnu99; GCC_NO_COMMON_BLOCKS = YES; + GCC_TREAT_WARNINGS_AS_ERRORS = YES; GCC_WARN_64_TO_32_BIT_CONVERSION = YES; GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; GCC_WARN_UNDECLARED_SELECTOR = YES; @@ -875,6 +877,7 @@ 635697DC1B14FC11007A7283 /* Debug */ = { isa = XCBuildConfiguration; buildSettings = { + GCC_TREAT_WARNINGS_AS_ERRORS = YES; PRODUCT_NAME = "$(TARGET_NAME)"; SKIP_INSTALL = YES; }; @@ -883,6 +886,7 @@ 635697DD1B14FC11007A7283 /* Release */ = { isa = XCBuildConfiguration; buildSettings = { + GCC_TREAT_WARNINGS_AS_ERRORS = YES; PRODUCT_NAME = "$(TARGET_NAME)"; SKIP_INSTALL = YES; }; From bb2bd6553938ca1a192414084e5800178f67c4a3 Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Thu, 16 Jun 2016 09:29:53 -0700 Subject: [PATCH 061/215] Clang format --- src/core/lib/surface/completion_queue.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/core/lib/surface/completion_queue.c b/src/core/lib/surface/completion_queue.c index de2b674d0e0..03de97bc2b5 100644 --- a/src/core/lib/surface/completion_queue.c +++ b/src/core/lib/surface/completion_queue.c @@ -93,12 +93,12 @@ static grpc_completion_queue *g_freelist; int grpc_cq_pluck_trace; int grpc_cq_event_timeout_trace; -#define GRPC_SURFACE_TRACE_RETURNED_EVENT(cq, event) \ - if (grpc_api_trace && \ - (grpc_cq_pluck_trace || (event)->type != GRPC_QUEUE_TIMEOUT)) { \ - char *_ev = grpc_event_string(event); \ - gpr_log(GPR_INFO, "RETURN_EVENT[%p]: %s", cq, _ev); \ - gpr_free(_ev); \ +#define GRPC_SURFACE_TRACE_RETURNED_EVENT(cq, event) \ + if (grpc_api_trace && \ + (grpc_cq_pluck_trace || (event)->type != GRPC_QUEUE_TIMEOUT)) { \ + char *_ev = grpc_event_string(event); \ + gpr_log(GPR_INFO, "RETURN_EVENT[%p]: %s", cq, _ev); \ + gpr_free(_ev); \ } From a7cc90055e91b1bda2e916c5884b5305c7f73b53 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 16 Jun 2016 09:59:51 -0700 Subject: [PATCH 062/215] Modified route_guide sample app to show RPC log Modified the screens to show meaningful information about RPC progress. When you click on each tab the screen now shows log messages that previously only went to the debug console. --- .../Misc/Base.lproj/Main.storyboard | 108 ++++++++++-------- .../objective-c/route_guide/ViewControllers.m | 41 +++++++ 2 files changed, 102 insertions(+), 47 deletions(-) diff --git a/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard b/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard index 9bf9498d62c..306320f7d89 100644 --- a/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard +++ b/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard @@ -1,7 +1,8 @@ - + - + + @@ -16,33 +17,35 @@ - - - - - + + + + - + @@ -56,29 +59,35 @@ - + + + - - - - + + + + + @@ -117,29 +126,32 @@ - - - - - + + + + + @@ -157,29 +169,31 @@ - - - - - + + + + diff --git a/examples/objective-c/route_guide/ViewControllers.m b/examples/objective-c/route_guide/ViewControllers.m index e32978240b6..b916a4ee0c4 100644 --- a/examples/objective-c/route_guide/ViewControllers.m +++ b/examples/objective-c/route_guide/ViewControllers.m @@ -83,6 +83,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface GetFeatureViewController : UIViewController { RTGRouteGuide *service; } +@property (weak, nonatomic) IBOutlet UILabel *output_label; @end @implementation GetFeatureViewController @@ -90,10 +91,16 @@ static NSString * const kHostAddress = @"localhost:50051"; - (void)execRequest { void (^handler)(RTGFeature *response, NSError *error) = ^(RTGFeature *response, NSError *error) { if (response.name.length) { + NSString *str =[NSString stringWithFormat:@"%@\nFound feature called %@ at %@.", self.output_label.text, response.location, response.name]; + self.output_label.text = str; NSLog(@"Found feature called %@ at %@.", response.name, response.location); } else if (response) { + NSString *str =[NSString stringWithFormat:@"%@\nFound no features at %@", self.output_label.text,response.location]; + self.output_label.text = str; NSLog(@"Found no features at %@", response.location); } else { + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; + self.output_label.text = str; NSLog(@"RPC error: %@", error); } }; @@ -116,6 +123,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { + self.output_label.text = @"RPC log:"; + self.output_label.numberOfLines = 0; + self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } @@ -131,6 +141,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface ListFeaturesViewController : UIViewController { RTGRouteGuide *service; } +@property (weak, nonatomic) IBOutlet UILabel *output_label; @end @@ -147,8 +158,12 @@ static NSString * const kHostAddress = @"localhost:50051"; [service listFeaturesWithRequest:rectangle eventHandler:^(BOOL done, RTGFeature *response, NSError *error) { if (response) { + NSString *str =[NSString stringWithFormat:@"%@\nFound feature at %@ called %@.", self.output_label.text, response.location, response.name]; + self.output_label.text = str; NSLog(@"Found feature at %@ called %@.", response.location, response.name); } else if (error) { + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; + self.output_label.text = str; NSLog(@"RPC error: %@", error); } }]; @@ -161,6 +176,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { + self.output_label.text = @"RPC log:"; + self.output_label.numberOfLines = 0; + self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } @@ -177,6 +195,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface RecordRouteViewController : UIViewController { RTGRouteGuide *service; } +@property (weak, nonatomic) IBOutlet UILabel *output_label; @end @@ -192,6 +211,8 @@ static NSString * const kHostAddress = @"localhost:50051"; RTGPoint *location = [RTGPoint message]; location.longitude = [((NSNumber *) feature[@"location"][@"longitude"]) intValue]; location.latitude = [((NSNumber *) feature[@"location"][@"latitude"]) intValue]; + NSString *str =[NSString stringWithFormat:@"%@\nVisiting point %@", self.output_label.text, location]; + self.output_label.text = str; NSLog(@"Visiting point %@", location); return location; }]; @@ -199,11 +220,19 @@ static NSString * const kHostAddress = @"localhost:50051"; [service recordRouteWithRequestsWriter:locations handler:^(RTGRouteSummary *response, NSError *error) { if (response) { + NSString *str =[NSString stringWithFormat: + @"%@\nFinished trip with %i points\nPassed %i features\n" + "Travelled %i meters\nIt took %i seconds", + self.output_label.text, response.pointCount, response.featureCount, + response.distance, response.elapsedTime]; + self.output_label.text = str; NSLog(@"Finished trip with %i points", response.pointCount); NSLog(@"Passed %i features", response.featureCount); NSLog(@"Travelled %i meters", response.distance); NSLog(@"It took %i seconds", response.elapsedTime); } else { + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; + self.output_label.text = str; NSLog(@"RPC error: %@", error); } }]; @@ -216,6 +245,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { + self.output_label.text = @"RPC log:"; + self.output_label.numberOfLines = 0; + self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } @@ -231,6 +263,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface RouteChatViewController : UIViewController { RTGRouteGuide *service; } +@property (weak, nonatomic) IBOutlet UILabel *output_label; @end @@ -249,8 +282,13 @@ static NSString * const kHostAddress = @"localhost:50051"; [service routeChatWithRequestsWriter:notesWriter eventHandler:^(BOOL done, RTGRouteNote *note, NSError *error) { if (note) { + NSString *str =[NSString stringWithFormat:@"%@\nGot message %@ at %@", + self.output_label.text, note.message, note.location]; + self.output_label.text = str; NSLog(@"Got message %@ at %@", note.message, note.location); } else if (error) { + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; + self.output_label.text = str; NSLog(@"RPC error: %@", error); } if (done) { @@ -266,6 +304,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { + self.output_label.text = @"RPC log:"; + self.output_label.numberOfLines = 0; + self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } From 3eba9075ef612f83c363868c24d106580ab25223 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 16 Jun 2016 10:15:32 -0700 Subject: [PATCH 063/215] using camelCase for output_label now --- .../Misc/Base.lproj/Main.storyboard | 8 +- .../objective-c/route_guide/ViewControllers.m | 74 +++++++++---------- 2 files changed, 41 insertions(+), 41 deletions(-) diff --git a/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard b/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard index 306320f7d89..5ca9f4642b9 100644 --- a/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard +++ b/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard @@ -40,7 +40,7 @@ - + @@ -86,7 +86,7 @@ - + @@ -150,7 +150,7 @@ - + @@ -192,7 +192,7 @@ - + diff --git a/examples/objective-c/route_guide/ViewControllers.m b/examples/objective-c/route_guide/ViewControllers.m index b916a4ee0c4..1e84c334728 100644 --- a/examples/objective-c/route_guide/ViewControllers.m +++ b/examples/objective-c/route_guide/ViewControllers.m @@ -37,7 +37,7 @@ #import #import -static NSString * const kHostAddress = @"localhost:50051"; +static NSString * const kHostAddress = @"192.168.1.120:50051"; /** Category to override RTGPoint's description. */ @interface RTGPoint (Description) @@ -83,7 +83,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface GetFeatureViewController : UIViewController { RTGRouteGuide *service; } -@property (weak, nonatomic) IBOutlet UILabel *output_label; +@property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @implementation GetFeatureViewController @@ -91,16 +91,16 @@ static NSString * const kHostAddress = @"localhost:50051"; - (void)execRequest { void (^handler)(RTGFeature *response, NSError *error) = ^(RTGFeature *response, NSError *error) { if (response.name.length) { - NSString *str =[NSString stringWithFormat:@"%@\nFound feature called %@ at %@.", self.output_label.text, response.location, response.name]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nFound feature called %@ at %@.", self.outputLabel.text, response.location, response.name]; + self.outputLabel.text = str; NSLog(@"Found feature called %@ at %@.", response.name, response.location); } else if (response) { - NSString *str =[NSString stringWithFormat:@"%@\nFound no features at %@", self.output_label.text,response.location]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nFound no features at %@", self.outputLabel.text,response.location]; + self.outputLabel.text = str; NSLog(@"Found no features at %@", response.location); } else { - NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.outputLabel.text, error]; + self.outputLabel.text = str; NSLog(@"RPC error: %@", error); } }; @@ -123,9 +123,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { - self.output_label.text = @"RPC log:"; - self.output_label.numberOfLines = 0; - self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; + self.outputLabel.text = @"RPC log:"; + self.outputLabel.numberOfLines = 0; + self.outputLabel.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } @@ -141,7 +141,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface ListFeaturesViewController : UIViewController { RTGRouteGuide *service; } -@property (weak, nonatomic) IBOutlet UILabel *output_label; +@property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @@ -158,12 +158,12 @@ static NSString * const kHostAddress = @"localhost:50051"; [service listFeaturesWithRequest:rectangle eventHandler:^(BOOL done, RTGFeature *response, NSError *error) { if (response) { - NSString *str =[NSString stringWithFormat:@"%@\nFound feature at %@ called %@.", self.output_label.text, response.location, response.name]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nFound feature at %@ called %@.", self.outputLabel.text, response.location, response.name]; + self.outputLabel.text = str; NSLog(@"Found feature at %@ called %@.", response.location, response.name); } else if (error) { - NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.outputLabel.text, error]; + self.outputLabel.text = str; NSLog(@"RPC error: %@", error); } }]; @@ -176,9 +176,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { - self.output_label.text = @"RPC log:"; - self.output_label.numberOfLines = 0; - self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; + self.outputLabel.text = @"RPC log:"; + self.outputLabel.numberOfLines = 0; + self.outputLabel.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } @@ -195,7 +195,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface RecordRouteViewController : UIViewController { RTGRouteGuide *service; } -@property (weak, nonatomic) IBOutlet UILabel *output_label; +@property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @@ -211,8 +211,8 @@ static NSString * const kHostAddress = @"localhost:50051"; RTGPoint *location = [RTGPoint message]; location.longitude = [((NSNumber *) feature[@"location"][@"longitude"]) intValue]; location.latitude = [((NSNumber *) feature[@"location"][@"latitude"]) intValue]; - NSString *str =[NSString stringWithFormat:@"%@\nVisiting point %@", self.output_label.text, location]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nVisiting point %@", self.outputLabel.text, location]; + self.outputLabel.text = str; NSLog(@"Visiting point %@", location); return location; }]; @@ -223,16 +223,16 @@ static NSString * const kHostAddress = @"localhost:50051"; NSString *str =[NSString stringWithFormat: @"%@\nFinished trip with %i points\nPassed %i features\n" "Travelled %i meters\nIt took %i seconds", - self.output_label.text, response.pointCount, response.featureCount, + self.outputLabel.text, response.pointCount, response.featureCount, response.distance, response.elapsedTime]; - self.output_label.text = str; + self.outputLabel.text = str; NSLog(@"Finished trip with %i points", response.pointCount); NSLog(@"Passed %i features", response.featureCount); NSLog(@"Travelled %i meters", response.distance); NSLog(@"It took %i seconds", response.elapsedTime); } else { - NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.outputLabel.text, error]; + self.outputLabel.text = str; NSLog(@"RPC error: %@", error); } }]; @@ -245,9 +245,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { - self.output_label.text = @"RPC log:"; - self.output_label.numberOfLines = 0; - self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; + self.outputLabel.text = @"RPC log:"; + self.outputLabel.numberOfLines = 0; + self.outputLabel.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } @@ -263,7 +263,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface RouteChatViewController : UIViewController { RTGRouteGuide *service; } -@property (weak, nonatomic) IBOutlet UILabel *output_label; +@property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @@ -283,12 +283,12 @@ static NSString * const kHostAddress = @"localhost:50051"; eventHandler:^(BOOL done, RTGRouteNote *note, NSError *error) { if (note) { NSString *str =[NSString stringWithFormat:@"%@\nGot message %@ at %@", - self.output_label.text, note.message, note.location]; - self.output_label.text = str; + self.outputLabel.text, note.message, note.location]; + self.outputLabel.text = str; NSLog(@"Got message %@ at %@", note.message, note.location); } else if (error) { - NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.outputLabel.text, error]; + self.outputLabel.text = str; NSLog(@"RPC error: %@", error); } if (done) { @@ -304,9 +304,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { - self.output_label.text = @"RPC log:"; - self.output_label.numberOfLines = 0; - self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; + self.outputLabel.text = @"RPC log:"; + self.outputLabel.numberOfLines = 0; + self.outputLabel.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } From 05998dc6939bb7ad4230775bb5ac87696ae9e29b Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Thu, 16 Jun 2016 10:16:44 -0700 Subject: [PATCH 064/215] More formatting fixes --- src/core/lib/surface/completion_queue.c | 1 - 1 file changed, 1 deletion(-) diff --git a/src/core/lib/surface/completion_queue.c b/src/core/lib/surface/completion_queue.c index 03de97bc2b5..d70f1721463 100644 --- a/src/core/lib/surface/completion_queue.c +++ b/src/core/lib/surface/completion_queue.c @@ -101,7 +101,6 @@ int grpc_cq_event_timeout_trace; gpr_free(_ev); \ } - static void on_pollset_shutdown_done(grpc_exec_ctx *exec_ctx, void *cc, bool success); From 4412c85a1079f2ac55a9a27641076ca34c1a2c62 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 16 Jun 2016 10:37:51 -0700 Subject: [PATCH 065/215] crud removal --- examples/objective-c/route_guide/ViewControllers.m | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/objective-c/route_guide/ViewControllers.m b/examples/objective-c/route_guide/ViewControllers.m index 1e84c334728..5fd411e3ba6 100644 --- a/examples/objective-c/route_guide/ViewControllers.m +++ b/examples/objective-c/route_guide/ViewControllers.m @@ -37,7 +37,7 @@ #import #import -static NSString * const kHostAddress = @"192.168.1.120:50051"; +static NSString * const kHostAddress = @"localhost:50051"; /** Category to override RTGPoint's description. */ @interface RTGPoint (Description) From 00f41c1eed57f84a2efdfbd96a59dd00d523cde0 Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Thu, 16 Jun 2016 12:46:52 -0700 Subject: [PATCH 066/215] Add comment about new trace flags --- src/core/lib/surface/completion_queue.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/core/lib/surface/completion_queue.h b/src/core/lib/surface/completion_queue.h index 108445b28f1..e4018695e27 100644 --- a/src/core/lib/surface/completion_queue.h +++ b/src/core/lib/surface/completion_queue.h @@ -39,6 +39,8 @@ #include #include "src/core/lib/iomgr/pollset.h" +/* These trace flags default to 1. The corresponding lines are only traced + if grpc_api_trace is also truthy */ extern int grpc_cq_pluck_trace; extern int grpc_cq_event_timeout_trace; From 2e12db9c319bcbdbb2fa570149f88e4b496b558c Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 16 Jun 2016 16:53:59 -0700 Subject: [PATCH 067/215] Test polling island merges --- Makefile | 36 ++++ build.yaml | 12 ++ src/core/lib/iomgr/ev_epoll_linux.c | 51 +++++- src/core/lib/iomgr/ev_epoll_linux.h | 6 + src/core/lib/iomgr/ev_posix.c | 7 + src/core/lib/iomgr/ev_posix.h | 3 + test/core/iomgr/ev_epoll_linux_test.c | 222 +++++++++++++++++++++++ tools/run_tests/sources_and_headers.json | 16 ++ tools/run_tests/tests.json | 15 ++ 9 files changed, 366 insertions(+), 2 deletions(-) create mode 100644 test/core/iomgr/ev_epoll_linux_test.c diff --git a/Makefile b/Makefile index e615704395b..825684cc2dd 100644 --- a/Makefile +++ b/Makefile @@ -905,6 +905,7 @@ dns_resolver_connectivity_test: $(BINDIR)/$(CONFIG)/dns_resolver_connectivity_te dns_resolver_test: $(BINDIR)/$(CONFIG)/dns_resolver_test dualstack_socket_test: $(BINDIR)/$(CONFIG)/dualstack_socket_test endpoint_pair_test: $(BINDIR)/$(CONFIG)/endpoint_pair_test +ev_epoll_linux_test: $(BINDIR)/$(CONFIG)/ev_epoll_linux_test fd_conservation_posix_test: $(BINDIR)/$(CONFIG)/fd_conservation_posix_test fd_posix_test: $(BINDIR)/$(CONFIG)/fd_posix_test fling_client: $(BINDIR)/$(CONFIG)/fling_client @@ -1242,6 +1243,7 @@ buildtests_c: privatelibs_c \ $(BINDIR)/$(CONFIG)/dns_resolver_test \ $(BINDIR)/$(CONFIG)/dualstack_socket_test \ $(BINDIR)/$(CONFIG)/endpoint_pair_test \ + $(BINDIR)/$(CONFIG)/ev_epoll_linux_test \ $(BINDIR)/$(CONFIG)/fd_conservation_posix_test \ $(BINDIR)/$(CONFIG)/fd_posix_test \ $(BINDIR)/$(CONFIG)/fling_client \ @@ -1512,6 +1514,8 @@ test_c: buildtests_c $(Q) $(BINDIR)/$(CONFIG)/dualstack_socket_test || ( echo test dualstack_socket_test failed ; exit 1 ) $(E) "[RUN] Testing endpoint_pair_test" $(Q) $(BINDIR)/$(CONFIG)/endpoint_pair_test || ( echo test endpoint_pair_test failed ; exit 1 ) + $(E) "[RUN] Testing ev_epoll_linux_test" + $(Q) $(BINDIR)/$(CONFIG)/ev_epoll_linux_test || ( echo test ev_epoll_linux_test failed ; exit 1 ) $(E) "[RUN] Testing fd_conservation_posix_test" $(Q) $(BINDIR)/$(CONFIG)/fd_conservation_posix_test || ( echo test fd_conservation_posix_test failed ; exit 1 ) $(E) "[RUN] Testing fd_posix_test" @@ -7130,6 +7134,38 @@ endif endif +EV_EPOLL_LINUX_TEST_SRC = \ + test/core/iomgr/ev_epoll_linux_test.c \ + +EV_EPOLL_LINUX_TEST_OBJS = $(addprefix $(OBJDIR)/$(CONFIG)/, $(addsuffix .o, $(basename $(EV_EPOLL_LINUX_TEST_SRC)))) +ifeq ($(NO_SECURE),true) + +# You can't build secure targets if you don't have OpenSSL. + +$(BINDIR)/$(CONFIG)/ev_epoll_linux_test: openssl_dep_error + +else + + + +$(BINDIR)/$(CONFIG)/ev_epoll_linux_test: $(EV_EPOLL_LINUX_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a + $(E) "[LD] Linking $@" + $(Q) mkdir -p `dirname $@` + $(Q) $(LD) $(LDFLAGS) $(EV_EPOLL_LINUX_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a $(LDLIBS) $(LDLIBS_SECURE) -o $(BINDIR)/$(CONFIG)/ev_epoll_linux_test + +endif + +$(OBJDIR)/$(CONFIG)/test/core/iomgr/ev_epoll_linux_test.o: $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a + +deps_ev_epoll_linux_test: $(EV_EPOLL_LINUX_TEST_OBJS:.o=.dep) + +ifneq ($(NO_SECURE),true) +ifneq ($(NO_DEPS),true) +-include $(EV_EPOLL_LINUX_TEST_OBJS:.o=.dep) +endif +endif + + FD_CONSERVATION_POSIX_TEST_SRC = \ test/core/iomgr/fd_conservation_posix_test.c \ diff --git a/build.yaml b/build.yaml index 7790e0c5174..84f4ea521be 100644 --- a/build.yaml +++ b/build.yaml @@ -1407,6 +1407,18 @@ targets: - grpc - gpr_test_util - gpr +- name: ev_epoll_linux_test + build: test + language: c + src: + - test/core/iomgr/ev_epoll_linux_test.c + deps: + - grpc_test_util + - grpc + - gpr_test_util + - gpr + platforms: + - linux - name: fd_conservation_posix_test build: test language: c diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 1fb59474640..ed2c494b783 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -317,8 +317,9 @@ static void polling_island_remove_all_fds_locked(polling_island *pi, if (err < 0 && errno != ENOENT) { /* TODO: sreek - We need a better way to bubble up this error instead of * just logging a message */ - gpr_log(GPR_ERROR, "epoll_ctl deleting fds[%zu]: %d failed with error: %s", - i, pi->fds[i]->fd, strerror(errno)); + gpr_log(GPR_ERROR, + "epoll_ctl deleting fds[%zu]: %d failed with error: %s", i, + pi->fds[i]->fd, strerror(errno)); } if (remove_fd_refs) { @@ -1458,6 +1459,52 @@ static void pollset_set_del_pollset_set(grpc_exec_ctx *exec_ctx, gpr_mu_unlock(&bag->mu); } +/* Test helper functions + * */ +void *grpc_fd_get_polling_island(grpc_fd *fd) { + polling_island *pi; + + gpr_mu_lock(&fd->pi_mu); + pi = fd->polling_island; + gpr_mu_unlock(&fd->pi_mu); + + return pi; +} + +void *grpc_pollset_get_polling_island(grpc_pollset *ps) { + polling_island *pi; + + gpr_mu_lock(&ps->pi_mu); + pi = ps->polling_island; + gpr_mu_unlock(&ps->pi_mu); + + return pi; +} + +static polling_island *get_polling_island(polling_island *p) { + if (p == NULL) { + return NULL; + } + + polling_island *next; + gpr_mu_lock(&p->mu); + while (p->merged_to != NULL) { + next = p->merged_to; + gpr_mu_unlock(&p->mu); + p = next; + gpr_mu_lock(&p->mu); + } + gpr_mu_unlock(&p->mu); + + return p; +} + +bool grpc_are_polling_islands_equal(void *p, void *q) { + p = get_polling_island(p); + q = get_polling_island(q); + return p == q; +} + /******************************************************************************* * Event engine binding */ diff --git a/src/core/lib/iomgr/ev_epoll_linux.h b/src/core/lib/iomgr/ev_epoll_linux.h index 8c819975a4c..7a494aba198 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.h +++ b/src/core/lib/iomgr/ev_epoll_linux.h @@ -38,4 +38,10 @@ const grpc_event_engine_vtable *grpc_init_epoll_linux(void); +#ifdef GPR_LINUX_EPOLL +void *grpc_fd_get_polling_island(grpc_fd *fd); +void *grpc_pollset_get_polling_island(grpc_pollset *ps); +bool grpc_are_polling_islands_equal(void *p, void *q); +#endif /* defined(GPR_LINUX_EPOLL) */ + #endif /* GRPC_CORE_LIB_IOMGR_EV_EPOLL_LINUX_H */ diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index 2b15967adcc..5b20600a6f7 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -54,6 +54,7 @@ grpc_poll_function_type grpc_poll_function = poll; static const grpc_event_engine_vtable *g_event_engine; +static const char* g_poll_strategy_name = NULL; typedef const grpc_event_engine_vtable *(*event_engine_factory_fn)(void); @@ -101,6 +102,7 @@ static void try_engine(const char *engine) { for (size_t i = 0; i < GPR_ARRAY_SIZE(g_factories); i++) { if (is(engine, g_factories[i].name)) { if ((g_event_engine = g_factories[i].factory())) { + g_poll_strategy_name = g_factories[i].name; gpr_log(GPR_DEBUG, "Using polling engine: %s", g_factories[i].name); return; } @@ -108,6 +110,11 @@ static void try_engine(const char *engine) { } } +/* Call this only after calling grpc_event_engine_init() */ +const char *grpc_get_poll_strategy_name() { + return g_poll_strategy_name; +} + void grpc_event_engine_init(void) { char *s = gpr_getenv("GRPC_POLL_STRATEGY"); if (s == NULL) { diff --git a/src/core/lib/iomgr/ev_posix.h b/src/core/lib/iomgr/ev_posix.h index 344bf63438a..3ed5a5f9562 100644 --- a/src/core/lib/iomgr/ev_posix.h +++ b/src/core/lib/iomgr/ev_posix.h @@ -98,6 +98,9 @@ typedef struct grpc_event_engine_vtable { void grpc_event_engine_init(void); void grpc_event_engine_shutdown(void); +/* Return the name of the poll strategy */ +const char* grpc_get_poll_strategy_name(); + /* Create a wrapped file descriptor. Requires fd is a non-blocking file descriptor. This takes ownership of closing fd. */ diff --git a/test/core/iomgr/ev_epoll_linux_test.c b/test/core/iomgr/ev_epoll_linux_test.c new file mode 100644 index 00000000000..51da15faa7a --- /dev/null +++ b/test/core/iomgr/ev_epoll_linux_test.c @@ -0,0 +1,222 @@ +/* + * + * Copyright 2015, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include "src/core/lib/iomgr/ev_epoll_linux.h" +#include "src/core/lib/iomgr/ev_posix.h" + +#include +#include +#include +#include + +#include +#include + +#include "src/core/lib/iomgr/iomgr.h" +#include "test/core/util/test_config.h" + +typedef struct test_pollset { + grpc_pollset *pollset; + gpr_mu *mu; +} test_pollset; + +typedef struct test_fd { + int inner_fd; + grpc_fd *fd; +} test_fd; + +static void test_fd_init(test_fd *fds, int num_fds) { + int i; + for (i = 0; i < num_fds; i++) { + fds[i].inner_fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + fds[i].fd = grpc_fd_create(fds[i].inner_fd, "test_fd"); + } +} + +static void test_fd_cleanup(grpc_exec_ctx *exec_ctx, test_fd *fds, + int num_fds) { + int release_fd; + int i; + + for (i = 0; i < num_fds; i++) { + grpc_fd_shutdown(exec_ctx, fds[i].fd); + grpc_exec_ctx_flush(exec_ctx); + + grpc_fd_orphan(exec_ctx, fds[i].fd, NULL, &release_fd, "test_fd_cleanup"); + grpc_exec_ctx_flush(exec_ctx); + + GPR_ASSERT(release_fd == fds[i].inner_fd); + close(fds[i].inner_fd); + } +} + +static void test_pollset_init(test_pollset *pollsets, int num_pollsets) { + int i; + for (i = 0; i < num_pollsets; i++) { + pollsets[i].pollset = gpr_malloc(grpc_pollset_size()); + grpc_pollset_init(pollsets[i].pollset, &pollsets[i].mu); + } +} + +static void destroy_pollset(grpc_exec_ctx *exec_ctx, void *p, bool success) { + grpc_pollset_destroy(p); +} + +static void test_pollset_cleanup(grpc_exec_ctx *exec_ctx, + test_pollset *pollsets, int num_pollsets) { + grpc_closure destroyed; + int i; + + for (i = 0; i < num_pollsets; i++) { + grpc_closure_init(&destroyed, destroy_pollset, pollsets[i].pollset); + grpc_pollset_shutdown(exec_ctx, pollsets[i].pollset, &destroyed); + + grpc_exec_ctx_flush(exec_ctx); + gpr_free(pollsets[i].pollset); + } +} + +#define NUM_FDS 8 +#define NUM_POLLSETS 4 +/* + * Cases to test: + * case 1) Polling islands of both fd and pollset are NULL + * case 2) Polling island of fd is NULL but that of pollset is not-NULL + * case 3) Polling island of fd is not-NULL but that of pollset is NULL + * case 4) Polling islands of both fd and pollset are not-NULL and: + * case 4.1) Polling islands of fd and pollset are equal + * case 4.2) Polling islands of fd and pollset are NOT-equal (This results + * in a merge) + * */ +static void test_add_fd_to_pollset() { + grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; + test_fd fds[NUM_FDS]; + test_pollset pollsets[NUM_POLLSETS]; + void *expected_pi = NULL; + int i; + + test_fd_init(fds, NUM_FDS); + test_pollset_init(pollsets, NUM_POLLSETS); + + /*Step 1. + * Create three polling islands (This will exercise test case 1 and 2) with + * the following configuration: + * polling island 0 = { fds:0,1,2, pollsets:0} + * polling island 1 = { fds:3,4, pollsets:1} + * polling island 2 = { fds:5,6,7 pollsets:2} + * + *Step 2. + * Add pollset 3 to polling island 0 (by adding fds 0 and 1 to pollset 3) + * (This will exercise test cases 3 and 4.1). The configuration becomes: + * polling island 0 = { fds:0,1,2, pollsets:0,3} <<< pollset 3 added here + * polling island 1 = { fds:3,4, pollsets:1} + * polling island 2 = { fds:5,6,7 pollsets:2} + * + *Step 3. + * Merge polling islands 0 and 1 by adding fd 0 to pollset 1 (This will + * exercise test case 4.2). The configuration becomes: + * polling island (merged) = {fds: 0,1,2,3,4, pollsets: 0,1,3} + * polling island 2 = {fds: 5,6,7 pollsets: 2} + * + *Step 4. + * Finally do one more merge by adding fd 3 to pollset 2. + * polling island (merged) = {fds: 0,1,2,3,4,5,6,7, pollsets: 0,1,2,3} + */ + + /* == Step 1 == */ + for (i = 0; i <= 2; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[0].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + for (i = 3; i <= 4; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + for (i = 5; i <= 7; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + /* == Step 2 == */ + for (i = 0; i <= 1; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[3].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + /* == Step 3 == */ + grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, fds[0].fd); + grpc_exec_ctx_flush(&exec_ctx); + + /* == Step 4 == */ + grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, fds[3].fd); + grpc_exec_ctx_flush(&exec_ctx); + + /* All polling islands are merged at this point */ + + /* Compare Fd:0's polling island with that of all other Fds */ + expected_pi = grpc_fd_get_polling_island(fds[0].fd); + for (i = 1; i < NUM_FDS; i++) { + GPR_ASSERT(grpc_are_polling_islands_equal( + expected_pi, grpc_fd_get_polling_island(fds[i].fd))); + } + + /* Compare Fd:0's polling island with that of all other pollsets */ + for (i = 0; i < NUM_POLLSETS; i++) { + GPR_ASSERT(grpc_are_polling_islands_equal( + expected_pi, grpc_pollset_get_polling_island(pollsets[i].pollset))); + } + + test_fd_cleanup(&exec_ctx, fds, NUM_FDS); + test_pollset_cleanup(&exec_ctx, pollsets, NUM_POLLSETS); + grpc_exec_ctx_finish(&exec_ctx); +} + +int main(int argc, char **argv) { + const char *poll_strategy = NULL; + grpc_test_init(argc, argv); + grpc_iomgr_init(); + + poll_strategy = grpc_get_poll_strategy_name(); + if (poll_strategy != NULL && strcmp(poll_strategy, "epoll") == 0) { + test_add_fd_to_pollset(); + } else { + gpr_log(GPR_INFO, + "Skipping the test. The test is only relevant for 'epoll' " + "strategy. and the current strategy is: '%s'", + poll_strategy); + } + grpc_iomgr_shutdown(); + return 0; +} diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index e8ff61dc3fb..e9df72e43a1 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -315,6 +315,22 @@ "third_party": false, "type": "target" }, + { + "deps": [ + "gpr", + "gpr_test_util", + "grpc", + "grpc_test_util" + ], + "headers": [], + "language": "c", + "name": "ev_epoll_linux_test", + "src": [ + "test/core/iomgr/ev_epoll_linux_test.c" + ], + "third_party": false, + "type": "target" + }, { "deps": [ "gpr", diff --git a/tools/run_tests/tests.json b/tools/run_tests/tests.json index 5a84a41b638..ba661840dad 100644 --- a/tools/run_tests/tests.json +++ b/tools/run_tests/tests.json @@ -377,6 +377,21 @@ "windows" ] }, + { + "args": [], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "gtest": false, + "language": "c", + "name": "ev_epoll_linux_test", + "platforms": [ + "linux" + ] + }, { "args": [], "ci_platforms": [ From 87e6770a1819d44f352e7ba5782ca1f7dd8cffca Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Thu, 16 Jun 2016 22:04:08 -0700 Subject: [PATCH 068/215] pr comments --- doc/compression.md | 40 +++++++++++++++++++++++----------------- 1 file changed, 23 insertions(+), 17 deletions(-) diff --git a/doc/compression.md b/doc/compression.md index ddbbff1c541..6e455636021 100644 --- a/doc/compression.md +++ b/doc/compression.md @@ -41,13 +41,14 @@ of the request, including not performing any compression, regardless of channel and RPC settings (for example, if compression would result in small or negative gains). -A compressed message from a client with an algorithm unsupported by a server, -WILL result in an INVALID\_ARGUMENT error, alongside the receiving peer's -`grpc-accept-encoding` header specifying the algorithms it accepts. If an -INTERNAL error is returned from the server despite having used one of the -algorithms from the `grpc-accept-encoding h`eader, the cause MUST NOT be related -to compression. Data sent from a server compressed with an algorithm not -supported by the client will also result in an INTERNAL error. +When a message from a client compressed with an unsupported algorithm is +processed by a server, it WILL result in an INVALID\_ARGUMENT error. The server +will include in its response a `grpc-accept-encoding` header specifying the +algorithms it does accept. If an INTERNAL error is returned from the server +despite having used one of the algorithms from the `grpc-accept-encoding` +header, the cause MUST NOT be related to compression. Data sent from a server +compressed with an algorithm not supported by the client WILL result in an +INTERNAL error on the client side. Note that a peer MAY choose to not disclose all the encodings it supports. However, if it receives a message compressed in an undisclosed but supported @@ -67,16 +68,21 @@ cases. ### Compression Levels and Algorithms -We currently (as of July 2015) support _gzip_ and _deflate_ as algorithms (with -the possible future addition of snappy). In order to simplify the public API, -it's intended to abstract the algorithms as _compression levels_ (such as "low", -"medium", "high") that'd map to concrete algorithms and/or their settings (such -as "low" mapping to "gzip -3" and "high" mapping to "gzip -9"). However, we -can't presently (July 2015) implement said compression levels at the client side -without either a initial negotiation of capabilities or an automatic retry -mechanism. Therefore, compression levels are only supported at the server side, -which is aware of the client's capabilities by virtue of the incoming -Message-Accept-Encoding header. +The set of supported algorithm is implementation dependent. In order to simplify +the public API and to operate seamlessly across implementations (both in terms +of languages but also different version of the same one), we introduce the idea +of _compression levels_ (such as "low", "medium", "high"). + +Levels map to concrete algorithms and/or their settings (such as "low" mapping +to "gzip -3" and "high" mapping to "gzip -9") automatically depending on what a +peer is known to support. A server is always aware of what its clients support, +as clients disclose it in their Message-Accept-Encoding header as part of their +initial call. A client doesn't a priori (presently) know which algorithms a +server supports. This issue can be addressed with an initial negotiation of +capabilities or an automatic retry mechanism. These features will be implemented +in the future. Currently however, compression levels are only supported at the +server side, which is aware of the client's capabilities by virtue of the +incoming. ### Propagation to child RPCs From b9df2760ed15930429305f01407914db106510d5 Mon Sep 17 00:00:00 2001 From: vjpai Date: Fri, 17 Jun 2016 00:27:00 -0700 Subject: [PATCH 069/215] Add proper documentation of ordering and thread-safety for streaming (sync and async) APIs --- include/grpc++/impl/codegen/async_stream.h | 10 +++++++++- include/grpc++/impl/codegen/completion_queue.h | 14 ++++++++++++-- include/grpc++/impl/codegen/sync_stream.h | 10 ++++++++-- 3 files changed, 29 insertions(+), 5 deletions(-) diff --git a/include/grpc++/impl/codegen/async_stream.h b/include/grpc++/impl/codegen/async_stream.h index 0606441549e..c74737ce5f8 100644 --- a/include/grpc++/impl/codegen/async_stream.h +++ b/include/grpc++/impl/codegen/async_stream.h @@ -52,11 +52,14 @@ class ClientAsyncStreamingInterface { /// Request notification of the reading of the initial metadata. Completion /// will be notified by \a tag on the associated completion queue. + /// This call is optional, but if it is used, it cannot be used concurrently + /// with Read /// /// \param[in] tag Tag identifying this request. virtual void ReadInitialMetadata(void* tag) = 0; - /// Request notification completion. + /// Indicate that the stream is to be finished and request notification + /// Should not be used concurrently with other operations /// /// \param[out] status To be updated with the operation status. /// \param[in] tag Tag identifying this request. @@ -71,6 +74,8 @@ class AsyncReaderInterface { /// Read a message of type \a R into \a msg. Completion will be notified by \a /// tag on the associated completion queue. + /// This is thread-safe with respect to other streaming APIs except for Finish + /// on the same stream. /// /// \param[out] msg Where to eventually store the read message. /// \param[in] tag The tag identifying the operation. @@ -88,6 +93,7 @@ class AsyncWriterInterface { /// Only one write may be outstanding at any given time. This means that /// after calling Write, one must wait to receive \a tag from the completion /// queue BEFORE calling Write again. + /// This is thread-safe with respect to Read /// /// \param[in] msg The message to be written. /// \param[in] tag The tag identifying the operation. @@ -158,6 +164,7 @@ class ClientAsyncWriterInterface : public ClientAsyncStreamingInterface, public AsyncWriterInterface { public: /// Signal the client is done with the writes. + /// Thread-safe with respect to Read /// /// \param[in] tag The tag identifying the operation. virtual void WritesDone(void* tag) = 0; @@ -229,6 +236,7 @@ class ClientAsyncReaderWriterInterface : public ClientAsyncStreamingInterface, public AsyncReaderInterface { public: /// Signal the client is done with the writes. + /// Thread-safe with respect to Read /// /// \param[in] tag The tag identifying the operation. virtual void WritesDone(void* tag) = 0; diff --git a/include/grpc++/impl/codegen/completion_queue.h b/include/grpc++/impl/codegen/completion_queue.h index 1b84b447050..f138ebe7de2 100644 --- a/include/grpc++/impl/codegen/completion_queue.h +++ b/include/grpc++/impl/codegen/completion_queue.h @@ -31,8 +31,18 @@ * */ -/// A completion queue implements a concurrent producer-consumer queue, with two -/// main methods, \a Next and \a AsyncNext. +/// A completion queue implements a concurrent producer-consumer queue, with +/// two main API-exposed methods: \a Next and \a AsyncNext. These +/// methods are the essential component of the gRPC C++ asynchronous API. +/// There is also a Shutdown method to indicate that a given completion queue +/// will no longer have regular events. This must be called before the +/// completion queue is destroyed. +/// All completion queue APIs are thread-safe and may be used concurrently with +/// any other completion queue API invocation; it is acceptable to have +/// multiple threads calling Next or AsyncNext on the same or different +/// completion queues, or to call these methods concurrently with a Shutdown +/// elsewhere. All of these should be completed, though, before a completion +/// queue destructor is called. #ifndef GRPCXX_IMPL_CODEGEN_COMPLETION_QUEUE_H #define GRPCXX_IMPL_CODEGEN_COMPLETION_QUEUE_H diff --git a/include/grpc++/impl/codegen/sync_stream.h b/include/grpc++/impl/codegen/sync_stream.h index e94ffe58422..9d7966ba04b 100644 --- a/include/grpc++/impl/codegen/sync_stream.h +++ b/include/grpc++/impl/codegen/sync_stream.h @@ -71,6 +71,8 @@ class ReaderInterface { virtual ~ReaderInterface() {} /// Blocking read a message and parse to \a msg. Returns \a true on success. + /// This is thread-safe with respect to other streaming APIs except for Finish + /// on the same stream. (Finish must be called as described above.) /// /// \param[out] msg The read message. /// @@ -87,6 +89,7 @@ class WriterInterface { virtual ~WriterInterface() {} /// Blocking write \a msg to the stream with options. + /// This is thread-safe with respect to Read /// /// \param msg The message to be written to the stream. /// \param options Options affecting the write operation. @@ -95,6 +98,7 @@ class WriterInterface { virtual bool Write(const W& msg, const WriteOptions& options) = 0; /// Blocking write \a msg to the stream with default options. + /// This is thread-safe with respect to Read /// /// \param msg The message to be written to the stream. /// @@ -174,7 +178,8 @@ class ClientWriterInterface : public ClientStreamingInterface, public WriterInterface { public: /// Half close writing from the client. - /// Block until writes are completed. + /// Block until currently-pending writes are completed. + /// Thread safe with respect to Read operations only /// /// \return Whether the writes were successful. virtual bool WritesDone() = 0; @@ -257,7 +262,8 @@ class ClientReaderWriterInterface : public ClientStreamingInterface, /// the metadata will be available in ClientContext after the first read. virtual void WaitForInitialMetadata() = 0; - /// Block until writes are completed. + /// Block until currently-pending writes are completed. + /// Thread-safe with respect to Read /// /// \return Whether the writes were successful. virtual bool WritesDone() = 0; From b187895fc43a84d75b64aa58d21d5caf9b80ba61 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Mon, 20 Jun 2016 09:11:54 -0700 Subject: [PATCH 070/215] Fix windows build error --- src/core/lib/iomgr/tcp_server_windows.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/core/lib/iomgr/tcp_server_windows.c b/src/core/lib/iomgr/tcp_server_windows.c index 2a51671ec71..4604f56dac7 100644 --- a/src/core/lib/iomgr/tcp_server_windows.c +++ b/src/core/lib/iomgr/tcp_server_windows.c @@ -103,6 +103,7 @@ struct grpc_tcp_server { /* Public function. Allocates the proper data structures to hold a grpc_tcp_server. */ grpc_error *grpc_tcp_server_create(grpc_closure *shutdown_complete, + const grpc_channel_args *args, grpc_tcp_server **server) { grpc_tcp_server *s = gpr_malloc(sizeof(grpc_tcp_server)); gpr_ref_init(&s->refs, 1); From 04e47f308ec57c42ec9a0dcdc25e72c03b66bc9e Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Mon, 20 Jun 2016 11:03:15 -0700 Subject: [PATCH 071/215] Direct additional log statement through custom logger --- src/node/src/server.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/node/src/server.js b/src/node/src/server.js index fd5153f1244..b3b414969ab 100644 --- a/src/node/src/server.js +++ b/src/node/src/server.js @@ -735,8 +735,8 @@ Server.prototype.addService = function(service, implementation) { } var impl; if (implementation[name] === undefined) { - console.warn('Method handler for %s expected but not provided', - attrs.path); + common.log(grpc.logVerbosity.ERROR, 'Method handler for ' + + attrs.path + ' expected but not provided'); impl = defaultHandler[method_type]; } else { impl = _.bind(implementation[name], implementation); From 36a58a7928e6a10b44b6cffdc118337ef7e0b8c1 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Mon, 20 Jun 2016 14:01:07 -0700 Subject: [PATCH 072/215] Enable treating warnings as errors for target gRPC --- .../GRPCClient/GRPCCall+ChannelCredentials.h | 4 ++-- src/objective-c/ProtoRPC/ProtoMethod.h | 3 +++ src/objective-c/ProtoRPC/ProtoRPC.h | 3 +++ src/objective-c/ProtoRPC/ProtoService.h | 3 +++ src/objective-c/ProtoRPC/ProtoService.m | 6 +++--- src/objective-c/tests/Podfile | 21 ++++++++++++++++--- 6 files changed, 32 insertions(+), 8 deletions(-) diff --git a/src/objective-c/GRPCClient/GRPCCall+ChannelCredentials.h b/src/objective-c/GRPCClient/GRPCCall+ChannelCredentials.h index 343dd48a14e..ac2a37d75f2 100644 --- a/src/objective-c/GRPCClient/GRPCCall+ChannelCredentials.h +++ b/src/objective-c/GRPCClient/GRPCCall+ChannelCredentials.h @@ -41,7 +41,7 @@ */ + (BOOL)setTLSPEMRootCerts:(nullable NSString *)pemRootCert forHost:(nonnull NSString *)host - error:(NSError **)errorPtr; + error:(NSError * _Nullable * _Nullable)errorPtr; /** * Configures @c host with TLS/SSL Client Credentials and optionally trusted root Certificate * Authorities. If @c pemRootCerts is nil, the default CA Certificates bundled with gRPC will be @@ -51,6 +51,6 @@ withPrivateKey:(nullable NSString *)pemPrivateKey withCertChain:(nullable NSString *)pemCertChain forHost:(nonnull NSString *)host - error:(NSError **)errorPtr; + error:(NSError * _Nullable * _Nullable)errorPtr; @end diff --git a/src/objective-c/ProtoRPC/ProtoMethod.h b/src/objective-c/ProtoRPC/ProtoMethod.h index f9fdbb35ffd..ae3a2723fc4 100644 --- a/src/objective-c/ProtoRPC/ProtoMethod.h +++ b/src/objective-c/ProtoRPC/ProtoMethod.h @@ -54,6 +54,9 @@ __attribute__((deprecated("Please use GRPCProtoMethod."))) * This subclass is empty now. Eventually we'll remove ProtoMethod class * to avoid potential naming conflict */ +#pragma clang diagnostic push +#pragma clang diagnostic ignored "-Wdeprecated-declarations" @interface GRPCProtoMethod : ProtoMethod +#pragma clang diagnostic pop @end diff --git a/src/objective-c/ProtoRPC/ProtoRPC.h b/src/objective-c/ProtoRPC/ProtoRPC.h index 5f91f6bce1e..04fc1e45f16 100644 --- a/src/objective-c/ProtoRPC/ProtoRPC.h +++ b/src/objective-c/ProtoRPC/ProtoRPC.h @@ -56,6 +56,9 @@ __attribute__((deprecated("Please use GRPCProtoCall."))) * This subclass is empty now. Eventually we'll remove ProtoRPC class * to avoid potential naming conflict */ +#pragma clang diagnostic push +#pragma clang diagnostic ignored "-Wdeprecated-declarations" @interface GRPCProtoCall : ProtoRPC +#pragma clang diagnostic pop @end diff --git a/src/objective-c/ProtoRPC/ProtoService.h b/src/objective-c/ProtoRPC/ProtoService.h index 87d06e1ae59..7faae1b49c9 100644 --- a/src/objective-c/ProtoRPC/ProtoService.h +++ b/src/objective-c/ProtoRPC/ProtoService.h @@ -55,6 +55,9 @@ __attribute__((deprecated("Please use GRPCProtoService."))) * This subclass is empty now. Eventually we'll remove ProtoService class * to avoid potential naming conflict */ +#pragma clang diagnostic push +#pragma clang diagnostic ignored "-Wdeprecated-declarations" @interface GRPCProtoService : ProtoService +#pragma clang diagnostic pop @end diff --git a/src/objective-c/ProtoRPC/ProtoService.m b/src/objective-c/ProtoRPC/ProtoService.m index 597c3cf0fed..4a14570d818 100644 --- a/src/objective-c/ProtoRPC/ProtoService.m +++ b/src/objective-c/ProtoRPC/ProtoService.m @@ -65,14 +65,14 @@ return self; } -- (ProtoRPC *)RPCToMethod:(NSString *)method +- (GRPCProtoCall *)RPCToMethod:(NSString *)method requestsWriter:(GRXWriter *)requestsWriter responseClass:(Class)responseClass responsesWriteable:(id)responsesWriteable { - ProtoMethod *methodName = [[ProtoMethod alloc] initWithPackage:_packageName + GRPCProtoMethod *methodName = [[GRPCProtoMethod alloc] initWithPackage:_packageName service:_serviceName method:method]; - return [[ProtoRPC alloc] initWithHost:_host + return [[GRPCProtoCall alloc] initWithHost:_host method:methodName requestsWriter:requestsWriter responseClass:responseClass diff --git a/src/objective-c/tests/Podfile b/src/objective-c/tests/Podfile index 508641d681a..53edf8c890a 100644 --- a/src/objective-c/tests/Podfile +++ b/src/objective-c/tests/Podfile @@ -1,9 +1,9 @@ source 'https://github.com/CocoaPods/Specs.git' platform :ios, '8.0' -pod 'Protobuf', :path => "../../../third_party/protobuf" -pod 'BoringSSL', :podspec => ".." -pod 'CronetFramework', :podspec => ".." +pod 'Protobuf', :path => "../../../third_party/protobuf", :inhibit_warnings => true +pod 'BoringSSL', :podspec => "..", :inhibit_warnings => true +pod 'CronetFramework', :podspec => "..", :inhibit_warnings => true pod 'gRPC', :path => "../../.." pod 'RemoteTest', :path => "RemoteTestClient" @@ -30,3 +30,18 @@ end target 'InteropTestsLocalCleartext' do end + +post_install do |installer| + installer.pods_project.targets.each do |target| + target.build_configurations.each do |config| + config.build_settings['GCC_TREAT_WARNINGS_AS_ERRORS'] = 'YES' + end + if target.name == 'gRPC' + target.build_configurations.each do |config| + # GPR_UNREACHABLE_CODE causes "Control may reach end of non-void + # function" warning + config.build_settings['GCC_WARN_ABOUT_RETURN_TYPE'] = 'NO' + end + end + end +end From abc7427a4d113aeb7fde6d732c63ffea1f615998 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Mon, 20 Jun 2016 14:40:57 -0700 Subject: [PATCH 073/215] Update podfiles for objc examples --- examples/objective-c/auth_sample/AuthTestService.podspec | 4 ++++ examples/objective-c/helloworld/HelloWorld.podspec | 4 ++++ examples/objective-c/route_guide/RouteGuide.podspec | 4 ++++ src/objective-c/examples/RemoteTestClient/RemoteTest.podspec | 4 ++++ 4 files changed, 16 insertions(+) diff --git a/examples/objective-c/auth_sample/AuthTestService.podspec b/examples/objective-c/auth_sample/AuthTestService.podspec index e9356260534..d246653ea79 100644 --- a/examples/objective-c/auth_sample/AuthTestService.podspec +++ b/examples/objective-c/auth_sample/AuthTestService.podspec @@ -2,6 +2,10 @@ Pod::Spec.new do |s| s.name = "AuthTestService" s.version = "0.0.1" s.license = "New BSD" + s.authors = { 'gRPC contributors' => 'grpc-io@googlegroups.com' } + s.homepage = "http://www.grpc.io/" + s.summary = "AuthTestService example" + s.source = { :git => 'https://github.com/grpc/grpc.git' } s.ios.deployment_target = "7.1" s.osx.deployment_target = "10.9" diff --git a/examples/objective-c/helloworld/HelloWorld.podspec b/examples/objective-c/helloworld/HelloWorld.podspec index bdf782f6eaf..17b016b31a1 100644 --- a/examples/objective-c/helloworld/HelloWorld.podspec +++ b/examples/objective-c/helloworld/HelloWorld.podspec @@ -2,6 +2,10 @@ Pod::Spec.new do |s| s.name = "HelloWorld" s.version = "0.0.1" s.license = "New BSD" + s.authors = { 'gRPC contributors' => 'grpc-io@googlegroups.com' } + s.homepage = "http://www.grpc.io/" + s.summary = "HelloWorld example" + s.source = { :git => 'https://github.com/grpc/grpc.git' } s.ios.deployment_target = "7.1" s.osx.deployment_target = "10.9" diff --git a/examples/objective-c/route_guide/RouteGuide.podspec b/examples/objective-c/route_guide/RouteGuide.podspec index 4bc2c42cbb8..97a61ff51a5 100644 --- a/examples/objective-c/route_guide/RouteGuide.podspec +++ b/examples/objective-c/route_guide/RouteGuide.podspec @@ -2,6 +2,10 @@ Pod::Spec.new do |s| s.name = "RouteGuide" s.version = "0.0.1" s.license = "New BSD" + s.authors = { 'gRPC contributors' => 'grpc-io@googlegroups.com' } + s.homepage = "http://www.grpc.io/" + s.summary = "RouteGuide example" + s.source = { :git => 'https://github.com/grpc/grpc.git' } s.ios.deployment_target = "7.1" s.osx.deployment_target = "10.9" diff --git a/src/objective-c/examples/RemoteTestClient/RemoteTest.podspec b/src/objective-c/examples/RemoteTestClient/RemoteTest.podspec index 5addf26fc46..107e6de4e21 100644 --- a/src/objective-c/examples/RemoteTestClient/RemoteTest.podspec +++ b/src/objective-c/examples/RemoteTestClient/RemoteTest.podspec @@ -2,6 +2,10 @@ Pod::Spec.new do |s| s.name = "RemoteTest" s.version = "0.0.1" s.license = "New BSD" + s.authors = { 'gRPC contributors' => 'grpc-io@googlegroups.com' } + s.homepage = "http://www.grpc.io/" + s.summary = "RemoteTest example" + s.source = { :git => 'https://github.com/grpc/grpc.git' } s.ios.deployment_target = '7.1' s.osx.deployment_target = '10.9' From 7981e1905f61471f8caece6eea6051b2522b9235 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Mon, 20 Jun 2016 15:38:15 -0700 Subject: [PATCH 074/215] Network status tracking Recreated the old PR (https://github.com/grpc/grpc/pull/6283) in a new repo since the old repo was destroyed. Removed changes to tcp_posix.c and tpc_windows.c, instead depending on the idempotent endpoint shutdown code from https://github.com/grpc/grpc/pull/6911. --- src/core/lib/iomgr/network_status_tracker.c | 82 ++++++ src/core/lib/iomgr/network_status_tracker.h | 40 +++ .../end2end/tests/network_status_change.c | 243 ++++++++++++++++++ 3 files changed, 365 insertions(+) create mode 100644 src/core/lib/iomgr/network_status_tracker.c create mode 100644 src/core/lib/iomgr/network_status_tracker.h create mode 100644 test/core/end2end/tests/network_status_change.c diff --git a/src/core/lib/iomgr/network_status_tracker.c b/src/core/lib/iomgr/network_status_tracker.c new file mode 100644 index 00000000000..6e432368cf1 --- /dev/null +++ b/src/core/lib/iomgr/network_status_tracker.c @@ -0,0 +1,82 @@ +/* + * + * Copyright 2015, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include "src/core/lib/iomgr/endpoint.h" +#include + +typedef struct endpoint_ll_node { + grpc_endpoint *ep; + struct endpoint_ll_node *next; +} endpoint_ll_node; + +static endpoint_ll_node *head = NULL; + +// TODO(makarandd): Install callback with OS to monitor network status. +void grpc_initialize_network_status_monitor() { +} + +void grpc_destroy_network_status_monitor() { + for (endpoint_ll_node *curr = head; curr != NULL; ) { + endpoint_ll_node *next = curr->next; + gpr_free(curr); + curr = next; + } +} + +void grpc_network_status_register_endpoint(grpc_endpoint *ep) { + if (head == NULL) { + head = (endpoint_ll_node *)gpr_malloc(sizeof(endpoint_ll_node)); + head->ep = ep; + head->next = NULL; + } else { + endpoint_ll_node *prev_head = head; + head = (endpoint_ll_node *)gpr_malloc(sizeof(endpoint_ll_node)); + head->ep = ep; + head->next = prev_head; + } +} + +// Walk the linked-list from head and execute shutdown. It is possible that +// other threads might be in the process of shutdown as well, but that has +// no side effect. +void grpc_network_status_shutdown_all_endpoints() { + if (head == NULL) { + return; + } + grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; + + for (endpoint_ll_node *curr = head; curr != NULL; curr = curr->next) { + curr->ep->vtable->shutdown(&exec_ctx, curr->ep); + } + grpc_exec_ctx_finish(&exec_ctx); +} diff --git a/src/core/lib/iomgr/network_status_tracker.h b/src/core/lib/iomgr/network_status_tracker.h new file mode 100644 index 00000000000..4154151927f --- /dev/null +++ b/src/core/lib/iomgr/network_status_tracker.h @@ -0,0 +1,40 @@ +/* + * + * Copyright 2016, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#ifndef GRPC_CORE_LIB_IOMGR_NETWORK_STATUS_TRACKER_H +#define GRPC_CORE_LIB_IOMGR_NETWORK_STATUS_TRACKER_H +#include "src/core/lib/iomgr/endpoint.h" + +void grpc_network_status_register_endpoint(grpc_endpoint *ep); +void grpc_network_status_shutdown_all_endpoints(); +#endif /* GRPC_CORE_LIB_IOMGR_NETWORK_STATUS_TRACKER_H */ diff --git a/test/core/end2end/tests/network_status_change.c b/test/core/end2end/tests/network_status_change.c new file mode 100644 index 00000000000..bbc85007022 --- /dev/null +++ b/test/core/end2end/tests/network_status_change.c @@ -0,0 +1,243 @@ +/* + * + * Copyright 2015, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include "test/core/end2end/end2end_tests.h" + +#include +#include + +#include +#include +#include +#include +#include +#include "test/core/end2end/cq_verifier.h" + +/* this is a private API but exposed here for testing*/ +extern void grpc_network_status_shutdown_all_endpoints(); + +enum { TIMEOUT = 200000 }; + +static void *tag(intptr_t t) { return (void *)t; } + +static grpc_end2end_test_fixture begin_test(grpc_end2end_test_config config, + const char *test_name, + grpc_channel_args *client_args, + grpc_channel_args *server_args) { + grpc_end2end_test_fixture f; + gpr_log(GPR_INFO, "%s/%s", test_name, config.name); + f = config.create_fixture(client_args, server_args); + config.init_server(&f, server_args); + config.init_client(&f, client_args); + return f; +} + +static gpr_timespec n_seconds_time(int n) { + return GRPC_TIMEOUT_SECONDS_TO_DEADLINE(n); +} + +static gpr_timespec five_seconds_time(void) { return n_seconds_time(500); } + +static void drain_cq(grpc_completion_queue *cq) { + grpc_event ev; + do { + ev = grpc_completion_queue_next(cq, five_seconds_time(), NULL); + } while (ev.type != GRPC_QUEUE_SHUTDOWN); +} + +static void shutdown_server(grpc_end2end_test_fixture *f) { + if (!f->server) return; + grpc_server_shutdown_and_notify(f->server, f->cq, tag(1000)); + GPR_ASSERT(grpc_completion_queue_pluck( + f->cq, tag(1000), GRPC_TIMEOUT_SECONDS_TO_DEADLINE(5), NULL) + .type == GRPC_OP_COMPLETE); + grpc_server_destroy(f->server); + f->server = NULL; +} + +static void shutdown_client(grpc_end2end_test_fixture *f) { + if (!f->client) return; + grpc_channel_destroy(f->client); + f->client = NULL; +} + +static void end_test(grpc_end2end_test_fixture *f) { + shutdown_server(f); + shutdown_client(f); + + grpc_completion_queue_shutdown(f->cq); + drain_cq(f->cq); + grpc_completion_queue_destroy(f->cq); +} + +/* Client sends a request with payload, server reads then returns status. */ +static void test_invoke_network_status_change(grpc_end2end_test_config config) { + grpc_call *c; + grpc_call *s; + gpr_slice request_payload_slice = gpr_slice_from_copied_string("hello world"); + grpc_byte_buffer *request_payload = + grpc_raw_byte_buffer_create(&request_payload_slice, 1); + gpr_timespec deadline = five_seconds_time(); + grpc_end2end_test_fixture f = + begin_test(config, "test_invoke_request_with_payload", NULL, NULL); + cq_verifier *cqv = cq_verifier_create(f.cq); + grpc_op ops[6]; + grpc_op *op; + grpc_metadata_array initial_metadata_recv; + grpc_metadata_array trailing_metadata_recv; + grpc_metadata_array request_metadata_recv; + grpc_byte_buffer *request_payload_recv = NULL; + grpc_call_details call_details; + grpc_status_code status; + grpc_call_error error; + char *details = NULL; + size_t details_capacity = 0; + int was_cancelled = 2; + + c = grpc_channel_create_call(f.client, NULL, GRPC_PROPAGATE_DEFAULTS, f.cq, + "/foo", "foo.test.google.fr", deadline, NULL); + GPR_ASSERT(c); + + grpc_metadata_array_init(&initial_metadata_recv); + grpc_metadata_array_init(&trailing_metadata_recv); + grpc_metadata_array_init(&request_metadata_recv); + grpc_call_details_init(&call_details); + + memset(ops, 0, sizeof(ops)); + op = ops; + op->op = GRPC_OP_SEND_INITIAL_METADATA; + op->data.send_initial_metadata.count = 0; + op->flags = 0; + op->reserved = NULL; + op++; + op->op = GRPC_OP_SEND_MESSAGE; + op->data.send_message = request_payload; + op->flags = 0; + op->reserved = NULL; + op++; + op->op = GRPC_OP_SEND_CLOSE_FROM_CLIENT; + op->flags = 0; + op->reserved = NULL; + op++; + op->op = GRPC_OP_RECV_INITIAL_METADATA; + op->data.recv_initial_metadata = &initial_metadata_recv; + op->flags = 0; + op->reserved = NULL; + op++; + op->op = GRPC_OP_RECV_STATUS_ON_CLIENT; + op->data.recv_status_on_client.trailing_metadata = &trailing_metadata_recv; + op->data.recv_status_on_client.status = &status; + op->data.recv_status_on_client.status_details = &details; + op->data.recv_status_on_client.status_details_capacity = &details_capacity; + op->flags = 0; + op->reserved = NULL; + op++; + error = grpc_call_start_batch(c, ops, (size_t)(op - ops), tag(1), NULL); + GPR_ASSERT(GRPC_CALL_OK == error); + + GPR_ASSERT(GRPC_CALL_OK == grpc_server_request_call( + f.server, &s, &call_details, + &request_metadata_recv, f.cq, f.cq, tag(101))); + cq_expect_completion(cqv, tag(101), 1); + cq_verify(cqv); + + op = ops; + op->op = GRPC_OP_SEND_INITIAL_METADATA; + op->data.send_initial_metadata.count = 0; + op->flags = 0; + op->reserved = NULL; + op++; + op->op = GRPC_OP_RECV_MESSAGE; + op->data.recv_message = &request_payload_recv; + op->flags = 0; + op->reserved = NULL; + op++; + error = grpc_call_start_batch(s, ops, (size_t)(op - ops), tag(102), NULL); + GPR_ASSERT(GRPC_CALL_OK == error); + + cq_expect_completion(cqv, tag(102), 1); + // Simulate the network loss event + grpc_network_status_shutdown_all_endpoints(); + cq_verify(cqv); + + op = ops; + op->op = GRPC_OP_RECV_CLOSE_ON_SERVER; + op->data.recv_close_on_server.cancelled = &was_cancelled; + op->flags = 0; + op->reserved = NULL; + op++; + op->op = GRPC_OP_SEND_STATUS_FROM_SERVER; + op->data.send_status_from_server.trailing_metadata_count = 0; + op->data.send_status_from_server.status = GRPC_STATUS_OK; + op->data.send_status_from_server.status_details = "xyz"; + op->flags = 0; + op->reserved = NULL; + op++; + error = grpc_call_start_batch(s, ops, (size_t)(op - ops), tag(103), NULL); + GPR_ASSERT(GRPC_CALL_OK == error); + void shutdown_all_endpoints(); + cq_expect_completion(cqv, tag(103), 1); + cq_expect_completion(cqv, tag(1), 1); + cq_verify(cqv); + + // Expected behavior of a RPC when network is lost. + GPR_ASSERT(status == GRPC_STATUS_UNAVAILABLE); + GPR_ASSERT(0 == strcmp(details, "")); + GPR_ASSERT(0 == strcmp(call_details.method, "/foo")); + GPR_ASSERT(0 == strcmp(call_details.host, "foo.test.google.fr")); + GPR_ASSERT(was_cancelled == 0); + + + gpr_free(details); + grpc_metadata_array_destroy(&initial_metadata_recv); + grpc_metadata_array_destroy(&trailing_metadata_recv); + grpc_metadata_array_destroy(&request_metadata_recv); + grpc_call_details_destroy(&call_details); + + grpc_call_destroy(c); + grpc_call_destroy(s); + + cq_verifier_destroy(cqv); + + grpc_byte_buffer_destroy(request_payload); + grpc_byte_buffer_destroy(request_payload_recv); + + end_test(&f); + config.tear_down_data(&f); +} + +void network_status_change(grpc_end2end_test_config config) { + test_invoke_network_status_change(config); +} + +void network_status_change_pre_init(void) {} From 0579cfc334794a32e243e971f77e01b5fd8f3c55 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Mon, 20 Jun 2016 15:45:24 -0700 Subject: [PATCH 075/215] more files after running build.yaml changes through --- BUILD | 8 + Makefile | 5 + binding.gyp | 1 + build.yaml | 2 + config.m4 | 1 + gRPC.podspec | 3 + grpc.gemspec | 2 + package.xml | 2 + src/core/lib/iomgr/tcp_posix.c | 3 + src/core/lib/iomgr/tcp_windows.c | 4 + src/python/grpcio/grpc_core_dependencies.py | 1 + test/core/end2end/end2end_nosec_tests.c | 8 + test/core/end2end/end2end_tests.c | 8 + test/core/end2end/gen_build_yaml.py | 1 + tools/doxygen/Doxyfile.core.internal | 2 + tools/run_tests/sources_and_headers.json | 5 + tools/run_tests/tests.json | 627 +++++++++++++++++- vsprojects/vcxproj/grpc/grpc.vcxproj | 3 + vsprojects/vcxproj/grpc/grpc.vcxproj.filters | 6 + .../grpc_unsecure/grpc_unsecure.vcxproj | 3 + .../grpc_unsecure.vcxproj.filters | 6 + .../end2end_nosec_tests.vcxproj | 2 + .../end2end_nosec_tests.vcxproj.filters | 3 + .../tests/end2end_tests/end2end_tests.vcxproj | 2 + .../end2end_tests.vcxproj.filters | 3 + 25 files changed, 702 insertions(+), 9 deletions(-) diff --git a/BUILD b/BUILD index ac0eb8e403d..55c2ea87c65 100644 --- a/BUILD +++ b/BUILD @@ -187,6 +187,7 @@ cc_library( "src/core/lib/iomgr/iomgr_internal.h", "src/core/lib/iomgr/iomgr_posix.h", "src/core/lib/iomgr/load_file.h", + "src/core/lib/iomgr/network_status_tracker.h", "src/core/lib/iomgr/polling_entity.h", "src/core/lib/iomgr/pollset.h", "src/core/lib/iomgr/pollset_set.h", @@ -338,6 +339,7 @@ cc_library( "src/core/lib/iomgr/iomgr_posix.c", "src/core/lib/iomgr/iomgr_windows.c", "src/core/lib/iomgr/load_file.c", + "src/core/lib/iomgr/network_status_tracker.c", "src/core/lib/iomgr/polling_entity.c", "src/core/lib/iomgr/pollset_set_windows.c", "src/core/lib/iomgr/pollset_windows.c", @@ -571,6 +573,7 @@ cc_library( "src/core/lib/iomgr/iomgr_internal.h", "src/core/lib/iomgr/iomgr_posix.h", "src/core/lib/iomgr/load_file.h", + "src/core/lib/iomgr/network_status_tracker.h", "src/core/lib/iomgr/polling_entity.h", "src/core/lib/iomgr/pollset.h", "src/core/lib/iomgr/pollset_set.h", @@ -712,6 +715,7 @@ cc_library( "src/core/lib/iomgr/iomgr_posix.c", "src/core/lib/iomgr/iomgr_windows.c", "src/core/lib/iomgr/load_file.c", + "src/core/lib/iomgr/network_status_tracker.c", "src/core/lib/iomgr/polling_entity.c", "src/core/lib/iomgr/pollset_set_windows.c", "src/core/lib/iomgr/pollset_windows.c", @@ -920,6 +924,7 @@ cc_library( "src/core/lib/iomgr/iomgr_internal.h", "src/core/lib/iomgr/iomgr_posix.h", "src/core/lib/iomgr/load_file.h", + "src/core/lib/iomgr/network_status_tracker.h", "src/core/lib/iomgr/polling_entity.h", "src/core/lib/iomgr/pollset.h", "src/core/lib/iomgr/pollset_set.h", @@ -1048,6 +1053,7 @@ cc_library( "src/core/lib/iomgr/iomgr_posix.c", "src/core/lib/iomgr/iomgr_windows.c", "src/core/lib/iomgr/load_file.c", + "src/core/lib/iomgr/network_status_tracker.c", "src/core/lib/iomgr/polling_entity.c", "src/core/lib/iomgr/pollset_set_windows.c", "src/core/lib/iomgr/pollset_windows.c", @@ -1805,6 +1811,7 @@ objc_library( "src/core/lib/iomgr/iomgr_posix.c", "src/core/lib/iomgr/iomgr_windows.c", "src/core/lib/iomgr/load_file.c", + "src/core/lib/iomgr/network_status_tracker.c", "src/core/lib/iomgr/polling_entity.c", "src/core/lib/iomgr/pollset_set_windows.c", "src/core/lib/iomgr/pollset_windows.c", @@ -2017,6 +2024,7 @@ objc_library( "src/core/lib/iomgr/iomgr_internal.h", "src/core/lib/iomgr/iomgr_posix.h", "src/core/lib/iomgr/load_file.h", + "src/core/lib/iomgr/network_status_tracker.h", "src/core/lib/iomgr/polling_entity.h", "src/core/lib/iomgr/pollset.h", "src/core/lib/iomgr/pollset_set.h", diff --git a/Makefile b/Makefile index d93ce337632..961485b3b80 100644 --- a/Makefile +++ b/Makefile @@ -2574,6 +2574,7 @@ LIBGRPC_SRC = \ src/core/lib/iomgr/iomgr_posix.c \ src/core/lib/iomgr/iomgr_windows.c \ src/core/lib/iomgr/load_file.c \ + src/core/lib/iomgr/network_status_tracker.c \ src/core/lib/iomgr/polling_entity.c \ src/core/lib/iomgr/pollset_set_windows.c \ src/core/lib/iomgr/pollset_windows.c \ @@ -2844,6 +2845,7 @@ LIBGRPC_CRONET_SRC = \ src/core/lib/iomgr/iomgr_posix.c \ src/core/lib/iomgr/iomgr_windows.c \ src/core/lib/iomgr/load_file.c \ + src/core/lib/iomgr/network_status_tracker.c \ src/core/lib/iomgr/polling_entity.c \ src/core/lib/iomgr/pollset_set_windows.c \ src/core/lib/iomgr/pollset_windows.c \ @@ -3183,6 +3185,7 @@ LIBGRPC_UNSECURE_SRC = \ src/core/lib/iomgr/iomgr_posix.c \ src/core/lib/iomgr/iomgr_windows.c \ src/core/lib/iomgr/load_file.c \ + src/core/lib/iomgr/network_status_tracker.c \ src/core/lib/iomgr/polling_entity.c \ src/core/lib/iomgr/pollset_set_windows.c \ src/core/lib/iomgr/pollset_windows.c \ @@ -6376,6 +6379,7 @@ LIBEND2END_TESTS_SRC = \ test/core/end2end/tests/max_concurrent_streams.c \ test/core/end2end/tests/max_message_length.c \ test/core/end2end/tests/negative_deadline.c \ + test/core/end2end/tests/network_status_change.c \ test/core/end2end/tests/no_op.c \ test/core/end2end/tests/payload.c \ test/core/end2end/tests/ping.c \ @@ -6452,6 +6456,7 @@ LIBEND2END_NOSEC_TESTS_SRC = \ test/core/end2end/tests/max_concurrent_streams.c \ test/core/end2end/tests/max_message_length.c \ test/core/end2end/tests/negative_deadline.c \ + test/core/end2end/tests/network_status_change.c \ test/core/end2end/tests/no_op.c \ test/core/end2end/tests/payload.c \ test/core/end2end/tests/ping.c \ diff --git a/binding.gyp b/binding.gyp index 3150d1083b0..9a4b46141ff 100644 --- a/binding.gyp +++ b/binding.gyp @@ -591,6 +591,7 @@ 'src/core/lib/iomgr/iomgr_posix.c', 'src/core/lib/iomgr/iomgr_windows.c', 'src/core/lib/iomgr/load_file.c', + 'src/core/lib/iomgr/network_status_tracker.c', 'src/core/lib/iomgr/polling_entity.c', 'src/core/lib/iomgr/pollset_set_windows.c', 'src/core/lib/iomgr/pollset_windows.c', diff --git a/build.yaml b/build.yaml index d41f4195947..54dae5b6ac2 100644 --- a/build.yaml +++ b/build.yaml @@ -183,6 +183,7 @@ filegroups: - src/core/lib/iomgr/iomgr_internal.h - src/core/lib/iomgr/iomgr_posix.h - src/core/lib/iomgr/load_file.h + - src/core/lib/iomgr/network_status_tracker.h - src/core/lib/iomgr/polling_entity.h - src/core/lib/iomgr/pollset.h - src/core/lib/iomgr/pollset_set.h @@ -261,6 +262,7 @@ filegroups: - src/core/lib/iomgr/iomgr_posix.c - src/core/lib/iomgr/iomgr_windows.c - src/core/lib/iomgr/load_file.c + - src/core/lib/iomgr/network_status_tracker.c - src/core/lib/iomgr/polling_entity.c - src/core/lib/iomgr/pollset_set_windows.c - src/core/lib/iomgr/pollset_windows.c diff --git a/config.m4 b/config.m4 index 716eb9841fb..6cf8b3a33fd 100644 --- a/config.m4 +++ b/config.m4 @@ -110,6 +110,7 @@ if test "$PHP_GRPC" != "no"; then src/core/lib/iomgr/iomgr_posix.c \ src/core/lib/iomgr/iomgr_windows.c \ src/core/lib/iomgr/load_file.c \ + src/core/lib/iomgr/network_status_tracker.c \ src/core/lib/iomgr/polling_entity.c \ src/core/lib/iomgr/pollset_set_windows.c \ src/core/lib/iomgr/pollset_windows.c \ diff --git a/gRPC.podspec b/gRPC.podspec index ff8373637a6..48da2322e73 100644 --- a/gRPC.podspec +++ b/gRPC.podspec @@ -190,6 +190,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/iomgr_internal.h', 'src/core/lib/iomgr/iomgr_posix.h', 'src/core/lib/iomgr/load_file.h', + 'src/core/lib/iomgr/network_status_tracker.h', 'src/core/lib/iomgr/polling_entity.h', 'src/core/lib/iomgr/pollset.h', 'src/core/lib/iomgr/pollset_set.h', @@ -375,6 +376,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/iomgr_posix.c', 'src/core/lib/iomgr/iomgr_windows.c', 'src/core/lib/iomgr/load_file.c', + 'src/core/lib/iomgr/network_status_tracker.c', 'src/core/lib/iomgr/polling_entity.c', 'src/core/lib/iomgr/pollset_set_windows.c', 'src/core/lib/iomgr/pollset_windows.c', @@ -570,6 +572,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/iomgr_internal.h', 'src/core/lib/iomgr/iomgr_posix.h', 'src/core/lib/iomgr/load_file.h', + 'src/core/lib/iomgr/network_status_tracker.h', 'src/core/lib/iomgr/polling_entity.h', 'src/core/lib/iomgr/pollset.h', 'src/core/lib/iomgr/pollset_set.h', diff --git a/grpc.gemspec b/grpc.gemspec index 184a5485917..5796cd68bc2 100755 --- a/grpc.gemspec +++ b/grpc.gemspec @@ -199,6 +199,7 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/iomgr_internal.h ) s.files += %w( src/core/lib/iomgr/iomgr_posix.h ) s.files += %w( src/core/lib/iomgr/load_file.h ) + s.files += %w( src/core/lib/iomgr/network_status_tracker.h ) s.files += %w( src/core/lib/iomgr/polling_entity.h ) s.files += %w( src/core/lib/iomgr/pollset.h ) s.files += %w( src/core/lib/iomgr/pollset_set.h ) @@ -354,6 +355,7 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/iomgr_posix.c ) s.files += %w( src/core/lib/iomgr/iomgr_windows.c ) s.files += %w( src/core/lib/iomgr/load_file.c ) + s.files += %w( src/core/lib/iomgr/network_status_tracker.c ) s.files += %w( src/core/lib/iomgr/polling_entity.c ) s.files += %w( src/core/lib/iomgr/pollset_set_windows.c ) s.files += %w( src/core/lib/iomgr/pollset_windows.c ) diff --git a/package.xml b/package.xml index 67e9bb2c282..623174ad016 100644 --- a/package.xml +++ b/package.xml @@ -206,6 +206,7 @@ + @@ -361,6 +362,7 @@ + diff --git a/src/core/lib/iomgr/tcp_posix.c b/src/core/lib/iomgr/tcp_posix.c index 017f52c3677..f7818353b0a 100644 --- a/src/core/lib/iomgr/tcp_posix.c +++ b/src/core/lib/iomgr/tcp_posix.c @@ -35,6 +35,7 @@ #ifdef GPR_POSIX_SOCKET +#include "src/core/lib/iomgr/network_status_tracker.h" #include "src/core/lib/iomgr/tcp_posix.h" #include @@ -474,6 +475,8 @@ grpc_endpoint *grpc_tcp_create(grpc_fd *em_fd, size_t slice_size, tcp->write_closure.cb = tcp_handle_write; tcp->write_closure.cb_arg = tcp; gpr_slice_buffer_init(&tcp->last_read_buffer); + /* Tell network status tracker about new endpoint */ + grpc_network_status_register_endpoint(&tcp->base); return &tcp->base; } diff --git a/src/core/lib/iomgr/tcp_windows.c b/src/core/lib/iomgr/tcp_windows.c index b2af8030aaa..8140d1d8cd2 100644 --- a/src/core/lib/iomgr/tcp_windows.c +++ b/src/core/lib/iomgr/tcp_windows.c @@ -37,6 +37,7 @@ #include +#include "src/core/lib/iomgr/network_status_tracker.h" #include "src/core/lib/iomgr/sockaddr_windows.h" #include @@ -401,6 +402,9 @@ grpc_endpoint *grpc_tcp_create(grpc_winsocket *socket, char *peer_string) { grpc_closure_init(&tcp->on_read, on_read, tcp); grpc_closure_init(&tcp->on_write, on_write, tcp); tcp->peer_string = gpr_strdup(peer_string); + /* Tell network status tracking code about the new endpoint */ + grpc_network_status_register_endpoint(&tcp->base); + return &tcp->base; } diff --git a/src/python/grpcio/grpc_core_dependencies.py b/src/python/grpcio/grpc_core_dependencies.py index 839c555f055..575dfa9dbde 100644 --- a/src/python/grpcio/grpc_core_dependencies.py +++ b/src/python/grpcio/grpc_core_dependencies.py @@ -104,6 +104,7 @@ CORE_SOURCE_FILES = [ 'src/core/lib/iomgr/iomgr_posix.c', 'src/core/lib/iomgr/iomgr_windows.c', 'src/core/lib/iomgr/load_file.c', + 'src/core/lib/iomgr/network_status_tracker.c', 'src/core/lib/iomgr/polling_entity.c', 'src/core/lib/iomgr/pollset_set_windows.c', 'src/core/lib/iomgr/pollset_windows.c', diff --git a/test/core/end2end/end2end_nosec_tests.c b/test/core/end2end/end2end_nosec_tests.c index b71299c09e0..09ed6e6d688 100644 --- a/test/core/end2end/end2end_nosec_tests.c +++ b/test/core/end2end/end2end_nosec_tests.c @@ -89,6 +89,8 @@ extern void max_message_length(grpc_end2end_test_config config); extern void max_message_length_pre_init(void); extern void negative_deadline(grpc_end2end_test_config config); extern void negative_deadline_pre_init(void); +extern void network_status_change(grpc_end2end_test_config config); +extern void network_status_change_pre_init(void); extern void no_op(grpc_end2end_test_config config); extern void no_op_pre_init(void); extern void payload(grpc_end2end_test_config config); @@ -144,6 +146,7 @@ void grpc_end2end_tests_pre_init(void) { max_concurrent_streams_pre_init(); max_message_length_pre_init(); negative_deadline_pre_init(); + network_status_change_pre_init(); no_op_pre_init(); payload_pre_init(); ping_pre_init(); @@ -190,6 +193,7 @@ void grpc_end2end_tests(int argc, char **argv, max_concurrent_streams(config); max_message_length(config); negative_deadline(config); + network_status_change(config); no_op(config); payload(config); ping(config); @@ -300,6 +304,10 @@ void grpc_end2end_tests(int argc, char **argv, negative_deadline(config); continue; } + if (0 == strcmp("network_status_change", argv[i])) { + network_status_change(config); + continue; + } if (0 == strcmp("no_op", argv[i])) { no_op(config); continue; diff --git a/test/core/end2end/end2end_tests.c b/test/core/end2end/end2end_tests.c index 00c9c44a78c..312700646cd 100644 --- a/test/core/end2end/end2end_tests.c +++ b/test/core/end2end/end2end_tests.c @@ -91,6 +91,8 @@ extern void max_message_length(grpc_end2end_test_config config); extern void max_message_length_pre_init(void); extern void negative_deadline(grpc_end2end_test_config config); extern void negative_deadline_pre_init(void); +extern void network_status_change(grpc_end2end_test_config config); +extern void network_status_change_pre_init(void); extern void no_op(grpc_end2end_test_config config); extern void no_op_pre_init(void); extern void payload(grpc_end2end_test_config config); @@ -147,6 +149,7 @@ void grpc_end2end_tests_pre_init(void) { max_concurrent_streams_pre_init(); max_message_length_pre_init(); negative_deadline_pre_init(); + network_status_change_pre_init(); no_op_pre_init(); payload_pre_init(); ping_pre_init(); @@ -194,6 +197,7 @@ void grpc_end2end_tests(int argc, char **argv, max_concurrent_streams(config); max_message_length(config); negative_deadline(config); + network_status_change(config); no_op(config); payload(config); ping(config); @@ -308,6 +312,10 @@ void grpc_end2end_tests(int argc, char **argv, negative_deadline(config); continue; } + if (0 == strcmp("network_status_change", argv[i])) { + network_status_change(config); + continue; + } if (0 == strcmp("no_op", argv[i])) { no_op(config); continue; diff --git a/test/core/end2end/gen_build_yaml.py b/test/core/end2end/gen_build_yaml.py index 325d9b3cad0..716dca20bb1 100755 --- a/test/core/end2end/gen_build_yaml.py +++ b/test/core/end2end/gen_build_yaml.py @@ -112,6 +112,7 @@ END2END_TESTS = { 'max_concurrent_streams': default_test_options._replace(proxyable=False), 'max_message_length': default_test_options, 'negative_deadline': default_test_options, + 'network_status_change': default_test_options, 'no_op': default_test_options, 'payload': default_test_options, 'ping_pong_streaming': default_test_options, diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal index e1555930e91..545536c97c4 100644 --- a/tools/doxygen/Doxyfile.core.internal +++ b/tools/doxygen/Doxyfile.core.internal @@ -818,6 +818,7 @@ src/core/lib/iomgr/iomgr.h \ src/core/lib/iomgr/iomgr_internal.h \ src/core/lib/iomgr/iomgr_posix.h \ src/core/lib/iomgr/load_file.h \ +src/core/lib/iomgr/network_status_tracker.h \ src/core/lib/iomgr/polling_entity.h \ src/core/lib/iomgr/pollset.h \ src/core/lib/iomgr/pollset_set.h \ @@ -973,6 +974,7 @@ src/core/lib/iomgr/iomgr.c \ src/core/lib/iomgr/iomgr_posix.c \ src/core/lib/iomgr/iomgr_windows.c \ src/core/lib/iomgr/load_file.c \ +src/core/lib/iomgr/network_status_tracker.c \ src/core/lib/iomgr/polling_entity.c \ src/core/lib/iomgr/pollset_set_windows.c \ src/core/lib/iomgr/pollset_windows.c \ diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index 57e3bd63f42..7f8cc8ad436 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -5387,6 +5387,7 @@ "test/core/end2end/tests/max_concurrent_streams.c", "test/core/end2end/tests/max_message_length.c", "test/core/end2end/tests/negative_deadline.c", + "test/core/end2end/tests/network_status_change.c", "test/core/end2end/tests/no_op.c", "test/core/end2end/tests/payload.c", "test/core/end2end/tests/ping.c", @@ -5445,6 +5446,7 @@ "test/core/end2end/tests/max_concurrent_streams.c", "test/core/end2end/tests/max_message_length.c", "test/core/end2end/tests/negative_deadline.c", + "test/core/end2end/tests/network_status_change.c", "test/core/end2end/tests/no_op.c", "test/core/end2end/tests/payload.c", "test/core/end2end/tests/ping.c", @@ -5732,6 +5734,7 @@ "src/core/lib/iomgr/iomgr_internal.h", "src/core/lib/iomgr/iomgr_posix.h", "src/core/lib/iomgr/load_file.h", + "src/core/lib/iomgr/network_status_tracker.h", "src/core/lib/iomgr/polling_entity.h", "src/core/lib/iomgr/pollset.h", "src/core/lib/iomgr/pollset_set.h", @@ -5847,6 +5850,8 @@ "src/core/lib/iomgr/iomgr_windows.c", "src/core/lib/iomgr/load_file.c", "src/core/lib/iomgr/load_file.h", + "src/core/lib/iomgr/network_status_tracker.c", + "src/core/lib/iomgr/network_status_tracker.h", "src/core/lib/iomgr/polling_entity.c", "src/core/lib/iomgr/polling_entity.h", "src/core/lib/iomgr/pollset.h", diff --git a/tools/run_tests/tests.json b/tools/run_tests/tests.json index 3ed7a6bc476..bf5ff5be191 100644 --- a/tools/run_tests/tests.json +++ b/tools/run_tests/tests.json @@ -4900,6 +4900,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_census_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -5736,6 +5758,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_compress_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -6548,6 +6592,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_fakesec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -7262,6 +7327,26 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_fd_test", + "platforms": [ + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -8030,6 +8115,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -8722,6 +8829,22 @@ "linux" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full+pipe_test", + "platforms": [ + "linux" + ] + }, { "args": [ "no_op" @@ -9452,6 +9575,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full+trace_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -10288,6 +10433,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_loadreporting_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -11100,6 +11267,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_oauth2_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -11814,6 +12002,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_proxy_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -12507,6 +12716,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_sockpair_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -13179,6 +13409,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_sockpair+trace_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -13874,7 +14125,7 @@ }, { "args": [ - "no_op" + "network_status_change" ], "ci_platforms": [ "windows", @@ -13895,7 +14146,7 @@ }, { "args": [ - "payload" + "no_op" ], "ci_platforms": [ "windows", @@ -13916,7 +14167,7 @@ }, { "args": [ - "ping_pong_streaming" + "payload" ], "ci_platforms": [ "windows", @@ -13937,7 +14188,7 @@ }, { "args": [ - "registered_call" + "ping_pong_streaming" ], "ci_platforms": [ "windows", @@ -13958,14 +14209,14 @@ }, { "args": [ - "request_with_flags" + "registered_call" ], "ci_platforms": [ "windows", "linux", "posix" ], - "cpu_cost": 0.1, + "cpu_cost": 1.0, "exclude_configs": [], "flaky": false, "language": "c", @@ -13979,14 +14230,14 @@ }, { "args": [ - "request_with_payload" + "request_with_flags" ], "ci_platforms": [ "windows", "linux", "posix" ], - "cpu_cost": 1.0, + "cpu_cost": 0.1, "exclude_configs": [], "flaky": false, "language": "c", @@ -14000,7 +14251,28 @@ }, { "args": [ - "server_finishes_request" + "request_with_payload" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_sockpair_1byte_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, + { + "args": [ + "server_finishes_request" ], "ci_platforms": [ "windows", @@ -14652,6 +14924,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_ssl_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -15488,6 +15782,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_ssl_cert_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -16216,6 +16532,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_ssl_proxy_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -16928,6 +17265,26 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_uds_test", + "platforms": [ + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -17714,6 +18071,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_census_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -18528,6 +18907,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_compress_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -19236,6 +19637,26 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_fd_nosec_test", + "platforms": [ + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -19982,6 +20403,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -20658,6 +21101,22 @@ "linux" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full+pipe_nosec_test", + "platforms": [ + "linux" + ] + }, { "args": [ "no_op" @@ -21366,6 +21825,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full+trace_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -22180,6 +22661,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_loadreporting_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -22887,6 +23390,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_proxy_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -23559,6 +24083,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_sockpair_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -24210,6 +24755,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_sockpair+trace_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -24922,6 +25488,29 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [ + "msan" + ], + "flaky": false, + "language": "c", + "name": "h2_sockpair_1byte_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -25638,6 +26227,26 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_uds_nosec_test", + "platforms": [ + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj b/vsprojects/vcxproj/grpc/grpc.vcxproj index 84e03f70561..be4a28d5ecf 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj @@ -327,6 +327,7 @@ + @@ -513,6 +514,8 @@ + + diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters index 0f817e0562d..086b62d572c 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters @@ -88,6 +88,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr @@ -731,6 +734,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj index ac9bb186b71..d88fc0eef0a 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj @@ -316,6 +316,7 @@ + @@ -480,6 +481,8 @@ + + diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters index 1341474142c..e1bc1801f2f 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters @@ -91,6 +91,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr @@ -638,6 +641,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr diff --git a/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj b/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj index 22cd102d116..5297d79e263 100644 --- a/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj +++ b/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj @@ -199,6 +199,8 @@ + + diff --git a/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj.filters b/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj.filters index 1bb208bba8e..897a42386f4 100644 --- a/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj.filters +++ b/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj.filters @@ -73,6 +73,9 @@ test\core\end2end\tests + + test\core\end2end\tests + test\core\end2end\tests diff --git a/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj b/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj index bfd437e8717..3bc61fc581d 100644 --- a/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj +++ b/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj @@ -201,6 +201,8 @@ + + diff --git a/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj.filters b/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj.filters index 61c065f77ca..b896ff8bf1f 100644 --- a/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj.filters +++ b/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj.filters @@ -76,6 +76,9 @@ test\core\end2end\tests + + test\core\end2end\tests + test\core\end2end\tests From 2f8ade0b9df48990e3617a302a5da946f032d4f6 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 17 Jun 2016 13:28:38 -0700 Subject: [PATCH 076/215] Significantly refactor the polling island locking and refcounting code --- src/core/lib/iomgr/ev_epoll_linux.c | 462 ++++++++++++++++------------ 1 file changed, 270 insertions(+), 192 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index ed2c494b783..72288889c02 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -140,18 +140,40 @@ static void fd_global_shutdown(void); #define CLOSURE_READY ((grpc_closure *)1) /******************************************************************************* - * Polling-island Declarations + * Polling island Declarations */ -/* TODO: sree: Consider making ref_cnt and merged_to to gpr_atm - This would - * significantly reduce the number of mutex acquisition calls. */ + +// #define GRPC_PI_REF_COUNT_DEBUG +#ifdef GRPC_PI_REF_COUNT_DEBUG + +#define PI_ADD_REF(p, r) pi_add_ref_dbg((p), 1, (r), __FILE__, __LINE__) +#define PI_UNREF(p, r) pi_unref_dbg((p), 1, (r), __FILE__, __LINE__) + +#else /* defined(GRPC_PI_REF_COUNT_DEBUG) */ + +#define PI_ADD_REF(p, r) pi_add_ref((p), 1) +#define PI_UNREF(p, r) pi_unref((p), 1) + +#endif /* !defined(GPRC_PI_REF_COUNT_DEBUG) */ + typedef struct polling_island { gpr_mu mu; - int ref_cnt; - - /* Points to the polling_island this merged into. - * If merged_to is not NULL, all the remaining fields (except mu and ref_cnt) - * are invalid and must be ignored */ - struct polling_island *merged_to; + /* Ref count. Use PI_ADD_REF() and PI_UNREF() macros to increment/decrement + the refcount. + Once the ref count becomes zero, this structure is destroyed which means + we should ensure that there is never a scenario where a PI_ADD_REF() is + racing with a PI_UNREF() that just made the ref_count zero. */ + gpr_atm ref_count; + + /* Pointer to the polling_island this merged into. + * merged_to value is only set once in polling_island's lifetime (and that too + * only if the island is merged with another island). Because of this, we can + * use gpr_atm type here so that we can do atomic access on this and reduce + * lock contention on 'mu' mutex. + * + * Note that if this field is not NULL (i.e not 0), all the remaining fields + * (except mu and ref_count) are invalid and must be ignored. */ + gpr_atm merged_to; /* The fd of the underlying epoll set */ int epoll_fd; @@ -236,6 +258,8 @@ static grpc_wakeup_fd polling_island_wakeup_fd; static gpr_mu g_pi_freelist_mu; static polling_island *g_pi_freelist = NULL; +static void polling_island_delete(); /* Forward declaration */ + #ifdef GRPC_TSAN /* Currently TSAN may incorrectly flag data races between epoll_ctl and epoll_wait for any grpc_fd structs that are added to the epoll set via @@ -247,6 +271,51 @@ static polling_island *g_pi_freelist = NULL; gpr_atm g_epoll_sync; #endif /* defined(GRPC_TSAN) */ +#ifdef GRPC_PI_REF_COUNT_DEBUG +long pi_add_ref(polling_island *pi, int ref_cnt); +long pi_unref(polling_island *pi, int ref_cnt); + +void pi_add_ref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, + int line) { + long old_cnt = pi_add_ref(pi, ref_cnt); + gpr_log(GPR_DEBUG, "Add ref pi: %p, old:%ld -> new:%ld (%s) - (%s, %d)", + (void *)pi, old_cnt, (old_cnt + ref_cnt), reason, file, line); +} + +void pi_unref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, + int line) { + long old_cnt = pi_unref(pi, ref_cnt); + gpr_log(GPR_DEBUG, "Unref pi: %p, old:%ld -> new:%ld (%s) - (%s, %d)", + (void *)pi, old_cnt, (old_cnt - ref_cnt), reason, file, line); +} +#endif + +long pi_add_ref(polling_island *pi, int ref_cnt) { + return gpr_atm_no_barrier_fetch_add(&pi->ref_count, ref_cnt); +} + +long pi_unref(polling_island *pi, int ref_cnt) { + long old_cnt = gpr_atm_no_barrier_fetch_add(&pi->ref_count, -ref_cnt); + + /* If ref count went to zero, delete the polling island. Note that this need + not be done under a lock. Once the ref count goes to zero, we are + guaranteed that no one else holds a reference to the polling island (and + that there is no racing pi_add_ref() call either. + + Also, if we are deleting the polling island and the merged_to field is + non-empty, we should remove a ref to the merged_to polling island + */ + if (old_cnt == ref_cnt) { + polling_island *next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + polling_island_delete(pi); + if (next != NULL) { + PI_UNREF(next, "pi_delete"); /* Recursive call */ + } + } + + return old_cnt; +} + /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, size_t fd_count, bool add_fd_refs) { @@ -355,8 +424,7 @@ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, } } -static polling_island *polling_island_create(grpc_fd *initial_fd, - int initial_ref_cnt) { +static polling_island *polling_island_create(grpc_fd *initial_fd) { polling_island *pi = NULL; /* Try to get one from the polling island freelist */ @@ -377,6 +445,9 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, pi->fds = NULL; } + gpr_atm_no_barrier_store(&pi->ref_count, 0); + gpr_atm_no_barrier_store(&pi->merged_to, NULL); + pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); if (pi->epoll_fd < 0) { @@ -387,14 +458,12 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, polling_island_add_wakeup_fd_locked(pi, &grpc_global_wakeup_fd); - pi->ref_cnt = initial_ref_cnt; - pi->merged_to = NULL; pi->next_free = NULL; if (initial_fd != NULL) { - /* It is not really needed to get the pi->mu lock here. If this is a newly - created polling island (or one that we got from the freelist), no one - else would be holding a lock to it anyway */ + /* Lock the polling island here just in case we got this structure from the + freelist and the polling island lock was not released yet (by the code + that adds the polling island to the freelist) */ gpr_mu_lock(&pi->mu); polling_island_add_fds_locked(pi, &initial_fd, 1, true); gpr_mu_unlock(&pi->mu); @@ -404,140 +473,136 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, } static void polling_island_delete(polling_island *pi) { - GPR_ASSERT(pi->ref_cnt == 0); GPR_ASSERT(pi->fd_cnt == 0); + gpr_atm_rel_store(&pi->merged_to, NULL); + close(pi->epoll_fd); pi->epoll_fd = -1; - pi->merged_to = NULL; - gpr_mu_lock(&g_pi_freelist_mu); pi->next_free = g_pi_freelist; g_pi_freelist = pi; gpr_mu_unlock(&g_pi_freelist_mu); } -void polling_island_unref_and_unlock(polling_island *pi, int unref_by) { - pi->ref_cnt -= unref_by; - int ref_cnt = pi->ref_cnt; - GPR_ASSERT(ref_cnt >= 0); - - gpr_mu_unlock(&pi->mu); - - if (ref_cnt == 0) { - polling_island_delete(pi); - } -} - -polling_island *polling_island_update_and_lock(polling_island *pi, int unref_by, - int add_ref_by) { +/* Gets the lock on the *latest* polling island i.e the last polling island in + the linked list (linked by 'merged_to' link). Call gpr_mu_unlock on the + returned polling island's mu. + Usage: To lock/unlock polling island "pi", do the following: + polling_island *pi_latest = polling_island_lock(pi); + ... + ... critical section .. + ... + gpr_mu_unlock(&pi_latest->mu); //NOTE: use pi_latest->mu. NOT pi->mu */ +polling_island *polling_island_lock(polling_island *pi) { polling_island *next = NULL; - gpr_mu_lock(&pi->mu); - while (pi->merged_to != NULL) { - next = pi->merged_to; - polling_island_unref_and_unlock(pi, unref_by); + while (true) { + next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + if (next == NULL) { + /* pi is the last node in the linked list. Get the lock and check again + (under the pi->mu lock) that pi is still the last node (because a merge + may have happend after the (next == NULL) check above and before + getting the pi->mu lock. + If pi is the last node, we are done. If not, unlock and continue + traversing the list */ + gpr_mu_lock(&pi->mu); + next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + if (next == NULL) { + break; + } + gpr_mu_unlock(&pi->mu); + } + pi = next; - gpr_mu_lock(&pi->mu); } - pi->ref_cnt += add_ref_by; return pi; } -void polling_island_pair_update_and_lock(polling_island **p, - polling_island **q) { +/* Gets the lock on the *latest* polling islands pointed by *p and *q. + This function is needed because calling the following block of code to obtain + locks on polling islands (*p and *q) is prone to deadlocks. + { + polling_island_lock(*p); + polling_island_lock(*q); + } + + Usage/exmaple: + polling_island *p1; + polling_island *p2; + .. + polling_island_lock_pair(&p1, &p2); + .. + .. Critical section with both p1 and p2 locked + .. + // Release locks + // **IMPORTANT**: Make sure you check p1 == p2 AFTER the function + // polling_island_lock_pair() was called and if so, release the lock only + // once. Note: Even if p1 != p2 beforec calling polling_island_lock_pair(), + // they might be after the function returns: + if (p1 == p2) { + gpr_mu_unlock(&p1->mu) + } else { + gpr_mu_unlock(&p1->mu); + gpr_mu_unlock(&p2->mu); + } + +*/ +void polling_island_lock_pair(polling_island **p, polling_island **q) { polling_island *pi_1 = *p; polling_island *pi_2 = *q; - polling_island *temp = NULL; - bool pi_1_locked = false; - bool pi_2_locked = false; - int num_swaps = 0; - - /* Loop until either pi_1 == pi_2 or until we acquired locks on both pi_1 - and pi_2 */ - while (pi_1 != pi_2 && !(pi_1_locked && pi_2_locked)) { - /* The following assertions are true at this point: - - pi_1 != pi_2 (else, the while loop would have exited) - - pi_1 MAY be locked - - pi_2 is NOT locked */ - - /* To maintain lock order consistency, always lock polling_island node with - lower address first. - First, make sure pi_1 < pi_2 before proceeding any further. If it turns - out that pi_1 > pi_2, unlock pi_1 if locked (because pi_2 is not locked - at this point and having pi_1 locked would violate the lock order) and - swap pi_1 and pi_2 so that pi_1 becomes less than pi_2 */ - if (pi_1 > pi_2) { - if (pi_1_locked) { - gpr_mu_unlock(&pi_1->mu); - pi_1_locked = false; - } + polling_island *next_1 = NULL; + polling_island *next_2 = NULL; + + /* The algorithm is simple: + - Go to the last polling islands in the linked lists *pi_1 and *pi_2 (and + keep updating pi_1 and pi_2) + - Then obtain locks on the islands by following a lock order rule of + locking polling_island with lower address first + Special case: Before obtaining the locks, check if pi_1 and pi_2 are + pointing to the same island. If that is the case, we can just call + polling_island_lock() + - After obtaining both the locks, double check that the polling islands + are still the last polling islands in their respective linked lists + (this is because there might have been polling island merges before + we got the lock) + - If the polling islands are the last islands, we are done. If not, + release the locks and continue the process from the first step */ + while (true) { + next_1 = (polling_island *)gpr_atm_acq_load(&pi_1->merged_to); + while (next_1 != NULL) { + pi_1 = next_1; + next_1 = (polling_island *)gpr_atm_acq_load(&pi_1->merged_to); + } - GPR_SWAP(polling_island *, pi_1, pi_2); - num_swaps++; + next_2 = (polling_island *)gpr_atm_acq_load(&pi_2->merged_to); + while (next_2 != NULL) { + pi_2 = next_2; + next_2 = (polling_island *)gpr_atm_acq_load(&pi_2->merged_to); } - /* The following assertions are true at this point: - - pi_1 != pi_2 - - pi_1 < pi_2 (address of pi_1 is less than that of pi_2) - - pi_1 MAYBE locked - - pi_2 is NOT locked */ + if (pi_1 == pi_2) { + pi_1 = pi_2 = polling_island_lock(pi_1); + break; + } - /* Lock pi_1 (if pi_1 is pointing to the terminal node in the list) */ - if (!pi_1_locked) { + if (pi_1 < pi_2) { + gpr_mu_lock(&pi_1->mu); + gpr_mu_lock(&pi_2->mu); + } else { + gpr_mu_lock(&pi_2->mu); gpr_mu_lock(&pi_1->mu); - pi_1_locked = true; - - /* If pi_1 is not terminal node (i.e pi_1->merged_to != NULL), we are not - done locking this polling_island yet. Release the lock on this node and - advance pi_1 to the next node in the list; and go to the beginning of - the loop (we can't proceed to locking pi_2 unless we locked pi_1 first) - */ - if (pi_1->merged_to != NULL) { - temp = pi_1->merged_to; - polling_island_unref_and_unlock(pi_1, 1); - pi_1 = temp; - pi_1_locked = false; - - continue; - } } - /* The following assertions are true at this point: - - pi_1 is locked - - pi_2 is unlocked - - pi_1 != pi_2 */ - - gpr_mu_lock(&pi_2->mu); - pi_2_locked = true; - - /* If pi_2 is not terminal node, we are not done locking this polling_island - yet. Release the lock and update pi_2 to the next node in the list */ - if (pi_2->merged_to != NULL) { - temp = pi_2->merged_to; - polling_island_unref_and_unlock(pi_2, 1); - pi_2 = temp; - pi_2_locked = false; + next_1 = (polling_island *)gpr_atm_acq_load(&pi_1->merged_to); + next_2 = (polling_island *)gpr_atm_acq_load(&pi_2->merged_to); + if (next_1 == NULL && next_2 == NULL) { + break; } - } - /* At this point, either pi_1 == pi_2 AND/OR we got both locks */ - if (pi_1 == pi_2) { - /* We may or may not have gotten the lock. If we didn't, walk the rest of - the polling_island list and get the lock */ - GPR_ASSERT(pi_1_locked || (!pi_1_locked && !pi_2_locked)); - if (!pi_1_locked) { - pi_1 = pi_2 = polling_island_update_and_lock(pi_1, 2, 0); - } - } else { - GPR_ASSERT(pi_1_locked && pi_2_locked); - /* If we swapped pi_1 and pi_2 odd number of times, do one more swap so that - pi_1 and pi_2 point to the same polling_island lists they started off - with at the beginning of this function (i.e *p and *q respectively) */ - if (num_swaps % 2 > 0) { - GPR_SWAP(polling_island *, pi_1, pi_2); - } + gpr_mu_unlock(&pi_1->mu); + gpr_mu_unlock(&pi_2->mu); } *p = pi_1; @@ -546,7 +611,7 @@ void polling_island_pair_update_and_lock(polling_island **p, polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Get locks on both the polling islands */ - polling_island_pair_update_and_lock(&p, &q); + polling_island_lock_pair(&p, &q); if (p == q) { /* Nothing needs to be done here */ @@ -568,15 +633,14 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Wakeup all the pollers (if any) on p so that they can pickup this change */ polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); - p->merged_to = q; + /* Add the 'merged_to' link from p --> q */ + gpr_atm_rel_store(&p->merged_to, q); + PI_ADD_REF(q, "pi_merge"); /* To account for the new incoming ref from p */ - /* - The merged polling island (i.e q) inherits all the ref counts of the - island merging with it (i.e p) - - The island p will lose a ref count */ - q->ref_cnt += p->ref_cnt; - polling_island_unref_and_unlock(p, 1); /* Decrement refcount */ - polling_island_unref_and_unlock(q, 0); /* Just Unlock. Don't decrement ref */ + gpr_mu_unlock(&p->mu); + gpr_mu_unlock(&q->mu); + /* Return the merged polling island */ return q; } @@ -667,6 +731,7 @@ static void unref_by(grpc_fd *fd, int n) { fd->freelist_next = fd_freelist; fd_freelist = fd; grpc_iomgr_unregister_object(&fd->iomgr_object); + gpr_mu_unlock(&fd_freelist_mu); } else { GPR_ASSERT(old > n); @@ -785,16 +850,20 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, REF_BY(fd, 1, reason); /* Remove the fd from the polling island: - - Update the fd->polling_island to point to the latest polling island - - Remove the fd from the polling island. - - Remove a ref to the polling island and set fd->polling_island to NULL */ + - Get a lock on the latest polling island (i.e the last island in the + linked list pointed by fd->polling_island). This is the island that + would actually contain the fd + - Remove the fd from the latest polling island + - Unlock the latest polling island + - Set fd->polling_island to NULL (but remove the ref on the polling island + before doing this.) */ gpr_mu_lock(&fd->pi_mu); if (fd->polling_island != NULL) { - fd->polling_island = - polling_island_update_and_lock(fd->polling_island, 1, 0); - polling_island_remove_fd_locked(fd->polling_island, fd, is_fd_closed); + polling_island *pi_latest = polling_island_lock(fd->polling_island); + polling_island_remove_fd_locked(pi_latest, fd, is_fd_closed); + gpr_mu_unlock(&pi_latest->mu); - polling_island_unref_and_unlock(fd->polling_island, 1); + PI_UNREF(fd->polling_island, "fd_orphan"); fd->polling_island = NULL; } gpr_mu_unlock(&fd->pi_mu); @@ -1050,17 +1119,13 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { gpr_mu_unlock(&fd->mu); } -/* Release the reference to pollset->polling_island and set it to NULL. - pollset->mu must be held */ -static void pollset_release_polling_island_locked(grpc_pollset *pollset) { - gpr_mu_lock(&pollset->pi_mu); - if (pollset->polling_island) { - pollset->polling_island = - polling_island_update_and_lock(pollset->polling_island, 1, 0); - polling_island_unref_and_unlock(pollset->polling_island, 1); - pollset->polling_island = NULL; +static void pollset_release_polling_island(grpc_pollset *ps, char *reason) { + gpr_mu_lock(&ps->pi_mu); + if (ps->polling_island != NULL) { + PI_UNREF(ps->polling_island, reason); } - gpr_mu_unlock(&pollset->pi_mu); + ps->polling_island = NULL; + gpr_mu_unlock(&ps->pi_mu); } static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, @@ -1069,8 +1134,9 @@ static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, GPR_ASSERT(!pollset_has_workers(pollset)); pollset->finish_shutdown_called = true; - pollset_release_polling_island_locked(pollset); + /* Release the ref and set pollset->polling_island to NULL */ + pollset_release_polling_island(pollset, "ps_shutdown"); grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); } @@ -1110,7 +1176,7 @@ static void pollset_reset(grpc_pollset *pollset) { pollset->finish_shutdown_called = false; pollset->kicked_without_pollers = false; pollset->shutdown_done = NULL; - pollset_release_polling_island_locked(pollset); + pollset_release_polling_island(pollset, "ps_reset"); } #define GRPC_EPOLL_MAX_EVENTS 1000 @@ -1124,28 +1190,37 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the - polling island pointed by pollset->polling_island. + latest polling island pointed by pollset->polling_island. Acquire the following locks: - pollset->mu (which we already have) - pollset->pi_mu - - pollset->polling_island->mu (call polling_island_update_and_lock())*/ + - pollset->polling_island lock */ gpr_mu_lock(&pollset->pi_mu); - pi = pollset->polling_island; - if (pi == NULL) { - pi = polling_island_create(NULL, 1); + if (pollset->polling_island == NULL) { + pollset->polling_island = polling_island_create(NULL); + PI_ADD_REF(pollset->polling_island, "ps"); } - /* In addition to locking the polling island, add a ref so that the island - does not get destroyed (which means the epoll_fd won't be closed) while - we are are doing an epoll_wait() on the epoll_fd */ - pi = polling_island_update_and_lock(pi, 1, 1); + pi = polling_island_lock(pollset->polling_island); epoll_fd = pi->epoll_fd; - /* Update the pollset->polling_island */ - pollset->polling_island = pi; + /* Update the pollset->polling_island since the island being pointed by + pollset->polling_island may not be the latest (i.e pi) */ + if (pollset->polling_island != pi) { + /* Always do PI_ADD_REF before PI_UNREF because PI_UNREF may cause the + polling island to be deleted */ + PI_ADD_REF(pi, "ps"); + PI_UNREF(pollset->polling_island, "ps"); + pollset->polling_island = pi; + } + + /* Add an extra ref so that the island does not get destroyed (which means + the epoll_fd won't be closed) while we are are doing an epoll_wait() on the + epoll_fd */ + PI_ADD_REF(pi, "ps_work"); - polling_island_unref_and_unlock(pollset->polling_island, 0); /* Keep the ref*/ + gpr_mu_unlock(&pi->mu); gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); @@ -1193,14 +1268,12 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, GPR_ASSERT(pi != NULL); - /* Before leaving, release the extra ref we added to the polling island */ - /* It is important to note that at this point 'pi' may not be the same as - * pollset->polling_island. This is because pollset->polling_island pointer - * gets updated whenever the underlying polling island is merged with another - * island and while we are doing epoll_wait() above, the polling island may - * have been merged */ - pi = polling_island_update_and_lock(pi, 1, 0); /* No new ref added */ - polling_island_unref_and_unlock(pi, 1); + /* Before leaving, release the extra ref we added to the polling island. It + is important to use "pi" here (i.e our old copy of pollset->polling_island + that we got before releasing the polling island lock). This is because + pollset->polling_island pointer might get udpated in other parts of the + code when there is an island merge while we are doing epoll_wait() above */ + PI_UNREF(pi, "ps_work"); GPR_TIMER_END("pollset_work_and_unlock", 0); } @@ -1297,20 +1370,34 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, if (fd->polling_island == pollset->polling_island) { pi_new = fd->polling_island; if (pi_new == NULL) { - pi_new = polling_island_create(fd, 2); + pi_new = polling_island_create(fd); } } else if (fd->polling_island == NULL) { - pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); - polling_island_add_fds_locked(pollset->polling_island, &fd, 1, true); + pi_new = polling_island_lock(pollset->polling_island); + polling_island_add_fds_locked(pi_new, &fd, 1, true); gpr_mu_unlock(&pi_new->mu); } else if (pollset->polling_island == NULL) { - pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); + pi_new = polling_island_lock(fd->polling_island); gpr_mu_unlock(&pi_new->mu); } else { pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); } - fd->polling_island = pollset->polling_island = pi_new; + if (fd->polling_island != pi_new) { + PI_ADD_REF(pi_new, "fd"); + if (fd->polling_island != NULL) { + PI_UNREF(fd->polling_island, "fd"); + } + fd->polling_island = pi_new; + } + + if (pollset->polling_island != pi_new) { + PI_ADD_REF(pi_new, "ps"); + if (pollset->polling_island != NULL) { + PI_UNREF(pollset->polling_island, "ps"); + } + pollset->polling_island = pi_new; + } gpr_mu_unlock(&fd->pi_mu); gpr_mu_unlock(&pollset->pi_mu); @@ -1481,28 +1568,19 @@ void *grpc_pollset_get_polling_island(grpc_pollset *ps) { return pi; } -static polling_island *get_polling_island(polling_island *p) { - if (p == NULL) { - return NULL; - } +bool grpc_are_polling_islands_equal(void *p, void *q) { + polling_island *p1 = p; + polling_island *p2 = q; - polling_island *next; - gpr_mu_lock(&p->mu); - while (p->merged_to != NULL) { - next = p->merged_to; - gpr_mu_unlock(&p->mu); - p = next; - gpr_mu_lock(&p->mu); + polling_island_lock_pair(&p1, &p2); + if (p1 == p2) { + gpr_mu_unlock(&p1->mu); + } else { + gpr_mu_unlock(&p1->mu); + gpr_mu_unlock(&p2->mu); } - gpr_mu_unlock(&p->mu); - - return p; -} -bool grpc_are_polling_islands_equal(void *p, void *q) { - p = get_polling_island(p); - q = get_polling_island(q); - return p == q; + return p1 == p2; } /******************************************************************************* From aa338326ed399832394078282664c6bcf66d11e7 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Mon, 20 Jun 2016 16:47:40 -0700 Subject: [PATCH 077/215] Revert ProtoService.m, add an exception for its incompatible-pointer-types warning --- src/objective-c/ProtoRPC/ProtoService.m | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/objective-c/ProtoRPC/ProtoService.m b/src/objective-c/ProtoRPC/ProtoService.m index 4a14570d818..cd9bc7aeac4 100644 --- a/src/objective-c/ProtoRPC/ProtoService.m +++ b/src/objective-c/ProtoRPC/ProtoService.m @@ -65,19 +65,22 @@ return self; } -- (GRPCProtoCall *)RPCToMethod:(NSString *)method +#pragma clang diagnostic push +#pragma clang diagnostic ignored "-Wincompatible-pointer-types" +- (ProtoRPC *)RPCToMethod:(NSString *)method requestsWriter:(GRXWriter *)requestsWriter responseClass:(Class)responseClass responsesWriteable:(id)responsesWriteable { - GRPCProtoMethod *methodName = [[GRPCProtoMethod alloc] initWithPackage:_packageName + ProtoMethod *methodName = [[ProtoMethod alloc] initWithPackage:_packageName service:_serviceName method:method]; - return [[GRPCProtoCall alloc] initWithHost:_host + return [[ProtoRPC alloc] initWithHost:_host method:methodName requestsWriter:requestsWriter responseClass:responseClass responsesWriteable:responsesWriteable]; } +#pragma clang diagnostic pop @end @implementation GRPCProtoService From cddf697ab44a7bab1821915e1e3f6a0f08ca1706 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 21 Jun 2016 08:27:07 -0700 Subject: [PATCH 078/215] Fix refcounting tsan failures and grab pollset lock in the function pollset_add_fd --- src/core/lib/iomgr/ev_epoll_linux.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 72288889c02..7cc69c876db 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -291,11 +291,11 @@ void pi_unref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, #endif long pi_add_ref(polling_island *pi, int ref_cnt) { - return gpr_atm_no_barrier_fetch_add(&pi->ref_count, ref_cnt); + return gpr_atm_full_fetch_add(&pi->ref_count, ref_cnt); } long pi_unref(polling_island *pi, int ref_cnt) { - long old_cnt = gpr_atm_no_barrier_fetch_add(&pi->ref_count, -ref_cnt); + long old_cnt = gpr_atm_full_fetch_add(&pi->ref_count, -ref_cnt); /* If ref count went to zero, delete the polling island. Note that this need not be done under a lock. Once the ref count goes to zero, we are @@ -311,6 +311,8 @@ long pi_unref(polling_island *pi, int ref_cnt) { if (next != NULL) { PI_UNREF(next, "pi_delete"); /* Recursive call */ } + } else { + GPR_ASSERT(old_cnt > ref_cnt); } return old_cnt; @@ -445,8 +447,8 @@ static polling_island *polling_island_create(grpc_fd *initial_fd) { pi->fds = NULL; } - gpr_atm_no_barrier_store(&pi->ref_count, 0); - gpr_atm_no_barrier_store(&pi->merged_to, NULL); + gpr_atm_rel_store(&pi->ref_count, 0); + gpr_atm_rel_store(&pi->merged_to, NULL); pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); @@ -1347,7 +1349,7 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { - /* TODO sreek - Double check if we need to get a pollset->mu lock here */ + gpr_mu_lock(&pollset->mu); gpr_mu_lock(&pollset->pi_mu); gpr_mu_lock(&fd->pi_mu); @@ -1401,6 +1403,7 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, gpr_mu_unlock(&fd->pi_mu); gpr_mu_unlock(&pollset->pi_mu); + gpr_mu_unlock(&pollset->mu); } /******************************************************************************* From e63246d1007516ca5c74043139304a371c0ce672 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Tue, 21 Jun 2016 09:03:11 -0700 Subject: [PATCH 079/215] clang-format --- test/core/iomgr/tcp_server_posix_test.c | 2 +- test/core/surface/server_chttp2_test.c | 2 +- test/core/util/test_tcp_server.c | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/test/core/iomgr/tcp_server_posix_test.c b/test/core/iomgr/tcp_server_posix_test.c index 6361e7afcbc..6e2d1d0fc9e 100644 --- a/test/core/iomgr/tcp_server_posix_test.c +++ b/test/core/iomgr/tcp_server_posix_test.c @@ -166,7 +166,7 @@ static void test_no_op_with_port_and_start(void) { grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; struct sockaddr_in addr; grpc_tcp_server *s; - GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, NULL, &s)); + GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, NULL, &s)); LOG_TEST("test_no_op_with_port_and_start"); int port; diff --git a/test/core/surface/server_chttp2_test.c b/test/core/surface/server_chttp2_test.c index 40cfa6b5989..6310b6f00b0 100644 --- a/test/core/surface/server_chttp2_test.c +++ b/test/core/surface/server_chttp2_test.c @@ -54,7 +54,7 @@ void test_add_same_port_twice() { a.key = GRPC_ARG_ALLOW_REUSEPORT; a.value.integer = 0; grpc_channel_args args = {1, &a}; - + int port = grpc_pick_unused_port_or_die(); char *addr = NULL; grpc_completion_queue *cq = grpc_completion_queue_create(NULL); diff --git a/test/core/util/test_tcp_server.c b/test/core/util/test_tcp_server.c index c1d307fc779..8a0b3932d86 100644 --- a/test/core/util/test_tcp_server.c +++ b/test/core/util/test_tcp_server.c @@ -72,8 +72,8 @@ void test_tcp_server_start(test_tcp_server *server, int port) { addr.sin_port = htons((uint16_t)port); memset(&addr.sin_addr, 0, sizeof(addr.sin_addr)); - grpc_error *error = - grpc_tcp_server_create(&server->shutdown_complete, NULL, &server->tcp_server); + grpc_error *error = grpc_tcp_server_create(&server->shutdown_complete, NULL, + &server->tcp_server); GPR_ASSERT(error == GRPC_ERROR_NONE); error = grpc_tcp_server_add_port(server->tcp_server, &addr, sizeof(addr), &port_added); From 39f9ac9b2a51fdcdbd3693d8a04eec3473eb9438 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 16 Jun 2016 16:53:59 -0700 Subject: [PATCH 080/215] Test polling island merges --- Makefile | 36 ++++ build.yaml | 12 ++ src/core/lib/iomgr/ev_epoll_linux.c | 51 +++++- src/core/lib/iomgr/ev_epoll_linux.h | 6 + src/core/lib/iomgr/ev_posix.c | 7 + src/core/lib/iomgr/ev_posix.h | 3 + test/core/iomgr/ev_epoll_linux_test.c | 222 +++++++++++++++++++++++ tools/run_tests/sources_and_headers.json | 16 ++ tools/run_tests/tests.json | 15 ++ 9 files changed, 366 insertions(+), 2 deletions(-) create mode 100644 test/core/iomgr/ev_epoll_linux_test.c diff --git a/Makefile b/Makefile index e615704395b..825684cc2dd 100644 --- a/Makefile +++ b/Makefile @@ -905,6 +905,7 @@ dns_resolver_connectivity_test: $(BINDIR)/$(CONFIG)/dns_resolver_connectivity_te dns_resolver_test: $(BINDIR)/$(CONFIG)/dns_resolver_test dualstack_socket_test: $(BINDIR)/$(CONFIG)/dualstack_socket_test endpoint_pair_test: $(BINDIR)/$(CONFIG)/endpoint_pair_test +ev_epoll_linux_test: $(BINDIR)/$(CONFIG)/ev_epoll_linux_test fd_conservation_posix_test: $(BINDIR)/$(CONFIG)/fd_conservation_posix_test fd_posix_test: $(BINDIR)/$(CONFIG)/fd_posix_test fling_client: $(BINDIR)/$(CONFIG)/fling_client @@ -1242,6 +1243,7 @@ buildtests_c: privatelibs_c \ $(BINDIR)/$(CONFIG)/dns_resolver_test \ $(BINDIR)/$(CONFIG)/dualstack_socket_test \ $(BINDIR)/$(CONFIG)/endpoint_pair_test \ + $(BINDIR)/$(CONFIG)/ev_epoll_linux_test \ $(BINDIR)/$(CONFIG)/fd_conservation_posix_test \ $(BINDIR)/$(CONFIG)/fd_posix_test \ $(BINDIR)/$(CONFIG)/fling_client \ @@ -1512,6 +1514,8 @@ test_c: buildtests_c $(Q) $(BINDIR)/$(CONFIG)/dualstack_socket_test || ( echo test dualstack_socket_test failed ; exit 1 ) $(E) "[RUN] Testing endpoint_pair_test" $(Q) $(BINDIR)/$(CONFIG)/endpoint_pair_test || ( echo test endpoint_pair_test failed ; exit 1 ) + $(E) "[RUN] Testing ev_epoll_linux_test" + $(Q) $(BINDIR)/$(CONFIG)/ev_epoll_linux_test || ( echo test ev_epoll_linux_test failed ; exit 1 ) $(E) "[RUN] Testing fd_conservation_posix_test" $(Q) $(BINDIR)/$(CONFIG)/fd_conservation_posix_test || ( echo test fd_conservation_posix_test failed ; exit 1 ) $(E) "[RUN] Testing fd_posix_test" @@ -7130,6 +7134,38 @@ endif endif +EV_EPOLL_LINUX_TEST_SRC = \ + test/core/iomgr/ev_epoll_linux_test.c \ + +EV_EPOLL_LINUX_TEST_OBJS = $(addprefix $(OBJDIR)/$(CONFIG)/, $(addsuffix .o, $(basename $(EV_EPOLL_LINUX_TEST_SRC)))) +ifeq ($(NO_SECURE),true) + +# You can't build secure targets if you don't have OpenSSL. + +$(BINDIR)/$(CONFIG)/ev_epoll_linux_test: openssl_dep_error + +else + + + +$(BINDIR)/$(CONFIG)/ev_epoll_linux_test: $(EV_EPOLL_LINUX_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a + $(E) "[LD] Linking $@" + $(Q) mkdir -p `dirname $@` + $(Q) $(LD) $(LDFLAGS) $(EV_EPOLL_LINUX_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a $(LDLIBS) $(LDLIBS_SECURE) -o $(BINDIR)/$(CONFIG)/ev_epoll_linux_test + +endif + +$(OBJDIR)/$(CONFIG)/test/core/iomgr/ev_epoll_linux_test.o: $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a + +deps_ev_epoll_linux_test: $(EV_EPOLL_LINUX_TEST_OBJS:.o=.dep) + +ifneq ($(NO_SECURE),true) +ifneq ($(NO_DEPS),true) +-include $(EV_EPOLL_LINUX_TEST_OBJS:.o=.dep) +endif +endif + + FD_CONSERVATION_POSIX_TEST_SRC = \ test/core/iomgr/fd_conservation_posix_test.c \ diff --git a/build.yaml b/build.yaml index 7790e0c5174..84f4ea521be 100644 --- a/build.yaml +++ b/build.yaml @@ -1407,6 +1407,18 @@ targets: - grpc - gpr_test_util - gpr +- name: ev_epoll_linux_test + build: test + language: c + src: + - test/core/iomgr/ev_epoll_linux_test.c + deps: + - grpc_test_util + - grpc + - gpr_test_util + - gpr + platforms: + - linux - name: fd_conservation_posix_test build: test language: c diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 1fb59474640..ed2c494b783 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -317,8 +317,9 @@ static void polling_island_remove_all_fds_locked(polling_island *pi, if (err < 0 && errno != ENOENT) { /* TODO: sreek - We need a better way to bubble up this error instead of * just logging a message */ - gpr_log(GPR_ERROR, "epoll_ctl deleting fds[%zu]: %d failed with error: %s", - i, pi->fds[i]->fd, strerror(errno)); + gpr_log(GPR_ERROR, + "epoll_ctl deleting fds[%zu]: %d failed with error: %s", i, + pi->fds[i]->fd, strerror(errno)); } if (remove_fd_refs) { @@ -1458,6 +1459,52 @@ static void pollset_set_del_pollset_set(grpc_exec_ctx *exec_ctx, gpr_mu_unlock(&bag->mu); } +/* Test helper functions + * */ +void *grpc_fd_get_polling_island(grpc_fd *fd) { + polling_island *pi; + + gpr_mu_lock(&fd->pi_mu); + pi = fd->polling_island; + gpr_mu_unlock(&fd->pi_mu); + + return pi; +} + +void *grpc_pollset_get_polling_island(grpc_pollset *ps) { + polling_island *pi; + + gpr_mu_lock(&ps->pi_mu); + pi = ps->polling_island; + gpr_mu_unlock(&ps->pi_mu); + + return pi; +} + +static polling_island *get_polling_island(polling_island *p) { + if (p == NULL) { + return NULL; + } + + polling_island *next; + gpr_mu_lock(&p->mu); + while (p->merged_to != NULL) { + next = p->merged_to; + gpr_mu_unlock(&p->mu); + p = next; + gpr_mu_lock(&p->mu); + } + gpr_mu_unlock(&p->mu); + + return p; +} + +bool grpc_are_polling_islands_equal(void *p, void *q) { + p = get_polling_island(p); + q = get_polling_island(q); + return p == q; +} + /******************************************************************************* * Event engine binding */ diff --git a/src/core/lib/iomgr/ev_epoll_linux.h b/src/core/lib/iomgr/ev_epoll_linux.h index 8c819975a4c..7a494aba198 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.h +++ b/src/core/lib/iomgr/ev_epoll_linux.h @@ -38,4 +38,10 @@ const grpc_event_engine_vtable *grpc_init_epoll_linux(void); +#ifdef GPR_LINUX_EPOLL +void *grpc_fd_get_polling_island(grpc_fd *fd); +void *grpc_pollset_get_polling_island(grpc_pollset *ps); +bool grpc_are_polling_islands_equal(void *p, void *q); +#endif /* defined(GPR_LINUX_EPOLL) */ + #endif /* GRPC_CORE_LIB_IOMGR_EV_EPOLL_LINUX_H */ diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index 2b15967adcc..5b20600a6f7 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -54,6 +54,7 @@ grpc_poll_function_type grpc_poll_function = poll; static const grpc_event_engine_vtable *g_event_engine; +static const char* g_poll_strategy_name = NULL; typedef const grpc_event_engine_vtable *(*event_engine_factory_fn)(void); @@ -101,6 +102,7 @@ static void try_engine(const char *engine) { for (size_t i = 0; i < GPR_ARRAY_SIZE(g_factories); i++) { if (is(engine, g_factories[i].name)) { if ((g_event_engine = g_factories[i].factory())) { + g_poll_strategy_name = g_factories[i].name; gpr_log(GPR_DEBUG, "Using polling engine: %s", g_factories[i].name); return; } @@ -108,6 +110,11 @@ static void try_engine(const char *engine) { } } +/* Call this only after calling grpc_event_engine_init() */ +const char *grpc_get_poll_strategy_name() { + return g_poll_strategy_name; +} + void grpc_event_engine_init(void) { char *s = gpr_getenv("GRPC_POLL_STRATEGY"); if (s == NULL) { diff --git a/src/core/lib/iomgr/ev_posix.h b/src/core/lib/iomgr/ev_posix.h index 344bf63438a..3ed5a5f9562 100644 --- a/src/core/lib/iomgr/ev_posix.h +++ b/src/core/lib/iomgr/ev_posix.h @@ -98,6 +98,9 @@ typedef struct grpc_event_engine_vtable { void grpc_event_engine_init(void); void grpc_event_engine_shutdown(void); +/* Return the name of the poll strategy */ +const char* grpc_get_poll_strategy_name(); + /* Create a wrapped file descriptor. Requires fd is a non-blocking file descriptor. This takes ownership of closing fd. */ diff --git a/test/core/iomgr/ev_epoll_linux_test.c b/test/core/iomgr/ev_epoll_linux_test.c new file mode 100644 index 00000000000..51da15faa7a --- /dev/null +++ b/test/core/iomgr/ev_epoll_linux_test.c @@ -0,0 +1,222 @@ +/* + * + * Copyright 2015, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include "src/core/lib/iomgr/ev_epoll_linux.h" +#include "src/core/lib/iomgr/ev_posix.h" + +#include +#include +#include +#include + +#include +#include + +#include "src/core/lib/iomgr/iomgr.h" +#include "test/core/util/test_config.h" + +typedef struct test_pollset { + grpc_pollset *pollset; + gpr_mu *mu; +} test_pollset; + +typedef struct test_fd { + int inner_fd; + grpc_fd *fd; +} test_fd; + +static void test_fd_init(test_fd *fds, int num_fds) { + int i; + for (i = 0; i < num_fds; i++) { + fds[i].inner_fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + fds[i].fd = grpc_fd_create(fds[i].inner_fd, "test_fd"); + } +} + +static void test_fd_cleanup(grpc_exec_ctx *exec_ctx, test_fd *fds, + int num_fds) { + int release_fd; + int i; + + for (i = 0; i < num_fds; i++) { + grpc_fd_shutdown(exec_ctx, fds[i].fd); + grpc_exec_ctx_flush(exec_ctx); + + grpc_fd_orphan(exec_ctx, fds[i].fd, NULL, &release_fd, "test_fd_cleanup"); + grpc_exec_ctx_flush(exec_ctx); + + GPR_ASSERT(release_fd == fds[i].inner_fd); + close(fds[i].inner_fd); + } +} + +static void test_pollset_init(test_pollset *pollsets, int num_pollsets) { + int i; + for (i = 0; i < num_pollsets; i++) { + pollsets[i].pollset = gpr_malloc(grpc_pollset_size()); + grpc_pollset_init(pollsets[i].pollset, &pollsets[i].mu); + } +} + +static void destroy_pollset(grpc_exec_ctx *exec_ctx, void *p, bool success) { + grpc_pollset_destroy(p); +} + +static void test_pollset_cleanup(grpc_exec_ctx *exec_ctx, + test_pollset *pollsets, int num_pollsets) { + grpc_closure destroyed; + int i; + + for (i = 0; i < num_pollsets; i++) { + grpc_closure_init(&destroyed, destroy_pollset, pollsets[i].pollset); + grpc_pollset_shutdown(exec_ctx, pollsets[i].pollset, &destroyed); + + grpc_exec_ctx_flush(exec_ctx); + gpr_free(pollsets[i].pollset); + } +} + +#define NUM_FDS 8 +#define NUM_POLLSETS 4 +/* + * Cases to test: + * case 1) Polling islands of both fd and pollset are NULL + * case 2) Polling island of fd is NULL but that of pollset is not-NULL + * case 3) Polling island of fd is not-NULL but that of pollset is NULL + * case 4) Polling islands of both fd and pollset are not-NULL and: + * case 4.1) Polling islands of fd and pollset are equal + * case 4.2) Polling islands of fd and pollset are NOT-equal (This results + * in a merge) + * */ +static void test_add_fd_to_pollset() { + grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; + test_fd fds[NUM_FDS]; + test_pollset pollsets[NUM_POLLSETS]; + void *expected_pi = NULL; + int i; + + test_fd_init(fds, NUM_FDS); + test_pollset_init(pollsets, NUM_POLLSETS); + + /*Step 1. + * Create three polling islands (This will exercise test case 1 and 2) with + * the following configuration: + * polling island 0 = { fds:0,1,2, pollsets:0} + * polling island 1 = { fds:3,4, pollsets:1} + * polling island 2 = { fds:5,6,7 pollsets:2} + * + *Step 2. + * Add pollset 3 to polling island 0 (by adding fds 0 and 1 to pollset 3) + * (This will exercise test cases 3 and 4.1). The configuration becomes: + * polling island 0 = { fds:0,1,2, pollsets:0,3} <<< pollset 3 added here + * polling island 1 = { fds:3,4, pollsets:1} + * polling island 2 = { fds:5,6,7 pollsets:2} + * + *Step 3. + * Merge polling islands 0 and 1 by adding fd 0 to pollset 1 (This will + * exercise test case 4.2). The configuration becomes: + * polling island (merged) = {fds: 0,1,2,3,4, pollsets: 0,1,3} + * polling island 2 = {fds: 5,6,7 pollsets: 2} + * + *Step 4. + * Finally do one more merge by adding fd 3 to pollset 2. + * polling island (merged) = {fds: 0,1,2,3,4,5,6,7, pollsets: 0,1,2,3} + */ + + /* == Step 1 == */ + for (i = 0; i <= 2; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[0].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + for (i = 3; i <= 4; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + for (i = 5; i <= 7; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + /* == Step 2 == */ + for (i = 0; i <= 1; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[3].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + /* == Step 3 == */ + grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, fds[0].fd); + grpc_exec_ctx_flush(&exec_ctx); + + /* == Step 4 == */ + grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, fds[3].fd); + grpc_exec_ctx_flush(&exec_ctx); + + /* All polling islands are merged at this point */ + + /* Compare Fd:0's polling island with that of all other Fds */ + expected_pi = grpc_fd_get_polling_island(fds[0].fd); + for (i = 1; i < NUM_FDS; i++) { + GPR_ASSERT(grpc_are_polling_islands_equal( + expected_pi, grpc_fd_get_polling_island(fds[i].fd))); + } + + /* Compare Fd:0's polling island with that of all other pollsets */ + for (i = 0; i < NUM_POLLSETS; i++) { + GPR_ASSERT(grpc_are_polling_islands_equal( + expected_pi, grpc_pollset_get_polling_island(pollsets[i].pollset))); + } + + test_fd_cleanup(&exec_ctx, fds, NUM_FDS); + test_pollset_cleanup(&exec_ctx, pollsets, NUM_POLLSETS); + grpc_exec_ctx_finish(&exec_ctx); +} + +int main(int argc, char **argv) { + const char *poll_strategy = NULL; + grpc_test_init(argc, argv); + grpc_iomgr_init(); + + poll_strategy = grpc_get_poll_strategy_name(); + if (poll_strategy != NULL && strcmp(poll_strategy, "epoll") == 0) { + test_add_fd_to_pollset(); + } else { + gpr_log(GPR_INFO, + "Skipping the test. The test is only relevant for 'epoll' " + "strategy. and the current strategy is: '%s'", + poll_strategy); + } + grpc_iomgr_shutdown(); + return 0; +} diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index e8ff61dc3fb..e9df72e43a1 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -315,6 +315,22 @@ "third_party": false, "type": "target" }, + { + "deps": [ + "gpr", + "gpr_test_util", + "grpc", + "grpc_test_util" + ], + "headers": [], + "language": "c", + "name": "ev_epoll_linux_test", + "src": [ + "test/core/iomgr/ev_epoll_linux_test.c" + ], + "third_party": false, + "type": "target" + }, { "deps": [ "gpr", diff --git a/tools/run_tests/tests.json b/tools/run_tests/tests.json index 5a84a41b638..ba661840dad 100644 --- a/tools/run_tests/tests.json +++ b/tools/run_tests/tests.json @@ -377,6 +377,21 @@ "windows" ] }, + { + "args": [], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "gtest": false, + "language": "c", + "name": "ev_epoll_linux_test", + "platforms": [ + "linux" + ] + }, { "args": [], "ci_platforms": [ From 94cda1a9c6ae86ab176d357b8822332d70283cde Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 17 Jun 2016 13:28:38 -0700 Subject: [PATCH 081/215] Significantly refactor the polling island locking and refcounting code --- src/core/lib/iomgr/ev_epoll_linux.c | 462 ++++++++++++++++------------ 1 file changed, 270 insertions(+), 192 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index ed2c494b783..72288889c02 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -140,18 +140,40 @@ static void fd_global_shutdown(void); #define CLOSURE_READY ((grpc_closure *)1) /******************************************************************************* - * Polling-island Declarations + * Polling island Declarations */ -/* TODO: sree: Consider making ref_cnt and merged_to to gpr_atm - This would - * significantly reduce the number of mutex acquisition calls. */ + +// #define GRPC_PI_REF_COUNT_DEBUG +#ifdef GRPC_PI_REF_COUNT_DEBUG + +#define PI_ADD_REF(p, r) pi_add_ref_dbg((p), 1, (r), __FILE__, __LINE__) +#define PI_UNREF(p, r) pi_unref_dbg((p), 1, (r), __FILE__, __LINE__) + +#else /* defined(GRPC_PI_REF_COUNT_DEBUG) */ + +#define PI_ADD_REF(p, r) pi_add_ref((p), 1) +#define PI_UNREF(p, r) pi_unref((p), 1) + +#endif /* !defined(GPRC_PI_REF_COUNT_DEBUG) */ + typedef struct polling_island { gpr_mu mu; - int ref_cnt; - - /* Points to the polling_island this merged into. - * If merged_to is not NULL, all the remaining fields (except mu and ref_cnt) - * are invalid and must be ignored */ - struct polling_island *merged_to; + /* Ref count. Use PI_ADD_REF() and PI_UNREF() macros to increment/decrement + the refcount. + Once the ref count becomes zero, this structure is destroyed which means + we should ensure that there is never a scenario where a PI_ADD_REF() is + racing with a PI_UNREF() that just made the ref_count zero. */ + gpr_atm ref_count; + + /* Pointer to the polling_island this merged into. + * merged_to value is only set once in polling_island's lifetime (and that too + * only if the island is merged with another island). Because of this, we can + * use gpr_atm type here so that we can do atomic access on this and reduce + * lock contention on 'mu' mutex. + * + * Note that if this field is not NULL (i.e not 0), all the remaining fields + * (except mu and ref_count) are invalid and must be ignored. */ + gpr_atm merged_to; /* The fd of the underlying epoll set */ int epoll_fd; @@ -236,6 +258,8 @@ static grpc_wakeup_fd polling_island_wakeup_fd; static gpr_mu g_pi_freelist_mu; static polling_island *g_pi_freelist = NULL; +static void polling_island_delete(); /* Forward declaration */ + #ifdef GRPC_TSAN /* Currently TSAN may incorrectly flag data races between epoll_ctl and epoll_wait for any grpc_fd structs that are added to the epoll set via @@ -247,6 +271,51 @@ static polling_island *g_pi_freelist = NULL; gpr_atm g_epoll_sync; #endif /* defined(GRPC_TSAN) */ +#ifdef GRPC_PI_REF_COUNT_DEBUG +long pi_add_ref(polling_island *pi, int ref_cnt); +long pi_unref(polling_island *pi, int ref_cnt); + +void pi_add_ref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, + int line) { + long old_cnt = pi_add_ref(pi, ref_cnt); + gpr_log(GPR_DEBUG, "Add ref pi: %p, old:%ld -> new:%ld (%s) - (%s, %d)", + (void *)pi, old_cnt, (old_cnt + ref_cnt), reason, file, line); +} + +void pi_unref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, + int line) { + long old_cnt = pi_unref(pi, ref_cnt); + gpr_log(GPR_DEBUG, "Unref pi: %p, old:%ld -> new:%ld (%s) - (%s, %d)", + (void *)pi, old_cnt, (old_cnt - ref_cnt), reason, file, line); +} +#endif + +long pi_add_ref(polling_island *pi, int ref_cnt) { + return gpr_atm_no_barrier_fetch_add(&pi->ref_count, ref_cnt); +} + +long pi_unref(polling_island *pi, int ref_cnt) { + long old_cnt = gpr_atm_no_barrier_fetch_add(&pi->ref_count, -ref_cnt); + + /* If ref count went to zero, delete the polling island. Note that this need + not be done under a lock. Once the ref count goes to zero, we are + guaranteed that no one else holds a reference to the polling island (and + that there is no racing pi_add_ref() call either. + + Also, if we are deleting the polling island and the merged_to field is + non-empty, we should remove a ref to the merged_to polling island + */ + if (old_cnt == ref_cnt) { + polling_island *next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + polling_island_delete(pi); + if (next != NULL) { + PI_UNREF(next, "pi_delete"); /* Recursive call */ + } + } + + return old_cnt; +} + /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, size_t fd_count, bool add_fd_refs) { @@ -355,8 +424,7 @@ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, } } -static polling_island *polling_island_create(grpc_fd *initial_fd, - int initial_ref_cnt) { +static polling_island *polling_island_create(grpc_fd *initial_fd) { polling_island *pi = NULL; /* Try to get one from the polling island freelist */ @@ -377,6 +445,9 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, pi->fds = NULL; } + gpr_atm_no_barrier_store(&pi->ref_count, 0); + gpr_atm_no_barrier_store(&pi->merged_to, NULL); + pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); if (pi->epoll_fd < 0) { @@ -387,14 +458,12 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, polling_island_add_wakeup_fd_locked(pi, &grpc_global_wakeup_fd); - pi->ref_cnt = initial_ref_cnt; - pi->merged_to = NULL; pi->next_free = NULL; if (initial_fd != NULL) { - /* It is not really needed to get the pi->mu lock here. If this is a newly - created polling island (or one that we got from the freelist), no one - else would be holding a lock to it anyway */ + /* Lock the polling island here just in case we got this structure from the + freelist and the polling island lock was not released yet (by the code + that adds the polling island to the freelist) */ gpr_mu_lock(&pi->mu); polling_island_add_fds_locked(pi, &initial_fd, 1, true); gpr_mu_unlock(&pi->mu); @@ -404,140 +473,136 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, } static void polling_island_delete(polling_island *pi) { - GPR_ASSERT(pi->ref_cnt == 0); GPR_ASSERT(pi->fd_cnt == 0); + gpr_atm_rel_store(&pi->merged_to, NULL); + close(pi->epoll_fd); pi->epoll_fd = -1; - pi->merged_to = NULL; - gpr_mu_lock(&g_pi_freelist_mu); pi->next_free = g_pi_freelist; g_pi_freelist = pi; gpr_mu_unlock(&g_pi_freelist_mu); } -void polling_island_unref_and_unlock(polling_island *pi, int unref_by) { - pi->ref_cnt -= unref_by; - int ref_cnt = pi->ref_cnt; - GPR_ASSERT(ref_cnt >= 0); - - gpr_mu_unlock(&pi->mu); - - if (ref_cnt == 0) { - polling_island_delete(pi); - } -} - -polling_island *polling_island_update_and_lock(polling_island *pi, int unref_by, - int add_ref_by) { +/* Gets the lock on the *latest* polling island i.e the last polling island in + the linked list (linked by 'merged_to' link). Call gpr_mu_unlock on the + returned polling island's mu. + Usage: To lock/unlock polling island "pi", do the following: + polling_island *pi_latest = polling_island_lock(pi); + ... + ... critical section .. + ... + gpr_mu_unlock(&pi_latest->mu); //NOTE: use pi_latest->mu. NOT pi->mu */ +polling_island *polling_island_lock(polling_island *pi) { polling_island *next = NULL; - gpr_mu_lock(&pi->mu); - while (pi->merged_to != NULL) { - next = pi->merged_to; - polling_island_unref_and_unlock(pi, unref_by); + while (true) { + next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + if (next == NULL) { + /* pi is the last node in the linked list. Get the lock and check again + (under the pi->mu lock) that pi is still the last node (because a merge + may have happend after the (next == NULL) check above and before + getting the pi->mu lock. + If pi is the last node, we are done. If not, unlock and continue + traversing the list */ + gpr_mu_lock(&pi->mu); + next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + if (next == NULL) { + break; + } + gpr_mu_unlock(&pi->mu); + } + pi = next; - gpr_mu_lock(&pi->mu); } - pi->ref_cnt += add_ref_by; return pi; } -void polling_island_pair_update_and_lock(polling_island **p, - polling_island **q) { +/* Gets the lock on the *latest* polling islands pointed by *p and *q. + This function is needed because calling the following block of code to obtain + locks on polling islands (*p and *q) is prone to deadlocks. + { + polling_island_lock(*p); + polling_island_lock(*q); + } + + Usage/exmaple: + polling_island *p1; + polling_island *p2; + .. + polling_island_lock_pair(&p1, &p2); + .. + .. Critical section with both p1 and p2 locked + .. + // Release locks + // **IMPORTANT**: Make sure you check p1 == p2 AFTER the function + // polling_island_lock_pair() was called and if so, release the lock only + // once. Note: Even if p1 != p2 beforec calling polling_island_lock_pair(), + // they might be after the function returns: + if (p1 == p2) { + gpr_mu_unlock(&p1->mu) + } else { + gpr_mu_unlock(&p1->mu); + gpr_mu_unlock(&p2->mu); + } + +*/ +void polling_island_lock_pair(polling_island **p, polling_island **q) { polling_island *pi_1 = *p; polling_island *pi_2 = *q; - polling_island *temp = NULL; - bool pi_1_locked = false; - bool pi_2_locked = false; - int num_swaps = 0; - - /* Loop until either pi_1 == pi_2 or until we acquired locks on both pi_1 - and pi_2 */ - while (pi_1 != pi_2 && !(pi_1_locked && pi_2_locked)) { - /* The following assertions are true at this point: - - pi_1 != pi_2 (else, the while loop would have exited) - - pi_1 MAY be locked - - pi_2 is NOT locked */ - - /* To maintain lock order consistency, always lock polling_island node with - lower address first. - First, make sure pi_1 < pi_2 before proceeding any further. If it turns - out that pi_1 > pi_2, unlock pi_1 if locked (because pi_2 is not locked - at this point and having pi_1 locked would violate the lock order) and - swap pi_1 and pi_2 so that pi_1 becomes less than pi_2 */ - if (pi_1 > pi_2) { - if (pi_1_locked) { - gpr_mu_unlock(&pi_1->mu); - pi_1_locked = false; - } + polling_island *next_1 = NULL; + polling_island *next_2 = NULL; + + /* The algorithm is simple: + - Go to the last polling islands in the linked lists *pi_1 and *pi_2 (and + keep updating pi_1 and pi_2) + - Then obtain locks on the islands by following a lock order rule of + locking polling_island with lower address first + Special case: Before obtaining the locks, check if pi_1 and pi_2 are + pointing to the same island. If that is the case, we can just call + polling_island_lock() + - After obtaining both the locks, double check that the polling islands + are still the last polling islands in their respective linked lists + (this is because there might have been polling island merges before + we got the lock) + - If the polling islands are the last islands, we are done. If not, + release the locks and continue the process from the first step */ + while (true) { + next_1 = (polling_island *)gpr_atm_acq_load(&pi_1->merged_to); + while (next_1 != NULL) { + pi_1 = next_1; + next_1 = (polling_island *)gpr_atm_acq_load(&pi_1->merged_to); + } - GPR_SWAP(polling_island *, pi_1, pi_2); - num_swaps++; + next_2 = (polling_island *)gpr_atm_acq_load(&pi_2->merged_to); + while (next_2 != NULL) { + pi_2 = next_2; + next_2 = (polling_island *)gpr_atm_acq_load(&pi_2->merged_to); } - /* The following assertions are true at this point: - - pi_1 != pi_2 - - pi_1 < pi_2 (address of pi_1 is less than that of pi_2) - - pi_1 MAYBE locked - - pi_2 is NOT locked */ + if (pi_1 == pi_2) { + pi_1 = pi_2 = polling_island_lock(pi_1); + break; + } - /* Lock pi_1 (if pi_1 is pointing to the terminal node in the list) */ - if (!pi_1_locked) { + if (pi_1 < pi_2) { + gpr_mu_lock(&pi_1->mu); + gpr_mu_lock(&pi_2->mu); + } else { + gpr_mu_lock(&pi_2->mu); gpr_mu_lock(&pi_1->mu); - pi_1_locked = true; - - /* If pi_1 is not terminal node (i.e pi_1->merged_to != NULL), we are not - done locking this polling_island yet. Release the lock on this node and - advance pi_1 to the next node in the list; and go to the beginning of - the loop (we can't proceed to locking pi_2 unless we locked pi_1 first) - */ - if (pi_1->merged_to != NULL) { - temp = pi_1->merged_to; - polling_island_unref_and_unlock(pi_1, 1); - pi_1 = temp; - pi_1_locked = false; - - continue; - } } - /* The following assertions are true at this point: - - pi_1 is locked - - pi_2 is unlocked - - pi_1 != pi_2 */ - - gpr_mu_lock(&pi_2->mu); - pi_2_locked = true; - - /* If pi_2 is not terminal node, we are not done locking this polling_island - yet. Release the lock and update pi_2 to the next node in the list */ - if (pi_2->merged_to != NULL) { - temp = pi_2->merged_to; - polling_island_unref_and_unlock(pi_2, 1); - pi_2 = temp; - pi_2_locked = false; + next_1 = (polling_island *)gpr_atm_acq_load(&pi_1->merged_to); + next_2 = (polling_island *)gpr_atm_acq_load(&pi_2->merged_to); + if (next_1 == NULL && next_2 == NULL) { + break; } - } - /* At this point, either pi_1 == pi_2 AND/OR we got both locks */ - if (pi_1 == pi_2) { - /* We may or may not have gotten the lock. If we didn't, walk the rest of - the polling_island list and get the lock */ - GPR_ASSERT(pi_1_locked || (!pi_1_locked && !pi_2_locked)); - if (!pi_1_locked) { - pi_1 = pi_2 = polling_island_update_and_lock(pi_1, 2, 0); - } - } else { - GPR_ASSERT(pi_1_locked && pi_2_locked); - /* If we swapped pi_1 and pi_2 odd number of times, do one more swap so that - pi_1 and pi_2 point to the same polling_island lists they started off - with at the beginning of this function (i.e *p and *q respectively) */ - if (num_swaps % 2 > 0) { - GPR_SWAP(polling_island *, pi_1, pi_2); - } + gpr_mu_unlock(&pi_1->mu); + gpr_mu_unlock(&pi_2->mu); } *p = pi_1; @@ -546,7 +611,7 @@ void polling_island_pair_update_and_lock(polling_island **p, polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Get locks on both the polling islands */ - polling_island_pair_update_and_lock(&p, &q); + polling_island_lock_pair(&p, &q); if (p == q) { /* Nothing needs to be done here */ @@ -568,15 +633,14 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Wakeup all the pollers (if any) on p so that they can pickup this change */ polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); - p->merged_to = q; + /* Add the 'merged_to' link from p --> q */ + gpr_atm_rel_store(&p->merged_to, q); + PI_ADD_REF(q, "pi_merge"); /* To account for the new incoming ref from p */ - /* - The merged polling island (i.e q) inherits all the ref counts of the - island merging with it (i.e p) - - The island p will lose a ref count */ - q->ref_cnt += p->ref_cnt; - polling_island_unref_and_unlock(p, 1); /* Decrement refcount */ - polling_island_unref_and_unlock(q, 0); /* Just Unlock. Don't decrement ref */ + gpr_mu_unlock(&p->mu); + gpr_mu_unlock(&q->mu); + /* Return the merged polling island */ return q; } @@ -667,6 +731,7 @@ static void unref_by(grpc_fd *fd, int n) { fd->freelist_next = fd_freelist; fd_freelist = fd; grpc_iomgr_unregister_object(&fd->iomgr_object); + gpr_mu_unlock(&fd_freelist_mu); } else { GPR_ASSERT(old > n); @@ -785,16 +850,20 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, REF_BY(fd, 1, reason); /* Remove the fd from the polling island: - - Update the fd->polling_island to point to the latest polling island - - Remove the fd from the polling island. - - Remove a ref to the polling island and set fd->polling_island to NULL */ + - Get a lock on the latest polling island (i.e the last island in the + linked list pointed by fd->polling_island). This is the island that + would actually contain the fd + - Remove the fd from the latest polling island + - Unlock the latest polling island + - Set fd->polling_island to NULL (but remove the ref on the polling island + before doing this.) */ gpr_mu_lock(&fd->pi_mu); if (fd->polling_island != NULL) { - fd->polling_island = - polling_island_update_and_lock(fd->polling_island, 1, 0); - polling_island_remove_fd_locked(fd->polling_island, fd, is_fd_closed); + polling_island *pi_latest = polling_island_lock(fd->polling_island); + polling_island_remove_fd_locked(pi_latest, fd, is_fd_closed); + gpr_mu_unlock(&pi_latest->mu); - polling_island_unref_and_unlock(fd->polling_island, 1); + PI_UNREF(fd->polling_island, "fd_orphan"); fd->polling_island = NULL; } gpr_mu_unlock(&fd->pi_mu); @@ -1050,17 +1119,13 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { gpr_mu_unlock(&fd->mu); } -/* Release the reference to pollset->polling_island and set it to NULL. - pollset->mu must be held */ -static void pollset_release_polling_island_locked(grpc_pollset *pollset) { - gpr_mu_lock(&pollset->pi_mu); - if (pollset->polling_island) { - pollset->polling_island = - polling_island_update_and_lock(pollset->polling_island, 1, 0); - polling_island_unref_and_unlock(pollset->polling_island, 1); - pollset->polling_island = NULL; +static void pollset_release_polling_island(grpc_pollset *ps, char *reason) { + gpr_mu_lock(&ps->pi_mu); + if (ps->polling_island != NULL) { + PI_UNREF(ps->polling_island, reason); } - gpr_mu_unlock(&pollset->pi_mu); + ps->polling_island = NULL; + gpr_mu_unlock(&ps->pi_mu); } static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, @@ -1069,8 +1134,9 @@ static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, GPR_ASSERT(!pollset_has_workers(pollset)); pollset->finish_shutdown_called = true; - pollset_release_polling_island_locked(pollset); + /* Release the ref and set pollset->polling_island to NULL */ + pollset_release_polling_island(pollset, "ps_shutdown"); grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); } @@ -1110,7 +1176,7 @@ static void pollset_reset(grpc_pollset *pollset) { pollset->finish_shutdown_called = false; pollset->kicked_without_pollers = false; pollset->shutdown_done = NULL; - pollset_release_polling_island_locked(pollset); + pollset_release_polling_island(pollset, "ps_reset"); } #define GRPC_EPOLL_MAX_EVENTS 1000 @@ -1124,28 +1190,37 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the - polling island pointed by pollset->polling_island. + latest polling island pointed by pollset->polling_island. Acquire the following locks: - pollset->mu (which we already have) - pollset->pi_mu - - pollset->polling_island->mu (call polling_island_update_and_lock())*/ + - pollset->polling_island lock */ gpr_mu_lock(&pollset->pi_mu); - pi = pollset->polling_island; - if (pi == NULL) { - pi = polling_island_create(NULL, 1); + if (pollset->polling_island == NULL) { + pollset->polling_island = polling_island_create(NULL); + PI_ADD_REF(pollset->polling_island, "ps"); } - /* In addition to locking the polling island, add a ref so that the island - does not get destroyed (which means the epoll_fd won't be closed) while - we are are doing an epoll_wait() on the epoll_fd */ - pi = polling_island_update_and_lock(pi, 1, 1); + pi = polling_island_lock(pollset->polling_island); epoll_fd = pi->epoll_fd; - /* Update the pollset->polling_island */ - pollset->polling_island = pi; + /* Update the pollset->polling_island since the island being pointed by + pollset->polling_island may not be the latest (i.e pi) */ + if (pollset->polling_island != pi) { + /* Always do PI_ADD_REF before PI_UNREF because PI_UNREF may cause the + polling island to be deleted */ + PI_ADD_REF(pi, "ps"); + PI_UNREF(pollset->polling_island, "ps"); + pollset->polling_island = pi; + } + + /* Add an extra ref so that the island does not get destroyed (which means + the epoll_fd won't be closed) while we are are doing an epoll_wait() on the + epoll_fd */ + PI_ADD_REF(pi, "ps_work"); - polling_island_unref_and_unlock(pollset->polling_island, 0); /* Keep the ref*/ + gpr_mu_unlock(&pi->mu); gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); @@ -1193,14 +1268,12 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, GPR_ASSERT(pi != NULL); - /* Before leaving, release the extra ref we added to the polling island */ - /* It is important to note that at this point 'pi' may not be the same as - * pollset->polling_island. This is because pollset->polling_island pointer - * gets updated whenever the underlying polling island is merged with another - * island and while we are doing epoll_wait() above, the polling island may - * have been merged */ - pi = polling_island_update_and_lock(pi, 1, 0); /* No new ref added */ - polling_island_unref_and_unlock(pi, 1); + /* Before leaving, release the extra ref we added to the polling island. It + is important to use "pi" here (i.e our old copy of pollset->polling_island + that we got before releasing the polling island lock). This is because + pollset->polling_island pointer might get udpated in other parts of the + code when there is an island merge while we are doing epoll_wait() above */ + PI_UNREF(pi, "ps_work"); GPR_TIMER_END("pollset_work_and_unlock", 0); } @@ -1297,20 +1370,34 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, if (fd->polling_island == pollset->polling_island) { pi_new = fd->polling_island; if (pi_new == NULL) { - pi_new = polling_island_create(fd, 2); + pi_new = polling_island_create(fd); } } else if (fd->polling_island == NULL) { - pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); - polling_island_add_fds_locked(pollset->polling_island, &fd, 1, true); + pi_new = polling_island_lock(pollset->polling_island); + polling_island_add_fds_locked(pi_new, &fd, 1, true); gpr_mu_unlock(&pi_new->mu); } else if (pollset->polling_island == NULL) { - pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); + pi_new = polling_island_lock(fd->polling_island); gpr_mu_unlock(&pi_new->mu); } else { pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); } - fd->polling_island = pollset->polling_island = pi_new; + if (fd->polling_island != pi_new) { + PI_ADD_REF(pi_new, "fd"); + if (fd->polling_island != NULL) { + PI_UNREF(fd->polling_island, "fd"); + } + fd->polling_island = pi_new; + } + + if (pollset->polling_island != pi_new) { + PI_ADD_REF(pi_new, "ps"); + if (pollset->polling_island != NULL) { + PI_UNREF(pollset->polling_island, "ps"); + } + pollset->polling_island = pi_new; + } gpr_mu_unlock(&fd->pi_mu); gpr_mu_unlock(&pollset->pi_mu); @@ -1481,28 +1568,19 @@ void *grpc_pollset_get_polling_island(grpc_pollset *ps) { return pi; } -static polling_island *get_polling_island(polling_island *p) { - if (p == NULL) { - return NULL; - } +bool grpc_are_polling_islands_equal(void *p, void *q) { + polling_island *p1 = p; + polling_island *p2 = q; - polling_island *next; - gpr_mu_lock(&p->mu); - while (p->merged_to != NULL) { - next = p->merged_to; - gpr_mu_unlock(&p->mu); - p = next; - gpr_mu_lock(&p->mu); + polling_island_lock_pair(&p1, &p2); + if (p1 == p2) { + gpr_mu_unlock(&p1->mu); + } else { + gpr_mu_unlock(&p1->mu); + gpr_mu_unlock(&p2->mu); } - gpr_mu_unlock(&p->mu); - - return p; -} -bool grpc_are_polling_islands_equal(void *p, void *q) { - p = get_polling_island(p); - q = get_polling_island(q); - return p == q; + return p1 == p2; } /******************************************************************************* From 65c6c59bcddd2847eb26eb7518747ebeea839d0b Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 21 Jun 2016 08:27:07 -0700 Subject: [PATCH 082/215] Fix refcounting tsan failures and grab pollset lock in the function pollset_add_fd --- src/core/lib/iomgr/ev_epoll_linux.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 72288889c02..7cc69c876db 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -291,11 +291,11 @@ void pi_unref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, #endif long pi_add_ref(polling_island *pi, int ref_cnt) { - return gpr_atm_no_barrier_fetch_add(&pi->ref_count, ref_cnt); + return gpr_atm_full_fetch_add(&pi->ref_count, ref_cnt); } long pi_unref(polling_island *pi, int ref_cnt) { - long old_cnt = gpr_atm_no_barrier_fetch_add(&pi->ref_count, -ref_cnt); + long old_cnt = gpr_atm_full_fetch_add(&pi->ref_count, -ref_cnt); /* If ref count went to zero, delete the polling island. Note that this need not be done under a lock. Once the ref count goes to zero, we are @@ -311,6 +311,8 @@ long pi_unref(polling_island *pi, int ref_cnt) { if (next != NULL) { PI_UNREF(next, "pi_delete"); /* Recursive call */ } + } else { + GPR_ASSERT(old_cnt > ref_cnt); } return old_cnt; @@ -445,8 +447,8 @@ static polling_island *polling_island_create(grpc_fd *initial_fd) { pi->fds = NULL; } - gpr_atm_no_barrier_store(&pi->ref_count, 0); - gpr_atm_no_barrier_store(&pi->merged_to, NULL); + gpr_atm_rel_store(&pi->ref_count, 0); + gpr_atm_rel_store(&pi->merged_to, NULL); pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); @@ -1347,7 +1349,7 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { - /* TODO sreek - Double check if we need to get a pollset->mu lock here */ + gpr_mu_lock(&pollset->mu); gpr_mu_lock(&pollset->pi_mu); gpr_mu_lock(&fd->pi_mu); @@ -1401,6 +1403,7 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, gpr_mu_unlock(&fd->pi_mu); gpr_mu_unlock(&pollset->pi_mu); + gpr_mu_unlock(&pollset->mu); } /******************************************************************************* From d263b925e8b8ce36042f34eda34715696b6eef3e Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Tue, 21 Jun 2016 09:15:57 -0700 Subject: [PATCH 083/215] Make sure to poll cq --- test/cpp/end2end/server_builder_plugin_test.cc | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/test/cpp/end2end/server_builder_plugin_test.cc b/test/cpp/end2end/server_builder_plugin_test.cc index 75f23b64a73..7d0b467d816 100644 --- a/test/cpp/end2end/server_builder_plugin_test.cc +++ b/test/cpp/end2end/server_builder_plugin_test.cc @@ -37,6 +37,7 @@ #include #include #include +#include #include #include #include @@ -188,6 +189,7 @@ class ServerBuilderPluginTest : public ::testing::TestWithParam { grpc::string server_address = "localhost:" + to_string(port_); builder_->AddListeningPort(server_address, InsecureServerCredentials()); cq_ = builder_->AddCompletionQueue(); + cq_thread_ = grpc::thread(std::bind(&ServerBuilderPluginTest::RunCQ, this)); server_ = builder_->BuildAndStart(); EXPECT_TRUE(CheckPresent()); } @@ -204,11 +206,8 @@ class ServerBuilderPluginTest : public ::testing::TestWithParam { EXPECT_TRUE(plugin->init_server_is_called()); EXPECT_TRUE(plugin->finish_is_called()); server_->Shutdown(); - void* tag; - bool ok; cq_->Shutdown(); - while (cq_->Next(&tag, &ok)) - ; + cq_thread_.join(); } string to_string(const int number) { @@ -223,6 +222,7 @@ class ServerBuilderPluginTest : public ::testing::TestWithParam { std::unique_ptr stub_; std::unique_ptr cq_; std::unique_ptr server_; + grpc::thread cq_thread_; TestServiceImpl service_; int port_; @@ -238,6 +238,13 @@ class ServerBuilderPluginTest : public ::testing::TestWithParam { return nullptr; } } + + void RunCQ() { + void* tag; + bool ok; + while (cq_->Next(&tag, &ok)) + ; + } }; TEST_P(ServerBuilderPluginTest, PluginWithoutServiceTest) { From 04a468122f0072cbd8a67d66dfd8243ce3ddface Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Tue, 21 Jun 2016 09:16:57 -0700 Subject: [PATCH 084/215] Add comment --- test/cpp/end2end/server_builder_plugin_test.cc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/test/cpp/end2end/server_builder_plugin_test.cc b/test/cpp/end2end/server_builder_plugin_test.cc index 7d0b467d816..778a2be573e 100644 --- a/test/cpp/end2end/server_builder_plugin_test.cc +++ b/test/cpp/end2end/server_builder_plugin_test.cc @@ -188,6 +188,8 @@ class ServerBuilderPluginTest : public ::testing::TestWithParam { void StartServer() { grpc::string server_address = "localhost:" + to_string(port_); builder_->AddListeningPort(server_address, InsecureServerCredentials()); + // we run some tests without a service, and for those we need to supply a + // frequently polled completion queue cq_ = builder_->AddCompletionQueue(); cq_thread_ = grpc::thread(std::bind(&ServerBuilderPluginTest::RunCQ, this)); server_ = builder_->BuildAndStart(); From 9f26a149a5da48364ec62dd607e34104bb851c6d Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 21 Jun 2016 10:15:01 -0700 Subject: [PATCH 085/215] fixed incomplete sentence --- doc/compression.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/compression.md b/doc/compression.md index 6e455636021..a116ac50a16 100644 --- a/doc/compression.md +++ b/doc/compression.md @@ -81,8 +81,8 @@ initial call. A client doesn't a priori (presently) know which algorithms a server supports. This issue can be addressed with an initial negotiation of capabilities or an automatic retry mechanism. These features will be implemented in the future. Currently however, compression levels are only supported at the -server side, which is aware of the client's capabilities by virtue of the -incoming. +server side, which is aware of the client's capabilities through the incoming +Message-Accept-Encoding header. ### Propagation to child RPCs From d5fd7ddc70e658a0364bd2e43c8a486a25db267c Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Tue, 21 Jun 2016 11:13:23 -0700 Subject: [PATCH 086/215] Addressed review comments Removed the silencing for incompatible-pointer-types Removed unused objects Fixed format issues --- src/objective-c/ProtoRPC/ProtoService.m | 9 +++------ src/objective-c/tests/GRPCClientTests.m | 24 ++++++++++++------------ src/objective-c/tests/Podfile | 3 ++- 3 files changed, 17 insertions(+), 19 deletions(-) diff --git a/src/objective-c/ProtoRPC/ProtoService.m b/src/objective-c/ProtoRPC/ProtoService.m index cd9bc7aeac4..97401908519 100644 --- a/src/objective-c/ProtoRPC/ProtoService.m +++ b/src/objective-c/ProtoRPC/ProtoService.m @@ -65,22 +65,19 @@ return self; } -#pragma clang diagnostic push -#pragma clang diagnostic ignored "-Wincompatible-pointer-types" - (ProtoRPC *)RPCToMethod:(NSString *)method requestsWriter:(GRXWriter *)requestsWriter responseClass:(Class)responseClass responsesWriteable:(id)responsesWriteable { - ProtoMethod *methodName = [[ProtoMethod alloc] initWithPackage:_packageName - service:_serviceName - method:method]; + GRPCProtoMethod *methodName = [[GRPCProtoMethod alloc] initWithPackage:_packageName + service:_serviceName + method:method]; return [[ProtoRPC alloc] initWithHost:_host method:methodName requestsWriter:requestsWriter responseClass:responseClass responsesWriteable:responsesWriteable]; } -#pragma clang diagnostic pop @end @implementation GRPCProtoService diff --git a/src/objective-c/tests/GRPCClientTests.m b/src/objective-c/tests/GRPCClientTests.m index 2eca7bf5498..1167a715bb9 100644 --- a/src/objective-c/tests/GRPCClientTests.m +++ b/src/objective-c/tests/GRPCClientTests.m @@ -110,14 +110,14 @@ static GRPCProtoMethod *kUnaryCallMethod; // This method isn't implemented by the remote server. kInexistentMethod = [[GRPCProtoMethod alloc] initWithPackage:kPackage - service:kService - method:@"Inexistent"]; + service:kService + method:@"Inexistent"]; kEmptyCallMethod = [[GRPCProtoMethod alloc] initWithPackage:kPackage - service:kService - method:@"EmptyCall"]; + service:kService + method:@"EmptyCall"]; kUnaryCallMethod = [[GRPCProtoMethod alloc] initWithPackage:kPackage - service:kService - method:@"UnaryCall"]; + service:kService + method:@"UnaryCall"]; } - (void)testConnectionToRemoteServer { @@ -303,9 +303,9 @@ static GRPCProtoMethod *kUnaryCallMethod; // Try to set parameters to nil for GRPCCall. This should cause an exception @try { - GRPCCall *call __unused = [[GRPCCall alloc] initWithHost:nil - path:nil - requestsWriter:nil]; + (void)[[GRPCCall alloc] initWithHost:nil + path:nil + requestsWriter:nil]; XCTFail(@"Did not receive an exception when parameters are nil"); } @catch(NSException *theException) { NSLog(@"Received exception as expected: %@", theException.name); @@ -316,9 +316,9 @@ static GRPCProtoMethod *kUnaryCallMethod; GRXWriter *requestsWriter = [GRXWriter emptyWriter]; [requestsWriter finishWithError:nil]; @try { - GRPCCall *call __unused = [[GRPCCall alloc] initWithHost:kHostAddress - path:kUnaryCallMethod.HTTPPath - requestsWriter:requestsWriter]; + (void)[[GRPCCall alloc] initWithHost:kHostAddress + path:kUnaryCallMethod.HTTPPath + requestsWriter:requestsWriter]; XCTFail(@"Did not receive an exception when GRXWriter has incorrect state."); } @catch(NSException *theException) { NSLog(@"Received exception as expected: %@", theException.name); diff --git a/src/objective-c/tests/Podfile b/src/objective-c/tests/Podfile index d51b18cc34d..6d5f94cbda1 100644 --- a/src/objective-c/tests/Podfile +++ b/src/objective-c/tests/Podfile @@ -6,7 +6,7 @@ install! 'cocoapods', :deterministic_uuids => false def shared_pods pod 'Protobuf', :path => "../../../third_party/protobuf", :inhibit_warnings => true pod 'BoringSSL', :podspec => "..", :inhibit_warnings => true - pod 'CronetFramework', :podspec => "..", :inhibit_warnings => true + pod 'CronetFramework', :podspec => ".." pod 'gRPC', :path => "../../.." pod 'RemoteTest', :path => "RemoteTestClient" end @@ -42,6 +42,7 @@ post_install do |installer| end if target.name == 'gRPC' target.build_configurations.each do |config| + # TODO(zyc) Remove this setting after the issue is resolved # GPR_UNREACHABLE_CODE causes "Control may reach end of non-void # function" warning config.build_settings['GCC_WARN_ABOUT_RETURN_TYPE'] = 'NO' From 8d9e83806d24ad5e1a47bb705d88816bfc8b4864 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Tue, 21 Jun 2016 11:21:48 -0700 Subject: [PATCH 087/215] Fixed format issues --- src/objective-c/tests/InteropTests.m | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/objective-c/tests/InteropTests.m b/src/objective-c/tests/InteropTests.m index 60a83259fa6..15ce120c551 100644 --- a/src/objective-c/tests/InteropTests.m +++ b/src/objective-c/tests/InteropTests.m @@ -285,8 +285,8 @@ static cronet_engine *cronetEngine = NULL; GRXBufferedPipe *requestsBuffer = [[GRXBufferedPipe alloc] init]; GRPCProtoCall *call = [_service RPCToStreamingInputCallWithRequestsWriter:requestsBuffer - handler:^(RMTStreamingInputCallResponse *response, - NSError *error) { + handler:^(RMTStreamingInputCallResponse *response, + NSError *error) { XCTAssertEqual(error.code, GRPC_STATUS_CANCELLED); [expectation fulfill]; }]; From b665dd2722371e5c4a28bbf6cb5f791f8ab5cca9 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 21 Jun 2016 11:52:29 -0700 Subject: [PATCH 088/215] reworded some --- doc/compression.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/compression.md b/doc/compression.md index a116ac50a16..15fae4d29bf 100644 --- a/doc/compression.md +++ b/doc/compression.md @@ -42,13 +42,13 @@ and RPC settings (for example, if compression would result in small or negative gains). When a message from a client compressed with an unsupported algorithm is -processed by a server, it WILL result in an INVALID\_ARGUMENT error. The server -will include in its response a `grpc-accept-encoding` header specifying the -algorithms it does accept. If an INTERNAL error is returned from the server -despite having used one of the algorithms from the `grpc-accept-encoding` -header, the cause MUST NOT be related to compression. Data sent from a server -compressed with an algorithm not supported by the client WILL result in an -INTERNAL error on the client side. +processed by a server, it WILL result in an INVALID\_ARGUMENT error on the +server. The server will then include in its response a `grpc-accept-encoding` +header specifying the algorithms it does accept. If an INTERNAL error is +returned from the server despite having used one of the algorithms from the +`grpc-accept-encoding` header, the cause MUST NOT be related to compression. +Data sent from a server compressed with an algorithm not supported by the client +WILL result in an INTERNAL error on the client side. Note that a peer MAY choose to not disclose all the encodings it supports. However, if it receives a message compressed in an undisclosed but supported From 3d399cb102510e46ca252213ad2799b40704d33d Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Tue, 21 Jun 2016 11:57:46 -0700 Subject: [PATCH 089/215] Fix format issues --- src/objective-c/tests/InteropTests.m | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/objective-c/tests/InteropTests.m b/src/objective-c/tests/InteropTests.m index 15ce120c551..3102cf597a0 100644 --- a/src/objective-c/tests/InteropTests.m +++ b/src/objective-c/tests/InteropTests.m @@ -284,9 +284,10 @@ static cronet_engine *cronetEngine = NULL; // A buffered pipe to which we never write any value acts as a writer that just hangs. GRXBufferedPipe *requestsBuffer = [[GRXBufferedPipe alloc] init]; - GRPCProtoCall *call = [_service RPCToStreamingInputCallWithRequestsWriter:requestsBuffer - handler:^(RMTStreamingInputCallResponse *response, - NSError *error) { + GRPCProtoCall *call = + [_service RPCToStreamingInputCallWithRequestsWriter:requestsBuffer + handler:^(RMTStreamingInputCallResponse *response, + NSError *error) { XCTAssertEqual(error.code, GRPC_STATUS_CANCELLED); [expectation fulfill]; }]; From ef01edf5b7c6eb008c7b0581354678aeeabd7680 Mon Sep 17 00:00:00 2001 From: vjpai Date: Tue, 21 Jun 2016 16:42:08 -0700 Subject: [PATCH 090/215] Fix review comments --- include/grpc++/impl/codegen/async_stream.h | 12 ++++++------ include/grpc++/impl/codegen/completion_queue.h | 11 ++++++----- include/grpc++/impl/codegen/sync_stream.h | 12 ++++++------ 3 files changed, 18 insertions(+), 17 deletions(-) diff --git a/include/grpc++/impl/codegen/async_stream.h b/include/grpc++/impl/codegen/async_stream.h index c74737ce5f8..cbc5d944295 100644 --- a/include/grpc++/impl/codegen/async_stream.h +++ b/include/grpc++/impl/codegen/async_stream.h @@ -53,7 +53,7 @@ class ClientAsyncStreamingInterface { /// Request notification of the reading of the initial metadata. Completion /// will be notified by \a tag on the associated completion queue. /// This call is optional, but if it is used, it cannot be used concurrently - /// with Read + /// with or after the \a Read method. /// /// \param[in] tag Tag identifying this request. virtual void ReadInitialMetadata(void* tag) = 0; @@ -74,8 +74,8 @@ class AsyncReaderInterface { /// Read a message of type \a R into \a msg. Completion will be notified by \a /// tag on the associated completion queue. - /// This is thread-safe with respect to other streaming APIs except for Finish - /// on the same stream. + /// This is thread-safe with respect to other streaming APIs except for \a + /// Finish on the same stream. /// /// \param[out] msg Where to eventually store the read message. /// \param[in] tag The tag identifying the operation. @@ -93,7 +93,7 @@ class AsyncWriterInterface { /// Only one write may be outstanding at any given time. This means that /// after calling Write, one must wait to receive \a tag from the completion /// queue BEFORE calling Write again. - /// This is thread-safe with respect to Read + /// This is thread-safe with respect to \a Read /// /// \param[in] msg The message to be written. /// \param[in] tag The tag identifying the operation. @@ -164,7 +164,7 @@ class ClientAsyncWriterInterface : public ClientAsyncStreamingInterface, public AsyncWriterInterface { public: /// Signal the client is done with the writes. - /// Thread-safe with respect to Read + /// Thread-safe with respect to \a Read /// /// \param[in] tag The tag identifying the operation. virtual void WritesDone(void* tag) = 0; @@ -236,7 +236,7 @@ class ClientAsyncReaderWriterInterface : public ClientAsyncStreamingInterface, public AsyncReaderInterface { public: /// Signal the client is done with the writes. - /// Thread-safe with respect to Read + /// Thread-safe with respect to \a Read /// /// \param[in] tag The tag identifying the operation. virtual void WritesDone(void* tag) = 0; diff --git a/include/grpc++/impl/codegen/completion_queue.h b/include/grpc++/impl/codegen/completion_queue.h index f138ebe7de2..03009e0561d 100644 --- a/include/grpc++/impl/codegen/completion_queue.h +++ b/include/grpc++/impl/codegen/completion_queue.h @@ -34,15 +34,16 @@ /// A completion queue implements a concurrent producer-consumer queue, with /// two main API-exposed methods: \a Next and \a AsyncNext. These /// methods are the essential component of the gRPC C++ asynchronous API. -/// There is also a Shutdown method to indicate that a given completion queue +/// There is also a \a Shutdown method to indicate that a given completion queue /// will no longer have regular events. This must be called before the /// completion queue is destroyed. /// All completion queue APIs are thread-safe and may be used concurrently with /// any other completion queue API invocation; it is acceptable to have -/// multiple threads calling Next or AsyncNext on the same or different -/// completion queues, or to call these methods concurrently with a Shutdown -/// elsewhere. All of these should be completed, though, before a completion -/// queue destructor is called. +/// multiple threads calling \a Next or \a AsyncNext on the same or different +/// completion queues, or to call these methods concurrently with a \a Shutdown +/// elsewhere. +/// \remark{All other API calls on completion queue should be completed before +/// a completion queue destructor is called.} #ifndef GRPCXX_IMPL_CODEGEN_COMPLETION_QUEUE_H #define GRPCXX_IMPL_CODEGEN_COMPLETION_QUEUE_H diff --git a/include/grpc++/impl/codegen/sync_stream.h b/include/grpc++/impl/codegen/sync_stream.h index 9d7966ba04b..e53717cabfc 100644 --- a/include/grpc++/impl/codegen/sync_stream.h +++ b/include/grpc++/impl/codegen/sync_stream.h @@ -71,8 +71,8 @@ class ReaderInterface { virtual ~ReaderInterface() {} /// Blocking read a message and parse to \a msg. Returns \a true on success. - /// This is thread-safe with respect to other streaming APIs except for Finish - /// on the same stream. (Finish must be called as described above.) + /// This is thread-safe with respect to other streaming APIs except for \a + /// Finish on the same stream. (\a Finish must be called as described above.) /// /// \param[out] msg The read message. /// @@ -89,7 +89,7 @@ class WriterInterface { virtual ~WriterInterface() {} /// Blocking write \a msg to the stream with options. - /// This is thread-safe with respect to Read + /// This is thread-safe with respect to \a Read /// /// \param msg The message to be written to the stream. /// \param options Options affecting the write operation. @@ -98,7 +98,7 @@ class WriterInterface { virtual bool Write(const W& msg, const WriteOptions& options) = 0; /// Blocking write \a msg to the stream with default options. - /// This is thread-safe with respect to Read + /// This is thread-safe with respect to \a Read /// /// \param msg The message to be written to the stream. /// @@ -179,7 +179,7 @@ class ClientWriterInterface : public ClientStreamingInterface, public: /// Half close writing from the client. /// Block until currently-pending writes are completed. - /// Thread safe with respect to Read operations only + /// Thread safe with respect to \a Read operations only /// /// \return Whether the writes were successful. virtual bool WritesDone() = 0; @@ -263,7 +263,7 @@ class ClientReaderWriterInterface : public ClientStreamingInterface, virtual void WaitForInitialMetadata() = 0; /// Block until currently-pending writes are completed. - /// Thread-safe with respect to Read + /// Thread-safe with respect to \a Read /// /// \return Whether the writes were successful. virtual bool WritesDone() = 0; From 3131c269c14f97294ebf8b6e3d1a235d4acf3317 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 21 Jun 2016 17:28:28 -0700 Subject: [PATCH 091/215] Integrate with unified error reporting --- src/core/lib/iomgr/ev_epoll_linux.c | 116 +++++++++++++++++++------- test/core/iomgr/ev_epoll_linux_test.c | 3 +- 2 files changed, 90 insertions(+), 29 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 7cc69c876db..d625b096a10 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -646,11 +646,18 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { return q; } -static void polling_island_global_init() { +static grpc_error *polling_island_global_init() { + grpc_error *error = GRPC_ERROR_NONE; + gpr_mu_init(&g_pi_freelist_mu); g_pi_freelist = NULL; - grpc_wakeup_fd_init(&polling_island_wakeup_fd); - grpc_wakeup_fd_wakeup(&polling_island_wakeup_fd); + + error = grpc_wakeup_fd_init(&polling_island_wakeup_fd); + if (error == GRPC_ERROR_NONE) { + error = grpc_wakeup_fd_wakeup(&polling_island_wakeup_fd); + } + + return error; } static void polling_island_global_shutdown() { @@ -870,21 +877,33 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, } gpr_mu_unlock(&fd->pi_mu); - grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); + grpc_exec_ctx_sched(exec_ctx, fd->on_done_closure, GRPC_ERROR_NONE, NULL); gpr_mu_unlock(&fd->mu); UNREF_BY(fd, 2, reason); /* Drop the reference */ } +static grpc_error *fd_shutdown_error(bool shutdown) { + if (!shutdown) { + return GRPC_ERROR_NONE; + } else { + return GRPC_ERROR_CREATE("FD shutdown"); + } +} + static void notify_on_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st, grpc_closure *closure) { - if (*st == CLOSURE_NOT_READY) { + if (fd->shutdown) { + grpc_exec_ctx_sched(exec_ctx, closure, GRPC_ERROR_CREATE("FD shutdown"), + NULL); + } else if (*st == CLOSURE_NOT_READY) { /* not ready ==> switch to a waiting state by setting the closure */ *st = closure; } else if (*st == CLOSURE_READY) { /* already ready ==> queue the closure to run immediately */ *st = CLOSURE_NOT_READY; - grpc_exec_ctx_enqueue(exec_ctx, closure, !fd->shutdown, NULL); + grpc_exec_ctx_sched(exec_ctx, closure, fd_shutdown_error(fd->shutdown), + NULL); } else { /* upcallptr was set to a different closure. This is an error! */ gpr_log(GPR_ERROR, @@ -906,7 +925,7 @@ static int set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, return 0; } else { /* waiting ==> queue closure */ - grpc_exec_ctx_enqueue(exec_ctx, *st, !fd->shutdown, NULL); + grpc_exec_ctx_sched(exec_ctx, *st, fd_shutdown_error(fd->shutdown), NULL); *st = CLOSURE_NOT_READY; return 1; } @@ -964,11 +983,11 @@ static void sig_handler(int sig_num) { static void poller_kick_init() { signal(grpc_wakeup_signal, sig_handler); } /* Global state management */ -static void pollset_global_init(void) { - grpc_wakeup_fd_init(&grpc_global_wakeup_fd); +static grpc_error *pollset_global_init(void) { gpr_tls_init(&g_current_thread_pollset); gpr_tls_init(&g_current_thread_worker); poller_kick_init(); + return grpc_wakeup_fd_init(&grpc_global_wakeup_fd); } static void pollset_global_shutdown(void) { @@ -977,8 +996,13 @@ static void pollset_global_shutdown(void) { gpr_tls_destroy(&g_current_thread_worker); } -static void pollset_worker_kick(grpc_pollset_worker *worker) { - pthread_kill(worker->pt_id, grpc_wakeup_signal); +static grpc_error *pollset_worker_kick(grpc_pollset_worker *worker) { + grpc_error *err = GRPC_ERROR_NONE; + int err_num = pthread_kill(worker->pt_id, grpc_wakeup_signal); + if (err_num != 0) { + err = GRPC_OS_ERROR(err_num, "pthread_kill"); + } + return err; } /* Return 1 if the pollset has active threads in pollset_work (pollset must @@ -1014,10 +1038,19 @@ static void push_front_worker(grpc_pollset *p, grpc_pollset_worker *worker) { worker->prev->next = worker->next->prev = worker; } +static void kick_append_error(grpc_error **composite, grpc_error *error) { + if (error == GRPC_ERROR_NONE) return; + if (*composite == GRPC_ERROR_NONE) { + *composite = GRPC_ERROR_CREATE("Kick Failure"); + } + *composite = grpc_error_add_child(*composite, error); +} + /* p->mu must be held before calling this function */ -static void pollset_kick(grpc_pollset *p, - grpc_pollset_worker *specific_worker) { +static grpc_error *pollset_kick(grpc_pollset *p, + grpc_pollset_worker *specific_worker) { GPR_TIMER_BEGIN("pollset_kick", 0); + grpc_error *error = GRPC_ERROR_NONE; grpc_pollset_worker *worker = specific_worker; if (worker != NULL) { @@ -1027,7 +1060,7 @@ static void pollset_kick(grpc_pollset *p, for (worker = p->root_worker.next; worker != &p->root_worker; worker = worker->next) { if (gpr_tls_get(&g_current_thread_worker) != (intptr_t)worker) { - pollset_worker_kick(worker); + kick_append_error(&error, pollset_worker_kick(worker)); } } } else { @@ -1037,7 +1070,7 @@ static void pollset_kick(grpc_pollset *p, } else { GPR_TIMER_MARK("kicked_specifically", 0); if (gpr_tls_get(&g_current_thread_worker) != (intptr_t)worker) { - pollset_worker_kick(worker); + kick_append_error(&error, pollset_worker_kick(worker)); } } } else if (gpr_tls_get(&g_current_thread_pollset) != (intptr_t)p) { @@ -1053,7 +1086,7 @@ static void pollset_kick(grpc_pollset *p, if (worker != NULL) { GPR_TIMER_MARK("finally_kick", 0); push_back_worker(p, worker); - pollset_worker_kick(worker); + kick_append_error(&error, pollset_worker_kick(worker)); } else { GPR_TIMER_MARK("kicked_no_pollers", 0); p->kicked_without_pollers = true; @@ -1061,9 +1094,13 @@ static void pollset_kick(grpc_pollset *p, } GPR_TIMER_END("pollset_kick", 0); + GRPC_LOG_IF_ERROR("pollset_kick", GRPC_ERROR_REF(error)); + return error; } -static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } +static grpc_error *kick_poller(void) { + return grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); +} static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { gpr_mu_init(&pollset->mu); @@ -1139,7 +1176,7 @@ static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, /* Release the ref and set pollset->polling_island to NULL */ pollset_release_polling_island(pollset, "ps_shutdown"); - grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); + grpc_exec_ctx_sched(exec_ctx, pollset->shutdown_done, GRPC_ERROR_NONE, NULL); } /* pollset->mu lock must be held by the caller before calling this */ @@ -1181,14 +1218,23 @@ static void pollset_reset(grpc_pollset *pollset) { pollset_release_polling_island(pollset, "ps_reset"); } +static void work_combine_error(grpc_error **composite, grpc_error *error) { + if (error == GRPC_ERROR_NONE) return; + if (*composite == GRPC_ERROR_NONE) { + *composite = GRPC_ERROR_CREATE("pollset_work"); + } + *composite = grpc_error_add_child(*composite, error); +} + #define GRPC_EPOLL_MAX_EVENTS 1000 -static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, int timeout_ms, - sigset_t *sig_mask) { +static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, + int timeout_ms, sigset_t *sig_mask) { struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; int epoll_fd = -1; int ep_rv; polling_island *pi = NULL; + grpc_error *error = GRPC_ERROR_NONE; GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the @@ -1232,6 +1278,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, if (ep_rv < 0) { if (errno != EINTR) { gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); + work_combine_error(&error, GRPC_OS_ERROR(errno, "epoll_pwait")); } else { /* We were interrupted. Save an interation by doing a zero timeout epoll_wait to see if there are any other events of interest */ @@ -1247,7 +1294,8 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, for (int i = 0; i < ep_rv; ++i) { void *data_ptr = ep_ev[i].data.ptr; if (data_ptr == &grpc_global_wakeup_fd) { - grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + work_combine_error( + &error, grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd)); } else if (data_ptr == &polling_island_wakeup_fd) { /* This means that our polling island is merged with a different island. We do not have to do anything here since the subsequent call @@ -1278,16 +1326,18 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, PI_UNREF(pi, "ps_work"); GPR_TIMER_END("pollset_work_and_unlock", 0); + return error; } /* pollset->mu lock must be held by the caller before calling this. The function pollset_work() may temporarily release the lock (pollset->mu) during the course of its execution but it will always re-acquire the lock and ensure that it is held by the time the function returns */ -static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_pollset_worker **worker_hdl, gpr_timespec now, - gpr_timespec deadline) { +static grpc_error *pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_pollset_worker **worker_hdl, + gpr_timespec now, gpr_timespec deadline) { GPR_TIMER_BEGIN("pollset_work", 0); + grpc_error *error = GRPC_ERROR_NONE; int timeout_ms = poll_deadline_to_millis_timeout(deadline, now); sigset_t new_mask; @@ -1316,7 +1366,7 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, push_front_worker(pollset, &worker); - pollset_work_and_unlock(exec_ctx, pollset, timeout_ms, &orig_mask); + error = pollset_work_and_unlock(exec_ctx, pollset, timeout_ms, &orig_mask); grpc_exec_ctx_flush(exec_ctx); gpr_mu_lock(&pollset->mu); @@ -1345,6 +1395,8 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, gpr_tls_set(&g_current_thread_pollset, (intptr_t)0); gpr_tls_set(&g_current_thread_worker, (intptr_t)0); GPR_TIMER_END("pollset_work", 0); + GRPC_LOG_IF_ERROR("pollset_work", GRPC_ERROR_REF(error)); + return error; } static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, @@ -1659,8 +1711,16 @@ const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { } fd_global_init(); - pollset_global_init(); - polling_island_global_init(); + + if (!GRPC_LOG_IF_ERROR("pollset_global_init", pollset_global_init())) { + return NULL; + } + + if (!GRPC_LOG_IF_ERROR("polling_island_global_init", + polling_island_global_init())) { + return NULL; + } + return &vtable; } diff --git a/test/core/iomgr/ev_epoll_linux_test.c b/test/core/iomgr/ev_epoll_linux_test.c index 51da15faa7a..034f17fd58b 100644 --- a/test/core/iomgr/ev_epoll_linux_test.c +++ b/test/core/iomgr/ev_epoll_linux_test.c @@ -88,7 +88,8 @@ static void test_pollset_init(test_pollset *pollsets, int num_pollsets) { } } -static void destroy_pollset(grpc_exec_ctx *exec_ctx, void *p, bool success) { +static void destroy_pollset(grpc_exec_ctx *exec_ctx, void *p, + grpc_error *error) { grpc_pollset_destroy(p); } From 0100b2f1c0b08800ba0f7f53fe9cb5fbec7881a7 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 21 Jun 2016 17:38:13 -0700 Subject: [PATCH 092/215] Make fd_shutdown idempotent --- src/core/lib/iomgr/ev_epoll_linux.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index d625b096a10..c077987c01b 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -942,15 +942,19 @@ static grpc_pollset *fd_get_read_notifier_pollset(grpc_exec_ctx *exec_ctx, return notifier; } +/* Might be called multiple times */ static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { gpr_mu_lock(&fd->mu); - GPR_ASSERT(!fd->shutdown); - fd->shutdown = true; - - /* Flush any pending read and write closures. Since fd->shutdown is 'true' at - this point, the closures would be called with 'success = false' */ - set_ready_locked(exec_ctx, fd, &fd->read_closure); - set_ready_locked(exec_ctx, fd, &fd->write_closure); + /* Do the actual shutdown only once */ + if (!fd->shutdown) { + fd->shutdown = true; + + shutdown(fd->fd, SHUT_RDWR); + /* Flush any pending read and write closures. Since fd->shutdown is 'true' + at this point, the closures would be called with 'success = false' */ + set_ready_locked(exec_ctx, fd, &fd->read_closure); + set_ready_locked(exec_ctx, fd, &fd->write_closure); + } gpr_mu_unlock(&fd->mu); } From 24b6eae1fc71a4f5d18eb2e7c1cbca5b4e54a46f Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 21 Jun 2016 18:01:14 -0700 Subject: [PATCH 093/215] Add missing function fd_is_shutdown --- src/core/lib/iomgr/ev_epoll_linux.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index c077987c01b..3a774a8876f 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -942,6 +942,13 @@ static grpc_pollset *fd_get_read_notifier_pollset(grpc_exec_ctx *exec_ctx, return notifier; } +static bool fd_is_shutdown(grpc_fd *fd) { + gpr_mu_lock(&fd->mu); + const bool r = fd->shutdown; + gpr_mu_unlock(&fd->mu); + return r; +} + /* Might be called multiple times */ static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { gpr_mu_lock(&fd->mu); @@ -1659,6 +1666,7 @@ static const grpc_event_engine_vtable vtable = { .fd_wrapped_fd = fd_wrapped_fd, .fd_orphan = fd_orphan, .fd_shutdown = fd_shutdown, + .fd_is_shutdown = fd_is_shutdown, .fd_notify_on_read = fd_notify_on_read, .fd_notify_on_write = fd_notify_on_write, .fd_get_read_notifier_pollset = fd_get_read_notifier_pollset, From bd28936c8648cde481fa20ca64ba00d9281cb119 Mon Sep 17 00:00:00 2001 From: vjpai Date: Tue, 21 Jun 2016 18:03:37 -0700 Subject: [PATCH 094/215] Properly handle reviewer comment re concurrent Read --- include/grpc++/impl/codegen/async_stream.h | 7 +++++-- include/grpc++/impl/codegen/sync_stream.h | 5 +++-- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/include/grpc++/impl/codegen/async_stream.h b/include/grpc++/impl/codegen/async_stream.h index cbc5d944295..e96d224ddbe 100644 --- a/include/grpc++/impl/codegen/async_stream.h +++ b/include/grpc++/impl/codegen/async_stream.h @@ -74,8 +74,11 @@ class AsyncReaderInterface { /// Read a message of type \a R into \a msg. Completion will be notified by \a /// tag on the associated completion queue. - /// This is thread-safe with respect to other streaming APIs except for \a - /// Finish on the same stream. + /// This is thread-safe with respect to \a Write or \a WritesDone methods. It + /// should not be called concurrently with other streaming APIs + /// on the same stream. It is not meaningful to call it concurrently + /// with another \a Read on the same stream since reads on the same stream + /// are delivered in order. /// /// \param[out] msg Where to eventually store the read message. /// \param[in] tag The tag identifying the operation. diff --git a/include/grpc++/impl/codegen/sync_stream.h b/include/grpc++/impl/codegen/sync_stream.h index e53717cabfc..cbfa4106995 100644 --- a/include/grpc++/impl/codegen/sync_stream.h +++ b/include/grpc++/impl/codegen/sync_stream.h @@ -71,8 +71,9 @@ class ReaderInterface { virtual ~ReaderInterface() {} /// Blocking read a message and parse to \a msg. Returns \a true on success. - /// This is thread-safe with respect to other streaming APIs except for \a - /// Finish on the same stream. (\a Finish must be called as described above.) + /// This is thread-safe with respect to \a Write or \WritesDone methods on + /// the same stream. It should not be called concurrently with another \a + /// Read on the same stream as the order of delivery will not be defined. /// /// \param[out] msg The read message. /// From 229533b1e68c4a4b8a67148f7fe25543584131f6 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 21 Jun 2016 20:42:52 -0700 Subject: [PATCH 095/215] Remove pollset->pi_mu since it is redundant. Also do not get polling island lock in the fast-path --- src/core/lib/iomgr/ev_epoll_linux.c | 82 ++++++++++++++++------------- src/core/lib/iomgr/ev_posix.c | 6 +-- 2 files changed, 46 insertions(+), 42 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 3a774a8876f..6464d3ba348 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -207,12 +207,7 @@ struct grpc_pollset { bool finish_shutdown_called; /* Is the 'finish_shutdown_locked()' called ? */ grpc_closure *shutdown_done; /* Called after after shutdown is complete */ - /* The polling island to which this pollset belongs to and the mutex - protecting the field */ - /* TODO: sreek: This lock might actually be adding more overhead to the - critical path (i.e pollset_work() function). Consider removing this lock - and just using the overall pollset lock */ - gpr_mu pi_mu; + /* The polling island to which this pollset belongs to */ struct polling_island *polling_island; }; @@ -488,31 +483,47 @@ static void polling_island_delete(polling_island *pi) { gpr_mu_unlock(&g_pi_freelist_mu); } +/* Attempts to gets the last polling island in the linked list (liked by the + * 'merged_to' field). Since this does not lock the polling island, there are no + * guarantees that the island returned is the last island */ +static polling_island *polling_island_maybe_get_latest(polling_island *pi) { + polling_island *next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + while (next != NULL) { + pi = next; + next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + } + + return pi; +} + /* Gets the lock on the *latest* polling island i.e the last polling island in - the linked list (linked by 'merged_to' link). Call gpr_mu_unlock on the + the linked list (linked by the 'merged_to' field). Call gpr_mu_unlock on the returned polling island's mu. Usage: To lock/unlock polling island "pi", do the following: polling_island *pi_latest = polling_island_lock(pi); ... ... critical section .. ... - gpr_mu_unlock(&pi_latest->mu); //NOTE: use pi_latest->mu. NOT pi->mu */ -polling_island *polling_island_lock(polling_island *pi) { + gpr_mu_unlock(&pi_latest->mu); // NOTE: use pi_latest->mu. NOT pi->mu */ +static polling_island *polling_island_lock(polling_island *pi) { polling_island *next = NULL; + while (true) { next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); if (next == NULL) { - /* pi is the last node in the linked list. Get the lock and check again - (under the pi->mu lock) that pi is still the last node (because a merge - may have happend after the (next == NULL) check above and before - getting the pi->mu lock. - If pi is the last node, we are done. If not, unlock and continue - traversing the list */ + /* Looks like 'pi' is the last node in the linked list but unless we check + this by holding the pi->mu lock, we cannot be sure (i.e without the + pi->mu lock, we don't prevent island merges). + To be absolutely sure, check once more by holding the pi->mu lock */ gpr_mu_lock(&pi->mu); next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); if (next == NULL) { + /* pi is infact the last node and we have the pi->mu lock. we're done */ break; } + + /* pi->merged_to is not NULL i.e pi isn't the last node anymore. pi->mu + * isn't the lock we are interested in. Continue traversing the list */ gpr_mu_unlock(&pi->mu); } @@ -526,11 +537,11 @@ polling_island *polling_island_lock(polling_island *pi) { This function is needed because calling the following block of code to obtain locks on polling islands (*p and *q) is prone to deadlocks. { - polling_island_lock(*p); - polling_island_lock(*q); + polling_island_lock(*p, true); + polling_island_lock(*q, true); } - Usage/exmaple: + Usage/example: polling_island *p1; polling_island *p2; .. @@ -551,7 +562,7 @@ polling_island *polling_island_lock(polling_island *pi) { } */ -void polling_island_lock_pair(polling_island **p, polling_island **q) { +static void polling_island_lock_pair(polling_island **p, polling_island **q) { polling_island *pi_1 = *p; polling_island *pi_2 = *q; polling_island *next_1 = NULL; @@ -611,7 +622,8 @@ void polling_island_lock_pair(polling_island **p, polling_island **q) { *q = pi_2; } -polling_island *polling_island_merge(polling_island *p, polling_island *q) { +static polling_island *polling_island_merge(polling_island *p, + polling_island *q) { /* Get locks on both the polling islands */ polling_island_lock_pair(&p, &q); @@ -1124,7 +1136,6 @@ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { pollset->finish_shutdown_called = false; pollset->shutdown_done = NULL; - gpr_mu_init(&pollset->pi_mu); pollset->polling_island = NULL; } @@ -1170,12 +1181,10 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { } static void pollset_release_polling_island(grpc_pollset *ps, char *reason) { - gpr_mu_lock(&ps->pi_mu); if (ps->polling_island != NULL) { PI_UNREF(ps->polling_island, reason); } ps->polling_island = NULL; - gpr_mu_unlock(&ps->pi_mu); } static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, @@ -1215,7 +1224,6 @@ static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, * here */ static void pollset_destroy(grpc_pollset *pollset) { GPR_ASSERT(!pollset_has_workers(pollset)); - gpr_mu_destroy(&pollset->pi_mu); gpr_mu_destroy(&pollset->mu); } @@ -1250,22 +1258,25 @@ static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the latest polling island pointed by pollset->polling_island. - Acquire the following locks: - - pollset->mu (which we already have) - - pollset->pi_mu - - pollset->polling_island lock */ - gpr_mu_lock(&pollset->pi_mu); + + Since epoll_fd is immutable, we can read it without obtaining the polling + island lock. There is however a possibility that the polling island (from + which we got the epoll_fd) got merged with another island while we are + in this function. This is still okay because in such a case, we will wakeup + right-away from epoll_wait() and pick up the latest polling_island the next + this function (i.e pollset_work_and_unlock()) is called. + */ if (pollset->polling_island == NULL) { pollset->polling_island = polling_island_create(NULL); PI_ADD_REF(pollset->polling_island, "ps"); } - pi = polling_island_lock(pollset->polling_island); + pi = polling_island_maybe_get_latest(pollset->polling_island); epoll_fd = pi->epoll_fd; /* Update the pollset->polling_island since the island being pointed by - pollset->polling_island may not be the latest (i.e pi) */ + pollset->polling_island maybe older than the one pointed by pi) */ if (pollset->polling_island != pi) { /* Always do PI_ADD_REF before PI_UNREF because PI_UNREF may cause the polling island to be deleted */ @@ -1278,9 +1289,6 @@ static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, the epoll_fd won't be closed) while we are are doing an epoll_wait() on the epoll_fd */ PI_ADD_REF(pi, "ps_work"); - - gpr_mu_unlock(&pi->mu); - gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); do { @@ -1413,7 +1421,6 @@ static grpc_error *pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { gpr_mu_lock(&pollset->mu); - gpr_mu_lock(&pollset->pi_mu); gpr_mu_lock(&fd->pi_mu); polling_island *pi_new = NULL; @@ -1465,7 +1472,6 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } gpr_mu_unlock(&fd->pi_mu); - gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); } @@ -1627,9 +1633,9 @@ void *grpc_fd_get_polling_island(grpc_fd *fd) { void *grpc_pollset_get_polling_island(grpc_pollset *ps) { polling_island *pi; - gpr_mu_lock(&ps->pi_mu); + gpr_mu_lock(&ps->mu); pi = ps->polling_island; - gpr_mu_unlock(&ps->pi_mu); + gpr_mu_unlock(&ps->mu); return pi; } diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index 4cdd13bbdba..a3c1e9db9a0 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -54,7 +54,7 @@ grpc_poll_function_type grpc_poll_function = poll; static const grpc_event_engine_vtable *g_event_engine; -static const char* g_poll_strategy_name = NULL; +static const char *g_poll_strategy_name = NULL; typedef const grpc_event_engine_vtable *(*event_engine_factory_fn)(void); @@ -111,9 +111,7 @@ static void try_engine(const char *engine) { } /* Call this only after calling grpc_event_engine_init() */ -const char *grpc_get_poll_strategy_name() { - return g_poll_strategy_name; -} +const char *grpc_get_poll_strategy_name() { return g_poll_strategy_name; } void grpc_event_engine_init(void) { char *s = gpr_getenv("GRPC_POLL_STRATEGY"); From caca0a15ded56b3732e0f67fe7d1507d9a7abf07 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Tue, 21 Jun 2016 23:07:49 -0700 Subject: [PATCH 096/215] Build objc examples as part of automatic tests --- src/objective-c/tests/build_example_test.sh | 56 +++++++++++++++++++ src/objective-c/tests/build_one_example.sh | 62 +++++++++++++++++++++ tools/run_tests/run_tests.py | 4 +- 3 files changed, 121 insertions(+), 1 deletion(-) create mode 100755 src/objective-c/tests/build_example_test.sh create mode 100755 src/objective-c/tests/build_one_example.sh diff --git a/src/objective-c/tests/build_example_test.sh b/src/objective-c/tests/build_example_test.sh new file mode 100755 index 00000000000..3cc0c552349 --- /dev/null +++ b/src/objective-c/tests/build_example_test.sh @@ -0,0 +1,56 @@ +#!/bin/bash +# Copyright 2015, Google Inc. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +# Don't run this script standalone. Instead, run from the repository root: +# ./tools/run_tests/run_tests.py -l objc + +set -eo pipefail + +cd `dirname $0` + +EXAMPLE_PATH=examples/objective-c/helloworld \ +SCHEME=HelloWorld \ +./build_one_example.sh + +EXAMPLE_PATH=examples/objective-c/route_guide \ +SCHEME=RouteGuideClient \ +./build_one_example.sh + +EXAMPLE_PATH=examples/objective-c/auth_sample \ +SCHEME=AuthSample \ +./build_one_example.sh + +EXAMPLE_PATH=src/objective-c/examples/Sample \ +SCHEME=Sample \ +./build_one_example.sh + +EXAMPLE_PATH=src/objective-c/examples/SwiftSample \ +SCHEME=SwiftSample \ +./build_one_example.sh diff --git a/src/objective-c/tests/build_one_example.sh b/src/objective-c/tests/build_one_example.sh new file mode 100755 index 00000000000..131fbd780df --- /dev/null +++ b/src/objective-c/tests/build_one_example.sh @@ -0,0 +1,62 @@ +#!/bin/bash +# Copyright 2015, Google Inc. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +# Don't run this script standalone. Instead, run from the repository root: +# ./tools/run_tests/run_tests.py -l objc + +set -e + +# Params: +# EXAMPLE_PATH - directory of the example +# SCHEME - scheme of the example, used by xcodebuild + +# CocoaPods requires the terminal to be using UTF-8 encoding. +export LANG=en_US.UTF-8 + +cd `dirname $0`/../../.. + +cd $EXAMPLE_PATH + +# clean the directory +git checkout . +git clean -df . +rm -rf Pods + +pod install + +set -o pipefail +XCODEBUILD_FILTER='(^===|^\*\*|\bfatal\b|\berror\b|\bwarning\b|\bfail)' +xcodebuild \ + clean build \ + -workspace *.xcworkspace \ + -scheme $SCHEME \ + -destination name="iPhone 6" \ + | egrep "$XCODEBUILD_FILTER" \ + | egrep -v "(GPBDictionary|GPBArray)" - diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 9ef12c5b07e..0d77f0dbf97 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -425,7 +425,7 @@ class PythonLanguage(object): return [] def build_steps(self): - return [['tools/run_tests/build_python.sh', tox_env] + return [['tools/run_tests/build_python.sh', tox_env] for tox_env in self._tox_envs] def post_tests_steps(self): @@ -602,6 +602,8 @@ class ObjCLanguage(object): def test_specs(self): return [self.config.job_spec(['src/objective-c/tests/run_tests.sh'], None, + environ=_FORCE_ENVIRON_FOR_WRAPPERS), + self.config.job_spec(['src/objective-c/tests/build_example_test.sh'], None, environ=_FORCE_ENVIRON_FOR_WRAPPERS)] def pre_build_steps(self): From f95a4898abd5888c34b31354cdbe9a909958aa08 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Tue, 21 Jun 2016 23:27:46 -0700 Subject: [PATCH 097/215] Fixed format issues --- src/objective-c/tests/build_example_test.sh | 2 +- src/objective-c/tests/build_one_example.sh | 2 +- tools/run_tests/run_tests.py | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/objective-c/tests/build_example_test.sh b/src/objective-c/tests/build_example_test.sh index 3cc0c552349..73405695e8e 100755 --- a/src/objective-c/tests/build_example_test.sh +++ b/src/objective-c/tests/build_example_test.sh @@ -1,5 +1,5 @@ #!/bin/bash -# Copyright 2015, Google Inc. +# Copyright 2016, Google Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/src/objective-c/tests/build_one_example.sh b/src/objective-c/tests/build_one_example.sh index 131fbd780df..56f686319e1 100755 --- a/src/objective-c/tests/build_one_example.sh +++ b/src/objective-c/tests/build_one_example.sh @@ -1,5 +1,5 @@ #!/bin/bash -# Copyright 2015, Google Inc. +# Copyright 2016, Google Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 0d77f0dbf97..580afc32de4 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -603,8 +603,8 @@ class ObjCLanguage(object): def test_specs(self): return [self.config.job_spec(['src/objective-c/tests/run_tests.sh'], None, environ=_FORCE_ENVIRON_FOR_WRAPPERS), - self.config.job_spec(['src/objective-c/tests/build_example_test.sh'], None, - environ=_FORCE_ENVIRON_FOR_WRAPPERS)] + self.config.job_spec(['src/objective-c/tests/build_example_test.sh'], + None, environ=_FORCE_ENVIRON_FOR_WRAPPERS)] def pre_build_steps(self): return [] From db3ffe13cbb37ea797890c455a4543c9d030adf5 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Wed, 22 Jun 2016 01:12:42 -0700 Subject: [PATCH 098/215] Update examples with Cocoapods 1.0.0 --- .../AuthSample.xcodeproj/project.pbxproj | 28 ++++++++--- .../HelloWorld.xcodeproj/project.pbxproj | 24 ++++----- .../project.pbxproj | 18 +++---- .../Sample/Sample.xcodeproj/project.pbxproj | 50 ++++++++++++------- .../SwiftSample.xcodeproj/project.pbxproj | 42 ++++++++-------- 5 files changed, 95 insertions(+), 67 deletions(-) diff --git a/examples/objective-c/auth_sample/AuthSample.xcodeproj/project.pbxproj b/examples/objective-c/auth_sample/AuthSample.xcodeproj/project.pbxproj index 51a39c578c7..ab7419c9bcd 100644 --- a/examples/objective-c/auth_sample/AuthSample.xcodeproj/project.pbxproj +++ b/examples/objective-c/auth_sample/AuthSample.xcodeproj/project.pbxproj @@ -116,11 +116,12 @@ isa = PBXNativeTarget; buildConfigurationList = 63E1E9A21B28CB2100EF0978 /* Build configuration list for PBXNativeTarget "AuthSample" */; buildPhases = ( - DAABBA7B5788A39108D7CA83 /* Check Pods Manifest.lock */, + DAABBA7B5788A39108D7CA83 /* [CP] Check Pods Manifest.lock */, 63E1E9781B28CB2000EF0978 /* Sources */, 63E1E9791B28CB2000EF0978 /* Frameworks */, 63E1E97A1B28CB2000EF0978 /* Resources */, - AEFCCC69DD59CE8F6EB769D7 /* Copy Pods Resources */, + AEFCCC69DD59CE8F6EB769D7 /* [CP] Copy Pods Resources */, + D24F6598302C412D4B863D6F /* [CP] Embed Pods Frameworks */, ); buildRules = ( ); @@ -177,14 +178,14 @@ /* End PBXResourcesBuildPhase section */ /* Begin PBXShellScriptBuildPhase section */ - AEFCCC69DD59CE8F6EB769D7 /* Copy Pods Resources */ = { + AEFCCC69DD59CE8F6EB769D7 /* [CP] Copy Pods Resources */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Copy Pods Resources"; + name = "[CP] Copy Pods Resources"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; @@ -192,14 +193,29 @@ shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-AuthSample/Pods-AuthSample-resources.sh\"\n"; showEnvVarsInLog = 0; }; - DAABBA7B5788A39108D7CA83 /* Check Pods Manifest.lock */ = { + D24F6598302C412D4B863D6F /* [CP] Embed Pods Frameworks */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Check Pods Manifest.lock"; + name = "[CP] Embed Pods Frameworks"; + outputPaths = ( + ); + runOnlyForDeploymentPostprocessing = 0; + shellPath = /bin/sh; + shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-AuthSample/Pods-AuthSample-frameworks.sh\"\n"; + showEnvVarsInLog = 0; + }; + DAABBA7B5788A39108D7CA83 /* [CP] Check Pods Manifest.lock */ = { + isa = PBXShellScriptBuildPhase; + buildActionMask = 2147483647; + files = ( + ); + inputPaths = ( + ); + name = "[CP] Check Pods Manifest.lock"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; diff --git a/examples/objective-c/helloworld/HelloWorld.xcodeproj/project.pbxproj b/examples/objective-c/helloworld/HelloWorld.xcodeproj/project.pbxproj index 250f009996f..df5c40cda20 100644 --- a/examples/objective-c/helloworld/HelloWorld.xcodeproj/project.pbxproj +++ b/examples/objective-c/helloworld/HelloWorld.xcodeproj/project.pbxproj @@ -7,7 +7,6 @@ objects = { /* Begin PBXBuildFile section */ - 3EF35C14BDC2B65E21837F02 /* libPods.a in Frameworks */ = {isa = PBXBuildFile; fileRef = 43AB08B32839A6700EA00DD4 /* libPods.a */; }; 5E3690661B2A23800040F884 /* main.m in Sources */ = {isa = PBXBuildFile; fileRef = 5E3690651B2A23800040F884 /* main.m */; }; 5E3690691B2A23800040F884 /* AppDelegate.m in Sources */ = {isa = PBXBuildFile; fileRef = 5E3690681B2A23800040F884 /* AppDelegate.m */; }; 5E36906C1B2A23800040F884 /* ViewController.m in Sources */ = {isa = PBXBuildFile; fileRef = 5E36906B1B2A23800040F884 /* ViewController.m */; }; @@ -18,7 +17,6 @@ /* Begin PBXFileReference section */ 0C432EF610DB15C0F47A66BB /* Pods-HelloWorld.release.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-HelloWorld.release.xcconfig"; path = "Pods/Target Support Files/Pods-HelloWorld/Pods-HelloWorld.release.xcconfig"; sourceTree = ""; }; - 43AB08B32839A6700EA00DD4 /* libPods.a */ = {isa = PBXFileReference; explicitFileType = archive.ar; includeInIndex = 0; path = libPods.a; sourceTree = BUILT_PRODUCTS_DIR; }; 5E3690601B2A23800040F884 /* HelloWorld.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = HelloWorld.app; sourceTree = BUILT_PRODUCTS_DIR; }; 5E3690641B2A23800040F884 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = ""; }; 5E3690651B2A23800040F884 /* main.m */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.objc; path = main.m; sourceTree = ""; }; @@ -37,7 +35,6 @@ buildActionMask = 2147483647; files = ( EF61CF6AE2536A31D47F0E63 /* libPods-HelloWorld.a in Frameworks */, - 3EF35C14BDC2B65E21837F02 /* libPods.a in Frameworks */, ); runOnlyForDeploymentPostprocessing = 0; }; @@ -88,7 +85,6 @@ isa = PBXGroup; children = ( 6B4E1F55F8A2EC95A0E7EE88 /* libPods-HelloWorld.a */, - 43AB08B32839A6700EA00DD4 /* libPods.a */, ); name = Frameworks; sourceTree = ""; @@ -109,12 +105,12 @@ isa = PBXNativeTarget; buildConfigurationList = 5E3690831B2A23810040F884 /* Build configuration list for PBXNativeTarget "HelloWorld" */; buildPhases = ( - ACF9162361FB8F24C70657DE /* Check Pods Manifest.lock */, + ACF9162361FB8F24C70657DE /* [CP] Check Pods Manifest.lock */, 5E36905C1B2A23800040F884 /* Sources */, 5E36905D1B2A23800040F884 /* Frameworks */, 5E36905E1B2A23800040F884 /* Resources */, - 4C7D815378D98AB3BFC1A7D5 /* Copy Pods Resources */, - BB76529986A8BFAF19A385B1 /* Embed Pods Frameworks */, + 4C7D815378D98AB3BFC1A7D5 /* [CP] Copy Pods Resources */, + BB76529986A8BFAF19A385B1 /* [CP] Embed Pods Frameworks */, ); buildRules = ( ); @@ -170,14 +166,14 @@ /* End PBXResourcesBuildPhase section */ /* Begin PBXShellScriptBuildPhase section */ - 4C7D815378D98AB3BFC1A7D5 /* Copy Pods Resources */ = { + 4C7D815378D98AB3BFC1A7D5 /* [CP] Copy Pods Resources */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Copy Pods Resources"; + name = "[CP] Copy Pods Resources"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; @@ -185,14 +181,14 @@ shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-HelloWorld/Pods-HelloWorld-resources.sh\"\n"; showEnvVarsInLog = 0; }; - ACF9162361FB8F24C70657DE /* Check Pods Manifest.lock */ = { + ACF9162361FB8F24C70657DE /* [CP] Check Pods Manifest.lock */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Check Pods Manifest.lock"; + name = "[CP] Check Pods Manifest.lock"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; @@ -200,19 +196,19 @@ shellScript = "diff \"${PODS_ROOT}/../Podfile.lock\" \"${PODS_ROOT}/Manifest.lock\" > /dev/null\nif [[ $? != 0 ]] ; then\n cat << EOM\nerror: The sandbox is not in sync with the Podfile.lock. Run 'pod install' or update your CocoaPods installation.\nEOM\n exit 1\nfi\n"; showEnvVarsInLog = 0; }; - BB76529986A8BFAF19A385B1 /* Embed Pods Frameworks */ = { + BB76529986A8BFAF19A385B1 /* [CP] Embed Pods Frameworks */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Embed Pods Frameworks"; + name = "[CP] Embed Pods Frameworks"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; shellPath = /bin/sh; - shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods/Pods-frameworks.sh\"\n"; + shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-HelloWorld/Pods-HelloWorld-frameworks.sh\"\n"; showEnvVarsInLog = 0; }; /* End PBXShellScriptBuildPhase section */ diff --git a/examples/objective-c/route_guide/RouteGuideClient.xcodeproj/project.pbxproj b/examples/objective-c/route_guide/RouteGuideClient.xcodeproj/project.pbxproj index f99775562c0..0bb84b3b905 100644 --- a/examples/objective-c/route_guide/RouteGuideClient.xcodeproj/project.pbxproj +++ b/examples/objective-c/route_guide/RouteGuideClient.xcodeproj/project.pbxproj @@ -116,12 +116,12 @@ isa = PBXNativeTarget; buildConfigurationList = 632527A31B1D0396003073D9 /* Build configuration list for PBXNativeTarget "RouteGuideClient" */; buildPhases = ( - C6FC30AD2376EC04317237C5 /* Check Pods Manifest.lock */, + C6FC30AD2376EC04317237C5 /* [CP] Check Pods Manifest.lock */, 632527791B1D0395003073D9 /* Sources */, 6325277A1B1D0395003073D9 /* Frameworks */, 6325277B1B1D0395003073D9 /* Resources */, - FFE0BCF30339E7A50A989EAB /* Copy Pods Resources */, - B5388EC5A25E89021740B916 /* Embed Pods Frameworks */, + FFE0BCF30339E7A50A989EAB /* [CP] Copy Pods Resources */, + B5388EC5A25E89021740B916 /* [CP] Embed Pods Frameworks */, ); buildRules = ( ); @@ -178,14 +178,14 @@ /* End PBXResourcesBuildPhase section */ /* Begin PBXShellScriptBuildPhase section */ - B5388EC5A25E89021740B916 /* Embed Pods Frameworks */ = { + B5388EC5A25E89021740B916 /* [CP] Embed Pods Frameworks */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Embed Pods Frameworks"; + name = "[CP] Embed Pods Frameworks"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; @@ -193,14 +193,14 @@ shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-RouteGuideClient/Pods-RouteGuideClient-frameworks.sh\"\n"; showEnvVarsInLog = 0; }; - C6FC30AD2376EC04317237C5 /* Check Pods Manifest.lock */ = { + C6FC30AD2376EC04317237C5 /* [CP] Check Pods Manifest.lock */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Check Pods Manifest.lock"; + name = "[CP] Check Pods Manifest.lock"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; @@ -208,14 +208,14 @@ shellScript = "diff \"${PODS_ROOT}/../Podfile.lock\" \"${PODS_ROOT}/Manifest.lock\" > /dev/null\nif [[ $? != 0 ]] ; then\n cat << EOM\nerror: The sandbox is not in sync with the Podfile.lock. Run 'pod install' or update your CocoaPods installation.\nEOM\n exit 1\nfi\n"; showEnvVarsInLog = 0; }; - FFE0BCF30339E7A50A989EAB /* Copy Pods Resources */ = { + FFE0BCF30339E7A50A989EAB /* [CP] Copy Pods Resources */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Copy Pods Resources"; + name = "[CP] Copy Pods Resources"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; diff --git a/src/objective-c/examples/Sample/Sample.xcodeproj/project.pbxproj b/src/objective-c/examples/Sample/Sample.xcodeproj/project.pbxproj index 611eb6032d5..5c2a6d14f96 100644 --- a/src/objective-c/examples/Sample/Sample.xcodeproj/project.pbxproj +++ b/src/objective-c/examples/Sample/Sample.xcodeproj/project.pbxproj @@ -7,16 +7,16 @@ objects = { /* Begin PBXBuildFile section */ + 426A5020E0E158A101BCA1D9 /* libPods-Sample.a in Frameworks */ = {isa = PBXBuildFile; fileRef = C20055928615A6F8434E26B4 /* libPods-Sample.a */; }; 6369A2701A9322E20015FC5C /* main.m in Sources */ = {isa = PBXBuildFile; fileRef = 6369A26F1A9322E20015FC5C /* main.m */; }; 6369A2731A9322E20015FC5C /* AppDelegate.m in Sources */ = {isa = PBXBuildFile; fileRef = 6369A2721A9322E20015FC5C /* AppDelegate.m */; }; 6369A2761A9322E20015FC5C /* ViewController.m in Sources */ = {isa = PBXBuildFile; fileRef = 6369A2751A9322E20015FC5C /* ViewController.m */; }; 6369A2791A9322E20015FC5C /* Main.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 6369A2771A9322E20015FC5C /* Main.storyboard */; }; 6369A27B1A9322E20015FC5C /* Images.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 6369A27A1A9322E20015FC5C /* Images.xcassets */; }; - FC81FE63CA655031F3524EC0 /* libPods.a in Frameworks */ = {isa = PBXBuildFile; fileRef = 2DC7B7C4C0410F43B9621631 /* libPods.a */; }; /* End PBXBuildFile section */ /* Begin PBXFileReference section */ - 2DC7B7C4C0410F43B9621631 /* libPods.a */ = {isa = PBXFileReference; explicitFileType = archive.ar; includeInIndex = 0; path = libPods.a; sourceTree = BUILT_PRODUCTS_DIR; }; + 5A8C9F4B28733B249DE4AB6D /* Pods-Sample.release.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-Sample.release.xcconfig"; path = "Pods/Target Support Files/Pods-Sample/Pods-Sample.release.xcconfig"; sourceTree = ""; }; 6369A26A1A9322E20015FC5C /* Sample.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = Sample.app; sourceTree = BUILT_PRODUCTS_DIR; }; 6369A26E1A9322E20015FC5C /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = ""; }; 6369A26F1A9322E20015FC5C /* main.m */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.objc; path = main.m; sourceTree = ""; }; @@ -26,8 +26,8 @@ 6369A2751A9322E20015FC5C /* ViewController.m */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.objc; path = ViewController.m; sourceTree = ""; }; 6369A2781A9322E20015FC5C /* Base */ = {isa = PBXFileReference; lastKnownFileType = file.storyboard; name = Base; path = Base.lproj/Main.storyboard; sourceTree = ""; }; 6369A27A1A9322E20015FC5C /* Images.xcassets */ = {isa = PBXFileReference; lastKnownFileType = folder.assetcatalog; path = Images.xcassets; sourceTree = ""; }; - AC29DD6FCDF962F519FEBB0D /* Pods.debug.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = Pods.debug.xcconfig; path = "Pods/Target Support Files/Pods/Pods.debug.xcconfig"; sourceTree = ""; }; - C68330F8D451CC6ACEABA09F /* Pods.release.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = Pods.release.xcconfig; path = "Pods/Target Support Files/Pods/Pods.release.xcconfig"; sourceTree = ""; }; + C20055928615A6F8434E26B4 /* libPods-Sample.a */ = {isa = PBXFileReference; explicitFileType = archive.ar; includeInIndex = 0; path = "libPods-Sample.a"; sourceTree = BUILT_PRODUCTS_DIR; }; + E3C01DF315C4E7433BCEC6E6 /* Pods-Sample.debug.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-Sample.debug.xcconfig"; path = "Pods/Target Support Files/Pods-Sample/Pods-Sample.debug.xcconfig"; sourceTree = ""; }; /* End PBXFileReference section */ /* Begin PBXFrameworksBuildPhase section */ @@ -35,7 +35,7 @@ isa = PBXFrameworksBuildPhase; buildActionMask = 2147483647; files = ( - FC81FE63CA655031F3524EC0 /* libPods.a in Frameworks */, + 426A5020E0E158A101BCA1D9 /* libPods-Sample.a in Frameworks */, ); runOnlyForDeploymentPostprocessing = 0; }; @@ -86,8 +86,8 @@ AB3331C9AE6488E61B2B094E /* Pods */ = { isa = PBXGroup; children = ( - AC29DD6FCDF962F519FEBB0D /* Pods.debug.xcconfig */, - C68330F8D451CC6ACEABA09F /* Pods.release.xcconfig */, + E3C01DF315C4E7433BCEC6E6 /* Pods-Sample.debug.xcconfig */, + 5A8C9F4B28733B249DE4AB6D /* Pods-Sample.release.xcconfig */, ); name = Pods; sourceTree = ""; @@ -95,7 +95,7 @@ C4C2C5219053E079C9EFB930 /* Frameworks */ = { isa = PBXGroup; children = ( - 2DC7B7C4C0410F43B9621631 /* libPods.a */, + C20055928615A6F8434E26B4 /* libPods-Sample.a */, ); name = Frameworks; sourceTree = ""; @@ -107,11 +107,12 @@ isa = PBXNativeTarget; buildConfigurationList = 6369A28D1A9322E20015FC5C /* Build configuration list for PBXNativeTarget "Sample" */; buildPhases = ( - 41F7486D8F66994B0BFB84AF /* Check Pods Manifest.lock */, + 41F7486D8F66994B0BFB84AF /* [CP] Check Pods Manifest.lock */, 6369A2661A9322E20015FC5C /* Sources */, 6369A2671A9322E20015FC5C /* Frameworks */, 6369A2681A9322E20015FC5C /* Resources */, - 04554623324BE4A838846086 /* Copy Pods Resources */, + 04554623324BE4A838846086 /* [CP] Copy Pods Resources */, + C7FAD018D05AB5F0B0FE81E2 /* [CP] Embed Pods Frameworks */, ); buildRules = ( ); @@ -167,29 +168,29 @@ /* End PBXResourcesBuildPhase section */ /* Begin PBXShellScriptBuildPhase section */ - 04554623324BE4A838846086 /* Copy Pods Resources */ = { + 04554623324BE4A838846086 /* [CP] Copy Pods Resources */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Copy Pods Resources"; + name = "[CP] Copy Pods Resources"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; shellPath = /bin/sh; - shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods/Pods-resources.sh\"\n"; + shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-Sample/Pods-Sample-resources.sh\"\n"; showEnvVarsInLog = 0; }; - 41F7486D8F66994B0BFB84AF /* Check Pods Manifest.lock */ = { + 41F7486D8F66994B0BFB84AF /* [CP] Check Pods Manifest.lock */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Check Pods Manifest.lock"; + name = "[CP] Check Pods Manifest.lock"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; @@ -197,6 +198,21 @@ shellScript = "diff \"${PODS_ROOT}/../Podfile.lock\" \"${PODS_ROOT}/Manifest.lock\" > /dev/null\nif [[ $? != 0 ]] ; then\n cat << EOM\nerror: The sandbox is not in sync with the Podfile.lock. Run 'pod install' or update your CocoaPods installation.\nEOM\n exit 1\nfi\n"; showEnvVarsInLog = 0; }; + C7FAD018D05AB5F0B0FE81E2 /* [CP] Embed Pods Frameworks */ = { + isa = PBXShellScriptBuildPhase; + buildActionMask = 2147483647; + files = ( + ); + inputPaths = ( + ); + name = "[CP] Embed Pods Frameworks"; + outputPaths = ( + ); + runOnlyForDeploymentPostprocessing = 0; + shellPath = /bin/sh; + shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-Sample/Pods-Sample-frameworks.sh\"\n"; + showEnvVarsInLog = 0; + }; /* End PBXShellScriptBuildPhase section */ /* Begin PBXSourcesBuildPhase section */ @@ -304,7 +320,7 @@ }; 6369A28E1A9322E20015FC5C /* Debug */ = { isa = XCBuildConfiguration; - baseConfigurationReference = AC29DD6FCDF962F519FEBB0D /* Pods.debug.xcconfig */; + baseConfigurationReference = E3C01DF315C4E7433BCEC6E6 /* Pods-Sample.debug.xcconfig */; buildSettings = { ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; INFOPLIST_FILE = Sample/Info.plist; @@ -315,7 +331,7 @@ }; 6369A28F1A9322E20015FC5C /* Release */ = { isa = XCBuildConfiguration; - baseConfigurationReference = C68330F8D451CC6ACEABA09F /* Pods.release.xcconfig */; + baseConfigurationReference = 5A8C9F4B28733B249DE4AB6D /* Pods-Sample.release.xcconfig */; buildSettings = { ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; INFOPLIST_FILE = Sample/Info.plist; diff --git a/src/objective-c/examples/SwiftSample/SwiftSample.xcodeproj/project.pbxproj b/src/objective-c/examples/SwiftSample/SwiftSample.xcodeproj/project.pbxproj index 2f5716082bf..2a1b30f2cf9 100644 --- a/src/objective-c/examples/SwiftSample/SwiftSample.xcodeproj/project.pbxproj +++ b/src/objective-c/examples/SwiftSample/SwiftSample.xcodeproj/project.pbxproj @@ -7,15 +7,14 @@ objects = { /* Begin PBXBuildFile section */ - 253D3A297105CA46DA960A11 /* libPods.a in Frameworks */ = {isa = PBXBuildFile; fileRef = DC58ACA18DCCB1553531B885 /* libPods.a */; }; 633BFFC81B950B210007E424 /* AppDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 633BFFC71B950B210007E424 /* AppDelegate.swift */; }; 633BFFCA1B950B210007E424 /* ViewController.swift in Sources */ = {isa = PBXBuildFile; fileRef = 633BFFC91B950B210007E424 /* ViewController.swift */; }; 633BFFCD1B950B210007E424 /* Main.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 633BFFCB1B950B210007E424 /* Main.storyboard */; }; 633BFFCF1B950B210007E424 /* Images.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 633BFFCE1B950B210007E424 /* Images.xcassets */; }; + 92EDB1408A1E1E7DDAB25D9C /* libPods-SwiftSample.a in Frameworks */ = {isa = PBXBuildFile; fileRef = 69BB5C6CA3C1F97E007AC527 /* libPods-SwiftSample.a */; }; /* End PBXBuildFile section */ /* Begin PBXFileReference section */ - 12C7B447AA80E624D93B5C54 /* Pods.debug.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = Pods.debug.xcconfig; path = "Pods/Target Support Files/Pods/Pods.debug.xcconfig"; sourceTree = ""; }; 633BFFC21B950B210007E424 /* SwiftSample.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = SwiftSample.app; sourceTree = BUILT_PRODUCTS_DIR; }; 633BFFC61B950B210007E424 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = ""; }; 633BFFC71B950B210007E424 /* AppDelegate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AppDelegate.swift; sourceTree = ""; }; @@ -23,8 +22,9 @@ 633BFFCC1B950B210007E424 /* Base */ = {isa = PBXFileReference; lastKnownFileType = file.storyboard; name = Base; path = Base.lproj/Main.storyboard; sourceTree = ""; }; 633BFFCE1B950B210007E424 /* Images.xcassets */ = {isa = PBXFileReference; lastKnownFileType = folder.assetcatalog; path = Images.xcassets; sourceTree = ""; }; 6367AD231B951655007FD3A4 /* Bridging-Header.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = "Bridging-Header.h"; sourceTree = ""; }; - C335CBC4C160E0D9EDEE646B /* Pods.release.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = Pods.release.xcconfig; path = "Pods/Target Support Files/Pods/Pods.release.xcconfig"; sourceTree = ""; }; - DC58ACA18DCCB1553531B885 /* libPods.a */ = {isa = PBXFileReference; explicitFileType = archive.ar; includeInIndex = 0; path = libPods.a; sourceTree = BUILT_PRODUCTS_DIR; }; + 69BB5C6CA3C1F97E007AC527 /* libPods-SwiftSample.a */ = {isa = PBXFileReference; explicitFileType = archive.ar; includeInIndex = 0; path = "libPods-SwiftSample.a"; sourceTree = BUILT_PRODUCTS_DIR; }; + A7E614A494D89D01BB395761 /* Pods-SwiftSample.debug.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-SwiftSample.debug.xcconfig"; path = "Pods/Target Support Files/Pods-SwiftSample/Pods-SwiftSample.debug.xcconfig"; sourceTree = ""; }; + C314E3E246AF23AC29B38FCF /* Pods-SwiftSample.release.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-SwiftSample.release.xcconfig"; path = "Pods/Target Support Files/Pods-SwiftSample/Pods-SwiftSample.release.xcconfig"; sourceTree = ""; }; /* End PBXFileReference section */ /* Begin PBXFrameworksBuildPhase section */ @@ -32,7 +32,7 @@ isa = PBXFrameworksBuildPhase; buildActionMask = 2147483647; files = ( - 253D3A297105CA46DA960A11 /* libPods.a in Frameworks */, + 92EDB1408A1E1E7DDAB25D9C /* libPods-SwiftSample.a in Frameworks */, ); runOnlyForDeploymentPostprocessing = 0; }; @@ -42,8 +42,8 @@ 31F283C976AE97586C17CCD9 /* Pods */ = { isa = PBXGroup; children = ( - 12C7B447AA80E624D93B5C54 /* Pods.debug.xcconfig */, - C335CBC4C160E0D9EDEE646B /* Pods.release.xcconfig */, + A7E614A494D89D01BB395761 /* Pods-SwiftSample.debug.xcconfig */, + C314E3E246AF23AC29B38FCF /* Pods-SwiftSample.release.xcconfig */, ); name = Pods; sourceTree = ""; @@ -90,7 +90,7 @@ 9D63A7F6423989BA306810CA /* Frameworks */ = { isa = PBXGroup; children = ( - DC58ACA18DCCB1553531B885 /* libPods.a */, + 69BB5C6CA3C1F97E007AC527 /* libPods-SwiftSample.a */, ); name = Frameworks; sourceTree = ""; @@ -102,12 +102,12 @@ isa = PBXNativeTarget; buildConfigurationList = 633BFFE11B950B210007E424 /* Build configuration list for PBXNativeTarget "SwiftSample" */; buildPhases = ( - 6BEEB33CA2705D7D2F2210E6 /* Check Pods Manifest.lock */, + 6BEEB33CA2705D7D2F2210E6 /* [CP] Check Pods Manifest.lock */, 633BFFBE1B950B210007E424 /* Sources */, 633BFFBF1B950B210007E424 /* Frameworks */, 633BFFC01B950B210007E424 /* Resources */, - AC2F6F9AB1C090BB0BEE6E4D /* Copy Pods Resources */, - A1738A987353B0BF2C64F0F7 /* Embed Pods Frameworks */, + AC2F6F9AB1C090BB0BEE6E4D /* [CP] Copy Pods Resources */, + A1738A987353B0BF2C64F0F7 /* [CP] Embed Pods Frameworks */, ); buildRules = ( ); @@ -164,14 +164,14 @@ /* End PBXResourcesBuildPhase section */ /* Begin PBXShellScriptBuildPhase section */ - 6BEEB33CA2705D7D2F2210E6 /* Check Pods Manifest.lock */ = { + 6BEEB33CA2705D7D2F2210E6 /* [CP] Check Pods Manifest.lock */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Check Pods Manifest.lock"; + name = "[CP] Check Pods Manifest.lock"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; @@ -179,34 +179,34 @@ shellScript = "diff \"${PODS_ROOT}/../Podfile.lock\" \"${PODS_ROOT}/Manifest.lock\" > /dev/null\nif [[ $? != 0 ]] ; then\n cat << EOM\nerror: The sandbox is not in sync with the Podfile.lock. Run 'pod install' or update your CocoaPods installation.\nEOM\n exit 1\nfi\n"; showEnvVarsInLog = 0; }; - A1738A987353B0BF2C64F0F7 /* Embed Pods Frameworks */ = { + A1738A987353B0BF2C64F0F7 /* [CP] Embed Pods Frameworks */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Embed Pods Frameworks"; + name = "[CP] Embed Pods Frameworks"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; shellPath = /bin/sh; - shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods/Pods-frameworks.sh\"\n"; + shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-SwiftSample/Pods-SwiftSample-frameworks.sh\"\n"; showEnvVarsInLog = 0; }; - AC2F6F9AB1C090BB0BEE6E4D /* Copy Pods Resources */ = { + AC2F6F9AB1C090BB0BEE6E4D /* [CP] Copy Pods Resources */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Copy Pods Resources"; + name = "[CP] Copy Pods Resources"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; shellPath = /bin/sh; - shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods/Pods-resources.sh\"\n"; + shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-SwiftSample/Pods-SwiftSample-resources.sh\"\n"; showEnvVarsInLog = 0; }; /* End PBXShellScriptBuildPhase section */ @@ -320,7 +320,7 @@ }; 633BFFE21B950B210007E424 /* Debug */ = { isa = XCBuildConfiguration; - baseConfigurationReference = 12C7B447AA80E624D93B5C54 /* Pods.debug.xcconfig */; + baseConfigurationReference = A7E614A494D89D01BB395761 /* Pods-SwiftSample.debug.xcconfig */; buildSettings = { ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; INFOPLIST_FILE = Info.plist; @@ -333,7 +333,7 @@ }; 633BFFE31B950B210007E424 /* Release */ = { isa = XCBuildConfiguration; - baseConfigurationReference = C335CBC4C160E0D9EDEE646B /* Pods.release.xcconfig */; + baseConfigurationReference = C314E3E246AF23AC29B38FCF /* Pods-SwiftSample.release.xcconfig */; buildSettings = { ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; INFOPLIST_FILE = Info.plist; From c0668c837b0ee48f64d0a912e2c6120f2ab719c5 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Wed, 22 Jun 2016 01:30:47 -0700 Subject: [PATCH 099/215] Increase timeout_seconds for build_example_test --- tools/run_tests/run_tests.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 580afc32de4..5c8f6f2a0e7 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -604,7 +604,8 @@ class ObjCLanguage(object): return [self.config.job_spec(['src/objective-c/tests/run_tests.sh'], None, environ=_FORCE_ENVIRON_FOR_WRAPPERS), self.config.job_spec(['src/objective-c/tests/build_example_test.sh'], - None, environ=_FORCE_ENVIRON_FOR_WRAPPERS)] + timeout_seconds=15*60, None, + environ=_FORCE_ENVIRON_FOR_WRAPPERS)] def pre_build_steps(self): return [] From a2dd8385f95259b8a6886902dac87d5df7c7b953 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 22 Jun 2016 10:26:03 -0700 Subject: [PATCH 100/215] Use pipe fds instead of event fds for the test --- test/core/iomgr/ev_epoll_linux_test.c | 55 +++++++++++++++++---------- 1 file changed, 34 insertions(+), 21 deletions(-) diff --git a/test/core/iomgr/ev_epoll_linux_test.c b/test/core/iomgr/ev_epoll_linux_test.c index 034f17fd58b..35eb6791302 100644 --- a/test/core/iomgr/ev_epoll_linux_test.c +++ b/test/core/iomgr/ev_epoll_linux_test.c @@ -34,9 +34,8 @@ #include "src/core/lib/iomgr/ev_epoll_linux.h" #include "src/core/lib/iomgr/ev_posix.h" -#include +#include #include -#include #include #include @@ -55,28 +54,29 @@ typedef struct test_fd { grpc_fd *fd; } test_fd; -static void test_fd_init(test_fd *fds, int num_fds) { +/* num_fds should be an even number */ +static void test_fd_init(test_fd *tfds, int *fds, int num_fds) { int i; for (i = 0; i < num_fds; i++) { - fds[i].inner_fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); - fds[i].fd = grpc_fd_create(fds[i].inner_fd, "test_fd"); + tfds[i].inner_fd = fds[i]; + tfds[i].fd = grpc_fd_create(fds[i], "test_fd"); } } -static void test_fd_cleanup(grpc_exec_ctx *exec_ctx, test_fd *fds, +static void test_fd_cleanup(grpc_exec_ctx *exec_ctx, test_fd *tfds, int num_fds) { int release_fd; int i; for (i = 0; i < num_fds; i++) { - grpc_fd_shutdown(exec_ctx, fds[i].fd); + grpc_fd_shutdown(exec_ctx, tfds[i].fd); grpc_exec_ctx_flush(exec_ctx); - grpc_fd_orphan(exec_ctx, fds[i].fd, NULL, &release_fd, "test_fd_cleanup"); + grpc_fd_orphan(exec_ctx, tfds[i].fd, NULL, &release_fd, "test_fd_cleanup"); grpc_exec_ctx_flush(exec_ctx); - GPR_ASSERT(release_fd == fds[i].inner_fd); - close(fds[i].inner_fd); + GPR_ASSERT(release_fd == tfds[i].inner_fd); + close(tfds[i].inner_fd); } } @@ -121,12 +121,25 @@ static void test_pollset_cleanup(grpc_exec_ctx *exec_ctx, * */ static void test_add_fd_to_pollset() { grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; - test_fd fds[NUM_FDS]; + test_fd tfds[NUM_FDS]; + int fds[NUM_FDS]; test_pollset pollsets[NUM_POLLSETS]; void *expected_pi = NULL; int i; + int r; + + /* Create some dummy file descriptors (using pipe fds for this test. Could be + anything). Also NUM_FDS should be even for this test. */ + for (i = 0; i < NUM_FDS; i = i + 2) { + r = pipe(fds + i); + if (r != 0) { + gpr_log(GPR_ERROR, "Error in creating pipe. %d (%s)", errno, + strerror(errno)); + return; + } + } - test_fd_init(fds, NUM_FDS); + test_fd_init(tfds, fds, NUM_FDS); test_pollset_init(pollsets, NUM_POLLSETS); /*Step 1. @@ -156,41 +169,41 @@ static void test_add_fd_to_pollset() { /* == Step 1 == */ for (i = 0; i <= 2; i++) { - grpc_pollset_add_fd(&exec_ctx, pollsets[0].pollset, fds[i].fd); + grpc_pollset_add_fd(&exec_ctx, pollsets[0].pollset, tfds[i].fd); grpc_exec_ctx_flush(&exec_ctx); } for (i = 3; i <= 4; i++) { - grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, fds[i].fd); + grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, tfds[i].fd); grpc_exec_ctx_flush(&exec_ctx); } for (i = 5; i <= 7; i++) { - grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, fds[i].fd); + grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, tfds[i].fd); grpc_exec_ctx_flush(&exec_ctx); } /* == Step 2 == */ for (i = 0; i <= 1; i++) { - grpc_pollset_add_fd(&exec_ctx, pollsets[3].pollset, fds[i].fd); + grpc_pollset_add_fd(&exec_ctx, pollsets[3].pollset, tfds[i].fd); grpc_exec_ctx_flush(&exec_ctx); } /* == Step 3 == */ - grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, fds[0].fd); + grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, tfds[0].fd); grpc_exec_ctx_flush(&exec_ctx); /* == Step 4 == */ - grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, fds[3].fd); + grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, tfds[3].fd); grpc_exec_ctx_flush(&exec_ctx); /* All polling islands are merged at this point */ /* Compare Fd:0's polling island with that of all other Fds */ - expected_pi = grpc_fd_get_polling_island(fds[0].fd); + expected_pi = grpc_fd_get_polling_island(tfds[0].fd); for (i = 1; i < NUM_FDS; i++) { GPR_ASSERT(grpc_are_polling_islands_equal( - expected_pi, grpc_fd_get_polling_island(fds[i].fd))); + expected_pi, grpc_fd_get_polling_island(tfds[i].fd))); } /* Compare Fd:0's polling island with that of all other pollsets */ @@ -199,7 +212,7 @@ static void test_add_fd_to_pollset() { expected_pi, grpc_pollset_get_polling_island(pollsets[i].pollset))); } - test_fd_cleanup(&exec_ctx, fds, NUM_FDS); + test_fd_cleanup(&exec_ctx, tfds, NUM_FDS); test_pollset_cleanup(&exec_ctx, pollsets, NUM_POLLSETS); grpc_exec_ctx_finish(&exec_ctx); } From 82393efe0c930f7756e9d0e427d2e9ae61e0a88e Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Wed, 22 Jun 2016 10:29:17 -0700 Subject: [PATCH 101/215] Fix run_test.py --- tools/run_tests/run_tests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 5c8f6f2a0e7..2a404df0d41 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -604,7 +604,7 @@ class ObjCLanguage(object): return [self.config.job_spec(['src/objective-c/tests/run_tests.sh'], None, environ=_FORCE_ENVIRON_FOR_WRAPPERS), self.config.job_spec(['src/objective-c/tests/build_example_test.sh'], - timeout_seconds=15*60, None, + None, timeout_seconds=15*60, environ=_FORCE_ENVIRON_FOR_WRAPPERS)] def pre_build_steps(self): From b16e4518b765f82b9b65222bd56738cda44e7259 Mon Sep 17 00:00:00 2001 From: Nathaniel Manista Date: Tue, 21 Jun 2016 23:24:06 +0000 Subject: [PATCH 102/215] Add a test of metadata, status code, and details --- src/python/grpcio/tests/tests.json | 1 + .../tests/unit/_metadata_code_details_test.py | 523 ++++++++++++++++++ 2 files changed, 524 insertions(+) create mode 100644 src/python/grpcio/tests/unit/_metadata_code_details_test.py diff --git a/src/python/grpcio/tests/tests.json b/src/python/grpcio/tests/tests.json index 8e509621a84..e384a2fc135 100644 --- a/src/python/grpcio/tests/tests.json +++ b/src/python/grpcio/tests/tests.json @@ -24,6 +24,7 @@ "_implementations_test.ChannelCredentialsTest", "_insecure_interop_test.InsecureInteropTest", "_logging_pool_test.LoggingPoolTest", + "_metadata_code_details_test.MetadataCodeDetailsTest", "_metadata_test.MetadataTest", "_not_found_test.NotFoundTest", "_python_plugin_test.PythonPluginTest", diff --git a/src/python/grpcio/tests/unit/_metadata_code_details_test.py b/src/python/grpcio/tests/unit/_metadata_code_details_test.py new file mode 100644 index 00000000000..dd74268cbf1 --- /dev/null +++ b/src/python/grpcio/tests/unit/_metadata_code_details_test.py @@ -0,0 +1,523 @@ +# Copyright 2016, Google Inc. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""Tests application-provided metadata, status code, and details.""" + +import threading +import unittest + +import grpc +from grpc.framework.foundation import logging_pool + +from tests.unit import test_common +from tests.unit.framework.common import test_constants +from tests.unit.framework.common import test_control + +_SERIALIZED_REQUEST = b'\x46\x47\x48' +_SERIALIZED_RESPONSE = b'\x49\x50\x51' + +_REQUEST_SERIALIZER = lambda unused_request: _SERIALIZED_REQUEST +_REQUEST_DESERIALIZER = lambda unused_serialized_request: object() +_RESPONSE_SERIALIZER = lambda unused_response: _SERIALIZED_RESPONSE +_RESPONSE_DESERIALIZER = lambda unused_serialized_resopnse: object() + +_SERVICE = b'test.TestService' +_UNARY_UNARY = b'UnaryUnary' +_UNARY_STREAM = b'UnaryStream' +_STREAM_UNARY = b'StreamUnary' +_STREAM_STREAM = b'StreamStream' + +_CLIENT_METADATA = ( + (b'client-md-key', b'client-md-key'), + (b'client-md-key-bin', b'\x00\x01') +) + +_SERVER_INITIAL_METADATA = ( + (b'server-initial-md-key', b'server-initial-md-value'), + (b'server-initial-md-key-bin', b'\x00\x02') +) + +_SERVER_TRAILING_METADATA = ( + (b'server-trailing-md-key', b'server-trailing-md-value'), + (b'server-trailing-md-key-bin', b'\x00\x03') +) + +_NON_OK_CODE = grpc.StatusCode.NOT_FOUND +_DETAILS = b'Test details!' + + +class _Servicer(object): + + def __init__(self): + self._lock = threading.Lock() + self._code = None + self._details = None + self._exception = False + self._return_none = False + self._received_client_metadata = None + + def unary_unary(self, request, context): + with self._lock: + self._received_client_metadata = context.invocation_metadata() + context.send_initial_metadata(_SERVER_INITIAL_METADATA) + context.set_trailing_metadata(_SERVER_TRAILING_METADATA) + if self._code is not None: + context.set_code(self._code) + if self._details is not None: + context.set_details(self._details) + if self._exception: + raise test_control.Defect() + else: + return None if self._return_none else object() + + def unary_stream(self, request, context): + with self._lock: + self._received_client_metadata = context.invocation_metadata() + context.send_initial_metadata(_SERVER_INITIAL_METADATA) + context.set_trailing_metadata(_SERVER_TRAILING_METADATA) + if self._code is not None: + context.set_code(self._code) + if self._details is not None: + context.set_details(self._details) + for _ in range(test_constants.STREAM_LENGTH // 2): + yield _SERIALIZED_RESPONSE + if self._exception: + raise test_control.Defect() + + def stream_unary(self, request_iterator, context): + with self._lock: + self._received_client_metadata = context.invocation_metadata() + context.send_initial_metadata(_SERVER_INITIAL_METADATA) + context.set_trailing_metadata(_SERVER_TRAILING_METADATA) + if self._code is not None: + context.set_code(self._code) + if self._details is not None: + context.set_details(self._details) + # TODO(https://github.com/grpc/grpc/issues/6891): just ignore the + # request iterator. + for ignored_request in request_iterator: + pass + if self._exception: + raise test_control.Defect() + else: + return None if self._return_none else _SERIALIZED_RESPONSE + + def stream_stream(self, request_iterator, context): + with self._lock: + self._received_client_metadata = context.invocation_metadata() + context.send_initial_metadata(_SERVER_INITIAL_METADATA) + context.set_trailing_metadata(_SERVER_TRAILING_METADATA) + if self._code is not None: + context.set_code(self._code) + if self._details is not None: + context.set_details(self._details) + # TODO(https://github.com/grpc/grpc/issues/6891): just ignore the + # request iterator. + for ignored_request in request_iterator: + pass + for _ in range(test_constants.STREAM_LENGTH // 3): + yield object() + if self._exception: + raise test_control.Defect() + + def set_code(self, code): + with self._lock: + self._code = code + + def set_details(self, details): + with self._lock: + self._details = details + + def set_exception(self): + with self._lock: + self._exception = True + + def set_return_none(self): + with self._lock: + self._return_none = True + + def received_client_metadata(self): + with self._lock: + return self._received_client_metadata + + +def _generic_handler(servicer): + method_handlers = { + _UNARY_UNARY: grpc.unary_unary_rpc_method_handler( + servicer.unary_unary, request_deserializer=_REQUEST_DESERIALIZER, + response_serializer=_RESPONSE_SERIALIZER), + _UNARY_STREAM: grpc.unary_stream_rpc_method_handler( + servicer.unary_stream), + _STREAM_UNARY: grpc.stream_unary_rpc_method_handler( + servicer.stream_unary), + _STREAM_STREAM: grpc.stream_stream_rpc_method_handler( + servicer.stream_stream, request_deserializer=_REQUEST_DESERIALIZER, + response_serializer=_RESPONSE_SERIALIZER), + } + return grpc.method_handlers_generic_handler(_SERVICE, method_handlers) + + +class MetadataCodeDetailsTest(unittest.TestCase): + + def setUp(self): + self._servicer = _Servicer() + self._server_pool = logging_pool.pool(test_constants.THREAD_CONCURRENCY) + self._server = grpc.server( + (_generic_handler(self._servicer),), self._server_pool) + port = self._server.add_insecure_port('[::]:0') + self._server.start() + + channel = grpc.insecure_channel('localhost:{}'.format(port)) + self._unary_unary = channel.unary_unary( + b'/'.join((b'', _SERVICE, _UNARY_UNARY,)), + request_serializer=_REQUEST_SERIALIZER, + response_deserializer=_RESPONSE_DESERIALIZER,) + self._unary_stream = channel.unary_stream( + b'/'.join((b'', _SERVICE, _UNARY_STREAM,)),) + self._stream_unary = channel.stream_unary( + b'/'.join((b'', _SERVICE, _STREAM_UNARY,)),) + self._stream_stream = channel.stream_stream( + b'/'.join((b'', _SERVICE, _STREAM_STREAM,)), + request_serializer=_REQUEST_SERIALIZER, + response_deserializer=_RESPONSE_DESERIALIZER,) + + + def testSuccessfulUnaryUnary(self): + self._servicer.set_details(_DETAILS) + + unused_response, call = self._unary_unary.with_call( + object(), metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, call.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, call.trailing_metadata())) + self.assertIs(grpc.StatusCode.OK, call.code()) + self.assertEqual(_DETAILS, call.details()) + + def testSuccessfulUnaryStream(self): + self._servicer.set_details(_DETAILS) + + call = self._unary_stream(_SERIALIZED_REQUEST, metadata=_CLIENT_METADATA) + received_initial_metadata = call.initial_metadata() + for _ in call: + pass + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, received_initial_metadata)) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, call.trailing_metadata())) + self.assertIs(grpc.StatusCode.OK, call.code()) + self.assertEqual(_DETAILS, call.details()) + + def testSuccessfulStreamUnary(self): + self._servicer.set_details(_DETAILS) + + unused_response, call = self._stream_unary.with_call( + iter([_SERIALIZED_REQUEST] * test_constants.STREAM_LENGTH), + metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, call.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, call.trailing_metadata())) + self.assertIs(grpc.StatusCode.OK, call.code()) + self.assertEqual(_DETAILS, call.details()) + + def testSuccessfulStreamStream(self): + self._servicer.set_details(_DETAILS) + + call = self._stream_stream( + iter([object()] * test_constants.STREAM_LENGTH), + metadata=_CLIENT_METADATA) + received_initial_metadata = call.initial_metadata() + for _ in call: + pass + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, received_initial_metadata)) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, call.trailing_metadata())) + self.assertIs(grpc.StatusCode.OK, call.code()) + self.assertEqual(_DETAILS, call.details()) + + def testCustomCodeUnaryUnary(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + + with self.assertRaises(grpc.RpcError) as exception_context: + self._unary_unary.with_call(object(), metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, + exception_context.exception.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, + exception_context.exception.trailing_metadata())) + self.assertIs(_NON_OK_CODE, exception_context.exception.code()) + self.assertEqual(_DETAILS, exception_context.exception.details()) + + def testCustomCodeUnaryStream(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + + call = self._unary_stream(_SERIALIZED_REQUEST, metadata=_CLIENT_METADATA) + received_initial_metadata = call.initial_metadata() + with self.assertRaises(grpc.RpcError): + for _ in call: + pass + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, received_initial_metadata)) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, call.trailing_metadata())) + self.assertIs(_NON_OK_CODE, call.code()) + self.assertEqual(_DETAILS, call.details()) + + def testCustomCodeStreamUnary(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + + with self.assertRaises(grpc.RpcError) as exception_context: + self._stream_unary.with_call( + iter([_SERIALIZED_REQUEST] * test_constants.STREAM_LENGTH), + metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, + exception_context.exception.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, + exception_context.exception.trailing_metadata())) + self.assertIs(_NON_OK_CODE, exception_context.exception.code()) + self.assertEqual(_DETAILS, exception_context.exception.details()) + + def testCustomCodeStreamStream(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + + call = self._stream_stream( + iter([object()] * test_constants.STREAM_LENGTH), + metadata=_CLIENT_METADATA) + received_initial_metadata = call.initial_metadata() + with self.assertRaises(grpc.RpcError) as exception_context: + for _ in call: + pass + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, received_initial_metadata)) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, + exception_context.exception.trailing_metadata())) + self.assertIs(_NON_OK_CODE, exception_context.exception.code()) + self.assertEqual(_DETAILS, exception_context.exception.details()) + + def testCustomCodeExceptionUnaryUnary(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + self._servicer.set_exception() + + with self.assertRaises(grpc.RpcError) as exception_context: + self._unary_unary.with_call(object(), metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, + exception_context.exception.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, + exception_context.exception.trailing_metadata())) + self.assertIs(_NON_OK_CODE, exception_context.exception.code()) + self.assertEqual(_DETAILS, exception_context.exception.details()) + + def testCustomCodeExceptionUnaryStream(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + self._servicer.set_exception() + + call = self._unary_stream(_SERIALIZED_REQUEST, metadata=_CLIENT_METADATA) + received_initial_metadata = call.initial_metadata() + with self.assertRaises(grpc.RpcError): + for _ in call: + pass + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, received_initial_metadata)) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, call.trailing_metadata())) + self.assertIs(_NON_OK_CODE, call.code()) + self.assertEqual(_DETAILS, call.details()) + + def testCustomCodeExceptionStreamUnary(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + self._servicer.set_exception() + + with self.assertRaises(grpc.RpcError) as exception_context: + self._stream_unary.with_call( + iter([_SERIALIZED_REQUEST] * test_constants.STREAM_LENGTH), + metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, + exception_context.exception.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, + exception_context.exception.trailing_metadata())) + self.assertIs(_NON_OK_CODE, exception_context.exception.code()) + self.assertEqual(_DETAILS, exception_context.exception.details()) + + def testCustomCodeExceptionStreamStream(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + self._servicer.set_exception() + + call = self._stream_stream( + iter([object()] * test_constants.STREAM_LENGTH), + metadata=_CLIENT_METADATA) + received_initial_metadata = call.initial_metadata() + with self.assertRaises(grpc.RpcError): + for _ in call: + pass + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, received_initial_metadata)) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, call.trailing_metadata())) + self.assertIs(_NON_OK_CODE, call.code()) + self.assertEqual(_DETAILS, call.details()) + + def testCustomCodeReturnNoneUnaryUnary(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + self._servicer.set_return_none() + + with self.assertRaises(grpc.RpcError) as exception_context: + self._unary_unary.with_call(object(), metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, + exception_context.exception.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, + exception_context.exception.trailing_metadata())) + self.assertIs(_NON_OK_CODE, exception_context.exception.code()) + self.assertEqual(_DETAILS, exception_context.exception.details()) + + def testCustomCodeReturnNoneStreamUnary(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + self._servicer.set_return_none() + + with self.assertRaises(grpc.RpcError) as exception_context: + self._stream_unary.with_call( + iter([_SERIALIZED_REQUEST] * test_constants.STREAM_LENGTH), + metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, + exception_context.exception.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, + exception_context.exception.trailing_metadata())) + self.assertIs(_NON_OK_CODE, exception_context.exception.code()) + self.assertEqual(_DETAILS, exception_context.exception.details()) + + +if __name__ == '__main__': + unittest.main(verbosity=2) From edfebc909a7e4a2ada5e07d2012a227fb15d4064 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Tue, 21 Jun 2016 16:39:21 -0700 Subject: [PATCH 103/215] dont generate NewClient method for C# services --- src/compiler/csharp_generator.cc | 15 --------------- 1 file changed, 15 deletions(-) diff --git a/src/compiler/csharp_generator.cc b/src/compiler/csharp_generator.cc index fc8feaf0fcc..8c1c6f2c826 100644 --- a/src/compiler/csharp_generator.cc +++ b/src/compiler/csharp_generator.cc @@ -485,20 +485,6 @@ void GenerateBindServiceMethod(Printer* out, const ServiceDescriptor *service) { out->Print("\n"); } -void GenerateNewStubMethods(Printer* out, const ServiceDescriptor *service) { - out->Print("/// Creates a new client for $servicename$\n", - "servicename", GetServiceClassName(service)); - out->Print("public static $classname$ NewClient(Channel channel)\n", - "classname", GetClientClassName(service)); - out->Print("{\n"); - out->Indent(); - out->Print("return new $classname$(channel);\n", "classname", - GetClientClassName(service)); - out->Outdent(); - out->Print("}\n"); - out->Print("\n"); -} - void GenerateService(Printer* out, const ServiceDescriptor *service, bool generate_client, bool generate_server, bool internal_access) { @@ -524,7 +510,6 @@ void GenerateService(Printer* out, const ServiceDescriptor *service, } if (generate_client) { GenerateClientStub(out, service); - GenerateNewStubMethods(out, service); } if (generate_server) { GenerateBindServiceMethod(out, service); From 614944324246b2b4eef9fc2e989af20375c49cd5 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Wed, 22 Jun 2016 13:34:59 -0700 Subject: [PATCH 104/215] add code to unregister endpoints --- src/core/lib/iomgr/network_status_tracker.c | 46 ++++++++++++++++++++- src/core/lib/iomgr/network_status_tracker.h | 1 + src/core/lib/iomgr/tcp_posix.c | 1 + src/core/lib/iomgr/tcp_windows.c | 1 + 4 files changed, 47 insertions(+), 2 deletions(-) diff --git a/src/core/lib/iomgr/network_status_tracker.c b/src/core/lib/iomgr/network_status_tracker.c index 6e432368cf1..f4f8e97a252 100644 --- a/src/core/lib/iomgr/network_status_tracker.c +++ b/src/core/lib/iomgr/network_status_tracker.c @@ -33,6 +33,7 @@ #include "src/core/lib/iomgr/endpoint.h" #include +#include typedef struct endpoint_ll_node { grpc_endpoint *ep; @@ -40,9 +41,13 @@ typedef struct endpoint_ll_node { } endpoint_ll_node; static endpoint_ll_node *head = NULL; +static gpr_mu g_endpoint_mutex; +static bool g_init_done = false; -// TODO(makarandd): Install callback with OS to monitor network status. void grpc_initialize_network_status_monitor() { + g_init_done = true; + gpr_mu_init(&g_endpoint_mutex); + // TODO(makarandd): Install callback with OS to monitor network status. } void grpc_destroy_network_status_monitor() { @@ -51,9 +56,15 @@ void grpc_destroy_network_status_monitor() { gpr_free(curr); curr = next; } + gpr_mu_destroy(&g_endpoint_mutex); } void grpc_network_status_register_endpoint(grpc_endpoint *ep) { + if (!g_init_done) { + grpc_initialize_network_status_monitor(); + } + gpr_mu_lock(&g_endpoint_mutex); + gpr_log(GPR_DEBUG, "Register endpoint %p", ep); if (head == NULL) { head = (endpoint_ll_node *)gpr_malloc(sizeof(endpoint_ll_node)); head->ep = ep; @@ -64,19 +75,50 @@ void grpc_network_status_register_endpoint(grpc_endpoint *ep) { head->ep = ep; head->next = prev_head; } + gpr_mu_unlock(&g_endpoint_mutex); +} + +void grpc_network_status_unregister_endpoint(grpc_endpoint *ep) { + gpr_mu_lock(&g_endpoint_mutex); + GPR_ASSERT(head); + gpr_log(GPR_DEBUG, "Unregister endpoint %p", ep); + bool found = false; + endpoint_ll_node *prev = head; + // if we're unregistering the head, just move head to the next + if (ep == head->ep) { + head = head->next; + gpr_free(prev); + found = true; + } else { + for (endpoint_ll_node *curr = head->next; curr != NULL; curr = curr->next) { + if (ep == curr->ep) { + prev->next = curr->next; + gpr_free(curr); + found = true; + break; + } + prev = curr; + } + } + gpr_mu_unlock(&g_endpoint_mutex); + GPR_ASSERT(found); } // Walk the linked-list from head and execute shutdown. It is possible that // other threads might be in the process of shutdown as well, but that has -// no side effect. +// no side effect since endpoint shutdown is idempotent. void grpc_network_status_shutdown_all_endpoints() { + gpr_mu_lock(&g_endpoint_mutex); if (head == NULL) { + gpr_mu_unlock(&g_endpoint_mutex); return; } grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; for (endpoint_ll_node *curr = head; curr != NULL; curr = curr->next) { + gpr_log(GPR_DEBUG, "Shutting down endpoint %p", curr->ep); curr->ep->vtable->shutdown(&exec_ctx, curr->ep); } + gpr_mu_unlock(&g_endpoint_mutex); grpc_exec_ctx_finish(&exec_ctx); } diff --git a/src/core/lib/iomgr/network_status_tracker.h b/src/core/lib/iomgr/network_status_tracker.h index 4154151927f..74a1aa8135f 100644 --- a/src/core/lib/iomgr/network_status_tracker.h +++ b/src/core/lib/iomgr/network_status_tracker.h @@ -36,5 +36,6 @@ #include "src/core/lib/iomgr/endpoint.h" void grpc_network_status_register_endpoint(grpc_endpoint *ep); +void grpc_network_status_unregister_endpoint(grpc_endpoint *ep); void grpc_network_status_shutdown_all_endpoints(); #endif /* GRPC_CORE_LIB_IOMGR_NETWORK_STATUS_TRACKER_H */ diff --git a/src/core/lib/iomgr/tcp_posix.c b/src/core/lib/iomgr/tcp_posix.c index f7818353b0a..56c1eef024c 100644 --- a/src/core/lib/iomgr/tcp_posix.c +++ b/src/core/lib/iomgr/tcp_posix.c @@ -153,6 +153,7 @@ static void tcp_ref(grpc_tcp *tcp) { gpr_ref(&tcp->refcount); } #endif static void tcp_destroy(grpc_exec_ctx *exec_ctx, grpc_endpoint *ep) { + grpc_network_status_unregister_endpoint(ep); grpc_tcp *tcp = (grpc_tcp *)ep; TCP_UNREF(exec_ctx, tcp, "destroy"); } diff --git a/src/core/lib/iomgr/tcp_windows.c b/src/core/lib/iomgr/tcp_windows.c index 8140d1d8cd2..37ab59021e3 100644 --- a/src/core/lib/iomgr/tcp_windows.c +++ b/src/core/lib/iomgr/tcp_windows.c @@ -379,6 +379,7 @@ static void win_shutdown(grpc_exec_ctx *exec_ctx, grpc_endpoint *ep) { } static void win_destroy(grpc_exec_ctx *exec_ctx, grpc_endpoint *ep) { + grpc_network_status_unregister_endpoint(ep); grpc_tcp *tcp = (grpc_tcp *)ep; TCP_UNREF(tcp, "destroy"); } From 76a0795b73ad2632c435fc338bb49368d1d68d9f Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 22 Jun 2016 15:09:06 -0700 Subject: [PATCH 105/215] Fix build errors on some configurations --- src/core/lib/iomgr/ev_epoll_linux.c | 13 +++++++------ test/core/iomgr/ev_epoll_linux_test.c | 10 ++++++++-- 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 6464d3ba348..88cbc586349 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -34,6 +34,7 @@ #include #include +/* This polling engine is only relevant on linux kernels supporting epoll() */ #ifdef GPR_LINUX_EPOLL #include "src/core/lib/iomgr/ev_epoll_linux.h" @@ -322,7 +323,7 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, #ifdef GRPC_TSAN /* See the definition of g_epoll_sync for more context */ - gpr_atm_rel_store(&g_epoll_sync, 0); + gpr_atm_rel_store(&g_epoll_sync, (gpr_atm) 0); #endif /* defined(GRPC_TSAN) */ for (i = 0; i < fd_count; i++) { @@ -442,8 +443,8 @@ static polling_island *polling_island_create(grpc_fd *initial_fd) { pi->fds = NULL; } - gpr_atm_rel_store(&pi->ref_count, 0); - gpr_atm_rel_store(&pi->merged_to, NULL); + gpr_atm_rel_store(&pi->ref_count, (gpr_atm) 0); + gpr_atm_rel_store(&pi->merged_to, (gpr_atm) NULL); pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); @@ -472,7 +473,7 @@ static polling_island *polling_island_create(grpc_fd *initial_fd) { static void polling_island_delete(polling_island *pi) { GPR_ASSERT(pi->fd_cnt == 0); - gpr_atm_rel_store(&pi->merged_to, NULL); + gpr_atm_rel_store(&pi->merged_to, (gpr_atm) NULL); close(pi->epoll_fd); pi->epoll_fd = -1; @@ -648,7 +649,7 @@ static polling_island *polling_island_merge(polling_island *p, polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); /* Add the 'merged_to' link from p --> q */ - gpr_atm_rel_store(&p->merged_to, q); + gpr_atm_rel_store(&p->merged_to, (gpr_atm) q); PI_ADD_REF(q, "pi_merge"); /* To account for the new incoming ref from p */ gpr_mu_unlock(&p->mu); @@ -810,7 +811,7 @@ static grpc_fd *fd_create(int fd, const char *name) { holding a lock to it anyway. */ gpr_mu_lock(&new_fd->mu); - gpr_atm_rel_store(&new_fd->refst, 1); + gpr_atm_rel_store(&new_fd->refst, (gpr_atm) 1); new_fd->fd = fd; new_fd->shutdown = false; new_fd->orphaned = false; diff --git a/test/core/iomgr/ev_epoll_linux_test.c b/test/core/iomgr/ev_epoll_linux_test.c index 35eb6791302..66a69f52cdb 100644 --- a/test/core/iomgr/ev_epoll_linux_test.c +++ b/test/core/iomgr/ev_epoll_linux_test.c @@ -30,7 +30,10 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * */ +#include +/* This test only relevant on linux systems where epoll() is available */ +#ifdef GPR_LINUX_EPOLL #include "src/core/lib/iomgr/ev_epoll_linux.h" #include "src/core/lib/iomgr/ev_posix.h" @@ -128,8 +131,10 @@ static void test_add_fd_to_pollset() { int i; int r; - /* Create some dummy file descriptors (using pipe fds for this test. Could be - anything). Also NUM_FDS should be even for this test. */ + /* Create some dummy file descriptors. Currently using pipe file descriptors + * for this test but we could use any other type of file descriptors. Also, + * since pipe() used in this test creates two fds in each call, NUM_FDS should + * be an even number */ for (i = 0; i < NUM_FDS; i = i + 2) { r = pipe(fds + i); if (r != 0) { @@ -234,3 +239,4 @@ int main(int argc, char **argv) { grpc_iomgr_shutdown(); return 0; } +#endif /* defined(GPR_LINUX_EPOLL) */ From fcb304f493f72fa5e487b364f78126e01c550288 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 22 Jun 2016 15:25:41 -0700 Subject: [PATCH 106/215] Compilation failure --- test/core/iomgr/ev_epoll_linux_test.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/test/core/iomgr/ev_epoll_linux_test.c b/test/core/iomgr/ev_epoll_linux_test.c index 66a69f52cdb..2547dc98718 100644 --- a/test/core/iomgr/ev_epoll_linux_test.c +++ b/test/core/iomgr/ev_epoll_linux_test.c @@ -239,4 +239,6 @@ int main(int argc, char **argv) { grpc_iomgr_shutdown(); return 0; } -#endif /* defined(GPR_LINUX_EPOLL) */ +#else /* defined(GPR_LINUX_EPOLL) */ +int main(int argc, char **argv) { return 0; } +#endif /* !defined(GPR_LINUX_EPOLL) */ From 932d02725200028415f8420bdaf35d9484f419d4 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Wed, 22 Jun 2016 16:17:15 -0700 Subject: [PATCH 107/215] clang-format fixes --- src/core/lib/iomgr/network_status_tracker.c | 10 +++++----- test/core/end2end/tests/network_status_change.c | 1 - 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/src/core/lib/iomgr/network_status_tracker.c b/src/core/lib/iomgr/network_status_tracker.c index f4f8e97a252..0e34605c81d 100644 --- a/src/core/lib/iomgr/network_status_tracker.c +++ b/src/core/lib/iomgr/network_status_tracker.c @@ -31,9 +31,9 @@ * */ -#include "src/core/lib/iomgr/endpoint.h" #include #include +#include "src/core/lib/iomgr/endpoint.h" typedef struct endpoint_ll_node { grpc_endpoint *ep; @@ -51,7 +51,7 @@ void grpc_initialize_network_status_monitor() { } void grpc_destroy_network_status_monitor() { - for (endpoint_ll_node *curr = head; curr != NULL; ) { + for (endpoint_ll_node *curr = head; curr != NULL;) { endpoint_ll_node *next = curr->next; gpr_free(curr); curr = next; @@ -86,9 +86,9 @@ void grpc_network_status_unregister_endpoint(grpc_endpoint *ep) { endpoint_ll_node *prev = head; // if we're unregistering the head, just move head to the next if (ep == head->ep) { - head = head->next; - gpr_free(prev); - found = true; + head = head->next; + gpr_free(prev); + found = true; } else { for (endpoint_ll_node *curr = head->next; curr != NULL; curr = curr->next) { if (ep == curr->ep) { diff --git a/test/core/end2end/tests/network_status_change.c b/test/core/end2end/tests/network_status_change.c index bbc85007022..7e7628391de 100644 --- a/test/core/end2end/tests/network_status_change.c +++ b/test/core/end2end/tests/network_status_change.c @@ -217,7 +217,6 @@ static void test_invoke_network_status_change(grpc_end2end_test_config config) { GPR_ASSERT(0 == strcmp(call_details.host, "foo.test.google.fr")); GPR_ASSERT(was_cancelled == 0); - gpr_free(details); grpc_metadata_array_destroy(&initial_metadata_recv); grpc_metadata_array_destroy(&trailing_metadata_recv); From 8891ef50eef2e858b2290de36691c4b535de3acc Mon Sep 17 00:00:00 2001 From: Ken Payson Date: Wed, 22 Jun 2016 16:47:52 -0700 Subject: [PATCH 108/215] Initialize port to -1 --- src/core/lib/iomgr/tcp_server_windows.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/core/lib/iomgr/tcp_server_windows.c b/src/core/lib/iomgr/tcp_server_windows.c index 2a51671ec71..86982bc1836 100644 --- a/src/core/lib/iomgr/tcp_server_windows.c +++ b/src/core/lib/iomgr/tcp_server_windows.c @@ -396,7 +396,7 @@ static grpc_error *add_socket_to_server(grpc_tcp_server *s, SOCKET sock, size_t addr_len, unsigned port_index, grpc_tcp_listener **listener) { grpc_tcp_listener *sp = NULL; - int port; + int port = -1; int status; GUID guid = WSAID_ACCEPTEX; DWORD ioctl_num_bytes; From 9c578c7a734937f67b928f4511dd40448c621174 Mon Sep 17 00:00:00 2001 From: Bill Clarke Date: Wed, 22 Jun 2016 16:48:08 -0700 Subject: [PATCH 109/215] Const correctness for ClientContext and ServerContext getters --- include/grpc++/impl/codegen/client_context.h | 10 +++++----- include/grpc++/impl/codegen/server_context.h | 7 ++++--- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/include/grpc++/impl/codegen/client_context.h b/include/grpc++/impl/codegen/client_context.h index e23fd4eda34..a132c9a57aa 100644 --- a/include/grpc++/impl/codegen/client_context.h +++ b/include/grpc++/impl/codegen/client_context.h @@ -193,7 +193,7 @@ class ClientContext { /// /// \return A multimap of initial metadata key-value pairs from the server. const std::multimap& - GetServerInitialMetadata() { + GetServerInitialMetadata() const { GPR_CODEGEN_ASSERT(initial_metadata_received_); return recv_initial_metadata_; } @@ -205,7 +205,7 @@ class ClientContext { /// /// \return A multimap of metadata trailing key-value pairs from the server. const std::multimap& - GetServerTrailingMetadata() { + GetServerTrailingMetadata() const { // TODO(yangg) check finished return trailing_metadata_; } @@ -230,13 +230,13 @@ class ClientContext { #ifndef GRPC_CXX0X_NO_CHRONO /// Return the deadline for the client call. - std::chrono::system_clock::time_point deadline() { + std::chrono::system_clock::time_point deadline() const { return Timespec2Timepoint(deadline_); } #endif // !GRPC_CXX0X_NO_CHRONO /// Return a \a gpr_timespec representation of the client call's deadline. - gpr_timespec raw_deadline() { return deadline_; } + gpr_timespec raw_deadline() const { return deadline_; } /// Set the per call authority header (see /// https://tools.ietf.org/html/rfc7540#section-8.1.2.3). @@ -337,7 +337,7 @@ class ClientContext { const InputMessage& request, OutputMessage* result); - grpc_call* call() { return call_; } + grpc_call* call() const { return call_; } void set_call(grpc_call* call, const std::shared_ptr& channel); uint32_t initial_metadata_flags() const { diff --git a/include/grpc++/impl/codegen/server_context.h b/include/grpc++/impl/codegen/server_context.h index a1e1ed176f6..cea13a513f6 100644 --- a/include/grpc++/impl/codegen/server_context.h +++ b/include/grpc++/impl/codegen/server_context.h @@ -94,12 +94,12 @@ class ServerContext { ~ServerContext(); #ifndef GRPC_CXX0X_NO_CHRONO - std::chrono::system_clock::time_point deadline() { + std::chrono::system_clock::time_point deadline() const { return Timespec2Timepoint(deadline_); } #endif // !GRPC_CXX0X_NO_CHRONO - gpr_timespec raw_deadline() { return deadline_; } + gpr_timespec raw_deadline() const { return deadline_; } void AddInitialMetadata(const grpc::string& key, const grpc::string& value); void AddTrailingMetadata(const grpc::string& key, const grpc::string& value); @@ -122,7 +122,8 @@ class ServerContext { // was called. void TryCancel() const; - const std::multimap& client_metadata() { + const std::multimap& client_metadata() + const { return client_metadata_; } From c072c927a885e49cb13afaad11a93379a1255b9f Mon Sep 17 00:00:00 2001 From: Bill Clarke Date: Wed, 22 Jun 2016 16:48:08 -0700 Subject: [PATCH 110/215] Const correctness for ClientContext and ServerContext getters --- include/grpc++/impl/codegen/client_context.h | 10 +++++----- include/grpc++/impl/codegen/server_context.h | 7 ++++--- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/include/grpc++/impl/codegen/client_context.h b/include/grpc++/impl/codegen/client_context.h index e23fd4eda34..a132c9a57aa 100644 --- a/include/grpc++/impl/codegen/client_context.h +++ b/include/grpc++/impl/codegen/client_context.h @@ -193,7 +193,7 @@ class ClientContext { /// /// \return A multimap of initial metadata key-value pairs from the server. const std::multimap& - GetServerInitialMetadata() { + GetServerInitialMetadata() const { GPR_CODEGEN_ASSERT(initial_metadata_received_); return recv_initial_metadata_; } @@ -205,7 +205,7 @@ class ClientContext { /// /// \return A multimap of metadata trailing key-value pairs from the server. const std::multimap& - GetServerTrailingMetadata() { + GetServerTrailingMetadata() const { // TODO(yangg) check finished return trailing_metadata_; } @@ -230,13 +230,13 @@ class ClientContext { #ifndef GRPC_CXX0X_NO_CHRONO /// Return the deadline for the client call. - std::chrono::system_clock::time_point deadline() { + std::chrono::system_clock::time_point deadline() const { return Timespec2Timepoint(deadline_); } #endif // !GRPC_CXX0X_NO_CHRONO /// Return a \a gpr_timespec representation of the client call's deadline. - gpr_timespec raw_deadline() { return deadline_; } + gpr_timespec raw_deadline() const { return deadline_; } /// Set the per call authority header (see /// https://tools.ietf.org/html/rfc7540#section-8.1.2.3). @@ -337,7 +337,7 @@ class ClientContext { const InputMessage& request, OutputMessage* result); - grpc_call* call() { return call_; } + grpc_call* call() const { return call_; } void set_call(grpc_call* call, const std::shared_ptr& channel); uint32_t initial_metadata_flags() const { diff --git a/include/grpc++/impl/codegen/server_context.h b/include/grpc++/impl/codegen/server_context.h index a1e1ed176f6..cea13a513f6 100644 --- a/include/grpc++/impl/codegen/server_context.h +++ b/include/grpc++/impl/codegen/server_context.h @@ -94,12 +94,12 @@ class ServerContext { ~ServerContext(); #ifndef GRPC_CXX0X_NO_CHRONO - std::chrono::system_clock::time_point deadline() { + std::chrono::system_clock::time_point deadline() const { return Timespec2Timepoint(deadline_); } #endif // !GRPC_CXX0X_NO_CHRONO - gpr_timespec raw_deadline() { return deadline_; } + gpr_timespec raw_deadline() const { return deadline_; } void AddInitialMetadata(const grpc::string& key, const grpc::string& value); void AddTrailingMetadata(const grpc::string& key, const grpc::string& value); @@ -122,7 +122,8 @@ class ServerContext { // was called. void TryCancel() const; - const std::multimap& client_metadata() { + const std::multimap& client_metadata() + const { return client_metadata_; } From 17233ceaa9a0150eeb0959cd7953ac8be100b413 Mon Sep 17 00:00:00 2001 From: Ken Payson Date: Wed, 22 Jun 2016 17:16:15 -0700 Subject: [PATCH 111/215] Ignore unused variables in ruby compilation --- Rakefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Rakefile b/Rakefile index f208a24fd33..f44946fe937 100755 --- a/Rakefile +++ b/Rakefile @@ -77,7 +77,7 @@ task 'dlls' do grpc_config = ENV['GRPC_CONFIG'] || 'opt' verbose = ENV['V'] || '0' - env = 'CPPFLAGS="-D_WIN32_WINNT=0x600 -DUNICODE -D_UNICODE" ' + env = 'CPPFLAGS="-D_WIN32_WINNT=0x600 -DUNICODE -D_UNICODE -Wno-unused-variable -Wno-unused-result" ' env += 'LDFLAGS=-static ' env += 'SYSTEM=MINGW32 ' env += 'EMBED_ZLIB=true ' From f41ebc3968ef06439d6d1468256ac5000940ea8f Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 22 Jun 2016 12:47:14 -0700 Subject: [PATCH 112/215] remove occurences of NewClient factory method --- examples/csharp/helloworld/GreeterClient/Program.cs | 2 +- examples/csharp/route_guide/RouteGuideClient/Program.cs | 2 +- src/csharp/Grpc.Examples.Tests/MathClientServerTests.cs | 2 +- src/csharp/Grpc.HealthCheck.Tests/HealthClientServerTest.cs | 2 +- src/csharp/Grpc.IntegrationTesting/ClientRunners.cs | 6 +++--- .../Grpc.IntegrationTesting/GeneratedServiceBaseTest.cs | 2 +- .../Grpc.IntegrationTesting/InteropClientServerTest.cs | 4 ++-- .../Grpc.IntegrationTesting/MetadataCredentialsTest.cs | 4 ++-- src/csharp/Grpc.IntegrationTesting/SslCredentialsTest.cs | 2 +- src/csharp/Grpc.IntegrationTesting/StressTestClient.cs | 2 +- 10 files changed, 14 insertions(+), 14 deletions(-) diff --git a/examples/csharp/helloworld/GreeterClient/Program.cs b/examples/csharp/helloworld/GreeterClient/Program.cs index 4393f2f3c29..444d4735095 100644 --- a/examples/csharp/helloworld/GreeterClient/Program.cs +++ b/examples/csharp/helloworld/GreeterClient/Program.cs @@ -39,7 +39,7 @@ namespace GreeterClient { Channel channel = new Channel("127.0.0.1:50051", ChannelCredentials.Insecure); - var client = Greeter.NewClient(channel); + var client = new Greeter.GreeterClient(channel); String user = "you"; var reply = client.SayHello(new HelloRequest { Name = user }); diff --git a/examples/csharp/route_guide/RouteGuideClient/Program.cs b/examples/csharp/route_guide/RouteGuideClient/Program.cs index ee51fbe8e04..8a173dcc3b3 100644 --- a/examples/csharp/route_guide/RouteGuideClient/Program.cs +++ b/examples/csharp/route_guide/RouteGuideClient/Program.cs @@ -231,7 +231,7 @@ namespace Routeguide static void Main(string[] args) { var channel = new Channel("127.0.0.1:50052", ChannelCredentials.Insecure); - var client = new RouteGuideClient(RouteGuide.NewClient(channel)); + var client = new RouteGuideClient(new RouteGuide.RouteGuideClient(channel)); // Looking for a valid feature client.GetFeature(409146138, -746188906); diff --git a/src/csharp/Grpc.Examples.Tests/MathClientServerTests.cs b/src/csharp/Grpc.Examples.Tests/MathClientServerTests.cs index 324c209ca1d..50dacc2eaaf 100644 --- a/src/csharp/Grpc.Examples.Tests/MathClientServerTests.cs +++ b/src/csharp/Grpc.Examples.Tests/MathClientServerTests.cs @@ -62,7 +62,7 @@ namespace Math.Tests }; server.Start(); channel = new Channel(Host, server.Ports.Single().BoundPort, ChannelCredentials.Insecure); - client = Math.NewClient(channel); + client = new Math.MathClient(channel); } [TestFixtureTearDown] diff --git a/src/csharp/Grpc.HealthCheck.Tests/HealthClientServerTest.cs b/src/csharp/Grpc.HealthCheck.Tests/HealthClientServerTest.cs index 070674bae9d..25a58fb3864 100644 --- a/src/csharp/Grpc.HealthCheck.Tests/HealthClientServerTest.cs +++ b/src/csharp/Grpc.HealthCheck.Tests/HealthClientServerTest.cs @@ -65,7 +65,7 @@ namespace Grpc.HealthCheck.Tests server.Start(); channel = new Channel(Host, server.Ports.Single().BoundPort, ChannelCredentials.Insecure); - client = Grpc.Health.V1.Health.NewClient(channel); + client = new Grpc.Health.V1.Health.HealthClient(channel); } [TestFixtureTearDown] diff --git a/src/csharp/Grpc.IntegrationTesting/ClientRunners.cs b/src/csharp/Grpc.IntegrationTesting/ClientRunners.cs index 39b9ae08e64..b9c0fe6d0d8 100644 --- a/src/csharp/Grpc.IntegrationTesting/ClientRunners.cs +++ b/src/csharp/Grpc.IntegrationTesting/ClientRunners.cs @@ -211,7 +211,7 @@ namespace Grpc.IntegrationTesting bool profilerReset = false; - var client = BenchmarkService.NewClient(channel); + var client = new BenchmarkService.BenchmarkServiceClient(channel); var request = CreateSimpleRequest(); var stopwatch = new Stopwatch(); @@ -237,7 +237,7 @@ namespace Grpc.IntegrationTesting private async Task RunUnaryAsync(Channel channel, IInterarrivalTimer timer) { - var client = BenchmarkService.NewClient(channel); + var client = new BenchmarkService.BenchmarkServiceClient(channel); var request = CreateSimpleRequest(); var stopwatch = new Stopwatch(); @@ -256,7 +256,7 @@ namespace Grpc.IntegrationTesting private async Task RunStreamingPingPongAsync(Channel channel, IInterarrivalTimer timer) { - var client = BenchmarkService.NewClient(channel); + var client = new BenchmarkService.BenchmarkServiceClient(channel); var request = CreateSimpleRequest(); var stopwatch = new Stopwatch(); diff --git a/src/csharp/Grpc.IntegrationTesting/GeneratedServiceBaseTest.cs b/src/csharp/Grpc.IntegrationTesting/GeneratedServiceBaseTest.cs index 001533ce315..4216dc1d6be 100644 --- a/src/csharp/Grpc.IntegrationTesting/GeneratedServiceBaseTest.cs +++ b/src/csharp/Grpc.IntegrationTesting/GeneratedServiceBaseTest.cs @@ -61,7 +61,7 @@ namespace Grpc.IntegrationTesting }; server.Start(); channel = new Channel(Host, server.Ports.Single().BoundPort, ChannelCredentials.Insecure); - client = TestService.NewClient(channel); + client = new TestService.TestServiceClient(channel); } [TearDown] diff --git a/src/csharp/Grpc.IntegrationTesting/InteropClientServerTest.cs b/src/csharp/Grpc.IntegrationTesting/InteropClientServerTest.cs index 4ee1ff5ec8a..f907f630dab 100644 --- a/src/csharp/Grpc.IntegrationTesting/InteropClientServerTest.cs +++ b/src/csharp/Grpc.IntegrationTesting/InteropClientServerTest.cs @@ -69,7 +69,7 @@ namespace Grpc.IntegrationTesting }; int port = server.Ports.Single().BoundPort; channel = new Channel(Host, port, TestCredentials.CreateSslCredentials(), options); - client = TestService.NewClient(channel); + client = new TestService.TestServiceClient(channel); } [TestFixtureTearDown] @@ -148,7 +148,7 @@ namespace Grpc.IntegrationTesting [Test] public void UnimplementedMethod() { - InteropClient.RunUnimplementedMethod(UnimplementedService.NewClient(channel)); + InteropClient.RunUnimplementedMethod(new UnimplementedService.UnimplementedServiceClient(channel)); } } } diff --git a/src/csharp/Grpc.IntegrationTesting/MetadataCredentialsTest.cs b/src/csharp/Grpc.IntegrationTesting/MetadataCredentialsTest.cs index 9fd575f1900..eb3bb8a28bb 100644 --- a/src/csharp/Grpc.IntegrationTesting/MetadataCredentialsTest.cs +++ b/src/csharp/Grpc.IntegrationTesting/MetadataCredentialsTest.cs @@ -88,7 +88,7 @@ namespace Grpc.IntegrationTesting var channelCredentials = ChannelCredentials.Create(TestCredentials.CreateSslCredentials(), CallCredentials.FromInterceptor(asyncAuthInterceptor)); channel = new Channel(Host, server.Ports.Single().BoundPort, channelCredentials, options); - client = TestService.NewClient(channel); + client = new TestService.TestServiceClient(channel); client.UnaryCall(new SimpleRequest { }); } @@ -97,7 +97,7 @@ namespace Grpc.IntegrationTesting public void MetadataCredentials_PerCall() { channel = new Channel(Host, server.Ports.Single().BoundPort, TestCredentials.CreateSslCredentials(), options); - client = TestService.NewClient(channel); + client = new TestService.TestServiceClient(channel); var callCredentials = CallCredentials.FromInterceptor(asyncAuthInterceptor); client.UnaryCall(new SimpleRequest { }, new CallOptions(credentials: callCredentials)); diff --git a/src/csharp/Grpc.IntegrationTesting/SslCredentialsTest.cs b/src/csharp/Grpc.IntegrationTesting/SslCredentialsTest.cs index 3df45b5f708..f85e272711f 100644 --- a/src/csharp/Grpc.IntegrationTesting/SslCredentialsTest.cs +++ b/src/csharp/Grpc.IntegrationTesting/SslCredentialsTest.cs @@ -79,7 +79,7 @@ namespace Grpc.IntegrationTesting }; channel = new Channel(Host, server.Ports.Single().BoundPort, clientCredentials, options); - client = TestService.NewClient(channel); + client = new TestService.TestServiceClient(channel); } [TestFixtureTearDown] diff --git a/src/csharp/Grpc.IntegrationTesting/StressTestClient.cs b/src/csharp/Grpc.IntegrationTesting/StressTestClient.cs index 4d6ca7ece57..74ee040ae49 100644 --- a/src/csharp/Grpc.IntegrationTesting/StressTestClient.cs +++ b/src/csharp/Grpc.IntegrationTesting/StressTestClient.cs @@ -148,7 +148,7 @@ namespace Grpc.IntegrationTesting channels.Add(channel); for (int j = 0; j < options.NumStubsPerChannel; j++) { - var client = TestService.NewClient(channel); + var client = new TestService.TestServiceClient(channel); var task = Task.Factory.StartNew(() => RunBodyAsync(client).GetAwaiter().GetResult(), TaskCreationOptions.LongRunning); tasks.Add(task); From bfcd6b627ece5ddab1aa690e4bf205a6c66d8fe3 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 22 Jun 2016 16:52:46 -0700 Subject: [PATCH 113/215] generate comments for C# client constructors --- src/compiler/csharp_generator.cc | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/src/compiler/csharp_generator.cc b/src/compiler/csharp_generator.cc index 8c1c6f2c826..f5a0876cf98 100644 --- a/src/compiler/csharp_generator.cc +++ b/src/compiler/csharp_generator.cc @@ -333,22 +333,28 @@ void GenerateClientStub(Printer* out, const ServiceDescriptor *service) { out->Indent(); // constructors + out->Print("/// Creates a new client for $servicename$\n" + "/// The channel to use to make remote calls.\n", + "servicename", GetServiceClassName(service)); out->Print("public $name$(Channel channel) : base(channel)\n", "name", GetClientClassName(service)); out->Print("{\n"); out->Print("}\n"); + out->Print("/// Creates a new client for $servicename$ that uses a custom CallInvoker.\n" + "/// The callInvoker to use to make remote calls.\n", + "servicename", GetServiceClassName(service)); out->Print("public $name$(CallInvoker callInvoker) : base(callInvoker)\n", "name", GetClientClassName(service)); out->Print("{\n"); out->Print("}\n"); - out->Print("///Protected parameterless constructor to allow creation" + out->Print("/// Protected parameterless constructor to allow creation" " of test doubles.\n"); out->Print("protected $name$() : base()\n", "name", GetClientClassName(service)); out->Print("{\n"); out->Print("}\n"); - out->Print("///Protected constructor to allow creation of configured" - " clients.\n"); + out->Print("/// Protected constructor to allow creation of configured clients.\n" + "/// The client configuration.\n"); out->Print("protected $name$(ClientBaseConfiguration configuration)" " : base(configuration)\n", "name", GetClientClassName(service)); From ed6ea4c691193483223fdb3a7ada5095b8a2494e Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 22 Jun 2016 12:38:44 -0700 Subject: [PATCH 114/215] regenerate protos --- src/csharp/Grpc.Examples/MathGrpc.cs | 15 +++---- src/csharp/Grpc.HealthCheck/HealthGrpc.cs | 15 +++---- .../Grpc.IntegrationTesting/MetricsGrpc.cs | 15 +++---- .../Grpc.IntegrationTesting/ServicesGrpc.cs | 30 ++++++------- .../Grpc.IntegrationTesting/TestGrpc.cs | 45 +++++++++---------- 5 files changed, 56 insertions(+), 64 deletions(-) diff --git a/src/csharp/Grpc.Examples/MathGrpc.cs b/src/csharp/Grpc.Examples/MathGrpc.cs index 4bbefcbe01a..25abc514195 100644 --- a/src/csharp/Grpc.Examples/MathGrpc.cs +++ b/src/csharp/Grpc.Examples/MathGrpc.cs @@ -128,17 +128,22 @@ namespace Math { /// Client for Math public class MathClient : ClientBase { + /// Creates a new client for Math + /// The channel to use to make remote calls. public MathClient(Channel channel) : base(channel) { } + /// Creates a new client for Math that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public MathClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected MathClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected MathClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -235,12 +240,6 @@ namespace Math { } } - /// Creates a new client for Math - public static MathClient NewClient(Channel channel) - { - return new MathClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(MathBase serviceImpl) { diff --git a/src/csharp/Grpc.HealthCheck/HealthGrpc.cs b/src/csharp/Grpc.HealthCheck/HealthGrpc.cs index d0ade7d02be..43eea0f22f5 100644 --- a/src/csharp/Grpc.HealthCheck/HealthGrpc.cs +++ b/src/csharp/Grpc.HealthCheck/HealthGrpc.cs @@ -71,17 +71,22 @@ namespace Grpc.Health.V1 { /// Client for Health public class HealthClient : ClientBase { + /// Creates a new client for Health + /// The channel to use to make remote calls. public HealthClient(Channel channel) : base(channel) { } + /// Creates a new client for Health that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public HealthClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected HealthClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected HealthClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -108,12 +113,6 @@ namespace Grpc.Health.V1 { } } - /// Creates a new client for Health - public static HealthClient NewClient(Channel channel) - { - return new HealthClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(HealthBase serviceImpl) { diff --git a/src/csharp/Grpc.IntegrationTesting/MetricsGrpc.cs b/src/csharp/Grpc.IntegrationTesting/MetricsGrpc.cs index 22bd27ec0aa..040798e3c21 100644 --- a/src/csharp/Grpc.IntegrationTesting/MetricsGrpc.cs +++ b/src/csharp/Grpc.IntegrationTesting/MetricsGrpc.cs @@ -97,17 +97,22 @@ namespace Grpc.Testing { /// Client for MetricsService public class MetricsServiceClient : ClientBase { + /// Creates a new client for MetricsService + /// The channel to use to make remote calls. public MetricsServiceClient(Channel channel) : base(channel) { } + /// Creates a new client for MetricsService that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public MetricsServiceClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected MetricsServiceClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected MetricsServiceClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -162,12 +167,6 @@ namespace Grpc.Testing { } } - /// Creates a new client for MetricsService - public static MetricsServiceClient NewClient(Channel channel) - { - return new MetricsServiceClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(MetricsServiceBase serviceImpl) { diff --git a/src/csharp/Grpc.IntegrationTesting/ServicesGrpc.cs b/src/csharp/Grpc.IntegrationTesting/ServicesGrpc.cs index 9c99296115c..e205dea93ea 100644 --- a/src/csharp/Grpc.IntegrationTesting/ServicesGrpc.cs +++ b/src/csharp/Grpc.IntegrationTesting/ServicesGrpc.cs @@ -93,17 +93,22 @@ namespace Grpc.Testing { /// Client for BenchmarkService public class BenchmarkServiceClient : ClientBase { + /// Creates a new client for BenchmarkService + /// The channel to use to make remote calls. public BenchmarkServiceClient(Channel channel) : base(channel) { } + /// Creates a new client for BenchmarkService that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public BenchmarkServiceClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected BenchmarkServiceClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected BenchmarkServiceClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -162,12 +167,6 @@ namespace Grpc.Testing { } } - /// Creates a new client for BenchmarkService - public static BenchmarkServiceClient NewClient(Channel channel) - { - return new BenchmarkServiceClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(BenchmarkServiceBase serviceImpl) { @@ -273,17 +272,22 @@ namespace Grpc.Testing { /// Client for WorkerService public class WorkerServiceClient : ClientBase { + /// Creates a new client for WorkerService + /// The channel to use to make remote calls. public WorkerServiceClient(Channel channel) : base(channel) { } + /// Creates a new client for WorkerService that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public WorkerServiceClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected WorkerServiceClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected WorkerServiceClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -398,12 +402,6 @@ namespace Grpc.Testing { } } - /// Creates a new client for WorkerService - public static WorkerServiceClient NewClient(Channel channel) - { - return new WorkerServiceClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(WorkerServiceBase serviceImpl) { diff --git a/src/csharp/Grpc.IntegrationTesting/TestGrpc.cs b/src/csharp/Grpc.IntegrationTesting/TestGrpc.cs index 6c252013f84..3e149da3e01 100644 --- a/src/csharp/Grpc.IntegrationTesting/TestGrpc.cs +++ b/src/csharp/Grpc.IntegrationTesting/TestGrpc.cs @@ -168,17 +168,22 @@ namespace Grpc.Testing { /// Client for TestService public class TestServiceClient : ClientBase { + /// Creates a new client for TestService + /// The channel to use to make remote calls. public TestServiceClient(Channel channel) : base(channel) { } + /// Creates a new client for TestService that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public TestServiceClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected TestServiceClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected TestServiceClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -315,12 +320,6 @@ namespace Grpc.Testing { } } - /// Creates a new client for TestService - public static TestServiceClient NewClient(Channel channel) - { - return new TestServiceClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(TestServiceBase serviceImpl) { @@ -373,17 +372,22 @@ namespace Grpc.Testing { /// Client for UnimplementedService public class UnimplementedServiceClient : ClientBase { + /// Creates a new client for UnimplementedService + /// The channel to use to make remote calls. public UnimplementedServiceClient(Channel channel) : base(channel) { } + /// Creates a new client for UnimplementedService that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public UnimplementedServiceClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected UnimplementedServiceClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected UnimplementedServiceClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -422,12 +426,6 @@ namespace Grpc.Testing { } } - /// Creates a new client for UnimplementedService - public static UnimplementedServiceClient NewClient(Channel channel) - { - return new UnimplementedServiceClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(UnimplementedServiceBase serviceImpl) { @@ -485,17 +483,22 @@ namespace Grpc.Testing { /// Client for ReconnectService public class ReconnectServiceClient : ClientBase { + /// Creates a new client for ReconnectService + /// The channel to use to make remote calls. public ReconnectServiceClient(Channel channel) : base(channel) { } + /// Creates a new client for ReconnectService that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public ReconnectServiceClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected ReconnectServiceClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected ReconnectServiceClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -538,12 +541,6 @@ namespace Grpc.Testing { } } - /// Creates a new client for ReconnectService - public static ReconnectServiceClient NewClient(Channel channel) - { - return new ReconnectServiceClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(ReconnectServiceBase serviceImpl) { From 0224dcc2dcda932a171776de325fa2e66c95478f Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 22 Jun 2016 18:04:00 -0700 Subject: [PATCH 115/215] clang format --- src/core/lib/iomgr/ev_epoll_linux.c | 12 ++++++------ src/core/lib/iomgr/ev_posix.h | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 88cbc586349..b1e9ac8a631 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -323,7 +323,7 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, #ifdef GRPC_TSAN /* See the definition of g_epoll_sync for more context */ - gpr_atm_rel_store(&g_epoll_sync, (gpr_atm) 0); + gpr_atm_rel_store(&g_epoll_sync, (gpr_atm)0); #endif /* defined(GRPC_TSAN) */ for (i = 0; i < fd_count; i++) { @@ -443,8 +443,8 @@ static polling_island *polling_island_create(grpc_fd *initial_fd) { pi->fds = NULL; } - gpr_atm_rel_store(&pi->ref_count, (gpr_atm) 0); - gpr_atm_rel_store(&pi->merged_to, (gpr_atm) NULL); + gpr_atm_rel_store(&pi->ref_count, (gpr_atm)0); + gpr_atm_rel_store(&pi->merged_to, (gpr_atm)NULL); pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); @@ -473,7 +473,7 @@ static polling_island *polling_island_create(grpc_fd *initial_fd) { static void polling_island_delete(polling_island *pi) { GPR_ASSERT(pi->fd_cnt == 0); - gpr_atm_rel_store(&pi->merged_to, (gpr_atm) NULL); + gpr_atm_rel_store(&pi->merged_to, (gpr_atm)NULL); close(pi->epoll_fd); pi->epoll_fd = -1; @@ -649,7 +649,7 @@ static polling_island *polling_island_merge(polling_island *p, polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); /* Add the 'merged_to' link from p --> q */ - gpr_atm_rel_store(&p->merged_to, (gpr_atm) q); + gpr_atm_rel_store(&p->merged_to, (gpr_atm)q); PI_ADD_REF(q, "pi_merge"); /* To account for the new incoming ref from p */ gpr_mu_unlock(&p->mu); @@ -811,7 +811,7 @@ static grpc_fd *fd_create(int fd, const char *name) { holding a lock to it anyway. */ gpr_mu_lock(&new_fd->mu); - gpr_atm_rel_store(&new_fd->refst, (gpr_atm) 1); + gpr_atm_rel_store(&new_fd->refst, (gpr_atm)1); new_fd->fd = fd; new_fd->shutdown = false; new_fd->orphaned = false; diff --git a/src/core/lib/iomgr/ev_posix.h b/src/core/lib/iomgr/ev_posix.h index 32260fe2ee5..579c84ef707 100644 --- a/src/core/lib/iomgr/ev_posix.h +++ b/src/core/lib/iomgr/ev_posix.h @@ -100,7 +100,7 @@ void grpc_event_engine_init(void); void grpc_event_engine_shutdown(void); /* Return the name of the poll strategy */ -const char* grpc_get_poll_strategy_name(); +const char *grpc_get_poll_strategy_name(); /* Create a wrapped file descriptor. Requires fd is a non-blocking file descriptor. From cc00745e8c6e160665e753c9fda6bf2650b1823d Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Thu, 23 Jun 2016 00:18:15 -0700 Subject: [PATCH 116/215] brought spec up to date --- doc/compression.md | 31 +++++++++++++++++++------------ 1 file changed, 19 insertions(+), 12 deletions(-) diff --git a/doc/compression.md b/doc/compression.md index 15fae4d29bf..9229ef9af10 100644 --- a/doc/compression.md +++ b/doc/compression.md @@ -42,13 +42,13 @@ and RPC settings (for example, if compression would result in small or negative gains). When a message from a client compressed with an unsupported algorithm is -processed by a server, it WILL result in an INVALID\_ARGUMENT error on the +processed by a server, it WILL result in a `UNIMPLEMENTED` error status on the server. The server will then include in its response a `grpc-accept-encoding` -header specifying the algorithms it does accept. If an INTERNAL error is -returned from the server despite having used one of the algorithms from the -`grpc-accept-encoding` header, the cause MUST NOT be related to compression. -Data sent from a server compressed with an algorithm not supported by the client -WILL result in an INTERNAL error on the client side. +header specifying the algorithms it does accept. If an `UNIMPLEMENTED` error +status is returned from the server despite having used one of the algorithms +from the `grpc-accept-encoding` header, the cause MUST NOT be related to +compression. Data sent from a server compressed with an algorithm not supported +by the client WILL result in an `INTERNAL` error status on the client side. Note that a peer MAY choose to not disclose all the encodings it supports. However, if it receives a message compressed in an undisclosed but supported @@ -99,13 +99,20 @@ compressed. channel compression configuration MUST be used. 1. When a compression method (including no compression) is specified for an outgoing message, the message MUST be compressed accordingly. -1. A message compressed in a way not supported by its endpoint MUST fail with -INVALID\_ARGUMENT status, its associated description indicating the unsupported -condition as well as the supported ones. The returned `grpc-accept-encoding` -header MUST NOT contain the compression method (encoding) used. +1. A message compressed by a client in a way not supported by its server MUST +fail with status `UNIMPLEMENTED`, its associated description indicating the +unsupported condition as well as the supported ones. The returned +`grpc-accept-encoding` header MUST NOT contain the compression method +(encoding) used. +1. A message compressed by a server in a way not supported by its client MUST +fail with status `INTERNAL`, its associated description indicating the +unsupported condition as well as the supported ones. The returned +`grpc-accept-encoding` header MUST NOT contain the compression method +(encoding) used. 1. An ill-constructed message with its [Compressed-Flag bit](PROTOCOL-HTTP2.md#compressed-flag) set but lacking a "[grpc-encoding](PROTOCOL-HTTP2.md#message-encoding)" -entry different from _identity_ in its metadata MUST fail with INTERNAL status, -its associated description indicating the invalid Compressed-Flag condition. +entry different from _identity_ in its metadata MUST fail with `INTERNAL` +status, its associated description indicating the invalid Compressed-Flag +condition. From c9b7ad0c0a541ac7559bb7081c260de195fe8315 Mon Sep 17 00:00:00 2001 From: Ken Payson Date: Thu, 23 Jun 2016 02:38:20 -0700 Subject: [PATCH 117/215] Updated grpcio c extension build for windows to use mingw. The default compiler for Python c extensions is VS2008 or VS2010 depending on the Python version. Since c-core moved onto the c99 standard, these compilers are not compatible. --- tools/run_tests/build_artifact_python.bat | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/tools/run_tests/build_artifact_python.bat b/tools/run_tests/build_artifact_python.bat index fea0275426f..295347e947c 100644 --- a/tools/run_tests/build_artifact_python.bat +++ b/tools/run_tests/build_artifact_python.bat @@ -37,10 +37,11 @@ set NUGET=C:\nuget\nuget.exe mkdir src\python\grpcio\grpc\_cython\_windows +@rem TODO(atash): maybe we could avoid the grpc_c.(32|64).python shim below if +@rem this used the right python build? copy /Y vsprojects\Release\grpc_dll.dll src\python\grpcio\grpc\_cython\_windows\grpc_c.32.python || goto :error copy /Y vsprojects\x64\Release\grpc_dll.dll src\python\grpcio\grpc\_cython\_windows\grpc_c.64.python || goto :error - set PATH=C:\%1;C:\%1\scripts;C:\msys64\mingw%2\bin;%PATH% pip install --upgrade six @@ -50,12 +51,6 @@ pip install -rrequirements.txt set GRPC_PYTHON_USE_CUSTOM_BDIST=0 set GRPC_PYTHON_BUILD_WITH_CYTHON=1 -@rem TODO(atash): maybe we could avoid the grpc_c.(32|64).python shim above if -@rem this used the right python build? -python setup.py bdist_wheel - -@rem Build gRPC Python tools -@rem @rem Because this is windows and *everything seems to hate Windows* we have to @rem set all of these flags ourselves because Python won't help us (see the @rem setup.py of the grpcio_tools project). @@ -70,6 +65,18 @@ set GRPC_PYTHON_CFLAGS=-fno-wrapv -frtti -std=c++11 python -c "from distutils.cygwinccompiler import get_msvcr; print(get_msvcr()[0])" > temp.txt set /p PYTHON_MSVCR= Date: Thu, 23 Jun 2016 08:22:30 -0700 Subject: [PATCH 118/215] Re-enable accidentally disabled traces --- src/core/lib/iomgr/tcp_posix.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/core/lib/iomgr/tcp_posix.c b/src/core/lib/iomgr/tcp_posix.c index 017f52c3677..1046b60bcc9 100644 --- a/src/core/lib/iomgr/tcp_posix.c +++ b/src/core/lib/iomgr/tcp_posix.c @@ -160,7 +160,7 @@ static void call_read_cb(grpc_exec_ctx *exec_ctx, grpc_tcp *tcp, grpc_error *error) { grpc_closure *cb = tcp->read_cb; - if (false && grpc_tcp_trace) { + if (grpc_tcp_trace) { size_t i; const char *str = grpc_error_string(error); gpr_log(GPR_DEBUG, "read: error=%s", str); @@ -394,7 +394,7 @@ static void tcp_write(grpc_exec_ctx *exec_ctx, grpc_endpoint *ep, grpc_tcp *tcp = (grpc_tcp *)ep; grpc_error *error = GRPC_ERROR_NONE; - if (false && grpc_tcp_trace) { + if (grpc_tcp_trace) { size_t i; for (i = 0; i < buf->count; i++) { From 75f6828636a432ff32435ac06a6121b8396c67fd Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Thu, 23 Jun 2016 10:16:16 -0700 Subject: [PATCH 119/215] Fixed typo --- doc/compression.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/compression.md b/doc/compression.md index 9229ef9af10..de245d90fee 100644 --- a/doc/compression.md +++ b/doc/compression.md @@ -42,7 +42,7 @@ and RPC settings (for example, if compression would result in small or negative gains). When a message from a client compressed with an unsupported algorithm is -processed by a server, it WILL result in a `UNIMPLEMENTED` error status on the +processed by a server, it WILL result in an `UNIMPLEMENTED` error status on the server. The server will then include in its response a `grpc-accept-encoding` header specifying the algorithms it does accept. If an `UNIMPLEMENTED` error status is returned from the server despite having used one of the algorithms From dde4fc66fb55291434a017c667d6a961332801e6 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 23 Jun 2016 10:35:19 -0700 Subject: [PATCH 120/215] changed service to _service, and added a TODO TODO for removing boilerplate code --- .../objective-c/route_guide/ViewControllers.m | 27 ++++++++++--------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/examples/objective-c/route_guide/ViewControllers.m b/examples/objective-c/route_guide/ViewControllers.m index 5fd411e3ba6..a5c03e9cfca 100644 --- a/examples/objective-c/route_guide/ViewControllers.m +++ b/examples/objective-c/route_guide/ViewControllers.m @@ -81,7 +81,7 @@ static NSString * const kHostAddress = @"localhost:50051"; * not to have a feature. */ @interface GetFeatureViewController : UIViewController { - RTGRouteGuide *service; + RTGRouteGuide *_service; } @property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @@ -90,6 +90,7 @@ static NSString * const kHostAddress = @"localhost:50051"; - (void)execRequest { void (^handler)(RTGFeature *response, NSError *error) = ^(RTGFeature *response, NSError *error) { + // TODO(makdharma): Remove boilerplate by consolidating into one log function. if (response.name.length) { NSString *str =[NSString stringWithFormat:@"%@\nFound feature called %@ at %@.", self.outputLabel.text, response.location, response.name]; self.outputLabel.text = str; @@ -109,8 +110,8 @@ static NSString * const kHostAddress = @"localhost:50051"; point.latitude = 409146138; point.longitude = -746188906; - [service getFeatureWithRequest:point handler:handler]; - [service getFeatureWithRequest:[RTGPoint message] handler:handler]; + [_service getFeatureWithRequest:point handler:handler]; + [_service getFeatureWithRequest:[RTGPoint message] handler:handler]; } - (void)viewDidLoad { @@ -119,7 +120,7 @@ static NSString * const kHostAddress = @"localhost:50051"; // This only needs to be done once per host, before creating service objects for that host. [GRPCCall useInsecureConnectionsForHost:kHostAddress]; - service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; + _service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; } - (void)viewDidAppear:(BOOL)animated { @@ -139,7 +140,7 @@ static NSString * const kHostAddress = @"localhost:50051"; * the pre-generated database. Prints each response as it comes in. */ @interface ListFeaturesViewController : UIViewController { - RTGRouteGuide *service; + RTGRouteGuide *_service; } @property (weak, nonatomic) IBOutlet UILabel *outputLabel; @@ -155,7 +156,7 @@ static NSString * const kHostAddress = @"localhost:50051"; rectangle.hi.longitude = -745E6; NSLog(@"Looking for features between %@ and %@", rectangle.lo, rectangle.hi); - [service listFeaturesWithRequest:rectangle + [_service listFeaturesWithRequest:rectangle eventHandler:^(BOOL done, RTGFeature *response, NSError *error) { if (response) { NSString *str =[NSString stringWithFormat:@"%@\nFound feature at %@ called %@.", self.outputLabel.text, response.location, response.name]; @@ -172,7 +173,7 @@ static NSString * const kHostAddress = @"localhost:50051"; - (void)viewDidLoad { [super viewDidLoad]; - service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; + _service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; } - (void)viewDidAppear:(BOOL)animated { @@ -193,7 +194,7 @@ static NSString * const kHostAddress = @"localhost:50051"; * server. */ @interface RecordRouteViewController : UIViewController { - RTGRouteGuide *service; + RTGRouteGuide *_service; } @property (weak, nonatomic) IBOutlet UILabel *outputLabel; @@ -217,7 +218,7 @@ static NSString * const kHostAddress = @"localhost:50051"; return location; }]; - [service recordRouteWithRequestsWriter:locations + [_service recordRouteWithRequestsWriter:locations handler:^(RTGRouteSummary *response, NSError *error) { if (response) { NSString *str =[NSString stringWithFormat: @@ -241,7 +242,7 @@ static NSString * const kHostAddress = @"localhost:50051"; - (void)viewDidLoad { [super viewDidLoad]; - service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; + _service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; } - (void)viewDidAppear:(BOOL)animated { @@ -261,7 +262,7 @@ static NSString * const kHostAddress = @"localhost:50051"; * the server. */ @interface RouteChatViewController : UIViewController { - RTGRouteGuide *service; + RTGRouteGuide *_service; } @property (weak, nonatomic) IBOutlet UILabel *outputLabel; @@ -279,7 +280,7 @@ static NSString * const kHostAddress = @"localhost:50051"; return note; }]; - [service routeChatWithRequestsWriter:notesWriter + [_service routeChatWithRequestsWriter:notesWriter eventHandler:^(BOOL done, RTGRouteNote *note, NSError *error) { if (note) { NSString *str =[NSString stringWithFormat:@"%@\nGot message %@ at %@", @@ -300,7 +301,7 @@ static NSString * const kHostAddress = @"localhost:50051"; - (void)viewDidLoad { [super viewDidLoad]; - service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; + _service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; } - (void)viewDidAppear:(BOOL)animated { From d139da8b9a9c482a83b23c926690f01b49025640 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Thu, 23 Jun 2016 10:37:21 -0700 Subject: [PATCH 121/215] Remove unnecessary changes --- tools/run_tests/run_tests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 2a404df0d41..532f7187408 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -425,7 +425,7 @@ class PythonLanguage(object): return [] def build_steps(self): - return [['tools/run_tests/build_python.sh', tox_env] + return [['tools/run_tests/build_python.sh', tox_env] for tox_env in self._tox_envs] def post_tests_steps(self): From 83a6a828b047cc24922a6d143e30f4ab6eb63239 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Thu, 23 Jun 2016 11:33:13 -0700 Subject: [PATCH 122/215] Fixed error messages for C++ interop client. --- test/cpp/interop/interop_client.cc | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/test/cpp/interop/interop_client.cc b/test/cpp/interop/interop_client.cc index 89f841dbe96..ec0729d9c5d 100644 --- a/test/cpp/interop/interop_client.cc +++ b/test/cpp/interop/interop_client.cc @@ -451,7 +451,7 @@ bool InteropClient::DoResponseStreaming() { // most likely due to connection failure. gpr_log(GPR_ERROR, "DoResponseStreaming(): Read fewer streams (%d) than " - "response_stream_sizes.size() (%" PRIuPTR ")", + "response_stream_sizes.size() (%zu)", i, response_stream_sizes.size()); return TransientFailureOrAbort(); } @@ -576,11 +576,10 @@ bool InteropClient::DoServerCompressedStreaming() { if (k < sizes.size()) { // stream->Read() failed before reading all the expected messages. This // is most likely due to a connection failure. - gpr_log(GPR_ERROR, "%s(): Responses read (k=%" PRIuPTR - ") is " - "less than the expected messages (i.e " - "response_stream_sizes.size() (%" PRIuPTR ")).", - __func__, k, response_stream_sizes.size()); + gpr_log(GPR_ERROR, + "%s(): Responses read (k=%zu) is less than the expected number of " + "messages (%zu).", + __func__, k, sizes.size()); return TransientFailureOrAbort(); } @@ -665,7 +664,7 @@ bool InteropClient::DoHalfDuplex() { // most likely due to a connection failure gpr_log(GPR_ERROR, "DoHalfDuplex(): Responses read (i=%d) are less than the expected " - "number of messages response_stream_sizes.size() (%" PRIuPTR ")", + "number of messages response_stream_sizes.size() (%zu)", i, response_stream_sizes.size()); return TransientFailureOrAbort(); } From a4b34c290d5f91866a3eff2f9793c00eb12f16f0 Mon Sep 17 00:00:00 2001 From: Ken Payson Date: Wed, 22 Jun 2016 15:31:15 -0700 Subject: [PATCH 123/215] Changed %lld to use mingw supported PRId64 --- src/core/ext/client_config/channel_connectivity.c | 6 +++--- src/core/lib/profiling/basic_timers.c | 4 ++-- .../lib/security/credentials/jwt/jwt_credentials.c | 4 ++-- src/core/lib/surface/channel.c | 10 +++++----- src/core/lib/surface/completion_queue.c | 8 ++++---- src/core/lib/transport/transport_op_string.c | 4 ++-- 6 files changed, 18 insertions(+), 18 deletions(-) diff --git a/src/core/ext/client_config/channel_connectivity.c b/src/core/ext/client_config/channel_connectivity.c index 20c01a9a7cf..2fd8e0e1232 100644 --- a/src/core/ext/client_config/channel_connectivity.c +++ b/src/core/ext/client_config/channel_connectivity.c @@ -189,10 +189,10 @@ void grpc_channel_watch_connectivity_state( GRPC_API_TRACE( "grpc_channel_watch_connectivity_state(" "channel=%p, last_observed_state=%d, " - "deadline=gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " "cq=%p, tag=%p)", - 7, (channel, (int)last_observed_state, (long long)deadline.tv_sec, - (int)deadline.tv_nsec, (int)deadline.clock_type, cq, tag)); + 7, (channel, (int)last_observed_state, deadline.tv_sec, + deadline.tv_nsec, (int)deadline.clock_type, cq, tag)); grpc_cq_begin_op(cq, tag); diff --git a/src/core/lib/profiling/basic_timers.c b/src/core/lib/profiling/basic_timers.c index 50082cd7ee4..ee464f310f4 100644 --- a/src/core/lib/profiling/basic_timers.c +++ b/src/core/lib/profiling/basic_timers.c @@ -141,9 +141,9 @@ static void write_log(gpr_timer_log *log) { entry->tm = gpr_time_0(entry->tm.clock_type); } fprintf(output_file, - "{\"t\": %lld.%09d, \"thd\": \"%d\", \"type\": \"%c\", \"tag\": " + "{\"t\": %"PRId64".%09d, \"thd\": \"%d\", \"type\": \"%c\", \"tag\": " "\"%s\", \"file\": \"%s\", \"line\": %d, \"imp\": %d}\n", - (long long)entry->tm.tv_sec, (int)entry->tm.tv_nsec, entry->thd, + entry->tm.tv_sec, entry->tm.tv_nsec, entry->thd, entry->type, entry->tagstr, entry->file, entry->line, entry->important); } diff --git a/src/core/lib/security/credentials/jwt/jwt_credentials.c b/src/core/lib/security/credentials/jwt/jwt_credentials.c index 973fb75eaab..f4dd9208610 100644 --- a/src/core/lib/security/credentials/jwt/jwt_credentials.c +++ b/src/core/lib/security/credentials/jwt/jwt_credentials.c @@ -149,10 +149,10 @@ grpc_call_credentials *grpc_service_account_jwt_access_credentials_create( "grpc_service_account_jwt_access_credentials_create(" "json_key=%s, " "token_lifetime=" - "gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " + "gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", 5, - (json_key, (long long)token_lifetime.tv_sec, (int)token_lifetime.tv_nsec, + (json_key, token_lifetime.tv_sec, token_lifetime.tv_nsec, (int)token_lifetime.clock_type, reserved)); GPR_ASSERT(reserved == NULL); return grpc_service_account_jwt_access_credentials_create_from_auth_json_key( diff --git a/src/core/lib/surface/channel.c b/src/core/lib/surface/channel.c index 8936e7164f4..952a769de1d 100644 --- a/src/core/lib/surface/channel.c +++ b/src/core/lib/surface/channel.c @@ -223,10 +223,10 @@ grpc_call *grpc_channel_create_call(grpc_channel *channel, "grpc_channel_create_call(" "channel=%p, parent_call=%p, propagation_mask=%x, cq=%p, method=%s, " "host=%s, " - "deadline=gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", 10, (channel, parent_call, (unsigned)propagation_mask, cq, method, host, - (long long)deadline.tv_sec, (int)deadline.tv_nsec, + deadline.tv_sec, deadline.tv_nsec, (int)deadline.clock_type, reserved)); GPR_ASSERT(!reserved); return grpc_channel_create_call_internal( @@ -282,11 +282,11 @@ grpc_call *grpc_channel_create_registered_call( "grpc_channel_create_registered_call(" "channel=%p, parent_call=%p, propagation_mask=%x, completion_queue=%p, " "registered_call_handle=%p, " - "deadline=gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", 9, (channel, parent_call, (unsigned)propagation_mask, completion_queue, - registered_call_handle, (long long)deadline.tv_sec, - (int)deadline.tv_nsec, (int)deadline.clock_type, reserved)); + registered_call_handle, deadline.tv_sec, + deadline.tv_nsec, (int)deadline.clock_type, reserved)); GPR_ASSERT(!reserved); return grpc_channel_create_call_internal( channel, parent_call, propagation_mask, completion_queue, NULL, diff --git a/src/core/lib/surface/completion_queue.c b/src/core/lib/surface/completion_queue.c index 2cc6aa74e02..aa35ae02fe2 100644 --- a/src/core/lib/surface/completion_queue.c +++ b/src/core/lib/surface/completion_queue.c @@ -316,9 +316,9 @@ grpc_event grpc_completion_queue_next(grpc_completion_queue *cc, GRPC_API_TRACE( "grpc_completion_queue_next(" "cc=%p, " - "deadline=gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", - 5, (cc, (long long)deadline.tv_sec, (int)deadline.tv_nsec, + 5, (cc, deadline.tv_sec, deadline.tv_nsec, (int)deadline.clock_type, reserved)); GPR_ASSERT(!reserved); @@ -428,9 +428,9 @@ grpc_event grpc_completion_queue_pluck(grpc_completion_queue *cc, void *tag, GRPC_API_TRACE( "grpc_completion_queue_pluck(" "cc=%p, tag=%p, " - "deadline=gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", - 6, (cc, tag, (long long)deadline.tv_sec, (int)deadline.tv_nsec, + 6, (cc, tag, deadline.tv_sec, deadline.tv_nsec, (int)deadline.clock_type, reserved)); GPR_ASSERT(!reserved); diff --git a/src/core/lib/transport/transport_op_string.c b/src/core/lib/transport/transport_op_string.c index df04c611270..d8fae7805e4 100644 --- a/src/core/lib/transport/transport_op_string.c +++ b/src/core/lib/transport/transport_op_string.c @@ -63,8 +63,8 @@ static void put_metadata_list(gpr_strvec *b, grpc_metadata_batch md) { } if (gpr_time_cmp(md.deadline, gpr_inf_future(md.deadline.clock_type)) != 0) { char *tmp; - gpr_asprintf(&tmp, " deadline=%lld.%09d", (long long)md.deadline.tv_sec, - (int)md.deadline.tv_nsec); + gpr_asprintf(&tmp, " deadline=%"PRId64".%09d", md.deadline.tv_sec, + md.deadline.tv_nsec); gpr_strvec_add(b, tmp); } } From fe754b4e996415300ff29b6e4c378d3d5704852a Mon Sep 17 00:00:00 2001 From: Ken Payson Date: Thu, 23 Jun 2016 12:32:07 -0700 Subject: [PATCH 124/215] Use GRPC_PYTHON_CFLAGS/GRPC_PYTHON_LDFLAGS in setup.py. This is needed for building grpcio with mingw, see https://github.com/grpc/grpc/pull/7012. --- setup.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/setup.py b/setup.py index f73501b8b35..7e625b937a7 100644 --- a/setup.py +++ b/setup.py @@ -31,6 +31,7 @@ import os import os.path +import shlex import shutil import sys import sysconfig @@ -99,8 +100,9 @@ if not "win32" in sys.platform: DEFINE_MACROS = (('OPENSSL_NO_ASM', 1), ('_WIN32_WINNT', 0x600), ('GPR_BACKWARDS_COMPATIBILITY_MODE', 1),) -LDFLAGS = () -CFLAGS = () +LDFLAGS = shlex.split(os.environ.get('GRPC_PYTHON_LDFLAGS', '')) +CFLAGS = shlex.split(os.environ.get('GRPC_PYTHON_CFLAGS', '')) + if "linux" in sys.platform: LDFLAGS += ('-Wl,-wrap,memcpy',) if "linux" in sys.platform or "darwin" in sys.platform: From 51d4f0194963e60cca2e28c08a53070f2135b4f0 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Thu, 23 Jun 2016 13:01:43 -0700 Subject: [PATCH 125/215] prevent race between grpcsharp_server_request_call and grpc_completion_queue_shutdown --- .../Grpc.Core/Internal/ServerSafeHandle.cs | 23 +++++++++++++------ 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/src/csharp/Grpc.Core/Internal/ServerSafeHandle.cs b/src/csharp/Grpc.Core/Internal/ServerSafeHandle.cs index 85813027064..014a8db78f2 100644 --- a/src/csharp/Grpc.Core/Internal/ServerSafeHandle.cs +++ b/src/csharp/Grpc.Core/Internal/ServerSafeHandle.cs @@ -54,7 +54,10 @@ namespace Grpc.Core.Internal public void RegisterCompletionQueue(CompletionQueueSafeHandle cq) { - Native.grpcsharp_server_register_completion_queue(this, cq); + using (cq.NewScope()) + { + Native.grpcsharp_server_register_completion_queue(this, cq); + } } public int AddInsecurePort(string addr) @@ -74,16 +77,22 @@ namespace Grpc.Core.Internal public void ShutdownAndNotify(BatchCompletionDelegate callback, CompletionQueueSafeHandle completionQueue) { - var ctx = BatchContextSafeHandle.Create(); - completionQueue.CompletionRegistry.RegisterBatchCompletion(ctx, callback); - Native.grpcsharp_server_shutdown_and_notify_callback(this, completionQueue, ctx); + using (completionQueue.NewScope()) + { + var ctx = BatchContextSafeHandle.Create(); + completionQueue.CompletionRegistry.RegisterBatchCompletion(ctx, callback); + Native.grpcsharp_server_shutdown_and_notify_callback(this, completionQueue, ctx); + } } public void RequestCall(BatchCompletionDelegate callback, CompletionQueueSafeHandle completionQueue) { - var ctx = BatchContextSafeHandle.Create(); - completionQueue.CompletionRegistry.RegisterBatchCompletion(ctx, callback); - Native.grpcsharp_server_request_call(this, completionQueue, ctx).CheckOk(); + using (completionQueue.NewScope()) + { + var ctx = BatchContextSafeHandle.Create(); + completionQueue.CompletionRegistry.RegisterBatchCompletion(ctx, callback); + Native.grpcsharp_server_request_call(this, completionQueue, ctx).CheckOk(); + } } protected override bool ReleaseHandle() From abd285aed813ff55329e76a9dc19eb3a1bc86166 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 23 Jun 2016 13:30:19 -0700 Subject: [PATCH 126/215] Added missing todo and moved _service To Implementation --- .../objective-c/route_guide/ViewControllers.m | 28 ++++++++++--------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/examples/objective-c/route_guide/ViewControllers.m b/examples/objective-c/route_guide/ViewControllers.m index a5c03e9cfca..26ca9d6220b 100644 --- a/examples/objective-c/route_guide/ViewControllers.m +++ b/examples/objective-c/route_guide/ViewControllers.m @@ -80,13 +80,14 @@ static NSString * const kHostAddress = @"localhost:50051"; * Run the getFeature demo. Calls getFeature with a point known to have a feature and a point known * not to have a feature. */ -@interface GetFeatureViewController : UIViewController { - RTGRouteGuide *_service; -} +@interface GetFeatureViewController : UIViewController + @property (weak, nonatomic) IBOutlet UILabel *outputLabel; + @end @implementation GetFeatureViewController +RTGRouteGuide *_service; - (void)execRequest { void (^handler)(RTGFeature *response, NSError *error) = ^(RTGFeature *response, NSError *error) { @@ -139,14 +140,14 @@ static NSString * const kHostAddress = @"localhost:50051"; * Run the listFeatures demo. Calls listFeatures with a rectangle containing all of the features in * the pre-generated database. Prints each response as it comes in. */ -@interface ListFeaturesViewController : UIViewController { - RTGRouteGuide *_service; -} +@interface ListFeaturesViewController : UIViewController + @property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @implementation ListFeaturesViewController +RTGRouteGuide *_service; - (void)execRequest { RTGRectangle *rectangle = [RTGRectangle message]; @@ -193,14 +194,14 @@ static NSString * const kHostAddress = @"localhost:50051"; * database with a variable delay in between. Prints the statistics when they are sent from the * server. */ -@interface RecordRouteViewController : UIViewController { - RTGRouteGuide *_service; -} +@interface RecordRouteViewController : UIViewController + @property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @implementation RecordRouteViewController +RTGRouteGuide *_service; - (void)execRequest { NSString *dataBasePath = [NSBundle.mainBundle pathForResource:@"route_guide_db" @@ -233,7 +234,7 @@ static NSString * const kHostAddress = @"localhost:50051"; NSLog(@"It took %i seconds", response.elapsedTime); } else { NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.outputLabel.text, error]; - self.outputLabel.text = str; + self.outputLabel.text = str; NSLog(@"RPC error: %@", error); } }]; @@ -261,14 +262,14 @@ static NSString * const kHostAddress = @"localhost:50051"; * Run the routeChat demo. Send some chat messages, and print any chat messages that are sent from * the server. */ -@interface RouteChatViewController : UIViewController { - RTGRouteGuide *_service; -} +@interface RouteChatViewController : UIViewController + @property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @implementation RouteChatViewController +RTGRouteGuide *_service; - (void)execRequest { NSArray *notes = @[[RTGRouteNote noteWithMessage:@"First message" latitude:0 longitude:0], @@ -305,6 +306,7 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { + // TODO(makarandd): Set these properties through UI builder self.outputLabel.text = @"RPC log:"; self.outputLabel.numberOfLines = 0; self.outputLabel.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; From f0f70a8a68c107f9fcff2dd35867fd762c2c13d6 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 23 Jun 2016 13:55:06 -0700 Subject: [PATCH 127/215] Make transport-level errors be reflected in status messages on calls Allows us to eliminate logging those errors by default (since they are explicitly passed up to the application). Required plumbing errors through the stack a little more deeply than we had previously. --- .../client_config/subchannel_call_holder.c | 7 +- .../chttp2/transport/chttp2_transport.c | 229 +++++++++++------- .../ext/transport/chttp2/transport/internal.h | 3 - src/core/lib/channel/channel_stack.c | 2 +- src/core/lib/iomgr/error.c | 14 ++ src/core/lib/iomgr/error.h | 16 +- .../security/transport/client_auth_filter.c | 3 +- src/core/lib/surface/call.c | 105 ++++---- src/core/lib/surface/completion_queue.c | 2 +- src/core/lib/transport/transport.c | 65 ++--- src/core/lib/transport/transport.h | 9 +- src/core/lib/transport/transport_op_string.c | 15 +- 12 files changed, 279 insertions(+), 191 deletions(-) diff --git a/src/core/ext/client_config/subchannel_call_holder.c b/src/core/ext/client_config/subchannel_call_holder.c index e31800edd98..b96a0ad0937 100644 --- a/src/core/ext/client_config/subchannel_call_holder.c +++ b/src/core/ext/client_config/subchannel_call_holder.c @@ -120,16 +120,13 @@ retry: return; } /* if this is a cancellation, then we can raise our cancelled flag */ - if (op->cancel_with_status != GRPC_STATUS_OK) { + if (op->cancel_error != GRPC_ERROR_NONE) { if (!gpr_atm_rel_cas(&holder->subchannel_call, 0, 1)) { goto retry; } else { switch (holder->creation_phase) { case GRPC_SUBCHANNEL_CALL_HOLDER_NOT_CREATING: - fail_locked(exec_ctx, holder, - grpc_error_set_int(GRPC_ERROR_CREATE("Cancelled"), - GRPC_ERROR_INT_GRPC_STATUS, - op->cancel_with_status)); + fail_locked(exec_ctx, holder, GRPC_ERROR_REF(op->cancel_error)); break; case GRPC_SUBCHANNEL_CALL_HOLDER_PICKING_SUBCHANNEL: holder->pick_subchannel(exec_ctx, holder->pick_subchannel_arg, NULL, diff --git a/src/core/ext/transport/chttp2/transport/chttp2_transport.c b/src/core/ext/transport/chttp2/transport/chttp2_transport.c index 9c99ff883af..f34ab98b036 100644 --- a/src/core/ext/transport/chttp2/transport/chttp2_transport.c +++ b/src/core/ext/transport/chttp2/transport/chttp2_transport.c @@ -106,14 +106,12 @@ static void perform_stream_op_locked(grpc_exec_ctx *exec_ctx, static void cancel_from_api(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport_global *transport_global, grpc_chttp2_stream_global *stream_global, - grpc_status_code status, - gpr_slice *optional_message); + grpc_error *error); static void close_from_api(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport_global *transport_global, grpc_chttp2_stream_global *stream_global, - grpc_status_code status, - gpr_slice *optional_message); + grpc_error *error); /** Add endpoint from this transport to pollset */ static void add_to_pollset_locked(grpc_exec_ctx *exec_ctx, @@ -163,8 +161,6 @@ static void destruct_transport(grpc_exec_ctx *exec_ctx, GPR_ASSERT(t->ep == NULL); - gpr_slice_unref(t->optional_drop_message); - gpr_slice_buffer_destroy(&t->global.qbuf); gpr_slice_buffer_destroy(&t->writing.outbuf); @@ -266,7 +262,6 @@ static void init_transport(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport *t, is_client ? GRPC_DTS_FH_0 : GRPC_DTS_CLIENT_PREFIX_0; t->parsing.is_first_frame = true; t->writing.is_client = is_client; - t->optional_drop_message = gpr_empty_slice(); grpc_connectivity_state_init( &t->channel_callback.state_tracker, GRPC_CHANNEL_READY, is_client ? "client_transport" : "server_transport"); @@ -876,7 +871,9 @@ static void maybe_start_some_streams( grpc_chttp2_list_pop_waiting_for_concurrency(transport_global, &stream_global)) { cancel_from_api(exec_ctx, transport_global, stream_global, - GRPC_STATUS_UNAVAILABLE, NULL); + grpc_error_set_int( + GRPC_ERROR_CREATE("Stream IDs exhausted"), + GRPC_ERROR_INT_GRPC_STATUS, GRPC_STATUS_UNAVAILABLE)); } } @@ -958,14 +955,14 @@ static void perform_stream_op_locked(grpc_exec_ctx *exec_ctx, on_complete->next_data.scratch |= CLOSURE_BARRIER_STATS_BIT; } - if (op->cancel_with_status != GRPC_STATUS_OK) { + if (op->cancel_error != GRPC_ERROR_NONE) { cancel_from_api(exec_ctx, transport_global, stream_global, - op->cancel_with_status, op->optional_close_message); + GRPC_ERROR_REF(op->cancel_error)); } - if (op->close_with_status != GRPC_STATUS_OK) { + if (op->close_error != GRPC_ERROR_NONE) { close_from_api(exec_ctx, transport_global, stream_global, - op->close_with_status, op->optional_close_message); + GRPC_ERROR_REF(op->close_error)); } if (op->send_initial_metadata != NULL) { @@ -979,12 +976,16 @@ static void perform_stream_op_locked(grpc_exec_ctx *exec_ctx, transport_global->settings[GRPC_PEER_SETTINGS] [GRPC_CHTTP2_SETTINGS_MAX_HEADER_LIST_SIZE]; if (metadata_size > metadata_peer_limit) { - gpr_log(GPR_DEBUG, - "to-be-sent initial metadata size exceeds peer limit " - "(%" PRIuPTR " vs. %" PRIuPTR ")", - metadata_size, metadata_peer_limit); - cancel_from_api(exec_ctx, transport_global, stream_global, - GRPC_STATUS_RESOURCE_EXHAUSTED, NULL); + cancel_from_api( + exec_ctx, transport_global, stream_global, + grpc_error_set_int( + grpc_error_set_int( + grpc_error_set_int( + GRPC_ERROR_CREATE("to-be-sent initial metadata size " + "exceeds peer limit"), + GRPC_ERROR_INT_SIZE, (intptr_t)metadata_size), + GRPC_ERROR_INT_LIMIT, (intptr_t)metadata_peer_limit), + GRPC_ERROR_INT_GRPC_STATUS, GRPC_STATUS_RESOURCE_EXHAUSTED)); } else { if (contains_non_ok_status(transport_global, op->send_initial_metadata)) { stream_global->seen_error = true; @@ -1038,12 +1039,16 @@ static void perform_stream_op_locked(grpc_exec_ctx *exec_ctx, transport_global->settings[GRPC_PEER_SETTINGS] [GRPC_CHTTP2_SETTINGS_MAX_HEADER_LIST_SIZE]; if (metadata_size > metadata_peer_limit) { - gpr_log(GPR_DEBUG, - "to-be-sent trailing metadata size exceeds peer limit " - "(%" PRIuPTR " vs. %" PRIuPTR ")", - metadata_size, metadata_peer_limit); - cancel_from_api(exec_ctx, transport_global, stream_global, - GRPC_STATUS_RESOURCE_EXHAUSTED, NULL); + cancel_from_api( + exec_ctx, transport_global, stream_global, + grpc_error_set_int( + grpc_error_set_int( + grpc_error_set_int( + GRPC_ERROR_CREATE("to-be-sent trailing metadata size " + "exceeds peer limit"), + GRPC_ERROR_INT_SIZE, (intptr_t)metadata_size), + GRPC_ERROR_INT_LIMIT, (intptr_t)metadata_peer_limit), + GRPC_ERROR_INT_GRPC_STATUS, GRPC_STATUS_RESOURCE_EXHAUSTED)); } else { if (contains_non_ok_status(transport_global, op->send_trailing_metadata)) { @@ -1235,8 +1240,12 @@ static void check_read_ops(grpc_exec_ctx *exec_ctx, incoming_byte_stream_destroy_locked(exec_ctx, NULL, NULL, bs); } if (stream_global->exceeded_metadata_size) { - cancel_from_api(exec_ctx, transport_global, stream_global, - GRPC_STATUS_RESOURCE_EXHAUSTED, NULL); + cancel_from_api( + exec_ctx, transport_global, stream_global, + grpc_error_set_int( + GRPC_ERROR_CREATE( + "received initial metadata size exceeds limit"), + GRPC_ERROR_INT_GRPC_STATUS, GRPC_STATUS_RESOURCE_EXHAUSTED)); } } grpc_chttp2_incoming_metadata_buffer_publish( @@ -1275,8 +1284,12 @@ static void check_read_ops(grpc_exec_ctx *exec_ctx, incoming_byte_stream_destroy_locked(exec_ctx, NULL, NULL, bs); } if (stream_global->exceeded_metadata_size) { - cancel_from_api(exec_ctx, transport_global, stream_global, - GRPC_STATUS_RESOURCE_EXHAUSTED, NULL); + cancel_from_api( + exec_ctx, transport_global, stream_global, + grpc_error_set_int( + GRPC_ERROR_CREATE( + "received trailing metadata size exceeds limit"), + GRPC_ERROR_INT_GRPC_STATUS, GRPC_STATUS_RESOURCE_EXHAUSTED)); } } if (stream_global->all_incoming_byte_streams_finished) { @@ -1340,35 +1353,67 @@ static void remove_stream(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport *t, GRPC_ERROR_UNREF(error); } +static void status_codes_from_error(grpc_error *error, + grpc_chttp2_error_code *http2_error, + grpc_status_code *grpc_status) { + intptr_t ip_http; + intptr_t ip_grpc; + bool have_http = + grpc_error_get_int(error, GRPC_ERROR_INT_HTTP2_ERROR, &ip_http); + bool have_grpc = + grpc_error_get_int(error, GRPC_ERROR_INT_GRPC_STATUS, &ip_grpc); + if (have_http) { + *http2_error = (grpc_chttp2_error_code)ip_http; + } else if (have_grpc) { + *http2_error = + grpc_chttp2_grpc_status_to_http2_error((grpc_status_code)ip_grpc); + } else { + *http2_error = GRPC_CHTTP2_INTERNAL_ERROR; + } + if (have_grpc) { + *grpc_status = (grpc_status_code)ip_grpc; + } else if (have_http) { + *grpc_status = + grpc_chttp2_http2_error_to_grpc_status((grpc_chttp2_error_code)ip_http); + } else { + *grpc_status = GRPC_STATUS_INTERNAL; + } +} + static void cancel_from_api(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport_global *transport_global, grpc_chttp2_stream_global *stream_global, - grpc_status_code status, - gpr_slice *optional_message) { + grpc_error *due_to_error) { if (!stream_global->read_closed || !stream_global->write_closed) { + grpc_status_code grpc_status; + grpc_chttp2_error_code http_error; + status_codes_from_error(due_to_error, &http_error, &grpc_status); + if (stream_global->id != 0) { gpr_slice_buffer_add( &transport_global->qbuf, - grpc_chttp2_rst_stream_create( - stream_global->id, - (uint32_t)grpc_chttp2_grpc_status_to_http2_error(status), - &stream_global->stats.outgoing)); + grpc_chttp2_rst_stream_create(stream_global->id, (uint32_t)http_error, + &stream_global->stats.outgoing)); } - if (optional_message) { - gpr_slice_ref(*optional_message); + const char *msg = + grpc_error_get_str(due_to_error, GRPC_ERROR_STR_GRPC_MESSAGE); + bool free_msg = false; + if (msg == NULL) { + free_msg = true; + msg = grpc_error_string(due_to_error); } - grpc_chttp2_fake_status(exec_ctx, transport_global, stream_global, status, - optional_message); + gpr_slice msg_slice = gpr_slice_from_copied_string(msg); + grpc_chttp2_fake_status(exec_ctx, transport_global, stream_global, + grpc_status, &msg_slice); + if (free_msg) grpc_error_free_string(msg); } - if (status != GRPC_STATUS_OK && !stream_global->seen_error) { + if (due_to_error != GRPC_ERROR_NONE && !stream_global->seen_error) { stream_global->seen_error = true; grpc_chttp2_list_add_check_read_ops(transport_global, stream_global); } - grpc_chttp2_mark_stream_closed( - exec_ctx, transport_global, stream_global, 1, 1, - grpc_error_set_int(GRPC_ERROR_CREATE("Cancelled"), - GRPC_ERROR_INT_GRPC_STATUS, status)); + grpc_chttp2_mark_stream_closed(exec_ctx, transport_global, stream_global, 1, + 1, due_to_error); } void grpc_chttp2_fake_status(grpc_exec_ctx *exec_ctx, @@ -1469,15 +1514,17 @@ void grpc_chttp2_mark_stream_closed( static void close_from_api(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport_global *transport_global, grpc_chttp2_stream_global *stream_global, - grpc_status_code status, - gpr_slice *optional_message) { + grpc_error *error) { gpr_slice hdr; gpr_slice status_hdr; gpr_slice message_pfx; uint8_t *p; uint32_t len = 0; + grpc_status_code grpc_status; + grpc_chttp2_error_code http_error; + status_codes_from_error(error, &http_error, &grpc_status); - GPR_ASSERT(status >= 0 && (int)status < 100); + GPR_ASSERT(grpc_status >= 0 && (int)grpc_status < 100); if (stream_global->id != 0 && !transport_global->is_client) { /* Hand roll a header block. @@ -1487,7 +1534,7 @@ static void close_from_api(grpc_exec_ctx *exec_ctx, time we got around to sending this, so instead we ignore HPACK compression and just write the uncompressed bytes onto the wire. */ - status_hdr = gpr_slice_malloc(15 + (status >= 10)); + status_hdr = gpr_slice_malloc(15 + (grpc_status >= 10)); p = GPR_SLICE_START_PTR(status_hdr); *p++ = 0x40; /* literal header */ *p++ = 11; /* len(grpc-status) */ @@ -1502,19 +1549,23 @@ static void close_from_api(grpc_exec_ctx *exec_ctx, *p++ = 't'; *p++ = 'u'; *p++ = 's'; - if (status < 10) { + if (grpc_status < 10) { *p++ = 1; - *p++ = (uint8_t)('0' + status); + *p++ = (uint8_t)('0' + grpc_status); } else { *p++ = 2; - *p++ = (uint8_t)('0' + (status / 10)); - *p++ = (uint8_t)('0' + (status % 10)); + *p++ = (uint8_t)('0' + (grpc_status / 10)); + *p++ = (uint8_t)('0' + (grpc_status % 10)); } GPR_ASSERT(p == GPR_SLICE_END_PTR(status_hdr)); len += (uint32_t)GPR_SLICE_LENGTH(status_hdr); - if (optional_message) { - GPR_ASSERT(GPR_SLICE_LENGTH(*optional_message) < 127); + const char *optional_message = + grpc_error_get_str(error, GRPC_ERROR_STR_GRPC_MESSAGE); + + if (optional_message != NULL) { + size_t msg_len = strlen(optional_message); + GPR_ASSERT(msg_len < 127); message_pfx = gpr_slice_malloc(15); p = GPR_SLICE_START_PTR(message_pfx); *p++ = 0x40; @@ -1531,10 +1582,10 @@ static void close_from_api(grpc_exec_ctx *exec_ctx, *p++ = 'a'; *p++ = 'g'; *p++ = 'e'; - *p++ = (uint8_t)GPR_SLICE_LENGTH(*optional_message); + *p++ = (uint8_t)msg_len; GPR_ASSERT(p == GPR_SLICE_END_PTR(message_pfx)); len += (uint32_t)GPR_SLICE_LENGTH(message_pfx); - len += (uint32_t)GPR_SLICE_LENGTH(*optional_message); + len += (uint32_t)msg_len; } hdr = gpr_slice_malloc(9); @@ -1555,53 +1606,54 @@ static void close_from_api(grpc_exec_ctx *exec_ctx, if (optional_message) { gpr_slice_buffer_add(&transport_global->qbuf, message_pfx); gpr_slice_buffer_add(&transport_global->qbuf, - gpr_slice_ref(*optional_message)); + gpr_slice_from_copied_string(optional_message)); } gpr_slice_buffer_add( &transport_global->qbuf, grpc_chttp2_rst_stream_create(stream_global->id, GRPC_CHTTP2_NO_ERROR, &stream_global->stats.outgoing)); - - if (optional_message) { - gpr_slice_ref(*optional_message); - } } - grpc_chttp2_fake_status(exec_ctx, transport_global, stream_global, status, - optional_message); - grpc_error *err = GRPC_ERROR_CREATE("Stream closed"); - err = grpc_error_set_int(err, GRPC_ERROR_INT_GRPC_STATUS, status); - if (optional_message) { - char *str = - gpr_dump_slice(*optional_message, GPR_DUMP_HEX | GPR_DUMP_ASCII); - err = grpc_error_set_str(err, GRPC_ERROR_STR_GRPC_MESSAGE, str); - gpr_free(str); + const char *msg = grpc_error_get_str(error, GRPC_ERROR_STR_GRPC_MESSAGE); + bool free_msg = false; + if (msg == NULL) { + free_msg = true; + msg = grpc_error_string(error); } + gpr_slice msg_slice = gpr_slice_from_copied_string(msg); + grpc_chttp2_fake_status(exec_ctx, transport_global, stream_global, + grpc_status, &msg_slice); + if (free_msg) grpc_error_free_string(msg); + grpc_chttp2_mark_stream_closed(exec_ctx, transport_global, stream_global, 1, - 1, err); + 1, error); } +typedef struct { + grpc_exec_ctx *exec_ctx; + grpc_error *error; +} cancel_stream_cb_args; + static void cancel_stream_cb(grpc_chttp2_transport_global *transport_global, void *user_data, grpc_chttp2_stream_global *stream_global) { - grpc_chttp2_transport *transport = TRANSPORT_FROM_GLOBAL(transport_global); - cancel_from_api(user_data, transport_global, stream_global, - GRPC_STATUS_UNAVAILABLE, - GPR_SLICE_IS_EMPTY(transport->optional_drop_message) - ? NULL - : &transport->optional_drop_message); + cancel_stream_cb_args *args = user_data; + cancel_from_api(args->exec_ctx, transport_global, stream_global, + GRPC_ERROR_REF(args->error)); } -static void end_all_the_calls(grpc_exec_ctx *exec_ctx, - grpc_chttp2_transport *t) { - grpc_chttp2_for_all_streams(&t->global, exec_ctx, cancel_stream_cb); +static void end_all_the_calls(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport *t, + grpc_error *error) { + cancel_stream_cb_args args = {exec_ctx, error}; + grpc_chttp2_for_all_streams(&t->global, &args, cancel_stream_cb); + GRPC_ERROR_UNREF(error); } static void drop_connection(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport *t, grpc_error *error) { - close_transport_locked(exec_ctx, t, error); - end_all_the_calls(exec_ctx, t); + close_transport_locked(exec_ctx, t, GRPC_ERROR_REF(error)); + end_all_the_calls(exec_ctx, t, error); } /** update window from a settings change */ @@ -1708,15 +1760,7 @@ static void parsing_action(grpc_exec_ctx *exec_ctx, void *arg, t->read_buffer.slices[i]); }; if (i != t->read_buffer.count) { - gpr_slice_unref(t->optional_drop_message); errors[2] = try_http_parsing(exec_ctx, t); - if (errors[2] != GRPC_ERROR_NONE) { - t->optional_drop_message = gpr_slice_from_copied_string( - "Connection dropped: received http1.x response"); - } else { - t->optional_drop_message = gpr_slice_from_copied_string( - "Connection dropped: received unparseable response"); - } } grpc_error *err = errors[0] == GRPC_ERROR_NONE && errors[1] == GRPC_ERROR_NONE && @@ -1784,6 +1828,10 @@ static void post_reading_action_locked(grpc_exec_ctx *exec_ctx, error = GRPC_ERROR_CREATE("Transport closed"); } if (error != GRPC_ERROR_NONE) { + if (!grpc_error_get_int(error, GRPC_ERROR_INT_GRPC_STATUS, NULL)) { + error = grpc_error_set_int(error, GRPC_ERROR_INT_GRPC_STATUS, + GRPC_STATUS_UNAVAILABLE); + } drop_connection(exec_ctx, t, GRPC_ERROR_REF(error)); t->endpoint_reading = 0; if (!t->executor.writing_active && t->ep) { @@ -1798,6 +1846,7 @@ static void post_reading_action_locked(grpc_exec_ctx *exec_ctx, prevent_endpoint_shutdown(t); } gpr_slice_buffer_reset_and_unref(&t->read_buffer); + GRPC_ERROR_UNREF(error); if (keep_reading) { grpc_endpoint_read(exec_ctx, t->ep, &t->read_buffer, &t->reading_action); @@ -1806,8 +1855,6 @@ static void post_reading_action_locked(grpc_exec_ctx *exec_ctx, } else { UNREF_TRANSPORT(exec_ctx, t, "reading_action"); } - - GRPC_LOG_IF_ERROR("close_transport", error); } /******************************************************************************* diff --git a/src/core/ext/transport/chttp2/transport/internal.h b/src/core/ext/transport/chttp2/transport/internal.h index d63170e350b..7e281f1b1a2 100644 --- a/src/core/ext/transport/chttp2/transport/internal.h +++ b/src/core/ext/transport/chttp2/transport/internal.h @@ -384,9 +384,6 @@ struct grpc_chttp2_transport { /** Transport op to be applied post-parsing */ grpc_transport_op *post_parsing_op; - - /** Message explaining the reason of dropping connection */ - gpr_slice optional_drop_message; }; typedef struct { diff --git a/src/core/lib/channel/channel_stack.c b/src/core/lib/channel/channel_stack.c index bbba85d80ba..42075b127bd 100644 --- a/src/core/lib/channel/channel_stack.c +++ b/src/core/lib/channel/channel_stack.c @@ -263,6 +263,6 @@ void grpc_call_element_send_cancel(grpc_exec_ctx *exec_ctx, grpc_call_element *cur_elem) { grpc_transport_stream_op op; memset(&op, 0, sizeof(op)); - op.cancel_with_status = GRPC_STATUS_CANCELLED; + op.cancel_error = GRPC_ERROR_CANCELLED; grpc_call_next_op(exec_ctx, cur_elem, &op); } diff --git a/src/core/lib/iomgr/error.c b/src/core/lib/iomgr/error.c index 540fb4fa7e4..da0d6972360 100644 --- a/src/core/lib/iomgr/error.c +++ b/src/core/lib/iomgr/error.c @@ -37,6 +37,7 @@ #include #include +#include #include #include #include @@ -115,6 +116,8 @@ static const char *error_int_name(grpc_error_ints key) { return "wsa_error"; case GRPC_ERROR_INT_HTTP_STATUS: return "http_status"; + case GRPC_ERROR_INT_LIMIT: + return "limit"; } GPR_UNREACHABLE_CODE(return "unknown"); } @@ -271,6 +274,12 @@ grpc_error *grpc_error_set_int(grpc_error *src, grpc_error_ints which, bool grpc_error_get_int(grpc_error *err, grpc_error_ints which, intptr_t *p) { void *pp; + if (is_special(err)) { + if (err == GRPC_ERROR_CANCELLED && which == GRPC_ERROR_INT_GRPC_STATUS) { + return GRPC_STATUS_CANCELLED; + } + return false; + } if (gpr_avl_maybe_get(err->ints, (void *)(uintptr_t)which, &pp)) { if (p != NULL) *p = (intptr_t)pp; return true; @@ -286,6 +295,11 @@ grpc_error *grpc_error_set_str(grpc_error *src, grpc_error_strs which, return new; } +const char *grpc_error_get_str(grpc_error *err, grpc_error_strs which) { + if (is_special(err)) return NULL; + return gpr_avl_get(err->strs, (void *)(uintptr_t)which); +} + grpc_error *grpc_error_add_child(grpc_error *src, grpc_error *child) { grpc_error *new = copy_error_and_unref(src); new->errs = gpr_avl_add(new->errs, (void *)(new->next_err++), child); diff --git a/src/core/lib/iomgr/error.h b/src/core/lib/iomgr/error.h index 69cdf3028e4..13f898e31ad 100644 --- a/src/core/lib/iomgr/error.h +++ b/src/core/lib/iomgr/error.h @@ -92,6 +92,8 @@ typedef enum { GRPC_ERROR_INT_FD, /// HTTP status (i.e. 404) GRPC_ERROR_INT_HTTP_STATUS, + /// context sensitive limit associated with the error + GRPC_ERROR_INT_LIMIT, } grpc_error_ints; typedef enum { @@ -163,23 +165,25 @@ void grpc_error_unref(grpc_error *err); #endif grpc_error *grpc_error_set_int(grpc_error *src, grpc_error_ints which, - intptr_t value); + intptr_t value) GRPC_MUST_USE_RESULT; bool grpc_error_get_int(grpc_error *error, grpc_error_ints which, intptr_t *p); grpc_error *grpc_error_set_time(grpc_error *src, grpc_error_times which, - gpr_timespec value); + gpr_timespec value) GRPC_MUST_USE_RESULT; grpc_error *grpc_error_set_str(grpc_error *src, grpc_error_strs which, - const char *value); + const char *value) GRPC_MUST_USE_RESULT; +const char *grpc_error_get_str(grpc_error *error, grpc_error_strs which); /// Add a child error: an error that is believed to have contributed to this /// error occurring. Allows root causing high level errors from lower level /// errors that contributed to them. -grpc_error *grpc_error_add_child(grpc_error *src, grpc_error *child); +grpc_error *grpc_error_add_child(grpc_error *src, + grpc_error *child) GRPC_MUST_USE_RESULT; grpc_error *grpc_os_error(const char *file, int line, int err, - const char *call_name); + const char *call_name) GRPC_MUST_USE_RESULT; /// create an error associated with errno!=0 (an 'operating system' error) #define GRPC_OS_ERROR(err, call_name) \ grpc_os_error(__FILE__, __LINE__, err, call_name) grpc_error *grpc_wsa_error(const char *file, int line, int err, - const char *call_name); + const char *call_name) GRPC_MUST_USE_RESULT; /// windows only: create an error associated with WSAGetLastError()!=0 #define GRPC_WSA_ERROR(err, call_name) \ grpc_wsa_error(__FILE__, __LINE__, err, call_name) diff --git a/src/core/lib/security/transport/client_auth_filter.c b/src/core/lib/security/transport/client_auth_filter.c index 76be2acd728..e053afc745c 100644 --- a/src/core/lib/security/transport/client_auth_filter.c +++ b/src/core/lib/security/transport/client_auth_filter.c @@ -220,8 +220,7 @@ static void auth_start_transport_op(grpc_exec_ctx *exec_ctx, grpc_linked_mdelem *l; grpc_client_security_context *sec_ctx = NULL; - if (calld->security_context_set == 0 && - op->cancel_with_status == GRPC_STATUS_OK) { + if (calld->security_context_set == 0 && op->cancel_error == GRPC_ERROR_NONE) { calld->security_context_set = 1; GPR_ASSERT(op->context); if (op->context[GRPC_CONTEXT_SECURITY].value == NULL) { diff --git a/src/core/lib/surface/call.c b/src/core/lib/surface/call.c index 04291b0ee0a..77c17a4975b 100644 --- a/src/core/lib/surface/call.c +++ b/src/core/lib/surface/call.c @@ -402,8 +402,50 @@ static void set_status_code(grpc_call *call, status_source source, call->status[source].is_set = 1; call->status[source].code = (grpc_status_code)status; +} + +static void set_status_details(grpc_call *call, status_source source, + grpc_mdstr *status) { + if (call->status[source].details != NULL) { + GRPC_MDSTR_UNREF(call->status[source].details); + } + call->status[source].details = status; +} + +static void get_final_status(grpc_call *call, + void (*set_value)(grpc_status_code code, + void *user_data), + void *set_value_user_data) { + int i; + for (i = 0; i < STATUS_SOURCE_COUNT; i++) { + if (call->status[i].is_set) { + set_value(call->status[i].code, set_value_user_data); + return; + } + } + if (call->is_client) { + set_value(GRPC_STATUS_UNKNOWN, set_value_user_data); + } else { + set_value(GRPC_STATUS_OK, set_value_user_data); + } +} - /* TODO(ctiller): what to do about the flush that was previously here */ +static void set_status_from_error(grpc_call *call, status_source source, + grpc_error *error) { + intptr_t status; + if (grpc_error_get_int(error, GRPC_ERROR_INT_GRPC_STATUS, &status)) { + set_status_code(call, source, (uint32_t)status); + } else { + set_status_code(call, source, GRPC_STATUS_INTERNAL); + } + const char *msg = grpc_error_get_str(error, GRPC_ERROR_STR_GRPC_MESSAGE); + bool free_msg = false; + if (msg == NULL) { + free_msg = true; + msg = grpc_error_string(error); + } + set_status_details(call, source, grpc_mdstr_from_string(msg)); + if (free_msg) grpc_error_free_string(msg); } static void set_incoming_compression_algorithm( @@ -492,32 +534,6 @@ uint32_t grpc_call_test_only_get_encodings_accepted_by_peer(grpc_call *call) { return encodings_accepted_by_peer; } -static void set_status_details(grpc_call *call, status_source source, - grpc_mdstr *status) { - if (call->status[source].details != NULL) { - GRPC_MDSTR_UNREF(call->status[source].details); - } - call->status[source].details = status; -} - -static void get_final_status(grpc_call *call, - void (*set_value)(grpc_status_code code, - void *user_data), - void *set_value_user_data) { - int i; - for (i = 0; i < STATUS_SOURCE_COUNT; i++) { - if (call->status[i].is_set) { - set_value(call->status[i].code, set_value_user_data); - return; - } - } - if (call->is_client) { - set_value(GRPC_STATUS_UNKNOWN, set_value_user_data); - } else { - set_value(GRPC_STATUS_OK, set_value_user_data); - } -} - static void get_final_details(grpc_call *call, char **out_details, size_t *out_details_capacity) { int i; @@ -741,8 +757,7 @@ grpc_call_error grpc_call_cancel_with_status(grpc_call *c, typedef struct termination_closure { grpc_closure closure; grpc_call *call; - grpc_status_code status; - gpr_slice optional_message; + grpc_error *error; grpc_closure *op_closure; enum { TC_CANCEL, TC_CLOSE } type; } termination_closure; @@ -758,7 +773,7 @@ static void done_termination(grpc_exec_ctx *exec_ctx, void *tcp, GRPC_CALL_INTERNAL_UNREF(exec_ctx, tc->call, "close"); break; } - gpr_slice_unref(tc->optional_message); + GRPC_ERROR_UNREF(tc->error); grpc_exec_ctx_sched(exec_ctx, tc->op_closure, GRPC_ERROR_NONE, NULL); gpr_free(tc); } @@ -767,7 +782,7 @@ static void send_cancel(grpc_exec_ctx *exec_ctx, void *tcp, grpc_error *error) { grpc_transport_stream_op op; termination_closure *tc = tcp; memset(&op, 0, sizeof(op)); - op.cancel_with_status = tc->status; + op.cancel_error = tc->error; /* reuse closure to catch completion */ grpc_closure_init(&tc->closure, done_termination, tc); op.on_complete = &tc->closure; @@ -778,8 +793,7 @@ static void send_close(grpc_exec_ctx *exec_ctx, void *tcp, grpc_error *error) { grpc_transport_stream_op op; termination_closure *tc = tcp; memset(&op, 0, sizeof(op)); - tc->optional_message = gpr_slice_ref(tc->optional_message); - grpc_transport_stream_op_add_close(&op, tc->status, &tc->optional_message); + op.close_error = tc->error; /* reuse closure to catch completion */ grpc_closure_init(&tc->closure, done_termination, tc); tc->op_closure = op.on_complete; @@ -789,14 +803,7 @@ static void send_close(grpc_exec_ctx *exec_ctx, void *tcp, grpc_error *error) { static grpc_call_error terminate_with_status(grpc_exec_ctx *exec_ctx, termination_closure *tc) { - grpc_mdstr *details = NULL; - if (GPR_SLICE_LENGTH(tc->optional_message) > 0) { - tc->optional_message = gpr_slice_ref(tc->optional_message); - details = grpc_mdstr_from_slice(tc->optional_message); - } - - set_status_code(tc->call, STATUS_FROM_API_OVERRIDE, (uint32_t)tc->status); - set_status_details(tc->call, STATUS_FROM_API_OVERRIDE, details); + set_status_from_error(tc->call, STATUS_FROM_API_OVERRIDE, tc->error); if (tc->type == TC_CANCEL) { grpc_closure_init(&tc->closure, send_cancel, tc); @@ -812,13 +819,15 @@ static grpc_call_error terminate_with_status(grpc_exec_ctx *exec_ctx, static grpc_call_error cancel_with_status(grpc_exec_ctx *exec_ctx, grpc_call *c, grpc_status_code status, const char *description) { + GPR_ASSERT(status != GRPC_STATUS_OK); termination_closure *tc = gpr_malloc(sizeof(*tc)); memset(tc, 0, sizeof(termination_closure)); tc->type = TC_CANCEL; tc->call = c; - tc->optional_message = gpr_slice_from_copied_string(description); - GPR_ASSERT(status != GRPC_STATUS_OK); - tc->status = status; + tc->error = grpc_error_set_int( + grpc_error_set_str(GRPC_ERROR_CREATE(description), + GRPC_ERROR_STR_GRPC_MESSAGE, description), + GRPC_ERROR_INT_GRPC_STATUS, status); return terminate_with_status(exec_ctx, tc); } @@ -826,13 +835,15 @@ static grpc_call_error cancel_with_status(grpc_exec_ctx *exec_ctx, grpc_call *c, static grpc_call_error close_with_status(grpc_exec_ctx *exec_ctx, grpc_call *c, grpc_status_code status, const char *description) { + GPR_ASSERT(status != GRPC_STATUS_OK); termination_closure *tc = gpr_malloc(sizeof(*tc)); memset(tc, 0, sizeof(termination_closure)); tc->type = TC_CLOSE; tc->call = c; - tc->optional_message = gpr_slice_from_copied_string(description); - GPR_ASSERT(status != GRPC_STATUS_OK); - tc->status = status; + tc->error = grpc_error_set_int( + grpc_error_set_str(GRPC_ERROR_CREATE(description), + GRPC_ERROR_STR_GRPC_MESSAGE, description), + GRPC_ERROR_INT_GRPC_STATUS, status); return terminate_with_status(exec_ctx, tc); } diff --git a/src/core/lib/surface/completion_queue.c b/src/core/lib/surface/completion_queue.c index 2cc6aa74e02..db8010ef9a1 100644 --- a/src/core/lib/surface/completion_queue.c +++ b/src/core/lib/surface/completion_queue.c @@ -240,7 +240,7 @@ void grpc_cq_end_op(grpc_exec_ctx *exec_ctx, grpc_completion_queue *cc, "grpc_cq_end_op(exec_ctx=%p, cc=%p, tag=%p, error=%s, done=%p, " "done_arg=%p, storage=%p)", 7, (exec_ctx, cc, tag, errmsg, done, done_arg, storage)); - if (grpc_trace_operation_failures) { + if (grpc_trace_operation_failures && error != GRPC_ERROR_NONE) { gpr_log(GPR_ERROR, "Operation failed: tag=%p, error=%s", tag, errmsg); } grpc_error_free_string(errmsg); diff --git a/src/core/lib/transport/transport.c b/src/core/lib/transport/transport.c index 1105494a85c..67920b05279 100644 --- a/src/core/lib/transport/transport.c +++ b/src/core/lib/transport/transport.c @@ -36,6 +36,7 @@ #include #include #include +#include "src/core/lib/support/string.h" #include "src/core/lib/transport/transport_impl.h" #ifdef GRPC_STREAM_REFCOUNT_DEBUG @@ -162,55 +163,63 @@ void grpc_transport_stream_op_finish_with_failure(grpc_exec_ctx *exec_ctx, grpc_exec_ctx_sched(exec_ctx, op->on_complete, error, NULL); } -void grpc_transport_stream_op_add_cancellation(grpc_transport_stream_op *op, - grpc_status_code status) { - GPR_ASSERT(status != GRPC_STATUS_OK); - if (op->cancel_with_status == GRPC_STATUS_OK) { - op->cancel_with_status = status; - } - if (op->close_with_status != GRPC_STATUS_OK) { - op->close_with_status = GRPC_STATUS_OK; - if (op->optional_close_message != NULL) { - gpr_slice_unref(*op->optional_close_message); - op->optional_close_message = NULL; - } - } -} - typedef struct { - gpr_slice message; + grpc_error *error; grpc_closure *then_call; grpc_closure closure; } close_message_data; static void free_message(grpc_exec_ctx *exec_ctx, void *p, grpc_error *error) { close_message_data *cmd = p; - gpr_slice_unref(cmd->message); + GRPC_ERROR_UNREF(cmd->error); if (cmd->then_call != NULL) { cmd->then_call->cb(exec_ctx, cmd->then_call->cb_arg, GRPC_ERROR_REF(error)); } gpr_free(cmd); } +static void add_error(grpc_transport_stream_op *op, grpc_error **which, + grpc_error *error) { + close_message_data *cmd; + cmd = gpr_malloc(sizeof(*cmd)); + cmd->error = error; + cmd->then_call = op->on_complete; + grpc_closure_init(&cmd->closure, free_message, cmd); + op->on_complete = &cmd->closure; + *which = error; +} + +void grpc_transport_stream_op_add_cancellation(grpc_transport_stream_op *op, + grpc_status_code status) { + GPR_ASSERT(status != GRPC_STATUS_OK); + if (op->cancel_error == GRPC_STATUS_OK) { + op->cancel_error = grpc_error_set_int(GRPC_ERROR_CANCELLED, + GRPC_ERROR_INT_GRPC_STATUS, status); + op->close_error = GRPC_ERROR_NONE; + } +} + void grpc_transport_stream_op_add_close(grpc_transport_stream_op *op, grpc_status_code status, gpr_slice *optional_message) { - close_message_data *cmd; GPR_ASSERT(status != GRPC_STATUS_OK); - if (op->cancel_with_status != GRPC_STATUS_OK || - op->close_with_status != GRPC_STATUS_OK) { + if (op->cancel_error != GRPC_ERROR_NONE || + op->close_error != GRPC_ERROR_NONE) { if (optional_message) { gpr_slice_unref(*optional_message); } return; } - if (optional_message) { - cmd = gpr_malloc(sizeof(*cmd)); - cmd->message = *optional_message; - cmd->then_call = op->on_complete; - grpc_closure_init(&cmd->closure, free_message, cmd); - op->on_complete = &cmd->closure; - op->optional_close_message = &cmd->message; + grpc_error *error; + if (optional_message != NULL) { + char *msg = gpr_dump_slice(*optional_message, GPR_DUMP_ASCII); + error = grpc_error_set_str(GRPC_ERROR_CREATE(msg), + GRPC_ERROR_STR_GRPC_MESSAGE, msg); + gpr_free(msg); + gpr_slice_unref(*optional_message); + } else { + error = GRPC_ERROR_CREATE("Call force closed"); } - op->close_with_status = status; + error = grpc_error_set_int(error, GRPC_ERROR_INT_GRPC_STATUS, status); + add_error(op, &op->close_error, error); } diff --git a/src/core/lib/transport/transport.h b/src/core/lib/transport/transport.h index a46ccb643ce..d2f6344ee3c 100644 --- a/src/core/lib/transport/transport.h +++ b/src/core/lib/transport/transport.h @@ -135,13 +135,12 @@ typedef struct grpc_transport_stream_op { /** Collect any stats into provided buffer, zero internal stat counters */ grpc_transport_stream_stats *collect_stats; - /** If != GRPC_STATUS_OK, cancel this stream */ - grpc_status_code cancel_with_status; + /** If != GRPC_ERROR_NONE, cancel this stream */ + grpc_error *cancel_error; - /** If != GRPC_STATUS_OK, send grpc-status, grpc-message, and close this + /** If != GRPC_ERROR, send grpc-status, grpc-message, and close this stream for both reading and writing */ - grpc_status_code close_with_status; - gpr_slice *optional_close_message; + grpc_error *close_error; /* Indexes correspond to grpc_context_index enum values */ grpc_call_context_element *context; diff --git a/src/core/lib/transport/transport_op_string.c b/src/core/lib/transport/transport_op_string.c index df04c611270..a862401df2c 100644 --- a/src/core/lib/transport/transport_op_string.c +++ b/src/core/lib/transport/transport_op_string.c @@ -119,10 +119,21 @@ char *grpc_transport_stream_op_string(grpc_transport_stream_op *op) { gpr_strvec_add(&b, gpr_strdup("RECV_TRAILING_METADATA")); } - if (op->cancel_with_status != GRPC_STATUS_OK) { + if (op->cancel_error != GRPC_STATUS_OK) { if (!first) gpr_strvec_add(&b, gpr_strdup(" ")); first = 0; - gpr_asprintf(&tmp, "CANCEL:%d", op->cancel_with_status); + const char *msg = grpc_error_string(op->cancel_error); + gpr_asprintf(&tmp, "CANCEL:%s", msg); + grpc_error_free_string(msg); + gpr_strvec_add(&b, tmp); + } + + if (op->close_error != GRPC_STATUS_OK) { + if (!first) gpr_strvec_add(&b, gpr_strdup(" ")); + first = 0; + const char *msg = grpc_error_string(op->close_error); + gpr_asprintf(&tmp, "CLOSE:%s", msg); + grpc_error_free_string(msg); gpr_strvec_add(&b, tmp); } From 5a3c6389edeaf19e7e0f0a6c0771c7548f0e0997 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 23 Jun 2016 14:02:33 -0700 Subject: [PATCH 128/215] Added braces around _service --- .../objective-c/route_guide/ViewControllers.m | 20 +++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/examples/objective-c/route_guide/ViewControllers.m b/examples/objective-c/route_guide/ViewControllers.m index 26ca9d6220b..b2f99c437e7 100644 --- a/examples/objective-c/route_guide/ViewControllers.m +++ b/examples/objective-c/route_guide/ViewControllers.m @@ -86,8 +86,9 @@ static NSString * const kHostAddress = @"localhost:50051"; @end -@implementation GetFeatureViewController -RTGRouteGuide *_service; +@implementation GetFeatureViewController { + RTGRouteGuide *_service; +} - (void)execRequest { void (^handler)(RTGFeature *response, NSError *error) = ^(RTGFeature *response, NSError *error) { @@ -146,8 +147,9 @@ RTGRouteGuide *_service; @end -@implementation ListFeaturesViewController -RTGRouteGuide *_service; +@implementation ListFeaturesViewController { + RTGRouteGuide *_service; +} - (void)execRequest { RTGRectangle *rectangle = [RTGRectangle message]; @@ -200,8 +202,9 @@ RTGRouteGuide *_service; @end -@implementation RecordRouteViewController -RTGRouteGuide *_service; +@implementation RecordRouteViewController { + RTGRouteGuide *_service; +} - (void)execRequest { NSString *dataBasePath = [NSBundle.mainBundle pathForResource:@"route_guide_db" @@ -268,8 +271,9 @@ RTGRouteGuide *_service; @end -@implementation RouteChatViewController -RTGRouteGuide *_service; +@implementation RouteChatViewController { + RTGRouteGuide *_service; +} - (void)execRequest { NSArray *notes = @[[RTGRouteNote noteWithMessage:@"First message" latitude:0 longitude:0], From 733e3fc209f78695896b2d721069afad866cd2f7 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 23 Jun 2016 14:07:11 -0700 Subject: [PATCH 129/215] Fix comparison --- src/core/lib/transport/transport_op_string.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/core/lib/transport/transport_op_string.c b/src/core/lib/transport/transport_op_string.c index a862401df2c..aede337e7cb 100644 --- a/src/core/lib/transport/transport_op_string.c +++ b/src/core/lib/transport/transport_op_string.c @@ -119,7 +119,7 @@ char *grpc_transport_stream_op_string(grpc_transport_stream_op *op) { gpr_strvec_add(&b, gpr_strdup("RECV_TRAILING_METADATA")); } - if (op->cancel_error != GRPC_STATUS_OK) { + if (op->cancel_error != GRPC_ERROR_NONE) { if (!first) gpr_strvec_add(&b, gpr_strdup(" ")); first = 0; const char *msg = grpc_error_string(op->cancel_error); @@ -128,7 +128,7 @@ char *grpc_transport_stream_op_string(grpc_transport_stream_op *op) { gpr_strvec_add(&b, tmp); } - if (op->close_error != GRPC_STATUS_OK) { + if (op->close_error != GRPC_ERROR_NONE) { if (!first) gpr_strvec_add(&b, gpr_strdup(" ")); first = 0; const char *msg = grpc_error_string(op->close_error); From ea0dc6af37469a45c2750df4bf852d12d75e9240 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 23 Jun 2016 14:10:38 -0700 Subject: [PATCH 130/215] Added a comment --- src/objective-c/GRPCClient/private/GRPCChannel.m | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/objective-c/GRPCClient/private/GRPCChannel.m b/src/objective-c/GRPCClient/private/GRPCChannel.m index 6cd4cb0ca30..7b7b79e1c62 100644 --- a/src/objective-c/GRPCClient/private/GRPCChannel.m +++ b/src/objective-c/GRPCClient/private/GRPCChannel.m @@ -199,7 +199,7 @@ grpc_channel_args * buildChannelArgs(NSDictionary *dictionary) { NULL, GRPC_PROPAGATE_DEFAULTS, queue.unmanagedQueue, path.UTF8String, - NULL, + NULL, // Passing NULL for host gpr_inf_future(GPR_CLOCK_REALTIME), NULL); } From eb429c3067e54b0dcdae21d005a4e98a022caad8 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 23 Jun 2016 14:18:49 -0700 Subject: [PATCH 131/215] Removed gpr_log statement --- src/core/lib/iomgr/network_status_tracker.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/src/core/lib/iomgr/network_status_tracker.c b/src/core/lib/iomgr/network_status_tracker.c index 0e34605c81d..38a1c9b7d41 100644 --- a/src/core/lib/iomgr/network_status_tracker.c +++ b/src/core/lib/iomgr/network_status_tracker.c @@ -64,7 +64,6 @@ void grpc_network_status_register_endpoint(grpc_endpoint *ep) { grpc_initialize_network_status_monitor(); } gpr_mu_lock(&g_endpoint_mutex); - gpr_log(GPR_DEBUG, "Register endpoint %p", ep); if (head == NULL) { head = (endpoint_ll_node *)gpr_malloc(sizeof(endpoint_ll_node)); head->ep = ep; @@ -81,7 +80,6 @@ void grpc_network_status_register_endpoint(grpc_endpoint *ep) { void grpc_network_status_unregister_endpoint(grpc_endpoint *ep) { gpr_mu_lock(&g_endpoint_mutex); GPR_ASSERT(head); - gpr_log(GPR_DEBUG, "Unregister endpoint %p", ep); bool found = false; endpoint_ll_node *prev = head; // if we're unregistering the head, just move head to the next @@ -116,7 +114,6 @@ void grpc_network_status_shutdown_all_endpoints() { grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; for (endpoint_ll_node *curr = head; curr != NULL; curr = curr->next) { - gpr_log(GPR_DEBUG, "Shutting down endpoint %p", curr->ep); curr->ep->vtable->shutdown(&exec_ctx, curr->ep); } gpr_mu_unlock(&g_endpoint_mutex); From d4d4c6f6f62a547da9136f57ddd6406facaf4ce5 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 23 Jun 2016 14:37:30 -0700 Subject: [PATCH 132/215] Fix comparison --- src/core/lib/transport/transport.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/core/lib/transport/transport.c b/src/core/lib/transport/transport.c index 67920b05279..79a20e12626 100644 --- a/src/core/lib/transport/transport.c +++ b/src/core/lib/transport/transport.c @@ -192,7 +192,7 @@ static void add_error(grpc_transport_stream_op *op, grpc_error **which, void grpc_transport_stream_op_add_cancellation(grpc_transport_stream_op *op, grpc_status_code status) { GPR_ASSERT(status != GRPC_STATUS_OK); - if (op->cancel_error == GRPC_STATUS_OK) { + if (op->cancel_error == GRPC_ERROR_NONE) { op->cancel_error = grpc_error_set_int(GRPC_ERROR_CANCELLED, GRPC_ERROR_INT_GRPC_STATUS, status); op->close_error = GRPC_ERROR_NONE; From 20d0a167beb287f61a7f33943fddfc34cae75860 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 23 Jun 2016 15:14:03 -0700 Subject: [PATCH 133/215] Better error handling and add polling_island_unlock_pair() helper --- src/core/lib/iomgr/ev_epoll_linux.c | 300 +++++++++++++++++----------- 1 file changed, 182 insertions(+), 118 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index b1e9ac8a631..a77044edc50 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -237,6 +237,19 @@ struct grpc_pollset_set { grpc_fd **fds; }; +/******************************************************************************* + * Common helpers + */ + +static void append_error(grpc_error **composite, grpc_error *error, + const char *desc) { + if (error == GRPC_ERROR_NONE) return; + if (*composite == GRPC_ERROR_NONE) { + *composite = GRPC_ERROR_CREATE(desc); + } + *composite = grpc_error_add_child(*composite, error); +} + /******************************************************************************* * Polling island Definitions */ @@ -316,10 +329,13 @@ long pi_unref(polling_island *pi, int ref_cnt) { /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, - size_t fd_count, bool add_fd_refs) { + size_t fd_count, bool add_fd_refs, + grpc_error **error) { int err; size_t i; struct epoll_event ev; + char *err_msg; + const char *err_desc = "polling_island_add_fds"; #ifdef GRPC_TSAN /* See the definition of g_epoll_sync for more context */ @@ -333,10 +349,12 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, if (err < 0) { if (errno != EEXIST) { - /* TODO: sreek - We need a better way to bubble up this error instead of - just logging a message */ - gpr_log(GPR_ERROR, "epoll_ctl add for fd: %d failed with error: %s", - fds[i]->fd, strerror(errno)); + gpr_asprintf( + &err_msg, + "epoll_ctl (epoll_fd: %d) add fd: %d failed with error: %d (%s)", + pi->epoll_fd, fds[i]->fd, errno, strerror(errno)); + append_error(error, GRPC_OS_ERROR(errno, err_msg), err_desc); + gpr_free(err_msg); } continue; @@ -356,37 +374,47 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, /* The caller is expected to hold pi->mu before calling this */ static void polling_island_add_wakeup_fd_locked(polling_island *pi, - grpc_wakeup_fd *wakeup_fd) { + grpc_wakeup_fd *wakeup_fd, + grpc_error **error) { struct epoll_event ev; int err; + char *err_msg; + const char *err_desc = "polling_island_add_wakeup_fd"; ev.events = (uint32_t)(EPOLLIN | EPOLLET); ev.data.ptr = wakeup_fd; err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, GRPC_WAKEUP_FD_GET_READ_FD(wakeup_fd), &ev); - if (err < 0) { - gpr_log(GPR_ERROR, - "Failed to add grpc_wake_up_fd (%d) to the epoll set (epoll_fd: %d)" - ". Error: %s", - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), pi->epoll_fd, - strerror(errno)); + if (err < 0 && errno != EEXIST) { + gpr_asprintf(&err_msg, + "epoll_ctl (epoll_fd: %d) add wakeup fd: %d failed with " + "error: %d (%s)", + pi->epoll_fd, + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), errno, + strerror(errno)); + append_error(error, GRPC_OS_ERROR(errno, err_msg), err_desc); + gpr_free(err_msg); } } /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_remove_all_fds_locked(polling_island *pi, - bool remove_fd_refs) { + bool remove_fd_refs, + grpc_error **error) { int err; size_t i; + char *err_msg; + const char *err_desc = "polling_island_remove_fds"; for (i = 0; i < pi->fd_cnt; i++) { err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, pi->fds[i]->fd, NULL); if (err < 0 && errno != ENOENT) { - /* TODO: sreek - We need a better way to bubble up this error instead of - * just logging a message */ - gpr_log(GPR_ERROR, - "epoll_ctl deleting fds[%zu]: %d failed with error: %s", i, - pi->fds[i]->fd, strerror(errno)); + gpr_asprintf(&err_msg, + "epoll_ctl (epoll_fd: %d) delete fds[%zu]: %d failed with " + "error: %d (%s)", + pi->epoll_fd, i, pi->fds[i]->fd, errno, strerror(errno)); + append_error(error, GRPC_OS_ERROR(errno, err_msg), err_desc); + gpr_free(err_msg); } if (remove_fd_refs) { @@ -399,17 +427,24 @@ static void polling_island_remove_all_fds_locked(polling_island *pi, /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, - bool is_fd_closed) { + bool is_fd_closed, + grpc_error **error) { int err; size_t i; + char *err_msg; + const char *err_desc = "polling_island_remove_fd"; /* If fd is already closed, then it would have been automatically been removed from the epoll set */ if (!is_fd_closed) { err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, fd->fd, NULL); if (err < 0 && errno != ENOENT) { - gpr_log(GPR_ERROR, "epoll_ctl deleting fd: %d failed with error; %s", - fd->fd, strerror(errno)); + gpr_asprintf( + &err_msg, + "epoll_ctl (epoll_fd: %d) del fd: %d failed with error: %d (%s)", + pi->epoll_fd, fd->fd, errno, strerror(errno)); + append_error(error, GRPC_OS_ERROR(errno, err_msg), err_desc); + gpr_free(err_msg); } } @@ -422,8 +457,12 @@ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, } } -static polling_island *polling_island_create(grpc_fd *initial_fd) { +/* Might return NULL in case of an error */ +static polling_island *polling_island_create(grpc_fd *initial_fd, + grpc_error **error) { polling_island *pi = NULL; + char *err_msg; + const char *err_desc = "polling_island_create"; /* Try to get one from the polling island freelist */ gpr_mu_lock(&g_pi_freelist_mu); @@ -449,22 +488,22 @@ static polling_island *polling_island_create(grpc_fd *initial_fd) { pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); if (pi->epoll_fd < 0) { - gpr_log(GPR_ERROR, "epoll_create1() failed with error: %s", - strerror(errno)); - } - GPR_ASSERT(pi->epoll_fd >= 0); - - polling_island_add_wakeup_fd_locked(pi, &grpc_global_wakeup_fd); - - pi->next_free = NULL; + gpr_asprintf(&err_msg, "epoll_create1 failed with error %d (%s)", errno, + strerror(errno)); + append_error(error, GRPC_OS_ERROR(errno, err_msg), err_desc); + gpr_free(err_msg); + } else { + polling_island_add_wakeup_fd_locked(pi, &grpc_global_wakeup_fd, error); + pi->next_free = NULL; - if (initial_fd != NULL) { - /* Lock the polling island here just in case we got this structure from the - freelist and the polling island lock was not released yet (by the code - that adds the polling island to the freelist) */ - gpr_mu_lock(&pi->mu); - polling_island_add_fds_locked(pi, &initial_fd, 1, true); - gpr_mu_unlock(&pi->mu); + if (initial_fd != NULL) { + /* Lock the polling island here just in case we got this structure from + the freelist and the polling island lock was not released yet (by the + code that adds the polling island to the freelist) */ + gpr_mu_lock(&pi->mu); + polling_island_add_fds_locked(pi, &initial_fd, 1, true, error); + gpr_mu_unlock(&pi->mu); + } } return pi; @@ -534,7 +573,9 @@ static polling_island *polling_island_lock(polling_island *pi) { return pi; } -/* Gets the lock on the *latest* polling islands pointed by *p and *q. +/* Gets the lock on the *latest* polling islands in the linked lists pointed by + *p and *q (and also updates *p and *q to point to the latest polling islands) + This function is needed because calling the following block of code to obtain locks on polling islands (*p and *q) is prone to deadlocks. { @@ -550,18 +591,8 @@ static polling_island *polling_island_lock(polling_island *pi) { .. .. Critical section with both p1 and p2 locked .. - // Release locks - // **IMPORTANT**: Make sure you check p1 == p2 AFTER the function - // polling_island_lock_pair() was called and if so, release the lock only - // once. Note: Even if p1 != p2 beforec calling polling_island_lock_pair(), - // they might be after the function returns: - if (p1 == p2) { - gpr_mu_unlock(&p1->mu) - } else { - gpr_mu_unlock(&p1->mu); - gpr_mu_unlock(&p2->mu); - } - + // Release locks: Always call polling_island_unlock_pair() to release locks + polling_island_unlock_pair(p1, p2); */ static void polling_island_lock_pair(polling_island **p, polling_island **q) { polling_island *pi_1 = *p; @@ -623,39 +654,46 @@ static void polling_island_lock_pair(polling_island **p, polling_island **q) { *q = pi_2; } -static polling_island *polling_island_merge(polling_island *p, - polling_island *q) { - /* Get locks on both the polling islands */ - polling_island_lock_pair(&p, &q); - +static void polling_island_unlock_pair(polling_island *p, polling_island *q) { if (p == q) { - /* Nothing needs to be done here */ gpr_mu_unlock(&p->mu); - return p; + } else { + gpr_mu_unlock(&p->mu); + gpr_mu_unlock(&q->mu); } +} - /* Make sure that p points to the polling island with fewer fds than q */ - if (p->fd_cnt > q->fd_cnt) { - GPR_SWAP(polling_island *, p, q); - } +static polling_island *polling_island_merge(polling_island *p, + polling_island *q, + grpc_error **error) { + /* Get locks on both the polling islands */ + polling_island_lock_pair(&p, &q); - /* "Merge" p with q i.e move all the fds from p (The one with fewer fds) to q - Note that the refcounts on the fds being moved will not change here. This - is why the last parameter in the following two functions is 'false') */ - polling_island_add_fds_locked(q, p->fds, p->fd_cnt, false); - polling_island_remove_all_fds_locked(p, false); + if (p != q) { + /* Make sure that p points to the polling island with fewer fds than q */ + if (p->fd_cnt > q->fd_cnt) { + GPR_SWAP(polling_island *, p, q); + } + + /* Merge p with q i.e move all the fds from p (The one with fewer fds) to q + Note that the refcounts on the fds being moved will not change here. + This is why the last param in the following two functions is 'false') */ + polling_island_add_fds_locked(q, p->fds, p->fd_cnt, false, error); + polling_island_remove_all_fds_locked(p, false, error); - /* Wakeup all the pollers (if any) on p so that they can pickup this change */ - polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); + /* Wakeup all the pollers (if any) on p so that they pickup this change */ + polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd, error); - /* Add the 'merged_to' link from p --> q */ - gpr_atm_rel_store(&p->merged_to, (gpr_atm)q); - PI_ADD_REF(q, "pi_merge"); /* To account for the new incoming ref from p */ + /* Add the 'merged_to' link from p --> q */ + gpr_atm_rel_store(&p->merged_to, (gpr_atm)q); + PI_ADD_REF(q, "pi_merge"); /* To account for the new incoming ref from p */ + } + /* else if p == q, nothing needs to be done */ - gpr_mu_unlock(&p->mu); - gpr_mu_unlock(&q->mu); + polling_island_unlock_pair(p, q); - /* Return the merged polling island */ + /* Return the merged polling island (Note that no merge would have happened + if p == q which is ok) */ return q; } @@ -853,6 +891,8 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure *on_done, int *release_fd, const char *reason) { bool is_fd_closed = false; + grpc_error *error = GRPC_ERROR_NONE; + gpr_mu_lock(&fd->mu); fd->on_done_closure = on_done; @@ -882,7 +922,7 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, gpr_mu_lock(&fd->pi_mu); if (fd->polling_island != NULL) { polling_island *pi_latest = polling_island_lock(fd->polling_island); - polling_island_remove_fd_locked(pi_latest, fd, is_fd_closed); + polling_island_remove_fd_locked(pi_latest, fd, is_fd_closed, &error); gpr_mu_unlock(&pi_latest->mu); PI_UNREF(fd->polling_island, "fd_orphan"); @@ -890,10 +930,11 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, } gpr_mu_unlock(&fd->pi_mu); - grpc_exec_ctx_sched(exec_ctx, fd->on_done_closure, GRPC_ERROR_NONE, NULL); + grpc_exec_ctx_sched(exec_ctx, fd->on_done_closure, error, NULL); gpr_mu_unlock(&fd->mu); UNREF_BY(fd, 2, reason); /* Drop the reference */ + GRPC_LOG_IF_ERROR("fd_orphan", GRPC_ERROR_REF(error)); } static grpc_error *fd_shutdown_error(bool shutdown) { @@ -1062,19 +1103,12 @@ static void push_front_worker(grpc_pollset *p, grpc_pollset_worker *worker) { worker->prev->next = worker->next->prev = worker; } -static void kick_append_error(grpc_error **composite, grpc_error *error) { - if (error == GRPC_ERROR_NONE) return; - if (*composite == GRPC_ERROR_NONE) { - *composite = GRPC_ERROR_CREATE("Kick Failure"); - } - *composite = grpc_error_add_child(*composite, error); -} - /* p->mu must be held before calling this function */ static grpc_error *pollset_kick(grpc_pollset *p, grpc_pollset_worker *specific_worker) { GPR_TIMER_BEGIN("pollset_kick", 0); grpc_error *error = GRPC_ERROR_NONE; + const char *err_desc = "Kick Failure"; grpc_pollset_worker *worker = specific_worker; if (worker != NULL) { @@ -1084,7 +1118,7 @@ static grpc_error *pollset_kick(grpc_pollset *p, for (worker = p->root_worker.next; worker != &p->root_worker; worker = worker->next) { if (gpr_tls_get(&g_current_thread_worker) != (intptr_t)worker) { - kick_append_error(&error, pollset_worker_kick(worker)); + append_error(&error, pollset_worker_kick(worker), err_desc); } } } else { @@ -1094,7 +1128,7 @@ static grpc_error *pollset_kick(grpc_pollset *p, } else { GPR_TIMER_MARK("kicked_specifically", 0); if (gpr_tls_get(&g_current_thread_worker) != (intptr_t)worker) { - kick_append_error(&error, pollset_worker_kick(worker)); + append_error(&error, pollset_worker_kick(worker), err_desc); } } } else if (gpr_tls_get(&g_current_thread_pollset) != (intptr_t)p) { @@ -1110,7 +1144,7 @@ static grpc_error *pollset_kick(grpc_pollset *p, if (worker != NULL) { GPR_TIMER_MARK("finally_kick", 0); push_back_worker(p, worker); - kick_append_error(&error, pollset_worker_kick(worker)); + append_error(&error, pollset_worker_kick(worker), err_desc); } else { GPR_TIMER_MARK("kicked_no_pollers", 0); p->kicked_without_pollers = true; @@ -1238,23 +1272,17 @@ static void pollset_reset(grpc_pollset *pollset) { pollset_release_polling_island(pollset, "ps_reset"); } -static void work_combine_error(grpc_error **composite, grpc_error *error) { - if (error == GRPC_ERROR_NONE) return; - if (*composite == GRPC_ERROR_NONE) { - *composite = GRPC_ERROR_CREATE("pollset_work"); - } - *composite = grpc_error_add_child(*composite, error); -} - #define GRPC_EPOLL_MAX_EVENTS 1000 -static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, - int timeout_ms, sigset_t *sig_mask) { +/* Note: sig_mask contains the signal mask to use *during* epoll_wait() */ +static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, int timeout_ms, + sigset_t *sig_mask, grpc_error **error) { struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; int epoll_fd = -1; int ep_rv; polling_island *pi = NULL; - grpc_error *error = GRPC_ERROR_NONE; + char *err_msg; + const char *err_desc = "pollset_work_and_unlock"; GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the @@ -1265,11 +1293,15 @@ static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, which we got the epoll_fd) got merged with another island while we are in this function. This is still okay because in such a case, we will wakeup right-away from epoll_wait() and pick up the latest polling_island the next - this function (i.e pollset_work_and_unlock()) is called. - */ + this function (i.e pollset_work_and_unlock()) is called */ if (pollset->polling_island == NULL) { - pollset->polling_island = polling_island_create(NULL); + pollset->polling_island = polling_island_create(NULL, error); + if (pollset->polling_island == NULL) { + GPR_TIMER_END("pollset_work_and_unlock", 0); + return; /* Fatal error. We cannot continue */ + } + PI_ADD_REF(pollset->polling_island, "ps"); } @@ -1297,8 +1329,10 @@ static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, sig_mask); if (ep_rv < 0) { if (errno != EINTR) { - gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); - work_combine_error(&error, GRPC_OS_ERROR(errno, "epoll_pwait")); + gpr_asprintf(&err_msg, + "epoll_wait() epoll fd: %d failed with error: %d (%s)", + epoll_fd, errno, strerror(errno)); + append_error(error, GRPC_OS_ERROR(errno, err_msg), err_desc); } else { /* We were interrupted. Save an interation by doing a zero timeout epoll_wait to see if there are any other events of interest */ @@ -1314,8 +1348,9 @@ static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, for (int i = 0; i < ep_rv; ++i) { void *data_ptr = ep_ev[i].data.ptr; if (data_ptr == &grpc_global_wakeup_fd) { - work_combine_error( - &error, grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd)); + append_error(error, + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd), + err_desc); } else if (data_ptr == &polling_island_wakeup_fd) { /* This means that our polling island is merged with a different island. We do not have to do anything here since the subsequent call @@ -1346,7 +1381,6 @@ static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, PI_UNREF(pi, "ps_work"); GPR_TIMER_END("pollset_work_and_unlock", 0); - return error; } /* pollset->mu lock must be held by the caller before calling this. @@ -1368,6 +1402,7 @@ static grpc_error *pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, worker.pt_id = pthread_self(); *worker_hdl = &worker; + gpr_tls_set(&g_current_thread_pollset, (intptr_t)pollset); gpr_tls_set(&g_current_thread_worker, (intptr_t)&worker); @@ -1379,14 +1414,37 @@ static grpc_error *pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); pollset->kicked_without_pollers = 0; } else if (!pollset->shutting_down) { + /* We use the posix-signal with number 'grpc_wakeup_signal' for waking up + (i.e 'kicking') a worker in the pollset. + A 'kick' is a way to inform that worker that there is some pending work + that needs immediate attention (like an event on the completion queue, + or a polling island merge that results in a new epoll-fd to wait on) and + that the worker should not spend time waiting in epoll_pwait(). + + A kick can come at anytime (i.e before/during or after the worker calls + epoll_pwait()) but in all cases we have to make sure that when a worker + gets a kick, it does not spend time in epoll_pwait(). In other words, one + kick should result in skipping/exiting of one epoll_pwait(); + + To accomplish this, we mask 'grpc_wakeup_signal' on this worker at all + times *except* when it is in epoll_pwait(). This way, the worker never + misses acting on a kick */ + sigemptyset(&new_mask); sigaddset(&new_mask, grpc_wakeup_signal); pthread_sigmask(SIG_BLOCK, &new_mask, &orig_mask); sigdelset(&orig_mask, grpc_wakeup_signal); + /* new_mask: The new thread mask which blocks 'grpc_wakeup_signal'. This is + the mask used at all times *except during epoll_wait()*" + orig_mask: The thread mask which allows 'grpc_wakeup_signal' and this is + the mask to use *during epoll_wait()* + + The new_mask is set on the worker before it is added to the pollset (i.e + before it can be kicked) */ - push_front_worker(pollset, &worker); + push_front_worker(pollset, &worker); /* Add worker to pollset */ - error = pollset_work_and_unlock(exec_ctx, pollset, timeout_ms, &orig_mask); + pollset_work_and_unlock(exec_ctx, pollset, timeout_ms, &orig_mask, &error); grpc_exec_ctx_flush(exec_ctx); gpr_mu_lock(&pollset->mu); @@ -1412,15 +1470,20 @@ static grpc_error *pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } *worker_hdl = NULL; + gpr_tls_set(&g_current_thread_pollset, (intptr_t)0); gpr_tls_set(&g_current_thread_worker, (intptr_t)0); + GPR_TIMER_END("pollset_work", 0); + GRPC_LOG_IF_ERROR("pollset_work", GRPC_ERROR_REF(error)); return error; } static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { + grpc_error *error = GRPC_ERROR_NONE; + gpr_mu_lock(&pollset->mu); gpr_mu_lock(&fd->pi_mu); @@ -1443,19 +1506,23 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, if (fd->polling_island == pollset->polling_island) { pi_new = fd->polling_island; if (pi_new == NULL) { - pi_new = polling_island_create(fd); + pi_new = polling_island_create(fd, &error); } } else if (fd->polling_island == NULL) { pi_new = polling_island_lock(pollset->polling_island); - polling_island_add_fds_locked(pi_new, &fd, 1, true); + polling_island_add_fds_locked(pi_new, &fd, 1, true, &error); gpr_mu_unlock(&pi_new->mu); } else if (pollset->polling_island == NULL) { pi_new = polling_island_lock(fd->polling_island); gpr_mu_unlock(&pi_new->mu); } else { - pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); + pi_new = polling_island_merge(fd->polling_island, pollset->polling_island, + &error); } + /* At this point, pi_new is the polling island that both fd->polling_island + and pollset->polling_island must be pointing to */ + if (fd->polling_island != pi_new) { PI_ADD_REF(pi_new, "fd"); if (fd->polling_island != NULL) { @@ -1645,13 +1712,10 @@ bool grpc_are_polling_islands_equal(void *p, void *q) { polling_island *p1 = p; polling_island *p2 = q; + /* Note: polling_island_lock_pair() may change p1 and p2 to point to the + latest polling islands in their respective linked lists */ polling_island_lock_pair(&p1, &p2); - if (p1 == p2) { - gpr_mu_unlock(&p1->mu); - } else { - gpr_mu_unlock(&p1->mu); - gpr_mu_unlock(&p2->mu); - } + polling_island_unlock_pair(p1, p2); return p1 == p2; } From 74189cd94b49be086e9320bf9536ab4bacfa6d61 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 23 Jun 2016 15:39:06 -0700 Subject: [PATCH 134/215] Remove caching of results by run_tests SIGNIFICANTLY increases the performance of actually running tests... --- tools/run_tests/jobset.py | 68 ++++++++---------------------------- tools/run_tests/run_tests.py | 63 ++------------------------------- 2 files changed, 18 insertions(+), 113 deletions(-) diff --git a/tools/run_tests/jobset.py b/tools/run_tests/jobset.py index d3259e724df..40409c43948 100755 --- a/tools/run_tests/jobset.py +++ b/tools/run_tests/jobset.py @@ -29,7 +29,6 @@ """Run a group of subprocesses and then finish.""" -import hashlib import multiprocessing import os import platform @@ -149,7 +148,7 @@ def which(filename): class JobSpec(object): """Specifies what to run for a job.""" - def __init__(self, cmdline, shortname=None, environ=None, hash_targets=None, + def __init__(self, cmdline, shortname=None, environ=None, cwd=None, shell=False, timeout_seconds=5*60, flake_retries=0, timeout_retries=0, kill_handler=None, cpu_cost=1.0, verbose_success=False): @@ -157,19 +156,14 @@ class JobSpec(object): Arguments: cmdline: a list of arguments to pass as the command line environ: a dictionary of environment variables to set in the child process - hash_targets: which files to include in the hash representing the jobs version - (or empty, indicating the job should not be hashed) kill_handler: a handler that will be called whenever job.kill() is invoked cpu_cost: number of cores per second this job needs """ if environ is None: environ = {} - if hash_targets is None: - hash_targets = [] self.cmdline = cmdline self.environ = environ self.shortname = cmdline[0] if shortname is None else shortname - self.hash_targets = hash_targets or [] self.cwd = cwd self.shell = shell self.timeout_seconds = timeout_seconds @@ -180,7 +174,7 @@ class JobSpec(object): self.verbose_success = verbose_success def identity(self): - return '%r %r %r' % (self.cmdline, self.environ, self.hash_targets) + return '%r %r' % (self.cmdline, self.environ) def __hash__(self): return hash(self.identity()) @@ -205,9 +199,8 @@ class JobResult(object): class Job(object): """Manages one job.""" - def __init__(self, spec, bin_hash, newline_on_success, travis, add_env): + def __init__(self, spec, newline_on_success, travis, add_env): self._spec = spec - self._bin_hash = bin_hash self._newline_on_success = newline_on_success self._travis = travis self._add_env = add_env.copy() @@ -249,7 +242,7 @@ class Job(object): self._process = try_start() self._state = _RUNNING - def state(self, update_cache): + def state(self): """Poll current state of the job. Prints messages at completion.""" def stdout(self=self): self._tempfile.seek(0) @@ -293,8 +286,6 @@ class Job(object): stdout() if self._spec.verbose_success else None, do_newline=self._newline_on_success or self._travis) self.result.state = 'PASSED' - if self._bin_hash: - update_cache.finished(self._spec.identity(), self._bin_hash) elif (self._state == _RUNNING and self._spec.timeout_seconds is not None and time.time() - self._start > self._spec.timeout_seconds): @@ -329,7 +320,7 @@ class Jobset(object): """Manages one run of jobs.""" def __init__(self, check_cancelled, maxjobs, newline_on_success, travis, - stop_on_failure, add_env, cache): + stop_on_failure, add_env): self._running = set() self._check_cancelled = check_cancelled self._cancelled = False @@ -338,9 +329,7 @@ class Jobset(object): self._maxjobs = maxjobs self._newline_on_success = newline_on_success self._travis = travis - self._cache = cache self._stop_on_failure = stop_on_failure - self._hashes = {} self._add_env = add_env self.resultset = {} self._remaining = None @@ -367,37 +356,21 @@ class Jobset(object): if current_cpu_cost + spec.cpu_cost <= self._maxjobs: break self.reap() if self.cancelled(): return False - if spec.hash_targets: - if spec.identity() in self._hashes: - bin_hash = self._hashes[spec.identity()] - else: - bin_hash = hashlib.sha1() - for fn in spec.hash_targets: - with open(which(fn)) as f: - bin_hash.update(f.read()) - bin_hash = bin_hash.hexdigest() - self._hashes[spec.identity()] = bin_hash - should_run = self._cache.should_run(spec.identity(), bin_hash) - else: - bin_hash = None - should_run = True - if should_run: - job = Job(spec, - bin_hash, - self._newline_on_success, - self._travis, - self._add_env) - self._running.add(job) - if not self.resultset.has_key(job.GetSpec().shortname): - self.resultset[job.GetSpec().shortname] = [] - return True + job = Job(spec, + self._newline_on_success, + self._travis, + self._add_env) + self._running.add(job) + if not self.resultset.has_key(job.GetSpec().shortname): + self.resultset[job.GetSpec().shortname] = [] + return True def reap(self): """Collect the dead jobs.""" while self._running: dead = set() for job in self._running: - st = job.state(self._cache) + st = job.state() if st == _RUNNING: continue if st == _FAILURE or st == _KILLED: self._failures += 1 @@ -450,15 +423,6 @@ def _never_cancelled(): return False -# cache class that caches nothing -class NoCache(object): - def should_run(self, cmdline, bin_hash): - return True - - def finished(self, cmdline, bin_hash): - pass - - def tag_remaining(xs): staging = [] for x in xs: @@ -477,12 +441,10 @@ def run(cmdlines, travis=False, infinite_runs=False, stop_on_failure=False, - cache=None, add_env={}): js = Jobset(check_cancelled, maxjobs if maxjobs is not None else _DEFAULT_MAX_JOBS, - newline_on_success, travis, stop_on_failure, add_env, - cache if cache is not None else NoCache()) + newline_on_success, travis, stop_on_failure, add_env) for cmdline, remaining in tag_remaining(cmdlines): if not js.start(cmdline): break diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index e4779e3a4e8..c571a81eb28 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -33,7 +33,6 @@ import argparse import ast import glob -import hashlib import itertools import json import multiprocessing @@ -78,24 +77,18 @@ class Config(object): if environ is None: environ = {} self.build_config = config - self.allow_hashing = (config != 'gcov') self.environ = environ self.environ['CONFIG'] = config self.tool_prefix = tool_prefix self.timeout_multiplier = timeout_multiplier - def job_spec(self, cmdline, hash_targets, timeout_seconds=5*60, + def job_spec(self, cmdline, timeout_seconds=5*60, shortname=None, environ={}, cpu_cost=1.0, flaky=False): """Construct a jobset.JobSpec for a test under this config Args: cmdline: a list of strings specifying the command line the test would like to run - hash_targets: either None (don't do caching of test results), or - a list of strings specifying files to include in a - binary hash to check if a test has changed - -- if used, all artifacts needed to run the test must - be listed """ actual_environ = self.environ.copy() for k, v in environ.iteritems(): @@ -105,8 +98,6 @@ class Config(object): environ=actual_environ, cpu_cost=cpu_cost, timeout_seconds=(self.timeout_multiplier * timeout_seconds if timeout_seconds else None), - hash_targets=hash_targets - if self.allow_hashing else None, flake_retries=5 if flaky or args.allow_flakes else 0, timeout_retries=3 if args.allow_flakes else 0) @@ -425,7 +416,7 @@ class PythonLanguage(object): return [] def build_steps(self): - return [['tools/run_tests/build_python.sh', tox_env] + return [['tools/run_tests/build_python.sh', tox_env] for tox_env in self._tox_envs] def post_tests_steps(self): @@ -1058,46 +1049,6 @@ runs_per_test = args.runs_per_test forever = args.forever -class TestCache(object): - """Cache for running tests.""" - - def __init__(self, use_cache_results): - self._last_successful_run = {} - self._use_cache_results = use_cache_results - self._last_save = time.time() - - def should_run(self, cmdline, bin_hash): - if cmdline not in self._last_successful_run: - return True - if self._last_successful_run[cmdline] != bin_hash: - return True - if not self._use_cache_results: - return True - return False - - def finished(self, cmdline, bin_hash): - self._last_successful_run[cmdline] = bin_hash - if time.time() - self._last_save > 1: - self.save() - - def dump(self): - return [{'cmdline': k, 'hash': v} - for k, v in self._last_successful_run.iteritems()] - - def parse(self, exdump): - self._last_successful_run = dict((o['cmdline'], o['hash']) for o in exdump) - - def save(self): - with open('.run_tests_cache', 'w') as f: - f.write(json.dumps(self.dump())) - self._last_save = time.time() - - def maybe_load(self): - if os.path.exists('.run_tests_cache'): - with open('.run_tests_cache') as f: - self.parse(json.loads(f.read())) - - def _start_port_server(port_server_port): # check if a compatible port server is running # if incompatible (version mismatch) ==> start a new one @@ -1217,7 +1168,7 @@ class BuildAndRunError(object): # returns a list of things that failed (or an empty list on success) def _build_and_run( - check_cancelled, newline_on_success, cache, xml_report=None, build_only=False): + check_cancelled, newline_on_success, xml_report=None, build_only=False): """Do one pass of building & running tests.""" # build latest sequentially num_failures, resultset = jobset.run( @@ -1266,7 +1217,6 @@ def _build_and_run( all_runs, check_cancelled, newline_on_success=newline_on_success, travis=args.travis, infinite_runs=infinite_runs, maxjobs=args.jobs, stop_on_failure=args.stop_on_failure, - cache=cache if not xml_report else None, add_env={'GRPC_TEST_PORT_SERVER': 'localhost:%d' % port_server_port}) if resultset: for k, v in sorted(resultset.items()): @@ -1295,14 +1245,9 @@ def _build_and_run( if num_test_failures: out.append(BuildAndRunError.TEST) - if cache: cache.save() - return out -test_cache = TestCache(runs_per_test == 1) -test_cache.maybe_load() - if forever: success = True while True: @@ -1312,7 +1257,6 @@ if forever: previous_success = success errors = _build_and_run(check_cancelled=have_files_changed, newline_on_success=False, - cache=test_cache, build_only=args.build_only) == 0 if not previous_success and not errors: jobset.message('SUCCESS', @@ -1324,7 +1268,6 @@ if forever: else: errors = _build_and_run(check_cancelled=lambda: False, newline_on_success=args.newline_on_success, - cache=test_cache, xml_report=args.xml_report, build_only=args.build_only) if not errors: From 98d31d1a40fe0d320f2488794d97a0bad5a8060d Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 23 Jun 2016 15:43:22 -0700 Subject: [PATCH 135/215] Fix special value lookup --- src/core/lib/iomgr/error.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/core/lib/iomgr/error.c b/src/core/lib/iomgr/error.c index da0d6972360..9f5ba76fd6c 100644 --- a/src/core/lib/iomgr/error.c +++ b/src/core/lib/iomgr/error.c @@ -276,7 +276,8 @@ bool grpc_error_get_int(grpc_error *err, grpc_error_ints which, intptr_t *p) { void *pp; if (is_special(err)) { if (err == GRPC_ERROR_CANCELLED && which == GRPC_ERROR_INT_GRPC_STATUS) { - return GRPC_STATUS_CANCELLED; + *p = GRPC_STATUS_CANCELLED; + return true; } return false; } From 6a29545c8c5ef61346af3a9b0bdd2ddb39ba15c8 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 23 Jun 2016 15:53:10 -0700 Subject: [PATCH 136/215] Change the type of 'ref_count' in polling_island from gpr_atm to gpr_refcount --- src/core/lib/iomgr/ev_epoll_linux.c | 69 +++++++++++++---------------- 1 file changed, 31 insertions(+), 38 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index a77044edc50..4dca551e1e1 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -121,6 +121,7 @@ struct grpc_fd { }; /* Reference counting for fds */ +// #define GRPC_FD_REF_COUNT_DEBUG #ifdef GRPC_FD_REF_COUNT_DEBUG static void fd_ref(grpc_fd *fd, const char *reason, const char *file, int line); static void fd_unref(grpc_fd *fd, const char *reason, const char *file, @@ -147,13 +148,13 @@ static void fd_global_shutdown(void); // #define GRPC_PI_REF_COUNT_DEBUG #ifdef GRPC_PI_REF_COUNT_DEBUG -#define PI_ADD_REF(p, r) pi_add_ref_dbg((p), 1, (r), __FILE__, __LINE__) -#define PI_UNREF(p, r) pi_unref_dbg((p), 1, (r), __FILE__, __LINE__) +#define PI_ADD_REF(p, r) pi_add_ref_dbg((p), (r), __FILE__, __LINE__) +#define PI_UNREF(p, r) pi_unref_dbg((p), (r), __FILE__, __LINE__) #else /* defined(GRPC_PI_REF_COUNT_DEBUG) */ -#define PI_ADD_REF(p, r) pi_add_ref((p), 1) -#define PI_UNREF(p, r) pi_unref((p), 1) +#define PI_ADD_REF(p, r) pi_add_ref((p)) +#define PI_UNREF(p, r) pi_unref((p)) #endif /* !defined(GPRC_PI_REF_COUNT_DEBUG) */ @@ -164,7 +165,7 @@ typedef struct polling_island { Once the ref count becomes zero, this structure is destroyed which means we should ensure that there is never a scenario where a PI_ADD_REF() is racing with a PI_UNREF() that just made the ref_count zero. */ - gpr_atm ref_count; + gpr_refcount ref_count; /* Pointer to the polling_island this merged into. * merged_to value is only set once in polling_island's lifetime (and that too @@ -281,50 +282,42 @@ gpr_atm g_epoll_sync; #endif /* defined(GRPC_TSAN) */ #ifdef GRPC_PI_REF_COUNT_DEBUG -long pi_add_ref(polling_island *pi, int ref_cnt); -long pi_unref(polling_island *pi, int ref_cnt); +void pi_add_ref(polling_island *pi); +void pi_unref(polling_island *pi); -void pi_add_ref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, - int line) { - long old_cnt = pi_add_ref(pi, ref_cnt); - gpr_log(GPR_DEBUG, "Add ref pi: %p, old:%ld -> new:%ld (%s) - (%s, %d)", - (void *)pi, old_cnt, (old_cnt + ref_cnt), reason, file, line); +void pi_add_ref_dbg(polling_island *pi, char *reason, char *file, int line) { + long old_cnt = gpr_atm_acq_load(&(pi->ref_count.count)); + pi_add_ref(pi); + gpr_log(GPR_DEBUG, "Add ref pi: %p, old: %ld -> new:%ld (%s) - (%s, %d)", + (void *)pi, old_cnt, old_cnt + 1, reason, file, line); } -void pi_unref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, - int line) { - long old_cnt = pi_unref(pi, ref_cnt); +void pi_unref_dbg(polling_island *pi, char *reason, char *file, int line) { + long old_cnt = gpr_atm_acq_load(&(pi->ref_count.count)); + pi_unref(pi); gpr_log(GPR_DEBUG, "Unref pi: %p, old:%ld -> new:%ld (%s) - (%s, %d)", - (void *)pi, old_cnt, (old_cnt - ref_cnt), reason, file, line); + (void *)pi, old_cnt, (old_cnt - 1), reason, file, line); } #endif -long pi_add_ref(polling_island *pi, int ref_cnt) { - return gpr_atm_full_fetch_add(&pi->ref_count, ref_cnt); -} - -long pi_unref(polling_island *pi, int ref_cnt) { - long old_cnt = gpr_atm_full_fetch_add(&pi->ref_count, -ref_cnt); +void pi_add_ref(polling_island *pi) { gpr_ref(&pi->ref_count); } - /* If ref count went to zero, delete the polling island. Note that this need - not be done under a lock. Once the ref count goes to zero, we are - guaranteed that no one else holds a reference to the polling island (and - that there is no racing pi_add_ref() call either. +void pi_unref(polling_island *pi) { + /* If ref count went to zero, delete the polling island. + Note that this deletion not be done under a lock. Once the ref count goes + to zero, we are guaranteed that no one else holds a reference to the + polling island (and that there is no racing pi_add_ref() call either). Also, if we are deleting the polling island and the merged_to field is non-empty, we should remove a ref to the merged_to polling island */ - if (old_cnt == ref_cnt) { + if (gpr_unref(&pi->ref_count)) { polling_island *next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); polling_island_delete(pi); if (next != NULL) { PI_UNREF(next, "pi_delete"); /* Recursive call */ } - } else { - GPR_ASSERT(old_cnt > ref_cnt); } - - return old_cnt; } /* The caller is expected to hold pi->mu lock before calling this function */ @@ -482,7 +475,7 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, pi->fds = NULL; } - gpr_atm_rel_store(&pi->ref_count, (gpr_atm)0); + gpr_ref_init(&pi->ref_count, 0); gpr_atm_rel_store(&pi->merged_to, (gpr_atm)NULL); pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); @@ -762,8 +755,8 @@ static gpr_mu fd_freelist_mu; #define UNREF_BY(fd, n, reason) unref_by(fd, n, reason, __FILE__, __LINE__) static void ref_by(grpc_fd *fd, int n, const char *reason, const char *file, int line) { - gpr_log(GPR_DEBUG, "FD %d %p ref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, - gpr_atm_no_barrier_load(&fd->refst), + gpr_log(GPR_DEBUG, "FD %d %p ref %d %ld -> %ld [%s; %s:%d]", fd->fd, + (void *)fd, n, gpr_atm_no_barrier_load(&fd->refst), gpr_atm_no_barrier_load(&fd->refst) + n, reason, file, line); #else #define REF_BY(fd, n, reason) ref_by(fd, n) @@ -777,8 +770,8 @@ static void ref_by(grpc_fd *fd, int n) { static void unref_by(grpc_fd *fd, int n, const char *reason, const char *file, int line) { gpr_atm old; - gpr_log(GPR_DEBUG, "FD %d %p unref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, - gpr_atm_no_barrier_load(&fd->refst), + gpr_log(GPR_DEBUG, "FD %d %p unref %d %ld -> %ld [%s; %s:%d]", fd->fd, + (void *)fd, n, gpr_atm_no_barrier_load(&fd->refst), gpr_atm_no_barrier_load(&fd->refst) - n, reason, file, line); #else static void unref_by(grpc_fd *fd, int n) { @@ -865,10 +858,10 @@ static grpc_fd *fd_create(int fd, const char *name) { char *fd_name; gpr_asprintf(&fd_name, "%s fd=%d", name, fd); grpc_iomgr_register_object(&new_fd->iomgr_object, fd_name); - gpr_free(fd_name); #ifdef GRPC_FD_REF_COUNT_DEBUG - gpr_log(GPR_DEBUG, "FD %d %p create %s", fd, r, fd_name); + gpr_log(GPR_DEBUG, "FD %d %p create %s", fd, (void *)new_fd, fd_name); #endif + gpr_free(fd_name); return new_fd; } From eb5e43762be40f78775dfd3728e105daf1169d90 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 23 Jun 2016 16:16:10 -0700 Subject: [PATCH 137/215] Fix ruby tests --- tools/run_tests/run_tests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index c571a81eb28..61cef0a9d57 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -451,7 +451,7 @@ class RubyLanguage(object): _check_compiler(self.args.compiler, ['default']) def test_specs(self): - return [self.config.job_spec(['tools/run_tests/run_ruby.sh'], None, + return [self.config.job_spec(['tools/run_tests/run_ruby.sh'], timeout_seconds=10*60, environ=_FORCE_ENVIRON_FOR_WRAPPERS)] From 2e1a1fe56f4276d670362c5aae1b493df0834c2e Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 23 Jun 2016 16:18:31 -0700 Subject: [PATCH 138/215] Fixes --- tools/run_tests/dockerjob.py | 4 ++-- tools/run_tests/run_performance_tests.py | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tools/run_tests/dockerjob.py b/tools/run_tests/dockerjob.py index 326c4faed95..e4ca3b7faaf 100755 --- a/tools/run_tests/dockerjob.py +++ b/tools/run_tests/dockerjob.py @@ -104,7 +104,7 @@ class DockerJob: def __init__(self, spec): self._spec = spec - self._job = jobset.Job(spec, bin_hash=None, newline_on_success=True, travis=True, add_env={}) + self._job = jobset.Job(spec, newline_on_success=True, travis=True, add_env={}) self._container_name = spec.container_name def mapped_port(self, port): @@ -118,4 +118,4 @@ class DockerJob: def is_running(self): """Polls a job and returns True if given job is still running.""" - return self._job.state(jobset.NoCache()) == jobset._RUNNING + return self._job.state() == jobset._RUNNING diff --git a/tools/run_tests/run_performance_tests.py b/tools/run_tests/run_performance_tests.py index f037d0d17d9..14901caf07f 100755 --- a/tools/run_tests/run_performance_tests.py +++ b/tools/run_tests/run_performance_tests.py @@ -61,11 +61,11 @@ class QpsWorkerJob: self._spec = spec self.language = language self.host_and_port = host_and_port - self._job = jobset.Job(spec, bin_hash=None, newline_on_success=True, travis=True, add_env={}) + self._job = jobset.Job(spec, newline_on_success=True, travis=True, add_env={}) def is_running(self): """Polls a job and returns True if given job is still running.""" - return self._job.state(jobset.NoCache()) == jobset._RUNNING + return self._job.state() == jobset._RUNNING def kill(self): return self._job.kill() From d6f8f0b7cd9c1e92219027919eb8a562763e029b Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Thu, 23 Jun 2016 17:33:01 -0700 Subject: [PATCH 139/215] Add TODO --- src/objective-c/examples/SwiftSample/ViewController.swift | 1 + 1 file changed, 1 insertion(+) diff --git a/src/objective-c/examples/SwiftSample/ViewController.swift b/src/objective-c/examples/SwiftSample/ViewController.swift index 80d7a47917f..2a95d2de516 100644 --- a/src/objective-c/examples/SwiftSample/ViewController.swift +++ b/src/objective-c/examples/SwiftSample/ViewController.swift @@ -71,6 +71,7 @@ class ViewController: UIViewController { NSLog("2. Response trailers: \(RPC.responseTrailers)") } + // TODO(jcanizales): Revert to using subscript syntax once XCode 8 is released. RPC.requestHeaders.setObject("My value", forKey: "My-Header") RPC.start() From 33b767a7701c373767bf29a49c93c9e515eb17f8 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Fri, 24 Jun 2016 09:13:03 -0700 Subject: [PATCH 140/215] fix build node package --- tools/run_tests/build_package_node.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/run_tests/build_package_node.sh b/tools/run_tests/build_package_node.sh index 6f7211b53fe..ff4cfdb8bf9 100755 --- a/tools/run_tests/build_package_node.sh +++ b/tools/run_tests/build_package_node.sh @@ -86,6 +86,7 @@ for arch in {x86,x64}; do cp $input_dir/protoc* bin/ cp $input_dir/grpc_node_plugin* bin/ mkdir -p bin/google/protobuf + mkdir -p bin/google/protobuf/compiler # needed for plugin.proto for proto in "${well_known_protos[@]}"; do cp $base/third_party/protobuf/src/google/protobuf/$proto.proto bin/google/protobuf/$proto.proto done From e07b83ba9fde162fe0e61f97c657492f4d948e95 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 24 Jun 2016 09:27:00 -0700 Subject: [PATCH 141/215] generate_projects.sh --- src/node/tools/package.json | 1 + 1 file changed, 1 insertion(+) diff --git a/src/node/tools/package.json b/src/node/tools/package.json index d4849c2e388..c34b259e2e0 100644 --- a/src/node/tools/package.json +++ b/src/node/tools/package.json @@ -34,6 +34,7 @@ "index.js", "bin/protoc.js", "bin/protoc_plugin.js", + "bin/google/protobuf", "LICENSE" ], "main": "index.js" From 886c512832464a5aa5e61b51f1fcd1da356ca847 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Fri, 24 Jun 2016 09:48:58 -0700 Subject: [PATCH 142/215] fix C# nuget build --- src/csharp/Grpc.Core/Grpc.Core.nuspec | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/src/csharp/Grpc.Core/Grpc.Core.nuspec b/src/csharp/Grpc.Core/Grpc.Core.nuspec index fa2c1fbff22..47593f787b1 100644 --- a/src/csharp/Grpc.Core/Grpc.Core.nuspec +++ b/src/csharp/Grpc.Core/Grpc.Core.nuspec @@ -24,11 +24,12 @@ - - - - - - + + + + + + + From d29a3bf004e23eb481b33b9a69fd10ae9221fc98 Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Fri, 24 Jun 2016 10:26:14 -0700 Subject: [PATCH 143/215] Update master branch to 0.16.0-dev --- Makefile | 2 +- build.yaml | 2 +- package.json | 2 +- package.xml | 4 ++-- src/core/lib/surface/version.c | 2 +- src/csharp/Grpc.Auth/project.json | 4 ++-- src/csharp/Grpc.Core/VersionInfo.cs | 4 ++-- src/csharp/Grpc.Core/project.json | 2 +- src/csharp/Grpc.HealthCheck/project.json | 4 ++-- src/csharp/build_packages.bat | 2 +- src/node/tools/package.json | 2 +- src/python/grpcio/grpc_version.py | 2 +- src/ruby/lib/grpc/version.rb | 2 +- src/ruby/tools/version.rb | 2 +- tools/distrib/python/grpcio_tools/grpc_version.py | 2 +- tools/doxygen/Doxyfile.c++ | 2 +- tools/doxygen/Doxyfile.c++.internal | 2 +- tools/doxygen/Doxyfile.core | 2 +- tools/doxygen/Doxyfile.core.internal | 2 +- 19 files changed, 23 insertions(+), 23 deletions(-) diff --git a/Makefile b/Makefile index 9be3e5784cc..2f49e7712fd 100644 --- a/Makefile +++ b/Makefile @@ -414,7 +414,7 @@ E = @echo Q = @ endif -VERSION = 0.15.0-dev +VERSION = 0.16.0-dev CPPFLAGS_NO_ARCH += $(addprefix -I, $(INCLUDES)) $(addprefix -D, $(DEFINES)) CPPFLAGS += $(CPPFLAGS_NO_ARCH) $(ARCH_FLAGS) diff --git a/build.yaml b/build.yaml index 1f06e20cf9b..7d570e4dab9 100644 --- a/build.yaml +++ b/build.yaml @@ -7,7 +7,7 @@ settings: '#3': Use "-preN" suffixes to identify pre-release versions '#4': Per-language overrides are possible with (eg) ruby_version tag here '#5': See the expand_version.py for all the quirks here - version: 0.15.0-dev + version: 0.16.0-dev filegroups: - name: census public_headers: diff --git a/package.json b/package.json index 5bdaa761e28..68a31d794c5 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "grpc", - "version": "0.15.0-dev", + "version": "0.16.0-dev", "author": "Google Inc.", "description": "gRPC Library for Node", "homepage": "http://www.grpc.io/", diff --git a/package.xml b/package.xml index 67e9bb2c282..c1ecae8b806 100644 --- a/package.xml +++ b/package.xml @@ -13,8 +13,8 @@ 2016-05-19 - 0.15.0 - 0.15.0 + 0.16.0 + 0.16.0 beta diff --git a/src/core/lib/surface/version.c b/src/core/lib/surface/version.c index aca76d2bb79..53f3c438541 100644 --- a/src/core/lib/surface/version.c +++ b/src/core/lib/surface/version.c @@ -36,4 +36,4 @@ #include -const char *grpc_version_string(void) { return "0.15.0-dev"; } +const char *grpc_version_string(void) { return "0.16.0-dev"; } diff --git a/src/csharp/Grpc.Auth/project.json b/src/csharp/Grpc.Auth/project.json index 1677565824b..4c5c960204e 100644 --- a/src/csharp/Grpc.Auth/project.json +++ b/src/csharp/Grpc.Auth/project.json @@ -1,5 +1,5 @@ { - "version": "0.15.0-dev", + "version": "0.16.0-dev", "title": "gRPC C# Auth", "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", @@ -13,7 +13,7 @@ "tags": [ "gRPC RPC Protocol HTTP/2 Auth OAuth2" ], }, "dependencies": { - "Grpc.Core": "0.15.0-dev", + "Grpc.Core": "0.16.0-dev", "Google.Apis.Auth": "1.11.1" }, "frameworks": { diff --git a/src/csharp/Grpc.Core/VersionInfo.cs b/src/csharp/Grpc.Core/VersionInfo.cs index e1609341d9a..cb20967680e 100644 --- a/src/csharp/Grpc.Core/VersionInfo.cs +++ b/src/csharp/Grpc.Core/VersionInfo.cs @@ -48,11 +48,11 @@ namespace Grpc.Core /// /// Current AssemblyFileVersion of gRPC C# assemblies /// - public const string CurrentAssemblyFileVersion = "0.15.0.0"; + public const string CurrentAssemblyFileVersion = "0.16.0.0"; /// /// Current version of gRPC C# /// - public const string CurrentVersion = "0.15.0-dev"; + public const string CurrentVersion = "0.16.0-dev"; } } diff --git a/src/csharp/Grpc.Core/project.json b/src/csharp/Grpc.Core/project.json index 7253107e04a..4729a9346cb 100644 --- a/src/csharp/Grpc.Core/project.json +++ b/src/csharp/Grpc.Core/project.json @@ -1,5 +1,5 @@ { - "version": "0.15.0-dev", + "version": "0.16.0-dev", "title": "gRPC C# Core", "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", diff --git a/src/csharp/Grpc.HealthCheck/project.json b/src/csharp/Grpc.HealthCheck/project.json index eb57608957a..c4895c2ad3c 100644 --- a/src/csharp/Grpc.HealthCheck/project.json +++ b/src/csharp/Grpc.HealthCheck/project.json @@ -1,5 +1,5 @@ { - "version": "0.15.0-dev", + "version": "0.16.0-dev", "title": "gRPC C# Healthchecking", "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", @@ -13,7 +13,7 @@ "tags": [ "gRPC health check" ] }, "dependencies": { - "Grpc.Core": "0.15.0-dev", + "Grpc.Core": "0.16.0-dev", "Google.Protobuf": "3.0.0-beta3" }, "frameworks": { diff --git a/src/csharp/build_packages.bat b/src/csharp/build_packages.bat index 63f8c30bc7e..272b30f385d 100644 --- a/src/csharp/build_packages.bat +++ b/src/csharp/build_packages.bat @@ -30,7 +30,7 @@ @rem Builds gRPC NuGet packages @rem Current package versions -set VERSION=0.15.0-dev +set VERSION=0.16.0-dev set PROTOBUF_VERSION=3.0.0-beta3 @rem Packages that depend on prerelease packages (like Google.Protobuf) need to have prerelease suffix as well. diff --git a/src/node/tools/package.json b/src/node/tools/package.json index c34b259e2e0..7c256d7ba0d 100644 --- a/src/node/tools/package.json +++ b/src/node/tools/package.json @@ -1,6 +1,6 @@ { "name": "grpc-tools", - "version": "0.15.0-dev", + "version": "0.16.0-dev", "author": "Google Inc.", "description": "Tools for developing with gRPC on Node.js", "homepage": "http://www.grpc.io/", diff --git a/src/python/grpcio/grpc_version.py b/src/python/grpcio/grpc_version.py index 0c13104d9da..0f4db9d972e 100644 --- a/src/python/grpcio/grpc_version.py +++ b/src/python/grpcio/grpc_version.py @@ -29,4 +29,4 @@ # AUTO-GENERATED FROM `$REPO_ROOT/templates/src/python/grpcio/grpc_version.py.template`!!! -VERSION='0.15.0.dev0' +VERSION='0.16.0.dev0' diff --git a/src/ruby/lib/grpc/version.rb b/src/ruby/lib/grpc/version.rb index 01c8c5ac8f1..5e6aaef2eb5 100644 --- a/src/ruby/lib/grpc/version.rb +++ b/src/ruby/lib/grpc/version.rb @@ -29,5 +29,5 @@ # GRPC contains the General RPC module. module GRPC - VERSION = '0.15.0.dev' + VERSION = '0.16.0.dev' end diff --git a/src/ruby/tools/version.rb b/src/ruby/tools/version.rb index dca7fd7e72c..68c1bf369d2 100644 --- a/src/ruby/tools/version.rb +++ b/src/ruby/tools/version.rb @@ -29,6 +29,6 @@ module GRPC module Tools - VERSION = '0.15.0.dev' + VERSION = '0.16.0.dev' end end diff --git a/tools/distrib/python/grpcio_tools/grpc_version.py b/tools/distrib/python/grpcio_tools/grpc_version.py index 1267d0e45dc..4b1e7fcd584 100644 --- a/tools/distrib/python/grpcio_tools/grpc_version.py +++ b/tools/distrib/python/grpcio_tools/grpc_version.py @@ -29,4 +29,4 @@ # AUTO-GENERATED FROM `$REPO_ROOT/templates/tools/distrib/python/grpcio_tools/grpc_version.py.template`!!! -VERSION='0.15.0.dev0' +VERSION='0.16.0.dev0' diff --git a/tools/doxygen/Doxyfile.c++ b/tools/doxygen/Doxyfile.c++ index 7f9d2df6f6c..de7acd7777e 100644 --- a/tools/doxygen/Doxyfile.c++ +++ b/tools/doxygen/Doxyfile.c++ @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC C++" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.16.0-dev # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a diff --git a/tools/doxygen/Doxyfile.c++.internal b/tools/doxygen/Doxyfile.c++.internal index dcf1a4c8c40..76bb3b6c59e 100644 --- a/tools/doxygen/Doxyfile.c++.internal +++ b/tools/doxygen/Doxyfile.c++.internal @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC C++" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.16.0-dev # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a diff --git a/tools/doxygen/Doxyfile.core b/tools/doxygen/Doxyfile.core index 72102b2fc50..53ae4e4cf4b 100644 --- a/tools/doxygen/Doxyfile.core +++ b/tools/doxygen/Doxyfile.core @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC Core" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.16.0-dev # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal index e1555930e91..1c75c941c27 100644 --- a/tools/doxygen/Doxyfile.core.internal +++ b/tools/doxygen/Doxyfile.core.internal @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC Core" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.16.0-dev # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a From f4c1bff0d8caedf716b460c4ee6114ac126e8872 Mon Sep 17 00:00:00 2001 From: Ken Payson Date: Sun, 12 Jun 2016 15:23:36 -0700 Subject: [PATCH 144/215] Moved grpc_shutdown to end of Py_Finalize() We currently rely on the __del__ method of a module scope object to call grpc_shutdown(). __del__ methods are not guaranteed to be called, and furthermore there are no guarantees about ordering, leading to shutdown race conditions. This moves grpc_shutdown to Py_Finalize(), which gets called after the Python context is completely cleaned up. --- .../grpcio/grpc/_cython/_cygrpc/grpc.pxi | 1 + src/python/grpcio/grpc/_cython/cygrpc.pyx | 32 +++++++------------ src/python/grpcio/grpc/_cython/loader.c | 7 ++++ src/python/grpcio/grpc/_cython/loader.h | 5 +++ 4 files changed, 24 insertions(+), 21 deletions(-) diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi index 05b8886df73..35e394d02f8 100644 --- a/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi +++ b/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi @@ -37,6 +37,7 @@ cdef extern from "grpc/_cython/loader.h": ctypedef long int64_t int pygrpc_load_core(char*) + int pygrpc_initialize_core() void *gpr_malloc(size_t size) nogil void gpr_free(void *ptr) nogil diff --git a/src/python/grpcio/grpc/_cython/cygrpc.pyx b/src/python/grpcio/grpc/_cython/cygrpc.pyx index cf146f5a048..c92a8d19a77 100644 --- a/src/python/grpcio/grpc/_cython/cygrpc.pyx +++ b/src/python/grpcio/grpc/_cython/cygrpc.pyx @@ -45,30 +45,20 @@ include "grpc/_cython/_cygrpc/security.pyx.pxi" include "grpc/_cython/_cygrpc/server.pyx.pxi" # -# Global state +# initialize gRPC # -cdef class _ModuleState: - cdef bint is_loaded +def _initialize(): + if 'win32' in sys.platform: + filename = pkg_resources.resource_filename( + 'grpc._cython', '_windows/grpc_c.64.python') + if not pygrpc_load_core(filename): + raise ImportError('failed to load core gRPC library') + if not pygrpc_initialize_core(): + raise ImportError('failed to initialize core gRPC library') - def __cinit__(self): - if 'win32' in sys.platform: - filename = pkg_resources.resource_filename( - 'grpc._cython', '_windows/grpc_c.64.python') - if not pygrpc_load_core(filename): - raise ImportError('failed to load core gRPC library') - with nogil: - grpc_init() - self.is_loaded = True - with nogil: - grpc_set_ssl_roots_override_callback( + grpc_set_ssl_roots_override_callback( ssl_roots_override_callback) - def __dealloc__(self): - if self.is_loaded: - with nogil: - grpc_shutdown() - -_module_state = _ModuleState() - +_initialize() diff --git a/src/python/grpcio/grpc/_cython/loader.c b/src/python/grpcio/grpc/_cython/loader.c index b909ad594ed..86b70dbb02d 100644 --- a/src/python/grpcio/grpc/_cython/loader.c +++ b/src/python/grpcio/grpc/_cython/loader.c @@ -31,6 +31,7 @@ * */ +#include #include "loader.h" #ifdef __cplusplus @@ -62,6 +63,12 @@ int pygrpc_load_core(char *path) { return 1; } #endif /* !GPR_WINDOWS */ +// Cython doesn't have Py_AtExit bindings, so we call the C_API directly +int pygrpc_initialize_core(void) { + grpc_init(); + return Py_AtExit(grpc_shutdown) < 0 ? 0 : 1; +} + #ifdef __cplusplus } #endif /* __cpluslus */ diff --git a/src/python/grpcio/grpc/_cython/loader.h b/src/python/grpcio/grpc/_cython/loader.h index 3b8796d39f7..eb4b1a1b018 100644 --- a/src/python/grpcio/grpc/_cython/loader.h +++ b/src/python/grpcio/grpc/_cython/loader.h @@ -46,6 +46,11 @@ extern "C" { /* Attempts to load the core if necessary, and return non-zero upon succes. */ int pygrpc_load_core(char *path); +/* Initializes grpc and registers grpc_shutdown() to be called right before + * interpreter exit. Returns non-zero upon success. + */ +int pygrpc_initialize_core(void); + #ifdef __cplusplus } #endif /* __cpluslus */ From 2dfcf14705f24bdf6d95d1b16265beb52fe33b02 Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Fri, 24 Jun 2016 11:06:28 -0700 Subject: [PATCH 145/215] Fix Node Windows build error --- src/node/ext/node_grpc.cc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/node/ext/node_grpc.cc b/src/node/ext/node_grpc.cc index ce988f9dfa9..745b5023d59 100644 --- a/src/node/ext/node_grpc.cc +++ b/src/node/ext/node_grpc.cc @@ -265,8 +265,8 @@ void InitLogConstants(Local exports) { Nan::Set(log_verbosity, Nan::New("DEBUG").ToLocalChecked(), DEBUG); Local INFO(Nan::New(GPR_LOG_SEVERITY_INFO)); Nan::Set(log_verbosity, Nan::New("INFO").ToLocalChecked(), INFO); - Local ERROR(Nan::New(GPR_LOG_SEVERITY_ERROR)); - Nan::Set(log_verbosity, Nan::New("ERROR").ToLocalChecked(), ERROR); + Local LOG_ERROR(Nan::New(GPR_LOG_SEVERITY_ERROR)); + Nan::Set(log_verbosity, Nan::New("ERROR").ToLocalChecked(), LOG_ERROR); } NAN_METHOD(MetadataKeyIsLegal) { From f519df8147e80b6ef21a576e8e08742e4f957e87 Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Fri, 24 Jun 2016 11:12:03 -0700 Subject: [PATCH 146/215] Update version to 0.15.0 --- Makefile | 2 +- build.yaml | 2 +- package.json | 2 +- src/core/lib/surface/version.c | 2 +- src/csharp/Grpc.Auth/project.json | 4 ++-- src/csharp/Grpc.Core/VersionInfo.cs | 2 +- src/csharp/Grpc.Core/project.json | 2 +- src/csharp/Grpc.HealthCheck/project.json | 4 ++-- src/csharp/build_packages.bat | 2 +- src/node/tools/package.json | 2 +- src/python/grpcio/grpc_version.py | 2 +- src/ruby/lib/grpc/version.rb | 2 +- src/ruby/tools/version.rb | 2 +- tools/distrib/python/grpcio_tools/grpc_version.py | 2 +- tools/doxygen/Doxyfile.c++ | 2 +- tools/doxygen/Doxyfile.c++.internal | 2 +- tools/doxygen/Doxyfile.core | 2 +- tools/doxygen/Doxyfile.core.internal | 2 +- 18 files changed, 20 insertions(+), 20 deletions(-) diff --git a/Makefile b/Makefile index 9be3e5784cc..d369653be3c 100644 --- a/Makefile +++ b/Makefile @@ -414,7 +414,7 @@ E = @echo Q = @ endif -VERSION = 0.15.0-dev +VERSION = 0.15.0 CPPFLAGS_NO_ARCH += $(addprefix -I, $(INCLUDES)) $(addprefix -D, $(DEFINES)) CPPFLAGS += $(CPPFLAGS_NO_ARCH) $(ARCH_FLAGS) diff --git a/build.yaml b/build.yaml index 1f06e20cf9b..ffc681d9028 100644 --- a/build.yaml +++ b/build.yaml @@ -7,7 +7,7 @@ settings: '#3': Use "-preN" suffixes to identify pre-release versions '#4': Per-language overrides are possible with (eg) ruby_version tag here '#5': See the expand_version.py for all the quirks here - version: 0.15.0-dev + version: 0.15.0 filegroups: - name: census public_headers: diff --git a/package.json b/package.json index 5bdaa761e28..1b2920c6bc4 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "grpc", - "version": "0.15.0-dev", + "version": "0.15.0", "author": "Google Inc.", "description": "gRPC Library for Node", "homepage": "http://www.grpc.io/", diff --git a/src/core/lib/surface/version.c b/src/core/lib/surface/version.c index aca76d2bb79..e4a3358c351 100644 --- a/src/core/lib/surface/version.c +++ b/src/core/lib/surface/version.c @@ -36,4 +36,4 @@ #include -const char *grpc_version_string(void) { return "0.15.0-dev"; } +const char *grpc_version_string(void) { return "0.15.0"; } diff --git a/src/csharp/Grpc.Auth/project.json b/src/csharp/Grpc.Auth/project.json index 1677565824b..ae83c3be2b8 100644 --- a/src/csharp/Grpc.Auth/project.json +++ b/src/csharp/Grpc.Auth/project.json @@ -1,5 +1,5 @@ { - "version": "0.15.0-dev", + "version": "0.15.0", "title": "gRPC C# Auth", "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", @@ -13,7 +13,7 @@ "tags": [ "gRPC RPC Protocol HTTP/2 Auth OAuth2" ], }, "dependencies": { - "Grpc.Core": "0.15.0-dev", + "Grpc.Core": "0.15.0", "Google.Apis.Auth": "1.11.1" }, "frameworks": { diff --git a/src/csharp/Grpc.Core/VersionInfo.cs b/src/csharp/Grpc.Core/VersionInfo.cs index e1609341d9a..d89a2b5c6ee 100644 --- a/src/csharp/Grpc.Core/VersionInfo.cs +++ b/src/csharp/Grpc.Core/VersionInfo.cs @@ -53,6 +53,6 @@ namespace Grpc.Core /// /// Current version of gRPC C# /// - public const string CurrentVersion = "0.15.0-dev"; + public const string CurrentVersion = "0.15.0"; } } diff --git a/src/csharp/Grpc.Core/project.json b/src/csharp/Grpc.Core/project.json index 7253107e04a..c7f2bf4fb12 100644 --- a/src/csharp/Grpc.Core/project.json +++ b/src/csharp/Grpc.Core/project.json @@ -1,5 +1,5 @@ { - "version": "0.15.0-dev", + "version": "0.15.0", "title": "gRPC C# Core", "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", diff --git a/src/csharp/Grpc.HealthCheck/project.json b/src/csharp/Grpc.HealthCheck/project.json index eb57608957a..98ea21a436a 100644 --- a/src/csharp/Grpc.HealthCheck/project.json +++ b/src/csharp/Grpc.HealthCheck/project.json @@ -1,5 +1,5 @@ { - "version": "0.15.0-dev", + "version": "0.15.0", "title": "gRPC C# Healthchecking", "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", @@ -13,7 +13,7 @@ "tags": [ "gRPC health check" ] }, "dependencies": { - "Grpc.Core": "0.15.0-dev", + "Grpc.Core": "0.15.0", "Google.Protobuf": "3.0.0-beta3" }, "frameworks": { diff --git a/src/csharp/build_packages.bat b/src/csharp/build_packages.bat index 63f8c30bc7e..e387efcc2da 100644 --- a/src/csharp/build_packages.bat +++ b/src/csharp/build_packages.bat @@ -30,7 +30,7 @@ @rem Builds gRPC NuGet packages @rem Current package versions -set VERSION=0.15.0-dev +set VERSION=0.15.0 set PROTOBUF_VERSION=3.0.0-beta3 @rem Packages that depend on prerelease packages (like Google.Protobuf) need to have prerelease suffix as well. diff --git a/src/node/tools/package.json b/src/node/tools/package.json index c34b259e2e0..b2cadd3f47a 100644 --- a/src/node/tools/package.json +++ b/src/node/tools/package.json @@ -1,6 +1,6 @@ { "name": "grpc-tools", - "version": "0.15.0-dev", + "version": "0.15.0", "author": "Google Inc.", "description": "Tools for developing with gRPC on Node.js", "homepage": "http://www.grpc.io/", diff --git a/src/python/grpcio/grpc_version.py b/src/python/grpcio/grpc_version.py index 0c13104d9da..c6c07afb443 100644 --- a/src/python/grpcio/grpc_version.py +++ b/src/python/grpcio/grpc_version.py @@ -29,4 +29,4 @@ # AUTO-GENERATED FROM `$REPO_ROOT/templates/src/python/grpcio/grpc_version.py.template`!!! -VERSION='0.15.0.dev0' +VERSION='0.15.0' diff --git a/src/ruby/lib/grpc/version.rb b/src/ruby/lib/grpc/version.rb index 01c8c5ac8f1..7f512e47aab 100644 --- a/src/ruby/lib/grpc/version.rb +++ b/src/ruby/lib/grpc/version.rb @@ -29,5 +29,5 @@ # GRPC contains the General RPC module. module GRPC - VERSION = '0.15.0.dev' + VERSION = '0.15.0' end diff --git a/src/ruby/tools/version.rb b/src/ruby/tools/version.rb index dca7fd7e72c..6a7a1d5bd33 100644 --- a/src/ruby/tools/version.rb +++ b/src/ruby/tools/version.rb @@ -29,6 +29,6 @@ module GRPC module Tools - VERSION = '0.15.0.dev' + VERSION = '0.15.0' end end diff --git a/tools/distrib/python/grpcio_tools/grpc_version.py b/tools/distrib/python/grpcio_tools/grpc_version.py index 1267d0e45dc..9a33c6e5d14 100644 --- a/tools/distrib/python/grpcio_tools/grpc_version.py +++ b/tools/distrib/python/grpcio_tools/grpc_version.py @@ -29,4 +29,4 @@ # AUTO-GENERATED FROM `$REPO_ROOT/templates/tools/distrib/python/grpcio_tools/grpc_version.py.template`!!! -VERSION='0.15.0.dev0' +VERSION='0.15.0' diff --git a/tools/doxygen/Doxyfile.c++ b/tools/doxygen/Doxyfile.c++ index 7f9d2df6f6c..066d29ac001 100644 --- a/tools/doxygen/Doxyfile.c++ +++ b/tools/doxygen/Doxyfile.c++ @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC C++" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.15.0 # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a diff --git a/tools/doxygen/Doxyfile.c++.internal b/tools/doxygen/Doxyfile.c++.internal index dcf1a4c8c40..6a0e8b2129c 100644 --- a/tools/doxygen/Doxyfile.c++.internal +++ b/tools/doxygen/Doxyfile.c++.internal @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC C++" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.15.0 # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a diff --git a/tools/doxygen/Doxyfile.core b/tools/doxygen/Doxyfile.core index 72102b2fc50..fa9fd5a3123 100644 --- a/tools/doxygen/Doxyfile.core +++ b/tools/doxygen/Doxyfile.core @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC Core" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.15.0 # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal index e1555930e91..e4c9f991d31 100644 --- a/tools/doxygen/Doxyfile.core.internal +++ b/tools/doxygen/Doxyfile.core.internal @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC Core" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.15.0 # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a From be1b9a718a09722e95ab04b08ccaf353dc3a51e9 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Fri, 24 Jun 2016 13:22:11 -0700 Subject: [PATCH 147/215] Fixes --- src/core/lib/iomgr/error.c | 5 ++++- src/core/lib/iomgr/tcp_posix.c | 4 ++-- src/core/lib/surface/call.c | 5 +++-- 3 files changed, 9 insertions(+), 5 deletions(-) diff --git a/src/core/lib/iomgr/error.c b/src/core/lib/iomgr/error.c index 9f5ba76fd6c..aa94bb7fdf7 100644 --- a/src/core/lib/iomgr/error.c +++ b/src/core/lib/iomgr/error.c @@ -248,7 +248,10 @@ static grpc_error *copy_error_and_unref(grpc_error *in) { if (is_special(in)) { if (in == GRPC_ERROR_NONE) return GRPC_ERROR_CREATE("no error"); if (in == GRPC_ERROR_OOM) return GRPC_ERROR_CREATE("oom"); - if (in == GRPC_ERROR_CANCELLED) return GRPC_ERROR_CREATE("cancelled"); + if (in == GRPC_ERROR_CANCELLED) + return grpc_error_set_int(GRPC_ERROR_CREATE("cancelled"), + GRPC_ERROR_INT_GRPC_STATUS, + GRPC_STATUS_CANCELLED); return GRPC_ERROR_CREATE("unknown"); } grpc_error *out = gpr_malloc(sizeof(*out)); diff --git a/src/core/lib/iomgr/tcp_posix.c b/src/core/lib/iomgr/tcp_posix.c index 017f52c3677..1046b60bcc9 100644 --- a/src/core/lib/iomgr/tcp_posix.c +++ b/src/core/lib/iomgr/tcp_posix.c @@ -160,7 +160,7 @@ static void call_read_cb(grpc_exec_ctx *exec_ctx, grpc_tcp *tcp, grpc_error *error) { grpc_closure *cb = tcp->read_cb; - if (false && grpc_tcp_trace) { + if (grpc_tcp_trace) { size_t i; const char *str = grpc_error_string(error); gpr_log(GPR_DEBUG, "read: error=%s", str); @@ -394,7 +394,7 @@ static void tcp_write(grpc_exec_ctx *exec_ctx, grpc_endpoint *ep, grpc_tcp *tcp = (grpc_tcp *)ep; grpc_error *error = GRPC_ERROR_NONE; - if (false && grpc_tcp_trace) { + if (grpc_tcp_trace) { size_t i; for (i = 0; i < buf->count; i++) { diff --git a/src/core/lib/surface/call.c b/src/core/lib/surface/call.c index 77c17a4975b..708ea3502af 100644 --- a/src/core/lib/surface/call.c +++ b/src/core/lib/surface/call.c @@ -407,9 +407,10 @@ static void set_status_code(grpc_call *call, status_source source, static void set_status_details(grpc_call *call, status_source source, grpc_mdstr *status) { if (call->status[source].details != NULL) { - GRPC_MDSTR_UNREF(call->status[source].details); + GRPC_MDSTR_UNREF(status); + } else { + call->status[source].details = status; } - call->status[source].details = status; } static void get_final_status(grpc_call *call, From bd96e8a6a38c0afe01c8bfcb0849d7f10ff634e8 Mon Sep 17 00:00:00 2001 From: Ken Payson Date: Fri, 24 Jun 2016 11:53:54 -0700 Subject: [PATCH 148/215] Fix clang formatting --- .../ext/client_config/channel_connectivity.c | 7 ++++--- src/core/lib/profiling/basic_timers.c | 8 ++++---- .../security/credentials/jwt/jwt_credentials.c | 8 ++++---- src/core/lib/surface/channel.c | 16 +++++++++------- src/core/lib/surface/completion_queue.c | 14 ++++++++------ src/core/lib/transport/transport_op_string.c | 2 +- 6 files changed, 30 insertions(+), 25 deletions(-) diff --git a/src/core/ext/client_config/channel_connectivity.c b/src/core/ext/client_config/channel_connectivity.c index 2fd8e0e1232..c1220e3a8c3 100644 --- a/src/core/ext/client_config/channel_connectivity.c +++ b/src/core/ext/client_config/channel_connectivity.c @@ -189,10 +189,11 @@ void grpc_channel_watch_connectivity_state( GRPC_API_TRACE( "grpc_channel_watch_connectivity_state(" "channel=%p, last_observed_state=%d, " - "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %" PRId64 + ", tv_nsec: %d, clock_type: %d }, " "cq=%p, tag=%p)", - 7, (channel, (int)last_observed_state, deadline.tv_sec, - deadline.tv_nsec, (int)deadline.clock_type, cq, tag)); + 7, (channel, (int)last_observed_state, deadline.tv_sec, deadline.tv_nsec, + (int)deadline.clock_type, cq, tag)); grpc_cq_begin_op(cq, tag); diff --git a/src/core/lib/profiling/basic_timers.c b/src/core/lib/profiling/basic_timers.c index ee464f310f4..51813d04617 100644 --- a/src/core/lib/profiling/basic_timers.c +++ b/src/core/lib/profiling/basic_timers.c @@ -141,11 +141,11 @@ static void write_log(gpr_timer_log *log) { entry->tm = gpr_time_0(entry->tm.clock_type); } fprintf(output_file, - "{\"t\": %"PRId64".%09d, \"thd\": \"%d\", \"type\": \"%c\", \"tag\": " + "{\"t\": %" PRId64 + ".%09d, \"thd\": \"%d\", \"type\": \"%c\", \"tag\": " "\"%s\", \"file\": \"%s\", \"line\": %d, \"imp\": %d}\n", - entry->tm.tv_sec, entry->tm.tv_nsec, entry->thd, - entry->type, entry->tagstr, entry->file, entry->line, - entry->important); + entry->tm.tv_sec, entry->tm.tv_nsec, entry->thd, entry->type, + entry->tagstr, entry->file, entry->line, entry->important); } } diff --git a/src/core/lib/security/credentials/jwt/jwt_credentials.c b/src/core/lib/security/credentials/jwt/jwt_credentials.c index f4dd9208610..1ae9f701fbd 100644 --- a/src/core/lib/security/credentials/jwt/jwt_credentials.c +++ b/src/core/lib/security/credentials/jwt/jwt_credentials.c @@ -149,11 +149,11 @@ grpc_call_credentials *grpc_service_account_jwt_access_credentials_create( "grpc_service_account_jwt_access_credentials_create(" "json_key=%s, " "token_lifetime=" - "gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " + "gpr_timespec { tv_sec: %" PRId64 + ", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", - 5, - (json_key, token_lifetime.tv_sec, token_lifetime.tv_nsec, - (int)token_lifetime.clock_type, reserved)); + 5, (json_key, token_lifetime.tv_sec, token_lifetime.tv_nsec, + (int)token_lifetime.clock_type, reserved)); GPR_ASSERT(reserved == NULL); return grpc_service_account_jwt_access_credentials_create_from_auth_json_key( grpc_auth_json_key_create_from_string(json_key), token_lifetime); diff --git a/src/core/lib/surface/channel.c b/src/core/lib/surface/channel.c index 952a769de1d..2cf6d8890a3 100644 --- a/src/core/lib/surface/channel.c +++ b/src/core/lib/surface/channel.c @@ -223,11 +223,12 @@ grpc_call *grpc_channel_create_call(grpc_channel *channel, "grpc_channel_create_call(" "channel=%p, parent_call=%p, propagation_mask=%x, cq=%p, method=%s, " "host=%s, " - "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %" PRId64 + ", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", - 10, (channel, parent_call, (unsigned)propagation_mask, cq, method, host, - deadline.tv_sec, deadline.tv_nsec, - (int)deadline.clock_type, reserved)); + 10, + (channel, parent_call, (unsigned)propagation_mask, cq, method, host, + deadline.tv_sec, deadline.tv_nsec, (int)deadline.clock_type, reserved)); GPR_ASSERT(!reserved); return grpc_channel_create_call_internal( channel, parent_call, propagation_mask, cq, NULL, @@ -282,11 +283,12 @@ grpc_call *grpc_channel_create_registered_call( "grpc_channel_create_registered_call(" "channel=%p, parent_call=%p, propagation_mask=%x, completion_queue=%p, " "registered_call_handle=%p, " - "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %" PRId64 + ", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", 9, (channel, parent_call, (unsigned)propagation_mask, completion_queue, - registered_call_handle, deadline.tv_sec, - deadline.tv_nsec, (int)deadline.clock_type, reserved)); + registered_call_handle, deadline.tv_sec, deadline.tv_nsec, + (int)deadline.clock_type, reserved)); GPR_ASSERT(!reserved); return grpc_channel_create_call_internal( channel, parent_call, propagation_mask, completion_queue, NULL, diff --git a/src/core/lib/surface/completion_queue.c b/src/core/lib/surface/completion_queue.c index aa35ae02fe2..b5f2f65e5c0 100644 --- a/src/core/lib/surface/completion_queue.c +++ b/src/core/lib/surface/completion_queue.c @@ -316,10 +316,11 @@ grpc_event grpc_completion_queue_next(grpc_completion_queue *cc, GRPC_API_TRACE( "grpc_completion_queue_next(" "cc=%p, " - "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %" PRId64 + ", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", - 5, (cc, deadline.tv_sec, deadline.tv_nsec, - (int)deadline.clock_type, reserved)); + 5, (cc, deadline.tv_sec, deadline.tv_nsec, (int)deadline.clock_type, + reserved)); GPR_ASSERT(!reserved); deadline = gpr_convert_clock_type(deadline, GPR_CLOCK_MONOTONIC); @@ -428,10 +429,11 @@ grpc_event grpc_completion_queue_pluck(grpc_completion_queue *cc, void *tag, GRPC_API_TRACE( "grpc_completion_queue_pluck(" "cc=%p, tag=%p, " - "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %" PRId64 + ", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", - 6, (cc, tag, deadline.tv_sec, deadline.tv_nsec, - (int)deadline.clock_type, reserved)); + 6, (cc, tag, deadline.tv_sec, deadline.tv_nsec, (int)deadline.clock_type, + reserved)); GPR_ASSERT(!reserved); deadline = gpr_convert_clock_type(deadline, GPR_CLOCK_MONOTONIC); diff --git a/src/core/lib/transport/transport_op_string.c b/src/core/lib/transport/transport_op_string.c index d8fae7805e4..aeaba5339fd 100644 --- a/src/core/lib/transport/transport_op_string.c +++ b/src/core/lib/transport/transport_op_string.c @@ -63,7 +63,7 @@ static void put_metadata_list(gpr_strvec *b, grpc_metadata_batch md) { } if (gpr_time_cmp(md.deadline, gpr_inf_future(md.deadline.clock_type)) != 0) { char *tmp; - gpr_asprintf(&tmp, " deadline=%"PRId64".%09d", md.deadline.tv_sec, + gpr_asprintf(&tmp, " deadline=%" PRId64 ".%09d", md.deadline.tv_sec, md.deadline.tv_nsec); gpr_strvec_add(b, tmp); } From ddf02c1c3ff6efb15a9c47b5da7bd76b3654ad94 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Fri, 24 Jun 2016 14:04:01 -0700 Subject: [PATCH 149/215] Fix bad indentation --- tools/run_tests/jobset.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/run_tests/jobset.py b/tools/run_tests/jobset.py index 40409c43948..4fe77487f96 100755 --- a/tools/run_tests/jobset.py +++ b/tools/run_tests/jobset.py @@ -363,7 +363,7 @@ class Jobset(object): self._running.add(job) if not self.resultset.has_key(job.GetSpec().shortname): self.resultset[job.GetSpec().shortname] = [] - return True + return True def reap(self): """Collect the dead jobs.""" From d3b0dda9b8c5ac7dda843e46b240e99a3973d3b6 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Fri, 24 Jun 2016 15:13:08 -0700 Subject: [PATCH 150/215] update grpc nugets to 0.15 --- examples/csharp/helloworld/.nuget/packages.config | 2 +- examples/csharp/helloworld/Greeter/Greeter.csproj | 10 +++++----- examples/csharp/helloworld/Greeter/packages.config | 6 +++--- .../helloworld/GreeterClient/GreeterClient.csproj | 10 +++++----- .../csharp/helloworld/GreeterClient/packages.config | 6 +++--- .../helloworld/GreeterServer/GreeterServer.csproj | 10 +++++----- .../csharp/helloworld/GreeterServer/packages.config | 6 +++--- examples/csharp/helloworld/generate_protos.bat | 2 +- examples/csharp/route_guide/.nuget/packages.config | 2 +- .../csharp/route_guide/RouteGuide/RouteGuide.csproj | 10 +++++----- examples/csharp/route_guide/RouteGuide/packages.config | 6 +++--- .../RouteGuideClient/RouteGuideClient.csproj | 10 +++++----- .../route_guide/RouteGuideClient/packages.config | 6 +++--- .../RouteGuideServer/RouteGuideServer.csproj | 10 +++++----- .../route_guide/RouteGuideServer/packages.config | 6 +++--- examples/csharp/route_guide/generate_protos.bat | 2 +- 16 files changed, 52 insertions(+), 52 deletions(-) diff --git a/examples/csharp/helloworld/.nuget/packages.config b/examples/csharp/helloworld/.nuget/packages.config index bfd6c6723d9..aa060800c19 100644 --- a/examples/csharp/helloworld/.nuget/packages.config +++ b/examples/csharp/helloworld/.nuget/packages.config @@ -1,4 +1,4 @@  - + \ No newline at end of file diff --git a/examples/csharp/helloworld/Greeter/Greeter.csproj b/examples/csharp/helloworld/Greeter/Greeter.csproj index 0270cc25f7a..20b85db8b60 100644 --- a/examples/csharp/helloworld/Greeter/Greeter.csproj +++ b/examples/csharp/helloworld/Greeter/Greeter.csproj @@ -10,7 +10,7 @@ Greeter Greeter v4.5 - 745ac60f + 2669b4f2 true @@ -33,11 +33,11 @@ False - ..\packages\Google.Protobuf.3.0.0-beta2\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll + ..\packages\Google.Protobuf.3.0.0-beta3\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll False - ..\packages\Grpc.Core.0.14.0\lib\net45\Grpc.Core.dll + ..\packages\Grpc.Core.0.15.0\lib\net45\Grpc.Core.dll @@ -61,11 +61,11 @@ - + This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. - + \ No newline at end of file diff --git a/examples/csharp/helloworld/Greeter/packages.config b/examples/csharp/helloworld/Greeter/packages.config index 617fe6da7be..ff9d6bbf73f 100644 --- a/examples/csharp/helloworld/Greeter/packages.config +++ b/examples/csharp/helloworld/Greeter/packages.config @@ -1,7 +1,7 @@  - - - + + + \ No newline at end of file diff --git a/examples/csharp/helloworld/GreeterClient/GreeterClient.csproj b/examples/csharp/helloworld/GreeterClient/GreeterClient.csproj index 877c450a50e..2b38ce290e9 100644 --- a/examples/csharp/helloworld/GreeterClient/GreeterClient.csproj +++ b/examples/csharp/helloworld/GreeterClient/GreeterClient.csproj @@ -10,7 +10,7 @@ GreeterClient GreeterClient v4.5 - 63b59176 + 5e942a7d true @@ -33,11 +33,11 @@ False - ..\packages\Google.Protobuf.3.0.0-beta2\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll + ..\packages\Google.Protobuf.3.0.0-beta3\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll False - ..\packages\Grpc.Core.0.14.0\lib\net45\Grpc.Core.dll + ..\packages\Grpc.Core.0.15.0\lib\net45\Grpc.Core.dll @@ -59,11 +59,11 @@ - + This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. - + \ No newline at end of file diff --git a/examples/csharp/helloworld/GreeterClient/packages.config b/examples/csharp/helloworld/GreeterClient/packages.config index 617fe6da7be..ff9d6bbf73f 100644 --- a/examples/csharp/helloworld/GreeterClient/packages.config +++ b/examples/csharp/helloworld/GreeterClient/packages.config @@ -1,7 +1,7 @@  - - - + + + \ No newline at end of file diff --git a/examples/csharp/helloworld/GreeterServer/GreeterServer.csproj b/examples/csharp/helloworld/GreeterServer/GreeterServer.csproj index 4d792dcf32e..43c633678b1 100644 --- a/examples/csharp/helloworld/GreeterServer/GreeterServer.csproj +++ b/examples/csharp/helloworld/GreeterServer/GreeterServer.csproj @@ -10,7 +10,7 @@ GreeterServer GreeterServer v4.5 - 25ac2e80 + 9c7b2963 true @@ -33,11 +33,11 @@ False - ..\packages\Google.Protobuf.3.0.0-beta2\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll + ..\packages\Google.Protobuf.3.0.0-beta3\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll False - ..\packages\Grpc.Core.0.14.0\lib\net45\Grpc.Core.dll + ..\packages\Grpc.Core.0.15.0\lib\net45\Grpc.Core.dll @@ -59,11 +59,11 @@ - + This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. - + \ No newline at end of file diff --git a/examples/csharp/helloworld/GreeterServer/packages.config b/examples/csharp/helloworld/GreeterServer/packages.config index 617fe6da7be..ff9d6bbf73f 100644 --- a/examples/csharp/helloworld/GreeterServer/packages.config +++ b/examples/csharp/helloworld/GreeterServer/packages.config @@ -1,7 +1,7 @@  - - - + + + \ No newline at end of file diff --git a/examples/csharp/helloworld/generate_protos.bat b/examples/csharp/helloworld/generate_protos.bat index b9f68d6745a..a952bb46cdc 100644 --- a/examples/csharp/helloworld/generate_protos.bat +++ b/examples/csharp/helloworld/generate_protos.bat @@ -34,7 +34,7 @@ setlocal @rem enter this directory cd /d %~dp0 -set TOOLS_PATH=packages\Grpc.Tools.0.14.0\tools\windows_x86 +set TOOLS_PATH=packages\Grpc.Tools.0.15.0\tools\windows_x86 %TOOLS_PATH%\protoc.exe -I../../protos --csharp_out Greeter ../../protos/helloworld.proto --grpc_out Greeter --plugin=protoc-gen-grpc=%TOOLS_PATH%\grpc_csharp_plugin.exe diff --git a/examples/csharp/route_guide/.nuget/packages.config b/examples/csharp/route_guide/.nuget/packages.config index bfd6c6723d9..aa060800c19 100644 --- a/examples/csharp/route_guide/.nuget/packages.config +++ b/examples/csharp/route_guide/.nuget/packages.config @@ -1,4 +1,4 @@  - + \ No newline at end of file diff --git a/examples/csharp/route_guide/RouteGuide/RouteGuide.csproj b/examples/csharp/route_guide/RouteGuide/RouteGuide.csproj index 4f7222ebba3..601d16ba24d 100644 --- a/examples/csharp/route_guide/RouteGuide/RouteGuide.csproj +++ b/examples/csharp/route_guide/RouteGuide/RouteGuide.csproj @@ -11,7 +11,7 @@ RouteGuide v4.5 512 - 0a9fcb7a + de2137f9 true @@ -33,11 +33,11 @@ False - ..\packages\Google.Protobuf.3.0.0-beta2\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll + ..\packages\Google.Protobuf.3.0.0-beta3\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll False - ..\packages\Grpc.Core.0.14.0\lib\net45\Grpc.Core.dll + ..\packages\Grpc.Core.0.15.0\lib\net45\Grpc.Core.dll False @@ -74,12 +74,12 @@ - + This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. - +