From 3165ff3d95d46e3e8f091bc64ad88e7e52dd8a65 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 8 Mar 2016 00:53:53 -0800 Subject: [PATCH 001/280] Added compression spec --- doc/PROTOCOL-HTTP2.md | 4 +- doc/compression.md | 105 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 107 insertions(+), 2 deletions(-) create mode 100644 doc/compression.md diff --git a/doc/PROTOCOL-HTTP2.md b/doc/PROTOCOL-HTTP2.md index 357ea72571b..31d694b803d 100644 --- a/doc/PROTOCOL-HTTP2.md +++ b/doc/PROTOCOL-HTTP2.md @@ -38,7 +38,7 @@ Request-Headers are delivered as HTTP2 headers in HEADERS + CONTINUATION frames. * **Nanosecond** → "n" * **Content-Type** → "content-type" "application/grpc" [("+proto" / "+json" / {_custom_})] * **Content-Coding** → "identity" / "gzip" / "deflate" / "snappy" / {_custom_} -* **Message-Encoding** → "grpc-encoding" Content-Coding +* **Message-Encoding** → "grpc-encoding" Content-Coding * **Message-Accept-Encoding** → "grpc-accept-encoding" Content-Coding \*("," Content-Coding) * **User-Agent** → "user-agent" {_structured user-agent string_} * **Message-Type** → "grpc-message-type" {_type name for message schema_} @@ -83,7 +83,7 @@ binary values' lengths being post-Base64. The repeated sequence of **Length-Prefixed-Message** items is delivered in DATA frames * **Length-Prefixed-Message** → Compressed-Flag Message-Length Message -* **Compressed-Flag** → 0 / 1 # encoded as 1 byte unsigned integer +* **Compressed-Flag** → 0 / 1 # encoded as 1 byte unsigned integer * **Message-Length** → {_length of Message_} # encoded as 4 byte unsigned integer * **Message** → \*{binary octet} diff --git a/doc/compression.md b/doc/compression.md new file mode 100644 index 00000000000..ddbbff1c541 --- /dev/null +++ b/doc/compression.md @@ -0,0 +1,105 @@ +## **gRPC Compression** + +The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", +"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be +interpreted as described in [RFC 2119](http://www.ietf.org/rfc/rfc2119.txt). + +### Intent + +Compression is used to reduce the amount of bandwidth used between peers. The +compression supported by gRPC acts _at the individual message level_, taking +_message_ [as defined in the wire format +document](PROTOCOL-HTTP2.md). + +The implementation supports different compression algorithms. A _default +compression level_, to be used in the absence of message-specific settings, MAY +be specified for during channel creation. + +The ability to control compression settings per call and to enable/disable +compression on a per message basis MAY be used to prevent CRIME/BEAST attacks. +It also allows for asymmetric compression communication, whereby a response MAY +be compressed differently, if at all. + +### Specification + +Compression MAY be configured by the Client Application by calling the +appropriate API method. There are two scenarios where compression MAY be +configured: + ++ At channel creation time, which sets the channel default compression and + therefore the compression that SHALL be used in the absence of per-RPC + compression configuration. ++ At response time, via: + + For unary RPCs, the {Client,Server}Context instance. + + For streaming RPCs, the {Client,Server}Writer instance. In this case, + configuration is reduced to disabling compression altogether. + +### Compression Method Asymmetry Between Peers + +A gRPC peer MAY choose to respond using a different compression method to that +of the request, including not performing any compression, regardless of channel +and RPC settings (for example, if compression would result in small or negative +gains). + +A compressed message from a client with an algorithm unsupported by a server, +WILL result in an INVALID\_ARGUMENT error, alongside the receiving peer's +`grpc-accept-encoding` header specifying the algorithms it accepts. If an +INTERNAL error is returned from the server despite having used one of the +algorithms from the `grpc-accept-encoding h`eader, the cause MUST NOT be related +to compression. Data sent from a server compressed with an algorithm not +supported by the client will also result in an INTERNAL error. + +Note that a peer MAY choose to not disclose all the encodings it supports. +However, if it receives a message compressed in an undisclosed but supported +encoding, it MUST include said encoding in the response's `grpc-accept-encoding +h`eader. + +For every message a server is requested to compress using an algorithm it knows +the client doesn't support (as indicated by the last `grpc-accept-encoding` +header received from the client), it SHALL send the message uncompressed. + +### Specific Disabling of Compression + +If the user (through the previously described mechanisms) requests to disable +compression the next message MUST be sent uncompressed. This is instrumental in +preventing BEAST/CRIME attacks. This applies to both the the unary and streaming +cases. + +### Compression Levels and Algorithms + +We currently (as of July 2015) support _gzip_ and _deflate_ as algorithms (with +the possible future addition of snappy). In order to simplify the public API, +it's intended to abstract the algorithms as _compression levels_ (such as "low", +"medium", "high") that'd map to concrete algorithms and/or their settings (such +as "low" mapping to "gzip -3" and "high" mapping to "gzip -9"). However, we +can't presently (July 2015) implement said compression levels at the client side +without either a initial negotiation of capabilities or an automatic retry +mechanism. Therefore, compression levels are only supported at the server side, +which is aware of the client's capabilities by virtue of the incoming +Message-Accept-Encoding header. + +### Propagation to child RPCs + +The inheritance of the compression configuration by child RPCs is left up to the +implementation. Note that in the absence of changes to the parent channel, its +configuration will be used. + +### Test cases + +1. When a compression level is not specified for either the channel or the +message, the default channel level _none_ is considered: data MUST NOT be +compressed. +1. When per-RPC compression configuration isn't present for a message, the +channel compression configuration MUST be used. +1. When a compression method (including no compression) is specified for an +outgoing message, the message MUST be compressed accordingly. +1. A message compressed in a way not supported by its endpoint MUST fail with +INVALID\_ARGUMENT status, its associated description indicating the unsupported +condition as well as the supported ones. The returned `grpc-accept-encoding` +header MUST NOT contain the compression method (encoding) used. +1. An ill-constructed message with its [Compressed-Flag +bit](PROTOCOL-HTTP2.md#compressed-flag) +set but lacking a +"[grpc-encoding](PROTOCOL-HTTP2.md#message-encoding)" +entry different from _identity_ in its metadata MUST fail with INTERNAL status, +its associated description indicating the invalid Compressed-Flag condition. From 9a4d0893881f6811646695cc73fe9752fed308ae Mon Sep 17 00:00:00 2001 From: Yosuke Ishikawa Date: Fri, 29 Apr 2016 01:05:39 +0900 Subject: [PATCH 002/280] Fix build error caused by immutability of value type --- src/objective-c/examples/SwiftSample/ViewController.swift | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/objective-c/examples/SwiftSample/ViewController.swift b/src/objective-c/examples/SwiftSample/ViewController.swift index a21ce07978d..80d7a47917f 100644 --- a/src/objective-c/examples/SwiftSample/ViewController.swift +++ b/src/objective-c/examples/SwiftSample/ViewController.swift @@ -71,7 +71,7 @@ class ViewController: UIViewController { NSLog("2. Response trailers: \(RPC.responseTrailers)") } - RPC.requestHeaders["My-Header"] = "My value" + RPC.requestHeaders.setObject("My value", forKey: "My-Header") RPC.start() @@ -84,7 +84,7 @@ class ViewController: UIViewController { let call = GRPCCall(host: RemoteHost, path: method.HTTPPath, requestsWriter: requestsWriter) - call.requestHeaders["My-Header"] = "My value" + call.requestHeaders.setObject("My value", forKey: "My-Header") call.startWithWriteable(GRXWriteable { response, error in if let response = response as? NSData { From 454432542a6cc6145621003ecd8303c82a4e2b7b Mon Sep 17 00:00:00 2001 From: Jorge Canizales Date: Fri, 29 Apr 2016 16:19:51 -0700 Subject: [PATCH 003/280] Add a maybe-temporary way for apps to clear the channel cache --- .../GRPCClient/GRPCCall+ChannelArg.h | 5 +++++ .../GRPCClient/GRPCCall+ChannelArg.m | 4 ++++ src/objective-c/GRPCClient/private/GRPCHost.h | 2 ++ src/objective-c/GRPCClient/private/GRPCHost.m | 21 ++++++++++++++----- 4 files changed, 27 insertions(+), 5 deletions(-) diff --git a/src/objective-c/GRPCClient/GRPCCall+ChannelArg.h b/src/objective-c/GRPCClient/GRPCCall+ChannelArg.h index bd6b064f166..646bf43b547 100644 --- a/src/objective-c/GRPCClient/GRPCCall+ChannelArg.h +++ b/src/objective-c/GRPCClient/GRPCCall+ChannelArg.h @@ -32,6 +32,8 @@ */ #import "GRPCCall.h" +#include + /** * Methods to configure GRPC channel options. */ @@ -43,4 +45,7 @@ */ + (void)setUserAgentPrefix:(NSString *)userAgentPrefix forHost:(NSString *)host; ++ (void)closeOpenConnections DEPRECATED_MSG_ATTRIBUTE("The API for this feature is experimental, " + "and might be removed or modified at any " + "time."); @end diff --git a/src/objective-c/GRPCClient/GRPCCall+ChannelArg.m b/src/objective-c/GRPCClient/GRPCCall+ChannelArg.m index 5f9932d86dc..bcc3b915075 100644 --- a/src/objective-c/GRPCClient/GRPCCall+ChannelArg.m +++ b/src/objective-c/GRPCClient/GRPCCall+ChannelArg.m @@ -46,4 +46,8 @@ hostConfig.userAgentPrefix = userAgentPrefix; } ++ (void)closeOpenConnections { + [GRPCHost flushChannelCache]; +} + @end diff --git a/src/objective-c/GRPCClient/private/GRPCHost.h b/src/objective-c/GRPCClient/private/GRPCHost.h index 9220e2a33db..350c69bf8e8 100644 --- a/src/objective-c/GRPCClient/private/GRPCHost.h +++ b/src/objective-c/GRPCClient/private/GRPCHost.h @@ -41,6 +41,8 @@ struct grpc_channel_credentials; @interface GRPCHost : NSObject ++ (void)flushChannelCache; + @property(nonatomic, readonly) NSString *address; @property(nonatomic, copy, nullable) NSString *userAgentPrefix; @property(nonatomic, nullable) struct grpc_channel_credentials *channelCreds; diff --git a/src/objective-c/GRPCClient/private/GRPCHost.m b/src/objective-c/GRPCClient/private/GRPCHost.m index 43166cbb527..0358cc62365 100644 --- a/src/objective-c/GRPCClient/private/GRPCHost.m +++ b/src/objective-c/GRPCClient/private/GRPCHost.m @@ -48,6 +48,8 @@ NS_ASSUME_NONNULL_BEGIN // templates/src/core/surface/version.c.template . #define GRPC_OBJC_VERSION_STRING @"0.13.0" +static NSMutableDictionary *kHostCache; + @implementation GRPCHost { // TODO(mlumish): Investigate whether caching channels with strong links is a good idea. GRPCChannel *_channel; @@ -79,13 +81,12 @@ NS_ASSUME_NONNULL_BEGIN } // Look up the GRPCHost in the cache. - static NSMutableDictionary *hostCache; static dispatch_once_t cacheInitialization; dispatch_once(&cacheInitialization, ^{ - hostCache = [NSMutableDictionary dictionary]; + kHostCache = [NSMutableDictionary dictionary]; }); - @synchronized(hostCache) { - GRPCHost *cachedHost = hostCache[address]; + @synchronized(kHostCache) { + GRPCHost *cachedHost = kHostCache[address]; if (cachedHost) { return cachedHost; } @@ -93,12 +94,22 @@ NS_ASSUME_NONNULL_BEGIN if ((self = [super init])) { _address = address; _secure = YES; - hostCache[address] = self; + kHostCache[address] = self; } } return self; } ++ (void)flushChannelCache { + @synchronized(kHostCache) { + [kHostCache enumerateKeysAndObjectsUsingBlock:^(id _Nonnull key, + GRPCHost * _Nonnull host, + BOOL * _Nonnull stop) { + [host disconnect]; + }]; + } +} + - (nullable grpc_call *)unmanagedCallWithPath:(NSString *)path completionQueue:(GRPCCompletionQueue *)queue { GRPCChannel *channel; From fa70dacf95b93486b7dfe0c21ada90d75a5d5bcd Mon Sep 17 00:00:00 2001 From: Jorge Canizales Date: Fri, 29 Apr 2016 19:58:52 -0700 Subject: [PATCH 004/280] =?UTF-8?q?Smoke=20test=20that=20things=20still=20?= =?UTF-8?q?work=20after=20=E2=80=9CcloseOpenConnections=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/objective-c/tests/InteropTests.m | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/src/objective-c/tests/InteropTests.m b/src/objective-c/tests/InteropTests.m index 26877b1ae84..379271a312d 100644 --- a/src/objective-c/tests/InteropTests.m +++ b/src/objective-c/tests/InteropTests.m @@ -35,6 +35,7 @@ #include +#import #import #import #import @@ -312,4 +313,25 @@ [self waitForExpectationsWithTimeout:8 handler:nil]; } +- (void)testRPCAfterClosingOpenConnections { + XCTAssertNotNil(self.class.host); + __weak XCTestExpectation *expectation = + [self expectationWithDescription:@"RPC after closing connection"]; + + RMTEmpty *request = [RMTEmpty message]; + + [_service emptyCallWithRequest:request handler:^(RMTEmpty *response, NSError *error) { + XCTAssertNil(error, @"First RPC finished with unexpected error: %@", error); + + [GRPCCall closeOpenConnections]; + + [_service emptyCallWithRequest:request handler:^(RMTEmpty *response, NSError *error) { + XCTAssertNil(error, @"Second RPC finished with unexpected error: %@", error); + [expectation fulfill]; + }]; + }]; + + [self waitForExpectationsWithTimeout:4 handler:nil]; +} + @end From 2e3c9ad6dd6fbf4f7532eedd9d5d2b52e7a3eb1f Mon Sep 17 00:00:00 2001 From: Julien Boeuf Date: Tue, 19 Jan 2016 17:14:38 -0800 Subject: [PATCH 005/280] Starting the work to fix #3803. - We still need a way to bubble up this error. --- src/core/lib/security/client_auth_filter.c | 3 +- src/core/lib/security/credentials.c | 35 +++++++------ src/core/lib/security/credentials.h | 4 +- test/core/security/credentials_test.c | 49 ++++++++++--------- test/core/security/oauth2_utils.c | 3 +- .../print_google_default_creds_token.c | 3 +- 6 files changed, 54 insertions(+), 43 deletions(-) diff --git a/src/core/lib/security/client_auth_filter.c b/src/core/lib/security/client_auth_filter.c index 8b58cb86bf9..f6de877021e 100644 --- a/src/core/lib/security/client_auth_filter.c +++ b/src/core/lib/security/client_auth_filter.c @@ -98,7 +98,8 @@ static void bubble_up_error(grpc_exec_ctx *exec_ctx, grpc_call_element *elem, static void on_credentials_metadata(grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, size_t num_md, - grpc_credentials_status status) { + grpc_credentials_status status, + const char *error_details) { grpc_call_element *elem = (grpc_call_element *)user_data; call_data *calld = elem->call_data; grpc_transport_stream_op *op = &calld->op; diff --git a/src/core/lib/security/credentials.c b/src/core/lib/security/credentials.c index fd5ad3589b7..1c9832333aa 100644 --- a/src/core/lib/security/credentials.c +++ b/src/core/lib/security/credentials.c @@ -122,7 +122,7 @@ void grpc_call_credentials_get_request_metadata( grpc_credentials_metadata_cb cb, void *user_data) { if (creds == NULL || creds->vtable->get_request_metadata == NULL) { if (cb != NULL) { - cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_OK); + cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_OK, NULL); } return; } @@ -497,10 +497,10 @@ static void jwt_get_request_metadata(grpc_exec_ctx *exec_ctx, if (jwt_md != NULL) { cb(exec_ctx, user_data, jwt_md->entries, jwt_md->num_entries, - GRPC_CREDENTIALS_OK); + GRPC_CREDENTIALS_OK, NULL); grpc_credentials_md_store_unref(jwt_md); } else { - cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_ERROR); + cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_ERROR, ""); } } @@ -660,10 +660,10 @@ static void on_oauth2_token_fetcher_http_response( c->token_expiration = gpr_time_add(gpr_now(GPR_CLOCK_REALTIME), token_lifetime); r->cb(exec_ctx, r->user_data, c->access_token_md->entries, - c->access_token_md->num_entries, status); + c->access_token_md->num_entries, status, NULL); } else { c->token_expiration = gpr_inf_past(GPR_CLOCK_REALTIME); - r->cb(exec_ctx, r->user_data, NULL, 0, status); + r->cb(exec_ctx, r->user_data, NULL, 0, status, ""); } gpr_mu_unlock(&c->mu); grpc_credentials_metadata_request_destroy(r); @@ -691,7 +691,7 @@ static void oauth2_token_fetcher_get_request_metadata( } if (cached_access_token_md != NULL) { cb(exec_ctx, user_data, cached_access_token_md->entries, - cached_access_token_md->num_entries, GRPC_CREDENTIALS_OK); + cached_access_token_md->num_entries, GRPC_CREDENTIALS_OK, NULL); grpc_credentials_md_store_unref(cached_access_token_md); } else { c->fetch_func( @@ -821,7 +821,7 @@ static void on_simulated_token_fetch_done(grpc_exec_ctx *exec_ctx, (grpc_credentials_metadata_request *)user_data; grpc_md_only_test_credentials *c = (grpc_md_only_test_credentials *)r->creds; r->cb(exec_ctx, r->user_data, c->md_store->entries, c->md_store->num_entries, - GRPC_CREDENTIALS_OK); + GRPC_CREDENTIALS_OK, NULL); grpc_credentials_metadata_request_destroy(r); } @@ -837,7 +837,7 @@ static void md_only_test_get_request_metadata( grpc_executor_enqueue( grpc_closure_create(on_simulated_token_fetch_done, cb_arg), true); } else { - cb(exec_ctx, user_data, c->md_store->entries, 1, GRPC_CREDENTIALS_OK); + cb(exec_ctx, user_data, c->md_store->entries, 1, GRPC_CREDENTIALS_OK, NULL); } } @@ -870,7 +870,8 @@ static void access_token_get_request_metadata( grpc_pollset *pollset, grpc_auth_metadata_context context, grpc_credentials_metadata_cb cb, void *user_data) { grpc_access_token_credentials *c = (grpc_access_token_credentials *)creds; - cb(exec_ctx, user_data, c->access_token_md->entries, 1, GRPC_CREDENTIALS_OK); + cb(exec_ctx, user_data, c->access_token_md->entries, 1, GRPC_CREDENTIALS_OK, + NULL); } static grpc_call_credentials_vtable access_token_vtable = { @@ -973,11 +974,12 @@ static void composite_call_md_context_destroy( static void composite_call_metadata_cb(grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, size_t num_md, - grpc_credentials_status status) { + grpc_credentials_status status, + const char *error_details) { grpc_composite_call_credentials_metadata_context *ctx = (grpc_composite_call_credentials_metadata_context *)user_data; if (status != GRPC_CREDENTIALS_OK) { - ctx->cb(exec_ctx, ctx->user_data, NULL, 0, status); + ctx->cb(exec_ctx, ctx->user_data, NULL, 0, status, NULL); return; } @@ -1002,7 +1004,7 @@ static void composite_call_metadata_cb(grpc_exec_ctx *exec_ctx, void *user_data, /* We're done!. */ ctx->cb(exec_ctx, ctx->user_data, ctx->md_elems->entries, - ctx->md_elems->num_entries, GRPC_CREDENTIALS_OK); + ctx->md_elems->num_entries, GRPC_CREDENTIALS_OK, NULL); composite_call_md_context_destroy(ctx); } @@ -1122,7 +1124,7 @@ static void iam_get_request_metadata(grpc_exec_ctx *exec_ctx, void *user_data) { grpc_google_iam_credentials *c = (grpc_google_iam_credentials *)creds; cb(exec_ctx, user_data, c->iam_md->entries, c->iam_md->num_entries, - GRPC_CREDENTIALS_OK); + GRPC_CREDENTIALS_OK, NULL); } static grpc_call_credentials_vtable iam_vtable = {iam_destruct, @@ -1178,7 +1180,8 @@ static void plugin_md_request_metadata_ready(void *request, gpr_log(GPR_ERROR, "Getting metadata from plugin failed with error: %s", error_details); } - r->cb(&exec_ctx, r->user_data, NULL, 0, GRPC_CREDENTIALS_ERROR); + r->cb(&exec_ctx, r->user_data, NULL, 0, GRPC_CREDENTIALS_ERROR, + error_details); } else { size_t i; grpc_credentials_md *md_array = NULL; @@ -1190,7 +1193,7 @@ static void plugin_md_request_metadata_ready(void *request, gpr_slice_from_copied_buffer(md[i].value, md[i].value_length); } } - r->cb(&exec_ctx, r->user_data, md_array, num_md, GRPC_CREDENTIALS_OK); + r->cb(&exec_ctx, r->user_data, md_array, num_md, GRPC_CREDENTIALS_OK, NULL); if (md_array != NULL) { for (i = 0; i < num_md; i++) { gpr_slice_unref(md_array[i].key); @@ -1218,7 +1221,7 @@ static void plugin_get_request_metadata(grpc_exec_ctx *exec_ctx, c->plugin.get_metadata(c->plugin.state, context, plugin_md_request_metadata_ready, request); } else { - cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_OK); + cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_OK, NULL); } } diff --git a/src/core/lib/security/credentials.h b/src/core/lib/security/credentials.h index 0373ceaa3fc..412b6e48fc9 100644 --- a/src/core/lib/security/credentials.h +++ b/src/core/lib/security/credentials.h @@ -160,11 +160,13 @@ void grpc_credentials_md_store_unref(grpc_credentials_md_store *store); /* --- grpc_call_credentials. --- */ +/* error_details must be NULL if status is GRPC_CREDENTIALS_OK. */ typedef void (*grpc_credentials_metadata_cb)(grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, size_t num_md, - grpc_credentials_status status); + grpc_credentials_status status, + const char *error_details); typedef struct { void (*destruct)(grpc_call_credentials *c); diff --git a/test/core/security/credentials_test.c b/test/core/security/credentials_test.c index 78672932787..36a4fdae14e 100644 --- a/test/core/security/credentials_test.c +++ b/test/core/security/credentials_test.c @@ -338,13 +338,15 @@ static void check_metadata(expected_md *expected, grpc_credentials_md *md_elems, static void check_google_iam_metadata(grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, size_t num_md, - grpc_credentials_status status) { + grpc_credentials_status status, + const char *error_details) { grpc_call_credentials *c = (grpc_call_credentials *)user_data; expected_md emd[] = {{GRPC_IAM_AUTHORIZATION_TOKEN_METADATA_KEY, test_google_iam_authorization_token}, {GRPC_IAM_AUTHORITY_SELECTOR_METADATA_KEY, test_google_iam_authority_selector}}; GPR_ASSERT(status == GRPC_CREDENTIALS_OK); + GPR_ASSERT(error_details == NULL); GPR_ASSERT(num_md == 2); check_metadata(emd, md_elems, num_md); grpc_call_credentials_unref(c); @@ -362,14 +364,13 @@ static void test_google_iam_creds(void) { grpc_exec_ctx_finish(&exec_ctx); } -static void check_access_token_metadata(grpc_exec_ctx *exec_ctx, - void *user_data, - grpc_credentials_md *md_elems, - size_t num_md, - grpc_credentials_status status) { +static void check_access_token_metadata( + grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, + size_t num_md, grpc_credentials_status status, const char *error_details) { grpc_call_credentials *c = (grpc_call_credentials *)user_data; expected_md emd[] = {{GRPC_AUTHORIZATION_METADATA_KEY, "Bearer blah"}}; GPR_ASSERT(status == GRPC_CREDENTIALS_OK); + GPR_ASSERT(error_details == NULL); GPR_ASSERT(num_md == 1); check_metadata(emd, md_elems, num_md); grpc_call_credentials_unref(c); @@ -418,7 +419,7 @@ static void test_channel_oauth2_composite_creds(void) { static void check_oauth2_google_iam_composite_metadata( grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, - size_t num_md, grpc_credentials_status status) { + size_t num_md, grpc_credentials_status status, const char *error_details) { grpc_call_credentials *c = (grpc_call_credentials *)user_data; expected_md emd[] = { {GRPC_AUTHORIZATION_METADATA_KEY, test_oauth2_bearer_token}, @@ -427,6 +428,7 @@ static void check_oauth2_google_iam_composite_metadata( {GRPC_IAM_AUTHORITY_SELECTOR_METADATA_KEY, test_google_iam_authority_selector}}; GPR_ASSERT(status == GRPC_CREDENTIALS_OK); + GPR_ASSERT(error_details == NULL); GPR_ASSERT(num_md == 3); check_metadata(emd, md_elems, num_md); grpc_call_credentials_unref(c); @@ -511,8 +513,9 @@ static void test_channel_oauth2_google_iam_composite_creds(void) { static void on_oauth2_creds_get_metadata_success( grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, - size_t num_md, grpc_credentials_status status) { + size_t num_md, grpc_credentials_status status, const char *error_details) { GPR_ASSERT(status == GRPC_CREDENTIALS_OK); + GPR_ASSERT(error_details == NULL); GPR_ASSERT(num_md == 1); GPR_ASSERT(gpr_slice_str_cmp(md_elems[0].key, "authorization") == 0); GPR_ASSERT(gpr_slice_str_cmp(md_elems[0].value, @@ -524,7 +527,7 @@ static void on_oauth2_creds_get_metadata_success( static void on_oauth2_creds_get_metadata_failure( grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, - size_t num_md, grpc_credentials_status status) { + size_t num_md, grpc_credentials_status status, const char *error_details) { GPR_ASSERT(status == GRPC_CREDENTIALS_ERROR); GPR_ASSERT(num_md == 0); GPR_ASSERT(user_data != NULL); @@ -760,14 +763,13 @@ static char *encode_and_sign_jwt_should_not_be_called( return NULL; } -static void on_jwt_creds_get_metadata_success(grpc_exec_ctx *exec_ctx, - void *user_data, - grpc_credentials_md *md_elems, - size_t num_md, - grpc_credentials_status status) { +static void on_jwt_creds_get_metadata_success( + grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, + size_t num_md, grpc_credentials_status status, const char *error_details) { char *expected_md_value; gpr_asprintf(&expected_md_value, "Bearer %s", test_signed_jwt); GPR_ASSERT(status == GRPC_CREDENTIALS_OK); + GPR_ASSERT(error_details == NULL); GPR_ASSERT(num_md == 1); GPR_ASSERT(gpr_slice_str_cmp(md_elems[0].key, "authorization") == 0); GPR_ASSERT(gpr_slice_str_cmp(md_elems[0].value, expected_md_value) == 0); @@ -776,11 +778,9 @@ static void on_jwt_creds_get_metadata_success(grpc_exec_ctx *exec_ctx, gpr_free(expected_md_value); } -static void on_jwt_creds_get_metadata_failure(grpc_exec_ctx *exec_ctx, - void *user_data, - grpc_credentials_md *md_elems, - size_t num_md, - grpc_credentials_status status) { +static void on_jwt_creds_get_metadata_failure( + grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, + size_t num_md, grpc_credentials_status status, const char *error_details) { GPR_ASSERT(status == GRPC_CREDENTIALS_ERROR); GPR_ASSERT(num_md == 0); GPR_ASSERT(user_data != NULL); @@ -1024,6 +1024,8 @@ static void plugin_get_metadata_success(void *state, cb(user_data, md, GPR_ARRAY_SIZE(md), GRPC_STATUS_OK, NULL); } +static const char *plugin_error_details = "Could not get metadata for plugin."; + static void plugin_get_metadata_failure(void *state, grpc_auth_metadata_context context, grpc_credentials_plugin_metadata_cb cb, @@ -1034,13 +1036,12 @@ static void plugin_get_metadata_failure(void *state, GPR_ASSERT(context.channel_auth_context == NULL); GPR_ASSERT(context.reserved == NULL); *s = PLUGIN_GET_METADATA_CALLED_STATE; - cb(user_data, NULL, 0, GRPC_STATUS_UNAUTHENTICATED, - "Could not get metadata for plugin."); + cb(user_data, NULL, 0, GRPC_STATUS_UNAUTHENTICATED, plugin_error_details); } static void on_plugin_metadata_received_success( grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, - size_t num_md, grpc_credentials_status status) { + size_t num_md, grpc_credentials_status status, const char *error_details) { size_t i = 0; GPR_ASSERT(user_data == NULL); GPR_ASSERT(md_elems != NULL); @@ -1053,11 +1054,13 @@ static void on_plugin_metadata_received_success( static void on_plugin_metadata_received_failure( grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, - size_t num_md, grpc_credentials_status status) { + size_t num_md, grpc_credentials_status status, const char *error_details) { GPR_ASSERT(user_data == NULL); GPR_ASSERT(md_elems == NULL); GPR_ASSERT(num_md == 0); GPR_ASSERT(status == GRPC_CREDENTIALS_ERROR); + GPR_ASSERT(error_details != NULL); + GPR_ASSERT(strcmp(error_details, plugin_error_details) == 0); } static void plugin_destroy(void *state) { diff --git a/test/core/security/oauth2_utils.c b/test/core/security/oauth2_utils.c index 20815d184cd..10155fccf59 100644 --- a/test/core/security/oauth2_utils.c +++ b/test/core/security/oauth2_utils.c @@ -53,7 +53,8 @@ typedef struct { static void on_oauth2_response(grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, size_t num_md, - grpc_credentials_status status) { + grpc_credentials_status status, + const char *error_details) { oauth2_request *request = user_data; char *token = NULL; gpr_slice token_slice; diff --git a/test/core/security/print_google_default_creds_token.c b/test/core/security/print_google_default_creds_token.c index 99bce4fbdfb..be900d84498 100644 --- a/test/core/security/print_google_default_creds_token.c +++ b/test/core/security/print_google_default_creds_token.c @@ -53,7 +53,8 @@ typedef struct { static void on_metadata_response(grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, size_t num_md, - grpc_credentials_status status) { + grpc_credentials_status status, + const char *error_details) { synchronizer *sync = user_data; if (status == GRPC_CREDENTIALS_ERROR) { fprintf(stderr, "Fetching token failed.\n"); From ef96264d8724a71f4f1fef5ebaf361d28b1a1752 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Wed, 18 May 2016 22:57:17 -0700 Subject: [PATCH 006/280] Support SO_REUSEPORT --- include/grpc/impl/codegen/grpc_types.h | 2 + .../chttp2/server/insecure/server_chttp2.c | 3 +- .../server/secure/server_secure_chttp2.c | 3 +- .../lib/iomgr/socket_utils_common_posix.c | 22 ++++ src/core/lib/iomgr/socket_utils_posix.h | 3 + src/core/lib/iomgr/tcp_server.h | 3 + src/core/lib/iomgr/tcp_server_posix.c | 122 ++++++++++++++++-- 7 files changed, 143 insertions(+), 15 deletions(-) diff --git a/include/grpc/impl/codegen/grpc_types.h b/include/grpc/impl/codegen/grpc_types.h index af3d0c21916..cd3cae71b6a 100644 --- a/include/grpc/impl/codegen/grpc_types.h +++ b/include/grpc/impl/codegen/grpc_types.h @@ -154,6 +154,8 @@ typedef struct { #define GRPC_SSL_TARGET_NAME_OVERRIDE_ARG "grpc.ssl_target_name_override" /* Maximum metadata size */ #define GRPC_ARG_MAX_METADATA_SIZE "grpc.max_metadata_size" +/** If non-zero, allow the use of SO_REUSEPORT if it's available (default 1) */ +#define GRPC_ARG_ALLOW_REUSEPORT "grpc.so_reuseport" /** Result of a grpc call. If the caller satisfies the prerequisites of a particular operation, the grpc_call_error returned will be GRPC_CALL_OK. diff --git a/src/core/ext/transport/chttp2/server/insecure/server_chttp2.c b/src/core/ext/transport/chttp2/server/insecure/server_chttp2.c index d1a58b66211..08a09a933a6 100644 --- a/src/core/ext/transport/chttp2/server/insecure/server_chttp2.c +++ b/src/core/ext/transport/chttp2/server/insecure/server_chttp2.c @@ -102,7 +102,8 @@ int grpc_server_add_insecure_http2_port(grpc_server *server, const char *addr) { goto error; } - err = grpc_tcp_server_create(NULL, &tcp); + err = + grpc_tcp_server_create(NULL, grpc_server_get_channel_args(server), &tcp); if (err != GRPC_ERROR_NONE) { goto error; } diff --git a/src/core/ext/transport/chttp2/server/secure/server_secure_chttp2.c b/src/core/ext/transport/chttp2/server/secure/server_secure_chttp2.c index 629959b7c03..64162cc7857 100644 --- a/src/core/ext/transport/chttp2/server/secure/server_secure_chttp2.c +++ b/src/core/ext/transport/chttp2/server/secure/server_secure_chttp2.c @@ -208,7 +208,8 @@ int grpc_server_add_secure_http2_port(grpc_server *server, const char *addr, state = gpr_malloc(sizeof(*state)); memset(state, 0, sizeof(*state)); grpc_closure_init(&state->destroy_closure, destroy_done, state); - err = grpc_tcp_server_create(&state->destroy_closure, &tcp); + err = grpc_tcp_server_create(&state->destroy_closure, + grpc_server_get_channel_args(server), &tcp); if (err != GRPC_ERROR_NONE) { goto error; } diff --git a/src/core/lib/iomgr/socket_utils_common_posix.c b/src/core/lib/iomgr/socket_utils_common_posix.c index d1721c910c4..bd8ef00c859 100644 --- a/src/core/lib/iomgr/socket_utils_common_posix.c +++ b/src/core/lib/iomgr/socket_utils_common_posix.c @@ -154,6 +154,28 @@ grpc_error *grpc_set_socket_reuse_addr(int fd, int reuse) { return GRPC_ERROR_NONE; } +/* set a socket to reuse old addresses */ +grpc_error *grpc_set_socket_reuse_port(int fd, int reuse) { +#ifndef SO_REUSEPORT + return GRPC_ERROR_CREATE("SO_REUSEPORT unavailable on compiling system"); +#else + int val = (reuse != 0); + int newval; + socklen_t intlen = sizeof(newval); + if (0 != setsockopt(fd, SOL_SOCKET, SO_REUSEPORT, &val, sizeof(val))) { + return GRPC_OS_ERROR(errno, "setsockopt(SO_REUSEPORT)"); + } + if (0 != getsockopt(fd, SOL_SOCKET, SO_REUSEPORT, &newval, &intlen)) { + return GRPC_OS_ERROR(errno, "getsockopt(SO_REUSEPORT)"); + } + if ((newval != 0) != val) { + return GRPC_ERROR_CREATE("Failed to set SO_REUSEPORT"); + } + + return GRPC_ERROR_NONE; +#endif +} + /* disable nagle */ grpc_error *grpc_set_socket_low_latency(int fd, int low_latency) { int val = (low_latency != 0); diff --git a/src/core/lib/iomgr/socket_utils_posix.h b/src/core/lib/iomgr/socket_utils_posix.h index 4e66cbc0c58..607fbc7a791 100644 --- a/src/core/lib/iomgr/socket_utils_posix.h +++ b/src/core/lib/iomgr/socket_utils_posix.h @@ -55,6 +55,9 @@ grpc_error *grpc_set_socket_reuse_addr(int fd, int reuse); /* disable nagle */ grpc_error *grpc_set_socket_low_latency(int fd, int low_latency); +/* set SO_REUSEPORT */ +grpc_error *grpc_set_socket_reuse_port(int fd, int reuse); + /* Returns true if this system can create AF_INET6 sockets bound to ::1. The value is probed once, and cached for the life of the process. diff --git a/src/core/lib/iomgr/tcp_server.h b/src/core/lib/iomgr/tcp_server.h index 924121f0c79..79f0242b26d 100644 --- a/src/core/lib/iomgr/tcp_server.h +++ b/src/core/lib/iomgr/tcp_server.h @@ -34,6 +34,8 @@ #ifndef GRPC_CORE_LIB_IOMGR_TCP_SERVER_H #define GRPC_CORE_LIB_IOMGR_TCP_SERVER_H +#include + #include "src/core/lib/iomgr/closure.h" #include "src/core/lib/iomgr/endpoint.h" @@ -58,6 +60,7 @@ typedef void (*grpc_tcp_server_cb)(grpc_exec_ctx *exec_ctx, void *arg, If shutdown_complete is not NULL, it will be used by grpc_tcp_server_unref() when the ref count reaches zero. */ grpc_error *grpc_tcp_server_create(grpc_closure *shutdown_complete, + const grpc_channel_args *args, grpc_tcp_server **server); /* Start listening to bound ports */ diff --git a/src/core/lib/iomgr/tcp_server_posix.c b/src/core/lib/iomgr/tcp_server_posix.c index db70fada96a..7047934780b 100644 --- a/src/core/lib/iomgr/tcp_server_posix.c +++ b/src/core/lib/iomgr/tcp_server_posix.c @@ -112,8 +112,10 @@ struct grpc_tcp_server { /* destroyed port count: how many ports are completely destroyed */ size_t destroyed_ports; - /* is this server shutting down? (boolean) */ - int shutdown; + /* is this server shutting down? */ + bool shutdown; + /* use SO_REUSEPORT */ + bool so_reuseport; /* linked list of server ports */ grpc_tcp_listener *head; @@ -132,14 +134,42 @@ struct grpc_tcp_server { size_t pollset_count; }; +static gpr_once check_init = GPR_ONCE_INIT; +static bool has_so_reuseport; + +static void init(void) { + int s = socket(AF_INET, SOCK_STREAM, 0); + if (s >= 0) { + has_so_reuseport = GRPC_LOG_IF_ERROR("check for SO_REUSEPORT", + grpc_set_socket_reuse_port(s, 1)); + close(s); + } +} + grpc_error *grpc_tcp_server_create(grpc_closure *shutdown_complete, + const grpc_channel_args *args, grpc_tcp_server **server) { + gpr_once_init(&check_init, init); + grpc_tcp_server *s = gpr_malloc(sizeof(grpc_tcp_server)); + s->so_reuseport = has_so_reuseport; + for (size_t i = 0; i < (args == NULL ? 0 : args->num_args); i++) { + if (0 == strcmp(GRPC_ARG_ALLOW_REUSEPORT, args->args[i].key)) { + if (args->args[i].type == GRPC_ARG_INTEGER) { + s->so_reuseport = + has_so_reuseport && (args->args[i].value.integer != 0); + } else { + gpr_free(s); + return GRPC_ERROR_CREATE(GRPC_ARG_ALLOW_REUSEPORT + " must be an integer"); + } + } + } gpr_ref_init(&s->refs, 1); gpr_mu_init(&s->mu); s->active_ports = 0; s->destroyed_ports = 0; - s->shutdown = 0; + s->shutdown = false; s->shutdown_starting.head = NULL; s->shutdown_starting.tail = NULL; s->shutdown_complete = shutdown_complete; @@ -214,7 +244,7 @@ static void tcp_server_destroy(grpc_exec_ctx *exec_ctx, grpc_tcp_server *s) { gpr_mu_lock(&s->mu); GPR_ASSERT(!s->shutdown); - s->shutdown = 1; + s->shutdown = true; /* shutdown all fd's */ if (s->active_ports) { @@ -264,13 +294,19 @@ static int get_max_accept_queue_size(void) { /* Prepare a recently-created socket for listening. */ static grpc_error *prepare_socket(int fd, const struct sockaddr *addr, - size_t addr_len, int *port) { + size_t addr_len, bool so_reuseport, + int *port) { struct sockaddr_storage sockname_temp; socklen_t sockname_len; grpc_error *err = GRPC_ERROR_NONE; GPR_ASSERT(fd >= 0); + if (so_reuseport) { + err = grpc_set_socket_reuse_port(fd, 1); + if (err != GRPC_ERROR_NONE) goto error; + } + err = grpc_set_socket_nonblocking(fd, 1); if (err != GRPC_ERROR_NONE) goto error; err = grpc_set_socket_cloexec(fd, 1); @@ -397,7 +433,7 @@ static grpc_error *add_socket_to_server(grpc_tcp_server *s, int fd, char *addr_str; char *name; - grpc_error *err = prepare_socket(fd, addr, addr_len, &port); + grpc_error *err = prepare_socket(fd, addr, addr_len, s->so_reuseport, &port); if (err == GRPC_ERROR_NONE) { GPR_ASSERT(port > 0); grpc_sockaddr_to_string(&addr_str, (struct sockaddr *)&addr, 1); @@ -433,6 +469,51 @@ static grpc_error *add_socket_to_server(grpc_tcp_server *s, int fd, return err; } +static grpc_error *clone_port(grpc_tcp_listener *listener, unsigned count) { + grpc_tcp_listener *sp = NULL; + char *addr_str; + char *name; + grpc_error *err; + + for (grpc_tcp_listener *l = listener->next; l && l->is_sibling; l = l->next) { + l->fd_index += count; + } + + for (unsigned i = 0; i < count; i++) { + int fd, port; + grpc_dualstack_mode dsmode; + err = grpc_create_dualstack_socket(&listener->addr.sockaddr, SOCK_STREAM, 0, + &dsmode, &fd); + if (err != GRPC_ERROR_NONE) return err; + err = prepare_socket(fd, &listener->addr.sockaddr, listener->addr_len, true, + &port); + if (err != GRPC_ERROR_NONE) return err; + grpc_sockaddr_to_string(&addr_str, &listener->addr.sockaddr, 1); + gpr_asprintf(&name, "tcp-server-listener:%s/clone-%d", addr_str, i); + sp = gpr_malloc(sizeof(grpc_tcp_listener)); + sp->next = listener->next; + listener->next = sp; + sp->server = listener->server; + sp->fd = fd; + sp->emfd = grpc_fd_create(fd, name); + memcpy(sp->addr.untyped, listener->addr.untyped, listener->addr_len); + sp->addr_len = listener->addr_len; + sp->port = port; + sp->port_index = listener->port_index; + sp->fd_index = listener->fd_index + count - i; + sp->is_sibling = 1; + sp->sibling = listener->is_sibling ? listener->sibling : listener; + GPR_ASSERT(sp->emfd); + while (listener->server->tail->next != NULL) { + listener->server->tail = listener->server->tail->next; + } + gpr_free(addr_str); + gpr_free(name); + } + + return GRPC_ERROR_NONE; +} + grpc_error *grpc_tcp_server_add_port(grpc_tcp_server *s, const void *addr, size_t addr_len, int *out_port) { grpc_tcp_listener *sp; @@ -589,14 +670,29 @@ void grpc_tcp_server_start(grpc_exec_ctx *exec_ctx, grpc_tcp_server *s, s->on_accept_cb_arg = on_accept_cb_arg; s->pollsets = pollsets; s->pollset_count = pollset_count; - for (sp = s->head; sp; sp = sp->next) { - for (i = 0; i < pollset_count; i++) { - grpc_pollset_add_fd(exec_ctx, pollsets[i], sp->emfd); + sp = s->head; + while (sp != NULL) { + if (s->so_reuseport && pollset_count > 1) { + GPR_ASSERT(GRPC_LOG_IF_ERROR( + "clone_port", clone_port(sp, (unsigned)(pollset_count - 1)))); + for (i = 0; i < pollset_count; i++) { + grpc_pollset_add_fd(exec_ctx, pollsets[i], sp->emfd); + sp->read_closure.cb = on_read; + sp->read_closure.cb_arg = sp; + grpc_fd_notify_on_read(exec_ctx, sp->emfd, &sp->read_closure); + s->active_ports++; + sp = sp->next; + } + } else { + for (i = 0; i < pollset_count; i++) { + grpc_pollset_add_fd(exec_ctx, pollsets[i], sp->emfd); + } + sp->read_closure.cb = on_read; + sp->read_closure.cb_arg = sp; + grpc_fd_notify_on_read(exec_ctx, sp->emfd, &sp->read_closure); + s->active_ports++; + sp = sp->next; } - sp->read_closure.cb = on_read; - sp->read_closure.cb_arg = sp; - grpc_fd_notify_on_read(exec_ctx, sp->emfd, &sp->read_closure); - s->active_ports++; } gpr_mu_unlock(&s->mu); } From c49464de3ed6108956561128b006c78777bd6db7 Mon Sep 17 00:00:00 2001 From: Julien Boeuf Date: Wed, 18 May 2016 23:11:50 -0700 Subject: [PATCH 007/280] replacing cancel op by close op in order to plumb error message. --- src/core/lib/security/client_auth_filter.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/src/core/lib/security/client_auth_filter.c b/src/core/lib/security/client_auth_filter.c index f6de877021e..3908b734a27 100644 --- a/src/core/lib/security/client_auth_filter.c +++ b/src/core/lib/security/client_auth_filter.c @@ -91,7 +91,8 @@ static void bubble_up_error(grpc_exec_ctx *exec_ctx, grpc_call_element *elem, grpc_status_code status, const char *error_msg) { call_data *calld = elem->call_data; gpr_log(GPR_ERROR, "Client side authentication failure: %s", error_msg); - grpc_transport_stream_op_add_cancellation(&calld->op, status); + gpr_slice error_slice = gpr_slice_from_copied_string(error_msg); + grpc_transport_stream_op_add_close(&calld->op, status, &error_slice); grpc_call_next_op(exec_ctx, elem, &calld->op); } @@ -108,7 +109,9 @@ static void on_credentials_metadata(grpc_exec_ctx *exec_ctx, void *user_data, reset_auth_metadata_context(&calld->auth_md_context); if (status != GRPC_CREDENTIALS_OK) { bubble_up_error(exec_ctx, elem, GRPC_STATUS_UNAUTHENTICATED, - "Credentials failed to get metadata."); + (error_details != NULL && strlen(error_details) > 0) + ? error_details + : "Credentials failed to get metadata."); return; } GPR_ASSERT(num_md <= MAX_CREDENTIALS_METADATA_COUNT); From 644da9857300151b27a1f297c40d971597a0845f Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 19 May 2016 09:19:28 -0700 Subject: [PATCH 008/280] Fixes --- src/core/lib/iomgr/tcp_server_posix.c | 1 + test/core/iomgr/tcp_server_posix_test.c | 10 +++++----- test/core/surface/concurrent_connectivity_test.c | 2 +- test/core/surface/server_chttp2_test.c | 8 +++++++- test/core/surface/server_test.c | 10 ++++++++-- test/core/util/test_tcp_server.c | 2 +- third_party/protobuf | 2 +- 7 files changed, 24 insertions(+), 11 deletions(-) diff --git a/src/core/lib/iomgr/tcp_server_posix.c b/src/core/lib/iomgr/tcp_server_posix.c index 7047934780b..83627e59d9d 100644 --- a/src/core/lib/iomgr/tcp_server_posix.c +++ b/src/core/lib/iomgr/tcp_server_posix.c @@ -488,6 +488,7 @@ static grpc_error *clone_port(grpc_tcp_listener *listener, unsigned count) { err = prepare_socket(fd, &listener->addr.sockaddr, listener->addr_len, true, &port); if (err != GRPC_ERROR_NONE) return err; + listener->server->nports++; grpc_sockaddr_to_string(&addr_str, &listener->addr.sockaddr, 1); gpr_asprintf(&name, "tcp-server-listener:%s/clone-%d", addr_str, i); sp = gpr_malloc(sizeof(grpc_tcp_listener)); diff --git a/test/core/iomgr/tcp_server_posix_test.c b/test/core/iomgr/tcp_server_posix_test.c index a37ff1773b2..54437f6a9bd 100644 --- a/test/core/iomgr/tcp_server_posix_test.c +++ b/test/core/iomgr/tcp_server_posix_test.c @@ -128,7 +128,7 @@ static void on_connect(grpc_exec_ctx *exec_ctx, void *arg, grpc_endpoint *tcp, static void test_no_op(void) { grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; grpc_tcp_server *s; - GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, &s)); + GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, NULL, &s)); grpc_tcp_server_unref(&exec_ctx, s); grpc_exec_ctx_finish(&exec_ctx); } @@ -136,7 +136,7 @@ static void test_no_op(void) { static void test_no_op_with_start(void) { grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; grpc_tcp_server *s; - GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, &s)); + GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, NULL, &s)); LOG_TEST("test_no_op_with_start"); grpc_tcp_server_start(&exec_ctx, s, NULL, 0, on_connect, NULL); grpc_tcp_server_unref(&exec_ctx, s); @@ -147,7 +147,7 @@ static void test_no_op_with_port(void) { grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; struct sockaddr_in addr; grpc_tcp_server *s; - GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, &s)); + GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, NULL, &s)); LOG_TEST("test_no_op_with_port"); memset(&addr, 0, sizeof(addr)); @@ -165,7 +165,7 @@ static void test_no_op_with_port_and_start(void) { grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; struct sockaddr_in addr; grpc_tcp_server *s; - GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, &s)); + GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, NULL, &s)); LOG_TEST("test_no_op_with_port_and_start"); int port; @@ -225,7 +225,7 @@ static void test_connect(unsigned n) { unsigned svr1_fd_count; int svr1_port; grpc_tcp_server *s; - GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, &s)); + GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, NULL, &s)); unsigned i; server_weak_ref weak_ref; server_weak_ref_init(&weak_ref); diff --git a/test/core/surface/concurrent_connectivity_test.c b/test/core/surface/concurrent_connectivity_test.c index 74e3183c26b..6d7d0413b28 100644 --- a/test/core/surface/concurrent_connectivity_test.c +++ b/test/core/surface/concurrent_connectivity_test.c @@ -112,7 +112,7 @@ void bad_server_thread(void *vargs) { socklen_t addr_len = sizeof(addr); int port; grpc_tcp_server *s; - grpc_error *error = grpc_tcp_server_create(NULL, &s); + grpc_error *error = grpc_tcp_server_create(NULL, NULL, &s); GPR_ASSERT(error == GRPC_ERROR_NONE); memset(&addr, 0, sizeof(addr)); addr.ss_family = AF_INET; diff --git a/test/core/surface/server_chttp2_test.c b/test/core/surface/server_chttp2_test.c index f42ca9f9cdf..40cfa6b5989 100644 --- a/test/core/surface/server_chttp2_test.c +++ b/test/core/surface/server_chttp2_test.c @@ -49,10 +49,16 @@ void test_unparsable_target(void) { } void test_add_same_port_twice() { + grpc_arg a; + a.type = GRPC_ARG_INTEGER; + a.key = GRPC_ARG_ALLOW_REUSEPORT; + a.value.integer = 0; + grpc_channel_args args = {1, &a}; + int port = grpc_pick_unused_port_or_die(); char *addr = NULL; grpc_completion_queue *cq = grpc_completion_queue_create(NULL); - grpc_server *server = grpc_server_create(NULL, NULL); + grpc_server *server = grpc_server_create(&args, NULL); grpc_server_credentials *fake_creds = grpc_fake_transport_security_server_credentials_create(); gpr_join_host_port(&addr, "localhost", port); diff --git a/test/core/surface/server_test.c b/test/core/surface/server_test.c index 3d2e25379a0..d1e646a9f15 100644 --- a/test/core/surface/server_test.c +++ b/test/core/surface/server_test.c @@ -76,9 +76,15 @@ void test_request_call_on_no_server_cq(void) { } void test_bind_server_twice(void) { + grpc_arg a; + a.type = GRPC_ARG_INTEGER; + a.key = GRPC_ARG_ALLOW_REUSEPORT; + a.value.integer = 0; + grpc_channel_args args = {1, &a}; + char *addr; - grpc_server *server1 = grpc_server_create(NULL, NULL); - grpc_server *server2 = grpc_server_create(NULL, NULL); + grpc_server *server1 = grpc_server_create(&args, NULL); + grpc_server *server2 = grpc_server_create(&args, NULL); grpc_completion_queue *cq = grpc_completion_queue_create(NULL); int port = grpc_pick_unused_port_or_die(); gpr_asprintf(&addr, "[::]:%d", port); diff --git a/test/core/util/test_tcp_server.c b/test/core/util/test_tcp_server.c index 27c16fc764a..c1d307fc779 100644 --- a/test/core/util/test_tcp_server.c +++ b/test/core/util/test_tcp_server.c @@ -73,7 +73,7 @@ void test_tcp_server_start(test_tcp_server *server, int port) { memset(&addr.sin_addr, 0, sizeof(addr.sin_addr)); grpc_error *error = - grpc_tcp_server_create(&server->shutdown_complete, &server->tcp_server); + grpc_tcp_server_create(&server->shutdown_complete, NULL, &server->tcp_server); GPR_ASSERT(error == GRPC_ERROR_NONE); error = grpc_tcp_server_add_port(server->tcp_server, &addr, sizeof(addr), &port_added); diff --git a/third_party/protobuf b/third_party/protobuf index a1938b2aa9c..d5fb408ddc2 160000 --- a/third_party/protobuf +++ b/third_party/protobuf @@ -1 +1 @@ -Subproject commit a1938b2aa9ca86ce7ce50c27ff9737c1008d2a03 +Subproject commit d5fb408ddc281ffcadeb08699e65bb694656d0bd From e48b1bc011e2b5783ce75f4122dfd5aba01f0d97 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 11 May 2016 15:17:22 -0700 Subject: [PATCH 009/280] Base changes. Create ev_epoll_posix.{c,h} files by making a copy of ev_poll_and_epoll.c file --- BUILD | 6 + Makefile | 2 + binding.gyp | 1 + build.yaml | 2 + config.m4 | 1 + gRPC.podspec | 3 + grpc.gemspec | 2 + package.xml | 2 + src/core/lib/iomgr/ev_epoll_posix.c | 1733 +++++++++++++++++ src/core/lib/iomgr/ev_epoll_posix.h | 41 + src/core/lib/iomgr/ev_posix.c | 3 +- src/python/grpcio/grpc_core_dependencies.py | 1 + tools/doxygen/Doxyfile.core.internal | 2 + tools/run_tests/sources_and_headers.json | 3 + vsprojects/vcxproj/grpc/grpc.vcxproj | 3 + vsprojects/vcxproj/grpc/grpc.vcxproj.filters | 6 + .../grpc_unsecure/grpc_unsecure.vcxproj | 3 + .../grpc_unsecure.vcxproj.filters | 6 + 18 files changed, 1819 insertions(+), 1 deletion(-) create mode 100644 src/core/lib/iomgr/ev_epoll_posix.c create mode 100644 src/core/lib/iomgr/ev_epoll_posix.h diff --git a/BUILD b/BUILD index fd03d52e3cb..0be8f27a01b 100644 --- a/BUILD +++ b/BUILD @@ -178,6 +178,7 @@ cc_library( "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", @@ -321,6 +322,7 @@ cc_library( "src/core/lib/iomgr/endpoint.c", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", "src/core/lib/iomgr/exec_ctx.c", @@ -546,6 +548,7 @@ cc_library( "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", @@ -666,6 +669,7 @@ cc_library( "src/core/lib/iomgr/endpoint.c", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", "src/core/lib/iomgr/exec_ctx.c", @@ -1358,6 +1362,7 @@ objc_library( "src/core/lib/iomgr/endpoint.c", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", "src/core/lib/iomgr/exec_ctx.c", @@ -1562,6 +1567,7 @@ objc_library( "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", diff --git a/Makefile b/Makefile index 817fcd072d7..29ebc0e5adf 100644 --- a/Makefile +++ b/Makefile @@ -2486,6 +2486,7 @@ LIBGRPC_SRC = \ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ + src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ @@ -2840,6 +2841,7 @@ LIBGRPC_UNSECURE_SRC = \ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ + src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ diff --git a/binding.gyp b/binding.gyp index 7b187070005..89774ead4da 100644 --- a/binding.gyp +++ b/binding.gyp @@ -581,6 +581,7 @@ 'src/core/lib/iomgr/endpoint.c', 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', + 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', 'src/core/lib/iomgr/exec_ctx.c', diff --git a/build.yaml b/build.yaml index 429dbb3351a..7ba65332972 100644 --- a/build.yaml +++ b/build.yaml @@ -165,6 +165,7 @@ filegroups: - src/core/lib/iomgr/closure.h - src/core/lib/iomgr/endpoint.h - src/core/lib/iomgr/endpoint_pair.h + - src/core/lib/iomgr/ev_epoll_posix.h - src/core/lib/iomgr/ev_poll_posix.h - src/core/lib/iomgr/ev_posix.h - src/core/lib/iomgr/exec_ctx.h @@ -239,6 +240,7 @@ filegroups: - src/core/lib/iomgr/endpoint.c - src/core/lib/iomgr/endpoint_pair_posix.c - src/core/lib/iomgr/endpoint_pair_windows.c + - src/core/lib/iomgr/ev_epoll_posix.c - src/core/lib/iomgr/ev_poll_posix.c - src/core/lib/iomgr/ev_posix.c - src/core/lib/iomgr/exec_ctx.c diff --git a/config.m4 b/config.m4 index 29df77ce1db..6987c741541 100644 --- a/config.m4 +++ b/config.m4 @@ -100,6 +100,7 @@ if test "$PHP_GRPC" != "no"; then src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ + src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ diff --git a/gRPC.podspec b/gRPC.podspec index cc8ba3de94e..3b4dd52380e 100644 --- a/gRPC.podspec +++ b/gRPC.podspec @@ -181,6 +181,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/closure.h', 'src/core/lib/iomgr/endpoint.h', 'src/core/lib/iomgr/endpoint_pair.h', + 'src/core/lib/iomgr/ev_epoll_posix.h', 'src/core/lib/iomgr/ev_poll_posix.h', 'src/core/lib/iomgr/ev_posix.h', 'src/core/lib/iomgr/exec_ctx.h', @@ -358,6 +359,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/endpoint.c', 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', + 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', 'src/core/lib/iomgr/exec_ctx.c', @@ -546,6 +548,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/closure.h', 'src/core/lib/iomgr/endpoint.h', 'src/core/lib/iomgr/endpoint_pair.h', + 'src/core/lib/iomgr/ev_epoll_posix.h', 'src/core/lib/iomgr/ev_poll_posix.h', 'src/core/lib/iomgr/ev_posix.h', 'src/core/lib/iomgr/exec_ctx.h', diff --git a/grpc.gemspec b/grpc.gemspec index ae7f9b7d2ee..71cccb6ca8d 100755 --- a/grpc.gemspec +++ b/grpc.gemspec @@ -190,6 +190,7 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/closure.h ) s.files += %w( src/core/lib/iomgr/endpoint.h ) s.files += %w( src/core/lib/iomgr/endpoint_pair.h ) + s.files += %w( src/core/lib/iomgr/ev_epoll_posix.h ) s.files += %w( src/core/lib/iomgr/ev_poll_posix.h ) s.files += %w( src/core/lib/iomgr/ev_posix.h ) s.files += %w( src/core/lib/iomgr/exec_ctx.h ) @@ -337,6 +338,7 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/endpoint.c ) s.files += %w( src/core/lib/iomgr/endpoint_pair_posix.c ) s.files += %w( src/core/lib/iomgr/endpoint_pair_windows.c ) + s.files += %w( src/core/lib/iomgr/ev_epoll_posix.c ) s.files += %w( src/core/lib/iomgr/ev_poll_posix.c ) s.files += %w( src/core/lib/iomgr/ev_posix.c ) s.files += %w( src/core/lib/iomgr/exec_ctx.c ) diff --git a/package.xml b/package.xml index 507a2a7ed6c..0fc5d0dee47 100644 --- a/package.xml +++ b/package.xml @@ -197,6 +197,7 @@ + @@ -344,6 +345,7 @@ + diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c new file mode 100644 index 00000000000..ce8d3981b3f --- /dev/null +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -0,0 +1,1733 @@ +/* + * + * Copyright 2016, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include + +#ifdef GPR_POSIX_SOCKET + +#include "src/core/lib/iomgr/ev_epoll_posix.h" + +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "src/core/lib/iomgr/iomgr_internal.h" +#include "src/core/lib/iomgr/wakeup_fd_posix.h" +#include "src/core/lib/profiling/timers.h" +#include "src/core/lib/support/block_annotate.h" + +/******************************************************************************* + * FD declarations + */ + +/* TODO(sreek) : Check if grpc_fd_watcher is needed (and if so, check if we can + * share this between ev_poll_posix.h and ev_epoll_posix versions */ + +typedef struct grpc_fd_watcher { + struct grpc_fd_watcher *next; + struct grpc_fd_watcher *prev; + grpc_pollset *pollset; + grpc_pollset_worker *worker; + grpc_fd *fd; +} grpc_fd_watcher; + +struct grpc_fd { + int fd; + /* refst format: + bit0: 1=active/0=orphaned + bit1-n: refcount + meaning that mostly we ref by two to avoid altering the orphaned bit, + and just unref by 1 when we're ready to flag the object as orphaned */ + gpr_atm refst; + + gpr_mu mu; + int shutdown; + int closed; + int released; + + /* The watcher list. + + The following watcher related fields are protected by watcher_mu. + + An fd_watcher is an ephemeral object created when an fd wants to + begin polling, and destroyed after the poll. + + It denotes the fd's interest in whether to read poll or write poll + or both or neither on this fd. + + If a watcher is asked to poll for reads or writes, the read_watcher + or write_watcher fields are set respectively. A watcher may be asked + to poll for both, in which case both fields will be set. + + read_watcher and write_watcher may be NULL if no watcher has been + asked to poll for reads or writes. + + If an fd_watcher is not asked to poll for reads or writes, it's added + to a linked list of inactive watchers, rooted at inactive_watcher_root. + If at a later time there becomes need of a poller to poll, one of + the inactive pollers may be kicked out of their poll loops to take + that responsibility. */ + grpc_fd_watcher inactive_watcher_root; + grpc_fd_watcher *read_watcher; + grpc_fd_watcher *write_watcher; + + grpc_closure *read_closure; + grpc_closure *write_closure; + + struct grpc_fd *freelist_next; + + grpc_closure *on_done_closure; + + grpc_iomgr_object iomgr_object; +}; + +/* Begin polling on an fd. + Registers that the given pollset is interested in this fd - so that if read + or writability interest changes, the pollset can be kicked to pick up that + new interest. + Return value is: + (fd_needs_read? read_mask : 0) | (fd_needs_write? write_mask : 0) + i.e. a combination of read_mask and write_mask determined by the fd's current + interest in said events. + Polling strategies that do not need to alter their behavior depending on the + fd's current interest (such as epoll) do not need to call this function. + MUST NOT be called with a pollset lock taken */ +static uint32_t fd_begin_poll(grpc_fd *fd, grpc_pollset *pollset, + grpc_pollset_worker *worker, uint32_t read_mask, + uint32_t write_mask, grpc_fd_watcher *rec); +/* Complete polling previously started with fd_begin_poll + MUST NOT be called with a pollset lock taken + if got_read or got_write are 1, also does the become_{readable,writable} as + appropriate. */ +static void fd_end_poll(grpc_exec_ctx *exec_ctx, grpc_fd_watcher *rec, + int got_read, int got_write); + +/* Return 1 if this fd is orphaned, 0 otherwise */ +static bool fd_is_orphaned(grpc_fd *fd); + +/* Reference counting for fds */ +/*#define GRPC_FD_REF_COUNT_DEBUG*/ +#ifdef GRPC_FD_REF_COUNT_DEBUG +static void fd_ref(grpc_fd *fd, const char *reason, const char *file, int line); +static void fd_unref(grpc_fd *fd, const char *reason, const char *file, + int line); +#define GRPC_FD_REF(fd, reason) fd_ref(fd, reason, __FILE__, __LINE__) +#define GRPC_FD_UNREF(fd, reason) fd_unref(fd, reason, __FILE__, __LINE__) +#else +static void fd_ref(grpc_fd *fd); +static void fd_unref(grpc_fd *fd); +#define GRPC_FD_REF(fd, reason) fd_ref(fd) +#define GRPC_FD_UNREF(fd, reason) fd_unref(fd) +#endif + +static void fd_global_init(void); +static void fd_global_shutdown(void); + +#define CLOSURE_NOT_READY ((grpc_closure *)0) +#define CLOSURE_READY ((grpc_closure *)1) + +/******************************************************************************* + * pollset declarations + */ + +typedef struct grpc_pollset_vtable grpc_pollset_vtable; + +typedef struct grpc_cached_wakeup_fd { + grpc_wakeup_fd fd; + struct grpc_cached_wakeup_fd *next; +} grpc_cached_wakeup_fd; + +struct grpc_pollset_worker { + grpc_cached_wakeup_fd *wakeup_fd; + int reevaluate_polling_on_wakeup; + int kicked_specifically; + struct grpc_pollset_worker *next; + struct grpc_pollset_worker *prev; +}; + +struct grpc_pollset { + /* pollsets under posix can mutate representation as fds are added and + removed. + For example, we may choose a poll() based implementation on linux for + few fds, and an epoll() based implementation for many fds */ + const grpc_pollset_vtable *vtable; + gpr_mu mu; + grpc_pollset_worker root_worker; + int in_flight_cbs; + int shutting_down; + int called_shutdown; + int kicked_without_pollers; + grpc_closure *shutdown_done; + grpc_closure_list idle_jobs; + union { + int fd; + void *ptr; + } data; + /* Local cache of eventfds for workers */ + grpc_cached_wakeup_fd *local_wakeup_cache; +}; + +struct grpc_pollset_vtable { + void (*add_fd)(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + struct grpc_fd *fd, int and_unlock_pollset); + void (*maybe_work_and_unlock)(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_pollset_worker *worker, + gpr_timespec deadline, gpr_timespec now); + void (*finish_shutdown)(grpc_pollset *pollset); + void (*destroy)(grpc_pollset *pollset); +}; + +/* Add an fd to a pollset */ +static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + struct grpc_fd *fd); + +static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd); + +/* Convert a timespec to milliseconds: + - very small or negative poll times are clamped to zero to do a + non-blocking poll (which becomes spin polling) + - other small values are rounded up to one millisecond + - longer than a millisecond polls are rounded up to the next nearest + millisecond to avoid spinning + - infinite timeouts are converted to -1 */ +static int poll_deadline_to_millis_timeout(gpr_timespec deadline, + gpr_timespec now); + +/* Allow kick to wakeup the currently polling worker */ +#define GRPC_POLLSET_CAN_KICK_SELF 1 +/* Force the wakee to repoll when awoken */ +#define GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP 2 +/* As per pollset_kick, with an extended set of flags (defined above) + -- mostly for fd_posix's use. */ +static void pollset_kick_ext(grpc_pollset *p, + grpc_pollset_worker *specific_worker, + uint32_t flags); + +/* turn a pollset into a multipoller: platform specific */ +typedef void (*platform_become_multipoller_type)(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, + struct grpc_fd **fds, + size_t fd_count); +static platform_become_multipoller_type platform_become_multipoller; + + +/* Return 1 if the pollset has active threads in pollset_work (pollset must + * be locked) */ +static int pollset_has_workers(grpc_pollset *pollset); + +static void remove_fd_from_all_epoll_sets(int fd); + +/******************************************************************************* + * pollset_set definitions + */ + +struct grpc_pollset_set { + gpr_mu mu; + + size_t pollset_count; + size_t pollset_capacity; + grpc_pollset **pollsets; + + size_t pollset_set_count; + size_t pollset_set_capacity; + struct grpc_pollset_set **pollset_sets; + + size_t fd_count; + size_t fd_capacity; + grpc_fd **fds; +}; + +/******************************************************************************* + * fd_posix.c + */ + +/* We need to keep a freelist not because of any concerns of malloc performance + * but instead so that implementations with multiple threads in (for example) + * epoll_wait deal with the race between pollset removal and incoming poll + * notifications. + * + * The problem is that the poller ultimately holds a reference to this + * object, so it is very difficult to know when is safe to free it, at least + * without some expensive synchronization. + * + * If we keep the object freelisted, in the worst case losing this race just + * becomes a spurious read notification on a reused fd. + */ +/* TODO(klempner): We could use some form of polling generation count to know + * when these are safe to free. */ +/* TODO(klempner): Consider disabling freelisting if we don't have multiple + * threads in poll on the same fd */ +/* TODO(klempner): Batch these allocations to reduce fragmentation */ +static grpc_fd *fd_freelist = NULL; +static gpr_mu fd_freelist_mu; + +static void freelist_fd(grpc_fd *fd) { + gpr_mu_lock(&fd_freelist_mu); + fd->freelist_next = fd_freelist; + fd_freelist = fd; + grpc_iomgr_unregister_object(&fd->iomgr_object); + gpr_mu_unlock(&fd_freelist_mu); +} + +static grpc_fd *alloc_fd(int fd) { + grpc_fd *r = NULL; + gpr_mu_lock(&fd_freelist_mu); + if (fd_freelist != NULL) { + r = fd_freelist; + fd_freelist = fd_freelist->freelist_next; + } + gpr_mu_unlock(&fd_freelist_mu); + if (r == NULL) { + r = gpr_malloc(sizeof(grpc_fd)); + gpr_mu_init(&r->mu); + } + + gpr_mu_lock(&r->mu); + gpr_atm_rel_store(&r->refst, 1); + r->shutdown = 0; + r->read_closure = CLOSURE_NOT_READY; + r->write_closure = CLOSURE_NOT_READY; + r->fd = fd; + r->inactive_watcher_root.next = r->inactive_watcher_root.prev = + &r->inactive_watcher_root; + r->freelist_next = NULL; + r->read_watcher = r->write_watcher = NULL; + r->on_done_closure = NULL; + r->closed = 0; + r->released = 0; + gpr_mu_unlock(&r->mu); + return r; +} + +static void destroy(grpc_fd *fd) { + gpr_mu_destroy(&fd->mu); + gpr_free(fd); +} + +#ifdef GRPC_FD_REF_COUNT_DEBUG +#define REF_BY(fd, n, reason) ref_by(fd, n, reason, __FILE__, __LINE__) +#define UNREF_BY(fd, n, reason) unref_by(fd, n, reason, __FILE__, __LINE__) +static void ref_by(grpc_fd *fd, int n, const char *reason, const char *file, + int line) { + gpr_log(GPR_DEBUG, "FD %d %p ref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, + gpr_atm_no_barrier_load(&fd->refst), + gpr_atm_no_barrier_load(&fd->refst) + n, reason, file, line); +#else +#define REF_BY(fd, n, reason) ref_by(fd, n) +#define UNREF_BY(fd, n, reason) unref_by(fd, n) +static void ref_by(grpc_fd *fd, int n) { +#endif + GPR_ASSERT(gpr_atm_no_barrier_fetch_add(&fd->refst, n) > 0); +} + +#ifdef GRPC_FD_REF_COUNT_DEBUG +static void unref_by(grpc_fd *fd, int n, const char *reason, const char *file, + int line) { + gpr_atm old; + gpr_log(GPR_DEBUG, "FD %d %p unref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, + gpr_atm_no_barrier_load(&fd->refst), + gpr_atm_no_barrier_load(&fd->refst) - n, reason, file, line); +#else +static void unref_by(grpc_fd *fd, int n) { + gpr_atm old; +#endif + old = gpr_atm_full_fetch_add(&fd->refst, -n); + if (old == n) { + freelist_fd(fd); + } else { + GPR_ASSERT(old > n); + } +} + +static void fd_global_init(void) { gpr_mu_init(&fd_freelist_mu); } + +static void fd_global_shutdown(void) { + gpr_mu_lock(&fd_freelist_mu); + gpr_mu_unlock(&fd_freelist_mu); + while (fd_freelist != NULL) { + grpc_fd *fd = fd_freelist; + fd_freelist = fd_freelist->freelist_next; + destroy(fd); + } + gpr_mu_destroy(&fd_freelist_mu); +} + +static grpc_fd *fd_create(int fd, const char *name) { + grpc_fd *r = alloc_fd(fd); + char *name2; + gpr_asprintf(&name2, "%s fd=%d", name, fd); + grpc_iomgr_register_object(&r->iomgr_object, name2); + gpr_free(name2); +#ifdef GRPC_FD_REF_COUNT_DEBUG + gpr_log(GPR_DEBUG, "FD %d %p create %s", fd, r, name); +#endif + return r; +} + +static bool fd_is_orphaned(grpc_fd *fd) { + return (gpr_atm_acq_load(&fd->refst) & 1) == 0; +} + +static void pollset_kick_locked(grpc_fd_watcher *watcher) { + gpr_mu_lock(&watcher->pollset->mu); + GPR_ASSERT(watcher->worker); + pollset_kick_ext(watcher->pollset, watcher->worker, + GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP); + gpr_mu_unlock(&watcher->pollset->mu); +} + +static void maybe_wake_one_watcher_locked(grpc_fd *fd) { + if (fd->inactive_watcher_root.next != &fd->inactive_watcher_root) { + pollset_kick_locked(fd->inactive_watcher_root.next); + } else if (fd->read_watcher) { + pollset_kick_locked(fd->read_watcher); + } else if (fd->write_watcher) { + pollset_kick_locked(fd->write_watcher); + } +} + +static void wake_all_watchers_locked(grpc_fd *fd) { + grpc_fd_watcher *watcher; + for (watcher = fd->inactive_watcher_root.next; + watcher != &fd->inactive_watcher_root; watcher = watcher->next) { + pollset_kick_locked(watcher); + } + if (fd->read_watcher) { + pollset_kick_locked(fd->read_watcher); + } + if (fd->write_watcher && fd->write_watcher != fd->read_watcher) { + pollset_kick_locked(fd->write_watcher); + } +} + +static int has_watchers(grpc_fd *fd) { + return fd->read_watcher != NULL || fd->write_watcher != NULL || + fd->inactive_watcher_root.next != &fd->inactive_watcher_root; +} + +static void close_fd_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + fd->closed = 1; + if (!fd->released) { + close(fd->fd); + } else { + remove_fd_from_all_epoll_sets(fd->fd); + } + grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); +} + +static int fd_wrapped_fd(grpc_fd *fd) { + if (fd->released || fd->closed) { + return -1; + } else { + return fd->fd; + } +} + +static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *on_done, int *release_fd, + const char *reason) { + fd->on_done_closure = on_done; + fd->released = release_fd != NULL; + if (!fd->released) { + shutdown(fd->fd, SHUT_RDWR); + } else { + *release_fd = fd->fd; + } + gpr_mu_lock(&fd->mu); + REF_BY(fd, 1, reason); /* remove active status, but keep referenced */ + if (!has_watchers(fd)) { + close_fd_locked(exec_ctx, fd); + } else { + wake_all_watchers_locked(fd); + } + gpr_mu_unlock(&fd->mu); + UNREF_BY(fd, 2, reason); /* drop the reference */ +} + +/* increment refcount by two to avoid changing the orphan bit */ +#ifdef GRPC_FD_REF_COUNT_DEBUG +static void fd_ref(grpc_fd *fd, const char *reason, const char *file, + int line) { + ref_by(fd, 2, reason, file, line); +} + +static void fd_unref(grpc_fd *fd, const char *reason, const char *file, + int line) { + unref_by(fd, 2, reason, file, line); +} +#else +static void fd_ref(grpc_fd *fd) { ref_by(fd, 2); } + +static void fd_unref(grpc_fd *fd) { unref_by(fd, 2); } +#endif + +static void notify_on_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure **st, grpc_closure *closure) { + if (*st == CLOSURE_NOT_READY) { + /* not ready ==> switch to a waiting state by setting the closure */ + *st = closure; + } else if (*st == CLOSURE_READY) { + /* already ready ==> queue the closure to run immediately */ + *st = CLOSURE_NOT_READY; + grpc_exec_ctx_enqueue(exec_ctx, closure, !fd->shutdown, NULL); + maybe_wake_one_watcher_locked(fd); + } else { + /* upcallptr was set to a different closure. This is an error! */ + gpr_log(GPR_ERROR, + "User called a notify_on function with a previous callback still " + "pending"); + abort(); + } +} + +/* returns 1 if state becomes not ready */ +static int set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure **st) { + if (*st == CLOSURE_READY) { + /* duplicate ready ==> ignore */ + return 0; + } else if (*st == CLOSURE_NOT_READY) { + /* not ready, and not waiting ==> flag ready */ + *st = CLOSURE_READY; + return 0; + } else { + /* waiting ==> queue closure */ + grpc_exec_ctx_enqueue(exec_ctx, *st, !fd->shutdown, NULL); + *st = CLOSURE_NOT_READY; + return 1; + } +} + +static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + gpr_mu_lock(&fd->mu); + GPR_ASSERT(!fd->shutdown); + fd->shutdown = 1; + set_ready_locked(exec_ctx, fd, &fd->read_closure); + set_ready_locked(exec_ctx, fd, &fd->write_closure); + gpr_mu_unlock(&fd->mu); +} + +static void fd_notify_on_read(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *closure) { + gpr_mu_lock(&fd->mu); + notify_on_locked(exec_ctx, fd, &fd->read_closure, closure); + gpr_mu_unlock(&fd->mu); +} + +static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *closure) { + gpr_mu_lock(&fd->mu); + notify_on_locked(exec_ctx, fd, &fd->write_closure, closure); + gpr_mu_unlock(&fd->mu); +} + +static uint32_t fd_begin_poll(grpc_fd *fd, grpc_pollset *pollset, + grpc_pollset_worker *worker, uint32_t read_mask, + uint32_t write_mask, grpc_fd_watcher *watcher) { + uint32_t mask = 0; + grpc_closure *cur; + int requested; + /* keep track of pollers that have requested our events, in case they change + */ + GRPC_FD_REF(fd, "poll"); + + gpr_mu_lock(&fd->mu); + + /* if we are shutdown, then don't add to the watcher set */ + if (fd->shutdown) { + watcher->fd = NULL; + watcher->pollset = NULL; + watcher->worker = NULL; + gpr_mu_unlock(&fd->mu); + GRPC_FD_UNREF(fd, "poll"); + return 0; + } + + /* if there is nobody polling for read, but we need to, then start doing so */ + cur = fd->read_closure; + requested = cur != CLOSURE_READY; + if (read_mask && fd->read_watcher == NULL && requested) { + fd->read_watcher = watcher; + mask |= read_mask; + } + /* if there is nobody polling for write, but we need to, then start doing so + */ + cur = fd->write_closure; + requested = cur != CLOSURE_READY; + if (write_mask && fd->write_watcher == NULL && requested) { + fd->write_watcher = watcher; + mask |= write_mask; + } + /* if not polling, remember this watcher in case we need someone to later */ + if (mask == 0 && worker != NULL) { + watcher->next = &fd->inactive_watcher_root; + watcher->prev = watcher->next->prev; + watcher->next->prev = watcher->prev->next = watcher; + } + watcher->pollset = pollset; + watcher->worker = worker; + watcher->fd = fd; + gpr_mu_unlock(&fd->mu); + + return mask; +} + +static void fd_end_poll(grpc_exec_ctx *exec_ctx, grpc_fd_watcher *watcher, + int got_read, int got_write) { + int was_polling = 0; + int kick = 0; + grpc_fd *fd = watcher->fd; + + if (fd == NULL) { + return; + } + + gpr_mu_lock(&fd->mu); + + if (watcher == fd->read_watcher) { + /* remove read watcher, kick if we still need a read */ + was_polling = 1; + if (!got_read) { + kick = 1; + } + fd->read_watcher = NULL; + } + if (watcher == fd->write_watcher) { + /* remove write watcher, kick if we still need a write */ + was_polling = 1; + if (!got_write) { + kick = 1; + } + fd->write_watcher = NULL; + } + if (!was_polling && watcher->worker != NULL) { + /* remove from inactive list */ + watcher->next->prev = watcher->prev; + watcher->prev->next = watcher->next; + } + if (got_read) { + if (set_ready_locked(exec_ctx, fd, &fd->read_closure)) { + kick = 1; + } + } + if (got_write) { + if (set_ready_locked(exec_ctx, fd, &fd->write_closure)) { + kick = 1; + } + } + if (kick) { + maybe_wake_one_watcher_locked(fd); + } + if (fd_is_orphaned(fd) && !has_watchers(fd) && !fd->closed) { + close_fd_locked(exec_ctx, fd); + } + gpr_mu_unlock(&fd->mu); + + GRPC_FD_UNREF(fd, "poll"); +} + +/******************************************************************************* + * pollset_posix.c + */ + +GPR_TLS_DECL(g_current_thread_poller); +GPR_TLS_DECL(g_current_thread_worker); + +/** The alarm system needs to be able to wakeup 'some poller' sometimes + * (specifically when a new alarm needs to be triggered earlier than the next + * alarm 'epoch'). + * This wakeup_fd gives us something to alert on when such a case occurs. */ +grpc_wakeup_fd grpc_global_wakeup_fd; + +static void remove_worker(grpc_pollset *p, grpc_pollset_worker *worker) { + worker->prev->next = worker->next; + worker->next->prev = worker->prev; +} + +static int pollset_has_workers(grpc_pollset *p) { + return p->root_worker.next != &p->root_worker; +} + +static grpc_pollset_worker *pop_front_worker(grpc_pollset *p) { + if (pollset_has_workers(p)) { + grpc_pollset_worker *w = p->root_worker.next; + remove_worker(p, w); + return w; + } else { + return NULL; + } +} + +static void push_back_worker(grpc_pollset *p, grpc_pollset_worker *worker) { + worker->next = &p->root_worker; + worker->prev = worker->next->prev; + worker->prev->next = worker->next->prev = worker; +} + +static void push_front_worker(grpc_pollset *p, grpc_pollset_worker *worker) { + worker->prev = &p->root_worker; + worker->next = worker->prev->next; + worker->prev->next = worker->next->prev = worker; +} + +static void pollset_kick_ext(grpc_pollset *p, + grpc_pollset_worker *specific_worker, + uint32_t flags) { + GPR_TIMER_BEGIN("pollset_kick_ext", 0); + + /* pollset->mu already held */ + if (specific_worker != NULL) { + if (specific_worker == GRPC_POLLSET_KICK_BROADCAST) { + GPR_TIMER_BEGIN("pollset_kick_ext.broadcast", 0); + GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); + for (specific_worker = p->root_worker.next; + specific_worker != &p->root_worker; + specific_worker = specific_worker->next) { + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + } + p->kicked_without_pollers = 1; + GPR_TIMER_END("pollset_kick_ext.broadcast", 0); + } else if (gpr_tls_get(&g_current_thread_worker) != + (intptr_t)specific_worker) { + GPR_TIMER_MARK("different_thread_worker", 0); + if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { + specific_worker->reevaluate_polling_on_wakeup = 1; + } + specific_worker->kicked_specifically = 1; + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + } else if ((flags & GRPC_POLLSET_CAN_KICK_SELF) != 0) { + GPR_TIMER_MARK("kick_yoself", 0); + if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { + specific_worker->reevaluate_polling_on_wakeup = 1; + } + specific_worker->kicked_specifically = 1; + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + } + } else if (gpr_tls_get(&g_current_thread_poller) != (intptr_t)p) { + GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); + GPR_TIMER_MARK("kick_anonymous", 0); + specific_worker = pop_front_worker(p); + if (specific_worker != NULL) { + if (gpr_tls_get(&g_current_thread_worker) == (intptr_t)specific_worker) { + GPR_TIMER_MARK("kick_anonymous_not_self", 0); + push_back_worker(p, specific_worker); + specific_worker = pop_front_worker(p); + if ((flags & GRPC_POLLSET_CAN_KICK_SELF) == 0 && + gpr_tls_get(&g_current_thread_worker) == + (intptr_t)specific_worker) { + push_back_worker(p, specific_worker); + specific_worker = NULL; + } + } + if (specific_worker != NULL) { + GPR_TIMER_MARK("finally_kick", 0); + push_back_worker(p, specific_worker); + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + } + } else { + GPR_TIMER_MARK("kicked_no_pollers", 0); + p->kicked_without_pollers = 1; + } + } + + GPR_TIMER_END("pollset_kick_ext", 0); +} + +static void pollset_kick(grpc_pollset *p, + grpc_pollset_worker *specific_worker) { + pollset_kick_ext(p, specific_worker, 0); +} + +/* global state management */ + +static void pollset_global_init(void) { + gpr_tls_init(&g_current_thread_poller); + gpr_tls_init(&g_current_thread_worker); + grpc_wakeup_fd_init(&grpc_global_wakeup_fd); +} + +static void pollset_global_shutdown(void) { + grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); + gpr_tls_destroy(&g_current_thread_poller); + gpr_tls_destroy(&g_current_thread_worker); +} + +static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } + +/* main interface */ + +static void become_basic_pollset(grpc_pollset *pollset, grpc_fd *fd_or_null); + +static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { + gpr_mu_init(&pollset->mu); + *mu = &pollset->mu; + pollset->root_worker.next = pollset->root_worker.prev = &pollset->root_worker; + pollset->in_flight_cbs = 0; + pollset->shutting_down = 0; + pollset->called_shutdown = 0; + pollset->kicked_without_pollers = 0; + pollset->idle_jobs.head = pollset->idle_jobs.tail = NULL; + pollset->local_wakeup_cache = NULL; + pollset->kicked_without_pollers = 0; + become_basic_pollset(pollset, NULL); +} + +static void pollset_destroy(grpc_pollset *pollset) { + GPR_ASSERT(pollset->in_flight_cbs == 0); + GPR_ASSERT(!pollset_has_workers(pollset)); + GPR_ASSERT(pollset->idle_jobs.head == pollset->idle_jobs.tail); + pollset->vtable->destroy(pollset); + while (pollset->local_wakeup_cache) { + grpc_cached_wakeup_fd *next = pollset->local_wakeup_cache->next; + grpc_wakeup_fd_destroy(&pollset->local_wakeup_cache->fd); + gpr_free(pollset->local_wakeup_cache); + pollset->local_wakeup_cache = next; + } + gpr_mu_destroy(&pollset->mu); +} + +static void pollset_reset(grpc_pollset *pollset) { + GPR_ASSERT(pollset->shutting_down); + GPR_ASSERT(pollset->in_flight_cbs == 0); + GPR_ASSERT(!pollset_has_workers(pollset)); + GPR_ASSERT(pollset->idle_jobs.head == pollset->idle_jobs.tail); + pollset->vtable->destroy(pollset); + pollset->shutting_down = 0; + pollset->called_shutdown = 0; + pollset->kicked_without_pollers = 0; + become_basic_pollset(pollset, NULL); +} + +static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_fd *fd) { + gpr_mu_lock(&pollset->mu); + pollset->vtable->add_fd(exec_ctx, pollset, fd, 1); +/* the following (enabled only in debug) will reacquire and then release + our lock - meaning that if the unlocking flag passed to add_fd above is + not respected, the code will deadlock (in a way that we have a chance of + debugging) */ +#ifndef NDEBUG + gpr_mu_lock(&pollset->mu); + gpr_mu_unlock(&pollset->mu); +#endif +} + +static void finish_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { + GPR_ASSERT(grpc_closure_list_empty(pollset->idle_jobs)); + pollset->vtable->finish_shutdown(pollset); + grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); +} + +static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_pollset_worker **worker_hdl, gpr_timespec now, + gpr_timespec deadline) { + grpc_pollset_worker worker; + *worker_hdl = &worker; + + /* pollset->mu already held */ + int added_worker = 0; + int locked = 1; + int queued_work = 0; + int keep_polling = 0; + GPR_TIMER_BEGIN("pollset_work", 0); + /* this must happen before we (potentially) drop pollset->mu */ + worker.next = worker.prev = NULL; + worker.reevaluate_polling_on_wakeup = 0; + if (pollset->local_wakeup_cache != NULL) { + worker.wakeup_fd = pollset->local_wakeup_cache; + pollset->local_wakeup_cache = worker.wakeup_fd->next; + } else { + worker.wakeup_fd = gpr_malloc(sizeof(*worker.wakeup_fd)); + grpc_wakeup_fd_init(&worker.wakeup_fd->fd); + } + worker.kicked_specifically = 0; + /* If there's work waiting for the pollset to be idle, and the + pollset is idle, then do that work */ + if (!pollset_has_workers(pollset) && + !grpc_closure_list_empty(pollset->idle_jobs)) { + GPR_TIMER_MARK("pollset_work.idle_jobs", 0); + grpc_exec_ctx_enqueue_list(exec_ctx, &pollset->idle_jobs, NULL); + goto done; + } + /* If we're shutting down then we don't execute any extended work */ + if (pollset->shutting_down) { + GPR_TIMER_MARK("pollset_work.shutting_down", 0); + goto done; + } + /* Give do_promote priority so we don't starve it out */ + if (pollset->in_flight_cbs) { + GPR_TIMER_MARK("pollset_work.in_flight_cbs", 0); + gpr_mu_unlock(&pollset->mu); + locked = 0; + goto done; + } + /* Start polling, and keep doing so while we're being asked to + re-evaluate our pollers (this allows poll() based pollers to + ensure they don't miss wakeups) */ + keep_polling = 1; + while (keep_polling) { + keep_polling = 0; + if (!pollset->kicked_without_pollers) { + if (!added_worker) { + push_front_worker(pollset, &worker); + added_worker = 1; + gpr_tls_set(&g_current_thread_worker, (intptr_t)&worker); + } + gpr_tls_set(&g_current_thread_poller, (intptr_t)pollset); + GPR_TIMER_BEGIN("maybe_work_and_unlock", 0); + pollset->vtable->maybe_work_and_unlock(exec_ctx, pollset, &worker, + deadline, now); + GPR_TIMER_END("maybe_work_and_unlock", 0); + locked = 0; + gpr_tls_set(&g_current_thread_poller, 0); + } else { + GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); + pollset->kicked_without_pollers = 0; + } + /* Finished execution - start cleaning up. + Note that we may arrive here from outside the enclosing while() loop. + In that case we won't loop though as we haven't added worker to the + worker list, which means nobody could ask us to re-evaluate polling). */ + done: + if (!locked) { + queued_work |= grpc_exec_ctx_flush(exec_ctx); + gpr_mu_lock(&pollset->mu); + locked = 1; + } + /* If we're forced to re-evaluate polling (via pollset_kick with + GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) then we land here and force + a loop */ + if (worker.reevaluate_polling_on_wakeup) { + worker.reevaluate_polling_on_wakeup = 0; + pollset->kicked_without_pollers = 0; + if (queued_work || worker.kicked_specifically) { + /* If there's queued work on the list, then set the deadline to be + immediate so we get back out of the polling loop quickly */ + deadline = gpr_inf_past(GPR_CLOCK_MONOTONIC); + } + keep_polling = 1; + } + } + if (added_worker) { + remove_worker(pollset, &worker); + gpr_tls_set(&g_current_thread_worker, 0); + } + /* release wakeup fd to the local pool */ + worker.wakeup_fd->next = pollset->local_wakeup_cache; + pollset->local_wakeup_cache = worker.wakeup_fd; + /* check shutdown conditions */ + if (pollset->shutting_down) { + if (pollset_has_workers(pollset)) { + pollset_kick(pollset, NULL); + } else if (!pollset->called_shutdown && pollset->in_flight_cbs == 0) { + pollset->called_shutdown = 1; + gpr_mu_unlock(&pollset->mu); + finish_shutdown(exec_ctx, pollset); + grpc_exec_ctx_flush(exec_ctx); + /* Continuing to access pollset here is safe -- it is the caller's + * responsibility to not destroy when it has outstanding calls to + * pollset_work. + * TODO(dklempner): Can we refactor the shutdown logic to avoid this? */ + gpr_mu_lock(&pollset->mu); + } else if (!grpc_closure_list_empty(pollset->idle_jobs)) { + grpc_exec_ctx_enqueue_list(exec_ctx, &pollset->idle_jobs, NULL); + gpr_mu_unlock(&pollset->mu); + grpc_exec_ctx_flush(exec_ctx); + gpr_mu_lock(&pollset->mu); + } + } + *worker_hdl = NULL; + GPR_TIMER_END("pollset_work", 0); +} + +static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_closure *closure) { + GPR_ASSERT(!pollset->shutting_down); + pollset->shutting_down = 1; + pollset->shutdown_done = closure; + pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); + if (!pollset_has_workers(pollset)) { + grpc_exec_ctx_enqueue_list(exec_ctx, &pollset->idle_jobs, NULL); + } + if (!pollset->called_shutdown && pollset->in_flight_cbs == 0 && + !pollset_has_workers(pollset)) { + pollset->called_shutdown = 1; + finish_shutdown(exec_ctx, pollset); + } +} + +static int poll_deadline_to_millis_timeout(gpr_timespec deadline, + gpr_timespec now) { + gpr_timespec timeout; + static const int64_t max_spin_polling_us = 10; + if (gpr_time_cmp(deadline, gpr_inf_future(deadline.clock_type)) == 0) { + return -1; + } + if (gpr_time_cmp(deadline, gpr_time_add(now, gpr_time_from_micros( + max_spin_polling_us, + GPR_TIMESPAN))) <= 0) { + return 0; + } + timeout = gpr_time_sub(deadline, now); + return gpr_time_to_millis(gpr_time_add( + timeout, gpr_time_from_nanos(GPR_NS_PER_MS - 1, GPR_TIMESPAN))); +} + +/* + * basic_pollset - a vtable that provides polling for zero or one file + * descriptor via poll() + */ + +typedef struct grpc_unary_promote_args { + const grpc_pollset_vtable *original_vtable; + grpc_pollset *pollset; + grpc_fd *fd; + grpc_closure promotion_closure; +} grpc_unary_promote_args; + +static void basic_do_promote(grpc_exec_ctx *exec_ctx, void *args, + bool success) { + grpc_unary_promote_args *up_args = args; + const grpc_pollset_vtable *original_vtable = up_args->original_vtable; + grpc_pollset *pollset = up_args->pollset; + grpc_fd *fd = up_args->fd; + + /* + * This is quite tricky. There are a number of cases to keep in mind here: + * 1. fd may have been orphaned + * 2. The pollset may no longer be a unary poller (and we can't let case #1 + * leak to other pollset types!) + * 3. pollset's fd (which may have changed) may have been orphaned + * 4. The pollset may be shutting down. + */ + + gpr_mu_lock(&pollset->mu); + /* First we need to ensure that nobody is polling concurrently */ + GPR_ASSERT(!pollset_has_workers(pollset)); + + gpr_free(up_args); + /* At this point the pollset may no longer be a unary poller. In that case + * we should just call the right add function and be done. */ + /* TODO(klempner): If we're not careful this could cause infinite recursion. + * That's not a problem for now because empty_pollset has a trivial poller + * and we don't have any mechanism to unbecome multipoller. */ + pollset->in_flight_cbs--; + if (pollset->shutting_down) { + /* We don't care about this pollset anymore. */ + if (pollset->in_flight_cbs == 0 && !pollset->called_shutdown) { + pollset->called_shutdown = 1; + finish_shutdown(exec_ctx, pollset); + } + } else if (fd_is_orphaned(fd)) { + /* Don't try to add it to anything, we'll drop our ref on it below */ + } else if (pollset->vtable != original_vtable) { + pollset->vtable->add_fd(exec_ctx, pollset, fd, 0); + } else if (fd != pollset->data.ptr) { + grpc_fd *fds[2]; + fds[0] = pollset->data.ptr; + fds[1] = fd; + + if (fds[0] && !fd_is_orphaned(fds[0])) { + platform_become_multipoller(exec_ctx, pollset, fds, GPR_ARRAY_SIZE(fds)); + GRPC_FD_UNREF(fds[0], "basicpoll"); + } else { + /* old fd is orphaned and we haven't cleaned it up until now, so remain a + * unary poller */ + /* Note that it is possible that fds[1] is also orphaned at this point. + * That's okay, we'll correct it at the next add or poll. */ + if (fds[0]) GRPC_FD_UNREF(fds[0], "basicpoll"); + pollset->data.ptr = fd; + GRPC_FD_REF(fd, "basicpoll"); + } + } + + gpr_mu_unlock(&pollset->mu); + + /* Matching ref in basic_pollset_add_fd */ + GRPC_FD_UNREF(fd, "basicpoll_add"); +} + +static void basic_pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_fd *fd, int and_unlock_pollset) { + grpc_unary_promote_args *up_args; + GPR_ASSERT(fd); + if (fd == pollset->data.ptr) goto exit; + + if (!pollset_has_workers(pollset)) { + /* Fast path -- no in flight cbs */ + /* TODO(klempner): Comment this out and fix any test failures or establish + * they are due to timing issues */ + grpc_fd *fds[2]; + fds[0] = pollset->data.ptr; + fds[1] = fd; + + if (fds[0] == NULL) { + pollset->data.ptr = fd; + GRPC_FD_REF(fd, "basicpoll"); + } else if (!fd_is_orphaned(fds[0])) { + platform_become_multipoller(exec_ctx, pollset, fds, GPR_ARRAY_SIZE(fds)); + GRPC_FD_UNREF(fds[0], "basicpoll"); + } else { + /* old fd is orphaned and we haven't cleaned it up until now, so remain a + * unary poller */ + GRPC_FD_UNREF(fds[0], "basicpoll"); + pollset->data.ptr = fd; + GRPC_FD_REF(fd, "basicpoll"); + } + goto exit; + } + + /* Now we need to promote. This needs to happen when we're not polling. Since + * this may be called from poll, the wait needs to happen asynchronously. */ + GRPC_FD_REF(fd, "basicpoll_add"); + pollset->in_flight_cbs++; + up_args = gpr_malloc(sizeof(*up_args)); + up_args->fd = fd; + up_args->original_vtable = pollset->vtable; + up_args->pollset = pollset; + up_args->promotion_closure.cb = basic_do_promote; + up_args->promotion_closure.cb_arg = up_args; + + grpc_closure_list_add(&pollset->idle_jobs, &up_args->promotion_closure, 1); + pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); + +exit: + if (and_unlock_pollset) { + gpr_mu_unlock(&pollset->mu); + } +} + +static void basic_pollset_maybe_work_and_unlock(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, + grpc_pollset_worker *worker, + gpr_timespec deadline, + gpr_timespec now) { +#define POLLOUT_CHECK (POLLOUT | POLLHUP | POLLERR) +#define POLLIN_CHECK (POLLIN | POLLHUP | POLLERR) + + struct pollfd pfd[3]; + grpc_fd *fd; + grpc_fd_watcher fd_watcher; + int timeout; + int r; + nfds_t nfds; + + fd = pollset->data.ptr; + if (fd && fd_is_orphaned(fd)) { + GRPC_FD_UNREF(fd, "basicpoll"); + fd = pollset->data.ptr = NULL; + } + timeout = poll_deadline_to_millis_timeout(deadline, now); + pfd[0].fd = GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd); + pfd[0].events = POLLIN; + pfd[0].revents = 0; + pfd[1].fd = GRPC_WAKEUP_FD_GET_READ_FD(&worker->wakeup_fd->fd); + pfd[1].events = POLLIN; + pfd[1].revents = 0; + nfds = 2; + if (fd) { + pfd[2].fd = fd->fd; + pfd[2].revents = 0; + GRPC_FD_REF(fd, "basicpoll_begin"); + gpr_mu_unlock(&pollset->mu); + pfd[2].events = + (short)fd_begin_poll(fd, pollset, worker, POLLIN, POLLOUT, &fd_watcher); + if (pfd[2].events != 0) { + nfds++; + } + } else { + gpr_mu_unlock(&pollset->mu); + } + + /* TODO(vpai): Consider first doing a 0 timeout poll here to avoid + even going into the blocking annotation if possible */ + /* poll fd count (argument 2) is shortened by one if we have no events + to poll on - such that it only includes the kicker */ + GPR_TIMER_BEGIN("poll", 0); + GRPC_SCHEDULING_START_BLOCKING_REGION; + r = grpc_poll_function(pfd, nfds, timeout); + GRPC_SCHEDULING_END_BLOCKING_REGION; + GPR_TIMER_END("poll", 0); + + if (r < 0) { + if (errno != EINTR) { + gpr_log(GPR_ERROR, "poll() failed: %s", strerror(errno)); + } + if (fd) { + fd_end_poll(exec_ctx, &fd_watcher, 0, 0); + } + } else if (r == 0) { + if (fd) { + fd_end_poll(exec_ctx, &fd_watcher, 0, 0); + } + } else { + if (pfd[0].revents & POLLIN_CHECK) { + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + } + if (pfd[1].revents & POLLIN_CHECK) { + grpc_wakeup_fd_consume_wakeup(&worker->wakeup_fd->fd); + } + if (nfds > 2) { + fd_end_poll(exec_ctx, &fd_watcher, pfd[2].revents & POLLIN_CHECK, + pfd[2].revents & POLLOUT_CHECK); + } else if (fd) { + fd_end_poll(exec_ctx, &fd_watcher, 0, 0); + } + } + + if (fd) { + GRPC_FD_UNREF(fd, "basicpoll_begin"); + } +} + +static void basic_pollset_destroy(grpc_pollset *pollset) { + if (pollset->data.ptr != NULL) { + GRPC_FD_UNREF(pollset->data.ptr, "basicpoll"); + pollset->data.ptr = NULL; + } +} + +static const grpc_pollset_vtable basic_pollset = { + basic_pollset_add_fd, basic_pollset_maybe_work_and_unlock, + basic_pollset_destroy, basic_pollset_destroy}; + +static void become_basic_pollset(grpc_pollset *pollset, grpc_fd *fd_or_null) { + pollset->vtable = &basic_pollset; + pollset->data.ptr = fd_or_null; + if (fd_or_null != NULL) { + GRPC_FD_REF(fd_or_null, "basicpoll"); + } +} + + +/******************************************************************************* + * pollset_multipoller_with_epoll_posix.c + */ + +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "src/core/lib/iomgr/ev_posix.h" +#include "src/core/lib/profiling/timers.h" +#include "src/core/lib/support/block_annotate.h" + +static void set_ready(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st) { + /* only one set_ready can be active at once (but there may be a racing + notify_on) */ + gpr_mu_lock(&fd->mu); + set_ready_locked(exec_ctx, fd, st); + gpr_mu_unlock(&fd->mu); +} + +static void fd_become_readable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + set_ready(exec_ctx, fd, &fd->read_closure); +} + +static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + set_ready(exec_ctx, fd, &fd->write_closure); +} + +struct epoll_fd_list { + int *epoll_fds; + size_t count; + size_t capacity; +}; + +static struct epoll_fd_list epoll_fd_global_list; +static gpr_once init_epoll_fd_list_mu = GPR_ONCE_INIT; +static gpr_mu epoll_fd_list_mu; + +static void init_mu(void) { gpr_mu_init(&epoll_fd_list_mu); } + +static void add_epoll_fd_to_global_list(int epoll_fd) { + gpr_once_init(&init_epoll_fd_list_mu, init_mu); + + gpr_mu_lock(&epoll_fd_list_mu); + if (epoll_fd_global_list.count == epoll_fd_global_list.capacity) { + epoll_fd_global_list.capacity = + GPR_MAX((size_t)8, epoll_fd_global_list.capacity * 2); + epoll_fd_global_list.epoll_fds = + gpr_realloc(epoll_fd_global_list.epoll_fds, + epoll_fd_global_list.capacity * sizeof(int)); + } + epoll_fd_global_list.epoll_fds[epoll_fd_global_list.count++] = epoll_fd; + gpr_mu_unlock(&epoll_fd_list_mu); +} + +static void remove_epoll_fd_from_global_list(int epoll_fd) { + gpr_mu_lock(&epoll_fd_list_mu); + GPR_ASSERT(epoll_fd_global_list.count > 0); + for (size_t i = 0; i < epoll_fd_global_list.count; i++) { + if (epoll_fd == epoll_fd_global_list.epoll_fds[i]) { + epoll_fd_global_list.epoll_fds[i] = + epoll_fd_global_list.epoll_fds[--(epoll_fd_global_list.count)]; + break; + } + } + gpr_mu_unlock(&epoll_fd_list_mu); +} + +static void remove_fd_from_all_epoll_sets(int fd) { + int err; + gpr_once_init(&init_epoll_fd_list_mu, init_mu); + gpr_mu_lock(&epoll_fd_list_mu); + if (epoll_fd_global_list.count == 0) { + gpr_mu_unlock(&epoll_fd_list_mu); + return; + } + for (size_t i = 0; i < epoll_fd_global_list.count; i++) { + err = epoll_ctl(epoll_fd_global_list.epoll_fds[i], EPOLL_CTL_DEL, fd, NULL); + if (err < 0 && errno != ENOENT) { + gpr_log(GPR_ERROR, "epoll_ctl del for %d failed: %s", fd, + strerror(errno)); + } + } + gpr_mu_unlock(&epoll_fd_list_mu); +} + +typedef struct { + grpc_pollset *pollset; + grpc_fd *fd; + grpc_closure closure; +} delayed_add; + +typedef struct { int epoll_fd; } epoll_hdr; + +static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_fd *fd) { + epoll_hdr *h = pollset->data.ptr; + struct epoll_event ev; + int err; + grpc_fd_watcher watcher; + + /* We pretend to be polling whilst adding an fd to keep the fd from being + closed during the add. This may result in a spurious wakeup being assigned + to this pollset whilst adding, but that should be benign. */ + GPR_ASSERT(fd_begin_poll(fd, pollset, NULL, 0, 0, &watcher) == 0); + if (watcher.fd != NULL) { + ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); + ev.data.ptr = fd; + err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); + if (err < 0) { + /* FDs may be added to a pollset multiple times, so EEXIST is normal. */ + if (errno != EEXIST) { + gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", fd->fd, + strerror(errno)); + } + } + } + fd_end_poll(exec_ctx, &watcher, 0, 0); +} + +static void perform_delayed_add(grpc_exec_ctx *exec_ctx, void *arg, + bool iomgr_status) { + delayed_add *da = arg; + + if (!fd_is_orphaned(da->fd)) { + finally_add_fd(exec_ctx, da->pollset, da->fd); + } + + gpr_mu_lock(&da->pollset->mu); + da->pollset->in_flight_cbs--; + if (da->pollset->shutting_down) { + /* We don't care about this pollset anymore. */ + if (da->pollset->in_flight_cbs == 0 && !da->pollset->called_shutdown) { + da->pollset->called_shutdown = 1; + grpc_exec_ctx_enqueue(exec_ctx, da->pollset->shutdown_done, true, NULL); + } + } + gpr_mu_unlock(&da->pollset->mu); + + GRPC_FD_UNREF(da->fd, "delayed_add"); + + gpr_free(da); +} + +static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, + grpc_fd *fd, + int and_unlock_pollset) { + if (and_unlock_pollset) { + gpr_mu_unlock(&pollset->mu); + finally_add_fd(exec_ctx, pollset, fd); + } else { + delayed_add *da = gpr_malloc(sizeof(*da)); + da->pollset = pollset; + da->fd = fd; + GRPC_FD_REF(fd, "delayed_add"); + grpc_closure_init(&da->closure, perform_delayed_add, da); + pollset->in_flight_cbs++; + grpc_exec_ctx_enqueue(exec_ctx, &da->closure, true, NULL); + } +} + +/* TODO(klempner): We probably want to turn this down a bit */ +#define GRPC_EPOLL_MAX_EVENTS 1000 + +static void multipoll_with_epoll_pollset_maybe_work_and_unlock( + grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, + gpr_timespec deadline, gpr_timespec now) { + struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; + int ep_rv; + int poll_rv; + epoll_hdr *h = pollset->data.ptr; + int timeout_ms; + struct pollfd pfds[2]; + + /* If you want to ignore epoll's ability to sanely handle parallel pollers, + * for a more apples-to-apples performance comparison with poll, add a + * if (pollset->counter != 0) { return 0; } + * here. + */ + + gpr_mu_unlock(&pollset->mu); + + timeout_ms = poll_deadline_to_millis_timeout(deadline, now); + + pfds[0].fd = GRPC_WAKEUP_FD_GET_READ_FD(&worker->wakeup_fd->fd); + pfds[0].events = POLLIN; + pfds[0].revents = 0; + pfds[1].fd = h->epoll_fd; + pfds[1].events = POLLIN; + pfds[1].revents = 0; + + /* TODO(vpai): Consider first doing a 0 timeout poll here to avoid + even going into the blocking annotation if possible */ + GPR_TIMER_BEGIN("poll", 0); + GRPC_SCHEDULING_START_BLOCKING_REGION; + poll_rv = grpc_poll_function(pfds, 2, timeout_ms); + GRPC_SCHEDULING_END_BLOCKING_REGION; + GPR_TIMER_END("poll", 0); + + if (poll_rv < 0) { + if (errno != EINTR) { + gpr_log(GPR_ERROR, "poll() failed: %s", strerror(errno)); + } + } else if (poll_rv == 0) { + /* do nothing */ + } else { + if (pfds[0].revents) { + grpc_wakeup_fd_consume_wakeup(&worker->wakeup_fd->fd); + } + if (pfds[1].revents) { + do { + /* The following epoll_wait never blocks; it has a timeout of 0 */ + ep_rv = epoll_wait(h->epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); + if (ep_rv < 0) { + if (errno != EINTR) { + gpr_log(GPR_ERROR, "epoll_wait() failed: %s", strerror(errno)); + } + } else { + int i; + for (i = 0; i < ep_rv; ++i) { + grpc_fd *fd = ep_ev[i].data.ptr; + /* TODO(klempner): We might want to consider making err and pri + * separate events */ + int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + int write_ev = ep_ev[i].events & EPOLLOUT; + if (fd == NULL) { + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + } else { + if (read_ev || cancel) { + fd_become_readable(exec_ctx, fd); + } + if (write_ev || cancel) { + fd_become_writable(exec_ctx, fd); + } + } + } + } + } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); + } + } +} + +static void multipoll_with_epoll_pollset_finish_shutdown( + grpc_pollset *pollset) {} + +static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset) { + epoll_hdr *h = pollset->data.ptr; + close(h->epoll_fd); + remove_epoll_fd_from_global_list(h->epoll_fd); + gpr_free(h); +} + +static const grpc_pollset_vtable multipoll_with_epoll_pollset = { + multipoll_with_epoll_pollset_add_fd, + multipoll_with_epoll_pollset_maybe_work_and_unlock, + multipoll_with_epoll_pollset_finish_shutdown, + multipoll_with_epoll_pollset_destroy}; + +static void epoll_become_multipoller(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, grpc_fd **fds, + size_t nfds) { + size_t i; + epoll_hdr *h = gpr_malloc(sizeof(epoll_hdr)); + struct epoll_event ev; + int err; + + pollset->vtable = &multipoll_with_epoll_pollset; + pollset->data.ptr = h; + h->epoll_fd = epoll_create1(EPOLL_CLOEXEC); + if (h->epoll_fd < 0) { + /* TODO(klempner): Fall back to poll here, especially on ENOSYS */ + gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); + abort(); + } + add_epoll_fd_to_global_list(h->epoll_fd); + + ev.events = (uint32_t)(EPOLLIN | EPOLLET); + ev.data.ptr = NULL; + err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); + if (err < 0) { + gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), + strerror(errno)); + } + + for (i = 0; i < nfds; i++) { + multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fds[i], 0); + } +} + +/******************************************************************************* + * pollset_set_posix.c + */ + +static grpc_pollset_set *pollset_set_create(void) { + grpc_pollset_set *pollset_set = gpr_malloc(sizeof(*pollset_set)); + memset(pollset_set, 0, sizeof(*pollset_set)); + gpr_mu_init(&pollset_set->mu); + return pollset_set; +} + +static void pollset_set_destroy(grpc_pollset_set *pollset_set) { + size_t i; + gpr_mu_destroy(&pollset_set->mu); + for (i = 0; i < pollset_set->fd_count; i++) { + GRPC_FD_UNREF(pollset_set->fds[i], "pollset_set"); + } + gpr_free(pollset_set->pollsets); + gpr_free(pollset_set->pollset_sets); + gpr_free(pollset_set->fds); + gpr_free(pollset_set); +} + +static void pollset_set_add_pollset(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, + grpc_pollset *pollset) { + size_t i, j; + gpr_mu_lock(&pollset_set->mu); + if (pollset_set->pollset_count == pollset_set->pollset_capacity) { + pollset_set->pollset_capacity = + GPR_MAX(8, 2 * pollset_set->pollset_capacity); + pollset_set->pollsets = + gpr_realloc(pollset_set->pollsets, pollset_set->pollset_capacity * + sizeof(*pollset_set->pollsets)); + } + pollset_set->pollsets[pollset_set->pollset_count++] = pollset; + for (i = 0, j = 0; i < pollset_set->fd_count; i++) { + if (fd_is_orphaned(pollset_set->fds[i])) { + GRPC_FD_UNREF(pollset_set->fds[i], "pollset_set"); + } else { + pollset_add_fd(exec_ctx, pollset, pollset_set->fds[i]); + pollset_set->fds[j++] = pollset_set->fds[i]; + } + } + pollset_set->fd_count = j; + gpr_mu_unlock(&pollset_set->mu); +} + +static void pollset_set_del_pollset(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, + grpc_pollset *pollset) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + for (i = 0; i < pollset_set->pollset_count; i++) { + if (pollset_set->pollsets[i] == pollset) { + pollset_set->pollset_count--; + GPR_SWAP(grpc_pollset *, pollset_set->pollsets[i], + pollset_set->pollsets[pollset_set->pollset_count]); + break; + } + } + gpr_mu_unlock(&pollset_set->mu); +} + +static void pollset_set_add_pollset_set(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *bag, + grpc_pollset_set *item) { + size_t i, j; + gpr_mu_lock(&bag->mu); + if (bag->pollset_set_count == bag->pollset_set_capacity) { + bag->pollset_set_capacity = GPR_MAX(8, 2 * bag->pollset_set_capacity); + bag->pollset_sets = + gpr_realloc(bag->pollset_sets, + bag->pollset_set_capacity * sizeof(*bag->pollset_sets)); + } + bag->pollset_sets[bag->pollset_set_count++] = item; + for (i = 0, j = 0; i < bag->fd_count; i++) { + if (fd_is_orphaned(bag->fds[i])) { + GRPC_FD_UNREF(bag->fds[i], "pollset_set"); + } else { + pollset_set_add_fd(exec_ctx, item, bag->fds[i]); + bag->fds[j++] = bag->fds[i]; + } + } + bag->fd_count = j; + gpr_mu_unlock(&bag->mu); +} + +static void pollset_set_del_pollset_set(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *bag, + grpc_pollset_set *item) { + size_t i; + gpr_mu_lock(&bag->mu); + for (i = 0; i < bag->pollset_set_count; i++) { + if (bag->pollset_sets[i] == item) { + bag->pollset_set_count--; + GPR_SWAP(grpc_pollset_set *, bag->pollset_sets[i], + bag->pollset_sets[bag->pollset_set_count]); + break; + } + } + gpr_mu_unlock(&bag->mu); +} + +static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + if (pollset_set->fd_count == pollset_set->fd_capacity) { + pollset_set->fd_capacity = GPR_MAX(8, 2 * pollset_set->fd_capacity); + pollset_set->fds = gpr_realloc( + pollset_set->fds, pollset_set->fd_capacity * sizeof(*pollset_set->fds)); + } + GRPC_FD_REF(fd, "pollset_set"); + pollset_set->fds[pollset_set->fd_count++] = fd; + for (i = 0; i < pollset_set->pollset_count; i++) { + pollset_add_fd(exec_ctx, pollset_set->pollsets[i], fd); + } + for (i = 0; i < pollset_set->pollset_set_count; i++) { + pollset_set_add_fd(exec_ctx, pollset_set->pollset_sets[i], fd); + } + gpr_mu_unlock(&pollset_set->mu); +} + +static void pollset_set_del_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + for (i = 0; i < pollset_set->fd_count; i++) { + if (pollset_set->fds[i] == fd) { + pollset_set->fd_count--; + GPR_SWAP(grpc_fd *, pollset_set->fds[i], + pollset_set->fds[pollset_set->fd_count]); + GRPC_FD_UNREF(fd, "pollset_set"); + break; + } + } + for (i = 0; i < pollset_set->pollset_set_count; i++) { + pollset_set_del_fd(exec_ctx, pollset_set->pollset_sets[i], fd); + } + gpr_mu_unlock(&pollset_set->mu); +} + +/******************************************************************************* + * event engine binding + */ + +static void shutdown_engine(void) { + fd_global_shutdown(); + pollset_global_shutdown(); +} + +static const grpc_event_engine_vtable vtable = { + .pollset_size = sizeof(grpc_pollset), + + .fd_create = fd_create, + .fd_wrapped_fd = fd_wrapped_fd, + .fd_orphan = fd_orphan, + .fd_shutdown = fd_shutdown, + .fd_notify_on_read = fd_notify_on_read, + .fd_notify_on_write = fd_notify_on_write, + + .pollset_init = pollset_init, + .pollset_shutdown = pollset_shutdown, + .pollset_reset = pollset_reset, + .pollset_destroy = pollset_destroy, + .pollset_work = pollset_work, + .pollset_kick = pollset_kick, + .pollset_add_fd = pollset_add_fd, + + .pollset_set_create = pollset_set_create, + .pollset_set_destroy = pollset_set_destroy, + .pollset_set_add_pollset = pollset_set_add_pollset, + .pollset_set_del_pollset = pollset_set_del_pollset, + .pollset_set_add_pollset_set = pollset_set_add_pollset_set, + .pollset_set_del_pollset_set = pollset_set_del_pollset_set, + .pollset_set_add_fd = pollset_set_add_fd, + .pollset_set_del_fd = pollset_set_del_fd, + + .kick_poller = kick_poller, + + .shutdown_engine = shutdown_engine, +}; + +const grpc_event_engine_vtable *grpc_init_epoll_posix(void) { + platform_become_multipoller = epoll_become_multipoller; + fd_global_init(); + pollset_global_init(); + return &vtable; +} + +#endif diff --git a/src/core/lib/iomgr/ev_epoll_posix.h b/src/core/lib/iomgr/ev_epoll_posix.h new file mode 100644 index 00000000000..35319b4fc5d --- /dev/null +++ b/src/core/lib/iomgr/ev_epoll_posix.h @@ -0,0 +1,41 @@ +/* + * + * Copyright 2015, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#ifndef GRPC_CORE_LIB_IOMGR_EV_EPOLL_POSIX_H +#define GRPC_CORE_LIB_IOMGR_EV_EPOLL_POSIX_H + +#include "src/core/lib/iomgr/ev_posix.h" + +const grpc_event_engine_vtable *grpc_init_epoll_posix(void); + +#endif /* GRPC_CORE_LIB_IOMGR_EV_EPOLL_POSIX_H */ diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index 95520b01d3a..baa3b9856ad 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -44,6 +44,7 @@ #include #include +#include "src/core/lib/iomgr/ev_epoll_posix.h" #include "src/core/lib/iomgr/ev_poll_posix.h" #include "src/core/lib/support/env.h" @@ -61,7 +62,7 @@ typedef struct { } event_engine_factory; static const event_engine_factory g_factories[] = { - {"poll", grpc_init_poll_posix}, + {"poll", grpc_init_poll_posix}, {"epoll", grpc_init_epoll_posix}, }; static void add(const char *beg, const char *end, char ***ss, size_t *ns) { diff --git a/src/python/grpcio/grpc_core_dependencies.py b/src/python/grpcio/grpc_core_dependencies.py index be24dc7cf02..49b4ddc457c 100644 --- a/src/python/grpcio/grpc_core_dependencies.py +++ b/src/python/grpcio/grpc_core_dependencies.py @@ -94,6 +94,7 @@ CORE_SOURCE_FILES = [ 'src/core/lib/iomgr/endpoint.c', 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', + 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', 'src/core/lib/iomgr/exec_ctx.c', diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal index c3fd59b3c23..36b25cca161 100644 --- a/tools/doxygen/Doxyfile.core.internal +++ b/tools/doxygen/Doxyfile.core.internal @@ -807,6 +807,7 @@ src/core/lib/http/parser.h \ src/core/lib/iomgr/closure.h \ src/core/lib/iomgr/endpoint.h \ src/core/lib/iomgr/endpoint_pair.h \ +src/core/lib/iomgr/ev_epoll_posix.h \ src/core/lib/iomgr/ev_poll_posix.h \ src/core/lib/iomgr/ev_posix.h \ src/core/lib/iomgr/exec_ctx.h \ @@ -954,6 +955,7 @@ src/core/lib/iomgr/closure.c \ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ +src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index 24d23fe28bd..43940495864 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -5530,6 +5530,7 @@ "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", @@ -5629,6 +5630,8 @@ "src/core/lib/iomgr/endpoint_pair.h", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_posix.c", + "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.c", diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj b/vsprojects/vcxproj/grpc/grpc.vcxproj index e5379dc6a49..55304af5869 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj @@ -316,6 +316,7 @@ + @@ -483,6 +484,8 @@ + + diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters index 95a5a73d268..7d1c90fda7c 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters @@ -55,6 +55,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr @@ -674,6 +677,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj index 90ad80f2fc2..3d0cdfc668b 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj @@ -304,6 +304,7 @@ + @@ -449,6 +450,8 @@ + + diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters index 2b19c0fb341..d2ff4c630fd 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters @@ -58,6 +58,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr @@ -572,6 +575,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr From a7786001a22f511130a4292893cc8e2ab0ccdf75 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 11 May 2016 18:13:31 -0700 Subject: [PATCH 010/280] Remove basic_pollset and the promotion related code --- src/core/lib/iomgr/ev_epoll_posix.c | 355 ++++++---------------------- 1 file changed, 71 insertions(+), 284 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index ce8d3981b3f..cb4a00a75c4 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -185,11 +185,6 @@ struct grpc_pollset_worker { }; struct grpc_pollset { - /* pollsets under posix can mutate representation as fds are added and - removed. - For example, we may choose a poll() based implementation on linux for - few fds, and an epoll() based implementation for many fds */ - const grpc_pollset_vtable *vtable; gpr_mu mu; grpc_pollset_worker root_worker; int in_flight_cbs; @@ -248,8 +243,6 @@ typedef void (*platform_become_multipoller_type)(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, struct grpc_fd **fds, size_t fd_count); -static platform_become_multipoller_type platform_become_multipoller; - /* Return 1 if the pollset has active threads in pollset_work (pollset must * be locked) */ @@ -796,8 +789,6 @@ static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } /* main interface */ -static void become_basic_pollset(grpc_pollset *pollset, grpc_fd *fd_or_null); - static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { gpr_mu_init(&pollset->mu); *mu = &pollset->mu; @@ -809,14 +800,20 @@ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { pollset->idle_jobs.head = pollset->idle_jobs.tail = NULL; pollset->local_wakeup_cache = NULL; pollset->kicked_without_pollers = 0; - become_basic_pollset(pollset, NULL); + pollset->data.ptr = NULL; } +/* TODO(sreek): Maybe merge multipoll_*_destroy() with pollset_destroy() + * function */ +static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset); + static void pollset_destroy(grpc_pollset *pollset) { GPR_ASSERT(pollset->in_flight_cbs == 0); GPR_ASSERT(!pollset_has_workers(pollset)); GPR_ASSERT(pollset->idle_jobs.head == pollset->idle_jobs.tail); - pollset->vtable->destroy(pollset); + + multipoll_with_epoll_pollset_destroy(pollset); + while (pollset->local_wakeup_cache) { grpc_cached_wakeup_fd *next = pollset->local_wakeup_cache->next; grpc_wakeup_fd_destroy(&pollset->local_wakeup_cache->fd); @@ -831,17 +828,24 @@ static void pollset_reset(grpc_pollset *pollset) { GPR_ASSERT(pollset->in_flight_cbs == 0); GPR_ASSERT(!pollset_has_workers(pollset)); GPR_ASSERT(pollset->idle_jobs.head == pollset->idle_jobs.tail); - pollset->vtable->destroy(pollset); + + multipoll_with_epoll_pollset_destroy(pollset); + pollset->shutting_down = 0; pollset->called_shutdown = 0; pollset->kicked_without_pollers = 0; - become_basic_pollset(pollset, NULL); } +/* TODO (sreek): Remove multipoll_with_epoll_add_fd declaration*/ +static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, + grpc_fd *fd, + int and_unlock_pollset); + static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { gpr_mu_lock(&pollset->mu); - pollset->vtable->add_fd(exec_ctx, pollset, fd, 1); + multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fd, 1); /* the following (enabled only in debug) will reacquire and then release our lock - meaning that if the unlocking flag passed to add_fd above is not respected, the code will deadlock (in a way that we have a chance of @@ -852,12 +856,21 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, #endif } +/* TODO (sreek): Remove multipoll_with_epoll_finish_shutdown() declaration */ +static void multipoll_with_epoll_pollset_finish_shutdown(grpc_pollset *pollset); + static void finish_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { GPR_ASSERT(grpc_closure_list_empty(pollset->idle_jobs)); - pollset->vtable->finish_shutdown(pollset); + multipoll_with_epoll_pollset_finish_shutdown(pollset); grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); } +/* TODO(sreek): Remove multipoll_with_epoll_*_maybe_work_and_unlock declaration + */ +static void multipoll_with_epoll_pollset_maybe_work_and_unlock( + grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, + gpr_timespec deadline, gpr_timespec now); + static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker **worker_hdl, gpr_timespec now, gpr_timespec deadline) { @@ -915,8 +928,10 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } gpr_tls_set(&g_current_thread_poller, (intptr_t)pollset); GPR_TIMER_BEGIN("maybe_work_and_unlock", 0); - pollset->vtable->maybe_work_and_unlock(exec_ctx, pollset, &worker, - deadline, now); + + multipoll_with_epoll_pollset_maybe_work_and_unlock( + exec_ctx, pollset, &worker, deadline, now); + GPR_TIMER_END("maybe_work_and_unlock", 0); locked = 0; gpr_tls_set(&g_current_thread_poller, 0); @@ -1013,233 +1028,6 @@ static int poll_deadline_to_millis_timeout(gpr_timespec deadline, timeout, gpr_time_from_nanos(GPR_NS_PER_MS - 1, GPR_TIMESPAN))); } -/* - * basic_pollset - a vtable that provides polling for zero or one file - * descriptor via poll() - */ - -typedef struct grpc_unary_promote_args { - const grpc_pollset_vtable *original_vtable; - grpc_pollset *pollset; - grpc_fd *fd; - grpc_closure promotion_closure; -} grpc_unary_promote_args; - -static void basic_do_promote(grpc_exec_ctx *exec_ctx, void *args, - bool success) { - grpc_unary_promote_args *up_args = args; - const grpc_pollset_vtable *original_vtable = up_args->original_vtable; - grpc_pollset *pollset = up_args->pollset; - grpc_fd *fd = up_args->fd; - - /* - * This is quite tricky. There are a number of cases to keep in mind here: - * 1. fd may have been orphaned - * 2. The pollset may no longer be a unary poller (and we can't let case #1 - * leak to other pollset types!) - * 3. pollset's fd (which may have changed) may have been orphaned - * 4. The pollset may be shutting down. - */ - - gpr_mu_lock(&pollset->mu); - /* First we need to ensure that nobody is polling concurrently */ - GPR_ASSERT(!pollset_has_workers(pollset)); - - gpr_free(up_args); - /* At this point the pollset may no longer be a unary poller. In that case - * we should just call the right add function and be done. */ - /* TODO(klempner): If we're not careful this could cause infinite recursion. - * That's not a problem for now because empty_pollset has a trivial poller - * and we don't have any mechanism to unbecome multipoller. */ - pollset->in_flight_cbs--; - if (pollset->shutting_down) { - /* We don't care about this pollset anymore. */ - if (pollset->in_flight_cbs == 0 && !pollset->called_shutdown) { - pollset->called_shutdown = 1; - finish_shutdown(exec_ctx, pollset); - } - } else if (fd_is_orphaned(fd)) { - /* Don't try to add it to anything, we'll drop our ref on it below */ - } else if (pollset->vtable != original_vtable) { - pollset->vtable->add_fd(exec_ctx, pollset, fd, 0); - } else if (fd != pollset->data.ptr) { - grpc_fd *fds[2]; - fds[0] = pollset->data.ptr; - fds[1] = fd; - - if (fds[0] && !fd_is_orphaned(fds[0])) { - platform_become_multipoller(exec_ctx, pollset, fds, GPR_ARRAY_SIZE(fds)); - GRPC_FD_UNREF(fds[0], "basicpoll"); - } else { - /* old fd is orphaned and we haven't cleaned it up until now, so remain a - * unary poller */ - /* Note that it is possible that fds[1] is also orphaned at this point. - * That's okay, we'll correct it at the next add or poll. */ - if (fds[0]) GRPC_FD_UNREF(fds[0], "basicpoll"); - pollset->data.ptr = fd; - GRPC_FD_REF(fd, "basicpoll"); - } - } - - gpr_mu_unlock(&pollset->mu); - - /* Matching ref in basic_pollset_add_fd */ - GRPC_FD_UNREF(fd, "basicpoll_add"); -} - -static void basic_pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_fd *fd, int and_unlock_pollset) { - grpc_unary_promote_args *up_args; - GPR_ASSERT(fd); - if (fd == pollset->data.ptr) goto exit; - - if (!pollset_has_workers(pollset)) { - /* Fast path -- no in flight cbs */ - /* TODO(klempner): Comment this out and fix any test failures or establish - * they are due to timing issues */ - grpc_fd *fds[2]; - fds[0] = pollset->data.ptr; - fds[1] = fd; - - if (fds[0] == NULL) { - pollset->data.ptr = fd; - GRPC_FD_REF(fd, "basicpoll"); - } else if (!fd_is_orphaned(fds[0])) { - platform_become_multipoller(exec_ctx, pollset, fds, GPR_ARRAY_SIZE(fds)); - GRPC_FD_UNREF(fds[0], "basicpoll"); - } else { - /* old fd is orphaned and we haven't cleaned it up until now, so remain a - * unary poller */ - GRPC_FD_UNREF(fds[0], "basicpoll"); - pollset->data.ptr = fd; - GRPC_FD_REF(fd, "basicpoll"); - } - goto exit; - } - - /* Now we need to promote. This needs to happen when we're not polling. Since - * this may be called from poll, the wait needs to happen asynchronously. */ - GRPC_FD_REF(fd, "basicpoll_add"); - pollset->in_flight_cbs++; - up_args = gpr_malloc(sizeof(*up_args)); - up_args->fd = fd; - up_args->original_vtable = pollset->vtable; - up_args->pollset = pollset; - up_args->promotion_closure.cb = basic_do_promote; - up_args->promotion_closure.cb_arg = up_args; - - grpc_closure_list_add(&pollset->idle_jobs, &up_args->promotion_closure, 1); - pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); - -exit: - if (and_unlock_pollset) { - gpr_mu_unlock(&pollset->mu); - } -} - -static void basic_pollset_maybe_work_and_unlock(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, - grpc_pollset_worker *worker, - gpr_timespec deadline, - gpr_timespec now) { -#define POLLOUT_CHECK (POLLOUT | POLLHUP | POLLERR) -#define POLLIN_CHECK (POLLIN | POLLHUP | POLLERR) - - struct pollfd pfd[3]; - grpc_fd *fd; - grpc_fd_watcher fd_watcher; - int timeout; - int r; - nfds_t nfds; - - fd = pollset->data.ptr; - if (fd && fd_is_orphaned(fd)) { - GRPC_FD_UNREF(fd, "basicpoll"); - fd = pollset->data.ptr = NULL; - } - timeout = poll_deadline_to_millis_timeout(deadline, now); - pfd[0].fd = GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd); - pfd[0].events = POLLIN; - pfd[0].revents = 0; - pfd[1].fd = GRPC_WAKEUP_FD_GET_READ_FD(&worker->wakeup_fd->fd); - pfd[1].events = POLLIN; - pfd[1].revents = 0; - nfds = 2; - if (fd) { - pfd[2].fd = fd->fd; - pfd[2].revents = 0; - GRPC_FD_REF(fd, "basicpoll_begin"); - gpr_mu_unlock(&pollset->mu); - pfd[2].events = - (short)fd_begin_poll(fd, pollset, worker, POLLIN, POLLOUT, &fd_watcher); - if (pfd[2].events != 0) { - nfds++; - } - } else { - gpr_mu_unlock(&pollset->mu); - } - - /* TODO(vpai): Consider first doing a 0 timeout poll here to avoid - even going into the blocking annotation if possible */ - /* poll fd count (argument 2) is shortened by one if we have no events - to poll on - such that it only includes the kicker */ - GPR_TIMER_BEGIN("poll", 0); - GRPC_SCHEDULING_START_BLOCKING_REGION; - r = grpc_poll_function(pfd, nfds, timeout); - GRPC_SCHEDULING_END_BLOCKING_REGION; - GPR_TIMER_END("poll", 0); - - if (r < 0) { - if (errno != EINTR) { - gpr_log(GPR_ERROR, "poll() failed: %s", strerror(errno)); - } - if (fd) { - fd_end_poll(exec_ctx, &fd_watcher, 0, 0); - } - } else if (r == 0) { - if (fd) { - fd_end_poll(exec_ctx, &fd_watcher, 0, 0); - } - } else { - if (pfd[0].revents & POLLIN_CHECK) { - grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); - } - if (pfd[1].revents & POLLIN_CHECK) { - grpc_wakeup_fd_consume_wakeup(&worker->wakeup_fd->fd); - } - if (nfds > 2) { - fd_end_poll(exec_ctx, &fd_watcher, pfd[2].revents & POLLIN_CHECK, - pfd[2].revents & POLLOUT_CHECK); - } else if (fd) { - fd_end_poll(exec_ctx, &fd_watcher, 0, 0); - } - } - - if (fd) { - GRPC_FD_UNREF(fd, "basicpoll_begin"); - } -} - -static void basic_pollset_destroy(grpc_pollset *pollset) { - if (pollset->data.ptr != NULL) { - GRPC_FD_UNREF(pollset->data.ptr, "basicpoll"); - pollset->data.ptr = NULL; - } -} - -static const grpc_pollset_vtable basic_pollset = { - basic_pollset_add_fd, basic_pollset_maybe_work_and_unlock, - basic_pollset_destroy, basic_pollset_destroy}; - -static void become_basic_pollset(grpc_pollset *pollset, grpc_fd *fd_or_null) { - pollset->vtable = &basic_pollset; - pollset->data.ptr = fd_or_null; - if (fd_or_null != NULL) { - GRPC_FD_REF(fd_or_null, "basicpoll"); - } -} - - /******************************************************************************* * pollset_multipoller_with_epoll_posix.c */ @@ -1274,6 +1062,7 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { set_ready(exec_ctx, fd, &fd->write_closure); } +/* TODO (sreek): Maybe this global list is not required. Double check*/ struct epoll_fd_list { int *epoll_fds; size_t count; @@ -1390,10 +1179,48 @@ static void perform_delayed_add(grpc_exec_ctx *exec_ctx, void *arg, gpr_free(da); } +/* Creates an epoll fd and initializes the pollset */ +static void multipoll_with_epoll_pollset_create_efd(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset) { + epoll_hdr *h = gpr_malloc(sizeof(epoll_hdr)); + struct epoll_event ev; + int err; + + /* Ensuring that the pollset is infact empty (with no epoll fd either) */ + GPR_ASSERT(pollset->data.ptr == NULL); + + pollset->data.ptr = h; + h->epoll_fd = epoll_create1(EPOLL_CLOEXEC); + if (h->epoll_fd < 0) { + gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); + abort(); + } + add_epoll_fd_to_global_list(h->epoll_fd); + + ev.events = (uint32_t)(EPOLLIN | EPOLLET); + ev.data.ptr = NULL; + + /* TODO (sreek): Double-check the use of grpc_global_wakeup_fd here (right now + * I do not know why this is used. I just copied this code from + * epoll_become_mutipoller() function in ev_poll_and_epoll_posix.c file */ + err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); + if (err < 0) { + gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), + strerror(errno)); + } +} + static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd, int and_unlock_pollset) { + /* If there is no epoll fd on the pollset, create one */ + if (pollset->data.ptr == NULL) { + multipoll_with_epoll_pollset_create_efd(exec_ctx, pollset); + } + if (and_unlock_pollset) { gpr_mu_unlock(&pollset->mu); finally_add_fd(exec_ctx, pollset, fd); @@ -1500,45 +1327,6 @@ static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset) { gpr_free(h); } -static const grpc_pollset_vtable multipoll_with_epoll_pollset = { - multipoll_with_epoll_pollset_add_fd, - multipoll_with_epoll_pollset_maybe_work_and_unlock, - multipoll_with_epoll_pollset_finish_shutdown, - multipoll_with_epoll_pollset_destroy}; - -static void epoll_become_multipoller(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, grpc_fd **fds, - size_t nfds) { - size_t i; - epoll_hdr *h = gpr_malloc(sizeof(epoll_hdr)); - struct epoll_event ev; - int err; - - pollset->vtable = &multipoll_with_epoll_pollset; - pollset->data.ptr = h; - h->epoll_fd = epoll_create1(EPOLL_CLOEXEC); - if (h->epoll_fd < 0) { - /* TODO(klempner): Fall back to poll here, especially on ENOSYS */ - gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); - abort(); - } - add_epoll_fd_to_global_list(h->epoll_fd); - - ev.events = (uint32_t)(EPOLLIN | EPOLLET); - ev.data.ptr = NULL; - err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); - if (err < 0) { - gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), - strerror(errno)); - } - - for (i = 0; i < nfds; i++) { - multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fds[i], 0); - } -} - /******************************************************************************* * pollset_set_posix.c */ @@ -1724,7 +1512,6 @@ static const grpc_event_engine_vtable vtable = { }; const grpc_event_engine_vtable *grpc_init_epoll_posix(void) { - platform_become_multipoller = epoll_become_multipoller; fd_global_init(); pollset_global_init(); return &vtable; From ab7f10ed61c7e0d104ea839206d2cd0340f5fed8 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 12 May 2016 20:21:37 -0700 Subject: [PATCH 011/280] Remove delayed_add --- src/core/lib/iomgr/ev_epoll_posix.c | 61 ++++++----------------------- 1 file changed, 13 insertions(+), 48 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index cb4a00a75c4..03c544b30c0 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -187,7 +187,7 @@ struct grpc_pollset_worker { struct grpc_pollset { gpr_mu mu; grpc_pollset_worker root_worker; - int in_flight_cbs; + int in_flight_cbs; /* TODO (sreek): Most likely this isn't needed anymore */ int shutting_down; int called_shutdown; int kicked_without_pollers; @@ -839,13 +839,12 @@ static void pollset_reset(grpc_pollset *pollset) { /* TODO (sreek): Remove multipoll_with_epoll_add_fd declaration*/ static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_fd *fd, - int and_unlock_pollset); + grpc_fd *fd); static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { gpr_mu_lock(&pollset->mu); - multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fd, 1); + multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fd); /* the following (enabled only in debug) will reacquire and then release our lock - meaning that if the unlocking flag passed to add_fd above is not respected, the code will deadlock (in a way that we have a chance of @@ -1121,12 +1120,6 @@ static void remove_fd_from_all_epoll_sets(int fd) { gpr_mu_unlock(&epoll_fd_list_mu); } -typedef struct { - grpc_pollset *pollset; - grpc_fd *fd; - grpc_closure closure; -} delayed_add; - typedef struct { int epoll_fd; } epoll_hdr; static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, @@ -1139,6 +1132,13 @@ static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, /* We pretend to be polling whilst adding an fd to keep the fd from being closed during the add. This may result in a spurious wakeup being assigned to this pollset whilst adding, but that should be benign. */ + /* TODO (sreek). This fd_begin_poll() really seem to accomplish adding + * GRPC_FD_REF() (i.e adding a refcount to the fd) and checking that the + * fd is not shutting down (in which case watcher.fd will be NULL and no + * refcount is added). The ref count is added only durng hte duration of + * adding it to the epoll set (after which fd_end_poll would be called and + * the fd's ref count is decremented by 1. So do we still need fd_begin_poll + * ??? */ GPR_ASSERT(fd_begin_poll(fd, pollset, NULL, 0, 0, &watcher) == 0); if (watcher.fd != NULL) { ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); @@ -1155,30 +1155,6 @@ static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, fd_end_poll(exec_ctx, &watcher, 0, 0); } -static void perform_delayed_add(grpc_exec_ctx *exec_ctx, void *arg, - bool iomgr_status) { - delayed_add *da = arg; - - if (!fd_is_orphaned(da->fd)) { - finally_add_fd(exec_ctx, da->pollset, da->fd); - } - - gpr_mu_lock(&da->pollset->mu); - da->pollset->in_flight_cbs--; - if (da->pollset->shutting_down) { - /* We don't care about this pollset anymore. */ - if (da->pollset->in_flight_cbs == 0 && !da->pollset->called_shutdown) { - da->pollset->called_shutdown = 1; - grpc_exec_ctx_enqueue(exec_ctx, da->pollset->shutdown_done, true, NULL); - } - } - gpr_mu_unlock(&da->pollset->mu); - - GRPC_FD_UNREF(da->fd, "delayed_add"); - - gpr_free(da); -} - /* Creates an epoll fd and initializes the pollset */ static void multipoll_with_epoll_pollset_create_efd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { @@ -1214,25 +1190,14 @@ static void multipoll_with_epoll_pollset_create_efd(grpc_exec_ctx *exec_ctx, static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_fd *fd, - int and_unlock_pollset) { + grpc_fd *fd) { /* If there is no epoll fd on the pollset, create one */ if (pollset->data.ptr == NULL) { multipoll_with_epoll_pollset_create_efd(exec_ctx, pollset); } - if (and_unlock_pollset) { - gpr_mu_unlock(&pollset->mu); - finally_add_fd(exec_ctx, pollset, fd); - } else { - delayed_add *da = gpr_malloc(sizeof(*da)); - da->pollset = pollset; - da->fd = fd; - GRPC_FD_REF(fd, "delayed_add"); - grpc_closure_init(&da->closure, perform_delayed_add, da); - pollset->in_flight_cbs++; - grpc_exec_ctx_enqueue(exec_ctx, &da->closure, true, NULL); - } + gpr_mu_unlock(&pollset->mu); + finally_add_fd(exec_ctx, pollset, fd); } /* TODO(klempner): We probably want to turn this down a bit */ From f6a2adf0cfa14e473e05b6cb2d6b8fdc7667945b Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 13 May 2016 16:00:20 -0700 Subject: [PATCH 012/280] Pollset_reset should not destroy the epoll_fd --- src/core/lib/iomgr/ev_epoll_posix.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index 03c544b30c0..19465280a1b 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -828,9 +828,6 @@ static void pollset_reset(grpc_pollset *pollset) { GPR_ASSERT(pollset->in_flight_cbs == 0); GPR_ASSERT(!pollset_has_workers(pollset)); GPR_ASSERT(pollset->idle_jobs.head == pollset->idle_jobs.tail); - - multipoll_with_epoll_pollset_destroy(pollset); - pollset->shutting_down = 0; pollset->called_shutdown = 0; pollset->kicked_without_pollers = 0; From 24f0f57ea16b77e710f7ff4ae8a048c888ab6b12 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 13 May 2016 19:22:44 -0700 Subject: [PATCH 013/280] Moving the creation of epoll_fd to pollset_init() instead of pollset_add_fd() [Verified stable. All tests pass] --- src/core/lib/iomgr/ev_epoll_posix.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index 19465280a1b..0ee0ee58a8b 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -787,6 +787,9 @@ static void pollset_global_shutdown(void) { static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } +/* TODO: sreek. Try to Remove this forward declaration*/ +static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset); + /* main interface */ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { @@ -800,7 +803,9 @@ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { pollset->idle_jobs.head = pollset->idle_jobs.tail = NULL; pollset->local_wakeup_cache = NULL; pollset->kicked_without_pollers = 0; + pollset->data.ptr = NULL; + multipoll_with_epoll_pollset_create_efd(pollset); } /* TODO(sreek): Maybe merge multipoll_*_destroy() with pollset_destroy() @@ -1153,13 +1158,15 @@ static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } /* Creates an epoll fd and initializes the pollset */ -static void multipoll_with_epoll_pollset_create_efd(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset) { +/* TODO: This has to be called ONLY from pollset_init function. and hence it + * does not acquire any lock */ +static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { epoll_hdr *h = gpr_malloc(sizeof(epoll_hdr)); struct epoll_event ev; int err; - /* Ensuring that the pollset is infact empty (with no epoll fd either) */ + /* TODO (sreek). remove this assert. Currently added this just to ensure that + * we do not overwrite h->epoll_fd without freeing the older one*/ GPR_ASSERT(pollset->data.ptr == NULL); pollset->data.ptr = h; @@ -1188,10 +1195,10 @@ static void multipoll_with_epoll_pollset_create_efd(grpc_exec_ctx *exec_ctx, static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { - /* If there is no epoll fd on the pollset, create one */ - if (pollset->data.ptr == NULL) { - multipoll_with_epoll_pollset_create_efd(exec_ctx, pollset); - } + GPR_ASSERT(pollset->data.ptr != NULL); + + /* TODO(sreek). Remove this unlock code (and also the code that acquires the + * lock before calling multipoll_with_epoll_add_fd() function */ gpr_mu_unlock(&pollset->mu); finally_add_fd(exec_ctx, pollset, fd); From 9ff57f67a0920e8e39baedfec5671fb6e662d257 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Sat, 14 May 2016 15:56:57 -0700 Subject: [PATCH 014/280] Remove idle_jobs and in_flight_cbs from pollset --- src/core/lib/iomgr/ev_epoll_posix.c | 50 ++--------------------------- 1 file changed, 3 insertions(+), 47 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index 0ee0ee58a8b..5776b6c1cf1 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -169,8 +169,6 @@ static void fd_global_shutdown(void); * pollset declarations */ -typedef struct grpc_pollset_vtable grpc_pollset_vtable; - typedef struct grpc_cached_wakeup_fd { grpc_wakeup_fd fd; struct grpc_cached_wakeup_fd *next; @@ -187,12 +185,10 @@ struct grpc_pollset_worker { struct grpc_pollset { gpr_mu mu; grpc_pollset_worker root_worker; - int in_flight_cbs; /* TODO (sreek): Most likely this isn't needed anymore */ int shutting_down; int called_shutdown; int kicked_without_pollers; grpc_closure *shutdown_done; - grpc_closure_list idle_jobs; union { int fd; void *ptr; @@ -201,16 +197,6 @@ struct grpc_pollset { grpc_cached_wakeup_fd *local_wakeup_cache; }; -struct grpc_pollset_vtable { - void (*add_fd)(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - struct grpc_fd *fd, int and_unlock_pollset); - void (*maybe_work_and_unlock)(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_pollset_worker *worker, - gpr_timespec deadline, gpr_timespec now); - void (*finish_shutdown)(grpc_pollset *pollset); - void (*destroy)(grpc_pollset *pollset); -}; - /* Add an fd to a pollset */ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, struct grpc_fd *fd); @@ -796,11 +782,9 @@ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { gpr_mu_init(&pollset->mu); *mu = &pollset->mu; pollset->root_worker.next = pollset->root_worker.prev = &pollset->root_worker; - pollset->in_flight_cbs = 0; pollset->shutting_down = 0; pollset->called_shutdown = 0; pollset->kicked_without_pollers = 0; - pollset->idle_jobs.head = pollset->idle_jobs.tail = NULL; pollset->local_wakeup_cache = NULL; pollset->kicked_without_pollers = 0; @@ -813,9 +797,7 @@ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset); static void pollset_destroy(grpc_pollset *pollset) { - GPR_ASSERT(pollset->in_flight_cbs == 0); GPR_ASSERT(!pollset_has_workers(pollset)); - GPR_ASSERT(pollset->idle_jobs.head == pollset->idle_jobs.tail); multipoll_with_epoll_pollset_destroy(pollset); @@ -830,9 +812,7 @@ static void pollset_destroy(grpc_pollset *pollset) { static void pollset_reset(grpc_pollset *pollset) { GPR_ASSERT(pollset->shutting_down); - GPR_ASSERT(pollset->in_flight_cbs == 0); GPR_ASSERT(!pollset_has_workers(pollset)); - GPR_ASSERT(pollset->idle_jobs.head == pollset->idle_jobs.tail); pollset->shutting_down = 0; pollset->called_shutdown = 0; pollset->kicked_without_pollers = 0; @@ -861,7 +841,6 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, static void multipoll_with_epoll_pollset_finish_shutdown(grpc_pollset *pollset); static void finish_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { - GPR_ASSERT(grpc_closure_list_empty(pollset->idle_jobs)); multipoll_with_epoll_pollset_finish_shutdown(pollset); grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); } @@ -895,26 +874,11 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_wakeup_fd_init(&worker.wakeup_fd->fd); } worker.kicked_specifically = 0; - /* If there's work waiting for the pollset to be idle, and the - pollset is idle, then do that work */ - if (!pollset_has_workers(pollset) && - !grpc_closure_list_empty(pollset->idle_jobs)) { - GPR_TIMER_MARK("pollset_work.idle_jobs", 0); - grpc_exec_ctx_enqueue_list(exec_ctx, &pollset->idle_jobs, NULL); - goto done; - } /* If we're shutting down then we don't execute any extended work */ if (pollset->shutting_down) { GPR_TIMER_MARK("pollset_work.shutting_down", 0); goto done; } - /* Give do_promote priority so we don't starve it out */ - if (pollset->in_flight_cbs) { - GPR_TIMER_MARK("pollset_work.in_flight_cbs", 0); - gpr_mu_unlock(&pollset->mu); - locked = 0; - goto done; - } /* Start polling, and keep doing so while we're being asked to re-evaluate our pollers (this allows poll() based pollers to ensure they don't miss wakeups) */ @@ -975,7 +939,7 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, if (pollset->shutting_down) { if (pollset_has_workers(pollset)) { pollset_kick(pollset, NULL); - } else if (!pollset->called_shutdown && pollset->in_flight_cbs == 0) { + } else if (!pollset->called_shutdown) { pollset->called_shutdown = 1; gpr_mu_unlock(&pollset->mu); finish_shutdown(exec_ctx, pollset); @@ -985,11 +949,6 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, * pollset_work. * TODO(dklempner): Can we refactor the shutdown logic to avoid this? */ gpr_mu_lock(&pollset->mu); - } else if (!grpc_closure_list_empty(pollset->idle_jobs)) { - grpc_exec_ctx_enqueue_list(exec_ctx, &pollset->idle_jobs, NULL); - gpr_mu_unlock(&pollset->mu); - grpc_exec_ctx_flush(exec_ctx); - gpr_mu_lock(&pollset->mu); } } *worker_hdl = NULL; @@ -1002,11 +961,8 @@ static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, pollset->shutting_down = 1; pollset->shutdown_done = closure; pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); - if (!pollset_has_workers(pollset)) { - grpc_exec_ctx_enqueue_list(exec_ctx, &pollset->idle_jobs, NULL); - } - if (!pollset->called_shutdown && pollset->in_flight_cbs == 0 && - !pollset_has_workers(pollset)) { + + if (!pollset->called_shutdown && !pollset_has_workers(pollset)) { pollset->called_shutdown = 1; finish_shutdown(exec_ctx, pollset); } From 97c2d6a269472a8a6e56d9e3d88f89bd27aff5ac Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Sat, 14 May 2016 16:33:16 -0700 Subject: [PATCH 015/280] Remove grpc_fd_watcher and related code from ev_epoll_posix.c --- src/core/lib/iomgr/ev_epoll_posix.c | 271 ++++------------------------ 1 file changed, 38 insertions(+), 233 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index 5776b6c1cf1..920be1951fa 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -59,17 +59,6 @@ * FD declarations */ -/* TODO(sreek) : Check if grpc_fd_watcher is needed (and if so, check if we can - * share this between ev_poll_posix.h and ev_epoll_posix versions */ - -typedef struct grpc_fd_watcher { - struct grpc_fd_watcher *next; - struct grpc_fd_watcher *prev; - grpc_pollset *pollset; - grpc_pollset_worker *worker; - grpc_fd *fd; -} grpc_fd_watcher; - struct grpc_fd { int fd; /* refst format: @@ -84,32 +73,6 @@ struct grpc_fd { int closed; int released; - /* The watcher list. - - The following watcher related fields are protected by watcher_mu. - - An fd_watcher is an ephemeral object created when an fd wants to - begin polling, and destroyed after the poll. - - It denotes the fd's interest in whether to read poll or write poll - or both or neither on this fd. - - If a watcher is asked to poll for reads or writes, the read_watcher - or write_watcher fields are set respectively. A watcher may be asked - to poll for both, in which case both fields will be set. - - read_watcher and write_watcher may be NULL if no watcher has been - asked to poll for reads or writes. - - If an fd_watcher is not asked to poll for reads or writes, it's added - to a linked list of inactive watchers, rooted at inactive_watcher_root. - If at a later time there becomes need of a poller to poll, one of - the inactive pollers may be kicked out of their poll loops to take - that responsibility. */ - grpc_fd_watcher inactive_watcher_root; - grpc_fd_watcher *read_watcher; - grpc_fd_watcher *write_watcher; - grpc_closure *read_closure; grpc_closure *write_closure; @@ -120,27 +83,6 @@ struct grpc_fd { grpc_iomgr_object iomgr_object; }; -/* Begin polling on an fd. - Registers that the given pollset is interested in this fd - so that if read - or writability interest changes, the pollset can be kicked to pick up that - new interest. - Return value is: - (fd_needs_read? read_mask : 0) | (fd_needs_write? write_mask : 0) - i.e. a combination of read_mask and write_mask determined by the fd's current - interest in said events. - Polling strategies that do not need to alter their behavior depending on the - fd's current interest (such as epoll) do not need to call this function. - MUST NOT be called with a pollset lock taken */ -static uint32_t fd_begin_poll(grpc_fd *fd, grpc_pollset *pollset, - grpc_pollset_worker *worker, uint32_t read_mask, - uint32_t write_mask, grpc_fd_watcher *rec); -/* Complete polling previously started with fd_begin_poll - MUST NOT be called with a pollset lock taken - if got_read or got_write are 1, also does the become_{readable,writable} as - appropriate. */ -static void fd_end_poll(grpc_exec_ctx *exec_ctx, grpc_fd_watcher *rec, - int got_read, int got_write); - /* Return 1 if this fd is orphaned, 0 otherwise */ static bool fd_is_orphaned(grpc_fd *fd); @@ -307,10 +249,7 @@ static grpc_fd *alloc_fd(int fd) { r->read_closure = CLOSURE_NOT_READY; r->write_closure = CLOSURE_NOT_READY; r->fd = fd; - r->inactive_watcher_root.next = r->inactive_watcher_root.prev = - &r->inactive_watcher_root; r->freelist_next = NULL; - r->read_watcher = r->write_watcher = NULL; r->on_done_closure = NULL; r->closed = 0; r->released = 0; @@ -387,43 +326,6 @@ static bool fd_is_orphaned(grpc_fd *fd) { return (gpr_atm_acq_load(&fd->refst) & 1) == 0; } -static void pollset_kick_locked(grpc_fd_watcher *watcher) { - gpr_mu_lock(&watcher->pollset->mu); - GPR_ASSERT(watcher->worker); - pollset_kick_ext(watcher->pollset, watcher->worker, - GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP); - gpr_mu_unlock(&watcher->pollset->mu); -} - -static void maybe_wake_one_watcher_locked(grpc_fd *fd) { - if (fd->inactive_watcher_root.next != &fd->inactive_watcher_root) { - pollset_kick_locked(fd->inactive_watcher_root.next); - } else if (fd->read_watcher) { - pollset_kick_locked(fd->read_watcher); - } else if (fd->write_watcher) { - pollset_kick_locked(fd->write_watcher); - } -} - -static void wake_all_watchers_locked(grpc_fd *fd) { - grpc_fd_watcher *watcher; - for (watcher = fd->inactive_watcher_root.next; - watcher != &fd->inactive_watcher_root; watcher = watcher->next) { - pollset_kick_locked(watcher); - } - if (fd->read_watcher) { - pollset_kick_locked(fd->read_watcher); - } - if (fd->write_watcher && fd->write_watcher != fd->read_watcher) { - pollset_kick_locked(fd->write_watcher); - } -} - -static int has_watchers(grpc_fd *fd) { - return fd->read_watcher != NULL || fd->write_watcher != NULL || - fd->inactive_watcher_root.next != &fd->inactive_watcher_root; -} - static void close_fd_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { fd->closed = 1; if (!fd->released) { @@ -454,11 +356,7 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, } gpr_mu_lock(&fd->mu); REF_BY(fd, 1, reason); /* remove active status, but keep referenced */ - if (!has_watchers(fd)) { - close_fd_locked(exec_ctx, fd); - } else { - wake_all_watchers_locked(fd); - } + close_fd_locked(exec_ctx, fd); gpr_mu_unlock(&fd->mu); UNREF_BY(fd, 2, reason); /* drop the reference */ } @@ -489,7 +387,6 @@ static void notify_on_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, /* already ready ==> queue the closure to run immediately */ *st = CLOSURE_NOT_READY; grpc_exec_ctx_enqueue(exec_ctx, closure, !fd->shutdown, NULL); - maybe_wake_one_watcher_locked(fd); } else { /* upcallptr was set to a different closure. This is an error! */ gpr_log(GPR_ERROR, @@ -540,111 +437,6 @@ static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, gpr_mu_unlock(&fd->mu); } -static uint32_t fd_begin_poll(grpc_fd *fd, grpc_pollset *pollset, - grpc_pollset_worker *worker, uint32_t read_mask, - uint32_t write_mask, grpc_fd_watcher *watcher) { - uint32_t mask = 0; - grpc_closure *cur; - int requested; - /* keep track of pollers that have requested our events, in case they change - */ - GRPC_FD_REF(fd, "poll"); - - gpr_mu_lock(&fd->mu); - - /* if we are shutdown, then don't add to the watcher set */ - if (fd->shutdown) { - watcher->fd = NULL; - watcher->pollset = NULL; - watcher->worker = NULL; - gpr_mu_unlock(&fd->mu); - GRPC_FD_UNREF(fd, "poll"); - return 0; - } - - /* if there is nobody polling for read, but we need to, then start doing so */ - cur = fd->read_closure; - requested = cur != CLOSURE_READY; - if (read_mask && fd->read_watcher == NULL && requested) { - fd->read_watcher = watcher; - mask |= read_mask; - } - /* if there is nobody polling for write, but we need to, then start doing so - */ - cur = fd->write_closure; - requested = cur != CLOSURE_READY; - if (write_mask && fd->write_watcher == NULL && requested) { - fd->write_watcher = watcher; - mask |= write_mask; - } - /* if not polling, remember this watcher in case we need someone to later */ - if (mask == 0 && worker != NULL) { - watcher->next = &fd->inactive_watcher_root; - watcher->prev = watcher->next->prev; - watcher->next->prev = watcher->prev->next = watcher; - } - watcher->pollset = pollset; - watcher->worker = worker; - watcher->fd = fd; - gpr_mu_unlock(&fd->mu); - - return mask; -} - -static void fd_end_poll(grpc_exec_ctx *exec_ctx, grpc_fd_watcher *watcher, - int got_read, int got_write) { - int was_polling = 0; - int kick = 0; - grpc_fd *fd = watcher->fd; - - if (fd == NULL) { - return; - } - - gpr_mu_lock(&fd->mu); - - if (watcher == fd->read_watcher) { - /* remove read watcher, kick if we still need a read */ - was_polling = 1; - if (!got_read) { - kick = 1; - } - fd->read_watcher = NULL; - } - if (watcher == fd->write_watcher) { - /* remove write watcher, kick if we still need a write */ - was_polling = 1; - if (!got_write) { - kick = 1; - } - fd->write_watcher = NULL; - } - if (!was_polling && watcher->worker != NULL) { - /* remove from inactive list */ - watcher->next->prev = watcher->prev; - watcher->prev->next = watcher->next; - } - if (got_read) { - if (set_ready_locked(exec_ctx, fd, &fd->read_closure)) { - kick = 1; - } - } - if (got_write) { - if (set_ready_locked(exec_ctx, fd, &fd->write_closure)) { - kick = 1; - } - } - if (kick) { - maybe_wake_one_watcher_locked(fd); - } - if (fd_is_orphaned(fd) && !has_watchers(fd) && !fd->closed) { - close_fd_locked(exec_ctx, fd); - } - gpr_mu_unlock(&fd->mu); - - GRPC_FD_UNREF(fd, "poll"); -} - /******************************************************************************* * pollset_posix.c */ @@ -1085,32 +877,45 @@ static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, epoll_hdr *h = pollset->data.ptr; struct epoll_event ev; int err; - grpc_fd_watcher watcher; - - /* We pretend to be polling whilst adding an fd to keep the fd from being - closed during the add. This may result in a spurious wakeup being assigned - to this pollset whilst adding, but that should be benign. */ - /* TODO (sreek). This fd_begin_poll() really seem to accomplish adding - * GRPC_FD_REF() (i.e adding a refcount to the fd) and checking that the - * fd is not shutting down (in which case watcher.fd will be NULL and no - * refcount is added). The ref count is added only durng hte duration of - * adding it to the epoll set (after which fd_end_poll would be called and - * the fd's ref count is decremented by 1. So do we still need fd_begin_poll - * ??? */ - GPR_ASSERT(fd_begin_poll(fd, pollset, NULL, 0, 0, &watcher) == 0); - if (watcher.fd != NULL) { - ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); - ev.data.ptr = fd; - err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); - if (err < 0) { - /* FDs may be added to a pollset multiple times, so EEXIST is normal. */ - if (errno != EEXIST) { - gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", fd->fd, - strerror(errno)); - } + + /* Hold a ref to the fd to keep it from being closed during the add. This may + result in a spurious wakeup being assigned to this pollset whilst adding, + but that should be benign. */ + /* TODO: (sreek): Understand how a spurious wake up migh be assinged to this + * pollset..and how holding a reference will prevent the fd from being closed + * (and perhaps more importantly, see how can an fd be closed while being + * added to the epollset */ + GRPC_FD_REF(fd, "add fd"); + + gpr_mu_lock(&fd->mu); + if (fd->shutdown) { + gpr_mu_unlock(&fd->mu); + GRPC_FD_UNREF(fd, "add fd"); + return; + } + gpr_mu_unlock(&fd->mu); + + ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); + ev.data.ptr = fd; + err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); + if (err < 0) { + /* FDs may be added to a pollset multiple times, so EEXIST is normal. */ + if (errno != EEXIST) { + gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", fd->fd, + strerror(errno)); } } - fd_end_poll(exec_ctx, &watcher, 0, 0); + + /* The fd might have been orphaned while we were adding it to the epoll set. + Close the fd in such a case (which will also take care of removing it from + the epoll set */ + gpr_mu_lock(&fd->mu); + if (fd_is_orphaned(fd) && !fd->closed) { + close_fd_locked(exec_ctx, fd); + } + gpr_mu_unlock(&fd->mu); + + GRPC_FD_UNREF(fd, "add fd"); } /* Creates an epoll fd and initializes the pollset */ From e9ee1f34b8efdc47ea12821351e5cc23125d62b2 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Sat, 14 May 2016 17:22:10 -0700 Subject: [PATCH 016/280] Minor refactor of add_fd path --- src/core/lib/iomgr/ev_epoll_posix.c | 24 +++++++----------------- 1 file changed, 7 insertions(+), 17 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index 920be1951fa..15126b3b62a 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -617,16 +617,13 @@ static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { + /* TODO (sreek) - Does reading pollset->data.ptr need pollset->mu lock ? + * because finally_add_fd() also reads it but without the lock! */ gpr_mu_lock(&pollset->mu); - multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fd); -/* the following (enabled only in debug) will reacquire and then release - our lock - meaning that if the unlocking flag passed to add_fd above is - not respected, the code will deadlock (in a way that we have a chance of - debugging) */ -#ifndef NDEBUG - gpr_mu_lock(&pollset->mu); + GPR_ASSERT(pollset->data.ptr != NULL); gpr_mu_unlock(&pollset->mu); -#endif + + multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fd); } /* TODO (sreek): Remove multipoll_with_epoll_finish_shutdown() declaration */ @@ -874,6 +871,8 @@ typedef struct { int epoll_fd; } epoll_hdr; static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { + /*TODO: (sree) Shouldn't this read (pollset->data.ptr) be done under a + pollset lock - i.e pollset->mu ? */ epoll_hdr *h = pollset->data.ptr; struct epoll_event ev; int err; @@ -941,9 +940,6 @@ static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { ev.events = (uint32_t)(EPOLLIN | EPOLLET); ev.data.ptr = NULL; - /* TODO (sreek): Double-check the use of grpc_global_wakeup_fd here (right now - * I do not know why this is used. I just copied this code from - * epoll_become_mutipoller() function in ev_poll_and_epoll_posix.c file */ err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); if (err < 0) { @@ -956,12 +952,6 @@ static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { - GPR_ASSERT(pollset->data.ptr != NULL); - - /* TODO(sreek). Remove this unlock code (and also the code that acquires the - * lock before calling multipoll_with_epoll_add_fd() function */ - - gpr_mu_unlock(&pollset->mu); finally_add_fd(exec_ctx, pollset, fd); } From c22eb5ac4dbf44209f0d431b6d1e9267210e0120 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Sat, 14 May 2016 19:22:36 -0700 Subject: [PATCH 017/280] Add epoll polling strategy to run_tests.py --- tools/run_tests/run_tests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 0538dce4198..ddaf96c345a 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -892,7 +892,7 @@ for l in languages: language_make_options=[] if any(language.make_options() for language in languages): - if not 'gcov' in args.config and len(languages) != 1: + if len(languages) != 1: print 'languages with custom make options cannot be built simultaneously with other languages' sys.exit(1) else: From 2ea165911b23099499da0af48088c47c47d659ba Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 17 May 2016 09:37:48 -0700 Subject: [PATCH 018/280] experiment with signals --- src/core/lib/iomgr/ev_epoll_posix.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index 15126b3b62a..4481bab4380 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include #include @@ -120,6 +121,7 @@ struct grpc_pollset_worker { grpc_cached_wakeup_fd *wakeup_fd; int reevaluate_polling_on_wakeup; int kicked_specifically; + pthread_t pt_id; struct grpc_pollset_worker *next; struct grpc_pollset_worker *prev; }; @@ -506,6 +508,8 @@ static void pollset_kick_ext(grpc_pollset *p, } specific_worker->kicked_specifically = 1; grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + /* TODO (sreek): Refactor this into a separate file*/ + pthread_kill(specific_worker->pt_id, SIGUSR1); } else if ((flags & GRPC_POLLSET_CAN_KICK_SELF) != 0) { GPR_TIMER_MARK("kick_yoself", 0); if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { @@ -551,10 +555,15 @@ static void pollset_kick(grpc_pollset *p, /* global state management */ +static void sig_handler(int sig_num) { + gpr_log(GPR_INFO, "Received signal %d", sig_num); +} + static void pollset_global_init(void) { gpr_tls_init(&g_current_thread_poller); gpr_tls_init(&g_current_thread_worker); grpc_wakeup_fd_init(&grpc_global_wakeup_fd); + signal(SIGUSR1, sig_handler); } static void pollset_global_shutdown(void) { @@ -663,6 +672,9 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_wakeup_fd_init(&worker.wakeup_fd->fd); } worker.kicked_specifically = 0; + + /* TODO(sreek): Abstract this thread id stuff out into a separate file */ + worker.pt_id = pthread_self(); /* If we're shutting down then we don't execute any extended work */ if (pollset->shutting_down) { GPR_TIMER_MARK("pollset_work.shutting_down", 0); From f448c34a6839f75476900a4a2b24b2160fe4d164 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 19 May 2016 10:51:24 -0700 Subject: [PATCH 019/280] Remove union { } data and epoll_hdr structures. Added ev_epoll_linux files --- BUILD | 6 + Makefile | 2 + binding.gyp | 1 + build.yaml | 2 + config.m4 | 1 + gRPC.podspec | 3 + grpc.gemspec | 2 + package.xml | 2 + src/core/lib/iomgr/ev_epoll_linux.c | 1335 +++++++++++++++++ src/core/lib/iomgr/ev_epoll_linux.h | 41 + src/core/lib/iomgr/ev_epoll_posix.c | 87 +- src/core/lib/iomgr/ev_posix.c | 7 +- src/python/grpcio/grpc_core_dependencies.py | 1 + tools/doxygen/Doxyfile.core.internal | 2 + tools/run_tests/sources_and_headers.json | 3 + vsprojects/vcxproj/grpc/grpc.vcxproj | 3 + vsprojects/vcxproj/grpc/grpc.vcxproj.filters | 6 + .../grpc_unsecure/grpc_unsecure.vcxproj | 3 + .../grpc_unsecure.vcxproj.filters | 6 + 19 files changed, 1442 insertions(+), 71 deletions(-) create mode 100644 src/core/lib/iomgr/ev_epoll_linux.c create mode 100644 src/core/lib/iomgr/ev_epoll_linux.h diff --git a/BUILD b/BUILD index 0be8f27a01b..a32352ebb30 100644 --- a/BUILD +++ b/BUILD @@ -178,6 +178,7 @@ cc_library( "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_linux.h", "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", @@ -322,6 +323,7 @@ cc_library( "src/core/lib/iomgr/endpoint.c", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_linux.c", "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", @@ -548,6 +550,7 @@ cc_library( "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_linux.h", "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", @@ -669,6 +672,7 @@ cc_library( "src/core/lib/iomgr/endpoint.c", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_linux.c", "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", @@ -1362,6 +1366,7 @@ objc_library( "src/core/lib/iomgr/endpoint.c", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_linux.c", "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", @@ -1567,6 +1572,7 @@ objc_library( "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_linux.h", "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", diff --git a/Makefile b/Makefile index 29ebc0e5adf..063698d943f 100644 --- a/Makefile +++ b/Makefile @@ -2486,6 +2486,7 @@ LIBGRPC_SRC = \ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ + src/core/lib/iomgr/ev_epoll_linux.c \ src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ @@ -2841,6 +2842,7 @@ LIBGRPC_UNSECURE_SRC = \ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ + src/core/lib/iomgr/ev_epoll_linux.c \ src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ diff --git a/binding.gyp b/binding.gyp index 89774ead4da..41e1b5bb416 100644 --- a/binding.gyp +++ b/binding.gyp @@ -581,6 +581,7 @@ 'src/core/lib/iomgr/endpoint.c', 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', + 'src/core/lib/iomgr/ev_epoll_linux.c', 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', diff --git a/build.yaml b/build.yaml index 7ba65332972..2f3d07071da 100644 --- a/build.yaml +++ b/build.yaml @@ -165,6 +165,7 @@ filegroups: - src/core/lib/iomgr/closure.h - src/core/lib/iomgr/endpoint.h - src/core/lib/iomgr/endpoint_pair.h + - src/core/lib/iomgr/ev_epoll_linux.h - src/core/lib/iomgr/ev_epoll_posix.h - src/core/lib/iomgr/ev_poll_posix.h - src/core/lib/iomgr/ev_posix.h @@ -240,6 +241,7 @@ filegroups: - src/core/lib/iomgr/endpoint.c - src/core/lib/iomgr/endpoint_pair_posix.c - src/core/lib/iomgr/endpoint_pair_windows.c + - src/core/lib/iomgr/ev_epoll_linux.c - src/core/lib/iomgr/ev_epoll_posix.c - src/core/lib/iomgr/ev_poll_posix.c - src/core/lib/iomgr/ev_posix.c diff --git a/config.m4 b/config.m4 index 6987c741541..4308295afda 100644 --- a/config.m4 +++ b/config.m4 @@ -100,6 +100,7 @@ if test "$PHP_GRPC" != "no"; then src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ + src/core/lib/iomgr/ev_epoll_linux.c \ src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ diff --git a/gRPC.podspec b/gRPC.podspec index 3b4dd52380e..de55880125a 100644 --- a/gRPC.podspec +++ b/gRPC.podspec @@ -181,6 +181,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/closure.h', 'src/core/lib/iomgr/endpoint.h', 'src/core/lib/iomgr/endpoint_pair.h', + 'src/core/lib/iomgr/ev_epoll_linux.h', 'src/core/lib/iomgr/ev_epoll_posix.h', 'src/core/lib/iomgr/ev_poll_posix.h', 'src/core/lib/iomgr/ev_posix.h', @@ -359,6 +360,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/endpoint.c', 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', + 'src/core/lib/iomgr/ev_epoll_linux.c', 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', @@ -548,6 +550,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/closure.h', 'src/core/lib/iomgr/endpoint.h', 'src/core/lib/iomgr/endpoint_pair.h', + 'src/core/lib/iomgr/ev_epoll_linux.h', 'src/core/lib/iomgr/ev_epoll_posix.h', 'src/core/lib/iomgr/ev_poll_posix.h', 'src/core/lib/iomgr/ev_posix.h', diff --git a/grpc.gemspec b/grpc.gemspec index 71cccb6ca8d..54ae2eb68dd 100755 --- a/grpc.gemspec +++ b/grpc.gemspec @@ -190,6 +190,7 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/closure.h ) s.files += %w( src/core/lib/iomgr/endpoint.h ) s.files += %w( src/core/lib/iomgr/endpoint_pair.h ) + s.files += %w( src/core/lib/iomgr/ev_epoll_linux.h ) s.files += %w( src/core/lib/iomgr/ev_epoll_posix.h ) s.files += %w( src/core/lib/iomgr/ev_poll_posix.h ) s.files += %w( src/core/lib/iomgr/ev_posix.h ) @@ -338,6 +339,7 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/endpoint.c ) s.files += %w( src/core/lib/iomgr/endpoint_pair_posix.c ) s.files += %w( src/core/lib/iomgr/endpoint_pair_windows.c ) + s.files += %w( src/core/lib/iomgr/ev_epoll_linux.c ) s.files += %w( src/core/lib/iomgr/ev_epoll_posix.c ) s.files += %w( src/core/lib/iomgr/ev_poll_posix.c ) s.files += %w( src/core/lib/iomgr/ev_posix.c ) diff --git a/package.xml b/package.xml index 0fc5d0dee47..d8e82a8bc37 100644 --- a/package.xml +++ b/package.xml @@ -197,6 +197,7 @@ + @@ -345,6 +346,7 @@ + diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c new file mode 100644 index 00000000000..f257ac8a1dd --- /dev/null +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -0,0 +1,1335 @@ +/* + * + * Copyright 2016, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include + +#ifdef GPR_POSIX_SOCKET + +#include "src/core/lib/iomgr/ev_epoll_posix.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "src/core/lib/iomgr/ev_posix.h" +#include "src/core/lib/iomgr/iomgr_internal.h" +#include "src/core/lib/iomgr/wakeup_fd_posix.h" +#include "src/core/lib/profiling/timers.h" +#include "src/core/lib/support/block_annotate.h" + +struct polling_island; + +/******************************************************************************* + * FD declarations + */ + +struct grpc_fd { + int fd; + /* refst format: + bit0: 1=active/0=orphaned + bit1-n: refcount + meaning that mostly we ref by two to avoid altering the orphaned bit, + and just unref by 1 when we're ready to flag the object as orphaned */ + gpr_atm refst; + + gpr_mu mu; + int shutdown; + int closed; + int released; + + grpc_closure *read_closure; + grpc_closure *write_closure; + + /* Mutex protecting the 'polling_island' field */ + gpr_mu pi_mu; + + /* The polling island to which this fd belongs to. An fd belongs to exactly + one polling island */ + struct polling_island *polling_island; + + struct grpc_fd *freelist_next; + + grpc_closure *on_done_closure; + + grpc_iomgr_object iomgr_object; +}; + +/* Return 1 if this fd is orphaned, 0 otherwise */ +static bool fd_is_orphaned(grpc_fd *fd); + +/* Reference counting for fds */ +/*#define GRPC_FD_REF_COUNT_DEBUG*/ +#ifdef GRPC_FD_REF_COUNT_DEBUG +static void fd_ref(grpc_fd *fd, const char *reason, const char *file, int line); +static void fd_unref(grpc_fd *fd, const char *reason, const char *file, + int line); +#define GRPC_FD_REF(fd, reason) fd_ref(fd, reason, __FILE__, __LINE__) +#define GRPC_FD_UNREF(fd, reason) fd_unref(fd, reason, __FILE__, __LINE__) +#else +static void fd_ref(grpc_fd *fd); +static void fd_unref(grpc_fd *fd); +#define GRPC_FD_REF(fd, reason) fd_ref(fd) +#define GRPC_FD_UNREF(fd, reason) fd_unref(fd) +#endif + +static void fd_global_init(void); +static void fd_global_shutdown(void); + +#define CLOSURE_NOT_READY ((grpc_closure *)0) +#define CLOSURE_READY ((grpc_closure *)1) + +/******************************************************************************* + * Polling Island + */ +typedef struct polling_island { + gpr_mu mu; + int ref_cnt; + + /* Pointer to the polling_island this merged into. If this is not NULL, all + the remaining fields in this pollset (i.e all fields except mu and ref_cnt) + are considered invalid and must be ignored */ + struct polling_island *merged_to; + + /* The fd of the underlying epoll set */ + int epoll_fd; + + /* The file descriptors in the epoll set */ + size_t fd_cnt; + size_t fd_capacity; + grpc_fd **fds; + + /* Polling islands that are no longer needed are kept in a freelist so that + they can be reused. This field points to the next polling island in the + free list. Note that this is only used if the polling island is in the + free list */ + struct polling_island *next_free; +} polling_island; + +/* Polling island freelist */ +static gpr_mu g_pi_freelist_mu; +static polling_island *g_pi_freelist = NULL; + +/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? */ +static void add_fd_to_polling_island_locked(polling_island *pi, grpc_fd *fd) { + int err; + struct epoll_event ev; + + ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); + ev.data.ptr = fd; + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); + + if (err < 0 && errno != EEXIST) { + gpr_log(GPR_ERROR, "epoll_ctl add for fd: %d failed with error: %s", fd->fd, + strerror(errno)); + return; + } + + pi->fd_capacity = GPR_MAX(pi->fd_capacity + 8, pi->fd_cnt * 3 / 2); + pi->fds = gpr_realloc(pi->fds, sizeof(grpc_fd *) * pi->fd_capacity); + pi->fds[pi->fd_cnt++] = fd; +} + +static polling_island *polling_island_create(int initial_ref_cnt, + grpc_fd *initial_fd) { + polling_island *pi = NULL; + gpr_mu_lock(&g_pi_freelist_mu); + if (g_pi_freelist != NULL) { + pi = g_pi_freelist; + g_pi_freelist = g_pi_freelist->next_free; + pi->next_free = NULL; + } + gpr_mu_unlock(&g_pi_freelist_mu); + + /* Create new polling island if we could not get one from the free list */ + if (pi == NULL) { + pi = gpr_malloc(sizeof(*pi)); + gpr_mu_init(&pi->mu); + pi->fd_cnt = 0; + pi->fd_capacity = 0; + pi->fds = NULL; + + pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); + if (pi->epoll_fd < 0) { + gpr_log(GPR_ERROR, "epoll_create1() failed with error: %s", + strerror(errno)); + } + GPR_ASSERT(pi->epoll_fd >= 0); + } + + pi->ref_cnt = initial_ref_cnt; + pi->merged_to = NULL; + pi->next_free = NULL; + + if (initial_fd != NULL) { + /* add_fd_to_polling_island_locked() expects the caller to hold a pi->mu + * lock. However, since this is a new polling island (and no one has a + * reference to it yet), it is okay to not acquire pi->mu here */ + add_fd_to_polling_island_locked(pi, initial_fd); + } + + return pi; +} + +static void polling_island_global_init() { + polling_island_create(0, NULL); /* TODO(sreek): Delete this line */ + gpr_mu_init(&g_pi_freelist_mu); + g_pi_freelist = NULL; +} + +/******************************************************************************* + * pollset declarations + */ + +typedef struct grpc_cached_wakeup_fd { + grpc_wakeup_fd fd; + struct grpc_cached_wakeup_fd *next; +} grpc_cached_wakeup_fd; + +struct grpc_pollset_worker { + grpc_cached_wakeup_fd *wakeup_fd; + int reevaluate_polling_on_wakeup; + int kicked_specifically; + pthread_t pt_id; + struct grpc_pollset_worker *next; + struct grpc_pollset_worker *prev; +}; + +struct grpc_pollset { + gpr_mu mu; + grpc_pollset_worker root_worker; + int shutting_down; + int called_shutdown; + int kicked_without_pollers; + grpc_closure *shutdown_done; + + int epoll_fd; + + /* Mutex protecting the 'polling_island' field */ + gpr_mu pi_mu; + + /* The polling island to which this fd belongs to. An fd belongs to exactly + one polling island */ + struct polling_island *polling_island; + + /* Local cache of eventfds for workers */ + grpc_cached_wakeup_fd *local_wakeup_cache; +}; + +/* Add an fd to a pollset */ +static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + struct grpc_fd *fd); + +static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd); + +/* Convert a timespec to milliseconds: + - very small or negative poll times are clamped to zero to do a + non-blocking poll (which becomes spin polling) + - other small values are rounded up to one millisecond + - longer than a millisecond polls are rounded up to the next nearest + millisecond to avoid spinning + - infinite timeouts are converted to -1 */ +static int poll_deadline_to_millis_timeout(gpr_timespec deadline, + gpr_timespec now); + +/* Allow kick to wakeup the currently polling worker */ +#define GRPC_POLLSET_CAN_KICK_SELF 1 +/* Force the wakee to repoll when awoken */ +#define GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP 2 +/* As per pollset_kick, with an extended set of flags (defined above) + -- mostly for fd_posix's use. */ +static void pollset_kick_ext(grpc_pollset *p, + grpc_pollset_worker *specific_worker, + uint32_t flags); + +/* turn a pollset into a multipoller: platform specific */ +typedef void (*platform_become_multipoller_type)(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, + struct grpc_fd **fds, + size_t fd_count); + +/* Return 1 if the pollset has active threads in pollset_work (pollset must + * be locked) */ +static int pollset_has_workers(grpc_pollset *pollset); + +static void remove_fd_from_all_epoll_sets(int fd); + +/******************************************************************************* + * pollset_set definitions + */ + +struct grpc_pollset_set { + gpr_mu mu; + + size_t pollset_count; + size_t pollset_capacity; + grpc_pollset **pollsets; + + size_t pollset_set_count; + size_t pollset_set_capacity; + struct grpc_pollset_set **pollset_sets; + + size_t fd_count; + size_t fd_capacity; + grpc_fd **fds; +}; + +/******************************************************************************* + * fd_posix.c + */ + +/* We need to keep a freelist not because of any concerns of malloc performance + * but instead so that implementations with multiple threads in (for example) + * epoll_wait deal with the race between pollset removal and incoming poll + * notifications. + * + * The problem is that the poller ultimately holds a reference to this + * object, so it is very difficult to know when is safe to free it, at least + * without some expensive synchronization. + * + * If we keep the object freelisted, in the worst case losing this race just + * becomes a spurious read notification on a reused fd. + */ +/* TODO(klempner): We could use some form of polling generation count to know + * when these are safe to free. */ +/* TODO(klempner): Consider disabling freelisting if we don't have multiple + * threads in poll on the same fd */ +/* TODO(klempner): Batch these allocations to reduce fragmentation */ +static grpc_fd *fd_freelist = NULL; +static gpr_mu fd_freelist_mu; + +static void freelist_fd(grpc_fd *fd) { + gpr_mu_lock(&fd_freelist_mu); + fd->freelist_next = fd_freelist; + fd_freelist = fd; + grpc_iomgr_unregister_object(&fd->iomgr_object); + gpr_mu_unlock(&fd_freelist_mu); +} + +static grpc_fd *alloc_fd(int fd) { + grpc_fd *r = NULL; + + gpr_mu_lock(&fd_freelist_mu); + if (fd_freelist != NULL) { + r = fd_freelist; + fd_freelist = fd_freelist->freelist_next; + } + gpr_mu_unlock(&fd_freelist_mu); + + if (r == NULL) { + r = gpr_malloc(sizeof(grpc_fd)); + gpr_mu_init(&r->mu); + gpr_mu_init(&r->pi_mu); + } + + /* TODO: sreek - check with ctiller on why we need to acquire a lock here */ + gpr_mu_lock(&r->mu); + gpr_atm_rel_store(&r->refst, 1); + r->shutdown = 0; + r->read_closure = CLOSURE_NOT_READY; + r->write_closure = CLOSURE_NOT_READY; + r->fd = fd; + r->polling_island = NULL; + r->freelist_next = NULL; + r->on_done_closure = NULL; + r->closed = 0; + r->released = 0; + gpr_mu_unlock(&r->mu); + return r; +} + +static void destroy(grpc_fd *fd) { + gpr_mu_destroy(&fd->mu); + gpr_free(fd); +} + +#ifdef GRPC_FD_REF_COUNT_DEBUG +#define REF_BY(fd, n, reason) ref_by(fd, n, reason, __FILE__, __LINE__) +#define UNREF_BY(fd, n, reason) unref_by(fd, n, reason, __FILE__, __LINE__) +static void ref_by(grpc_fd *fd, int n, const char *reason, const char *file, + int line) { + gpr_log(GPR_DEBUG, "FD %d %p ref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, + gpr_atm_no_barrier_load(&fd->refst), + gpr_atm_no_barrier_load(&fd->refst) + n, reason, file, line); +#else +#define REF_BY(fd, n, reason) ref_by(fd, n) +#define UNREF_BY(fd, n, reason) unref_by(fd, n) +static void ref_by(grpc_fd *fd, int n) { +#endif + GPR_ASSERT(gpr_atm_no_barrier_fetch_add(&fd->refst, n) > 0); +} + +#ifdef GRPC_FD_REF_COUNT_DEBUG +static void unref_by(grpc_fd *fd, int n, const char *reason, const char *file, + int line) { + gpr_atm old; + gpr_log(GPR_DEBUG, "FD %d %p unref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, + gpr_atm_no_barrier_load(&fd->refst), + gpr_atm_no_barrier_load(&fd->refst) - n, reason, file, line); +#else +static void unref_by(grpc_fd *fd, int n) { + gpr_atm old; +#endif + old = gpr_atm_full_fetch_add(&fd->refst, -n); + if (old == n) { + freelist_fd(fd); + } else { + GPR_ASSERT(old > n); + } +} + +static void fd_global_init(void) { gpr_mu_init(&fd_freelist_mu); } + +static void fd_global_shutdown(void) { + gpr_mu_lock(&fd_freelist_mu); + gpr_mu_unlock(&fd_freelist_mu); + while (fd_freelist != NULL) { + grpc_fd *fd = fd_freelist; + fd_freelist = fd_freelist->freelist_next; + destroy(fd); + } + gpr_mu_destroy(&fd_freelist_mu); +} + +static grpc_fd *fd_create(int fd, const char *name) { + grpc_fd *r = alloc_fd(fd); + char *name2; + gpr_asprintf(&name2, "%s fd=%d", name, fd); + grpc_iomgr_register_object(&r->iomgr_object, name2); + gpr_free(name2); +#ifdef GRPC_FD_REF_COUNT_DEBUG + gpr_log(GPR_DEBUG, "FD %d %p create %s", fd, r, name); +#endif + return r; +} + +static bool fd_is_orphaned(grpc_fd *fd) { + return (gpr_atm_acq_load(&fd->refst) & 1) == 0; +} + +static void close_fd_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + fd->closed = 1; + if (!fd->released) { + close(fd->fd); + } else { + remove_fd_from_all_epoll_sets(fd->fd); + } + grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); +} + +static int fd_wrapped_fd(grpc_fd *fd) { + if (fd->released || fd->closed) { + return -1; + } else { + return fd->fd; + } +} + +/* TODO: sreek - do something here with the pollset island link */ +static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *on_done, int *release_fd, + const char *reason) { + fd->on_done_closure = on_done; + fd->released = release_fd != NULL; + if (!fd->released) { + shutdown(fd->fd, SHUT_RDWR); + } else { + *release_fd = fd->fd; + } + gpr_mu_lock(&fd->mu); + REF_BY(fd, 1, reason); /* remove active status, but keep referenced */ + close_fd_locked(exec_ctx, fd); + gpr_mu_unlock(&fd->mu); + UNREF_BY(fd, 2, reason); /* drop the reference */ +} + +/* increment refcount by two to avoid changing the orphan bit */ +#ifdef GRPC_FD_REF_COUNT_DEBUG +static void fd_ref(grpc_fd *fd, const char *reason, const char *file, + int line) { + ref_by(fd, 2, reason, file, line); +} + +static void fd_unref(grpc_fd *fd, const char *reason, const char *file, + int line) { + unref_by(fd, 2, reason, file, line); +} +#else +static void fd_ref(grpc_fd *fd) { ref_by(fd, 2); } + +static void fd_unref(grpc_fd *fd) { unref_by(fd, 2); } +#endif + +static void notify_on_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure **st, grpc_closure *closure) { + if (*st == CLOSURE_NOT_READY) { + /* not ready ==> switch to a waiting state by setting the closure */ + *st = closure; + } else if (*st == CLOSURE_READY) { + /* already ready ==> queue the closure to run immediately */ + *st = CLOSURE_NOT_READY; + grpc_exec_ctx_enqueue(exec_ctx, closure, !fd->shutdown, NULL); + } else { + /* upcallptr was set to a different closure. This is an error! */ + gpr_log(GPR_ERROR, + "User called a notify_on function with a previous callback still " + "pending"); + abort(); + } +} + +/* returns 1 if state becomes not ready */ +static int set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure **st) { + if (*st == CLOSURE_READY) { + /* duplicate ready ==> ignore */ + return 0; + } else if (*st == CLOSURE_NOT_READY) { + /* not ready, and not waiting ==> flag ready */ + *st = CLOSURE_READY; + return 0; + } else { + /* waiting ==> queue closure */ + grpc_exec_ctx_enqueue(exec_ctx, *st, !fd->shutdown, NULL); + *st = CLOSURE_NOT_READY; + return 1; + } +} + +/* Do something here with the pollset island link (?) */ +static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + gpr_mu_lock(&fd->mu); + GPR_ASSERT(!fd->shutdown); + fd->shutdown = 1; + set_ready_locked(exec_ctx, fd, &fd->read_closure); + set_ready_locked(exec_ctx, fd, &fd->write_closure); + gpr_mu_unlock(&fd->mu); +} + +static void fd_notify_on_read(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *closure) { + gpr_mu_lock(&fd->mu); + notify_on_locked(exec_ctx, fd, &fd->read_closure, closure); + gpr_mu_unlock(&fd->mu); +} + +static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *closure) { + gpr_mu_lock(&fd->mu); + notify_on_locked(exec_ctx, fd, &fd->write_closure, closure); + gpr_mu_unlock(&fd->mu); +} + +/******************************************************************************* + * pollset_posix.c + */ + +GPR_TLS_DECL(g_current_thread_poller); +GPR_TLS_DECL(g_current_thread_worker); + +/** The alarm system needs to be able to wakeup 'some poller' sometimes + * (specifically when a new alarm needs to be triggered earlier than the next + * alarm 'epoch'). + * This wakeup_fd gives us something to alert on when such a case occurs. */ +grpc_wakeup_fd grpc_global_wakeup_fd; + +static void remove_worker(grpc_pollset *p, grpc_pollset_worker *worker) { + worker->prev->next = worker->next; + worker->next->prev = worker->prev; +} + +static int pollset_has_workers(grpc_pollset *p) { + return p->root_worker.next != &p->root_worker; +} + +static grpc_pollset_worker *pop_front_worker(grpc_pollset *p) { + if (pollset_has_workers(p)) { + grpc_pollset_worker *w = p->root_worker.next; + remove_worker(p, w); + return w; + } else { + return NULL; + } +} + +static void push_back_worker(grpc_pollset *p, grpc_pollset_worker *worker) { + worker->next = &p->root_worker; + worker->prev = worker->next->prev; + worker->prev->next = worker->next->prev = worker; +} + +static void push_front_worker(grpc_pollset *p, grpc_pollset_worker *worker) { + worker->prev = &p->root_worker; + worker->next = worker->prev->next; + worker->prev->next = worker->next->prev = worker; +} + +static void pollset_kick_ext(grpc_pollset *p, + grpc_pollset_worker *specific_worker, + uint32_t flags) { + GPR_TIMER_BEGIN("pollset_kick_ext", 0); + + /* pollset->mu already held */ + if (specific_worker != NULL) { + if (specific_worker == GRPC_POLLSET_KICK_BROADCAST) { + GPR_TIMER_BEGIN("pollset_kick_ext.broadcast", 0); + GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); + for (specific_worker = p->root_worker.next; + specific_worker != &p->root_worker; + specific_worker = specific_worker->next) { + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + } + p->kicked_without_pollers = 1; + GPR_TIMER_END("pollset_kick_ext.broadcast", 0); + } else if (gpr_tls_get(&g_current_thread_worker) != + (intptr_t)specific_worker) { + GPR_TIMER_MARK("different_thread_worker", 0); + if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { + specific_worker->reevaluate_polling_on_wakeup = 1; + } + specific_worker->kicked_specifically = 1; + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + /* TODO (sreek): Refactor this into a separate file*/ + pthread_kill(specific_worker->pt_id, SIGUSR1); + } else if ((flags & GRPC_POLLSET_CAN_KICK_SELF) != 0) { + GPR_TIMER_MARK("kick_yoself", 0); + if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { + specific_worker->reevaluate_polling_on_wakeup = 1; + } + specific_worker->kicked_specifically = 1; + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + } + } else if (gpr_tls_get(&g_current_thread_poller) != (intptr_t)p) { + GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); + GPR_TIMER_MARK("kick_anonymous", 0); + specific_worker = pop_front_worker(p); + if (specific_worker != NULL) { + if (gpr_tls_get(&g_current_thread_worker) == (intptr_t)specific_worker) { + GPR_TIMER_MARK("kick_anonymous_not_self", 0); + push_back_worker(p, specific_worker); + specific_worker = pop_front_worker(p); + if ((flags & GRPC_POLLSET_CAN_KICK_SELF) == 0 && + gpr_tls_get(&g_current_thread_worker) == + (intptr_t)specific_worker) { + push_back_worker(p, specific_worker); + specific_worker = NULL; + } + } + if (specific_worker != NULL) { + GPR_TIMER_MARK("finally_kick", 0); + push_back_worker(p, specific_worker); + grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + } + } else { + GPR_TIMER_MARK("kicked_no_pollers", 0); + p->kicked_without_pollers = 1; + } + } + + GPR_TIMER_END("pollset_kick_ext", 0); +} + +static void pollset_kick(grpc_pollset *p, + grpc_pollset_worker *specific_worker) { + pollset_kick_ext(p, specific_worker, 0); +} + +/* global state management */ + +static void sig_handler(int sig_num) { + gpr_log(GPR_INFO, "Received signal %d", sig_num); +} + +static void pollset_global_init(void) { + gpr_tls_init(&g_current_thread_poller); + gpr_tls_init(&g_current_thread_worker); + grpc_wakeup_fd_init(&grpc_global_wakeup_fd); + signal(SIGUSR1, sig_handler); +} + +static void pollset_global_shutdown(void) { + grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); + gpr_tls_destroy(&g_current_thread_poller); + gpr_tls_destroy(&g_current_thread_worker); +} + +static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } + +/* TODO: sreek. Try to Remove this forward declaration*/ +static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset); + +/* main interface */ + +static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { + gpr_mu_init(&pollset->mu); + *mu = &pollset->mu; + pollset->root_worker.next = pollset->root_worker.prev = &pollset->root_worker; + gpr_mu_init(&pollset->pi_mu); + pollset->polling_island = NULL; + pollset->shutting_down = 0; + pollset->called_shutdown = 0; + pollset->kicked_without_pollers = 0; + pollset->local_wakeup_cache = NULL; + pollset->kicked_without_pollers = 0; + + multipoll_with_epoll_pollset_create_efd(pollset); +} + +/* TODO(sreek): Maybe merge multipoll_*_destroy() with pollset_destroy() + * function */ +static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset); + +static void pollset_destroy(grpc_pollset *pollset) { + GPR_ASSERT(!pollset_has_workers(pollset)); + + multipoll_with_epoll_pollset_destroy(pollset); + + while (pollset->local_wakeup_cache) { + grpc_cached_wakeup_fd *next = pollset->local_wakeup_cache->next; + grpc_wakeup_fd_destroy(&pollset->local_wakeup_cache->fd); + gpr_free(pollset->local_wakeup_cache); + pollset->local_wakeup_cache = next; + } + gpr_mu_destroy(&pollset->pi_mu); + gpr_mu_destroy(&pollset->mu); +} + +/* TODO(sreek) - Do something with the pollset island link (??) */ +static void pollset_reset(grpc_pollset *pollset) { + GPR_ASSERT(pollset->shutting_down); + GPR_ASSERT(!pollset_has_workers(pollset)); + pollset->shutting_down = 0; + pollset->called_shutdown = 0; + pollset->kicked_without_pollers = 0; +} + +/* TODO (sreek): Remove multipoll_with_epoll_finish_shutdown() declaration */ +static void multipoll_with_epoll_pollset_finish_shutdown(grpc_pollset *pollset); + +static void finish_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { + multipoll_with_epoll_pollset_finish_shutdown(pollset); + grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); +} + +/* TODO(sreek): Remove multipoll_with_epoll_*_maybe_work_and_unlock declaration + */ +static void multipoll_with_epoll_pollset_maybe_work_and_unlock( + grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, + gpr_timespec deadline, gpr_timespec now); + +static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_pollset_worker **worker_hdl, gpr_timespec now, + gpr_timespec deadline) { + grpc_pollset_worker worker; + *worker_hdl = &worker; + + /* pollset->mu already held */ + int added_worker = 0; + int locked = 1; + int queued_work = 0; + int keep_polling = 0; + GPR_TIMER_BEGIN("pollset_work", 0); + /* this must happen before we (potentially) drop pollset->mu */ + worker.next = worker.prev = NULL; + worker.reevaluate_polling_on_wakeup = 0; + if (pollset->local_wakeup_cache != NULL) { + worker.wakeup_fd = pollset->local_wakeup_cache; + pollset->local_wakeup_cache = worker.wakeup_fd->next; + } else { + worker.wakeup_fd = gpr_malloc(sizeof(*worker.wakeup_fd)); + grpc_wakeup_fd_init(&worker.wakeup_fd->fd); + } + worker.kicked_specifically = 0; + + /* TODO(sreek): Abstract this thread id stuff out into a separate file */ + worker.pt_id = pthread_self(); + /* If we're shutting down then we don't execute any extended work */ + if (pollset->shutting_down) { + GPR_TIMER_MARK("pollset_work.shutting_down", 0); + goto done; + } + /* Start polling, and keep doing so while we're being asked to + re-evaluate our pollers (this allows poll() based pollers to + ensure they don't miss wakeups) */ + keep_polling = 1; + while (keep_polling) { + keep_polling = 0; + if (!pollset->kicked_without_pollers) { + if (!added_worker) { + push_front_worker(pollset, &worker); + added_worker = 1; + gpr_tls_set(&g_current_thread_worker, (intptr_t)&worker); + } + gpr_tls_set(&g_current_thread_poller, (intptr_t)pollset); + GPR_TIMER_BEGIN("maybe_work_and_unlock", 0); + + multipoll_with_epoll_pollset_maybe_work_and_unlock( + exec_ctx, pollset, &worker, deadline, now); + + GPR_TIMER_END("maybe_work_and_unlock", 0); + locked = 0; + gpr_tls_set(&g_current_thread_poller, 0); + } else { + GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); + pollset->kicked_without_pollers = 0; + } + /* Finished execution - start cleaning up. + Note that we may arrive here from outside the enclosing while() loop. + In that case we won't loop though as we haven't added worker to the + worker list, which means nobody could ask us to re-evaluate polling). */ + done: + if (!locked) { + queued_work |= grpc_exec_ctx_flush(exec_ctx); + gpr_mu_lock(&pollset->mu); + locked = 1; + } + /* If we're forced to re-evaluate polling (via pollset_kick with + GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) then we land here and force + a loop */ + if (worker.reevaluate_polling_on_wakeup) { + worker.reevaluate_polling_on_wakeup = 0; + pollset->kicked_without_pollers = 0; + if (queued_work || worker.kicked_specifically) { + /* If there's queued work on the list, then set the deadline to be + immediate so we get back out of the polling loop quickly */ + deadline = gpr_inf_past(GPR_CLOCK_MONOTONIC); + } + keep_polling = 1; + } + } + if (added_worker) { + remove_worker(pollset, &worker); + gpr_tls_set(&g_current_thread_worker, 0); + } + /* release wakeup fd to the local pool */ + worker.wakeup_fd->next = pollset->local_wakeup_cache; + pollset->local_wakeup_cache = worker.wakeup_fd; + /* check shutdown conditions */ + if (pollset->shutting_down) { + if (pollset_has_workers(pollset)) { + pollset_kick(pollset, NULL); + } else if (!pollset->called_shutdown) { + pollset->called_shutdown = 1; + gpr_mu_unlock(&pollset->mu); + finish_shutdown(exec_ctx, pollset); + grpc_exec_ctx_flush(exec_ctx); + /* Continuing to access pollset here is safe -- it is the caller's + * responsibility to not destroy when it has outstanding calls to + * pollset_work. + * TODO(dklempner): Can we refactor the shutdown logic to avoid this? */ + gpr_mu_lock(&pollset->mu); + } + } + *worker_hdl = NULL; + GPR_TIMER_END("pollset_work", 0); +} + +/* TODO: (sreek) Do something with the pollset island link */ +static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_closure *closure) { + GPR_ASSERT(!pollset->shutting_down); + pollset->shutting_down = 1; + pollset->shutdown_done = closure; + pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); + + if (!pollset->called_shutdown && !pollset_has_workers(pollset)) { + pollset->called_shutdown = 1; + finish_shutdown(exec_ctx, pollset); + } +} + +static int poll_deadline_to_millis_timeout(gpr_timespec deadline, + gpr_timespec now) { + gpr_timespec timeout; + static const int64_t max_spin_polling_us = 10; + if (gpr_time_cmp(deadline, gpr_inf_future(deadline.clock_type)) == 0) { + return -1; + } + if (gpr_time_cmp(deadline, gpr_time_add(now, gpr_time_from_micros( + max_spin_polling_us, + GPR_TIMESPAN))) <= 0) { + return 0; + } + timeout = gpr_time_sub(deadline, now); + return gpr_time_to_millis(gpr_time_add( + timeout, gpr_time_from_nanos(GPR_NS_PER_MS - 1, GPR_TIMESPAN))); +} + +/******************************************************************************* + * pollset_multipoller_with_epoll_posix.c + */ + +static void set_ready(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st) { + /* only one set_ready can be active at once (but there may be a racing + notify_on) */ + gpr_mu_lock(&fd->mu); + set_ready_locked(exec_ctx, fd, st); + gpr_mu_unlock(&fd->mu); +} + +static void fd_become_readable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + set_ready(exec_ctx, fd, &fd->read_closure); +} + +static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + set_ready(exec_ctx, fd, &fd->write_closure); +} + +/* TODO (sreek): Maybe this global list is not required. Double check*/ +struct epoll_fd_list { + int *epoll_fds; + size_t count; + size_t capacity; +}; + +static struct epoll_fd_list epoll_fd_global_list; +static gpr_once init_epoll_fd_list_mu = GPR_ONCE_INIT; +static gpr_mu epoll_fd_list_mu; + +static void init_mu(void) { gpr_mu_init(&epoll_fd_list_mu); } + +static void add_epoll_fd_to_global_list(int epoll_fd) { + gpr_once_init(&init_epoll_fd_list_mu, init_mu); + + gpr_mu_lock(&epoll_fd_list_mu); + if (epoll_fd_global_list.count == epoll_fd_global_list.capacity) { + epoll_fd_global_list.capacity = + GPR_MAX((size_t)8, epoll_fd_global_list.capacity * 2); + epoll_fd_global_list.epoll_fds = + gpr_realloc(epoll_fd_global_list.epoll_fds, + epoll_fd_global_list.capacity * sizeof(int)); + } + epoll_fd_global_list.epoll_fds[epoll_fd_global_list.count++] = epoll_fd; + gpr_mu_unlock(&epoll_fd_list_mu); +} + +static void remove_epoll_fd_from_global_list(int epoll_fd) { + gpr_mu_lock(&epoll_fd_list_mu); + GPR_ASSERT(epoll_fd_global_list.count > 0); + for (size_t i = 0; i < epoll_fd_global_list.count; i++) { + if (epoll_fd == epoll_fd_global_list.epoll_fds[i]) { + epoll_fd_global_list.epoll_fds[i] = + epoll_fd_global_list.epoll_fds[--(epoll_fd_global_list.count)]; + break; + } + } + gpr_mu_unlock(&epoll_fd_list_mu); +} + +static void remove_fd_from_all_epoll_sets(int fd) { + int err; + gpr_once_init(&init_epoll_fd_list_mu, init_mu); + gpr_mu_lock(&epoll_fd_list_mu); + if (epoll_fd_global_list.count == 0) { + gpr_mu_unlock(&epoll_fd_list_mu); + return; + } + for (size_t i = 0; i < epoll_fd_global_list.count; i++) { + err = epoll_ctl(epoll_fd_global_list.epoll_fds[i], EPOLL_CTL_DEL, fd, NULL); + if (err < 0 && errno != ENOENT) { + gpr_log(GPR_ERROR, "epoll_ctl del for %d failed: %s", fd, + strerror(errno)); + } + } + gpr_mu_unlock(&epoll_fd_list_mu); +} + +/* TODO: sreek - This function multipoll_with_epoll_pollset_add_fd() and + * finally_add_fd() in ev_poll_and_epoll_posix.c */ +static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_fd *fd) { + + /* TODO sreek - Check if we need to get a pollset->mu lock here */ + + struct epoll_event ev; + int err; + + /* Hold a ref to the fd to keep it from being closed during the add. This may + result in a spurious wakeup being assigned to this pollset whilst adding, + but that should be benign. */ + /* TODO: (sreek): Understand how a spurious wake up migh be assinged to this + * pollset..and how holding a reference will prevent the fd from being closed + * (and perhaps more importantly, see how can an fd be closed while being + * added to the epollset */ + GRPC_FD_REF(fd, "add fd"); + + gpr_mu_lock(&fd->mu); + if (fd->shutdown) { + gpr_mu_unlock(&fd->mu); + GRPC_FD_UNREF(fd, "add fd"); + return; + } + gpr_mu_unlock(&fd->mu); + + ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); + ev.data.ptr = fd; + err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); + if (err < 0) { + /* FDs may be added to a pollset multiple times, so EEXIST is normal. */ + if (errno != EEXIST) { + gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", fd->fd, + strerror(errno)); + } + } + + /* The fd might have been orphaned while we were adding it to the epoll set. + Close the fd in such a case (which will also take care of removing it from + the epoll set */ + gpr_mu_lock(&fd->mu); + if (fd_is_orphaned(fd) && !fd->closed) { + close_fd_locked(exec_ctx, fd); + } + gpr_mu_unlock(&fd->mu); + + GRPC_FD_UNREF(fd, "add fd"); +} + +/* Creates an epoll fd and initializes the pollset */ +/* TODO: This has to be called ONLY from pollset_init function. and hence it + * does not acquire any lock */ +static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { + struct epoll_event ev; + int err; + + pollset->epoll_fd = epoll_create1(EPOLL_CLOEXEC); + if (pollset->epoll_fd < 0) { + gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); + abort(); + } + add_epoll_fd_to_global_list(pollset->epoll_fd); + + ev.events = (uint32_t)(EPOLLIN | EPOLLET); + ev.data.ptr = NULL; + + err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); + if (err < 0) { + gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), + strerror(errno)); + } +} + +/* TODO(klempner): We probably want to turn this down a bit */ +#define GRPC_EPOLL_MAX_EVENTS 1000 + +static void multipoll_with_epoll_pollset_maybe_work_and_unlock( + grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, + gpr_timespec deadline, gpr_timespec now) { + struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; + int epoll_fd = pollset->epoll_fd; + int ep_rv; + int poll_rv; + int timeout_ms; + struct pollfd pfds[2]; + + /* If you want to ignore epoll's ability to sanely handle parallel pollers, + * for a more apples-to-apples performance comparison with poll, add a + * if (pollset->counter != 0) { return 0; } + * here. + */ + + gpr_mu_unlock(&pollset->mu); + + timeout_ms = poll_deadline_to_millis_timeout(deadline, now); + + pfds[0].fd = GRPC_WAKEUP_FD_GET_READ_FD(&worker->wakeup_fd->fd); + pfds[0].events = POLLIN; + pfds[0].revents = 0; + pfds[1].fd = epoll_fd; + pfds[1].events = POLLIN; + pfds[1].revents = 0; + + /* TODO(vpai): Consider first doing a 0 timeout poll here to avoid + even going into the blocking annotation if possible */ + GPR_TIMER_BEGIN("poll", 0); + GRPC_SCHEDULING_START_BLOCKING_REGION; + poll_rv = grpc_poll_function(pfds, 2, timeout_ms); + GRPC_SCHEDULING_END_BLOCKING_REGION; + GPR_TIMER_END("poll", 0); + + if (poll_rv < 0) { + if (errno != EINTR) { + gpr_log(GPR_ERROR, "poll() failed: %s", strerror(errno)); + } + } else if (poll_rv == 0) { + /* do nothing */ + } else { + if (pfds[0].revents) { + grpc_wakeup_fd_consume_wakeup(&worker->wakeup_fd->fd); + } + if (pfds[1].revents) { + do { + /* The following epoll_wait never blocks; it has a timeout of 0 */ + ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); + if (ep_rv < 0) { + if (errno != EINTR) { + gpr_log(GPR_ERROR, "epoll_wait() failed: %s", strerror(errno)); + } + } else { + int i; + for (i = 0; i < ep_rv; ++i) { + grpc_fd *fd = ep_ev[i].data.ptr; + /* TODO(klempner): We might want to consider making err and pri + * separate events */ + int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + int write_ev = ep_ev[i].events & EPOLLOUT; + if (fd == NULL) { + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + } else { + if (read_ev || cancel) { + fd_become_readable(exec_ctx, fd); + } + if (write_ev || cancel) { + fd_become_writable(exec_ctx, fd); + } + } + } + } + } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); + } + } +} + +static void multipoll_with_epoll_pollset_finish_shutdown( + grpc_pollset *pollset) {} + +static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset) { + close(pollset->epoll_fd); + remove_epoll_fd_from_global_list(pollset->epoll_fd); +} + +/******************************************************************************* + * pollset_set_posix.c + */ + +static grpc_pollset_set *pollset_set_create(void) { + grpc_pollset_set *pollset_set = gpr_malloc(sizeof(*pollset_set)); + memset(pollset_set, 0, sizeof(*pollset_set)); + gpr_mu_init(&pollset_set->mu); + return pollset_set; +} + +static void pollset_set_destroy(grpc_pollset_set *pollset_set) { + size_t i; + gpr_mu_destroy(&pollset_set->mu); + for (i = 0; i < pollset_set->fd_count; i++) { + GRPC_FD_UNREF(pollset_set->fds[i], "pollset_set"); + } + gpr_free(pollset_set->pollsets); + gpr_free(pollset_set->pollset_sets); + gpr_free(pollset_set->fds); + gpr_free(pollset_set); +} + +static void pollset_set_add_pollset(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, + grpc_pollset *pollset) { + size_t i, j; + gpr_mu_lock(&pollset_set->mu); + if (pollset_set->pollset_count == pollset_set->pollset_capacity) { + pollset_set->pollset_capacity = + GPR_MAX(8, 2 * pollset_set->pollset_capacity); + pollset_set->pollsets = + gpr_realloc(pollset_set->pollsets, pollset_set->pollset_capacity * + sizeof(*pollset_set->pollsets)); + } + pollset_set->pollsets[pollset_set->pollset_count++] = pollset; + for (i = 0, j = 0; i < pollset_set->fd_count; i++) { + if (fd_is_orphaned(pollset_set->fds[i])) { + GRPC_FD_UNREF(pollset_set->fds[i], "pollset_set"); + } else { + pollset_add_fd(exec_ctx, pollset, pollset_set->fds[i]); + pollset_set->fds[j++] = pollset_set->fds[i]; + } + } + pollset_set->fd_count = j; + gpr_mu_unlock(&pollset_set->mu); +} + +static void pollset_set_del_pollset(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, + grpc_pollset *pollset) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + for (i = 0; i < pollset_set->pollset_count; i++) { + if (pollset_set->pollsets[i] == pollset) { + pollset_set->pollset_count--; + GPR_SWAP(grpc_pollset *, pollset_set->pollsets[i], + pollset_set->pollsets[pollset_set->pollset_count]); + break; + } + } + gpr_mu_unlock(&pollset_set->mu); +} + +static void pollset_set_add_pollset_set(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *bag, + grpc_pollset_set *item) { + size_t i, j; + gpr_mu_lock(&bag->mu); + if (bag->pollset_set_count == bag->pollset_set_capacity) { + bag->pollset_set_capacity = GPR_MAX(8, 2 * bag->pollset_set_capacity); + bag->pollset_sets = + gpr_realloc(bag->pollset_sets, + bag->pollset_set_capacity * sizeof(*bag->pollset_sets)); + } + bag->pollset_sets[bag->pollset_set_count++] = item; + for (i = 0, j = 0; i < bag->fd_count; i++) { + if (fd_is_orphaned(bag->fds[i])) { + GRPC_FD_UNREF(bag->fds[i], "pollset_set"); + } else { + pollset_set_add_fd(exec_ctx, item, bag->fds[i]); + bag->fds[j++] = bag->fds[i]; + } + } + bag->fd_count = j; + gpr_mu_unlock(&bag->mu); +} + +static void pollset_set_del_pollset_set(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *bag, + grpc_pollset_set *item) { + size_t i; + gpr_mu_lock(&bag->mu); + for (i = 0; i < bag->pollset_set_count; i++) { + if (bag->pollset_sets[i] == item) { + bag->pollset_set_count--; + GPR_SWAP(grpc_pollset_set *, bag->pollset_sets[i], + bag->pollset_sets[bag->pollset_set_count]); + break; + } + } + gpr_mu_unlock(&bag->mu); +} + +static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + if (pollset_set->fd_count == pollset_set->fd_capacity) { + pollset_set->fd_capacity = GPR_MAX(8, 2 * pollset_set->fd_capacity); + pollset_set->fds = gpr_realloc( + pollset_set->fds, pollset_set->fd_capacity * sizeof(*pollset_set->fds)); + } + GRPC_FD_REF(fd, "pollset_set"); + pollset_set->fds[pollset_set->fd_count++] = fd; + for (i = 0; i < pollset_set->pollset_count; i++) { + pollset_add_fd(exec_ctx, pollset_set->pollsets[i], fd); + } + for (i = 0; i < pollset_set->pollset_set_count; i++) { + pollset_set_add_fd(exec_ctx, pollset_set->pollset_sets[i], fd); + } + gpr_mu_unlock(&pollset_set->mu); +} + +static void pollset_set_del_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + for (i = 0; i < pollset_set->fd_count; i++) { + if (pollset_set->fds[i] == fd) { + pollset_set->fd_count--; + GPR_SWAP(grpc_fd *, pollset_set->fds[i], + pollset_set->fds[pollset_set->fd_count]); + GRPC_FD_UNREF(fd, "pollset_set"); + break; + } + } + for (i = 0; i < pollset_set->pollset_set_count; i++) { + pollset_set_del_fd(exec_ctx, pollset_set->pollset_sets[i], fd); + } + gpr_mu_unlock(&pollset_set->mu); +} + +/******************************************************************************* + * event engine binding + */ + +static void shutdown_engine(void) { + fd_global_shutdown(); + pollset_global_shutdown(); +} + +static const grpc_event_engine_vtable vtable = { + .pollset_size = sizeof(grpc_pollset), + + .fd_create = fd_create, + .fd_wrapped_fd = fd_wrapped_fd, + .fd_orphan = fd_orphan, + .fd_shutdown = fd_shutdown, + .fd_notify_on_read = fd_notify_on_read, + .fd_notify_on_write = fd_notify_on_write, + + .pollset_init = pollset_init, + .pollset_shutdown = pollset_shutdown, + .pollset_reset = pollset_reset, + .pollset_destroy = pollset_destroy, + .pollset_work = pollset_work, + .pollset_kick = pollset_kick, + .pollset_add_fd = pollset_add_fd, + + .pollset_set_create = pollset_set_create, + .pollset_set_destroy = pollset_set_destroy, + .pollset_set_add_pollset = pollset_set_add_pollset, + .pollset_set_del_pollset = pollset_set_del_pollset, + .pollset_set_add_pollset_set = pollset_set_add_pollset_set, + .pollset_set_del_pollset_set = pollset_set_del_pollset_set, + .pollset_set_add_fd = pollset_set_add_fd, + .pollset_set_del_fd = pollset_set_del_fd, + + .kick_poller = kick_poller, + + .shutdown_engine = shutdown_engine, +}; + +const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { + fd_global_init(); + pollset_global_init(); + polling_island_global_init(); + return &vtable; +} + +#endif diff --git a/src/core/lib/iomgr/ev_epoll_linux.h b/src/core/lib/iomgr/ev_epoll_linux.h new file mode 100644 index 00000000000..8c819975a4c --- /dev/null +++ b/src/core/lib/iomgr/ev_epoll_linux.h @@ -0,0 +1,41 @@ +/* + * + * Copyright 2015, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#ifndef GRPC_CORE_LIB_IOMGR_EV_EPOLL_LINUX_H +#define GRPC_CORE_LIB_IOMGR_EV_EPOLL_LINUX_H + +#include "src/core/lib/iomgr/ev_posix.h" + +const grpc_event_engine_vtable *grpc_init_epoll_linux(void); + +#endif /* GRPC_CORE_LIB_IOMGR_EV_EPOLL_LINUX_H */ diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c index 4481bab4380..5abd5b2a94c 100644 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ b/src/core/lib/iomgr/ev_epoll_posix.c @@ -42,6 +42,7 @@ #include #include #include +#include #include #include @@ -51,11 +52,13 @@ #include #include +#include "src/core/lib/iomgr/ev_posix.h" #include "src/core/lib/iomgr/iomgr_internal.h" #include "src/core/lib/iomgr/wakeup_fd_posix.h" #include "src/core/lib/profiling/timers.h" #include "src/core/lib/support/block_annotate.h" + /******************************************************************************* * FD declarations */ @@ -133,10 +136,9 @@ struct grpc_pollset { int called_shutdown; int kicked_without_pollers; grpc_closure *shutdown_done; - union { - int fd; - void *ptr; - } data; + + int epoll_fd; + /* Local cache of eventfds for workers */ grpc_cached_wakeup_fd *local_wakeup_cache; }; @@ -589,7 +591,6 @@ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { pollset->local_wakeup_cache = NULL; pollset->kicked_without_pollers = 0; - pollset->data.ptr = NULL; multipoll_with_epoll_pollset_create_efd(pollset); } @@ -619,22 +620,6 @@ static void pollset_reset(grpc_pollset *pollset) { pollset->kicked_without_pollers = 0; } -/* TODO (sreek): Remove multipoll_with_epoll_add_fd declaration*/ -static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, - grpc_fd *fd); - -static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_fd *fd) { - /* TODO (sreek) - Does reading pollset->data.ptr need pollset->mu lock ? - * because finally_add_fd() also reads it but without the lock! */ - gpr_mu_lock(&pollset->mu); - GPR_ASSERT(pollset->data.ptr != NULL); - gpr_mu_unlock(&pollset->mu); - - multipoll_with_epoll_pollset_add_fd(exec_ctx, pollset, fd); -} - /* TODO (sreek): Remove multipoll_with_epoll_finish_shutdown() declaration */ static void multipoll_with_epoll_pollset_finish_shutdown(grpc_pollset *pollset); @@ -790,20 +775,6 @@ static int poll_deadline_to_millis_timeout(gpr_timespec deadline, * pollset_multipoller_with_epoll_posix.c */ -#include -#include -#include -#include -#include - -#include -#include -#include - -#include "src/core/lib/iomgr/ev_posix.h" -#include "src/core/lib/profiling/timers.h" -#include "src/core/lib/support/block_annotate.h" - static void set_ready(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st) { /* only one set_ready can be active at once (but there may be a racing notify_on) */ @@ -879,13 +850,13 @@ static void remove_fd_from_all_epoll_sets(int fd) { gpr_mu_unlock(&epoll_fd_list_mu); } -typedef struct { int epoll_fd; } epoll_hdr; - -static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, +/* TODO: sreek - This function multipoll_with_epoll_pollset_add_fd() and + * finally_add_fd() in ev_poll_and_epoll_posix.c */ +static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { - /*TODO: (sree) Shouldn't this read (pollset->data.ptr) be done under a - pollset lock - i.e pollset->mu ? */ - epoll_hdr *h = pollset->data.ptr; + + /* TODO sreek - Check if we need to get a pollset->mu lock here */ + struct epoll_event ev; int err; @@ -908,7 +879,7 @@ static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); ev.data.ptr = fd; - err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); + err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); if (err < 0) { /* FDs may be added to a pollset multiple times, so EEXIST is normal. */ if (errno != EEXIST) { @@ -933,26 +904,20 @@ static void finally_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, /* TODO: This has to be called ONLY from pollset_init function. and hence it * does not acquire any lock */ static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { - epoll_hdr *h = gpr_malloc(sizeof(epoll_hdr)); struct epoll_event ev; int err; - /* TODO (sreek). remove this assert. Currently added this just to ensure that - * we do not overwrite h->epoll_fd without freeing the older one*/ - GPR_ASSERT(pollset->data.ptr == NULL); - - pollset->data.ptr = h; - h->epoll_fd = epoll_create1(EPOLL_CLOEXEC); - if (h->epoll_fd < 0) { + pollset->epoll_fd = epoll_create1(EPOLL_CLOEXEC); + if (pollset->epoll_fd < 0) { gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); abort(); } - add_epoll_fd_to_global_list(h->epoll_fd); + add_epoll_fd_to_global_list(pollset->epoll_fd); ev.events = (uint32_t)(EPOLLIN | EPOLLET); ev.data.ptr = NULL; - err = epoll_ctl(h->epoll_fd, EPOLL_CTL_ADD, + err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); if (err < 0) { gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", @@ -961,12 +926,6 @@ static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { } } -static void multipoll_with_epoll_pollset_add_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, - grpc_fd *fd) { - finally_add_fd(exec_ctx, pollset, fd); -} - /* TODO(klempner): We probably want to turn this down a bit */ #define GRPC_EPOLL_MAX_EVENTS 1000 @@ -974,9 +933,9 @@ static void multipoll_with_epoll_pollset_maybe_work_and_unlock( grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, gpr_timespec deadline, gpr_timespec now) { struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; + int epoll_fd = pollset->epoll_fd; int ep_rv; int poll_rv; - epoll_hdr *h = pollset->data.ptr; int timeout_ms; struct pollfd pfds[2]; @@ -993,7 +952,7 @@ static void multipoll_with_epoll_pollset_maybe_work_and_unlock( pfds[0].fd = GRPC_WAKEUP_FD_GET_READ_FD(&worker->wakeup_fd->fd); pfds[0].events = POLLIN; pfds[0].revents = 0; - pfds[1].fd = h->epoll_fd; + pfds[1].fd = epoll_fd; pfds[1].events = POLLIN; pfds[1].revents = 0; @@ -1018,7 +977,7 @@ static void multipoll_with_epoll_pollset_maybe_work_and_unlock( if (pfds[1].revents) { do { /* The following epoll_wait never blocks; it has a timeout of 0 */ - ep_rv = epoll_wait(h->epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); + ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); if (ep_rv < 0) { if (errno != EINTR) { gpr_log(GPR_ERROR, "epoll_wait() failed: %s", strerror(errno)); @@ -1053,10 +1012,8 @@ static void multipoll_with_epoll_pollset_finish_shutdown( grpc_pollset *pollset) {} static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset) { - epoll_hdr *h = pollset->data.ptr; - close(h->epoll_fd); - remove_epoll_fd_from_global_list(h->epoll_fd); - gpr_free(h); + close(pollset->epoll_fd); + remove_epoll_fd_from_global_list(pollset->epoll_fd); } /******************************************************************************* diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index baa3b9856ad..404ef2a64b9 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -44,7 +44,7 @@ #include #include -#include "src/core/lib/iomgr/ev_epoll_posix.h" +#include "src/core/lib/iomgr/ev_epoll_linux.h" #include "src/core/lib/iomgr/ev_poll_posix.h" #include "src/core/lib/support/env.h" @@ -163,11 +163,6 @@ void grpc_fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, g_event_engine->fd_notify_on_write(exec_ctx, fd, closure); } -grpc_pollset *grpc_fd_get_read_notifier_pollset(grpc_exec_ctx *exec_ctx, - grpc_fd *fd) { - return g_event_engine->fd_get_read_notifier_pollset(exec_ctx, fd); -} - size_t grpc_pollset_size(void) { return g_event_engine->pollset_size; } void grpc_pollset_init(grpc_pollset *pollset, gpr_mu **mu) { diff --git a/src/python/grpcio/grpc_core_dependencies.py b/src/python/grpcio/grpc_core_dependencies.py index 49b4ddc457c..13bc6888d66 100644 --- a/src/python/grpcio/grpc_core_dependencies.py +++ b/src/python/grpcio/grpc_core_dependencies.py @@ -94,6 +94,7 @@ CORE_SOURCE_FILES = [ 'src/core/lib/iomgr/endpoint.c', 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', + 'src/core/lib/iomgr/ev_epoll_linux.c', 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal index 36b25cca161..d968278f2a4 100644 --- a/tools/doxygen/Doxyfile.core.internal +++ b/tools/doxygen/Doxyfile.core.internal @@ -807,6 +807,7 @@ src/core/lib/http/parser.h \ src/core/lib/iomgr/closure.h \ src/core/lib/iomgr/endpoint.h \ src/core/lib/iomgr/endpoint_pair.h \ +src/core/lib/iomgr/ev_epoll_linux.h \ src/core/lib/iomgr/ev_epoll_posix.h \ src/core/lib/iomgr/ev_poll_posix.h \ src/core/lib/iomgr/ev_posix.h \ @@ -955,6 +956,7 @@ src/core/lib/iomgr/closure.c \ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ +src/core/lib/iomgr/ev_epoll_linux.c \ src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index 43940495864..97cc55db36e 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -5530,6 +5530,7 @@ "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_linux.h", "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", @@ -5630,6 +5631,8 @@ "src/core/lib/iomgr/endpoint_pair.h", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_linux.c", + "src/core/lib/iomgr/ev_epoll_linux.h", "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.c", diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj b/vsprojects/vcxproj/grpc/grpc.vcxproj index 55304af5869..a67e4d16dad 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj @@ -316,6 +316,7 @@ + @@ -484,6 +485,8 @@ + + diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters index 7d1c90fda7c..bf9b7dc7dcf 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters @@ -55,6 +55,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr @@ -677,6 +680,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj index 3d0cdfc668b..afc9a2ca1b2 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj @@ -304,6 +304,7 @@ + @@ -450,6 +451,8 @@ + + diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters index d2ff4c630fd..b7507f9a963 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters @@ -58,6 +58,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr @@ -575,6 +578,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr From 9442bab5d303d7bd33e9406129ad897588d07111 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 20 May 2016 17:54:06 -0700 Subject: [PATCH 020/280] Write most of the methods in the new epoll implementation --- src/core/lib/iomgr/ev_epoll_linux.c | 301 ++++++++++++++++++++++------ 1 file changed, 244 insertions(+), 57 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index f257ac8a1dd..0d30bb659b6 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -150,28 +150,84 @@ typedef struct polling_island { static gpr_mu g_pi_freelist_mu; static polling_island *g_pi_freelist = NULL; -/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? */ -static void add_fd_to_polling_island_locked(polling_island *pi, grpc_fd *fd) { +/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? + * TODO: sreek - Should this add a ref to the grpc_fd ? */ +/* The caller is expected to hold pi->mu lock before calling this function */ +static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, + size_t fd_count) { int err; + size_t i; struct epoll_event ev; - ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); - ev.data.ptr = fd; - err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); + for (i = 0; i < fd_count; i++) { + ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); + ev.data.ptr = fds[i]; + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, fds[i]->fd, &ev); + + if (err < 0 && errno != EEXIST) { + gpr_log(GPR_ERROR, "epoll_ctl add for fd: %d failed with error: %s", + fds[i]->fd, strerror(errno)); + /* TODO: sreek - Not sure if it is a good idea to continue here. We need a + * better way to bubble up this error instead of doing an abort() */ + continue; + } - if (err < 0 && errno != EEXIST) { - gpr_log(GPR_ERROR, "epoll_ctl add for fd: %d failed with error: %s", fd->fd, - strerror(errno)); - return; + if (pi->fd_cnt == pi->fd_capacity) { + pi->fd_capacity = GPR_MAX(pi->fd_capacity + 8, pi->fd_cnt * 3 / 2); + pi->fds = gpr_realloc(pi->fds, sizeof(grpc_fd *) * pi->fd_capacity); + } + + pi->fds[pi->fd_cnt++] = fds[i]; + } +} + +/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? + * TODO: sreek - Might have to unref the fds (assuming whether we add a ref to + * the fd when adding it to the epollset) */ +/* The caller is expected to hold pi->mu lock before calling this function */ +static void polling_island_clear_fds_locked(polling_island *pi) { + int err; + size_t i; + + for (i = 0; i < pi->fd_cnt; i++) { + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, pi->fds[i]->fd, NULL); + + if (err < 0 && errno != ENOENT) { + gpr_log(GPR_ERROR, + "epoll_ctl delete for fds[i]: %d failed with error: %s", i, + pi->fds[i]->fd, strerror(errno)); + /* TODO: sreek - Not sure if it is a good idea to continue here. We need a + * better way to bubble up this error instead of doing an abort() */ + continue; + } + } + + pi->fd_cnt = 0; +} + +/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? + * TODO: sreek - Might have to unref the fd (assuming whether we add a ref to + * the fd when adding it to the epollset) */ +/* The caller is expected to hold pi->mu lock before calling this function */ +static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd) { + int err; + size_t i; + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, fd->fd, NULL); + if (err < 0 && errno != ENOENT) { + gpr_log(GPR_ERROR, "epoll_ctl delete for fd: %d failed with error; %s", + fd->fd, strerror(errno)); } - pi->fd_capacity = GPR_MAX(pi->fd_capacity + 8, pi->fd_cnt * 3 / 2); - pi->fds = gpr_realloc(pi->fds, sizeof(grpc_fd *) * pi->fd_capacity); - pi->fds[pi->fd_cnt++] = fd; + for (i = 0; i < pi->fd_cnt; i++) { + if (pi->fds[i] == fd) { + pi->fds[i] = pi->fds[--pi->fd_cnt]; + break; + } + } } -static polling_island *polling_island_create(int initial_ref_cnt, - grpc_fd *initial_fd) { +static polling_island *polling_island_create(grpc_fd *initial_fd, + int initial_ref_cnt) { polling_island *pi = NULL; gpr_mu_lock(&g_pi_freelist_mu); if (g_pi_freelist != NULL) { @@ -202,17 +258,151 @@ static polling_island *polling_island_create(int initial_ref_cnt, pi->next_free = NULL; if (initial_fd != NULL) { - /* add_fd_to_polling_island_locked() expects the caller to hold a pi->mu + /* polling_island_add_fds_locked() expects the caller to hold a pi->mu * lock. However, since this is a new polling island (and no one has a * reference to it yet), it is okay to not acquire pi->mu here */ - add_fd_to_polling_island_locked(pi, initial_fd); + polling_island_add_fds_locked(pi, &initial_fd, 1); } return pi; } +static void polling_island_delete(polling_island *pi) { + GPR_ASSERT(pi->ref_cnt == 0); + GPR_ASSERT(pi->fd_cnt == 0); + + pi->merged_to = NULL; + + gpr_mu_lock(&g_pi_freelist_mu); + pi->next_free = g_pi_freelist; + g_pi_freelist = pi; + gpr_mu_unlock(&g_pi_freelist_mu); +} + +void polling_island_unref_and_unlock(polling_island *pi, int unref_by) { + pi->ref_cnt -= unref_by; + int ref_cnt = pi->ref_cnt; + GPR_ASSERT(ref_cnt >= 0); + + gpr_mu_unlock(&pi->mu); + + if (ref_cnt == 0) { + polling_island_delete(pi); + } +} + +polling_island *polling_island_update_and_lock(polling_island *pi, int unref_by, + int add_ref_by) { + polling_island *next = NULL; + gpr_mu_lock(&pi->mu); + while (pi->merged_to != NULL) { + next = pi->merged_to; + polling_island_unref_and_unlock(pi, unref_by); + pi = next; + gpr_mu_lock(&pi->mu); + } + + pi->ref_cnt += add_ref_by; + return pi; +} + +void polling_island_pair_update_and_lock(polling_island **p, + polling_island **q) { + polling_island *pi_1 = *p; + polling_island *pi_2 = *q; + polling_island *temp = NULL; + bool pi_1_locked = false; + bool pi_2_locked = false; + int num_swaps = 0; + + while (pi_1 != pi_2 && !(pi_1_locked && pi_2_locked)) { + // pi_1 is NOT equal to pi_2 + // pi_1 MAY be locked + + if (pi_1 > pi_2) { + if (pi_1_locked) { + gpr_mu_unlock(&pi_1->mu); + pi_1_locked = false; + } + + GPR_SWAP(polling_island *, pi_1, pi_2); + num_swaps++; + } + + // p1 < p2 + // p1 MAY BE locked + // p2 is NOT locked + + if (!pi_1_locked) { + gpr_mu_lock(&pi_1->mu); + pi_1_locked = true; + + if (pi_1->merged_to != NULL) { + temp = pi_1->merged_to; + polling_island_unref_and_unlock(pi_1, 1); + pi_1 = temp; + pi_1_locked = false; + + continue; + } + } + + // p1 is LOCKED + // p2 is UNLOCKED + // p1 != p2 + + gpr_mu_lock(&pi_2->mu); + pi_2_locked = true; + + if (pi_2->merged_to != NULL) { + temp = pi_2->merged_to; + polling_island_unref_and_unlock(pi_2, 1); + pi_2 = temp; + pi_2_locked = false; + } + } + + // Either pi_1 == pi_2 OR we got both locks! + if (pi_1 == pi_2) { + GPR_ASSERT(pi_1_locked || (!pi_1_locked && !pi_2_locked)); + if (!pi_1_locked) { + pi_1 = pi_2 = polling_island_update_and_lock(pi_1, 2, 0); + } + } else { + GPR_ASSERT(pi_1_locked && pi_2_locked); + if (num_swaps % 2 > 0) { + GPR_SWAP(polling_island *, pi_1, pi_2); + } + } + + *p = pi_1; + *q = pi_2; +} + +polling_island *polling_island_merge(polling_island *p, polling_island *q) { + polling_island *merged = NULL; + + polling_island_pair_update_and_lock(&p, &q); + + /* TODO: sreek: Think about this scenario some more. Is it possible ?. what + * does it mean, when would this happen */ + if (p == q) { + merged = p; + } + + // Move all the fds from polling_island p to polling_island q + polling_island_add_fds_locked(q, p->fds, p->fd_cnt); + polling_island_clear_fds_locked(p); + + q->ref_cnt += p->ref_cnt; + + gpr_mu_unlock(&p->mu); + gpr_mu_unlock(&q->mu); + + return merged; +} + static void polling_island_global_init() { - polling_island_create(0, NULL); /* TODO(sreek): Delete this line */ gpr_mu_init(&g_pi_freelist_mu); g_pi_freelist = NULL; } @@ -245,7 +435,7 @@ struct grpc_pollset { int epoll_fd; - /* Mutex protecting the 'polling_island' field */ + /* Mutex protecting the 'polling_island' field */ gpr_mu pi_mu; /* The polling island to which this fd belongs to. An fd belongs to exactly @@ -319,7 +509,8 @@ struct grpc_pollset_set { * fd_posix.c */ -/* We need to keep a freelist not because of any concerns of malloc performance +/* We need to keep a freelist not because of any concerns of malloc + * performance * but instead so that implementations with multiple threads in (for example) * epoll_wait deal with the race between pollset removal and incoming poll * notifications. @@ -434,6 +625,7 @@ static void fd_global_shutdown(void) { static grpc_fd *fd_create(int fd, const char *name) { grpc_fd *r = alloc_fd(fd); + char *name2; gpr_asprintf(&name2, "%s fd=%d", name, fd); grpc_iomgr_register_object(&r->iomgr_object, name2); @@ -453,6 +645,20 @@ static void close_fd_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { if (!fd->released) { close(fd->fd); } else { + /* TODO: sreek - Check for deadlocks */ + + gpr_mu_lock(&fd->pi_mu); + fd->polling_island = + polling_island_update_and_lock(fd->polling_island, 1, 0); + + polling_island_remove_fd_locked(fd->polling_island, fd); + polling_island_unref_and_unlock(fd->polling_island, 1); + + fd->polling_island = NULL; + gpr_mu_unlock(&fd->pi_mu); + + + /* TODO: sreek - This should be no longer needed */ remove_fd_from_all_epoll_sets(fd->fd); } grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); @@ -752,7 +958,8 @@ static void finish_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); } -/* TODO(sreek): Remove multipoll_with_epoll_*_maybe_work_and_unlock declaration +/* TODO(sreek): Remove multipoll_with_epoll_*_maybe_work_and_unlock + * declaration */ static void multipoll_with_epoll_pollset_maybe_work_and_unlock( grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, @@ -979,50 +1186,30 @@ static void remove_fd_from_all_epoll_sets(int fd) { * finally_add_fd() in ev_poll_and_epoll_posix.c */ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { - /* TODO sreek - Check if we need to get a pollset->mu lock here */ + gpr_mu_lock(&pollset->pi_mu); + gpr_mu_lock(&fd->pi_mu); - struct epoll_event ev; - int err; - - /* Hold a ref to the fd to keep it from being closed during the add. This may - result in a spurious wakeup being assigned to this pollset whilst adding, - but that should be benign. */ - /* TODO: (sreek): Understand how a spurious wake up migh be assinged to this - * pollset..and how holding a reference will prevent the fd from being closed - * (and perhaps more importantly, see how can an fd be closed while being - * added to the epollset */ - GRPC_FD_REF(fd, "add fd"); + polling_island *pi_new = NULL; - gpr_mu_lock(&fd->mu); - if (fd->shutdown) { - gpr_mu_unlock(&fd->mu); - GRPC_FD_UNREF(fd, "add fd"); - return; - } - gpr_mu_unlock(&fd->mu); - - ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); - ev.data.ptr = fd; - err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); - if (err < 0) { - /* FDs may be added to a pollset multiple times, so EEXIST is normal. */ - if (errno != EEXIST) { - gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", fd->fd, - strerror(errno)); + if (fd->polling_island == pollset->polling_island) { + pi_new = fd->polling_island; + if (pi_new == NULL) { + pi_new = polling_island_create(fd, 2); } - } + } else if (fd->polling_island == NULL) { + pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); - /* The fd might have been orphaned while we were adding it to the epoll set. - Close the fd in such a case (which will also take care of removing it from - the epoll set */ - gpr_mu_lock(&fd->mu); - if (fd_is_orphaned(fd) && !fd->closed) { - close_fd_locked(exec_ctx, fd); + } else if (pollset->polling_island == NULL) { + pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); + } else { // Non null and different + pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); } - gpr_mu_unlock(&fd->mu); - GRPC_FD_UNREF(fd, "add fd"); + fd->polling_island = pollset->polling_island = pi_new; + + gpr_mu_unlock(&fd->pi_mu); + gpr_mu_unlock(&pollset->pi_mu); } /* Creates an epoll fd and initializes the pollset */ From d806145573f8e78a52012b4b8ab94ff46b855d58 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 20 May 2016 18:12:30 -0700 Subject: [PATCH 021/280] Removed epoll_fd_global_list --- src/core/lib/iomgr/ev_epoll_linux.c | 68 +---------------------------- 1 file changed, 1 insertion(+), 67 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 0d30bb659b6..7793a952016 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -483,8 +483,6 @@ typedef void (*platform_become_multipoller_type)(grpc_exec_ctx *exec_ctx, * be locked) */ static int pollset_has_workers(grpc_pollset *pollset); -static void remove_fd_from_all_epoll_sets(int fd); - /******************************************************************************* * pollset_set definitions */ @@ -656,11 +654,8 @@ static void close_fd_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { fd->polling_island = NULL; gpr_mu_unlock(&fd->pi_mu); - - - /* TODO: sreek - This should be no longer needed */ - remove_fd_from_all_epoll_sets(fd->fd); } + grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); } @@ -1123,65 +1118,6 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { set_ready(exec_ctx, fd, &fd->write_closure); } -/* TODO (sreek): Maybe this global list is not required. Double check*/ -struct epoll_fd_list { - int *epoll_fds; - size_t count; - size_t capacity; -}; - -static struct epoll_fd_list epoll_fd_global_list; -static gpr_once init_epoll_fd_list_mu = GPR_ONCE_INIT; -static gpr_mu epoll_fd_list_mu; - -static void init_mu(void) { gpr_mu_init(&epoll_fd_list_mu); } - -static void add_epoll_fd_to_global_list(int epoll_fd) { - gpr_once_init(&init_epoll_fd_list_mu, init_mu); - - gpr_mu_lock(&epoll_fd_list_mu); - if (epoll_fd_global_list.count == epoll_fd_global_list.capacity) { - epoll_fd_global_list.capacity = - GPR_MAX((size_t)8, epoll_fd_global_list.capacity * 2); - epoll_fd_global_list.epoll_fds = - gpr_realloc(epoll_fd_global_list.epoll_fds, - epoll_fd_global_list.capacity * sizeof(int)); - } - epoll_fd_global_list.epoll_fds[epoll_fd_global_list.count++] = epoll_fd; - gpr_mu_unlock(&epoll_fd_list_mu); -} - -static void remove_epoll_fd_from_global_list(int epoll_fd) { - gpr_mu_lock(&epoll_fd_list_mu); - GPR_ASSERT(epoll_fd_global_list.count > 0); - for (size_t i = 0; i < epoll_fd_global_list.count; i++) { - if (epoll_fd == epoll_fd_global_list.epoll_fds[i]) { - epoll_fd_global_list.epoll_fds[i] = - epoll_fd_global_list.epoll_fds[--(epoll_fd_global_list.count)]; - break; - } - } - gpr_mu_unlock(&epoll_fd_list_mu); -} - -static void remove_fd_from_all_epoll_sets(int fd) { - int err; - gpr_once_init(&init_epoll_fd_list_mu, init_mu); - gpr_mu_lock(&epoll_fd_list_mu); - if (epoll_fd_global_list.count == 0) { - gpr_mu_unlock(&epoll_fd_list_mu); - return; - } - for (size_t i = 0; i < epoll_fd_global_list.count; i++) { - err = epoll_ctl(epoll_fd_global_list.epoll_fds[i], EPOLL_CTL_DEL, fd, NULL); - if (err < 0 && errno != ENOENT) { - gpr_log(GPR_ERROR, "epoll_ctl del for %d failed: %s", fd, - strerror(errno)); - } - } - gpr_mu_unlock(&epoll_fd_list_mu); -} - /* TODO: sreek - This function multipoll_with_epoll_pollset_add_fd() and * finally_add_fd() in ev_poll_and_epoll_posix.c */ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, @@ -1224,7 +1160,6 @@ static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); abort(); } - add_epoll_fd_to_global_list(pollset->epoll_fd); ev.events = (uint32_t)(EPOLLIN | EPOLLET); ev.data.ptr = NULL; @@ -1325,7 +1260,6 @@ static void multipoll_with_epoll_pollset_finish_shutdown( static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset) { close(pollset->epoll_fd); - remove_epoll_fd_from_global_list(pollset->epoll_fd); } /******************************************************************************* From 96b2554313120949dd26e3c0968e8aea9b8a650f Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 20 May 2016 18:14:48 -0700 Subject: [PATCH 022/280] ctiller's ev_epoll_linux.c file (for reference) --- src/core/lib/iomgr/ctiller_ev_epoll_linux.c | 461 ++++++++++++++++++++ 1 file changed, 461 insertions(+) create mode 100644 src/core/lib/iomgr/ctiller_ev_epoll_linux.c diff --git a/src/core/lib/iomgr/ctiller_ev_epoll_linux.c b/src/core/lib/iomgr/ctiller_ev_epoll_linux.c new file mode 100644 index 00000000000..23c20a77aae --- /dev/null +++ b/src/core/lib/iomgr/ctiller_ev_epoll_linux.c @@ -0,0 +1,461 @@ +/* + * + * Copyright 2015-2016, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include "src/core/lib/iomgr/ev_epoll_linux.h" + +#include +#include +#include +#include + +#include +#include +#include +#include + +#include "src/core/lib/iomgr/iomgr_internal.h" + +/* TODO(sreek) Remove this file */ + + +//////////////////////////////////////////////////////////////////////////////// +// Definitions + +#define STATE_NOT_READY ((gpr_atm)0) +#define STATE_READY ((gpr_atm)1) + +typedef enum { POLLABLE_FD, POLLABLE_EPOLL_SET } pollable_type; + +typedef struct { + pollable_type type; + int fd; + grpc_iomgr_object iomgr_object; +} pollable_object; + +typedef struct polling_island { + pollable_object pollable; + gpr_mu mu; + int refs; + grpc_fd *only_fd; + struct polling_island *became; + struct polling_island *next; +} polling_island; + +struct grpc_fd { + pollable_object pollable; + + // each event atomic is a tri state: + // STATE_NOT_READY - no event received, nobody waiting for it either + // STATE_READY - event received, nobody waiting for it + // closure pointer - no event received, upper layer is waiting for it + gpr_atm on_readable; + gpr_atm on_writable; + + // mutex guarding set_ready & shutdown state + gpr_mu set_ready_mu; + bool shutdown; + + // mutex protecting polling_island + gpr_mu polling_island_mu; + // current polling island + polling_island *polling_island; + + grpc_fd *next_free; +}; + +struct grpc_pollset_worker {}; + +struct grpc_pollset { + gpr_mu mu; + // current polling island + polling_island *polling_island; +}; + +//////////////////////////////////////////////////////////////////////////////// +// Polling island implementation + +static gpr_mu g_pi_freelist_mu; +static polling_island *g_first_free_pi; + +static void add_pollable_to_epoll_set(pollable_object *pollable, int epoll_set, + uint32_t events) { + struct epoll_event ev; + ev.events = events; + ev.data.ptr = pollable; + int err = epoll_ctl(epoll_set, EPOLL_CTL_ADD, pollable->fd, &ev); + if (err < 0) { + gpr_log(GPR_ERROR, "epoll_ctl add for %d faild: %s", pollable->fd, + strerror(errno)); + } +} + +static void add_fd_to_epoll_set(grpc_fd *fd, int epoll_set) { + add_pollable_to_epoll_set(&fd->pollable, epoll_set, + EPOLLIN | EPOLLOUT | EPOLLET); +} + +static void add_island_to_epoll_set(polling_island *pi, int epoll_set) { + add_pollable_to_epoll_set(&pi->pollable, epoll_set, EPOLLIN | EPOLLET); +} + +static polling_island *polling_island_create(grpc_fd *initial_fd) { + polling_island *r = NULL; + gpr_mu_lock(&g_pi_freelist_mu); + if (g_first_free_pi == NULL) { + r = gpr_malloc(sizeof(*r)); + r->pollable.type = POLLABLE_EPOLL_SET; + gpr_mu_init(&r->mu); + } else { + r = g_first_free_pi; + g_first_free_pi = r->next; + } + gpr_mu_unlock(&g_pi_freelist_mu); + + r->pollable.fd = epoll_create1(EPOLL_CLOEXEC); + GPR_ASSERT(r->pollable.fd >= 0); + + gpr_mu_lock(&r->mu); + r->only_fd = initial_fd; + r->refs = 2; // creation of a polling island => a referencing pollset & fd + gpr_mu_unlock(&r->mu); + + add_fd_to_epoll_set(initial_fd, r->pollable.fd); + return r; +} + +static void polling_island_delete(polling_island *p) { + gpr_mu_lock(&g_pi_freelist_mu); + p->next = g_first_free_pi; + g_first_free_pi = p; + gpr_mu_unlock(&g_pi_freelist_mu); +} + +static polling_island *polling_island_add(polling_island *p, grpc_fd *fd) { + gpr_mu_lock(&p->mu); + p->only_fd = NULL; + p->refs++; // new fd picks up a ref + gpr_mu_unlock(&p->mu); + + add_fd_to_epoll_set(fd, p->pollable.fd); + + return p; +} + +static void add_siblings_to(polling_island *siblings, polling_island *dest) { + polling_island *sibling_tail = dest; + while (sibling_tail->next != NULL) { + sibling_tail = sibling_tail->next; + } + sibling_tail->next = siblings; +} + +static polling_island *polling_island_merge(polling_island *a, + polling_island *b) { + GPR_ASSERT(a != b); + polling_island *out; + + gpr_mu_lock(&GPR_MIN(a, b)->mu); + gpr_mu_lock(&GPR_MAX(a, b)->mu); + + GPR_ASSERT(a->became == NULL); + GPR_ASSERT(b->became == NULL); + + if (a->only_fd == NULL && b->only_fd == NULL) { + b->became = a; + add_siblings_to(b, a); + add_island_to_epoll_set(b, a->pollable.fd); + out = a; + } else if (a->only_fd == NULL) { + GPR_ASSERT(b->only_fd != NULL); + add_fd_to_epoll_set(b->only_fd, a->pollable.fd); + b->became = a; + out = a; + } else if (b->only_fd == NULL) { + GPR_ASSERT(a->only_fd != NULL); + add_fd_to_epoll_set(a->only_fd, b->pollable.fd); + a->became = b; + out = b; + } else { + add_fd_to_epoll_set(b->only_fd, a->pollable.fd); + a->only_fd = NULL; + b->only_fd = NULL; + b->became = a; + out = a; + } + + gpr_mu_unlock(&a->mu); + gpr_mu_unlock(&b->mu); + + return out; +} + +static polling_island *polling_island_update_and_lock(polling_island *p) { + gpr_mu_lock(&p->mu); + if (p->became != NULL) { + do { + polling_island *from = p; + p = p->became; + gpr_mu_lock(&p->mu); + bool delete_from = 0 == --from->refs; + p->refs++; + gpr_mu_unlock(&from->mu); + if (delete_from) { + polling_island_delete(from); + } + } while (p->became != NULL); + } + return p; +} + +static polling_island *polling_island_ref(polling_island *p) { + gpr_mu_lock(&p->mu); + gpr_mu_unlock(&p->mu); + return p; +} + +static void polling_island_drop(polling_island *p) {} + +static polling_island *polling_island_update(polling_island *p, + int updating_owner_count) { + p = polling_island_update_and_lock(p); + GPR_ASSERT(p->refs != 0); + p->refs += updating_owner_count; + gpr_mu_unlock(&p->mu); + return p; +} + +//////////////////////////////////////////////////////////////////////////////// +// FD implementation + +static gpr_mu g_fd_freelist_mu; +static grpc_fd *g_first_free_fd; + +static grpc_fd *fd_create(int fd, const char *name) { + grpc_fd *r = NULL; + gpr_mu_lock(&g_fd_freelist_mu); + if (g_first_free_fd == NULL) { + r = gpr_malloc(sizeof(*r)); + r->pollable.type = POLLABLE_FD; + gpr_atm_rel_store(&r->on_readable, 0); + gpr_atm_rel_store(&r->on_writable, 0); + gpr_mu_init(&r->polling_island_mu); + gpr_mu_init(&r->set_ready_mu); + } else { + r = g_first_free_fd; + g_first_free_fd = r->next_free; + } + gpr_mu_unlock(&g_fd_freelist_mu); + + r->pollable.fd = fd; + grpc_iomgr_register_object(&r->pollable.iomgr_object, name); + r->next_free = NULL; + return r; +} + +static int fd_wrapped_fd(grpc_fd *fd) { return fd->pollable.fd; } + +static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *on_done, int *release_fd, + const char *reason) { + if (release_fd != NULL) { + *release_fd = fd->pollable.fd; + } else { + close(fd->pollable.fd); + } + + gpr_mu_lock(&fd->polling_island_mu); + if (fd->polling_island != NULL) { + polling_island_drop(fd->polling_island); + } + gpr_mu_unlock(&fd->polling_island_mu); + + gpr_mu_lock(&g_fd_freelist_mu); + fd->next_free = g_first_free_fd; + g_first_free_fd = fd; + grpc_iomgr_unregister_object(&fd->pollable.iomgr_object); + gpr_mu_unlock(&g_fd_freelist_mu); + + grpc_exec_ctx_enqueue(exec_ctx, on_done, true, NULL); +} + +static void notify_on(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *closure, gpr_atm *state) { + if (gpr_atm_acq_cas(state, STATE_NOT_READY, (gpr_atm)closure)) { + // state was not ready, and is now the closure - we're done */ + } else { + // cas failed - we MUST be in STATE_READY (can't request two notifications + // for the same event) + // flip back to not ready, enqueue the closure directly + GPR_ASSERT(gpr_atm_rel_cas(state, STATE_READY, STATE_NOT_READY)); + grpc_exec_ctx_enqueue(exec_ctx, closure, true, NULL); + } +} + +static void fd_notify_on_read(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *closure) { + notify_on(exec_ctx, fd, closure, &fd->on_readable); +} + +static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_closure *closure) { + notify_on(exec_ctx, fd, closure, &fd->on_readable); +} + +static void destroy_fd_freelist(void) { + while (g_first_free_fd) { + grpc_fd *next = g_first_free_fd->next_free; + gpr_mu_destroy(&g_first_free_fd->polling_island_mu); + gpr_free(next); + g_first_free_fd = next; + } +} + +static void set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + gpr_atm *state) { + if (gpr_atm_acq_cas(state, STATE_NOT_READY, STATE_READY)) { + // state was not ready, and is now ready - we're done + } else { + // cas failed - either there's a closure queued which we should consume OR + // the state was already STATE_READY + gpr_atm cur_state = gpr_atm_acq_load(state); + if (cur_state != STATE_READY) { + // state wasn't STATE_READY - it *must* have been a closure + // since it's illegal to ask for notification twice, it's safe to assume + // that we'll resume being the closure + GPR_ASSERT(gpr_atm_rel_cas(state, cur_state, STATE_NOT_READY)); + grpc_exec_ctx_enqueue(exec_ctx, (grpc_closure *)cur_state, !fd->shutdown, + NULL); + } + } +} + +static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { + gpr_mu_lock(&fd->set_ready_mu); + GPR_ASSERT(!fd->shutdown); + fd->shutdown = 1; + set_ready_locked(exec_ctx, fd, &fd->on_readable); + set_ready_locked(exec_ctx, fd, &fd->on_writable); + gpr_mu_unlock(&fd->set_ready_mu); +} + +//////////////////////////////////////////////////////////////////////////////// +// Pollset implementation + +static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { + gpr_mu_init(&pollset->mu); + *mu = &pollset->mu; + pollset->polling_island = NULL; +} + +static void pollset_destroy(grpc_pollset *pollset) { + gpr_mu_destroy(&pollset->mu); + if (pollset->polling_island) { + polling_island_drop(pollset->polling_island); + } +} + +static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + struct grpc_fd *fd) { + gpr_mu_lock(&pollset->mu); + gpr_mu_lock(&fd->polling_island_mu); + + polling_island *new; + + if (fd->polling_island == NULL) { + if (pollset->polling_island == NULL) { + new = polling_island_create(fd); + } else { + new = polling_island_add(pollset->polling_island, fd); + } + } else if (pollset->polling_island == NULL) { + new = polling_island_ref(fd->polling_island); + } else if (pollset->polling_island != fd->polling_island) { + new = polling_island_merge(pollset->polling_island, fd->polling_island); + } else { + new = polling_island_update(pollset->polling_island, 1); + } + + fd->polling_island = pollset->polling_island = new; + + gpr_mu_unlock(&fd->polling_island_mu); + gpr_mu_unlock(&pollset->mu); +} + +//////////////////////////////////////////////////////////////////////////////// +// Engine binding + +static void shutdown_engine(void) { destroy_fd_freelist(); } + +static const grpc_event_engine_vtable vtable = { + .pollset_size = sizeof(grpc_pollset), + + .fd_create = fd_create, + .fd_wrapped_fd = fd_wrapped_fd, + .fd_orphan = fd_orphan, + .fd_shutdown = fd_shutdown, + .fd_notify_on_read = fd_notify_on_read, + .fd_notify_on_write = fd_notify_on_write, + + .pollset_init = pollset_init, + .pollset_shutdown = pollset_shutdown, + .pollset_reset = pollset_reset, + .pollset_destroy = pollset_destroy, + .pollset_work = pollset_work, + .pollset_kick = pollset_kick, + .pollset_add_fd = pollset_add_fd, + + .pollset_set_create = pollset_set_create, + .pollset_set_destroy = pollset_set_destroy, + .pollset_set_add_pollset = pollset_set_add_pollset, + .pollset_set_del_pollset = pollset_set_del_pollset, + .pollset_set_add_pollset_set = pollset_set_add_pollset_set, + .pollset_set_del_pollset_set = pollset_set_del_pollset_set, + .pollset_set_add_fd = pollset_set_add_fd, + .pollset_set_del_fd = pollset_set_del_fd, + + .kick_poller = kick_poller, + + .shutdown_engine = shutdown_engine, +}; + +static bool is_epoll_available(void) { + abort(); + return false; +} + +const grpc_event_engine_vtable *grpc_init_poll_posix(void) { + if (!is_epoll_available()) { + return NULL; + } + return &vtable; +} From d7d6eed78822ed8de14fdf9b39255365946cf8c4 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 31 May 2016 09:44:08 -0700 Subject: [PATCH 023/280] Correct typo --- src/core/lib/iomgr/ev_posix.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index 404ef2a64b9..96399ef8379 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -44,7 +44,7 @@ #include #include -#include "src/core/lib/iomgr/ev_epoll_linux.h" +#include "src/core/lib/iomgr/ev_epoll_posix.h" #include "src/core/lib/iomgr/ev_poll_posix.h" #include "src/core/lib/support/env.h" From 8c6c9067bc001a7cd94c2689570a6ec27affee82 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 31 May 2016 10:57:20 -0700 Subject: [PATCH 024/280] run epoll tests too --- tools/run_tests/run_tests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index ddaf96c345a..f3b191feab7 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -157,7 +157,7 @@ class CLanguage(object): 'windows': ['all'], 'mac': ['all'], 'posix': ['all'], - 'linux': ['poll'], + 'linux': ['poll', 'epoll'], } for target in binaries: polling_strategies = (POLLING_STRATEGIES[self.platform] From 5098f91159f2a5c0494688b8cfaff4debef5686f Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 31 May 2016 10:58:17 -0700 Subject: [PATCH 025/280] Rewrite all the pollset and fd functions in ev_epoll_linux.c --- src/core/lib/iomgr/ev_epoll_linux.c | 161 +++++++--------------------- 1 file changed, 38 insertions(+), 123 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 7793a952016..1201c10a7e0 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -67,10 +67,10 @@ struct polling_island; struct grpc_fd { int fd; /* refst format: - bit0: 1=active/0=orphaned - bit1-n: refcount - meaning that mostly we ref by two to avoid altering the orphaned bit, - and just unref by 1 when we're ready to flag the object as orphaned */ + bit 0 : 1=Active / 0=Orphaned + bits 1-n : refcount + - ref/unref by two to avoid altering the orphaned bit + - To orphan, unref by 1 */ gpr_atm refst; gpr_mu mu; @@ -84,12 +84,11 @@ struct grpc_fd { /* Mutex protecting the 'polling_island' field */ gpr_mu pi_mu; - /* The polling island to which this fd belongs to. An fd belongs to exactly - one polling island */ + /* The polling island to which this fd belongs to. + * An fd belongs to exactly one polling island */ struct polling_island *polling_island; struct grpc_fd *freelist_next; - grpc_closure *on_done_closure; grpc_iomgr_object iomgr_object; @@ -141,7 +140,6 @@ typedef struct polling_island { /* Polling islands that are no longer needed are kept in a freelist so that they can be reused. This field points to the next polling island in the - free list. Note that this is only used if the polling island is in the free list */ struct polling_island *next_free; } polling_island; @@ -185,7 +183,7 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, * TODO: sreek - Might have to unref the fds (assuming whether we add a ref to * the fd when adding it to the epollset) */ /* The caller is expected to hold pi->mu lock before calling this function */ -static void polling_island_clear_fds_locked(polling_island *pi) { +static void polling_island_remove_all_fds_locked(polling_island *pi) { int err; size_t i; @@ -392,7 +390,7 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { // Move all the fds from polling_island p to polling_island q polling_island_add_fds_locked(q, p->fds, p->fd_cnt); - polling_island_clear_fds_locked(p); + polling_island_remove_all_fds_locked(p); q->ref_cnt += p->ref_cnt; @@ -411,14 +409,7 @@ static void polling_island_global_init() { * pollset declarations */ -typedef struct grpc_cached_wakeup_fd { - grpc_wakeup_fd fd; - struct grpc_cached_wakeup_fd *next; -} grpc_cached_wakeup_fd; - struct grpc_pollset_worker { - grpc_cached_wakeup_fd *wakeup_fd; - int reevaluate_polling_on_wakeup; int kicked_specifically; pthread_t pt_id; struct grpc_pollset_worker *next; @@ -441,9 +432,6 @@ struct grpc_pollset { /* The polling island to which this fd belongs to. An fd belongs to exactly one polling island */ struct polling_island *polling_island; - - /* Local cache of eventfds for workers */ - grpc_cached_wakeup_fd *local_wakeup_cache; }; /* Add an fd to a pollset */ @@ -465,8 +453,6 @@ static int poll_deadline_to_millis_timeout(gpr_timespec deadline, /* Allow kick to wakeup the currently polling worker */ #define GRPC_POLLSET_CAN_KICK_SELF 1 -/* Force the wakee to repoll when awoken */ -#define GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP 2 /* As per pollset_kick, with an extended set of flags (defined above) -- mostly for fd_posix's use. */ static void pollset_kick_ext(grpc_pollset *p, @@ -815,34 +801,25 @@ static void pollset_kick_ext(grpc_pollset *p, if (specific_worker != NULL) { if (specific_worker == GRPC_POLLSET_KICK_BROADCAST) { GPR_TIMER_BEGIN("pollset_kick_ext.broadcast", 0); - GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); for (specific_worker = p->root_worker.next; specific_worker != &p->root_worker; specific_worker = specific_worker->next) { - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + pthread_kill(specific_worker->pt_id, SIGUSR1); } p->kicked_without_pollers = 1; GPR_TIMER_END("pollset_kick_ext.broadcast", 0); } else if (gpr_tls_get(&g_current_thread_worker) != (intptr_t)specific_worker) { GPR_TIMER_MARK("different_thread_worker", 0); - if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { - specific_worker->reevaluate_polling_on_wakeup = 1; - } specific_worker->kicked_specifically = 1; - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); /* TODO (sreek): Refactor this into a separate file*/ pthread_kill(specific_worker->pt_id, SIGUSR1); } else if ((flags & GRPC_POLLSET_CAN_KICK_SELF) != 0) { GPR_TIMER_MARK("kick_yoself", 0); - if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { - specific_worker->reevaluate_polling_on_wakeup = 1; - } specific_worker->kicked_specifically = 1; - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + pthread_kill(specific_worker->pt_id, SIGUSR1); } } else if (gpr_tls_get(&g_current_thread_poller) != (intptr_t)p) { - GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); GPR_TIMER_MARK("kick_anonymous", 0); specific_worker = pop_front_worker(p); if (specific_worker != NULL) { @@ -860,7 +837,7 @@ static void pollset_kick_ext(grpc_pollset *p, if (specific_worker != NULL) { GPR_TIMER_MARK("finally_kick", 0); push_back_worker(p, specific_worker); - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); + pthread_kill(specific_worker->pt_id, SIGUSR1); } } else { GPR_TIMER_MARK("kicked_no_pollers", 0); @@ -911,8 +888,6 @@ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { pollset->shutting_down = 0; pollset->called_shutdown = 0; pollset->kicked_without_pollers = 0; - pollset->local_wakeup_cache = NULL; - pollset->kicked_without_pollers = 0; multipoll_with_epoll_pollset_create_efd(pollset); } @@ -926,12 +901,6 @@ static void pollset_destroy(grpc_pollset *pollset) { multipoll_with_epoll_pollset_destroy(pollset); - while (pollset->local_wakeup_cache) { - grpc_cached_wakeup_fd *next = pollset->local_wakeup_cache->next; - grpc_wakeup_fd_destroy(&pollset->local_wakeup_cache->fd); - gpr_free(pollset->local_wakeup_cache); - pollset->local_wakeup_cache = next; - } gpr_mu_destroy(&pollset->pi_mu); gpr_mu_destroy(&pollset->mu); } @@ -974,14 +943,6 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, GPR_TIMER_BEGIN("pollset_work", 0); /* this must happen before we (potentially) drop pollset->mu */ worker.next = worker.prev = NULL; - worker.reevaluate_polling_on_wakeup = 0; - if (pollset->local_wakeup_cache != NULL) { - worker.wakeup_fd = pollset->local_wakeup_cache; - pollset->local_wakeup_cache = worker.wakeup_fd->next; - } else { - worker.wakeup_fd = gpr_malloc(sizeof(*worker.wakeup_fd)); - grpc_wakeup_fd_init(&worker.wakeup_fd->fd); - } worker.kicked_specifically = 0; /* TODO(sreek): Abstract this thread id stuff out into a separate file */ @@ -1026,27 +987,12 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, gpr_mu_lock(&pollset->mu); locked = 1; } - /* If we're forced to re-evaluate polling (via pollset_kick with - GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) then we land here and force - a loop */ - if (worker.reevaluate_polling_on_wakeup) { - worker.reevaluate_polling_on_wakeup = 0; - pollset->kicked_without_pollers = 0; - if (queued_work || worker.kicked_specifically) { - /* If there's queued work on the list, then set the deadline to be - immediate so we get back out of the polling loop quickly */ - deadline = gpr_inf_past(GPR_CLOCK_MONOTONIC); - } - keep_polling = 1; - } } if (added_worker) { remove_worker(pollset, &worker); gpr_tls_set(&g_current_thread_worker, 0); } - /* release wakeup fd to the local pool */ - worker.wakeup_fd->next = pollset->local_wakeup_cache; - pollset->local_wakeup_cache = worker.wakeup_fd; + /* check shutdown conditions */ if (pollset->shutting_down) { if (pollset_has_workers(pollset)) { @@ -1135,10 +1081,9 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } } else if (fd->polling_island == NULL) { pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); - } else if (pollset->polling_island == NULL) { pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); - } else { // Non null and different + } else { pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); } @@ -1182,9 +1127,7 @@ static void multipoll_with_epoll_pollset_maybe_work_and_unlock( struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; int epoll_fd = pollset->epoll_fd; int ep_rv; - int poll_rv; int timeout_ms; - struct pollfd pfds[2]; /* If you want to ignore epoll's ability to sanely handle parallel pollers, * for a more apples-to-apples performance comparison with poll, add a @@ -1196,63 +1139,35 @@ static void multipoll_with_epoll_pollset_maybe_work_and_unlock( timeout_ms = poll_deadline_to_millis_timeout(deadline, now); - pfds[0].fd = GRPC_WAKEUP_FD_GET_READ_FD(&worker->wakeup_fd->fd); - pfds[0].events = POLLIN; - pfds[0].revents = 0; - pfds[1].fd = epoll_fd; - pfds[1].events = POLLIN; - pfds[1].revents = 0; - - /* TODO(vpai): Consider first doing a 0 timeout poll here to avoid - even going into the blocking annotation if possible */ - GPR_TIMER_BEGIN("poll", 0); - GRPC_SCHEDULING_START_BLOCKING_REGION; - poll_rv = grpc_poll_function(pfds, 2, timeout_ms); - GRPC_SCHEDULING_END_BLOCKING_REGION; - GPR_TIMER_END("poll", 0); - - if (poll_rv < 0) { - if (errno != EINTR) { - gpr_log(GPR_ERROR, "poll() failed: %s", strerror(errno)); - } - } else if (poll_rv == 0) { - /* do nothing */ - } else { - if (pfds[0].revents) { - grpc_wakeup_fd_consume_wakeup(&worker->wakeup_fd->fd); - } - if (pfds[1].revents) { - do { - /* The following epoll_wait never blocks; it has a timeout of 0 */ - ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); - if (ep_rv < 0) { - if (errno != EINTR) { - gpr_log(GPR_ERROR, "epoll_wait() failed: %s", strerror(errno)); - } + do { + /* The following epoll_wait never blocks; it has a timeout of 0 */ + ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms); + if (ep_rv < 0) { + if (errno != EINTR) { + gpr_log(GPR_ERROR, "epoll_wait() failed: %s", strerror(errno)); + } + } else { + int i; + for (i = 0; i < ep_rv; ++i) { + grpc_fd *fd = ep_ev[i].data.ptr; + /* TODO(klempner): We might want to consider making err and pri + * separate events */ + int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + int write_ev = ep_ev[i].events & EPOLLOUT; + if (fd == NULL) { + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); } else { - int i; - for (i = 0; i < ep_rv; ++i) { - grpc_fd *fd = ep_ev[i].data.ptr; - /* TODO(klempner): We might want to consider making err and pri - * separate events */ - int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); - int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); - int write_ev = ep_ev[i].events & EPOLLOUT; - if (fd == NULL) { - grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); - } else { - if (read_ev || cancel) { - fd_become_readable(exec_ctx, fd); - } - if (write_ev || cancel) { - fd_become_writable(exec_ctx, fd); - } - } + if (read_ev || cancel) { + fd_become_readable(exec_ctx, fd); + } + if (write_ev || cancel) { + fd_become_writable(exec_ctx, fd); } } - } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); + } } - } + } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); } static void multipoll_with_epoll_pollset_finish_shutdown( From 894900734662b7012d7b858db716c3cc1e9f8179 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 1 Jun 2016 08:52:43 -0700 Subject: [PATCH 026/280] Add a epoll_test.c file to experiment. REMOVE this from the final commit --- Makefile | 36 +++ build.yaml | 12 + test/core/network_benchmarks/epoll_test.c | 263 ++++++++++++++++++ .../network_benchmarks/low_level_ping_pong.c | 2 + tools/run_tests/sources_and_headers.json | 16 ++ tools/run_tests/tests.json | 15 + 6 files changed, 344 insertions(+) create mode 100644 test/core/network_benchmarks/epoll_test.c diff --git a/Makefile b/Makefile index 063698d943f..235f32d9a30 100644 --- a/Makefile +++ b/Makefile @@ -903,6 +903,7 @@ dns_resolver_connectivity_test: $(BINDIR)/$(CONFIG)/dns_resolver_connectivity_te dns_resolver_test: $(BINDIR)/$(CONFIG)/dns_resolver_test dualstack_socket_test: $(BINDIR)/$(CONFIG)/dualstack_socket_test endpoint_pair_test: $(BINDIR)/$(CONFIG)/endpoint_pair_test +epoll_test: $(BINDIR)/$(CONFIG)/epoll_test fd_conservation_posix_test: $(BINDIR)/$(CONFIG)/fd_conservation_posix_test fd_posix_test: $(BINDIR)/$(CONFIG)/fd_posix_test fling_client: $(BINDIR)/$(CONFIG)/fling_client @@ -1234,6 +1235,7 @@ buildtests_c: privatelibs_c \ $(BINDIR)/$(CONFIG)/dns_resolver_test \ $(BINDIR)/$(CONFIG)/dualstack_socket_test \ $(BINDIR)/$(CONFIG)/endpoint_pair_test \ + $(BINDIR)/$(CONFIG)/epoll_test \ $(BINDIR)/$(CONFIG)/fd_conservation_posix_test \ $(BINDIR)/$(CONFIG)/fd_posix_test \ $(BINDIR)/$(CONFIG)/fling_client \ @@ -1497,6 +1499,8 @@ test_c: buildtests_c $(Q) $(BINDIR)/$(CONFIG)/dualstack_socket_test || ( echo test dualstack_socket_test failed ; exit 1 ) $(E) "[RUN] Testing endpoint_pair_test" $(Q) $(BINDIR)/$(CONFIG)/endpoint_pair_test || ( echo test endpoint_pair_test failed ; exit 1 ) + $(E) "[RUN] Testing epoll_test" + $(Q) $(BINDIR)/$(CONFIG)/epoll_test || ( echo test epoll_test failed ; exit 1 ) $(E) "[RUN] Testing fd_conservation_posix_test" $(Q) $(BINDIR)/$(CONFIG)/fd_conservation_posix_test || ( echo test fd_conservation_posix_test failed ; exit 1 ) $(E) "[RUN] Testing fd_posix_test" @@ -6577,6 +6581,38 @@ endif endif +EPOLL_TEST_SRC = \ + test/core/network_benchmarks/epoll_test.c \ + +EPOLL_TEST_OBJS = $(addprefix $(OBJDIR)/$(CONFIG)/, $(addsuffix .o, $(basename $(EPOLL_TEST_SRC)))) +ifeq ($(NO_SECURE),true) + +# You can't build secure targets if you don't have OpenSSL. + +$(BINDIR)/$(CONFIG)/epoll_test: openssl_dep_error + +else + + + +$(BINDIR)/$(CONFIG)/epoll_test: $(EPOLL_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a + $(E) "[LD] Linking $@" + $(Q) mkdir -p `dirname $@` + $(Q) $(LD) $(LDFLAGS) $(EPOLL_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a $(LDLIBS) $(LDLIBS_SECURE) -o $(BINDIR)/$(CONFIG)/epoll_test + +endif + +$(OBJDIR)/$(CONFIG)/test/core/network_benchmarks/epoll_test.o: $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a + +deps_epoll_test: $(EPOLL_TEST_OBJS:.o=.dep) + +ifneq ($(NO_SECURE),true) +ifneq ($(NO_DEPS),true) +-include $(EPOLL_TEST_OBJS:.o=.dep) +endif +endif + + FD_CONSERVATION_POSIX_TEST_SRC = \ test/core/iomgr/fd_conservation_posix_test.c \ diff --git a/build.yaml b/build.yaml index 2f3d07071da..db9787546ad 100644 --- a/build.yaml +++ b/build.yaml @@ -1321,6 +1321,18 @@ targets: - grpc - gpr_test_util - gpr +- name: epoll_test + build: test + language: c + src: + - test/core/network_benchmarks/epoll_test.c + deps: + - grpc_test_util + - grpc + - gpr_test_util + - gpr + platforms: + - linux - name: fd_conservation_posix_test build: test language: c diff --git a/test/core/network_benchmarks/epoll_test.c b/test/core/network_benchmarks/epoll_test.c new file mode 100644 index 00000000000..a918dd9bb94 --- /dev/null +++ b/test/core/network_benchmarks/epoll_test.c @@ -0,0 +1,263 @@ +/* + * + * Copyright 2015, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +/* TODO: sreek: REMOVE THIS FILE */ + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +int g_signal_num = SIGUSR1; + +int g_timeout_secs = 2; + +int g_eventfd_create = 1; +int g_eventfd_wakeup = 0; +int g_eventfd_teardown = 0; +int g_close_epoll_fd = 1; + +typedef struct thread_args { + gpr_thd_id id; + int epoll_fd; + int thread_num; +} thread_args; + +static int eventfd_create() { + if (!g_eventfd_create) { + return -1; + } + + int efd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + GPR_ASSERT(efd >= 0); + return efd; +} + +static void eventfd_wakeup(int efd) { + if (!g_eventfd_wakeup) { + return; + } + + int err; + do { + err = eventfd_write(efd, 1); + } while (err < 0 && errno == EINTR); +} + +static void epoll_teardown(int epoll_fd, int fd) { + if (!g_eventfd_teardown) { + return; + } + + if (epoll_ctl(epoll_fd, EPOLL_CTL_DEL, fd, NULL) < 0) { + if (errno != ENOENT) { + gpr_log(GPR_ERROR, "epoll_ctl: %s", strerror(errno)); + GPR_ASSERT(0); + } + } +} + +/* Special case for epoll, where we need to create the fd ahead of time. */ +static int epoll_setup(int fd) { + int epoll_fd; + struct epoll_event ev; + + epoll_fd = epoll_create(1); + if (epoll_fd < 0) { + gpr_log(GPR_ERROR, "epoll_create: %s", strerror(errno)); + return -1; + } + + ev.events = (uint32_t)EPOLLIN; + ev.data.fd = fd; + if (epoll_ctl(epoll_fd, EPOLL_CTL_ADD, fd, &ev) < 0) { + if (errno != EEXIST) { + gpr_log(GPR_ERROR, "epoll_ctl: %s", strerror(errno)); + return -1; + } + + gpr_log(GPR_ERROR, "epoll_ctl: The fd %d already exists", fd); + } + + return epoll_fd; +} + +#define GRPC_EPOLL_MAX_EVENTS 1000 +static void thread_main(void *args) { + int ep_rv; + struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; + int fd; + int i; + int cancel; + int read; + int write; + thread_args *thd_args = args; + sigset_t new_mask; + sigset_t orig_mask; + int keep_polling = 0; + + gpr_log(GPR_INFO, "Thread: %d Started", thd_args->thread_num); + + do { + keep_polling = 0; + + /* Mask the signal before getting the epoll_fd */ + gpr_log(GPR_INFO, "Thread: %d Blocking signal: %d", thd_args->thread_num, + g_signal_num); + sigemptyset(&new_mask); + sigaddset(&new_mask, g_signal_num); + pthread_sigmask(SIG_BLOCK, &new_mask, &orig_mask); + + gpr_log(GPR_INFO, "Thread: %d Waiting on epoll_wait()", + thd_args->thread_num); + ep_rv = epoll_pwait(thd_args->epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, + g_timeout_secs * 5000, &orig_mask); + gpr_log(GPR_INFO, "Thread: %d out of epoll_wait. ep_rv = %d", + thd_args->thread_num, ep_rv); + + if (ep_rv < 0) { + if (errno != EINTR) { + gpr_log(GPR_ERROR, "Thread: %d. epoll_wait failed with error: %d", + thd_args->thread_num, errno); + } else { + gpr_log(GPR_INFO, + "Thread: %d. epoll_wait was interrupted. Polling again >>>>>>>", + thd_args->thread_num); + keep_polling = 1; + } + } else { + if (ep_rv == 0) { + gpr_log(GPR_INFO, + "Thread: %d - epoll_wait returned 0. Most likely a timeout. " + "Polling again", + thd_args->thread_num); + keep_polling = 1; + } + + for (i = 0; i < ep_rv; i++) { + fd = ep_ev[i].data.fd; + cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + read = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + write = ep_ev[i].events & EPOLLOUT; + gpr_log(GPR_INFO, + "Thread: %d. epoll_wait returned that fd: %d has event of " + "interest. read: %d, write: %d, cancel: %d", + thd_args->thread_num, fd, read, write, cancel); + } + } + } while (keep_polling); +} + +static void close_fd(int fd) { + if (!g_close_epoll_fd) { + return; + } + + gpr_log(GPR_INFO, "*** Closing fd : %d ****", fd); + close(fd); + gpr_log(GPR_INFO, "*** Closed fd : %d ****", fd); +} + + +static void sig_handler(int sig_num) { + gpr_log(GPR_INFO, "<<<<< Received signal %d", sig_num); +} + +static void set_signal_handler() { + gpr_log(GPR_INFO, "Setting signal handler"); + signal(g_signal_num, sig_handler); +} + +#define NUM_THREADS 2 +int main(int argc, char **argv) { + int efd; + int epoll_fd; + int i; + thread_args thd_args[NUM_THREADS]; + gpr_thd_options options = gpr_thd_options_default(); + + set_signal_handler(); + + gpr_log(GPR_INFO, "Starting.."); + efd = eventfd_create(); + gpr_log(GPR_INFO, "Created event fd: %d", efd); + epoll_fd = epoll_setup(efd); + gpr_log(GPR_INFO, "Created epoll_fd: %d", epoll_fd); + + gpr_thd_options_set_joinable(&options); + for (i = 0; i < NUM_THREADS; i++) { + thd_args[i].thread_num = i; + thd_args[i].epoll_fd = epoll_fd; + gpr_log(GPR_INFO, "Starting thread: %d", i); + gpr_thd_new(&thd_args[i].id, thread_main, &thd_args[i], &options); + } + + sleep((unsigned)g_timeout_secs * 2); + + /* Send signals first */ + for (i = 0; i < NUM_THREADS; i++) { + gpr_log(GPR_INFO, "Sending signal to thread: %d", thd_args->thread_num); + pthread_kill(thd_args[i].id, g_signal_num); + gpr_log(GPR_INFO, "Sent signal to thread: %d >>>>>> ", + thd_args->thread_num); + } + + sleep((unsigned)g_timeout_secs * 2); + + close_fd(epoll_fd); + + sleep((unsigned)g_timeout_secs * 2); + + eventfd_wakeup(efd); + epoll_teardown(epoll_fd, efd); + + for (i = 0; i < NUM_THREADS; i++) { + gpr_thd_join(thd_args[i].id); + gpr_log(GPR_INFO, "Thread: %d joined", i); + } + + return 0; +} diff --git a/test/core/network_benchmarks/low_level_ping_pong.c b/test/core/network_benchmarks/low_level_ping_pong.c index 1b40895a719..b72a07778e0 100644 --- a/test/core/network_benchmarks/low_level_ping_pong.c +++ b/test/core/network_benchmarks/low_level_ping_pong.c @@ -44,6 +44,7 @@ #include #include #include +#include #ifdef __linux__ #include #endif @@ -84,6 +85,7 @@ typedef struct thread_args { static int read_bytes(int fd, char *buf, size_t read_size, int spin) { size_t bytes_read = 0; ssize_t err; + do { err = read(fd, buf + bytes_read, read_size - bytes_read); if (err < 0) { diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index 97cc55db36e..85b71a8255a 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -301,6 +301,22 @@ "third_party": false, "type": "target" }, + { + "deps": [ + "gpr", + "gpr_test_util", + "grpc", + "grpc_test_util" + ], + "headers": [], + "language": "c", + "name": "epoll_test", + "src": [ + "test/core/network_benchmarks/epoll_test.c" + ], + "third_party": false, + "type": "target" + }, { "deps": [ "gpr", diff --git a/tools/run_tests/tests.json b/tools/run_tests/tests.json index 850f9474aec..be7b72f61d2 100644 --- a/tools/run_tests/tests.json +++ b/tools/run_tests/tests.json @@ -356,6 +356,21 @@ "windows" ] }, + { + "args": [], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "gtest": false, + "language": "c", + "name": "epoll_test", + "platforms": [ + "linux" + ] + }, { "args": [], "ci_platforms": [ From 9b9708a9c484c02ad8ef27060370ae605a83942c Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Wed, 1 Jun 2016 11:42:20 -0700 Subject: [PATCH 027/280] Allow Node users to set a custom logger and log verbosity. Defaults to existing core logger --- src/node/ext/node_grpc.cc | 117 ++++++++++++++++++++++++++++++++++++ src/node/index.js | 33 ++++++++++ src/node/src/common.js | 21 +++++++ src/node/src/credentials.js | 4 +- 4 files changed, 174 insertions(+), 1 deletion(-) diff --git a/src/node/ext/node_grpc.cc b/src/node/ext/node_grpc.cc index 6b6e42737b5..45762cad2e6 100644 --- a/src/node/ext/node_grpc.cc +++ b/src/node/ext/node_grpc.cc @@ -31,12 +31,15 @@ * */ +#include + #include #include #include #include "grpc/grpc.h" #include "grpc/grpc_security.h" #include "grpc/support/alloc.h" +#include "grpc/support/log.h" #include "call.h" #include "call_credentials.h" @@ -49,10 +52,22 @@ using v8::FunctionTemplate; using v8::Local; using v8::Value; +using v8::Number; using v8::Object; using v8::Uint32; using v8::String; +typedef struct logger_state { + Nan::Callback *callback; + std::list *pending_args; + uv_mutex_t mutex; + uv_async_t async; + // Indicates that a logger has been set + bool logger_set; +} logger_state; + +logger_state grpc_logger_state; + static char *pem_root_certs = NULL; void InitStatusConstants(Local exports) { @@ -235,6 +250,18 @@ void InitWriteFlags(Local exports) { Nan::Set(write_flags, Nan::New("NO_COMPRESS").ToLocalChecked(), NO_COMPRESS); } +void InitLogConstants(Local exports) { + Nan::HandleScope scope; + Local log_verbosity = Nan::New(); + Nan::Set(exports, Nan::New("logVerbosity").ToLocalChecked(), log_verbosity); + Local DEBUG(Nan::New(GPR_LOG_SEVERITY_DEBUG)); + Nan::Set(log_verbosity, Nan::New("DEBUG").ToLocalChecked(), DEBUG); + Local INFO(Nan::New(GPR_LOG_SEVERITY_INFO)); + Nan::Set(log_verbosity, Nan::New("INFO").ToLocalChecked(), INFO); + Local ERROR(Nan::New(GPR_LOG_SEVERITY_ERROR)); + Nan::Set(log_verbosity, Nan::New("ERROR").ToLocalChecked(), ERROR); +} + NAN_METHOD(MetadataKeyIsLegal) { if (!info[0]->IsString()) { return Nan::ThrowTypeError( @@ -298,16 +325,98 @@ NAN_METHOD(SetDefaultRootsPem) { } } +NAUV_WORK_CB(LogMessagesCallback) { + Nan::HandleScope scope; + std::list args; + uv_mutex_lock(&grpc_logger_state.mutex); + args.splice(args.begin(), *grpc_logger_state.pending_args); + uv_mutex_unlock(&grpc_logger_state.mutex); + /* Call the callback with each log message */ + while (!args.empty()) { + gpr_log_func_args *arg = args.front(); + args.pop_front(); + Local file = Nan::New(arg->file).ToLocalChecked(); + Local line = Nan::New(arg->line); + Local severity = Nan::New( + gpr_log_severity_string(arg->severity)).ToLocalChecked(); + Local message = Nan::New(arg->message).ToLocalChecked(); + const int argc = 4; + Local argv[argc] = {file, line, severity, message}; + grpc_logger_state.callback->Call(argc, argv); + delete[] arg->message; + delete arg; + } +} + +void node_log_func(gpr_log_func_args *args) { + // TODO(mlumish): Use the core's log formatter when it becomes available + gpr_log_func_args *args_copy = new gpr_log_func_args; + size_t message_len = strlen(args->message) + 1; + char *message = new char[message_len]; + memcpy(message, args->message, message_len); + memcpy(args_copy, args, sizeof(gpr_log_func_args)); + args_copy->message = message; + + uv_mutex_lock(&grpc_logger_state.mutex); + grpc_logger_state.pending_args->push_back(args_copy); + uv_mutex_unlock(&grpc_logger_state.mutex); + + uv_async_send(&grpc_logger_state.async); +} + +void init_logger() { + memset(&grpc_logger_state, 0, sizeof(logger_state)); + grpc_logger_state.pending_args = new std::list(); + uv_mutex_init(&grpc_logger_state.mutex); + uv_async_init(uv_default_loop(), + &grpc_logger_state.async, + LogMessagesCallback); + uv_unref((uv_handle_t*)&grpc_logger_state.async); + grpc_logger_state.logger_set = false; + + gpr_log_verbosity_init(); +} + +/* This registers a JavaScript logger for messages from the gRPC core. Because + that handler has to be run in the context of the JavaScript event loop, it + will be run asynchronously. To minimize the problems that could cause for + debugging, we leave core to do its default synchronous logging until a + JavaScript logger is set */ +NAN_METHOD(SetDefaultLoggerCallback) { + if (!info[0]->IsFunction()) { + return Nan::ThrowTypeError( + "setDefaultLoggerCallback's argument must be a function"); + } + if (!grpc_logger_state.logger_set) { + gpr_set_log_function(node_log_func); + grpc_logger_state.logger_set = true; + } + grpc_logger_state.callback = new Nan::Callback(info[0].As()); +} + +NAN_METHOD(SetLogVerbosity) { + if (!info[0]->IsUint32()) { + return Nan::ThrowTypeError( + "setLogVerbosity's argument must be a number"); + } + gpr_log_severity severity = static_cast( + Nan::To(info[0]).FromJust()); + gpr_set_log_verbosity(severity); +} + void init(Local exports) { Nan::HandleScope scope; grpc_init(); grpc_set_ssl_roots_override_callback(get_ssl_roots_override); + init_logger(); + InitStatusConstants(exports); InitCallErrorConstants(exports); InitOpTypeConstants(exports); InitPropagateConstants(exports); InitConnectivityStateConstants(exports); InitWriteFlags(exports); + InitLogConstants(exports); grpc::node::Call::Init(exports); grpc::node::CallCredentials::Init(exports); @@ -333,6 +442,14 @@ void init(Local exports) { Nan::GetFunction( Nan::New(SetDefaultRootsPem) ).ToLocalChecked()); + Nan::Set(exports, Nan::New("setDefaultLoggerCallback").ToLocalChecked(), + Nan::GetFunction( + Nan::New(SetDefaultLoggerCallback) + ).ToLocalChecked()); + Nan::Set(exports, Nan::New("setLogVerbosity").ToLocalChecked(), + Nan::GetFunction( + Nan::New(SetLogVerbosity) + ).ToLocalChecked()); } NODE_MODULE(grpc_node, init) diff --git a/src/node/index.js b/src/node/index.js index 66664d94b5a..b85cec68414 100644 --- a/src/node/index.js +++ b/src/node/index.js @@ -46,6 +46,8 @@ var client = require('./src/client.js'); var server = require('./src/server.js'); +var common = require('./src/common.js'); + var Metadata = require('./src/metadata.js'); var grpc = require('./src/grpc_extension'); @@ -122,6 +124,32 @@ exports.load = function load(filename, format, options) { return loadObject(builder.ns, options); }; +/** + * Sets the logger function for the gRPC module. For debugging purposes, the C + * core will log synchronously directly to stdout unless this function is + * called. Note: the output format here is intended to be informational, and + * is not guaranteed to stay the same in the future. + * Logs will be directed to logger.error. + * @param {Console} logger A Console-like object. + */ +exports.setLogger = function setLogger(logger) { + common.logger = logger; + grpc.setDefaultLoggerCallback(function(file, line, severity, message) { + file = path.basename(file); + logger.error(severity + '\t' + file + ':' + line + ']\t' + message); + }); +}; + +/** + * Sets the logger verbosity for gRPC module logging. The options are members + * of the grpc.logVerbosity map. + * @param {Number} verbosity The minimum severity to log + */ +exports.setLogVerbosity = function setLogVerbosity(verbosity) { + common.logVerbosity = verbosity; + grpc.setLogVerbosity(verbosity); +}; + /** * @see module:src/server.Server */ @@ -152,6 +180,11 @@ exports.callError = grpc.callError; */ exports.writeFlags = grpc.writeFlags; +/** + * Log verbosity setting name to code number mapping + */ +exports.logVerbosity = grpc.logVerbosity; + /** * Credentials factories */ diff --git a/src/node/src/common.js b/src/node/src/common.js index 8cf43b7a84c..e2d1a50df4a 100644 --- a/src/node/src/common.js +++ b/src/node/src/common.js @@ -157,3 +157,24 @@ exports.getProtobufServiceAttrs = function getProtobufServiceAttrs(service, }]; })); }; + +/** + * The logger object for the gRPC module. Defaults to console. + */ +exports.logger = console; + +/** + * The current logging verbosity. UNSET corresponds to logging everything + */ +exports.logVerbosity = -1; + +/** + * Log a message if the severity is at least as high as the current verbosity + * @param {Number} severity A value of the grpc.logVerbosity map + * @param {String} message The message to log + */ +exports.log = function log(severity, message) { + if (severity >= exports.logVerbosity) { + exports.logger.error(message); + } +}; diff --git a/src/node/src/credentials.js b/src/node/src/credentials.js index a12eade4e1f..b746d0625df 100644 --- a/src/node/src/credentials.js +++ b/src/node/src/credentials.js @@ -69,6 +69,8 @@ var ChannelCredentials = grpc.ChannelCredentials; var Metadata = require('./metadata.js'); +var common = require('./common.js'); + /** * Create an SSL Credentials object. If using a client-side certificate, both * the second and third arguments must be passed. @@ -120,7 +122,7 @@ exports.createFromGoogleCredential = function(google_credential) { var service_url = auth_context.service_url; google_credential.getRequestMetadata(service_url, function(err, header) { if (err) { - console.log('Auth error:', err); + common.log(grpc.logVerbosity.INFO, 'Auth error:' + err); callback(err); return; } From 4a43fe4a2fc62ab179e6fff7ca10f2c9c13dd1fc Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Wed, 1 Jun 2016 16:05:48 -0700 Subject: [PATCH 028/280] Minor change to common.js --- src/node/src/common.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/node/src/common.js b/src/node/src/common.js index e2d1a50df4a..22159dd39f7 100644 --- a/src/node/src/common.js +++ b/src/node/src/common.js @@ -164,9 +164,9 @@ exports.getProtobufServiceAttrs = function getProtobufServiceAttrs(service, exports.logger = console; /** - * The current logging verbosity. UNSET corresponds to logging everything + * The current logging verbosity. 0 corresponds to logging everything */ -exports.logVerbosity = -1; +exports.logVerbosity = 0; /** * Log a message if the severity is at least as high as the current verbosity From 1d2f28913e93481cc4b425c608dc7c59ebe026e4 Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Thu, 2 Jun 2016 14:33:22 -0700 Subject: [PATCH 029/280] Add timestamps to custom log output --- src/node/ext/node_grpc.cc | 38 ++++++++++++++++++++++++-------------- src/node/index.js | 16 +++++++++++++--- 2 files changed, 37 insertions(+), 17 deletions(-) diff --git a/src/node/ext/node_grpc.cc b/src/node/ext/node_grpc.cc index 45762cad2e6..6daa1039342 100644 --- a/src/node/ext/node_grpc.cc +++ b/src/node/ext/node_grpc.cc @@ -40,6 +40,7 @@ #include "grpc/grpc_security.h" #include "grpc/support/alloc.h" #include "grpc/support/log.h" +#include "grpc/support/time.h" #include "call.h" #include "call_credentials.h" @@ -48,6 +49,7 @@ #include "server.h" #include "completion_queue_async_worker.h" #include "server_credentials.h" +#include "timeval.h" using v8::FunctionTemplate; using v8::Local; @@ -57,9 +59,14 @@ using v8::Object; using v8::Uint32; using v8::String; +typedef struct log_args { + gpr_log_func_args core_args; + gpr_timespec timestamp; +} log_args; + typedef struct logger_state { Nan::Callback *callback; - std::list *pending_args; + std::list *pending_args; uv_mutex_t mutex; uv_async_t async; // Indicates that a logger has been set @@ -327,35 +334,38 @@ NAN_METHOD(SetDefaultRootsPem) { NAUV_WORK_CB(LogMessagesCallback) { Nan::HandleScope scope; - std::list args; + std::list args; uv_mutex_lock(&grpc_logger_state.mutex); args.splice(args.begin(), *grpc_logger_state.pending_args); uv_mutex_unlock(&grpc_logger_state.mutex); /* Call the callback with each log message */ while (!args.empty()) { - gpr_log_func_args *arg = args.front(); + log_args *arg = args.front(); args.pop_front(); - Local file = Nan::New(arg->file).ToLocalChecked(); - Local line = Nan::New(arg->line); + Local file = Nan::New(arg->core_args.file).ToLocalChecked(); + Local line = Nan::New(arg->core_args.line); Local severity = Nan::New( - gpr_log_severity_string(arg->severity)).ToLocalChecked(); - Local message = Nan::New(arg->message).ToLocalChecked(); - const int argc = 4; - Local argv[argc] = {file, line, severity, message}; + gpr_log_severity_string(arg->core_args.severity)).ToLocalChecked(); + Local message = Nan::New(arg->core_args.message).ToLocalChecked(); + Local timestamp = Nan::New( + grpc::node::TimespecToMilliseconds(arg->timestamp)).ToLocalChecked(); + const int argc = 5; + Local argv[argc] = {file, line, severity, message, timestamp}; grpc_logger_state.callback->Call(argc, argv); - delete[] arg->message; + delete[] arg->core_args.message; delete arg; } } void node_log_func(gpr_log_func_args *args) { // TODO(mlumish): Use the core's log formatter when it becomes available - gpr_log_func_args *args_copy = new gpr_log_func_args; + log_args *args_copy = new log_args; size_t message_len = strlen(args->message) + 1; char *message = new char[message_len]; memcpy(message, args->message, message_len); - memcpy(args_copy, args, sizeof(gpr_log_func_args)); - args_copy->message = message; + memcpy(&args_copy->core_args, args, sizeof(gpr_log_func_args)); + args_copy->core_args.message = message; + args_copy->timestamp = gpr_now(GPR_CLOCK_REALTIME); uv_mutex_lock(&grpc_logger_state.mutex); grpc_logger_state.pending_args->push_back(args_copy); @@ -366,7 +376,7 @@ void node_log_func(gpr_log_func_args *args) { void init_logger() { memset(&grpc_logger_state, 0, sizeof(logger_state)); - grpc_logger_state.pending_args = new std::list(); + grpc_logger_state.pending_args = new std::list(); uv_mutex_init(&grpc_logger_state.mutex); uv_async_init(uv_default_loop(), &grpc_logger_state.async, diff --git a/src/node/index.js b/src/node/index.js index b85cec68414..9fb6faa5d7c 100644 --- a/src/node/index.js +++ b/src/node/index.js @@ -124,6 +124,10 @@ exports.load = function load(filename, format, options) { return loadObject(builder.ns, options); }; +var log_template = _.template( + '{severity} {timestamp}\t{file}:{line}]\t{message}', + {interpolate: /{([\s\S]+?)}/g}); + /** * Sets the logger function for the gRPC module. For debugging purposes, the C * core will log synchronously directly to stdout unless this function is @@ -134,9 +138,15 @@ exports.load = function load(filename, format, options) { */ exports.setLogger = function setLogger(logger) { common.logger = logger; - grpc.setDefaultLoggerCallback(function(file, line, severity, message) { - file = path.basename(file); - logger.error(severity + '\t' + file + ':' + line + ']\t' + message); + grpc.setDefaultLoggerCallback(function(file, line, severity, + message, timestamp) { + logger.error(log_template({ + file: path.basename(file), + line: line, + severity: severity, + message: message, + timestamp: timestamp.toISOString() + })); }); }; From 0bcbd79baa57c16d4ab64a070b5fbfc93293f543 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 1 Jun 2016 15:43:03 -0700 Subject: [PATCH 030/280] Functionality complete in ev_epoll_linux.c --- src/core/lib/iomgr/ev_epoll_linux.c | 1202 +++++++++++++-------------- 1 file changed, 591 insertions(+), 611 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 1201c10a7e0..ce42a9e7ce6 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -61,31 +61,27 @@ struct polling_island; /******************************************************************************* - * FD declarations + * Fd Declarations */ - struct grpc_fd { int fd; /* refst format: bit 0 : 1=Active / 0=Orphaned bits 1-n : refcount - - ref/unref by two to avoid altering the orphaned bit - - To orphan, unref by 1 */ + Ref/Unref by two to avoid altering the orphaned bit */ gpr_atm refst; gpr_mu mu; - int shutdown; + bool shutdown; int closed; - int released; + bool released; grpc_closure *read_closure; grpc_closure *write_closure; - /* Mutex protecting the 'polling_island' field */ + /* The polling island to which this fd belongs to and the mutex protecting the + the field */ gpr_mu pi_mu; - - /* The polling island to which this fd belongs to. - * An fd belongs to exactly one polling island */ struct polling_island *polling_island; struct grpc_fd *freelist_next; @@ -94,11 +90,7 @@ struct grpc_fd { grpc_iomgr_object iomgr_object; }; -/* Return 1 if this fd is orphaned, 0 otherwise */ -static bool fd_is_orphaned(grpc_fd *fd); - /* Reference counting for fds */ -/*#define GRPC_FD_REF_COUNT_DEBUG*/ #ifdef GRPC_FD_REF_COUNT_DEBUG static void fd_ref(grpc_fd *fd, const char *reason, const char *file, int line); static void fd_unref(grpc_fd *fd, const char *reason, const char *file, @@ -119,15 +111,15 @@ static void fd_global_shutdown(void); #define CLOSURE_READY ((grpc_closure *)1) /******************************************************************************* - * Polling Island + * Polling-island Declarations */ typedef struct polling_island { gpr_mu mu; int ref_cnt; - /* Pointer to the polling_island this merged into. If this is not NULL, all - the remaining fields in this pollset (i.e all fields except mu and ref_cnt) - are considered invalid and must be ignored */ + /* Points to the polling_island this merged into. + * If merged_to is not NULL, all the remaining fields (except mu and ref_cnt) + * are invalid and must be ignored */ struct polling_island *merged_to; /* The fd of the underlying epoll set */ @@ -144,15 +136,62 @@ typedef struct polling_island { struct polling_island *next_free; } polling_island; +/******************************************************************************* + * Pollset Declarations + */ + +struct grpc_pollset_worker { + int kicked_specifically; + pthread_t pt_id; /* TODO (sreek) - Add an abstraction here */ + struct grpc_pollset_worker *next; + struct grpc_pollset_worker *prev; +}; + +struct grpc_pollset { + gpr_mu mu; + grpc_pollset_worker root_worker; + bool kicked_without_pollers; + + bool shutting_down; /* Is the pollset shutting down ? */ + bool finish_shutdown_called; /* Is the 'finish_shutdown_locked()' called ? */ + grpc_closure *shutdown_done; /* Called after after shutdown is complete */ + + /* The polling island to which this pollset belongs to and the mutex + protecting the field */ + gpr_mu pi_mu; + struct polling_island *polling_island; +}; + +/******************************************************************************* + * Pollset-set Declarations + */ +struct grpc_pollset_set { + gpr_mu mu; + + size_t pollset_count; + size_t pollset_capacity; + grpc_pollset **pollsets; + + size_t pollset_set_count; + size_t pollset_set_capacity; + struct grpc_pollset_set **pollset_sets; + + size_t fd_count; + size_t fd_capacity; + grpc_fd **fds; +}; + +/******************************************************************************* + * Polling-island Definitions + */ + /* Polling island freelist */ static gpr_mu g_pi_freelist_mu; static polling_island *g_pi_freelist = NULL; -/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? - * TODO: sreek - Should this add a ref to the grpc_fd ? */ /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, - size_t fd_count) { + size_t fd_count, bool add_fd_refs) { int err; size_t i; struct epoll_event ev; @@ -162,11 +201,14 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, ev.data.ptr = fds[i]; err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, fds[i]->fd, &ev); - if (err < 0 && errno != EEXIST) { - gpr_log(GPR_ERROR, "epoll_ctl add for fd: %d failed with error: %s", - fds[i]->fd, strerror(errno)); - /* TODO: sreek - Not sure if it is a good idea to continue here. We need a - * better way to bubble up this error instead of doing an abort() */ + if (err < 0) { + if (errno != EEXIST) { + /* TODO: sreek - We need a better way to bubble up this error instead of + just logging a message */ + gpr_log(GPR_ERROR, "epoll_ctl add for fd: %d failed with error: %s", + fds[i]->fd, strerror(errno)); + } + continue; } @@ -176,26 +218,30 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, } pi->fds[pi->fd_cnt++] = fds[i]; + if (add_fd_refs) { + GRPC_FD_REF(fds[i], "polling_island"); + } } } -/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? - * TODO: sreek - Might have to unref the fds (assuming whether we add a ref to - * the fd when adding it to the epollset) */ /* The caller is expected to hold pi->mu lock before calling this function */ -static void polling_island_remove_all_fds_locked(polling_island *pi) { +static void polling_island_remove_all_fds_locked(polling_island *pi, + bool remove_fd_refs) { int err; size_t i; for (i = 0; i < pi->fd_cnt; i++) { - err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, pi->fds[i]->fd, NULL); + if (remove_fd_refs) { + GRPC_FD_UNREF(pi->fds[i], "polling_island"); + } + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, pi->fds[i]->fd, NULL); if (err < 0 && errno != ENOENT) { gpr_log(GPR_ERROR, "epoll_ctl delete for fds[i]: %d failed with error: %s", i, pi->fds[i]->fd, strerror(errno)); - /* TODO: sreek - Not sure if it is a good idea to continue here. We need a - * better way to bubble up this error instead of doing an abort() */ + /* TODO: sreek - We need a better way to bubble up this error instead of + * just logging a message */ continue; } } @@ -203,22 +249,31 @@ static void polling_island_remove_all_fds_locked(polling_island *pi) { pi->fd_cnt = 0; } -/* TODO: sreek - Should we hold a lock on fd or add a ref to the fd ? - * TODO: sreek - Might have to unref the fd (assuming whether we add a ref to - * the fd when adding it to the epollset) */ /* The caller is expected to hold pi->mu lock before calling this function */ -static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd) { +static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, + bool close_fd, bool remove_fd_ref) { int err; size_t i; - err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, fd->fd, NULL); - if (err < 0 && errno != ENOENT) { - gpr_log(GPR_ERROR, "epoll_ctl delete for fd: %d failed with error; %s", - fd->fd, strerror(errno)); + + /* Calling close() on the fd will automatically remove it from the epoll set. + If not calling close(), the fd must be explicitly removed from the epoll + set */ + if (close_fd) { + close(fd->fd); + } else { + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, fd->fd, NULL); + if (err < 0 && errno != ENOENT) { + gpr_log(GPR_ERROR, "epoll_ctl delete for fd: %d failed with error; %s", + fd->fd, strerror(errno)); + } } for (i = 0; i < pi->fd_cnt; i++) { if (pi->fds[i] == fd) { pi->fds[i] = pi->fds[--pi->fd_cnt]; + if (remove_fd_ref) { + GRPC_FD_UNREF(fd, "polling_island"); + } break; } } @@ -227,6 +282,10 @@ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd) { static polling_island *polling_island_create(grpc_fd *initial_fd, int initial_ref_cnt) { polling_island *pi = NULL; + struct epoll_event ev; + int err; + + /* Try to get one from the polling island freelist */ gpr_mu_lock(&g_pi_freelist_mu); if (g_pi_freelist != NULL) { pi = g_pi_freelist; @@ -242,13 +301,25 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, pi->fd_cnt = 0; pi->fd_capacity = 0; pi->fds = NULL; + } - pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); - if (pi->epoll_fd < 0) { - gpr_log(GPR_ERROR, "epoll_create1() failed with error: %s", - strerror(errno)); - } - GPR_ASSERT(pi->epoll_fd >= 0); + pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); + if (pi->epoll_fd < 0) { + gpr_log(GPR_ERROR, "epoll_create1() failed with error: %s", + strerror(errno)); + } + GPR_ASSERT(pi->epoll_fd >= 0); + + ev.events = (uint32_t)(EPOLLIN | EPOLLET); + ev.data.ptr = NULL; + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); + if (err < 0) { + gpr_log(GPR_ERROR, + "Failed to add grpc_global_wake_up_fd (%d) to the epoll set " + "(epoll_fd: %d) with error: %s", + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), pi->epoll_fd, + strerror(errno)); } pi->ref_cnt = initial_ref_cnt; @@ -256,10 +327,12 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, pi->next_free = NULL; if (initial_fd != NULL) { - /* polling_island_add_fds_locked() expects the caller to hold a pi->mu - * lock. However, since this is a new polling island (and no one has a - * reference to it yet), it is okay to not acquire pi->mu here */ - polling_island_add_fds_locked(pi, &initial_fd, 1); + /* It is not really needed to get the pi->mu lock here. If this is a newly + created polling island (or one that we got from the freelist), no one + else would be holding a lock to it anyway */ + gpr_mu_lock(&pi->mu); + polling_island_add_fds_locked(pi, &initial_fd, 1, true); + gpr_mu_unlock(&pi->mu); } return pi; @@ -269,6 +342,9 @@ static void polling_island_delete(polling_island *pi) { GPR_ASSERT(pi->ref_cnt == 0); GPR_ASSERT(pi->fd_cnt == 0); + close(pi->epoll_fd); + pi->epoll_fd = -1; + pi->merged_to = NULL; gpr_mu_lock(&g_pi_freelist_mu); @@ -313,10 +389,20 @@ void polling_island_pair_update_and_lock(polling_island **p, bool pi_2_locked = false; int num_swaps = 0; + /* Loop until either pi_1 == pi_2 or until we acquired locks on both pi_1 + and pi_2 */ while (pi_1 != pi_2 && !(pi_1_locked && pi_2_locked)) { - // pi_1 is NOT equal to pi_2 - // pi_1 MAY be locked - + /* The following assertions are true at this point: + - pi_1 != pi_2 (else, the while loop would have exited) + - pi_1 MAY be locked + - pi_2 is NOT locked */ + + /* To maintain lock order consistency, always lock polling_island node with + lower address first. + First, make sure pi_1 < pi_2 before proceeding any further. If it turns + out that pi_1 > pi_2, unlock pi_1 if locked (because pi_2 is not locked + at this point and having pi_1 locked would violate the lock order) and + swap pi_1 and pi_2 so that pi_1 becomes less than pi_2 */ if (pi_1 > pi_2) { if (pi_1_locked) { gpr_mu_unlock(&pi_1->mu); @@ -327,14 +413,22 @@ void polling_island_pair_update_and_lock(polling_island **p, num_swaps++; } - // p1 < p2 - // p1 MAY BE locked - // p2 is NOT locked + /* The following assertions are true at this point: + - pi_1 != pi_2 + - pi_1 < pi_2 (address of pi_1 is less than that of pi_2) + - pi_1 MAYBE locked + - pi_2 is NOT locked */ + /* Lock pi_1 (if pi_1 is pointing to the terminal node in the list) */ if (!pi_1_locked) { gpr_mu_lock(&pi_1->mu); pi_1_locked = true; + /* If pi_1 is not terminal node (i.e pi_1->merged_to != NULL), we are not + done locking this polling_island yet. Release the lock on this node and + advance pi_1 to the next node in the list; and go to the beginning of + the loop (we can't proceed to locking pi_2 unless we locked pi_1 first) + */ if (pi_1->merged_to != NULL) { temp = pi_1->merged_to; polling_island_unref_and_unlock(pi_1, 1); @@ -345,13 +439,16 @@ void polling_island_pair_update_and_lock(polling_island **p, } } - // p1 is LOCKED - // p2 is UNLOCKED - // p1 != p2 + /* The following assertions are true at this point: + - pi_1 is locked + - pi_2 is unlocked + - pi_1 != pi_2 */ gpr_mu_lock(&pi_2->mu); pi_2_locked = true; + /* If pi_2 is not terminal node, we are not done locking this polling_island + yet. Release the lock and update pi_2 to the next node in the list */ if (pi_2->merged_to != NULL) { temp = pi_2->merged_to; polling_island_unref_and_unlock(pi_2, 1); @@ -360,14 +457,19 @@ void polling_island_pair_update_and_lock(polling_island **p, } } - // Either pi_1 == pi_2 OR we got both locks! + /* At this point, either pi_1 == pi_2 AND/OR we got both locks */ if (pi_1 == pi_2) { + /* We may or may not have gotten the lock. If we didn't, walk the rest of + the polling_island list and get the lock */ GPR_ASSERT(pi_1_locked || (!pi_1_locked && !pi_2_locked)); if (!pi_1_locked) { pi_1 = pi_2 = polling_island_update_and_lock(pi_1, 2, 0); } } else { GPR_ASSERT(pi_1_locked && pi_2_locked); + /* If we swapped pi_1 and pi_2 odd number of times, do one more swap so that + pi_1 and pi_2 point to the same polling_island lists they started off + with at the beginning of this function (i.e *p and *q respectively) */ if (num_swaps % 2 > 0) { GPR_SWAP(polling_island *, pi_1, pi_2); } @@ -378,26 +480,37 @@ void polling_island_pair_update_and_lock(polling_island **p, } polling_island *polling_island_merge(polling_island *p, polling_island *q) { - polling_island *merged = NULL; - + /* Get locks on both the polling islands */ polling_island_pair_update_and_lock(&p, &q); /* TODO: sreek: Think about this scenario some more. Is it possible ?. what * does it mean, when would this happen */ if (p == q) { - merged = p; + /* Nothing needs to be done here */ + gpr_mu_unlock(&p->mu); + return p; } - // Move all the fds from polling_island p to polling_island q - polling_island_add_fds_locked(q, p->fds, p->fd_cnt); - polling_island_remove_all_fds_locked(p); + /* Make sure that p points to the polling island with fewer fds than q */ + if (p->fd_cnt > q->fd_cnt) { + GPR_SWAP(polling_island *, p, q); + } + + /* "Merge" p with q i.e move all the fds from p (the polling_island with fewer + fds) to q. + Note: Not altering the ref counts on the affected fds here because they + would effectively remain unchanged */ + polling_island_add_fds_locked(q, p->fds, p->fd_cnt, false); + polling_island_remove_all_fds_locked(p, false); + /* The merged polling island inherits all the ref counts of the island merging + with it */ q->ref_cnt += p->ref_cnt; gpr_mu_unlock(&p->mu); gpr_mu_unlock(&q->mu); - return merged; + return q; } static void polling_island_global_init() { @@ -406,95 +519,10 @@ static void polling_island_global_init() { } /******************************************************************************* - * pollset declarations - */ - -struct grpc_pollset_worker { - int kicked_specifically; - pthread_t pt_id; - struct grpc_pollset_worker *next; - struct grpc_pollset_worker *prev; -}; - -struct grpc_pollset { - gpr_mu mu; - grpc_pollset_worker root_worker; - int shutting_down; - int called_shutdown; - int kicked_without_pollers; - grpc_closure *shutdown_done; - - int epoll_fd; - - /* Mutex protecting the 'polling_island' field */ - gpr_mu pi_mu; - - /* The polling island to which this fd belongs to. An fd belongs to exactly - one polling island */ - struct polling_island *polling_island; -}; - -/* Add an fd to a pollset */ -static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - struct grpc_fd *fd); - -static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, grpc_fd *fd); - -/* Convert a timespec to milliseconds: - - very small or negative poll times are clamped to zero to do a - non-blocking poll (which becomes spin polling) - - other small values are rounded up to one millisecond - - longer than a millisecond polls are rounded up to the next nearest - millisecond to avoid spinning - - infinite timeouts are converted to -1 */ -static int poll_deadline_to_millis_timeout(gpr_timespec deadline, - gpr_timespec now); - -/* Allow kick to wakeup the currently polling worker */ -#define GRPC_POLLSET_CAN_KICK_SELF 1 -/* As per pollset_kick, with an extended set of flags (defined above) - -- mostly for fd_posix's use. */ -static void pollset_kick_ext(grpc_pollset *p, - grpc_pollset_worker *specific_worker, - uint32_t flags); - -/* turn a pollset into a multipoller: platform specific */ -typedef void (*platform_become_multipoller_type)(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, - struct grpc_fd **fds, - size_t fd_count); - -/* Return 1 if the pollset has active threads in pollset_work (pollset must - * be locked) */ -static int pollset_has_workers(grpc_pollset *pollset); - -/******************************************************************************* - * pollset_set definitions - */ - -struct grpc_pollset_set { - gpr_mu mu; - - size_t pollset_count; - size_t pollset_capacity; - grpc_pollset **pollsets; - - size_t pollset_set_count; - size_t pollset_set_capacity; - struct grpc_pollset_set **pollset_sets; - - size_t fd_count; - size_t fd_capacity; - grpc_fd **fds; -}; - -/******************************************************************************* - * fd_posix.c + * Fd Definitions */ -/* We need to keep a freelist not because of any concerns of malloc - * performance +/* We need to keep a freelist not because of any concerns of malloc performance * but instead so that implementations with multiple threads in (for example) * epoll_wait deal with the race between pollset removal and incoming poll * notifications. @@ -506,58 +534,16 @@ struct grpc_pollset_set { * If we keep the object freelisted, in the worst case losing this race just * becomes a spurious read notification on a reused fd. */ -/* TODO(klempner): We could use some form of polling generation count to know - * when these are safe to free. */ -/* TODO(klempner): Consider disabling freelisting if we don't have multiple - * threads in poll on the same fd */ -/* TODO(klempner): Batch these allocations to reduce fragmentation */ -static grpc_fd *fd_freelist = NULL; -static gpr_mu fd_freelist_mu; - -static void freelist_fd(grpc_fd *fd) { - gpr_mu_lock(&fd_freelist_mu); - fd->freelist_next = fd_freelist; - fd_freelist = fd; - grpc_iomgr_unregister_object(&fd->iomgr_object); - gpr_mu_unlock(&fd_freelist_mu); -} - -static grpc_fd *alloc_fd(int fd) { - grpc_fd *r = NULL; - - gpr_mu_lock(&fd_freelist_mu); - if (fd_freelist != NULL) { - r = fd_freelist; - fd_freelist = fd_freelist->freelist_next; - } - gpr_mu_unlock(&fd_freelist_mu); - - if (r == NULL) { - r = gpr_malloc(sizeof(grpc_fd)); - gpr_mu_init(&r->mu); - gpr_mu_init(&r->pi_mu); - } - /* TODO: sreek - check with ctiller on why we need to acquire a lock here */ - gpr_mu_lock(&r->mu); - gpr_atm_rel_store(&r->refst, 1); - r->shutdown = 0; - r->read_closure = CLOSURE_NOT_READY; - r->write_closure = CLOSURE_NOT_READY; - r->fd = fd; - r->polling_island = NULL; - r->freelist_next = NULL; - r->on_done_closure = NULL; - r->closed = 0; - r->released = 0; - gpr_mu_unlock(&r->mu); - return r; -} +/* The alarm system needs to be able to wakeup 'some poller' sometimes + * (specifically when a new alarm needs to be triggered earlier than the next + * alarm 'epoch'). This wakeup_fd gives us something to alert on when such a + * case occurs. */ +/* TODO: sreek: Right now, this wakes up all pollers */ +grpc_wakeup_fd grpc_global_wakeup_fd; -static void destroy(grpc_fd *fd) { - gpr_mu_destroy(&fd->mu); - gpr_free(fd); -} +static grpc_fd *fd_freelist = NULL; +static gpr_mu fd_freelist_mu; #ifdef GRPC_FD_REF_COUNT_DEBUG #define REF_BY(fd, n, reason) ref_by(fd, n, reason, __FILE__, __LINE__) @@ -588,12 +574,33 @@ static void unref_by(grpc_fd *fd, int n) { #endif old = gpr_atm_full_fetch_add(&fd->refst, -n); if (old == n) { - freelist_fd(fd); + /* Add the fd to the freelist */ + gpr_mu_lock(&fd_freelist_mu); + fd->freelist_next = fd_freelist; + fd_freelist = fd; + grpc_iomgr_unregister_object(&fd->iomgr_object); + gpr_mu_unlock(&fd_freelist_mu); } else { GPR_ASSERT(old > n); } } +/* Increment refcount by two to avoid changing the orphan bit */ +#ifdef GRPC_FD_REF_COUNT_DEBUG +static void fd_ref(grpc_fd *fd, const char *reason, const char *file, + int line) { + ref_by(fd, 2, reason, file, line); +} + +static void fd_unref(grpc_fd *fd, const char *reason, const char *file, + int line) { + unref_by(fd, 2, reason, file, line); +} +#else +static void fd_ref(grpc_fd *fd) { ref_by(fd, 2); } +static void fd_unref(grpc_fd *fd) { unref_by(fd, 2); } +#endif + static void fd_global_init(void) { gpr_mu_init(&fd_freelist_mu); } static void fd_global_shutdown(void) { @@ -602,91 +609,111 @@ static void fd_global_shutdown(void) { while (fd_freelist != NULL) { grpc_fd *fd = fd_freelist; fd_freelist = fd_freelist->freelist_next; - destroy(fd); + gpr_mu_destroy(&fd->mu); + gpr_free(fd); } gpr_mu_destroy(&fd_freelist_mu); } static grpc_fd *fd_create(int fd, const char *name) { - grpc_fd *r = alloc_fd(fd); + grpc_fd *new_fd = NULL; + + gpr_mu_lock(&fd_freelist_mu); + if (fd_freelist != NULL) { + new_fd = fd_freelist; + fd_freelist = fd_freelist->freelist_next; + } + gpr_mu_unlock(&fd_freelist_mu); + + if (new_fd == NULL) { + new_fd = gpr_malloc(sizeof(grpc_fd)); + gpr_mu_init(&new_fd->mu); + gpr_mu_init(&new_fd->pi_mu); + } - char *name2; - gpr_asprintf(&name2, "%s fd=%d", name, fd); - grpc_iomgr_register_object(&r->iomgr_object, name2); - gpr_free(name2); + /* Note: It is not really needed to get the new_fd->mu lock here. If this is a + newly created fd (or an fd we got from the freelist), no one else would be + holding a lock to it anyway. */ + gpr_mu_lock(&new_fd->mu); + + gpr_atm_rel_store(&new_fd->refst, 1); + new_fd->shutdown = false; + new_fd->read_closure = CLOSURE_NOT_READY; + new_fd->write_closure = CLOSURE_NOT_READY; + new_fd->fd = fd; + new_fd->polling_island = NULL; + new_fd->freelist_next = NULL; + new_fd->on_done_closure = NULL; + new_fd->closed = 0; + new_fd->released = false; + + gpr_mu_unlock(&new_fd->mu); + + char *fd_name; + gpr_asprintf(&fd_name, "%s fd=%d", name, fd); + grpc_iomgr_register_object(&new_fd->iomgr_object, fd_name); + gpr_free(fd_name); #ifdef GRPC_FD_REF_COUNT_DEBUG - gpr_log(GPR_DEBUG, "FD %d %p create %s", fd, r, name); + gpr_log(GPR_DEBUG, "FD %d %p create %s", fd, r, fd_name); #endif - return r; + return new_fd; } static bool fd_is_orphaned(grpc_fd *fd) { return (gpr_atm_acq_load(&fd->refst) & 1) == 0; } -static void close_fd_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - fd->closed = 1; - if (!fd->released) { - close(fd->fd); - } else { - /* TODO: sreek - Check for deadlocks */ - - gpr_mu_lock(&fd->pi_mu); - fd->polling_island = - polling_island_update_and_lock(fd->polling_island, 1, 0); - - polling_island_remove_fd_locked(fd->polling_island, fd); - polling_island_unref_and_unlock(fd->polling_island, 1); - - fd->polling_island = NULL; - gpr_mu_unlock(&fd->pi_mu); - } - - grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); -} - static int fd_wrapped_fd(grpc_fd *fd) { - if (fd->released || fd->closed) { - return -1; - } else { - return fd->fd; + int ret_fd = -1; + gpr_mu_lock(&fd->mu); + if (!fd->released && !fd->closed) { + ret_fd = fd->fd; } + gpr_mu_unlock(&fd->mu); + + return ret_fd; } -/* TODO: sreek - do something here with the pollset island link */ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure *on_done, int *release_fd, const char *reason) { + /* TODO(sreek) In ev_poll_posix.c,the lock is acquired a little later. Why? */ + gpr_mu_lock(&fd->mu); fd->on_done_closure = on_done; + + /* If release_fd is not NULL, we should be relinquishing control of the file + descriptor fd->fd (but we still own the grpc_fd structure). */ fd->released = release_fd != NULL; if (!fd->released) { shutdown(fd->fd, SHUT_RDWR); } else { *release_fd = fd->fd; } - gpr_mu_lock(&fd->mu); - REF_BY(fd, 1, reason); /* remove active status, but keep referenced */ - close_fd_locked(exec_ctx, fd); - gpr_mu_unlock(&fd->mu); - UNREF_BY(fd, 2, reason); /* drop the reference */ -} -/* increment refcount by two to avoid changing the orphan bit */ -#ifdef GRPC_FD_REF_COUNT_DEBUG -static void fd_ref(grpc_fd *fd, const char *reason, const char *file, - int line) { - ref_by(fd, 2, reason, file, line); -} + REF_BY(fd, 1, reason); /* Remove active status, but keep referenced */ + fd->closed = 1; -static void fd_unref(grpc_fd *fd, const char *reason, const char *file, - int line) { - unref_by(fd, 2, reason, file, line); -} -#else -static void fd_ref(grpc_fd *fd) { ref_by(fd, 2); } + /* Remove the fd from the polling island: + - Update the fd->polling_island to point to the latest polling island + - Remove the fd from the polling island. Also, call close() on the file + descriptor fd->fd ONLY if we haven't relinquised control (i.e + fd->released is 'false') + - Decrement the ref count on the polling island and det fd->polling_island + to NULL */ + gpr_mu_lock(&fd->pi_mu); -static void fd_unref(grpc_fd *fd) { unref_by(fd, 2); } -#endif + fd->polling_island = polling_island_update_and_lock(fd->polling_island, 1, 0); + polling_island_remove_fd_locked(fd->polling_island, fd, !fd->released, true); + polling_island_unref_and_unlock(fd->polling_island, 1); + fd->polling_island = NULL; + + gpr_mu_unlock(&fd->pi_mu); + + grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); + + gpr_mu_unlock(&fd->mu); + UNREF_BY(fd, 2, reason); /* Drop the reference */ +} static void notify_on_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st, grpc_closure *closure) { @@ -724,11 +751,13 @@ static int set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, } } -/* Do something here with the pollset island link (?) */ static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { gpr_mu_lock(&fd->mu); GPR_ASSERT(!fd->shutdown); - fd->shutdown = 1; + fd->shutdown = true; + + /* Flush any pending read and write closures. Since fd->shutdown is 'true' at + this point, the closures would be called with 'success = false' */ set_ready_locked(exec_ctx, fd, &fd->read_closure); set_ready_locked(exec_ctx, fd, &fd->write_closure); gpr_mu_unlock(&fd->mu); @@ -749,27 +778,39 @@ static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, } /******************************************************************************* - * pollset_posix.c + * Pollset Definitions */ -GPR_TLS_DECL(g_current_thread_poller); -GPR_TLS_DECL(g_current_thread_worker); +static void sig_handler(int sig_num) { + /* TODO: sreek - Remove this expensive log line */ + gpr_log(GPR_INFO, "Received signal %d", sig_num); +} -/** The alarm system needs to be able to wakeup 'some poller' sometimes - * (specifically when a new alarm needs to be triggered earlier than the next - * alarm 'epoch'). - * This wakeup_fd gives us something to alert on when such a case occurs. */ -grpc_wakeup_fd grpc_global_wakeup_fd; +/* Global state management */ +static void pollset_global_init(void) { + gpr_tls_init(&g_current_thread_poller); + gpr_tls_init(&g_current_thread_worker); + grpc_wakeup_fd_init(&grpc_global_wakeup_fd); + signal(SIGUSR1, sig_handler); +} -static void remove_worker(grpc_pollset *p, grpc_pollset_worker *worker) { - worker->prev->next = worker->next; - worker->next->prev = worker->prev; +static void pollset_global_shutdown(void) { + grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); + gpr_tls_destroy(&g_current_thread_poller); + gpr_tls_destroy(&g_current_thread_worker); } +/* Return 1 if the pollset has active threads in pollset_work (pollset must + * be locked) */ static int pollset_has_workers(grpc_pollset *p) { return p->root_worker.next != &p->root_worker; } +static void remove_worker(grpc_pollset *p, grpc_pollset_worker *worker) { + worker->prev->next = worker->next; + worker->next->prev = worker->prev; +} + static grpc_pollset_worker *pop_front_worker(grpc_pollset *p) { if (pollset_has_workers(p)) { grpc_pollset_worker *w = p->root_worker.next; @@ -792,241 +833,69 @@ static void push_front_worker(grpc_pollset *p, grpc_pollset_worker *worker) { worker->prev->next = worker->next->prev = worker; } -static void pollset_kick_ext(grpc_pollset *p, - grpc_pollset_worker *specific_worker, - uint32_t flags) { - GPR_TIMER_BEGIN("pollset_kick_ext", 0); - - /* pollset->mu already held */ - if (specific_worker != NULL) { - if (specific_worker == GRPC_POLLSET_KICK_BROADCAST) { - GPR_TIMER_BEGIN("pollset_kick_ext.broadcast", 0); - for (specific_worker = p->root_worker.next; - specific_worker != &p->root_worker; - specific_worker = specific_worker->next) { - pthread_kill(specific_worker->pt_id, SIGUSR1); +/* p->mu must be held before calling this function */ +static void pollset_kick(grpc_pollset *p, + grpc_pollset_worker *specific_worker) { + GPR_TIMER_BEGIN("pollset_kick", 0); + + grpc_pollset_worker *worker = specific_worker; + if (worker != NULL) { + if (worker == GRPC_POLLSET_KICK_BROADCAST) { + GPR_TIMER_BEGIN("pollset_kick.broadcast", 0); + if (pollset_has_workers(p)) { + for (worker = p->root_worker.next; worker != &p->root_worker; + worker = worker->next) { + pthread_kill(worker->pt_id, SIGUSR1); + } + } else { + p->kicked_without_pollers = true; } - p->kicked_without_pollers = 1; - GPR_TIMER_END("pollset_kick_ext.broadcast", 0); - } else if (gpr_tls_get(&g_current_thread_worker) != - (intptr_t)specific_worker) { - GPR_TIMER_MARK("different_thread_worker", 0); - specific_worker->kicked_specifically = 1; - /* TODO (sreek): Refactor this into a separate file*/ - pthread_kill(specific_worker->pt_id, SIGUSR1); - } else if ((flags & GRPC_POLLSET_CAN_KICK_SELF) != 0) { - GPR_TIMER_MARK("kick_yoself", 0); - specific_worker->kicked_specifically = 1; - pthread_kill(specific_worker->pt_id, SIGUSR1); + GPR_TIMER_END("pollset_kick.broadcast", 0); + } else { + GPR_TIMER_MARK("kicked_specifically", 0); + worker->kicked_specifically = true; + pthread_kill(worker->pt_id, SIGUSR1); } - } else if (gpr_tls_get(&g_current_thread_poller) != (intptr_t)p) { + } else { GPR_TIMER_MARK("kick_anonymous", 0); - specific_worker = pop_front_worker(p); - if (specific_worker != NULL) { - if (gpr_tls_get(&g_current_thread_worker) == (intptr_t)specific_worker) { - GPR_TIMER_MARK("kick_anonymous_not_self", 0); - push_back_worker(p, specific_worker); - specific_worker = pop_front_worker(p); - if ((flags & GRPC_POLLSET_CAN_KICK_SELF) == 0 && - gpr_tls_get(&g_current_thread_worker) == - (intptr_t)specific_worker) { - push_back_worker(p, specific_worker); - specific_worker = NULL; - } - } - if (specific_worker != NULL) { - GPR_TIMER_MARK("finally_kick", 0); - push_back_worker(p, specific_worker); - pthread_kill(specific_worker->pt_id, SIGUSR1); - } + worker = pop_front_worker(p); + if (worker != NULL) { + GPR_TIMER_MARK("finally_kick", 0); + push_back_worker(p, worker); + pthread_kill(worker->pt_id, SIGUSR1); } else { GPR_TIMER_MARK("kicked_no_pollers", 0); - p->kicked_without_pollers = 1; + p->kicked_without_pollers = true; } } - GPR_TIMER_END("pollset_kick_ext", 0); + GPR_TIMER_END("pollset_kick", 0); } -static void pollset_kick(grpc_pollset *p, - grpc_pollset_worker *specific_worker) { - pollset_kick_ext(p, specific_worker, 0); -} +static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } -/* global state management */ +static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { + gpr_mu_init(&pollset->mu); + *mu = &pollset->mu; -static void sig_handler(int sig_num) { - gpr_log(GPR_INFO, "Received signal %d", sig_num); -} + pollset->root_worker.next = pollset->root_worker.prev = &pollset->root_worker; + pollset->kicked_without_pollers = false; -static void pollset_global_init(void) { - gpr_tls_init(&g_current_thread_poller); - gpr_tls_init(&g_current_thread_worker); - grpc_wakeup_fd_init(&grpc_global_wakeup_fd); - signal(SIGUSR1, sig_handler); -} + pollset->shutting_down = false; + pollset->finish_shutdown_called = false; + pollset->shutdown_done = NULL; -static void pollset_global_shutdown(void) { - grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); - gpr_tls_destroy(&g_current_thread_poller); - gpr_tls_destroy(&g_current_thread_worker); -} - -static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } - -/* TODO: sreek. Try to Remove this forward declaration*/ -static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset); - -/* main interface */ - -static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { - gpr_mu_init(&pollset->mu); - *mu = &pollset->mu; - pollset->root_worker.next = pollset->root_worker.prev = &pollset->root_worker; gpr_mu_init(&pollset->pi_mu); pollset->polling_island = NULL; - pollset->shutting_down = 0; - pollset->called_shutdown = 0; - pollset->kicked_without_pollers = 0; - - multipoll_with_epoll_pollset_create_efd(pollset); -} - -/* TODO(sreek): Maybe merge multipoll_*_destroy() with pollset_destroy() - * function */ -static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset); - -static void pollset_destroy(grpc_pollset *pollset) { - GPR_ASSERT(!pollset_has_workers(pollset)); - - multipoll_with_epoll_pollset_destroy(pollset); - - gpr_mu_destroy(&pollset->pi_mu); - gpr_mu_destroy(&pollset->mu); -} - -/* TODO(sreek) - Do something with the pollset island link (??) */ -static void pollset_reset(grpc_pollset *pollset) { - GPR_ASSERT(pollset->shutting_down); - GPR_ASSERT(!pollset_has_workers(pollset)); - pollset->shutting_down = 0; - pollset->called_shutdown = 0; - pollset->kicked_without_pollers = 0; -} - -/* TODO (sreek): Remove multipoll_with_epoll_finish_shutdown() declaration */ -static void multipoll_with_epoll_pollset_finish_shutdown(grpc_pollset *pollset); - -static void finish_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { - multipoll_with_epoll_pollset_finish_shutdown(pollset); - grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); -} - -/* TODO(sreek): Remove multipoll_with_epoll_*_maybe_work_and_unlock - * declaration - */ -static void multipoll_with_epoll_pollset_maybe_work_and_unlock( - grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, - gpr_timespec deadline, gpr_timespec now); - -static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_pollset_worker **worker_hdl, gpr_timespec now, - gpr_timespec deadline) { - grpc_pollset_worker worker; - *worker_hdl = &worker; - - /* pollset->mu already held */ - int added_worker = 0; - int locked = 1; - int queued_work = 0; - int keep_polling = 0; - GPR_TIMER_BEGIN("pollset_work", 0); - /* this must happen before we (potentially) drop pollset->mu */ - worker.next = worker.prev = NULL; - worker.kicked_specifically = 0; - - /* TODO(sreek): Abstract this thread id stuff out into a separate file */ - worker.pt_id = pthread_self(); - /* If we're shutting down then we don't execute any extended work */ - if (pollset->shutting_down) { - GPR_TIMER_MARK("pollset_work.shutting_down", 0); - goto done; - } - /* Start polling, and keep doing so while we're being asked to - re-evaluate our pollers (this allows poll() based pollers to - ensure they don't miss wakeups) */ - keep_polling = 1; - while (keep_polling) { - keep_polling = 0; - if (!pollset->kicked_without_pollers) { - if (!added_worker) { - push_front_worker(pollset, &worker); - added_worker = 1; - gpr_tls_set(&g_current_thread_worker, (intptr_t)&worker); - } - gpr_tls_set(&g_current_thread_poller, (intptr_t)pollset); - GPR_TIMER_BEGIN("maybe_work_and_unlock", 0); - - multipoll_with_epoll_pollset_maybe_work_and_unlock( - exec_ctx, pollset, &worker, deadline, now); - - GPR_TIMER_END("maybe_work_and_unlock", 0); - locked = 0; - gpr_tls_set(&g_current_thread_poller, 0); - } else { - GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); - pollset->kicked_without_pollers = 0; - } - /* Finished execution - start cleaning up. - Note that we may arrive here from outside the enclosing while() loop. - In that case we won't loop though as we haven't added worker to the - worker list, which means nobody could ask us to re-evaluate polling). */ - done: - if (!locked) { - queued_work |= grpc_exec_ctx_flush(exec_ctx); - gpr_mu_lock(&pollset->mu); - locked = 1; - } - } - if (added_worker) { - remove_worker(pollset, &worker); - gpr_tls_set(&g_current_thread_worker, 0); - } - - /* check shutdown conditions */ - if (pollset->shutting_down) { - if (pollset_has_workers(pollset)) { - pollset_kick(pollset, NULL); - } else if (!pollset->called_shutdown) { - pollset->called_shutdown = 1; - gpr_mu_unlock(&pollset->mu); - finish_shutdown(exec_ctx, pollset); - grpc_exec_ctx_flush(exec_ctx); - /* Continuing to access pollset here is safe -- it is the caller's - * responsibility to not destroy when it has outstanding calls to - * pollset_work. - * TODO(dklempner): Can we refactor the shutdown logic to avoid this? */ - gpr_mu_lock(&pollset->mu); - } - } - *worker_hdl = NULL; - GPR_TIMER_END("pollset_work", 0); -} - -/* TODO: (sreek) Do something with the pollset island link */ -static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_closure *closure) { - GPR_ASSERT(!pollset->shutting_down); - pollset->shutting_down = 1; - pollset->shutdown_done = closure; - pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); - - if (!pollset->called_shutdown && !pollset_has_workers(pollset)) { - pollset->called_shutdown = 1; - finish_shutdown(exec_ctx, pollset); - } } +/* Convert a timespec to milliseconds: + - Very small or negative poll times are clamped to zero to do a non-blocking + poll (which becomes spin polling) + - Other small values are rounded up to one millisecond + - Longer than a millisecond polls are rounded up to the next nearest + millisecond to avoid spinning + - Infinite timeouts are converted to -1 */ static int poll_deadline_to_millis_timeout(gpr_timespec deadline, gpr_timespec now) { gpr_timespec timeout; @@ -1034,6 +903,7 @@ static int poll_deadline_to_millis_timeout(gpr_timespec deadline, if (gpr_time_cmp(deadline, gpr_inf_future(deadline.clock_type)) == 0) { return -1; } + if (gpr_time_cmp(deadline, gpr_time_add(now, gpr_time_from_micros( max_spin_polling_us, GPR_TIMESPAN))) <= 0) { @@ -1044,10 +914,6 @@ static int poll_deadline_to_millis_timeout(gpr_timespec deadline, timeout, gpr_time_from_nanos(GPR_NS_PER_MS - 1, GPR_TIMESPAN))); } -/******************************************************************************* - * pollset_multipoller_with_epoll_posix.c - */ - static void set_ready(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st) { /* only one set_ready can be active at once (but there may be a racing notify_on) */ @@ -1064,94 +930,46 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { set_ready(exec_ctx, fd, &fd->write_closure); } -/* TODO: sreek - This function multipoll_with_epoll_pollset_add_fd() and - * finally_add_fd() in ev_poll_and_epoll_posix.c */ -static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_fd *fd) { - /* TODO sreek - Check if we need to get a pollset->mu lock here */ - gpr_mu_lock(&pollset->pi_mu); - gpr_mu_lock(&fd->pi_mu); - - polling_island *pi_new = NULL; - - if (fd->polling_island == pollset->polling_island) { - pi_new = fd->polling_island; - if (pi_new == NULL) { - pi_new = polling_island_create(fd, 2); - } - } else if (fd->polling_island == NULL) { - pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); - } else if (pollset->polling_island == NULL) { - pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); - } else { - pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); - } - - fd->polling_island = pollset->polling_island = pi_new; - - gpr_mu_unlock(&fd->pi_mu); - gpr_mu_unlock(&pollset->pi_mu); -} - -/* Creates an epoll fd and initializes the pollset */ -/* TODO: This has to be called ONLY from pollset_init function. and hence it - * does not acquire any lock */ -static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { - struct epoll_event ev; - int err; - - pollset->epoll_fd = epoll_create1(EPOLL_CLOEXEC); - if (pollset->epoll_fd < 0) { - gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); - abort(); - } - - ev.events = (uint32_t)(EPOLLIN | EPOLLET); - ev.data.ptr = NULL; - - err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); - if (err < 0) { - gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), - strerror(errno)); - } -} - /* TODO(klempner): We probably want to turn this down a bit */ #define GRPC_EPOLL_MAX_EVENTS 1000 - -static void multipoll_with_epoll_pollset_maybe_work_and_unlock( - grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, - gpr_timespec deadline, gpr_timespec now) { +static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, int timeout_ms, + sigset_t *sig_mask) { struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; - int epoll_fd = pollset->epoll_fd; + int epoll_fd; int ep_rv; - int timeout_ms; + GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); + + /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the + polling island pointed by pollset->polling_island. + Acquire the following locks: + - pollset->mu (which we already have) + - pollset->pi_mu + - pollset->polling_island->mu */ + gpr_mu_lock(&pollset->pi_mu); + pollset->polling_island = + polling_island_update_and_lock(pollset->polling_island, 1, 0); - /* If you want to ignore epoll's ability to sanely handle parallel pollers, - * for a more apples-to-apples performance comparison with poll, add a - * if (pollset->counter != 0) { return 0; } - * here. - */ + epoll_fd = pollset->polling_island->epoll_fd; + /* Release the locks */ + polling_island_unref_and_unlock(pollset->polling_island, 0); /* Keep the ref*/ + gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); - timeout_ms = poll_deadline_to_millis_timeout(deadline, now); - do { - /* The following epoll_wait never blocks; it has a timeout of 0 */ - ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms); + ep_rv = epoll_pwait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms, + sig_mask); + if (ep_rv < 0) { if (errno != EINTR) { - gpr_log(GPR_ERROR, "epoll_wait() failed: %s", strerror(errno)); + /* TODO (sreek) - Check for bad file descriptor error */ + gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); } } else { int i; for (i = 0; i < ep_rv; ++i) { grpc_fd *fd = ep_ev[i].data.ptr; - /* TODO(klempner): We might want to consider making err and pri - * separate events */ int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); int write_ev = ep_ev[i].events & EPOLLOUT; @@ -1168,17 +986,179 @@ static void multipoll_with_epoll_pollset_maybe_work_and_unlock( } } } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); + + GPR_TIMER_END("pollset_work_and_unlock", 0); +} + +/* Release the reference to pollset->polling_island and set it to NULL. + pollset->mu must be held */ +static void pollset_release_polling_island_locked(grpc_pollset *pollset) { + gpr_mu_lock(&pollset->pi_mu); + if (pollset->polling_island) { + pollset->polling_island = + polling_island_update_and_lock(pollset->polling_island, 1, 0); + polling_island_unref_and_unlock(pollset->polling_island, 1); + pollset->polling_island = NULL; + } + gpr_mu_unlock(&pollset->pi_mu); +} + +static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset) { + /* The pollset cannot have any workers if we are at this stage */ + GPR_ASSERT(!pollset_has_workers(pollset)); + + pollset->finish_shutdown_called = true; + pollset_release_polling_island_locked(pollset); + + grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); +} + +/* pollset->mu lock must be held by the caller before calling this */ +static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_closure *closure) { + GPR_TIMER_BEGIN("pollset_shutdown", 0); + GPR_ASSERT(!pollset->shutting_down); + pollset->shutting_down = true; + pollset->shutdown_done = closure; + pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); + + /* If the pollset has any workers, we cannot call finish_shutdown_locked() + because it would release the underlying polling island. In such a case, we + let the last worker call finish_shutdown_locked() from pollset_work() */ + if (!pollset_has_workers(pollset)) { + GPR_ASSERT(!pollset->finish_shutdown_called); + GPR_TIMER_MARK("pollset_shutdown.finish_shutdown_locked", 0); + finish_shutdown_locked(exec_ctx, pollset); + } + GPR_TIMER_END("pollset_shutdown", 0); } -static void multipoll_with_epoll_pollset_finish_shutdown( - grpc_pollset *pollset) {} +/* TODO(sreek) Is pollset_shutdown() guranteed to be called before this? */ +static void pollset_destroy(grpc_pollset *pollset) { + GPR_ASSERT(!pollset_has_workers(pollset)); + gpr_mu_destroy(&pollset->pi_mu); + gpr_mu_destroy(&pollset->mu); +} -static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset) { - close(pollset->epoll_fd); +static void pollset_reset(grpc_pollset *pollset) { + GPR_ASSERT(pollset->shutting_down); + GPR_ASSERT(!pollset_has_workers(pollset)); + pollset->shutting_down = false; + pollset->finish_shutdown_called = false; + pollset->kicked_without_pollers = false; + /* TODO(sreek) - Should pollset->shutdown closure be set to NULL here? */ + pollset_release_polling_island_locked(pollset); +} + +/* pollset->mu lock must be held by the caller before calling this. + The function pollset_work() may temporarily release the lock (pollset->mu) + during the course of its execution but it will always re-acquire the lock and + ensure that it is held by the time the function returns */ +static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_pollset_worker **worker_hdl, gpr_timespec now, + gpr_timespec deadline) { + GPR_TIMER_BEGIN("pollset_work", 0); + + int timeout_ms = poll_deadline_to_millis_timeout(deadline, now); + + sigset_t new_mask; + sigset_t orig_mask; + + grpc_pollset_worker worker; + worker.next = worker.prev = NULL; + worker.kicked_specifically = 0; + worker.pt_id = pthread_self(); + + *worker_hdl = &worker; + + if (pollset->kicked_without_pollers) { + /* If the pollset was kicked without pollers, pretend that the current + worker got the kick and skip polling. A kick indicates that there is some + work that needs attention like an event on the completion queue or an + alarm */ + GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); + pollset->kicked_without_pollers = 0; + } else if (!pollset->shutting_down) { + sigemptyset(&new_mask); + sigaddset(&new_mask, SIGUSR1); + pthread_sigmask(SIG_BLOCK, &new_mask, &orig_mask); + sigdelset(&orig_mask, SIGUSR1); + + push_front_worker(pollset, &worker); + + pollset_work_and_unlock(exec_ctx, pollset, timeout_ms, &orig_mask); + grpc_exec_ctx_flush(exec_ctx); + + gpr_mu_lock(&pollset->mu); + remove_worker(pollset, &worker); + } + + /* If we are the last worker on the pollset (i.e pollset_has_workers() is + false at this point) and the pollset is shutting down, we may have to + finish the shutdown process by calling finish_shutdown_locked(). + See pollset_shutdown() for more details. + + Note: Continuing to access pollset here is safe; it is the caller's + responsibility to not destroy a pollset when it has outstanding calls to + pollset_work() */ + if (pollset->shutting_down && !pollset_has_workers(pollset) && + !pollset->finish_shutdown_called) { + GPR_TIMER_MARK("pollset_work.finish_shutdown_locked", 0); + finish_shutdown_locked(exec_ctx, pollset); + + gpr_mu_unlock(&pollset->mu); + grpc_exec_ctx_flush(exec_ctx); + gpr_mu_lock(&pollset->mu); + } + + *worker_hdl = NULL; + GPR_TIMER_END("pollset_work", 0); +} + +static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_fd *fd) { + /* TODO sreek - Check if we need to get a pollset->mu lock here */ + gpr_mu_lock(&pollset->pi_mu); + gpr_mu_lock(&fd->pi_mu); + + polling_island *pi_new = NULL; + + /* 1) If fd->polling_island and pollset->polling_island are both non-NULL and + * equal, do nothing. + * 2) If fd->polling_island and pollset->polling_island are both NULL, create + * a new polling island (with a refcount of 2) and make the polling_island + * fields in both fd and pollset to point to the new island + * 3) If one of fd->polling_island or pollset->polling_island is NULL, update + * the NULL polling_island field to point to the non-NULL polling_island + * field (ensure that the refcount on the polling island is incremented by + * 1 to account for the newly added reference) + * 4) Finally, if fd->polling_island and pollset->polling_island are non-NULL + * and different, merge both the polling islands and update the + * polling_island fields in both fd and pollset to point to the merged + * polling island. + */ + if (fd->polling_island == pollset->polling_island) { + pi_new = fd->polling_island; + if (pi_new == NULL) { + pi_new = polling_island_create(fd, 2); + } + } else if (fd->polling_island == NULL) { + pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); + } else if (pollset->polling_island == NULL) { + pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); + } else { + pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); + } + + fd->polling_island = pollset->polling_island = pi_new; + + gpr_mu_unlock(&fd->pi_mu); + gpr_mu_unlock(&pollset->pi_mu); } /******************************************************************************* - * pollset_set_posix.c + * Pollset-set Definitions */ static grpc_pollset_set *pollset_set_create(void) { @@ -1200,6 +1180,45 @@ static void pollset_set_destroy(grpc_pollset_set *pollset_set) { gpr_free(pollset_set); } +static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + if (pollset_set->fd_count == pollset_set->fd_capacity) { + pollset_set->fd_capacity = GPR_MAX(8, 2 * pollset_set->fd_capacity); + pollset_set->fds = gpr_realloc( + pollset_set->fds, pollset_set->fd_capacity * sizeof(*pollset_set->fds)); + } + GRPC_FD_REF(fd, "pollset_set"); + pollset_set->fds[pollset_set->fd_count++] = fd; + for (i = 0; i < pollset_set->pollset_count; i++) { + pollset_add_fd(exec_ctx, pollset_set->pollsets[i], fd); + } + for (i = 0; i < pollset_set->pollset_set_count; i++) { + pollset_set_add_fd(exec_ctx, pollset_set->pollset_sets[i], fd); + } + gpr_mu_unlock(&pollset_set->mu); +} + +static void pollset_set_del_fd(grpc_exec_ctx *exec_ctx, + grpc_pollset_set *pollset_set, grpc_fd *fd) { + size_t i; + gpr_mu_lock(&pollset_set->mu); + for (i = 0; i < pollset_set->fd_count; i++) { + if (pollset_set->fds[i] == fd) { + pollset_set->fd_count--; + GPR_SWAP(grpc_fd *, pollset_set->fds[i], + pollset_set->fds[pollset_set->fd_count]); + GRPC_FD_UNREF(fd, "pollset_set"); + break; + } + } + for (i = 0; i < pollset_set->pollset_set_count; i++) { + pollset_set_del_fd(exec_ctx, pollset_set->pollset_sets[i], fd); + } + gpr_mu_unlock(&pollset_set->mu); +} + static void pollset_set_add_pollset(grpc_exec_ctx *exec_ctx, grpc_pollset_set *pollset_set, grpc_pollset *pollset) { @@ -1281,47 +1300,8 @@ static void pollset_set_del_pollset_set(grpc_exec_ctx *exec_ctx, gpr_mu_unlock(&bag->mu); } -static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, grpc_fd *fd) { - size_t i; - gpr_mu_lock(&pollset_set->mu); - if (pollset_set->fd_count == pollset_set->fd_capacity) { - pollset_set->fd_capacity = GPR_MAX(8, 2 * pollset_set->fd_capacity); - pollset_set->fds = gpr_realloc( - pollset_set->fds, pollset_set->fd_capacity * sizeof(*pollset_set->fds)); - } - GRPC_FD_REF(fd, "pollset_set"); - pollset_set->fds[pollset_set->fd_count++] = fd; - for (i = 0; i < pollset_set->pollset_count; i++) { - pollset_add_fd(exec_ctx, pollset_set->pollsets[i], fd); - } - for (i = 0; i < pollset_set->pollset_set_count; i++) { - pollset_set_add_fd(exec_ctx, pollset_set->pollset_sets[i], fd); - } - gpr_mu_unlock(&pollset_set->mu); -} - -static void pollset_set_del_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, grpc_fd *fd) { - size_t i; - gpr_mu_lock(&pollset_set->mu); - for (i = 0; i < pollset_set->fd_count; i++) { - if (pollset_set->fds[i] == fd) { - pollset_set->fd_count--; - GPR_SWAP(grpc_fd *, pollset_set->fds[i], - pollset_set->fds[pollset_set->fd_count]); - GRPC_FD_UNREF(fd, "pollset_set"); - break; - } - } - for (i = 0; i < pollset_set->pollset_set_count; i++) { - pollset_set_del_fd(exec_ctx, pollset_set->pollset_sets[i], fd); - } - gpr_mu_unlock(&pollset_set->mu); -} - /******************************************************************************* - * event engine binding + * Event engine binding */ static void shutdown_engine(void) { From 73ef9154024290e86a3566a574c04992afc93d00 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 3 Jun 2016 15:25:05 -0700 Subject: [PATCH 031/280] epoll polling strategy now points to the new code --- src/core/lib/iomgr/ev_epoll_linux.c | 4 ---- src/core/lib/iomgr/ev_posix.c | 4 ++-- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index ce42a9e7ce6..ab4224b2d56 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -788,16 +788,12 @@ static void sig_handler(int sig_num) { /* Global state management */ static void pollset_global_init(void) { - gpr_tls_init(&g_current_thread_poller); - gpr_tls_init(&g_current_thread_worker); grpc_wakeup_fd_init(&grpc_global_wakeup_fd); signal(SIGUSR1, sig_handler); } static void pollset_global_shutdown(void) { grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); - gpr_tls_destroy(&g_current_thread_poller); - gpr_tls_destroy(&g_current_thread_worker); } /* Return 1 if the pollset has active threads in pollset_work (pollset must diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index 96399ef8379..b6b113aed3f 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -44,7 +44,7 @@ #include #include -#include "src/core/lib/iomgr/ev_epoll_posix.h" +#include "src/core/lib/iomgr/ev_epoll_linux.h" #include "src/core/lib/iomgr/ev_poll_posix.h" #include "src/core/lib/support/env.h" @@ -62,7 +62,7 @@ typedef struct { } event_engine_factory; static const event_engine_factory g_factories[] = { - {"poll", grpc_init_poll_posix}, {"epoll", grpc_init_epoll_posix}, + {"poll", grpc_init_poll_posix}, {"epoll", grpc_init_epoll_linux}, }; static void add(const char *beg, const char *end, char ***ss, size_t *ns) { From 88ee12fbe98685e736366d6a151a10ed103f8979 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 3 Jun 2016 19:26:48 -0700 Subject: [PATCH 032/280] Handle pollsets and fds witn no polling islands and fix locking bug in pollset_add_fd --- src/core/lib/iomgr/ev_epoll_linux.c | 83 ++++++++++++++++------------- 1 file changed, 45 insertions(+), 38 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index ab4224b2d56..0fb1ccfa0fe 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -701,12 +701,14 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - Decrement the ref count on the polling island and det fd->polling_island to NULL */ gpr_mu_lock(&fd->pi_mu); - - fd->polling_island = polling_island_update_and_lock(fd->polling_island, 1, 0); - polling_island_remove_fd_locked(fd->polling_island, fd, !fd->released, true); - polling_island_unref_and_unlock(fd->polling_island, 1); - fd->polling_island = NULL; - + if (fd->polling_island != NULL) { + fd->polling_island = + polling_island_update_and_lock(fd->polling_island, 1, 0); + polling_island_remove_fd_locked(fd->polling_island, fd, !fd->released, + true); + polling_island_unref_and_unlock(fd->polling_island, 1); + fd->polling_island = NULL; + } gpr_mu_unlock(&fd->pi_mu); grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); @@ -926,13 +928,12 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { set_ready(exec_ctx, fd, &fd->write_closure); } -/* TODO(klempner): We probably want to turn this down a bit */ #define GRPC_EPOLL_MAX_EVENTS 1000 static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, int timeout_ms, sigset_t *sig_mask) { struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; - int epoll_fd; + int epoll_fd = -1; int ep_rv; GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); @@ -943,45 +944,49 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, - pollset->pi_mu - pollset->polling_island->mu */ gpr_mu_lock(&pollset->pi_mu); - pollset->polling_island = - polling_island_update_and_lock(pollset->polling_island, 1, 0); - epoll_fd = pollset->polling_island->epoll_fd; + if (pollset->polling_island != NULL) { + pollset->polling_island = + polling_island_update_and_lock(pollset->polling_island, 1, 0); + epoll_fd = pollset->polling_island->epoll_fd; + gpr_mu_unlock(&pollset->polling_island->mu); + } - /* Release the locks */ - polling_island_unref_and_unlock(pollset->polling_island, 0); /* Keep the ref*/ gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); - do { - ep_rv = epoll_pwait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms, - sig_mask); + /* If epoll_fd == -1, this is a blank pollset and does not have any fds yet */ + if (epoll_fd != -1) { + do { + ep_rv = epoll_pwait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms, + sig_mask); - if (ep_rv < 0) { - if (errno != EINTR) { - /* TODO (sreek) - Check for bad file descriptor error */ - gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); - } - } else { - int i; - for (i = 0; i < ep_rv; ++i) { - grpc_fd *fd = ep_ev[i].data.ptr; - int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); - int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); - int write_ev = ep_ev[i].events & EPOLLOUT; - if (fd == NULL) { - grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); - } else { - if (read_ev || cancel) { - fd_become_readable(exec_ctx, fd); - } - if (write_ev || cancel) { - fd_become_writable(exec_ctx, fd); + if (ep_rv < 0) { + if (errno != EINTR) { + /* TODO (sreek) - Check for bad file descriptor error */ + gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); + } + } else { + int i; + for (i = 0; i < ep_rv; ++i) { + grpc_fd *fd = ep_ev[i].data.ptr; + int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + int write_ev = ep_ev[i].events & EPOLLOUT; + if (fd == NULL) { + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + } else { + if (read_ev || cancel) { + fd_become_readable(exec_ctx, fd); + } + if (write_ev || cancel) { + fd_become_writable(exec_ctx, fd); + } } } } - } - } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); + } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); + } GPR_TIMER_END("pollset_work_and_unlock", 0); } @@ -1141,8 +1146,10 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } } else if (fd->polling_island == NULL) { pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); + gpr_mu_unlock(&pi_new->mu); } else if (pollset->polling_island == NULL) { pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); + gpr_mu_unlock(&pi_new->mu); } else { pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); } From 79a6233bef501e1f51250351a55abc61cc024827 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Sat, 4 Jun 2016 14:01:03 -0700 Subject: [PATCH 033/280] Fix a few bugs in ev_epoll_linux.c 1. pollset_add_fd: Add fd to epoll set if fd->polling_island == NULL 2. close(fd) in fd_orphan instead of polling_island_remove_fd_locked() since fd->polling_island may be NULL 3. If pollset work() is interrupted, do a zero timeout epoll_wait(). pollset_work may be called without a polling island --- src/core/lib/iomgr/ev_epoll_linux.c | 125 +++++++++++++++++----------- 1 file changed, 78 insertions(+), 47 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 0fb1ccfa0fe..3aa26109f22 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -72,9 +72,14 @@ struct grpc_fd { gpr_atm refst; gpr_mu mu; + + /* Indicates that the fd is shutdown and that any pending read/write closures + should fail */ bool shutdown; - int closed; - bool released; + + /* The fd is either closed or we relinquished control of it. In either cases, + this indicates that the 'fd' on this structure is no longer valid */ + bool orphaned; grpc_closure *read_closure; grpc_closure *write_closure; @@ -251,16 +256,13 @@ static void polling_island_remove_all_fds_locked(polling_island *pi, /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, - bool close_fd, bool remove_fd_ref) { + bool is_fd_closed) { int err; size_t i; - /* Calling close() on the fd will automatically remove it from the epoll set. - If not calling close(), the fd must be explicitly removed from the epoll - set */ - if (close_fd) { - close(fd->fd); - } else { + /* If fd is already closed, then it would have been automatically been removed + from the epoll set */ + if (!is_fd_closed) { err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, fd->fd, NULL); if (err < 0 && errno != ENOENT) { gpr_log(GPR_ERROR, "epoll_ctl delete for fd: %d failed with error; %s", @@ -271,9 +273,7 @@ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, for (i = 0; i < pi->fd_cnt; i++) { if (pi->fds[i] == fd) { pi->fds[i] = pi->fds[--pi->fd_cnt]; - if (remove_fd_ref) { - GRPC_FD_UNREF(fd, "polling_island"); - } + GRPC_FD_UNREF(fd, "polling_island"); break; } } @@ -644,8 +644,7 @@ static grpc_fd *fd_create(int fd, const char *name) { new_fd->polling_island = NULL; new_fd->freelist_next = NULL; new_fd->on_done_closure = NULL; - new_fd->closed = 0; - new_fd->released = false; + new_fd->orphaned = false; gpr_mu_unlock(&new_fd->mu); @@ -666,7 +665,7 @@ static bool fd_is_orphaned(grpc_fd *fd) { static int fd_wrapped_fd(grpc_fd *fd) { int ret_fd = -1; gpr_mu_lock(&fd->mu); - if (!fd->released && !fd->closed) { + if (!fd->orphaned) { ret_fd = fd->fd; } gpr_mu_unlock(&fd->mu); @@ -678,34 +677,35 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure *on_done, int *release_fd, const char *reason) { /* TODO(sreek) In ev_poll_posix.c,the lock is acquired a little later. Why? */ + bool is_fd_closed = false; gpr_mu_lock(&fd->mu); fd->on_done_closure = on_done; /* If release_fd is not NULL, we should be relinquishing control of the file descriptor fd->fd (but we still own the grpc_fd structure). */ - fd->released = release_fd != NULL; - if (!fd->released) { - shutdown(fd->fd, SHUT_RDWR); - } else { + if (release_fd != NULL) { *release_fd = fd->fd; + } else { + close(fd->fd); + is_fd_closed = true; } - REF_BY(fd, 1, reason); /* Remove active status, but keep referenced */ - fd->closed = 1; + fd->orphaned = true; + + /* Remove the active status but keep referenced. We want this grpc_fd struct + to be alive (and not added to freelist) until the end of this function */ + REF_BY(fd, 1, reason); /* Remove the fd from the polling island: - Update the fd->polling_island to point to the latest polling island - - Remove the fd from the polling island. Also, call close() on the file - descriptor fd->fd ONLY if we haven't relinquised control (i.e - fd->released is 'false') - - Decrement the ref count on the polling island and det fd->polling_island - to NULL */ + - Remove the fd from the polling island. + - Remove a ref to the polling island and set fd->polling_island to NULL */ gpr_mu_lock(&fd->pi_mu); if (fd->polling_island != NULL) { fd->polling_island = polling_island_update_and_lock(fd->polling_island, 1, 0); - polling_island_remove_fd_locked(fd->polling_island, fd, !fd->released, - true); + polling_island_remove_fd_locked(fd->polling_island, fd, is_fd_closed); + polling_island_unref_and_unlock(fd->polling_island, 1); fd->polling_island = NULL; } @@ -839,17 +839,20 @@ static void pollset_kick(grpc_pollset *p, grpc_pollset_worker *worker = specific_worker; if (worker != NULL) { if (worker == GRPC_POLLSET_KICK_BROADCAST) { - GPR_TIMER_BEGIN("pollset_kick.broadcast", 0); + gpr_log(GPR_DEBUG, "pollset_kick: broadcast!"); if (pollset_has_workers(p)) { + GPR_TIMER_BEGIN("pollset_kick.broadcast", 0); for (worker = p->root_worker.next; worker != &p->root_worker; worker = worker->next) { pthread_kill(worker->pt_id, SIGUSR1); } } else { + gpr_log(GPR_DEBUG, "pollset_kick: (broadcast) Kicked without pollers"); p->kicked_without_pollers = true; } GPR_TIMER_END("pollset_kick.broadcast", 0); } else { + gpr_log(GPR_DEBUG, "pollset_kick: kicked kicked_specifically"); GPR_TIMER_MARK("kicked_specifically", 0); worker->kicked_specifically = true; pthread_kill(worker->pt_id, SIGUSR1); @@ -860,9 +863,11 @@ static void pollset_kick(grpc_pollset *p, if (worker != NULL) { GPR_TIMER_MARK("finally_kick", 0); push_back_worker(p, worker); + gpr_log(GPR_DEBUG, "pollset_kick: anonymous kick"); pthread_kill(worker->pt_id, SIGUSR1); } else { GPR_TIMER_MARK("kicked_no_pollers", 0); + gpr_log(GPR_DEBUG, "pollset_kick: kicked without pollers"); p->kicked_without_pollers = true; } } @@ -935,6 +940,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; int epoll_fd = -1; int ep_rv; + gpr_log(GPR_DEBUG, "pollset_work_and_unlock: Entering.."); GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the @@ -949,6 +955,16 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, pollset->polling_island = polling_island_update_and_lock(pollset->polling_island, 1, 0); epoll_fd = pollset->polling_island->epoll_fd; + if (pollset->polling_island->fd_cnt == 0) { + gpr_log(GPR_DEBUG, "pollset_work_and_unlock: epoll_fd: %d, No other fds", + epoll_fd); + } + for (size_t i = 0; i < pollset->polling_island->fd_cnt; i++) { + gpr_log(GPR_DEBUG, + "pollset_work_and_unlock: epoll_fd: %d, fd_count: %d, fd[%d]: %d", + epoll_fd, pollset->polling_island->fd_cnt, i, + pollset->polling_island->fds[i]->fd); + } gpr_mu_unlock(&pollset->polling_island->mu); } @@ -958,36 +974,47 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, /* If epoll_fd == -1, this is a blank pollset and does not have any fds yet */ if (epoll_fd != -1) { do { + gpr_timespec before_epoll = gpr_now(GPR_CLOCK_PRECISE); + gpr_log(GPR_DEBUG, "pollset_work_and_unlock: epoll_wait()...."); ep_rv = epoll_pwait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms, sig_mask); + gpr_timespec after_epoll = gpr_now(GPR_CLOCK_PRECISE); + int dur = gpr_time_to_millis(gpr_time_sub(after_epoll, before_epoll)); + gpr_log(GPR_DEBUG, + "pollset_work_and_unlock: DONE epoll_wait() : %d ms, ep_rv: %d", + dur, ep_rv); if (ep_rv < 0) { if (errno != EINTR) { /* TODO (sreek) - Check for bad file descriptor error */ gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); + } else { + gpr_log(GPR_DEBUG, "pollset_work_and_unlock: 0-timeout epoll_wait()"); + ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); + gpr_log(GPR_DEBUG, "pollset_work_and_unlock: ep_rv: %d", ep_rv); } - } else { - int i; - for (i = 0; i < ep_rv; ++i) { - grpc_fd *fd = ep_ev[i].data.ptr; - int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); - int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); - int write_ev = ep_ev[i].events & EPOLLOUT; - if (fd == NULL) { - grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); - } else { - if (read_ev || cancel) { - fd_become_readable(exec_ctx, fd); - } - if (write_ev || cancel) { - fd_become_writable(exec_ctx, fd); - } + } + + int i; + for (i = 0; i < ep_rv; ++i) { + grpc_fd *fd = ep_ev[i].data.ptr; + int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + int write_ev = ep_ev[i].events & EPOLLOUT; + if (fd == NULL) { + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + } else { + if (read_ev || cancel) { + fd_become_readable(exec_ctx, fd); + } + if (write_ev || cancel) { + fd_become_writable(exec_ctx, fd); } } } } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); } - + gpr_log(GPR_DEBUG, "pollset_work_and_unlock: Leaving.."); GPR_TIMER_END("pollset_work_and_unlock", 0); } @@ -1060,7 +1087,7 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker **worker_hdl, gpr_timespec now, gpr_timespec deadline) { GPR_TIMER_BEGIN("pollset_work", 0); - + gpr_log(GPR_DEBUG, "pollset_work: enter"); int timeout_ms = poll_deadline_to_millis_timeout(deadline, now); sigset_t new_mask; @@ -1079,6 +1106,7 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, work that needs attention like an event on the completion queue or an alarm */ GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); + gpr_log(GPR_INFO, "pollset_work: kicked without pollers.."); pollset->kicked_without_pollers = 0; } else if (!pollset->shutting_down) { sigemptyset(&new_mask); @@ -1113,12 +1141,14 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, gpr_mu_lock(&pollset->mu); } + gpr_log(GPR_DEBUG, "pollset_work(): leaving"); *worker_hdl = NULL; GPR_TIMER_END("pollset_work", 0); } static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { + gpr_log(GPR_DEBUG, "pollset_add_fd: pollset: %p, fd: %d", pollset, fd->fd); /* TODO sreek - Check if we need to get a pollset->mu lock here */ gpr_mu_lock(&pollset->pi_mu); gpr_mu_lock(&fd->pi_mu); @@ -1146,6 +1176,7 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } } else if (fd->polling_island == NULL) { pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); + polling_island_add_fds_locked(pollset->polling_island, &fd, 1, true); gpr_mu_unlock(&pi_new->mu); } else if (pollset->polling_island == NULL) { pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); From b5fe44ea0f7e5ef6f013d6c97725f93a74fed5ef Mon Sep 17 00:00:00 2001 From: Julien Boeuf Date: Mon, 6 Jun 2016 09:50:38 +0200 Subject: [PATCH 034/280] Adding error details for JWT and oauth2. --- src/core/lib/security/credentials/jwt/jwt_credentials.c | 3 ++- .../lib/security/credentials/oauth2/oauth2_credentials.c | 5 +++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/src/core/lib/security/credentials/jwt/jwt_credentials.c b/src/core/lib/security/credentials/jwt/jwt_credentials.c index 0d29729da17..7f96e5247b0 100644 --- a/src/core/lib/security/credentials/jwt/jwt_credentials.c +++ b/src/core/lib/security/credentials/jwt/jwt_credentials.c @@ -116,7 +116,8 @@ static void jwt_get_request_metadata(grpc_exec_ctx *exec_ctx, GRPC_CREDENTIALS_OK, NULL); grpc_credentials_md_store_unref(jwt_md); } else { - cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_ERROR, NULL); + cb(exec_ctx, user_data, NULL, 0, GRPC_CREDENTIALS_ERROR, + "Could not generate JWT."); } } diff --git a/src/core/lib/security/credentials/oauth2/oauth2_credentials.c b/src/core/lib/security/credentials/oauth2/oauth2_credentials.c index 81457183d2d..a0e4b6fe8e9 100644 --- a/src/core/lib/security/credentials/oauth2/oauth2_credentials.c +++ b/src/core/lib/security/credentials/oauth2/oauth2_credentials.c @@ -233,10 +233,11 @@ static void on_oauth2_token_fetcher_http_response( c->token_expiration = gpr_time_add(gpr_now(GPR_CLOCK_REALTIME), token_lifetime); r->cb(exec_ctx, r->user_data, c->access_token_md->entries, - c->access_token_md->num_entries, status, NULL); + c->access_token_md->num_entries, GRPC_CREDENTIALS_OK, NULL); } else { c->token_expiration = gpr_inf_past(GPR_CLOCK_REALTIME); - r->cb(exec_ctx, r->user_data, NULL, 0, status, NULL); + r->cb(exec_ctx, r->user_data, NULL, 0, status, + "Error occured when fetching oauth2 token."); } gpr_mu_unlock(&c->mu); grpc_credentials_metadata_request_destroy(r); From 38c0cdee34697bacb95f34d1a4d31131d0df3af5 Mon Sep 17 00:00:00 2001 From: Julien Boeuf Date: Mon, 6 Jun 2016 14:46:08 +0200 Subject: [PATCH 035/280] Fix double-free issue for optional_message. Also better end to end testing of plumbing of plugin error message. --- .../ext/transport/chttp2/transport/chttp2_transport.c | 8 +++----- test/cpp/end2end/end2end_test.cc | 6 +++++- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/src/core/ext/transport/chttp2/transport/chttp2_transport.c b/src/core/ext/transport/chttp2/transport/chttp2_transport.c index 1abc49f435e..b0da0578f5e 100644 --- a/src/core/ext/transport/chttp2/transport/chttp2_transport.c +++ b/src/core/ext/transport/chttp2/transport/chttp2_transport.c @@ -1504,17 +1504,15 @@ static void close_from_api(grpc_exec_ctx *exec_ctx, gpr_slice_buffer_add(&transport_global->qbuf, gpr_slice_ref(*optional_message)); } - gpr_slice_buffer_add( &transport_global->qbuf, grpc_chttp2_rst_stream_create(stream_global->id, GRPC_CHTTP2_NO_ERROR, &stream_global->stats.outgoing)); - - if (optional_message) { - gpr_slice_ref(*optional_message); - } } + if (optional_message) { + gpr_slice_ref(*optional_message); + } grpc_chttp2_fake_status(exec_ctx, transport_global, stream_global, status, optional_message); grpc_chttp2_mark_stream_closed(exec_ctx, transport_global, stream_global, 1, diff --git a/test/cpp/end2end/end2end_test.cc b/test/cpp/end2end/end2end_test.cc index f52aa52f39f..4602145f54b 100644 --- a/test/cpp/end2end/end2end_test.cc +++ b/test/cpp/end2end/end2end_test.cc @@ -75,6 +75,8 @@ bool CheckIsLocalhost(const grpc::string& addr) { addr.substr(0, kIpv6.size()) == kIpv6; } +const char kTestCredsPluginErrorMsg[] = "Could not find plugin metadata."; + class TestMetadataCredentialsPlugin : public MetadataCredentialsPlugin { public: static const char kMetadataKey[]; @@ -99,7 +101,7 @@ class TestMetadataCredentialsPlugin : public MetadataCredentialsPlugin { metadata->insert(std::make_pair(kMetadataKey, metadata_value_)); return Status::OK; } else { - return Status(StatusCode::NOT_FOUND, "Could not find plugin metadata."); + return Status(StatusCode::NOT_FOUND, kTestCredsPluginErrorMsg); } } @@ -1318,6 +1320,7 @@ TEST_P(SecureEnd2endTest, NonBlockingAuthMetadataPluginFailure) { Status s = stub_->Echo(&context, request, &response); EXPECT_FALSE(s.ok()); EXPECT_EQ(s.error_code(), StatusCode::UNAUTHENTICATED); + EXPECT_EQ(s.error_message(), kTestCredsPluginErrorMsg); } TEST_P(SecureEnd2endTest, NonBlockingAuthMetadataPluginAndProcessorSuccess) { @@ -1375,6 +1378,7 @@ TEST_P(SecureEnd2endTest, BlockingAuthMetadataPluginFailure) { Status s = stub_->Echo(&context, request, &response); EXPECT_FALSE(s.ok()); EXPECT_EQ(s.error_code(), StatusCode::UNAUTHENTICATED); + EXPECT_EQ(s.error_message(), kTestCredsPluginErrorMsg); } TEST_P(SecureEnd2endTest, ClientAuthContext) { From 7d3d4d48cd3a0b916d7f561f6fa525886fb34d8a Mon Sep 17 00:00:00 2001 From: Julien Boeuf Date: Mon, 6 Jun 2016 14:48:41 +0200 Subject: [PATCH 036/280] Fixing clang-format. --- src/core/lib/security/credentials/credentials.h | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/src/core/lib/security/credentials/credentials.h b/src/core/lib/security/credentials/credentials.h index c6df6cdf2f7..b097233bee4 100644 --- a/src/core/lib/security/credentials/credentials.h +++ b/src/core/lib/security/credentials/credentials.h @@ -156,12 +156,9 @@ void grpc_credentials_md_store_unref(grpc_credentials_md_store *store); /* --- grpc_call_credentials. --- */ /* error_details must be NULL if status is GRPC_CREDENTIALS_OK. */ -typedef void (*grpc_credentials_metadata_cb)(grpc_exec_ctx *exec_ctx, - void *user_data, - grpc_credentials_md *md_elems, - size_t num_md, - grpc_credentials_status status, - const char *error_details); +typedef void (*grpc_credentials_metadata_cb)( + grpc_exec_ctx *exec_ctx, void *user_data, grpc_credentials_md *md_elems, + size_t num_md, grpc_credentials_status status, const char *error_details); typedef struct { void (*destruct)(grpc_call_credentials *c); From 4c11a20bf0b6b1d64a2800bcb06f76404294aa84 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 6 Jun 2016 09:23:25 -0700 Subject: [PATCH 037/280] Remove unused files --- BUILD | 6 - Makefile | 38 - binding.gyp | 1 - build.yaml | 14 - config.m4 | 1 - gRPC.podspec | 3 - grpc.gemspec | 2 - package.xml | 2 - src/core/lib/iomgr/ctiller_ev_epoll_linux.c | 461 ------- src/core/lib/iomgr/ev_epoll_linux.c | 2 +- src/core/lib/iomgr/ev_epoll_posix.c | 1209 ----------------- src/core/lib/iomgr/ev_epoll_posix.h | 41 - src/python/grpcio/grpc_core_dependencies.py | 1 - test/core/network_benchmarks/epoll_test.c | 263 ---- .../network_benchmarks/low_level_ping_pong.c | 2 - tools/doxygen/Doxyfile.core.internal | 2 - tools/run_tests/sources_and_headers.json | 19 - tools/run_tests/tests.json | 15 - vsprojects/vcxproj/grpc/grpc.vcxproj | 3 - vsprojects/vcxproj/grpc/grpc.vcxproj.filters | 6 - .../grpc_unsecure/grpc_unsecure.vcxproj | 3 - .../grpc_unsecure.vcxproj.filters | 6 - 22 files changed, 1 insertion(+), 2099 deletions(-) delete mode 100644 src/core/lib/iomgr/ctiller_ev_epoll_linux.c delete mode 100644 src/core/lib/iomgr/ev_epoll_posix.c delete mode 100644 src/core/lib/iomgr/ev_epoll_posix.h delete mode 100644 test/core/network_benchmarks/epoll_test.c diff --git a/BUILD b/BUILD index a32352ebb30..70354a88101 100644 --- a/BUILD +++ b/BUILD @@ -179,7 +179,6 @@ cc_library( "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", "src/core/lib/iomgr/ev_epoll_linux.h", - "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", @@ -324,7 +323,6 @@ cc_library( "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", "src/core/lib/iomgr/ev_epoll_linux.c", - "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", "src/core/lib/iomgr/exec_ctx.c", @@ -551,7 +549,6 @@ cc_library( "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", "src/core/lib/iomgr/ev_epoll_linux.h", - "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", @@ -673,7 +670,6 @@ cc_library( "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", "src/core/lib/iomgr/ev_epoll_linux.c", - "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", "src/core/lib/iomgr/exec_ctx.c", @@ -1367,7 +1363,6 @@ objc_library( "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", "src/core/lib/iomgr/ev_epoll_linux.c", - "src/core/lib/iomgr/ev_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", "src/core/lib/iomgr/exec_ctx.c", @@ -1573,7 +1568,6 @@ objc_library( "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", "src/core/lib/iomgr/ev_epoll_linux.h", - "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", diff --git a/Makefile b/Makefile index 235f32d9a30..1c83aec21e7 100644 --- a/Makefile +++ b/Makefile @@ -903,7 +903,6 @@ dns_resolver_connectivity_test: $(BINDIR)/$(CONFIG)/dns_resolver_connectivity_te dns_resolver_test: $(BINDIR)/$(CONFIG)/dns_resolver_test dualstack_socket_test: $(BINDIR)/$(CONFIG)/dualstack_socket_test endpoint_pair_test: $(BINDIR)/$(CONFIG)/endpoint_pair_test -epoll_test: $(BINDIR)/$(CONFIG)/epoll_test fd_conservation_posix_test: $(BINDIR)/$(CONFIG)/fd_conservation_posix_test fd_posix_test: $(BINDIR)/$(CONFIG)/fd_posix_test fling_client: $(BINDIR)/$(CONFIG)/fling_client @@ -1235,7 +1234,6 @@ buildtests_c: privatelibs_c \ $(BINDIR)/$(CONFIG)/dns_resolver_test \ $(BINDIR)/$(CONFIG)/dualstack_socket_test \ $(BINDIR)/$(CONFIG)/endpoint_pair_test \ - $(BINDIR)/$(CONFIG)/epoll_test \ $(BINDIR)/$(CONFIG)/fd_conservation_posix_test \ $(BINDIR)/$(CONFIG)/fd_posix_test \ $(BINDIR)/$(CONFIG)/fling_client \ @@ -1499,8 +1497,6 @@ test_c: buildtests_c $(Q) $(BINDIR)/$(CONFIG)/dualstack_socket_test || ( echo test dualstack_socket_test failed ; exit 1 ) $(E) "[RUN] Testing endpoint_pair_test" $(Q) $(BINDIR)/$(CONFIG)/endpoint_pair_test || ( echo test endpoint_pair_test failed ; exit 1 ) - $(E) "[RUN] Testing epoll_test" - $(Q) $(BINDIR)/$(CONFIG)/epoll_test || ( echo test epoll_test failed ; exit 1 ) $(E) "[RUN] Testing fd_conservation_posix_test" $(Q) $(BINDIR)/$(CONFIG)/fd_conservation_posix_test || ( echo test fd_conservation_posix_test failed ; exit 1 ) $(E) "[RUN] Testing fd_posix_test" @@ -2491,7 +2487,6 @@ LIBGRPC_SRC = \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ src/core/lib/iomgr/ev_epoll_linux.c \ - src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ @@ -2847,7 +2842,6 @@ LIBGRPC_UNSECURE_SRC = \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ src/core/lib/iomgr/ev_epoll_linux.c \ - src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ @@ -6581,38 +6575,6 @@ endif endif -EPOLL_TEST_SRC = \ - test/core/network_benchmarks/epoll_test.c \ - -EPOLL_TEST_OBJS = $(addprefix $(OBJDIR)/$(CONFIG)/, $(addsuffix .o, $(basename $(EPOLL_TEST_SRC)))) -ifeq ($(NO_SECURE),true) - -# You can't build secure targets if you don't have OpenSSL. - -$(BINDIR)/$(CONFIG)/epoll_test: openssl_dep_error - -else - - - -$(BINDIR)/$(CONFIG)/epoll_test: $(EPOLL_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a - $(E) "[LD] Linking $@" - $(Q) mkdir -p `dirname $@` - $(Q) $(LD) $(LDFLAGS) $(EPOLL_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a $(LDLIBS) $(LDLIBS_SECURE) -o $(BINDIR)/$(CONFIG)/epoll_test - -endif - -$(OBJDIR)/$(CONFIG)/test/core/network_benchmarks/epoll_test.o: $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a - -deps_epoll_test: $(EPOLL_TEST_OBJS:.o=.dep) - -ifneq ($(NO_SECURE),true) -ifneq ($(NO_DEPS),true) --include $(EPOLL_TEST_OBJS:.o=.dep) -endif -endif - - FD_CONSERVATION_POSIX_TEST_SRC = \ test/core/iomgr/fd_conservation_posix_test.c \ diff --git a/binding.gyp b/binding.gyp index 41e1b5bb416..255998aafd8 100644 --- a/binding.gyp +++ b/binding.gyp @@ -582,7 +582,6 @@ 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', 'src/core/lib/iomgr/ev_epoll_linux.c', - 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', 'src/core/lib/iomgr/exec_ctx.c', diff --git a/build.yaml b/build.yaml index db9787546ad..75c7a76bdb3 100644 --- a/build.yaml +++ b/build.yaml @@ -166,7 +166,6 @@ filegroups: - src/core/lib/iomgr/endpoint.h - src/core/lib/iomgr/endpoint_pair.h - src/core/lib/iomgr/ev_epoll_linux.h - - src/core/lib/iomgr/ev_epoll_posix.h - src/core/lib/iomgr/ev_poll_posix.h - src/core/lib/iomgr/ev_posix.h - src/core/lib/iomgr/exec_ctx.h @@ -242,7 +241,6 @@ filegroups: - src/core/lib/iomgr/endpoint_pair_posix.c - src/core/lib/iomgr/endpoint_pair_windows.c - src/core/lib/iomgr/ev_epoll_linux.c - - src/core/lib/iomgr/ev_epoll_posix.c - src/core/lib/iomgr/ev_poll_posix.c - src/core/lib/iomgr/ev_posix.c - src/core/lib/iomgr/exec_ctx.c @@ -1321,18 +1319,6 @@ targets: - grpc - gpr_test_util - gpr -- name: epoll_test - build: test - language: c - src: - - test/core/network_benchmarks/epoll_test.c - deps: - - grpc_test_util - - grpc - - gpr_test_util - - gpr - platforms: - - linux - name: fd_conservation_posix_test build: test language: c diff --git a/config.m4 b/config.m4 index 4308295afda..e2d1c00b6e4 100644 --- a/config.m4 +++ b/config.m4 @@ -101,7 +101,6 @@ if test "$PHP_GRPC" != "no"; then src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ src/core/lib/iomgr/ev_epoll_linux.c \ - src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ diff --git a/gRPC.podspec b/gRPC.podspec index de55880125a..736ae98b543 100644 --- a/gRPC.podspec +++ b/gRPC.podspec @@ -182,7 +182,6 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/endpoint.h', 'src/core/lib/iomgr/endpoint_pair.h', 'src/core/lib/iomgr/ev_epoll_linux.h', - 'src/core/lib/iomgr/ev_epoll_posix.h', 'src/core/lib/iomgr/ev_poll_posix.h', 'src/core/lib/iomgr/ev_posix.h', 'src/core/lib/iomgr/exec_ctx.h', @@ -361,7 +360,6 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', 'src/core/lib/iomgr/ev_epoll_linux.c', - 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', 'src/core/lib/iomgr/exec_ctx.c', @@ -551,7 +549,6 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/endpoint.h', 'src/core/lib/iomgr/endpoint_pair.h', 'src/core/lib/iomgr/ev_epoll_linux.h', - 'src/core/lib/iomgr/ev_epoll_posix.h', 'src/core/lib/iomgr/ev_poll_posix.h', 'src/core/lib/iomgr/ev_posix.h', 'src/core/lib/iomgr/exec_ctx.h', diff --git a/grpc.gemspec b/grpc.gemspec index 54ae2eb68dd..01b2890493f 100755 --- a/grpc.gemspec +++ b/grpc.gemspec @@ -191,7 +191,6 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/endpoint.h ) s.files += %w( src/core/lib/iomgr/endpoint_pair.h ) s.files += %w( src/core/lib/iomgr/ev_epoll_linux.h ) - s.files += %w( src/core/lib/iomgr/ev_epoll_posix.h ) s.files += %w( src/core/lib/iomgr/ev_poll_posix.h ) s.files += %w( src/core/lib/iomgr/ev_posix.h ) s.files += %w( src/core/lib/iomgr/exec_ctx.h ) @@ -340,7 +339,6 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/endpoint_pair_posix.c ) s.files += %w( src/core/lib/iomgr/endpoint_pair_windows.c ) s.files += %w( src/core/lib/iomgr/ev_epoll_linux.c ) - s.files += %w( src/core/lib/iomgr/ev_epoll_posix.c ) s.files += %w( src/core/lib/iomgr/ev_poll_posix.c ) s.files += %w( src/core/lib/iomgr/ev_posix.c ) s.files += %w( src/core/lib/iomgr/exec_ctx.c ) diff --git a/package.xml b/package.xml index d8e82a8bc37..ba6e11fadc7 100644 --- a/package.xml +++ b/package.xml @@ -198,7 +198,6 @@ - @@ -347,7 +346,6 @@ - diff --git a/src/core/lib/iomgr/ctiller_ev_epoll_linux.c b/src/core/lib/iomgr/ctiller_ev_epoll_linux.c deleted file mode 100644 index 23c20a77aae..00000000000 --- a/src/core/lib/iomgr/ctiller_ev_epoll_linux.c +++ /dev/null @@ -1,461 +0,0 @@ -/* - * - * Copyright 2015-2016, Google Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are - * met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following disclaimer - * in the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Google Inc. nor the names of its - * contributors may be used to endorse or promote products derived from - * this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - * - */ - -#include "src/core/lib/iomgr/ev_epoll_linux.h" - -#include -#include -#include -#include - -#include -#include -#include -#include - -#include "src/core/lib/iomgr/iomgr_internal.h" - -/* TODO(sreek) Remove this file */ - - -//////////////////////////////////////////////////////////////////////////////// -// Definitions - -#define STATE_NOT_READY ((gpr_atm)0) -#define STATE_READY ((gpr_atm)1) - -typedef enum { POLLABLE_FD, POLLABLE_EPOLL_SET } pollable_type; - -typedef struct { - pollable_type type; - int fd; - grpc_iomgr_object iomgr_object; -} pollable_object; - -typedef struct polling_island { - pollable_object pollable; - gpr_mu mu; - int refs; - grpc_fd *only_fd; - struct polling_island *became; - struct polling_island *next; -} polling_island; - -struct grpc_fd { - pollable_object pollable; - - // each event atomic is a tri state: - // STATE_NOT_READY - no event received, nobody waiting for it either - // STATE_READY - event received, nobody waiting for it - // closure pointer - no event received, upper layer is waiting for it - gpr_atm on_readable; - gpr_atm on_writable; - - // mutex guarding set_ready & shutdown state - gpr_mu set_ready_mu; - bool shutdown; - - // mutex protecting polling_island - gpr_mu polling_island_mu; - // current polling island - polling_island *polling_island; - - grpc_fd *next_free; -}; - -struct grpc_pollset_worker {}; - -struct grpc_pollset { - gpr_mu mu; - // current polling island - polling_island *polling_island; -}; - -//////////////////////////////////////////////////////////////////////////////// -// Polling island implementation - -static gpr_mu g_pi_freelist_mu; -static polling_island *g_first_free_pi; - -static void add_pollable_to_epoll_set(pollable_object *pollable, int epoll_set, - uint32_t events) { - struct epoll_event ev; - ev.events = events; - ev.data.ptr = pollable; - int err = epoll_ctl(epoll_set, EPOLL_CTL_ADD, pollable->fd, &ev); - if (err < 0) { - gpr_log(GPR_ERROR, "epoll_ctl add for %d faild: %s", pollable->fd, - strerror(errno)); - } -} - -static void add_fd_to_epoll_set(grpc_fd *fd, int epoll_set) { - add_pollable_to_epoll_set(&fd->pollable, epoll_set, - EPOLLIN | EPOLLOUT | EPOLLET); -} - -static void add_island_to_epoll_set(polling_island *pi, int epoll_set) { - add_pollable_to_epoll_set(&pi->pollable, epoll_set, EPOLLIN | EPOLLET); -} - -static polling_island *polling_island_create(grpc_fd *initial_fd) { - polling_island *r = NULL; - gpr_mu_lock(&g_pi_freelist_mu); - if (g_first_free_pi == NULL) { - r = gpr_malloc(sizeof(*r)); - r->pollable.type = POLLABLE_EPOLL_SET; - gpr_mu_init(&r->mu); - } else { - r = g_first_free_pi; - g_first_free_pi = r->next; - } - gpr_mu_unlock(&g_pi_freelist_mu); - - r->pollable.fd = epoll_create1(EPOLL_CLOEXEC); - GPR_ASSERT(r->pollable.fd >= 0); - - gpr_mu_lock(&r->mu); - r->only_fd = initial_fd; - r->refs = 2; // creation of a polling island => a referencing pollset & fd - gpr_mu_unlock(&r->mu); - - add_fd_to_epoll_set(initial_fd, r->pollable.fd); - return r; -} - -static void polling_island_delete(polling_island *p) { - gpr_mu_lock(&g_pi_freelist_mu); - p->next = g_first_free_pi; - g_first_free_pi = p; - gpr_mu_unlock(&g_pi_freelist_mu); -} - -static polling_island *polling_island_add(polling_island *p, grpc_fd *fd) { - gpr_mu_lock(&p->mu); - p->only_fd = NULL; - p->refs++; // new fd picks up a ref - gpr_mu_unlock(&p->mu); - - add_fd_to_epoll_set(fd, p->pollable.fd); - - return p; -} - -static void add_siblings_to(polling_island *siblings, polling_island *dest) { - polling_island *sibling_tail = dest; - while (sibling_tail->next != NULL) { - sibling_tail = sibling_tail->next; - } - sibling_tail->next = siblings; -} - -static polling_island *polling_island_merge(polling_island *a, - polling_island *b) { - GPR_ASSERT(a != b); - polling_island *out; - - gpr_mu_lock(&GPR_MIN(a, b)->mu); - gpr_mu_lock(&GPR_MAX(a, b)->mu); - - GPR_ASSERT(a->became == NULL); - GPR_ASSERT(b->became == NULL); - - if (a->only_fd == NULL && b->only_fd == NULL) { - b->became = a; - add_siblings_to(b, a); - add_island_to_epoll_set(b, a->pollable.fd); - out = a; - } else if (a->only_fd == NULL) { - GPR_ASSERT(b->only_fd != NULL); - add_fd_to_epoll_set(b->only_fd, a->pollable.fd); - b->became = a; - out = a; - } else if (b->only_fd == NULL) { - GPR_ASSERT(a->only_fd != NULL); - add_fd_to_epoll_set(a->only_fd, b->pollable.fd); - a->became = b; - out = b; - } else { - add_fd_to_epoll_set(b->only_fd, a->pollable.fd); - a->only_fd = NULL; - b->only_fd = NULL; - b->became = a; - out = a; - } - - gpr_mu_unlock(&a->mu); - gpr_mu_unlock(&b->mu); - - return out; -} - -static polling_island *polling_island_update_and_lock(polling_island *p) { - gpr_mu_lock(&p->mu); - if (p->became != NULL) { - do { - polling_island *from = p; - p = p->became; - gpr_mu_lock(&p->mu); - bool delete_from = 0 == --from->refs; - p->refs++; - gpr_mu_unlock(&from->mu); - if (delete_from) { - polling_island_delete(from); - } - } while (p->became != NULL); - } - return p; -} - -static polling_island *polling_island_ref(polling_island *p) { - gpr_mu_lock(&p->mu); - gpr_mu_unlock(&p->mu); - return p; -} - -static void polling_island_drop(polling_island *p) {} - -static polling_island *polling_island_update(polling_island *p, - int updating_owner_count) { - p = polling_island_update_and_lock(p); - GPR_ASSERT(p->refs != 0); - p->refs += updating_owner_count; - gpr_mu_unlock(&p->mu); - return p; -} - -//////////////////////////////////////////////////////////////////////////////// -// FD implementation - -static gpr_mu g_fd_freelist_mu; -static grpc_fd *g_first_free_fd; - -static grpc_fd *fd_create(int fd, const char *name) { - grpc_fd *r = NULL; - gpr_mu_lock(&g_fd_freelist_mu); - if (g_first_free_fd == NULL) { - r = gpr_malloc(sizeof(*r)); - r->pollable.type = POLLABLE_FD; - gpr_atm_rel_store(&r->on_readable, 0); - gpr_atm_rel_store(&r->on_writable, 0); - gpr_mu_init(&r->polling_island_mu); - gpr_mu_init(&r->set_ready_mu); - } else { - r = g_first_free_fd; - g_first_free_fd = r->next_free; - } - gpr_mu_unlock(&g_fd_freelist_mu); - - r->pollable.fd = fd; - grpc_iomgr_register_object(&r->pollable.iomgr_object, name); - r->next_free = NULL; - return r; -} - -static int fd_wrapped_fd(grpc_fd *fd) { return fd->pollable.fd; } - -static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure *on_done, int *release_fd, - const char *reason) { - if (release_fd != NULL) { - *release_fd = fd->pollable.fd; - } else { - close(fd->pollable.fd); - } - - gpr_mu_lock(&fd->polling_island_mu); - if (fd->polling_island != NULL) { - polling_island_drop(fd->polling_island); - } - gpr_mu_unlock(&fd->polling_island_mu); - - gpr_mu_lock(&g_fd_freelist_mu); - fd->next_free = g_first_free_fd; - g_first_free_fd = fd; - grpc_iomgr_unregister_object(&fd->pollable.iomgr_object); - gpr_mu_unlock(&g_fd_freelist_mu); - - grpc_exec_ctx_enqueue(exec_ctx, on_done, true, NULL); -} - -static void notify_on(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure *closure, gpr_atm *state) { - if (gpr_atm_acq_cas(state, STATE_NOT_READY, (gpr_atm)closure)) { - // state was not ready, and is now the closure - we're done */ - } else { - // cas failed - we MUST be in STATE_READY (can't request two notifications - // for the same event) - // flip back to not ready, enqueue the closure directly - GPR_ASSERT(gpr_atm_rel_cas(state, STATE_READY, STATE_NOT_READY)); - grpc_exec_ctx_enqueue(exec_ctx, closure, true, NULL); - } -} - -static void fd_notify_on_read(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure *closure) { - notify_on(exec_ctx, fd, closure, &fd->on_readable); -} - -static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure *closure) { - notify_on(exec_ctx, fd, closure, &fd->on_readable); -} - -static void destroy_fd_freelist(void) { - while (g_first_free_fd) { - grpc_fd *next = g_first_free_fd->next_free; - gpr_mu_destroy(&g_first_free_fd->polling_island_mu); - gpr_free(next); - g_first_free_fd = next; - } -} - -static void set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - gpr_atm *state) { - if (gpr_atm_acq_cas(state, STATE_NOT_READY, STATE_READY)) { - // state was not ready, and is now ready - we're done - } else { - // cas failed - either there's a closure queued which we should consume OR - // the state was already STATE_READY - gpr_atm cur_state = gpr_atm_acq_load(state); - if (cur_state != STATE_READY) { - // state wasn't STATE_READY - it *must* have been a closure - // since it's illegal to ask for notification twice, it's safe to assume - // that we'll resume being the closure - GPR_ASSERT(gpr_atm_rel_cas(state, cur_state, STATE_NOT_READY)); - grpc_exec_ctx_enqueue(exec_ctx, (grpc_closure *)cur_state, !fd->shutdown, - NULL); - } - } -} - -static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - gpr_mu_lock(&fd->set_ready_mu); - GPR_ASSERT(!fd->shutdown); - fd->shutdown = 1; - set_ready_locked(exec_ctx, fd, &fd->on_readable); - set_ready_locked(exec_ctx, fd, &fd->on_writable); - gpr_mu_unlock(&fd->set_ready_mu); -} - -//////////////////////////////////////////////////////////////////////////////// -// Pollset implementation - -static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { - gpr_mu_init(&pollset->mu); - *mu = &pollset->mu; - pollset->polling_island = NULL; -} - -static void pollset_destroy(grpc_pollset *pollset) { - gpr_mu_destroy(&pollset->mu); - if (pollset->polling_island) { - polling_island_drop(pollset->polling_island); - } -} - -static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - struct grpc_fd *fd) { - gpr_mu_lock(&pollset->mu); - gpr_mu_lock(&fd->polling_island_mu); - - polling_island *new; - - if (fd->polling_island == NULL) { - if (pollset->polling_island == NULL) { - new = polling_island_create(fd); - } else { - new = polling_island_add(pollset->polling_island, fd); - } - } else if (pollset->polling_island == NULL) { - new = polling_island_ref(fd->polling_island); - } else if (pollset->polling_island != fd->polling_island) { - new = polling_island_merge(pollset->polling_island, fd->polling_island); - } else { - new = polling_island_update(pollset->polling_island, 1); - } - - fd->polling_island = pollset->polling_island = new; - - gpr_mu_unlock(&fd->polling_island_mu); - gpr_mu_unlock(&pollset->mu); -} - -//////////////////////////////////////////////////////////////////////////////// -// Engine binding - -static void shutdown_engine(void) { destroy_fd_freelist(); } - -static const grpc_event_engine_vtable vtable = { - .pollset_size = sizeof(grpc_pollset), - - .fd_create = fd_create, - .fd_wrapped_fd = fd_wrapped_fd, - .fd_orphan = fd_orphan, - .fd_shutdown = fd_shutdown, - .fd_notify_on_read = fd_notify_on_read, - .fd_notify_on_write = fd_notify_on_write, - - .pollset_init = pollset_init, - .pollset_shutdown = pollset_shutdown, - .pollset_reset = pollset_reset, - .pollset_destroy = pollset_destroy, - .pollset_work = pollset_work, - .pollset_kick = pollset_kick, - .pollset_add_fd = pollset_add_fd, - - .pollset_set_create = pollset_set_create, - .pollset_set_destroy = pollset_set_destroy, - .pollset_set_add_pollset = pollset_set_add_pollset, - .pollset_set_del_pollset = pollset_set_del_pollset, - .pollset_set_add_pollset_set = pollset_set_add_pollset_set, - .pollset_set_del_pollset_set = pollset_set_del_pollset_set, - .pollset_set_add_fd = pollset_set_add_fd, - .pollset_set_del_fd = pollset_set_del_fd, - - .kick_poller = kick_poller, - - .shutdown_engine = shutdown_engine, -}; - -static bool is_epoll_available(void) { - abort(); - return false; -} - -const grpc_event_engine_vtable *grpc_init_poll_posix(void) { - if (!is_epoll_available()) { - return NULL; - } - return &vtable; -} diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 3aa26109f22..61106faef90 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -35,7 +35,7 @@ #ifdef GPR_POSIX_SOCKET -#include "src/core/lib/iomgr/ev_epoll_posix.h" +#include "src/core/lib/iomgr/ev_epoll_linux.h" #include #include diff --git a/src/core/lib/iomgr/ev_epoll_posix.c b/src/core/lib/iomgr/ev_epoll_posix.c deleted file mode 100644 index 5abd5b2a94c..00000000000 --- a/src/core/lib/iomgr/ev_epoll_posix.c +++ /dev/null @@ -1,1209 +0,0 @@ -/* - * - * Copyright 2016, Google Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are - * met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following disclaimer - * in the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Google Inc. nor the names of its - * contributors may be used to endorse or promote products derived from - * this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - * - */ - -#include - -#ifdef GPR_POSIX_SOCKET - -#include "src/core/lib/iomgr/ev_epoll_posix.h" - -#include -#include -#include -#include -#include -#include -#include -#include - -#include -#include -#include -#include -#include - -#include "src/core/lib/iomgr/ev_posix.h" -#include "src/core/lib/iomgr/iomgr_internal.h" -#include "src/core/lib/iomgr/wakeup_fd_posix.h" -#include "src/core/lib/profiling/timers.h" -#include "src/core/lib/support/block_annotate.h" - - -/******************************************************************************* - * FD declarations - */ - -struct grpc_fd { - int fd; - /* refst format: - bit0: 1=active/0=orphaned - bit1-n: refcount - meaning that mostly we ref by two to avoid altering the orphaned bit, - and just unref by 1 when we're ready to flag the object as orphaned */ - gpr_atm refst; - - gpr_mu mu; - int shutdown; - int closed; - int released; - - grpc_closure *read_closure; - grpc_closure *write_closure; - - struct grpc_fd *freelist_next; - - grpc_closure *on_done_closure; - - grpc_iomgr_object iomgr_object; -}; - -/* Return 1 if this fd is orphaned, 0 otherwise */ -static bool fd_is_orphaned(grpc_fd *fd); - -/* Reference counting for fds */ -/*#define GRPC_FD_REF_COUNT_DEBUG*/ -#ifdef GRPC_FD_REF_COUNT_DEBUG -static void fd_ref(grpc_fd *fd, const char *reason, const char *file, int line); -static void fd_unref(grpc_fd *fd, const char *reason, const char *file, - int line); -#define GRPC_FD_REF(fd, reason) fd_ref(fd, reason, __FILE__, __LINE__) -#define GRPC_FD_UNREF(fd, reason) fd_unref(fd, reason, __FILE__, __LINE__) -#else -static void fd_ref(grpc_fd *fd); -static void fd_unref(grpc_fd *fd); -#define GRPC_FD_REF(fd, reason) fd_ref(fd) -#define GRPC_FD_UNREF(fd, reason) fd_unref(fd) -#endif - -static void fd_global_init(void); -static void fd_global_shutdown(void); - -#define CLOSURE_NOT_READY ((grpc_closure *)0) -#define CLOSURE_READY ((grpc_closure *)1) - -/******************************************************************************* - * pollset declarations - */ - -typedef struct grpc_cached_wakeup_fd { - grpc_wakeup_fd fd; - struct grpc_cached_wakeup_fd *next; -} grpc_cached_wakeup_fd; - -struct grpc_pollset_worker { - grpc_cached_wakeup_fd *wakeup_fd; - int reevaluate_polling_on_wakeup; - int kicked_specifically; - pthread_t pt_id; - struct grpc_pollset_worker *next; - struct grpc_pollset_worker *prev; -}; - -struct grpc_pollset { - gpr_mu mu; - grpc_pollset_worker root_worker; - int shutting_down; - int called_shutdown; - int kicked_without_pollers; - grpc_closure *shutdown_done; - - int epoll_fd; - - /* Local cache of eventfds for workers */ - grpc_cached_wakeup_fd *local_wakeup_cache; -}; - -/* Add an fd to a pollset */ -static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - struct grpc_fd *fd); - -static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, grpc_fd *fd); - -/* Convert a timespec to milliseconds: - - very small or negative poll times are clamped to zero to do a - non-blocking poll (which becomes spin polling) - - other small values are rounded up to one millisecond - - longer than a millisecond polls are rounded up to the next nearest - millisecond to avoid spinning - - infinite timeouts are converted to -1 */ -static int poll_deadline_to_millis_timeout(gpr_timespec deadline, - gpr_timespec now); - -/* Allow kick to wakeup the currently polling worker */ -#define GRPC_POLLSET_CAN_KICK_SELF 1 -/* Force the wakee to repoll when awoken */ -#define GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP 2 -/* As per pollset_kick, with an extended set of flags (defined above) - -- mostly for fd_posix's use. */ -static void pollset_kick_ext(grpc_pollset *p, - grpc_pollset_worker *specific_worker, - uint32_t flags); - -/* turn a pollset into a multipoller: platform specific */ -typedef void (*platform_become_multipoller_type)(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, - struct grpc_fd **fds, - size_t fd_count); - -/* Return 1 if the pollset has active threads in pollset_work (pollset must - * be locked) */ -static int pollset_has_workers(grpc_pollset *pollset); - -static void remove_fd_from_all_epoll_sets(int fd); - -/******************************************************************************* - * pollset_set definitions - */ - -struct grpc_pollset_set { - gpr_mu mu; - - size_t pollset_count; - size_t pollset_capacity; - grpc_pollset **pollsets; - - size_t pollset_set_count; - size_t pollset_set_capacity; - struct grpc_pollset_set **pollset_sets; - - size_t fd_count; - size_t fd_capacity; - grpc_fd **fds; -}; - -/******************************************************************************* - * fd_posix.c - */ - -/* We need to keep a freelist not because of any concerns of malloc performance - * but instead so that implementations with multiple threads in (for example) - * epoll_wait deal with the race between pollset removal and incoming poll - * notifications. - * - * The problem is that the poller ultimately holds a reference to this - * object, so it is very difficult to know when is safe to free it, at least - * without some expensive synchronization. - * - * If we keep the object freelisted, in the worst case losing this race just - * becomes a spurious read notification on a reused fd. - */ -/* TODO(klempner): We could use some form of polling generation count to know - * when these are safe to free. */ -/* TODO(klempner): Consider disabling freelisting if we don't have multiple - * threads in poll on the same fd */ -/* TODO(klempner): Batch these allocations to reduce fragmentation */ -static grpc_fd *fd_freelist = NULL; -static gpr_mu fd_freelist_mu; - -static void freelist_fd(grpc_fd *fd) { - gpr_mu_lock(&fd_freelist_mu); - fd->freelist_next = fd_freelist; - fd_freelist = fd; - grpc_iomgr_unregister_object(&fd->iomgr_object); - gpr_mu_unlock(&fd_freelist_mu); -} - -static grpc_fd *alloc_fd(int fd) { - grpc_fd *r = NULL; - gpr_mu_lock(&fd_freelist_mu); - if (fd_freelist != NULL) { - r = fd_freelist; - fd_freelist = fd_freelist->freelist_next; - } - gpr_mu_unlock(&fd_freelist_mu); - if (r == NULL) { - r = gpr_malloc(sizeof(grpc_fd)); - gpr_mu_init(&r->mu); - } - - gpr_mu_lock(&r->mu); - gpr_atm_rel_store(&r->refst, 1); - r->shutdown = 0; - r->read_closure = CLOSURE_NOT_READY; - r->write_closure = CLOSURE_NOT_READY; - r->fd = fd; - r->freelist_next = NULL; - r->on_done_closure = NULL; - r->closed = 0; - r->released = 0; - gpr_mu_unlock(&r->mu); - return r; -} - -static void destroy(grpc_fd *fd) { - gpr_mu_destroy(&fd->mu); - gpr_free(fd); -} - -#ifdef GRPC_FD_REF_COUNT_DEBUG -#define REF_BY(fd, n, reason) ref_by(fd, n, reason, __FILE__, __LINE__) -#define UNREF_BY(fd, n, reason) unref_by(fd, n, reason, __FILE__, __LINE__) -static void ref_by(grpc_fd *fd, int n, const char *reason, const char *file, - int line) { - gpr_log(GPR_DEBUG, "FD %d %p ref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, - gpr_atm_no_barrier_load(&fd->refst), - gpr_atm_no_barrier_load(&fd->refst) + n, reason, file, line); -#else -#define REF_BY(fd, n, reason) ref_by(fd, n) -#define UNREF_BY(fd, n, reason) unref_by(fd, n) -static void ref_by(grpc_fd *fd, int n) { -#endif - GPR_ASSERT(gpr_atm_no_barrier_fetch_add(&fd->refst, n) > 0); -} - -#ifdef GRPC_FD_REF_COUNT_DEBUG -static void unref_by(grpc_fd *fd, int n, const char *reason, const char *file, - int line) { - gpr_atm old; - gpr_log(GPR_DEBUG, "FD %d %p unref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, - gpr_atm_no_barrier_load(&fd->refst), - gpr_atm_no_barrier_load(&fd->refst) - n, reason, file, line); -#else -static void unref_by(grpc_fd *fd, int n) { - gpr_atm old; -#endif - old = gpr_atm_full_fetch_add(&fd->refst, -n); - if (old == n) { - freelist_fd(fd); - } else { - GPR_ASSERT(old > n); - } -} - -static void fd_global_init(void) { gpr_mu_init(&fd_freelist_mu); } - -static void fd_global_shutdown(void) { - gpr_mu_lock(&fd_freelist_mu); - gpr_mu_unlock(&fd_freelist_mu); - while (fd_freelist != NULL) { - grpc_fd *fd = fd_freelist; - fd_freelist = fd_freelist->freelist_next; - destroy(fd); - } - gpr_mu_destroy(&fd_freelist_mu); -} - -static grpc_fd *fd_create(int fd, const char *name) { - grpc_fd *r = alloc_fd(fd); - char *name2; - gpr_asprintf(&name2, "%s fd=%d", name, fd); - grpc_iomgr_register_object(&r->iomgr_object, name2); - gpr_free(name2); -#ifdef GRPC_FD_REF_COUNT_DEBUG - gpr_log(GPR_DEBUG, "FD %d %p create %s", fd, r, name); -#endif - return r; -} - -static bool fd_is_orphaned(grpc_fd *fd) { - return (gpr_atm_acq_load(&fd->refst) & 1) == 0; -} - -static void close_fd_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - fd->closed = 1; - if (!fd->released) { - close(fd->fd); - } else { - remove_fd_from_all_epoll_sets(fd->fd); - } - grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); -} - -static int fd_wrapped_fd(grpc_fd *fd) { - if (fd->released || fd->closed) { - return -1; - } else { - return fd->fd; - } -} - -static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure *on_done, int *release_fd, - const char *reason) { - fd->on_done_closure = on_done; - fd->released = release_fd != NULL; - if (!fd->released) { - shutdown(fd->fd, SHUT_RDWR); - } else { - *release_fd = fd->fd; - } - gpr_mu_lock(&fd->mu); - REF_BY(fd, 1, reason); /* remove active status, but keep referenced */ - close_fd_locked(exec_ctx, fd); - gpr_mu_unlock(&fd->mu); - UNREF_BY(fd, 2, reason); /* drop the reference */ -} - -/* increment refcount by two to avoid changing the orphan bit */ -#ifdef GRPC_FD_REF_COUNT_DEBUG -static void fd_ref(grpc_fd *fd, const char *reason, const char *file, - int line) { - ref_by(fd, 2, reason, file, line); -} - -static void fd_unref(grpc_fd *fd, const char *reason, const char *file, - int line) { - unref_by(fd, 2, reason, file, line); -} -#else -static void fd_ref(grpc_fd *fd) { ref_by(fd, 2); } - -static void fd_unref(grpc_fd *fd) { unref_by(fd, 2); } -#endif - -static void notify_on_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure **st, grpc_closure *closure) { - if (*st == CLOSURE_NOT_READY) { - /* not ready ==> switch to a waiting state by setting the closure */ - *st = closure; - } else if (*st == CLOSURE_READY) { - /* already ready ==> queue the closure to run immediately */ - *st = CLOSURE_NOT_READY; - grpc_exec_ctx_enqueue(exec_ctx, closure, !fd->shutdown, NULL); - } else { - /* upcallptr was set to a different closure. This is an error! */ - gpr_log(GPR_ERROR, - "User called a notify_on function with a previous callback still " - "pending"); - abort(); - } -} - -/* returns 1 if state becomes not ready */ -static int set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure **st) { - if (*st == CLOSURE_READY) { - /* duplicate ready ==> ignore */ - return 0; - } else if (*st == CLOSURE_NOT_READY) { - /* not ready, and not waiting ==> flag ready */ - *st = CLOSURE_READY; - return 0; - } else { - /* waiting ==> queue closure */ - grpc_exec_ctx_enqueue(exec_ctx, *st, !fd->shutdown, NULL); - *st = CLOSURE_NOT_READY; - return 1; - } -} - -static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - gpr_mu_lock(&fd->mu); - GPR_ASSERT(!fd->shutdown); - fd->shutdown = 1; - set_ready_locked(exec_ctx, fd, &fd->read_closure); - set_ready_locked(exec_ctx, fd, &fd->write_closure); - gpr_mu_unlock(&fd->mu); -} - -static void fd_notify_on_read(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure *closure) { - gpr_mu_lock(&fd->mu); - notify_on_locked(exec_ctx, fd, &fd->read_closure, closure); - gpr_mu_unlock(&fd->mu); -} - -static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, - grpc_closure *closure) { - gpr_mu_lock(&fd->mu); - notify_on_locked(exec_ctx, fd, &fd->write_closure, closure); - gpr_mu_unlock(&fd->mu); -} - -/******************************************************************************* - * pollset_posix.c - */ - -GPR_TLS_DECL(g_current_thread_poller); -GPR_TLS_DECL(g_current_thread_worker); - -/** The alarm system needs to be able to wakeup 'some poller' sometimes - * (specifically when a new alarm needs to be triggered earlier than the next - * alarm 'epoch'). - * This wakeup_fd gives us something to alert on when such a case occurs. */ -grpc_wakeup_fd grpc_global_wakeup_fd; - -static void remove_worker(grpc_pollset *p, grpc_pollset_worker *worker) { - worker->prev->next = worker->next; - worker->next->prev = worker->prev; -} - -static int pollset_has_workers(grpc_pollset *p) { - return p->root_worker.next != &p->root_worker; -} - -static grpc_pollset_worker *pop_front_worker(grpc_pollset *p) { - if (pollset_has_workers(p)) { - grpc_pollset_worker *w = p->root_worker.next; - remove_worker(p, w); - return w; - } else { - return NULL; - } -} - -static void push_back_worker(grpc_pollset *p, grpc_pollset_worker *worker) { - worker->next = &p->root_worker; - worker->prev = worker->next->prev; - worker->prev->next = worker->next->prev = worker; -} - -static void push_front_worker(grpc_pollset *p, grpc_pollset_worker *worker) { - worker->prev = &p->root_worker; - worker->next = worker->prev->next; - worker->prev->next = worker->next->prev = worker; -} - -static void pollset_kick_ext(grpc_pollset *p, - grpc_pollset_worker *specific_worker, - uint32_t flags) { - GPR_TIMER_BEGIN("pollset_kick_ext", 0); - - /* pollset->mu already held */ - if (specific_worker != NULL) { - if (specific_worker == GRPC_POLLSET_KICK_BROADCAST) { - GPR_TIMER_BEGIN("pollset_kick_ext.broadcast", 0); - GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); - for (specific_worker = p->root_worker.next; - specific_worker != &p->root_worker; - specific_worker = specific_worker->next) { - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); - } - p->kicked_without_pollers = 1; - GPR_TIMER_END("pollset_kick_ext.broadcast", 0); - } else if (gpr_tls_get(&g_current_thread_worker) != - (intptr_t)specific_worker) { - GPR_TIMER_MARK("different_thread_worker", 0); - if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { - specific_worker->reevaluate_polling_on_wakeup = 1; - } - specific_worker->kicked_specifically = 1; - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); - /* TODO (sreek): Refactor this into a separate file*/ - pthread_kill(specific_worker->pt_id, SIGUSR1); - } else if ((flags & GRPC_POLLSET_CAN_KICK_SELF) != 0) { - GPR_TIMER_MARK("kick_yoself", 0); - if ((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) != 0) { - specific_worker->reevaluate_polling_on_wakeup = 1; - } - specific_worker->kicked_specifically = 1; - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); - } - } else if (gpr_tls_get(&g_current_thread_poller) != (intptr_t)p) { - GPR_ASSERT((flags & GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) == 0); - GPR_TIMER_MARK("kick_anonymous", 0); - specific_worker = pop_front_worker(p); - if (specific_worker != NULL) { - if (gpr_tls_get(&g_current_thread_worker) == (intptr_t)specific_worker) { - GPR_TIMER_MARK("kick_anonymous_not_self", 0); - push_back_worker(p, specific_worker); - specific_worker = pop_front_worker(p); - if ((flags & GRPC_POLLSET_CAN_KICK_SELF) == 0 && - gpr_tls_get(&g_current_thread_worker) == - (intptr_t)specific_worker) { - push_back_worker(p, specific_worker); - specific_worker = NULL; - } - } - if (specific_worker != NULL) { - GPR_TIMER_MARK("finally_kick", 0); - push_back_worker(p, specific_worker); - grpc_wakeup_fd_wakeup(&specific_worker->wakeup_fd->fd); - } - } else { - GPR_TIMER_MARK("kicked_no_pollers", 0); - p->kicked_without_pollers = 1; - } - } - - GPR_TIMER_END("pollset_kick_ext", 0); -} - -static void pollset_kick(grpc_pollset *p, - grpc_pollset_worker *specific_worker) { - pollset_kick_ext(p, specific_worker, 0); -} - -/* global state management */ - -static void sig_handler(int sig_num) { - gpr_log(GPR_INFO, "Received signal %d", sig_num); -} - -static void pollset_global_init(void) { - gpr_tls_init(&g_current_thread_poller); - gpr_tls_init(&g_current_thread_worker); - grpc_wakeup_fd_init(&grpc_global_wakeup_fd); - signal(SIGUSR1, sig_handler); -} - -static void pollset_global_shutdown(void) { - grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); - gpr_tls_destroy(&g_current_thread_poller); - gpr_tls_destroy(&g_current_thread_worker); -} - -static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } - -/* TODO: sreek. Try to Remove this forward declaration*/ -static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset); - -/* main interface */ - -static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { - gpr_mu_init(&pollset->mu); - *mu = &pollset->mu; - pollset->root_worker.next = pollset->root_worker.prev = &pollset->root_worker; - pollset->shutting_down = 0; - pollset->called_shutdown = 0; - pollset->kicked_without_pollers = 0; - pollset->local_wakeup_cache = NULL; - pollset->kicked_without_pollers = 0; - - multipoll_with_epoll_pollset_create_efd(pollset); -} - -/* TODO(sreek): Maybe merge multipoll_*_destroy() with pollset_destroy() - * function */ -static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset); - -static void pollset_destroy(grpc_pollset *pollset) { - GPR_ASSERT(!pollset_has_workers(pollset)); - - multipoll_with_epoll_pollset_destroy(pollset); - - while (pollset->local_wakeup_cache) { - grpc_cached_wakeup_fd *next = pollset->local_wakeup_cache->next; - grpc_wakeup_fd_destroy(&pollset->local_wakeup_cache->fd); - gpr_free(pollset->local_wakeup_cache); - pollset->local_wakeup_cache = next; - } - gpr_mu_destroy(&pollset->mu); -} - -static void pollset_reset(grpc_pollset *pollset) { - GPR_ASSERT(pollset->shutting_down); - GPR_ASSERT(!pollset_has_workers(pollset)); - pollset->shutting_down = 0; - pollset->called_shutdown = 0; - pollset->kicked_without_pollers = 0; -} - -/* TODO (sreek): Remove multipoll_with_epoll_finish_shutdown() declaration */ -static void multipoll_with_epoll_pollset_finish_shutdown(grpc_pollset *pollset); - -static void finish_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset) { - multipoll_with_epoll_pollset_finish_shutdown(pollset); - grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); -} - -/* TODO(sreek): Remove multipoll_with_epoll_*_maybe_work_and_unlock declaration - */ -static void multipoll_with_epoll_pollset_maybe_work_and_unlock( - grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, - gpr_timespec deadline, gpr_timespec now); - -static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_pollset_worker **worker_hdl, gpr_timespec now, - gpr_timespec deadline) { - grpc_pollset_worker worker; - *worker_hdl = &worker; - - /* pollset->mu already held */ - int added_worker = 0; - int locked = 1; - int queued_work = 0; - int keep_polling = 0; - GPR_TIMER_BEGIN("pollset_work", 0); - /* this must happen before we (potentially) drop pollset->mu */ - worker.next = worker.prev = NULL; - worker.reevaluate_polling_on_wakeup = 0; - if (pollset->local_wakeup_cache != NULL) { - worker.wakeup_fd = pollset->local_wakeup_cache; - pollset->local_wakeup_cache = worker.wakeup_fd->next; - } else { - worker.wakeup_fd = gpr_malloc(sizeof(*worker.wakeup_fd)); - grpc_wakeup_fd_init(&worker.wakeup_fd->fd); - } - worker.kicked_specifically = 0; - - /* TODO(sreek): Abstract this thread id stuff out into a separate file */ - worker.pt_id = pthread_self(); - /* If we're shutting down then we don't execute any extended work */ - if (pollset->shutting_down) { - GPR_TIMER_MARK("pollset_work.shutting_down", 0); - goto done; - } - /* Start polling, and keep doing so while we're being asked to - re-evaluate our pollers (this allows poll() based pollers to - ensure they don't miss wakeups) */ - keep_polling = 1; - while (keep_polling) { - keep_polling = 0; - if (!pollset->kicked_without_pollers) { - if (!added_worker) { - push_front_worker(pollset, &worker); - added_worker = 1; - gpr_tls_set(&g_current_thread_worker, (intptr_t)&worker); - } - gpr_tls_set(&g_current_thread_poller, (intptr_t)pollset); - GPR_TIMER_BEGIN("maybe_work_and_unlock", 0); - - multipoll_with_epoll_pollset_maybe_work_and_unlock( - exec_ctx, pollset, &worker, deadline, now); - - GPR_TIMER_END("maybe_work_and_unlock", 0); - locked = 0; - gpr_tls_set(&g_current_thread_poller, 0); - } else { - GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); - pollset->kicked_without_pollers = 0; - } - /* Finished execution - start cleaning up. - Note that we may arrive here from outside the enclosing while() loop. - In that case we won't loop though as we haven't added worker to the - worker list, which means nobody could ask us to re-evaluate polling). */ - done: - if (!locked) { - queued_work |= grpc_exec_ctx_flush(exec_ctx); - gpr_mu_lock(&pollset->mu); - locked = 1; - } - /* If we're forced to re-evaluate polling (via pollset_kick with - GRPC_POLLSET_REEVALUATE_POLLING_ON_WAKEUP) then we land here and force - a loop */ - if (worker.reevaluate_polling_on_wakeup) { - worker.reevaluate_polling_on_wakeup = 0; - pollset->kicked_without_pollers = 0; - if (queued_work || worker.kicked_specifically) { - /* If there's queued work on the list, then set the deadline to be - immediate so we get back out of the polling loop quickly */ - deadline = gpr_inf_past(GPR_CLOCK_MONOTONIC); - } - keep_polling = 1; - } - } - if (added_worker) { - remove_worker(pollset, &worker); - gpr_tls_set(&g_current_thread_worker, 0); - } - /* release wakeup fd to the local pool */ - worker.wakeup_fd->next = pollset->local_wakeup_cache; - pollset->local_wakeup_cache = worker.wakeup_fd; - /* check shutdown conditions */ - if (pollset->shutting_down) { - if (pollset_has_workers(pollset)) { - pollset_kick(pollset, NULL); - } else if (!pollset->called_shutdown) { - pollset->called_shutdown = 1; - gpr_mu_unlock(&pollset->mu); - finish_shutdown(exec_ctx, pollset); - grpc_exec_ctx_flush(exec_ctx); - /* Continuing to access pollset here is safe -- it is the caller's - * responsibility to not destroy when it has outstanding calls to - * pollset_work. - * TODO(dklempner): Can we refactor the shutdown logic to avoid this? */ - gpr_mu_lock(&pollset->mu); - } - } - *worker_hdl = NULL; - GPR_TIMER_END("pollset_work", 0); -} - -static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_closure *closure) { - GPR_ASSERT(!pollset->shutting_down); - pollset->shutting_down = 1; - pollset->shutdown_done = closure; - pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); - - if (!pollset->called_shutdown && !pollset_has_workers(pollset)) { - pollset->called_shutdown = 1; - finish_shutdown(exec_ctx, pollset); - } -} - -static int poll_deadline_to_millis_timeout(gpr_timespec deadline, - gpr_timespec now) { - gpr_timespec timeout; - static const int64_t max_spin_polling_us = 10; - if (gpr_time_cmp(deadline, gpr_inf_future(deadline.clock_type)) == 0) { - return -1; - } - if (gpr_time_cmp(deadline, gpr_time_add(now, gpr_time_from_micros( - max_spin_polling_us, - GPR_TIMESPAN))) <= 0) { - return 0; - } - timeout = gpr_time_sub(deadline, now); - return gpr_time_to_millis(gpr_time_add( - timeout, gpr_time_from_nanos(GPR_NS_PER_MS - 1, GPR_TIMESPAN))); -} - -/******************************************************************************* - * pollset_multipoller_with_epoll_posix.c - */ - -static void set_ready(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st) { - /* only one set_ready can be active at once (but there may be a racing - notify_on) */ - gpr_mu_lock(&fd->mu); - set_ready_locked(exec_ctx, fd, st); - gpr_mu_unlock(&fd->mu); -} - -static void fd_become_readable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - set_ready(exec_ctx, fd, &fd->read_closure); -} - -static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - set_ready(exec_ctx, fd, &fd->write_closure); -} - -/* TODO (sreek): Maybe this global list is not required. Double check*/ -struct epoll_fd_list { - int *epoll_fds; - size_t count; - size_t capacity; -}; - -static struct epoll_fd_list epoll_fd_global_list; -static gpr_once init_epoll_fd_list_mu = GPR_ONCE_INIT; -static gpr_mu epoll_fd_list_mu; - -static void init_mu(void) { gpr_mu_init(&epoll_fd_list_mu); } - -static void add_epoll_fd_to_global_list(int epoll_fd) { - gpr_once_init(&init_epoll_fd_list_mu, init_mu); - - gpr_mu_lock(&epoll_fd_list_mu); - if (epoll_fd_global_list.count == epoll_fd_global_list.capacity) { - epoll_fd_global_list.capacity = - GPR_MAX((size_t)8, epoll_fd_global_list.capacity * 2); - epoll_fd_global_list.epoll_fds = - gpr_realloc(epoll_fd_global_list.epoll_fds, - epoll_fd_global_list.capacity * sizeof(int)); - } - epoll_fd_global_list.epoll_fds[epoll_fd_global_list.count++] = epoll_fd; - gpr_mu_unlock(&epoll_fd_list_mu); -} - -static void remove_epoll_fd_from_global_list(int epoll_fd) { - gpr_mu_lock(&epoll_fd_list_mu); - GPR_ASSERT(epoll_fd_global_list.count > 0); - for (size_t i = 0; i < epoll_fd_global_list.count; i++) { - if (epoll_fd == epoll_fd_global_list.epoll_fds[i]) { - epoll_fd_global_list.epoll_fds[i] = - epoll_fd_global_list.epoll_fds[--(epoll_fd_global_list.count)]; - break; - } - } - gpr_mu_unlock(&epoll_fd_list_mu); -} - -static void remove_fd_from_all_epoll_sets(int fd) { - int err; - gpr_once_init(&init_epoll_fd_list_mu, init_mu); - gpr_mu_lock(&epoll_fd_list_mu); - if (epoll_fd_global_list.count == 0) { - gpr_mu_unlock(&epoll_fd_list_mu); - return; - } - for (size_t i = 0; i < epoll_fd_global_list.count; i++) { - err = epoll_ctl(epoll_fd_global_list.epoll_fds[i], EPOLL_CTL_DEL, fd, NULL); - if (err < 0 && errno != ENOENT) { - gpr_log(GPR_ERROR, "epoll_ctl del for %d failed: %s", fd, - strerror(errno)); - } - } - gpr_mu_unlock(&epoll_fd_list_mu); -} - -/* TODO: sreek - This function multipoll_with_epoll_pollset_add_fd() and - * finally_add_fd() in ev_poll_and_epoll_posix.c */ -static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_fd *fd) { - - /* TODO sreek - Check if we need to get a pollset->mu lock here */ - - struct epoll_event ev; - int err; - - /* Hold a ref to the fd to keep it from being closed during the add. This may - result in a spurious wakeup being assigned to this pollset whilst adding, - but that should be benign. */ - /* TODO: (sreek): Understand how a spurious wake up migh be assinged to this - * pollset..and how holding a reference will prevent the fd from being closed - * (and perhaps more importantly, see how can an fd be closed while being - * added to the epollset */ - GRPC_FD_REF(fd, "add fd"); - - gpr_mu_lock(&fd->mu); - if (fd->shutdown) { - gpr_mu_unlock(&fd->mu); - GRPC_FD_UNREF(fd, "add fd"); - return; - } - gpr_mu_unlock(&fd->mu); - - ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); - ev.data.ptr = fd; - err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, fd->fd, &ev); - if (err < 0) { - /* FDs may be added to a pollset multiple times, so EEXIST is normal. */ - if (errno != EEXIST) { - gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", fd->fd, - strerror(errno)); - } - } - - /* The fd might have been orphaned while we were adding it to the epoll set. - Close the fd in such a case (which will also take care of removing it from - the epoll set */ - gpr_mu_lock(&fd->mu); - if (fd_is_orphaned(fd) && !fd->closed) { - close_fd_locked(exec_ctx, fd); - } - gpr_mu_unlock(&fd->mu); - - GRPC_FD_UNREF(fd, "add fd"); -} - -/* Creates an epoll fd and initializes the pollset */ -/* TODO: This has to be called ONLY from pollset_init function. and hence it - * does not acquire any lock */ -static void multipoll_with_epoll_pollset_create_efd(grpc_pollset *pollset) { - struct epoll_event ev; - int err; - - pollset->epoll_fd = epoll_create1(EPOLL_CLOEXEC); - if (pollset->epoll_fd < 0) { - gpr_log(GPR_ERROR, "epoll_create1 failed: %s", strerror(errno)); - abort(); - } - add_epoll_fd_to_global_list(pollset->epoll_fd); - - ev.events = (uint32_t)(EPOLLIN | EPOLLET); - ev.data.ptr = NULL; - - err = epoll_ctl(pollset->epoll_fd, EPOLL_CTL_ADD, - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); - if (err < 0) { - gpr_log(GPR_ERROR, "epoll_ctl add for %d failed: %s", - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), - strerror(errno)); - } -} - -/* TODO(klempner): We probably want to turn this down a bit */ -#define GRPC_EPOLL_MAX_EVENTS 1000 - -static void multipoll_with_epoll_pollset_maybe_work_and_unlock( - grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker *worker, - gpr_timespec deadline, gpr_timespec now) { - struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; - int epoll_fd = pollset->epoll_fd; - int ep_rv; - int poll_rv; - int timeout_ms; - struct pollfd pfds[2]; - - /* If you want to ignore epoll's ability to sanely handle parallel pollers, - * for a more apples-to-apples performance comparison with poll, add a - * if (pollset->counter != 0) { return 0; } - * here. - */ - - gpr_mu_unlock(&pollset->mu); - - timeout_ms = poll_deadline_to_millis_timeout(deadline, now); - - pfds[0].fd = GRPC_WAKEUP_FD_GET_READ_FD(&worker->wakeup_fd->fd); - pfds[0].events = POLLIN; - pfds[0].revents = 0; - pfds[1].fd = epoll_fd; - pfds[1].events = POLLIN; - pfds[1].revents = 0; - - /* TODO(vpai): Consider first doing a 0 timeout poll here to avoid - even going into the blocking annotation if possible */ - GPR_TIMER_BEGIN("poll", 0); - GRPC_SCHEDULING_START_BLOCKING_REGION; - poll_rv = grpc_poll_function(pfds, 2, timeout_ms); - GRPC_SCHEDULING_END_BLOCKING_REGION; - GPR_TIMER_END("poll", 0); - - if (poll_rv < 0) { - if (errno != EINTR) { - gpr_log(GPR_ERROR, "poll() failed: %s", strerror(errno)); - } - } else if (poll_rv == 0) { - /* do nothing */ - } else { - if (pfds[0].revents) { - grpc_wakeup_fd_consume_wakeup(&worker->wakeup_fd->fd); - } - if (pfds[1].revents) { - do { - /* The following epoll_wait never blocks; it has a timeout of 0 */ - ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); - if (ep_rv < 0) { - if (errno != EINTR) { - gpr_log(GPR_ERROR, "epoll_wait() failed: %s", strerror(errno)); - } - } else { - int i; - for (i = 0; i < ep_rv; ++i) { - grpc_fd *fd = ep_ev[i].data.ptr; - /* TODO(klempner): We might want to consider making err and pri - * separate events */ - int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); - int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); - int write_ev = ep_ev[i].events & EPOLLOUT; - if (fd == NULL) { - grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); - } else { - if (read_ev || cancel) { - fd_become_readable(exec_ctx, fd); - } - if (write_ev || cancel) { - fd_become_writable(exec_ctx, fd); - } - } - } - } - } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); - } - } -} - -static void multipoll_with_epoll_pollset_finish_shutdown( - grpc_pollset *pollset) {} - -static void multipoll_with_epoll_pollset_destroy(grpc_pollset *pollset) { - close(pollset->epoll_fd); - remove_epoll_fd_from_global_list(pollset->epoll_fd); -} - -/******************************************************************************* - * pollset_set_posix.c - */ - -static grpc_pollset_set *pollset_set_create(void) { - grpc_pollset_set *pollset_set = gpr_malloc(sizeof(*pollset_set)); - memset(pollset_set, 0, sizeof(*pollset_set)); - gpr_mu_init(&pollset_set->mu); - return pollset_set; -} - -static void pollset_set_destroy(grpc_pollset_set *pollset_set) { - size_t i; - gpr_mu_destroy(&pollset_set->mu); - for (i = 0; i < pollset_set->fd_count; i++) { - GRPC_FD_UNREF(pollset_set->fds[i], "pollset_set"); - } - gpr_free(pollset_set->pollsets); - gpr_free(pollset_set->pollset_sets); - gpr_free(pollset_set->fds); - gpr_free(pollset_set); -} - -static void pollset_set_add_pollset(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, - grpc_pollset *pollset) { - size_t i, j; - gpr_mu_lock(&pollset_set->mu); - if (pollset_set->pollset_count == pollset_set->pollset_capacity) { - pollset_set->pollset_capacity = - GPR_MAX(8, 2 * pollset_set->pollset_capacity); - pollset_set->pollsets = - gpr_realloc(pollset_set->pollsets, pollset_set->pollset_capacity * - sizeof(*pollset_set->pollsets)); - } - pollset_set->pollsets[pollset_set->pollset_count++] = pollset; - for (i = 0, j = 0; i < pollset_set->fd_count; i++) { - if (fd_is_orphaned(pollset_set->fds[i])) { - GRPC_FD_UNREF(pollset_set->fds[i], "pollset_set"); - } else { - pollset_add_fd(exec_ctx, pollset, pollset_set->fds[i]); - pollset_set->fds[j++] = pollset_set->fds[i]; - } - } - pollset_set->fd_count = j; - gpr_mu_unlock(&pollset_set->mu); -} - -static void pollset_set_del_pollset(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, - grpc_pollset *pollset) { - size_t i; - gpr_mu_lock(&pollset_set->mu); - for (i = 0; i < pollset_set->pollset_count; i++) { - if (pollset_set->pollsets[i] == pollset) { - pollset_set->pollset_count--; - GPR_SWAP(grpc_pollset *, pollset_set->pollsets[i], - pollset_set->pollsets[pollset_set->pollset_count]); - break; - } - } - gpr_mu_unlock(&pollset_set->mu); -} - -static void pollset_set_add_pollset_set(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *bag, - grpc_pollset_set *item) { - size_t i, j; - gpr_mu_lock(&bag->mu); - if (bag->pollset_set_count == bag->pollset_set_capacity) { - bag->pollset_set_capacity = GPR_MAX(8, 2 * bag->pollset_set_capacity); - bag->pollset_sets = - gpr_realloc(bag->pollset_sets, - bag->pollset_set_capacity * sizeof(*bag->pollset_sets)); - } - bag->pollset_sets[bag->pollset_set_count++] = item; - for (i = 0, j = 0; i < bag->fd_count; i++) { - if (fd_is_orphaned(bag->fds[i])) { - GRPC_FD_UNREF(bag->fds[i], "pollset_set"); - } else { - pollset_set_add_fd(exec_ctx, item, bag->fds[i]); - bag->fds[j++] = bag->fds[i]; - } - } - bag->fd_count = j; - gpr_mu_unlock(&bag->mu); -} - -static void pollset_set_del_pollset_set(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *bag, - grpc_pollset_set *item) { - size_t i; - gpr_mu_lock(&bag->mu); - for (i = 0; i < bag->pollset_set_count; i++) { - if (bag->pollset_sets[i] == item) { - bag->pollset_set_count--; - GPR_SWAP(grpc_pollset_set *, bag->pollset_sets[i], - bag->pollset_sets[bag->pollset_set_count]); - break; - } - } - gpr_mu_unlock(&bag->mu); -} - -static void pollset_set_add_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, grpc_fd *fd) { - size_t i; - gpr_mu_lock(&pollset_set->mu); - if (pollset_set->fd_count == pollset_set->fd_capacity) { - pollset_set->fd_capacity = GPR_MAX(8, 2 * pollset_set->fd_capacity); - pollset_set->fds = gpr_realloc( - pollset_set->fds, pollset_set->fd_capacity * sizeof(*pollset_set->fds)); - } - GRPC_FD_REF(fd, "pollset_set"); - pollset_set->fds[pollset_set->fd_count++] = fd; - for (i = 0; i < pollset_set->pollset_count; i++) { - pollset_add_fd(exec_ctx, pollset_set->pollsets[i], fd); - } - for (i = 0; i < pollset_set->pollset_set_count; i++) { - pollset_set_add_fd(exec_ctx, pollset_set->pollset_sets[i], fd); - } - gpr_mu_unlock(&pollset_set->mu); -} - -static void pollset_set_del_fd(grpc_exec_ctx *exec_ctx, - grpc_pollset_set *pollset_set, grpc_fd *fd) { - size_t i; - gpr_mu_lock(&pollset_set->mu); - for (i = 0; i < pollset_set->fd_count; i++) { - if (pollset_set->fds[i] == fd) { - pollset_set->fd_count--; - GPR_SWAP(grpc_fd *, pollset_set->fds[i], - pollset_set->fds[pollset_set->fd_count]); - GRPC_FD_UNREF(fd, "pollset_set"); - break; - } - } - for (i = 0; i < pollset_set->pollset_set_count; i++) { - pollset_set_del_fd(exec_ctx, pollset_set->pollset_sets[i], fd); - } - gpr_mu_unlock(&pollset_set->mu); -} - -/******************************************************************************* - * event engine binding - */ - -static void shutdown_engine(void) { - fd_global_shutdown(); - pollset_global_shutdown(); -} - -static const grpc_event_engine_vtable vtable = { - .pollset_size = sizeof(grpc_pollset), - - .fd_create = fd_create, - .fd_wrapped_fd = fd_wrapped_fd, - .fd_orphan = fd_orphan, - .fd_shutdown = fd_shutdown, - .fd_notify_on_read = fd_notify_on_read, - .fd_notify_on_write = fd_notify_on_write, - - .pollset_init = pollset_init, - .pollset_shutdown = pollset_shutdown, - .pollset_reset = pollset_reset, - .pollset_destroy = pollset_destroy, - .pollset_work = pollset_work, - .pollset_kick = pollset_kick, - .pollset_add_fd = pollset_add_fd, - - .pollset_set_create = pollset_set_create, - .pollset_set_destroy = pollset_set_destroy, - .pollset_set_add_pollset = pollset_set_add_pollset, - .pollset_set_del_pollset = pollset_set_del_pollset, - .pollset_set_add_pollset_set = pollset_set_add_pollset_set, - .pollset_set_del_pollset_set = pollset_set_del_pollset_set, - .pollset_set_add_fd = pollset_set_add_fd, - .pollset_set_del_fd = pollset_set_del_fd, - - .kick_poller = kick_poller, - - .shutdown_engine = shutdown_engine, -}; - -const grpc_event_engine_vtable *grpc_init_epoll_posix(void) { - fd_global_init(); - pollset_global_init(); - return &vtable; -} - -#endif diff --git a/src/core/lib/iomgr/ev_epoll_posix.h b/src/core/lib/iomgr/ev_epoll_posix.h deleted file mode 100644 index 35319b4fc5d..00000000000 --- a/src/core/lib/iomgr/ev_epoll_posix.h +++ /dev/null @@ -1,41 +0,0 @@ -/* - * - * Copyright 2015, Google Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are - * met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following disclaimer - * in the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Google Inc. nor the names of its - * contributors may be used to endorse or promote products derived from - * this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - * - */ - -#ifndef GRPC_CORE_LIB_IOMGR_EV_EPOLL_POSIX_H -#define GRPC_CORE_LIB_IOMGR_EV_EPOLL_POSIX_H - -#include "src/core/lib/iomgr/ev_posix.h" - -const grpc_event_engine_vtable *grpc_init_epoll_posix(void); - -#endif /* GRPC_CORE_LIB_IOMGR_EV_EPOLL_POSIX_H */ diff --git a/src/python/grpcio/grpc_core_dependencies.py b/src/python/grpcio/grpc_core_dependencies.py index 13bc6888d66..50a6f196d8d 100644 --- a/src/python/grpcio/grpc_core_dependencies.py +++ b/src/python/grpcio/grpc_core_dependencies.py @@ -95,7 +95,6 @@ CORE_SOURCE_FILES = [ 'src/core/lib/iomgr/endpoint_pair_posix.c', 'src/core/lib/iomgr/endpoint_pair_windows.c', 'src/core/lib/iomgr/ev_epoll_linux.c', - 'src/core/lib/iomgr/ev_epoll_posix.c', 'src/core/lib/iomgr/ev_poll_posix.c', 'src/core/lib/iomgr/ev_posix.c', 'src/core/lib/iomgr/exec_ctx.c', diff --git a/test/core/network_benchmarks/epoll_test.c b/test/core/network_benchmarks/epoll_test.c deleted file mode 100644 index a918dd9bb94..00000000000 --- a/test/core/network_benchmarks/epoll_test.c +++ /dev/null @@ -1,263 +0,0 @@ -/* - * - * Copyright 2015, Google Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are - * met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following disclaimer - * in the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Google Inc. nor the names of its - * contributors may be used to endorse or promote products derived from - * this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - * - */ - -/* TODO: sreek: REMOVE THIS FILE */ - -#include - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include -#include -#include -#include - -int g_signal_num = SIGUSR1; - -int g_timeout_secs = 2; - -int g_eventfd_create = 1; -int g_eventfd_wakeup = 0; -int g_eventfd_teardown = 0; -int g_close_epoll_fd = 1; - -typedef struct thread_args { - gpr_thd_id id; - int epoll_fd; - int thread_num; -} thread_args; - -static int eventfd_create() { - if (!g_eventfd_create) { - return -1; - } - - int efd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); - GPR_ASSERT(efd >= 0); - return efd; -} - -static void eventfd_wakeup(int efd) { - if (!g_eventfd_wakeup) { - return; - } - - int err; - do { - err = eventfd_write(efd, 1); - } while (err < 0 && errno == EINTR); -} - -static void epoll_teardown(int epoll_fd, int fd) { - if (!g_eventfd_teardown) { - return; - } - - if (epoll_ctl(epoll_fd, EPOLL_CTL_DEL, fd, NULL) < 0) { - if (errno != ENOENT) { - gpr_log(GPR_ERROR, "epoll_ctl: %s", strerror(errno)); - GPR_ASSERT(0); - } - } -} - -/* Special case for epoll, where we need to create the fd ahead of time. */ -static int epoll_setup(int fd) { - int epoll_fd; - struct epoll_event ev; - - epoll_fd = epoll_create(1); - if (epoll_fd < 0) { - gpr_log(GPR_ERROR, "epoll_create: %s", strerror(errno)); - return -1; - } - - ev.events = (uint32_t)EPOLLIN; - ev.data.fd = fd; - if (epoll_ctl(epoll_fd, EPOLL_CTL_ADD, fd, &ev) < 0) { - if (errno != EEXIST) { - gpr_log(GPR_ERROR, "epoll_ctl: %s", strerror(errno)); - return -1; - } - - gpr_log(GPR_ERROR, "epoll_ctl: The fd %d already exists", fd); - } - - return epoll_fd; -} - -#define GRPC_EPOLL_MAX_EVENTS 1000 -static void thread_main(void *args) { - int ep_rv; - struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; - int fd; - int i; - int cancel; - int read; - int write; - thread_args *thd_args = args; - sigset_t new_mask; - sigset_t orig_mask; - int keep_polling = 0; - - gpr_log(GPR_INFO, "Thread: %d Started", thd_args->thread_num); - - do { - keep_polling = 0; - - /* Mask the signal before getting the epoll_fd */ - gpr_log(GPR_INFO, "Thread: %d Blocking signal: %d", thd_args->thread_num, - g_signal_num); - sigemptyset(&new_mask); - sigaddset(&new_mask, g_signal_num); - pthread_sigmask(SIG_BLOCK, &new_mask, &orig_mask); - - gpr_log(GPR_INFO, "Thread: %d Waiting on epoll_wait()", - thd_args->thread_num); - ep_rv = epoll_pwait(thd_args->epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, - g_timeout_secs * 5000, &orig_mask); - gpr_log(GPR_INFO, "Thread: %d out of epoll_wait. ep_rv = %d", - thd_args->thread_num, ep_rv); - - if (ep_rv < 0) { - if (errno != EINTR) { - gpr_log(GPR_ERROR, "Thread: %d. epoll_wait failed with error: %d", - thd_args->thread_num, errno); - } else { - gpr_log(GPR_INFO, - "Thread: %d. epoll_wait was interrupted. Polling again >>>>>>>", - thd_args->thread_num); - keep_polling = 1; - } - } else { - if (ep_rv == 0) { - gpr_log(GPR_INFO, - "Thread: %d - epoll_wait returned 0. Most likely a timeout. " - "Polling again", - thd_args->thread_num); - keep_polling = 1; - } - - for (i = 0; i < ep_rv; i++) { - fd = ep_ev[i].data.fd; - cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); - read = ep_ev[i].events & (EPOLLIN | EPOLLPRI); - write = ep_ev[i].events & EPOLLOUT; - gpr_log(GPR_INFO, - "Thread: %d. epoll_wait returned that fd: %d has event of " - "interest. read: %d, write: %d, cancel: %d", - thd_args->thread_num, fd, read, write, cancel); - } - } - } while (keep_polling); -} - -static void close_fd(int fd) { - if (!g_close_epoll_fd) { - return; - } - - gpr_log(GPR_INFO, "*** Closing fd : %d ****", fd); - close(fd); - gpr_log(GPR_INFO, "*** Closed fd : %d ****", fd); -} - - -static void sig_handler(int sig_num) { - gpr_log(GPR_INFO, "<<<<< Received signal %d", sig_num); -} - -static void set_signal_handler() { - gpr_log(GPR_INFO, "Setting signal handler"); - signal(g_signal_num, sig_handler); -} - -#define NUM_THREADS 2 -int main(int argc, char **argv) { - int efd; - int epoll_fd; - int i; - thread_args thd_args[NUM_THREADS]; - gpr_thd_options options = gpr_thd_options_default(); - - set_signal_handler(); - - gpr_log(GPR_INFO, "Starting.."); - efd = eventfd_create(); - gpr_log(GPR_INFO, "Created event fd: %d", efd); - epoll_fd = epoll_setup(efd); - gpr_log(GPR_INFO, "Created epoll_fd: %d", epoll_fd); - - gpr_thd_options_set_joinable(&options); - for (i = 0; i < NUM_THREADS; i++) { - thd_args[i].thread_num = i; - thd_args[i].epoll_fd = epoll_fd; - gpr_log(GPR_INFO, "Starting thread: %d", i); - gpr_thd_new(&thd_args[i].id, thread_main, &thd_args[i], &options); - } - - sleep((unsigned)g_timeout_secs * 2); - - /* Send signals first */ - for (i = 0; i < NUM_THREADS; i++) { - gpr_log(GPR_INFO, "Sending signal to thread: %d", thd_args->thread_num); - pthread_kill(thd_args[i].id, g_signal_num); - gpr_log(GPR_INFO, "Sent signal to thread: %d >>>>>> ", - thd_args->thread_num); - } - - sleep((unsigned)g_timeout_secs * 2); - - close_fd(epoll_fd); - - sleep((unsigned)g_timeout_secs * 2); - - eventfd_wakeup(efd); - epoll_teardown(epoll_fd, efd); - - for (i = 0; i < NUM_THREADS; i++) { - gpr_thd_join(thd_args[i].id); - gpr_log(GPR_INFO, "Thread: %d joined", i); - } - - return 0; -} diff --git a/test/core/network_benchmarks/low_level_ping_pong.c b/test/core/network_benchmarks/low_level_ping_pong.c index b72a07778e0..1b40895a719 100644 --- a/test/core/network_benchmarks/low_level_ping_pong.c +++ b/test/core/network_benchmarks/low_level_ping_pong.c @@ -44,7 +44,6 @@ #include #include #include -#include #ifdef __linux__ #include #endif @@ -85,7 +84,6 @@ typedef struct thread_args { static int read_bytes(int fd, char *buf, size_t read_size, int spin) { size_t bytes_read = 0; ssize_t err; - do { err = read(fd, buf + bytes_read, read_size - bytes_read); if (err < 0) { diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal index d968278f2a4..21ee1e6ff8a 100644 --- a/tools/doxygen/Doxyfile.core.internal +++ b/tools/doxygen/Doxyfile.core.internal @@ -808,7 +808,6 @@ src/core/lib/iomgr/closure.h \ src/core/lib/iomgr/endpoint.h \ src/core/lib/iomgr/endpoint_pair.h \ src/core/lib/iomgr/ev_epoll_linux.h \ -src/core/lib/iomgr/ev_epoll_posix.h \ src/core/lib/iomgr/ev_poll_posix.h \ src/core/lib/iomgr/ev_posix.h \ src/core/lib/iomgr/exec_ctx.h \ @@ -957,7 +956,6 @@ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ src/core/lib/iomgr/ev_epoll_linux.c \ -src/core/lib/iomgr/ev_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ src/core/lib/iomgr/exec_ctx.c \ diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index 85b71a8255a..304a0e1e3a9 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -301,22 +301,6 @@ "third_party": false, "type": "target" }, - { - "deps": [ - "gpr", - "gpr_test_util", - "grpc", - "grpc_test_util" - ], - "headers": [], - "language": "c", - "name": "epoll_test", - "src": [ - "test/core/network_benchmarks/epoll_test.c" - ], - "third_party": false, - "type": "target" - }, { "deps": [ "gpr", @@ -5547,7 +5531,6 @@ "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", "src/core/lib/iomgr/ev_epoll_linux.h", - "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", "src/core/lib/iomgr/exec_ctx.h", @@ -5649,8 +5632,6 @@ "src/core/lib/iomgr/endpoint_pair_windows.c", "src/core/lib/iomgr/ev_epoll_linux.c", "src/core/lib/iomgr/ev_epoll_linux.h", - "src/core/lib/iomgr/ev_epoll_posix.c", - "src/core/lib/iomgr/ev_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.c", diff --git a/tools/run_tests/tests.json b/tools/run_tests/tests.json index be7b72f61d2..850f9474aec 100644 --- a/tools/run_tests/tests.json +++ b/tools/run_tests/tests.json @@ -356,21 +356,6 @@ "windows" ] }, - { - "args": [], - "ci_platforms": [ - "linux" - ], - "cpu_cost": 1.0, - "exclude_configs": [], - "flaky": false, - "gtest": false, - "language": "c", - "name": "epoll_test", - "platforms": [ - "linux" - ] - }, { "args": [], "ci_platforms": [ diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj b/vsprojects/vcxproj/grpc/grpc.vcxproj index a67e4d16dad..ce523725e8e 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj @@ -317,7 +317,6 @@ - @@ -487,8 +486,6 @@ - - diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters index bf9b7dc7dcf..d46676f2296 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters @@ -58,9 +58,6 @@ src\core\lib\iomgr - - src\core\lib\iomgr - src\core\lib\iomgr @@ -683,9 +680,6 @@ src\core\lib\iomgr - - src\core\lib\iomgr - src\core\lib\iomgr diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj index afc9a2ca1b2..d4dd428c2db 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj @@ -305,7 +305,6 @@ - @@ -453,8 +452,6 @@ - - diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters index b7507f9a963..d14e7e7ab41 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters @@ -61,9 +61,6 @@ src\core\lib\iomgr - - src\core\lib\iomgr - src\core\lib\iomgr @@ -581,9 +578,6 @@ src\core\lib\iomgr - - src\core\lib\iomgr - src\core\lib\iomgr From 9bc3d2d67f32f4dad8cd1319dd4f3fce48c1abee Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 6 Jun 2016 10:27:56 -0700 Subject: [PATCH 038/280] Minor comments --- src/core/lib/iomgr/ev_epoll_linux.c | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 61106faef90..d3abf3bd84f 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -144,10 +144,9 @@ typedef struct polling_island { /******************************************************************************* * Pollset Declarations */ - struct grpc_pollset_worker { int kicked_specifically; - pthread_t pt_id; /* TODO (sreek) - Add an abstraction here */ + pthread_t pt_id; /* Thread id of this worker */ struct grpc_pollset_worker *next; struct grpc_pollset_worker *prev; }; @@ -483,8 +482,7 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Get locks on both the polling islands */ polling_island_pair_update_and_lock(&p, &q); - /* TODO: sreek: Think about this scenario some more. Is it possible ?. what - * does it mean, when would this happen */ + /* TODO: sreek: Think about this scenario some more */ if (p == q) { /* Nothing needs to be done here */ gpr_mu_unlock(&p->mu); @@ -539,7 +537,10 @@ static void polling_island_global_init() { * (specifically when a new alarm needs to be triggered earlier than the next * alarm 'epoch'). This wakeup_fd gives us something to alert on when such a * case occurs. */ -/* TODO: sreek: Right now, this wakes up all pollers */ + +/* TODO: sreek: Right now, this wakes up all pollers. In future we should make + * sure to wake up one polling thread (which can wake up other threads if + * needed) */ grpc_wakeup_fd grpc_global_wakeup_fd; static grpc_fd *fd_freelist = NULL; @@ -676,7 +677,6 @@ static int fd_wrapped_fd(grpc_fd *fd) { static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure *on_done, int *release_fd, const char *reason) { - /* TODO(sreek) In ev_poll_posix.c,the lock is acquired a little later. Why? */ bool is_fd_closed = false; gpr_mu_lock(&fd->mu); fd->on_done_closure = on_done; @@ -784,8 +784,9 @@ static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, */ static void sig_handler(int sig_num) { - /* TODO: sreek - Remove this expensive log line */ +#ifdef GPRC_EPOLL_DEBUG gpr_log(GPR_INFO, "Received signal %d", sig_num); +#endif } /* Global state management */ @@ -986,7 +987,10 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, if (ep_rv < 0) { if (errno != EINTR) { - /* TODO (sreek) - Check for bad file descriptor error */ + /* TODO (sreek) - Do not log an error in case of bad file descriptor + * (A bad file descriptor here would just mean that the epoll set was + * merged with another epoll set and that the current epoll_fd is + * closed) */ gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); } else { gpr_log(GPR_DEBUG, "pollset_work_and_unlock: 0-timeout epoll_wait()"); @@ -1062,7 +1066,9 @@ static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, GPR_TIMER_END("pollset_shutdown", 0); } -/* TODO(sreek) Is pollset_shutdown() guranteed to be called before this? */ +/* pollset_shutdown is guaranteed to be called before pollset_destroy. So other + * than destroying the mutexes, there is nothing special that needs to be done + * here */ static void pollset_destroy(grpc_pollset *pollset) { GPR_ASSERT(!pollset_has_workers(pollset)); gpr_mu_destroy(&pollset->pi_mu); @@ -1075,7 +1081,7 @@ static void pollset_reset(grpc_pollset *pollset) { pollset->shutting_down = false; pollset->finish_shutdown_called = false; pollset->kicked_without_pollers = false; - /* TODO(sreek) - Should pollset->shutdown closure be set to NULL here? */ + pollset->shutdown_done = NULL; pollset_release_polling_island_locked(pollset); } @@ -1149,7 +1155,7 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { gpr_log(GPR_DEBUG, "pollset_add_fd: pollset: %p, fd: %d", pollset, fd->fd); - /* TODO sreek - Check if we need to get a pollset->mu lock here */ + /* TODO sreek - Double check if we need to get a pollset->mu lock here */ gpr_mu_lock(&pollset->pi_mu); gpr_mu_lock(&fd->pi_mu); From d627c105847f0d262ce3886fb5b463dc914ddbd7 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 6 Jun 2016 15:49:32 -0700 Subject: [PATCH 039/280] Fix asan failures (i.e add pollset_global_shutdown), remove debug log lines --- src/core/lib/iomgr/ev_epoll_linux.c | 76 ++++++++++++++--------------- 1 file changed, 38 insertions(+), 38 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index d3abf3bd84f..0e00d4d216a 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -516,6 +516,21 @@ static void polling_island_global_init() { g_pi_freelist = NULL; } +static void polling_island_global_shutdown() { + polling_island *next; + gpr_mu_lock(&g_pi_freelist_mu); + gpr_mu_unlock(&g_pi_freelist_mu); + while (g_pi_freelist != NULL) { + next = g_pi_freelist->next_free; + gpr_mu_destroy(&g_pi_freelist->mu); + gpr_free(g_pi_freelist->fds); + gpr_free(g_pi_freelist); + g_pi_freelist = next; + } + + gpr_mu_destroy(&g_pi_freelist_mu); +} + /******************************************************************************* * Fd Definitions */ @@ -784,7 +799,7 @@ static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, */ static void sig_handler(int sig_num) { -#ifdef GPRC_EPOLL_DEBUG +#ifdef GRPC_EPOLL_DEBUG gpr_log(GPR_INFO, "Received signal %d", sig_num); #endif } @@ -792,7 +807,7 @@ static void sig_handler(int sig_num) { /* Global state management */ static void pollset_global_init(void) { grpc_wakeup_fd_init(&grpc_global_wakeup_fd); - signal(SIGUSR1, sig_handler); + signal(SIGUSR1, sig_handler); /* TODO: sreek - Do not hardcode SIGUSR1 */ } static void pollset_global_shutdown(void) { @@ -840,7 +855,6 @@ static void pollset_kick(grpc_pollset *p, grpc_pollset_worker *worker = specific_worker; if (worker != NULL) { if (worker == GRPC_POLLSET_KICK_BROADCAST) { - gpr_log(GPR_DEBUG, "pollset_kick: broadcast!"); if (pollset_has_workers(p)) { GPR_TIMER_BEGIN("pollset_kick.broadcast", 0); for (worker = p->root_worker.next; worker != &p->root_worker; @@ -848,12 +862,10 @@ static void pollset_kick(grpc_pollset *p, pthread_kill(worker->pt_id, SIGUSR1); } } else { - gpr_log(GPR_DEBUG, "pollset_kick: (broadcast) Kicked without pollers"); p->kicked_without_pollers = true; } GPR_TIMER_END("pollset_kick.broadcast", 0); } else { - gpr_log(GPR_DEBUG, "pollset_kick: kicked kicked_specifically"); GPR_TIMER_MARK("kicked_specifically", 0); worker->kicked_specifically = true; pthread_kill(worker->pt_id, SIGUSR1); @@ -864,11 +876,9 @@ static void pollset_kick(grpc_pollset *p, if (worker != NULL) { GPR_TIMER_MARK("finally_kick", 0); push_back_worker(p, worker); - gpr_log(GPR_DEBUG, "pollset_kick: anonymous kick"); pthread_kill(worker->pt_id, SIGUSR1); } else { GPR_TIMER_MARK("kicked_no_pollers", 0); - gpr_log(GPR_DEBUG, "pollset_kick: kicked without pollers"); p->kicked_without_pollers = true; } } @@ -941,7 +951,6 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; int epoll_fd = -1; int ep_rv; - gpr_log(GPR_DEBUG, "pollset_work_and_unlock: Entering.."); GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the @@ -952,22 +961,27 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, - pollset->polling_island->mu */ gpr_mu_lock(&pollset->pi_mu); - if (pollset->polling_island != NULL) { - pollset->polling_island = - polling_island_update_and_lock(pollset->polling_island, 1, 0); - epoll_fd = pollset->polling_island->epoll_fd; - if (pollset->polling_island->fd_cnt == 0) { - gpr_log(GPR_DEBUG, "pollset_work_and_unlock: epoll_fd: %d, No other fds", - epoll_fd); - } - for (size_t i = 0; i < pollset->polling_island->fd_cnt; i++) { - gpr_log(GPR_DEBUG, - "pollset_work_and_unlock: epoll_fd: %d, fd_count: %d, fd[%d]: %d", - epoll_fd, pollset->polling_island->fd_cnt, i, - pollset->polling_island->fds[i]->fd); - } - gpr_mu_unlock(&pollset->polling_island->mu); + if (pollset->polling_island == NULL) { + pollset->polling_island = polling_island_create(NULL, 1); + } + + pollset->polling_island = + polling_island_update_and_lock(pollset->polling_island, 1, 0); + epoll_fd = pollset->polling_island->epoll_fd; + +#ifdef GRPC_EPOLL_DEBUG + if (pollset->polling_island->fd_cnt == 0) { + gpr_log(GPR_DEBUG, "pollset_work_and_unlock: epoll_fd: %d, No other fds", + epoll_fd); + } + for (size_t i = 0; i < pollset->polling_island->fd_cnt; i++) { + gpr_log(GPR_DEBUG, + "pollset_work_and_unlock: epoll_fd: %d, fd_count: %d, fd[%d]: %d", + epoll_fd, pollset->polling_island->fd_cnt, i, + pollset->polling_island->fds[i]->fd); } +#endif + gpr_mu_unlock(&pollset->polling_island->mu); gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); @@ -975,16 +989,8 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, /* If epoll_fd == -1, this is a blank pollset and does not have any fds yet */ if (epoll_fd != -1) { do { - gpr_timespec before_epoll = gpr_now(GPR_CLOCK_PRECISE); - gpr_log(GPR_DEBUG, "pollset_work_and_unlock: epoll_wait()...."); ep_rv = epoll_pwait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms, sig_mask); - gpr_timespec after_epoll = gpr_now(GPR_CLOCK_PRECISE); - int dur = gpr_time_to_millis(gpr_time_sub(after_epoll, before_epoll)); - gpr_log(GPR_DEBUG, - "pollset_work_and_unlock: DONE epoll_wait() : %d ms, ep_rv: %d", - dur, ep_rv); - if (ep_rv < 0) { if (errno != EINTR) { /* TODO (sreek) - Do not log an error in case of bad file descriptor @@ -993,9 +999,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, * closed) */ gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); } else { - gpr_log(GPR_DEBUG, "pollset_work_and_unlock: 0-timeout epoll_wait()"); ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); - gpr_log(GPR_DEBUG, "pollset_work_and_unlock: ep_rv: %d", ep_rv); } } @@ -1018,7 +1022,6 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, } } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); } - gpr_log(GPR_DEBUG, "pollset_work_and_unlock: Leaving.."); GPR_TIMER_END("pollset_work_and_unlock", 0); } @@ -1093,7 +1096,6 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker **worker_hdl, gpr_timespec now, gpr_timespec deadline) { GPR_TIMER_BEGIN("pollset_work", 0); - gpr_log(GPR_DEBUG, "pollset_work: enter"); int timeout_ms = poll_deadline_to_millis_timeout(deadline, now); sigset_t new_mask; @@ -1112,7 +1114,6 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, work that needs attention like an event on the completion queue or an alarm */ GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); - gpr_log(GPR_INFO, "pollset_work: kicked without pollers.."); pollset->kicked_without_pollers = 0; } else if (!pollset->shutting_down) { sigemptyset(&new_mask); @@ -1147,14 +1148,12 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, gpr_mu_lock(&pollset->mu); } - gpr_log(GPR_DEBUG, "pollset_work(): leaving"); *worker_hdl = NULL; GPR_TIMER_END("pollset_work", 0); } static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { - gpr_log(GPR_DEBUG, "pollset_add_fd: pollset: %p, fd: %d", pollset, fd->fd); /* TODO sreek - Double check if we need to get a pollset->mu lock here */ gpr_mu_lock(&pollset->pi_mu); gpr_mu_lock(&fd->pi_mu); @@ -1347,6 +1346,7 @@ static void pollset_set_del_pollset_set(grpc_exec_ctx *exec_ctx, static void shutdown_engine(void) { fd_global_shutdown(); pollset_global_shutdown(); + polling_island_global_shutdown(); } static const grpc_event_engine_vtable vtable = { From e5012bac7a57df6b1993a1eaa9b8b3c4d7671975 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 6 Jun 2016 16:01:45 -0700 Subject: [PATCH 040/280] Remove redundant code --- src/core/lib/iomgr/ev_epoll_linux.c | 61 ++++++++++++++--------------- 1 file changed, 29 insertions(+), 32 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 0e00d4d216a..f7ac4ae1ff4 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -986,42 +986,39 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); - /* If epoll_fd == -1, this is a blank pollset and does not have any fds yet */ - if (epoll_fd != -1) { - do { - ep_rv = epoll_pwait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms, - sig_mask); - if (ep_rv < 0) { - if (errno != EINTR) { - /* TODO (sreek) - Do not log an error in case of bad file descriptor - * (A bad file descriptor here would just mean that the epoll set was - * merged with another epoll set and that the current epoll_fd is - * closed) */ - gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); - } else { - ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); - } + do { + ep_rv = epoll_pwait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, timeout_ms, + sig_mask); + if (ep_rv < 0) { + if (errno != EINTR) { + /* TODO (sreek) - Do not log an error in case of bad file descriptor + * (A bad file descriptor here would just mean that the epoll set was + * merged with another epoll set and that the current epoll_fd is + * closed) */ + gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); + } else { + ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); } + } - int i; - for (i = 0; i < ep_rv; ++i) { - grpc_fd *fd = ep_ev[i].data.ptr; - int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); - int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); - int write_ev = ep_ev[i].events & EPOLLOUT; - if (fd == NULL) { - grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); - } else { - if (read_ev || cancel) { - fd_become_readable(exec_ctx, fd); - } - if (write_ev || cancel) { - fd_become_writable(exec_ctx, fd); - } + int i; + for (i = 0; i < ep_rv; ++i) { + grpc_fd *fd = ep_ev[i].data.ptr; + int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + int write_ev = ep_ev[i].events & EPOLLOUT; + if (fd == NULL) { + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + } else { + if (read_ev || cancel) { + fd_become_readable(exec_ctx, fd); + } + if (write_ev || cancel) { + fd_become_writable(exec_ctx, fd); } } - } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); - } + } + } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); GPR_TIMER_END("pollset_work_and_unlock", 0); } From ad162ba5a9f91481b71f233d1f4f8c15b01de644 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 6 Jun 2016 16:23:37 -0700 Subject: [PATCH 041/280] Core review comments and remove 'kicked_specifically' field as its not needed --- src/core/lib/iomgr/ev_epoll_linux.c | 21 ++++++++------------- 1 file changed, 8 insertions(+), 13 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index f7ac4ae1ff4..d5aac96fa46 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -145,7 +145,6 @@ typedef struct polling_island { * Pollset Declarations */ struct grpc_pollset_worker { - int kicked_specifically; pthread_t pt_id; /* Thread id of this worker */ struct grpc_pollset_worker *next; struct grpc_pollset_worker *prev; @@ -235,18 +234,16 @@ static void polling_island_remove_all_fds_locked(polling_island *pi, size_t i; for (i = 0; i < pi->fd_cnt; i++) { - if (remove_fd_refs) { - GRPC_FD_UNREF(pi->fds[i], "polling_island"); - } - err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, pi->fds[i]->fd, NULL); if (err < 0 && errno != ENOENT) { - gpr_log(GPR_ERROR, - "epoll_ctl delete for fds[i]: %d failed with error: %s", i, - pi->fds[i]->fd, strerror(errno)); /* TODO: sreek - We need a better way to bubble up this error instead of - * just logging a message */ - continue; + * just logging a message */ + gpr_log(GPR_ERROR, "epoll_ctl deleting fds[%d]: %d failed with error: %s", + i, pi->fds[i]->fd, strerror(errno)); + } + + if (remove_fd_refs) { + GRPC_FD_UNREF(pi->fds[i], "polling_island"); } } @@ -264,7 +261,7 @@ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, if (!is_fd_closed) { err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, fd->fd, NULL); if (err < 0 && errno != ENOENT) { - gpr_log(GPR_ERROR, "epoll_ctl delete for fd: %d failed with error; %s", + gpr_log(GPR_ERROR, "epoll_ctl deleting fd: %d failed with error; %s", fd->fd, strerror(errno)); } } @@ -867,7 +864,6 @@ static void pollset_kick(grpc_pollset *p, GPR_TIMER_END("pollset_kick.broadcast", 0); } else { GPR_TIMER_MARK("kicked_specifically", 0); - worker->kicked_specifically = true; pthread_kill(worker->pt_id, SIGUSR1); } } else { @@ -1100,7 +1096,6 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_pollset_worker worker; worker.next = worker.prev = NULL; - worker.kicked_specifically = 0; worker.pt_id = pthread_self(); *worker_hdl = &worker; From 5855c478c69508f000baa4878f515d72b5f5a1e9 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 8 Jun 2016 12:56:56 -0700 Subject: [PATCH 042/280] Use poll if not linux, add read notifier pollset support and some groundwork for adding API that allows users to register custom kick signal number --- include/grpc/impl/codegen/port_platform.h | 1 + src/core/lib/iomgr/ev_epoll_linux.c | 72 ++++++++++++++++------- src/core/lib/iomgr/ev_posix.c | 2 +- 3 files changed, 54 insertions(+), 21 deletions(-) diff --git a/include/grpc/impl/codegen/port_platform.h b/include/grpc/impl/codegen/port_platform.h index be4215a54bf..7a6ec53fb4c 100644 --- a/include/grpc/impl/codegen/port_platform.h +++ b/include/grpc/impl/codegen/port_platform.h @@ -189,6 +189,7 @@ #define GPR_GCC_ATOMIC 1 #define GPR_GCC_TLS 1 #define GPR_LINUX 1 +#define GPR_LINUX_EPOLL 1 #define GPR_LINUX_LOG #define GPR_LINUX_MULTIPOLL_WITH_EPOLL 1 #define GPR_POSIX_WAKEUP_FD 1 diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index d5aac96fa46..69ab665e155 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -33,7 +33,7 @@ #include -#ifdef GPR_POSIX_SOCKET +#ifdef GPR_LINUX_EPOLL #include "src/core/lib/iomgr/ev_epoll_linux.h" @@ -60,6 +60,8 @@ struct polling_island; +static int grpc_poller_kick_signum; + /******************************************************************************* * Fd Declarations */ @@ -92,6 +94,9 @@ struct grpc_fd { struct grpc_fd *freelist_next; grpc_closure *on_done_closure; + /* The pollset that last noticed that the fd is readable */ + grpc_pollset *read_notifier_pollset; + grpc_iomgr_object iomgr_object; }; @@ -650,14 +655,15 @@ static grpc_fd *fd_create(int fd, const char *name) { gpr_mu_lock(&new_fd->mu); gpr_atm_rel_store(&new_fd->refst, 1); + new_fd->fd = fd; new_fd->shutdown = false; + new_fd->orphaned = false; new_fd->read_closure = CLOSURE_NOT_READY; new_fd->write_closure = CLOSURE_NOT_READY; - new_fd->fd = fd; new_fd->polling_island = NULL; new_fd->freelist_next = NULL; new_fd->on_done_closure = NULL; - new_fd->orphaned = false; + new_fd->read_notifier_pollset = NULL; gpr_mu_unlock(&new_fd->mu); @@ -765,6 +771,17 @@ static int set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, } } +static grpc_pollset *fd_get_read_notifier_pollset(grpc_exec_ctx *exec_ctx, + grpc_fd *fd) { + grpc_pollset *notifier = NULL; + + gpr_mu_lock(&fd->mu); + notifier = fd->read_notifier_pollset; + gpr_mu_unlock(&fd->mu); + + return notifier; +} + static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { gpr_mu_lock(&fd->mu); GPR_ASSERT(!fd->shutdown); @@ -801,16 +818,25 @@ static void sig_handler(int sig_num) { #endif } +static void poller_kick_init() { + grpc_poller_kick_signum = SIGRTMIN + 2; + signal(grpc_poller_kick_signum, sig_handler); +} + /* Global state management */ static void pollset_global_init(void) { grpc_wakeup_fd_init(&grpc_global_wakeup_fd); - signal(SIGUSR1, sig_handler); /* TODO: sreek - Do not hardcode SIGUSR1 */ + poller_kick_init(); } static void pollset_global_shutdown(void) { grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); } +static void pollset_worker_kick(grpc_pollset_worker *worker) { + pthread_kill(worker->pt_id, grpc_poller_kick_signum); +} + /* Return 1 if the pollset has active threads in pollset_work (pollset must * be locked) */ static int pollset_has_workers(grpc_pollset *p) { @@ -856,7 +882,7 @@ static void pollset_kick(grpc_pollset *p, GPR_TIMER_BEGIN("pollset_kick.broadcast", 0); for (worker = p->root_worker.next; worker != &p->root_worker; worker = worker->next) { - pthread_kill(worker->pt_id, SIGUSR1); + pollset_worker_kick(worker); } } else { p->kicked_without_pollers = true; @@ -864,7 +890,7 @@ static void pollset_kick(grpc_pollset *p, GPR_TIMER_END("pollset_kick.broadcast", 0); } else { GPR_TIMER_MARK("kicked_specifically", 0); - pthread_kill(worker->pt_id, SIGUSR1); + pollset_worker_kick(worker); } } else { GPR_TIMER_MARK("kick_anonymous", 0); @@ -872,7 +898,7 @@ static void pollset_kick(grpc_pollset *p, if (worker != NULL) { GPR_TIMER_MARK("finally_kick", 0); push_back_worker(p, worker); - pthread_kill(worker->pt_id, SIGUSR1); + pollset_worker_kick(worker); } else { GPR_TIMER_MARK("kicked_no_pollers", 0); p->kicked_without_pollers = true; @@ -924,20 +950,20 @@ static int poll_deadline_to_millis_timeout(gpr_timespec deadline, timeout, gpr_time_from_nanos(GPR_NS_PER_MS - 1, GPR_TIMESPAN))); } -static void set_ready(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st) { - /* only one set_ready can be active at once (but there may be a racing - notify_on) */ +static void fd_become_readable(grpc_exec_ctx *exec_ctx, grpc_fd *fd, + grpc_pollset *notifier) { + /* Need the fd->mu since we might be racing with fd_notify_on_read */ gpr_mu_lock(&fd->mu); - set_ready_locked(exec_ctx, fd, st); + set_ready_locked(exec_ctx, fd, &fd->read_closure); + fd->read_notifier_pollset = notifier; gpr_mu_unlock(&fd->mu); } -static void fd_become_readable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - set_ready(exec_ctx, fd, &fd->read_closure); -} - static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { - set_ready(exec_ctx, fd, &fd->write_closure); + /* Need the fd->mu since we might be racing with fd_notify_on_write */ + gpr_mu_lock(&fd->mu); + set_ready_locked(exec_ctx, fd, &fd->write_closure); + gpr_mu_unlock(&fd->mu); } #define GRPC_EPOLL_MAX_EVENTS 1000 @@ -1007,7 +1033,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); } else { if (read_ev || cancel) { - fd_become_readable(exec_ctx, fd); + fd_become_readable(exec_ctx, fd, pollset); } if (write_ev || cancel) { fd_become_writable(exec_ctx, fd); @@ -1109,9 +1135,9 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, pollset->kicked_without_pollers = 0; } else if (!pollset->shutting_down) { sigemptyset(&new_mask); - sigaddset(&new_mask, SIGUSR1); + sigaddset(&new_mask, grpc_poller_kick_signum); pthread_sigmask(SIG_BLOCK, &new_mask, &orig_mask); - sigdelset(&orig_mask, SIGUSR1); + sigdelset(&orig_mask, grpc_poller_kick_signum); push_front_worker(pollset, &worker); @@ -1350,6 +1376,7 @@ static const grpc_event_engine_vtable vtable = { .fd_shutdown = fd_shutdown, .fd_notify_on_read = fd_notify_on_read, .fd_notify_on_write = fd_notify_on_write, + .fd_get_read_notifier_pollset = fd_get_read_notifier_pollset, .pollset_init = pollset_init, .pollset_shutdown = pollset_shutdown, @@ -1380,4 +1407,9 @@ const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { return &vtable; } -#endif +#else /* defined(GPR_LINUX_EPOLL) */ +/* If GPR_LINUX_EPOLL is not defined, it means epoll is not available. Return + * NULL */ +const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { return NULL; } + +#endif /* !defined(GPR_LINUX_EPOLL) */ diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index e0c3558a51e..2b15967adcc 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -63,8 +63,8 @@ typedef struct { } event_engine_factory; static const event_engine_factory g_factories[] = { - {"poll", grpc_init_poll_posix}, {"epoll", grpc_init_epoll_linux}, + {"poll", grpc_init_poll_posix}, {"legacy", grpc_init_poll_and_epoll_posix}, }; From 485cf40ab7d68b2624a5ab3be3842444f5835f14 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Wed, 8 Jun 2016 13:35:25 -0700 Subject: [PATCH 043/280] Fixed wrong spec and reworked streaming compressed case --- doc/interop-test-descriptions.md | 36 +++++++++++------------------- test/cpp/interop/interop_client.cc | 17 ++++++++------ 2 files changed, 23 insertions(+), 30 deletions(-) diff --git a/doc/interop-test-descriptions.md b/doc/interop-test-descriptions.md index 7fd21c7022e..6a76acd3a89 100644 --- a/doc/interop-test-descriptions.md +++ b/doc/interop-test-descriptions.md @@ -224,7 +224,7 @@ Procedure: size: 31415 } response_parameters:{ - size: 59 + size: 9 } response_parameters:{ size: 2653 @@ -261,20 +261,19 @@ Procedure: request_compressed_response: bool response_type:COMPRESSABLE response_parameters:{ - size: 31415 - } - response_parameters:{ - size: 59 - } - response_parameters:{ - size: 2653 + size: 31424 } response_parameters:{ - size: 58979 + size: 61632 } } ``` + Note that the `response_parameters` sizes are the sum of the usual streaming + response sizes (31415, 9, 2653, 58979) taken in successive pairs. This way, + we only keep a single list of sizes while making sure the individual message + sizes are large enough to trigger compression in all implementations. + Client asserts: * call was successful * exactly four responses @@ -283,7 +282,7 @@ Procedure: NOT have the compressed message flag set. * if `request_compressed_response` is true, the response's messages MUST have the compressed message flag set. - * response payload bodies are sized (in order): 31415, 59, 2653, 58979 + * response payload bodies are sized (in order): 31424, 61632 * clients are free to assert that the response payload body contents are zero and comparing the entire response messages against golden responses @@ -295,16 +294,10 @@ Procedure: request_compressed_response: bool response_type:UNCOMPRESSABLE response_parameters:{ - size: 31415 - } - response_parameters:{ - size: 59 - } - response_parameters:{ - size: 2653 + size: 31424 } response_parameters:{ - size: 58979 + size: 61632 } } ``` @@ -316,10 +309,7 @@ Procedure: * the response MAY have the compressed message flag set. Some implementations will choose to compress the payload even when the output size if larger than the input. - * response payload bodies are sized (in order): 31415, 59, 2653, 58979 - * clients are free to assert that the body of the responses are identical to - the golden uncompressable data at `test/cpp/interop/rnd.dat`. - + * response payload bodies are sized (in order): 31424, 61632 ### ping_pong @@ -350,7 +340,7 @@ Procedure: { response_type: COMPRESSABLE response_parameters:{ - size: 59 + size: 9 } payload:{ body: 8 bytes of zeros diff --git a/test/cpp/interop/interop_client.cc b/test/cpp/interop/interop_client.cc index a0479e8f689..7705bb15922 100644 --- a/test/cpp/interop/interop_client.cc +++ b/test/cpp/interop/interop_client.cc @@ -58,7 +58,7 @@ namespace testing { namespace { // The same value is defined by the Java client. const std::vector request_stream_sizes = {27182, 8, 1828, 45904}; -const std::vector response_stream_sizes = {31415, 59, 2653, 58979}; +const std::vector response_stream_sizes = {31415, 9, 2653, 58979}; const int kNumResponseMessages = 2000; const int kResponseMessageSize = 1030; const int kReceiveDelayMilliSeconds = 20; @@ -466,10 +466,11 @@ bool InteropClient::DoResponseCompressedStreaming() { request.set_response_type(payload_types[i]); request.set_request_compressed_response(request_compression[j]); - for (size_t k = 0; k < response_stream_sizes.size(); ++k) { + for (size_t k = 0; k < response_stream_sizes.size() / 2; ++k) { ResponseParameters* response_parameter = request.add_response_parameters(); - response_parameter->set_size(response_stream_sizes[k]); + response_parameter->set_size(response_stream_sizes[k] + + response_stream_sizes[k + 1]); } StreamingOutputCallResponse response; @@ -483,7 +484,9 @@ bool InteropClient::DoResponseCompressedStreaming() { switch (response.payload().type()) { case PayloadType::COMPRESSABLE: GPR_ASSERT(response.payload().body() == - grpc::string(response_stream_sizes[k], '\0')); + grpc::string(response_stream_sizes[k] + + response_stream_sizes[k + 1], + '\0')); break; case PayloadType::UNCOMPRESSABLE: break; @@ -513,14 +516,14 @@ bool InteropClient::DoResponseCompressedStreaming() { gpr_log(GPR_DEBUG, "Response streaming done %s.", log_suffix); gpr_free(log_suffix); - if (k < response_stream_sizes.size()) { + if (k < response_stream_sizes.size() / 2) { // stream->Read() failed before reading all the expected messages. This // is most likely due to a connection failure. gpr_log(GPR_ERROR, "DoResponseCompressedStreaming(): Responses read (k=%d) is " "less than the expected messages (i.e " - "response_stream_sizes.size() (%d)). (i=%d, j=%d)", - k, response_stream_sizes.size(), i, j); + "response_stream_sizes.size()/2 (%d)). (i=%d, j=%d)", + k, response_stream_sizes.size() / 2, i, j); return TransientFailureOrAbort(); } From 9065c8b1df3a106947eab71db678e2506aa4af5d Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Wed, 8 Jun 2016 14:31:21 -0700 Subject: [PATCH 044/280] more changes after offline chat with ejona --- doc/interop-test-descriptions.md | 85 ++++++++------------------------ 1 file changed, 20 insertions(+), 65 deletions(-) diff --git a/doc/interop-test-descriptions.md b/doc/interop-test-descriptions.md index 6a76acd3a89..63b0022c3f6 100644 --- a/doc/interop-test-descriptions.md +++ b/doc/interop-test-descriptions.md @@ -90,11 +90,11 @@ Client asserts: * clients are free to assert that the response payload body contents are zero and comparing the entire response message against a golden response -### large_compressed_unary +### server_compressed_unary -This test verifies compressed unary calls succeed in sending messages. It -sends one unary request for every payload type, with and without requesting a -compressed response from the server. +This test verifies compressed server-only unary calls succeed in sending +messages. It sends one unary request for every payload type, with and without +requesting a compressed response from the server. In all scenarios, whether compression was actually performed is determined by the compression bit in the response's message flags. @@ -103,7 +103,6 @@ the compression bit in the response's message flags. Server features: * [UnaryCall][] * [Compressable Payload][] -* [Uncompressable Payload][] Procedure: 1. Client calls UnaryCall with: @@ -130,26 +129,6 @@ Procedure: zero and comparing the entire response message against a golden response - 2. Client calls UnaryCall with: - ``` - { - request_compressed_response: bool - response_type: UNCOMPRESSABLE - response_size: 314159 - payload:{ - body: 271828 bytes of zeros - } - } - ``` - Client asserts: - * call was successful - * response payload type is UNCOMPRESSABLE - * the response MAY have the compressed message flag set. Some - implementations will choose to compress the payload even when the output - size if larger than the input. - * response payload body is 314159 bytes in size - - ### client_streaming This test verifies that client-only streaming succeeds. @@ -250,7 +229,6 @@ This test verifies that server-only compressed streaming succeeds. Server features: * [StreamingOutputCall][] * [Compressable Payload][] -* [Uncompressable Payload][] Procedure: @@ -258,46 +236,26 @@ Procedure: ``` { - request_compressed_response: bool + request_compressed_response: true response_type:COMPRESSABLE response_parameters:{ - size: 31424 + size: 31415 } response_parameters:{ - size: 61632 + size: 58979 } } ``` - Note that the `response_parameters` sizes are the sum of the usual streaming - response sizes (31415, 9, 2653, 58979) taken in successive pairs. This way, - we only keep a single list of sizes while making sure the individual message - sizes are large enough to trigger compression in all implementations. - - Client asserts: - * call was successful - * exactly four responses - * response payloads are COMPRESSABLE - * if `request_compressed_response` is false, the response's messages MUST - NOT have the compressed message flag set. - * if `request_compressed_response` is true, the response's messages MUST - have the compressed message flag set. - * response payload bodies are sized (in order): 31424, 61632 - * clients are free to assert that the response payload body contents are - zero and comparing the entire response messages against golden responses - - - 2. Client calls StreamingOutputCall with: - ``` { - request_compressed_response: bool - response_type:UNCOMPRESSABLE + request_compressed_response: false + response_type:COMPRESSABLE response_parameters:{ - size: 31424 + size: 31415 } response_parameters:{ - size: 61632 + size: 58979 } } ``` @@ -305,11 +263,15 @@ Procedure: Client asserts: * call was successful * exactly four responses - * response payloads are UNCOMPRESSABLE - * the response MAY have the compressed message flag set. Some - implementations will choose to compress the payload even when the output - size if larger than the input. - * response payload bodies are sized (in order): 31424, 61632 + * response payloads are COMPRESSABLE + * if `request_compressed_response` is false, the response's messages MUST + NOT have the compressed message flag set. + * if `request_compressed_response` is true, the response's messages MUST + have the compressed message flag set. + * response payload bodies are sized (in order): 31415, 58979 + * clients are free to assert that the response payload body contents are + zero and comparing the entire response messages against golden responses + ### ping_pong @@ -910,13 +872,6 @@ When the client requests COMPRESSABLE payload, the response includes a payload of the size requested containing all zeros and the payload type is COMPRESSABLE. -### Uncompressable Payload -[Uncompressable Payload]: #uncompressable-payload - -When the client requests UNCOMPRESSABLE payload, the response includes a payload -of the size requested containing uncompressable data and the payload type is -UNCOMPRESSABLE. - ### Echo Status [Echo Status]: #echo-status When the client sends a response_status in the request payload, the server closes From 24b1062f42ef01bd47a458e94423f068ec1765f0 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 8 Jun 2016 15:20:17 -0700 Subject: [PATCH 045/280] Do not close epoll_fd while there are any pollers and add the ability to wake up all pollers when an island is merged --- src/core/lib/iomgr/ev_epoll_linux.c | 113 ++++++++++++++++++++-------- 1 file changed, 80 insertions(+), 33 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 69ab665e155..3a3c136a5aa 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -190,9 +190,18 @@ struct grpc_pollset_set { }; /******************************************************************************* - * Polling-island Definitions + * Polling island Definitions */ +/* The wakeup fd that is used to wake up all threads in a Polling island. This + is useful in the polling island merge operation where we need to wakeup all + the threads currently polling the smaller polling island (so that they can + start polling the new/merged polling island) + + NOTE: This fd is initialized to be readable and MUST NOT be consumed i.e the + threads that woke up MUST NOT call grpc_wakeup_fd_consume_wakeup() */ +static grpc_wakeup_fd polling_island_wakeup_fd; + /* Polling island freelist */ static gpr_mu g_pi_freelist_mu; static polling_island *g_pi_freelist = NULL; @@ -232,6 +241,25 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, } } +/* The caller is expected to hold pi->mu before calling this */ +static void polling_island_add_wakeup_fd_locked(polling_island *pi, + grpc_wakeup_fd *wakeup_fd) { + struct epoll_event ev; + int err; + + ev.events = (uint32_t)(EPOLLIN | EPOLLET); + ev.data.ptr = wakeup_fd; + err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, + GRPC_WAKEUP_FD_GET_READ_FD(wakeup_fd), &ev); + if (err < 0) { + gpr_log(GPR_ERROR, + "Failed to add grpc_wake_up_fd (%d) to the epoll set (epoll_fd: %d)" + ". Error: %s", + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), pi->epoll_fd, + strerror(errno)); + } +} + /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_remove_all_fds_locked(polling_island *pi, bool remove_fd_refs) { @@ -283,8 +311,6 @@ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, static polling_island *polling_island_create(grpc_fd *initial_fd, int initial_ref_cnt) { polling_island *pi = NULL; - struct epoll_event ev; - int err; /* Try to get one from the polling island freelist */ gpr_mu_lock(&g_pi_freelist_mu); @@ -311,17 +337,7 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, } GPR_ASSERT(pi->epoll_fd >= 0); - ev.events = (uint32_t)(EPOLLIN | EPOLLET); - ev.data.ptr = NULL; - err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), &ev); - if (err < 0) { - gpr_log(GPR_ERROR, - "Failed to add grpc_global_wake_up_fd (%d) to the epoll set " - "(epoll_fd: %d) with error: %s", - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), pi->epoll_fd, - strerror(errno)); - } + polling_island_add_wakeup_fd_locked(pi, &grpc_global_wakeup_fd); pi->ref_cnt = initial_ref_cnt; pi->merged_to = NULL; @@ -496,13 +512,15 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { GPR_SWAP(polling_island *, p, q); } - /* "Merge" p with q i.e move all the fds from p (the polling_island with fewer - fds) to q. - Note: Not altering the ref counts on the affected fds here because they - would effectively remain unchanged */ + /* "Merge" p with q i.e move all the fds from p (The one with fewer fds) to q + )Note that the refcounts on the fds being moved will not change here. This + is why the last parameter in the following two functions is 'false') */ polling_island_add_fds_locked(q, p->fds, p->fd_cnt, false); polling_island_remove_all_fds_locked(p, false); + /* Wakeup all the pollers (if any) on p so that they can pickup this change */ + polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); + /* The merged polling island inherits all the ref counts of the island merging with it */ q->ref_cnt += p->ref_cnt; @@ -516,6 +534,8 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { static void polling_island_global_init() { gpr_mu_init(&g_pi_freelist_mu); g_pi_freelist = NULL; + grpc_wakeup_fd_init(&polling_island_wakeup_fd); + grpc_wakeup_fd_wakeup(&polling_island_wakeup_fd); } static void polling_island_global_shutdown() { @@ -529,8 +549,9 @@ static void polling_island_global_shutdown() { gpr_free(g_pi_freelist); g_pi_freelist = next; } - gpr_mu_destroy(&g_pi_freelist_mu); + + grpc_wakeup_fd_destroy(&polling_island_wakeup_fd); } /******************************************************************************* @@ -973,6 +994,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; int epoll_fd = -1; int ep_rv; + polling_island *pi = NULL; GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the @@ -983,13 +1005,19 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, - pollset->polling_island->mu */ gpr_mu_lock(&pollset->pi_mu); - if (pollset->polling_island == NULL) { - pollset->polling_island = polling_island_create(NULL, 1); + pi = pollset->polling_island; + if (pi == NULL) { + pi = polling_island_create(NULL, 1); } - pollset->polling_island = - polling_island_update_and_lock(pollset->polling_island, 1, 0); - epoll_fd = pollset->polling_island->epoll_fd; + /* In addition to locking the polling island, add a ref so that the island + does not get destroyed (which means the epoll_fd won't be closed) while + we are are doing an epoll_wait() on the epoll_fd */ + pi = polling_island_update_and_lock(pi, 1, 1); + epoll_fd = pi->epoll_fd; + + /* Update the pollset->polling_island */ + pollset->polling_island = pi; #ifdef GRPC_EPOLL_DEBUG if (pollset->polling_island->fd_cnt == 0) { @@ -1013,25 +1041,29 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, sig_mask); if (ep_rv < 0) { if (errno != EINTR) { - /* TODO (sreek) - Do not log an error in case of bad file descriptor - * (A bad file descriptor here would just mean that the epoll set was - * merged with another epoll set and that the current epoll_fd is - * closed) */ gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); } else { + /* We were interrupted. Save an interation by doing a zero timeout + epoll_wait to see if there are any other events of interest */ ep_rv = epoll_wait(epoll_fd, ep_ev, GRPC_EPOLL_MAX_EVENTS, 0); } } int i; for (i = 0; i < ep_rv; ++i) { - grpc_fd *fd = ep_ev[i].data.ptr; - int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); - int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); - int write_ev = ep_ev[i].events & EPOLLOUT; - if (fd == NULL) { + void *data_ptr = ep_ev[i].data.ptr; + if (data_ptr == &grpc_global_wakeup_fd) { grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + } else if (data_ptr == &polling_island_wakeup_fd) { + /* This means that our polling island is merged with a different + island. We do not have to do anything here since the subsequent call + to the function pollset_work_and_unlock() will pick up the correct + epoll_fd */ } else { + grpc_fd *fd = data_ptr; + int cancel = ep_ev[i].events & (EPOLLERR | EPOLLHUP); + int read_ev = ep_ev[i].events & (EPOLLIN | EPOLLPRI); + int write_ev = ep_ev[i].events & EPOLLOUT; if (read_ev || cancel) { fd_become_readable(exec_ctx, fd, pollset); } @@ -1041,6 +1073,21 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, } } } while (ep_rv == GRPC_EPOLL_MAX_EVENTS); + + GPR_ASSERT(pi != NULL); + + /* Before leaving, release the extra ref we added to the polling island */ + /* It is important to note that at this point 'pi' may not be the same as + * pollset->polling_island. This is because pollset->polling_island pointer + * gets updated whenever the underlying polling island is merged with another + * island and while we are doing epoll_wait() above, the polling island may + * have been merged */ + + /* TODO (sreek) - Change the ref count on polling island to gpr_atm so that + * we do not have to do this here */ + gpr_mu_lock(&pi->mu); + polling_island_unref_and_unlock(pi, 1); + GPR_TIMER_END("pollset_work_and_unlock", 0); } From e682e46a9e451a94cfcb1cf9c927185abd81fceb Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 8 Jun 2016 15:40:21 -0700 Subject: [PATCH 046/280] Add TODOs --- src/core/lib/iomgr/ev_epoll_linux.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 3a3c136a5aa..2e871a4f1ba 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -83,6 +83,7 @@ struct grpc_fd { this indicates that the 'fd' on this structure is no longer valid */ bool orphaned; + /* TODO: sreek - Move this a lockfree implementation */ grpc_closure *read_closure; grpc_closure *write_closure; @@ -166,6 +167,9 @@ struct grpc_pollset { /* The polling island to which this pollset belongs to and the mutex protecting the field */ + /* TODO: sreek: This lock might actually be adding more overhead to the + critical path (i.e pollset_work() function). Consider removing this lock + and just using the overall pollset lock */ gpr_mu pi_mu; struct polling_island *polling_island; }; From 3dbf4d61b26e2364a974e47f16f3a655d3eda908 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 8 Jun 2016 16:26:45 -0700 Subject: [PATCH 047/280] More TODOs --- src/core/lib/iomgr/ev_epoll_linux.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 2e871a4f1ba..046ec5e7407 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -83,7 +83,7 @@ struct grpc_fd { this indicates that the 'fd' on this structure is no longer valid */ bool orphaned; - /* TODO: sreek - Move this a lockfree implementation */ + /* TODO: sreek - Move this to a lockfree implementation */ grpc_closure *read_closure; grpc_closure *write_closure; @@ -124,6 +124,8 @@ static void fd_global_shutdown(void); /******************************************************************************* * Polling-island Declarations */ +/* TODO: sree: Consider making ref_cnt and merged_to to gpr_atm - This would + * significantly reduce the number of mutex acquisition calls. */ typedef struct polling_island { gpr_mu mu; int ref_cnt; @@ -177,6 +179,12 @@ struct grpc_pollset { /******************************************************************************* * Pollset-set Declarations */ +/* TODO: sreek - Change the pollset_set implementation such that a pollset_set + * directly points to a polling_island (and adding an fd/pollset/pollset_set to + * the current pollset_set would result in polling island merges. This would + * remove the need to maintain fd_count here. This will also significantly + * simplify the grpc_fd structure since we would no longer need to explicitly + * maintain the orphaned state */ struct grpc_pollset_set { gpr_mu mu; From 8e4926c0eeb0df9e5c8029136e39f6c8700f0814 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 8 Jun 2016 20:33:19 -0700 Subject: [PATCH 048/280] pollset_kick optimization (do not kick any other thread if the current thread can be kicked) --- src/core/lib/iomgr/ev_epoll_linux.c | 153 ++++++++++++++++------------ 1 file changed, 87 insertions(+), 66 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 046ec5e7407..d45f87c2f8e 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -844,6 +844,8 @@ static void fd_notify_on_write(grpc_exec_ctx *exec_ctx, grpc_fd *fd, /******************************************************************************* * Pollset Definitions */ +GPR_TLS_DECL(g_current_thread_pollset); +GPR_TLS_DECL(g_current_thread_worker); static void sig_handler(int sig_num) { #ifdef GRPC_EPOLL_DEBUG @@ -859,11 +861,15 @@ static void poller_kick_init() { /* Global state management */ static void pollset_global_init(void) { grpc_wakeup_fd_init(&grpc_global_wakeup_fd); + gpr_tls_init(&g_current_thread_pollset); + gpr_tls_init(&g_current_thread_worker); poller_kick_init(); } static void pollset_global_shutdown(void) { grpc_wakeup_fd_destroy(&grpc_global_wakeup_fd); + gpr_tls_destroy(&g_current_thread_pollset); + gpr_tls_destroy(&g_current_thread_worker); } static void pollset_worker_kick(grpc_pollset_worker *worker) { @@ -915,7 +921,9 @@ static void pollset_kick(grpc_pollset *p, GPR_TIMER_BEGIN("pollset_kick.broadcast", 0); for (worker = p->root_worker.next; worker != &p->root_worker; worker = worker->next) { - pollset_worker_kick(worker); + if (gpr_tls_get(&g_current_thread_worker) != (intptr_t)worker) { + pollset_worker_kick(worker); + } } } else { p->kicked_without_pollers = true; @@ -923,9 +931,18 @@ static void pollset_kick(grpc_pollset *p, GPR_TIMER_END("pollset_kick.broadcast", 0); } else { GPR_TIMER_MARK("kicked_specifically", 0); - pollset_worker_kick(worker); + if (gpr_tls_get(&g_current_thread_worker) != (intptr_t)worker) { + pollset_worker_kick(worker); + } } - } else { + } else if (gpr_tls_get(&g_current_thread_pollset) != (intptr_t)p) { + /* Since worker == NULL, it means that we can kick "any" worker on this + pollset 'p'. If 'p' happens to be the same pollset this thread is + currently polling (i.e in pollset_work() function), then there is no need + to kick any other worker since the current thread can just absorb the + kick. This is the reason why we enter this case only when + g_current_thread_pollset is != p */ + GPR_TIMER_MARK("kick_anonymous", 0); worker = pop_front_worker(p); if (worker != NULL) { @@ -999,6 +1016,69 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { gpr_mu_unlock(&fd->mu); } +/* Release the reference to pollset->polling_island and set it to NULL. + pollset->mu must be held */ +static void pollset_release_polling_island_locked(grpc_pollset *pollset) { + gpr_mu_lock(&pollset->pi_mu); + if (pollset->polling_island) { + pollset->polling_island = + polling_island_update_and_lock(pollset->polling_island, 1, 0); + polling_island_unref_and_unlock(pollset->polling_island, 1); + pollset->polling_island = NULL; + } + gpr_mu_unlock(&pollset->pi_mu); +} + +static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset) { + /* The pollset cannot have any workers if we are at this stage */ + GPR_ASSERT(!pollset_has_workers(pollset)); + + pollset->finish_shutdown_called = true; + pollset_release_polling_island_locked(pollset); + + grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); +} + +/* pollset->mu lock must be held by the caller before calling this */ +static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_closure *closure) { + GPR_TIMER_BEGIN("pollset_shutdown", 0); + GPR_ASSERT(!pollset->shutting_down); + pollset->shutting_down = true; + pollset->shutdown_done = closure; + pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); + + /* If the pollset has any workers, we cannot call finish_shutdown_locked() + because it would release the underlying polling island. In such a case, we + let the last worker call finish_shutdown_locked() from pollset_work() */ + if (!pollset_has_workers(pollset)) { + GPR_ASSERT(!pollset->finish_shutdown_called); + GPR_TIMER_MARK("pollset_shutdown.finish_shutdown_locked", 0); + finish_shutdown_locked(exec_ctx, pollset); + } + GPR_TIMER_END("pollset_shutdown", 0); +} + +/* pollset_shutdown is guaranteed to be called before pollset_destroy. So other + * than destroying the mutexes, there is nothing special that needs to be done + * here */ +static void pollset_destroy(grpc_pollset *pollset) { + GPR_ASSERT(!pollset_has_workers(pollset)); + gpr_mu_destroy(&pollset->pi_mu); + gpr_mu_destroy(&pollset->mu); +} + +static void pollset_reset(grpc_pollset *pollset) { + GPR_ASSERT(pollset->shutting_down); + GPR_ASSERT(!pollset_has_workers(pollset)); + pollset->shutting_down = false; + pollset->finish_shutdown_called = false; + pollset->kicked_without_pollers = false; + pollset->shutdown_done = NULL; + pollset_release_polling_island_locked(pollset); +} + #define GRPC_EPOLL_MAX_EVENTS 1000 static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, int timeout_ms, @@ -1103,69 +1183,6 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, GPR_TIMER_END("pollset_work_and_unlock", 0); } -/* Release the reference to pollset->polling_island and set it to NULL. - pollset->mu must be held */ -static void pollset_release_polling_island_locked(grpc_pollset *pollset) { - gpr_mu_lock(&pollset->pi_mu); - if (pollset->polling_island) { - pollset->polling_island = - polling_island_update_and_lock(pollset->polling_island, 1, 0); - polling_island_unref_and_unlock(pollset->polling_island, 1); - pollset->polling_island = NULL; - } - gpr_mu_unlock(&pollset->pi_mu); -} - -static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset) { - /* The pollset cannot have any workers if we are at this stage */ - GPR_ASSERT(!pollset_has_workers(pollset)); - - pollset->finish_shutdown_called = true; - pollset_release_polling_island_locked(pollset); - - grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); -} - -/* pollset->mu lock must be held by the caller before calling this */ -static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_closure *closure) { - GPR_TIMER_BEGIN("pollset_shutdown", 0); - GPR_ASSERT(!pollset->shutting_down); - pollset->shutting_down = true; - pollset->shutdown_done = closure; - pollset_kick(pollset, GRPC_POLLSET_KICK_BROADCAST); - - /* If the pollset has any workers, we cannot call finish_shutdown_locked() - because it would release the underlying polling island. In such a case, we - let the last worker call finish_shutdown_locked() from pollset_work() */ - if (!pollset_has_workers(pollset)) { - GPR_ASSERT(!pollset->finish_shutdown_called); - GPR_TIMER_MARK("pollset_shutdown.finish_shutdown_locked", 0); - finish_shutdown_locked(exec_ctx, pollset); - } - GPR_TIMER_END("pollset_shutdown", 0); -} - -/* pollset_shutdown is guaranteed to be called before pollset_destroy. So other - * than destroying the mutexes, there is nothing special that needs to be done - * here */ -static void pollset_destroy(grpc_pollset *pollset) { - GPR_ASSERT(!pollset_has_workers(pollset)); - gpr_mu_destroy(&pollset->pi_mu); - gpr_mu_destroy(&pollset->mu); -} - -static void pollset_reset(grpc_pollset *pollset) { - GPR_ASSERT(pollset->shutting_down); - GPR_ASSERT(!pollset_has_workers(pollset)); - pollset->shutting_down = false; - pollset->finish_shutdown_called = false; - pollset->kicked_without_pollers = false; - pollset->shutdown_done = NULL; - pollset_release_polling_island_locked(pollset); -} - /* pollset->mu lock must be held by the caller before calling this. The function pollset_work() may temporarily release the lock (pollset->mu) during the course of its execution but it will always re-acquire the lock and @@ -1184,6 +1201,8 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, worker.pt_id = pthread_self(); *worker_hdl = &worker; + gpr_tls_set(&g_current_thread_pollset, (intptr_t)pollset); + gpr_tls_set(&g_current_thread_worker, (intptr_t)&worker); if (pollset->kicked_without_pollers) { /* If the pollset was kicked without pollers, pretend that the current @@ -1226,6 +1245,8 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } *worker_hdl = NULL; + gpr_tls_set(&g_current_thread_pollset, (intptr_t)0); + gpr_tls_set(&g_current_thread_worker, (intptr_t)0); GPR_TIMER_END("pollset_work", 0); } From 0553a436610201b252cfce0ed5d2cea69da15e85 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 9 Jun 2016 00:42:41 -0700 Subject: [PATCH 049/280] Fix refcounting bug in polling_island_merge --- src/core/lib/iomgr/ev_epoll_linux.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index d45f87c2f8e..66bbae52b28 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -512,7 +512,6 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Get locks on both the polling islands */ polling_island_pair_update_and_lock(&p, &q); - /* TODO: sreek: Think about this scenario some more */ if (p == q) { /* Nothing needs to be done here */ gpr_mu_unlock(&p->mu); @@ -525,7 +524,7 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { } /* "Merge" p with q i.e move all the fds from p (The one with fewer fds) to q - )Note that the refcounts on the fds being moved will not change here. This + Note that the refcounts on the fds being moved will not change here. This is why the last parameter in the following two functions is 'false') */ polling_island_add_fds_locked(q, p->fds, p->fd_cnt, false); polling_island_remove_all_fds_locked(p, false); @@ -533,9 +532,11 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Wakeup all the pollers (if any) on p so that they can pickup this change */ polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); - /* The merged polling island inherits all the ref counts of the island merging - with it */ + /* - The merged polling island (i.e q) inherits all the ref counts of the + island merging with it (i.e p) + - The island p will lose a ref count */ q->ref_cnt += p->ref_cnt; + p->ref_cnt--; gpr_mu_unlock(&p->mu); gpr_mu_unlock(&q->mu); From 727440216553c01925c0f5bc88293108bb3f051f Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 9 Jun 2016 09:42:06 -0700 Subject: [PATCH 050/280] Check epoll is actually available. set GPR_LINUX_EPOLL only in GLIBC ver 2.9 and above --- include/grpc/impl/codegen/port_platform.h | 2 +- src/core/lib/iomgr/ev_epoll_linux.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/include/grpc/impl/codegen/port_platform.h b/include/grpc/impl/codegen/port_platform.h index 7a6ec53fb4c..affef9e66b6 100644 --- a/include/grpc/impl/codegen/port_platform.h +++ b/include/grpc/impl/codegen/port_platform.h @@ -189,7 +189,6 @@ #define GPR_GCC_ATOMIC 1 #define GPR_GCC_TLS 1 #define GPR_LINUX 1 -#define GPR_LINUX_EPOLL 1 #define GPR_LINUX_LOG #define GPR_LINUX_MULTIPOLL_WITH_EPOLL 1 #define GPR_POSIX_WAKEUP_FD 1 @@ -201,6 +200,7 @@ #ifdef __GLIBC_PREREQ #if __GLIBC_PREREQ(2, 9) #define GPR_LINUX_EVENTFD 1 +#define GPR_LINUX_EPOLL 1 #endif #if __GLIBC_PREREQ(2, 10) #define GPR_LINUX_SOCKETUTILS 1 diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 66bbae52b28..d2d5d2852b9 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -1481,7 +1481,26 @@ static const grpc_event_engine_vtable vtable = { .shutdown_engine = shutdown_engine, }; +/* It is possible that GLIBC has epoll but the underlying kernel doesn't. + * Create a dummy epoll_fd to make sure epoll support is available */ +static bool is_epoll_available() { + int fd = epoll_create1(EPOLL_CLOEXEC); + if (fd < 0) { + gpr_log( + GPR_ERROR, + "epoll_create1 failed with error: %d. Not using epoll polling engine", + fd); + return false; + } + close(fd); + return true; +} + const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { + if (!is_epoll_available()) { + return NULL; + } + fd_global_init(); pollset_global_init(); polling_island_global_init(); From 74686ce7c4354020b4e48ba067ac7635d0e4edb2 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Thu, 9 Jun 2016 15:33:33 -0700 Subject: [PATCH 051/280] Completed interop spec, as well as code for c++ --- Makefile | 6 +- build.yaml | 2 +- doc/interop-test-descriptions.md | 161 +++++++++++++++--- src/proto/grpc/testing/messages.proto | 17 +- test/cpp/interop/client.cc | 63 +++++-- test/cpp/interop/interop_client.cc | 99 ++++++----- test/cpp/interop/interop_client.h | 6 +- .../{server_main.cc => interop_server.cc} | 61 +++++-- test/cpp/interop/rnd.dat | Bin 524288 -> 0 bytes test/cpp/interop/server_helper.cc | 4 + test/cpp/interop/server_helper.h | 1 + tools/run_tests/sources_and_headers.json | 2 +- .../interop_server_main.vcxproj | 2 +- .../interop_server_main.vcxproj.filters | 2 +- 14 files changed, 325 insertions(+), 101 deletions(-) rename test/cpp/interop/{server_main.cc => interop_server.cc} (83%) delete mode 100644 test/cpp/interop/rnd.dat diff --git a/Makefile b/Makefile index 5a5dd5e242c..7738a8d8ff0 100644 --- a/Makefile +++ b/Makefile @@ -4194,7 +4194,7 @@ LIBINTEROP_SERVER_MAIN_SRC = \ $(GENDIR)/src/proto/grpc/testing/empty.pb.cc $(GENDIR)/src/proto/grpc/testing/empty.grpc.pb.cc \ $(GENDIR)/src/proto/grpc/testing/messages.pb.cc $(GENDIR)/src/proto/grpc/testing/messages.grpc.pb.cc \ $(GENDIR)/src/proto/grpc/testing/test.pb.cc $(GENDIR)/src/proto/grpc/testing/test.grpc.pb.cc \ - test/cpp/interop/server_main.cc \ + test/cpp/interop/interop_server.cc \ PUBLIC_HEADERS_CXX += \ @@ -4240,7 +4240,7 @@ ifneq ($(NO_DEPS),true) -include $(LIBINTEROP_SERVER_MAIN_OBJS:.o=.dep) endif endif -$(OBJDIR)/$(CONFIG)/test/cpp/interop/server_main.o: $(GENDIR)/src/proto/grpc/testing/empty.pb.cc $(GENDIR)/src/proto/grpc/testing/empty.grpc.pb.cc $(GENDIR)/src/proto/grpc/testing/messages.pb.cc $(GENDIR)/src/proto/grpc/testing/messages.grpc.pb.cc $(GENDIR)/src/proto/grpc/testing/test.pb.cc $(GENDIR)/src/proto/grpc/testing/test.grpc.pb.cc +$(OBJDIR)/$(CONFIG)/test/cpp/interop/interop_server.o: $(GENDIR)/src/proto/grpc/testing/empty.pb.cc $(GENDIR)/src/proto/grpc/testing/empty.grpc.pb.cc $(GENDIR)/src/proto/grpc/testing/messages.pb.cc $(GENDIR)/src/proto/grpc/testing/messages.grpc.pb.cc $(GENDIR)/src/proto/grpc/testing/test.pb.cc $(GENDIR)/src/proto/grpc/testing/test.grpc.pb.cc LIBQPS_SRC = \ @@ -14569,8 +14569,8 @@ test/cpp/end2end/test_service_impl.cc: $(OPENSSL_DEP) test/cpp/interop/client.cc: $(OPENSSL_DEP) test/cpp/interop/client_helper.cc: $(OPENSSL_DEP) test/cpp/interop/interop_client.cc: $(OPENSSL_DEP) +test/cpp/interop/interop_server.cc: $(OPENSSL_DEP) test/cpp/interop/server_helper.cc: $(OPENSSL_DEP) -test/cpp/interop/server_main.cc: $(OPENSSL_DEP) test/cpp/qps/client_async.cc: $(OPENSSL_DEP) test/cpp/qps/client_sync.cc: $(OPENSSL_DEP) test/cpp/qps/driver.cc: $(OPENSSL_DEP) diff --git a/build.yaml b/build.yaml index 302e3f99e04..335e68409f7 100644 --- a/build.yaml +++ b/build.yaml @@ -1101,7 +1101,7 @@ libs: - src/proto/grpc/testing/empty.proto - src/proto/grpc/testing/messages.proto - src/proto/grpc/testing/test.proto - - test/cpp/interop/server_main.cc + - test/cpp/interop/interop_server.cc deps: - interop_server_helper - grpc++_test_util diff --git a/doc/interop-test-descriptions.md b/doc/interop-test-descriptions.md index 63b0022c3f6..a023d80c50d 100644 --- a/doc/interop-test-descriptions.md +++ b/doc/interop-test-descriptions.md @@ -90,26 +90,84 @@ Client asserts: * clients are free to assert that the response payload body contents are zero and comparing the entire response message against a golden response +### client_compressed_unary + +This test verifies the client can compress unary messages. It sends one +unary request for a compressable payload type, with and without compression. + +Server features: +* [UnaryCall][] +* [Compressed Request][] + +Procedure: + 1. Client calls UnaryCall with: + + ``` + { + expect_compressed_request: true + response_type: COMPRESSABLE + response_size: 314159 + payload:{ + body: 271828 bytes of zeros + } + } + ``` + + ``` + { + expect_compressed_request: false + response_type: COMPRESSABLE + response_size: 314159 + payload:{ + body: 271828 bytes of zeros + } + } + ``` + + Client asserts: + * call was successful + * response payload type is COMPRESSABLE + * if `request_compressed_response` is false, the response MUST NOT have the + compressed message flag set. + * if `request_compressed_response` is true, the response MUST have the + compressed message flag set. + * response payload body is 314159 bytes in size + * clients are free to assert that the response payload body contents are + zero and comparing the entire response message against a golden response + + ### server_compressed_unary -This test verifies compressed server-only unary calls succeed in sending -messages. It sends one unary request for every payload type, with and without -requesting a compressed response from the server. +This test verifies the server can compress unary messages. It sends one unary +request for a COMPRESSABLE payload type, with and without requesting a +compressed response from the server. -In all scenarios, whether compression was actually performed is determined by -the compression bit in the response's message flags. +Whether compression was actually performed is determined by the compression bit +in the response's message flags. Server features: * [UnaryCall][] * [Compressable Payload][] +* [Compressed Response][] Procedure: 1. Client calls UnaryCall with: ``` { - request_compressed_response: bool + request_compressed_response: true + response_type: COMPRESSABLE + response_size: 314159 + payload:{ + body: 271828 bytes of zeros + } + } + ``` + + ``` + { + request_compressed_response: false response_type: COMPRESSABLE response_size: 314159 payload:{ @@ -120,10 +178,10 @@ Procedure: Client asserts: * call was successful * response payload type is COMPRESSABLE - * if `request_compressed_response` is false, the response MUST NOT have the - compressed message flag set. - * if `request_compressed_response` is true, the response MUST have the + * when `request_compressed_response` is true, the response MUST have the compressed message flag set. + * when `request_compressed_response` is false, the response MUST NOT have + the compressed message flag set. * response payload body is 314159 bytes in size * clients are free to assert that the response payload body contents are zero and comparing the entire response message against a golden response @@ -224,11 +282,12 @@ Client asserts: ### server_compressed_streaming -This test verifies that server-only compressed streaming succeeds. +This test verifies that the server can compress streaming messages. Server features: * [StreamingOutputCall][] * [Compressable Payload][] +* [Compressed Response][] Procedure: @@ -262,17 +321,56 @@ Procedure: Client asserts: * call was successful - * exactly four responses + * exactly two responses * response payloads are COMPRESSABLE - * if `request_compressed_response` is false, the response's messages MUST + * when `request_compressed_response` is false, the response's messages MUST NOT have the compressed message flag set. - * if `request_compressed_response` is true, the response's messages MUST + * when `request_compressed_response` is true, the response's messages MUST have the compressed message flag set. * response payload bodies are sized (in order): 31415, 58979 * clients are free to assert that the response payload body contents are zero and comparing the entire response messages against golden responses +### client_compressed_streaming + +This test verifies that the client can compress streaming messages. + +Server features: +* [StreamingInputCall][] +* [Compressed Request][] + +Procedure: + 1. Client calls StreamingInputCall + 1. Client sends: + + ``` + { + expect_compressed_request: true + payload:{ + body: 27182 bytes of zeros + } + } + ``` + + 1. Client then sends: + + ``` + { + expect_compressed_request: false + payload:{ + body: 45904 bytes of zeros + } + } + ``` + + 6. Client half-closes + + Client asserts: + * call was successful + * response aggregated_payload_size is 73086 + + ### ping_pong This test verifies that full duplex bidi is supported. @@ -373,7 +471,8 @@ with desired oauth scope. The test uses `--default_service_account` with GCE service account email and `--oauth_scope` with the OAuth scope to use. For testing against -grpc-test.sandbox.googleapis.com, "https://www.googleapis.com/auth/xapi.zoo" should +grpc-test.sandbox.googleapis.com, "https://www.googleapis.com/auth/xapi.zoo" +should be passed in as `--oauth_scope`. Server features: @@ -400,7 +499,8 @@ Procedure: Client asserts: * call was successful -* received SimpleResponse.username equals the value of `--default_service_account` flag +* received SimpleResponse.username equals the value of + `--default_service_account` flag * received SimpleResponse.oauth_scope is in `--oauth_scope` * response payload body is 314159 bytes in size * clients are free to assert that the response payload body contents are zero @@ -444,7 +544,8 @@ Client asserts: * call was successful * received SimpleResponse.username is not empty and is in the json key file used by the auth library. The client can optionally check the username matches the -email address in the key file or equals the value of `--default_service_account` flag. +email address in the key file or equals the value of `--default_service_account` +flag. * response payload body is 314159 bytes in size * clients are free to assert that the response payload body contents are zero and comparing the entire response message against a golden response @@ -470,8 +571,8 @@ variable GOOGLE_APPLICATION_CREDENTIALS, *OR* if GCE credentials is used to fetch the token, `--default_service_account` can be used to pass in GCE service account email. - uses the flag `--oauth_scope` for the oauth scope. For testing against -grpc-test.sandbox.googleapis.com, "https://www.googleapis.com/auth/xapi.zoo" should -be passed as the `--oauth_scope`. +grpc-test.sandbox.googleapis.com, "https://www.googleapis.com/auth/xapi.zoo" +should be passed as the `--oauth_scope`. Server features: * [UnaryCall][] @@ -481,7 +582,8 @@ Server features: Procedure: 1. Client uses the auth library to obtain an authorization token - 2. Client configures the channel to use AccessTokenCredentials with the access token obtained in step 1 + 2. Client configures the channel to use AccessTokenCredentials with the access + token obtained in step 1 3. Client calls UnaryCall with the following message ``` @@ -502,17 +604,17 @@ json key file or GCE default service account email. Similar to the other auth tests, this test is only for cloud-to-prod path. -This test verifies unary calls succeed in sending messages using a JWT or a service account -credentials set on the RPC. +This test verifies unary calls succeed in sending messages using a JWT or a +service account credentials set on the RPC. The test - uses the flag `--service_account_key_file` with the path to a json key file downloaded from https://console.developers.google.com. Alternately, if using a usable auth implementation, it may specify the file location in the environment variable GOOGLE_APPLICATION_CREDENTIALS -- optionally uses the flag `--oauth_scope` for the oauth scope if implementator +- optionally uses the flag `--oauth_scope` for the oauth scope if implementator wishes to use service account credential instead of JWT credential. For testing -against grpc-test.sandbox.googleapis.com, oauth scope +against grpc-test.sandbox.googleapis.com, oauth scope "https://www.googleapis.com/auth/xapi.zoo" should be used. Server features: @@ -839,6 +941,19 @@ payload body of size `SimpleRequest.response_size` bytes and type as appropriate for the `SimpleRequest.response_type`. If the server does not support the `response_type`, then it should fail the RPC with `INVALID_ARGUMENT`. +### CompressedResponse +[CompressedResponse]: #compressedresponse + +When the client sets `SimpleRequest.request_compressed_response` to true, the +response is sent back compressed. + +### CompressedRequest +[CompressedRequest]: #compressedrequest + +When the client sets `SimpleRequest.expect_compressed_request ` to true, the +server expects the client request to be compressed. If it's not, it fails +the RPC with `INVALID_ARGUMENT`. + ### StreamingInputCall [StreamingInputCall]: #streaminginputcall diff --git a/src/proto/grpc/testing/messages.proto b/src/proto/grpc/testing/messages.proto index e1090156ab4..99b75dea3d5 100644 --- a/src/proto/grpc/testing/messages.proto +++ b/src/proto/grpc/testing/messages.proto @@ -38,9 +38,6 @@ package grpc.testing; enum PayloadType { // Compressable text format. COMPRESSABLE = 0; - - // Uncompressable binary format. - UNCOMPRESSABLE = 1; } // A block of data, to simply increase gRPC message size. @@ -82,6 +79,12 @@ message SimpleRequest { // Whether server should return a given status EchoStatus response_status = 7; + + // Whether the server should expect this request to be compressed. + bool expect_compressed_request = 8; + + // The type of payload. + PayloadType payload_type = 9; } // Unary response, as configured by the request. @@ -100,6 +103,12 @@ message StreamingInputCallRequest { // Optional input payload sent along with the request. Payload payload = 1; + // The type of payload. + PayloadType payload_type = 2; + + // Whether the server should expect this request to be compressed. + bool expect_compressed_request = 3; + // Not expecting any payload from the response. } @@ -135,7 +144,7 @@ message StreamingOutputCallRequest { Payload payload = 3; // Whether to request the server to compress the response. - bool request_compressed_response = 6; + bool request_compressed_response = 4; // Whether server should return a given status EchoStatus response_status = 7; diff --git a/test/cpp/interop/client.cc b/test/cpp/interop/client.cc index 77278249794..c7d081100e2 100644 --- a/test/cpp/interop/client.cc +++ b/test/cpp/interop/client.cc @@ -40,7 +40,9 @@ #include #include #include +#include +#include "src/core/lib/support/string.h" #include "test/cpp/interop/client_helper.h" #include "test/cpp/interop/interop_client.h" #include "test/cpp/util/test_config.h" @@ -55,10 +57,14 @@ DEFINE_string(test_case, "large_unary", "Configure different test cases. Valid options are: " "empty_unary : empty (zero bytes) request and response; " "large_unary : single request and (large) response; " - "large_compressed_unary : single request and compressed (large) " - "response; " + + "client_compressed_unary : single compressed request; " + "server_compressed_unary : single compressed response; " + "client_streaming : request streaming with single response; " "server_streaming : single request with response streaming; " + "client_compressed_streaming : compressed request streaming with " + "single response; " "server_compressed_streaming : single request with compressed " "response streaming; " "slow_consumer : single request with response; " @@ -104,14 +110,18 @@ int main(int argc, char** argv) { client.DoEmpty(); } else if (FLAGS_test_case == "large_unary") { client.DoLargeUnary(); - } else if (FLAGS_test_case == "large_compressed_unary") { - client.DoLargeCompressedUnary(); + } else if (FLAGS_test_case == "server_compressed_unary") { + client.DoServerCompressedUnary(); + } else if (FLAGS_test_case == "client_compressed_unary") { + client.DoClientCompressedUnary(); } else if (FLAGS_test_case == "client_streaming") { client.DoRequestStreaming(); } else if (FLAGS_test_case == "server_streaming") { client.DoResponseStreaming(); } else if (FLAGS_test_case == "server_compressed_streaming") { - client.DoResponseCompressedStreaming(); + client.DoServerCompressedStreaming(); + } else if (FLAGS_test_case == "client_compressed_streaming") { + client.DoClientCompressedStreaming(); } else if (FLAGS_test_case == "slow_consumer") { client.DoResponseStreamingWithSlowConsumer(); } else if (FLAGS_test_case == "half_duplex") { @@ -144,9 +154,12 @@ int main(int argc, char** argv) { } else if (FLAGS_test_case == "all") { client.DoEmpty(); client.DoLargeUnary(); + client.DoClientCompressedUnary(); + client.DoServerCompressedUnary(); client.DoRequestStreaming(); client.DoResponseStreaming(); - client.DoResponseCompressedStreaming(); + client.DoClientCompressedStreaming(); + client.DoServerCompressedStreaming(); client.DoHalfDuplex(); client.DoPingPong(); client.DoCancelAfterBegin(); @@ -165,14 +178,36 @@ int main(int argc, char** argv) { } // compute_engine_creds only runs in GCE. } else { - gpr_log( - GPR_ERROR, - "Unsupported test case %s. Valid options are all|empty_unary|" - "large_unary|large_compressed_unary|client_streaming|server_streaming|" - "server_compressed_streaming|half_duplex|ping_pong|cancel_after_begin|" - "cancel_after_first_response|timeout_on_sleeping_server|empty_stream|" - "compute_engine_creds|jwt_token_creds|oauth2_auth_token|per_rpc_creds", - "status_code_and_message|custom_metadata", FLAGS_test_case.c_str()); + const char* testcases[] = + { "all", + "cancel_after_begin", + "cancel_after_first_response", + "client_compressed_streaming", + "client_compressed_unary", + "client_streaming", + "compute_engine_creds", + "custom_metadata", + "empty_stream", + "empty_unary", + "half_duplex", + "jwt_token_creds", + "large_unary", + "oauth2_auth_token", + "oauth2_auth_token", + "per_rpc_creds", + "per_rpc_creds", + "ping_pong", + "server_compressed_streaming", + "server_compressed_unary", + "server_streaming", + "status_code_and_message", + "timeout_on_sleeping_server"}; + char* joined_testcases = + gpr_strjoin_sep(testcases, GPR_ARRAY_SIZE(testcases), "\n", NULL); + + gpr_log(GPR_ERROR, "Unsupported test case %s. Valid options are\n%s", + FLAGS_test_case.c_str(), joined_testcases); + gpr_free(joined_testcases); ret = 1; } diff --git a/test/cpp/interop/interop_client.cc b/test/cpp/interop/interop_client.cc index 7705bb15922..e5d37514028 100644 --- a/test/cpp/interop/interop_client.cc +++ b/test/cpp/interop/interop_client.cc @@ -73,23 +73,22 @@ void CompressionChecks(const InteropClientContextInspector& inspector, const SimpleResponse* response) { const grpc_compression_algorithm received_compression = inspector.GetCallCompressionAlgorithm(); - if (request->request_compressed_response() && - received_compression == GRPC_COMPRESS_NONE) { - if (request->request_compressed_response() && - received_compression == GRPC_COMPRESS_NONE) { + if (request->request_compressed_response()) { + if (received_compression == GRPC_COMPRESS_NONE) { // Requested some compression, got NONE. This is an error. gpr_log(GPR_ERROR, "Failure: Requested compression but got uncompressed response " "from server."); abort(); } - } - if (!request->request_compressed_response()) { + if (request->response_type() == PayloadType::COMPRESSABLE) { + // requested compression and compressable response => results should + // always be compressed. + GPR_ASSERT(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS); + } + } else { + // Didn't request compression -> make sure the response is uncompressed GPR_ASSERT(!(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS)); - } else if (request->response_type() == PayloadType::COMPRESSABLE) { - // requested compression and compressable response => results should always - // be compressed. - GPR_ASSERT(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS); } } } // namespace @@ -211,17 +210,6 @@ bool InteropClient::PerformLargeUnary(SimpleRequest* request, GPR_ASSERT(response->payload().body() == grpc::string(kLargeResponseSize, '\0')); break; - case PayloadType::UNCOMPRESSABLE: { - // We don't really check anything: We can't assert that the payload is - // uncompressed because it's the server's prerogative to decide on that, - // and different implementations decide differently (ie, Java always - // compresses when requested to do so, whereas C core throws away the - // compressed payload if the output is larger than the input). - // In addition, we don't compare the actual random bytes received because - // asserting that data is sent/received properly isn't the purpose of this - // test. Moreover, different implementations are also free to use - // different sets of random bytes. - } break; default: GPR_ASSERT(false); } @@ -336,9 +324,39 @@ bool InteropClient::DoLargeUnary() { return true; } -bool InteropClient::DoLargeCompressedUnary() { +bool InteropClient::DoClientCompressedUnary() { + const bool expect_compression[] = {false, true}; + const PayloadType payload_types[] = {COMPRESSABLE}; + for (size_t i = 0; i < GPR_ARRAY_SIZE(payload_types); i++) { + for (size_t j = 0; j < GPR_ARRAY_SIZE(expect_compression); j++) { + char* log_suffix; + gpr_asprintf(&log_suffix, "(compression=%s; payload=%s)", + expect_compression[j] ? "true" : "false", + PayloadType_Name(payload_types[i]).c_str()); + + gpr_log(GPR_DEBUG, "Sending compressed unary request %s.", log_suffix); + SimpleRequest request; + SimpleResponse response; + request.set_response_type(payload_types[i]); + request.set_expect_compressed_request(expect_compression[j]); + + if (!PerformLargeUnary(&request, &response, CompressionChecks)) { + gpr_log(GPR_ERROR, "Compressed unary request failed %s", log_suffix); + gpr_free(log_suffix); + return false; + } + + gpr_log(GPR_DEBUG, "Compressed unary request failed %s", log_suffix); + gpr_free(log_suffix); + } + } + + return true; +} + +bool InteropClient::DoServerCompressedUnary() { const bool request_compression[] = {false, true}; - const PayloadType payload_types[] = {COMPRESSABLE, UNCOMPRESSABLE}; + const PayloadType payload_types[] = {COMPRESSABLE}; for (size_t i = 0; i < GPR_ARRAY_SIZE(payload_types); i++) { for (size_t j = 0; j < GPR_ARRAY_SIZE(request_compression); j++) { char* log_suffix; @@ -346,7 +364,7 @@ bool InteropClient::DoLargeCompressedUnary() { request_compression[j] ? "true" : "false", PayloadType_Name(payload_types[i]).c_str()); - gpr_log(GPR_DEBUG, "Sending a large compressed unary rpc %s.", + gpr_log(GPR_DEBUG, "Sending unary request for compressed response %s.", log_suffix); SimpleRequest request; SimpleResponse response; @@ -354,12 +372,13 @@ bool InteropClient::DoLargeCompressedUnary() { request.set_request_compressed_response(request_compression[j]); if (!PerformLargeUnary(&request, &response, CompressionChecks)) { - gpr_log(GPR_ERROR, "Large compressed unary failed %s", log_suffix); + gpr_log(GPR_ERROR, "Request for compressed unary failed %s", + log_suffix); gpr_free(log_suffix); return false; } - gpr_log(GPR_DEBUG, "Large compressed unary done %s.", log_suffix); + gpr_log(GPR_DEBUG, "Request for compressed unary failed %s", log_suffix); gpr_free(log_suffix); } } @@ -447,9 +466,16 @@ bool InteropClient::DoResponseStreaming() { return true; } -bool InteropClient::DoResponseCompressedStreaming() { +bool InteropClient::DoClientCompressedStreaming() { + // XXX + return false; +} + +bool InteropClient::DoServerCompressedStreaming() { const bool request_compression[] = {false, true}; - const PayloadType payload_types[] = {COMPRESSABLE, UNCOMPRESSABLE}; + const PayloadType payload_types[] = {COMPRESSABLE}; + const std::vector response_stream_sizes = {31415, 58979}; + for (size_t i = 0; i < GPR_ARRAY_SIZE(payload_types); i++) { for (size_t j = 0; j < GPR_ARRAY_SIZE(request_compression); j++) { ClientContext context; @@ -466,11 +492,10 @@ bool InteropClient::DoResponseCompressedStreaming() { request.set_response_type(payload_types[i]); request.set_request_compressed_response(request_compression[j]); - for (size_t k = 0; k < response_stream_sizes.size() / 2; ++k) { + for (size_t k = 0; k < response_stream_sizes.size(); ++k) { ResponseParameters* response_parameter = request.add_response_parameters(); - response_parameter->set_size(response_stream_sizes[k] + - response_stream_sizes[k + 1]); + response_parameter->set_size(response_stream_sizes[k]); } StreamingOutputCallResponse response; @@ -484,11 +509,7 @@ bool InteropClient::DoResponseCompressedStreaming() { switch (response.payload().type()) { case PayloadType::COMPRESSABLE: GPR_ASSERT(response.payload().body() == - grpc::string(response_stream_sizes[k] + - response_stream_sizes[k + 1], - '\0')); - break; - case PayloadType::UNCOMPRESSABLE: + grpc::string(response_stream_sizes[k], '\0')); break; default: GPR_ASSERT(false); @@ -516,14 +537,14 @@ bool InteropClient::DoResponseCompressedStreaming() { gpr_log(GPR_DEBUG, "Response streaming done %s.", log_suffix); gpr_free(log_suffix); - if (k < response_stream_sizes.size() / 2) { + if (k < response_stream_sizes.size()) { // stream->Read() failed before reading all the expected messages. This // is most likely due to a connection failure. gpr_log(GPR_ERROR, - "DoResponseCompressedStreaming(): Responses read (k=%d) is " + "DoServerCompressedStreaming(): Responses read (k=%d) is " "less than the expected messages (i.e " "response_stream_sizes.size()/2 (%d)). (i=%d, j=%d)", - k, response_stream_sizes.size() / 2, i, j); + k, response_stream_sizes.size(), i, j); return TransientFailureOrAbort(); } diff --git a/test/cpp/interop/interop_client.h b/test/cpp/interop/interop_client.h index ae75762bb8f..ea44986fbc4 100644 --- a/test/cpp/interop/interop_client.h +++ b/test/cpp/interop/interop_client.h @@ -64,12 +64,14 @@ class InteropClient { bool DoEmpty(); bool DoLargeUnary(); - bool DoLargeCompressedUnary(); + bool DoServerCompressedUnary(); + bool DoClientCompressedUnary(); bool DoPingPong(); bool DoHalfDuplex(); bool DoRequestStreaming(); bool DoResponseStreaming(); - bool DoResponseCompressedStreaming(); + bool DoServerCompressedStreaming(); + bool DoClientCompressedStreaming(); bool DoResponseStreamingWithSlowConsumer(); bool DoCancelAfterBegin(); bool DoCancelAfterFirstResponse(); diff --git a/test/cpp/interop/server_main.cc b/test/cpp/interop/interop_server.cc similarity index 83% rename from test/cpp/interop/server_main.cc rename to test/cpp/interop/interop_server.cc index bbedda14d25..b328f478fa0 100644 --- a/test/cpp/interop/server_main.cc +++ b/test/cpp/interop/interop_server.cc @@ -48,6 +48,7 @@ #include #include +#include "src/core/lib/transport/byte_stream.h" #include "src/proto/grpc/testing/empty.grpc.pb.h" #include "src/proto/grpc/testing/messages.grpc.pb.h" #include "src/proto/grpc/testing/test.grpc.pb.h" @@ -78,7 +79,6 @@ using grpc::testing::TestService; using grpc::Status; static bool got_sigint = false; -static const char* kRandomFile = "test/cpp/interop/rnd.dat"; const char kEchoInitialMetadataKey[] = "x-grpc-test-echo-initial"; const char kEchoTrailingBinMetadataKey[] = "x-grpc-test-echo-trailing-bin"; @@ -117,16 +117,8 @@ bool SetPayload(PayloadType response_type, int size, Payload* payload) { std::unique_ptr body(new char[size]()); payload->set_body(body.get(), size); } break; - case PayloadType::UNCOMPRESSABLE: { - std::unique_ptr body(new char[size]()); - std::ifstream rnd_file(kRandomFile); - GPR_ASSERT(rnd_file.good()); - rnd_file.read(body.get(), size); - GPR_ASSERT(!rnd_file.eof()); // Requested more rnd bytes than available - payload->set_body(body.get(), size); - } break; default: - GPR_ASSERT(false); + return false; } return true; } @@ -140,6 +132,41 @@ void SetResponseCompression(ServerContext* context, } } +template +bool CheckExpectedCompression(const ServerContext& context, + const RequestType& request) { + const InteropServerContextInspector inspector(context); + const grpc_compression_algorithm received_compression = + inspector.GetCallCompressionAlgorithm(); + + if (request.expect_compressed_request()) { + if (received_compression == GRPC_COMPRESS_NONE) { + // Expected some compression, got NONE. This is an error. + gpr_log(GPR_ERROR, + "Failure: Expected compression but got uncompressed request " + "from client."); + return false; + } + if (request.payload_type() == PayloadType::COMPRESSABLE) { + if (!(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS)) { + gpr_log(GPR_ERROR, + "Failure: Requested compression in a compressable request, but " + "compression bit in message flags not set."); + return false; + } + } + } else { + // Didn't expect compression -> make sure the request is uncompressed + if (inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS) { + gpr_log(GPR_ERROR, + "Failure: Didn't requested compression, but compression bit in " + "message flags set."); + return false; + } + } + return true; +} + class TestServiceImpl : public TestService::Service { public: Status EmptyCall(ServerContext* context, const grpc::testing::Empty* request, @@ -152,10 +179,15 @@ class TestServiceImpl : public TestService::Service { SimpleResponse* response) { MaybeEchoMetadata(context); SetResponseCompression(context, *request); + if (!CheckExpectedCompression(*context, *request)) { + return Status(grpc::StatusCode::INVALID_ARGUMENT, + "Compressed request expectation not met."); + } if (request->response_size() > 0) { if (!SetPayload(request->response_type(), request->response_size(), response->mutable_payload())) { - return Status(grpc::StatusCode::INTERNAL, "Error creating payload."); + return Status(grpc::StatusCode::INVALID_ARGUMENT, + "Error creating payload."); } } @@ -179,7 +211,8 @@ class TestServiceImpl : public TestService::Service { if (!SetPayload(request->response_type(), request->response_parameters(i).size(), response.mutable_payload())) { - return Status(grpc::StatusCode::INTERNAL, "Error creating payload."); + return Status(grpc::StatusCode::INVALID_ARGUMENT, + "Error creating payload."); } write_success = writer->Write(response); } @@ -196,6 +229,10 @@ class TestServiceImpl : public TestService::Service { StreamingInputCallRequest request; int aggregated_payload_size = 0; while (reader->Read(&request)) { + if (!CheckExpectedCompression(*context, request)) { + return Status(grpc::StatusCode::INVALID_ARGUMENT, + "Compressed request expectation not met."); + } if (request.has_payload()) { aggregated_payload_size += request.payload().body().size(); } diff --git a/test/cpp/interop/rnd.dat b/test/cpp/interop/rnd.dat deleted file mode 100644 index 8c7f38f9e0e01b33798cbe743bd5f873b6e0a1af..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 524288 zcmV(pK=8kv?7tv1DE+tQT%$)WUjgz!7gW_ie^{E{v2nb-0l>d60CCAP;RHo8=7TVI zX=?~j4{pUaww5d{@K#XuW&-Z~xivlDcGX1So5`!1E9192zjmb)a0Pza>H(KOCgN*v zqPwm?Mg;4o#y{}#I+`l$-Rq=EA^3YIhY5|QJP*I7@GhoS>C)Fx%95J7-knGrwLlw2 zXC)e98V|$+^J@ivRL4q!s2EX+=aaabN{{H);FLqPQ%IdpO-P`Pu^%TKjLI-?9T1BL zN@}dJH0OR?aq&Co`LDS%xwR&tnU#0}#-G_EB_^|FnBGmA7@bZ9f$M9NW%Jn>P%DGt>NgXk~r5M;v-3I~1$p?3)hYrVXH3acNBwAw+m}I)R!=j!? zKMrA_pCNt`<8^V1g&rP;?hb7Wki zNJPQ7gcVL~z+eZmJ+N5|Yfe0q8RpcN7IKo(#eMCqC3(Q2dls@AnDcq^@27N6?)_Ov z_xB{@4i8ObPyX;{W(-A_yf_OGvdt~2St^73Lq9(q`a>oy06 zp%8KzuPVfqjI+BZ(%v12C0YNqY3nr{+GlW^XP`)OE%YWxutHmQT-Xy~i5UqJYUK`; z0~pu%Pr$U?y<@?5S^l3ey*90G-(QLkmsF<=Vj^l6kctZn#zl>_&+$DLd1>dgItIXb zIPyU|mjS!$#mSq!A_|n>)oSU38FoTKV-NN#Y^tMOovgmM7`;KGjL#1J@D&b1BNEha zZRL{E-x?Z-4EO7U`P8`P`_&i+%!vx83D2FpUC0wO^HKaH=i0WWE5NpxT$Cn5xv z6lW?0CG9qW+-s%UwEmUN+aXGSW1DwTIu1hXHmunBpB6_Kr|6gzed!k07_=el7u7(v2^z{k(L zXJ9vF9a;y3smEH!+H!D9%`!Js3BTI&^#2!`1h~Kv3?wF644Xkm@Iiv%Lut3`$JGCxoFJ`Z2fz&+o;-fHi zGPTplrR$1Cb-H|x%6krl)crr;oS*EP$<1&vmmt<+HHe&#)%T;24mI9V4tp?Sn3o%* zd2n%tJ;<5IA*TkkY#;^W(;gCty_maRdQfSe+wxX(3cA@)c7lrd>!9JG$!a?Xz0l-B zKz4~O=OMe3Ye9LR$B#;UX7Tv;DBAJq(%1JrhN;p(wc1KI5nR_?4zEv9oJ?oT>^?@e zx=CYYFoj?nN;X}+%mk)9xhs^SUD#QdJBWcy2rzAJ-po!w$8<|njb$fWnxe%gerqvA z#&I6MMp_qojq4kAqU@cl$gzJ1ZZ!|B0Ul8XvFN%^BTj&GlRy3~|NUYb9PII+?%vR{ z8@j4NTS>Qezn9FQ-&D}RtH`UmHF=hRT}s~T$_QhBFiL+1A(jcE3_Wj_X40X69-i&a z=!MOy2n@%TMoy7|>8DFwi7j%++y?(TT5^HVHF06ZG|WU*3vNqpkiVSDKzLgjbQ9_gC2fZQ+qSy^x_(Dy#Yv)%BuhnciW=%8l# z2aqP#`3XOA2lsUZEZw$U8~8T~XW;&s-g?mKL$S1dqOD$YpIRgVwCmxtuxNMQP^V#C z$68v|2bn@R4E8xtXa!=cs)edfA=@i(NfMndtkStEn@FhFyy4Cty`Tk9Z1%;$ld}IE z)HmOmXRc*L&7#ouUUG&>HR6*<()QMfcGcew<;2W>cqH}Brx630UYJ$GQoXVH2yCDW zeW0?uV4Rn27bo*<=ANzwC9t{?YVWFF-`x&Qcvclhcmb!ETOLJCjMXok$yI(#e5UsB zMfP~3lkR#+o1t8$QjEkVWP!Q8hv}_jnK@HG)SDhrO{jtfsjo;%X-UjW{(!IFGkRul z`zA-_0^jEy^GYFne}EQWYEa1dvvcr2h?Q&1 zK`q`A?y0jZC`hJ3n|uACPpCbV_X7^);+-tP`U>tzGZ|_HKU6B){L2^G+`C|mA{Z6_ zm}y^h!Naj(IcaJLAQYj#ngrO*BO@z*5wlR2ojWA&Ua%OWMS1GCyQpl1gG&Cpc&LdK zxD6)a^lP$iO8AJt=_{oMfGkgJ9d3ou`K0xr(XHf?!#kUmDc4NryP-4bwmTJ`ZQOrg zFrWdFbZL{i0#ZdqXBF+mGkm%&*N1S`b>)hlGvf+^14{~Zi+b%wV&lTBmx$*THp45} zr)<&{GsPfotIWu{cv_+vv?uKR-xyU6w7Ut{{;F3AgFHGwulUXSxE>`C=jMlhT?JP3 zYq|bTcg)>Fn=EEYBwYwOWNK6aC-x&np)x8oz*viQs5tJ~&%JZZg6(B(I`hlviTBPl zf&a+dzOmTB+eBIlg_i$EgTqlD(XUy#s|0wxs~nLsiAvZViH)Cl`5ye~)cGBduA4L`xK?#cgHSN9yd8a8F^y9IBM+ugikUJ88ylEv#_EsA zkw#iC0?QmtPDA)1rkAzy!F7q*iAm^U{hhX7suU9mW;Hrje%DX2fc+%iFYCWp6%S4N zDup%AWMnQdOhR}C8Vhsx4P1tSK7fX}^r=$7Bq(gms+c##aRjV`Lhs^TgGwHf z(F$(UYRlr^qPeEpz39|qjEns1`zkuRieIf5pe5PzWKE{k_rb5GdydQ_om^LMz{fNX zsMwZVwz_VYou+8)@WyIC6aZONcTnO}YPvyvr&dUmNJncw*?)dpiRBj$3)J)ZV$h{+ zp-3RmFS}|*&<2m8D#L{|QFZzxzf{wW|g2CJ%}mv=gWz@5p{Q;c=Om6MtTv7pR{ z!963Ao7I}oc(g_gm*I{oD{Bk{D7B_&Js*%Ha6V7B?b#r3Li@E+BcBb>V&mkg-IT*e zA2f}7z@`d49M^Ia+~Ac~IE3h?v+lh8wqUo`j~l6zX|d&K#{7|HF8}}BV9P=|C&BPs z4i>vO%LNY|YQg-7R8V7Gt!`DQ;&Gp{x?6og_F!L&zqI;|TwhESA1SLe9G?TKmg>)2 z<8Zi?LwcfUqpbOz1btJ;o=W=CL_;6}7e)|K)yb=d&S!6oxBao2>WYD?MMt0zB7Gf% z&$(h&Oi^A?j&i>XO~3HUSs4aT&_v8RNk3FzV36P`k*KwFAg)FtVjQWvKeoc z0Fr+h3?u2J!}gUfT7sepC;E8cS>K&c*eNOS08LR#2OJOm>FBw0qR7Df&yf&LsEKb8 z_^{JvfQa6)(}yBzbT6J@a^lqfmW9B&p5I=--*6d@rk6PAn zU}RTA?&8qMkB1hY=q@;*`uU=;*!;s?ilbDY1N;k590GSY-E385I}HoXd_?a{!df0; z)FQ09z*<$js*bh{G6;g>3%#PG-A#Bx6f$}cZG!)d-YsH?GPO3x&$icSpM>QaM#F%8 z_-D;uHv;Uxs52FO-ecr0OIkhOme@(h%5I`3W-+2irAO+n^np>=G0Lxi$-^cw`XT6O zN!q~)746U&tNp-PD)5YkSg5(m2R45Ava>Zk4`&ZxX9I5wj(oEwxgtxWf+F6_sd7o9 z?}whLM}G9BG`u4IQP%}wQ_@Aee+nD@a_@dArh{ksdnIdxgKAvbT(>LiG^7!!x|MyX~BN#|C$f-k%+b;-NCpk~&+lJ8e|b@sCfw4LX+RjLMb@@NY&(ta zo?tjuP7Ln6<*&^z?XNW16q4`ko(vtx#NcsccpWu;llbC936E6sxf~g{ltpWOR(-zy z#8M^CA^Kz=QUX*pZ_i;+&HTk`-ZaGF+|_dZ8boH(pn#tqJ%;NAK@}#e<*aRZ?*pF6 zA}^-_sjlfw!wD3?pzol0$zrhWj@&@ydTDA+`fz}S9NG)0&YZM{rjKpr2eE)N33ite zgH#m?%R*b&l2jGe{I^Zc9)PbwQgZ7pAInrWDyH!n#el)O5#G4>h`GA89hr~=k>>N> ztnBx|&6={fBkP#}q0ai3W3r)}w#K=RM#_!j%TnZe8EiQ7x`kD2qIffGo0P+v*|#FD z)q7SqITLI6P5}8Ic*QFg%C$g|J4&AO6vfSw=hg9$b4PbSpsl6Cj$f7V?C9Tw(=xtE zLJtoTpF+RfT2meNnA`j^Dgyo|4CQPv%hN>lH9D!#y!dE4-}Tpdk>tfN-&G6k1_e6B zfWUs~pVd!WtvA(kcL0}JrBk$nc68;jNI7^tfgXcEmZhVYCuc0Y+bx-7>e5MW^yjfSJ)e=Hc<5lzOf-tYm@_$XP}5G4ehA30T{3T{q3aj{Y_eI+A^ph758xYbCFt2Oek z=PooMc;*VlLglNC)hK)nfVtz)6hEon4vuR}7M$Y7)XDbeGvIQ$O!`t4h=|?P5=8%7 z2V^rlsazIUJQ*vI<@Vdqfz#8q2)kFw7QDyPEK__ElqqH5k1+0UAIRu9UFkz)q?>U$(#%8=s8+e(lS68TS4>m|)C))^{`}f{FVv z>0#zumPqCh8-a(KwlYZ$et1b<2Ncg3ia@9eAW{8EZnK(W@shZI{=kfGw4#RplM8ne!-F3|RM4t+{bU{HhqS0^Oof zfb#`9I&r=>(1>1Ec*rTTC}Fju?^k|7Il_w*6CW4oHV-k*yBG*Mi7RNx$~fXgw7b_1 z>6c6yaS?LGM(B5e@9Mi-eMwasb4AMUKADIRli$7MPeB)Ip?@(~1qg2gd6SyQSdJcU z2H0_TB&n=LyQfl=pHDU`NaV$`t0Q>#lN`q!qM2~sxG_ySeyO$Sj;*&Kz;47d<~?gB zipmPT;_yblQY%ap4xC;K4ac|s2Sx*Fi#)6_O4@~($~j%3&Ke59RC?;dBqd(z-Ve0@ zl&*)vdk=Z&#JWNcvHuW;@iS)%%)B`gL;Zj-^sUh6r` zU>Ysmd-9jT&bANQUXTSQ=-i&BfGkh28cHu?mb;zFS^`O}T6O#b>4xv_aTUZpB=!@N zgN;TJ4ZUTrh##dt?8<3fvYJTZ_~jIOMxUpm;%~3l$x;hAhEo%$)D1K*?X!EU2a#gv zI}^Ioj4;lkZaP&oh`$HPH^v}smQpS$C%Wsd-`Fprao;PFmcEP?$ybT&W;dvrW#!9f zc21h_adHbJz~X~&spdcw@Oih*Nrpre_Q ztLb_~uZSuU(s#W5Pso6-uQbD$kEPTt$Tt|=#JJ!;lMuyWzgTt`9`%02MpYqt(m3TppWPff;8*R=OG|J-?fdJAL7pTsdDH-u9Ex?8>RJ1I6)+R z%Z~W20oUJs`G}=pyy4F~qx=94g1jZH$dyd!VV!2j11pDS2E|a~|v;IfBOY#N7%y;OQd>lN3l3 z*u$-8H7)4yp1M^|0f!$>+orCz=C3ummw!6oi^V%_sldq^=AqtN3?EgwOv^4)B;8O; z3c>3oU7K}nyQ>G%;ZZho5SH>HD`=ppN2%J|Q{WPwub53q@C4h&UZkF(>%pycB-Pc} zg}6}$2c&9;cm}&Cpi;=K!4Hq>6p)OuySx>6CKyI=lUh{=2bR$ST;J16>m-XtmOyTh zF9MX3oB%B?JklgeO6Mj2ot#6&&LB~Hm04+e60+P8_nx*4i}drNS);~jMqGgi-&0p} zI)iAazQ*p<R>77i&D-!R$ z{qkrxDvibj-3<+xgJ`M0~QPH&gs!`GRqr0QVo&NxufzWSB>6VM^C* z>iG4RjJ+ke9O@nE1$@2pmObcHnRmNbuJ8=~q5;kdam!NB7>0Gm+kdvYe!BWDJce-8I=paxq>+D%~1ijQ9T=_Sznj!8a zG2n7p=rdc&v$jfAU}|7gJ+Iu%eX-qTJEWDiXqhl`P!);&^T@&Vl=D#%=X(wGG)05> zFy^5&|LA7ix(_{3QcKgR0Dz<=S0?eMGwad4?xIZD(5?vU7N_VeJ#6mJz&wJELz<{-Usde8UqfFIm7yt(I`f#LKR>q2j~X_gG1 zaYg`pxh-A67^;}*Y+!e+3FiaQ`^4qrUBg>}$f@Z?o(AtcCQ!ZlUFbD!<_mj;`m^%ba0;|VH=Kmp-+QxQosztW^ zWo`1&8)M9902m+=7$nPZ&Bs1*yNQDM?$~N@+0TTZdysCl?(9U;u(or;flm~^9%J4_ z8ePj$dF8Z+WJA5UFk6BO2$<7QtxUJMm(pFqi$O7Yc3mkOK{;*tA*1DZ^D%|-=Z8Se z6NG`i^4wKdtXx-w*fycmpL2>@oNeZeqY)d742#%1t6O(fl;ABsUQbrzh{aRRcj@pE zQm{7DDLbSvzGyT$YyYu^#EKB}bljW`S+wSI7@NGme;ia-uBuO{gtL@_X`+LF={GWF#P+FgO%U+YpDMXc7V}zxTMl;ea;X2@_VHKw1iq9Ye zScy}%kL?MX#B+SNf!JLVZoW*e=un>P;zTV%1^ZD&qx@4k+pi0L;%ny_J9ZyX)RhTz zk>k~wnaok*{mS?0v#<#DBGvj+skdZ|vJ(=~Xy+uM$Qf{av8TU%hNi5yEa>nqmpj`k zUz2dKACzR*dQvl{zY7p0Wjm^QsbWhP6xPuS78qAXb<9@J#94|I5Z2HSVgW^H1dbsC zBQmRjXB1^|aq4Urg~-|-!TvU~LfQ45rX4#~oVtJQJSC1<-XtE@$OOj-t5x=I{M`v) zSi-);pshjmG8ojRdzzbp7j0aj6)|4PK4(Djv*N7-4~}gw=;Ywe%*ae^RUwTL&i74A z0zd1CJEi!Q&`Xf~D8v@-P~}A=vLmnt|K#*w?9u;=j&mn?d)CU+g>fhPjX<{L7%!$` z6D}H$3p#biUl|Qs{i$Z4xrz$S2V)8SGnhp^3ISC8BT^3Yj@|%PBHoXlIIkvt__K9D z@kx)|mY3Mc@6qYg6rkRwTo?ApF~$Y}6uY>k=E(rJ7{nVg3{~_`0PAEH9q%K2+YB70 z_O7nNU7i#2QOgNKY9HEtXLsFHsWv1a%n)4S;uBu~gfXCT9XSG~G^p$C_3WZ8!7mQ} zF$O~KvDOLXzV?KI>KB1f8j}zi5ip7}-IFrlG4J+^o!gfT7E`W!I=ZLl5~aCSD0au!Q|FWq3iA$vY(E|78uQ z@bY7!Qgy$m3iTA^G=}Cu`x$`GQ8($FnxYQdm|3&J5E3eV{q=L?S7-ZN`GtDAb0QS{ zZDp9X&Z#0#ir^G>9TL;Lx9f`_C1L4c^f#$@KwP7BPi7(zk9(9Crc0821ZcW9p_fbY zN(j>|!r8;_sfhWtDN34haF*w%#G$-#EmtDKuv2%s4ak)sa69!7 zyES8`+g|15cbPXeg)dZWA3ooN{~{eG6w$*AcL}(gn>iv?HCV35m{ijm@=1ZNiRL5J zP(ZA*wS^&i@UJ~Af$_<6D+@8mVjjTjBnyNYb6?DyNW~m~64nZMSek}aU?-!AOAwoJ zZ+TdQ6LTdob0|`@$Ep~KtM;2-U_Rfm`nKZ?#2?bGbHCXEqJcX1H!a4S533wV*gy~S z6UVO5&yyj4JupYZEwF3Zy`fOUqdlATpaa=tf1jz%=y&Rx*6&}INPAzW-*P{ z$>V>{nT#}^?yPcVvOxjIE-%kER$OJCpNtg2^Th51?yNNI@8A!Uc)Y#@H%Q~1GK~A5 zW=F)#ug1f8_b#z~Tmoolo{kg5+J4F`J01QH0Rbw}VYyj^k$96XKNVpGwDgbNjfRu1 z3BDditu^QA|HSe|OjvjZ{i-J$O-msJ^2qhWfZx5aY)d%|)CYcc-o0+qwcfw?AXK7~ zY>5a2;ge&x-*}3sA*YV5lkC+8Zl5gCpmlY_*&#Ja8uM8hG@A-Ee(=TKf7WMV*w(dy zqjk0>13>HNTikTj@oIZ`!x&P+&nmy*T|89k*;_IaBO+|^pxMgg3-Ae)9Xm6`3=K=! zBp`zeITi6%FkS=0(9wb^b0ERVXtYw^+5pF1M4mKG8(YS?FI7)ENe7US zR#*-B=n8&-%2GY`{W>$_gx{+23WD_4p!_8tn%uALP6!dGD*`OI>z6-N<*XJP*eXez zBIcwR(rrhTuuqljP`)tHq@O-T&7w+N`V}7THB)IR=VdM+vcV#3@FT@-BfyV~$5a2w zb)+|zX0##&oz0&}lqsIdKj5Nm*zbtTbTVc(8+qPIAe_&o@>0QdyH%Cq1-aPn{(1wJ zp0Kq`o$aByGOhsrlGumIc2QaLrf4!}TNR>ZSc4}5wEc&6hig>HUXoTF!PYoo_8LR( z-XdoI;Ta^PoJO@u`#5ux4)D^#$o^H*m?vd73 z)?JJe#>KGqJ9KZE!r>e2=;Onf^cs<8bcEKl9!-jwiGmd!FxH`|ICvbvFuo@C5@Oa{ z@Jw1&IL{4H;+Wu;XG$x~@wpE!=XJxVB~f=l?4UCLKHl+dXHc{MQQJuSmQ7>we zRvnv7S**T<7=f%9Fk!p_ffuS(-Gzo&;s}^v|My&!38MBnDL#2+jM#~twf6Dbt2gOE z#shq((4#?!SvofBzdI<9imPIeS~0oQq-a!z*gC6mv|;bB{Tf=J=L;?3FMz!5tADrFd+W4c*>>g^{K z_*?=P$u9{=m)m?b9Sjk|tyb41bs@>{1unrOezqtA*tn^XSk%}}w#f^xAwpAnK|T7X z*G)i!$QR6@1Vv0RgbD#0b($ zH=JK+HfxYM(hh#V=Me^^QEzb5>ZX^0nnDTd;)B6O1TMS)WJl2T@C9!32osFYm#*(} z`Y^_^V2RRY!kl6p$ar&g;TeM2rdg1att(8dl=e{w=ztO2rBo>#pr1HU+JTvoF~%H8 zr7WtO2QR3%rTMzrRtC>p7Oc_RH8_sUk z!b)nBRv}W54CGS8K%GyYR)daPtW^dd_&ZX=49+@ZYi!~G5e_YQV~GfSeS;mv~jOH z*wm-TSD|J-ofkKKH2NVS!&x*{(CuyQLdOVoosk^{5TPm`!X{G0QCgk7Py9?l0m|>@ zvTf59QLwc+b)g1lQD~>&G>O*m43${=rpFmJd$S4Vg&=f;v``-oLe4kF4XXP<1Mhz; z?F)gR!Y310O0-18^h4x+ZF~utDWAsCzZc_y6+WsnOSe^lOVT~7Q4en1=?F&Cfm~sc z-G~q+3F**t zpbFD;R?eD&Hw$w1S@8!3%#L0x#2al5b)w{OBg#t*{p;6flFSEO2=b_8{qCmF(>2># zM7~D@a*A*>fFjtF zmLd3Wf%nH04L6bQK@M6i!EGPfn?s#OM~-4Ua=!>wI1U?b6p~{RNWRZ-Y;v%a1l=Oy z4WllbY#~T|gN1;H@I(M>!#7eDy_Db~A{<>|td~X6d}-8i#R*Q)c{Fk6`Yu9zK3|)cx&uwBIA+)nk#A`)$@rE3P#ATOEfrS=Oi`J<%C_Pt}9D4OCNbX~Ue! zpUM=#QDBYE)k$lY9Qm_N3t4p3@L6ka;1<0QwHN#?a0k@&%ts|7xOk_UmoX{RB>huJ{@1c zBY~h%o&E2(h9De0L@MFHc><)i9l2yXXB9hTtT~Dv(deebdC$p(7InoO&G6o00HM}t z*(PW_I&+vs0C{Q)%1L%t`rR669&DkJ3P0#t#vxXxDj~TXM+VN}E%Zirfj?E#6M-Mo zjzDs*ONlTtnZrb}k4=)7aX9}DyMil5l%1%+y&NW%%+%Hjt+^9_($DQ8zbbRVPR1pR z(X76xRORl>LFZHn@63ZL81mh&1bdx2IwiY)BUc9Z<3wHv+S}`r>?DKN%`_+kh)-BGKH2$dZe?CsiAH8u3OIXGmT`(@gKP`YbRoM@R zc4bBuYX;&%rCy|AWO@Kgv^_7>N<*eJE6fX*3oxY7IV0Avw&MZ?qTwh7W%g+pC{yB0 zX%h1tL;5hgw>kD1rs89DFC75t*w=k91anC8ITZ3(2<^W@g9b63<;56#r?uk^ZW%LCl=33wX`+JF5e+)>UfSb<(~foLf* z((do&E4mpIAm1R=S_0FYG%jUR3SH5Kx(n@M1wi)BUsQczYLz`AMONA=7T>0S>L$Yf z>EV4_f;w1WLO~>Ly>2wG6Jw=wOU+v`Ox{?_pvzF%XPr-Dzy@fsTh7m-$o!|2$aKSFF3!2)@yZjEikx?m=ccEh>xC0u z;QW4nNLTpYDyyJ*s>CdNs_Zr>e{B31aymQ)RtV52t37GE<)@YXf(4U&Srq@29kL>O z#t;_W6cN>%;VNgLIX;Dn9w^h!iZ8!9x?TGHz!j$&mw-SE8ZCIM)}od_Q4oS+x*RGf zLRCbDTt%)R&x|FnJV@R6nto|67aFNLs)^LLXN{7V=WM!p;|CeA`5($S8~L8>tykXu z<9)0u8-SA1AjJGk$fc6cMk~h)cMcgJ@lVyZ+H1AOBTM0rr>;#J=wsd7dVaUOIqT zRn(S_xo2mY1hGGdlEzE|84E!M?-m)hz0~e>WlRY2su|B`FF|=sLo$CZU?uR=8lm*u zu01t=a!MU4I7`i#xV(Mtw&H%(&tX^P&~d^aCG1iBQR+)km@}@w?0-v`220M@r2c?x z0yI|RRx_tv&(CHY7vcZe8V^*JT{TOcUamnC^Tdsa?ce5&9yu*Qy$Y)unm@OP0d;U}HqF(r#~(=YnIoUQ{}>XM31?`HW@T381Pln5bx10WC$%v| z5Y8bIw+%xES9E{I!iWs|7%`6-R1uG?pk~=D;9`lfHrC6n0Hb}6{jw;=w=^YtJOF~W zU+aq;croup1N6HhmqsUfH5jEI))IE;9bqgK-b&C6OrHCNm&0(vK;FUU$N;+0ejs6W;E zAZ$S41r*@Y%l0x#gl(8|(KsDi#dJ8BLd*@Q4W>Hn8^mMZi?(*o-47j zi=hs&N53aSp+K(JuMRm=tbSDAdCm{g(!A-Vv$fMwwEdlE4-zHauV}X zAZuw=aQp=@2zDI#AFA-N`~yLZwZU{E*ROAzq|adg$xdN!@PVJhM84V)YH^>SE#AP; z2dWYM1E=W}A$~i!Mw-8*xH7rS%Py+TamJS9_y5vora(7iwOvAP{f9kV`s>V+L|^iI zm{L(Gc;wb91-BoW{vHM_+TqH0y0wymK_tVDI=^pg?-v3mjghiB;>a9<;PTWWbS z)0K3Prm$*Cp_$;>QAm?rCy4P#J*nI|0%;5Vj0b(uCPCm4FZrW+B){A~`6mqVjsQ}; zngo(LP|mndbUc6ZUnCCgK(cQEMy|M#Y z+(({d(omG!+DzW~;I!+kmc^YBU`{s)v$6uI&h!P&A9mka@mW`lgpWzhOh$ zj+I+PcL>IuYD4D7B&-Rd%21{Hp-n@^XjiFHA*!-5X}~=pcImQ3$q9)n7J{WF9NGv2 z*mdY*>rr+TZitgKh&$O~E>)l|A40Mm$b3qMNAtTb5xZWpZ)J#0%KbkB6$_9Ta9+oB zw>*4MAJ-b*@$?coU%HgVmY|>v!g*dcxvH57s9D%5{rmD9F&qf=VMgM!Tj`ze553@^ zpA43ReRe$ja#6j^+ug7!OpJRS3X;l@msZ5DdHUNK3tew_h^nrO6UPlRYQaB@P1cHs zsO~gY`&c7gN33n7agx#j)v_bMKpnHC$Mnq-`T03Mb(WB}vA(K-lqt4{E8ZQpXr)3SB7CE2RWKz3_43D zkTAD^vm6QKU*@9OYt&M-xkkP&ivT#c%=fBr34%q|8p0ADLZ4;jf(4 zW@?84AYpf4^v0MkhbGKpb0nF-0<%M{9x3?lKe_( zcMr}VnmNKEUafEY`lT2#z5u+YfLy->6)drE9}8JKBqQ$ch9@?5Ut+kRQ#%~pO zUuBco0gd8@^Qq15n*gfpDV;b zo{HrSAn-@tcls;hn-!tNkIYJ6yq?It;p?}fnrZ$+hvU5_Q737TY`}dO{xLE+aNFwo zC&NCQ;hRVEsIcJQif~1!P3%)!BrA@CSdLF7Vw*ox09jDeiAIj^kS)K=Y7LM#0*%g` z@7rMcsHelLS&cF|njC9u(Sp6UHL|n~ z^b~?A4-pKzLWsc-QB!X0*HVz15tJu_B4^sQHWw&l(sb}+rJ)b#_6dWVsa(X5IyCdJ zm*97JJEDT~^TMNFB}E#~X6!AQpzXzYPSz^DmJHZtU>JV7!3MF@h{6kI6d_BXxwlRb zV&|%2fQ#ml1XDl3ps0&_D_y$^ss3e{*<=6ormx6X#$(nXFMY&)=US4d! zIGRKiSwtn&^lKZmxKU&7=VZxD!H*&;Nq`&gVs#0zqNtJ=-|qZMN3tEJf;pS{_xE+lhp9-_PlgKGXN~Lg80fV7l9$7&Ej&@i+9_Sf#0`^9~4@5dZ z6oM?P8{ma|n$KOq@xlEhA4;^yzHF{Cp_g9YQ(BkqQA#Y_V<+gL#UzjK1In6teT-nr zgqTKh$`q6Igweuli@4`PuINllA5WyCpWF6$PZ) z$+Z(JwB$V$u0}%(P(Q@*A#=UtYrQqvdHg|0mOQ@)pc-3Pl#zEdLR{V+<@y}11)392 zlNOTo`yP^0bSo7S#@5ohq&fyDlRLgpxZt;j^jH-fVZ;EcQw-hEZ)g@v_8rjy$fJ~` z4v`q$(&FN;L)ZHExQ(VpYgXxrl{tqQ$rdD`ioOQxnh_^=FLrP>T?VaHU@+vaA!AMW zI((EDGtUHLb_=7%-%iSRf(RgF%$rZ%u`ArL5PxZx_WoJx?%#AF79DVB6cac4w(#u_ zu{|kHX`@xjn7f<8w%7593Zcr`SG8?ha5udGsR@SDVn3BTdU)4<%e5DyFU24Cz~K|O zd&u&)J&SS}Y?=|NEC`MeWk_V!=jQ@(0RfO@ncOJZm4skUF&hQ&aNqAmMhhHmNHDp^ z>^Yg;Y|UGyXV|KSIEze(h(eG3x(bj}QtnSa!xF5B^~J#IW+daA5q2viL>zV?N{Y0z z2a#Dg<&*7(y+qYwpQtT4N4@`23T5vFV zT`}jr2(u3$Pz|8KG`U66;ARfA1-w<2i#|QrxyL4(m4lR16YXGaMg&(4RyZ3}n&>s< zXIc&Jr3gVX&>qGfghLC|C~ni~-Q%sF3KaSF`63B-e4Z8pe(qZ-AusFD%ONnH5q39c=(N)k~WK+L+`|`41~T8rppH-A}4F3D&mKfv^2M6zbRx$<$l-!aR^Vd z9*JP?gqw~Er7{tEyA3aw+P#)I!sE;_A(zMiz?8rUH9c5&45>pXb^7T&(;^)wiGm4R zR=X*L>nwfW9JW$a%YL8}-tb`Sj3RXEGa^*B%Lvk-6+*(2*A`?lKY?d`R0h+45QKkX zQ?;YmZX=|{$l}`mVRY;;6wvOS@Sl4B>;V$7$(^LX2?1iJXzPb6Z|%pNQ?5|0!i_A@ z5t<`|Ls(Y$rP11j2QfHGCY;01GMWDZqJn=bOID70*2%wSPXvk-|HgQ$rKVK$S&jB~ zCp4N$ON;IGQP%M4L=hA0t9L4&pslhmeXQlKXfASGu^bN!|D48Y3!YHkAyZn(N1Rz_ zB;pTK_8Bf>ZCU?UZ=jreyw$G*P$$po)yXq4Ca29W94{rileZfe`fhJp9`0@vq;sAcnJ+ss{NffrNHf0n-7yCXS+GX*CrOc6(FOmvh14j7y8Fl+ev^-+m zf^0U=+THWe{SX=@U679w659H%X2-BuV})I$74%`?d@cUEC8Oqs?qF7c z3Um#}CP{6a7z^f(P~JHrI}ZwQC92LaTUX$wx`V3$R!oHjA<1!wIA4z*1GsvzoEO9Ng6$wF7CH(+OH z*_=CP2P9=6b~V!i;9uR`Mgt2`2OXWurqR8RgEr z#HJ5F6=eRh85kx%KCj0^$CMR9GrUT4!t=6sK)Vgy=QxQKx&5#hLMGo>Lo+(*BFGx6 zNu&lX#}fYYjg}uw=Vya$bjD}sIN^0<`(|9Kq`jcAo<8JM1%pMr%O6DRYX2OxL>x})kBcZXfp0JKg=nl zB}f;p@bzQz`j~O6W0i%HCBS8u&7R~eLeu~sK;XYq8%#;poxovk?K+Pmr67&0?P-(c zwH#auhfTD|v-^<_OyL;us~4Lzt+}s(a#Li0BkBhbOw1WUK6U7gEF_i6#`ne5+-*_$xpok0Y8Tgi}#1 zt8f5Urdm?|Jj~T=-%k(jsm|)S+%t?twVu(%jNT3?`C1B+28+_67REqAlJP>%cZYww zxRp73Z0;813Po3-DbD`(jq2s?Saz2%ydgaAAo&ZU`}A2!i-9}K+upDt9(z7?NHzs2 z!4R$sMBA-=@$wlj7Wt=v$NbOyqMWox8aSg<^=@ZuJfpl~k~-r1P7Fs|ilse0cr_w* z?4UT*XWpUQhZlr0eV=(X(^5^am~%1;x>C}YPAMv;OK_@z z;O{rxs~Kk%-VSmjCJka1Fo3;igjnM?3Y=tW3A&-bIzz|$%=A>gxUCmWI`rTS+Xx`P z8t@zoQ4hq>+*i1vxH8y|;y4q)G#G1Mucy7O#FwEr<=B5eM5H2-)zx>`w~wgNSQHow4p0#}!_&t+HnCcIm)Of8zU;ICbg zxU3vG@=)p^yDXUH?sV3|y}xnGmI-WovCr$to9lwE5HRnjTQ2bBSd7(u)I03)impnS zWfl8=HffIWj`B%VtQxk52H55*tdH)~Sblvt(>yl1+QiY^P{m)~wk382M%te?CY{sD zVR%t=!ORy6@jwTD3o*`D9pEa@INmo|F6L{tHs8U+d5xZ!(C+^V!{m|=KsDv)I*^L5 zNG@vMxJU;kR1=YpCBkvC!x#@s*mXHo1gMprHS33|wJ<_FrBhW7Gnrx4NJ~;8B z^XNJ}Q$!lApn0L7fx68H&;E;ZQ$Adn@q3hxfj$`4?83mOBEY3JVtbfy`SXC?$eQP& z#74DP#`K5CWUrBC^LjZgeWE&LY;IoCqJQutOIVkS;>Y_D(BTw=@Qcu?Lx}HGX_HW1 zQ@*68*P022bJ$9cMuUb!Y4FbP>yf#PVM27!W234l*L+NrYENe;Foi`7ydAsCD8$5l z?Q;V$j_J;@AF@xj#J4nD22EobQ8LrPvQ+Hl_}{vOJPL<(sSrN2O`HBuEtAYKF)Zb6 zZ?!$wMeibrCb!K*Oe(c@6_ljV)EKx1>4_Lq z0c{C{v2N8V<8Cd9-Eu)ZSs zH>n0$myP<9B|Y@9>EG|u(h!*y`^rJvFt_R1Kb~ zCkR4vCt|ES@(ewe0%20Wy3ah%<$s_J*J!b+_E0rN*vgJ4_?>9EKI(-pUl@wcgk4Pk z&*wkyr0^Wdq%p_1F@gxKK^#tB5uCGX#$LX5<#pFX>sYYzSn@4A)`#H}SYJBh?<$$1`$qr~m6Z!W)exHcs zQD4Bc>eWw_ohOhc4HQ2jI^Z%bBV&AKBNXm>GsdA5O~*OGCl9 z)`3=~tIzq@j8AR0d&}UFt_$}<_&l2xQl3*IXm5aA4?z~u8<`}loPI=BXm&c|km5~| z(pk_YkmHwn**kQ12@5(H*#)xwASgQEdpc~Qi%-F9?a6P%mBoiQ z%bN8c>#lefV)Q=9)rjLIC@!vcI|wT=r4n?yoJP~4pF5~iVZOLNP!drj*T%hmb}1PY z>5C6c-}WS@*f#Ocf3Zy)8t&jeG-_r!6H`cOS1f>zX6&o3v*?N5H-oX@z@NnsrbvsB zC7-oJW<3#6t@D$&AelrzBJ7aI=TTizBX)k>UX7*rMqs8ARmfM*!n@~Ma7JwVI8$sW z(&%kfmOJaXqHZQFWGbqSJr}+Qqrh6X!|0#Kg>2RIE|3cHk7j^n;}zCmh28uk>1^{SQVB(^8TKo@tFtiMZEhy%*7u+B6h&MY_Ig~a4jl8Ps$Alw^P+k98)i&L@IjQ z3fjSKOuvkG?vjEa%@;ZJ-9KSYKsh#c>Gn#{I@n$F#sG5Pg*h&vC4F{CM**!xnoujf zp2p*VXhXodG7_$xGia5iwy%l5X4ts1aZmzdD9k4l(=l* zqJ$gkVRpj)D$dWqa3o&hRbvty&%bWn!WVF4`DbQqj%~Vu!&D7cZzdZL}$hO!ndC1D1lZd~k^0UtLxT;YO4a{^ihp%J+`lD2(}_JDFzaTYEb4iIIFVPk;AMlxyY*Kn|lsk@t2Dm_c|>Cn2X2TXOd#6ShZuE)jjVU1p+Y%^L&;sjPw9`-F`S&&hR5S|{v(ebI4*Oii9e zOhTxx!Tinr2%qHYT`iLWg<3GIKmdYdI(Ted-G1&U533&NIlsi%hETPdlxd^(060kN zODa5Sqm%Kdx4+R9uLbAJ>&x8XvGmj_uhg$M6UiSG(IP48|dj) zX3#ImCF0p#@)wo<&wCJ>&d+UKD%8~K#EP}pt=V5EJM$BytOy`4fT6!!?A`9(3Ss#V zC=(X|E^D*)GY^t2W>CW~tgx-@vu zFaIGMmvc6`0806Ja>*@M()DQ!FuTF{P3(w=K9h(H3SLccog?pr1EHlWIa2v;u%5cG z3kQHGj7=m4nNpnFktn_LO%oXIk!}6+auI3U_6B^>CSHWy-1?M-Wg;E!nK*v#Z_}Lr zd1J`Q57gE&J)Y_CDa$u0uPs%;615=iHyQVZPDdQn%?a^nda(O{OhD5RGPlq>)v+h% zW{lvcHP>=Bi%Rh`~8#O0HQo?zfTbv*6c=-L7?ls8r-z_i{@30jskK)IVzL`m=r3(;ZN3BupalP`K6wF$oY} zO+?hSpVQ9ndX=S#-S%Hr8Z}KJatgSq7*=%y4j@t%o`vE(I6+>cajtObpno~f2k|JH zr!U$*#kon3k2OPTb*4Z6)Dz)9bA&_>bxdxf?DKoi3G#V>+>lAZ}(mgC^>=2g7w ze^*cYaG5clC|BtzO#WfVFlh9~u=4RRF#)DI+=Bci|9684fp#WP+rs7;mo3qycQ3Z} zvvl}oo>|4&9Yr^rza&+){N1y3Do$^Do!)nNd;UT*I;794C`df+yjw{Qm zb~2#o9s(mIHZ>^U!Vi+^uM88z;KC>(kt?!{g>zLgmZ4%ag}1;bwHs<=xQ>Y$rAL+) zk!#o8;DkVm!I`nF9}Qk4&%8}(|Bx%Wn!8rcyoynr-&uuuLyB_sR?e6CBgeZ3f;E}i}w}0ySbujX<(4{*#w?JXyejVc=pO`z{lfIjL#5mO8+B@jL)+= z{L?YWD72oYM8a>$TLtf>#U`aARl}Q86MRP#7g#UENq601_-MUC)1Fv;?}9k38gjv%f8nVqk>`N%uA0dvkQ!xF zPAr{7%3vIL?RTO$E7M9IcAEN|WBrAQ0Uv#6az+_uK?Rl#7a~AmtOzn#PPGd-UBbKM ztcg9$>^m3HP&jDHX=?kOxp_{S>gqLV^Om?#+!Tc3YDy)0NG zQKiX2EJR(ViHjG$ovhBtEv24Ay+~S@Xf3J#(b={)4gn5{+>;khQrj1cL~*jQ-uesQ zRu{TKA!aNlAQ=4G!G(#s;u{i;u&Ie$p1mGo_g-5YCv2j+=3(d`-Bp6Z;kya+5UdnQ_3M;%GFG2ny$C2-A#SalF zWB_#p=(2Nsn)6R@uiiWvP%s1ivL7AA6Mw~loaeoPCKF)X6AZXb;to8&Q<)M{KrBTg z^<>H%jeB}EF$4ezQz?aSZzmc}0T1hTR_~GG%sZ*}lO*<83pzz&Mq`-+DYap;1|8I;u%kHjTqAAr7&X`dE25JVy&$ z;t~7*MI+|;SGyj=)@A3N-P$PY($KxmH4$MWj}-KG@b&vuc9t?U4Fty>MsGt{?TwDe4+F~61e&B4}>7}i)?q@K86 zCd+09uKR`K^gdhvy8{)!i*m0}x?Y!%#qBY?yD7JKPIYaSqDgec=%5=Ge-~JPsUJoM z8ji=?4z7Xijj;5_e16%QCW{REaks*DguExep~?iC=%_KBAG>5f{vq5WT4hqrjc(~O za!lxoO@tq(`?JlEa1L6xbvk+mJmreAb5#OIf9*!~hFCK6ko$8bE)9&7kW=EWu30D@OG^)W6m3nL5WTK?}+N)-lXM8MdL@|LO_M0eP4T#gH57B^h^@Yz$j#5nm8h}b}quA1*Z443ODTN4z z;vn4LK@LxU%XX@tLG=quOwU(5dcKB>sq^j=4f@rhp^AZvEqE zFBcN8wcw4!aP%|E-SHkZvN&(b2X`yw6U5M8E{wau&w^XiEX=gJcO2zoYS@s*!;*1< zp?JzLKD%SG#biJcsmate<=ADL_F{I-gjoLkWhws^P? zk!U1iMJ0_mx(Pka?CYngXSLR1iT1n31^DUY7;#Zw4&o3+=5+J*{a98US;6@Mc^LyO zCti9ZYRXCOt}bVSyo$GjLIzCOz`5OKj<>;j6s&%Bq3}%YxMwyZGZdtD;CQ=fL`c3U z)i1DIQLh}~;S&(&14ofFRoCtaK%LgAl*$*)eG-VgXd&0V(HG9 z-Ka?4N*A7FS@3^#L2E@BO+nza#7RJd|-+BG!>*qBRt3?9m6n3o{ zIy`X6V)68q+AP&{qfc1#%UVuj3cFCryHzUq8TH0L3^|3S>hAZLGC+^Xcb>dKvyjM@ zt7-4`(|8LQ`)D0XP$jMwi*R>XD-!DG-^0Fp7fX9%7S*F+5+=jIYj7xQi{6=FUwoe8 zZ6m3pa?JR8s~_Z+GGh87|WaDXEezfFQ-#2^uT88` zhP_6EJyyzzhA9n`fBj}(Qmr;al`O99+Cou;=T)Qt^*xX5@-(K9k+Z8Y#LZl;8H-V0 zC-Tqs{<<56>YAJ{*vI!g^Fv%z?)&jC3~W~I;|mTZi7NNh%bzw~WqlB)BAp;->SxEo zAZw+-;LttY-o(Pbte3X%)*9nISdHMQ2d?VI+yWbv`Q{+bz9)Yw8gxi*LRIPFY;a82 zy8u`XhiXIEd-3(T#@F2z!e0yZ?O=<`UoZS3M5$g3z06!xWgWDw=;*mv(k#seqs5>@joCs@s0*QY!7{pz_&z?!`it8$F=)PbI`**{Dq$jJWXEccF{<+T zS!V{@;ePPbeV86)4%lz98|k2F1?^NkO7^ARMoHW&cZJJR zsPB>=(OSgk>;-gaGK07hgjHYC$8+(m-ueie7M^6R8dzMGxHbV%Nl!!jkg&bNKqY1G z&Y*jPBX3mQR54H^%E<*YXQB8M0n|_a0hv%QzxN;h>Iu?i(xVUO7LW6$;pvK|F@k-b z@%6*~#6M%`-FB=4F<;MTxdJ8Rilj;)TO-7A%G^(ly`&HD&!hv6be&<0Tx{?JN7OzE zv*_5I`=d}c@$K^=x2+Vnr=lB(iwIU%2HYh^rOg(jSxI~VEnkM4Tv{VOX0 zdPB+)YWT||d}jF2uxl*pFB*)@(%WSt0|Qjs5Su(-aiA1}XA$1p!@BLJnnekfKrZ!N zpR=!1Q#83q9z4t54c5(Vf{Z$NlAz|Bo1(A#g9htlg1Egb4T-P%>9*0AwLTg3t7@ez z-nhN^XRfB$941{F5mUN{NM$H}+S+STe1S|1%uTe~2uGv6gaRu?B`t*K5@x{Q6Y9r$&ZR9p+jvRDJA_fGF}+l+Mfj>9^BmAKIy)BC^XJ7e4-&l@`k#MhS&M>oT^-p>G zRAiAI^Lhs0yH@;cl*E7Q8X6{iEMCKYno2`q4@tjtjnMTKH5NB6Jz2_s_PjtF1DNn? zwVvKJ5=PS96zlnOj9T3?9upG)A!Ovzdi(I`)^E@>G)vE>?6->#LID?HHcoCQ?~lg4 zq%uE?e?T|Iy#p|KkT@gyQ;B8-X7hJ&f}2(6X~p!JrTQAW5CtMuw$o9YDY|~fsx&5l znUfF&()z!PxhhbL{h&Lw0+BH*jmG@TvL9$UXywawH;t2Zb2|?&{F$F&*&g?K_XA3j z($#cZZjH)G`w9bX+&|?0D2JbU)J5}j)6H?~!`gx;8qfTqmAyN_muxYMH=eKO#L& zkSpZ4Phj%ws_Tc2A^{$~OMI`_X`Y>+l17-L`o5lvMWpJiVnf( z95`e#j^^h^BLVL6Wh5;}(|1~_%r#7GxRd$WOUua$&vV~`9faALx{J>yCs3w@3x@vJ zf9A4KWy2Rq>umoW3;RZurX@N}jSy=L_0{Qv%3!AhOjb`jjYh|Rn@=GODiE`!2vuA8 zepXoYiZ!ph&Iy>m!f<5~?23o>g(1(4;BPjkbJrJX|3Vte4PM(l=lQ^)iuj5;+o2tC zhX;)tYi{HP_ac!ajbzu6M7PD<#gI%y<`{~D~xPb6-l(wX!s*RFvi2X~lJHO}UgUg1T6A`Lc4O#5O_4;uNC@>k)DmD$zHqL8JJGq@>`Ji^zq zKOeby03W`Ry}M|KC-PY@tomGY_7MLuKUu_(Fy0!K;pEXkxj{^GHq+DZd<^*ql&%F|Qh zdV|pP8?0C>AXw=c*8V&3ebfKeC9N$7@HKBqkp-sp?strl2Q`dbJbNMIB(aU#Q0GPNQqzy}f;%q`)}8nCJ_L z&K|76;1eOO{j5XNrWK;lO{7zdYIDw?X=%*glfEMun}}Js&gf(;?Z%q$V!0H2#j_$Q zJM(b_jk?*7>nWl7Pq)Jk?)U#10bt<#>EzEpGEZteYdv%_GD?j5=AmAFklk}6CYe?p z-OD&+RGf10h>QTa1KLyti^-7A%FL-FkQWnOn;&o5PXL|eXbnw!uY72-z^~P7+$!{8 zM>$s6X?s2k|By>e;UaxDI31uwTS6XIKh!!!lx(;n$ln%zPB}0`(i}|4t%~U-Bwrp1 z(8Sshg7~i={Nz9x`Cmt(?L1YdGJ<1UyBvL`q+E zRPxR@Uhf|n2ut{^4e^y)va>`9&^@P1xS5TlckKTuAJ0(8ov6)b#BxXJ$S}p>@6y)eLGMPq{SakJCG5D!B5+N#Q?yw>I zckQAM`(?<+9IqkYgQgXtZlqh3=tzR#Gpo^Shhy*Gg5+m;#NmN=!OGrHPJZTpAi z4HU`uaP$W=si3ah@v!>Fq5)Sa)R7#{$C=h-7#t!a`G+Q!zFyzusFe`@CJUvHoo|Zi z{H4q*E1vZqJSCO~0^aU&#^qP{X019^bxC6ZI5cm+J8hh(aillIUm<-KYN>_$4gF$rV6sLhj}VUu7X;w$VlG zMVek(?+}*T)igDZU(c-GO%Cw~>kzt|q`+KYgo1(gZVVa?ApWUMR)OmKPgg z9Jx?$izYf&++P~M;3uQ0fjw~&Q;0w1DE;X@ya^!09f5;p^cVtOSVP9)yD1NjrNR2i zl{^pRyry?Q=)%=Mn68E4hz<@Szkv0_9>Zh@W%Ud zcvFF2<6G54E*n2T;QCNpY!WRv4v`?0e3D|Y;E7-DXD1qP8`Z7Nju|K%aKd`kUEtYA zhSja{>ClIhKPCn9Jm>YqhL<(w(}=Uuc*6qicfFbLYv*Y~@V95c-!%IgPXv)E^d7Zs0k zQyr7B6cLUI`uCGbd79|pvkS(ODln*;gH?B>&}OhIGv{@D<|dpp3vxlO7KZfuOSdbR=R1NcyL_89w5D7aYPs?b)ah{dAAuULEh)0>kdrX(<|8+U7}z0R zMGnrzqs3N??O^DeE0KG2ktvU}un9PA-wIlUl;xi9oE9 zCBw)^cf{(x3JN;$Dq_p9cnHwL!W*fUThl19Q88_$#ZwgFn@!B?49I+~uJ@AJ`zeM4 z2VH!Eepm@tc;(3w5F6EXA(HX*VE^XzJifElK>GOKs9>9WHmc^orHXC)YMafavvZt; zWCedq72Ss+4Yt*optX^5|4l%F_L{%olWK4i&7H6Z<0z2yn97jx+4<862!Fv{mlLPh zb{aDOZlG_}>j^joo8@gI>^hg2Czh@PjwX^9UoWMeE)J$KK;zXMq&1O2_pg~Jy&3hW z>=)1y?1!Cxp2Q;PZ4WU03CGAGl`%e0IQ>4I_}R9fLoT)NH|HY_r0A~*=J2I~wKlv! z0f$)0s9^6MGG@K?Q4{-K;VpbY@n8*Ift?wBg6_|vd@sv@0lOc_jbIH~idY6TDnLCG zI@m+wi`1T?YVKaMvc9!Cbwab(15+RrC_QZJFcgFIao)e0uUUC6)^9GMlZ7SB?t+SI#CzAX#S7BJDgrAaa9k_iS2|glA70w#YH4 zMG@0-{_@Rzno+lVZbt!8=|cMcw2qrQ?-qSDmJmM)J|XD&^ul1=DqyFN!(=IM4_xo) zV?lShC_f8gEr^TZuGi>j8%jRN=7_cafTw~>O{MgT*>?4FX|%sdVaJext}$hs-|)Pe z#5up^{lzmL@!AJG=ogQ4%k{7s>-Y0e8(?u$1XX|P z&G~w%9S&(jL_f*Y;({p!3sv&D8V1xQga^05Ov5pgc7-S`RD?b#RJ=T4dD#ul$??zM z&LzAa$$p@_;pe9ZK+4qEZGz3t{F*(Jt7aoT;cLaLDE~s{eIG4QtqSRJ;(bgLdgOUu zwPLaXjbvY5_EUZji3?{n>GB`{54(?7`*k&~E=o|s_&t|1ieaEoFO$wj5(hMc33Kwj z!-OI9__d?W#B;2tI}c4f{yWQ*e{ccgmKo!lPWU|VeeVXAG4hxp4Te+LCq>zWoKQ~1 zTUi#dJu&=BnMJ;0K|$?_^Nrfgi&;apADCHeT0g(<5}-3k*+b9|Pz3T-`PoXnoxS*D=#k#>1E7cNgXWbMeE0ba?8b82-9=@Ule~Mk~qFzQaTS zS0;Q86IjG;#u}gkplRsrL2e70M9}7V+Lkd`_r_Gj9fbGKx~;H_*J)f}#+|voZYx0{ zzb_Xrm8_4x0Luz(+9#2(?$c8<(Nrmg>eCpI5mJiAR>E#$be-}7M8ht5h!9dAoHynY z`FTf5%|iBOqtsA!FwV4vZScRC0g#_1hF*lSw-h^R>wn$8m|=6K7M4&HTXQXu)CT8R zem+0N)Q&nVH>eC^eiy$2!_gF0)K3UMW5b-Huf4W?GF|Vwii5>j zFY&ZQIh59|fda>VyO#7fvP0uX;YF-M0(sE!w3T?nFl0mY8vf`E=llzdi+Zm6VLi~j zQDtt*^hwc-{CjTg9ZA`q6iVMp%pWVyf2Y^QE8;OLh`M9G%`Susz7b{#`=Fk zh?_k&!rp$pvxlK(<X2))*5NPF9WDqZ^nE-a( zc_CaE6~fh@P|t8)qHpSeYDl(=!O)QZI|LX)d2=GZ1ymM5cd$u#hieYMsI2E`+hFT% z`NQ_x9$8g4388oLi{;6?6Gql&AA>YI==;U5N)n=*pE^9wf4G2cDca|o5188-N2q|z zCX+>_N+k4O;aiwc{E~gvo69@2VWIHOpekf|#4Vb%wniwfhoRal(gl0;SJ0J861yU1 z18zJQUbx8~MF;w@&7}l58X0GH&|Xpyn6KRVAjCwSR$$3iO@!&=Du8OQbFAk&8oN^d z5+^Ddl(xKp{{;VsDr1IfA`%Q?`@P=+Xj#)n4ceA#)HtRP=!L5q9OI{li^l7{f)nkX z1PL#I+tO)|>j`H&$N*Z;>37~*#c-9ruaamCdToAh_=svd*eZ9}FIt5%Y&-G;^SQvn zQY=7cD8-5KT^0I_{0}Ut^|%NMhr;^ybna?ncT7$S-XZ2eJ37JDZ^V(DH&gYWEq)2s z=(2v4-J*5a1uR5ytki>HPsj$CL4|F$?i}LMhM2%30s$80qfP7H)V_jBb^O27B8f&N*LZGjM3zSf-d$PwuH z3ws7YvRn9yW#cA5rkp-X{0GX6Q+A|TGxY4V>YP7mgJrxPwifP!m0jO?54pr2xVg%8>8!5# zwm9KkzX&;R6($Kw8ZX6t6(qfSQ@aai880!_*R_V6@B?MkaS2O3XMxCtoJ$kRoAc&f zz(xC2^ZOD^*u(pkjai*E-ku#436394DJvl3!tuo zrJvASH5zcCn@1%uvpl+V^06qVZxn^In#L1@-goq_s};vynd;00<~Tr+kC>bltM!79 zGKU@HLCxr*@oh*;ZF;c^u|Hcvl2r~nxvg3Eo2dZR(^|nZ@?Y**%EAkXPiYr@*KdYh z$|r3G(=$mbfin3?^D+_c(}TMx5_In+Ym!tG3L9;}6n8s4H)prVU1ma?Cq>`^yvrh6 zYpVu2UbkQONCFk=4;)Xi`S8U?T181tYv+sc7pm&85IElK@u8*H)yS|qg*imlDjBA4 zvT{=bCwie1?t?WSzo}0#_`hdc4T6%GM!InTml@k6FB`t`K{-q{8l5k86V6!=xk zVwnc!g*?uPOzqdpn$Gaw7Dcw|i+=0pf_U>{!>*s%tq0)Nuy* z8;74H0`** zO}$B&PSZd=XXb-HAyCP^o+Q~rM`C7k>OS7dR+Tbz>9eImzC1+$QxfV?CXUOJHi4lO zrRjb{@8fCaQ2T`uBlJJuiyj8`tJo!(?U*f#xfU>Ds9(pmPWRW5>T6$xG57015d+8$ zL^=Sg)(`=O9alEWc{6$aWKtu4;GIPEzvt!&G0_d_q)*{77{iBc}gRZxZQvnhV0EruD8Ysg|m?Y$UwM*_XB{^|uHEUh0^3{SM@TES+bPfu7>GMeL8 zsW7UAxOV$$`hI#jTPyXSFuTsBqe|*42=PktVpsr;k*{hJIyab`f3+f@VSyg1J=8?t z+!&I_tJW1A(3vZWm*5>HyUymnLm^)B=r-1~^zW(kH~6G+R{L^a;cg<44w0f8<>ogi zYse!(Bp`g9^o674yz<-{ahWsRA`{SBDgy=ycHgkdj8La!qWh;Xt_j5Q|3XwVt`j;H zbPMO&hyfb?6<|?WgUImD^`bJ^ZcZexq%An1ik$TYwrlm_(W6V4V$h?uE8Vv3$UMn~ zHjk2v;Y`8CmBMCkv!>F|1}`K`v$&Ciuypp6!0vtb+O`kbiAkeLv7Sl>d9XO2%jOa} zd<=983do(voZ2MVvA2@aEn72Z(Zsp8%+Tuv3dvi|5w+KLW`2jOG86(zYOcK&klpw@ zR4U|Pf?yX_$m2_%IY<&0ITf`Fq(Oq$@{idGZ`tiLJ<>_;xLV`!vsYJ4 zS3P*JJx#!HrrOR|ztRIX&z4p`bL51B13YM(XQ;~yWbCHZw*f-z;enJ5`e^cKoTO~i zT#SH!^VCkvpG9O2or)kHA&?&)>I#r59r9by93O4hDiWv$aAJimxyyrpn`7rDH8vTp zd!*l)C50iziD8&MZp@TGv)QPq92?w+qQ16 zM|#n-%kUA0wQj5{Q2^IhI|pMo+(^XCd%@HEvVe74f2yp3EH_qd4r_>fHO~UuwIr#a zV8#F{Q9uBtap1f^WeM$~eez%)=(-8zRXLvdlHW%HXzcfTJ z9_4hS|CF;xLK5_n)29prYCV;7i&HkgRkyK57aH3im1wG@d=M_+fYNo%kmIX6%`}YJ zd8%y2+4`Utq+_B<@e%4OCB(&(*^%tf4&o=8S?5NFl-2K;i$*!$1rglvu!@5zQeZyA zG!1WOTa*8{>{IUE?6tu4l0(a~|3Xdf-msh9HFeJ@@y0u(!hCD!N_7Qd@U7q%s1kco zNzG&w{29!}cXJ}&|AFmVGwUd;@q}9HeChQn`(nl#H(WssBXq5^Rb^*Ug8o_y?;?XI zjMAMyvuW%XhZ1IBeFRkcV7q0h$K1JS<%aVI8nsk*V&JR1lewo$vNd_!vtwv0ULWOU zxiA>2_fR>+Q@VRw2tBDHsQ}XQ#k}`@8LN^7#5f#~0^ixA%&}qCM9Z)*51t0G7N=<{ zRYFhjxbajZoN++M(pwp+15bzCxZGA}2EftIjwFrf#eH6yHrk!(F(T~@eXbzUogWdd zf>4!2fMjb-nyz1K@Y2tj&P-wRnw7Vx>&W(zT+|-nOcLeI?0snbWy*MVj32k+k zji%!*dZ%0L9atuBem#>$moA#8Esp3WG{yG;hqn3%X(#_5MmZsSxMM8O-w~K9_}om% z+VEWditXNJ{+a>ko|Ky^D3}heze48p`&M|1iDWU^ATCs zmK89CV5C==0?X7H(Bs>2+L_7!REWkw%3%Bzdk9N_z)576`rU`?NW3QI1~W$|RHSvM z+ne`|JL4Z4C!KUl_5MSq@!zROdEIt2Ot!Z&(l}|_1q)WlMM#Dk`wCOFXDF_ zC(%Hh}sZxSuWE^DR1B?7~MsnV~^@ z(%*W&8O>YGgxDzmzSZ4y5OX>{bB4Rx#w#IfUPZ~#4dA0V89yImNm9qy&MAcOK0)_+ zWQrF5?((K5ii=w|ZS?JXA+4MQ=JN<^X0<>?epbt5^= znvTW!o??QmKzRxM#h&`5K&NgU=D?G`T!BT3B?j4(WZyC9hn0VMxog*rODM78KO^rK zAGyKfILUE5(Ga1;z-{70jcjEQb--@bpOnFQcG0A86xhEgJi3@_Nzv${k^^yt|CmJU zH2VkMhcXUZB-ls=>rCxA?jSv|3u1yO^;PJaxU zL7RxzqXLzOl9HIa2T9e9n6^e47|4|o^I?8tIwhr>Zzv$?XiPD=f1qWAxgCfKRG5KV zgo?tdQ(zwMhwSJpGbyE`O#Ud0)!n+7QvySAZ@R?Z zN(gXC*ZvgaLeA{z8Y9~mHF$2~;PmK~H??_ui6s?z#iwD$9xY;|u9+aVYZCSmQ~_uR ztwH)#V^yU1j!{7!S?|1qqAg@!qmr;QgCznp<30k6>KRoarC)=7#hJLq^2Bz?6Kymc}Yaz zHqI`f)E)wG88Id#@zwz=P6K}PZ7mh73}(s4BR<@o>|?!arHJ8&2+{S{+aH7D@p-Qr zjp^5gfNRUbi`g#X+8*d`8S71kU(Pp4tEF8@UNCI|^rJ!OoZGj`qx1)|TEEy-k3dXF zNm|I-8F?Rl57&5lL&DY#p=uVsZ0zdstUV5IrirfVQXGj!xz*yUoMFbPWwDs&ow@<*$cC^`h_b7CL_w+oA z%zCwcTm9$5a9Lv#AA^FF2%#L85|BN+twx9HN+=AJpw0JY&5rmUbo4**prcUSsUKYT zw-&SO3I@p<X0&;H zdR&oLwtzk<&{+HefuaB=K-s^zH5gyQU|rC9hFjOfl}J3sj(Fjv@7n+vOTjjF#|7=+ z93JfRXlAfY$Fuj z`&we7aTV>XMQP)|p6D-;tm)7>hTNq0?Xz*<_HvgaiHBQYQ~EV3ir`G?k_~NWGoFI} zE)3JzW1?8us!PYgBc!zQim)i_eT%TUNKbh4q`G?YWJ+eVj?%IFKDvu{KeMROHjFB= zdw-r>^W);-X+|T*6o9A~5pLDq#B9mB;mlRo=ZF)A+oAAEVtr(a?Fc*AaEwFP)xj&o z5M-kl?Oqwq4XfAahtx_Ble(N{3x6 zptp)&X6_vC5-K$>`mY>uQC@3eVH>koOGjd-%U{67RO}r4^<)fUTUoTq+KkO z7#hx|$(dHa#z|(`LgXj;26b40SYyY?OC`0gwzq@U!O`GT?Y5EXf9g&TqAGs1SF%XK zIVQ3Ids>q%i!)Hsfz02}=VVUa=X03V>=V>Esb#k@F?b3$XbMk(k%^n;rk=n+25w{G zo;b9fleRg3;MvO)7bUk8aqGX7jpQqP9eENRO_F-#Ye2ezNFzVmVy1nd<%5L=Q(aZZ z-KV*msRo)X0zdp2T>vSNPzfD;SFONspPF(qXPYvL>uu!Ce5Go3UzY|MjzlnyjlM5Q z2|khh^c7c^J5T2gyNo8yR(Mx2=x~2~r0)!Ywf-c96MH4BVvGoBTlIjS{H^L}nvRg; zG%Yq9aqdPSozKdsKD#pL3wGn($VLC;b-6tk?zX>y%A!*y38D~}%K)hjI4F$jBv0dp zPgZ)s!beMQ2-9r^s0-P}+%(Fmh!YDs{gwlvxN1JdXHBSj&RDUc_s(x5_KJWy^P6n| zs;=@?N_^4gp_Z}4mE(F?LlDahXvlxU#Y#?BiVQ6@on*{{eB7;vq+kJCnW$h$r8qEd z;Rr%^tcw}*SO`b#@zQ_(i*iag3gB#$xoG?QmV23yTzz8MIYW z9k&y!!g(K#bGiEnAu$kyMi@hTI*4sv5%%La?wwhug9Hv)d4HH_bzxog55;;($>Pac zRGN;18EJe?6AeqVGS|xgS)4ZsZX)30S_l}vcFhanzR&kF1LJLq)B$)lviv;tb0c1b z(6wL0KYC(>`QPN#f%p?c%n&i%=Jumq?#G!UN2AEb(>LxXw3?8qvV0(D*=$D7B}HG$ zzDPV*qH^MM=SYAMd~mr(504YI;aHFrKbkme@!F~N7Dv)$PR6$e&`jofx`x<(XR9aW z#jTMr4r0r~hW$dXXYR|G1Z>{smvQ1R63wfZB=aQb_>o?9qbmb0rX9ks`TU!bV90B0 zox=)C^Ia4l!;lvS3^aF06GYW^?pO=(Yf+O^*rb1~L%MLC?;W_KUA#OaOrBl5O1F}c zy|c1Xm(nYi<1?Vz7kNa0*%i2|Ebm%BikBhxg}wW{jb&^gqhO~jzjCBjjEf{OA7C`p z31S2R19m zRq%B-!nLU-ezYsb0XpPU2Qb=Jd#ac8_S9TW@SWQ@fV{#SMbNL*8{9?Ky$z%gml7ht zMZP9H$^mS2g8cVnd;n*Bl#U<|x!{EpRls$O>TL!azQak<)rwoDgiNq@?r4lMeNpsh z@U>^AEwkx)_~?$I4Hy={v?6w{;(C((9$Zk@E4nUo{$~gtNE|M@wSzJ@t&xeBgSaWa zVU$LQqzVN(2L0@o+)zd3raMdHBxcCyk}n9m9{gHL`1gJ`QZNoY6lsp?5(8X3%Zm_p zV2?xxMHs1i^_C7DjSVno2erz^<>bM4ZFNjQvlZ7mOWvB@zWu57HSWj`FSIoc)8a!2qBjMzSlFS$y zB|_jRV8%jNRMi$&)V;i7E=AvSo0d$ib3(w!hY<7g4=6CynS^(w?pRrb#P?shg?bKc zR4}VBU@UbD2ZjJp+xfSJl0agf2K`k>0~ zkUo! za%meyC=Eb?genX`8#W^ap>MH9^SNqdjCZj?9u4+R#;d$h>K!M)t;3%+0C(t#w23r# zjdb}~6N)2m8*}z0zHpRh22%M%r#euqXa(s@B~QThMa z^@w;K*ZT@m!>C_l41jjD*$mxf);6?af-Lts)NNv^OFb8H&|&^ z{15+@R4PW0G$ifk+HFO?zO=UO5;w7rQC_a9ikAAR`mdwlPN-UEd>j1TQ^BMWlf%ya zh}oBTZtrBkzMC|jzwxAxPsg`UT$VV>2Z7a^Vqcr?RD6?90>kr*sK(jP1RoJQzM~`K zj~G`S{oa(M`m3yq&8;PC2!iN#tVVZzf%bENEp521n1tj?|I#k7^vEgQg6C@1v7JZ> zHkQnLFAMczk7M6S?%m>{zx!nb!Su&-Icz_wu%ncLg(Id+4U1uo@})sslZ6w!0}S^O z8zcfCXG|F7{L+Cnt0{@ZyE-#$7wvqx9g3WuVhRJ8skmt1FpNbQF4>+(ar&*C0XjD% z;U@sUQvt+cnjjHy4W>!?!`y?4w&l9N2rh^rWK4E`#*eg3_hA@xWwI7qZV#4gjpei6 z;?!kPGH(2a#=Qb3EZrh=5V{^S6R9t%YOEffN30g$Obs%I51)&yIGs>w@;3+id;O zw`8+Hl|%EycA~4U(lj6o{ppe~gO(Ju_mK9nl<-vdIp?-~?Cvo-2>LW^u=7NR!D8ZDHdrgbP+1dJ<&% zK|~MvujhxY5fsK($uPhI`wV(@!oYGjbNM#~L^@#)*oRYwA)wk>z4~l7YN&SW0c6u9 z9@swT>QjP2B}1HoqSwR3Y%f;FsU%QJ<`>mdMb^htQQhBkg$Rm&+P3~r2Cmt;!X^nl zB`On~dro_VVo{$9&Pw~`e>wMCs`xR>?b#usi#v2{5kD@m0M+gAz#)6Q$5qZ_(0wVi z^-@ooj65n(LcJ$+rI5x6l7k^Syu{ynSKSE90(7o zH_Zz)`FTsLSB&in-Sv@LE~+fyN6|?is}b5~>(59OpCwzCDfD{>{#fnZx?X=#IyRfGD-fV#--jBBqfDKE!W3{hlFSX#oCN_@r1Y#t%qUk-!kcVgvyTaWOpV)A>4 zC1euF5r#5Z3+7@e143_8i975ZWSy{?DR?5D$)T~`oHaHisg}I+s$N&u^krEW{m>q>X#MO90lfDf@ zi3=QA$}&~aMSAMac5nfJ)L%FS+92vC!%HkU1cqFIZxIi3(VNibj!*dJ06;0(xfP;0 z(Sn3Z4u2N#U~(Zye}TIkP|n$f=b=5jVzG0;qzEm>%OG}d&}l>8k<88yl@-Bjj!JfB z;y?>-<*e%a2Zmzx^hD~nZsF}`vhB*ZT5iMSdLXfO0f@^`aJG$EPQac;D)q@ zzE)#-h#j`v?uDD3nItGv2!cj>*)y#Z$FP%R_AKQB?1@OL*$}1O0*sk)OBsVm6J+kn_J9+5fP14&DV+sNj zUxHPWtC>pC_~Jm$$!zI+YB4k4oe9AkTI{Od+%UZgg{IJjd&a^>f@7Ca3nr+*$5d$g z&S>oIm&7^llN8XE7&DKtR#(RmQ@rs%Ifp8l3?OkNZAuK2X#*hV7vy&0Gq#Pw0>cnN zt;2hrik8ASs~WBFPdhRa?&SY>Q2$7_8ncB&N9;>*Zx7=%(A@3nNO~gueyy!|aH7+A z|JlX-&G-KVOsn#2AUQ-FK#mr!OIYi51@QdD>Vn~4HUGK6(4S=YKbrtk3wwQGJ8q_V z>6yOtFOSkK!^?&zy+ME2^|b|DkIGBZVDKL^-W8J#j7ujJF8WIwE;H_eeu^-4B*Olx zVzQWwcG|64Oj@NPh8V=_tjFAbf>aQ{m@Yl3s7-T`Aq-_mCJw@o9WM%EYf9$vcPrCe z^}}iAJ`aqmf>hF-HT64zP&8{rukyf{+l`|?(+4?BkeU=b`$hN83b$2UP!3Mw$Xjp< zJJ3VyVE){_^tyM^oh{o;qX79iFeZa^=tOV7v?Y45aRF$Q6CfH0!V0Zmg8?z zE@whnAknD~t~#&mDp$}?5Q7NRIu;mgg!B7m-k%{Xtb3^W~~2LhZ|vo{h#*b8`$@f zbs(?i6daiNo6{R?_vTGTg^}iI9Qovj(5V@I;216@O>6ipdL?>N(^@EoDYw!s9j`~b zY#$aXjTk%P-FPJ;T4~-%T6VsLzmxI@sdPf@xnG|>rdDEG8=(hhG`7-&-hb-j zeCE8S^&GYS;HhfD__jV-z~2<^v!-U#t5_WJeZwsCy}8NI2#uvyVr;jCD=2~TRQVS9 zjS2HOcb>I01s8ULWzSG9m{N$`z>R{tRdDWgviE{-iKtTPXTl?xn-9&+dsLDf`v6K( zPM@ALP?tuf-2f>49p76_dw)M`Xt9nGV#@BK)_zyjFHb90p^($+niURU^1?|n6XoWN z_&X$!t1FIaYKnc&mPEqI-r{ry684*1Hq2EfSb%JYmzA9a+nU zCNrgAT?5yt{(I9vqh<%jqEE?RP&mS^zwc5VzB|__Gzb9Xy2bqTenvVeB%?pGz@QgH zSt5WLiQxNsIOJnz&8wdKewslES{p`D6bCZhf`oxO+lQm1emqPo=DKQ#noY zNwimygH(Jrn%m7d>l%t|B@oY}WG}2%n0A;ScVOFO^I;L_#oLomOa}zCHJa(f&Htde z{!kC!2u}!Cl0ad%Dtdx?h3#3_KZp+LP1GAPs(K;e7^;RGvk44u+eF+3L%6jf8>s?M zp;ekqpVSa{#_FZDRq56Uh|+JNp3T2~2eE29+mZS=XHYP8vA)LR??Vvg9_PmLW?Rp* ze!OJ!hwrgYjQlR>+G$+k6h&MY*au8%{gMZ~CR*I6@2}`v{<~fDf@nm6h&g5vCljuj zJ3C95$bgPW*6aNzcR**)u?N};o%~?I_}lOY-_2#ns^~bmKw6~*e8z0< zeY9yf6he~%sVJo<*{aB*UoWQ9##(vL6gEr(a<1iLD7^06UIPjqT+ z{OEr~)a&f_ZrQ*^cE+_CE3j6ympFzefzy`5rY89JSZ;f7+Tg`b*JFr?d z=r@W0y|)5d4UbF7g$_guB@OgDX)re&gpAa0bjHG;;;@<~JGqGS<#9KCgm<8`A6par znNF|_8(+^)VM|u3SZbrnBfLgT+2wnkzyWT-aKp`el`F{6XZ^;K$jztL*`!;3Hf2%f zc@<&+oXETC_d34D=?>uZq!y_uAlk#Gtb6&3-i9_@+{XDv%$3eXTgqu`aDssH??b?& z$PnK|f^=)D=QGtlfeW7#H3Sgg0hJhaa5fI#7|I#)fokY9E!xLIb1F`Pid95|?@Dz_Qb)L#Qbc7_L_Z6kF+)(kx0 z%aC5HS`Aplp83C)u;Ds=R~#sTVK&3zUXm(n>K6vX=8{FFM<)4Dq<4f`M^;TnGnq&I z{i94rSj3OilESu$taKMIakh?|V1-{NZ5p73J6x$5j<{@}3Pn(c4 zGOTLOq<80;ZR{6YVcHyRC}<6L2KA9d0?Xs|2m1tmZvOXn8JWw0et|NXei6S!`_VG3 zc^|1nQOUJM2nD)du3~gS(n{E0S{7~pLYw;r^0k=pJf3G?uGCp)PvB1vuk*FpBiR8n zy-^=U?@S3wOt;6U?`2~?>Cm}oX+nH_@D0f+@=^GxM&2t3TR+e6`nduT+=p*L_6zO+ zak6xa-C6*pHVgJXFg9Fp}}7+i%XylY=OO=^>qu+52m z0_mL7>jalsj6Df#pwFQ9ky`5kCP`Oz&eC)ABdX3sKfr;(P!Fsc#8C6D@>J+(tn|S) zhc<4U@pzNaAe*2xlj&6eu7wl(2VnRACfUkjhsGkLYdE=X)F~{DL8S!-B?M@_Y@5;q zPn%?Z^oOP?XU~rxeQ^mPB`WY0v9bdUdVdEbsdqm47tN72Sce1=)00|gOzh=~kA>6i zF#<^!5cJlOt7X1sPtM$PXd~L`6BbioOcCU(Z1V^zMz~oGgAk5?y4HVOm9;FiN5%xP zY^)9Q-%wF3C~xGyiWeKTYI-k;`SoLh>{W1cm9v0Su7fJZ+U8st996791wkO|8U{`k z?)SfWy=6Q1krOVfQj^9%y8cIM(fM!RdF@fHW)l-$QAXDDo*@#CiVCe%2hb;n_waW7EqR_RN+#&Wy@%}RD zFmW?QtD{f%QF!5!`a}7G2v(~aW3@YQK6%FT7FmAMKR6`-9z%!djSpUL55#Sr3s_cd zYq?!@11a32Idfbq*N7B|h@whcUtZ|HhVT|pugS8%yc|JQO}yt1!N{R&&PZ#VN(Mw) zi%H~)VD8`qM&dJfZe%HY#vvAWk|{5?J1@l!nk8`na1e*yes)@9R3ZorKQ1c{=q}Pu zRdY*}yvCg1m|kd}4w#DzKHHzST*v}aH3af012xu#-ZlWXD zTzab*27#l7gIV~~G1zDLbu@jJov_mO9fiodr$@1wr)-ojf0YtXT&g0_*NaP+K#2_o zquYKaT{T;Fu7d7v7JO648zxIGElRhSD+e%OhYq|-1ro~ z7w|=^4~G2ozAl70JWA_Zb{MoXk+CbXSS9}B!gy4#p*88{eOIaC%*IPv?Obd81$?!v z!13|&lK}Psvtvx_KC6f)f;gJ3KV!&y*r^f4X9d!@KfspOp<}bsrzzYl!LVkf3WeGr zg6a3|irEB?K(MY=YLl05C1OT58cSskLdHWr#ZI~|#@#a~#oKk%1hU(U~{eK#sYYnYaxLPq}|Z_d1nf|3O<$UNuO)uSc~lJKD53UQ9? z_-?> zuBd$DQ6D+0$iWiA7DDX=3r@8(JU*ew!J2GkU^0$8AgOU3hY$P`M^=nrSC$GdgX8yo zART;?2_xdhOoeZi`C(LhLvNVFfe|pcN@;F}D!;er|I~VxoW9Dgjv(7fYtOAjJJ~8~q-W;g~Yl zyr3A3(w{3`6}WqOyW*pg|0f5(NL?iKF#E}QD#IOR7+3U|j;_z@BK6c0n&K32z1R31RuZd< zfU>cOGju{jbJTyP&RD_a26shP9ro`4P~St~LH?z+|AN)zY*uxxt%$VC&Kvi~<IOO`no=DXfX&K^TBT)-xTr}$#{`f&SoRW7)&DB|@1*gT-IcOjWJxfzo~4_gxjzt2uoTX6Vo?e| zMU-yB|G^;G?u13K8)r}Y*r3`4Ene-nB`IjG$261xH;6pMg}2Kvxe9a5Mgx4JE@xuU zTYKq{N;ehfJbEOVym#X32x-Y&|AWWqiT+`9^TBqzUALu4wCAN_nsH%gv*IYA*SZA3 z1xh!JT_JGI^;>2qAV{CBfOtI$DWwk>^P(ZLl3s0zx2&TYX#~WMCn$1`HhA8S7S@Q_ zVGuqi9Uocpu2&HRo3~QN3$N0Bhr>aLi1FAKB+Q(tM4{}eBE~!OhOvtL{G9GMUkneQ z>GH*nxwbLDvl$PGAD}4JX-K4rS7vLOD|kK{(;|(8nX;r19?ftRF?Qo7xLcI4-8(r1 zRO4UXFa+>3IGvo>u5TwuldGQccmwqx)ldO-x^13Nf=hpj6jX|jNnj@udo3fci`rzOiD3C z2FaD?ht@7pzN9mnDG0^Cg)g+k9SCc~T%P*qfXec*b^w7KW5pPWnk;JD;)LNtSkt7M zsRTlKEi%+KE9b{EB^wM2{;}diNA4QXGo~{`YE#cE=R+3(L48K-;+HqAM8c1w#>(Y5 zwsbwiAPT;C$;YgDZ#_wT^XYN3xWMQ2b1`Bv*pAAzF+OD-X#}#!SzonvNFfHqKTK1G z|LM-g9QL)X$+R<3X6JD?M2}5yDWH+ZdS7#?nxg8h$^(al0nQdxJ_{o{FJ!=L0Ao02 zyAP_uFdtjj%5^07e5fSL1|C3d%mew|&rF7qAY^V@=%D(Q0d-|+_Eq(i{B)>^4QZ9x6UaD9#i16Q+YEs9DZ+|v0iaKtQ`l$HjDHz z^Jeal6;grL>vCu|$psa`t#L^JNBv2nrXGrB@~~sU+oqkbvHWUw*M;$v75L4?B7jzh z&+&|Z`}N_E7EYe++!e_P!Y8|tv;mjF!>>i3+YDIbsGLo0PsMA93=)>|Uus0Cc{}Dt za(#)`PJ!@=Z^BtO6`hr%EbjgB_#%hv^z4G;C#TpsA<@C_a={a%ZUFxx*ukSdV%e<|*!*)1xdOsNQ*CIq0uE$;*) z7`eO-Z%f}L>6eF(ap_-u5*c7ZgUU<&!op7td#j0uoFhsExm`V%?!JiJLi=6g;pA8K z(-|LU+$An#RhF?2*{wD*ThI*;YJKAgW)y)-N$+d7qR~cli{|zDwYR`c0G6$Bu7imm z=DXsFUn|8FzY?)WDek%5uix%{@PQ+%X+jW|cL!LCZQFYDuD=WLky;MWcTPZ}iCocn zOa6G1E}lQamZ4+HSEZ{} zTGyVcw#NQG#fVwdd%ic0S7#f4acqAm{#QB4L>*@Hm2xq~*I$}>>kiaXgm`SavOa?R z@e{(xZD~JFr|fVeQJ-T!60x$W{&+p0fXTDKsB>cpStxRCmXxs0K{-jiR$~Jn4k1Wt z!+{AB_cDXRUBH_Jv{zZTZtv_x*98FU+al{QZpv{Rm~7)3vR{xxoK^KNB1WYO|m86szX=chFHuL(ZBcZCv~!3|mau$PT|u#ZP7h2#h>)9MPi`@<*@42FvVCf-cjWoxnxk_4eWY4-Q&W|^4|6{{XKl@I zWICm0EY`OB!oLgZS2C|hop~))jrNcf zD62pycZ~zLIUG1A>j>7$$T|0E*2?u*#&)xdXJ`l9i-m-AzX}|ImwkcsFv)$d)A$V8 znc757KDJPtd@t!t@sLTc3WL8jll#CP1oG`xW+`S^3wtU3@(JtoZX2_h*V>+>`9D|< zB%Cnb?-~8mZL+2l)Cby*_^q?^+9CVU-k$lg4f+@iu1h7xjoW2=;_p!l$@-#G5G;aj zo>&T*jAKkCs<2QFjtcJDQ6RQ!p9$`6Kl>4i6faE2AK2ohY4x$i@fd4j$5j-__BW#{ zfkemNyRg|*3>X=P7QYz?bt~<;GYCv1rf677*?l}dB)Hd#?uqtsB1lnAB8sMX9>QoJ z4#gVF_kECdEj+%0A^H$B%&A)%*t0}r{<;4h9Y@-L1 zBJRKIsK;zig(s+B4I%8n{A)KxhwLh<6G^-W@2V zyhS!b$;RqLoZmOt(owE=qUtdMdr)P>8i$Vpo|=`*%*oI?~3q|Nbav z7&W~0oHw)Kyr2kk0P<@mmRYVO%EtSOK>#UiDIx`esg<4%Fqm8JXQpN#EK`uX{Bv1# z9K5Q^Fpiole$VH$H*yogdwWYX2>@kOh{EHagQ~_66Mz-g^Zhv9E4p{#YUji+RPULN z`5A zR$KQ}jc<6DV%i23H6t`w$W6F-H%HltNTwN`N#m?6Dveh9ycfH>hJ&l!8|B#yWLn*q ztp%d{3U|m1_h9qowEHt%l<58^VF?)_$O>M0^i2D_WT@os+|Kl1M-(4Vkf%z@-aC})ez#C8S%0XV)b=?G0|ta3~|m*c{VPshpPUx3G0FN<^8pd`y< z49x|MH~COUiJB`|&;`~MwgU|4|B;KOProNk8<>J`l3yC}w2&@lM5-2({`137H#pY| znw!;}hZ)R{GH~r{a@ub*=rO<)w*=N4Ya~r$30M{t(86Hi(v1j&-)KK;4zLZ8WlBj? zkVGN~xq>7mVMW7k)<_Xx@!2*E>*<;`u)KPxI;YDeOF>129McpapI9xM$Jf%D zZcv$lt)3?Qdjgt1y3drY!8$*?m|4zbMSqJgajsnGv7(dP8u*b z19pJp8!a2?#ayh|)a>LsWrdM17yh#)}k(`bB@^o}GPq^x!XS zQHaRwUOAEebtu7N2-`uO7SwyqOs;y*oa5)%LXIf&gNv$kNQ!*nL9-Z*L~v>jN}w_F^GvqT6@frCCUlu`Cf$v~Ez0N8fga<=w@<+pr!GUW;dN}}6qSU2?HwqDRw$R%% zkcHm?SVizJwI#xWY4BQ9oCH$#IK8X3{hQ-8wT`^mppua!B?WUMn#&JmKzh?)m3F47 zvTVrsa0Y22bl$gg7-8)%ts}*XOw3|`rSF1?X2P|$4C?R`qm&$%mJaBxuG49hZ%~;p zeOQ+Ku9dkrBzK6p`Ryky8W(Iwnus^y)atV#$v#)Q8`%!%YDD?mR#?>5H6*S9?p}Y9tEMx}{83l)4Rq+qFp6kwURc^|bOjZoOrNh&S3R^58z||;}(?G(f?1Zl! zQpk9?I$^F<c8~tdhtGfw`tUQ*o+S7WgW9D7 z)(Lq$kDwm3^?n!1w5v3mLv(1LVj*4;-jHsm02Oq;zmRNXt)&2Hm?z=r*y%Z&f#Fgq zFpA%YwS+%P8(EtQd9^T|ieC?HRn2nF=!&^N4*mbM)P90%g|$;NL7%b&O$qas@KW7u z?UeUkcbQ=|%f;tX{)17^jtKnh$ZOmn!HFFVt49sl`bKmcWIW3e$PZ(6srK&K^dt_g zj9eoNA5iWICtXE-txp@RD3G9UHJjRko1?4T42$%8)#cL6I@3j5Yu+~xvyDIV_Y)C* z`kG&Hut8J`@kTu}{cdW6&FBb#y~xwQ-r98Elvc0)mHyyl9=pqUOipzKI5U1Kd&J^6 zr`E6vz_C0MgPz-gTh5?Llx}%sW0hr8=HhxpLccBI^<-@uy@K%7a)(|h0E`hlUoRH~ z2%tqSuSA`1)8lesZcKOU|8d;thV8>{#3l_4f{9uet#H=3*cE!bQN{7TzTF~*Y@Y7Ec{*lN&2-F&a!v!D{~(t{L?E2l}A4mGkfMY3v(&s(LFfAU=`$Q#l< zdnLAmjK`0k`l*7i@uQsys=QxCe5T{y;_JHW$Yn|{jWDCY*NRtf#w$0BkXNBO%}#KW z)oU3)plbc=2v1e|2b6YVMLH!ps9^BJeH+yOTsu+<`Z_owyvo*mRtTag8*b#y{X7k7 z^1aiN;=3g3+9NZvcdd7M*wmGlO%5TQ zWS}!#s#ZXICEf?kr2_GGXzJHKA>Mq9#>Q=-{SAJjY@YrIyvM%dTN(>n(KcnJaIo5Qsh=S9XUL4Q}E%k;YaOH-5WO;}vUuCQ6#bj3`@ zwA3(!Eb0cr@{Jh1giVH=gXC-l{8;RPcMp5Xt1j8`fKUgCt1)hdUEy!%&WvN;mZXg% zLf~8EmkB>S^4bM=VUc^sf(OsdM7}3gPZCq!X(>_a@l7RVm(+aK>=Fo~J&rWj8oZ0X?SS}3X}c=%-EKEXONAVC7*_hT zdd~=8{5TE{giJu;joXcAcwU-T3mR(%8L|9EbUkPrGr`V8%EF23Xp8hPJ%7sXDZ~`s(mqyuB!GZNw9ZJ|m{grY;XhZfImk6J*i>RgnMMoZ!uxR?h@GWKkjtU6Q}y(hwY; z5a)}{>?^ls@SybbfFh)^t1}Osj6v6!<%j8OTL7GH!$q%upg@+=G}NK5?7A&DT!c}V z&tm)d0zrkfrL3`XWxUMNicyU%Be%UtH|j-*9xLtpawxJ-_gY-qS4%j#U@>Sk9lC_! zvbv|MjPyW9N6ho=wGH_if=fr4`MZkkCwgi)?H{V5(_}Uc^jw?7lVLj^);@b}b|;3p ze+F<*Un{b-7Q7k(|&1Gvp6TAJN$o)ZQg1M zQn@4O?|g{5o;l7rKcm+>H_kuo$U~54KGwfJ^s|h*CFru&5;(##2az96Py|y;G%1QR zWGVD7`D-RIE!o)rTMjCC+C^(i7iHv}QNv&geI0fH_~3xYAt}Z?t>nlt5hdGsm7?&M zF8^kRsy-%M&dW}9i-+y4guPSpQP5s@R8D>lgdRq4b>|Dae@^ov|7)53ZmW?h3g__c zkXbKQFJo@0gZDqouG54t{th6js|x~36^JzI?J)L!vY_R$J#A*cX9wfO>kUG+-`hS0 zb5Y3Q4)ue5R3)b435hEntNaNGznq34To&8eobYcV2P7*NCB;e5#Ro8@H;sAA>m{<+ zLxeh?1N~Aa4Ic}0ZEL!uMMMwwsM06OxOSoUG%VE1&FEXe-uai>+`=9Vo)#TrtG#?O z80M(JgvB7;koEh`hTbA+L>=n8W4#3BPS|*N9=zPtRgqR;k(vNIS~Y}Je7G@jt1zLD z0Y1+Zsk`!_bTB{fXLW8fO;XiO7ha@5{MK__?#)3NEv-9*_{I;LtOx(tY!tTC zb>y}3WuDZ3_T_lhbEGR=S7NsQYMB6a`Dbwx4ecLP%r z*&Rqarhx~fk3*v5FdD~BEW(&!_{=gQV(!Mt7fKSL^iTRYCCR$y^ngJsG>*O8UPCMq zzUGC&{-g95ZJ_jmt6CPV-5@SlntuEpCw3b63{OMPb<~@ZMz<&4MwRL7hGK&lmWgXd zq#O-m-~Q7SzwW+?7#Dc6{VBt=a$LZ!vtnOpVqz(+kny8_xTCx8spbERNRP;amc!>bVEaeL%e#*C?8s_yh4VuLsg z@V&#M1(&x}aS<6}g+U03e~ZaFsE@3ftNlHDPVUaD@m9$~;!>hMYK1t&P@t%V46|>i z0RrO{4-^i4zd4Qr)of8qG0AdiV>iAK58iVStqOkcTS4|bCLLByb4jG61YX)H9Ak9r zDpTU4tlA?vhW&z^tjBuM(9_FP=-7X8l9dudf1KbN4OEpaoIFA7AGIpC^gS`ip^Ra= zb9$3f3B0!#D?=LuEaB%APS4NkmzEKBrpW06UvNv3pA-T86tfdc@9PGLI=lZ+))&zc z6=K^9n9tgWse9K3Sz5@A1l$I3S+8mII##kt8HNGqIprRV42{h_f|eo;EY@NbMQ`lF zM8?~wrQ0hPeL#^tpiuD-IBl6VF7ZHTe;*D=0H#%131;x%KOy{A0n3r^H`d_LnFh)3 ztVPfKDU}UYD;Um^_T!e`g2s!T;EMn;`Vm1NZ|cJk4{IZ(z(E~9eYK6#mD96EDATg} zkuzyvV~!(;&w!owjQ{Q!@QIwm>vVPbO|osUA2W}(4Wpc&Ka==GpHYX6%d*BNf8tf+ zkJkt11%ycetS737w)1RX$m9u)F$9ZU`$Y*od%VD?ON?+>iNGnV;6nD2?u6cN)fRRs zan!(XXI3Y9$pn9on#zlYJ;ZGw@x@0$?P63-s3vxpUzi=wD14HczLPz4@IAJtUG&MK z{7_?AJ?p=aMpm75JI+3p_YTp7ScTU={&f^Kf(FM=zJPmt&$}N$@;|{B#eNb^NyfMT z-=T~QEbcxl?}^L8rPM%&sGn6U^s7Ddw+NLG^Z#Ek^#dhMrD~64Lx~g6>ISqU@EDrU z%4fl${lSVFApkc($iFFs5y)%5fM2H#ryO}zFc$GMKfDKBzr;v|@7i)AIjhJ_{uF!+ z+}Id(XQtxLk@;4f3aOE#04Q)YXmF(6E1~5);~f@mwrYQ&Edzx70i8`rf>T7NtgEb# znO$d&!J2{{TkQt4@^rV)e8o`VnjCI{B|hD((Uq&kO}K?M(!Iy@4Q3&}Hldo=6Buo&lfM@^*T^+I-?~%v zwlta6R^cQptX@@v+ z=vUneimg@!{y$$N)C#{n=kH2x|M?%Ex2|e}ssu^Q%KHehoMreXo6NGP~)X z1K^lP+P!lDg#j#A`QcU(x@w*aX994Q3J7oQKMP2Yd-BcPSm;aB{2l(T&W?OC`3@Xa z$Am6bhq6t`o}1DX?LC#z{<=s_*yUyO&WVKs$060QGs{u09<2>a`aA++1dPYQ zKd(HGHfkKkMu?V7 z@4|$6Yxr@kqG|ny4PWU;%#83@ZQ>x}gTmT$o+4_Xy0@vL9$2fCs_v3SwC z;vTB{(9%N&R1D9mr_?j)d!gTII^bYC_(7})Hpq-43VO!!}O7eZGHZ_v42 zdFqdL-W_DyL~+XUyiMZnP{^+@=d5a^03}+N|Adw-Y8xd*=}M6_D0q&wMAZ$yK)HZF zSKnH%^+LB&IdE0GR;cQC;-t?ja74`$bh#34c4dQZ=Pz6cRO-=^f?w}R8T)nBYaFn^ z$Ki8c7}$z}&@qZ}kRthE@x|eEXY@Fiv%|F9xZMIG5RatRuo}KiqDtzS{D(}@CMbD1 zmW|e!nOVzFKaf`)Ll+;ZG4DFb%Og_n#@t4I9Yczdml8z`6NtQU?b&=ZoNwpAH5}Vc zt@$3*`cS0BCmc<*YU$3(Y}rtvkCYO}f_=+^)H|%#5?&f1rHBn=F04}!G77{i9N_at zz3QOHJs5Yi#;c=PZO6}Q8bFuik_1C`1f)|UL**Q`QX=5Mgn7&G!2m~f7t~4 z#Y0|gbf%6?{3*@zvnAYFKCax&-G;eZF}Cs=&0-Ok!AFheLY=%nfvd&HCDGuCMT#S{ z0qtwsDtCN8u_nb&qrJDV{2dyDzFO5;o(4$_zM!ln7(<5fl&H@(3T~wea-^rn~>~J9+HH-BwNvc zE**Gvf3`$8joIPTWli07R$6phHP_t^Ro;$OrG}&4s7_9RwNk-4|;bX8&bAjll78y))|T(V^7qGaaFuko+C3;s)*NC zEZ_MTVg1Ennx3Z=kmNdh(K65BKMHZd1gOJ*a(Ld)zz) zPC<=Z+GQ%TU=$Uu5={~@{Ejr0p5YH)kt3vVCbG+7hs1tsi`)-P&|Iym`V5@xn*w)J z_Q2AEdx!Xh5A;j5RykJVf;+WXOX7QEzw-$vpbRey{q!TWerMEgXId_jpHGm>nC++{ z-FvWpLQ06nK#wu1VR?^r_F}S=N;kN88E!mHhyow2MbJyBw4DBIyluC?yRjFPeD*6! zfOcI_i-}Ve%D@`R&`v>{5@27EbP_yY;QJ)rJMC(D0KVgwS+4|0D>uWd2Ryb$$u|UY z$yRNhuAc1LJ@^Jq-+|KQm}h4gke$qQc2?A|raXu z+gUGJqeq04!FP%kx#OEpZOlq&*1QrKocrIuzr${7Rvq+)1u`TKNMV8=)X^@eisE5w zFk$qcUwTV11Mo%skb*%&s5qJT(F1WGvTe*IZih=EH|aAX`{7MPaSSEOF`})Uj~GqkvAja+>zt?WM7!68d0jN@ERF6e> zL!iQbi|$qFIV$h=r27Sr9&x$zut^(Py;GZBAl^QGk2CHVyvM1&xE+XG8ahEtID%0@ zICo#E^~wkrn1zqv5)_BOP+bMfW$G&H*1Hih&Q;2Q53?s=%#kUPeDhLhSdI*Im2~1? zW%X068U9xkD{b+{WNm*g$?ym}6Fi~J!F?VO<=7AEd1mnp5>h{OuydAISxl*z zOHcN5*UT9`7f_ca+a{k!N2Ex4T8CXhI()_-Es+_|`2zkE=~3Y8U)5)Bau=*4i-=&q zkXWDeqX(bdz1e!5K#}=AWq&+D^KwZQ8G^KR&I(bWJ2BDuK&@+#=l9(zmi?+v=SNxy0AQ;XG$Efcg%BTosZ6R2 zyCsx7K?!-BA#T+YX`IqLQ zv*_ZUb|bv{S$zSd+gS6Zevn&?KvLwsG!I{b;1;idh*vO>Z?luaksiOj(X9u#Ap5;L zjAW{ED_O$DqUzi_V4fQc(4H-IC>zsIcn~5Ov&Q6W&S+sXg4q%jO`a!%6NH!y7tS8Y zsvfD^@dU}OWw`n99!Assse?$B2(u7aC%EKtmGBXU49UgyOt_-KK~>2;8r|B$e05v! zy6&o?qHWH6cAtvF3|aS6VW|@(Av{P6{)=?Ik>(5#WaVHJKU2$gk0CrCQdP$qP0Cg0 zk2$axZvb|;@xXq@Z}_7QC5Q4HVXJ7I8OYPE#loRfrY%)c-)J0WteMYI(rL4zGQ=mJ zjnSYQS4E!zD$qel1@0Pv2LPmHnTc+$v8%O$?OOqVz)JJ{7Jv`KzrZ#Fpus@&0-~q} zBA%?JPD8mivwAhgQ^xm?&;s38tJeFhXS+oUCH`KFu8(U?G8;exHw}&>kiHMSYribx zn*F#mMy`~zGb$L#543v#(PDEg4`rc)c{=-I*FIqht>DTOFM{L*3u%0q+Mh0Jc|wmd zqaD!3TO?MP;i#^TG0tWncD|JH@iyW7`^2^e5^r7EnHP+Vyz)L;CZ)88O-l;1ts-Sa zXjMWgG?f*M^J%%5Q;UI7PTbA6OXAWfQSEXIFdy=qCSpjqP5+*9>(< zyQY+nRkohDY_3Agn}V{5AoQ43;U=#m%YhqZqebT=$k=qYaAOX*qmxJoO*o2b;D|Kt z`z7UIK2(M_5Z9z+DXpDXR+I8=_ea?_)yI6MZMMk#meAUHnB}G| zZ0N0NmmmRbVLim>sfC5mq*WV5%Mm->XA2m7%amz)E80NL^^U$(2Oc-zSuaP%;0{F_>l_e>%L2!}A(0GTF1|7(6aR>iTyIh?)D`VAbDAQLY-&WkhTXpqH! z$(%!^U~6fiin&rJCCNQ5Ar?I5lLVbifd!$l5eFX?07p;a1p`1va3GQ9Q=wLesQYgu zP#i=vK}vID8gH7rqelWP%gltUBG9(4 zk3)(lQh_ex0Qb0bmjRSKKltFrWVsF{SsyeDKH5PT6TwUYe}qV%`H}K;se57i^-=GT*YY<SUrsDYMh)~`p|;ikc$xq zkc)Ew5O4h+K^(+*woSk`!b{UgVD19@yN)=!Jw8_MK$vO8ZKI=|7O8#HcnuRlP|Jm} z26p!eX&kC6k0`6@QdfAcuG1hX{ zF@eH*gsl-S^CbAZJ z%A;Jje2@qfs|Jnte(pz%gx5(uI+0#gZ%E9RjYQ|WH?ve*y$GtPnLEoN-?hk#cN54{ z`lRqz>nO1G>nFlXWj81zjC+cKl17I7hMVNz{gTIlAgn)-l#bokrS0yvdbj;kZMzSCKm2to?;vmPk$R&cG-+ z6ROOHs1p{Ljm|WTW4Se}p*=}h`BS!3HERV_csZVTxymVrqcSiy z{SOc`^H=<((`#{F1PD_0IMhI92cztD7U&JN7!DO6NC0DEy$AE3LhlOg;e~oKtxF?N zpgB>>HwQh~H9dn&pn{p4F$2fPOq7u_$N>&__{F2wlm`F#PD)=x`JF+h$0}CPFQck? zxB)}?9HiZ9#V-xh*3PYoQdJ@vM+a4%B56@34*V7+@_bVd?;5Yj*{{R}fY9-vD&NNv zIiwIU;8~>p3<=N zWb^4fSh17Qh@$C%TS1BmPi1_kwnkW@?p|^i`wzF&9(2vnu&y(NQLhnVb*L_es}Di{ z^JM~cR9p5MbI0nOd7YV|Gb-AN;)F&%7@K}R21C!?)#}Gg(tQFeW zKsG~oOV6NuM~tSo*8a_*LXcx!`@*5<7X^JD{Uy^6aR;PN9m^0;TywiSFQQ&IPz&lo z`$puxTQCZ&B0yi?$&Ph>yLNvU(u(B!zhxsHi@8T4wdQJ##AR5GFs?o$_wn2)9ZgMrx@yhX$>K|lQZdZWa$}DPp2@xv(PF>%YG}bD0P_2EnqX@b& z9XZKC3}5efF--*EN$$Nbw~B>7xpbXMb93OS$r=h$`EdeN#Rltg*4loiCZ196=?hQ8 z3H1!9;NctW*>H%^_Z$F#bpH=V{hHEr8CDhpEbnjU=G!kZQq_;c*jr`r$;OE|@W`O2 zeZNg2AMp+CM|jL!-?(Y;>BcbCI~?gD5RrdY_5Vt-<_=wUVD%i;G;elO-iE@*o4uG6 zjsVJEfb69=q$ICzxjyhLU}>Iu^Ub>|-b$gyg<^}8)j$^zyh#jWcyFoE#OjZGr@-6~ zH$xS4Y@^=Ar*A1%8Z@cS*-6rPXHMMMTG7hZuP%2z{)+?#8El>Mb8VA?ycKUW7*hIS z)a*)K)m8qTqLiKu?;lVxeTN#&C}Pnx{~6Qm$|9EjUW3@Ofxehw!A==0@T+rt z&5NUNqM7>hF>?Ieclp-6YMZBajS}_#RWWXqJSz5Ey?AM`EZ4`3Co%f^qeaquECaHj zuB$ReR4AqvkZmD+^*{ESIMjb}+V~*pWc`ZsL0awl@BuZz}NlOm6CmoVs}K zYh57nI4=m0`OKcDp|gm-YM=N5pr@b;^t^a5l{KBHtO8s&78GmU_s+>is_qB-NJFy_ z2GQp7?K~8Rf)pS@z!^y_6&ePY@@I5#7BUlh3X%{>3{YMIT>A({;Sf=TQ3S8!lRMCGXle*|9PM_htI2@Ii&plY zjz(K;Fb~uc$EG$H>X^z$VEvx;T%7W^q622IWvj|MBQc2JBsOeTnga=!NYq;#wkJKP zfRGaK+cVd|ELRKYSGro>8M0_Vu_pA*#x?aWUa%iVM}=l#ThoLcoNinUJqm@$9aHZ} z2{;iR0+xC2Spr#=zgwHu21E}X+}K9Qy}a1uT7m9pX*w;P)-%?tX&|kUe3cZfE4B;h zHjxgJuaOGmpYoD)0zA7_z1FEhdO#Y^CgZ9K&kWPU#}iwN&Byz~j>^O7z(b1|{?0Qg z6DK%3wOUlQd{XI1?*QwMFWi@0bgxZTiRwF0m5w%x*Z+>A&!}q-axVY{EfVmzX*9F? z1sceD>*}Ia5L89Zt9ka?Z#pL3&z2~)DTeFINB|%e#CaU?O3{La`HOD*56HN5(jfZn zL8@xAT8u!(ica~CM1{vcD+h-V5ng~q>h#bM2Cy0o9)dDQFroXe^!Je-oCg!C9&Vl} zRTNa4KbmKV9kQ)Hb0<@7DQmt5|5M4OSq#?Mnduz8-}!^9G&Cg50iFn|SC3LxK}!mf zYAPY)){1FM$CNkeeI5P0LfGzRfT_pgue@9rK9x1kC9~C)fBsquQiAb8jFw1jYnvUS z7XjnT1fJg>)^x`*T0NDzd6u{dz2@ zk0mwhqI9@u8A}Xpq$Sk4eG4dx>1VK!*<4F1w^z-=3$SLi@WfxQ$z^HkyVzdLpA2xVW&dxw=QGyobOQC03Nx}_uv0$kI4)TOHOB7%U0e6<+!Y>7q+5+ zw&d$U;B7*d9c|vg$mrQA5)0uqbyXJ#HD|3dFiw}c#xW>+9Jqm4306=fR{SU-W^!0b zN#Tk_(QZoE_#Ry!%+xV-U-9y&52HI>jby#6@prz-3Y9O*&h@tyjy1^47b_xF+hOCcQlfb? z5on7m*_$CAgKRY&PQ7>7?_^VanDtwb?szxM=#XgGnmQ**`WLK{8#QL7K|ArSxpaAT zs$4e+XR{_znxANDL|{Wq`jQD(+_=C+OaTMpyLi{jDlLK<4n?=|dISbzqe?#VKN~$J zl}{N%9hKq_Q%R3S?%-p(n~i#rvHC*94wCo`XZZFw#P@00B{7I*M5FyS7RdQ~1j_g# zFK{w4599BO*}^l-zJk)fzxf_gmXmUUZO1-nbRss#^t=x@gwTicHRV zOKydy;CRq)NsjRek&+t$pWG0O887EDBQDM&3T4yuuweWw-Qvf|#n{{`P|!7&r!MfG zxdpG=QnJW`*pIcjB`viHDYmU78ZE1oVOX%wG<3ldMrip}+`E)CA9H1xt#%Wwnn>qn z>8YLF>g1H$Ek;Tg)1MXVA}ihTeTPWdlMKIzxsf<6_z<30!im!V5u#>uo~ce{_7aH~ zvFc1=bCA9p!yRFl?A&Iz9B-WfTc_2%jl$}T#q;t*gW5iS9gTq*t}u3379{UI>!#mI z@&+;&nRR|}JI}{6+S2tTE6Pm%Ym?pdnXYZ}HhNrR`}VLJat<`;tyR}^^~XM6%k*DQ z^s$4T`1UC>4kJ#$9p8yxgT)QNB%fNp>}I8Q2q;1-XUi|BlR#a)q3P;(yoC00IAkeOw*m}qSPDqH%O3Wn4r_W1M;zPfigWo)DzQrS`2^Iab88 z)}DIwOYkag{{G;GKw|?WW<8jfH1ow)mX5mxo*8tEO^>Kh?&mbV`wY-7M7#E}XEKk+ zp%whnd4Z_y4g%91LrhE}U>XBYw27*{_Dlx=oMmLDN?N=0x$ViH?VQ5$ zdkt1qL@!od%ZUdJ7#r)SeR6Yp5lZbrYPmQKr%ZkBC4m&@tCvqR(Cv<*>=~Il6G)gk z^=rzv?Dhf;_hG^;3P*00FV1GNOo$k0z!KEa@+`UNczoL=nL)b2!4)DIoJ{NVBr?my z+wJ12<_%IL5n9NMz4Z~{yqfK~ZJmoXc3)a0;-byNm9zd9OL&4f$M^92!Ssxx5Q7H% zgf6@&0$>-jZ*w7KL6}o4mhW$4(!&r9R3Eg{by_97mQyiqs#j4u7!fA|1HnJoY}Xo8zFI+pb+}YT4T$c#kf2gC#D6XW%(e_Bckm1Jdn zh5b0)IsCm_uti>xeYsDEuVC_kzcD_GTRP26W^^^*2l!^*+@p2s^#fyutONnzJay40 zR|Ok^FczNeDAp6O*ES#z02$$C#FA|DX7+m9Xs`6Hiqi6?^7i-3swkrB5m#QMn__Tn z2#Ro>a5ESHMOA(j-O>aKyHs@=2j8k28lXn}I&pWE`1}A-T6P>L!PJ2OpD771*J|(c z-l20b`b3Y5#PfxBS;!^h-k!&2MbfFA*&Gt>jZb;uSZJ|7F3eg%`*s z1@Fjnb2v(7U0Q?vd)ODID@rO1S`myXspSBHl@YHCT6ORgq7gvs8p|Laus<(G?x@V! zEwf3&e;y#c;@hm2wk$)kMKwcC(6wgAIy#{{YUv0~rveIXsz;TNyx7j?0D(Z~Ecm9@ z)oH5R0}>Vge|a+8G9U1S6hasHr6NF8UVEh7mIO|h3g(qPi;tJQ`8UOZYxR*(@yRVT z_^7qeBe_3yPZ3SsQXv+0u_!<4c;6Wa==8+n{7>uLBzIDPUy4e>@8x13+4b7+;)#3C zeZC-PA;AlS5(4<6g*OM4I{j2vDhTVk#Hm&&+xcnVltpUoH{)80^d=trl%g%j-16*{ z$Dl-vXL1P{O$gI7MnuPbNtXaGC^J_shNSPc#oAM0!QdiS-6|$s%gu2PH@9e>%B6vt zctLz#7)7Zx$JT)~Z zopdPQ00plID3SHU{DCBJ*n=(2g%tyY2%hj1KJ_^a*P(!;gb%dl-}9eiOG%`B%>dQPput3gArL#)M|?A3uhh{K+v?g@Kyu<7yWdXbo0Qbf z@Gt=O_A=?I=^uu@R1K0?0Wr&eLb@W|@-S{-KAA140`{{#A#DRkh=~g#yu4Rt95FMV zK@jSAOIX{gbFevw))Yc#aER&$hI!pUzoJ0kPly?8|9Y#@u!meAR$FEGvo$#sSYroK zBzz?JDOal995*B!ulX>d2vff9>!7kYTCI*+Dt(2Y#=Cwt-PfiEbP@EnjjE}wS)u_S zbNnAU7RdRAGrZc{{Mi47vt3rK;LoKh4`$8!?5fAx?}DvLRpMz6 zapDvQ=L~Vp-&hVkS(}(=%4l>lnKM8q=i$?``FCf1J5Jm$#OgJQRp51w&K|ZWd9yG; zv!*_6e!;JlX@PeJfviT`F77wpAHO1J%5KC(Bb|AHk=Es6oC~t^*%voS?sQ^ZsEw%a zK_-=?L!;I=u@}yY*iM!0iaz-DsP!_&+>W)*P4`FcqnAwkIbuI>h;A9I_;{GZlA+^7 zbs~TRal3TGm5yCxK<&!Ij^1cLkC3?0gk7gXVh_ z8@0dR@q2p_Byu995Z;1?^r z+$;}EI$&c%3Dl*;UIZ-mq5I5J{|iW(S|CP?;V$y6+WdxQOrx^VcsU8tc($XY6L9AO zUMY1kjxY-U_I(1qWNe$`ID1df{V!rA5N1PpWGJ6_>Gk-3NqF)jwZ<)OJt4oF14~5r ztT2&_yOuj8Q&1CqYz}uqFAWCzKCU+P`8zd01vMG3qmRO{;iTRmF;l$bo~Ct2r@_jD z#%s;cf--gRIXuXgIAt%+(>7Rcx2h8rnl)~3wF}RN;Im`>oO4^om(+u;du!z>TUEEn z!x8yN>9%cU-qTwy*P)Gds;gtt9s9N{DG}{KBHAW#<}t~2%tjEJodNH)BUVG7ljT8h z5Gqx9EgUuAJ)>!Pxe*V!k3!Ar3Z;=zCx6)3SPm5$P);Ow*u-1Q<_9nNQ@HKvi{Lt} zkm?HuMTvwq!GcEl-U>KUA+_sLr}E&O`wP@Usy!6kF4v#)G%>T-z3K!EFbEQ#a|`GV zOZUauE=~rl=B$;pS!wabCAwPO(8W~ifj+nPk+Ba>u{(tFHV{gwW&elR%2n;JdGz<~ zaf9K+K3q{mpZYJwQ-VG5rvQ&1F@Fng;NPGBt>yR#(0&5uaNDhC5>+w!xOtpSs4%(zwr&`wnw@jMxuV#S5xdWVTHi>HpKrGtk+PO;vrgW< zvQ2M-Zk-O&^rz1f!2|?{d@&Y&9;aOL%MW5f3TBd-mqsV7h!HB&cA3vmM5&lZ6)0`vprrAF zQnnmqC3Rs>Dxh?0kYP2InDo|A_pwtJJ z89R0(6AXf^1-DqE@a0X zl)NjVIq17{{Q^?X&I3`a+55Ah{8Y$mjsgUYR_5+61cL$UMsSV8=*|!}Q2DlR@1=Rl z>q4#l^JEeFW5Hhjeo&(Y^Y;fR}1=8 z5Zh4pa?h`pUWUr34`m5vrq$^a?+PXAwM@=4?XQ*We+fFkRyadKg{6@JWCd33%F<273PAgrB$#RPmc2*0p=?#DhA|0qn6 z$0^hsne%?dh|?n4P7nSsnEGp=s{2$qBhHkQ6`9AjS99Yy`Hs$MveDM zpJpVyX%~K;TlJ5`rmEq1=?>IHZ)EudQ3+fhyD@y`YAK#yB;%awi^VA3NHQi@UsC1I zq7RF6`9XKbn!|ZCG)jkj^OvA!og-~NypPQI+Uw*@pa#7GnK41P>pXzKEP9Q^L!$LP zj-0LS(c0j>A+QjQ9Vr)_MY)ddKW_kG?;xl|BQF)05>HI#I8PxhjoB%w*?q6mZBXLC9bsLeLst;Y&II_4;*kkDzM zBTpDEH%Tc%S3mHGa7T1o{Cg=E6tY-a@7Im%fk~9FbXcj2~lCX5}U`TR;JpD%OlA^kMK-=nfxkzLm*zX`(`eLK@^kgW< zYE$a1sGjE?9o!H}K-F`a;fp}~Il0o6O$5qSbvHjU*`XV2pYyaKIrx#54QR(ncjF&e zY6@>_98#Ai6c7m>kl& zGBq3(M<>A9^5^%&kIat}R-(Av&hX^+@xWm`fmI>AwI;2^Q}O^I2nfm+c1B81hA! zo*2f2EJJqPdS$`rCEgA@{dJVe^b9e~S4+gLoFsIJy4sITH~gX;RE1#yw}vhbBR#}) zz(-{Ny{@c+x{NNxD%m!dh`+=kFi?9yHb3J@l~Y}|+G1sa|J{eMNlgNXW|Xm+n57YE z8_E#VyvGzRU7$^5Mn$v|>#RPgp{PMJUHS)ys#cQ0fB6+xa;*Y7YSlzh#@|ZEVjl!< zF7h%GS$gCBM{#$t9^32FE`Mp`%i=cWy=T~T*J?Q3}IkkN)k zQsPtHPAhwk737;Js(;qd&)I{|fsJigrJhXmzjpvSXbh3RH5ltJ$c!erkAkz~%J#~9zSfwkK?#aF`_ z=msR28lgZ%D0%>+gL1GD+n&%f7nqrr*mk>hE=i{hMJUXCl9diC9hbIP;fm-;aS&~A z=jLUy2*IO!i#_o)KZvle)MaM|VH}1R%?)%vnupnx#g^*}!*@}+(#{L(c8kVu$%e^J zx!-RCer<=c<5W(ijlqDybgb{Jrvbwr6K7de?p`7_P>~TVM!l%>66O=uCq*lLm(m?o~?)wtrJwVs}qu&a8y5}Z2C|d#vxU$ z%E8*)bdNZh9t8wi1Go5?g8d2gD(WUQxe1j^0Yr8EGZAfmnzI}ruaRl*Z0|Med8Cx+ zI!;Jm8d*&j@HFP3QIqAi6Encv&_65-)|3I&{oq3czvszTRbC2>9CQkmKP)Pq6vgDU zSiEG=%Y(;1Tt+Z<0B;&ue%&UHdOqAi0HryZ?X?HI1@!Ly66YdHyX#ZVB@J78i< z{K8<|Bnm-2zWcMhJ)>nDUIsS*Df>hbyH4T-6|p`p@c5$CTGgaXTgD#F6iw>@#LY`=$vM&^)ZbUC{YgsDWl@=z~yBDd%uY z(nJNo?jpG(E(eAGS7#3#*Y9Nb#fRu}!&;RbQ4HS5^(rzEgt*Lbbyfje%?R_Iu^65Q zZECNzFz`S`=6RuJRo*5K4NJwYrz_yK-(gkok-^^lC3TF&GiJU&ZZ{mDu$JWU0Um3h z7YXWFxGb~V6RhEim1IVX>9Fb;78qfbI%_S_SBKn|M2{Cl>!;>S(Q1}VtAC5Pk!h*{ zj#6}?iU|0DD`-p<->X#Uvzk9xeuPVUVgfFS&me7;V?hD}-3V`gVal6&5$2;YfQUNd zI;1|wsZP(=TFbwY;w$ahL7?aOZ3D&}0mnQ%a?EugdtU(BG8;5Q)3 zf(c)AuFg_%NVBh$S;i@dFjwraf8;ps^Q5J%-HZzahd>^Hv{NQHNgh{0h4}%IzuaHBR>QTYzNt)KX@OoB0 zph7TKpC*7@Ov0AC7wx;hfEN9hRAW}sF-Vp1#$b&=oB zO=D%v45ZfqAGJ8uT5Yl+T#fZ7X~jM_D%%pzb-jn#vAf{rj}t! zR?{CBn$JDBnyA`k@0?7%P#bfcN8kg4o{8$EAtn!HG@txo4Or$2z-@f1ZC=0--fV{v zL$;5@=9f!YkdZ>+NEAZ+I}juPWFx%k3_}vQB0&wU3!KNe;b6bhX8P)_djAR*+58i9 z&8k=&%e4?(I(GxvKTSY_w^=!uX9a3w31rne-p)b=VWSzKU5t_{xtAC}%gcvN;dM4q zkVRv(dN4)aiecAl(4OInMthYX+JJqRbZpJZNF^qd5(u5-Ke9i&hQMo-mw4 zbJ+eFri?{*=4mcHzb}}%4o^INz5#b%agsWUCPZGVw%w zfTt8nu6AA74Em_C62Cm?eqfWp#D5X}PENG_b%|kUaY#2YkN$S<@Y-$H$3jF*>|9X)q14~HRl&j!l)0rpGr-X91!uQgxY|LHNrv2Q znKegDly>bzus65%&Tvb%gMs)ah*Sz;oQDGqp#_E^Un|MFMAI>ePyk=Qdj29_5|n9P z%k7`tj|wZEixu;7grFS^!lUPhVX~P$Ebcwyum`Hw(B;VA22e(He0^8f!%VH7$ugpY z>{6vLRqMdBop=UAGKuEm>0gPf-I=C)@dr$ag*h#Lt3DBBw048nSxB!b1)8;q!I55E z^5Rtnr)rCm-CE^)q*i(LYg@K923IYs zecTx>o%#i~j2U64ET5+@7&7(TR-ksgV<&?Y!DLe)GV{hewp7J7+W+fCpXGHe zG(R4m$C#l|$zpNM+`C#t5Ngh~zP6kQ+h zkLCYZx0w&v9{Y-~Gg}d>b5dBMDm!FNpvb@6eD45vZ~SzdvYn$(5c( zh{Mrb0@BdCTQV^$3rorvN9>?~bZa;`kAm8!w^*NKTU463`Dq)jF(1dAZ=d9$RX7~R zkzQ7xb~(8WiX1Dm3h<}789P>-ZZ$X4)0q*$3Wpt3GOg`TeZH?E;r11ZA*WGtuhECz zcfw4|2b^##V?@Rr;eK;8E}rNeVqU%;Lylm_bqM)Q43808h!Y>FUI7-O>rojxv>!Q) zKT7R;SN88@7HBTE);~(07~zJa<_sxq?F;;^ge&`12=7YWqG0AE_bT~N>OvS>wymQb zh}2$M4ly!aWm6~z3@zBZ?&e@ov^O+DY{J+G=oE7=F*rjS!@#s6iwtKh%ZgOf6$Kn^ zywVPQ@QJO@-dsP_lI~4?p3$ju@4E0U+%3%cRfnIvoE(JCT6l!x96An3mLwc7BUEoAbS<5zE085xOZY)#Wko`<0^eO4&=1#y}VQd_2#=;TSW zU)ezHQyk)&>=D^#arU!h%+Q)~L>bx1=GkwYojlIF@GPkI1N%nS6qZvLK~RwKO}Us& zs+(U}4Fs705vjP5coR8MW-T(af+4PLrUb zzKOv4yMTTejGMBPCAyUgkWh|>d{)1TRMRQboNC~AHu`0i0$lsu(89wQvY~=XL(m*) z1=RV4#(8#zB3Nmiq#1>1qWOi(pyuHKAU`Y`_BV|ntY6ozps>Zari+fslKbfVT+cod zwHCi4qm53w_CGS|Ra50O&g-3Y!{p7Xel64{X{K|e;^APhMSUYJ*z+pca|@cA&a6zC zia(e`DQWQB5z#1bd!X8YXgCrW?-Ez%f}`!09X>$ib|&8M0!Xt0{$$*U2DriW2~x!6 z+7mEg5j=0nsHc(2A&m&nV85tfv~w23G(IfKPV+dnaL9FYd+Js*z}gJ_j0)@;Xf7Ps z=wS-AxfZkR^R+3~G+~)W>BSyLK)D1`=E{}GOt;ZGWh>TkP&$RU_v-Y>lUU(^CfE|U zYJv(GTPdqxEObe<$Gu6|&jwNHDcVp_G|>W~j2*da*YiexapF+-qljoW$n1WXX&uwx zjG<9smuCjpjwKousW#s$-jDZs{e&tN`^@Bg5Ey(S)y-BehShOk$sVV+o^8%9Y`9iV z5$!#wCJ6O`Fe6~7|CNE2bMlR@OgnoLd@vvlV;!Y8umaBkoC?%|g$x@!cO-;gd_>Gu z4l*Y?-I8;xcd#=5%hz0E^R5Bsdo})X7I!S!30smdf6+b+G8!zgmn8&AV9P~Bj*c1i zcE|Ce{%CO;!KYFJgMrCt?sYyP55O<}|m z+;K2ZJG&TB=+Mb<0S15A%1(mh|?434Qv1^r|Z!rx7@lcVJQ(IxPTPlPeiLoMux z*BG+2*Q}f%A9HJ(vfT|#qi>nj%}17=+Q00YxBEbI=|nU(ohV*NOCeKwPa9TEj9{aI z?6ln!bH3SqPH>|RbWIF3dY_}xD6o^(wlPmIgpeJNO~7W}{_{n>4Nb9;z=!2)d2DD7 zWHX^@g_~A`W0b?R#${ME!D3kEcS(p3Ze0|RZ$WwufhUid9SuDYS10MG2Q?Cn^8;x$ zAR~*>BpZNYZoE=_)VQ1UGBSP}Go<-X3EZY#YwJXiT<5n*>jm-DdQxGlb?pVp%rbok zau(VO>}2DflHo>ZOj42VM0dPLMTBUY3W*b_&2;Sx@-e1H^Olia&2;#P9Z47xf@bjv zLVPqaqR;enR0Fd6mFpq`Mg%#eL1*K0{0SZCHv|#evF5Xtbi}_C_ZD`cXyth61NO%i zoGR){5xz8OJ3w8{dP8`MbF{vF2yr!0f_ruGB7oHM_K(RT!^E+z?7!3Ok|@lS?a60} zC^-_~(40!fH;1*a^^??JEUF1r-6|~2hV)BAuj2=>ta76aVGi&;nsyq?wW_q~v^DVn zoRzURMxIKp_eCKa;rc=lNhR~XG^_a$i;JDNdC%&-x(Yq8odIoT`R~lhGNE4>P>#tt z{@Jl*4Bx4u3gTm?#l44EjksLu#{}pXGsS%P#C>>kJk*n*+*w?%}ppcb2rX};@SCDXYqzZ$o_}t^c{RL6@9&pG6YdYLJ zTe^H~l9J+XkMx`DV1flAZBcKr(}^SNB9=`1rpKwIMeFskR_x?e;f%xgvk{)BscDG7 z>9>{tfa?AR>EoANkm{iZKD#Ey(AOOoFc18^1&cHmP%Sv?X=;3_d|+c4%`S#Pgw z9p&v(*4okJGOH&hvL;eNpZ_op7W=L(##zl@9W8XhFSrw0uqP_gb7A~9voOerm*Lef zPrCUAn~xZt<9#M<`M~K)1etWIyIXA~p4vre;-(xwoCus9+!^LOGWHT$nNc(?FDPzV zqNjK3etSR)g(q5;gX-$&)7s<7SjlP9ny(G4qYLp}@8-wMC3>LFkRn0q*Y3X{>|C(w+8 zkP>k55~yinTv;(%gSkK*G<#m`8cCKxX0mEmtX7FEpfr)rn83!fx|bb)%Z6`dVbAGq zj8=vNx|7yGG?ZVC?Kqrf^V%Md<(4q-3zUMY-Sa;D5x!t|L3Q%2s&)NQLN74sPEZoNQW+V|STZe_yziFZTXX!PrFToRi>DJ&>;v9Vgi!WDW zeV%vI8a(aJlAoF7^j7^9%r_1}_V`acl7^t&_lF8)IcbZ|8vDSEHrH_Kau8>#!DegD z(}m4gjHq%%AC0wGe0`UUlC8IkX7dYvCKrc0tA6@`g6;HkZ{3K z_5O(L;=LNgdJfxlyt5Jm&+=Q%(^O#Dkx;G%dybt@&>UY-a+D2fZ`-=o%D}VaCvOJP zpKQqK;3Hr8f+<7asU0jVS^}RzNf{c$s899x&ZnGfSC`lTorMz$*7mXkd6kqafnrOk zTefm_QxDHHp>{S%E@{TM>67RlmMnY>A;~DQ_N5zp8y+!!#V^}GTT(1}Gng7EywAF; zzjS@7y>HK^!vA)P!v*#X=2f~9 z%KKih4}HjI*wN)5T*MZ5e><)vI8wX`LtUOz%od`JvDR#;b6Oiowi$T8l?AZx8Ijb; z{u9yppM&R5ym~*%j^?V8EdfS{%ba{;JqP)zt64oE4Sk4@)4~i^)4|u<4$boUO1fSc zN09E|j_fxs9dJLx@?;5zpV;$ULNZU0 z_T}J%%RBdH6GKfox*t-jNAxocnfbg9cLQldjzSDfNp(5ngI&-I4k6&8*k{()nzs^!y?S-#<75T4{G0zGR1 z4U5++*AcLL4IE;bt)5QjDW5*YVC`|WgEnRn(DQSjBlKo$ZsRe8xyWSvU?loN9@L(P zFdu)QK_;tkd4;B%RSmxK`9+_uW+g%+p-_*OsJtdv7i_p^&4K-JbmSl>Kc;l)wwFZn zy#-fr?@j0`0?xSNR35YSCSYM>m*D&uMEl=HE`mKxSr=uMuU@Lk7h8^z*G)F`r;80= zDMv3=sCCs)Ha>I5-1W4~V4g`1#BoBV%hFdr`W_BVb318u{N6fA>S>vzQ+H4gbxL=s z?W$Qbf?_bvT=*WfX0}5gsj)VuhAOSdE|PB(&wGRIP@X7&<0~Pc@g(+97Pj{Ky2^!x zqZEPAMATg=$a=*P?F>F@KX)ghrkIWypgx_JWxYW);qz1JFqc*<&qXQQhj}Q~8Ayl; zu8XZ-q>{Zy^l?)HX(i$Bzo{qE$O1WNv5DuIS2ZgZr4MPvel$}hmu;e!ywa|&tJz;y z8p+r-$@n2_;~zT(xpiJ^NKwmpa88yn*xgYzXOIygDv3K0?Ig|#}V$WUt85tLQ z{+3k|_hfPV>#e$(_(kL@9qT0ktZiWLId4bJYw-#LI+SJr5OGOkqw4WoA6-QjTM<}4 znRylBShvlqchZ!f?5j6dZ2mH2STNlH+gXMEwTPUioB#Jg$JXANl%A;yB&4vy7A8w# zBhyGPPCKEFyB(N3GO}H~`u=<``!K(=4ZG{Kt*a(-} z`1@1zzy;P@wz5b=FPoh3o>3neI^L8xh^k3El2DKN%VXg;T|T==fO$m9=3{rGZBhFP zmhxh`LsZ9n$H=#b=O!wN1DL{driZRNW0*HVT18r*o=F2zpgK-@bO!mVjNMz^-AxQ7 zp+iK1Hs^kamJr7U&c{h8*P;P)?v1Y~_fd1HIB^Ou-2mnDmy7;av))Ki{LUnQMc;MK`XZa7X!R!d$VtsLe@!2#iB5n^ z>^wtPILk7zuJ}<6!>a&HfiJ8~Mwh!)hFM0Uewt%8K48WprjO8cY6eDzvcNDKDjL*v#g?ICH(f&vsZjSEmU6QhQIg|Qd?q;G; za~LYXg&KwK)q9SS58F62=J2Z2afu69Fe$WYRzJcq*p=Mjxv;wT<0?(a$z+7jqkqj8?MkiN(t+|^Oe=jd{#~4B3aL_KT*Cgw;DZIAI zcCm&o8QeCt*54Oxt*DLKf|68+Rqq(Zk;E<`_j#6mJF(uJF~#nHpf(fT@HFrN$_^?T z>Q!?#(N?O%Ap;tF4B0ahvGxlMtUGWkLznv?!yxmLkS#IK3NXX(mr+DR=18j(fc(m< z1W;9!gu~CG297gCP>@AZc|A;23=kHy72`S_e(KdjkWPyw7{($3Y+)iSqCkm0+1y@a zI46&OZ+Jc()VH9V{tt6~UbWBudDx=BY}hh6w*Q?20LpyKDBkhIH_7p?kVeofZiA0Y zYrY}?m_5dB5?U;BR2^Ft9c{w{^~Xkg1a8c4v|#$Q1_@e!TuTcwJY~Lq3%zi<+E(2h z@Lege*fKFR6PMkk=xU?ndFV|Ynl}=PZ7K>e=?SieHdmonE5fN5TDT6;9|FvoiL%>& zNg$w8%j%KHer)oU6oLv(0*gV)bQ%sE!j(FhpO)X-4+X5{2MHX>3iXA}@fexa?D0Jn zUZI^CA{oAIsCPyjoma-u{s5|4&^Hbh@j=ed@lgF%ZL3rM@S)O`-8;U&- zNxFn7&<&vHQv;Z#*~9p(PFw?Qr5Q0WHM>^Y@5C%`1{=THSP4)xP_CSMy_Q7^`dJIL zp4GOQ!9Tk%zTK7d>JSH>xLKXt zdky)Ep~xJSvD3ax4p!g;e4bHtorOi%G^pb%)X=QWO(&O`b%CozI8+{N2}6K zC7j;lCw)d6C0m)eiF$xg{v^vOa7PXYIxMxug=462?Z=^c7A)9=!DHSSd~8MVIpkVZ z`I@o-rE00B7VYVq$ipO*l9C2)1)Fnym6W<$83J1p&ClER!rK?J$1IJudgLG))tQ&c z7)npB1$}3U3;BM9J0`x@tGHzgs&yc#$>~BLh8yJu^+6Ww3x{P4tr|3yht$HAlIz_j z@|EK)u^_YQ7Y=$?VGHCSQp2q!XF$f_fJJr&0B+hpA$O7@myh4I&#+N8PX2P#WQ!pk zh}9ai;XC#)HbJmg8MH8ylNJQCqQIy725 z11w65AaAUgim27c_3DP5cvDtc_SsciHpGW?z(6f#G^(UtR!@Qj5eVKc#gHA__scGlY}^Td45`4dHqz6H~dX z!zq}j`l)a}g4J=>-`B(de_^qPb>6D`q?7(ep`{=5$OCm&n0+~l<}hxf+CzIT~@|#hhh`DE-6m_32ac|F|HR>iv?*?)EmCR-*oq*!sx~ zn#|Eop$>2?Zt+K&d4Ka_VFi|A!c*-BsMGb?6v%jlj+4YM zlE00#tDeqbh-jGZGpIub93s!%dt@leCrtP_-#&*z=)`G`Oteg9t6(m}@!;oIwxst& zz$!(;Kz5ka$H`wG-GZD)TVRcbw~iopKwg>xnU`#BZd>beA*697^(aPWD76j_qB({^ zZ%fk`xjn55-xVMaud0$@%KBxa%g%1=)|iLHW=Y8&!jxN{5UN+pt#`!A@==>*+`7e& z=S!xmaIoPf3#kyHKH!WqOkX;%i_FF2{;4b(3x1nHV?CW3&0$tr5>j2ix4F=^d|5SM_m%J`hGO13_T@4LO$ z+VIHzF{-d)vEs_pHJ^=cI|H|gaSGPG_tciai>Px+m`&kmb<3jswW?);fMRx=+P;^w zPGhvE@J^W=yRQrb_LY_;O5|gBxjWFxxET`3aS%U_y>Z5!X?0KmT{xSCPQfR6Qz=>N z0{7hMb?(0vMVd0wCTFgMo^9V_JYl>bJFZZ8KRaBvYX=amyzwIq{YCcx zN?A!?u{!SnI^kZz7C)eo>7$wDcWe><=Y8^v~H(*S17x7 zs*dW4xhUIWBOTSpZ#tsCZ{eTAFp5S|pbMy&&pBqnZo-(6|JE9x0_ zzi`vcj=eLGTz9>NSsl|!U4Ebqv@Hos>Vh8e)M_!;1fz2X-5_K1K{araDB!M_1H@~i zzOT?GMujZOtT_`V>u z`Hs97Id>$^vmMkP^i4stm4UK(x$bc8Xn4N@HNPkAiUcTo0h}TTE;J9QP1(HsEae&% z3Q4Y3ZyBkFS7{2%pcTl6fQBoqUTe@)YI(k}bSbdAqmLBIulP_N=s+FA++fvc97g{e zR-R$f-7K#C8-UWq15KI=-tJ>wdO~L=NI|p`7XGxWw4~$HD5g0@$gPA$#*V!420f{d zaW*^KTAbK47l}v*Ya9ae%ezGLg8n&FwlFQN;WBi_RmK+xDgVmS!~(M2Drg8748s6u zZjxnfabqwBB7ncIzALS`4EqOdFBuY0NxVf$14r3N{aAubHHtNv3f>4~#$%x)YZy*A z!LC8HL;&gN8e~`=L%nerp~UiU^yIv6_hWpPt1Q#FCNz-cS(d}}mc z)-+hiM(%q!3U?_yQA!*w%7p4$#qG6YAktt;oY?tp=2n)jF8#cQZG)_Wfkv#(w&nBx zFa3Zlm*wg%RKCuN@(V;ERDa=g!(JR7WpP*ehZLw=hD0VB?dE&ba&Q!L79W&k{}_2_ z3@3+erd0l%o1gIuInGh435A^7oF^lxlb-%OnI?zO z{GU@#tW6XXb}^j`{MWVi)5dmga-R_kneqTNCYZ8DTh29TL{7k;nGw{ zbs4UoBxV95uybK!7@LI3R_jY}c=VOZ`%+#=D=G6}@LXH(XiX zOpq8%n$IJ9mHK!zLLf28wvLpAuze6O)Mbk>2=0T&s_)*ZDajh(Mfw4WMk3ovXi6vJq39 zFfpd>>m**k4iA|SAT5DNz+0CIN8=DBDoY#MJ#!I_O4TSear=7idVvO+zB`j-_(wRm zmqnOf%)z>UFWy0&|J?BoR71I6R8JO|(ohj1?jI~P6`%qO;+{>;nSJ)G-FA92=$%>=ZtKLoAWa=Wb*{Qcdgu*2jac;M1Xg>NbF276Av&(@haxhLq-V0! zJs{l>>Zv?I1i=oFzxL>6G2%GcMk{$_{}g_+C!N%ObE6>8_FYpO!JremOp$Qik-<`}l~^pu5HB z^Z?RW{l=#^rlFzqyafW!ZF|FXIgRyt#9*}l>uW}dK{~I{s!rROGzEWsZDxC%_=agH zsUifHo3O}>oeAZFL~JVyb$@$&8+{9Q$;5aXfiho-3kEj^vA*{P2lYEX8`!|&K3gMj z8j3Uai|)5QM~A>8ADEKvpU(fRwT$^;<3p(@sv;Zx2BN0(vbR}2HYnfxkRyNS9e_y- z<5;J8>aCZ+>{aPP2S8?(m{Trl-dW7-@`y4dFa=u#opZ>O_m4U52_Mr;fNBTp?92dE z)mut!7dvwkt(T-T`=qETSbkuz_M~VGJ2n{H{mvA!)q8JE3DtU0)T2WXY7|q&F~}`9 z*aC12r9qOQq6KWYypl%{zROS;A30J_I<>->+WY&n^FIb(L zA@UzAyh{PmpYh97D%|@feC3Gj^=!gBm^RN}%T!DBx$?&TjFvJKjsoD~{Z|p-#VE0U z-eMaEE8N=IvTlR)3ZQ*I=aC1@ap}(G2a19tRZ5{ZnH zvxPd}6seA~&VOfUywQS0p1~}sx}xaMHx$Xle}w|H&^bEcBC$?$VJ^UD-YzrJD?SsG z0{EvmbpAoip7FW$$~{(Id=|2I$q#Yp>T8pgs1f{FG5i3fCeV?XSk3$28F`NZq0IoB za;7+q9VXGW?u0Z4_-I^lfN{&&=q)5%Tj;Oh^XEp5MrWQy)Au&HF$*M84i-PocZhgd zL<8Ej|6j;WOe=GX!lFkfS5zV<=^tF~WTu7f@$Nk54Mf?;did7DfE?_tSlD6s3mOk| z6s>kb{8(QeahrG)TMjhEdEF-GLE=%E`|0YusXp`8l7094)=592AI8Ux>FSAhlsbv{ zun!QvoE4)FrVZ?XRXA#RsFCh)e6U7|0F@iC4lfLJ?obC-I zd*@C8V@NY9FV4y0sA)CX8Yr#PaQJc&!bO^Ou4}{Vz&|}ordqlS{&XINCq7~>hZB`y z@q^~3wCyJ!SK_rHs+MzVWPQC$6eZMOz!vyW=TG_+s?mCQjPq(o*;r6+n8D^evdlF}l=x z#q7Vp?w3m4AP{szNaqAcXL#ug!`!HcArrV1l7A@}$$d*9Pw}xkFiBEaN$nZIR*~#s zrhs05os=ILzV|BwMhQm0DJa<*u2<-x2nb?-$RthcURwa)>4-=A zTBwlX=j%Q3h{5yW(KGv#J1{XMDo%)uL7h@q0>FkiT@r-sdq!9MTEKrr$glxqT(+Nl zI~i8?l8zch8ev$9RN!HtdhEAxwkZfD*T_q^TDn&+yFz&Ss4E3a_p_Qd%^)K5N+4n& zMuL|NLkBxXX6?4$DB=;)CUV8x=KC>$3-@mw>@Z~-D}<0wwCfz1VIcoSa|k*nZQnQ> zzj=ZjqSsGCDGWH@X(~!bWBmB;19y?T&0Fy4>S(P^)}`#?^&mOBICfO=^-pUz=Lhz- zqbYAsq6I;GY_`vrbiL(Z;=V0~Q5||$gwsv2r_!wCF>VTFH??qf=_xveC^JTxp?K8- z6OUsNvH8DaVQWhm(isX#K865KCKxRo`_*3D+YlS}oulXI`9v*FhjSpFZ3lF$?oD@` zg&4w%1SasNtOIb}aoJjWVZdimIc&5I38C+e4Y>W4@B_R(51ey79XC89w9*vGp5gq? zBneiL5KRQj!~+4sZ?ajsWqEP_wv)4X8~R*iv=vf zj&R$OTFKrd0rDMFQnDP}CD|UDm6qz1^4{H3I^s^m*D#1c91EKScN08oLQY`I?L3QJ zW5)QYcF#MtcIC7+pIfhl#M`0!s*+F{Bn&fXx%fpm8G$<+5a$$xc$WsP=E_5pcbrn2 zXtfd}5svZ(Wb@NN0Ll^p*yjP~EGTaiBo&ZiB{voIt)7w9zf~vGt|}#8k?KYKDX}fE zWr}7aL?SZu0nA&|6!1yb=Rc>k`6!n2QFY_MhuriRcBlfY7ip~fOPH`80Eu@5T1f5k z+E*<<noMj<_QksRR|_B%34ms@+K7~e%8_2;Ov`jL!a|Kefe2vV;hwd*d7-N>w&XaV5fhIN3qGmEgKBYn~?m_b|Kmq3NHR$Md3m1Y=qNmNCADy8= zOCiYkvesWRC?n5>l-Y)THXapbb*ko;fi}kU)|sNq`XSddgBvAb!X@#GO4i#0TYrix z_-OUHeZxYCg#ZZq*%_0~2R+qEB58IrdsXCC6&J0md%7lzkO@6f*{s2jO_;-65SCyxYA}F zrf8>1fKp15Za+G;?VZFZJ;Jli2mBUDD4)fFx=u5I*8Y2i#7JBD*hm- zHiw_j@8`$I)OuT6TNkZu227U!fo0Ujd3u29LhPUoK+rTJ^g@AO8pr%9L`fGB7I9?q0CPqx(=Ef3w3wjlTWfjXDB>;|`G%Nds%(l=YsF&Di137lZa zQf~eO^boMoEt!Yv0kW?m$!94=%uN$ugMggzu$f}xj0ji{Hc2sTZs;ZY51XUIHpb~* z6)|2CGqk+05`SAcx-vYpX30Q)=>p!Ts5P0&_tgiL#i`Lp)M`nxPxXqYhfA$LqKNO_ zTbf;2|I|GX;Rjn1CtO3_3j{@#5Td+X`Ym)@;O8Fk2$5)b*Qr)=3^^^oFZixlEu_P^(wvwlh!p|!21 zeTz9%XO*Q)T-vnNuDn(2QS7|#zaLi9xdx@%{I*NeY!QbD3-ek|Z^&Ap_}=&1w)DbX z2GxY%dabilfPBMaLM8>uQ4`gVW)z?gy3y-zcV@;y_7Nee_&%Z2fMD(SApN1U-JG2F z7TBulqO;*r_13?w{@Lsvu6dLCMJZ59%ScA?oHbwazP%=R9jQy;M^v&+svc355W2_L zOt&QXu?5A3N_z#u_b7X79Wk&9=IbGnK%_%o%aM}*G|O)?Yi~M+s+RyNXR)+4h{Ku> z`_;+3darM@huEI2yKDaIVrnMfbUdvE3xUu3OU_nvlG%3YxX0)=US`*OsTpqhKa#4t z0hCC-T0ku(F=X4u{CS2Z*;)W@$hj7~LOls>-VE-l@c`)(1Xj2*et6`g?6E{YXso0s zhQyZ;{nR*<>vdbpTffLA>`x&VOc1OtzWCw^wDjphRO8Gz{CVhBk?(t4Azs;#%r9p` zswigrV!+e(ZHqi2b*=0!EJn0#9J03?yLtdl7J;?{-pg94cI)QG+6HyAk#CIrPvfNq z9w?{-ytt?e^_?~rz;ZJqC+Fl^f!JJzbRWPvvKaz{W-B=mlQFG(0kcZ*d5ws&zKJS8 z@Ej0n=l9_IB!N0=LQohQ%4S!4)PP`J009;B@w+Mn%?^O`H^<>(VpWcH3FurYL_z4+7lPQ+P= z>))4CqT#KrFy?0yG=>f4G#@$9K41QV68*MLrX;TiV?2w%`1<8o>SyX4GJ#PaKX+xATt=RWI0 zunqIML$+KYJKU?(Ez;3^y?oo&9bSw^v0v=|_YxYAx&db{vh~Z!n;gk@L-ggYTbsrP zfQ1%Lf5* z14D@>Je8EP+&m8If zFP~v&{4#JxehND#^*gBFtp-3cnuHa`qb%qtj-|R=RA!D%T;E(9-+W^kOb`uSLY4ClyAxNNA+KzY@v5dIjd1-Pf%~YlA zki9$rQbGyIl&FnCH2j#(lIqR6DFzjCh+0eP#6%1FLa4!k)aHkH(j6a`p;{VhOa_a! z;|Y~1OON~LZ*Iuf6ox#>Hx*_J4Ta@c@O5V-0z9P1|9nUZeVEs*ee_(e>)xb9c4ba@ zi1zFH<$ePx2CV;;zeGDa$7P;DO6dpB#*mZXR327ZBbng((CcS>y0fX=6@Ld?rv?{$ zofdTffdy4(oq3P2bQahr7T!isjdd4Bjpk&F9I!NxPNu~hgbhiDCfT_E=MG~9d9Pfg_^d}S1ZSO1dS3%?q)S{-dgNn7 zb`&>8Dn+|*V6lG8&m(!f6?7G8L*TB!ygMy4oF}b2;^+GO zA|grne@Jpc=7mV0+|F1cYka;@Ka3f|)%BX#%ua&^30HFQLCzx?@k$IHXA~=x7s{${Jp?sliylh0Q%6BxTQ9)>{|p$ zqo=%|Dx4{I&?eUyTmah#}{RNn<%?avx-rW{#ZA~A#{H$8w<>3K-|L+K{)@U+F z;BdFhpzSn#ba3()8R6Q`;brwQUR07Cz7K8qLcNwx;7QQRf}b6P4Hh7_k^7 zi?)X}2hj-@Jlyekd_xL4oDkeOFG{i7jrm-VUnrfwa~Pq_?h>-gZKBPPkEcGn5>su( zbf_-f7J{kCK|j|e6H&cH4eqb;cCgQuI}s&id9C!lLq-0N50J=NSq33d=VMD4U>g}_ zky^_0Olp1N91%<2z9y7R9>v?Uusve<*kDGSED?DQpSs_4CXm?&OtCb1C2fK})eDbJ zfEuWsZn?s;!3iKSBQcyCu+>RAIZD{&`w`bcfgEgQ#iK-0&x|Jdc2Wk{%*fFoEE0kQ z=1@k%`N=*5w%9!WQ=#$<ynW=OKl~8cS^-8o*rmL zE!imqssD@Hc87K5%92EXzz}{8LsgCpnOIq93q;D4S1Ci zLt#YS;LACJHnXDUok#N_dC18QY6jeco>8LaqQsY&9D{a5J*s@Y*&#Ort_8&>IaXn_ z3_*!!`yCIhF6osPY!QL}3UD+~#(6_UW^t3?RS zByFP9yT~QUmj194^BQw>8vC5j4ZUK6t?`Nv`fx)u$U%2e?eGvA)sYP9&*M z;MYZf;^@E!KiT|eKT*_31Ayg5`}6>4p}jNqyyc`Tw?WN3wsZjQ+kDCV&@R{{dbBIiz{I_IN&OMQNeWp_G7OM3@ISj2Dn-V4p6V)g zTW+5agiQ%K9NYE|icDpfP<;4VtJxZahg4_N4)U<2M_%e3WiPYdEF36Oq%+**Qetiq z#OIrYt4v}D6jXK=062~)p!kPvJl03{3eOKEUTe4lj{R~?6QZSTS|1s@;01>_(76#O z3j`d4+HWAX4IZj+y3+O@R+_ndX3FKeG;?G6g-HzHnCRZtR!xXEuzn8}uHEb-^+cjB zfGlAEPrXwRu^rpbFjO<=E7IyrDfZ;u09*9mL%vX7W#l>p^m6y)xnaKM|x_i=A8 z9h@`!l!xrGU*hC#>oq*lzyO1!iz_pZImPU$puw!8g2y++pyMN<0Zea0k#>+Jv;_T- zE9*$C`pYAi)0Q5VoaYh4I|84Y&J%NpEcM?3vK;Q6t}UA(-q_rt3k2)qfs2*NMI-8Q zm=$=-GQX+nVv>8Zt*94p5zl3HwI*C4WU5q0!MP3PNg=)crNA*%u&#yw6t8d0x*&-Z zOmN;XbZ*Z8E$`!tJsr}z_1og0rj(;&k+Co>CF9IZ3EBH`bg8uf&zcZ8RP-zGt%X5P zs>=+81Z*yasPqiNzf<&N1RKK=#S*w~iMxl2mo3{JIf9@V=vF~nUTt*PQ%Nn%>qhQP z3}S}HPM5Ttj&ANi{<9l!nQu~$OG*M72n4-&VQL%lEb9@L9Vp#SmC0MqIaxAI>mzH! zF>!rX4mWh%GdRTmGy0AMhn$0LEBHxj_Y1>*Pmk7NrV-at`8vcH^w&sB7BVJxJx?eX zh?xri1m1kUr=02KmfTT}$5p2*V39V5Rr_Zqwv8v-Xqh2VtJzbJ|--zk|Y8bGMkqI^@8N!_TKOHW<)rhBI z`1)uRD;kN4?#apr#Mb<@haB3M z!)xnTmRy2CFXPaIPPBokO#9xbW39{QD^y{jzJJKIl9i=gnk?g^D<`2eDi;fdm3nAQ z;h8To3|N(0fk8D>1EP!4hS&Q5Si-G{x@LhC*{61(#IF-odirR7w=L|Hd3G7oqB#!R z;s0ng0-LQvI+a8+q#B!%&;nmu$OSbRs2X^SKMw=fmB>ml0s^n_J2?KnCF_Nn^@5I- z^9TbyMoP=}&?F--%jjR(63ur(F0^44dLi!gKyI2TS+pa49{7RKl_wEc#ciP%_mYM={2yjA~8H;r3nE6NWS(;Bq&1a&{$$-u!=i2);b;68@Q;IO{)NYM+c`k*Ttbc&N8_(C3QhwMg<0I zyy+^_sv`(hC7BRn#l+n7lH1>{8(fOGJeVj?cJ%nqS)At>BaTnFYv+4#_dHP&p{Jn? zVR?`&VS^<>PmI%t%uX$yWHczRb@iDJ+58N_EXB>Hz4Qx|jq{$ker!PDbcVIX`Og+@ zV1sdP1cdxd^hO7*km{Tpgrtg^YDz;?t|x|Rd`9tL6UQ-m)|_@UOXup+_eRx3SR#{- zQ8mPnXjSIh9`Ln{aO!>x(OMuNE+1)=0bJZWORLK`f^5lE7lzrsNdbcjPB0b1C2Fz0 zMc!b{*cUowBPNv@Pk8>|8^6Bwq9_?N!_q#}qFZK7%=^|Y>)NrnlbdCDo)Fg@$t!LS z_cYKad?(+59^;nTeD=o3vbp|9(^j{Z0O8kaqE4MY{OChdy~%QyUjsaLa^YZ zN|aMy(?+{^*5$sYjv5Tv9A2e7Xnu9{k^nzIz`sO#n#e&|3o>(2$iXFvJrJBJoe(Vr zIiiTtK}l-E`tZi^V+<->eVuh6k$m{YHd=i;Q#k=ssr*xmMa8)7^=!w?uGa<}MFF)w z%P7)cDLCoIV{1T)$H9lP8QPV+EfgH}5|WN7$y@f%%dj7D4HVmD!R4ZPnH!aaxfRbv z5^k$RtU@8K{?j(k>8yC-=e&NJu?%k`XJ1Oxgs2<)*d$imNIAe{V+-}koKrinM;Mwp>;*jSy|cD4yPTmJFYz3+E; z&88m`B^-uU$$;$99EEJ9z$Oj0Hs1n-b2?RQsUfh<3CUl$KX<l zkg8K5B7#kp0#VSHV5?{YowoG&C8Q8?6y-2N=?Htap4%e;t0RLZC51;@N4uvK$0fQ_ zF2OEKMIy{UQu>h>OfzfIi7<*MkAz9)*l$gn$=*@MzGNpbYRYZyG^{X0;c3R~=7u8P zrzN7ynS#(?@gYD<&_r(lI_>(@sqS(9Ij>)Xlsr7x8x|fMa3j1f^O_(l`1m>&si=B% zkJ|gOy6<3~-CA9uY~eCmc+QC&q0wwgsv}x)Lbjnf&~#loW_Lrz52~0_@$Eu~`rlgL zx;O+4PKJRL%s1xOl6i zaPi>OeXyg8WMgcDK2^loY?A?mXr`@tU?1Qe?GW-h?50UB9_;W_Y$d(XOXE3p}Z9gP=QsCuHP#{lP zZ8WV%CU);d#MC2W9nMj@Sha&lbaN)8^%d0hvMSMmIjA&QcO4Z1U!cqj0d;DTE!TPW z5{in*MB=jt`#j!#33=3Rcfih(<8KCj-nwwHf(0zsM z%e|%-a-7=q$YWuvL{UMIheDhLKgDPx{yoJH-mD^Y*1UUK)$}LvDzcD+cxBvZ@3sTLsOs(%d5?2b$!+XbCC4NkzwD<@$-{D zTM;jmYd!L*5rLJBJEZ8dNf!CbMAL7uYy6lV2ws$38z}}*32>lE%HkyvOP3w|Fe_j` z+P|-SkLp_UjA$o^#(L>WNSaq$-f!heBkH$GD8Dz;{?09Y_-6c$yh{4LR{yJ1i$TLq zx3rF4bpd!SjorxIJ)%8WuAdv>h zJ9d<+Q`nx6T&1M)dsDt;V3-58gWe$T17*{?j~Y^t^2*Ae9a3Hf_?G5$a8aJyB%kdb z4~9-HU}jp_K3QJ9in|3%q{n*_NrqAG04pGk`k?r^-T4R_Ue?=_NN%J~`%b2I*Zd>^ zk>rt{qGAF8U1*SdIJbo=6_jI-_8JfU^7^`@nN-?q>CetDxKMRFdlaJT6Fq48T<0y4 z5)&s-vG2$?1C?mP^AOTFuQ07gvGT#6n;Ng;*lE4^K2eXMZfbOG=kYHsAKu)e$OFDH zxqEOk6=rj|*^yA?Do2DLtJ5ael6#qX2@QRuoG3k|I;(a~s%I*zy!0^jm=HYmh91uQ zxAR6G9~}`tImkLWJgDMwoKUJrok7Kk3-d!Jy!cS*V8S<7ZX9=dnn6u6v*7B_k;vM= z3Q0?O8j?!0KNi%Macv(nnX3I3Iz)&}S`jCD-hEC8HNF*5;S99Yf+VUdJaC-tJsv@c z$b^&19*sbXZzz~Z`@4b=e-0z=@+}ZIZNFnmI7gePSk3W?D9_8*#~*4dn575=TzM1; zE)ff`Zz0j)2iVm;)XICnnNN!KBvlGb*IP!1JzcsP^Z(R!FNhkhMK|hk7M5uE55q9; z83Ic)Ya7Lg4b(yUl5SULe!w0}y`M?ZXKNqgbj(IG(^wJtHie8emx;R|9+me#2+|U- z2+X9P%i5$|%)l|pr7NacI>XMGU{JN1B%=^P8g7oFF(bj~?(Xv@1{_9RVY^ff3pYpj zXPhfX9G8E758myFT33HRYECII8 z!X2^E6Hok}v@>1|#b$i*pzYbHe%csRIR_w7LVTaga4UYAO`i;<+XT?zo9lps3 zk??a(TH4q3c9NE&Z)*~7D3PyJV;0ST5b`Qd?FVyRr+=|EzgAMv(^zIUgzD%Ona!?@ zXGxb?ipet>m$Z!bb$eR9C4)c;{n=Q~w~-4$dYtXk!@Vw^nj0(K!-l3XlB40@1HY_u zZ=5f{+G~Yzqf9>+lt+ISMlteCHA`G;LzyZD5N%pM z683%+?I#@7LuGns58HE}Mt|!JNQ$E#6kV+#?ojZrhH2;9O%gA2Ziqpu%yuO=(A4$l zY=U&@7Ba4}#&n`?Zsqm|exOAn32~c61cr2CBpmukUi8v=$}uExm)BZQ*XQI;essag zfEcS~R;R~ea#KHAFY%rb4+Op&}jD1AX7u{y|5>yNc`{lMI)SW}1DLxxOxJd)Oi|bEH5ujVvW-v=#iQRSZ zZN7W`hlTOAVWcoS7X*qcOh0SexcIos*;{4m;xBO%(KU`PpJP?bnt7qp+3Ro;f3$rR z^2RBl=$LtiA*TGRA*2`4Ew=}rXEJ}c5N^>lHh|2jAMFy^Hr^sW zufEU{(YGi-ir5&H9`znU`g&?zwr5X)ldbmbT90f}XUQqnPJ~!;!X~C->XB>2Jmb?3V9tvG!3DxIvJ;#2G-}%jHKtZFo*zzu zWfIOnqTa-Dg!j3dfZBdMu{Svb)rWCYLnHf2WTwR9m`_|bz%r;?xGB-^T2|f#Y;k`I z1rf@9Nn|e0Hqpr;-}QR63<RQ7VLG4#$j;k8O6(>%lQW$dulfs?h7L-mr#%q>&(%+`?yR$zY2ZT`ZvF?4nUZ40zV0Ac4K8#R0CAVtKFGo zF5gE(aRkroQVPPobU)29bY;#gaUmVfZU@`c&*ie=!glD>qksSP4U{3%>p-{Onykff z5@%x&L<73xa`v3CN3n}AqW-`D29JDxN&OW}XxNLF~ zDg9o{9`uqHMtYJiUu|!7111Y`KQT$3!|jx9BZYpQ)Bfkn=Y;DF3HhYM&JLr5u&A(= z?4(`K^MfAC<`h7f`k^O89c^Js0vK|wjDH`8TKB9~FkRG0$f8;mYHOSh`VcgpbWKPU z!;chQ*~XV9?*_8vM`0SfUuHXZ0ST5FtT>$Xa_*wK#zN}$QG8sgQGzlB2x8A_5x!Pd z4aPL{y$A_}hodWx5CV#Xfs0pJD?M}cm8}J7BgR{D8A*dBlBDD{WTj`^0tpT!Pw^t` z1y9Ly5AnxX6cRk=kw~*3$VI&H>20C0M0{>9%05h_u;sbNf>f$i7@-Od)CoikMst-# zK5+T6bFwog%YdZ%Ipewrd8M!}RgW{{dhr_w$iqz^!pAPCJfsDLuq`9G*$B-bffQ{ZKhvF?nkkMsw7Wb^C?sp!wQ($Had{dqk>` zccZQ?Xir;|Y_+NdlT8>owM&e1{Mu#Pqk~x=$xPC6IPTN(p~7%wG3&K;^U!?AwIU4J z*3$GVKl@2gDa#-`>PNs=7)Ol9x~X-qH=T(f8dHI>;*kI%0eA^&5e_os$fXDiD1x*N zG~vJ6!PCDUr3}eOc7SkkBX#gQTsg(b;Q+iyMQ4REMqZ4d;Gb?!V~fZ!R<#@DJgC#o zqF!>#_uFB%i{C|oFg>!NJ5*UxPGY)9P0$;o%ll^wA}5jj`w8NO;EwAhy!y@QEAQ|> zaD7iFf{0xHlVnkH!?4pNMmy)wYXAf~$}>u+!8>4$xaC%DrRqYumlL3fU3{}RHI4Dm z;aPG$#>-xd++{#=NvC^U>E;CD3|UYdZ>7=|lhGsfD?R6X0YqnS6AP>dJrm!^QRNUV z=PbN3t(@{;_dgkZ{1MXoj<283&Lf?k4C8SfG--U&LE2@9a&$zOf)`hnR@Uic*xukd zFQUponPxah5CnyjXzIs4n`SUr@+*reA(8V)aPFunvEo2~7|IXo9lN-Lef#^2y%yN- zRj13^5;5~BosEkN5nv3Nz_gq@5s`l-9@jW!gzNidfHRMl4g{e;!L1YjcI!LTB0paj z#pZZ?fnV#d&u$<_Z`ggjBQEr`&=itAU7}^1%$uB!9IGmq2+|4sHBfLdm7ixBa@xit zEo)VJAu9OZC9?21c2 z>T;bCWpgBaWSfLYNKE)%24ap zZ|Pc-n`YKC^-aus3ZBX+9Tg+{VfZq$MI=5jjN9-aOHa;my!cz`BmXuVhc(` z%3c2WU`TTRWGkVE%K|A(T2s_M1j;LZ*%?_A5h832wc~o56+zUz```}-BvpO06{9E9 zgcm->27hc=$A$QtcU+&8CFmlzirYGlCb}0vJJJw3m|CV%AIh5mb$bl^H2cf?T2`WY zK87u142nY!`bRDlf!|B8|85PEL8bQRM>gT|23jwHg^@$y2^FOMmDjW=^ zPO3`>O-?}+-iB-SM&#-us))CYp1n&Cdi4nH)#c5W7@?1y!kE<`o}gfH^6ePMFAqD^ zM_H`zfXar-#$Uo_YW?#ad9XWn&jrs8N|X2D^fWSeSeo)h2mFNuqj015T)+|!s?!jj zTk6e!yK^1IHlmQTp^es#Qk-GZZhsdcXH9A09SN5}oX~TtqTrq#eGcJV9~PT7vARDf zD|Q1ixE3`P9(a_Y49M6-Vd$TMXZMNAnfZX59FFE73&9erK5?Dnh4%(%Zt@ zGOX}TA_Ogqc9(=R9T?6xKjx)(Yay7Tq0zquR515IH9`-8Cgu>Ove|zhdLpv`l}1oD z%=(Y8RjSnkqp1@q;=#w}a6Tgh>Szgz>Jv7lgfIp)T~#OuJS1|_YLu@M!MF~O2ip0- zo|>25VzCl>|8efOsq{kd+l6uG@Q&4GD7hr}lM*Q50=Q00&t!*o=TG|`)nJPDyD>U! zu~8}*?jQ^!p|8qn-h6{9l%vI59#;6Q;?pC^dL7GCgdd|~$8 z=}F#v^uWEWzut@>h*|~LUf5Q^wmUc+fvl~b5Duw6sCpU*LfWw(j;h&*fVsknP{GXF zmLiEF%tt%3n7ON(kyr4okOy4KF$17g%gRaUH@iFXLkx&At95tMNGFL0{+*L?%J@!s{1}Zxh4lrrz!(@ z8$IJl9f^Mo_ol!E%9hzH5#?717_a!}zXBrv;0y*Z`C_r+Z<@<@{54%qqHxcPZKhr% z+Er|&yKunzH8uDC#k^Cmg$ya0_<+Kxt?O-%9dxiDGt-Juj}2V5eVppVvNNzHh}x;J z1|aaNTXPyTs3X_aWd*7vyL^8sy^V@e@E?(rS&vTcYBc%w5^_Okx{CHMj7JQnL7$DFpm_n`OhQe9(U-Pn$YfO$t#Xc;RUS#^*9YqWSgg4k;y(v1$`KCDBpVg%rR zvXoTwW)j;#6|hh{48p+*vL7ZhD^-F1;$xi++2L}^3ZlKhp1aF}jIK}q$%^mYrybn7 zV5Qk0s`_jL3=T1{T%L9Hb3QpcnN{?Q7Gu}K=fB0#shB#q8UgFj(T%-RPRST0ODk*T zm;wJHN;vG)@#zCGy)VD`A?t`!*c2~+CnMb3$QrLX!+ci{z6v{r%yA;k7<-Ya&RC&d z<G(qE&pG1>`zHsIiD)Z#93-ce|UnU|o_3`TmXB*nPTefgB37 zMhx5x%2nc6>`E?OBIn9Qb{`41FFHpPXwr|9brQwbE1MG6;%(NlJX3(+~!UzkwPD60R_=9DeRh_2N6M;?C{{y0PhOuZK7%OHyB;i-+%8W2lhCo6y9?UERU~@BO!T4MzfOZn&Q%^ zZvu=F9KySX?m2vEo2*d}l1OCaxseiI*~WV9ww=qP%x{3gD7+>tikCayf2V0H5HM=x zp=Fn=Q6x2t*DCBD1ou{;xdG}lT?6W-i__B3$d%Rp>bYWjmyAD8gs2&U^04V*ss=}{ zY{5N58PpH|n+}~B*b^xYft9W1lwyU>Sf!VTk$ZmTo~U<|ZH=2nVjMh;(1LA7IWS`gB{(B3hg_nf@LymhMpn`-W*Phstoj@r;vNhH+n z2%pb_kv;td9Mo!Eb-_4ia~m$vMs{!h%u+vGSgb-XuI?DYDbsrSLAU2mf#dJG`^5K=;L-(IYErHN$%(GN6$BW`O8 z2rFd@^$R4oKwIIa`NLNEu6iD{LQCFj#^oDrG2VLso$gap*_7$^*Cw8g zLgKJs*Nlz~hRpho@Pg}0W050TEYPDgSY&T$`JB_95`A%_OFUXG^HL{(%5K4+HYY9hRMDQQ(!V_tA9cd;k#JGd&GkuEk6lqekuE#;T+L z%El7EdPy7Bpp&_d-RFAubm(kRD3tu6x45y8Pef?Ss>=58;c}4R*|4c-3j`2@?ACS+ znY!)v9T)&RC_((sTAA;G-J9IAXtRp{CuvZ91B7+)`u~yABToOnomhvmH6*9o&)(Jg zO4T)Fwtg}5GW3gIyHgPq4F!|-Du*G$G>*Cc2X`IJI{qyz5zY7t15yO7S+t1Zr=-Ew2) z%TIB?UQvp_5YF)!9qk&j>X26Bi-79hKpG~$K3TRP3k`6`0}LE;wz@1?`>J>Xf6kCaraa ziDEadl%>ZM)hlPBGxYV(AMt~;k*xvoegKhJUH+yL=$0BwzxLK|tTSN3OeDqs|9uu>+m_ z^C7;CIxAS{hJJVVeQnVoYucM=AMFnS4t6$_^0yvv$&+xUL)|dfJ^rv2o#2q>*a@eo zwOm>-N}qP&`G{rwT3(anRY9EHXP6j>tgU`JJ&uUG#019dfz>neJ@;9>nvR2&4 zKpUC1-_yV1V&@z;>}QV9rOd)FZF6R`&ZkHC2uMdv)Qpq0_rjYSJQ1axn533;g59vX z4he>CjjWw5@SaDCN3NG4Ik_Wq9E`xP<&oSr_bqH$Xk^}h&#f_XXUC|RLap5BC1$G4 zJSr3bh6Vx8x5xW5JAS-OsKy#worhCKJRTb&UD z!!S02GX#GK5{T^@f9bdHj z2UvuLuet_IKpbAF8!tYd(WECvL5OZ4Q!bYMl{hdmE2G01=W>&NAYA1vi->_Ba>Q*Z zfGvGLX0Rm4qxbyzTAC?98H=gr0sil(x$1|AjAtUIPXr7FI**t3_kh_0Cw3p z4XG$VEA*5*EWTQ~94IVA_UMIf{5A~lYEW&=N;q=tZlu*;FByA+rW#pjW_9-pnfXQF zkr>PKJLM&>VrDNIPI?etuI-`Kuk)N^D&b&F)7@0^OklXA9ivY+bSyfU@mr{?Oyg8&`7 zbuo8>{zIRUR_Mi|1!*4@v2&R8NX2C|UDX%tjbJFSTac9c%A&ERM=rDWEjeI#YpAbbh$RcYO{qorb7)M+(?%tKG4-rggF(cF>pbAc4CIx?4|i(n2t7FmEn3UJh?9jsWaPs%BhSFg4ShYNL7|{s z_AG>7lAG_~yQF|aCd{Y-l8RRrFS0!s`X($xX?3shtJk)Iy9u1sl=0~rsJpUx6oDd+ z^_s^%<2Z`$+-ZhJ^$e*dwBp%*b#WUCpoLYbk|5`P?UZOQUekphB2Zt6=Z0YRMHJ`D zQB0U!8|KO<<&$ecsO(K}rJ9}I)!4eCU=WRp-tMeRHH+84J7no_nJo`g1iQlXEb-V6 zRtlOZ0PMG&gCV9YIRR~n-P9Z@NV&z}TRCvM+A)%i2p~}@#Y|;8epRdEZ%U#YG`zmX zM%$=VpJ--Eda5qsT0f~MJb5fT-*Eybt*VA$jU2+QlUObQxz zVrEDWu`{`RWBaOUMfH%3vwT%)*ES%r3Cv3-`<2At*FVOH4uZ41f>KwcjL`Rq(oTA` zh{+yzV!?&FK+=2QcjW{0g<+1Z(5e5zv9TvPyx;B$=!9;B;v9E5`bxvJwUN0 zT)xSU#*Tw2dl!p!)G_@Oa_fJgUMp0Fz!?1+=5NC&Q zVaFoihEjAsc6dhT*oMXZJPkclPZ3`NUX}j;$(WKcdrQB(ScE0(0m^TzsQU)bZ#$+( zUZKwk31RBJP#>W_vQPAny!|K3l;be6xcQnZ=rc5iX`Iu#{k`Xz8M7UeEgN+C3^%T|)otw-4?@spsOgZ9ZCeA;cN|C34 z{W)DXkFpLVoJa?30IQyxG`^NDP5kSn1#svPowyEVoGx5UN}^XthPliHVgchqNE+_{ z+hq$;7LRQJiO?IEel_pDO=Q6*Mf)T6cA-cdYblU!GlS!9;PtkM=lviF7)=qX4}*)s z=Zu^FtQrYVa48U4r$ZO0$>fo zRwnP@8(9X)mT?dEwqY0SO=-mYevfjgEOenlBH9CRafX2N#V@+A-PhK=E~S)?VJ4M# z0Hl~3q0ogDw89?q4>frFlxXmub3l_roZ=Nh8U@lb{|9^8L;7m2D)@~~^`RuDn2Y=* zTCN5-kF+!3)6c0k@sd#-ouB@x-0AUq3<&iJnzt2=2I`u|gsRJ)zc^sy{Ve%$pI^@B z0(+bU&&2N_ASg<}Py^y>?<87J%W#Ha_~?8lzSXyoBpE@Q^6&H#{HMw6`J-ZIUX`pA z|5bomJZ&^bo9mx;q_Br+N|=`(KgoM5vS+N%;FL~y$-;WjlgwXcmld2KkvmBcArm4I z-J?9aSi=!gZGBTY9PAc6fvLA@5R&ls`Is)3B)rvTr#HW+$h3|M#qp%?8@Q~01V~8M zfgd0kemrD-j=U=#3y*qnljkqNTvm?G_=;{9b~^^<_p3VyOMq~sqwNMR#fnHq$dPP< z|0l}g5A3#fI&0vt+OP zyq;K#9T_wie6EwWdN1d%ctQW70;=~DT0AjV0!!imNiSNdE=Ol_OCA!a3WXZC&1R9S z*Fo-wpr9-9%WmMnhc|gIyn^IhsNpf2dZ|J@4f$q7`5_qAV_65DHwX%YOU`CE!yEK^ zY)Hw=mkS+j-pmSH&noiT4a<6T#2x;Sm9r&hxAi!K>cp&BjtyRuHtQuVZJszF<}^KX zW|i(MBrV-Yi#+^j^r3|=UjBnDe+Yyy_@69ms;U%*{*u5&IpZvk0<#hWPFCY6p|!|Y zKzxVFHkIxpZWt&kYphwvgN06M6sWfS{&MUSc0+yuxfZc3J$`7~MHoTdK?Ve~t~Khw z=ar5=gVjwC-$sx@UGXxE3rx8c4pa!pdGuU&@hR^(q*vGG_Z`V)!*%(ZHD@peNU}un z#n8;nkTEkrO~+Cokx`FNkJAiC)qhPh{1LJ>5s_tV#8lcj~olCw{XScUwDxQcbH~C7yZ!ZOP zh4pvG&5bE<#s_B9yi?FCBR9uF&fvgZN4sZr3UZ~8WbX`T41R-Dsb^)bVz+i|g|6g# z>loBxwnF2ug5+U@->XGK)7sI#s)PWJXjPM1{@7!d;(*H-{_@;(bIeVmDkFDQZl~}f zuNq&xcoxpOL&-M){!R+jRf~nOZUmNEoZ=cfC8?JXmvV?M_PZiK?c%Sgq@>h59j8b#7U&5 zvrIQPVaNq~$?AC>G3ai`8Y7ZF9Yo`Ll2Z0k7umtdm`;pstz%=Mx0fB7Ef7$4`+OcP5s$N6PC+f>jX&HQWtpiB{| zA^J{9SgiMT2|M*o#a$bTWwHuD9{4I3xP1eiC(}s4sn&$Ig9!O^)#tui8RPehL+td- zosnfVK`_n34C=KQP zs+K;L_X?CJer}_Zj+`Y-a)FeJPVOMcrP=+mbi#%;yCZDQk)V(E_J3xan_W8lA2VdK zC@dVeY7Nw>ukL;^;*GY&wOHTg0I|vJ7RtZP(>bqmKFLC$VbHwzD2Xk*-o34+_KPi| zLUKyKn@)mbTN;Jxs zIhnB)B11RD(D=JA@wLu6L9wri=H`nJqJ>px{`$3D{ZV`cN_kV4o6sN~v^W1og#F80 zsDRQXI>pO5qmX27Bon3Rh8DHy8n~_#@b~b_jeeWWUBuXVEmUxuRQ4>|^KaLTE%M-f zL00TMQn-_L!8Mx7LWbdQwV;V5I`juG;oI->!P6rPt2$m2OOUl8)?&!&^2CvzJEH{N z)OgVCR*H%+QjS(Mej?W>EJ+6LsPj0J$LK}AZZ{1P-i{5#*>9=qIxYh)>nbbdhNl6J6d>$2CZ{Of6a<(N=mOPw#Z6tzzS)l72GYt_85q<*5 z5^Sg6&Me4_C-&qXQf_tSO1}&}pcG5h?}WNH=J?*EP)Zs#Gxu`FsiqQ03WjpddwtNm z5K}p&Of|FL?p&KLDFPUWI})b-UikbZMu}iX%+F%(s2pggigvUqmg0 ziI(s}S?ykYy;hIrO{={Z3RMyXm%SHkK$ z9HoGqmQL3V4jdQ=uEzOIXv6z}3@z6d{{rQEivqf1AP3mhxpOuWo+akkrBwZfw?`_L zaZ|}^FLm7ysN!NCD_a?gW7m}>7SeY?M@@7l*$H6U2SMm|j^lR^`3E8#c~v1GZXOA3 z<_nsB!V~r!JU8uIIS>qA{02!KJdQQFD3~e%x*^VOmY^ob}0`Un2wPjtE4G3#8tkcxj=Vg#I%Eq|bYP zjj75&FVl)Zjj8 z;bsO*Xz(9ZN4>;MdzZkGd}H}2&JD(aJQ||9zmQa8FsW4G5fscu0*A5iLW&RC7#eZ1 z$oNjwq*IH;H74)b+jjN(T8{rE|I|p!@}}?Zc2Oa;e+-n+D&kU3Tp`qt_aH-Hsd*92 z++uTcuF?TY_% zC1%>YD3~=})E#2QRdVQKU*P$)*ib=G&^pB$?9~Y=b`I#nmzdT-%C?V?_-<=8HyvfVYIwB z8n=HDZ3+~X&G`MY+xKdGPjmZ}%rPuu3t0+}*>+=k4hy06_D3*^wW(ak=uV9|Py#k8 zNnG4-nh%)+TsykXVqwyXyDW+@A|zK}t&VR3$z8=pb)9#b)~M5GVnWn3VViT-(yiqG zrWeoIDXbtEzYLKZ0*2J@dKELSql3oXdeBGGL<`>0uOp~@4*0p6X6#P3bib&m=%@s! z@O-tC67hV%F0F~-l>M&TG4a*4?7G}Z6g`d9OL`J_h5P|jii2pJ{KYU3qAUy6#`*w~ zm1$3HDNI-zp-Ci7#x-Ut;+qPlzr)8sDvN`SNF8Y6e#f{PBaQBe! z=qn~WefM155;DFaTO-sMn%-Z%(A}$z-93p?NpNJq`feQ*yV|M2qNO&*dPv%&`lJp( z8@#P92S|2Li|x1?PysZ0BZ_`NdcTb(!U@tnw)!$VuhU?-NmA_7VM>?t#EZ_!NlMki z)K%)rGW%+86$B1aRptDl99$}NZQ!%i;8~XkFpCRY4B4UhJdNJ&{c7u@7NIDu?xdTk z6KodbPs!lEV&sKFHPK`CG8(f{SI*v4=J6{=aPqS?`3CBlgAHoARM)?oEYAjcR%s#H z#=}=T!U_V-*m79b?V(aG);6$!Nd0MIbtkO6Y>1bD$D7xiNHG~dxw(aaLFus+a7bRZ zqk#tK%Qpy4b`dzo(e1_tizzK&`o3)KjqpT{7wILVt7mseYgn=;6;Crtug0PU9p3T? z64L1Bh1qfzo1kGdbLos{^s`9sqj#&W#EJJCNYJ#5(!CA#+sM^+E(gFLVixpjEWs_N zx>T&3(v>fi?0w)s8VT&3h6hw~yYDiVe}iP0%?`%)9^Bbe;KpL=5-7iaYm_h9`(;^q z|Hh>et2r6pJ{iaPO;hnv>lnvQVF5AKGnkUN?UeaepUr#Zj!?xe7UL!zYy^dDk=})S z^!X}&C?Df6_G+hkYYV-S!Nxp+jR zw0;#)k_8W$fxr;*kP>8;>H8XDv{9;c*UrDv0j;OWiYSCCWxVR}hk1#GW2r)z-6$bE zV3M7!{Wj8Qqxz^M^#G8QVa2xAmo(jh+-I}usL-dk?Uyt2BGN9BfeVr89b7sh>xi`V zcxDc1Ld!0eMAjsL|ADM(s9*pWWJOD*aIhN*36Ea1<)%mw=Kh1;vjNK1@`wMUuy-oL zvoBTRJ>R%$1){AA4ld#FXuG~TOV7dl5$?NTZ6}44Z*?ku-|??WyuC`}He*!BjJ!Tr zDV3b+ved{BA+o>grXC)Tx(vtGFlgP`j{n@4gy;H zDJw|L;VEA6c6u(PNG>=vO~m=2!Eroknr=@L>vv(Ip%3M-*G zbb~k;?QDnMHEDod9@dz@NZl-c!QPP}Nt!yEo4K5oWQ7 zGq9vsf)iLZEd24G1gf(T{m1%nGyBIur?MxN`*IYd4_xdKo@bkO|0B^!Xghx(*MV`k zyT0||4Nq(?`jwDrYh8^bd8I!i$A&Osyzl8RzEVa9Te8wlyP+@oK>syE9$c{aK&7t& zxnu9}oEybQ-Y0nkjg$8QyF<;ZTbS=@H^w@qMdgCL==zQc%)Hjz-?6AnK~ zVp481uXx=4R6~OW4qK0NClOnV`*LZgO9t5E@tB6TyAD+mAv2zIit&tRK^t1?l;>D= z><+}-0$;w-^&wrBjV0cbu9y#N`o>x_*GP2UsA42+=ulg4vE}Q(Map~rt0a^O5O#@$ zlQjzh{Et)7It>V&TKt_~eio<1MsvzGewdvPy4b7iVnPK%7m#2d3MN?GLI=V*2(HaE zgc6uEZuilp(5h7sN`kk%plV@JBliHEIkTFlIGKYRQncx!?QXxLD~}5CuzkKjaEc~RpF%5a;&}E=vk$U1 z1_#qgQtz;<|5kopB{<&=V81tmSWmvnfzjWC{(sIQc<^@ud^_d?cW72$0OjLC?UW-R zCoqz0p$#Dl`5TrghpeHnjh^pwO4czk*rDma1^PL>kz)Rr+NX2hU$tC!$kIVAi0>Kw zK;500U!q)|#t0*#F>X5)MRuCym+=JNGT^t$H}Jz{Os@lxko|0qe;c<` zx)h(LzNdB#zu3{0W?-YrDSfCumGBvFWAjy-qn)iGcpD5Z_a8@a-kmPTc6?iy*>V5J ziBH?7bbGEn%10m1@T&QjDDFmFWc@Pw#wZZEpD&|k6c!Hz=o>Co^g(j1MM!fX7s2wd zBOtfAn-69Dzg>VjQ>L|*xVhZqx&@qp4x3!?7;x8b|0ZIuSO%Sc1PU?anywGh%MFKx zmI)Vj$K)}2ed{BGLJo7-1{y#C^~Xrq{KjrAkK{i?kaC3!V^h7Bj`vjL({ms{D`KE} zXNQbHola&^H&7*1G#4}(M)SzJww^73FfY@MKqrv>3IY-R@;`}I9zF6oJKu}?(&|bo zqpm5CiMRtfhdRqQgviT8O!kt0A6BdWX{8VLMIZHSWq9`ZXGv60m6DY3-&TJ(q{Xvf zuIotWJR>f>d+xY;4Y)mQlJ7)tkL2J>ytU9o{JF>r5_{SSE$KMd}wgLlO~5J9QJqK)mQz`;7S_aCwh zh+n57w?YB+ihxF_zUo#O4;d$D!Zm3AaYC@l6-GdvLv%LTP)R#jVO?t3*ww+%xcHuLlE03BH+J;vf&T-yokSG9TcS>{ zgm|V+59jj0In=0uZ~eBc$79VN=XF@8W7XEBmZX=m_y~Z%xz_gzU;qxJ{($koe=o^D zR^hvrJ6Ii0W!Z5<>A>AQBRu32{MFs-2rwg|V%M<-Lp8ko*S5TDW}X!YsVX#_Gy0SL=GN*3jTJS3b?xN?gy*#pXt>9pt(-#scA6sSDru1nFh8t?2nGrW#};xDd*2OaTgM~(fB z3Ms`U-I(qMp$|3(RN`?FpnDJrE zg(j6|)KQA%ccy%Vrab;-EvI}d-vj7`s%pT|jMkK+qS+#ZbE{lFT}>bs&%7GtUr+R^ z)Z{9g>f!=?Xt6ppv!hU^9UTAk?W;}K33t3<2u@ncD#MYy{AI%8N8HQd-2ap}xu$3^ zWrdFLZp!}QI7?4qvCC1#)|;N`+qkc^RP2Y`GKyVe2A&9bj8Z>3qc}TB@|{|VoL;$b zfBSfw-y$g@*#9TAv8N4F$L)4qcZ?y<;b4R1_7e2!p(4>xGEny&ev=NIkT`uMbms=i z1~@klO8Uv9h-qIDH7I(nAoh7u_PVv{j*K|N?!a~VT38$<$QmDgN5`_czqeER9$sBel6vzv5uotc7kBvTJN;(7I@eU;sxzxWD() zz!oTqbiP^Lo!FmNlZF3C?ICcu_>zpfT1o}Kr-iE)>*)&Fx8BeZ8yZeoKh6KX9tS;p zr3W4gD=#LPby;9N6KD;T*pYDCj{0dweu<$So>Z8~R_Rvd9Tm~CgvUT1s& zDySzEA5wX46)+=rvy{-n|BS3OhIF)F3M=sv3Dxl~X2>ro0NWP1=96rxup4jY5-`*kCU=-x zrk=8|_sy!I5$(x|)928rGJzlQrRyOMmnVN)0JsHQ&!p2!;L_J1(H z+7R_7{YcviQzQBjYm5-zqaon|>RbSp9Z--9&xZ1_dddl0>>!$a5b&!81S7p^T_2}B zwf&|YQXg8(?EeCMoXX--SH~3A*w}VcS{pZJEEXgo_t*Bmxp+>CcvJ^3uHe>|u=F>1 zmmqg7>e$eRBREERpZt=Ce!Hoa;zJ+{ZRgDz+i9dZ?6=jmOT+QWP|w*7i+mI*BM3B7 zUh!Y*%If z=J>rf)RRMh$KI!;K)G`EPn(m{%8RIueijTS6@yY%Hru6whtkX1S*;e#iJw7Y7iCa@ zcGX~wb{NweQp_gb<2Uu{`(d?1_3k+@K3nn14nbe?|lNg|gmc}R61mmBr$E0NMW+lM190y_3c{R*A)bQ9BT3Fi= z({DE4>QX@0?PZkWs$xMU$^$h=*!ZZW5x2V2FC^NZ!n>IYU&cEAET z)dy&yaF;7_#d{az7^kHgbSXgrYWY*8Cs}2_^>K0D3E9ER!$8JMXOkY@UwQk?IruK= zbjV1-?Xgp;ygr;<#Hh&hkw?Cpx5FNn`itV!q1_#0RoRH>43kD#t>-)jIVRWzahznv zyFvlKRdiT0`N@ZBJE3IAhFJ~@g#SmWh=BeAkbzS=o6`_R$`hOWY*PaHc$D9j)p)-~ zsn#a4Wa&5mVlp4~2qCFBLLUF|Dv|QlPEKMLiHYPkAdf{qLH&Eb_HN|I=(o5pQhSW) zbPKZ+Ft$&!o0m|=k6p|Swx*)>2@as6DVy-b3Y*naC;Qo0+8sBli|Zu){e)mv{b8ps z?Pq#>Mqr3y)8{Qp^?__C*f%Y7+Pf>dv#W>G(%vfr2eC#|>mgGNK6{JS&Kvl3A{#mW z3dN8FsHWD?;yzlAh5E^xEq2^{Dpx>>5q;Bl>uObY^BCv zv<|1E-K&M=R>PK)Hnm{uZCs7)5c^H=QAZK}Tc?UUfFBgsQU>VsXUo%aOv&&ZW6^d8 zW4@AmfKwK)LCD$i5+_Sz$p;A4kmI?8T%<-X=+@Xrah(;zh0eY5p|7_0kDFKKi48kw z{mq!?F3JUw+76V_+pnvoYj>bqIIl)}o*S~2tw8A{)5KQGhk#*Olh?5c=f8rmZKK-h`uFwd1><|}wN**Rm>nmD{POd0H0249{nTi%MB=+YfjicwN z3?@8$7?q{gR$hzVZk0IAOWe?2MhLh3>*ZF-*C;>8{89q36ASV&(pl1j3fVk$!5*@b zpzGLfI+^Mvo1r|MaM2vNcUzL$0wtiegZWxJ-^|JmL_in5DlBry+f!#YM95h-+7)?i#$cpTbCws!lR@lT>Ct}7f z2T;XG95Kbq%Di3YG0dZ9^4cX`MLg&JQG!rN5(Xd`w_;5Vcg~@Fy}Y_p=s6VJh2Z)q zv5~{w7mZp~t)!n>UrJ1qi$f%zF`EwM3{>z-bXRX$c(;#i|2j!Yz+swAQFy};m!=t+ z=m5!c`h9kni21<%A`j1u()=VFD49#{^-ZV9c8Z7~S zA=HvY9E13cNVo~7vpGl377yw84Gidd?VD&v9jP&ZhL@4vj7u%GdI%8zEzO8`B}+YC zs~VLND4Z7D)e6jE4ON4i2t4nch8a&H3_X9jc8(~d=YW0R#7YPFKBaHxZ-6MduDvgzZcL%zv>P;+NuoRj<*HH>8RDPe*Yxx;05%84e_w zKMW^udXmys0f`{2g#08V_9H3c*?7n~!A-7ae#+3c=1XGyo9yE6I9p2Wu>h)_VZUu5 z5%R@}u;!_A)ONcbPt-e_maw_{raLthId#ZTam3P}r}l!4s859*ZPY6gGjDe40rgdi z^AG+OafxMNV%djmc6xpNTpIiLnn(~fHB06#Yt#HR>CIS$&q&qGcz4D)Qu>U}pzG68 zXCnaNoIPfslM@Nf?jMpMz-t80s`^UKKxJ@SC1@?hyZA8@gn^XvoMsczHBV8H$f3rS z#)ozWfT9QD?;Q}Wf8sEgimXNwT|`DJ=JqrXgM%ET(@)xEP`wLa2i8|VX&D@dgw2g} z(y=^Xn?bqwTOu~=7WU8zccCxv$C(gi#*(vp@7O{-kaGHu-rPll|My6F`vH{a)7K>- zn&e_pzY4MYTe~`>5NnJ2FW-fWZ+?y z0+Ld)^~kzcuOAi*+nlm-=mNjOTNC81*!>YgwA;Iw)_xV|2`l;9*G_60{38CIUpQ7M z!A4Hte>a1qmr}pT$95Q!oHF{Rgxg91e)H2$sJ7Nz0$bv^GN}G1Kk;c}tx+kMXAXq# z*%WG^Hk55{@D%~WK7i|W6~@8lll;p}T_RAYMsWWdEK$mFmNQz&hI}mcNn(>UW<871 zGGXViI*)9nJS$WWST8=Ob~ffc$5W(S@{=;r=pI!oIm`n3D^_lSN4EP1xePF6lyZ6A zE#6R>5t1uM=I7IHkg)$RWWKlXMr`2z{NLd5lmk{+!N4U(+M-7T*Qqgb)q=-FRym$( z)%WzP|MN?#fYmH}>V48##&m~C{qgcR85#iPD>%4fpZa$zPvn*Z(I0oKkS_Rpoz4LD zas5ym$8B#Sz~o{ra*5II)N%4(Fk40;L9}r*Q$bnERki}J2KzbR5;dCKk*nCP*3-Ld zF&^tKu5n6PR$G$QwP;@!+-){}jxCh3U#8*wC%>8plG(X3@m1<~c&;rDnjn$p-%_2D z`u*D+8>{d?)%+&L4H9sZQ%%36W(0C1FWoF-T&&pAb=_SZ4zI7ew3LeDaO?Z{4Dbt5 zgh2`ufwKiw=$Ll=ITH6CAh?_*`g_#|%+lF?zJDAxxB-#W`*7%`R`3oKZUY>s>}hka zYo{I+;3tVR1CUxBeJ3j`EvCZ~3IRZNBp0@!r?k&5bf5|4wPCSy1n({hFjp=F!gy! zI4N_sPpg9brD$ie#~=*iQ7!z5K!>)mG|oow!Q3gL-wb^&dej{OEQKxVxV_N!aN*+Z zOY#gAL)NN%<>pyt1;@Ooq!_1~p}=#uST_|l;a=V-O5Ng1QQt^pc_~uey*4WQxqswI zY)COKgAcnT=UGe}F~#j@Yx@6DAT~(US@JbO1DA=v5Xf_v4A9$fAQg*^Ig!FPBO}UAN(`$41ep8=#apr7G zP&N}HfD!MPG14Jo^IB?tN&y@j7)vafz+=9nj*3WAVIK5M4$sx>S$${x46^ogU3eF8 zGib|Ld61svD#rW@W{GDI#X2~9W);ZZ$BaH`?wr^ABK z;aa%}pSdTZGD0)17>{5{q=g71B=}pKtaJpIPcO4JTrAHZCd!ZL&39pr5U|m|Vb}@a_r@`=)W@N`IdgHJ*D5$%Y)^r?%C(dZ6 z4J4K=f%vD(hXgS8%FWGiIP`pg)7Bp!ry~RS=QelCJYz(6KIWPQ!wmyGI`t~c$>XCe z_joL^9P~9wQGt=@%`8iKQigD`k14fvrJ1hc$K54pT>{=^6~`_UiixDF!~xO!+>tS!XhC<;E_(r_zAZ~qWrwo5rq({G6x<%bnJUJ8uo z;@r-JqA;))xIeYP%EoR``jZuBA>9^WF#UT=lNu8DnPUxkCQUMi&YTwPF@*nJ$p}Yh zM)*W8Ps>)u0W2@wO);MwLT9l8TJNWHCyLn;f?Q%{9!R+_CZQI7ZkWkzKl*=-I!OQX zG7JVDjRHb*0IF0+{%LfeF=JzfDa3_mn~QgL2QRAEr>Ez2Nj}441{n^Xd!gSR2G4P#I zxCd9DXkg*#uFHHSk8}+9HU{w;!t)f^JM?~hxndj7Og#%TkQk>S3LPsPA_*33(N#jM%+R1w0eX0vwj34^sl_fj8bGPch6kOe`ByfOoS`X7GNA(rytJ{;}Kz_@kItN%zeT zReWc{YA83fPSMZ05zsV+wvt?vj=VacnZ|VAABD;x41uytoW;rlQ z@#gtF!w4>(qPQ%H<<(u??%fF=wnV&!T@v;P&fJ)ht;rn8F6}M6MAcaRw(}pQ8l_i9 zW5k#@&+AvxO zLzH3MsBJWM{;v-~IV<)rgIZ3jC-maG=`Rq*Ys1{K{qCj`GI&8<)mto? zmvNor!#@`NL)z}GVbLvaw^psr$k}r9wo|ytD*t7wP%tZP@!&r|hRmLhgf-b??g<93 z6x$}MAh*8^cB%r3f<**!0LrjqzOI^Ff*Z)US0yFlI3QEi|Gc+1Y^_u34KK-ipF48c z<9skaCzu3Ls@)GCmro$C#96$N62-u5lV%DM2gG;&+z#OrR-!vY=tU6gaFaqXNfV$b zHoOW+z3C{G#N~{|(P6JZ9TLZ{v>fet;N0^pUd>s?lOszvAG5V+*FPQgs@MT(3nA$% zg^Y5kr+z7suBxDf=V`&98_h@;usHAlSvVzVlV1u&z9Y@pmjGb78MDqI6~sj!7wf=xGj?59X@BL6I=O^24p|_~0 zZ3%87k*J%~wmv3HekcMxlf1rCwvEK5k=~WVp%}8x%ak!sxg}we!Q&^=L`X^P=p@W0 zesk$RmaRaVf7~+NF53-pk6{$@{kj7B8A#<&cO)X8T7KEr!!W|l`d)nlT};A4`ls!< z2&+webHMC;M>UjAYrgbUbyN0W*G4JhlQOlxbR^Y_ojP(Akx5*e`2#V{huk7CSLU?1 zdxV0TS!>BWm?v?{OUzWwXK->>tIFb(ct(*I6&D?e!y92rPS=~#?g4*CXaVBu8Eb1Q z03ZkRJ#n5;m7V9++B`mqAXu9k&sq}6dmhdjyyo)t)N)3q< zN;dONMZB0IF+r4NSh`&6V8J=RBK(q&%WWVi7_5wRBEK%sQJ)AsWLXvAS*cu74ad|m zG{)AoC>QIgFVvA`hLHf+?*tw%DvrK4esUMO_@GK@l*Z<#TlfRAgJDOpZX7ty4^|0%zG zdD;7Y;A8m|&8-vwVi>K&8y$Pki#BlLN&Iw~NUzJbLIz*V8XS%~3gUE5F-b#gL_hIg zqlS#Sa~X$cF<<`J@-FiYD8Ox#OLZ&_CLzVFU?T)M*=9MZW{BsW1Jo@4Iq=nUgKy#{ z*B#m`21AHmq8`y5+X3Ii!N3xA=~)SvVMis6cEx-@^;^#=ckTaut~fNC!NZ z*G@Ysr|(a4)kc6l?jKw9+rY8SisaCd>D1)cE@O>SjIN9W-rJzV{0QZnaEn*0r>;c) zs>XpIE+AtTI9SO$o1zC9jD9ehmxEU-M_*AyD8@W1csMaH&#?I2Y%psPP?jJoEK?Ls zbQ7Qu?#fLYY)EKWO;i?}D7m-zDOLUL$-i?&F3&*9g4!#T%jOG5ILD_seGWX=PpUqu zjKhze0g;!7)indV5xTNZRx)?gFus$AM!07&DlFld_B zQUc8V+PeM@>{*r78Mv4dF5Q^Oh#?K65){)G> z?sxT+xtkLdD2%0;7QBAG@Cnd z%NjES|8RtS*1NI>Z%qcxW2XvuICzYxD0s2ha~R2?XDyB6CPjj^9Y^Sr*l)VyL$ukp z>#@<(`Xw)t^rruIJ)zE$$hyN+FW#)S^gU=dqD8sS79RH&F`TI3waa54{({fG}xH zL26Ns))LFdMWKPRwoQmyk(mKac>8PSHVqs|MsL`^mxsf-SkE6!A;oV)zT zGLwjMli;mo=NR67bEJztfY*zg1$4~l8rDB!KzGlB1wUWV*Ywo)Vl#9^H!wt+m#CM1 zi(4a}u&@_(unsOTcsQ_@MD3JC9CSt0dHID!sV#}@!?5;+U{&Zdi4@_I=QLbDxD{w; zlaT*!P&#!@wc{{iO7er#!2lYd_z)q$YxIIg1%Uo{6p-`vA9i8&wr#C3Zfb#+@KWHR z=8pt!o}aoCLIaELr5ZmFtTGIxDz)75&ft<5SKqLF&i229w@5xFi^bVfv9i!o`Uw&p6m|HhfC8 z5U2x-gKH0zqIpM>qQ6PYn12Qwpmj?&jQ zXMj(yP3Kz2R@CXp0Xx(Z_RLGNR~~qSMEsQ=#mUC(Euuq~;x4_#@JA7I1n&jO*1 zL}(uv0!U1tlfbY@l%KTys;|36TIgvPX3&EQ?t~-np?K;wra`J7 zt~Z3+mb_}YH$}YT?m^WNd+8k~ZdJK@*_i0Z_ZmtW30$;bgB|%UK}E7%9P24a(fO4m z9K#D4l@kQuUl(UnMu1zsmr>Gv%1DVJ=k_%KNG_G!A$R!!vIffPy081%@O+&w^&=6uM z5iYrMnBh&}UIfz1cYn;RUgdOKMnkl!=alk(!rn`Ienl=Tjqkb_W<+F#f7W5if?;2a}3q z>qt_(4-Iyi^*+c&(8UITz5++Bq!sC*iSO4wjZnlDlj`e#b846v{@7OA=@R#ld01;s z8vg2o8mA~u`}}-s6&6GHfW^SC;m7=5qE(-T__wz6? zr#Q%nd;DWg*LubdHnm(sOn$gOiSc!n=3|lH(yB2VkkqLyb83lxj|Zt1{Llj;LH!T6 zzYzIH)iEbP1AdL?PZ_7`)6}UjJ^A>8_#n;@_Oyo!}L(!<0J%*WW!=Q!T#Z z#8ZdFN_S1tb#0ATDmI)<3BE~P#v^}HK|ncxm7J{ViLA{7_F)PVtQ8-oT{ z@B^$YxU@z8p6t!v1D8@1%mlLj{}k3`48mN|UkMK|u|s*+37fv_NHb~a+MZ_+z9rl{uB=LR;7NHw>h z3pWwks_dVCCs@A3>VvZi(N{3_8{ZnicYkjsbs(r@cWEUfUewcadWD)1KhGG9w3VF=$~Y5c*>fH6?_-eN%y+8iCV4`OY}7A(<`+(s8@%0H`47 zunp1B!QwR*RB-hrG;x|zuI}G9eI=sfy_0j2LazC<4r4VAc*`{PI=P?4_%~KGv%DEp zgc&PtvJ-t4W9fM2$3x?*y32}sD_Y=Onyr9y9b2z+?hRXs7*914RAEKaZngZu? zj}mKnz+?)9w&|M!Q+5kQ%$kf3nC{ipDogOY)v@;Kqj{IW7wiWag4g;0z9`UK zlTnaTSSU`%{H74f5cGlO=uR~Z8+9%r^B7#ICj+mwd0`ZM@SXuM7p}>Mo#tHss1&6% zrJ`U7_Sg>UNVsUO27#{sMBqLwc&7nNinnu&{gAS6ek~a=%-=rW4_coxFIuQy;#>uJ zMcp3{1n>ULMWy5frUrVEB2;_4j7Xpe(6p$ug#_YCRoI7|VZjT$2xztM!1v1v#raFjxzv{T)kbe)e9erl<{T)Iccj=q7c=Tx?cr2Sr{dJF^eUZy- zdn!i*{HtTz-cU|CLyr-{jCov&znFF2;TLzg3rT#3_zdHO{e5UACq|X1Bed(CW=_>G zl;pj9z3|Zk5)(-v=7PIq~?=ynQ{O|(kZ@K`qNg6!E$w(Y0oo=ml z%9KFc7co`9Je7-Ee|rKnOAu^X)zGZsI(+1u3;g5j#7^Q4p4Zy1%r(V~1XnF{*&`tC z!JE{x?dV|)4WrM_NY88cPX-k zt-~K{e(UcYP9wlSw z2xi2h;PPxPjm`b{+FHCp-kcP8U!&`dch9EL&qYRC<$&5X6Ev|1ZUZrHSwCoU+WeR= zN*Bnr01~1irVaqB&P7LLQ1Fn77^OM#2?gqJI>J_oPOeO9cF97PJBh~0`xbhv24GAt z=x!!i0WR4U*u4Zm#_lV@_R%kiJ1-USor=usY_8SRtrR9*=_9a{OX)XRUQ7TqvHs*D zlbq+=IotrWwH>P-T!!mR{m9Ci3buuow3-{~3h9tlo%-eN>*=B}(J3*Ysrb=NH zBZYpdBQcku-y2QJ*T31a-0W|igBcK@rCEsNnvv3anlB+tshAqY%gBc4r@ZTB6;_1B zgCW7xiV-sPwr-bNkaG&b(XYiwCzQBRf^%E@ssaTfOyP8e+pin8X)rQ7!N(0>@Bo}O z!Z&jhUTagky>3@k403cKrK;kS5zJ6V+n>sUgPvJceZrCnqe6sbmKCiE7ZKC?D5Bu zpM-G?LY-x)&Jg2z$zdr_8fyWSvoIT?W6fnCjKC9sAndK(%u`v@h1bf7?ebW9Rzy)- znKv&XT`-0Gk4r<2W}WBM+J;Ae1N)%E*oso{F#5MA401$=$@VLO2_{H6BZFGRRrtIE z8_xpQ;TF1B=E`XXf@cnps#Hm-qjlQs*k^s1i*QFd;n7MQK81}LegI_j3?D%NHNg4T zbyEq+MlxI-*`R$YUe(=l_lQ+_)L#M4Ie4CO#iBs7S_{5v+XA(_$)FPwa=fkr3}~J0 zB48L|Zu0g$ne}&ARA5J4fif1hPZugox2^e&1^Jy!m!v`In(s8iLP?l>(bLLPk0;`> zKn{erOad+{k{A9;GVGW7g12eRTFr5L|KjFc9ozSCqwi%MT98#{6s9O0|2O}Gs~2YR zRJ#%BKY}_$b!1Tb-AD;zdq9{dhrzYOw=7#~&GVK~HimUpi@#@Xe2}`j6D&lQWSwJy z>6Q)3V)qgSH6>AkS}@iM>Q~5Hum}$~*=fNo_6GVB+Z0sLwPZ^6dDu95~)N-pQ4$K4hlHgC@Ok$qF0*b3szDhcMJhqDV2stpJoi zFRxSk2(5g#g@Y`LHcXpJ!5TE29N-iM|2zPRt66m<7px)_AoiTb28M0DLcza^bk&Rr z8a^Dkp*=@d-USaVcBtWvl73?1L-sABSV;}sEZHt$vXJ4{Aks(PspwMJybn$zrGN^u zM_xOM8^>D*Xo!))R6SsCbbt}9L))Xs5Kt4`n-E|3N#+#lH7sL@Vn6`=1jI=*`dj-~ zZ-@=b$e11)8WxKB=v|9@l zI^oM68Xs#pL}4 z6lr7#h4ry$VoplmQi(D7xjLx_phs#?7krJPzsE6Kg0+~4N|3VC&w`Q(SwtrU2_e7M zFl7YUrd6w9p}BYKBfWvg38|~z0e@T9_Z0A}mY4QZ)%}`5kQNKJ0HQZT`Nf994{TFq z;k{Rjb8z-Uh`D?nO&0#wm1IJN?(N$@A=Ib=a}*&98BSH5LDGYRt2}t}`fM|;>4K$* zof`_O3BoE0MU6{pns)HtIb^5;mSQ8Vo3PFZ{plkwC&8;0e*6sjx#aWP8_S+vU3 zVMR6Qxcj2$(H2t*FxESZ?T|3Dz{OcXLj9iP=x2nSNsht4%6nU$+Lfo|`{9jsC0BaM zTR9&8aXUX59&1&p*p+dLuWHVd&KHguRKg=+m~HBs3*vf*p{+Kh@(|9xd$e7TNeqKT znzHTq=Fnc%TaQg+WW&44M&{INubo8Jxb_vE&;SF>adeJP&>MxS?tvCJ2i})>v$q!h zZ`*?M=DFndR*(PSD~_d%wi-b0ZJ5C>M)dkE7`y+wE1k(@r#MD~ayu2BQ@vr~Gu9wG zUqf-$?o(B4^GGZkz?PB)o-s;)E0I2_j(_@b*^!XQGssAe?SE~r)iNldlAi%48Xgkt z7#TGuCC30>PLRIGpT-d>+LcP8^PlfgAGc3vwtjR~Wl`hwH31~d2Qjq-WywS`e3*m8 zqWd37r}_`d8c++`M&TCP=C+N@>N;5ekA!~pMYK38TtYrt^XDu*P`&VmbkLf; zXON`&?V}cB?td)U>x>~6t<#OVsN$TzWq2FIu!1NHVg&Y;U`O0r_M_?4SL^d>pa{*< z*%=O;`zlw=<9Ppt5dDg*L5(3q(Q$vLyCMAkK=kv9PiEtH9cnruz_&VQ;5!wlsz>wU8nM!}Od;zz;hSZE}A^EVgyHs9;GCt`tWZXX$~BH;{CV-X}s7r+}NfGF%L|eFMuFn{eG+EiBMOi zMO@W0OYr}ZOe=^IAGu3y<%A;hHRXxlsyg${O^%8Cjr%T$?OW&P>wrW%I#m4W!YNV- z$2SWk?uCI_ z++!2Ho>>K_YFiJmrLR)U6%(0@Ejr?lFnDp6?SPYo7e5yW&yU{@ag}I&IKiW&({t{A zqQ?;F97y--qwf%12gn}zEg^*cWwDg|6sbW}_q2lc#?Ue_>6ixr{mdcT)R6-1CCU1- za0)#;#BW0FpDld>OfBCTv6M=RuRZz?)3S6B*8(O2jid4R`W+3iXSdqhvZeha&<^=k z`ewaW{;4F;fmPd@Zn#o!z*><>p66y9i==IWyeADH^~F+o8p<~M9h@niDElaIFe5VtS~%^4CDf zDVAc`{JiMX@ED*qpj|W`I7uEW-Gh?F>K3wzf9}B8E^l}kOX&ws`f)KTVXHe~d?5RC z&g)3}I2lKW&i|-c@_cK~x`y79iGXI&r}Mi5h^(_|3S;}i!kVS**R-t3$Iu9b1j`G@ zI1B3VbvP~U?CAPubuEy;qXd}0q?sF6cW@?g@|vB7vlyJ(b+gf>M3vg zz>Y-65IdgD$%n`1c(*CM(h@*a;D@ZHo4A< z;kxechL0fet&x1&#?^1mUBj3*O0C5;vriA0pHaZNhN`?#!kR*`Mm7z0uT!6B-!&uQ z@D+|F;Ln6(xgqQJ-`1k~)x5=Fauu7n6k;cC?g5id1dncDcwSwvFhlM_AXg$ddaxR?lkX9T4C{VM4Dw31+n&y%xa@?eRx+wwf?o zD^WKkVgtkLi{fSRFy|dB{Ty(3k(2o+fhpJ)m#j(V4fkOPk)s;nfy6wD95emek~DPG09+T9Yx3PqH?Al> z%efy{FC9wFC)yq^`J*`=e*5464LM1{lX_zP2^0dA-fH6H)n3IL2tr z+$4{ILwSboHZGHJ0MH^jAO^Rr2d_G0oOw-QwzKh90s9SFX9(0CI!gu(=qsk)2YT6!*GtxqCQlZ5A}`fBfHE`;Jl z_(0`)uy`MuH<0FR&O}Rmn&*lZT~lj--j>Z8d0kK;4mZqQIq{|nYDAl$;Sk=9O|GUd z4+7SivD-F#(G#BMH9TodNsJMYukzYXD8D8+n7(%!DK=8KTGHbS$-J41l_-%qgL55A z52x!f=047xd#7-BRdh@2Sc>Myxi}>@iSFVaPf*5F&xzH z=s!-A{HQh2`n!>*-HykKEWMsjOwBG`F^k;PzBqd}0?x=5@G*q=$>5mOnCjOox=3 z%G4^g1_Eprh5odRGn*|o=9)3YU5<0k)j5{$VB_o>9#!wLZ`cA%h79aFr5mJSBt!@(_JGcocjfW z^Olum;3f(}HG6Q~av|AL&-6Qjqknd+ zv_EP7Pexqs-9G5{!m{AK@r@Y_BC-cEHWRed%bj1MfABt=;7Bl*p^j5(H2t>fN`7!A9Z*b;qw+31$rZzc~w0TT(dx3 z7x(jIKmVrGSU+mffsxk;)}QXaZQ`3H#U&agND<&*9?zO~^PC{s+Ph;grCBZ`0N)sD zYB;^@WN5(&(X~#CFfWg>nXYP4V+vb}Vez~~k_Oo2y0LAGg7BwLrzYx{S4AX?UmjPc zM#D2^>EDF_>eW#t@R*1h)2*u=P8gN!83>1LpQT;Me`V7=4-(MaP2Ek@<*(UeoX)u~ z#{6`VrY~_{ zslEGGWrmZ47k{{GQtnG4Jn!cpofVoO!TdoxbZY z?h4#bM3U5Gk>lQuPL3^oXcGL(5U)`5Sg<0F5j_wXM;O}nbjS2`7t#p4znZ9+@3Xefc;hOhcS(xY(vf9!uqgMMIy>MRx)};19j@?KHw|ThX3ys81 z$iM>?9CNXrGR|VsDsf{z+1q#A6K{!zXWs-ZRN?PFnC@Di$?qC4B*#JZp~|Xz!nZIy zlH?P$U0gB*UKV(3XcjoFA}vn5_d7y)5dv6gjbjGSN`0Sf!TD4Fp|K#vi%vRz|AzvR z0UGgO4x_npXD}*m8mz^yr-T0A3%3^FY>i-)ebFG?g*|CDacUy=I&A^_e}_QP=*~l& zhx%eR%62zc(EAMCP63Nn99EopP5?zfy1yWfTtC;Son!Kije zsjynGJDMkT$&y~h(y59w6GHpu@(gv#a?aPgg{4KA!2+;ve$Hou1-&iYJ{`UBrJJpf zBrx<32!>l1YnXsFV9%sJ`)XOzVE);_!=;)KD#v?u$Y znYc!_p*9bbAA+yyCl35Mi?+Foy4oRhpF*Gmx7GgMX95-G*)*KnC~fiI?e{>Y6fBKJ z$O#O@u_h?@l>HSYVmzzDdgH;4c6^*suVSIhCFa3h%4r|32wz|ZX&?;`DN~NmY}{^Z zSUWKyC+QjJs+5^*{K~8jK~I4aax)cTT7#@ug##{|BMGF&Uh44F*lK5H%pwm{$y*y+ zNyq1K9JQaurikA|4lYE;CjC4}aua?y$1gGo!YaJWbTU3-fJi_*X(9*L(030~ZtQ9CioZT!_RmYLY49m(AV~WdTyOij@dDyUdbJ z9SI!S8$6b|L_Tqe<$v&2rMCZKU71nXok;NNiv?gL8Oj-6U5(z1g|HZ>P!=(`cgqds za8CWiR^TRh&{5(9-f>6WHC1i7-E)@5vGaX*@!5X=4U*MSQFZn!zcVhBntod!Zggjj zXD@3+=eRn@Nk6%303V%Yy1i&m7*rzKAd=lFr@!BZh!@Y|Z0Vb^i^v`g3IsC(}B1`g>E-~$r zG=};sL-S!i-cso8@Om(T#wFDV;gs=4lIxN_J!z2X8Hw^g*A>T%L~h1@MR#iAZ0GO< zAC-TaS515U9kz(qRGh^HpO)~VHwm=O*8!o=i)uaY1;Gyh!AALJn4URJP-6b$Sqw;u zUuP+MZnjS8%0F9i`^>U~(lw+vrN=UGmG7o(;+bCd>Tdwm$omE@LsME}E@}E6h;%1y z3+Bf6t;Xl!Pu}h{W^?%=tz0APWHXF7Xp?akl-fL!GgtE5?9{^l#=RftUmr>r&<@SY z(2j7xRlyis8D|f4G10LRz-6{V9SpuVOInp@jiLh`zzC$cDavGxAoE{y!=LkqUwL<8 zVQuHGZh4Q0kP02gaYTb!cj16miM(pPw5J;pMe@Sr=(Hwg0^kD1*k11@TKf=eWSEw+ zX0_dM$Vuh*=Hhc({w8-4Z)BhR!XB!d3gr9pG+8oxPr3wv}biAP1Oad4FynAO;rs9H`n{3 z-R{nKgzK&Hed+WXQCy%DiiB!YFxZ-Q)R|H!Sj`4|8|)a7?KC_onnrT#Qa0R5B(Ts* zo#Z4?zU&%-$FMw?Q~}x11>uw~?kNHvs1b!qkwWR`;0!j4`Lr^O_#n*Ic~SK|q6AT5 zvKUFTRdyA zPmS~dBLac+#0MsV{@RcS%sy#;LaW8_1NS>D*lx29>wv9aHYMzV*1fWH+c5J0_{RU&>aTh1*~y_rG5EDW<2d=w!H6vZh$fX5i8|tlc`vPe)K%Cf<^L8&mi6ZUo z(W+~rch9;8e+haCUhlj5Yb-_>&qiEP!A_G*GXNFWgGXlEyqr&*SXSB}WYL-mTDW){ zdv;C?DE~WOgPI8bj#QI1axx%;d)hA50k^_w2 zR`wE*RP-w%X#=oqx$L@ z5fI1^751=_76x#G4(t*k50Hz8jfcJL#yAZuwK-14(Q5{qx6h`%95m^+QsH~!UGuKB zAYk1VeNmf+Q)3t=+AXOZyLOfp*s`D?x?GY5JLcSu@F9Z9sGh&cBu-`w<4M~~3V!Eg z8C{cbvR_mnG{bl+B!jezQzp>cg}sfP2^j+suEbgGKy7SjdyPZa2go3kkc2!hcdoZ@XJS8I4{#=^;IO*~$onjTJ z&QnD=MpgPL(L(x4T74{p_02+}WvF^3FL$4OVX|a{M@HWxyM%a{$%BE;C?CkqnR0&Q z;w12^b3xPl!A2{%SKddm4}@1em&L1$c}m_# zUtY7Cn+$dvn-p(*0+cgZe2w@Ak7)tF1n;h;6;DdwZv)oIUY1IVsEPPaJ>u z>W{nGIcww@C0EI7g^=d14f6QA^|1&1Q$$$er#cDod{d{7j(l&AyraWNSpIxlx8I+qYNFGF zbZd%Aa)%)X>Ss)f;?C~_b@c2S%!JElq-&Ymff!Ygk1zr`H^d8mXMF;=N-S-m9Ifl# zL>-Km>4+Y%ZEF{c)MydB&Ah449MX>j7u)@PFpSufZihQPjd;oN)yUCe415$_iN%vN z!oG_Yv&b8ybEfK(PTJr~)t|)-^&xcG@4wyN-o}I=-TZI<0;iuc$odi`O~es2bENS% zBQ94&{jHC{u{phl;4cl&&nBa?^FQD8`w!y-n1IVfT8I`Y$Qe-PU)FAWD3}uNIS#5l z23*Wh$mw4J6h#MH3N#Qz>U`!ab7J{Fq=sd!F)liXLI>9sTOV!NsCf%Mmj*ce5G|co zt~$u#05T4up4A-i2JVcCKy@~0cRgLQ=e5HX9~1@WKXH&+8NW%j~ z$@3>qk4?vC<|LMyA+Ot%^By0FmDaQgm7K2?``V#cEVAH@JPtv^zIn{y=k(h7MO`Dm zE&p}bVkBr?f@k{V(B_!@f>NX!J>*byRPF$3s@`7_b`ykzngc)2*` zD|+bP2SJ6&@RT0DdoGH0P03cT4{g$T2Q^zgrNKPr?-J{P9__MFyrYr;<7LSQXk%{f zWR_(>gHlVK)~TrD2izh_j}UU$v-y#)qCRMN)4{F8abEu5+Ag>mxjl!gF$t{E`~FqwN1YGwOq7P1epCG#FwzS=N$q1JGA^VIaxVHr#HZe7}rZU|=v^>%$%VGYvSr=b+N_QQR!Arn_(4&o@Frl%{ zAbZJ$-j-ym6QhXdRa9i#o6Z}U>ZOxgdZWG(*Eshuhh|1g2{QE!LJv0i35bvzbzbH( z4N$~mt}{0+`cPs94i4w#`LXvradpJ1+YBu6kGg6kQH12KCqJk zl@gRE^`z%m(|NvXQmU7K^XCy?C|d2K?0;1h6|Jtbw{wl|9#lsLh|=+YKD~4R;Wh)* zgsMEtuUVU#AT)WvaA?juE;C59OMYXo7WcLG38SwcX|VrLac@xO691`6_Q{GwC@D=^ zxQC2D3HFAFkDr;KlPT^dEv0(i7rxKI6tYVOPu_P_v*fSZ$&yZ+$GfSB;iZwUpP zJzH(z7c1p&>!4OkXq^tkd%H|H(9n<|=QD<=dQ}aNxgy5L1mmCs*51|n^$Q_O6u@K- z8M6Y$AQzFCwM01tUOEKArf5ZcTqigqQE=}6-6nbfCZbICjzQ8~4^|rvAIq5P><2up zO?t$FDwIn~q`&KkZ@oV4xGO{BJHWIuME@3}U1GhMonSb|IOoaMo;d%HEHH&naFMRO zv616V(`IRfBVeCzkQ?ip(U_PDyNZC6{Uls<^ZLr9dASN?{^#s%@tFTONFME73pQ6v zT>3XrIRcgsKw*=t;bo-(w8~inQ&;NrZo@|PX_7{t^d`^I_I(il-m?^+Nf)up*&~P( zH9W;gil*!~`1h(8oURyJ3jC!(?vC1y?pNlMWlFPt|o62ISMvT9sP z;4qGLCt~6wTVx7YIw3J|k@eBv2}i3M(n?*v-jUG8AXk&!pE2PAYp<*Jw>KJ081p4u zsuk)^eKo>8`rnJ4kdXMgqDX}x3>)~u$EX|YRiY^Usu6u%(dvP|M)i23T>E5-6$8V( z5yijLFpJ>^y{xVh*h2Sknn0ZaVFgFM%4$9K35E23ow2r0q#P&GgQim7Lw0>#?$pu) z2Y9613fRPm2m1hXX0b)Pb`vFxH`~DsY1wrcf^bw+*|4k%do)+RG7gai8Mz_Phqk_u zWSlsW-V4~`;K?c={tC?Mx#Ru7QtXO`O7r}l6yvV;j-raHc8LArYw9~fFe3)Ab)%Cx zf_A?lPxuyz5r!_eiOdYl5cj!_LM|sQ@&SPj@4j4;dF~i{7lp7qc{V!^AA3JM zr0&c;yB-HIp4~weq4EW7eQ(3lw2Jmth3++&krj1~s~v1p4n30DT(qU8VC`?F6BUro zlvFofJBqOBh0EZVL91$;CI6ct!BU1q|8?F9qF5O(iE)HRO+>#=z#-dRyr&= zt4dvp)4`OK#JK4^G>2vP0>|tT9xbR^2T4IoIa){El+o@|yOI{*r`lwj)6f^qtp&OjSm=oi{&Yj zHe1tR3>}s0`n{pI@UZ|U$UAiP?sS$pK_A~ZH$_LSZPL=!1AzyFgj^he@+r=A$xbQD z%*s-3L)HNUiOU`Ay_*OeqeZN6{cvaw+X;p?UEo|UCjnRg;G^4d6k&TLadmRt>^}7c zWx@os&AL8#WQLsrIaTV)W3J6+$R!lAWaivXPF;AM80RMj+lSO?co6-d%Hf}^swh&^ z3N{(`@XHgXS$K7{mC|10*8uE&$&e%zI<5^j6jerUR!~m(zdZ&U0ih6$-e%2s6kR9z z8GUEYueBDOV$JR8(V3?x5l``LWA^&&S455fz~nm#MKpcikj7Gv8F3QKTh5RRev34c+>#-KTHEho5O`$&O(MlWm%-uc*$sj)BYy7L`p zasxvne|=oyHvyp?qpri0Rom<|ZA;`sj&Q?|EEP@$Phk?4TaaCPyYA~h$qzLOKhX~j z5&Zlfy5ka?oEDzfo+BdZvWC;-9J`fuq) z*o-p0ohhfL0`QJ4prqtJ3H!v$b=GuB|Bs0EqfYX$G)pWsjw{sK+!A71=C9?%WmU`R zKZotPr|PMfOVlw z;~IeZhma{po()bQnm%&1Jt7+;k!Fr_m+kaj+tBeGKx+P#*6-SQN^s_zJq*dgsJ3X< z-P3oGU406~_sh5rM1d8VW{R?^FEx}>GL(WKf7Qb|v;Jpu*l8~oHaGkzV!`@@YC21O zLFUHso?!Vq&l8@1%SxTcZlj;QdTVo@Q7fJ&EscT-J#gG0FR<4b@*h1lAD08pKAOit zPKv4Wdj}0zf;R|5tsZeA$0N59(TL)eUdJou9Hg$I1)ld(!^V=DjermEUr^Sc3j_9T5 z!%U%L(}9A^k&&ggs`E8Q{gT175SfK5Q8ot7CX|nRHYSC6X=~%zTp%&Z*Td@O$tfg$ z_?tbk_ydDQ@>)x97Xo(RAQl8$$s<@Stkq@6$hQ3i4=CPL6ok6f?-~-%C7j_-A7t%oL z{O!dGd~$RS)d$pCAv9{ya+ol`kl=z#&e)Lxac34b}BT&{5=1R?m}-ArhIp0_&zaXR%vO?*-< zzj*S@5GC!#@E=q?(5_jlAFmCQJ7fLGs~T27b-D4Yb#mQ6N687b2w$LCBb>w+hw-U# zeUWCsjZZ_95>ybRx^7zVN}^BE)2Nw%3Fyb* z>;Pu;x|Lnwfq7Gy)$)hr$WIsyJv1OGBF31<~D#Z=w2n)&Gv z{2CuQWzspX&k-o%Hk4t+?g+@fzWmE;Q87qv4HK@}sP`YoJ@f0W0bN85#77ht;;Cu5*tlu5F_inkI z>ooSQX%I)f&`P=yqT{%$-s1g1?3;ZUWD+^Xvphfo?LYhN7p{h<$q1@M@Dbx$WUd;y zkt1uV%pmzWkYk#d`m{%YLr>_i9@CrxM;=`5_%;K>551aqwJRk~Vm{QZIWpK6EogM5 zKo!l^k9gG{Fo{X%3P6wyM}udKG6<9M$03;NnTkUk_mV*dPr7CLP=d6PuDoamlMg7n zeSX)L`TD&*MmH6r0?e>v0W?&|`Msvo>$O21Mca}nw(gO0(nHS!Z(sfx;_-B4NV5ab6ahQ)>8wA_zf#CFOgwN)FFx)m zq|H|Lx~|jjX3dm*f`s#?zXP|llc<&EbSCup3qU$CtmT;JxN_LX`xv>BfSF>qzp#kq zolFlbx=c4|@3m8OBUsG7@Z^0bUmG1WYhe*2AvB~@^Px7gVAW;FqR`_7JkNJHg72tg zYe>%fwJ3y!?(z-!3=Ak3guVm`#FW z?K^9ff#bKCLAopPzFVW-p#8Ak*I~!A7md$7D-=;e!ka3GKr6vD9i?>OVs;R+r;f1G zZ?^;I3Fd6;q-?){b4WSv(o|4-eW)}t7DpZ;OrG`ai8+XV*Eo_J&VUh$w6QAo-IF*U zrnk9&nb16UMdM5^RZ7OI$sTz%QNQueN|;@UnP>B(yImsr6_|^ zclQN1&3IDALyw?1WZ06G81pf;bvRy8yvt-SB;K5$|1Y`IMh%3cdSeg@4alT7V_l-l z`=j-sm30U2>QVPQTK|6~_^^d7XCCd6dkRyhjECyebq#`~(3mB`plBvv+&I{%$)lGX zgpa;zIB&qU^j!R^C;6^2nqhL9nkOovfZA-8atxvFRCTUUPc(y}BOli%2vx#EIwtrB z`_8!H2BOAb`F6;s5X>g36Of0bxJ;b(F#|g~dCOfu44(;#S^%?rCwqhBQ;9`Zw&0?s zR~XMNW46z%rkP0kOU|M73Uf`yLpmrb+^hIUV$OIzFUUFxloxx3mX5RBf)yS}G5rpb{kbX#JORnE3iCI1N+d3hWf`d?f)I!TA#UOl1NqJYw2J2+9&H zg3z+%_h>EPA8EyMzhdl2V(RmgIQ@8Dke6YDDY?jd zEb6S8KT;cvnoxfBC!=xNl&%KcD}8Ijcq}ZAKPERLy~^oZGNzXSf=OU~?vh|Zji{o` z0o1amNGmB;Sok28ajsW0Ye zu#xzLi>Z_-nu79-W9{2aH48knZnRg5+=P^w*ICl-htP=S4NoAf9zht(gmRB@Z0dH8 zh}yUlAZ93Ykb^4T2yi6#S>QAbV@|Dtlp>MS%}&0+ZqI zXec?mSCgZ})mYBt-VhqAvcX`sXE!JN`97w4gs=Fab8^R{Bg=ORTPz z()7WcHZFoKV<7saJ?;Ir;RMpHrNO=M9v

urCNydN4+Gxc4#)lTVbrDOlb zw{sHV{}h<-CjNt#fh{RYtn7~t10Qha-Qpf_CQ+5p0!w<#Dl#gr)P;h1%CT876i+-{!U58dq5meB9V_6VVKQI=VE}xswnIxhc%`BAU?kde+Jxf; zC28dAy;#d0jmJ*&Dqslio0m)bJ$R5 zQZ?x?#1w=L;t}NO#o&)2y0Y9?7g$|yhUfGDPR6z8bO#_+>I99)^1Ph0jV&2#laxPGfp&54*Muk*(1Z$jt67UuNIoEw6+r2u4Bw!q41 zMOMQnZt5~QGc^<#x7-b;vE11;I|e#wS5i*Gw0;h zb%j`x`hIq}k%Ztx%HX!cuQXi1F6I=O#ikp#KTd|yBHX1)B&#Np%)88MTx(`mRjKa; z2e*yM!(0;X=7`7M7YP+1%Lwu^vvh>6^P%lhKKOgFX8alVG~oCQalMPgkR^j^F;t+O zyqa2$i(E&2DF;?e%-K8m40hjdSG5E{Se=hLET}259Yk%?llFWfjN)Xi-$?jHKE#x@ zk8ArY5+t#}0@-e&?VTZ!JICQN3s1Y`#4&lX17F#vnBm*&8pd>s#lSY#_@?bT{MI9< z^UhMI=1DQ9<~6zmjN9>lQY<}4z=@C)Z7+Kp;B|g)>lpDWsE9&oGi%AX7o5eT<-j^C zkon3Ar=EGTc@!=p;V1M0MT}XsmUcwp(I*VMU|aSbG29DE*1Rg=tYM|9gyb+lL5jA*zR;X9~Yt9c+0#%y9U5A15|v^krp^=tFBT z^(7dPzn?_jF@s%fn?q_N**(AxR4e9b4_en^EQ7{0=fgZjx7IXj`X912JoMO5njz^2W2E!hSq-$ar@Wd7SJEC>VNan);toB#clP1 zqG=r|n53FYqWOS%jq1o{A3xPQ_Y;)UTpV%VxIQvY6ew^I<;h01p?Q8r!`0^8KlSy( zpLdw^a-w&DM)yYYdLaIkJyLZj*WN3CP9QUkRd}3g%Wc*!%p%3Zmi&tgOZs@B z`u2zTFR5bv*{+mOs~;A2dR4}@93t_`@@FQ zM$3q>Of4QnS9r~dU1?kLyhkDgmBnu2^;*7!g-A9VcoN>Oi&O$^UH$wl=Z6PZ$_+ll z!(mTR{nI?F*oH#_q7;}aAJ&`&eKRpphrjXfNe*XcG*T9}gT5OqJx)$dkQ!lnVqBdF z)7NNM$h87F08!F^x_`X#BE5S+w;6dB@ zk&@;GOH~!0kI20p&3<}a4*5>Ni@HZ2G!`ixw{M)`rY}@rK_AO8Zb@W<32;7WILNn` zNjhPQKztG>B&D4ucFmLtY26TsDMwdnTtp%rV);!g|A($mDFrmB@5qjEo471L22_HC z=TLu{{kfr3&LvZO+vohXtXi*lu1pR;;?cbrZ$;%8)2;-o>W)~jvD2B|W8z5|6xYuC zBKJwAIwRM>HcTq4+NS|LP)yq_T_DY!xfbwZ<<zGFq zn&0x$0>i34i1Jisj+$uS{~8!rm^h-t;MG+IST-eRGJSPOd#~UnV2ny)bRr33*}9I_ zUtl0?=S4-KM4FJ3w{<4r6uB_z2uh*{Jg!ozgVQoRcXgg6#_(Eek|mp{B~dr#lyMu- zQ_$swOYsQ}zFYfu&j3*kqU^p;Yb*C6MyJ1ZL_51(8EqfySuX2zTnDOsJ^bG-1P!0BkT|_J41s;M>rfE=K$O2fr{JyX(|0sQioC zDTVlK$JvP7Ix4dxWr~71rj6>yb-+HL%)diL zV&Q_UnbXw}2Sp(CMiRELY!;bc+?9E9^+ZDe=at62AvAeE^jd5;`vTU$8E8JQGf@wi zMgBXhns+VxFzO`aF%_L(e0qy=o|Rs>#qynoWz&S$^t4jUYN0KVS!(4`Qu?EMWalG? zi+;h{7PqFN9hFD)+q@GI&|gp#$vQVe|FP&vi1)+I5b6!h>(~N;F+EsSBl&K^#^~4l0Na=&UGj*p%Tr|+yVI{ zFLtA+p@A-rW<1Oz*2Aj}l4QNmT=&JZwDg+7~Y)#Qbj02tOc08%b^41m_}-nvikh1}`ddxJzyB`GWpAMIv!c-Q7q2emuqedoIDn zm%l)qeeX2BWJ?6sHF{|Pb}L&EoIqZd^zaIyWsAAOr`qx%nW|M0UmR~+Rz2nL=y8T9 zHkc^H33EGeW@2}QbVq3xD}QOQ>PHUd>Ks2(35kvndQ#cMR(KybsV^F0?1loNUvica zQQ@{Nfb8p>W9Fy{IMl@KYn z@Yj673)>(=XmQIc`e)6gV==izOK+XX`WFU~T@(UXv%<7SL9f(gW&ws73nqlD$7ft) zt^B$QE=rjF0$^e`MuRbOQe0K*DVE7_JJ&Vt7>~ zZY`C^lflob&Do!L=_N977g z$~8PldR`oS=5G^NeMGX?OpIZDy2YYAPZf4Jw#k;RZ1|(4=Udu-L*ALWiC^8Xi_&pg z>k~f61KE#nzM>JUEb9zgXPmn-?rX~x8FT?N`QTbubL?F38VH1?6<|bTAY&yh5rtl# zVJ3uwG>`okEu69UjX+;K3={4Wr;CKc%G%-b_;u{01%uKl#T_fP)TuE`AIZ;#eUhw4 z0W$NUWh*9=*OYwP3(I=@Kd}~sh%*v?ZoUA-@%Xk}5C-(8+PA^N_@!?Ax-W(#7fjyO zsd)a{a%SsUrcRIs#O?_4<$Pcu!bqZLSU(tDzga10@Urk{vTRnuL^P#}^9{OU-khZ( ze{9LfR}gfde|3_1{GZUpaqfr4%56#u|F#7b;l}-$EOkHDp#VPYFUp{Eogrx;GRn9E ziy51@)OXY8arB{odonD|Ti-!a{Szy=KAL*Jh2q_l?+1TjH&`UddhH%x?;| z^KthuSM`{3F^n&|s!1RC7S~93CPkkR{!pxDdkKWu5<4`|F;T8Gy$hNP z{pqWK$8;cm0Srdo$)(N2<^w7=m6SZLB2IQ-RzNVpeCqZ7(79|nG_7K#GN;yL{+xfI zts~_)Wv^?|i|=Go&>(6oSG2ygG}0)09L1Z15IZj-4U`Lb zJK|mZq;e}DrJO`zo?X}|?Vf$fLcclhJut`Yy z&e7b;K-NtgZLin-w>+|!c6#RcHgroM8pe$`^x|q&M+pFVMOnUXv%wDXahAo zKroIi94BZe?LZOPbJ-3LyM1T6yC4bB`2tEUR4U27^armEyh*4S{vGGZJ>bq!rr*{v zC3D7D+4ra$U+y%a7bKnQubaya8o5eQ=TnR)y-+Yv^es-aN*N||(jos2F&Ll9ip%r- z*3PFsrjAusPOerCq4JJhI6~s?-}%KHRyJ3JI#I8P8Sk{hBMzkxTjd^Qw7YkWHj!V}qQg?2CjCjZ^XA}6lq!;h{6&aCF|WIifW)MS#ydKV;*nZ)ttOer&X76FjA$~e z{)N+fpd4O*qJiNv-q5*&Ce!y&xUQgPw@wR@fE4m0v*uN`@wYuJz)b`2tUA2mx=QyT za>EY%C5|KWJxGxhc=M{PXu?z$9p+K0sB5)Oz(*;eeH0b)Px? zvVTelgZ(J?i+_@zNDpRRu-Cj)_5&J*8_s_ONE{8Z5DVK1HLAhR>mRX5H52VOzDFCD zY;r;3yZb>)#l`W)^xfQb!Dik?RiZU=M3kfrk<+zfL`q2o4Kq0e z8Wj^HoX7EMfD`(yjLo#h?TESJ2T17u6>)K97E>ky{_@7b@A)y&M*i z6R3*DYK9|O^v{#O?RaPo`?!wdQL$x^7Z@__*uQR#e}nK1?~D*K_EoWU7$>=W8Gc}>O2k?T6o;!X>!5KMk2lDfY^5?6pm^o6zAWCP5)+4$hC$RYq(F{E9@e6HWLD2Bt7?(ZrkAUrQ)Qv{oSU)A}|`7TAkc?mhZV+6cu@ zpsbTP%ZY&(D3$~cFv&YD%|e|UN>r?O$)29QZAKNcO0z&FJ(I1!q`C+^Y--LKI1&5F zD&oW#Pk}ksC(nc;&ZXoCwq`<@Opfz0+t2quGWsTpU-aa$Sh5R1i}#*LIX394aNm-qM1Kk^$IFFATel|S!E^UHR#2n*H$62e|a{sj%W`d>B_gKB@QiVUYZ)o`_HP~3KlP39`O=kG5>UlwkwRGG99 zFID`q$_$oJs3JlXXoe)1fyD^cO*e%rXtmsKZ^vq!_@z%)5B#mvoKi?|z)@*%nEyl$ zdE-eIp28vXwo%NeiVHZDh$UNK0um%xy4algP<5#8yMfy;jk(aUkeK?SCFb~~pkq;< z#|i=y7^&jhkHPi~kMg{Jyu|#dBa**(H-aKIhZ?45y8|mih=|vG*yp^cyl=su8lJ2O zs;DYIPFt=)5XOaqbiGu9ZBLKpxy}s|R>9)(%w1;%hxoHhni!JY88?94qySA1;6vmZ zy(&9kC$D^_hGE*KQHJYHm*sBnk+Zk}!zvsIQYezGND)P@{HMS?R11XP4WivlwLv|r zlg#w@lZn3>s6in#t=qiU#eRz440e?zc~136psvu8H8+(L(xGPM;5P7Py#>xl`Xe&iX?X?D&4MOn^sCQ)_6k@p=8 zLrG&N5s10veCBvIwjft{9SaR1U0Y|Q@~cMxjkiYf%g9tcmGql^z&?rAdwdE?JkWGa z65u{>NVxE>dMIfF^zB7nrZG5ktF{bY4A}mzQ;UNdn?+FFUE7vt;Jz!M^kz~5fZg1r z{n1vv^HD1#+Tuj1{zsz`0fr?4Hs)`tSGMbLZ&bqfyh3sW)=c>@yzws6Gk=T#+BzY=4jW&|+?Ipi_^SiW0@n~i{ zUy%>%;P6t*d+~W~i;DBK`*>M>UjNjSXX2L@qs52}%^?T$NYUo$7>!D^!=W*Q<5KEL zZJ=Ab34Oa{+$xu1WS6)j*Asd23w2+g5=V?DotE2I!C+#GB7N}9WtP6@QXWOU6{6Vv zM+ADe4D5sWhMOVZ)#OOHIq^F9ez!PmM#jr03_e%`t4%TFd!o_+rM)j8_%?cT1RClN z`f}ukKHHSA#dG+-EAYuIX{*u*F4bq91GH!7AGW;omXSIUb*j}ej?0c!*!3B;r6F2c z)N7iLf)$^C3k&7>*lUk8xG*+jBH{IB;V{(6Vx-a}?e1Sx1li>8w#7vagQ{^hX$^BE zFYE-H@|Y}+h7>Tku|TVClIy^-9+2A0S-kE8e7%loxZwH9$+5p7W}|+Pc_tav$atge z#{82cw@x=oAQA_s9zp_e36)3v`p3I$6-%|+2$Ycm!(~D=V+iyKPSVp{h(NC@Vlmy} zB$O=JmHAuQ9RZ-qk*6gUdr^r*bzuIK3kWsOW!FWWNOpbt?7MOEy)t|EGyJE)O*D8~ zsw~h;4+jf{P4o&;ywY7JFJKLi?br@WL=st3x|DBM?&s1GEbTO74ZgHRI^!T=)xH#q@fb38OZ0h^{j z_(FAK*2Vf`45NZwxgnTrcUUHn9_z-g`M)%?~dsZKXr&93W2d(MPykMiL>K?90DVGZFVDFnnO4iOk2 ziML1VmQ#B4RA@T`IY=zolVtEuLE1meHx3aN^n#sIpz*3NdUHa&#)0ua0XngrAM^Sf zw)2qN#!QQn?H<+O`76Lju>}e9Gb|n0nFTyjGstANm}@dKSsWvjkO&Hqm2X02g&K!A zCD3iCNgBT((ODQOU53%GWyhwU7o88EqPaFMZj^%&6$^w<_LSsCoFBg=98yGT6FYS_ zm{OR0s;LE58|q3@Kn z=oS}wGFW!{VLOr9t0=;wKJ&98uwOcK-E&ej@PANO)D7C9lJ_&fXm^QRjqKRL z804i{?Td4UE&WqrDWsYaj+Ws00KtFd&Z?q?m3w?zv@$gq(3_!WH+92#NPC|v@`0{X z6lLtEUBidETju`<5i*) z20$n}102as6j}+_hB&OR6PZ~e5F-xE!++C@*HH2s8`1?|vql-Z<6ui`!F=Iy@Ow}? zqZPS;_SMH*O191FsxfFRPHuypJ#kod&}fE9UsJDk`(LOPNs|X{9XcNXcxz!ssS{r0 z{J4b05W(jeTxo!LQ6D&|%ri$Q$0(V9`$k;bY`^mnDD_Jb%cVhx+Tq?0hBj8y0`hRT z>ttfFTH}-=mEAD7^L#(x^gsplsEZ()+Un)Hh%8vcj2`*D?UpR*^<0OqWuQb6dFzhn z%|uz`cm!6Durr2N=_U8Z_C@NL@Z6rRXt7H>UF@lN`z&dIe=`q>2 z<+cvZYn_l>^a+kW)Yi6M-DHxc#cM8F32hz^mXyt>B2|s^tnL?NL&DzJWznbWN^6R^ zH+jVCicpNN+C#HXRCva6tD+7U?E}~EPA2kGOhduL)U{e7f{Z}9AlUGiU5O5ahrcVR zdd8_!nD7c)X%t?NmREl--%(L4PW#XZN$%_q!&^nIglnTOvq#gXHJU}S3qCHp%%)_&8hKxX+gq@ z!}A5fAxGW_wxTiIuHb50HgqgkM=&3?j;AEDg63R1sg&Rkjr-L{qJBSBIWFgq`_+we znQJLfGv1vwHJc0_tY6)-Q!@i;lsBZhKZy3AAQ!=7^M_*;QZ-cxflZTddrgN65O(mW z{n}iy7Clohq5VoacfDnO)yPODY7TNccQ!<(KScdN(elX!^stwqP;;=+AV(tDP~dXE zoFELdhF0gp+tCXSBxlb7_(3@YAO82+xJ*1=?Gi{c9CAg0F~H1H;NB_GV%8J=+O`<3 z$w$tk$YUijF;9Qw`4MY4#F=SHGy2U=8J!*|Q;{#zeER-YKroK3AsgPNVr$;WzM5GR zj_biez0&DWp13=!HYhoVI)4;q;SkM+3K3*5oQ%eVZI=VG1s1f<-3Y$_O1Hzh=6cn* zQGi2KUFOI`5Q9@*ffu?_N*$}!3frDThvU#6g=l=)3%VEUgQetNu6AF|6a*Oxm9^^# zS7@pTSG%Z`BrBxP7NW#PXk!A=3Bm1GxTQ~bky}G9QeZe;^gQWBPSVT z(6npQ{Bq0UdsyrcSv0KE)fd zmjrbDw$^8U6(XT&XQmRR5%uMvN)k|g=KarOZGsj57^`CppH_5D2LxJUXd z;Hv(fczhiPE;=?j73#Veb39%srpl{#>mkU{zq_p9WKBGEEvKYfZq2038ZX5 zN}1V{-JU+ntypsXrV5VDJq&ccv3nn9O2&8wOG zM9|S9tn0~FQ9PCR{}w=0VW)aBlA*^)H&?GH2*ySU(v5)l$K%6ZRQQKaI z8pGvA#2*Mf_r(NrH|k}pzp|o-JO{}utZR)zSFr?cYmo=JUB2G*q&(WHd&Re@Txb)@ zr7u*xy-v*sBHMF{-Lo-abjaeW(M&Xqsq;&p2$AwDA%wg?6a{@SRt-1C) zJD~2sp}5kf7yM&sa~u4wMv0K{HK2~ImwPfB$j5guAhSX_iQPG8uWtn|UlP1uJ?fq8 zCf)G%Bbv(}z-60rZ7U66{htw|2+|+&h^9oXVT07rnZ<#0YMT$XmcjK3zu0A-MXHs$ z+v}mW8`YrQ%h++qE--Atiu@0>+GCb6?Jq%XjF-N>-)W@ftfEC~+9W%QKk1h(iPM=O z-d3$$;ID5Td4AhHJGr?0fvX3TD@i&tDQ%6_Ef5UvA?y+ndZ=oncj5W=Ha*eYf?~lb zTY^QJ*Ac-=Tu1b08QNvqNsqCRys3K~hhM$E8tjQFEtFO!ZXk3N zH&`R)PWM9VwY&6E&CM2dpT$l^+XxnDAhu7T?{T5>032`UI4|?|TXt2q7d~v2J)0$o52Afw1N&3m`3r}zkhzJp0aX@5Js}Vv-VPS42 zw>_^Gvu)LY@rlq+r0|3YLU5xD+eo7=n~ruqqVLVGNj;$uZRraOP)c?nlToTDM}RF~ z?~NifJM#H{z*obih{GwK(oVZ=GC>56i|$6S2;Fd|;1!O8K&l7F0E$Ba{4GsMVmKE$ zB6o9$hIq2^7U|uWu}pNpFI%8VvZ+}LeuW4ap4C`K-K$5+6?*Su)yC7mXH=Wwk5)%{Yj@9^uRkMZpSNdG@>SnU0_EMuHyfFMeby$qvvXOW%!Y6OUAy12MSP+w3lmtBArZ-!YXw!K!-iaSY&q0o(BsHhnZ z9;%!CZL^JU^YZLuey9$%{jxTsqtA0bkfCW6lV;0XHtcTnrb?SY5xQzR&>;VT~X-4tY+VQ&jx+35EnZgb3i7KJnk zu16S9Za=348KW8>2Y7iO>qQBo=0C5T4oWk|$C$2Jtu{vas%2`)7^&E}2v* z2=;GXxO*CK_?M^^wuYiY8+3>1>|?%m4cD5z>~M^z!v~gK@ZW8+0NOuSG5*^WR|>Ux zvnnl50bMgfd33EIEeM}{2>IFFnzfI4^I8V4$D}nx@0mgD147S1Y658cdLwnvoUSjB z1}-4AVHjuhAkJBaC!q@V{Sg&x9N{RUfVo);>8XX%BZ)IjN)a}K`jBg?Y(p3WEgc2{)~bi zJAh4ffh&Dup@&WaW*7^haA1)@3)ZVPsr?@Z?}u|EC!wn;^IY}g^xT{ushEk)8yf|r ztiecT({;hQtk$PB^NB!iEv@$Zb7aa-Z>=3R3GvV$DilIweQn^#; z3=t-g9*CZUY|#fBa7_DNS~YTuT!OK?{_>AwQp~T&HAT0S2y#?G(Ta705laq(Iy7_K zwA2YaC3SY8*d^xeF`lSx(1%<9gG46fhh8~qQLU?9wf{N-9Oy}z58tXu3pW>o-&)*o zyl}*sT$cEkw-A}Xaj+@zPF#z7KRKPN1_rNn^V-4hrZ_i6kSRLL^Jj_(+fx$`Pq8*aW_D2e94EuSxs5=#H1%i$d_m@&2hb(gM@G|xjU-AC{H0;8i?-d5hx zy4WZE0mM10Qbg4dCH$gx@8aTqc=>GiZZj9<;Po>29q?IXS_G1AoZ}f#*5Wws)M)2}?Vrqs)lE zTXNg{v9Au)azSZ^&(nnF3da%c_Dan&pXADXtvo?)gpuKxG3eD#srz=4ufH->fJ>;j z{3_~9i>H=BeL~&_jQPU)Hw{$CFCixQ?ySZ2)S)>0mkRzK-h(X%6+R&+?6*0En3(f2 zNoEUe3e`Jp$T`!)>GC9W<9z1?>FD!yMbMAzwBiUNy!@g`VMt{+u)w1ShN^US>)ixx zY-0-Eg~`0OfyfHxFlF|J)~-^>V2UfB^UBAfZZ}W#*dM_&nNpTWB(a0n5^r>EuDvK_Lr#SJ)0Br#QH<>H%}q+90;xB|XlRH%s6A_@ych zG|B#(f{8R}I(%!pA#XR6p>PL%n|l;)H|l?`O9gS+=-#Gg^%l;hUxbd$WqK{a?PzXG zpvR9$de)ciV()=#`u57JLah@&u@;%inzv9)Qpe`{{A=K>*tS~l`4EetH(Zz|i=a}x zNI7$Mc-fLAopjT@-xG*l6V^7rbEmLAkV)y($(Li_4BrZ9&i8=9CIgUaCz)*-f&Z$I z2MMtTJhhjTHzQm?jY>f76B@bzf_~u))Ab(q$#LiAg5amM#%1~g^HrE76d?q4kggGap+KlTu=SZ#ikvLbHl1ZXMg%nzYNjK+>&4k4~cEK zUnvff^pR?;!WLaC9VNi78$H(`(Yg38X(|66TxM*RwMbk*5g^W=>Y|E~*3HmXX+zHU zD-7|LhE2p+JMWp19cVsLeOw_ey}G~t@;JzRfh3wrGmUs?GRLUS{cV@w)^5``Vo?E$ z4)sO^r?%o^;2>eUGfp599N&i5)KVO1DXq4mpM*b9~eBz%3dxUTWKp*pR}HX#x--S7J>YoYc0N(T{Yh-8b20 z#p#hX!{}Y2JlUI*9vk(*HGQY|KA$^jYGwga+nwhuX5%gquiBJ4Klsvs3N5;3X2z&X zg@#{VAH8F^DhJQ%THqI>Kk;D53z|2pA+f1q9K~(9JUytCdR%S=TA)p+z~f(2oYyFo zu+N3I{7YgJoMXr^*sVK{OP2b?>#boAAM>|vEsvWRbD$Wm%Hg8wBSBpG`4)~4+Uh9h z`(~Q~D43xamF__Or!v-h^#Q)s*hG7b#gr7il8W5;c!#F3<}pFSz{b4W4vj8 zqpR7$_ruC1pgYXwFVf|`q)+GshaJ4>x|oKp?8||2&VkbeP`Ue#QwjS~E9XWktw{Bu zLQU6 zT8zoZz~57ic*<^I@RJsS&z(&Gj6^FvYLjVf7rsKT{~=&IGh39puBwvJ=(Hw>SG~D+ z>n5-!bp)%QU2ESQ%Ns*15LZ53!~Vn8=l^D9ifQ9fncuB~i(5_L<4l#Q*D(u>^Km?~6H~7eA@s~=S{3wdLT#say3?shjrI%AFae(H;DSq$vYnWO0p|fOv zFy+ljA8@*I{|gRwOYiwMKZ)!ZsnEX!0q6-*>i(@_zNZg$Q{M@$?RYf;WgI6R%)^2) zA{jth=nG6BS(C!WWb6KEKIn8EaG~B-@yd5?NW8QTQ(DjuGW{f)is^1eJLP0e@y`bc@tmogrK*e2Sp1P+S-)yUyI4ffKz6F)J$B7Cm@n*;~XFF?pTMht57wPP2x6 z1W?Z0LIF^9u4$B*is2cqmI1cv*7S^ljaxJ#5Bge-84$ZzWCa^zT~L3!9kmQlGLk|Z z<8km`NmMg+tb+mki>mO!#w;LODt;@`Kpe;om@@Fe7bOkypwHW0U`45>Xoci)?QSv$ z2bc?+owLURdRp0LcvE%yAj%?cDl`qj&kNie(fLmL&})=J>1or%T_YK*-kxd4HOly+3E}2 za+DV*RQ((zhOE>qQZ0FlL37D0cD#Si<|W78d>6wXEc}#ovgN29{MCUmE3=8qOPv%+ zIzO`pxGEbQ5+ga=aifkrApB;@N6)r`D5++W?ISzSJ8sks6O}D$|BMIGhcHGCbV%QN z8MD)wc74xYVl6#o9n9N|UFav?wuci-PReZ?bQ|>urBb&k)>z1g;raoVezTxCd7B14 z#b-4MBRFHB%mK_oq5DvvxsMDsLf3h-_B-ydtT4q1aUs=wpVPUxXg%sA8>z0;VW>fg zn^?LSWE;hm9m{m|i4yrJYWyl+{2=*)@GWmN`mB;(fqLLcT0d^O-St0vS89A!nz3*@ zp{clhKr(V};v&FY=$k)o6`iNA=>u$j0F=*KYt}q~1awKfCK$rDr8z*R7@_&lML>7N zZI=_$fwSQ=yNkbamtl^?N~_d7WSBL^h-vv%#%@+g5v|~7{@a3ZF>PE152w98yRzNUY?_^I zN8#&-G`t@Mukn2$rnd}A&tuufs-j)&uuD~pyb!ueYD#p0nn}lKWIJFkb+E41({qcwon&02YBvp<087O8y%Eb8AP@~xY?EO3*d7^m~3QR*W>;ud-?&uv##x6UYdq|Dy=nNBOqNG{u<2 zdv4~&;6SBxnkn5wE0{H}B>}1i8*@P(ME>a|!C(sHUW4{QIFGdKyS6W52C|U?=9wGG z@i0>q0_%;>;lBJ*ZqTN@M1F(XGcG(F)X;E(5!h{VZ#5x8`J-Qv@Kur2Pur1i>LzNLw=%T1nG{su?bgs0}>hQ9Z0>+UGN??%P9<~)Pzr5v598+)PsLLaa37wh^Ly1ls_x5mz z04WnC@o$1D@aG>5Gn0d(PBZxUrkw|on~<{DXOapd2bPtJX8}ARixXzkHmj08;BevA ziK9dsiO?r!$YbIdTRk(nk*A_6EJOdhl1h|+5*D5%pfbhz*Zh(fYrFmAM&Vq{nSi?S zGy%NBqQJm9d zh9m|QeX1`(Zv}LClkqvoOBT%i5XW=%Y{GaFKp@yG#1&kQ?Mu*e8JLxsA7#Bm?=J+% zm`fv3Q?k;*C)K8w?PWv=ae;nLC2L=W|3?hakA8Z|cIboYvjS{O_&+JVc+17z>kU2jwSI*u29L+_&8lm{2 zpyD*nB(f-AN`;r2KJ3A%y(4!i0J?ta6{Bxm@RgZZVDg3f&2r#M_-e<^jr5Gnu3i)> z5UneV*kV|s4V3`4$9@&%akdM06IFb}Xn~Kz*cD~4^z*Prnn?D@)}}f4^MK3U1>6sY z`dl0?hvVaQ2_tEGdEwKBou7XQ=2lm&^4u`G8@{zogwi24;L=MKk0@ybC#}jsJ z?D3j}zi#8G#<-k&c{&j_^Tl3BWlZAzXFs6ARPGMlqIj%Y zA`F_+shV_g;PzuToUrU;X3FzVOkJp|lLysU#%D}LPS;h7`iWIYsyEIN&9<>&@MNyG zP)30%u>6ga+vB{g=_&m1BHdzKI%IGP0PC65A;qCu-jI(~{9(Y5{Gc`xm2QF0kKrw#qj-(&=Dq!)U0t$QDSGPfdjDv}#_OD1 z7U_r}85X~y{p7!-040CB-N9YW9n9+z2M`dKt^UG&ZI&deQN#!-R#s?5FUyKgG{{6EOU2pc|shts<;!R5-ZPY@#V1E(^M~%>T#a?`7tjssv z{Uqa;U{6;u?9o0H;>NB1IP|lcnAcayZjgW*7S)~tV7 zgH5F=Mo&Jyzb?jjfRy$Ht6vjvl^s~6OGl#@2`?CaCPC&9l{=N>C9_b*MV$>gWQ;g= zd859F(ke&`%M`Cq7_bdn+@pZ!#k#E+Ib*{UeE2!7#^!&axc%#cyIBEj%L<@)5$V5^ zq|1Ux{3-RVTGoQFiDqs9*g%FJ3jwH-peO%Nle|)!XG_d$Fw`*$2gxdv{juVn!M%wC zB#r3w&BS}uhp!`-$df~wUq}wlSdn+@&VK!WxIl)D*sbXNjzOL&{pb&rv=4Z87F3o; zj_&W38n%{QeC!ONs4bE;@xiLhC4F{7XVlcDTDKI5Oa02XHdZInmm67j^{D=i{_K$a zRjElhAC`i>a=nMAPrqHSF8>B)9z!D2#gX>vj=4DWXB{4Bh$)O`zBYnsLI( z$DK0Hk7^oLcG#T}>M>$GPKB@j0?%_p^$_&VfA06pRQGO3P{H*Sfwu8}X^&68nTg3; z=afItddDmdK^B74J5DpU$|;;4l6{m{*QAt;CXW8OLm#d38v`>YvKfmtFIPdDh=6ev z*WkuF{<`w)6SkJLm?A+2m*kIVy|YX|_Nl5&nLkF*;j7r}u-Qw~FULEP8EwnykNT_N zG-694i?Bzg+OGGkB+h~;!3^@EUS91-#JG`x7h<#aYV=T4&STUUz>YfH(Hz^%kvS}b ziT8-IAeZ^y=ZF+;BOE=#bZbdwVty%H^Toe#wK^lRlYMXK7-ilDdKT6$gr_c*-7&(b zrq%t2Kwp_m=q(PZ1VUM$dRVqW21EM+M!9%I%{@2`JyS4zHb7}ic{W~$N@sTYmrCoZ zRlibWsH{QtFk1_hvjD`HJDJsjGMOo|-xVy>aO7bzuGa9Aq(|{^h1VV)jN@ z&Zz?sjh70FhbSeW$>O!F>tu=CMsi11MqH4YP}XPM1-CE3hMMyH9S5DrG@;0wYU!f- zC(lM(O>C|h55t)!xfF%o^Ta>^Uc3ttrfw}invl>q4HEmh*>x$x1zDa0DPmt{{?f@6 zwG@mto1_8rVy*Zl%HMe$0dADhGss%1$q3)9EMr7$cr3j8xG>Z5UUgijnp(pA*u!o( zZWxTrlbwyIOIDNS_hqK^NA0oOdrWEEh9xk+kXOS9BN#BQDBj>Y1LKET2{Mp)RiwnE zfiaWBxy)h$UWZ@!neQ``PA=7S$CaZidqT)O@N{xc76d`rG_bn+$$iX-=tVaS9s`n z?I8Eh!O-+YwW+-@$_ZP!3%A_)W3nfVVu5mO8(KA4tzOM^o@RV-f~~ z38hN}nT*A#P?CfyH~`+dC$FAzGtS?Am|8C72tx7t_43x`_wJc(f@BW$w#cO)-ck=A z0ncP_1yz7l>1gx$kP>g(jO&52uHk-+q!N#Qix>L=GTaZl{|6~;HNm3h)z#l$z121J zy3gGrido>6c3LjbJpGL9eqSF&7!&W23d3{3dS5P#S-Zt;S7;TryQ7zid6&z^t7xeG^U%eiJk8d>!$QsIp&KngbT3pDy zmAUcbYCop{N5B)hQiQ^7(^3J6GvsuB2t_O}-p8KUg|lI-gM7PMNo^UBcWq1^&Q?%>RsE_DuU-;6Av)Am(c%T`3- z0oy63Y|XRv3=FQ(H$vBUGFsp_9BczttCNO*FZGVz~I-1aglEmV6i=inMocv8u9X#Y`&#7j>0bl@KbeVvC&w^Un5v^edqI=O`(|D>aJnYHgs6b-PThNLU1<(Pp?m zh6$uL5cKARTbDf?gV6+_jqu<*4~)3vF^i?zeYl(5Q$tibYo1t@k2`Q?0!~GQGYxp_ z-{9d%nEcB+YNpdCHQD3?<3Or2?&Okjf!0yDs=X!fgc+~F6ASN~T9B188{fTfN*C-F zx0(gwL=~>kJDti;h2pvABoD5d`}lbgLLD+#sCT0nfA8;tEglaUu;C&2r^FC z>W#A~7ZoKtvZl<3)Rs7@!F#W9D&gM&89kfCgHy>m(gcvz#Kwo3(VTrg!xFA$Gho8p zh%v=2V-^jR;i&S|UQg~R)xLBt24w-d0ib0A>~a^6!=LX&;k4je*9&{Kyolc%KyRWJpFdl)e-2;fhExj1qLac~jxi^+7-toq zYWFiJ=RoeIn6}7cRgs+}=x$9kQnqN2*_LbtB;vHO3$Nn3FVW5kAZ9bE zVxD@tMc?0TBgo|D&na`RiS{k#(1t%CdU@%xkZi*dR7Pe3_b;DdGzqZ0%QDmg_kt@_ zqLYG@4OW*OuMcRf0Wmlv!s)>TVpT#0JN-iw$$4TC@ubRtr39M;J?%7flCA-C*5YCA7m3$hV z@z|+`f%{$U#npyCmZi)@N(WTa{f6Bg`3`wi={?VaBR$pr>|>7;q3_5fzkwq@D>Ay= z=4%HYQHk}oTaDsjjViGYz$*0l)a(lbCrWn#c1!`ybM0-rJbErF^k87S$+FR%sGO8@ z6u!{c)DixIsQ+`i2mpxuVM?C+wYuc_3QA!a-zp-8b)JfuHq{XGw**l5q5{TdR#r1( z9-RqMxxbyZX_=nPuBjt1yM>1R4FR{ke=ufK;HPgAm7kbdb}_ILnE5ksJ}U>a-9Vsn zGlFM-Piu_lK>RNOohfn^L&Gb}hsVP#620?6Q<1o=(5x7#fmw{1s&H4Fa0Q+Z_9q4G zt&iJKA^fD$SGBgi$-x@CCVsiIGHCC%71B4l%_KtfagPRr$O?DOPz%OR#@oN}p826(`p zVd_&B5m;++sM3gz9j@o0*Np?4e-m|?eiB#9WE+u{<7e;}xVS64mqs-qp6hou zBu(CMgVTV^X)ua%o_n&@SnB$I+188>5A$So$an89TIzQTa?t+nUmOge|#xMAe?%O}YReL&m5x}rc36)_Dt zD&G1^2yqL$`C~zq1306~bm5!`Xb2(sx2|6s8D>n#_Nw)MKVDd3+W*Y4n3l;55;CV9 zxddoGa{|{WuBSTXCpL1Wmp%O(;JzpNN9ihBXin1?^ZO_tJg`kC%c<}lR{+G0$-=%IN zdTpL%FEO&r>>U!b&ePT5aXiLMim^|ffv7SaeAd!({>@5n){r)NNzv{5b^2?RbC$HL z5fXaQD-(_6mCol&bl`XTHq;LZEO>2Vz@`-7TzY*aak6h(9nV(|$j6K9}>{W`=_M35nLZU;zduk>?FE4_BJHHE0VBJOE+5bdLgjS|vqN;ZGrIEJ#T&?tZ}; zHc*~6vq&6gZ=8~;vBKC?&z5>`;oKeqO@nX6I~@8)%5aBy9=#4CISe6a(tnf<1IUl` zcP`zvLUvSGFxVc^78~xKms>gfvIvdFwU93YXFXUnB{QuyW&1=x&?$RM&6kCPtP7qd zB&Xm9iFT1qj@CriW*OM@9t)8c-o%%0@tVA0%*Zrf)f{mS!rVC1V!(z_oStUPP4NUy zf(@P+^C`5B^^XkymO1H)sDh87InF_QV=x(UI?P$6ufC38wFtoC70U%Ija57Uv-L;l z#tvFg_58UQGLb?&K&ETL-_+0RX0vx4umfQF4 zUa`!`jAj0t)g_d|u9iU2*^>%$L!MrD>@eQ7E62v&JHacNoeM7#PW`la#~KvR&q@7h z|N1bqS}x!g+we?WC9yQI)D>~CjM@1067d0u%$R$V<2cB-iPxz~IHi^wv6|g1mFzx! z^=@+CUJa~Dz%?OudCyHr7i+&ihrQVd@}GTqku$__CbxKFg``=GMEzkc4l3<$6hUv( z@Z~eRi@;l*NSxqT#kq~b{oob11D#PzV^PgC_#t6{Uo+4={e32tH3W!x;%Dj`&AgJj z5sI`>IV_eZqa!ga{IQthi`FC)rYt$^|yJq2CzZqLK_{J z4F(AO;5YX>k(feOyq|yoq;b2^{WLigyA?}gnyTg-xQY~y&&L5}mA4n3c>dS3IS-py z%=_Ocj4VU2rIhq?I(`QWPm|DO2mfVF#nHGFj|Nk7#m!$PMX2Tsmr2>vj^}RpJ`)dO=(L)EQOb4wuLSy)9ySm;tqYx-X~kQ^JK|FaaV3uNH7Vku$y6DomCHn5fC8|NfD5fxvCmgQ_AC+A0fI|>*LFC+Z;%mF! z5QZVkFfUFI|_Nf$=@pRn#@M^$CGBdM|}d zj}RCq(I!ONBqu{MW}Voic3)>3;q!_$KzuDxM~9~UfGzPYTb$5$Q!sCKIHqfYbjFlG zmo7B3Byrp+kZ-&t6!ubER7BOKA<(+IDC2;dyI*F$u;5b2fh+9~x)(5hSD6MUcEz*O zi{W3OwX52R^MJO$z)=p3vHv96x1Okb3S$A|I}SoD=$=!+&B0BEv46;=dx5SbHJ_Zg zseV1j%Rq~nH=Uy0atm}pgJDX7V2EGXzHj62XtBj5zsceJv#6XJ16rXoU3^izfpqF- zY8CTgI=u{A!sTrs-vdd}6$C)SZ_JnOM2^+U5Z7P8&{_?ZQ;=yfczB{p;Id6y)t?=S zHm*!`ux{e&O0?#oZn|rjJxGqV5}7}$_U>M`Q$sYZYJ4vFy@-kZ2EoRL3@F;aJ;#4b zb;W;SvxcD&jvl*Q6J+Nz(<$h%Ag_7R)rB7we*RTEeQ^OPJv;d0nq|4-#VV3D-5j8G zy~hT@I%(*Y8`ONXHUx<@cgH{>JC3gC`7o?7WUSS5f<_Rkz)dL(JS!)GI22_Ny@I28 zCR{89z{F;W5HkqX+JWN4J6>qb@&X0QK|ODNt=3k8kF5K^{a9UBGroW=g&lW=C{48%z6YoBL-TDPhaQ%FzN6B#KqlYDT7!`rI#2v8vWvm%B zPO=NSzIO=TI=ffF^eO?<6}|*w@;&XjhfY-x4yvO3xlnid>Y-0|kVQAj*ytS+%fDw5 zcuf$Zjl8a(8#Q!W?$<7WlpV$9U~j`b(;;8&1_fybe2&xywtt+glRD}1$f zF)eBoe&ZRK#)E5nH~wn%|BFt)!4oeynL7mpy=moK{#oX&RZGm}2+m6VW2|sZH?$bs z0XQbQCi4u3{u1w$)#XlI(lLv_k4A|h4CyRL647b5M4q$a)uJ9Y@t zFiI(k^{KcUM`^-H8F>y%>8-WvF;2;cA1X?Ak|y4N_6+woNVvf<(_0!z%f84Yae*yQ z7zcn5p$PUG`>ut7gT2E4QMzyhBudZxA>y4*(qFfh-&1pbH!%D-1QRk*tQ};=yUS^k ziIZf}#``4?#OscFU`;X;uOuU_swrQ$R3-#~^+q`3b}hEeT#6Oyh?jOGOXZ zmhkXEkNGRa-cabFbq>$9X%XYc!A_mQ8K$u-1)Nr6#?ZqeB(U2`ox)>$AWEcHQ`|Ou zRCWKPlQqA_MR6i>x;sChK>Z6A`6Ep?)YFu!fZdV;pn+O8H+S2+U{xU*?uaPkT#*>0 zOS)m!oxK)7#&5v0BvP>kVTB@wkqgr3=MjgZV3nAuTV_(hdX+^*+`i}LT@_z|e-tHC z`m5=z4dnM48Tl|t4WJS;1I!C|-0sy0)qOe3ZxZ^>1-U|F!Y){?;cMK-2T73GCx1aC zS+0y!$fral_dLIgzBq?Qz%C3C z7j)~Exm^JK{SN7*iwy?_3Veumq|($xn2^uKSUwk7R?;_bxxE*{sVOp`Csxj$T=#iJ zZa?IG(J#mFMBM|^Tg?BLZ3)!Xl>M*sI**dHf1bhf_2$PTe%hGXa$S?YHh{p6X{AFM(AJgfcoUlPp+UNd#B6wGLi zKWaoY*FnY!kjjwt)jI>TK_OH<=PBTmWL}FAVt6oiw|nnIg(JFhHG~9Z_-G*;)W)D^ z;Q2Y^Uw%H*6z|$M*EB;O`oFqO!!+S_e7}GwiZ=1dbOaCimFa>WG~yGw8wz6d!x zo93vBGTlXelm=QN7nu$cM|jxf^awi1)^;hCK^`!;-oyfaz`fMJwezAv3Er4GH{l|yF!>hnGxgUz@T7qTn6ac$i1&-6tf1VE9M+-VU9REl3C^`U2#Y;bas>J{-b0 zwr7@&i{3^`xE~j!X@N^d0o=T21~|Wyj&v|x;$W?rquXL(E;sp1r8)fR<7+-9=>9|! zEGQd#eMbYAFd3F|Y^*NbGK@@0rnQEzWfaWyjwGUD;jIzl!0UahdfW`rPr0K!N{5rH zTWL4`E2Q1shQWFme!%=6b>XD{oJB#$mrk^cYT{GI5;2VsZdkxBm$j|%QY_C2tE8f6 zDxWVnETwxu*puG0Q^-+Iy29swSm2yJx!DY1R_4g~nBoLIXx!S~tE+Z#Myt)umbVdrIrI!>h;QKlAqTEZQ~=e54h;&jiRq-ifGB&_fA2PxFk z0?>fTk?t`%OXuL;=FeV4aeiV#x)6_InD}+j75#HgI|Utq`fu$*y|?L-l;gB!$H(e! zfRky0u(dl^LBW0=g(cxrPdG^X`Uoj>GjIgycwHnn`yS`GYY3B=vk$ylyci99_nEQ@ zDHPf`WF>RxobPKgf=esNf1BoK5#*|!Rvs`LuG?2@Ri=NlTjjI~WbLLM^9EkIIo9-D zU8?gsIBkQv#~hzeJjcXPFc;d8*9?(P=yn~u#&z2jPsx`3`8I1qj;jQ|>y^PE?;Q>P ztklR@&57vmQd2oL5HK6lAR)3t_$e9^n8&P8ekD%r+(VNdPDwbro}tX@AuY__LVw7g zH$3cQkq7THDD6BMfic zf-#myhaX;p;>JUkDwocB#?UEnqaA^FLj=N8br#ySZ~hgr1#4-z)X~Olf?n#qxg9p0 zK_!VOcVk&7Co57JO}yM)>_-#T#8rmAQD+C&m-zl=UGU>>Is1;(Nk47&T@zAG&oQYa zTT4SrJ))FxSp8XM%tH(qg&rqvIMlqImSI#1q0$7X571#QDJhN3Dv?Adc8C(Rp0l;bDxReqv$+BzF?CgwQE?4#GZ6cFuYT|3EC8{< zU4!0hsUm30J2~26%=-wR7I3WDI*4V|BJPNoBx4T{$zf-^OT5JV9=o-ht8a%l! z|K4jCIuMDKvE5QE2O*j1ZxMYNwNepP#ScFG-;#IbkkkZE{M)BW|M;-OOEaq)x=j(C}&Igwc2=R8~9nEg-h7%A8{1;^$aOH_?LaI zA3zZ(-Ev~%J?*cA@ae?{$~2S{^&|R49~o7LQKV7wT!k{gB3MAlIOaEGr^dqNkAwUy zfy@v9YS=JmiJ&uO9octpJSt7B-S2vXFgn5a;uV`%)&dg5u@ZG>%azjZQ&^yq422 z4K&7Jj%A8j2}y+*z1WTj z1q8Q){x%o`H-#;O#SwG~x;6R!W=Xtj!DJbnNxV;vea_AIA09ebZgUHDJCbkt3ko%; z_pzhxHH(UYz2@cy!501l>P|!mAw4n9O26sb4r8}+*R}b9F@D3mor62;`T^X*59(p5 zw?D8vqzNNC`99=)G5{?=(!ZksKm*pet%}-~ z&e?7$4B1muY{Rq`@Yr%RnlK~- z#ozzw1y%bwazesQ#NAI&>8|OqZ^6)M(($3bf>u-T$L^9K+GM7 zvI#r+WQ9HX(jc}wXU1QQkT|ZcsMzbqiDUY9#A|}n<1OJ6sJu82d=5Wf0(6kF<84Dq z&|U9tAZb2)aK+Mobz=z1qN)Tj^Iea<%F|_`eh$jhvMjp zQPyreAXj4o#HwQ)(wCT;*pluItL1(D<<^6XNxbr>t0czJ=9VhYs9iOFnoNIxB;eXM}&gYMG^+-H1rZT-UGBSQt z2RtgD4Lu!vIfM$U&`6cuwMYvm--DhFU8mxoin z2lPqkuUSlHes60}nHY$Dd)!r%Otvaz?T^finwVHCFcJMs)sQsrp z5|D&GH}q})#dTv6z|>cDc9g)9#UeHfe0(yM7d_<05$a^sn6xtRwbVrhLZQ?O(AkKf zv0J#=gcQ??C~mUfkTMg+AfwS^Q=1EsW45>YKhtaO*z7h7s8^lzX6Juue#2K=iB5$8 zHe(l$%H+@*mW`O*mXL`uk>7b^4%ey8BR{SluE>3cCn*)c_Ac z9|0`C{#xtp3NDuX@t*RzqZ63r4rrNzc5b#BWFfv&=-eUJB!{oh0aq$|T&gDIrD0>I zN$6dwd=j)PjJJHXi5*u*ymqB_ZKDM@Y2Xx4r7uZb-|HC+uMatid!?qAN-aqEtu0Y* z!nFm<+O3Fb5l4gP@Ve_P$^SGK=TZz)KQ(!UE;^P;B$!c3vfU-(x%w{x#why7o5&{M zL@#Qr6unpYrnXdY%gJ8vH+3RWo}wLVZ9kIv3TPVFxzH%i;Rj}90lGtjIh;=K-kUVG zmejfGk|KB%(B(l?+Gw&Zemo0k+E}}e6`ku)2M#)nWT5{BBE)5!ivaZ_a$-lm3Eu9m z5#0|DNj!LapaKmjfyrCeBl@g^6^h=L=U*cmYR=YEcL~Bo)*-oOie1$Dpjm!y8cX&& zs4_EuK*U~HuX9nyygbi|%eEk9?y!YA`v@w&WiAY^zsFH?FQk(jsrYSV>HfdtGNN*i zT*CL0r>?|qC@u%S^bDzXN;BHb?FmPZ$B{y?vzIGO#r*=xYuBmqFvIWOA(}jgIGy*2 z9M-j>|8!znN?T+!jO>L_lD8C#j<)&tS-vf?Su<=`ZFLk3Mpiv_XsHiSmLl|BHlVeW zSS01FdO)4RQ91zLix;Fqrnx+Ql;4pM;zWuVIndC3r4K+$YQUqb1mkVP^(HR0d!?@i z0CTzfy~FNKoZ6&Nn@k-zqwk31PyHzppv$pKHLic4UlnbytVO6S#I#g{rmFKThyZjb_B z0m@}FP)?9IG*O?zUzD%R_mlj^jFBQbse(N;5j06rP#X#8!`CB-`d5(ba3MD@Ig{bhX{ z>X2gVB+FyK@_8jbzV0f|tWVyUXD5vcvTRHf>9X_(R#EooM{5jLiOhHX(TNWX-c06R z^dl;t1C4Sj^a~z9EW%mfMIl9+q0TER83Dk(AT5AQ|3L!>9-8z@6C~!|C%7aMI+Uuc znfvzSVI&MsFYJvP|Kx*4fw!_)b}r*Lhixki+S1i+IYJUxFviPsTNH7&td5P@%7M4eTNF+N_Pqzbyno$Pi%ITwIdp~pZRdHtJiOR+cZ?3LSP4F7q{F~7bz43O z_71B6ZQFcu=QW6u^Vl)4kDu83H9d8ki%okcZ8hx6RLS@{_&3NHgfv>#2b`P!p3<}p zt5s33*}#5Oy4L0~k4E0HULjl!7?8ekeXzHs=p&OyGo8BJ(F)XI36<8O-o@&?L*7AF z96wo&O7sLd#^K2nRAmQ5fssU|9lK<)z!|XYTxzKE_^XbxGcXWh=SQZu-v zWl;F#qbx3*zGM*E!&xJC1YNg2IgC?wdVmRdDAR>k6{~}zOG)Ddo!V=!SIY7Cp{}ne zWy{^4ln8spmO2<63#W<2_MKQ>%BzVhdkCsy$9mODXM8-;B}m9l6x`k(biF5#SN<`4 zA*lshjkuU76`oWFg2-!x`+&<+;&%MO@J9szh-=n#(>7QLo3Q-X`_69gM=I@gwr+X6 za4z{V)B4FNK{XplTL`uA7;MZ_p=~RXW&^~{U*GXx=`1BcL1~I4NtT+*ArxPOZ*dnqY$7lRl*{#KR~ahzc76RE?SPfT?_kkE_}HMbQ)w3 z(_9MVMB`6%Bf+vt*rn!hvQf?0nqmA@N(VG$*dZ};vD$J`7F9cCc>ME)fc^I0szCeP z6}KB%v^$kj#;Y#`^Q+@9j?HmC(Sy}7Eh-6!T=H+;4IK4G&-zyEQMD&TE#jeF_a8}I z4!45q5t{+G$#!=+Enj&{&sje?-(NdX7RwR1az|>FhqsmmMsCXc80VL-jNSE-a#mD> zDzPy<&r+{YX(UAN2$2HRm|pLiHklMacm;E_lQL6xabbPA$0g)t7fC8QuZdivza|+w zUv|kFph$tFX(5(VIQUq${Qfw%Huy23`tyi2jk^sKeEGYiTg1@IX&pzo8l1rY(cm>eDsXF#FCy8>H9gnN6bmg0N<%}e~oHx^|nRA*qe8?&(*qOy42045MfKJJaqD(nmF8@)bHd?vT z(w$s@2J*NWmog!|1Sb*=P~kpZ@DiX+du*~%VT`QYX1CB}W_Gby0N~6gNeq=R7h&xr z>m3Z6`ZoDLs}QI-q#%$<5j*uGUzJUPU0ox(8Fw|NMVL>fGegY#hFC3^&ud{GzWz9E za^P1k`afd`Zr4wzR=wIW`h>kA^QV{3`W|-|*{!_Wce4&SrlSG%7gAj<=VJ6LXBa+! zoLqoWq8<7?TVUr_qB40fm~P1GmyQ(c?5a%dR`1NDs?A+-*GadE*Nx9(?fK7VW?Vof zY8A2YdORsO`OYHPcs3By_;^avGazw_u8=G0&x7j0qyY?N4AtilUDd?~Rmujkx{+^r z#-_I5H_%p;cTlAa@d5wi7$V%O6u)jF_Y=ypoWQ`MV#aM@;VuSSm|2PrZUIY^nCk`` zpmUbxC&u02|F#P((sV_GC1pUCJJ4-F4J6ie8h|kMjtWGD_M@R#Ni5d}Ka*@=_fvMP zfhe`Xm&$9dqofK~jELMFrRpH2sBMvk=fjiC;7cqyT^UsW@en^$M^&GOuBR0{bJrk# zl!>~2Z&fx#dd5_WNII>L&aCcCl1qp2*WtNgKHMv0M6{lP;z`|8Xs(aTQ@T@h3PV{z zE8*QxaEKM#xTwci-%ootzE5+~e7ytz%gjEXe4Q&m5QHPr6orFbev9Hf7&ovQJ5kNp2uHWMKtk?A z^uu9|Fgb|GO)@&eKxq`vT;-v9cGa!Nbua%5HLh}d>Wg#upUq=ZGMc%<66u)joN4f2 zMX;qw+K^PHUYXPz!ixaHt3#=&tAZ#_;-U8*d>~Go9Wo6AIi(GI)g{j?U<}d)j=2CB z2mg5}j4xdOWGpi#_sBpQrHW!R9{YbJi8Ddhbt?b;dM;(7dKoj&+&^#`w&gFT=Qx#1Mppg6ULv`qQ9!XRWSPExBz zcI$ABQ|ap*$nvsL_g2ZZq!G`ICEKmy3JIXQhsE&;c3)L{pVv&cV8NFv zWrsBwdcVF7PxhCh7x0_PhN;%B8We|5(Hrj9f8;WoCf^FU2}tDy^4U|p)i5(%?K#|r+JR+<^3V@P8Y77us`OTV3AglJb<>rt z^SPK$ijP42&@mM1@swokhVHcA8dkMC!{t~rDumn)+cHV_3aHxdol78 zS*W$+s&J>Z07@hc3-3>$a<|q@iLaEU--q-)*P%FBoZ;*1m8r^Gmj0cu@j4pdjnHo6 zG-Hw}E$y>Wt8Nxb^c&vy+I)rq6ol8Kz%~R$BaOme5InY*HMtZDc7t*@{hOLY=8v20 zsvQ%{!IoYY;~ZQ>0j7uinESzGhVS#$kdiTK#nJ)Z9(vbvQ3uH~sUv1>ee;&pzL1CQ z_rQB)rHmPIoB{cr2IlSnGEbtD9k_OqAdkt04)!fnWot=Vb>uyL z7o|Lj!|Qf}QaqH;uMuL?04k~*`6+y%56`JV2-{2K@d)on@y!gre#LzWYxV#2;}-#Y z^+mkTy>x7XI$kwE>?Tcq^bjG>6!fo&f2aQ5_XhurfVU}IF_Rii(`KR)sg`F0T4eAD z;Nu(6FEls7Ke^if>9l15xIhR7iyccLS(=oDJ3Qw&JU)!_AsHGg+Q_lEeD?7E8 z10oy8VW;zp%W)-kepfY>fEl4mV}?cK3MDkqEbgNUvR9X|^erY{zRUAHNO@C-K6)V6 zEc0l(@SH~5;%7qo!63XweMMC>BxKUV9F6d41AsMFaUMpKt}&7WFf(wc09ZRf-_Y_m zS2&il(!Ts4>iBH(erfmJNA_d{(NA^T4q~VYP&x`thdHLCV z)>X9`NCI-$P@1Ba5t#R+lxD;ldTsTGh3GK7>D3C~npKiY3f0C);cJMS;9*5y59y>3|ogQPQ388*d zA)=^D$&V^ar?L2F7W>JF7A?4a@#UUWxTEbfym+weLLD|Wfg z;DGRt%Wjt1+Zo@v>Whj5__g6v9#Fxy*3GW!dAkGvPbH?QGUBj>*#O?CfKfY7vh^ar zm*MIfMlj#Y1$&$zbIsMq>J~{5k!-r5$fXRo?hPr}mfm=0+psYfzBY8_Tu()0_>*)~ zozMSY^GvdM_H&GHqkH7zV}A5L5bBC-hivt-1s9szBg<=9GI1GAa`^N*_Nn-bMG9Bri*BzYjP5dusOMp1l*ThScj*$E zf@_`5W{V3-24#7KcQ)GaAF7Y_V9H!sPTXpZ4@_`6qh-#wl^I4M#Z~2O$V;l0iMdg# zp~DYJBd4pSS&6TrL1^;BwV_&e8o*7vZ^sRqJwzE;2P zS#~(u%9dch@(-p)i|p8M6_O20IFlf6zH2Jb&?hi+GLPB%Wi`fv5G&MO#&3M;;rdZR z3SpL6F3QCXbq8(5h0liY_ffveCT0XaiN^NKZ0@7s8euU*fyL8;}B{#hcoepy&* zOdUvjx{bFy9Z)kj1i0Lre6~e(dZ!sxoagbtGR^9NuiHCDCh%Vuz@$X2V&Ut1Rov<1 zW4iK~D%U-n8YX`Mh>E=O5~+z5&(Vgy5SneL^~uq-)3|&+MEUS!rMl;W3&$Q&(3$?j zYDT9@L%dWVpP7(9-Xhb701SQU!Iz(j9?UEWve3n!NPB;)EfaA?n4e;U4{m7H&+36EWNVmxRQs<} z#@D=qHDgGiZNahW-UEJ#9sr?5NWeLZkb>1X2`MY^7yv74-`^W*duQJumVW)J5&!Rk z40|n|G+cgQ-sHQN1e1*lf(1#Jl(j9b+>602!8aueB#;*NKW8&@%-uUd1ufTP4Lr{A zPZ%|p*K8G#F){KC-TNvxGh-%L-O0BOpL!*U0 z%%HpU_{z83i6e+@?-2J~IuQp$uMHUV)erwbAcCEwy@1o`IIVC1Y)A&4)$dG#v zL+&XIE)SHO>7znqByi`}@jh$oExa0Zepr#_*cH~3ijBXA!$}fB24G=i46bNBt`~0W zFd#<{z(7RYAOK^&7G<}rDLgTymL0Pt6_pU{V49SL>`FZ(Fz21dyt;zaf{Gvts(co5 zW#sphzeTKnGxhAj1z6}pkJRVyeoOcl3ny;zH@GGtptYX8VR>nU!e3-0LwN^JZjNDi z+Io%o8qN03xlmjN#w!aIj;95t)}$jvVcgjhSB^*m7wK!ufQAwQR%A{u%duWl9DmQq z@TSzi7 zFNdQcZ!Pi4Bnoylb8ab_>lCcjS?y-9|F%GCm2+(QRKQCcQC@P~m+||xnwYSHczL2# z2-y%CuCkPf8tAL-xT2`_-~cQz$6SfhpRbv2mE;eOK*t-|&!A!nBdW^l)! z=mtI1{=y|i?rCiuwt8lS93r_AL+3@^-i5&Fhh61&j(YXUIlUBoh$gYo zezLf@x`k0M&z}JhO$SK{MH>XlK4%{QUOo!YaHGN4Hk;3_e)WHWgkk|B?2oWgrUKEy zJYS5s#kFuIy10-WCZU&Gt>rC#so%s|@e=yKyLvCmZ!LzZ4_%=KKcg8l4W8kP7bw7@ zKu;LN-oN^!M0ob%dE+OFbgH-`GolL_F8MX(RL0NHNS|yb!aVGkIC-=uUd6Dp0MEwA zC5(fz9(Qyb)dDAQ&X3Awr0-`+5ZN9ImrX;iu-KFMxYF|*$?s@@Nxg8JGwH4T>aSR- zLEm2zMgB}1L#_SqeZrdIJOqDg;I_*ro?QSQsDR(D6O%XX9d-z=#DtfoIvG1R^HSsw zhOt6Vpj-Hsf-#3%IlxU$j^5aLJ}{o( z;K$TWiafe10S7Xl^; zq&ic`+ineMI-X+X*^q z*y}V*VKk$dVH|7j`e1yReRLO;h?EEZbyG(@lo%#{g4i(3SCtTJwfyKFK=mYhTn={Q z#T241=G*Uscc&*~{S@&X{!pp8x;9bBZeTHKYh2&2n*`O(2`doPbRG(^UfCz&+p_tp z(>Xasz{&ioXyfB1OK0*-F`CD!h3-vgp=UcgBrCY95?Zun`{P_HRIbis%cz<2spsNRY>nL@uo?2-G?vS)&@fIT=EsR*|c9QB%f z4D|6H+{#Uh5#c*bo$+0)^@5{PJN;}2qQg3A-eByDKA_m_&9~Zf; z0r|4nR(=zb^Tba;;vrFenR4j*P%Kx3JD=xNhUC#c;WEK99=7qD`dr`k%Iz}?E$9wH z7+_+t?*9%ZfSR-v*6<9X*mOVE#(Vz-cPAvKP^ZPWhqL#K2c61?x5Wk%KS3kledt(J zF@w?>>412wjnH#_+5yDIM`O)mGoy8dTV2v3@plrG5RGI}gP_UJHg#64F1K_D2{W{! zhHBocDeY9|ZqujaztsN3FP~;)Wk$>S{X^M5vgp#48jOh7$|G*0y*e&k5~>%HlxDSxjcRA+R0N~8sL?L3;reW$=|98{#X53|sInQ2@*8yk+&sYB|y$Pc?q(QTj>^t!hY2l4NEmQIN+xM<6z zHLI1mF(DOC3sXg{-Az^%u)h03(m>m5J9bY1I@>VI=iN#S0?0sm;)SD|01OL#o_dEY z>acg2Sq0x@6lxE~+xFD)B{~y`_G3#%ZQK<_XL6DPq%WhOSdl-12%hg{Ow@;VsqyB2 z9^te%iY=p$tS=*SC-VFp(jF($Z zBvjTJGJV>ygfq?|t#i=LPX0(WCI-V%>a;>>?y}Ajz3B*75twENx!vw^(T}PI4pOPt zJ^4hyT6VgW!gBjFX#NI8)Ogqx2} zIEe7?E7P36pgso7>dr;ImnFQQ3X{y%$D!Lu>Ny!cn<^c48<@ze;J4uwDtT;HK!%xj9OWUIRj#$GX0K*y`{%VG^}*%jeN)oZ1lk z-TJXIC;#^NB5PS0E$kx|Z%3ulyos!$uEp?pRERzJSr{^90WAZ0WndDx&1hISyzZyo zhw7k25PQ*Dx@QxFC1QIz76uCHHpa!Y`h`}$H0RrlHRsHdzC5t6E~EatJiQfJdUz$e zK3$FX3}c0d5j61S4vM`eD^Ka@0@{isj0Tu_h;k@@2HFD@7B~9!w{Ug(Lr*x^!B%#F zSbLU*2MmIl?^R`1;D1t<){c(90($HK^|mIvWJ-#Gh8!E*iZ8&rQoxhRz`#@C2l<_j zLm$h3wzs1LkCKC(0lkq2Nn>F+5f41{-Q7cy25nc}I=v8?XJEbK>T!y&5eo&du#gRDL)qVHc^ zE}174oJ|DV>l3sq${{P_Vvng}65yb$ew)C&laFpIaFieo9;lF>0CRLTfhQ5)SsoIj zBD*0SsAmF6Dvn0`C$MJ*`mxZ2q)7+lUhmBbRa!iAbGmCR)QBz*LX>|dq351coP3Pk zJ?48*V$bYSUWS;n9ghBg66%!!nc@hiSI()RI~WbU&XS;AG-71Hb-idIs@t=snF=qR;MEkk}CsH@>#Y;!X%x`y>VsiI^x_~XK zz)x6wBz)+#2lj7qX3!S7@LdI{UK{0nE6G>FE@WiIQMEnDW+W2uKaCk~hHyQX6>=B! zGfxU(Q}u%Vw5IjDQjPb~Qu#{QS%9=P{c@7+5ZJN^39h;;D?J9!8m6rGS2!|sj~{>l z7U>=S1;omb3`)I%d+)(Z{5juruCNvIRK^4Xo%9;mvW3=o;KAk)I(+YF_;VitHH!d% zkE%t%F^*#|^>LjiJYCWp-zNS1L+9Sj?gwP!k9M;IqKwSo!k5Ox;ZehG0i4-OE`}|* zIGM|K9h*HJw46mQ^WgbkM2s!a4}lB8Gxw_nbQ|{?LWQ8J?E+n!fjJw|-M+<`@8bg4 zzIcUe*iyg~Cuof~pZ^@3!inX!)J@O>aWDlevA)AB8aS(DKr&tcvG`2j#=fqvF)>$I zx#fOTY9lN+Q$!3X&*GBQq$e$mc@2@?Y(u5hXy#X@Qpd`&}g@v$G?b@cu4riO=JBI@nUvmo14wI}KuCd5t}ab}>M} z&}i;#dkHBV2*8;!hclyZc8#>^U^32V3#R)@S1KITM!bll?hL{4T^8EKSDcboD-r*> zD_?s7LC_ryLW=~-{k+gnbhzb#&voQU8UHBGK32u zZiUy4mM8B1GhHp@nS^Zd@)j5`>MCVHnlUszO>3!d`VqA#Xro7H6XCU zuSV%bSkbx*jt`cmyX9C0Mx(JQ*ABBmm=Y0jHaNFzKNzgdlx-p6xa1kL?(P80c#*h- z2=SMbEN7C4(<~lnINumg+&3Dhd!;$86V_bBGrmzftcb7~u$rW&Mbb{&7I$SG=UghIvXdF)N+Liva zK^=Y@9WH>3&pB=ICFq<-Im6*sPDEYrppkG4CQkkaVhep?pesm#x=JKB*^n>32vJjE z`uDmY5Y7dAW^Mr-k}TJyB}sVDL^wY#Iu99rI^rj}!=FD61~I~Bzt52$F-^XsHFrTu zl2rR`k1V@7HE^8urHk;^Lcl-|X$-Tqly$4YGudBd)&OjBe26j3H4DPnaa77gHm5P1 zT{{>X=PI|{kv|C5sIOAquofLjGKe6O&Ni(D2K5*;8)#bpY>c;>oT+rlZMb^!vv}8>cQ^hA8 zMYw>-brr$)Rr|#62W|I)OBk*Q$tf%44m!#WN1|JCkvCB*4CB{rQe~$rxeSmRV%+KU%A=_fB?`&y|2nb*lHKTH%aL2z@H#7lj(1Z zy#7e>)XNQt?svkmEU+Ti&?fS!zaa2Z-%s2u`)nO)Og?)}ZDxYSlL=qq#-&E>6oCzN zG>QH=m&mVmtkQi^pTGgPl%=+S7Hm=LgjCya8+BRtt|yDcRI{F4fq&1zb-_>GWVKJK z=+C>ns+*MJ{LYGR`P(;k!AphT27e6!->DjH;fm4&mhwr-cmLq5TzptMHc;8~{QPk6 z!JEevqv{dd208mU)WHBk4V;RWnQ(=Xt+5HnFMdqrqY`S?%}o<4F0c_fLqv!*iT;?o zo&ww{Kr#MCo&03aTx`S6V)qxN7DJvHyw0gQ&xIu93{AE@nHDT@FhWhECPPFa7Y4f! zJH?R*_Y5*4y=fx|NZd>!3J?wU3vLMP&BKnkJppqHn)Ynkbd|&Z^E0fWLpr`Ps;Qj@ z%=6V*1&jwYqra|#?QLpjZYZME^t;IsoB!8S-4yDhreYd!;|q5H6Nv@8Y1>>F#E&}- z>-Ah_@LqTR0j@#fXFgZl7`#Y|Nc_<^RZq!Xkjxv?uB70-S1SE4t^^)bpDDm3LDwQLM$)3(Z? znT{;*+8mLZHTsNE7~Zt?#((>e)N#B1yHc#sqlOqF7s(%>&F!q_&+co?eslI>|gBrEeWt>48z+SRDsF@Q!>z4>eKlMTY z=Kv%^mj(CPo%FPhMIT56U~{bb*c8)un7NPzMzQc~x(#I921t98yJ+*W*}jLi`~}oB zWB|a9trL5aU5#KOOJA_HU^&R*6or1tmUdLn1sKQ4L>G`ryOhRq2muC+6dhejNDf?} zf__A2F5qiM%zuP8QrH)%KsP3hZt1&*ztX^K2HUX`W=X;Xdd`OB|{7Tt1sBuvMH zPZ<2BjWXn!t+_@RiAPmkS6I!Z39bpTy{ga-pyu>BFu_zJZipu45W!jXQ(g?|c8!9# z+@1Ty(fZbcX+72Rd7E^mm)<*nW||~HrOzs>*BT909rD{r)(`8Vp>a1f6oUl4L$QHq zn>pwc6QiKmMj54&e7c-t0dSarW(A=%qfMT{+at;VTRqd1OA0(^Or6KrnP}4jd_bh$ zksYyIe8Z-At=XtX0*%P}90q^>ge+3lxVf35nLj#WuA3}Wh>2E-SfOt@M?O)-j!)Nd zDC~7M&uyE8`{+(GIVqSgyRPl2q@)+;V#-X*F-`F>vVI~ycQ8K+%*-WQa;Td`%S>}S z&nyleqj3oHlJ#R#s|1pA_*lAJ*RbM{CO4?Ic$sAI9d1le^9w$E=bi(T;0zY8>^ZrK zy1yt^njM*E^c@ag-W0%6@o=$|Zr!0KtcDXa$oBm4nsTfw^HtjT5E-kwoC&3{DY(#q zFQnM{iZf|xugiCpp=K$0t6A-ilhKhc+h`-Z^7|)c>^CA;5fE?W8Pff;YO-kG`t;V9 zmQ%Eah+M+Ps1hX5J!{od-j)p%D_9mRl=9)p8SWqWNWsp+LHViYyN6-zW7w-(m}KKK zcY>{_Y;JLtz>jjHQ$#Q6_ERv(TWFjpj2Btx@Yq;sMzEHgTn^1{Ob?c7{?Yid?H3ID zKo9pegOuK@AvbF<1j@5aqPuxVq7XSJ49=0h>TahA;h6=COEIhW9QNXV zKw7RWOFB6Fx9wx`oc!s5b_P27$2GlmLp?@YM(oL4Byj zh${F~U(I;B;AO90?UuD2dy4C9@9JIz1^Sw>l*OvuDxub1-tFRb_a~LsKZZuAd_jIr zRrGZe9N;izLB}`A@W?*u@~>5Ro1d0Jk7SEieuAqsaud;)9UasyAHQ`>xeij!&#vMr zWLxe7=253+P64ays~C)r-)-xut{5a-J`)|QTEetWC4HD54Deb+7-yD6qG&RoA ztmf)GFjYX}9gC7em=9tt>_q|t;3-ELR#TEXPVnR6cuQG^ipiVSwwoZ z-eykT&uO%dl zj)a-=l=VWEh+06hj79`9QmF1(t)1?K`%w^!&fGYyx@z;j4qug4Y_iE=M4mx_Py{ld zhTaX<-mZ`7T&FzCybwXA5OyM+2bK5$iTIOeq_QTt5ncs#${P{ytxpDB>;)o@%cVqr zxFpG;q>A$S7waOT0SX4H13U+&CH{ZYW+^}2u^=RdhE93jWEUU6N;%wOxOwpat%BPsw` zW&xSQ;%*L3Umu?WB61G(U*4rtRmi`GdH^U|k^*P8n>ayJ*l`EoUIMwa8pX z7oJjIRzbW3L&-lP3v^snut#{HATIG{?~N6E`FyQ_80R;aj;_nxY~()_;S9{I67+xA zGFcu+-qvAp^39(?IR;^WpW7~#eMGi)0uQtyA%$q-dm>NzQ>Hx+np@~wEdKY0M_N|@ zZBvq$Dcn)LS!piTEi7(!evB^^sch0vW%kyj=ob;HnWLr{08S!i~44C`F3 zr|IWH`W{5S)kllk-3`7Q@5$H*ZEaSmd9b=v6&OJ<&AB_A++DFpBFcyxQaNydnaXKJ z0dzI#E?EqFCKzX}#74_dhW# z9#0i>rw4R<9D3)oXZhtZ4PgPZRSdz+D5Wq#P|>y=5B6YU@yE&-@hGd-^OP>zvvB_b=8IL;~PnMf*Y@AB-%?HFC9FVp5^NaCneB z$h#Fogo#Nrz6)uxNp;QoDKc-B%g5nqJQB_EfIdpSf%*du_|6j zS!6C8K(%`M3ovT99eL2+x6P}M3{5awUUSiXUp=ld3R@N!EGczNQ(6DQZACMZ41pLc zCZgo33(TDz;6qBn3P%1ok1yl#*#sb~_4DOVI7F;f#7-Ukm6h_PdaD=7v!hn;CP&}R zf(=k#D7iU~e|@FNd@ooP21sHz8=Xg|sc&`g=kITgyP_3`bGlp2sw*&YP8#9Y%$O{dGsMJK-(2 zkgmMGxS1|P9F?DdvX;6Ik_+AVO-IH%(k+=_Vf;)EqPD(aoDAzNVsjO|qfjDbHV-Qkx0oU!Ji<|)b0v&AJ{y3-WIPSNp`!Wt)F2X@124{t@ z%7=?UMftV9Y?eJ-Q#g>GWoK#`3juU4CNq+=*oMX=G>pGcEsDfm?!H50R4~paw&P70 z>kzTGnApwfHMnjwBOlt$sdPX+?j_VbGbooB_$rOF%RpJHsJv6k9@n{jGiS?|f**4v zHb&rX(v)qur?QSOuD)D}_F@Nf&PqNmQyrowjJH4DYct(skJW;~-NoP1|Fsg8Vy#DM zKe=-2*MC+F($22SJowepaxP&N6b(R3xg&yDVq-eIa*i<|DodF4>>vu`IW;WP-HNXZ zQD1}O0pO0hhIKlyCRJ8e&jaUU%?Zy{gkj@npS;qJ4&x~_LofLZs}jITqRSmN08jc+ ztKeU{kpd`Mne}Xoq`dMCSsu8o1QV)j9aJ1|uF6O#--TjDM%^(MML@-K-WmnJFeY;J zVv}Cf$V=)@T~7~YaRJ%S3X7ftaxP`~4x3EV1VQs1mbm~XtqaFx9sc-%a@>_sk?3FU z1tl-4c^*2RzF_@M2_|4Q`(^u(V>!@ahp8TmTinJaeaAJK`?9rYwX%cE0iPutpSPf_ zM3j_?RU8e*QLjup?An*-mv1u#;9m@n%L~ae#ZIc)>Qdjihh>t&%-#BiOE%?$TGhT| zyGIlI-^qTCzoFy`$R;ZI z&z%mP$9e)bAGO6SnziqFbfaArO;&|F?ObKQfkmfr;Fx6Zbn3ONQ&j%2=;^zaLSqdv z^5tm)T6Nq=ry6*n0L@?tozZ%K4@DJ)&uk+@fr4XVZY=tT)}pMFvVN-5$3=u|oX!arp=)RzLisT{F$CAsH> z!_@R0Xih>8;p|0kW$gE--9z&q+*`AmB5|7aFO4a7RA@V#XoL;)oT)(?{Pz;MNnb}e z$f^to8*^B`XZ)Z38#XuUpNAlqt&Ph}@yS<^6yE8F4iM^BHrb zI*=p)tGMju8rB$)Q)+D3ZsAiOJ}C84ShPk8M7kH_QRkf4;- zq(Lolm*?dJ?6X?d{c}^vm5HQc^Sh_4L$=zKDLf}@&p>MMR$x6ir z2*riIQl(6@m>WcVAaKt7r9XSeag6$=5Jnlxf6eJ%X}h5yB?73iYdOj{sH4owksMqL zvL$6o6Tt|gLFN2!5$6`1$#}%smM{Ta`HZp;^Ivk`g7D`jhf4B=Fe(NiPj#uzd#%Le zf8};BP2R++};9M0c5spA*9p^udGGr-jLm%*ip@+`C zVAg~bE#8+SA%=TGx=hg0?#5q`!|TO`HY6FcD{XzH3v_l_njXV41(kW4=|OcEa(pr| zTrafg23Tu)BHF^n77O>ZB~)RW>1sU=tDIq5PQ$QG6_dbZ)s90yc7htg^uXXV@AYWA zgl?arQ1ey~h~O>HMdqP~(<5Dsc11Lct4Y8D!1+2OVVV4RJOCgKy9np6NzL!Y( z+LgX0d*58Hjbj7aFmU;!(uXWS=>>kl!#be`1chO@LxVzw=EnFumDRi4!JqLvZz54M zVA&rYx0T+4pO=0J-w+iM^h~j|TCDpZmmz7v(F_alY%?HX!s2ZxvHJEWyWHB$sJhfYmbu}uY)#9iCK|sF0Pv20fbt7|j zCN`|~=n&t~|7aUil|?{+=yY51*JeKvczP{CzjI7gam3;p1r!l|?N=e3;*nEHsWY9Q z6}y`7=Yvs7k^?tVqCrVM*LxiSlpxjaer7xP?~KG=!QFcF(w_qSgX|-rf}|}N4Tt&5 z$;$Ja?xR84ecFb@9$>*s*q|h54~e*x5FA@RgCv?s>RcI+vQ?LWb2VwelVYUZ*3cF9 zY@CU#@V0omf(-~IfxGTN|k(J1_ zCqWI>c}a=vgwJPOBtk}8(UFelumrjV&Y>gLPJIYQ!*ek(hU{};Sc#Ld4mu`ZrsHE@ z@!L%F?z2P_vA`*Mw*`y{zVRtmSZ2x#Czh#GNzb{AiBOOJ)o;|SCZ@|1o&`z~$Iv~a zoRaA?;z${{sj-~qPLv}~G;8g17xRGbm+5JWbN6wyk0H_;w(U>3%}8XPL!a^iM7iA$? zDbg~M&F?sd-&080%xJ3PO55t<4qhWW_Fpo>s3jVd*VZw+>LR>7f6g1)ZXT+9-i8Sn ztHad6mnv%Y@TroRsTkm}DxHtSTpj(godx9F{ zqkraNeU@atu9V`ZUAOx|A+(e@k47jfQsL(>m5%$&C@4{p`>>Bi?$w0}<86D7_;!$z zH%$uE-Jz4f$laGvKZVtT+2PoD^Dffy+$@(#^(o;9W+R?WGg|DRHIUjlHkYN!|6mB~ zLA^jg9wRrHeBl4>&lx|MHs9@O!9Na!&}K1#S@AI>uP+ln@Vf~5wb%rG($1`D5xd%$ z0+$)8cXOrMmB;}L(xFlj+F}+RuIJ5ZDiz>tYEFH5t3I&JuPupEhlG9a#LNK$^4>%3 zU6d5)vhM?qg39EeG^mGVq59V}V85T5^mMgpdZtgr@SW>sq$DJmFrT{^AE%JFDsD!< zNAGZLPgl0c*}(!DBmEbQ*iqyAx+WgOVo9{PSqm>C;F8_bUt$;ybI4wkvlRyIV2bn4 z5`GGV;f~PC=wPQC-+#ntF&D5PFNa0X%=Q;i@FQ(EQ#6`U+cm_y2t1U*w^cQ`*0g^6 zaGd-uIkf0SNxa(Tt3S7;+m7czr%u_itcb4FK^ZSKM7;Q2N7V^|Ll15deV8U*PUuba zU$Dj%6-1k?<;VT)i^f$SX(p-4X?aKy!MhDio*@Jg>qYk9pkC#rO3p4bC+kRIzt#B3 zm!jKcn%9l4C#pV0ABfL$nHgXyQv~JkT3RGD1$*6-v$PBO-|5Cs-dAJ7PA5|NO&v7pxs)Z)2E`diQ;m(`I9@I)nsuZ( zn7@N3;J*7=ttaV<`TK*d2_)Wyq+|dj$4wST?6zo7YP@4@ig-`^f6|O;PZd2f4}f#y z#7?-?UTNTM`7;C#%9X%wTl~9hRD5A~iKf#rC?J&92GQ~F`)rUM`!0KCF@0|jCgMko z$yz$NhMJ>u0Vspeg{PP(wd}A2ku1#V7zD&8&0OvPgN|3lty_7LX9k+2AaaU?u;FGC zSUwlUOs8!5TPu!fPhaU%#9DiCf5Snhw6~lCIA}})kiMMIHFQ09zAuHaY0U#~C2Wh3 zwY`|qZZreNIFo0ARRDBo7v>EiD>iUKU?_LCb7gkpf}VR*uyBB!Y>^|5(>WU&WRcv%5hkW88NRe8YasJfmN z;n^P)`4_6f8(<$dwG~b!@qtjXX{ZiL1j3WC;`Dw}%R%+L?M)>E?Qp>Rzh=~VZP?^S z-VPM%mT07$3>za?v69MvSFQlFpFE@KaI+FOKCKqeW-!TEIb^w*WrLUTU2*x(3~tjd ztMTLw2v2k2zy3I(oR*+VOV?-qtO3uAs2ULz+4rWjx_{)mXk_q`u%-?E2Ob)l6tZt; ziYZM%##&`t?geKM&e4s>c9`mRTW+*)MRw`-oD!TYvhhe11Y5|v*uOC!a-H@+q2^4h z`HY2jAhLjUA=s)>E3`IzH4ED=$Os1aH2bZY3+hN^Y5>!Vsj%PCS*4in+1d6sbUeN9 zDreaaBN0~4>=t=1hJkElRP%5@2ja+gHfedrLRP92QVns7{U0yjK&aRFneNr5p{Vcm z9*?;n2nEM&T6d zwtt0iYtB0f@-Q7;TAAISVaYA^PF9lo@LjlcZcs9A@O{3;c>+T=AW<)U(Ci7!Ml|fA zRA#-RD}aGzE54Kts3cOk%{AhWR)N2R~flVtjeY?ASa0x)I#~)m%!b?JiL_Sbgx+}yW?Ix+- z(cQhYT98*Vp-siz4lt^_W@2Y!x6T_+_CGE{f0~syYunb4J#&0BGZg_TI~Yl(1zWsL zcDQ6v!FYL;=r=5*2+PMJX%h@}kR^UiJBSMF-t1SkT1fD7|<^G`tD**0da8(y#S3jYL zRaEnHBZ^VT^-~i6sX>;n@V4ZovorTo->6RR)MMA9-PX6w;#fS8?SE_5l%F?5Y1V5x z=DX$G%DC*_7gOldu3!qtTjWf1gw{X4OM@W;oMLN$+)4fHn7zX>F4UUIdB|kAk^1P zuM3<(2Xb9Uv7gY|%L%h}pIj#bBclY7^uON2@d20IN!ngGtArd=;KW0p#~&Z#MJaXg zSbI?oYgi6!6`Zc&o*0)iD-!p--aj882FkL&`6;s~u64u3J9j`v*0eu(BA9+9lpz=!7m|(M2q2fwBgot2?tZTM?S)(&h>?6W-*fG7FuTQ)S)fQ?KJwO$i5 zqf^_lT$Ut26|j@FVRN5$Oq5NyAWc*U-W&B{S$D~Z$DBw5QkM5ISI`@`T8X4W)v+D0 z2$u**$7BP?R@Ed_j+a)=9iY#5yUHvND@}^XB$?0rNld<)FL+%iLE6phTVUKz7H zwj6dex~2vFxWP^@z^BGwT>m~hem7O0+IlpnCj~2^kIQ~WRIAsi44$b5tl!j5RsCA0 zyeJ5)DgHiXxgCaUr;#P@k{X>E92nNdMHP|xl2tIrnVGW%2>M)%x2qY7mbcyQ@-lUq z%hGj_Iql~~dVmi-Jecc*knqS=GigTi{Jh;tQ|0b9xrnL4|IkSk44%jLFR0MDe9 z&oRswW7Nd!8RB`lEP`|sR^zJoWhkr|gqj^Z^He^m~7#Y;kL2 zwSdU-csNRMM%TV2{8~P}6<(mMVcqW3B6Bmfl2hF*h5z&)d6Dyb8MT9ARw zx{twtbvp5n6Nnx6F8CJdnK84gC#3fFLtIiPPJ4+)1jBakJJ3nUCND4`c3P{}4m6TQ zLqKAcN38S`fX5WjF@Ya!QUrI#B+q; z)UA6JLPL!2m_m2-FoB(IkvGp!SV>a@QOtp>l1%_%#I16esktw5qWwgKRmgi`PwZ6m zsd;}iKyr`tUc`VDaJ`|*A4q2WSndGv%KlAevu?;LtQHKaLYdHO1Sgcxo|ViJR)f2= zrYm0>vE0A^Z8$~Ft5-9xt-p_{P+KlB2%q}h7N^YL@l$n=s4h(ssycFCD2$(-n15dQ zzI{!tS8W^R(IB0=D8|y!xMJn+Xad#TPM7wE(zMV#R&3L-Uv5T{Kf8Pyyn4KJhW59{ zJs`|XapT_vqvG|}B%ii)b$GMQG-oV)@5vo8-i&iP6{+XENn7vDLRkO@bcAkArz0qv;sbl)0dZ`*}RuIW+I^O0Y$ zAm^7>touy*Y$-)Dsm$|XheVo(rbRS?cRw><^tq2M8;^46rwG1y1ia*Vyk&DTAlB*f1qi|M$?&{T7i&kJ;%0(;u3jeSucN91UTrB9~Pg|5S%c?B7Gm;~Jf8B`k9U_b)AdFo6(b}ORsQ|^B?P((s%NN-+xK@gGu-x`j1 z&!+-0LMY%`3GPx-Q%UKn4(4UZLs}sa@e>P6gUlfX$zwql;UdX2$0nLF#dULQ{p+Gp zLH7-L_B(mb>o?}^&|B>(h4@{A;g3_xbWV?s>;|47XSEx(CNO%QDf4Qn;`Gduzg97* zauNC^t%C430r?%|pn$LlYU)J9k7jEa3~Z)30n6O9oTJ*jH3xuM8z4mSPyN`ZMx*W5Hfydjb{ zHr_*_;v~g3QMr*BW0-2L*o>huFkS&((dhZI7bAM*LQTf0ul#L>xv)BPR~>GNNcsK4 z=PaompHr(ZS6b4fEpz&UC7OPb-OQ(qf~4)d-3q1E)zIzeTU0qt&5)UocQ`RHp;ZM6 z>dIinpejW&frmO?Q*b!-x-+eD7>@j>IKZlUnl}P8c1i-+Da$(ZMZw6C+;%Z96&aws zQfIcwg-aEVZE!p+2VD)-=K)P|C34Ko@Ia(YFc~@DH-YkufFBjiq*P8$$g4bg0TVUt;kGdMkaV<|As=eg_EfJKu};YuD#c#9(;@wh z#ra8Cw{5CZX+@!=l+AY_lmPzZa2g!ozLu-S4|0=-*X2I{-)7jsu+0SzTZ0M7rgiR70W~{O!khZKc3`!!l92{ zO(uq&i^bewXR>N+9?X(4WxmQPF5n%?pF-&PFPlq0#<={#z;+i(@ zyl>l9qb$(Sc%;Ww?Zv`d9zEI#k;x8H78Xuzosc9Nd&IRnOhu*MHoO1JZO-hHGWxym z=*2mQ!{txJ`U$|?VL0&Hl3`I7uUnneP#xH#ZfbNvfVtl0`ek+DtQbACKg+hR%sC~- zHi~`nsrYx(sP%C#Ts9l+%xdT$n6`O3TDk$%ggb@|xuDeaM*~*aRhFej3z2){6}uDm z*$4VEe*C|?+5X<$W|r#M4w~J5stDLNy%z<(eTP4!KABOF*SPBYt3ZtD7S&rbBhjRQ z?SyR&?dmNEFc2NstWA&eKkT8?GVwwRlGhk0BIX%4>;EkkAFprhWqp-HGt-6d6zwWG zvwS~)JrCNxaPc_EYb{ zH*ot+1k7c?2c%BT>LAtU6w6D4>q3tJt5@-_gAqiTc{KgN{yZlv;ON8wM{&dPdO4E{ zc)!gDYyrXeVLxpJeYXJM#H;@^$E)=&PcjcvbBysoo{|B6>hjl-zWvAx2$SuX^FWq<&G_XpkrW0p|gdJ{zt5>e~( zG!lFeu3byqhmn|if|8HKQKgSTeF!Cbt*Vl_L6_ztQa?>!ip+zkZ)S{wT>tfSNohP2 z#DQobXY9@^F|^ni;79%{6F9>%<%%5&QvNKT7@$ z69^Jo(D*XBPT)r$4jJ09y9+p5XSRI4RHFk(Sq1f$SGcaiUMAu#U-IHJ^iY`diZG>0 z;o(q=_Wr}3`39r@$7X$nUwN@_MfXR4xQfe=R6x`s8o|;svbvJ>`#|7zavRw)n#mQ&EZ+>iHo^cTi^?t}Dhh!{u zGYjjD*wf9+>FivfQZ=EIQ#}yS#N~l}TGt&&!?%9w+vP?B;*5y|H;dcKKrJao&}ED5 zr8buq2%~7y-nP|zrB3JaA-6p}cp64g-YwPX#Vk^9EQj^n2B`p7bSb*8Y{73TjbCPo z4X=qzi9XfRKODg@U8XPuSLZ!KOy>BIQLFs=iTgGp+RHuES<}iVMED9-LzUfm0bW-3 ztK-M}CcfW%DQz4y?t$tBq$D7q13W=TlsW=^C!u8EWgWy1w$P;f7HJ*Iw}|XI1)MduNjIl?V8cD z^=$~x*w2=cSSuR9boC~zDVMNR3vTjgrqP!UoC#?}I_wQE^jMLGUSBk+c z0J1Ee{l$?W?Id z&{CRP5*bg+Xr|L}7s1Rjs^%M>!g7@W%>Pbr_n8-jl~jflcYsmR_;jeV`K61&Z$BQz zh>0dD@#|ee*TC0$=(a6#H^+mAsKgSE$3fU$rjNd=O84D^Kuih|$j3Yq99|1*Ti6oJ zYCMKM2T?^s=Y*@H^LV1B&qOp*`tG`RTGQ>wiQ3GFO$|l1*W)r07omIdwRJt{ON235Kl{@ zySVfuEjo`{$uZsb=^p7C@58{GKu;V>!~&Bf7H`9$#}S~wZI(+8!3o*`Q3p4|kRr() zkr}gi(H0tl@UBHeZNQRbVD53pr2)znu=-OIPc7+DwtqcY&Cs95%i1-$80hb*)%$7+ zpB0?jD>LNCaj1JC%BzQH!XomVFmeF2J;>-*y47by_DliGde*9g&75HxrbvglxFIP| zAYdf@AqIx+GweIbOL`Tg9lIxfXHT2_oSEmxjPP%=hS~5Ah#|haLWjP z>Y7BiU^Z~3E}V?cb|h8rFQ#0-;E4m1%V1TtFo(p$YvD$Dx~Zf}&TN!Nd4604)i94m zUx|AHrjJE15rZ17A%jHmolh*FZ!`|Q!e=f$HG#ICwm1&QjEX(#&dG+gdvSu$2eTKN zhHR?pA(|W2PDI6N+t}M~e*28EwC@BX5ph06Z1J&7kcUQw`Q%@VKvZnWNU8D*bZCVM zBh)fZ%!#UFZLSO0WCt=ykqDlfgurZDK{Zr-7`j(bkX*G3`!>(4<(iPRO6U02i)s3c zM+H=#+qm>{zcL@1{N-cA2>hv4t(IY)+_DO{Ny_KrbrurR=c|PeUiMD{FfWWg0%RF` z;9ON`Srs1bDxjM)GQ%ANnHzuLu?mK*%6%(Mel)yc%)+aIqa(?RSL!r~#?7Iyfzsrv z{Y-XZf#q_8@lz+F=LzQKe7u@EbTXD*5d_p#zAMGP^yApp{Mv`N*faA??+2;`hnQW z`@2=D`si89g+@*mPFF+4Ao21oWjz+|k9-~{TY)#3dp0UIGybM|&*X>r6D z{`Av}ta@hbY6j?3ijvmn|J%NtUyMhm)RG~9$7;4>5y}cgO%bht5kJq7)!zw`IS{)7 zph)gu;VFXg-$rMTbgOQ5VnPj-#6w&_-FaxfY@$v+6Iboqj7tl9yYbY>0xTQK`Jp+( zzO`$B2t$b3A`_}IkSuev4Pd!uj^Q#=X2aHonKd1+7u@oo|8P!)D+iJNKo(Iw7jZo) zv~YdEtcF<>+{@3R{dFveebzk@-aW3HDidy1NQzQEedAy|p@YiN@c1V3GaWM!mDh#& z!NoP$eQjPChQ4W)R#*557OzqLtFqX!ByjsHTK~h%J#R$-?ZzKlY0DEn55oFGkU(tzFbxFd>HzqSQJrK*!TGGNczXIFX^7PzY zyEe)9)7LXsAGfpt zL=F0WhLP}{>r1mXg&0{{8YcgZ6!o5{1{Ds*$seH^1nk5#&Ssv_!&Qp?9>9Ym7nKb| z2n7OJ#0eK5B91}q48t$Nj-hD*gaFRH{>JT>yx-81S(`@U<^=ahT<8G;V5uY5Owzv_ zA8qfQ`CNtZXfaL|a_zLaa$38q%kAd6Z}tpWFi%nT8mmh*D$X!GL9@)n=+V{pp@Rax z(5EwM3kBOpn0S?#6)w3K!X!;0<+2>5BAat>Khc*@$K1xM271cn!QB3-FjSLFO)L}f9PUEOD zttV&R_^W~dcev6K0(}1NL^V4yS~PAerqEbJIQ!|y+AE%CMt{($IdZ#Pc?CxO;E+FW zv9DwnM$yB5)pJ6QD$x`JRHwR**TvK%zw^=-ygrJtc$dt<%#jwBz`G(}YtVe1-E^CE zToDS(XHeK7LeiAYnVC8F3kqEcGT~ESD;e@quud2L4Lp03u1mjv;i23b^vHHtvk2RA zm_k(gURD4 zliqQnNRqi#4mt{izuB}aSAckK5?zD3K(?Av9iW|ie&*z1jFl^gS#QxHyXEfO-dJfV z7i>$SOQ#9Dk&fHOb|zygR%!%WNQnLVjzMbafm;QfFdF9iz+p%RwEmN(LXx zSROPsfWCVLm_tlBsKZsrGkZsYCwt8HU$!|-d`~Ske~;?7%oa8n$?-TCWeZL+ z*Q2HGQg&o!^&;t#LaNjcd%H-5s-jd;EPvsL~qxw`et}#aMZa=IO#e-+0@u>nQNEKd#X{!`mS4=?zp ztZVRVP`@uS{z}G_NI_*O_DVAq6F0vS=)+|7lx;|2#e`L!TM=3U==nZGIiyL(iCAye z+@=Fcxa^S)zi8aGJ2G~SAuJ;4ZgHEh^n1Be8nTC&e=V??UxFC&aPsd24gTOj)~+%c zZds$cn}jpCka^OrV*{m5h(C2xxo)N555xV3s|+>GaA#Rzm=y}`uAoOFC-314M^2<3 zJp*(LZauEe$29jseu;sTO zc1ncG>@Lt4ln$^%o4SRL@-Q(p4jAeVagCo| z)-;chz*$qHbj4NrSnU#Vw=s+Pe%Ag=XmK+odY22@e)q!`R%yVrf*1HQS}3bYiZ(-CjoDy?^(DmLtq_pHQn&LG>6ZJ6qny`gDq&SvD1%Zif>f$Hs9k9X)2m;wZ5& z^Uo@yx>4d&Jb%v~fs`3b)iZ?9w@2tN<}zyh?N(9bi`A>pv}8o+AfyYS#*Uf#e3G~G zO)EcUs61b=QB{mcQVR#wmMw5M!a1yl=n~l*{b@3K-B)QJ{PNfu28#A}<2VAg4q<;s z?b?>?V|@>%;+1v%S1dBf#cDhnc;`AA6SnfMB*D=~tZtKu>&44lj+ol^G4mU7O2Q7s zQ`Ph8Yd&1r=}@G|Y=R7cGSBSHa&Wyay}Mz6S1&3Wb`kVvk+ zMrKT@7C|swapv4$tc*BXcLcu=r6+3prVP34Ds^{D-J=@^>CHF)XEm9hVIBOGsqsZ+ z1G$ih3>hktl40;@O;6fFvGzq8XmlX}7Z{alH|=}vI)gO`s&9Ri_#2Ggp#j5+^%qj} zRGL7i-`~Vmw5-3ALScC;VN1-ZyO`WXm&oq%^7yr0fJNMwYav@U@QeOajN|NY;!+SY;E01PR3tp<4Adkxt#}CX?=$pX!1PFH; z0=SCQxT+%}#-2LXF1FAerBcyBx=1X0y4WIb!oB&SO;Vh$UbLM$U`~^LZl3%|7Z0aC zoKq{_F-&Oj%;@U?W5TL9Dj8R_5xXg+-<4$=A1$LjPQLhN8w_nBPm^s-5$D{;xVsLZ zTK19OTlLZf(Gw8v#_Zr+Xm)4z`E(MK8e-P%Cp+bj@ywvTp_8>T@g{Q&hOfoMLMDHEq$Cd2C+~<{;0Y*&=MNH++TEz_1HQa9Nl_^P~bng!?kl! z)96R+%+Vm;*%TJQ4&rnpnE{i%^*EMsKobrVd3K+_cZuX&IsBn+k@eB76K$hy1wNKN zzpRhvMW{(=tGKsC89vk3T)0?Km$!_BnjW@eI~@P6Aw?8Y`(nfQFCUH#9WAeusr7oT zRt(ISk-R;XxThAfT4h`koaZnv&_o3o->4U1YV*(72hn)S=nl=f06s=J2ZMHJ5kR;d zA7g!EtYH<}hF^W-MqoQ%j4*%Pta^!o7x$~}BQsW}X6M4>^I{M|1UciwT`THgYKF;A zm0A$&z^h7nK~S_1TQp+P9j_eWC8lVu*iiy>Yf#M+*!B$pQtu=#e4=0WAQ4`;U$=d3 zBKy=fA_KueD7ecV0UU&D3R|B&x$oW>J+wVFDL5oT5d=6oB1o&)JZ<{4%|ko;FfgV& zr=1-Z?Zb5b8KZiDs}n#?bw$Dqrc*=X-w}!+-rN%x1$>lx!1xd}d&a)MZ%Eg2j32X} zEBPNWfw3x(;fP7xUKOl(S~#_h+>5GC!dXk)HsZK%O$3`d2z96w{8fms=a12-yC0U( zcUK?S!-80g0J*UbQTt{$_@$#JfU<{XxFgKeEIDe$EoC0IGv0BdQfmvdAYLkAF@TDc zCw`flj*dewOp$IeaGuh9y zJ+?>JHG?ZSd2@-f=rs-u%ZjL$O{}BHh15aZ?7*{$eo82yLd%poBc7BHZspKg_e(1q zy+7_%#F#R&N?lsU)hhDF0&>EP89nI8;T{trXydX$Iqxc5YuIT<+%|cccf(AyRXqyi8RiY_Z_~y1c!>6NL>Dwv6AqLs9o%O{ng7#4^!1$em@M4@ z|5aFmUPPirzhQFDz2+VATE(ZNcr3O$0y6i%ov_k)w~r1DX*b_t1dGjf1WCmhgHmf4ksV`{7Ng}i(3*sN5 z9sCr;m?$~FUY^R}*XGI)%k-8F9vkR(>z;lc$dWA~#Dqb< zeIJpRF77@i?ZcE*EgqJJaG(L;l$DeJ#gTC%878!b%eW zc&1;(0M+6_$aQ?ps_y^^<1KFUKr_sIVcfm6_J$_wa51_OlrO9oYA zk0(|DmyCjv7}dJCU$H_C{RmCWBzn!y^UL}gh5%zQ;Yq2ttxF~KbjHqMfmGxt4#iL} za!~G*LBR$_CSG;X{m|Lyc&p|-(%<4>fWY)VsZT#R7P)dsYET{Ce*x)Nnslou_Tw@h zo4FlF7*JKa zh{srjzlJap5e0SpmzkUgqe1^cLarJ{Z=~b)*^)VFHs09PXseJ9xW=Wi67E2tYbevr zXoS2rE@^zB&i!Bj`c~kybi8f-LTT#^TB7n^?lYpLAFu$(j1p2c1nt@8Ri-zh zB~5R-ak*G$qB4Sc>hFI_yAmJ11YJ+a2DM>+BxDM#p5jcp-2pfqi12tF5W52@w0W$| zFtPq{>%t;+xrK3iv;@k(!6JHE?T(+*(KAOr;97-353pmyT>2Fdv{cy~#*M|*i?m&w?|&a)td zh~XR$+$G3Z#n;4vT3y)Sk{!k?Cs%6-Drc=Bg3o=2ez8Ra;{T~3`b&~|!`>hfK=j5J znLn5H=!9+C)WSzISCsw2SBxS&ck;mC1t;E-x5|R5ekynZqoS8b2-G(v#$pQlT8m^`GM7J2|gld*)vJPaPSg3Ept{84lUR&qoIAKIkqou z7nmsC{N)beZb{$HpG(nl^4G+PDu9*s4L-9K0Ta_DzA$s zvvFZE0vP2W0u%+n|7oyc71UN4SP&C3uq{&72uIze3piTXTKfb8k5+4qlAG27~CQ!zT|WjRJY)g5IlIN(k7OAt;Sxg{8_$F9sbcUqCxivmPT6k1C8xNr?~o5U40hNzgN{3+y0z5 z+x7b)GFBpGnaZgH?U_&q@-m|~Guu6u6=H&u_74)JW z7u!`Q6fjQ~j5zKXDFG2~3L>eKBT$6c1!(&_S`S&jxo^;R!sDV=93ZlwWB1g#W?A0O zN5&$(9E@;aBoiw(c21nKdEPgH1QA5sF|V1vhQOqi5mT?q=J*z#(!YvKbp5Tc-v8m4 zlOU?B*rbBibbr24AT#)TlB$4Ix%#D2eeTPqVnMoQXr#q>C;ma<-#7!YRs&r`l-OCx z5TIJQfHNSDGdNe_Us6tT(?=?v+&tq;1GIxs4@`BGwS5~jcZk3%_W$~NKN~JA#mnvt zvnJiXqj_L0I7CBeF*_c)#bsK+u7AoELikq4-s2a-I#~Kwk+zY zU<=bV+B1{*lw_pxz~<0}8UuMuRx98aR#|7+O<*dZ_j|j1A~(aw0QIcxqz|8wkK69r z{Na--ukVPCvAg^;EJ@pobOn3`rgI0a%KPH?{*-ZA3swd)|#)Hxp(s~+H5qFwgO0@l|HfhktY zzpZdSr5#)acMlz-xkrZ`vJHo&{BNd8UzUBm60DTmS;diE z>C^G*ygBihd}olT*omOq-lR0#nPm!x7Rlq2PBs{av{ULXs0=y!Td!*G z;d{sD)KAVOddMUl7z)MlnXBNNU-n&ZkjPZ?D4ggf|MghQx{gm3LeaoNou}jsJ7e^P zNZ}c@m@hQFeR&v$X#PlS;Jp3K7AIdF_$*EQQZfdz$?DYzxKK1AkZ93i*F~nVm)PM>%EvFle*>6X#8WdMihk2U7@3*P z;m;H5#+$~@%?ZjSQ4EJd+A(RVSwpt?E^e5pcUrX8-%%6x<0Tjh_i(3-_v`FG>g`M$ z=Qx*gXb!VrN%^;WibmkK1IpDx^)seP5Re5UMejxazjF|XPml!p>jxyr<}ywPxp(|i zWqj7J%=gC~hzVSKsC(<9uZRau{=5AQ<)P&~;{$!Fy|q7$_2%Yz{gY%WDCkYQdm0S} zj0dEj@?wVD>ymIDjj6S!ao{?}4(Qr&mqP)M0XR%%*>EpF{n`xmV(V@efz?TC?1lN1 z)ZA)|qxwU}8Zu$%Vs6hf%W=)|vRY1gQl^0~a;Oov@D5Y~8Hx=A>c6+U){yw>kja64 zf%*s9*wE%zGC2`HDG_obc--`661XE_px8~1$b`;o417RCdn?T0zaw7*oi}tlpCr|4 z?WyChsd~MOZ6r<@AkyUAXY8|Nvts^W{N=Ohilmg&r7q~(9}laqQ>IUX7$$9_7G^D>*bYBV#htWN3i(g=XQIblD+U|dxfGaTdD-F}%GSI@|* zlJvp$uqXMQN9UoWp7O9j>7F@?w@WPjF&4O@&jzm(V%`sXDIO@OyhLYF+WJY!U047Wzt)EPJ+_dd$Y$2=x3`!I*k#5hkP0Lpb1nDMS%yyJB7NtIj__X3k57gR5c zhwdEVR2YMBbKa!5E0gR)^`btckW)bmGwb+DoP1|jBv zTi$t4<;Kd)4I!&5&H4M}ptweCb<7C_PJiC>8VE6E6a9NS-*-Rw42ds7S-VM#$v|PV;U9Q@F@F#h;L+dx?LZfEK&JM z&KV_qIqP3?NdB4F(Y$KE;gb1a%fs(0n+UJXONjc9-tqMV|&Eq#}3Cns_rg&(!|e7T(l z7}_thL@}Ax=qn ztZ|CYoL;cPCXP%xnDSjm@FXu7B^deQ-rk;Av8DqJ=1UBI)E%CoLNgxSYNIGSd#|A> z{Vp!8z}KXPQvufOuU$SZNq@V*n;0d_G^p5kQcneH#)d!V?4orK%#PYq#%C)by;FDE z{PS-l7^*56k0GeV;{gBQg_UL~^<^G5dMh<5vQQ;w5QM=479;?XTpfE-d47R7El5e{ zUKd~N#r*~QQmsAG>-*%tW=bK5CjatWdut;~SHQ*8!;z~xVMPQ|t(8(eA0VE>A>f5` z{vY}CSSbq2Meq~@nYZ#pzibBrc6R7IrPMwj`}iu0+E!$QZSR0c*~yx>=)^sPyJ$cE zYb=PHbtIYCd662-H%2|}DiA#OboAJCw{W=c0x$vX8zjF2?!?slaW=lgXdn&cO>cSS+;7+`jh8r{A3$dGiuY41(YaBo|)4x58^LbTG5js!scu2g-)~UMbfB z9tIXES->eqn2>LBOX*fJw9F1}h97}03nCC(eCeX>NY|=}ikMQ~3cd5qM>`svF>^U! z*Xzo94kCfz+Q9=p46b$?mOn1Gfp?I6&X>q7m`V`Zb%d>haz!3JaDQ5TcJUsH;*@q) zbY?lj z8EN1YOU{E6Jw~vvLYL>tN;wj*^$*ed)`w{56XSGJ|Cj>Z*l~Vj@~uP+km98v4#ljw ztmM|jb~0iEt&rA$S#W633y}U0Vlaq8m%q1=gAkHi02JLog88;taIRg`;&QTkLDI}y z^}@5xe`+VWd;ipE)>$yx?bDXHv1v`kk)w^}d$fDfv)Luet2JpGXz;<5RL)?t2ZYb-EuyH#ECXTwadQQ;1W%U))lC{Kc>5(p_>vd&ez7OLe73K=^rq#QBmM z)C5J}fCRU)wfp@2gF)I{UTq$QSut|W#}!_+8TV)%+~Ge%8=zkXL%HA$8vLZn@ZQ6E zBZmBexp~*hnOrePAFrsdMf5zZ-EQFlAp53BHQ7f8IxE2Znq8k6h7<7Pa+ z6(KWwm?S4aNP7tTZ%}&*mai7c5=I2^J2d1mM}`alx#xzcmKzFci8cE_CIpL`&~kI6 zZl`6!*0-QBK@q+e@HX#<#5=J`oup4ds0jDW@lU@~_a%Hh@TgiMAhZqJV)=^6yH_>m zD_N+1PtEQLJ{+2%OdQyi8n!R4yjI?f!1p}*DDq&AO5{A_L zVXF530C9`S&=Lw|1z4d7W@^xgmsPV3S-P){*@9`b-8-7OsWaU`rq5*{6iWH}bI=h? ztoQ6OXzxj}zNTJ3l(%Hv@CSQwe}(s0rlFSe*?D0+d4_8Mve!v+9Kn17PC6nH-(%Iin$>(_;}&s~#qh17Hep>VdD{$+XTKwem#~AR105O+SpFK! z-jfoJMNA#805-g~aAJ>zZ0pbfF_o77c`m-@cIO6b)=187Bb}WD(Y4MDFzhtk$HLpB z*nPrhTGzxPjc(uBKM>2%3Eau})_~}vUXo=RKR?Qn1G`Cv+A$5#+1@ZL=iYq36YffK z^h*t-ykOYivYXIk;-HO!t> zn6}HzQ3xN;6kOyPS3qdYLDsJwGmPBZhcx_TkY`Rekd?i~5uMYxaA3<#og=PHcSLS7 ze58Qpvd~WQ^bdc6pKxBGrHrZGUMtoi_-H;V5(pz$vmG0@1f%`e8ug5GzIi892Q00B zzXCAaGzw$lsXm8Iyx+lOyeCTzYVZSu+09^qgUt$m9IAV|#w@wV8t&4s6#>WiuzLV1 znL$PShNEj<)_3U>EC00xEuz8{6qSTpp#US`T>;-zPZ3!*9VUY#S$C+Je3VxMw;#3g zi2|PnK01gBwNN1(EYbzgxgAEso;1SHM~E+b@DPtr#60qq{U7c8{Q;^|25F zFISf8+1`;i0rRBg0V-teT90v)0qr$ba$_i@9>%$%B_9mwpy+Pn{1CfUyiV@aJ6gQU zfEGH(N?s13flcf3{t%whjHVi&o}|c|kU!?X>FNivj1*DC??l7x7LxPMF=k$Gz#II+ zfUW^-UIR<)s=A4;N~tlRU9LRNf9!xW$9!5N?4z|P6{4|%1eRb*V?Pj(U*}(UYCPc# zK~6VgEscxp{}kTDxwa8hXk|k9X8;iEM!T}P@u+4P)+;D&6e?xU_lpZ|+rJ%*mr1z& zTdYR*kThY-Q@U_5+}YW$ftAz+vxETuct)sSrWeYhELxkLTtE?SK?T?tJw<~Ym>d7B zmEVU4DPeLi)L``KjtAlR6F+>oKQz{@w(Gj^s2b~zhgtG=3iA^)p?NyvVjuQ>*N8Mr z=$5P~W8kOeGi#I|jnZ$ll+!E{d~BKym&W=^p|!ZlstIqlbB7&&)IsTtd=4|ozMu#r zDYW)#jNA#W<c`YJ}un4@Y=_WhjhKj=dI$>Km+7!UHynPM*+Dw#5z`=8s zyxw=?ZtkFDHC>7=ZzL9Vz<+e)wI*YQMyd(Srv6mjwgTCJ9+DW<;+MgO_4C81dX**5 zMe^$naN9;&-?Jy9@-(prTKyrK5}4DmI}@pNhqr#GdxZaWU3Lf;Nubz3%deEssnE>0 z=VCZ^)k=SZe6r#jDRs#{L(tLCC(a2~FW$G&U=Mo+YzD~$6-A5harkx$T>+E^YVQtS z$n`MK9}Nk0z30hAT9V}t&q8P?Mm$Nyl@uc7O+3=V#X4yLjoLe-0L5iQ!9L+x}#yx3K9@r77=_8hR=NwZ`VoKuEL)MgFUh;-hz5Hm$DNu>J zebUTF3w-OEt%Dbwm|Twv0#YcEDi1HMXMfqFoiq)H<9-FPvIok=>g4I$UV>AO7`;=6 z%EK>~yTX4uqt+ZxH=aN4f=CLZ9E*l+OYst~jH zs2?1%d(#tjCy{OkOTeqwOu_;LPgbmBC^mx*GP9myly`wxmHy7d{*>lL8^$LzUgUjF z_;8Y-@WdvlLp`pEnB!Man4!L7Fr$SR6U;$Nzq-MXZFT6DSwnauOv;~ddu!O~P?KKU ztC)-E%yb1IVwCOSC)Ngej()FUs{9XBEZBo#LD@Y^2MF*}s1EYdvI9?s=|+OTqD^h1^8cmW`)W1}z~h+gYX-IvUL&6+DV1 z#bqMx*Vz!qhbm)w>p-VUY-ACW26Zn)l0fVN%wb3gaS15k=o047+baEfGq*@-0tp0ZWMAKc8qG zL%}AAM5&HBU&0aX>NQb9%tMc=#UFRJKr3m!vgfJpL~_$i*UP)_u*%MbesJ%nb}RyK zI;>XL7nQRZz_(^J5Fum!jZE`Hxraaa#PO680$BKBEw3Ovp*5l*7OKHeQrnAumRd*w zMktoZU=P!81VM<6Smb|xWQiEb5{OSPddd?#>0!FUBy;Dz<~2=y=+f%OZIY8yr5s$a zm)Re*uo((ik`;EdX#}{?d>XVS8NH{T3kn&IQSmyzDg*=19;D(OMY9B|yi_kF>S+uf=Z{wV6uqwtqzyjP5T z2SOW1A?@&9uy$4-xsx$;lF&~42#)#Y#4gKK#aF#7ajymP|aW zc+jJ84!V{^Zd`+pTSW_{Js^sn$nF>lwo{OP?-!#5esS?z6e7JW%oTf@5FM^VRgqS9yd zLzJle^F38Iyd!h14iEeo&}v$`2XF?swv_#vwV?Z;Ro#(p{)#nsZCBos1exh;S$b&o zbH*tbqS@XmiFa8lur&PCakxu0Wp4h2OS;-Z>A))xe8|x^e1}qn9y$gaWNHnR70sc$ zaEfQwS3r`qI7ms9hHrH&9x>ejIuh9-*OU<3W36Wl!hL;FPCO0yg_UagrTL>iE!Q|} zK~AM>H_9=)Sa6W>EN?3b6xDoZaJqET`cgLKSBl#kFs*R_+OG|?EwEoH+bI6Nlb>4N zQItz*Ug=#KLzHXvU7ERs{D0f=E+-vY-H&(S22QQ^&9-IN#QFmw*;3OJ9p`!58_0aO zAm0#r+QT$V>qV-PQ6y`FgLfta{~A|TS$L234lAl-(>!P9j4=MO=^hKGf=UZKBP{Q- zUcg-T^hp4@7!_*e?c=tM3)Ta-f+cOBaph_jiKtEKqQO6qOx+mOFK)Las8f>Xb3xAB zK5t))H`5{`GkkABi)|R7^aS%ZYmE4SCRWwFts;~REc+2-Wra9rxhsf{IER_NX%<29}h6fYO!+77^K`Z_TVD&GFZ}We6DvM0D>hC@sFU7sEUS zL8;gYs13yXr-}fgMVYk*u83EOC%+Vu!Mq&X0&3+I$J8y?5SYrCXz7rYyZJotuS82_ z_MSSnz8`SOsgThkpWg9lR9d0zG!mF9^wMXuMVpa8uFlO5b_HB?(Cl_2IOSei8qX$o zB_T(;Ou7jRJO7ky?^96$Z>IS2+_1W=w_j|jO(eXSGb}>O#DvJ@u;%HD#{=rX+ukxea4xIXiy z!&du8bwJtwSlLSNoWae)jN)QUgAN9ah4hT^vOLEcr}^>UJ;p13MD**{|EHms>r;2hm7p zr-A1wGEipp{u4re+A(-q$c+G$L;JGyFCd{l&o;_*r;QelYbP~aD5F{vSv=Ub=Oo~a znP`H2ueb+CATo&(2^$no>76~c0XJ`)Y!SwUC3yv53N5RTaa%s$ym95Gub!YOs>e9x zSrFIWTd)2VQAyM4(Glt9DaMcx>E(WO_F%hf2-Cq?>8%QT3_`Y6??Fm^Su$t*=M3Rk zwPx%i3u3`=?nBkp&F*mGJcufzT!olG6f=uvv4A~>S!gGF=l z#+3+(Blza#bLD~KaIpkVW3)Y0R1=tSBIomVMwS8m5)ynUO5VU!y#>b^^D|3|I`N+= zps_SQ^=Y1=HR;hXWl8Ed>=^b=n+cg-elBS+UWnZO2h#)MOXF?q1!mIq3s+!;H52mm zyU~EX+;KV5r$a)zbIh~9OG?vQWn#f#dymXZ*uK&+4Ewn`uY{TNsorjfX%H1N)5QmH5 z!Kd(>J%ZsPM~qlJE)iz|5QMf4WHW=@^ndRuNf=PY0?W0^FrZ^-t|z4WF2l`gN}gKA z{%he;oh*8zCQ;ZL@KR29qdQ(o*2k&hm|+%fW8fnj6b#@g?hv#gMwfksplSL3^>WBw zE5%`w(7E%appv-7w8URz_&;Oz}|AgMxpgk{%Xd_CI4%M_#GNofp)J>BXO@k ztKyu@%Zt=ForI_AOUs$Cc$h`7`GngWZ zp!Obwu}$VgJ=(PUA|9(RyL@%K=?b)T&IwP1Q9=e{oSS6$-6!M`oUJ?ND+t1*g(@1R@NWMbPzd6H9v`7ILS>{VDSx(g5VTeoXaP0|up5SBixKQSk+qr7a!rsm?v?o$F-V?3 z(sRz_TNwhV#*C7QzoxA&|MEpp+zF@>ptt6al-yp;>L5_X^~H;yMZBiB^V`{#N!j9| zPT6vKyRaM%D}CDi@%0d$p%rm-3wn;w>hVlmS?(|^*qPxB&Xk*X%BSh6+G3}1j%UAN zxJ#kc6rqz)lks(WMeZUo@tOKpWOpv+p}sgkuB zyI2$z1NR2IjDrS7@ogl^7ReGd*g9)2I-O&H{ljs!%+1|$z;I-Zx~Y;Vxxf&&ivw$~ z(44hpU<4=|$(Vatlhdc!4T)nND+4TCl8t6#^&?sgkwv(*<|((|Ew zy=QriKecXdi;fSfIHH(34t-pR2D@}KQdRSy zncA&rBnJrIQ1yQ*-m>~`%%)wV5zLQ(+^ zYJOq|=6Dp7yR^~(CQ+-z1@zl}UOmXT-_N)Ua)x@NYe$6fG(u|A$$S$vvFKgz@`t^C|IYf@`FMFurf zCSpgV4%{yZze)p9Ms5{M{OcM|NMrEa)^S%tMvfyi`@5ngB*PRjS(E&D4C8htQODy_ z>IvO#TQIS?t7dXtB?5y94|y5Mqg^!k`lV?(_YW4^qY)1zb&AJ<6A()iL3J5*c4t%^ z21?b-Msg1KERroX=9s;D(P|;EM*vxT+7_Vx?LQGSNYz}vwpSJXg3gY|^a78xPLVu2 zCAe(agndI-az9_k7MBCB!TSkAA@#Mdl;nvuUO7Z1WpH8VZHL=$sB@y$Nb$%6w@+|b zO_TLsTOuQwS*n1pp~*siD+pNYvCL64M6q!jCvN|U82#P^S6u8MOmls+86fo(h$B>7 zyFo1+Ko6A~JAUy@ilaf)#-q4#ovXXe{*gZR@{vCbA}zVu?r%o$E7Y2o=84whYm7`O0M{j0GvR>F-t=2`VX( z84yF?(y(Gz(+xyRIH`qcbAl4}7xc?))7l0`tQ&U|0;)A8u`%%rWUb8}9yOyGGC5*y zS)&U}z6#?OeDtSY&>h4g-=(%x%2NN|9J zuF@f-+t38MLBd2n%N-tXpq^Pfj;D)8ptHw|$mz9BQ>t(|Oizf!7KkO7^zfT*4*w}4 zOqgAG(G$&-_)6N1CUpge;>B>64`<=yllHX=V$K?|$*H{yS9T}vx%9CSCn1u$q0P7@ z#NA$BBb`+Wt4?$7W&o%AuKDJzPqsPDcs89Z0Vrzd3^Kn9t9^NQ&O^idE1227p2M|8<@5$x zAjUT;uOAM`0UtHCT*&<);8<=F1zM8Y5&T5XAw8`oP-v}Zes=@e{OGFda39kl=uNo0 zOg;gVearbrc&vu{ZLF!1?gSU7M>#s(GJ#Crm8YSNX97N#r&7gWM^JP<#m2@HSxG?6 z<+lW-UCK)%9Y-`0rUD%z+5ylqC6-i%SG0^HnUw;5{Da? zHzZOxeNXA8$Q=rF(h#yw829VOBcTb7Rs7~P$M-`ozXqDpgfU%?=R4LN?VkeCZsC)x z)^1}f*0XRy4tM)no@`=m#E1I9K56Vie!U)vEw@>+T0zYEaHx)lj{)rDiu932W`0l# zFRt{ZJLngE9Srw7`p>T7L z29z7+tz9(@Y8>Lb>>*$yg{B;0lFOdP8rAmLeCTq81P^p`nNmmbkyFaHGY79j1IZqEykF2qhJlJ?>k2+L8Mj6I2 z48Xk;kwjHi*{8ogxnN}-sS%Wp!-t=I3L+dAg0E8FrY}OyngbwRavEfsxi08;F9nOY zU*1Qc_#e9ZlRo&zS>wl?@PUC<<5Y9Q&ql66G<`{|YnZ&MOk(}r#wZ;;<|erLdQqJ^ z6^_H!tbj!Q>)pO(=rh=mY=0~De?H`7sdmPhAn{?pw>{S&=+dmzP*2VN)@lJ?nql7X{=JH)YaFtsv&nTIs;iGz(_ zPk(yd#FP{JVe_th0}-ovZWx|{b?juQQz+Ewkk+LaAkoKq0z!(12H7snSYqea<_G)-UK`nKOsz;VrWHW8t> z&193d2)o{zkGG(n$B~-7Ihlx0`oP)4NH=107IX~##s@ilS1FKzKOuNUx+ng%u4BGz^LAN*GM z(sCQ=Tnk-VH>tk@Uz9PVx@MZaN5_QwS<*=MV%aT1J_eQ{i~T_*azx#!RqFl?Z*v zwV^Bc5lfV!<6G@k_&`Y${bHJUYbbesUF;=m_g10}iIlaF8qA;?Qok^X&&+TW%i(`-i7EnU)L_Jp8$FS13Z=`eIW%KrkQxUyu)?p#thH zoSB8sT*tNSlA-+}XHPY*hlgLuez+O(8vvPy#asJy{`=3?=!>|FRI0T2_rxxJ_8Jit{V+tTkhj5A3AKB$_G6ecIknsEbzW(jO<}tCLSxm zo|M5jdyPGx(jBsL7^T$fkpv3N82rWGW$*!*On9C?-Da)eSJKr_92>a6$&$chblU$U zT#k-qEWj3*LSC>ZYh+anJ10acIbvi6Xt_*aL@Wdk3~p>TD%p=pXja$y8)E7ny(ivd zaC7s&8=?z52G|HzL5$%2EV+A4SZ5}8)GL)F`KA2Ao3H^+t`}zG6SYk5%g_9+P4hl#o{&TLZDF5M0BRst{YfVL8ovl~ zovjl29Yoy(Y220rC5i=R$&m;~?im)?x# zh#FQ4=Q(PAu+nsWo`DD4ri(5+4CWa=AkP|siSJMwNpczhi5vWku8gp`<;R-BW-R== z?KaIX!V6?1w`~N6LL#xEqvP;HpLj~>I2NF$h7-zU+G(MVN3<3bm>1=q~weH3vKpA&3Td8Q<#>klh4 zpRTAhU7wGj_mJX!+CKb1da>Gi<=el#(LhZy-UKU6bV-?h92c1fpw7m$5zkijY|9Bo z`Q(Up05r9;Nq`U*s}susdh^0bWIO2YqB+i*n%WCL-JB3i11LrrTFHVXvRG#8Kc*E`o0vJ~O+$zcq&D9>aSFSANc4J) z#x_(Gt2?!>3dr0o)1CA)p#8-AHEd;F2lc0(xJWeao#4%1P@>8f?uKSI?6e}!sMg+< z9+ToVJmeA3-jE`)uQ;!G)Df!vvePvQn#hW^4Y_8LQXV8-rA7=ax_}Irl$g*YUM2E1 zEO3i7WN3<%isFMfXbcYR{%TeMQP1MD z7P_7jrgB|WCaSin!e0Mnx2+68!>iw8mlfu`bXAN}-9d|Cl9Y*pQ=)EqY!P4wYv*x1 zUvV@km#}tBE(R!PoLUK39x%Q4Pbc2Mlzhj;3NmM%Ie2^PDdNWl!^HR5a`5SL(19{B z-yO6>k1x=8M&qfTu57JiAtT z1@ia$r1b2zR$UEkgH&idSe!L5sd4p`T{xZP+1AC8C|FlO!?#m%^{Lli3RcHxI?fIrdpON!YDpvC+x6j$Xw(QeP3 z_0_H&#|d&(YL1G8*;Cn=lZp}^z4kpLlE-2FBpw9gk=b4|rn2GcsIzB>rT&Y% z(-#CQzSf`!SZp|T6_@-hJM|__D(vma^#;&mH-t6@M&_cv(x&^@LibwwL5Xhzka z8_uE~^cX@${miLx0isp~d>RZwBm-rPxnS71V10<}>SBB7voSKTj7S$mFtY5U=B&NF1yJr#xYf(>cOg<|hF!cuiC?jKrpTLv(t6U1r-Ms7z zJ-+40uDL^1o27444I;0w##N0_nh0tv1Tbl~p>>IGHg-&R{J)>cv3pRJ(H+Xoj|w+{ zt%t!mZ9(Dal+FpTQAx)2u}G`k#DkZG^qVv@t84Z@0CcJm;EI%Z6^tt8dCvI$%q$$; z=zbeuWEb`JS}5pO-B|`@r#qjAS98|6m&kJ({!*pJcoA?T@Rn9j74Zo_L4^^icD@0O ztn~1 zV>+7f=}}otTAB1(d|6iXuQ!&oc0!qBlY+<%%w@5w?Oh4=bQ1v~d>WK*eJJ`kHtrk* zFZU|eJe1E0qS7?X<2v2I>8-K+w!$MdJmT)8w_!&yC~GJzX;#G739#-{lv>#IszcP< z(^V)>Fbl1EQWytNwqg}dBc1POn$t1sDq zoS3_PU&?#L(_8h*o@|heX!hg$R_{kjXHl2|eO4vmt4xjx)Yd5mV!M)60?jlYXd3jg zQA11hc3E5d&Evx^=TTd?-y$n%`F!EZLm{s-Bq5bgcmB?%E@Uf8n?Y@bocTEw&@J$= zmsD)8qm58xvmk%2!o47q<6XuXMe<-47=09@14^Le#ekdhLT2g=qz4(=6RRTph``T91>E{}#T{Tys^8Beqta^6lQwW;)Qt z*(0ZSRZ)^qy_^H=7z$V&aiHE?6A{KSl{q?jrikgHE>N5k0z7IoX{KD$#`OQlTLT3N zE))&4?5Hamhho4})LkgCvDeYnI*GN2;Na55A8L1;T-*=rH{!o&p2+~kxxFx3t}+&1 z7U*OIx$ej&#@g~;yJhjx&}m)qyxb}NxCr;JpCqOID8-mWop;o(rZW2P@!1<`(a#{I zu9VLe6-D~?o%o9f`lAjoY9}2acb>;>SX+0UqRFYGzl0-LKC<^vb#TG{?vO&zO2LR% zYaS7PVemTJS+g>^;88vHFhj-HIusw%T6Q2_L)xV1ekDmdJ{s8lmo<%7$p5T?6Aqb_AdiSH?zHKmLg0v!lHdO~PL=<2$v0!KdWiLvh{(J)AhHn|9nI z1Fb~0K6aoU5x}3%kz=>U=B5QIDg~mmBd&HsV0XLNnC8K|bg7oS@FEod^;k_;1$E5E z&T53QYG?{~lRME-Ln<~r(0mm7X@n&3@0r#E8A+<#b_@GxJNy%emLfhGztPzAZ*1_#4_xFqDRw;oR8HlADQcE5 zzM)FYByQ%r6o%uLk9d(nH#PshTmDW#d8z`OJKVPPJ=Y=O6s(+SHBNHQddxLSg~Dj& zfv&t1V8wcoEP2oYn3?rgrk8yqS1DmUs@Zw>DEPt*p8 zmKK9STCF>8m83kj`!?uywq_PcoYmH_+yq^boJ+zoy6o5~L=y2VrsWGqU+f(f+f?WM zE2|EH`Ym3N5U8EfA_qt_eYm(|!i$nlNT` z+mslyr*9bVdvzqw3H<4(Qm)HdpD&1@o~^1im}bLZ9-v|TsQ75aMzA!O`{6AMG51^M zX~Urwda%KvLAI5G@;J|4mTNq9kNaFRg>_*T(dyl~IlK%H{B*VQjDb~HW6eP@Qw z0fTYK_HL=|yUKl-I|l?GeTBV0bz7V`*Il!7c>L;G-= z)8~8>L`i+5;~u`>qs!l;ldLTvKG089KJ8DUebkZ0bdr&15qs}RP=Gq*K#+{v8>kjz z`-uSm+o)z50_>QHzmC$TOOI-&C{_nb7E_bSi`<+~xV@Nia+0R*yyH>;o)Ak9cycBs zGzz^INE`$Ch%Q8~JQ+KnUESJjK7h+2qpJ)~hem2$o_LG#1k^8$9@_I3Kxp*oEwbHn z9|HdX5lO=9;$iFcRgLA9PTIH-2|XPkvi#!}n{BU?!9GrH{(?h4kL}5`Pdr6FP{~-gX0v98enk|B z%=#cAwcgcV5Es`T3`#tH2l)^BlV2G-iV)YQz$F5Ce(_g+03{l@U#?75@3+b%_4YE; zpbQGOgn$S`q%uzqQ#W^e-lYz8PgA>yrYBQGcASUq$m_VPLpGWcs>4co%#0BmzE)Qp z1D~(wa<-eFL53T)8_(ss@#1MEZ8L*z7&+bkHZV!sy&Gg3+cQrB2RsRyS2Jkv+Mb#+nP3qvE++wmoD z4?q9CPfR4?5N@19`GSt}YNA}|Yow=CU|UZR#j%Q(e_fhnoPCvLUt#MDBQb^y80E}W zHX8V?7~g7>1Fsrnqb%>n0%#1lVv%rf7^YgCDhaFKqUip7pbqTpo~TTHlJ0sH(S4@n zj|RvZvSPCzY*9RSW)z>WfenA0Y%yZnNb#p+hmP$kn zS0&*BMv~XW5{VXABQT9Q2t?1pq6Poc3S2VAx4Zu1`rOdM(l4UO{WW=%&xBsXS9a=J z#Q4*olmfBxIZGr(v@n2p?(SsSF|<@NhjA=q6`ImuBRvJ`ER6S|2hM*3zYNcC-OtjuOdDWwN)sJiL`oB{-vDg(RjzMz@GGq!vB~*~)bnQTtl=O|z<2pgZzVPxEGW+H5uv|DSTxGoh!IlAe(KGAKOe2Z`Bzfr-1wver z`s?1+P=5_zkAw7t<&3c@ENKHl&F4G-AK|)^pgFTi+P#LYjTo<9(;SB?qL7Zk7bxY2 zWjT*&e@oJu9n~VzdRhM~f}0|?lJ~ks(i+QK%qPeO+c`Ejcr*_n1MHJL(5%& zyG`T$Q#*RHXhZ)8_}eopRE+J|9$2%^cif>_S`ecGV9ME#kh@Yi2BT}pqKfT3litYfQCkTyjkS_=S~@z1uLFPYJf`!z#ba6Hjx)+Jt^7>-Q(ybj(U9^O3&MuGF^!R(q| zi>QoP?ARq|qf}Y}ul@e}B?W;nPd)BM=N-Xb-;bxDVO$5`Ta01AD84mZMs?=)g0NK4 zSevPAJMdH1EJSzm5yK+MBNXjM3qBQZp1n?>E@t6k{?vp^`iEQRadn5ge zy(0R(9b8Nlc43Vy1R{%>_iA=J6(|Qz7=uHJ*}XVv@X9DlgO+vfOZWaQZ2(^kmq3c~PPZcdVHJ^@d`T5_NFg(3r8yQ@e$C`n2X3oN4MvQ`$YcK!YrVaPJM zdvZ)X1E$#PqRiEcrzO=hlJb0uIQfk}`k zHA0|6AEoW_hsX6O2pk|2lfl9hz~KE<2e*}pk}iQt{dRue`3_&IUNq8pH*Yg6B~10u zX6OcU3<**PRmn(9f!&vK%lNKbJhwfk`hQPhRBuR+F9Ds*=2Z@RDqXuS4SgU4 z7vp@{#lIa9I@L(xQw$|tm7?;%{C7K!dXAoOoBAqDzGH%C5M^Z_(%tu_v&P-%Y4M{D z>FgIkaU_m@Rjf%;XAe#}qK2?uE8?Uh`}`?M3?y%~|PB`_1V#>v)=gO2xt59=mCZt|Atj zLut7R(*<+%jx%w<&k@Bm)WL%`s`!KZ#FUzLbefMvWH-NFEyJ6yD?jm=z1)Gub34QQ7hxn2StCPfKs-Kp zJ+K=6Iu3x|pqN{J@IVG!)z#IuH)sr^d`scE+$5i2sJY5t=>IAyG~}u0w2xpk4&ZgT zX#i7p3V|574M>kx!(&F12ciCkSt=Slu{fo)deCkW;vgm{R{tL3O2u%iOq$-;Kyw3< zRvIYYI-a>+o$~R<8W}ux3}Ind;>{6*!g)FT>^fF1hA~$XVr?$ecqbyL>ROwvA~)y2 z3p(VjLD0Ql*3IGXb)4Rib-aLbcH<|xC&~2y7k=qYp3VNVR@R)8?tD>%bP7_2Ggz{) zGo0?Q+1qSwLfqm6mg!!af92=SoODT&Y=yP+%!kNV7-JgAVw}%Z$x!4lypq<+K{F!Uw7j86M@&0mCP(OU)1}r^3a(Y!b9i)B6F32K7P z;kErc_#d6{vaHFiwnw#-OoKF)&|J}~%u)^=ov0*M>O9acWoEt0YbL%74GmI0%rEbb z`X)zHnT11@doAXq%Z@lG{gQ*RN@C5Rf`d3X32Mk()rKW#Ci($Avr#Q%_VI76Yg#;N zzsG9MAo0hITzss&eM>qfQSb|7I`RAj5^udK0|1efT8uCG6Yn!zf9j}V2)@tGE8^FR zfmw#8=AFy5Xa3L`s=yxcF{|>CFj@-?;pl|XGxwVJ(ZHGAr0BXzyDcw~ zA!1Y#5+^Qt@0>(2tV9cq>BVd2dFRXtw_s(-)Jw`p&=+8xJl4ac)dK03-LvM+V0?yT z0&Q(${y*R>LZ&2}uCjtKR8!P&Xu&^kLAjm18RyunOeBH5v(y3SiWsY>rMKpJj>GvH z?rVL%{;kt9$-(NIN!{~o`3G3uWhGGj9YjK$w26@9ath*ZyszPQYq#CV<4W zUy8LQd0uLV4ULAb{M%_wIQyzcvw)FSC?HLA0qoj}WR9vxUf)QFZ~f6*_%xlaUh~u0 z#|#&R1_X>S0W>pvUt1;ImvL2ESGmcY9GeZ7?G%V=r*}b8_)(P3jU}GNJl%|q< zo@owyx12MPT_neKmWB8V63f_!W+Pcv5q&JHl(#~SIG)xYlL{Vq3n4l zn)`r*(_JSs%e4pMYEi1B^9`L<@(%O8?SL4pIOaxN5b@4XIu3c~P}Wehx@r612=3`Q zkxZP2^ein}UQkNxn&yHVIOY5TBv-{Q_tuWW4}C!q45dUD3@knOU8np z;sxYLj)52LTWa1^7=9(hLS+uy4Va1M_na1PG(`zhU{*zR!Vf*7k53sX`EPQ%mAAOTx!BHX;y3V z&M>ryMgcH>R5+r;(S3&}II$Wxfb4q2f}0&Z8I(GaKQevsE}ZcT{0&3x36vpBxpfas zffRD;K>DRK)O8b;#fPK^lU1DlE#VJL!lAQ6byzgCsgPR2n6aa~I=IOuHkYNtL0zB7 zB_ncv1VZ)T`aqVfs}SFBShTk~2Z5ae3Tx|lqR$P-&p$NsMGP#gPn6#o{z+ z@{ZIbUfUBDO-wx3G;kZ}33%m{3^VgHLGw9%Z{nLrU87*kMKo&MFz~5l2jO%UJ7~Zl zYkGmQi+sFI0>f&>84em4(S#;0mbAaFO46{ya^q+-jB!N2PO0i&Mo;pu3%7sb=wquj z{TkY8@!8*|qO3kn5MYrDlJu3zAI@{d@sVCTP7v!pEtytHdIGTWAeN<+ zcHElJ?q7mIus8$5P0vn9Zf%&>PKBi8OiUdf-tYnB(9yF{>b8-jlA^aN&6vck^! z)^nl^k~p(WR9SXYIl!~V_$ zg%DH4;brYjy^k9ud1@Bt&~5`=(B$57M`B?0W*@h(C&a7~J~XVbMU*p0S~NlMp7wkF zY2o-GP8r$menPW>cOQVCV)qXv#}TkK8_9^8w1ABLO49>WTdTOntl}+Jc)T80uCxk$ zq3(m~%>Y7+H8Xl+g4##T#z`yj=t;Q5RmKE>-pA{TK^ZA9sv;N;mK9k)n+c@hXe8oG zf&(Vms|&5n*U5OEJ>&ZnDP}Z+;|kxf0(+*}1*dCf`-Y5lMteeG}BWr=$DiCj4 zhY)6k3T{6W=s+!s&JIcBEgSx!*j*qD%|1Q$ly^z=dD&kfB&9v#B(7rq6jrB64c+C2M4aJB`%PFVJ)AaMDt0ds%X|(cQV4X5>%WOu|o-xLkH=%}((W zafXI_+uE28J(r?N6;+I_N5Zq#y%ALk3N9O(d9&@7XPv7DNhWs1dqGvKr~wo#YXCh! z!oS(aYJoi1%6|T-dQCxufoLU;#zrsxGDb?&lO=KHwPgA$0g zaCyJI6z4#R9*)fez{g-59~9NgE!Bb0zFz6RUUxjS#VlhIj))s_k|BmV?M!t-2(Ep` z&qCC#m>cLc`$SB!H5yLo(?ne4srezY^&%j)y%7HY$s%&s z;|X3mszx^S;n<7=@{?*(lP7~zdi)i-xL}^-6rKgyq7KQ_tY<9yNbco#Y1?3E*F4i{ zTulGj1gnxaVB6x%z|?D!eqi`eQ|R537W0vmrBubHUDnTG)I>t~ff_($-eEKdnB)!J z<2~F<;5w}gN9HN?v^2#y9SnROxS;m|NZ-(tu@182_6p)&96ay>^sE@~m_6igDg{FIsVBR_PRuAuv`5uqqP@zpa9O>yBmMWBz}i#!2Jka-Bj~V4s$_Qe zb=<^cjq-A8cnYmNMXw!0vku|Ai@6|^W_Iz*v8%jzjOs&7o$%#Q#pN5Jb!HhERWu(hL10nd7A{X=~0%`A>4kIXGd zvc(1tzurgq#HPiD5hwG?9V4QQvd7)5ehb;Y{b-HfIr<2dCS2Y0oks#67O2l=2E+e! zprzCAa^()-xD&dNqCpRrQ9ozi4yh4wZleodKwIH#Nl-xCUYoblxhFlN?snTZG;aK9 z0V>SB{qQNza;ojCQoB8RO>2SI;2qDx_ti#pc&`m$LqkWMqJyp^(M2tVF7L8(0}SM; z=kN6_f4|gkKMAnWKH!1-!J|akIYumy({jtd;#2zKNvT|xJ3#b{LL~%|>Ndk8>Of7G z`C6KJ^!EkSxiyf3OV0n%R2xz@XXkWb$iftrC}FY=fKT07~qx;c&%@PnTtzwOWZkkkQ- zwQuLK0bC~*4f3w&Y)k)lfZnb~z_?rwHaPpu_;_}6C+8Ri91cCFCbwNnF9q;i-9Dv) z&cinx>LSHrk1;pF2&lv!Z(=Hvp9|V{0EB8r=mA$IDf_VT$*NK>91^pL%7NjV2+jyJ z;|s{M?75zajUx$Uwf^!zANe?auMF)~V~#GSRb4zz_3rGM4%X#~7!PzrRidzI%dVj@ zDd(-oF>9SazJJ%M<>2;d`_A06_O3wiWfH3)|G0u%5C}I%e_s;8 zaB(=~F7_2PfYj8#wHMzQPz}ru_-0rVgg;M(e9wJob=Bp}1BQJEkNU$*9&1(g%5(+f zfq8ON7_R5fEvw-5o#VNiV9Zwd_h*}x*{A{Z43*%%gRX6AJ&aEDLW7iSaF;l}M5A;F zj?8h-3&E+eMkNq5yFmPKrnb07<*SdLe1%jWF0&>c6KZr1)F8roTZ_2x)hW2r#h;Vf zc8!yGxvq{yZkay<`_I23aRk1b1aKV$&H{` z_E$mLs23j{7=hDecF9-3uO|#<*tJQh+$4Im2SLkB;JZ+vrf!ckKzA;d`+^=2SZM6!3^LhYfO?w5}M2n0-2$d)oG;vA0OGg zk|cR@|H_83oG`u+l*djY(!z5oDN$lKm8*?Lgf_(d==n*Hb&L2Qj?xBh6|k>uBk8Kr z8cRZ9R?WF@-LZC6&< zXOA+%kTw&`Mw`_q_~Ow~;VT1CiYb`x<(pmq}fjnk@w8>4SNJ6Y!u0# z!La7sPt+5$J|U0vJbd0nT(4cBAf9}PLX08Sg#&riI0;YNG(7M^T4Lg^XZR>Sc3+;`%_{Xi2%c4tL20F`lMr?)m3Ol-49=J`#L55uEsvKsYT(3aB1up+a!pvahkCB9E#2J)+wb$B^;~m)gFCwDrwS@;f z<>h`elj;{ z<`6Z4P=$b%?8A&~SUlLil(wvqAWr{ptj+@qDL*V^GnPYsTW|c`PI&|+FxU65jHQCT zLzECyrj04a~ju-gkO)z`1u8x78+6`D96hl}(_!EE0$p zG5Be**u}SJLJN}vZKZCHTsG-``(W~8U~eI1Yk$V%*)8@E_6b$2JA*R^s&3fU4B$2Z zEgWW|#H=xot!)idhm2%DHBa9MNiNPZ!xHrvRN$4#<8Eqp&fIW2^7#p|p^YNTq9#){ zXxX9td5fsLe)Y+t)3|+^ZAav0Zg8#ZnJQGe&M8GbGR56A+C=9fTm0qeUm+)&*gKG^ zs$4Mw{{__L6frO^yS6Pjc`_d^ubMa# zi=4K_#caKzW&lwE3QUbRmTco7XRE!U@@r_axAWxN^ESp0h(2nH?OdS^aPBzxMt;6O zhEd~XB+(Ul*O8?{qK5(((AF6t+iM(js2;>Sb{rC@(_PTxw9ka2+SJhbwCnxSnfl_d z^%c5B%iFqZGZwbk@&h$9CtqjO=HE#Ir z08`+@ z)IzPjKJ4A2)$G%RCNLV*IDQWM-om6KMPgYh?3(Cx@Xe}xpM}kgW*yA|)J>P3MQId1X)A(^*WTm6__{P^o?{;6x87v18 zR!2*^iW#BFn=QSHzfmXgOSik9F2jbuaWUFD*e}jp=I10zSisc`3*b!Q@%5A13kV>V zf)Oj6YRc{dCoipI0d5Si<2lb+Dk?N(t-<;2wO)OCS<_Xs3bOv6Z3$$}Y};1g;AKX^ z7BqG4KpXpW^fR_(w@QvoG`n%yCI5z_j~twaH6>;CuLh@L-bo4ZU8QBb-K$2m;&E!< z2H44p!pp{HQMje2^Hq6rVmX9Y*m)wl_^Ep|&N^GO>~pf!ms0;Rjz62w-JXj)yyVay zH&(wCZ}}Mj%Nlxudu7IY*e!BobF@nfCR3{G)c0Iz=vtD3d_mCs&X3kp0?Lzx^iA#3 z7x9;}E^vrKts8svnC^z(W_D`+L1fWUCJO5D%7+wK z6K?8ewgrjf9bYCJAF&o?RU*PSEA15z=C1`mu4>#kOZA@ed|B?Y3O?geCD-3P9wl14 z@*UVBd<^v1eDFaX<*6mfm5wxi-a6E8yuyE{<2y8IB!)B$K14jkF0;ILrrS|Tj7T@C zSM2ZAH)@>!_@~YhIbr3?v4YiBX2fD^0_YOnV5O(;XUgWC*MP=vcTgJ^^bhskr@@o}X zXYFeu2uTr>9D;ai`M=nd^OIi9S%^g^2fYZOoZ1om9;UR#U+{3lKmZt)B?}H$D;{{T z-LPTp%mDe%EUoUk-Z7pdEntDT9nSZo4U(%L?H>EQrZ==M;GoqgDJMD)@MT1<-uV5A z?CE{UYQ}~`x6{C-(@Q>|)qv}wX&xsW-EbgPYm%ANt!TscGff8>ay#N#RLp=QDn?!O zW~nmbSA=Dp?DbQ;a$mb%c?p;F3l`x5eZz9cYg>!}-GCkQpUBuYdW1m_N2cFhIb$Ks;ru}H5iO!mZ0z}C?2eg%tBj{()IyU`nc|NT0^!i{-RS6( zR0?oPsAtsDb&G6bu}vCY1jBM4`CK;zrJdpz^AuXsPsD{Z8Dt-Mt;-oRe5H-)MJtPj z7~0NQlN*-}d5tsJk3m^mo=<|g0e`EsYWR|Mtzrb_^o0tG#YA`}$aK4WCNttnKUrPd z_|n0Th+GWB0&PW}>K_a1K`Q&!8XRv#M`@5Gby>cs5*S_Hb|aP6OArhrqn*nkrcrz` zP(Fx#JtYGP1UQn<5GBi;Zn_&iO@@bZA@Y3G@3!0;4E((I@kAAh&FP(GD9R9{=PB|9 z!h>2*r;DRJb7M{iyq~@>JMq&c2f0M{JneCsTK)`enSYUT+N&?SH;U>$KU9E0E9VRzf!&K+eD<-$zi+#KxG%i$GzJ&swaFBg}X8Q8< z>zPBs#P$O5UEH&LPj}HqKZ+k{K)xv~!^hV9@t0H` zvD1_e*%jHPxRP?p7UP_p0ORZ|L#8SAqc;f6c)SW!m-wHp_PPPU)r%2aEBXlBXXLE) z25#W*QN+GEM2@31F%*AOc+6Xs=o|`rW4`!>YM4cEYQpV^Sxuje_vUV!|AXcd*qY%^ z+3U{~p}F4NTTl$9G}2xTTm3oNN_Oi8Dd4-D$$W;Ea`g22UbJdKAyauUg@6?ce{O;L zYUhRR#|eppuJ4x-*b?2k$8!X*>KjMdfU2@zFI_CFTis%V{-ijLESXmzA7fT1-z*Vd z3H=KA)PLzy?tvf+Of>&Ek*_T(exYf)y8DvNVn|Vdcp(cJv}6Z1`1~rLz&x@-`#2fZ z2_nNIjS#%{;omeT;ou?pj}0`_BGLr6${|&T(6PeYk+WSm$9=KE5TyE7W>^;?8(XV#VhatJ-4Z55CVz|kIkRl9+rQ^&`bugO>86L;x*l7eq*U*ZqSn|oqVO}@Zm=2*Ww|Ic$t zXQK&IhPv^FePG?|=t_|M_0gt37qTD7iVsr#pf&6VBpVvUS>!&ee^_@f0sPgc+OcezTDj!BA#})=) zoGA%26=_je6hkb$@}B0Uw*!@AZRyCC&Ymjb5B}eqBNF#78m{-hqX@iG7oQQ=0ayKr z#p7qNQdpMxdnTpqTc=zU zmnJuPoE);gKrD(zrF|&%x(r+HY6b8pO9t{=g{m?-YN~ljXhXYjV7=Mbjth|fjm{RuAfzj@E&ZQ>IKlj*kYna zve2uh$8`Vul&!m&kOsH@`uB~JpF5j{*NnzO>)AhtUWH-7$1XK(2c%+$IIq=FohW}k zUMC?L(WNJ1@}uoSC)}Yqq2z>6j^R2SYET{U5PXf52{GFZD2C}Ah3pWfzDB2TbT)Pl zc-7kM>(J*y0NAjA%2fyc<`}{`aPa;d8$V>)jV?H7Q)NC45^w zKd%~d4B!QXt-)My)Q{ZJ(-AkgW0Z2+QQ`De9N znJ0}?mqpAKil1$9vfPFjq>5~aH)gCS9Os*;>E#Ez;w&DU3`4#(jEyTQ1vHZO@P5P~ z#r|)jA9JwdA}aQtj-hJcI?w^E_Iac1CFrnM$S!rNWhU&RfAx%TlnO!x@!Q^Lg(OG+ zz%!0^t>Rq!QAnTmDt0=|rkf{@gYR7MN74{BGlK8-hg`X6i)r!GsXI>)Hh zO1zl^QQn$yYT|AlGK@0L(&iLYP`=6l!C6ToX~6YT$M38o);FQ($J8@$EG$`MhPrmX$Nz00H`Y5ra-=wr!K!)|1?nQX7YIt z@$z*7Uzm!ZffBW;=_{n1N-@06-;ByX@2mTCPhGs6H+0B7zUqEbwGEH&1~mDXknhF^ zVqCo7mcN_X)-NBkKtd2WFZbspb@5RZaR-pr|7ZT{T$usSzPTJYMl}Vt3eY23t)wYJ zyNj8NZjc@$eg9EH#*K`{VQF zI9Beb0>TXb#zp`DA{ny-7mOvl1 zA?03xto~90irX`_x+z9`kSW%=Z6>O)XLIm(#RHiF`S)o7SjUMR(#^=$Zq7cv%1)BA#u zdMv#vD7o~b7}a4{LbVbp_m%MQ{juq?>9ZNoD96-8ZCF_p?Z9Mh$=vz}!@Q>0#y-yL z9nfjlBHrArp00EG7od9|-4j1N;_nRPz3p~7j4efW;~ z0&P;$DiiC7MK`{y=!n=?X1v@oK0u*QuwTl%M$!QlPxzsmBkAC#ZlMt^+6GM2oQkqW;;4`#Ha+URM}}?i3K!nx=y{N%=^ZZYXb;=LhhtqneBJvDYM@E$eqJzdF4X zWH=lg&u_)e-X;J#WKZ!gq^f}&$OOp@xls!RZ_jl=GTaJaWd%p#vbjCiSZFXeOv1B1 zQ4pq?N_N-r}i?!;A{E*{BS95#~~Tg6+DY*5u9m(O{^isLQ{~D2+Cw zcjWJlYfpoi6M4^OXysC}M;;6!?mz$N?tD!^5J6LecbC|REeVR-MInGpK%VX?8=u*t z*hb#CDa=eOQV3Ag3aPHCfvqY|u=N{0GJ@rM|0HZ)wUP%>p$}oLt&=;=RABsZxW35yL!HgW$~yevwI+my--6%kOf%Vb zVg%aWX=2Zq!6)Q&Y#5xhz}mHEM>EKPTw>^)=|}Jm_!I2+1s5~>2kU+3|>d`gBswgOWzaUNCUH8cq@CX>H#|O5|7;G62Aay4lPE^AFy$j+&936uLET@^w zOm;h{F|pON_;W}JI~I~)VLRIn=hF=)TC`Z7AT-n?rf%$RR6e+KG}b~Yl@i@qeglW7 z{gKd2EVBouZYvE6WsqJdFpX4t|8Jk) zQt*c{*&*HY8q=k0gs$TA>A)chRq$enj7v|(YYSv(_qY1CW1bxO0=adEwAf$WTrvra z4Yq$1f#($7o$qHne-AT#!?hctVF2aci_RbvNMZ;6lT9S6W1@`mrO8mcjQi##lBJm_ z4}SygM0H*M=7;L&<{Ls6o_~1N zlY8n)I-%15taimx%JAr?cIdWM9nete$TIYK8MtXxE?}pS^$9JC5~!$Z~ze$?j01HXDO=ch;A4E|lhN zcrs3({E)e3kmNG)fy0L~3*5d5@DHN9;aEusm%pvF> zr@ExSWDS*5R{EpxUdkfPC9=d-Fe2nUSbd_BfyJ4e?vhlCtm7xye@siH z)^v&h4aAqLA|47D{;J% zvmQ-%??I{)EbOp~@V*hk`44jUe&azo#1s(wRTC8}^Xh{JYBHZa%FB?h^|ujOFkGt0 z%C{`@C=`2>0$Y8DUxAX-ZPlJZxADllvl!}6Oq7OELoDJ?O5eok*%V6UPYfEuqC{(0 zE?*;h^l%2ac7gnGQt5D@YZf>9uEQRRk~ODV;Bd1-9EFU=@B#at^}R_RL>bYJXb|zm zD_2}7Ozl2U1uuV-N?XngDsw2}@f2?CA!-RzVIrAM z8~%IFF^L;@8Mn@pXV9*B#+vVr_Cdb{VwDkfiZY=_)9ZvD+l^FZkcmfAH{S*iI&`91 zB_Qb4!HI-{aIm}%{EV|{3Oy0gT0(H_V+h+Uis~%BMysWME}b8Q=|MYXQGC&gZXeKv zaisM+dZpkmt>P!{rq~oj>6T^>frD&3uMRs!#tDQ69=R%Kmr429{dpj?C7f*jrJWX! z`qgfZHZNg9%)(c>aF?{;E2w6LWnb8y_O>C_%x14HAz+{=X2KwHmh@2`oFcL5?E8PwsHas%$ zm+Og9J>uRTw?v_OZQ0Azf!)T?alZ%uD+}cH#|0CKaNx@z5zq0JgQln?bmlsm=IPLb zto=G61<%&FDAZi|LCUBURKx5#>-?F2~t?CN8r6B#C^`_M`p&Lb)q#E?6T_3 zwtUZ8nx@sN+(@S++$N8lTQvf{S?4x04Si?6v*@hpQHA-UEuOwxAKKzvHtsB-apL`l z;vS1Wi+a+E>6HYd6BL!@Iym9@DO&{rq6_ik~Y?Br9vGnDK{W$=$KeJC82qf>e;BAs_FqsYn#7Wq; zWk-Gj;AYshd%237{fxd_WnNP(rz-3c1J!L=-8VA@3G5Wy^h&mYFg=$<-l-_Gd6L=% z3tiQw(h7a@HG(_V&z>~}PcsQS>x%}GYhB5%RpT2(EB-M1#PyX=2WJ;ldc8aX^+zdB zv#qExIb*vkVi)szXFZM!uNBUDukd_{9@7l%uc%;L)SHw`lsu73Fo@$-7)!qK1P)JT z3tJaS5x5P6hiv?oA}3;v1?N~T2T?I3aBVHPMFkZ$l&`?vNzc1ae8|qZis)S4q)7~V zwgl85Hcg5t*acie}efO1J6Dm3nf)cF~@B$|-%4iuUaK{I)Y6CIO4qBe854JR^20}48k3Nu6&C*Kh!y`U4($D&Bise7uvD~J%i@*yA7l-R zx&z$XM)0kEE82DTP9lmN2&PQlv-ws5cf=fH)L^k09-`_7@A%)vVv%6h#xVK+=w!;R zbpZ5-zIfgg8d&3UJ!fAp#X+g0+yBbi^F))X6`etwqEFb;uY+h}1a@`%yj8tgT5S2E z(S4sH`>W6S+nju-{QSs>f`l+Vqb1$9P#Z*G_>~S4mwS2_UBaIxS|3g$B5GW@qwghE zsNHI0&Z~tFOgUO0&A@CL_>*bWb&yq}5^2lmqTYI-qE?0Oj>`SHVXn)4m;n8;!}nKx zk90VVl&rA!9u`&Nyy}$n!nj3=FmMUiBL$qIx0z>*hHGu%0%+7{^O%B-x)N*~3@6@l zVC(I!xkI9xHm)ROzgGLSAp;Bas+2zvVjE}ZzdsfrMy69k0WXl$cJN<>SS>BMl^u&v zyHn-6RAFViO1;1!4~g`&_*cS-YFojlYTc!dvfx&@`9H*&&F2}TS}efJJoH}m%Q8JK zMCGUZXnmnQMy8jE;UIGczH=Y7KVvwB}xz!O`S2kSXK) zPK=dBR;=9p*_f411FqL`m%+o;K1oYl39IBT5OMiGeM#}~xjBlTDZh#&{j6B&->Yi7 zq^T;i)3v&!w{n2(T|lrg3*reIjff&8BSX83NxdLkWc70JC%#%JpYsKcwb;bSr^K#& zVUUs1f_-D1Gw_zWHYHk31F_nxxHa>aXEC^|{HTD=KDK~9rye~bNujy7l#+Ngv(5x)tn~~V}ujBxUR}0xhc$; zL^q<8DCPM(%t|g!sss-CduMyONB>d|cHBo}gqT;w9Rxo6S}33l|CVqwOP#yCT1^JEZlh^a>5)u*-0YOk_ewE$s;-+G>F4x6rpmE_;q!>o|m zo^%1D&f4IlBA@v<5rNX)WQ=r8C*e)l5h{26vw||Syj3t;GZ~ZH0yIRpg{=i=!EmvF zkih1p%}M>8ngg4-`YKj&x%Aq)piC4`g5YazxX)F@7RGHo|JwRnADsG#!3w#l65EvpwY2vsKxx3|h{Rka!ypqzw%kBE^RTpvkWx5!#_F3*4(?&cZ85AjBUP!s~WpuSF3<_loijW}&n} zX`sGZdv$|8$&rZybIf=C=~=estNC@(0qh4}0Jhc)^_;~suh1m)#xruD409~b?@+(x zS&2#uy;lSekmxrGfrOmwbAkX4R9KjPs;ZtO z&>hC!-4bC{j|BQ&eMQX;>NT@@oUfDwd>KT@_?xfh;jaS-dEmFb$he>_#8ez$W&m-O zdzuW9f9!DvwqaT$DDaRG#9&sVbf*RDDT1k_zFmyj{uH5tfF_huFBmU0HTcUGU&~g> z$QLTQ5&t&FI4EGUzK*kQiHX+Fmzy>6@DKw+lK zlWC#nSLd}=HkC?Vut*gkLr*JT&+7FS`2$4XXnaYZzM6IDmAa51LC2W+AYNB~2mVd=`>I_~b_1Y2kmHhHV z!<>NlWKF$A4S!SM0{n@E=yUuzjLHblYdTUZxoBR{E&t0-W--jPg=Uqw>l1&)G zb-GP>Ym@X-KPBSd2+J5)_7Z}{?GY=8gJ>!TkPi(N_*2!f%`KQp^{?$VX#hkiJ6-b^UZiOkyCy|6NN zSMkN=hLlzJ^FrLMWn4k$cLH*Gi$Z>njtUmU0hqpnJ?G2f4LQV-#SM$gGbyy?d044S z4~W%}mv;2#IJeY`&Yt573t`m^j+(vfmb3f4I2MDCEUm6uA1wI1NBmO!H;tpQz*vvf zohIphF8?29QKY^=@tH2bPWKK1kJuw47wI!l1Q#}cTfnB$+5dSLT7CQJ=RwjiqR?c0 z{D)n!2hBCSPI2ANBDO|+ykoAf!Ky1Z_LJ!wxnI|%ZGHcJ-8+?`44<~vG7-24cK&3w zwBq_aiw*Zfi-<+3;T1RbDFPvBEiL>c(g(5Obg{`!wRPp6vsuJD25fcP6Rb1BwNT%7 zIU_XI@7Ylga4eR{MzSDTwn5_j;(yzMO3m>+Sj`dZaNXqj%D zB_x_alOg9wCeuSBhy@B~JgQE(`<27wpg{bLg|Y;In_QE z1>M8*7~gFFADfKCrezcPskdnbC;A~+;yVg4n??w<(-JG6lY(4)we<>T+EM{yXaYLe zx+A~~`%@JSsfXv2s3NLCT%;QwC=<6C2DuB$NKv^%2w4h!ovh@b1grS zpY@p6p`q1D+SN;=Ue!8ky^tA;6zJ&+I5q#iPstzPAiNvY*^OhMn`qJ4r9n49sJgHa zVR(9Cyge>C6up{;&4STSH5fcfw{$0TfBg=wL4@Fu;|;^ifB5pkw-X(YU&c|zbP+F0=Fl1nV8II`3im7y-0^++a@m=2FJi- zV|K|s@CijQ+~hUs|I0j!nD$wy4`}eI{+V&k#r5XI!vQLoT>9!iROG70-1-(Uw=;j% z$djICh%z~^r{ktVV$&x0f+GY0REi)g*?@FcdNF@U-Q;IILZ!>>@7DBe?vGoT%x^#GEZ! zY^5tmrn2SEctvPj3beLEJZxsi-bigsh4t`%dyObA`K6<-qb&uB4Y>2|^LZWm>}n+; z5RY-VY!DR@9&@@c9JS)qo$&jcNK21Z4RFz=ad^ox-D~BoDymOqcZ_&epW(>o!}=S) zNMHlDy1b%&q9CPgc&yVbhaVN+ef$&JSZiW*Suol>x#rzwCA69leN2(kp;+smBtiK0 zra#b@-8dT&O0CLVy2+M-%zHC73a@FVi4{hZ{#)eX`+$pGM<=087RAS*sq@WpK6>NR zw*ib1C&DZjH#K2HZOO3SA&S7qQVngBFQCUV-UdpPhT15+f?!xhxbu@No7@f`jGR{) zlu1oH?0`}!IlQMoOt3f9SjoeQjN^^&1)+)M(3~;IMBv=XY1MDJK!Z)?;a6%Tujnf= zQ9jrfJV2JB>*wC&@trGCo*LyHU5ps!=4tu2YB9hfLWcx2WRsXjRpemHLHKpmRbw~T zoapLy@K>}%0E95%6O0MoX8k5YxZFUtk$@|Cl|pbDXbGdKg@FVKGJHqM^!09K1anry z*}uYsU@MC2L!B+c`fLAk2H)Q^~$K_Rx#D+}`XrA}}<%!aV$@#o^{?|R?9ZG@?W^}OhRh#Yzl03(n z+M!g&=UK)gbWI%9i~i((Rj~4M;p{SXr2Z1qUY0wH#-}Lq)Rht`;+tz)%%kBE}~&9=WS|~bA}}Y zXwvvr4Pq^qQ!UGUWnM#g02}1d_r=M@%a+u@=E?<+{F`dRS`a5Cyzjqd?SBbhh$554 z$M|7`W^W!YiXsD&X#UJ`LYCtJ3V&(2Z!<5P^So-ZiD;X&)3$}3#vQP1%g*zLQ{e#? zuV-^xV`dwh^bqp~29h(>UbWo2#ABCFolBg3By_LL3`?H24?-Hphgd@&N&ix>gapTcRL- zwAW;k``+m@Dwvk0ZvZ*V6~MTss>4Cskj2fuhiC;?7PT$mg{nwT8@`5kbec2*3iqVH zllY(EKl!1cQpOW^hv)Zi*xuLGUZIDxD@v5yFs66|s$a12=saywfpP-`vcA->&hzMF zUj-2&@cI={zQD{5iJ8qFzFtA3e`M<-7XpkyMKlA+Vb{dR)CgZ?AU#!LHVAb67V#pR z`Vp(^#0f{s#4ILmCSM9ctI!Bbe`hx%_`$poU4_zr?%4_ z`Y?I;{0>`oYlLVAIR&?eRq??gykBS39$(Anm$Fd&D9enw(34wDSf-kQgRAo@mrn9H z6)0WZq4wK|Gd^z#m_z1;8rT=DuYMo65rQv1X65U|Q=!En@&!ad2JJENL26UV=FpZS zEkdZy(`7)^4N$wH6pN376gM(hGq4ecN7eHrDGwbvKA0Fot(Tn-lzj$apBsh9k~xIt zoFoiA-048Wb%-)sJWuNC7!i)eXVl)FJV|Y-7&qZx+CNoVzn-0Xs<+5AZr)Ui_Rm3J zKk*h0I;E7A`PnC6`Z`8TwhU zVjB;}V8kdVTL4uiLiA&(Q^}uPO0;(03mI6R9Tbj~wt)dSB=K-OBY+&eIs2nZciF2> zT5>BXRAWHy(NfWn0h^6abrl0mcB-sA68xsb?;*zrP8z_+zUzpzMfWO^V!sr)WiBpJwfm(HNq{b&~? zqk`?Jn4~pX%Fcd=TrxlgffC2>L786=&&+n(_EJBGe`6z zas&**Z!Isq+EzJgItE$PmC7-=dDI!$kU&Q=B%pXo%3)YT?<&Eo_QA+hsh0`LD33DI zjpY;Ne0@@x=r0a8Hqf3^;W9L~u{h}4JHf7n3iBF^YDh(rtAj4V*|fZ=?cx(kHur-5 z_IHSb=6Zo(={+-55M%(PchLB>w}4wVYkyDJaA(qLjXpbG6hIL^XI`@rP zDhhzVf;xng34>%Ho$!3*_(Ll}N|7wX9r=#_LyG1g^LXNh?5lv{S2P%ls0TyR9wjFU z7a!A_0~LrkRwONyP)8Qb2D9nVHQX`6i)l$qHIR8dcMa^t!-7$*#w~tt$3W6ahmV)X zqQ!?xTrjyJCYQP?DVkov9h%-_G{VE$5Vsm_1}B@8oNK%#XF)7jhG{SEHZi9MW}Q|y zyj+&VXs8DEF!<{oyKskZHxAp#yyvuc| z4M#&gP!W8`4h?n#;wbg@6^{P~Ssuxk)#64?F0N$0bf13+?{`X9hpvj{B?)ykZC&)O z;>~Sr(L~1)(&>_%a5|7mZ#IcAslE@khuRa*lB-aseekuD)_q;Q6oe&Gzp_GLc?FCV zkTj*B;UGTCXGF8`3=%tfT+*KaSXnG3z0Fjj&5!us$p~-8#L8lgP4b<_T;)a3svu#} zb~$_!&RIb9EIBINOD_M;Rv8lT*^etNvnIgcz#?<4@ZE7@I}XMRUO%SDyTXLONGbHh z*WQ+9k>FL_^8x@{UO!?5ypyk;1=6#@^T^(wS#)O!Zon2kq@VQgH+!=7@JPT~mvF(E zn;!9{;nNo3Y%l2whJ(R_jyl^d|H^?B|MBoKXN&oOhR6{P>)t^ zbN}zV9-!(h$8Z^Om<;{^ML@d0>HtX4q1IoYFiYn=kR7q^desLb9O~e z<@Jysl<{xA=4NJP<1Hys+O1k(pn-J@M%(!i7%3k?@VR}F!}S~z18VF?`xtac z`}fpn0}<-~wS|7SDd_VTU-yu#N;Be7)H^vmjUo&Q$8viFkZ9<&tnVvfWf9r%i2^`r z`#-)|5BA`GjOByc#G7p(IEX*$&MTX)h!31I8-!z&8!XIu3b_x=UQ9s~`dy^aYOJNWdk5B<$;yW!<@^r3&vjoR&oss(t*|Ms*fldvsex2?h8IyS)}skiJ-{ zSQ%QwSJ9hKg7CM?mnh6I{1TsPl!cgVFfvj1;nU_NN~@%}sn67I+h8q$ECKH(HFrK` z*?&GGT!3N)aLy9k%bfa|1-Crc?E0ac_20_2CtDxgkJ~9PM2aN zh=d;=Lnr3mWc)_sgn1mXz)3JyRUoM9=BQ2O&y|r=judS@lqksEv&9Z67$7DGX>O?9 z_~nL#?=ZhKPFEvmRcE5$qlG6DS#||AznSk+?O0LpzHvQJXHS;_3N7Nx9URNh!y6&~Q1{SAM#6=6U+Zb`TO zhJU=Blk^T!CpdU!W{X9Q%*y^BiCNJrKi8LgHM#*Sy1%g~C!l2BD>ayclc4=iW@o)RBZF~YLozOK-{n+}35_wV}8M_u&?b+FT4 zGiY#OR9V(J>lPM%mZLttflDGnR%|GupG~GLX$T!P!$>FXP`%mT<|mz#Aq^Ie=ZM61 z&4b+f0jZfcRfxIcDf-XEA0vs8|G;{l$1J#^yYD-~KAjf+WhDG?56|9^L`?)D&_1dy z)GL2l(xqHVZUr>filH$ko`gW5Z(eQSI;80y5Ggy`k7~cC*1)d|14uIBl(I~gjqU;Z ze3r3C@k#v^X;zSKUY#_{NTUdHPNH>>_M$ACjK|}ITF#JJcoQaFj~bY)?*h_Hdcs`; z@8@GF1(I7V@G@9Vu3(;7MCDQU( zW-N1D( z)Lf^`L;AXh6um``<^8#GGdLi7C*|LAROqrza^y_eNX3J?@fG}H4Y3fLKK`v0dBl5Eco(EZ=* z5^4Jj^rVrHHBT+Vk)%BygboJPFOM+#!e@%zHOPU(ct8vbK>&ETP&s4p1};KSf%}R& z`W{=98-%Q+Xph1jk#x~-;y>(sap^C`7<2dobl;3$w)3dRQ$L4M34Y7is@3=!(eD(E zKNPwBkwLf$b(d)u^6E6I>=H-{Rq{z2Z(H{*bpB)}Sj25P_!sCHruF(A`u2m>LPL$^ zZu>qQ_fXfqPujZd?R{N) zucZ@RkMgtDeJ zExl{A`Xgug761-+w?QyY-&$n-q6)(G9AG5jeIdR|A407vnh>m4F=>DX-_OoLwuyAl z8>7gcYPti;c^@O-9&|7s1M}f7R^kC}mi^1aC(;%#+`Dxj7)?Zz`1&IoGY2Qw5Il#N z`~$D*e$2_xwXUR~grT|%i_jc}7n)Vn9^RtMj-h#mP~AJfmK1wr!R3g@HHQ{0Ex_5S z*~li22vT!nhqIMGN?^)9rDpF_6A3n~gQ9P0X`WHCVWyZk++ZPixpS@Ts=4tn$IQ*4 z`bFoH*&4iDS#RQK=D|m$iMAtqWJ~G)P1ZQzB)-~tUQl{IFF68Ln5i;o)NwzDRfV8_ zR15p%1t^vL7hd)0Eisv;M0H^#bl74j=$n|3P@5x4Tz3|Yz!0-O;A z>``0sFr4olk`!k+Gosb|{oo`5X{YkmMBmNwBnBM-|WuDdVGgXxwiO?~Ddtzyqm1KpgTEJ>enO0(e*nb2<$0 zs6~Z%5Av@CiZMJWzN=Z&$Q@uVLFiZP8V4FMZ>Oq#7SO$bPjWFe64K2gd~+HyRc-;( z13F-n@9fik7^=yh>Mt5tgmaJ@f#(}2PasfS--gYByK6Xvq#n%Set#Y^u=yY%W3POS zIapU7deg$ zfnXBlyIu1IS5XZ zPB7G*B8fM7a3}$%IBs|F8_@3i$UF; z8Jid@9is{f-?Q~hyQ@^*GQX*^R{PN(vLX52ZsI@%+sl(Uk-0|~PX>uVFuvl0yY3X; z1$xlf50C)V@YBL^Nr*39?<5Acnjuh&V98OHzQCh-`*_|)2EW>lHhfqZ5dlQ;SsJD6 zp`8z?dKx!%(nUnG6bZ*n0K$RkM1qh!WcVgr8MmR?`eV=<5;4xDo+TSN2=&HSQE(Rq zU^S4my(5#@e9b9{BPgs`e^g}uRI%}jyzMtTbQi2BgZB>w`QJ=akeJwt>j(!Yd;Z@` zk|T&rl52Y|@(1%|a_ffltI7pH9ST=^bEfIs!92oSNCd(#IkHHPKLu;6_9;&)-lE!v zlXXh~U@!!7>kJSp$I$U{d?Lmv`oJD2e0aNQqw? zCwB@jBpjk%!dljE;bJO&@RwYmvh|p2cZyq>eQS)ii{o15=$FiURN5CRh6$#wYd>CV z*l_rk?f@hluOD8rGo|hCAZVsYggJ*DZQWsFoi@dT@=VvFxp&G2Xf z(VRSeq_j)*(>WF7?1@sQn47T|YK|-!9qFN4$KWCr@nKOoh^pwk_3VBnO=LepVLGxo zgObt@y^*>&N*T(-7si$M>ycOziOw*lS-PAkV$Jqx<5~bUXFlUpf#U;*Uf*Oc+4><}Bn0gWlR*@_Itaghn6GIt^ma z$YWoPnNM(4xK|~~qm9ST1UlFC{@N%~CFS9N@mRbOz8vT6R4^kR<-@!AmwjhbOYbX< zy{UM|Kd*b|i$&tT*)-}Eh+};gNE@-4vK-zMhnBDd;FSEQOTd#{^xnNzXnt7S$^7<< zhbWjR{?F&7{0lPty1N$_fRCc5D6ym32=!9G(G@)u$h~}oBBtUhIyy^O2E&`~TmpMI z=hGsLjcIgs5CjFBZetrgLAw|)E8?MR)Rf>jr2Eysp};HX6v%k#4X{$y-QHAs31V*j zWlNZug7bLOj)m8&@eukxF@eVx*KYglU6Re=B19ga$Plj$8abCfa^1Ya z_8NdQO<*6c?RAd@jMKsB*Kg}%!&ab4MQ!X@= zwM_jamRzOmqlP<#e4E^e+-MdJI z)21#|mzS(lL*@JbEa6c<`hXm2^x+Fj<})1bK4r&eNhvU=?toy;ML`$p3Zimyz&QW> zj??pssfK4XM~Jx|t;#pS@r}906{4wlmegaDN+70Ep9pm-wVoi*z}W*kvv>u~*|Mr7*W$~jp~K-zJHrbm z6y6!w>RycV0%b2g-T@?9`DB3yLwr&fqxUlF0pNC9iG%mA$iF);XcW~vM5?sb znfvkJ8{+6%6b65m)>OI#hi3=>w_e_l9ZG-vAJXjJdLtujuzy9ahCD;>Fp$(#t@yAg zLjs|NtNA8?guGhgVCx&DNS>!Nh=q?_0EavJ0-Za$#GSIC_UJDsy_>;k*K7-x7K4H6 zXSCQIf(QHk{gvKYT(F+rJyNL!uz6xOUoG0d zWZL(im1x9`m!nW#*IqnjiV7ss+8fpft?0N}%6z$>mg1r2`*6wF4U8_HJr6`e?5T$x z!_~W6V`m4Tcl8P-Tui>!Kv3q|nQ=l`i21z_jc??GYo146f*#8D(JkSi)bYNvmN-q- z;o=);O^6b`3lPb(EF9h{8+~qRcFV+4MV)G*i@J7OvY47CBCRYucRgi&H9O3M_=S9r z1=#v+`eEvXVUv!L!(w8OpM@m$qJydLen$vQr=ceXf6C$Qhp3;EZ4aYBuj43aXzzoz~Ipc zm$8C@fLdQmc~9cnXKmjh;U@`QhXO^?gd%9mU0+v9`r#eG*-ruT!MYPn9lpEE1Q?c$ zwg9^5pUJe8zTXYyoKBzZj6!lh`D?VS{u_jQrxZ^T|B7MvHf1QXIAtEH%eb` zpknOqsj76>4Sd)46vf{g-Clz@?wU(v-WogYT`$lRgW^ z7J^e{>P}S+MU*h|p*RiByIl1@+SJi;Z2_hF&04cizm!XTUyH=Bd*)wg$qYPP-H<^m zD4ZNe~(>D|Wxe*$szfhzqZ(FA` zs1lAAU`6k8moN$!yR#bD{A^E(+0=v}7ruj@w{{DA*W!+YaAEgi)uNBdl4X@}Luz5iA%S85d3QenPBAa| zSnm8@gVc_cJdR?q#JM$HVMXJyTZ$RMl?uG9HvTh*%F}4kp(hlQax@D(%Ri7ru415z z(E6%1Tj0gVc`EraWu9N@+#%_>V9D;ESb-uBL9kcQn@>@A+s+?JzycKmS99g2W`YMx zSCdqL7#^TENnAzHZ~nA9ef+rskBGNZpez^oDHtKL>7Z3TTEY!3kW^Es z_XLD$fHuOWF$c4+^jesY!RY>P2Ic2oiHG@N5zQF%UWo|gvnE2vLms`^M4!g*<_RgE z&D6H4|F>yPkckG~0D7=z&M=7;ZPEZLmc}I3kOrBDXGLNkRL-JIS-6^ zvXpZGk0$QF!rhnN|7gGC@l_}?`V2=X2)O(tU8{Xv9RYZYb)o|MDgm#0#9)Tb|2*ks z-D6yXOeW>|gQb}~W9q~1J0#K~tR1C&hD=d`_$9D&RFeLQys7VCKh`^RA*!Q9pI7ao$r0cR*0`lhozv!n%9#B zvX~%G#+hOGHBA9e`Yzr(#X^l-52fyX6L%<^oBSe8SSe&1dz&B#C~G-kTpqqY zqwL|S1VNT1E0dT`Tw#AOBM8l|ChD!uH4U1ajFCta`Cu9>&8=bn9M5NxE5_nWGu#RJ zQ$_MA`pD=LhC+|(`7p@>X-AZS9-phKj#pF_z#f}N4wOYEx*3V9Sp(53u`ZLU zr5JBz$@^^2YZAF{=CU&9>BVNBz@*1ej2&pWe4q2K2eQ+kXKc4&qMYzfg3GFV3zDeJ z8r+92cHBH?zAnv?>6fvOiBw{|Al6R$A8+S4Tl}!f$6p;jp9+1y{i)_0#aaq_<=%RM zxN8i%uqVDXyW#}n?H1eMqKKBLxr-T>= zQ{JQxV7r-B>_tDtAeR?4E#8tl@`VOweHa&=QD0+|(=AW%#|fAnsFC~iTU~jx(M?Ya zdimtJ;C9Hyn|;5p`-~xvD!zR)qE|P`PGbcM4@LjjN_BZvEX%`CM@=l`KNvJ!cNq8~ zIo7^P#I2&?X>!MyAo#^E^$0q%7%dFUMyrzmhLD${D;MoCh_f}n?{5_ZkTZf8^Sm)2 z*ZjlKs?!Rb|E&-k=o6TXd#ylNEk=IXFZs@n{IZr0T!H-dIBl3jSCK*+xm!#SEm9*i zXpB2@4i6z6)|%G#{Du&8)<~c74Dg5uvZX<6_Yy9YhWwL1^__Kf!u#IC&nP!OdcpHZ z6Sv{lkVDC?F2cj_c;b=#u38tpbbr1g8R6C9OY0?tg*eBDK#e$cUm6lbEj?W6MK)RE z4YEYL$b*xd;!^k`Z2{VXCfdLRCUG6U6-JYie|jxJMFv|(tkowl2u}5o%e7Ne|uv7er+*s?w(Mk!BWavqvxzi2*&4doMDZAc5}2{&%$zG!b51Qbo}}jL&{u{8|2(pz5L{IToKH*CJ#;#+%%q_O)?17@uiBNEix&%&S7U( z@$<bWr%u5nqUhfI$p9pX+WO(gDEgxOLh^2s9TG^khX%FhXJLtW6>c>w@AOvYGM(|azTy#po>dr>f#LzS61`l{ zg^%n-AB`qr|M0cMD%gW+8)26cha)bNK2fn%i?7Pzs0xRB2P9tMnbXyhOa6reU&d{w zq7d8HO2mCWn>PLmAMwD{4fH=RA{0?oc}TG1r(hJuvVq`gFk~T3iSOeTPPn!}F3k{; zjG3}n_?8ZaGlO6(YE(|mBvrJyh$;+Y>|K(Odwiw=Mss)ovIc~ zs+~aXLnm&eeF>i*`Mc)9`Gb@B@xhx>)6k*&Jb~_-aK7=r(;c=PNCaUrekZ=ji%s5j zRB)bK1>ic$Q!#2O&6waFac1JGH%jNrLaRJE;=xZvJP2w$m|F$eQK%Y95gISHlJft^ zD@4{C9mE=&_%*_VKwJokZwPknR`gjI0MK*yk(7in4$Eo!yz8n*dJ(=-hbk4mt7Cqw z%e}oxpNJ-=#7Ea~`udV($os6@p8x?oo}I1B4)~FCayX^;TX6w$Yj%R<)xxWApuL{5 ziN8ayk-U9Nohq;air`59jlq8uNNGKdTfp6-hZVgyAqlywcY1j1a&hSHN6qE}5Xsq_ z!);Cy<1$c##y8ha5y9YlYW zI{)rRmp1)dyYb*CRFQWQdY}QlJ*{2*O~p&r_x1z{(iDjCOXW-hl7?V76EG?fSswN$ z9d5i+l=52&h0-tAuaB8J^&!UVh$FPcj#|BD@92T9nJ@OZ8%@pxIbuxCki^;u(|O}8 zN&{bDD_2woUixP;f9jl(7`+06&#XV**$(Xc#m026j%$fG#}TYAZOBlkvE7J%_P*I} z1U=}49(yE1Qm_*^+?vQJv3pn$R4_yyWaI}H4K&`-cvuuKy5v4i`gSEtO5cUHhgE*u zth5mxez!HsEm6m`Hy^v8!hpW>Nu%rdu3m3PNI5kw_wiUjwaXx z5l%4h5a$v!z>o&%>&IXiTt^mjfZZ!s{1ych7QudiXy*USWp2wAM|dBu=^httdaD%X zjdvjWr)D?!}7_~p@HL_S@1C2%gE=^Nw`w!gI=4_yOHcmeG0ZKtaBJNfO{|My=sqopV(Jb z!u4LLnx!JfZpR)xsr@p_qkm6;&EU^DJhq#(Vb%Rj)NZ2(L4nz=Pqr)iEy*F_S$Ha(2eY(y#J@i-`ts6*yCUGv6tZR)CV7Ksml7oGd}#wG^hs4 z?MZy3PI4}Oxe9Hs(q9B)`K9DGhoqc5I$%=pyq`-}<7AJz&u{f4yl)&;ePTLNBrftB z=ypytV-~tBqn;^P2?<(jOtHBTf!CUJvv!b|EjsS{9!fkv^CwXoLvQROr?ZzSKbEe+ zc;~?^KKlY)S@k&wejpZy zGHR=p;*PYXjkac85SyKu0YJ^2E@g|1{k?mcR2`~LF3=OYYfG?LI&}-ug^_-30*+)M z()p%k@C`N3u3$gC5>hxKBW$#sg`ZbMU}i;0^V6%OvnabwspJ0XoHJ-*3f^~$ZH8nE zZ~LWYB<>l(`%tUEaiTe=1UL{R;EV!5+!s+5ug*odCRrv%6Z7!pUf z7c$Q?I!@dMDy^VuQ_ffx))zSpp<%g&2$!^32<=eeGlsE4as>MXLNRN9rVVPzeKZny z_%^}bTUF$-iV*Af5J|F+8d`N+P_QTHL4ixGy7z?0|Z_bj#W~bt&n_J%uVZ{G1`#7JsD!3098lKP+1x>2;;bD5kX|&pZ#~A>FyJE zso-EQ&Ee(fM&}uvdkt{f)P}BW+O(fy`LLg{(#O`LN4~?~2fO*c|7pfby7l~x?9CUp z7W(y1!39@x;tLz;zD-n|Z1EF?z?05&u?H!drU-5kDRcXvkU0UpW%WV*)*$8JbcWZV zc0VsmzMh8&(x}wrf=H;_+xQ+DZgG;A>*ob0(Rg}-8Nv}eKW|6~9~i7&_Dm3|03E{S zODkQ=L4oJ8LGm^ez*K^xD&QUym}eDynfT7CYMs=`Ub8ct48QG=zmwe5_5#J)y-3do zPrxkeRq5!q-6Lw~#l%m7y?0_0Oo^o8Ypc4Y7r}|@XEQsPgMrUuhpTm3M$ssio2F^( z`eBAnvZrMS@P&^(W0^;n4&WJSh;?T0*V>3E(C)-OVN zic;ka0*|cT7(-lxqsJzSWfJ^4AP}uUD#{mi!cTPfu7;2Pzm8TFcqb;JIuzjsZ-5q{ zII$icawng{Rixo~Y?27yCR(EYc#VD3;k&ejt+Z|2xN7WN{~*(J_lgjJauyFBhWR>A zI0ri`E|IsN)zddSbBk9;*v{#QYr>;R3kGQ8OUw%988e&KQ2w8?m2F;87WDb*-glJa zz2FQqjLdcH31KU=Igz{`Qaqq3RI;^jCcj;k2z*jbb3KRj8=rbY_n{WY4dG^W1bF-X zY`2udwdBY;E5x0{XIc()in7M2c%^~?LNH#1_d4Ax2QA^)ZN{Eoo|u1cY200C;)uW8-ik!X@I~{AXZ@kGu0TuQ3H#0 zA4xcIolC`603ldP2d+tLHXK;O3dy>CP6^=FIyPyBl|)Idm9zm6j^9OPj}7R2vdKN7 z+0bemE!J`YuF398Qjvgs5?#hPlR}TA8rZ2aG7@W1!cK-{hi?lHkPouic8LwibHKHX z7{(q{%*>J$jzTe*ns~2x&|vZNN0k<)AP&san0061wp!`ORrY4-mD}J2GU!JPGn(bq zBVL$^GFvgEeyf;dDgRymZZ0u){49iL(sb{VGYbkFk2dVI#S~Maawj|e28B^AFmk1< zu*oREHJOd!pySRn$J}mkh7DG$V$jyhYu;;mZ-u{o^Mo^HOS+=YO5`x3;MFHBq=)t3&n4I~wtt7)Lt#T$(> zQ1Gn87T^zZ(Uc*7<>bUIdtDGH_kY$qX9JEOL&NH$Zd0y%YNzTpP4J8g{7Ge9ZoJbf zFcDB^K; z4ftV`r`^Hzjx8tj@;XVUyteN%=tw{{DtxUrO6-kAXsIC36kuOl;Z7dtNN zQ65(m;`XP?FaASh8H~RZ8i9%nAH$|qj8*nx<Wdtz$|^zcI_xo%2#S z!QLsFWwNVbsHc32T5zw-%!lmd*Y}Te{1V9x?W0UGI=c6zpke#!#dH2jTuCYFf^`D2 zA~UQwn?XoK=Qz=SgTPrU$({cKdryd)I9A3=t{0LJXWinfn@qK`$7M9HaV3vL zS|%y9Q_a*BIY%!DmqwTV3ADWk5C=G{S!(BFZwKac3nrXRy2WbKvGh0YKu3OrOTfFt z&9p^CM{EQMC?c09T5m#1k8z0*?GLm7&%W4p6i7QvU^Rx5vyC1^Ox=sCQqrfKy`yo- zG5X4wBTPZy4(#$SB|m&xOhfq_p7TqH7Eh`VMd8o5ES5E5r>1O`@wgaX^nldQmZH4X z4Pfy&*5Oj$TU-ROcc*5rqE|+~z)W&g3&)kawzWSZqwoOp$x_A1B?d}mV!OiqtK8Kt zVhp9Oa9T5wxZ%kojAIsO_)mq>c*-y$W9}&M*(#W(%ktOEA}q50`fWvSx!l!UxkXyD zRDOMojla_(w3Ty?jx!~-FD=R*tZU8R)lQd%9nt!~+E{E2lQJ_dw%Y#ccHx(vRyj_g z66%Cnnc}Zc%JaQ=n2HJP(Rz5n%D_1=g)lFJUFPW0Kq^6;LmUL#9|3O;@fRC_jc{4% z<=|>hOaqf+9&ndJg-?ED7zE||QcFh$zEhBr>7|{aUR$TocFDWNnH!3e{4AxkuntWq z{2YuQz{0C|r?lc|KGA+GbH-k8DQf0<%upqWfwr%^CYpcyWns!p!wTW>!s}mB*+Vb0 zD;Os;sM-WNjeeow)voxdG|QqI^Us9Hr_)U+GiAn~}mq5!h0ZLz)GuO12>DuuM-WHO8ZylCS!I77`# z;msV}>RS^rDO=I)t0XW*Nv}(PE|`M+5{dwQm+-pj!zwfQpM_#J8F>pliV0yPpAN7d z5qYR#MX42Zn4yu$YNz9EUB1=FP9Z7V^tXsn1xhba8ZW31-1Bfz83GMZfb#1A7YXX& zuubfV7trsr3>YWpucPB_#I} z`}_UD56l7o-*IOY7Hk4gq_Lrza$Z?qOH&0zL0a$)(qLP57Ev!GgqYCqbS1=ZeKYFL z)Pv&GiCVKk*aUvPmT}Iz=u;|oEP(|Ba_GR+OCFMv=US$*Ie{Obv>ty(NQ6UmZUSV3 z2`?rWJ?2=^#Q9HSkVNg&?I(j<Qzng31Q@riJk@vh*uUu(cd z6t1?A1_!!e*kzP_;Bs|o%?IQpf63l^gDBD*L#`%g3h^b!@qEB02z1aBm>>8MA@ZO% zTt;*?yyAJ_as-Xh-*X(IgFN(O5}(3`Co5}^%K+tYaf~j{*3BbLX$V>I%tPJ-8PT(A zbi^+KS+03swh{yUx_wV1xn>CG=7xy_ZjIzB7iqwyN0qz0){B2Tg9dMaS%B8WK3w&2 zj+&Nk&kHxfv`RJhkYM_UbD7Wf8a?v-vV^6&Hoqn%+ zXg&=??^7>&BJTLUAeTaMQC9|7HO;ieSj|y{09T^$TIy`Zk>_F52b46AgHtXue2E~@q-EqCPCZbUPHg~uG&A7nV|MsCKa0$yS9 zvZf5%xy6IT$^(rEbK|XMa{x{+YK5YAlZ=)G&CbX8%hxPA7krrRGkB1(z2#ips zh~l-3U3Q)n{?uwW)FQ(zsM9J|pv?gs5NXgKNW@lkLT6<;`tepNN8xqZQ|da7{D=GD z#zDJQLf3T_S$~|jPN+g4MZ3-vvLKTOFm!D_Z=o9TSWO=%8;@|l?>vM>K=ETRp1`(a zO_>oOtjM-3x!V^(=9N7oo(AI=%p1@51`rax@T581*n0|>`H2qssMi&(*E}Og#uvRJ zbCSfmPzbVR&FC_1S)_WF_aEgWLqruH%%h=4z%%i?IeWPs4;@QXH0aAAF&16P(^ogJ z9n{F|L>)Yu8c403g<^@2{E+$fZP0lc*@N8%9q&wP0`F*AJ`8{+3gS%V=ISvF0jI0< zIPcpH0s@)W);**kNw*H=_K{YsDziD9SP(3Nh)$O)V;VvoKd1gtLx}H(7dQA3?IG{^+&p8W8UrH0d-1_8)9? zD4Sq*#u|X`jWWs7@tFNkS;$_{M z`{l1ww||ki=0P}!ot&;E5eGMs{!7$D41XkIQ$!P|35-YVvluei>psK@`PbsN03CfD zTPN>GP*-fQNqG??&>6hpCLT#4hoc?D?KwHf&PLpXx)@9Z)-oU8ifInz;54tq&-^=> zpiy!$$zyE)Rg^vI%q{}E8nnK{ksd2~u(}Blyx!v}x-0#rpep;wMo0W08DaSR*O4gi zLcg}1xs`}(!7W72gdc5>jYH=(`+x1yJ@IEx^ z&PZr%n352rBu_f$Q^1={j6m*tPF0pPMy51uCTOmA6HU6%ipUw;2&#>1DrDOt~4+NO0% z$rNAJ5VT*b>+kwM;`1k&Q_Q^uAit*QB-ZafbG+fQpuXP$fsse7cZH#Vfg>4ZO0WKYblEoi z`?j>zd-W4<0IFXX6{=b+8rxr2s=F55iSnJvm2go0CrC!VSgU@73_)no`)0&#Z!Df~ zW#~s!82uRD)1&MZe;-{7AgHP888_5dY^_rL^c{m^;CZbgc;kC5fz!opHLRZwr9ZjO z1plqPQ}J_$jV}osbN;M)(zNWB#5R0s=O*M?QPnp2HhUFA_zXT!)jO4SieWKARH1jj zM9Tk-)X(F%o`JvrLLYb)FtbZ6f3+yX2>fAT{@+*Nu(P=fF@Z=v-UAc*zl)C?!Objh2Ht_a zuOEgst=Lv1GE(;UOao4RZM0FSx(_VDTLOx6^7`H6k-0}gX(ziS>fmU1kxGdZG!Jh^ zG81Cb18LYm1uk#h-6hnj_*ef-LZjvhTxZno@!Hss=KYef*ZmiLKP5?{)oRQ9kqy^D ze&&mW;oT8l9>~1V6F4j;jm8jbb~TwlsYb}0oYjLq=Fu;@_rf1^eSxYV^~{7d7i_#d z*bKE=@In~UcsuF1kB-zK6nSVw`h*LFJqK>*36`##$Q(pqm`uM!>YVdf#46?m=`Cwv zl)=%G5_%ihIU}JaSnxw)f!ab>db!IwaBOOLIbWL=wouy=x?|0L?1^LDz!T|~+h#5B zH}&bIZqqj;3i6vz{J@?sjZH|TwB0{T3Or<1zbjGL!ly*GjP|8DT9tKQH7R=Q0}X)t zZ%_0qToBtjK8~gF%Qn6DA@3eAn!F$ULif0~eR9n^1`*DP-1cjnube3DV+I=~q&0Yc z&-&lHL0!rBgoCjtveSD?8!0M@11-M3CtvW}y4mhY>f5v1R0c@6E@jBU2%)9JSgEj; zBxn!QGG6Y*pO*Ny0+v^Nlf9{<>fd<$5F-Zj9}~jynUEHQ`H;*sp44hdSfpEzlANax z4d7gIL8_zqaiw?#9gjH?z5p++Z149#g6W_b5va@aVyTd}^oe!0DfD|y;2s$^9Iy9< zBR`)mH*+tZ!FZw)#G4>5YpCX+d8L`un!?n#xG!*|IT+zhD(Sp?J-Pytbwt*!Qr6c0 zD;VzQ9%wKadu4iUR!wR;0x`>^Ug1CV6&|>-`Ey}4RiuKEt17Vv7 zfx9B%EQ46}*XK}ygj0f)>-Va-u3mrh`&wF-Y{|W*NnQ%*I99CS7$m0LsuAN1?xK?49(KPSn)_rgSuNPD^#y7OAx#cM>MoHZ+&%cr<~ z8|*-intPk~Pf5WG)8nmcBIAm=zoMmVe9<5RaKv@nSLATZV@BL{$Hl`f6v>H@{sXe>mDJZH9Pd~b)9EaA^J8e?#}RK=&Y4hk>_Ny zWpe~xJ?-~kmA6if0lqDd%^{@Bb%AP&s}A`2hl;E8Cf$rRxd2k!3Rt9MaLM5NHkl#< zh4{BLjEee#RK=1f2clafxJgq8>Txt#{uCln(cOXiQqW&5kK4;+-<;$QHbT9S5%=hN zP82<3+$d`BikYLRA3GUZGBr;(aS^3+hg3iMTq*>=6zX-^Ktn?Ge$-Wyc`}}Kmgch| zAY&n=`0@R|%iI!ygv%!}JnN=gLmA-o%YM6$GBMS({u;k`^$^i+k^Jp+TKQ@iVIr4# zs(8lyuJF2#ed<;Yj^W5-P|S_ZfBkt|3l`vdn7*=3lst-@_AArR^ayQoW&vLGv~Mst zy+t4LrrHjTlf0d2wbW!`PkYLWH zJ5luDoabi)uR4da44BtT?;mcLv_}G~pkMLK74gu}sM#@NRm4iCE99FnQIdng7H6F{Wh`amEtH46yWN7#tF0_wf?;-?bo1qgUo=T84dL|{lyNF50w zfL zh$`N8mVp)5+abK1`|$l|W|K4(FdhZL^jK!XqEF7r%z(^QV4)b$IGn%1DxGkY(pJ2+ zlaa4xD9UnjI4VwPDPwK`UBy;BtnHRoc$P{N>sHFqIcJm=9W}3PE0!lkVs~kBe_||x z4u5WnWNqag(-xtQ*+UF?M5&F_7EjC6>Y6pQ*vU8yj~{I?6>@LF$#tc-Z1=WSqM4>1NcPg+?sKVmoPRp4pZB(Zy_` z!Ou{Uj_b?dKmc;**IDVNe!KF%`5a6qnw}MI6ZS~O^Q2`xA7k1?5Y&(CAmxLmF;?rxDBJ@wE4nE|0 z_u}uEEj3gRKaTiEpm=^{eME}~ zf-~CJf53^f>P7vvhNiQZTh;kv)}K>@NehPGfz^Z${JHeEb`Nd*3)TgR%%%=MZ*oWa zA(PrNLa=&R!K6qAjw%CVs&fnxmqKX%1MY|#Bt(j-B$^EER5eOA{h3@j>by{G>q_j3 z@5a8T9E>N(2P9Fp5;$<4;bDT^(kLhPJL;{_k#EId9=Wt64|5r3ws#+=0R=&Rlu;uX=Ie{hK)L{aUEf9&600rOs#BmJn0 zuaZ%d=1@Lsp;Q-EleKZmyt}BgHfd5PF`^V*J@^f}=HU9QI*VJsa{yI9s=wqs_c2Ww zg&y^L2s|b)zGIfm=Uulf$csy(4ks48DeKy_&|JH9&C0+|y$`BgNE)rKBOpjS&_l~? zl%o|}$M$?XXjvNQ)yC7|^6XERlHJZcBh<$PV0bF=R|baj=_L?N3p)T%xk;sgV9#rL-9E3__&rM*m^$X4+a9UNK6aCh1fXql zP#8Op*Zi9PM$>a{8mtirKUT{%j>Kx?4r?iHJ0^s8Q@JS_k!6xge0tx(wG2jT{l^Jh1ngjC>42P$8Uj zY>NDFpMUzSP2!D&fK%qiQ;7R8Se=1GQ+wCO&bG*n!8v3Dh8b8PO1f6>$bOyqa$Yr} z7u&}dV<00bzXWW2y2h8bQ&b?E^!dfyTntrT+Vg1-nT+l8+XEX<)hU#iy2?JRED_ye zehDX^rPw@Jyr*qnR4HQvdgvY5Dqu434IYj^Ck7>fh<#ZNE0i=o3s9_u9e8Kz&tsCw ztP&r|Gxf?Q{0Xli&SV#yDY2=O6&{n7nUJ;_M*+C^@XL+X*nODsGx}E8N~zbDrHDVW z%SNGP(RH&=PB9j|!lXr%ad5C)kRgrQ!sNv(nz*TZL>h9SdoL->wf(QjIaD!zpMOHw zIK-A=JI{-&)|Dvbuz`;TS>Ml6!2@>4N=JCsbxetlq2Psyw8W9cuyyxVYkRw6>tvWH z_v%|SQA!L!Ir}i|c)T1sY#gcQwipZa_PleR{ND@>sxF3EeJ`$NpJH8+WWMhordUS@ zFAWI=Z9?dIhgL0ZJ~V`AEG5xM;Un@Ee;v*0n;3{`ZR=ADT^05lMvt#CaRx_tMZ^Hd zAY!V%+mVl>^){4B7*9?q>im~cQeUE7;JVYgr@B3=C+d;XB|r*FTXE(CUb*Gf3asvi zLm@ymaTrpU2U!oJ3zDXG2qfNN5!aZ%_Ub0ToR^G2r3w=_-(}SKQ3Np59-Wt3jo9{u zE@RHm_mJQI7+ZR=GD#0D884+CJge9S)OH+%KSv|peU}YI36@~1rw}TSj?<#p#Uqb! zZfjlab{7q#x#-r9DXGxB!A+|%k! z#^JpX=?F7&*^#`YQJny(i!5bMRpwzMzVdpfx(k#Sq&u$tcBEOR; zZGh(w=24w>LeVGuH=TY04Tcr*u?qE}0Yl$4b;L(p zxw*t)1NYLHi4c7MEz;$47rkiWT5ar>D*q;C*}lZ67?l|VQ`fmvSlhBvL^I-18hfsP zF#@v_18y%8E#|7Y46rM7VH~hJsV|S!QSfm%9EzUT>r)*A#yKZoh_;lpyoJ1hPq62k zs%3dS0cWzzA^79nrWRg6f~^iH9Mu@NFWQL3GNLc}#yzw&4vJ(_Q2-C5t``6dU}iA^ zTzPaUWhCdfqu8|Yi-syq=+WymbK@;UAy{dvclnVcIFhsZISd;15DdIS z{deu@*JHJm^jZMmbUgRQhoKpBsQrEIt4-1!5h%~n=+b&^^5l2#sGYeODfzt9$~5IZ z_o=)tHtt2KlBj;tjK>FDSB9l#)teqbNfR>fy-Y^}(u-6*M&Xfr)Cr-pyrF8&b3R~*Z$LSg6hM9ZfT39ambe&ml*yW4 zy2z!s0~5)dxG)OW>dVKucqie#h+s>R)fAvXuDbq89wCvz4c@#SR5)yColX)Jbc|Dg zFGxR{KWbAvjYG8ALEWHFC%qnfbk&W3ZF%^fS_YhH$6w{NH7(u3j30hAp)}p7;|L6?al6Ag$cym(Rd6IB_4pU@^?X_9QyyrjNbX>ei-w(!KGj zb%bhu^$pkk;5Cto`Z<9^i}yB*%AxR3GD`y{w>@lT^L5?_exywv^7SO%%yuHajfd<@ zM#ejiWv^VdxPV;^0uec#ZdNMTL*kt?=)(xJu+Xhjo|m{|*qhlXgX_2;fz2^@D-AD} zKuisAp$DY4zAxhlwgSffbYoty8XvxE`lo<;U{=}Anja49;j3Png64}A1v07X@$Cco zS2Eo#%=&HE%6d1q3lmX&FZy&8R~h=JWirTolKru3Zo~}0o;_cNo#>VMa#7OW!1y-I zg;K1$D-^ZuF>J4zXR5FaazW{pQ=;xPKyVF! zBnGEtf)bOAuY5M{`@?==47ZOiHFZg#9fWunVOyf)?$WTh*5NN-vKHb3@i(D9lrN6D z>Bq@Pz}fEQMlx#VB-9paifL6nQxS6OV&XpsKsiZAeo53Zs!kG z{@&_0h-0}GM?XMN5>;f6Pq7aLtS7SMj~yt>YM1+W1yY8@#6;>73VgMc14@$Bzaw{!)eD@Qh^&|3)WIjKaA5i*e5SaRj+Q&hl7ROB9-IJo9lgl=D zvkP7V(@!;9#VGS2k=!{PW=pPd%Ix=~Wz5f!)^S;A2yF6tfHh8MdhS2TIje9fJmc-9 zvVbfkS7HB--VS6GRxOE14K}Li7(mpd5gZ(ANnya@wbk?kjzNclT8t0Fv_$c!XIo+&G%dWE~4uK5Qw~itDkE21=AGow>melTr9fGnRkb4{P}#!`r!t zz?+H1*b$Y&)D1X=XVdHKMdG{dtX_w`{=g_^;Zgum=a z7gV+L?c>sX z%uND>DK_~ScX{j_6E)GK>T`tFY9DS$&rV2gcJQvJN!FCuDY>NjX~U{d577LU0GQ@a zGv1p+S=?uzueorlUO|(AAo;&u<>YW{H${BINkBq-sa`rR3Qv!cO6_J7WN;GdkXemC z_y`Yx2H31c7n_*Z&)(iTx4IqwP7U(#n#-YKVLiS^$6>c<o{o(5_%m5I4Y8))TVH2QoBv9)an%@K2hJS-H|KBc*&GsN&HlAsx;B zeX-0481v)hC>|}<4d7EFL)p!yFmNs6ZO*TDzl_ZtsHZ%wN5@_m+|{`TMsdpBsZ_#zgm zKuurXGW)?^!?0;IcYgnSazy*B$DV^n=Ob}D&0OeaJkoDpR(q`+li`=!iUky1#eU*R z5wX${enyNnpqU&^Y}>t(g#jQ%;|z`dnS7uWF7O*U_~TWpozF7@;MPbC5~xU$iKOMl z>Q|l&6JUj?eqWO8mRX^<_BxUh)|cS$1lAf_+O!TDdLxL+(She`fViDLWx-$QM@G!X zh5Q^LL8(@yJvCs5$M!MC$T` zAu&Pr%B!C|U_HD$IJPuoYcG;-U{W{_rOKP-8sF7dJv|Io=lA145O+oaF)bS(WOg<( zW-A#XcO0*)jRxr1;wR1Z=t@Tx< z{o2i|W0&#D5tak7v>Ua2wND`2fEa(k2Mau?hlI1=ZjlzHNjhn}0bDk*K5}u`C7$ht z7v74L4~E|^q^M8|MPhvpoA32GfuRYqv}Wk?2kskE8c%c9Fc#?`lrEmNxdOlX>2%IA zPV}LxqVjbRd^_0z1^>d-z|C(}#C($a^5hVT6OJQgp1Cx0Yz+F>Ncu*eFmUCCMU)xu zao@~#Kmo~YGHPb7@bwa#Nf>$#dL1!F#zjAC>HPgkQ7l6p-NFq%7Nb?5MJMasJq^=- zA!#{F({tkPJooeSHl8kXzAG8H2v6gSaXVL2*L}~}9LU<>d510fK9yDh1Q62y@JGfd zcAP*~0uXAY>-+b_{;+xFX@zM)Oj`S#)m`mOR5nepQ)E+AxYw)jj!kd_^HU|yHo|7n zdJa8b-kZ{DLwF=`lM_ly-r|D4_~CUIy156^z$Q)`ex*lRv;$pM_+Ce)18`)GgYhlk zft*K#WeE_qVo7_@IlND1YxLYVlSE(X_xWI*2Hi@GdRpOGKI&cn&H8VkBpg)%2Fy)W zD`&c{_5+h=l&Sy>c7r&S4IeJ{nOPE&lPUKg>P7i6K_J85D8{s?bBQG>xP05t+-R_e zCsXK!oEGvOI>#j5MWaGR$7R}R1Hi}Y3Tsq|vpW%KQ2!mM98kCP;{uByjOr=T@7Zoz zgY@HBd2L!yl_8wKu%4cv5N<~~yO+TBICAkFRC}_kk*FX1LfJc9 zsz0;`xq-I0|IO;q-rt#7Xf;ZbEUT<9I57Z8s9IM;B?P%ubI02}FHBtC1<~jhUf#_N z1HRH{aaQKwF-iX`4vj~cdCNK&akXO3EKLtBFh09TmKmFH!1`-ufF*8fW% zXm|0GssUDlC;m9XLw#48A4HLW~>4i#|Hvm%?MHakC7T3jW#$1Ag zgGPQ9$^eh;E{EVKH~iBcsz;3y$0NXa@w`p=h6v6%2s+=jrKLDZ@-2I*PlPcD%*cBL z<(^~?1rK>Q$8aJ*IzyiAhz#X8IVSjnYC-i*^Y79+4IdrB2_35ECmyXGUz6U0r z@f%mM$8|JMW>$N_X2G+MOiE6iIYdZ_)gR;*X-JOg4Y|kFO{p6!DH#h=RyLR*A{b1;%7C6wjlI-E z!QUr)IuXlx{)~6V@~(t4-`uEr@n9G-vnNb1&Pm)x0~4E`PUlq4*!bK4$x)yS ztVq~ifiEu_kI!oBnTdJB!F{5=8Q{=uXbG7(>ZJAbFdzfUSZ5M}7VyKD&|r6FlOI7H zY=wa7Rc=z%YqiWH1KK?ibocSS5CdK&5v3|;!!n%AC>Mtb0vx!w@G~G84*%MyHyA** zxHgaYPFrLs1IIx%j4Vp`g{l*|Sg35qE@$=7;gV0!esl~9s9kr)AN|fti6oTd3!;FV zPWm}@;PGAb6iNpG9Dv}V-=b_o8IjxMrH@^p4FpqaPvlm5n%u-)PBU;rtB!Yr8&y}M-{(%xw{ z`UQDS|jqKBKrW-eF>>6$Y zBA7}m-5K#pOfYLqS7!(EYT~W$9u?rn;U1h`%O#C zW6fpJ>gB=h4hwD3bdidufNkvvK!-AjV}OB}HLSIJPgp2P#_BXh-gblTj*b ze2u&b+?RE<+i&RMl`-K^QWX3oI+*UiTr~=$_#jR8Y!GZJohh$d?(zNhxPc)Y0?#-9 zjxh59sxkc+3RVhfJ9gQE<)Bwnj^4WNImd+MZ(Xgk%g(oR%Cx%{jDSq}Isnt^hrLLC z<4HC`vQ_Dl#Q;S|wFDY;?y;sMkUiM=G3c#ptV6#ur)BS18XZr!gTIc)IK#2Lf>`{3 zf4LxFhXa!beP^xUGY*#BI zyf1RcFGC`0UZeMaf6(XPhC}kRSUjX9Y>ojqiK95&!bK*Q5<0yeX}*v?mW8g`^G9_$ zGV;E7ef0yYLjnQ0(|{iA;A?~z-ZFeUQr{`swX=Xa516-B(UJ%x<4iq)v4T)>#}NKp zjYHEU>ZEO5-48o>wyuPM#j!;Q*^#dP~2KMFewGB14l+2Sonmj%7vudq=5SdJs&eXrU0uIrN-oBd{6TNa zgLp^nAjS}wYSX$FRit`sw_g)8MTI!mv!`?1G=58hVw%gbE1~4Lvh?N@AVCtLr@y*I zAk~OJcbzSxeQkA7)ECLr1i@PZ(_Ds>l4qVgCaU$9jsmW&(sWjMkkDd{>LFV0vb`UrAWDafB6?1VBy*0b`%=IL^|g?7$rLfLt|cGe1|X z0c+NI44eCCdTy&S;$U8)75C(iNURKNWf%@B$&<%B5_Yl;R^N6N|F zMzv&e%#}gk!Jsk8U z$)Yzkkf&?~*TQHlXivgeSdSjok~>Ot#(-toUq-K4Rlo1twinEDvC;&6wMt?7@~M=X zE;GnG)tytJQdb)*He_@oZ&`viV%95K{3I5;0BVgN-Gh8} zVFR!j#tiK^t)X`A-auD#OA!Wrt;k?RtR01M_pbm7sLy_k0k)B8cJCd(7@rHZK!{3x?Mgc87X{?NfFc(FzTpbXS*ls&2o33#110Wa7QCEr z+5<(aqpY5GmwB>?_vxfp7fA6$-ChXiucSmyF2j-tK2c@?BB#@orMA(RZzG$zxFeJk zB)1dEFuY5Xccza_-gJg#jb{txFIX7p6o#c_lg)OfHn=@Y{!4UwKKYIPOzUS~4LHmegTWxE#&2 zSI>b%5-Eu&xLjolJJH&c76=rxw^;?Bak>_QT=@r06H7U071VT{KN$Zxy|h3TR^sUi zf;mR92^L@eXv^3E?|R(tq%u4^uCU`Hg+Ls^`$)g z`u%0m40XxI8M80ruHAwj@qBE=Yn2u{3s+lCbL3B-NUE`!qea5X6tZU(zg+_5Lb|M{ z*%2_w{Bx0dGC#%WV@@IPKz8pQt)OM1d`*$;i7tI{?<9>K{eajI69N_wKN%&imfi!= z0IfUfzK@ZTvlm)4{mbl2+QYJKwLQ11GiRsT?dr?F-r$NNbIG!u^gAr_1^ z#0Mim#(XX9CK%c7g^JDVC(c6hz4&HBz(LHvh8?4MxJh*|`L53>U7wTsj(nL(B##Ro ze621G!Iy+44%irh?vQo(Ji_}*m$v#|45XKd<+~(ku09HPJ#ZH8^ml^z%WkLX)NuzX z8Zx}WgGx=K6P96L`16WpMMCc|_scCL0VJLb^nHzs(nk)idKmjdo$&)KEr$Cwf60_0 zqTYCL`v|$osF;WiFfIf%_+X~yhdNU>?ZZi&Qf+OLMezP+?>TI4*B<#G_fVA*F0+SW!4qe%@|$J!>T_~1oQc(g0j)`j38Rw_PoAvB_8E;7?Btynn}QVL<%V5xy9V@P$A;j$c0l zlHmWJ%Hfil2l=5}KD3cFVx+&Sdk!o)P{>9F2eBZPX0g)}Z|)Ld#qwQ5fslvJ4A$e^ zSsg*=z(7*9RhQ@$hOTsX+^ys1N>%#!t;lAU-q!3W*C=7x_a*JkQDOfNBJ{ghmVPGz zC$7BsEiw*wNXuHeZ^6S&LV=4zBzE#IFglxH0mm4mYfMgR8tZzZg2}81mS)y>OjFpW zQG&87D=Ti#ZwRfLNOZbb+rjS}uN(h&uAKJI4zO7QcW+y^l!w}0q@)VUn)wfpkH`q6?)jm#YDkBUdz+R55mO%}?gFvFr$7*dsRYrL zxZN$juHeT*Q4o+~Gvj%VZJ(cC^Md%zRt9^ykm8}QU63yugl*bdiYkERS-RSavAXvf zh^y5NTKxFLPjl@64(@`o%|}hTc%51r0Xd`exW!RuZn=0+nC@btQv2tK`cM&Fiuh8TYyfKf?TsNK01#0RJsNlK zJt^-B!T^YAnOqccAKz= zCT`ApyP&gyZH$j=neQR*;w(x!8kk;=&W+Kh1ZZDgJix2Xse~ro12efqf%cIlOFATp znhsl^>lDg_2%&;Nm6;1R$fB@Da@@azRmwpreKB=QodF%gT?@sflkLwSw%kCjfX&>y zjhDS!y!nE?9cD`4Z%1QecFSQFnb~vCuQhx%aZ+dIhy1_5qW>HC${VmdyA;Gv+|c`> z2C$Q)E^C3?y`<9oStwt$ahHS%?CpJ~P9t;tsTLc3Y`~_q8a)q*rGw7Z+_bX@SkJ20 zK2~}rM<4WxZ58@BnZ<|YE1*462T;tZ%9I(bYS}7nq)TaER*!x6(;QuLQsv-TjTIni zj5*oA#bdx1*@RG$(vB1`gtGXRYGWw!qGJWcg-E15F}J>C5lr~*g-qBTfq0`mciXFL zqV3u+&_&}Zq%^_EH0f<3kMr$aoiaXVO0>rzI5U@Tq-skT-t~0%y`(Upsp4}K%C7^# zyRmb#T4H7>2%x0#;j_8r{F~Mu#nW_RZKkNAxRAeq7mm1MF)ob(Bk*@90FY}#8L^0{{{!MJjkp^A29<%o z8YAdvKlpTT6|Fm{oTJ>SkU*N<@=DvXx~)5ho=(WmuuJ@v=d@~m4k{HJ{BJ|^He{$u z4pN*T8V0=K&hC3tp6?{Opb3Mjp7AFW!f_98x_TNO>;X_86ABoOI;t%D&WD~yzzRHis zr#p3mJyVfU6}=TUH6~^U&S>d$ z)cP;E>=XTA9+_`@TGpO~4wP5c2}(sBFB3620R^e(49&BcqWkz2{g^bSc#L3lyAXyW z4}0-vBrYKJZ|1Lr*}|{|NSf|ex)K2v@i19%+n&ovIH%1)H|bCH!bqu1us#yO#b%z5 z^b{CqL?LQ=xuQ$?DFW#eFv5HZ^S?RJrU0zv*R2?tQ9oV%dRq9`@X%5P#;o8n2|5ct zVEoV3uY2)JPz(TR;gf&-aLaS@ux!e?BfHYXzlGq-JQAX8CcXD}WRR^kFNV+Lv}d=( zxX-z*#g;vq89%p80^QBc9nqOX0TInV@2&rz-*bhJth?P=7}|YKi?@>xK&Rizmeih( zbnE6}0bTZPK#wc;IS<(z4z}lMxzj(||1(*ph}%@1tk~|TwXOkLnNYRAN{oOiR`ZQ1 zV;6h&O|<>o`UR*L`yqg^49XkYvi}2d@fwN%m=2$Fsa1^f7$O zmLdz2kd5G%fz>r<9JE9vhAs?ou-|VV6@S6!Z-wpKu6u&L7Ku|SyoAv7Hq@s>@y<;* zk$?Lzl{61chMu8QUYwgja|-asZ#!cKTR4z%^&Pg6%yd$}8xkO4V z`rlj$KPy7A`zvadjQVOM>{_p#2}u}FhIVEC?R}iCAtP;5l}p11{Ubxc07HH9hFXD| zimsyQlvKBk*}}!~2UV{(CFTn4YIn zQIxmsa_ud>L;~jEi(3-exjH9i{8^#TZ(MjRJGOnD-mx2hx&qxSqbLdnEOygPwx+l4 zhd+ih?UB@t&=TD&D!%$L=iaQtOW2%T3&xal_%Oi`r~OKvic>gM%y5y}nB}l`dY@V9 z_aBsFfO@42*FLwmr-7Mmk6&{(brbmPR0HZmgR0wYcEmD#{zZM5of#OdqEI!XehFQf z9i%AvqBvo>Avn4%T(7bP4;q6+%E3APHO2R|(A^VT26(&RO@+EYYP_vN>SN4Q*w1*) z=KnpwSk2V!-I1UG&zpXXz>(6W+SJR$vSO_@Lox<^+)h7ambHi%7y=7J4HV=RL2g+# z<(e(q+*$wJE~j(zD2y1pwhcm@4wB2VJZUxfZ5GJn3$YDpl=7Q76R*a z)(_PsLMpdCoiuweDx(Rc#XXxd;tupgw!lR6*pR-EOu^=yNo1~2K{b&Aw;*vY3kX`e z{CXhlR&LIRO`^WY2NwevE}Jxo7#Iwi)#>L?`8@KP=R^Kz6Hn(vI_u*i*;*l++g^g8=Bw!g*q4$qYAs%0rQ>-uvMpY>$gmKb_bb~zTE3I{8M;JB zmNyXm(iuGjDS~cO=F(%S0#lZ_bjyZbn^wD^$=uCkpP?T2wjn46ICJdv+tRf+Pc@hm z*UPKwmt5lxl9#`O-|(40-h|}ZF;$gIy_b^Gdq32;?${>j%Yh&BbMg2zyg2GyXwJWH z7N0nPIa(iW_)&Y5IxH%IKMB zi_}Ujv1E_+8s}0tXKQ)I+j0A)ADm_qVnQ(xhP;>zsEPFKNFOrZ4&pE!BX%8r@OI{u zF}mYl!UOoA$8V>;c4x)s``uwtpg^1W@FaJcY{u}Pi>Q1h$G~p+Wl-CaCet=>T><}} z3|3H6d0C=MO>lQ#X$+U%$uwp)cMkSho>!iFo8RWEB;*U6T%NtwezwI20W#IfaAK|x zCH;s$`;ZmJqJD30lyyUKs4>dSLeRGIh?J4_-?qL5A41mBkxNu-RYrJaBF`UJ> zWG0Lm0Xj15pIx>!#9QvB``m)-4`qeR12&4=QoI&_uBLi*{bN0#$!+K_hlILtAO(^h3#auq_| zjV{cX#!D}xDYvnWW66ke_y49X61RJ9R~Vj}wJ3HP`PDMEy$*3iIdZU@f4ZD_3lCLf zORcJn9)YrpNr6L+d`W0*Y_J8yj{a9ffAcpCCHClr#>~1zfA~)ueJf3;uvE9R(KugN zD{=Y0qu_udKqJy_FM8J8yfkBko@cHQ3P--r;N~1Uf59=s@ow-Q>5w6D-)TIXh;*}a zg~BNgU7?z@+yMaOja3z_38U!b1WZnJfV&6*zf`%rbI-)caltw5l!1I$k{Zj;C0>S& z?!Ac%C{ZFqS@VHuB4-exKY)qFOtZRZHiOwaReaM~!c%Q^{v4%V`G;A+lP8K5bsKHA zQ)3vQ$qElCYQi+zGSuD=^Amow_^zbA?{yPaK>kFfS){dfE?I<;OuTSi@cT`n-I z6uw8!py??_?9Hm>q#paT62e-(#K%D<|8kGQuK#bLnmPP~azek(BPo(+!U-qZ7Q#%$iIA|*EAMOu z2z%llCPg>wJmUcMoCm>El?&!oU*Q)~@I|R+X4binbz%JP?NCDG(_&t#-*#E7mKShl z6hf%XaOdstzDH$Xz&3MFK*}(?U(Q7h#A>n?VP}0~`c^fAM?Rd3?$^(FY?~9bXbJ}; zVm)W%Vfu8PMk4CJva8S#`t%rQX7^J}Kzs`+UO6sX<%m81tm4%63uWSBcJcbbh4iS3 zDv;MXo057+lv`{ykhHm%V~?wWO>%iDy6O|B=Lhz7MIA8KY>*>6{TW?$ts1Aa%ea5_ z7j8G8OoBjAoYn2S)4RbhvCU<$s8FeJ2JfPz@c7H$)ph(8+|P$XN2eoaTZ7Fb0YYrA z#cOp-A+zTy&2L?KnD=2Oe{9P?%(V^4hqvl${LS*%ng|57Nq$9??vK{Sp6^YmbZIQ6 z$=nCP1zj@!W}0XJQq_{_j3?XnGcd-0G`-Tk%p1w^uD5mUsU1nFiViG*Zbe8?LNr&u z(QQ{C4OuIa%Yh7BN-KBev3VMrOo{XZb5;O1;b=k?@+Z!N@Z<}TDUDfIPDW@dKhMvS zIf^aa$NR9m%ia*O4+(5SsKqpTbte1N%MG8o?)QSdO;H0<{uLxuBs#;`$6-)XzGwlW zL%Z9Uk6vlzLH3x0?1C<8_^p{Q% zF(SG#ot0sK`6cH57KZWCJ zKJRV=z(z7Q9$`cbkR=V3-JqdQ)Y7CuB7-HEQw<3|AhBqz>_yYp4agX z{-`2%%WZ*v4sC0}0-qauo9Pnzp-st95x%5u(`=;t-m;w<=pu0io|PD-J}%UqZlZF= zsuF|#H?xui2QCof4q;H9p4sTBv2)8n>)J{03CuH;RFGCW)`dNB46Kfsob@BcBVW~c z8xVa(>Gj?F>Hg4npQzMyAxL2pjM6k4s13kj2GEz+i{vD#tlC_ z?CvvCg#9w&$y3wzGo=Cg5NnbFfv5&tNiVU&k&6<4tv#SZkn9*kBb&cxDrO)$m^|+* zW0X2e9Y!6+U`e$%rjWi!E{l~;IcL?l5nZ)equrAnx~l?JJ-#g)Z=|XX4jDVP%Hs@r z_Y27xVv?`i|Fs9AWp!Oa7uZM&>wx7Nl$J;ztX_dy#2XpREltYhnKfV|mHk=*tR<@Z z)sy+$^DEEw&~V6RI7zF1wVVMtWi9&d$P^ISr?Gn>^=Fqc(0qa(%)2n~Y|!wP=NDp- z2I|ZbgS2p=c&i7@=jsSlu}TgVs=HHyIS+=jK~Sk%ERJOXR`4sw0D~uXamN@;Kr@)J z1BH`J_F@M*F{$+0ejFt|MgVeFamEXJ6h4{VtA5KeOPUQq9_eFcrngoFDM!G?Nwys~ zJ#tTz8?#cT+UM_3Vgy`xFB21n7~oF6ND0NhC3EYU!}lF$KqJW!>mnI@U)e?wxH4SU zVC?qqa8cH%)%+$;&;Qm|S{@dk5}tj1nh&mqv0{usY_!bb41bhQis`Oa0~U3AA?qE2 z5N&I`;bQ)Q2au>Nll8gu@Yj=^tMLq=u8V`9w>CJI*H(VAK{*tfDbsV{cg+%+T;NP~W13zR7?dz0HAb|W%Hj7HPOAYuKryAA-WHH;=0i78NCYI@ zMSx|Hp3~}u&Nd2&<^hR}GSN(84~Xd)!KuS6wO463*Z}2bR9P=Nq?k!3KQORm?y+Xg8(D%FC)!;o@$VTQ~akvT^ z;f+{5#`Gz&cpg%f^$|3?7|5r=L+^)d--z8Sp$zyDoXla|E|gC6d|v_y0Q}6IgGOrj z-ckxhBi`lO83x2~QRrBw1sN52<2QSEJ{HjWf_NLJ)WKrw9{y?4BPRh)P&>HPa7O#& zsbF*%M;vLhGls-dh1>_$xD3@V<77I>$GiJ*@jDlO2SU?NUj&g^J$1c*>L}Z}6J1xY}5V)}Sg=UI9vo6jY=WYfp zG=#c${OK)GOyPm`hz7?+WW`mrv*?u1)2{hhQM`M1LoG*3-qDK#S2HA3_P2D#@;0MJDja;I5aZ$8`p0sY+QA7x;f|;q#9t z*@4F5*sHyZE2d@ER^;tT41!9$LBr5$2LDGN7jc4gArC6qs!%ya<0Pm@%kQVf&jh>% z!M+cU7m8eM*iujdJGk15Um8m!i8I#G)IvD0Jt zBZL0dG}<5BbZ1TmVjV{J$L8}gfm{N*2QVval&~{auEsT+9{rAZk&yd2Z48&!Bx}AD z(Vx6}A+=pWUAMmlYrU1)DrEW<+QQ7e(Hfkwy-ni`w=20nR}Q52*6IGdP)7a~lV6>$ zv97m|mH)xtlCA~gWo%n4xtqsO5PkGP6wCpaD|zb8H#ybpvmOC2xoT#Hg&g;EWi)DtLf1uOoBmul`x~-A6fmm|LO*<)f7AT|oaH87#&=)vgJVXJM2~k^$ z1e@s-ld1uuFq5bo4&2Sete}|ia!c-cxo-4y|J9=6)a`u`Z<8ro;I$(%j|rpgy8oQC#CSY(yUQopQ7(qyDHVaRiKi^~}@CWK8l2^w2&Q?Lsk z)D6+uI<}@Y05b7>Pr`FHK4#CRLZbqmh&oW-==Cv*<_OXCJ3;|3gm33-d2N$+K@dFOSG{mLdk-7DGO z&2#?gpe0PLoxv?S(MI-!acMWhW}-V%X}@57qU0P2Id6^-H9{+u#$pC_h>AW#=W_Z=G!q%6C;ka@LyzMQ8+3t75fK zOA99Qf3)upQiI=WZj^j!$=gFI8C4?|THDE*FQf&Ss|e~rZ30iH3*6^F_t0jrT7}h} z#?*kT)RXt~hn(J19mWgy)?z0NyX&DVC+YIC@}J+FTvwRW!(HK(z&JYG=c|zO|G3Pa z7euCu40B%BMSQT^rKq;XY~dT==sR%OAEG!d!|^wNEFvfyBhYD#9z%5oK3)-SSAlSVSw&I?}qx0t;nLC#6`6r76s7T-6iFvsdt{ zssR^wn+9<{`<)*OvipbZzS!K>-RA!>$?nEtrW^usM~S@p>K+^|QHeZb2m8hvuo%EC#g*lrx7m2IgG@Rqw_-ChClM3}o?@eEc9iT~Z{5SS-y$~6 z%uPqW_v^;Gb~SOfUqiCzeKM6(xe}yESYar|`hsTvs$h8YV04jQ4eIk?NK z6gO+Va9f&Agn_2pHDbH82%nh)72lFyJDI>Bln}DbX4!U^R-<&YKq$=-@PJC5IDnm` zS2ru{6+DHxjE~+_OyX|FN*BtLIhxRF3b`|9#7KiAk=c8b#0!l2p4mYNg2<>93e8kDs4yp4-XcP{s?3 zR%RPGz#_(r8w@Y+D=`n{z3vS4GYQ@(*d8bNS2^bjU%V{W`Ipjx`pq_CWc4Q)o?lY4 zXRmnJzCnf`df}?!?3fwQ@(JEwEk0M1zrvgyHC#E|`gvZGkazQJET{8F!JLLwXy_#1 ziT~{n8nrmXxkXBS_l4+^PaJ4_zk0z-7cXiO2>woiHRr?z+&L^!>~f#8hm%@Ca~mmg z`(&4u@N_oYgE2Q~ySo!i>YI@(VHsb|4%tIN#;~>9sI>D_%vD!E!^Iw^MS8nrMS88X z0?(GZ5htr%4lW2kn1^!`C0%iqpgPD>gVT3D!)APhUfhV??*l&;yAS`pcIAV_WB~ak zjC6#1_Y6b^cGH?3g#|wthAW1xIwY=1W4C$CUtPewJF%RrVs5T%PLU+WuVjET%twC5$S8!Orw`sS?liUp| z#)#Hyf5@7WR?lUz5%4Q_W=m8gw>ysU=L6>lJuQPsa;g{lXJ&L~N%}lkgDA=){K|!n zEI-5&AFwdI`p$Qe!KFqUBDme9Jbz1u9$4#A6|et(hiL?ch^nvM=q)OSk|y78ug*#e z&Z)zTKK=c(5Dkg{w~J{G;GaY9sDQQ*)7H=|A? z?jm+B{_n|+Dx}Jk;0%AG1ry1NoI}K<+bvGw^z;L@gQh3uI$R;-kRw#;67q0zQf8@4`s^JU|pLH zvNrVk@OWm6dySogmGfCBKWQu~OC*2TZFV5VF+u0{Z%;G=hDHo!aYA(LmE!kd*6c{wh&Cvp~LS&C<~KKJkXQ<%ZAn1?1=a1Wfd{cfLi*Em`(a}d#o zYIpGgwoP8I?DiL^i>z|0ue#J+ZSte|%<96N{XK6#R`Kl2gx@PZtwG7%l8FjO^={GM zA*bQpw0on?%<*_lN82n^=+-RLX6GnN*&!r)4#u+Z9sOI+Gol&cM@y z&jI^Bai1I0OdemlAGLeDxBmF`jH;zdJZ~r%6eY^tod?T3#B8!Br}>kl3Ylc9;){B} zsv4H!sXD;O{)wxu=2qL3dBqQ7eCl-Z4rvWNQihD<@!YmDeQ1VBR}|U zx^VJPl`{?OnmBpe`d4{&AEDPb8>Mz7U3-@3EyqbcK@~FO{da*_Bv$VQ!z<%!NXeHy zUB+?NI|X4c@i%6N$_8glTwe8;&O84tPNsd>jnvMfKCE+7`i#qXMXxO+;jTZ8*5CAT z_ix=;pOaz<)@E$om^N(bSZT!!8TR{TypYRR=E8Bdedt$2&&e5^phYH6M7FMISrbUNu;LgH%xdk}jL_`i-AZ;YEm(L~tx{a`8a zCp6b>9=0>io-98rsW8k($P&h+sXGSNIdtJb^XCP-e_xa1_stmLu2bYN@9nS3$M2%C zg|!0_3j?SRZuL(la%$UZ;?I&N7$v?8pyVH-u^g+|5$>U?IH%?04KR8tB)8w=|4ks` zVRyP%PQBU{G1v;5F4W;KMo4Rg%+cI)7!am9XaacT6$@U2*Th)=g|!(Sv1+*62k%|E z^*Cb(G2{)sp?eBY{zTwYqAd$-s2@6XrrDX>;n+eK^lXIsRL4RytC3^IH4R7^;aj{0 z_5f%{ACB-!i)$h}^I;X8aZ^s9qie!uA)2+IN~@Bm@0ALegH7uFtN=7Zx#BchCu|rx z-P0o(zxp5|M&S`0Jr#}av0R905-Cu!-BsiL)?pa!eyqoQ~kLtbys-{5!&tX;Y{n-k5t_G2#OO%HH}bnu@*8D z)m|h|y^7#V6?ddkwT zrPaNh4SowKX$9pTEZh%Ew>{9;emD4=lD&kWfLVs#@AVLbB}e<0mOFhtB?xzczqAIO z7XL~o1kQr4#Z}62LXU}l2SG@nWA(r*!jbPKF)O6PomsoI7JHPdENp#xBZ`Nnk?#aT z6?4#uR(!&mw2n%kqEyS;vMBS`K3_%$W@qr*T>7U-Y1v>r=CY&lF-iM)A%H+0iFM@# z0>u1uR1mDq69OY~2@?7k7$iD{zo|2HCBP69*BNt>FAUu?8!5gU`QOGH1EOLpYX!g( zCe0Z0@J+FIueQ%g(-^$Sl45Ru7CTg+i(RONcqb530>H@id0Iz=|3|W zbb}2!R-EJ=UF8#IL@T>XrDA&DMy}||R8n!?yjdBkR1lDH{N_GgIj2CSMMJU`>o-kB z5Gt>G_FW?3s($NmU&u)m4IiqHjkZ=s`M|NCgkH{C+~lJ|tlbRamIBSF;o6;)W{T1J zOxb8>$fQ$`{e8FPTEeyWtKkaG8h4#;RWC+nL}hCpsQUugh?$F_Iv|)?zx) zIlsF(VaVzj{co}V?{|-gGSnW-dS8uoDhdbfEXgJo*8-x*v?ToJd%J;ip6`AXmBQRu zNrjcAJmxG3td3ZB$v{4SVitKNc1cxTd#pTncF?cSGK@ex@AoSW!P}GUM%IQV-~74L z&UZLb*8b25v#RQu@eBAX!2=1~1GM&)kC$VNgD>o4xN4YO<7erR2hC5cmWAo2zDIwc zg4`D@WKt+{PhYTry_;8Mzh}lR`DgRUDnU*UwW#<)Hs0)NeJUpoGg?rh*u*fU{7|gm zwtR97@){J2n@FP=lB?P?7a%zh=M!57Vv1LHF#KYw?egfRwRi5=d{v@>^zSWjj)a(AM`xs$9x_Zf1P5Q^K%qEdrPo-o!$nkj15C zD;+dceN6B&dC>nuR{>SIkoPiyWGUSaTU@f=vDS4w09Vqgy?G*v_pJdIQRllCde%pW zD+1rpr)3{eXJ*0zw3AgF%8U@>49tn8o^>PCiw2q0F`4bQ zVz39f;CKhsqHkhicZSCDQUxM$iyF)?_bq2xV;dZiWNM=sQ9g4v1c->*c`aL56 z%=Ek5)MDu#d+2I-pvFEpPw*~IxjS>0I->ID)iVD@DG`Jvuk>d^UiMM}UKMbvSi;4j z05-H%W5@An_f?7XYs}5M=$`PDs&~o36oc?@#J^J` zUlc9J(myh+{au-xo0A|#g}1id!kFf|$ZuY5^hd3h5u)#T^CUd9!toG9+Ca88gmAz6 z_)JJcCVJ3yz$cLhA{3IEV0pogXjYvp_bW`)@5gf5NE23UfqU(4dWy|NftpAWt*l}U ze5XpA@Fym$Q|V)AiN_@qUNcS$(MhtvbE5d{4i9iQq|z!J6^fxqb0nOW3YWS>trKY$ zy+rQMwK+Urt^1cXQ;I<{*v}8jU6mat|I-Mo?n*Z*odN1yJJ0baX$6SKin;mBWeT%zw<^wOrUYTf4*vt{ zco98(^0wtwW9C3)|6u{sfSI_AxmOC!`eW>!S3l(O#JVAVbGOy@bjUmmqgd+HbM5Rs z?J}9=#Oru1H*Vb~;yuX@Nm~XK0sk%t&ZVd%>RpP_%+*iyn)41s4VXA}1G%pL2+Jb$ zFa9et>XHadQhm0hTmz`8_sHWkY#3c(qCvohTMSLH1RB~da+$a8bqN%f{I=K9&v7SfGwSP#>nYC^RNzpA(xv^ zfCEC-YSJ|a9k(`La)&O0C}yReQjib|3A8H~Q4iIDwnNnQdx1{%6T59zGKNk3t9F@S z&UT!Nq^4)nObS|;vW!mm4W%e?E*PcaXZp1Pf0NgQ=_MW~S`a!_qAiUfDLqZ! zMC!5^t0>-^KYv(4JtOJe8-BvZNMIjUp{W?(t*lT_)38xHWBo$_U|E2g_I*trp{9%7 zh?V91rli+#7O=-7px|Us<(t_qFc~D)^_~%j_E+YyLO$sffwaoS`kOyzrdwfJ| z=nrcNDk=Kc`D|TsCyKzxmWJ#(P<%j5M|~21qZotCm#bG$S$GFSwrZBUZv_B}3RmuT zDqDw0$dCV3p-eYe{|>=fA#^ zvz4u!>zIYOEc@W9h`ovs+vT~1@Z!WQ^dCjeD%LP)-qa`PdZzd8f&dI=niot@{0sfr z){y?QU=yaw)eIkiI?kuw3ta|q{eKx#q#r!N6?-yWyfPyHV{--1s00`N=3I}-O0jw_ zN>KigpF_oPyqomq?-UNSxE9aVOj5hK%V!u^q~ltwgD0EmM=<||0q`OPfpMCeUIl54 z`?fZR^da&m7R#w3Wg7WI5ly)=GjZrs3^#bgDXmUqGDLzVbih8`f!&J;AR3 z`F%B{87E8K_j*Eqj-?#5FP20Coev{Q36WP}ZM5%tP#%KA!6J=(>}XSgSb=DoJ@abX zXTIWGX8Lr-Y*u&sTn=VCur<6fKxR#Qy`QebZ-7gd-J@W_ei3Jdk@lQZ@?}r>qS*FR z_GiUJ#8zA+gW1A5m(qs6R7Y)PVUe{p*PQ4pZO6o)wLEvS`OyHTA@#Ow8U4{zhkEIg z6iFYNS4HO~uF$cfC4vxu^KVdze{i|`mRig_g+m)S=m2;h33EadYrfz+ zhLj`2VKw>LS0_P{PdDY$XuqY48!?YCL#*q-MS81WP6X(djS=NrtE6YuFPZ8klLuYd zju6N2@Z5v;jg~|YpRevefIr3alnH#fJbfvA;77;qmbAW3A*C?A;^9RogRChLIs6;A zN4j(t%NFt}WlHcbaM?&ptS8ye`hiG%x_C+Xoc(`+Ge{WhL>f4;vNWASy!Z;zvMlso@O4j|6X4Kxi8>t zS+Ph3qURb*Ljl_#Nhsv_68P6RHw!kcGx@x4Vb96T(Emiq%;fodN_pFbm6#9u!?SBh zi4xlDug|h)TU3%8b&pGZfR+%bM08Eh*-&={^0rz)b2A_A25f3&W7DC|9uw5eZ! zb7{8Lkl`U^-;0H;1d$ENAD@3xA>j#fbRYe+XuDBKkF0hAf4smv0*RE~cqoz(@Ma?t zphvLnOlAzek)%jp!Wg4A!xQJt!)5_c-l^Ip+7EBmmtQ-hDH1y0UPJfFCT|qJWPd$qXocPt;N*2PXyqq0wqhV z*j-Ml?Rot5EXa@FpJK*MGEH*su;&>B#f zz|7pszFe#+)3NXDWu_EU6-u{2N8)79&%+C~ByRKNj#Rk=ngq3a(JW99R+^tn*UV=QqbnIvyA^1x5e z?R8v@hOJh^2KPa5S$Mw1O4gLhey3^EspBoe1+Vrg8rGxEwb}`p>4F;s4xfnc=Po~; zT2N{YR<%^x%scp<$&L=(96+b)5Rv~PE#op)U#ZCoK(f4NH%Xw)H%aN6-ZEkbeJ=kP z-n>(pf1mx)Sy_$y*C=#`@SPUhI~Z4PY`&Qa+y212t1z*l^E?& zE}lXk13dUk6A8yGTdelQL35M_uY()bB@~rF(rg2RZij1{Hrhb2~8IayTmAygX9LJ2-HblZQB}897Gu-__$X3PNo!V z^sYUdwRF0YFT%O&0Pe*|%(;@{bxRWgfCOt}1b!UEP>%|c>G@YQ9eR3zg903~FOrb} z-x+B7m*(!=9>J8;qe?7=7_GwkeoW+7J${vOO_bCaP&N@Ui7jn%hH-wL?YV9X*g~xI zsmrs>?d>1Io@N@36A%AIChHo?yYA5}^mUti=}j;JB&=ouhTb_J(>_~mkiT|vC=gU{ zHxH!@P%8xgU>8+~(T;9e=mRWITC2w)#y;q(Q__T1_gvn&{|+j5m8(qST_f-H6~S-? zo58$44mT_RCF}PH;&33%ZO%Q;_&^Eh$H=o&A3nlgN(aUmbio0`ezLEP0GkL_fj$g6 zK0@?Zz|&J|MC7Nc;t*~guX3mytWK-7{U6SS?R}#MY~q_Vak9*yOP7Wvj>!gh`!3Y! zSYy6~9tA?ZEMcEP&jYDxF>%=>+c3Nrb`>LJO3*UDENt`4$>0bdAKVBOq6FB>N#4Vz zKQG%{Ftr>j!swwEx%b~&qe2-+4ocDYa)C;%_ZXvWOHpb}opxB@S|elBvq`CWo@26{ zD#Y*$Ndj@Zjvy^;xq*A>Pr#rB>(|oO2C4?bwV~Qr174E{|0OtV7*mGWn^v6 zF*;S6p~95%aokZA&kW-!c36Lk4*5mH8=xbMU)=gny`c6^VB{>ekHLU#89^1mnC_s{ z1c9tDr+s-WBZrU}4?+_=%&Z&>jx_f}%HXABoK>_8k~#T7(3VqqTpGX=WH{d7J1df3 za}Ekd>Z|)S^{!VO)zz(V4g=#X05FlP1TXr|aO=75>DACBV1Y=FwYLw%nXJH8a_5n% zfX`RdN&SPX<;eqvk_Ni#ukf|3@xb=I30#_@Bgy9tIE)_iHIqt_W5jGBu9n-S-K^n( zbd=#9U70MndS50-)}AthJw+fcJ|`{L(|GufuuuHDc0D~oVC-=XpTPk2A} z*KHe^(+MYM-IiFxkPXFgY*bCbuh7a#6tJGR6wrN&P=}jexclzqj?!O>^-{Z>;ss8T zx+_zg?SOnXu&pK$!_2+nR5?iry3OE>;$JWYjePAu(qR%xsRpL#U}k??{!cuEH#{{5 z0VfL^JhT^pM4Qk*66;UdmAw==2Oc+Zq za6d4Tx&_cv6|WSnrCOT*rzc1Q@$k+lQQao4=B!jX=Ju#4=ESZp;iBjvRME(MP_PzM z=D$Re9&pmT3nwwW~!6n1!2aPt$oJ(!#)%x3ZD**r5)+N(MATTDH zT2Jt~fc03pUd;Y82gIP3mD;Xey5}2^K9Tib<=`W36|{G<;q)l2+0r7YBOJ7j@rn}8{MgG3 zb{Mkzukw%_dQYrGsqry^aLk=0J9ZnY&bJN*T)_h~lGC79vdwB?>v6Hr-!%98{aSf3 zoB~?Ov2HN)&idn07>Y&#tj+CBxPo;XucHkrSCHr5Y-wf13%N%(VX|Kk7Ck)1khGxF z=q(lKfq1aA)R|1^mZ*ynvm5=bxwS!l%Kx(z`HWed_PnIzE2~esg9w?Fw|FPZgw7Zq z5&uj9phfTtI~22SttSnUtqwbM_*y{uDB%>O z5-bD{%E&40RD%$m7J1O~m!7TqOZrL^RyQS~@`ZEF$IOHO!Y;gBAdftSch^}Tg5a46 zqA%9#VTQ~H%@QnTz_sKC;9x)4B_j|OV$P5UCfT5!_{Kz1bmou01eo5W*5h30DDaIexd{ zy7AoJ^{=9LnWS?Ah~B=~j}|Um1IUtoMxU6zgSDvBQ-j!t2UfffGq37qS;$EmsnHal zlH!hs9wVs&xWWSn0HS0lc~syWvF&g4eNsA=n@DDsR&W3aK6I|1)loH(p+<9rbEk_N zipuBca8fPcYHxomR!mro(xoqnmW~?hz4%7Xr}%7nn_z~WTV6QkaW1vJtTT#1=6bDZ zZ7CHhnaCM&w#?ENc0@YG0Q3`qWKUd!iYDXWT6jsc-W0%#Mw|8IsQv@{) zW82+Q@dYSgw%qpbl+ONgnVpCG?)*db?e8n_3KxGmU-3R?RGp5E?8{xn+XTIL=n$YW z4|^SE?Y)$1BPgT#F-KdPehut43I61b8mBx>#7q(l4Ed24amcAkLMdZ-I3ZqmEg$Cd zhm|1IlaWF+u<~lvO6>J6$YrWm=C=Dvp&pzCv}Ii}@cTUn(4lO*m&{@Orv;6P@X<&$ z=~pE^7TTyMYOdkXp(d{GLe5!}%6$eI^+^JpTahhnooXrz`JVK2WFHK4#G}2tldG^L z+#gWfDfZuF^{chc!N|J#()OMfHtAoJuk)wH_8&7MWJt3trv9b=sY$B4sGmfj`13l# zF&F0BoV@k|EfA~cFGzGH!%}l6qI!(5@@9L7_*(-6^v_YGgeAIG+ZHhz_0-{M??a{G zX}p{29EB1hI``>)sjLWTd75|RFx@6{?k!VZ>M>0KuMgzDaQC4|{?g9*0cy}7GeMS~ z)Zu6)XX@y<2QbE;2mkzwOZa^#V;x5U`PJ}kL^?c>`SPjWm9R5>P4n%4W#6V}ioSsQ zX7+r5{ZI)zEnD$Jf{CwZqQZfIAtv1tD_HZ#eda)-Dz^^lP0D%it+)6FQck8klN5rj zWa!IZGY9{GxI6l&PY48XEJ~c6ASd9(U3Z1)1_x76Qhy5?!Vits8g2% z6|Cg!b>=I*k~Xa^?xfc;e_zumrHVFl1d*2PlGmvDX8P5?kb>stlze05k>~LYi2kR8a7Sp~1x=$q?86W+)_Zvy$BA zeqx;_h(|uTUzNs-W2o4`9r9l~*yW*>7gc#D{o1F_wP8C^6A#J6sr;!Zydo15W%q3q z(wAtSKHL!>67Xf_qibA># zKyaj%}P*=nCx%8Qg^_#h7hp0UaXzm?~0XAJ{_DxrG#s4EjrTzQH8*BG1E;yIt?FS9vdt^6A`h~E$y5voN~@M&AYjFXljeB$6a?6Lbp zv(GPM%Ed8lp2~7c_PBN8jEC|ymG5+(cIYOhoDAmSb9jF}hkfalgb4lw`3^8H=jay} zxBeqAI9FS*H!-t-ITaU8gYFUIap)vFQ(mU%e`FiUUoR^%9lMc_v+lvTnZy2Twf&1@ z%7@k?)Uj-;*5>*EO7C@p*ZG3>HL-^$&hZ4ebqH0#_L!moz4+-=KKg!bZwIs4B2$T> za$916Zy{SEn(<79^c9`B?PFs%5}H7;tZ^Dr4zs1gRM^)Cs%lOu>nO)CPTJk_cKp3t^9;IG)%s^QSZxakMtc z{$lQ=hWVncaGQz^S4HX^Y(hpsft{@`=#vhW7e^`pWbzAK{S?BxelPrw!LIVbB^1s_ zFQ|_K+Abg6NPKUI-S2Zr~^jao<4JC8v1>}9f&Mf#WDupr=Q^Ifpf?qBT$LvlgN`VkkI>28YZx1 z7sb_t|4jJ%!eVVHDMlkm;|rwY(MesXslp-lH!PeoAJ?_xU;&YK5ok72pGkntU$DPhswj4Q|JpP1#8_MyRn~^Uv*wRv~OtlNRjl2~Y z+aVTF`!?0YkNmySR~$e=m8~hjspzIP9W{;_|Pv$0279aN>>lL}EqRJ>!lRe+O9i+LIgVtg; z34&No-ypRDWcRJy6b?C~NhAbf+py|tY}M*~gE#Nx8=$~!I#t{BVH)KXurtoYm>!N@ zb!QNRIL&P30GrDwc^ipR#<#jL%DU#mcKG%l5{JdPe?J+%hqnH~6pewz9{ujTs%4yH zu$;uYzO;_<&AN^WjKeMnph!l-Y9@G{ZIcYC=R}kjIOhMcXtk6fjr9ja(2a7>2aF#@ zD&V|?7Ix3YissxkY3p#QO&E!<-)Yir;xU*|aAA^7NFP%5YYM_~f+s-_quCzg*q{jLjG*l2)sH&whdCVyG)hUhG6lL}cu z@Kjp(@dQw5kk(i-(Mtm%gR*+*WCnJlP#(2sFLm{(Qzi^a8sda43Y=DlPO8aOq;7bS z9hfTs1ccd$d$oHIMgaSCQ95s6L{RNEh*!+d^ay<)l6v~0Srb;UW96SyE^>(kAhz|W zyk}RXp7B1tD25Q%K(1H0lM@Pu?jjgFisf)|NCtKb!{jKO*w7nH!>n<)5{WEE@!LWr zDV6rn1lB7ZDG*+P98Qr5jE6V}6e=V-=?6w^E(igY*d|NmhNC}oa6*AX|Do4#7Z4CB zOr}5Ddw8vo?M_;JX2=Pv^qoGNsaz#K#Ju(22a2=IxC}KNwGZmBS6oOB=+R+3_Zp)q zDr5Cimz~U~Xg>gn6h{Kjz-7%Dao|$Xa4C}#KPru0H7{M#m|5n6=;VGra+!_`tD>H0 zQ#Mva2!4dS5fsgLqk0Q(61~L7m|l@#C|G{?Y_gf%^RT}%2!Z2)p1)OG{fmTryYN6? zBw+T%rfdA`%F%mOoe7WPI`ZG6k9^I;j}CTh`)-#}4iL4@ZFLwL=s|0uP%%_=r40nX zAD>>jwmn0W1wh-KnSC!nj=P-SEvSH@r<8Y5%w)GJX1QF-6oYJ%1`S48dJn&-D%$Ut zBO)QM1}dJ+Fyi7)O{yKl1Y^%fq+$u*^(Mj8u!`v{d=c^|WiPLKF8}9zblhA|v$BdH zF-6+t5MX$VwpNS`%rG4eI^qf#kR=BL^k?K(R8tDpZs|TYxON>kTfJ-33OF&{XFwK`iLyYl4`l1kFKa6~{hEGGHR zEeRx@bzc%bb%6fBKoR=Sjv zV3)gc-%kJ@KV40+st*J4b($u~A!81dOJH;-`7(#y-PJd>R>L%2jVnjF>=Z`R&`Qgg zk$kwXqRh$X?59mRiSX!X2n7D=lHvM>7QvB6o{#5R+$(7Zy{Z}1ELDuNfl$JuTR!_)*b032s^W}@JjAk03h3*a;K}ifFHrbSx~F3?}R<} z1F~+vsVr%Mdr;f!!$g^4n$?U<0BH!vBRyy>jtaAXkM}@keA8;lsfb`=MPYQEs50nX zp<#TtHP~yebPvj$v^RT)SOPUsfSxNOtxGqE>^!1Ob=IUgU$MSpzq6%ts?$D1-sO{i zYiTzjy?WE(fyr+L=%zURlw5GiK1j7+r9i=6l=ULK`@IapjwA*Obm&2Lo(szn+J_S% zbD*?SlvsXYaYAOtofJdwhlRsYsek>ql|~QRLf;qfY{TxUr8SI%bf!@)&X#9E%EpZk zAg8*yXC&9;mQ#(Rus*LKr5jAL*b;r3#;&;qo7*dWq|rPh85`MArNVu%zbVdege z{I0E#Id4KQV83JR6id8MlwU9D#)UvH)U)b!dk;XEBi6v5ZrDq#3iNzBoP{KHQ^F>IBNt%i@$jZIohjvJSOZ zGA5}yCcjO<90m)t7Z2A>UeVP?m)-VC_&rkG@clNzO1E2chS7*MCN8Xb5A?j){s>qn~1C=A|^bI5!Ko0+N%alD_wek0~no*ZUd{KVwTp0gl!e#7AP0m6*bVo$K!_1Q9 zt%}3%q{o!SB;L4TFFS0}w5TVdHMLVtsfx+2{E~1MqF#@S)frKWqCuGw^N{Vy5o(&F z@h17W_C6oThkX3k9vh{F{;;1NiN5!;M^6nCAnXd$9VGShaI}}Q?zr8a@mFy7@8k=q z{XFaV%(2Go800V=o%RTA1+1Iu#t_Bf@D|e>L+RFM1jghi+VPQC9~jUWzcpUuI=*Pk zj)OX(RHR&c|KgBxca)mX+XwNsrxrbuSSHru;zb{8q#NfK3N|{p1k`VN3NqWAhfV!1 zMz9LF-n7b$gKnrM(!c@J#Q@Jl<72?Brdv>p{e13NF_Z|eXwl>eW;0}Yqfm~eP+KP+5N;8w8JnCMHz-@`kIg*w!6HD z;UVyozkK^D6YvbOSePmR2!NdsHHW)d**)L`;)@)~3$$RWaF!@(jckkp`Fj3w*pEjz zk8kT8w>hHmn%cHU6iyMtv8HMC{--)a#H|%@Oy3DNYhYj#*w4O}D3!4|T?IE2PO!sr zpPD+PXP8IU8*3G(wgk4v*hs^%yf?8q`zYP$y1oU6(fw%)%!u zppOwa={wVm-qIlkr5W&kUPCJs>o|EpfHPa{{!o)*H|)Xs1S{h{Sb=vz_1;xe*zne# zD#HDRS5X(s00;kTxn(S^bV47o{M)C>`LCzOsGTm4D}%8r4oug996O@@BA#W2+c?WG zsoD~ONkzU*(owA$i6krsA@bf7j_2-yyOCxA5}x!~YEFntpxH1s>JXvOY*l zkY33M+S!f0L=G|J7Bg?*AIU30RZ}lH+o@xS{8>;wf*slDOftfm-j~ZcM#Njjyczi> zbb?00fd89pC#=BC0SRe9{b&i%&^)7US7d6_p>A`HOW}@Id1*2kwcnPNbH!#;%CxrX5f1eN4wp;IO7v@g^9s8V6jKOG0 z;-z7dQHk4vR|G2Vyq;)v^BSC0@E+;b&iMj%OGa`L4zp_3w=oNsi>co}C}@~zm^@{A zP;@n98F005{-N`^PhorpCS97NRrw&q$u)maysr)X(&L2WQ7g;jb)CZx*_i-G(8z7~ zx$uH4g$0xY^gHO__F)(N#nV%7QC8;V4azWa@Ex4nfZckpltMc#Vs@D4MxfX&22Tt+ z&^rH7RdSxi;(qWOuRhxc_}&BtUbh8jOE#b0A!9dC_HFdFc+-^q5Y>*1(zzam`t&o0 zcaJGf&+y3e?v)KP4)N8{q^*JBQ-N!C+{wBS+p)%D-_QIF+_JlA4jLZ6CmD!6c?6!D z_a=j@ZS1TejJ7tNfcG93VjG=BW~(ZPL6<*ppup}(dRUY>aH;13jc!xV^^pP>$#Hlw zPClH!lca8B#yea?uz5Uf?4?#d?Y{O~66H>&cg~rq_qS(^QP{v9Le_e7yVCe5PH*AI zva1zmu8TJ2s@vjF+bNXXTpIY5I5{nz_|;Kk3#mXKE{ zn3aA$iKc-`CHa_F!i6zw-ir0+f>w8#M6?8N~AGS^S>&LwaQ5v6f8 z*h6zHiO|v~N{A6}8;~xoE#Zgs*{8z6#g}|%!E-Yuc|Vx4bj9Zwq%TO|wgQbSY=c(% zg@r(yCGI1MAhz2k=y13Y*{%~SInq8kS(O&6Qy7LUEMGxVel4r^=Cpyyc&!1RlDS^) zo(DIR2h?VxQ}u|jQH4OZd__{F{ts9HGm?ITlZ8WXDrC+DrNK3kwVal6p)MO0<+p~h90S<^ z#SPT4Bgp}*#r7c~2KdCC`4t>AcR*pI_fT7eoyte<#}0pLGyC_5yJFe}YoQ|SdI?V2 z+Tey}@OMAilrd2x-Pj@{25IYcaOaDacVI=}*tyc_^)qFI307UdX@m166*T(s1a#(? z2JT7&$STeFK8pbCLh$m3)OG{0F(E`RGnj;}894eHxU2V4?{WM4dNZU$?{y*!K`mdCDS7Of{SQ@gv8s(3a8!X=BtLQ{#0@aCNAqhobs6Ut zaZr%D0*OB_JKd`Ag&g;r4##4@@!)vgfx!yJSmya}aPs-855Yc^>sXAdvtKM%Xv;_! zwkPRHb1>{KL2E}4mGXfK6hhmG;)t1ockFV=;s)vt{-@Hj z=0p^jlxpOk^{RLUQnkWVV}64j;^vG+7^t3JJ)^Tc6Y!AV;o`u847g1NT@)0nbHiYk zKzaJNBPK*Ik{vgT9aN1TS}-xO6||qdz>=QF5#PwJffTo|Hf^j)Y?ihaLP=LyR|{)T z#gT<98iPcdWXrZh51kC=&}s0V>I`Z3jBdx+vo5g`T9BCQWhtV)ID~jlqVAuO#wo2D zTrz~NI;L7NI0$XsZ~RshE>=}r$`*-Kj1>GO!$UTAzIskh9Ive>i3;(VVgofibdeoW zN#;UqMG)F9#AfA~HP4oVe7QceQa$2OYKkPLJqnz^c_APWA1}A6ZgLzSP!?jJ=qoe6 zERPX0*Ll%$bhtGr_>)hH0YtaO80~pe_SnUK4tU`0Y z8ZqEiVoo-v3V2PXonY~t)lYH9nNB0Y(tcdaum}aM21+UCwliN{-?!6Y2_cVEi_5QB z3h0ss_>N`Wz+-XNLc%{Z`I-(r>S%#{Li`30ytn+86>%HZgHCV$%R#JbW0VrHTtwbB z&;}pmg+?3teSm1^p-}i*XK)+61y_ng(s?R&|3{PBs6eC(EQts8m?}(N+Ay#~S5h;iIVt@BC`` z!G&S0;G2XxH7DtMDCZ9VU|Geptb2`lg6E@5Mk-=SblDNhGANwrg!O9S9Z{WRQ?}_v|L#l?W>0GWnAOC@FiTU%5VZH5Wev z;zhw0bfOxqkr3;nVhmbdO@Pn2{$9>_<2|-nodht|Alv+_99%Ul( z&0s-*kUc4Nr5&&E!_D)t8<1AEaEJ!6Dc8F^I%VR!Kak26Q3vEg$1JRzxd3-69u^f= zudfIHnh@6zT<-0Dk4TFWu{9D%tQ!!0I(5?*{~hC@-gPPD4qBJyK=-%OiCeAd`$zND zAFqVXNQ&_(3hRX6v4Gy-dg$IsGN5G0r5hc?sfnXw{+#I64m8eC^6&UFC*9MwpHG7^ zAM;m9&;M1HD+u~_n(wT)2a|+hm2Why(xCi6Mx8#}I~1ASo}x~0;o{FGuVeIK`>Nyt zuL_brJiM5k$2wdIH=`E2AYoD9=c1Exbi>elGcw^2GuSV9lrP*Os0bl2mt&>8dtdg+ zCRh;wPy^sc)29PtNRKi{K4_hnHWR9_5|t;C@eq4G)!S^-lOIEMUZta2GiIAIsNOR?!rqileY+3&F%F8JdoZ2!@=E3@V*67xQ(LvtzI}L-ia4h4o5fvAjr~D`)8F8tWM`14_ zZa$=$`r}Naf@*%7L}%4v8s%5B>zT&b<4`E~Fbjm>i%@Y#cUWwn_xd&`Jp8R!ABtuz zbU$$~dyU~Nc~^+|=W`f=sFGyHQ|kyNgU~_QpOC+sS;Ch^Bv@X4`6$#A)4Y%@vz zJP<>?D}~kL_TZaO#+bCw=+8am6B%I&1;nvxmIN15wE?}H4vo{|d)J$0Qii>J-Hdw9 zh_M2+k1+wWZ8hgKX^PDk+BtnMxHU+1`{YwxW@yV1yD&*ne@)B@A(vKQW2@tOb~L{< zg`?OPCqC9zY+t-0;oA-)kb|V3KO&UAvr&I-M8{r>F4`moDlUMT%Z=eCc1#?OE*t9wzZChJ@2gQb*;0m-EX_(6wy z8WzBrU#iYquXu@QUajMp%q&&Uuu}VgR(L$G0trSw9bIJ?Y=j#dX|fS83Lnn~Y)Lz7 z*U5hKqND|r0%%)rcJ<^@O{wOEu;J**A`fPVZwLz(0jw=26^@#lF|i0DYJh?$R{%Re z#J>Z<77ISbS^bNtx__?_ZYTfycr(t~`NGTF!^K{ESZ_|XY5P;K&l?Y0%hV$ZjVJ+0 zib4skbY$>=3k)lMAFJ^#3r`WZp9Gn4UA8H7uy|3(N%w9M`bAc~_qJ&_)1TL%c?hWK zQ6IEbPhjs1K?73enzgRRjNZ=19rhb>y9A1WcZsoXh4LU`hreL zqy(P3fTR~hkH};E2v;aH#@sVXI@0o`!2jfPx1x>JECo;C7Hsg~oaGxy%=i0Mu{SUz zExL*C1+{`Fm%`ope^1Qu&X;Y6g_IGAS?W}o*)ZaGkB4pUDtV2qv8C4oAwoO>>KIPm zM>nw-46Vjk^E*a97xV(LR|l4s93KBRclV5()LrqCb+J>K;nNl-D4U&`t>W`VTeIrp z{RHJbG(p67GN6MzSX(M$FHeoV<(#=z3S3I;=WICmBV%ObxRh=KA^t zc`O>T_7!rf4Mw3Y*z_#G8%0}0C1bgc%;g)2O7_nr32K=bf>7?Q_Grjwf{Hbx-w7xC zPBkU8v#9h)RlXOEn{w{4)5vO_)Vgx+$1ru0$~bY?IgPja9)3@-kGYy=5^OOd687fc z?>^Juc)l50fiQoSF3{t@#yEjjST!oYa4GxTu%rI`%3{>5Lz$c{_Zip+Z~9A93rEKW z&Va&KWne_|%%Ed!HC$s(5UCho8Bye}d=N|PoUoO+u@AaVj8DnHLxn4L7)s(eIihxW zl~dkNcrDfZ)!3Q6uT^RzOn%88t+EK~KXIiQDS;xYtFU?iF9_f#$f*RKIxNw#wQv0N z@|lk!MN+H0p_z2WEgOOPJllL1O7)XLKCJ*siBF{`E8rMllnOR5YPT?n7YJ*V4!Ck0@KiDpb+f=vk%uV z>-z3aq5~X#R2|_-*`jqywNLZ+d18GTbWF|j5K+Gy7Snh$JeR)eR_Q7Nxi}Zy;1aFN zd2ZqUIgy(a$5V(=UI84AGfg7XM=NhVvod`e%d7*88_7F`=!p z1`s-L-v(Wl9n|Z=jO~)a^+ZgH*ipO7fMnF9*_BM~SLErTzPR@@2Ah1VdUw6iaM+S@ zQ$7T>Ob0AsBVg0Y4W9nXgAIW^HFVQMC44Ks!%4->YkUXf@t5Q7610N)Pu7zq>84r$ zKpZjKP`>-xdodVw*B@f*Eo(K~5l@fL$r2TC$Y_vq4f-UNa*(>Pm1*Doswsr*wW}MG z7b}Ak2?_Sr(Bh;2&b@r-3khmHl&`&QiZY_Z__~m=O2T{q!1jy`@}uwc%; zKuv=tD*1h(M8h|*vg6Ibe%bW zG-oidf(?c+yx>*()UB0dZ0m;3OYI@+!%O=0uqsq_@k83k&Z8~ zMWNIhc6Q9k&mPEp0ak$Xy1R6&mEqZ{YroHQRmjN+{n(s%_c!RSiP#)!@=@u z{U#%7x;FkceL(5g2zlqo zRJS6gtr2{))nUFp0yp!Vb5Qg}1$H*F4>)UUJN+LAklPbU#~*c>RO8sF@zSNXV*lS0 zPz69Npe{qi9y+TWh@0aX9p(wvY#Pq~svX75nyN!c<4>yuRYs9szpBl=^NeR1xhJB1 zI9`j(&DPWQ#6dPq#W|@RwJF5^-+|q20zkn(U+~Exq!e@GXd8q^bVri%!0XP;MVGdum?UL(i;m+M zu+)9uJmlrY!E=DfG5SDIH^Ty0E7FzeD13EhyPa3ugizsG6Y|{L1>6ZZTX` z;?BaJ2DaW;^Wyc55{X!rzstp56Qh2#R{wN1l4BdMzTgbMJ$VFsn~E6FXAPE~`}< z|8Qx)Jy}`ZpBzXHdo!N^_ceJRa&NDlSM+aH6s-tIx4G~EDfV?H}f ze=Z~YPdXTtl^&p*(hhlDkSPBZfIN5v?74XmCE4cRS)w3pA3s;smKuuj;^Ox$G2=z$ z|Gr#GBdRD%xA$3PA|9%rwchF+^i7nPOXQ{OlXp|D!otZZ>~%tZ@RHQQHR}v8pK4Cy zp@itYmV-?eKj`2X@6c!*b>Ec+k%|-L2`yDQgQQ!P%a@U=d)m1u2@mL5+P|)8Z>iE9 z(B~C$7WHM2!H>X!q(oya9S2ZSZB@%kwpn9d8{`}T$am740@#K_l_NvkD9pTA-XX!e zeK54u=n+B>9)`^fDrY&yGzHvq)}zQy2ftWpkXXa@tIp+CZ__NYZdjfZ{ui~Ak}=E- zO0ry$xBk0e6AR-=MT%#5^kZvd5Kw%eWg!~+@S4P}wwtH`AX0a_s_8pRUMNI@o|<6W zT)*t*uwJUe`NPFy6{bN{F;tfmvu6|;r!US0*1}I1d6kW#2{&1XmpXb`wrjH2^1>rk&)<%c)};d(yd zt;?Gaownw!XWZ82r6s5cb{zv6G)znRptn<78<{@*D72fRI`3=@!-7JyJa|RQh^%)sXHh|D8-^9 zNZMJZj3{oP+*NbxJB^5~pl(L>UoICpXX4*FHXV2@TcK4Wi6jpfdII-M-hE^uBs0JD z4BZ-|LC<$?Gd5fUT1jceW*(Fs8vxZ(c3tl>j*~#@6DIkXL`7610Z!SM2v>Xc! zS5PFqHa_k8B_y+d{d9b&37=>ck+H?s^jshR3|M+jc@41%ukKsv7C8i>7N(YEb2^`( zrjUHWJy4P}^KR}vV+``d!<6+(8$^|T6Ce503`V004Ku9HYTKtV8%+BntM7{sYEVHd0R1z%nvO42vV#Rv$5azgeukoeYEaoRK0T=0Naabry#q7r0V`k4>_?NBhHd=i!62*X=<3KmFtb_-M z#L=E zKr|e^GLx56Kr;rmFD-Obz5FcBsll=bpj=GcXU1A}&7Eg$`cl+r=K$<}LnB4uAd&7- z)bKpTQzCzjf#jjQZDvqoOv}G%RIA;zx-BARtXMm1a*iC3qhr~jTBz6cB?bWxe7C?U zMdMnFz(1&^YOtH^o7#j~O4Js39wfHNqwjWyZgox8&|%0oQmXt>*2r+>^0CF^>xbsZ z0vrE7UJlqr0M5EjnqqIw+Mc&V|O8kvj#aj~# zowrwj+@DP#`JPOf(Q$m_(rA;9LrT@I{r=}(?kDDS%c4;qtuO63f5xq#X(I9yE zlpfkSHp>_Z$ec>-G#exQ+qYx@GJ3m6FDCkPjwW@A^?}Gq-VN~F%e#(?}qrrBg2R8nEUmLxL`SOR5TkT5J-1HS{i zH0@GgAps+(WioF~!4lY372^iD8Eqm4Ux1Gn2YD@K`t-MobOxq$EwtU|;PK!+_Sze- zeloY{IMLQzun`N?W)Q8%oIbm5Pet}YRZ=<*jA%w^_qd6gxm&g`8Nic43#0WMxZ@Pb z-01^r-Sm8tSw_>4r|!Y^kiXU|&*1K8XJLK+mW70{Z6C}ThF2@xDd~E-@s$uo>IF~q zg+p`9dfmcMmL^;Ib1k5Ko;g^qzKwA>vt*tsUp5N6Y#m{w$~Ozmdx$GebrOwE+u`gK z3i1t)E{3VB-cPy_8w>Fp=)>^(dwI6h0t=ibU&@3CS+Efbpl~i?N+v&dV;q3xOi$MF zH!hENBsz;O?Aj3z5{@-$5>3OmA2+NXM}pOJ_mJZEF5TxO=+IMqb3nErV>|5Lr))tY zu<0(xJDVi`RJtYDdOplHbo$~tE8?_w+b{UH6PmZp2_}omS`ayI3(Yhdf)ZKV{K|El zxQsu91*G5~%jSn!V|Hb{FrWU#-F)Hv7u7vUHF6RXEOpQx!WmUH*GW1z$k!yr*1|Rm z^DOubbC1oCmJ)rBa3*h`_zTq9J*I89H_M8|^XSb=sP?ed)rZRO*GAf*CERu;Vb_!{ z-rio&XMm)H(XJv1%4Jpf&K+8W?BP{tR%dL)?i_eX{Eiya?gMRbIe4`_~Kf zT*Qo6@LpBjmKI}zx9DS%Z7=rhP>wMQV|X5v7zL2QwQSYI)VVZa=FQ)v09R zrIwn6&Q4L6yu-6^#*x`Uj+^ z4&`|~&hfnJ@6jDU!=q~_#bEDi+HtBhhr*n&%e5_iQBVh=Z_bU%Mi3-=!n7JZg|hya z+;!qdTMGFQHT>}Y zJY7P-gsP+r zD)`*RViq;i_kkolJZgUC54gP2-1}C>@VQmJp`S$<1&@q^vuAasZ>>Cb!z8h`8DMd- z#2X0u=tKlSGcC3TnVqyxnim^;^uXNp|K+JV8>_ak*nw)J+C+)6M`xmZ@blEpaAUj( zTwu^1(`l`&)-|I^SN|X~uEW!=&|{V9MhNl?ma5!8F&obiTcBGEXyGpc3rSmv2h&2% z>LNeUTRcHFLS06s>U`H~%pV}gQSI370kRq#DCp-T7G0ELCSA<>fu3p#teg{UpQ-xh zR9WVPgv^O0%<@%iMx!N!V(?J|m@Nj;)9#^$a}o^SI3UQru!j2J@zAfr<^efCcTj*j z6!j~kM*q=Lc}`~EIPRWK(m%irXymY~b&ivvaS2?)zv$?z)_sC;Y0_-hQgJZMNoxSv zU5@dI%Gr*=^LYkXP8UQpQ8cpz@aY7QOLW{8RI=_(Cemv|Jg8qogqP&+YV!y-dl(|w zqcyWr{ixFctC%EvB!ULIW&KcvGMExYPjG@QfLT%kNhzcN{?L}VT9r`F-U9ps?bXtGtBgo%+MkeeZ?EH*Vr-K2WfHi$ zg@@lsjL^nNH?+Kva~t>E6*;lF=)weqF6{hHsPdN2TvO=22%WbA?+ce0YVZ=8I-Qh)rq-Y>*$<`I+j{hWXXs(mj!}k2HQI&~cVy{TqRj*{s z&h_q%-1Fpm=6Eo;V2Dov!rK?2eyj77)Fv~s9@(6k&UN7^_5H$2{?u{&f@9CL_9e@Cy>Wt! z;Ej1z_hZt!a(20U4+6f*w+f!^S8@G)A&qfyx@sReZx=xj<-XO~pkC)gBl~VCggqp! zv_ZPAguAcu-cSfF$D>}&`3>ENQ0V|o?7k;LFmiYL-jdtqskTfLjG6>3 zyNX@lyOl>i_Vgh#uQW8n*Fnc82DgJCN4{kB0qD3wd0}K3K9?nU}Sn?>qJ#V!<>3I{|+hEJ5 z#S%UCD12=Aaa;+7T~HThBeZsUfF)rJ?3CBr>D4Ts^B>igbAuU4|37|LFF>QX3Tfhk zh0ZO$q96EcUWBl_%(X0SbaNbi7iFx_hQ8))ZVeb28AMGG-7^!{9>kLK^+REe*uI60 zK)3D;M;Q`v6IHj18Dmyu3mP5@A;%StNl{5hYIU3q3J;AU;y|!!W1hq;(Boh5B>Cm6 zaP$9pho&k1tBx>*XKa2`{!M!*wAmSj4oWdSg29xY;gXo zN(d48agn~(xSq3b8$C5u=fPdyV4Y(T-(oa+k>}1R*eQ<5Z{&0hYU?r@9~4x(rF|0Q zL?H_HEy@rZ_BMd8TD0>K3U0_YHE3z&{P9x)8-gKsZEf0-vZN8O2*RsJxRAH#xBBaR z@B!Lzzp&H2Ox43$4Pq;klM|$$$ILojB*!=LVT!CDcnEV3drQs@oUNDBir|U;sdD#<05%n&33{X2@sqb?Q<%uGw!m0?=IhW^>oAS|;O?Ume z1sLArV&4J1w=e3G3hG`1!{Q>L{Xm`G%NCzftn_An+5@roPE#I)QHKLuY3+8C`a~Ld z3(Z68hQ)iu@;+(sNGmHobz*6aZCEW*?ho*MpU1^bz58+d_gMMgF|t9*I^|)5%8`fg zL;9&bJGC?=0U`CnHl8XBRe+f?Xr)Ye?44^irQ@I(Ket(F`#VH|%oNl;k{}8m#T9;? z?Y2cl1?G!Z@UsmmW#Tvt>NIxzD{scXJ!V%w2!3c56SBVorqF9rkW!wFgk3r{vrI~v zas2;_zE2)A&q*}${MOmp@lB)Xe_VW5{9z5$p=Rd>7pOlWY{y;M`=(LO)lK{GflVY~ z&Z1locS!rnz%bQ8qp)u@W1K_WGdl3FJPLu-_AswOq|uB5DVda{Ww~B?;G3A{`->_m zX@m~Xb4#wwdo@)Zxz1n6EViH1suqYiD2C6YLQe)) z_v9`6q0{r-{g|`6V{u@HZ(#m_`gI!RpzV1udCZZ)^+!n7Ikt*vrXbAb*e;VKd`6sK zkqi#*Fc8K@hZ=(kGhpz90EhBrh-%-8Vco18{6#tvuo!;y-HLCjCYye&7;*rTYXQ6; zfE+xP>;4INq-`0`MFh)&$&Tr8y9-)Dd#7*@eMHA36t}l8REn&?JgqQ>*%o# zI4S!Lp$Y10X8mE(02Cdl-W_DZQVm?Ch|`Q+4!!#O|h3WGq#Ag~Hz~nQtebw)l}2v-9Osll2^pyI*Q!A`=1Gd7Yu}#2qmx!ZkFuMfcFqc*$er z21-T1G3)LSk8;c*jOZ>d;A;ga$KOq4wQqT=HJ)ksn?1rEeAIbER5AQ6 z{L^n+UPD*QJQ{9$tC?+UPD^gnwSf$Fj|m8OA>QdpCn$uts~g651vtm}Yz+uwkNLZd zUmuRJ8}D?X{A33d3%+N3A}PGsG(dB+GF5`TY1k2<^F9tnFBpe73dC!Z94;})^kdH-+4$PZEKX8BD89uVP+ z*JwVSpvqN)nVu*mP%lF~IJo{~cEq_*dPGKxaks(T-#is7vF>E~O4p_i(5C22h@e+G zEWUiFXkU|RUgdE`+X(02Pe*2ODc`)#iN$-6`K^-4_I#4~i|u-r14eLWiqMcEU#0Ew z)+GG8;6nF|lyLP?R?F66D2)12|ISbXaj{u=0mhd~_}e|Ppl?+la1Tq)m+?KBQw!Y* zebx9sFPEKC4Kbxmk9c|&{pBYb4Xpp2$T(ARFmxP@icbn3d2TrSjG3|&otrC)VOS}$ z`c00}pO|2a#k8oP0tYNhdij!|snx%EQ>dgbt9ekWbe_=>gqSSAhpsl4MXv7$HvWf4 zRvh9PvBYAGDra`S4!bk({{YMDNv9qN>uSb?ovMP1%nFVnob~e(9lM}WBKwWJ`mn~P z=}QS^N*}ZjU=ShxV*53MRR>c@h9Hya{N0KlY|^NS;IRUgc27O({KEi z3zlu!H@atT*xuS$2_n_FCeW8VETG^8VGkH_Cse3{4HDtkp7 zH!qaQG$Ip}wwO;E-*rRgckw$iF?$sn)P$h+(Kf33W!p2)IfjG8N8*557o{e^6D7nP&Hv1Gx|kyD8#PqW<+YQi`=s|Cjxt8UBSK$I8ayr(ryQfRpeo{jz2rq z|BAV(vPB&n$TzXjSb>Y8WfSC{I00z76#jzj?lYR+C_+-$`6UyG07UWkj11o_NVNWb zgx3CGuZmneE0%^VPo}qhL^Tv}+%Nr+lvsxrB!9RUgVQaze{C1R)yQ#^`J#Nsw1pgyM zy#pp8M3W?gQPDU~D)8d9M@|R_VQ|MIm419PWEZV~QNB2G*-><@^A;tv0WR~MSDk=! z54Qs4S}E_D+D4V}-< zR*4o7l38=g9Zq4B1yJ<<6rawL%PZSdWz>htL;1Ikbg#V>{FLhHKcF>uL6|dOy%m7n zQNopPT;B!$iZ$|pw$qim4^Y7dfv`6Z5!6@WndftQfh~mSA6Bw3aN_5KMj)uV0o)0) z46dTtC&>$pX$b9ED`F$=n0G{&Cgk+2yW%<0ZIy1St3Tny7ub;;u$@N}l=>;x8iDcu zk)Ln4QSs6+dK6S^mxDf|P;nA1_U#hvLbJt9@vvvT$UMD5FdSqI&{B(MwB{r_oEk0R zoRB-OYQVv1J|c`Akaqo28DQ10c|b+gd8iFsc;$TX9IP^IuA<@p#bUhN3E7JT2-cvG zUJ^M(BbBX_d~UsgP9X&cEAm2LO`k4_`JfdSZ;!4#NO15`pEqyA?d( zMaClU2!JtCIBXT<_O6p8k5;-5w^L~kAgzOTBM4|S%+Q zr~|(=9HayyiH6A|=L4XHC3nli`czzg-k(t$K~+^I`+NnFWvDHR#OR#rg9t7dx45Xr zirKl9QLYO0!U|_W{}Lre*;K9Wla;LaSd`dz0nMv$5nTy|5+i0e!POl@{wPGws|w1G zZ4YUpC22tc2I?58YdS121kT?5`_@{=q7HQw%@r2Y8U9kkTuf@w$FV5TJwHpqQ^%fC`1N-71w>qlxE%&)|O2c2RtnTHC zWKP`<$++Z+Ch7J(alo?6#wG?U3Nmsbq6UT%^PGuU`dx@;)z17CBL2yi6*ep}(i;4ELDxHuT5_7~s<}??Rs;hUlh2erSR+$k> z0B$3|Ud}oFyH>_W8m=%`=ytJ15=|v@ZmL~r84GIR5;{GJoiMsE#U|8{$Wfz#L`IKfv{#eXO!_a2%~ z)}TtikEXJr7TzHQoB=9vAf;PNMY-hochO5Gx^3X9g7ClruUJD1!!=dV06r0hBKxAz!Sm z7~E}!(!{XbagqaHprD>7UM zzwyz;R99&d^1ii|8+@!LDlE~!j0WMa7oWV7y^^ky>8aTc96ER#cuCiQv4C~Qrf}Lq(jI@s5n*`3%^vr(U78?)0XnZig9t9!voPK5C+mv}mpTT4O(% zw$a}zbP5pXE>OfUI)G)~LS`J@l`Sv+-J%gwdN)W`g>!SlgPXilL z+iUin=_ZpYgx8Su;-wKy#}zUB1Ko|*-nx)xN@zZFt2`u`*(GhZv=f6MV_L#A%|Uw^ zv*>3jr?Y&0UG0qHp`<{8Sf~##(?MJ3MNrq~7pXz1Tb{9UYV(W`)zvmMj=mBoElP*Y zK${D7Q2hEEHamP5ca?)>Wv>nYmRkwR5wWc2M^#8+WBv*M`s$I;X6|O@&sk^jCDST@ z`hMp6IDWDE{_=sd+YA#XR zVKVZQ;UujW&SPDxqSfED9YMqZ8vzzi1@7?E5}wz` zjIradFwDP5_*?4C2c5JrognzOFb;`Is5>&ROIlGPGrf}?pdIjdC`^UYsi zALkYON_e2-{cFAKmSn<3DHT&*u$pvSHnoQg6m>kPK`ih?*e0AY1PGAtmV_QDrT2jGDhXLx*XBZ zY%v!TahTe`R=Y?QziiHVy?da)bC?E;P=6si6MIYva`h-8sSvUDES2a8R$97E_HQt6 zEqaA3U(siF?KXG?<=V&30AW=EKiEWOVSvZ+{a^wUp6q(-xIL<)m|}0P_O!W1lnmby4kvRb@-R zNjw57Ij?IzkYnFSCvFGQ-ep2hSNf_wktqlafUii5rrmmJ**3XECI4DuBO|=E1#clLk)r+fs*#Ioxo-+f zup#<>;KTmHzSpuB^_#$eGa!Q10U%jwe_Q05s%BR|$NZFgnri3za-v&(iY-;7)IRQy zh};eGw3UPAb@T16`Sk$i{(Z2r0^D4*)bk`X)RiK>qmP=tWsb+sc>=ay5+fsVpTxPP zw-04@e13GCZYmSDCx_irB2dXk7W;K#retd{KOi&E1rIDd$Z~kFkl`+ov*s;h?7i&G zPuCFG=d|lJyh*mm%IgI2E{ec>WIDu++KPYXh8HSh=zjlNtJKQ%9(u^wZZvH`rPyQQ-E-2j_k?rz zc>{*3nXVtDT~llp3xau%Rt9He`MIB)$}`Kl)^0#hzm!eQU7PqI(9j$-7(oj)5ji7|!I3yC2 z?#g!Opie)6jZ4%`6wyfimm3CC?qSo1KO&iVCpLr%e9c2-7Iik+JQB=+i*NABF@>a+E5>jY{<@Bs86_hIIs%sE z$SBD&J87zA^OzctoeVThcKnD?>{b12I1y9rDn}zcN>oHe=G7_Cg})fGTGl>R>sEc!C8JYOJORNU7FMzf%(u4Cb-EY z2wEsyq+&U8pZ#)!IGYDpJEg~mJUcuxQrQ+TO~5E7A*^&=W0kKIf+%#+s3wI~>mf_~6H`FQ~mgaW75SO|_ip^ZVf!gK*1nesAG(DaUf zl?E(TBykekXUvI4-%eIL&Bm4bc^$^te*M}nB|GQcy%dS=40#g*;g+=X9-=f5DLf*O z!`n$*{}Xcpg(<$ld;qT?L%~1|_kj~syR|?XY1h)jhWDB*UTo!)mS9!QBX*hmdrLdd zA2X-~=3Ygv-*}+`3)JtYOn6zEr$D2I^Sz#O@?qQ?+wrq=%xMGu-+YRAwmJnESP1`; zV?2Iw7X+B19GV;tUm%IBi!g)d6=$l>7Na%fs5o5nsRFXdR(SS&*YIC2bm(7d=GTwt zWaWqY&^!tRAf(4E=#4*RvZCEj%L zz=O3U>Jup9Y((UXv(awI-R7&s$jy+HUlGbm@v!eP-Ekm&nEipd$GAF~otUbz7X`;* zw3xYIeG9ONS-%2HWSr(Yu%&vPniQKz6IsDUa?bcj!nw%w5HEQ?<;hS3&JQr_?8)OF zuv2N!eA2LX_^PT)Z7X^=*dbZS$R9;Y3_aB-n$9lU+1VSYLw`i=#jszKDVn1{F#i8- zP$=Si**v}so%M3R-C}GDUu@Va7oq3^F<+Rnok?_DlZh06*6noH@eiW|_mC=y7Q4{+ zyEf@)$Wi6j&7ydLn!VW-R_fMIZ)*fD@PgogyzZEn><%Dtdwpph89rsLQ*O9V#r4lj z1-DfE6_NpWQfKZw@a-QOc{Dj^?w4zH@NqEjTmCCMCG>V&F0-uIDq#`CI+7~+*K5V; z3X&iW;X~;IKp)HYnXP^>k)zfGsbz{}sD^+838*TA+FhwN3X+QcDo>#ZB@y(^uROl#UFe#ds}8 zL@&uZ&0b}N=gBjuZ}xZ8?ly=#ZK$xS++9J1&Oj9;Df68yl`mv?O8qB_gCKq($h%`~ z;J7H#;iHlgh&Gs!bs}vs65M{PMa+8X4>GxFcatwW)b!@qR;|h;!cnAHqvVDdoJu+} zzT_wMK$%w})+Qa)_i~`8kP2oJhWOgaa@LmZz=Cbp!{4ZwhiHkUofCgs39!B&K_n2n z9W=nix`iV^qnsTAz6CuSmB4BBFg*|r3N6HCQ%3;)vS$t7iDJ-{96BKxsv323`g4dC zH%AA#izsc3BPojn2u)rCdk@+QPAEniNZE&ck!%_&V=^ddeRf{WC@47yu&yAZ*{T8d z)i00C13o=qsUV7uM!f8?PwHfwDpCbXkGWxN%B0Jh*o5?%upt%%&?SH&JbKq5wP7O# z<}3O>atZvp-xmhf!r@Kx1hAKnJd%(?%TZjP;fS}05X=~N>i^Efd&=*K|&uq;5o4_4MDyC=NN+ zVMgaYkSw$A(6*2R1qwBj>(ERHUvLue>#A({1iwViRpndWK;Ua8w&(9B-)|nP{yzx*PH?JnnhBESUl zYr&YhIbdJSLhVSp3oaFBW@?6P%Z9j*FPyOlcAO{S8Yn;iaixQyuWnWv2xWj7=E|-g z0vgMaZU!m3-43bnRgW@R-zKyMrL&-`@%xA7cj>n$-xZWlV*9RZyT1V+e}!BPvtFI# ziWsZ%YcOkq4RyXs*@0*2BepB*?3r3rT8a&szxt3jdJiw6I)h@L$^W;`J()5z*}G+= zb)QH#|E=e!=Vs)minaTS4$|GXHNWkRVzJzqk&TfeBZM8#U;C1;Cma){zwAERNZ#JX z&4G9HH#buV35JI17JkbP@HR|&>|Ukg69q7Ygs}uiOTW8K);juEE<|UhUs8+91Hwzq zX7(!xyOk6(AmLtkR7K?kx+zN1SKO2P%b;Q;Ep>5k0nK_Zz^{-~z!2Bv0yeMUmK|Mp z{49a;4y&Q@fK@5^c^8A`<-RV6;DuMu(S6D+YKj8%4sZS@co(5g#``5dC#=S}Zx<9l zcY2|pokl(Pofe>PA18$z<5q`1FKk98yqZ;%3$1*0)0j0w5lxA~dl#%;Y@3--Vd7g@ z*Q5h+8KsJWl0gP^!#VLH8(KiJ!Hrs+iWzs)1k(dtP0&D1uK`JwH2ltp{)wF~V7c!1 z_4}b8MLulZh2Rj~T?&|e!eNBUz$RkX{oM0V(Rj~lH!Fz#8rZib)p}P3ua4{_`oiI} zp?*Rog`i$L$ur?gYWxyIVu;vGeJGXsQZod2*QQe*ZSp|VB7Tth1UZE{f)d@E?)?d* zKYx)mZA)YtRD`);VWjiTqhtrIF0j&fj^ug7W$6=^xx{U}^YD}$06NtR2Y|56*%BZT z2ckLpL!0o(H}?YWf4G;bWe3IA2sJs6UnSWq8bVcGMt8KYEsErB=tsf^5syO2}N zY#5x`09t%QT~dN&k;&c|q&{g(U}TZWx*ZCzH{1{i?p}p}U^^5K((~NefxBIjw7pSc zwLMC$ZkKi?ZuOf@UdTv?UJ}B8U3iRQw{k?3ef?O$SHg70_A8IR7^iYxx;DUgeFC%C z{0(*Erl)e{msyq+gmY7N$#Tt7jDBk4J(z9rO(*s)vK|RR0_lxx{s96tYsxUA$Iz%8 z9|?^}T>jS+?ffoH6Cwj)Sx7YsNM$=D{79)Xh8PtYz_WgZ?@;?jx2^b|EJ08BUY1Q!DPX z*P$|^=3=%7>K&#eWd<{9B?0g0NSh$f8LKZ;3ib1^d@4y?ds7Zf#RTXrMy!xDGs~J`$#QMZ54mq?e!A z9G_mcfD$&Su5G$s(}uKH1c`Fi1Z6x`t!DU5+wu1dLS6~=fqhnS;1%qC5ZPXHXt`;( zG}|p+GprAS;})Ll%WmwInw=-I17*W%CqkUE?2ZqAOUI~a(@}-iM&R#HEHDR3<5m7I z75Eu91xSR6kQDOqDr}!)Eclp5D2T6bNUf$+i4CNNi#-qYX9D#%L`qmrP2*-?!*FUX zuq4!<1BHy3^KSV%pAWLiq^Y3lB#iH6-Qm#8EYC_oph?26-Kx5Bj7Kb~gmXDr5H*F^ z#QTa@dz$qU63vv>RcRNag>H%xP`3Bn_z;~|Z~Juy*sw;fjsrl?;r-dj&~l_O!*gD6 z%1(HZ?reoIq{8KI+DvI%l_a1>U2qW68PYb4@I=oo{3fm|!}AO(BzoIG*(>H=l#sK3 z_RKm+5|F-Ot7Z~90x_Yi2gM$9KUs9n**)$>M-e3p`ZgPFCVxg)!U30(C3lFz@J-J` z$-H6(>l9jId-?*|YOPl$2k*eGj(Vk%($z(X#I{occ1fbhSv2m9iBBfjx?c~wLcDwv zY=!dmyGOC@N-(aAV0dlhJ2q@Z?`YtnJkvFJp#_gBQi{_ef10xHlPINgKGyIxh<5l& zR#F{k1AbO5Q*Rv=6U=ZmxKukvPXvd~W)aIWx|Nbn{~uC-7hOgF3uB&x)n_Sr^r^gG z*foIMQ}DCgH>CLrs)Mkf`_p?ogyrQjaHGNJg=rjg?-VQfjpbWqVG?I{2ik&r99lrk zwfm_}&R2xh#@m=m>1pZh=hSpQ{Z{PjUk59$s0HfQ*=DUl^SFUTp((ZeV^?FH-UptbbV~oSA_QxwzW4>Oyo~C~A3BrfDXu z=C>)STK%PPqM%4ZJcgeU)yIzYet{8#9b%k0*d^tN$SK0SZFkmOw+p%_VX`9>Xfc1P z`$_rh?}fKEaZnW~)U?_}+a7mP`~Rlo?F>i_^-Aj^)D>Iq36uyVRKN%{B=eb0wSMkGC?6r ze}7(h2>?Bh&AUW%m-!JLFv^o=(tDZxz$woZe=^_ZJv?!8n$6=y%5yOx?eB?v zcFuxz$z(@TC$_v}*!%uh4lXT=f|sPg9lOoYdV^fQLx$PAN-(>X53C~I(Knf=Ag~KE z#ZxKtLVZq%_AmQsThpO0b*Tg0m?%uCH-9)Y2&k3AMTsvIOqU~0Nu7qJD=|TlLO&0s zs6p5~zfHD5A71#;Ec3pOW)0W`lSV=pa2)?p!j5nZAga(L18$Lhij$h$%lsr9cA}B8WUncGG~m zpQxATgn@rXi&|ezc-%A0lbd!TS?P^WvWqnAk9(akU@nNE!32N8cHioAm2sn|O5j@^ z5d-E_SzuOXIjp^m%6oNY!8+e~I6eBYc1oI+ez8n9aO3w(U$#w))L;v8Iqv)SEU9jJ zBZd=Ldy9P~W5GJnlz=V0o$(0H?9pk0l!Wq?(0T74L8VS+PW!*33$97Td*Dz+;qdsHJ8KCl2hYQulA(uKCI4&>Kw2#D0bN zl|lgu?7k!wG}-hMg+F#xFP-HEGgAU5gncTvoOv%+;mw}2oLsPRttmg414ddp;x(p# zYXFSwig^gxJR$z$$c(o5j*o-;XUtk5#{vi}LNDO1C{ESWJ72D1(aZV3?UEqI^o*ycIX*etr z%bdAy&z=1$_5zO!EEvi*LU>5KR}VaQSK$}K_lF55ABlddxO+EgwsB<(Q(~a%zFOWAi{e)_nJcII(xrj+uL7}H@u<%#ET-A zXPZrp{&pyYe3FUg;n!%sg)oECeRIXz1TZwAGgvc%1JMZOc8& zu>(W8N;s4$&A%?OYr)nf`_491D9<;8!$`X&I1iqd#*5Nx(;&U)?MfvMAFJH!%H}s% zJj7wSTWH(HY1VTsA@9w4V3hg1%I?{R8}pU#H2AePi3^8ms|ny|t1k2&&p&apJ2nsA zy~Qit?sLF>Ww5|W{e$_)0_mRPlZh*3Y8w4g-=(tj;H>(xWs($JTzahqtM*PA#O?dt z&5kObptNT8f5x@jk*vm3`G-A@A2p1KDsQpr!a?E zbbmN#;BtK*;FFfAX!BqyrBR;GPP~PBf9M!BtAR=iz^+<_i6M%yzeSQ5GhQX<{)o@i zuo18@q-JB=KB=^lV;1F?(P+RzDA<1uBZC%c1yzw+uZv+gmWZ1^pECncb|?p$x4@ zO1dUKPb2S5hG7&KlF2$$(sY;83ZRE0Jt;Ws9o^?J??NX>7Z_Iee21nu#u^X+Z(sUY zol}45LBEfi7~QtUIZ}O+JS2p5V$NdLABI}B+OpWn8n{PDG;qCRsonh+X6aCsu}Fc? z-(Jwjl*|*sFC`pnd{<3E{7H~yq_Xqy3W@m zfutAgD$Kj6S4bDeeDJlGCSkb^8IYX22G6WZ?PyrV-~Wa*|MrUxiSwsQQw3p7y9@u( zP(4Q%{Y|JZ0B((BdG3$@AdQVg(7X`pH3eif-!hyAK9z^gIdF`xhmvtPGf)WnR-~mT` z3MKL6zaZ_rgW2H=m{)ECoO$u^XHv?#fs{Gy%qwWCL!s}@Jkb`ZM3w|~znS{DGs2au zBq-#qUVqr57^zmgCWou#SDe|1y}14eB;aygVNh`tqv2sH|3wmYinOA~q9}x-0C2Tr z+ht$5{sCaD{?)SHk#`3}Fvj3T_|#v+gl`E%WhVe{3=&5n8_FTaj1AyS|(r#5% z$P>swN)pztq@o|$XYi14 z{?wqZ3@~zU<~~|UeLWaywo(9AVuP%Rvftrw=4Z07ATX7LjMD8v)Rk8^+ERr_(=bTF zK)y+|K?V(G4Jmd#XoIc$ykU)WyRi?>lT>E%A@O{jYWq^M^^(cWoQe*Q*{4PFUBKv; zUY`^Sk33Q^qnSqnRp?;RiSEnD$Lwp-(VgVy1W@5pGhXoV`Stxsq)f6q30Xw~b9Q|G zieWkSGe%0GcJSYp1Au3C*ou?3unQQPB>Q&mK?J z@~8JOrtDlQc>1UHBM{}Q$GLWV`ULyW;}ZN}pGVd_ktQzvLvZt_>tI7Gtldbxq2dE$ z$~_i@+H)68Ye{e1F+Gj+EmWr!I4@LWsE)P*T2v+^d&OEc>DK;_G}XUD*O)cEpqF;P_;T6-{=&w2h`gfT=IN^udC6IxcrhgKG5qMSTI_At+!l4G3PQ5X}$Gn z<^3tn@;p>bZjV&>7s~&*cNU=~97`UqtIRNk<%`SPLsH|s|7st*MIvUow#?xeWPtM* zFO3Z)y!D*}(L|{nDWdNB&ubl%l`MIFXR6KXo^{j#8?wW9Zv#SoUjZ;QyBsZLbt!n)h;y>v;#EC!04p7`j^5fM zA(ZVW1RioLz_SU;QQ{C&%OhmDF86P7KcZ7~L^fX<48eJO%D|3A$3ub>kEs9VjPt+R zWME!bun5LMn4U)6e4L-a(4 z=4si?2BJs>zyo2xmpH!TCRY^=a0_v?Dfs*&CdjF?T|zfaNi}e0?>z3#_eBde&V@{l z;kOoT|Mi9b$32V-CzAFBld3rX7GeGWtIpZPpbL5T@FbagA0^n;Q7hqvY>|L=Uk5k; z)pTDqH2uqt^~D+7oPE2L21cx0)RPt`9==tJ9eh;AwZwUe3q1;>){ zGg+`S=I#U}`mJ9Swca}=xEEYb%PMxq6Mm5(S!f@U(B(%BLWd^fw-(Yzn!FG(GfO>f za6xY90Tu5rMgq$r=b(QIr+~Lz!}Ue$7#6wL&wXkL*ybjH!?scS-&W#kjhEI9vg6rq z8I?LD3??gPl@{p9j|VoN5~aqKIJ(Zp$6{mJn}+}{)zwnAyi*iDGL&V0lQqa707Rq? z8){wpI1i4|OqMj_Qu36Laz%na&*GKpo=&}&m9}U4!5N_y^of`yrS6Kk`oPZsm#`Q>`80#YPK+rLI?auld}~!o^$8$sO)LPh1l=DZ%E_Eq zen>)k3wR@UC`n`i#{(ggD9ZHaiPkd!AZ@F=FVf^@P*{=eCY?GDfv$W=a7b`!3V89# zg*sHbA!oi`mmR%e$DF3`9;fr4>P!#t_VX5fUVsA_#FVGn2&>Skc--?}(?Nosi@Xi- zQl-_4tI81~%6nD#Mr!xpxlo&!Z=m`lB#VOz%UbqUcsB{U5>B|V2&^oA#4`xvUzu#> z(ynY-csp##Pf8E!Ux)XR04=%-a*lW7 zi^Is*!(s(BuU#8K8>de;HWNnUHh$)M$8T{t2n#bTnc{|Z7FsMLYt>6s_|942Kl@Xu zKWX$}EB6mmo~v@lIPQF9*x#k9-Ouaek+)JVVQKco(-hE%79O{aZgo)00-08@^ox3T zy}3RpnF5OZz+t(WOA$|meb_q`awO9&$8d=U7Y$u*2**Mci zLOR4b*bB$h;H~s?!li7j3D~b7Pf7+HX#=iE5MUB%GM?UDW)*+^qb_EBC(?CyruB*7 zvN_q#Pan#yJQeZMMm$!U@03jUxMs!LUF2vENa@ni8b;)9i?-9hN>rJGczZbE1m5Qr zc>;UDkByubTxE6yn7bao02hpVGbOz7q<7E!rnA~OH!-HpKB%`g&&{&9LCGj2DAd+_ zXfFzc1fyuxUoHJ}NDnA5J(@(r!|WJtp~fPEl@CLJpWVRxy;m~9a>c8?IgjBKZIW`y zl4A#x@AGb=@;HS3Xeq07H*j|DWs<3)?_cArAvoi&2k*aLaAn;m*u1OynY)E)&rdgSE#d zrCj~_s60*}^;qxD2;0t*OIOl5X$dv1cY?L+g5PPGLuq3 z%2)&boW``Sr1o=WXmj^2J1}n3gtN)O*eLhY6*B9Y$=gNsGk!cHat)zB7^H?kx-8U+ zi5pdR@`X!^^aXK-Bg59IY!9L!sNOgAl7PI%2#1)p9?AbJ!hjE_#mY&4<)poDU=cu^ zfQVHD&Bte_0dM0X6l#UZGGac6r_9)CbwV*^XAcaL7H;W0t&h!fDNJ%(xg z@}(icz}n?i2`1a@kKwtjLbV3AN6RTgus`!0^>K0k$F{| zsa!O!*0eS?fNQ;%`USjpnDEYu0qas-n*Z>TC&n7``1I>0?yY|vXWhUH(oU5nsG^0PDsbGJN*J?VtW`_;FZ8oL8tS)iWKLZ8x@EJ#|Lx zQbJtxr+w}c()=yFvKyPLd_!MITADs3nb$D6zFiJ|hXuTX^}MyHrdA}uvIS*^9LZOV zYkkJ%J&VNPppo&22XvOUihBX)K}=L?q11^Ai=NsvK3pjcP@93Ep`^nFbR$tY2{#c* zn|c)ckN~3Sa9<_XpmT;diL6aFHyGUn?8^f@FcMn1SdRGb2}EXg-egk-U5-%DOFKs$ z5I#Sj^bR0t%9nSpNLw+b%CVe%qHLr0gQP~I^JIM!&r1Q@D2J#;?j#!nc4y5q#v4!# z%2&N7hv-^7*^FYuZoP}y_Et#)wDpYvPI?ug-YZukM)kR&_@m@{o{KR|NJf55c|2G6 z)oS4Z#@@UpyAr=dcEG1q+0EWnj=th zL>Oi>!~i1!_F9bIqPyG*PuwN$=*I#}GyA6oF4{r78*Eq{b%oPZ&KJ%OdUUXO=aN+J zb3O#iI|Qca#W}p9y1O=$VKoi>9AiS=7I0VtS#$tjddEViriP|iG^pF4ERVb~T&u^$ zO7t)LRbF$zoaDD5F!rOJ!RQC!hVV`Zl&8KA>zLz1r`n?F!0%3?HKGTlN9;$d~Yn&_XnUP%#UboK+rH9-imNs4??8~r886cCA7 z--?`U6m6>H+0LFK`!>vzhZ{^^fzJhz-7QsZZe$_R!+I>09?>`?)%5U6t}?Y+l=2oR zy+1N9PxCn*C$EF7~K3$uu*W6}2sX3#L%W?QFqacZdp{8n!2oZKUm0A?X-X z<))L3FLuPjQR=>LeId$wrmB0jxYzTt5G?g@p0OVmSQ^x4gIhhj?en+2f0~M| zpOYXYQ|@hN4PnG%pc~+T$3THBV1=OwvAc7U4W6$D0RhlC)7xY*WFul^S>`Wq( zxt1W4j$9GV1B3_}l~_dGzG$VOF1$Y(zTR9`qTTa1gsK>F3wNVc00r2|fiy;@mr@~5 zGA91dyv&xRVNGa1+`?S_R; zs-zYvWi68ntG;1}eH<;uOs9p^NTF&wymMZ+(oJl@Hf9`YSW7uX#r$o@BDfytxzxFM zWAzaFvjfx+CO(Z868m-d&yWk{G7YSRDIU`h&vbHhVRv>NM3{7e+UY!!9e?|Rmb6r5 zRENuz1am4~(BcW!n(uZoY{%S8>&>Lx|I0rA&IBGL&?%J%MVu48V>g=Zyyr5UzSL4= zGcemt4_Dir#nO9FzFn`_2TAdaTz5$B@AnRZd;*fgfqNMY)1%{4dvV;)edELD>Ei(xsEVP9!%LBF=0npr!u!B999Q!zrGJvLva{-Z84sB6Jbqs0Cu&ae3yAuT^gD+mMy6c6Q*GMOD?mfAwV z&;9JJH#8N{mAqL7!YtxW{vpw_^eofV;tA5u%Z?`l*Ky*gF9hdkknLKMja16h=P+tu zubCUQ>Cu8K3ZHrl-$lVREq zb*-gAZH+E44-uzGn_S>n9#P4)%J)xe&8-LQVOE?5R@T;sIZ5ukZ%H&ATP+Xfz(>9H zTV_~L!W@(+NVBn78lYOjhDt%r2e=#HyCr9XObY^u{%;i$qsLX#zW~6_=#umq{_IES zb(DddvNb$o&&9gKaIVQTys`>2G`a3Z@4TEUUayw6(_? z8BigGVxG6ji)A;~N{ys*K)0*x%8P_R-dN-jdcI3i0#KY=e>={qKp=!^ZRgG!6>^-j zj1D%ZJpvYx(~;XWnNaiJ%Qbc}4fuHX+=NA*Y^P2aXy8=A^ThM&x~7F5ZGJ6L2wT4& zJmC6;w{D~6c%1n;*x*gu_GY4T%Zx*vKmp2PeJhd|jiu`|l+}Jq-+LX)&^7yhCr!47 z##4e;t3w4eNvKCNhvB-^!q5hNrpyk8vbG2#7OKEAWseINsc2WI z&pDkp4tyh%XgqJIY|oJylA7@)SWBlytDO<5pX)8 zAugMqi9HRTJWq`!J(*EH;l)nn@4|&eg+j};0d&BdxPD}5&`$X(2&}l`>J^hswS(H{%AJ`&>n%@ zo4n%pkr8QoCYW|3l1n9vyZ%I;Mhy>h-Bc84vAOip>lidkPwsC|n^ULxe|{PvA_=+txV27jtKbIG4rK*(9&?e_icL*e zyN?%ARG9#MsUU^r$dE#%!dxLc2|cR%XEQUHcDBZlIYZ_V&x(bht@6nO=5*ptd0nxM zkmd7sz(h@zGja7+*3fsw-=Ct+LeT<}8}oW&ByNl}W$mDQ%rOrx&wdt~E^*}V3Ua{_+N4KHa(~GxVU!6^ zLIK!=q=YoW6kweMyuM0h9ZUAV27n8vhG7)#rw`z`@lhV-`WV!B6^D&7+5o!TQ;<35 zo0~N-u|4-qlVrO+P;+f*irjf&ujUN+Vj!k+j7CA4d^xCvNTfv12B2H{Bp^Ix^J2bi zYZTe5^-H|iXc>QbIC4GM4-J9{#(MUfbhVx1Ve73~bLoXABHVY-g29=1HA#%pcHHfZRsn z;IgN!!WF1~VO?zcih$m%!5x@#igwtAus^$n$vyL%=D#QS)>ly(H2%EMnOe0y(jB4L z4NJ6i{xz+3`CPi|yxZDOq;=`uoWA`<;Vhw;gt7j4_|41P>dz4XONQk!E z&$4sWsc8upFjj=w;T5&DU{)LeFxLJaFqEb~YEw57mI=wU+z5%JVi2{Nwf}-obM5-` z1$Tf;Vb16B5_Vs37vkY!&~EJMT+)Vda2gFL^`U_GS%YsGen31N4X9N{NZ_=TZB!YL zkT7*>>61d3@riL68O2TSK3DA#vJ;n`ebv+3Dg+E>uH%XEH&Wf|ZJ>|ff-kswbTSH? zQv`DT9%1mgCFx(BZM4SXrWDK-(CRVF)JIN-?lL6xrha_qI263G^z3xhYCQBX1}w&S z4BJ@oW($p@dnZq;UZ?57j{T8tNPFeE`Jjxy%ZJiOa)1-(WK4ByJr?JUZ4LQY+-eU1 zzEnrLA&Fpp*iB-JkVao;H#stSadSXf@+nI-b=XHOI?M+~q~Z+N%&uNytD2k&2S9vh zPMNMwwLd?$6BlfdPvk6Cj`HAAH@aF)H$s@N2Ce+0+}LQsM%~$6349CkUOPQnw8D4a z>-^xzy*^tj(Vy>3_vzCL#71SAaC-#<4k9jBR)7_0E79H3Nb;S~`&18Qn=zj7Ym^;) z7(o`iv{*8qtilIIf8POy0Vd&R?XVGnc0>ezmEA@uV>*D$^ne%FMU3 zl*mnW%T(U>Ny!2L_6x8a;EqD(QpQr;*d;TY>%VpL?}8d>>(YL zY|REB_#cs2ra!R@85U%oY5pSxJV0cG4L~3^ZPxg zyRjD$#%2vem%r z>a1Z5e8!f{7+l`;!(?Jx2^xL<;5XjGF_i|(78f;h-mQhDWq@S8GGwMLJG??zWo3YJ zJyy8C*R(k1U1bsWv!fu4R9q$u?rao-X_xfwPhNYAkJ<>FkZ65F_z>(N#wlQ4M9+R8 zGlNQni?LT75^=tb-$S5e#?^&*UCz*tj*LsH=cOnUA_YeBU&(J^eHV4~7CPY4ePuL4 zbaSCbVXhS9pINDt;gDQ*D?K zz>Ff#t=#Ph?LcQtf)`2984#nEBY}2|SR4WuU2)_2U{E@og1w^O)ZSeo7T7cJTUz>i zJ&dKO4EJ5mVV2t1+?Iml_fv?Tp9SI>fAB9G*9ixc_eI1d<91@UB}Y$turb2pI~(6Z z+ZX+H%1=$@VX}4mqvaih7m0iv_=pIz@{8kb$-aWALJiO1iQx}Rp0#?sIT zbg1TOG_M$iV+&wu)l@pp(67o>m3h@tcvQr*31&;wo6`ql1Y%rljb^%VT2UnS$|(+Y2N zDpN3g&_$}(vtUl7ojz+pUpavClzB{Hw+GeiLEW0?JNEg=geHP3$tmT{G-cAbdvsFG z`bJ((Sx;L3#bR+gUr|M5JQHH2Y&J|^t#mG*hk8CA*v0HNzgNKg3 zVD-{}BC0&!;kTSdC)wy`X)cpcU1Fyld{STNr|2k!nGV23TE^QS9*{j+&$7OjJ-yA4 zg;!Yper=P+XtVrzhLcI#5mT#%OE`%oW*qLQ^UM!79AMaE<(3G4_Hpx1vZe9$At z(e7f`i*m#_6Bq9HTHuH*0Wp9L#)LJsNzGaKE(;(+=Fg?}Sn2@n67TS2t^NWAdy;$I z>cxV^+HcKXE~emsL}2I}Nv#Ah$oK7wljSc7HTqGLl;quIIKY#fp;v4Q+Db@Ul;{LLG9aa3^ zDMI_$LWcS>*G?PR*SQS6V9K_98JQZ(qfrd&F*+hxZe0EG?!)bj1w2?>vlDcW&JOaJ zIl)?F{ZZh1!Rl2UIh&NS;wjZg;9F1*dQ+6DL9^1?Ua9wH&iq@0lZ~yrK0Q`|{3hiI ziWonh%NBejirh-Tt^%pq8Pi7XNDLOLa9T5`9w>}4STv!H<)GT1{{pQLtIGFb97S;* zHL1Q>92R{_vqeE#U?2qZ7kHkdv^~_6>6i+N>S|jvQVw^BD5aw6YwE}ZmZ*&eMm~nj zxqNe6bItP=iC>91!Li(>JQWy<+1G_bsvv0gi&s|tISsiZJn9Z)L>5$3Fq%$MQ&k#` zbECu`c|N0;Tl$c6K@PPVci_jjU@hD;^Mr!D0D2};h-#+|AMwl(Q&j-p3S`N^5jI(V>7=xQ8(G$~^+PjV z7F^Llqsic>igDd2&_>9maEvlw1mgYm`znG4iR=|J)I1k@vMyI(HBYc{x8J=}b-+M(?&$A_wQxPzCGlRaq=4Q?yxD)>#`3T92}=++);;q{eCBe!$+9VS zSG+a}hi3*2DIOUpYv%PVeVy|p#$Q=*7@~VaP47Rh6`owhiu>)J_p)@HGG@7>kZY^| zH}T$zVpQ%rCD8+{v1I)m+Bd`du9plar5=+ORk{G>pdXld^EY%ZGD-pY4{a<$;ZM{E z$36&5@UW3)k3{w<#CVW`|4`MQ14D6RRp_bO$dl$&q)38?q_bmFVKYSFD&1u#+4jhL zSYAN>{n=BGl*#T&*mF7V$f`G+mZFo$%9U7&bHHI*I$j`CJMip9gHbl~bLO!~n2vDO zGvf1*9mY!&_>ZapkLzc)I2JWs?G&V5F#2~Q;x2ki_S{4RmAX^Fs!^_dd4lrXKWDRn zELC`mem^79fj1mz;5sol_m>LAyvo7bppM|sI&@Ovq04(EuSB8NBz0Yk_Zf$mq-R}- z&w|UBsNC&k4kbRK%NYIq>}5#~i6Gm#tJdp3LP1t(0{(q8VKESK=BJ@B5c(*X zGxhtWH8rF!rsup!y(dD&vrRv{Axe20_4~ePWf*qVH$DO$(_)Qx4*|p}Wl|YPyVNoMcQEOge3RJK9Qe1(xW$5j}og zT23QmVu|EeU)pcV4Qz)&S{rZ{muvL(X9fT?EihhAf+-m@MYE_ zobEl+r6n}Eo)1k_(Bc0U=QJy{<-mHmQ$ zslZ zG~i_^s^5#!59Dn14QUf@sDD%Vg22RIoUaOiM zcc^1N5j3=4kXJ*73=KniTL26Z2FG~28T04W!a^sgSF=?oPN5{Fr|4Y5J)oZRYMTN` z3rnF^JszC8YXi`Z)>t~#rqHoHDnOZP;C^q^Dy)$k`c#8fr9p`e{6ndJg5eNBQE$91 zpzakI7R(Wfb_S0g$$}pn68PZGHP)uClmke^#jJXHl*WwXDbL>(sUv6%g8reg6Y<NHYh;N=BtCt#KKx4pW@W;D(%v=@O!j9Y7=$g7YwKsoDqMV9JL7) z1>oy6B4hL#{Y!73GlU`VHUWYD^yX@MO)Q5HuC0z*x;VWA6-b>jreNtRV{(|ADG<0NzIK4&Hj{on-D~WzL zh#F%Fd_;UWWjIfru8?|?)lUTf5l}&_o=OOBTD<$3uCSBVff$gGFE*kApKgSKHQ9y3 ztGTP9gu+%v$o_MdNMJ`*3GATO_c~u}tr$sTsi(nKA`>#~{yDk*{Y_lTP zAGi*qni`Zsei;~20|sq%CR*41WxrLPu-239F1`s(_~JR43b66Du_~4|inf+ut{(bj zXJjW1VO1CCtn9pu1}uHlScU=xaC;gxJjON`51%3I${jKrfDWP|Z#_7_*<1f8M#TeE z{f@BN{khwPsiP#r4baC>oh2!1JVl?cOQw^uU= zY&HDm8(cXp)d~r97&|c zaAOVXGCyG7o#^gxjEmBr(ZJ979*c68m)={xb=>4=_!8tRYsz>v=QIq<(W=F7L%yYC<$(M8XfPSV zpwxO|#VC8NDZ2jv(>LL?6bor-GPoX3!wFY9a)q)|gI>go;v-pqGA*y``pzw_|kdT;eZ+vIYW-)&JF# znJ^$2KSi$NxeCJSXpzk>rdPO)<1YZ=5<;;}LB?gupxxb`Ut#oCN(t@9>ZZzwdxN6l zw%_=R*#zMgQNE@z}0}~W$enkU6H0!Bv#i^O10QUOm5^h(5K^1vp z!IG~zm%EEA#G7`)?8M;}*g81J_+i6H)~!iogCfMF(j2vGWZ{zM#DO$UfCwzMB@g7kH#Tblr?fF%%W$ z!U9GL1O&ne(6G&N0?lquarFBfW_GNH+bbqBO%t~G!a%HV^jU)l!>XLaA3Sn%ApzB2 z?=s*f0H_$glh$M5R|?Ue_q?`vt~t@7y%LO^l=!j9y*i^ z%c~Dh9JW3n<3G?96qg=cH2W$T@&`HV^t6cd1rjS?F8er`c=xVQ^;oBlld~RFxot2O zCsvcmD*4&mN&wQXh7J>)8@Sft0^M7iS7FxebX)2mY*=68x;JP<1gR(%Ahv2mnMn94 z*G0HLbf*{5+^3yBDX=8Ol#?(G9FtWmMAOk~y{=p`_x2KnfGW7e;BlylQapaIwB1px zP+7o>9RSVTffp&Us`u}nCiPT+b~ZV(Na(&XRsbw%9oWWBP~wjBYaf~webF8aS(M)Q0Rx#wro zObLBAM~i%Cr^GDL<(CFpKW4^_81lcTnapmM(ke zOKy6S$!_GgMrcDLFS*e6x2^I!C6jm!@KITX@VA2@7n)p5f{Zgfmf5a^CM)38BZ8{F z0C8b6=x*BIHpaxixoD|aW>$r_S-@*&`YL?u$@WTlgDAMvJ=`8Rq37sV zJf0DvY2BJc^9hcn0mSq_$v3c^X*)$ZC1*h;@C6-LzxD9FW2hWsJ1dNZ*D&Wj_jPCi z#~hG}tReprq{ux3ym!GxBL|%0yQn#52%}8=aF$6OH8Ngh2~Vul!9G((J8Q${OH4|O zJKmlW;m?C9jW+X#&}XU`f|}(Jj#;!?XCd=V=zc(jR^!m6SVSMEt)5E-`3uJ1Oq-hs zD7ogb9cJjg>cIjtE+h`M1)kjZ2(7e^u6xfNxt9C4F%3)|D~0r&5o(=+;z7^W$x7yo zV7_RK2vtmf$*$Dz@o?E1W;ZdUv3NFc5k#}Jg{qS3t=%KbXi*sK9s$fNXa)lZ4R9`& zdq4X%nnXae+;75wE4p2HO%)*sC#aVoY#XW`Yw+hu$dc-jK5gFo|UZJqEJasx&Z6OPo?H#R&M zj`?24$MyRf?J$N#8-dbio!zlL*g00fESpSLa~xQ0V`PydY=8ypU`LPiou-6xLIivR z?&B$8h9B4f&#)&%CX93y4CrZ9p*VEWd_Yk8adhM3M(ck=eo5d3Wx7KefxlP#z^-3r z;yB^jZFcv4ofZCTgnU4tYYv&UA%V!2T%(mJLZaCC!@jzB%NCW6n@@1^iE+*T-LlC|B=gb% zq=N2SF}bCrwY|2s9u_)dgEC`VYqANSpTb&{eJU$0TN=iv>V!+|-;y(^2p8!ZW@pp# z#Sf&#(mJ#(9YEDp5y;33*B@ViA=Wo-`O7+w9q}&t9Sh2$E82NwqB*}-U=Lz$47!J{ zE0ezaRL9Qp{?81zvIfSve3ohDvcfSd{e0thM3-!Ux!Y2&giRjkQTCS3(;&Uk zUMUMGCR>d|%nA_zPYO~eXY=}XmVlj_4~JQCus;`+JmW`Nz2vR4ol!0&?F1 zefEnr0yowF!Rq06Bx?MS_d5A4N%R}*?f1n zi=4B^D`~5aQ{mK|1JMo&j(&e8P_)isM+=-2Z{b?_N{sETdY zwCV)Z*6aS=hW#tllz>yoILev2j~lEf-dgD2-~3+3@H+_Rd}Say6}#&&#NQoLKAKGT z^ZS72&gaz4(hG#@&xV%b7pv73H~!%cN~hbtkxI(aZ}pl|MQPN88a=H&qhUr6n`YTm z7PMRj85dol8zyl6aTH!0dfz2|F^+bT*7K{B!sMk#gOi=5>#8%e+ARm$zLz!9mJ#)s zQnpx}(aGkFo?b7_tzyozuYr>9C)=(w*p@@H=)+M}h8rQR5iL!jtkz6?DiPzb4ULk{ zkxF^}WrBj(vYOs$s9>>>49h1;Kf?MX7UV~}!RyKM1!Mq=csdW;6|ulw)D4jJC6f|5 zdSj$hgI*@twZ^KpNdx!Y*aEj(S+k-_qOJC>+UygRM>jyLqVZIgQ&3^aOND4@^|Ml` z^Ua>kn}fe zpI4#{{G2Jv$wO;)^VsV4A=*VI&be}6AZrqR7!F&Z+gXcdbC6#E!4!?I+(&}=O;n)O zP0}YZ4g4mgHDq{e^c+(*Idndraym1JzE52xdU}Ruu0I-o90+=3J7U@t&7q!(!&C$~ zF2R>1&bpME;mn0W8d~rVE%?rylNquOEN(A+&H?3*B;$62Kh)wzGLf}oh&{^1fm)C)3V${3R8~(DLnCthl-BTdQl%7XN=_!y@-AiP$EC2dkKy=CCBYZ$ zK%0*_JbstwQ&{`I`+| zWi?O9ZSin&%Dm;U*AJ11o&Yi2RHQ(2Ac{H4zWN*`&28J3YI|OhxguU-Dbj9Ovii>( zEnxpN0V~B=99UjEygU6Y>K!9C8~mIYG-%Ukb)^Dmw6vG%2U^8)CH*lXgYtT+rdZ=z&0d)W>)Q`)5pm~h%I{{Hm4G8B-nLU9B<~J*l{tqZU$xsXoPA>0%&I;jM3DiT z(5w^pWbh05M|EV}FYi55gHk7M>}=BcrEe}`|MV2KMy!}V1ZRu+mGi;=i#YlgC7FGu z`XWOWom6t@)}ghlg?lVs7Q~a-iYxcFZTQzr99%_*eVBK@RT8ul>oqPjvu48y5@R_G z!xaX1`@$X)y#z@H@`5CA(7moo1=sB|%U!Dt5|(H*)4SS~_ag6NHcrfq!p!p-oLHoL z$krWyTwm~Lp4NL`gS^%?B4O{gyR<$=3(Qi$-7e+X&h6Sz8RlS!Pti=*lLUFI>x;jb zf25?g%WW12xHn?L4&?oZ;8CY=5{BV$If+Uo>^F#CsMy4)Q&==iX|DQSG603=;eHQZ)W3EHpj+Wi3nR$u-) zOUmRDqCK{*jSY8(!n{50S<1~G^AO%}zBv9+?BS2(bK}K0Z5=Zk z|2sJG-V=xrj=5XacRXPRE7bq|6~#VVTewS43RY+y<7C8i`C;QE;sIE6Gi2~~_BxyK5%m2dG8A1p568KgTosLMqKJ=|7-{G)tk|M5x zLvm%FVDD=l_LPeKzjd^sqBW!XuZwn9K^ zxVE)9cN_Qo3Rm`?sMc-XFX002z>sicsnNFKZOVGNiI)6vxHN)TPSoHyZK~#WezG-2 zSF0%Ejw;o+Fq$2%1>6P!*YwRXnC}WaJl&s&he31gstoEFTzM0h{(`3BwMGQb z5QA0XaAQs!ST(gm3gXf!5E$$vTF>DN0Ox9d1{Hes`Gni4iL%c|*Jp*6Sk(N@gj}g; zpB%pr#O*JpQ6?xs+X!YM;T2Y3CYL^$e@_Y8dDad_#NV`h(0f)m-vPiL?2h~yc`7)g(lGIiZ7Sb&Bqu8dSZra#twuQNl*w9 z_-M?HQJq-_WRXJPaOcl;hV+hZXeXtHQnD6iU30h36kd(_)6E>iWd7V|(VY{-{`4dS zkjIm0bakz7B1id!(*Ywl9N2B1g_x_NS~VG zJj+6TIpUcIYIsIZjEpm1xU7yXV%V_xki?hgxKtexCiw1_sWNM(-#HeVylVCKWjQ00 zhGz0L0_5;o93HNcDE zpAPgR6YpYA5M~rcc9-^Y6UK}fd*zcXujnoh$!gq5yN_sYt{jfs;qUwqz_YAx;1NazW z7Nhz+E=|wxaaUm4nT5w4khUgD`xlG4d>(%20&WNk2tx~-<8|x^b2Y%ok=_BaBgc8` zkbgUw7All&q2&PaD)4(;MOC1}RF;NjjfSrbMGBd$i|Wrhaay$UfF&@qPLd_J4re+N zLjNJX62HLc1;e||&OQV(cRerTnFJ&FOlzU^U(JS4T30>>%q24VYi6{nI+ zyMFoLduq`bj>wR5p~Ucm^Kbd16w7aGI(m?lBh9hI_jG2&xhI{;fqpJ2K)>Er6sZpo zhs?EDdx6kIYU@3UvL?d`hteNAMNK#%YonMK5w99sUKehRbf9KoG59tpSN&?Xhd3YY3urjHsR1jQ!4UKxEf%Kq_6&EDbplvZ*D-e!}ogiSphukWG!IM zUQYbnL=j1p`f6YxE!-V@hU~_d{qiDX<;wzE=%}Z8E?mJRmUF6k$A+$iE!zPqfd)NJ<1P!D5bS@SaYC4vY7=A0dZ55qdo);vcDJB0N?hju*zD8}+jGYwEDYdL4 zo9>G654r#?X8u@vQgMBFT^f3Bb%!@_UL6!#cbq%|zNjDdU9qFO|W(EuL z4i_Qs+Ft~Pj^`fC{}8r55}Y^=7i%eKZ;q9O4dM^Ym4dJu_q2IdB$HeqfhfJ*z^B3{ zGxHL~q^J!^o4auC8l><}S#&xdZ&r0GqKH;rLWZXE!NMOj@$C&#x`>si%d=+T*@HzM z!`G{bT3`dLmfGdxs2&Npl?wV97RO==qm$JaT2nY2tUPa;s3L8A$=9)YCoXnusKv-W z`BGwOQN#uP?1@l)V>ES#48sZ^Fc{Nsq4>Gp^V4CWMp@eLy-=b0-Y69@9Kxx@e zfDZ1Ovv^Km36dIgbWl|f>zf`W)upWlGr8cQfrzC+SDKDKzo`&1QhaL97R z{%jr=!PIMBU{zt{%1y4Fxx` zgFj=Z26Nzg9oRx#>L?Yb0*B&L;n&SjY1#L8Mp=nx>hY$+cZ9fawjze`ey3V@^`?i$ zai@D3q`|X%cLn>FlT-A~xL<`q79oZ9i>heC-?Om?Om#+yfj<7Ma}5FF2?fqx!(M2v=rp2Sv3x{LcJR&13v$g;kj z0t64 z;!;9|wA(i&BZt`6od9F}_yZLOs+j$CUKu`*&a6={p!}m)n~Nf`x|WhYuqXGzR4pSw z(T?^aW!I#Nc7N0)w8za(Vg+WfnNIcQ9TK9=tbk>~HyP4VRIMIlgSTrLIf=u7;OpQBk^HCH=u@n@6?P{HwD~Jw`cL2`dP;91zKC0 zfDZOk8)w2}H3vBb1G>hV12K`T`iUeaQ8IgmrbMfF5PrE1QN5LUF809RVkC;U?R^Kd z4_{S9<0m$PLjhq<4wkWw?)8MP5l@(+jCEb+g5^U(^cx^K&I))8Z<~BwadA0Ev8B^+ zZ_k!a<(nbl=Yh=w_sa#=qgRS@XBKJZ?yJo_t;0EWM+J`qCF$cNVdq7oz#{?Dm@gGC z7b@yL-699AHc{C$ZAPo!{OItlK_xj0oQny+ud!=D)*w`Y2qdet1KctKFQ7(UlTh6^ z6r=!}X>yKbSs_x+F5GHMi3FQx*g`izu9Dy&taxEBI^ zffV*;jm8j-Zwlx#F9piSvH?vso0A35Ck@;os49XGc7Qp~Wex*+S^jYq82@V{izbYSS|F*?d;Qh4-{`8iX44f>tr;K}||{tD9v>im$n zr=O#Gwew~tsJ?2IE@6`oF3e0}4JxrHD^0n1X{%%U>Q_Tv^Nh*_Y=cCdG}{rE8*B~REoyz zR&AgLTO~1j=Gs-#wg=Tu{UWXV<*#^+>fHG5sC_?xKl60RkuJgcd3g3rN-@v#Mh0sCMu}3*0#3Y{{Delff-qEq|T34i*--tnjg=-#<)vZP5d00FD@)DoW={C=5tON zNbT-vfcy}`RF79&5GR6|Ly&WaeS@whP?l&{(1ivW#bktvk{q@@aa!y`!6dau_MV#3 zOla-%h{v6&U;!Y7bnqY}5b|3d5ITjvB0@Ij-XCd z$$BkDr>$yn`DOfB3p=;z{j}@+vn)%u56b(;b;bX;pCl{BP?elE!BW!p(=B?ufDu5l z=DMjfp7-Wc)Q1#Bq7TxJzQ1G zefREq>i5-XVQRHvo1qGAY7~+h{6r$`q+UAjhOlu0ZY66=uaAQ5TY#9AhN%0e>R%)~3n-Nou z^vH^OM*}OwutoKZYi-~$cX)iZZ3I_byVuJ-9@-NPma#~N$eu8+R6^1jYEvga+R)-| zL?kXVl_0_sML&(%h4HU+FJbA;OTsiz!7~1xQZCS#D3cMcBbjpVHGg-Q0|iu#H+$FJ zg$~>@l3>Vup>uY5S=@9QUm4*yvb(*JEJ)je7@V_JRGy!iR-Y=uz@8 zB)J^T>N1K)GFYG}a(n`!rKpKp3I)r=1KaqFSUS9_y#P1zvl#gHY)CB~LNXy&7s{Uw z=;|1G*l60q@}iTTy8;MhGM-{~bAIlXjG2b5+y#O$c9>u;{6LSP*fq${k=WQE0_pta zd^YbcUFPU>9^#TXkB7p70T3$%Ye(mr+~kJR9^PE3K1nybaB0TkbK>B0F)r$8&(#mz-eXn@6vKxz70-}b z_4zmR%u?80pbx7p%nVl+DkQb^NtDDPWGAN(lmX;LC69nJUV*UgWC%G3Fe5;N(OU&1dKdb916 zYYnZKCO@Fg0Uvsm6MZ+C`0W7>Zz$@@8mgF6Z)X7qw&XlgxJtN`mV?1ny59Y^3(uoa zzgj^~;NmVpMt67wlAKnmXIX|3<{CZpjpc$Fk%$AlIaSA^+uZL$+e(hE0b)u*7eg^| z1;6PyJ`_d;)-$Iy(A{x-m`_}o)iaKc-FL}fvDM%A{v}FDv)hv8VpCbx9M)Fl`{jh- z?0fncGw=y&O_*2J0(O}Wf{%Gtw=!#LGHgzyl%WOPEdm{zJIhvlPSsI13J)qx2!S{J z;WS?{&*`Hkl3H4Ls#lv;lx>u?MTs)OARztiZ>S>x@1+-J>bf}b>H zB?#Uq!2kB`Y9w8Rd53-)C~m8xyU-IcMv_Ch=U0MMqy+|<-1*1X|5Lx5=-S3l2faU$ z(Ng~|?DU9x*?FepW~@3&r=c_9RoSyJCn}84ttdWk5S~MUy;ReT1{mqb2ug|q!3uB; zY`sqTufe@v%&#hmz1y;86`igo$sAb_`U~O?I0sT(n87FWwck02_}B zV7Tg`6OJ0fH)KDW)Lw2;`xu7#yNleuE+$*wmquA*{oSq=U=kYzZ0!e+HJrMn9S696 zy6~)HE;Z&=X7i8>E~ig1mea4(X%gs-jVbu)k+!DAs7XPB;kiMjGLIi6p!O?yJKNXdW}xvPnn4zgEX zaInPfix8pRrMmG84nK}ZS#DC};^Y<&BAnQQN^(b~fTsyGF~5aUg%SmWu{SEkLB?zpAwOXD-5#HzbTICB_&c=F?BAvo?c}Mt_mP=>ucV<6C+2EH(HKhU<<|K#) zWq3zkp2j-W4!<-I83bmIgAwW!=^Q^BU~88rvFR123UQP%hT!tybP;hqBW3_gbW#IlO(p|K|MEIUT1hDBn!j>K%6kX?o~{RP8bu; z#HPzSj5<@v*=YML4q`C!vj*n5NDZS(+BGV?XYw4a>>=-Fy zBp)8LOdETHk@5(qyrQAeFX(G|HeU?wJf&K3!#q{{VP07WewjpR8wp~2L0jWzuI%ii z!U;3B3Gzx=O*W0$S9%q_O?3>;5n-Pi$9mg}$l2#8X?@4YgMn!c@%^k$f<{8yzYC;` zad>;SDZwUh?rGBH(H47FZ_(#t&nw-}>2Hzc|dM0(N+ zzZ)2r@s4NEzaJoc%qF1iyCYNK(QhmVP}Ac8Sv*m= z_iVDCF@!-fdMiI^*>zX7{*O5>L_TwW-yo;{96Norq9Edqg00&f=`}C=aXQ!?w5K;r zyd=^p<#c_7qre%L9@6jzpc_Uy2C~wL16U_grVmqgX($Qg*l#agbd|zu?~KvOnbt}=xr1|2D*r-a$rf-rnrYM|ZHUNm0c>=m(FLzkaWOD94Yd?v3qR5gOtGyZ zNx>Y(^KtbrY0Af!I}cVKi+b>FFSqJvI#`Y#3x>5@(IzMEP`7vW=Id?S)S%yIA! z2jr+{t4P_k7{CB|#0xMY5gr-c;-Ik8Q3`_J!XRHbLQ7SC3VF!(8XiE?za~&XJ##5V zI#)9&NA&zXXq9Vrb3|4Ge=f9!+@W#hNGzAb1$=RQvx2`PB0R4$itlG0V;{e!(*wx$ z{G@H_E50J@K?rk?sF=9_j2YTu>l}8?joF}dIuSaOhK*!AZ$%cW#3B2f?$$tTI)NAv z7KqB;0TtElsn$)1EF#9w=a$d=x!(@XY$SL2X4pZXKj*XUHMAK*{B|sqAuWAgsn0}t z>(8@;W?=}Z^nYY;CqohQ)AxFQi`Mp}N=#;5%Eg%{K{|9e^!XopP91x4OwKq}|(r9d=gNRQJe8C?Q(SPH1X&I5bQ1cnd;SSZI$6o*gt zOzTd&3ZcMcE@`mXVnUMs%q`z(e>EI=-WDhk#y1SuVCLR>@GR$;B$CsjEav1K2LKsU zjNY4=j~oznHJB20UhvWnFG0v0BeZ&My{Q8{Rh>>zcqrhhLDCPp{MtHd0z?8hyQ7~A z!hm*w2!5yKUw+(&jWNYYaevj$i3EcZuSBSNd2Bp`r2|TsA{+a>8}~v~7%T+dDsGWw z9W4vNo2$RG8F7J31MgAgoFGwgo7y*kUc=m3?pC&*+%n`gKEvsR@~%8chkla!F8Q8* zlb3>SQ_U3>J2r!am@Wm%vIYqhF|6I=NTxc7Regs&?l-(f+Eqod_q#tqy+@`JLI+(Q z+d`z+dALF!X{c6M2>lY$8NH6FSxd3-Ixx!`lK#pvo%{GLF|ygaTIR35Xxi5?a(%0r>YB;2uI zP4qzWxaSJ)xg!yiAtab6&&%5;Q4J1GDSywg^iTdQ9jnPc?p3;4(8hmdDvCbaRatdu z`9ONwdl7Tu0N)p-3$z?-5!SM;4fHWQ3KvB zx{(&bLY2mAD0Oxaw?#YLh}Zw4^>l8Uo{ANhO=oZ`tK1aNu;uxcrA2m${McI_mj3Ga`XR9XSsk0Yd)gAVuKd9OVuJ}tG$yTJ{&$Xvy z$t}g>3_jS5^GQYkO)VvUBlBuC!8N;fU4m8qSI43ZM6ZZ_@uHRiJg=9?DoPX1xW4r3OV7TkD zfA`O_V&CNr8E#LbqKkpD72}@!7F*pIsNORf73zPi_=__YcbfitJRc%4o5yrbtX}9016c8dj)C0NlekiuC&2Tm@v;osAqpy@%AKuN#RNoeIA)M9q?a0F)B-y9KA}Z7{`IFsWobIRDkM() z96eRO&Ch;zgaKe_4Up@Kis5@!P-`dk||t$LoR`N9!>NG zCvJSRlO=J1!csMgx8nxv0ZpmNQp1#q_nTesYk;e?gAk;L^mg3l$9q20OoyuzR)~t_ znItKy=-TZtQu@klJ+JVo*Rsm)L2gZWv<&4&;nCv``eT(9Svdp3Ct_f%>CZ==wQq+!5`YOd$6!y_?_9*5~%c^!8>%GC$hL)l{ zb$78{b(tX|OuYROlr&VA-{rJaxz1LmwhVn#3P4C zJPk&@xY?rc>IChjY9)^$(Po%fHLvjr9>gqLPc4Xki!1M)(_Cxa!B)}pf#0f8UdJz` z$i*~CeSrhqRArA7AMlDthQ|;b$W>t^1J}oxX&}qrTbFGmP3O!=0;lLrJxL_t-Hx-w z+BhQtIB`^d6n=J5qpp;Zg_W~4GAM5}=Zr%4Jkb`ux`mFZn3Rc_GJ{GKMWUp_5k-TZ z9Z4@~)*mVI+#8)^#2$CgIht022%W!OKe_nOQNjdLSJaY@UUQ!CO97+lJT$a0 z```ucPxPos>3r1eUQ%+rT`r;kTJNa|mRGQu;j}lOxwnuV&}2|Cgj&N<^x^VN48u+C z0pTM8+jorD$zvw@${2bi>3v!`7Lp~9pmO|7y}dRJm7QaX6@SgHHm&j2H2Mu6@om8S zutfo7nX-))8yk1LU+926w{I{X>>>;hoEDY{Nh^v-d1D}>10xiTe#WO?vD@&N!2cq& z9;e|GS|dvEw$}TXvb4+?BsKW8nJEB(ESejew7z%E)BfmY$SvG6P}LyC>>cX3${q6VGVyBNrhkHUU$3AdT38D zN)RWS4L(gWTf)W-)XZ4#&DFP|=_o+n=?l2c1-;JMn%kTnRTd&tBX?d3N$p#)8t$SA zZ-oP`NwmP)Cy7H0rk^~*<5K|1b)`JO6loMS3G^nWKll)UiK3zA3G7_|zUQnkD%j;A zvk)&%qb8j+I4xQzhatLhhB0Gz$h<^iB}8bo{la2qefC~BLnSJZv|>JELJZ<%#*BvV zRmJlCLs(;EY4PN4R&pYjbORqg=XL7F|NM;g_F>H%;3kecE`NT4T*cXQcnK#`^u4PG zZ`JeqpL#+d${mBk-aww3p2l1<=-AydV2RmHJ!BI_4NuUm&)mkOhC)Pk>?-lR%pd%B z-qwo&2h+n5 z@6m;N%I!=3f=!ZBeDV(nl0->#)0uNH&FeY^?+e&S(8cbH>F&ItIE^fws%B^T@Xs`n z@dNoS`3AUtNkz+^S|SdFIJGp7qcQ3pQHN{kKVNI$k9lIITa4HPmWiq# zwOXArIG518TIH?G4Nrq_^ImG1GU7j#IHqDHZxDN6$=miTQ^PY?1FmW#lY%$6rDh}7 zgxC!RkzYDoUL-tU(oq#-mG9$PH6lQkn(_eojntggYu;9GTsI7a=zSoh3FnemlZN?< zT(a@y;DH_nSy3RTyG}9Y)@d&#Z{@5{Fksp_$dEs7H8!cyy*kD#8eXd%IY#h%t)Pz4 zmS7fe-_#$dh@a-IZca1UDi4W(Po&dGuo_d?uhx$4tYAj6B<~xJ;G!8!70kLWIw1D< zf3&tLZ+S)P?~RYQvK+q{ovx~jSz^h3A{&kSbGzLiv^IT0z$e_?4~_VYrB8y>5>F zuVd!E6(H6z-+I`K46?t${V+caC4`T`Z!!?w@Sj#Iu#+gS%9oCdLcq~26ubZdV4r}{ zzA>m(x<3H^I{6y#kbJBQRUpqlq)A=5vLqknOBvOOxN2P$@4nl%f^Zmyn<@2ErBehU z*^VO|_;WmlDUoE3&Src;WBPIrf$gi{>3o9fhm8bDcy>t}3EIf&Da^QSxy0X> zqJ5crK{TM~*eViwLIL4)*{dSoE3fIdh<_y=RC|#54*#4^pxTF#MX9W`fE~4pq~Cgw$U_G&K4^7~ zU#wXe7bCQ@@0dm?7f1tw-hTC}#MEphh$VcA$ti`iJZ|^llqW#^TaL94N|}IgDxT1y ze+B6%73*h9&g(AU?*eKrDbfo(OcGhxMJwsI+4rl!3r;@A@h`;=v_tg(~u zyZXp2j=ykm)Whl`#|NG4l>7vIH4O`CanzGu*jV(`&nz@%J-Zu}G47${wZd9GuU7~XpNB8yx9a#N*o*%Z6zr)RADM^r;UUdkaX4gggC#nEqB!Z= zOBVd}uliN+hZIcS8-4wl*#mMc*jHZK!&2=ZCj!B306I#GKSLdA(b%_Toveb(O) zRg$2xOT@0u5#%y-PUm!13B37au3qiOjeno&dXfDc8T!sub9^W+5Brf z{ne|Jb5`pF%sgz~&68D1AaL#t%ip zFud4v)P!Y8DTACmvEP&-lLK5e2@T`4U)>LHmY#!L&O|8N!$u}PEUpo_dFT(GYkwzJ zOvHpXGEq@t1Bo8QGoa)Ww&YNhkCYMAWD9?~PcI43=6Ff4IUfC3ZAGKt6`jl4io3+K zXeH){tEQoX00!*|yqIRB1A&K11p9JLy2z4S&o zjlD~bSPOJ^m$Neh)gL9=7R6#VH&GLJopn0=diCdgTMmy0i&7nbno!mEl?FAC-?>%vR`?;y*! zfA`G>$Qkpjq8M~FhIf9HwvH@b)UHoB%Br8?fl+}(e}o->aN4$DHyk?|2A&5&ULS$9 z6IpqK^_kvXD=$+`w9U@ozI6BFEIZpq+}`>!i=mhJV16z}ECuxQ8m-kAOEHg2qUgg2 zMcgbBP7EiJK)ZKj#)iV;CxD*-m5V~nFNc`#J|bgIw>+EGE^?Q5**%vAcHQ+NenoK+ zpH3%XpIdUnBcp-XvEwj$pPRbnp7X=QU<15gju*L4EJ5tL#W-Z|omKQ63xhuvi`hkI|H4hQiIcYY11PdU#KE+T~SRL%4?x9a*{t=8`2AHHL`N$P8eQBne>L#gYHXS~&-YON%iPBp!oaEg;q_`{7grWunDY8TxADq%GEF^P*>{r4*>EqO9; zGMa(({hAL%;%Jc6H;U+q=t@O`QYqT*UwvYL&R`*sKo&w9aAbfa7j)?lM;FAWRU$G~ z@TCjZi?W<~4WPkB6W#XLd{njtJ>pg{!^f7;QjoV}@c25pvr{n=V6H@4POqeZaKRH+ z*)yzx@R=A2xGo~cUZKk`M`3(mhx@0086kaqAd4?lk?cSdlBoo+-gL9r<4_ULe5qsqLm2KI5?b3@OoqxMq4#HZ?0 z96Go$>|^qpk*V8gdKD$yw*758q^Q6558s>ty<)%o;veWezdBr-RVDNR0mB-5wN?T1 zvn#vir z1JxX73!& z%4nm!*SR+3^&$0|D$N>u?m(@Kb#Ln~Y)ajk>RUdD;V4NnD^n#PB-N2ttK=`hOYoMZ zL;Bh9$>`itS!1~g8jotELs3Aw1~N+#+p z2bEaEJ|{+u5}H89^ZWD=)*lI679)m_wgL;N8fT2?5ttOcv7+%z6;r2hOaE|YAcJ>I zSmu@{stHBjK$vNPdPh`}jHv<(?k&CsWM?3FwiMnLgWa57r#e5KKoTT}2w~MqdZoe( z=v3GS&b0C7dTQxy`z6hG`nN{265!y=%zgWv#|Nkk@c94)IbGRaTNkommFt_^ zs;EQ&eJS5JOwMR+XlEFzn@;N&Jzn!cFs+(Go%hDO{(lV7ee}MWL38<674n~^YAEd| zagkDM=OHhl64rM+(b1D<%1}@H_%hoZH034^Lb^nMjmIj;~-g12lKmhc{JCFpZ1;Oe3I`-BYZxC6F%Fo4oaTcQyUMx!ZaQ8mLOW~Tz5`+xw)8#R~lf_kDozbZT~ znUglzxo`N7Rx5)WCH1%L4xaj%v@2fKQpJs~pB~0xHy}9F9Av$4kpAb>`$QF;G-(5+ z=DxQ^g(~!sY^TF3Dz)=1BGlaQE0Z&zDm3SY-qTh;BZ9A$qFHhg@4Aiyy%xaj=DTT5 z-H$IgW_kRaWe$WN6jd7TFg-F7^WjxilVnr|>AVU_pI7$_`$C5RIaS1XXT4;wVXkop zkmNLCg;iA_%4nWMuzFlf;~R;-U_U+ME-QVqsdWwrLDjN+r5G_1|JAhlo97Hpu9mK) ze{y{RMyAohlzijKTfC(2fkr{z954CvTlGnqPZ&m&>tk&-TU4hE`M#PsogpM3diE2x>#y?O>G6k87(%x@n`?-7QO`GHS_j+CbL{({JO$iS0 zyMwdxrzjc=N(Vi0Y;kHfWYyb0V`)>*WncFsjM0|*H~zv|->@9-^<$7GEyQA?i)c8K=$H{BLBxdNXU+{9<{w zWsjIS!uO%>Z9fA?7=@+m6~t86*loK#?NeY0t?cD*;T)(Q5=g$=v_g&C^Tm$QH;D$6 zZ4i%>FEz0Ec0M(B9Vr(AY_h6`D7nd36`|bFuR=b?&^zv#R?j+kQPFdAUe8hMBZz$U z3c~jwvpVF@!W<_JC2Dr0bD;SYFq=*7|A4pJ$D=;q34Tl<2UC|TD0e;J2M+ul!cte8 z54q?YoT%00Y(gi=YHyO$gJM-4%KttmG*GfMC|w^S7pOQDph80fcfi@1?Nm&Lq!cpI zYx&@v0q56vbtd7T5Q;OfGtb{b;kI0cVMU2zJ=;!K$I*POaWB?y< z?)onVvtQq)VbS@^naASvEs6GzJ&nJZR@l}Nd*7sa2%lgGbrKXbY`K5_;gzUVUls`9 zET$>ey2Eb@{9q7G5XZIRCj=<=xCm5x3oCtgF$m+effM6Q1)Of%l6NrZXjJMSPPMth zptI3DZDwcrZsE3OAT(fQ48ymZuB1q~v%{|;TTi8X4;3X4tN#tIq3h-nJLvgU?|!Mv zmodMJcOjxzQ5nVBLw;F~=SylG0y2BDj0A!qPGDe|P{0!Y;#?p#_YAiMWRkE~Ph;4M z(Zcj2LlY_@{qgXH+Rr38nnuqLD{<}&KR!_7NzUP>Z!y0R$cee+RdG5u7t^FT#SMW=eWKC6u?^ZTcp>BgTHq_1le`~j(e zq>h;umGGExqyEv)p#V)lvcJtx+SS^(K6?y-pus@d!*GU^(c~b9=<}q(|7A8BXcV)V zCgmbR4ET{~4H|;d<_E&eI)7?Vbp{z%XR>ZVpBm&uy;#JnGlCjEXqp+yWZO-l4Mq6ipNh5 zQ`R&Xv9ge?*7U|kna%e^AAX0^XT)S$m++ya^Vk{XI(%y73bo6SHnkXp=;b$)k(>8E zyKVSjtKmMfI;MH&$lwv^XdDvaY^h72^_Y2GE*>h5pXa*!BFv3Rv15$u5$YWviA(;x z8e@T1Z;$nq@0FBU*>6wVu_2E%akd*#^^pV2;(Uv|fO}xWJd_j6BSGk#ZK!6;9IYQ#f%g_lidQXJ{fs9mLu(bPE)8l|XF$HxIpY(c0H9)_$!?AQ~rvynBIX(+-QmGy5< z-4YRY6`L>VDCkI$EAyu)c92S*H|(N|S^ddwf`Y zoN1!vI1h5=KZ8R@>V$mle~c~na6!@cpZ*5?1c#4~ujhRc#4Xn;3}jd~YK0BIbj*HG zPunW4az|t0GXKs61_g=;hL^bH$J2iLrQ1lew3C6HGwd6erR1)epp#Zz_r8Xx!>^P| zgH2=o!jv4$51`5Hdcbl!;BKmw)J`Yt5LLh#6l~V$Ia@3Pe;9aGF&zkBxX8n6MJ3Ej z{bhU|3yHPpiOjHW>Sn-d$0yICp5C#|1B{5x(r+lbId57>=Y>5cN`LuVq{?03mOWKJ z-Cm2$JLitBe}DLt`WmjeKN?E7gQ)dQ-QPiZvWVpSf;Fgw)ZqwM9SD*D>}oBR$kEua zVb5E@*OyYOC*=LW_w>!vC}@|?8mI`mbuJ;laYoY?#G`Q(REukkUyHF}?*@P?HAWfKf5$58-ItGJw4 z-84m76OQAQCCyL9`RyFS!6n@D5DDjn$VbfOrGeH3-Lz5-q76XHD{&tb4%C4wLI0Q~ z;%YdtaJBPEhh<+g)s5>9!jDTZ zM$5ASmZXHu#O*0mZhIVdr0V5YM{3$(wUB)?zEV8GK3>4QefX3AJDtdRe^PBATV+c= zJWsnwEi`Eap%w2xjABO${V9nnt6fr(R&u4>FFILCQ^P=iIB*SR091_c>iI-?)#XEQ zo_^i(52+*GLN0LwKMQ5gmB#BcQJ8GU(>{)1?lbp+%J8>^PH1A1=BM`DoM@ws$^9xe znz5;o3>>%gFD}h#Gl2y8u6ngb^y`^|S5j}M@R)RD?Fg zLRsC`;B_=LGt6 zn=U&!1qr5$nIQL;+idtS&Bmw-9M7e1rwn5K&&v*g?(mU?H{E0x#q`x7=M)D-DP04` zLFPrUl;zub^(KFdB~!>6tP-x2sc(A7hHPNLIG10(tPOV34ta|-w}$9L*N0dwI@f=& zL|1yA`;q<|&&a{t!97xg9Ixg)7Ob!Nx4}8)^-^f0;LqwNrq$!#z+I&DC`rrrAWH0Fx= z{*^#Nd`=HfCbV~?_4Pc3LoVr0*}xjgKiiJlS~hKHsnh`8mziKVC#XuM-Gt6nxQtyW zeC|;Bzpe6q6xC1IC8ei`fTkm}jHdgQw$<0g1R4v+-O&naVn}VR-Sx}sg~%)t3$uyD zrGUcHvzds#1`jyF#E(_KLL>d z8qClN{n^oN9COW2_o~PkrM8lIWeX3LeCyJ>y;yU4aCMU_d=>>sSPCEYe z!rP!N1Nsj)C?Qb+w{{?LYdxBMDlmr}pi zDf<-9m zD%GAUjH@^BH|wO`IU*S{GoR(No3G57^N|FdNqp3Ap#c;Sd-ZZMkcOmf3bgRcs|fv@ zY}iul*ROAc$~QmBz#q@A?V{xshVL-ChN@)DS_w{p&JMzX5>4&bihnxfT}rAcA`N4d zmDx2)z>3;Sn|jpNSH}i$BuFe=nwgXJMpk2sv3o_i#6f(vekabEk(2RNE)Db}8tHHk z6ZkHO>~H%9f8=8^1*JLy1SKVXqoRjkjzGRW`c5Z~b(oqdDE!I9P)_@<-YfcJT&h_@ z0_4PdE*aCLE`SpmEggnTZyb=mCf?2p@sO&M7m>CRWaJ}&u?PG_u(E7RR?~KDtIUV~ zxo9Ox+Kf~-TNDl>*wtQbN1o|)vGY!$fC%WXWk+RAiE3#_X}F%MBsG*O<_Q;-7*NEs zG0ul%qiSg?o@)Ws(#;GJ{?m1Dir7OIi1yTPJu`5!3Y!^)5?NfDu0!b5PZ-V^P)t1W)yaf25Uc+ushuJbh4FQetd!JZx2PgS~0Pr_rVlQqf<478OL(M`8l1WFnZ z3NTmBvW{i(XUqR6%C^HYJem!}|((8-MrxoR+cUR{rSI5Dh7 z{Y;-*QgJE2WgZS41fK>V(GmEhxM;5G@@RpQl;iOLvczl2){jO!z*5dK67H^X3lUN$ z&fVIX6Jmos(ToJWLTePcfBs<6AT)S6MQg4E&giLr{upotRdrx+M(M}F0Ey<;CzmP^ zhogq@@z@iS3euEH(DDtz9y>_B>KN*ACYO+|;Pa+{_)pL|?%nKVJ|a0Y1)D8xu`;8A zhxx?<{tPIv%a@v)R9R*CnA~%5$(r75a=x*M!d)2AEfo%8>FX6oQ#&*++=O;{E@`mu zolwp%L%CLFGsP;mk!R9J6)*j~n0TzJ|>%eREhCWE7ctP*hoCY(wT6#4lj^qpuV z(X473^vwusOqCo17f>@X$~jzbDv}wy28MXVCI&4PtDHw{{<+YY0#O9~_%GVp_SZ2Y zruZ8MJkRYRRq_)D=$+VYu! z3o(o0fWZfor3<_U{_gjd;}Pe`YIsoVCWzgoxq$CL?8BFed`}je$~}+4R>9+#8Ykcp!wZXvO%hKuQpin*wj%RM&Ycv`72gCRFn9sNaL% z=xTD%=FFp^Ago~4nD`f#CCTSGs?1xdr(rUMfSrv?&A?~3_QIZ5=*{2^(aLnM#}Bzh zygM?3Wjs=4_)lBe^s@Iu=($Khy_%BrKuvcUIY$xFc3W13W~XR=^xc44sIojS`-;uK z7TvQH=4PgVEVdiVBF}I_93_VV*lEJ<-5W~jtWMlK4}@gdkTFW*^=n>qdw45WhcR&@ z&ZCd)d(M!&`{qxE9!czCQ`PF>*7`)lL0Py+1()--Sr|#l1$eH{u6Z%2Nn3LVTxju| zE?wP%-#ZD(al;?|qo#S?#(<%)nLfVmTuR;)YP#rk#anYINxJR>>kyABT;+r}UNJzQ zRDba9zd6yF3Fsh*yV2}9?yp3f@!A`ZRZswh2$XCWQkP!{r9C^@D_Rm`YMD94kN^Mi z;fJi-IcZ*?-YF4$UBl><+xz|JiJAwKuB_fZD~xck>H6%;QM0mpeF-(F=;aV1K)3;C zQxLgps0za7pCXs}elzx+%vUs*DjmZ*2#}t#gXdhnyIMUxMJ05Ej3PL*EuhL<60wQ< zz#i9R)i61FqQ?bRl2lqPqJOljf>t%3^?l4w^j(wPheg|EhBX$W-{PCaRUDNa;$~Wd zgQb~@Hw&MfHqfskPu8X;o;KR<+)JLgRCyHEyx=7C)m8BMeZwDxfaLw9S4Vf(4fb1B zJUa;BtQ!B_WaBsCYZ^M0<_8*KYWmazc12m6dG-R&2{MdZ)`$mQ_gN<3so(d)nHWyj zXCMD*HORc&Fje*{r7ttaqhy~qvSM~@95lo!e&HLAcbb$oM((u0T;0)VDQaqIs8qPY zF!eb{xumJ~c|`5B_rF>ffEKbHSR$UCD-x6DLCegzQ6$V8p0muFUiXd*vns?dMVPgJ z#q5F6R5d2CnD*Lwygur07@81z0w~l>sdZ-o0rX^zr5%}hl)(olbwDphSakORZ;M)% zn52P64y2o+**72l;zV1GsK}la(iFj#HI|YJcPWoIhUNmR9jfnSpwANIV;dv&ziVV^ z8?pAy(zY$C+gQ>~jZNvZPgv{r|7y(>?C&R;B1kLnTYmkKg7yusZo+Hs|L&99cx+#i z)LrN&AL1T0)}@scU(aF+;$-UA9xSoq`$pfbuP*FmO8w=f@Dnv77NBhc8_UwnZXEs# zySZ<#UFRI7fv^-c#yZ30fsV&DQxlxNxFqU{WVf4nBcgPxJLU(6&QI1jv>*e8T{Bha z_B)!T!oCuJG}#hSfsjSsn*BjCO+29ggXRkk#H`toX{z=GDb*sR5DlP|@g~%IR|GU` zH`daI;rnU%L>%`HpW_$Ynh>3?W>h^uqy65u_St*r2g>$JV7Wq>D_S|Qbha5xjP+M1 zrTjiRwctw@aP}g$u~QDFTO~Rtfv@~(D}bJ;*0x3uUM=rWwh}PEm!jl%JgynbFN?#^ekx zhCh*-h&%N{H)p9MG-}n4a)e~G@`VX{S-!hU{q6{QIDi6)>+@$dqGTK=`(6q;T0CMy z#nBI-6%BbEz&$p@b5(x-a5dJzfnY8Exo*HFohyYa^XW{{3`>EIn)v>H^ZWY`(v0_!?c<2+ z*tpfLj=)f7WG~0tjsv{WPSyGl!`JqI@o$i@vHOcCdD!G!ES+7zK?q)~YHN=GrMiJk zhuMJX_4wz#4o*VcL5?X5^Ekl#dBW&F$e}fshix)3@9`)*WNjTMAA7Ce;s$Dm)OS#! zM2I`h)jcpIzim4P88Ujbbe=yZYtV2rJ{O;|9$Qj<>m)f*%R)lMaGZg`bNkR9Y1aw7 z?u7!2c)0}eZ`AdBCP13$XY16iv%!BjZIz%QJdc6Gzp)0RO^(Zh zg{zNOeplEMNF_f{kY4iut8;Yy%pEjXZdljr+TK1gR!n9dKtD>)Kf2Nzl z5)*vXT7@1V-I=}W(k?=Z>7qu9Z9*#2_zzSZa;N)quSOw=rE(vNvx#_XQDh%;lg|Kj z$^x#qtk*0}@Wudu8Nrl9j2Qsz+zAR4cygp=v93Vjs?%%rp?K$7oMQLub^{C`yeWf1 z#trCrBIkB*1XIZ4FE8MGS#jh1e1D_bJ9uMH-Ln@TKIgNs(lfU1CDrxVI^w3F0M^WN zQ$8(syW~9WFcB+tag$7vC8u&bt96a4Vq8?h9YqvmI_ zM@|jM3%C4QCb8 zCDn!qB)?#X>i<&XvkjaR5hotPc>d+hJThU;P z>EC?PqPyyN6DWCWx=7Z_BQnAuzZ<9pP+BJ-bV0dLBQEN2l}tnE9<0eVxU)TVGHWXo z!*BtXj5|(Y%KM6iM~FqNm1c;oVC)(LurCG7v%w4<{}bA(Llf_!kb^Alp@O_okl2-mX z@&AG`nnwjI8)8$jl=UC1TTZRN06h{@0xUm5GlzchZu()qeb2IdZ1IJ1#8GPu*Tih5 z?;Uo>u|LglQP1x@*Z&jBni0uX0zGfcZD}%}r__0*;4xB7o2158tL9q%?HoBYTa7#q zu2{XcVC&dYHd>#5aOqm?L(>yfN=D2Pg=sy=sR&2suD803w|fa6SlGWcVz*et~ti)Yp<6PoO$)finq2;g%gLmL{Ci~?A3z;)1L z>1=zQ)ldsE9It2sM->-GO?+u{R#6yQs#FPp~1q{E`)6ZO4Ia;AMX& zEoaCGV6D3QoVFVgJ`cx4g%e=_nMqb|dY!OU@n%urhKG%!LEAs{OQJCiYwYfg{@5Tk z3s*hCkVD)IoQX{zwrV40On56~2>M;CUi~orbDxrw9JM6?EsF`$OPDAmc)Ae4d#Nq6 zhS|9)$2`LK>Vr%|M{h{ZeFriPCmCl_?}-){*f^@KCv#R6wdZ1W@fR*2$Ov^N;{V-Q z%)gTYBx{}RO^aSHE7Ceu{hqis%gVSnL{HcEK`_VgNcNr0{bJS9yj8IkCbm3VSKp%t ztbHfhO!oS zB_pv_rS(B;m?2<^S2yY|FCq=zXaXkq+|v>dj5tO0j=*4@|LRp#M2gWY?pCRP2+)R@ zV`UyvuVO;Dz33uM{0XC87QGwu zC5N0&O}@mRNGP%N%O{Ir2)v3>jP4!jN~s`;AWWZG{;tx?pbG?POzdFf80xnhHH#u> zS>KJ-KJ5$EB&1XL;|1Hybco+%u!u7eH(U-DqnV)T0>;2S9e}E!p!8LL=qEo-$c-Hj zK%9V&4pRqfYa#=WG|Dh>5t_R)6NP2XYeA7_9Vm{D{#LkF&=1gIOQ{vbnSgq`Bq-++zNE;L_1pnVx?ZXDg86ENF=9m;7BhvjdM0N|Nrnn#{v z0L6IrU5$(}&}B?Ln?u3W3&wmNWqLAPmN_JqFi)J{1Ra4@jr}}Vr!JU1iK8snCUZhP4o;VdR5l~6c2leN#sC@+sgszu6n@wf)0G`$K;oMp5Q%?R(kD#P&n{7IM0S0 zXmG0gwn8{l-`fl=HCX;(_InJxoqp3mVo$r`BzulX8tP5C1I1Qu5YQ>^7-r@Qt#d$` zPhCJ#8XmyL21)Xkb(6M|ya~TBef2+_E?xI(OdinEG?5 zQ*=sKEgBu1_?@lW-9*Qbf0I7AhUf@v51ZiEYdvyfNeEY&yP&4!OR>Yv0}E2r@8mRQ zF%!*X&yrldlY+TxG8Bd;n2!~_27Rn*u{$C?UYT&NToV=lI%BQnbAA&Sj{339V`5Gl zO}BD_}e2shpIHa$sBi()shwXnB2UQ5yOmN=5MGe9r?>} zd^C`;bFLHXZjRH=z){jXo}*3E^nPckU;=7tXr&)-*gX$td>{WR$D@3Gh-H**=p^0+ z;5b}@j8i)g=6-zBn@Hjh10bFRpRS)urZA6=Bv}7${&OreEso76bR%6IgB2A>wBT;q zN_dZ=HBkpXxUUD%UMAzS)&~I7tX`UtH5-B`Q|X+~x(r8!mDA_h#l&>rxtw!&WWWi0 zE#bpp4Tc(oDiPJ5PtYz9IkZPs-fFjQHIg|GuhKISmf@=0x5V@>Y3htUvN2+kZQW|n zAP~Kh+|`Y1sg%gv&e8{GBS=-75hW9%-ObwIB?)2f~FN)*?C`$XWDypi9ScP!eVlWo}dZ0A6 z|8I&T$>Gl4|3`0|PGj~NaK`PI=7ez$`aWOi*XBz2sj#6z=(^eZkyoNC*`+LKp2%31 zVwULm@!t%!3<1g<5tStoUkqB1M#W6X#Md)P-E8V(5nuW)N;1*~sE)Aap9zhmoPjpy z3E0Bvrvb?u0^5t-G)jk`JBhW%BCoC~V^Z4-@&fxVcI6JhNM_o|yX^e^UL=(rBfkW# z#8SnEGrFC$%txBao?j2Td&^_h#Su-C0G(*u6pLnJU2LE%fc-MgLLf(2C`yRuX|7o7r$?-Wbr)ksR()!hCOe zd|)!9rD2v<`qQFc@TvaYzF*_M18DYo7~W6V0Z9b6aU@j&mdV>HYmiU-0v*y@vk=RA z1Rl%MdRLL^FeL+ zzi{^MbL9b&-vnNQzC#_^9R`~SV^Ul!CReBk65^?%m z?hTsW+X{deegB_d+qGFd7EWQ6AC`TNTT2OH3EO(VQv&vY0TbdZ7p!J-6O(^j7k4}x z+ls5Kb)XH`BuPk4%!Y`r#DK@jGh^sZogjy)2I{eQlS8CqS8^XQtk;URo;nrvC|@@kM{%ZOC`VJ zy#K?Lv(PX*0h5rM2MiF89U*e00ydX_yM z!(ob}kC_kWN6r;E%T}C*NIP<6(KDPJThY&?l@{BFgY=_H;FjR<@Kb1m{pw*O1dS(@ zgPVw(nG7h)Q!Y0og*fJSX9_>vz9&Y{brRx=lte{#ekex<*?J6>y6WgEN{K8xQ|Cqgow8(c#l<3<8Yjh| zIcBS|BZ|H9m0nl~|F;M8ZBYh&HZEqvWH1|+2VD)81Ff?m==S&C-=Sj$mWv>@td${KT=_o(P2X3FngaweeI;j_=U)#s*5kUNKnEUZINf8#6fyoKK zt%!v!Lsqbwzm6sqYCO3yYxT{DE-%7rS1OO-XJ7&yG^G@GX+BVG@{=EFUezWulpygu zY6t{P6+UVyRqlv7R#=}twP7wvO<3ih1tI7V+H*%vu%WDGd)#9bGt%aECot`VVPt?F|y4oy6p z%y?rnA!CDkLr0^gMvW8XegG0gpN8@7>S*yx16O&SV&(3f4>?3JmB$n#xiY zkON2oG+Uk8)9E<8e6vh+*)j~WGQ+Ph_y;b?tuD5~NCSpXWAck*xN@p7FHK1wJd-Mb z#o>%}8Nqe3BgkAny7HmUN-h(-VTtEW{E696&1rFR+jZLpFYz zkyuSrH7UL8=~_dLN4K+I)T3{20RRdFdIJ!a5SJln5kb=@EKqaWu@VHx~IQVORN%DJD?)JY3>(#dXlfKTWEl)h)#5 zHkt|(&$`qmBJEm|bce{gl=OdWn>z9RjE*((AlTPH*+6!-a-w0rRSO)BGs8Rna1PF= z2o;-o8f0cLiXIXT&EUbH79-(O?*_z90ocCh%aI+O5l3y2Zqcbc9xMwB9g)r7xceHJ z8TZ>Sxwh2TO~}7Mfh$%3h-b$OFix4iBo4jfF^VZlWYDDa(K1}L2Fi}o_@R}=#Kd=( z$C=513h*F%{9)F(IxG4LeVsTEHDV=hx{U#Gh=7<@QQOsx>3b16ldhx2Q#s7s)@Rx-%di1p}ot;eD`gZ>Al$>@Ow@AF!&1sZ?fDiabPxuN9HXSfr^6Vg*yDHcQSY* zCb?9iZ0?bdwecWg=enlxUKkB16i%i7HK&<}G1S(P1g!M`oFHI8b_}5>KqsetWvJRz ztz+1~2(lW#pRDPndzc+T(5g0Z1sGI_EzOBJa{61m~Tju9V2nzkW;EUFh9I>jK zKpqOOUWeQWcBub1mRlMQykdA;DAvt>w;r*Qaqy)**Z;LaM%VN;pznltQQVCZx(o@y zCHtp0ihbd)gxCVhd40vIqu)t6B|Ag_CmmQcgEbf!TAGTHNZMY1NGPsXc=c8&>Vx(R z?)VON6^M)Cb^aeXNQ!y#ORYT$J5FMu#q%ucr^>42L#3zg`du5X!BQ&=UF3^VrKw5X z9p#G04-7wMA#u?lGBfxudkdVdPPjp`1Gh?hl17GPbshFMZp$BI7vK{731F4Od=}>w zg68yT453?Hm|jGfX5?_lEQB`N`STQT{6Ybk!{mg`+582afq;nI(@Fn>g8fDrvRS;n z@m`jt(4)2F$htj!v~fje+sEsB6L){Y7~Pv*{Q^tcCof$Xo4~VX(cT!0Z@d-j20d^> z3?1E$P#2n}DF2O)R(6IU>VXIvqK-f_r^sSEGsGY#l<qpBM&PkuEwwcf=`sUCetbwvzyG_YosVn*WYm!NH7<^l{e1 z9axVN zb8G-1yEE(aeZNFnK}fQ*!5+YjuNokd2G-rTS}cJ$SDc0w&OybOA;7<;B+i+z3UDW> zbpO=jH-EGAW6R-bMMFkUWr)=5SqZy%k zy{_1w*!Ijar}ASK?VVI@c_JA+{UHIj%K#C(5S&6!<1W!ksiur&d7zI<@};Yz4P&-% zB4RYlnNlR&_LDjMsv+;|B6$lJg@jmKsu#IF50blF|LO{avZ-Qz7seOgrsh7xbhpfj?^wq&IncC9}BC|oa` zB22|zfzaAFXagi8;54P1gt? z_58DQp(%uua6{UN2g%gBg7p+m%hhD0?S^D-Mo0Q2w0K4ygQQe51~;ndpEp?*G zOSMG$8o4b`q3^8`u*EmxTm=!mUvg*Vn+`&Ht7AogGS;kM&1&TfUW94t*w1_9#fe0o z3UDDUVwfS!F4Pg7ur`fpiHSY;Cvw!@`uNj8>_igbs=CmVp9&+a?olG$$g0;+5*d#% z@ua1T^o&7mO`!WBo^MDYL8&l=6%-~PSF;;U8*mN$w)(t>j&GE=UuBJIuMViQmzFP| zrtVf^mmc!crAVecmoKl=X!Mr;`CP4%43bc!ruaUUu5MzC!8d2qghwJ(edC&t>SBrm5C`?>1^g%s+05I-&~rB>uBj< zUz~fZv-no)?pnoshSmJ|u)qt;?cW-210wJki5 zLK`fYIa-@!RIA|-72cs{!qyr%Ku3a@I%oi@oMq$JP$Yl)*=+#yIyS%+<^X{|;PVI#u!)RCd$p?Zx;hq=E+XFqC^c zK=js`m+|&sgnN!b@+@f8W90e?OQz$mvv6GSgs9xDn?xaYf80EWF2ntjKf-C#BvN^* z;!F&Z$PazXKi*@otTXx=xrvoxUgFwKX>|Yhl}Lq6VqHgJx9b4_snoUu zbl?(!%v4<9e+^Q1Uqeo@gY<#pR`a+ObW}_#?VbJlf-?2 zd~Ljx-WW2#1-K18fSXkg{X5RlT&ah73Mhov8^zwq>=9*4s&5}0N&_!#Dojwg=Pe?n zkE&~+bY*dLmx?wMiC&UZ>ZL3ts14_gXwl(e2Gb_*ptIlK0_#}*%T~*|P;pTLQxFG% zOXL6WWJyASptFMecMVKpghrb9M79Qo!~)+vikzmwdwlbV&p>R4 zBAwYC_BJF%QCumsqBuxsp#IngrO-`$7O$FgZDc56i5vDEcqgu6U&@aKJFuXr)7Z_p zIl}kyysCj}$UqZb!UbSZk}bg~teekeNHr*#0NxS0EMu`?^%5FD#5{$hEO1z=C#C;l z)Bwf5!Ac2w3r!<8$w}yk;+=VY3u_I{Otb2V9wcu@hYgUD>{U&a|EKH6HereJ zvi~#Dn#5MPCvnum|Lp&%@4jW8#q2Kbbm!{xs9?I^d*1O3vub zBc_oux=UCy?oFnUTSQ{9(+Bi6f0}4ezGjVij53@svZZwe=%95kdvcX2)G4bgJuayk zTt+PH^{6xWVLY0?+_CMa4moW)EnSb7L&p~Jw@@UBHy9#F0fE8x(03Bs7N}4dLhxwu z!j%@1BBlrgmWKd88V@hE*UA|F7&2Uee#zc?jwKZ9ER8GaANw#QnsuWW&(#<0f?&s@ z>sGPxjRgM2)VrvC)T^c-BHIPa^o@eWq-9~+CH=rmL2Ws8d51Clze;AK`V0#%qu_`@ z{rSlF_6cH1G=G$)<86(iLZ6T=r03jD4;ROz1q#MFNJLmZd(?^t_EN9SO0x;2U6MsE z6WVf0bSTw2G#;BNY&6Hlv~@Ttc$H8aqPx}FZaYrl=1-_k39i& zycVPEWI)-;MP+NpWe%#b61glvW|z8(R|M|s?6nBJywMTC<)$=bL8^!mDqT^k(dt*& zOTm9GbtTAjIJqPrDndbBW6h);I)D}xX}nYIXDRFivsvDV7yrPPjjlE8s^N&F;~juQ z%_}G_?*g26{TXRG$|NkQ{-@NOR4Zy)WV7vmBQ<*J|Wjxe1_*3gb^< znHoMoRvAzS9{jK*+_%RqLUkE;8SeT;=b=#&m_Ko%upKEtni5~?=8jM^WmbYRwGKDU zg%U(y-mEv>!=wT>xF<*xj25!Mo3Ch%36em-_&rw~_19n;eDUW_z z4n7_JxWT@bHu)my)biW~^=_iYESh08ph$|?F5ABx?!iylLwC%;;iXs@d4pRu$S}*) zJZ&~i0S%^Af$rFgQYqYqbmKRTVi>EC9iS|NniSkF%uV-(AWo9ot~a0Y{f%{+*z;5F zVDB9=*x9&;K=2;(XG!?p+ib{MJ+7S~$AQ--+U3j-_Od6F0=R_0t{a@nsi%K6q7>X+ zo9>W2zjY|CWUicT(_n*57Ah$$%n=g-Y#GNiyV3TPgd7ePh03YA&Gro?*dh^w!6MGG z7xaqaj9F znH|MDS#{xjQ>dVwA6)WD{KRWL?X#Eg_%u@2$RdTYRp)zK7;Lj0i4uKL`*O7K9BXpm zyqXV3Gq~d^=->k2Ww{4GXWkTst?w6_m1C^+vGfd7{)>7*F4o-$#sbIZz36a2Ek%dJ zg+;FI*Ut(FTr>QZ6APLRTByM2|17(aRn0Uqp_3D5r)j;jSyzl} zr2X6&sPW@6P*w>V@@{6R?TO^{eNl8e`99IhNT^1)g67i=ioLKss*-!?arXeu+cj)d ztRHIHm*kn!F)?3AWZI}FcH+efBAUi)AI&){FVr8Py>1F(B_&+xl`UlO0-G@PVvk*o zkK@r|aeQ2!4|6$+_?GO+^R$Nu32LI0u=zF@zx+OnMWU=BQ@Q_@ywO_K-*J#BtrQ!x zzyzay-33YXZ;L*H#5e?0BEyYlN#6y_IMo0?`ISSQ01r@JPa0=BU4Y++PP^LN;N}sf z8k(q#Zxnw6T=Nz|(mzN{J?C|Q17}tDT+oszxQu3PGVxK)01%N)NLR4aUIiy7s%XM0 zX2JL(DnE`f-X_Ns-fSmjc^|s-Kbo+uS8(Z0!q%wN+OCs;2tmRITK|ot+EL`PuBy=P zubyELJQ5;v`+HbYkdIZt04Sf5(cPEJ@U!^_OU zp$AE6Uet~k9b2y@BMSO5w|JY2`k-E;SIuWkl6b_bN^C$WJ7A=wtlEL0v`Y#V`^Nt=&aw=JochmMcg6}BK2d- zVV1GHv4qJgUleda$iA>W4RUlrtg^5lcblAgq75u%&PH* zBccyAM+O>C<}lL}I2|4Kq>b3>JaNbvH71r-LrAqiuG=T-n+mvra;QQX)WTw8!HZI#vQ?*M6D&Rbw5;} zb=Kmlg3`wfx7X%yE2tzL2}_h z246TX6jhL@D$;y@FYeU8@mqbvt${$Bg-|~he27^XS1!SsmK!r3Ul5-D!(K}nMYxft zw7Fs(nCa+W$@$=$A{OE@LxA>D^cXx=HdJ7U<-*TFQP-sbc0xmB#;<&`-YZZ!-)tev zqP#b9yFjZ#&9;V1V0DZ}Ge3s=*#+K=04~G5-kSNf(6utfaUl)O=W!SmanM;|$r65I zB!CBA$gPAwqbB3<@go=!X+d`37>}v{RF3)Tt~+12L%iVWOE{67Dn|0lU(87BWNCeFX7lq6;<`m#F8k+wv;klP(MO#UzqN3n+V&Y~?bDj&f zuqTR2C9L{ry8YZxeNLFy4!@T9f~^|ZhLz9Pi>?qgM;-1md_Q;$sb~T?A zMUswXYP@5xx4(6n*B$+Ty>jm#@f4Fkc%9I$Ap;KjtGEk|XIp;eknyMOOXFs%QA$c> z=Au?@c*YLaRw#oV-;UDu`w1N!;q3mj>KXH#K<8P{rd&O~+-34mmbMP*0+&2(+WTe+ z*thOJ&cSDI7)xz)oK|T1K1Xm!kov*0gTs@wlC&sruEo`awNAx5|9mye>qj2g-=+$X z78jB_D5CnBLI4=fzh^7UfB-Olp-sqLXMy;e8>xcnw6Pw*v`$Q@c`}S?FsrSqm3~__ zVK%6Y56W9fZt)!o6?*$(VzQ=yIHC{{Ic>6^Q=g}!q+qo1$l3lt6+WZU^~+iN#IY!l+$L~Ag#J$iDYB= z$-^12Jo4!o>F?d*fiS}>X9^E0=sj(l9{9%;`c0pvY~~+3?Rtbx)mg^BQm{6r_w_}* z1V=j8h|r`C*MD@J$)o&?jM$x3Vo(#}O7DssWOgnHIU3CGfZ~*JhFSq9{6_*9+uI++ zAa7kCN?`AdGtY4}p}RcV;lYyaIeN*SMZRzQXXyCau$g8N4=csl^JkJnb7@?v)SicO zHr4OjZmifZvG2P+zhOo9W2e`FfKpO2iWV5_pE1ziQsxVTYVp2g)MO6l+|i3Nh`i54 z3XCET^|YV;ii{tq_Msf`4-G(Zewus7(^~<;Cm~(Byu!x$np0I9f+v#CZlmb7+P^*1 zH5$HCY;ICoI#634L%jTA@E|ESJ8x8SSLCwGO=OI*okMJC^Jv}7%4#;zZx}k%Ngm6E zJ8h~&e!a+au?KaBww({VDYlSmgZvYEh|JNLBJwdp0rqO5u=in=C7z;zALJy}=0`GQ zRQHzhXqMVh!Sv&Qfty|T6e2tt3i?iWcgDDrrdg7oPri%`^B;TK$j`MtTBiIsWwR?c z>ZU%3J?pXLPlAeJ8IqAY8MV(DYJkJ<0k`B4?p?kKq^IqGYMJNFm-WzJ@)V+$N(j8< z8%D{5I{8AP#sxKm1|WCA9{+u#pX3$|MuEyIV*XTHRsV$+WQuW@Oq#GPBd>IR{eQT0 zkpHj$uL7rC{8yZU_e}sCl@v#7^%{L{JW%_9_sLtG=I|gKjCTc_=*(m$IR9D;v$#HR zTr+B}UrYEUt#{gvFSS95wGQ=W#~JiNI9Z%G?GKVegA3vgQdWf1a6?`cFAf(x01hkN z@)aK-e2#8D^g~iMK}08-82W|n==7yNvrN9b+O(GPPdYjmXhfaT>-VG2*ZgLk;dqfC zp=Oj|N_=vlWT%i7AG)rMoz#mkpRur2r zMF;x)&kIZ37C)W;j!6-x-mkTh5-0!V0KlYrDvHI{M56k4M$?K`+;x6+gOfYh5N#aUO?8y+iLC%Mwe5t$re%41h;z%WS)S^*${WR@hF=;Y*tmO) z{KA1`XHtgyx*=+{@Igk0eKCP?Wi&o|9v3lphr zvc29D@;NPv#xEPtE0k~L^NN<}ZJfqLvoM5A;y;=5 z^s#IxaHLPWe(AXYn=cFdjk&Vh_ZlXy^Cb}FC@VvbQjh8_S>3itFRit-dL>*tA=Hwbm9eVH9P9ISvB(JBQvnF%tt&Uf+MlNDGTVg zbT3Q)aRZ?F)xsq;=0|3f$Nropv%-i`>>Fj_H{c8^Ho9BKgcigRJT8m%3#)Fu@#2!2 z`2ctG8L9oc(4XLT264?jS!^rloFO<7dw{7Pfw%{a^l7CvX(IJ~kjgyVFPa0GPq}~= z3EHzU#8xk1pS<0>v2Ha$`+~~I3Xtn&FuZy^#9J1VRFu61)Q5=wZU$A(nV(+%SZEDq z8TLIwHOS7=iD)|RsLiP+izG&1?9i^fWFu};E4w&w?k;R9Bgk7F3M-W83-j0yB-N{S}myncU-$sWkf<46iw+HJxq>vArWmXZQJ61^m z>AH-;d8*+?jlJ54#KlIu>@B|JXP1Rfkd?M%%m$B4@wgg7(PN#pB zNk#mx_FC_h-np6fE)h-cS4U=_@pC`M>9*QB27}zllHB>!aG@aEN%`)XtVk}_aj6os4&7R#Kk0*aIR!hu=% zMx!wsJb7;-;o&qPoEv&0IcfY>-5=YLuD4yLg+LHczEuVx>R_zjyIowX@MBd}Oe_@^ zmMhUx;RnPO-=4C7*Z-_qC;X)Fz|d5D-*bZ={&YpF9$%e$vAbH0Vdj{_%hDC%2Jx+V z7w~^!NbpD{+3C>I13^A^Y|*sc@hKQ*wKVbJxg;DnsiCMWaKg|d!9d1*#~!h4SYxPB zL?jCbf74b)E%UKS5j+xiqOM=w{Q4o)$)Qn+srtRMNXI=@_axstFE8P|tE#n1Ha&~o zsCO!p*{m=kc#lUuJT5#b7O0-!nEa2l5d-0hZ~UM`t4KmFS6vNfEZO~A1|#tU`67-- zl-(5$2t>aZB!Km2qUkH@oOVdf20_iAnMIm^cXo)*)5U9uBc(5ew}z3gU4~$Du+^Qf z)V76&7DO5jf8cADWYaO zJo8CP2_%Hw5Q|kwUD8@?xB9@-v9r~6uxU{5| z5juNj&!uwz{B1(8_-BT_Lu`>r&RcycQk$FqSVDrukLhmPUd7P~p2f}8lNP;1*q-aO zE(fX`f{SADx~kOV*LhQ7`eiX_zTsZp9p>b&u!NQR=c+!ioo{6lN~B;Zn%Q=Dj~nDi zR=6)ku2k*6W*@e{ZwTG%t@$L$Y)KakPmf#}A&mrN+S_1tLv_9FS;!mq_{0Bpxb?b0 z4L~NHa0gb}xz-bSBrWtE_pt47W{A=?jRg#q<@T!X&*pc(*%KJ|% zVcglkOFN*v;FFdd16M)(HQuzHiN6lx*Y?PP>uw4rpTcKeC;foa+>>PzeJDTjqXX() z(AX`a>z$+X;wn2E4=>wLOkXDYP>JuQrHaE06!&3H%U;NdW}Bu#4=J(c~*)#agPvEU#)iWxfkR z=2^ktTF%Je=CY`(4fT0MW6=G|f(tp(tXyo}qwi7`AS!dtn($t^O7B2eL6Z5~(F5{8 zdZEd6BW_=EV_0XJKVhYDM?QgRE9)b|X4Ox#9lh}R8CpwO2n} z5P_y@jcg7smUbIgt+UsrQBgccz++^Z&eBAo)8p$bbXl@Zvk-()ksq=%mHk0xE?)&j zl>e!(PXl+$xu291pj<6Zw6pN{X5T62hgHlc$;lox**0HuIy8h>0BU%7VIW=1zO`5r zCPW;H%I9sl-kzSxyHXyXpRth!#bvy!r*@K@^0jmo-d1uQ4+oD@A3KUuSmi(jl4O2# zBI4lbt@h}PC$`PJL3^j)^JS`9yDWEAc4WkQtcK|NYjrqL`ZnAMEpm0aO&PL)?rO@dhq3?5J_^TvqHi^Ha|(ED9KE zi8mq!9V7SE-%sIX%%qK=CkkJ1N^2LIVP6TR`ms%hZMA)L%W%uEl_8R>GMa9&OY*Ap zpp5{bQ2R7F@C0y3d{}J=K;i9@%`N}1Da}Kux^HRr#Jwy3Hl>&o^XB%u=eF&s5>Wq+ z7XThpMf1S?HJn@x2-L|_<-Y%QALF}gVjiQhz}5%GjHL{XtprYBGh&uv#lo*y$Sk{6VH-IH_ksFB%) z`ieo*ZxPE{q5rt)XuAlV0+%!(WmpS|odeJ{4o0Y)G|Jt%&x%4d@Caf!d>INKkVX?@ zwVRJ6YD`)o$jTg!$EvtaB)#4Q9x$Vek3bS_`$GDI&Rdh-QPw{DxVh%2#6o|p)>7jO z55S=wp7!nCdTT9{&t7^1ua7e5Bn9cWNG<|%96#D6w5F@1(S4`&XwESNd?OHASn6(QUkscg3V&ggt1_GCNqL-7kad z;Y*jPnuG{YY#w-^S<2*?O0WM^!)$#kQmIE}vBymDZJ?PNb#(;goXcHm5D_*9ie<%U zpk1`DBTBiRtQc7!;t9xdo>;Zj{0_2}Q^9<;-fuk#{vqS2>n3$I`m!()NGv*Jqf5ue zHF>F0x0$+w!CyX`hI7dv*2i z@K?p@4rYe(-0ghB=CzZ(+vsYzb}ShJQ7aA;LE)>O=pDN6+=uS@u_S}ahkG86F*eZ* z-wHI9!ZJV&>~==G(16b*g1(t-?h3myFXrC1%hRz-u^I0=v@TUjoou+2exQzN^l z^2kh90)dI#vVu6NDNR$fog`5qbf#bRg1xJcl$dPtYOJ0W zg(W1)*PQe#N|YCea{Da8*z%uR-&ibwJr;WSB;a%C%SXuhj1Lx1*KXI_q5#Z-hm7l< zNVzPsO*(%dgAQzy1N$&SSb#@Myrxp2>^r)YcW{O6`j+|x%Opw|ClnisiLF+4E(Gdj z5MgpN)Fe|<4rIl5A~{oMYCj0<79{fGfa-eAvXPgSdY3&WL^-(hYEoK5CfYF})nY(1gua_}KMt@X}`>gZqFL5&5l(tvssO>aH0|pwq{zPHm>`*h?~c=Dq}Z zX}Mjhc5~ZHbZWJO@p8 z7(&VuWM)_`_MeIE(4myRpi)0scplo2*ZJn)d8NeDi=(ewl zcf3ahSN9P}2qS?-#Uev@JYwM&LDIpC_mg8YqEhC=>dx1Hu)iqkbnjV3x58Ft%1x(; zUe1W$`c|AL=BYs&D<=te7e5j61njoO;(Xz31GYI;WiaN#)ad&-V|}}ay6@R@9((e| zbz4wxrYSNA%>WVQ1D7vr$>rn^2m6<|fs&MLI7|XV)=)|)@(SzKV4VM-8Sp1jQrFAo zr$Axvk|sXQ53wj+^I9no6@D>^gYZ=3Mky%YR(U(&&2_lc>^3!<`2y=Pnx!Y`=FIw? zCAX-k!XCKCpx~C&Zms8B5+E45yd2%F}FL{FzRPkI* z1JLjS)71fnIT22&JUg;d`KP64;Y%bVQM^G$fH+rK!IC<0fKF$#FEGbJ()1XFzQLys z_C!fK0oz?2A{b0`83E4*;c(P03DRm5Ix6S{MVV5IU!QY*t5tqtVYYTV$I+RC?1$HF zm;Ziv2t9d)^Mh^i8T|dlc zBm?%dE)P^D!(9DhTa_5sV9}T3NKt0?g9H}zhJ{}*mIL+x`B=VlDxd9v`c&JdJglV@ ziDTfnNqy~34q{&pC?%i>56Wo_;J>2IrZ}AE0VU~_yamh#URlpn{sdM$=DxCXfp4iZ zLAc~iMsx+ag9$28Y?jFxBMj@*!x&ar^sc{I;ftnPu>=!S!MDUmB(zQ=?iX|P=6w@N z83@`1i~>}~VJ|$iL^l<(M8Gmx=hUwh<-_p#6qlNtBYU+ABha8il?G5A7GqLh z91}znJb&Rz4N7yDi4^5p=gUac%m{g4tKSf3Di@X)i zuT9*`B_fDG_5)=TXiHAAAn8!@=GWw3FKd8i`RHW5kiJmC=XUU=OB$ZLZ$VP?8efrK z?bVz~jgJmCe16S^%oPv#b+RbNl_%xq){7i@a9n`-10p%DEpluX>%<+~xL+G~5m1EL zj?lwF7eDOV9{uss?}`Za;%&TrQGz^G!e_3=?7zi#on~cKg_0@h6~uf^!&8ROeQ@LDc2hmuW>1Pk zi0cxNOO|4w9zs!A42ccE<0`0=FLGcqv+v269;!vLa*l>IS;|fQ8O~P+A)IrWhR~$M=XY{<-&Zq@9=luBS|mM746R)_p!1E znH)nh0hA&Mt-A^wwvO;hS-)#OifUQ18}h~Gy@zyIgSY!mRDcwp2Mq`3^A7(2G3YcoJg#S{SZYBE3$Ladcpv+jEEGBht$F%zL1RdQPvHT8R zf4oyl?@?DMI`nax0fycZ?qO9-3nzq}0zxW(809x9!pLa;N_y4i{WIb|o}z{;#8%bZ zO}({}9R;EGF{Y4FyfRfyfj$z6>O=!vZg!dM;Ko8;CLUM6)+;WM+Ly~IJ~BZuwf$00 z2U4ph34%R%lXk!4E3@LB-SPE$$&tneSZQaICTys(x^hu@sdsy@hzc^?YHWCKBt(QZ z{jHHvL%kqv96)11iP3wQxZ^pVaa%Q`<&Dc^bXJG$=#$293?G|SA$8_8In%hhwQVzq zVR>7JvsMjA@p$%QrCNtG5Z+y6iYS&Yv*OF>o@sWfdFhS}6xtu?h_@EK3KehgP95t3 zA4yrIg=R{-t*kwXg0bchFGe~g3kMPr&kt4;+gr+NH0jPTxtm9jHojH|w9t>2gfUfU zw*|I`ekMY2B+(4f7=E=}780c?5F#+PuYk7cD%a4(x<6m*B-5cDwCVKY6wnZBQ-tSb zLf&L1Oa^_pj-MgYa-3u)LesY&^_;5~;-piczc-M23AFElwrTE&;w27k%7KmSgEw3N z^;h$)kXj!t9Md1Oz?2BSxMi|d=K+({s@J#UZDR&x7xgRz z@7asKI%LDhd{drUmaZMBO_PWt^q(Q5sEd|{u<7(=%{42KYU;3MLm;vnuu}?tG7Kfq zWQ%$>@tN6ZlS#@Tqbs33(KoOEs9{k8&+7`ts6*m1Y*Sfqx&^w_8F=PXPAvx#UyT^y zkP%9;mREJv`?8Tud8E81-E;m;L_{NrYey4HVZq9{VOB0!y&9udN%d6>iVE|-%IK9} ziPGO~jn#(@Vskg2S1K{Ua)iqxs^)Aea!uk5%x{ToBaS))U`^j2C*_m#IT?%!YZCs#)Vot z1Kyrg!xDCWYYK#Vj7u@*Zgv$*Z#F5fZt)xhH6*o$Ks9h79x8~Bd!4%^zk z|H9QA=>+?F=Y5`$=u&G!uWw z^sq!fQLrTJ^;<5MT1_NJIFn<^rY_1o7u_(oZC@_>&a`2{BU(5wYOitui#S+<=drKQ$!<#Jfv6 zmz^&707E1*e(&}0qX-ePYjK!%*f_yAC%*3h%&#{a%vGF-JZA6oT3+Vq9Bdb@Y#oN~ zcM-AMt#D%aP)Wb1tt3zHcNHYPpBrYH&rZQ&Z1A8V8frHaA%uJEC>vVAK|79~Rix}_ zGBYp_4{Z_$qHw^K%fWfD2`RpTS6=9yF)|N`;<^^d?kz(MQv~#Pfqf!eU#%NlY<04G zyMY|L_Nty1lZ8ZC8_HS1r*>XAh*t&kWc2`tuJ|N|_(9lA+d%kT zHBeZmct^PL$NvX4kPx@SE7N0{mE7gTspJd7a@^WVRL%g9Vt0Q!Y@pWxbuHY_hs?e) z-e1nuvP}W*Jds*N2UAB!OeDcE+08d^W9R^cRQ>Wd@=a~oS_7Q^@E13S4!__Wtzzby za-Nl#T)^Iy${H;e5#BhE(V3f-pg+g(kYMk$h`Lc6=#~|EO!gX0eD(>F-K@2a#O!3- zT1_v2$Cr|=`XmD5CoM5}^}{Ap5pG{!yhAWo_ssbo=B+Hp)*;F~{ms$jo5*5rP)cGS z|C}gmLF>L*xRr3TjD#nlBx|N01EF2_$4t*DCq8`6na62DWdIBIo9tnC;O@KSIN*m6 za%6RGjguuPCsk|*tx=PDKH)meuy2ks=eZa)F@biqs$xwk%n%zyR{HDa;Hj-5g&9|C zEi3WQNRiYP6HI>FSh%j!FnelI+KCmwvZX`WUPF2`xk`3Ef*8D~6rBlnUu(fREUTqD zM=He5W#0Cw0M97+oW_&g)kj!%tc{u_wlV$dKM0vaNTygSC1G8?M~^wyAQU)#;Ip{H z>Hy-rSHsErctGa`M{tBG5;2g55$7!wY4+1TLCMmo3j*#@ z)$xI5huj!Cs(l(h-IZ{Hi>hFs^9G`!Ycs4 z_eeqV1}glQQzaegksM@kYA#$r-Okl|7lp!V!R<*zSEZmNf26vIyiAY#9A$1l`1m$d(Jds>tQg)1A@DHMVk6^6;>_%F5Z;Xf?Lw zaPFnf)yv3cWe;3Y#NaqiqwzqeG2|&O_k7sP{Ag!CYSv{_1R~S1tljawEoC)S{4z1! zoR`L!upxr?%{p1pQN2JX zwS2=R=IB58?naY=SjFP>rP}J@QRORuqtX%1-wvb&B}pw-)?_)t&*# z|7BH9OPlI`b=BGmp#w$QK?B0`K-XT-E4#CjsW|~?T(JvLI;AVJ@^AZGu=LEz?N4J@ z*O8~A!Fi$W-amh1I2ro}In+}V{X_qoaFCI<&pfiHWL(mtJphC`QG6jM`_=RF{-x!RJ*3(*Rlm3cnVb~`eF@HMR(%B&XUbD6BZSt83z3DGaA#hVO0=;! z=~yd(Ahv@4SI{-TM){ay{n8p^xzfaP1=l}Q~2tnk5mM%!RyV!_tL+McIr zB7*lKbc@QN*`A1qmoOMlvb_Lck*{6Ftd1Av0%l=+Z_bBoFZ)lZvPv|DJJ3rKkV?>i zQ&w8w)loaO=28Chx8#M$zsa~qK^q3|n0qzXH4Y{F;7T=qwYH6JF_b|^I>lOE1S-!a&{aSH|mP7Y#$ z(@r?puezp%$=)*NGW3m_YbrJCB~&ef6_9sYGaW!=-%0`!2G7 zrNakb+@+m%i~QF`e6fVWLR?>StnjY5v2{<|4V!g5XBDk45-&$;>pglYem_lqgWQC% z>iP=@V`89G=!h`Gl*fP_($__0d>WV-Nx18UC+mAC@=GeQ-YGj{3mRq7mlACuuN$qA zJW-L~hxtOT8}gsZ9}JalD8Tn$eBWiRVZo6(!7Gci4g**L>jCh0&+ICKWBZJW=`tsKk^SYFM6oyLjoas>P~Cb(}R6A}SWa1O=5| z+@4t6`zyXsB9aEC8bSs0oTu5ztMWD7k^|Tyv78IL7Bet$S*-_OAv6g~o^ij;H?0Rr z!dS&4EF2O%bY>E=}zbyzfgZ9j<<+`1bPoi!0J8PGQBoDwW4YFKxt0TY^=-ja- z_#b5pA#wl?a2a279!qdvw z?m+qUHZ{{B%e!TM{o@HDQjECZ^6f>vkXrBZ$Ra%g&d=TdS>@)DHHfgRD161bX_}8% zX2CT>*-U?A0%lt`OQI@9cS=m<7JbiegErtv+RNXw26|8DbR(jH9eiUDVMRIRm*L!A zk!tN@;9mpCpX8BUo_rHv*7aB?#@YM*@gHrGs@!?BIb#*Tu8g#|FoQ(*q<2Vpaa`$Q z+KQw&07%aq9&RZzEgyG#WcXg#m%Ut#0E-p?Tc%l{%uc$-iM`}f6d~d+WS>QA*yrUg zcebATp}$QebK43$w;vA;3u2dHZmo!x9|vw{TGhN%|C7;cI9&h9Pq?>=HG1^_{O^blPv)^(2Tc zJ2H`qgJ-72xW}{VHN}|1r^IN(ZgTuAbSFAiOug-3`8?`>ucwiiS~B{ny?lc_YQ8&P zJeM3Oiv4tw)m#>!55Fm9lVScxD>}*24emsuy7fvw{tKm;9*G-QYRZ@k*7PQwZ;VCC zIu~>x0_hS5rnHA~avE%jKBD$L12es{!g2EA?fd?kC_HBggbOWPbcd?(g`kSRZgObT7n~#N3Gl^15aWHVDNb4aU$+e|5vzAlcrL$r* zXb!oNd=2ufX7PX-NAtqR3DDzoQ)^5m2M;03#+XATiX0M;{oEA*Zh!>|8n{x3z|WYbBnG?KO0Za;W#|89wc7b7+3(W z4ksg%AktzK{|Pk?M$7l<)*}BDK;K&kS}{^wv@sezrMLSQ=@+=pf`EGvq&|#voHjCa4KOs(u z%nBj%B3_$Uu|~8^ERA>1nTsFi%O2Nvi(00m#@eP_Ylk9mDBcm+C5yi=d4(5d;k$2 zT>oElm)Q^Obz1S};dF-1!FH`GW3jZ<2avf>!s#NZjo4jHiwdmC`M#T*ZHVO8`J3G@ z(e|9Sn{D8!F6Zh%X;%XCf=2&)dKe8yxqzFKfRqOoK6gHOjIs95Tx}KK$T8T>-%YFg zJxCU?w|pj(KL7*WPQ1@gQpQ3Z4>dp6hN%2QSRSIWa3CJ#tvA<@HVmYL`}qFaxvj~# z>zFHz%$@Z`=1plQbNB z(!@Zh{S<4u%ZBj!tx^R0ViK8>J0T5Wp+x8SgzTNAe~SBE&{gDd-$|7zRJlQf04{Si zYN~#?4h&FSIatSNd$XIdNWm2!DG1Xj*(B*+>D-WN!m=V=*nn#w@L?ksqyn@;y)f@R zR9&lUY+sv9Umc$>N;WAf?y;>#d#$OFhG<5`Y`h#>;)wM>2VbjyGr>%06-TzFwWuyK zXmQW`+zfy4m?y?Oh|6Zx3UTvH2gjO*$hn-_Cp6o3q|Qt&){92QNL!X57Nc1lpC1h> zobEeATOyIxHRIb8W>bNF3d;^azP$o-qjw3QQ)tdGl1+@MQa7#Ccz3P~xR?6i zi+CA;ff-n8L_CDmn!>-kMsq0AHBKNd=S?_hg8k4_IZE^}d!!aIjlK*!lq4t`L8}cw zZb(r(TbZ_XuGt->WNV{V)g_7D$HmAAQK&h|MPfn^o;oD-TU=qkA7swW<&&0zd=bzA zC5>T@dpI4YYCUQ^z6Wxa#|mvhz1yp3e-Hb8C9l=T+|Esd6Ku48PK)R zcO9R}XWWVb8-8LF3hzfpYPJosr-`^2Vg;V8tmSSNU8&E}o94E}nD_|{`=0cS?*o>; zL;S#=2ksLwI9^l~Yd7?_7)&dO^022ge$8cETA#JYas&}Vc0uC*4wUt+gnFyWIes85 z%0DdIp3tJ9JXd&wFHB86r&?fKF|vWg#_D}-1Jzp8vjUKYPH#6Ts?&D~mtl?2gdI&(0D>Vi^q8Z`0P zd09}D8tx zNju>C?$y8PT@sbz{Q~st)|P9dI*W`YSQipA`uK=5k)jg#EBzYDbAIJ}{}Ap!o$VLD z9=-Wrltemo`*o1OFoOO|(&`y$Lb(4;56RDIyTCX6Wi-~>D23+~ zh}3f$hUga$iaKRXGrj4H_ypoa93FDB4o%x)x&WIh^u(F!V0y)T1p4XnyoWfv47WYc zgVJ}fb)?+~`d?yu;tNU;l`yIGXK_0(JoKp96F~A}3uI8fq=7{Hdq6Lp^IPZ15?G-` zQB&QS3tE(b9M`K?4ztx;h3?iXNa#LzSltbj!kq#jtxUN3F<*yg;LLr&4OVpb8AN_-QkMY|&%decP!7!UGxW;a>fJ4}5YHUFY=ZkL3J!?jYjs|4;x^F^{iDgL zzUl{*k19*|;hw2TbgzF%3z6&!f0tPe-5Pp-elRnaM0LKsD@Eav9m*JmYgR#MrjE@$v-0jd7V2c?j7n)vzf2!VwYVfZy;2K^RF)S90 zHx?zA!pBvi0`IY4tt)v0Rrn_Q8T>G1tObOSUSImrawjX zOq6s92-K`~5HI>5kBm<|uukat7G0>4<&(c+TvzIfg@C2$n4#lfOcu8*gU8Ng3-J?@z|l-R>`yV4lWT6nTM}O=uD1BoHL0IDY#7 zkMHfy8^7Xd*tktPheH3hB~Qwom}Ltqg~&#JW0*vCsGA$UF=%fvo)W6#W9TB!Y+*zD z-_?fLkG7?nyKx>9tdJ;z_w?q393pP!5*C<9uKW!S8>SlH>}5K{HNe@;^HK z_*jKQbGhce&T>Nj#mL(}9-*MjoBEy~UBc~($0y6kII;lHciV(`rz2#ev+=RJc@NMK z)5@-+w2~pj$AwhSp+|uI3oj6Hsw1H)VYHV2>gLJd^qBRa;XY@yjC{5f(&oQMiYj!h z@+WW}1|_6u@9lFb~!?cl&S`X`=3TuQb}0ZAu+#V*=FTJ>rAN|Ax~1UQc}g z5}EQT$Mi?#?kdwVW|#c+-HCC}_X=LZ$6L&=6a+_!lF`RpL7 zU1)94d(mJFpX0}O=#{5E7L`b$t~jhLm8Odsrzcwv4hu_&SFHxa@OWkmFhkyHu)#H5 zTpRgs$A$YqDf*W0|J(L+FD7lZcVTCX4TTO_1KQMsE)6L7VH-RSE+A3yGT_OJt>6lLTSOV2k=M$jf=?>P|5VnN{ZfZR z0`k(daz^8*prXB=xEZsXJ>0O3C9eTqXoI1d{IdNHI9Hk5P6iAtmv#FcxLzVnkWX^k zW~}4?dDWHiA0!$~gncrYT>{Z_T6Gn>4bZGOTZn_PSTh-izy;0UgR$LYD+bj0o#mSu z4Q)CK)=fjEvT4!R2N`j8UPCnZ_UlUn99-4F-jI@)tqN!Ki7W7*#Fg;DM{M(viPoyA z2C%C*;>nKv(elyVBgL_$0?3%7@#!6nF{%F2BqG!5WKaOix^!`}aE3NwFF}veE`}hznrmnu z7^5^>a74nwXA?$}94Wi_QSUDL-!>UH;xNRY3A3eES$Vf%z z={p-RbCa^;{KD*3(7M6?r9ULnEvpd<+cBxYI{a06?YC=j4Z_mi%%r9(2F}#CK8fh- zvD8S@nw1DBqwHPgQ#Rcd`wB_pl0H)LY8nIwcho7=1Ez^Wf{5#fQB#K@*lqyb4%{4y zOfEwQ92b4B2;;ZQ;-D97RG@GTwesK<)8BYb7M*M1lm4F9ppA+I{mX}h#Kp!ywC(%( z3{~}6k1x*$&hy{cSZgsTW;AeJOpGLHV>NOp5Wjk=1#2ATtsa%pCgyx+t{X{qjjixT zHqBx*wQ-J+ZDEfTVaQeQ+><3=n@iYNQV z7d7j%`+-&xKIwcmxu6GQE%_ zVOvUbYi(}JAwIWR9SS!M<*<1|bRJVQJ2upUP2^%9JDN+K%NquUVu?0SFhIw?9cQbU zK&v^;4EafZ>+46&V1@go_Lu=_)+oLy(x^a5yUN~IkkOtT)AoNJ&?s7Ygn0V{hME`0JGQELbT0bxzJBf2`a#&||B z;vU;yU7W)?^dkbyUN?sj(Amw@R6lkzy?hL^i2kdo&W;obfqL;K9YFn{PjaBnay!7q z+DHWjzGO2;@OGIu_8&qal}70oc%5oPV2S9QIW(-O06BtwwAkll-9nl^ag#K(}FIhtPsTijH`(qQEcR#MER{pi;QY*Y$d$c|` zA<&`B7ksh#TAOZ`+quz8ZfB}nCP0qxGFk;9xru6dUhE^wn)zZ%jOEryP+<-54mN{3 z2>OX%3tT3*@k6o$!1Dg`*E6L-4Kb+ehtt{E4D+KeQe})mgI+TFq~D}Vyje7{R-_LP zIql+WoJ_>q-;31fVB&N=XRMT10~hM=PZu1Psr`VGyTGVffp8SUkMi0&tcKwAZ7a)| z5~30AAcyjS|B)HffchX(oofRUlVl?T-Pk7BG_%^Q0MTS=R5nA}I9~@${Cfelq*?EPYZY&QB-DC7 zwmM5-5s8dLx{ygEs4}+fJOo|n8-t0`025Ci@8LyoV2{K)b)!^2_D~b!MEuC|X;{G4 z!`BUY8*Fh-x9};Q&?8)Yx^Tq}YfUH;oEUrzHeiRxX@$20)Jfe=95oh+Nbz0&5Bc}m zcwx>LqR^ys<~3)i-1oQY!XT?WAhQSyL}Cs3*oj485B>rTFq((qgKWaJs!cd$5R;s- zI-s?C($W^;%6hWSp!frTYAReLwJhX5?fA??9*GPMnprOZJwU?0Ia7eu!L;LO*@}WS!c@@TOUgK9R#Kuy`|wzf=c>XVBAs(B9Q^9Npzg5GJz`AgVcJC~iO zx&cEiLF*RzHhl=Rh7S(Vj7f7&L0NErx>kaO;^-KjAc#G|6J+{ z(tlk22HY+PBjqxOdUh8$30nd*|HFAy`fA(E}ecG_VaxbBAaLLY1W`Q+jeIkcS5y_ z(YRVNjJ1&XDqK0BV{()cU=G`{*1uJrtNi_7a6RE?ZsqY?W=vbUFdX{&Pl1`8B)*zj zZ-kq^Vh%1)wm?2mw1Yl~AR`8MElw|buKL%*F4Zxg1btT6iwZP3chPXP+j0yFB%Crl zx@z;gLSRuHO#So!2c_@!knwZ5UCp{-Rvz&|YHdnrz@~$&E9y7JO&5~Xee$GJ81_yp zhC&r)LK&>b!1+8+q3}#}J|qz#j{2!a0il2;zH-bmshe*+@(G&Y-pvj9W2_o1aB_lV((j$AsJwy~=azO&i#u zNJ&X5q2k>aa>ZNhB+#=Ee`yt5#mv&c`$lk$io0=i3$m28$be-jO8TlQ72J5scLjrt z=x(saK#7IH*qO-Rdri4KzF)+<%juj7!81cTe(`Uhywnc+>j793v8xU%MSESO`FU+g!&5&dz*H&IE z1dSg~dlla~VbPn6(fsr4q>zw`M+;L@*0|iS@GQHS=wJ;OkbCaEX=0X1UCj1Gyu*8# zWbeawCeUrJiQizx8R9Pov!I;3_o_(>fBzKsBfgEiKJuf=?(kq0dk!t3!e`e4zpzhD z!8T*&M};;Q3cBIFFc&oTfD}~;K*1PRW{1RzntFK{Wa%~c82(w{G|}N(Z`P{B3M%h{ zboPOKLU=6tR$DVI3q=8T)6^d(Y3qtLhT0vCpq1T);x%C<*UWD6`Xsb#+$G<=R@7@^ zxSF6?Z5@nTpmgDaKiX|GVTRX>Rx00&O$EJe(bZFryffnPdk1rE=lXhMMQ3N*T`Xy& z9fk4~jP(vsCH`@cZv>0ZuPW>y&r#SmKgPgJCRwkv^~y-N0gdq$14yoWlwmoORPtw- ztXXN-x6n;IBrNk4uI}GN6%fU4j42U?j7ealbGi%8GSBtFpB2~W9?})h%xO1%H%0OF z*6LD#$0;{g^7pMQ;ghc))XLUw8D?i7mC^_c$+VF+PAbn$rHs!o#%iat&~O+7+E;H3AwgiMqD>mw?WtTkTr zgV7DdmlyGX?zFQqehF8whqUpEzs&}#Y&W5jSCYhNM058Pe_j(?DnOxWjbZ%7VwdlG z`-{x2CGHU(-yELD$ssRFFz;i)VuW;9 z4w$IVWeKhw*<%dm=P$C-MZO6!f z9srLf{%tCEottEK{CuPGA0%4-E5k``crWBz=l{=Xz+FRlLcUAsSPlx_qyH$+`{9A9 zGKA^UTXHXRQhW}ixbWeMDyw(z3Y8=hBN4i0naMC!qBrkvy|-Ub z;$$5aF{f1dKhE!>vB=T+VjL^2*~i=q#FtGr*o{<4>R@9X9%0`du1+^GeFi7KEZV5n zfV)GpJ(lE(G|)-@?uiL5;;Z@AnBpru?p$#&-mo%C9VtH!dUOp1uguEdeE=$3yH6=g_emn5l#ttWFWTsckl@rc z1(<_+^$R?__o~XOcpFe@MoM@Piv0IGAy~`Lk0(2~3trd@ECYdAn7|r2-1+A*=l1Nc zHd4h8HU7x8aA zyFT2JrB(9fKMb8U4+h+1D;Rhdm|C8x3RvVImFg?n^g~M}Q;#FN`RbaqPlNpzF$Iqp zQ?6Pb?BctGm^j2>|E&72eSi1JXzF4R0SMAWHrO(^a?@dT+01hPE0+dob}vt(*?TDH zM^v>0WNcng`oE2h!oXxDx92ovc%KPo@gmxO<2Uy*p;SZ0$9{U324UR7+b3DEi_hEH z_VHO^8^JZ3gR-AeFO5BT?iMDMg#bBWcb)AIo+kKL`aU&E6o5+)+-e=h6iAKNUGocgt&vGk-Np=ePmuvx&!YUB=lXhb>@yy zf+Wlv`?5wa%Oo&EkZHVJ(xJ2CfYXnQk_wiLV;6Ly_*M;GhtYUP5V?NeW~y&7STe5Z{-;y}mk1?0eJj zFpnwsQ*p@SZXln^C_rcx)sn@9uyuPem`Sxt(sIzia$`BFP}}=RjB>R^PUtC95-GPF z%SiolL{Ee=<{L&iZ05lF`1V<2WJjC}DJ6OMH0j9q!pNKMYb%hmz=TpZS_q@PXR5Y- zejzs8j(?w|#>a41{?Qfo8USF!x`mC;u8#CU$jH{|liA?V8FdAq9Fg6ae*dOpfrAah zykvcYIbm?yk@qUbR4AR-QfWIb!to=xZ`*1lG7lCvHZ0H`q@vk{Mo6voBe17K=2Nz; zs2G4IKX|I1EPI)Hwq7DlZ&|Ay^bV*O0-DK|jbC_f@H(kGHR7W9LZRVjS%lP0rHYZY zhKUZ7|n@m z4&OmbO-dO(pvl)RyUfz9=3bvMI1#H>ROA{0KorV5z8@ceww)AL`P_+a(l}-=^3*qNO*(>)f zmt>3LBe0CvlFcVNKAVf3%Lp2Q#Wmb`b;F#iUhkIA87Ie47DjS2 z{?J*upIJ~+gfPV;x#U4&1cj#+eK`eGdwi6_&VX1%lb0?gKxYi!STYE-QUMGg6p2!_ z+0``HMY5BL3@I|U;l5?d4BY!K<52R|#(7{1{=i!EHuZv85Zzc4-HuKfbJAu?>8xd1 ziPA&;n^kHZGXtjmh}ONnbjP^=^jp>+{xGPjkCVRmM)k&4Q^1kZ4`_&dhya9SmTIYF zsj~c8^PGI2ES9!bZHXiB)w*%E-JZlDEbZNJA|6fH@nqlzgFFn&1H29EB?QY5C&5L! zEwJXg|17uyhbh1b@)~uGZ*i#v#i?zTebc))sS|5GbDG8nQ)(I^+~{~9%A)bMSc-4- zs7g9EwY1d*{EMIlT5=E^6+Sc+Sb+rOjqD4EjQ7BR)xB0 zUW8>uNj-hEuT^$OjIYj?o`s-^qMw_{#6?B|5yI5Js*okj4;mmkSv7s%vM{iLlY|TP z5U7LZRE#d^TgeJh9TrQ$ibbp~|88d*Ie|wmBHn7uzn0Bh3jAQvw=BnR#|>YPf`O_b?-!1lwamrvc884rJ2r)C7dzR~%Wdj@ zq1vm~wavkG)S*9=S%=(yfppWnbfc`XbIAJ3>ya@Oc(nf4tR<`$KdBTT2y%0al!n2U zlf%@f5qI?&r3(&b9gvuz<`X~h-IwnwHNY7n#7Ph2E?$}*joFJ8WV zyws-y!E^eQ7*A#(`Ye>cUoU-Z;a&Af5})dc&>|LFAWF3^t-5FkU>ZYg3s6YOypBm9 zeBeh90>(Vn`H%*@oZ#o%Z6n96nF(N2(MrAiXpp1s-HAG zNaP27KPxUCq7>wB^b^Vsi_;EBrA^hz3`CEer7HqJxJJV8YlT+~HW&iX)1l~dMjA+o zt=*ZpQ$h`tMmA}*!GOw^L+fFj0`2o_z_pC~XjI=Y{tcJCQjj??ezezc?=MIlALv{= zEsKUFJpnav4VGI|?lt+2$j+XPOx zTm&Ly?Xb`gXmdpZ>ymOUX>vxu4E8k!8W+katRtIZf*2ICjvD}pGH|k965d-GyKlQb zV!$hHgxM7Ool);P_g57@o_=FGy#Yx&jEgMLC(^vNb*cC?;19U$#PjMItbXRU_a-|~ zP80uLU^;(fx11{@p7C%XK}?YIi+rIAnh~&g1A^^uroUR99XmXBp7*UNL@Gp9lluO0 znNZA&)U_V8?^!(&^24B1{Iuj)>k;<;Bq)8!2A$<++rZAs-?h!+ZQY;N&s&oi;pXYC zu6_nIs7i?54CMLRr*21vnNXnVy27)B7<$(B^VXoy{2P&_aM-mGzx+v_j=e;%(O1!| zY@O;>lu^28wYkHTw0fb6%s~29%cKQ8^WTV|6M_Z2!S65W;5c^lv8|fgoOEJh%{&Ay zthb~STNCa40y&*2cIno}iS1PZ3V=4AX5H0BrqeJxi}5vYq)E7dXnu1pHG`=i1gq5_ zVm%8#@1i;qbn@O&a}91A?Qpo@NY~iI5g{kc?Vzh)oKpT>?_`Sd&H7YoSPbtbI}-!52Al%$tF;nV1bM&rr&-i&AG}Wd$4baJ zX(93R!#Jv_(yHdQbi;FrTF5J%_23sE*#ulvo`yjb zszbLHv>iHqcDs$~lVKnsm^0nKm<5{MTf2hUC19&Us>)X7f{Y{`nLlf|1YZ&jBglMr zt;i#RLaKm%xvLJhwUcnPwNDVZS^3mSQX)Oar zll$@@59B^GPwW8$0NN?pJd0EF1$CkeW$w=cwP(!Ne$LV_#s8`ffN(i$~IGds_@yp zx`$3yHa;*Pu|q|~cPfqn>fi2wG!1zvl=&mJGZi~IqvDedQv#ye{@a*TY#n($mM2xm zoNL@M9cu;%!3`@5U)^x}6hMtBaPNm*s>si7Kv?mrPd|w$fzj*TECHi4puH-0FI*)X zC@lgpC@sTAF$L??Sh=M}hq>)<0UU%LiUW)eCAGPgn3Tj&eCuZR8NOLV9a%rPuxgb(s*7P2O_1D-B0H7 z9)0GBeK6h&lxYG>*K~7ti3rt)Y>AaqA!kX?8O&R9_MtlhNiP{?Z{T6yz;m3OzYRZg zlcVmu()3AwsOM$3X*0gY=L5+)Ibyv8$a}y~Dp2Obh=EluI*^ErwJDySK5R9-0qnfy ztwS&&?=mi{D4)uN_@`GQ4vdzL%+0xhnteBl(FDy|3?{Ba#`yM5K^e5@Rt|+m-15BnNrH_aGAtbG?sUM?HG=bB8m71kP`puXcwkHZ0p z%2CWxoDL3xHEPr-N2!y>*c| z`ux=5eZP%cSKy>6tCkrm8O?4UF_q5u6{CcD}>JFH<|6P}VEW7k@O3g+SO&n6~ z^?`!q2zfwiHl7{ZL1vZTciwxlCP8>Qo+kbT&K_(Qs0dcXe<1VlU$LRU$Q{-BlCHhY zWYTTh2Ggw?d}O*pd}{X6tQ!iJmD6;Tt=Kh3HP<^RKl!u{fM(aX6)fE8kS^CS`LZ6_cp$(qWK?|#rZRATo%8+t)2 zS8+GA8zvYBR|~?qmzlQYY%Gr8kKS~#+Esq%AJV2yjTL#R?vm-&yf@(R0~Qmpyz zhrZrFt|{=?FX&Q!5S>9PFgp;aQGoc2qVEd<&ngY~NL+>e9MrlTD#4J~x6P=n<3CG_ zq$SYprYWl0J%tr}os1(HmPJHZM~cO#_Rlw(cQF1zfilR9V2OwUBW%{Ux*HDj@ZySg zJ9cKnFLna?_*|@SVoa?(xuz%gmFjRvpO?%|zHDSu(E{%e`*u+iE)CT{UWo4%#L}H` zX_T@P_ql5VHqZFi_Cq8`OohlLUcc)@UGtj-C8K(%XmUH4v)eK@Kpg=AS~n>5M|Ww2U; zsMJPXO0a6;GldKn?ZI3+B=arCgb@K&!s& zIk(`I+g3%T^4}@4?VOW{0WPlAY93n;)vjI)bzNw`}Z zfsWU@_^$)5iVp!G)8ZSY2+E?p!+-s5fLnc`OG{YQ$>m7H9)wXkr|=iM89(~j$ryW| z!He;*&pTcP;qT$c-jbuPFostjm-*nS;`#ZopQkEMfvRhh;@&-z>USmAAYb@ELlVpn zPO1E@J!<~y3|E3M;pKyMS-f7_J}#4h8f7ECz{BpvF9UOwXRK_Meo@2wTd_%D)A_DR z{xKZ}=L8FxEAdI4O&q+Ri+=#wSv}LvAIopvdkc{tAsCAVt0Q>SNR;og<;R;Nlhy+W zv1{VPMaY_eL!CB|o+BlZX?Q?U+SmCE(s4zZl$LVXkV!_fRAtksLnk;uqgjTM7M&kX zQI3|evKg*mYgB&pmu$EjwKHc!Jmf!8~*rGups?*cmFt7{WdlI13h`ddo z4%sD|*(!97TyqwLa4MOD0u4-=qd>*9NY|Koq+5>n{gq17kS3kZfk>617LOl50|u0K z=JyqV=LN7ebN4AexM{x+bgmaq*9(9)na-h~p3EB(gT#`xUx7u+2Q0ybB1e>)5MSX; z+FXcPsTbU-u)i|h9ecEK6B|j}*BIeZfjGgloiwSN5&>H{hq)86>m6I3N|+}{x{UyC zSXb|l>KeN|q(HhGDMb)mDa2(MFy*1%rB%Sq*v!>$Tjq`{3aF5wPEkMS>{a%}?!owx zR~kf@$q>WVO^u%cH|XNM@iWY5Uhm&}KdGcT?L0^!@O8c&dNL(qt%!`S4{Y%@CIjJl z$puaIBO6Z^2gODJ+z{#`aqy;!3>RVFDG;3Mr7~(%jRaoDlq?A60-*`;^K?zYvFw+N z7>@-q;Pm86K8ya5@Qk#Ym^wvrV8HO%AZcu{|7f;{(!QU1_nW(W>6q%s8mY)A_@&y_ zin!xs{bR6*Re)y3deBdro(IpG$Z{Cclv&-AmDso1Sa0Ko}}&nA2p43;m+6^e4a zFPFrvgAZB9aY8ry@SUIZ7{XGrokEasnt%&>{t~wmw?vPu!1($|I2E?a&xwW2zW6=S z#L?Kz`u%?*Cd1@pLFo{^YLhbvU}wO)>bFmn!Aouv?oFiH#h7j}9WexPnd0F)4DkN0 zO)+e5OmWpJ{lt``>outabQX~GSlPEA{s>&DP%F^Pgh2Gt@N?cQ2zml>Vx_9h!jda8 zVq$HzFSiYDQBZ-Ic1yQJvGBA@1<8T4jgF+L`wqb4Zv`Z zFjgPdNzS`iYha|+1OG4XAgsPT}#)rkS3hT`s5ptApgeXq_Seb<5G;WnPjtIQm0s#1wTy&PNYElHO^YhgpylJ)ByT{B3rsmw6q=FN@w`gj(VF@zG0xuaiUpy zgZY7W>-s0c)zl%P5^-UL&e?e@@ir^s(TvDnAHpcC@k&d?Y0D7CtBMaGMDyM?v5YU9 z<4YH6LfWcXcPHKtBME`B$`mX85@~#zq!0Eg2Ui?ij`1HW)gl%pma0JxFY!1s5JN6$ zOSC?^6&?E_CqUy>Q$_7re&C4lxoNk&^kI9__NZJcO>a&0AS;6kbwG)k71?a7F`2>- z7EPd!hdo= z>njtGWvGP#t!bv8{DZ?wFY}WPy18DWUKDR2KW&LEg~SSvwAEL}8uHQs@cb+QQHRUk z*uKSH8p{WN;=cbuQo>$TAst!cUTzwfBL?ozKjJa@-0IiOv*2jqq+=OLml z`%`&_u)or@C<*X!ZM2}IntbS)badd&I##^mmKMJWpz2hs?9@qwF%4Odbno%MTZwN* zE$jP&(xq|^+3m`U+}E`=^2r32>DzKF*&{mJK(&U{(D?N8ENW;Iy)D@rXvdq8`jhsZ z@H~s%A=2G&>ws9|OO14r zcZo&R(2T(C(TNQ323Ua){SkAySHzhjIp&mN9R#nn^w00FdR2&fvHNt*T#}4{WcyQ~ zXx)ZxGqX%w*A__U%XDsMKc&FC)Ke_L5@e}CXnGmzBzGv-B)&fhuPDb?D zW5KI^c!{bnEGKPX?ioPedu=ec)L+P4yRSW-(SX%NoDzlTxaPx)=3>1U5yd1dwQkm8 z@R@5-_W8hDs`@pf3EoZZY9k3~A14-shTqS8q_(HGG0XM5xHJhipU&q|U5{m0UWh)E z1T|<}J}v$Apvo5_62ak6OFB|m>RoA+HD}PB3hr9@+r`DH-6jQKTw9UyM2C7gnkrOn zqC%G9-Lb5r7!uadbwXH=dihmAMUJx}( z4XKg=4IDd zj(v~HGDmbEw%s1#?pt9TXb{cYyF=Hk%bT*VC#v?8Ih}Zd9rk<2x<6O1&~+(TQ5J=| z#tk(qR0=<%4T0^XN(Krf+y~t^bApX2?)4M8l?}=X7bBH`%eg5NTCSl|{7u3r(F#8D zzphnsugUv~V>BkF%fpDlF&lNkyHHb$-h3_7CfTp1onun{<#(&T28TaQWP9(4DH#8C zJSR8*wKD`#2HtcKpv1%jqY=ZC9zTz$*MZB9mValp>vNI7weknvFQ05i& z@_wuUtya(!9zQrKI{ZXj^SK4|+;o3)_%_y{v~q1@wm{aJ3WjbaRE|{QAUM)mgbN1_ zGE-V0oT#@we=Fa28IRn7c$E~mvec+4rF zRd&^_79x_ch7MsdKk=syy7CfJygSN(O(=KXC`8-FR>A`9E~}b*0g<`=+s{=AppFy? zV&l9z-SFp(8%k*D=JTmdM0h8+O+$WE^jX|}!autyDD*bVC~%UrR*Z`Iu|CWgokA>#wxqQ@ z@m2#4F)H;M5UV?1l>ze-n3Y^~2GjE^zo%b8&Trj zX0q?mCggb7VO5S_^3~G-Z4zI!6AWiqvxQuIuopD+ePiB^Gr9}KIriAVDJw`FAKYH5 z$Y|Bv7_%&;TSoBRre5Q`<`!hYqwI`8;I>?)vCE@ZoQ5{z^T@t zGB{Jv=@*Sb7b-W;Yk!v(=8y2SA4Z-X!f^byNDW5Z0P?laI!&$odt z5a@Mk5F%3&_kkr1LWE|d`g7GGNRz*Q4%q|HlNs}!16ZiP@1rD`&ghswpBf^bV@SXD z_m#leYR-7N;Ka2|pz1H0uU*2XoA(z)U!-U}R-TUj8Yw`^A<`_5bjN``YtZl%s-IL6 zeuTi1R%2psAXO^l7k&^@PF2mTdz^3n)*-NbvFSg{&u2Wg{*SDtt%C+?vezS#R~{p( z(@oYPpnqFQMWwVfwLf}vn&r>YBB4h)>B5my#uefG_k_mu!8wd@>xJZghgl)yJfi+# zQEA5~-C0WPBaojSoq(}c#|Fp!4|Q}wNQpCk2|KJ0iJQlUFatIJiMyaF)pKu){+ML? zI5V?|vzD6e$7~~nN5?z?RCPk%4$(2dW`fgPVoL!$o$(>UN-9#dIv%f zl5@$9k!P0Zyi0?Az3>yHAk4DST%H+x15P78*H+1;X(Bcv)>-S|ZjBF4$P{LFtpDHuF_aZWl?MP`DJ*e>6=XBM;i%!4@xU;d5ta zpyx!nF|8(|TeEU}XMF;2jFKk5L?Ce?1hur@Axfl78KQSW=8Jqvgs;@uSQ$L9lnA3s+5J=V093o% zI@(e|9Tw=3Px5JwVtX+x@cOgm?6uw~&mb-~b}t)y@EJSzZjIj`x}9PQRI=$aq69HQBtL4GLD+nTZ#Rs0@*) z{6K5A*IUcgr4V%8`lU=l-$ONQdZqoLhA}u?i1Q6JVx0ubi}LXPb8-kc+ju z)+}+FgV7>OeEMVQ_?(#rg3r)MSJtswuI_kI)ZiXcAjp2+qI|UgKHFwV(ND>{WHSU4 z6Xgt;bev63tQ2$GMI^wE5lLoE4%q47Csy=$Wwf91nOib+JJz2F0uLjr5SCO=I8JlB zvz_r&^u@?Z{zMM7H2WE>Yb)H2`}MTdZy4Pj%){Lwu*=xp^*H?QvHG1B*b^asz5xj5 zKM=tI)WrFRm)Tr$rs;X~)v|9-mZMO`$Qvd>0aA1CF6=+nfB+vI1IWlDOjz#*R&7)4V#{k>zu5UvA5{Mf(bX?BlXxqiiM;b3IWH)Z_S8;eS}YZ`P6I z;IixM>gu{u#ykqr8Qq(W2$skxMhpBcohS|QHag&2z10!Yq{9@Y;aznpC(FE#yp66v zouA073&t`)%zr9F8C$}_VjoaQkdm(!jfO!Mx--2h6W8;_Yg&n5t5wuxWWb?B5_Pt> zT5$WQ0%>Mo)w&_48y|p^B-Hz3C}PF0tsFG&vBG|bRo(CDWsC8HHe%W$2bJq%`LnVW-Ew1;drSQsIOD8%J{J#oM=UM%wCbc+*F#?mKQg{~%4K1E+iZu!1EN*-5b{ zdq{Chy?v?(BK<4m3L+|AW6W3LkVcu|N>eg|A8$zEuxiyy$flm$icN67Q=yrkpOXP* zSSjZ*9EYKGuJ7f+Y zjsh3iWym=kZiKLby$sqh984U@{=MDqLH?H9fD3mX-zT}|iBapNh{DTysUtnxW^p!V`nTDzygbdk;UHMymjB_ois|Jw1E+}hmI-YI8r<* z<|cYbDqVQpFtD=Zw^^6aPBTuaygE-6sZ`C29!)*D12q)Ubj8yntl^X9QyO|-7ErZl z!7V415%cfem+M1avGkNfKD+ja?Zl5)1Za`RcW-b~4_MO`j}Z7h1Y7|nqFR*5S6DUX z2UPIr(L(i~%6)LO93uS~TNNQ=8w-18$3;G>hWzV0oV>Ig`Y^H{!%a&aJ1pmwvnYcM4awU?90R zhDb(ooA(ISeOcEpz?KZfS|5-3Xmg>5`0c|5U+#~Y;kA2Tu&c{0ndG=$M;lb!QBOZF zB2;iBE7M*E+D9~&;YHyfD2-SBRlRLDGw(auFV@5@5*r&g8M#NN$c?AbbpGadSmJ+f zRsZ1cWLgQ_>b3&~H`+aXowyVPBTI$T$n>#$X~zn=%){mMRP#!PkPu0_hrE#DptOc> zWYq<{>j8@WLQL-QN1s%4-MKTNP#XPy*Kg#OPkwB%HR0s(5q260lSW#vMzm?LV8V8P zcQ|^F0;xLz$v+T&DW>saTRe#sH#Gb^e>g%5B28T~MVs7DC~KLFH(|Ldpo3uty81Dm zT-)NcTkJkKi|KR60zHPIPJ6GLWfhXp{-Kf$7Se6lw~CspU5%Y)$svqnA^PU7DKZ^- zTWrsHX6ou5nL>n}Joc$0-W3i0+?^wZ)3OW z8O1zJ@Nl{_Sj5tH9DZLI#@VCwO7R$q^zw}x`?T!`WoYMzgSVOeRN1-|@H(ReRbn;8 z?eRfoz7{|!g+n8oM$oFa(JxHwmTMuNXwaQ4thSj@SVss%F%Kpo3ib@?@2A%A@fX=T z8fDYpYvXA|E1Xe@+%{-wTcpJ@uZ-H&5pb5A&FJkil!+1z2d6VoMWg)UWun18eG?7_ zqkI{oV_s2h{5U6=*G)u+$7YLS?&NyvVAr+{y0!s=!XO7mi};mE+ecR~K7mO}2T8Gc zByF}zfiBk$DIp1$t z2uyDxXVV=*aAteEGd8|jGH`1ugwCn&(JkIrEb~u3HJqlz+1Frx=&&N7)x}wZNbg%} ztFc!w_9x6vWEBL8(%pAh>{jf~3L~5$-D9wHaZIsCgD@c(K!w^Cg)`%m_;DloQSqi) zZdbc&E(=u1`kV%j?nQt?4c)_OX=RZoF9|FNgvr&V)WlcgJ35jqSN|L|p@o|fvb8s1 zF`+u@VqED{z5{!?F;@@mVmH~fYDPDF+cTzsxs%QZ96ZDWa1n!iLv3eLZHnrh^!U>F z4NM%STO9Yhq`fpT1;Vq20cSc&l3O`IuusIf)^ZkK7*rikrUXPnR3^dp4rD))e*FjK zD^|!7HRKScbA$A$6p>X6u8(?u3e0aXa#)x5J_W{jk@FW~0f;?zdu&4-y&Z%es$;p( znWETxkY&`xoxlOW#7cPdyJUC2FjF=Ux<%lA06BfK31UPvFS|tSTP6Skfvj?3kQi1@q@WQT=k^DIDq-l_r%8*b zdDl@HL4%31#5YyAtu z1W{uH*qI~n-K4HQsiJou?dAWey!{mX)2YN#>Yf8568?5qYfqMeK3x+(prFkk!dPq4 zufJ|HsCR-&E?hF>Szn0A1ocDNQ*sw@(SX=&5cMdHKxJ7^Oucb(*SDhHWLk`(3|O>> z$pWHpi0r|XSvp3T&((6acU+sUo$(Vvo4J%DjxCio5e7}$IJP0(QPeBFHy=d>-piSzi(9Vv zQ>h46xo1}|s?Iuo%r{wS?mv`ds%>(xf-rG?i}wuOpOI9>ap6RJAfS;F{C3wb?@qH==x0^r->tT z_t8+bh;%mNeh?iG20|mQz`XOcA7Y=qZK%r*sYpn%KMk`arUmkYpY}`ScSbE45BC$5 z!&Z7%fq$jyjCtChQ~M$+Q5y0qMs6=86@qZN*)cff+Y^sp2Nj-4c{y9G6YHu?yngWt z3aIhe;lNW*7LININyquzv}jzsB)Xow8XVz!=~Q*EYFn45Hu1oaR*to3SOeji&yE3gOi z`6HrN5LN>0n)xSD*k7iAY(Yz^58b=+Yif8g8X*o|Z$W}oGz1j=5za!Vw*@^*x@;*3q3G}X+gD;bBs?UO!f9JMlWWsLXPF%u^U>8i+q@X<(DkdnPnbUL+4CRI2=Z70 zz7B3I(bTENvbDrsN?5s6W_bLb6rg9p|kPm9qF^I)dvp>phTU$(3MB#*stPB;k7guB{Aw4 zC{&d-G4}NlGA1tkJ!l{PH$gwwUH|{=A}83Z z1!jx3b0=@lqU6&sq=*&eCN}apWWS1gCt9T=95 QZiorw&PpQM)~y_2dHJ4qa8sb zXA;_;hRd48gUk6N6c(1{+r+0SZNUKV+U`$x{CPl&@!BTd>rGq4KmHp7WVNtc*&-A3 zuz?1lxs~j!pSA^4`J4MJJarAnvn}(*JIzW}M`yk&T?X|JUev3BA}S5BO|pprK|EH$ z=o-=K%m)L_?X-GMF{0{IQdR`UKkjIX_xi4(N`Zw9oXvgUKC*H=}Z=8etROX->xRY*+ed z=VX84J=La@!;ajG{RGIkjob^;`319@ai2O9P}eTSv){=(f{Ja_1%wN|+%wj~ZJgr1 z_J?knvph@YQz(WKQ3Ezu#cQmT=*z$um}2anX4_`ah$idh`v}+uj`0ac*QZK*kvR9M zr7oaid^NjM#lw|SEO+viNEV6(9ruRFP?OYi8-6RerWF{QXfTMsIl$(zvSsh%o@8tf z-;n&Uk$(zM{f< z)yC-<8_k*qhY{VHt)!SA6=Hg{S{;2JadOib*whcQL0`2-EdTPHf&5KlDX*VI%m?44 zdyc6 z{E2`fuVkjS4x^c^s9f)B#2sZ6EntL)D7RCdtW*nvT_|StcYYG`O^&54% z$IL<{B8i)Bpd*p{hGPvlm%dYvHk+u)Nb*IXRrEax5L#WZcvT!4tER(rvkQ+USx*Ci z9RzwrS|~0z10HykabF{fY_K?*%(IZUp%)=3Q{$~esoQ?mig1jm4u)d_ZC9HiWU;Q$ z)>ctW#*xvCK{02e`dH1ka}Mie>!dIdpAozR3VnqyCOeXqBZo9Db(7G7(s;bE+0BIo z8>#QAA5+B3sn;$Jobijgv--v_RmEzQ020Ur4p@;G$Ty65N8ka_@Rr zmL(xH@Bx%e8xV>?ZI#@*Cl+C}x4 z+kyBvAjAtxFxO#s4DI|_=gL|JPHlyKd@NzQ35>QsP-Nwq`AXYKXMRUd5rA`GYJk&S zGT^7xGnYJ=pjX`B@bx{N>cC;TltVtSkA1rvA<|JbSgysIdIouNQXDuH_;de~H^GPe*-*{8|D0uhbgq zm;lr*LFiivZ6yFUt^8r)j4y0__UgdL1G!Z@aS1g((cFHU+wSd0$;u+hgcdtxMXTDf z`9NK%V1R5GQ4TT0M^D@UD;3~H0J0Kx#~`CaHW|y??Q*cFAp5?YMI)2FGF{h_>B+Gz zc1YZ;$@5ZJVQzWUSnp=t{`v;SOJDHsx`&0 ze*x#|H6MJ$s3;>hpdQE&uCR1RbGX$J7_hoEM!46dfWJsVPx1>x;zJq_+Y5O^=uCV* z5BsbL=YrDqCTI8v6b11^WVU_dQA72FJp~_=y}6FSzAwjZn?R^{d%gyt5S23<%tcU^ zF3`pP0q@PMO&0$1F3Q|{)A|`f_$C;eLQZxFc%Y0iZF6Zw=$hv%5v1HtP@fyUe2zWQRQm3O~RH-S#@wo1r_?`f3a zi%NOaCjU8aju~lbVZXeLmgvJzZwU2!=S3ukc;ZPwoffGyp+w z0%nkz=#oQ4a}!$gY@=Z)hDfy51??aIgl-R*I2hgF5hJ&9TOpRsMbs|=ozOtPm z$(`NWfs6~RUv?1WsExijjDp57TItJ-PUBm<iN0CKl@N`b>I!YjVfMq~mE(`I@#Pposy^Sn8U`uSyAIWsU1}1jjJh_a;-3u16 zb_Yj%7$C6?c1LK*Sk$&JwmL#y;fiXNuY`l^tV2(PEZ?w|fbS6J2Ri4yZPnOzN3r>z zYbQ#9srFkY))W^hRknCn$6cX8aa=S4mJ_;EPcnHl86G9g%3lJ6L#}$tfR7Hgws+rx zXat@5gRAK?R@uYAEPMckK-0KB&z}Z{@i*q_)ZwsDvjgb5IS< z(!Xq?Z~%!K;J1t>0Me?Lj8AzN)Soz>P6_Vo1gDk8GiRU1*@s#_fu%Bhr2+q9bXdNv z70=SoVKFUZ+sm^`Q5mWh9(V2Of%zFyshst_W%Gm@_83TAMM@n`6w(F!G`PM{Y!i06 zoES)(l(7MZqzWWCDTveWYffss{6-!uj!f7CQCEWmi3z1ht^S35#_7^tT8}-Ct{LC z;_HtbU0`{l`lKq(Q{>rCSaGWvZp2ili#9h_H$vI3V>No{we_+OSzBTe_o)0}hYWZE zn4F(6%PSLKR1*7LFD2-Um-ueyBM}}Lu=IPn2JccUU;xoXHU6z0EU&tF_+xeQ)H>se zsbJouZ|RVhZ{V0eHIr!R>KeFbj%__VLg!Yl?UWt-G`b|N@LU-G6@;ZY)Z!|e5;^`q zxYjS>UUtP$LvpOF1j{~MESEt&9tt!i#!fjqUwR=er#|Zv#>O6Esq9VBAMZV*xKTuhos3yCx_OQRtPFe+0X zT$tqn3dj99T(uQ?jQ1;Kfo!QBHMkdne22oKX7dEYxz%iQTU0Jp3y9Q6!~cZNfRV!U zjKTsQk*tpVl@oC9P;h%==CZ4am~C#ASzFDemk|16E}gwikG~Mv?i*ORE;6Q4R)T|b zk5j=omDe@JIb88S_C;^dHsZ*_ZJrH zmW{YK;OV)xPUWE1jjq%}C3b5E+XxeU@Fn>zg&X|lVn?Bv+ErXtuwS%1tfHig%r_VD z3c3SLXdFBfsjP>R4)d1yXC=X)(FXb#XIsoN_Z*vWH3*N6YkM%g)mc*R2nSv__KkqP zH9|!Ti_sZ=+k7|ddrJs(Gg}QovZr|#bidDL5q_lRZ+`^T&ZnC~%f*}Ni?i#D$_o_M ze0k7@1Vn*0YNd1()x9tD{5e!R<$eMhhzF^W_OT6-71Qc(avFh(m@c_$eJINL!uDl# z58KZbDSnbafq!%M2!lY>1uV9EO$^ zcDjSna$+-tM>1)XbR9~;Q_V|4sU6?J_EsC!8^2W`sUsj!R6tDw# z%O`n7T7iz=l>T`bCt{e-Dx?yuz)IgSsb6!)zGbbV;)iQ*OPc=_eRg*NPUaB(jTqe- z|I2K3RbL>>kT5P%SS$A73+L3RNbH_jt}O{w?|)y%9$P3^lZB218LOSUXsQl&>@rkr z1cY_(Iv=uU`(umHXxZl4)?;NSq*T{yH4>XS_5zo-xcatP>w}3vbSnPs7BC9RYrUU_ z^z`uNFPH(kTS0gCCKhk|#6NFknPj;kufatG${iM_w0UP4jJ3NJi2QUZRc|`bjUpY{ z^7JBf7i2aSgTfr6#|9qUC)*HQBw!X0gum0IBdmuqDcFr7}7Grvk5_%mzG5b z6)@}fGCf=`U~O}h_oaaGeaQv8XFo?o*#q3IQyz9loTwNg4X_a?PawKym0_vFHh4(~ zTb9J>6}o|#mdWWW(0c@xL0pFnGyN{u7%-&pqH6b);y|Fb&3Uam{ALAq{eHXtAA5 z!UCBRwhU6FP_1^~rI_qXihjOs90%$_ItH0f;B(}@euVOyS)^-(KL?ZIjrXhmkZnil z8Zvb|es&X28itF4x9(^wS)Cyv%hAY5LCq1wdOjJG)8W#o=StQm0(Pre;AJows9sIL=Gs7l?WfBM;P)jhSx6wp>5pF4|QQCj#jiRDWbT_H=JowllY1( z$zpLqWdJ4eGLD*I8f#Ts*16@8P$<~_l&$KvrKI)`jc*wM%XI$eG({eJnERqO4|%0eO9t!CSnWzFcc*Nf)De+FYc7DeHPJw&H(IqUCUX%O4LXI%P7RnOG4oua!E)H zi-(sDWI5qVZ^S(_CUM+AohEw-$9GoBs2}dY-MsS8pU>=x3_iY=?rGn?p8Lz}jZPYY zisD^6rZ(Hu{K>aYKv2a5;lFHX8{eRc1Hv{%UPA&;|JEVoNed3VjJ11#TlSC)J8ES` zWxc#sbN@E@*OrcJGM>>nN4P5H*wcVW%2^f=4_6JxV|NH(Bbz}d#g|^y+532S;@faI zS+M;EinPKo#9ytYp7ZV4_<0nRP{X0@d02$2!_`V8zP$_Dv1)!w{^D4Lw*|OtX1_vY zdHyXF_xB7HdUinPwU8Q*GU=E+Qd zs-XoY{P-(3uUfM9aFBN4BB20s*=$?^hkHqIW_Vtko@R%v5?hz)&;&zXN+1w4rkv1K zWH`~}U%V~nD(GE;Rxh@FK{ZPVB_WDn=WJaL?w+a7{{SDz@Z0>Iqy1ECQC45r2HOFC zo<-@vtPo>YDorU{g{+}YhJLnU^Xtesnlz*ZPHRvaO*Yc54R@U%W5RV$?GM0owx@XN z>8j=kjB%=gPdo!4YuP3r4`fB^olyQHA_`2Rj%_lN;MF}dVevv!3k~%(8k9B%qm-11;W@ z4hh8O^h1yLCjB`6>%z(0i(6&%Lmn756m#K6NEv&{XNJ~NSbx@t@s?&muw80jpl+|V z@M?0%6!uHkEo}+^Rj56^;9@%kJD4VFw<8@-Pv6TH7xhvs*=E{(`luA*(25N^Mr&z* zvovl-4arDACE5E6R~wh-Q7LVkf0V1Ay$t$VW#qZP{tP0MJFOhbyl&1|t`&g=b{mj9 zl4f-QL|7`C{L2LGjxLIp;hLJ}poU>%L~RkLXJb=reBH*jKhc{Dc?~svKG1J?cmAG0 z8Ex?t#Z2HQKQBIp2bfb}ZHdJ0)*NE%V;gEtDX2}Gqm5zGEHa2FfHm}x+6i^`ZTfVi?2 zM6SQu*i0%qk0Qm{KlOzDSLSg!e(r}%5Jd`~zblBJB>82&Fq<<K^lf(QlyR#wqrAZoJ_UUrm!|Dxjb{PO2xHAPq+JXWGguK~JS;+9!1!dm^)fE&HB5R8i@ zhP>9w5R`YmzQf;6h5;7f8jx=Ux_xYwTQ)7E1UZw%`Xhlb;SLc~93hLo@{7rvt=P7( z)j+TS5CjMuSsh3$4%Zl0u`SA!Lr{^rl5u2Z(gngaachq`O8J+73_0jW0vKtY*KJde5<``F~{?h%cXY=GHM@y1=EXf=Qwz}GA zc1OuywG9)Ll;34IJ5%KpI#K+ne8)$)p(i)<=9`ep-lQPwT*iP+I+YG|l83`M=~gUB zPyvb`2C^TeYi#;tQhJ`7T@V{;nPD|?dR~rYMGHcdHgXX;iK5?U_}q+*;6)KNefE~v zRcQqCSU)wdY19*kPR|>_>O;+(~By z+?gV*H^VElP3H=MMKlC?WCto&T6SQfRDS+zqm1*9|4QUWJGGJn_vAXmqx@z*klDsd z>XJ`~p`-U3#!PT2PE*;CYy`(-ju&nMmqo~h7(^O@CNCTS@O03h&kX4(_uYe1fJCkIx-d`0)W?iWg|wbp%}OE+B+>EcQea(PRXNttIM>p@DD! z;gW{wz|m7u{)D|Il!{kxOpnyb*eBR0Um934wm}wkDi(dsxqq!nSW z5g7no{SMr>ii=wC->T#MC6!juk2zbQ`kNYC2wWX;>n@z;`u~}istC?C5 zsCqJg-ASmR^jkY9dOunE5e)8?)M-Q*7X-C0QW5^WlN--wFgvA3T)%}drF}V&nf(7A@S*~( zA2b}o=jM;8-I1WWR3DK;63Q`?9|ITIC8S(hy;?T(ypYbajB$5}IpWJD5(Y0=Li zOFh^>7Z-O)%qk}QsD`x@M?tOf7Df?8_L5O(>u%R@yLNwg*vrmXRU-@{d7{;K28N4|K)gE|h$v9UM!;+NGJnLev0evszn(Sv@1snBr0$J6 zOfbd&H#Sk ztf8l9W-Z^t&hbuxYE@U{_M{!~j$*M@9kknqxurHeNx(F$b!-Lz=rU6W9giCIc!J_I z)mls-NWUOl<^l;?HZtJUk=O(e5tD1Bt{$58NuCB@!UvpAGYHAWU=66F*pO_R$}~1c zqaRwSQ#vI6p7KN0?b1YfAink?tKIqS5*lNbBlG?V`u_xvt@a1by7`~FszFaR-fknX z1_`N=Qyu&6tZaY~&U91+8Zzhk01Dm6 zqL~#S$^__d_T7q+FOiEJsM(vYGOhIuZtscOK1TAN8MC*>4?+-Rn>vAIrF~k8EUh3e z|8s#Pg7~CmB%H|=a}X|kjS3ee#&)-0E;q3U?dVd{;(FPCQe3Y~KdHC=0b27u(}4}D zGl!Gx1tc%A{Vq}i6;exp$BT7f5U|QylFChn{>;K!jKT-T8p&u)W<7}R;LgOiJ8Wna za26@lYY3o8hYmWhj?H`_=GR~MdG{ilawfYQU`wakl;H{3V+#+m_L+jRT%@jPMWCf# zwaFSH6z&VIor95>q4}oY2Y>so!g%$_M4lb_#hx+zxkkH>jr<_eLO>FwyFhiBZ z{cw#eSgjTv9W_A1KW@lI5d;TG6jCc$dCR=?iD-6>nWYYUF@U5oWcwLWUXs;a|Dcte zpa>4@TGAxrqyVkk-sgyU7;YnsOSs2D_JI&1;-9M(7%FS~Rpo8wU6>O>(2|(zV*gFA zP_gab73i8RF3%hlzW=K~pp<&C_c2|mtdvIpNP&{*xIqJ7<#N(0Q19&e{)3O_Z`e(T zq|=k82k8kfd;>yX+K01Q4fq*=qO0q8Co9MCw~9QuJv@uQ01i#JGCsWt>RgjI+6YS+ zWTT!t`HGu+B&np;+^+FAH>?7XzL?9^L>J2!=Ugd@azL6BA)V1F6=3Bx6jontRh&rd^1V z;=16KM;gWvBEewVpD4fBNtF{k+Yf^^)v_QNIC&ilN-sd`rXu3 z3I>!;%Bbv1M+*3>8knisVEX|*gPqVpchEZ@x$b=m@!YCzCs$aC5(P?kD zk1asU0k?c@jM?sPV#}w0lcGmLL&{0RO<_;X1oUsVD#^ONBs~1-U@hIz#||G?eqqtU zgP|rD$Bowa;!&eU=2|wTyTVT5sPencKEr%(GQDv?>RYI+{V8MgLVugg^J@smkWbY8 z=!4wLnoERytL5xHM|kPyyc!22_y^U!z8=X3BY$It%vN?EnIDz4x%VSjx~M_Q*3=k*l?9 zU>&S%<33^g=Y{Czt=IklrPe0VP2ryQKNWIJ0b8zyt+8J0?2SeWm(?*~ab4;%DA1l9 z;Y0w3vKu{vFK5uDLQGg1N>tlmlIdGjGYB2T@B4QFHf?G3^Oil}$_A$vA|ZjcUS!&n z%8P3^4qLgxjU_sR`-o_X?4KSEgTJ2ctGw*iAB#^15HJD5%l=nS%c?Y%ko*Ij_+k+X zlSOHx@lM;eiCuQOf;}`79CXQh3nm^lsVgWWBU$h`|DZT)+%g9{>Z-gD_Qv~x>hv8S zuSe#MP(-DKBd;f3-Nrx2iSLIbrnS-3sQC?#H=Uvn!f2dO(L0g|(ZdUfG4gRcM za=;*;dMIARIm8H(1>kJ7bI2VdCfbXNTM}oJJ(v?VUyc}}W2Q^o!u#(Wckz8&0IU*i z>sA|#Xsl1k%Rfyl^W&FF>Kl%@4rR)RQ??}kqQ2ccX;xs-#|@8G{X z5~O{gb-t5vGXdjUy>sU9;`vj1bWSXjmHBlFJK+mTl(ki|`%>6cen7g#NK0gU1#qKO zL)CdUr%I#b)Xsl`J)Xf$ zhf+Oit|4a*vG9~Vt#g9pd!pQL6O|Jnym7qnO7>IA*!$)Ph?J*@GZthRBPY?mktC!E z>1df}DHL6hB-8-;VRqe`M#M?>{0yeS5Ua*paA_?jcAUaaNEGqiT+RJgqv7fLY4^jF zS$Gi&3ZJ2@?DDZk4a1^eO6@4CGpC~q{a;$BGW&_X>dqsJyifvtTk6$GmzeKhWvuag zYIG$Dq!Q>FTUT>%SBvltF+DeYs$zs!J|yha-W5?!Amhpsn!`Mvv$E#0jX;rN1)35C zKVN;%FK>U?O5&P|trq0fm#m`6^qW{xLi&XTml&0JsS1LG11y!%^Wg#t{)FbT?;hzE&#xLaWf zLgs%pRpSIqq@^M)1RZ~N1Hl|eHBS!VnTipcW`aNk%!qU*^s}K0aU+=-m;>deo?T|j$LV~9-6u~?X7VtFk zkY@iEeRieCRb-*SR~NCKppvb_JJmQ8kh_Eu(uUUKtZ?m?>W!S~{b&zGmATOf&MHs+ z%qe$JZQd|#p4Ocl?DvVLldp8b9^3*Crl7|5(z2}sT)k@^cHM$VEcGlM?CftpD;-@w zVvP~58ToL}Op3NjQ03in4`zMb?&I0L<;Zv&M3Kpgw^57ur4U3Joq@@l%US%gB+-mz zqdXbzl}0<=J@I>6iLw08srGa)I12ysJqA_?6#*ow?+(Ac_DXN^VCcWzMpVzLNY`xn z{h_5UqTDQFDNdHgK5qL%a0y#4)$WE`!G0&_ToeH1 zZ}0AkUd@oHvhfHO7n8}}8S}UjL=Qz{EDgVo(hF5NEH{V^w;aI0j!8D z?3}fAQie4R6k~b?HmJ>ErBDTfL8oe9h|ZJ5i*sEta1e$o zXN(}1F!~?cc*C#C^*qVii(4j>UBZ-7NpzGKpL7A?r}F(PxyOJ9`64GD@`)uAIfm}5J?A5h@wxCYzRyR=gG_tZQ-%T} zfLHAv0#j)$$s)!imq|5+4IK0(4la5%Z6on5Ikk0^TCXE#Y2a0YYQ0b&bOhtSw|)gfn7z z8^cEaW}>=;&pM!_zjIx@44zxMXzLV8wf-X6AMW&`tGt>?*!LH({2i~xKeWeZvU!I$ zRh#BU)SHk0hI}Jtt<(6i;t>hWt-0NgQ-|HSi%5T@=OI^TK;`9z!JQG6Bt5*9IvMF4 z@=lc0aC3QRRBvH+sGrqaYqM>&kCmUGYZoVScK$D-R|Z1H%k0b}Fl`z#Y0%oTDiv7` z4njr!U2{3kmWot>8Cb74{Zu7#n&pBV8P(wRu`}FAO{PqvX z45LR?)86|_lxVK3?{Eq{EL5QF0V~+Ma?V4*CJGm{X}}n3v$O0$0E(`lE{wG+SpivX zwqF6rXdw=HHv9w^j#NZg`hQMtHiTC*T{1(d$J6 zJdJU^;jCU@A&9MgO$_QKuNK-7EZm&NbSm_27jnqM`PaPDNhpU4ZUcONp;Ely6gkf! zPpjPs@F*vb+(ZSy%uG!S8S4DNtDW?LUsOK!8Aboj0~|Qj3&E{Qz2ys|Q64$ot>i9W zX7A^y3u~ZN+b*IL9RKMAN>{h}ZIyR$UH`4+{2|DRpp=8o5x@aY!*{Zq1<6pS(|=ux zWY;qOod`t05!1$7dxPzzM{$~&(ch?&4zb>^?y`$Wm81W4#pzzybRO86+dpGl2zGJY z?lCW2r8eDI+J-jelqUY+WkD8|KAlI1DteafQL?$|P^=+tw^amkz3ObJ&^@@d>o<(l z$z$MR67ah{3iKQUEe<5>=0b@Nf_0M4yw`U7>f1DjK#CJ7;&rfbl8wT^0Z!n`szlpY9>=XUQnsnQD~S(a!frfNq-M z>DCavS~PE6x85M)x8eN*@6&p;B)=n%Qvjs(?j$jBa|_@d!2G>0m+T6|2u;VJlzxGn zuFZkKcX%L(rs4UTK1=_}t3y05wVk13RqVv&&cpQHKc_3i28L33KheTD z3XxQ$XIqw2qiMV@K`)V=oCv3{M2#G+4Y9~2rwq&8D1QhVO1L3!SPqyQq&i6~lHo*- z`b*5_Sh8=OBHZn#-0UMmX3hb}){V1^D~S8BTBe~-I!pnqKU`LAdBsgX?H`NExu@Fn zSUhm=xN<=s2rq2G|8z>RdRTi_vyw(-EBC{B4r|2*Dgh0F=gKhFqU*Jb(&qEG1Vn-z zeQHz=3N<8}762_DM?)7!NVZgaNfO`|f3iIGZy9p~DSrKTxU^&jQY1W+1+aIz8G{lr zzGjHicza}fO;)~nVB!BE`MIvPr*uDJ0n%C-=R{FIR$#>#5sz~n0ep0-Wo|+g>Y)YQ z0H!jNlFSmkQYxQNHFxO0V5;>6jv=UpLJxnKm7aM=UqL3&xLfYRP`hgu%J<_r?!|Utz~CeY6E18bAT~=t6 z4WJY9ylL2IL^WtG|3K4H=*-Oq?4{a9;b?it$sUMvs$-ANLVycDgd|VCIlaMQBD`7z zE=Y?C%CwmL$sEei{Z+0#E+k)*(?%z;!u2*3cCLzeP6x(??_@96h+E#PxAFy1Fdv%Q zs1>1g`U>VkkPW()Z-T!NlnIx{f$!S`-6DB}|2*ol?;c?Y9`_455tq6kyLUjK-T$(p zxZ~B%`^)F_(bbzdDzy+nbzE7c!;J&ZCh-Y+Y7MrqQ-q;4)=v%w4l1=JK-6w)<)S#C z&La$M#FuNu=0@nVLp{2b`#Em1ew||0yrPs@fa%+G9s?^nw28b6L9p@k6!ZzeZQucW z+i%0l*ra|_C>RMq3~+3Q9OUHIf}jw_No2*Hr#)n7@BrE?V`-Y9pU!qgTBg_!GA#~^ zg+kIpz?iOG>LbEAxaW`NA84P+vS)Ix-R>_0-|=xYod(&CyF3B~SI@BQ0OCNb!Oe3H#SC-MZwcD4Dn328qV zX~~4c6}o9&#lt}ucH{IT9PRc8FS13LnVLJZM>Gh=6%Gfhcp{7Q=q6tG6N<_81yQ?g z8HB5bZ?7)n9PCYGgls~y!%-0^@6T8vb7y_VisrDc0@WJ^07W1H3TXxeB0TOJMg!1Z zMr*Zr1s82g6uXxbXBOnI$2qNq+n^J|pdGv6$+^8@OAR5ppfq9J_b7IMxvqxD29RXD zb@J#uAJg9@a@8dIaTB8E%Jv1{+hkLKE!kg~p-0rqVdHvsrjw zuGoSBO}nN)hu1D%;$Jx)SQl zyKFi!B3X@Po)Z~HG?t^{Ha;K)n#w+>5z^k}!?zJ&K9tQ}?1aWJQf)rQ>52IWfX@wJ zu$%OyAD*JiAN&MKN!t67d&o6nuX=k}0a%XOqE-{CPxL*nP?p$`xP0dQFunkL5psIY zQiw%+t~NbY+o%gDy~~6G|53>m1a3Hhs{rXsL{88H81j{ zZp7M<9*>PVlWNcmO9oo~M{d4&xu#SBnpy`d0s%d3>www&1HD1UbLnrXrP);s(cu#G*XnL$RyC{j)$#nKTmG6TO9Ub zr2w%VMKYMTEMC}!Jt>ucmvVFLRI#-YCgf*AZ|QGBweRJQmtM2N=+Mt45tq(gwCni- zPpPn`*a41pZYqnDf{wj{>K+hkyx?u|N$nWNS6}%kG60k!PKS6#6)&KUFULUudt{=< zU{9u9Iwx(3%|3rYFYe59xSIRFxyTWTsFxZ2JKS%?>pg|ky+G{PbP{cE_kVVUT5fu= z^#rTXN5dL`=h!E@1zj@iZkeqw#Pgj4Ix;+gaVN`^m0v89#h2`;z-8C;G(s`4Rok?C zv-;&mvOkhE?-aZ<6Q3Bz2?7|mxzTNRMx{?;Tw`Uqfq^_f7Gz|SC;{JdIl_%GxANIE zlrW&y5iL*rKx7U zA$gD};T*$AF^$TMhX^4(x~M5O&Xxt!@swz>N3|BegffZn}k%qqUQ zjBV~e^=>PgWelt`j;?=B{rx+CfIjcV%J#;;Y>3bVj3R}9U=^$j_`W+GfQS$V^=gX^ zS_AwQv(Z&klrr#KC{?h->8$&`I!XcXULVjp*P!-FwB7XHS-Z^rjVho-0Np}fxRxoK zDuVr{5WNNEfSYgJU@{Bl4PG^?kJ|83PxDD61nazB*ytGmnlTuy7|2|chxRGM+f9hn zDpw4ux{?nk!cRH7dNPZh^sACZW<0WV#zKIrcN%VfUhR+!8W>@dX$V^9FvW7>Sy89< z@P-`~6pN$iZ6e}ri{@nxv42%#Hy{e7;At48k=K)+t{Vy4s<*4X=2!vcNixbuz51Vy zU8|5aoXrAwv7Wv^H%R&FrTr~|plQb&i?Ui3wIIjx#~pWu5By^yfR7_R3kYCy9WV4- z<9!o!0}b=de!sa=J=cD6b+aiuKM_R6t$2m(z7a#yUsNI3HR1p=9yq$4DmPF&|&_p^b{I26=18#2-w>C}oqBB4&!_Aa^5kpXrOXp(O zAhB{kM^&oVy->DO5m4`q zo-)!egJeIq7sg{Z&VLwCd8EPMTCvTTSVpX5Eue z6=P+@9o7Z>>V)X|ul@-gqChr|=j@`^aA8jG*QUysrD%KEiXLZE~>tNd3{G4`pK#7cgv!F&ad) zUSoFtnF)l{$~ZGzC1Rg0e%+CluyxEKbtCejEDjmiu-d~M#8PZj1MXw^rl8y|x$q5l zslM`~2J~(-sP*MX8bc^hy_C3>Xl7{Vr|c-%Y&EKn07_HAyxQTXJ~qtiJ%$FXfLTKC zszW@@L97ZEW^?~RQ2R{-9ZrKL?%I$U^`aee9vbv=UiW!Tl8Z~FU6>8Z7gtx`gYKW7 z=h%R?=_t-zwB8G}N@eIy;<7vHpgPH zu}J?G`NcD`SV^`5-e!4z3B*tVI0B{*jSz*USfRem5=E5&I;4F@if0D5(QYIsEubP@^FAon=2%K1)RLkLj8-3 z$0NQI^)wF}cRh?ju)#mK2r~RL8#L>39_g80ke0>WnZ`YNyvRu2X;#$fj$s)@y2tv| zRbVR#QJ=j(*3!Fq2rzj$@_ZutK$mBX%sZX4x{E5JY#ivMMFI-(j^U%aa@kYG*EpF) z9^F~Lh04EC__^aZ>|d$u#s3`pB@8{6uhX!UA}pLD!oD26FVV7WR?>Y(NuY)@bdMZo zG^h_N8~w`R2(sP11e##GQ9{y+XNT!#w_0=|2t!nxZ2T00US~MS?X<|oxPT*at=oiO zM}!ZNprecyO35O)F<{ul$3Ik^)@CkJ6!X-z&b}0S_QEKxrQ>g5r>XRnh8E{Clp#Z5 z$L3pAf)rA^1jFW&k;ryPV)i*BJGv#rI60ec8ls8>8CZ8e7dCmAgD{3lh>=lzn9uy? zAZ9rd)}?W`@QG-)liWa!VL6)&8?LOeBPLOiVGMwt-AJLzLZCy(h}5xfl&x;Oz9KyP zoYW3sm}WRAibZ8KWN&DcVf&mzXM+hm(s3r#W0!RO;uM=}%QThX6|7*`6V5>qd!X*6LcCAjmfc&Oa;S8K6U|GS(hQP;Gc6_=EoZwNu2f?U;$36D0> z)}+wyjY@Qx0~rIk>7pu)fz@=`xP427L}b!Xm+DULf?zed8*eeti&=-*TtOtl*bw>cmxuB7++*$oI#w>qjGS&(t!M$_YMfAjwHgYUcDZIQ=#D> z>Uqy~(RaqSN$FMEsbw6}giEnhde&xKi>WVb(!Qh_Wy%O{=T4k)|RZEMP!bao!y_fsoE}3!chKiu?YIG)8rg?9IOF3-bLrDqVa`gmOElS z=u^`VRyrDV^yA70TRV61ivD8VXYvnf$ljR`kbd!flb2~(?-1*+S()*BF zVkOZ`BCw;cc22t@|8fLwIppCH)OHDKFzp*~Zy2?fIr=^FlVuA8Iv}8#cM^r83_oCn zSc$4$2RBPHlTdw+OJ=P$#q0?L0^vy^>IFso3WpGTZ$9*yw1JsqW#>KR!yB;N&U)Xz z0H@x~1nXD_cRf+1zeMo&vl_3MzeEbhSEfZXoaSZZ4jV0T7GW!!vdFJW$_d~?!}@!O zV0Qf511`YNL$0#`KXMu#s9TH%(!x41k)5>{RR3H^XAsNs zUU5&2WweMsxf8bie5CLW|mO@Z;*`#NSz7VYVkm*A+kE<>AwR5F=Eu$hl z`3)QpSnEpr3052XR~n%oZbVT5BWqz-`U}j})=7NistR9C_Z#*g=sg9!e-n`vGd178vF)b+iNS|{Dr}RHYy|(UrRhfD?Q5xwb`y781nal@N!T?u3x?JZe zCttZeGVyFnNYk@q1|7KvqVn~+T#@=qT~c@SxDnJb$7GDS71`>}vl zNkjX9kFFnUi_BJP&fFt2HEFkqEA$Gi{&ARdrXLG#DBcBMoK-JR|5}~YgocnZE z5i1ZzhX_Es)Vu;dK859a$Mf!?j2BMmn@ep*{h9c)pwK~}*316L^E~&fRBj1u*L4I^ zZE~n0ceO(X<~xr4%b#&`F!O=QBPV&kDh$mxjSk zORI@upkuh~nU*#yytjs&7@xI%ytz4 z4ro-KM1Eq}XV0!hBOn_+lwzj+QFwA_ZO;lFgdh$Nc8{=;ae^{pGzMip zwRh*-2C0G#gJBL7a{x5;yu<3YS-q4uXEO!Pt$Vy-f{pT&JY|UGA`|g0Qt;m#3htUb zMUpl)^L$xFkE_6v+}VKi=t0N9?$RaNVbRdyR;0-%_z${r51BZwXUc9Mv~VQ8ok|(K zn-Fc6<5MpCDHFCxkkoTz!aJnik&5aEY;zq*qu=mO4HBRQyBwsEk(#w_eiVnAhLjd5 zt6ZWMCFvo;o7_>{|F>Tu5&S~ayZkr&30iv)f;9? zg>XhKkjX-vHuHpq6p1en6>@sm#J`C1aQEJC3ZbZ{9+koOX2qu}u@ijB?8+K!4AYX= z??~2e)qx#hBTR=pq*ph4WLTbBrrqSRuU`=vtlNbm@n0m#^efD(IX3VahMc`tKRm8H z1$`RUBv$IP z&L~s<{}J4ewGpq2>S*R;X#7`?F*`;U6<2N16Efe@PaJRVKz5O^pd{qena3y_#F zNrzKutRXYxeK)axZEFO{0a}3{CB+oWQ!-o-`05$aCh|MN3a}pbcBpDM_o#raFH4_P z1!0vPe60sMB_eF*U+CqBeN&->yudmMD4|TEnOd)&NJp!x)av?H zKWUJz8ycG{yc9xcle;4+LztC>Z>8G2KM4Xl!4?pG9v}74?Ao>;9b14ONjW^qF*XQJ3eae~jfL;4igyfUK`o=~NDY(3U$JaDM3>sjt#J zHiITxA&mnJw4jfC>^Ps1>?Z=aVQ<+i_K0`JU0~lqix1K#;Iw|KMlY2>{H5*_o?mA3 zC4CQdLc7ab$G+f~Tws$IsyPCWd23B#)2PotSIr%{i37o%9vxA8Dz8Vh%2lLvkhajJ zU=pDO8N#RYMdJ4}t8^)TdoxzOeJ&rxNE<6h9D@qRa2E;b$Deg7Kuuf_XRoW|MEdf< zN+`S{DYdTIe4{p%K8XUv2uqQP3YXQOz7$!mW}CtV+FwO&fy@wuH$$nPK3XH~O$IGK zJ>TG2Sld`#c6BQrYcPrYGGj+DpuEQp-4KV3RrDpeH2^wzrdoG}_NY={ZSB21uH&kr zB5A@l**p>Bk8=74s&2{+-6 z*~YfS)Pj9@j3JD(b1h(!*urHi`Ww|Ttx-{=i?+-cLS4Tfuf%rNl!|k5PIDG4br_hW zrgClwVlvyc-4ytwOwABY`@ zuSmCXR=0AmL+!Lv2?aUH(IObeBb98vyR?0!dvsF--q3S&xpg_Q;irH*In14uS7mFd zuA|q3^5E9(=wn$L8)#@+mS;0G%?|CxF?R!W`Uqj$r40*p9Ihjt0{5{Mfbjx%)K{Z^ zEinx8B=YWIHIAyf@-#0Rb!~NSeA~=1k z@C!c2rJq$$+V2|mWDrDIJZeyk2rr5j1;3W{k1X=zDjE%h@n9b5g zN&I}Uy>z;F_O&&{3<2-01uo#l5Uvh?Tix?0NoNxf`=z{pKw%p-%V);bn%XUbfZH(iDN!lg<3m{m6V)g= zJ&-MicnekV!z9%`A*bmL&K|G8Uw23t%UF<&(yi|V$YuixwpJ`L!XqQ`hF{714zEBt zRbPfvdOq|nzPDySlgkk;9V+Hrpwy?$K%6W#)+2-#86|_i(cFrZ2+={JQnt%GB6Q3# zAV1B|aAS$CyJTS+C!$5JioT_o&Vru9ZvG2`Ee4{kN=PCTp)+59LP^L)!6*V6ztXk5 zx|87haYhTcj*;k4Y;mtg=w7vJ*!m_5WB|L2MP;ZEQLza)pY2Xg^nq2nty`V z@clxpdPns?Q&HJwr`FQs6$}%9h-ax-SgX1AXf|p?3+>M8SD7WT4&Th7wlKis=xp&t z2f~GQj8m z6v>Wrh;odI=gZF`--_WV6ZzXnT^5+7Z(j=lcA(q^8vVd9;@N+M(983v$KetZE1!lJ zuflqJ?L2uBIXOquO)E8=VSq9~9elZnqm9}g) z!=MBaP<~3wVEC|@vNHbtHZC6=GS+(LGBGejeko*1M>WsXI^l!rBBs0`e<}QO^!Z>m zvbW6b`3*rVi?e0y_veAl(E(w7&~x6EY+oxpfGAtXH--o1wcCtW7PS8oM`ha&OuzaFFXLa5qU3a~dWf-H0UzTW-MTd7+O{cK`fCO-=M8^5UctaU0 zcT(2M$`+CzF-69Y`q6nH5+QIUopS17?XNr<5xhDgmIV_RC#h^9$Vr3{yP@B{B1jcW z!nj%;Lip(hE~HAS5)H-15?P}vG5811d=WO=_|^Zg_WMpV*VfOw1Cby<`IvU_D&~-G zaf-h5UYg%-1+ssU;y+$W1v>}ns47nlzPnZ%-rz`XfMlTH5>uTfj`78R!N~(qy>z+5 zBMR#WVy|Y{%vY>MPMzo&G(57$AX%v?Riad?=i=1|XVQstC5M_a6TxkKjXZHAtJbHO zyatTZU;GwS+o8dG=nUh%`cf$7Wt)3RV(`JTh1?lJSifL}pXJo6BwN4-_v%{)JP0dA zozH}R2rRl;grfw@LYHax$v{oox=ZGI$1O+}v@m<3A)^Jq7PX}wNoq&^&hw4!1mUuj_=G0< zdfn2(Vw!84q-{^ed>-TkpID4*tvaiWH&=evJ4F|tG= zr3K8agEx+A0XVhhilS77c&esUfAE)Kr&YWl>Bvo%*0>00-3(pqd|8Kwg<3uKIWqzg zbCbR^q!yxbv0O_2b575TmD(P)t1H7(xGF#W?Aa7P$$6B%wOaVYSQ>Mv#<2{4jb7Bp z(dJWaWVUTup2Z0Lvm)*Sm1phdho9XP?{^i+EYLld!O}!|aC6Jk%-+F# zgYuW}zb~ul%61s3z88p+XZT%Tw=}ThYMB=sFVPJ7(~m4SA-_*qn4~An8N73CCdUU~ z)_uEl0}UwwEz|dv2nV@FPsW!`b<=B!|8Y2fN3@VUwe0B=4?aBk{7|%VH6nDV(>5$Q zc?|0m4Re7TB^Ou2WvGr{uVs!lc)5M$N(Zq4i2Rz;Z{oc1r0C{|2T1lhgcWBwrcz%} zi93>HU!uYLN&*pte&-K;czTww`(<71g zkQvDnDk4J~W9e^lkih{`m$}r|FDKiZNc%s6UIES`V&FFU*m8fGRm^*^b&H(uT5w5F zq2({EfD>VPG?t8J%b7$GN3Y<98$VFufknjjdnr9O&Lf3e)TlN3GhA!SMxz^=+Kj2a znpu(hBJHSE^mI0u^EbedpL1)s&i5U&-Lp9#NfTiY&qh?uq1WX&oMXdlaaN}=J{FyL ziO7~*TWjRBj-=zxGoTbdYHgzV6LdDMc9CjRK^Qmz9~ju%(VVK}*A))F`@F(H8zomV zW)(7n{H+QrjFP5TwX28<YXaDqTS-BYiE_eubcNj?yiv0>0 zG9QHfv=hxtXW5>C3f&szLb~^IuK`vXt&Xm_R=@lHOPSQ?lZvy(kLuJp0#d zc2va^jOg3U7>l)ZBKAYn9*sq5YXs#__~HM&0;0ehYYR;!Vc9Bd?Cu@S(+5LiK0G4( zTWN&EP^(~I`!mJ%AnEncx&dXS{H8e&(G}vIh8N=+W#RoZzFAw|5gb3vxH+3Hw;m?UhO97@{Gab?nB#q;HS7xJCL8U92WOj`QO<4@lRBQ=cg$qn^21CoO`?I( zP&TP8!8l{B^p5w?MK4-yo9G4F>5LPtc(>$l`~jqQcWt7AMi{p7_`3QoZ?I)dh1e)C zVfK_J?DxHC7~Gd4X|xW0 zL_G=~vC(Nt(~OU;^3Bv@@*S3kh8L zcN-=Xhz2*kHj8!m`e4OGb)^VTOzToXr2GpO({8;_%jVMKGIbCjzXK?&Ha$W~UK~Tx zkuCuh0by{zm9W=GMthP52ywstNOI=-EWNfrQ^r~@l*!M|6!*#=9u)D~RdJo0uanP* z1!c_!r0{Xr0h=6Zdn}o?T?dgZ-faT@=iTa08SmkGJb3o_`zm~u2F>@ng6XT2G$hrcUxI-PU zs2cA_-iegkD8YKy$;suR-PEz&CBaoa<`{#5g$wzuX*m@4?R@ww7a@#1)W{ zVKIRQ4iTV*V>nM{>7R?$P$s5zW%W!tVwPDShz7kc zV*1{)rT6y@HY1b`lNvWFPK7Pw%joH14}$_N#!toZKd-IAMu7^${+mM%n1FyB8-EuH zGB>QFI-Q&;Rh{Z1ptI^^gv&cVxHA-r>g*+ztTSjtLU)+e!)O1IY=v87Y_=&QSU#el z*%vfOwe48fAla7(4${pkc899PG_Y(dmSb01X4m6)GIt;rdqH(-z`2`o; zCOOPJ7ygX=qed)bF0C=8`zv@?jOk#?Sb6S60fKar*iuV-5rVA-0s9rYDCs6vzAdpj zI4%e)&yLdX#sc?XXhQHAR~??d3aWQ?VuS}&$zVGk-b7?N&2o?tG-`1I*9~fmQGnOP zcqi|B`kyqH9%?03;L5-oSqo@fBqu?>F;HnEF511_{J(ApcTfQ(Wl;z~QugoO4i0(F zbiVWwDxIN`6+m5X*pNm**tFtviFm;teSS4>=$fp2;g0p;bkjVSuU^8_8Xe0&KyQL* zE$T1sf;LMgwr!J-G!(j>^?!4hf3jUr)jeQ53kwS-RoUM{>rKVbhK|4n2oojmVJP*H z6zIENun>1vq86p_3lE=r2Ops~H7ubz`S$sKFYn>g%MeIs);QKjl1;0z1amVg;(5^q z2eNF!iP!Y7G&YzA@nW&XG6e_;chWW+JVgd?QXbzY=cGSHEE?8xIEG&4hLF$RwDYSS zVf>by)FwGXqheMN&XC=pb^406%U^8v=WOfC#6VNVEJc-Q8%Y#$IxU;1vg?(Y@S_?6 zCU!mDyzDmb1hyz`c(L zJEPpGb0#Q~t5Ak!DdMD*hd{wYr#ePjZCkLw`A6QzKiv( z;rpL3jxH%XO%rsA1^4cm{@(qLYN&Y3QqU4K5XQzBn1JH1)|e;b%Z#h6^L{gjqQliP zjqRNlzHZJ2P={~te7{w^dS8$Wj9p}4*{!cp8qi4=MNS($zL#yRtY$~?2S zISz`9dQ@S>DDazoJ;&~l|9sIgLI8AEjFX0^vOz}RBkDnakv%B{l= zlQ&f>FvnNxS*SJE;4n(7Bm-gPTE`wt--Ajswg#C9rAuMy&4>cUYVZxgqNzn6fE-f# z?;M%Ill7Czw`WOCOr&6^dP??RI(vrQ>KZF-YGKR&e_Mh7#R29PpWG`;fy$V%75}Il zVw(pXxuARKW$5Y|d@D?`sq48*D$~NFHVrA4tYO^>R%kP4-F;!+z#s4_SDP_RJ6g|^ zOL$=ZKfo9xKh(&i;);h`Bh+1#gPD2%+e>?#HB-RCeY>`S$`DMWao{JOO0B}p@aQxY z@U3YcsnwlySe|$F8T!s2ru?eO_S63Y9tFg? z%%@=g>+G=$0RCieV)&F*8VJpBhwG&6&G^R;Y<+s_oU#1bqGXMBrGFDflHs<)^nzXLUA`7sb)_+-r+DhRnp9a>;8OBJx!`o((nuH$&W@<)K|o*lym{Gy6==mgMJseOfIg#V-!Lc< zeJ(<4`ZaJjJIyN>RIRlwuDfXl97FVUtd#bFYnwI)n2bYi%?Tmi7;YTCz=Bted4?$| zaPVyV8-h`r>87yZ;nXb`=nSYiNWJ+2KdH|XJWG_su(&9~qn$3+C7H1{4EuQef{>zF z5O@jwp{UB3zwP?m65x!d;!*mi9j)^D@bDn7igVxT%X^L7mBQ4lDoP>IRz;j}Xi~dF&w|PkdN7b)HD1Z)m+yn8a_)$w z{keWWV-^um+^~o5ONKC4UZudfCsZ@p%SUWJ6#m_Dcq4i9^~nra6ygn$TdLbvC~|$D zJ=nmsMK|@cWww)O2zUi4?8t3{f3OeO(QwF#OjMDXPf(F2RQvMQmu{H?942qwQ8yOfHovY-dIAF zRKS!IH$aSiR8bHap%Me&)p_VU9+6) z$L5r*v(uDi%tA>uj(B|wT-X&sj6@zIK>el0OkD05@QyTXy&^~vZKSeVbVn~84Of!1 zd`{TlY(5UTp9@E`qaD~+Dx=LdI?jcX2T?z z_4c)+!mi2-kcsH;FuIFC?6b)nX;Hx)(ui!Mr~P-(S-}Nz$YrUI9jHsdEi@#`sztw2 zFQIjo!k7}Hohm3xPYOzq3D3nW!&H|d^nEWXfa!CUxD8j5;$mlb`B7p(F3BWB?AeC( zE{7n-&-uDx7j7eaSOwESZm;PXw?~BOEMP=L6D)fTAM) zmvHg)AE-DP(H2sxVZZ)@Kq_`C;OY5@CWXT{#ESlDT;#+3Sg*sDU2n_Q+pM$1+)`&M z11$w63CEhyWe|V-0!T3sX9jN0C z+w#dI5n4~lpCY1(%KgGG3VpHVRAn-;2J#8Nc&_}hlRz*k)ieRtmjG{tMoHn(NNDB^ z&bYOkzv#MRIJ@Kz!Tu)MlBE(6g;ShR578Z5g5S~X+Wh%&1Pf5%#1`mK<%7GO8A(m# zsGv+gLOQv|D99!!v>4=xHwuc2#gO+HuXkWM7K}YzCWl9(92nV?Z-^qP#7A5es_uUl z*WPdtj7Sq?U?N^e5C?b9+mhwA3bFU!TV`WuAc*z-yD;G)A;d4-F_)fqOcy^2)#Km< zP%Ooxr+3k!ZcKt!l&{wRhhQvp*yJ@^E*`nIurLSyypa#lyTXB&+#${8NTuys zrdu?S@o7l=M@DH4(Yc=Mxxo_jV0Ztvx~|)J4-9p)gUBMaLjFv?V|MB}`;&3}&*K>IuAO(afWE7d$UTYU_TxoMWs1M^xI_v-2!C)hyZ z_aIq4Qw5$qt>A(?8T`;cUFwzu5|J(@8)4^M3MQK24)_DQu4X~do3a^8Pu|PS z+Au5b^6s*+dXb1pr5L7Xz3N7?dNIqt`JkB^{t9%?1r^2;YmtPj&;JYrXwIwgn#Ra) z>NF-Zu~a_Ju~hM^D;QghOLUJe0&~QL1zFfT<1umgvD3y-r}sI&)W^g8b&BY&GKKf= zD+B*+w0xntI<#Bv{fW;~-3tFct!6WD3MNFJ=gxJyFC*6_L>(KwLlg5qZUgDC9d1O7 zrUBYo0}vCPzYyQ^vhj4}BWhtus%hpFIhN?+c3%*weiEh{-k?q^fs$%r1Zg|O?JUJ^ z;>9S`n4gM|^jlRW%9d+WTqcg53b;u-*W7>fKWnXcM?zSGRUob11BzBJ2hdiWX50zb z8Z{9IWwjsLZWuJpHtS`H$GeCF&Jj!_FslZ6C<9Mk3v)=5rspF7C%!d5`ksHETK=s7 zf@TvUbe?TZlY%sb+Ac8ROx<~Cjo-xR6WOPMn+Ym}%0^|w%ysy_ftxD|Z=T2C!3eSG zt??7g#&bagu?4jM?w)Y`aDzHHZ7TT=)Be92uASicD8!4ZdD$mkt@(6kYU!(K>Jks0uY_AoA32B)TNBNBbVv$T6+?YGTjX&;EXOv0}RIy)W1b{l!8Ns8yH0 z$W@z#&k##984^ZyZ~8ZM`|ltwngsxmP{mkBRUNwv&k@A86>|kU7iS*EKuj#ZqM9u8 z5ORq4Fz@n~<{vmK@KT~@@GufZ1v9d#-1Bnb`=V!iBJv%&LJkjZA!HE%8jP-VsUJwC z@e7{P-IS5ocm}VGRyKvGZBnsQJ{-5G&m8D^l( zwS3Ar+BfU6^Z;`~653h`M^(DvIUK)Az-?tmc*jq2w<|yff2U%=(e3dvY4760kV7bL z>CkmS&qipx8;wPT0K|NUN*9)SU3$mZ`qw-o+cv=r(dV-BayJc^Ws&2d>y*bG*>nCq z$mgbZYJdilZf1?_OG(sTiu zZRNQF*ppLlK#0fy|FvyymQ)W>RR~8;t|TWZMl-e)Uximcf2u5IYW&WpAGLRbTP=t3 z$`!^nzte|FJGDqB<#?R%O|c@NW(Tu3Py{ZizI99 zkB*r6GmQxm)os%rbi$cuCkEu5dotCQ4ve!`_$z))da9vxHMbqW>nsK6X}3G`0V=a-&Cf+bD47TEZvC%d;UPbY&+UX6PsHhPn6E&I3|jMW&*kRL1Y%3!%4lce9d!SBsGLXu>nSd^!YDtZ(%|USaf)!s zDYin=!hSNn9^;NhzxfZ)KYlY5*^fN2B-N-gp1iKLeNDy5L$S48mQ6`7!EEZdT9t7p zZ_yJ$uolT6)1;u`4JC%!ey)U@jp8^x6LG-EpcPyDBc70@pl6Q?FN|T?h9{3zSLC$- zpY#JM?|IhXcXG7W@eyp8WL;Q0tc5B?Z+4qB@$ribT zp?ko1cUJ|WVIy%Uw#6tZPS5jJ9Y%?CI?!GuN-YJQ52>H>CEjLalj-nSsqX#TS7EVT7 zkVfO1tO|%UWxipBG9rw10$t^v3EmV_8ZX9&v0yR_Ym@x3N)3kk!l5jZF_z4MD&N;% z_{b)+!|R(s(CNf<%K&~zM89r44cgOYu#o;#0T6P%&qOmn&roX6P-4F#`ryhfZ#?qz z>I#Mel5lSf=56AGB+;iAZyTgjWcIT~teiKQm1T=$ZX|4%&KfQZF)QX57N8~q*vFC5GkJRsj?m(y=sM>H_>8_k(0UjxK^1~Kp|%)9!%Q@_7q2zp$Uk$`2!APt@X zg1T0yUHz@Z3ckju@6~+ZosE(J%zKrjPPr4FqLFH7>A?*QHK$%?2IhPK6t+&;BY8%s z5-d7h;#XyG*<8m_MKju(wvM$iK=}wO&_w2D4V%VU8V@ zTZe}h1^S+n!adx2@S?pBU!IMVI167kHRkSvN#<~^kGLxsd0rRE647wWpaM+6t?LRyNVO9jBF>!%O_~tE!5DiO53NZ_d+7^i&rREWi2_hWPKz^RGX^RI~8;gjQ4sU%9lU z^Ut7i_n06YmKwAz2Ird7`Y$I zqaZFVsMZ}~w|J>Z9N-;c?N=^I%_OmIA-J7NP?_QiL2|hLuf*4g*(cSrl4&>0d(n@B zxn-8u_nSs$rk7Je;CU||_s`(G6TGe2de3_nQu&;Nf0A>v)YZ$MC19{$x(<2#x6NSg z8D#lHpK!I3lRszuemi*p4>ftH>3Cz5`f*kDkt*;0E9NG-Tv7hv_&zC+?bO3Vh&CrG<_ z49fp^HQQYRhCXEX)SB;+o;H``MqXireh?aS-%*(@WEcbdsFX21C1wBl*+Djwr z6ev14adTd5nCmZ4dW*N1OkQ+I4OfA~(}b`R0bfMHgf?En0&Cj*Vc}N7#&jnxhsn{& z?r9@`+LwC&CVtE4=d{CTD~4~ZUTD)Lx&_kd%>euJ&}Q1PrmgtL9jJwvEEZ2^EB4`( z`wq}6K@c8Vyo2oj$Nky^R)Wz!CPY=DvcXr9#YKlp%cY#G^-w+$a{~6h8e9D1j+UZp zDY}^$>70!EJX|@R^;7%_&`ybhEe@|s`Oycr%!J~5W>>eQF6;w;pzgfINVuZ&B*^G< zy>0}t=mx!RAZ11>#;XjSLlarW;8}<_ajlXCnb!S{APlRX?S%F8%!JFF#zB`px-$lR zOuA3b(stnzV_!CAE-49*^*Pxr(UFspVTK`b{NrC=P~jCuKo;xXm;mVBpQwPMpDtS5 z9%Y(7AA)$=^~SawZWI6&__R>X<&#Y-$Ap&jj25&@bfm1c zdcfrC1!3UwFQVWZmWB+2)ST^-D6T-Zs7r)|AVmG6&BB}=MiQX?1$z70V7}yes~|$1 z=2076Af=W@i{fNrdmA8o$-cY#j`a6eOT>(t)Lh|IMl)>SgrZ)3JaSUdF)@TsY!ikO z`{(bIjFb#~jI1Jr+oF{2;>29_3(Arv!Z)@gtKHJYUf8LvYYZ>rn>gi^K=ymJbH63F z1sSOUQ193+K$8xW*2DNZCb2l$r1fJUAp$I?;1vKexV^!pf|w*^Br)mKaF=xGkpv>U zfCilwX)gN$k{fOAp1c@Xr)vx^b;#aP(G2FlmgV zIed}|4dPGjMtc99*WABv26WHNm?}$LV~Q*hj8a;%Rhd9W^)@kti93?Cy-_+__iL8{ z;^EbyWlVnau^NpLiGCJ~VCJ^;1PlA#nwRGEc*K1pBKDI3=Km+Y;sv=&55Hp0B2#a>GZ`XVBA``M9AG907zuQYxwQ1 zcfT8cM=1i)YA`Hb z78_iH?!KheZh@6k0%CEGb=mL{2PNNu{E6aoo^buM#8MM zY{IfbR7a5$UD03j8^sIHK%#l@{p`_fQS$GSe zHImr%VY2nx-78LM5r|^2ueC~adP3m`-C0hJQ!=kx$A?gZRNvlb;htoUJZT^aO8hzxLRX)TiY0_#pd7Ch(AhekyrAuE&$Rf z-0az#m%gE-kM7S+%@salk(azFX3)p>`Em3l-*}aBp4BP9jmZ9w(?`X!X#)wgqc%?1+5+E#y_Up_YW z-DRCM1uHfc>_TG^_VjC-jRzCH3}FGTk@=95MLd1ZQp8u0dER+q^A6FBP^Kl1_g2@d zZ%bW&;IINoxPr;^;E5VBmiGd=;(h9N-{a9|hrjFX$OPnbpHQPO0A<=_`C5h|o z=a|w=wif7JLu{D294`J~xxqMu5ZOtYFm`>>2a8%S3DJ^hjV7cbAsEdnA4TRbJAp>y zYy-@2F$FcmicUqX)zJ9a6v;hx#NGotiUm%0dqAQW-d}`N1&Gq}wJ?X(80H!Piw_*tp#Ia)Zl%x^xQ&D+$fiNLX|IyAPXvWCz1hcdp!Xhu(&JnQ_R z$WV4pOxC8c%=18`tW{lc@^Ihn9naeP)Twm0;Aum|F@1eo5jdkm&&x!r1@K;E)#vFj z6Ls9~M~ocg->=e$`3YjfzM1S+d+yvdw%4sMHiKLI*Eu!-x$T_})L?WD!bvx>y5uFztnjrkC{1HCw!@7-R7gTO9+!AriDEazY`Acj4vW{jtC|n2tje5dltF# z3JRL$?R7%iGE-myxeO>WMHqBY$${UL9ZG-eeSl8{3Oa)XAq*Lo_HQ_|L5(LI48PU)nwM7(En` zvlCcs+`7TNzX|Qis~i?#o-{J#?c(Z~L$#1Cm(F^aqprfDfuYL6{QQhz6tMf!orIJ1 zU#CxlCnEJwT(mB96G1J!=Mm_zWKm?5lo~(t_ZqU;nstr}!9!EX_9{GXQNRO9=HgtX z@K_y1T~NT9|6Qp&_nO8b$oRt#oBkR0IWY`Xzc(a*4>u;+mGKf-L7Lga=t9dY{rZ0R zj(}mquo|R+_dRuwQo2NU7hf$+v37ve{5tG`Pp=L{LNemLIz#Da}GlrXzn@c)DcGC3nK@L6{RoFb;eZ!p_31SD{;9 z>5X;R7J!4kH1h0nd~{0=N&QvW5t-xiyVgsAQR~GAYykT>J=f)v7%#8SpQO)Qtap)x11(Ntpq#(V0+i-y1@f;Iv({N?J!pUD{BGRd^xN6ohw7%~w+wOUz;Y5qGf}5|c*{ zsZo0IcHejhaG|`Zvf|-!D}36D2mJlWf|@c3>YRYp45__(Q3&Ta`KNXR5C=J#@<@KQ zM{y3YAa*nBz3E1})6coxIGDoxJ)C|3?nXQD)FWjgUD)~ zur4pm$f~~zo`p2`Lnp186i3LsT~>?kDymNYoT}<9O!bY(=FPSotQ5~dO8j=Kdn0$~ zjn9D{*m_KWSd@aI=uS@r5aEC_evB-z_NrgvhPRep)o`8E>axf8c+uGdD#T#swC5d8 zaU**5Df0{rk}bN@E!%3yvhRuUC+`b=ZKBO*& z%0J=TTy4?vi2r zsQe~sI@R(sv++G>cnW>r{*>>IDhlVA&WEETn44~_*9KZ9sm-!mp*&G?7^0~8c!dBt z05(a5yowzM4!&G9S>;#R0o5t_tXB4|&i0G+*L*f>Im6j) z-!s8|+0hBQ(;qkdJR3#F#)xC12oJT$kUr($(g+05sPcdAFA(F1g_jVwLY}tYlT_as zXGU>OY(tlrvAMQW;VP_L!@MUW=ooxE1_p*nEh=PbQb$4Vk>P+l9e0038<^FPu(4?q zqC)>Qm=cd`Ze!Rk&`x9wxw}K1LsNek(sv=H@Z%W;dRv0pD-pbA;;)n$tXHLdWla8X zJB#6cKgbhT?t2D}YAgl`!!M>N54aGGf;sR+0XHrvd5mVL;1)J}eyzFe3v{x(NJ7*< z#aVNhGv7na>m7%ho#HdTgl>5F-^ zQU#(O*!m6uPZ7+iAo<5M_~Tu5$xXNE5Yu}!OJ)Dw-o;bDx!8f&@*cpTW|p06=gv7d z>!ZLhu?=DOt5WgVMe=36G60X)R6&r5O5W-^%1@imReO>WvBvY7Ro0gGPp{p6`u(#7Fx@HlTtuzZEB<++3JC61| z4ggI+vcF8Gi38GjRF!;@6Is3HeK|qw(ppERb7#9V|KKmDcmxnOYgiYpOiWzys@}$% zL4$#dDMbqX)6Bauk1Kryzfi8;d84KGOA831SWvN^)N7fP1bVF~_1W>?COn~9Ctf#R z^HJp7`nBP^%!jQ#q5HaorJwnhQ^y>+OuSk0H-2h(+7A$rY>;;t|DTSruopegTkG@3 z2_n$rW##5~FW0hB}^G zu^2$+^h>OzayW0oegG8zaZmo^v-OCNO z;n3&Sf*pODG02~D^K>2`z+QTC)LuRRa9Y#vb<$>x5^-yD<(@UTGC6uDNdlowdp$hM zNmMbMU1P*Zm(w?fVppf!`~4gVp?`1a()xuwJ_-PhC56H*lE^JEH>_ddZb-@A8)^$E zn3my@&kt_?h!noQEsFWlSk$x|>>0sh^ROEv_*tFFLl#+*aNFX4 zjt!k%Zz`LP_p@<X~B()A-VDjeD;>g9K9e;90XDj9k z4AdBc(dpo`$dR#aT%ITHCrcP?>p-Qnn{THfG@s5sBlpf)iD|Z6sTqndD&*jjH7N56w-(_L2pgZBRX@$`MT_up+tQ!PaOMwv&@!h}Ff488Dd~Sj|6A4Q>4c%Y;u0 z*@4TJ?|}%U3z8Sc-$-`#Nv&3VMe&+iQ+rV8}$To&cOKoFE@_X zgbykx4C5;F3;uE{@8Ex0O37?t=f)2?cZs0=A*uV7-thIALrHycj-U)5eyGNxz)gcP znNFr#xztD^8V^|HrH1y2QEyvJZ7!4ZI@~QpY(*q_dd?i367To7ja7(PE=`=o^M;a% zce{Gd)LvYpA6J@!9TE41sLBW0zdN98zx8=A6N3(Nz?QByO!yIHaJRTXym43q8kSJc zU0(RVRXuza3VF5F5J=l;0q()I7k ztY&^XNYC*6O%|A9MyCJuP)6Z;H?77y!?1MV4b~B@cU)KX_LRj z^?JhA>wfuGHzWZT&orZQWI2tkUMFk+PiUa+Q z$UGu0i^$6|;o5#d@gT2rSE(YlwtQ`?1j2BS>~iA${!AKN!QVn&$kvzzD1I{@iQ#8F zwpw4Ax*Ao#A-7PS5O=>wAH*q(D-(g{Z;9J##a0y;A>Ku24#rfzpRd}Pz%L?FGwyvo zxUNo>`B-?lk;(cgAi#MS+O(oIr{4A|Zhp%qehnYhJ)#Q8th3FCokUZ@CO7RmX+7j26e+}pmm>c6Kd*-xrGOy}I3brPUWZ8!-j z^@+oto`D)EAYY1W|I3r1iVR>Mab7KyytO_um(=vg<@@LgM`WW-wTp#c*etW=9cc({(f6mFTrNTZ+$-QII^$gIhG!3OMhc{ zo6Czh^;)}w-$fk|LhNK4?yAt5O3Jg`{R7$h{W-l&WW)uZwiXy=$;)WcpL$ve(3-8~ zr$3`9-4HyBk)lC;^_64_b;`eLOz3`~XqlQr1E2nq9{MfT&#l_5U`Ele1O;lJB{VJ< zhD0uHn$M%;Qj7`5B17-4ULEua8O#r6U;Dr*=Uy|trO;=*~T2PVr=q-|q5T2OK z4)zse^ww&{;b7MZ|b|pT0y0aUlNEggxeN9;MTJnIZ2b^+;PEE(|IHOkWhh zOc-ZRgIWRb2MxukE}AHdeY&IKSpl1-mlp7x5oB3p`P4;zWI+wn1-g7|o{RcBPCzQ} zoL6=uYnF#ifJu+kBfXU)#94L-L|}L5Bgnr$8-2}9s1VVRrQ={5P#F{wfw14VA#$KU zCIF)&KG7&LMy~{tdtvwNo|vQBdkSjh7aOfT%*eq@ib96+Q5Tp}`@1Iu6;08==6TF! zi=3C(FmjKhbY_C3a-vQDaL{USlm;^OczHy>&!)!{JduAXM3_ zp$>VKP%2F;ghKVSKI9vX8wcEWXJJ z!a`?I-A{pHcXAqe#uNkaQOiq~)d`yycbl>L_UZ^D!#0pB| z#;N>DUs4ic$Sq zosGAhX>)#5j89Q{hR1U{` zAe~o&ebfU@T6RI~C!f}73m;1l&TJQ?+;QkDNKUCLiz8G4qL!Wj$zb|_*X^DX7QFPb z=>Qq?oNt|%Xs28W9ko*7W@{BBqSoxXks*O}x<64HYa0ru`dOW!x62!6?m`pP?M{x0 z#23CpmZ;TMvCa)+fR7#v*2YqY?Z_UIo`p;XTC-*Go^6k$ycAyXpeg3CKo`EVpr&d( z{t9#w!Q7$l`?K53A7kEuf>`0pO(>ttL z(m+Unu^Fq#W{S=7P+Iv^i%^(6yH=7V_BgYG=eCu^zBSg6QCt-zB2||+21R*~*-X&h z=dO?^_~RQ`B%}ksHJ?8B*xqcX5hDg4b}`<%N73WV!q?V{)Xf+_Fc1f zKZZTPqp^*xYexk^Alg zcB$!F&S;Dc4lPHI%eR50c-_+v$Lf(^tM0Qoyb8H<7Lga~ z%*~Yzqz|SrLh~Z4}$aHr?*!hejl( zfrm>=FMG2*b#XgfZFZgo;JNJbR%gBt_X2>`6XV6WD{GC5Ga}pMQF7H%zaX&5b#x0# z<#nn~@^UN9!#zLDi`43|+(|SRO`n-}13epP9=XgMeL^&Lc17C#p>)66`njR%V{C3$0l00T_hc6di)?_WOO18R9K>CaZS@{EZWTox zu$ibJJ}d7s(=TLON%__M2xgP@64%Y$7;Oi^{qFRRUz5#L8#xcao}G{J54!~8$~xPJ z{#%f?Icy4O7DfQENy>&K7Yc4DL^9|%t1@ZI@-z#pHueBLmtLkX_W5~uaQkUY?$L4t zN`sCxqNNBY_{5QO{ncA%lF~vLe5}fCv6(fnrmeW(hzbH1WD(qohRN6*7)`HhlXfxW zXxkT*-ghYL?!@$@3Wqcqd22q&8A$2OpBH;(CUy^Vz7bFxf-xCnmRJ3plSN2w5Ypl_ zW$gJ^u|uIcIFWflpR93Vf5Nv#E_}9*)WFxVz`(e=U%DH8hfh~s$$WzDs z*)^9){Zlpq?83dimxk(NmW2xwHiZ#@SJ*fQ^3h?4ULx4b`3F-1AVJtye}w;|CYP!e z#3wR07rzne>tVrjDitecmL9O{o7481u1h^}08Q;^Tg?FZkJCUY8j-OZwCC2qfy_?F zHTcO~sd%8iTLrfj?%vq!IwI4XJN%@R*QjJ8zjS@M&kE*QJF8XcRWeP$tj0vxaDn5N z2xm;=Pp#K0=Op}9L}x^NPMFQSRu_jxSLiBb*pPLb(G1)? z(qv`$o1=0Pl*Jy2j-n40vr_#J`;9cn>by4w^Uh`qXPx6B*T}a+bR(W(;XxsDN%JB+ zM%nw%AtQKStP#|-z#@>vm+U`(&o_^AMtD&-#$wp+sAhc`n6;*+XjeezKy83d;T+kAQ#!s7j9$#g{8kn?1=GWV{~(lE zsaQpV9c0^hApMdHgv+}ZkbA&#;qIfrcKpm88x0h+@8SzLkJn3<2z*r*h0%Z1QaiXD zGPrPm8JGZp!ai(2Fnc9#ZF|=MLm-eZY`fY@qIu2sbE{Xn1I@j*QeRyp2l6O%Ec4db zrY@~sE%ouU=(`MC!NQR-C3yZmxYyjIE$$&PQ>1%UnavtPZdv=(^IUg^gyMH)DN*2N zQ2TL%Jz}oGh0Rng;B&SYdoI-yTf#ZkgPqfm)DI_QOZV{(ioZDj;U`p$uzRTFO*xXo zzTG#hZ|P~bJT*0OEV4#VKe6sp)9r@#N-*MW0WqWKJ*WQY)S3zF6#jm6P!{SJw!m+P83FOx z=g&3T*}x?}*jpwj7kk>v6rS9YQ6p%cjDpiR53uZsM)V6T26#y^z1uijz<-{EfN(rs z7c!n9m`yRolgnBb6&*;16<`AXhq+#40cM9*dT^Wave{-pPFlhL=h_6De!%5MG@PbMgD(?+mdqO_< zG?$yj=!)fTzrfbtx;c6YzPImAVWq`|*UiitdO5AkkV0G%!|=q!qo~otSD;AIDzbFe zKJ^E=FHPQ zATj`TRZ5LrA2!_aK`c%bA$b&b-y2TQUFafCPOWApB#{|esrgU18~)qO{nOYOMie34Nt{IWrh6q(iEPh_g>(_1e0czalYjn z65#dg!F8`{CCI;}o9U8%`-NSSMF18=Y7>Csa~6h6;IWZKp)SZ|dAztWWC95dlAU8YMG`ghOq zH60m9e>Rg}XMB52l?-_KWScarSFp9-8c5I*muBa)F^q2Zh_Wt*4R__t``4khaZ6RB z`Mgysu9>=;;vYJ%p}R5tlTAW5f@Tpc?(YyQVMiqc9P(3Y{XBSeYedlWpyLSu$|$*7*$cd5E)$7VdMf+x1etUZB)ZFsjrJAm^##bh&F_W& z(D0B4ZINT&XJM|j{()gWZn%GgHpF-cky*k*EJOE;8Upo;vrQ2~itlzhzeJ8m znug4&v8!qtT{d8*_>MDViG8oak~A{;?5xVd>_zU!WNZlxSK-$<{;eduMzSF@y!La~ z+AMD0r4nG7Z!J7k*M+)PuBL;{Ob|Uv^6J7pOZfRII~)J4H}hxl0*O8lVKd^wo$=c4 z9k&DS#+gn%YZ|=s*%z2mb;)1ol4T4kKM~+@VbH@;hc;``Ku|)%V?k0UVipapC1`%I+SG2X6lpI&{wTD@c^rsb*{!sK{J(>m2v zZA}TSAdu|j1+J%799f?PWk-ZQ4!-Zi^sbjRLR5c|7BPp148Gob=!fGV+*EdESY1RS zF)@0GY{p)|xFEBh$mk};;*&+mkqe91Ay40n3Ml|H0|+IaVizf`6Ybt-ahdkDTses> zjpf3F`RJj!1>v8zdz)%arS@}?i)=m7}J=?(Um!C17TU(vbT z<>mWWZ-hGi#>~Ifl5Hr5<3t4Mz9LQpEXt-@!x$0bnmqdra6nmOBkWrPaYbq73X8;F z+)OJC`BatxcH>;$6y~4ZUc->a^eKntc`8mcE>xV~5o?bJa^V_A)1QKXhc>q+HC;Tm zvzSNQrMTktJ{{^_UeSt=xU6}Y3-_Ok8h>%LZ;}S0)z^O=i=>yMchZPx@28BDCWU1 z7_lk7VQ*DnKP3mLu`3hWV$KFrCYs)=0j~sE+6pBi*Y>5YD<+SgF`_q@I3uLrg_$Gf zT4lB!lH$jbPz;Ej_Ggr}`W>0lU4{z`lmYw5;3vD6E~h`uoRA!kOLGIWWcU^Wj_I1RKALFW9{dU{$-V_6& zrd?urPB>*!ER*&yo-w6wk_R-vl2$`d=OjaNTVSmj*SX;tBjVtX*q^u+j2CxdU2EFS zuk%?=OmLAImH#kDtu%mrmt26=Onh6G#VX;3Q$!*UOrhILpz_%pTj~ia&`OWFzzOiR+%p-L zx(v1I^43VaUu`vPRx-oa@z*f6vRiEU9X>(7nNiA!!fdoGz;%VXsAdoT(mlgSixP+{ ztp~a>$U3{Tb~CfEGA{~?<(<#IV;d(zXfoZ5Kc+vwAC&SLlJ>zdmX6la^gl#eZ@%x* z)KiZXonm+-yL!3IRO?w-XOTiH( zDN9c%yMx14t3|bSY<0G5bJ{lEOE(0-yXsW*}Yx3$DPZN)tAo(8I zr4Yrj2@8;xXh~*SW=p4L*Ei^h>BCcX(UF*jiV|mNpPF?3bKql${Qen`elE@FC zBbBZda4<%8vi8AOf%PaY9*%>!Ns$r}!tPVB!&?>!hm?%I`Q%uxv( z>0*blX7vbVs`#311HYSJ}3;|f>xWiZg3~cI?XB;*wozqlQk~8h1=h*-Ysp<$^?R&;CN^F3~s+W9$tuH_- z{ZohHhA?D=`jP|AL|;pva)n0o-|e&nQBw<9M>mM+{mNb zengqH#4nD_2B%$;WQ-aUT7AZUnT*+wRiH9meha)$!b!c8t!=iBht>UOMNw8=XVJ8+9B+u^a%Dg@am0W&~)xW8M8G*V-n zG}-NT8>1&vPi`I3hXe=-17HDIyyVw`0XI2=vvf4&oiz_$LWmV2shC zdDmxeqLAg9T@Vu6O6BuN8`u=~AHFzPhp|A_l^A7Pi=VcwkK(XUgK|z=VDqX1DHVAp zlJ?JU2oA=%sOY30g6}LYAgE}}w(M_@F!_iQFtJn?d2&N>7h*3+bGtxq2e*sK;Tt5^(L!if)E1eX;^p+pY^kAkN#q=WoYkO z-{~^n4bmoFy!UQ!86#>%yh>~}akLbB_8D&VZcW_^4XuxB3#N7zLSkMX)kUVm9j5#I z&Aw{hqu~?Ku07S?)T7l{&Wi+EJb5TJDm?W)}c{_TLCXXfde8HtiJCP4c0>TMn709U%|MQ~9c)gJ6RVa?<&CqWi(UxOQy^eMO6zrRA@`82l|>&vs+t3e zs#fvb;MCz6rG)V)ACTCesx_Ef$M6~p&=Xeg|WYM24oK~k^>PFJWUqo zhkCnFDzCf|tb*f<4Ty|LPK@eN;JM_}5hwXf_O=~H-_tE|K35la+x|9(yo5@ecLd9% zET(xiFvqB9m{)C-4HGx{S-662<-VAeZdK4c&fvV253 z3#?&HpamxS}kz$(g6az3S5Z*&pcr^GO-=%3U^@CzW z*;ZZW9&zdS$wU31jb)c+ADY0LE{}iXmm~k7eFe#{Sb4u*Yvj;Na|TSJs$!0Lx@7M?dkJJ5?9tEK zXHy=Ap$@=ht9}rmj>SzxqKaNR^g%UF*91E*FOK41ae4il&uq<@$wk_@1ROWYWGCdm zi`2Wj_-Vjw(l2N&zYhHUIur4Qf6JzcuH)j-&TDH-d)RAvs!Sdt5DH2UW*iBOQ^`kT z#4)+XxOi(Uh>e`;zG`6ia-n;fyv6)1sdi7^9D0}17 zd!x>>U5*y)Kwz-7e9n3!6(V*yIz_?N>*MsYJ6yH|-m|ue?94d(C#2(r)qf^cU?nB% z7IHp4)$b!;5`d4Oss4tKVMf1@$Z|lF>$X)q;Vv~sNdv2Q_D4?YAT5>U z?{vkXsdWBwUtMcu@mUAnf#$DbIwE&xA0eIUd7je(2_IW90Bufj1YJ((qH-rOQoYyz zqIU#w7Fsa%0vu0E=7a4v@@x!}H96S`%h_@OuOF?$p&Y!;LUtwK&@9y}xqi0M&j33rlyYgt zzn6_JYFla3PhY-VAgdfe+1-%au42emSyH#0$BbMEAwroiN%fkrnVgJ!PCB_;qjeob zV+^)w?G-5heCA^}8n2SIez$X~keNhVA(nho+AcSOtJ156E2brme5eE{_-&SMt(8P^ zHP^_R*~^;SUy=XP2P-`bdA?=SqSZ8`Z>Wkh`%uIxDFJ)F^`J5$B;U!#+&Qf-0A{~-59`lfz)R_(K z*j&hzSwXn4r7`)L)F*0R@bKHIxmql4mFu9~0%4L2nVe(3?~m-@n{9zW^u;W#@C}@# zya6rZdYeaaz6)gveS`RmCU2XRf~wpzpcx%K(dt;PW=2Tyl^}7)ROG~Xl3AcA#iubI z5U_YD7?1YG>jmz)vNKWbM|ekj4AuA1bcNTCnH+e{)@4nA(Z(NP?xX%b$;AuUKI%8>IaB5bMj)Zur~fY zcLWWx5vZslOVpIb8hqk%L38EeH;RK5GpM0DQq+CD({Ev&*WY(l0)uW+Q@0VXKq>43 z@-PcYM=)w>W0om7JGEa?wz?KGd0Dp!?QS?;D3jx@ucdbXV8?a^>Sl(nS9|-Whizi9 zoxJuWA3SJ(#QqFrWG1U^p`^ocj`jr!B3_b$9;0aV2gBWnq-L;`gIh}p)1k3Y8oZ^yc#1qhldHpH$!7;o(>%REKi>$*XpDetPbf4LX;Iz z>c274y52C_C3o+B1K=_No;k#d7FF%ZlNT6Xap*(43>$SKy?}2m!_}s+ZgyjVsi#rh zA>I7vn&r{RtYm8x%sPv+@vn=%hSV$Ay(Pv`ZuP93-@$iocp@4h>1vPjJ4_jD6-_N`#z@ z+**$FlR0@geOXJdx542BkC6h-)w&u+m85+%r_HIWd^X{{1mT1Jt?xnakE$*wRYjrC z8vsF$UF0|{0xJ8jk$S3@8V>?4>VNuoERsUV;~UpH!wZF+yN@I_dmZF2?MB8MIb8Ek zV3EXtwbkcVQTfWL*yNek1qBo%UE)^wo8T7%Us7mQY0-*OC*_3K?Gd}*@ zi9|a0$w-GU(X}5}p}ggzec1t(EuNh$*J_QjQI!_g`Y`G2dZ9RTa_VmN>OM=AX&V^; z(PL}sG9B)F@~U7&c%n@m=fA0VguL1GVv>Rrp6vJTmWTKlW8TcvwTGLSJieDQfcqa4R;O)w}!Kg1{%Cw zFKAg0zhssCjbgb?fILb|tINP-i<{`&z0VoSBKGnMyYX<$rUB(Zv4rRPX9aZ3IApX= zcksTFULT4x6|8Pz4L=9^`CS`1T~)<#KcW=!+qqog{|C z*q|6=WL#ND>^2$$McusYiias5$xlCs=i+GIxLGr{$^eZdmI)OBL+hsK`jA@e>FT6j5w1mC*!-v#qMD0@)}3ATj-~ zeMC{6&19I*$X}6dGZUyk&n)1)A5p)=QatA87uxL>8lAOo$$aY^J8Kck-_2=clXSWELSV+_`dTty?6l zi=qad4<0%1b*R^n6jZk-2fI=7R;56EZC^2_nveKV+!r#z6=p`H2mE@~gn zZWKu9;OwcZX$PC$w!M$9#AGm=WAIvA4J8U)J@`%ZR{iNo6P+d z8eW=|x-!*`=u9Ld)lek5Tl2W!m$Q6h-6R;t-CG~=X$AgMK{wX!h&6ZeUJZ6i`&^I{ zt|gb*^5X$U_h(AI_^-WuX!0k8NHTMk5k|6m0o7Zk;>W9(amAuVpw>0yp|*>Y9i7a0 z;~b4(id^JuM4J+@#m>qtN=Oi$ZHVZ3sFj0H`UWTLT3po+KfQ27GS62aZdD4@b z2FGb$I@V)9%^c$^b=m>zq=jUcP4-ydsHZ7xSEpO}HE;P8?D=;;>MS}xBv12UQ5PoL zx<*Qy{}lQ-Z`15Y)1KB(-2n={t26yFVi(r+Q+FhgtR}640YX(LAq5oB2*~_91jIds zQNol2>AA45B}sc4e_ZG9bc22RUI@RqpN7@Cz^N!wR5yE#8SD%RMDk00`)HqI3vSNDh+r3XjYNuqIm ztN%@waF{+{Nbh^l=7>eExNoqK-RC ziGHY1nXeO{f?oWak=c^to7ch@d$Q45l_d~fbgN9UX9>*}UYh}D(YeH}2}!=xX9e#g z?+j_X0zJIBA%vy3@f6jEYp{PFPwo3~4%5Q~<|HvTc)3s^l+5Y(F-D~8Ucx`*+KmTW zBjS2lb4W+%r_v-e_s-iC5i4zx%4L~^p*Qp*=9DDTORh*=8dieF_RJTX_BB6l_*2aJ z${Fs4hdro@!7O<*)bMXXf47f8%Eu&XR{Kt6B>>(N-?C_McPT{$U?1c=q=IS6In__Q z%)iy}9iwLC-X=pVN$}y!_Z1}V?#)-wOzP`Mn++}a%U<;P-OePY5aS~|)HFvC%};yY z6*b3$fd&uc)!2!XMTuSD5ax1@VtApVPlV6bt`V$3ED|6wZ2VK$`MRe3oi#a|jr7^tfn}vnr!)R(0squ6P}8a_thtJuds`Wjndqz4Ip>Z&oQQwY zaW@3<8cCAe7E_vp@TmE02jbpl7(7`fI>$lT#W242OI1kM{Le_RUFX@=9NdWhHvGP^ zSJPW)J=e(${*J*b0E!(FoCD0w)@*WSX%inmILqGg+a#k^1N2w61O-Zu)XE%guvZ=* zk)}>W!iKu)dt|l>BE!?tbrKDFa6S&0qC#o1HEa$)bTm5bUsx+B6ZIECn7n`qBz2M~ z3&NKg4s(i~X*|!>Aosc9&_P&23fcnBBoFj>4=$G6iez<-0K?{X{WoGfXTO67i~Rm2 zZrUHlq$Q#fPdc0z4mL%WepEPQTMJI*V!-@0c8=R*DrIE=hFte|QM-UZf9a?zP&9_c zSCcP&6JWX^kINJP9>Ng9p}+U#n4qjAJJqsM@ua)nq*YN5YrHsJ0dF@|ks%z>h^{sJ~Iipt=6@`UKm+&1|@s$ zWgZ~TnO1D-`rP&H!(m({?EQ#iL4slz%vi{M%cjL0bS7ZS2cHYk#^KoaUdlaf;gbVg zEdk7o%CVeZ+R>MeMl7me?CEr_YUrH6Y(^_0&u=aGE^KWT$;6 zGB$C(sKfN1LdfmKa>la&80HXKmCOc~^KWdbi#Y^O8{sEI&|l}G>I?AOe`wFa+LTw( z=Jl`^#z=^ArdprXhK8%tBX^797k`I(H-q$wApO{4&{S#oXXZ}Ih#)2^n=dC#8*`n^ z3gdVSFTe|jZw_)ZjXshF4R5-)&J8}mV{Cuz7OXp1+_+EkgXzb)b-Jp;XzLm=d1)eb zwP{^C%na8jLSS_#7t=AbBF_o$WgIAhOZRi4xE#I|COhMRF8)2Y?%`CMfb}SjFJ;oA zW_KCHhfDAcJ=@j)l;Xv6P?o=MpC8}sXKCY}y{D4~fcNZ8;QE5Pq|XiM7^HjQcvns3 zIe~jqQY@2&J4cWFgB4Q`T8Kzhxm}PdQwK<5B;BIr)79V6>$%O0$1lcjf5uT%}13gFhCGTCZcq zD^XC(Qy6PEsm}=oV^1v0UycN4Har9GYy$j-7fHYGLp}?ICuQ@51k!kaCN~>@AYMRB zk$VaScO6W!35}#0>tR2n(-^^H$M8Ng2yhW?USNm8zPZQU*VH}a0HgH^6W7kO0%<4h zA-Z7`q*hT0pV=`n3r~`B-kqz(EE!?+Xq>*qsV2UNc2O88-Fk@}e98=ENDm$u3+ig{ z^u8(S|?lV-*&6Jb6Qnn8j^ds`697yq7@^s~>feCFyz#TCDTU|ef{@grD} zqw;S!zreQwUDgDJ{7BB^k85TTsr~aDCQA{{RK>XQUktuNyaC? zp(c{Th!`?;V{X*DKO~kHd{)nUr40oQu&_)*clViIQ&1&lRMJ&9eKNK?fS4D=qRrPb zTJdwm)xkGROeM+6<5l9=S`!*mr=*7;1er8w|9C9R*6J{$2tZbOR#i5fSQfGy|!hvat|))qbWQNmyBe-YE;RV6X%H6*K*+z}?PDbAiM;QNktV zn75&B&IXmsQ0Xh<%gCD{KHXV)*B`Dsw{A5RrjXGp4#)ABBuJMM2Z`E|;#}RnNSyWD zs6awp-n$mPk)Kw~Buv`jU1U_XT#=BG}8 zWs=H765c7#SwD0pRUZa6cFcGi*@R3xfKy^@U2SbaKMWlTHNKaqod0_RkQK@{V>prs zS;ay0Im!HMsjmEiQ0DYYB6|iVj?hv#%lVigr_^^E>zB z)G#Ndp;s5n0Ayf~dAVBy9Oa0~>giQ)TwL(U#j;O#8b1X(mrTSPJ&qB18cs!kcExDe zR$|4yBS*I#l6CA=-0qtl6~H^<=L;Ubs`UysR9E>dZb=N{O#f?(d)@|PIug_Z4@GMO zEsCnH(q!hYjo~$O3EEA(cXil)1yhmRFNcqVOUg1_DwkeTGaUB$6|VPJmw8LQDml~k zB1)f@(?CE1BaF4Z+}cK0U(iGr#5WX8)*C>KmM^y)#>^iSs-z+1U!p+g6Y5E+FBr`I z!+Z(^8lCGK%$M-tlJMf-VfTO`(%r8?Nqb!xAC=tTDJ5$y!;#R>^QM0$DRiE6(tp}C z`knipGrh&F^-&!hSGv{ zeImTSYFOH5_!_9htQbjIZ?J)V{g>?FJSR~X-3m4l*L>j%V#{af({Ad#Z@%i-MnnG8h4BD|s(306g;& zapm4(PcFIPy}-xM%LHn+12p<-IJQJYEInFZAY!vpRZH3dj&1hC?h0Y`XsfLZ7#hOe zQgTK1p+_@X>zna3xUE~FzUM~C;0nBCi`jA)#)fA0CrMFpA7Hh`>{;Tv_}t$Cz4^EV zMtF7|v-Pr%M{ZxAoZbpR49NexN9&PJLN8b+4l`$#e4HK>jko_r=vur;%6DTo4ldpw znW2B-np)IT*$bx(^;bo;I*j|FKdmYl4i*v5Y(%WJuCP93U3uRW_F@84PkM}%gq;H+ zD}<}CEfXz!9}n}VnS$Npe~*UQwOL_4_TY&Q7ToEI`vD`)nU`9e6C(i1pSpyk_2>fu z2v`eA*C}?YTYwj6Bm&Fqq-sHmf-VgWow>Vf@ai8h$7s_|n5;1EFjp+6WN(&LW%M;t zk2kIPZm`a=SzbjT9XhgbxwdvSG~matXU2@%qHgaJZw|u*-mqgqI;-s_ zq>C3yIcGl!1oq0sSci*lPk?f2p?x=R)90wWxL8FO$q9Vx{B;k~R+lG!CDOZ9aJ8V5hCPo%M z4C^^4U1Ou3A*02L(FVSUBLK#rYli~Zsk^t55~cD^Yi3f}(gZ2D!h>rV)X))^c7=ob z(Hd>@wh;-1_L;@A;s8%VeNmpDUb4@Aqs$tNj?J|_Vpvx}Q4+5!a;NQlQ&Mq%;|& zApCnYFH_7+-W1UX495xMa8NN+jLV@C;R+EsvKG}P58}IF|BWR{<~F!<38##;DN0!l zVvzA7Mq2H3uX&QX|8_5NZ1M30m8^ZPWw&~@R=rmwKkE@f#vs}Yke7AA%YFme#=uco z^MOqBz+T{_%hESZ!*e%7CblLdHdP+uJmXW^_K#akhiUi=)NmMaQMni}yXqT41^`yR z-Yq2K{gL1!yl6TGf7ICcM|9$YFD@W}5lc!jG!AfQ3$siIT9E1T(}-D%?jUK$xsf)v zR4rJwuLy0>a)CGj}QjxW#fOjSg`#UzuCceUROlC%O5xIPOt4b;^g~GbNQ0}`6 zGTurvez3BNJ{{RQMwmBNPCQMrFj#}-YO@9+TXv)>^adjUS3s!0k*Ec{C7M>eg%Oq< z8^kkN!e?W{5uWl*BdE3u((_F}{&7ygdx8bE1y48+?+3Vm03WX0{MM8A{?)Ef)c@AQ zj2pa1$lf_#1UhygW1Y|>0t|O^2igiDK&V1cK%R8Zr9(zJS4s54>fX0R;EEJR1n16d zCpDz3L`v^0+Km^tzr=16$r?xP6fnH9uSk z5Z2*bnEO_Ky262FaN6BKs%P#6k$g#JY&Km0&hE_5|}uw2Ku5UPQ6+rIa4C^G6| z>C$yuF~|w~^ad$lqNR3c1Y3LCVsUn}lftNJ^-0<+P#cS6y{=AzwLsa5ZK&KQdr%S( zb)cijQo1dKhs3mdW6w!9gar4OV5dyDitnlb<)NX zQwOOwNq14@7a+?uCR34dDQSGodC&|bY)70g|7y)hH@@g=EgeeS$c038<#eX+vcbU8 zr*AaY5A0Kj06v#$f+G!LMDl^-@rNj}ZHnD>xRD&n z&Gnz=p3WKffM2X4eNUh-Fn%>xd>cEcE9i`y)s^K6kuVnjD!H#e9?U?LG`HMsKq$P3 z_shw9N^UtMOC?lLLXA1&l!qc!n%LaZRKUy=*cwO1B|{aQr_&NRP5Jaar&^*A_Kp+9 z_n5JU-bj@L`jf};;kQ|g-GO=j&}U1Q>EG9Ua{?ehW{gmd@rR8fVGWklVMo}^Nxv%E zxx2}nbzOC_x@4Xx!lONGRX%Lboet<5y{A$J_Ux$xwLO;w z#D~GP(5z2sZ0cdij{a91qy{kiw<9ekU_ST8<2s2 z6aW#4N4nEH3XRRIqo7JZa)&K@eC!4Va(@BLkhe!m?~n-ft~^yr#l@;Dxk*X*4ak~n zMih-Tv&2wz^0#YTSCd{{1EKA02|7Ss#5zIyXFQ6kXcK{hhSavoWLc*~e<31cX>NZu z;Aq$W0L);Wta2}1HvtNJq7oe{37Jx_wtWR=OK78{7_s;iVj7S$-F4f`zk@`8xTeMS z7P&&sI4#6dC+$TE&)e>IYapWu)lfw-Z(h{=*vQEXbJ*?Ho#6%y#HwagyjRv2NjUer%-D~f)<3O95=)4wF}e&Q zgBC7~5$z6SQ)ZOxTY?#Ua)+A~6sg!-eZ*H(fY6-Lu+a3~9w9dZqeehQ6^gA3@(YP*@++(H zYhM&OHb#QBOlCglBtO$vLkDLAywQw3GpFIHzzr|A3O^?Y+@I}1F3rPv9_WS@h3vFE z{P!@>bS)KReUqS-{t!iTJ+(98el#j4BD>ulj`QbZ@ATnM+xYCQ4EI8#qZu%Cb|Xbf z0O;HiymLD?Ll1E6Q0@13C^(4xWSk9YYQV8E&W8Ru^fV+@6x#ZEK#-kI=?X~KlP6t$ zrthUC>oH4&x6gX(H1l-57!L-0&Z0?fWaSqI%K8BHJWy&JKnHlq!Dl4z3DzyWwEbAN zJ3LZN8r!Zpu1|C=W=#oSw73S~fF)Us_8wO9AA|~Zix;5U-s1@*uVO&NiUHoCJ%yzg zC#U~ojuZlCW}4N%Tptn8scipi(ZuYCBA?4oT|4I~EdNakc15hhDZQ$@_b4aTcFc6j zj@HsH!%==df(RMkMKX8Va|@Dsk`{+sSwGlhqnOyP7FSH|47S(wvVuPWEC48+Kp8$IKF96*Tp6>4sSVUAI{8 z@#obbQ`CC(K+h=IgNaDx%^LO*OMs)gcXo*t$;lg=Rk9EPNW)*d&TJt5!nz^JT@GJvD7jotuKe zpL7QwIl2HM#b2xM#HpLE9-VSN)w}EPDgmj2t_UP%?lh|nx+iJB2Jlf0I58@Q3)g!2 zKAByAsY3z7ChPYTFWBBkF#`LHRB<>Bs~{GdUqsyXNx$C2{(6DhhiRm}i%;U-wenISv)DHohD7NTST(X_Y(&sVRN5haeis1q#R5fl=qLlhPNFv& zoeUb^W0WWP;abVrK!uvLjtrgB=awR@*~$4{OCCEzs16~=iA_8M>6D3T(*519ag#Vb zh#$6XDvHUn_xFr=*J#*=M-e=%mAdp?*57KtaZ=J*D9VrW6wv4A8>SDsX`DC?!yAh2 z5BUjlEp}57<^F>yHbwnu-pg5NYf?4v5X#y7)^iNrz5213<8%kII?(RTN##myNn50T zD&e$Z?Iz;VY`(GFsr6qG3NV@_K$`I-2Dm51e#5##MQ)w5&Lc0m;Ini# zO2l?7tXdyuhKga#Cw&PiGoBEIPDW!D1!fKK-Ox^=zG!n6{D~bNX@}0)UiF?jWYyBW z*xj2c$ZJ4eJiHtfen+Io9lgLqown{8ElcqI9h1Y!XACzgO<%rZgt?d21UhX4rQpoU z^;?S>8dzWFL42bHjU_lR@!D%2orl~T=O6E`alXN};vQZ?GVH7txgul;)L%y$!To)N zjmZNnb!GHGS+lT=0vQo0x?B(eq*i4^zgkJj{R zTtx9Hn@_&h2{bbG!TX3y@E2Xft=EMi(MrJc;`ky*EO)VdbB!S|I478JwS{Ow+h>B9 zVg9^t(08jCvTchRII&y#BO~~A8+~j7`4chCCCg(6L=b) zBkfSFWa6X9JK@myT$gXX<8c1bVG1hX0Jp$r ztDjhFP|VZpxpo;YmwL{+c^BI`=`J`pF4L@PM@=CT{8ekNqXuG71@1AF>y`bXKG3G1 za9VjGQBv9Gpy(td`tQXkvV@djp`Uulqk{_aG4+=?Z6s24T9Jg2Yjx&2Yq;b75YSBC z{|Vsv4JQ|Kmk_D|Ju-#lS)I{yIq?}52i`JdCMr#q>zQ(>JPl%li*;x^%^>ZmZV-31 zg*`^`-oK)r7*UW4>|{Vyp&htJ3-^YAFD$HrCvuNQ^FEd@{FLpQWQO~P5Nu|kUur@Z zk)KC7Wzo3VJGP2SJ01r?+Et4)<4-1D`HQLV-c#gxNnN}j(BSTBMNK+z*v>3@XgoL< z2rRB-?|v=Q*75@?+0tsr$?@jX+GL8GNeu0bpqM@^RiRQI!Nd#aU5Lc3%K)-7WXP2O zgY6s?-8l*lAxDTR;7cwQ7i>HUT3;k^W&8PF+PhOJbh{)CNz~Iw*eDw|7k1><#d(ES zujAfPDta;fwJY zCwR|Zr75QrUd)ua&pd@2 zqeS#8kQz5$LKtq=xHo~<=J}?qP9)#bHFraB_8hzVh4eq=V?&lzpZ`^Lq4i@_*RDtA z2sIuyQYbg-*u;qR>k6rYXO;)2*!Zc($CI7-1ann+kvSs3{v8QL4kIw9+c6kKgq{*m zU{w6y=%}cer`Ce?S{NP?WeH=P)*B6N{UuS$+y$$*Kot6)JzdUGI%xN5e95lhhDB1OJQ48JiIgZ2Nz3x8-b?^M+UN#t#- zBW-fMwVYTlPF^4EYFkdRiT=k*A#k_yyeE=JWksuj7RHnnGIU(RGAf36^76lMCFCUH!5)s9mk8c2z zdLr8{rr|Av+=l65Vu;R(7RV^9;GJ{?M{#SG=&doCsPiQn(G_>aPGzi~$p<%426BxZ zeeFins30E2u9SR3rCcZI>E^NA&s5=VxDMjVbxR`SN_+98XR4NZErO)8S^Dv8ur>T( zW=%~6UM>o_z^tOujW{BoW|e`~QY_r9q2Ft6ux~jMqzrG753XT~ym8_tq&3LM1vTQz z=l&YWEG(1G^!-6#$PC6*Pzj}hz5UAr-|A>U0I|CgIBQ=UNaSn)vvP^LFR)Qo3(Fgc*1GmzA$7MVUNb}#}UC2Hya=4!+q z=f|p8Xvc0)hqw?rFqQ;NgWSBw%o!op`+TqUP@|ppo@4<@H;pPxw7-RPl4rAk(DjCU z#%9t1KtylQp@|_0&)Ym=`Z|OLOQxfBPz3I2wfs4>?H#65g$RmH^7Oh<*NvmbfD^&S zV60&gwn~P-fr%5zYBhi@BPH@leJP4p(rN>ADF$#Ly($NN3;}}C@z9a}Bp7aw+;ni* z3bK~ol2+1Z1w)q#cO?H47*btI?lyE9>~zIhb{?6kRaDxgdrk8T?`CiolJ@sWB^KT(vy-ucB?UMAKbU@E z)&W;-))()Bmv7E=7bVmPUz_qF4L`?zGo6Gk&&sI8HJRIKI0lWG_+zj#(0VZcZLcL` zyqkJ9|InT_c$Syb>;N?)7R)NV zUr}gv{TPD}cn1us>{VD9ZKn7WG=Gy|p+^SbhIJ?>o(k6bXs9heBm-eWh2J z7H~obhJgPBo4SFJ@zvY;qzqbdcd1btaEDf+91=n%d1BU!mcmh-m82Xw`oRS~1&-R` zD;eI@{PZ8&BUa-~YC1=6ba;%TEVh6TmuA~Dy=`JMwH#nn{xCbx+%6nLse@ZRZCEmD z6Vhto%ia}zD2i77(IcPJfKH1PXjycn`A}OB@c5`)bQT8iXps=13YZlqua9@f9aAu4 zxGt+ngA*cQW`y1>)+B2Qvhv=;n+hOzfhAPc`g;bHbFac$OF<6=!Wz9GsmF37|FDl7rcVv6oLOV%Oo5+UE{17T&xK zk|m)ksy#Y2K?PavFKFBON%Ba#NHnE?=nAhp2AV@EiE|mWZ-Y}pqv}L{>t{}8TV^Xb zj;H%jb{m@2D2Kj)#G{E-8|As*qx!Xl=WTxJoR}h8%Ki%Wk^&7IYnW0@xXlFkQEW?# zG3$4f3T+oHYz~H;@jpLWq#T`zFV*W32NR0EPWbD zYx_9d9+Thvd#J;ETu7r?tcIg0o2jqsFA_AVI@or0@Y+YMNE5aT8#+S;eI2cnMi@$9 zesNs&OUX*&UbY8IEfwZ13BFdY$su=(@YeWv6zlkal$TL z@Im_oLtT@{bCnr=x5)Se-iC?dw*{QGY|Pl`F~qpeeKEL& zgw}h#dB*V_v=%>h$uXtpdZ4)TgUBvA>W9lYYKZv2p{3;gs0{gJU-h2IOP-=c7658q zrg}7=i5rtNSB4{Rk+R@`<=VPODCOFLo!@&f0DP3&b(`ab7`g;% z^vDJ&D@z-Zo9-r{gX`tN^x!Dhx+Qcdy6wYZ49v9qlvOzU;yu=;K9#&ibbMi8dh0mh zBG*RSCG}l7rC-6z<1ewgQ}*Z3!LC4z)`O1~6`u%EO5`OB=x}v?YAI8}w=rf&Qa6b5 zf(I!8KX);WW@BnKE66a#A@xjHG7BMjK^J!>ZA3%(g;6i_K3ImIEl5O=FcZG`^wVsR zrBvbQKge6mj6HA=b}w0m>NA+b$9pU=8yRQQPdp=i&v;}%*qiYymwQi%deI~ zOD5EYZjk&lJF?jt^LB?1@g@U=ZY6nd&QZxlv*Olub-5LhOvYD-S<53Vz>RhCQh=}7 zTA&05xZG}+GAL~(p2=R-pi4sz(&74<&v8y9Xq8R8w42$v%x_(OyUCQ4Ee+={PDk*uh+`K0@Z zb0W+W@1q_U==2FE&F#yCU9~F z8@Y>9NZs-$G3InS>zHcjN8H^}Jr_w<5#B4>LM-#+Y)$w{na~l}{m0k0Hh@tJZ8?V* zy;F;al32OPL^FM*=;3H<2}W^P*xknJ5CEO2dKEMh54TIsSTUEMB5PR^ZTh>b{W8B? z`TxC(Tv7l|)qSd5WJB%#2sJ6;_xQ2$J^yZHE6hx7e$^xa9nUh?$VX#$wQi{j>@rUY ziKCn=hc_>$Vjzm9$4k8X@L6zzrjZjTE5g}Jc+xNduXmw1cjQbvFTEX4rIx~x{|FR4dX z{13R<&P<45o_)~NJT)fEV}fz`82q&G2BYP{C*B`0Hccn>B3B#zv|h+6bE(Tm122W7d6lOW(Ki3PK{oG*>4%-Lyq8iA-3j zdmbAk#lbWR|3fo5)WK~<3rszp(t-*)py_L)NC-WjRlK#$a7z(!GP~T%4~cFb`fF*C z1k~*eBd>`YBCpNH6R06gpC=F#ahdJ))#!u#H!;R|7^dL>A~*%qW*A)8=L8HX6!k>- zcPz1c<)yIVM=da272y(p|56+IPWp?a%hPdJlEgVvw{yPpKj2X@h-Q;@xsCPmqO;w0BzgIF*M5&?Pc0z`x)ffR#y)p4+{rG446Jl-U~3Y9 zh0Z1cz|+#@5N$W_rtPN5oqTNa`O5i~jARP)yj9yH=3zAU90#Rw;+d7~nts-1!v7L; z(X8Y$Vg(C6Tiv&7GC{-Mszi7-puWNuz&GJsWf%en)rTz}t&qH-4Oe&vY-sXfl)Bcr z2ib5M^3P0b8CNbFCZUk9^A7x0NnO_o#CwCD<$yQ;>%V%$S9ik8%_KMWCr?rhyL^zR zL@E%tsM}YRO)_W$uL+j_SfolTZ|GC=wSXWUlaPi!=dd&%T#VS%y*z1LdCAoB?oPo? zcv>8<-#Nu$HYO!J?|?`!QlPn~KufHE<-1Z~9sI#wT-3{~C38JU_C9=zEI&=6*F_2V zN1eesOXR~MQYx$;vBP_TNS0w>RnX)2_F`6n+$tyj)mLNR3qga`>~FC&F=15OVk-qGXxp=;&{ z=>?7DK{##<`Bk7;Xf%Qe59Sxj7qjXc8aOIM4E0#D_D&$J#9{}yz_tb`+Moj{nJW;e zvtsTTc;LPS-#1^{1g?9H(pNYWA9s<-_FY{I-)8~1r^DPN9|H@`*vPbASGJU|gI^k4_a!_GaB)X-9f{_4Q8Vd$T9iq%$Opzi=QJ9)zho^+&(`62)e*VKLD@UMwZq-z}G+&?8r!Lr!#ZE zxAQDdsR#>jvw&`*^h4nYQ65#rSTqQg!Bcq#)ahEF7|tu*`2pB^*CvI(%ehz9r)0^< zHuw+Y>m@ORe^s-8*h3G5!Px5OyaI7uQuttOsxT>$?ZX!Zb@5vDAAFM#(L1CfoO!Mc zY5s;#QgF6p5BOl+}-`$7SL9hOe_&%67G)n5W&-jgAeBQ0U(=^IotJz~Al{?Vj zP4krs8Q3B#X_}w_dOs6Ar%{IaBS{_CBK~{OdBc>_6=!>y}PJSxBkeLd3p8IiIee5 zFmBvy$?>^K%>K(qaQ~b5%EIigKN@Uz9VUKf7i<3;fEvx1n#V6SUiOdnO=d^arU@yJ zz+*F1Qrl$7)h9JJ8>6JY-^CU_%KLpBA z=v>Rl@%^x4%S{Axn!r)`3w9Vd zllv~$bpZ?B`wxS7Pv`-q(CegohFq?Fq0we6ViEz!W%1bc@IZrDpS1$v6d@TOV~aMO zSAOdBP$4eF@Pz)JKBn}jkzgCY;jR}9=5)VZM?z{0tw{u7<(2x`N0NzUt{bh6GCD)| z=b2wNnUXDN;en=Ju6~Wz)n&62SQjcpdGjryNZX;n8Z-1t_X-Q4yd0IV^z9D zX9_H}dIr$Pg=IgbhuG0p2@H!AiRQ}nU*vfZzqbTLYGbM3!{?)KOfk#K|I$rafcTNi30WuS0MQWP+ zB&Cj<&aA_p?O{dqnAs4sy6z~XW)doLH-hGR7G(^mh56XsDt6q(K|?Ir9cr{kK_}pn@Gfr4rEjLCpQ6CBjg* z7bBX%adSkDyAe2$I+OV`O)-s_G1dPr!KRSn59~H;h8L)B)K-i~G_X^;5$78r!L4QE zj6$=uH`vx8P{KT^$DLu!6RU#O#@e#C@9YRuJyU_Xb?EgDrdn{5s^#Lx%?QtD|1o68 zGs}6>4+7`?rkN3M*m40L#pt~ePu&D5g-yL;10Z5j)>Oq>mN881 zC^{(-sf>N*;rUtycbW#Q!=k*O!id@(-B9Z(sHu6!?9tN6aBAP1(6`s_a*jPXw>M^SYYnLS=iAJZjG@dYKYNCs{ zw;LeSdjfpN2#dPy*WxM+Lrt61$~uQ6oxN>5AcWog*H(Eeym(3QDaKyltIH|)wZesR z)DVLPWuob`7x2%Gcu*dU0$xl}27U5-VG$`0xL0j~38A8J!fWZrv_NUN097$12ms5r z3JT4+I7Xt$B^Sm{ovrzugsbIF3gIs45GmGP$;QZi2Ez!5D=S(7)#5Y;Eo4LGps3_@OjlS%;JYB!Wn)UCgSZrVY2SRC zeCV^7`+1GA!^*GVFyTG1THT?XBZaWrJsSLqa}Zxd4b5qqGEf50GJ_4J1NCc!n_t4{ zduD^&rzX`ZR)*EtMwVP;TLnR;`_kwjH043T6@H5Rv87j7ja-R0P`!u4`O9 zMGWsQ2OWfzz)ELzvGNQIeC;r*;dqLSow_Itc_EQJb=*4D(b`!D8p^mz;JYf^{wHTElD(X|3W_dCXRleC@%qgU*Ih!VG$kYCz;!D)` zyJPog!$2BT2(g-Zb~)$FP}RM{c+FD8j9L)y4n4~Ui!VAQmWeTR$AW_{AGdvWZ;W}g zw7m~s%NP006~qxBYpGn!Fz-MKBRzaMp%`2WM&B}lMxt;x*|n~vZ&H<0Jg>5;&bjI; zN-6dvTN^TSw?^aVw*1-oJQKBnRo6)cl;F;A0&q6IVjZM_Y23d$vo9H}_x^y1hL=X$ zWdGEeaRoFo@f=GDXj}q+ICh8x6w5x~TDh6+qqRR0R0pHs$v*QMZ4qEv|3QD=`^}6a zCNn(#Pw`l#SwwJwI%otNI3JK=XP$2x02}3|$O<-DoGC^?lNoA>o#p|`AnI(;fLBEM zr>!giZG+Mz1ieJ*J!9RO={PfePIoBFtu)Y}1l=PeyTGV=)|;2m16B|TRx$+`+6AJ6p?F{*e zRCailc}PKVH4Y~iUvZ&AQ=Du8X4mX32%;Yg`oskvH2o}Hqj3i+2m3Psp%_imUM-~P9j9$_r{HnbhNgW2bu_hi7R`% zBQ5C_)Pw(9cwFk=zTk0gDo;ZqcOIF|S<5Ebnf(``BRhONVt(bEBxEXi6(QRt(CxDa zfn|@}hxcYA{t@<#&ru@kM4Qf3(fb?OP^Z~}@jq4@;uabBS;Guuz~0eT+Dt2&@}&%s zm06ro4VHlUS(*OHrC(%ZZ){Nl2=EczV=7tasS`=EMnlAfZQhl>l)vJ27hR>onDC&O zwFlQE!FGR>c1LJ$rwNBfZ69E`l>{^n65mHMU8Bcn$H+un7#InlZ-keU@U~j$1(3*x zVETBj>3&uxGjIgtLM$ery6q=_2OmJz=@lfwRwbRDQ62;JG4+n`z01fCY;JsGRVUX1 zDyR10GD<9i)iI)iS;7u6I_d!#Q*I59I}x)c!(KoHQoL0gK@QE>f(ozy&5|8%aR~99E{rpgeS*k00E{C^IAs5mSN1tp z$!Hi+u|a3wgg>F)#W{Go`rZoOmZMVTL;A;a1-#p5uqNwuIGL%e*GsWQ~+b>Xm_yhGl$p zPyFAlaRqPhbh>IKKJ4xTEKo$IHymvqCmqZ_h^T{j2JvJK4dQ?V5IZ5`1a0C%qyEc{ z&8Av2DFt>TEWR}x3?lHNJh!-xWHNq;sq)}NYCrW=<5I7!JJ5iv?D-kanBaYRRb88%` zLoRN&&f2SQ%7G}LycX4>HK$=8D^`m|Zp2^eu1 z^qO&TQ*bbKTq?Nq{yD;S7>Jk!`mV;d9Gw(%K?U$ii;XBZT?Ea$vgsW6aiYv`+)^{{|?|qTkplyjpC5>;!&EK&(+Z&M!PHH|{mKMZsf{T3) z@-DIj->vK%Q9Ou#_%Hekr#{-0?{tq1(T=;LUZUSYXT24n8xV;o=P0ZRBnQ|tpQ_1i zTqV{O)Km-wZIlzryGa9a~PF%tOH-!y=cBR zYKUQ*>c=nnO`J#%4*4RP&)eV$LQo-hj4GLM{IgA)t;&Y7D-0hPxH1naU=X`cuTw4t zyry~g;BlIHFEh zTk8_D@_6IvGA8N-+3g5$#uGQVFBI{N{+lHKCh{bo%xIuv(eME`V8!+ijafCyF`uE` zeDnER5j?yO7o`r*+d`s2stqv9Kee=vgG(+=_B_sYKdzZWWO<_p)VAz>c74y`9c`tD z-5MmcYy-S(Pa;St^vKHEL}jY|Ie$550UQ%6#Cy)($NkB-2Y->N**LB%cf_y4 zj6?9uAE}v|Mj{`;}{OoiZWb=7mu+LfOvHv}Uf zxT_tU3NXvU{8DmaToc+NgpGjZVX?GJhtv8k;MOssEvu)~O8MEr1cqSyY|vxejyO!C z2(l1gZBb@UiPYi3kR(h(?eOkYUCVZmKFbU<59s>`Ja{I7g_S|ujc1cQdp*q;~Pj}&8u2ns})&!hIEB9Jx#Xj0v&|e4-;%?`iGTe?7t!pKmc14I_7$ZaYq@bpg|X@Q*B-LWk5yg8@P>=J z1@_IbBFbC2fOak#$#4yaskDe>Xrd9>pP-v>S`ZzGdR_jriGi2VMVeYM19BFD(tu3H z;1RIIhdXsvk@-3vg-;_sOLPG+{rXTAH;64b95=~AG=cVKh!oyJ)d}h|5PAg`xZBJa z|A_{f7-7Muuft@3rU4LR&m+FIeTI*GBPy|j-6Wwh@6zuor;e6XfpD(d9rYh$Oi z28`RWrRD~_0<}x=VhJC=Xm2>z0~;l7biCEF#+P=3*Fs6nz|KEOD5~L6>RVuV}>Bv-$ z23>RhKE>2DGcNH1cRzBL0hn4})z)NtfSZt!7%K*nm^BzLuW@AUH)yhGK?g2thb8Ax zdvWK)*3O!ob3qqS5LbfH%#*0&;ZDxE2Iu}Lwx)X#2EE8DudVq<`Byapy`L%U1Z?7* zU=;3dHaEpEYFwPk@9=4E$zZ-EB@Y+(;FzwIthd{qHXpi4`bdn&wmbu{sa7O9S?tsP ze8UF$N8s>o5&~sA+tepdAnDsrY*1fNQ!B#r@Ev!zt|fpsHtLp?EaDE*@%?d+%K*{q z5zwI*ZjSpvJcZ7tZ@fLkn)znkqPQ%tHpKfX6&kTj1z54`JueDu#Ad`Bbalj{qLx4= z{y_M|{O_lpmc0KE7#YeW02)R+Y12O7rm)6FBN8~s1Ruznbp)3+3MK%{AX304j`uYu zhXA@l0Up1lpYtpzTL(32#F&HsnMD6u&SokhUZ;>9j)7z5UxlfjPklV&?$Sr1+p^ZW z<)ybvjK)VUQvsrkBT`v+-O=U(r<-;aXmhbvG2<{^V<=Lc8UB|}-Xa$LfMvffuc5k?;>!L> z>2S;-!ygLb(!5|>nD>(S5TGEz+$Vz!!9Ktw;;ZuEn{+*8^SrhS;!kDcYi-EvQX|zP zs2zjYVVoyDk1vC_wBx-DJD0I=*B2b+sUFw z~T9n z5+=3f3u4HC@_bMr!$wi#Z0ulSbv^515whmZ-KC+&4pgk!|)-y&$Z8`Q+mqR6n++AF;Sw#tg2obEoUwE$p(Mk#dDF?;hOTyHM{QJgGH&!^-%< zvDry_Nuc}g2vRrKqJKjX&|M#@xie}DV0y83*ZC8t3slau0Zuw1`Nj~Kj!o|}_Pkzmg60r0> z$ToE`4g`{E{;l)Z23js$Bo}c%9nTe5ocIrv7`O;UchW;qz)2UZRu9y)tQJW23$g>X z4i3D(#0w$3><@1x>=Y!cKAQiij{WSUAQ7SsivbB|XH&>0R-JuT2Ukwwz|EZaCBfOb z-WIHB@{%{`|1uwSE#MFum0=DUaKZU0)}_FAn&(-6c-rWqW2}*qcr+})j-@E6 z{#xhE>&PPeNf$TmuYn3LW@3W^ekAwaV!H1^aaOZaI7~fJ)sm+PtF{Q5j;=ud@&VhB zRB=Gkk0vZX>EZWmogXVc>&{5FA6<@(3~xd#yU|e>3)@7NKPYBU1D!Qb>6%AQF=22X z#)RS`GKywhdmzOy4D?D z(ZU1ICkoizGY|@00fiM>en8sc*wU}^#U!1$x(;%X%C==*HeGH}vHIjfL{WG%l-V%k zYGf{GiKRo{=2u?vhr!B?u2D4Y<}+2*2Azwr!m^C)lUr*?LEnK0<6-i0_buMM9aPP< zOr5&qp#VD2Jau;ekBPK5l2%}7pYVZ2!due&J|BW2qMe0XGMPtqWfH+j3>N&9wUBPXfCrVBXFbEGap zq2sQozZE&H{3M>!;Bc3wh6Sxqjqi=+Wb^+@FXk{#?Nvfqcs03WSc>(Aaa4#5?&L}j z=rq6+azFyz{zG%H{;s9HGUEoglhF`ucs$uc+5Xkup_JNvu6fyPr90$vh%Vpq#`Wjx zPg1Um{pl22_3&-!svltZ+Rc|)$=sKVC96tC;&`L^)*c3Kxv(#Ep}8!40U`6buy=F% z9)2XKxnqc-#2T)|jl_wZF^L+XX>V9oi3$$T6*>cvHTnt!Bh0=-iPV-Se^cD5gWKr# zhIMpn0-PX1SoVqARThqeoQgDimhzGnDr+G43{`3dL*dSZLBfDk%J3{+#sl{cT#Q5O z3q(Om!thQB?6+me$r(hXa)xtTKyB0whJi*g!1VEk(IyK=hSg*5E=vOLjeBSFGqT1=7cBf#lRQ3f!Rwl&+*%ys=YVbK;6Kb$8=!bIugqBh{GN`^?YFC^P zMXJ<7qpP|Dl%}-6Z=NEgpsRy{s*^4Hd%vt}8K78SHjaXfB-}WArwM>vcnY7RERWDLxCaR2{l%xEQJqF z2U}6?@WQDa6)8tqzfN-k`@l_Nt9k0=iD+B&kfjqGNky!e&UsjYJR(vn9dr9P?dAVr zzj~UVYB5|}A%k!ZuaaaU4T&!aZuwDMC)u|EML@d0BaqZ0Md5o(Zv^fgR*M0v2rmKT zNo_C7jWL{EF%-e+YT}LvMlcGvP%Z@^{_fIDbLHfWmM2xH%}yEXYAn5`|c zb^q?2#zrGx73>uVZNUb?L)CU~K+R(lkQ}i|_#ZBF^8N%zYnloP7iZ+|_n%MC)J~eZ z=9gh}*-ZpY^UZp1T{xh9N^QlNG`3w6Eu$kgXQwN|&=0O%PB?m&go#F2gE8)+9CEPO zd3OML{PI8_1ado=4`_JD2I)VRMpkmFQY>bB1ZL5xRHWig)|2f^Vfi5Ktpv>7kDb`f z73bQs z%tmt3Ly#+8nu|7Q(mlc6<{timtN;&%VBF$VtpC{dLA6nP{7|Kuu7+|I$wy%GnM-`@ zxphAp=^*qIBdvK5?B<`EF;qz-i!FKWc@20aLpByYO_VGk-7)kdWiLZq*~c*6kBrMM zMN13q`^*|IkMLTt6TH_7^@sPh+fX;bw%L#M+z(@>;mqYXSWesR&iIgO5yh2w} zN{fz5HF#NquYQH7^E z)gK1AykCuKR^{2Y%5>aYN2y?A;GUd1!2sKY3AK1gShDA(VX_4Y=^pHV@WRp8kCrY; zEs`i|3v}mzc@>F)LU4VLiOutGJL`k>y0!`J3<4U5pZ4qalgt#s(PBC~&KJK}Q7Eq1 z_>khfP}9>Y=D&FuO0?&%43qg+KjIBOgcua%`=^K5328K2&usb{pF=rO&g+}XqP(>1 zLWwv+M2@yrqAZBs^EXG^u#W;x;0whS8O9!_R5h15@}FKC=Fa9CWTAC=ME$-Qfb*ZA z?r3qgsF2jz@ZRkC@`v{v_-Haya7a&YNC$P6rJb>&&%GfxgNRLLkRU9YNlZp0LdX7m zT4#R6PFBWwTA)wNa2A3Vf0%6=rAQ@03gf(JD3!5gyJ~)$M3+E@RUhqX#Yzt6~jol|eR|Ly#c|9UN8KMme(HdZ(K%|J!V(b2HwpYm*ctkt&yEV^Ak z>EM&ozvWCboj_Fw?TBaMLLf6G4#+`JcqlyUy=^VfuQ2-#bdSi{r>|Pn55S_$lNqNW zMdOzHgWyPKLE@KTfI~O=(Y*bV7QqhPEf0V}Cl8E@0S~7ymp%9DTVoKB)Bdt#nVf6` z=V`f2brfr|0Dv?8n#kN7^Ne-psn@r!mbm1$oU8RSukCz#7|ih3wJf94v^o9k{8o7v zR4M7mf-Ej1*sU!QZCrgDH`oxV>mw(<^0C@C+-n%;v`W?)G;ziHM~Tw10A~pSwjauoa{92>sK{M0K1N5H^oHOy?O}41W&xB~YT-?{~jTDLK2} zq;vn3kz9NMkKQ$Vb&C$1`IVsmH6USP)(?)U)yzirbxGsL<^R*_Oem|QMNq1COVbZ- zLBm05G3cVG0B@ta&d$W=IYnIh;L%d-d04nD+kzTQXgv2%(@@WI*=SZ{}vIKQ^7 z$lfh5DwjX1Zl4T)*w&cu=(tv z5;>ntS5_`plwQp znJL_WhAz@Z4=$GOwTL@ze6wxw^ZD2kCjb+k?8}39jT3RJYK$Ud*g8iq*7Sy*Gf2gi z4_Hl0D?KWA3e&@<3WXQ6+~rfVN^EFdE)tFAff^m0sCnD=0gt7Vaiwt>EkGFO@^pI6 z1Q4OSDuVY9sVyhG0*b-F4i27-+v;rKHyB7I-}Ohu;-BiU6=Bd%tn^4$7`)FZ=Q8Q`-HL~6ISg);ztQv%Crl1oT^@Q&9Q=o@&CR4u`y%g+wfH5q5^#NX8`L zSgfHvFr5?BLdDiF{$DM7Xzr(A_j3G~NxXtuEz&0jFdSWyj$&G8FVBc(-~ zNB*W+y&1erEI`;9haj=WroW#JuINcHn+b??oH-ax3xrf}zs{RLRr#%; zUTob&hCk|k4=>>&;uFVR{0nuOX;YvvVJ2?)FGYex8S@4hT!K)<_Yo5CAM0iN0$2b@ zm95B2I0mIN>X)e>%N^g9Eka;c@~!UH$(dHaFmsIBLPAKP@d!wTYpW0anMOa)xK{uP zIS}4G{0r6=Z6855xc-w|pqe)0Do&$2@cf3UPO(h|mz&7qPd^^7&3w8jCg-`!6byD9 zV8m9M73x#2!M)E| z!|wH9qd6rK89s^@3dD;xjM>IVab#H-j=iIx_xyx=>@3V?r-G1L${1K%Nh)Hjpdr*_ zqD^Gp6Q?d{WfUPgJd$%2q(KQ1idW;dgoiLji{ThpD!u4t-g)Uz=Bi+T=dr)_t|6wt zx#s#guIGF2@FZyzN_=hehOF?*X+Rf4G9l^qtDiA$CZ%`|YhaNJS=+PG zen$P>xDBTkJtU=qPrONxd4e}j&xV^9g2EbUfc3fz0-sbs-Ik(!X!0&;_9mQWSgeRz=;Rh ztdeI;1vJH4W&UE!_3vjMhA&VTQI!kJB9kuvj8a_ z3jYM|TRv-6d4~L<(6_34vKX)BqM<08N3PO@j6)@9mCUy*`LGFk?GYaEpFqMppo@QS zpSVWlr3+zdz2zBB#PB(R2sk8o0}Zury+?RVW|K!|%FVk)ZE0P&*}d7zc+~@O=$5Wx zV``JVt}2RYoyqBB`#wXps(kPRJbn|AAgMew4NuVZnqOr?hgBj8E$j(|UinH3dw-!o zkZRzKY+E3s=EKEgT;1noPw>i-r@f#Xz?{ypeQ&v@YUL))7U5AyD~pQT!Le(n(3iKu z+NeHwjdvDl?ILA)R^j)n%}C8wk#`_598LloOPdijjYv@sNkOi7UI^)M`aFO{-1dCy zno*r4cvd_H*BcrWYrES2{+baeE+NlEypXv(ITF+lV8kI3pM2*ga~&M^F#g<4mLP0% zjC<3x6Dq~O6*OSi710eh*1NON9+o6O7TZ=wV`$qp=cJ)Z&7O2H!SF5Lq!~5}&SRSI zKLD=U>eO5!Wai~R?up?NpAE2Dr|gOD;hOx6Rf6RweROf25kLq>r!G`Oc>p>aGX`F+ zr^ArBN$qZ|R6`pt19w~`w5tn$^J@qD-x=*-MkWf!{a3iS)5Ih*2^BmyxnKiGA8(O%)UmXpXx_Q{)xpLIG3TR#zlT=72@V6ibfY z<=T{iV>Q)PDR!=MDRE=_CS2&tni2dJ|7L&=wy*~8tp=N|GrFF4x&{}Z!CU2UFn6P+ zpLDIQe=Nw?PjdQH;eh+~vfWda?1ga^-9sKGsrPqP*2pP#&6;h(bz}&0NZZh7OhZQ~ z=7l#5VSRfDWoRH##ODl3u(Fp>a`xDayHClfuYMJTk{D)NI|pI^#_}hSY3EWUF6iso zij<1(-6~KJ^?YbQKhatw8u{t(4}A8?Gtj%iR6(;wu9+E32f1n1oBrUvKe}(8)s<@o%ZHtbod-)EDy!{fr zse@*|JY7BcdM#;_Y*na-wjRgzGyVG$%y=%vBrmJo0nkTtP318xJJc9x;p~s?)H&g4VxEPb^%wGUe7^n z!2~Y^f4||qo?X53>&PX-=v6tDd_~O`R_Y{v=ixP z<=@jM>Ojzc(sNeNRjYHNyM#w>d85?Smj0$>EE{CuyZlKwQ9M)9Mx>cEnI{WAs6XY= zrE_VUnOETXn0QZ2ltSi^((NoWDK7PJ5*i$oWXz>e5xPp$6Lu*2JLWe0kv5lY9dzv0 zO?Y(eeb@@(<_8#=n3DOv%mJw^4>^33%DB^n{^G0%b5egSnyN-gU?{$VI$`c(>nYl8 zhFaNo!z1Z`Ibb3WMDudUK0wk#{!Qm&a{}<#E>ZH!5#Dk-I^Fwp+GvlmnkQJwGo>(E zT**ytFLvCOGBl?+tTehmd^eC5_Qo_GE#Pr zm{D6_Dm<&fYk%i$W^WjuD8J#MH*V|d zcj1<-84mM==jYASLhBhX{wxr(?^cYbYSXB3w>at0>C-OPhDlwT+lXX9mh=Ol%=$FT z?eKFm55yR-Cf~4;t%0Ww;lWy?6TJsRcE&)*YQj^g|Ay6tjer~>4POs(k|?mse4<#8 ziCSrYZV|^jVtD3l+TK&!Ut8cYV6>6+ z-Ic658-wG!4sok#{l(JV-gV=i8&Zkhj?u$)+ga*N5{L0+uxsx63}``mlt;Bo4K`H+(9+R z;c`gvDDR=7yNp)Wed!E|a zTulG|fyX}6fs;zrZ!0t=3>M{5e-ZJ?_~Z+bY>Z68TW~&#GIi6$zz!Xs=xyuqjACwl>7rg@`obn#=5> zcW9oD_#ox-&8fBlgQ;A`u8_MEW`7`Y#1i%S{y$)Y1YRTpni&qgVL1_x@ZA9FaMZE;~CBIuXqbNB?L-#j;k3nDI&b zC4fR*nDcWTFL+V7C` zK6lB-hBY-co(@<<(oA4vyt*-gu(d;NnM(O6;7GKI3?0h+pueWNeKQ#cL${c`^t%-l zC+FCZ)tYS;@hy^(a@{a;_|wnZfNIJaKD>RuEM)N6s!Z?u!lOwxfm(yvq??GI)!zEH zV$jh$d|hrPf3SAlb&w(@lnlV>y-xg^2ig_O(@+upcDxc->Opk6A)iMgFU=ne#N$0&`ZE$8wh^4>PCGqhQdbk{Qn?~QyX7ZxBVNfZ7$0A~(fMi~qZfW3msyw2?S6U81*=c79nsge}OB=ysKw zx|c|}S7Iwc_z%{G&=iToTD*1k*p85&|jA=K&c0=lpJ*FEa;O8uNRPqN~SH^Entv}j7Q6D zO9Cp>a2YY!GUEjU5AM{SAU#BfeQU)v;+P+~AlZ8`^Rrs+de%_nDQW=D=o}rZ!6X#z z7o$l{siVWrKElon0cc^z+b-k}If_S#O$Z}a*cuhKb2e1vktkH~^EN9aE>>C%h zV9&?utT}B_J9zBpURr#wh>tBAtFNxo$yjOr{?nb}Pcuv8YO`94}6AK!Z{?-5cN^6lc`7{Jw`}~k;NJz%x$#bz2oF@^^DKGB@W>CqX zI|x^zhfBT1RfQw!b1I<>yX1Ihq_P;19|j~0c=bjg2JO1hX5|2~%u+y}TrjKe zv+OK6sKV(@)e7|^s{@fP^=!uUaNPXvtCkHuGx|rxL%fC~vR@t)1Z*L$WgmcDC%_J0 zG@GFd=?M@b5h_qZElb8#FdtVWKEYO*;NY9 z9tx4}P%KWLVqkG^KD*MXZbsKiLA}q+A)9NBF*pNQ6!dJr(%;k_O=?3lA2s)xD_clT zj&H#d14g6ij(&@QVY4@Pwm*dB08w<^#eNeN&i%(?JwdK$P*7o78X`$4+iZx@H@a*r z9XdK(JgU3ITZ@{Zt1#@-Xg)wI)2Dcuy_BXifPo3|P*qFAdiPgN0aBTrGnLwK-;Ryc;se z?8H7@{P^KZcg@aC}W9sXVjV{dyBu322m6IF;T7NoHjtOIT_-89oX*nBg0MY z0{AA9ED8L(SVA`(j9=Y%(s(&BBZF2ATO;P5m`*5L zM`d)!L1;noHkK21_EnO!xQXQlao#El8vPQgWVu$m>w+A;NWziGb}Y7_+xXD#IZ#^i zC3)9LTPmrrp47|Jee%hH_?k?!sxPS$R;HE|#5x!l73xf=q)7em6l1BP26x4ucM}cB zJtq~;Ns6q=we%C%8?l_AWRd1H*EW&sP3!#+dl!x6v>0_?JA6}@j8XBKh{9rEZdcAo zC%Ypk6wUV4v3S+At-ko9(m}@qUQUatvuZ|p47BYcxGE&aQX2{6ZBHNi`pP?OiVdh+ zq)f&3oLPoKU+8A{Po5pvO-_^?`*sj1volkrNJ3$c(mpQulfJ~l&5wsTbZqtly&N>5 zA;wqEgm>oayaZwUD#WZ&n&dd{(;dv3`Cl4U1vb}?q7lc4;B@#AF4FQcwXRj}Zsk@_ zn=2)Yb{Qg}D9tR@f{++o=j6o7p3nK`G&-mK@vVBGH!y#>{xYQPn0%#QMAwKM;qQ+8 znvG9Sq1a+QtL1r)>fZ79g*;+ijfvF84I`#OTzZk)_w94HAHQB16$$_!oQ4b<%idN! zb*jXemH?}MfSy#b8%PU*Qd8t3^es@*b}bt-N-w+K3G+k~dkIXES`}bz|IjZDF>duccMwg=w~1rMfuzWjM- zCl44j6~|P;Q>Z1ZYum!uS=~U#8}#KzQt%55e|9JD_Ky==8^Z&)g}GP@Dn1v!?GLqb z@h2PxEW>Olf^N!S**mh}j|mTZ_JL4N+r7g;jyKIhmxdyB5E2~D6{UClk<&pdIXAyP z+>wDVX(mIS%guhS;?%U6(?oa#&>BmB94XyuLzPH6ZO0VNjB9quq57WSc8?bp-*!0O zFb6@nE01c{)SKAW3Id-kf#j!nO=(?;CZ8FH-X1Qpc-Qo&a<(YE<%it;uqwW8CyG#O z#8(QpK2Qg*M`z8u$pi|NhEW!M;YK4qY9+bx7<;9iFH=ByiL1#$``0~fcRlnt1k%^5 zEH3-C6nD#jUDIU24vVuJsxTynkpYQ*2l>kEh*T2PZ3;vI4l=>n`wPw46$h?~10&w# zM?Gu8fgyHfV>o-NW3`RbC3)r6b$QPn<g1~X?N5Mmu(LlKyr&k7qPzy;e%3NB zTyPpQgTWSeJgIiCjcDqZryV3msUIq`mtefBQ-p8nS!b(mfNt<-<1X#H%{>P)_6S$0 zXnmh>`;<0>38j%g{xhN)#T>OMX@4AEW9g3+UG2jIA=J56dqC{VLfm>2NIs$D^6V2T z2l+M{_2*#o2mGVN*kld~VwB_G;U~Z*Jig=3K?djz^UZ-3fbeiDLObnygM9fg)NTTr}VuE*tV4N}`i5%)Q+MdVlg%f*VR7~DY{%X=mmRH&9i38GJ z8U-H`*xmES_6<{^<^0-2+?~aq9B~M$eICADl|BaSr`R}xIqVC1O!6YU6nqN%Zw_L; zvMa%U6^saP4_Dk9Og>oBwE|r`=@Cf+pof>;PNPONXcb$Z14%OXs?-d9`~ zdAbvw<0AlY8b~rc!MCKrq|SEQcRwZD>|#(qh}FG!vE*$HrCZ)GuV6iCT8p_bQ6cOv zsPxot(XivhHJ(+XW!%O>J@m2i!;S^xHf$#1)mAUXe8ZG-pr)gIJaVgMd5VoW=l8?w@j?ziLBIY zio$QzR}=zKY~}IECY{^d+25H0i@5&7DR;K>a&6wO0Cn_Jrip~ULS9Il-RvV_l&6)! z{2Uu^xvLy>z~lI02E3a+1jmVCeY3y}@+vno>N{VsK$a*5UrD)B;ikd1W5fcqRg zewpnxo;Ymnqdq|^kV6Cl8C^~wk`M79bb&}AkH`J~KMxZ_%>Gbg=#+?1)*yJUTsF3d zu^ClxK55<^a4XF#G3Tz8TIHTAftyq6#l}@(o!|iOe|*f8xftpTFWpXF__~(KQC!up z1>5*@`eHe3k*oEU&Zc^(8VV<0WO3alv4^ys*K~l7MQG*lZ6nXRe=dRtJ+E=-OA9C@ zy}2rc7xQP7e(*mB78lksG80vJ+*-Uqs)Z(_f{n2XRYi1psfG%5e$7nWs2NLy&mH}C z1PbZr8px!jNC)hJ!M#HkY*rRtGXSJU4*yx)>*&4O+2XFc*qS-YTVwiawH1HHEVT!z zHaUKvm-nga4LI49o3dgp)9UQW0$X4uXKK@qqvGiUVa>CNZP=&H{-KJ?ioRTQ5i59n z)hla)ROUqiN!G41u@Z?QxE7umJJ-)`omsq_Ko#d3#@pv$%H5WLoyD|83cf{n?H^0b zRIA*#AmQEeaQ}lH@K$ibe@sI>xty54Ai`<{F-}yiO|(>RjAeator364{@q7kZz!7B zf^Seaasj_yKV})8F7l0d!5hQdK$N;epcrP`F*jj?Nrfv6ov|dc%lg%pBaoP@-Y+Ma zRL!lRdxPT4xSaQxx}tRXP0}25jt}X2qQc`=1|R1u7M?XNn~5-nI*$ukMTpZ1ZBIES zn#nTOt0(VXJ)d${Da}hp@JQ9j>%9AGqo$?&@|jk=mo7nwA2kFRg>%J1kvr~k6-WrU zl?N8{urr5x5QS1i=F~TSV&Ah4JY5BE*&Hae=cOxrn+?HkdMD?RC~ZuK`7D3rpOpL`X&?0e}9qj5egm8^tKc*mIdev^XW%40l!7Ki{JTcG!1? zMmBTQF?r=4(^pty!9q}eE*c=7ZJVz#mZ0U9V}DQH`fsd~5(`C*DV2<}e+M8of~E;QuvI`# z9d33+ZV59q>g6?X9dYDC!rd}Cy{xXPH8y$F#?~mXuW~Leoxm>}SX=2ICs4 z`Sw^Uy!Dd0B{}sFdB(GJLeQ<9e*uoyoBX2;@AX~j3hq}<4MuiFWDCSq2IH)NVt?$e zMXu;zX=j(WV;Am;gPO;d(Mlv_Cr-LDa=^ZaQ%?j4nVRQEIcF{D$Q_IW`dgw<@cn4i zZRBwTN&&9vGRDy>LjxU;ZBKlPk#)_9x%p7Fu|JR$Exe%($w))o76l0?8Wmo&9hFC+ z9U#uW<*%G!BJ26cPMk!Jnd#(; zu|s95__e<&%@+mTF%u!F`UmI7ziFjJa_O(*6)X{!bC-w+=zeZ1Pd=J?4;vN4i1fNh z&k94!*^+M_=^V?fCbMy_iK%I`;&u`#t)q6uluA}QfT9d>fqm{UOLbI9rWyAOT}_7n zqr7(a=F4EMXiQwwH%*T1XL`q; zX|3^PKyvf8b@=bm&(v+CoU^*OR8G7%CDk-4)uWnit2eB6tH~;aC8uoWBVkoZjAE$x9L%E)emZWO(UPkwz1I$_&_M{L6WYPzWM4mW^e5OqwwP{BBIm9!RGP z0%_Sb>Heu_+)ZwHIgRv6Y0iwdPcm1P9zzExYf(FS8KX%zU}Z3XusMau$YthCymYc~@1$hdZZ zA%LPMYQ<Fx9wcko zeqaFs@j(EGuj4;;VBx}>BCHWK!YpTR+plhZzhx0VVLZ?u3^1=8LmtJMT zBTI&GFR0ehegt4Jn^h%g4bp)+Bjx<(s)UKHgRAcv7;w^yTyrkY)U6@pfAl;DGdg}| z&}6(DY^&LIv&3MR7Biy|{ZaaPhDoB6EbhevyetU>G1}rEKVInaIUM0Ub^6;-j$m2U zpp#nYy7an$(SyaPMYI@`iBk@f!T!(O^ODaB|Bms)zW#+Ns-77shhS>(|VnbI~tCX}C-kRezx5uxR3OIbhKV)kI~i zaxl%ua0<&ir0>?W0@KoymI5^xzbszZlEn={z@DI5e8#(+H=hZ;?~1^7JS)0R;1)I7 zq@RY2Ro_2fCipwoG6B;?JcWTa4!9AruToE3-SYkapB9%*K^mVk}07VhCD>p`a2ZdugBl zfnoQum-D3_h*c`uD`C-*F8q*}82**o)hw+d&zMjjgaVr2;3-dtGu_3FSKe7|KYTW1 zC9ZVH;D@{gH}O`E5`Rz$0R`lrdjCMNfy7?RjQ19!&o z`Sj3~YP$@&9;cXY@uTKiiF`skEb9=iEr5%WYJZ~--*4vqke59?eE+(gx&O&1Mkq8V zLX18WVW0;qEFH{}NC~p}H%yZ1!7=#0#*C|S5CeDd=#5YAqi1+|rr(nqZuuZhPEw4* z7}dds2m%F@?t=K+Sy+#)=5Kwja_Cm7(czXVLiZwm6qUuuIOf={nGQ#2T8+e(xC|xc zsOJC7#d>Xh^6GB7Z^H)^BZf z+TA-W9*VkHt@o3PovvK!pAPiZN&4+oV$Zx(|{%q@oH;cQQ;} z_!FCkoQybPTi=u@b(Qc8jn(@EQ#=>P`yMpY`K3J$W?i5`2~*cn^#AwVHM>_QSUN`G zV{Bv(h|%~I1}t+ zx<^Y^a6nr8xaYj36WYap1EgHsfv`o%5YFiPGvvfi3`~)4dU7HO5DP!_MA9W9UK5z4 zAEEk5qzJhl9Cz#@?so&tBO!QI6<)1_Q3OH{w+UIr@O_V0Le}I!cANL=`19gFH zBImpNx2^mY7{|h*Rl6gTU3Vv>APVZ_)K*XCVq-e>W4KRERXGBL{gpoF-H?WIFHS4# z>$>(Cdmb*!oF~K?^$yNV_uS6k(CC6U-;xniZ86RY9>0;yv;wg8$0X7=ek7e5wHw}3p8tlv~ zhyFbQ?Y`-l-6pv!I=0ub9D~$;Ep6|9omVOGjOi8fKuzZe=JFa3Cy(Wt|J8$fGo(A% zk<%x`>7q2Wel2ektR~oatr%3P8wR7S^J#KDs>2PGoDODlWM(eb0sr-=nkNV^E}(T< zDg3WOezyCu&1TTwjac#GNqxbC{@+(C_mbbF$Ot$gc?q45u6xh-eQ*Zw8RaPD(x*pd zT15GUArfUTKDD1mCiBxqg8vDxVI4D79X5ru3ud)nz6i#C0WX_53s$A6=9evV09s`C zyQD(nvTfKR5b=Ok0Sq!yL*YQ@Jud1pjW@S)pzecTC2;YG(P2(>de)0io%eH&5m{!wn?5N?%F$c)kTqv&Ijuz`hg~65BrTWWZX`bl*nmN0gF8y z6Y`H!=lh-DvWGrD(bYO$+7oKZJpu2gAjAh`)sAI7Weq={wBN7rD)~V`% zQjE0OxU5EVB*}^*0fjYieF;n5=vjcJwqA*5H$(B+3ob2-Tct5@Tw3O$b7Y`IG7nRetMM?Of#2er?^sfinDef}4NHSQHE` zcF)z6@eW;YaP<=n!kW#=^P07?e&zV9?(dg#~#dP*e7I0);PNC3M@c0bU)=_12?wDvil$8k8JW&MNTKTV)i zM@1B+XMk<@65l5QC0&MG;9DmTupM8MHJ#p zAbCP@2y}y_g+znc{=p=6DLb)iyy?4W8W^)4^Tp4#MtQ<$Y4}1|(Wwra;)j$I=9rE> z_d4sS($JUSawP-$(4C%!O8=Dw&y8ytCh#Q`NYH<^RxLRn9Re!{t_K_yUEROY zL&Hfr2~=j;%2X;o{N{r8uQHJA!&gro08~|#Q&RRlamv+=>w$JjZG=x{-g9IYK?Z9^ssRJ!Kq%#*qv6-yU>H;6dyU53cf5 zQP0^y>$1l_%Gg%hTI}ZW5vEuR^!!e`k)w(T*TT8F!Q%Utsnp!rtL*; zevz_0-JO~IquszH4ctTJ92@c>nliF{1m%nc!H@#=J^gLgOXjktqPp~&5=j^%%7i44 zNf9v<4w0u?O$d?L8S{p)>t;Skh~`gH25$-Z)B!vrD#M}RX`AWxbWHbdQAJUX-?qqt z8G8pRRh?T4S8WP`6FsR&&D>y{$vmiYEMk3Y+k&Z9U?PQtCN72HYh56| ze0}OSn1=Hw+{=HS7kED7R#qMpkihp0tRW-F%CLb?lv^Carm_aXqCvY_$HN| zAvgjir;{v$9yN@KUhr==M)ry$nMn3is^*HoTD=ZC^2{??LQ0aNe8DZgsOZ3yUnZ@{ zsI}(CSu^RABz!;0$QR)^-0q=;dz$-I4R&{=VnD>8k|WfCV<0{`S{AceGd(a-bWP`2 zeS-lYUvFMwPPSRQkg~nyi58yWCwP<_DrT$;NO7kTSN^Q@1jykWNO-gR;A}iYS^6oV zbW)>%4@>K(b?u&9@omD2_oMmn_!nn`mroZ{M9(b?j$0nu6%TG_=Ta{N8wQI~|afum&*(b!m-QuZ%ULlgsk{P5|6v4+Gp>)f8P;yHG zy_KRH7v5TE6#6Zdhe_pTiP%7C#l&re`*hZ4Kl70U>R8=l671n%W_=xCsI@js=bE%v zcqgjM-b10qd7%H%2yRt|%o{Xri{>h&F>-u^WCO9`iW2uGNabANhU-fO40E0y^@T3; zMU#j~yFH4dw~tunIdF;91g-x>{WA2O5m<7#IW! zaj}Z+Rf14&&UqB2pF*)u{NRU=*GsOsqqTAB$QnLMFQ(QStMZNq;uj^I5yQJCTDAV#<&E#>TwG@GmO!FC*(p~vf0htOi z3$zAdUc&I6O~uyw==HF=JXAE1QiS6EuVU4;#NOp}LTPg!MyFx&*PX~vKfeXp1);HgG%P&F$nZr{G~m*I0a z00a$MMZ!R}wqJA{9!!nwmq-RsQ3k%H+JZ6xW<>--EV=%W!k#w39%!EHQhZ)nCMj&t zJkO2F=UdpNCp6UQ5qi0INjw1&tQmI3Hy+(HDpW=me@Dm~vOTg6xFo671p>hV2K+kj zK5oJQBD`#G0YA!MW!L^tL`f^a4JXvU_UV|?M)kCb1t{^wu$;nMF{B*xUAB1MJP+Y7jFFch#Iq@xqrpXXnz8bDq|~{hDij zQ??(|ajAl^vz(Uyz?fCNGe|RZ69OL*36CS)@4A7WOr zfTDNc=0jjj+y{mG$>YKM3`h2{Z>Z;B8Mv1MV7uIidAQg;vnwooYWq6J%R3%5>_<$? z0IlfNOL*yz%Nqe0nKJVfeCzKateyk>7x<@7P~S1q9ckvx=$ZoY82ORYv$L(yNZB0Y zbD4q=k+d0KTv57E?oJe#W7DD{diz;+?_ju97f*(QCb|NSuNvDrZaso;;?9cSf7(Q2 zJ^>6JmBKWs@@gUzOOIMc+ithXbFL1u#z}XExF}Cd9?Yq?{(8tPzK#{upFl$YY+htX zP#NDHD^x=hTC^nHa3b;KN4)HMB;s5XY|kd>nK6aK3BYUjw>!bDV5lx^f3vKte8@0E z)J7W0sw--|!fs=7+MDhG0X+%i>raK6a> z)wSi?l_#w`11!T#c`JWP7yk9tFRg}s8OTHJRzm=u+;1UGywW~>uD3c*W8iI;!PxkG z0rH>2GCm70F)=|#!IlH8S0!h3So}YS`FZ+(93oY@-hsBrF)I>L%eT?4ee+hjK0n^7 z_Y7()>*rsow(IbxJE+s|7zqI)_Kbh#8nBAw;JxO_;KrauoRvngtDTv~R7j}eEnL%A zq-nX512L@)3ie{{oqV=1dV1e}J%pdIQ0^5z zNAw#SIk79I8QurpZl4#Kzgc;JHZs9+CUL9aiLod}0~>hG)~zU5b`2dbj+5Nf{Rak4 zpQ&TGocg9=Tj6h=i~`6%EXSD+oWU#-`;R$cjoF=+b?JE6aq`r*p@$8Nv5NA^%gz<2kk>~mycP&vpS7PLFw%YRtGS-+Y(AvDmc&0|4Ip~s4ZFNhQgBlb zXPdrNyp}IO0izP$7#0*K#4;a`%sJ%-fd)t8u4u8gFa2_YJ!+hR!n$P{t~84tl+QTB z0;`v@SK7DPy&&rppET?v8KLNlU1*Z*(k=Fej9aBSYur_%S-2oeW&I%mbJp~4>;vtvnd}&SJ4|`7BUJ!)M zMNHE?8gXJA?=V@Yj#nsr#&@z(8--i=sE7YnCQnb=p(^6KXF_M;;Z?TaT)z`45U53~ zwEl^W&2c|L*2qh`31d=wGrnV!JktFtL1SCg2n}ESQiQc{Qq>BHKun-*zN3x`g2;`y@4u258Kk3v*`T^gyrXxQ6$8o>46Hm%M5f zU)j(2->NsCRWOgNT$J?JvlsF`XWYqBp?@rz*IoK`8?PYMoK-f9CE$JCf|*=!;cEer zrT~|_kYX_z70xR14(nASPo9Iy5}`hYQM*SXz+(nitl{P3arLmVNH{QkM>nlLH&woCR9K029{*=q~IAJXxV;rer z3vfmQ?sEWWM&Fl@=C3*xUD0=&Ps~1O*0gG1vwA?880a!RW{@5v8nwATGi%B>KWDK? zYgcv3!C70}AW@{LjpPJR)>c5cOc+(cp!%y&3Rn1d1!9{qp6nnrE z9~lU`C!oaC38NIi6@QsZ?L{|MdHi}&u(CWG*R>_@^jh1{pc~&L)@R2cwke7Ivt>T5g?&Xk&y;54O1<4T0K)$X0 zg5Sn0boPIeOz4Q-%yaF{C(r)v!MDyVW*<`SGOWkn^D|dhAB~Iz0}-MMO#zT=u&6@8 z$fh0YhtJo$>PX~QY^t+!L`I9_Vq;be1s5|VOEohY#UrCpr6(Ja;Zc~<9tpzZlBxjY_ ziGKXznX&*kaAO8@usN|@w~pKhmWAfA1q|D=O(UqkWMZa)ZZe`1))^b~i&$c?C1A0u z=E-jx_;1Ho2IXWbGI*&qK{$BQ@`BZ?-36Op*&wx=*^GEd3ag1dmSUIe5pB8`2s0yO z__IF3;2(fQLlZjR-0mxiS^t5lwj>1E(w1-k1xc=f;^Yz*MNXB%~gIZL2TEi zrRyDbU#&+M2laBbjJXFLY7cJHCNpCe2r&k6qDjIix47Gak>c1z(Mi>a_&vSL$CrXg zVdn@e4cEc~WHSk@V^T3nKLy!@mD~3y+OJ-~al(_(2~Vg=%tS;9=*z9q1E)j^*Mgla zcqq`Jz6R0U`w*F!tg{X^-&zkqO@!hv8lfbfn~0Jwf%?&IVg>V?&2-LtraQv3J1+na zz%j0L38A0TQ#@#IAKG__@1;g%fu(~d(NYoa3G)EV9VVO6F>bIwR4-@JOG=9y{l6sz zsQK&Po=5a9wrgt+W(z(7tx}@8CK;xA8v*9;Xh!8H!F!+w^pP(M+^aW!;MEI)tJvN? z^CkJf{+g{`NqNPI&+nq@&|P4; zS?KIjU_mgia19Cm$XQ}LL@V`a-rGo@nx2+~a@YO~-~Bo@8WLfx_T328nQttv>;zC` z7IwhEJp7n1PhKnfE*;-elHqDyW0Q(cpWv@@FIEzR80j<;fOPwOO2qs4#SyZi`_YY*MVUIxUYHQ@=sqmbw_y3OJH zR<-Yn!ftf$Nz~I2mxbdzAS8Z`;?bSGosslz;iDYD=4GY?q{uT)SrK5k&%AcIwOzI? z>zkD+>$E?I)(LCsw?P5ds__w{1(GHPxwLjH{ql4Jacp4*)%r3QHv;cyV8w1JwvRk0 zx=FgzQJJ1f69QDRXk~rsfS8(9XTwE|tw&7oMwT==609Z(m*?c}1WEheK`CfmssV1U z)83#nRwM>|qmaHrrEG0zo4d_mrlB@fMmkzs9$uDvXUs@c58QR9=gGP*{!rW;f9#>1 zM|J}xsQM^w$ljFXRJJoHIh}97d!e*}|E0**zd1OUixyTvc!nxgRbthR1P&lan0YM; z&VE~ry}xx+2x<1k#e9)aE33FK=1|plx^Ot#cWpwduV`{E;UFY55lIt%9Y<+eo^2`A zpWg9!HTUIoW7Nl$?g-53;`&$d_kL1dFt%zRvv*xCYRPJB{9a~K#fk8%Uf6i_d-{lx ztqY`Bp+kGagrz3H2tjR+c2P!yQuXlY7?8D+@fbR!h=4TZSU^V|&9YD#q(-RO-V!y1FL%y=Y8rTZ9t*cB4J4|%NBzEFK z9u%I^%TH~Bpy>;w5@_#l1C4N!DD}LNk3}4>UrbY=RYfXYo-)@@!IU-qeynJ z(%q8$1}k>1!kt2Q8hvGg0s1&zu$h`pngdP?4B41A9A{N=&@BjFK}7kHIgRMwvvUFd zX5Wd5sd2=iKU=uEYX8{AlnMQ?02%LR3%=kMVS>|J2+^gw8g-!NuKrqYmw4hB-7*X6 z-ZwSVz6S?sxjJ}^Mb1E5E;J(gDF_IK; zYB{iTqg`V(XS}{J=B$a9ZKYs_5wugXlSLn~PSRtCk)x+T92I#T z+3=#~zpm507Zx0UPZaeCTq>0>6dAFGIe_ra#)6rvw`H@r)k(E!F z?z8u4RgdaAW!va(tc8iXpfeX;MHkdr&iQDkN{V;589pIL`TCjw%98Xrc6!&xxW5y2 zj|Zb^x1v4)B}G6CshuABndi8_wS-7b_5yDdYfO^l8?igdpTjd6N;opHUDLQe*3wT~ zn_wbG5lRdBQeq?ph*^$f<0#`6nYz5c3KS!U8D4ikgLR6gc!pgAI{p<*d|xF8A}qvd|M$0AYS!_k*5T4rf0xKpUS4`W=J{=i$@4yvA}220?VY;3!I@T0ONd zH9P&VPnUM~U&gc;e@?F`z5J8g=Guw!x3<0yAn(d(&Tpy(0Ur z;~L)etA%WO*qbbMRPYIH2ELv7F%Ik?Gc@z?>0VAh#bv>th}l%NP_H5INqY6aRXJzA z9vvPp_WC8{Z0Z9kq8H%OY@JtejtZ&Sz591vWWVD~I(dX8El$TOp9o5guf3ONZ(Ssm z^wi?D9!#qf!z+4D6Q@1HOI4kM3CA3U*OL3CB6Wo)=QpnzgY=UNpQ@a^oz+8MtM;UB zhfQbtIxBZI+GS^LWc~ELc`)}9M`cbA)D)@AeGwiG;@C-L4=S>WH#S=waEk4zXqBD`48N#FWyp=x zaEo5LL>g&Zp$j%Dk+3|b4;}jW^t27-FzX{3-f!Q8H}3XDLpFK3~$C}L442q?3*{1bHFc{bNzJ{!Do;8MiRC_xw4q#}h z;)FGMA?p=~B7xOY0zPZFTk}oDv8?TJ-3Zz#MMlqIYJRj*lU)X$8kB&}bG!Q7>C{ewvulb%# z*|YwL1}=L%G#;qpi#!1|R}Cy6)5@+-A!h>H$PV+Z|8;sp-xC&c zZuxrWTq3FE$@pRnhbnTtnA^K*;&+!VDbhdsb*v|IBnUnHG3;}(N{4w)2F=)A<-UT1 z&4<&zwj>1~HC=zg5}TQBL^h+@)=0UEIqxY0yJ3SEmwFy4A3`?fC6$X5ADq31f|mEk zaYq4=q7Z^?zhwK+8VB?JO z7C&QY^p$EmywH`-QN$G2jQbHL8f|PCB)TASTy+fwpG<%@HS%Z)i91^VSqDHFh2NsN z3UcKuonZ-}s2d&H({irozV-2D{OncRHQIupMO@9*;j`IZVeqWad`3w?iNgtEfTVf4 zjoVz>xQDYLj2wr6x$D}IrM3L;$$_q!J8`z9REoiTa|n(#O6;I>f~%c{mAz8A1LJ6g zFK{mTk&l9~w(Ua|W$ul%;E86Y5lh8_yVEUi7JcxS@oselTzhj!0C$jf95Q=da{`H?c>HB*%wO2Ae z3dfG@e!Q`1trI;{Y}e05%2g1=PY)T;f+KSzAnoZl!)~3egmA6#CqtYsS6x-bz!&-;gvdy_;S!;g(2H~pg7}tNQ+|_p zrwRc%fw4TjuC?t!WqK`w!g3IMkqYuM?rau>a6uIh`;r%POQxJN;+evwKL-i^$<{lf zL?y2snVdAV&D}BJ`|PC!*A!}1xa>k!R?YNoi8peu00nyps2W!we-H}#@s zWm%aFj6`+Q(5h!xm8Hb8OzjaGdomisz@&6L6YTJx6gI9pesesHE{4pzq`vA{9>ohC zB|7Gr<-R&2s+V$5%vWTtITii_KL^$qvX_5xEPZ8$0mQRL>JHCq0q*q;R0}^3P2_R% zW>k&vTe3lqRfRuQvZ@@mFd>h0>Il#hvXZ-Vl`^=N_q)}8e_M;OAT#`f)?sn6hz~#? zEBiWC!8rf8KiKUSwba;yqzi>b()AwWgXl1K@TYKE#qBZC);WSse?=@trvOZjOU3SL zNwA_<+0cxU_KT%WvY%21aNYivWbfHXQ4g>=NpqfKgk(N*buV&u?LhK0IBBK5Hh*Xn zuSN;2%racad1$9tLl{G66G}jN5jrb${cr{#nCCey`*G}3WR`+CTt{PQ^Pfnf;_9Sm z=J!_j|}%+-a3;OM1S_Hri+uWt5;;Ph0jeaxM;T=V4?&JblD~RqniTRc1}KvLi`}7{eU7z9-}Kk3E*80&ob>1GtdvpeFw@8)pUP97 zko&DwWLw84p#`1M`r?nc@Sm%aTi3abo$ZvW%{{E6>(GJ*QhSPGSQ>!@(GB#<5J_6! z$e~NIvgTtM`Lz1T*FqmW4Y@i2w(nOmtI5PmH7z4#VMM#H;ml{Wl1*t7B_N&?6^j=B z++Hgc7KKZ2C>6VQ5NK!Ch;w2tiBoYr9OHDiA7lTBcd3l!u_*uR> zca}_4ES{*9;4LMHawGc5ColnEwMFt=dB>QnS%}043KYq&Cmp34jJ0c%{#q+gu(Jq~ zBU~`GHM}(h3fKCkED-6u)h}*Najk5`r-0_rXRRTl2kj70u9$mc+It;|fTRJ(L2?VY zXs?x#T~d)k70ym0lZc|uOb^>*9QV}JJ5(^EO4N)+xGM8{j*=PGagNSyB|9>RccKHG ztFX`E@=n!_so9bh&an|MfR{nl@dUwy&iJN>X91yONi|GsT=^wG9uFkg>V_(|ii2mKALtmKBHy!` zrP~6r6$}(2LF?B6%Y$#`sr;l)YxfXZxb?^`C{6&q{oW~@=Q~BNaYMf|JOptx%#@9) z{D|>(f&uS`&%os}a<#9D38kS;U^BILO@tjA_P^fU_{bL;82p&??_$~_&<43j1t1>I z4^}5|1x#UiW(YBG`zCI?gz@ddO68u&R)C(uwBxFp0mR&gZ*_wg7ct%^r}k zV>IaWQV|`O!klrUuS%Nv4%PT^ZMpeQyBsN6l4p`9TmlSz*BBlxz4Xt@$>i`wy;!A` zfGaiCwxZ_a;Sq21PnSEhWo22KjP+3BUH={CUtlJZZrXv=9l4{_+h!J0Fc`@GHaNCD z(r7mvXm^#l){?!_L{j_pIe%_^*BlbpZAAH4^`J0}WN`c#Lg za>!qZl+ADiiOv7b1H=_~+DMw;8Q$ENBEQA(Y`P#vD>cQ6ZYKJY2_St&UU8kC!i?a8 z3ye|rdN3NwHW@O2w$*?x&6(J?+=-kht9Y9Qu)mau&sTf`wgf1{Bf+>9Cts)0DuYu6 zbD@x&>j&V9(3axJ|C(b#b9SpE4*+#+xqgwlR2+IaF*MTc67ZQ3UUbDFP(r`ft4;9S zY4t>(5V@|CTZq}@NhR0(nX|!3o1Sz_Ro1(5?0;aeYn5;B*${oc9>y4&ge-!8S1xP3 zK;~eT0^%G8mxL>+&1JKS(7lnk9zxE=eh-F=wjlOx&g~!&B%~Sy5wqdDn1`^}zqW^1 zUqscQgqe!ub583ddV;c%jV|dO4%Zf%QnE_hz8)Se*LX^fv@G?o*C6b?xXKDJu|Vm> z^j%+E3KR_b^o~CkYM5pE?Fymv*7v4&Tp6~tBW2%CS~v-b)uhJbYV`W)5Mm6y#d*wy z0G8)%eH_G$hdAZil7s($g+VF-n8oIl$Gars4cN_Ev9QpUYjJc|1<-x(ade(GKt^mv6Sm|)TPVrv}; z?nnyGMTptON3?KGkP?xa@AcYck9rX0=80wFIqQrIXI0~Sr7;%I$}V0Vte0*Gfh7Y# zdy_@>T9qU^=*ZSbonmELRIN1$j;lKC5|X3OXtHXA9B*>YiayVJR1)9ii_$dI(PZOM^#7s%&e7u79* zEST;`^;uSQB8?*>&Vr`hShzcwrdO*{|1P;kj?j&bx*{@YL>6u?`XwP?MFOAf^Nn!06RcEG=r?5zaYss)+ zxwqrz`*<^tK^Cg9Six4N?Nh-UAZM%VY7ppZ+;>)C-XH?JzQGY6Q3s!rGx*~ zW}>{9338fqpR|G;jBI@>Foe@j)G`|}GLSgO8nmjbgC9<%wXhC|TXK4HM4q8mwj@3M zxA%g6Gsl)d8Kh0AnftIBO$?FY zX2G^*H$hz*cR{_jR)vjjP!R3z4C!~hFyDc}P*89#4Pu~OX`iu|-#dRTLe*}4^0npn zibP@!w)~LnM)sXIOF&aze%UtHcY9L~$~I|;mQEm^78@e{E*6T$U$S75*6FGl5n{CG zEo3hJpsE5WVX(?~r1>-%{t@<)x`^ilKMM=6`znU=#(1TKm99%yPhh>(w9&RGRdc&> zE9`U5G+_D@e!w4^9!+ZlO>BV+kEVdUY0VR#+?Fr=s`FjCz_Q<}#)S=&|MxFe486IP z!WG3ur$iHMU`(R(zwvj322y@+;)B{H97r9xUs!QG1X;&u(Wn9R0T=_HJEXtri79)L z1m_NaOp+;mKxXSqZQlD7_nWB2%lNSlSIP0$a?_T_x;+LzZD01riBLb$MErmHaJQuI zR@Zk|{{x%*c@6+5s;Dm1bupH4f$}eJ{_?68$ARb;vD3PZ0^r$&bxRR#7?3a1dr2-^ z3_N}7VHSY7QK}XFaUNi^&API9>DroeB)=g@c06C6E1c|E4dM>!g-0Njo(J&<;G)tR zbtOsM1wwHjjgijmY;)X+#jTC5Bbkx z4*#9QNQ(N4_9DJ+SGnHOr%E!2?5qcv4whtRW;X_Z23|4SV zswK18j7OBB;(Zhkz^_$EAUf;fs`$fgD^S?F?m!=RF;4HgZDo;;So@rQL3)t@w2@Vl zG@jvxM6+D?nMc^RYp9*OL}9t0s&dr%P4)JmDh?F-`&9Aeh%oNhQLQ3Sx#`+h3sDKeL=eZRQYs@qJRDY~{Be&+sSiLybN7gj zs4(bO1+J#rspgam2&bspF@@;FG}bvq{g>Q9^%azNJv|pi+T+}l7bIXn#lXp1Gp`gaEU_Bzh(marLWTiJ47fh!h zlDAnEXTxM%ob$Ms2qgo9RIAlLyS-OImseIz`W2MuNP z84Osi#v)c8ewC#9pd2yT@G6ZSVB4Jgj3zYYKf6ivzZh8;q8dJyuFlRClgD&I?`X#5 zS0`TL*ynsS^8xNpxP^#zO z`}9lFVa2$bJi;+68E*V~%C1Ts&$?Pzr0$xu?ERYfBnV+{50$;wZ7?<2T zp@ZKbK#0luxKwvda#VhFd_#c1J;vms&_3om4EsxCcfY$#4DsgU%<`d)7tT&Orh$`p zpO3HJrcHevG@c$ZAo65}TCGT?k~<4-S^#fzXt7c|K57Esy&m_;tA+3I!T5W>j!+G^ zqLo|Sw^jeOme4&EuFF+N3?dCIu+^g0AYbWfzaTl}tV{t)R4V2=o1deUo1n&&;tb}) zi8iwPKdf`UX-5PO*gMoCIlm2vVM!e@h+8|%K^fy++CQB7c5L!Czxe`6a4URh*6|W38MZPRsQxz0PMbmOQySNXXyv zMW7NYqut;|-$H>+L`+526SV&}F0%1583W6%pq#p_c&4G2@o!~DaZoAqX4N~_j4nXw zUxz8{XOMe-pC~R87@~eJhhG}zneld?`yN+^BP@bHej}=`L;1=!HFeJUfdwa3Jf~#7 z7#H>k)Q1`PNI1fJ6E_*=kTdTXLCS-X4`7rsOy;4a43ZQMi#hBoh_U{&WQjQJc=zRs z2?-YuR|Qp`3I0#PQIDaVpK6%H;k^*Hh<8qYrZU{ML z?B70N9&+u(tU+_f=G6X1KqQ2)_1R+B>x&7bN~iU}{$nMlf0Gd&G5E3)lcSXLj&7qg z=D><{?k9d@FqEBoDE|*;G*16ngi>vIW1T#+L^=|ymZ!2sM%I>42xJJqV?H7;(ma)q zmRo}?FP`;+(R54KcxHV&$tR@J5U9nUU)Za|F5b3gDF9=!JUH-dmaM*J9X`~xsD0PB zYInA#Ub}sE%iM{QnPX2)NYTKsUOH+GOOy&U7Iwwcw(@n`)bapks6e z5@<6*vbUoVs61N+wqZd`X*`3g#4L;O>p*a@j}k*=?G8boB1j^9W6T!3H>x|Law)(9-nJKQzjc8$$miNHj~w%wQOC zK<$?CT;8%VNqv`^vCGKVlqX?B;cn(8bH#evVQ8XM$;uPt^~8fP7DCFB)A~ZE7y?0o zoSQ=M3l=>IGg2}kLCa1neB*N*cU>8TkzN@#{DK3-De@%*HyuW>YckFP9R+ughCd`=QGXA>8CM+Kq@FnDW|0>!SZUJFk`L98*@SyK3tQ5N{C&uK?N$=9Y zZ5|x&DH(K%Fwg}~QmB;AQ#9bYt5u1Yno41Z5BZ_=n0#WiU}U6~3{5>g2QE56tR8GC zg53l{7XPajgGH)dzROo|yt$mPr(+|Hl#X|-bB8J?H3r*__kp_1nkL=V`NT(r0%dni zHA0BpFZbI0m|5UV@N%kYjJIYQoo);Z;A4!ct(+fcm49a;DSD2T7+DTDQ{G+hb@-Qr zd?+o?EEgPtfq6yLn$j(xk^X8fpuS5kG`n|zI>C){d_P>!w$p_YPs9*fG>lC-RaPg{!NsBtg^0=2B*S(4xd$~=+nGEIVBdToLUsSO`L@;DYWTI+sq#OVFc8&!#RN1U8f9 zKCy{77MHc-xL@s{k%vggZ7c|ELL)xkH@O)0p`5}FNRIqo5`|8@-8=I)E5;IC!Z^}E zqL9{!us3b*LB#jUAEX-J3g@UB{&FtyUr}_T%f~f|k+L6nCae4x9V!v}q8d~`Jqs(m z)oXm$3S;t4W`OGXM=r>QRlvzbhCx@Oj5I8$!YZ{^jad%kRYTaF;P7DCBG*G_P4Ige zF_q`&Ln&11EAK9^9oOBFM-_&5dgHI8qebmqgS2QK#DUjJqkViMdPSyWJvQ8k&BN?3!Drs1W~8 zW0*X`V_@$j-n^G68NhF?_HWw-*_8kP(BYnzztf`*tP{~i-$gWYO&nE-iNAz(^rGme zGD@T?{o#)4_KjG)K2RMAyo?4~D%SGfFP!}!WL|4qFzb|hT+_6P z_qCedzKPJ_fvVz8LPnFPj<%BWx+?--m{?cat{7%5`n-YLWeFp7*SK(al~jF{6F=Nw zq3a zBIaMo?8{7k7$(ZXCZOwND!5w4y}(CE$`2$gHK4)F+F=Y3fd=XKn72KfeK0O;#<$fm zl({Yew`v>U=jtnS(G%RGvOPwLJ@Z62U$<@sdUTrj12ft3z3)dIKB$6+u0|PicV#9m1&uA zr#uw;#2ktc*Sqv{pmapD{rQ+Ohh{RKm}kBlX4wHbl5nPt-bkMJFN7k8ADUm}(9%e8 zsw|<&joAR+wa+QIUb3#YT$XA#J)m_4m_5;V|54kQm~=mKKxH=7R8#_i5)6;5R;vq3 zYTmkgF+lyyrXKv}I4^#x-m2x1O8h&cd}{66GAX;zd*g28@vf)2P)g&iUt~PH6!FFu;6q0S%h&4JF2 z3%ge%v#a3=lkF;vbu&8(R0iMlm*-+T@~HZ~xS$&L$F4Dh%_%NLjsAp!euxbkL#Cb=@apzBMoK$8m|NrVs$udN>Uc``+NiIaKFTx=$&(odiOEM5nYKT94~ zN^8-=cLtF7jA89PI0zh&<^k!zLm%B=^xp$z^Iuh;7=v4@-_rIo1cSqSfVx8uvgwvix$fv!5!;W*K}hA&FXXX;J?^k)=zWLBQm zFq$Bju<%lE-fzhH7mmz0FwA$~Od&1QU7)m&8tHx%)bk8j)*gPDy)!)@0Y7;8{{Y#$ zu^r#aYutBn2za#(L2g8nqHafpTq_re2JM@^TLh=)D^DjGm}`OYviZhrrasgB`Jo*8 z6+nqRb5tBk2Xx}PBsMZHf0a^N!vC3CB;*$+V15OkX%Z&@FOkp{<>Z61E8Bruc%@iN z%tPU&xesYh(RKQU!H<#Jd2IX{Dx=2Jfbw(pOoju^N%e{GTY~}&hxaN*v)Sm@T=_n3 zp=m0*P*lK?$wVGf5b{`s%352oJ9S!MSlrmr=yh&4c%V$#@iN@hZ0a=Lp3!;WqbTl= z#Ec;?cyZbp@h;utA+lDZFu`0d6ertQAZ{i=5u$!G(N#LGG3{lKG4u(dl)oRbba;Pg zBhTjoQP$I80s+Swx1xE47N_%L^)sykvH}Qmyb^Pc%8$tQ(euuaIGKc*tEin{???AivXDn>$0Yq zXVMpI%WyrT%McG#-r)E5LPg?dNf zxEVkJu;z#NZ6v?vu(ILrStIxM317BmDJ0o&TJ6h#KZ56}=DnzvHO((y`RhP>-iT-9}_ z0{m*tSV^?>#W`#vm1~KHKmGMeE*hM-Z|o5&Q(~S6SbJ%aeZ_%MWo=$q|xg?K7uDjzjbKuZ|CIc)l;+6uxFYEjzrc1OR9T|v?Rtf-N3|u7*>G1Yer#3)~|*@upo!Lf6!J& zFN24BK7b#RNAvUXwL^xwqFj&`$ypdgg-crm95?61ssM1IH$N zp+f5oVj`ehn1aO|ArW6Pm(ZKNwKMAvj4lQ8;=NqlnB8bJHhv-ZZ&d*rXsK2qplY*r z3B4++-I9BlrbFbhe!lKFnd=>e%?WD+=1~x-6GoVLa!}$Cw9;-8{{w&##6VKTTqZ79 zgk;E)=ss_0S5>fExR=X+*-dTJoN%NWGg$rbV{NE@w3c*f^G>ISfIU{|=OWS-)G3{i!!Lit<0%6$(dPj3Q_Rfh;dV1?XTXZNEAeM5KQ^ zuZ^hhyKb)+SD|MTVm6J2YoA)ANUHXHFD^&kZ9%{~w~&Nx)kQ_3t$=~*UGwIU)+ytH z-(R{sk-uGR&(xO@@qFfar{4Z~4xUP($qFj}2k-^JSY|A^gG+x-@S$`E#`mXkHw!!U zRo!Eed%0t(Bs3JIVW5_-Xsd75NJk8RL5EyvF)=QU7iYm$dz_G*4AKGjva%e}1SZ*4 zyfoomh=SR`)9|-Sl1BiiK$Q^*+iuwsH&M=17zBDkjTt#%~vae;l+W-+)OArvPyj>%sAuv+QT24O~mU?`A3X$J9< zCfm8gm3MRXTyO^KlB%=ctt&NBd^uyIspw6WHbC%Y^G>8}?pf8Ax41tQfIDQ^Yg@s3 zJ`MGCjUD{w5o{~*KTPi)Ry|K=$4Lbo>SI0Yp?<+H>v9|)M1{uzk>VsiQ5|#@*V$%| zi=BDH1R(#jvW>dOZFJ43O%Xc5qNnkRJJKVGudJfl7-p7NB-b;%=5WB4zz z;Iur;?3e6V>YoLcVBG3JOqc@BmH1|vc^z)}qepnMI5j$(1s$St6r@mWe}b1TWLWwk zkSXn#ao37>3Z=6p9Hf8hzRsZ}FQ(Fxj+~Si?Q?H?f^`W9jhQ~s6&{4by}L^?EEvZ<=MJ(z+2+ZjO$l(WvIrs9>*R~5zZZhbZR{8N~1JOH2 z79zv+7r_K zSTOA4zyHK4%4n1)O_oYxDo-oS%?g9?iHf&7f+@A_8^eAC6_TOn;qy0n2A$63>ahA; z;OU)#-b-u4WC}ZN`+rmWGZayD8C{87RBjOmAS-?Y1;KQxnjj}!u&cTzPiUY7KOTeI z25ytfky0bn3lN2N0!dE>4GedMvLjy!h*jrcy~9pQduX4*+#fmZlvcTUlz@aqr`cm2?kKHxR^m9zN zT^iRS@aF^pJTq#Jh)z56Z3Ncp!2 zDSgw!_i@TkCba}24@Lfl;t(~vE+g(8-0p{v{KMfyQ{YB=JbiR*WEiYY7~NJ%6u=!k z64|&^eMvl7&0=dcLY$lpX?BHVgZEPVf~GLyp6x+j-UbKtCmhVGCkk%WSdMgqY=?~? z6m%QDuw9tEuuuvjA&RRcDa@?W}O32{+3*@kJ&2=j?lpsQ5 zumcqsnX8?YW1skMAckZQa=3QpS0#3Boxj0|l6%Ud=U>%-XEsa<)0)flM#z(XbS0Bz zX@9goLU;8CMa)N`|G1vL+8ozPa{fAQOG` zq((*#vHHb;ZA&6hJ>B-rlFxVPP3C6|W%evckE)9pg>0wwA3i{}51^Ty20A%eg+z#1 zgq=qW+$XzB`)O&L^?YnrfA&B`r5LhQ*0ul+yzOQc#mkGJ<{^1|>x z1gJ>I@LXe(uM@Y|MDMx(D=&{MmW8y8y!SHB9Q^j6rin;$dbIiEE(VFK;sUsTDR#R3 zub$IOE`h@{&8zqfq7p;3!;!$C)U-hIJB}MP;!TC7Df~A+W5&mk1VR*Zfjv)(0zjm*5bjvp0(?!li=TuW=kF71_IqsFn( zH@Ha|_K`pU2Cb*h!>pTFBrUf$RotnZuhX9EYVvyh`DH;j#Ce$nmRR4<<{<^VKZ1ud zI8Drp;GC*Yn(Bj*8%Gtz{h~&H!Je3z_h}Jd1)}c;lfsU5gK%h`pys=7Z+s*J5g_3z zjJ?p+J0LrUGzZW*4FN{Oeg1u;lLXOc$;VCJ)V9<;W!npqn&6?a3@L{*w>UjMaZIYwH8^N41J9?98L)#%T zYUa~S0#ayc*jC%m<#oUTglN89tc(Xl`$7<3W$>hb7efM$MWu) ziM#JhR!LIg|1dznk?Kc#>8L3bA@bCdp))~^YQV7u5V7V;leI2tX6XC>$|w7xP8q|X ztv^h_u|up*gt3hCnaF!|X4~NyR?6xeVwlV67sUQUjMKU>U&P0@33$pegHqnuX_>(F zB%5-*qwZ8@Ix!(>VmMhDU%y|rN7}v0!Ct1-w$0#PDjmuCjfL&1hzJ~Nk(LQPQ*kLh z5aE^XWz;LX!Mqb?bS@GMBlMcx&27sZ<8B?fmfLZ7I`n2_P}BWxLip3pYhiCCNOZPB z(EM)fWkCFJC$YfRr z_1VUkP(m~s(*aH0bo{pUgYls?K^u+AA)|bwf>w_q=*^W$?rGp7|6Ma)WpD;KSN7zGw;oh#1u@G@ z$OW>KCY1y_1j2GTi9-$fO0fiNB1b zWM5<$O-8v1FP*d2Jmgc(4jjr<%C|6Va=l26A)i&S zK~nFmzO04^`~1qC`HsJip}C13=EC&WM)hvKrh|ufMT0)6oAyC^`ew>LLa3&1dq7k( z_F_X(S4N;I!n0rm0CI{se6+1AtJ@^gk|L_jT2du=A$<^exP5{!y(J?TE3kn81am2b z1wFu3>CPfXh(@A+Xf%{-UB@(5ARXzX+;tb(xbIkF(&DwhSR@5L3Z&)WlCkYW&_BXg zc*|!!LLCtTyG*z4D(tETr}7mrB$>KQT-ekj4&nj{H)lkHtAJRJRG(s{&ebdFtm+qP z0bR?#!PU<>6}*LnqXyKaUiUvvguh85&9dZwUib=RA%XMG0rR&4SiXF%NjXRq`BL+B zoS4WPHFc<0xL8$-0S)z8cO=uhMNr|9hf~=%yCdSUpmegq^*f|!K*y?$iIu@o$a_S0 z$;|z#Z)=rBI5Y5aV<27nI15<`R_sg8NO62~c25%iK-QIAmux91_^8~A0kGn4%567h zZRCm?vs~eClrhmIL8!MBpPWZ`(9tcLasShWr0QO#*;Zb+4fr))gL{3N8)4v{FquP{ z>UDgYdu&4V$5sS&(++D7qltVwEe5{9*aUY?mVyctZkSa2efq^VYm{1MqNt9ZZ1-5Y zNJp@7Cu9SkdNzZhy9gT&yr*XV*A!&rHv*x_rs;iIOh5fPwn5>05u(2{@5}+&@Rqk4 z`V?b$209Ex&=jG)?4TMa*d#WDG(K}S_CWPRqF56)CzX|=f{?qP4~gft`!t>7qR z3scIeH(+z|u~!8wz45+(gr8_g(sU_5Ri>uCUVn)eM+{RTivlOq*> zuL)s*#t~3f|GXk)={32u(5mHVL|A@ZvYCKbTu}wj}ig844Hs?WZgkT~MG0sj&ZMc<0S`_-H zMDJDaVF5#)0&~*x)(SY zQ;DmNXPWN}iTK*3Ww0Q}x_Pp-cH{CMd4jV$+@m+R% zd6|Z3C3rmN?__w%q84g^6YbYy#n`RM3vVZrhHldr1?Ki@()(rkGj(K7fgDSK+sroW z4nqi7JWCu~uX=^<0Vh+B@$I+LTwgsoo!?L&19XX%pl?_@HPu-HL|4@Pu1_9)l16(H zCYk~&?ayXA7lL60wIh(U$e!!8%q;i5yD$2gf@`5_Qn0Sb?7oe5zp;2zONsOcdu*G@ zmUQ`YiZ9N-oWK)pg4NSpvNt8?i|Dj%=n*Okt%=(uRH;+12v1vgF>DTHQHcU|k%e#% z@2%VE6-+Fj+xI{9pldl@i}H}j{(uc;B26Ck+cl@`R|uFp5<~&)sD(>ZuX(bPJ=9@} z{|eLPr-SNd_rVL z-#*;^?(h1A3_nC`2IK~7Y}W2%I>VC{(@iIW(bq^K3^|ZNqkP&!bDrRu797@!yt^6L z4~lz`gwohoIsY|b<(Oz1Kx&J5i>--7WA5(P=V*G7_m`<~T@iqEV6@>@q}ARSGAT=`}TlXxM-u zRAuuOAa!GjoBi@y>}>I$s~cD|K+R!~uFpvjE6Sh>RJ8t-cjHPRJreGGQCu!?BN()K z5JC23A-Jd?S(VU6?Ux1Cm=*V%^9LQjRxo0A{<@jmQQMYxR3QJ6wFo1LzYk+}kf8=n z+o~ZU2|6A|r9$$^0lNXvcan7DU&{PKQ%tB*QDXa=9J5WOvA-0r$W?sjO;JoeG@%9} zZZk!YB5B@C#+_VofZV_k z5uTO=G3hF1&BQxABUI1|ukvcm6Sl)cDZN(w-Uwxgca^}yW^}mY^swwW?{@B=&@$2boplkr>Qq4K!izxj)~AS1*ht}FJF5;KiqFq z;6lt((q>mSxTHHjiPs`@EVsk^7hj*Z-5-?@zc$a;%l7zg1~m?^AgwG>ybWj~{IW6c zPfZ^*5qWP4gur`BTn72Ky`T+``HtgjxV#q5j#SW%!1*5bi(i>jUay9H9HD`YAK>Xq z4hY2CtYF#c)xlEsfZ7729Glj&8+i_90Wz$l&wD-9wbQXhajHw#9hy4VMMAK#x7T=7 z@DqH1^&d5~ezWIU{+=u)u6n0Os4d${YMXxXfsb!VBgv|%2Zpw}LBMex^OPhN9Hj+PX`rM#-& zDiz&^Z;?b{X>CSd(m>q`F{(Odya95R3x8`m?f8O4VPfsmL$CbBKx%MU&ow&ZIGpZb z)s>fcK4sUe;^z5w|88kv49RSu`lvI85`DZ1IQszxb@^3>MXV6bY!o?O|Cp&cS7zi# zZz^HO7>3_t+3*#|8>0H5ip*#VA=yC&Cqatvu0F7AsNlt7cLJIvY;yOGgaEg^;u&ZL zj2A)a9WM^v7H%{n0IfP}uP~lsgZ-K^d6Er-R{aYByAjo~Uvk7(Qi4Icn>%mp^j#XsJvZx6J$X4KT&_o!20z4#? z>a*-I@cA}3e~c-7P#qkst!gP$^+gi&|MY!AcbrpRFi6DX%vyn1x_++c8COeTOF6Ls zyxtBAKyY1nCL;uQPDoW=S#PNYY^o;6k%L{2){D&8)yPBTT57x@>XcJz>T4}LR{JhU zOT~_>Zwo5ivMWlmx%WPgytbNuKo3I&hu0y@2DyU7kdc(v{*Z>a4s3f@W5;Tb6PQ6} z^p9rz5#tfW80-EdDTE;(Ob?WI+sCT*oB|d9ZPU^G^q|HuCe*{Nu_E1>7 zxq#;-t*lCA6_QK6Ak&;(6;YosZD0G_#WpB^3^7MRLZG4WxEjendurQ1-B$Hd7LaAs>;5xC4n1K{Q!GOw3f14xG0a$2e@8VCzCS32SMAi$y)Qf&Du6BcF{U2|$= z0NU)JWB26n8D+|*)*`tyCX{kYooP@7jH)GmV#i<(+CNEbHZ|$vB9fOWH}k`=GC`?j z06DnZ!NDiqL{(PB7p1|no#N4YiI1sdZze<-h7H+>lI0cHJdnooCE)@p z-KsVb+InxJUg$75&meOaOIj5|0k865-{)C#r&+ULyn=|MHjB1FCE^p~+$=|RNXWNN zNfod+RBmad2?op!=hZvQBDxR|e>A_E7lk>W*0U!`qv2w%Q53Vt2vB?_tbl;eybSxc ziMF;)uZ;i3KJ!!0WQ(sc^p>gJ5M1ge#{M;e@-z?^`|QoV43yBV075xRcWb>#*?u1_ zq^LX<3==V8k^rS7OEB@^GP1r^2>K#6Dt6vF8k%ejKO|ySqB0ZRNus^4Hc)vBNjtCJ zEUrZ-EG^ma&DuYewGg2Q_@air0LUBxqd^TbdvibR+T)m5x(iF%!*9djUKCXv9YHV+ zoYKCTsU$u8DV1^D`ow>-oaW-vcJKl{{qS z4eQI$2qF9DHEwKhA<$bBgU+4u!rTvHqG8Zf#UbiI#Et_sP>n0lJrU}DfN;5(a>T#5 zG1S0Nx(x47&1d$B#S1AU5{Oi46o|R_3Wej~;)60xNAbp_UHnvoEG`f+A9i?16!a>~OFpnoB&gWiSLWl&>I&`ecXG!x_6h+y!q)dm%lCL7pYA zQvoX*A+z8)-#JeXWBO+LiwOV|8FCIDnci)30oMC@{dSg}rRx}IRs<}xW<@k|OT&gj z&mqQ7E2|5T1?FPUdEb%00{xIEISa-UG0G_7u@86I`zY=jyl@Gi@L<+kpLUM)n>-+- z9>Pwe^$~gtEb4MCC$2uM8L#}I2_g6GE=3ca*_tw?s9^9MjR7CNFP4(T*(2_?wCLSm0#^fXJJ zVCF8$#*+YSfAMF5gEYuq2#S8=5<&v#L0-V+boBvh&RTC^{{yBe&Ii-Oeu^De2Glya zr=wIfp3H|X#mesgsRhB-?%J2jndAI*Elkc+?A@jW=(Z8y@Z;hGdX+|^S;L$P(GgpLr1X!jp&u(G~-iE-B> zx95?Ra1wQ~=)14juyQE=8DI|6hW{y@JRU8V)bh}GmkFd4-TV~(k3ITLvr8(Nf-P7B zWUik{3kFi0KTb|Vg?yoR8y@Aj0VDv2y{jLr4gz;l;Y#~s~t09ffg2_ZhSEiCS9_)%EVC{tZ^1h_+v;B+Uwo;?R^A5kiuq7p!e z05+0KBoEOMv`PGyMMigz$~HaC-Wing?!gMa?V~4b%&yj-JHlB)mo<>`yfvfkp&Cp6 zd)GU7=Zr`@+ONBTiFt+^fvmXg0QuSdj@ndci6PKejXhxHB@kA*KUD39gYxw8GVgxg z4?$0&pKydb8-OQ;nLGMm3wkP@4tY|%on8*ZSP}nu7B*7zbyrQ)JihW?fm4J2rlbB2 zE{Dk#^#=f6N@GkQ7QLp^T|9f52+Xi9z0sXj{C5EfQ%LOoC7qr2TVo?td=V!?~w08UU^}1|=(We_X0FZS$0**65}7Di0I6 z2yBQSFUSy)582TUwuA;4$+{iMB|@nIP{|R9%cc;trj-#=_)-RGeAXK=K52W1E=9R| z!RV0m1Gxq7VIf0KP*ASsB3J~FDNJ)j-ieC2XPv}*X|=SFH9y^)qA(hO|o%_pB>t0FO!mpb6s`wMbksMtxNXbBG!k-AxHRu^*+rqd%f|Z1sH)1-PA5RrE~SU0d;qCiU_1#$*Cc zqd+y9kIn>$W1nrN6QaNK`Kq(5Z0}wCiJ4PkbWvp6gG;5%%IQSZHigjwm2Db-jabiH zXBhQvjrw$=ADBhZ++--A$U_r}nDshg?(Z8GF&`4a-Y*>Z*Dv(`qkfvvL7|ESk)%fR zXNcwsv1}A)VvrRWU?#|1&W)k?7_Jl1D&egDtQLO&a|vH9k|R9uw}pO(|CaUJLX&iq ziu8ndxv-Klq(@&Fthg3PAB*c_P)aQ;!yx^7%Z#crTY(VWJD_Ci03X7pC8?CbD3=D4 zs_9jbpTK8$DQrT?)wdD0692E74j1fVG5fxe`=M9-IfXc&ho6J^3J^WIr+4g6jEDm6 zIoUZ<&1nR>^Q-4%!*8qu_GR*ct4Q zXMgpQ(;8r7<)C#k_=U`1P*$iHvtMhN437tNm=nGpHB-w8xxmFS#Q53nt2H>Q3`ru_ zpUX<9I`lsa7mipol;0r+Op;jz}im{FId|dbI383?+QF_$rXg z67<6^WMMRm>RHd9Q(0Nnz78i;cFE=HKPIxL2x&|8KzkK!lzw!(Qe`N#r%@KfoG$&8 zq4d>|#%_PtSX+yo&M}3P>F|9JGDqcV&3TAy(s~~C6pq&{3Vc*Cw~Uj>G`1g!m@|1=Ir}P3-pZOYTnBW>L;{UL6QsH0{-;t?lRi}PN49De}y zk!Pd>>23yQV)DczFQE}gBQI;Aydax5KKPFM<@7=G0cV*myz387c0vUkjMgG$f&X-jBB>Qgs5 zQ^3m*9D#{aQ$1UIBPX{MqdHExn7gBv-7Gj_G@OdCpZR`8t1QUM!VRlrln?6zl>*0j zw~G%jaB#!*un#FI(P`Ls>s%7ow8JL!Vz>T`lUf{2aELKimpAJDycsLO9UmLKoxHCj zU^?7l{wHBRasLVy0J+r*%-cmg)q}Wb-;Tjz0VcSAd@sCB&OIr_6#5zUUDXK@=_EC4 z5qjV%L+D-@{)nUlM!{iko`}C2{XH@kvPU+!57-#l>tSg zJ(4;*^T`fi+1;zq8aER;W8H6ZW7-&BpJ5ZvQ}C~YRgLOpH1gh=6_>8m%EC^FzL989 z%bER~2|(tj?iEaXQhg!$U zv!a}sHREnFl&UALFd;p(<~8?Gtqtk5ei??(wWPg02~&+l_&F4C6j62j=vQ66J;!(G zot$|dNecyAt;3w_WFx&egr5UX-Ru`C)XEq(fA^kpHBWBhhyx4GUz=t3HvR@=0v3?M z@?|}Bq$F&>W@BeGrjECw+HL{e{fxA=eZ&0ToejHtY{SpvrGV2%&>cF#6OqwbRnGdF zIl?k`I>vyNd)tr@+^XjZW|ZYFxF}L%?^)kEnk728S>E|~y`X23e`3b9G(#3I+|a<> zJN?*`P$Hp-jJE!-HHGV}SGeaSo9teo2(-=X(H?j`V4YFd@DwFbiyOkAGNfcrH{3@Q zU|Zh@L>nG}qERXi3T?2|r4)FgnXA_nyYJH~vT8Kk=RUM{|J~hi@n>82l6o$aMz$?6yac?emR4A6v=6RNz z;_b5?2mF#}y$^ji*7;Hfm)ECTC@IWI=fhzV33 zw$)h);nN4jA$c}KH|<`sfD6wUad-!gm9Eu{|AU;$M4Zm5Qt9dmySay6Kr2!N@UL5I zw4}|!;o9R;MvHX!mIc+e8M@0r2Lhb#33QH>^WLd^Uxy5y$ftm0F-Rto)p{PeduiQ9 zw)1JCo7_$=79|c|4%~mVob3MSOF29` z&DCf%N5P#N*cvNhGeBM}c#?hGE0DhYBIAd5tQFB0vEd%}x}Df`x2JoUI?!Pzu=sAd z*mk9GpmYjrY0OOQrtrgfN-F9E01&&%mql&zRi3}y7#aNVfWKf;F0W<- zqiU1RjOc&2JjHc=+7YT>)SjBS}>!?U?{{vao3DPV^N|j$Vs{e@%SfHyK*;u zXy56RSDRy@9*L^6un8x8dUYu|4qf{0JTNK%oW zfViNSJqmD#ZK4)VOU8jAo=T?otnffvdZ0cbG5HkImA%tLGO%6+g9+$F3@WURE^X5J zNB_@KwuN>9%G-()fSp(*H}W9rt>7>&b7`0p=KiUq2&UYt9lZ9O+g)d!zljTZN@4oy zSPP3t%w{2nFtvbOdbDFVt=GTE6+}rG@l$ioE2^?8?7!8vxU{FeqDAKZeD#EljDfve zsm$bIPDfZAA26#!ZYBR-;ItfHyYBs>BUvu|(7VVXE!Aydl9T$F+=pO86J$6k_B2M{ zX)saD3k}o#Yk??XkN26+a4s*%<=yflwd0Cax2RJFEV)CbTvdJHshoWqBv7R*)%h*O z{2FQY*kLchllV+C09s<+Ob%6dAlD@=zmNT2Ijr(Q&a_q#HY`wtcshCUR3F>95)YQ? zGs<1KVWgFwy|1-Gm?nJR4{LGY*57E;SE~r9F{-p=!jG3jrGv=XeJaKK+!=F*!oo95 zP>YIAVZo{Cj{cb4=Z< z@a3VT2Qh9=YV%xvdO54@jO3vmt0BMEf>-Ak;u_0=BW>NfhD>A}I1=mkobt`lDkLN` zp!9&bTsREEi$@8iF-iasqY?QFMo-fU-xFL>8Ve zrW*dzev(PmfSdxSeoZI&H^E80HoTDk%6E{N?r7vI)C$h@YF(C1-o!P4CFXhRxqxG<36u(`{ms3Ql%%d*o^W&Rbn7@e8V@5YOhNi z#ASNx6YO3`Hi*Ec0UDoky*C!A!;O)?!_M{7sx6;Iy{}S^_;9k)7gE}4;;O0Tz+)0= zwryVQe3ppJ%R%bg?UH0R*Tim#zDQ*gYKqd%n`HdB9G*>w!3r+-ZSUSz-2>sZahh~H zHG#oB0O-dr^}R=?c&_02Gt~YPgUK;tBo$415DbY3%^$$Dl>)qP8k-XEPm%E5~FX+f2xcaC9o)pYNXNcAufl_b@eLesm{7`lCcSRJdOajvF|c^W$X z!^5Ovyb5=|J0`+7-9@jOc}fRjDP6(7ezC8PfM>DBoYsT_xTKmbBIk|Xahor`2o^STCMtU{L}a2Hj23Srllq93KCMuoMh6#UxbhrI{gpD#1hNf9il zO5yp@E(J4I-ZGE;{b@gOI7 zx{lyT=cu;m%mvBHX5w)o3Y)c)f?Ln}kA@K5+!8+{b~^R66A8*n^KxIPSm<#`7ENPt zd67OZbJ3YX>1oa7oMFmjrNGFcCSL`M#f1gHa8G^6QYs5^ zo6&Pfk_sz#(p-`iiu_$8)DR869h|VG2|dCewwj}V|FX#wgJr7hKpdhu69ebTY{k^X zs}JN8>&;DQKw;Xo#SuY9o@=5TVbs5jhM&)0E^-Cr6==#sbKM9BDMcIgNUQ`i? zqEwY=L|*xJl~bI$dJ6ZL7jA$Z^u=wcqanqA&-fv@^w6V7Ga%4d=kuZ;!}J0S=au&# z%<#?eVNwb?xZio?n-%s4=P$R1l+_3E;e>wN9%4Zym@Nmbi7{FR8|}ze;FdQK#mlfa zxSr`kg`MI=NZNihx*8E9ItcsmGS&{EAn?8;WaG3M_@m-q8QATrXSypeKGQ(}V<}Ln zt}(CJh~)d9YE1dEUNBiS_6K5!>-vMy=!FWHrwAY)Yd8^AM&2jrU@?}<)weTYC1LPa z;f={W8qXdZw@f3UYH6sbT}Glym?ZtstJd;1`Yi|8#-P&n1}YXb-^oRrt&iGNir=U} zr!^nL2Mh&oBp@Gz0A95l-_EVh=A&a;_Q&o%XiV=(=R9SMPe5q$3$r2-2UT}?D_WZu z)X4J#t&3=g>fEeqR0PnUW;Y!nfrU6M$Ag5#XtZ8FB_!vqRU?FT$w2N)zf~&AgiSMP z!4EZ*Jok-V@%!v5Cw8hVc$8md8B5@=ZyG2~J$k&#z3uiJOZF6c9IrbPIP%g4Y@v&P z@!n(;fqe8rLxO?HK2(qO^NjBR*3a74dALpDjl820nHVKrqAlN9PHPTN$AwH2i5r*PNfEo+ z&z!mF1}ss|KR74H7IQQNGLyrE_lTm5o=b~|5C*A!^K`7W#x1S{Q^u@A;RpxY!IVci zdN_JkTNS4H;9gY5L;b!z@$dmTQfNLUM!amEDq`%k(ZuPxz~O~OF={m^-Ak*9 zPO4b1K$&YESg$b6ym?=#=-qx)Dp`>CGz85@5~fs+?Vz#lio+D#lY)KOh2-v8I2HOB zS)2qSIA@5HNrinzZvh$zUM`|B2b2=Qwxzy7u6`|p2?^FkKrNDItHqjB_b!UCKQlBO zh^p-*%!p9Q?KX>8Vh;1sf$(BU=0+R3cnv&Ut2y2cw}LZN#I&|B+uh5OMaSrVCHh++ zPu%EByg~J4me6SJ=^vO;CNSf~=+YtQHnJU!K8d0KPt0k=5gh-mNLkV-wF(roecE*d z0*Hi^JQC}5)!)+#6QeY(GWd2&ub5o-%e-D<(f|CEhuArv=cREp^~{%8x)G=u?~2Jp zAEYh#Hw`(jl5R!pTA)Xsh+LVxcS~e022hrYbqrOJVeXAqOlk`qrcr#7c*sVR_wkwy zF0%@qxiIi^<84s!0oz+2{=taAZqO++eA<&{`YR*Juyc8FEFh z>@EBR*}x*`UNHhiE@??lQ^Cci6PYi-!+Ik8S>}POBzI}RbJ1ey&&-$ytDep?uqMNp z`0TC?Vg22bWHC`0=kdwkB_$I7>OCQoMq93HNNb8>0Q{wxTm^_RaDH-gA{4Ozh0ghSvm91QHt5b0by=Fz$9DbXJW45~a2I znKLpm8ooLz6m2ldk3NXl?e`e%oHj)bnoW^Ghhp^s4k(cNm=A+1yqn-iLY-a^mbZv@ zihxCSud2P43nz(oR_bEzBNw$pWYPNvNPotM-UH=nnjSAjXqsX9BZ~H9Wg_nqcX${f zm`TA7-E5HRh7u)ZR;2v~K;#r9#kCr;9W=*I__~D_T7DiK6@u4iB3=kSyM*KBbE2hx zd6Lov-ZY-grjXw-a9ZPDf!k?;4=0hcoh-ix6NZ2K;mn8wKP$8Rw(_bWLHlH3uq{>^ zr%tAb=SZ(9>Pw8Jih5+;NHVSh77H7yZ}N?PG~A?V4a&8>AVDq=R%!k3_<(D10Ml8) z&0cmJ<`fdoXW--`Wdrb_&vti~Yaj#mj$=TA7fRIvD zQte$dihfT3Q+#&aF=c4;T`Oe@**X<>)b(J%&ubSCgeFcEyO)w5<{Pk4DMV63@gm|N zR^gg*iRd=PrU?MdrJ9p#K0w+IFNo)%p$hdj$?Ih*F3Hsr_`RaLTKl6|47<3EQiFIL6)L=#WKm!%x|uFBz0vS};VWwK2KmVv zb5zklBvdUbGNrA6#iP}Y?EKUNHWlE!=+*T2c?SB}QTb&}jSL@GwBU7h>%Ie-xvRKj zc6C^_%xYXRUR5EiqTK2nL%Yw{0ys1<%~E0+&9=ek@!U(#swmQs!)z+W zhAG^=6Ng2Iab8@+*+GDK%cjDm7x5xRQj%62UxQSPS9f&Zof|G^ox99X&Hs3hLcc2E zBN|MZur9*-6SOZMM%P1RHt<)#$4dj`comMDt>=>SwAMGAw0N$TgI;}O~D_r#aP2Y7NJ0eA-? z2T_6$;v!`~X{ZB!@i%^sz4^0A+(~y3HxSiEr`Hs57*1iRs=QH-zPcajMCN7|6N=%^&#?>=`jVD�(EG#S!)Co9H zi+Eqksmsv_rMAVgEcbj=f<>y@n;UD2O9pd|S!Oxn->4mnYXyr{{^t#Z%&q;6rQd<` zuFH&B&uc!mzT`RvuHL_{YYEk6CN5*(!L~14lZ#1&K?qQ>%Hwz%AsJCp10?2ew8kdb z9F@y*+Roc^KCbPba4)>50N}^r05GafnMu=T8@tI}Wl`$_+(Y^wRG<6K<`Q5}+q`9o!Q*CW6ERb5lABM6KHmUeNK8y{& zz>E8+ol`LWL)~8lgk!czqs?MHE%Y+hu-7$wJ4ix&%fN=8p#jw84xrEYbDWy(S&p=x zzucGuXSq7$w}vWV?}j9JthgPa%~?ZoM`{&J1(mDn;1$UTQ{cg=IXQ0Wtol0Ot`GjP z&5F!Vj4?XNk>NI9)%g)hEM4S;6`W9hL8Ww&h}O+@%~k z|2t8}ru8JaA{r;BVf&PYQI4dz%cutwd8JAxy|PraopL2rmT!Orh{ckls}|T?P$yeQ z`S_!7jsHST#h>sh6+M1S?`V6_F&+?vQM%Z3QX8HI>M3s5ij*158ljkT&R3QY0J-rJ zt$t&bH3-XhHJ^;Nbvij4Oqn!hSIcZn`EHtRT#21Y2JjzPLxae#ctR6_E%44i78Q`W zbgDXj&X9+$q_4Ntaw!Sna1SM^DSg&RR0aj=QAQ$2DL**%wO5fN@KMGOi&-3<;~5a< zWA>~AgUU2rgPT4alLu@dDs~Eec}ccakEMpwuu_9*+;fj)JcLAb9HqWPlZrWuT2xhg zL{dCl{6bu%5aO%js$s=DkDrLIW4yceYHiQ0nAKkO8wXpG)lJ)P-l8lp`ymJ~pA!2#-3Q9?dlsic|4 zF4o}w+W9n?wt1N(y`&1%gCbmfk;sChx5z(C;d-~?=6~Wa%cn)mj9Z;YN@Y%QCR3UhTL_3^PPatpE0*w3&qyeUt%d@d@pbI>Gxj0xk3Ll}na z$mhWH@t|>dWWLY+=g6cFn`&(hA_m&^?7^noRH{KN6AJ)}2i@=C40`4D`W#+ZuTu7g z(Dh(5arBp=_||afxww(h3tKXg%UG$Ck+0Qv7)j);OI)8TGN<)ZUmdy#m%8s0>P}Tg zF#s6k&Mfsho7%?RpP~^iXi!51gn7kE&p=@^lYyk=(m)1yJR9v?>@m!x8YCEUoBlHG zZjg`Q6levrtqV9|UmErcg9Tz(V~s+J6q_hFVug-$u3hrR?;X@;bMUZE>?pXkPFSY zX(cffQV9>hy}gEJG#(azNBa^u_47xkBTV?9B+J^0>F~La%_gnVQ}z1(g9*`caV#Py zZok4iYNz$UeTT|p_Cyn~hT&h@*OwUoHxtu7zJ+vcQC{0H-lYJgOFBnbuMLn)n|{Um zaQZ4~;>;G783#LjLibie-q#kj)K?|#9Y{NLmWPXBvM_pD{@&7m8_43}8u)X3^hFKCTd*Ifx<8_nSDTla5(B8r zsk_U~D#K%B$`^CeA8@@6T|FA*iOJv_fh@mVsb6dmKSpIM5$tgTs$H}WRg7_=(tR6^ zSn2V7Oq?Co+5!(BxW+Ln>Bh0>MG|Qbc_9ZRMI|#r(hdAO)n*{+PpeSNL7&1#u8989 zmfx|o;$rum@_=NUh!5(X0!n3pjiv59IHN#Zwq}Y zIBq-W0suewRBhk}GNFf+fE^JIjP9aeY9!0krpWOJJ~^t^owk`oPca~u;KjFWl3Wc= z?;gyw1qg@sJ|W0PgQTG%Lh|{OaD{0CA|JHqN!K9zwZI!$Rql&~)Y~bd74Nl}_5V}L zqSdqm)-G6=hVYC$P0S7SNdh25iOotT2%7c?!{ZZ&D-2@FXnQp>tzo7g^jJPKuC?-g zH;e&Vn);K_p+#rs8N8fnHa$@ubvK-XJBz1kD0gU17hHbwGPc;RHK&BVDQMG6a%4_W z^b4x@KU7F)MhbaTsA{wOd$wqi%VPe&AT^7X!37098`VqFuUrYAq`ZhM)Z|?NF-)&W zNR720nOFO%DYP^mKCCX!v@WE&8mZ6q6o$CO|5HnohFPG6mQwvP7JaZ1F*NevaM8t_ z`my{I7|C$2WfFeZs{!r@sxN_+s4`M9wl?As;G9{R??aUlm4roz@ki8JU;f{!j^5%{ znnmayWL(p@5iZ{;9{t#Gzxr}d6hN8bF-ABbSEA@fL96o8e5j}gwMx7Oh;;Uj;r3^R zvWG>lrWDU%mpeNV6D&(F^&a%78Ojj7G>~~jdeF0kkPa{?tv2ns+&5GcZd-bnb9{M2 zxNsr2o9Uft{aJ?{92u^ln#6*b`4MHZtL(e z=&60zs?$ho0N$OqvoT>~T_yZoe!~hyHJsEw{xfLH$Vsg-MQ9ChQ+F8K3+2ou^>XQ? zY8w`XY8tT#5lORBB`25#DksS(W!h>Z{|ES9u65L+RwTzUtAq8WbC7X2>{Le?CsC2h zgMB&-irI86Bu=L8?Dd==cS?NT-3Yf3I0GlvhxAWd<)4J_3e-$w>8R_qf6|D#NTA=u z+jpqcec`FJ^iAI#_Yh@@up#m=5S2n#fLFAV+uII-F1EH|_fr4ig^jtQ7r}WgL4ZeR zRS1o5yH8HfSA#9R(SYAZs(Rb=S|`~`u}5wAtyO{!kKHUxC{qbk`k^TcGtMiN*0nRg z-}yoD7lI~=iLo*00B=Fvpms7`OcIsiyh{dp!Er7-;C%|ZQRzw{mEpZO$&!ncDhp3O zvY(VJIdIW#IfthcD8a@W_&WaPe?{Zz5=$Mv{?tA084UEGPD$mz(W-BZ!-@L&ATGDc z`PoCPU=|w{4q8l!v#iY^;Wl%vvm(YGJlIt4OjEaKbflplBe_)A>krUY8Wz3`Yyup7 zRak`1JJLfvr$ts22ohGn0!LV!^~?``iPgj;@@}^?t!-<{ivD1KzSOf)@1q$Z?`tNv!1kJtgA-AI?8X14n+HP^+HQc&7RT_qum%IU{}e)@zEPkPlW0_`#WVD!Tck zdE1aSzDE5%UT@O0@$)3((XI9gfb*-t7?J0Ftg&~owxFbQ63qL4c>zB6Z?ub59)$!Ak5w?_{t6X1@kbz8Q31n0eCjsGoBzi|CrE zE;$8+V1hiSVh33`T14i|F_?eG0um+h2*>NpqAgQUZ(B*R-I}yG>ykM%9fz}xyEza_ zy_CCb8(&xdRBP9X%H^B^lAD;JQlBW>jws9-VuJcF_D8a-r*mU>Y>VIgkRj8H_R+VI zaYR0kOKeU+fqFY|2UB+77T4{z9|VO_{^SN9x0tI{RZ~YM8?I#}x6-ju15I0fWfi#{o8C$Nz8^B`7wQAO>w&f#if0UiPXketn%;0wr#rw10f4a^KyY)R7=kl&=HvuLBS+ z$YW8KJlzryA!jT)PV6f3W;vqXa1I{Tqaqa_yk|Bp@9zFU9 zU}*uYEW)D}3^^L+0g_3Y^gq&P{M5%mms}nJ&`NtlYW}t*m9rKb{?QS+7>6EwdMuK7OM9lVu%Ix>H}ONBmy15S z`58>LfPqzOI7?0hhGOvs%!H$$LleweSOBAP!d`sVzy4fvO+)N4YDD}AED4rP&L76V zbL^D0ibbhKU|j+7y~&9l?o*Xu6oYZ~^jQ;!^EG_F6JjAo{0xwZ8`ZLtth4ai!RQtI z<2-VauoH`5 zCmk+QXcXD1!%JXr@r<>@F}^VeQcf?LdS^ChoK+Wc{4^Ctuu9t+qU(&{S?pR&aK7m$ z2%B$gj$K$jLY(n(mh#QcF&=W=-{rcgnWx+aPv5V{TBXK?7U)}V%k48M0GXm$9(azg zKy|=kTU%(tm%c2y0I|tQ0G>O%8=7k#{_^^O5ZlQVLrZf!7Ej2dt-X@ufOf>KpvYPb z0|{GRB_@n*wq8;Fo80^zNY^NSS_-x;pdTsJcOUDn2lNnIl-~_8#d8c6P1^dZ|A!&; zv>&ZTl_Nv#eUxyMjCn0OgR#D>a{~VRc0g$7h3h@cK|4+G#Q~5VgSny$Kr0{*uesmu z>aih&2O%iwkv(6rdap`QGFn|DX|(I(b5hz7BG4!v4qpM`Vw`LqK8OIb#}pp-zTr54 zX%73NjcxiU);9XhcVdca6ere-B6oJB9ib>6I#*93m1jvE!j~83un6+J=$L!`TRvgRpY~R#=8nq`93sy=JFl?FUA{ ze&%D%Mt@d^Bb(f*9fyU}?By1rbec#Qv?+{fX`adfR~iODL8;hwSi)9lywan6F%{q- zMqOA^N?=qa9C`TdWk}Zpu{)Zo$NFi0{`uUap?sPEq>JguefoX|&>8~yrI-1h&kVWj z>hbd(8~HGWS3jc=epL_jFceuEls_V9#~%4I-Rfokh7-N>vbgd>}GA|kooqC9NitfWBAe@Vmw)D9P9)JkyVUlm(0MBzH;%`sh zb1(m_wz=y-$IE{gZ8P?+f&3d^MKptYIO}YkX!k%NRT5YJhQfg&;!(lX4YaYJNMkSn z?mUR$gTV%BdzkYu_ceD;BrdyM`N6}}^-Nir3Iy2oKS0re$E1GYdMAw4Z7~ai&eShh z`q6W9{wtcl?z1!}>MBt~9DI1ouI8y~_5`u3U`vLeE`!ZJNH^L$q|iD9fB7e9g^7l~ zLfD*=UpMzG_J?rB>UulUwP6+5csQlHxNMqpn%4K5%3#fA#Aa%=6`zSBxNMvi=w_f{ zQ_kg@6S}q1&bDx21s4E(Tf3`~DQJ25-IY%G;D#4SaV9ae=38IJndd%&FFt;{8(294 zQcI0#dh+jV(D8YLBAJ~=*6hf7)tUV@Rsnl=$WaTO)?Y#xlJYC+djnW~qf@z#c-Pl6 z!YCJZ2k5oZu$Sy|^H<0zNyJK{%)Z;~L5j@}5p_?S}}9hsiL`kL3riy(XommL~f_zi0GC0 zXt+m)r#grWVa|S_KSRJzR#1;cZt;`}f)vbUabLx+a@xZhfyW{wX=0S5Q^+F{NThA} zP<6E9VCu9q-yQJ@iWq8^WRp(ZA6~I`aVI|;CIR7R@?>tjNQuhimS+2K`%?^HnP#To zTY29}*W$4!lX<$uff`CadmDE$>nnc>33oMz&yx#aw;9OAv;HgPMPNa=dX4O`G16x) z=z-KTaL!k`sp}QU56PSauB%g6B$YK}{eMst*g8iBvBa1a^zYZp*rA3BmRUJx0ginJ z&vfr{)64ThEzjnRc~X3~s1L;(Ro?cc3wc7*0){OHeFFBM<72w+p{K^e-SM>@5?_L` z&8(0#d{uQNbVTe+{J}ksjALhgLN)(gnmR!3p?8s|U+#-{Vz|I=bNP?(NnISUht?4Y zuExII$c*dEvG}#W5$m)an4K>OnS;3Bb{^zMu7@;%`o`&uTFu$Fw{<>JRitU6RgxLF zRQr3P$dMs601nYr-m`q(G#&KdjTtBXj#x_s7s`UpY4JACmudaW%CT~mfg|XnGz09k zaIVO3*(ikzBR_%oZt2WBZoQa*f=Q|F;FL=^BKZ)AeU#=r7qp+I>lD525&D6y*Q2fO zIIvAvTarP)5_nRD;XG}Wik-D0_y07Q(VgUWFWzos^HsI|K0`bO8ZRZ-(mrFo@2x;fSOYgcp>lo$~cx<}9dpfD+xCOB@cR6s)Z0~s|YC3Rl zc6*T*5XN-;dJHoce*}caqB$M2gl_swPJ|NmhSEi_yml09FCc7 z6EZ`!y=Vj5gp9};om1f`4C|-vDxrGN0BEU)a@8xpL=l_$g{zHp+arfIw4ZsFqZ_?Q z&oUPehC;5EEme1eam$N+{S;&;F0q@ap75M5M%h#w5eDTOq1UtPLgNZComlpx3L*K_ zDcDcg(qwIKh)I6-G0R06uBXi?Bo;_`5R&~#(lNgzkcg-x|6&_U5fu89mPkpA(jkYj z`bxS3$704I?RMH8QTkB}R(>Noj?*@qzAQIesG1%ickI@=SmX$#Upl3N>5W7%Jp+20 z;!jfKUS6_qiI|%adakRu;0kvtJ*+y<{MOvt>lxcCx2&q3iOrAxo08IRLoQ`FQt=}Y zkCBV5VpPtW4wLMWlBbfdZvmg52HgBc*PqM(7>TeQdgPB z>SHHh^CkX07N*E8^s=N?TVy0odR-IvY4U~o3j+p(@q$9mHb~mibmakU1l7ic*rhAB zS5r{luuh{)Lg_dqts70=$gj*O*})I%xPV-^fJ0YJ%?y?p11ggodtR#pynQWr^gWk_ zQOFHazlq~n(`wX7GX*=Cdn2*&`}6{ekl9pQ5DV?mGPixxW)4I;SG=1J!iKBdcgQet zgQs)QOW>K2tSE7=SlTVXF?BlOS}RF)Uo4U)(RPl9Y*ZKVvfZ@BJNod!OVeg*CkEc1 zX2o9MM43FiIO=CpddRv^GQ1kgU0`i~_<8=Ov!ow1I3e?Fxvsnyq^a5VxsgkS?(v+) z-B*LCDyUG01)?&q! zCWD-TY!3oJ`-*1L^xd{_i9gp)O&A|>J+pp3C0Zyp`3c$$_#NuN1@>;AAu$d6n<=i` zBpNEb3Bp7iRLCfnGsiaKFPp8$Os+Vdr)pQ&S@qgP@tb-gx7$14ck(Z;U=RFAJ^0f) zISM`93lgZVdF3t_Tq~kE1za4*GBeqmFH1aCr8H{Ib`UQPotl8vAnsAAN%NumQG5N{ z09eF3_e}qQA4Ip1gIJ>j^(;f;Sk-Y*sgr+>_-CZRFQKVZ?O>`8NNv2S(!bl0{~HV& zmRStL8HL^2F5-z&vB1B~AQ(Zg!tIe;DnO8G4)HT zZx;jChXu}fuWn|ZbXNeai!R_CurLNQVm2xv{jd`Q{{%hk1e*DTY@&^Y+%S^Nd;Lkr zqFCL0>vN_@TgxlD_Zx8vMv!uVcuTZ4B~7nW47dz2zpUfY)G_Be9p3p(q=x!KddKA*nOU3bdP}o zjV#9oTU9w6EodBq4==yn({x4`h9e*-7stP9uvHSBPK_Wi9F%L5g8_;o=#U58|`=ZKet3AhEfDsmoTGpuZA z3|w;|%3k2f2U&AMO)AQNzw8e-M6IK%4IvEiw`aG&I6e{2mVuko2y^26d2UeTOVw^7 zm*|KMw|G4GB!z(QEjHX!0ji>;zmhr`8h}+mz<-1gA^Shd$ioc(6TD!(7lk4%DvH^h zXoOb07}T#hX_67t63@H$mHR)pl)z1#@plg%Xr9CFbY%Ld7tFmS`F=6GC@+kIzk)pq zF9!@!g~6RKjaGuuq`b0qgkrs`rTZQmM;gjO$@^g4e(DbxX2s&?rP8Ys0@QAc*Y3`Z z@z@~szP1lb@f;N`AALJu=dvg5coY{FV^=T#GDQZ|H=#rdPia)e)Y`0Ju~KC5nkt(! zUWFa+gwTR2lW;LAFdr9>{;hvMK9bS$P6q6Oe$d**@jZxCxp{_@JY z6}gdt@AQ#w;hBzE$H`+>Q_DB;X8}(_@R{V#*siwu&6rqRWZPBk;H@ovKp{2!C^gBb(^hy*$T8&ghxOA;-0rhCB;->@Se)MtV)$ zX~t`QRS&NH6EwQ$D=H8{k0BZbF&8*vjgkH~{Che8YppRw<~qg)A)iOtac#uCvIhYItS5%gjT9Nk$7e7q;#hh z235Xk4Bj{K?YWp%RNF|N#q3@Bs2$s()Q~bq{G|>9lXn}_hTHk& z?t_uoh9a1qLWb$BY}T*xEeEH5w}S#N9cUx%->Nw1FqB-ioS39+Et{7Y?rHNDt>I+$ ze$^tL#x!U|>uY#O!BSf?KdCBMcJ;94G(7Rjc2yc?$J%|KWO26?zYt-a)h5jgZ;-Ew zYpv$I{26?P_*B^*4!)D0;o1(+aZg{w^~YD;P}=)U=hUo^Czz2Ai0G;a#YiSTI#ZZ- z0q2z>S`p;D4wvjLkg{xY{}oYQ|5KIN{KXfrhpp3OQG{?Bs^6dsLILbfRSs)%@e{V( z>#~!>j}NC~w-e8BiKxgmM72s! zfwEo)nRLmi4I0!I@myxGX?}^xVR$m94OX+JJ~+aICfw(RIrjAUll4qOnxF5W6Fc=u7cF>z2*?z5k!B zWpSnY+T9bOy5_q?-X=77YMoNm&4`DnGUL^a&Vdv1q!h6Ar*%^`$+g%1^tMdH`vhF) zh)l|jK-31?5ru7qbVoPb^KI;#y-LGAfiGn)lCgF1O_fXF6{6+un07%6#j5Zje4*)c zD8s@YWoAM>U)mv9KYI6+x$G1vRUG?8t3o>B|OujVWMzIb0|=;tj&C^OCA zcP#LO_B-d(f-^LqIs2YRCl*Otr|V|@ilCbjtGE1C1>y1CCK;sQ)`|9V*qizP~K6W3IqDBWs1i zYU;~F@dcKwF9%5hHFs$XD~L3xmDj!CkXY^(l&=<*XV$1fj+T##{g5nSS>|SUTV;=i zfWYL=H%xjZ^qAzS|2UIp@&)RR+BsV}?Pjt}=IjaHwmn?-&{Hyd=t5{oNUPo&8)2)4 zPsFK4cb@M3z~ut-69f#Ke^_xQ7R^1E&iuOk;fH3y-)_LFp!ZaeHcrTlG>5tdqFY0v zu5g>Mf$4t?{(O}#p3G6C*cZ)`t8&SU$vJ(YI!VpSW^3fU7UlKir2(DCi*6Pi=iui7 zpJ?By3JaHT{wNw+=ySg zEbZC{zKt*$97f+zQ}}L`p>=aj)~K_TVyCmWPBRY`U14U>;S%XpE<~6;!F?DC{)nJ- zf2>!|WQ2ngL_udn*aDm37ZBgQFhA)dx=ChC=z;6&n0KxF@ai=` zKtt5^yvRa6C~d+5L9CQ$$so^A-fdd;66N$=1Jie}pT@DqCT>zoBkUcJm$=G5_ieUQ zEhsH^cQgpx%1`f?Y~fy|TaWe)Ww$UhQQY}4v7u*IQ~rz5EJ#$r6fDo|X)saQp5d5K zSKzvjmP>})OfV-=8-TmarrIc#LP^|D^QgAJyxPA0$-WYAIOf(fw%2J`C4dBww0 zF!0G094#4bYUaVRRAEU%2j`8W!Ts1B6LM_D$@^GW(n4ogtW3(gkZ4=#vRI*MbRI!f zDGQq&D@QX~4lKdOJ8(Zc>Wz-CpfHdv1CjTil3I@&+sh@QwtzyTsKF&z+s9JbLk*g= zA@G6R&4xu$WdMfZ4Aan2EQd`RG()CuDs_@_q&HtSxVm01On>72Xt@F#r_Ugfd?JXq1uBMm@H zZ~)lIfh~MwIu9Zsu87FsfX=6Ry^3dFHRw!}8X~qV_>67f|Ox0igDV&5&fCmS{8k~ z(h0h6dy2Crizca^s*WKjgKF6NoXOX{6C(8u5G0KCyLui}D?$rFaY3q1171Pi#^(7u zU>&uokOGaesDFgSsFJFpLyH;_F~8R1@mH5wg`+U-V7s<9U$M3@*ZAEi^dfR8XRecB zi~NA#EO0lfFLu&gxqbc#qmXI&gKU`DG8V7{hVnibNtJAo&2)(o7b#nAb;DWxj6JT@ zGHM?)c>56#Gp^RHzvxl|g{(HutPdfPQr*!4BZGm`dM->-afk9#02oCzIlsz=SLoe> zbh0X)mzaq!XJnTgb?*De_F|$c6nM-2ZhOF_=Pi?LJ8g=J|L`ne6E%tCVa%6e zt}+wfa>=WRFJ_6c?OH^0l!OA^^2~pEOZ|2WV%wA8%*H{q-8at`{x~@li_whld1UQD z!@_fM^*I<|08Ct(EAem96TZ9m?umY?y2vGZri%s1)PqeX}e1tIP2l0|+!;+0Ez0z3PWO=b10CLm;9eM>C~<9DxD-CSZ0p zAgc2#-v5Tg;>EZ=V;}4O;7PDymA7f+p1YDOd^3eR?YCjT$To|v@*{u~%%%yXx~@5# zJ?O5A(&UAjl#jJDqS{oL=XYWF`Nj-{;sCp=LO(kdIy)JvJEsXuHFw{$B;#?M1P${o zDQ7utESbQb-G8U=E-%I7;s;C8az}G+!_h(YTCc4@r=8iW-&@AUhJ%JNb2xZmtj7JN z<|bmx7B|FDW~~wXsmQX1_?y@t=+PLmDBwVKo^y0Fx(cVvIo39)L!bs3>6i@cZcLr^ z@__{L&d5K=IYl5X=YIZp8=^m(uSD#LjuWF>V0%p&i#&nISRcljxScwH^-ek4YL-sfn7|iQ(x< zCMFXvAHD*MpQ?*7TbKvZIm5T1q*OO^)+E^p2pxf}vk151KS20F`mlGc@60(MYFe{; zvl7fE+DQ0J-bSCi`nyyAjVWd=AAjjY*U6hx-x}@^Kr^H~(y8%6%s%V5v#z>Qu8>?t zh5DxKcP8GsVT>tJbgc6og9y|bGnUQorqxnmF^4LD+P>uVWaK0ikDJNMlIP<8*LHQ` z0@;Bg>y$=FZ6x|M<5_LgG}`&k_fceWn+JrhO^ChUcumN*ckUT1T$&qeuYp^j9J@Mf zQG0HEC{rvgeaB2D+7X#&>!tfx@cQ~GYB z#|3hE*|eWsh(5hQNaLEr!0SD~nt{%`4pJ(nzEhBnAJIDQGR-?p#F#JJWZeY9c%8JZ zq~$wb@}y-}Hy?64c!AA0fMUJ$Z@cEWD^SvRU()x4f`)x{eXQ29`18~sf2Deba%5=0 z@z00s;pR_IA=f=hu9ghoUb$GisoYy zBh||8G#RK_31mtX_ybs)Fp4Y@Yi;F_xpcEeQPRcd|2cmqJrJ}Z9vP=B08O{L^i2P^ z>mUZ*pVRz=IS3ZTsT*}7I0s|l{2{tH?a5{zkw3h@;W__zd@_NqEB@J&nk&D6!{#RXtS@u{x` zN|Sy$H`;kLUH96BSD7zanhm)+BBhK^XZgkEbJkCEtVUMQ9_M8nX%FGa(da@vv)tMu zB12P8k81jZjr4SJMjtU!;!x_1O~1O3cuuQN`2V~n5aLD+5cy8BpjTc1S=v`Z)2~Kb z$H1Bgz}{{LrWHh<1i6BPipnJtPHFpp=nql?9rBCc=WbwQ)y|v&PQCIIpt$lrU5dlJ z-uDj@0ppz}3>3=E7?v3pQ-c6u8S?t0E$-B=0Ct|DRLtnrs$()Ol&HBpK{=ieBsP?n zv=8K+ybZ4fQFvsAxz>8BdY<((X0X`q+lfO|h|oN`Yjm6|MaV7S`%FO!8`u}JL~w(6Mz5hGpasC2lMF6&a@f%NfOKR&llN+-y4}&-@rCZGN&Pfw(8Q( zI}apH0X0x$l-bshNPu61T;5w|pueOzes!zOlk^6O92u=<5^rju?KXt(YG^tc8)*Q8 zY2>MC4%{iXZMLE<{iz$2cb|cCD)b8vZRmLosqCd2plOkoq<}ytiUxOBMXV2Z;CUBl zvkwnUbY#MhURSf*{VENkx>X!C+8=n}*R8<05MtPbwq*ydEH$tRFKNC{Atmk7vkmsK zyp}64hxGbh*wd63#c=$OzH?Jc8fBhJ`N1d`ADl>7(TPv<5EMxZoU~`)bG}t9jMp-| zZo`Uxj<%kz_LrD0(^JN#>+4RK(VsY6)@rlgQqW`JYpu|lGzJQ-Gs+W+Vbm5zF*A=4 z=WR<)7GT(JOd)-vG9S#i+uI0(;Yh0R@BEc+fi8jl0fvtYQ2%O^@-I4 zccNPMSp)Lf?ozOzv{t_snI+Xby_qD0;kugu>lobd@e$W!u_HvYlabsbQ$|Oihiiac z&pfpT*gx>9C=X))y!l2HVt4*2IXu<{ zkyHum;Z3`RF%20KqJvu{^ReXtJ~)ZB2zteuJj^1JY089HDc^biSIDu5{m0rtKSUqq z4+$|pcW9*7`iFHA+rjGLED2;t4l{yz}0Q_rH-@ zIp1zE=QSQT{I zFFp2J%U#5onExPhg$5NTM0A(xJmxw`;HN%u+x0>(E<>~w8)v5L=p>f=HSRdS19M}> zs{Q&8IUd^==Mpu^8ICb>er;!Ww~MrKvgv033)S=0r$J*~;%_p7u)e*f3VEa#|C%uY zF6^JZXj&_ZR=M$D_i+OKtJM|umyPXx`Z(xByG;WHvLW-RwPrrRdF&Q0)XTM@OT1@V z?LSM5_jWC6JB7Tf6ESjvKgi#n7JuZa--*?hE3OCMSfd>Q?6L#NRjNc!E*~RUaa7@-DpfOxx zNYf8x6i9REwxZ<>*wMJTOm`Ei@8#NQqb+(GW*9MhSYpVGg`{N@^er`bxaD=&nxT#W z6C=ahU|<-I7v3FD$EpjCURP+Q1a3Z<)9xqX;_k4vpEzW%La=!Nox z*u0iu;j-9|ENwEC@J-Ho-hyKFY1I`lBKb|n^39>YvUtKdy!VjhzTA5eh;1k|eiZzB zLnm`Gt*?~7OCF1$zVGwed%)XX)QTl`V@yzX6#k`4j>Gr-y_?Z(^dvL6!ofw#2JWHtgMfQ+f` zk?XT5`!@H3`XJ+eXVb2Y{k46*hc%ASgP1aQA0Vt&7eyWr3w>NEX)L=yRorJe^qYV| z=SlF3=RYngFn*B6o?)azuhArmoE4g=Pm~HW>F2-KEFMp&=J2cE7B) z9DMSNFz#QkB+g+8NKZ2u4Y6}L6nggP$B0yp%skk-PJ&@R>S>OeVKZp_+YO5%mNmE38xbIu^hqhpQSr+xE{mL z8+YgHmM=NTm&gL``evbn#T0m9Nkp@@RR^|_%V{Z4@5 zn)7v3IA9PaHogLUzi#lF{`Kw8r^|XSn~O2JurDx+#Y)S8k&g+Wdk23X#hVcr(ojnH z1O;j{Bz>w5$$;EC@w+gGZWtncmAXMRbmFd=qe~AWQFibBPp1cgoc!IZo2>!eQi8}@ z%L{LqL$hRi!wa2lETIcXh1&baEU0&Jtm|SPE4`@5N*pP~D#{HsN%W_Qodm4cGU3GA z)cn|>!J%$@Z$5qbz?g4v-v*)0w;y{@Y)Nk_hhDmMEE$e83ccVoZJs7bL)JH3nu=A7 zn^s1zl+trErfsOy7xw(Hk_XMDi}bHwQ#iuCefU?XfpRGkd=k%3qZH)>z|hw^&y1;U z)rJ@mERFFUEsPxgCk(|8R)_NNafp3_rxshp9)~Bkc*yhs)uH^JJwW#U94|CbVA_Ox zUHcQ9Q;`-o699j%-`dFZz;a5#AqX87tS+&d+C*Ac^g;qSc<&9~29@8vw+g-CN5tuX z$KsZ&FDyen>)An=kp=14GKd#+hscr9MDWSHW#uaDEM?h;vU-SdcE)1A-C^tJ4rmnQ zEFBFLTCo3#?k(47dcsr4pNIq~fi-bp#u54|Q-2Sk8#c?e-e{C%lQpQT8J$wc0ua+{R-v1< zI};2?R6a^}vOm8}RhN#u-8W0d9$G(?sw>Cg8%2?$qrx4vX}(hbA!JekIgN(Ex%_w< zdo19P0<}q2^8OFeNo(t{zcHXI5;B5g$NsYgvGRw4r2x&=uby-;!jZx1B6{==n(r|n z+@bo21=>heFD`xtORE@$+e9sVlMkZ*2WQ+*K<*`5Z}BL}!X-k&hoJe-4k*Dsb*Bri zfX^&k-~EhqU1D-aY}T0+6?eCJ;CPk9A{8LyWhv$qk#~6uwrv`ljIA&92fy;mF^-l& zX7`Sbr~#-=8FD{xh%j3XG~KTIqaGZ>3(+j_L$qlZ%!&lCe30mY4q86ZXJ{U0*fK23 zIc8PS+9t5aFUhjonn!Hss7xC;7#hs*s=)b@x~s$d(r;w>@@@i_9+h%-Oa`WV5gV+Z zicy5d(-;pH(W^2kH^D*laxP|K^}K^^|CbRoUuUh73StrWgv`bd=s@7^2{2Z#V-adK z&GnWPV|D``{sOB?m1bN6H3jHjvA6$}+!m4b{>C33tp3$+m-MMQ+7w-@f?s zx)26C!~c2{4VCOQWSOJsHhD04Vg^N$@|D0B0IGm!Chqd_5qTojK{J8V6br{Q{(Y#_ zx8(IjEO1`K67Zuym#Y7;kNoY0l=_=@F_Br|ulxNVGfPa+4qzOD-^6|C8j#yl*Ne>* z#a^Y?o38fb<}+(KLHJndb9R~u?s(MmKWhs0TNnf64iivjt!&p_3D0R ze#KtCfM6)UEYRZrz-Ng0d>pJkp3z4a^Ch3!2oXd&QnPtG?SVkNKZhqTJMhh{o4%!lFNAU#co!9C}?P6*xOgvWKAsG$ci=TRO9erk_PoE|!|v^iLRIb~9(^i1pf zKLKAOX@{h0sird)LKq=}bH!GIIV?^`)s)%IStlOYxpD9aM0Jra%mkG=MX8!3A0`J7 z$X3sxX>n=5eRoD<3Z8_{&)8W}0|6gz9HB;|a&F&I4n|N>%xi4uwFL~>*G?kdPNXZE z54D&#Nw=IrMA9%sanM>L#4d{^h7f;jr7<0Lp>V+R5+6>|Q$dkUIoJ1m`y|2OSoE#f zx{eE!M6xa$YA38+T396q?0^P@#PFh-+}aSsU0h-=!j^ULD4R@sKJf)H+pp{vjsdKY zKUzngu=%D8w+IC<KH7zWG~KL53?^t(p=~QO@`q@tL58JOlM01 z4CKva(u-oR6H#s9;F4resx_mtsTkzw^t#$+mm3ammQ(A6KRL!^n-ana5Wm;uq7hsV z)iF9dG=Eg+PP;=qYM<#MUiYH3ru^_7{Uh#*Ey3-EkLkRfeQ>JG-94P6nEnO0fL~v- z^qg21I>cGrTzOXhn)o87>dT0aBQPw^l93589^j5`jEIT>hqQdqC zLq|*U#qk-ay1+mR*KQVgQ*z~%Hg;%n8xeyoeP%z|qdms${4E^tKEFg>f$F=1$!V4^B^ObV|?WB z91Wulqd&+-CT;0g8F*B?uEm3{J4Q_V6M2KHp8US>4Aq~M6e?8hS6Qhc6MRSt^Z7#G z-}3+(?1cv6IXeI z$nuO7=hU6W6$S?dtyHKI2==L2xu`y&IHYkKH`M}K)?WMXQz@|Is#=62G!cc0JvV2P z+xaM=_a9|Lu*T0{4Y4^HH052ROoYSTfneN&!*1nbJwiFk{fBHo>i)iT+YeFbPX8Av zthoe%OGbeRx6kcPqoEqE(2sfLx_D&XpdF49S6 zPu{5{dfr=P2e;g*2mO?39cU;^`(bsIurC=D%R{(kdbix2EDqj5BR<-)q@sBOMduNQ;i9_C7pt-(FVliG_ejQovUd#!eP6i>iyxDQNZmtFmR-DL`d~0 z&ezL5fpsIz-^JVGW|lsL=hs$aTWy>Sk3Ky`5clsQNmKUTuS~YYHjs4ylpvoe<68LA z^zLwwn0Ekf(g$RN?k?v>;2^+OqSd$K=G|+t0Z|au2Sx1}bYa{;-@4SW-T)p){`RO| z^4?tB+fljsNJU8}4*)cb&N%WI{ncC>0wPf+6IPGRK_-jI(5#4NRUmwg7N+2j`AWL? zq=84K#YG0MBw20`nj+fXmn?!+LB7!{8eoPrnQyv~Xw}0R;E?p+ohBlJ0zA~njt*vz zCzg>Fk|FNla=nPJ;|30vT(RMZR@aaXi0Ts_JG9&@wHk5UIobOVA`x}&;eCmne0viJ z`oJ%Po=#AUhkv1>T!$3T^s{!WP-J|8A7JJRR_>NH ztf@qS9%{#G`SEB&EI5!ZvgBUE<`iO9rvrwqNlS(PI~CMj)K(zW8aGqo`y0~`?hz`s z{d%mrpVz$rnqx$#TKDEI7N7iG*-kU82bHitA;ABLhdgYHd3RyWCx{gcrY^?$mYPb6 z@az*{-+c&4@Ui;_Y2Hss&07Z4_W^C2R$h^ETkj2UVydCGeGCC>o~9vV)-MWdGwsAxaX5sUhGNiKEU{RN57 z5Oi=E_w8}SE$4?AVCsew_Vd)iPZ0V)TZB-rp2Fzn$C)>rj!7(yk7YG_p@I=(HIjBJ zdpJ2lmzx6|&w;KG{Lqe+q$aV@Ki;od6PJU;ctV^}M2}uM4ueYlgrCX2-JTn(HBlZF z8G1rQH`r^**$naH@+n9QqZKZK>3UYNi__g+)_r?Gsl@BVbCj`pA|(JVo!+wj4o@nd z4reOU1}3Tq>F5^~+#8GS3Uh_TrNI&DRbGxw5zsoD`M{!7tqHN+b0sZ>C8L!1$4FnH zLZy6R4Lbg}V{+y=0SB0JLe>(v^T%xkH0*rgX6vUMWLsKguFL z0iOpkJCsn^yjF3{lU-03?m4jL({Pp}3kWt6JNHD9c;kuNsIdqqX1?+zKA9ElFPH|= z=$bEb2K9S6JGtc)no~eE4WY9f5S3TbWlD>q4gZ0TrO5o44SQ%8GqSOU;kDD*te&Q9 zFzB=j(**q#2l3fquV!yFvK%a2yhDXMok=KX3`2SiVs1i{IjFH++TbP7K%cl^HJ^hqcO06Jih?*>1RW&mm2QEA`hDkL8wW+F zJU&)Z;V1;(kYe8v;xtg(L*NWvlFqqnZXiTHc792R?>q^J*APr3o)OrU*j(!RGm;4L1`FDfi;`7u^3(mc1x5{(& z7SLCv{-YaoLde!+Y7kmr*+~>Hfi_6HDE5^$=A&YDBZSQ#NP0ln$hVKS#(k*PCfE_Y zrTbWBO(2naucdPr3~9e?1Qqf2-##H2+T05Ij=nsd9OqoPoqx^MSU(7UtgnSpobA>R z&ti&$x1>#eP1O0!tD#%%v>UL_E6g~kh4vkI44a0-j zLU}hQ{XTR50$g{c!_5giqo8i}%%os~F_x&v7nx?4rK_&@)nW-RTc}_1Z}bLDG46b8 zW*v8xI2hUY#z^)*zfD&VJmx-P-JRlyWQ7lHIw?V__z<1!Z*bRmy=hnYzk$6HtMYK5 zXYT_bRGl&y4dz|iT~4~U#>H#&A5W*%xRP+`VGz(QGELZ$sa6*$(+>i}eI3~~mT5#@ z9csG}5ssc4W>V3rQ_*?#65S{1;x;i!k6J>HRt$MLIjYHd0q;#GF$}RgvfJB0+{hOV zfuGoqRF`dFrp_*k5jS|jLTMCd1L zp!&2_NL%Q5G2r(|3BM{c>(e+t{}2BUGcqAprk({oL5&=7rY$$EGQura@G~Skk>}g)W#C9!UvvQNR03T~CTtqBXZ@qb+iv0x}Z;7ZBUJf8%LW0PA#ye#QIs z8>^EJh_2HwVYk~#G-EuyBVuoeD(EcDygXxOsCnw&AL4szOCvu1c|}2<0kKo7KSy)Y zo;fYPDjTVU1y_@d7w0lB{#izBO0r~3zAaH%!-*MGUwozBa2ylQ{T!srOtZ>iumvWJ zsE)t&)MnuUWh5r|s`iE#BjpIW4bPD;lAuXYQPiqJNb;hX33ao9z=O7(DT6NqdihTm_4#~&5=oBdSC;gs(7x*OB`3r;q2+e`|44i=cvSbpJEJXk~`UODqxAxWrH&8pLS8xl2ENEZ<9eD7yVj5X6Y3t5|amls8_x z>~5kM5b9RcJ&Ie>8Q9OgnweVt^!C1RsMNj`s(a)fnH1OBd?^I5-JCr;J_*2>{zQfz zx7VBU=n2oBXK2F+1$imiEp2yN%5c%gRTnpYw)k;eLEogbsTIYLJY3kXUC3t9HM`4H z3np+&dRqzhOrE1kPfYn_g}*DV*;=I>#xSNYwU=FKWDOHXuwLR1RsjQ-obkwpTq}QG zZi7Bmpk>U23T0cxMdiG_NGmwp5m7Zz5cJl7*>OS; z>xcWTob1_tGS3+xdri+BVn$>62|TG5OqSZ}XsS`8xZc?JXJa4_x)6A|M63_xOftP5 z$@Wo|VjeiYcspAX9EMlkap=!yPdT2tcPwb1KNfz)Dwpb9_IW?#C81ZJ852D0AlFpNCrk*?-S5Osq$zYIU-#^aKl`AiJ+#Woo6(wR(-0XiDCb4&LpJzBvd$7eyChgh z=HpjiN6@15i7{BEOWq7MF>w|V)y$Q;s>KI2uJsU)3*&V6c-%Ihe$Ys7|XDDtKNk z*`G3WMATkKVeR`F{N9XsGr{(oH6k;w#P(bsZLW*W%ga$ZSFEx+<3{)rsufrFrksY* zWUDtA__n{r!MNBp}yx>B6wr zI#9iQyJAf0(5gmBErfgOd$}VJI0{%mUl9EN=n>fu+Fv%cBMY;xT@HBX_He+M z^J#zXc=Z&{e|7=$hZGNUV8s)8?tU}$0g8K8kr|}~Q>SC4d4vkM)IBB@&bsFM7ATgk zLNWCIz6d{*)SR2c15S0G;U6M5Pgx4jLI_BU>9iLp}49%=R6QdKQ zv39M(lj*Qea~X+X)F)77b`bi&;B4^uFe2aZG}O7FxvRdv&{9c5KWE>074Ml*@CpdB z8{j$j{7=0iJ2|=jjJ{^2LTJeoGF+6K%|8$)d*=v}gm0Y0#dxr-8YdSV`QY4%#jo;M z4KNb-KA&Wzl+eTDD0F%@5F#-1DiH6k!Ei-c1BmUtmJ;cMwb=lPbl6Em-M<&BNQNiN z(mMrsbBoXRTwGa=W6mRf1{6ygCm(mu1R&ewK0o}ZCj7ZPWN#6Nfo)prKhf{@>z2Z1 zsx{|KLGU_}J9jyljG@_r{k-sFATAE?uLwskIrPu2W*~rGtt3 zPtve9QwJX?ZEPtlf6I9UR?!><%HwBiMFNwNS&5nL-HIjn44#Lc#HcQv1xaUBoacV2C^mSMWxRBOS9_iQ&V(;A#WG1X5K$n=-kdtp{im}Jk#8Z!vKTtZ$N>;{K&O%-jRKXHC z!9Khxl>x)aUO}t5|DqAKi-#JRtU4{U1dkylY%-gUrLW)SGS92q=X5zBuz6okMi9x- zwA_7QsG@Ac!~`RC`D@Ia7bnJ3R@WbyDt?5?W2l~R7?yb}ebq*gOXxK?n*xls)=HmT zulBk=m3z}}182x@nnY!&1CRMuEBX)DoCdi6c^ABJi{=*WZp3=85>6j-1u*2Il$!)z z!!gilARB6$@F3XiXIdId1B5mI?_ltZ`mLF-g3V8O-@Da#CR%i|Fz$z}m@qgB{(X2x zP(YqgyY-Dw1M*?$>oB1}lwZO)r?ToHlBr%oKFbl48G%?zah*EBT7m1nuTQNQxx|aH z8ZCNt{Zc$gk+zNDwshRcaD zwC$X}E%M0WWl|b?m~eX3EJEODdfVbpI_sfE0<2?Jb)+d@K?HNR*Z)8_R?VOl+H3W` zBFNn?^RVVHJ-)N)=fWN7_v*3u z4~#fbfe!JPUQ<{((u(}ktp-_z6EJh|4uKEj8@quBdi&87mc$aXQ{320%MF-uAT==o z9K4mp$-7?8?0kgDk?+A|?;WI4aFX~&h&8=z2c~Js6?FMbrt<|Y?InI;L|=PZ8?neW zg9Mb4+M6pehdSm~6m5KaKkz*i>x2=80aU7tl#x1!(8%nF*kW_PDx1xKX5_OP#J)Kr z=p|42+~=-0WNs(4VWCWw=@V0bZ*NWE>&x!`b(P_!G!xzMA<_PXHa6vd;% zIl%eg{jOVoX|>L$HnASyl@yqUwvzR53X>Mbma&08#IoRU;77p)Vm)5nWx^OX+Yq7j zT_6n(^4x41$`lM~T6ZXs9$oTlfr@ZbBEW1ZCc3^|>1q_&TRgH}7>_*V$7+ocyCFG}X7{c4d;cQHv@!=H>a#ULTjfWYHF* zwK)6|45Dsi$L*l=TWf6*5obX=Zzc{9*Y>peYg$xgSbj|opJ8m+n?=GoU2kWuDWZRF zA6p+M3Ccu{jjI8|m-rO(D6m$lL{xXsuj}Bx8n@Po;7>~v5*Nj7Q$^8R>dMpERuCpg z5Cv28I*6{!vYhEd!tqPP%Zb9KI0d$aa^)^SSlucDw{Fblo^OobOFD!-Y+L|^i zFhHaiH?%q7>zwu4T>G7tyaZ!elWV}f@LycI`IU$D7$b^*(ia=P43@4QjZy3N+JP9w z+&C%}5>Ks%s6v#}B_1g;C**uRPx9aNo;puKrWXw9@6&g{90PL~tWkTYPi`nVs1E{i zYO;d8&5wz-rx`Xx9M|y5V-As7T3*{Kkhu~^)J1k0kNS2FCyN_B-8OpJE zA$J)8x`b(kvTjpvio5=-fCgFB@r~XCmmu=&1Mx^a)bGQ6d)rSj*TAG;vG>>{v5>It zdDSh6WQ19({k-hX+I;ij{zY#7cGfNTN&g(sQQr+uyDJwb{gr1dE;&?MVmHtrZ-$`t zFJo4;#v&AKhc~FNxd9Xr+y&i1c^Sh}&P5=6R{bV#rnQT zVRAHjovbFC#5Cq1#}fvvw|fq|4Ih*xrKWRwp# zK^ONu1<9QE{^oBtrx~w1BE2|{DFbdK!;D|IUxdiXUuXP$+vIKCq~XJv!}9aqP%B+= zomEUvfXWF8D(!0O_4cJqqwb|{kw+7MVn~L(rhKtU;I~W0f>E5e$h`))P+|#7Id-*H zuDJ)ImL!1uKt0{Gs^PeZx~GKXLpba-Z@lXq<=-MgUi0A8bK;F0rppH2@oN%K7gGMO z2c=S^KM221Cj)thzmh^<26$h2^oH`QBY7YAua)+G3rl~X-@=>mIbIg#ix+q`S^y$O-D2(CFm?cC zs-hGYxvmU8&6kwNm?=Rzsv9TeWDq4{1GQQ5*~Thb?kis~gWsDz!ovknkq>caFcaZT zH{4I`ob;@1>8++Nof1OCe~F?IBYNQ~zm%i0OAU=9;v+4ikh_vmjueXfVxP)RZ0NF| zC0@AGLTkuDZT(%do!aqvSAWOPykDqabA&$0lUmQbE+Nf7F_b0VvH(NCXLpO;3~U>0 z_0|nWj`uUQwc}nac1O{dRh>rYmw2y^ zQdZ8>IN(rc?IMs67+nADGzSPvHt+Ll!h;PxI3_ivD1)CRc%Doo{ZJip6WK?f22}K` zb2{vT!?xuX37TDNJqZw3Dh^d@HW;%wG#T=t2Jwt<(CiMd53RY-7Kt$>i}}>T9Bwta zxQZGxtk|N#GatpTsdhwATt3K)fQ;D`T=33XGW@fdX+1g1p4hWCpp!m(DRd6c z%XpQ@#9ssnLL}k3Y+rn}P}wx^Nmtlvg37hU8iRHF-qWsJKV` zrY)lt6>UAUomOt^SZGLIqB`B-Rh!EJO#bbG{e$N4md396zIS(!$QFk4uBWM3C!SCfx$=L{E#^m#XC8Zf6u zFSh;3ID0)g3z>x|sFMuDRyU!(wR8w=<+JZJ0I={3w%>~s(6e(c^Ii*Ej@*jFyC$3_ z2ZW>iKG*!nj7|_2A~v_WzPrs12+4jK=}lSwwan|ZoJ(uTu6K5A1Oiesll^$C!px}G z@i}h|OYCVPGFurhP2jyQQbLyn{KQytxb|&+V|ed1WYeDH1l+!POd!Er5SA&1VSrG( z1{R`OtnF(~4i;e}+x1U~VjH$sHy|CsXUI)1g)?Z<9=HcRMhj(|m!@qv$6FTu?fil? z4L{5fjQoBC@P8F) zwh_?e$VcV@m=!d=&ONP)%PKAuDhnTSmSg&8TP56!@v9?e`LV}&d-4#FgUC5oy+kU- zz(2E`OIXV;&IE;dgECVB2^^Ljfk-fPb#6$qATrvd_NR;1B7@%)GOV&Th7A+*7x$wB z6+NIH?1HT(0oJjk8&-ToJxyd9B^6fcPIOxZ#4Fjfnq6b9Po5z&istBiG8wYh&cVdX zUm7&#UkE1^n5K89gTvs<>t`D@@*d3tHGPWDj_ouQpRu&Fu&{{$&#fQsJ`f>18;4ja z0;?<&{$NKr!n7Ds`lO%)0oN{n6AGJ1dK=fW)GeW!IDd=Nk zuyzg55LT|)o4|*7K9K?mpmch`zE7~pQwTq2e3C3XK^2^0?{_FAYg}=|`P=AE!uaC>p z?Brsv={TS)fwE40S3mCWF;02J(Mod-68sEKt{Cz;Ogw@{0^7y$qdJ&DNIE8GAy_Iu z;zhtV_zM7_BQGG?zg;!No3O6umtk93>=kC?A{Cs+;7)ddwI2l4`n2@Q?F7xGU-9)V zy*^xxh7Z`Yk1M;5pr~j&;0w?x%(lh)q@8nCL8UV#M_lw>8*llx9+ zH4)C;nTG*=2^sf&-qUHp=+4bVOZ6kWKnj1;igMKi5Y)G)(=Jf(!bEB6QKPxtV@cyUK)fedPqbnS3;-# zC-gR7!(aj~_vDIwzeJnfhr}?u!l!V>V~*R&tT2x1T)%}6aljIa<6HX2Sj7t{N>Y(n zeGeOZhTDMB=@Qz@8P}dEw6>b&dyG*)1EIl1X*O&Dq#rqM0dc9n>!~ro{zv7VI@a%~ zzN}J_9*&tbK$7LMgVaDa(Pu^PJ~HJ;4Q z-IuOdoRkXdb(;qg#b7%6&4KjfszXM`v1NN0y_&ZIWys@Sx=X`1D)C+c#y|4+#uz&h zW=Bi1@pwGxCt(n3w6b-FtQP418WpijdvznEOG;&H_Q?T4@K;D8vTMSIXenb zQeN4u@(YY&I$@X$t{zXA4x5mlb1jxq?;781*)T>_gONqk`^gE5ZWVLu8bb?eiaukz z6bMF&!y`#c)}EV{MlTi5$|;(G_mw}=%Sv2}{9uQ8H5PHd*vu5ZQ7r|qEKN6qC@($F#>lu3zB|e%pvTmPR>ek4`9aH4YD{Sqdz@41b?2$LOZ^`(ER}_(- zuUGGpRM#_WVOi(8qe1U0M2_2x@(*bk!FAS0Q%(_`wt8xxC>(b+8B~VVG6D4-4}=GO z6D77q7HmUm;fP)Oy5~ivJVucx zR9oVX{vwCxG1|D7`xIKtJ3U)6rsui<02*-7qYPqffAItW0#Nt8G*E%O%yMReU+HXf zj3eM7v0&6rjE+G1UAkv)(%|rq683rW)0LC(;nRf2Eu&AT3wBH#q97koFFvq^A%MQH zO|n9WXA!Urq8S8Jz2>5NCeQ#X8nvjN;5N@`Am*^-mJ4-NCW$rgl+m(e>3RZZF&2$C*R*R{8~g*>i~A;o zN#^>ZfqwR8_KU(XMLdWsobgl40(;_RyG|<)mzS#!TkrJ&VpJiVpt!S*wres*4X%_% zthf%Ue4{l5`^ej;%*o0NtiH&t?DwvEPj>ST5|d%f+LH3O*h<-iMB|8m-ynK&eveq3 z8|e(fmVRkGV0|}ERnT_C;C4rX8pFTf@p&CFUVB^dC|`&LuKB7M)?prDcoqB7gsr9D zziy3ukN()AhFB?%kp-;+{ev>(?hI6)oJC!%$c}M)$pwfv3pcb{RyBKyovuqp;RiI% z{NhAp-0)|3%nEw6TcR}e_!kr5nIKu0l(g@>adzk&(q-#co4p!t4 zdC*6aZAN2zLn9`_8!0CkIYUPR_l;%Ot*;3@oSR*{CVR*A0&|L|u7$OM7>@+Ak|q zK?#6DFZk>RNd?KydoN%znah$yg&PLR*hg%t>=01cvwJC3n!t<-(XzZX!r;}VN2e(@ z4>OQh&KMb)kJIygxQQ~VXaRxvh6G*CG2S)n=ZJLee(5p6_5&fQPX3uBNtsgGGuTfw z4}R;?9WNXyn;*^Nr|pVRpCh72<_@?G6Ww1W0lTiu-!~^x9Fp`PPwq{}^wwbV)%I8= z-6N9_wlTS%6E*`+e&h%W^dw(iaI_YYhkOGqyUUicV`0bn!E{bE^X1sOzgZuxDT2GB z7Oo){-}TsM39Ynv7WULctH5wQ!V{4Ix*DF0sw?j&nmxJAhR=O)TCFs=_}EI`!S9sR zm~MR2Sg`Fy+AfM}Zeg~z(?uz~UcCtvCZaKbtkaf)3nO60wsW(S*=;L)!cd#n9HgBr zM2NYc1KNC&rh1Wi$+eoB(il!Bw6~EkYL2vik|0eTH9>p@>`gKOXt`Y}yBBF@w|3je zJXZ;u#&ilK4B9tMhBaIHSYp()yKMlR#=CU@&==49t9AneAvk?=+1$DV6Ul^vz-{jkTo}7;q)o~b3xFr>NeGCsRlsv!6hxE zx!dpj=K6);XQU!Ra&`8@|%m~#BH+nhUi8SVWR~RF$=K%X^}uIY|j)? zC$Nvz6Aul7<@FnWNCG>5SHgBT!C`1#qrF-pZ0|P%Uo_ka>NL$bEP;SL{pjWvymPs5 zk3Q%R;{Jj$WOjP}(MBIM%%>~()KPx5RY=ZYcCnNDiSVYAc)x!o6rtDfLJ9xl4L$jY z$Es?-xk6Cz*LPQ<9Bp(9MYfgSAmF&@%GfHUVclgdJ4O)cdBT9cboN$mt&t;By z#Pba==9+xG7Fk&>5_*dREa^tz74k?@eF4BOuV2F4C8!FSE}2oMFDxQeTPXDyV;UGM z7y2aK4#$N^_E|v;5yZqm>=sa|HqUEW-kRCdMpe@u1-E7c6CVR|1el>wSO?2s_;d%t zP_=w~`|RzPn7m|oSKQN`Bi(|lhS?`_$$zAEq7nYQP`-7WyYC?XBWu-xb*P*yEQ}VO z0E~U%(IqALdE|vAIxfO+Kbh|lX8n48@Cz#cd?`(GNwb%n-rhV!0z^o0FT7=Vl)uv} z#fUuGO^Lwy1(gYXkgh#&S%60&9|&`=D>)21BJ1-RhfuJz+1I-(>CUwT89QA(J%E?N zOp%>Y4-do<0wL_FkV?eKkfy(7`Zl=U-)bGr*5qFX{B_kbWt1HH(gw~mB1Tjyh%w4? zyWNUHYtI&L{1n#5Nk7w9)}~F{F_D8SJVB+Z_CR{I?m1#D3WK30CUXV+pz$PP43U4M zxZ3*CZbiH7N4x82{V* zpDTp-0woI}>Gm%xnc3e0pz@M!s>)2EDFN0f=@>rpa)@OKp1Evz?K{Ke|3ltkhRCSU zGKqpoS`ym7I6VomKP~<-KahglH&Gyz!w{C&jpW9or`d)56s0Pksv1%f7SI(B>Lp8k zP`$#t8ww4ya|S7j@1@%(Ah>p#UZH_{qk!o<*%sH;#Uo-U(H2+3X$Bn41{~4DfS+@3 zdu_7@5<2_I1DKLB?&x^c`nX)tvDP{W%%?-{X0~t<0(tLaXiK+kr$qQFmL6cL!{;q@ zpU`s&HAm=mWT27+qEF*H{eCpu{(Wo*nJl+Wre9$%#+YG~v<*-pTy91Or-!#W4I5#- zq5|T_n4aP99O;)S)~qOz#bw$ECGC!CF;LS#X%`Yv(h54^g^2PUL1D$g8DJpb&!VJE zk7rFWuojiQnKDBmwP1N$2q8O2hX9K%Yk1s@p3^ybL7P)DfuozYJ^Fuq=MRWMU)V#J zK&79O7?hD(6RKR>f*oq^b}G3v*?qGovm&?jd7k`K#4oee6(!L^j02!_gVGld4ap_WSYi0|OTF#)ypXKFz9dcg~u_{@)3A2 zORjYYXTHh{_?t){nv9^+#ySu+E{Px(P6uxNO7p}W00YP){1>JV7ZMZ0%@{8;f&g6P zQX{uP=J==Xoc|OAV0eJjUm+9fdp1jq(U0vR`9frD4bXfR)YP?x>P$5klZD(VP^ zO5{*KCsw0MfyJZIcex!$P4iH*i@pc>fi&|K%q1Te)%!T!2p_t@q6NyotzNY!V=}i| zB|EIW<`u_JZ%t#DrxWTv=Kq_c=NR+#i|e*%R*{e5^Z!?D>H_}g;zLp@DLsvXQja8P zO9upo@?1x^gJb~A{W|a!gN-YK;qo!WH}y@>z8X;A%7*3)NzXhd-a%J&xoJ4kmBDB5Lu+slf+HuY&F@sPI zKDGBCT?DY7lhZ)oQhuUoxHJMW6<_(cM%j%tWKVvLVi}fPqz%pmcm|>UyjzOSUVP6CbJt>9E!acsA zm-J36$T5R%_KBSQy;95)zkTB7fg+VYjCrWXz$&&oOw3#A2rH@;v@Z0R^*98z4)87h?UM%!9_; zF~Oisafp{x=7&I@dfcJMfOgy2`cz`q50iqevT?SuBrZT^X zMfC}i_F%7{^~P6t;Hjolf#tqqHc~ewd6SlttLKmuoD3-g9Hb$mEt*-CV=F;bnFiit zgbtIZIBxpiZV|wbi}8C8Kw1+YY90|F{}aKT6~!>;3Nv2$Y2JyiI6obj)=axlMo|(dgD8%G_jYjhM;yEpIgUZsXk}AVF(%#*mgKL2Tqb zA(lFYg?98O5q<*P!l3Qa%S5OWj}L!r&??A4Ec8bbglT2TGe_R&E_e)o^e=%FzQ}I}7C{sKWeoHNF);4> z0#CO$Ks1-X0G@b0FAoE`8+wZ9ygGUefvA~Ck1{zM3XvsH+egN6EROGa5>p_ghj<}A zg}}Ovk8Zv=HKJk*%jaeiaju3U!LQhsaA_72t?mrW8k`IuTH+=(B3UDPT@6 zqgy^~iEa*~d?ly9g}ll@D3T;Zp%N{COFBR*9!r?ZW}I7nhwHpA8)P2xKE<#5ljacn zF&7Ktl9LD-qYf*Np&l$+0ALxSiVLMwKr2P5WlL8*VF`%SJ9_HROK)YM*M5%0n^pc5 zO8YkfV;0b&Ayc9&nZS{g_Y3Q@YiD^Sx%El94uv0DxOK_r2LGfA*MjV>BzlfH0>8HL z9YB`;5D3Fi&56MWcV!m2Cab`ok$-%~T2AXat?j4oVP{=e{Wt^l+UJJz`&wQm9x} z`~e}PH#gCDsuW(nGBRQu@TYyo0D@6@{1rf}5eBylRxop)D-g7?+!e$Ys7Ouh@1+XR z^blzN5&=_faIqMC`Pjv*A5CF7EC~pS5oh}AtPRR8#hOm*&9b5XA1i%OO>BZYN?)Ma zh<8)B-+ds5(C*9iwZ}H_K_yWD!tR`#Kt#SIJEMP0o!dKjrt-NMV+d)T2IyLru$k)+ zr)bISl>LiL;1{usd)9;aNBCZ?j++LD`pyw|-Prd{e?E5HK%!cQl-cd$xD$j`Hs8e3 zd}o(rs^&1H4;fq~)|oF?wrKuFx?BzO_B-ooA0mE{7Ym05*d5tp3i5OodNzzpao%)o zByZMVT({pQ#8CoNRV4{`K4RK-9RK7x*eW9c;eXi~PN*9_La%?+%py>AHfo~#eZv(~ zSt+VcX%ev#QeVwiHf1n~CXo5WtfXCWkY+!gnG_miVtaY*y{a%Ax~#J~)7*D{;7Fwg z_AjiBkyaKZR|0!cgrM<9KFvnu6r{he2YcT=Oo_Gf%@5a_l`fR#JT@0GOl9}Da}F~c z9LTYlPFYC13RRnW3NXCCdmL)9G64usYwmzdL5CQa4VRPgz8*}|Aa%gq)H$Zoy@_n0 zmQ#+A9_BknqO-glwJ^u1p6uWT>9!Nta7fBI{DB?ne}VvQCT9n(R{3Fw=?taxOP)$` z4P=%%;acLScQa=cnr^A#5H|-9uOn*U`vkKx z29&Z}wcNhA|GnGWlk;t)voPhV;p~)J`7o5;n~oRnm>i{q!}*4u@5M0f!I-`%Bx%~^ z8$t$Sh*iQ@K{X>EihGu1skMpMAc%%1ksWX$%wrq3!1K7i0E4n|4m8wveFEgHi8Wtk z=OvF`Jy-Rl60Dv}K7bqnwNDG3qOD|sk-DOhU6^TWfy^4;9yJlkx=KVny61#)n9@c_ z@AY-hrBe!I3a{*lzBOza00FdFmWm!FGm9uX@$Gg>)~Cz@@pklxnR|yLW>zYCs~&2Bn5+^t=!7k~3b)O+ zy@wDE3`Wv)KH*#JsXj7sp<0WbFCOip3k6O7ZP0_K-UCHyZ+IGTd=VxHu0vb& zh+_5M+_RzYJaOLKN81(1{4D04i@4q~RnsQT^`G#iCYLJ$Kb9Eg2o^-JU3IaNq5l&+ z%yqg&)cL4aa01`ibDDt+z`5uy=iFWiIyu0e>u0Wnd83@if|*{V@Mc(h$m) zD`|MxEW#(KOtnfr=L}oRKhC*ld(vS9PQH|ouqqZy)flLtp|sIGfUBfcNtOKQ^tl=adPK6a#p+5tx#}B^jl*r~+Phmd`H*SKh%!Rb7y+c7Fu) zLNriC5_Sr3R_RPi2F;WHt&1U|WW)ka#m}rIx=yZjrV2Bm*-K?4CKi%tJ7> zpld#7+=cuLz?m-0V~ZN%tfb2RRSVuq$3nOKoX1|{eEC%Y%CBGPJg3sbdA^_+}+>`i7s;;q)ez z9P2==W(qYwg%OfGXcHo#IbFZH4x0Fn>vcN}_Pk=JoJE*J&t#5T0H8{T3Zi7rjZEKh zY|)p`C}}B^9`St^4RGq$MG$3*IOS=;sD@0&rE6A(g=C$Ioa| zq_8l)>89Et7o%g*QxoqJV=u0YdGJiVtX@$VNL^rQmW5G99^2b)Ro-;oJxG7duA z{NqKP7$3bkrj-hbz=$x}REcA*!U-3sjZrcX-$SC^3bxLrq*Px6OO(_ze?m>-^h{I+ zXkqHXTH+O6J=@0EVV z0xc~dB%ndTLKKeZ<7-CEoeP^W4qM0X6$3R8`SFk5VMAIQk5HUy?&JhYsBR#?h%5?0 zMe^M{-G?8CGOzkM!1_ilpH+?eEitu;f}4Xf-|SF1UPV{F0pNf$+`(ChdBjN|d18m- zrzdywEggLPrQS@a)e0LM@WcC8%$9GR3qAV4QC|I1HM7$O{+_IX3drsvUC%rF-j}=s zr!eFHa}8+@Xu715;)JX2_(n~b^xF=SuGGCF{q@n}Pi^*Ll!=q{=ab^!cl|*O#aYDd zm(rfzlk_DrPBd%;_s2B+f{XNfiI#g%DxMpFD=}!;_+rj!3Md|bz=GKBGzYaqsvegb zvnAuQV&o=lZ(Vw1bh97|kEY}dH?@_-j?rK?GZWJteW3oRo{bpvOqrn1Lu*0zbrYJr zn2Qm;C`xe5M)*(JZhh!iT1*#W=DgWa>cm4w9wsU3IzTY~O}26pDNN|OnhbrY2=N!v1r)xh^Yo<{!k0HW!*Oz zu<|t!m-4hFGjZADUhi(JkQ-3S|J)i?4^lK&E!}!1xBj>cq*JH(7XbksTs)E1z{6zY z%r;C%8T#m%mri@^+A@i`*tMur-snws;MqvL6KDslvXe}BMC2~5xa_F895nI4m}r@H z2#!Pq6bfCR`14hwU34}w_W<;2Do^huJDDAxKi)E|)*oK%&v7X3b56h;S`FWn*WyqP z9YCB%EEJK@<0g;g${wJu9GSbfkpWEd^qqk6? zQQQn(jgB9ZUH^wVAPXa4VJM*Ob8?0S|MI3G^aL+JX~_&*E48f?ZWlF?e6*BP=MVnS1w) zIvD9PV-tihQ%Ps=nl0E|uEV2{&53R8?3f4*XCaK~un0wAz*_6R*uneBX&Sj&a)>=y zq{|-d1)0rOT*Y=I^%%Bx=2zG%Yo7)SP5ajp08W5?MDn*%+$@}n2F?V&9Z8>3GK>r-AZ?7Lt2D>e6cm>4f>&$QV9pwV?s8%2l(175b-yFICyxiT{ zUk9Pkvt*nN{|p-eew)0VL*1BXsWcGM$vp+pqaAF1sfhPEpYKA74)=UIIkp=VV)}SQ z+^C~e;YB?V&Jb%xq20aYc}Wl=yX5)gaend5|LQF)ZvOt=Ld3N2vJ99{H|BW|#?ZZm zkwk-4Me&uT`R#b@6|L)+cD==;CfD{B|L#RwI%v+W2;HyF8ff4fB7B4{S!2JmW!1wx zRYb$1Jj=a*4~Q*_3_Ho4KEmLrq}LNB+T7Y z>NmRZbyC~Nu1k%jMCvZWKXWc)d8Ad-=Tw7zJ6gV`lJQ zVr!qT6ET;Kd~{>V?es4g)Ffg`4IF!yImi+QJmqnx>E}ueV+w6ei*PC;l{$8Cji`)% z_kHgZXGdGQ5k=ZiVH2d82@X|riX#pl2J+j910yd6%)a8jrnYeY`6)UdF=gWom1H~V z@rYotetkH4)yj5+P8Dv=)0G)hj{4+>K>`F~jcPjpf&lT$1Wta+)o?x%W2CmofVQWJ z_zYrBU_INPs3P&~BT^6B^Leiagh;KZbmy=(u5m2 zNRULlmBGaR4wfbB_%-gbdYWqXFFU(9 zI&bL$lB=*>6_S!?uB2d2}*^n0Eg z$CX|?Ue5nSB&}b-K>d0pEFXq?nzu2TQjV?q=4H$ z5LL{99#if@oA)*e-q6}P>cqP$NAGn9QRh~s>YvemonAEhD#5sForeK|_ zGu>T4@@5VMJVzA#gVtHT3fYGM!wMD_XxsK!Z*u-;&52be-iJMIShwe62JhKvRv7 zzrVb{yaz_a2hC#F$V1EE2|p-Km;et`S9M?!x1jYiAR37LBo0&8LT^ccnQGl6P5fkA zOTH#kdM_s?kD_x;el`tPeZ13$F+o5c0nJ9n?oOupk8Lm~e{A5`%E%*GQN8GA7~&CL zI#>(AP@-~f*4bI%{e1uSa(`<9PtRKpWnqD!;IAYX6ElSMz&qxz*Sys3xHbFAP{hB= z1twdH50m)UbB(x)GfovMMtJl9zpw`=fzET{|kdc!21E;4^I`(%L%WTlX$(jQ=QG6 zxT*0Tj|`Mk3i2#I787uTm~5!OHM;J!lFB9_P2$I7>UudWN;~EgtCTiYV7HSVe)868 z)P}%`HGgg0vXR`bFcPAz0q5)50n=$$i}DJtYo|o!$OvF&i;)__IULsC$A8v zBKwY}3FRV&HyFKF!LBO9!AjGW} zW*Sg?as0c2QNw~ieT#cx?z_X(RV3P{A5#Sa0_Yf$AMzzo>Ys|e|8{G~4hH>RJbF=l zy2mB`v0_}4OB(RcMz0R^(U0=EI~E79}VrJY{H1IR7y!?KD&a)dn5x>7_tB z(g_{g`%p3dbzmruE#*it8&?iMFx=%|>K%&E8tvhi1B6H%F_|97^VrCPUNmdq6YSWa zKZ$%#+&m`#>)8+JpW7*>42$wOn2R$MHrjB_3Fg47OW)Mt+1WM{ z3oXdo_L`u*gjZTg+HaNsNTN)4W8O-c?4c`kBUSAZG79M{7ULd!391wS8yw5?{D=to zuvRk7-7gsntj06I{mo<$eNlvv%S#EV`h26^jITMt4>X}KO~Y(fTb|+ucu=^)FhF`N zarh{IopbyKe|U!s7z+&=?SM#JptXCTwQ2tq7vNX+2)B8yZBXo>^K%Mxv~|^bhU4g~ zBSv3;XV}ufh=_IBxYV{OR8bdR?6a?d>2=l^MVCD9@=jyuxJ8U|c{oP62O`3?M|d!5 z36-uFX3vVV^(ed;f%?Rm^n(We`dGZ$K9hZf7JMird?Zwk;+I|iNMCV`ohD|){q-}% zoQCy1Zjl`YfZze(q9R0sE=E=h-*@}L1YjX}`YsTiD-mXcZ0DlXC^ctfm_ z@+GS#%XIOpSYJt2=Sc&NCQ_v~^?q$60Y{g?dNSQW96P@F3nN!QwPU$71BP$nlu^A{ zvZO##L4-AVdw~1RtybYM!crks3kK`D&u|y1#`#Zv0Vb>$WGIvbslVl{ zSF$$glc*abDzVAU`_v1)zwoHoQ6Zd|11JiXYRaOK@;XkG?L$Phz?jtPyXmix^+J_^~S$~6| zYa;yf0<<1L;mU11R*Z~Py0ruf6kU&nNb0T?O?vYiD($tA^tpw}S1%cNe974Y()~?} zwtd#1xX`qhb^RsOB5zWc6(lLyahuLAvCE~tFPCXT2#7NZ2%$`(XRk?QF z&Yjd*c4IShb)7cBRF*C|oyvUz(MB7t36U@(O^UpdtYQIuttEODe|36t@5Xv|e^*=o zTMNc8dUv8tep8L}g9;U$Fj55QEI@wScdj{#R~^98KmK*GCBk#T)U0&JOYrE?x<{c~OcM(d8$W@I!f@{aOrU752;keTU_UJ3N|$L>a*))K)U5>K?pC#LG!jZ4Ltgu-E{0zB1X8n=kvb!i8;^>XvAI$ZRM zZ$+GCzP+NP$IDz#KHhHK{<{_0w0+E}x{1z|4_rZXxRc&#zqO1&7;iMW< zo0nWXg+=79W;25msZ_9KjytjFD;*jf?-m25kJ6vk&e+ zWMFP;`%h^Rrpc7}pM#KLykM5%Gb(c>`tWCI z8a;q43f*V^8)oH7t??m;2d$GobTySMwz?rMu=YMn@g^G+OjM@0Pes#CmUD4!8I>fI@?n7iBifcEJACq#!P{V+0kQp~ z*(l>mSoE{uOg2d~jOyT&4w0DrwU{1)^Ko1oGv5fPh716Q>j>#CmdijEM9=W3lw}g-IfKQF?_-oRXU#<5bKZeVb+vMax~IX7e5( zxF--PrfVn25Rs9tsZu#CZ8t#TjqogG_0hGiTjscqb3;uCUwgO;I)WK+R6DD_BTOC{ z!x=IDNsa_cYaY}6Qw zSj+{{*yBd@i^?qK7CXpN1RSB4=n_we0pvFPDjFOI3a1mT0yl>m&n&UG{v}7vy|O*e zC&^>EaV}9JoV~0Y|)f0Hn;0+rsTuprVk(--yL^iwZ zq=bTgr{cj(!({_7m8mrbsxZG!=4Ga5Esv|q<`4G)>%&P?QOrD>{&+&SzDjX48W6Xp-xj1e&+135q;N2>G2tm6MvIp{) zvgE8!fNwM`8I3>jiURQNNE~%z+uxvZG`CL85;1Me@(qvJ($CKr0rySnyeS{NzP1WLTYH_Z#qZ` z=HOLbBpbK6SceTB)I)-K6G{jN*jV=4=CN-=!@YM|C4W`0_FVWgYMn~Aov<2lJs~Xv z6q8C?&&BRLd6PV#Q`@(2soJAxfu`z^Jl?MJVWJA_$CDC=+x-XqT{d~RX&G*QvpXce zSwH{smmu-46C^ZrhIB3gm3asrdcYxt*p&sEyA*P0iCiuyc{X9!BJpS&!eln;F608u zzwxq&>>c}r{9Mz-+0LLkU08xd6F^79%+olxv4E9Q;jI|PjG}qOzhqj(V!hrhdu%4C z&I{Ca_}8YJVEYW@K2(oj5rQ)ZE_QIGZtZJE;EZH0UF6T^Bg+tQ9Q*5g=&U}a{-(IZ)%J$A z7(GcULK{+i@p$69$nZVJeI>&+M*)Eir9O&3y0C{VsIGO_F8f6ugQt~_%j&S9d`qx0 zgvI7TtM||m!jA!#gQ7&Y0q2_r%5XzsU}-(@7{!pMmMz)|0jH@|kOd#f*RgjEKDE@) z&8h2)$VDjIB&O|6fINKymuy5DbR^2|I}*~`Mz5aG$_-OfN!GTU*WvcdH5OW83w4g^ z#b=sPXa~uv(BlE&&F9BZL4a_pC30T*>%9`}V*@Zj7qlUwNbg;+k9)@F9)sRBMz(9fZccK0v2wY`W8IIvh-X z$r?$TiC;p@m^fT*bq)O-zP|>3RCd6iI;e>>Yg@X#Wj~cviB`u{%pE?syvCNo)zOuQ+C;^(OdKwzattH)UIOZS4TMmk6Aj8fpb#+ z--N--uwCfozDOvKSmg03m-do5^#S%TPR~W=uE>U!LRrrq!(0&Z$j{{uF9UEz(>y4! zn3clL9!J6=SRe%h{(Rp!cXSe{bKRI|iI>^txV7K3uY%lvSOa}Dhu9K*(-Hs=xZ(yiDD@qgrcraa6yzo9XN?KhZKJy z76(>}9mEKxb>FYe$sj7yBp&_sXri1-$>o})Hhn*GIy;TBi_{;MeKxzs^L+lkZa%9x zNgl;kBp=~R=3%1(TjcI?T|JUQ2nP+XE`|tR0%9`km&b=G`PT6D=tLvWIC@lk8$KCs zP$IaZVHooE9tRpV9bO(%?|6~!tB2>5Rv)z;Uz`UQP$Z-YaIi)%SE<5Sg3sW3tz^b+W?KWwUEGTk*5qAK>tXR#eK%8P)l$>W~JI^0w*$ORp zb?HX6KczQ}8{D+`?z2x6@%mWleKdJoJ0Zqod8)b;n0?au*oXe4Tz+ZO8F{ITUze6T zxJ%uamR4yf@Qo3A3b%38Mx!d6I8C@WV~WMD*tO{N%Oe(Su4CCAblob@!a}v|F)MB^ z!msjhddPl?J?nYnFS7yVHXI1u#}?nczR;+8W%_j&+hwKf!ITD-oJyggk?c!HswFf{i=rvV&?@Y)#0!;aRhg#c(NlqNg% zA)hXCmkfX$MvhRI#HE&yMo8KTLN%Jr)ErICR-4qlC>G#i6+^KP+qdxUHr-SLBuzH za`Xy>1!b8%os2U*WIUCD$vZ!)X>2kRTijhEGvt;o&A?^0bJA9&@HOjG&!n3~J8qQ$ zt5H7LO<6kww>`|Y{R64mVcQO;r97KwPhyu^7p`M-$SnyL&7-HHO8o77Jqx{KQj9i z#`-hR(9zRq{06QkbD_kGPs})VJQUYtSJDqwEY$Bl5(}`vs-s_de2rnFAF!qk00QMS zTFiG7Fub0QRCJnC@Nm##Le>CJ<2~hEU9_~htaixQPl1%3U5-)ZeBe!Y(eJ%9UM!*@ zs6L3L_nsGc})` zRhj>$n&$cZV?SboveWfj%};MOjn0tUZ%DPU3+o7I9y&Rok!*52S0W>RDb7lMIaG4_ ze`j=`&xR3iJQpImCYpU)WqU>#wfh!(txB zMt<>d#3~?I~!J#yuO6zcyiZ1PUEH{P@LU3?Kh{Z`gA{(pH77fu*}A-HxRC^ zq;hR|qCBV&pZR4#G`g3?h9xnH)DCF87}gP?$Q%f zB2Dxp9GiH!8EQbMc8(`fO21OO$iCGC>I8{@t1w$RD~UtM$83?qflZk+;M-n8b!%LD zyBQcZElUgvWyxEkg#k>Eev4zIjEFW|W9zT@V?xyqs0IX#{9ad01{imV`CITnvB)Ga z^#JZj6)6fv$-is_4mg0;0{AYRBh5M_&n4ptFmQ9cPwoB1yo{U*=MI@gyxc=~N-^y& zZK#bM_YF`b3%5qqxwg7c0Et2=cNbgNxu4=dC7WL_Ka(Fdz%o007#T93avw1)ozQ{; zkp^lW0Z13rcjl$_&`Q75L;fvuwGwSVs@c{}=3dNcBdr4ptwb#>OzMJzs(euLjRs`r z4pCJJJuqaIk+|eoOVwvlG^%XmicPehunLxeu>cH6nQ71CLs*?sOz0u8jvc6I0k|(TZ>1@ue~UO|z}3FYLfPOk zU3d!E3pZKhL-)$YIrII$FEcvKx4Z5PgQi&A#l=x;g0&1Lm5re!QkzpVolAHACF4(i zoH4<(so5}jvN*oR=+O%&PV8&?jlnVUa1B~w*U{;3*Ns~93e}d5BhAfFgMnYpf-k(f zC%L}kfum|g@*G6TxHg@yEwYE|%#5DUB5j(N$?%{5KAC7-#kx$S>+CNF+7-!V(Wx7bf8ZfDa(TU6=8cc*@@M(YT@joj=$osFS5rFfh>gjunXT@QDHJd(74 zNzhHcX}t;y*ycal#VG?^B?XKh%a}cgm)tanmWK(7w^04&c zop0yWZXj%i|9_b{CnHqnxbtpmN#OeJ_`c$eK*FY}pYTtGa1x+yKiO5yl^baj1l=ah zf}7jt7BMpTs)+2~PZujNqNSRnlQY9|Y9VY48kx7+OSO>iaDgfT5H3zrH#R&zeF}#@ zI6OAfdMz>mHnR!NVungw6@f1!Wf9hqc}R~yNnQ$$_lLL|oYn}eB=T=5(i1VO%k(oT z%}f%_(FjFoF>FKcz$Amm%Qpf2RhYBzbtQt$2(mX~2MrjAZ|dVAY(tsNs}q&EfBGJ$ zHMa+38T%`eeXhI+Z1e|lJsz!COu`t9ulVWW-_R;g5KP*rFHM<6UX=qVyck;~!Y(E; zOL|b0{Eio<3+0SfJx-n*7V8i9hvbyuc7JM~7TWr^(9F##I_j8Byj$E1$}8sfwU)DJ ze8cvV>?>{l?WOrSC6YVvBTyX@I6alERrnA_qKO3>YH>oyGn9`MY$)H8opVj{PQH3O z?_JZvfU6K!y~bdqwIxd?VU3XBU98!xlRT3!6{-{Fs<)t~1~F_dXc zc;p<4Ak0@@c!x#7$TabAhuuOT_lEwkmKeiN3Yn9 z#ApPi(bz8+p_j&P;_9U^3}ik5DMP5w9tzi1=l5O=CmL_u`P6XLw)*yW67TETuT<$y zoEn$m`#0_+|A+^$SOTa*0u~7gT@22=JlCMe*zlRIiSI6|ja6~d8b+!K@a$W6t@68I zkcWD^srkaYU#Ol`g0)2R^uLw6=AkYVKw4qoIh0N#YhCv;$X*2dQ16ss-`R1Wh8i2A zJc}cKp(?5-K~OXn4C&kN!v~;~H4SoSJ%xR%%&>$yOnbvrwx`Z@51eHV3_1cgaULT{ zBn^8kqk#%S!_Q)Ogaw*hA)J!z4*qJcdIGU!RTZ}c81D2ohG#(@fX_HnWxzeSAczCX z{Cx!7Z+Sp$XE_KHGx7Ow#)%R%`IS#Eh&~hRG!{a$CXtNT0q5GhH;h>=CZ3sutp;{- zaY=j8q>d@&oVR-qStM=*+dh8HGOK%(b{w8$^tZ2lf4}KcgWC$iC-T?3|BM;M>9K)4 za2V)sEh`zXMfyG&Rdb+2PAa%IeVhycgrRV6YswPhCE?XYYLC?F4X@;jR{FGZF^Kl= z%)9nIt#xML_O!Z7abnYT{dxuQ=Oj%zm#gF^Pw3+nXr?>;_7lF21VC;xR_i21+AZrv z=-Od687@E5B~9kB8pkV0)l@O*a_WdeIIpmDpRa=tV*ottZVy2_Rk2e*tm{9&B@Y`F z)8ONuPC=qOFhvxq=X=Qx>4jS=4#WzYikVw>Addjc_mmmDT5Gex&ho9 zc8MzwyP^r2I3N?vImo5wIBT#)5qSeJ_ZglR_uO7-H~=tWeGD*|5{E@q-bLFdQ)?UP z+8c?V#J)pF#`vu&QhWY}89l9vG=ZtzU2YdXlitls&Y=9m+x&Y4Zda^yhMj~t(HArS z<)8DY=IxsYlvr9o5FBlGkymO`S}NNo(9qL6ZjkX_#w(Glis1tx<7tEt-ny48K$4Pb z{x-CiZS=cS%a8=w^>moAsQe`vYwziY#p}0;Yy*D9mob62m}2+(YerawS}@amCA}Ng zY_@89E1W2Evt*}Vzo@cSGTGp*;VUbXs5kX66cEqtGuIq$$nE?&oVF6pyW{sR^}FrJ zI!+`K6NgVBIeC_dG}<{HCuA#W4&M)louo*{pV8&M4@ zNj+k5y`97w&XWdI<*%t72z3m8%$Gk7mmCw1=-GGvrZY;Q_L0H~F^Xo_gy$Z{=Q8s1 zqN4IzWZ_Kwdx6M8U(duyl>Wbtpb1w}+7I&B<=cHhL6?U;_)$bMi%Bxsew>k$4ihhh z!!-4?hfHn&f;1e+e*M0ofT1R%UoMAc8tI4~SDL5bUSJBmJSU^H`?JWIk8=qV6|yH@ z-5?bw*)L-)0E-Fk%-F5R8ENyH4T3s=jT}PT65fb8Ky0N27zqH!K0~s;qh;2z12F8w zDxp_o9&+qB?#hLh?E<|w&v7dQ}QyjE^bllvS#nD3XkYy3E4(TVIGz%dTj z1q~zx%3Ij0%S`XUo&Jns{+v5E7}vHoo+)g$)ub? z@474RBH(df*+c>ns}?J=Id2VmRBU84KM#>WU-y`*c;w}fh2we3wT6g_;Z;H60HSY@ zApr4^80%cDq7^$U1I7P{7qj=MD;H`a9Sm<%L(2<3Jiui!3WP`T?{fxw>ws`Jf=blM zt>QT^K!EAAn>py+*hFF_5@kk?Hek<6EnfAcOYL}U)WbMkTnz6I2l;mdmw3&`xu8ni z=(#R5@233sM|C>_WJvmnsOu!X1*U0?@ccx+`ReRcgX3kZEI}%LF zF);Y7!hL-{0svBoOac4XKRLba4<755dbP)ToR_Q_;I&($@^z|&fVzx95Zz~=C(u>% zg=@~_2(&oBTl8A8IPPO5aukR#1;Of43pZ^I>#HT&&p z(+l*ryS3JY*e(EXUH2#{20qkc%x_rmWtnIys;o|Vges7u%@)d{w47Q7{xh>hKHKa0 z41w%|1-}7Ff$BI=^YrM*B-(2l=))8$wiN>d4{E;M!dYJJ7;Cv#Km2qM0GRzEV$rja zO+`zPhiKaOo%|MFnF?jtj(DEfyY@Wj?O1R9J$(5$OH|tL)dsB8JYGzB5l^T(elyRhLo_N8gj|dR~89R-nLO3RLvXi0xj5h)I-MT*nr62 z_JH72C%59t4m^;tl@#;hE;GpjX*+WSHLq%DiT_AWksP)-Pr&TJQ*I0QVUl z5=JVP$H1;(&jo#InU+BqURZSQl63ysk&FXj2LNi~&TUMHnFy1x)dys#5Ecu%u%I(^ z{MFYPs|2AR1BrHUzQmax`LXOvGDP%20syCW==Lkpdy8_>e?~OVt4Ggy8{S!&PHPN= zOY5>wJjyWEm-Il|>F(dz^8vaLZp@sQ%>40N@sf`3=~oq>`~}l5>VLPB3kbsQRcV^{Bn$GYFVed)F9=uj?%w7S;@6(qy(@L2wkl4I-c-8#E}b zd=Ew5Gxp=e?K$pQQf%g2Q9vj~ZkqZMLVJ}N(I_|g1TE$)?ubm>QO@}nt`v5;Nm(O&}Rn?EajskU z%7m9uUyw}Wj^h@yNx%edUa20?VO6$vbwYHyta(|DJ7xtFm{Hrq0<^^&7#fa3tYjIt zC-$mU>9&K+TfIx!2V9?q0m2MTxhF(fO$u=mo|unzHK=UuoLmZLCjDU)U!tly>Df8J z19hN9>kZR4)QLIGy|_hA+mp0V&dG#EzdNZVCo6U%m{rG8yUq_c-L)wAd)NjEm;iCb z9c96u)yumhJ1weWXV}*F)BA2Nmy7bi z5Auh&tzAX*4E&J~1xbxqjCfXvJ_JMD<(G6*WnKOlNRLJGgRMRtb1&B-6s+VZKYTjG zJ?s|R)7(;YPk_puZ^E z1&|$tJ`=X1{A0`#;#4gdbNpgVle#Y_YoZLCZ+jHjMq`2Y=YZtsd#jjAlq4`3^pu~I zcgSio1Rpmm8C&3rodyUG)^PcB1gVk_4vxwvMf5-PLT_y0q7|S#A)Y+#Rnjh3+9W4& zKc!87zwECqU<@KeF32b)EPbO08GH0+s)hh_F5pQRe8U&lK<$N~@r~YoChNFJI)+V& zcDlRbcQ{2B?)8Ikt_rTY?z)rUst@MiF3An^A&Lj%pgVfoR0?wLc{qu2*c!Ak-1WjO z2JP(=`!d`j>C2YlNy)=^tAxZ35f}|thbzdZ2vB5LjMs!B*Rp&`04ZJ6osF~S z+XM`iF2vzR1)gi}4WL=Yp}XW>q!=Yb^7NDM`I?amfRME;?w z>)WKU*bx*gleV8eJIHW3p|QnpodaVI_}kIP;j()^|1E-a9##c4TjSwk`9$|Z<9y_B zPyNy02lAuFqz1)wZ+Cdzy|+9)2Jx|abs$&SXUT5Zpc@~n@J|Smtz8)Jvv|S=G$ChT zC_^=MZYuZ2&(=o*uAFaSI1!;(yTBzGZs3qqEQ&i@J_`NYz0UzzG2`>86lvO3F7OEJ1AuMT-G)`z6+Rqi5;F3{M#VhZLAtk zQIXJVK!>&kLAE`U1^RTupUtyh*US)!=$HO0a&d7o^ndxDmztqW*P+Dd# z8!)TmZr*-z^aHAx$&j0%h8Y5V&nf(Uec^S*B44_$MsZMC-$k*pUSPi@>IEB8+qhDL zRr|vV?0OfOZxKA@+Wf_-T7(Z474{-ck2kK`{D3=mWv}`Fg9YZb9eY{UtC9<2J7sxp z5e#yEhXE|q84}(X2$1klUQ+t|TMNt+{#yTTv~d>9nWAOaFZ?+IN}kpq$rRD73v^&F zf~i|);TCb@LMx0S0$wJXo8B0S;alWijR**S1YsM}cUD7Z%J#1r+Akf%JedSH=dfiu z-DBSUD#*eBQ+4mwWAM^B|6sM|&-3?{*Gz$Lu>A2ShsDwQ+;^uPXc^-|1Bh9=2!!~r z;l~}%_cPBFodgkcNN#Mu!@`bDb#G2TN$D{{!NR-3fGM6h>uBvqSOrTE%I$TMjdX8d zRgva*yaF)w=N)>Udxv%{m=B6nJsGBx%d9g?k|z>Rqjl(C!q>kDqA+63{{b~Z`ydAV zlhKhQ#=sWNZ=%}Qd!td9(h>;#NgW=@&LLp^YQMb*Q~N?yw}U>5cb9DAasjsL8;w5z zg9yB#*zVLG#5EncvLoEg`Ikuyf{El4l^}8>7wUTY5_BJIkwb3Iyfh4p#H2W)U%s@b44 zm4+((tXo}NLsObh=C#s2zb8_R>>cKlbNK_Lh1!}*9`fLZp~zKBuebg_7#pW3ELqzt zyDIj}&w*1bKn!?Of|VY%P{Q2%fQkaBt?2U){QN4R>*CIg!f|ld8E7IB-GiS9Uv=gK zzNh9E69eE^)L5}6m~uB{W~oS2Q*uulwzz-<)>C{6`ajF5og=XPdpyoQ+tYiu0AzQ@N+;4LO1hGel;YO}EX zh5~KxUKS-#3mph;{%s`euLX*+%JHePXS|eRU-lzD;6jY-y7C=ZpS29Qx|G`Ibq`oF za>Na|(|?bgvGO|*l%-9Dm#e*}WlPlu@3z-GrsI6fQl;+AgO7gYA?Gt$;|p zU0bKSzczrP?nZRk2&4t{{5D>yDY_yNL)|o zbhQm*yUUNr^=(e;gD!E4IAJHXT;h4fLWy0VQbsik;d|2KX+-d8`)k&Y2ZK}Whjl#I zp6P>7NKF)~{4xC(vQ(F#^G_vP_xnQi{`?%-03QwDP&97!8YT@DtcH|(@6;&DfN;!M zW6x-D1NqM|U=`(J0tEJ!da?&fB}UfR&VNblsqVha^ULJ^?PF7js<+c(eGOy=q7=%G zv;%QG%40_wlod3WPF@M&Z&xhBkb@bgQi_}nP?(#UA96sCb??-|r(1mgFlUFQ2#%r^1*9^G^zfG@D)Bq!}}d zu*u*!Hk7JDg2!5#Pw3XScIy?RTeBJ9V#CI}xoh5Xro03VDlMY_UE>MxC}J+xkU?dG zTj>%~N~-(h{d8i)?G}xX%M}90&m10x9Lesv#Ejm>teDEl7UFvc$-+UnK~heW7mSccoyE~_T&Jj^9>N=W3tvu>^sisB=d zQF&3$MutQd(@BEM7=$B?d3sdwX1D z0!oh3`}lJOa|pu_ysG}Z_`R7Yhr-CICY?yk!uSvYHGe~AP;dODsM(d<9`XT9REb6YcjdIpi+(NA?uVS?D@J)=XdnK{?EK* zE#q_zo#|6M*&D_D12HO+yq=O$shhVf@BF(D=>cf|PoO=TE*X!@&Y;2bA2iw<E43AgWJNv1FxlwG+Ac7kK|S^1WiD%IJrqJHC>?vU#?`6jAEVTNfIPin7U*xT^? zdW>#_l(VtI1?WUV$|_dfuV|y&u*{A-Y*B_&h~bIh>cy=`u13M{Qt1!M%uF-a+=HV; z1@VL07+7;J#8#xB*|`Grc0_}4<}+xwNH{g;{^vQk)2?Ngl(c~A>YtmFM0lGImO!^1 z(F7ODY(_MLj3?p}5-5S#OK9fbswg)sgP2#ZHSA9A4f>c)>sIGYGz| z$XX~j_y$x)aaCWOv24D|gEZq*ib)Q2c?^Mkr%eVl6mucgg9RW}$f{Gti90J_td;CB zzNqxgOJ|CKYVnR`>lqxv2H@oV-uD%zWGb3uKR%lAe{nXt`RLO+L`<_YEPjdu1Kj|1 zznzF&v(0n$9jiE?=%yLflR7PbOnK7Rc_GKj30^m)aB8#z0Bdvt=X)#~dd+W%}tqGP~0 z^F-m?NRTf-l-2%Jys2&t%Wr{F-SkG5WOsyJQSE>kIE5boKZ@NwqVSHCSf%$8mQrQz zY`=jrCWAm!rGl50JXnk3F`^e?cCz~ft&uVIGJPiMa&7JaV%jx|>D*QhQB2&x?Y)ZH z?e!uL@MhSyl1kpq#4dc1Xmy-P5YWg~2i1Lr%IpB3Lcu(nRW#u2;+qQ}_g+_BN@^t# zXm&EaWSc7MW|fznFdqu-tO^axAfyzjx_t*Ps-NTNRcnllWL-?gZ4(Y*C1;PR8~FOu zMp3cs1LG^CYP_v@hX!wcn6XdhJVg&`6EegqB2rrl%(d>DdR$TZTTRQMvC}zzVan#-s5LvvBvh(}&ch?Gc(&2bwt!UQ2VTPoN8UKsvLnX<`PiIDCA9li#z+J5PLc%dZk*J61U zPyEXVmRK5b)7p*oawS0CwVp)K_TPi@uFl&KJH}VG{JbF+Orl3C`H0pz0Nsc~7 zrXu37y|1?2sa!&lFFPLYfbE$JaNCO!6oe;Mg^|Cw}|LeeLCyxEaV z1tS8_%8{H-&lo=9vlrKL6yGxOO|O#{63BrpLA4C`2-$S#x<8cP<6%MFXC>(38-)Yb z2vc#;Bu?D)=9HPCX;FRPP z#y(1vBm+PYJr#9(Lbeng5nxQo0XJaqoC~lWu$B3@QK{D%c2pjF^cg2eWAe!futQLR zL#LkpKZ6y)zcOx*@&rmgKch1{vm_%$CK!sQQpP_SGMRwkg2rDscSqFva=BT6QS)f? z{ZR$b438dNid^U)58T|MG%zl|^KT0V;;wxAdb}~r$DhXKo*$$3hhNxruXy!FSe;AJwKEo}xg}Z4Bn_STHVc=}0uycy$N0&+ zKEqQx#R0p_^47#LfSIV72!Oxk=+|eIF{gx|sbr;~&mW2q=fQ4T;DaH4qg^M8e-9q1 z8>a^rgTW{d@Yj>!(TFUDH976S+~+Pe$y zbCltXzk0yEH6GWEEkx)jSm5wK&U(e_I*qh$RuIU0M!1QA9_ z!IFUv9>)lJy4&2nVJ>K{+Zwkls9e3Rdv9UEwP~dhrr*PCVUX4Cygd|;J9(`PaSF)_ ziKL-&M@cYw%nL#vJbDddu@LOF!5EkGq%|%di&eDejDrm~%Yn;4NqnC@Yj5qe%0mD& z`k3jC;B-r*!?if0VnL@=C6q;HDPU(`l*g@S*W3iSg;b{)x5Sp55K&s0X>il(>SP}u zQNJY>umLCXAQy-{&D7bIR#)ZDjlJyS1h2#%$l_PKs=+;{Id(WOoOcc0-} zT=vcgg?K0_PHGVCq`jk`g1@~1%u4mr(CdOX z6}3_~$rU5IqAyq1tc1hfHVurw&_b%ev0MXu60tuv{aKc8`bw4pZwKu+K{JkR!3IcO z8FAND^ETcC;PIEESL}Folb0?_aI&DSWl6d%S{9Nw(*oUnjPOR7Mow5e`cw4lZ_=@1 z?j*@ZE0bc?B1U5fjYr8|OGeOA>XglBL?AbZskF#UlRE%?*d8Cf2vuG_N2oFUcF)IN znON3x5TEy`YsdC2>ZxdLslg%EA&kMyLryDC@$=t_*_>F0;QAT}6{m0vqyl~1T(fh; z9`3ClRH;A?3>QFxj3)HMWQWf_I|w)2xAA(of9?h~Osw}mi+E~R)-giC&lBFYcbc*uYebzM%-?M zQ4Id!!*@_0_1HnxchS?-cu_6iUTpN9)u;gI;aPLuziYu9@YhWor^i*}Q=~sj3MeFU ztkElt?{ja`C)00HB(%VuexPx^L0@#N*>VkI_eRd1(&>0?OWpm1QCzwm{AuwuiBCq< zH2ahpJBlE_pT@m@?Pk@rHP;?Ge-& zE?1~6zbXXRq}U9P|)qS?C-~Al5eQ7HD!wHQobXxPF z&Ljx8)VZV*Xo`j0*qL2%0Gza0>8u0AigxrFPGw<>p8|{!1{@tMEP-aB%c+ZT;bXGE z{jiwRnHl}@M()WSJt|gB4KRO}g3WU|ol&Nk!;f=@IPqgJzq8@5WKiBhFA+p1W}(pI zE_&G6TG*q(Auu_geSvK11Fel_YpACAXeKTs?cCPpQ1AoyBubl4DD;)hRo+F{oF_X* z0p8;O=Tv8<)b%k=w#pdI`?hUP=J#S7G&H5puLxrP&HdYXiWLTn7#S}=PxvKNbMtr3 zKWlJX{+QSdF5ZN0s?i(j%GgTth4XfVX9rhi+{Wq=2s`Afmu%edJD^-j3sPOLQA|`n zTBYy%+7<_xu+RYTcz?n ztJy>jlT*d2gwwr;06WHl{m7u482pN2i@x>Xj)}PN2#F3ZMS9+?XnZd3YhTy7R1-O= zR00qKy!Owv{g7!=?OcF6Vx1%daiFI^6gRyagT=DWEE|`=Wp~%G@@Bh;!fK9HUR%Lk z(6_K6J7FPL&K~LyP;CX+0aD{ynWSl)H%+g-aWPOF8wj5>I>AYswHq9-Tz83VlU`pL zNFL9d1aU3f&SBCYVI@Ce&3HTti0prEy4VA6p+eBarvmnasj{%REg9Z4K0g%z5+q zd*|HZ$DDO{LfB@RT*v|B2Xjf1yR^>+hAqEjTB@}%`K$gB+;Nu1W=(TWcyns-VUZGv z7pPJFjjfN~(fh7&ODY9?|2+!iIJ@xOs}S#ebJ#1jT(OU%kdyJO2(%U#iI|s`W9{6f zBrl?drU0k~-7?m;8tLwZ6I=vLm(4FEzoX>-DB=5vbi$J|9B#a6h`yZZ@%JU^jk*hxcqE>gOWEI1eq&1s=rUe(q$N8+__f0Wq`#_|A(yKt7h& zQhV}FE2Hv$ako6wlqo&vv1lN76Ymxrr6o@+#o0}=({(P0c!)}8d~``-ifUy2v)=iu zBNfF$cgdipLaQx!Livqi3~aML`#vE9yPM4Y5RB`oueQU@fao(}(`vCm6KyrZ(tO550#{ z;2cYfB%9jxQFea+iK~=vo@v(yh*qb4>yb|H-3Iu&BF7Lv?$3=c{-b7-bwwi`l+_lk z=$$0YRP{&%JS%_IK&g%@v93^MRGE3%pxzBVw`-AY$$(_1{_jyK<-TiIE`mSTK1JDO;>1nRp7}X-kD=_xA`;H=Cm>9KYJyzI z)l2hT>iI93*vOUF;{K>JhVbT_VP4M}*XB`bT=&#!vm(*mSx43~XJZ}Hylg2(Pt%@N z79NHlhqPYj8oXS0W!{VqU#Dk)BXd_lUQQ}1SN0100Jpx%>vhxwMu8<68HPUmDa^kn zLb4=xGA1z)K2_1;4BK&zBh0e;?{U6D=K$`#WOPPRrcVF!QpMkue1Cjt26u`fOKenk z7Qk#E8VU%gyM~Th3LyWy?}g)-Q74QgfjCK{n|^7Pzk_Dt6966Hrysnka^@=544 zkL=~3q~_RLrkvd(k02`=@=*zDXYC)h%%i`sNpnguW&PsY_6R7*$$Mm$jd7ZE%`GVs z{{fCrR(QVWPgWv3Q{+wr^Jt_4UTjILEpsJ2vX_#|p$bdbL%O*t%k6egjL8~#LpIVr zze#1~ayu*6;SYlU~f?_~gr?gj;&9qkRa(BwIDua=m`X-9=`vgzU0zk25i8wRV0%r4;fFDJMB_`^@F$<3vZyK_e0Ik}SlSJe^wHoq!`Zzne9Y*7 zxMj2zoU``@DRY#CP9HhV7-0(0Wodueqn|6VT)N+K=UO` z<+a0J@kJ!!#}^e;*M9E|`9UU6VoPXXs~{z^D0{tPe4ykmSHX1dPRk-OpU zZ6$ZWs+_!!ArGeO`mr+}90g}FYTCjqigknum0%IyRVMu(v4Wh#nzr{-W-E~7pCz+S hY_RWnvXt!;6BB^L!h&!yDt`#9*7Iy)KL39)t@N~W73lx~ diff --git a/test/cpp/interop/server_helper.cc b/test/cpp/interop/server_helper.cc index c6d891ad71b..8b0b511bcb8 100644 --- a/test/cpp/interop/server_helper.cc +++ b/test/cpp/interop/server_helper.cc @@ -72,6 +72,10 @@ uint32_t InteropServerContextInspector::GetEncodingsAcceptedByClient() const { return grpc_call_test_only_get_encodings_accepted_by_peer(context_.call_); } +uint32_t InteropServerContextInspector::GetMessageFlags() const { + return grpc_call_test_only_get_message_flags(context_.call_); +} + std::shared_ptr InteropServerContextInspector::GetAuthContext() const { return context_.auth_context(); diff --git a/test/cpp/interop/server_helper.h b/test/cpp/interop/server_helper.h index 12865e40324..a1da14a4c8d 100644 --- a/test/cpp/interop/server_helper.h +++ b/test/cpp/interop/server_helper.h @@ -54,6 +54,7 @@ class InteropServerContextInspector { bool IsCancelled() const; grpc_compression_algorithm GetCallCompressionAlgorithm() const; uint32_t GetEncodingsAcceptedByClient() const; + uint32_t GetMessageFlags() const; private: const ::grpc::ServerContext& context_; diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index e23c1cb6002..5aea0af0a2f 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -4519,7 +4519,7 @@ "language": "c++", "name": "interop_server_main", "src": [ - "test/cpp/interop/server_main.cc" + "test/cpp/interop/interop_server.cc" ], "third_party": false, "type": "lib" diff --git a/vsprojects/vcxproj/interop_server_main/interop_server_main.vcxproj b/vsprojects/vcxproj/interop_server_main/interop_server_main.vcxproj index 075750afc6d..18971d6a341 100644 --- a/vsprojects/vcxproj/interop_server_main/interop_server_main.vcxproj +++ b/vsprojects/vcxproj/interop_server_main/interop_server_main.vcxproj @@ -171,7 +171,7 @@ - + diff --git a/vsprojects/vcxproj/interop_server_main/interop_server_main.vcxproj.filters b/vsprojects/vcxproj/interop_server_main/interop_server_main.vcxproj.filters index 51a6b9e73c3..4ee8135c045 100644 --- a/vsprojects/vcxproj/interop_server_main/interop_server_main.vcxproj.filters +++ b/vsprojects/vcxproj/interop_server_main/interop_server_main.vcxproj.filters @@ -10,7 +10,7 @@ src\proto\grpc\testing - + test\cpp\interop From 7b3fa9e54fda047237f47be2c13fc015bb0546a4 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Thu, 9 Jun 2016 16:12:53 -0700 Subject: [PATCH 052/280] Reworded spec. --- doc/interop-test-descriptions.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/doc/interop-test-descriptions.md b/doc/interop-test-descriptions.md index a023d80c50d..ebeb62753b4 100644 --- a/doc/interop-test-descriptions.md +++ b/doc/interop-test-descriptions.md @@ -92,8 +92,10 @@ Client asserts: ### client_compressed_unary -This test verifies the client can compress unary messages. It sends one -unary request for a compressable payload type, with and without compression. +This test verifies the client can compress unary messages. It sends two +unary requests with their payloads marked as COMPRESSABLE. One request will be +sent compressed and its `expect_compressed_request` set to true. Conversely for +the uncompressed case. Server features: * [UnaryCall][] @@ -138,9 +140,9 @@ Procedure: ### server_compressed_unary -This test verifies the server can compress unary messages. It sends one unary -request for a COMPRESSABLE payload type, with and without requesting a -compressed response from the server. +This test verifies the server can compress unary messages. It sends two unary +requests for a COMPRESSABLE payload type, expecting the server response to be +compressed or not according to the `request_compressed_response` boolean. Whether compression was actually performed is determined by the compression bit in the response's message flags. From c7be7c688829281b543428ec22029d4d09bd2a9c Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 9 Jun 2016 17:08:50 -0700 Subject: [PATCH 053/280] Add an API at the core level to disable signals or use a different signal number --- include/grpc/grpc_posix.h | 8 +++++ src/core/lib/iomgr/ev_epoll_linux.c | 45 ++++++++++++++++++++++------- 2 files changed, 43 insertions(+), 10 deletions(-) diff --git a/include/grpc/grpc_posix.h b/include/grpc/grpc_posix.h index 9742b83374a..5e89ae3b1ee 100644 --- a/include/grpc/grpc_posix.h +++ b/include/grpc/grpc_posix.h @@ -63,6 +63,14 @@ GRPCAPI void grpc_server_add_insecure_channel_from_fd(grpc_server *server, grpc_completion_queue *cq, int fd); +/** GRPC Core POSIX library may internally use signals to optimize some work. + The library uses (SIGRTMIN + 2) signal by default. Use this API to instruct + the library to use a different signal i.e 'signum' instead. + Note: + - To prevent GRPC library from using any signals, pass a 'signum' of -1 + - This API is optional but if called, it MUST be called before grpc_init() */ +GRPCAPI void grpc_use_signal(int signum); + #ifdef __cplusplus } #endif diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index d2d5d2852b9..7e01ac144f2 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -31,6 +31,7 @@ * */ +#include #include #ifdef GPR_LINUX_EPOLL @@ -58,9 +59,26 @@ #include "src/core/lib/profiling/timers.h" #include "src/core/lib/support/block_annotate.h" -struct polling_island; +static int grpc_wakeup_signal = -1; +static bool is_grpc_wakeup_signal_initialized = false; + +/* Implements the function defined in grpc_posix.h. This function might be + * called before even calling grpc_init() to set either a different signal to + * use. If signum == -1, then the use of signals is disabled */ +void grpc_use_signal(int signum) { + grpc_wakeup_signal = signum; + is_grpc_wakeup_signal_initialized = true; -static int grpc_poller_kick_signum; + if (grpc_wakeup_signal < 0) { + gpr_log(GPR_INFO, + "Use of signals is disabled. Epoll engine will not be used"); + } else { + gpr_log(GPR_INFO, "epoll engine will be using signal: %d", + grpc_wakeup_signal); + } +} + +struct polling_island; /******************************************************************************* * Fd Declarations @@ -854,10 +872,7 @@ static void sig_handler(int sig_num) { #endif } -static void poller_kick_init() { - grpc_poller_kick_signum = SIGRTMIN + 2; - signal(grpc_poller_kick_signum, sig_handler); -} +static void poller_kick_init() { signal(grpc_wakeup_signal, sig_handler); } /* Global state management */ static void pollset_global_init(void) { @@ -874,7 +889,7 @@ static void pollset_global_shutdown(void) { } static void pollset_worker_kick(grpc_pollset_worker *worker) { - pthread_kill(worker->pt_id, grpc_poller_kick_signum); + pthread_kill(worker->pt_id, grpc_wakeup_signal); } /* Return 1 if the pollset has active threads in pollset_work (pollset must @@ -1214,9 +1229,9 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, pollset->kicked_without_pollers = 0; } else if (!pollset->shutting_down) { sigemptyset(&new_mask); - sigaddset(&new_mask, grpc_poller_kick_signum); + sigaddset(&new_mask, grpc_wakeup_signal); pthread_sigmask(SIG_BLOCK, &new_mask, &orig_mask); - sigdelset(&orig_mask, grpc_poller_kick_signum); + sigdelset(&orig_mask, grpc_wakeup_signal); push_front_worker(pollset, &worker); @@ -1497,19 +1512,29 @@ static bool is_epoll_available() { } const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { + /* If use of signals is disabled, we cannot use epoll engine*/ + if (is_grpc_wakeup_signal_initialized && grpc_wakeup_signal < 0) { + return NULL; + } + if (!is_epoll_available()) { return NULL; } + if (!is_grpc_wakeup_signal_initialized) { + grpc_use_signal(SIGRTMIN + 2); + } + fd_global_init(); pollset_global_init(); polling_island_global_init(); return &vtable; } -#else /* defined(GPR_LINUX_EPOLL) */ +#else /* defined(GPR_LINUX_EPOLL) */ /* If GPR_LINUX_EPOLL is not defined, it means epoll is not available. Return * NULL */ const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { return NULL; } +void grpc_use_signal(int signum) {} #endif /* !defined(GPR_LINUX_EPOLL) */ From 492fd961824447e7f12974893dd9bbf37291e5cc Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 10 Jun 2016 09:03:34 -0700 Subject: [PATCH 054/280] generate_projects.sh after adding grpc_use_signal() API --- BUILD | 2 ++ Makefile | 1 + grpc.def | 1 + src/python/grpcio/grpc/_cython/imports.generated.c | 2 ++ src/python/grpcio/grpc/_cython/imports.generated.h | 3 +++ src/ruby/ext/grpc/rb_grpc_imports.generated.c | 2 ++ src/ruby/ext/grpc/rb_grpc_imports.generated.h | 3 +++ 7 files changed, 14 insertions(+) diff --git a/BUILD b/BUILD index 68b066d5593..43855751cbc 100644 --- a/BUILD +++ b/BUILD @@ -556,6 +556,7 @@ cc_library( "src/core/lib/iomgr/closure.h", "src/core/lib/iomgr/endpoint.h", "src/core/lib/iomgr/endpoint_pair.h", + "src/core/lib/iomgr/ev_epoll_linux.h", "src/core/lib/iomgr/ev_poll_and_epoll_posix.h", "src/core/lib/iomgr/ev_poll_posix.h", "src/core/lib/iomgr/ev_posix.h", @@ -693,6 +694,7 @@ cc_library( "src/core/lib/iomgr/endpoint.c", "src/core/lib/iomgr/endpoint_pair_posix.c", "src/core/lib/iomgr/endpoint_pair_windows.c", + "src/core/lib/iomgr/ev_epoll_linux.c", "src/core/lib/iomgr/ev_poll_and_epoll_posix.c", "src/core/lib/iomgr/ev_poll_posix.c", "src/core/lib/iomgr/ev_posix.c", diff --git a/Makefile b/Makefile index 620299ed15b..4d8b060760a 100644 --- a/Makefile +++ b/Makefile @@ -2764,6 +2764,7 @@ LIBGRPC_CRONET_SRC = \ src/core/lib/iomgr/endpoint.c \ src/core/lib/iomgr/endpoint_pair_posix.c \ src/core/lib/iomgr/endpoint_pair_windows.c \ + src/core/lib/iomgr/ev_epoll_linux.c \ src/core/lib/iomgr/ev_poll_and_epoll_posix.c \ src/core/lib/iomgr/ev_poll_posix.c \ src/core/lib/iomgr/ev_posix.c \ diff --git a/grpc.def b/grpc.def index 0046028949e..9f70ce61a30 100644 --- a/grpc.def +++ b/grpc.def @@ -90,6 +90,7 @@ EXPORTS grpc_call_error_to_string grpc_insecure_channel_create_from_fd grpc_server_add_insecure_channel_from_fd + grpc_use_signal grpc_auth_property_iterator_next grpc_auth_context_property_iterator grpc_auth_context_peer_identity diff --git a/src/python/grpcio/grpc/_cython/imports.generated.c b/src/python/grpcio/grpc/_cython/imports.generated.c index 5c49f6cf3e9..a025b99047d 100644 --- a/src/python/grpcio/grpc/_cython/imports.generated.c +++ b/src/python/grpcio/grpc/_cython/imports.generated.c @@ -128,6 +128,7 @@ grpc_is_binary_header_type grpc_is_binary_header_import; grpc_call_error_to_string_type grpc_call_error_to_string_import; grpc_insecure_channel_create_from_fd_type grpc_insecure_channel_create_from_fd_import; grpc_server_add_insecure_channel_from_fd_type grpc_server_add_insecure_channel_from_fd_import; +grpc_use_signal_type grpc_use_signal_import; grpc_auth_property_iterator_next_type grpc_auth_property_iterator_next_import; grpc_auth_context_property_iterator_type grpc_auth_context_property_iterator_import; grpc_auth_context_peer_identity_type grpc_auth_context_peer_identity_import; @@ -401,6 +402,7 @@ void pygrpc_load_imports(HMODULE library) { grpc_call_error_to_string_import = (grpc_call_error_to_string_type) GetProcAddress(library, "grpc_call_error_to_string"); grpc_insecure_channel_create_from_fd_import = (grpc_insecure_channel_create_from_fd_type) GetProcAddress(library, "grpc_insecure_channel_create_from_fd"); grpc_server_add_insecure_channel_from_fd_import = (grpc_server_add_insecure_channel_from_fd_type) GetProcAddress(library, "grpc_server_add_insecure_channel_from_fd"); + grpc_use_signal_import = (grpc_use_signal_type) GetProcAddress(library, "grpc_use_signal"); grpc_auth_property_iterator_next_import = (grpc_auth_property_iterator_next_type) GetProcAddress(library, "grpc_auth_property_iterator_next"); grpc_auth_context_property_iterator_import = (grpc_auth_context_property_iterator_type) GetProcAddress(library, "grpc_auth_context_property_iterator"); grpc_auth_context_peer_identity_import = (grpc_auth_context_peer_identity_type) GetProcAddress(library, "grpc_auth_context_peer_identity"); diff --git a/src/python/grpcio/grpc/_cython/imports.generated.h b/src/python/grpcio/grpc/_cython/imports.generated.h index 16bb5cdfab6..5bdbbce89b5 100644 --- a/src/python/grpcio/grpc/_cython/imports.generated.h +++ b/src/python/grpcio/grpc/_cython/imports.generated.h @@ -335,6 +335,9 @@ extern grpc_insecure_channel_create_from_fd_type grpc_insecure_channel_create_fr typedef void(*grpc_server_add_insecure_channel_from_fd_type)(grpc_server *server, grpc_completion_queue *cq, int fd); extern grpc_server_add_insecure_channel_from_fd_type grpc_server_add_insecure_channel_from_fd_import; #define grpc_server_add_insecure_channel_from_fd grpc_server_add_insecure_channel_from_fd_import +typedef void(*grpc_use_signal_type)(int signum); +extern grpc_use_signal_type grpc_use_signal_import; +#define grpc_use_signal grpc_use_signal_import typedef const grpc_auth_property *(*grpc_auth_property_iterator_next_type)(grpc_auth_property_iterator *it); extern grpc_auth_property_iterator_next_type grpc_auth_property_iterator_next_import; #define grpc_auth_property_iterator_next grpc_auth_property_iterator_next_import diff --git a/src/ruby/ext/grpc/rb_grpc_imports.generated.c b/src/ruby/ext/grpc/rb_grpc_imports.generated.c index c13d1a00d74..8e24ef38ed5 100644 --- a/src/ruby/ext/grpc/rb_grpc_imports.generated.c +++ b/src/ruby/ext/grpc/rb_grpc_imports.generated.c @@ -128,6 +128,7 @@ grpc_is_binary_header_type grpc_is_binary_header_import; grpc_call_error_to_string_type grpc_call_error_to_string_import; grpc_insecure_channel_create_from_fd_type grpc_insecure_channel_create_from_fd_import; grpc_server_add_insecure_channel_from_fd_type grpc_server_add_insecure_channel_from_fd_import; +grpc_use_signal_type grpc_use_signal_import; grpc_auth_property_iterator_next_type grpc_auth_property_iterator_next_import; grpc_auth_context_property_iterator_type grpc_auth_context_property_iterator_import; grpc_auth_context_peer_identity_type grpc_auth_context_peer_identity_import; @@ -397,6 +398,7 @@ void grpc_rb_load_imports(HMODULE library) { grpc_call_error_to_string_import = (grpc_call_error_to_string_type) GetProcAddress(library, "grpc_call_error_to_string"); grpc_insecure_channel_create_from_fd_import = (grpc_insecure_channel_create_from_fd_type) GetProcAddress(library, "grpc_insecure_channel_create_from_fd"); grpc_server_add_insecure_channel_from_fd_import = (grpc_server_add_insecure_channel_from_fd_type) GetProcAddress(library, "grpc_server_add_insecure_channel_from_fd"); + grpc_use_signal_import = (grpc_use_signal_type) GetProcAddress(library, "grpc_use_signal"); grpc_auth_property_iterator_next_import = (grpc_auth_property_iterator_next_type) GetProcAddress(library, "grpc_auth_property_iterator_next"); grpc_auth_context_property_iterator_import = (grpc_auth_context_property_iterator_type) GetProcAddress(library, "grpc_auth_context_property_iterator"); grpc_auth_context_peer_identity_import = (grpc_auth_context_peer_identity_type) GetProcAddress(library, "grpc_auth_context_peer_identity"); diff --git a/src/ruby/ext/grpc/rb_grpc_imports.generated.h b/src/ruby/ext/grpc/rb_grpc_imports.generated.h index 9c86a3690c5..d6e2a367999 100644 --- a/src/ruby/ext/grpc/rb_grpc_imports.generated.h +++ b/src/ruby/ext/grpc/rb_grpc_imports.generated.h @@ -335,6 +335,9 @@ extern grpc_insecure_channel_create_from_fd_type grpc_insecure_channel_create_fr typedef void(*grpc_server_add_insecure_channel_from_fd_type)(grpc_server *server, grpc_completion_queue *cq, int fd); extern grpc_server_add_insecure_channel_from_fd_type grpc_server_add_insecure_channel_from_fd_import; #define grpc_server_add_insecure_channel_from_fd grpc_server_add_insecure_channel_from_fd_import +typedef void(*grpc_use_signal_type)(int signum); +extern grpc_use_signal_type grpc_use_signal_import; +#define grpc_use_signal grpc_use_signal_import typedef const grpc_auth_property *(*grpc_auth_property_iterator_next_type)(grpc_auth_property_iterator *it); extern grpc_auth_property_iterator_next_type grpc_auth_property_iterator_next_import; #define grpc_auth_property_iterator_next grpc_auth_property_iterator_next_import From dd8d84ebb576630f7114b97a6677c1983687cfb4 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Fri, 10 Jun 2016 19:10:42 -0700 Subject: [PATCH 055/280] spec comments addressed --- doc/interop-test-descriptions.md | 212 ++++++++++++-------------- src/proto/grpc/testing/messages.proto | 20 +-- 2 files changed, 106 insertions(+), 126 deletions(-) diff --git a/doc/interop-test-descriptions.md b/doc/interop-test-descriptions.md index ebeb62753b4..3dd7807dec0 100644 --- a/doc/interop-test-descriptions.md +++ b/doc/interop-test-descriptions.md @@ -68,14 +68,12 @@ control (even if compression is enabled on the channel). Server features: * [UnaryCall][] -* [Compressable Payload][] Procedure: 1. Client calls UnaryCall with: ``` { - response_type: COMPRESSABLE response_size: 314159 payload:{ body: 271828 bytes of zeros @@ -85,29 +83,39 @@ Procedure: Client asserts: * call was successful -* response payload type is COMPRESSABLE * response payload body is 314159 bytes in size * clients are free to assert that the response payload body contents are zero and comparing the entire response message against a golden response ### client_compressed_unary -This test verifies the client can compress unary messages. It sends two -unary requests with their payloads marked as COMPRESSABLE. One request will be -sent compressed and its `expect_compressed_request` set to true. Conversely for -the uncompressed case. +This test verifies the client can compress unary messages. It sends an initial +inconsistent request to verify whether the server supports the +[CompressedRequest][] feature. If it does, it should catch the inconsistency and +fail the call with an `INVALID_ARGUMENT` status. If the feature is supported, it +proceeds with two unary calls, for compressed and uncompressed payloads. Server features: * [UnaryCall][] -* [Compressed Request][] +* [CompressedRequest][] Procedure: - 1. Client calls UnaryCall with: + 1. Client calls UnaryCall with the feature probe, an **uncompressed** message: + ``` + { + expect_compressed: false + response_size: 314159 + payload:{ + body: 271828 bytes of zeros + } + } + ``` + + 1. Client calls UnaryCall with the *compressed* message: ``` { - expect_compressed_request: true - response_type: COMPRESSABLE + expect_compressed: true response_size: 314159 payload:{ body: 271828 bytes of zeros @@ -115,10 +123,11 @@ Procedure: } ``` + 1. Client calls UnaryCall with the *uncompressed* message: + ``` { - expect_compressed_request: false - response_type: COMPRESSABLE + expect_compressed: false response_size: 314159 payload:{ body: 271828 bytes of zeros @@ -127,22 +136,18 @@ Procedure: ``` Client asserts: - * call was successful - * response payload type is COMPRESSABLE - * if `request_compressed_response` is false, the response MUST NOT have the - compressed message flag set. - * if `request_compressed_response` is true, the response MUST have the - compressed message flag set. - * response payload body is 314159 bytes in size - * clients are free to assert that the response payload body contents are - zero and comparing the entire response message against a golden response + * First call was unsuccessful with `INVALID_ARGUMENT` status. Subsequent + calls were successful. + * Response payload body is 314159 bytes in size. + * Clients are free to assert that the response payload body contents are + zero and comparing the entire response message against a golden response. ### server_compressed_unary This test verifies the server can compress unary messages. It sends two unary -requests for a COMPRESSABLE payload type, expecting the server response to be -compressed or not according to the `request_compressed_response` boolean. +requests, expecting the server response to be +compressed or not according to the `response_compressed` boolean. Whether compression was actually performed is determined by the compression bit in the response's message flags. @@ -150,16 +155,14 @@ in the response's message flags. Server features: * [UnaryCall][] -* [Compressable Payload][] -* [Compressed Response][] +* [CompressedResponse][] Procedure: 1. Client calls UnaryCall with: ``` { - request_compressed_response: true - response_type: COMPRESSABLE + response_compressed: true response_size: 314159 payload:{ body: 271828 bytes of zeros @@ -169,8 +172,7 @@ Procedure: ``` { - request_compressed_response: false - response_type: COMPRESSABLE + response_compressed: false response_size: 314159 payload:{ body: 271828 bytes of zeros @@ -179,10 +181,9 @@ Procedure: ``` Client asserts: * call was successful - * response payload type is COMPRESSABLE - * when `request_compressed_response` is true, the response MUST have the + * when `response_compressed` is true, the response MUST have the compressed message flag set. - * when `request_compressed_response` is false, the response MUST NOT have + * when `response_compressed` is false, the response MUST NOT have the compressed message flag set. * response payload body is 314159 bytes in size * clients are free to assert that the response payload body contents are @@ -195,7 +196,6 @@ This test verifies that client-only streaming succeeds. Server features: * [StreamingInputCall][] -* [Compressable Payload][] Procedure: 1. Client calls StreamingInputCall @@ -245,20 +245,70 @@ Client asserts: * call was successful * response aggregated_payload_size is 74922 + +### client_compressed_streaming + +This test verifies the client can compress streaming messages. It sends an +initial inconsistent streaming call comprised of a single message to verify if +the server implements the [CompressedRequest][] feature. If it does, the client +will then send another streaming call, comprised of two messages: the first one +compressed with `expect_compressed` true; the second one uncompressed with +`expected_compressed` false. + +Procedure: + 1. Client calls StreamingInputCall + 1. Client sends the following feature-probing *uncompressed* message + + ``` + { + expect_compressed: true + payload:{ + body: 27182 bytes of zeros + } + } + ``` + If the call fails with `INVALID_ARGUMENT`, the test fails. Otherwise, we + continue. + + 1. Client then sends the *compressed* message + + ``` + { + expect_compressed: true + payload:{ + body: 27182 bytes of zeros + } + } + ``` + 1. And finally, the *uncompressed* message: + ``` + { + expect_compressed: false + payload:{ + body: 45904 bytes of zeros + } + } + ``` + 1. Client half-closes + + Client asserts: + * First call was unsuccessful with `INVALID_ARGUMENT` status. Subsequent + calls were successful. + * Response aggregated_payload_size is 73086. + + ### server_streaming This test verifies that server-only streaming succeeds. Server features: * [StreamingOutputCall][] -* [Compressable Payload][] Procedure: 1. Client calls StreamingOutputCall with: ``` { - response_type:COMPRESSABLE response_parameters:{ size: 31415 } @@ -277,7 +327,6 @@ Procedure: Client asserts: * call was successful * exactly four responses -* response payloads are COMPRESSABLE * response payload bodies are sized (in order): 31415, 9, 2653, 58979 * clients are free to assert that the response payload body contents are zero and comparing the entire response messages against golden responses @@ -288,8 +337,7 @@ This test verifies that the server can compress streaming messages. Server features: * [StreamingOutputCall][] -* [Compressable Payload][] -* [Compressed Response][] +* [CompressedResponse][] Procedure: @@ -297,8 +345,7 @@ Procedure: ``` { - request_compressed_response: true - response_type:COMPRESSABLE + response_compressed: true response_parameters:{ size: 31415 } @@ -310,8 +357,7 @@ Procedure: ``` { - request_compressed_response: false - response_type:COMPRESSABLE + response_compressed: false response_parameters:{ size: 31415 } @@ -324,69 +370,27 @@ Procedure: Client asserts: * call was successful * exactly two responses - * response payloads are COMPRESSABLE - * when `request_compressed_response` is false, the response's messages MUST + * when `response_compressed` is false, the response's messages MUST NOT have the compressed message flag set. - * when `request_compressed_response` is true, the response's messages MUST + * when `response_compressed` is true, the response's messages MUST have the compressed message flag set. * response payload bodies are sized (in order): 31415, 58979 * clients are free to assert that the response payload body contents are zero and comparing the entire response messages against golden responses -### client_compressed_streaming - -This test verifies that the client can compress streaming messages. - -Server features: -* [StreamingInputCall][] -* [Compressed Request][] - -Procedure: - 1. Client calls StreamingInputCall - 1. Client sends: - - ``` - { - expect_compressed_request: true - payload:{ - body: 27182 bytes of zeros - } - } - ``` - - 1. Client then sends: - - ``` - { - expect_compressed_request: false - payload:{ - body: 45904 bytes of zeros - } - } - ``` - - 6. Client half-closes - - Client asserts: - * call was successful - * response aggregated_payload_size is 73086 - - ### ping_pong This test verifies that full duplex bidi is supported. Server features: * [FullDuplexCall][] -* [Compressable Payload][] Procedure: 1. Client calls FullDuplexCall with: ``` { - response_type: COMPRESSABLE response_parameters:{ size: 31415 } @@ -400,7 +404,6 @@ Procedure: ``` { - response_type: COMPRESSABLE response_parameters:{ size: 9 } @@ -414,7 +417,6 @@ Procedure: ``` { - response_type: COMPRESSABLE response_parameters:{ size: 2653 } @@ -428,7 +430,6 @@ Procedure: ``` { - response_type: COMPRESSABLE response_parameters:{ size: 58979 } @@ -443,7 +444,6 @@ Procedure: Client asserts: * call was successful * exactly four responses -* response payloads are COMPRESSABLE * response payload bodies are sized (in order): 31415, 9, 2653, 58979 * clients are free to assert that the response payload body contents are zero and comparing the entire response messages against golden responses @@ -479,7 +479,6 @@ be passed in as `--oauth_scope`. Server features: * [UnaryCall][] -* [Compressable Payload][] * [Echo Authenticated Username][] * [Echo OAuth Scope][] @@ -489,7 +488,6 @@ Procedure: ``` { - response_type: COMPRESSABLE response_size: 314159 payload:{ body: 271828 bytes of zeros @@ -523,7 +521,6 @@ variable GOOGLE_APPLICATION_CREDENTIALS. Server features: * [UnaryCall][] -* [Compressable Payload][] * [Echo Authenticated Username][] * [Echo OAuth Scope][] @@ -533,7 +530,6 @@ Procedure: ``` { - response_type: COMPRESSABLE response_size: 314159 payload:{ body: 271828 bytes of zeros @@ -578,7 +574,6 @@ should be passed as the `--oauth_scope`. Server features: * [UnaryCall][] -* [Compressable Payload][] * [Echo Authenticated Username][] * [Echo OAuth Scope][] @@ -621,7 +616,6 @@ against grpc-test.sandbox.googleapis.com, oauth scope Server features: * [UnaryCall][] -* [Compressable Payload][] * [Echo Authenticated Username][] * [Echo OAuth Scope][] @@ -652,7 +646,6 @@ by the server. Server features: * [UnaryCall][] * [FullDuplexCall][] -* [Compressable Payload][] * [Echo Metadata][] Procedure: @@ -667,7 +660,6 @@ Procedure: ``` { - response_type: COMPRESSABLE response_size: 314159 payload:{ body: 271828 bytes of zeros @@ -686,7 +678,6 @@ Procedure: ``` { - response_type: COMPRESSABLE response_size: 314159 payload:{ body: 271828 bytes of zeros @@ -792,14 +783,12 @@ from the server. Server features: * [FullDuplexCall][] -* [Compressable Payload][] Procedure: 1. Client starts FullDuplexCall with ``` { - response_type: COMPRESSABLE response_parameters:{ size: 31415 } @@ -946,15 +935,17 @@ for the `SimpleRequest.response_type`. If the server does not support the ### CompressedResponse [CompressedResponse]: #compressedresponse -When the client sets `SimpleRequest.request_compressed_response` to true, the -response is sent back compressed. +When the client sets `response_compressed` to true, the server's response is +sent back compressed. Note that `response_compressed` is present on both +`SimpleRequest` (unary) and `StreamingOutputCallRequest` (streaming). ### CompressedRequest [CompressedRequest]: #compressedrequest -When the client sets `SimpleRequest.expect_compressed_request ` to true, the -server expects the client request to be compressed. If it's not, it fails -the RPC with `INVALID_ARGUMENT`. +When the client sets `expect_compressed` to true, the server expects the client +request to be compressed. If it's not, it fails the RPC with `INVALID_ARGUMENT`. +Note that `response_compressed` is present on both `SimpleRequest` (unary) and +`StreamingOutputCallRequest` (streaming). ### StreamingInputCall [StreamingInputCall]: #streaminginputcall @@ -982,13 +973,6 @@ payload body of size ResponseParameters.size bytes, as specified by its respective ResponseParameters. After receiving half close and sending all responses, it closes with OK. -### Compressable Payload -[Compressable Payload]: #compressable-payload - -When the client requests COMPRESSABLE payload, the response includes a payload -of the size requested containing all zeros and the payload type is -COMPRESSABLE. - ### Echo Status [Echo Status]: #echo-status When the client sends a response_status in the request payload, the server closes diff --git a/src/proto/grpc/testing/messages.proto b/src/proto/grpc/testing/messages.proto index 99b75dea3d5..782fd40989d 100644 --- a/src/proto/grpc/testing/messages.proto +++ b/src/proto/grpc/testing/messages.proto @@ -34,6 +34,7 @@ syntax = "proto3"; package grpc.testing; +// DEPRECATED, don't use. To be removed shortly. // The type of payload that should be returned. enum PayloadType { // Compressable text format. @@ -42,6 +43,7 @@ enum PayloadType { // A block of data, to simply increase gRPC message size. message Payload { + // DEPRECATED, don't use. To be removed shortly. // The type of data in body. PayloadType type = 1; // Primary contents of payload. @@ -57,12 +59,12 @@ message EchoStatus { // Unary request. message SimpleRequest { + // DEPRECATED, don't use. To be removed shortly. // Desired payload type in the response from the server. // If response_type is RANDOM, server randomly chooses one from other formats. PayloadType response_type = 1; // Desired payload size in the response from the server. - // If response_type is COMPRESSABLE, this denotes the size before compression. int32 response_size = 2; // Optional input payload sent along with the request. @@ -75,16 +77,13 @@ message SimpleRequest { bool fill_oauth_scope = 5; // Whether to request the server to compress the response. - bool request_compressed_response = 6; + bool response_compressed = 6; // Whether server should return a given status EchoStatus response_status = 7; // Whether the server should expect this request to be compressed. - bool expect_compressed_request = 8; - - // The type of payload. - PayloadType payload_type = 9; + bool expect_compressed = 8; } // Unary response, as configured by the request. @@ -103,11 +102,8 @@ message StreamingInputCallRequest { // Optional input payload sent along with the request. Payload payload = 1; - // The type of payload. - PayloadType payload_type = 2; - // Whether the server should expect this request to be compressed. - bool expect_compressed_request = 3; + bool expect_compressed = 2; // Not expecting any payload from the response. } @@ -121,7 +117,6 @@ message StreamingInputCallResponse { // Configuration for a particular response. message ResponseParameters { // Desired payload sizes in responses from the server. - // If response_type is COMPRESSABLE, this denotes the size before compression. int32 size = 1; // Desired interval between consecutive responses in the response stream in @@ -131,6 +126,7 @@ message ResponseParameters { // Server-streaming request. message StreamingOutputCallRequest { + // DEPRECATED, don't use. To be removed shortly. // Desired payload type in the response from the server. // If response_type is RANDOM, the payload from each response in the stream // might be of different types. This is to simulate a mixed type of payload @@ -144,7 +140,7 @@ message StreamingOutputCallRequest { Payload payload = 3; // Whether to request the server to compress the response. - bool request_compressed_response = 4; + bool response_compressed = 6; // Whether server should return a given status EchoStatus response_status = 7; From eb16b3dc3cd579931d730ba3fef1f7008f649003 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 10 Jun 2016 23:06:25 -0700 Subject: [PATCH 056/280] Fix ref counting bug --- src/core/lib/iomgr/ev_epoll_linux.c | 20 ++------------------ 1 file changed, 2 insertions(+), 18 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 7e01ac144f2..617afad1975 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -1127,20 +1127,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, /* Update the pollset->polling_island */ pollset->polling_island = pi; -#ifdef GRPC_EPOLL_DEBUG - if (pollset->polling_island->fd_cnt == 0) { - gpr_log(GPR_DEBUG, "pollset_work_and_unlock: epoll_fd: %d, No other fds", - epoll_fd); - } - for (size_t i = 0; i < pollset->polling_island->fd_cnt; i++) { - gpr_log(GPR_DEBUG, - "pollset_work_and_unlock: epoll_fd: %d, fd_count: %d, fd[%d]: %d", - epoll_fd, pollset->polling_island->fd_cnt, i, - pollset->polling_island->fds[i]->fd); - } -#endif - gpr_mu_unlock(&pollset->polling_island->mu); - + polling_island_unref_and_unlock(pollset->polling_island, 0); /* Keep the ref*/ gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); @@ -1190,10 +1177,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, * gets updated whenever the underlying polling island is merged with another * island and while we are doing epoll_wait() above, the polling island may * have been merged */ - - /* TODO (sreek) - Change the ref count on polling island to gpr_atm so that - * we do not have to do this here */ - gpr_mu_lock(&pi->mu); + polling_island_update_and_lock(pi, 1, 0); /* No new ref added */ polling_island_unref_and_unlock(pi, 1); GPR_TIMER_END("pollset_work_and_unlock", 0); From 58e589644403b10afb31ffd45befabe13b652db8 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 13 Jun 2016 00:52:56 -0700 Subject: [PATCH 057/280] Fix bad merge --- src/core/lib/iomgr/ev_epoll_linux.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 617afad1975..a8a874cd4b0 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -550,14 +550,14 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Wakeup all the pollers (if any) on p so that they can pickup this change */ polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); + p->merged_to = q; + /* - The merged polling island (i.e q) inherits all the ref counts of the island merging with it (i.e p) - The island p will lose a ref count */ q->ref_cnt += p->ref_cnt; - p->ref_cnt--; - - gpr_mu_unlock(&p->mu); - gpr_mu_unlock(&q->mu); + polling_island_unref_and_unlock(p, 1); /* Decrement refcount */ + polling_island_unref_and_unlock(q, 0); /* Just Unlock. Don't decrement ref */ return q; } @@ -1110,7 +1110,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, Acquire the following locks: - pollset->mu (which we already have) - pollset->pi_mu - - pollset->polling_island->mu */ + - pollset->polling_island->mu (call polling_island_update_and_lock())*/ gpr_mu_lock(&pollset->pi_mu); pi = pollset->polling_island; @@ -1144,8 +1144,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, } } - int i; - for (i = 0; i < ep_rv; ++i) { + for (int i = 0; i < ep_rv; ++i) { void *data_ptr = ep_ev[i].data.ptr; if (data_ptr == &grpc_global_wakeup_fd) { grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); @@ -1177,7 +1176,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, * gets updated whenever the underlying polling island is merged with another * island and while we are doing epoll_wait() above, the polling island may * have been merged */ - polling_island_update_and_lock(pi, 1, 0); /* No new ref added */ + pi = polling_island_update_and_lock(pi, 1, 0); /* No new ref added */ polling_island_unref_and_unlock(pi, 1); GPR_TIMER_END("pollset_work_and_unlock", 0); From ddc3ebb6ea694b252c27e149507bec4df427a592 Mon Sep 17 00:00:00 2001 From: yang-g Date: Mon, 13 Jun 2016 10:40:32 -0700 Subject: [PATCH 058/280] add missing language tag --- build.yaml | 11 +++++----- tools/run_tests/sources_and_headers.json | 26 ++++++++++++------------ 2 files changed, 19 insertions(+), 18 deletions(-) diff --git a/build.yaml b/build.yaml index a94fb052a5d..8589ba28f8e 100644 --- a/build.yaml +++ b/build.yaml @@ -143,11 +143,6 @@ filegroups: - include/grpc/impl/codegen/sync_posix.h - include/grpc/impl/codegen/sync_windows.h - include/grpc/impl/codegen/time.h -- name: grpc++_codegen_base_src - src: - - src/cpp/codegen/codegen_init.cc - uses: - - grpc++_codegen_base - name: grpc_base public_headers: - include/grpc/byte_buffer.h @@ -747,6 +742,12 @@ filegroups: - include/grpc++/impl/codegen/time.h uses: - grpc_codegen +- name: grpc++_codegen_base_src + language: c++ + src: + - src/cpp/codegen/codegen_init.cc + uses: + - grpc++_codegen_base - name: grpc++_codegen_proto language: c++ public_headers: diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index fd522ee1739..1d73434f3dc 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -5607,19 +5607,6 @@ "third_party": false, "type": "filegroup" }, - { - "deps": [ - "grpc++_codegen_base" - ], - "headers": [], - "language": "c", - "name": "grpc++_codegen_base_src", - "src": [ - "src/cpp/codegen/codegen_init.cc" - ], - "third_party": false, - "type": "filegroup" - }, { "deps": [ "gpr", @@ -6636,6 +6623,19 @@ "third_party": false, "type": "filegroup" }, + { + "deps": [ + "grpc++_codegen_base" + ], + "headers": [], + "language": "c++", + "name": "grpc++_codegen_base_src", + "src": [ + "src/cpp/codegen/codegen_init.cc" + ], + "third_party": false, + "type": "filegroup" + }, { "deps": [ "grpc++_codegen_base", From 773a8825bcb9cae06af18263b112b8f8fd96f10d Mon Sep 17 00:00:00 2001 From: yang-g Date: Mon, 13 Jun 2016 11:09:29 -0700 Subject: [PATCH 059/280] Minor fixes --- src/core/lib/iomgr/udp_server.c | 4 ++-- src/cpp/server/server_posix.cc | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/core/lib/iomgr/udp_server.c b/src/core/lib/iomgr/udp_server.c index 16150687d3d..1ebccf2ee2e 100644 --- a/src/core/lib/iomgr/udp_server.c +++ b/src/core/lib/iomgr/udp_server.c @@ -243,13 +243,13 @@ static int prepare_socket(int fd, const struct sockaddr *addr, if (!grpc_set_socket_sndbuf(fd, buffer_size_bytes)) { gpr_log(GPR_ERROR, "Failed to set send buffer size to %d bytes", - buf_size_bytes); + buffer_size_bytes); goto error; } if (!grpc_set_socket_rcvbuf(fd, buffer_size_bytes)) { gpr_log(GPR_ERROR, "Failed to set receive buffer size to %d bytes", - buf_size_bytes); + buffer_size_bytes); goto error; } diff --git a/src/cpp/server/server_posix.cc b/src/cpp/server/server_posix.cc index 8cb9753a125..c3aa2adc60e 100644 --- a/src/cpp/server/server_posix.cc +++ b/src/cpp/server/server_posix.cc @@ -42,8 +42,8 @@ namespace grpc { void AddInsecureChannelFromFd(Server* server, int fd) { grpc_server_add_insecure_channel_from_fd( server->c_server(), server->completion_queue()->cq(), fd); +} #endif // GPR_SUPPORT_CHANNELS_FROM_FD -} } // namespace grpc From 41622a8e389e8eda38d6d3bfbf34cbf35f437156 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 13 Jun 2016 16:43:14 -0700 Subject: [PATCH 060/280] Fix tsan failures --- Makefile | 1 + build.yaml | 1 + src/core/lib/iomgr/ev_epoll_linux.c | 27 ++++++++++++++++++++++++++- 3 files changed, 28 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 4d8b060760a..28d6842c76c 100644 --- a/Makefile +++ b/Makefile @@ -200,6 +200,7 @@ LD_tsan = clang LDXX_tsan = clang++ CPPFLAGS_tsan = -O0 -fsanitize=thread -fno-omit-frame-pointer -Wno-unused-command-line-argument -DGPR_NO_DIRECT_SYSCALLS LDFLAGS_tsan = -fsanitize=thread +DEFINES_tsan = _GRPC_TSAN DEFINES_tsan += GRPC_TEST_SLOWDOWN_BUILD_FACTOR=5 VALID_CONFIG_stapprof = 1 diff --git a/build.yaml b/build.yaml index 85b66d985bb..139ab3e8bca 100644 --- a/build.yaml +++ b/build.yaml @@ -3231,6 +3231,7 @@ configs: CPPFLAGS: -O0 -fsanitize=thread -fno-omit-frame-pointer -Wno-unused-command-line-argument -DGPR_NO_DIRECT_SYSCALLS CXX: clang++ + DEFINES: _GRPC_TSAN LD: clang LDFLAGS: -fsanitize=thread LDXX: clang++ diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index a8a874cd4b0..35a15e00c96 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -236,6 +236,17 @@ static grpc_wakeup_fd polling_island_wakeup_fd; static gpr_mu g_pi_freelist_mu; static polling_island *g_pi_freelist = NULL; +#ifdef _GRPC_TSAN +/* Currently TSAN may incorrectly flag data races between epoll_ctl and + epoll_wait for any grpc_fd structs that are added to the epoll set via + epoll_ctl and are returned (within a very short window) via epoll_wait(). + + To work-around this race, we establish a happens-before relation between + the code just-before epoll_ctl() and the code after epoll_wait() by using + this atomic */ +gpr_atm g_epoll_sync; +#endif + /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, size_t fd_count, bool add_fd_refs) { @@ -243,6 +254,11 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, size_t i; struct epoll_event ev; +#ifdef _GRPC_TSAN + /* See the definition of g_epoll_sync for more context */ + gpr_atm_rel_store(&g_epoll_sync, 0); +#endif + for (i = 0; i < fd_count; i++) { ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); ev.data.ptr = fds[i]; @@ -361,6 +377,7 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, } pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); + if (pi->epoll_fd < 0) { gpr_log(GPR_ERROR, "epoll_create1() failed with error: %s", strerror(errno)); @@ -1144,6 +1161,11 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, } } +#ifdef _GRPC_TSAN + /* See the definition of g_poll_sync for more details */ + gpr_atm_acq_load(&g_epoll_sync); +#endif + for (int i = 0; i < ep_rv; ++i) { void *data_ptr = ep_ev[i].data.ptr; if (data_ptr == &grpc_global_wakeup_fd) { @@ -1514,10 +1536,13 @@ const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { return &vtable; } -#else /* defined(GPR_LINUX_EPOLL) */ +#else /* defined(GPR_LINUX_EPOLL) */ +#if defined(GPR_POSIX_SOCKET) +#include "src/core/lib/iomgr/ev_posix.h" /* If GPR_LINUX_EPOLL is not defined, it means epoll is not available. Return * NULL */ const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { return NULL; } +#endif /* defined(GPR_POSIX_SOCKET) */ void grpc_use_signal(int signum) {} #endif /* !defined(GPR_LINUX_EPOLL) */ From ad2c4778fc560f10f38550428189c97c9e2bc5a1 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Mon, 13 Jun 2016 19:06:54 -0700 Subject: [PATCH 061/280] Rename _GRPC_TSAN to GRPC_TSAN --- Makefile | 2 +- build.yaml | 2 +- src/core/lib/iomgr/ev_epoll_linux.c | 12 ++++++------ 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Makefile b/Makefile index 040ebd4102b..e35e360785b 100644 --- a/Makefile +++ b/Makefile @@ -200,7 +200,7 @@ LD_tsan = clang LDXX_tsan = clang++ CPPFLAGS_tsan = -O0 -fsanitize=thread -fno-omit-frame-pointer -Wno-unused-command-line-argument -DGPR_NO_DIRECT_SYSCALLS LDFLAGS_tsan = -fsanitize=thread -DEFINES_tsan = _GRPC_TSAN +DEFINES_tsan = GRPC_TSAN DEFINES_tsan += GRPC_TEST_SLOWDOWN_BUILD_FACTOR=5 VALID_CONFIG_stapprof = 1 diff --git a/build.yaml b/build.yaml index 0847232b504..3d327f8bffd 100644 --- a/build.yaml +++ b/build.yaml @@ -3266,7 +3266,7 @@ configs: CPPFLAGS: -O0 -fsanitize=thread -fno-omit-frame-pointer -Wno-unused-command-line-argument -DGPR_NO_DIRECT_SYSCALLS CXX: clang++ - DEFINES: _GRPC_TSAN + DEFINES: GRPC_TSAN LD: clang LDFLAGS: -fsanitize=thread LDXX: clang++ diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 35a15e00c96..006c2a8ee7f 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -236,7 +236,7 @@ static grpc_wakeup_fd polling_island_wakeup_fd; static gpr_mu g_pi_freelist_mu; static polling_island *g_pi_freelist = NULL; -#ifdef _GRPC_TSAN +#ifdef GRPC_TSAN /* Currently TSAN may incorrectly flag data races between epoll_ctl and epoll_wait for any grpc_fd structs that are added to the epoll set via epoll_ctl and are returned (within a very short window) via epoll_wait(). @@ -245,7 +245,7 @@ static polling_island *g_pi_freelist = NULL; the code just-before epoll_ctl() and the code after epoll_wait() by using this atomic */ gpr_atm g_epoll_sync; -#endif +#endif /* defined(GRPC_TSAN) */ /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, @@ -254,10 +254,10 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, size_t i; struct epoll_event ev; -#ifdef _GRPC_TSAN +#ifdef GRPC_TSAN /* See the definition of g_epoll_sync for more context */ gpr_atm_rel_store(&g_epoll_sync, 0); -#endif +#endif /* defined(GRPC_TSAN) */ for (i = 0; i < fd_count; i++) { ev.events = (uint32_t)(EPOLLIN | EPOLLOUT | EPOLLET); @@ -1161,10 +1161,10 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, } } -#ifdef _GRPC_TSAN +#ifdef GRPC_TSAN /* See the definition of g_poll_sync for more details */ gpr_atm_acq_load(&g_epoll_sync); -#endif +#endif /* defined(GRPC_TSAN) */ for (int i = 0; i < ep_rv; ++i) { void *data_ptr = ep_ev[i].data.ptr; From ec4359ddc1b905b12784babfcb4cebb3a1e67478 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Mon, 11 Apr 2016 20:55:44 -0700 Subject: [PATCH 062/280] add support for CoreCLR --- src/csharp/.gitignore | 2 + src/csharp/Grpc.Auth/Grpc.Auth.xproj | 19 ++++ src/csharp/Grpc.Auth/project.json | 35 +++++++ .../Grpc.Core.Tests/Grpc.Core.Tests.xproj | 19 ++++ .../Grpc.Core.Tests/NUnitVersionTest.cs | 2 +- src/csharp/Grpc.Core.Tests/SanityTest.cs | 2 + src/csharp/Grpc.Core.Tests/project.json | 28 ++++++ src/csharp/Grpc.Core/Grpc.Core.csproj | 2 +- src/csharp/Grpc.Core/Grpc.Core.nuspec | 12 +-- src/csharp/Grpc.Core/Grpc.Core.xproj | 19 ++++ .../Internal/DefaultSslRootsOverride.cs | 2 +- .../Grpc.Core/Internal/NativeExtension.cs | 2 +- src/csharp/Grpc.Core/Internal/PlatformApis.cs | 2 +- .../Internal/SafeHandleZeroIsInvalid.cs | 2 +- src/csharp/Grpc.Core/project.json | 56 +++++++++++ src/csharp/Grpc.Dnx.sln | 94 +++++++++++++++++++ .../Grpc.Examples.MathClient.xproj | 19 ++++ .../Grpc.Examples.MathClient/project.json | 26 +++++ .../Grpc.Examples.MathServer.xproj | 19 ++++ .../Grpc.Examples.MathServer/project.json | 26 +++++ .../Grpc.Examples.Tests.xproj | 19 ++++ .../MathClientServerTests.cs | 8 +- src/csharp/Grpc.Examples.Tests/project.json | 28 ++++++ src/csharp/Grpc.Examples/Grpc.Examples.xproj | 19 ++++ src/csharp/Grpc.Examples/project.json | 24 +++++ .../Grpc.HealthCheck.Tests.xproj | 19 ++++ .../Grpc.HealthCheck.Tests/project.json | 28 ++++++ .../Grpc.HealthCheck/Grpc.HealthCheck.xproj | 19 ++++ src/csharp/Grpc.HealthCheck/project.json | 32 +++++++ .../Grpc.IntegrationTesting.Client.xproj | 19 ++++ .../project.json | 15 +++ .../Grpc.IntegrationTesting.QpsWorker.xproj | 19 ++++ .../project.json | 15 +++ .../Grpc.IntegrationTesting.Server.xproj | 19 ++++ .../project.json | 15 +++ .../Grpc.IntegrationTesting.xproj | 19 ++++ .../Grpc.IntegrationTesting/project.json | 21 +++++ src/csharp/build_packages.bat | 12 +-- .../grpc.native.csharp.nuspec | 27 ------ .../src/csharp/build_packages.bat.template | 12 +-- 40 files changed, 722 insertions(+), 55 deletions(-) create mode 100644 src/csharp/Grpc.Auth/Grpc.Auth.xproj create mode 100644 src/csharp/Grpc.Auth/project.json create mode 100644 src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.xproj create mode 100644 src/csharp/Grpc.Core.Tests/project.json create mode 100644 src/csharp/Grpc.Core/Grpc.Core.xproj create mode 100644 src/csharp/Grpc.Core/project.json create mode 100644 src/csharp/Grpc.Dnx.sln create mode 100644 src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.xproj create mode 100644 src/csharp/Grpc.Examples.MathClient/project.json create mode 100644 src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.xproj create mode 100644 src/csharp/Grpc.Examples.MathServer/project.json create mode 100644 src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.xproj create mode 100644 src/csharp/Grpc.Examples.Tests/project.json create mode 100644 src/csharp/Grpc.Examples/Grpc.Examples.xproj create mode 100644 src/csharp/Grpc.Examples/project.json create mode 100644 src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.xproj create mode 100644 src/csharp/Grpc.HealthCheck.Tests/project.json create mode 100644 src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.xproj create mode 100644 src/csharp/Grpc.HealthCheck/project.json create mode 100644 src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.xproj create mode 100644 src/csharp/Grpc.IntegrationTesting.Client/project.json create mode 100644 src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.xproj create mode 100644 src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json create mode 100644 src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.xproj create mode 100644 src/csharp/Grpc.IntegrationTesting.Server/project.json create mode 100644 src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.xproj create mode 100644 src/csharp/Grpc.IntegrationTesting/project.json delete mode 100644 src/csharp/grpc.native.csharp/grpc.native.csharp.nuspec diff --git a/src/csharp/.gitignore b/src/csharp/.gitignore index 0f96a482219..fc2875a1dd5 100644 --- a/src/csharp/.gitignore +++ b/src/csharp/.gitignore @@ -1,5 +1,7 @@ +*.xproj.user *.userprefs *.csproj.user +*.lock.json StyleCop.Cache test-results packages diff --git a/src/csharp/Grpc.Auth/Grpc.Auth.xproj b/src/csharp/Grpc.Auth/Grpc.Auth.xproj new file mode 100644 index 00000000000..c3a6fa2947b --- /dev/null +++ b/src/csharp/Grpc.Auth/Grpc.Auth.xproj @@ -0,0 +1,19 @@ + + + + 14.0.25123 + $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) + + + + c82631ed-06d1-4458-87bc-8257d12307a8 + Grpc.Auth + ..\Grpc.Core\artifacts\obj\$(MSBuildProjectName) + ..\Grpc.Core\artifacts\bin\$(MSBuildProjectName)\ + + + + 2.0 + + + \ No newline at end of file diff --git a/src/csharp/Grpc.Auth/project.json b/src/csharp/Grpc.Auth/project.json new file mode 100644 index 00000000000..513325f7491 --- /dev/null +++ b/src/csharp/Grpc.Auth/project.json @@ -0,0 +1,35 @@ +{ + "version": "0.14.0-anexperiment", + + "title": "gRPC C# Auth", + "summary": "Auth library for C# implementation of gRPC - an RPC library and framework", + "description": "Auth library for C# implementation of gRPC - an RPC library and framework. See project site for more info.", + "authors": ["Google Inc."], + "owners": ["grpc-packages"], + "licenseUrl": "https://github.com/grpc/grpc/blob/master/LICENSE", + "projectUrl": "https://github.com/grpc/grpc", + "requireLicenseAcceptance": false, + "copyright": "Copyright 2015, Google Inc.", + "tags": ["gRPC RPC Protocol HTTP/2 Auth OAuth2"], + + "compile": "**/*.cs", + "resourceFiles": ["../../../etc/roots.pem"], + + "dependencies": { + "Grpc.Core": "0.14.0-anexperiment", + "Google.Apis.Auth": "1.11.1" + }, + "frameworks": { + "net45": { }, + "dotnet54": { + "imports": [ + "net45" + ], + "dependencies": { + "Microsoft.CSharp": "4.0.1-beta-23516", + "Microsoft.NETCore.Portable.Compatibility": "1.0.1-beta-23516", + "System.Threading.Tasks": "4.0.11-beta-23516" + } + } + } +} diff --git a/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.xproj b/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.xproj new file mode 100644 index 00000000000..e6595118fb1 --- /dev/null +++ b/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.xproj @@ -0,0 +1,19 @@ + + + + 14.0.25123 + $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) + + + + 759e23b2-fc04-4695-902d-b073cded3599 + Grpc.Core.Tests + ..\artifacts\obj\$(MSBuildProjectName) + ..\artifacts\bin\$(MSBuildProjectName)\ + + + + 2.0 + + + \ No newline at end of file diff --git a/src/csharp/Grpc.Core.Tests/NUnitVersionTest.cs b/src/csharp/Grpc.Core.Tests/NUnitVersionTest.cs index 3fa6ad09c02..1a9e441611d 100644 --- a/src/csharp/Grpc.Core.Tests/NUnitVersionTest.cs +++ b/src/csharp/Grpc.Core.Tests/NUnitVersionTest.cs @@ -56,7 +56,7 @@ namespace Grpc.Core.Tests Console.Error.WriteLine("You are using and old version of NUnit that doesn't support async tests and skips them instead. " + "This test has failed to indicate that."); Console.Error.Flush(); - Environment.Exit(1); + throw new Exception("NUnitVersionTest has failed."); } } diff --git a/src/csharp/Grpc.Core.Tests/SanityTest.cs b/src/csharp/Grpc.Core.Tests/SanityTest.cs index 3830f0cbacf..9e995d40c03 100644 --- a/src/csharp/Grpc.Core.Tests/SanityTest.cs +++ b/src/csharp/Grpc.Core.Tests/SanityTest.cs @@ -45,6 +45,7 @@ namespace Grpc.Core.Tests { public class SanityTest { +#if !DOTNET5_4 ///

/// Because we depend on a native library, sometimes when things go wrong, the /// entire NUnit test process crashes. To be able to track down problems better, @@ -121,5 +122,6 @@ namespace Grpc.Core.Tests } return result; } +#endif } } diff --git a/src/csharp/Grpc.Core.Tests/project.json b/src/csharp/Grpc.Core.Tests/project.json new file mode 100644 index 00000000000..0c5d9354358 --- /dev/null +++ b/src/csharp/Grpc.Core.Tests/project.json @@ -0,0 +1,28 @@ +{ + "compile": "**/*.cs", + "compilationOptions": { + "emitEntryPoint": true + }, + + "content": "../nativelibs/**", + + "dependencies": { + "Grpc.Core": "0.14.0-anexperiment", + "NUnit": "3.2.0", + "NUnitLite": "3.2.0-*" + }, + "frameworks": { + "net45": { }, + "dotnet54": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "Microsoft.NETCore.App": { + "type": "platform", + "version": "1.0.0-rc2-23931" + } + } + } + } +} diff --git a/src/csharp/Grpc.Core/Grpc.Core.csproj b/src/csharp/Grpc.Core/Grpc.Core.csproj index a796911b99f..e460f004473 100644 --- a/src/csharp/Grpc.Core/Grpc.Core.csproj +++ b/src/csharp/Grpc.Core/Grpc.Core.csproj @@ -148,7 +148,7 @@ - Resources\roots.pem + roots.pem \ No newline at end of file diff --git a/src/csharp/Grpc.Core/Grpc.Core.nuspec b/src/csharp/Grpc.Core/Grpc.Core.nuspec index 0ada0049c2a..fa2c1fbff22 100644 --- a/src/csharp/Grpc.Core/Grpc.Core.nuspec +++ b/src/csharp/Grpc.Core/Grpc.Core.nuspec @@ -24,11 +24,11 @@ - - - - - - + + + + + + diff --git a/src/csharp/Grpc.Core/Grpc.Core.xproj b/src/csharp/Grpc.Core/Grpc.Core.xproj new file mode 100644 index 00000000000..f5d1bf2eef5 --- /dev/null +++ b/src/csharp/Grpc.Core/Grpc.Core.xproj @@ -0,0 +1,19 @@ + + + + 14.0 + $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) + + + + dc9908b6-f291-4fc8-a46d-2ea2551790ec + Grpc.Core + ..\artifacts\obj\$(MSBuildProjectName) + ..\artifacts\bin\$(MSBuildProjectName)\ + + + + 2.0 + + + \ No newline at end of file diff --git a/src/csharp/Grpc.Core/Internal/DefaultSslRootsOverride.cs b/src/csharp/Grpc.Core/Internal/DefaultSslRootsOverride.cs index aa4dafd7f2a..2a96e9920c0 100644 --- a/src/csharp/Grpc.Core/Internal/DefaultSslRootsOverride.cs +++ b/src/csharp/Grpc.Core/Internal/DefaultSslRootsOverride.cs @@ -46,7 +46,7 @@ namespace Grpc.Core.Internal /// internal static class DefaultSslRootsOverride { - const string RootsPemResourceName = "Grpc.Core.Resources.roots.pem"; + const string RootsPemResourceName = "Grpc.Core.roots.pem"; static object staticLock = new object(); /// diff --git a/src/csharp/Grpc.Core/Internal/NativeExtension.cs b/src/csharp/Grpc.Core/Internal/NativeExtension.cs index b45ba19c24d..6c219621df5 100644 --- a/src/csharp/Grpc.Core/Internal/NativeExtension.cs +++ b/src/csharp/Grpc.Core/Internal/NativeExtension.cs @@ -118,7 +118,7 @@ namespace Grpc.Core.Internal { var assembly = typeof(NativeExtension).GetTypeInfo().Assembly; #if DOTNET5_4 - // Assembly.EscapedCodeBase does not exit under CoreCLR, but assemblies imported from a nuget package + // Assembly.EscapedCodeBase does not exist under CoreCLR, but assemblies imported from a nuget package // don't seem to be shadowed by DNX-based projects at all. return assembly.Location; #else diff --git a/src/csharp/Grpc.Core/Internal/PlatformApis.cs b/src/csharp/Grpc.Core/Internal/PlatformApis.cs index 5d8c44b589e..3331a1f9fac 100644 --- a/src/csharp/Grpc.Core/Internal/PlatformApis.cs +++ b/src/csharp/Grpc.Core/Internal/PlatformApis.cs @@ -53,7 +53,7 @@ namespace Grpc.Core.Internal static PlatformApis() { -#if DNXCORE50 +#if DOTNET5_4 isLinux = RuntimeInformation.IsOSPlatform(OSPlatform.Linux); isMacOSX = RuntimeInformation.IsOSPlatform(OSPlatform.OSX); isWindows = RuntimeInformation.IsOSPlatform(OSPlatform.Windows); diff --git a/src/csharp/Grpc.Core/Internal/SafeHandleZeroIsInvalid.cs b/src/csharp/Grpc.Core/Internal/SafeHandleZeroIsInvalid.cs index 702aea2883a..230faacff63 100644 --- a/src/csharp/Grpc.Core/Internal/SafeHandleZeroIsInvalid.cs +++ b/src/csharp/Grpc.Core/Internal/SafeHandleZeroIsInvalid.cs @@ -39,7 +39,7 @@ namespace Grpc.Core.Internal /// /// Safe handle to wrap native objects. /// - internal abstract class SafeHandleZeroIsInvalid : SafeHandle + internal abstract class SafeHandleZeroIsInvalid : System.Runtime.InteropServices.SafeHandle { public SafeHandleZeroIsInvalid() : base(IntPtr.Zero, true) { diff --git a/src/csharp/Grpc.Core/project.json b/src/csharp/Grpc.Core/project.json new file mode 100644 index 00000000000..8aece57856c --- /dev/null +++ b/src/csharp/Grpc.Core/project.json @@ -0,0 +1,56 @@ +{ + "version": "0.14.0-anexperiment", + + "title": "gRPC C# Core", + "summary": "Core C# implementation of gRPC - an RPC library and framework", + "description": "Core C# implementation of gRPC - an RPC library and framework. See project site for more info.", + "authors": [ "Google Inc." ], + "owners": [ "grpc-packages" ], + "licenseUrl": "https://github.com/grpc/grpc/blob/master/LICENSE", + "projectUrl": "https://github.com/grpc/grpc", + "requireLicenseAcceptance": false, + "copyright": "Copyright 2015, Google Inc.", + "tags": [ "gRPC RPC Protocol HTTP/2" ], + + "compile": "**/*.cs", + "resourceFiles": [ "../../../etc/roots.pem" ], + + "content": "../nativelibs/**", + + "packInclude": { + "build/net45/": "Grpc.Core.targets", + "build/native/bin/windows_x86/": "../nativelibs/windows_x86/grpc_csharp_ext.dll", + "build/native/bin/windows_x64/": "../nativelibs/windows_x64/grpc_csharp_ext.dll", + "build/native/bin/linux_x86/": "../nativelibs/linux_x86/libgrpc_csharp_ext.so", + "build/native/bin/linux_x64/": "../nativelibs/linux_x64/libgrpc_csharp_ext.so", + "build/native/bin/macosx_x86/": "../nativelibs/macosx_x86/libgrpc_csharp_ext.dylib", + "build/native/bin/macosx_x64/": "../nativelibs/macosx_x64/libgrpc_csharp_ext.dylib" + }, + + "dependencies": { + "Ix-Async": "1.2.5" + }, + "frameworks": { + "net45": { }, + "dotnet54": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "Microsoft.CSharp": "4.0.1-beta-23516", + "System.Collections": "4.0.11-beta-23516", + "System.Collections.Concurrent": "4.0.11-beta-23516", + "System.Console": "4.0.0-beta-23516", + "System.Linq": "4.0.1-beta-23516", + "System.Threading": "4.0.11-beta-23516", + "System.Threading.Thread": "4.0.0-beta-23516", + "System.Reflection": "4.1.0-beta-23516", + "System.Text.Encoding": "4.0.11-beta-23516", + "System.Text.RegularExpressions": "4.0.11-beta-23516", + "System.IO": "4.0.11-beta-23516", + "System.IO.FileSystem": "4.0.1-beta-23516", + "System.Runtime.InteropServices.RuntimeInformation": "4.0.0-beta-23516" + } + } + } +} diff --git a/src/csharp/Grpc.Dnx.sln b/src/csharp/Grpc.Dnx.sln new file mode 100644 index 00000000000..6a7e2e27482 --- /dev/null +++ b/src/csharp/Grpc.Dnx.sln @@ -0,0 +1,94 @@ + +Microsoft Visual Studio Solution File, Format Version 12.00 +# Visual Studio 14 +VisualStudioVersion = 14.0.25123.0 +MinimumVisualStudioVersion = 10.0.40219.1 +Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.Core", "Grpc.Core\Grpc.Core.xproj", "{DC9908B6-F291-4FC8-A46D-2EA2551790EC}" +EndProject +Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.Auth", "Grpc.Auth\Grpc.Auth.xproj", "{C82631ED-06D1-4458-87BC-8257D12307A8}" +EndProject +Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.Core.Tests", "Grpc.Core.Tests\Grpc.Core.Tests.xproj", "{759E23B2-FC04-4695-902D-B073CDED3599}" +EndProject +Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.Examples", "Grpc.Examples\Grpc.Examples.xproj", "{C77B792D-FC78-4CE2-9522-B40B0803C636}" +EndProject +Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.Examples.MathClient", "Grpc.Examples.MathClient\Grpc.Examples.MathClient.xproj", "{FD48DECA-1622-4173-B1D9-2101CF5E7C5F}" +EndProject +Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.Examples.MathServer", "Grpc.Examples.MathServer\Grpc.Examples.MathServer.xproj", "{58579368-5372-4E67-ACD6-9B59CB9FA698}" +EndProject +Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.Examples.Tests", "Grpc.Examples.Tests\Grpc.Examples.Tests.xproj", "{C61714A6-F633-44FB-97F4-C91F425C1D15}" +EndProject +Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.HealthCheck", "Grpc.HealthCheck\Grpc.HealthCheck.xproj", "{3BE4AD0B-2BF0-4D68-B625-F6018EF0DCFA}" +EndProject +Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.HealthCheck.Tests", "Grpc.HealthCheck.Tests\Grpc.HealthCheck.Tests.xproj", "{43DAFAC6-5343-4621-960E-A8A977EA3F0B}" +EndProject +Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.IntegrationTesting", "Grpc.IntegrationTesting\Grpc.IntegrationTesting.xproj", "{20354386-3E71-4046-A269-3BC2A06F3EC8}" +EndProject +Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.IntegrationTesting.Client", "Grpc.IntegrationTesting.Client\Grpc.IntegrationTesting.Client.xproj", "{48EA5BBE-70E2-4198-869D-D7E59C45F30D}" +EndProject +Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.IntegrationTesting.QpsWorker", "Grpc.IntegrationTesting.QpsWorker\Grpc.IntegrationTesting.QpsWorker.xproj", "{661B70D7-F56A-46E0-9B81-6227B591B5E7}" +EndProject +Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.IntegrationTesting.Server", "Grpc.IntegrationTesting.Server\Grpc.IntegrationTesting.Server.xproj", "{881F7AD1-A84E-47A2-9402-115C63C4031E}" +EndProject +Global + GlobalSection(SolutionConfigurationPlatforms) = preSolution + Debug|Any CPU = Debug|Any CPU + Release|Any CPU = Release|Any CPU + EndGlobalSection + GlobalSection(ProjectConfigurationPlatforms) = postSolution + {DC9908B6-F291-4FC8-A46D-2EA2551790EC}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {DC9908B6-F291-4FC8-A46D-2EA2551790EC}.Debug|Any CPU.Build.0 = Debug|Any CPU + {DC9908B6-F291-4FC8-A46D-2EA2551790EC}.Release|Any CPU.ActiveCfg = Release|Any CPU + {DC9908B6-F291-4FC8-A46D-2EA2551790EC}.Release|Any CPU.Build.0 = Release|Any CPU + {C82631ED-06D1-4458-87BC-8257D12307A8}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {C82631ED-06D1-4458-87BC-8257D12307A8}.Debug|Any CPU.Build.0 = Debug|Any CPU + {C82631ED-06D1-4458-87BC-8257D12307A8}.Release|Any CPU.ActiveCfg = Release|Any CPU + {C82631ED-06D1-4458-87BC-8257D12307A8}.Release|Any CPU.Build.0 = Release|Any CPU + {759E23B2-FC04-4695-902D-B073CDED3599}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {759E23B2-FC04-4695-902D-B073CDED3599}.Debug|Any CPU.Build.0 = Debug|Any CPU + {759E23B2-FC04-4695-902D-B073CDED3599}.Release|Any CPU.ActiveCfg = Release|Any CPU + {759E23B2-FC04-4695-902D-B073CDED3599}.Release|Any CPU.Build.0 = Release|Any CPU + {C77B792D-FC78-4CE2-9522-B40B0803C636}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {C77B792D-FC78-4CE2-9522-B40B0803C636}.Debug|Any CPU.Build.0 = Debug|Any CPU + {C77B792D-FC78-4CE2-9522-B40B0803C636}.Release|Any CPU.ActiveCfg = Release|Any CPU + {C77B792D-FC78-4CE2-9522-B40B0803C636}.Release|Any CPU.Build.0 = Release|Any CPU + {FD48DECA-1622-4173-B1D9-2101CF5E7C5F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {FD48DECA-1622-4173-B1D9-2101CF5E7C5F}.Debug|Any CPU.Build.0 = Debug|Any CPU + {FD48DECA-1622-4173-B1D9-2101CF5E7C5F}.Release|Any CPU.ActiveCfg = Release|Any CPU + {FD48DECA-1622-4173-B1D9-2101CF5E7C5F}.Release|Any CPU.Build.0 = Release|Any CPU + {58579368-5372-4E67-ACD6-9B59CB9FA698}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {58579368-5372-4E67-ACD6-9B59CB9FA698}.Debug|Any CPU.Build.0 = Debug|Any CPU + {58579368-5372-4E67-ACD6-9B59CB9FA698}.Release|Any CPU.ActiveCfg = Release|Any CPU + {58579368-5372-4E67-ACD6-9B59CB9FA698}.Release|Any CPU.Build.0 = Release|Any CPU + {C61714A6-F633-44FB-97F4-C91F425C1D15}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {C61714A6-F633-44FB-97F4-C91F425C1D15}.Debug|Any CPU.Build.0 = Debug|Any CPU + {C61714A6-F633-44FB-97F4-C91F425C1D15}.Release|Any CPU.ActiveCfg = Release|Any CPU + {C61714A6-F633-44FB-97F4-C91F425C1D15}.Release|Any CPU.Build.0 = Release|Any CPU + {3BE4AD0B-2BF0-4D68-B625-F6018EF0DCFA}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {3BE4AD0B-2BF0-4D68-B625-F6018EF0DCFA}.Debug|Any CPU.Build.0 = Debug|Any CPU + {3BE4AD0B-2BF0-4D68-B625-F6018EF0DCFA}.Release|Any CPU.ActiveCfg = Release|Any CPU + {3BE4AD0B-2BF0-4D68-B625-F6018EF0DCFA}.Release|Any CPU.Build.0 = Release|Any CPU + {43DAFAC6-5343-4621-960E-A8A977EA3F0B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {43DAFAC6-5343-4621-960E-A8A977EA3F0B}.Debug|Any CPU.Build.0 = Debug|Any CPU + {43DAFAC6-5343-4621-960E-A8A977EA3F0B}.Release|Any CPU.ActiveCfg = Release|Any CPU + {43DAFAC6-5343-4621-960E-A8A977EA3F0B}.Release|Any CPU.Build.0 = Release|Any CPU + {20354386-3E71-4046-A269-3BC2A06F3EC8}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {20354386-3E71-4046-A269-3BC2A06F3EC8}.Debug|Any CPU.Build.0 = Debug|Any CPU + {20354386-3E71-4046-A269-3BC2A06F3EC8}.Release|Any CPU.ActiveCfg = Release|Any CPU + {20354386-3E71-4046-A269-3BC2A06F3EC8}.Release|Any CPU.Build.0 = Release|Any CPU + {48EA5BBE-70E2-4198-869D-D7E59C45F30D}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {48EA5BBE-70E2-4198-869D-D7E59C45F30D}.Debug|Any CPU.Build.0 = Debug|Any CPU + {48EA5BBE-70E2-4198-869D-D7E59C45F30D}.Release|Any CPU.ActiveCfg = Release|Any CPU + {48EA5BBE-70E2-4198-869D-D7E59C45F30D}.Release|Any CPU.Build.0 = Release|Any CPU + {661B70D7-F56A-46E0-9B81-6227B591B5E7}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {661B70D7-F56A-46E0-9B81-6227B591B5E7}.Debug|Any CPU.Build.0 = Debug|Any CPU + {661B70D7-F56A-46E0-9B81-6227B591B5E7}.Release|Any CPU.ActiveCfg = Release|Any CPU + {661B70D7-F56A-46E0-9B81-6227B591B5E7}.Release|Any CPU.Build.0 = Release|Any CPU + {881F7AD1-A84E-47A2-9402-115C63C4031E}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {881F7AD1-A84E-47A2-9402-115C63C4031E}.Debug|Any CPU.Build.0 = Debug|Any CPU + {881F7AD1-A84E-47A2-9402-115C63C4031E}.Release|Any CPU.ActiveCfg = Release|Any CPU + {881F7AD1-A84E-47A2-9402-115C63C4031E}.Release|Any CPU.Build.0 = Release|Any CPU + EndGlobalSection + GlobalSection(SolutionProperties) = preSolution + HideSolutionNode = FALSE + EndGlobalSection +EndGlobal diff --git a/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.xproj b/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.xproj new file mode 100644 index 00000000000..a8dc195a6dc --- /dev/null +++ b/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.xproj @@ -0,0 +1,19 @@ + + + + 14.0.25123 + $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) + + + + fd48deca-1622-4173-b1d9-2101cf5e7c5f + Grpc.Examples.MathClient + ..\artifacts\obj\$(MSBuildProjectName) + ..\artifacts\bin\$(MSBuildProjectName)\ + + + + 2.0 + + + \ No newline at end of file diff --git a/src/csharp/Grpc.Examples.MathClient/project.json b/src/csharp/Grpc.Examples.MathClient/project.json new file mode 100644 index 00000000000..67af09721ce --- /dev/null +++ b/src/csharp/Grpc.Examples.MathClient/project.json @@ -0,0 +1,26 @@ +{ + "compile": "**/*.cs", + "compilationOptions": { + "emitEntryPoint": true + }, + + "content": "../nativelibs/**", + + "dependencies": { + "Grpc.Examples": "1.0.0" + }, + "frameworks": { + "net45": { }, + "dotnet54": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "Microsoft.NETCore.App": { + "type": "platform", + "version": "1.0.0-rc2-23931" + } + } + } + } +} diff --git a/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.xproj b/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.xproj new file mode 100644 index 00000000000..40746f1f946 --- /dev/null +++ b/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.xproj @@ -0,0 +1,19 @@ + + + + 14.0.25123 + $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) + + + + 58579368-5372-4e67-acd6-9b59cb9fa698 + Grpc.Examples.MathServer + ..\artifacts\obj\$(MSBuildProjectName) + ..\artifacts\bin\$(MSBuildProjectName)\ + + + + 2.0 + + + \ No newline at end of file diff --git a/src/csharp/Grpc.Examples.MathServer/project.json b/src/csharp/Grpc.Examples.MathServer/project.json new file mode 100644 index 00000000000..67af09721ce --- /dev/null +++ b/src/csharp/Grpc.Examples.MathServer/project.json @@ -0,0 +1,26 @@ +{ + "compile": "**/*.cs", + "compilationOptions": { + "emitEntryPoint": true + }, + + "content": "../nativelibs/**", + + "dependencies": { + "Grpc.Examples": "1.0.0" + }, + "frameworks": { + "net45": { }, + "dotnet54": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "Microsoft.NETCore.App": { + "type": "platform", + "version": "1.0.0-rc2-23931" + } + } + } + } +} diff --git a/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.xproj b/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.xproj new file mode 100644 index 00000000000..c744816a811 --- /dev/null +++ b/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.xproj @@ -0,0 +1,19 @@ + + + + 14.0.25123 + $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) + + + + c61714a6-f633-44fb-97f4-c91f425c1d15 + Grpc.Examples.Tests + ..\artifacts\obj\$(MSBuildProjectName) + ..\artifacts\bin\$(MSBuildProjectName)\ + + + + 2.0 + + + \ No newline at end of file diff --git a/src/csharp/Grpc.Examples.Tests/MathClientServerTests.cs b/src/csharp/Grpc.Examples.Tests/MathClientServerTests.cs index ee11105efe7..324c209ca1d 100644 --- a/src/csharp/Grpc.Examples.Tests/MathClientServerTests.cs +++ b/src/csharp/Grpc.Examples.Tests/MathClientServerTests.cs @@ -110,7 +110,7 @@ namespace Math.Tests { var responses = await call.ResponseStream.ToListAsync(); CollectionAssert.AreEqual(new List { 1, 1, 2, 3, 5, 8 }, - responses.ConvertAll((n) => n.Num_)); + responses.Select((n) => n.Num_)); } } @@ -162,7 +162,7 @@ namespace Math.Tests { using (var call = client.Sum()) { - var numbers = new List { 10, 20, 30 }.ConvertAll(n => new Num { Num_ = n }); + var numbers = new List { 10, 20, 30 }.Select(n => new Num { Num_ = n }); await call.RequestStream.WriteAllAsync(numbers); var result = await call.ResponseAsync; @@ -185,8 +185,8 @@ namespace Math.Tests await call.RequestStream.WriteAllAsync(divArgsList); var result = await call.ResponseStream.ToListAsync(); - CollectionAssert.AreEqual(new long[] { 3, 4, 3 }, result.ConvertAll((divReply) => divReply.Quotient)); - CollectionAssert.AreEqual(new long[] { 1, 16, 1 }, result.ConvertAll((divReply) => divReply.Remainder)); + CollectionAssert.AreEqual(new long[] { 3, 4, 3 }, result.Select((divReply) => divReply.Quotient)); + CollectionAssert.AreEqual(new long[] { 1, 16, 1 }, result.Select((divReply) => divReply.Remainder)); } } } diff --git a/src/csharp/Grpc.Examples.Tests/project.json b/src/csharp/Grpc.Examples.Tests/project.json new file mode 100644 index 00000000000..e61aa65abfc --- /dev/null +++ b/src/csharp/Grpc.Examples.Tests/project.json @@ -0,0 +1,28 @@ +{ + "compile": "**/*.cs", + "compilationOptions": { + "emitEntryPoint": true + }, + + "content": "../nativelibs/**", + + "dependencies": { + "Grpc.Examples": "1.0.0", + "NUnit": "3.2.0", + "NUnitLite": "3.2.0-*" + }, + "frameworks": { + "net45": { }, + "dotnet54": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "Microsoft.NETCore.App": { + "type": "platform", + "version": "1.0.0-rc2-23931" + } + } + } + } +} diff --git a/src/csharp/Grpc.Examples/Grpc.Examples.xproj b/src/csharp/Grpc.Examples/Grpc.Examples.xproj new file mode 100644 index 00000000000..aa3ad680fd8 --- /dev/null +++ b/src/csharp/Grpc.Examples/Grpc.Examples.xproj @@ -0,0 +1,19 @@ + + + + 14.0.25123 + $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) + + + + c77b792d-fc78-4ce2-9522-b40b0803c636 + Grpc.Examples + ..\artifacts\obj\$(MSBuildProjectName) + ..\artifacts\bin\$(MSBuildProjectName)\ + + + + 2.0 + + + \ No newline at end of file diff --git a/src/csharp/Grpc.Examples/project.json b/src/csharp/Grpc.Examples/project.json new file mode 100644 index 00000000000..c78de89cf89 --- /dev/null +++ b/src/csharp/Grpc.Examples/project.json @@ -0,0 +1,24 @@ +{ + "compile": "**/*.cs", + + "content": "../nativelibs/**", + + "dependencies": { + "Grpc.Core": "0.0.1", + "Google.Protobuf": "3.0.0-beta2" + }, + "frameworks": { + "net45": { }, + "dotnet54": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "Microsoft.NETCore.App": { + "type": "platform", + "version": "1.0.0-rc2-23931" + } + } + } + } +} diff --git a/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.xproj b/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.xproj new file mode 100644 index 00000000000..db05ccf7a4f --- /dev/null +++ b/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.xproj @@ -0,0 +1,19 @@ + + + + 14.0.25123 + $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) + + + + 43dafac6-5343-4621-960e-a8a977ea3f0b + Grpc.HealthCheck.Tests + ..\artifacts\obj\$(MSBuildProjectName) + ..\artifacts\bin\$(MSBuildProjectName)\ + + + + 2.0 + + + \ No newline at end of file diff --git a/src/csharp/Grpc.HealthCheck.Tests/project.json b/src/csharp/Grpc.HealthCheck.Tests/project.json new file mode 100644 index 00000000000..5ecb17c9671 --- /dev/null +++ b/src/csharp/Grpc.HealthCheck.Tests/project.json @@ -0,0 +1,28 @@ +{ + "compile": "**/*.cs", + "compilationOptions": { + "emitEntryPoint": true + }, + + "content": "../nativelibs/**", + + "dependencies": { + "Grpc.HealthCheck": "1.0.0", + "NUnit": "3.2.0", + "NUnitLite": "3.2.0-*" + }, + "frameworks": { + "net45": { }, + "dotnet54": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "Microsoft.NETCore.App": { + "type": "platform", + "version": "1.0.0-rc2-23931" + } + } + } + } +} diff --git a/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.xproj b/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.xproj new file mode 100644 index 00000000000..18701779d19 --- /dev/null +++ b/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.xproj @@ -0,0 +1,19 @@ + + + + 14.0.25123 + $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) + + + + 3be4ad0b-2bf0-4d68-b625-f6018ef0dcfa + Grpc.HealthCheck + ..\artifacts\obj\$(MSBuildProjectName) + ..\artifacts\bin\$(MSBuildProjectName)\ + + + + 2.0 + + + \ No newline at end of file diff --git a/src/csharp/Grpc.HealthCheck/project.json b/src/csharp/Grpc.HealthCheck/project.json new file mode 100644 index 00000000000..0984d43deca --- /dev/null +++ b/src/csharp/Grpc.HealthCheck/project.json @@ -0,0 +1,32 @@ +{ + "version": "0.14.0-anexperiment", + + "title": "gRPC C# Healthchecking", + "summary": "Implementation of gRPC health service", + "description": "Example implementation of grpc.health.v1 service that can be used for health-checking.", + "authors": [ "Google Inc." ], + "owners": [ "grpc-packages" ], + "licenseUrl": "https://github.com/grpc/grpc/blob/master/LICENSE", + "projectUrl": "https://github.com/grpc/grpc", + "requireLicenseAcceptance": false, + "copyright": "Copyright 2015, Google Inc.", + "tags": [ "gRPC health check" ], + + "compile": "**/*.cs", + + "dependencies": { + "Grpc.Core": "0.14.0-anexperiment", + "Google.Protobuf": "3.0.0-beta2" + }, + "frameworks": { + "net45": { }, + "dotnet54": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "Microsoft.CSharp": "4.0.1-beta-23516" + } + } + } +} diff --git a/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.xproj b/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.xproj new file mode 100644 index 00000000000..82e044ec212 --- /dev/null +++ b/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.xproj @@ -0,0 +1,19 @@ + + + + 14.0.25123 + $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) + + + + 48ea5bbe-70e2-4198-869d-d7e59c45f30d + Grpc.IntegrationTesting.Client + ..\artifacts\obj\$(MSBuildProjectName) + ..\artifacts\bin\$(MSBuildProjectName)\ + + + + 2.0 + + + \ No newline at end of file diff --git a/src/csharp/Grpc.IntegrationTesting.Client/project.json b/src/csharp/Grpc.IntegrationTesting.Client/project.json new file mode 100644 index 00000000000..e023a9815b1 --- /dev/null +++ b/src/csharp/Grpc.IntegrationTesting.Client/project.json @@ -0,0 +1,15 @@ +{ + "compile": "**/*.cs", + "compilationOptions": { + "emitEntryPoint": true + }, + + "content": "../nativelibs/**", + + "dependencies": { + "Grpc.IntegrationTesting": "1.0.0" + }, + "frameworks": { + "net45": { } + } +} diff --git a/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.xproj b/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.xproj new file mode 100644 index 00000000000..7348004c982 --- /dev/null +++ b/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.xproj @@ -0,0 +1,19 @@ + + + + 14.0.25123 + $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) + + + + 661b70d7-f56a-46e0-9b81-6227b591b5e7 + Grpc.IntegrationTesting.QpsWorker + ..\artifacts\obj\$(MSBuildProjectName) + ..\artifacts\bin\$(MSBuildProjectName)\ + + + + 2.0 + + + \ No newline at end of file diff --git a/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json b/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json new file mode 100644 index 00000000000..e023a9815b1 --- /dev/null +++ b/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json @@ -0,0 +1,15 @@ +{ + "compile": "**/*.cs", + "compilationOptions": { + "emitEntryPoint": true + }, + + "content": "../nativelibs/**", + + "dependencies": { + "Grpc.IntegrationTesting": "1.0.0" + }, + "frameworks": { + "net45": { } + } +} diff --git a/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.xproj b/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.xproj new file mode 100644 index 00000000000..fc43c41fd89 --- /dev/null +++ b/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.xproj @@ -0,0 +1,19 @@ + + + + 14.0.25123 + $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) + + + + 881f7ad1-a84e-47a2-9402-115c63c4031e + Grpc.IntegrationTesting.Server + ..\artifacts\obj\$(MSBuildProjectName) + ..\artifacts\bin\$(MSBuildProjectName)\ + + + + 2.0 + + + \ No newline at end of file diff --git a/src/csharp/Grpc.IntegrationTesting.Server/project.json b/src/csharp/Grpc.IntegrationTesting.Server/project.json new file mode 100644 index 00000000000..e023a9815b1 --- /dev/null +++ b/src/csharp/Grpc.IntegrationTesting.Server/project.json @@ -0,0 +1,15 @@ +{ + "compile": "**/*.cs", + "compilationOptions": { + "emitEntryPoint": true + }, + + "content": "../nativelibs/**", + + "dependencies": { + "Grpc.IntegrationTesting": "1.0.0" + }, + "frameworks": { + "net45": { } + } +} diff --git a/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.xproj b/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.xproj new file mode 100644 index 00000000000..69e89fa5770 --- /dev/null +++ b/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.xproj @@ -0,0 +1,19 @@ + + + + 14.0.25123 + $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) + + + + 20354386-3e71-4046-a269-3bc2a06f3ec8 + Grpc.IntegrationTesting + ..\artifacts\obj\$(MSBuildProjectName) + ..\artifacts\bin\$(MSBuildProjectName)\ + + + + 2.0 + + + \ No newline at end of file diff --git a/src/csharp/Grpc.IntegrationTesting/project.json b/src/csharp/Grpc.IntegrationTesting/project.json new file mode 100644 index 00000000000..f2cda96e7be --- /dev/null +++ b/src/csharp/Grpc.IntegrationTesting/project.json @@ -0,0 +1,21 @@ +{ + "compile": "**/*.cs", + "compilationOptions": { + "emitEntryPoint": true + }, + + "content": "../nativelibs/**", + + "dependencies": { + "Grpc.Auth": "0.0.1", + "Grpc.Core": "0.0.1", + "Google.Protobuf": "3.0.0-beta2", + "CommandLineParser": "1.9.71", + "Moq": "4.2.1510.2205", + "NUnit": "3.2.0", + "NUnitLite": "3.2.0-*" + }, + "frameworks": { + "net45": { } + } +} diff --git a/src/csharp/build_packages.bat b/src/csharp/build_packages.bat index 1cc63da9703..63f8c30bc7e 100644 --- a/src/csharp/build_packages.bat +++ b/src/csharp/build_packages.bat @@ -41,12 +41,12 @@ set NUGET=C:\nuget\nuget.exe @rem Collect the artifacts built by the previous build step if running on Jenkins @rem TODO(jtattermusch): is there a better way to do this? -xcopy /Y /I ..\..\architecture=x86,language=csharp,platform=windows\artifacts\* Grpc.Core\windows_x86\ -xcopy /Y /I ..\..\architecture=x64,language=csharp,platform=windows\artifacts\* Grpc.Core\windows_x64\ -xcopy /Y /I ..\..\architecture=x86,language=csharp,platform=linux\artifacts\* Grpc.Core\linux_x86\ -xcopy /Y /I ..\..\architecture=x64,language=csharp,platform=linux\artifacts\* Grpc.Core\linux_x64\ -xcopy /Y /I ..\..\architecture=x86,language=csharp,platform=macos\artifacts\* Grpc.Core\macosx_x86\ -xcopy /Y /I ..\..\architecture=x64,language=csharp,platform=macos\artifacts\* Grpc.Core\macosx_x64\ +xcopy /Y /I ..\..\architecture=x86,language=csharp,platform=windows\artifacts\* nativelibs\windows_x86\ +xcopy /Y /I ..\..\architecture=x64,language=csharp,platform=windows\artifacts\* nativelibs\windows_x64\ +xcopy /Y /I ..\..\architecture=x86,language=csharp,platform=linux\artifacts\* nativelibs\linux_x86\ +xcopy /Y /I ..\..\architecture=x64,language=csharp,platform=linux\artifacts\* nativelibs\linux_x64\ +xcopy /Y /I ..\..\architecture=x86,language=csharp,platform=macos\artifacts\* nativelibs\macosx_x86\ +xcopy /Y /I ..\..\architecture=x64,language=csharp,platform=macos\artifacts\* nativelibs\macosx_x64\ @rem Collect protoc artifacts built by the previous build step xcopy /Y /I ..\..\architecture=x86,language=protoc,platform=windows\artifacts\* protoc_plugins\windows_x86\ diff --git a/src/csharp/grpc.native.csharp/grpc.native.csharp.nuspec b/src/csharp/grpc.native.csharp/grpc.native.csharp.nuspec deleted file mode 100644 index cc688e2bc71..00000000000 --- a/src/csharp/grpc.native.csharp/grpc.native.csharp.nuspec +++ /dev/null @@ -1,27 +0,0 @@ - - - - grpc.native.csharp - $version$ - Google Inc. - grpc-packages - https://github.com/grpc/grpc/blob/master/LICENSE - http://github.com/grpc/grpc - false - Native extension needed by gRPC C# library. This is not the package you are looking for, it is only meant to be used as a dependency. - Release of gRPC C core $version$ libraries. - Copyright 2015 - gRPC C# Native Extension - Native library required by gRPC C# - gRPC native - - - - - - - - - - - diff --git a/templates/src/csharp/build_packages.bat.template b/templates/src/csharp/build_packages.bat.template index 122435af2e8..ea2acb661ee 100644 --- a/templates/src/csharp/build_packages.bat.template +++ b/templates/src/csharp/build_packages.bat.template @@ -43,12 +43,12 @@ @rem Collect the artifacts built by the previous build step if running on Jenkins @rem TODO(jtattermusch): is there a better way to do this? - xcopy /Y /I ..\..\architecture=x86,language=csharp,platform=windows\artifacts\* Grpc.Core\windows_x86${"\\"} - xcopy /Y /I ..\..\architecture=x64,language=csharp,platform=windows\artifacts\* Grpc.Core\windows_x64${"\\"} - xcopy /Y /I ..\..\architecture=x86,language=csharp,platform=linux\artifacts\* Grpc.Core\linux_x86${"\\"} - xcopy /Y /I ..\..\architecture=x64,language=csharp,platform=linux\artifacts\* Grpc.Core\linux_x64${"\\"} - xcopy /Y /I ..\..\architecture=x86,language=csharp,platform=macos\artifacts\* Grpc.Core\macosx_x86${"\\"} - xcopy /Y /I ..\..\architecture=x64,language=csharp,platform=macos\artifacts\* Grpc.Core\macosx_x64${"\\"} + xcopy /Y /I ..\..\architecture=x86,language=csharp,platform=windows\artifacts\* nativelibs\windows_x86${"\\"} + xcopy /Y /I ..\..\architecture=x64,language=csharp,platform=windows\artifacts\* nativelibs\windows_x64${"\\"} + xcopy /Y /I ..\..\architecture=x86,language=csharp,platform=linux\artifacts\* nativelibs\linux_x86${"\\"} + xcopy /Y /I ..\..\architecture=x64,language=csharp,platform=linux\artifacts\* nativelibs\linux_x64${"\\"} + xcopy /Y /I ..\..\architecture=x86,language=csharp,platform=macos\artifacts\* nativelibs\macosx_x86${"\\"} + xcopy /Y /I ..\..\architecture=x64,language=csharp,platform=macos\artifacts\* nativelibs\macosx_x64${"\\"} @rem Collect protoc artifacts built by the previous build step xcopy /Y /I ..\..\architecture=x86,language=protoc,platform=windows\artifacts\* protoc_plugins\windows_x86${"\\"} From 88651de8a713b068eaf499d36bf0b67c0cc9e8e3 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Tue, 14 Jun 2016 16:30:58 -0700 Subject: [PATCH 063/280] Fixes #2646 Pass NULL in the host parameter of grpc_channel_create_call --- src/objective-c/GRPCClient/private/GRPCChannel.m | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/src/objective-c/GRPCClient/private/GRPCChannel.m b/src/objective-c/GRPCClient/private/GRPCChannel.m index d3192c983d1..6cd4cb0ca30 100644 --- a/src/objective-c/GRPCClient/private/GRPCChannel.m +++ b/src/objective-c/GRPCClient/private/GRPCChannel.m @@ -199,9 +199,7 @@ grpc_channel_args * buildChannelArgs(NSDictionary *dictionary) { NULL, GRPC_PROPAGATE_DEFAULTS, queue.unmanagedQueue, path.UTF8String, - // Get "host" from "host:port" - // TODO(jcanizales): Use NSURLs throughout, to clarify these. - [_host componentsSeparatedByString:@":"][0].UTF8String, + NULL, gpr_inf_future(GPR_CLOCK_REALTIME), NULL); } From 350c1517819c883f8a31b75f243e2a699d8b57a4 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 14 Jun 2016 17:08:27 -0700 Subject: [PATCH 064/280] pr comments --- doc/interop-test-descriptions.md | 39 ++++++++++----------------- src/proto/grpc/testing/messages.proto | 25 ++++++++++++----- 2 files changed, 32 insertions(+), 32 deletions(-) diff --git a/doc/interop-test-descriptions.md b/doc/interop-test-descriptions.md index 3dd7807dec0..480625c6ba7 100644 --- a/doc/interop-test-descriptions.md +++ b/doc/interop-test-descriptions.md @@ -89,11 +89,10 @@ Client asserts: ### client_compressed_unary -This test verifies the client can compress unary messages. It sends an initial -inconsistent request to verify whether the server supports the -[CompressedRequest][] feature. If it does, it should catch the inconsistency and -fail the call with an `INVALID_ARGUMENT` status. If the feature is supported, it -proceeds with two unary calls, for compressed and uncompressed payloads. +This test verifies the client can compress unary messages by sending two unary +calls, for compressed and uncompressed payloads. It sends an initial probing +request to verify whether the server supports the [CompressedRequest][] feature. +If it doesn't, the call is expected to fail with an `INVALID_ARGUMENT` status. Server features: * [UnaryCall][] @@ -140,14 +139,14 @@ Procedure: calls were successful. * Response payload body is 314159 bytes in size. * Clients are free to assert that the response payload body contents are - zero and comparing the entire response message against a golden response. + zeros and comparing the entire response message against a golden response. ### server_compressed_unary This test verifies the server can compress unary messages. It sends two unary -requests, expecting the server response to be -compressed or not according to the `response_compressed` boolean. +requests, expecting the server's response to be compressed or not according to +the `response_compressed` boolean. Whether compression was actually performed is determined by the compression bit in the response's message flags. @@ -158,7 +157,7 @@ Server features: * [CompressedResponse][] Procedure: - 1. Client calls UnaryCall with: + 1. Client calls UnaryCall with `SimpleRequest`s: ``` { @@ -257,7 +256,8 @@ compressed with `expect_compressed` true; the second one uncompressed with Procedure: 1. Client calls StreamingInputCall - 1. Client sends the following feature-probing *uncompressed* message + 1. Client sends the following feature-probing *uncompressed* + `StreamingInputCallRequest` message ``` { @@ -305,7 +305,7 @@ Server features: * [StreamingOutputCall][] Procedure: - 1. Client calls StreamingOutputCall with: + 1. Client calls StreamingOutputCall with `StreamingOutputCallRequest`: ``` { @@ -341,27 +341,16 @@ Server features: Procedure: - 1. Client calls StreamingOutputCall with: - - ``` - { - response_compressed: true - response_parameters:{ - size: 31415 - } - response_parameters:{ - size: 58979 - } - } - ``` + 1. Client calls StreamingOutputCall with `StreamingOutputCallRequest`: ``` { - response_compressed: false response_parameters:{ + response_compressed: true size: 31415 } response_parameters:{ + response_compressed: false size: 58979 } } diff --git a/src/proto/grpc/testing/messages.proto b/src/proto/grpc/testing/messages.proto index 782fd40989d..1812dc74029 100644 --- a/src/proto/grpc/testing/messages.proto +++ b/src/proto/grpc/testing/messages.proto @@ -32,6 +32,8 @@ syntax = "proto3"; +import "google/protobuf/wrappers.proto"; + package grpc.testing; // DEPRECATED, don't use. To be removed shortly. @@ -76,8 +78,11 @@ message SimpleRequest { // Whether SimpleResponse should include OAuth scope. bool fill_oauth_scope = 5; - // Whether to request the server to compress the response. - bool response_compressed = 6; + // Whether to request the server to compress the response. This field is + // "nullable" in order to interoperate seamlessly with clients not able to + // implement the full compression tests by introspecting the call to verify + // the response's compression status. + google.protobuf.BoolValue response_compressed = 6; // Whether server should return a given status EchoStatus response_status = 7; @@ -102,8 +107,11 @@ message StreamingInputCallRequest { // Optional input payload sent along with the request. Payload payload = 1; - // Whether the server should expect this request to be compressed. - bool expect_compressed = 2; + // Whether the server should expect this request to be compressed. This field + // is "nullable" in order to interoperate seamlessly with servers not able to + // implement the full compression tests by introspecting the call to verify + // the request's compression status. + BoolValue expect_compressed = 2; // Not expecting any payload from the response. } @@ -122,6 +130,12 @@ message ResponseParameters { // Desired interval between consecutive responses in the response stream in // microseconds. int32 interval_us = 2; + + // Whether to request the server to compress the response. This field is + // "nullable" in order to interoperate seamlessly with clients not able to + // implement the full compression tests by introspecting the call to verify + // the response's compression status. + BoolValue compressed = 3; } // Server-streaming request. @@ -139,9 +153,6 @@ message StreamingOutputCallRequest { // Optional input payload sent along with the request. Payload payload = 3; - // Whether to request the server to compress the response. - bool response_compressed = 6; - // Whether server should return a given status EchoStatus response_status = 7; } From 8d9dff511811bd8e0c92bd56fd682db14033424b Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 15 Jun 2016 09:35:39 -0700 Subject: [PATCH 065/280] prevent interfering of project.json files with .csproj files in VS2015 --- src/csharp/Grpc.Auth/Grpc.Auth.csproj | 1 + src/csharp/Grpc.Auth/Grpc.Auth.project.json | 8 ++++++++ src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.csproj | 1 + src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.project.json | 8 ++++++++ src/csharp/Grpc.Core/Grpc.Core.csproj | 1 + src/csharp/Grpc.Core/Grpc.Core.project.json | 8 ++++++++ .../Grpc.Examples.MathClient.csproj | 3 +++ .../Grpc.Examples.MathClient.project.json | 8 ++++++++ .../Grpc.Examples.MathServer.csproj | 3 +++ .../Grpc.Examples.MathServer.project.json | 8 ++++++++ src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.csproj | 1 + .../Grpc.Examples.Tests/Grpc.Examples.Tests.project.json | 8 ++++++++ src/csharp/Grpc.Examples/Grpc.Examples.csproj | 1 + src/csharp/Grpc.Examples/Grpc.Examples.project.json | 8 ++++++++ .../Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.csproj | 1 + .../Grpc.HealthCheck.Tests.project.json | 8 ++++++++ src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.csproj | 1 + src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.project.json | 8 ++++++++ .../Grpc.IntegrationTesting.Client.csproj | 1 + .../Grpc.IntegrationTesting.Client.project.json | 8 ++++++++ .../Grpc.IntegrationTesting.QpsWorker.csproj | 1 + .../Grpc.IntegrationTesting.QpsWorker.project.json | 8 ++++++++ .../Grpc.IntegrationTesting.Server.csproj | 1 + .../Grpc.IntegrationTesting.Server.project.json | 8 ++++++++ .../Grpc.IntegrationTesting.StressClient.csproj | 5 ++++- .../Grpc.IntegrationTesting.StressClient.project.json | 8 ++++++++ .../Grpc.IntegrationTesting.csproj | 1 + .../Grpc.IntegrationTesting.project.json | 8 ++++++++ 28 files changed, 133 insertions(+), 1 deletion(-) create mode 100644 src/csharp/Grpc.Auth/Grpc.Auth.project.json create mode 100644 src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.project.json create mode 100644 src/csharp/Grpc.Core/Grpc.Core.project.json create mode 100644 src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.project.json create mode 100644 src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.project.json create mode 100644 src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.project.json create mode 100644 src/csharp/Grpc.Examples/Grpc.Examples.project.json create mode 100644 src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.project.json create mode 100644 src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.project.json create mode 100644 src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.project.json create mode 100644 src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.project.json create mode 100644 src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.project.json create mode 100644 src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.project.json create mode 100644 src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.project.json diff --git a/src/csharp/Grpc.Auth/Grpc.Auth.csproj b/src/csharp/Grpc.Auth/Grpc.Auth.csproj index 3acea7d2f83..1fa14fc3dfb 100644 --- a/src/csharp/Grpc.Auth/Grpc.Auth.csproj +++ b/src/csharp/Grpc.Auth/Grpc.Auth.csproj @@ -81,6 +81,7 @@ + \ No newline at end of file diff --git a/src/csharp/Grpc.Auth/Grpc.Auth.project.json b/src/csharp/Grpc.Auth/Grpc.Auth.project.json new file mode 100644 index 00000000000..c2f5bcb1637 --- /dev/null +++ b/src/csharp/Grpc.Auth/Grpc.Auth.project.json @@ -0,0 +1,8 @@ +{ + "frameworks": { + "net45": { } + }, + "runtimes": { + "win": { } + } +} diff --git a/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.csproj b/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.csproj index 074c9603dcf..f6c226567d9 100644 --- a/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.csproj +++ b/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.csproj @@ -99,6 +99,7 @@ + Designer diff --git a/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.project.json b/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.project.json new file mode 100644 index 00000000000..c2f5bcb1637 --- /dev/null +++ b/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.project.json @@ -0,0 +1,8 @@ +{ + "frameworks": { + "net45": { } + }, + "runtimes": { + "win": { } + } +} diff --git a/src/csharp/Grpc.Core/Grpc.Core.csproj b/src/csharp/Grpc.Core/Grpc.Core.csproj index e460f004473..1952ee37121 100644 --- a/src/csharp/Grpc.Core/Grpc.Core.csproj +++ b/src/csharp/Grpc.Core/Grpc.Core.csproj @@ -141,6 +141,7 @@ + diff --git a/src/csharp/Grpc.Core/Grpc.Core.project.json b/src/csharp/Grpc.Core/Grpc.Core.project.json new file mode 100644 index 00000000000..c2f5bcb1637 --- /dev/null +++ b/src/csharp/Grpc.Core/Grpc.Core.project.json @@ -0,0 +1,8 @@ +{ + "frameworks": { + "net45": { } + }, + "runtimes": { + "win": { } + } +} diff --git a/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.csproj b/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.csproj index 35c0646a3fd..65bf236def6 100644 --- a/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.csproj +++ b/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.csproj @@ -57,4 +57,7 @@ Grpc.Examples + + + \ No newline at end of file diff --git a/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.project.json b/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.project.json new file mode 100644 index 00000000000..c2f5bcb1637 --- /dev/null +++ b/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.project.json @@ -0,0 +1,8 @@ +{ + "frameworks": { + "net45": { } + }, + "runtimes": { + "win": { } + } +} diff --git a/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.csproj b/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.csproj index 74d79b44d98..26b42b6936d 100644 --- a/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.csproj +++ b/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.csproj @@ -57,4 +57,7 @@ Grpc.Examples + + + \ No newline at end of file diff --git a/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.project.json b/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.project.json new file mode 100644 index 00000000000..c2f5bcb1637 --- /dev/null +++ b/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.project.json @@ -0,0 +1,8 @@ +{ + "frameworks": { + "net45": { } + }, + "runtimes": { + "win": { } + } +} diff --git a/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.csproj b/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.csproj index 3fd28c65282..4c7d89309af 100644 --- a/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.csproj +++ b/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.csproj @@ -69,6 +69,7 @@ + diff --git a/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.project.json b/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.project.json new file mode 100644 index 00000000000..c2f5bcb1637 --- /dev/null +++ b/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.project.json @@ -0,0 +1,8 @@ +{ + "frameworks": { + "net45": { } + }, + "runtimes": { + "win": { } + } +} diff --git a/src/csharp/Grpc.Examples/Grpc.Examples.csproj b/src/csharp/Grpc.Examples/Grpc.Examples.csproj index 30170ab03c7..3dfa84e896e 100644 --- a/src/csharp/Grpc.Examples/Grpc.Examples.csproj +++ b/src/csharp/Grpc.Examples/Grpc.Examples.csproj @@ -69,6 +69,7 @@ + \ No newline at end of file diff --git a/src/csharp/Grpc.Examples/Grpc.Examples.project.json b/src/csharp/Grpc.Examples/Grpc.Examples.project.json new file mode 100644 index 00000000000..c2f5bcb1637 --- /dev/null +++ b/src/csharp/Grpc.Examples/Grpc.Examples.project.json @@ -0,0 +1,8 @@ +{ + "frameworks": { + "net45": { } + }, + "runtimes": { + "win": { } + } +} diff --git a/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.csproj b/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.csproj index a5ee4fdb46c..aefacfbcc01 100644 --- a/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.csproj +++ b/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.csproj @@ -74,6 +74,7 @@ + diff --git a/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.project.json b/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.project.json new file mode 100644 index 00000000000..c2f5bcb1637 --- /dev/null +++ b/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.project.json @@ -0,0 +1,8 @@ +{ + "frameworks": { + "net45": { } + }, + "runtimes": { + "win": { } + } +} diff --git a/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.csproj b/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.csproj index 2697b74f59e..7db8b2d38e2 100644 --- a/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.csproj +++ b/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.csproj @@ -65,6 +65,7 @@ + diff --git a/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.project.json b/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.project.json new file mode 100644 index 00000000000..c2f5bcb1637 --- /dev/null +++ b/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.project.json @@ -0,0 +1,8 @@ +{ + "frameworks": { + "net45": { } + }, + "runtimes": { + "win": { } + } +} diff --git a/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.csproj b/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.csproj index 339a754c02c..91fb3ce5bcf 100644 --- a/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.csproj +++ b/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.csproj @@ -83,6 +83,7 @@ + \ No newline at end of file diff --git a/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.project.json b/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.project.json new file mode 100644 index 00000000000..c2f5bcb1637 --- /dev/null +++ b/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.project.json @@ -0,0 +1,8 @@ +{ + "frameworks": { + "net45": { } + }, + "runtimes": { + "win": { } + } +} diff --git a/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.csproj b/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.csproj index 0dc2751b045..dda26a68923 100644 --- a/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.csproj +++ b/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.csproj @@ -59,5 +59,6 @@ + \ No newline at end of file diff --git a/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.project.json b/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.project.json new file mode 100644 index 00000000000..c2f5bcb1637 --- /dev/null +++ b/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.project.json @@ -0,0 +1,8 @@ +{ + "frameworks": { + "net45": { } + }, + "runtimes": { + "win": { } + } +} diff --git a/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.csproj b/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.csproj index 27a56503086..f73d99dbd1a 100644 --- a/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.csproj +++ b/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.csproj @@ -83,6 +83,7 @@ + \ No newline at end of file diff --git a/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.project.json b/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.project.json new file mode 100644 index 00000000000..c2f5bcb1637 --- /dev/null +++ b/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.project.json @@ -0,0 +1,8 @@ +{ + "frameworks": { + "net45": { } + }, + "runtimes": { + "win": { } + } +} diff --git a/src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.csproj b/src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.csproj index d6eba74289e..8bd3d789138 100644 --- a/src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.csproj +++ b/src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.csproj @@ -1,4 +1,4 @@ - + Debug @@ -57,4 +57,7 @@ Grpc.IntegrationTesting + + + \ No newline at end of file diff --git a/src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.project.json b/src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.project.json new file mode 100644 index 00000000000..c2f5bcb1637 --- /dev/null +++ b/src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.project.json @@ -0,0 +1,8 @@ +{ + "frameworks": { + "net45": { } + }, + "runtimes": { + "win": { } + } +} diff --git a/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.csproj b/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.csproj index 00890494088..3a0764230d6 100644 --- a/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.csproj +++ b/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.csproj @@ -129,6 +129,7 @@ + Designer diff --git a/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.project.json b/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.project.json new file mode 100644 index 00000000000..c2f5bcb1637 --- /dev/null +++ b/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.project.json @@ -0,0 +1,8 @@ +{ + "frameworks": { + "net45": { } + }, + "runtimes": { + "win": { } + } +} From c58b3bbbaf01f3492aedce5bc88cd11873a4b8a0 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 15 Jun 2016 09:47:31 -0700 Subject: [PATCH 066/280] rename Grpc.Dnx.sln --- src/csharp/{Grpc.Dnx.sln => Grpc.Dotnet.sln} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename src/csharp/{Grpc.Dnx.sln => Grpc.Dotnet.sln} (100%) diff --git a/src/csharp/Grpc.Dnx.sln b/src/csharp/Grpc.Dotnet.sln similarity index 100% rename from src/csharp/Grpc.Dnx.sln rename to src/csharp/Grpc.Dotnet.sln From 5ff32fa9fde85d664b60b0a9ff4d0799e2e0a2b3 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 15 Jun 2016 09:49:40 -0700 Subject: [PATCH 067/280] update .xproj files --- src/csharp/Grpc.Auth/Grpc.Auth.xproj | 3 +-- src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.xproj | 3 +-- src/csharp/Grpc.Core/Grpc.Core.xproj | 3 +-- .../Grpc.Examples.MathClient/Grpc.Examples.MathClient.xproj | 3 +-- .../Grpc.Examples.MathServer/Grpc.Examples.MathServer.xproj | 3 +-- src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.xproj | 3 +-- src/csharp/Grpc.Examples/Grpc.Examples.xproj | 3 +-- src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.xproj | 3 +-- src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.xproj | 3 +-- .../Grpc.IntegrationTesting.Client.xproj | 3 +-- .../Grpc.IntegrationTesting.QpsWorker.xproj | 3 +-- .../Grpc.IntegrationTesting.Server.xproj | 3 +-- .../Grpc.IntegrationTesting/Grpc.IntegrationTesting.xproj | 3 +-- 13 files changed, 13 insertions(+), 26 deletions(-) diff --git a/src/csharp/Grpc.Auth/Grpc.Auth.xproj b/src/csharp/Grpc.Auth/Grpc.Auth.xproj index c3a6fa2947b..dd3d94c574a 100644 --- a/src/csharp/Grpc.Auth/Grpc.Auth.xproj +++ b/src/csharp/Grpc.Auth/Grpc.Auth.xproj @@ -9,9 +9,8 @@ c82631ed-06d1-4458-87bc-8257d12307a8 Grpc.Auth ..\Grpc.Core\artifacts\obj\$(MSBuildProjectName) - ..\Grpc.Core\artifacts\bin\$(MSBuildProjectName)\ + .\bin\ - 2.0 diff --git a/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.xproj b/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.xproj index e6595118fb1..05823291542 100644 --- a/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.xproj +++ b/src/csharp/Grpc.Core.Tests/Grpc.Core.Tests.xproj @@ -9,9 +9,8 @@ 759e23b2-fc04-4695-902d-b073cded3599 Grpc.Core.Tests ..\artifacts\obj\$(MSBuildProjectName) - ..\artifacts\bin\$(MSBuildProjectName)\ + .\bin\ - 2.0 diff --git a/src/csharp/Grpc.Core/Grpc.Core.xproj b/src/csharp/Grpc.Core/Grpc.Core.xproj index f5d1bf2eef5..137236ffdb6 100644 --- a/src/csharp/Grpc.Core/Grpc.Core.xproj +++ b/src/csharp/Grpc.Core/Grpc.Core.xproj @@ -9,9 +9,8 @@ dc9908b6-f291-4fc8-a46d-2ea2551790ec Grpc.Core ..\artifacts\obj\$(MSBuildProjectName) - ..\artifacts\bin\$(MSBuildProjectName)\ + .\bin\ - 2.0 diff --git a/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.xproj b/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.xproj index a8dc195a6dc..4655bd43774 100644 --- a/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.xproj +++ b/src/csharp/Grpc.Examples.MathClient/Grpc.Examples.MathClient.xproj @@ -9,9 +9,8 @@ fd48deca-1622-4173-b1d9-2101cf5e7c5f Grpc.Examples.MathClient ..\artifacts\obj\$(MSBuildProjectName) - ..\artifacts\bin\$(MSBuildProjectName)\ + .\bin\ - 2.0 diff --git a/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.xproj b/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.xproj index 40746f1f946..38a449e8f29 100644 --- a/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.xproj +++ b/src/csharp/Grpc.Examples.MathServer/Grpc.Examples.MathServer.xproj @@ -9,9 +9,8 @@ 58579368-5372-4e67-acd6-9b59cb9fa698 Grpc.Examples.MathServer ..\artifacts\obj\$(MSBuildProjectName) - ..\artifacts\bin\$(MSBuildProjectName)\ + .\bin\ - 2.0 diff --git a/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.xproj b/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.xproj index c744816a811..9cecd18b2e4 100644 --- a/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.xproj +++ b/src/csharp/Grpc.Examples.Tests/Grpc.Examples.Tests.xproj @@ -9,9 +9,8 @@ c61714a6-f633-44fb-97f4-c91f425c1d15 Grpc.Examples.Tests ..\artifacts\obj\$(MSBuildProjectName) - ..\artifacts\bin\$(MSBuildProjectName)\ + .\bin\ - 2.0 diff --git a/src/csharp/Grpc.Examples/Grpc.Examples.xproj b/src/csharp/Grpc.Examples/Grpc.Examples.xproj index aa3ad680fd8..d1d7e6d9816 100644 --- a/src/csharp/Grpc.Examples/Grpc.Examples.xproj +++ b/src/csharp/Grpc.Examples/Grpc.Examples.xproj @@ -9,9 +9,8 @@ c77b792d-fc78-4ce2-9522-b40b0803c636 Grpc.Examples ..\artifacts\obj\$(MSBuildProjectName) - ..\artifacts\bin\$(MSBuildProjectName)\ + .\bin\ - 2.0 diff --git a/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.xproj b/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.xproj index db05ccf7a4f..724c5b2a160 100644 --- a/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.xproj +++ b/src/csharp/Grpc.HealthCheck.Tests/Grpc.HealthCheck.Tests.xproj @@ -9,9 +9,8 @@ 43dafac6-5343-4621-960e-a8a977ea3f0b Grpc.HealthCheck.Tests ..\artifacts\obj\$(MSBuildProjectName) - ..\artifacts\bin\$(MSBuildProjectName)\ + .\bin\ - 2.0 diff --git a/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.xproj b/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.xproj index 18701779d19..5806a7af979 100644 --- a/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.xproj +++ b/src/csharp/Grpc.HealthCheck/Grpc.HealthCheck.xproj @@ -9,9 +9,8 @@ 3be4ad0b-2bf0-4d68-b625-f6018ef0dcfa Grpc.HealthCheck ..\artifacts\obj\$(MSBuildProjectName) - ..\artifacts\bin\$(MSBuildProjectName)\ + .\bin\ - 2.0 diff --git a/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.xproj b/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.xproj index 82e044ec212..7f456cfaef1 100644 --- a/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.xproj +++ b/src/csharp/Grpc.IntegrationTesting.Client/Grpc.IntegrationTesting.Client.xproj @@ -9,9 +9,8 @@ 48ea5bbe-70e2-4198-869d-d7e59c45f30d Grpc.IntegrationTesting.Client ..\artifacts\obj\$(MSBuildProjectName) - ..\artifacts\bin\$(MSBuildProjectName)\ + .\bin\ - 2.0 diff --git a/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.xproj b/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.xproj index 7348004c982..15bec443d6c 100644 --- a/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.xproj +++ b/src/csharp/Grpc.IntegrationTesting.QpsWorker/Grpc.IntegrationTesting.QpsWorker.xproj @@ -9,9 +9,8 @@ 661b70d7-f56a-46e0-9b81-6227b591b5e7 Grpc.IntegrationTesting.QpsWorker ..\artifacts\obj\$(MSBuildProjectName) - ..\artifacts\bin\$(MSBuildProjectName)\ + .\bin\ - 2.0 diff --git a/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.xproj b/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.xproj index fc43c41fd89..689eb0b8425 100644 --- a/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.xproj +++ b/src/csharp/Grpc.IntegrationTesting.Server/Grpc.IntegrationTesting.Server.xproj @@ -9,9 +9,8 @@ 881f7ad1-a84e-47a2-9402-115c63c4031e Grpc.IntegrationTesting.Server ..\artifacts\obj\$(MSBuildProjectName) - ..\artifacts\bin\$(MSBuildProjectName)\ + .\bin\ - 2.0 diff --git a/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.xproj b/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.xproj index 69e89fa5770..357300ecb9b 100644 --- a/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.xproj +++ b/src/csharp/Grpc.IntegrationTesting/Grpc.IntegrationTesting.xproj @@ -9,9 +9,8 @@ 20354386-3e71-4046-a269-3bc2a06f3ec8 Grpc.IntegrationTesting ..\artifacts\obj\$(MSBuildProjectName) - ..\artifacts\bin\$(MSBuildProjectName)\ + .\bin\ - 2.0 From ae7b048de4c6b8691dcd89b1a4442aa08ce3392d Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 15 Jun 2016 09:52:57 -0700 Subject: [PATCH 068/280] upgrade protobuf to 3.0.0-beta3 in project.json files --- src/csharp/Grpc.Examples/project.json | 2 +- src/csharp/Grpc.HealthCheck/project.json | 2 +- src/csharp/Grpc.IntegrationTesting/project.json | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/csharp/Grpc.Examples/project.json b/src/csharp/Grpc.Examples/project.json index c78de89cf89..f259459a717 100644 --- a/src/csharp/Grpc.Examples/project.json +++ b/src/csharp/Grpc.Examples/project.json @@ -5,7 +5,7 @@ "dependencies": { "Grpc.Core": "0.0.1", - "Google.Protobuf": "3.0.0-beta2" + "Google.Protobuf": "3.0.0-beta3" }, "frameworks": { "net45": { }, diff --git a/src/csharp/Grpc.HealthCheck/project.json b/src/csharp/Grpc.HealthCheck/project.json index 0984d43deca..220224819f8 100644 --- a/src/csharp/Grpc.HealthCheck/project.json +++ b/src/csharp/Grpc.HealthCheck/project.json @@ -16,7 +16,7 @@ "dependencies": { "Grpc.Core": "0.14.0-anexperiment", - "Google.Protobuf": "3.0.0-beta2" + "Google.Protobuf": "3.0.0-beta3" }, "frameworks": { "net45": { }, diff --git a/src/csharp/Grpc.IntegrationTesting/project.json b/src/csharp/Grpc.IntegrationTesting/project.json index f2cda96e7be..93cabf21bcf 100644 --- a/src/csharp/Grpc.IntegrationTesting/project.json +++ b/src/csharp/Grpc.IntegrationTesting/project.json @@ -9,7 +9,7 @@ "dependencies": { "Grpc.Auth": "0.0.1", "Grpc.Core": "0.0.1", - "Google.Protobuf": "3.0.0-beta2", + "Google.Protobuf": "3.0.0-beta3", "CommandLineParser": "1.9.71", "Moq": "4.2.1510.2205", "NUnit": "3.2.0", From cf4205dff54d8e3920f98dac28049770b5f8f044 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 15 Jun 2016 11:22:20 -0700 Subject: [PATCH 069/280] Compilation error --- src/core/lib/iomgr/ev_epoll_linux.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 006c2a8ee7f..1fb59474640 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -317,7 +317,7 @@ static void polling_island_remove_all_fds_locked(polling_island *pi, if (err < 0 && errno != ENOENT) { /* TODO: sreek - We need a better way to bubble up this error instead of * just logging a message */ - gpr_log(GPR_ERROR, "epoll_ctl deleting fds[%d]: %d failed with error: %s", + gpr_log(GPR_ERROR, "epoll_ctl deleting fds[%zu]: %d failed with error: %s", i, pi->fds[i]->fd, strerror(errno)); } From c61c235fac8e2fbb1968852108f95029b67970d4 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 14 Jun 2016 18:20:38 -0700 Subject: [PATCH 070/280] moar pr comments --- doc/interop-test-descriptions.md | 95 ++++++++++++++++----------- src/proto/grpc/testing/messages.proto | 6 +- 2 files changed, 61 insertions(+), 40 deletions(-) diff --git a/doc/interop-test-descriptions.md b/doc/interop-test-descriptions.md index 480625c6ba7..a4f9abecfae 100644 --- a/doc/interop-test-descriptions.md +++ b/doc/interop-test-descriptions.md @@ -90,19 +90,21 @@ Client asserts: ### client_compressed_unary This test verifies the client can compress unary messages by sending two unary -calls, for compressed and uncompressed payloads. It sends an initial probing -request to verify whether the server supports the [CompressedRequest][] feature. -If it doesn't, the call is expected to fail with an `INVALID_ARGUMENT` status. +calls, for compressed and uncompressed payloads. It also sends an initial +probing request to verify whether the server supports the [CompressedRequest][] +feature by checking if the probing call fails with an `INVALID_ARGUMENT` status. Server features: * [UnaryCall][] * [CompressedRequest][] Procedure: - 1. Client calls UnaryCall with the feature probe, an **uncompressed** message: + 1. Client calls UnaryCall with the feature probe, an *uncompressed* message: ``` { - expect_compressed: false + expect_compressed:{ + value: true + } response_size: 314159 payload:{ body: 271828 bytes of zeros @@ -114,7 +116,9 @@ Procedure: ``` { - expect_compressed: true + expect_compressed:{ + value: true + } response_size: 314159 payload:{ body: 271828 bytes of zeros @@ -126,7 +130,9 @@ Procedure: ``` { - expect_compressed: false + expect_compressed:{ + value: false + } response_size: 314159 payload:{ body: 271828 bytes of zeros @@ -135,8 +141,8 @@ Procedure: ``` Client asserts: - * First call was unsuccessful with `INVALID_ARGUMENT` status. Subsequent - calls were successful. + * First call failed with `INVALID_ARGUMENT` status. + * Subsequent calls were successful. * Response payload body is 314159 bytes in size. * Clients are free to assert that the response payload body contents are zeros and comparing the entire response message against a golden response. @@ -149,7 +155,8 @@ requests, expecting the server's response to be compressed or not according to the `response_compressed` boolean. Whether compression was actually performed is determined by the compression bit -in the response's message flags. +in the response's message flags. *Note that some languages may not have access +to the message flags*. Server features: @@ -157,11 +164,13 @@ Server features: * [CompressedResponse][] Procedure: - 1. Client calls UnaryCall with `SimpleRequest`s: + 1. Client calls UnaryCall with `SimpleRequest`: ``` { - response_compressed: true + response_compressed:{ + value: true + } response_size: 314159 payload:{ body: 271828 bytes of zeros @@ -171,7 +180,9 @@ Procedure: ``` { - response_compressed: false + response_compressed:{ + value: false + } response_size: 314159 payload:{ body: 271828 bytes of zeros @@ -184,7 +195,7 @@ Procedure: compressed message flag set. * when `response_compressed` is false, the response MUST NOT have the compressed message flag set. - * response payload body is 314159 bytes in size + * response payload body is 314159 bytes in size in both cases. * clients are free to assert that the response payload body contents are zero and comparing the entire response message against a golden response @@ -247,21 +258,20 @@ Client asserts: ### client_compressed_streaming -This test verifies the client can compress streaming messages. It sends an -initial inconsistent streaming call comprised of a single message to verify if -the server implements the [CompressedRequest][] feature. If it does, the client -will then send another streaming call, comprised of two messages: the first one -compressed with `expect_compressed` true; the second one uncompressed with -`expected_compressed` false. +This test verifies the client can compress requests on per-message basis by +performing a two-request streaming call. It also sends an initial probing +request to verify whether the server supports the [CompressedRequest][] feature +by checking if the probing call fails with an `INVALID_ARGUMENT` status. Procedure: - 1. Client calls StreamingInputCall - 1. Client sends the following feature-probing *uncompressed* - `StreamingInputCallRequest` message + 1. Client calls `StreamingInputCall` and sends the following feature-probing + *uncompressed* `StreamingInputCallRequest` message ``` { - expect_compressed: true + expect_compressed:{ + value: true + } payload:{ body: 27182 bytes of zeros } @@ -270,31 +280,37 @@ Procedure: If the call fails with `INVALID_ARGUMENT`, the test fails. Otherwise, we continue. - 1. Client then sends the *compressed* message + 1. Client calls `StreamingInputCall` again, sending the *compressed* message ``` { - expect_compressed: true + expect_compressed:{ + value: true + } payload:{ body: 27182 bytes of zeros } } ``` - 1. And finally, the *uncompressed* message: + + 1. And finally, the *uncompressed* message ``` { - expect_compressed: false + expect_compressed:{ + value: false + } payload:{ body: 45904 bytes of zeros } } ``` + 1. Client half-closes - Client asserts: - * First call was unsuccessful with `INVALID_ARGUMENT` status. Subsequent - calls were successful. - * Response aggregated_payload_size is 73086. +Client asserts: +* First call fails with `INVALID_ARGUMENT`. +* Next calls succeeds. +* Response aggregated payload size is 73086. ### server_streaming @@ -333,7 +349,8 @@ Client asserts: ### server_compressed_streaming -This test verifies that the server can compress streaming messages. +This test verifies that the server can compress streaming messages and disable +compression on individual messages. Server features: * [StreamingOutputCall][] @@ -346,12 +363,16 @@ Procedure: ``` { response_parameters:{ - response_compressed: true + compressed: { + value: true + } size: 31415 } response_parameters:{ - response_compressed: false - size: 58979 + compressed: { + value: false + } + size: 92653 } } ``` @@ -363,7 +384,7 @@ Procedure: NOT have the compressed message flag set. * when `response_compressed` is true, the response's messages MUST have the compressed message flag set. - * response payload bodies are sized (in order): 31415, 58979 + * response payload bodies are sized (in order): 31415, 92653 * clients are free to assert that the response payload body contents are zero and comparing the entire response messages against golden responses diff --git a/src/proto/grpc/testing/messages.proto b/src/proto/grpc/testing/messages.proto index 1812dc74029..e4e748a6913 100644 --- a/src/proto/grpc/testing/messages.proto +++ b/src/proto/grpc/testing/messages.proto @@ -88,7 +88,7 @@ message SimpleRequest { EchoStatus response_status = 7; // Whether the server should expect this request to be compressed. - bool expect_compressed = 8; + google.protobuf.BoolValue expect_compressed = 8; } // Unary response, as configured by the request. @@ -111,7 +111,7 @@ message StreamingInputCallRequest { // is "nullable" in order to interoperate seamlessly with servers not able to // implement the full compression tests by introspecting the call to verify // the request's compression status. - BoolValue expect_compressed = 2; + google.protobuf.BoolValue expect_compressed = 2; // Not expecting any payload from the response. } @@ -135,7 +135,7 @@ message ResponseParameters { // "nullable" in order to interoperate seamlessly with clients not able to // implement the full compression tests by introspecting the call to verify // the response's compression status. - BoolValue compressed = 3; + google.protobuf.BoolValue compressed = 3; } // Server-streaming request. From a6c9a9121ec68a6754f4020bed357498bb237b37 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 15 Jun 2016 10:11:43 -0700 Subject: [PATCH 071/280] update project.json --- src/csharp/Grpc.Auth/project.json | 25 +++++------ src/csharp/Grpc.Core.Tests/project.json | 14 +++--- src/csharp/Grpc.Core/project.json | 44 +++++++++---------- .../Grpc.Examples.MathClient/project.json | 13 +++--- .../Grpc.Examples.MathServer/project.json | 13 +++--- src/csharp/Grpc.Examples.Tests/project.json | 13 +++--- src/csharp/Grpc.Examples/project.json | 13 ++++-- .../Grpc.HealthCheck.Tests/project.json | 15 ++++--- src/csharp/Grpc.HealthCheck/project.json | 30 ++++++++----- .../project.json | 13 +++--- .../project.json | 13 +++--- .../project.json | 13 +++--- .../Grpc.IntegrationTesting/project.json | 14 +++--- 13 files changed, 133 insertions(+), 100 deletions(-) diff --git a/src/csharp/Grpc.Auth/project.json b/src/csharp/Grpc.Auth/project.json index 513325f7491..25579982a9d 100644 --- a/src/csharp/Grpc.Auth/project.json +++ b/src/csharp/Grpc.Auth/project.json @@ -1,20 +1,17 @@ { "version": "0.14.0-anexperiment", - "title": "gRPC C# Auth", - "summary": "Auth library for C# implementation of gRPC - an RPC library and framework", - "description": "Auth library for C# implementation of gRPC - an RPC library and framework. See project site for more info.", - "authors": ["Google Inc."], - "owners": ["grpc-packages"], - "licenseUrl": "https://github.com/grpc/grpc/blob/master/LICENSE", - "projectUrl": "https://github.com/grpc/grpc", - "requireLicenseAcceptance": false, + "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", - "tags": ["gRPC RPC Protocol HTTP/2 Auth OAuth2"], - - "compile": "**/*.cs", - "resourceFiles": ["../../../etc/roots.pem"], - + "packOptions": { + "summary": "Auth library for C# implementation of gRPC - an RPC library and framework", + "description": "Auth library for C# implementation of gRPC - an RPC library and framework. See project site for more info.", + "owners": [ "grpc-packages" ], + "licenseUrl": "https://github.com/grpc/grpc/blob/master/LICENSE", + "projectUrl": "https://github.com/grpc/grpc", + "requireLicenseAcceptance": false, + "tags": [ "gRPC RPC Protocol HTTP/2 Auth OAuth2" ], + }, "dependencies": { "Grpc.Core": "0.14.0-anexperiment", "Google.Apis.Auth": "1.11.1" @@ -28,7 +25,7 @@ "dependencies": { "Microsoft.CSharp": "4.0.1-beta-23516", "Microsoft.NETCore.Portable.Compatibility": "1.0.1-beta-23516", - "System.Threading.Tasks": "4.0.11-beta-23516" + "System.Threading.Tasks": "4.0.11-beta-23516" } } } diff --git a/src/csharp/Grpc.Core.Tests/project.json b/src/csharp/Grpc.Core.Tests/project.json index 0c5d9354358..dc90e04ccfb 100644 --- a/src/csharp/Grpc.Core.Tests/project.json +++ b/src/csharp/Grpc.Core.Tests/project.json @@ -1,13 +1,17 @@ { - "compile": "**/*.cs", - "compilationOptions": { + "buildOptions": { + "compile": "**/*.cs", + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + } + }, "emitEntryPoint": true }, - - "content": "../nativelibs/**", - "dependencies": { "Grpc.Core": "0.14.0-anexperiment", + "Newtonsoft.Json": "8.0.3", "NUnit": "3.2.0", "NUnitLite": "3.2.0-*" }, diff --git a/src/csharp/Grpc.Core/project.json b/src/csharp/Grpc.Core/project.json index 8aece57856c..a916bbd09ab 100644 --- a/src/csharp/Grpc.Core/project.json +++ b/src/csharp/Grpc.Core/project.json @@ -1,32 +1,30 @@ { "version": "0.14.0-anexperiment", - "title": "gRPC C# Core", - "summary": "Core C# implementation of gRPC - an RPC library and framework", - "description": "Core C# implementation of gRPC - an RPC library and framework. See project site for more info.", "authors": [ "Google Inc." ], - "owners": [ "grpc-packages" ], - "licenseUrl": "https://github.com/grpc/grpc/blob/master/LICENSE", - "projectUrl": "https://github.com/grpc/grpc", - "requireLicenseAcceptance": false, "copyright": "Copyright 2015, Google Inc.", - "tags": [ "gRPC RPC Protocol HTTP/2" ], - - "compile": "**/*.cs", - "resourceFiles": [ "../../../etc/roots.pem" ], - - "content": "../nativelibs/**", - - "packInclude": { - "build/net45/": "Grpc.Core.targets", - "build/native/bin/windows_x86/": "../nativelibs/windows_x86/grpc_csharp_ext.dll", - "build/native/bin/windows_x64/": "../nativelibs/windows_x64/grpc_csharp_ext.dll", - "build/native/bin/linux_x86/": "../nativelibs/linux_x86/libgrpc_csharp_ext.so", - "build/native/bin/linux_x64/": "../nativelibs/linux_x64/libgrpc_csharp_ext.so", - "build/native/bin/macosx_x86/": "../nativelibs/macosx_x86/libgrpc_csharp_ext.dylib", - "build/native/bin/macosx_x64/": "../nativelibs/macosx_x64/libgrpc_csharp_ext.dylib" + "packOptions": { + "summary": "Core C# implementation of gRPC - an RPC library and framework", + "description": "Core C# implementation of gRPC - an RPC library and framework. See project site for more info.", + "owners": [ "grpc-packages" ], + "licenseUrl": "https://github.com/grpc/grpc/blob/master/LICENSE", + "projectUrl": "https://github.com/grpc/grpc", + "requireLicenseAcceptance": false, + "tags": [ "gRPC RPC Protocol HTTP/2" ], + "files": { + "build/net45/": "Grpc.Core.targets", + "build/native/bin/windows_x86/": "../nativelibs/windows_x86/grpc_csharp_ext.dll", + "build/native/bin/windows_x64/": "../nativelibs/windows_x64/grpc_csharp_ext.dll", + "build/native/bin/linux_x86/": "../nativelibs/linux_x86/libgrpc_csharp_ext.so", + "build/native/bin/linux_x64/": "../nativelibs/linux_x64/libgrpc_csharp_ext.so", + "build/native/bin/macosx_x86/": "../nativelibs/macosx_x86/libgrpc_csharp_ext.dylib", + "build/native/bin/macosx_x64/": "../nativelibs/macosx_x64/libgrpc_csharp_ext.dylib" + } + }, + "buildOptions": { + "compile": "**/*.cs", + "embed": [ "../../../etc/roots.pem" ] }, - "dependencies": { "Ix-Async": "1.2.5" }, diff --git a/src/csharp/Grpc.Examples.MathClient/project.json b/src/csharp/Grpc.Examples.MathClient/project.json index 67af09721ce..4abf8a5e34f 100644 --- a/src/csharp/Grpc.Examples.MathClient/project.json +++ b/src/csharp/Grpc.Examples.MathClient/project.json @@ -1,11 +1,14 @@ { - "compile": "**/*.cs", - "compilationOptions": { + "buildOptions": { + "compile": "**/*.cs", + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + } + }, "emitEntryPoint": true }, - - "content": "../nativelibs/**", - "dependencies": { "Grpc.Examples": "1.0.0" }, diff --git a/src/csharp/Grpc.Examples.MathServer/project.json b/src/csharp/Grpc.Examples.MathServer/project.json index 67af09721ce..4abf8a5e34f 100644 --- a/src/csharp/Grpc.Examples.MathServer/project.json +++ b/src/csharp/Grpc.Examples.MathServer/project.json @@ -1,11 +1,14 @@ { - "compile": "**/*.cs", - "compilationOptions": { + "buildOptions": { + "compile": "**/*.cs", + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + } + }, "emitEntryPoint": true }, - - "content": "../nativelibs/**", - "dependencies": { "Grpc.Examples": "1.0.0" }, diff --git a/src/csharp/Grpc.Examples.Tests/project.json b/src/csharp/Grpc.Examples.Tests/project.json index e61aa65abfc..bd74812c153 100644 --- a/src/csharp/Grpc.Examples.Tests/project.json +++ b/src/csharp/Grpc.Examples.Tests/project.json @@ -1,11 +1,14 @@ { - "compile": "**/*.cs", - "compilationOptions": { + "buildOptions": { + "compile": "**/*.cs", + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + } + }, "emitEntryPoint": true }, - - "content": "../nativelibs/**", - "dependencies": { "Grpc.Examples": "1.0.0", "NUnit": "3.2.0", diff --git a/src/csharp/Grpc.Examples/project.json b/src/csharp/Grpc.Examples/project.json index f259459a717..e2b4c10422f 100644 --- a/src/csharp/Grpc.Examples/project.json +++ b/src/csharp/Grpc.Examples/project.json @@ -1,14 +1,19 @@ { - "compile": "**/*.cs", - - "content": "../nativelibs/**", + "buildOptions": { + "compile": "**/*.cs" + }, "dependencies": { "Grpc.Core": "0.0.1", "Google.Protobuf": "3.0.0-beta3" }, "frameworks": { - "net45": { }, + "net45": { + "frameworkAssemblies": { + "System.Runtime": "", + "System.IO": "" + } + }, "dotnet54": { "imports": [ "portable-net45" diff --git a/src/csharp/Grpc.HealthCheck.Tests/project.json b/src/csharp/Grpc.HealthCheck.Tests/project.json index 5ecb17c9671..248a1324f60 100644 --- a/src/csharp/Grpc.HealthCheck.Tests/project.json +++ b/src/csharp/Grpc.HealthCheck.Tests/project.json @@ -1,13 +1,16 @@ { - "compile": "**/*.cs", - "compilationOptions": { + "buildOptions": { + "compile": "**/*.cs", + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + } + }, "emitEntryPoint": true }, - - "content": "../nativelibs/**", - "dependencies": { - "Grpc.HealthCheck": "1.0.0", + "Grpc.HealthCheck": "0.0.1", "NUnit": "3.2.0", "NUnitLite": "3.2.0-*" }, diff --git a/src/csharp/Grpc.HealthCheck/project.json b/src/csharp/Grpc.HealthCheck/project.json index 220224819f8..ad42df595d4 100644 --- a/src/csharp/Grpc.HealthCheck/project.json +++ b/src/csharp/Grpc.HealthCheck/project.json @@ -1,25 +1,31 @@ { "version": "0.14.0-anexperiment", - "title": "gRPC C# Healthchecking", - "summary": "Implementation of gRPC health service", - "description": "Example implementation of grpc.health.v1 service that can be used for health-checking.", "authors": [ "Google Inc." ], - "owners": [ "grpc-packages" ], - "licenseUrl": "https://github.com/grpc/grpc/blob/master/LICENSE", - "projectUrl": "https://github.com/grpc/grpc", - "requireLicenseAcceptance": false, "copyright": "Copyright 2015, Google Inc.", - "tags": [ "gRPC health check" ], - - "compile": "**/*.cs", - + "packOptions": { + "summary": "Implementation of gRPC health service", + "description": "Example implementation of grpc.health.v1 service that can be used for health-checking.", + "owners": [ "grpc-packages" ], + "licenseUrl": "https://github.com/grpc/grpc/blob/master/LICENSE", + "projectUrl": "https://github.com/grpc/grpc", + "requireLicenseAcceptance": false, + "tags": [ "gRPC health check" ] + }, + "buildOptions": { + "compile": "**/*.cs" + }, "dependencies": { "Grpc.Core": "0.14.0-anexperiment", "Google.Protobuf": "3.0.0-beta3" }, "frameworks": { - "net45": { }, + "net45": { + "frameworkAssemblies": { + "System.Runtime": "", + "System.IO": "" + } + }, "dotnet54": { "imports": [ "portable-net45" diff --git a/src/csharp/Grpc.IntegrationTesting.Client/project.json b/src/csharp/Grpc.IntegrationTesting.Client/project.json index e023a9815b1..6ad74d59984 100644 --- a/src/csharp/Grpc.IntegrationTesting.Client/project.json +++ b/src/csharp/Grpc.IntegrationTesting.Client/project.json @@ -1,11 +1,14 @@ { - "compile": "**/*.cs", - "compilationOptions": { + "buildOptions": { + "compile": "**/*.cs", + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + } + }, "emitEntryPoint": true }, - - "content": "../nativelibs/**", - "dependencies": { "Grpc.IntegrationTesting": "1.0.0" }, diff --git a/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json b/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json index e023a9815b1..6ad74d59984 100644 --- a/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json +++ b/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json @@ -1,11 +1,14 @@ { - "compile": "**/*.cs", - "compilationOptions": { + "buildOptions": { + "compile": "**/*.cs", + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + } + }, "emitEntryPoint": true }, - - "content": "../nativelibs/**", - "dependencies": { "Grpc.IntegrationTesting": "1.0.0" }, diff --git a/src/csharp/Grpc.IntegrationTesting.Server/project.json b/src/csharp/Grpc.IntegrationTesting.Server/project.json index e023a9815b1..6ad74d59984 100644 --- a/src/csharp/Grpc.IntegrationTesting.Server/project.json +++ b/src/csharp/Grpc.IntegrationTesting.Server/project.json @@ -1,11 +1,14 @@ { - "compile": "**/*.cs", - "compilationOptions": { + "buildOptions": { + "compile": "**/*.cs", + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + } + }, "emitEntryPoint": true }, - - "content": "../nativelibs/**", - "dependencies": { "Grpc.IntegrationTesting": "1.0.0" }, diff --git a/src/csharp/Grpc.IntegrationTesting/project.json b/src/csharp/Grpc.IntegrationTesting/project.json index 93cabf21bcf..0093531fa9e 100644 --- a/src/csharp/Grpc.IntegrationTesting/project.json +++ b/src/csharp/Grpc.IntegrationTesting/project.json @@ -1,11 +1,8 @@ { - "compile": "**/*.cs", - "compilationOptions": { + "buildOptions": { + "compile": "**/*.cs", "emitEntryPoint": true }, - - "content": "../nativelibs/**", - "dependencies": { "Grpc.Auth": "0.0.1", "Grpc.Core": "0.0.1", @@ -16,6 +13,11 @@ "NUnitLite": "3.2.0-*" }, "frameworks": { - "net45": { } + "net45": { + "frameworkAssemblies": { + "System.Runtime": "", + "System.IO": "" + } + } } } From 635fafc398df035d45e0f56fe24fb21477805067 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 15 Jun 2016 15:10:51 -0700 Subject: [PATCH 072/280] dont register shutdownhooks for dotnet5.4 --- src/csharp/Grpc.Core.Tests/AppDomainUnloadTest.cs | 14 ++++++++------ src/csharp/Grpc.Core/GrpcEnvironment.cs | 4 ++++ 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/src/csharp/Grpc.Core.Tests/AppDomainUnloadTest.cs b/src/csharp/Grpc.Core.Tests/AppDomainUnloadTest.cs index e605a310f9e..d2c641cf2f8 100644 --- a/src/csharp/Grpc.Core.Tests/AppDomainUnloadTest.cs +++ b/src/csharp/Grpc.Core.Tests/AppDomainUnloadTest.cs @@ -32,13 +32,7 @@ #endregion using System; -using System.Diagnostics; -using System.Linq; -using System.Reflection; -using System.Threading; using System.Threading.Tasks; -using Grpc.Core; -using Grpc.Core.Internal; using Grpc.Core.Utils; using NUnit.Framework; @@ -46,6 +40,13 @@ namespace Grpc.Core.Tests { public class AppDomainUnloadTest { +#if DOTNET5_4 + [Test] + [Ignore("Not supported for CoreCLR")] + public void AppDomainUnloadHookCanCleanupAbandonedCall() + { + } +#else [Test] public void AppDomainUnloadHookCanCleanupAbandonedCall() { @@ -86,5 +87,6 @@ namespace Grpc.Core.Tests readyToShutdown.Task.Wait(); // make sure handler is running } } +#endif } } diff --git a/src/csharp/Grpc.Core/GrpcEnvironment.cs b/src/csharp/Grpc.Core/GrpcEnvironment.cs index e9e4cb4cbb3..37f9dce7c2f 100644 --- a/src/csharp/Grpc.Core/GrpcEnvironment.cs +++ b/src/csharp/Grpc.Core/GrpcEnvironment.cs @@ -352,8 +352,12 @@ namespace Grpc.Core { if (!hooksRegistered) { + // TODO(jtattermusch): register shutdownhooks for CoreCLR as well +#if !DOTNET5_4 + AppDomain.CurrentDomain.ProcessExit += ShutdownHookHandler; AppDomain.CurrentDomain.DomainUnload += ShutdownHookHandler; +#endif } hooksRegistered = true; } From 9f6a80597771eae74760b23a66737eb8960f8fd7 Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Wed, 15 Jun 2016 17:39:03 -0700 Subject: [PATCH 073/280] Allow disabling traces, add trace variables for pluck and timeout events --- BUILD | 4 -- build.yaml | 1 - gRPC.podspec | 2 - grpc.gemspec | 1 - package.xml | 1 - src/core/lib/debug/trace.c | 8 +++- src/core/lib/surface/call.h | 1 - src/core/lib/surface/completion_queue.c | 29 +++++++---- src/core/lib/surface/completion_queue.h | 3 ++ src/core/lib/surface/init.c | 7 ++- src/core/lib/surface/surface_trace.h | 48 ------------------- tools/doxygen/Doxyfile.core.internal | 1 - tools/run_tests/sources_and_headers.json | 2 - vsprojects/vcxproj/grpc/grpc.vcxproj | 1 - vsprojects/vcxproj/grpc/grpc.vcxproj.filters | 3 -- .../grpc_unsecure/grpc_unsecure.vcxproj | 1 - .../grpc_unsecure.vcxproj.filters | 3 -- 17 files changed, 36 insertions(+), 80 deletions(-) delete mode 100644 src/core/lib/surface/surface_trace.h diff --git a/BUILD b/BUILD index f049e3c4056..bf001ac7273 100644 --- a/BUILD +++ b/BUILD @@ -228,7 +228,6 @@ cc_library( "src/core/lib/surface/init.h", "src/core/lib/surface/lame_client.h", "src/core/lib/surface/server.h", - "src/core/lib/surface/surface_trace.h", "src/core/lib/transport/byte_stream.h", "src/core/lib/transport/connectivity_state.h", "src/core/lib/transport/metadata.h", @@ -606,7 +605,6 @@ cc_library( "src/core/lib/surface/init.h", "src/core/lib/surface/lame_client.h", "src/core/lib/surface/server.h", - "src/core/lib/surface/surface_trace.h", "src/core/lib/transport/byte_stream.h", "src/core/lib/transport/connectivity_state.h", "src/core/lib/transport/metadata.h", @@ -949,7 +947,6 @@ cc_library( "src/core/lib/surface/init.h", "src/core/lib/surface/lame_client.h", "src/core/lib/surface/server.h", - "src/core/lib/surface/surface_trace.h", "src/core/lib/transport/byte_stream.h", "src/core/lib/transport/connectivity_state.h", "src/core/lib/transport/metadata.h", @@ -2061,7 +2058,6 @@ objc_library( "src/core/lib/surface/init.h", "src/core/lib/surface/lame_client.h", "src/core/lib/surface/server.h", - "src/core/lib/surface/surface_trace.h", "src/core/lib/transport/byte_stream.h", "src/core/lib/transport/connectivity_state.h", "src/core/lib/transport/metadata.h", diff --git a/build.yaml b/build.yaml index a83ebc595ad..11cc9347cb0 100644 --- a/build.yaml +++ b/build.yaml @@ -224,7 +224,6 @@ filegroups: - src/core/lib/surface/init.h - src/core/lib/surface/lame_client.h - src/core/lib/surface/server.h - - src/core/lib/surface/surface_trace.h - src/core/lib/transport/byte_stream.h - src/core/lib/transport/connectivity_state.h - src/core/lib/transport/metadata.h diff --git a/gRPC.podspec b/gRPC.podspec index d3665d51744..622be89ca74 100644 --- a/gRPC.podspec +++ b/gRPC.podspec @@ -231,7 +231,6 @@ Pod::Spec.new do |s| 'src/core/lib/surface/init.h', 'src/core/lib/surface/lame_client.h', 'src/core/lib/surface/server.h', - 'src/core/lib/surface/surface_trace.h', 'src/core/lib/transport/byte_stream.h', 'src/core/lib/transport/connectivity_state.h', 'src/core/lib/transport/metadata.h', @@ -606,7 +605,6 @@ Pod::Spec.new do |s| 'src/core/lib/surface/init.h', 'src/core/lib/surface/lame_client.h', 'src/core/lib/surface/server.h', - 'src/core/lib/surface/surface_trace.h', 'src/core/lib/transport/byte_stream.h', 'src/core/lib/transport/connectivity_state.h', 'src/core/lib/transport/metadata.h', diff --git a/grpc.gemspec b/grpc.gemspec index 9f3ae048a70..e0a8ff00a42 100755 --- a/grpc.gemspec +++ b/grpc.gemspec @@ -240,7 +240,6 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/surface/init.h ) s.files += %w( src/core/lib/surface/lame_client.h ) s.files += %w( src/core/lib/surface/server.h ) - s.files += %w( src/core/lib/surface/surface_trace.h ) s.files += %w( src/core/lib/transport/byte_stream.h ) s.files += %w( src/core/lib/transport/connectivity_state.h ) s.files += %w( src/core/lib/transport/metadata.h ) diff --git a/package.xml b/package.xml index b25e42c1c01..fef06ef9eed 100644 --- a/package.xml +++ b/package.xml @@ -247,7 +247,6 @@ - diff --git a/src/core/lib/debug/trace.c b/src/core/lib/debug/trace.c index 555f497b784..c56046785b3 100644 --- a/src/core/lib/debug/trace.c +++ b/src/core/lib/debug/trace.c @@ -88,7 +88,11 @@ static void parse(const char *s) { split(s, &strings, &nstrings); for (i = 0; i < nstrings; i++) { - grpc_tracer_set_enabled(strings[i], 1); + if (strings[i][0] == '-') { + grpc_tracer_set_enabled(strings[i] + 1, 0); + } else { + grpc_tracer_set_enabled(strings[i], 1); + } } for (i = 0; i < nstrings; i++) { @@ -117,7 +121,7 @@ int grpc_tracer_set_enabled(const char *name, int enabled) { tracer *t; if (0 == strcmp(name, "all")) { for (t = tracers; t; t = t->next) { - *t->flag = 1; + *t->flag = enabled; } } else { int found = 0; diff --git a/src/core/lib/surface/call.h b/src/core/lib/surface/call.h index b640345c21d..3a78fe3aa36 100644 --- a/src/core/lib/surface/call.h +++ b/src/core/lib/surface/call.h @@ -37,7 +37,6 @@ #include "src/core/lib/channel/channel_stack.h" #include "src/core/lib/channel/context.h" #include "src/core/lib/surface/api_trace.h" -#include "src/core/lib/surface/surface_trace.h" #include #include diff --git a/src/core/lib/surface/completion_queue.c b/src/core/lib/surface/completion_queue.c index 5eb7cf1bf4f..de2b674d0e0 100644 --- a/src/core/lib/surface/completion_queue.c +++ b/src/core/lib/surface/completion_queue.c @@ -48,7 +48,6 @@ #include "src/core/lib/surface/api_trace.h" #include "src/core/lib/surface/call.h" #include "src/core/lib/surface/event_string.h" -#include "src/core/lib/surface/surface_trace.h" typedef struct { grpc_pollset_worker **worker; @@ -91,6 +90,18 @@ struct grpc_completion_queue { static gpr_mu g_freelist_mu; static grpc_completion_queue *g_freelist; +int grpc_cq_pluck_trace; +int grpc_cq_event_timeout_trace; + +#define GRPC_SURFACE_TRACE_RETURNED_EVENT(cq, event) \ + if (grpc_api_trace && \ + (grpc_cq_pluck_trace || (event)->type != GRPC_QUEUE_TIMEOUT)) { \ + char *_ev = grpc_event_string(event); \ + gpr_log(GPR_INFO, "RETURN_EVENT[%p]: %s", cq, _ev); \ + gpr_free(_ev); \ + } + + static void on_pollset_shutdown_done(grpc_exec_ctx *exec_ctx, void *cc, bool success); @@ -396,13 +407,15 @@ grpc_event grpc_completion_queue_pluck(grpc_completion_queue *cc, void *tag, GPR_TIMER_BEGIN("grpc_completion_queue_pluck", 0); - GRPC_API_TRACE( - "grpc_completion_queue_pluck(" - "cc=%p, tag=%p, " - "deadline=gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " - "reserved=%p)", - 6, (cc, tag, (long long)deadline.tv_sec, (int)deadline.tv_nsec, - (int)deadline.clock_type, reserved)); + if (grpc_cq_pluck_trace) { + GRPC_API_TRACE( + "grpc_completion_queue_pluck(" + "cc=%p, tag=%p, " + "deadline=gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " + "reserved=%p)", + 6, (cc, tag, (long long)deadline.tv_sec, (int)deadline.tv_nsec, + (int)deadline.clock_type, reserved)); + } GPR_ASSERT(!reserved); deadline = gpr_convert_clock_type(deadline, GPR_CLOCK_MONOTONIC); diff --git a/src/core/lib/surface/completion_queue.h b/src/core/lib/surface/completion_queue.h index 3d0dd13c53b..108445b28f1 100644 --- a/src/core/lib/surface/completion_queue.h +++ b/src/core/lib/surface/completion_queue.h @@ -39,6 +39,9 @@ #include #include "src/core/lib/iomgr/pollset.h" +extern int grpc_cq_pluck_trace; +extern int grpc_cq_event_timeout_trace; + typedef struct grpc_cq_completion { /** user supplied tag */ void *tag; diff --git a/src/core/lib/surface/init.c b/src/core/lib/surface/init.c index 1c8b7090156..499ecffc2ae 100644 --- a/src/core/lib/surface/init.c +++ b/src/core/lib/surface/init.c @@ -57,7 +57,6 @@ #include "src/core/lib/surface/init.h" #include "src/core/lib/surface/lame_client.h" #include "src/core/lib/surface/server.h" -#include "src/core/lib/surface/surface_trace.h" #include "src/core/lib/transport/connectivity_state.h" #include "src/core/lib/transport/transport_impl.h" @@ -165,6 +164,12 @@ void grpc_init(void) { &grpc_trace_channel_stack_builder); grpc_register_tracer("http1", &grpc_http1_trace); grpc_register_tracer("compression", &grpc_compression_trace); + grpc_register_tracer("queue_pluck", &grpc_cq_pluck_trace); + // Default pluck trace to 1 + grpc_cq_pluck_trace = 1; + grpc_register_tracer("queue_timeout", &grpc_cq_event_timeout_trace); + // Default timeout trace to 1 + grpc_cq_event_timeout_trace = 1; grpc_security_pre_init(); grpc_iomgr_init(); grpc_executor_init(); diff --git a/src/core/lib/surface/surface_trace.h b/src/core/lib/surface/surface_trace.h deleted file mode 100644 index a69a0fff577..00000000000 --- a/src/core/lib/surface/surface_trace.h +++ /dev/null @@ -1,48 +0,0 @@ -/* - * - * Copyright 2015, Google Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are - * met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following disclaimer - * in the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Google Inc. nor the names of its - * contributors may be used to endorse or promote products derived from - * this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - * - */ - -#ifndef GRPC_CORE_LIB_SURFACE_SURFACE_TRACE_H -#define GRPC_CORE_LIB_SURFACE_SURFACE_TRACE_H - -#include -#include "src/core/lib/debug/trace.h" -#include "src/core/lib/surface/api_trace.h" - -#define GRPC_SURFACE_TRACE_RETURNED_EVENT(cq, event) \ - if (grpc_api_trace) { \ - char *_ev = grpc_event_string(event); \ - gpr_log(GPR_INFO, "RETURN_EVENT[%p]: %s", cq, _ev); \ - gpr_free(_ev); \ - } - -#endif /* GRPC_CORE_LIB_SURFACE_SURFACE_TRACE_H */ diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal index aee866adf46..337ce16551a 100644 --- a/tools/doxygen/Doxyfile.core.internal +++ b/tools/doxygen/Doxyfile.core.internal @@ -857,7 +857,6 @@ src/core/lib/surface/event_string.h \ src/core/lib/surface/init.h \ src/core/lib/surface/lame_client.h \ src/core/lib/surface/server.h \ -src/core/lib/surface/surface_trace.h \ src/core/lib/transport/byte_stream.h \ src/core/lib/transport/connectivity_state.h \ src/core/lib/transport/metadata.h \ diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index 4aad52c69d0..f9c497afcdd 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -5762,7 +5762,6 @@ "src/core/lib/surface/init.h", "src/core/lib/surface/lame_client.h", "src/core/lib/surface/server.h", - "src/core/lib/surface/surface_trace.h", "src/core/lib/transport/byte_stream.h", "src/core/lib/transport/connectivity_state.h", "src/core/lib/transport/metadata.h", @@ -5921,7 +5920,6 @@ "src/core/lib/surface/metadata_array.c", "src/core/lib/surface/server.c", "src/core/lib/surface/server.h", - "src/core/lib/surface/surface_trace.h", "src/core/lib/surface/validate_metadata.c", "src/core/lib/surface/version.c", "src/core/lib/transport/byte_stream.c", diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj b/vsprojects/vcxproj/grpc/grpc.vcxproj index ec0c984f2e7..c69623bf736 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj @@ -366,7 +366,6 @@ - diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters index 273f73b2a30..593024993fa 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters @@ -839,9 +839,6 @@ src\core\lib\surface - - src\core\lib\surface - src\core\lib\transport diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj index bba099d803c..28dcd8269eb 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj @@ -355,7 +355,6 @@ - diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters index 78a8777f942..058ae653cad 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters @@ -749,9 +749,6 @@ src\core\lib\surface - - src\core\lib\surface - src\core\lib\transport From dbe2b9e976d50ce7971df32dc9a3122525dad3ce Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Wed, 15 Jun 2016 20:23:04 -0700 Subject: [PATCH 074/280] Enable treating warnings as errors in objc tests --- src/objective-c/tests/GRPCClientTests.m | 16 ++++++++-------- src/objective-c/tests/InteropTests.m | 10 ++++++---- .../tests/Tests.xcodeproj/project.pbxproj | 4 ++++ 3 files changed, 18 insertions(+), 12 deletions(-) diff --git a/src/objective-c/tests/GRPCClientTests.m b/src/objective-c/tests/GRPCClientTests.m index 9a8d4253241..2eca7bf5498 100644 --- a/src/objective-c/tests/GRPCClientTests.m +++ b/src/objective-c/tests/GRPCClientTests.m @@ -48,9 +48,9 @@ static NSString * const kPackage = @"grpc.testing"; static NSString * const kService = @"TestService"; static NSString * const kRemoteSSLHost = @"grpc-test.sandbox.googleapis.com"; -static ProtoMethod *kInexistentMethod; -static ProtoMethod *kEmptyCallMethod; -static ProtoMethod *kUnaryCallMethod; +static GRPCProtoMethod *kInexistentMethod; +static GRPCProtoMethod *kEmptyCallMethod; +static GRPCProtoMethod *kUnaryCallMethod; /** Observer class for testing that responseMetadata is KVO-compliant */ @interface PassthroughObserver : NSObject @@ -109,13 +109,13 @@ static ProtoMethod *kUnaryCallMethod; [GRPCCall useInsecureConnectionsForHost:kHostAddress]; // This method isn't implemented by the remote server. - kInexistentMethod = [[ProtoMethod alloc] initWithPackage:kPackage + kInexistentMethod = [[GRPCProtoMethod alloc] initWithPackage:kPackage service:kService method:@"Inexistent"]; - kEmptyCallMethod = [[ProtoMethod alloc] initWithPackage:kPackage + kEmptyCallMethod = [[GRPCProtoMethod alloc] initWithPackage:kPackage service:kService method:@"EmptyCall"]; - kUnaryCallMethod = [[ProtoMethod alloc] initWithPackage:kPackage + kUnaryCallMethod = [[GRPCProtoMethod alloc] initWithPackage:kPackage service:kService method:@"UnaryCall"]; } @@ -303,7 +303,7 @@ static ProtoMethod *kUnaryCallMethod; // Try to set parameters to nil for GRPCCall. This should cause an exception @try { - GRPCCall *call = [[GRPCCall alloc] initWithHost:nil + GRPCCall *call __unused = [[GRPCCall alloc] initWithHost:nil path:nil requestsWriter:nil]; XCTFail(@"Did not receive an exception when parameters are nil"); @@ -316,7 +316,7 @@ static ProtoMethod *kUnaryCallMethod; GRXWriter *requestsWriter = [GRXWriter emptyWriter]; [requestsWriter finishWithError:nil]; @try { - GRPCCall *call = [[GRPCCall alloc] initWithHost:kHostAddress + GRPCCall *call __unused = [[GRPCCall alloc] initWithHost:kHostAddress path:kUnaryCallMethod.HTTPPath requestsWriter:requestsWriter]; XCTFail(@"Did not receive an exception when GRXWriter has incorrect state."); diff --git a/src/objective-c/tests/InteropTests.m b/src/objective-c/tests/InteropTests.m index 781c500f73e..60a83259fa6 100644 --- a/src/objective-c/tests/InteropTests.m +++ b/src/objective-c/tests/InteropTests.m @@ -58,7 +58,7 @@ requestedResponseSize:(NSNumber *)responseSize { RMTStreamingOutputCallRequest *request = [self message]; RMTResponseParameters *parameters = [RMTResponseParameters message]; - parameters.size = responseSize.integerValue; + parameters.size = (int)responseSize.integerValue; [request.responseParametersArray addObject:parameters]; request.payload.body = [NSMutableData dataWithLength:payloadSize.unsignedIntegerValue]; return request; @@ -80,7 +80,9 @@ #pragma mark Tests +#ifdef GRPC_COMPILE_WITH_CRONET static cronet_engine *cronetEngine = NULL; +#endif @implementation InteropTests { RMTTestService *_service; @@ -186,7 +188,7 @@ static cronet_engine *cronetEngine = NULL; RMTStreamingOutputCallRequest *request = [RMTStreamingOutputCallRequest message]; for (NSNumber *size in expectedSizes) { RMTResponseParameters *parameters = [RMTResponseParameters message]; - parameters.size = [size integerValue]; + parameters.size = (int)[size integerValue]; [request.responseParametersArray addObject:parameters]; } @@ -282,7 +284,7 @@ static cronet_engine *cronetEngine = NULL; // A buffered pipe to which we never write any value acts as a writer that just hangs. GRXBufferedPipe *requestsBuffer = [[GRXBufferedPipe alloc] init]; - ProtoRPC *call = [_service RPCToStreamingInputCallWithRequestsWriter:requestsBuffer + GRPCProtoCall *call = [_service RPCToStreamingInputCallWithRequestsWriter:requestsBuffer handler:^(RMTStreamingInputCallResponse *response, NSError *error) { XCTAssertEqual(error.code, GRPC_STATUS_CANCELLED); @@ -313,7 +315,7 @@ static cronet_engine *cronetEngine = NULL; [requestsBuffer writeValue:request]; - __block ProtoRPC *call = + __block GRPCProtoCall *call = [_service RPCToFullDuplexCallWithRequestsWriter:requestsBuffer eventHandler:^(BOOL done, RMTStreamingOutputCallResponse *response, diff --git a/src/objective-c/tests/Tests.xcodeproj/project.pbxproj b/src/objective-c/tests/Tests.xcodeproj/project.pbxproj index b0429617c01..59a6af2e11e 100644 --- a/src/objective-c/tests/Tests.xcodeproj/project.pbxproj +++ b/src/objective-c/tests/Tests.xcodeproj/project.pbxproj @@ -823,6 +823,7 @@ "$(inherited)", ); GCC_SYMBOLS_PRIVATE_EXTERN = NO; + GCC_TREAT_WARNINGS_AS_ERRORS = YES; GCC_WARN_64_TO_32_BIT_CONVERSION = YES; GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; GCC_WARN_UNDECLARED_SELECTOR = YES; @@ -859,6 +860,7 @@ ENABLE_STRICT_OBJC_MSGSEND = YES; GCC_C_LANGUAGE_STANDARD = gnu99; GCC_NO_COMMON_BLOCKS = YES; + GCC_TREAT_WARNINGS_AS_ERRORS = YES; GCC_WARN_64_TO_32_BIT_CONVERSION = YES; GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; GCC_WARN_UNDECLARED_SELECTOR = YES; @@ -875,6 +877,7 @@ 635697DC1B14FC11007A7283 /* Debug */ = { isa = XCBuildConfiguration; buildSettings = { + GCC_TREAT_WARNINGS_AS_ERRORS = YES; PRODUCT_NAME = "$(TARGET_NAME)"; SKIP_INSTALL = YES; }; @@ -883,6 +886,7 @@ 635697DD1B14FC11007A7283 /* Release */ = { isa = XCBuildConfiguration; buildSettings = { + GCC_TREAT_WARNINGS_AS_ERRORS = YES; PRODUCT_NAME = "$(TARGET_NAME)"; SKIP_INSTALL = YES; }; From bb2bd6553938ca1a192414084e5800178f67c4a3 Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Thu, 16 Jun 2016 09:29:53 -0700 Subject: [PATCH 075/280] Clang format --- src/core/lib/surface/completion_queue.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/core/lib/surface/completion_queue.c b/src/core/lib/surface/completion_queue.c index de2b674d0e0..03de97bc2b5 100644 --- a/src/core/lib/surface/completion_queue.c +++ b/src/core/lib/surface/completion_queue.c @@ -93,12 +93,12 @@ static grpc_completion_queue *g_freelist; int grpc_cq_pluck_trace; int grpc_cq_event_timeout_trace; -#define GRPC_SURFACE_TRACE_RETURNED_EVENT(cq, event) \ - if (grpc_api_trace && \ - (grpc_cq_pluck_trace || (event)->type != GRPC_QUEUE_TIMEOUT)) { \ - char *_ev = grpc_event_string(event); \ - gpr_log(GPR_INFO, "RETURN_EVENT[%p]: %s", cq, _ev); \ - gpr_free(_ev); \ +#define GRPC_SURFACE_TRACE_RETURNED_EVENT(cq, event) \ + if (grpc_api_trace && \ + (grpc_cq_pluck_trace || (event)->type != GRPC_QUEUE_TIMEOUT)) { \ + char *_ev = grpc_event_string(event); \ + gpr_log(GPR_INFO, "RETURN_EVENT[%p]: %s", cq, _ev); \ + gpr_free(_ev); \ } From a7cc90055e91b1bda2e916c5884b5305c7f73b53 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 16 Jun 2016 09:59:51 -0700 Subject: [PATCH 076/280] Modified route_guide sample app to show RPC log Modified the screens to show meaningful information about RPC progress. When you click on each tab the screen now shows log messages that previously only went to the debug console. --- .../Misc/Base.lproj/Main.storyboard | 108 ++++++++++-------- .../objective-c/route_guide/ViewControllers.m | 41 +++++++ 2 files changed, 102 insertions(+), 47 deletions(-) diff --git a/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard b/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard index 9bf9498d62c..306320f7d89 100644 --- a/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard +++ b/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard @@ -1,7 +1,8 @@ - + - + + @@ -16,33 +17,35 @@ - - - - - + + + + - + @@ -56,29 +59,35 @@ - + + + - - - - + + + + + @@ -117,29 +126,32 @@ - - - - - + + + + + @@ -157,29 +169,31 @@ - - - - - + + + + diff --git a/examples/objective-c/route_guide/ViewControllers.m b/examples/objective-c/route_guide/ViewControllers.m index e32978240b6..b916a4ee0c4 100644 --- a/examples/objective-c/route_guide/ViewControllers.m +++ b/examples/objective-c/route_guide/ViewControllers.m @@ -83,6 +83,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface GetFeatureViewController : UIViewController { RTGRouteGuide *service; } +@property (weak, nonatomic) IBOutlet UILabel *output_label; @end @implementation GetFeatureViewController @@ -90,10 +91,16 @@ static NSString * const kHostAddress = @"localhost:50051"; - (void)execRequest { void (^handler)(RTGFeature *response, NSError *error) = ^(RTGFeature *response, NSError *error) { if (response.name.length) { + NSString *str =[NSString stringWithFormat:@"%@\nFound feature called %@ at %@.", self.output_label.text, response.location, response.name]; + self.output_label.text = str; NSLog(@"Found feature called %@ at %@.", response.name, response.location); } else if (response) { + NSString *str =[NSString stringWithFormat:@"%@\nFound no features at %@", self.output_label.text,response.location]; + self.output_label.text = str; NSLog(@"Found no features at %@", response.location); } else { + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; + self.output_label.text = str; NSLog(@"RPC error: %@", error); } }; @@ -116,6 +123,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { + self.output_label.text = @"RPC log:"; + self.output_label.numberOfLines = 0; + self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } @@ -131,6 +141,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface ListFeaturesViewController : UIViewController { RTGRouteGuide *service; } +@property (weak, nonatomic) IBOutlet UILabel *output_label; @end @@ -147,8 +158,12 @@ static NSString * const kHostAddress = @"localhost:50051"; [service listFeaturesWithRequest:rectangle eventHandler:^(BOOL done, RTGFeature *response, NSError *error) { if (response) { + NSString *str =[NSString stringWithFormat:@"%@\nFound feature at %@ called %@.", self.output_label.text, response.location, response.name]; + self.output_label.text = str; NSLog(@"Found feature at %@ called %@.", response.location, response.name); } else if (error) { + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; + self.output_label.text = str; NSLog(@"RPC error: %@", error); } }]; @@ -161,6 +176,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { + self.output_label.text = @"RPC log:"; + self.output_label.numberOfLines = 0; + self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } @@ -177,6 +195,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface RecordRouteViewController : UIViewController { RTGRouteGuide *service; } +@property (weak, nonatomic) IBOutlet UILabel *output_label; @end @@ -192,6 +211,8 @@ static NSString * const kHostAddress = @"localhost:50051"; RTGPoint *location = [RTGPoint message]; location.longitude = [((NSNumber *) feature[@"location"][@"longitude"]) intValue]; location.latitude = [((NSNumber *) feature[@"location"][@"latitude"]) intValue]; + NSString *str =[NSString stringWithFormat:@"%@\nVisiting point %@", self.output_label.text, location]; + self.output_label.text = str; NSLog(@"Visiting point %@", location); return location; }]; @@ -199,11 +220,19 @@ static NSString * const kHostAddress = @"localhost:50051"; [service recordRouteWithRequestsWriter:locations handler:^(RTGRouteSummary *response, NSError *error) { if (response) { + NSString *str =[NSString stringWithFormat: + @"%@\nFinished trip with %i points\nPassed %i features\n" + "Travelled %i meters\nIt took %i seconds", + self.output_label.text, response.pointCount, response.featureCount, + response.distance, response.elapsedTime]; + self.output_label.text = str; NSLog(@"Finished trip with %i points", response.pointCount); NSLog(@"Passed %i features", response.featureCount); NSLog(@"Travelled %i meters", response.distance); NSLog(@"It took %i seconds", response.elapsedTime); } else { + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; + self.output_label.text = str; NSLog(@"RPC error: %@", error); } }]; @@ -216,6 +245,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { + self.output_label.text = @"RPC log:"; + self.output_label.numberOfLines = 0; + self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } @@ -231,6 +263,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface RouteChatViewController : UIViewController { RTGRouteGuide *service; } +@property (weak, nonatomic) IBOutlet UILabel *output_label; @end @@ -249,8 +282,13 @@ static NSString * const kHostAddress = @"localhost:50051"; [service routeChatWithRequestsWriter:notesWriter eventHandler:^(BOOL done, RTGRouteNote *note, NSError *error) { if (note) { + NSString *str =[NSString stringWithFormat:@"%@\nGot message %@ at %@", + self.output_label.text, note.message, note.location]; + self.output_label.text = str; NSLog(@"Got message %@ at %@", note.message, note.location); } else if (error) { + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; + self.output_label.text = str; NSLog(@"RPC error: %@", error); } if (done) { @@ -266,6 +304,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { + self.output_label.text = @"RPC log:"; + self.output_label.numberOfLines = 0; + self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } From 3eba9075ef612f83c363868c24d106580ab25223 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 16 Jun 2016 10:15:32 -0700 Subject: [PATCH 077/280] using camelCase for output_label now --- .../Misc/Base.lproj/Main.storyboard | 8 +- .../objective-c/route_guide/ViewControllers.m | 74 +++++++++---------- 2 files changed, 41 insertions(+), 41 deletions(-) diff --git a/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard b/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard index 306320f7d89..5ca9f4642b9 100644 --- a/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard +++ b/examples/objective-c/route_guide/Misc/Base.lproj/Main.storyboard @@ -40,7 +40,7 @@ - + @@ -86,7 +86,7 @@ - + @@ -150,7 +150,7 @@ - + @@ -192,7 +192,7 @@ - + diff --git a/examples/objective-c/route_guide/ViewControllers.m b/examples/objective-c/route_guide/ViewControllers.m index b916a4ee0c4..1e84c334728 100644 --- a/examples/objective-c/route_guide/ViewControllers.m +++ b/examples/objective-c/route_guide/ViewControllers.m @@ -37,7 +37,7 @@ #import #import -static NSString * const kHostAddress = @"localhost:50051"; +static NSString * const kHostAddress = @"192.168.1.120:50051"; /** Category to override RTGPoint's description. */ @interface RTGPoint (Description) @@ -83,7 +83,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface GetFeatureViewController : UIViewController { RTGRouteGuide *service; } -@property (weak, nonatomic) IBOutlet UILabel *output_label; +@property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @implementation GetFeatureViewController @@ -91,16 +91,16 @@ static NSString * const kHostAddress = @"localhost:50051"; - (void)execRequest { void (^handler)(RTGFeature *response, NSError *error) = ^(RTGFeature *response, NSError *error) { if (response.name.length) { - NSString *str =[NSString stringWithFormat:@"%@\nFound feature called %@ at %@.", self.output_label.text, response.location, response.name]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nFound feature called %@ at %@.", self.outputLabel.text, response.location, response.name]; + self.outputLabel.text = str; NSLog(@"Found feature called %@ at %@.", response.name, response.location); } else if (response) { - NSString *str =[NSString stringWithFormat:@"%@\nFound no features at %@", self.output_label.text,response.location]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nFound no features at %@", self.outputLabel.text,response.location]; + self.outputLabel.text = str; NSLog(@"Found no features at %@", response.location); } else { - NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.outputLabel.text, error]; + self.outputLabel.text = str; NSLog(@"RPC error: %@", error); } }; @@ -123,9 +123,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { - self.output_label.text = @"RPC log:"; - self.output_label.numberOfLines = 0; - self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; + self.outputLabel.text = @"RPC log:"; + self.outputLabel.numberOfLines = 0; + self.outputLabel.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } @@ -141,7 +141,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface ListFeaturesViewController : UIViewController { RTGRouteGuide *service; } -@property (weak, nonatomic) IBOutlet UILabel *output_label; +@property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @@ -158,12 +158,12 @@ static NSString * const kHostAddress = @"localhost:50051"; [service listFeaturesWithRequest:rectangle eventHandler:^(BOOL done, RTGFeature *response, NSError *error) { if (response) { - NSString *str =[NSString stringWithFormat:@"%@\nFound feature at %@ called %@.", self.output_label.text, response.location, response.name]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nFound feature at %@ called %@.", self.outputLabel.text, response.location, response.name]; + self.outputLabel.text = str; NSLog(@"Found feature at %@ called %@.", response.location, response.name); } else if (error) { - NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.outputLabel.text, error]; + self.outputLabel.text = str; NSLog(@"RPC error: %@", error); } }]; @@ -176,9 +176,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { - self.output_label.text = @"RPC log:"; - self.output_label.numberOfLines = 0; - self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; + self.outputLabel.text = @"RPC log:"; + self.outputLabel.numberOfLines = 0; + self.outputLabel.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } @@ -195,7 +195,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface RecordRouteViewController : UIViewController { RTGRouteGuide *service; } -@property (weak, nonatomic) IBOutlet UILabel *output_label; +@property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @@ -211,8 +211,8 @@ static NSString * const kHostAddress = @"localhost:50051"; RTGPoint *location = [RTGPoint message]; location.longitude = [((NSNumber *) feature[@"location"][@"longitude"]) intValue]; location.latitude = [((NSNumber *) feature[@"location"][@"latitude"]) intValue]; - NSString *str =[NSString stringWithFormat:@"%@\nVisiting point %@", self.output_label.text, location]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nVisiting point %@", self.outputLabel.text, location]; + self.outputLabel.text = str; NSLog(@"Visiting point %@", location); return location; }]; @@ -223,16 +223,16 @@ static NSString * const kHostAddress = @"localhost:50051"; NSString *str =[NSString stringWithFormat: @"%@\nFinished trip with %i points\nPassed %i features\n" "Travelled %i meters\nIt took %i seconds", - self.output_label.text, response.pointCount, response.featureCount, + self.outputLabel.text, response.pointCount, response.featureCount, response.distance, response.elapsedTime]; - self.output_label.text = str; + self.outputLabel.text = str; NSLog(@"Finished trip with %i points", response.pointCount); NSLog(@"Passed %i features", response.featureCount); NSLog(@"Travelled %i meters", response.distance); NSLog(@"It took %i seconds", response.elapsedTime); } else { - NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.outputLabel.text, error]; + self.outputLabel.text = str; NSLog(@"RPC error: %@", error); } }]; @@ -245,9 +245,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { - self.output_label.text = @"RPC log:"; - self.output_label.numberOfLines = 0; - self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; + self.outputLabel.text = @"RPC log:"; + self.outputLabel.numberOfLines = 0; + self.outputLabel.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } @@ -263,7 +263,7 @@ static NSString * const kHostAddress = @"localhost:50051"; @interface RouteChatViewController : UIViewController { RTGRouteGuide *service; } -@property (weak, nonatomic) IBOutlet UILabel *output_label; +@property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @@ -283,12 +283,12 @@ static NSString * const kHostAddress = @"localhost:50051"; eventHandler:^(BOOL done, RTGRouteNote *note, NSError *error) { if (note) { NSString *str =[NSString stringWithFormat:@"%@\nGot message %@ at %@", - self.output_label.text, note.message, note.location]; - self.output_label.text = str; + self.outputLabel.text, note.message, note.location]; + self.outputLabel.text = str; NSLog(@"Got message %@ at %@", note.message, note.location); } else if (error) { - NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.output_label.text, error]; - self.output_label.text = str; + NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.outputLabel.text, error]; + self.outputLabel.text = str; NSLog(@"RPC error: %@", error); } if (done) { @@ -304,9 +304,9 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { - self.output_label.text = @"RPC log:"; - self.output_label.numberOfLines = 0; - self.output_label.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; + self.outputLabel.text = @"RPC log:"; + self.outputLabel.numberOfLines = 0; + self.outputLabel.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; [self execRequest]; } From 05998dc6939bb7ad4230775bb5ac87696ae9e29b Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Thu, 16 Jun 2016 10:16:44 -0700 Subject: [PATCH 078/280] More formatting fixes --- src/core/lib/surface/completion_queue.c | 1 - 1 file changed, 1 deletion(-) diff --git a/src/core/lib/surface/completion_queue.c b/src/core/lib/surface/completion_queue.c index 03de97bc2b5..d70f1721463 100644 --- a/src/core/lib/surface/completion_queue.c +++ b/src/core/lib/surface/completion_queue.c @@ -101,7 +101,6 @@ int grpc_cq_event_timeout_trace; gpr_free(_ev); \ } - static void on_pollset_shutdown_done(grpc_exec_ctx *exec_ctx, void *cc, bool success); From 4412c85a1079f2ac55a9a27641076ca34c1a2c62 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 16 Jun 2016 10:37:51 -0700 Subject: [PATCH 079/280] crud removal --- examples/objective-c/route_guide/ViewControllers.m | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/objective-c/route_guide/ViewControllers.m b/examples/objective-c/route_guide/ViewControllers.m index 1e84c334728..5fd411e3ba6 100644 --- a/examples/objective-c/route_guide/ViewControllers.m +++ b/examples/objective-c/route_guide/ViewControllers.m @@ -37,7 +37,7 @@ #import #import -static NSString * const kHostAddress = @"192.168.1.120:50051"; +static NSString * const kHostAddress = @"localhost:50051"; /** Category to override RTGPoint's description. */ @interface RTGPoint (Description) From 00f41c1eed57f84a2efdfbd96a59dd00d523cde0 Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Thu, 16 Jun 2016 12:46:52 -0700 Subject: [PATCH 080/280] Add comment about new trace flags --- src/core/lib/surface/completion_queue.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/core/lib/surface/completion_queue.h b/src/core/lib/surface/completion_queue.h index 108445b28f1..e4018695e27 100644 --- a/src/core/lib/surface/completion_queue.h +++ b/src/core/lib/surface/completion_queue.h @@ -39,6 +39,8 @@ #include #include "src/core/lib/iomgr/pollset.h" +/* These trace flags default to 1. The corresponding lines are only traced + if grpc_api_trace is also truthy */ extern int grpc_cq_pluck_trace; extern int grpc_cq_event_timeout_trace; From 474a574ba7ebd425d140c31b264c39b8e48d303d Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 15 Jun 2016 16:17:56 -0700 Subject: [PATCH 081/280] update readme.md --- src/csharp/README.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/csharp/README.md b/src/csharp/README.md index 201c5ab0b56..acfb5ec6ddb 100644 --- a/src/csharp/README.md +++ b/src/csharp/README.md @@ -100,6 +100,12 @@ different languages. tools/run_tests/run_tests.py -l csharp ``` +ON .NET CORE SUPPORT +------------------ + +We are committed to providing full support for [.NET Core](https://dotnet.github.io/) in near future, +but currently, the support is for .NET Core is experimental/work-in-progress. + DOCUMENTATION ------------- - the gRPC C# reference documentation is available online at [grpc.io][] From 34f1b555aef416917a38cdbbb4191d388c6a4c83 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Thu, 16 Jun 2016 16:51:20 -0700 Subject: [PATCH 082/280] better test_case flag help formating --- test/cpp/interop/client.cc | 96 ++++++++++++++++++-------------------- 1 file changed, 46 insertions(+), 50 deletions(-) diff --git a/test/cpp/interop/client.cc b/test/cpp/interop/client.cc index c7d081100e2..e8ae6ee5723 100644 --- a/test/cpp/interop/client.cc +++ b/test/cpp/interop/client.cc @@ -54,34 +54,31 @@ DEFINE_string(server_host, "127.0.0.1", "Server host to connect to"); DEFINE_string(server_host_override, "foo.test.google.fr", "Override the server host which is sent in HTTP header"); DEFINE_string(test_case, "large_unary", - "Configure different test cases. Valid options are: " - "empty_unary : empty (zero bytes) request and response; " - "large_unary : single request and (large) response; " - - "client_compressed_unary : single compressed request; " - "server_compressed_unary : single compressed response; " - - "client_streaming : request streaming with single response; " - "server_streaming : single request with response streaming; " + "Configure different test cases. Valid options are:\n\n" + "all : all test cases;\n" + "cancel_after_begin : cancel stream after starting it;\n" + "cancel_after_first_response: cancel on first response;\n" "client_compressed_streaming : compressed request streaming with " - "single response; " + "client_compressed_unary : single compressed request;\n" + "client_streaming : request streaming with single response;\n" + "compute_engine_creds: large_unary with compute engine auth;\n" + "custom_metadata: server will echo custom metadata;\n" + "empty_stream : bi-di stream with no request/response;\n" + "empty_unary : empty (zero bytes) request and response;\n" + "half_duplex : half-duplex streaming;\n" + "jwt_token_creds: large_unary with JWT token auth;\n" + "large_unary : single request and (large) response;\n" + "oauth2_auth_token: raw oauth2 access token auth;\n" + "per_rpc_creds: raw oauth2 access token on a single rpc;\n" + "ping_pong : full-duplex streaming;\n" + "response streaming;\n" "server_compressed_streaming : single request with compressed " - "response streaming; " - "slow_consumer : single request with response; " - " streaming with slow client consumer; " - "half_duplex : half-duplex streaming; " - "ping_pong : full-duplex streaming; " - "cancel_after_begin : cancel stream after starting it; " - "cancel_after_first_response: cancel on first response; " - "timeout_on_sleeping_server: deadline exceeds on stream; " - "empty_stream : bi-di stream with no request/response; " - "compute_engine_creds: large_unary with compute engine auth; " - "jwt_token_creds: large_unary with JWT token auth; " - "oauth2_auth_token: raw oauth2 access token auth; " - "per_rpc_creds: raw oauth2 access token on a single rpc; " - "status_code_and_message: verify status code & message; " - "custom_metadata: server will echo custom metadata;" - "all : all of above."); + "server_compressed_unary : single compressed response;\n" + "server_streaming : single request with response streaming;\n" + "slow_consumer : single request with response streaming with " + "slow client consumer;\n" + "status_code_and_message: verify status code & message;\n" + "timeout_on_sleeping_server: deadline exceeds on stream;\n"); DEFINE_string(default_service_account, "", "Email of GCE default service account"); DEFINE_string(service_account_key_file, "", @@ -178,30 +175,29 @@ int main(int argc, char** argv) { } // compute_engine_creds only runs in GCE. } else { - const char* testcases[] = - { "all", - "cancel_after_begin", - "cancel_after_first_response", - "client_compressed_streaming", - "client_compressed_unary", - "client_streaming", - "compute_engine_creds", - "custom_metadata", - "empty_stream", - "empty_unary", - "half_duplex", - "jwt_token_creds", - "large_unary", - "oauth2_auth_token", - "oauth2_auth_token", - "per_rpc_creds", - "per_rpc_creds", - "ping_pong", - "server_compressed_streaming", - "server_compressed_unary", - "server_streaming", - "status_code_and_message", - "timeout_on_sleeping_server"}; + const char* testcases[] = {"all", + "cancel_after_begin", + "cancel_after_first_response", + "client_compressed_streaming", + "client_compressed_unary", + "client_streaming", + "compute_engine_creds", + "custom_metadata", + "empty_stream", + "empty_unary", + "half_duplex", + "jwt_token_creds", + "large_unary", + "oauth2_auth_token", + "oauth2_auth_token", + "per_rpc_creds", + "per_rpc_creds", + "ping_pong", + "server_compressed_streaming", + "server_compressed_unary", + "server_streaming", + "status_code_and_message", + "timeout_on_sleeping_server"}; char* joined_testcases = gpr_strjoin_sep(testcases, GPR_ARRAY_SIZE(testcases), "\n", NULL); From 560875239e4ce883b9c02d0bf003ce07d430154b Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Thu, 16 Jun 2016 16:52:11 -0700 Subject: [PATCH 083/280] c++ client & server changes to bring code up to spec --- test/cpp/interop/interop_client.cc | 337 ++++++++++++++++------------- test/cpp/interop/interop_server.cc | 86 ++++---- 2 files changed, 227 insertions(+), 196 deletions(-) diff --git a/test/cpp/interop/interop_client.cc b/test/cpp/interop/interop_client.cc index e5d37514028..509e8b97aba 100644 --- a/test/cpp/interop/interop_client.cc +++ b/test/cpp/interop/interop_client.cc @@ -1,6 +1,6 @@ /* * - * Copyright 2015, Google Inc. + * Copyright 2015-2016, Google Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -68,12 +68,12 @@ const int kLargeResponseSize = 314159; void NoopChecks(const InteropClientContextInspector& inspector, const SimpleRequest* request, const SimpleResponse* response) {} -void CompressionChecks(const InteropClientContextInspector& inspector, - const SimpleRequest* request, - const SimpleResponse* response) { +void UnaryCompressionChecks(const InteropClientContextInspector& inspector, + const SimpleRequest* request, + const SimpleResponse* response) { const grpc_compression_algorithm received_compression = inspector.GetCallCompressionAlgorithm(); - if (request->request_compressed_response()) { + if (request->response_compressed().value()) { if (received_compression == GRPC_COMPRESS_NONE) { // Requested some compression, got NONE. This is an error. gpr_log(GPR_ERROR, @@ -81,11 +81,7 @@ void CompressionChecks(const InteropClientContextInspector& inspector, "from server."); abort(); } - if (request->response_type() == PayloadType::COMPRESSABLE) { - // requested compression and compressable response => results should - // always be compressed. - GPR_ASSERT(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS); - } + GPR_ASSERT(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS); } else { // Didn't request compression -> make sure the response is uncompressed GPR_ASSERT(!(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS)); @@ -190,11 +186,16 @@ bool InteropClient::PerformLargeUnary(SimpleRequest* request, CheckerFn custom_checks_fn) { ClientContext context; InteropClientContextInspector inspector(context); - // If the request doesn't already specify the response type, default to - // COMPRESSABLE. request->set_response_size(kLargeResponseSize); grpc::string payload(kLargeRequestSize, '\0'); request->mutable_payload()->set_body(payload.c_str(), kLargeRequestSize); + if (request->has_expect_compressed()) { + if (request->expect_compressed().value()) { + context.set_compression_algorithm(GRPC_COMPRESS_GZIP); + } else { + context.set_compression_algorithm(GRPC_COMPRESS_NONE); + } + } Status s = serviceStub_.Get()->UnaryCall(&context, *request, response); if (!AssertStatusOk(s)) { @@ -204,16 +205,8 @@ bool InteropClient::PerformLargeUnary(SimpleRequest* request, custom_checks_fn(inspector, request, response); // Payload related checks. - GPR_ASSERT(response->payload().type() == request->response_type()); - switch (response->payload().type()) { - case PayloadType::COMPRESSABLE: - GPR_ASSERT(response->payload().body() == - grpc::string(kLargeResponseSize, '\0')); - break; - default: - GPR_ASSERT(false); - } - + GPR_ASSERT(response->payload().body() == + grpc::string(kLargeResponseSize, '\0')); return true; } @@ -226,7 +219,6 @@ bool InteropClient::DoComputeEngineCreds( SimpleResponse response; request.set_fill_username(true); request.set_fill_oauth_scope(true); - request.set_response_type(PayloadType::COMPRESSABLE); if (!PerformLargeUnary(&request, &response)) { return false; @@ -300,7 +292,6 @@ bool InteropClient::DoJwtTokenCreds(const grpc::string& username) { SimpleRequest request; SimpleResponse response; request.set_fill_username(true); - request.set_response_type(PayloadType::COMPRESSABLE); if (!PerformLargeUnary(&request, &response)) { return false; @@ -316,7 +307,6 @@ bool InteropClient::DoLargeUnary() { gpr_log(GPR_DEBUG, "Sending a large unary rpc..."); SimpleRequest request; SimpleResponse response; - request.set_response_type(PayloadType::COMPRESSABLE); if (!PerformLargeUnary(&request, &response)) { return false; } @@ -325,62 +315,72 @@ bool InteropClient::DoLargeUnary() { } bool InteropClient::DoClientCompressedUnary() { - const bool expect_compression[] = {false, true}; - const PayloadType payload_types[] = {COMPRESSABLE}; - for (size_t i = 0; i < GPR_ARRAY_SIZE(payload_types); i++) { - for (size_t j = 0; j < GPR_ARRAY_SIZE(expect_compression); j++) { - char* log_suffix; - gpr_asprintf(&log_suffix, "(compression=%s; payload=%s)", - expect_compression[j] ? "true" : "false", - PayloadType_Name(payload_types[i]).c_str()); - - gpr_log(GPR_DEBUG, "Sending compressed unary request %s.", log_suffix); - SimpleRequest request; - SimpleResponse response; - request.set_response_type(payload_types[i]); - request.set_expect_compressed_request(expect_compression[j]); - - if (!PerformLargeUnary(&request, &response, CompressionChecks)) { - gpr_log(GPR_ERROR, "Compressed unary request failed %s", log_suffix); - gpr_free(log_suffix); - return false; - } - - gpr_log(GPR_DEBUG, "Compressed unary request failed %s", log_suffix); + // Probing for compression-checks support. + ClientContext probe_context; + SimpleRequest probe_req; + SimpleResponse probe_res; + + probe_context.set_compression_algorithm(GRPC_COMPRESS_NONE); + probe_req.mutable_expect_compressed()->set_value(true); // lies! + + probe_req.set_response_size(kLargeResponseSize); + probe_req.mutable_payload()->set_body(grpc::string(kLargeRequestSize, '\0')); + + gpr_log(GPR_DEBUG, "Sending probe for compressed unary request."); + const Status s = + serviceStub_.Get()->UnaryCall(&probe_context, probe_req, &probe_res); + if (s.error_code() != grpc::StatusCode::INVALID_ARGUMENT) { + // The server isn't able to evaluate incoming compression, making the rest + // of this test moot. + gpr_log(GPR_DEBUG, "Compressed unary request probe failed %s"); + return false; + } + gpr_log(GPR_DEBUG, "Compressed unary request probe succeeded. Proceeding."); + + const std::vector compressions = {true, false}; + for (size_t i = 0; i < compressions.size(); i++) { + char* log_suffix; + gpr_asprintf(&log_suffix, "(compression=%s)", + compressions[i] ? "true" : "false"); + + gpr_log(GPR_DEBUG, "Sending compressed unary request %s.", log_suffix); + SimpleRequest request; + SimpleResponse response; + request.mutable_expect_compressed()->set_value(compressions[i]); + if (!PerformLargeUnary(&request, &response, UnaryCompressionChecks)) { + gpr_log(GPR_ERROR, "Compressed unary request failed %s", log_suffix); gpr_free(log_suffix); + return false; } + + gpr_log(GPR_DEBUG, "Compressed unary request failed %s", log_suffix); + gpr_free(log_suffix); } return true; } bool InteropClient::DoServerCompressedUnary() { - const bool request_compression[] = {false, true}; - const PayloadType payload_types[] = {COMPRESSABLE}; - for (size_t i = 0; i < GPR_ARRAY_SIZE(payload_types); i++) { - for (size_t j = 0; j < GPR_ARRAY_SIZE(request_compression); j++) { - char* log_suffix; - gpr_asprintf(&log_suffix, "(compression=%s; payload=%s)", - request_compression[j] ? "true" : "false", - PayloadType_Name(payload_types[i]).c_str()); - - gpr_log(GPR_DEBUG, "Sending unary request for compressed response %s.", - log_suffix); - SimpleRequest request; - SimpleResponse response; - request.set_response_type(payload_types[i]); - request.set_request_compressed_response(request_compression[j]); - - if (!PerformLargeUnary(&request, &response, CompressionChecks)) { - gpr_log(GPR_ERROR, "Request for compressed unary failed %s", - log_suffix); - gpr_free(log_suffix); - return false; - } - - gpr_log(GPR_DEBUG, "Request for compressed unary failed %s", log_suffix); + const std::vector compressions = {true, false}; + for (size_t i = 0; i < compressions.size(); i++) { + char* log_suffix; + gpr_asprintf(&log_suffix, "(compression=%s)", + compressions[i] ? "true" : "false"); + + gpr_log(GPR_DEBUG, "Sending unary request for compressed response %s.", + log_suffix); + SimpleRequest request; + SimpleResponse response; + request.mutable_response_compressed()->set_value(compressions[i]); + + if (!PerformLargeUnary(&request, &response, UnaryCompressionChecks)) { + gpr_log(GPR_ERROR, "Request for compressed unary failed %s", log_suffix); gpr_free(log_suffix); + return false; } + + gpr_log(GPR_DEBUG, "Request for compressed unary failed %s", log_suffix); + gpr_free(log_suffix); } return true; @@ -407,7 +407,7 @@ bool InteropClient::DoRequestStreaming() { serviceStub_.Get()->StreamingInputCall(&context, &response)); int aggregated_payload_size = 0; - for (unsigned int i = 0; i < request_stream_sizes.size(); ++i) { + for (size_t i = 0; i < request_stream_sizes.size(); ++i) { Payload* payload = request.mutable_payload(); payload->set_body(grpc::string(request_stream_sizes[i], '\0')); if (!stream->Write(request)) { @@ -416,7 +416,7 @@ bool InteropClient::DoRequestStreaming() { } aggregated_payload_size += request_stream_sizes[i]; } - stream->WritesDone(); + GPR_ASSERT(stream->WritesDone()); Status s = stream->Finish(); if (!AssertStatusOk(s)) { @@ -467,94 +467,128 @@ bool InteropClient::DoResponseStreaming() { } bool InteropClient::DoClientCompressedStreaming() { - // XXX - return false; + // Probing for compression-checks support. + ClientContext probe_context; + StreamingInputCallRequest probe_req; + StreamingInputCallResponse probe_res; + + probe_context.set_compression_algorithm(GRPC_COMPRESS_NONE); + probe_req.mutable_expect_compressed()->set_value(true); // lies! + probe_req.mutable_payload()->set_body(grpc::string(27182, '\0')); + + gpr_log(GPR_DEBUG, "Sending probe for compressed streaming request."); + + std::unique_ptr> probe_stream( + serviceStub_.Get()->StreamingInputCall(&probe_context, &probe_res)); + + if (!probe_stream->Write(probe_req)) { + gpr_log(GPR_ERROR, "%s(): stream->Write() failed", __func__); + return TransientFailureOrAbort(); + } + Status s = probe_stream->Finish(); + if (s.error_code() != grpc::StatusCode::INVALID_ARGUMENT) { + // The server isn't able to evaluate incoming compression, making the rest + // of this test moot. + gpr_log(GPR_DEBUG, "Compressed streaming request probe failed %s"); + return false; + } + gpr_log(GPR_DEBUG, + "Compressed streaming request probe succeeded. Proceeding."); + + ClientContext context; + StreamingInputCallRequest request; + StreamingInputCallResponse response; + + context.set_compression_algorithm(GRPC_COMPRESS_GZIP); + std::unique_ptr> stream( + serviceStub_.Get()->StreamingInputCall(&context, &response)); + + request.mutable_payload()->set_body(grpc::string(27182, '\0')); + request.mutable_expect_compressed()->set_value(true); + gpr_log(GPR_DEBUG, "Sending streaming request with compression enabled"); + if (!stream->Write(request)) { + gpr_log(GPR_ERROR, "%s(): stream->Write() failed", __func__); + return TransientFailureOrAbort(); + } + + WriteOptions wopts; + wopts.set_no_compression(); + request.mutable_payload()->set_body(grpc::string(45904, '\0')); + request.mutable_expect_compressed()->set_value(false); + gpr_log(GPR_DEBUG, "Sending streaming request with compression disabled"); + if (!stream->Write(request, wopts)) { + gpr_log(GPR_ERROR, "%s(): stream->Write() failed", __func__); + return TransientFailureOrAbort(); + } + GPR_ASSERT(stream->WritesDone()); + + s = stream->Finish(); + if (!AssertStatusOk(s)) { + return false; + } + + return true; } bool InteropClient::DoServerCompressedStreaming() { - const bool request_compression[] = {false, true}; - const PayloadType payload_types[] = {COMPRESSABLE}; - const std::vector response_stream_sizes = {31415, 58979}; + const std::vector compressions = {true, false}; + const std::vector sizes = {31415, 92653}; - for (size_t i = 0; i < GPR_ARRAY_SIZE(payload_types); i++) { - for (size_t j = 0; j < GPR_ARRAY_SIZE(request_compression); j++) { - ClientContext context; - InteropClientContextInspector inspector(context); - StreamingOutputCallRequest request; + ClientContext context; + InteropClientContextInspector inspector(context); + StreamingOutputCallRequest request; + for (size_t i = 0; i < compressions.size(); i++) { + for (size_t j = 0; j < sizes.size(); j++) { char* log_suffix; - gpr_asprintf(&log_suffix, "(compression=%s; payload=%s)", - request_compression[j] ? "true" : "false", - PayloadType_Name(payload_types[i]).c_str()); - - gpr_log(GPR_DEBUG, "Receiving response streaming rpc %s.", log_suffix); - - request.set_response_type(payload_types[i]); - request.set_request_compressed_response(request_compression[j]); - - for (size_t k = 0; k < response_stream_sizes.size(); ++k) { - ResponseParameters* response_parameter = - request.add_response_parameters(); - response_parameter->set_size(response_stream_sizes[k]); - } - StreamingOutputCallResponse response; - - std::unique_ptr> stream( - serviceStub_.Get()->StreamingOutputCall(&context, request)); - - size_t k = 0; - while (stream->Read(&response)) { - // Payload related checks. - GPR_ASSERT(response.payload().type() == request.response_type()); - switch (response.payload().type()) { - case PayloadType::COMPRESSABLE: - GPR_ASSERT(response.payload().body() == - grpc::string(response_stream_sizes[k], '\0')); - break; - default: - GPR_ASSERT(false); - } - - // Compression related checks. - if (request.request_compressed_response()) { - GPR_ASSERT(inspector.GetCallCompressionAlgorithm() > - GRPC_COMPRESS_NONE); - if (request.response_type() == PayloadType::COMPRESSABLE) { - // requested compression and compressable response => results should - // always be compressed. - GPR_ASSERT(inspector.GetMessageFlags() & - GRPC_WRITE_INTERNAL_COMPRESS); - } - } else { - // requested *no* compression. - GPR_ASSERT( - !(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS)); - } - - ++k; - } - - gpr_log(GPR_DEBUG, "Response streaming done %s.", log_suffix); + gpr_asprintf(&log_suffix, "(compression=%s; size=%d)", + compressions[i] ? "true" : "false", sizes[j]); + + gpr_log(GPR_DEBUG, "Sending request streaming rpc %s.", log_suffix); gpr_free(log_suffix); - if (k < response_stream_sizes.size()) { - // stream->Read() failed before reading all the expected messages. This - // is most likely due to a connection failure. - gpr_log(GPR_ERROR, - "DoServerCompressedStreaming(): Responses read (k=%d) is " - "less than the expected messages (i.e " - "response_stream_sizes.size()/2 (%d)). (i=%d, j=%d)", - k, response_stream_sizes.size(), i, j); - return TransientFailureOrAbort(); - } - - Status s = stream->Finish(); - if (!AssertStatusOk(s)) { - return false; - } + ResponseParameters* const response_parameter = + request.add_response_parameters(); + response_parameter->mutable_compressed()->set_value(compressions[i]); + response_parameter->set_size(sizes[j]); } } + std::unique_ptr> stream( + serviceStub_.Get()->StreamingOutputCall(&context, request)); + + size_t k = 0; + StreamingOutputCallResponse response; + while (stream->Read(&response)) { + // Payload size checks. + GPR_ASSERT(response.payload().body() == + grpc::string(request.response_parameters(k).size(), '\0')); + + // Compression checks. + GPR_ASSERT(request.response_parameters(k).has_compressed()); + if (request.response_parameters(k).compressed().value()) { + GPR_ASSERT(inspector.GetCallCompressionAlgorithm() > GRPC_COMPRESS_NONE); + GPR_ASSERT(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS); + } else { + // requested *no* compression. + GPR_ASSERT(!(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS)); + } + ++k; + } + + if (k < response_stream_sizes.size()) { + // stream->Read() failed before reading all the expected messages. This + // is most likely due to a connection failure. + gpr_log(GPR_ERROR, + "%s(): Responses read (k=%d) is " + "less than the expected number of messages (%d)).", + __func__, k, response_stream_sizes.size()); + return TransientFailureOrAbort(); + } + Status s = stream->Finish(); + if (!AssertStatusOk(s)) { + return false; + } return true; } @@ -655,7 +689,6 @@ bool InteropClient::DoPingPong() { stream(serviceStub_.Get()->FullDuplexCall(&context)); StreamingOutputCallRequest request; - request.set_response_type(PayloadType::COMPRESSABLE); ResponseParameters* response_parameter = request.add_response_parameters(); Payload* payload = request.mutable_payload(); StreamingOutputCallResponse response; @@ -722,7 +755,6 @@ bool InteropClient::DoCancelAfterFirstResponse() { stream(serviceStub_.Get()->FullDuplexCall(&context)); StreamingOutputCallRequest request; - request.set_response_type(PayloadType::COMPRESSABLE); ResponseParameters* response_parameter = request.add_response_parameters(); response_parameter->set_size(31415); request.mutable_payload()->set_body(grpc::string(27182, '\0')); @@ -862,7 +894,6 @@ bool InteropClient::DoCustomMetadata() { stream(serviceStub_.Get()->FullDuplexCall(&context)); StreamingOutputCallRequest request; - request.set_response_type(PayloadType::COMPRESSABLE); ResponseParameters* response_parameter = request.add_response_parameters(); response_parameter->set_size(kLargeResponseSize); grpc::string payload(kLargeRequestSize, '\0'); diff --git a/test/cpp/interop/interop_server.cc b/test/cpp/interop/interop_server.cc index b328f478fa0..8c5c0e24e19 100644 --- a/test/cpp/interop/interop_server.cc +++ b/test/cpp/interop/interop_server.cc @@ -1,6 +1,6 @@ /* * - * Copyright 2015, Google Inc. + * Copyright 2015-2016, Google Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -65,10 +65,10 @@ using grpc::ServerCredentials; using grpc::ServerReader; using grpc::ServerReaderWriter; using grpc::ServerWriter; +using grpc::WriteOptions; using grpc::SslServerCredentialsOptions; using grpc::testing::InteropServerContextInspector; using grpc::testing::Payload; -using grpc::testing::PayloadType; using grpc::testing::SimpleRequest; using grpc::testing::SimpleResponse; using grpc::testing::StreamingInputCallRequest; @@ -110,50 +110,30 @@ void MaybeEchoMetadata(ServerContext* context) { } } -bool SetPayload(PayloadType response_type, int size, Payload* payload) { - payload->set_type(response_type); - switch (response_type) { - case PayloadType::COMPRESSABLE: { - std::unique_ptr body(new char[size]()); - payload->set_body(body.get(), size); - } break; - default: - return false; - } +bool SetPayload(int size, Payload* payload) { + std::unique_ptr body(new char[size]()); + payload->set_body(body.get(), size); return true; } -template -void SetResponseCompression(ServerContext* context, - const RequestType& request) { - if (request.request_compressed_response()) { - // Any level would do, let's go for HIGH because we are overachievers. - context->set_compression_level(GRPC_COMPRESS_LEVEL_HIGH); - } -} - -template bool CheckExpectedCompression(const ServerContext& context, - const RequestType& request) { + const bool compression_expected) { const InteropServerContextInspector inspector(context); const grpc_compression_algorithm received_compression = inspector.GetCallCompressionAlgorithm(); - if (request.expect_compressed_request()) { + if (compression_expected) { if (received_compression == GRPC_COMPRESS_NONE) { // Expected some compression, got NONE. This is an error. gpr_log(GPR_ERROR, - "Failure: Expected compression but got uncompressed request " - "from client."); + "Expected compression but got uncompressed request from client."); return false; } - if (request.payload_type() == PayloadType::COMPRESSABLE) { - if (!(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS)) { - gpr_log(GPR_ERROR, - "Failure: Requested compression in a compressable request, but " - "compression bit in message flags not set."); - return false; - } + if (!(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS)) { + gpr_log(GPR_ERROR, + "Failure: Requested compression in a compressable request, but " + "compression bit in message flags not set."); + return false; } } else { // Didn't expect compression -> make sure the request is uncompressed @@ -178,14 +158,24 @@ class TestServiceImpl : public TestService::Service { Status UnaryCall(ServerContext* context, const SimpleRequest* request, SimpleResponse* response) { MaybeEchoMetadata(context); - SetResponseCompression(context, *request); - if (!CheckExpectedCompression(*context, *request)) { + if (request->has_response_compressed()) { + const bool compression_requested = request->response_compressed().value(); + gpr_log(GPR_DEBUG, "Request for compression (%s) present for %s", + compression_requested ? "enabled" : "disabled", __func__); + if (compression_requested) { + // Any level would do, let's go for HIGH because we are overachievers. + context->set_compression_level(GRPC_COMPRESS_LEVEL_HIGH); + } else { + context->set_compression_level(GRPC_COMPRESS_LEVEL_NONE); + } + } + if (!CheckExpectedCompression(*context, + request->expect_compressed().value())) { return Status(grpc::StatusCode::INVALID_ARGUMENT, "Compressed request expectation not met."); } if (request->response_size() > 0) { - if (!SetPayload(request->response_type(), request->response_size(), - response->mutable_payload())) { + if (!SetPayload(request->response_size(), response->mutable_payload())) { return Status(grpc::StatusCode::INVALID_ARGUMENT, "Error creating payload."); } @@ -203,18 +193,28 @@ class TestServiceImpl : public TestService::Service { Status StreamingOutputCall( ServerContext* context, const StreamingOutputCallRequest* request, ServerWriter* writer) { - SetResponseCompression(context, *request); StreamingOutputCallResponse response; + // Compress by default. Disabled on a per-message basis. + context->set_compression_level(GRPC_COMPRESS_LEVEL_HIGH); bool write_success = true; for (int i = 0; write_success && i < request->response_parameters_size(); i++) { - if (!SetPayload(request->response_type(), - request->response_parameters(i).size(), + if (!SetPayload(request->response_parameters(i).size(), response.mutable_payload())) { return Status(grpc::StatusCode::INVALID_ARGUMENT, "Error creating payload."); } - write_success = writer->Write(response); + WriteOptions wopts; + if (request->response_parameters(i).has_compressed()) { + const bool compression_requested = + request->response_parameters(i).compressed().value(); + gpr_log(GPR_DEBUG, "Request for compression (%s) present for %s", + compression_requested ? "enabled" : "disabled", __func__); + if (!compression_requested) { + wopts.set_no_compression(); + } // else, compression is already enabled via the context. + } + write_success = writer->Write(response, wopts); } if (write_success) { return Status::OK; @@ -229,7 +229,8 @@ class TestServiceImpl : public TestService::Service { StreamingInputCallRequest request; int aggregated_payload_size = 0; while (reader->Read(&request)) { - if (!CheckExpectedCompression(*context, request)) { + if (!CheckExpectedCompression(*context, + request.expect_compressed().value())) { return Status(grpc::StatusCode::INVALID_ARGUMENT, "Compressed request expectation not met."); } @@ -250,7 +251,6 @@ class TestServiceImpl : public TestService::Service { StreamingOutputCallResponse response; bool write_success = true; while (write_success && stream->Read(&request)) { - SetResponseCompression(context, request); if (request.response_parameters_size() != 0) { response.mutable_payload()->set_type(request.payload().type()); response.mutable_payload()->set_body( From 317b8acf995ef3360df4c04ba180355e6b62e59c Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Thu, 16 Jun 2016 09:36:11 -0700 Subject: [PATCH 084/280] migrate everything to netstandard1.5 --- src/csharp/Grpc.Auth/project.json | 8 ++--- .../Grpc.Core.Tests/AppDomainUnloadTest.cs | 2 +- src/csharp/Grpc.Core.Tests/NUnitMain.cs | 2 +- src/csharp/Grpc.Core.Tests/SanityTest.cs | 3 +- src/csharp/Grpc.Core.Tests/project.json | 15 ++++---- src/csharp/Grpc.Core/GrpcEnvironment.cs | 2 +- .../Grpc.Core/Internal/NativeExtension.cs | 4 +-- src/csharp/Grpc.Core/Internal/PlatformApis.cs | 2 +- src/csharp/Grpc.Core/project.json | 17 ++-------- .../Grpc.Examples.MathClient/project.json | 15 ++++---- .../Grpc.Examples.MathServer/project.json | 15 ++++---- src/csharp/Grpc.Examples.Tests/NUnitMain.cs | 2 +- src/csharp/Grpc.Examples.Tests/project.json | 15 ++++---- src/csharp/Grpc.Examples/project.json | 12 +++---- .../Grpc.HealthCheck.Tests/NUnitMain.cs | 2 +- .../Grpc.HealthCheck.Tests/project.json | 15 ++++---- src/csharp/Grpc.HealthCheck/project.json | 4 +-- .../project.json | 19 +++++++++-- .../project.json | 19 +++++++++-- .../project.json | 19 +++++++++-- .../GeneratedClientTest.cs | 12 ++++--- .../GeneratedServiceBaseTest.cs | 1 - .../Grpc.IntegrationTesting/InteropClient.cs | 32 ++++++++++++++--- .../MetadataCredentialsTest.cs | 21 +++++------- .../Grpc.IntegrationTesting/NUnitMain.cs | 2 +- .../TestCredentials.cs | 2 +- .../Grpc.IntegrationTesting/project.json | 34 +++++++++++++++++-- 27 files changed, 198 insertions(+), 98 deletions(-) diff --git a/src/csharp/Grpc.Auth/project.json b/src/csharp/Grpc.Auth/project.json index 25579982a9d..57539f2976b 100644 --- a/src/csharp/Grpc.Auth/project.json +++ b/src/csharp/Grpc.Auth/project.json @@ -18,14 +18,14 @@ }, "frameworks": { "net45": { }, - "dotnet54": { + "netstandard1.5": { "imports": [ "net45" ], "dependencies": { - "Microsoft.CSharp": "4.0.1-beta-23516", - "Microsoft.NETCore.Portable.Compatibility": "1.0.1-beta-23516", - "System.Threading.Tasks": "4.0.11-beta-23516" + "Microsoft.NETCore.Portable.Compatibility": "1.0.1-rc2-24027", + "NETStandard.Library": "1.5.0-rc2-24027", + "System.Threading.Tasks": "4.0.11-rc2-24027" } } } diff --git a/src/csharp/Grpc.Core.Tests/AppDomainUnloadTest.cs b/src/csharp/Grpc.Core.Tests/AppDomainUnloadTest.cs index d2c641cf2f8..064bc13cabb 100644 --- a/src/csharp/Grpc.Core.Tests/AppDomainUnloadTest.cs +++ b/src/csharp/Grpc.Core.Tests/AppDomainUnloadTest.cs @@ -40,7 +40,7 @@ namespace Grpc.Core.Tests { public class AppDomainUnloadTest { -#if DOTNET5_4 +#if NETSTANDARD1_5 [Test] [Ignore("Not supported for CoreCLR")] public void AppDomainUnloadHookCanCleanupAbandonedCall() diff --git a/src/csharp/Grpc.Core.Tests/NUnitMain.cs b/src/csharp/Grpc.Core.Tests/NUnitMain.cs index 9c1d7bf3c8a..24a9f846d10 100644 --- a/src/csharp/Grpc.Core.Tests/NUnitMain.cs +++ b/src/csharp/Grpc.Core.Tests/NUnitMain.cs @@ -49,7 +49,7 @@ namespace Grpc.Core.Tests { // Make logger immune to NUnit capturing stdout and stderr to workaround https://github.com/nunit/nunit/issues/1406. GrpcEnvironment.SetLogger(new TextWriterLogger(Console.Error)); -#if DOTNET5_4 +#if NETSTANDARD1_5 return new AutoRun(typeof(NUnitMain).GetTypeInfo().Assembly).Execute(args, new ExtendedTextWrapper(Console.Out), Console.In); #else return new AutoRun().Execute(args); diff --git a/src/csharp/Grpc.Core.Tests/SanityTest.cs b/src/csharp/Grpc.Core.Tests/SanityTest.cs index 9e995d40c03..501992c5695 100644 --- a/src/csharp/Grpc.Core.Tests/SanityTest.cs +++ b/src/csharp/Grpc.Core.Tests/SanityTest.cs @@ -45,7 +45,8 @@ namespace Grpc.Core.Tests { public class SanityTest { -#if !DOTNET5_4 + // TODO: make sanity test work for CoreCLR as well +#if !NETSTANDARD1_5 /// /// Because we depend on a native library, sometimes when things go wrong, the /// entire NUnit test process crashes. To be able to track down problems better, diff --git a/src/csharp/Grpc.Core.Tests/project.json b/src/csharp/Grpc.Core.Tests/project.json index dc90e04ccfb..06718043142 100644 --- a/src/csharp/Grpc.Core.Tests/project.json +++ b/src/csharp/Grpc.Core.Tests/project.json @@ -10,23 +10,26 @@ "emitEntryPoint": true }, "dependencies": { - "Grpc.Core": "0.14.0-anexperiment", + "Grpc.Core": { + "version": "0.0.1", + "taget": "project" + }, "Newtonsoft.Json": "8.0.3", "NUnit": "3.2.0", "NUnitLite": "3.2.0-*" }, "frameworks": { "net45": { }, - "dotnet54": { + "netstandard1.5": { "imports": [ "portable-net45" ], "dependencies": { - "Microsoft.NETCore.App": { - "type": "platform", - "version": "1.0.0-rc2-23931" - } + "NETStandard.Library": "1.5.0-rc2-24027" } } + }, + "runtimes": { + "win7-x64": { } } } diff --git a/src/csharp/Grpc.Core/GrpcEnvironment.cs b/src/csharp/Grpc.Core/GrpcEnvironment.cs index 37f9dce7c2f..f4642c91577 100644 --- a/src/csharp/Grpc.Core/GrpcEnvironment.cs +++ b/src/csharp/Grpc.Core/GrpcEnvironment.cs @@ -353,7 +353,7 @@ namespace Grpc.Core if (!hooksRegistered) { // TODO(jtattermusch): register shutdownhooks for CoreCLR as well -#if !DOTNET5_4 +#if !NETSTANDARD1_5 AppDomain.CurrentDomain.ProcessExit += ShutdownHookHandler; AppDomain.CurrentDomain.DomainUnload += ShutdownHookHandler; diff --git a/src/csharp/Grpc.Core/Internal/NativeExtension.cs b/src/csharp/Grpc.Core/Internal/NativeExtension.cs index 6c219621df5..a6d79258162 100644 --- a/src/csharp/Grpc.Core/Internal/NativeExtension.cs +++ b/src/csharp/Grpc.Core/Internal/NativeExtension.cs @@ -117,7 +117,7 @@ namespace Grpc.Core.Internal private static string GetAssemblyPath() { var assembly = typeof(NativeExtension).GetTypeInfo().Assembly; -#if DOTNET5_4 +#if NETSTANDARD1_5 // Assembly.EscapedCodeBase does not exist under CoreCLR, but assemblies imported from a nuget package // don't seem to be shadowed by DNX-based projects at all. return assembly.Location; @@ -136,7 +136,7 @@ namespace Grpc.Core.Internal #endif } -#if !DOTNET5_4 +#if !NETSTANDARD1_5 private static bool IsFileUri(string uri) { return uri.ToLowerInvariant().StartsWith(Uri.UriSchemeFile); diff --git a/src/csharp/Grpc.Core/Internal/PlatformApis.cs b/src/csharp/Grpc.Core/Internal/PlatformApis.cs index 3331a1f9fac..15391ddc647 100644 --- a/src/csharp/Grpc.Core/Internal/PlatformApis.cs +++ b/src/csharp/Grpc.Core/Internal/PlatformApis.cs @@ -53,7 +53,7 @@ namespace Grpc.Core.Internal static PlatformApis() { -#if DOTNET5_4 +#if NETSTANDARD1_5 isLinux = RuntimeInformation.IsOSPlatform(OSPlatform.Linux); isMacOSX = RuntimeInformation.IsOSPlatform(OSPlatform.OSX); isWindows = RuntimeInformation.IsOSPlatform(OSPlatform.Windows); diff --git a/src/csharp/Grpc.Core/project.json b/src/csharp/Grpc.Core/project.json index a916bbd09ab..b5b7722bba2 100644 --- a/src/csharp/Grpc.Core/project.json +++ b/src/csharp/Grpc.Core/project.json @@ -30,24 +30,13 @@ }, "frameworks": { "net45": { }, - "dotnet54": { + "netstandard1.5": { "imports": [ "portable-net45" ], "dependencies": { - "Microsoft.CSharp": "4.0.1-beta-23516", - "System.Collections": "4.0.11-beta-23516", - "System.Collections.Concurrent": "4.0.11-beta-23516", - "System.Console": "4.0.0-beta-23516", - "System.Linq": "4.0.1-beta-23516", - "System.Threading": "4.0.11-beta-23516", - "System.Threading.Thread": "4.0.0-beta-23516", - "System.Reflection": "4.1.0-beta-23516", - "System.Text.Encoding": "4.0.11-beta-23516", - "System.Text.RegularExpressions": "4.0.11-beta-23516", - "System.IO": "4.0.11-beta-23516", - "System.IO.FileSystem": "4.0.1-beta-23516", - "System.Runtime.InteropServices.RuntimeInformation": "4.0.0-beta-23516" + "NETStandard.Library": "1.5.0-rc2-24027", + "System.Threading.Thread": "4.0.0-rc2-24027" } } } diff --git a/src/csharp/Grpc.Examples.MathClient/project.json b/src/csharp/Grpc.Examples.MathClient/project.json index 4abf8a5e34f..791cd1dcb80 100644 --- a/src/csharp/Grpc.Examples.MathClient/project.json +++ b/src/csharp/Grpc.Examples.MathClient/project.json @@ -10,20 +10,23 @@ "emitEntryPoint": true }, "dependencies": { - "Grpc.Examples": "1.0.0" + "Grpc.Examples": { + "version": "1.0.0", + "taget": "project" + } }, "frameworks": { "net45": { }, - "dotnet54": { + "netstandard1.5": { "imports": [ "portable-net45" ], "dependencies": { - "Microsoft.NETCore.App": { - "type": "platform", - "version": "1.0.0-rc2-23931" - } + "NETStandard.Library": "1.5.0-rc2-24027" } } + }, + "runtimes": { + "win7-x64": { } } } diff --git a/src/csharp/Grpc.Examples.MathServer/project.json b/src/csharp/Grpc.Examples.MathServer/project.json index 4abf8a5e34f..791cd1dcb80 100644 --- a/src/csharp/Grpc.Examples.MathServer/project.json +++ b/src/csharp/Grpc.Examples.MathServer/project.json @@ -10,20 +10,23 @@ "emitEntryPoint": true }, "dependencies": { - "Grpc.Examples": "1.0.0" + "Grpc.Examples": { + "version": "1.0.0", + "taget": "project" + } }, "frameworks": { "net45": { }, - "dotnet54": { + "netstandard1.5": { "imports": [ "portable-net45" ], "dependencies": { - "Microsoft.NETCore.App": { - "type": "platform", - "version": "1.0.0-rc2-23931" - } + "NETStandard.Library": "1.5.0-rc2-24027" } } + }, + "runtimes": { + "win7-x64": { } } } diff --git a/src/csharp/Grpc.Examples.Tests/NUnitMain.cs b/src/csharp/Grpc.Examples.Tests/NUnitMain.cs index ea87802766b..1a522cab932 100644 --- a/src/csharp/Grpc.Examples.Tests/NUnitMain.cs +++ b/src/csharp/Grpc.Examples.Tests/NUnitMain.cs @@ -49,7 +49,7 @@ namespace Grpc.Examples.Tests { // Make logger immune to NUnit capturing stdout and stderr to workaround https://github.com/nunit/nunit/issues/1406. GrpcEnvironment.SetLogger(new TextWriterLogger(Console.Error)); -#if DOTNET5_4 +#if NETSTANDARD1_5 return new AutoRun(typeof(NUnitMain).GetTypeInfo().Assembly).Execute(args, new ExtendedTextWrapper(Console.Out), Console.In); #else return new AutoRun().Execute(args); diff --git a/src/csharp/Grpc.Examples.Tests/project.json b/src/csharp/Grpc.Examples.Tests/project.json index bd74812c153..6ef440e27dd 100644 --- a/src/csharp/Grpc.Examples.Tests/project.json +++ b/src/csharp/Grpc.Examples.Tests/project.json @@ -10,22 +10,25 @@ "emitEntryPoint": true }, "dependencies": { - "Grpc.Examples": "1.0.0", + "Grpc.Examples": { + "version": "1.0.0", + "taget": "project" + }, "NUnit": "3.2.0", "NUnitLite": "3.2.0-*" }, "frameworks": { "net45": { }, - "dotnet54": { + "netstandard1.5": { "imports": [ "portable-net45" ], "dependencies": { - "Microsoft.NETCore.App": { - "type": "platform", - "version": "1.0.0-rc2-23931" - } + "NETStandard.Library": "1.5.0-rc2-24027" } } + }, + "runtimes": { + "win7-x64": { } } } diff --git a/src/csharp/Grpc.Examples/project.json b/src/csharp/Grpc.Examples/project.json index e2b4c10422f..4a3810596b0 100644 --- a/src/csharp/Grpc.Examples/project.json +++ b/src/csharp/Grpc.Examples/project.json @@ -4,7 +4,10 @@ }, "dependencies": { - "Grpc.Core": "0.0.1", + "Grpc.Core": { + "version": "0.0.1", + "taget": "project" + }, "Google.Protobuf": "3.0.0-beta3" }, "frameworks": { @@ -14,15 +17,12 @@ "System.IO": "" } }, - "dotnet54": { + "netstandard1.5": { "imports": [ "portable-net45" ], "dependencies": { - "Microsoft.NETCore.App": { - "type": "platform", - "version": "1.0.0-rc2-23931" - } + "NETStandard.Library": "1.5.0-rc2-24027" } } } diff --git a/src/csharp/Grpc.HealthCheck.Tests/NUnitMain.cs b/src/csharp/Grpc.HealthCheck.Tests/NUnitMain.cs index 0820523f35c..44634671ce5 100644 --- a/src/csharp/Grpc.HealthCheck.Tests/NUnitMain.cs +++ b/src/csharp/Grpc.HealthCheck.Tests/NUnitMain.cs @@ -49,7 +49,7 @@ namespace Grpc.HealthCheck.Tests { // Make logger immune to NUnit capturing stdout and stderr to workaround https://github.com/nunit/nunit/issues/1406. GrpcEnvironment.SetLogger(new TextWriterLogger(Console.Error)); -#if DOTNET5_4 +#if NETSTANDARD1_5 return new AutoRun(typeof(NUnitMain).GetTypeInfo().Assembly).Execute(args, new ExtendedTextWrapper(Console.Out), Console.In); #else return new AutoRun().Execute(args); diff --git a/src/csharp/Grpc.HealthCheck.Tests/project.json b/src/csharp/Grpc.HealthCheck.Tests/project.json index 248a1324f60..fde21f1b7e0 100644 --- a/src/csharp/Grpc.HealthCheck.Tests/project.json +++ b/src/csharp/Grpc.HealthCheck.Tests/project.json @@ -10,22 +10,25 @@ "emitEntryPoint": true }, "dependencies": { - "Grpc.HealthCheck": "0.0.1", + "Grpc.HealthCheck": { + "version": "0.0.1", + "taget": "project" + }, "NUnit": "3.2.0", "NUnitLite": "3.2.0-*" }, "frameworks": { "net45": { }, - "dotnet54": { + "netstandard1.5": { "imports": [ "portable-net45" ], "dependencies": { - "Microsoft.NETCore.App": { - "type": "platform", - "version": "1.0.0-rc2-23931" - } + "NETStandard.Library": "1.5.0-rc2-24027" } } + }, + "runtimes": { + "win7-x64": { } } } diff --git a/src/csharp/Grpc.HealthCheck/project.json b/src/csharp/Grpc.HealthCheck/project.json index ad42df595d4..c69db5f997d 100644 --- a/src/csharp/Grpc.HealthCheck/project.json +++ b/src/csharp/Grpc.HealthCheck/project.json @@ -26,12 +26,12 @@ "System.IO": "" } }, - "dotnet54": { + "netstandard1.5": { "imports": [ "portable-net45" ], "dependencies": { - "Microsoft.CSharp": "4.0.1-beta-23516" + "NETStandard.Library": "1.5.0-rc2-24027" } } } diff --git a/src/csharp/Grpc.IntegrationTesting.Client/project.json b/src/csharp/Grpc.IntegrationTesting.Client/project.json index 6ad74d59984..a5a87f9882d 100644 --- a/src/csharp/Grpc.IntegrationTesting.Client/project.json +++ b/src/csharp/Grpc.IntegrationTesting.Client/project.json @@ -10,9 +10,24 @@ "emitEntryPoint": true }, "dependencies": { - "Grpc.IntegrationTesting": "1.0.0" + "Grpc.IntegrationTesting": { + "version": "1.0.0", + "taget": "project" + } }, "frameworks": { - "net45": { } + "net45": { }, + "netstandard1.5": { + "imports": [ + "portable-net45", + "net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027" + } + } + }, + "runtimes": { + "win7-x64": { } } } diff --git a/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json b/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json index 6ad74d59984..a5a87f9882d 100644 --- a/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json +++ b/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json @@ -10,9 +10,24 @@ "emitEntryPoint": true }, "dependencies": { - "Grpc.IntegrationTesting": "1.0.0" + "Grpc.IntegrationTesting": { + "version": "1.0.0", + "taget": "project" + } }, "frameworks": { - "net45": { } + "net45": { }, + "netstandard1.5": { + "imports": [ + "portable-net45", + "net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027" + } + } + }, + "runtimes": { + "win7-x64": { } } } diff --git a/src/csharp/Grpc.IntegrationTesting.Server/project.json b/src/csharp/Grpc.IntegrationTesting.Server/project.json index 6ad74d59984..a5a87f9882d 100644 --- a/src/csharp/Grpc.IntegrationTesting.Server/project.json +++ b/src/csharp/Grpc.IntegrationTesting.Server/project.json @@ -10,9 +10,24 @@ "emitEntryPoint": true }, "dependencies": { - "Grpc.IntegrationTesting": "1.0.0" + "Grpc.IntegrationTesting": { + "version": "1.0.0", + "taget": "project" + } }, "frameworks": { - "net45": { } + "net45": { }, + "netstandard1.5": { + "imports": [ + "portable-net45", + "net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027" + } + } + }, + "runtimes": { + "win7-x64": { } } } diff --git a/src/csharp/Grpc.IntegrationTesting/GeneratedClientTest.cs b/src/csharp/Grpc.IntegrationTesting/GeneratedClientTest.cs index 37786b6c30a..eb7b55a2863 100644 --- a/src/csharp/Grpc.IntegrationTesting/GeneratedClientTest.cs +++ b/src/csharp/Grpc.IntegrationTesting/GeneratedClientTest.cs @@ -40,7 +40,6 @@ using System.Threading.Tasks; using Grpc.Core; using Grpc.Core.Utils; using Grpc.Testing; -using Moq; using NUnit.Framework; namespace Grpc.IntegrationTesting @@ -49,14 +48,16 @@ namespace Grpc.IntegrationTesting { TestService.TestServiceClient unimplementedClient = new UnimplementedTestServiceClient(); + // TODO: replace Moq by some mocking library with CoreCLR support. +#if !NETSTANDARD1_5 [Test] public void ExpandedParamOverloadCanBeMocked() { var expected = new SimpleResponse(); - var mockClient = new Mock(); + var mockClient = new Moq.Mock(); // mocking is relatively clumsy because one needs to specify value for all the optional params. - mockClient.Setup(m => m.UnaryCall(It.IsAny(), null, null, CancellationToken.None)).Returns(expected); + mockClient.Setup(m => m.UnaryCall(Moq.It.IsAny(), null, null, CancellationToken.None)).Returns(expected); Assert.AreSame(expected, mockClient.Object.UnaryCall(new SimpleRequest())); } @@ -66,11 +67,12 @@ namespace Grpc.IntegrationTesting { var expected = new SimpleResponse(); - var mockClient = new Mock(); - mockClient.Setup(m => m.UnaryCall(It.IsAny(), It.IsAny())).Returns(expected); + var mockClient = new Moq.Mock(); + mockClient.Setup(m => m.UnaryCall(Moq.It.IsAny(), Moq.It.IsAny())).Returns(expected); Assert.AreSame(expected, mockClient.Object.UnaryCall(new SimpleRequest(), new CallOptions())); } +#endif [Test] public void DefaultMethodStubThrows_UnaryCall() diff --git a/src/csharp/Grpc.IntegrationTesting/GeneratedServiceBaseTest.cs b/src/csharp/Grpc.IntegrationTesting/GeneratedServiceBaseTest.cs index 5fd0e14e78d..001533ce315 100644 --- a/src/csharp/Grpc.IntegrationTesting/GeneratedServiceBaseTest.cs +++ b/src/csharp/Grpc.IntegrationTesting/GeneratedServiceBaseTest.cs @@ -40,7 +40,6 @@ using System.Threading.Tasks; using Grpc.Core; using Grpc.Core.Utils; using Grpc.Testing; -using Moq; using NUnit.Framework; namespace Grpc.IntegrationTesting diff --git a/src/csharp/Grpc.IntegrationTesting/InteropClient.cs b/src/csharp/Grpc.IntegrationTesting/InteropClient.cs index aea40afee2c..d273867a6a7 100644 --- a/src/csharp/Grpc.IntegrationTesting/InteropClient.cs +++ b/src/csharp/Grpc.IntegrationTesting/InteropClient.cs @@ -145,16 +145,26 @@ namespace Grpc.IntegrationTesting if (options.TestCase == "jwt_token_creds") { +#if !NETSTANDARD1_5 var googleCredential = await GoogleCredential.GetApplicationDefaultAsync(); Assert.IsTrue(googleCredential.IsCreateScopedRequired); credentials = ChannelCredentials.Create(credentials, googleCredential.ToCallCredentials()); +#else + // TODO(jtattermusch): implement this + throw new NotImplementedException("Not supported on CoreCLR yet"); +#endif } if (options.TestCase == "compute_engine_creds") { +#if !NETSTANDARD1_5 var googleCredential = await GoogleCredential.GetApplicationDefaultAsync(); Assert.IsFalse(googleCredential.IsCreateScopedRequired); credentials = ChannelCredentials.Create(credentials, googleCredential.ToCallCredentials()); +#else + // TODO(jtattermusch): implement this + throw new NotImplementedException("Not supported on CoreCLR yet"); +#endif } return credentials; } @@ -245,7 +255,7 @@ namespace Grpc.IntegrationTesting { Console.WriteLine("running client_streaming"); - var bodySizes = new List { 27182, 8, 1828, 45904 }.ConvertAll((size) => new StreamingInputCallRequest { Payload = CreateZerosPayload(size) }); + var bodySizes = new List { 27182, 8, 1828, 45904 }.Select((size) => new StreamingInputCallRequest { Payload = CreateZerosPayload(size) }); using (var call = client.StreamingInputCall()) { @@ -266,7 +276,7 @@ namespace Grpc.IntegrationTesting var request = new StreamingOutputCallRequest { ResponseType = PayloadType.Compressable, - ResponseParameters = { bodySizes.ConvertAll((size) => new ResponseParameters { Size = size }) } + ResponseParameters = { bodySizes.Select((size) => new ResponseParameters { Size = size }) } }; using (var call = client.StreamingOutputCall(request)) @@ -276,7 +286,7 @@ namespace Grpc.IntegrationTesting { Assert.AreEqual(PayloadType.Compressable, res.Payload.Type); } - CollectionAssert.AreEqual(bodySizes, responseList.ConvertAll((item) => item.Payload.Body.Length)); + CollectionAssert.AreEqual(bodySizes, responseList.Select((item) => item.Payload.Body.Length)); } Console.WriteLine("Passed!"); } @@ -398,6 +408,7 @@ namespace Grpc.IntegrationTesting public static async Task RunOAuth2AuthTokenAsync(TestService.TestServiceClient client, string oauthScope) { +#if !NETSTANDARD1_5 Console.WriteLine("running oauth2_auth_token"); ITokenAccess credential = (await GoogleCredential.GetApplicationDefaultAsync()).CreateScoped(new[] { oauthScope }); string oauth2Token = await credential.GetAccessTokenForRequestAsync(); @@ -415,10 +426,15 @@ namespace Grpc.IntegrationTesting Assert.True(oauthScope.Contains(response.OauthScope)); Assert.AreEqual(GetEmailFromServiceAccountFile(), response.Username); Console.WriteLine("Passed!"); +#else + // TODO(jtattermusch): implement this + throw new NotImplementedException("Not supported on CoreCLR yet"); +#endif } public static async Task RunPerRpcCredsAsync(TestService.TestServiceClient client, string oauthScope) { +#if !NETSTANDARD1_5 Console.WriteLine("running per_rpc_creds"); ITokenAccess googleCredential = await GoogleCredential.GetApplicationDefaultAsync(); @@ -432,6 +448,10 @@ namespace Grpc.IntegrationTesting Assert.AreEqual(GetEmailFromServiceAccountFile(), response.Username); Console.WriteLine("Passed!"); +#else + // TODO(jtattermusch): implement this + throw new NotImplementedException("Not supported on CoreCLR yet"); +#endif } public static async Task RunCancelAfterBeginAsync(TestService.TestServiceClient client) @@ -626,13 +646,17 @@ namespace Grpc.IntegrationTesting // extracts the client_email field from service account file used for auth test cases private static string GetEmailFromServiceAccountFile() { +#if !NETSTANDARD1_5 string keyFile = Environment.GetEnvironmentVariable("GOOGLE_APPLICATION_CREDENTIALS"); Assert.IsNotNull(keyFile); - var jobject = JObject.Parse(File.ReadAllText(keyFile)); string email = jobject.GetValue("client_email").Value(); Assert.IsTrue(email.Length > 0); // spec requires nonempty client email. return email; +#else + // TODO(jtattermusch): implement this + throw new NotImplementedException("Not supported on CoreCLR yet"); +#endif } private static Metadata CreateTestMetadata() diff --git a/src/csharp/Grpc.IntegrationTesting/MetadataCredentialsTest.cs b/src/csharp/Grpc.IntegrationTesting/MetadataCredentialsTest.cs index f95af5008f1..9fd575f1900 100644 --- a/src/csharp/Grpc.IntegrationTesting/MetadataCredentialsTest.cs +++ b/src/csharp/Grpc.IntegrationTesting/MetadataCredentialsTest.cs @@ -40,7 +40,6 @@ using System.Threading.Tasks; using Grpc.Core; using Grpc.Core.Utils; using Grpc.Testing; -using Moq; using NUnit.Framework; namespace Grpc.IntegrationTesting @@ -52,19 +51,14 @@ namespace Grpc.IntegrationTesting Channel channel; TestService.TestServiceClient client; List options; - Mock serviceMock; AsyncAuthInterceptor asyncAuthInterceptor; [SetUp] public void Init() { - serviceMock = new Mock(); - serviceMock.Setup(m => m.UnaryCall(It.IsAny(), It.IsAny())) - .Returns(new Func>(UnaryCallHandler)); - server = new Server { - Services = { TestService.BindService(serviceMock.Object) }, + Services = { TestService.BindService(new FakeTestService()) }, Ports = { { Host, ServerPort.PickUnused, TestCredentials.CreateSslServerCredentials() } } }; server.Start(); @@ -96,7 +90,7 @@ namespace Grpc.IntegrationTesting channel = new Channel(Host, server.Ports.Single().BoundPort, channelCredentials, options); client = TestService.NewClient(channel); - client.UnaryCall(new SimpleRequest {}); + client.UnaryCall(new SimpleRequest { }); } [Test] @@ -109,11 +103,14 @@ namespace Grpc.IntegrationTesting client.UnaryCall(new SimpleRequest { }, new CallOptions(credentials: callCredentials)); } - private Task UnaryCallHandler(SimpleRequest request, ServerCallContext context) + private class FakeTestService : TestService.TestServiceBase { - var authToken = context.RequestHeaders.First((entry) => entry.Key == "authorization").Value; - Assert.AreEqual("SECRET_TOKEN", authToken); - return Task.FromResult(new SimpleResponse()); + public override Task UnaryCall(SimpleRequest request, ServerCallContext context) + { + var authToken = context.RequestHeaders.First((entry) => entry.Key == "authorization").Value; + Assert.AreEqual("SECRET_TOKEN", authToken); + return Task.FromResult(new SimpleResponse()); + } } } } diff --git a/src/csharp/Grpc.IntegrationTesting/NUnitMain.cs b/src/csharp/Grpc.IntegrationTesting/NUnitMain.cs index d8902de08f5..100ff0b5de9 100644 --- a/src/csharp/Grpc.IntegrationTesting/NUnitMain.cs +++ b/src/csharp/Grpc.IntegrationTesting/NUnitMain.cs @@ -49,7 +49,7 @@ namespace Grpc.IntegrationTesting { // Make logger immune to NUnit capturing stdout and stderr to workaround https://github.com/nunit/nunit/issues/1406. GrpcEnvironment.SetLogger(new TextWriterLogger(Console.Error)); -#if DOTNET5_4 +#if NETSTANDARD1_5 return new AutoRun(typeof(NUnitMain).GetTypeInfo().Assembly).Execute(args, new ExtendedTextWrapper(Console.Out), Console.In); #else return new AutoRun().Execute(args); diff --git a/src/csharp/Grpc.IntegrationTesting/TestCredentials.cs b/src/csharp/Grpc.IntegrationTesting/TestCredentials.cs index 774563d752f..60b9cf4e0b7 100644 --- a/src/csharp/Grpc.IntegrationTesting/TestCredentials.cs +++ b/src/csharp/Grpc.IntegrationTesting/TestCredentials.cs @@ -90,7 +90,7 @@ namespace Grpc.IntegrationTesting private static string GetPath(string relativePath) { - var assemblyDir = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location); + var assemblyDir = Path.GetDirectoryName(typeof(TestCredentials).GetTypeInfo().Assembly.Location); return Path.Combine(assemblyDir, relativePath); } } diff --git a/src/csharp/Grpc.IntegrationTesting/project.json b/src/csharp/Grpc.IntegrationTesting/project.json index 0093531fa9e..6c1d31b3c45 100644 --- a/src/csharp/Grpc.IntegrationTesting/project.json +++ b/src/csharp/Grpc.IntegrationTesting/project.json @@ -1,23 +1,51 @@ { "buildOptions": { "compile": "**/*.cs", + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + }, + "include": "data/*" + }, "emitEntryPoint": true }, "dependencies": { - "Grpc.Auth": "0.0.1", - "Grpc.Core": "0.0.1", + "Grpc.Auth": { + "version": "0.0.1", + "taget": "project" + }, + "Grpc.Core": { + "version": "0.0.1", + "taget": "project" + }, "Google.Protobuf": "3.0.0-beta3", "CommandLineParser": "1.9.71", - "Moq": "4.2.1510.2205", "NUnit": "3.2.0", "NUnitLite": "3.2.0-*" }, "frameworks": { "net45": { + "dependencies": { + "Moq": "4.2.1510.2205" + }, "frameworkAssemblies": { "System.Runtime": "", "System.IO": "" } + }, + "netstandard1.5": { + "imports": [ + "portable-net45", + "net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027", + "System.Linq.Expressions": "4.0.11-rc2-24027" + } } + }, + "runtimes": { + "win7-x64": { } } } From 2e12db9c319bcbdbb2fa570149f88e4b496b558c Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 16 Jun 2016 16:53:59 -0700 Subject: [PATCH 085/280] Test polling island merges --- Makefile | 36 ++++ build.yaml | 12 ++ src/core/lib/iomgr/ev_epoll_linux.c | 51 +++++- src/core/lib/iomgr/ev_epoll_linux.h | 6 + src/core/lib/iomgr/ev_posix.c | 7 + src/core/lib/iomgr/ev_posix.h | 3 + test/core/iomgr/ev_epoll_linux_test.c | 222 +++++++++++++++++++++++ tools/run_tests/sources_and_headers.json | 16 ++ tools/run_tests/tests.json | 15 ++ 9 files changed, 366 insertions(+), 2 deletions(-) create mode 100644 test/core/iomgr/ev_epoll_linux_test.c diff --git a/Makefile b/Makefile index e615704395b..825684cc2dd 100644 --- a/Makefile +++ b/Makefile @@ -905,6 +905,7 @@ dns_resolver_connectivity_test: $(BINDIR)/$(CONFIG)/dns_resolver_connectivity_te dns_resolver_test: $(BINDIR)/$(CONFIG)/dns_resolver_test dualstack_socket_test: $(BINDIR)/$(CONFIG)/dualstack_socket_test endpoint_pair_test: $(BINDIR)/$(CONFIG)/endpoint_pair_test +ev_epoll_linux_test: $(BINDIR)/$(CONFIG)/ev_epoll_linux_test fd_conservation_posix_test: $(BINDIR)/$(CONFIG)/fd_conservation_posix_test fd_posix_test: $(BINDIR)/$(CONFIG)/fd_posix_test fling_client: $(BINDIR)/$(CONFIG)/fling_client @@ -1242,6 +1243,7 @@ buildtests_c: privatelibs_c \ $(BINDIR)/$(CONFIG)/dns_resolver_test \ $(BINDIR)/$(CONFIG)/dualstack_socket_test \ $(BINDIR)/$(CONFIG)/endpoint_pair_test \ + $(BINDIR)/$(CONFIG)/ev_epoll_linux_test \ $(BINDIR)/$(CONFIG)/fd_conservation_posix_test \ $(BINDIR)/$(CONFIG)/fd_posix_test \ $(BINDIR)/$(CONFIG)/fling_client \ @@ -1512,6 +1514,8 @@ test_c: buildtests_c $(Q) $(BINDIR)/$(CONFIG)/dualstack_socket_test || ( echo test dualstack_socket_test failed ; exit 1 ) $(E) "[RUN] Testing endpoint_pair_test" $(Q) $(BINDIR)/$(CONFIG)/endpoint_pair_test || ( echo test endpoint_pair_test failed ; exit 1 ) + $(E) "[RUN] Testing ev_epoll_linux_test" + $(Q) $(BINDIR)/$(CONFIG)/ev_epoll_linux_test || ( echo test ev_epoll_linux_test failed ; exit 1 ) $(E) "[RUN] Testing fd_conservation_posix_test" $(Q) $(BINDIR)/$(CONFIG)/fd_conservation_posix_test || ( echo test fd_conservation_posix_test failed ; exit 1 ) $(E) "[RUN] Testing fd_posix_test" @@ -7130,6 +7134,38 @@ endif endif +EV_EPOLL_LINUX_TEST_SRC = \ + test/core/iomgr/ev_epoll_linux_test.c \ + +EV_EPOLL_LINUX_TEST_OBJS = $(addprefix $(OBJDIR)/$(CONFIG)/, $(addsuffix .o, $(basename $(EV_EPOLL_LINUX_TEST_SRC)))) +ifeq ($(NO_SECURE),true) + +# You can't build secure targets if you don't have OpenSSL. + +$(BINDIR)/$(CONFIG)/ev_epoll_linux_test: openssl_dep_error + +else + + + +$(BINDIR)/$(CONFIG)/ev_epoll_linux_test: $(EV_EPOLL_LINUX_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a + $(E) "[LD] Linking $@" + $(Q) mkdir -p `dirname $@` + $(Q) $(LD) $(LDFLAGS) $(EV_EPOLL_LINUX_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a $(LDLIBS) $(LDLIBS_SECURE) -o $(BINDIR)/$(CONFIG)/ev_epoll_linux_test + +endif + +$(OBJDIR)/$(CONFIG)/test/core/iomgr/ev_epoll_linux_test.o: $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a + +deps_ev_epoll_linux_test: $(EV_EPOLL_LINUX_TEST_OBJS:.o=.dep) + +ifneq ($(NO_SECURE),true) +ifneq ($(NO_DEPS),true) +-include $(EV_EPOLL_LINUX_TEST_OBJS:.o=.dep) +endif +endif + + FD_CONSERVATION_POSIX_TEST_SRC = \ test/core/iomgr/fd_conservation_posix_test.c \ diff --git a/build.yaml b/build.yaml index 7790e0c5174..84f4ea521be 100644 --- a/build.yaml +++ b/build.yaml @@ -1407,6 +1407,18 @@ targets: - grpc - gpr_test_util - gpr +- name: ev_epoll_linux_test + build: test + language: c + src: + - test/core/iomgr/ev_epoll_linux_test.c + deps: + - grpc_test_util + - grpc + - gpr_test_util + - gpr + platforms: + - linux - name: fd_conservation_posix_test build: test language: c diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 1fb59474640..ed2c494b783 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -317,8 +317,9 @@ static void polling_island_remove_all_fds_locked(polling_island *pi, if (err < 0 && errno != ENOENT) { /* TODO: sreek - We need a better way to bubble up this error instead of * just logging a message */ - gpr_log(GPR_ERROR, "epoll_ctl deleting fds[%zu]: %d failed with error: %s", - i, pi->fds[i]->fd, strerror(errno)); + gpr_log(GPR_ERROR, + "epoll_ctl deleting fds[%zu]: %d failed with error: %s", i, + pi->fds[i]->fd, strerror(errno)); } if (remove_fd_refs) { @@ -1458,6 +1459,52 @@ static void pollset_set_del_pollset_set(grpc_exec_ctx *exec_ctx, gpr_mu_unlock(&bag->mu); } +/* Test helper functions + * */ +void *grpc_fd_get_polling_island(grpc_fd *fd) { + polling_island *pi; + + gpr_mu_lock(&fd->pi_mu); + pi = fd->polling_island; + gpr_mu_unlock(&fd->pi_mu); + + return pi; +} + +void *grpc_pollset_get_polling_island(grpc_pollset *ps) { + polling_island *pi; + + gpr_mu_lock(&ps->pi_mu); + pi = ps->polling_island; + gpr_mu_unlock(&ps->pi_mu); + + return pi; +} + +static polling_island *get_polling_island(polling_island *p) { + if (p == NULL) { + return NULL; + } + + polling_island *next; + gpr_mu_lock(&p->mu); + while (p->merged_to != NULL) { + next = p->merged_to; + gpr_mu_unlock(&p->mu); + p = next; + gpr_mu_lock(&p->mu); + } + gpr_mu_unlock(&p->mu); + + return p; +} + +bool grpc_are_polling_islands_equal(void *p, void *q) { + p = get_polling_island(p); + q = get_polling_island(q); + return p == q; +} + /******************************************************************************* * Event engine binding */ diff --git a/src/core/lib/iomgr/ev_epoll_linux.h b/src/core/lib/iomgr/ev_epoll_linux.h index 8c819975a4c..7a494aba198 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.h +++ b/src/core/lib/iomgr/ev_epoll_linux.h @@ -38,4 +38,10 @@ const grpc_event_engine_vtable *grpc_init_epoll_linux(void); +#ifdef GPR_LINUX_EPOLL +void *grpc_fd_get_polling_island(grpc_fd *fd); +void *grpc_pollset_get_polling_island(grpc_pollset *ps); +bool grpc_are_polling_islands_equal(void *p, void *q); +#endif /* defined(GPR_LINUX_EPOLL) */ + #endif /* GRPC_CORE_LIB_IOMGR_EV_EPOLL_LINUX_H */ diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index 2b15967adcc..5b20600a6f7 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -54,6 +54,7 @@ grpc_poll_function_type grpc_poll_function = poll; static const grpc_event_engine_vtable *g_event_engine; +static const char* g_poll_strategy_name = NULL; typedef const grpc_event_engine_vtable *(*event_engine_factory_fn)(void); @@ -101,6 +102,7 @@ static void try_engine(const char *engine) { for (size_t i = 0; i < GPR_ARRAY_SIZE(g_factories); i++) { if (is(engine, g_factories[i].name)) { if ((g_event_engine = g_factories[i].factory())) { + g_poll_strategy_name = g_factories[i].name; gpr_log(GPR_DEBUG, "Using polling engine: %s", g_factories[i].name); return; } @@ -108,6 +110,11 @@ static void try_engine(const char *engine) { } } +/* Call this only after calling grpc_event_engine_init() */ +const char *grpc_get_poll_strategy_name() { + return g_poll_strategy_name; +} + void grpc_event_engine_init(void) { char *s = gpr_getenv("GRPC_POLL_STRATEGY"); if (s == NULL) { diff --git a/src/core/lib/iomgr/ev_posix.h b/src/core/lib/iomgr/ev_posix.h index 344bf63438a..3ed5a5f9562 100644 --- a/src/core/lib/iomgr/ev_posix.h +++ b/src/core/lib/iomgr/ev_posix.h @@ -98,6 +98,9 @@ typedef struct grpc_event_engine_vtable { void grpc_event_engine_init(void); void grpc_event_engine_shutdown(void); +/* Return the name of the poll strategy */ +const char* grpc_get_poll_strategy_name(); + /* Create a wrapped file descriptor. Requires fd is a non-blocking file descriptor. This takes ownership of closing fd. */ diff --git a/test/core/iomgr/ev_epoll_linux_test.c b/test/core/iomgr/ev_epoll_linux_test.c new file mode 100644 index 00000000000..51da15faa7a --- /dev/null +++ b/test/core/iomgr/ev_epoll_linux_test.c @@ -0,0 +1,222 @@ +/* + * + * Copyright 2015, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include "src/core/lib/iomgr/ev_epoll_linux.h" +#include "src/core/lib/iomgr/ev_posix.h" + +#include +#include +#include +#include + +#include +#include + +#include "src/core/lib/iomgr/iomgr.h" +#include "test/core/util/test_config.h" + +typedef struct test_pollset { + grpc_pollset *pollset; + gpr_mu *mu; +} test_pollset; + +typedef struct test_fd { + int inner_fd; + grpc_fd *fd; +} test_fd; + +static void test_fd_init(test_fd *fds, int num_fds) { + int i; + for (i = 0; i < num_fds; i++) { + fds[i].inner_fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + fds[i].fd = grpc_fd_create(fds[i].inner_fd, "test_fd"); + } +} + +static void test_fd_cleanup(grpc_exec_ctx *exec_ctx, test_fd *fds, + int num_fds) { + int release_fd; + int i; + + for (i = 0; i < num_fds; i++) { + grpc_fd_shutdown(exec_ctx, fds[i].fd); + grpc_exec_ctx_flush(exec_ctx); + + grpc_fd_orphan(exec_ctx, fds[i].fd, NULL, &release_fd, "test_fd_cleanup"); + grpc_exec_ctx_flush(exec_ctx); + + GPR_ASSERT(release_fd == fds[i].inner_fd); + close(fds[i].inner_fd); + } +} + +static void test_pollset_init(test_pollset *pollsets, int num_pollsets) { + int i; + for (i = 0; i < num_pollsets; i++) { + pollsets[i].pollset = gpr_malloc(grpc_pollset_size()); + grpc_pollset_init(pollsets[i].pollset, &pollsets[i].mu); + } +} + +static void destroy_pollset(grpc_exec_ctx *exec_ctx, void *p, bool success) { + grpc_pollset_destroy(p); +} + +static void test_pollset_cleanup(grpc_exec_ctx *exec_ctx, + test_pollset *pollsets, int num_pollsets) { + grpc_closure destroyed; + int i; + + for (i = 0; i < num_pollsets; i++) { + grpc_closure_init(&destroyed, destroy_pollset, pollsets[i].pollset); + grpc_pollset_shutdown(exec_ctx, pollsets[i].pollset, &destroyed); + + grpc_exec_ctx_flush(exec_ctx); + gpr_free(pollsets[i].pollset); + } +} + +#define NUM_FDS 8 +#define NUM_POLLSETS 4 +/* + * Cases to test: + * case 1) Polling islands of both fd and pollset are NULL + * case 2) Polling island of fd is NULL but that of pollset is not-NULL + * case 3) Polling island of fd is not-NULL but that of pollset is NULL + * case 4) Polling islands of both fd and pollset are not-NULL and: + * case 4.1) Polling islands of fd and pollset are equal + * case 4.2) Polling islands of fd and pollset are NOT-equal (This results + * in a merge) + * */ +static void test_add_fd_to_pollset() { + grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; + test_fd fds[NUM_FDS]; + test_pollset pollsets[NUM_POLLSETS]; + void *expected_pi = NULL; + int i; + + test_fd_init(fds, NUM_FDS); + test_pollset_init(pollsets, NUM_POLLSETS); + + /*Step 1. + * Create three polling islands (This will exercise test case 1 and 2) with + * the following configuration: + * polling island 0 = { fds:0,1,2, pollsets:0} + * polling island 1 = { fds:3,4, pollsets:1} + * polling island 2 = { fds:5,6,7 pollsets:2} + * + *Step 2. + * Add pollset 3 to polling island 0 (by adding fds 0 and 1 to pollset 3) + * (This will exercise test cases 3 and 4.1). The configuration becomes: + * polling island 0 = { fds:0,1,2, pollsets:0,3} <<< pollset 3 added here + * polling island 1 = { fds:3,4, pollsets:1} + * polling island 2 = { fds:5,6,7 pollsets:2} + * + *Step 3. + * Merge polling islands 0 and 1 by adding fd 0 to pollset 1 (This will + * exercise test case 4.2). The configuration becomes: + * polling island (merged) = {fds: 0,1,2,3,4, pollsets: 0,1,3} + * polling island 2 = {fds: 5,6,7 pollsets: 2} + * + *Step 4. + * Finally do one more merge by adding fd 3 to pollset 2. + * polling island (merged) = {fds: 0,1,2,3,4,5,6,7, pollsets: 0,1,2,3} + */ + + /* == Step 1 == */ + for (i = 0; i <= 2; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[0].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + for (i = 3; i <= 4; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + for (i = 5; i <= 7; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + /* == Step 2 == */ + for (i = 0; i <= 1; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[3].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + /* == Step 3 == */ + grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, fds[0].fd); + grpc_exec_ctx_flush(&exec_ctx); + + /* == Step 4 == */ + grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, fds[3].fd); + grpc_exec_ctx_flush(&exec_ctx); + + /* All polling islands are merged at this point */ + + /* Compare Fd:0's polling island with that of all other Fds */ + expected_pi = grpc_fd_get_polling_island(fds[0].fd); + for (i = 1; i < NUM_FDS; i++) { + GPR_ASSERT(grpc_are_polling_islands_equal( + expected_pi, grpc_fd_get_polling_island(fds[i].fd))); + } + + /* Compare Fd:0's polling island with that of all other pollsets */ + for (i = 0; i < NUM_POLLSETS; i++) { + GPR_ASSERT(grpc_are_polling_islands_equal( + expected_pi, grpc_pollset_get_polling_island(pollsets[i].pollset))); + } + + test_fd_cleanup(&exec_ctx, fds, NUM_FDS); + test_pollset_cleanup(&exec_ctx, pollsets, NUM_POLLSETS); + grpc_exec_ctx_finish(&exec_ctx); +} + +int main(int argc, char **argv) { + const char *poll_strategy = NULL; + grpc_test_init(argc, argv); + grpc_iomgr_init(); + + poll_strategy = grpc_get_poll_strategy_name(); + if (poll_strategy != NULL && strcmp(poll_strategy, "epoll") == 0) { + test_add_fd_to_pollset(); + } else { + gpr_log(GPR_INFO, + "Skipping the test. The test is only relevant for 'epoll' " + "strategy. and the current strategy is: '%s'", + poll_strategy); + } + grpc_iomgr_shutdown(); + return 0; +} diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index e8ff61dc3fb..e9df72e43a1 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -315,6 +315,22 @@ "third_party": false, "type": "target" }, + { + "deps": [ + "gpr", + "gpr_test_util", + "grpc", + "grpc_test_util" + ], + "headers": [], + "language": "c", + "name": "ev_epoll_linux_test", + "src": [ + "test/core/iomgr/ev_epoll_linux_test.c" + ], + "third_party": false, + "type": "target" + }, { "deps": [ "gpr", diff --git a/tools/run_tests/tests.json b/tools/run_tests/tests.json index 5a84a41b638..ba661840dad 100644 --- a/tools/run_tests/tests.json +++ b/tools/run_tests/tests.json @@ -377,6 +377,21 @@ "windows" ] }, + { + "args": [], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "gtest": false, + "language": "c", + "name": "ev_epoll_linux_test", + "platforms": [ + "linux" + ] + }, { "args": [], "ci_platforms": [ From 87e6770a1819d44f352e7ba5782ca1f7dd8cffca Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Thu, 16 Jun 2016 22:04:08 -0700 Subject: [PATCH 086/280] pr comments --- doc/compression.md | 40 +++++++++++++++++++++++----------------- 1 file changed, 23 insertions(+), 17 deletions(-) diff --git a/doc/compression.md b/doc/compression.md index ddbbff1c541..6e455636021 100644 --- a/doc/compression.md +++ b/doc/compression.md @@ -41,13 +41,14 @@ of the request, including not performing any compression, regardless of channel and RPC settings (for example, if compression would result in small or negative gains). -A compressed message from a client with an algorithm unsupported by a server, -WILL result in an INVALID\_ARGUMENT error, alongside the receiving peer's -`grpc-accept-encoding` header specifying the algorithms it accepts. If an -INTERNAL error is returned from the server despite having used one of the -algorithms from the `grpc-accept-encoding h`eader, the cause MUST NOT be related -to compression. Data sent from a server compressed with an algorithm not -supported by the client will also result in an INTERNAL error. +When a message from a client compressed with an unsupported algorithm is +processed by a server, it WILL result in an INVALID\_ARGUMENT error. The server +will include in its response a `grpc-accept-encoding` header specifying the +algorithms it does accept. If an INTERNAL error is returned from the server +despite having used one of the algorithms from the `grpc-accept-encoding` +header, the cause MUST NOT be related to compression. Data sent from a server +compressed with an algorithm not supported by the client WILL result in an +INTERNAL error on the client side. Note that a peer MAY choose to not disclose all the encodings it supports. However, if it receives a message compressed in an undisclosed but supported @@ -67,16 +68,21 @@ cases. ### Compression Levels and Algorithms -We currently (as of July 2015) support _gzip_ and _deflate_ as algorithms (with -the possible future addition of snappy). In order to simplify the public API, -it's intended to abstract the algorithms as _compression levels_ (such as "low", -"medium", "high") that'd map to concrete algorithms and/or their settings (such -as "low" mapping to "gzip -3" and "high" mapping to "gzip -9"). However, we -can't presently (July 2015) implement said compression levels at the client side -without either a initial negotiation of capabilities or an automatic retry -mechanism. Therefore, compression levels are only supported at the server side, -which is aware of the client's capabilities by virtue of the incoming -Message-Accept-Encoding header. +The set of supported algorithm is implementation dependent. In order to simplify +the public API and to operate seamlessly across implementations (both in terms +of languages but also different version of the same one), we introduce the idea +of _compression levels_ (such as "low", "medium", "high"). + +Levels map to concrete algorithms and/or their settings (such as "low" mapping +to "gzip -3" and "high" mapping to "gzip -9") automatically depending on what a +peer is known to support. A server is always aware of what its clients support, +as clients disclose it in their Message-Accept-Encoding header as part of their +initial call. A client doesn't a priori (presently) know which algorithms a +server supports. This issue can be addressed with an initial negotiation of +capabilities or an automatic retry mechanism. These features will be implemented +in the future. Currently however, compression levels are only supported at the +server side, which is aware of the client's capabilities by virtue of the +incoming. ### Propagation to child RPCs From b9df2760ed15930429305f01407914db106510d5 Mon Sep 17 00:00:00 2001 From: vjpai Date: Fri, 17 Jun 2016 00:27:00 -0700 Subject: [PATCH 087/280] Add proper documentation of ordering and thread-safety for streaming (sync and async) APIs --- include/grpc++/impl/codegen/async_stream.h | 10 +++++++++- include/grpc++/impl/codegen/completion_queue.h | 14 ++++++++++++-- include/grpc++/impl/codegen/sync_stream.h | 10 ++++++++-- 3 files changed, 29 insertions(+), 5 deletions(-) diff --git a/include/grpc++/impl/codegen/async_stream.h b/include/grpc++/impl/codegen/async_stream.h index 0606441549e..c74737ce5f8 100644 --- a/include/grpc++/impl/codegen/async_stream.h +++ b/include/grpc++/impl/codegen/async_stream.h @@ -52,11 +52,14 @@ class ClientAsyncStreamingInterface { /// Request notification of the reading of the initial metadata. Completion /// will be notified by \a tag on the associated completion queue. + /// This call is optional, but if it is used, it cannot be used concurrently + /// with Read /// /// \param[in] tag Tag identifying this request. virtual void ReadInitialMetadata(void* tag) = 0; - /// Request notification completion. + /// Indicate that the stream is to be finished and request notification + /// Should not be used concurrently with other operations /// /// \param[out] status To be updated with the operation status. /// \param[in] tag Tag identifying this request. @@ -71,6 +74,8 @@ class AsyncReaderInterface { /// Read a message of type \a R into \a msg. Completion will be notified by \a /// tag on the associated completion queue. + /// This is thread-safe with respect to other streaming APIs except for Finish + /// on the same stream. /// /// \param[out] msg Where to eventually store the read message. /// \param[in] tag The tag identifying the operation. @@ -88,6 +93,7 @@ class AsyncWriterInterface { /// Only one write may be outstanding at any given time. This means that /// after calling Write, one must wait to receive \a tag from the completion /// queue BEFORE calling Write again. + /// This is thread-safe with respect to Read /// /// \param[in] msg The message to be written. /// \param[in] tag The tag identifying the operation. @@ -158,6 +164,7 @@ class ClientAsyncWriterInterface : public ClientAsyncStreamingInterface, public AsyncWriterInterface { public: /// Signal the client is done with the writes. + /// Thread-safe with respect to Read /// /// \param[in] tag The tag identifying the operation. virtual void WritesDone(void* tag) = 0; @@ -229,6 +236,7 @@ class ClientAsyncReaderWriterInterface : public ClientAsyncStreamingInterface, public AsyncReaderInterface { public: /// Signal the client is done with the writes. + /// Thread-safe with respect to Read /// /// \param[in] tag The tag identifying the operation. virtual void WritesDone(void* tag) = 0; diff --git a/include/grpc++/impl/codegen/completion_queue.h b/include/grpc++/impl/codegen/completion_queue.h index 1b84b447050..f138ebe7de2 100644 --- a/include/grpc++/impl/codegen/completion_queue.h +++ b/include/grpc++/impl/codegen/completion_queue.h @@ -31,8 +31,18 @@ * */ -/// A completion queue implements a concurrent producer-consumer queue, with two -/// main methods, \a Next and \a AsyncNext. +/// A completion queue implements a concurrent producer-consumer queue, with +/// two main API-exposed methods: \a Next and \a AsyncNext. These +/// methods are the essential component of the gRPC C++ asynchronous API. +/// There is also a Shutdown method to indicate that a given completion queue +/// will no longer have regular events. This must be called before the +/// completion queue is destroyed. +/// All completion queue APIs are thread-safe and may be used concurrently with +/// any other completion queue API invocation; it is acceptable to have +/// multiple threads calling Next or AsyncNext on the same or different +/// completion queues, or to call these methods concurrently with a Shutdown +/// elsewhere. All of these should be completed, though, before a completion +/// queue destructor is called. #ifndef GRPCXX_IMPL_CODEGEN_COMPLETION_QUEUE_H #define GRPCXX_IMPL_CODEGEN_COMPLETION_QUEUE_H diff --git a/include/grpc++/impl/codegen/sync_stream.h b/include/grpc++/impl/codegen/sync_stream.h index e94ffe58422..9d7966ba04b 100644 --- a/include/grpc++/impl/codegen/sync_stream.h +++ b/include/grpc++/impl/codegen/sync_stream.h @@ -71,6 +71,8 @@ class ReaderInterface { virtual ~ReaderInterface() {} /// Blocking read a message and parse to \a msg. Returns \a true on success. + /// This is thread-safe with respect to other streaming APIs except for Finish + /// on the same stream. (Finish must be called as described above.) /// /// \param[out] msg The read message. /// @@ -87,6 +89,7 @@ class WriterInterface { virtual ~WriterInterface() {} /// Blocking write \a msg to the stream with options. + /// This is thread-safe with respect to Read /// /// \param msg The message to be written to the stream. /// \param options Options affecting the write operation. @@ -95,6 +98,7 @@ class WriterInterface { virtual bool Write(const W& msg, const WriteOptions& options) = 0; /// Blocking write \a msg to the stream with default options. + /// This is thread-safe with respect to Read /// /// \param msg The message to be written to the stream. /// @@ -174,7 +178,8 @@ class ClientWriterInterface : public ClientStreamingInterface, public WriterInterface { public: /// Half close writing from the client. - /// Block until writes are completed. + /// Block until currently-pending writes are completed. + /// Thread safe with respect to Read operations only /// /// \return Whether the writes were successful. virtual bool WritesDone() = 0; @@ -257,7 +262,8 @@ class ClientReaderWriterInterface : public ClientStreamingInterface, /// the metadata will be available in ClientContext after the first read. virtual void WaitForInitialMetadata() = 0; - /// Block until writes are completed. + /// Block until currently-pending writes are completed. + /// Thread-safe with respect to Read /// /// \return Whether the writes were successful. virtual bool WritesDone() = 0; From dc44e179507697ba23ce2d6180872af660b185a9 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Fri, 17 Jun 2016 12:47:10 -0700 Subject: [PATCH 088/280] Added third_party/protobuf/src to protoc's include path --- Makefile | 52 ++++++++++++++++++------------------- templates/Makefile.template | 4 +-- 2 files changed, 28 insertions(+), 28 deletions(-) diff --git a/Makefile b/Makefile index f500c7472f1..3df98b9f386 100644 --- a/Makefile +++ b/Makefile @@ -1891,12 +1891,12 @@ else $(GENDIR)/src/proto/grpc/lb/v1/load_balancer.pb.cc: src/proto/grpc/lb/v1/load_balancer.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[PROTOC] Generating protobuf CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --cpp_out=$(GENDIR) $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --cpp_out=$(GENDIR) $< $(GENDIR)/src/proto/grpc/lb/v1/load_balancer.grpc.pb.cc: src/proto/grpc/lb/v1/load_balancer.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[GRPC] Generating gRPC's protobuf service CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< endif ifeq ($(NO_PROTOC),true) @@ -1906,12 +1906,12 @@ else $(GENDIR)/src/proto/grpc/testing/compiler_test.pb.cc: src/proto/grpc/testing/compiler_test.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[PROTOC] Generating protobuf CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --cpp_out=$(GENDIR) $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --cpp_out=$(GENDIR) $< $(GENDIR)/src/proto/grpc/testing/compiler_test.grpc.pb.cc: src/proto/grpc/testing/compiler_test.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[GRPC] Generating gRPC's protobuf service CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< endif ifeq ($(NO_PROTOC),true) @@ -1921,12 +1921,12 @@ else $(GENDIR)/src/proto/grpc/testing/control.pb.cc: src/proto/grpc/testing/control.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(GENDIR)/src/proto/grpc/testing/payloads.pb.cc $(GENDIR)/src/proto/grpc/testing/stats.pb.cc $(E) "[PROTOC] Generating protobuf CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --cpp_out=$(GENDIR) $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --cpp_out=$(GENDIR) $< $(GENDIR)/src/proto/grpc/testing/control.grpc.pb.cc: src/proto/grpc/testing/control.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(GENDIR)/src/proto/grpc/testing/payloads.pb.cc $(GENDIR)/src/proto/grpc/testing/payloads.grpc.pb.cc $(GENDIR)/src/proto/grpc/testing/stats.pb.cc $(GENDIR)/src/proto/grpc/testing/stats.grpc.pb.cc $(E) "[GRPC] Generating gRPC's protobuf service CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< endif ifeq ($(NO_PROTOC),true) @@ -1936,12 +1936,12 @@ else $(GENDIR)/src/proto/grpc/testing/duplicate/echo_duplicate.pb.cc: src/proto/grpc/testing/duplicate/echo_duplicate.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(GENDIR)/src/proto/grpc/testing/echo_messages.pb.cc $(E) "[PROTOC] Generating protobuf CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --cpp_out=$(GENDIR) $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --cpp_out=$(GENDIR) $< $(GENDIR)/src/proto/grpc/testing/duplicate/echo_duplicate.grpc.pb.cc: src/proto/grpc/testing/duplicate/echo_duplicate.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(GENDIR)/src/proto/grpc/testing/echo_messages.pb.cc $(GENDIR)/src/proto/grpc/testing/echo_messages.grpc.pb.cc $(E) "[GRPC] Generating gRPC's protobuf service CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< endif ifeq ($(NO_PROTOC),true) @@ -1951,12 +1951,12 @@ else $(GENDIR)/src/proto/grpc/testing/echo.pb.cc: src/proto/grpc/testing/echo.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(GENDIR)/src/proto/grpc/testing/echo_messages.pb.cc $(E) "[PROTOC] Generating protobuf CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --cpp_out=$(GENDIR) $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --cpp_out=$(GENDIR) $< $(GENDIR)/src/proto/grpc/testing/echo.grpc.pb.cc: src/proto/grpc/testing/echo.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(GENDIR)/src/proto/grpc/testing/echo_messages.pb.cc $(GENDIR)/src/proto/grpc/testing/echo_messages.grpc.pb.cc $(E) "[GRPC] Generating gRPC's protobuf service CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< endif ifeq ($(NO_PROTOC),true) @@ -1966,12 +1966,12 @@ else $(GENDIR)/src/proto/grpc/testing/echo_messages.pb.cc: src/proto/grpc/testing/echo_messages.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[PROTOC] Generating protobuf CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --cpp_out=$(GENDIR) $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --cpp_out=$(GENDIR) $< $(GENDIR)/src/proto/grpc/testing/echo_messages.grpc.pb.cc: src/proto/grpc/testing/echo_messages.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[GRPC] Generating gRPC's protobuf service CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< endif ifeq ($(NO_PROTOC),true) @@ -1981,12 +1981,12 @@ else $(GENDIR)/src/proto/grpc/testing/empty.pb.cc: src/proto/grpc/testing/empty.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[PROTOC] Generating protobuf CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --cpp_out=$(GENDIR) $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --cpp_out=$(GENDIR) $< $(GENDIR)/src/proto/grpc/testing/empty.grpc.pb.cc: src/proto/grpc/testing/empty.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[GRPC] Generating gRPC's protobuf service CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< endif ifeq ($(NO_PROTOC),true) @@ -1996,12 +1996,12 @@ else $(GENDIR)/src/proto/grpc/testing/messages.pb.cc: src/proto/grpc/testing/messages.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[PROTOC] Generating protobuf CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --cpp_out=$(GENDIR) $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --cpp_out=$(GENDIR) $< $(GENDIR)/src/proto/grpc/testing/messages.grpc.pb.cc: src/proto/grpc/testing/messages.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[GRPC] Generating gRPC's protobuf service CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< endif ifeq ($(NO_PROTOC),true) @@ -2011,12 +2011,12 @@ else $(GENDIR)/src/proto/grpc/testing/metrics.pb.cc: src/proto/grpc/testing/metrics.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[PROTOC] Generating protobuf CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --cpp_out=$(GENDIR) $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --cpp_out=$(GENDIR) $< $(GENDIR)/src/proto/grpc/testing/metrics.grpc.pb.cc: src/proto/grpc/testing/metrics.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[GRPC] Generating gRPC's protobuf service CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< endif ifeq ($(NO_PROTOC),true) @@ -2026,12 +2026,12 @@ else $(GENDIR)/src/proto/grpc/testing/payloads.pb.cc: src/proto/grpc/testing/payloads.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[PROTOC] Generating protobuf CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --cpp_out=$(GENDIR) $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --cpp_out=$(GENDIR) $< $(GENDIR)/src/proto/grpc/testing/payloads.grpc.pb.cc: src/proto/grpc/testing/payloads.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[GRPC] Generating gRPC's protobuf service CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< endif ifeq ($(NO_PROTOC),true) @@ -2041,12 +2041,12 @@ else $(GENDIR)/src/proto/grpc/testing/services.pb.cc: src/proto/grpc/testing/services.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(GENDIR)/src/proto/grpc/testing/messages.pb.cc $(GENDIR)/src/proto/grpc/testing/control.pb.cc $(E) "[PROTOC] Generating protobuf CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --cpp_out=$(GENDIR) $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --cpp_out=$(GENDIR) $< $(GENDIR)/src/proto/grpc/testing/services.grpc.pb.cc: src/proto/grpc/testing/services.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(GENDIR)/src/proto/grpc/testing/messages.pb.cc $(GENDIR)/src/proto/grpc/testing/messages.grpc.pb.cc $(GENDIR)/src/proto/grpc/testing/control.pb.cc $(GENDIR)/src/proto/grpc/testing/control.grpc.pb.cc $(E) "[GRPC] Generating gRPC's protobuf service CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< endif ifeq ($(NO_PROTOC),true) @@ -2056,12 +2056,12 @@ else $(GENDIR)/src/proto/grpc/testing/stats.pb.cc: src/proto/grpc/testing/stats.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[PROTOC] Generating protobuf CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --cpp_out=$(GENDIR) $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --cpp_out=$(GENDIR) $< $(GENDIR)/src/proto/grpc/testing/stats.grpc.pb.cc: src/proto/grpc/testing/stats.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(E) "[GRPC] Generating gRPC's protobuf service CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< endif ifeq ($(NO_PROTOC),true) @@ -2071,12 +2071,12 @@ else $(GENDIR)/src/proto/grpc/testing/test.pb.cc: src/proto/grpc/testing/test.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(GENDIR)/src/proto/grpc/testing/empty.pb.cc $(GENDIR)/src/proto/grpc/testing/messages.pb.cc $(E) "[PROTOC] Generating protobuf CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --cpp_out=$(GENDIR) $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --cpp_out=$(GENDIR) $< $(GENDIR)/src/proto/grpc/testing/test.grpc.pb.cc: src/proto/grpc/testing/test.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) $(GENDIR)/src/proto/grpc/testing/empty.pb.cc $(GENDIR)/src/proto/grpc/testing/empty.grpc.pb.cc $(GENDIR)/src/proto/grpc/testing/messages.pb.cc $(GENDIR)/src/proto/grpc/testing/messages.grpc.pb.cc $(E) "[GRPC] Generating gRPC's protobuf service CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< endif diff --git a/templates/Makefile.template b/templates/Makefile.template index 079fef65ae8..f3e21c10ba4 100644 --- a/templates/Makefile.template +++ b/templates/Makefile.template @@ -1227,12 +1227,12 @@ $(GENDIR)/${p}.pb.cc: ${p}.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) ${' '.join('$(GENDIR)/%s.pb.cc' % q for q in proto_deps.get(p, []))} $(E) "[PROTOC] Generating protobuf CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --cpp_out=$(GENDIR) $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --cpp_out=$(GENDIR) $< $(GENDIR)/${p}.grpc.pb.cc: ${p}.proto $(PROTOBUF_DEP) $(PROTOC_PLUGINS) ${' '.join('$(GENDIR)/%s.pb.cc $(GENDIR)/%s.grpc.pb.cc' % (q,q) for q in proto_deps.get(p, []))} $(E) "[GRPC] Generating gRPC's protobuf service CC file from $<" $(Q) mkdir -p `dirname $@` - $(Q) $(PROTOC) --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< + $(Q) $(PROTOC) -Ithird_party/protobuf/src -I. --grpc_out=$(GENDIR) --plugin=protoc-gen-grpc=$(BINDIR)/$(CONFIG)/grpc_cpp_plugin $< endif % endfor From 3c65d24fabbee3f88fc505b54d82280c31b5e0f5 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Fri, 17 Jun 2016 15:18:18 -0700 Subject: [PATCH 089/280] updated stress test with new tests cases --- test/cpp/interop/stress_interop_client.cc | 14 +++++++-- test/cpp/interop/stress_interop_client.h | 36 +++++++++++++---------- 2 files changed, 31 insertions(+), 19 deletions(-) diff --git a/test/cpp/interop/stress_interop_client.cc b/test/cpp/interop/stress_interop_client.cc index aa95682e741..1d5fc80cf26 100644 --- a/test/cpp/interop/stress_interop_client.cc +++ b/test/cpp/interop/stress_interop_client.cc @@ -138,8 +138,12 @@ bool StressTestInteropClient::RunTest(TestCaseType test_case) { is_success = interop_client_->DoLargeUnary(); break; } - case LARGE_COMPRESSED_UNARY: { - is_success = interop_client_->DoLargeCompressedUnary(); + case CLIENT_COMPRESSED_UNARY: { + is_success = interop_client_->DoClientCompressedUnary(); + break; + } + case CLIENT_COMPRESSED_STREAMING: { + is_success = interop_client_->DoClientCompressedStreaming(); break; } case CLIENT_STREAMING: { @@ -150,8 +154,12 @@ bool StressTestInteropClient::RunTest(TestCaseType test_case) { is_success = interop_client_->DoResponseStreaming(); break; } + case SERVER_COMPRESSED_UNARY: { + is_success = interop_client_->DoServerCompressedUnary(); + break; + } case SERVER_COMPRESSED_STREAMING: { - is_success = interop_client_->DoResponseCompressedStreaming(); + is_success = interop_client_->DoServerCompressedStreaming(); break; } case SLOW_CONSUMER: { diff --git a/test/cpp/interop/stress_interop_client.h b/test/cpp/interop/stress_interop_client.h index aa93b58b4a7..cf6a7134739 100644 --- a/test/cpp/interop/stress_interop_client.h +++ b/test/cpp/interop/stress_interop_client.h @@ -51,29 +51,33 @@ using std::vector; enum TestCaseType { UNKNOWN_TEST = -1, - EMPTY_UNARY = 0, - LARGE_UNARY = 1, - LARGE_COMPRESSED_UNARY = 2, - CLIENT_STREAMING = 3, - SERVER_STREAMING = 4, - SERVER_COMPRESSED_STREAMING = 5, - SLOW_CONSUMER = 6, - HALF_DUPLEX = 7, - PING_PONG = 8, - CANCEL_AFTER_BEGIN = 9, - CANCEL_AFTER_FIRST_RESPONSE = 10, - TIMEOUT_ON_SLEEPING_SERVER = 11, - EMPTY_STREAM = 12, - STATUS_CODE_AND_MESSAGE = 13, - CUSTOM_METADATA = 14 + EMPTY_UNARY, + LARGE_UNARY, + CLIENT_COMPRESSED_UNARY, + CLIENT_COMPRESSED_STREAMING, + CLIENT_STREAMING, + SERVER_STREAMING, + SERVER_COMPRESSED_UNARY, + SERVER_COMPRESSED_STREAMING, + SLOW_CONSUMER, + HALF_DUPLEX, + PING_PONG, + CANCEL_AFTER_BEGIN, + CANCEL_AFTER_FIRST_RESPONSE, + TIMEOUT_ON_SLEEPING_SERVER, + EMPTY_STREAM, + STATUS_CODE_AND_MESSAGE, + CUSTOM_METADATA }; const vector> kTestCaseList = { {EMPTY_UNARY, "empty_unary"}, {LARGE_UNARY, "large_unary"}, - {LARGE_COMPRESSED_UNARY, "large_compressed_unary"}, + {CLIENT_COMPRESSED_UNARY, "client_compressed_unary"}, + {CLIENT_COMPRESSED_STREAMING, "client_compressed_streaming"}, {CLIENT_STREAMING, "client_streaming"}, {SERVER_STREAMING, "server_streaming"}, + {SERVER_COMPRESSED_UNARY, "server_compressed_unary"}, {SERVER_COMPRESSED_STREAMING, "server_compressed_streaming"}, {SLOW_CONSUMER, "slow_consumer"}, {HALF_DUPLEX, "half_duplex"}, From fab857eefc20a0dc96cd4e6a39a95892e3c105dc Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Fri, 17 Jun 2016 16:53:43 -0700 Subject: [PATCH 090/280] address review comments --- src/csharp/Grpc.Core.Tests/project.json | 3 +- .../Grpc.Examples.MathClient/project.json | 3 +- .../Grpc.Examples.MathServer/project.json | 3 +- src/csharp/Grpc.Examples.Tests/project.json | 3 +- src/csharp/Grpc.Examples/project.json | 3 +- .../Grpc.HealthCheck.Tests/project.json | 3 +- .../project.json | 3 +- .../project.json | 3 +- .../project.json | 3 +- .../project.json | 32 +++++++++++++++++++ .../Grpc.IntegrationTesting/project.json | 6 ++-- 11 files changed, 43 insertions(+), 22 deletions(-) create mode 100644 src/csharp/Grpc.IntegrationTesting.StressClient/project.json diff --git a/src/csharp/Grpc.Core.Tests/project.json b/src/csharp/Grpc.Core.Tests/project.json index 06718043142..13705a22227 100644 --- a/src/csharp/Grpc.Core.Tests/project.json +++ b/src/csharp/Grpc.Core.Tests/project.json @@ -11,8 +11,7 @@ }, "dependencies": { "Grpc.Core": { - "version": "0.0.1", - "taget": "project" + "target": "project" }, "Newtonsoft.Json": "8.0.3", "NUnit": "3.2.0", diff --git a/src/csharp/Grpc.Examples.MathClient/project.json b/src/csharp/Grpc.Examples.MathClient/project.json index 791cd1dcb80..be9e123ff8b 100644 --- a/src/csharp/Grpc.Examples.MathClient/project.json +++ b/src/csharp/Grpc.Examples.MathClient/project.json @@ -11,8 +11,7 @@ }, "dependencies": { "Grpc.Examples": { - "version": "1.0.0", - "taget": "project" + "target": "project" } }, "frameworks": { diff --git a/src/csharp/Grpc.Examples.MathServer/project.json b/src/csharp/Grpc.Examples.MathServer/project.json index 791cd1dcb80..be9e123ff8b 100644 --- a/src/csharp/Grpc.Examples.MathServer/project.json +++ b/src/csharp/Grpc.Examples.MathServer/project.json @@ -11,8 +11,7 @@ }, "dependencies": { "Grpc.Examples": { - "version": "1.0.0", - "taget": "project" + "target": "project" } }, "frameworks": { diff --git a/src/csharp/Grpc.Examples.Tests/project.json b/src/csharp/Grpc.Examples.Tests/project.json index 6ef440e27dd..86f06a5aa9b 100644 --- a/src/csharp/Grpc.Examples.Tests/project.json +++ b/src/csharp/Grpc.Examples.Tests/project.json @@ -11,8 +11,7 @@ }, "dependencies": { "Grpc.Examples": { - "version": "1.0.0", - "taget": "project" + "target": "project" }, "NUnit": "3.2.0", "NUnitLite": "3.2.0-*" diff --git a/src/csharp/Grpc.Examples/project.json b/src/csharp/Grpc.Examples/project.json index 4a3810596b0..610712f6460 100644 --- a/src/csharp/Grpc.Examples/project.json +++ b/src/csharp/Grpc.Examples/project.json @@ -5,8 +5,7 @@ "dependencies": { "Grpc.Core": { - "version": "0.0.1", - "taget": "project" + "target": "project" }, "Google.Protobuf": "3.0.0-beta3" }, diff --git a/src/csharp/Grpc.HealthCheck.Tests/project.json b/src/csharp/Grpc.HealthCheck.Tests/project.json index fde21f1b7e0..9b28d2b4ba0 100644 --- a/src/csharp/Grpc.HealthCheck.Tests/project.json +++ b/src/csharp/Grpc.HealthCheck.Tests/project.json @@ -11,8 +11,7 @@ }, "dependencies": { "Grpc.HealthCheck": { - "version": "0.0.1", - "taget": "project" + "target": "project" }, "NUnit": "3.2.0", "NUnitLite": "3.2.0-*" diff --git a/src/csharp/Grpc.IntegrationTesting.Client/project.json b/src/csharp/Grpc.IntegrationTesting.Client/project.json index a5a87f9882d..d8e47b1169b 100644 --- a/src/csharp/Grpc.IntegrationTesting.Client/project.json +++ b/src/csharp/Grpc.IntegrationTesting.Client/project.json @@ -11,8 +11,7 @@ }, "dependencies": { "Grpc.IntegrationTesting": { - "version": "1.0.0", - "taget": "project" + "target": "project" } }, "frameworks": { diff --git a/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json b/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json index a5a87f9882d..d8e47b1169b 100644 --- a/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json +++ b/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json @@ -11,8 +11,7 @@ }, "dependencies": { "Grpc.IntegrationTesting": { - "version": "1.0.0", - "taget": "project" + "target": "project" } }, "frameworks": { diff --git a/src/csharp/Grpc.IntegrationTesting.Server/project.json b/src/csharp/Grpc.IntegrationTesting.Server/project.json index a5a87f9882d..d8e47b1169b 100644 --- a/src/csharp/Grpc.IntegrationTesting.Server/project.json +++ b/src/csharp/Grpc.IntegrationTesting.Server/project.json @@ -11,8 +11,7 @@ }, "dependencies": { "Grpc.IntegrationTesting": { - "version": "1.0.0", - "taget": "project" + "target": "project" } }, "frameworks": { diff --git a/src/csharp/Grpc.IntegrationTesting.StressClient/project.json b/src/csharp/Grpc.IntegrationTesting.StressClient/project.json new file mode 100644 index 00000000000..d8e47b1169b --- /dev/null +++ b/src/csharp/Grpc.IntegrationTesting.StressClient/project.json @@ -0,0 +1,32 @@ +{ + "buildOptions": { + "compile": "**/*.cs", + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + } + }, + "emitEntryPoint": true + }, + "dependencies": { + "Grpc.IntegrationTesting": { + "target": "project" + } + }, + "frameworks": { + "net45": { }, + "netstandard1.5": { + "imports": [ + "portable-net45", + "net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027" + } + } + }, + "runtimes": { + "win7-x64": { } + } +} diff --git a/src/csharp/Grpc.IntegrationTesting/project.json b/src/csharp/Grpc.IntegrationTesting/project.json index 6c1d31b3c45..8964dcc1970 100644 --- a/src/csharp/Grpc.IntegrationTesting/project.json +++ b/src/csharp/Grpc.IntegrationTesting/project.json @@ -12,12 +12,10 @@ }, "dependencies": { "Grpc.Auth": { - "version": "0.0.1", - "taget": "project" + "target": "project" }, "Grpc.Core": { - "version": "0.0.1", - "taget": "project" + "target": "project" }, "Google.Protobuf": "3.0.0-beta3", "CommandLineParser": "1.9.71", From 39ec1cbd7aed31ad1ce95cd263dff5f28d15fe1a Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Fri, 17 Jun 2016 10:00:43 -0700 Subject: [PATCH 091/280] add csharp CoreCLR docker image --- .../csharp_coreclr_x64/Dockerfile.template | 38 +++++++++ .../test/csharp_coreclr_x64/Dockerfile | 82 +++++++++++++++++++ 2 files changed, 120 insertions(+) create mode 100644 templates/tools/dockerfile/test/csharp_coreclr_x64/Dockerfile.template create mode 100644 tools/dockerfile/test/csharp_coreclr_x64/Dockerfile diff --git a/templates/tools/dockerfile/test/csharp_coreclr_x64/Dockerfile.template b/templates/tools/dockerfile/test/csharp_coreclr_x64/Dockerfile.template new file mode 100644 index 00000000000..35782d6665f --- /dev/null +++ b/templates/tools/dockerfile/test/csharp_coreclr_x64/Dockerfile.template @@ -0,0 +1,38 @@ +%YAML 1.2 +--- | + # Copyright 2015, Google Inc. + # All rights reserved. + # + # Redistribution and use in source and binary forms, with or without + # modification, are permitted provided that the following conditions are + # met: + # + # * Redistributions of source code must retain the above copyright + # notice, this list of conditions and the following disclaimer. + # * Redistributions in binary form must reproduce the above + # copyright notice, this list of conditions and the following disclaimer + # in the documentation and/or other materials provided with the + # distribution. + # * Neither the name of Google Inc. nor the names of its + # contributors may be used to endorse or promote products derived from + # this software without specific prior written permission. + # + # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + FROM microsoft/dotnet:1.0.0-preview1 + + <%include file="../../apt_get_basic.include"/> + <%include file="../../run_tests_addons.include"/> + # Define the default command. + CMD ["bash"] + diff --git a/tools/dockerfile/test/csharp_coreclr_x64/Dockerfile b/tools/dockerfile/test/csharp_coreclr_x64/Dockerfile new file mode 100644 index 00000000000..9dfc040d736 --- /dev/null +++ b/tools/dockerfile/test/csharp_coreclr_x64/Dockerfile @@ -0,0 +1,82 @@ +# Copyright 2015, Google Inc. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +FROM microsoft/dotnet:1.0.0-preview1 + +# Install Git and basic packages. +RUN apt-get update && apt-get install -y \ + autoconf \ + autotools-dev \ + build-essential \ + bzip2 \ + ccache \ + curl \ + gcc \ + gcc-multilib \ + git \ + golang \ + gyp \ + lcov \ + libc6 \ + libc6-dbg \ + libc6-dev \ + libgtest-dev \ + libtool \ + make \ + perl \ + strace \ + python-dev \ + python-setuptools \ + python-yaml \ + telnet \ + unzip \ + wget \ + zip && apt-get clean + +#================ +# Build profiling +RUN apt-get update && apt-get install -y time && apt-get clean + +# Prepare ccache +RUN ln -s /usr/bin/ccache /usr/local/bin/gcc +RUN ln -s /usr/bin/ccache /usr/local/bin/g++ +RUN ln -s /usr/bin/ccache /usr/local/bin/cc +RUN ln -s /usr/bin/ccache /usr/local/bin/c++ +RUN ln -s /usr/bin/ccache /usr/local/bin/clang +RUN ln -s /usr/bin/ccache /usr/local/bin/clang++ + +#====================== +# Zookeeper dependencies +# TODO(jtattermusch): is zookeeper still needed? +RUN apt-get install -y libzookeeper-mt-dev + +RUN mkdir /var/local/jenkins + +# Define the default command. +CMD ["bash"] From 099cbf84aad8da27fba9ee7cb698d8e2d5c01fd1 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Fri, 17 Jun 2016 16:58:11 -0700 Subject: [PATCH 092/280] add debian.8-x64 to the list of runtimes --- src/csharp/Grpc.Core.Tests/project.json | 3 ++- src/csharp/Grpc.Examples.MathClient/project.json | 3 ++- src/csharp/Grpc.Examples.MathServer/project.json | 3 ++- src/csharp/Grpc.Examples.Tests/project.json | 3 ++- src/csharp/Grpc.HealthCheck.Tests/project.json | 3 ++- src/csharp/Grpc.IntegrationTesting.Client/project.json | 3 ++- src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json | 3 ++- src/csharp/Grpc.IntegrationTesting.Server/project.json | 3 ++- src/csharp/Grpc.IntegrationTesting.StressClient/project.json | 3 ++- src/csharp/Grpc.IntegrationTesting/project.json | 3 ++- 10 files changed, 20 insertions(+), 10 deletions(-) diff --git a/src/csharp/Grpc.Core.Tests/project.json b/src/csharp/Grpc.Core.Tests/project.json index 13705a22227..a59e6390d9d 100644 --- a/src/csharp/Grpc.Core.Tests/project.json +++ b/src/csharp/Grpc.Core.Tests/project.json @@ -29,6 +29,7 @@ } }, "runtimes": { - "win7-x64": { } + "win7-x64": { }, + "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.Examples.MathClient/project.json b/src/csharp/Grpc.Examples.MathClient/project.json index be9e123ff8b..9c070c76ba5 100644 --- a/src/csharp/Grpc.Examples.MathClient/project.json +++ b/src/csharp/Grpc.Examples.MathClient/project.json @@ -26,6 +26,7 @@ } }, "runtimes": { - "win7-x64": { } + "win7-x64": { }, + "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.Examples.MathServer/project.json b/src/csharp/Grpc.Examples.MathServer/project.json index be9e123ff8b..9c070c76ba5 100644 --- a/src/csharp/Grpc.Examples.MathServer/project.json +++ b/src/csharp/Grpc.Examples.MathServer/project.json @@ -26,6 +26,7 @@ } }, "runtimes": { - "win7-x64": { } + "win7-x64": { }, + "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.Examples.Tests/project.json b/src/csharp/Grpc.Examples.Tests/project.json index 86f06a5aa9b..7dd938cfb16 100644 --- a/src/csharp/Grpc.Examples.Tests/project.json +++ b/src/csharp/Grpc.Examples.Tests/project.json @@ -28,6 +28,7 @@ } }, "runtimes": { - "win7-x64": { } + "win7-x64": { }, + "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.HealthCheck.Tests/project.json b/src/csharp/Grpc.HealthCheck.Tests/project.json index 9b28d2b4ba0..be2b3a0459f 100644 --- a/src/csharp/Grpc.HealthCheck.Tests/project.json +++ b/src/csharp/Grpc.HealthCheck.Tests/project.json @@ -28,6 +28,7 @@ } }, "runtimes": { - "win7-x64": { } + "win7-x64": { }, + "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.IntegrationTesting.Client/project.json b/src/csharp/Grpc.IntegrationTesting.Client/project.json index d8e47b1169b..fabf906a738 100644 --- a/src/csharp/Grpc.IntegrationTesting.Client/project.json +++ b/src/csharp/Grpc.IntegrationTesting.Client/project.json @@ -27,6 +27,7 @@ } }, "runtimes": { - "win7-x64": { } + "win7-x64": { }, + "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json b/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json index d8e47b1169b..fabf906a738 100644 --- a/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json +++ b/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json @@ -27,6 +27,7 @@ } }, "runtimes": { - "win7-x64": { } + "win7-x64": { }, + "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.IntegrationTesting.Server/project.json b/src/csharp/Grpc.IntegrationTesting.Server/project.json index d8e47b1169b..fabf906a738 100644 --- a/src/csharp/Grpc.IntegrationTesting.Server/project.json +++ b/src/csharp/Grpc.IntegrationTesting.Server/project.json @@ -27,6 +27,7 @@ } }, "runtimes": { - "win7-x64": { } + "win7-x64": { }, + "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.IntegrationTesting.StressClient/project.json b/src/csharp/Grpc.IntegrationTesting.StressClient/project.json index d8e47b1169b..fabf906a738 100644 --- a/src/csharp/Grpc.IntegrationTesting.StressClient/project.json +++ b/src/csharp/Grpc.IntegrationTesting.StressClient/project.json @@ -27,6 +27,7 @@ } }, "runtimes": { - "win7-x64": { } + "win7-x64": { }, + "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.IntegrationTesting/project.json b/src/csharp/Grpc.IntegrationTesting/project.json index 8964dcc1970..d5ac6f108ba 100644 --- a/src/csharp/Grpc.IntegrationTesting/project.json +++ b/src/csharp/Grpc.IntegrationTesting/project.json @@ -44,6 +44,7 @@ } }, "runtimes": { - "win7-x64": { } + "win7-x64": { }, + "debian.8-x64": {} } } From 76511a5cb7b8e70ce43c36a152a69d64a4b92c9a Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Fri, 17 Jun 2016 14:00:57 -0700 Subject: [PATCH 093/280] attempt for run_tests.py --- tools/run_tests/build_csharp_coreclr.sh | 38 ++++++++++++++++++++++ tools/run_tests/run_tests.py | 43 +++++++++++++++++++------ 2 files changed, 71 insertions(+), 10 deletions(-) create mode 100755 tools/run_tests/build_csharp_coreclr.sh diff --git a/tools/run_tests/build_csharp_coreclr.sh b/tools/run_tests/build_csharp_coreclr.sh new file mode 100755 index 00000000000..733b1a2083c --- /dev/null +++ b/tools/run_tests/build_csharp_coreclr.sh @@ -0,0 +1,38 @@ +#!/bin/bash +# Copyright 2015, Google Inc. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +set -ex + +cd $(dirname $0)/../../src/csharp + +# TODO(jtattermusch): introduce caching +dotnet restore . + +dotnet build -f netstandard1.5 --configuration $MSBUILD_CONFIG '**/project.json' diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 3080d19c8d1..473eb460b46 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -504,7 +504,14 @@ class CSharpLanguage(object): self._make_options = [_windows_toolset_option(self.args.compiler), _windows_arch_option(self.args.arch)] else: - _check_compiler(self.args.compiler, ['default']) + if self.platform == 'linux': + if self.args.compiler == 'coreclr': + self._docker_distro = 'coreclr' + else: + self._docker_distro = 'jessie' + else: + _check_compiler(self.args.compiler, ['default']) + if self.platform == 'mac': # On Mac, official distribution of mono is 32bit. # TODO(jtattermusch): EMBED_ZLIB=true currently breaks the mac build @@ -521,14 +528,25 @@ class CSharpLanguage(object): nunit_args = ['--labels=All', '--noresult', '--workers=1'] - if self.platform == 'windows': + assembly_subdir = 'bin/%s' % msbuild_config + assembly_extension = '.exe' + + if self.args.compiler == 'coreclr': + # TODO(jtattermusch): make the runtime string platform-specific + assembly_subdir += '/netstandard1.5/debian.8-x64' + assembly_extension = '' + runtime_cmd = [] + elif self.platform == 'windows': runtime_cmd = [] else: runtime_cmd = ['mono'] specs = [] for assembly in tests_by_assembly.iterkeys(): - assembly_file = 'src/csharp/%s/bin/%s/%s.exe' % (assembly, msbuild_config, assembly) + assembly_file = 'src/csharp/%s/%s/%s%s' % (assembly, + assembly_subdir, + assembly, + assembly_extension) if self.config.build_config != 'gcov' or self.platform != 'windows': # normally, run each test as a separate process for test in tests_by_assembly[assembly]: @@ -571,12 +589,15 @@ class CSharpLanguage(object): return self._make_options; def build_steps(self): - if self.platform == 'windows': - return [[_windows_build_bat(self.args.compiler), - 'src/csharp/Grpc.sln', - '/p:Configuration=%s' % _MSBUILD_CONFIG[self.config.build_config]]] + if self.args.compiler == 'coreclr': + return [['tools/run_tests/build_csharp_coreclr.sh']] else: - return [['tools/run_tests/build_csharp.sh']] + if self.platform == 'windows': + return [[_windows_build_bat(self.args.compiler), + 'src/csharp/Grpc.sln', + '/p:Configuration=%s' % _MSBUILD_CONFIG[self.config.build_config]]] + else: + return [['tools/run_tests/build_csharp.sh']] def post_tests_steps(self): if self.platform == 'windows': @@ -588,7 +609,8 @@ class CSharpLanguage(object): return 'Makefile' def dockerfile_dir(self): - return 'tools/dockerfile/test/csharp_jessie_%s' % _docker_arch_suffix(self.args.arch) + return 'tools/dockerfile/test/csharp_%s_%s' % (self._docker_distro, + _docker_arch_suffix(self.args.arch)) def __str__(self): return 'csharp' @@ -838,7 +860,8 @@ argp.add_argument('--compiler', 'clang3.4', 'clang3.5', 'clang3.6', 'clang3.7', 'vs2010', 'vs2013', 'vs2015', 'python2.7', 'python3.4', - 'node0.12', 'node4', 'node5'], + 'node0.12', 'node4', 'node5', + 'coreclr'], default='default', help='Selects compiler to use. Allowed values depend on the platform and language.') argp.add_argument('--build_only', From e5f0f95101b40e00f3f7147d18514dbedf81d5a2 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Fri, 17 Jun 2016 18:56:55 -0700 Subject: [PATCH 094/280] fixed syntax errors in proto --- src/proto/grpc/binary_log/v1alpha/log.proto | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/src/proto/grpc/binary_log/v1alpha/log.proto b/src/proto/grpc/binary_log/v1alpha/log.proto index 83166cd4104..46656109bc9 100644 --- a/src/proto/grpc/binary_log/v1alpha/log.proto +++ b/src/proto/grpc/binary_log/v1alpha/log.proto @@ -29,20 +29,20 @@ syntax = "proto3"; -import "google/protobuf/timestamp.proto" +import "google/protobuf/timestamp.proto"; package grpc.binary_log.v1alpha; enum Direction { - SERVER_SEND; - SERVER_RECV; - CLIENT_SEND; - CLIENT_RECV; + SERVER_SEND = 0; + SERVER_RECV = 1; + CLIENT_SEND = 2; + CLIENT_RECV = 3; } message KeyValuePair { - string key; - string value; + string key = 1; + string value = 2; } // Any sort of metadata that may be sent in either direction during a call From 377bfdef920ef36ab53f409f7037f3f683020f9f Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Fri, 17 Jun 2016 18:17:47 -0700 Subject: [PATCH 095/280] added extra path to python protoc --- src/python/grpcio/commands.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/python/grpcio/commands.py b/src/python/grpcio/commands.py index 3e974eba0a3..7f9c292d659 100644 --- a/src/python/grpcio/commands.py +++ b/src/python/grpcio/commands.py @@ -182,6 +182,8 @@ class BuildProtoModules(setuptools.Command): '--plugin=protoc-gen-python-grpc={}'.format( self.grpc_python_plugin_command), '-I {}'.format(GRPC_STEM), + '-I .'.format(GRPC_STEM), + '-I {}/third_party/protobuf/src'.format(GRPC_STEM), '--python_out={}'.format(PROTO_GEN_STEM), '--python-grpc_out={}'.format(PROTO_GEN_STEM), ] + [path] From 5ad70a6953635da45542d1b3cb93672e6b51ae64 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Fri, 17 Jun 2016 19:03:10 -0700 Subject: [PATCH 096/280] removed spurious .format() --- src/python/grpcio/commands.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/python/grpcio/commands.py b/src/python/grpcio/commands.py index 7f9c292d659..f498ed41901 100644 --- a/src/python/grpcio/commands.py +++ b/src/python/grpcio/commands.py @@ -182,7 +182,7 @@ class BuildProtoModules(setuptools.Command): '--plugin=protoc-gen-python-grpc={}'.format( self.grpc_python_plugin_command), '-I {}'.format(GRPC_STEM), - '-I .'.format(GRPC_STEM), + '-I .', '-I {}/third_party/protobuf/src'.format(GRPC_STEM), '--python_out={}'.format(PROTO_GEN_STEM), '--python-grpc_out={}'.format(PROTO_GEN_STEM), From 78a73333b705a9d184da7f3174533500d05b2a21 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Mon, 20 Jun 2016 08:24:44 -0700 Subject: [PATCH 097/280] Fix memory leak upon receiving two RST_STREAMs --- .../chttp2/transport/frame_rst_stream.c | 14 +- .../2c452818a10ddef09b90c89a53db14b9b56b21f3 | Bin 0 -> 52 bytes .../42ead79c94eccdf8a8c3d8036be73e14fa260dd5 | Bin 0 -> 64 bytes .../4e05d6cf1c3f0c04f6ee92d09a53ee0fe35c085a | Bin 0 -> 64 bytes .../8f980dd25f1c77e3536131c2c620aa32e8c13180 | Bin 0 -> 14 bytes .../aef36c49d7dec0dcf8cdc224d9e9221fa2cb1db0 | Bin 0 -> 53 bytes ...h-14ed70cd9ea7987cdd0c8f6e39398ee7c60ee2ff | Bin 0 -> 719 bytes .../dcb06a6e34cbed15515e5b3581ca666f704777bd | Bin 0 -> 238 bytes .../ea46b684f1e67a27c231f2d536c41da631189b9c | Bin 0 -> 696 bytes tools/run_tests/tests.json | 152 ++++++++++++++++++ 10 files changed, 160 insertions(+), 6 deletions(-) create mode 100644 test/core/end2end/fuzzers/client_fuzzer_corpus/2c452818a10ddef09b90c89a53db14b9b56b21f3 create mode 100644 test/core/end2end/fuzzers/client_fuzzer_corpus/42ead79c94eccdf8a8c3d8036be73e14fa260dd5 create mode 100644 test/core/end2end/fuzzers/client_fuzzer_corpus/4e05d6cf1c3f0c04f6ee92d09a53ee0fe35c085a create mode 100644 test/core/end2end/fuzzers/client_fuzzer_corpus/8f980dd25f1c77e3536131c2c620aa32e8c13180 create mode 100644 test/core/end2end/fuzzers/client_fuzzer_corpus/aef36c49d7dec0dcf8cdc224d9e9221fa2cb1db0 create mode 100644 test/core/end2end/fuzzers/client_fuzzer_corpus/crash-14ed70cd9ea7987cdd0c8f6e39398ee7c60ee2ff create mode 100644 test/core/end2end/fuzzers/client_fuzzer_corpus/dcb06a6e34cbed15515e5b3581ca666f704777bd create mode 100644 test/core/end2end/fuzzers/client_fuzzer_corpus/ea46b684f1e67a27c231f2d536c41da631189b9c diff --git a/src/core/ext/transport/chttp2/transport/frame_rst_stream.c b/src/core/ext/transport/chttp2/transport/frame_rst_stream.c index a7aefb99158..e3a3c9e4a7c 100644 --- a/src/core/ext/transport/chttp2/transport/frame_rst_stream.c +++ b/src/core/ext/transport/chttp2/transport/frame_rst_stream.c @@ -102,12 +102,14 @@ grpc_error *grpc_chttp2_rst_stream_parser_parse( if (p->byte == 4) { GPR_ASSERT(is_last); stream_parsing->received_close = 1; - stream_parsing->forced_close_error = grpc_error_set_int( - GRPC_ERROR_CREATE("RST_STREAM"), GRPC_ERROR_INT_HTTP2_ERROR, - (intptr_t)((((uint32_t)p->reason_bytes[0]) << 24) | - (((uint32_t)p->reason_bytes[1]) << 16) | - (((uint32_t)p->reason_bytes[2]) << 8) | - (((uint32_t)p->reason_bytes[3])))); + if (stream_parsing->forced_close_error == GRPC_ERROR_NONE) { + stream_parsing->forced_close_error = grpc_error_set_int( + GRPC_ERROR_CREATE("RST_STREAM"), GRPC_ERROR_INT_HTTP2_ERROR, + (intptr_t)((((uint32_t)p->reason_bytes[0]) << 24) | + (((uint32_t)p->reason_bytes[1]) << 16) | + (((uint32_t)p->reason_bytes[2]) << 8) | + (((uint32_t)p->reason_bytes[3])))); + } } return GRPC_ERROR_NONE; diff --git a/test/core/end2end/fuzzers/client_fuzzer_corpus/2c452818a10ddef09b90c89a53db14b9b56b21f3 b/test/core/end2end/fuzzers/client_fuzzer_corpus/2c452818a10ddef09b90c89a53db14b9b56b21f3 new file mode 100644 index 0000000000000000000000000000000000000000..059634fda10a679b7bb4f753fe06660888b26dd4 GIT binary patch literal 52 zcmZS3WMpJ#`1cCr>)SNic&kq1P C_Z2e$ literal 0 HcmV?d00001 diff --git a/test/core/end2end/fuzzers/client_fuzzer_corpus/42ead79c94eccdf8a8c3d8036be73e14fa260dd5 b/test/core/end2end/fuzzers/client_fuzzer_corpus/42ead79c94eccdf8a8c3d8036be73e14fa260dd5 new file mode 100644 index 0000000000000000000000000000000000000000..b9c53b26edd94105bc4bdb24e73697cc4be93dc4 GIT binary patch literal 64 zcmZQz&|+j^U|?Wm;7KnkNY*XM%uUTNE#U@pic1npN{a=Jfbu+y3``8!3=GU5r40Z7 LGqQs;X{`VNxT6g} literal 0 HcmV?d00001 diff --git a/test/core/end2end/fuzzers/client_fuzzer_corpus/4e05d6cf1c3f0c04f6ee92d09a53ee0fe35c085a b/test/core/end2end/fuzzers/client_fuzzer_corpus/4e05d6cf1c3f0c04f6ee92d09a53ee0fe35c085a new file mode 100644 index 0000000000000000000000000000000000000000..8a4a279998d72b2aa9b94310364f40c2580515c2 GIT binary patch literal 64 zcmZQz&|+j^U|?Wm;7KnkNY*XM%uUTNE#X!Naf(Y4OG=9cjeznzj0{W+*$fQKAf*id M|1+|KG-<5>0Is79{{R30 literal 0 HcmV?d00001 diff --git a/test/core/end2end/fuzzers/client_fuzzer_corpus/8f980dd25f1c77e3536131c2c620aa32e8c13180 b/test/core/end2end/fuzzers/client_fuzzer_corpus/8f980dd25f1c77e3536131c2c620aa32e8c13180 new file mode 100644 index 0000000000000000000000000000000000000000..fcebab7a64f25b8dfad2cbb19851b76833a674b7 GIT binary patch literal 14 TcmZS3WMpJ#`1cCKLz? literal 0 HcmV?d00001 diff --git a/test/core/end2end/fuzzers/client_fuzzer_corpus/aef36c49d7dec0dcf8cdc224d9e9221fa2cb1db0 b/test/core/end2end/fuzzers/client_fuzzer_corpus/aef36c49d7dec0dcf8cdc224d9e9221fa2cb1db0 new file mode 100644 index 0000000000000000000000000000000000000000..6b015fe66e6131d746ff41da4b961b0ea074a02f GIT binary patch literal 53 wcmZQzU||3OMg|5&AjJ%3v*hM7Fz|zz42+Dqxr(`e{sRfWsS`IUP6RU40etxhaR2}S literal 0 HcmV?d00001 diff --git a/test/core/end2end/fuzzers/client_fuzzer_corpus/crash-14ed70cd9ea7987cdd0c8f6e39398ee7c60ee2ff b/test/core/end2end/fuzzers/client_fuzzer_corpus/crash-14ed70cd9ea7987cdd0c8f6e39398ee7c60ee2ff new file mode 100644 index 0000000000000000000000000000000000000000..be6366049d93b585d80d625c72ec8eabc458f8f5 GIT binary patch literal 719 zcma))!A%1(5Jl~fL&S-#01X$%d*ezZL|E_oKS9FwrjM^T*-qe*OC#i~h0zgCtcA1q!ShX>2v)Qh6$Z`_$uwt&j-F4ZR; z5Sv7_I-eYEmYW?vcl;IP1o{i~ad|8#&gc-zlgX7X`Jb@uP`1h)s-Wx}23A)%p@>wE za!C@8^?ad=qK}-dq9bit8{GOE-Go^{jRIXY;!;t6Li0vv6=jI|D`%R^2vry~vi7F? zj%DqTNjKogO;hm+{BgT~2z2|($`EkI19~LJC`TB;F;RQr8{}99X|={K^4w4OLpkCM RBFep{S=%Njy`zq%`UA}C*FFFM literal 0 HcmV?d00001 diff --git a/test/core/end2end/fuzzers/client_fuzzer_corpus/dcb06a6e34cbed15515e5b3581ca666f704777bd b/test/core/end2end/fuzzers/client_fuzzer_corpus/dcb06a6e34cbed15515e5b3581ca666f704777bd new file mode 100644 index 0000000000000000000000000000000000000000..92750f94a32152bdd71d388f47c4bcefb6ec34b8 GIT binary patch literal 238 zcmY+7F%AMj2t~o2!q`|@8)C7Jm9en!5Zu87aF}11*jjo!PvH2O)odig1Rng4NKyI5 zR^;7UGq%I&coxyJ30Y55)dt5*_uO3%k6!p!;!iRS_{tfJY8zgY`Q<}Y|ly-<3XI{!F o4-v0C#t0){Z=6m|-N-+eun0W93o~cbV+vGj(5A~7+}Qfx0kNN|?f?J) literal 0 HcmV?d00001 diff --git a/tools/run_tests/tests.json b/tools/run_tests/tests.json index 3ed7a6bc476..2353ac85831 100644 --- a/tools/run_tests/tests.json +++ b/tools/run_tests/tests.json @@ -54188,6 +54188,25 @@ ], "uses_polling": false }, + { + "args": [ + "test/core/end2end/fuzzers/client_fuzzer_corpus/2c452818a10ddef09b90c89a53db14b9b56b21f3" + ], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 0.1, + "exclude_configs": [ + "tsan" + ], + "flaky": false, + "language": "c", + "name": "client_fuzzer_one_entry", + "platforms": [ + "linux" + ], + "uses_polling": false + }, { "args": [ "test/core/end2end/fuzzers/client_fuzzer_corpus/2c6e69067c68c145dc5d3a60b86d8081fdf95d0d" @@ -54986,6 +55005,25 @@ ], "uses_polling": false }, + { + "args": [ + "test/core/end2end/fuzzers/client_fuzzer_corpus/42ead79c94eccdf8a8c3d8036be73e14fa260dd5" + ], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 0.1, + "exclude_configs": [ + "tsan" + ], + "flaky": false, + "language": "c", + "name": "client_fuzzer_one_entry", + "platforms": [ + "linux" + ], + "uses_polling": false + }, { "args": [ "test/core/end2end/fuzzers/client_fuzzer_corpus/43202ad9b1a689d919ab9ae91c2d0223394867bf" @@ -55328,6 +55366,25 @@ ], "uses_polling": false }, + { + "args": [ + "test/core/end2end/fuzzers/client_fuzzer_corpus/4e05d6cf1c3f0c04f6ee92d09a53ee0fe35c085a" + ], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 0.1, + "exclude_configs": [ + "tsan" + ], + "flaky": false, + "language": "c", + "name": "client_fuzzer_one_entry", + "platforms": [ + "linux" + ], + "uses_polling": false + }, { "args": [ "test/core/end2end/fuzzers/client_fuzzer_corpus/4e21c4b5c454df51c102f09ea1ba78c42133ee16" @@ -57247,6 +57304,25 @@ ], "uses_polling": false }, + { + "args": [ + "test/core/end2end/fuzzers/client_fuzzer_corpus/8f980dd25f1c77e3536131c2c620aa32e8c13180" + ], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 0.1, + "exclude_configs": [ + "tsan" + ], + "flaky": false, + "language": "c", + "name": "client_fuzzer_one_entry", + "platforms": [ + "linux" + ], + "uses_polling": false + }, { "args": [ "test/core/end2end/fuzzers/client_fuzzer_corpus/90a9c3390752b94ca19a58cb2fe6267bc818f718" @@ -58463,6 +58539,25 @@ ], "uses_polling": false }, + { + "args": [ + "test/core/end2end/fuzzers/client_fuzzer_corpus/aef36c49d7dec0dcf8cdc224d9e9221fa2cb1db0" + ], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 0.1, + "exclude_configs": [ + "tsan" + ], + "flaky": false, + "language": "c", + "name": "client_fuzzer_one_entry", + "platforms": [ + "linux" + ], + "uses_polling": false + }, { "args": [ "test/core/end2end/fuzzers/client_fuzzer_corpus/af8b24ffaecdfaf96c0cd7c76f31dc9e1b4d0935" @@ -59508,6 +59603,25 @@ ], "uses_polling": false }, + { + "args": [ + "test/core/end2end/fuzzers/client_fuzzer_corpus/crash-14ed70cd9ea7987cdd0c8f6e39398ee7c60ee2ff" + ], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 0.1, + "exclude_configs": [ + "tsan" + ], + "flaky": false, + "language": "c", + "name": "client_fuzzer_one_entry", + "platforms": [ + "linux" + ], + "uses_polling": false + }, { "args": [ "test/core/end2end/fuzzers/client_fuzzer_corpus/crash-3bd02c98286bfa7be8e13c5500ddb587bba74fbb" @@ -60173,6 +60287,25 @@ ], "uses_polling": false }, + { + "args": [ + "test/core/end2end/fuzzers/client_fuzzer_corpus/dcb06a6e34cbed15515e5b3581ca666f704777bd" + ], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 0.1, + "exclude_configs": [ + "tsan" + ], + "flaky": false, + "language": "c", + "name": "client_fuzzer_one_entry", + "platforms": [ + "linux" + ], + "uses_polling": false + }, { "args": [ "test/core/end2end/fuzzers/client_fuzzer_corpus/dccd1fd6d3394f5f68c87950ed7356a2e9ef0f6f" @@ -60667,6 +60800,25 @@ ], "uses_polling": false }, + { + "args": [ + "test/core/end2end/fuzzers/client_fuzzer_corpus/ea46b684f1e67a27c231f2d536c41da631189b9c" + ], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 0.1, + "exclude_configs": [ + "tsan" + ], + "flaky": false, + "language": "c", + "name": "client_fuzzer_one_entry", + "platforms": [ + "linux" + ], + "uses_polling": false + }, { "args": [ "test/core/end2end/fuzzers/client_fuzzer_corpus/eb969b9ab1b0d6b5d197795223ba7a091ebd8460" From 56565fca7532fcc559cd8149e437eb0a7ecdfd9b Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Mon, 20 Jun 2016 08:36:53 -0700 Subject: [PATCH 098/280] Force receiving a SETTINGS frame as the first frame --- .../ext/transport/chttp2/transport/chttp2_transport.c | 3 ++- src/core/ext/transport/chttp2/transport/internal.h | 1 + src/core/ext/transport/chttp2/transport/parsing.c | 11 +++++++++++ 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/src/core/ext/transport/chttp2/transport/chttp2_transport.c b/src/core/ext/transport/chttp2/transport/chttp2_transport.c index 9aa39ba26c2..a8c4b271fe1 100644 --- a/src/core/ext/transport/chttp2/transport/chttp2_transport.c +++ b/src/core/ext/transport/chttp2/transport/chttp2_transport.c @@ -264,6 +264,7 @@ static void init_transport(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport *t, t->parsing.is_client = is_client; t->parsing.deframe_state = is_client ? GRPC_DTS_FH_0 : GRPC_DTS_CLIENT_PREFIX_0; + t->parsing.is_first_frame = true; t->writing.is_client = is_client; t->optional_drop_message = gpr_empty_slice(); grpc_connectivity_state_init( @@ -1805,7 +1806,7 @@ static void post_reading_action_locked(grpc_exec_ctx *exec_ctx, UNREF_TRANSPORT(exec_ctx, t, "reading_action"); } - GRPC_ERROR_UNREF(error); + GRPC_LOG_IF_ERROR("close_transport", error); } /******************************************************************************* diff --git a/src/core/ext/transport/chttp2/transport/internal.h b/src/core/ext/transport/chttp2/transport/internal.h index 54e72ebd249..64385b43724 100644 --- a/src/core/ext/transport/chttp2/transport/internal.h +++ b/src/core/ext/transport/chttp2/transport/internal.h @@ -265,6 +265,7 @@ struct grpc_chttp2_transport_parsing { uint8_t incoming_frame_type; uint8_t incoming_frame_flags; uint8_t header_eof; + bool is_first_frame; uint32_t expect_continuation_stream_id; uint32_t incoming_frame_size; uint32_t incoming_stream_id; diff --git a/src/core/ext/transport/chttp2/transport/parsing.c b/src/core/ext/transport/chttp2/transport/parsing.c index 785134091f5..991d7729af4 100644 --- a/src/core/ext/transport/chttp2/transport/parsing.c +++ b/src/core/ext/transport/chttp2/transport/parsing.c @@ -468,6 +468,17 @@ grpc_error *grpc_chttp2_perform_read( static grpc_error *init_frame_parser( grpc_exec_ctx *exec_ctx, grpc_chttp2_transport_parsing *transport_parsing) { + if (transport_parsing->is_first_frame && + transport_parsing->incoming_frame_type != GRPC_CHTTP2_FRAME_SETTINGS) { + char *msg; + gpr_asprintf( + &msg, "Expected SETTINGS frame as the first frame, got frame type %d", + transport_parsing->incoming_frame_type); + grpc_error *err = GRPC_ERROR_CREATE(msg); + gpr_free(msg); + return err; + } + transport_parsing->is_first_frame = false; if (transport_parsing->expect_continuation_stream_id != 0) { if (transport_parsing->incoming_frame_type != GRPC_CHTTP2_FRAME_CONTINUATION) { From b187895fc43a84d75b64aa58d21d5caf9b80ba61 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Mon, 20 Jun 2016 09:11:54 -0700 Subject: [PATCH 099/280] Fix windows build error --- src/core/lib/iomgr/tcp_server_windows.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/core/lib/iomgr/tcp_server_windows.c b/src/core/lib/iomgr/tcp_server_windows.c index 2a51671ec71..4604f56dac7 100644 --- a/src/core/lib/iomgr/tcp_server_windows.c +++ b/src/core/lib/iomgr/tcp_server_windows.c @@ -103,6 +103,7 @@ struct grpc_tcp_server { /* Public function. Allocates the proper data structures to hold a grpc_tcp_server. */ grpc_error *grpc_tcp_server_create(grpc_closure *shutdown_complete, + const grpc_channel_args *args, grpc_tcp_server **server) { grpc_tcp_server *s = gpr_malloc(sizeof(grpc_tcp_server)); gpr_ref_init(&s->refs, 1); From daec5f94d2805db094cd748dbb691e25185af3c6 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Mon, 20 Jun 2016 09:24:45 -0700 Subject: [PATCH 100/280] We dont need to account for parsing or not when writing --- src/core/ext/transport/chttp2/transport/chttp2_transport.c | 3 +-- src/core/ext/transport/chttp2/transport/internal.h | 3 +-- src/core/ext/transport/chttp2/transport/writing.c | 4 ++-- 3 files changed, 4 insertions(+), 6 deletions(-) diff --git a/src/core/ext/transport/chttp2/transport/chttp2_transport.c b/src/core/ext/transport/chttp2/transport/chttp2_transport.c index a8c4b271fe1..4eb5ba8270e 100644 --- a/src/core/ext/transport/chttp2/transport/chttp2_transport.c +++ b/src/core/ext/transport/chttp2/transport/chttp2_transport.c @@ -644,8 +644,7 @@ static void finish_global_actions(grpc_exec_ctx *exec_ctx, for (;;) { if (!t->executor.writing_active && !t->closed && - grpc_chttp2_unlocking_check_writes(exec_ctx, &t->global, &t->writing, - t->executor.parsing_active)) { + grpc_chttp2_unlocking_check_writes(exec_ctx, &t->global, &t->writing)) { t->executor.writing_active = 1; REF_TRANSPORT(t, "writing"); prevent_endpoint_shutdown(t); diff --git a/src/core/ext/transport/chttp2/transport/internal.h b/src/core/ext/transport/chttp2/transport/internal.h index 64385b43724..909c406f8a8 100644 --- a/src/core/ext/transport/chttp2/transport/internal.h +++ b/src/core/ext/transport/chttp2/transport/internal.h @@ -525,8 +525,7 @@ struct grpc_chttp2_stream { are required, and schedule them if so */ int grpc_chttp2_unlocking_check_writes(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport_global *global, - grpc_chttp2_transport_writing *writing, - int is_parsing); + grpc_chttp2_transport_writing *writing); void grpc_chttp2_perform_writes( grpc_exec_ctx *exec_ctx, grpc_chttp2_transport_writing *transport_writing, grpc_endpoint *endpoint); diff --git a/src/core/ext/transport/chttp2/transport/writing.c b/src/core/ext/transport/chttp2/transport/writing.c index add76781827..b19f5f068df 100644 --- a/src/core/ext/transport/chttp2/transport/writing.c +++ b/src/core/ext/transport/chttp2/transport/writing.c @@ -45,7 +45,7 @@ static void finalize_outbuf(grpc_exec_ctx *exec_ctx, int grpc_chttp2_unlocking_check_writes( grpc_exec_ctx *exec_ctx, grpc_chttp2_transport_global *transport_global, - grpc_chttp2_transport_writing *transport_writing, int is_parsing) { + grpc_chttp2_transport_writing *transport_writing) { grpc_chttp2_stream_global *stream_global; grpc_chttp2_stream_writing *stream_writing; @@ -61,7 +61,7 @@ int grpc_chttp2_unlocking_check_writes( [GRPC_CHTTP2_SETTINGS_HEADER_TABLE_SIZE]); if (transport_global->dirtied_local_settings && - !transport_global->sent_local_settings && !is_parsing) { + !transport_global->sent_local_settings) { gpr_slice_buffer_add( &transport_writing->outbuf, grpc_chttp2_settings_create( From 04e47f308ec57c42ec9a0dcdc25e72c03b66bc9e Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Mon, 20 Jun 2016 11:03:15 -0700 Subject: [PATCH 101/280] Direct additional log statement through custom logger --- src/node/src/server.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/node/src/server.js b/src/node/src/server.js index fd5153f1244..b3b414969ab 100644 --- a/src/node/src/server.js +++ b/src/node/src/server.js @@ -735,8 +735,8 @@ Server.prototype.addService = function(service, implementation) { } var impl; if (implementation[name] === undefined) { - console.warn('Method handler for %s expected but not provided', - attrs.path); + common.log(grpc.logVerbosity.ERROR, 'Method handler for ' + + attrs.path + ' expected but not provided'); impl = defaultHandler[method_type]; } else { impl = _.bind(implementation[name], implementation); From fa1f74e2267d8dfe97d617357033d2a0bf8423be Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Mon, 20 Jun 2016 11:11:44 -0700 Subject: [PATCH 102/280] Fix errored streams prematurely terminating, add a test --- Makefile | 2 + .../chttp2/transport/chttp2_transport.c | 4 +- .../ext/transport/chttp2/transport/internal.h | 1 + test/core/end2end/end2end_nosec_tests.c | 8 + test/core/end2end/end2end_tests.c | 8 + test/core/end2end/gen_build_yaml.py | 1 + .../end2end/tests/streaming_error_response.c | 278 ++++++++ tools/run_tests/sources_and_headers.json | 2 + tools/run_tests/tests.json | 629 +++++++++++++++++- .../end2end_nosec_tests.vcxproj | 2 + .../end2end_nosec_tests.vcxproj.filters | 3 + .../tests/end2end_tests/end2end_tests.vcxproj | 2 + .../end2end_tests.vcxproj.filters | 3 + 13 files changed, 932 insertions(+), 11 deletions(-) create mode 100644 test/core/end2end/tests/streaming_error_response.c diff --git a/Makefile b/Makefile index 13bc2f1871c..23e40cfb075 100644 --- a/Makefile +++ b/Makefile @@ -6389,6 +6389,7 @@ LIBEND2END_TESTS_SRC = \ test/core/end2end/tests/simple_delayed_request.c \ test/core/end2end/tests/simple_metadata.c \ test/core/end2end/tests/simple_request.c \ + test/core/end2end/tests/streaming_error_response.c \ test/core/end2end/tests/trailing_metadata.c \ PUBLIC_HEADERS_C += \ @@ -6465,6 +6466,7 @@ LIBEND2END_NOSEC_TESTS_SRC = \ test/core/end2end/tests/simple_delayed_request.c \ test/core/end2end/tests/simple_metadata.c \ test/core/end2end/tests/simple_request.c \ + test/core/end2end/tests/streaming_error_response.c \ test/core/end2end/tests/trailing_metadata.c \ PUBLIC_HEADERS_C += \ diff --git a/src/core/ext/transport/chttp2/transport/chttp2_transport.c b/src/core/ext/transport/chttp2/transport/chttp2_transport.c index 9aa39ba26c2..63d07108a7c 100644 --- a/src/core/ext/transport/chttp2/transport/chttp2_transport.c +++ b/src/core/ext/transport/chttp2/transport/chttp2_transport.c @@ -1093,6 +1093,7 @@ static void perform_stream_op_locked(grpc_exec_ctx *exec_ctx, stream_global->recv_trailing_metadata_finished = add_closure_barrier(on_complete); stream_global->recv_trailing_metadata = op->recv_trailing_metadata; + stream_global->final_metadata_requested = true; grpc_chttp2_list_add_check_read_ops(transport_global, stream_global); } @@ -1246,7 +1247,8 @@ static void check_read_ops(grpc_exec_ctx *exec_ctx, stream_global->recv_initial_metadata_ready = NULL; } if (stream_global->recv_message_ready != NULL) { - while (stream_global->seen_error && + while (stream_global->final_metadata_requested && + stream_global->seen_error && (bs = grpc_chttp2_incoming_frame_queue_pop( &stream_global->incoming_frames)) != NULL) { incoming_byte_stream_destroy_locked(exec_ctx, NULL, NULL, bs); diff --git a/src/core/ext/transport/chttp2/transport/internal.h b/src/core/ext/transport/chttp2/transport/internal.h index 54e72ebd249..48602fb47ce 100644 --- a/src/core/ext/transport/chttp2/transport/internal.h +++ b/src/core/ext/transport/chttp2/transport/internal.h @@ -440,6 +440,7 @@ typedef struct { bool published_initial_metadata; bool published_trailing_metadata; + bool final_metadata_requested; grpc_chttp2_incoming_metadata_buffer received_initial_metadata; grpc_chttp2_incoming_metadata_buffer received_trailing_metadata; diff --git a/test/core/end2end/end2end_nosec_tests.c b/test/core/end2end/end2end_nosec_tests.c index b71299c09e0..2893bddc43c 100644 --- a/test/core/end2end/end2end_nosec_tests.c +++ b/test/core/end2end/end2end_nosec_tests.c @@ -115,6 +115,8 @@ extern void simple_metadata(grpc_end2end_test_config config); extern void simple_metadata_pre_init(void); extern void simple_request(grpc_end2end_test_config config); extern void simple_request_pre_init(void); +extern void streaming_error_response(grpc_end2end_test_config config); +extern void streaming_error_response_pre_init(void); extern void trailing_metadata(grpc_end2end_test_config config); extern void trailing_metadata_pre_init(void); @@ -157,6 +159,7 @@ void grpc_end2end_tests_pre_init(void) { simple_delayed_request_pre_init(); simple_metadata_pre_init(); simple_request_pre_init(); + streaming_error_response_pre_init(); trailing_metadata_pre_init(); } @@ -203,6 +206,7 @@ void grpc_end2end_tests(int argc, char **argv, simple_delayed_request(config); simple_metadata(config); simple_request(config); + streaming_error_response(config); trailing_metadata(config); return; } @@ -352,6 +356,10 @@ void grpc_end2end_tests(int argc, char **argv, simple_request(config); continue; } + if (0 == strcmp("streaming_error_response", argv[i])) { + streaming_error_response(config); + continue; + } if (0 == strcmp("trailing_metadata", argv[i])) { trailing_metadata(config); continue; diff --git a/test/core/end2end/end2end_tests.c b/test/core/end2end/end2end_tests.c index 00c9c44a78c..96a38e76dcc 100644 --- a/test/core/end2end/end2end_tests.c +++ b/test/core/end2end/end2end_tests.c @@ -117,6 +117,8 @@ extern void simple_metadata(grpc_end2end_test_config config); extern void simple_metadata_pre_init(void); extern void simple_request(grpc_end2end_test_config config); extern void simple_request_pre_init(void); +extern void streaming_error_response(grpc_end2end_test_config config); +extern void streaming_error_response_pre_init(void); extern void trailing_metadata(grpc_end2end_test_config config); extern void trailing_metadata_pre_init(void); @@ -160,6 +162,7 @@ void grpc_end2end_tests_pre_init(void) { simple_delayed_request_pre_init(); simple_metadata_pre_init(); simple_request_pre_init(); + streaming_error_response_pre_init(); trailing_metadata_pre_init(); } @@ -207,6 +210,7 @@ void grpc_end2end_tests(int argc, char **argv, simple_delayed_request(config); simple_metadata(config); simple_request(config); + streaming_error_response(config); trailing_metadata(config); return; } @@ -360,6 +364,10 @@ void grpc_end2end_tests(int argc, char **argv, simple_request(config); continue; } + if (0 == strcmp("streaming_error_response", argv[i])) { + streaming_error_response(config); + continue; + } if (0 == strcmp("trailing_metadata", argv[i])) { trailing_metadata(config); continue; diff --git a/test/core/end2end/gen_build_yaml.py b/test/core/end2end/gen_build_yaml.py index 325d9b3cad0..6d3d8f8d3c4 100755 --- a/test/core/end2end/gen_build_yaml.py +++ b/test/core/end2end/gen_build_yaml.py @@ -126,6 +126,7 @@ END2END_TESTS = { 'simple_delayed_request': connectivity_test_options, 'simple_metadata': default_test_options, 'simple_request': default_test_options, + 'streaming_error_response': default_test_options, 'trailing_metadata': default_test_options, } diff --git a/test/core/end2end/tests/streaming_error_response.c b/test/core/end2end/tests/streaming_error_response.c new file mode 100644 index 00000000000..c1d6f73ecdc --- /dev/null +++ b/test/core/end2end/tests/streaming_error_response.c @@ -0,0 +1,278 @@ +/* + * + * Copyright 2015, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include "test/core/end2end/end2end_tests.h" + +#include +#include + +#include +#include +#include +#include +#include +#include "test/core/end2end/cq_verifier.h" + +enum { TIMEOUT = 200000 }; + +static void *tag(intptr_t t) { return (void *)t; } + +static grpc_end2end_test_fixture begin_test(grpc_end2end_test_config config, + const char *test_name, + grpc_channel_args *client_args, + grpc_channel_args *server_args, + bool request_status_early) { + grpc_end2end_test_fixture f; + gpr_log(GPR_INFO, "%s/%s/request_status_early=%s", test_name, config.name, + request_status_early ? "true" : "false"); + f = config.create_fixture(client_args, server_args); + config.init_server(&f, server_args); + config.init_client(&f, client_args); + return f; +} + +static gpr_timespec n_seconds_time(int n) { + return GRPC_TIMEOUT_SECONDS_TO_DEADLINE(n); +} + +static gpr_timespec five_seconds_time(void) { return n_seconds_time(5); } + +static void drain_cq(grpc_completion_queue *cq) { + grpc_event ev; + do { + ev = grpc_completion_queue_next(cq, five_seconds_time(), NULL); + } while (ev.type != GRPC_QUEUE_SHUTDOWN); +} + +static void shutdown_server(grpc_end2end_test_fixture *f) { + if (!f->server) return; + grpc_server_shutdown_and_notify(f->server, f->cq, tag(1000)); + GPR_ASSERT(grpc_completion_queue_pluck( + f->cq, tag(1000), GRPC_TIMEOUT_SECONDS_TO_DEADLINE(5), NULL) + .type == GRPC_OP_COMPLETE); + grpc_server_destroy(f->server); + f->server = NULL; +} + +static void shutdown_client(grpc_end2end_test_fixture *f) { + if (!f->client) return; + grpc_channel_destroy(f->client); + f->client = NULL; +} + +static void end_test(grpc_end2end_test_fixture *f) { + shutdown_server(f); + shutdown_client(f); + + grpc_completion_queue_shutdown(f->cq); + drain_cq(f->cq); + grpc_completion_queue_destroy(f->cq); +} + +/* Client sends a request with payload, server reads then returns status. */ +static void test(grpc_end2end_test_config config, bool request_status_early) { + grpc_call *c; + grpc_call *s; + gpr_slice response_payload1_slice = gpr_slice_from_copied_string("hello"); + grpc_byte_buffer *response_payload1 = + grpc_raw_byte_buffer_create(&response_payload1_slice, 1); + gpr_slice response_payload2_slice = gpr_slice_from_copied_string("world"); + grpc_byte_buffer *response_payload2 = + grpc_raw_byte_buffer_create(&response_payload2_slice, 1); + gpr_timespec deadline = five_seconds_time(); + grpc_end2end_test_fixture f = begin_test(config, "streaming_error_response", + NULL, NULL, request_status_early); + cq_verifier *cqv = cq_verifier_create(f.cq); + grpc_op ops[6]; + grpc_op *op; + grpc_metadata_array initial_metadata_recv; + grpc_metadata_array trailing_metadata_recv; + grpc_metadata_array request_metadata_recv; + grpc_byte_buffer *response_payload1_recv = NULL; + grpc_byte_buffer *response_payload2_recv = NULL; + grpc_call_details call_details; + grpc_status_code status; + grpc_call_error error; + char *details = NULL; + size_t details_capacity = 0; + int was_cancelled = 2; + + c = grpc_channel_create_call(f.client, NULL, GRPC_PROPAGATE_DEFAULTS, f.cq, + "/foo", "foo.test.google.fr", deadline, NULL); + GPR_ASSERT(c); + + grpc_metadata_array_init(&initial_metadata_recv); + grpc_metadata_array_init(&trailing_metadata_recv); + grpc_metadata_array_init(&request_metadata_recv); + grpc_call_details_init(&call_details); + + memset(ops, 0, sizeof(ops)); + op = ops; + op->op = GRPC_OP_SEND_INITIAL_METADATA; + op->data.send_initial_metadata.count = 0; + op++; + op->op = GRPC_OP_SEND_CLOSE_FROM_CLIENT; + op++; + op->op = GRPC_OP_RECV_INITIAL_METADATA; + op->data.recv_initial_metadata = &initial_metadata_recv; + op++; + op->op = GRPC_OP_RECV_MESSAGE; + op->data.recv_message = &response_payload1_recv; + op++; + if (request_status_early) { + op->op = GRPC_OP_RECV_STATUS_ON_CLIENT; + op->data.recv_status_on_client.trailing_metadata = &trailing_metadata_recv; + op->data.recv_status_on_client.status = &status; + op->data.recv_status_on_client.status_details = &details; + op->data.recv_status_on_client.status_details_capacity = &details_capacity; + op++; + } + error = grpc_call_start_batch(c, ops, (size_t)(op - ops), tag(1), NULL); + GPR_ASSERT(GRPC_CALL_OK == error); + + GPR_ASSERT(GRPC_CALL_OK == grpc_server_request_call( + f.server, &s, &call_details, + &request_metadata_recv, f.cq, f.cq, tag(101))); + cq_expect_completion(cqv, tag(101), 1); + cq_verify(cqv); + + memset(ops, 0, sizeof(ops)); + op = ops; + op->op = GRPC_OP_SEND_INITIAL_METADATA; + op->data.send_initial_metadata.count = 0; + op++; + op->op = GRPC_OP_SEND_MESSAGE; + op->data.send_message = response_payload1; + op++; + error = grpc_call_start_batch(s, ops, (size_t)(op - ops), tag(102), NULL); + GPR_ASSERT(GRPC_CALL_OK == error); + + cq_expect_completion(cqv, tag(102), 1); + cq_verify(cqv); + + memset(ops, 0, sizeof(ops)); + op = ops; + op->op = GRPC_OP_SEND_MESSAGE; + op->data.send_message = response_payload2; + op++; + error = grpc_call_start_batch(s, ops, (size_t)(op - ops), tag(103), NULL); + GPR_ASSERT(GRPC_CALL_OK == error); + + cq_expect_completion(cqv, tag(103), 1); + if (!request_status_early) { + cq_expect_completion(cqv, tag(1), 1); + } + cq_verify(cqv); + + memset(ops, 0, sizeof(ops)); + op = ops; + op->op = GRPC_OP_RECV_CLOSE_ON_SERVER; + op->data.recv_close_on_server.cancelled = &was_cancelled; + op++; + op->op = GRPC_OP_SEND_STATUS_FROM_SERVER; + op->data.send_status_from_server.trailing_metadata_count = 0; + op->data.send_status_from_server.status = GRPC_STATUS_FAILED_PRECONDITION; + op->data.send_status_from_server.status_details = "xyz"; + op++; + error = grpc_call_start_batch(s, ops, (size_t)(op - ops), tag(104), NULL); + GPR_ASSERT(GRPC_CALL_OK == error); + + if (!request_status_early) { + memset(ops, 0, sizeof(ops)); + op = ops; + op->op = GRPC_OP_RECV_MESSAGE; + op->data.recv_message = &response_payload2_recv; + op++; + error = grpc_call_start_batch(c, ops, (size_t)(op - ops), tag(2), NULL); + GPR_ASSERT(GRPC_CALL_OK == error); + } + + cq_expect_completion(cqv, tag(104), 1); + if (request_status_early) { + cq_expect_completion(cqv, tag(1), 1); + } else { + cq_expect_completion(cqv, tag(2), 1); + } + cq_verify(cqv); + + if (!request_status_early) { + memset(ops, 0, sizeof(ops)); + op = ops; + op->op = GRPC_OP_RECV_STATUS_ON_CLIENT; + op->data.recv_status_on_client.trailing_metadata = &trailing_metadata_recv; + op->data.recv_status_on_client.status = &status; + op->data.recv_status_on_client.status_details = &details; + op->data.recv_status_on_client.status_details_capacity = &details_capacity; + op++; + error = grpc_call_start_batch(c, ops, (size_t)(op - ops), tag(3), NULL); + GPR_ASSERT(GRPC_CALL_OK == error); + + cq_expect_completion(cqv, tag(3), 1); + cq_verify(cqv); + + GPR_ASSERT(response_payload1_recv != NULL); + GPR_ASSERT(response_payload2_recv != NULL); + } + + GPR_ASSERT(status == GRPC_STATUS_FAILED_PRECONDITION); + GPR_ASSERT(0 == strcmp(details, "xyz")); + GPR_ASSERT(0 == strcmp(call_details.method, "/foo")); + GPR_ASSERT(0 == strcmp(call_details.host, "foo.test.google.fr")); + GPR_ASSERT(was_cancelled == 1); + + gpr_free(details); + grpc_metadata_array_destroy(&initial_metadata_recv); + grpc_metadata_array_destroy(&trailing_metadata_recv); + grpc_metadata_array_destroy(&request_metadata_recv); + grpc_call_details_destroy(&call_details); + + grpc_call_destroy(c); + grpc_call_destroy(s); + + cq_verifier_destroy(cqv); + + grpc_byte_buffer_destroy(response_payload1); + grpc_byte_buffer_destroy(response_payload2); + grpc_byte_buffer_destroy(response_payload1_recv); + grpc_byte_buffer_destroy(response_payload2_recv); + + end_test(&f); + config.tear_down_data(&f); +} + +void streaming_error_response(grpc_end2end_test_config config) { + test(config, false); + test(config, true); +} + +void streaming_error_response_pre_init(void) {} diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index 57e3bd63f42..750d98b0be0 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -5400,6 +5400,7 @@ "test/core/end2end/tests/simple_delayed_request.c", "test/core/end2end/tests/simple_metadata.c", "test/core/end2end/tests/simple_request.c", + "test/core/end2end/tests/streaming_error_response.c", "test/core/end2end/tests/trailing_metadata.c" ], "third_party": false, @@ -5458,6 +5459,7 @@ "test/core/end2end/tests/simple_delayed_request.c", "test/core/end2end/tests/simple_metadata.c", "test/core/end2end/tests/simple_request.c", + "test/core/end2end/tests/streaming_error_response.c", "test/core/end2end/tests/trailing_metadata.c" ], "third_party": false, diff --git a/tools/run_tests/tests.json b/tools/run_tests/tests.json index 3ed7a6bc476..f0e4849ec99 100644 --- a/tools/run_tests/tests.json +++ b/tools/run_tests/tests.json @@ -5186,6 +5186,28 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_census_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -6022,6 +6044,28 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_compress_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -6821,6 +6865,27 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_fakesec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -7482,6 +7547,26 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_fd_test", + "platforms": [ + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -8316,6 +8401,28 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -8930,6 +9037,22 @@ "linux" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full+pipe_test", + "platforms": [ + "linux" + ] + }, { "args": [ "trailing_metadata" @@ -9738,6 +9861,28 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full+trace_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -10574,6 +10719,28 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_loadreporting_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -11373,6 +11540,27 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_oauth2_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -12045,6 +12233,27 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_proxy_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -12738,6 +12947,27 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_sockpair_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -13410,6 +13640,27 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_sockpair+trace_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -14105,7 +14356,7 @@ }, { "args": [ - "trailing_metadata" + "streaming_error_response" ], "ci_platforms": [ "windows", @@ -14126,19 +14377,18 @@ }, { "args": [ - "bad_hostname" + "trailing_metadata" ], "ci_platforms": [ "windows", "linux", - "mac", "posix" ], "cpu_cost": 1.0, "exclude_configs": [], "flaky": false, "language": "c", - "name": "h2_ssl_test", + "name": "h2_sockpair_1byte_test", "platforms": [ "windows", "linux", @@ -14148,7 +14398,7 @@ }, { "args": [ - "binary_metadata" + "bad_hostname" ], "ci_platforms": [ "windows", @@ -14170,7 +14420,7 @@ }, { "args": [ - "call_creds" + "binary_metadata" ], "ci_platforms": [ "windows", @@ -14192,7 +14442,7 @@ }, { "args": [ - "cancel_after_accept" + "call_creds" ], "ci_platforms": [ "windows", @@ -14200,7 +14450,7 @@ "mac", "posix" ], - "cpu_cost": 0.1, + "cpu_cost": 1.0, "exclude_configs": [], "flaky": false, "language": "c", @@ -14214,7 +14464,7 @@ }, { "args": [ - "cancel_after_client_done" + "cancel_after_accept" ], "ci_platforms": [ "windows", @@ -14222,7 +14472,29 @@ "mac", "posix" ], - "cpu_cost": 1.0, + "cpu_cost": 0.1, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_ssl_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, + { + "args": [ + "cancel_after_client_done" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, "exclude_configs": [], "flaky": false, "language": "c", @@ -14938,6 +15210,28 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_ssl_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -15774,6 +16068,28 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_ssl_cert_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -16447,6 +16763,27 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_ssl_proxy_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -17188,6 +17525,26 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_uds_test", + "platforms": [ + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -18000,6 +18357,28 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_census_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -18814,6 +19193,28 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_compress_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -19456,6 +19857,26 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_fd_nosec_test", + "platforms": [ + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -20268,6 +20689,28 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -20866,6 +21309,22 @@ "linux" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full+pipe_nosec_test", + "platforms": [ + "linux" + ] + }, { "args": [ "trailing_metadata" @@ -21652,6 +22111,28 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full+trace_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -22466,6 +22947,28 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_loadreporting_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -23118,6 +23621,27 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_proxy_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -23790,6 +24314,27 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_sockpair_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -24441,6 +24986,27 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_sockpair+trace_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -25175,6 +25741,29 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [ + "msan" + ], + "flaky": false, + "language": "c", + "name": "h2_sockpair_1byte_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" @@ -25898,6 +26487,26 @@ "posix" ] }, + { + "args": [ + "streaming_error_response" + ], + "ci_platforms": [ + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_uds_nosec_test", + "platforms": [ + "linux", + "mac", + "posix" + ] + }, { "args": [ "trailing_metadata" diff --git a/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj b/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj index 22cd102d116..923c1d1ab41 100644 --- a/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj +++ b/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj @@ -225,6 +225,8 @@ + + diff --git a/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj.filters b/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj.filters index 1bb208bba8e..6533eaa057b 100644 --- a/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj.filters +++ b/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj.filters @@ -112,6 +112,9 @@ test\core\end2end\tests + + test\core\end2end\tests + test\core\end2end\tests diff --git a/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj b/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj index bfd437e8717..0b859e25ce4 100644 --- a/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj +++ b/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj @@ -227,6 +227,8 @@ + + diff --git a/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj.filters b/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj.filters index 61c065f77ca..ea1c5e3c237 100644 --- a/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj.filters +++ b/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj.filters @@ -115,6 +115,9 @@ test\core\end2end\tests + + test\core\end2end\tests + test\core\end2end\tests From 89dde5e5681a7a718b234311e302897f5861723c Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Mon, 20 Jun 2016 11:13:11 -0700 Subject: [PATCH 103/280] Fix copyright --- test/core/end2end/tests/streaming_error_response.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/core/end2end/tests/streaming_error_response.c b/test/core/end2end/tests/streaming_error_response.c index c1d6f73ecdc..e15c132d633 100644 --- a/test/core/end2end/tests/streaming_error_response.c +++ b/test/core/end2end/tests/streaming_error_response.c @@ -1,6 +1,6 @@ /* * - * Copyright 2015, Google Inc. + * Copyright 2016, Google Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without From ffaafe6fa3fbb308c00619bef45e9f43a3d8e4ba Mon Sep 17 00:00:00 2001 From: Nathaniel Manista Date: Mon, 20 Jun 2016 19:36:49 +0000 Subject: [PATCH 104/280] Change with_call from parameter to attribute --- src/python/grpcio/grpc/__init__.py | 57 +++++++++++++++---- src/python/grpcio/grpc/_channel.py | 32 ++++++++--- .../grpcio/grpc/beta/_client_adaptations.py | 8 +-- .../grpcio/tests/unit/_metadata_test.py | 8 +-- src/python/grpcio/tests/unit/_rpc_test.py | 21 +++---- 5 files changed, 87 insertions(+), 39 deletions(-) diff --git a/src/python/grpcio/grpc/__init__.py b/src/python/grpcio/grpc/__init__.py index 28adca37723..f9b09b4cf14 100644 --- a/src/python/grpcio/grpc/__init__.py +++ b/src/python/grpcio/grpc/__init__.py @@ -438,9 +438,7 @@ class UnaryUnaryMultiCallable(six.with_metaclass(abc.ABCMeta)): """Affords invoking a unary-unary RPC.""" @abc.abstractmethod - def __call__( - self, request, timeout=None, metadata=None, credentials=None, - with_call=False): + def __call__(self, request, timeout=None, metadata=None, credentials=None): """Synchronously invokes the underlying RPC. Args: @@ -449,12 +447,30 @@ class UnaryUnaryMultiCallable(six.with_metaclass(abc.ABCMeta)): metadata: An optional sequence of pairs of bytes to be transmitted to the service-side of the RPC. credentials: An optional CallCredentials for the RPC. - with_call: Whether or not to include return a Call for the RPC in addition - to the response. Returns: - The response value for the RPC, and a Call for the RPC if with_call was - set to True at invocation. + The response value for the RPC. + + Raises: + RpcError: Indicating that the RPC terminated with non-OK status. The + raised RpcError will also be a Call for the RPC affording the RPC's + metadata, status code, and details. + """ + raise NotImplementedError() + + @abc.abstractmethod + def with_call(self, request, timeout=None, metadata=None, credentials=None): + """Synchronously invokes the underlying RPC. + + Args: + request: The request value for the RPC. + timeout: An optional durating of time in seconds to allow for the RPC. + metadata: An optional sequence of pairs of bytes to be transmitted to the + service-side of the RPC. + credentials: An optional CallCredentials for the RPC. + + Returns: + The response value for the RPC and a Call value for the RPC. Raises: RpcError: Indicating that the RPC terminated with non-OK status. The @@ -510,8 +526,7 @@ class StreamUnaryMultiCallable(six.with_metaclass(abc.ABCMeta)): @abc.abstractmethod def __call__( - self, request_iterator, timeout=None, metadata=None, credentials=None, - with_call=False): + self, request_iterator, timeout=None, metadata=None, credentials=None): """Synchronously invokes the underlying RPC. Args: @@ -520,8 +535,6 @@ class StreamUnaryMultiCallable(six.with_metaclass(abc.ABCMeta)): metadata: An optional sequence of pairs of bytes to be transmitted to the service-side of the RPC. credentials: An optional CallCredentials for the RPC. - with_call: Whether or not to include return a Call for the RPC in addition - to the response. Returns: The response value for the RPC, and a Call for the RPC if with_call was @@ -534,6 +547,28 @@ class StreamUnaryMultiCallable(six.with_metaclass(abc.ABCMeta)): """ raise NotImplementedError() + @abc.abstractmethod + def with_call( + self, request_iterator, timeout=None, metadata=None, credentials=None): + """Synchronously invokes the underlying RPC. + + Args: + request_iterator: An iterator that yields request values for the RPC. + timeout: An optional duration of time in seconds to allow for the RPC. + metadata: An optional sequence of pairs of bytes to be transmitted to the + service-side of the RPC. + credentials: An optional CallCredentials for the RPC. + + Returns: + The response value for the RPC and a Call for the RPC. + + Raises: + RpcError: Indicating that the RPC terminated with non-OK status. The + raised RpcError will also be a Call for the RPC affording the RPC's + metadata, status code, and details. + """ + raise NotImplementedError() + @abc.abstractmethod def future( self, request_iterator, timeout=None, metadata=None, credentials=None): diff --git a/src/python/grpcio/grpc/_channel.py b/src/python/grpcio/grpc/_channel.py index d9315d2e6cd..8b774c44488 100644 --- a/src/python/grpcio/grpc/_channel.py +++ b/src/python/grpcio/grpc/_channel.py @@ -449,9 +449,7 @@ class _UnaryUnaryMultiCallable(grpc.UnaryUnaryMultiCallable): ) return state, operations, deadline, deadline_timespec, None - def __call__( - self, request, timeout=None, metadata=None, credentials=None, - with_call=False): + def _blocking(self, request, timeout, metadata, credentials): state, operations, deadline, deadline_timespec, rendezvous = self._prepare( request, timeout, metadata) if rendezvous: @@ -464,7 +462,15 @@ class _UnaryUnaryMultiCallable(grpc.UnaryUnaryMultiCallable): call.set_credentials(credentials._credentials) call.start_batch(cygrpc.Operations(operations), None) _handle_event(completion_queue.poll(), state, self._response_deserializer) - return _end_unary_response_blocking(state, with_call, deadline) + return state, deadline + + def __call__(self, request, timeout=None, metadata=None, credentials=None): + state, deadline, = self._blocking(request, timeout, metadata, credentials) + return _end_unary_response_blocking(state, False, deadline) + + def with_call(self, request, timeout=None, metadata=None, credentials=None): + state, deadline, = self._blocking(request, timeout, metadata, credentials) + return _end_unary_response_blocking(state, True, deadline) def future(self, request, timeout=None, metadata=None, credentials=None): state, operations, deadline, deadline_timespec, rendezvous = self._prepare( @@ -532,9 +538,7 @@ class _StreamUnaryMultiCallable(grpc.StreamUnaryMultiCallable): self._request_serializer = request_serializer self._response_deserializer = response_deserializer - def __call__( - self, request_iterator, timeout=None, metadata=None, credentials=None, - with_call=False): + def _blocking(self, request_iterator, timeout, metadata, credentials): deadline, deadline_timespec = _deadline(timeout) state = _RPCState(_STREAM_UNARY_INITIAL_DUE, None, None, None, None) completion_queue = cygrpc.CompletionQueue() @@ -563,7 +567,19 @@ class _StreamUnaryMultiCallable(grpc.StreamUnaryMultiCallable): state.condition.notify_all() if not state.due: break - return _end_unary_response_blocking(state, with_call, deadline) + return state, deadline + + def __call__( + self, request_iterator, timeout=None, metadata=None, credentials=None): + state, deadline, = self._blocking( + request_iterator, timeout, metadata, credentials) + return _end_unary_response_blocking(state, False, deadline) + + def with_call( + self, request_iterator, timeout=None, metadata=None, credentials=None): + state, deadline, = self._blocking( + request_iterator, timeout, metadata, credentials) + return _end_unary_response_blocking(state, True, deadline) def future( self, request_iterator, timeout=None, metadata=None, credentials=None): diff --git a/src/python/grpcio/grpc/beta/_client_adaptations.py b/src/python/grpcio/grpc/beta/_client_adaptations.py index 024808c5404..56456cc117f 100644 --- a/src/python/grpcio/grpc/beta/_client_adaptations.py +++ b/src/python/grpcio/grpc/beta/_client_adaptations.py @@ -186,9 +186,9 @@ def _blocking_unary_unary( response_deserializer=response_deserializer) effective_metadata = _effective_metadata(metadata, metadata_transformer) if with_call: - response, call = multi_callable( + response, call = multi_callable.with_call( request, timeout=timeout, metadata=effective_metadata, - credentials=_credentials(protocol_options), with_call=True) + credentials=_credentials(protocol_options)) return response, _Rendezvous(None, None, call) else: return multi_callable( @@ -237,9 +237,9 @@ def _blocking_stream_unary( response_deserializer=response_deserializer) effective_metadata = _effective_metadata(metadata, metadata_transformer) if with_call: - response, call = multi_callable( + response, call = multi_callable.with_call( request_iterator, timeout=timeout, metadata=effective_metadata, - credentials=_credentials(protocol_options), with_call=True) + credentials=_credentials(protocol_options)) return response, _Rendezvous(None, None, call) else: return multi_callable( diff --git a/src/python/grpcio/tests/unit/_metadata_test.py b/src/python/grpcio/tests/unit/_metadata_test.py index 77b39012619..2cb13f236bd 100644 --- a/src/python/grpcio/tests/unit/_metadata_test.py +++ b/src/python/grpcio/tests/unit/_metadata_test.py @@ -173,8 +173,8 @@ class MetadataTest(unittest.TestCase): def testUnaryUnary(self): multi_callable = self._channel.unary_unary(_UNARY_UNARY) - unused_response, call = multi_callable( - _REQUEST, metadata=_CLIENT_METADATA, with_call=True) + unused_response, call = multi_callable.with_call( + _REQUEST, metadata=_CLIENT_METADATA) self.assertTrue(test_common.metadata_transmitted( _SERVER_INITIAL_METADATA, call.initial_metadata())) self.assertTrue(test_common.metadata_transmitted( @@ -192,9 +192,9 @@ class MetadataTest(unittest.TestCase): def testStreamUnary(self): multi_callable = self._channel.stream_unary(_STREAM_UNARY) - unused_response, call = multi_callable( + unused_response, call = multi_callable.with_call( [_REQUEST] * test_constants.STREAM_LENGTH, - metadata=_CLIENT_METADATA, with_call=True) + metadata=_CLIENT_METADATA) self.assertTrue(test_common.metadata_transmitted( _SERVER_INITIAL_METADATA, call.initial_metadata())) self.assertTrue(test_common.metadata_transmitted( diff --git a/src/python/grpcio/tests/unit/_rpc_test.py b/src/python/grpcio/tests/unit/_rpc_test.py index 8407593c86d..9814504edff 100644 --- a/src/python/grpcio/tests/unit/_rpc_test.py +++ b/src/python/grpcio/tests/unit/_rpc_test.py @@ -27,7 +27,7 @@ # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -"""Test of gRPC Python's application-layer API.""" +"""Test of RPCs made against gRPC Python's application-layer API.""" import itertools import threading @@ -216,10 +216,9 @@ class RPCTest(unittest.TestCase): expected_response = self._handler.handle_unary_unary(request, None) multi_callable = _unary_unary_multi_callable(self._channel) - response, call = multi_callable( + response, call = multi_callable.with_call( request, metadata=( - (b'test', b'SuccessfulUnaryRequestBlockingUnaryResponseWithCall'),), - with_call=True) + (b'test', b'SuccessfulUnaryRequestBlockingUnaryResponseWithCall'),)) self.assertEqual(expected_response, response) self.assertIs(grpc.StatusCode.OK, call.code()) @@ -266,11 +265,11 @@ class RPCTest(unittest.TestCase): request_iterator = iter(requests) multi_callable = _stream_unary_multi_callable(self._channel) - response, call = multi_callable( + response, call = multi_callable.with_call( request_iterator, metadata=( (b'test', b'SuccessfulStreamRequestBlockingUnaryResponseWithCall'), - ), with_call=True) + )) self.assertEqual(expected_response, response) self.assertIs(grpc.StatusCode.OK, call.code()) @@ -525,10 +524,9 @@ class RPCTest(unittest.TestCase): multi_callable = _unary_unary_multi_callable(self._channel) with self._control.pause(): with self.assertRaises(grpc.RpcError) as exception_context: - multi_callable( + multi_callable.with_call( request, timeout=test_constants.SHORT_TIMEOUT, - metadata=((b'test', b'ExpiredUnaryRequestBlockingUnaryResponse'),), - with_call=True) + metadata=((b'test', b'ExpiredUnaryRequestBlockingUnaryResponse'),)) self.assertIsNotNone(exception_context.exception.initial_metadata()) self.assertIs( @@ -640,10 +638,9 @@ class RPCTest(unittest.TestCase): multi_callable = _unary_unary_multi_callable(self._channel) with self._control.fail(): with self.assertRaises(grpc.RpcError) as exception_context: - multi_callable( + multi_callable.with_call( request, - metadata=((b'test', b'FailedUnaryRequestBlockingUnaryResponse'),), - with_call=True) + metadata=((b'test', b'FailedUnaryRequestBlockingUnaryResponse'),)) self.assertIs(grpc.StatusCode.UNKNOWN, exception_context.exception.code()) From 36a58a7928e6a10b44b6cffdc118337ef7e0b8c1 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Mon, 20 Jun 2016 14:01:07 -0700 Subject: [PATCH 105/280] Enable treating warnings as errors for target gRPC --- .../GRPCClient/GRPCCall+ChannelCredentials.h | 4 ++-- src/objective-c/ProtoRPC/ProtoMethod.h | 3 +++ src/objective-c/ProtoRPC/ProtoRPC.h | 3 +++ src/objective-c/ProtoRPC/ProtoService.h | 3 +++ src/objective-c/ProtoRPC/ProtoService.m | 6 +++--- src/objective-c/tests/Podfile | 21 ++++++++++++++++--- 6 files changed, 32 insertions(+), 8 deletions(-) diff --git a/src/objective-c/GRPCClient/GRPCCall+ChannelCredentials.h b/src/objective-c/GRPCClient/GRPCCall+ChannelCredentials.h index 343dd48a14e..ac2a37d75f2 100644 --- a/src/objective-c/GRPCClient/GRPCCall+ChannelCredentials.h +++ b/src/objective-c/GRPCClient/GRPCCall+ChannelCredentials.h @@ -41,7 +41,7 @@ */ + (BOOL)setTLSPEMRootCerts:(nullable NSString *)pemRootCert forHost:(nonnull NSString *)host - error:(NSError **)errorPtr; + error:(NSError * _Nullable * _Nullable)errorPtr; /** * Configures @c host with TLS/SSL Client Credentials and optionally trusted root Certificate * Authorities. If @c pemRootCerts is nil, the default CA Certificates bundled with gRPC will be @@ -51,6 +51,6 @@ withPrivateKey:(nullable NSString *)pemPrivateKey withCertChain:(nullable NSString *)pemCertChain forHost:(nonnull NSString *)host - error:(NSError **)errorPtr; + error:(NSError * _Nullable * _Nullable)errorPtr; @end diff --git a/src/objective-c/ProtoRPC/ProtoMethod.h b/src/objective-c/ProtoRPC/ProtoMethod.h index f9fdbb35ffd..ae3a2723fc4 100644 --- a/src/objective-c/ProtoRPC/ProtoMethod.h +++ b/src/objective-c/ProtoRPC/ProtoMethod.h @@ -54,6 +54,9 @@ __attribute__((deprecated("Please use GRPCProtoMethod."))) * This subclass is empty now. Eventually we'll remove ProtoMethod class * to avoid potential naming conflict */ +#pragma clang diagnostic push +#pragma clang diagnostic ignored "-Wdeprecated-declarations" @interface GRPCProtoMethod : ProtoMethod +#pragma clang diagnostic pop @end diff --git a/src/objective-c/ProtoRPC/ProtoRPC.h b/src/objective-c/ProtoRPC/ProtoRPC.h index 5f91f6bce1e..04fc1e45f16 100644 --- a/src/objective-c/ProtoRPC/ProtoRPC.h +++ b/src/objective-c/ProtoRPC/ProtoRPC.h @@ -56,6 +56,9 @@ __attribute__((deprecated("Please use GRPCProtoCall."))) * This subclass is empty now. Eventually we'll remove ProtoRPC class * to avoid potential naming conflict */ +#pragma clang diagnostic push +#pragma clang diagnostic ignored "-Wdeprecated-declarations" @interface GRPCProtoCall : ProtoRPC +#pragma clang diagnostic pop @end diff --git a/src/objective-c/ProtoRPC/ProtoService.h b/src/objective-c/ProtoRPC/ProtoService.h index 87d06e1ae59..7faae1b49c9 100644 --- a/src/objective-c/ProtoRPC/ProtoService.h +++ b/src/objective-c/ProtoRPC/ProtoService.h @@ -55,6 +55,9 @@ __attribute__((deprecated("Please use GRPCProtoService."))) * This subclass is empty now. Eventually we'll remove ProtoService class * to avoid potential naming conflict */ +#pragma clang diagnostic push +#pragma clang diagnostic ignored "-Wdeprecated-declarations" @interface GRPCProtoService : ProtoService +#pragma clang diagnostic pop @end diff --git a/src/objective-c/ProtoRPC/ProtoService.m b/src/objective-c/ProtoRPC/ProtoService.m index 597c3cf0fed..4a14570d818 100644 --- a/src/objective-c/ProtoRPC/ProtoService.m +++ b/src/objective-c/ProtoRPC/ProtoService.m @@ -65,14 +65,14 @@ return self; } -- (ProtoRPC *)RPCToMethod:(NSString *)method +- (GRPCProtoCall *)RPCToMethod:(NSString *)method requestsWriter:(GRXWriter *)requestsWriter responseClass:(Class)responseClass responsesWriteable:(id)responsesWriteable { - ProtoMethod *methodName = [[ProtoMethod alloc] initWithPackage:_packageName + GRPCProtoMethod *methodName = [[GRPCProtoMethod alloc] initWithPackage:_packageName service:_serviceName method:method]; - return [[ProtoRPC alloc] initWithHost:_host + return [[GRPCProtoCall alloc] initWithHost:_host method:methodName requestsWriter:requestsWriter responseClass:responseClass diff --git a/src/objective-c/tests/Podfile b/src/objective-c/tests/Podfile index 508641d681a..53edf8c890a 100644 --- a/src/objective-c/tests/Podfile +++ b/src/objective-c/tests/Podfile @@ -1,9 +1,9 @@ source 'https://github.com/CocoaPods/Specs.git' platform :ios, '8.0' -pod 'Protobuf', :path => "../../../third_party/protobuf" -pod 'BoringSSL', :podspec => ".." -pod 'CronetFramework', :podspec => ".." +pod 'Protobuf', :path => "../../../third_party/protobuf", :inhibit_warnings => true +pod 'BoringSSL', :podspec => "..", :inhibit_warnings => true +pod 'CronetFramework', :podspec => "..", :inhibit_warnings => true pod 'gRPC', :path => "../../.." pod 'RemoteTest', :path => "RemoteTestClient" @@ -30,3 +30,18 @@ end target 'InteropTestsLocalCleartext' do end + +post_install do |installer| + installer.pods_project.targets.each do |target| + target.build_configurations.each do |config| + config.build_settings['GCC_TREAT_WARNINGS_AS_ERRORS'] = 'YES' + end + if target.name == 'gRPC' + target.build_configurations.each do |config| + # GPR_UNREACHABLE_CODE causes "Control may reach end of non-void + # function" warning + config.build_settings['GCC_WARN_ABOUT_RETURN_TYPE'] = 'NO' + end + end + end +end From abc7427a4d113aeb7fde6d732c63ffea1f615998 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Mon, 20 Jun 2016 14:40:57 -0700 Subject: [PATCH 106/280] Update podfiles for objc examples --- examples/objective-c/auth_sample/AuthTestService.podspec | 4 ++++ examples/objective-c/helloworld/HelloWorld.podspec | 4 ++++ examples/objective-c/route_guide/RouteGuide.podspec | 4 ++++ src/objective-c/examples/RemoteTestClient/RemoteTest.podspec | 4 ++++ 4 files changed, 16 insertions(+) diff --git a/examples/objective-c/auth_sample/AuthTestService.podspec b/examples/objective-c/auth_sample/AuthTestService.podspec index e9356260534..d246653ea79 100644 --- a/examples/objective-c/auth_sample/AuthTestService.podspec +++ b/examples/objective-c/auth_sample/AuthTestService.podspec @@ -2,6 +2,10 @@ Pod::Spec.new do |s| s.name = "AuthTestService" s.version = "0.0.1" s.license = "New BSD" + s.authors = { 'gRPC contributors' => 'grpc-io@googlegroups.com' } + s.homepage = "http://www.grpc.io/" + s.summary = "AuthTestService example" + s.source = { :git => 'https://github.com/grpc/grpc.git' } s.ios.deployment_target = "7.1" s.osx.deployment_target = "10.9" diff --git a/examples/objective-c/helloworld/HelloWorld.podspec b/examples/objective-c/helloworld/HelloWorld.podspec index bdf782f6eaf..17b016b31a1 100644 --- a/examples/objective-c/helloworld/HelloWorld.podspec +++ b/examples/objective-c/helloworld/HelloWorld.podspec @@ -2,6 +2,10 @@ Pod::Spec.new do |s| s.name = "HelloWorld" s.version = "0.0.1" s.license = "New BSD" + s.authors = { 'gRPC contributors' => 'grpc-io@googlegroups.com' } + s.homepage = "http://www.grpc.io/" + s.summary = "HelloWorld example" + s.source = { :git => 'https://github.com/grpc/grpc.git' } s.ios.deployment_target = "7.1" s.osx.deployment_target = "10.9" diff --git a/examples/objective-c/route_guide/RouteGuide.podspec b/examples/objective-c/route_guide/RouteGuide.podspec index 4bc2c42cbb8..97a61ff51a5 100644 --- a/examples/objective-c/route_guide/RouteGuide.podspec +++ b/examples/objective-c/route_guide/RouteGuide.podspec @@ -2,6 +2,10 @@ Pod::Spec.new do |s| s.name = "RouteGuide" s.version = "0.0.1" s.license = "New BSD" + s.authors = { 'gRPC contributors' => 'grpc-io@googlegroups.com' } + s.homepage = "http://www.grpc.io/" + s.summary = "RouteGuide example" + s.source = { :git => 'https://github.com/grpc/grpc.git' } s.ios.deployment_target = "7.1" s.osx.deployment_target = "10.9" diff --git a/src/objective-c/examples/RemoteTestClient/RemoteTest.podspec b/src/objective-c/examples/RemoteTestClient/RemoteTest.podspec index 5addf26fc46..107e6de4e21 100644 --- a/src/objective-c/examples/RemoteTestClient/RemoteTest.podspec +++ b/src/objective-c/examples/RemoteTestClient/RemoteTest.podspec @@ -2,6 +2,10 @@ Pod::Spec.new do |s| s.name = "RemoteTest" s.version = "0.0.1" s.license = "New BSD" + s.authors = { 'gRPC contributors' => 'grpc-io@googlegroups.com' } + s.homepage = "http://www.grpc.io/" + s.summary = "RemoteTest example" + s.source = { :git => 'https://github.com/grpc/grpc.git' } s.ios.deployment_target = '7.1' s.osx.deployment_target = '10.9' From 3b5ae53cd149ecb821c8950b19b06c8330e288c0 Mon Sep 17 00:00:00 2001 From: Nathaniel Manista Date: Mon, 20 Jun 2016 22:09:39 +0000 Subject: [PATCH 107/280] Correct lingering FATAL_FAILURE sites This should have been done as part of 5444bed32f1405ebb53b0c37d3b. --- src/python/grpcio/tests/unit/_channel_connectivity_test.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/python/grpcio/tests/unit/_channel_connectivity_test.py b/src/python/grpcio/tests/unit/_channel_connectivity_test.py index a1575efada7..ae8de523ecf 100644 --- a/src/python/grpcio/tests/unit/_channel_connectivity_test.py +++ b/src/python/grpcio/tests/unit/_channel_connectivity_test.py @@ -135,12 +135,12 @@ class ChannelConnectivityTest(unittest.TestCase): self.assertNotIn( grpc.ChannelConnectivity.TRANSIENT_FAILURE, third_connectivities) self.assertNotIn( - grpc.ChannelConnectivity.FATAL_FAILURE, third_connectivities) + grpc.ChannelConnectivity.SHUTDOWN, third_connectivities) self.assertNotIn( grpc.ChannelConnectivity.TRANSIENT_FAILURE, fourth_connectivities) self.assertNotIn( - grpc.ChannelConnectivity.FATAL_FAILURE, fourth_connectivities) + grpc.ChannelConnectivity.SHUTDOWN, fourth_connectivities) def test_reachable_then_unreachable_channel_connectivity(self): server = _server.Server((), futures.ThreadPoolExecutor(max_workers=0)) From 7981e1905f61471f8caece6eea6051b2522b9235 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Mon, 20 Jun 2016 15:38:15 -0700 Subject: [PATCH 108/280] Network status tracking Recreated the old PR (https://github.com/grpc/grpc/pull/6283) in a new repo since the old repo was destroyed. Removed changes to tcp_posix.c and tpc_windows.c, instead depending on the idempotent endpoint shutdown code from https://github.com/grpc/grpc/pull/6911. --- src/core/lib/iomgr/network_status_tracker.c | 82 ++++++ src/core/lib/iomgr/network_status_tracker.h | 40 +++ .../end2end/tests/network_status_change.c | 243 ++++++++++++++++++ 3 files changed, 365 insertions(+) create mode 100644 src/core/lib/iomgr/network_status_tracker.c create mode 100644 src/core/lib/iomgr/network_status_tracker.h create mode 100644 test/core/end2end/tests/network_status_change.c diff --git a/src/core/lib/iomgr/network_status_tracker.c b/src/core/lib/iomgr/network_status_tracker.c new file mode 100644 index 00000000000..6e432368cf1 --- /dev/null +++ b/src/core/lib/iomgr/network_status_tracker.c @@ -0,0 +1,82 @@ +/* + * + * Copyright 2015, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include "src/core/lib/iomgr/endpoint.h" +#include + +typedef struct endpoint_ll_node { + grpc_endpoint *ep; + struct endpoint_ll_node *next; +} endpoint_ll_node; + +static endpoint_ll_node *head = NULL; + +// TODO(makarandd): Install callback with OS to monitor network status. +void grpc_initialize_network_status_monitor() { +} + +void grpc_destroy_network_status_monitor() { + for (endpoint_ll_node *curr = head; curr != NULL; ) { + endpoint_ll_node *next = curr->next; + gpr_free(curr); + curr = next; + } +} + +void grpc_network_status_register_endpoint(grpc_endpoint *ep) { + if (head == NULL) { + head = (endpoint_ll_node *)gpr_malloc(sizeof(endpoint_ll_node)); + head->ep = ep; + head->next = NULL; + } else { + endpoint_ll_node *prev_head = head; + head = (endpoint_ll_node *)gpr_malloc(sizeof(endpoint_ll_node)); + head->ep = ep; + head->next = prev_head; + } +} + +// Walk the linked-list from head and execute shutdown. It is possible that +// other threads might be in the process of shutdown as well, but that has +// no side effect. +void grpc_network_status_shutdown_all_endpoints() { + if (head == NULL) { + return; + } + grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; + + for (endpoint_ll_node *curr = head; curr != NULL; curr = curr->next) { + curr->ep->vtable->shutdown(&exec_ctx, curr->ep); + } + grpc_exec_ctx_finish(&exec_ctx); +} diff --git a/src/core/lib/iomgr/network_status_tracker.h b/src/core/lib/iomgr/network_status_tracker.h new file mode 100644 index 00000000000..4154151927f --- /dev/null +++ b/src/core/lib/iomgr/network_status_tracker.h @@ -0,0 +1,40 @@ +/* + * + * Copyright 2016, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#ifndef GRPC_CORE_LIB_IOMGR_NETWORK_STATUS_TRACKER_H +#define GRPC_CORE_LIB_IOMGR_NETWORK_STATUS_TRACKER_H +#include "src/core/lib/iomgr/endpoint.h" + +void grpc_network_status_register_endpoint(grpc_endpoint *ep); +void grpc_network_status_shutdown_all_endpoints(); +#endif /* GRPC_CORE_LIB_IOMGR_NETWORK_STATUS_TRACKER_H */ diff --git a/test/core/end2end/tests/network_status_change.c b/test/core/end2end/tests/network_status_change.c new file mode 100644 index 00000000000..bbc85007022 --- /dev/null +++ b/test/core/end2end/tests/network_status_change.c @@ -0,0 +1,243 @@ +/* + * + * Copyright 2015, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include "test/core/end2end/end2end_tests.h" + +#include +#include + +#include +#include +#include +#include +#include +#include "test/core/end2end/cq_verifier.h" + +/* this is a private API but exposed here for testing*/ +extern void grpc_network_status_shutdown_all_endpoints(); + +enum { TIMEOUT = 200000 }; + +static void *tag(intptr_t t) { return (void *)t; } + +static grpc_end2end_test_fixture begin_test(grpc_end2end_test_config config, + const char *test_name, + grpc_channel_args *client_args, + grpc_channel_args *server_args) { + grpc_end2end_test_fixture f; + gpr_log(GPR_INFO, "%s/%s", test_name, config.name); + f = config.create_fixture(client_args, server_args); + config.init_server(&f, server_args); + config.init_client(&f, client_args); + return f; +} + +static gpr_timespec n_seconds_time(int n) { + return GRPC_TIMEOUT_SECONDS_TO_DEADLINE(n); +} + +static gpr_timespec five_seconds_time(void) { return n_seconds_time(500); } + +static void drain_cq(grpc_completion_queue *cq) { + grpc_event ev; + do { + ev = grpc_completion_queue_next(cq, five_seconds_time(), NULL); + } while (ev.type != GRPC_QUEUE_SHUTDOWN); +} + +static void shutdown_server(grpc_end2end_test_fixture *f) { + if (!f->server) return; + grpc_server_shutdown_and_notify(f->server, f->cq, tag(1000)); + GPR_ASSERT(grpc_completion_queue_pluck( + f->cq, tag(1000), GRPC_TIMEOUT_SECONDS_TO_DEADLINE(5), NULL) + .type == GRPC_OP_COMPLETE); + grpc_server_destroy(f->server); + f->server = NULL; +} + +static void shutdown_client(grpc_end2end_test_fixture *f) { + if (!f->client) return; + grpc_channel_destroy(f->client); + f->client = NULL; +} + +static void end_test(grpc_end2end_test_fixture *f) { + shutdown_server(f); + shutdown_client(f); + + grpc_completion_queue_shutdown(f->cq); + drain_cq(f->cq); + grpc_completion_queue_destroy(f->cq); +} + +/* Client sends a request with payload, server reads then returns status. */ +static void test_invoke_network_status_change(grpc_end2end_test_config config) { + grpc_call *c; + grpc_call *s; + gpr_slice request_payload_slice = gpr_slice_from_copied_string("hello world"); + grpc_byte_buffer *request_payload = + grpc_raw_byte_buffer_create(&request_payload_slice, 1); + gpr_timespec deadline = five_seconds_time(); + grpc_end2end_test_fixture f = + begin_test(config, "test_invoke_request_with_payload", NULL, NULL); + cq_verifier *cqv = cq_verifier_create(f.cq); + grpc_op ops[6]; + grpc_op *op; + grpc_metadata_array initial_metadata_recv; + grpc_metadata_array trailing_metadata_recv; + grpc_metadata_array request_metadata_recv; + grpc_byte_buffer *request_payload_recv = NULL; + grpc_call_details call_details; + grpc_status_code status; + grpc_call_error error; + char *details = NULL; + size_t details_capacity = 0; + int was_cancelled = 2; + + c = grpc_channel_create_call(f.client, NULL, GRPC_PROPAGATE_DEFAULTS, f.cq, + "/foo", "foo.test.google.fr", deadline, NULL); + GPR_ASSERT(c); + + grpc_metadata_array_init(&initial_metadata_recv); + grpc_metadata_array_init(&trailing_metadata_recv); + grpc_metadata_array_init(&request_metadata_recv); + grpc_call_details_init(&call_details); + + memset(ops, 0, sizeof(ops)); + op = ops; + op->op = GRPC_OP_SEND_INITIAL_METADATA; + op->data.send_initial_metadata.count = 0; + op->flags = 0; + op->reserved = NULL; + op++; + op->op = GRPC_OP_SEND_MESSAGE; + op->data.send_message = request_payload; + op->flags = 0; + op->reserved = NULL; + op++; + op->op = GRPC_OP_SEND_CLOSE_FROM_CLIENT; + op->flags = 0; + op->reserved = NULL; + op++; + op->op = GRPC_OP_RECV_INITIAL_METADATA; + op->data.recv_initial_metadata = &initial_metadata_recv; + op->flags = 0; + op->reserved = NULL; + op++; + op->op = GRPC_OP_RECV_STATUS_ON_CLIENT; + op->data.recv_status_on_client.trailing_metadata = &trailing_metadata_recv; + op->data.recv_status_on_client.status = &status; + op->data.recv_status_on_client.status_details = &details; + op->data.recv_status_on_client.status_details_capacity = &details_capacity; + op->flags = 0; + op->reserved = NULL; + op++; + error = grpc_call_start_batch(c, ops, (size_t)(op - ops), tag(1), NULL); + GPR_ASSERT(GRPC_CALL_OK == error); + + GPR_ASSERT(GRPC_CALL_OK == grpc_server_request_call( + f.server, &s, &call_details, + &request_metadata_recv, f.cq, f.cq, tag(101))); + cq_expect_completion(cqv, tag(101), 1); + cq_verify(cqv); + + op = ops; + op->op = GRPC_OP_SEND_INITIAL_METADATA; + op->data.send_initial_metadata.count = 0; + op->flags = 0; + op->reserved = NULL; + op++; + op->op = GRPC_OP_RECV_MESSAGE; + op->data.recv_message = &request_payload_recv; + op->flags = 0; + op->reserved = NULL; + op++; + error = grpc_call_start_batch(s, ops, (size_t)(op - ops), tag(102), NULL); + GPR_ASSERT(GRPC_CALL_OK == error); + + cq_expect_completion(cqv, tag(102), 1); + // Simulate the network loss event + grpc_network_status_shutdown_all_endpoints(); + cq_verify(cqv); + + op = ops; + op->op = GRPC_OP_RECV_CLOSE_ON_SERVER; + op->data.recv_close_on_server.cancelled = &was_cancelled; + op->flags = 0; + op->reserved = NULL; + op++; + op->op = GRPC_OP_SEND_STATUS_FROM_SERVER; + op->data.send_status_from_server.trailing_metadata_count = 0; + op->data.send_status_from_server.status = GRPC_STATUS_OK; + op->data.send_status_from_server.status_details = "xyz"; + op->flags = 0; + op->reserved = NULL; + op++; + error = grpc_call_start_batch(s, ops, (size_t)(op - ops), tag(103), NULL); + GPR_ASSERT(GRPC_CALL_OK == error); + void shutdown_all_endpoints(); + cq_expect_completion(cqv, tag(103), 1); + cq_expect_completion(cqv, tag(1), 1); + cq_verify(cqv); + + // Expected behavior of a RPC when network is lost. + GPR_ASSERT(status == GRPC_STATUS_UNAVAILABLE); + GPR_ASSERT(0 == strcmp(details, "")); + GPR_ASSERT(0 == strcmp(call_details.method, "/foo")); + GPR_ASSERT(0 == strcmp(call_details.host, "foo.test.google.fr")); + GPR_ASSERT(was_cancelled == 0); + + + gpr_free(details); + grpc_metadata_array_destroy(&initial_metadata_recv); + grpc_metadata_array_destroy(&trailing_metadata_recv); + grpc_metadata_array_destroy(&request_metadata_recv); + grpc_call_details_destroy(&call_details); + + grpc_call_destroy(c); + grpc_call_destroy(s); + + cq_verifier_destroy(cqv); + + grpc_byte_buffer_destroy(request_payload); + grpc_byte_buffer_destroy(request_payload_recv); + + end_test(&f); + config.tear_down_data(&f); +} + +void network_status_change(grpc_end2end_test_config config) { + test_invoke_network_status_change(config); +} + +void network_status_change_pre_init(void) {} From 0579cfc334794a32e243e971f77e01b5fd8f3c55 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Mon, 20 Jun 2016 15:45:24 -0700 Subject: [PATCH 109/280] more files after running build.yaml changes through --- BUILD | 8 + Makefile | 5 + binding.gyp | 1 + build.yaml | 2 + config.m4 | 1 + gRPC.podspec | 3 + grpc.gemspec | 2 + package.xml | 2 + src/core/lib/iomgr/tcp_posix.c | 3 + src/core/lib/iomgr/tcp_windows.c | 4 + src/python/grpcio/grpc_core_dependencies.py | 1 + test/core/end2end/end2end_nosec_tests.c | 8 + test/core/end2end/end2end_tests.c | 8 + test/core/end2end/gen_build_yaml.py | 1 + tools/doxygen/Doxyfile.core.internal | 2 + tools/run_tests/sources_and_headers.json | 5 + tools/run_tests/tests.json | 627 +++++++++++++++++- vsprojects/vcxproj/grpc/grpc.vcxproj | 3 + vsprojects/vcxproj/grpc/grpc.vcxproj.filters | 6 + .../grpc_unsecure/grpc_unsecure.vcxproj | 3 + .../grpc_unsecure.vcxproj.filters | 6 + .../end2end_nosec_tests.vcxproj | 2 + .../end2end_nosec_tests.vcxproj.filters | 3 + .../tests/end2end_tests/end2end_tests.vcxproj | 2 + .../end2end_tests.vcxproj.filters | 3 + 25 files changed, 702 insertions(+), 9 deletions(-) diff --git a/BUILD b/BUILD index ac0eb8e403d..55c2ea87c65 100644 --- a/BUILD +++ b/BUILD @@ -187,6 +187,7 @@ cc_library( "src/core/lib/iomgr/iomgr_internal.h", "src/core/lib/iomgr/iomgr_posix.h", "src/core/lib/iomgr/load_file.h", + "src/core/lib/iomgr/network_status_tracker.h", "src/core/lib/iomgr/polling_entity.h", "src/core/lib/iomgr/pollset.h", "src/core/lib/iomgr/pollset_set.h", @@ -338,6 +339,7 @@ cc_library( "src/core/lib/iomgr/iomgr_posix.c", "src/core/lib/iomgr/iomgr_windows.c", "src/core/lib/iomgr/load_file.c", + "src/core/lib/iomgr/network_status_tracker.c", "src/core/lib/iomgr/polling_entity.c", "src/core/lib/iomgr/pollset_set_windows.c", "src/core/lib/iomgr/pollset_windows.c", @@ -571,6 +573,7 @@ cc_library( "src/core/lib/iomgr/iomgr_internal.h", "src/core/lib/iomgr/iomgr_posix.h", "src/core/lib/iomgr/load_file.h", + "src/core/lib/iomgr/network_status_tracker.h", "src/core/lib/iomgr/polling_entity.h", "src/core/lib/iomgr/pollset.h", "src/core/lib/iomgr/pollset_set.h", @@ -712,6 +715,7 @@ cc_library( "src/core/lib/iomgr/iomgr_posix.c", "src/core/lib/iomgr/iomgr_windows.c", "src/core/lib/iomgr/load_file.c", + "src/core/lib/iomgr/network_status_tracker.c", "src/core/lib/iomgr/polling_entity.c", "src/core/lib/iomgr/pollset_set_windows.c", "src/core/lib/iomgr/pollset_windows.c", @@ -920,6 +924,7 @@ cc_library( "src/core/lib/iomgr/iomgr_internal.h", "src/core/lib/iomgr/iomgr_posix.h", "src/core/lib/iomgr/load_file.h", + "src/core/lib/iomgr/network_status_tracker.h", "src/core/lib/iomgr/polling_entity.h", "src/core/lib/iomgr/pollset.h", "src/core/lib/iomgr/pollset_set.h", @@ -1048,6 +1053,7 @@ cc_library( "src/core/lib/iomgr/iomgr_posix.c", "src/core/lib/iomgr/iomgr_windows.c", "src/core/lib/iomgr/load_file.c", + "src/core/lib/iomgr/network_status_tracker.c", "src/core/lib/iomgr/polling_entity.c", "src/core/lib/iomgr/pollset_set_windows.c", "src/core/lib/iomgr/pollset_windows.c", @@ -1805,6 +1811,7 @@ objc_library( "src/core/lib/iomgr/iomgr_posix.c", "src/core/lib/iomgr/iomgr_windows.c", "src/core/lib/iomgr/load_file.c", + "src/core/lib/iomgr/network_status_tracker.c", "src/core/lib/iomgr/polling_entity.c", "src/core/lib/iomgr/pollset_set_windows.c", "src/core/lib/iomgr/pollset_windows.c", @@ -2017,6 +2024,7 @@ objc_library( "src/core/lib/iomgr/iomgr_internal.h", "src/core/lib/iomgr/iomgr_posix.h", "src/core/lib/iomgr/load_file.h", + "src/core/lib/iomgr/network_status_tracker.h", "src/core/lib/iomgr/polling_entity.h", "src/core/lib/iomgr/pollset.h", "src/core/lib/iomgr/pollset_set.h", diff --git a/Makefile b/Makefile index d93ce337632..961485b3b80 100644 --- a/Makefile +++ b/Makefile @@ -2574,6 +2574,7 @@ LIBGRPC_SRC = \ src/core/lib/iomgr/iomgr_posix.c \ src/core/lib/iomgr/iomgr_windows.c \ src/core/lib/iomgr/load_file.c \ + src/core/lib/iomgr/network_status_tracker.c \ src/core/lib/iomgr/polling_entity.c \ src/core/lib/iomgr/pollset_set_windows.c \ src/core/lib/iomgr/pollset_windows.c \ @@ -2844,6 +2845,7 @@ LIBGRPC_CRONET_SRC = \ src/core/lib/iomgr/iomgr_posix.c \ src/core/lib/iomgr/iomgr_windows.c \ src/core/lib/iomgr/load_file.c \ + src/core/lib/iomgr/network_status_tracker.c \ src/core/lib/iomgr/polling_entity.c \ src/core/lib/iomgr/pollset_set_windows.c \ src/core/lib/iomgr/pollset_windows.c \ @@ -3183,6 +3185,7 @@ LIBGRPC_UNSECURE_SRC = \ src/core/lib/iomgr/iomgr_posix.c \ src/core/lib/iomgr/iomgr_windows.c \ src/core/lib/iomgr/load_file.c \ + src/core/lib/iomgr/network_status_tracker.c \ src/core/lib/iomgr/polling_entity.c \ src/core/lib/iomgr/pollset_set_windows.c \ src/core/lib/iomgr/pollset_windows.c \ @@ -6376,6 +6379,7 @@ LIBEND2END_TESTS_SRC = \ test/core/end2end/tests/max_concurrent_streams.c \ test/core/end2end/tests/max_message_length.c \ test/core/end2end/tests/negative_deadline.c \ + test/core/end2end/tests/network_status_change.c \ test/core/end2end/tests/no_op.c \ test/core/end2end/tests/payload.c \ test/core/end2end/tests/ping.c \ @@ -6452,6 +6456,7 @@ LIBEND2END_NOSEC_TESTS_SRC = \ test/core/end2end/tests/max_concurrent_streams.c \ test/core/end2end/tests/max_message_length.c \ test/core/end2end/tests/negative_deadline.c \ + test/core/end2end/tests/network_status_change.c \ test/core/end2end/tests/no_op.c \ test/core/end2end/tests/payload.c \ test/core/end2end/tests/ping.c \ diff --git a/binding.gyp b/binding.gyp index 3150d1083b0..9a4b46141ff 100644 --- a/binding.gyp +++ b/binding.gyp @@ -591,6 +591,7 @@ 'src/core/lib/iomgr/iomgr_posix.c', 'src/core/lib/iomgr/iomgr_windows.c', 'src/core/lib/iomgr/load_file.c', + 'src/core/lib/iomgr/network_status_tracker.c', 'src/core/lib/iomgr/polling_entity.c', 'src/core/lib/iomgr/pollset_set_windows.c', 'src/core/lib/iomgr/pollset_windows.c', diff --git a/build.yaml b/build.yaml index d41f4195947..54dae5b6ac2 100644 --- a/build.yaml +++ b/build.yaml @@ -183,6 +183,7 @@ filegroups: - src/core/lib/iomgr/iomgr_internal.h - src/core/lib/iomgr/iomgr_posix.h - src/core/lib/iomgr/load_file.h + - src/core/lib/iomgr/network_status_tracker.h - src/core/lib/iomgr/polling_entity.h - src/core/lib/iomgr/pollset.h - src/core/lib/iomgr/pollset_set.h @@ -261,6 +262,7 @@ filegroups: - src/core/lib/iomgr/iomgr_posix.c - src/core/lib/iomgr/iomgr_windows.c - src/core/lib/iomgr/load_file.c + - src/core/lib/iomgr/network_status_tracker.c - src/core/lib/iomgr/polling_entity.c - src/core/lib/iomgr/pollset_set_windows.c - src/core/lib/iomgr/pollset_windows.c diff --git a/config.m4 b/config.m4 index 716eb9841fb..6cf8b3a33fd 100644 --- a/config.m4 +++ b/config.m4 @@ -110,6 +110,7 @@ if test "$PHP_GRPC" != "no"; then src/core/lib/iomgr/iomgr_posix.c \ src/core/lib/iomgr/iomgr_windows.c \ src/core/lib/iomgr/load_file.c \ + src/core/lib/iomgr/network_status_tracker.c \ src/core/lib/iomgr/polling_entity.c \ src/core/lib/iomgr/pollset_set_windows.c \ src/core/lib/iomgr/pollset_windows.c \ diff --git a/gRPC.podspec b/gRPC.podspec index ff8373637a6..48da2322e73 100644 --- a/gRPC.podspec +++ b/gRPC.podspec @@ -190,6 +190,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/iomgr_internal.h', 'src/core/lib/iomgr/iomgr_posix.h', 'src/core/lib/iomgr/load_file.h', + 'src/core/lib/iomgr/network_status_tracker.h', 'src/core/lib/iomgr/polling_entity.h', 'src/core/lib/iomgr/pollset.h', 'src/core/lib/iomgr/pollset_set.h', @@ -375,6 +376,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/iomgr_posix.c', 'src/core/lib/iomgr/iomgr_windows.c', 'src/core/lib/iomgr/load_file.c', + 'src/core/lib/iomgr/network_status_tracker.c', 'src/core/lib/iomgr/polling_entity.c', 'src/core/lib/iomgr/pollset_set_windows.c', 'src/core/lib/iomgr/pollset_windows.c', @@ -570,6 +572,7 @@ Pod::Spec.new do |s| 'src/core/lib/iomgr/iomgr_internal.h', 'src/core/lib/iomgr/iomgr_posix.h', 'src/core/lib/iomgr/load_file.h', + 'src/core/lib/iomgr/network_status_tracker.h', 'src/core/lib/iomgr/polling_entity.h', 'src/core/lib/iomgr/pollset.h', 'src/core/lib/iomgr/pollset_set.h', diff --git a/grpc.gemspec b/grpc.gemspec index 184a5485917..5796cd68bc2 100755 --- a/grpc.gemspec +++ b/grpc.gemspec @@ -199,6 +199,7 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/iomgr_internal.h ) s.files += %w( src/core/lib/iomgr/iomgr_posix.h ) s.files += %w( src/core/lib/iomgr/load_file.h ) + s.files += %w( src/core/lib/iomgr/network_status_tracker.h ) s.files += %w( src/core/lib/iomgr/polling_entity.h ) s.files += %w( src/core/lib/iomgr/pollset.h ) s.files += %w( src/core/lib/iomgr/pollset_set.h ) @@ -354,6 +355,7 @@ Gem::Specification.new do |s| s.files += %w( src/core/lib/iomgr/iomgr_posix.c ) s.files += %w( src/core/lib/iomgr/iomgr_windows.c ) s.files += %w( src/core/lib/iomgr/load_file.c ) + s.files += %w( src/core/lib/iomgr/network_status_tracker.c ) s.files += %w( src/core/lib/iomgr/polling_entity.c ) s.files += %w( src/core/lib/iomgr/pollset_set_windows.c ) s.files += %w( src/core/lib/iomgr/pollset_windows.c ) diff --git a/package.xml b/package.xml index 67e9bb2c282..623174ad016 100644 --- a/package.xml +++ b/package.xml @@ -206,6 +206,7 @@ + @@ -361,6 +362,7 @@ + diff --git a/src/core/lib/iomgr/tcp_posix.c b/src/core/lib/iomgr/tcp_posix.c index 017f52c3677..f7818353b0a 100644 --- a/src/core/lib/iomgr/tcp_posix.c +++ b/src/core/lib/iomgr/tcp_posix.c @@ -35,6 +35,7 @@ #ifdef GPR_POSIX_SOCKET +#include "src/core/lib/iomgr/network_status_tracker.h" #include "src/core/lib/iomgr/tcp_posix.h" #include @@ -474,6 +475,8 @@ grpc_endpoint *grpc_tcp_create(grpc_fd *em_fd, size_t slice_size, tcp->write_closure.cb = tcp_handle_write; tcp->write_closure.cb_arg = tcp; gpr_slice_buffer_init(&tcp->last_read_buffer); + /* Tell network status tracker about new endpoint */ + grpc_network_status_register_endpoint(&tcp->base); return &tcp->base; } diff --git a/src/core/lib/iomgr/tcp_windows.c b/src/core/lib/iomgr/tcp_windows.c index b2af8030aaa..8140d1d8cd2 100644 --- a/src/core/lib/iomgr/tcp_windows.c +++ b/src/core/lib/iomgr/tcp_windows.c @@ -37,6 +37,7 @@ #include +#include "src/core/lib/iomgr/network_status_tracker.h" #include "src/core/lib/iomgr/sockaddr_windows.h" #include @@ -401,6 +402,9 @@ grpc_endpoint *grpc_tcp_create(grpc_winsocket *socket, char *peer_string) { grpc_closure_init(&tcp->on_read, on_read, tcp); grpc_closure_init(&tcp->on_write, on_write, tcp); tcp->peer_string = gpr_strdup(peer_string); + /* Tell network status tracking code about the new endpoint */ + grpc_network_status_register_endpoint(&tcp->base); + return &tcp->base; } diff --git a/src/python/grpcio/grpc_core_dependencies.py b/src/python/grpcio/grpc_core_dependencies.py index 839c555f055..575dfa9dbde 100644 --- a/src/python/grpcio/grpc_core_dependencies.py +++ b/src/python/grpcio/grpc_core_dependencies.py @@ -104,6 +104,7 @@ CORE_SOURCE_FILES = [ 'src/core/lib/iomgr/iomgr_posix.c', 'src/core/lib/iomgr/iomgr_windows.c', 'src/core/lib/iomgr/load_file.c', + 'src/core/lib/iomgr/network_status_tracker.c', 'src/core/lib/iomgr/polling_entity.c', 'src/core/lib/iomgr/pollset_set_windows.c', 'src/core/lib/iomgr/pollset_windows.c', diff --git a/test/core/end2end/end2end_nosec_tests.c b/test/core/end2end/end2end_nosec_tests.c index b71299c09e0..09ed6e6d688 100644 --- a/test/core/end2end/end2end_nosec_tests.c +++ b/test/core/end2end/end2end_nosec_tests.c @@ -89,6 +89,8 @@ extern void max_message_length(grpc_end2end_test_config config); extern void max_message_length_pre_init(void); extern void negative_deadline(grpc_end2end_test_config config); extern void negative_deadline_pre_init(void); +extern void network_status_change(grpc_end2end_test_config config); +extern void network_status_change_pre_init(void); extern void no_op(grpc_end2end_test_config config); extern void no_op_pre_init(void); extern void payload(grpc_end2end_test_config config); @@ -144,6 +146,7 @@ void grpc_end2end_tests_pre_init(void) { max_concurrent_streams_pre_init(); max_message_length_pre_init(); negative_deadline_pre_init(); + network_status_change_pre_init(); no_op_pre_init(); payload_pre_init(); ping_pre_init(); @@ -190,6 +193,7 @@ void grpc_end2end_tests(int argc, char **argv, max_concurrent_streams(config); max_message_length(config); negative_deadline(config); + network_status_change(config); no_op(config); payload(config); ping(config); @@ -300,6 +304,10 @@ void grpc_end2end_tests(int argc, char **argv, negative_deadline(config); continue; } + if (0 == strcmp("network_status_change", argv[i])) { + network_status_change(config); + continue; + } if (0 == strcmp("no_op", argv[i])) { no_op(config); continue; diff --git a/test/core/end2end/end2end_tests.c b/test/core/end2end/end2end_tests.c index 00c9c44a78c..312700646cd 100644 --- a/test/core/end2end/end2end_tests.c +++ b/test/core/end2end/end2end_tests.c @@ -91,6 +91,8 @@ extern void max_message_length(grpc_end2end_test_config config); extern void max_message_length_pre_init(void); extern void negative_deadline(grpc_end2end_test_config config); extern void negative_deadline_pre_init(void); +extern void network_status_change(grpc_end2end_test_config config); +extern void network_status_change_pre_init(void); extern void no_op(grpc_end2end_test_config config); extern void no_op_pre_init(void); extern void payload(grpc_end2end_test_config config); @@ -147,6 +149,7 @@ void grpc_end2end_tests_pre_init(void) { max_concurrent_streams_pre_init(); max_message_length_pre_init(); negative_deadline_pre_init(); + network_status_change_pre_init(); no_op_pre_init(); payload_pre_init(); ping_pre_init(); @@ -194,6 +197,7 @@ void grpc_end2end_tests(int argc, char **argv, max_concurrent_streams(config); max_message_length(config); negative_deadline(config); + network_status_change(config); no_op(config); payload(config); ping(config); @@ -308,6 +312,10 @@ void grpc_end2end_tests(int argc, char **argv, negative_deadline(config); continue; } + if (0 == strcmp("network_status_change", argv[i])) { + network_status_change(config); + continue; + } if (0 == strcmp("no_op", argv[i])) { no_op(config); continue; diff --git a/test/core/end2end/gen_build_yaml.py b/test/core/end2end/gen_build_yaml.py index 325d9b3cad0..716dca20bb1 100755 --- a/test/core/end2end/gen_build_yaml.py +++ b/test/core/end2end/gen_build_yaml.py @@ -112,6 +112,7 @@ END2END_TESTS = { 'max_concurrent_streams': default_test_options._replace(proxyable=False), 'max_message_length': default_test_options, 'negative_deadline': default_test_options, + 'network_status_change': default_test_options, 'no_op': default_test_options, 'payload': default_test_options, 'ping_pong_streaming': default_test_options, diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal index e1555930e91..545536c97c4 100644 --- a/tools/doxygen/Doxyfile.core.internal +++ b/tools/doxygen/Doxyfile.core.internal @@ -818,6 +818,7 @@ src/core/lib/iomgr/iomgr.h \ src/core/lib/iomgr/iomgr_internal.h \ src/core/lib/iomgr/iomgr_posix.h \ src/core/lib/iomgr/load_file.h \ +src/core/lib/iomgr/network_status_tracker.h \ src/core/lib/iomgr/polling_entity.h \ src/core/lib/iomgr/pollset.h \ src/core/lib/iomgr/pollset_set.h \ @@ -973,6 +974,7 @@ src/core/lib/iomgr/iomgr.c \ src/core/lib/iomgr/iomgr_posix.c \ src/core/lib/iomgr/iomgr_windows.c \ src/core/lib/iomgr/load_file.c \ +src/core/lib/iomgr/network_status_tracker.c \ src/core/lib/iomgr/polling_entity.c \ src/core/lib/iomgr/pollset_set_windows.c \ src/core/lib/iomgr/pollset_windows.c \ diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index 57e3bd63f42..7f8cc8ad436 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -5387,6 +5387,7 @@ "test/core/end2end/tests/max_concurrent_streams.c", "test/core/end2end/tests/max_message_length.c", "test/core/end2end/tests/negative_deadline.c", + "test/core/end2end/tests/network_status_change.c", "test/core/end2end/tests/no_op.c", "test/core/end2end/tests/payload.c", "test/core/end2end/tests/ping.c", @@ -5445,6 +5446,7 @@ "test/core/end2end/tests/max_concurrent_streams.c", "test/core/end2end/tests/max_message_length.c", "test/core/end2end/tests/negative_deadline.c", + "test/core/end2end/tests/network_status_change.c", "test/core/end2end/tests/no_op.c", "test/core/end2end/tests/payload.c", "test/core/end2end/tests/ping.c", @@ -5732,6 +5734,7 @@ "src/core/lib/iomgr/iomgr_internal.h", "src/core/lib/iomgr/iomgr_posix.h", "src/core/lib/iomgr/load_file.h", + "src/core/lib/iomgr/network_status_tracker.h", "src/core/lib/iomgr/polling_entity.h", "src/core/lib/iomgr/pollset.h", "src/core/lib/iomgr/pollset_set.h", @@ -5847,6 +5850,8 @@ "src/core/lib/iomgr/iomgr_windows.c", "src/core/lib/iomgr/load_file.c", "src/core/lib/iomgr/load_file.h", + "src/core/lib/iomgr/network_status_tracker.c", + "src/core/lib/iomgr/network_status_tracker.h", "src/core/lib/iomgr/polling_entity.c", "src/core/lib/iomgr/polling_entity.h", "src/core/lib/iomgr/pollset.h", diff --git a/tools/run_tests/tests.json b/tools/run_tests/tests.json index 3ed7a6bc476..bf5ff5be191 100644 --- a/tools/run_tests/tests.json +++ b/tools/run_tests/tests.json @@ -4900,6 +4900,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_census_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -5736,6 +5758,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_compress_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -6548,6 +6592,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_fakesec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -7262,6 +7327,26 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_fd_test", + "platforms": [ + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -8030,6 +8115,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -8722,6 +8829,22 @@ "linux" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full+pipe_test", + "platforms": [ + "linux" + ] + }, { "args": [ "no_op" @@ -9452,6 +9575,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full+trace_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -10288,6 +10433,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_loadreporting_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -11100,6 +11267,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_oauth2_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -11814,6 +12002,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_proxy_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -12507,6 +12716,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_sockpair_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -13179,6 +13409,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_sockpair+trace_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -13874,7 +14125,7 @@ }, { "args": [ - "no_op" + "network_status_change" ], "ci_platforms": [ "windows", @@ -13895,7 +14146,7 @@ }, { "args": [ - "payload" + "no_op" ], "ci_platforms": [ "windows", @@ -13916,7 +14167,7 @@ }, { "args": [ - "ping_pong_streaming" + "payload" ], "ci_platforms": [ "windows", @@ -13937,7 +14188,7 @@ }, { "args": [ - "registered_call" + "ping_pong_streaming" ], "ci_platforms": [ "windows", @@ -13958,14 +14209,14 @@ }, { "args": [ - "request_with_flags" + "registered_call" ], "ci_platforms": [ "windows", "linux", "posix" ], - "cpu_cost": 0.1, + "cpu_cost": 1.0, "exclude_configs": [], "flaky": false, "language": "c", @@ -13979,14 +14230,14 @@ }, { "args": [ - "request_with_payload" + "request_with_flags" ], "ci_platforms": [ "windows", "linux", "posix" ], - "cpu_cost": 1.0, + "cpu_cost": 0.1, "exclude_configs": [], "flaky": false, "language": "c", @@ -14000,7 +14251,28 @@ }, { "args": [ - "server_finishes_request" + "request_with_payload" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_sockpair_1byte_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, + { + "args": [ + "server_finishes_request" ], "ci_platforms": [ "windows", @@ -14652,6 +14924,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_ssl_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -15488,6 +15782,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_ssl_cert_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -16216,6 +16532,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_ssl_proxy_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -16928,6 +17265,26 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_uds_test", + "platforms": [ + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -17714,6 +18071,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_census_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -18528,6 +18907,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_compress_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -19236,6 +19637,26 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_fd_nosec_test", + "platforms": [ + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -19982,6 +20403,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -20658,6 +21101,22 @@ "linux" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full+pipe_nosec_test", + "platforms": [ + "linux" + ] + }, { "args": [ "no_op" @@ -21366,6 +21825,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_full+trace_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -22180,6 +22661,28 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_loadreporting_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -22887,6 +23390,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_proxy_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -23559,6 +24083,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_sockpair_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -24210,6 +24755,27 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_sockpair+trace_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -24922,6 +25488,29 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "windows", + "linux", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [ + "msan" + ], + "flaky": false, + "language": "c", + "name": "h2_sockpair_1byte_nosec_test", + "platforms": [ + "windows", + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" @@ -25638,6 +26227,26 @@ "posix" ] }, + { + "args": [ + "network_status_change" + ], + "ci_platforms": [ + "linux", + "mac", + "posix" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "language": "c", + "name": "h2_uds_nosec_test", + "platforms": [ + "linux", + "mac", + "posix" + ] + }, { "args": [ "no_op" diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj b/vsprojects/vcxproj/grpc/grpc.vcxproj index 84e03f70561..be4a28d5ecf 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj @@ -327,6 +327,7 @@ + @@ -513,6 +514,8 @@ + + diff --git a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters index 0f817e0562d..086b62d572c 100644 --- a/vsprojects/vcxproj/grpc/grpc.vcxproj.filters +++ b/vsprojects/vcxproj/grpc/grpc.vcxproj.filters @@ -88,6 +88,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr @@ -731,6 +734,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj index ac9bb186b71..d88fc0eef0a 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj @@ -316,6 +316,7 @@ + @@ -480,6 +481,8 @@ + + diff --git a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters index 1341474142c..e1bc1801f2f 100644 --- a/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters +++ b/vsprojects/vcxproj/grpc_unsecure/grpc_unsecure.vcxproj.filters @@ -91,6 +91,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr @@ -638,6 +641,9 @@ src\core\lib\iomgr + + src\core\lib\iomgr + src\core\lib\iomgr diff --git a/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj b/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj index 22cd102d116..5297d79e263 100644 --- a/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj +++ b/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj @@ -199,6 +199,8 @@ + + diff --git a/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj.filters b/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj.filters index 1bb208bba8e..897a42386f4 100644 --- a/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj.filters +++ b/vsprojects/vcxproj/test/end2end/tests/end2end_nosec_tests/end2end_nosec_tests.vcxproj.filters @@ -73,6 +73,9 @@ test\core\end2end\tests + + test\core\end2end\tests + test\core\end2end\tests diff --git a/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj b/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj index bfd437e8717..3bc61fc581d 100644 --- a/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj +++ b/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj @@ -201,6 +201,8 @@ + + diff --git a/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj.filters b/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj.filters index 61c065f77ca..b896ff8bf1f 100644 --- a/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj.filters +++ b/vsprojects/vcxproj/test/end2end/tests/end2end_tests/end2end_tests.vcxproj.filters @@ -76,6 +76,9 @@ test\core\end2end\tests + + test\core\end2end\tests + test\core\end2end\tests From 2f8ade0b9df48990e3617a302a5da946f032d4f6 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 17 Jun 2016 13:28:38 -0700 Subject: [PATCH 110/280] Significantly refactor the polling island locking and refcounting code --- src/core/lib/iomgr/ev_epoll_linux.c | 462 ++++++++++++++++------------ 1 file changed, 270 insertions(+), 192 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index ed2c494b783..72288889c02 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -140,18 +140,40 @@ static void fd_global_shutdown(void); #define CLOSURE_READY ((grpc_closure *)1) /******************************************************************************* - * Polling-island Declarations + * Polling island Declarations */ -/* TODO: sree: Consider making ref_cnt and merged_to to gpr_atm - This would - * significantly reduce the number of mutex acquisition calls. */ + +// #define GRPC_PI_REF_COUNT_DEBUG +#ifdef GRPC_PI_REF_COUNT_DEBUG + +#define PI_ADD_REF(p, r) pi_add_ref_dbg((p), 1, (r), __FILE__, __LINE__) +#define PI_UNREF(p, r) pi_unref_dbg((p), 1, (r), __FILE__, __LINE__) + +#else /* defined(GRPC_PI_REF_COUNT_DEBUG) */ + +#define PI_ADD_REF(p, r) pi_add_ref((p), 1) +#define PI_UNREF(p, r) pi_unref((p), 1) + +#endif /* !defined(GPRC_PI_REF_COUNT_DEBUG) */ + typedef struct polling_island { gpr_mu mu; - int ref_cnt; - - /* Points to the polling_island this merged into. - * If merged_to is not NULL, all the remaining fields (except mu and ref_cnt) - * are invalid and must be ignored */ - struct polling_island *merged_to; + /* Ref count. Use PI_ADD_REF() and PI_UNREF() macros to increment/decrement + the refcount. + Once the ref count becomes zero, this structure is destroyed which means + we should ensure that there is never a scenario where a PI_ADD_REF() is + racing with a PI_UNREF() that just made the ref_count zero. */ + gpr_atm ref_count; + + /* Pointer to the polling_island this merged into. + * merged_to value is only set once in polling_island's lifetime (and that too + * only if the island is merged with another island). Because of this, we can + * use gpr_atm type here so that we can do atomic access on this and reduce + * lock contention on 'mu' mutex. + * + * Note that if this field is not NULL (i.e not 0), all the remaining fields + * (except mu and ref_count) are invalid and must be ignored. */ + gpr_atm merged_to; /* The fd of the underlying epoll set */ int epoll_fd; @@ -236,6 +258,8 @@ static grpc_wakeup_fd polling_island_wakeup_fd; static gpr_mu g_pi_freelist_mu; static polling_island *g_pi_freelist = NULL; +static void polling_island_delete(); /* Forward declaration */ + #ifdef GRPC_TSAN /* Currently TSAN may incorrectly flag data races between epoll_ctl and epoll_wait for any grpc_fd structs that are added to the epoll set via @@ -247,6 +271,51 @@ static polling_island *g_pi_freelist = NULL; gpr_atm g_epoll_sync; #endif /* defined(GRPC_TSAN) */ +#ifdef GRPC_PI_REF_COUNT_DEBUG +long pi_add_ref(polling_island *pi, int ref_cnt); +long pi_unref(polling_island *pi, int ref_cnt); + +void pi_add_ref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, + int line) { + long old_cnt = pi_add_ref(pi, ref_cnt); + gpr_log(GPR_DEBUG, "Add ref pi: %p, old:%ld -> new:%ld (%s) - (%s, %d)", + (void *)pi, old_cnt, (old_cnt + ref_cnt), reason, file, line); +} + +void pi_unref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, + int line) { + long old_cnt = pi_unref(pi, ref_cnt); + gpr_log(GPR_DEBUG, "Unref pi: %p, old:%ld -> new:%ld (%s) - (%s, %d)", + (void *)pi, old_cnt, (old_cnt - ref_cnt), reason, file, line); +} +#endif + +long pi_add_ref(polling_island *pi, int ref_cnt) { + return gpr_atm_no_barrier_fetch_add(&pi->ref_count, ref_cnt); +} + +long pi_unref(polling_island *pi, int ref_cnt) { + long old_cnt = gpr_atm_no_barrier_fetch_add(&pi->ref_count, -ref_cnt); + + /* If ref count went to zero, delete the polling island. Note that this need + not be done under a lock. Once the ref count goes to zero, we are + guaranteed that no one else holds a reference to the polling island (and + that there is no racing pi_add_ref() call either. + + Also, if we are deleting the polling island and the merged_to field is + non-empty, we should remove a ref to the merged_to polling island + */ + if (old_cnt == ref_cnt) { + polling_island *next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + polling_island_delete(pi); + if (next != NULL) { + PI_UNREF(next, "pi_delete"); /* Recursive call */ + } + } + + return old_cnt; +} + /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, size_t fd_count, bool add_fd_refs) { @@ -355,8 +424,7 @@ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, } } -static polling_island *polling_island_create(grpc_fd *initial_fd, - int initial_ref_cnt) { +static polling_island *polling_island_create(grpc_fd *initial_fd) { polling_island *pi = NULL; /* Try to get one from the polling island freelist */ @@ -377,6 +445,9 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, pi->fds = NULL; } + gpr_atm_no_barrier_store(&pi->ref_count, 0); + gpr_atm_no_barrier_store(&pi->merged_to, NULL); + pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); if (pi->epoll_fd < 0) { @@ -387,14 +458,12 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, polling_island_add_wakeup_fd_locked(pi, &grpc_global_wakeup_fd); - pi->ref_cnt = initial_ref_cnt; - pi->merged_to = NULL; pi->next_free = NULL; if (initial_fd != NULL) { - /* It is not really needed to get the pi->mu lock here. If this is a newly - created polling island (or one that we got from the freelist), no one - else would be holding a lock to it anyway */ + /* Lock the polling island here just in case we got this structure from the + freelist and the polling island lock was not released yet (by the code + that adds the polling island to the freelist) */ gpr_mu_lock(&pi->mu); polling_island_add_fds_locked(pi, &initial_fd, 1, true); gpr_mu_unlock(&pi->mu); @@ -404,140 +473,136 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, } static void polling_island_delete(polling_island *pi) { - GPR_ASSERT(pi->ref_cnt == 0); GPR_ASSERT(pi->fd_cnt == 0); + gpr_atm_rel_store(&pi->merged_to, NULL); + close(pi->epoll_fd); pi->epoll_fd = -1; - pi->merged_to = NULL; - gpr_mu_lock(&g_pi_freelist_mu); pi->next_free = g_pi_freelist; g_pi_freelist = pi; gpr_mu_unlock(&g_pi_freelist_mu); } -void polling_island_unref_and_unlock(polling_island *pi, int unref_by) { - pi->ref_cnt -= unref_by; - int ref_cnt = pi->ref_cnt; - GPR_ASSERT(ref_cnt >= 0); - - gpr_mu_unlock(&pi->mu); - - if (ref_cnt == 0) { - polling_island_delete(pi); - } -} - -polling_island *polling_island_update_and_lock(polling_island *pi, int unref_by, - int add_ref_by) { +/* Gets the lock on the *latest* polling island i.e the last polling island in + the linked list (linked by 'merged_to' link). Call gpr_mu_unlock on the + returned polling island's mu. + Usage: To lock/unlock polling island "pi", do the following: + polling_island *pi_latest = polling_island_lock(pi); + ... + ... critical section .. + ... + gpr_mu_unlock(&pi_latest->mu); //NOTE: use pi_latest->mu. NOT pi->mu */ +polling_island *polling_island_lock(polling_island *pi) { polling_island *next = NULL; - gpr_mu_lock(&pi->mu); - while (pi->merged_to != NULL) { - next = pi->merged_to; - polling_island_unref_and_unlock(pi, unref_by); + while (true) { + next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + if (next == NULL) { + /* pi is the last node in the linked list. Get the lock and check again + (under the pi->mu lock) that pi is still the last node (because a merge + may have happend after the (next == NULL) check above and before + getting the pi->mu lock. + If pi is the last node, we are done. If not, unlock and continue + traversing the list */ + gpr_mu_lock(&pi->mu); + next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + if (next == NULL) { + break; + } + gpr_mu_unlock(&pi->mu); + } + pi = next; - gpr_mu_lock(&pi->mu); } - pi->ref_cnt += add_ref_by; return pi; } -void polling_island_pair_update_and_lock(polling_island **p, - polling_island **q) { +/* Gets the lock on the *latest* polling islands pointed by *p and *q. + This function is needed because calling the following block of code to obtain + locks on polling islands (*p and *q) is prone to deadlocks. + { + polling_island_lock(*p); + polling_island_lock(*q); + } + + Usage/exmaple: + polling_island *p1; + polling_island *p2; + .. + polling_island_lock_pair(&p1, &p2); + .. + .. Critical section with both p1 and p2 locked + .. + // Release locks + // **IMPORTANT**: Make sure you check p1 == p2 AFTER the function + // polling_island_lock_pair() was called and if so, release the lock only + // once. Note: Even if p1 != p2 beforec calling polling_island_lock_pair(), + // they might be after the function returns: + if (p1 == p2) { + gpr_mu_unlock(&p1->mu) + } else { + gpr_mu_unlock(&p1->mu); + gpr_mu_unlock(&p2->mu); + } + +*/ +void polling_island_lock_pair(polling_island **p, polling_island **q) { polling_island *pi_1 = *p; polling_island *pi_2 = *q; - polling_island *temp = NULL; - bool pi_1_locked = false; - bool pi_2_locked = false; - int num_swaps = 0; - - /* Loop until either pi_1 == pi_2 or until we acquired locks on both pi_1 - and pi_2 */ - while (pi_1 != pi_2 && !(pi_1_locked && pi_2_locked)) { - /* The following assertions are true at this point: - - pi_1 != pi_2 (else, the while loop would have exited) - - pi_1 MAY be locked - - pi_2 is NOT locked */ - - /* To maintain lock order consistency, always lock polling_island node with - lower address first. - First, make sure pi_1 < pi_2 before proceeding any further. If it turns - out that pi_1 > pi_2, unlock pi_1 if locked (because pi_2 is not locked - at this point and having pi_1 locked would violate the lock order) and - swap pi_1 and pi_2 so that pi_1 becomes less than pi_2 */ - if (pi_1 > pi_2) { - if (pi_1_locked) { - gpr_mu_unlock(&pi_1->mu); - pi_1_locked = false; - } + polling_island *next_1 = NULL; + polling_island *next_2 = NULL; + + /* The algorithm is simple: + - Go to the last polling islands in the linked lists *pi_1 and *pi_2 (and + keep updating pi_1 and pi_2) + - Then obtain locks on the islands by following a lock order rule of + locking polling_island with lower address first + Special case: Before obtaining the locks, check if pi_1 and pi_2 are + pointing to the same island. If that is the case, we can just call + polling_island_lock() + - After obtaining both the locks, double check that the polling islands + are still the last polling islands in their respective linked lists + (this is because there might have been polling island merges before + we got the lock) + - If the polling islands are the last islands, we are done. If not, + release the locks and continue the process from the first step */ + while (true) { + next_1 = (polling_island *)gpr_atm_acq_load(&pi_1->merged_to); + while (next_1 != NULL) { + pi_1 = next_1; + next_1 = (polling_island *)gpr_atm_acq_load(&pi_1->merged_to); + } - GPR_SWAP(polling_island *, pi_1, pi_2); - num_swaps++; + next_2 = (polling_island *)gpr_atm_acq_load(&pi_2->merged_to); + while (next_2 != NULL) { + pi_2 = next_2; + next_2 = (polling_island *)gpr_atm_acq_load(&pi_2->merged_to); } - /* The following assertions are true at this point: - - pi_1 != pi_2 - - pi_1 < pi_2 (address of pi_1 is less than that of pi_2) - - pi_1 MAYBE locked - - pi_2 is NOT locked */ + if (pi_1 == pi_2) { + pi_1 = pi_2 = polling_island_lock(pi_1); + break; + } - /* Lock pi_1 (if pi_1 is pointing to the terminal node in the list) */ - if (!pi_1_locked) { + if (pi_1 < pi_2) { + gpr_mu_lock(&pi_1->mu); + gpr_mu_lock(&pi_2->mu); + } else { + gpr_mu_lock(&pi_2->mu); gpr_mu_lock(&pi_1->mu); - pi_1_locked = true; - - /* If pi_1 is not terminal node (i.e pi_1->merged_to != NULL), we are not - done locking this polling_island yet. Release the lock on this node and - advance pi_1 to the next node in the list; and go to the beginning of - the loop (we can't proceed to locking pi_2 unless we locked pi_1 first) - */ - if (pi_1->merged_to != NULL) { - temp = pi_1->merged_to; - polling_island_unref_and_unlock(pi_1, 1); - pi_1 = temp; - pi_1_locked = false; - - continue; - } } - /* The following assertions are true at this point: - - pi_1 is locked - - pi_2 is unlocked - - pi_1 != pi_2 */ - - gpr_mu_lock(&pi_2->mu); - pi_2_locked = true; - - /* If pi_2 is not terminal node, we are not done locking this polling_island - yet. Release the lock and update pi_2 to the next node in the list */ - if (pi_2->merged_to != NULL) { - temp = pi_2->merged_to; - polling_island_unref_and_unlock(pi_2, 1); - pi_2 = temp; - pi_2_locked = false; + next_1 = (polling_island *)gpr_atm_acq_load(&pi_1->merged_to); + next_2 = (polling_island *)gpr_atm_acq_load(&pi_2->merged_to); + if (next_1 == NULL && next_2 == NULL) { + break; } - } - /* At this point, either pi_1 == pi_2 AND/OR we got both locks */ - if (pi_1 == pi_2) { - /* We may or may not have gotten the lock. If we didn't, walk the rest of - the polling_island list and get the lock */ - GPR_ASSERT(pi_1_locked || (!pi_1_locked && !pi_2_locked)); - if (!pi_1_locked) { - pi_1 = pi_2 = polling_island_update_and_lock(pi_1, 2, 0); - } - } else { - GPR_ASSERT(pi_1_locked && pi_2_locked); - /* If we swapped pi_1 and pi_2 odd number of times, do one more swap so that - pi_1 and pi_2 point to the same polling_island lists they started off - with at the beginning of this function (i.e *p and *q respectively) */ - if (num_swaps % 2 > 0) { - GPR_SWAP(polling_island *, pi_1, pi_2); - } + gpr_mu_unlock(&pi_1->mu); + gpr_mu_unlock(&pi_2->mu); } *p = pi_1; @@ -546,7 +611,7 @@ void polling_island_pair_update_and_lock(polling_island **p, polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Get locks on both the polling islands */ - polling_island_pair_update_and_lock(&p, &q); + polling_island_lock_pair(&p, &q); if (p == q) { /* Nothing needs to be done here */ @@ -568,15 +633,14 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Wakeup all the pollers (if any) on p so that they can pickup this change */ polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); - p->merged_to = q; + /* Add the 'merged_to' link from p --> q */ + gpr_atm_rel_store(&p->merged_to, q); + PI_ADD_REF(q, "pi_merge"); /* To account for the new incoming ref from p */ - /* - The merged polling island (i.e q) inherits all the ref counts of the - island merging with it (i.e p) - - The island p will lose a ref count */ - q->ref_cnt += p->ref_cnt; - polling_island_unref_and_unlock(p, 1); /* Decrement refcount */ - polling_island_unref_and_unlock(q, 0); /* Just Unlock. Don't decrement ref */ + gpr_mu_unlock(&p->mu); + gpr_mu_unlock(&q->mu); + /* Return the merged polling island */ return q; } @@ -667,6 +731,7 @@ static void unref_by(grpc_fd *fd, int n) { fd->freelist_next = fd_freelist; fd_freelist = fd; grpc_iomgr_unregister_object(&fd->iomgr_object); + gpr_mu_unlock(&fd_freelist_mu); } else { GPR_ASSERT(old > n); @@ -785,16 +850,20 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, REF_BY(fd, 1, reason); /* Remove the fd from the polling island: - - Update the fd->polling_island to point to the latest polling island - - Remove the fd from the polling island. - - Remove a ref to the polling island and set fd->polling_island to NULL */ + - Get a lock on the latest polling island (i.e the last island in the + linked list pointed by fd->polling_island). This is the island that + would actually contain the fd + - Remove the fd from the latest polling island + - Unlock the latest polling island + - Set fd->polling_island to NULL (but remove the ref on the polling island + before doing this.) */ gpr_mu_lock(&fd->pi_mu); if (fd->polling_island != NULL) { - fd->polling_island = - polling_island_update_and_lock(fd->polling_island, 1, 0); - polling_island_remove_fd_locked(fd->polling_island, fd, is_fd_closed); + polling_island *pi_latest = polling_island_lock(fd->polling_island); + polling_island_remove_fd_locked(pi_latest, fd, is_fd_closed); + gpr_mu_unlock(&pi_latest->mu); - polling_island_unref_and_unlock(fd->polling_island, 1); + PI_UNREF(fd->polling_island, "fd_orphan"); fd->polling_island = NULL; } gpr_mu_unlock(&fd->pi_mu); @@ -1050,17 +1119,13 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { gpr_mu_unlock(&fd->mu); } -/* Release the reference to pollset->polling_island and set it to NULL. - pollset->mu must be held */ -static void pollset_release_polling_island_locked(grpc_pollset *pollset) { - gpr_mu_lock(&pollset->pi_mu); - if (pollset->polling_island) { - pollset->polling_island = - polling_island_update_and_lock(pollset->polling_island, 1, 0); - polling_island_unref_and_unlock(pollset->polling_island, 1); - pollset->polling_island = NULL; +static void pollset_release_polling_island(grpc_pollset *ps, char *reason) { + gpr_mu_lock(&ps->pi_mu); + if (ps->polling_island != NULL) { + PI_UNREF(ps->polling_island, reason); } - gpr_mu_unlock(&pollset->pi_mu); + ps->polling_island = NULL; + gpr_mu_unlock(&ps->pi_mu); } static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, @@ -1069,8 +1134,9 @@ static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, GPR_ASSERT(!pollset_has_workers(pollset)); pollset->finish_shutdown_called = true; - pollset_release_polling_island_locked(pollset); + /* Release the ref and set pollset->polling_island to NULL */ + pollset_release_polling_island(pollset, "ps_shutdown"); grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); } @@ -1110,7 +1176,7 @@ static void pollset_reset(grpc_pollset *pollset) { pollset->finish_shutdown_called = false; pollset->kicked_without_pollers = false; pollset->shutdown_done = NULL; - pollset_release_polling_island_locked(pollset); + pollset_release_polling_island(pollset, "ps_reset"); } #define GRPC_EPOLL_MAX_EVENTS 1000 @@ -1124,28 +1190,37 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the - polling island pointed by pollset->polling_island. + latest polling island pointed by pollset->polling_island. Acquire the following locks: - pollset->mu (which we already have) - pollset->pi_mu - - pollset->polling_island->mu (call polling_island_update_and_lock())*/ + - pollset->polling_island lock */ gpr_mu_lock(&pollset->pi_mu); - pi = pollset->polling_island; - if (pi == NULL) { - pi = polling_island_create(NULL, 1); + if (pollset->polling_island == NULL) { + pollset->polling_island = polling_island_create(NULL); + PI_ADD_REF(pollset->polling_island, "ps"); } - /* In addition to locking the polling island, add a ref so that the island - does not get destroyed (which means the epoll_fd won't be closed) while - we are are doing an epoll_wait() on the epoll_fd */ - pi = polling_island_update_and_lock(pi, 1, 1); + pi = polling_island_lock(pollset->polling_island); epoll_fd = pi->epoll_fd; - /* Update the pollset->polling_island */ - pollset->polling_island = pi; + /* Update the pollset->polling_island since the island being pointed by + pollset->polling_island may not be the latest (i.e pi) */ + if (pollset->polling_island != pi) { + /* Always do PI_ADD_REF before PI_UNREF because PI_UNREF may cause the + polling island to be deleted */ + PI_ADD_REF(pi, "ps"); + PI_UNREF(pollset->polling_island, "ps"); + pollset->polling_island = pi; + } + + /* Add an extra ref so that the island does not get destroyed (which means + the epoll_fd won't be closed) while we are are doing an epoll_wait() on the + epoll_fd */ + PI_ADD_REF(pi, "ps_work"); - polling_island_unref_and_unlock(pollset->polling_island, 0); /* Keep the ref*/ + gpr_mu_unlock(&pi->mu); gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); @@ -1193,14 +1268,12 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, GPR_ASSERT(pi != NULL); - /* Before leaving, release the extra ref we added to the polling island */ - /* It is important to note that at this point 'pi' may not be the same as - * pollset->polling_island. This is because pollset->polling_island pointer - * gets updated whenever the underlying polling island is merged with another - * island and while we are doing epoll_wait() above, the polling island may - * have been merged */ - pi = polling_island_update_and_lock(pi, 1, 0); /* No new ref added */ - polling_island_unref_and_unlock(pi, 1); + /* Before leaving, release the extra ref we added to the polling island. It + is important to use "pi" here (i.e our old copy of pollset->polling_island + that we got before releasing the polling island lock). This is because + pollset->polling_island pointer might get udpated in other parts of the + code when there is an island merge while we are doing epoll_wait() above */ + PI_UNREF(pi, "ps_work"); GPR_TIMER_END("pollset_work_and_unlock", 0); } @@ -1297,20 +1370,34 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, if (fd->polling_island == pollset->polling_island) { pi_new = fd->polling_island; if (pi_new == NULL) { - pi_new = polling_island_create(fd, 2); + pi_new = polling_island_create(fd); } } else if (fd->polling_island == NULL) { - pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); - polling_island_add_fds_locked(pollset->polling_island, &fd, 1, true); + pi_new = polling_island_lock(pollset->polling_island); + polling_island_add_fds_locked(pi_new, &fd, 1, true); gpr_mu_unlock(&pi_new->mu); } else if (pollset->polling_island == NULL) { - pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); + pi_new = polling_island_lock(fd->polling_island); gpr_mu_unlock(&pi_new->mu); } else { pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); } - fd->polling_island = pollset->polling_island = pi_new; + if (fd->polling_island != pi_new) { + PI_ADD_REF(pi_new, "fd"); + if (fd->polling_island != NULL) { + PI_UNREF(fd->polling_island, "fd"); + } + fd->polling_island = pi_new; + } + + if (pollset->polling_island != pi_new) { + PI_ADD_REF(pi_new, "ps"); + if (pollset->polling_island != NULL) { + PI_UNREF(pollset->polling_island, "ps"); + } + pollset->polling_island = pi_new; + } gpr_mu_unlock(&fd->pi_mu); gpr_mu_unlock(&pollset->pi_mu); @@ -1481,28 +1568,19 @@ void *grpc_pollset_get_polling_island(grpc_pollset *ps) { return pi; } -static polling_island *get_polling_island(polling_island *p) { - if (p == NULL) { - return NULL; - } +bool grpc_are_polling_islands_equal(void *p, void *q) { + polling_island *p1 = p; + polling_island *p2 = q; - polling_island *next; - gpr_mu_lock(&p->mu); - while (p->merged_to != NULL) { - next = p->merged_to; - gpr_mu_unlock(&p->mu); - p = next; - gpr_mu_lock(&p->mu); + polling_island_lock_pair(&p1, &p2); + if (p1 == p2) { + gpr_mu_unlock(&p1->mu); + } else { + gpr_mu_unlock(&p1->mu); + gpr_mu_unlock(&p2->mu); } - gpr_mu_unlock(&p->mu); - - return p; -} -bool grpc_are_polling_islands_equal(void *p, void *q) { - p = get_polling_island(p); - q = get_polling_island(q); - return p == q; + return p1 == p2; } /******************************************************************************* From bc98af18fabd94fbc6673dc021fdc45b20432028 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Fri, 17 Jun 2016 18:38:27 -0700 Subject: [PATCH 111/280] make run_tests.py support coreclr on windows --- tools/run_tests/build_csharp_coreclr.bat | 44 ++++++++++++++++++++++++ tools/run_tests/run_tests.py | 38 ++++++++++---------- 2 files changed, 64 insertions(+), 18 deletions(-) create mode 100644 tools/run_tests/build_csharp_coreclr.bat diff --git a/tools/run_tests/build_csharp_coreclr.bat b/tools/run_tests/build_csharp_coreclr.bat new file mode 100644 index 00000000000..cead6d0e02c --- /dev/null +++ b/tools/run_tests/build_csharp_coreclr.bat @@ -0,0 +1,44 @@ +@rem Copyright 2016, Google Inc. +@rem All rights reserved. +@rem +@rem Redistribution and use in source and binary forms, with or without +@rem modification, are permitted provided that the following conditions are +@rem met: +@rem +@rem * Redistributions of source code must retain the above copyright +@rem notice, this list of conditions and the following disclaimer. +@rem * Redistributions in binary form must reproduce the above +@rem copyright notice, this list of conditions and the following disclaimer +@rem in the documentation and/or other materials provided with the +@rem distribution. +@rem * Neither the name of Google Inc. nor the names of its +@rem contributors may be used to endorse or promote products derived from +@rem this software without specific prior written permission. +@rem +@rem THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +@rem "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +@rem LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +@rem A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +@rem OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +@rem SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +@rem LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +@rem DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +@rem THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +@rem (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +@rem OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +setlocal + +cd /d %~dp0\..\..\src\csharp + +dotnet restore . || goto :error + +dotnet build -f netstandard1.5 --configuration %MSBUILD_CONFIG% "**/project.json" || goto :error + +endlocal + +goto :EOF + +:error +echo Failed! +exit /b %errorlevel% diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 473eb460b46..3cb0be579da 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -504,13 +504,9 @@ class CSharpLanguage(object): self._make_options = [_windows_toolset_option(self.args.compiler), _windows_arch_option(self.args.arch)] else: - if self.platform == 'linux': - if self.args.compiler == 'coreclr': - self._docker_distro = 'coreclr' - else: - self._docker_distro = 'jessie' - else: - _check_compiler(self.args.compiler, ['default']) + _check_compiler(self.args.compiler, ['default', 'coreclr']) + if self.platform == 'linux' and self.args.compiler == 'coreclr': + self._docker_distro = 'coreclr' if self.platform == 'mac': # On Mac, official distribution of mono is 32bit. @@ -525,21 +521,22 @@ class CSharpLanguage(object): tests_by_assembly = json.load(f) msbuild_config = _MSBUILD_CONFIG[self.config.build_config] - nunit_args = ['--labels=All', - '--noresult', - '--workers=1'] + nunit_args = ['--labels=All'] assembly_subdir = 'bin/%s' % msbuild_config assembly_extension = '.exe' if self.args.compiler == 'coreclr': # TODO(jtattermusch): make the runtime string platform-specific - assembly_subdir += '/netstandard1.5/debian.8-x64' - assembly_extension = '' - runtime_cmd = [] - elif self.platform == 'windows': + #assembly_subdir += '/netstandard1.5/debian.8-x64' + #assembly_extension = '' + assembly_subdir += '/netstandard1.5/win7-x64' runtime_cmd = [] else: - runtime_cmd = ['mono'] + nunit_args += ['--noresult', '--workers=1'] + if self.platform == 'windows': + runtime_cmd = [] + else: + runtime_cmd = ['mono'] specs = [] for assembly in tests_by_assembly.iterkeys(): @@ -590,7 +587,10 @@ class CSharpLanguage(object): def build_steps(self): if self.args.compiler == 'coreclr': - return [['tools/run_tests/build_csharp_coreclr.sh']] + if self.platform == 'windows': + return [['tools\\run_tests\\build_csharp_coreclr.bat']] + else: + return [['tools/run_tests/build_csharp_coreclr.sh']] else: if self.platform == 'windows': return [[_windows_build_bat(self.args.compiler), @@ -752,7 +752,8 @@ def _check_arch_option(arch): def _windows_build_bat(compiler): """Returns name of build.bat for selected compiler.""" - if compiler == 'default' or compiler == 'vs2013': + # For CoreCLR, fall back to the default compiler for C core + if compiler == 'default' or compiler == 'vs2013' or compiler == 'coreclr': return 'vsprojects\\build_vs2013.bat' elif compiler == 'vs2015': return 'vsprojects\\build_vs2015.bat' @@ -765,7 +766,8 @@ def _windows_build_bat(compiler): def _windows_toolset_option(compiler): """Returns msbuild PlatformToolset for selected compiler.""" - if compiler == 'default' or compiler == 'vs2013': + # For CoreCLR, fall back to the default compiler for C core + if compiler == 'default' or compiler == 'vs2013' or compiler == 'coreclr': return '/p:PlatformToolset=v120' elif compiler == 'vs2015': return '/p:PlatformToolset=v140' From 74330130a3becaced929b2502fdea72bc5589314 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Mon, 20 Jun 2016 15:54:27 -0700 Subject: [PATCH 112/280] improve project.json files --- src/csharp/Grpc.Core.Tests/project.json | 1 - src/csharp/Grpc.Core/project.json | 1 - src/csharp/Grpc.Dotnet.sln | 6 ++++++ .../Grpc.Examples.MathClient/project.json | 1 - .../Grpc.Examples.MathServer/project.json | 1 - src/csharp/Grpc.Examples.Tests/project.json | 1 - src/csharp/Grpc.Examples/project.json | 4 ---- .../Grpc.HealthCheck.Tests/project.json | 1 - src/csharp/Grpc.HealthCheck/project.json | 3 --- .../project.json | 1 - .../project.json | 1 - .../project.json | 1 - ...Grpc.IntegrationTesting.StressClient.xproj | 19 +++++++++++++++++++ .../project.json | 1 - .../Grpc.IntegrationTesting/project.json | 1 - 15 files changed, 25 insertions(+), 18 deletions(-) create mode 100644 src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.xproj diff --git a/src/csharp/Grpc.Core.Tests/project.json b/src/csharp/Grpc.Core.Tests/project.json index a59e6390d9d..95f40fdbbde 100644 --- a/src/csharp/Grpc.Core.Tests/project.json +++ b/src/csharp/Grpc.Core.Tests/project.json @@ -1,6 +1,5 @@ { "buildOptions": { - "compile": "**/*.cs", "copyToOutput": { "mappings": { "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", diff --git a/src/csharp/Grpc.Core/project.json b/src/csharp/Grpc.Core/project.json index b5b7722bba2..ba3dc15495d 100644 --- a/src/csharp/Grpc.Core/project.json +++ b/src/csharp/Grpc.Core/project.json @@ -22,7 +22,6 @@ } }, "buildOptions": { - "compile": "**/*.cs", "embed": [ "../../../etc/roots.pem" ] }, "dependencies": { diff --git a/src/csharp/Grpc.Dotnet.sln b/src/csharp/Grpc.Dotnet.sln index 6a7e2e27482..98b3cd54abb 100644 --- a/src/csharp/Grpc.Dotnet.sln +++ b/src/csharp/Grpc.Dotnet.sln @@ -29,6 +29,8 @@ Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.IntegrationTesting.Qps EndProject Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.IntegrationTesting.Server", "Grpc.IntegrationTesting.Server\Grpc.IntegrationTesting.Server.xproj", "{881F7AD1-A84E-47A2-9402-115C63C4031E}" EndProject +Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Grpc.IntegrationTesting.StressClient", "Grpc.IntegrationTesting.StressClient\Grpc.IntegrationTesting.StressClient.xproj", "{0EBC910B-8867-4D3E-8686-91F34183D839}" +EndProject Global GlobalSection(SolutionConfigurationPlatforms) = preSolution Debug|Any CPU = Debug|Any CPU @@ -87,6 +89,10 @@ Global {881F7AD1-A84E-47A2-9402-115C63C4031E}.Debug|Any CPU.Build.0 = Debug|Any CPU {881F7AD1-A84E-47A2-9402-115C63C4031E}.Release|Any CPU.ActiveCfg = Release|Any CPU {881F7AD1-A84E-47A2-9402-115C63C4031E}.Release|Any CPU.Build.0 = Release|Any CPU + {0EBC910B-8867-4D3E-8686-91F34183D839}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {0EBC910B-8867-4D3E-8686-91F34183D839}.Debug|Any CPU.Build.0 = Debug|Any CPU + {0EBC910B-8867-4D3E-8686-91F34183D839}.Release|Any CPU.ActiveCfg = Release|Any CPU + {0EBC910B-8867-4D3E-8686-91F34183D839}.Release|Any CPU.Build.0 = Release|Any CPU EndGlobalSection GlobalSection(SolutionProperties) = preSolution HideSolutionNode = FALSE diff --git a/src/csharp/Grpc.Examples.MathClient/project.json b/src/csharp/Grpc.Examples.MathClient/project.json index 9c070c76ba5..4c1ea780412 100644 --- a/src/csharp/Grpc.Examples.MathClient/project.json +++ b/src/csharp/Grpc.Examples.MathClient/project.json @@ -1,6 +1,5 @@ { "buildOptions": { - "compile": "**/*.cs", "copyToOutput": { "mappings": { "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", diff --git a/src/csharp/Grpc.Examples.MathServer/project.json b/src/csharp/Grpc.Examples.MathServer/project.json index 9c070c76ba5..4c1ea780412 100644 --- a/src/csharp/Grpc.Examples.MathServer/project.json +++ b/src/csharp/Grpc.Examples.MathServer/project.json @@ -1,6 +1,5 @@ { "buildOptions": { - "compile": "**/*.cs", "copyToOutput": { "mappings": { "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", diff --git a/src/csharp/Grpc.Examples.Tests/project.json b/src/csharp/Grpc.Examples.Tests/project.json index 7dd938cfb16..f41be82bd53 100644 --- a/src/csharp/Grpc.Examples.Tests/project.json +++ b/src/csharp/Grpc.Examples.Tests/project.json @@ -1,6 +1,5 @@ { "buildOptions": { - "compile": "**/*.cs", "copyToOutput": { "mappings": { "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", diff --git a/src/csharp/Grpc.Examples/project.json b/src/csharp/Grpc.Examples/project.json index 610712f6460..fe580eb165b 100644 --- a/src/csharp/Grpc.Examples/project.json +++ b/src/csharp/Grpc.Examples/project.json @@ -1,8 +1,4 @@ { - "buildOptions": { - "compile": "**/*.cs" - }, - "dependencies": { "Grpc.Core": { "target": "project" diff --git a/src/csharp/Grpc.HealthCheck.Tests/project.json b/src/csharp/Grpc.HealthCheck.Tests/project.json index be2b3a0459f..5a5f063258a 100644 --- a/src/csharp/Grpc.HealthCheck.Tests/project.json +++ b/src/csharp/Grpc.HealthCheck.Tests/project.json @@ -1,6 +1,5 @@ { "buildOptions": { - "compile": "**/*.cs", "copyToOutput": { "mappings": { "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", diff --git a/src/csharp/Grpc.HealthCheck/project.json b/src/csharp/Grpc.HealthCheck/project.json index c69db5f997d..e9fdfad4731 100644 --- a/src/csharp/Grpc.HealthCheck/project.json +++ b/src/csharp/Grpc.HealthCheck/project.json @@ -12,9 +12,6 @@ "requireLicenseAcceptance": false, "tags": [ "gRPC health check" ] }, - "buildOptions": { - "compile": "**/*.cs" - }, "dependencies": { "Grpc.Core": "0.14.0-anexperiment", "Google.Protobuf": "3.0.0-beta3" diff --git a/src/csharp/Grpc.IntegrationTesting.Client/project.json b/src/csharp/Grpc.IntegrationTesting.Client/project.json index fabf906a738..b19b76c6ccf 100644 --- a/src/csharp/Grpc.IntegrationTesting.Client/project.json +++ b/src/csharp/Grpc.IntegrationTesting.Client/project.json @@ -1,6 +1,5 @@ { "buildOptions": { - "compile": "**/*.cs", "copyToOutput": { "mappings": { "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", diff --git a/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json b/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json index fabf906a738..b19b76c6ccf 100644 --- a/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json +++ b/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json @@ -1,6 +1,5 @@ { "buildOptions": { - "compile": "**/*.cs", "copyToOutput": { "mappings": { "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", diff --git a/src/csharp/Grpc.IntegrationTesting.Server/project.json b/src/csharp/Grpc.IntegrationTesting.Server/project.json index fabf906a738..b19b76c6ccf 100644 --- a/src/csharp/Grpc.IntegrationTesting.Server/project.json +++ b/src/csharp/Grpc.IntegrationTesting.Server/project.json @@ -1,6 +1,5 @@ { "buildOptions": { - "compile": "**/*.cs", "copyToOutput": { "mappings": { "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", diff --git a/src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.xproj b/src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.xproj new file mode 100644 index 00000000000..2f4fdcbb470 --- /dev/null +++ b/src/csharp/Grpc.IntegrationTesting.StressClient/Grpc.IntegrationTesting.StressClient.xproj @@ -0,0 +1,19 @@ + + + + 14.0.25123 + $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) + + + + 0ebc910b-8867-4d3e-8686-91f34183d839 + Grpc.IntegrationTesting.StressClient + .\obj + .\bin\ + + + + 2.0 + + + \ No newline at end of file diff --git a/src/csharp/Grpc.IntegrationTesting.StressClient/project.json b/src/csharp/Grpc.IntegrationTesting.StressClient/project.json index fabf906a738..b19b76c6ccf 100644 --- a/src/csharp/Grpc.IntegrationTesting.StressClient/project.json +++ b/src/csharp/Grpc.IntegrationTesting.StressClient/project.json @@ -1,6 +1,5 @@ { "buildOptions": { - "compile": "**/*.cs", "copyToOutput": { "mappings": { "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", diff --git a/src/csharp/Grpc.IntegrationTesting/project.json b/src/csharp/Grpc.IntegrationTesting/project.json index d5ac6f108ba..be857279892 100644 --- a/src/csharp/Grpc.IntegrationTesting/project.json +++ b/src/csharp/Grpc.IntegrationTesting/project.json @@ -1,6 +1,5 @@ { "buildOptions": { - "compile": "**/*.cs", "copyToOutput": { "mappings": { "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", From aa338326ed399832394078282664c6bcf66d11e7 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Mon, 20 Jun 2016 16:47:40 -0700 Subject: [PATCH 113/280] Revert ProtoService.m, add an exception for its incompatible-pointer-types warning --- src/objective-c/ProtoRPC/ProtoService.m | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/objective-c/ProtoRPC/ProtoService.m b/src/objective-c/ProtoRPC/ProtoService.m index 4a14570d818..cd9bc7aeac4 100644 --- a/src/objective-c/ProtoRPC/ProtoService.m +++ b/src/objective-c/ProtoRPC/ProtoService.m @@ -65,19 +65,22 @@ return self; } -- (GRPCProtoCall *)RPCToMethod:(NSString *)method +#pragma clang diagnostic push +#pragma clang diagnostic ignored "-Wincompatible-pointer-types" +- (ProtoRPC *)RPCToMethod:(NSString *)method requestsWriter:(GRXWriter *)requestsWriter responseClass:(Class)responseClass responsesWriteable:(id)responsesWriteable { - GRPCProtoMethod *methodName = [[GRPCProtoMethod alloc] initWithPackage:_packageName + ProtoMethod *methodName = [[ProtoMethod alloc] initWithPackage:_packageName service:_serviceName method:method]; - return [[GRPCProtoCall alloc] initWithHost:_host + return [[ProtoRPC alloc] initWithHost:_host method:methodName requestsWriter:requestsWriter responseClass:responseClass responsesWriteable:responsesWriteable]; } +#pragma clang diagnostic pop @end @implementation GRPCProtoService From bbe601ab7ca1ba763f95415fbcd391947987341a Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Mon, 20 Jun 2016 17:49:34 -0700 Subject: [PATCH 114/280] add a hack --- tools/run_tests/build_csharp_coreclr.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tools/run_tests/build_csharp_coreclr.sh b/tools/run_tests/build_csharp_coreclr.sh index 733b1a2083c..68c19cb6c9d 100755 --- a/tools/run_tests/build_csharp_coreclr.sh +++ b/tools/run_tests/build_csharp_coreclr.sh @@ -36,3 +36,7 @@ cd $(dirname $0)/../../src/csharp dotnet restore . dotnet build -f netstandard1.5 --configuration $MSBUILD_CONFIG '**/project.json' + +# Grpc.IntegrationTesting doesn't get built by the previous command for some reason. +# TODO(jtattermusch): get rid of the hack +dotnet build -f netstandard1.5 --configuration $MSBUILD_CONFIG Grpc.IntegrationTesting/project.json From 64c137c4f5e23e898d5fbc6306b191daaf147ec0 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Mon, 20 Jun 2016 17:54:19 -0700 Subject: [PATCH 115/280] select runtime dir --- tools/run_tests/run_tests.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 3cb0be579da..c5cf0135f73 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -526,10 +526,11 @@ class CSharpLanguage(object): assembly_extension = '.exe' if self.args.compiler == 'coreclr': - # TODO(jtattermusch): make the runtime string platform-specific - #assembly_subdir += '/netstandard1.5/debian.8-x64' - #assembly_extension = '' - assembly_subdir += '/netstandard1.5/win7-x64' + if self.platform == 'linux': + assembly_subdir += '/netstandard1.5/debian.8-x64' + assembly_extension = '' + else: + assembly_subdir += '/netstandard1.5/win7-x64' runtime_cmd = [] else: nunit_args += ['--noresult', '--workers=1'] From 74ea91d55229a654d0987d2f39cddf8514926f14 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Mon, 20 Jun 2016 18:19:41 -0700 Subject: [PATCH 116/280] use qps_json_driver for latency profiling --- .../latency_profile/run_latency_profile.sh | 94 ++++++++++++------- 1 file changed, 58 insertions(+), 36 deletions(-) diff --git a/tools/profiling/latency_profile/run_latency_profile.sh b/tools/profiling/latency_profile/run_latency_profile.sh index 54a25a9cb73..40c6fcb4314 100755 --- a/tools/profiling/latency_profile/run_latency_profile.sh +++ b/tools/profiling/latency_profile/run_latency_profile.sh @@ -1,5 +1,5 @@ #!/bin/bash -# Copyright 2015, Google Inc. +# Copyright 2016, Google Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without @@ -28,17 +28,61 @@ # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +# format argument via +# $ echo '{...}' | python -mjson.tool +read -r -d '' SCENARIOS_JSON_ARG <<'EOF' +{ + "scenarios": [ + { + "benchmark_seconds": 5, + "client_config": { + "client_channels": 1, + "client_type": "SYNC_CLIENT", + "histogram_params": { + "max_possible": 60000000000.0, + "resolution": 0.01 + }, + "load_params": { + "closed_loop": {} + }, + "outstanding_rpcs_per_channel": 1, + "payload_config": { + "simple_params": { + "req_size": 0, + "resp_size": 0 + } + }, + "rpc_type": "UNARY", + "security_params": { + "server_host_override": "foo.test.google.fr", + "use_test_ca": true + } + }, + "name": "cpp_protobuf_sync_unary_ping_pong_secure", + "num_clients": 1, + "num_servers": 1, + "server_config": { + "core_limit": 1, + "security_params": { + "server_host_override": "foo.test.google.fr", + "use_test_ca": true + }, + "server_type": "SYNC_SERVER" + }, + "spawn_local_worker_count": 2, + "warmup_seconds": 5 + } + ] +} + +EOF + set -ex cd $(dirname $0)/../../.. -BINS="sync_unary_ping_pong_test sync_streaming_ping_pong_test" CPUS=`python -c 'import multiprocessing; print multiprocessing.cpu_count()'` -make CONFIG=basicprof -j$CPUS $BINS - -mkdir -p reports - # try to use pypy for generating reports # each trace dumps 7-8gig of text to disk, and processing this into a report is # heavyweight - so any speed boost is worthwhile @@ -49,35 +93,13 @@ else PYTHON=python2.7 fi -# start processes, interleaving report index generation +make CONFIG=basicprof -j$CPUS qps_json_driver + +mkdir -p reports echo '' > reports/index.html -for bin in $BINS -do - bins/basicprof/$bin - mv latency_trace.txt $bin.trace - echo "$bin
" >> reports/index.html -done -pids="" -# generate report pages... this will take some time -# run them in parallel: they take 1 cpu each -for bin in $BINS -do - $PYTHON tools/profiling/latency_profile/profile_analyzer.py \ - --source=$bin.trace --fmt=simple > reports/$bin.txt & - pids+=" $!" -done +bins/basicprof/qps_json_driver --scenarios_json="$SCENARIOS_JSON_ARG" +echo '
' >> reports/index.html
+$PYTHON tools/profiling/latency_profile/profile_analyzer.py \
+    --source=latency_trace.txt --fmt=simple >> reports/index.html
+echo '
' >> reports/index.html echo '' >> reports/index.html - -# make sure we kill the report generation if something goes wrong -trap "kill $pids || true" 0 - -# finally, wait for the background report generation to finish -for pid in $pids -do - if wait $pid - then - echo "Finished $pid" - else - exit 1 - fi -done From cddf697ab44a7bab1821915e1e3f6a0f08ca1706 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 21 Jun 2016 08:27:07 -0700 Subject: [PATCH 117/280] Fix refcounting tsan failures and grab pollset lock in the function pollset_add_fd --- src/core/lib/iomgr/ev_epoll_linux.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 72288889c02..7cc69c876db 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -291,11 +291,11 @@ void pi_unref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, #endif long pi_add_ref(polling_island *pi, int ref_cnt) { - return gpr_atm_no_barrier_fetch_add(&pi->ref_count, ref_cnt); + return gpr_atm_full_fetch_add(&pi->ref_count, ref_cnt); } long pi_unref(polling_island *pi, int ref_cnt) { - long old_cnt = gpr_atm_no_barrier_fetch_add(&pi->ref_count, -ref_cnt); + long old_cnt = gpr_atm_full_fetch_add(&pi->ref_count, -ref_cnt); /* If ref count went to zero, delete the polling island. Note that this need not be done under a lock. Once the ref count goes to zero, we are @@ -311,6 +311,8 @@ long pi_unref(polling_island *pi, int ref_cnt) { if (next != NULL) { PI_UNREF(next, "pi_delete"); /* Recursive call */ } + } else { + GPR_ASSERT(old_cnt > ref_cnt); } return old_cnt; @@ -445,8 +447,8 @@ static polling_island *polling_island_create(grpc_fd *initial_fd) { pi->fds = NULL; } - gpr_atm_no_barrier_store(&pi->ref_count, 0); - gpr_atm_no_barrier_store(&pi->merged_to, NULL); + gpr_atm_rel_store(&pi->ref_count, 0); + gpr_atm_rel_store(&pi->merged_to, NULL); pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); @@ -1347,7 +1349,7 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { - /* TODO sreek - Double check if we need to get a pollset->mu lock here */ + gpr_mu_lock(&pollset->mu); gpr_mu_lock(&pollset->pi_mu); gpr_mu_lock(&fd->pi_mu); @@ -1401,6 +1403,7 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, gpr_mu_unlock(&fd->pi_mu); gpr_mu_unlock(&pollset->pi_mu); + gpr_mu_unlock(&pollset->mu); } /******************************************************************************* From e63246d1007516ca5c74043139304a371c0ce672 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Tue, 21 Jun 2016 09:03:11 -0700 Subject: [PATCH 118/280] clang-format --- test/core/iomgr/tcp_server_posix_test.c | 2 +- test/core/surface/server_chttp2_test.c | 2 +- test/core/util/test_tcp_server.c | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/test/core/iomgr/tcp_server_posix_test.c b/test/core/iomgr/tcp_server_posix_test.c index 6361e7afcbc..6e2d1d0fc9e 100644 --- a/test/core/iomgr/tcp_server_posix_test.c +++ b/test/core/iomgr/tcp_server_posix_test.c @@ -166,7 +166,7 @@ static void test_no_op_with_port_and_start(void) { grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; struct sockaddr_in addr; grpc_tcp_server *s; - GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, NULL, &s)); + GPR_ASSERT(GRPC_ERROR_NONE == grpc_tcp_server_create(NULL, NULL, &s)); LOG_TEST("test_no_op_with_port_and_start"); int port; diff --git a/test/core/surface/server_chttp2_test.c b/test/core/surface/server_chttp2_test.c index 40cfa6b5989..6310b6f00b0 100644 --- a/test/core/surface/server_chttp2_test.c +++ b/test/core/surface/server_chttp2_test.c @@ -54,7 +54,7 @@ void test_add_same_port_twice() { a.key = GRPC_ARG_ALLOW_REUSEPORT; a.value.integer = 0; grpc_channel_args args = {1, &a}; - + int port = grpc_pick_unused_port_or_die(); char *addr = NULL; grpc_completion_queue *cq = grpc_completion_queue_create(NULL); diff --git a/test/core/util/test_tcp_server.c b/test/core/util/test_tcp_server.c index c1d307fc779..8a0b3932d86 100644 --- a/test/core/util/test_tcp_server.c +++ b/test/core/util/test_tcp_server.c @@ -72,8 +72,8 @@ void test_tcp_server_start(test_tcp_server *server, int port) { addr.sin_port = htons((uint16_t)port); memset(&addr.sin_addr, 0, sizeof(addr.sin_addr)); - grpc_error *error = - grpc_tcp_server_create(&server->shutdown_complete, NULL, &server->tcp_server); + grpc_error *error = grpc_tcp_server_create(&server->shutdown_complete, NULL, + &server->tcp_server); GPR_ASSERT(error == GRPC_ERROR_NONE); error = grpc_tcp_server_add_port(server->tcp_server, &addr, sizeof(addr), &port_added); From 39f9ac9b2a51fdcdbd3693d8a04eec3473eb9438 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 16 Jun 2016 16:53:59 -0700 Subject: [PATCH 119/280] Test polling island merges --- Makefile | 36 ++++ build.yaml | 12 ++ src/core/lib/iomgr/ev_epoll_linux.c | 51 +++++- src/core/lib/iomgr/ev_epoll_linux.h | 6 + src/core/lib/iomgr/ev_posix.c | 7 + src/core/lib/iomgr/ev_posix.h | 3 + test/core/iomgr/ev_epoll_linux_test.c | 222 +++++++++++++++++++++++ tools/run_tests/sources_and_headers.json | 16 ++ tools/run_tests/tests.json | 15 ++ 9 files changed, 366 insertions(+), 2 deletions(-) create mode 100644 test/core/iomgr/ev_epoll_linux_test.c diff --git a/Makefile b/Makefile index e615704395b..825684cc2dd 100644 --- a/Makefile +++ b/Makefile @@ -905,6 +905,7 @@ dns_resolver_connectivity_test: $(BINDIR)/$(CONFIG)/dns_resolver_connectivity_te dns_resolver_test: $(BINDIR)/$(CONFIG)/dns_resolver_test dualstack_socket_test: $(BINDIR)/$(CONFIG)/dualstack_socket_test endpoint_pair_test: $(BINDIR)/$(CONFIG)/endpoint_pair_test +ev_epoll_linux_test: $(BINDIR)/$(CONFIG)/ev_epoll_linux_test fd_conservation_posix_test: $(BINDIR)/$(CONFIG)/fd_conservation_posix_test fd_posix_test: $(BINDIR)/$(CONFIG)/fd_posix_test fling_client: $(BINDIR)/$(CONFIG)/fling_client @@ -1242,6 +1243,7 @@ buildtests_c: privatelibs_c \ $(BINDIR)/$(CONFIG)/dns_resolver_test \ $(BINDIR)/$(CONFIG)/dualstack_socket_test \ $(BINDIR)/$(CONFIG)/endpoint_pair_test \ + $(BINDIR)/$(CONFIG)/ev_epoll_linux_test \ $(BINDIR)/$(CONFIG)/fd_conservation_posix_test \ $(BINDIR)/$(CONFIG)/fd_posix_test \ $(BINDIR)/$(CONFIG)/fling_client \ @@ -1512,6 +1514,8 @@ test_c: buildtests_c $(Q) $(BINDIR)/$(CONFIG)/dualstack_socket_test || ( echo test dualstack_socket_test failed ; exit 1 ) $(E) "[RUN] Testing endpoint_pair_test" $(Q) $(BINDIR)/$(CONFIG)/endpoint_pair_test || ( echo test endpoint_pair_test failed ; exit 1 ) + $(E) "[RUN] Testing ev_epoll_linux_test" + $(Q) $(BINDIR)/$(CONFIG)/ev_epoll_linux_test || ( echo test ev_epoll_linux_test failed ; exit 1 ) $(E) "[RUN] Testing fd_conservation_posix_test" $(Q) $(BINDIR)/$(CONFIG)/fd_conservation_posix_test || ( echo test fd_conservation_posix_test failed ; exit 1 ) $(E) "[RUN] Testing fd_posix_test" @@ -7130,6 +7134,38 @@ endif endif +EV_EPOLL_LINUX_TEST_SRC = \ + test/core/iomgr/ev_epoll_linux_test.c \ + +EV_EPOLL_LINUX_TEST_OBJS = $(addprefix $(OBJDIR)/$(CONFIG)/, $(addsuffix .o, $(basename $(EV_EPOLL_LINUX_TEST_SRC)))) +ifeq ($(NO_SECURE),true) + +# You can't build secure targets if you don't have OpenSSL. + +$(BINDIR)/$(CONFIG)/ev_epoll_linux_test: openssl_dep_error + +else + + + +$(BINDIR)/$(CONFIG)/ev_epoll_linux_test: $(EV_EPOLL_LINUX_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a + $(E) "[LD] Linking $@" + $(Q) mkdir -p `dirname $@` + $(Q) $(LD) $(LDFLAGS) $(EV_EPOLL_LINUX_TEST_OBJS) $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a $(LDLIBS) $(LDLIBS_SECURE) -o $(BINDIR)/$(CONFIG)/ev_epoll_linux_test + +endif + +$(OBJDIR)/$(CONFIG)/test/core/iomgr/ev_epoll_linux_test.o: $(LIBDIR)/$(CONFIG)/libgrpc_test_util.a $(LIBDIR)/$(CONFIG)/libgrpc.a $(LIBDIR)/$(CONFIG)/libgpr_test_util.a $(LIBDIR)/$(CONFIG)/libgpr.a + +deps_ev_epoll_linux_test: $(EV_EPOLL_LINUX_TEST_OBJS:.o=.dep) + +ifneq ($(NO_SECURE),true) +ifneq ($(NO_DEPS),true) +-include $(EV_EPOLL_LINUX_TEST_OBJS:.o=.dep) +endif +endif + + FD_CONSERVATION_POSIX_TEST_SRC = \ test/core/iomgr/fd_conservation_posix_test.c \ diff --git a/build.yaml b/build.yaml index 7790e0c5174..84f4ea521be 100644 --- a/build.yaml +++ b/build.yaml @@ -1407,6 +1407,18 @@ targets: - grpc - gpr_test_util - gpr +- name: ev_epoll_linux_test + build: test + language: c + src: + - test/core/iomgr/ev_epoll_linux_test.c + deps: + - grpc_test_util + - grpc + - gpr_test_util + - gpr + platforms: + - linux - name: fd_conservation_posix_test build: test language: c diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 1fb59474640..ed2c494b783 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -317,8 +317,9 @@ static void polling_island_remove_all_fds_locked(polling_island *pi, if (err < 0 && errno != ENOENT) { /* TODO: sreek - We need a better way to bubble up this error instead of * just logging a message */ - gpr_log(GPR_ERROR, "epoll_ctl deleting fds[%zu]: %d failed with error: %s", - i, pi->fds[i]->fd, strerror(errno)); + gpr_log(GPR_ERROR, + "epoll_ctl deleting fds[%zu]: %d failed with error: %s", i, + pi->fds[i]->fd, strerror(errno)); } if (remove_fd_refs) { @@ -1458,6 +1459,52 @@ static void pollset_set_del_pollset_set(grpc_exec_ctx *exec_ctx, gpr_mu_unlock(&bag->mu); } +/* Test helper functions + * */ +void *grpc_fd_get_polling_island(grpc_fd *fd) { + polling_island *pi; + + gpr_mu_lock(&fd->pi_mu); + pi = fd->polling_island; + gpr_mu_unlock(&fd->pi_mu); + + return pi; +} + +void *grpc_pollset_get_polling_island(grpc_pollset *ps) { + polling_island *pi; + + gpr_mu_lock(&ps->pi_mu); + pi = ps->polling_island; + gpr_mu_unlock(&ps->pi_mu); + + return pi; +} + +static polling_island *get_polling_island(polling_island *p) { + if (p == NULL) { + return NULL; + } + + polling_island *next; + gpr_mu_lock(&p->mu); + while (p->merged_to != NULL) { + next = p->merged_to; + gpr_mu_unlock(&p->mu); + p = next; + gpr_mu_lock(&p->mu); + } + gpr_mu_unlock(&p->mu); + + return p; +} + +bool grpc_are_polling_islands_equal(void *p, void *q) { + p = get_polling_island(p); + q = get_polling_island(q); + return p == q; +} + /******************************************************************************* * Event engine binding */ diff --git a/src/core/lib/iomgr/ev_epoll_linux.h b/src/core/lib/iomgr/ev_epoll_linux.h index 8c819975a4c..7a494aba198 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.h +++ b/src/core/lib/iomgr/ev_epoll_linux.h @@ -38,4 +38,10 @@ const grpc_event_engine_vtable *grpc_init_epoll_linux(void); +#ifdef GPR_LINUX_EPOLL +void *grpc_fd_get_polling_island(grpc_fd *fd); +void *grpc_pollset_get_polling_island(grpc_pollset *ps); +bool grpc_are_polling_islands_equal(void *p, void *q); +#endif /* defined(GPR_LINUX_EPOLL) */ + #endif /* GRPC_CORE_LIB_IOMGR_EV_EPOLL_LINUX_H */ diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index 2b15967adcc..5b20600a6f7 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -54,6 +54,7 @@ grpc_poll_function_type grpc_poll_function = poll; static const grpc_event_engine_vtable *g_event_engine; +static const char* g_poll_strategy_name = NULL; typedef const grpc_event_engine_vtable *(*event_engine_factory_fn)(void); @@ -101,6 +102,7 @@ static void try_engine(const char *engine) { for (size_t i = 0; i < GPR_ARRAY_SIZE(g_factories); i++) { if (is(engine, g_factories[i].name)) { if ((g_event_engine = g_factories[i].factory())) { + g_poll_strategy_name = g_factories[i].name; gpr_log(GPR_DEBUG, "Using polling engine: %s", g_factories[i].name); return; } @@ -108,6 +110,11 @@ static void try_engine(const char *engine) { } } +/* Call this only after calling grpc_event_engine_init() */ +const char *grpc_get_poll_strategy_name() { + return g_poll_strategy_name; +} + void grpc_event_engine_init(void) { char *s = gpr_getenv("GRPC_POLL_STRATEGY"); if (s == NULL) { diff --git a/src/core/lib/iomgr/ev_posix.h b/src/core/lib/iomgr/ev_posix.h index 344bf63438a..3ed5a5f9562 100644 --- a/src/core/lib/iomgr/ev_posix.h +++ b/src/core/lib/iomgr/ev_posix.h @@ -98,6 +98,9 @@ typedef struct grpc_event_engine_vtable { void grpc_event_engine_init(void); void grpc_event_engine_shutdown(void); +/* Return the name of the poll strategy */ +const char* grpc_get_poll_strategy_name(); + /* Create a wrapped file descriptor. Requires fd is a non-blocking file descriptor. This takes ownership of closing fd. */ diff --git a/test/core/iomgr/ev_epoll_linux_test.c b/test/core/iomgr/ev_epoll_linux_test.c new file mode 100644 index 00000000000..51da15faa7a --- /dev/null +++ b/test/core/iomgr/ev_epoll_linux_test.c @@ -0,0 +1,222 @@ +/* + * + * Copyright 2015, Google Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Google Inc. nor the names of its + * contributors may be used to endorse or promote products derived from + * this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include "src/core/lib/iomgr/ev_epoll_linux.h" +#include "src/core/lib/iomgr/ev_posix.h" + +#include +#include +#include +#include + +#include +#include + +#include "src/core/lib/iomgr/iomgr.h" +#include "test/core/util/test_config.h" + +typedef struct test_pollset { + grpc_pollset *pollset; + gpr_mu *mu; +} test_pollset; + +typedef struct test_fd { + int inner_fd; + grpc_fd *fd; +} test_fd; + +static void test_fd_init(test_fd *fds, int num_fds) { + int i; + for (i = 0; i < num_fds; i++) { + fds[i].inner_fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + fds[i].fd = grpc_fd_create(fds[i].inner_fd, "test_fd"); + } +} + +static void test_fd_cleanup(grpc_exec_ctx *exec_ctx, test_fd *fds, + int num_fds) { + int release_fd; + int i; + + for (i = 0; i < num_fds; i++) { + grpc_fd_shutdown(exec_ctx, fds[i].fd); + grpc_exec_ctx_flush(exec_ctx); + + grpc_fd_orphan(exec_ctx, fds[i].fd, NULL, &release_fd, "test_fd_cleanup"); + grpc_exec_ctx_flush(exec_ctx); + + GPR_ASSERT(release_fd == fds[i].inner_fd); + close(fds[i].inner_fd); + } +} + +static void test_pollset_init(test_pollset *pollsets, int num_pollsets) { + int i; + for (i = 0; i < num_pollsets; i++) { + pollsets[i].pollset = gpr_malloc(grpc_pollset_size()); + grpc_pollset_init(pollsets[i].pollset, &pollsets[i].mu); + } +} + +static void destroy_pollset(grpc_exec_ctx *exec_ctx, void *p, bool success) { + grpc_pollset_destroy(p); +} + +static void test_pollset_cleanup(grpc_exec_ctx *exec_ctx, + test_pollset *pollsets, int num_pollsets) { + grpc_closure destroyed; + int i; + + for (i = 0; i < num_pollsets; i++) { + grpc_closure_init(&destroyed, destroy_pollset, pollsets[i].pollset); + grpc_pollset_shutdown(exec_ctx, pollsets[i].pollset, &destroyed); + + grpc_exec_ctx_flush(exec_ctx); + gpr_free(pollsets[i].pollset); + } +} + +#define NUM_FDS 8 +#define NUM_POLLSETS 4 +/* + * Cases to test: + * case 1) Polling islands of both fd and pollset are NULL + * case 2) Polling island of fd is NULL but that of pollset is not-NULL + * case 3) Polling island of fd is not-NULL but that of pollset is NULL + * case 4) Polling islands of both fd and pollset are not-NULL and: + * case 4.1) Polling islands of fd and pollset are equal + * case 4.2) Polling islands of fd and pollset are NOT-equal (This results + * in a merge) + * */ +static void test_add_fd_to_pollset() { + grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; + test_fd fds[NUM_FDS]; + test_pollset pollsets[NUM_POLLSETS]; + void *expected_pi = NULL; + int i; + + test_fd_init(fds, NUM_FDS); + test_pollset_init(pollsets, NUM_POLLSETS); + + /*Step 1. + * Create three polling islands (This will exercise test case 1 and 2) with + * the following configuration: + * polling island 0 = { fds:0,1,2, pollsets:0} + * polling island 1 = { fds:3,4, pollsets:1} + * polling island 2 = { fds:5,6,7 pollsets:2} + * + *Step 2. + * Add pollset 3 to polling island 0 (by adding fds 0 and 1 to pollset 3) + * (This will exercise test cases 3 and 4.1). The configuration becomes: + * polling island 0 = { fds:0,1,2, pollsets:0,3} <<< pollset 3 added here + * polling island 1 = { fds:3,4, pollsets:1} + * polling island 2 = { fds:5,6,7 pollsets:2} + * + *Step 3. + * Merge polling islands 0 and 1 by adding fd 0 to pollset 1 (This will + * exercise test case 4.2). The configuration becomes: + * polling island (merged) = {fds: 0,1,2,3,4, pollsets: 0,1,3} + * polling island 2 = {fds: 5,6,7 pollsets: 2} + * + *Step 4. + * Finally do one more merge by adding fd 3 to pollset 2. + * polling island (merged) = {fds: 0,1,2,3,4,5,6,7, pollsets: 0,1,2,3} + */ + + /* == Step 1 == */ + for (i = 0; i <= 2; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[0].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + for (i = 3; i <= 4; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + for (i = 5; i <= 7; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + /* == Step 2 == */ + for (i = 0; i <= 1; i++) { + grpc_pollset_add_fd(&exec_ctx, pollsets[3].pollset, fds[i].fd); + grpc_exec_ctx_flush(&exec_ctx); + } + + /* == Step 3 == */ + grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, fds[0].fd); + grpc_exec_ctx_flush(&exec_ctx); + + /* == Step 4 == */ + grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, fds[3].fd); + grpc_exec_ctx_flush(&exec_ctx); + + /* All polling islands are merged at this point */ + + /* Compare Fd:0's polling island with that of all other Fds */ + expected_pi = grpc_fd_get_polling_island(fds[0].fd); + for (i = 1; i < NUM_FDS; i++) { + GPR_ASSERT(grpc_are_polling_islands_equal( + expected_pi, grpc_fd_get_polling_island(fds[i].fd))); + } + + /* Compare Fd:0's polling island with that of all other pollsets */ + for (i = 0; i < NUM_POLLSETS; i++) { + GPR_ASSERT(grpc_are_polling_islands_equal( + expected_pi, grpc_pollset_get_polling_island(pollsets[i].pollset))); + } + + test_fd_cleanup(&exec_ctx, fds, NUM_FDS); + test_pollset_cleanup(&exec_ctx, pollsets, NUM_POLLSETS); + grpc_exec_ctx_finish(&exec_ctx); +} + +int main(int argc, char **argv) { + const char *poll_strategy = NULL; + grpc_test_init(argc, argv); + grpc_iomgr_init(); + + poll_strategy = grpc_get_poll_strategy_name(); + if (poll_strategy != NULL && strcmp(poll_strategy, "epoll") == 0) { + test_add_fd_to_pollset(); + } else { + gpr_log(GPR_INFO, + "Skipping the test. The test is only relevant for 'epoll' " + "strategy. and the current strategy is: '%s'", + poll_strategy); + } + grpc_iomgr_shutdown(); + return 0; +} diff --git a/tools/run_tests/sources_and_headers.json b/tools/run_tests/sources_and_headers.json index e8ff61dc3fb..e9df72e43a1 100644 --- a/tools/run_tests/sources_and_headers.json +++ b/tools/run_tests/sources_and_headers.json @@ -315,6 +315,22 @@ "third_party": false, "type": "target" }, + { + "deps": [ + "gpr", + "gpr_test_util", + "grpc", + "grpc_test_util" + ], + "headers": [], + "language": "c", + "name": "ev_epoll_linux_test", + "src": [ + "test/core/iomgr/ev_epoll_linux_test.c" + ], + "third_party": false, + "type": "target" + }, { "deps": [ "gpr", diff --git a/tools/run_tests/tests.json b/tools/run_tests/tests.json index 5a84a41b638..ba661840dad 100644 --- a/tools/run_tests/tests.json +++ b/tools/run_tests/tests.json @@ -377,6 +377,21 @@ "windows" ] }, + { + "args": [], + "ci_platforms": [ + "linux" + ], + "cpu_cost": 1.0, + "exclude_configs": [], + "flaky": false, + "gtest": false, + "language": "c", + "name": "ev_epoll_linux_test", + "platforms": [ + "linux" + ] + }, { "args": [], "ci_platforms": [ From 94cda1a9c6ae86ab176d357b8822332d70283cde Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 17 Jun 2016 13:28:38 -0700 Subject: [PATCH 120/280] Significantly refactor the polling island locking and refcounting code --- src/core/lib/iomgr/ev_epoll_linux.c | 462 ++++++++++++++++------------ 1 file changed, 270 insertions(+), 192 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index ed2c494b783..72288889c02 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -140,18 +140,40 @@ static void fd_global_shutdown(void); #define CLOSURE_READY ((grpc_closure *)1) /******************************************************************************* - * Polling-island Declarations + * Polling island Declarations */ -/* TODO: sree: Consider making ref_cnt and merged_to to gpr_atm - This would - * significantly reduce the number of mutex acquisition calls. */ + +// #define GRPC_PI_REF_COUNT_DEBUG +#ifdef GRPC_PI_REF_COUNT_DEBUG + +#define PI_ADD_REF(p, r) pi_add_ref_dbg((p), 1, (r), __FILE__, __LINE__) +#define PI_UNREF(p, r) pi_unref_dbg((p), 1, (r), __FILE__, __LINE__) + +#else /* defined(GRPC_PI_REF_COUNT_DEBUG) */ + +#define PI_ADD_REF(p, r) pi_add_ref((p), 1) +#define PI_UNREF(p, r) pi_unref((p), 1) + +#endif /* !defined(GPRC_PI_REF_COUNT_DEBUG) */ + typedef struct polling_island { gpr_mu mu; - int ref_cnt; - - /* Points to the polling_island this merged into. - * If merged_to is not NULL, all the remaining fields (except mu and ref_cnt) - * are invalid and must be ignored */ - struct polling_island *merged_to; + /* Ref count. Use PI_ADD_REF() and PI_UNREF() macros to increment/decrement + the refcount. + Once the ref count becomes zero, this structure is destroyed which means + we should ensure that there is never a scenario where a PI_ADD_REF() is + racing with a PI_UNREF() that just made the ref_count zero. */ + gpr_atm ref_count; + + /* Pointer to the polling_island this merged into. + * merged_to value is only set once in polling_island's lifetime (and that too + * only if the island is merged with another island). Because of this, we can + * use gpr_atm type here so that we can do atomic access on this and reduce + * lock contention on 'mu' mutex. + * + * Note that if this field is not NULL (i.e not 0), all the remaining fields + * (except mu and ref_count) are invalid and must be ignored. */ + gpr_atm merged_to; /* The fd of the underlying epoll set */ int epoll_fd; @@ -236,6 +258,8 @@ static grpc_wakeup_fd polling_island_wakeup_fd; static gpr_mu g_pi_freelist_mu; static polling_island *g_pi_freelist = NULL; +static void polling_island_delete(); /* Forward declaration */ + #ifdef GRPC_TSAN /* Currently TSAN may incorrectly flag data races between epoll_ctl and epoll_wait for any grpc_fd structs that are added to the epoll set via @@ -247,6 +271,51 @@ static polling_island *g_pi_freelist = NULL; gpr_atm g_epoll_sync; #endif /* defined(GRPC_TSAN) */ +#ifdef GRPC_PI_REF_COUNT_DEBUG +long pi_add_ref(polling_island *pi, int ref_cnt); +long pi_unref(polling_island *pi, int ref_cnt); + +void pi_add_ref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, + int line) { + long old_cnt = pi_add_ref(pi, ref_cnt); + gpr_log(GPR_DEBUG, "Add ref pi: %p, old:%ld -> new:%ld (%s) - (%s, %d)", + (void *)pi, old_cnt, (old_cnt + ref_cnt), reason, file, line); +} + +void pi_unref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, + int line) { + long old_cnt = pi_unref(pi, ref_cnt); + gpr_log(GPR_DEBUG, "Unref pi: %p, old:%ld -> new:%ld (%s) - (%s, %d)", + (void *)pi, old_cnt, (old_cnt - ref_cnt), reason, file, line); +} +#endif + +long pi_add_ref(polling_island *pi, int ref_cnt) { + return gpr_atm_no_barrier_fetch_add(&pi->ref_count, ref_cnt); +} + +long pi_unref(polling_island *pi, int ref_cnt) { + long old_cnt = gpr_atm_no_barrier_fetch_add(&pi->ref_count, -ref_cnt); + + /* If ref count went to zero, delete the polling island. Note that this need + not be done under a lock. Once the ref count goes to zero, we are + guaranteed that no one else holds a reference to the polling island (and + that there is no racing pi_add_ref() call either. + + Also, if we are deleting the polling island and the merged_to field is + non-empty, we should remove a ref to the merged_to polling island + */ + if (old_cnt == ref_cnt) { + polling_island *next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + polling_island_delete(pi); + if (next != NULL) { + PI_UNREF(next, "pi_delete"); /* Recursive call */ + } + } + + return old_cnt; +} + /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, size_t fd_count, bool add_fd_refs) { @@ -355,8 +424,7 @@ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, } } -static polling_island *polling_island_create(grpc_fd *initial_fd, - int initial_ref_cnt) { +static polling_island *polling_island_create(grpc_fd *initial_fd) { polling_island *pi = NULL; /* Try to get one from the polling island freelist */ @@ -377,6 +445,9 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, pi->fds = NULL; } + gpr_atm_no_barrier_store(&pi->ref_count, 0); + gpr_atm_no_barrier_store(&pi->merged_to, NULL); + pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); if (pi->epoll_fd < 0) { @@ -387,14 +458,12 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, polling_island_add_wakeup_fd_locked(pi, &grpc_global_wakeup_fd); - pi->ref_cnt = initial_ref_cnt; - pi->merged_to = NULL; pi->next_free = NULL; if (initial_fd != NULL) { - /* It is not really needed to get the pi->mu lock here. If this is a newly - created polling island (or one that we got from the freelist), no one - else would be holding a lock to it anyway */ + /* Lock the polling island here just in case we got this structure from the + freelist and the polling island lock was not released yet (by the code + that adds the polling island to the freelist) */ gpr_mu_lock(&pi->mu); polling_island_add_fds_locked(pi, &initial_fd, 1, true); gpr_mu_unlock(&pi->mu); @@ -404,140 +473,136 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, } static void polling_island_delete(polling_island *pi) { - GPR_ASSERT(pi->ref_cnt == 0); GPR_ASSERT(pi->fd_cnt == 0); + gpr_atm_rel_store(&pi->merged_to, NULL); + close(pi->epoll_fd); pi->epoll_fd = -1; - pi->merged_to = NULL; - gpr_mu_lock(&g_pi_freelist_mu); pi->next_free = g_pi_freelist; g_pi_freelist = pi; gpr_mu_unlock(&g_pi_freelist_mu); } -void polling_island_unref_and_unlock(polling_island *pi, int unref_by) { - pi->ref_cnt -= unref_by; - int ref_cnt = pi->ref_cnt; - GPR_ASSERT(ref_cnt >= 0); - - gpr_mu_unlock(&pi->mu); - - if (ref_cnt == 0) { - polling_island_delete(pi); - } -} - -polling_island *polling_island_update_and_lock(polling_island *pi, int unref_by, - int add_ref_by) { +/* Gets the lock on the *latest* polling island i.e the last polling island in + the linked list (linked by 'merged_to' link). Call gpr_mu_unlock on the + returned polling island's mu. + Usage: To lock/unlock polling island "pi", do the following: + polling_island *pi_latest = polling_island_lock(pi); + ... + ... critical section .. + ... + gpr_mu_unlock(&pi_latest->mu); //NOTE: use pi_latest->mu. NOT pi->mu */ +polling_island *polling_island_lock(polling_island *pi) { polling_island *next = NULL; - gpr_mu_lock(&pi->mu); - while (pi->merged_to != NULL) { - next = pi->merged_to; - polling_island_unref_and_unlock(pi, unref_by); + while (true) { + next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + if (next == NULL) { + /* pi is the last node in the linked list. Get the lock and check again + (under the pi->mu lock) that pi is still the last node (because a merge + may have happend after the (next == NULL) check above and before + getting the pi->mu lock. + If pi is the last node, we are done. If not, unlock and continue + traversing the list */ + gpr_mu_lock(&pi->mu); + next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + if (next == NULL) { + break; + } + gpr_mu_unlock(&pi->mu); + } + pi = next; - gpr_mu_lock(&pi->mu); } - pi->ref_cnt += add_ref_by; return pi; } -void polling_island_pair_update_and_lock(polling_island **p, - polling_island **q) { +/* Gets the lock on the *latest* polling islands pointed by *p and *q. + This function is needed because calling the following block of code to obtain + locks on polling islands (*p and *q) is prone to deadlocks. + { + polling_island_lock(*p); + polling_island_lock(*q); + } + + Usage/exmaple: + polling_island *p1; + polling_island *p2; + .. + polling_island_lock_pair(&p1, &p2); + .. + .. Critical section with both p1 and p2 locked + .. + // Release locks + // **IMPORTANT**: Make sure you check p1 == p2 AFTER the function + // polling_island_lock_pair() was called and if so, release the lock only + // once. Note: Even if p1 != p2 beforec calling polling_island_lock_pair(), + // they might be after the function returns: + if (p1 == p2) { + gpr_mu_unlock(&p1->mu) + } else { + gpr_mu_unlock(&p1->mu); + gpr_mu_unlock(&p2->mu); + } + +*/ +void polling_island_lock_pair(polling_island **p, polling_island **q) { polling_island *pi_1 = *p; polling_island *pi_2 = *q; - polling_island *temp = NULL; - bool pi_1_locked = false; - bool pi_2_locked = false; - int num_swaps = 0; - - /* Loop until either pi_1 == pi_2 or until we acquired locks on both pi_1 - and pi_2 */ - while (pi_1 != pi_2 && !(pi_1_locked && pi_2_locked)) { - /* The following assertions are true at this point: - - pi_1 != pi_2 (else, the while loop would have exited) - - pi_1 MAY be locked - - pi_2 is NOT locked */ - - /* To maintain lock order consistency, always lock polling_island node with - lower address first. - First, make sure pi_1 < pi_2 before proceeding any further. If it turns - out that pi_1 > pi_2, unlock pi_1 if locked (because pi_2 is not locked - at this point and having pi_1 locked would violate the lock order) and - swap pi_1 and pi_2 so that pi_1 becomes less than pi_2 */ - if (pi_1 > pi_2) { - if (pi_1_locked) { - gpr_mu_unlock(&pi_1->mu); - pi_1_locked = false; - } + polling_island *next_1 = NULL; + polling_island *next_2 = NULL; + + /* The algorithm is simple: + - Go to the last polling islands in the linked lists *pi_1 and *pi_2 (and + keep updating pi_1 and pi_2) + - Then obtain locks on the islands by following a lock order rule of + locking polling_island with lower address first + Special case: Before obtaining the locks, check if pi_1 and pi_2 are + pointing to the same island. If that is the case, we can just call + polling_island_lock() + - After obtaining both the locks, double check that the polling islands + are still the last polling islands in their respective linked lists + (this is because there might have been polling island merges before + we got the lock) + - If the polling islands are the last islands, we are done. If not, + release the locks and continue the process from the first step */ + while (true) { + next_1 = (polling_island *)gpr_atm_acq_load(&pi_1->merged_to); + while (next_1 != NULL) { + pi_1 = next_1; + next_1 = (polling_island *)gpr_atm_acq_load(&pi_1->merged_to); + } - GPR_SWAP(polling_island *, pi_1, pi_2); - num_swaps++; + next_2 = (polling_island *)gpr_atm_acq_load(&pi_2->merged_to); + while (next_2 != NULL) { + pi_2 = next_2; + next_2 = (polling_island *)gpr_atm_acq_load(&pi_2->merged_to); } - /* The following assertions are true at this point: - - pi_1 != pi_2 - - pi_1 < pi_2 (address of pi_1 is less than that of pi_2) - - pi_1 MAYBE locked - - pi_2 is NOT locked */ + if (pi_1 == pi_2) { + pi_1 = pi_2 = polling_island_lock(pi_1); + break; + } - /* Lock pi_1 (if pi_1 is pointing to the terminal node in the list) */ - if (!pi_1_locked) { + if (pi_1 < pi_2) { + gpr_mu_lock(&pi_1->mu); + gpr_mu_lock(&pi_2->mu); + } else { + gpr_mu_lock(&pi_2->mu); gpr_mu_lock(&pi_1->mu); - pi_1_locked = true; - - /* If pi_1 is not terminal node (i.e pi_1->merged_to != NULL), we are not - done locking this polling_island yet. Release the lock on this node and - advance pi_1 to the next node in the list; and go to the beginning of - the loop (we can't proceed to locking pi_2 unless we locked pi_1 first) - */ - if (pi_1->merged_to != NULL) { - temp = pi_1->merged_to; - polling_island_unref_and_unlock(pi_1, 1); - pi_1 = temp; - pi_1_locked = false; - - continue; - } } - /* The following assertions are true at this point: - - pi_1 is locked - - pi_2 is unlocked - - pi_1 != pi_2 */ - - gpr_mu_lock(&pi_2->mu); - pi_2_locked = true; - - /* If pi_2 is not terminal node, we are not done locking this polling_island - yet. Release the lock and update pi_2 to the next node in the list */ - if (pi_2->merged_to != NULL) { - temp = pi_2->merged_to; - polling_island_unref_and_unlock(pi_2, 1); - pi_2 = temp; - pi_2_locked = false; + next_1 = (polling_island *)gpr_atm_acq_load(&pi_1->merged_to); + next_2 = (polling_island *)gpr_atm_acq_load(&pi_2->merged_to); + if (next_1 == NULL && next_2 == NULL) { + break; } - } - /* At this point, either pi_1 == pi_2 AND/OR we got both locks */ - if (pi_1 == pi_2) { - /* We may or may not have gotten the lock. If we didn't, walk the rest of - the polling_island list and get the lock */ - GPR_ASSERT(pi_1_locked || (!pi_1_locked && !pi_2_locked)); - if (!pi_1_locked) { - pi_1 = pi_2 = polling_island_update_and_lock(pi_1, 2, 0); - } - } else { - GPR_ASSERT(pi_1_locked && pi_2_locked); - /* If we swapped pi_1 and pi_2 odd number of times, do one more swap so that - pi_1 and pi_2 point to the same polling_island lists they started off - with at the beginning of this function (i.e *p and *q respectively) */ - if (num_swaps % 2 > 0) { - GPR_SWAP(polling_island *, pi_1, pi_2); - } + gpr_mu_unlock(&pi_1->mu); + gpr_mu_unlock(&pi_2->mu); } *p = pi_1; @@ -546,7 +611,7 @@ void polling_island_pair_update_and_lock(polling_island **p, polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Get locks on both the polling islands */ - polling_island_pair_update_and_lock(&p, &q); + polling_island_lock_pair(&p, &q); if (p == q) { /* Nothing needs to be done here */ @@ -568,15 +633,14 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { /* Wakeup all the pollers (if any) on p so that they can pickup this change */ polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); - p->merged_to = q; + /* Add the 'merged_to' link from p --> q */ + gpr_atm_rel_store(&p->merged_to, q); + PI_ADD_REF(q, "pi_merge"); /* To account for the new incoming ref from p */ - /* - The merged polling island (i.e q) inherits all the ref counts of the - island merging with it (i.e p) - - The island p will lose a ref count */ - q->ref_cnt += p->ref_cnt; - polling_island_unref_and_unlock(p, 1); /* Decrement refcount */ - polling_island_unref_and_unlock(q, 0); /* Just Unlock. Don't decrement ref */ + gpr_mu_unlock(&p->mu); + gpr_mu_unlock(&q->mu); + /* Return the merged polling island */ return q; } @@ -667,6 +731,7 @@ static void unref_by(grpc_fd *fd, int n) { fd->freelist_next = fd_freelist; fd_freelist = fd; grpc_iomgr_unregister_object(&fd->iomgr_object); + gpr_mu_unlock(&fd_freelist_mu); } else { GPR_ASSERT(old > n); @@ -785,16 +850,20 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, REF_BY(fd, 1, reason); /* Remove the fd from the polling island: - - Update the fd->polling_island to point to the latest polling island - - Remove the fd from the polling island. - - Remove a ref to the polling island and set fd->polling_island to NULL */ + - Get a lock on the latest polling island (i.e the last island in the + linked list pointed by fd->polling_island). This is the island that + would actually contain the fd + - Remove the fd from the latest polling island + - Unlock the latest polling island + - Set fd->polling_island to NULL (but remove the ref on the polling island + before doing this.) */ gpr_mu_lock(&fd->pi_mu); if (fd->polling_island != NULL) { - fd->polling_island = - polling_island_update_and_lock(fd->polling_island, 1, 0); - polling_island_remove_fd_locked(fd->polling_island, fd, is_fd_closed); + polling_island *pi_latest = polling_island_lock(fd->polling_island); + polling_island_remove_fd_locked(pi_latest, fd, is_fd_closed); + gpr_mu_unlock(&pi_latest->mu); - polling_island_unref_and_unlock(fd->polling_island, 1); + PI_UNREF(fd->polling_island, "fd_orphan"); fd->polling_island = NULL; } gpr_mu_unlock(&fd->pi_mu); @@ -1050,17 +1119,13 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { gpr_mu_unlock(&fd->mu); } -/* Release the reference to pollset->polling_island and set it to NULL. - pollset->mu must be held */ -static void pollset_release_polling_island_locked(grpc_pollset *pollset) { - gpr_mu_lock(&pollset->pi_mu); - if (pollset->polling_island) { - pollset->polling_island = - polling_island_update_and_lock(pollset->polling_island, 1, 0); - polling_island_unref_and_unlock(pollset->polling_island, 1); - pollset->polling_island = NULL; +static void pollset_release_polling_island(grpc_pollset *ps, char *reason) { + gpr_mu_lock(&ps->pi_mu); + if (ps->polling_island != NULL) { + PI_UNREF(ps->polling_island, reason); } - gpr_mu_unlock(&pollset->pi_mu); + ps->polling_island = NULL; + gpr_mu_unlock(&ps->pi_mu); } static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, @@ -1069,8 +1134,9 @@ static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, GPR_ASSERT(!pollset_has_workers(pollset)); pollset->finish_shutdown_called = true; - pollset_release_polling_island_locked(pollset); + /* Release the ref and set pollset->polling_island to NULL */ + pollset_release_polling_island(pollset, "ps_shutdown"); grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); } @@ -1110,7 +1176,7 @@ static void pollset_reset(grpc_pollset *pollset) { pollset->finish_shutdown_called = false; pollset->kicked_without_pollers = false; pollset->shutdown_done = NULL; - pollset_release_polling_island_locked(pollset); + pollset_release_polling_island(pollset, "ps_reset"); } #define GRPC_EPOLL_MAX_EVENTS 1000 @@ -1124,28 +1190,37 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the - polling island pointed by pollset->polling_island. + latest polling island pointed by pollset->polling_island. Acquire the following locks: - pollset->mu (which we already have) - pollset->pi_mu - - pollset->polling_island->mu (call polling_island_update_and_lock())*/ + - pollset->polling_island lock */ gpr_mu_lock(&pollset->pi_mu); - pi = pollset->polling_island; - if (pi == NULL) { - pi = polling_island_create(NULL, 1); + if (pollset->polling_island == NULL) { + pollset->polling_island = polling_island_create(NULL); + PI_ADD_REF(pollset->polling_island, "ps"); } - /* In addition to locking the polling island, add a ref so that the island - does not get destroyed (which means the epoll_fd won't be closed) while - we are are doing an epoll_wait() on the epoll_fd */ - pi = polling_island_update_and_lock(pi, 1, 1); + pi = polling_island_lock(pollset->polling_island); epoll_fd = pi->epoll_fd; - /* Update the pollset->polling_island */ - pollset->polling_island = pi; + /* Update the pollset->polling_island since the island being pointed by + pollset->polling_island may not be the latest (i.e pi) */ + if (pollset->polling_island != pi) { + /* Always do PI_ADD_REF before PI_UNREF because PI_UNREF may cause the + polling island to be deleted */ + PI_ADD_REF(pi, "ps"); + PI_UNREF(pollset->polling_island, "ps"); + pollset->polling_island = pi; + } + + /* Add an extra ref so that the island does not get destroyed (which means + the epoll_fd won't be closed) while we are are doing an epoll_wait() on the + epoll_fd */ + PI_ADD_REF(pi, "ps_work"); - polling_island_unref_and_unlock(pollset->polling_island, 0); /* Keep the ref*/ + gpr_mu_unlock(&pi->mu); gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); @@ -1193,14 +1268,12 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, GPR_ASSERT(pi != NULL); - /* Before leaving, release the extra ref we added to the polling island */ - /* It is important to note that at this point 'pi' may not be the same as - * pollset->polling_island. This is because pollset->polling_island pointer - * gets updated whenever the underlying polling island is merged with another - * island and while we are doing epoll_wait() above, the polling island may - * have been merged */ - pi = polling_island_update_and_lock(pi, 1, 0); /* No new ref added */ - polling_island_unref_and_unlock(pi, 1); + /* Before leaving, release the extra ref we added to the polling island. It + is important to use "pi" here (i.e our old copy of pollset->polling_island + that we got before releasing the polling island lock). This is because + pollset->polling_island pointer might get udpated in other parts of the + code when there is an island merge while we are doing epoll_wait() above */ + PI_UNREF(pi, "ps_work"); GPR_TIMER_END("pollset_work_and_unlock", 0); } @@ -1297,20 +1370,34 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, if (fd->polling_island == pollset->polling_island) { pi_new = fd->polling_island; if (pi_new == NULL) { - pi_new = polling_island_create(fd, 2); + pi_new = polling_island_create(fd); } } else if (fd->polling_island == NULL) { - pi_new = polling_island_update_and_lock(pollset->polling_island, 1, 1); - polling_island_add_fds_locked(pollset->polling_island, &fd, 1, true); + pi_new = polling_island_lock(pollset->polling_island); + polling_island_add_fds_locked(pi_new, &fd, 1, true); gpr_mu_unlock(&pi_new->mu); } else if (pollset->polling_island == NULL) { - pi_new = polling_island_update_and_lock(fd->polling_island, 1, 1); + pi_new = polling_island_lock(fd->polling_island); gpr_mu_unlock(&pi_new->mu); } else { pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); } - fd->polling_island = pollset->polling_island = pi_new; + if (fd->polling_island != pi_new) { + PI_ADD_REF(pi_new, "fd"); + if (fd->polling_island != NULL) { + PI_UNREF(fd->polling_island, "fd"); + } + fd->polling_island = pi_new; + } + + if (pollset->polling_island != pi_new) { + PI_ADD_REF(pi_new, "ps"); + if (pollset->polling_island != NULL) { + PI_UNREF(pollset->polling_island, "ps"); + } + pollset->polling_island = pi_new; + } gpr_mu_unlock(&fd->pi_mu); gpr_mu_unlock(&pollset->pi_mu); @@ -1481,28 +1568,19 @@ void *grpc_pollset_get_polling_island(grpc_pollset *ps) { return pi; } -static polling_island *get_polling_island(polling_island *p) { - if (p == NULL) { - return NULL; - } +bool grpc_are_polling_islands_equal(void *p, void *q) { + polling_island *p1 = p; + polling_island *p2 = q; - polling_island *next; - gpr_mu_lock(&p->mu); - while (p->merged_to != NULL) { - next = p->merged_to; - gpr_mu_unlock(&p->mu); - p = next; - gpr_mu_lock(&p->mu); + polling_island_lock_pair(&p1, &p2); + if (p1 == p2) { + gpr_mu_unlock(&p1->mu); + } else { + gpr_mu_unlock(&p1->mu); + gpr_mu_unlock(&p2->mu); } - gpr_mu_unlock(&p->mu); - - return p; -} -bool grpc_are_polling_islands_equal(void *p, void *q) { - p = get_polling_island(p); - q = get_polling_island(q); - return p == q; + return p1 == p2; } /******************************************************************************* From 65c6c59bcddd2847eb26eb7518747ebeea839d0b Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 21 Jun 2016 08:27:07 -0700 Subject: [PATCH 121/280] Fix refcounting tsan failures and grab pollset lock in the function pollset_add_fd --- src/core/lib/iomgr/ev_epoll_linux.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 72288889c02..7cc69c876db 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -291,11 +291,11 @@ void pi_unref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, #endif long pi_add_ref(polling_island *pi, int ref_cnt) { - return gpr_atm_no_barrier_fetch_add(&pi->ref_count, ref_cnt); + return gpr_atm_full_fetch_add(&pi->ref_count, ref_cnt); } long pi_unref(polling_island *pi, int ref_cnt) { - long old_cnt = gpr_atm_no_barrier_fetch_add(&pi->ref_count, -ref_cnt); + long old_cnt = gpr_atm_full_fetch_add(&pi->ref_count, -ref_cnt); /* If ref count went to zero, delete the polling island. Note that this need not be done under a lock. Once the ref count goes to zero, we are @@ -311,6 +311,8 @@ long pi_unref(polling_island *pi, int ref_cnt) { if (next != NULL) { PI_UNREF(next, "pi_delete"); /* Recursive call */ } + } else { + GPR_ASSERT(old_cnt > ref_cnt); } return old_cnt; @@ -445,8 +447,8 @@ static polling_island *polling_island_create(grpc_fd *initial_fd) { pi->fds = NULL; } - gpr_atm_no_barrier_store(&pi->ref_count, 0); - gpr_atm_no_barrier_store(&pi->merged_to, NULL); + gpr_atm_rel_store(&pi->ref_count, 0); + gpr_atm_rel_store(&pi->merged_to, NULL); pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); @@ -1347,7 +1349,7 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { - /* TODO sreek - Double check if we need to get a pollset->mu lock here */ + gpr_mu_lock(&pollset->mu); gpr_mu_lock(&pollset->pi_mu); gpr_mu_lock(&fd->pi_mu); @@ -1401,6 +1403,7 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, gpr_mu_unlock(&fd->pi_mu); gpr_mu_unlock(&pollset->pi_mu); + gpr_mu_unlock(&pollset->mu); } /******************************************************************************* From d263b925e8b8ce36042f34eda34715696b6eef3e Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Tue, 21 Jun 2016 09:15:57 -0700 Subject: [PATCH 122/280] Make sure to poll cq --- test/cpp/end2end/server_builder_plugin_test.cc | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/test/cpp/end2end/server_builder_plugin_test.cc b/test/cpp/end2end/server_builder_plugin_test.cc index 75f23b64a73..7d0b467d816 100644 --- a/test/cpp/end2end/server_builder_plugin_test.cc +++ b/test/cpp/end2end/server_builder_plugin_test.cc @@ -37,6 +37,7 @@ #include #include #include +#include #include #include #include @@ -188,6 +189,7 @@ class ServerBuilderPluginTest : public ::testing::TestWithParam { grpc::string server_address = "localhost:" + to_string(port_); builder_->AddListeningPort(server_address, InsecureServerCredentials()); cq_ = builder_->AddCompletionQueue(); + cq_thread_ = grpc::thread(std::bind(&ServerBuilderPluginTest::RunCQ, this)); server_ = builder_->BuildAndStart(); EXPECT_TRUE(CheckPresent()); } @@ -204,11 +206,8 @@ class ServerBuilderPluginTest : public ::testing::TestWithParam { EXPECT_TRUE(plugin->init_server_is_called()); EXPECT_TRUE(plugin->finish_is_called()); server_->Shutdown(); - void* tag; - bool ok; cq_->Shutdown(); - while (cq_->Next(&tag, &ok)) - ; + cq_thread_.join(); } string to_string(const int number) { @@ -223,6 +222,7 @@ class ServerBuilderPluginTest : public ::testing::TestWithParam { std::unique_ptr stub_; std::unique_ptr cq_; std::unique_ptr server_; + grpc::thread cq_thread_; TestServiceImpl service_; int port_; @@ -238,6 +238,13 @@ class ServerBuilderPluginTest : public ::testing::TestWithParam { return nullptr; } } + + void RunCQ() { + void* tag; + bool ok; + while (cq_->Next(&tag, &ok)) + ; + } }; TEST_P(ServerBuilderPluginTest, PluginWithoutServiceTest) { From 04a468122f0072cbd8a67d66dfd8243ce3ddface Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Tue, 21 Jun 2016 09:16:57 -0700 Subject: [PATCH 123/280] Add comment --- test/cpp/end2end/server_builder_plugin_test.cc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/test/cpp/end2end/server_builder_plugin_test.cc b/test/cpp/end2end/server_builder_plugin_test.cc index 7d0b467d816..778a2be573e 100644 --- a/test/cpp/end2end/server_builder_plugin_test.cc +++ b/test/cpp/end2end/server_builder_plugin_test.cc @@ -188,6 +188,8 @@ class ServerBuilderPluginTest : public ::testing::TestWithParam { void StartServer() { grpc::string server_address = "localhost:" + to_string(port_); builder_->AddListeningPort(server_address, InsecureServerCredentials()); + // we run some tests without a service, and for those we need to supply a + // frequently polled completion queue cq_ = builder_->AddCompletionQueue(); cq_thread_ = grpc::thread(std::bind(&ServerBuilderPluginTest::RunCQ, this)); server_ = builder_->BuildAndStart(); From 939e9ca6d82f2558702e5fa859d16ca7052f746b Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Tue, 21 Jun 2016 08:00:42 -0700 Subject: [PATCH 124/280] use template to generate project.json files --- .../csharp/Grpc.Auth/project.json.template | 34 ++++++++++++++ .../Grpc.Core.Tests/project.json.template | 24 ++++++++++ .../csharp/Grpc.Core/project.json.template | 44 ++++++++++++++++++ .../project.json.template | 21 +++++++++ .../project.json.template | 21 +++++++++ .../Grpc.Examples.Tests/project.json.template | 23 ++++++++++ .../Grpc.Examples/project.json.template | 27 +++++++++++ .../project.json.template | 23 ++++++++++ .../Grpc.HealthCheck/project.json.template | 37 +++++++++++++++ .../project.json.template | 22 +++++++++ .../project.json.template | 22 +++++++++ .../project.json.template | 22 +++++++++ .../project.json.template | 22 +++++++++ .../project.json.template | 38 ++++++++++++++++ templates/src/csharp/build_options.include | 45 +++++++++++++++++++ 15 files changed, 425 insertions(+) create mode 100644 templates/src/csharp/Grpc.Auth/project.json.template create mode 100644 templates/src/csharp/Grpc.Core.Tests/project.json.template create mode 100644 templates/src/csharp/Grpc.Core/project.json.template create mode 100644 templates/src/csharp/Grpc.Examples.MathClient/project.json.template create mode 100644 templates/src/csharp/Grpc.Examples.MathServer/project.json.template create mode 100644 templates/src/csharp/Grpc.Examples.Tests/project.json.template create mode 100644 templates/src/csharp/Grpc.Examples/project.json.template create mode 100644 templates/src/csharp/Grpc.HealthCheck.Tests/project.json.template create mode 100644 templates/src/csharp/Grpc.HealthCheck/project.json.template create mode 100644 templates/src/csharp/Grpc.IntegrationTesting.Client/project.json.template create mode 100644 templates/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json.template create mode 100644 templates/src/csharp/Grpc.IntegrationTesting.Server/project.json.template create mode 100644 templates/src/csharp/Grpc.IntegrationTesting.StressClient/project.json.template create mode 100644 templates/src/csharp/Grpc.IntegrationTesting/project.json.template create mode 100644 templates/src/csharp/build_options.include diff --git a/templates/src/csharp/Grpc.Auth/project.json.template b/templates/src/csharp/Grpc.Auth/project.json.template new file mode 100644 index 00000000000..90ad0eb0891 --- /dev/null +++ b/templates/src/csharp/Grpc.Auth/project.json.template @@ -0,0 +1,34 @@ +%YAML 1.2 +--- | + { + "version": "${settings.csharp_version}", + "title": "gRPC C# Auth", + "authors": [ "Google Inc." ], + "copyright": "Copyright 2015, Google Inc.", + "packOptions": { + "summary": "Auth library for C# implementation of gRPC - an RPC library and framework", + "description": "Auth library for C# implementation of gRPC - an RPC library and framework. See project site for more info.", + "owners": [ "grpc-packages" ], + "licenseUrl": "https://github.com/grpc/grpc/blob/master/LICENSE", + "projectUrl": "https://github.com/grpc/grpc", + "requireLicenseAcceptance": false, + "tags": [ "gRPC RPC Protocol HTTP/2 Auth OAuth2" ], + }, + "dependencies": { + "Grpc.Core": "${settings.csharp_version}", + "Google.Apis.Auth": "1.11.1" + }, + "frameworks": { + "net45": { }, + "netstandard1.5": { + "imports": [ + "net45" + ], + "dependencies": { + "Microsoft.NETCore.Portable.Compatibility": "1.0.1-rc2-24027", + "NETStandard.Library": "1.5.0-rc2-24027", + "System.Threading.Tasks": "4.0.11-rc2-24027" + } + } + } + } diff --git a/templates/src/csharp/Grpc.Core.Tests/project.json.template b/templates/src/csharp/Grpc.Core.Tests/project.json.template new file mode 100644 index 00000000000..bc9fa3e63a9 --- /dev/null +++ b/templates/src/csharp/Grpc.Core.Tests/project.json.template @@ -0,0 +1,24 @@ +%YAML 1.2 +--- | + { + <%include file="../build_options.include" args="executable=True"/> + "dependencies": { + "Grpc.Core": { + "target": "project" + }, + "Newtonsoft.Json": "8.0.3", + "NUnit": "3.2.0", + "NUnitLite": "3.2.0-*" + }, + "frameworks": { + "net45": { }, + "netstandard1.5": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027" + } + } + }, + } diff --git a/templates/src/csharp/Grpc.Core/project.json.template b/templates/src/csharp/Grpc.Core/project.json.template new file mode 100644 index 00000000000..6f9197f572f --- /dev/null +++ b/templates/src/csharp/Grpc.Core/project.json.template @@ -0,0 +1,44 @@ +%YAML 1.2 +--- | + { + "version": "${settings.csharp_version}", + "title": "gRPC C# Core", + "authors": [ "Google Inc." ], + "copyright": "Copyright 2015, Google Inc.", + "packOptions": { + "summary": "Core C# implementation of gRPC - an RPC library and framework", + "description": "Core C# implementation of gRPC - an RPC library and framework. See project site for more info.", + "owners": [ "grpc-packages" ], + "licenseUrl": "https://github.com/grpc/grpc/blob/master/LICENSE", + "projectUrl": "https://github.com/grpc/grpc", + "requireLicenseAcceptance": false, + "tags": [ "gRPC RPC Protocol HTTP/2" ], + "files": { + "build/net45/": "Grpc.Core.targets", + "build/native/bin/windows_x86/": "../nativelibs/windows_x86/grpc_csharp_ext.dll", + "build/native/bin/windows_x64/": "../nativelibs/windows_x64/grpc_csharp_ext.dll", + "build/native/bin/linux_x86/": "../nativelibs/linux_x86/libgrpc_csharp_ext.so", + "build/native/bin/linux_x64/": "../nativelibs/linux_x64/libgrpc_csharp_ext.so", + "build/native/bin/macosx_x86/": "../nativelibs/macosx_x86/libgrpc_csharp_ext.dylib", + "build/native/bin/macosx_x64/": "../nativelibs/macosx_x64/libgrpc_csharp_ext.dylib" + } + }, + "buildOptions": { + "embed": [ "../../../etc/roots.pem" ] + }, + "dependencies": { + "Ix-Async": "1.2.5" + }, + "frameworks": { + "net45": { }, + "netstandard1.5": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027", + "System.Threading.Thread": "4.0.0-rc2-24027" + } + } + } + } diff --git a/templates/src/csharp/Grpc.Examples.MathClient/project.json.template b/templates/src/csharp/Grpc.Examples.MathClient/project.json.template new file mode 100644 index 00000000000..fba401c3a47 --- /dev/null +++ b/templates/src/csharp/Grpc.Examples.MathClient/project.json.template @@ -0,0 +1,21 @@ +%YAML 1.2 +--- | + { + <%include file="../build_options.include" args="executable=True"/> + "dependencies": { + "Grpc.Examples": { + "target": "project" + } + }, + "frameworks": { + "net45": { }, + "netstandard1.5": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027" + } + } + } + } diff --git a/templates/src/csharp/Grpc.Examples.MathServer/project.json.template b/templates/src/csharp/Grpc.Examples.MathServer/project.json.template new file mode 100644 index 00000000000..fba401c3a47 --- /dev/null +++ b/templates/src/csharp/Grpc.Examples.MathServer/project.json.template @@ -0,0 +1,21 @@ +%YAML 1.2 +--- | + { + <%include file="../build_options.include" args="executable=True"/> + "dependencies": { + "Grpc.Examples": { + "target": "project" + } + }, + "frameworks": { + "net45": { }, + "netstandard1.5": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027" + } + } + } + } diff --git a/templates/src/csharp/Grpc.Examples.Tests/project.json.template b/templates/src/csharp/Grpc.Examples.Tests/project.json.template new file mode 100644 index 00000000000..21765f0565c --- /dev/null +++ b/templates/src/csharp/Grpc.Examples.Tests/project.json.template @@ -0,0 +1,23 @@ +%YAML 1.2 +--- | + { + <%include file="../build_options.include" args="executable=True"/> + "dependencies": { + "Grpc.Examples": { + "target": "project" + }, + "NUnit": "3.2.0", + "NUnitLite": "3.2.0-*" + }, + "frameworks": { + "net45": { }, + "netstandard1.5": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027" + } + } + } + } diff --git a/templates/src/csharp/Grpc.Examples/project.json.template b/templates/src/csharp/Grpc.Examples/project.json.template new file mode 100644 index 00000000000..715fc087256 --- /dev/null +++ b/templates/src/csharp/Grpc.Examples/project.json.template @@ -0,0 +1,27 @@ +%YAML 1.2 +--- | + { + <%include file="../build_options.include" args="executable=False"/> + "dependencies": { + "Grpc.Core": { + "target": "project" + }, + "Google.Protobuf": "3.0.0-beta3" + }, + "frameworks": { + "net45": { + "frameworkAssemblies": { + "System.Runtime": "", + "System.IO": "" + } + }, + "netstandard1.5": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027" + } + } + } + } diff --git a/templates/src/csharp/Grpc.HealthCheck.Tests/project.json.template b/templates/src/csharp/Grpc.HealthCheck.Tests/project.json.template new file mode 100644 index 00000000000..79e67226cb4 --- /dev/null +++ b/templates/src/csharp/Grpc.HealthCheck.Tests/project.json.template @@ -0,0 +1,23 @@ +%YAML 1.2 +--- | + { + <%include file="../build_options.include" args="executable=True"/> + "dependencies": { + "Grpc.HealthCheck": { + "target": "project" + }, + "NUnit": "3.2.0", + "NUnitLite": "3.2.0-*" + }, + "frameworks": { + "net45": { }, + "netstandard1.5": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027" + } + } + } + } diff --git a/templates/src/csharp/Grpc.HealthCheck/project.json.template b/templates/src/csharp/Grpc.HealthCheck/project.json.template new file mode 100644 index 00000000000..59073af7eec --- /dev/null +++ b/templates/src/csharp/Grpc.HealthCheck/project.json.template @@ -0,0 +1,37 @@ +%YAML 1.2 +--- | + { + "version": "${settings.csharp_version}", + "title": "gRPC C# Healthchecking", + "authors": [ "Google Inc." ], + "copyright": "Copyright 2015, Google Inc.", + "packOptions": { + "summary": "Implementation of gRPC health service", + "description": "Example implementation of grpc.health.v1 service that can be used for health-checking.", + "owners": [ "grpc-packages" ], + "licenseUrl": "https://github.com/grpc/grpc/blob/master/LICENSE", + "projectUrl": "https://github.com/grpc/grpc", + "requireLicenseAcceptance": false, + "tags": [ "gRPC health check" ] + }, + "dependencies": { + "Grpc.Core": "${settings.csharp_version}", + "Google.Protobuf": "3.0.0-beta3" + }, + "frameworks": { + "net45": { + "frameworkAssemblies": { + "System.Runtime": "", + "System.IO": "" + } + }, + "netstandard1.5": { + "imports": [ + "portable-net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027" + } + } + } + } diff --git a/templates/src/csharp/Grpc.IntegrationTesting.Client/project.json.template b/templates/src/csharp/Grpc.IntegrationTesting.Client/project.json.template new file mode 100644 index 00000000000..10ed5493477 --- /dev/null +++ b/templates/src/csharp/Grpc.IntegrationTesting.Client/project.json.template @@ -0,0 +1,22 @@ +%YAML 1.2 +--- | + { + <%include file="../build_options.include" args="executable=True,includeData=True"/> + "dependencies": { + "Grpc.IntegrationTesting": { + "target": "project" + } + }, + "frameworks": { + "net45": { }, + "netstandard1.5": { + "imports": [ + "portable-net45", + "net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027" + } + } + } + } diff --git a/templates/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json.template b/templates/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json.template new file mode 100644 index 00000000000..10ed5493477 --- /dev/null +++ b/templates/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json.template @@ -0,0 +1,22 @@ +%YAML 1.2 +--- | + { + <%include file="../build_options.include" args="executable=True,includeData=True"/> + "dependencies": { + "Grpc.IntegrationTesting": { + "target": "project" + } + }, + "frameworks": { + "net45": { }, + "netstandard1.5": { + "imports": [ + "portable-net45", + "net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027" + } + } + } + } diff --git a/templates/src/csharp/Grpc.IntegrationTesting.Server/project.json.template b/templates/src/csharp/Grpc.IntegrationTesting.Server/project.json.template new file mode 100644 index 00000000000..10ed5493477 --- /dev/null +++ b/templates/src/csharp/Grpc.IntegrationTesting.Server/project.json.template @@ -0,0 +1,22 @@ +%YAML 1.2 +--- | + { + <%include file="../build_options.include" args="executable=True,includeData=True"/> + "dependencies": { + "Grpc.IntegrationTesting": { + "target": "project" + } + }, + "frameworks": { + "net45": { }, + "netstandard1.5": { + "imports": [ + "portable-net45", + "net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027" + } + } + } + } diff --git a/templates/src/csharp/Grpc.IntegrationTesting.StressClient/project.json.template b/templates/src/csharp/Grpc.IntegrationTesting.StressClient/project.json.template new file mode 100644 index 00000000000..10ed5493477 --- /dev/null +++ b/templates/src/csharp/Grpc.IntegrationTesting.StressClient/project.json.template @@ -0,0 +1,22 @@ +%YAML 1.2 +--- | + { + <%include file="../build_options.include" args="executable=True,includeData=True"/> + "dependencies": { + "Grpc.IntegrationTesting": { + "target": "project" + } + }, + "frameworks": { + "net45": { }, + "netstandard1.5": { + "imports": [ + "portable-net45", + "net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027" + } + } + } + } diff --git a/templates/src/csharp/Grpc.IntegrationTesting/project.json.template b/templates/src/csharp/Grpc.IntegrationTesting/project.json.template new file mode 100644 index 00000000000..31815114857 --- /dev/null +++ b/templates/src/csharp/Grpc.IntegrationTesting/project.json.template @@ -0,0 +1,38 @@ +%YAML 1.2 +--- | + { + <%include file="../build_options.include" args="executable=True,includeData=True"/> + "dependencies": { + "Grpc.Auth": { + "target": "project" + }, + "Grpc.Core": { + "target": "project" + }, + "Google.Protobuf": "3.0.0-beta3", + "CommandLineParser": "1.9.71", + "NUnit": "3.2.0", + "NUnitLite": "3.2.0-*" + }, + "frameworks": { + "net45": { + "dependencies": { + "Moq": "4.2.1510.2205" + }, + "frameworkAssemblies": { + "System.Runtime": "", + "System.IO": "" + } + }, + "netstandard1.5": { + "imports": [ + "portable-net45", + "net45" + ], + "dependencies": { + "NETStandard.Library": "1.5.0-rc2-24027", + "System.Linq.Expressions": "4.0.11-rc2-24027" + } + } + } + } diff --git a/templates/src/csharp/build_options.include b/templates/src/csharp/build_options.include new file mode 100644 index 00000000000..468d281618c --- /dev/null +++ b/templates/src/csharp/build_options.include @@ -0,0 +1,45 @@ +<%page args="executable=False,includeData=False"/>\ +"buildOptions": { + % if executable: + "emitEntryPoint": true + % endif + }, + % if executable: + "configurations": { + "Debug": { + "buildOptions": { + "copyToOutput": { + % if includeData: + "include": "data/*", + % endif + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/dbg/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/dbg/libgrpc_csharp_ext.dylib" + } + } + } + }, + "Release": { + "buildOptions": { + "copyToOutput": { + % if includeData: + "include": "data/*", + % endif + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Release/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Release/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/opt/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/opt/libgrpc_csharp_ext.dylib" + } + } + } + } + }, + "runtimes": { + "win7-x64": { }, + "debian.8-x64": { }, + "osx.10.11-x64": { } + }, + % endif \ No newline at end of file From 381e26a21eb0d23f55653da1f611ace55ee8b662 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Tue, 21 Jun 2016 08:37:22 -0700 Subject: [PATCH 125/280] regenerate project.json files --- src/csharp/Grpc.Auth/project.json | 4 +- src/csharp/Grpc.Core.Tests/project.json | 40 +++++++++++++---- src/csharp/Grpc.Core/project.json | 2 +- .../Grpc.Examples.MathClient/project.json | 40 +++++++++++++---- .../Grpc.Examples.MathServer/project.json | 40 +++++++++++++---- src/csharp/Grpc.Examples.Tests/project.json | 40 +++++++++++++---- src/csharp/Grpc.Examples/project.json | 3 ++ .../Grpc.HealthCheck.Tests/project.json | 40 +++++++++++++---- src/csharp/Grpc.HealthCheck/project.json | 4 +- .../project.json | 42 +++++++++++++---- .../project.json | 42 +++++++++++++---- .../project.json | 42 +++++++++++++---- .../project.json | 42 +++++++++++++---- .../Grpc.IntegrationTesting/project.json | 45 ++++++++++++++----- 14 files changed, 329 insertions(+), 97 deletions(-) diff --git a/src/csharp/Grpc.Auth/project.json b/src/csharp/Grpc.Auth/project.json index 57539f2976b..1677565824b 100644 --- a/src/csharp/Grpc.Auth/project.json +++ b/src/csharp/Grpc.Auth/project.json @@ -1,5 +1,5 @@ { - "version": "0.14.0-anexperiment", + "version": "0.15.0-dev", "title": "gRPC C# Auth", "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", @@ -13,7 +13,7 @@ "tags": [ "gRPC RPC Protocol HTTP/2 Auth OAuth2" ], }, "dependencies": { - "Grpc.Core": "0.14.0-anexperiment", + "Grpc.Core": "0.15.0-dev", "Google.Apis.Auth": "1.11.1" }, "frameworks": { diff --git a/src/csharp/Grpc.Core.Tests/project.json b/src/csharp/Grpc.Core.Tests/project.json index 95f40fdbbde..3ad081df39e 100644 --- a/src/csharp/Grpc.Core.Tests/project.json +++ b/src/csharp/Grpc.Core.Tests/project.json @@ -1,13 +1,39 @@ { "buildOptions": { - "copyToOutput": { - "mappings": { - "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", - "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + "emitEntryPoint": true + }, + "configurations": { + "Debug": { + "buildOptions": { + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/dbg/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/dbg/libgrpc_csharp_ext.dylib" + } + } } }, - "emitEntryPoint": true + "Release": { + "buildOptions": { + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Release/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Release/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/opt/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/opt/libgrpc_csharp_ext.dylib" + } + } + } + } }, + "runtimes": { + "win7-x64": { }, + "debian.8-x64": { }, + "osx.10.11-x64": { } + }, + "dependencies": { "Grpc.Core": { "target": "project" @@ -27,8 +53,4 @@ } } }, - "runtimes": { - "win7-x64": { }, - "debian.8-x64": {} - } } diff --git a/src/csharp/Grpc.Core/project.json b/src/csharp/Grpc.Core/project.json index ba3dc15495d..7253107e04a 100644 --- a/src/csharp/Grpc.Core/project.json +++ b/src/csharp/Grpc.Core/project.json @@ -1,5 +1,5 @@ { - "version": "0.14.0-anexperiment", + "version": "0.15.0-dev", "title": "gRPC C# Core", "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", diff --git a/src/csharp/Grpc.Examples.MathClient/project.json b/src/csharp/Grpc.Examples.MathClient/project.json index 4c1ea780412..b254f15af87 100644 --- a/src/csharp/Grpc.Examples.MathClient/project.json +++ b/src/csharp/Grpc.Examples.MathClient/project.json @@ -1,13 +1,39 @@ { "buildOptions": { - "copyToOutput": { - "mappings": { - "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", - "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + "emitEntryPoint": true + }, + "configurations": { + "Debug": { + "buildOptions": { + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/dbg/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/dbg/libgrpc_csharp_ext.dylib" + } + } } }, - "emitEntryPoint": true + "Release": { + "buildOptions": { + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Release/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Release/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/opt/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/opt/libgrpc_csharp_ext.dylib" + } + } + } + } }, + "runtimes": { + "win7-x64": { }, + "debian.8-x64": { }, + "osx.10.11-x64": { } + }, + "dependencies": { "Grpc.Examples": { "target": "project" @@ -23,9 +49,5 @@ "NETStandard.Library": "1.5.0-rc2-24027" } } - }, - "runtimes": { - "win7-x64": { }, - "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.Examples.MathServer/project.json b/src/csharp/Grpc.Examples.MathServer/project.json index 4c1ea780412..b254f15af87 100644 --- a/src/csharp/Grpc.Examples.MathServer/project.json +++ b/src/csharp/Grpc.Examples.MathServer/project.json @@ -1,13 +1,39 @@ { "buildOptions": { - "copyToOutput": { - "mappings": { - "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", - "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + "emitEntryPoint": true + }, + "configurations": { + "Debug": { + "buildOptions": { + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/dbg/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/dbg/libgrpc_csharp_ext.dylib" + } + } } }, - "emitEntryPoint": true + "Release": { + "buildOptions": { + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Release/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Release/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/opt/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/opt/libgrpc_csharp_ext.dylib" + } + } + } + } }, + "runtimes": { + "win7-x64": { }, + "debian.8-x64": { }, + "osx.10.11-x64": { } + }, + "dependencies": { "Grpc.Examples": { "target": "project" @@ -23,9 +49,5 @@ "NETStandard.Library": "1.5.0-rc2-24027" } } - }, - "runtimes": { - "win7-x64": { }, - "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.Examples.Tests/project.json b/src/csharp/Grpc.Examples.Tests/project.json index f41be82bd53..d2779e814f9 100644 --- a/src/csharp/Grpc.Examples.Tests/project.json +++ b/src/csharp/Grpc.Examples.Tests/project.json @@ -1,13 +1,39 @@ { "buildOptions": { - "copyToOutput": { - "mappings": { - "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", - "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + "emitEntryPoint": true + }, + "configurations": { + "Debug": { + "buildOptions": { + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/dbg/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/dbg/libgrpc_csharp_ext.dylib" + } + } } }, - "emitEntryPoint": true + "Release": { + "buildOptions": { + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Release/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Release/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/opt/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/opt/libgrpc_csharp_ext.dylib" + } + } + } + } }, + "runtimes": { + "win7-x64": { }, + "debian.8-x64": { }, + "osx.10.11-x64": { } + }, + "dependencies": { "Grpc.Examples": { "target": "project" @@ -25,9 +51,5 @@ "NETStandard.Library": "1.5.0-rc2-24027" } } - }, - "runtimes": { - "win7-x64": { }, - "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.Examples/project.json b/src/csharp/Grpc.Examples/project.json index fe580eb165b..7d3f4dcbb1e 100644 --- a/src/csharp/Grpc.Examples/project.json +++ b/src/csharp/Grpc.Examples/project.json @@ -1,4 +1,7 @@ { + "buildOptions": { + }, + "dependencies": { "Grpc.Core": { "target": "project" diff --git a/src/csharp/Grpc.HealthCheck.Tests/project.json b/src/csharp/Grpc.HealthCheck.Tests/project.json index 5a5f063258a..74599bd4b9e 100644 --- a/src/csharp/Grpc.HealthCheck.Tests/project.json +++ b/src/csharp/Grpc.HealthCheck.Tests/project.json @@ -1,13 +1,39 @@ { "buildOptions": { - "copyToOutput": { - "mappings": { - "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", - "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + "emitEntryPoint": true + }, + "configurations": { + "Debug": { + "buildOptions": { + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/dbg/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/dbg/libgrpc_csharp_ext.dylib" + } + } } }, - "emitEntryPoint": true + "Release": { + "buildOptions": { + "copyToOutput": { + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Release/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Release/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/opt/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/opt/libgrpc_csharp_ext.dylib" + } + } + } + } }, + "runtimes": { + "win7-x64": { }, + "debian.8-x64": { }, + "osx.10.11-x64": { } + }, + "dependencies": { "Grpc.HealthCheck": { "target": "project" @@ -25,9 +51,5 @@ "NETStandard.Library": "1.5.0-rc2-24027" } } - }, - "runtimes": { - "win7-x64": { }, - "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.HealthCheck/project.json b/src/csharp/Grpc.HealthCheck/project.json index e9fdfad4731..eb57608957a 100644 --- a/src/csharp/Grpc.HealthCheck/project.json +++ b/src/csharp/Grpc.HealthCheck/project.json @@ -1,5 +1,5 @@ { - "version": "0.14.0-anexperiment", + "version": "0.15.0-dev", "title": "gRPC C# Healthchecking", "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", @@ -13,7 +13,7 @@ "tags": [ "gRPC health check" ] }, "dependencies": { - "Grpc.Core": "0.14.0-anexperiment", + "Grpc.Core": "0.15.0-dev", "Google.Protobuf": "3.0.0-beta3" }, "frameworks": { diff --git a/src/csharp/Grpc.IntegrationTesting.Client/project.json b/src/csharp/Grpc.IntegrationTesting.Client/project.json index b19b76c6ccf..e5ba04d7173 100644 --- a/src/csharp/Grpc.IntegrationTesting.Client/project.json +++ b/src/csharp/Grpc.IntegrationTesting.Client/project.json @@ -1,13 +1,41 @@ { "buildOptions": { - "copyToOutput": { - "mappings": { - "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", - "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + "emitEntryPoint": true + }, + "configurations": { + "Debug": { + "buildOptions": { + "copyToOutput": { + "include": "data/*", + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/dbg/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/dbg/libgrpc_csharp_ext.dylib" + } + } } }, - "emitEntryPoint": true + "Release": { + "buildOptions": { + "copyToOutput": { + "include": "data/*", + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Release/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Release/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/opt/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/opt/libgrpc_csharp_ext.dylib" + } + } + } + } }, + "runtimes": { + "win7-x64": { }, + "debian.8-x64": { }, + "osx.10.11-x64": { } + }, + "dependencies": { "Grpc.IntegrationTesting": { "target": "project" @@ -24,9 +52,5 @@ "NETStandard.Library": "1.5.0-rc2-24027" } } - }, - "runtimes": { - "win7-x64": { }, - "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json b/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json index b19b76c6ccf..e5ba04d7173 100644 --- a/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json +++ b/src/csharp/Grpc.IntegrationTesting.QpsWorker/project.json @@ -1,13 +1,41 @@ { "buildOptions": { - "copyToOutput": { - "mappings": { - "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", - "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + "emitEntryPoint": true + }, + "configurations": { + "Debug": { + "buildOptions": { + "copyToOutput": { + "include": "data/*", + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/dbg/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/dbg/libgrpc_csharp_ext.dylib" + } + } } }, - "emitEntryPoint": true + "Release": { + "buildOptions": { + "copyToOutput": { + "include": "data/*", + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Release/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Release/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/opt/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/opt/libgrpc_csharp_ext.dylib" + } + } + } + } }, + "runtimes": { + "win7-x64": { }, + "debian.8-x64": { }, + "osx.10.11-x64": { } + }, + "dependencies": { "Grpc.IntegrationTesting": { "target": "project" @@ -24,9 +52,5 @@ "NETStandard.Library": "1.5.0-rc2-24027" } } - }, - "runtimes": { - "win7-x64": { }, - "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.IntegrationTesting.Server/project.json b/src/csharp/Grpc.IntegrationTesting.Server/project.json index b19b76c6ccf..e5ba04d7173 100644 --- a/src/csharp/Grpc.IntegrationTesting.Server/project.json +++ b/src/csharp/Grpc.IntegrationTesting.Server/project.json @@ -1,13 +1,41 @@ { "buildOptions": { - "copyToOutput": { - "mappings": { - "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", - "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + "emitEntryPoint": true + }, + "configurations": { + "Debug": { + "buildOptions": { + "copyToOutput": { + "include": "data/*", + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/dbg/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/dbg/libgrpc_csharp_ext.dylib" + } + } } }, - "emitEntryPoint": true + "Release": { + "buildOptions": { + "copyToOutput": { + "include": "data/*", + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Release/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Release/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/opt/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/opt/libgrpc_csharp_ext.dylib" + } + } + } + } }, + "runtimes": { + "win7-x64": { }, + "debian.8-x64": { }, + "osx.10.11-x64": { } + }, + "dependencies": { "Grpc.IntegrationTesting": { "target": "project" @@ -24,9 +52,5 @@ "NETStandard.Library": "1.5.0-rc2-24027" } } - }, - "runtimes": { - "win7-x64": { }, - "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.IntegrationTesting.StressClient/project.json b/src/csharp/Grpc.IntegrationTesting.StressClient/project.json index b19b76c6ccf..e5ba04d7173 100644 --- a/src/csharp/Grpc.IntegrationTesting.StressClient/project.json +++ b/src/csharp/Grpc.IntegrationTesting.StressClient/project.json @@ -1,13 +1,41 @@ { "buildOptions": { - "copyToOutput": { - "mappings": { - "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", - "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" + "emitEntryPoint": true + }, + "configurations": { + "Debug": { + "buildOptions": { + "copyToOutput": { + "include": "data/*", + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/dbg/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/dbg/libgrpc_csharp_ext.dylib" + } + } } }, - "emitEntryPoint": true + "Release": { + "buildOptions": { + "copyToOutput": { + "include": "data/*", + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Release/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Release/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/opt/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/opt/libgrpc_csharp_ext.dylib" + } + } + } + } }, + "runtimes": { + "win7-x64": { }, + "debian.8-x64": { }, + "osx.10.11-x64": { } + }, + "dependencies": { "Grpc.IntegrationTesting": { "target": "project" @@ -24,9 +52,5 @@ "NETStandard.Library": "1.5.0-rc2-24027" } } - }, - "runtimes": { - "win7-x64": { }, - "debian.8-x64": {} } } diff --git a/src/csharp/Grpc.IntegrationTesting/project.json b/src/csharp/Grpc.IntegrationTesting/project.json index be857279892..3493ab0c228 100644 --- a/src/csharp/Grpc.IntegrationTesting/project.json +++ b/src/csharp/Grpc.IntegrationTesting/project.json @@ -1,14 +1,41 @@ { "buildOptions": { - "copyToOutput": { - "mappings": { - "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", - "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll" - }, - "include": "data/*" - }, "emitEntryPoint": true }, + "configurations": { + "Debug": { + "buildOptions": { + "copyToOutput": { + "include": "data/*", + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Debug/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Debug/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/dbg/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/dbg/libgrpc_csharp_ext.dylib" + } + } + } + }, + "Release": { + "buildOptions": { + "copyToOutput": { + "include": "data/*", + "mappings": { + "nativelibs/windows_x64/grpc_csharp_ext.dll": "../../../vsprojects/x64/Release/grpc_csharp_ext.dll", + "nativelibs/windows_x86/grpc_csharp_ext.dll": "../../../vsprojects/Release/grpc_csharp_ext.dll", + "nativelibs/linux_x64/libgrpc_csharp_ext.so": "../../../libs/opt/libgrpc_csharp_ext.so", + "nativelibs/macosx_x64/libgrpc_csharp_ext.dylib": "../../../libs/opt/libgrpc_csharp_ext.dylib" + } + } + } + } + }, + "runtimes": { + "win7-x64": { }, + "debian.8-x64": { }, + "osx.10.11-x64": { } + }, + "dependencies": { "Grpc.Auth": { "target": "project" @@ -41,9 +68,5 @@ "System.Linq.Expressions": "4.0.11-rc2-24027" } } - }, - "runtimes": { - "win7-x64": { }, - "debian.8-x64": {} } } From 6d082203eb23b4abe31e3e80d4ab01bc9bb6f3af Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Tue, 21 Jun 2016 10:03:38 -0700 Subject: [PATCH 126/280] try making C# tests work on coreclr on mac --- tools/run_tests/run_tests.py | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index c5cf0135f73..3881c075d46 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -509,10 +509,11 @@ class CSharpLanguage(object): self._docker_distro = 'coreclr' if self.platform == 'mac': - # On Mac, official distribution of mono is 32bit. # TODO(jtattermusch): EMBED_ZLIB=true currently breaks the mac build - self._make_options = ['EMBED_OPENSSL=true', - 'CFLAGS=-m32', 'LDFLAGS=-m32'] + self._make_options = ['EMBED_OPENSSL=true'] + if self.args.compiler != 'coreclr': + # On Mac, official distribution of mono is 32bit. + self._make_options += ['CFLAGS=-m32', 'LDFLAGS=-m32'] else: self._make_options = ['EMBED_OPENSSL=true', 'EMBED_ZLIB=true'] @@ -529,6 +530,9 @@ class CSharpLanguage(object): if self.platform == 'linux': assembly_subdir += '/netstandard1.5/debian.8-x64' assembly_extension = '' + if self.platform == 'mac': + assembly_subdir += '/netstandard1.5/osx.10.11-x64' + assembly_extension = '' else: assembly_subdir += '/netstandard1.5/win7-x64' runtime_cmd = [] From 9f26a149a5da48364ec62dd607e34104bb851c6d Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 21 Jun 2016 10:15:01 -0700 Subject: [PATCH 127/280] fixed incomplete sentence --- doc/compression.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/compression.md b/doc/compression.md index 6e455636021..a116ac50a16 100644 --- a/doc/compression.md +++ b/doc/compression.md @@ -81,8 +81,8 @@ initial call. A client doesn't a priori (presently) know which algorithms a server supports. This issue can be addressed with an initial negotiation of capabilities or an automatic retry mechanism. These features will be implemented in the future. Currently however, compression levels are only supported at the -server side, which is aware of the client's capabilities by virtue of the -incoming. +server side, which is aware of the client's capabilities through the incoming +Message-Accept-Encoding header. ### Propagation to child RPCs From 6e2f88c9fd21a06ad265c494325999c679c53375 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Tue, 21 Jun 2016 10:52:40 -0700 Subject: [PATCH 128/280] compile 64bit extension for coreclr on windows --- tools/run_tests/run_tests.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 3881c075d46..d3819c08840 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -501,8 +501,10 @@ class CSharpLanguage(object): if self.platform == 'windows': # Explicitly choosing between x86 and x64 arch doesn't work yet _check_arch(self.args.arch, ['default']) + # CoreCLR use 64bit runtime by default. + arch_option = 'x64' if self.args.compiler == 'coreclr' else self.args.arch self._make_options = [_windows_toolset_option(self.args.compiler), - _windows_arch_option(self.args.arch)] + _windows_arch_option(arch_option)] else: _check_compiler(self.args.compiler, ['default', 'coreclr']) if self.platform == 'linux' and self.args.compiler == 'coreclr': From d5fd7ddc70e658a0364bd2e43c8a486a25db267c Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Tue, 21 Jun 2016 11:13:23 -0700 Subject: [PATCH 129/280] Addressed review comments Removed the silencing for incompatible-pointer-types Removed unused objects Fixed format issues --- src/objective-c/ProtoRPC/ProtoService.m | 9 +++------ src/objective-c/tests/GRPCClientTests.m | 24 ++++++++++++------------ src/objective-c/tests/Podfile | 3 ++- 3 files changed, 17 insertions(+), 19 deletions(-) diff --git a/src/objective-c/ProtoRPC/ProtoService.m b/src/objective-c/ProtoRPC/ProtoService.m index cd9bc7aeac4..97401908519 100644 --- a/src/objective-c/ProtoRPC/ProtoService.m +++ b/src/objective-c/ProtoRPC/ProtoService.m @@ -65,22 +65,19 @@ return self; } -#pragma clang diagnostic push -#pragma clang diagnostic ignored "-Wincompatible-pointer-types" - (ProtoRPC *)RPCToMethod:(NSString *)method requestsWriter:(GRXWriter *)requestsWriter responseClass:(Class)responseClass responsesWriteable:(id)responsesWriteable { - ProtoMethod *methodName = [[ProtoMethod alloc] initWithPackage:_packageName - service:_serviceName - method:method]; + GRPCProtoMethod *methodName = [[GRPCProtoMethod alloc] initWithPackage:_packageName + service:_serviceName + method:method]; return [[ProtoRPC alloc] initWithHost:_host method:methodName requestsWriter:requestsWriter responseClass:responseClass responsesWriteable:responsesWriteable]; } -#pragma clang diagnostic pop @end @implementation GRPCProtoService diff --git a/src/objective-c/tests/GRPCClientTests.m b/src/objective-c/tests/GRPCClientTests.m index 2eca7bf5498..1167a715bb9 100644 --- a/src/objective-c/tests/GRPCClientTests.m +++ b/src/objective-c/tests/GRPCClientTests.m @@ -110,14 +110,14 @@ static GRPCProtoMethod *kUnaryCallMethod; // This method isn't implemented by the remote server. kInexistentMethod = [[GRPCProtoMethod alloc] initWithPackage:kPackage - service:kService - method:@"Inexistent"]; + service:kService + method:@"Inexistent"]; kEmptyCallMethod = [[GRPCProtoMethod alloc] initWithPackage:kPackage - service:kService - method:@"EmptyCall"]; + service:kService + method:@"EmptyCall"]; kUnaryCallMethod = [[GRPCProtoMethod alloc] initWithPackage:kPackage - service:kService - method:@"UnaryCall"]; + service:kService + method:@"UnaryCall"]; } - (void)testConnectionToRemoteServer { @@ -303,9 +303,9 @@ static GRPCProtoMethod *kUnaryCallMethod; // Try to set parameters to nil for GRPCCall. This should cause an exception @try { - GRPCCall *call __unused = [[GRPCCall alloc] initWithHost:nil - path:nil - requestsWriter:nil]; + (void)[[GRPCCall alloc] initWithHost:nil + path:nil + requestsWriter:nil]; XCTFail(@"Did not receive an exception when parameters are nil"); } @catch(NSException *theException) { NSLog(@"Received exception as expected: %@", theException.name); @@ -316,9 +316,9 @@ static GRPCProtoMethod *kUnaryCallMethod; GRXWriter *requestsWriter = [GRXWriter emptyWriter]; [requestsWriter finishWithError:nil]; @try { - GRPCCall *call __unused = [[GRPCCall alloc] initWithHost:kHostAddress - path:kUnaryCallMethod.HTTPPath - requestsWriter:requestsWriter]; + (void)[[GRPCCall alloc] initWithHost:kHostAddress + path:kUnaryCallMethod.HTTPPath + requestsWriter:requestsWriter]; XCTFail(@"Did not receive an exception when GRXWriter has incorrect state."); } @catch(NSException *theException) { NSLog(@"Received exception as expected: %@", theException.name); diff --git a/src/objective-c/tests/Podfile b/src/objective-c/tests/Podfile index d51b18cc34d..6d5f94cbda1 100644 --- a/src/objective-c/tests/Podfile +++ b/src/objective-c/tests/Podfile @@ -6,7 +6,7 @@ install! 'cocoapods', :deterministic_uuids => false def shared_pods pod 'Protobuf', :path => "../../../third_party/protobuf", :inhibit_warnings => true pod 'BoringSSL', :podspec => "..", :inhibit_warnings => true - pod 'CronetFramework', :podspec => "..", :inhibit_warnings => true + pod 'CronetFramework', :podspec => ".." pod 'gRPC', :path => "../../.." pod 'RemoteTest', :path => "RemoteTestClient" end @@ -42,6 +42,7 @@ post_install do |installer| end if target.name == 'gRPC' target.build_configurations.each do |config| + # TODO(zyc) Remove this setting after the issue is resolved # GPR_UNREACHABLE_CODE causes "Control may reach end of non-void # function" warning config.build_settings['GCC_WARN_ABOUT_RETURN_TYPE'] = 'NO' From 8d9e83806d24ad5e1a47bb705d88816bfc8b4864 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Tue, 21 Jun 2016 11:21:48 -0700 Subject: [PATCH 130/280] Fixed format issues --- src/objective-c/tests/InteropTests.m | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/objective-c/tests/InteropTests.m b/src/objective-c/tests/InteropTests.m index 60a83259fa6..15ce120c551 100644 --- a/src/objective-c/tests/InteropTests.m +++ b/src/objective-c/tests/InteropTests.m @@ -285,8 +285,8 @@ static cronet_engine *cronetEngine = NULL; GRXBufferedPipe *requestsBuffer = [[GRXBufferedPipe alloc] init]; GRPCProtoCall *call = [_service RPCToStreamingInputCallWithRequestsWriter:requestsBuffer - handler:^(RMTStreamingInputCallResponse *response, - NSError *error) { + handler:^(RMTStreamingInputCallResponse *response, + NSError *error) { XCTAssertEqual(error.code, GRPC_STATUS_CANCELLED); [expectation fulfill]; }]; From 743decdafa3fe9b5e315748e61ff368245ec4b9d Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Tue, 21 Jun 2016 11:40:47 -0700 Subject: [PATCH 131/280] fix C# --use_docker on linux --- tools/run_tests/run_tests.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index d3819c08840..2a80706e6a8 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -509,6 +509,8 @@ class CSharpLanguage(object): _check_compiler(self.args.compiler, ['default', 'coreclr']) if self.platform == 'linux' and self.args.compiler == 'coreclr': self._docker_distro = 'coreclr' + else: + self._docker_distro = 'jessie' if self.platform == 'mac': # TODO(jtattermusch): EMBED_ZLIB=true currently breaks the mac build From b665dd2722371e5c4a28bbf6cb5f791f8ab5cca9 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 21 Jun 2016 11:52:29 -0700 Subject: [PATCH 132/280] reworded some --- doc/compression.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/compression.md b/doc/compression.md index a116ac50a16..15fae4d29bf 100644 --- a/doc/compression.md +++ b/doc/compression.md @@ -42,13 +42,13 @@ and RPC settings (for example, if compression would result in small or negative gains). When a message from a client compressed with an unsupported algorithm is -processed by a server, it WILL result in an INVALID\_ARGUMENT error. The server -will include in its response a `grpc-accept-encoding` header specifying the -algorithms it does accept. If an INTERNAL error is returned from the server -despite having used one of the algorithms from the `grpc-accept-encoding` -header, the cause MUST NOT be related to compression. Data sent from a server -compressed with an algorithm not supported by the client WILL result in an -INTERNAL error on the client side. +processed by a server, it WILL result in an INVALID\_ARGUMENT error on the +server. The server will then include in its response a `grpc-accept-encoding` +header specifying the algorithms it does accept. If an INTERNAL error is +returned from the server despite having used one of the algorithms from the +`grpc-accept-encoding` header, the cause MUST NOT be related to compression. +Data sent from a server compressed with an algorithm not supported by the client +WILL result in an INTERNAL error on the client side. Note that a peer MAY choose to not disclose all the encodings it supports. However, if it receives a message compressed in an undisclosed but supported From 3d399cb102510e46ca252213ad2799b40704d33d Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Tue, 21 Jun 2016 11:57:46 -0700 Subject: [PATCH 133/280] Fix format issues --- src/objective-c/tests/InteropTests.m | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/objective-c/tests/InteropTests.m b/src/objective-c/tests/InteropTests.m index 15ce120c551..3102cf597a0 100644 --- a/src/objective-c/tests/InteropTests.m +++ b/src/objective-c/tests/InteropTests.m @@ -284,9 +284,10 @@ static cronet_engine *cronetEngine = NULL; // A buffered pipe to which we never write any value acts as a writer that just hangs. GRXBufferedPipe *requestsBuffer = [[GRXBufferedPipe alloc] init]; - GRPCProtoCall *call = [_service RPCToStreamingInputCallWithRequestsWriter:requestsBuffer - handler:^(RMTStreamingInputCallResponse *response, - NSError *error) { + GRPCProtoCall *call = + [_service RPCToStreamingInputCallWithRequestsWriter:requestsBuffer + handler:^(RMTStreamingInputCallResponse *response, + NSError *error) { XCTAssertEqual(error.code, GRPC_STATUS_CANCELLED); [expectation fulfill]; }]; From 7edfcb021f56a1fe0cf1154740cfad7a4be21cc7 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 21 Jun 2016 12:02:33 -0700 Subject: [PATCH 134/280] Using inlined BoolValue in lieu of wrappers.proto due to lack of node support --- src/proto/grpc/testing/messages.proto | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/src/proto/grpc/testing/messages.proto b/src/proto/grpc/testing/messages.proto index e4e748a6913..367752d77bb 100644 --- a/src/proto/grpc/testing/messages.proto +++ b/src/proto/grpc/testing/messages.proto @@ -32,7 +32,13 @@ syntax = "proto3"; -import "google/protobuf/wrappers.proto"; +// TODO(dgq): Go back to using well-known types once +// https://github.com/grpc/grpc/issues/6980 has been fixed. +// import "google/protobuf/wrappers.proto"; +message BoolValue { + // The bool value. + bool value = 1; +} package grpc.testing; @@ -82,13 +88,13 @@ message SimpleRequest { // "nullable" in order to interoperate seamlessly with clients not able to // implement the full compression tests by introspecting the call to verify // the response's compression status. - google.protobuf.BoolValue response_compressed = 6; + BoolValue response_compressed = 6; // Whether server should return a given status EchoStatus response_status = 7; // Whether the server should expect this request to be compressed. - google.protobuf.BoolValue expect_compressed = 8; + BoolValue expect_compressed = 8; } // Unary response, as configured by the request. @@ -111,7 +117,7 @@ message StreamingInputCallRequest { // is "nullable" in order to interoperate seamlessly with servers not able to // implement the full compression tests by introspecting the call to verify // the request's compression status. - google.protobuf.BoolValue expect_compressed = 2; + BoolValue expect_compressed = 2; // Not expecting any payload from the response. } @@ -135,7 +141,7 @@ message ResponseParameters { // "nullable" in order to interoperate seamlessly with clients not able to // implement the full compression tests by introspecting the call to verify // the response's compression status. - google.protobuf.BoolValue compressed = 3; + BoolValue compressed = 3; } // Server-streaming request. From ad8723f6473034aacbfd7c0e80c0ad2b93156db8 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 21 Jun 2016 13:09:29 -0700 Subject: [PATCH 135/280] moved up 'package' statement. --- src/proto/grpc/testing/messages.proto | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/proto/grpc/testing/messages.proto b/src/proto/grpc/testing/messages.proto index 367752d77bb..64998c2f231 100644 --- a/src/proto/grpc/testing/messages.proto +++ b/src/proto/grpc/testing/messages.proto @@ -32,6 +32,8 @@ syntax = "proto3"; +package grpc.testing; + // TODO(dgq): Go back to using well-known types once // https://github.com/grpc/grpc/issues/6980 has been fixed. // import "google/protobuf/wrappers.proto"; @@ -40,8 +42,6 @@ message BoolValue { bool value = 1; } -package grpc.testing; - // DEPRECATED, don't use. To be removed shortly. // The type of payload that should be returned. enum PayloadType { From f8bbe72a00ef3827879ea8d377c595c8e96d6a66 Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Tue, 21 Jun 2016 14:05:16 -0700 Subject: [PATCH 136/280] Make src/node/tools/package.json consistent with its template --- templates/src/node/tools/package.json.template | 1 + 1 file changed, 1 insertion(+) diff --git a/templates/src/node/tools/package.json.template b/templates/src/node/tools/package.json.template index 69ad71a3b83..02824259767 100644 --- a/templates/src/node/tools/package.json.template +++ b/templates/src/node/tools/package.json.template @@ -36,6 +36,7 @@ "index.js", "bin/protoc.js", "bin/protoc_plugin.js", + "bin/google/protobuf", "LICENSE" ], "main": "index.js" From be3745529715745901245073233bf2fd8f282b00 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Tue, 21 Jun 2016 15:11:33 -0700 Subject: [PATCH 137/280] add missing ConfigureAwait(false) --- src/csharp/Grpc.Core/GrpcEnvironment.cs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/csharp/Grpc.Core/GrpcEnvironment.cs b/src/csharp/Grpc.Core/GrpcEnvironment.cs index e9e4cb4cbb3..58bb1802510 100644 --- a/src/csharp/Grpc.Core/GrpcEnvironment.cs +++ b/src/csharp/Grpc.Core/GrpcEnvironment.cs @@ -105,7 +105,7 @@ namespace Grpc.Core if (instanceToShutdown != null) { - await instanceToShutdown.ShutdownAsync(); + await instanceToShutdown.ShutdownAsync().ConfigureAwait(false); } } From 9d393a5d3c1a2609c3aa44caa3879e590f659c21 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 21 Jun 2016 15:20:19 -0700 Subject: [PATCH 138/280] slightly fancier output --- tools/profiling/latency_profile/run_latency_profile.sh | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/tools/profiling/latency_profile/run_latency_profile.sh b/tools/profiling/latency_profile/run_latency_profile.sh index 40c6fcb4314..618db202dc4 100755 --- a/tools/profiling/latency_profile/run_latency_profile.sh +++ b/tools/profiling/latency_profile/run_latency_profile.sh @@ -96,10 +96,11 @@ fi make CONFIG=basicprof -j$CPUS qps_json_driver mkdir -p reports -echo '' > reports/index.html bins/basicprof/qps_json_driver --scenarios_json="$SCENARIOS_JSON_ARG" -echo '
' >> reports/index.html
+
+echo 'Latency profile for:
' > reports/index.html +echo "

${SCENARIOS_JSON_ARG}

" >> reports/index.html +echo '

' >> reports/index.html
 $PYTHON tools/profiling/latency_profile/profile_analyzer.py \
     --source=latency_trace.txt --fmt=simple >> reports/index.html
-echo '
' >> reports/index.html -echo '' >> reports/index.html +echo '

' >> reports/index.html From dacce7a1e11543ee3b3b49becc3466d967daa733 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 21 Jun 2016 16:40:20 -0700 Subject: [PATCH 139/280] fixed bad merge. c++ tests should work again --- test/cpp/interop/interop_client.cc | 34 ++++++++++++++---------------- test/cpp/interop/interop_server.cc | 2 +- 2 files changed, 17 insertions(+), 19 deletions(-) diff --git a/test/cpp/interop/interop_client.cc b/test/cpp/interop/interop_client.cc index cfa17e5b76c..89f841dbe96 100644 --- a/test/cpp/interop/interop_client.cc +++ b/test/cpp/interop/interop_client.cc @@ -537,20 +537,19 @@ bool InteropClient::DoServerCompressedStreaming() { InteropClientContextInspector inspector(context); StreamingOutputCallRequest request; - for (size_t i = 0; i < compressions.size(); i++) { - for (size_t j = 0; j < sizes.size(); j++) { - char* log_suffix; - gpr_asprintf(&log_suffix, "(compression=%s; size=%d)", - compressions[i] ? "true" : "false", sizes[j]); + GPR_ASSERT(compressions.size() == sizes.size()); + for (size_t i = 0; i < sizes.size(); i++) { + char* log_suffix; + gpr_asprintf(&log_suffix, "(compression=%s; size=%d)", + compressions[i] ? "true" : "false", sizes[i]); - gpr_log(GPR_DEBUG, "Sending request streaming rpc %s.", log_suffix); - gpr_free(log_suffix); + gpr_log(GPR_DEBUG, "Sending request streaming rpc %s.", log_suffix); + gpr_free(log_suffix); - ResponseParameters* const response_parameter = - request.add_response_parameters(); - response_parameter->mutable_compressed()->set_value(compressions[i]); - response_parameter->set_size(sizes[j]); - } + ResponseParameters* const response_parameter = + request.add_response_parameters(); + response_parameter->mutable_compressed()->set_value(compressions[i]); + response_parameter->set_size(sizes[i]); } std::unique_ptr> stream( serviceStub_.Get()->StreamingOutputCall(&context, request)); @@ -574,14 +573,13 @@ bool InteropClient::DoServerCompressedStreaming() { ++k; } - if (k < response_stream_sizes.size()) { + if (k < sizes.size()) { // stream->Read() failed before reading all the expected messages. This // is most likely due to a connection failure. - gpr_log(GPR_ERROR, - "%s(): Responses read (k=%" PRIuPTR - ") is " - "less than the expected messages (i.e " - "response_stream_sizes.size() (%" PRIuPTR ")).", + gpr_log(GPR_ERROR, "%s(): Responses read (k=%" PRIuPTR + ") is " + "less than the expected messages (i.e " + "response_stream_sizes.size() (%" PRIuPTR ")).", __func__, k, response_stream_sizes.size()); return TransientFailureOrAbort(); } diff --git a/test/cpp/interop/interop_server.cc b/test/cpp/interop/interop_server.cc index f0a182f2309..199fef5455b 100644 --- a/test/cpp/interop/interop_server.cc +++ b/test/cpp/interop/interop_server.cc @@ -222,7 +222,7 @@ class TestServiceImpl : public TestService::Service { gpr_time_from_micros(time_us, GPR_TIMESPAN)); gpr_sleep_until(sleep_time); } - write_success = writer->Write(response); + write_success = writer->Write(response, wopts); } if (write_success) { return Status::OK; From ef01edf5b7c6eb008c7b0581354678aeeabd7680 Mon Sep 17 00:00:00 2001 From: vjpai Date: Tue, 21 Jun 2016 16:42:08 -0700 Subject: [PATCH 140/280] Fix review comments --- include/grpc++/impl/codegen/async_stream.h | 12 ++++++------ include/grpc++/impl/codegen/completion_queue.h | 11 ++++++----- include/grpc++/impl/codegen/sync_stream.h | 12 ++++++------ 3 files changed, 18 insertions(+), 17 deletions(-) diff --git a/include/grpc++/impl/codegen/async_stream.h b/include/grpc++/impl/codegen/async_stream.h index c74737ce5f8..cbc5d944295 100644 --- a/include/grpc++/impl/codegen/async_stream.h +++ b/include/grpc++/impl/codegen/async_stream.h @@ -53,7 +53,7 @@ class ClientAsyncStreamingInterface { /// Request notification of the reading of the initial metadata. Completion /// will be notified by \a tag on the associated completion queue. /// This call is optional, but if it is used, it cannot be used concurrently - /// with Read + /// with or after the \a Read method. /// /// \param[in] tag Tag identifying this request. virtual void ReadInitialMetadata(void* tag) = 0; @@ -74,8 +74,8 @@ class AsyncReaderInterface { /// Read a message of type \a R into \a msg. Completion will be notified by \a /// tag on the associated completion queue. - /// This is thread-safe with respect to other streaming APIs except for Finish - /// on the same stream. + /// This is thread-safe with respect to other streaming APIs except for \a + /// Finish on the same stream. /// /// \param[out] msg Where to eventually store the read message. /// \param[in] tag The tag identifying the operation. @@ -93,7 +93,7 @@ class AsyncWriterInterface { /// Only one write may be outstanding at any given time. This means that /// after calling Write, one must wait to receive \a tag from the completion /// queue BEFORE calling Write again. - /// This is thread-safe with respect to Read + /// This is thread-safe with respect to \a Read /// /// \param[in] msg The message to be written. /// \param[in] tag The tag identifying the operation. @@ -164,7 +164,7 @@ class ClientAsyncWriterInterface : public ClientAsyncStreamingInterface, public AsyncWriterInterface { public: /// Signal the client is done with the writes. - /// Thread-safe with respect to Read + /// Thread-safe with respect to \a Read /// /// \param[in] tag The tag identifying the operation. virtual void WritesDone(void* tag) = 0; @@ -236,7 +236,7 @@ class ClientAsyncReaderWriterInterface : public ClientAsyncStreamingInterface, public AsyncReaderInterface { public: /// Signal the client is done with the writes. - /// Thread-safe with respect to Read + /// Thread-safe with respect to \a Read /// /// \param[in] tag The tag identifying the operation. virtual void WritesDone(void* tag) = 0; diff --git a/include/grpc++/impl/codegen/completion_queue.h b/include/grpc++/impl/codegen/completion_queue.h index f138ebe7de2..03009e0561d 100644 --- a/include/grpc++/impl/codegen/completion_queue.h +++ b/include/grpc++/impl/codegen/completion_queue.h @@ -34,15 +34,16 @@ /// A completion queue implements a concurrent producer-consumer queue, with /// two main API-exposed methods: \a Next and \a AsyncNext. These /// methods are the essential component of the gRPC C++ asynchronous API. -/// There is also a Shutdown method to indicate that a given completion queue +/// There is also a \a Shutdown method to indicate that a given completion queue /// will no longer have regular events. This must be called before the /// completion queue is destroyed. /// All completion queue APIs are thread-safe and may be used concurrently with /// any other completion queue API invocation; it is acceptable to have -/// multiple threads calling Next or AsyncNext on the same or different -/// completion queues, or to call these methods concurrently with a Shutdown -/// elsewhere. All of these should be completed, though, before a completion -/// queue destructor is called. +/// multiple threads calling \a Next or \a AsyncNext on the same or different +/// completion queues, or to call these methods concurrently with a \a Shutdown +/// elsewhere. +/// \remark{All other API calls on completion queue should be completed before +/// a completion queue destructor is called.} #ifndef GRPCXX_IMPL_CODEGEN_COMPLETION_QUEUE_H #define GRPCXX_IMPL_CODEGEN_COMPLETION_QUEUE_H diff --git a/include/grpc++/impl/codegen/sync_stream.h b/include/grpc++/impl/codegen/sync_stream.h index 9d7966ba04b..e53717cabfc 100644 --- a/include/grpc++/impl/codegen/sync_stream.h +++ b/include/grpc++/impl/codegen/sync_stream.h @@ -71,8 +71,8 @@ class ReaderInterface { virtual ~ReaderInterface() {} /// Blocking read a message and parse to \a msg. Returns \a true on success. - /// This is thread-safe with respect to other streaming APIs except for Finish - /// on the same stream. (Finish must be called as described above.) + /// This is thread-safe with respect to other streaming APIs except for \a + /// Finish on the same stream. (\a Finish must be called as described above.) /// /// \param[out] msg The read message. /// @@ -89,7 +89,7 @@ class WriterInterface { virtual ~WriterInterface() {} /// Blocking write \a msg to the stream with options. - /// This is thread-safe with respect to Read + /// This is thread-safe with respect to \a Read /// /// \param msg The message to be written to the stream. /// \param options Options affecting the write operation. @@ -98,7 +98,7 @@ class WriterInterface { virtual bool Write(const W& msg, const WriteOptions& options) = 0; /// Blocking write \a msg to the stream with default options. - /// This is thread-safe with respect to Read + /// This is thread-safe with respect to \a Read /// /// \param msg The message to be written to the stream. /// @@ -179,7 +179,7 @@ class ClientWriterInterface : public ClientStreamingInterface, public: /// Half close writing from the client. /// Block until currently-pending writes are completed. - /// Thread safe with respect to Read operations only + /// Thread safe with respect to \a Read operations only /// /// \return Whether the writes were successful. virtual bool WritesDone() = 0; @@ -263,7 +263,7 @@ class ClientReaderWriterInterface : public ClientStreamingInterface, virtual void WaitForInitialMetadata() = 0; /// Block until currently-pending writes are completed. - /// Thread-safe with respect to Read + /// Thread-safe with respect to \a Read /// /// \return Whether the writes were successful. virtual bool WritesDone() = 0; From e89aad02bf236190f1855d1481b5549a02cf8749 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 21 Jun 2016 16:43:31 -0700 Subject: [PATCH 141/280] updated node interop server --- src/node/interop/interop_server.js | 13 +------------ 1 file changed, 1 insertion(+), 12 deletions(-) diff --git a/src/node/interop/interop_server.js b/src/node/interop/interop_server.js index 72807623054..05f52a1083d 100644 --- a/src/node/interop/interop_server.js +++ b/src/node/interop/interop_server.js @@ -45,9 +45,6 @@ var testProto = grpc.load({ var ECHO_INITIAL_KEY = 'x-grpc-test-echo-initial'; var ECHO_TRAILING_KEY = 'x-grpc-test-echo-trailing-bin'; -var incompressible_data = fs.readFileSync( - __dirname + '/../../../test/cpp/interop/rnd.dat'); - /** * Create a buffer filled with size zeroes * @param {number} size The length of the buffer @@ -88,15 +85,7 @@ function getEchoTrailer(call) { } function getPayload(payload_type, size) { - if (payload_type === 'RANDOM') { - payload_type = ['COMPRESSABLE', - 'UNCOMPRESSABLE'][Math.random() < 0.5 ? 0 : 1]; - } - var body; - switch (payload_type) { - case 'COMPRESSABLE': body = zeroBuffer(size); break; - case 'UNCOMPRESSABLE': incompressible_data.slice(size); break; - } + var body = zeroBuffer(size); return {type: payload_type, body: body}; } From ff32a8648270ccf9fce9d8950eee06e917177715 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 21 Jun 2016 17:07:20 -0700 Subject: [PATCH 142/280] updated the interop tests driver --- tools/run_tests/run_interop_tests.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/tools/run_tests/run_interop_tests.py b/tools/run_tests/run_interop_tests.py index 0787637d758..cc146f800ac 100755 --- a/tools/run_tests/run_interop_tests.py +++ b/tools/run_tests/run_interop_tests.py @@ -54,7 +54,9 @@ os.chdir(ROOT) _DEFAULT_SERVER_PORT=8080 -_SKIP_COMPRESSION = ['large_compressed_unary', +_SKIP_COMPRESSION = ['client_compressed_unary', + 'client_compressed_streaming' + 'server_compressed_unary', 'server_compressed_streaming'] _SKIP_ADVANCED = ['custom_metadata', 'status_code_and_message', @@ -345,7 +347,8 @@ _TEST_CASES = ['large_unary', 'empty_unary', 'ping_pong', 'cancel_after_begin', 'cancel_after_first_response', 'timeout_on_sleeping_server', 'custom_metadata', 'status_code_and_message', 'unimplemented_method', - 'large_compressed_unary', 'server_compressed_streaming'] + 'client_compressed_unary', 'server_compressed_unary', + 'client_compressed_streaming', 'server_compressed_streaming'] _AUTH_TEST_CASES = ['compute_engine_creds', 'jwt_token_creds', 'oauth2_auth_token', 'per_rpc_creds'] From 47031a8b14a501a40d8217b2a2e37764da8b5afb Mon Sep 17 00:00:00 2001 From: Ken Payson Date: Wed, 4 May 2016 16:39:06 -0700 Subject: [PATCH 143/280] Add egg-info to python distribution. Currently, grpcio cannot be used as a dependency for egg packages (https://github.com/grpc/grpc/issues/6939) There is likely a better solution, but this is intended as a patch for the 0.15.0 release. --- PYTHON-MANIFEST.in | 1 + 1 file changed, 1 insertion(+) diff --git a/PYTHON-MANIFEST.in b/PYTHON-MANIFEST.in index 534f4c1251a..3ebba6ec3fc 100644 --- a/PYTHON-MANIFEST.in +++ b/PYTHON-MANIFEST.in @@ -1,6 +1,7 @@ recursive-include src/python/grpcio/grpc *.c *.h *.py *.pyx *.pxd *.pxi *.python *.pem recursive-exclude src/python/grpcio/grpc/_cython *.so *.pyd graft src/python/grpcio/tests +graft src/python/grpcio/grpcio.egg-info graft src/core graft src/boringssl graft include/grpc From 446f70e3843d9cdf40b6cf80d74bcd153032c2cf Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 21 Jun 2016 17:13:28 -0700 Subject: [PATCH 144/280] fixed faulty server streaming c++ test case --- test/cpp/interop/interop_server.cc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/cpp/interop/interop_server.cc b/test/cpp/interop/interop_server.cc index 199fef5455b..ebef0002a3c 100644 --- a/test/cpp/interop/interop_server.cc +++ b/test/cpp/interop/interop_server.cc @@ -194,8 +194,6 @@ class TestServiceImpl : public TestService::Service { ServerContext* context, const StreamingOutputCallRequest* request, ServerWriter* writer) { StreamingOutputCallResponse response; - // Compress by default. Disabled on a per-message basis. - context->set_compression_level(GRPC_COMPRESS_LEVEL_HIGH); bool write_success = true; for (int i = 0; write_success && i < request->response_parameters_size(); i++) { @@ -206,6 +204,8 @@ class TestServiceImpl : public TestService::Service { } WriteOptions wopts; if (request->response_parameters(i).has_compressed()) { + // Compress by default. Disabled on a per-message basis. + context->set_compression_level(GRPC_COMPRESS_LEVEL_HIGH); const bool compression_requested = request->response_parameters(i).compressed().value(); gpr_log(GPR_DEBUG, "Request for compression (%s) present for %s", From 396f9435d5c24c4f7179cfc5f789669d3900f7fe Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Tue, 21 Jun 2016 17:33:03 -0700 Subject: [PATCH 145/280] added freaking missing comma --- tools/run_tests/run_interop_tests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/run_tests/run_interop_tests.py b/tools/run_tests/run_interop_tests.py index cc146f800ac..e3af721ee53 100755 --- a/tools/run_tests/run_interop_tests.py +++ b/tools/run_tests/run_interop_tests.py @@ -55,7 +55,7 @@ os.chdir(ROOT) _DEFAULT_SERVER_PORT=8080 _SKIP_COMPRESSION = ['client_compressed_unary', - 'client_compressed_streaming' + 'client_compressed_streaming', 'server_compressed_unary', 'server_compressed_streaming'] From 3131c269c14f97294ebf8b6e3d1a235d4acf3317 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 21 Jun 2016 17:28:28 -0700 Subject: [PATCH 146/280] Integrate with unified error reporting --- src/core/lib/iomgr/ev_epoll_linux.c | 116 +++++++++++++++++++------- test/core/iomgr/ev_epoll_linux_test.c | 3 +- 2 files changed, 90 insertions(+), 29 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 7cc69c876db..d625b096a10 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -646,11 +646,18 @@ polling_island *polling_island_merge(polling_island *p, polling_island *q) { return q; } -static void polling_island_global_init() { +static grpc_error *polling_island_global_init() { + grpc_error *error = GRPC_ERROR_NONE; + gpr_mu_init(&g_pi_freelist_mu); g_pi_freelist = NULL; - grpc_wakeup_fd_init(&polling_island_wakeup_fd); - grpc_wakeup_fd_wakeup(&polling_island_wakeup_fd); + + error = grpc_wakeup_fd_init(&polling_island_wakeup_fd); + if (error == GRPC_ERROR_NONE) { + error = grpc_wakeup_fd_wakeup(&polling_island_wakeup_fd); + } + + return error; } static void polling_island_global_shutdown() { @@ -870,21 +877,33 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, } gpr_mu_unlock(&fd->pi_mu); - grpc_exec_ctx_enqueue(exec_ctx, fd->on_done_closure, true, NULL); + grpc_exec_ctx_sched(exec_ctx, fd->on_done_closure, GRPC_ERROR_NONE, NULL); gpr_mu_unlock(&fd->mu); UNREF_BY(fd, 2, reason); /* Drop the reference */ } +static grpc_error *fd_shutdown_error(bool shutdown) { + if (!shutdown) { + return GRPC_ERROR_NONE; + } else { + return GRPC_ERROR_CREATE("FD shutdown"); + } +} + static void notify_on_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure **st, grpc_closure *closure) { - if (*st == CLOSURE_NOT_READY) { + if (fd->shutdown) { + grpc_exec_ctx_sched(exec_ctx, closure, GRPC_ERROR_CREATE("FD shutdown"), + NULL); + } else if (*st == CLOSURE_NOT_READY) { /* not ready ==> switch to a waiting state by setting the closure */ *st = closure; } else if (*st == CLOSURE_READY) { /* already ready ==> queue the closure to run immediately */ *st = CLOSURE_NOT_READY; - grpc_exec_ctx_enqueue(exec_ctx, closure, !fd->shutdown, NULL); + grpc_exec_ctx_sched(exec_ctx, closure, fd_shutdown_error(fd->shutdown), + NULL); } else { /* upcallptr was set to a different closure. This is an error! */ gpr_log(GPR_ERROR, @@ -906,7 +925,7 @@ static int set_ready_locked(grpc_exec_ctx *exec_ctx, grpc_fd *fd, return 0; } else { /* waiting ==> queue closure */ - grpc_exec_ctx_enqueue(exec_ctx, *st, !fd->shutdown, NULL); + grpc_exec_ctx_sched(exec_ctx, *st, fd_shutdown_error(fd->shutdown), NULL); *st = CLOSURE_NOT_READY; return 1; } @@ -964,11 +983,11 @@ static void sig_handler(int sig_num) { static void poller_kick_init() { signal(grpc_wakeup_signal, sig_handler); } /* Global state management */ -static void pollset_global_init(void) { - grpc_wakeup_fd_init(&grpc_global_wakeup_fd); +static grpc_error *pollset_global_init(void) { gpr_tls_init(&g_current_thread_pollset); gpr_tls_init(&g_current_thread_worker); poller_kick_init(); + return grpc_wakeup_fd_init(&grpc_global_wakeup_fd); } static void pollset_global_shutdown(void) { @@ -977,8 +996,13 @@ static void pollset_global_shutdown(void) { gpr_tls_destroy(&g_current_thread_worker); } -static void pollset_worker_kick(grpc_pollset_worker *worker) { - pthread_kill(worker->pt_id, grpc_wakeup_signal); +static grpc_error *pollset_worker_kick(grpc_pollset_worker *worker) { + grpc_error *err = GRPC_ERROR_NONE; + int err_num = pthread_kill(worker->pt_id, grpc_wakeup_signal); + if (err_num != 0) { + err = GRPC_OS_ERROR(err_num, "pthread_kill"); + } + return err; } /* Return 1 if the pollset has active threads in pollset_work (pollset must @@ -1014,10 +1038,19 @@ static void push_front_worker(grpc_pollset *p, grpc_pollset_worker *worker) { worker->prev->next = worker->next->prev = worker; } +static void kick_append_error(grpc_error **composite, grpc_error *error) { + if (error == GRPC_ERROR_NONE) return; + if (*composite == GRPC_ERROR_NONE) { + *composite = GRPC_ERROR_CREATE("Kick Failure"); + } + *composite = grpc_error_add_child(*composite, error); +} + /* p->mu must be held before calling this function */ -static void pollset_kick(grpc_pollset *p, - grpc_pollset_worker *specific_worker) { +static grpc_error *pollset_kick(grpc_pollset *p, + grpc_pollset_worker *specific_worker) { GPR_TIMER_BEGIN("pollset_kick", 0); + grpc_error *error = GRPC_ERROR_NONE; grpc_pollset_worker *worker = specific_worker; if (worker != NULL) { @@ -1027,7 +1060,7 @@ static void pollset_kick(grpc_pollset *p, for (worker = p->root_worker.next; worker != &p->root_worker; worker = worker->next) { if (gpr_tls_get(&g_current_thread_worker) != (intptr_t)worker) { - pollset_worker_kick(worker); + kick_append_error(&error, pollset_worker_kick(worker)); } } } else { @@ -1037,7 +1070,7 @@ static void pollset_kick(grpc_pollset *p, } else { GPR_TIMER_MARK("kicked_specifically", 0); if (gpr_tls_get(&g_current_thread_worker) != (intptr_t)worker) { - pollset_worker_kick(worker); + kick_append_error(&error, pollset_worker_kick(worker)); } } } else if (gpr_tls_get(&g_current_thread_pollset) != (intptr_t)p) { @@ -1053,7 +1086,7 @@ static void pollset_kick(grpc_pollset *p, if (worker != NULL) { GPR_TIMER_MARK("finally_kick", 0); push_back_worker(p, worker); - pollset_worker_kick(worker); + kick_append_error(&error, pollset_worker_kick(worker)); } else { GPR_TIMER_MARK("kicked_no_pollers", 0); p->kicked_without_pollers = true; @@ -1061,9 +1094,13 @@ static void pollset_kick(grpc_pollset *p, } GPR_TIMER_END("pollset_kick", 0); + GRPC_LOG_IF_ERROR("pollset_kick", GRPC_ERROR_REF(error)); + return error; } -static void kick_poller(void) { grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); } +static grpc_error *kick_poller(void) { + return grpc_wakeup_fd_wakeup(&grpc_global_wakeup_fd); +} static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { gpr_mu_init(&pollset->mu); @@ -1139,7 +1176,7 @@ static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, /* Release the ref and set pollset->polling_island to NULL */ pollset_release_polling_island(pollset, "ps_shutdown"); - grpc_exec_ctx_enqueue(exec_ctx, pollset->shutdown_done, true, NULL); + grpc_exec_ctx_sched(exec_ctx, pollset->shutdown_done, GRPC_ERROR_NONE, NULL); } /* pollset->mu lock must be held by the caller before calling this */ @@ -1181,14 +1218,23 @@ static void pollset_reset(grpc_pollset *pollset) { pollset_release_polling_island(pollset, "ps_reset"); } +static void work_combine_error(grpc_error **composite, grpc_error *error) { + if (error == GRPC_ERROR_NONE) return; + if (*composite == GRPC_ERROR_NONE) { + *composite = GRPC_ERROR_CREATE("pollset_work"); + } + *composite = grpc_error_add_child(*composite, error); +} + #define GRPC_EPOLL_MAX_EVENTS 1000 -static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, int timeout_ms, - sigset_t *sig_mask) { +static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, + int timeout_ms, sigset_t *sig_mask) { struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; int epoll_fd = -1; int ep_rv; polling_island *pi = NULL; + grpc_error *error = GRPC_ERROR_NONE; GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the @@ -1232,6 +1278,7 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, if (ep_rv < 0) { if (errno != EINTR) { gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); + work_combine_error(&error, GRPC_OS_ERROR(errno, "epoll_pwait")); } else { /* We were interrupted. Save an interation by doing a zero timeout epoll_wait to see if there are any other events of interest */ @@ -1247,7 +1294,8 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, for (int i = 0; i < ep_rv; ++i) { void *data_ptr = ep_ev[i].data.ptr; if (data_ptr == &grpc_global_wakeup_fd) { - grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd); + work_combine_error( + &error, grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd)); } else if (data_ptr == &polling_island_wakeup_fd) { /* This means that our polling island is merged with a different island. We do not have to do anything here since the subsequent call @@ -1278,16 +1326,18 @@ static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, PI_UNREF(pi, "ps_work"); GPR_TIMER_END("pollset_work_and_unlock", 0); + return error; } /* pollset->mu lock must be held by the caller before calling this. The function pollset_work() may temporarily release the lock (pollset->mu) during the course of its execution but it will always re-acquire the lock and ensure that it is held by the time the function returns */ -static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, - grpc_pollset_worker **worker_hdl, gpr_timespec now, - gpr_timespec deadline) { +static grpc_error *pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, + grpc_pollset_worker **worker_hdl, + gpr_timespec now, gpr_timespec deadline) { GPR_TIMER_BEGIN("pollset_work", 0); + grpc_error *error = GRPC_ERROR_NONE; int timeout_ms = poll_deadline_to_millis_timeout(deadline, now); sigset_t new_mask; @@ -1316,7 +1366,7 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, push_front_worker(pollset, &worker); - pollset_work_and_unlock(exec_ctx, pollset, timeout_ms, &orig_mask); + error = pollset_work_and_unlock(exec_ctx, pollset, timeout_ms, &orig_mask); grpc_exec_ctx_flush(exec_ctx); gpr_mu_lock(&pollset->mu); @@ -1345,6 +1395,8 @@ static void pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, gpr_tls_set(&g_current_thread_pollset, (intptr_t)0); gpr_tls_set(&g_current_thread_worker, (intptr_t)0); GPR_TIMER_END("pollset_work", 0); + GRPC_LOG_IF_ERROR("pollset_work", GRPC_ERROR_REF(error)); + return error; } static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, @@ -1659,8 +1711,16 @@ const grpc_event_engine_vtable *grpc_init_epoll_linux(void) { } fd_global_init(); - pollset_global_init(); - polling_island_global_init(); + + if (!GRPC_LOG_IF_ERROR("pollset_global_init", pollset_global_init())) { + return NULL; + } + + if (!GRPC_LOG_IF_ERROR("polling_island_global_init", + polling_island_global_init())) { + return NULL; + } + return &vtable; } diff --git a/test/core/iomgr/ev_epoll_linux_test.c b/test/core/iomgr/ev_epoll_linux_test.c index 51da15faa7a..034f17fd58b 100644 --- a/test/core/iomgr/ev_epoll_linux_test.c +++ b/test/core/iomgr/ev_epoll_linux_test.c @@ -88,7 +88,8 @@ static void test_pollset_init(test_pollset *pollsets, int num_pollsets) { } } -static void destroy_pollset(grpc_exec_ctx *exec_ctx, void *p, bool success) { +static void destroy_pollset(grpc_exec_ctx *exec_ctx, void *p, + grpc_error *error) { grpc_pollset_destroy(p); } From 0100b2f1c0b08800ba0f7f53fe9cb5fbec7881a7 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 21 Jun 2016 17:38:13 -0700 Subject: [PATCH 147/280] Make fd_shutdown idempotent --- src/core/lib/iomgr/ev_epoll_linux.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index d625b096a10..c077987c01b 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -942,15 +942,19 @@ static grpc_pollset *fd_get_read_notifier_pollset(grpc_exec_ctx *exec_ctx, return notifier; } +/* Might be called multiple times */ static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { gpr_mu_lock(&fd->mu); - GPR_ASSERT(!fd->shutdown); - fd->shutdown = true; - - /* Flush any pending read and write closures. Since fd->shutdown is 'true' at - this point, the closures would be called with 'success = false' */ - set_ready_locked(exec_ctx, fd, &fd->read_closure); - set_ready_locked(exec_ctx, fd, &fd->write_closure); + /* Do the actual shutdown only once */ + if (!fd->shutdown) { + fd->shutdown = true; + + shutdown(fd->fd, SHUT_RDWR); + /* Flush any pending read and write closures. Since fd->shutdown is 'true' + at this point, the closures would be called with 'success = false' */ + set_ready_locked(exec_ctx, fd, &fd->read_closure); + set_ready_locked(exec_ctx, fd, &fd->write_closure); + } gpr_mu_unlock(&fd->mu); } From 24b6eae1fc71a4f5d18eb2e7c1cbca5b4e54a46f Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 21 Jun 2016 18:01:14 -0700 Subject: [PATCH 148/280] Add missing function fd_is_shutdown --- src/core/lib/iomgr/ev_epoll_linux.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index c077987c01b..3a774a8876f 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -942,6 +942,13 @@ static grpc_pollset *fd_get_read_notifier_pollset(grpc_exec_ctx *exec_ctx, return notifier; } +static bool fd_is_shutdown(grpc_fd *fd) { + gpr_mu_lock(&fd->mu); + const bool r = fd->shutdown; + gpr_mu_unlock(&fd->mu); + return r; +} + /* Might be called multiple times */ static void fd_shutdown(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { gpr_mu_lock(&fd->mu); @@ -1659,6 +1666,7 @@ static const grpc_event_engine_vtable vtable = { .fd_wrapped_fd = fd_wrapped_fd, .fd_orphan = fd_orphan, .fd_shutdown = fd_shutdown, + .fd_is_shutdown = fd_is_shutdown, .fd_notify_on_read = fd_notify_on_read, .fd_notify_on_write = fd_notify_on_write, .fd_get_read_notifier_pollset = fd_get_read_notifier_pollset, From bd28936c8648cde481fa20ca64ba00d9281cb119 Mon Sep 17 00:00:00 2001 From: vjpai Date: Tue, 21 Jun 2016 18:03:37 -0700 Subject: [PATCH 149/280] Properly handle reviewer comment re concurrent Read --- include/grpc++/impl/codegen/async_stream.h | 7 +++++-- include/grpc++/impl/codegen/sync_stream.h | 5 +++-- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/include/grpc++/impl/codegen/async_stream.h b/include/grpc++/impl/codegen/async_stream.h index cbc5d944295..e96d224ddbe 100644 --- a/include/grpc++/impl/codegen/async_stream.h +++ b/include/grpc++/impl/codegen/async_stream.h @@ -74,8 +74,11 @@ class AsyncReaderInterface { /// Read a message of type \a R into \a msg. Completion will be notified by \a /// tag on the associated completion queue. - /// This is thread-safe with respect to other streaming APIs except for \a - /// Finish on the same stream. + /// This is thread-safe with respect to \a Write or \a WritesDone methods. It + /// should not be called concurrently with other streaming APIs + /// on the same stream. It is not meaningful to call it concurrently + /// with another \a Read on the same stream since reads on the same stream + /// are delivered in order. /// /// \param[out] msg Where to eventually store the read message. /// \param[in] tag The tag identifying the operation. diff --git a/include/grpc++/impl/codegen/sync_stream.h b/include/grpc++/impl/codegen/sync_stream.h index e53717cabfc..cbfa4106995 100644 --- a/include/grpc++/impl/codegen/sync_stream.h +++ b/include/grpc++/impl/codegen/sync_stream.h @@ -71,8 +71,9 @@ class ReaderInterface { virtual ~ReaderInterface() {} /// Blocking read a message and parse to \a msg. Returns \a true on success. - /// This is thread-safe with respect to other streaming APIs except for \a - /// Finish on the same stream. (\a Finish must be called as described above.) + /// This is thread-safe with respect to \a Write or \WritesDone methods on + /// the same stream. It should not be called concurrently with another \a + /// Read on the same stream as the order of delivery will not be defined. /// /// \param[out] msg The read message. /// From 229533b1e68c4a4b8a67148f7fe25543584131f6 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Tue, 21 Jun 2016 20:42:52 -0700 Subject: [PATCH 150/280] Remove pollset->pi_mu since it is redundant. Also do not get polling island lock in the fast-path --- src/core/lib/iomgr/ev_epoll_linux.c | 82 ++++++++++++++++------------- src/core/lib/iomgr/ev_posix.c | 6 +-- 2 files changed, 46 insertions(+), 42 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 3a774a8876f..6464d3ba348 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -207,12 +207,7 @@ struct grpc_pollset { bool finish_shutdown_called; /* Is the 'finish_shutdown_locked()' called ? */ grpc_closure *shutdown_done; /* Called after after shutdown is complete */ - /* The polling island to which this pollset belongs to and the mutex - protecting the field */ - /* TODO: sreek: This lock might actually be adding more overhead to the - critical path (i.e pollset_work() function). Consider removing this lock - and just using the overall pollset lock */ - gpr_mu pi_mu; + /* The polling island to which this pollset belongs to */ struct polling_island *polling_island; }; @@ -488,31 +483,47 @@ static void polling_island_delete(polling_island *pi) { gpr_mu_unlock(&g_pi_freelist_mu); } +/* Attempts to gets the last polling island in the linked list (liked by the + * 'merged_to' field). Since this does not lock the polling island, there are no + * guarantees that the island returned is the last island */ +static polling_island *polling_island_maybe_get_latest(polling_island *pi) { + polling_island *next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + while (next != NULL) { + pi = next; + next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); + } + + return pi; +} + /* Gets the lock on the *latest* polling island i.e the last polling island in - the linked list (linked by 'merged_to' link). Call gpr_mu_unlock on the + the linked list (linked by the 'merged_to' field). Call gpr_mu_unlock on the returned polling island's mu. Usage: To lock/unlock polling island "pi", do the following: polling_island *pi_latest = polling_island_lock(pi); ... ... critical section .. ... - gpr_mu_unlock(&pi_latest->mu); //NOTE: use pi_latest->mu. NOT pi->mu */ -polling_island *polling_island_lock(polling_island *pi) { + gpr_mu_unlock(&pi_latest->mu); // NOTE: use pi_latest->mu. NOT pi->mu */ +static polling_island *polling_island_lock(polling_island *pi) { polling_island *next = NULL; + while (true) { next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); if (next == NULL) { - /* pi is the last node in the linked list. Get the lock and check again - (under the pi->mu lock) that pi is still the last node (because a merge - may have happend after the (next == NULL) check above and before - getting the pi->mu lock. - If pi is the last node, we are done. If not, unlock and continue - traversing the list */ + /* Looks like 'pi' is the last node in the linked list but unless we check + this by holding the pi->mu lock, we cannot be sure (i.e without the + pi->mu lock, we don't prevent island merges). + To be absolutely sure, check once more by holding the pi->mu lock */ gpr_mu_lock(&pi->mu); next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); if (next == NULL) { + /* pi is infact the last node and we have the pi->mu lock. we're done */ break; } + + /* pi->merged_to is not NULL i.e pi isn't the last node anymore. pi->mu + * isn't the lock we are interested in. Continue traversing the list */ gpr_mu_unlock(&pi->mu); } @@ -526,11 +537,11 @@ polling_island *polling_island_lock(polling_island *pi) { This function is needed because calling the following block of code to obtain locks on polling islands (*p and *q) is prone to deadlocks. { - polling_island_lock(*p); - polling_island_lock(*q); + polling_island_lock(*p, true); + polling_island_lock(*q, true); } - Usage/exmaple: + Usage/example: polling_island *p1; polling_island *p2; .. @@ -551,7 +562,7 @@ polling_island *polling_island_lock(polling_island *pi) { } */ -void polling_island_lock_pair(polling_island **p, polling_island **q) { +static void polling_island_lock_pair(polling_island **p, polling_island **q) { polling_island *pi_1 = *p; polling_island *pi_2 = *q; polling_island *next_1 = NULL; @@ -611,7 +622,8 @@ void polling_island_lock_pair(polling_island **p, polling_island **q) { *q = pi_2; } -polling_island *polling_island_merge(polling_island *p, polling_island *q) { +static polling_island *polling_island_merge(polling_island *p, + polling_island *q) { /* Get locks on both the polling islands */ polling_island_lock_pair(&p, &q); @@ -1124,7 +1136,6 @@ static void pollset_init(grpc_pollset *pollset, gpr_mu **mu) { pollset->finish_shutdown_called = false; pollset->shutdown_done = NULL; - gpr_mu_init(&pollset->pi_mu); pollset->polling_island = NULL; } @@ -1170,12 +1181,10 @@ static void fd_become_writable(grpc_exec_ctx *exec_ctx, grpc_fd *fd) { } static void pollset_release_polling_island(grpc_pollset *ps, char *reason) { - gpr_mu_lock(&ps->pi_mu); if (ps->polling_island != NULL) { PI_UNREF(ps->polling_island, reason); } ps->polling_island = NULL; - gpr_mu_unlock(&ps->pi_mu); } static void finish_shutdown_locked(grpc_exec_ctx *exec_ctx, @@ -1215,7 +1224,6 @@ static void pollset_shutdown(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, * here */ static void pollset_destroy(grpc_pollset *pollset) { GPR_ASSERT(!pollset_has_workers(pollset)); - gpr_mu_destroy(&pollset->pi_mu); gpr_mu_destroy(&pollset->mu); } @@ -1250,22 +1258,25 @@ static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the latest polling island pointed by pollset->polling_island. - Acquire the following locks: - - pollset->mu (which we already have) - - pollset->pi_mu - - pollset->polling_island lock */ - gpr_mu_lock(&pollset->pi_mu); + + Since epoll_fd is immutable, we can read it without obtaining the polling + island lock. There is however a possibility that the polling island (from + which we got the epoll_fd) got merged with another island while we are + in this function. This is still okay because in such a case, we will wakeup + right-away from epoll_wait() and pick up the latest polling_island the next + this function (i.e pollset_work_and_unlock()) is called. + */ if (pollset->polling_island == NULL) { pollset->polling_island = polling_island_create(NULL); PI_ADD_REF(pollset->polling_island, "ps"); } - pi = polling_island_lock(pollset->polling_island); + pi = polling_island_maybe_get_latest(pollset->polling_island); epoll_fd = pi->epoll_fd; /* Update the pollset->polling_island since the island being pointed by - pollset->polling_island may not be the latest (i.e pi) */ + pollset->polling_island maybe older than the one pointed by pi) */ if (pollset->polling_island != pi) { /* Always do PI_ADD_REF before PI_UNREF because PI_UNREF may cause the polling island to be deleted */ @@ -1278,9 +1289,6 @@ static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, the epoll_fd won't be closed) while we are are doing an epoll_wait() on the epoll_fd */ PI_ADD_REF(pi, "ps_work"); - - gpr_mu_unlock(&pi->mu); - gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); do { @@ -1413,7 +1421,6 @@ static grpc_error *pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { gpr_mu_lock(&pollset->mu); - gpr_mu_lock(&pollset->pi_mu); gpr_mu_lock(&fd->pi_mu); polling_island *pi_new = NULL; @@ -1465,7 +1472,6 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } gpr_mu_unlock(&fd->pi_mu); - gpr_mu_unlock(&pollset->pi_mu); gpr_mu_unlock(&pollset->mu); } @@ -1627,9 +1633,9 @@ void *grpc_fd_get_polling_island(grpc_fd *fd) { void *grpc_pollset_get_polling_island(grpc_pollset *ps) { polling_island *pi; - gpr_mu_lock(&ps->pi_mu); + gpr_mu_lock(&ps->mu); pi = ps->polling_island; - gpr_mu_unlock(&ps->pi_mu); + gpr_mu_unlock(&ps->mu); return pi; } diff --git a/src/core/lib/iomgr/ev_posix.c b/src/core/lib/iomgr/ev_posix.c index 4cdd13bbdba..a3c1e9db9a0 100644 --- a/src/core/lib/iomgr/ev_posix.c +++ b/src/core/lib/iomgr/ev_posix.c @@ -54,7 +54,7 @@ grpc_poll_function_type grpc_poll_function = poll; static const grpc_event_engine_vtable *g_event_engine; -static const char* g_poll_strategy_name = NULL; +static const char *g_poll_strategy_name = NULL; typedef const grpc_event_engine_vtable *(*event_engine_factory_fn)(void); @@ -111,9 +111,7 @@ static void try_engine(const char *engine) { } /* Call this only after calling grpc_event_engine_init() */ -const char *grpc_get_poll_strategy_name() { - return g_poll_strategy_name; -} +const char *grpc_get_poll_strategy_name() { return g_poll_strategy_name; } void grpc_event_engine_init(void) { char *s = gpr_getenv("GRPC_POLL_STRATEGY"); From caca0a15ded56b3732e0f67fe7d1507d9a7abf07 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Tue, 21 Jun 2016 23:07:49 -0700 Subject: [PATCH 151/280] Build objc examples as part of automatic tests --- src/objective-c/tests/build_example_test.sh | 56 +++++++++++++++++++ src/objective-c/tests/build_one_example.sh | 62 +++++++++++++++++++++ tools/run_tests/run_tests.py | 4 +- 3 files changed, 121 insertions(+), 1 deletion(-) create mode 100755 src/objective-c/tests/build_example_test.sh create mode 100755 src/objective-c/tests/build_one_example.sh diff --git a/src/objective-c/tests/build_example_test.sh b/src/objective-c/tests/build_example_test.sh new file mode 100755 index 00000000000..3cc0c552349 --- /dev/null +++ b/src/objective-c/tests/build_example_test.sh @@ -0,0 +1,56 @@ +#!/bin/bash +# Copyright 2015, Google Inc. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +# Don't run this script standalone. Instead, run from the repository root: +# ./tools/run_tests/run_tests.py -l objc + +set -eo pipefail + +cd `dirname $0` + +EXAMPLE_PATH=examples/objective-c/helloworld \ +SCHEME=HelloWorld \ +./build_one_example.sh + +EXAMPLE_PATH=examples/objective-c/route_guide \ +SCHEME=RouteGuideClient \ +./build_one_example.sh + +EXAMPLE_PATH=examples/objective-c/auth_sample \ +SCHEME=AuthSample \ +./build_one_example.sh + +EXAMPLE_PATH=src/objective-c/examples/Sample \ +SCHEME=Sample \ +./build_one_example.sh + +EXAMPLE_PATH=src/objective-c/examples/SwiftSample \ +SCHEME=SwiftSample \ +./build_one_example.sh diff --git a/src/objective-c/tests/build_one_example.sh b/src/objective-c/tests/build_one_example.sh new file mode 100755 index 00000000000..131fbd780df --- /dev/null +++ b/src/objective-c/tests/build_one_example.sh @@ -0,0 +1,62 @@ +#!/bin/bash +# Copyright 2015, Google Inc. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +# Don't run this script standalone. Instead, run from the repository root: +# ./tools/run_tests/run_tests.py -l objc + +set -e + +# Params: +# EXAMPLE_PATH - directory of the example +# SCHEME - scheme of the example, used by xcodebuild + +# CocoaPods requires the terminal to be using UTF-8 encoding. +export LANG=en_US.UTF-8 + +cd `dirname $0`/../../.. + +cd $EXAMPLE_PATH + +# clean the directory +git checkout . +git clean -df . +rm -rf Pods + +pod install + +set -o pipefail +XCODEBUILD_FILTER='(^===|^\*\*|\bfatal\b|\berror\b|\bwarning\b|\bfail)' +xcodebuild \ + clean build \ + -workspace *.xcworkspace \ + -scheme $SCHEME \ + -destination name="iPhone 6" \ + | egrep "$XCODEBUILD_FILTER" \ + | egrep -v "(GPBDictionary|GPBArray)" - diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 9ef12c5b07e..0d77f0dbf97 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -425,7 +425,7 @@ class PythonLanguage(object): return [] def build_steps(self): - return [['tools/run_tests/build_python.sh', tox_env] + return [['tools/run_tests/build_python.sh', tox_env] for tox_env in self._tox_envs] def post_tests_steps(self): @@ -602,6 +602,8 @@ class ObjCLanguage(object): def test_specs(self): return [self.config.job_spec(['src/objective-c/tests/run_tests.sh'], None, + environ=_FORCE_ENVIRON_FOR_WRAPPERS), + self.config.job_spec(['src/objective-c/tests/build_example_test.sh'], None, environ=_FORCE_ENVIRON_FOR_WRAPPERS)] def pre_build_steps(self): From f95a4898abd5888c34b31354cdbe9a909958aa08 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Tue, 21 Jun 2016 23:27:46 -0700 Subject: [PATCH 152/280] Fixed format issues --- src/objective-c/tests/build_example_test.sh | 2 +- src/objective-c/tests/build_one_example.sh | 2 +- tools/run_tests/run_tests.py | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/objective-c/tests/build_example_test.sh b/src/objective-c/tests/build_example_test.sh index 3cc0c552349..73405695e8e 100755 --- a/src/objective-c/tests/build_example_test.sh +++ b/src/objective-c/tests/build_example_test.sh @@ -1,5 +1,5 @@ #!/bin/bash -# Copyright 2015, Google Inc. +# Copyright 2016, Google Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/src/objective-c/tests/build_one_example.sh b/src/objective-c/tests/build_one_example.sh index 131fbd780df..56f686319e1 100755 --- a/src/objective-c/tests/build_one_example.sh +++ b/src/objective-c/tests/build_one_example.sh @@ -1,5 +1,5 @@ #!/bin/bash -# Copyright 2015, Google Inc. +# Copyright 2016, Google Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 0d77f0dbf97..580afc32de4 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -603,8 +603,8 @@ class ObjCLanguage(object): def test_specs(self): return [self.config.job_spec(['src/objective-c/tests/run_tests.sh'], None, environ=_FORCE_ENVIRON_FOR_WRAPPERS), - self.config.job_spec(['src/objective-c/tests/build_example_test.sh'], None, - environ=_FORCE_ENVIRON_FOR_WRAPPERS)] + self.config.job_spec(['src/objective-c/tests/build_example_test.sh'], + None, environ=_FORCE_ENVIRON_FOR_WRAPPERS)] def pre_build_steps(self): return [] From 892f8f952ae28696ff4d900aeffb0c05f1699f63 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Tue, 21 Jun 2016 23:46:32 -0700 Subject: [PATCH 153/280] Update c# readme. --- src/csharp/README.md | 62 ++++++++++---------------------------------- 1 file changed, 14 insertions(+), 48 deletions(-) diff --git a/src/csharp/README.md b/src/csharp/README.md index 201c5ab0b56..b33d87ed383 100644 --- a/src/csharp/README.md +++ b/src/csharp/README.md @@ -19,33 +19,13 @@ PREREQUISITES HOW TO USE -------------- -**Windows** +**Windows, Linux, Mac OS X** -- Open Visual Studio and start a new project/solution. +- Open Visual Studio / MonoDevelop / Xamarin Studio and start a new project/solution. - Add NuGet package `Grpc` as a dependency (Project options -> Manage NuGet Packages). - That will also pull all the transitive dependencies (including the gRPC native library that - gRPC C# is using internally). - -**Linux (Debian)** - -- Open MonoDevelop and start a new project/solution. - -- Add NuGet package `Grpc` as a dependency (Project -> Add NuGet packages). - That will also pull all the transitive dependencies (including the gRPC native library that - gRPC C# is using internally). - -- NOTE: gRPC C# doesn't have a good story yet for shipping precompiled Linux version of Protocol Buffers compiler (_protoc_) and the gRPC _protoc_ plugin. You can install them using [gRPC Linuxbrew instructions][]. - -**Mac OS X** - -- Open Xamarin Studio and start a new project/solution. - -- Add NuGet package `Grpc` as a dependency (Project -> Add NuGet packages). - That will also pull all the transitive dependencies (including the gRPC native library that - gRPC C# is using internally). -- NOTE: gRPC C# doesn't have a good story yet for shipping precompiled Mac OS X version of Protocol Buffers compiler (_protoc_) and the gRPC _protoc_ plugin. You can install them using [gRPC Homebrew instructions][]. +- To be able to generate code from Protocol Buffer (`.proto`) file definitions, add NuGet package `Grpc.Tools` that contains Protocol Buffers compiler (_protoc_) and the gRPC _protoc_ plugin. BUILD FROM SOURCE ----------------- @@ -61,26 +41,15 @@ If you are a user of gRPC C#, go to Usage section above. - Open `src\csharp\Grpc.sln` (path is relative to gRPC repository root) using Visual Studio -**Linux** +**Linux and Mac OS X** - The grpc_csharp_ext native library needs to be built so you can build the gRPC C# solution: - ```sh - # from the gRPC repository root - $ make CONFIG=dbg grpc_csharp_ext - ``` - -- Use MonoDevelop to open the solution Grpc.sln - -**Mac OS X** - -- The grpc_csharp_ext native library needs to be built so you can build the gRPC C# solution. - ```sh # from the gRPC repository root $ tools/run_tests/run_tests.py -c dbg -l csharp --build_only ``` -- Use Xamarin Studio to open the solution Grpc.sln +- Use MonoDevelop / Xamarin Studio to open the solution Grpc.sln RUNNING TESTS ------------- @@ -102,8 +71,9 @@ tools/run_tests/run_tests.py -l csharp DOCUMENTATION ------------- -- the gRPC C# reference documentation is available online at [grpc.io][] -- [Helloworld example][] +- [API Reference][] +- [Helloworld Example][] +- [RouteGuide Tutorial][] CONTENTS -------- @@ -111,15 +81,15 @@ CONTENTS - ext: The extension library that wraps C API to be more digestible by C#. - Grpc.Auth: - gRPC OAuth2 support. + gRPC OAuth2/JWT support. - Grpc.Core: The main gRPC C# library. - Grpc.Examples: API examples for math.proto - Grpc.Examples.MathClient: - An example client that sends some requests to math server. + An example client that sends requests to math server. - Grpc.Examples.MathServer: - An example client that sends some requests to math server. + An example server that implements a simple math service. - Grpc.IntegrationTesting: Cross-language gRPC implementation testing (interop testing). @@ -130,10 +100,6 @@ Internally, gRPC C# uses a native library written in C (gRPC C core) and invokes Prior to version 0.13, installing `grpc_csharp_ext` was required to make gRPC work on Linux and MacOS. Starting with version 0.13, we have improved the packaging story significantly and precompiled versions of the native library for all supported platforms are now shipped with the NuGet package. Just installing the `Grpc` NuGet package should be the only step needed to use gRPC C#, regardless of your platform (Windows, Linux or Mac) and the bitness (32 or 64bit). -[gRPC Linuxbrew instructions]:https://github.com/grpc/homebrew-grpc#quick-install-linux -[gRPC Homebrew instructions]:https://github.com/grpc/homebrew-grpc#quick-install-linux -[homebrew]:http://brew.sh -[gRPC install script]:https://raw.githubusercontent.com/grpc/homebrew-grpc/master/scripts/install -[grpc.io]: http://www.grpc.io/docs/installation/csharp.html -[Debian jessie-backports]:http://backports.debian.org/Instructions/ -[Helloworld example]:../../examples/csharp/helloworld +[API Reference]: http://www.grpc.io/grpc/csharp/ +[Helloworld Example]: ../../examples/csharp/helloworld +[RouteGuide Tutorial]: http://www.grpc.io/docs/tutorials/basic/csharp.html From db3ffe13cbb37ea797890c455a4543c9d030adf5 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Wed, 22 Jun 2016 01:12:42 -0700 Subject: [PATCH 154/280] Update examples with Cocoapods 1.0.0 --- .../AuthSample.xcodeproj/project.pbxproj | 28 ++++++++--- .../HelloWorld.xcodeproj/project.pbxproj | 24 ++++----- .../project.pbxproj | 18 +++---- .../Sample/Sample.xcodeproj/project.pbxproj | 50 ++++++++++++------- .../SwiftSample.xcodeproj/project.pbxproj | 42 ++++++++-------- 5 files changed, 95 insertions(+), 67 deletions(-) diff --git a/examples/objective-c/auth_sample/AuthSample.xcodeproj/project.pbxproj b/examples/objective-c/auth_sample/AuthSample.xcodeproj/project.pbxproj index 51a39c578c7..ab7419c9bcd 100644 --- a/examples/objective-c/auth_sample/AuthSample.xcodeproj/project.pbxproj +++ b/examples/objective-c/auth_sample/AuthSample.xcodeproj/project.pbxproj @@ -116,11 +116,12 @@ isa = PBXNativeTarget; buildConfigurationList = 63E1E9A21B28CB2100EF0978 /* Build configuration list for PBXNativeTarget "AuthSample" */; buildPhases = ( - DAABBA7B5788A39108D7CA83 /* Check Pods Manifest.lock */, + DAABBA7B5788A39108D7CA83 /* [CP] Check Pods Manifest.lock */, 63E1E9781B28CB2000EF0978 /* Sources */, 63E1E9791B28CB2000EF0978 /* Frameworks */, 63E1E97A1B28CB2000EF0978 /* Resources */, - AEFCCC69DD59CE8F6EB769D7 /* Copy Pods Resources */, + AEFCCC69DD59CE8F6EB769D7 /* [CP] Copy Pods Resources */, + D24F6598302C412D4B863D6F /* [CP] Embed Pods Frameworks */, ); buildRules = ( ); @@ -177,14 +178,14 @@ /* End PBXResourcesBuildPhase section */ /* Begin PBXShellScriptBuildPhase section */ - AEFCCC69DD59CE8F6EB769D7 /* Copy Pods Resources */ = { + AEFCCC69DD59CE8F6EB769D7 /* [CP] Copy Pods Resources */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Copy Pods Resources"; + name = "[CP] Copy Pods Resources"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; @@ -192,14 +193,29 @@ shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-AuthSample/Pods-AuthSample-resources.sh\"\n"; showEnvVarsInLog = 0; }; - DAABBA7B5788A39108D7CA83 /* Check Pods Manifest.lock */ = { + D24F6598302C412D4B863D6F /* [CP] Embed Pods Frameworks */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Check Pods Manifest.lock"; + name = "[CP] Embed Pods Frameworks"; + outputPaths = ( + ); + runOnlyForDeploymentPostprocessing = 0; + shellPath = /bin/sh; + shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-AuthSample/Pods-AuthSample-frameworks.sh\"\n"; + showEnvVarsInLog = 0; + }; + DAABBA7B5788A39108D7CA83 /* [CP] Check Pods Manifest.lock */ = { + isa = PBXShellScriptBuildPhase; + buildActionMask = 2147483647; + files = ( + ); + inputPaths = ( + ); + name = "[CP] Check Pods Manifest.lock"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; diff --git a/examples/objective-c/helloworld/HelloWorld.xcodeproj/project.pbxproj b/examples/objective-c/helloworld/HelloWorld.xcodeproj/project.pbxproj index 250f009996f..df5c40cda20 100644 --- a/examples/objective-c/helloworld/HelloWorld.xcodeproj/project.pbxproj +++ b/examples/objective-c/helloworld/HelloWorld.xcodeproj/project.pbxproj @@ -7,7 +7,6 @@ objects = { /* Begin PBXBuildFile section */ - 3EF35C14BDC2B65E21837F02 /* libPods.a in Frameworks */ = {isa = PBXBuildFile; fileRef = 43AB08B32839A6700EA00DD4 /* libPods.a */; }; 5E3690661B2A23800040F884 /* main.m in Sources */ = {isa = PBXBuildFile; fileRef = 5E3690651B2A23800040F884 /* main.m */; }; 5E3690691B2A23800040F884 /* AppDelegate.m in Sources */ = {isa = PBXBuildFile; fileRef = 5E3690681B2A23800040F884 /* AppDelegate.m */; }; 5E36906C1B2A23800040F884 /* ViewController.m in Sources */ = {isa = PBXBuildFile; fileRef = 5E36906B1B2A23800040F884 /* ViewController.m */; }; @@ -18,7 +17,6 @@ /* Begin PBXFileReference section */ 0C432EF610DB15C0F47A66BB /* Pods-HelloWorld.release.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-HelloWorld.release.xcconfig"; path = "Pods/Target Support Files/Pods-HelloWorld/Pods-HelloWorld.release.xcconfig"; sourceTree = ""; }; - 43AB08B32839A6700EA00DD4 /* libPods.a */ = {isa = PBXFileReference; explicitFileType = archive.ar; includeInIndex = 0; path = libPods.a; sourceTree = BUILT_PRODUCTS_DIR; }; 5E3690601B2A23800040F884 /* HelloWorld.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = HelloWorld.app; sourceTree = BUILT_PRODUCTS_DIR; }; 5E3690641B2A23800040F884 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = ""; }; 5E3690651B2A23800040F884 /* main.m */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.objc; path = main.m; sourceTree = ""; }; @@ -37,7 +35,6 @@ buildActionMask = 2147483647; files = ( EF61CF6AE2536A31D47F0E63 /* libPods-HelloWorld.a in Frameworks */, - 3EF35C14BDC2B65E21837F02 /* libPods.a in Frameworks */, ); runOnlyForDeploymentPostprocessing = 0; }; @@ -88,7 +85,6 @@ isa = PBXGroup; children = ( 6B4E1F55F8A2EC95A0E7EE88 /* libPods-HelloWorld.a */, - 43AB08B32839A6700EA00DD4 /* libPods.a */, ); name = Frameworks; sourceTree = ""; @@ -109,12 +105,12 @@ isa = PBXNativeTarget; buildConfigurationList = 5E3690831B2A23810040F884 /* Build configuration list for PBXNativeTarget "HelloWorld" */; buildPhases = ( - ACF9162361FB8F24C70657DE /* Check Pods Manifest.lock */, + ACF9162361FB8F24C70657DE /* [CP] Check Pods Manifest.lock */, 5E36905C1B2A23800040F884 /* Sources */, 5E36905D1B2A23800040F884 /* Frameworks */, 5E36905E1B2A23800040F884 /* Resources */, - 4C7D815378D98AB3BFC1A7D5 /* Copy Pods Resources */, - BB76529986A8BFAF19A385B1 /* Embed Pods Frameworks */, + 4C7D815378D98AB3BFC1A7D5 /* [CP] Copy Pods Resources */, + BB76529986A8BFAF19A385B1 /* [CP] Embed Pods Frameworks */, ); buildRules = ( ); @@ -170,14 +166,14 @@ /* End PBXResourcesBuildPhase section */ /* Begin PBXShellScriptBuildPhase section */ - 4C7D815378D98AB3BFC1A7D5 /* Copy Pods Resources */ = { + 4C7D815378D98AB3BFC1A7D5 /* [CP] Copy Pods Resources */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Copy Pods Resources"; + name = "[CP] Copy Pods Resources"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; @@ -185,14 +181,14 @@ shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-HelloWorld/Pods-HelloWorld-resources.sh\"\n"; showEnvVarsInLog = 0; }; - ACF9162361FB8F24C70657DE /* Check Pods Manifest.lock */ = { + ACF9162361FB8F24C70657DE /* [CP] Check Pods Manifest.lock */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Check Pods Manifest.lock"; + name = "[CP] Check Pods Manifest.lock"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; @@ -200,19 +196,19 @@ shellScript = "diff \"${PODS_ROOT}/../Podfile.lock\" \"${PODS_ROOT}/Manifest.lock\" > /dev/null\nif [[ $? != 0 ]] ; then\n cat << EOM\nerror: The sandbox is not in sync with the Podfile.lock. Run 'pod install' or update your CocoaPods installation.\nEOM\n exit 1\nfi\n"; showEnvVarsInLog = 0; }; - BB76529986A8BFAF19A385B1 /* Embed Pods Frameworks */ = { + BB76529986A8BFAF19A385B1 /* [CP] Embed Pods Frameworks */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Embed Pods Frameworks"; + name = "[CP] Embed Pods Frameworks"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; shellPath = /bin/sh; - shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods/Pods-frameworks.sh\"\n"; + shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-HelloWorld/Pods-HelloWorld-frameworks.sh\"\n"; showEnvVarsInLog = 0; }; /* End PBXShellScriptBuildPhase section */ diff --git a/examples/objective-c/route_guide/RouteGuideClient.xcodeproj/project.pbxproj b/examples/objective-c/route_guide/RouteGuideClient.xcodeproj/project.pbxproj index f99775562c0..0bb84b3b905 100644 --- a/examples/objective-c/route_guide/RouteGuideClient.xcodeproj/project.pbxproj +++ b/examples/objective-c/route_guide/RouteGuideClient.xcodeproj/project.pbxproj @@ -116,12 +116,12 @@ isa = PBXNativeTarget; buildConfigurationList = 632527A31B1D0396003073D9 /* Build configuration list for PBXNativeTarget "RouteGuideClient" */; buildPhases = ( - C6FC30AD2376EC04317237C5 /* Check Pods Manifest.lock */, + C6FC30AD2376EC04317237C5 /* [CP] Check Pods Manifest.lock */, 632527791B1D0395003073D9 /* Sources */, 6325277A1B1D0395003073D9 /* Frameworks */, 6325277B1B1D0395003073D9 /* Resources */, - FFE0BCF30339E7A50A989EAB /* Copy Pods Resources */, - B5388EC5A25E89021740B916 /* Embed Pods Frameworks */, + FFE0BCF30339E7A50A989EAB /* [CP] Copy Pods Resources */, + B5388EC5A25E89021740B916 /* [CP] Embed Pods Frameworks */, ); buildRules = ( ); @@ -178,14 +178,14 @@ /* End PBXResourcesBuildPhase section */ /* Begin PBXShellScriptBuildPhase section */ - B5388EC5A25E89021740B916 /* Embed Pods Frameworks */ = { + B5388EC5A25E89021740B916 /* [CP] Embed Pods Frameworks */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Embed Pods Frameworks"; + name = "[CP] Embed Pods Frameworks"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; @@ -193,14 +193,14 @@ shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-RouteGuideClient/Pods-RouteGuideClient-frameworks.sh\"\n"; showEnvVarsInLog = 0; }; - C6FC30AD2376EC04317237C5 /* Check Pods Manifest.lock */ = { + C6FC30AD2376EC04317237C5 /* [CP] Check Pods Manifest.lock */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Check Pods Manifest.lock"; + name = "[CP] Check Pods Manifest.lock"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; @@ -208,14 +208,14 @@ shellScript = "diff \"${PODS_ROOT}/../Podfile.lock\" \"${PODS_ROOT}/Manifest.lock\" > /dev/null\nif [[ $? != 0 ]] ; then\n cat << EOM\nerror: The sandbox is not in sync with the Podfile.lock. Run 'pod install' or update your CocoaPods installation.\nEOM\n exit 1\nfi\n"; showEnvVarsInLog = 0; }; - FFE0BCF30339E7A50A989EAB /* Copy Pods Resources */ = { + FFE0BCF30339E7A50A989EAB /* [CP] Copy Pods Resources */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Copy Pods Resources"; + name = "[CP] Copy Pods Resources"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; diff --git a/src/objective-c/examples/Sample/Sample.xcodeproj/project.pbxproj b/src/objective-c/examples/Sample/Sample.xcodeproj/project.pbxproj index 611eb6032d5..5c2a6d14f96 100644 --- a/src/objective-c/examples/Sample/Sample.xcodeproj/project.pbxproj +++ b/src/objective-c/examples/Sample/Sample.xcodeproj/project.pbxproj @@ -7,16 +7,16 @@ objects = { /* Begin PBXBuildFile section */ + 426A5020E0E158A101BCA1D9 /* libPods-Sample.a in Frameworks */ = {isa = PBXBuildFile; fileRef = C20055928615A6F8434E26B4 /* libPods-Sample.a */; }; 6369A2701A9322E20015FC5C /* main.m in Sources */ = {isa = PBXBuildFile; fileRef = 6369A26F1A9322E20015FC5C /* main.m */; }; 6369A2731A9322E20015FC5C /* AppDelegate.m in Sources */ = {isa = PBXBuildFile; fileRef = 6369A2721A9322E20015FC5C /* AppDelegate.m */; }; 6369A2761A9322E20015FC5C /* ViewController.m in Sources */ = {isa = PBXBuildFile; fileRef = 6369A2751A9322E20015FC5C /* ViewController.m */; }; 6369A2791A9322E20015FC5C /* Main.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 6369A2771A9322E20015FC5C /* Main.storyboard */; }; 6369A27B1A9322E20015FC5C /* Images.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 6369A27A1A9322E20015FC5C /* Images.xcassets */; }; - FC81FE63CA655031F3524EC0 /* libPods.a in Frameworks */ = {isa = PBXBuildFile; fileRef = 2DC7B7C4C0410F43B9621631 /* libPods.a */; }; /* End PBXBuildFile section */ /* Begin PBXFileReference section */ - 2DC7B7C4C0410F43B9621631 /* libPods.a */ = {isa = PBXFileReference; explicitFileType = archive.ar; includeInIndex = 0; path = libPods.a; sourceTree = BUILT_PRODUCTS_DIR; }; + 5A8C9F4B28733B249DE4AB6D /* Pods-Sample.release.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-Sample.release.xcconfig"; path = "Pods/Target Support Files/Pods-Sample/Pods-Sample.release.xcconfig"; sourceTree = ""; }; 6369A26A1A9322E20015FC5C /* Sample.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = Sample.app; sourceTree = BUILT_PRODUCTS_DIR; }; 6369A26E1A9322E20015FC5C /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = ""; }; 6369A26F1A9322E20015FC5C /* main.m */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.objc; path = main.m; sourceTree = ""; }; @@ -26,8 +26,8 @@ 6369A2751A9322E20015FC5C /* ViewController.m */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.objc; path = ViewController.m; sourceTree = ""; }; 6369A2781A9322E20015FC5C /* Base */ = {isa = PBXFileReference; lastKnownFileType = file.storyboard; name = Base; path = Base.lproj/Main.storyboard; sourceTree = ""; }; 6369A27A1A9322E20015FC5C /* Images.xcassets */ = {isa = PBXFileReference; lastKnownFileType = folder.assetcatalog; path = Images.xcassets; sourceTree = ""; }; - AC29DD6FCDF962F519FEBB0D /* Pods.debug.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = Pods.debug.xcconfig; path = "Pods/Target Support Files/Pods/Pods.debug.xcconfig"; sourceTree = ""; }; - C68330F8D451CC6ACEABA09F /* Pods.release.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = Pods.release.xcconfig; path = "Pods/Target Support Files/Pods/Pods.release.xcconfig"; sourceTree = ""; }; + C20055928615A6F8434E26B4 /* libPods-Sample.a */ = {isa = PBXFileReference; explicitFileType = archive.ar; includeInIndex = 0; path = "libPods-Sample.a"; sourceTree = BUILT_PRODUCTS_DIR; }; + E3C01DF315C4E7433BCEC6E6 /* Pods-Sample.debug.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-Sample.debug.xcconfig"; path = "Pods/Target Support Files/Pods-Sample/Pods-Sample.debug.xcconfig"; sourceTree = ""; }; /* End PBXFileReference section */ /* Begin PBXFrameworksBuildPhase section */ @@ -35,7 +35,7 @@ isa = PBXFrameworksBuildPhase; buildActionMask = 2147483647; files = ( - FC81FE63CA655031F3524EC0 /* libPods.a in Frameworks */, + 426A5020E0E158A101BCA1D9 /* libPods-Sample.a in Frameworks */, ); runOnlyForDeploymentPostprocessing = 0; }; @@ -86,8 +86,8 @@ AB3331C9AE6488E61B2B094E /* Pods */ = { isa = PBXGroup; children = ( - AC29DD6FCDF962F519FEBB0D /* Pods.debug.xcconfig */, - C68330F8D451CC6ACEABA09F /* Pods.release.xcconfig */, + E3C01DF315C4E7433BCEC6E6 /* Pods-Sample.debug.xcconfig */, + 5A8C9F4B28733B249DE4AB6D /* Pods-Sample.release.xcconfig */, ); name = Pods; sourceTree = ""; @@ -95,7 +95,7 @@ C4C2C5219053E079C9EFB930 /* Frameworks */ = { isa = PBXGroup; children = ( - 2DC7B7C4C0410F43B9621631 /* libPods.a */, + C20055928615A6F8434E26B4 /* libPods-Sample.a */, ); name = Frameworks; sourceTree = ""; @@ -107,11 +107,12 @@ isa = PBXNativeTarget; buildConfigurationList = 6369A28D1A9322E20015FC5C /* Build configuration list for PBXNativeTarget "Sample" */; buildPhases = ( - 41F7486D8F66994B0BFB84AF /* Check Pods Manifest.lock */, + 41F7486D8F66994B0BFB84AF /* [CP] Check Pods Manifest.lock */, 6369A2661A9322E20015FC5C /* Sources */, 6369A2671A9322E20015FC5C /* Frameworks */, 6369A2681A9322E20015FC5C /* Resources */, - 04554623324BE4A838846086 /* Copy Pods Resources */, + 04554623324BE4A838846086 /* [CP] Copy Pods Resources */, + C7FAD018D05AB5F0B0FE81E2 /* [CP] Embed Pods Frameworks */, ); buildRules = ( ); @@ -167,29 +168,29 @@ /* End PBXResourcesBuildPhase section */ /* Begin PBXShellScriptBuildPhase section */ - 04554623324BE4A838846086 /* Copy Pods Resources */ = { + 04554623324BE4A838846086 /* [CP] Copy Pods Resources */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Copy Pods Resources"; + name = "[CP] Copy Pods Resources"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; shellPath = /bin/sh; - shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods/Pods-resources.sh\"\n"; + shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-Sample/Pods-Sample-resources.sh\"\n"; showEnvVarsInLog = 0; }; - 41F7486D8F66994B0BFB84AF /* Check Pods Manifest.lock */ = { + 41F7486D8F66994B0BFB84AF /* [CP] Check Pods Manifest.lock */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Check Pods Manifest.lock"; + name = "[CP] Check Pods Manifest.lock"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; @@ -197,6 +198,21 @@ shellScript = "diff \"${PODS_ROOT}/../Podfile.lock\" \"${PODS_ROOT}/Manifest.lock\" > /dev/null\nif [[ $? != 0 ]] ; then\n cat << EOM\nerror: The sandbox is not in sync with the Podfile.lock. Run 'pod install' or update your CocoaPods installation.\nEOM\n exit 1\nfi\n"; showEnvVarsInLog = 0; }; + C7FAD018D05AB5F0B0FE81E2 /* [CP] Embed Pods Frameworks */ = { + isa = PBXShellScriptBuildPhase; + buildActionMask = 2147483647; + files = ( + ); + inputPaths = ( + ); + name = "[CP] Embed Pods Frameworks"; + outputPaths = ( + ); + runOnlyForDeploymentPostprocessing = 0; + shellPath = /bin/sh; + shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-Sample/Pods-Sample-frameworks.sh\"\n"; + showEnvVarsInLog = 0; + }; /* End PBXShellScriptBuildPhase section */ /* Begin PBXSourcesBuildPhase section */ @@ -304,7 +320,7 @@ }; 6369A28E1A9322E20015FC5C /* Debug */ = { isa = XCBuildConfiguration; - baseConfigurationReference = AC29DD6FCDF962F519FEBB0D /* Pods.debug.xcconfig */; + baseConfigurationReference = E3C01DF315C4E7433BCEC6E6 /* Pods-Sample.debug.xcconfig */; buildSettings = { ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; INFOPLIST_FILE = Sample/Info.plist; @@ -315,7 +331,7 @@ }; 6369A28F1A9322E20015FC5C /* Release */ = { isa = XCBuildConfiguration; - baseConfigurationReference = C68330F8D451CC6ACEABA09F /* Pods.release.xcconfig */; + baseConfigurationReference = 5A8C9F4B28733B249DE4AB6D /* Pods-Sample.release.xcconfig */; buildSettings = { ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; INFOPLIST_FILE = Sample/Info.plist; diff --git a/src/objective-c/examples/SwiftSample/SwiftSample.xcodeproj/project.pbxproj b/src/objective-c/examples/SwiftSample/SwiftSample.xcodeproj/project.pbxproj index 2f5716082bf..2a1b30f2cf9 100644 --- a/src/objective-c/examples/SwiftSample/SwiftSample.xcodeproj/project.pbxproj +++ b/src/objective-c/examples/SwiftSample/SwiftSample.xcodeproj/project.pbxproj @@ -7,15 +7,14 @@ objects = { /* Begin PBXBuildFile section */ - 253D3A297105CA46DA960A11 /* libPods.a in Frameworks */ = {isa = PBXBuildFile; fileRef = DC58ACA18DCCB1553531B885 /* libPods.a */; }; 633BFFC81B950B210007E424 /* AppDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 633BFFC71B950B210007E424 /* AppDelegate.swift */; }; 633BFFCA1B950B210007E424 /* ViewController.swift in Sources */ = {isa = PBXBuildFile; fileRef = 633BFFC91B950B210007E424 /* ViewController.swift */; }; 633BFFCD1B950B210007E424 /* Main.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 633BFFCB1B950B210007E424 /* Main.storyboard */; }; 633BFFCF1B950B210007E424 /* Images.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 633BFFCE1B950B210007E424 /* Images.xcassets */; }; + 92EDB1408A1E1E7DDAB25D9C /* libPods-SwiftSample.a in Frameworks */ = {isa = PBXBuildFile; fileRef = 69BB5C6CA3C1F97E007AC527 /* libPods-SwiftSample.a */; }; /* End PBXBuildFile section */ /* Begin PBXFileReference section */ - 12C7B447AA80E624D93B5C54 /* Pods.debug.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = Pods.debug.xcconfig; path = "Pods/Target Support Files/Pods/Pods.debug.xcconfig"; sourceTree = ""; }; 633BFFC21B950B210007E424 /* SwiftSample.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = SwiftSample.app; sourceTree = BUILT_PRODUCTS_DIR; }; 633BFFC61B950B210007E424 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = ""; }; 633BFFC71B950B210007E424 /* AppDelegate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AppDelegate.swift; sourceTree = ""; }; @@ -23,8 +22,9 @@ 633BFFCC1B950B210007E424 /* Base */ = {isa = PBXFileReference; lastKnownFileType = file.storyboard; name = Base; path = Base.lproj/Main.storyboard; sourceTree = ""; }; 633BFFCE1B950B210007E424 /* Images.xcassets */ = {isa = PBXFileReference; lastKnownFileType = folder.assetcatalog; path = Images.xcassets; sourceTree = ""; }; 6367AD231B951655007FD3A4 /* Bridging-Header.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = "Bridging-Header.h"; sourceTree = ""; }; - C335CBC4C160E0D9EDEE646B /* Pods.release.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = Pods.release.xcconfig; path = "Pods/Target Support Files/Pods/Pods.release.xcconfig"; sourceTree = ""; }; - DC58ACA18DCCB1553531B885 /* libPods.a */ = {isa = PBXFileReference; explicitFileType = archive.ar; includeInIndex = 0; path = libPods.a; sourceTree = BUILT_PRODUCTS_DIR; }; + 69BB5C6CA3C1F97E007AC527 /* libPods-SwiftSample.a */ = {isa = PBXFileReference; explicitFileType = archive.ar; includeInIndex = 0; path = "libPods-SwiftSample.a"; sourceTree = BUILT_PRODUCTS_DIR; }; + A7E614A494D89D01BB395761 /* Pods-SwiftSample.debug.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-SwiftSample.debug.xcconfig"; path = "Pods/Target Support Files/Pods-SwiftSample/Pods-SwiftSample.debug.xcconfig"; sourceTree = ""; }; + C314E3E246AF23AC29B38FCF /* Pods-SwiftSample.release.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-SwiftSample.release.xcconfig"; path = "Pods/Target Support Files/Pods-SwiftSample/Pods-SwiftSample.release.xcconfig"; sourceTree = ""; }; /* End PBXFileReference section */ /* Begin PBXFrameworksBuildPhase section */ @@ -32,7 +32,7 @@ isa = PBXFrameworksBuildPhase; buildActionMask = 2147483647; files = ( - 253D3A297105CA46DA960A11 /* libPods.a in Frameworks */, + 92EDB1408A1E1E7DDAB25D9C /* libPods-SwiftSample.a in Frameworks */, ); runOnlyForDeploymentPostprocessing = 0; }; @@ -42,8 +42,8 @@ 31F283C976AE97586C17CCD9 /* Pods */ = { isa = PBXGroup; children = ( - 12C7B447AA80E624D93B5C54 /* Pods.debug.xcconfig */, - C335CBC4C160E0D9EDEE646B /* Pods.release.xcconfig */, + A7E614A494D89D01BB395761 /* Pods-SwiftSample.debug.xcconfig */, + C314E3E246AF23AC29B38FCF /* Pods-SwiftSample.release.xcconfig */, ); name = Pods; sourceTree = ""; @@ -90,7 +90,7 @@ 9D63A7F6423989BA306810CA /* Frameworks */ = { isa = PBXGroup; children = ( - DC58ACA18DCCB1553531B885 /* libPods.a */, + 69BB5C6CA3C1F97E007AC527 /* libPods-SwiftSample.a */, ); name = Frameworks; sourceTree = ""; @@ -102,12 +102,12 @@ isa = PBXNativeTarget; buildConfigurationList = 633BFFE11B950B210007E424 /* Build configuration list for PBXNativeTarget "SwiftSample" */; buildPhases = ( - 6BEEB33CA2705D7D2F2210E6 /* Check Pods Manifest.lock */, + 6BEEB33CA2705D7D2F2210E6 /* [CP] Check Pods Manifest.lock */, 633BFFBE1B950B210007E424 /* Sources */, 633BFFBF1B950B210007E424 /* Frameworks */, 633BFFC01B950B210007E424 /* Resources */, - AC2F6F9AB1C090BB0BEE6E4D /* Copy Pods Resources */, - A1738A987353B0BF2C64F0F7 /* Embed Pods Frameworks */, + AC2F6F9AB1C090BB0BEE6E4D /* [CP] Copy Pods Resources */, + A1738A987353B0BF2C64F0F7 /* [CP] Embed Pods Frameworks */, ); buildRules = ( ); @@ -164,14 +164,14 @@ /* End PBXResourcesBuildPhase section */ /* Begin PBXShellScriptBuildPhase section */ - 6BEEB33CA2705D7D2F2210E6 /* Check Pods Manifest.lock */ = { + 6BEEB33CA2705D7D2F2210E6 /* [CP] Check Pods Manifest.lock */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Check Pods Manifest.lock"; + name = "[CP] Check Pods Manifest.lock"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; @@ -179,34 +179,34 @@ shellScript = "diff \"${PODS_ROOT}/../Podfile.lock\" \"${PODS_ROOT}/Manifest.lock\" > /dev/null\nif [[ $? != 0 ]] ; then\n cat << EOM\nerror: The sandbox is not in sync with the Podfile.lock. Run 'pod install' or update your CocoaPods installation.\nEOM\n exit 1\nfi\n"; showEnvVarsInLog = 0; }; - A1738A987353B0BF2C64F0F7 /* Embed Pods Frameworks */ = { + A1738A987353B0BF2C64F0F7 /* [CP] Embed Pods Frameworks */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Embed Pods Frameworks"; + name = "[CP] Embed Pods Frameworks"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; shellPath = /bin/sh; - shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods/Pods-frameworks.sh\"\n"; + shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-SwiftSample/Pods-SwiftSample-frameworks.sh\"\n"; showEnvVarsInLog = 0; }; - AC2F6F9AB1C090BB0BEE6E4D /* Copy Pods Resources */ = { + AC2F6F9AB1C090BB0BEE6E4D /* [CP] Copy Pods Resources */ = { isa = PBXShellScriptBuildPhase; buildActionMask = 2147483647; files = ( ); inputPaths = ( ); - name = "Copy Pods Resources"; + name = "[CP] Copy Pods Resources"; outputPaths = ( ); runOnlyForDeploymentPostprocessing = 0; shellPath = /bin/sh; - shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods/Pods-resources.sh\"\n"; + shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-SwiftSample/Pods-SwiftSample-resources.sh\"\n"; showEnvVarsInLog = 0; }; /* End PBXShellScriptBuildPhase section */ @@ -320,7 +320,7 @@ }; 633BFFE21B950B210007E424 /* Debug */ = { isa = XCBuildConfiguration; - baseConfigurationReference = 12C7B447AA80E624D93B5C54 /* Pods.debug.xcconfig */; + baseConfigurationReference = A7E614A494D89D01BB395761 /* Pods-SwiftSample.debug.xcconfig */; buildSettings = { ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; INFOPLIST_FILE = Info.plist; @@ -333,7 +333,7 @@ }; 633BFFE31B950B210007E424 /* Release */ = { isa = XCBuildConfiguration; - baseConfigurationReference = C335CBC4C160E0D9EDEE646B /* Pods.release.xcconfig */; + baseConfigurationReference = C314E3E246AF23AC29B38FCF /* Pods-SwiftSample.release.xcconfig */; buildSettings = { ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; INFOPLIST_FILE = Info.plist; From c0668c837b0ee48f64d0a912e2c6120f2ab719c5 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Wed, 22 Jun 2016 01:30:47 -0700 Subject: [PATCH 155/280] Increase timeout_seconds for build_example_test --- tools/run_tests/run_tests.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 580afc32de4..5c8f6f2a0e7 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -604,7 +604,8 @@ class ObjCLanguage(object): return [self.config.job_spec(['src/objective-c/tests/run_tests.sh'], None, environ=_FORCE_ENVIRON_FOR_WRAPPERS), self.config.job_spec(['src/objective-c/tests/build_example_test.sh'], - None, environ=_FORCE_ENVIRON_FOR_WRAPPERS)] + timeout_seconds=15*60, None, + environ=_FORCE_ENVIRON_FOR_WRAPPERS)] def pre_build_steps(self): return [] From a2dd8385f95259b8a6886902dac87d5df7c7b953 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 22 Jun 2016 10:26:03 -0700 Subject: [PATCH 156/280] Use pipe fds instead of event fds for the test --- test/core/iomgr/ev_epoll_linux_test.c | 55 +++++++++++++++++---------- 1 file changed, 34 insertions(+), 21 deletions(-) diff --git a/test/core/iomgr/ev_epoll_linux_test.c b/test/core/iomgr/ev_epoll_linux_test.c index 034f17fd58b..35eb6791302 100644 --- a/test/core/iomgr/ev_epoll_linux_test.c +++ b/test/core/iomgr/ev_epoll_linux_test.c @@ -34,9 +34,8 @@ #include "src/core/lib/iomgr/ev_epoll_linux.h" #include "src/core/lib/iomgr/ev_posix.h" -#include +#include #include -#include #include #include @@ -55,28 +54,29 @@ typedef struct test_fd { grpc_fd *fd; } test_fd; -static void test_fd_init(test_fd *fds, int num_fds) { +/* num_fds should be an even number */ +static void test_fd_init(test_fd *tfds, int *fds, int num_fds) { int i; for (i = 0; i < num_fds; i++) { - fds[i].inner_fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); - fds[i].fd = grpc_fd_create(fds[i].inner_fd, "test_fd"); + tfds[i].inner_fd = fds[i]; + tfds[i].fd = grpc_fd_create(fds[i], "test_fd"); } } -static void test_fd_cleanup(grpc_exec_ctx *exec_ctx, test_fd *fds, +static void test_fd_cleanup(grpc_exec_ctx *exec_ctx, test_fd *tfds, int num_fds) { int release_fd; int i; for (i = 0; i < num_fds; i++) { - grpc_fd_shutdown(exec_ctx, fds[i].fd); + grpc_fd_shutdown(exec_ctx, tfds[i].fd); grpc_exec_ctx_flush(exec_ctx); - grpc_fd_orphan(exec_ctx, fds[i].fd, NULL, &release_fd, "test_fd_cleanup"); + grpc_fd_orphan(exec_ctx, tfds[i].fd, NULL, &release_fd, "test_fd_cleanup"); grpc_exec_ctx_flush(exec_ctx); - GPR_ASSERT(release_fd == fds[i].inner_fd); - close(fds[i].inner_fd); + GPR_ASSERT(release_fd == tfds[i].inner_fd); + close(tfds[i].inner_fd); } } @@ -121,12 +121,25 @@ static void test_pollset_cleanup(grpc_exec_ctx *exec_ctx, * */ static void test_add_fd_to_pollset() { grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; - test_fd fds[NUM_FDS]; + test_fd tfds[NUM_FDS]; + int fds[NUM_FDS]; test_pollset pollsets[NUM_POLLSETS]; void *expected_pi = NULL; int i; + int r; + + /* Create some dummy file descriptors (using pipe fds for this test. Could be + anything). Also NUM_FDS should be even for this test. */ + for (i = 0; i < NUM_FDS; i = i + 2) { + r = pipe(fds + i); + if (r != 0) { + gpr_log(GPR_ERROR, "Error in creating pipe. %d (%s)", errno, + strerror(errno)); + return; + } + } - test_fd_init(fds, NUM_FDS); + test_fd_init(tfds, fds, NUM_FDS); test_pollset_init(pollsets, NUM_POLLSETS); /*Step 1. @@ -156,41 +169,41 @@ static void test_add_fd_to_pollset() { /* == Step 1 == */ for (i = 0; i <= 2; i++) { - grpc_pollset_add_fd(&exec_ctx, pollsets[0].pollset, fds[i].fd); + grpc_pollset_add_fd(&exec_ctx, pollsets[0].pollset, tfds[i].fd); grpc_exec_ctx_flush(&exec_ctx); } for (i = 3; i <= 4; i++) { - grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, fds[i].fd); + grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, tfds[i].fd); grpc_exec_ctx_flush(&exec_ctx); } for (i = 5; i <= 7; i++) { - grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, fds[i].fd); + grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, tfds[i].fd); grpc_exec_ctx_flush(&exec_ctx); } /* == Step 2 == */ for (i = 0; i <= 1; i++) { - grpc_pollset_add_fd(&exec_ctx, pollsets[3].pollset, fds[i].fd); + grpc_pollset_add_fd(&exec_ctx, pollsets[3].pollset, tfds[i].fd); grpc_exec_ctx_flush(&exec_ctx); } /* == Step 3 == */ - grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, fds[0].fd); + grpc_pollset_add_fd(&exec_ctx, pollsets[1].pollset, tfds[0].fd); grpc_exec_ctx_flush(&exec_ctx); /* == Step 4 == */ - grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, fds[3].fd); + grpc_pollset_add_fd(&exec_ctx, pollsets[2].pollset, tfds[3].fd); grpc_exec_ctx_flush(&exec_ctx); /* All polling islands are merged at this point */ /* Compare Fd:0's polling island with that of all other Fds */ - expected_pi = grpc_fd_get_polling_island(fds[0].fd); + expected_pi = grpc_fd_get_polling_island(tfds[0].fd); for (i = 1; i < NUM_FDS; i++) { GPR_ASSERT(grpc_are_polling_islands_equal( - expected_pi, grpc_fd_get_polling_island(fds[i].fd))); + expected_pi, grpc_fd_get_polling_island(tfds[i].fd))); } /* Compare Fd:0's polling island with that of all other pollsets */ @@ -199,7 +212,7 @@ static void test_add_fd_to_pollset() { expected_pi, grpc_pollset_get_polling_island(pollsets[i].pollset))); } - test_fd_cleanup(&exec_ctx, fds, NUM_FDS); + test_fd_cleanup(&exec_ctx, tfds, NUM_FDS); test_pollset_cleanup(&exec_ctx, pollsets, NUM_POLLSETS); grpc_exec_ctx_finish(&exec_ctx); } From 82393efe0c930f7756e9d0e427d2e9ae61e0a88e Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Wed, 22 Jun 2016 10:29:17 -0700 Subject: [PATCH 157/280] Fix run_test.py --- tools/run_tests/run_tests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 5c8f6f2a0e7..2a404df0d41 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -604,7 +604,7 @@ class ObjCLanguage(object): return [self.config.job_spec(['src/objective-c/tests/run_tests.sh'], None, environ=_FORCE_ENVIRON_FOR_WRAPPERS), self.config.job_spec(['src/objective-c/tests/build_example_test.sh'], - timeout_seconds=15*60, None, + None, timeout_seconds=15*60, environ=_FORCE_ENVIRON_FOR_WRAPPERS)] def pre_build_steps(self): From 0d896ef906f6a57f10832764611598d457d0e947 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Tue, 21 Jun 2016 18:17:27 -0700 Subject: [PATCH 158/280] fix reading of compressed byte_buffer in C# --- src/csharp/ext/grpc_csharp_ext.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/csharp/ext/grpc_csharp_ext.c b/src/csharp/ext/grpc_csharp_ext.c index 4782654250c..9b8d050ea51 100644 --- a/src/csharp/ext/grpc_csharp_ext.c +++ b/src/csharp/ext/grpc_csharp_ext.c @@ -249,10 +249,12 @@ grpcsharp_batch_context_recv_initial_metadata( GPR_EXPORT intptr_t GPR_CALLTYPE grpcsharp_batch_context_recv_message_length( const grpcsharp_batch_context *ctx) { + grpc_byte_buffer_reader reader; if (!ctx->recv_message) { return -1; } - return (intptr_t)grpc_byte_buffer_length(ctx->recv_message); + grpc_byte_buffer_reader_init(&reader, ctx->recv_message); + return (intptr_t)grpc_byte_buffer_length(reader.buffer_out); } /* From 4047d5d4b642a3eb33100feda4b82d04179263f6 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Tue, 21 Jun 2016 20:53:56 -0700 Subject: [PATCH 159/280] add test that C# can read compressed messages --- src/csharp/Grpc.Core.Tests/CompressionTest.cs | 26 +++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/src/csharp/Grpc.Core.Tests/CompressionTest.cs b/src/csharp/Grpc.Core.Tests/CompressionTest.cs index 378c81851c0..f2f2931e480 100644 --- a/src/csharp/Grpc.Core.Tests/CompressionTest.cs +++ b/src/csharp/Grpc.Core.Tests/CompressionTest.cs @@ -34,6 +34,7 @@ using System; using System.Diagnostics; using System.Linq; +using System.Text; using System.Threading; using System.Threading.Tasks; using Grpc.Core; @@ -118,5 +119,30 @@ namespace Grpc.Core.Tests await call.ResponseStream.ToListAsync(); } + + [Test] + public void CanReadCompressedMessages() + { + var compressionMetadata = new Metadata + { + { new Metadata.Entry("grpc-internal-encoding-request", "gzip") } + }; + + helper.UnaryHandler = new UnaryServerMethod(async (req, context) => + { + await context.WriteResponseHeadersAsync(compressionMetadata); + return req; + }); + + var stringBuilder = new StringBuilder(); + for (int i = 0; i < 200000; i++) + { + stringBuilder.Append('a'); + } + var request = stringBuilder.ToString(); + var response = Calls.BlockingUnaryCall(helper.CreateUnaryCall(new CallOptions(compressionMetadata)), request); + + Assert.AreEqual(request, response); + } } } From 67ceba531971a2344a11be21ef5685b66e2609f1 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Tue, 21 Jun 2016 18:29:09 -0700 Subject: [PATCH 160/280] regenerate messages.proto --- .../Grpc.IntegrationTesting/Messages.cs | 414 +++++++++++++----- 1 file changed, 301 insertions(+), 113 deletions(-) diff --git a/src/csharp/Grpc.IntegrationTesting/Messages.cs b/src/csharp/Grpc.IntegrationTesting/Messages.cs index d42501aa5b8..1240db128b0 100644 --- a/src/csharp/Grpc.IntegrationTesting/Messages.cs +++ b/src/csharp/Grpc.IntegrationTesting/Messages.cs @@ -24,46 +24,48 @@ namespace Grpc.Testing { byte[] descriptorData = global::System.Convert.FromBase64String( string.Concat( "CiVzcmMvcHJvdG8vZ3JwYy90ZXN0aW5nL21lc3NhZ2VzLnByb3RvEgxncnBj", - "LnRlc3RpbmciQAoHUGF5bG9hZBInCgR0eXBlGAEgASgOMhkuZ3JwYy50ZXN0", - "aW5nLlBheWxvYWRUeXBlEgwKBGJvZHkYAiABKAwiKwoKRWNob1N0YXR1cxIM", - "CgRjb2RlGAEgASgFEg8KB21lc3NhZ2UYAiABKAkioQIKDVNpbXBsZVJlcXVl", - "c3QSMAoNcmVzcG9uc2VfdHlwZRgBIAEoDjIZLmdycGMudGVzdGluZy5QYXls", - "b2FkVHlwZRIVCg1yZXNwb25zZV9zaXplGAIgASgFEiYKB3BheWxvYWQYAyAB", - "KAsyFS5ncnBjLnRlc3RpbmcuUGF5bG9hZBIVCg1maWxsX3VzZXJuYW1lGAQg", - "ASgIEhgKEGZpbGxfb2F1dGhfc2NvcGUYBSABKAgSOwoUcmVzcG9uc2VfY29t", - "cHJlc3Npb24YBiABKA4yHS5ncnBjLnRlc3RpbmcuQ29tcHJlc3Npb25UeXBl", - "EjEKD3Jlc3BvbnNlX3N0YXR1cxgHIAEoCzIYLmdycGMudGVzdGluZy5FY2hv", - "U3RhdHVzIl8KDlNpbXBsZVJlc3BvbnNlEiYKB3BheWxvYWQYASABKAsyFS5n", - "cnBjLnRlc3RpbmcuUGF5bG9hZBIQCgh1c2VybmFtZRgCIAEoCRITCgtvYXV0", - "aF9zY29wZRgDIAEoCSJDChlTdHJlYW1pbmdJbnB1dENhbGxSZXF1ZXN0EiYK", - "B3BheWxvYWQYASABKAsyFS5ncnBjLnRlc3RpbmcuUGF5bG9hZCI9ChpTdHJl", - "YW1pbmdJbnB1dENhbGxSZXNwb25zZRIfChdhZ2dyZWdhdGVkX3BheWxvYWRf", - "c2l6ZRgBIAEoBSI3ChJSZXNwb25zZVBhcmFtZXRlcnMSDAoEc2l6ZRgBIAEo", - "BRITCgtpbnRlcnZhbF91cxgCIAEoBSKlAgoaU3RyZWFtaW5nT3V0cHV0Q2Fs", - "bFJlcXVlc3QSMAoNcmVzcG9uc2VfdHlwZRgBIAEoDjIZLmdycGMudGVzdGlu", - "Zy5QYXlsb2FkVHlwZRI9ChNyZXNwb25zZV9wYXJhbWV0ZXJzGAIgAygLMiAu", - "Z3JwYy50ZXN0aW5nLlJlc3BvbnNlUGFyYW1ldGVycxImCgdwYXlsb2FkGAMg", - "ASgLMhUuZ3JwYy50ZXN0aW5nLlBheWxvYWQSOwoUcmVzcG9uc2VfY29tcHJl", - "c3Npb24YBiABKA4yHS5ncnBjLnRlc3RpbmcuQ29tcHJlc3Npb25UeXBlEjEK", - "D3Jlc3BvbnNlX3N0YXR1cxgHIAEoCzIYLmdycGMudGVzdGluZy5FY2hvU3Rh", - "dHVzIkUKG1N0cmVhbWluZ091dHB1dENhbGxSZXNwb25zZRImCgdwYXlsb2Fk", - "GAEgASgLMhUuZ3JwYy50ZXN0aW5nLlBheWxvYWQiMwoPUmVjb25uZWN0UGFy", - "YW1zEiAKGG1heF9yZWNvbm5lY3RfYmFja29mZl9tcxgBIAEoBSIzCg1SZWNv", - "bm5lY3RJbmZvEg4KBnBhc3NlZBgBIAEoCBISCgpiYWNrb2ZmX21zGAIgAygF", - "Kj8KC1BheWxvYWRUeXBlEhAKDENPTVBSRVNTQUJMRRAAEhIKDlVOQ09NUFJF", - "U1NBQkxFEAESCgoGUkFORE9NEAIqMgoPQ29tcHJlc3Npb25UeXBlEggKBE5P", - "TkUQABIICgRHWklQEAESCwoHREVGTEFURRACYgZwcm90bzM=")); + "LnRlc3RpbmciGgoJQm9vbFZhbHVlEg0KBXZhbHVlGAEgASgIIkAKB1BheWxv", + "YWQSJwoEdHlwZRgBIAEoDjIZLmdycGMudGVzdGluZy5QYXlsb2FkVHlwZRIM", + "CgRib2R5GAIgASgMIisKCkVjaG9TdGF0dXMSDAoEY29kZRgBIAEoBRIPCgdt", + "ZXNzYWdlGAIgASgJIs4CCg1TaW1wbGVSZXF1ZXN0EjAKDXJlc3BvbnNlX3R5", + "cGUYASABKA4yGS5ncnBjLnRlc3RpbmcuUGF5bG9hZFR5cGUSFQoNcmVzcG9u", + "c2Vfc2l6ZRgCIAEoBRImCgdwYXlsb2FkGAMgASgLMhUuZ3JwYy50ZXN0aW5n", + "LlBheWxvYWQSFQoNZmlsbF91c2VybmFtZRgEIAEoCBIYChBmaWxsX29hdXRo", + "X3Njb3BlGAUgASgIEjQKE3Jlc3BvbnNlX2NvbXByZXNzZWQYBiABKAsyFy5n", + "cnBjLnRlc3RpbmcuQm9vbFZhbHVlEjEKD3Jlc3BvbnNlX3N0YXR1cxgHIAEo", + "CzIYLmdycGMudGVzdGluZy5FY2hvU3RhdHVzEjIKEWV4cGVjdF9jb21wcmVz", + "c2VkGAggASgLMhcuZ3JwYy50ZXN0aW5nLkJvb2xWYWx1ZSJfCg5TaW1wbGVS", + "ZXNwb25zZRImCgdwYXlsb2FkGAEgASgLMhUuZ3JwYy50ZXN0aW5nLlBheWxv", + "YWQSEAoIdXNlcm5hbWUYAiABKAkSEwoLb2F1dGhfc2NvcGUYAyABKAkidwoZ", + "U3RyZWFtaW5nSW5wdXRDYWxsUmVxdWVzdBImCgdwYXlsb2FkGAEgASgLMhUu", + "Z3JwYy50ZXN0aW5nLlBheWxvYWQSMgoRZXhwZWN0X2NvbXByZXNzZWQYAiAB", + "KAsyFy5ncnBjLnRlc3RpbmcuQm9vbFZhbHVlIj0KGlN0cmVhbWluZ0lucHV0", + "Q2FsbFJlc3BvbnNlEh8KF2FnZ3JlZ2F0ZWRfcGF5bG9hZF9zaXplGAEgASgF", + "ImQKElJlc3BvbnNlUGFyYW1ldGVycxIMCgRzaXplGAEgASgFEhMKC2ludGVy", + "dmFsX3VzGAIgASgFEisKCmNvbXByZXNzZWQYAyABKAsyFy5ncnBjLnRlc3Rp", + "bmcuQm9vbFZhbHVlIugBChpTdHJlYW1pbmdPdXRwdXRDYWxsUmVxdWVzdBIw", + "Cg1yZXNwb25zZV90eXBlGAEgASgOMhkuZ3JwYy50ZXN0aW5nLlBheWxvYWRU", + "eXBlEj0KE3Jlc3BvbnNlX3BhcmFtZXRlcnMYAiADKAsyIC5ncnBjLnRlc3Rp", + "bmcuUmVzcG9uc2VQYXJhbWV0ZXJzEiYKB3BheWxvYWQYAyABKAsyFS5ncnBj", + "LnRlc3RpbmcuUGF5bG9hZBIxCg9yZXNwb25zZV9zdGF0dXMYByABKAsyGC5n", + "cnBjLnRlc3RpbmcuRWNob1N0YXR1cyJFChtTdHJlYW1pbmdPdXRwdXRDYWxs", + "UmVzcG9uc2USJgoHcGF5bG9hZBgBIAEoCzIVLmdycGMudGVzdGluZy5QYXls", + "b2FkIjMKD1JlY29ubmVjdFBhcmFtcxIgChhtYXhfcmVjb25uZWN0X2JhY2tv", + "ZmZfbXMYASABKAUiMwoNUmVjb25uZWN0SW5mbxIOCgZwYXNzZWQYASABKAgS", + "EgoKYmFja29mZl9tcxgCIAMoBSofCgtQYXlsb2FkVHlwZRIQCgxDT01QUkVT", + "U0FCTEUQAGIGcHJvdG8z")); descriptor = pbr::FileDescriptor.FromGeneratedCode(descriptorData, new pbr::FileDescriptor[] { }, - new pbr::GeneratedClrTypeInfo(new[] {typeof(global::Grpc.Testing.PayloadType), typeof(global::Grpc.Testing.CompressionType), }, new pbr::GeneratedClrTypeInfo[] { + new pbr::GeneratedClrTypeInfo(new[] {typeof(global::Grpc.Testing.PayloadType), }, new pbr::GeneratedClrTypeInfo[] { + new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.BoolValue), global::Grpc.Testing.BoolValue.Parser, new[]{ "Value" }, null, null, null), new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.Payload), global::Grpc.Testing.Payload.Parser, new[]{ "Type", "Body" }, null, null, null), new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.EchoStatus), global::Grpc.Testing.EchoStatus.Parser, new[]{ "Code", "Message" }, null, null, null), - new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.SimpleRequest), global::Grpc.Testing.SimpleRequest.Parser, new[]{ "ResponseType", "ResponseSize", "Payload", "FillUsername", "FillOauthScope", "ResponseCompression", "ResponseStatus" }, null, null, null), + new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.SimpleRequest), global::Grpc.Testing.SimpleRequest.Parser, new[]{ "ResponseType", "ResponseSize", "Payload", "FillUsername", "FillOauthScope", "ResponseCompressed", "ResponseStatus", "ExpectCompressed" }, null, null, null), new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.SimpleResponse), global::Grpc.Testing.SimpleResponse.Parser, new[]{ "Payload", "Username", "OauthScope" }, null, null, null), - new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.StreamingInputCallRequest), global::Grpc.Testing.StreamingInputCallRequest.Parser, new[]{ "Payload" }, null, null, null), + new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.StreamingInputCallRequest), global::Grpc.Testing.StreamingInputCallRequest.Parser, new[]{ "Payload", "ExpectCompressed" }, null, null, null), new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.StreamingInputCallResponse), global::Grpc.Testing.StreamingInputCallResponse.Parser, new[]{ "AggregatedPayloadSize" }, null, null, null), - new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.ResponseParameters), global::Grpc.Testing.ResponseParameters.Parser, new[]{ "Size", "IntervalUs" }, null, null, null), - new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.StreamingOutputCallRequest), global::Grpc.Testing.StreamingOutputCallRequest.Parser, new[]{ "ResponseType", "ResponseParameters", "Payload", "ResponseCompression", "ResponseStatus" }, null, null, null), + new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.ResponseParameters), global::Grpc.Testing.ResponseParameters.Parser, new[]{ "Size", "IntervalUs", "Compressed" }, null, null, null), + new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.StreamingOutputCallRequest), global::Grpc.Testing.StreamingOutputCallRequest.Parser, new[]{ "ResponseType", "ResponseParameters", "Payload", "ResponseStatus" }, null, null, null), new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.StreamingOutputCallResponse), global::Grpc.Testing.StreamingOutputCallResponse.Parser, new[]{ "Payload" }, null, null, null), new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.ReconnectParams), global::Grpc.Testing.ReconnectParams.Parser, new[]{ "MaxReconnectBackoffMs" }, null, null, null), new pbr::GeneratedClrTypeInfo(typeof(global::Grpc.Testing.ReconnectInfo), global::Grpc.Testing.ReconnectInfo.Parser, new[]{ "Passed", "BackoffMs" }, null, null, null) @@ -74,6 +76,7 @@ namespace Grpc.Testing { } #region Enums /// + /// DEPRECATED, don't use. To be removed shortly. /// The type of payload that should be returned. /// public enum PayloadType { @@ -81,31 +84,122 @@ namespace Grpc.Testing { /// Compressable text format. ///
[pbr::OriginalName("COMPRESSABLE")] Compressable = 0, - /// - /// Uncompressable binary format. - /// - [pbr::OriginalName("UNCOMPRESSABLE")] Uncompressable = 1, - /// - /// Randomly chosen from all other formats defined in this enum. - /// - [pbr::OriginalName("RANDOM")] Random = 2, } + #endregion + + #region Messages /// - /// Compression algorithms + /// TODO(dgq): Go back to using well-known types once + /// https://github.com/grpc/grpc/issues/6980 has been fixed. + /// import "google/protobuf/wrappers.proto"; /// - public enum CompressionType { + [global::System.Diagnostics.DebuggerNonUserCodeAttribute()] + public sealed partial class BoolValue : pb::IMessage { + private static readonly pb::MessageParser _parser = new pb::MessageParser(() => new BoolValue()); + public static pb::MessageParser Parser { get { return _parser; } } + + public static pbr::MessageDescriptor Descriptor { + get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[0]; } + } + + pbr::MessageDescriptor pb::IMessage.Descriptor { + get { return Descriptor; } + } + + public BoolValue() { + OnConstruction(); + } + + partial void OnConstruction(); + + public BoolValue(BoolValue other) : this() { + value_ = other.value_; + } + + public BoolValue Clone() { + return new BoolValue(this); + } + + /// Field number for the "value" field. + public const int ValueFieldNumber = 1; + private bool value_; /// - /// No compression + /// The bool value. /// - [pbr::OriginalName("NONE")] None = 0, - [pbr::OriginalName("GZIP")] Gzip = 1, - [pbr::OriginalName("DEFLATE")] Deflate = 2, - } + public bool Value { + get { return value_; } + set { + value_ = value; + } + } - #endregion + public override bool Equals(object other) { + return Equals(other as BoolValue); + } + + public bool Equals(BoolValue other) { + if (ReferenceEquals(other, null)) { + return false; + } + if (ReferenceEquals(other, this)) { + return true; + } + if (Value != other.Value) return false; + return true; + } + + public override int GetHashCode() { + int hash = 1; + if (Value != false) hash ^= Value.GetHashCode(); + return hash; + } + + public override string ToString() { + return pb::JsonFormatter.ToDiagnosticString(this); + } + + public void WriteTo(pb::CodedOutputStream output) { + if (Value != false) { + output.WriteRawTag(8); + output.WriteBool(Value); + } + } + + public int CalculateSize() { + int size = 0; + if (Value != false) { + size += 1 + 1; + } + return size; + } + + public void MergeFrom(BoolValue other) { + if (other == null) { + return; + } + if (other.Value != false) { + Value = other.Value; + } + } + + public void MergeFrom(pb::CodedInputStream input) { + uint tag; + while ((tag = input.ReadTag()) != 0) { + switch(tag) { + default: + input.SkipLastField(); + break; + case 8: { + Value = input.ReadBool(); + break; + } + } + } + } + + } - #region Messages /// /// A block of data, to simply increase gRPC message size. /// @@ -115,7 +209,7 @@ namespace Grpc.Testing { public static pb::MessageParser Parser { get { return _parser; } } public static pbr::MessageDescriptor Descriptor { - get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[0]; } + get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[1]; } } pbr::MessageDescriptor pb::IMessage.Descriptor { @@ -141,6 +235,7 @@ namespace Grpc.Testing { public const int TypeFieldNumber = 1; private global::Grpc.Testing.PayloadType type_ = 0; /// + /// DEPRECATED, don't use. To be removed shortly. /// The type of data in body. /// public global::Grpc.Testing.PayloadType Type { @@ -255,7 +350,7 @@ namespace Grpc.Testing { public static pb::MessageParser Parser { get { return _parser; } } public static pbr::MessageDescriptor Descriptor { - get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[1]; } + get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[2]; } } pbr::MessageDescriptor pb::IMessage.Descriptor { @@ -388,7 +483,7 @@ namespace Grpc.Testing { public static pb::MessageParser Parser { get { return _parser; } } public static pbr::MessageDescriptor Descriptor { - get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[2]; } + get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[3]; } } pbr::MessageDescriptor pb::IMessage.Descriptor { @@ -407,8 +502,9 @@ namespace Grpc.Testing { Payload = other.payload_ != null ? other.Payload.Clone() : null; fillUsername_ = other.fillUsername_; fillOauthScope_ = other.fillOauthScope_; - responseCompression_ = other.responseCompression_; + ResponseCompressed = other.responseCompressed_ != null ? other.ResponseCompressed.Clone() : null; ResponseStatus = other.responseStatus_ != null ? other.ResponseStatus.Clone() : null; + ExpectCompressed = other.expectCompressed_ != null ? other.ExpectCompressed.Clone() : null; } public SimpleRequest Clone() { @@ -419,6 +515,7 @@ namespace Grpc.Testing { public const int ResponseTypeFieldNumber = 1; private global::Grpc.Testing.PayloadType responseType_ = 0; /// + /// DEPRECATED, don't use. To be removed shortly. /// Desired payload type in the response from the server. /// If response_type is RANDOM, server randomly chooses one from other formats. /// @@ -434,7 +531,6 @@ namespace Grpc.Testing { private int responseSize_; /// /// Desired payload size in the response from the server. - /// If response_type is COMPRESSABLE, this denotes the size before compression. /// public int ResponseSize { get { return responseSize_; } @@ -482,16 +578,19 @@ namespace Grpc.Testing { } } - /// Field number for the "response_compression" field. - public const int ResponseCompressionFieldNumber = 6; - private global::Grpc.Testing.CompressionType responseCompression_ = 0; + /// Field number for the "response_compressed" field. + public const int ResponseCompressedFieldNumber = 6; + private global::Grpc.Testing.BoolValue responseCompressed_; /// - /// Compression algorithm to be used by the server for the response (stream) + /// Whether to request the server to compress the response. This field is + /// "nullable" in order to interoperate seamlessly with clients not able to + /// implement the full compression tests by introspecting the call to verify + /// the response's compression status. /// - public global::Grpc.Testing.CompressionType ResponseCompression { - get { return responseCompression_; } + public global::Grpc.Testing.BoolValue ResponseCompressed { + get { return responseCompressed_; } set { - responseCompression_ = value; + responseCompressed_ = value; } } @@ -508,6 +607,19 @@ namespace Grpc.Testing { } } + /// Field number for the "expect_compressed" field. + public const int ExpectCompressedFieldNumber = 8; + private global::Grpc.Testing.BoolValue expectCompressed_; + /// + /// Whether the server should expect this request to be compressed. + /// + public global::Grpc.Testing.BoolValue ExpectCompressed { + get { return expectCompressed_; } + set { + expectCompressed_ = value; + } + } + public override bool Equals(object other) { return Equals(other as SimpleRequest); } @@ -524,8 +636,9 @@ namespace Grpc.Testing { if (!object.Equals(Payload, other.Payload)) return false; if (FillUsername != other.FillUsername) return false; if (FillOauthScope != other.FillOauthScope) return false; - if (ResponseCompression != other.ResponseCompression) return false; + if (!object.Equals(ResponseCompressed, other.ResponseCompressed)) return false; if (!object.Equals(ResponseStatus, other.ResponseStatus)) return false; + if (!object.Equals(ExpectCompressed, other.ExpectCompressed)) return false; return true; } @@ -536,8 +649,9 @@ namespace Grpc.Testing { if (payload_ != null) hash ^= Payload.GetHashCode(); if (FillUsername != false) hash ^= FillUsername.GetHashCode(); if (FillOauthScope != false) hash ^= FillOauthScope.GetHashCode(); - if (ResponseCompression != 0) hash ^= ResponseCompression.GetHashCode(); + if (responseCompressed_ != null) hash ^= ResponseCompressed.GetHashCode(); if (responseStatus_ != null) hash ^= ResponseStatus.GetHashCode(); + if (expectCompressed_ != null) hash ^= ExpectCompressed.GetHashCode(); return hash; } @@ -566,14 +680,18 @@ namespace Grpc.Testing { output.WriteRawTag(40); output.WriteBool(FillOauthScope); } - if (ResponseCompression != 0) { - output.WriteRawTag(48); - output.WriteEnum((int) ResponseCompression); + if (responseCompressed_ != null) { + output.WriteRawTag(50); + output.WriteMessage(ResponseCompressed); } if (responseStatus_ != null) { output.WriteRawTag(58); output.WriteMessage(ResponseStatus); } + if (expectCompressed_ != null) { + output.WriteRawTag(66); + output.WriteMessage(ExpectCompressed); + } } public int CalculateSize() { @@ -593,12 +711,15 @@ namespace Grpc.Testing { if (FillOauthScope != false) { size += 1 + 1; } - if (ResponseCompression != 0) { - size += 1 + pb::CodedOutputStream.ComputeEnumSize((int) ResponseCompression); + if (responseCompressed_ != null) { + size += 1 + pb::CodedOutputStream.ComputeMessageSize(ResponseCompressed); } if (responseStatus_ != null) { size += 1 + pb::CodedOutputStream.ComputeMessageSize(ResponseStatus); } + if (expectCompressed_ != null) { + size += 1 + pb::CodedOutputStream.ComputeMessageSize(ExpectCompressed); + } return size; } @@ -624,8 +745,11 @@ namespace Grpc.Testing { if (other.FillOauthScope != false) { FillOauthScope = other.FillOauthScope; } - if (other.ResponseCompression != 0) { - ResponseCompression = other.ResponseCompression; + if (other.responseCompressed_ != null) { + if (responseCompressed_ == null) { + responseCompressed_ = new global::Grpc.Testing.BoolValue(); + } + ResponseCompressed.MergeFrom(other.ResponseCompressed); } if (other.responseStatus_ != null) { if (responseStatus_ == null) { @@ -633,6 +757,12 @@ namespace Grpc.Testing { } ResponseStatus.MergeFrom(other.ResponseStatus); } + if (other.expectCompressed_ != null) { + if (expectCompressed_ == null) { + expectCompressed_ = new global::Grpc.Testing.BoolValue(); + } + ExpectCompressed.MergeFrom(other.ExpectCompressed); + } } public void MergeFrom(pb::CodedInputStream input) { @@ -665,8 +795,11 @@ namespace Grpc.Testing { FillOauthScope = input.ReadBool(); break; } - case 48: { - responseCompression_ = (global::Grpc.Testing.CompressionType) input.ReadEnum(); + case 50: { + if (responseCompressed_ == null) { + responseCompressed_ = new global::Grpc.Testing.BoolValue(); + } + input.ReadMessage(responseCompressed_); break; } case 58: { @@ -676,6 +809,13 @@ namespace Grpc.Testing { input.ReadMessage(responseStatus_); break; } + case 66: { + if (expectCompressed_ == null) { + expectCompressed_ = new global::Grpc.Testing.BoolValue(); + } + input.ReadMessage(expectCompressed_); + break; + } } } } @@ -691,7 +831,7 @@ namespace Grpc.Testing { public static pb::MessageParser Parser { get { return _parser; } } public static pbr::MessageDescriptor Descriptor { - get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[3]; } + get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[4]; } } pbr::MessageDescriptor pb::IMessage.Descriptor { @@ -867,7 +1007,7 @@ namespace Grpc.Testing { public static pb::MessageParser Parser { get { return _parser; } } public static pbr::MessageDescriptor Descriptor { - get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[4]; } + get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[5]; } } pbr::MessageDescriptor pb::IMessage.Descriptor { @@ -882,6 +1022,7 @@ namespace Grpc.Testing { public StreamingInputCallRequest(StreamingInputCallRequest other) : this() { Payload = other.payload_ != null ? other.Payload.Clone() : null; + ExpectCompressed = other.expectCompressed_ != null ? other.ExpectCompressed.Clone() : null; } public StreamingInputCallRequest Clone() { @@ -901,6 +1042,22 @@ namespace Grpc.Testing { } } + /// Field number for the "expect_compressed" field. + public const int ExpectCompressedFieldNumber = 2; + private global::Grpc.Testing.BoolValue expectCompressed_; + /// + /// Whether the server should expect this request to be compressed. This field + /// is "nullable" in order to interoperate seamlessly with servers not able to + /// implement the full compression tests by introspecting the call to verify + /// the request's compression status. + /// + public global::Grpc.Testing.BoolValue ExpectCompressed { + get { return expectCompressed_; } + set { + expectCompressed_ = value; + } + } + public override bool Equals(object other) { return Equals(other as StreamingInputCallRequest); } @@ -913,12 +1070,14 @@ namespace Grpc.Testing { return true; } if (!object.Equals(Payload, other.Payload)) return false; + if (!object.Equals(ExpectCompressed, other.ExpectCompressed)) return false; return true; } public override int GetHashCode() { int hash = 1; if (payload_ != null) hash ^= Payload.GetHashCode(); + if (expectCompressed_ != null) hash ^= ExpectCompressed.GetHashCode(); return hash; } @@ -931,6 +1090,10 @@ namespace Grpc.Testing { output.WriteRawTag(10); output.WriteMessage(Payload); } + if (expectCompressed_ != null) { + output.WriteRawTag(18); + output.WriteMessage(ExpectCompressed); + } } public int CalculateSize() { @@ -938,6 +1101,9 @@ namespace Grpc.Testing { if (payload_ != null) { size += 1 + pb::CodedOutputStream.ComputeMessageSize(Payload); } + if (expectCompressed_ != null) { + size += 1 + pb::CodedOutputStream.ComputeMessageSize(ExpectCompressed); + } return size; } @@ -951,6 +1117,12 @@ namespace Grpc.Testing { } Payload.MergeFrom(other.Payload); } + if (other.expectCompressed_ != null) { + if (expectCompressed_ == null) { + expectCompressed_ = new global::Grpc.Testing.BoolValue(); + } + ExpectCompressed.MergeFrom(other.ExpectCompressed); + } } public void MergeFrom(pb::CodedInputStream input) { @@ -967,6 +1139,13 @@ namespace Grpc.Testing { input.ReadMessage(payload_); break; } + case 18: { + if (expectCompressed_ == null) { + expectCompressed_ = new global::Grpc.Testing.BoolValue(); + } + input.ReadMessage(expectCompressed_); + break; + } } } } @@ -982,7 +1161,7 @@ namespace Grpc.Testing { public static pb::MessageParser Parser { get { return _parser; } } public static pbr::MessageDescriptor Descriptor { - get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[5]; } + get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[6]; } } pbr::MessageDescriptor pb::IMessage.Descriptor { @@ -1091,7 +1270,7 @@ namespace Grpc.Testing { public static pb::MessageParser Parser { get { return _parser; } } public static pbr::MessageDescriptor Descriptor { - get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[6]; } + get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[7]; } } pbr::MessageDescriptor pb::IMessage.Descriptor { @@ -1107,6 +1286,7 @@ namespace Grpc.Testing { public ResponseParameters(ResponseParameters other) : this() { size_ = other.size_; intervalUs_ = other.intervalUs_; + Compressed = other.compressed_ != null ? other.Compressed.Clone() : null; } public ResponseParameters Clone() { @@ -1118,7 +1298,6 @@ namespace Grpc.Testing { private int size_; /// /// Desired payload sizes in responses from the server. - /// If response_type is COMPRESSABLE, this denotes the size before compression. /// public int Size { get { return size_; } @@ -1141,6 +1320,22 @@ namespace Grpc.Testing { } } + /// Field number for the "compressed" field. + public const int CompressedFieldNumber = 3; + private global::Grpc.Testing.BoolValue compressed_; + /// + /// Whether to request the server to compress the response. This field is + /// "nullable" in order to interoperate seamlessly with clients not able to + /// implement the full compression tests by introspecting the call to verify + /// the response's compression status. + /// + public global::Grpc.Testing.BoolValue Compressed { + get { return compressed_; } + set { + compressed_ = value; + } + } + public override bool Equals(object other) { return Equals(other as ResponseParameters); } @@ -1154,6 +1349,7 @@ namespace Grpc.Testing { } if (Size != other.Size) return false; if (IntervalUs != other.IntervalUs) return false; + if (!object.Equals(Compressed, other.Compressed)) return false; return true; } @@ -1161,6 +1357,7 @@ namespace Grpc.Testing { int hash = 1; if (Size != 0) hash ^= Size.GetHashCode(); if (IntervalUs != 0) hash ^= IntervalUs.GetHashCode(); + if (compressed_ != null) hash ^= Compressed.GetHashCode(); return hash; } @@ -1177,6 +1374,10 @@ namespace Grpc.Testing { output.WriteRawTag(16); output.WriteInt32(IntervalUs); } + if (compressed_ != null) { + output.WriteRawTag(26); + output.WriteMessage(Compressed); + } } public int CalculateSize() { @@ -1187,6 +1388,9 @@ namespace Grpc.Testing { if (IntervalUs != 0) { size += 1 + pb::CodedOutputStream.ComputeInt32Size(IntervalUs); } + if (compressed_ != null) { + size += 1 + pb::CodedOutputStream.ComputeMessageSize(Compressed); + } return size; } @@ -1200,6 +1404,12 @@ namespace Grpc.Testing { if (other.IntervalUs != 0) { IntervalUs = other.IntervalUs; } + if (other.compressed_ != null) { + if (compressed_ == null) { + compressed_ = new global::Grpc.Testing.BoolValue(); + } + Compressed.MergeFrom(other.Compressed); + } } public void MergeFrom(pb::CodedInputStream input) { @@ -1217,6 +1427,13 @@ namespace Grpc.Testing { IntervalUs = input.ReadInt32(); break; } + case 26: { + if (compressed_ == null) { + compressed_ = new global::Grpc.Testing.BoolValue(); + } + input.ReadMessage(compressed_); + break; + } } } } @@ -1232,7 +1449,7 @@ namespace Grpc.Testing { public static pb::MessageParser Parser { get { return _parser; } } public static pbr::MessageDescriptor Descriptor { - get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[7]; } + get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[8]; } } pbr::MessageDescriptor pb::IMessage.Descriptor { @@ -1249,7 +1466,6 @@ namespace Grpc.Testing { responseType_ = other.responseType_; responseParameters_ = other.responseParameters_.Clone(); Payload = other.payload_ != null ? other.Payload.Clone() : null; - responseCompression_ = other.responseCompression_; ResponseStatus = other.responseStatus_ != null ? other.ResponseStatus.Clone() : null; } @@ -1261,6 +1477,7 @@ namespace Grpc.Testing { public const int ResponseTypeFieldNumber = 1; private global::Grpc.Testing.PayloadType responseType_ = 0; /// + /// DEPRECATED, don't use. To be removed shortly. /// Desired payload type in the response from the server. /// If response_type is RANDOM, the payload from each response in the stream /// might be of different types. This is to simulate a mixed type of payload @@ -1298,19 +1515,6 @@ namespace Grpc.Testing { } } - /// Field number for the "response_compression" field. - public const int ResponseCompressionFieldNumber = 6; - private global::Grpc.Testing.CompressionType responseCompression_ = 0; - /// - /// Compression algorithm to be used by the server for the response (stream) - /// - public global::Grpc.Testing.CompressionType ResponseCompression { - get { return responseCompression_; } - set { - responseCompression_ = value; - } - } - /// Field number for the "response_status" field. public const int ResponseStatusFieldNumber = 7; private global::Grpc.Testing.EchoStatus responseStatus_; @@ -1338,7 +1542,6 @@ namespace Grpc.Testing { if (ResponseType != other.ResponseType) return false; if(!responseParameters_.Equals(other.responseParameters_)) return false; if (!object.Equals(Payload, other.Payload)) return false; - if (ResponseCompression != other.ResponseCompression) return false; if (!object.Equals(ResponseStatus, other.ResponseStatus)) return false; return true; } @@ -1348,7 +1551,6 @@ namespace Grpc.Testing { if (ResponseType != 0) hash ^= ResponseType.GetHashCode(); hash ^= responseParameters_.GetHashCode(); if (payload_ != null) hash ^= Payload.GetHashCode(); - if (ResponseCompression != 0) hash ^= ResponseCompression.GetHashCode(); if (responseStatus_ != null) hash ^= ResponseStatus.GetHashCode(); return hash; } @@ -1367,10 +1569,6 @@ namespace Grpc.Testing { output.WriteRawTag(26); output.WriteMessage(Payload); } - if (ResponseCompression != 0) { - output.WriteRawTag(48); - output.WriteEnum((int) ResponseCompression); - } if (responseStatus_ != null) { output.WriteRawTag(58); output.WriteMessage(ResponseStatus); @@ -1386,9 +1584,6 @@ namespace Grpc.Testing { if (payload_ != null) { size += 1 + pb::CodedOutputStream.ComputeMessageSize(Payload); } - if (ResponseCompression != 0) { - size += 1 + pb::CodedOutputStream.ComputeEnumSize((int) ResponseCompression); - } if (responseStatus_ != null) { size += 1 + pb::CodedOutputStream.ComputeMessageSize(ResponseStatus); } @@ -1409,9 +1604,6 @@ namespace Grpc.Testing { } Payload.MergeFrom(other.Payload); } - if (other.ResponseCompression != 0) { - ResponseCompression = other.ResponseCompression; - } if (other.responseStatus_ != null) { if (responseStatus_ == null) { responseStatus_ = new global::Grpc.Testing.EchoStatus(); @@ -1442,10 +1634,6 @@ namespace Grpc.Testing { input.ReadMessage(payload_); break; } - case 48: { - responseCompression_ = (global::Grpc.Testing.CompressionType) input.ReadEnum(); - break; - } case 58: { if (responseStatus_ == null) { responseStatus_ = new global::Grpc.Testing.EchoStatus(); @@ -1468,7 +1656,7 @@ namespace Grpc.Testing { public static pb::MessageParser Parser { get { return _parser; } } public static pbr::MessageDescriptor Descriptor { - get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[8]; } + get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[9]; } } pbr::MessageDescriptor pb::IMessage.Descriptor { @@ -1584,7 +1772,7 @@ namespace Grpc.Testing { public static pb::MessageParser Parser { get { return _parser; } } public static pbr::MessageDescriptor Descriptor { - get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[9]; } + get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[10]; } } pbr::MessageDescriptor pb::IMessage.Descriptor { @@ -1692,7 +1880,7 @@ namespace Grpc.Testing { public static pb::MessageParser Parser { get { return _parser; } } public static pbr::MessageDescriptor Descriptor { - get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[10]; } + get { return global::Grpc.Testing.MessagesReflection.Descriptor.MessageTypes[11]; } } pbr::MessageDescriptor pb::IMessage.Descriptor { From f8135f6b05238237ec5ed635ecf1f5a4ba9b3301 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Tue, 21 Jun 2016 18:41:09 -0700 Subject: [PATCH 161/280] remove occurences of compressable payload type --- .../Grpc.IntegrationTesting/InteropClient.cs | 23 ------------------- 1 file changed, 23 deletions(-) diff --git a/src/csharp/Grpc.IntegrationTesting/InteropClient.cs b/src/csharp/Grpc.IntegrationTesting/InteropClient.cs index d273867a6a7..834c276c64d 100644 --- a/src/csharp/Grpc.IntegrationTesting/InteropClient.cs +++ b/src/csharp/Grpc.IntegrationTesting/InteropClient.cs @@ -240,13 +240,11 @@ namespace Grpc.IntegrationTesting Console.WriteLine("running large_unary"); var request = new SimpleRequest { - ResponseType = PayloadType.Compressable, ResponseSize = 314159, Payload = CreateZerosPayload(271828) }; var response = client.UnaryCall(request); - Assert.AreEqual(PayloadType.Compressable, response.Payload.Type); Assert.AreEqual(314159, response.Payload.Body.Length); Console.WriteLine("Passed!"); } @@ -275,17 +273,12 @@ namespace Grpc.IntegrationTesting var request = new StreamingOutputCallRequest { - ResponseType = PayloadType.Compressable, ResponseParameters = { bodySizes.Select((size) => new ResponseParameters { Size = size }) } }; using (var call = client.StreamingOutputCall(request)) { var responseList = await call.ResponseStream.ToListAsync(); - foreach (var res in responseList) - { - Assert.AreEqual(PayloadType.Compressable, res.Payload.Type); - } CollectionAssert.AreEqual(bodySizes, responseList.Select((item) => item.Payload.Body.Length)); } Console.WriteLine("Passed!"); @@ -299,46 +292,38 @@ namespace Grpc.IntegrationTesting { await call.RequestStream.WriteAsync(new StreamingOutputCallRequest { - ResponseType = PayloadType.Compressable, ResponseParameters = { new ResponseParameters { Size = 31415 } }, Payload = CreateZerosPayload(27182) }); Assert.IsTrue(await call.ResponseStream.MoveNext()); - Assert.AreEqual(PayloadType.Compressable, call.ResponseStream.Current.Payload.Type); Assert.AreEqual(31415, call.ResponseStream.Current.Payload.Body.Length); await call.RequestStream.WriteAsync(new StreamingOutputCallRequest { - ResponseType = PayloadType.Compressable, ResponseParameters = { new ResponseParameters { Size = 9 } }, Payload = CreateZerosPayload(8) }); Assert.IsTrue(await call.ResponseStream.MoveNext()); - Assert.AreEqual(PayloadType.Compressable, call.ResponseStream.Current.Payload.Type); Assert.AreEqual(9, call.ResponseStream.Current.Payload.Body.Length); await call.RequestStream.WriteAsync(new StreamingOutputCallRequest { - ResponseType = PayloadType.Compressable, ResponseParameters = { new ResponseParameters { Size = 2653 } }, Payload = CreateZerosPayload(1828) }); Assert.IsTrue(await call.ResponseStream.MoveNext()); - Assert.AreEqual(PayloadType.Compressable, call.ResponseStream.Current.Payload.Type); Assert.AreEqual(2653, call.ResponseStream.Current.Payload.Body.Length); await call.RequestStream.WriteAsync(new StreamingOutputCallRequest { - ResponseType = PayloadType.Compressable, ResponseParameters = { new ResponseParameters { Size = 58979 } }, Payload = CreateZerosPayload(45904) }); Assert.IsTrue(await call.ResponseStream.MoveNext()); - Assert.AreEqual(PayloadType.Compressable, call.ResponseStream.Current.Payload.Type); Assert.AreEqual(58979, call.ResponseStream.Current.Payload.Body.Length); await call.RequestStream.CompleteAsync(); @@ -367,7 +352,6 @@ namespace Grpc.IntegrationTesting var request = new SimpleRequest { - ResponseType = PayloadType.Compressable, ResponseSize = 314159, Payload = CreateZerosPayload(271828), FillUsername = true, @@ -377,7 +361,6 @@ namespace Grpc.IntegrationTesting // not setting credentials here because they were set on channel already var response = client.UnaryCall(request); - Assert.AreEqual(PayloadType.Compressable, response.Payload.Type); Assert.AreEqual(314159, response.Payload.Body.Length); Assert.False(string.IsNullOrEmpty(response.OauthScope)); Assert.True(oauthScope.Contains(response.OauthScope)); @@ -391,7 +374,6 @@ namespace Grpc.IntegrationTesting var request = new SimpleRequest { - ResponseType = PayloadType.Compressable, ResponseSize = 314159, Payload = CreateZerosPayload(271828), FillUsername = true, @@ -400,7 +382,6 @@ namespace Grpc.IntegrationTesting // not setting credentials here because they were set on channel already var response = client.UnaryCall(request); - Assert.AreEqual(PayloadType.Compressable, response.Payload.Type); Assert.AreEqual(314159, response.Payload.Body.Length); Assert.AreEqual(GetEmailFromServiceAccountFile(), response.Username); Console.WriteLine("Passed!"); @@ -480,13 +461,11 @@ namespace Grpc.IntegrationTesting { await call.RequestStream.WriteAsync(new StreamingOutputCallRequest { - ResponseType = PayloadType.Compressable, ResponseParameters = { new ResponseParameters { Size = 31415 } }, Payload = CreateZerosPayload(27182) }); Assert.IsTrue(await call.ResponseStream.MoveNext()); - Assert.AreEqual(PayloadType.Compressable, call.ResponseStream.Current.Payload.Type); Assert.AreEqual(31415, call.ResponseStream.Current.Payload.Body.Length); cts.Cancel(); @@ -546,7 +525,6 @@ namespace Grpc.IntegrationTesting // step 1: test unary call var request = new SimpleRequest { - ResponseType = PayloadType.Compressable, ResponseSize = 314159, Payload = CreateZerosPayload(271828) }; @@ -565,7 +543,6 @@ namespace Grpc.IntegrationTesting // step 2: test full duplex call var request = new StreamingOutputCallRequest { - ResponseType = PayloadType.Compressable, ResponseParameters = { new ResponseParameters { Size = 31415 } }, Payload = CreateZerosPayload(27182) }; From a7daf1edc2b336908918589c4de632197ebb92dc Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Tue, 21 Jun 2016 19:11:49 -0700 Subject: [PATCH 162/280] implement C# client compression interop tests --- .../Grpc.IntegrationTesting/InteropClient.cs | 108 ++++++++++++++++++ 1 file changed, 108 insertions(+) diff --git a/src/csharp/Grpc.IntegrationTesting/InteropClient.cs b/src/csharp/Grpc.IntegrationTesting/InteropClient.cs index 834c276c64d..17ef587d168 100644 --- a/src/csharp/Grpc.IntegrationTesting/InteropClient.cs +++ b/src/csharp/Grpc.IntegrationTesting/InteropClient.cs @@ -222,6 +222,12 @@ namespace Grpc.IntegrationTesting case "unimplemented_method": RunUnimplementedMethod(new UnimplementedService.UnimplementedServiceClient(channel)); break; + case "client_compressed_unary": + RunClientCompressedUnary(client); + break; + case "client_compressed_streaming": + await RunClientCompressedStreamingAsync(client); + break; default: throw new ArgumentException("Unknown test case " + options.TestCase); } @@ -615,11 +621,113 @@ namespace Grpc.IntegrationTesting Console.WriteLine("Passed!"); } + public static void RunClientCompressedUnary(TestService.TestServiceClient client) + { + Console.WriteLine("running client_compressed_unary"); + var probeRequest = new SimpleRequest + { + ExpectCompressed = new BoolValue + { + Value = true // lie about compression + }, + ResponseSize = 314159, + Payload = CreateZerosPayload(271828) + }; + var e = Assert.Throws(() => client.UnaryCall(probeRequest, CreateClientCompressionMetadata(false))); + Assert.AreEqual(StatusCode.InvalidArgument, e.Status.StatusCode); + + var compressedRequest = new SimpleRequest + { + ExpectCompressed = new BoolValue + { + Value = true + }, + ResponseSize = 314159, + Payload = CreateZerosPayload(271828) + }; + var response1 = client.UnaryCall(compressedRequest, CreateClientCompressionMetadata(true)); + Assert.AreEqual(314159, response1.Payload.Body.Length); + + var uncompressedRequest = new SimpleRequest + { + ExpectCompressed = new BoolValue + { + Value = false + }, + ResponseSize = 314159, + Payload = CreateZerosPayload(271828) + }; + var response2 = client.UnaryCall(uncompressedRequest, CreateClientCompressionMetadata(false)); + Assert.AreEqual(314159, response2.Payload.Body.Length); + + Console.WriteLine("Passed!"); + } + + public static async Task RunClientCompressedStreamingAsync(TestService.TestServiceClient client) + { + Console.WriteLine("running client_compressed_streaming"); + try + { + var probeCall = client.StreamingInputCall(CreateClientCompressionMetadata(false)); + await probeCall.RequestStream.WriteAsync(new StreamingInputCallRequest + { + ExpectCompressed = new BoolValue + { + Value = true + }, + Payload = CreateZerosPayload(27182) + }); + + // cannot use Assert.ThrowsAsync because it uses Task.Wait and would deadlock. + await probeCall; + Assert.Fail(); + } + catch (RpcException e) + { + Assert.AreEqual(StatusCode.InvalidArgument, e.Status.StatusCode); + } + + var call = client.StreamingInputCall(CreateClientCompressionMetadata(true)); + await call.RequestStream.WriteAsync(new StreamingInputCallRequest + { + ExpectCompressed = new BoolValue + { + Value = true + }, + Payload = CreateZerosPayload(27182) + }); + + call.RequestStream.WriteOptions = new WriteOptions(WriteFlags.NoCompress); + await call.RequestStream.WriteAsync(new StreamingInputCallRequest + { + ExpectCompressed = new BoolValue + { + Value = false + }, + Payload = CreateZerosPayload(45904) + }); + await call.RequestStream.CompleteAsync(); + + var response = await call.ResponseAsync; + Assert.AreEqual(73086, response.AggregatedPayloadSize); + + Console.WriteLine("Passed!"); + } + private static Payload CreateZerosPayload(int size) { return new Payload { Body = ByteString.CopyFrom(new byte[size]) }; } + private static Metadata CreateClientCompressionMetadata(bool compressed) + { + var algorithmName = compressed ? "gzip" : "identity"; + return new Metadata + { + { new Metadata.Entry("grpc-internal-encoding-request", algorithmName) } + }; + } + // extracts the client_email field from service account file used for auth test cases private static string GetEmailFromServiceAccountFile() { From 9fc079fddd2fc1bad958d6f7f7063363437426a5 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Tue, 21 Jun 2016 19:53:31 -0700 Subject: [PATCH 163/280] enable client compression interop tests for C# --- tools/run_tests/run_interop_tests.py | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/tools/run_tests/run_interop_tests.py b/tools/run_tests/run_interop_tests.py index e3af721ee53..d76dd4b7d26 100755 --- a/tools/run_tests/run_interop_tests.py +++ b/tools/run_tests/run_interop_tests.py @@ -54,10 +54,13 @@ os.chdir(ROOT) _DEFAULT_SERVER_PORT=8080 -_SKIP_COMPRESSION = ['client_compressed_unary', - 'client_compressed_streaming', - 'server_compressed_unary', - 'server_compressed_streaming'] +_SKIP_CLIENT_COMPRESSION = ['client_compressed_unary', + 'client_compressed_streaming'] + +_SKIP_SERVER_COMPRESSION = ['server_compressed_unary', + 'server_compressed_streaming'] + +_SKIP_COMPRESSION = _SKIP_CLIENT_COMPRESSION + _SKIP_SERVER_COMPRESSION _SKIP_ADVANCED = ['custom_metadata', 'status_code_and_message', 'unimplemented_method'] @@ -113,7 +116,7 @@ class CSharpLanguage: return {} def unimplemented_test_cases(self): - return _SKIP_COMPRESSION + return _SKIP_SERVER_COMPRESSION def unimplemented_test_cases_server(self): return _SKIP_COMPRESSION From 606e35a4fb2a6e7dd711e7a2eec6c59bf6f2258b Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 22 Jun 2016 12:26:36 -0700 Subject: [PATCH 164/280] add C# constant for GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY --- src/csharp/Grpc.Core.Tests/CompressionTest.cs | 2 +- src/csharp/Grpc.Core/Metadata.cs | 7 +++++++ src/csharp/Grpc.IntegrationTesting/InteropClient.cs | 2 +- 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/src/csharp/Grpc.Core.Tests/CompressionTest.cs b/src/csharp/Grpc.Core.Tests/CompressionTest.cs index f2f2931e480..16be210508a 100644 --- a/src/csharp/Grpc.Core.Tests/CompressionTest.cs +++ b/src/csharp/Grpc.Core.Tests/CompressionTest.cs @@ -125,7 +125,7 @@ namespace Grpc.Core.Tests { var compressionMetadata = new Metadata { - { new Metadata.Entry("grpc-internal-encoding-request", "gzip") } + { new Metadata.Entry(Metadata.CompressionRequestAlgorithmMetadataKey, "gzip") } }; helper.UnaryHandler = new UnaryServerMethod(async (req, context) => diff --git a/src/csharp/Grpc.Core/Metadata.cs b/src/csharp/Grpc.Core/Metadata.cs index f73f720094a..915bec146c9 100644 --- a/src/csharp/Grpc.Core/Metadata.cs +++ b/src/csharp/Grpc.Core/Metadata.cs @@ -63,6 +63,13 @@ namespace Grpc.Core /// public static readonly Metadata Empty = new Metadata().Freeze(); + /// + /// To be used in initial metadata to request specific compression algorithm + /// for given call. Direct selection of compression algorithms is an internal + /// feature and is not part of public API. + /// + internal const string CompressionRequestAlgorithmMetadataKey = "grpc-internal-encoding-request"; + readonly List entries; bool readOnly; diff --git a/src/csharp/Grpc.IntegrationTesting/InteropClient.cs b/src/csharp/Grpc.IntegrationTesting/InteropClient.cs index 17ef587d168..e27fe5b3d80 100644 --- a/src/csharp/Grpc.IntegrationTesting/InteropClient.cs +++ b/src/csharp/Grpc.IntegrationTesting/InteropClient.cs @@ -724,7 +724,7 @@ namespace Grpc.IntegrationTesting var algorithmName = compressed ? "gzip" : "identity"; return new Metadata { - { new Metadata.Entry("grpc-internal-encoding-request", algorithmName) } + { new Metadata.Entry(Metadata.CompressionRequestAlgorithmMetadataKey, algorithmName) } }; } From b16e4518b765f82b9b65222bd56738cda44e7259 Mon Sep 17 00:00:00 2001 From: Nathaniel Manista Date: Tue, 21 Jun 2016 23:24:06 +0000 Subject: [PATCH 165/280] Add a test of metadata, status code, and details --- src/python/grpcio/tests/tests.json | 1 + .../tests/unit/_metadata_code_details_test.py | 523 ++++++++++++++++++ 2 files changed, 524 insertions(+) create mode 100644 src/python/grpcio/tests/unit/_metadata_code_details_test.py diff --git a/src/python/grpcio/tests/tests.json b/src/python/grpcio/tests/tests.json index 8e509621a84..e384a2fc135 100644 --- a/src/python/grpcio/tests/tests.json +++ b/src/python/grpcio/tests/tests.json @@ -24,6 +24,7 @@ "_implementations_test.ChannelCredentialsTest", "_insecure_interop_test.InsecureInteropTest", "_logging_pool_test.LoggingPoolTest", + "_metadata_code_details_test.MetadataCodeDetailsTest", "_metadata_test.MetadataTest", "_not_found_test.NotFoundTest", "_python_plugin_test.PythonPluginTest", diff --git a/src/python/grpcio/tests/unit/_metadata_code_details_test.py b/src/python/grpcio/tests/unit/_metadata_code_details_test.py new file mode 100644 index 00000000000..dd74268cbf1 --- /dev/null +++ b/src/python/grpcio/tests/unit/_metadata_code_details_test.py @@ -0,0 +1,523 @@ +# Copyright 2016, Google Inc. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""Tests application-provided metadata, status code, and details.""" + +import threading +import unittest + +import grpc +from grpc.framework.foundation import logging_pool + +from tests.unit import test_common +from tests.unit.framework.common import test_constants +from tests.unit.framework.common import test_control + +_SERIALIZED_REQUEST = b'\x46\x47\x48' +_SERIALIZED_RESPONSE = b'\x49\x50\x51' + +_REQUEST_SERIALIZER = lambda unused_request: _SERIALIZED_REQUEST +_REQUEST_DESERIALIZER = lambda unused_serialized_request: object() +_RESPONSE_SERIALIZER = lambda unused_response: _SERIALIZED_RESPONSE +_RESPONSE_DESERIALIZER = lambda unused_serialized_resopnse: object() + +_SERVICE = b'test.TestService' +_UNARY_UNARY = b'UnaryUnary' +_UNARY_STREAM = b'UnaryStream' +_STREAM_UNARY = b'StreamUnary' +_STREAM_STREAM = b'StreamStream' + +_CLIENT_METADATA = ( + (b'client-md-key', b'client-md-key'), + (b'client-md-key-bin', b'\x00\x01') +) + +_SERVER_INITIAL_METADATA = ( + (b'server-initial-md-key', b'server-initial-md-value'), + (b'server-initial-md-key-bin', b'\x00\x02') +) + +_SERVER_TRAILING_METADATA = ( + (b'server-trailing-md-key', b'server-trailing-md-value'), + (b'server-trailing-md-key-bin', b'\x00\x03') +) + +_NON_OK_CODE = grpc.StatusCode.NOT_FOUND +_DETAILS = b'Test details!' + + +class _Servicer(object): + + def __init__(self): + self._lock = threading.Lock() + self._code = None + self._details = None + self._exception = False + self._return_none = False + self._received_client_metadata = None + + def unary_unary(self, request, context): + with self._lock: + self._received_client_metadata = context.invocation_metadata() + context.send_initial_metadata(_SERVER_INITIAL_METADATA) + context.set_trailing_metadata(_SERVER_TRAILING_METADATA) + if self._code is not None: + context.set_code(self._code) + if self._details is not None: + context.set_details(self._details) + if self._exception: + raise test_control.Defect() + else: + return None if self._return_none else object() + + def unary_stream(self, request, context): + with self._lock: + self._received_client_metadata = context.invocation_metadata() + context.send_initial_metadata(_SERVER_INITIAL_METADATA) + context.set_trailing_metadata(_SERVER_TRAILING_METADATA) + if self._code is not None: + context.set_code(self._code) + if self._details is not None: + context.set_details(self._details) + for _ in range(test_constants.STREAM_LENGTH // 2): + yield _SERIALIZED_RESPONSE + if self._exception: + raise test_control.Defect() + + def stream_unary(self, request_iterator, context): + with self._lock: + self._received_client_metadata = context.invocation_metadata() + context.send_initial_metadata(_SERVER_INITIAL_METADATA) + context.set_trailing_metadata(_SERVER_TRAILING_METADATA) + if self._code is not None: + context.set_code(self._code) + if self._details is not None: + context.set_details(self._details) + # TODO(https://github.com/grpc/grpc/issues/6891): just ignore the + # request iterator. + for ignored_request in request_iterator: + pass + if self._exception: + raise test_control.Defect() + else: + return None if self._return_none else _SERIALIZED_RESPONSE + + def stream_stream(self, request_iterator, context): + with self._lock: + self._received_client_metadata = context.invocation_metadata() + context.send_initial_metadata(_SERVER_INITIAL_METADATA) + context.set_trailing_metadata(_SERVER_TRAILING_METADATA) + if self._code is not None: + context.set_code(self._code) + if self._details is not None: + context.set_details(self._details) + # TODO(https://github.com/grpc/grpc/issues/6891): just ignore the + # request iterator. + for ignored_request in request_iterator: + pass + for _ in range(test_constants.STREAM_LENGTH // 3): + yield object() + if self._exception: + raise test_control.Defect() + + def set_code(self, code): + with self._lock: + self._code = code + + def set_details(self, details): + with self._lock: + self._details = details + + def set_exception(self): + with self._lock: + self._exception = True + + def set_return_none(self): + with self._lock: + self._return_none = True + + def received_client_metadata(self): + with self._lock: + return self._received_client_metadata + + +def _generic_handler(servicer): + method_handlers = { + _UNARY_UNARY: grpc.unary_unary_rpc_method_handler( + servicer.unary_unary, request_deserializer=_REQUEST_DESERIALIZER, + response_serializer=_RESPONSE_SERIALIZER), + _UNARY_STREAM: grpc.unary_stream_rpc_method_handler( + servicer.unary_stream), + _STREAM_UNARY: grpc.stream_unary_rpc_method_handler( + servicer.stream_unary), + _STREAM_STREAM: grpc.stream_stream_rpc_method_handler( + servicer.stream_stream, request_deserializer=_REQUEST_DESERIALIZER, + response_serializer=_RESPONSE_SERIALIZER), + } + return grpc.method_handlers_generic_handler(_SERVICE, method_handlers) + + +class MetadataCodeDetailsTest(unittest.TestCase): + + def setUp(self): + self._servicer = _Servicer() + self._server_pool = logging_pool.pool(test_constants.THREAD_CONCURRENCY) + self._server = grpc.server( + (_generic_handler(self._servicer),), self._server_pool) + port = self._server.add_insecure_port('[::]:0') + self._server.start() + + channel = grpc.insecure_channel('localhost:{}'.format(port)) + self._unary_unary = channel.unary_unary( + b'/'.join((b'', _SERVICE, _UNARY_UNARY,)), + request_serializer=_REQUEST_SERIALIZER, + response_deserializer=_RESPONSE_DESERIALIZER,) + self._unary_stream = channel.unary_stream( + b'/'.join((b'', _SERVICE, _UNARY_STREAM,)),) + self._stream_unary = channel.stream_unary( + b'/'.join((b'', _SERVICE, _STREAM_UNARY,)),) + self._stream_stream = channel.stream_stream( + b'/'.join((b'', _SERVICE, _STREAM_STREAM,)), + request_serializer=_REQUEST_SERIALIZER, + response_deserializer=_RESPONSE_DESERIALIZER,) + + + def testSuccessfulUnaryUnary(self): + self._servicer.set_details(_DETAILS) + + unused_response, call = self._unary_unary.with_call( + object(), metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, call.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, call.trailing_metadata())) + self.assertIs(grpc.StatusCode.OK, call.code()) + self.assertEqual(_DETAILS, call.details()) + + def testSuccessfulUnaryStream(self): + self._servicer.set_details(_DETAILS) + + call = self._unary_stream(_SERIALIZED_REQUEST, metadata=_CLIENT_METADATA) + received_initial_metadata = call.initial_metadata() + for _ in call: + pass + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, received_initial_metadata)) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, call.trailing_metadata())) + self.assertIs(grpc.StatusCode.OK, call.code()) + self.assertEqual(_DETAILS, call.details()) + + def testSuccessfulStreamUnary(self): + self._servicer.set_details(_DETAILS) + + unused_response, call = self._stream_unary.with_call( + iter([_SERIALIZED_REQUEST] * test_constants.STREAM_LENGTH), + metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, call.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, call.trailing_metadata())) + self.assertIs(grpc.StatusCode.OK, call.code()) + self.assertEqual(_DETAILS, call.details()) + + def testSuccessfulStreamStream(self): + self._servicer.set_details(_DETAILS) + + call = self._stream_stream( + iter([object()] * test_constants.STREAM_LENGTH), + metadata=_CLIENT_METADATA) + received_initial_metadata = call.initial_metadata() + for _ in call: + pass + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, received_initial_metadata)) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, call.trailing_metadata())) + self.assertIs(grpc.StatusCode.OK, call.code()) + self.assertEqual(_DETAILS, call.details()) + + def testCustomCodeUnaryUnary(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + + with self.assertRaises(grpc.RpcError) as exception_context: + self._unary_unary.with_call(object(), metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, + exception_context.exception.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, + exception_context.exception.trailing_metadata())) + self.assertIs(_NON_OK_CODE, exception_context.exception.code()) + self.assertEqual(_DETAILS, exception_context.exception.details()) + + def testCustomCodeUnaryStream(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + + call = self._unary_stream(_SERIALIZED_REQUEST, metadata=_CLIENT_METADATA) + received_initial_metadata = call.initial_metadata() + with self.assertRaises(grpc.RpcError): + for _ in call: + pass + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, received_initial_metadata)) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, call.trailing_metadata())) + self.assertIs(_NON_OK_CODE, call.code()) + self.assertEqual(_DETAILS, call.details()) + + def testCustomCodeStreamUnary(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + + with self.assertRaises(grpc.RpcError) as exception_context: + self._stream_unary.with_call( + iter([_SERIALIZED_REQUEST] * test_constants.STREAM_LENGTH), + metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, + exception_context.exception.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, + exception_context.exception.trailing_metadata())) + self.assertIs(_NON_OK_CODE, exception_context.exception.code()) + self.assertEqual(_DETAILS, exception_context.exception.details()) + + def testCustomCodeStreamStream(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + + call = self._stream_stream( + iter([object()] * test_constants.STREAM_LENGTH), + metadata=_CLIENT_METADATA) + received_initial_metadata = call.initial_metadata() + with self.assertRaises(grpc.RpcError) as exception_context: + for _ in call: + pass + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, received_initial_metadata)) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, + exception_context.exception.trailing_metadata())) + self.assertIs(_NON_OK_CODE, exception_context.exception.code()) + self.assertEqual(_DETAILS, exception_context.exception.details()) + + def testCustomCodeExceptionUnaryUnary(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + self._servicer.set_exception() + + with self.assertRaises(grpc.RpcError) as exception_context: + self._unary_unary.with_call(object(), metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, + exception_context.exception.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, + exception_context.exception.trailing_metadata())) + self.assertIs(_NON_OK_CODE, exception_context.exception.code()) + self.assertEqual(_DETAILS, exception_context.exception.details()) + + def testCustomCodeExceptionUnaryStream(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + self._servicer.set_exception() + + call = self._unary_stream(_SERIALIZED_REQUEST, metadata=_CLIENT_METADATA) + received_initial_metadata = call.initial_metadata() + with self.assertRaises(grpc.RpcError): + for _ in call: + pass + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, received_initial_metadata)) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, call.trailing_metadata())) + self.assertIs(_NON_OK_CODE, call.code()) + self.assertEqual(_DETAILS, call.details()) + + def testCustomCodeExceptionStreamUnary(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + self._servicer.set_exception() + + with self.assertRaises(grpc.RpcError) as exception_context: + self._stream_unary.with_call( + iter([_SERIALIZED_REQUEST] * test_constants.STREAM_LENGTH), + metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, + exception_context.exception.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, + exception_context.exception.trailing_metadata())) + self.assertIs(_NON_OK_CODE, exception_context.exception.code()) + self.assertEqual(_DETAILS, exception_context.exception.details()) + + def testCustomCodeExceptionStreamStream(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + self._servicer.set_exception() + + call = self._stream_stream( + iter([object()] * test_constants.STREAM_LENGTH), + metadata=_CLIENT_METADATA) + received_initial_metadata = call.initial_metadata() + with self.assertRaises(grpc.RpcError): + for _ in call: + pass + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, received_initial_metadata)) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, call.trailing_metadata())) + self.assertIs(_NON_OK_CODE, call.code()) + self.assertEqual(_DETAILS, call.details()) + + def testCustomCodeReturnNoneUnaryUnary(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + self._servicer.set_return_none() + + with self.assertRaises(grpc.RpcError) as exception_context: + self._unary_unary.with_call(object(), metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, + exception_context.exception.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, + exception_context.exception.trailing_metadata())) + self.assertIs(_NON_OK_CODE, exception_context.exception.code()) + self.assertEqual(_DETAILS, exception_context.exception.details()) + + def testCustomCodeReturnNoneStreamUnary(self): + self._servicer.set_code(_NON_OK_CODE) + self._servicer.set_details(_DETAILS) + self._servicer.set_return_none() + + with self.assertRaises(grpc.RpcError) as exception_context: + self._stream_unary.with_call( + iter([_SERIALIZED_REQUEST] * test_constants.STREAM_LENGTH), + metadata=_CLIENT_METADATA) + + self.assertTrue( + test_common.metadata_transmitted( + _CLIENT_METADATA, self._servicer.received_client_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_INITIAL_METADATA, + exception_context.exception.initial_metadata())) + self.assertTrue( + test_common.metadata_transmitted( + _SERVER_TRAILING_METADATA, + exception_context.exception.trailing_metadata())) + self.assertIs(_NON_OK_CODE, exception_context.exception.code()) + self.assertEqual(_DETAILS, exception_context.exception.details()) + + +if __name__ == '__main__': + unittest.main(verbosity=2) From edfebc909a7e4a2ada5e07d2012a227fb15d4064 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Tue, 21 Jun 2016 16:39:21 -0700 Subject: [PATCH 166/280] dont generate NewClient method for C# services --- src/compiler/csharp_generator.cc | 15 --------------- 1 file changed, 15 deletions(-) diff --git a/src/compiler/csharp_generator.cc b/src/compiler/csharp_generator.cc index fc8feaf0fcc..8c1c6f2c826 100644 --- a/src/compiler/csharp_generator.cc +++ b/src/compiler/csharp_generator.cc @@ -485,20 +485,6 @@ void GenerateBindServiceMethod(Printer* out, const ServiceDescriptor *service) { out->Print("\n"); } -void GenerateNewStubMethods(Printer* out, const ServiceDescriptor *service) { - out->Print("/// Creates a new client for $servicename$\n", - "servicename", GetServiceClassName(service)); - out->Print("public static $classname$ NewClient(Channel channel)\n", - "classname", GetClientClassName(service)); - out->Print("{\n"); - out->Indent(); - out->Print("return new $classname$(channel);\n", "classname", - GetClientClassName(service)); - out->Outdent(); - out->Print("}\n"); - out->Print("\n"); -} - void GenerateService(Printer* out, const ServiceDescriptor *service, bool generate_client, bool generate_server, bool internal_access) { @@ -524,7 +510,6 @@ void GenerateService(Printer* out, const ServiceDescriptor *service, } if (generate_client) { GenerateClientStub(out, service); - GenerateNewStubMethods(out, service); } if (generate_server) { GenerateBindServiceMethod(out, service); From 614944324246b2b4eef9fc2e989af20375c49cd5 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Wed, 22 Jun 2016 13:34:59 -0700 Subject: [PATCH 167/280] add code to unregister endpoints --- src/core/lib/iomgr/network_status_tracker.c | 46 ++++++++++++++++++++- src/core/lib/iomgr/network_status_tracker.h | 1 + src/core/lib/iomgr/tcp_posix.c | 1 + src/core/lib/iomgr/tcp_windows.c | 1 + 4 files changed, 47 insertions(+), 2 deletions(-) diff --git a/src/core/lib/iomgr/network_status_tracker.c b/src/core/lib/iomgr/network_status_tracker.c index 6e432368cf1..f4f8e97a252 100644 --- a/src/core/lib/iomgr/network_status_tracker.c +++ b/src/core/lib/iomgr/network_status_tracker.c @@ -33,6 +33,7 @@ #include "src/core/lib/iomgr/endpoint.h" #include +#include typedef struct endpoint_ll_node { grpc_endpoint *ep; @@ -40,9 +41,13 @@ typedef struct endpoint_ll_node { } endpoint_ll_node; static endpoint_ll_node *head = NULL; +static gpr_mu g_endpoint_mutex; +static bool g_init_done = false; -// TODO(makarandd): Install callback with OS to monitor network status. void grpc_initialize_network_status_monitor() { + g_init_done = true; + gpr_mu_init(&g_endpoint_mutex); + // TODO(makarandd): Install callback with OS to monitor network status. } void grpc_destroy_network_status_monitor() { @@ -51,9 +56,15 @@ void grpc_destroy_network_status_monitor() { gpr_free(curr); curr = next; } + gpr_mu_destroy(&g_endpoint_mutex); } void grpc_network_status_register_endpoint(grpc_endpoint *ep) { + if (!g_init_done) { + grpc_initialize_network_status_monitor(); + } + gpr_mu_lock(&g_endpoint_mutex); + gpr_log(GPR_DEBUG, "Register endpoint %p", ep); if (head == NULL) { head = (endpoint_ll_node *)gpr_malloc(sizeof(endpoint_ll_node)); head->ep = ep; @@ -64,19 +75,50 @@ void grpc_network_status_register_endpoint(grpc_endpoint *ep) { head->ep = ep; head->next = prev_head; } + gpr_mu_unlock(&g_endpoint_mutex); +} + +void grpc_network_status_unregister_endpoint(grpc_endpoint *ep) { + gpr_mu_lock(&g_endpoint_mutex); + GPR_ASSERT(head); + gpr_log(GPR_DEBUG, "Unregister endpoint %p", ep); + bool found = false; + endpoint_ll_node *prev = head; + // if we're unregistering the head, just move head to the next + if (ep == head->ep) { + head = head->next; + gpr_free(prev); + found = true; + } else { + for (endpoint_ll_node *curr = head->next; curr != NULL; curr = curr->next) { + if (ep == curr->ep) { + prev->next = curr->next; + gpr_free(curr); + found = true; + break; + } + prev = curr; + } + } + gpr_mu_unlock(&g_endpoint_mutex); + GPR_ASSERT(found); } // Walk the linked-list from head and execute shutdown. It is possible that // other threads might be in the process of shutdown as well, but that has -// no side effect. +// no side effect since endpoint shutdown is idempotent. void grpc_network_status_shutdown_all_endpoints() { + gpr_mu_lock(&g_endpoint_mutex); if (head == NULL) { + gpr_mu_unlock(&g_endpoint_mutex); return; } grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; for (endpoint_ll_node *curr = head; curr != NULL; curr = curr->next) { + gpr_log(GPR_DEBUG, "Shutting down endpoint %p", curr->ep); curr->ep->vtable->shutdown(&exec_ctx, curr->ep); } + gpr_mu_unlock(&g_endpoint_mutex); grpc_exec_ctx_finish(&exec_ctx); } diff --git a/src/core/lib/iomgr/network_status_tracker.h b/src/core/lib/iomgr/network_status_tracker.h index 4154151927f..74a1aa8135f 100644 --- a/src/core/lib/iomgr/network_status_tracker.h +++ b/src/core/lib/iomgr/network_status_tracker.h @@ -36,5 +36,6 @@ #include "src/core/lib/iomgr/endpoint.h" void grpc_network_status_register_endpoint(grpc_endpoint *ep); +void grpc_network_status_unregister_endpoint(grpc_endpoint *ep); void grpc_network_status_shutdown_all_endpoints(); #endif /* GRPC_CORE_LIB_IOMGR_NETWORK_STATUS_TRACKER_H */ diff --git a/src/core/lib/iomgr/tcp_posix.c b/src/core/lib/iomgr/tcp_posix.c index f7818353b0a..56c1eef024c 100644 --- a/src/core/lib/iomgr/tcp_posix.c +++ b/src/core/lib/iomgr/tcp_posix.c @@ -153,6 +153,7 @@ static void tcp_ref(grpc_tcp *tcp) { gpr_ref(&tcp->refcount); } #endif static void tcp_destroy(grpc_exec_ctx *exec_ctx, grpc_endpoint *ep) { + grpc_network_status_unregister_endpoint(ep); grpc_tcp *tcp = (grpc_tcp *)ep; TCP_UNREF(exec_ctx, tcp, "destroy"); } diff --git a/src/core/lib/iomgr/tcp_windows.c b/src/core/lib/iomgr/tcp_windows.c index 8140d1d8cd2..37ab59021e3 100644 --- a/src/core/lib/iomgr/tcp_windows.c +++ b/src/core/lib/iomgr/tcp_windows.c @@ -379,6 +379,7 @@ static void win_shutdown(grpc_exec_ctx *exec_ctx, grpc_endpoint *ep) { } static void win_destroy(grpc_exec_ctx *exec_ctx, grpc_endpoint *ep) { + grpc_network_status_unregister_endpoint(ep); grpc_tcp *tcp = (grpc_tcp *)ep; TCP_UNREF(tcp, "destroy"); } From 76a0795b73ad2632c435fc338bb49368d1d68d9f Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 22 Jun 2016 15:09:06 -0700 Subject: [PATCH 168/280] Fix build errors on some configurations --- src/core/lib/iomgr/ev_epoll_linux.c | 13 +++++++------ test/core/iomgr/ev_epoll_linux_test.c | 10 ++++++++-- 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 6464d3ba348..88cbc586349 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -34,6 +34,7 @@ #include #include +/* This polling engine is only relevant on linux kernels supporting epoll() */ #ifdef GPR_LINUX_EPOLL #include "src/core/lib/iomgr/ev_epoll_linux.h" @@ -322,7 +323,7 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, #ifdef GRPC_TSAN /* See the definition of g_epoll_sync for more context */ - gpr_atm_rel_store(&g_epoll_sync, 0); + gpr_atm_rel_store(&g_epoll_sync, (gpr_atm) 0); #endif /* defined(GRPC_TSAN) */ for (i = 0; i < fd_count; i++) { @@ -442,8 +443,8 @@ static polling_island *polling_island_create(grpc_fd *initial_fd) { pi->fds = NULL; } - gpr_atm_rel_store(&pi->ref_count, 0); - gpr_atm_rel_store(&pi->merged_to, NULL); + gpr_atm_rel_store(&pi->ref_count, (gpr_atm) 0); + gpr_atm_rel_store(&pi->merged_to, (gpr_atm) NULL); pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); @@ -472,7 +473,7 @@ static polling_island *polling_island_create(grpc_fd *initial_fd) { static void polling_island_delete(polling_island *pi) { GPR_ASSERT(pi->fd_cnt == 0); - gpr_atm_rel_store(&pi->merged_to, NULL); + gpr_atm_rel_store(&pi->merged_to, (gpr_atm) NULL); close(pi->epoll_fd); pi->epoll_fd = -1; @@ -648,7 +649,7 @@ static polling_island *polling_island_merge(polling_island *p, polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); /* Add the 'merged_to' link from p --> q */ - gpr_atm_rel_store(&p->merged_to, q); + gpr_atm_rel_store(&p->merged_to, (gpr_atm) q); PI_ADD_REF(q, "pi_merge"); /* To account for the new incoming ref from p */ gpr_mu_unlock(&p->mu); @@ -810,7 +811,7 @@ static grpc_fd *fd_create(int fd, const char *name) { holding a lock to it anyway. */ gpr_mu_lock(&new_fd->mu); - gpr_atm_rel_store(&new_fd->refst, 1); + gpr_atm_rel_store(&new_fd->refst, (gpr_atm) 1); new_fd->fd = fd; new_fd->shutdown = false; new_fd->orphaned = false; diff --git a/test/core/iomgr/ev_epoll_linux_test.c b/test/core/iomgr/ev_epoll_linux_test.c index 35eb6791302..66a69f52cdb 100644 --- a/test/core/iomgr/ev_epoll_linux_test.c +++ b/test/core/iomgr/ev_epoll_linux_test.c @@ -30,7 +30,10 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * */ +#include +/* This test only relevant on linux systems where epoll() is available */ +#ifdef GPR_LINUX_EPOLL #include "src/core/lib/iomgr/ev_epoll_linux.h" #include "src/core/lib/iomgr/ev_posix.h" @@ -128,8 +131,10 @@ static void test_add_fd_to_pollset() { int i; int r; - /* Create some dummy file descriptors (using pipe fds for this test. Could be - anything). Also NUM_FDS should be even for this test. */ + /* Create some dummy file descriptors. Currently using pipe file descriptors + * for this test but we could use any other type of file descriptors. Also, + * since pipe() used in this test creates two fds in each call, NUM_FDS should + * be an even number */ for (i = 0; i < NUM_FDS; i = i + 2) { r = pipe(fds + i); if (r != 0) { @@ -234,3 +239,4 @@ int main(int argc, char **argv) { grpc_iomgr_shutdown(); return 0; } +#endif /* defined(GPR_LINUX_EPOLL) */ From fcb304f493f72fa5e487b364f78126e01c550288 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 22 Jun 2016 15:25:41 -0700 Subject: [PATCH 169/280] Compilation failure --- test/core/iomgr/ev_epoll_linux_test.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/test/core/iomgr/ev_epoll_linux_test.c b/test/core/iomgr/ev_epoll_linux_test.c index 66a69f52cdb..2547dc98718 100644 --- a/test/core/iomgr/ev_epoll_linux_test.c +++ b/test/core/iomgr/ev_epoll_linux_test.c @@ -239,4 +239,6 @@ int main(int argc, char **argv) { grpc_iomgr_shutdown(); return 0; } -#endif /* defined(GPR_LINUX_EPOLL) */ +#else /* defined(GPR_LINUX_EPOLL) */ +int main(int argc, char **argv) { return 0; } +#endif /* !defined(GPR_LINUX_EPOLL) */ From 932d02725200028415f8420bdaf35d9484f419d4 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Wed, 22 Jun 2016 16:17:15 -0700 Subject: [PATCH 170/280] clang-format fixes --- src/core/lib/iomgr/network_status_tracker.c | 10 +++++----- test/core/end2end/tests/network_status_change.c | 1 - 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/src/core/lib/iomgr/network_status_tracker.c b/src/core/lib/iomgr/network_status_tracker.c index f4f8e97a252..0e34605c81d 100644 --- a/src/core/lib/iomgr/network_status_tracker.c +++ b/src/core/lib/iomgr/network_status_tracker.c @@ -31,9 +31,9 @@ * */ -#include "src/core/lib/iomgr/endpoint.h" #include #include +#include "src/core/lib/iomgr/endpoint.h" typedef struct endpoint_ll_node { grpc_endpoint *ep; @@ -51,7 +51,7 @@ void grpc_initialize_network_status_monitor() { } void grpc_destroy_network_status_monitor() { - for (endpoint_ll_node *curr = head; curr != NULL; ) { + for (endpoint_ll_node *curr = head; curr != NULL;) { endpoint_ll_node *next = curr->next; gpr_free(curr); curr = next; @@ -86,9 +86,9 @@ void grpc_network_status_unregister_endpoint(grpc_endpoint *ep) { endpoint_ll_node *prev = head; // if we're unregistering the head, just move head to the next if (ep == head->ep) { - head = head->next; - gpr_free(prev); - found = true; + head = head->next; + gpr_free(prev); + found = true; } else { for (endpoint_ll_node *curr = head->next; curr != NULL; curr = curr->next) { if (ep == curr->ep) { diff --git a/test/core/end2end/tests/network_status_change.c b/test/core/end2end/tests/network_status_change.c index bbc85007022..7e7628391de 100644 --- a/test/core/end2end/tests/network_status_change.c +++ b/test/core/end2end/tests/network_status_change.c @@ -217,7 +217,6 @@ static void test_invoke_network_status_change(grpc_end2end_test_config config) { GPR_ASSERT(0 == strcmp(call_details.host, "foo.test.google.fr")); GPR_ASSERT(was_cancelled == 0); - gpr_free(details); grpc_metadata_array_destroy(&initial_metadata_recv); grpc_metadata_array_destroy(&trailing_metadata_recv); From 8891ef50eef2e858b2290de36691c4b535de3acc Mon Sep 17 00:00:00 2001 From: Ken Payson Date: Wed, 22 Jun 2016 16:47:52 -0700 Subject: [PATCH 171/280] Initialize port to -1 --- src/core/lib/iomgr/tcp_server_windows.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/core/lib/iomgr/tcp_server_windows.c b/src/core/lib/iomgr/tcp_server_windows.c index 2a51671ec71..86982bc1836 100644 --- a/src/core/lib/iomgr/tcp_server_windows.c +++ b/src/core/lib/iomgr/tcp_server_windows.c @@ -396,7 +396,7 @@ static grpc_error *add_socket_to_server(grpc_tcp_server *s, SOCKET sock, size_t addr_len, unsigned port_index, grpc_tcp_listener **listener) { grpc_tcp_listener *sp = NULL; - int port; + int port = -1; int status; GUID guid = WSAID_ACCEPTEX; DWORD ioctl_num_bytes; From 9c578c7a734937f67b928f4511dd40448c621174 Mon Sep 17 00:00:00 2001 From: Bill Clarke Date: Wed, 22 Jun 2016 16:48:08 -0700 Subject: [PATCH 172/280] Const correctness for ClientContext and ServerContext getters --- include/grpc++/impl/codegen/client_context.h | 10 +++++----- include/grpc++/impl/codegen/server_context.h | 7 ++++--- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/include/grpc++/impl/codegen/client_context.h b/include/grpc++/impl/codegen/client_context.h index e23fd4eda34..a132c9a57aa 100644 --- a/include/grpc++/impl/codegen/client_context.h +++ b/include/grpc++/impl/codegen/client_context.h @@ -193,7 +193,7 @@ class ClientContext { /// /// \return A multimap of initial metadata key-value pairs from the server. const std::multimap& - GetServerInitialMetadata() { + GetServerInitialMetadata() const { GPR_CODEGEN_ASSERT(initial_metadata_received_); return recv_initial_metadata_; } @@ -205,7 +205,7 @@ class ClientContext { /// /// \return A multimap of metadata trailing key-value pairs from the server. const std::multimap& - GetServerTrailingMetadata() { + GetServerTrailingMetadata() const { // TODO(yangg) check finished return trailing_metadata_; } @@ -230,13 +230,13 @@ class ClientContext { #ifndef GRPC_CXX0X_NO_CHRONO /// Return the deadline for the client call. - std::chrono::system_clock::time_point deadline() { + std::chrono::system_clock::time_point deadline() const { return Timespec2Timepoint(deadline_); } #endif // !GRPC_CXX0X_NO_CHRONO /// Return a \a gpr_timespec representation of the client call's deadline. - gpr_timespec raw_deadline() { return deadline_; } + gpr_timespec raw_deadline() const { return deadline_; } /// Set the per call authority header (see /// https://tools.ietf.org/html/rfc7540#section-8.1.2.3). @@ -337,7 +337,7 @@ class ClientContext { const InputMessage& request, OutputMessage* result); - grpc_call* call() { return call_; } + grpc_call* call() const { return call_; } void set_call(grpc_call* call, const std::shared_ptr& channel); uint32_t initial_metadata_flags() const { diff --git a/include/grpc++/impl/codegen/server_context.h b/include/grpc++/impl/codegen/server_context.h index a1e1ed176f6..cea13a513f6 100644 --- a/include/grpc++/impl/codegen/server_context.h +++ b/include/grpc++/impl/codegen/server_context.h @@ -94,12 +94,12 @@ class ServerContext { ~ServerContext(); #ifndef GRPC_CXX0X_NO_CHRONO - std::chrono::system_clock::time_point deadline() { + std::chrono::system_clock::time_point deadline() const { return Timespec2Timepoint(deadline_); } #endif // !GRPC_CXX0X_NO_CHRONO - gpr_timespec raw_deadline() { return deadline_; } + gpr_timespec raw_deadline() const { return deadline_; } void AddInitialMetadata(const grpc::string& key, const grpc::string& value); void AddTrailingMetadata(const grpc::string& key, const grpc::string& value); @@ -122,7 +122,8 @@ class ServerContext { // was called. void TryCancel() const; - const std::multimap& client_metadata() { + const std::multimap& client_metadata() + const { return client_metadata_; } From c072c927a885e49cb13afaad11a93379a1255b9f Mon Sep 17 00:00:00 2001 From: Bill Clarke Date: Wed, 22 Jun 2016 16:48:08 -0700 Subject: [PATCH 173/280] Const correctness for ClientContext and ServerContext getters --- include/grpc++/impl/codegen/client_context.h | 10 +++++----- include/grpc++/impl/codegen/server_context.h | 7 ++++--- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/include/grpc++/impl/codegen/client_context.h b/include/grpc++/impl/codegen/client_context.h index e23fd4eda34..a132c9a57aa 100644 --- a/include/grpc++/impl/codegen/client_context.h +++ b/include/grpc++/impl/codegen/client_context.h @@ -193,7 +193,7 @@ class ClientContext { /// /// \return A multimap of initial metadata key-value pairs from the server. const std::multimap& - GetServerInitialMetadata() { + GetServerInitialMetadata() const { GPR_CODEGEN_ASSERT(initial_metadata_received_); return recv_initial_metadata_; } @@ -205,7 +205,7 @@ class ClientContext { /// /// \return A multimap of metadata trailing key-value pairs from the server. const std::multimap& - GetServerTrailingMetadata() { + GetServerTrailingMetadata() const { // TODO(yangg) check finished return trailing_metadata_; } @@ -230,13 +230,13 @@ class ClientContext { #ifndef GRPC_CXX0X_NO_CHRONO /// Return the deadline for the client call. - std::chrono::system_clock::time_point deadline() { + std::chrono::system_clock::time_point deadline() const { return Timespec2Timepoint(deadline_); } #endif // !GRPC_CXX0X_NO_CHRONO /// Return a \a gpr_timespec representation of the client call's deadline. - gpr_timespec raw_deadline() { return deadline_; } + gpr_timespec raw_deadline() const { return deadline_; } /// Set the per call authority header (see /// https://tools.ietf.org/html/rfc7540#section-8.1.2.3). @@ -337,7 +337,7 @@ class ClientContext { const InputMessage& request, OutputMessage* result); - grpc_call* call() { return call_; } + grpc_call* call() const { return call_; } void set_call(grpc_call* call, const std::shared_ptr& channel); uint32_t initial_metadata_flags() const { diff --git a/include/grpc++/impl/codegen/server_context.h b/include/grpc++/impl/codegen/server_context.h index a1e1ed176f6..cea13a513f6 100644 --- a/include/grpc++/impl/codegen/server_context.h +++ b/include/grpc++/impl/codegen/server_context.h @@ -94,12 +94,12 @@ class ServerContext { ~ServerContext(); #ifndef GRPC_CXX0X_NO_CHRONO - std::chrono::system_clock::time_point deadline() { + std::chrono::system_clock::time_point deadline() const { return Timespec2Timepoint(deadline_); } #endif // !GRPC_CXX0X_NO_CHRONO - gpr_timespec raw_deadline() { return deadline_; } + gpr_timespec raw_deadline() const { return deadline_; } void AddInitialMetadata(const grpc::string& key, const grpc::string& value); void AddTrailingMetadata(const grpc::string& key, const grpc::string& value); @@ -122,7 +122,8 @@ class ServerContext { // was called. void TryCancel() const; - const std::multimap& client_metadata() { + const std::multimap& client_metadata() + const { return client_metadata_; } From 17233ceaa9a0150eeb0959cd7953ac8be100b413 Mon Sep 17 00:00:00 2001 From: Ken Payson Date: Wed, 22 Jun 2016 17:16:15 -0700 Subject: [PATCH 174/280] Ignore unused variables in ruby compilation --- Rakefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Rakefile b/Rakefile index f208a24fd33..f44946fe937 100755 --- a/Rakefile +++ b/Rakefile @@ -77,7 +77,7 @@ task 'dlls' do grpc_config = ENV['GRPC_CONFIG'] || 'opt' verbose = ENV['V'] || '0' - env = 'CPPFLAGS="-D_WIN32_WINNT=0x600 -DUNICODE -D_UNICODE" ' + env = 'CPPFLAGS="-D_WIN32_WINNT=0x600 -DUNICODE -D_UNICODE -Wno-unused-variable -Wno-unused-result" ' env += 'LDFLAGS=-static ' env += 'SYSTEM=MINGW32 ' env += 'EMBED_ZLIB=true ' From f41ebc3968ef06439d6d1468256ac5000940ea8f Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 22 Jun 2016 12:47:14 -0700 Subject: [PATCH 175/280] remove occurences of NewClient factory method --- examples/csharp/helloworld/GreeterClient/Program.cs | 2 +- examples/csharp/route_guide/RouteGuideClient/Program.cs | 2 +- src/csharp/Grpc.Examples.Tests/MathClientServerTests.cs | 2 +- src/csharp/Grpc.HealthCheck.Tests/HealthClientServerTest.cs | 2 +- src/csharp/Grpc.IntegrationTesting/ClientRunners.cs | 6 +++--- .../Grpc.IntegrationTesting/GeneratedServiceBaseTest.cs | 2 +- .../Grpc.IntegrationTesting/InteropClientServerTest.cs | 4 ++-- .../Grpc.IntegrationTesting/MetadataCredentialsTest.cs | 4 ++-- src/csharp/Grpc.IntegrationTesting/SslCredentialsTest.cs | 2 +- src/csharp/Grpc.IntegrationTesting/StressTestClient.cs | 2 +- 10 files changed, 14 insertions(+), 14 deletions(-) diff --git a/examples/csharp/helloworld/GreeterClient/Program.cs b/examples/csharp/helloworld/GreeterClient/Program.cs index 4393f2f3c29..444d4735095 100644 --- a/examples/csharp/helloworld/GreeterClient/Program.cs +++ b/examples/csharp/helloworld/GreeterClient/Program.cs @@ -39,7 +39,7 @@ namespace GreeterClient { Channel channel = new Channel("127.0.0.1:50051", ChannelCredentials.Insecure); - var client = Greeter.NewClient(channel); + var client = new Greeter.GreeterClient(channel); String user = "you"; var reply = client.SayHello(new HelloRequest { Name = user }); diff --git a/examples/csharp/route_guide/RouteGuideClient/Program.cs b/examples/csharp/route_guide/RouteGuideClient/Program.cs index ee51fbe8e04..8a173dcc3b3 100644 --- a/examples/csharp/route_guide/RouteGuideClient/Program.cs +++ b/examples/csharp/route_guide/RouteGuideClient/Program.cs @@ -231,7 +231,7 @@ namespace Routeguide static void Main(string[] args) { var channel = new Channel("127.0.0.1:50052", ChannelCredentials.Insecure); - var client = new RouteGuideClient(RouteGuide.NewClient(channel)); + var client = new RouteGuideClient(new RouteGuide.RouteGuideClient(channel)); // Looking for a valid feature client.GetFeature(409146138, -746188906); diff --git a/src/csharp/Grpc.Examples.Tests/MathClientServerTests.cs b/src/csharp/Grpc.Examples.Tests/MathClientServerTests.cs index 324c209ca1d..50dacc2eaaf 100644 --- a/src/csharp/Grpc.Examples.Tests/MathClientServerTests.cs +++ b/src/csharp/Grpc.Examples.Tests/MathClientServerTests.cs @@ -62,7 +62,7 @@ namespace Math.Tests }; server.Start(); channel = new Channel(Host, server.Ports.Single().BoundPort, ChannelCredentials.Insecure); - client = Math.NewClient(channel); + client = new Math.MathClient(channel); } [TestFixtureTearDown] diff --git a/src/csharp/Grpc.HealthCheck.Tests/HealthClientServerTest.cs b/src/csharp/Grpc.HealthCheck.Tests/HealthClientServerTest.cs index 070674bae9d..25a58fb3864 100644 --- a/src/csharp/Grpc.HealthCheck.Tests/HealthClientServerTest.cs +++ b/src/csharp/Grpc.HealthCheck.Tests/HealthClientServerTest.cs @@ -65,7 +65,7 @@ namespace Grpc.HealthCheck.Tests server.Start(); channel = new Channel(Host, server.Ports.Single().BoundPort, ChannelCredentials.Insecure); - client = Grpc.Health.V1.Health.NewClient(channel); + client = new Grpc.Health.V1.Health.HealthClient(channel); } [TestFixtureTearDown] diff --git a/src/csharp/Grpc.IntegrationTesting/ClientRunners.cs b/src/csharp/Grpc.IntegrationTesting/ClientRunners.cs index 39b9ae08e64..b9c0fe6d0d8 100644 --- a/src/csharp/Grpc.IntegrationTesting/ClientRunners.cs +++ b/src/csharp/Grpc.IntegrationTesting/ClientRunners.cs @@ -211,7 +211,7 @@ namespace Grpc.IntegrationTesting bool profilerReset = false; - var client = BenchmarkService.NewClient(channel); + var client = new BenchmarkService.BenchmarkServiceClient(channel); var request = CreateSimpleRequest(); var stopwatch = new Stopwatch(); @@ -237,7 +237,7 @@ namespace Grpc.IntegrationTesting private async Task RunUnaryAsync(Channel channel, IInterarrivalTimer timer) { - var client = BenchmarkService.NewClient(channel); + var client = new BenchmarkService.BenchmarkServiceClient(channel); var request = CreateSimpleRequest(); var stopwatch = new Stopwatch(); @@ -256,7 +256,7 @@ namespace Grpc.IntegrationTesting private async Task RunStreamingPingPongAsync(Channel channel, IInterarrivalTimer timer) { - var client = BenchmarkService.NewClient(channel); + var client = new BenchmarkService.BenchmarkServiceClient(channel); var request = CreateSimpleRequest(); var stopwatch = new Stopwatch(); diff --git a/src/csharp/Grpc.IntegrationTesting/GeneratedServiceBaseTest.cs b/src/csharp/Grpc.IntegrationTesting/GeneratedServiceBaseTest.cs index 001533ce315..4216dc1d6be 100644 --- a/src/csharp/Grpc.IntegrationTesting/GeneratedServiceBaseTest.cs +++ b/src/csharp/Grpc.IntegrationTesting/GeneratedServiceBaseTest.cs @@ -61,7 +61,7 @@ namespace Grpc.IntegrationTesting }; server.Start(); channel = new Channel(Host, server.Ports.Single().BoundPort, ChannelCredentials.Insecure); - client = TestService.NewClient(channel); + client = new TestService.TestServiceClient(channel); } [TearDown] diff --git a/src/csharp/Grpc.IntegrationTesting/InteropClientServerTest.cs b/src/csharp/Grpc.IntegrationTesting/InteropClientServerTest.cs index 4ee1ff5ec8a..f907f630dab 100644 --- a/src/csharp/Grpc.IntegrationTesting/InteropClientServerTest.cs +++ b/src/csharp/Grpc.IntegrationTesting/InteropClientServerTest.cs @@ -69,7 +69,7 @@ namespace Grpc.IntegrationTesting }; int port = server.Ports.Single().BoundPort; channel = new Channel(Host, port, TestCredentials.CreateSslCredentials(), options); - client = TestService.NewClient(channel); + client = new TestService.TestServiceClient(channel); } [TestFixtureTearDown] @@ -148,7 +148,7 @@ namespace Grpc.IntegrationTesting [Test] public void UnimplementedMethod() { - InteropClient.RunUnimplementedMethod(UnimplementedService.NewClient(channel)); + InteropClient.RunUnimplementedMethod(new UnimplementedService.UnimplementedServiceClient(channel)); } } } diff --git a/src/csharp/Grpc.IntegrationTesting/MetadataCredentialsTest.cs b/src/csharp/Grpc.IntegrationTesting/MetadataCredentialsTest.cs index 9fd575f1900..eb3bb8a28bb 100644 --- a/src/csharp/Grpc.IntegrationTesting/MetadataCredentialsTest.cs +++ b/src/csharp/Grpc.IntegrationTesting/MetadataCredentialsTest.cs @@ -88,7 +88,7 @@ namespace Grpc.IntegrationTesting var channelCredentials = ChannelCredentials.Create(TestCredentials.CreateSslCredentials(), CallCredentials.FromInterceptor(asyncAuthInterceptor)); channel = new Channel(Host, server.Ports.Single().BoundPort, channelCredentials, options); - client = TestService.NewClient(channel); + client = new TestService.TestServiceClient(channel); client.UnaryCall(new SimpleRequest { }); } @@ -97,7 +97,7 @@ namespace Grpc.IntegrationTesting public void MetadataCredentials_PerCall() { channel = new Channel(Host, server.Ports.Single().BoundPort, TestCredentials.CreateSslCredentials(), options); - client = TestService.NewClient(channel); + client = new TestService.TestServiceClient(channel); var callCredentials = CallCredentials.FromInterceptor(asyncAuthInterceptor); client.UnaryCall(new SimpleRequest { }, new CallOptions(credentials: callCredentials)); diff --git a/src/csharp/Grpc.IntegrationTesting/SslCredentialsTest.cs b/src/csharp/Grpc.IntegrationTesting/SslCredentialsTest.cs index 3df45b5f708..f85e272711f 100644 --- a/src/csharp/Grpc.IntegrationTesting/SslCredentialsTest.cs +++ b/src/csharp/Grpc.IntegrationTesting/SslCredentialsTest.cs @@ -79,7 +79,7 @@ namespace Grpc.IntegrationTesting }; channel = new Channel(Host, server.Ports.Single().BoundPort, clientCredentials, options); - client = TestService.NewClient(channel); + client = new TestService.TestServiceClient(channel); } [TestFixtureTearDown] diff --git a/src/csharp/Grpc.IntegrationTesting/StressTestClient.cs b/src/csharp/Grpc.IntegrationTesting/StressTestClient.cs index 4d6ca7ece57..74ee040ae49 100644 --- a/src/csharp/Grpc.IntegrationTesting/StressTestClient.cs +++ b/src/csharp/Grpc.IntegrationTesting/StressTestClient.cs @@ -148,7 +148,7 @@ namespace Grpc.IntegrationTesting channels.Add(channel); for (int j = 0; j < options.NumStubsPerChannel; j++) { - var client = TestService.NewClient(channel); + var client = new TestService.TestServiceClient(channel); var task = Task.Factory.StartNew(() => RunBodyAsync(client).GetAwaiter().GetResult(), TaskCreationOptions.LongRunning); tasks.Add(task); From bfcd6b627ece5ddab1aa690e4bf205a6c66d8fe3 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 22 Jun 2016 16:52:46 -0700 Subject: [PATCH 176/280] generate comments for C# client constructors --- src/compiler/csharp_generator.cc | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/src/compiler/csharp_generator.cc b/src/compiler/csharp_generator.cc index 8c1c6f2c826..f5a0876cf98 100644 --- a/src/compiler/csharp_generator.cc +++ b/src/compiler/csharp_generator.cc @@ -333,22 +333,28 @@ void GenerateClientStub(Printer* out, const ServiceDescriptor *service) { out->Indent(); // constructors + out->Print("/// Creates a new client for $servicename$\n" + "/// The channel to use to make remote calls.\n", + "servicename", GetServiceClassName(service)); out->Print("public $name$(Channel channel) : base(channel)\n", "name", GetClientClassName(service)); out->Print("{\n"); out->Print("}\n"); + out->Print("/// Creates a new client for $servicename$ that uses a custom CallInvoker.\n" + "/// The callInvoker to use to make remote calls.\n", + "servicename", GetServiceClassName(service)); out->Print("public $name$(CallInvoker callInvoker) : base(callInvoker)\n", "name", GetClientClassName(service)); out->Print("{\n"); out->Print("}\n"); - out->Print("///Protected parameterless constructor to allow creation" + out->Print("/// Protected parameterless constructor to allow creation" " of test doubles.\n"); out->Print("protected $name$() : base()\n", "name", GetClientClassName(service)); out->Print("{\n"); out->Print("}\n"); - out->Print("///Protected constructor to allow creation of configured" - " clients.\n"); + out->Print("/// Protected constructor to allow creation of configured clients.\n" + "/// The client configuration.\n"); out->Print("protected $name$(ClientBaseConfiguration configuration)" " : base(configuration)\n", "name", GetClientClassName(service)); From ed6ea4c691193483223fdb3a7ada5095b8a2494e Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 22 Jun 2016 12:38:44 -0700 Subject: [PATCH 177/280] regenerate protos --- src/csharp/Grpc.Examples/MathGrpc.cs | 15 +++---- src/csharp/Grpc.HealthCheck/HealthGrpc.cs | 15 +++---- .../Grpc.IntegrationTesting/MetricsGrpc.cs | 15 +++---- .../Grpc.IntegrationTesting/ServicesGrpc.cs | 30 ++++++------- .../Grpc.IntegrationTesting/TestGrpc.cs | 45 +++++++++---------- 5 files changed, 56 insertions(+), 64 deletions(-) diff --git a/src/csharp/Grpc.Examples/MathGrpc.cs b/src/csharp/Grpc.Examples/MathGrpc.cs index 4bbefcbe01a..25abc514195 100644 --- a/src/csharp/Grpc.Examples/MathGrpc.cs +++ b/src/csharp/Grpc.Examples/MathGrpc.cs @@ -128,17 +128,22 @@ namespace Math { /// Client for Math public class MathClient : ClientBase { + /// Creates a new client for Math + /// The channel to use to make remote calls. public MathClient(Channel channel) : base(channel) { } + /// Creates a new client for Math that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public MathClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected MathClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected MathClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -235,12 +240,6 @@ namespace Math { } } - /// Creates a new client for Math - public static MathClient NewClient(Channel channel) - { - return new MathClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(MathBase serviceImpl) { diff --git a/src/csharp/Grpc.HealthCheck/HealthGrpc.cs b/src/csharp/Grpc.HealthCheck/HealthGrpc.cs index d0ade7d02be..43eea0f22f5 100644 --- a/src/csharp/Grpc.HealthCheck/HealthGrpc.cs +++ b/src/csharp/Grpc.HealthCheck/HealthGrpc.cs @@ -71,17 +71,22 @@ namespace Grpc.Health.V1 { /// Client for Health public class HealthClient : ClientBase { + /// Creates a new client for Health + /// The channel to use to make remote calls. public HealthClient(Channel channel) : base(channel) { } + /// Creates a new client for Health that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public HealthClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected HealthClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected HealthClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -108,12 +113,6 @@ namespace Grpc.Health.V1 { } } - /// Creates a new client for Health - public static HealthClient NewClient(Channel channel) - { - return new HealthClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(HealthBase serviceImpl) { diff --git a/src/csharp/Grpc.IntegrationTesting/MetricsGrpc.cs b/src/csharp/Grpc.IntegrationTesting/MetricsGrpc.cs index 22bd27ec0aa..040798e3c21 100644 --- a/src/csharp/Grpc.IntegrationTesting/MetricsGrpc.cs +++ b/src/csharp/Grpc.IntegrationTesting/MetricsGrpc.cs @@ -97,17 +97,22 @@ namespace Grpc.Testing { /// Client for MetricsService public class MetricsServiceClient : ClientBase { + /// Creates a new client for MetricsService + /// The channel to use to make remote calls. public MetricsServiceClient(Channel channel) : base(channel) { } + /// Creates a new client for MetricsService that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public MetricsServiceClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected MetricsServiceClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected MetricsServiceClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -162,12 +167,6 @@ namespace Grpc.Testing { } } - /// Creates a new client for MetricsService - public static MetricsServiceClient NewClient(Channel channel) - { - return new MetricsServiceClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(MetricsServiceBase serviceImpl) { diff --git a/src/csharp/Grpc.IntegrationTesting/ServicesGrpc.cs b/src/csharp/Grpc.IntegrationTesting/ServicesGrpc.cs index 9c99296115c..e205dea93ea 100644 --- a/src/csharp/Grpc.IntegrationTesting/ServicesGrpc.cs +++ b/src/csharp/Grpc.IntegrationTesting/ServicesGrpc.cs @@ -93,17 +93,22 @@ namespace Grpc.Testing { /// Client for BenchmarkService public class BenchmarkServiceClient : ClientBase { + /// Creates a new client for BenchmarkService + /// The channel to use to make remote calls. public BenchmarkServiceClient(Channel channel) : base(channel) { } + /// Creates a new client for BenchmarkService that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public BenchmarkServiceClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected BenchmarkServiceClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected BenchmarkServiceClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -162,12 +167,6 @@ namespace Grpc.Testing { } } - /// Creates a new client for BenchmarkService - public static BenchmarkServiceClient NewClient(Channel channel) - { - return new BenchmarkServiceClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(BenchmarkServiceBase serviceImpl) { @@ -273,17 +272,22 @@ namespace Grpc.Testing { /// Client for WorkerService public class WorkerServiceClient : ClientBase { + /// Creates a new client for WorkerService + /// The channel to use to make remote calls. public WorkerServiceClient(Channel channel) : base(channel) { } + /// Creates a new client for WorkerService that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public WorkerServiceClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected WorkerServiceClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected WorkerServiceClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -398,12 +402,6 @@ namespace Grpc.Testing { } } - /// Creates a new client for WorkerService - public static WorkerServiceClient NewClient(Channel channel) - { - return new WorkerServiceClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(WorkerServiceBase serviceImpl) { diff --git a/src/csharp/Grpc.IntegrationTesting/TestGrpc.cs b/src/csharp/Grpc.IntegrationTesting/TestGrpc.cs index 6c252013f84..3e149da3e01 100644 --- a/src/csharp/Grpc.IntegrationTesting/TestGrpc.cs +++ b/src/csharp/Grpc.IntegrationTesting/TestGrpc.cs @@ -168,17 +168,22 @@ namespace Grpc.Testing { /// Client for TestService public class TestServiceClient : ClientBase { + /// Creates a new client for TestService + /// The channel to use to make remote calls. public TestServiceClient(Channel channel) : base(channel) { } + /// Creates a new client for TestService that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public TestServiceClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected TestServiceClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected TestServiceClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -315,12 +320,6 @@ namespace Grpc.Testing { } } - /// Creates a new client for TestService - public static TestServiceClient NewClient(Channel channel) - { - return new TestServiceClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(TestServiceBase serviceImpl) { @@ -373,17 +372,22 @@ namespace Grpc.Testing { /// Client for UnimplementedService public class UnimplementedServiceClient : ClientBase { + /// Creates a new client for UnimplementedService + /// The channel to use to make remote calls. public UnimplementedServiceClient(Channel channel) : base(channel) { } + /// Creates a new client for UnimplementedService that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public UnimplementedServiceClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected UnimplementedServiceClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected UnimplementedServiceClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -422,12 +426,6 @@ namespace Grpc.Testing { } } - /// Creates a new client for UnimplementedService - public static UnimplementedServiceClient NewClient(Channel channel) - { - return new UnimplementedServiceClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(UnimplementedServiceBase serviceImpl) { @@ -485,17 +483,22 @@ namespace Grpc.Testing { /// Client for ReconnectService public class ReconnectServiceClient : ClientBase { + /// Creates a new client for ReconnectService + /// The channel to use to make remote calls. public ReconnectServiceClient(Channel channel) : base(channel) { } + /// Creates a new client for ReconnectService that uses a custom CallInvoker. + /// The callInvoker to use to make remote calls. public ReconnectServiceClient(CallInvoker callInvoker) : base(callInvoker) { } - ///Protected parameterless constructor to allow creation of test doubles. + /// Protected parameterless constructor to allow creation of test doubles. protected ReconnectServiceClient() : base() { } - ///Protected constructor to allow creation of configured clients. + /// Protected constructor to allow creation of configured clients. + /// The client configuration. protected ReconnectServiceClient(ClientBaseConfiguration configuration) : base(configuration) { } @@ -538,12 +541,6 @@ namespace Grpc.Testing { } } - /// Creates a new client for ReconnectService - public static ReconnectServiceClient NewClient(Channel channel) - { - return new ReconnectServiceClient(channel); - } - /// Creates service definition that can be registered with a server public static ServerServiceDefinition BindService(ReconnectServiceBase serviceImpl) { From 0224dcc2dcda932a171776de325fa2e66c95478f Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Wed, 22 Jun 2016 18:04:00 -0700 Subject: [PATCH 178/280] clang format --- src/core/lib/iomgr/ev_epoll_linux.c | 12 ++++++------ src/core/lib/iomgr/ev_posix.h | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index 88cbc586349..b1e9ac8a631 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -323,7 +323,7 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, #ifdef GRPC_TSAN /* See the definition of g_epoll_sync for more context */ - gpr_atm_rel_store(&g_epoll_sync, (gpr_atm) 0); + gpr_atm_rel_store(&g_epoll_sync, (gpr_atm)0); #endif /* defined(GRPC_TSAN) */ for (i = 0; i < fd_count; i++) { @@ -443,8 +443,8 @@ static polling_island *polling_island_create(grpc_fd *initial_fd) { pi->fds = NULL; } - gpr_atm_rel_store(&pi->ref_count, (gpr_atm) 0); - gpr_atm_rel_store(&pi->merged_to, (gpr_atm) NULL); + gpr_atm_rel_store(&pi->ref_count, (gpr_atm)0); + gpr_atm_rel_store(&pi->merged_to, (gpr_atm)NULL); pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); @@ -473,7 +473,7 @@ static polling_island *polling_island_create(grpc_fd *initial_fd) { static void polling_island_delete(polling_island *pi) { GPR_ASSERT(pi->fd_cnt == 0); - gpr_atm_rel_store(&pi->merged_to, (gpr_atm) NULL); + gpr_atm_rel_store(&pi->merged_to, (gpr_atm)NULL); close(pi->epoll_fd); pi->epoll_fd = -1; @@ -649,7 +649,7 @@ static polling_island *polling_island_merge(polling_island *p, polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); /* Add the 'merged_to' link from p --> q */ - gpr_atm_rel_store(&p->merged_to, (gpr_atm) q); + gpr_atm_rel_store(&p->merged_to, (gpr_atm)q); PI_ADD_REF(q, "pi_merge"); /* To account for the new incoming ref from p */ gpr_mu_unlock(&p->mu); @@ -811,7 +811,7 @@ static grpc_fd *fd_create(int fd, const char *name) { holding a lock to it anyway. */ gpr_mu_lock(&new_fd->mu); - gpr_atm_rel_store(&new_fd->refst, (gpr_atm) 1); + gpr_atm_rel_store(&new_fd->refst, (gpr_atm)1); new_fd->fd = fd; new_fd->shutdown = false; new_fd->orphaned = false; diff --git a/src/core/lib/iomgr/ev_posix.h b/src/core/lib/iomgr/ev_posix.h index 32260fe2ee5..579c84ef707 100644 --- a/src/core/lib/iomgr/ev_posix.h +++ b/src/core/lib/iomgr/ev_posix.h @@ -100,7 +100,7 @@ void grpc_event_engine_init(void); void grpc_event_engine_shutdown(void); /* Return the name of the poll strategy */ -const char* grpc_get_poll_strategy_name(); +const char *grpc_get_poll_strategy_name(); /* Create a wrapped file descriptor. Requires fd is a non-blocking file descriptor. From cc00745e8c6e160665e753c9fda6bf2650b1823d Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Thu, 23 Jun 2016 00:18:15 -0700 Subject: [PATCH 179/280] brought spec up to date --- doc/compression.md | 31 +++++++++++++++++++------------ 1 file changed, 19 insertions(+), 12 deletions(-) diff --git a/doc/compression.md b/doc/compression.md index 15fae4d29bf..9229ef9af10 100644 --- a/doc/compression.md +++ b/doc/compression.md @@ -42,13 +42,13 @@ and RPC settings (for example, if compression would result in small or negative gains). When a message from a client compressed with an unsupported algorithm is -processed by a server, it WILL result in an INVALID\_ARGUMENT error on the +processed by a server, it WILL result in a `UNIMPLEMENTED` error status on the server. The server will then include in its response a `grpc-accept-encoding` -header specifying the algorithms it does accept. If an INTERNAL error is -returned from the server despite having used one of the algorithms from the -`grpc-accept-encoding` header, the cause MUST NOT be related to compression. -Data sent from a server compressed with an algorithm not supported by the client -WILL result in an INTERNAL error on the client side. +header specifying the algorithms it does accept. If an `UNIMPLEMENTED` error +status is returned from the server despite having used one of the algorithms +from the `grpc-accept-encoding` header, the cause MUST NOT be related to +compression. Data sent from a server compressed with an algorithm not supported +by the client WILL result in an `INTERNAL` error status on the client side. Note that a peer MAY choose to not disclose all the encodings it supports. However, if it receives a message compressed in an undisclosed but supported @@ -99,13 +99,20 @@ compressed. channel compression configuration MUST be used. 1. When a compression method (including no compression) is specified for an outgoing message, the message MUST be compressed accordingly. -1. A message compressed in a way not supported by its endpoint MUST fail with -INVALID\_ARGUMENT status, its associated description indicating the unsupported -condition as well as the supported ones. The returned `grpc-accept-encoding` -header MUST NOT contain the compression method (encoding) used. +1. A message compressed by a client in a way not supported by its server MUST +fail with status `UNIMPLEMENTED`, its associated description indicating the +unsupported condition as well as the supported ones. The returned +`grpc-accept-encoding` header MUST NOT contain the compression method +(encoding) used. +1. A message compressed by a server in a way not supported by its client MUST +fail with status `INTERNAL`, its associated description indicating the +unsupported condition as well as the supported ones. The returned +`grpc-accept-encoding` header MUST NOT contain the compression method +(encoding) used. 1. An ill-constructed message with its [Compressed-Flag bit](PROTOCOL-HTTP2.md#compressed-flag) set but lacking a "[grpc-encoding](PROTOCOL-HTTP2.md#message-encoding)" -entry different from _identity_ in its metadata MUST fail with INTERNAL status, -its associated description indicating the invalid Compressed-Flag condition. +entry different from _identity_ in its metadata MUST fail with `INTERNAL` +status, its associated description indicating the invalid Compressed-Flag +condition. From c9b7ad0c0a541ac7559bb7081c260de195fe8315 Mon Sep 17 00:00:00 2001 From: Ken Payson Date: Thu, 23 Jun 2016 02:38:20 -0700 Subject: [PATCH 180/280] Updated grpcio c extension build for windows to use mingw. The default compiler for Python c extensions is VS2008 or VS2010 depending on the Python version. Since c-core moved onto the c99 standard, these compilers are not compatible. --- tools/run_tests/build_artifact_python.bat | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/tools/run_tests/build_artifact_python.bat b/tools/run_tests/build_artifact_python.bat index fea0275426f..295347e947c 100644 --- a/tools/run_tests/build_artifact_python.bat +++ b/tools/run_tests/build_artifact_python.bat @@ -37,10 +37,11 @@ set NUGET=C:\nuget\nuget.exe mkdir src\python\grpcio\grpc\_cython\_windows +@rem TODO(atash): maybe we could avoid the grpc_c.(32|64).python shim below if +@rem this used the right python build? copy /Y vsprojects\Release\grpc_dll.dll src\python\grpcio\grpc\_cython\_windows\grpc_c.32.python || goto :error copy /Y vsprojects\x64\Release\grpc_dll.dll src\python\grpcio\grpc\_cython\_windows\grpc_c.64.python || goto :error - set PATH=C:\%1;C:\%1\scripts;C:\msys64\mingw%2\bin;%PATH% pip install --upgrade six @@ -50,12 +51,6 @@ pip install -rrequirements.txt set GRPC_PYTHON_USE_CUSTOM_BDIST=0 set GRPC_PYTHON_BUILD_WITH_CYTHON=1 -@rem TODO(atash): maybe we could avoid the grpc_c.(32|64).python shim above if -@rem this used the right python build? -python setup.py bdist_wheel - -@rem Build gRPC Python tools -@rem @rem Because this is windows and *everything seems to hate Windows* we have to @rem set all of these flags ourselves because Python won't help us (see the @rem setup.py of the grpcio_tools project). @@ -70,6 +65,18 @@ set GRPC_PYTHON_CFLAGS=-fno-wrapv -frtti -std=c++11 python -c "from distutils.cygwinccompiler import get_msvcr; print(get_msvcr()[0])" > temp.txt set /p PYTHON_MSVCR= Date: Thu, 23 Jun 2016 08:22:30 -0700 Subject: [PATCH 181/280] Re-enable accidentally disabled traces --- src/core/lib/iomgr/tcp_posix.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/core/lib/iomgr/tcp_posix.c b/src/core/lib/iomgr/tcp_posix.c index 017f52c3677..1046b60bcc9 100644 --- a/src/core/lib/iomgr/tcp_posix.c +++ b/src/core/lib/iomgr/tcp_posix.c @@ -160,7 +160,7 @@ static void call_read_cb(grpc_exec_ctx *exec_ctx, grpc_tcp *tcp, grpc_error *error) { grpc_closure *cb = tcp->read_cb; - if (false && grpc_tcp_trace) { + if (grpc_tcp_trace) { size_t i; const char *str = grpc_error_string(error); gpr_log(GPR_DEBUG, "read: error=%s", str); @@ -394,7 +394,7 @@ static void tcp_write(grpc_exec_ctx *exec_ctx, grpc_endpoint *ep, grpc_tcp *tcp = (grpc_tcp *)ep; grpc_error *error = GRPC_ERROR_NONE; - if (false && grpc_tcp_trace) { + if (grpc_tcp_trace) { size_t i; for (i = 0; i < buf->count; i++) { From 75f6828636a432ff32435ac06a6121b8396c67fd Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Thu, 23 Jun 2016 10:16:16 -0700 Subject: [PATCH 182/280] Fixed typo --- doc/compression.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/compression.md b/doc/compression.md index 9229ef9af10..de245d90fee 100644 --- a/doc/compression.md +++ b/doc/compression.md @@ -42,7 +42,7 @@ and RPC settings (for example, if compression would result in small or negative gains). When a message from a client compressed with an unsupported algorithm is -processed by a server, it WILL result in a `UNIMPLEMENTED` error status on the +processed by a server, it WILL result in an `UNIMPLEMENTED` error status on the server. The server will then include in its response a `grpc-accept-encoding` header specifying the algorithms it does accept. If an `UNIMPLEMENTED` error status is returned from the server despite having used one of the algorithms From dde4fc66fb55291434a017c667d6a961332801e6 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 23 Jun 2016 10:35:19 -0700 Subject: [PATCH 183/280] changed service to _service, and added a TODO TODO for removing boilerplate code --- .../objective-c/route_guide/ViewControllers.m | 27 ++++++++++--------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/examples/objective-c/route_guide/ViewControllers.m b/examples/objective-c/route_guide/ViewControllers.m index 5fd411e3ba6..a5c03e9cfca 100644 --- a/examples/objective-c/route_guide/ViewControllers.m +++ b/examples/objective-c/route_guide/ViewControllers.m @@ -81,7 +81,7 @@ static NSString * const kHostAddress = @"localhost:50051"; * not to have a feature. */ @interface GetFeatureViewController : UIViewController { - RTGRouteGuide *service; + RTGRouteGuide *_service; } @property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @@ -90,6 +90,7 @@ static NSString * const kHostAddress = @"localhost:50051"; - (void)execRequest { void (^handler)(RTGFeature *response, NSError *error) = ^(RTGFeature *response, NSError *error) { + // TODO(makdharma): Remove boilerplate by consolidating into one log function. if (response.name.length) { NSString *str =[NSString stringWithFormat:@"%@\nFound feature called %@ at %@.", self.outputLabel.text, response.location, response.name]; self.outputLabel.text = str; @@ -109,8 +110,8 @@ static NSString * const kHostAddress = @"localhost:50051"; point.latitude = 409146138; point.longitude = -746188906; - [service getFeatureWithRequest:point handler:handler]; - [service getFeatureWithRequest:[RTGPoint message] handler:handler]; + [_service getFeatureWithRequest:point handler:handler]; + [_service getFeatureWithRequest:[RTGPoint message] handler:handler]; } - (void)viewDidLoad { @@ -119,7 +120,7 @@ static NSString * const kHostAddress = @"localhost:50051"; // This only needs to be done once per host, before creating service objects for that host. [GRPCCall useInsecureConnectionsForHost:kHostAddress]; - service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; + _service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; } - (void)viewDidAppear:(BOOL)animated { @@ -139,7 +140,7 @@ static NSString * const kHostAddress = @"localhost:50051"; * the pre-generated database. Prints each response as it comes in. */ @interface ListFeaturesViewController : UIViewController { - RTGRouteGuide *service; + RTGRouteGuide *_service; } @property (weak, nonatomic) IBOutlet UILabel *outputLabel; @@ -155,7 +156,7 @@ static NSString * const kHostAddress = @"localhost:50051"; rectangle.hi.longitude = -745E6; NSLog(@"Looking for features between %@ and %@", rectangle.lo, rectangle.hi); - [service listFeaturesWithRequest:rectangle + [_service listFeaturesWithRequest:rectangle eventHandler:^(BOOL done, RTGFeature *response, NSError *error) { if (response) { NSString *str =[NSString stringWithFormat:@"%@\nFound feature at %@ called %@.", self.outputLabel.text, response.location, response.name]; @@ -172,7 +173,7 @@ static NSString * const kHostAddress = @"localhost:50051"; - (void)viewDidLoad { [super viewDidLoad]; - service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; + _service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; } - (void)viewDidAppear:(BOOL)animated { @@ -193,7 +194,7 @@ static NSString * const kHostAddress = @"localhost:50051"; * server. */ @interface RecordRouteViewController : UIViewController { - RTGRouteGuide *service; + RTGRouteGuide *_service; } @property (weak, nonatomic) IBOutlet UILabel *outputLabel; @@ -217,7 +218,7 @@ static NSString * const kHostAddress = @"localhost:50051"; return location; }]; - [service recordRouteWithRequestsWriter:locations + [_service recordRouteWithRequestsWriter:locations handler:^(RTGRouteSummary *response, NSError *error) { if (response) { NSString *str =[NSString stringWithFormat: @@ -241,7 +242,7 @@ static NSString * const kHostAddress = @"localhost:50051"; - (void)viewDidLoad { [super viewDidLoad]; - service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; + _service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; } - (void)viewDidAppear:(BOOL)animated { @@ -261,7 +262,7 @@ static NSString * const kHostAddress = @"localhost:50051"; * the server. */ @interface RouteChatViewController : UIViewController { - RTGRouteGuide *service; + RTGRouteGuide *_service; } @property (weak, nonatomic) IBOutlet UILabel *outputLabel; @@ -279,7 +280,7 @@ static NSString * const kHostAddress = @"localhost:50051"; return note; }]; - [service routeChatWithRequestsWriter:notesWriter + [_service routeChatWithRequestsWriter:notesWriter eventHandler:^(BOOL done, RTGRouteNote *note, NSError *error) { if (note) { NSString *str =[NSString stringWithFormat:@"%@\nGot message %@ at %@", @@ -300,7 +301,7 @@ static NSString * const kHostAddress = @"localhost:50051"; - (void)viewDidLoad { [super viewDidLoad]; - service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; + _service = [[RTGRouteGuide alloc] initWithHost:kHostAddress]; } - (void)viewDidAppear:(BOOL)animated { From d139da8b9a9c482a83b23c926690f01b49025640 Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Thu, 23 Jun 2016 10:37:21 -0700 Subject: [PATCH 184/280] Remove unnecessary changes --- tools/run_tests/run_tests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 2a404df0d41..532f7187408 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -425,7 +425,7 @@ class PythonLanguage(object): return [] def build_steps(self): - return [['tools/run_tests/build_python.sh', tox_env] + return [['tools/run_tests/build_python.sh', tox_env] for tox_env in self._tox_envs] def post_tests_steps(self): From 83a6a828b047cc24922a6d143e30f4ab6eb63239 Mon Sep 17 00:00:00 2001 From: David Garcia Quintas Date: Thu, 23 Jun 2016 11:33:13 -0700 Subject: [PATCH 185/280] Fixed error messages for C++ interop client. --- test/cpp/interop/interop_client.cc | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/test/cpp/interop/interop_client.cc b/test/cpp/interop/interop_client.cc index 89f841dbe96..ec0729d9c5d 100644 --- a/test/cpp/interop/interop_client.cc +++ b/test/cpp/interop/interop_client.cc @@ -451,7 +451,7 @@ bool InteropClient::DoResponseStreaming() { // most likely due to connection failure. gpr_log(GPR_ERROR, "DoResponseStreaming(): Read fewer streams (%d) than " - "response_stream_sizes.size() (%" PRIuPTR ")", + "response_stream_sizes.size() (%zu)", i, response_stream_sizes.size()); return TransientFailureOrAbort(); } @@ -576,11 +576,10 @@ bool InteropClient::DoServerCompressedStreaming() { if (k < sizes.size()) { // stream->Read() failed before reading all the expected messages. This // is most likely due to a connection failure. - gpr_log(GPR_ERROR, "%s(): Responses read (k=%" PRIuPTR - ") is " - "less than the expected messages (i.e " - "response_stream_sizes.size() (%" PRIuPTR ")).", - __func__, k, response_stream_sizes.size()); + gpr_log(GPR_ERROR, + "%s(): Responses read (k=%zu) is less than the expected number of " + "messages (%zu).", + __func__, k, sizes.size()); return TransientFailureOrAbort(); } @@ -665,7 +664,7 @@ bool InteropClient::DoHalfDuplex() { // most likely due to a connection failure gpr_log(GPR_ERROR, "DoHalfDuplex(): Responses read (i=%d) are less than the expected " - "number of messages response_stream_sizes.size() (%" PRIuPTR ")", + "number of messages response_stream_sizes.size() (%zu)", i, response_stream_sizes.size()); return TransientFailureOrAbort(); } From a4b34c290d5f91866a3eff2f9793c00eb12f16f0 Mon Sep 17 00:00:00 2001 From: Ken Payson Date: Wed, 22 Jun 2016 15:31:15 -0700 Subject: [PATCH 186/280] Changed %lld to use mingw supported PRId64 --- src/core/ext/client_config/channel_connectivity.c | 6 +++--- src/core/lib/profiling/basic_timers.c | 4 ++-- .../lib/security/credentials/jwt/jwt_credentials.c | 4 ++-- src/core/lib/surface/channel.c | 10 +++++----- src/core/lib/surface/completion_queue.c | 8 ++++---- src/core/lib/transport/transport_op_string.c | 4 ++-- 6 files changed, 18 insertions(+), 18 deletions(-) diff --git a/src/core/ext/client_config/channel_connectivity.c b/src/core/ext/client_config/channel_connectivity.c index 20c01a9a7cf..2fd8e0e1232 100644 --- a/src/core/ext/client_config/channel_connectivity.c +++ b/src/core/ext/client_config/channel_connectivity.c @@ -189,10 +189,10 @@ void grpc_channel_watch_connectivity_state( GRPC_API_TRACE( "grpc_channel_watch_connectivity_state(" "channel=%p, last_observed_state=%d, " - "deadline=gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " "cq=%p, tag=%p)", - 7, (channel, (int)last_observed_state, (long long)deadline.tv_sec, - (int)deadline.tv_nsec, (int)deadline.clock_type, cq, tag)); + 7, (channel, (int)last_observed_state, deadline.tv_sec, + deadline.tv_nsec, (int)deadline.clock_type, cq, tag)); grpc_cq_begin_op(cq, tag); diff --git a/src/core/lib/profiling/basic_timers.c b/src/core/lib/profiling/basic_timers.c index 50082cd7ee4..ee464f310f4 100644 --- a/src/core/lib/profiling/basic_timers.c +++ b/src/core/lib/profiling/basic_timers.c @@ -141,9 +141,9 @@ static void write_log(gpr_timer_log *log) { entry->tm = gpr_time_0(entry->tm.clock_type); } fprintf(output_file, - "{\"t\": %lld.%09d, \"thd\": \"%d\", \"type\": \"%c\", \"tag\": " + "{\"t\": %"PRId64".%09d, \"thd\": \"%d\", \"type\": \"%c\", \"tag\": " "\"%s\", \"file\": \"%s\", \"line\": %d, \"imp\": %d}\n", - (long long)entry->tm.tv_sec, (int)entry->tm.tv_nsec, entry->thd, + entry->tm.tv_sec, entry->tm.tv_nsec, entry->thd, entry->type, entry->tagstr, entry->file, entry->line, entry->important); } diff --git a/src/core/lib/security/credentials/jwt/jwt_credentials.c b/src/core/lib/security/credentials/jwt/jwt_credentials.c index 973fb75eaab..f4dd9208610 100644 --- a/src/core/lib/security/credentials/jwt/jwt_credentials.c +++ b/src/core/lib/security/credentials/jwt/jwt_credentials.c @@ -149,10 +149,10 @@ grpc_call_credentials *grpc_service_account_jwt_access_credentials_create( "grpc_service_account_jwt_access_credentials_create(" "json_key=%s, " "token_lifetime=" - "gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " + "gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", 5, - (json_key, (long long)token_lifetime.tv_sec, (int)token_lifetime.tv_nsec, + (json_key, token_lifetime.tv_sec, token_lifetime.tv_nsec, (int)token_lifetime.clock_type, reserved)); GPR_ASSERT(reserved == NULL); return grpc_service_account_jwt_access_credentials_create_from_auth_json_key( diff --git a/src/core/lib/surface/channel.c b/src/core/lib/surface/channel.c index 8936e7164f4..952a769de1d 100644 --- a/src/core/lib/surface/channel.c +++ b/src/core/lib/surface/channel.c @@ -223,10 +223,10 @@ grpc_call *grpc_channel_create_call(grpc_channel *channel, "grpc_channel_create_call(" "channel=%p, parent_call=%p, propagation_mask=%x, cq=%p, method=%s, " "host=%s, " - "deadline=gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", 10, (channel, parent_call, (unsigned)propagation_mask, cq, method, host, - (long long)deadline.tv_sec, (int)deadline.tv_nsec, + deadline.tv_sec, deadline.tv_nsec, (int)deadline.clock_type, reserved)); GPR_ASSERT(!reserved); return grpc_channel_create_call_internal( @@ -282,11 +282,11 @@ grpc_call *grpc_channel_create_registered_call( "grpc_channel_create_registered_call(" "channel=%p, parent_call=%p, propagation_mask=%x, completion_queue=%p, " "registered_call_handle=%p, " - "deadline=gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", 9, (channel, parent_call, (unsigned)propagation_mask, completion_queue, - registered_call_handle, (long long)deadline.tv_sec, - (int)deadline.tv_nsec, (int)deadline.clock_type, reserved)); + registered_call_handle, deadline.tv_sec, + deadline.tv_nsec, (int)deadline.clock_type, reserved)); GPR_ASSERT(!reserved); return grpc_channel_create_call_internal( channel, parent_call, propagation_mask, completion_queue, NULL, diff --git a/src/core/lib/surface/completion_queue.c b/src/core/lib/surface/completion_queue.c index 2cc6aa74e02..aa35ae02fe2 100644 --- a/src/core/lib/surface/completion_queue.c +++ b/src/core/lib/surface/completion_queue.c @@ -316,9 +316,9 @@ grpc_event grpc_completion_queue_next(grpc_completion_queue *cc, GRPC_API_TRACE( "grpc_completion_queue_next(" "cc=%p, " - "deadline=gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", - 5, (cc, (long long)deadline.tv_sec, (int)deadline.tv_nsec, + 5, (cc, deadline.tv_sec, deadline.tv_nsec, (int)deadline.clock_type, reserved)); GPR_ASSERT(!reserved); @@ -428,9 +428,9 @@ grpc_event grpc_completion_queue_pluck(grpc_completion_queue *cc, void *tag, GRPC_API_TRACE( "grpc_completion_queue_pluck(" "cc=%p, tag=%p, " - "deadline=gpr_timespec { tv_sec: %lld, tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", - 6, (cc, tag, (long long)deadline.tv_sec, (int)deadline.tv_nsec, + 6, (cc, tag, deadline.tv_sec, deadline.tv_nsec, (int)deadline.clock_type, reserved)); GPR_ASSERT(!reserved); diff --git a/src/core/lib/transport/transport_op_string.c b/src/core/lib/transport/transport_op_string.c index df04c611270..d8fae7805e4 100644 --- a/src/core/lib/transport/transport_op_string.c +++ b/src/core/lib/transport/transport_op_string.c @@ -63,8 +63,8 @@ static void put_metadata_list(gpr_strvec *b, grpc_metadata_batch md) { } if (gpr_time_cmp(md.deadline, gpr_inf_future(md.deadline.clock_type)) != 0) { char *tmp; - gpr_asprintf(&tmp, " deadline=%lld.%09d", (long long)md.deadline.tv_sec, - (int)md.deadline.tv_nsec); + gpr_asprintf(&tmp, " deadline=%"PRId64".%09d", md.deadline.tv_sec, + md.deadline.tv_nsec); gpr_strvec_add(b, tmp); } } From fe754b4e996415300ff29b6e4c378d3d5704852a Mon Sep 17 00:00:00 2001 From: Ken Payson Date: Thu, 23 Jun 2016 12:32:07 -0700 Subject: [PATCH 187/280] Use GRPC_PYTHON_CFLAGS/GRPC_PYTHON_LDFLAGS in setup.py. This is needed for building grpcio with mingw, see https://github.com/grpc/grpc/pull/7012. --- setup.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/setup.py b/setup.py index f73501b8b35..7e625b937a7 100644 --- a/setup.py +++ b/setup.py @@ -31,6 +31,7 @@ import os import os.path +import shlex import shutil import sys import sysconfig @@ -99,8 +100,9 @@ if not "win32" in sys.platform: DEFINE_MACROS = (('OPENSSL_NO_ASM', 1), ('_WIN32_WINNT', 0x600), ('GPR_BACKWARDS_COMPATIBILITY_MODE', 1),) -LDFLAGS = () -CFLAGS = () +LDFLAGS = shlex.split(os.environ.get('GRPC_PYTHON_LDFLAGS', '')) +CFLAGS = shlex.split(os.environ.get('GRPC_PYTHON_CFLAGS', '')) + if "linux" in sys.platform: LDFLAGS += ('-Wl,-wrap,memcpy',) if "linux" in sys.platform or "darwin" in sys.platform: From 51d4f0194963e60cca2e28c08a53070f2135b4f0 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Thu, 23 Jun 2016 13:01:43 -0700 Subject: [PATCH 188/280] prevent race between grpcsharp_server_request_call and grpc_completion_queue_shutdown --- .../Grpc.Core/Internal/ServerSafeHandle.cs | 23 +++++++++++++------ 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/src/csharp/Grpc.Core/Internal/ServerSafeHandle.cs b/src/csharp/Grpc.Core/Internal/ServerSafeHandle.cs index 85813027064..014a8db78f2 100644 --- a/src/csharp/Grpc.Core/Internal/ServerSafeHandle.cs +++ b/src/csharp/Grpc.Core/Internal/ServerSafeHandle.cs @@ -54,7 +54,10 @@ namespace Grpc.Core.Internal public void RegisterCompletionQueue(CompletionQueueSafeHandle cq) { - Native.grpcsharp_server_register_completion_queue(this, cq); + using (cq.NewScope()) + { + Native.grpcsharp_server_register_completion_queue(this, cq); + } } public int AddInsecurePort(string addr) @@ -74,16 +77,22 @@ namespace Grpc.Core.Internal public void ShutdownAndNotify(BatchCompletionDelegate callback, CompletionQueueSafeHandle completionQueue) { - var ctx = BatchContextSafeHandle.Create(); - completionQueue.CompletionRegistry.RegisterBatchCompletion(ctx, callback); - Native.grpcsharp_server_shutdown_and_notify_callback(this, completionQueue, ctx); + using (completionQueue.NewScope()) + { + var ctx = BatchContextSafeHandle.Create(); + completionQueue.CompletionRegistry.RegisterBatchCompletion(ctx, callback); + Native.grpcsharp_server_shutdown_and_notify_callback(this, completionQueue, ctx); + } } public void RequestCall(BatchCompletionDelegate callback, CompletionQueueSafeHandle completionQueue) { - var ctx = BatchContextSafeHandle.Create(); - completionQueue.CompletionRegistry.RegisterBatchCompletion(ctx, callback); - Native.grpcsharp_server_request_call(this, completionQueue, ctx).CheckOk(); + using (completionQueue.NewScope()) + { + var ctx = BatchContextSafeHandle.Create(); + completionQueue.CompletionRegistry.RegisterBatchCompletion(ctx, callback); + Native.grpcsharp_server_request_call(this, completionQueue, ctx).CheckOk(); + } } protected override bool ReleaseHandle() From abd285aed813ff55329e76a9dc19eb3a1bc86166 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 23 Jun 2016 13:30:19 -0700 Subject: [PATCH 189/280] Added missing todo and moved _service To Implementation --- .../objective-c/route_guide/ViewControllers.m | 28 ++++++++++--------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/examples/objective-c/route_guide/ViewControllers.m b/examples/objective-c/route_guide/ViewControllers.m index a5c03e9cfca..26ca9d6220b 100644 --- a/examples/objective-c/route_guide/ViewControllers.m +++ b/examples/objective-c/route_guide/ViewControllers.m @@ -80,13 +80,14 @@ static NSString * const kHostAddress = @"localhost:50051"; * Run the getFeature demo. Calls getFeature with a point known to have a feature and a point known * not to have a feature. */ -@interface GetFeatureViewController : UIViewController { - RTGRouteGuide *_service; -} +@interface GetFeatureViewController : UIViewController + @property (weak, nonatomic) IBOutlet UILabel *outputLabel; + @end @implementation GetFeatureViewController +RTGRouteGuide *_service; - (void)execRequest { void (^handler)(RTGFeature *response, NSError *error) = ^(RTGFeature *response, NSError *error) { @@ -139,14 +140,14 @@ static NSString * const kHostAddress = @"localhost:50051"; * Run the listFeatures demo. Calls listFeatures with a rectangle containing all of the features in * the pre-generated database. Prints each response as it comes in. */ -@interface ListFeaturesViewController : UIViewController { - RTGRouteGuide *_service; -} +@interface ListFeaturesViewController : UIViewController + @property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @implementation ListFeaturesViewController +RTGRouteGuide *_service; - (void)execRequest { RTGRectangle *rectangle = [RTGRectangle message]; @@ -193,14 +194,14 @@ static NSString * const kHostAddress = @"localhost:50051"; * database with a variable delay in between. Prints the statistics when they are sent from the * server. */ -@interface RecordRouteViewController : UIViewController { - RTGRouteGuide *_service; -} +@interface RecordRouteViewController : UIViewController + @property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @implementation RecordRouteViewController +RTGRouteGuide *_service; - (void)execRequest { NSString *dataBasePath = [NSBundle.mainBundle pathForResource:@"route_guide_db" @@ -233,7 +234,7 @@ static NSString * const kHostAddress = @"localhost:50051"; NSLog(@"It took %i seconds", response.elapsedTime); } else { NSString *str =[NSString stringWithFormat:@"%@\nRPC error: %@", self.outputLabel.text, error]; - self.outputLabel.text = str; + self.outputLabel.text = str; NSLog(@"RPC error: %@", error); } }]; @@ -261,14 +262,14 @@ static NSString * const kHostAddress = @"localhost:50051"; * Run the routeChat demo. Send some chat messages, and print any chat messages that are sent from * the server. */ -@interface RouteChatViewController : UIViewController { - RTGRouteGuide *_service; -} +@interface RouteChatViewController : UIViewController + @property (weak, nonatomic) IBOutlet UILabel *outputLabel; @end @implementation RouteChatViewController +RTGRouteGuide *_service; - (void)execRequest { NSArray *notes = @[[RTGRouteNote noteWithMessage:@"First message" latitude:0 longitude:0], @@ -305,6 +306,7 @@ static NSString * const kHostAddress = @"localhost:50051"; } - (void)viewDidAppear:(BOOL)animated { + // TODO(makarandd): Set these properties through UI builder self.outputLabel.text = @"RPC log:"; self.outputLabel.numberOfLines = 0; self.outputLabel.font = [UIFont fontWithName:@"Helvetica Neue" size:8.0]; From f0f70a8a68c107f9fcff2dd35867fd762c2c13d6 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 23 Jun 2016 13:55:06 -0700 Subject: [PATCH 190/280] Make transport-level errors be reflected in status messages on calls Allows us to eliminate logging those errors by default (since they are explicitly passed up to the application). Required plumbing errors through the stack a little more deeply than we had previously. --- .../client_config/subchannel_call_holder.c | 7 +- .../chttp2/transport/chttp2_transport.c | 229 +++++++++++------- .../ext/transport/chttp2/transport/internal.h | 3 - src/core/lib/channel/channel_stack.c | 2 +- src/core/lib/iomgr/error.c | 14 ++ src/core/lib/iomgr/error.h | 16 +- .../security/transport/client_auth_filter.c | 3 +- src/core/lib/surface/call.c | 105 ++++---- src/core/lib/surface/completion_queue.c | 2 +- src/core/lib/transport/transport.c | 65 ++--- src/core/lib/transport/transport.h | 9 +- src/core/lib/transport/transport_op_string.c | 15 +- 12 files changed, 279 insertions(+), 191 deletions(-) diff --git a/src/core/ext/client_config/subchannel_call_holder.c b/src/core/ext/client_config/subchannel_call_holder.c index e31800edd98..b96a0ad0937 100644 --- a/src/core/ext/client_config/subchannel_call_holder.c +++ b/src/core/ext/client_config/subchannel_call_holder.c @@ -120,16 +120,13 @@ retry: return; } /* if this is a cancellation, then we can raise our cancelled flag */ - if (op->cancel_with_status != GRPC_STATUS_OK) { + if (op->cancel_error != GRPC_ERROR_NONE) { if (!gpr_atm_rel_cas(&holder->subchannel_call, 0, 1)) { goto retry; } else { switch (holder->creation_phase) { case GRPC_SUBCHANNEL_CALL_HOLDER_NOT_CREATING: - fail_locked(exec_ctx, holder, - grpc_error_set_int(GRPC_ERROR_CREATE("Cancelled"), - GRPC_ERROR_INT_GRPC_STATUS, - op->cancel_with_status)); + fail_locked(exec_ctx, holder, GRPC_ERROR_REF(op->cancel_error)); break; case GRPC_SUBCHANNEL_CALL_HOLDER_PICKING_SUBCHANNEL: holder->pick_subchannel(exec_ctx, holder->pick_subchannel_arg, NULL, diff --git a/src/core/ext/transport/chttp2/transport/chttp2_transport.c b/src/core/ext/transport/chttp2/transport/chttp2_transport.c index 9c99ff883af..f34ab98b036 100644 --- a/src/core/ext/transport/chttp2/transport/chttp2_transport.c +++ b/src/core/ext/transport/chttp2/transport/chttp2_transport.c @@ -106,14 +106,12 @@ static void perform_stream_op_locked(grpc_exec_ctx *exec_ctx, static void cancel_from_api(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport_global *transport_global, grpc_chttp2_stream_global *stream_global, - grpc_status_code status, - gpr_slice *optional_message); + grpc_error *error); static void close_from_api(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport_global *transport_global, grpc_chttp2_stream_global *stream_global, - grpc_status_code status, - gpr_slice *optional_message); + grpc_error *error); /** Add endpoint from this transport to pollset */ static void add_to_pollset_locked(grpc_exec_ctx *exec_ctx, @@ -163,8 +161,6 @@ static void destruct_transport(grpc_exec_ctx *exec_ctx, GPR_ASSERT(t->ep == NULL); - gpr_slice_unref(t->optional_drop_message); - gpr_slice_buffer_destroy(&t->global.qbuf); gpr_slice_buffer_destroy(&t->writing.outbuf); @@ -266,7 +262,6 @@ static void init_transport(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport *t, is_client ? GRPC_DTS_FH_0 : GRPC_DTS_CLIENT_PREFIX_0; t->parsing.is_first_frame = true; t->writing.is_client = is_client; - t->optional_drop_message = gpr_empty_slice(); grpc_connectivity_state_init( &t->channel_callback.state_tracker, GRPC_CHANNEL_READY, is_client ? "client_transport" : "server_transport"); @@ -876,7 +871,9 @@ static void maybe_start_some_streams( grpc_chttp2_list_pop_waiting_for_concurrency(transport_global, &stream_global)) { cancel_from_api(exec_ctx, transport_global, stream_global, - GRPC_STATUS_UNAVAILABLE, NULL); + grpc_error_set_int( + GRPC_ERROR_CREATE("Stream IDs exhausted"), + GRPC_ERROR_INT_GRPC_STATUS, GRPC_STATUS_UNAVAILABLE)); } } @@ -958,14 +955,14 @@ static void perform_stream_op_locked(grpc_exec_ctx *exec_ctx, on_complete->next_data.scratch |= CLOSURE_BARRIER_STATS_BIT; } - if (op->cancel_with_status != GRPC_STATUS_OK) { + if (op->cancel_error != GRPC_ERROR_NONE) { cancel_from_api(exec_ctx, transport_global, stream_global, - op->cancel_with_status, op->optional_close_message); + GRPC_ERROR_REF(op->cancel_error)); } - if (op->close_with_status != GRPC_STATUS_OK) { + if (op->close_error != GRPC_ERROR_NONE) { close_from_api(exec_ctx, transport_global, stream_global, - op->close_with_status, op->optional_close_message); + GRPC_ERROR_REF(op->close_error)); } if (op->send_initial_metadata != NULL) { @@ -979,12 +976,16 @@ static void perform_stream_op_locked(grpc_exec_ctx *exec_ctx, transport_global->settings[GRPC_PEER_SETTINGS] [GRPC_CHTTP2_SETTINGS_MAX_HEADER_LIST_SIZE]; if (metadata_size > metadata_peer_limit) { - gpr_log(GPR_DEBUG, - "to-be-sent initial metadata size exceeds peer limit " - "(%" PRIuPTR " vs. %" PRIuPTR ")", - metadata_size, metadata_peer_limit); - cancel_from_api(exec_ctx, transport_global, stream_global, - GRPC_STATUS_RESOURCE_EXHAUSTED, NULL); + cancel_from_api( + exec_ctx, transport_global, stream_global, + grpc_error_set_int( + grpc_error_set_int( + grpc_error_set_int( + GRPC_ERROR_CREATE("to-be-sent initial metadata size " + "exceeds peer limit"), + GRPC_ERROR_INT_SIZE, (intptr_t)metadata_size), + GRPC_ERROR_INT_LIMIT, (intptr_t)metadata_peer_limit), + GRPC_ERROR_INT_GRPC_STATUS, GRPC_STATUS_RESOURCE_EXHAUSTED)); } else { if (contains_non_ok_status(transport_global, op->send_initial_metadata)) { stream_global->seen_error = true; @@ -1038,12 +1039,16 @@ static void perform_stream_op_locked(grpc_exec_ctx *exec_ctx, transport_global->settings[GRPC_PEER_SETTINGS] [GRPC_CHTTP2_SETTINGS_MAX_HEADER_LIST_SIZE]; if (metadata_size > metadata_peer_limit) { - gpr_log(GPR_DEBUG, - "to-be-sent trailing metadata size exceeds peer limit " - "(%" PRIuPTR " vs. %" PRIuPTR ")", - metadata_size, metadata_peer_limit); - cancel_from_api(exec_ctx, transport_global, stream_global, - GRPC_STATUS_RESOURCE_EXHAUSTED, NULL); + cancel_from_api( + exec_ctx, transport_global, stream_global, + grpc_error_set_int( + grpc_error_set_int( + grpc_error_set_int( + GRPC_ERROR_CREATE("to-be-sent trailing metadata size " + "exceeds peer limit"), + GRPC_ERROR_INT_SIZE, (intptr_t)metadata_size), + GRPC_ERROR_INT_LIMIT, (intptr_t)metadata_peer_limit), + GRPC_ERROR_INT_GRPC_STATUS, GRPC_STATUS_RESOURCE_EXHAUSTED)); } else { if (contains_non_ok_status(transport_global, op->send_trailing_metadata)) { @@ -1235,8 +1240,12 @@ static void check_read_ops(grpc_exec_ctx *exec_ctx, incoming_byte_stream_destroy_locked(exec_ctx, NULL, NULL, bs); } if (stream_global->exceeded_metadata_size) { - cancel_from_api(exec_ctx, transport_global, stream_global, - GRPC_STATUS_RESOURCE_EXHAUSTED, NULL); + cancel_from_api( + exec_ctx, transport_global, stream_global, + grpc_error_set_int( + GRPC_ERROR_CREATE( + "received initial metadata size exceeds limit"), + GRPC_ERROR_INT_GRPC_STATUS, GRPC_STATUS_RESOURCE_EXHAUSTED)); } } grpc_chttp2_incoming_metadata_buffer_publish( @@ -1275,8 +1284,12 @@ static void check_read_ops(grpc_exec_ctx *exec_ctx, incoming_byte_stream_destroy_locked(exec_ctx, NULL, NULL, bs); } if (stream_global->exceeded_metadata_size) { - cancel_from_api(exec_ctx, transport_global, stream_global, - GRPC_STATUS_RESOURCE_EXHAUSTED, NULL); + cancel_from_api( + exec_ctx, transport_global, stream_global, + grpc_error_set_int( + GRPC_ERROR_CREATE( + "received trailing metadata size exceeds limit"), + GRPC_ERROR_INT_GRPC_STATUS, GRPC_STATUS_RESOURCE_EXHAUSTED)); } } if (stream_global->all_incoming_byte_streams_finished) { @@ -1340,35 +1353,67 @@ static void remove_stream(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport *t, GRPC_ERROR_UNREF(error); } +static void status_codes_from_error(grpc_error *error, + grpc_chttp2_error_code *http2_error, + grpc_status_code *grpc_status) { + intptr_t ip_http; + intptr_t ip_grpc; + bool have_http = + grpc_error_get_int(error, GRPC_ERROR_INT_HTTP2_ERROR, &ip_http); + bool have_grpc = + grpc_error_get_int(error, GRPC_ERROR_INT_GRPC_STATUS, &ip_grpc); + if (have_http) { + *http2_error = (grpc_chttp2_error_code)ip_http; + } else if (have_grpc) { + *http2_error = + grpc_chttp2_grpc_status_to_http2_error((grpc_status_code)ip_grpc); + } else { + *http2_error = GRPC_CHTTP2_INTERNAL_ERROR; + } + if (have_grpc) { + *grpc_status = (grpc_status_code)ip_grpc; + } else if (have_http) { + *grpc_status = + grpc_chttp2_http2_error_to_grpc_status((grpc_chttp2_error_code)ip_http); + } else { + *grpc_status = GRPC_STATUS_INTERNAL; + } +} + static void cancel_from_api(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport_global *transport_global, grpc_chttp2_stream_global *stream_global, - grpc_status_code status, - gpr_slice *optional_message) { + grpc_error *due_to_error) { if (!stream_global->read_closed || !stream_global->write_closed) { + grpc_status_code grpc_status; + grpc_chttp2_error_code http_error; + status_codes_from_error(due_to_error, &http_error, &grpc_status); + if (stream_global->id != 0) { gpr_slice_buffer_add( &transport_global->qbuf, - grpc_chttp2_rst_stream_create( - stream_global->id, - (uint32_t)grpc_chttp2_grpc_status_to_http2_error(status), - &stream_global->stats.outgoing)); + grpc_chttp2_rst_stream_create(stream_global->id, (uint32_t)http_error, + &stream_global->stats.outgoing)); } - if (optional_message) { - gpr_slice_ref(*optional_message); + const char *msg = + grpc_error_get_str(due_to_error, GRPC_ERROR_STR_GRPC_MESSAGE); + bool free_msg = false; + if (msg == NULL) { + free_msg = true; + msg = grpc_error_string(due_to_error); } - grpc_chttp2_fake_status(exec_ctx, transport_global, stream_global, status, - optional_message); + gpr_slice msg_slice = gpr_slice_from_copied_string(msg); + grpc_chttp2_fake_status(exec_ctx, transport_global, stream_global, + grpc_status, &msg_slice); + if (free_msg) grpc_error_free_string(msg); } - if (status != GRPC_STATUS_OK && !stream_global->seen_error) { + if (due_to_error != GRPC_ERROR_NONE && !stream_global->seen_error) { stream_global->seen_error = true; grpc_chttp2_list_add_check_read_ops(transport_global, stream_global); } - grpc_chttp2_mark_stream_closed( - exec_ctx, transport_global, stream_global, 1, 1, - grpc_error_set_int(GRPC_ERROR_CREATE("Cancelled"), - GRPC_ERROR_INT_GRPC_STATUS, status)); + grpc_chttp2_mark_stream_closed(exec_ctx, transport_global, stream_global, 1, + 1, due_to_error); } void grpc_chttp2_fake_status(grpc_exec_ctx *exec_ctx, @@ -1469,15 +1514,17 @@ void grpc_chttp2_mark_stream_closed( static void close_from_api(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport_global *transport_global, grpc_chttp2_stream_global *stream_global, - grpc_status_code status, - gpr_slice *optional_message) { + grpc_error *error) { gpr_slice hdr; gpr_slice status_hdr; gpr_slice message_pfx; uint8_t *p; uint32_t len = 0; + grpc_status_code grpc_status; + grpc_chttp2_error_code http_error; + status_codes_from_error(error, &http_error, &grpc_status); - GPR_ASSERT(status >= 0 && (int)status < 100); + GPR_ASSERT(grpc_status >= 0 && (int)grpc_status < 100); if (stream_global->id != 0 && !transport_global->is_client) { /* Hand roll a header block. @@ -1487,7 +1534,7 @@ static void close_from_api(grpc_exec_ctx *exec_ctx, time we got around to sending this, so instead we ignore HPACK compression and just write the uncompressed bytes onto the wire. */ - status_hdr = gpr_slice_malloc(15 + (status >= 10)); + status_hdr = gpr_slice_malloc(15 + (grpc_status >= 10)); p = GPR_SLICE_START_PTR(status_hdr); *p++ = 0x40; /* literal header */ *p++ = 11; /* len(grpc-status) */ @@ -1502,19 +1549,23 @@ static void close_from_api(grpc_exec_ctx *exec_ctx, *p++ = 't'; *p++ = 'u'; *p++ = 's'; - if (status < 10) { + if (grpc_status < 10) { *p++ = 1; - *p++ = (uint8_t)('0' + status); + *p++ = (uint8_t)('0' + grpc_status); } else { *p++ = 2; - *p++ = (uint8_t)('0' + (status / 10)); - *p++ = (uint8_t)('0' + (status % 10)); + *p++ = (uint8_t)('0' + (grpc_status / 10)); + *p++ = (uint8_t)('0' + (grpc_status % 10)); } GPR_ASSERT(p == GPR_SLICE_END_PTR(status_hdr)); len += (uint32_t)GPR_SLICE_LENGTH(status_hdr); - if (optional_message) { - GPR_ASSERT(GPR_SLICE_LENGTH(*optional_message) < 127); + const char *optional_message = + grpc_error_get_str(error, GRPC_ERROR_STR_GRPC_MESSAGE); + + if (optional_message != NULL) { + size_t msg_len = strlen(optional_message); + GPR_ASSERT(msg_len < 127); message_pfx = gpr_slice_malloc(15); p = GPR_SLICE_START_PTR(message_pfx); *p++ = 0x40; @@ -1531,10 +1582,10 @@ static void close_from_api(grpc_exec_ctx *exec_ctx, *p++ = 'a'; *p++ = 'g'; *p++ = 'e'; - *p++ = (uint8_t)GPR_SLICE_LENGTH(*optional_message); + *p++ = (uint8_t)msg_len; GPR_ASSERT(p == GPR_SLICE_END_PTR(message_pfx)); len += (uint32_t)GPR_SLICE_LENGTH(message_pfx); - len += (uint32_t)GPR_SLICE_LENGTH(*optional_message); + len += (uint32_t)msg_len; } hdr = gpr_slice_malloc(9); @@ -1555,53 +1606,54 @@ static void close_from_api(grpc_exec_ctx *exec_ctx, if (optional_message) { gpr_slice_buffer_add(&transport_global->qbuf, message_pfx); gpr_slice_buffer_add(&transport_global->qbuf, - gpr_slice_ref(*optional_message)); + gpr_slice_from_copied_string(optional_message)); } gpr_slice_buffer_add( &transport_global->qbuf, grpc_chttp2_rst_stream_create(stream_global->id, GRPC_CHTTP2_NO_ERROR, &stream_global->stats.outgoing)); - - if (optional_message) { - gpr_slice_ref(*optional_message); - } } - grpc_chttp2_fake_status(exec_ctx, transport_global, stream_global, status, - optional_message); - grpc_error *err = GRPC_ERROR_CREATE("Stream closed"); - err = grpc_error_set_int(err, GRPC_ERROR_INT_GRPC_STATUS, status); - if (optional_message) { - char *str = - gpr_dump_slice(*optional_message, GPR_DUMP_HEX | GPR_DUMP_ASCII); - err = grpc_error_set_str(err, GRPC_ERROR_STR_GRPC_MESSAGE, str); - gpr_free(str); + const char *msg = grpc_error_get_str(error, GRPC_ERROR_STR_GRPC_MESSAGE); + bool free_msg = false; + if (msg == NULL) { + free_msg = true; + msg = grpc_error_string(error); } + gpr_slice msg_slice = gpr_slice_from_copied_string(msg); + grpc_chttp2_fake_status(exec_ctx, transport_global, stream_global, + grpc_status, &msg_slice); + if (free_msg) grpc_error_free_string(msg); + grpc_chttp2_mark_stream_closed(exec_ctx, transport_global, stream_global, 1, - 1, err); + 1, error); } +typedef struct { + grpc_exec_ctx *exec_ctx; + grpc_error *error; +} cancel_stream_cb_args; + static void cancel_stream_cb(grpc_chttp2_transport_global *transport_global, void *user_data, grpc_chttp2_stream_global *stream_global) { - grpc_chttp2_transport *transport = TRANSPORT_FROM_GLOBAL(transport_global); - cancel_from_api(user_data, transport_global, stream_global, - GRPC_STATUS_UNAVAILABLE, - GPR_SLICE_IS_EMPTY(transport->optional_drop_message) - ? NULL - : &transport->optional_drop_message); + cancel_stream_cb_args *args = user_data; + cancel_from_api(args->exec_ctx, transport_global, stream_global, + GRPC_ERROR_REF(args->error)); } -static void end_all_the_calls(grpc_exec_ctx *exec_ctx, - grpc_chttp2_transport *t) { - grpc_chttp2_for_all_streams(&t->global, exec_ctx, cancel_stream_cb); +static void end_all_the_calls(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport *t, + grpc_error *error) { + cancel_stream_cb_args args = {exec_ctx, error}; + grpc_chttp2_for_all_streams(&t->global, &args, cancel_stream_cb); + GRPC_ERROR_UNREF(error); } static void drop_connection(grpc_exec_ctx *exec_ctx, grpc_chttp2_transport *t, grpc_error *error) { - close_transport_locked(exec_ctx, t, error); - end_all_the_calls(exec_ctx, t); + close_transport_locked(exec_ctx, t, GRPC_ERROR_REF(error)); + end_all_the_calls(exec_ctx, t, error); } /** update window from a settings change */ @@ -1708,15 +1760,7 @@ static void parsing_action(grpc_exec_ctx *exec_ctx, void *arg, t->read_buffer.slices[i]); }; if (i != t->read_buffer.count) { - gpr_slice_unref(t->optional_drop_message); errors[2] = try_http_parsing(exec_ctx, t); - if (errors[2] != GRPC_ERROR_NONE) { - t->optional_drop_message = gpr_slice_from_copied_string( - "Connection dropped: received http1.x response"); - } else { - t->optional_drop_message = gpr_slice_from_copied_string( - "Connection dropped: received unparseable response"); - } } grpc_error *err = errors[0] == GRPC_ERROR_NONE && errors[1] == GRPC_ERROR_NONE && @@ -1784,6 +1828,10 @@ static void post_reading_action_locked(grpc_exec_ctx *exec_ctx, error = GRPC_ERROR_CREATE("Transport closed"); } if (error != GRPC_ERROR_NONE) { + if (!grpc_error_get_int(error, GRPC_ERROR_INT_GRPC_STATUS, NULL)) { + error = grpc_error_set_int(error, GRPC_ERROR_INT_GRPC_STATUS, + GRPC_STATUS_UNAVAILABLE); + } drop_connection(exec_ctx, t, GRPC_ERROR_REF(error)); t->endpoint_reading = 0; if (!t->executor.writing_active && t->ep) { @@ -1798,6 +1846,7 @@ static void post_reading_action_locked(grpc_exec_ctx *exec_ctx, prevent_endpoint_shutdown(t); } gpr_slice_buffer_reset_and_unref(&t->read_buffer); + GRPC_ERROR_UNREF(error); if (keep_reading) { grpc_endpoint_read(exec_ctx, t->ep, &t->read_buffer, &t->reading_action); @@ -1806,8 +1855,6 @@ static void post_reading_action_locked(grpc_exec_ctx *exec_ctx, } else { UNREF_TRANSPORT(exec_ctx, t, "reading_action"); } - - GRPC_LOG_IF_ERROR("close_transport", error); } /******************************************************************************* diff --git a/src/core/ext/transport/chttp2/transport/internal.h b/src/core/ext/transport/chttp2/transport/internal.h index d63170e350b..7e281f1b1a2 100644 --- a/src/core/ext/transport/chttp2/transport/internal.h +++ b/src/core/ext/transport/chttp2/transport/internal.h @@ -384,9 +384,6 @@ struct grpc_chttp2_transport { /** Transport op to be applied post-parsing */ grpc_transport_op *post_parsing_op; - - /** Message explaining the reason of dropping connection */ - gpr_slice optional_drop_message; }; typedef struct { diff --git a/src/core/lib/channel/channel_stack.c b/src/core/lib/channel/channel_stack.c index bbba85d80ba..42075b127bd 100644 --- a/src/core/lib/channel/channel_stack.c +++ b/src/core/lib/channel/channel_stack.c @@ -263,6 +263,6 @@ void grpc_call_element_send_cancel(grpc_exec_ctx *exec_ctx, grpc_call_element *cur_elem) { grpc_transport_stream_op op; memset(&op, 0, sizeof(op)); - op.cancel_with_status = GRPC_STATUS_CANCELLED; + op.cancel_error = GRPC_ERROR_CANCELLED; grpc_call_next_op(exec_ctx, cur_elem, &op); } diff --git a/src/core/lib/iomgr/error.c b/src/core/lib/iomgr/error.c index 540fb4fa7e4..da0d6972360 100644 --- a/src/core/lib/iomgr/error.c +++ b/src/core/lib/iomgr/error.c @@ -37,6 +37,7 @@ #include #include +#include #include #include #include @@ -115,6 +116,8 @@ static const char *error_int_name(grpc_error_ints key) { return "wsa_error"; case GRPC_ERROR_INT_HTTP_STATUS: return "http_status"; + case GRPC_ERROR_INT_LIMIT: + return "limit"; } GPR_UNREACHABLE_CODE(return "unknown"); } @@ -271,6 +274,12 @@ grpc_error *grpc_error_set_int(grpc_error *src, grpc_error_ints which, bool grpc_error_get_int(grpc_error *err, grpc_error_ints which, intptr_t *p) { void *pp; + if (is_special(err)) { + if (err == GRPC_ERROR_CANCELLED && which == GRPC_ERROR_INT_GRPC_STATUS) { + return GRPC_STATUS_CANCELLED; + } + return false; + } if (gpr_avl_maybe_get(err->ints, (void *)(uintptr_t)which, &pp)) { if (p != NULL) *p = (intptr_t)pp; return true; @@ -286,6 +295,11 @@ grpc_error *grpc_error_set_str(grpc_error *src, grpc_error_strs which, return new; } +const char *grpc_error_get_str(grpc_error *err, grpc_error_strs which) { + if (is_special(err)) return NULL; + return gpr_avl_get(err->strs, (void *)(uintptr_t)which); +} + grpc_error *grpc_error_add_child(grpc_error *src, grpc_error *child) { grpc_error *new = copy_error_and_unref(src); new->errs = gpr_avl_add(new->errs, (void *)(new->next_err++), child); diff --git a/src/core/lib/iomgr/error.h b/src/core/lib/iomgr/error.h index 69cdf3028e4..13f898e31ad 100644 --- a/src/core/lib/iomgr/error.h +++ b/src/core/lib/iomgr/error.h @@ -92,6 +92,8 @@ typedef enum { GRPC_ERROR_INT_FD, /// HTTP status (i.e. 404) GRPC_ERROR_INT_HTTP_STATUS, + /// context sensitive limit associated with the error + GRPC_ERROR_INT_LIMIT, } grpc_error_ints; typedef enum { @@ -163,23 +165,25 @@ void grpc_error_unref(grpc_error *err); #endif grpc_error *grpc_error_set_int(grpc_error *src, grpc_error_ints which, - intptr_t value); + intptr_t value) GRPC_MUST_USE_RESULT; bool grpc_error_get_int(grpc_error *error, grpc_error_ints which, intptr_t *p); grpc_error *grpc_error_set_time(grpc_error *src, grpc_error_times which, - gpr_timespec value); + gpr_timespec value) GRPC_MUST_USE_RESULT; grpc_error *grpc_error_set_str(grpc_error *src, grpc_error_strs which, - const char *value); + const char *value) GRPC_MUST_USE_RESULT; +const char *grpc_error_get_str(grpc_error *error, grpc_error_strs which); /// Add a child error: an error that is believed to have contributed to this /// error occurring. Allows root causing high level errors from lower level /// errors that contributed to them. -grpc_error *grpc_error_add_child(grpc_error *src, grpc_error *child); +grpc_error *grpc_error_add_child(grpc_error *src, + grpc_error *child) GRPC_MUST_USE_RESULT; grpc_error *grpc_os_error(const char *file, int line, int err, - const char *call_name); + const char *call_name) GRPC_MUST_USE_RESULT; /// create an error associated with errno!=0 (an 'operating system' error) #define GRPC_OS_ERROR(err, call_name) \ grpc_os_error(__FILE__, __LINE__, err, call_name) grpc_error *grpc_wsa_error(const char *file, int line, int err, - const char *call_name); + const char *call_name) GRPC_MUST_USE_RESULT; /// windows only: create an error associated with WSAGetLastError()!=0 #define GRPC_WSA_ERROR(err, call_name) \ grpc_wsa_error(__FILE__, __LINE__, err, call_name) diff --git a/src/core/lib/security/transport/client_auth_filter.c b/src/core/lib/security/transport/client_auth_filter.c index 76be2acd728..e053afc745c 100644 --- a/src/core/lib/security/transport/client_auth_filter.c +++ b/src/core/lib/security/transport/client_auth_filter.c @@ -220,8 +220,7 @@ static void auth_start_transport_op(grpc_exec_ctx *exec_ctx, grpc_linked_mdelem *l; grpc_client_security_context *sec_ctx = NULL; - if (calld->security_context_set == 0 && - op->cancel_with_status == GRPC_STATUS_OK) { + if (calld->security_context_set == 0 && op->cancel_error == GRPC_ERROR_NONE) { calld->security_context_set = 1; GPR_ASSERT(op->context); if (op->context[GRPC_CONTEXT_SECURITY].value == NULL) { diff --git a/src/core/lib/surface/call.c b/src/core/lib/surface/call.c index 04291b0ee0a..77c17a4975b 100644 --- a/src/core/lib/surface/call.c +++ b/src/core/lib/surface/call.c @@ -402,8 +402,50 @@ static void set_status_code(grpc_call *call, status_source source, call->status[source].is_set = 1; call->status[source].code = (grpc_status_code)status; +} + +static void set_status_details(grpc_call *call, status_source source, + grpc_mdstr *status) { + if (call->status[source].details != NULL) { + GRPC_MDSTR_UNREF(call->status[source].details); + } + call->status[source].details = status; +} + +static void get_final_status(grpc_call *call, + void (*set_value)(grpc_status_code code, + void *user_data), + void *set_value_user_data) { + int i; + for (i = 0; i < STATUS_SOURCE_COUNT; i++) { + if (call->status[i].is_set) { + set_value(call->status[i].code, set_value_user_data); + return; + } + } + if (call->is_client) { + set_value(GRPC_STATUS_UNKNOWN, set_value_user_data); + } else { + set_value(GRPC_STATUS_OK, set_value_user_data); + } +} - /* TODO(ctiller): what to do about the flush that was previously here */ +static void set_status_from_error(grpc_call *call, status_source source, + grpc_error *error) { + intptr_t status; + if (grpc_error_get_int(error, GRPC_ERROR_INT_GRPC_STATUS, &status)) { + set_status_code(call, source, (uint32_t)status); + } else { + set_status_code(call, source, GRPC_STATUS_INTERNAL); + } + const char *msg = grpc_error_get_str(error, GRPC_ERROR_STR_GRPC_MESSAGE); + bool free_msg = false; + if (msg == NULL) { + free_msg = true; + msg = grpc_error_string(error); + } + set_status_details(call, source, grpc_mdstr_from_string(msg)); + if (free_msg) grpc_error_free_string(msg); } static void set_incoming_compression_algorithm( @@ -492,32 +534,6 @@ uint32_t grpc_call_test_only_get_encodings_accepted_by_peer(grpc_call *call) { return encodings_accepted_by_peer; } -static void set_status_details(grpc_call *call, status_source source, - grpc_mdstr *status) { - if (call->status[source].details != NULL) { - GRPC_MDSTR_UNREF(call->status[source].details); - } - call->status[source].details = status; -} - -static void get_final_status(grpc_call *call, - void (*set_value)(grpc_status_code code, - void *user_data), - void *set_value_user_data) { - int i; - for (i = 0; i < STATUS_SOURCE_COUNT; i++) { - if (call->status[i].is_set) { - set_value(call->status[i].code, set_value_user_data); - return; - } - } - if (call->is_client) { - set_value(GRPC_STATUS_UNKNOWN, set_value_user_data); - } else { - set_value(GRPC_STATUS_OK, set_value_user_data); - } -} - static void get_final_details(grpc_call *call, char **out_details, size_t *out_details_capacity) { int i; @@ -741,8 +757,7 @@ grpc_call_error grpc_call_cancel_with_status(grpc_call *c, typedef struct termination_closure { grpc_closure closure; grpc_call *call; - grpc_status_code status; - gpr_slice optional_message; + grpc_error *error; grpc_closure *op_closure; enum { TC_CANCEL, TC_CLOSE } type; } termination_closure; @@ -758,7 +773,7 @@ static void done_termination(grpc_exec_ctx *exec_ctx, void *tcp, GRPC_CALL_INTERNAL_UNREF(exec_ctx, tc->call, "close"); break; } - gpr_slice_unref(tc->optional_message); + GRPC_ERROR_UNREF(tc->error); grpc_exec_ctx_sched(exec_ctx, tc->op_closure, GRPC_ERROR_NONE, NULL); gpr_free(tc); } @@ -767,7 +782,7 @@ static void send_cancel(grpc_exec_ctx *exec_ctx, void *tcp, grpc_error *error) { grpc_transport_stream_op op; termination_closure *tc = tcp; memset(&op, 0, sizeof(op)); - op.cancel_with_status = tc->status; + op.cancel_error = tc->error; /* reuse closure to catch completion */ grpc_closure_init(&tc->closure, done_termination, tc); op.on_complete = &tc->closure; @@ -778,8 +793,7 @@ static void send_close(grpc_exec_ctx *exec_ctx, void *tcp, grpc_error *error) { grpc_transport_stream_op op; termination_closure *tc = tcp; memset(&op, 0, sizeof(op)); - tc->optional_message = gpr_slice_ref(tc->optional_message); - grpc_transport_stream_op_add_close(&op, tc->status, &tc->optional_message); + op.close_error = tc->error; /* reuse closure to catch completion */ grpc_closure_init(&tc->closure, done_termination, tc); tc->op_closure = op.on_complete; @@ -789,14 +803,7 @@ static void send_close(grpc_exec_ctx *exec_ctx, void *tcp, grpc_error *error) { static grpc_call_error terminate_with_status(grpc_exec_ctx *exec_ctx, termination_closure *tc) { - grpc_mdstr *details = NULL; - if (GPR_SLICE_LENGTH(tc->optional_message) > 0) { - tc->optional_message = gpr_slice_ref(tc->optional_message); - details = grpc_mdstr_from_slice(tc->optional_message); - } - - set_status_code(tc->call, STATUS_FROM_API_OVERRIDE, (uint32_t)tc->status); - set_status_details(tc->call, STATUS_FROM_API_OVERRIDE, details); + set_status_from_error(tc->call, STATUS_FROM_API_OVERRIDE, tc->error); if (tc->type == TC_CANCEL) { grpc_closure_init(&tc->closure, send_cancel, tc); @@ -812,13 +819,15 @@ static grpc_call_error terminate_with_status(grpc_exec_ctx *exec_ctx, static grpc_call_error cancel_with_status(grpc_exec_ctx *exec_ctx, grpc_call *c, grpc_status_code status, const char *description) { + GPR_ASSERT(status != GRPC_STATUS_OK); termination_closure *tc = gpr_malloc(sizeof(*tc)); memset(tc, 0, sizeof(termination_closure)); tc->type = TC_CANCEL; tc->call = c; - tc->optional_message = gpr_slice_from_copied_string(description); - GPR_ASSERT(status != GRPC_STATUS_OK); - tc->status = status; + tc->error = grpc_error_set_int( + grpc_error_set_str(GRPC_ERROR_CREATE(description), + GRPC_ERROR_STR_GRPC_MESSAGE, description), + GRPC_ERROR_INT_GRPC_STATUS, status); return terminate_with_status(exec_ctx, tc); } @@ -826,13 +835,15 @@ static grpc_call_error cancel_with_status(grpc_exec_ctx *exec_ctx, grpc_call *c, static grpc_call_error close_with_status(grpc_exec_ctx *exec_ctx, grpc_call *c, grpc_status_code status, const char *description) { + GPR_ASSERT(status != GRPC_STATUS_OK); termination_closure *tc = gpr_malloc(sizeof(*tc)); memset(tc, 0, sizeof(termination_closure)); tc->type = TC_CLOSE; tc->call = c; - tc->optional_message = gpr_slice_from_copied_string(description); - GPR_ASSERT(status != GRPC_STATUS_OK); - tc->status = status; + tc->error = grpc_error_set_int( + grpc_error_set_str(GRPC_ERROR_CREATE(description), + GRPC_ERROR_STR_GRPC_MESSAGE, description), + GRPC_ERROR_INT_GRPC_STATUS, status); return terminate_with_status(exec_ctx, tc); } diff --git a/src/core/lib/surface/completion_queue.c b/src/core/lib/surface/completion_queue.c index 2cc6aa74e02..db8010ef9a1 100644 --- a/src/core/lib/surface/completion_queue.c +++ b/src/core/lib/surface/completion_queue.c @@ -240,7 +240,7 @@ void grpc_cq_end_op(grpc_exec_ctx *exec_ctx, grpc_completion_queue *cc, "grpc_cq_end_op(exec_ctx=%p, cc=%p, tag=%p, error=%s, done=%p, " "done_arg=%p, storage=%p)", 7, (exec_ctx, cc, tag, errmsg, done, done_arg, storage)); - if (grpc_trace_operation_failures) { + if (grpc_trace_operation_failures && error != GRPC_ERROR_NONE) { gpr_log(GPR_ERROR, "Operation failed: tag=%p, error=%s", tag, errmsg); } grpc_error_free_string(errmsg); diff --git a/src/core/lib/transport/transport.c b/src/core/lib/transport/transport.c index 1105494a85c..67920b05279 100644 --- a/src/core/lib/transport/transport.c +++ b/src/core/lib/transport/transport.c @@ -36,6 +36,7 @@ #include #include #include +#include "src/core/lib/support/string.h" #include "src/core/lib/transport/transport_impl.h" #ifdef GRPC_STREAM_REFCOUNT_DEBUG @@ -162,55 +163,63 @@ void grpc_transport_stream_op_finish_with_failure(grpc_exec_ctx *exec_ctx, grpc_exec_ctx_sched(exec_ctx, op->on_complete, error, NULL); } -void grpc_transport_stream_op_add_cancellation(grpc_transport_stream_op *op, - grpc_status_code status) { - GPR_ASSERT(status != GRPC_STATUS_OK); - if (op->cancel_with_status == GRPC_STATUS_OK) { - op->cancel_with_status = status; - } - if (op->close_with_status != GRPC_STATUS_OK) { - op->close_with_status = GRPC_STATUS_OK; - if (op->optional_close_message != NULL) { - gpr_slice_unref(*op->optional_close_message); - op->optional_close_message = NULL; - } - } -} - typedef struct { - gpr_slice message; + grpc_error *error; grpc_closure *then_call; grpc_closure closure; } close_message_data; static void free_message(grpc_exec_ctx *exec_ctx, void *p, grpc_error *error) { close_message_data *cmd = p; - gpr_slice_unref(cmd->message); + GRPC_ERROR_UNREF(cmd->error); if (cmd->then_call != NULL) { cmd->then_call->cb(exec_ctx, cmd->then_call->cb_arg, GRPC_ERROR_REF(error)); } gpr_free(cmd); } +static void add_error(grpc_transport_stream_op *op, grpc_error **which, + grpc_error *error) { + close_message_data *cmd; + cmd = gpr_malloc(sizeof(*cmd)); + cmd->error = error; + cmd->then_call = op->on_complete; + grpc_closure_init(&cmd->closure, free_message, cmd); + op->on_complete = &cmd->closure; + *which = error; +} + +void grpc_transport_stream_op_add_cancellation(grpc_transport_stream_op *op, + grpc_status_code status) { + GPR_ASSERT(status != GRPC_STATUS_OK); + if (op->cancel_error == GRPC_STATUS_OK) { + op->cancel_error = grpc_error_set_int(GRPC_ERROR_CANCELLED, + GRPC_ERROR_INT_GRPC_STATUS, status); + op->close_error = GRPC_ERROR_NONE; + } +} + void grpc_transport_stream_op_add_close(grpc_transport_stream_op *op, grpc_status_code status, gpr_slice *optional_message) { - close_message_data *cmd; GPR_ASSERT(status != GRPC_STATUS_OK); - if (op->cancel_with_status != GRPC_STATUS_OK || - op->close_with_status != GRPC_STATUS_OK) { + if (op->cancel_error != GRPC_ERROR_NONE || + op->close_error != GRPC_ERROR_NONE) { if (optional_message) { gpr_slice_unref(*optional_message); } return; } - if (optional_message) { - cmd = gpr_malloc(sizeof(*cmd)); - cmd->message = *optional_message; - cmd->then_call = op->on_complete; - grpc_closure_init(&cmd->closure, free_message, cmd); - op->on_complete = &cmd->closure; - op->optional_close_message = &cmd->message; + grpc_error *error; + if (optional_message != NULL) { + char *msg = gpr_dump_slice(*optional_message, GPR_DUMP_ASCII); + error = grpc_error_set_str(GRPC_ERROR_CREATE(msg), + GRPC_ERROR_STR_GRPC_MESSAGE, msg); + gpr_free(msg); + gpr_slice_unref(*optional_message); + } else { + error = GRPC_ERROR_CREATE("Call force closed"); } - op->close_with_status = status; + error = grpc_error_set_int(error, GRPC_ERROR_INT_GRPC_STATUS, status); + add_error(op, &op->close_error, error); } diff --git a/src/core/lib/transport/transport.h b/src/core/lib/transport/transport.h index a46ccb643ce..d2f6344ee3c 100644 --- a/src/core/lib/transport/transport.h +++ b/src/core/lib/transport/transport.h @@ -135,13 +135,12 @@ typedef struct grpc_transport_stream_op { /** Collect any stats into provided buffer, zero internal stat counters */ grpc_transport_stream_stats *collect_stats; - /** If != GRPC_STATUS_OK, cancel this stream */ - grpc_status_code cancel_with_status; + /** If != GRPC_ERROR_NONE, cancel this stream */ + grpc_error *cancel_error; - /** If != GRPC_STATUS_OK, send grpc-status, grpc-message, and close this + /** If != GRPC_ERROR, send grpc-status, grpc-message, and close this stream for both reading and writing */ - grpc_status_code close_with_status; - gpr_slice *optional_close_message; + grpc_error *close_error; /* Indexes correspond to grpc_context_index enum values */ grpc_call_context_element *context; diff --git a/src/core/lib/transport/transport_op_string.c b/src/core/lib/transport/transport_op_string.c index df04c611270..a862401df2c 100644 --- a/src/core/lib/transport/transport_op_string.c +++ b/src/core/lib/transport/transport_op_string.c @@ -119,10 +119,21 @@ char *grpc_transport_stream_op_string(grpc_transport_stream_op *op) { gpr_strvec_add(&b, gpr_strdup("RECV_TRAILING_METADATA")); } - if (op->cancel_with_status != GRPC_STATUS_OK) { + if (op->cancel_error != GRPC_STATUS_OK) { if (!first) gpr_strvec_add(&b, gpr_strdup(" ")); first = 0; - gpr_asprintf(&tmp, "CANCEL:%d", op->cancel_with_status); + const char *msg = grpc_error_string(op->cancel_error); + gpr_asprintf(&tmp, "CANCEL:%s", msg); + grpc_error_free_string(msg); + gpr_strvec_add(&b, tmp); + } + + if (op->close_error != GRPC_STATUS_OK) { + if (!first) gpr_strvec_add(&b, gpr_strdup(" ")); + first = 0; + const char *msg = grpc_error_string(op->close_error); + gpr_asprintf(&tmp, "CLOSE:%s", msg); + grpc_error_free_string(msg); gpr_strvec_add(&b, tmp); } From 5a3c6389edeaf19e7e0f0a6c0771c7548f0e0997 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 23 Jun 2016 14:02:33 -0700 Subject: [PATCH 191/280] Added braces around _service --- .../objective-c/route_guide/ViewControllers.m | 20 +++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/examples/objective-c/route_guide/ViewControllers.m b/examples/objective-c/route_guide/ViewControllers.m index 26ca9d6220b..b2f99c437e7 100644 --- a/examples/objective-c/route_guide/ViewControllers.m +++ b/examples/objective-c/route_guide/ViewControllers.m @@ -86,8 +86,9 @@ static NSString * const kHostAddress = @"localhost:50051"; @end -@implementation GetFeatureViewController -RTGRouteGuide *_service; +@implementation GetFeatureViewController { + RTGRouteGuide *_service; +} - (void)execRequest { void (^handler)(RTGFeature *response, NSError *error) = ^(RTGFeature *response, NSError *error) { @@ -146,8 +147,9 @@ RTGRouteGuide *_service; @end -@implementation ListFeaturesViewController -RTGRouteGuide *_service; +@implementation ListFeaturesViewController { + RTGRouteGuide *_service; +} - (void)execRequest { RTGRectangle *rectangle = [RTGRectangle message]; @@ -200,8 +202,9 @@ RTGRouteGuide *_service; @end -@implementation RecordRouteViewController -RTGRouteGuide *_service; +@implementation RecordRouteViewController { + RTGRouteGuide *_service; +} - (void)execRequest { NSString *dataBasePath = [NSBundle.mainBundle pathForResource:@"route_guide_db" @@ -268,8 +271,9 @@ RTGRouteGuide *_service; @end -@implementation RouteChatViewController -RTGRouteGuide *_service; +@implementation RouteChatViewController { + RTGRouteGuide *_service; +} - (void)execRequest { NSArray *notes = @[[RTGRouteNote noteWithMessage:@"First message" latitude:0 longitude:0], From 733e3fc209f78695896b2d721069afad866cd2f7 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 23 Jun 2016 14:07:11 -0700 Subject: [PATCH 192/280] Fix comparison --- src/core/lib/transport/transport_op_string.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/core/lib/transport/transport_op_string.c b/src/core/lib/transport/transport_op_string.c index a862401df2c..aede337e7cb 100644 --- a/src/core/lib/transport/transport_op_string.c +++ b/src/core/lib/transport/transport_op_string.c @@ -119,7 +119,7 @@ char *grpc_transport_stream_op_string(grpc_transport_stream_op *op) { gpr_strvec_add(&b, gpr_strdup("RECV_TRAILING_METADATA")); } - if (op->cancel_error != GRPC_STATUS_OK) { + if (op->cancel_error != GRPC_ERROR_NONE) { if (!first) gpr_strvec_add(&b, gpr_strdup(" ")); first = 0; const char *msg = grpc_error_string(op->cancel_error); @@ -128,7 +128,7 @@ char *grpc_transport_stream_op_string(grpc_transport_stream_op *op) { gpr_strvec_add(&b, tmp); } - if (op->close_error != GRPC_STATUS_OK) { + if (op->close_error != GRPC_ERROR_NONE) { if (!first) gpr_strvec_add(&b, gpr_strdup(" ")); first = 0; const char *msg = grpc_error_string(op->close_error); From ea0dc6af37469a45c2750df4bf852d12d75e9240 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 23 Jun 2016 14:10:38 -0700 Subject: [PATCH 193/280] Added a comment --- src/objective-c/GRPCClient/private/GRPCChannel.m | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/objective-c/GRPCClient/private/GRPCChannel.m b/src/objective-c/GRPCClient/private/GRPCChannel.m index 6cd4cb0ca30..7b7b79e1c62 100644 --- a/src/objective-c/GRPCClient/private/GRPCChannel.m +++ b/src/objective-c/GRPCClient/private/GRPCChannel.m @@ -199,7 +199,7 @@ grpc_channel_args * buildChannelArgs(NSDictionary *dictionary) { NULL, GRPC_PROPAGATE_DEFAULTS, queue.unmanagedQueue, path.UTF8String, - NULL, + NULL, // Passing NULL for host gpr_inf_future(GPR_CLOCK_REALTIME), NULL); } From eb429c3067e54b0dcdae21d005a4e98a022caad8 Mon Sep 17 00:00:00 2001 From: Makarand Dharmapurikar Date: Thu, 23 Jun 2016 14:18:49 -0700 Subject: [PATCH 194/280] Removed gpr_log statement --- src/core/lib/iomgr/network_status_tracker.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/src/core/lib/iomgr/network_status_tracker.c b/src/core/lib/iomgr/network_status_tracker.c index 0e34605c81d..38a1c9b7d41 100644 --- a/src/core/lib/iomgr/network_status_tracker.c +++ b/src/core/lib/iomgr/network_status_tracker.c @@ -64,7 +64,6 @@ void grpc_network_status_register_endpoint(grpc_endpoint *ep) { grpc_initialize_network_status_monitor(); } gpr_mu_lock(&g_endpoint_mutex); - gpr_log(GPR_DEBUG, "Register endpoint %p", ep); if (head == NULL) { head = (endpoint_ll_node *)gpr_malloc(sizeof(endpoint_ll_node)); head->ep = ep; @@ -81,7 +80,6 @@ void grpc_network_status_register_endpoint(grpc_endpoint *ep) { void grpc_network_status_unregister_endpoint(grpc_endpoint *ep) { gpr_mu_lock(&g_endpoint_mutex); GPR_ASSERT(head); - gpr_log(GPR_DEBUG, "Unregister endpoint %p", ep); bool found = false; endpoint_ll_node *prev = head; // if we're unregistering the head, just move head to the next @@ -116,7 +114,6 @@ void grpc_network_status_shutdown_all_endpoints() { grpc_exec_ctx exec_ctx = GRPC_EXEC_CTX_INIT; for (endpoint_ll_node *curr = head; curr != NULL; curr = curr->next) { - gpr_log(GPR_DEBUG, "Shutting down endpoint %p", curr->ep); curr->ep->vtable->shutdown(&exec_ctx, curr->ep); } gpr_mu_unlock(&g_endpoint_mutex); From d4d4c6f6f62a547da9136f57ddd6406facaf4ce5 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 23 Jun 2016 14:37:30 -0700 Subject: [PATCH 195/280] Fix comparison --- src/core/lib/transport/transport.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/core/lib/transport/transport.c b/src/core/lib/transport/transport.c index 67920b05279..79a20e12626 100644 --- a/src/core/lib/transport/transport.c +++ b/src/core/lib/transport/transport.c @@ -192,7 +192,7 @@ static void add_error(grpc_transport_stream_op *op, grpc_error **which, void grpc_transport_stream_op_add_cancellation(grpc_transport_stream_op *op, grpc_status_code status) { GPR_ASSERT(status != GRPC_STATUS_OK); - if (op->cancel_error == GRPC_STATUS_OK) { + if (op->cancel_error == GRPC_ERROR_NONE) { op->cancel_error = grpc_error_set_int(GRPC_ERROR_CANCELLED, GRPC_ERROR_INT_GRPC_STATUS, status); op->close_error = GRPC_ERROR_NONE; From 20d0a167beb287f61a7f33943fddfc34cae75860 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 23 Jun 2016 15:14:03 -0700 Subject: [PATCH 196/280] Better error handling and add polling_island_unlock_pair() helper --- src/core/lib/iomgr/ev_epoll_linux.c | 300 +++++++++++++++++----------- 1 file changed, 182 insertions(+), 118 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index b1e9ac8a631..a77044edc50 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -237,6 +237,19 @@ struct grpc_pollset_set { grpc_fd **fds; }; +/******************************************************************************* + * Common helpers + */ + +static void append_error(grpc_error **composite, grpc_error *error, + const char *desc) { + if (error == GRPC_ERROR_NONE) return; + if (*composite == GRPC_ERROR_NONE) { + *composite = GRPC_ERROR_CREATE(desc); + } + *composite = grpc_error_add_child(*composite, error); +} + /******************************************************************************* * Polling island Definitions */ @@ -316,10 +329,13 @@ long pi_unref(polling_island *pi, int ref_cnt) { /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, - size_t fd_count, bool add_fd_refs) { + size_t fd_count, bool add_fd_refs, + grpc_error **error) { int err; size_t i; struct epoll_event ev; + char *err_msg; + const char *err_desc = "polling_island_add_fds"; #ifdef GRPC_TSAN /* See the definition of g_epoll_sync for more context */ @@ -333,10 +349,12 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, if (err < 0) { if (errno != EEXIST) { - /* TODO: sreek - We need a better way to bubble up this error instead of - just logging a message */ - gpr_log(GPR_ERROR, "epoll_ctl add for fd: %d failed with error: %s", - fds[i]->fd, strerror(errno)); + gpr_asprintf( + &err_msg, + "epoll_ctl (epoll_fd: %d) add fd: %d failed with error: %d (%s)", + pi->epoll_fd, fds[i]->fd, errno, strerror(errno)); + append_error(error, GRPC_OS_ERROR(errno, err_msg), err_desc); + gpr_free(err_msg); } continue; @@ -356,37 +374,47 @@ static void polling_island_add_fds_locked(polling_island *pi, grpc_fd **fds, /* The caller is expected to hold pi->mu before calling this */ static void polling_island_add_wakeup_fd_locked(polling_island *pi, - grpc_wakeup_fd *wakeup_fd) { + grpc_wakeup_fd *wakeup_fd, + grpc_error **error) { struct epoll_event ev; int err; + char *err_msg; + const char *err_desc = "polling_island_add_wakeup_fd"; ev.events = (uint32_t)(EPOLLIN | EPOLLET); ev.data.ptr = wakeup_fd; err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_ADD, GRPC_WAKEUP_FD_GET_READ_FD(wakeup_fd), &ev); - if (err < 0) { - gpr_log(GPR_ERROR, - "Failed to add grpc_wake_up_fd (%d) to the epoll set (epoll_fd: %d)" - ". Error: %s", - GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), pi->epoll_fd, - strerror(errno)); + if (err < 0 && errno != EEXIST) { + gpr_asprintf(&err_msg, + "epoll_ctl (epoll_fd: %d) add wakeup fd: %d failed with " + "error: %d (%s)", + pi->epoll_fd, + GRPC_WAKEUP_FD_GET_READ_FD(&grpc_global_wakeup_fd), errno, + strerror(errno)); + append_error(error, GRPC_OS_ERROR(errno, err_msg), err_desc); + gpr_free(err_msg); } } /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_remove_all_fds_locked(polling_island *pi, - bool remove_fd_refs) { + bool remove_fd_refs, + grpc_error **error) { int err; size_t i; + char *err_msg; + const char *err_desc = "polling_island_remove_fds"; for (i = 0; i < pi->fd_cnt; i++) { err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, pi->fds[i]->fd, NULL); if (err < 0 && errno != ENOENT) { - /* TODO: sreek - We need a better way to bubble up this error instead of - * just logging a message */ - gpr_log(GPR_ERROR, - "epoll_ctl deleting fds[%zu]: %d failed with error: %s", i, - pi->fds[i]->fd, strerror(errno)); + gpr_asprintf(&err_msg, + "epoll_ctl (epoll_fd: %d) delete fds[%zu]: %d failed with " + "error: %d (%s)", + pi->epoll_fd, i, pi->fds[i]->fd, errno, strerror(errno)); + append_error(error, GRPC_OS_ERROR(errno, err_msg), err_desc); + gpr_free(err_msg); } if (remove_fd_refs) { @@ -399,17 +427,24 @@ static void polling_island_remove_all_fds_locked(polling_island *pi, /* The caller is expected to hold pi->mu lock before calling this function */ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, - bool is_fd_closed) { + bool is_fd_closed, + grpc_error **error) { int err; size_t i; + char *err_msg; + const char *err_desc = "polling_island_remove_fd"; /* If fd is already closed, then it would have been automatically been removed from the epoll set */ if (!is_fd_closed) { err = epoll_ctl(pi->epoll_fd, EPOLL_CTL_DEL, fd->fd, NULL); if (err < 0 && errno != ENOENT) { - gpr_log(GPR_ERROR, "epoll_ctl deleting fd: %d failed with error; %s", - fd->fd, strerror(errno)); + gpr_asprintf( + &err_msg, + "epoll_ctl (epoll_fd: %d) del fd: %d failed with error: %d (%s)", + pi->epoll_fd, fd->fd, errno, strerror(errno)); + append_error(error, GRPC_OS_ERROR(errno, err_msg), err_desc); + gpr_free(err_msg); } } @@ -422,8 +457,12 @@ static void polling_island_remove_fd_locked(polling_island *pi, grpc_fd *fd, } } -static polling_island *polling_island_create(grpc_fd *initial_fd) { +/* Might return NULL in case of an error */ +static polling_island *polling_island_create(grpc_fd *initial_fd, + grpc_error **error) { polling_island *pi = NULL; + char *err_msg; + const char *err_desc = "polling_island_create"; /* Try to get one from the polling island freelist */ gpr_mu_lock(&g_pi_freelist_mu); @@ -449,22 +488,22 @@ static polling_island *polling_island_create(grpc_fd *initial_fd) { pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); if (pi->epoll_fd < 0) { - gpr_log(GPR_ERROR, "epoll_create1() failed with error: %s", - strerror(errno)); - } - GPR_ASSERT(pi->epoll_fd >= 0); - - polling_island_add_wakeup_fd_locked(pi, &grpc_global_wakeup_fd); - - pi->next_free = NULL; + gpr_asprintf(&err_msg, "epoll_create1 failed with error %d (%s)", errno, + strerror(errno)); + append_error(error, GRPC_OS_ERROR(errno, err_msg), err_desc); + gpr_free(err_msg); + } else { + polling_island_add_wakeup_fd_locked(pi, &grpc_global_wakeup_fd, error); + pi->next_free = NULL; - if (initial_fd != NULL) { - /* Lock the polling island here just in case we got this structure from the - freelist and the polling island lock was not released yet (by the code - that adds the polling island to the freelist) */ - gpr_mu_lock(&pi->mu); - polling_island_add_fds_locked(pi, &initial_fd, 1, true); - gpr_mu_unlock(&pi->mu); + if (initial_fd != NULL) { + /* Lock the polling island here just in case we got this structure from + the freelist and the polling island lock was not released yet (by the + code that adds the polling island to the freelist) */ + gpr_mu_lock(&pi->mu); + polling_island_add_fds_locked(pi, &initial_fd, 1, true, error); + gpr_mu_unlock(&pi->mu); + } } return pi; @@ -534,7 +573,9 @@ static polling_island *polling_island_lock(polling_island *pi) { return pi; } -/* Gets the lock on the *latest* polling islands pointed by *p and *q. +/* Gets the lock on the *latest* polling islands in the linked lists pointed by + *p and *q (and also updates *p and *q to point to the latest polling islands) + This function is needed because calling the following block of code to obtain locks on polling islands (*p and *q) is prone to deadlocks. { @@ -550,18 +591,8 @@ static polling_island *polling_island_lock(polling_island *pi) { .. .. Critical section with both p1 and p2 locked .. - // Release locks - // **IMPORTANT**: Make sure you check p1 == p2 AFTER the function - // polling_island_lock_pair() was called and if so, release the lock only - // once. Note: Even if p1 != p2 beforec calling polling_island_lock_pair(), - // they might be after the function returns: - if (p1 == p2) { - gpr_mu_unlock(&p1->mu) - } else { - gpr_mu_unlock(&p1->mu); - gpr_mu_unlock(&p2->mu); - } - + // Release locks: Always call polling_island_unlock_pair() to release locks + polling_island_unlock_pair(p1, p2); */ static void polling_island_lock_pair(polling_island **p, polling_island **q) { polling_island *pi_1 = *p; @@ -623,39 +654,46 @@ static void polling_island_lock_pair(polling_island **p, polling_island **q) { *q = pi_2; } -static polling_island *polling_island_merge(polling_island *p, - polling_island *q) { - /* Get locks on both the polling islands */ - polling_island_lock_pair(&p, &q); - +static void polling_island_unlock_pair(polling_island *p, polling_island *q) { if (p == q) { - /* Nothing needs to be done here */ gpr_mu_unlock(&p->mu); - return p; + } else { + gpr_mu_unlock(&p->mu); + gpr_mu_unlock(&q->mu); } +} - /* Make sure that p points to the polling island with fewer fds than q */ - if (p->fd_cnt > q->fd_cnt) { - GPR_SWAP(polling_island *, p, q); - } +static polling_island *polling_island_merge(polling_island *p, + polling_island *q, + grpc_error **error) { + /* Get locks on both the polling islands */ + polling_island_lock_pair(&p, &q); - /* "Merge" p with q i.e move all the fds from p (The one with fewer fds) to q - Note that the refcounts on the fds being moved will not change here. This - is why the last parameter in the following two functions is 'false') */ - polling_island_add_fds_locked(q, p->fds, p->fd_cnt, false); - polling_island_remove_all_fds_locked(p, false); + if (p != q) { + /* Make sure that p points to the polling island with fewer fds than q */ + if (p->fd_cnt > q->fd_cnt) { + GPR_SWAP(polling_island *, p, q); + } + + /* Merge p with q i.e move all the fds from p (The one with fewer fds) to q + Note that the refcounts on the fds being moved will not change here. + This is why the last param in the following two functions is 'false') */ + polling_island_add_fds_locked(q, p->fds, p->fd_cnt, false, error); + polling_island_remove_all_fds_locked(p, false, error); - /* Wakeup all the pollers (if any) on p so that they can pickup this change */ - polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd); + /* Wakeup all the pollers (if any) on p so that they pickup this change */ + polling_island_add_wakeup_fd_locked(p, &polling_island_wakeup_fd, error); - /* Add the 'merged_to' link from p --> q */ - gpr_atm_rel_store(&p->merged_to, (gpr_atm)q); - PI_ADD_REF(q, "pi_merge"); /* To account for the new incoming ref from p */ + /* Add the 'merged_to' link from p --> q */ + gpr_atm_rel_store(&p->merged_to, (gpr_atm)q); + PI_ADD_REF(q, "pi_merge"); /* To account for the new incoming ref from p */ + } + /* else if p == q, nothing needs to be done */ - gpr_mu_unlock(&p->mu); - gpr_mu_unlock(&q->mu); + polling_island_unlock_pair(p, q); - /* Return the merged polling island */ + /* Return the merged polling island (Note that no merge would have happened + if p == q which is ok) */ return q; } @@ -853,6 +891,8 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, grpc_closure *on_done, int *release_fd, const char *reason) { bool is_fd_closed = false; + grpc_error *error = GRPC_ERROR_NONE; + gpr_mu_lock(&fd->mu); fd->on_done_closure = on_done; @@ -882,7 +922,7 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, gpr_mu_lock(&fd->pi_mu); if (fd->polling_island != NULL) { polling_island *pi_latest = polling_island_lock(fd->polling_island); - polling_island_remove_fd_locked(pi_latest, fd, is_fd_closed); + polling_island_remove_fd_locked(pi_latest, fd, is_fd_closed, &error); gpr_mu_unlock(&pi_latest->mu); PI_UNREF(fd->polling_island, "fd_orphan"); @@ -890,10 +930,11 @@ static void fd_orphan(grpc_exec_ctx *exec_ctx, grpc_fd *fd, } gpr_mu_unlock(&fd->pi_mu); - grpc_exec_ctx_sched(exec_ctx, fd->on_done_closure, GRPC_ERROR_NONE, NULL); + grpc_exec_ctx_sched(exec_ctx, fd->on_done_closure, error, NULL); gpr_mu_unlock(&fd->mu); UNREF_BY(fd, 2, reason); /* Drop the reference */ + GRPC_LOG_IF_ERROR("fd_orphan", GRPC_ERROR_REF(error)); } static grpc_error *fd_shutdown_error(bool shutdown) { @@ -1062,19 +1103,12 @@ static void push_front_worker(grpc_pollset *p, grpc_pollset_worker *worker) { worker->prev->next = worker->next->prev = worker; } -static void kick_append_error(grpc_error **composite, grpc_error *error) { - if (error == GRPC_ERROR_NONE) return; - if (*composite == GRPC_ERROR_NONE) { - *composite = GRPC_ERROR_CREATE("Kick Failure"); - } - *composite = grpc_error_add_child(*composite, error); -} - /* p->mu must be held before calling this function */ static grpc_error *pollset_kick(grpc_pollset *p, grpc_pollset_worker *specific_worker) { GPR_TIMER_BEGIN("pollset_kick", 0); grpc_error *error = GRPC_ERROR_NONE; + const char *err_desc = "Kick Failure"; grpc_pollset_worker *worker = specific_worker; if (worker != NULL) { @@ -1084,7 +1118,7 @@ static grpc_error *pollset_kick(grpc_pollset *p, for (worker = p->root_worker.next; worker != &p->root_worker; worker = worker->next) { if (gpr_tls_get(&g_current_thread_worker) != (intptr_t)worker) { - kick_append_error(&error, pollset_worker_kick(worker)); + append_error(&error, pollset_worker_kick(worker), err_desc); } } } else { @@ -1094,7 +1128,7 @@ static grpc_error *pollset_kick(grpc_pollset *p, } else { GPR_TIMER_MARK("kicked_specifically", 0); if (gpr_tls_get(&g_current_thread_worker) != (intptr_t)worker) { - kick_append_error(&error, pollset_worker_kick(worker)); + append_error(&error, pollset_worker_kick(worker), err_desc); } } } else if (gpr_tls_get(&g_current_thread_pollset) != (intptr_t)p) { @@ -1110,7 +1144,7 @@ static grpc_error *pollset_kick(grpc_pollset *p, if (worker != NULL) { GPR_TIMER_MARK("finally_kick", 0); push_back_worker(p, worker); - kick_append_error(&error, pollset_worker_kick(worker)); + append_error(&error, pollset_worker_kick(worker), err_desc); } else { GPR_TIMER_MARK("kicked_no_pollers", 0); p->kicked_without_pollers = true; @@ -1238,23 +1272,17 @@ static void pollset_reset(grpc_pollset *pollset) { pollset_release_polling_island(pollset, "ps_reset"); } -static void work_combine_error(grpc_error **composite, grpc_error *error) { - if (error == GRPC_ERROR_NONE) return; - if (*composite == GRPC_ERROR_NONE) { - *composite = GRPC_ERROR_CREATE("pollset_work"); - } - *composite = grpc_error_add_child(*composite, error); -} - #define GRPC_EPOLL_MAX_EVENTS 1000 -static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, - grpc_pollset *pollset, - int timeout_ms, sigset_t *sig_mask) { +/* Note: sig_mask contains the signal mask to use *during* epoll_wait() */ +static void pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, + grpc_pollset *pollset, int timeout_ms, + sigset_t *sig_mask, grpc_error **error) { struct epoll_event ep_ev[GRPC_EPOLL_MAX_EVENTS]; int epoll_fd = -1; int ep_rv; polling_island *pi = NULL; - grpc_error *error = GRPC_ERROR_NONE; + char *err_msg; + const char *err_desc = "pollset_work_and_unlock"; GPR_TIMER_BEGIN("pollset_work_and_unlock", 0); /* We need to get the epoll_fd to wait on. The epoll_fd is in inside the @@ -1265,11 +1293,15 @@ static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, which we got the epoll_fd) got merged with another island while we are in this function. This is still okay because in such a case, we will wakeup right-away from epoll_wait() and pick up the latest polling_island the next - this function (i.e pollset_work_and_unlock()) is called. - */ + this function (i.e pollset_work_and_unlock()) is called */ if (pollset->polling_island == NULL) { - pollset->polling_island = polling_island_create(NULL); + pollset->polling_island = polling_island_create(NULL, error); + if (pollset->polling_island == NULL) { + GPR_TIMER_END("pollset_work_and_unlock", 0); + return; /* Fatal error. We cannot continue */ + } + PI_ADD_REF(pollset->polling_island, "ps"); } @@ -1297,8 +1329,10 @@ static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, sig_mask); if (ep_rv < 0) { if (errno != EINTR) { - gpr_log(GPR_ERROR, "epoll_pwait() failed: %s", strerror(errno)); - work_combine_error(&error, GRPC_OS_ERROR(errno, "epoll_pwait")); + gpr_asprintf(&err_msg, + "epoll_wait() epoll fd: %d failed with error: %d (%s)", + epoll_fd, errno, strerror(errno)); + append_error(error, GRPC_OS_ERROR(errno, err_msg), err_desc); } else { /* We were interrupted. Save an interation by doing a zero timeout epoll_wait to see if there are any other events of interest */ @@ -1314,8 +1348,9 @@ static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, for (int i = 0; i < ep_rv; ++i) { void *data_ptr = ep_ev[i].data.ptr; if (data_ptr == &grpc_global_wakeup_fd) { - work_combine_error( - &error, grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd)); + append_error(error, + grpc_wakeup_fd_consume_wakeup(&grpc_global_wakeup_fd), + err_desc); } else if (data_ptr == &polling_island_wakeup_fd) { /* This means that our polling island is merged with a different island. We do not have to do anything here since the subsequent call @@ -1346,7 +1381,6 @@ static grpc_error *pollset_work_and_unlock(grpc_exec_ctx *exec_ctx, PI_UNREF(pi, "ps_work"); GPR_TIMER_END("pollset_work_and_unlock", 0); - return error; } /* pollset->mu lock must be held by the caller before calling this. @@ -1368,6 +1402,7 @@ static grpc_error *pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, worker.pt_id = pthread_self(); *worker_hdl = &worker; + gpr_tls_set(&g_current_thread_pollset, (intptr_t)pollset); gpr_tls_set(&g_current_thread_worker, (intptr_t)&worker); @@ -1379,14 +1414,37 @@ static grpc_error *pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, GPR_TIMER_MARK("pollset_work.kicked_without_pollers", 0); pollset->kicked_without_pollers = 0; } else if (!pollset->shutting_down) { + /* We use the posix-signal with number 'grpc_wakeup_signal' for waking up + (i.e 'kicking') a worker in the pollset. + A 'kick' is a way to inform that worker that there is some pending work + that needs immediate attention (like an event on the completion queue, + or a polling island merge that results in a new epoll-fd to wait on) and + that the worker should not spend time waiting in epoll_pwait(). + + A kick can come at anytime (i.e before/during or after the worker calls + epoll_pwait()) but in all cases we have to make sure that when a worker + gets a kick, it does not spend time in epoll_pwait(). In other words, one + kick should result in skipping/exiting of one epoll_pwait(); + + To accomplish this, we mask 'grpc_wakeup_signal' on this worker at all + times *except* when it is in epoll_pwait(). This way, the worker never + misses acting on a kick */ + sigemptyset(&new_mask); sigaddset(&new_mask, grpc_wakeup_signal); pthread_sigmask(SIG_BLOCK, &new_mask, &orig_mask); sigdelset(&orig_mask, grpc_wakeup_signal); + /* new_mask: The new thread mask which blocks 'grpc_wakeup_signal'. This is + the mask used at all times *except during epoll_wait()*" + orig_mask: The thread mask which allows 'grpc_wakeup_signal' and this is + the mask to use *during epoll_wait()* + + The new_mask is set on the worker before it is added to the pollset (i.e + before it can be kicked) */ - push_front_worker(pollset, &worker); + push_front_worker(pollset, &worker); /* Add worker to pollset */ - error = pollset_work_and_unlock(exec_ctx, pollset, timeout_ms, &orig_mask); + pollset_work_and_unlock(exec_ctx, pollset, timeout_ms, &orig_mask, &error); grpc_exec_ctx_flush(exec_ctx); gpr_mu_lock(&pollset->mu); @@ -1412,15 +1470,20 @@ static grpc_error *pollset_work(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, } *worker_hdl = NULL; + gpr_tls_set(&g_current_thread_pollset, (intptr_t)0); gpr_tls_set(&g_current_thread_worker, (intptr_t)0); + GPR_TIMER_END("pollset_work", 0); + GRPC_LOG_IF_ERROR("pollset_work", GRPC_ERROR_REF(error)); return error; } static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, grpc_fd *fd) { + grpc_error *error = GRPC_ERROR_NONE; + gpr_mu_lock(&pollset->mu); gpr_mu_lock(&fd->pi_mu); @@ -1443,19 +1506,23 @@ static void pollset_add_fd(grpc_exec_ctx *exec_ctx, grpc_pollset *pollset, if (fd->polling_island == pollset->polling_island) { pi_new = fd->polling_island; if (pi_new == NULL) { - pi_new = polling_island_create(fd); + pi_new = polling_island_create(fd, &error); } } else if (fd->polling_island == NULL) { pi_new = polling_island_lock(pollset->polling_island); - polling_island_add_fds_locked(pi_new, &fd, 1, true); + polling_island_add_fds_locked(pi_new, &fd, 1, true, &error); gpr_mu_unlock(&pi_new->mu); } else if (pollset->polling_island == NULL) { pi_new = polling_island_lock(fd->polling_island); gpr_mu_unlock(&pi_new->mu); } else { - pi_new = polling_island_merge(fd->polling_island, pollset->polling_island); + pi_new = polling_island_merge(fd->polling_island, pollset->polling_island, + &error); } + /* At this point, pi_new is the polling island that both fd->polling_island + and pollset->polling_island must be pointing to */ + if (fd->polling_island != pi_new) { PI_ADD_REF(pi_new, "fd"); if (fd->polling_island != NULL) { @@ -1645,13 +1712,10 @@ bool grpc_are_polling_islands_equal(void *p, void *q) { polling_island *p1 = p; polling_island *p2 = q; + /* Note: polling_island_lock_pair() may change p1 and p2 to point to the + latest polling islands in their respective linked lists */ polling_island_lock_pair(&p1, &p2); - if (p1 == p2) { - gpr_mu_unlock(&p1->mu); - } else { - gpr_mu_unlock(&p1->mu); - gpr_mu_unlock(&p2->mu); - } + polling_island_unlock_pair(p1, p2); return p1 == p2; } From 74189cd94b49be086e9320bf9536ab4bacfa6d61 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 23 Jun 2016 15:39:06 -0700 Subject: [PATCH 197/280] Remove caching of results by run_tests SIGNIFICANTLY increases the performance of actually running tests... --- tools/run_tests/jobset.py | 68 ++++++++---------------------------- tools/run_tests/run_tests.py | 63 ++------------------------------- 2 files changed, 18 insertions(+), 113 deletions(-) diff --git a/tools/run_tests/jobset.py b/tools/run_tests/jobset.py index d3259e724df..40409c43948 100755 --- a/tools/run_tests/jobset.py +++ b/tools/run_tests/jobset.py @@ -29,7 +29,6 @@ """Run a group of subprocesses and then finish.""" -import hashlib import multiprocessing import os import platform @@ -149,7 +148,7 @@ def which(filename): class JobSpec(object): """Specifies what to run for a job.""" - def __init__(self, cmdline, shortname=None, environ=None, hash_targets=None, + def __init__(self, cmdline, shortname=None, environ=None, cwd=None, shell=False, timeout_seconds=5*60, flake_retries=0, timeout_retries=0, kill_handler=None, cpu_cost=1.0, verbose_success=False): @@ -157,19 +156,14 @@ class JobSpec(object): Arguments: cmdline: a list of arguments to pass as the command line environ: a dictionary of environment variables to set in the child process - hash_targets: which files to include in the hash representing the jobs version - (or empty, indicating the job should not be hashed) kill_handler: a handler that will be called whenever job.kill() is invoked cpu_cost: number of cores per second this job needs """ if environ is None: environ = {} - if hash_targets is None: - hash_targets = [] self.cmdline = cmdline self.environ = environ self.shortname = cmdline[0] if shortname is None else shortname - self.hash_targets = hash_targets or [] self.cwd = cwd self.shell = shell self.timeout_seconds = timeout_seconds @@ -180,7 +174,7 @@ class JobSpec(object): self.verbose_success = verbose_success def identity(self): - return '%r %r %r' % (self.cmdline, self.environ, self.hash_targets) + return '%r %r' % (self.cmdline, self.environ) def __hash__(self): return hash(self.identity()) @@ -205,9 +199,8 @@ class JobResult(object): class Job(object): """Manages one job.""" - def __init__(self, spec, bin_hash, newline_on_success, travis, add_env): + def __init__(self, spec, newline_on_success, travis, add_env): self._spec = spec - self._bin_hash = bin_hash self._newline_on_success = newline_on_success self._travis = travis self._add_env = add_env.copy() @@ -249,7 +242,7 @@ class Job(object): self._process = try_start() self._state = _RUNNING - def state(self, update_cache): + def state(self): """Poll current state of the job. Prints messages at completion.""" def stdout(self=self): self._tempfile.seek(0) @@ -293,8 +286,6 @@ class Job(object): stdout() if self._spec.verbose_success else None, do_newline=self._newline_on_success or self._travis) self.result.state = 'PASSED' - if self._bin_hash: - update_cache.finished(self._spec.identity(), self._bin_hash) elif (self._state == _RUNNING and self._spec.timeout_seconds is not None and time.time() - self._start > self._spec.timeout_seconds): @@ -329,7 +320,7 @@ class Jobset(object): """Manages one run of jobs.""" def __init__(self, check_cancelled, maxjobs, newline_on_success, travis, - stop_on_failure, add_env, cache): + stop_on_failure, add_env): self._running = set() self._check_cancelled = check_cancelled self._cancelled = False @@ -338,9 +329,7 @@ class Jobset(object): self._maxjobs = maxjobs self._newline_on_success = newline_on_success self._travis = travis - self._cache = cache self._stop_on_failure = stop_on_failure - self._hashes = {} self._add_env = add_env self.resultset = {} self._remaining = None @@ -367,37 +356,21 @@ class Jobset(object): if current_cpu_cost + spec.cpu_cost <= self._maxjobs: break self.reap() if self.cancelled(): return False - if spec.hash_targets: - if spec.identity() in self._hashes: - bin_hash = self._hashes[spec.identity()] - else: - bin_hash = hashlib.sha1() - for fn in spec.hash_targets: - with open(which(fn)) as f: - bin_hash.update(f.read()) - bin_hash = bin_hash.hexdigest() - self._hashes[spec.identity()] = bin_hash - should_run = self._cache.should_run(spec.identity(), bin_hash) - else: - bin_hash = None - should_run = True - if should_run: - job = Job(spec, - bin_hash, - self._newline_on_success, - self._travis, - self._add_env) - self._running.add(job) - if not self.resultset.has_key(job.GetSpec().shortname): - self.resultset[job.GetSpec().shortname] = [] - return True + job = Job(spec, + self._newline_on_success, + self._travis, + self._add_env) + self._running.add(job) + if not self.resultset.has_key(job.GetSpec().shortname): + self.resultset[job.GetSpec().shortname] = [] + return True def reap(self): """Collect the dead jobs.""" while self._running: dead = set() for job in self._running: - st = job.state(self._cache) + st = job.state() if st == _RUNNING: continue if st == _FAILURE or st == _KILLED: self._failures += 1 @@ -450,15 +423,6 @@ def _never_cancelled(): return False -# cache class that caches nothing -class NoCache(object): - def should_run(self, cmdline, bin_hash): - return True - - def finished(self, cmdline, bin_hash): - pass - - def tag_remaining(xs): staging = [] for x in xs: @@ -477,12 +441,10 @@ def run(cmdlines, travis=False, infinite_runs=False, stop_on_failure=False, - cache=None, add_env={}): js = Jobset(check_cancelled, maxjobs if maxjobs is not None else _DEFAULT_MAX_JOBS, - newline_on_success, travis, stop_on_failure, add_env, - cache if cache is not None else NoCache()) + newline_on_success, travis, stop_on_failure, add_env) for cmdline, remaining in tag_remaining(cmdlines): if not js.start(cmdline): break diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index e4779e3a4e8..c571a81eb28 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -33,7 +33,6 @@ import argparse import ast import glob -import hashlib import itertools import json import multiprocessing @@ -78,24 +77,18 @@ class Config(object): if environ is None: environ = {} self.build_config = config - self.allow_hashing = (config != 'gcov') self.environ = environ self.environ['CONFIG'] = config self.tool_prefix = tool_prefix self.timeout_multiplier = timeout_multiplier - def job_spec(self, cmdline, hash_targets, timeout_seconds=5*60, + def job_spec(self, cmdline, timeout_seconds=5*60, shortname=None, environ={}, cpu_cost=1.0, flaky=False): """Construct a jobset.JobSpec for a test under this config Args: cmdline: a list of strings specifying the command line the test would like to run - hash_targets: either None (don't do caching of test results), or - a list of strings specifying files to include in a - binary hash to check if a test has changed - -- if used, all artifacts needed to run the test must - be listed """ actual_environ = self.environ.copy() for k, v in environ.iteritems(): @@ -105,8 +98,6 @@ class Config(object): environ=actual_environ, cpu_cost=cpu_cost, timeout_seconds=(self.timeout_multiplier * timeout_seconds if timeout_seconds else None), - hash_targets=hash_targets - if self.allow_hashing else None, flake_retries=5 if flaky or args.allow_flakes else 0, timeout_retries=3 if args.allow_flakes else 0) @@ -425,7 +416,7 @@ class PythonLanguage(object): return [] def build_steps(self): - return [['tools/run_tests/build_python.sh', tox_env] + return [['tools/run_tests/build_python.sh', tox_env] for tox_env in self._tox_envs] def post_tests_steps(self): @@ -1058,46 +1049,6 @@ runs_per_test = args.runs_per_test forever = args.forever -class TestCache(object): - """Cache for running tests.""" - - def __init__(self, use_cache_results): - self._last_successful_run = {} - self._use_cache_results = use_cache_results - self._last_save = time.time() - - def should_run(self, cmdline, bin_hash): - if cmdline not in self._last_successful_run: - return True - if self._last_successful_run[cmdline] != bin_hash: - return True - if not self._use_cache_results: - return True - return False - - def finished(self, cmdline, bin_hash): - self._last_successful_run[cmdline] = bin_hash - if time.time() - self._last_save > 1: - self.save() - - def dump(self): - return [{'cmdline': k, 'hash': v} - for k, v in self._last_successful_run.iteritems()] - - def parse(self, exdump): - self._last_successful_run = dict((o['cmdline'], o['hash']) for o in exdump) - - def save(self): - with open('.run_tests_cache', 'w') as f: - f.write(json.dumps(self.dump())) - self._last_save = time.time() - - def maybe_load(self): - if os.path.exists('.run_tests_cache'): - with open('.run_tests_cache') as f: - self.parse(json.loads(f.read())) - - def _start_port_server(port_server_port): # check if a compatible port server is running # if incompatible (version mismatch) ==> start a new one @@ -1217,7 +1168,7 @@ class BuildAndRunError(object): # returns a list of things that failed (or an empty list on success) def _build_and_run( - check_cancelled, newline_on_success, cache, xml_report=None, build_only=False): + check_cancelled, newline_on_success, xml_report=None, build_only=False): """Do one pass of building & running tests.""" # build latest sequentially num_failures, resultset = jobset.run( @@ -1266,7 +1217,6 @@ def _build_and_run( all_runs, check_cancelled, newline_on_success=newline_on_success, travis=args.travis, infinite_runs=infinite_runs, maxjobs=args.jobs, stop_on_failure=args.stop_on_failure, - cache=cache if not xml_report else None, add_env={'GRPC_TEST_PORT_SERVER': 'localhost:%d' % port_server_port}) if resultset: for k, v in sorted(resultset.items()): @@ -1295,14 +1245,9 @@ def _build_and_run( if num_test_failures: out.append(BuildAndRunError.TEST) - if cache: cache.save() - return out -test_cache = TestCache(runs_per_test == 1) -test_cache.maybe_load() - if forever: success = True while True: @@ -1312,7 +1257,6 @@ if forever: previous_success = success errors = _build_and_run(check_cancelled=have_files_changed, newline_on_success=False, - cache=test_cache, build_only=args.build_only) == 0 if not previous_success and not errors: jobset.message('SUCCESS', @@ -1324,7 +1268,6 @@ if forever: else: errors = _build_and_run(check_cancelled=lambda: False, newline_on_success=args.newline_on_success, - cache=test_cache, xml_report=args.xml_report, build_only=args.build_only) if not errors: From 98d31d1a40fe0d320f2488794d97a0bad5a8060d Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 23 Jun 2016 15:43:22 -0700 Subject: [PATCH 198/280] Fix special value lookup --- src/core/lib/iomgr/error.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/core/lib/iomgr/error.c b/src/core/lib/iomgr/error.c index da0d6972360..9f5ba76fd6c 100644 --- a/src/core/lib/iomgr/error.c +++ b/src/core/lib/iomgr/error.c @@ -276,7 +276,8 @@ bool grpc_error_get_int(grpc_error *err, grpc_error_ints which, intptr_t *p) { void *pp; if (is_special(err)) { if (err == GRPC_ERROR_CANCELLED && which == GRPC_ERROR_INT_GRPC_STATUS) { - return GRPC_STATUS_CANCELLED; + *p = GRPC_STATUS_CANCELLED; + return true; } return false; } From 6a29545c8c5ef61346af3a9b0bdd2ddb39ba15c8 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Thu, 23 Jun 2016 15:53:10 -0700 Subject: [PATCH 199/280] Change the type of 'ref_count' in polling_island from gpr_atm to gpr_refcount --- src/core/lib/iomgr/ev_epoll_linux.c | 69 +++++++++++++---------------- 1 file changed, 31 insertions(+), 38 deletions(-) diff --git a/src/core/lib/iomgr/ev_epoll_linux.c b/src/core/lib/iomgr/ev_epoll_linux.c index a77044edc50..4dca551e1e1 100644 --- a/src/core/lib/iomgr/ev_epoll_linux.c +++ b/src/core/lib/iomgr/ev_epoll_linux.c @@ -121,6 +121,7 @@ struct grpc_fd { }; /* Reference counting for fds */ +// #define GRPC_FD_REF_COUNT_DEBUG #ifdef GRPC_FD_REF_COUNT_DEBUG static void fd_ref(grpc_fd *fd, const char *reason, const char *file, int line); static void fd_unref(grpc_fd *fd, const char *reason, const char *file, @@ -147,13 +148,13 @@ static void fd_global_shutdown(void); // #define GRPC_PI_REF_COUNT_DEBUG #ifdef GRPC_PI_REF_COUNT_DEBUG -#define PI_ADD_REF(p, r) pi_add_ref_dbg((p), 1, (r), __FILE__, __LINE__) -#define PI_UNREF(p, r) pi_unref_dbg((p), 1, (r), __FILE__, __LINE__) +#define PI_ADD_REF(p, r) pi_add_ref_dbg((p), (r), __FILE__, __LINE__) +#define PI_UNREF(p, r) pi_unref_dbg((p), (r), __FILE__, __LINE__) #else /* defined(GRPC_PI_REF_COUNT_DEBUG) */ -#define PI_ADD_REF(p, r) pi_add_ref((p), 1) -#define PI_UNREF(p, r) pi_unref((p), 1) +#define PI_ADD_REF(p, r) pi_add_ref((p)) +#define PI_UNREF(p, r) pi_unref((p)) #endif /* !defined(GPRC_PI_REF_COUNT_DEBUG) */ @@ -164,7 +165,7 @@ typedef struct polling_island { Once the ref count becomes zero, this structure is destroyed which means we should ensure that there is never a scenario where a PI_ADD_REF() is racing with a PI_UNREF() that just made the ref_count zero. */ - gpr_atm ref_count; + gpr_refcount ref_count; /* Pointer to the polling_island this merged into. * merged_to value is only set once in polling_island's lifetime (and that too @@ -281,50 +282,42 @@ gpr_atm g_epoll_sync; #endif /* defined(GRPC_TSAN) */ #ifdef GRPC_PI_REF_COUNT_DEBUG -long pi_add_ref(polling_island *pi, int ref_cnt); -long pi_unref(polling_island *pi, int ref_cnt); +void pi_add_ref(polling_island *pi); +void pi_unref(polling_island *pi); -void pi_add_ref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, - int line) { - long old_cnt = pi_add_ref(pi, ref_cnt); - gpr_log(GPR_DEBUG, "Add ref pi: %p, old:%ld -> new:%ld (%s) - (%s, %d)", - (void *)pi, old_cnt, (old_cnt + ref_cnt), reason, file, line); +void pi_add_ref_dbg(polling_island *pi, char *reason, char *file, int line) { + long old_cnt = gpr_atm_acq_load(&(pi->ref_count.count)); + pi_add_ref(pi); + gpr_log(GPR_DEBUG, "Add ref pi: %p, old: %ld -> new:%ld (%s) - (%s, %d)", + (void *)pi, old_cnt, old_cnt + 1, reason, file, line); } -void pi_unref_dbg(polling_island *pi, int ref_cnt, char *reason, char *file, - int line) { - long old_cnt = pi_unref(pi, ref_cnt); +void pi_unref_dbg(polling_island *pi, char *reason, char *file, int line) { + long old_cnt = gpr_atm_acq_load(&(pi->ref_count.count)); + pi_unref(pi); gpr_log(GPR_DEBUG, "Unref pi: %p, old:%ld -> new:%ld (%s) - (%s, %d)", - (void *)pi, old_cnt, (old_cnt - ref_cnt), reason, file, line); + (void *)pi, old_cnt, (old_cnt - 1), reason, file, line); } #endif -long pi_add_ref(polling_island *pi, int ref_cnt) { - return gpr_atm_full_fetch_add(&pi->ref_count, ref_cnt); -} - -long pi_unref(polling_island *pi, int ref_cnt) { - long old_cnt = gpr_atm_full_fetch_add(&pi->ref_count, -ref_cnt); +void pi_add_ref(polling_island *pi) { gpr_ref(&pi->ref_count); } - /* If ref count went to zero, delete the polling island. Note that this need - not be done under a lock. Once the ref count goes to zero, we are - guaranteed that no one else holds a reference to the polling island (and - that there is no racing pi_add_ref() call either. +void pi_unref(polling_island *pi) { + /* If ref count went to zero, delete the polling island. + Note that this deletion not be done under a lock. Once the ref count goes + to zero, we are guaranteed that no one else holds a reference to the + polling island (and that there is no racing pi_add_ref() call either). Also, if we are deleting the polling island and the merged_to field is non-empty, we should remove a ref to the merged_to polling island */ - if (old_cnt == ref_cnt) { + if (gpr_unref(&pi->ref_count)) { polling_island *next = (polling_island *)gpr_atm_acq_load(&pi->merged_to); polling_island_delete(pi); if (next != NULL) { PI_UNREF(next, "pi_delete"); /* Recursive call */ } - } else { - GPR_ASSERT(old_cnt > ref_cnt); } - - return old_cnt; } /* The caller is expected to hold pi->mu lock before calling this function */ @@ -482,7 +475,7 @@ static polling_island *polling_island_create(grpc_fd *initial_fd, pi->fds = NULL; } - gpr_atm_rel_store(&pi->ref_count, (gpr_atm)0); + gpr_ref_init(&pi->ref_count, 0); gpr_atm_rel_store(&pi->merged_to, (gpr_atm)NULL); pi->epoll_fd = epoll_create1(EPOLL_CLOEXEC); @@ -762,8 +755,8 @@ static gpr_mu fd_freelist_mu; #define UNREF_BY(fd, n, reason) unref_by(fd, n, reason, __FILE__, __LINE__) static void ref_by(grpc_fd *fd, int n, const char *reason, const char *file, int line) { - gpr_log(GPR_DEBUG, "FD %d %p ref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, - gpr_atm_no_barrier_load(&fd->refst), + gpr_log(GPR_DEBUG, "FD %d %p ref %d %ld -> %ld [%s; %s:%d]", fd->fd, + (void *)fd, n, gpr_atm_no_barrier_load(&fd->refst), gpr_atm_no_barrier_load(&fd->refst) + n, reason, file, line); #else #define REF_BY(fd, n, reason) ref_by(fd, n) @@ -777,8 +770,8 @@ static void ref_by(grpc_fd *fd, int n) { static void unref_by(grpc_fd *fd, int n, const char *reason, const char *file, int line) { gpr_atm old; - gpr_log(GPR_DEBUG, "FD %d %p unref %d %d -> %d [%s; %s:%d]", fd->fd, fd, n, - gpr_atm_no_barrier_load(&fd->refst), + gpr_log(GPR_DEBUG, "FD %d %p unref %d %ld -> %ld [%s; %s:%d]", fd->fd, + (void *)fd, n, gpr_atm_no_barrier_load(&fd->refst), gpr_atm_no_barrier_load(&fd->refst) - n, reason, file, line); #else static void unref_by(grpc_fd *fd, int n) { @@ -865,10 +858,10 @@ static grpc_fd *fd_create(int fd, const char *name) { char *fd_name; gpr_asprintf(&fd_name, "%s fd=%d", name, fd); grpc_iomgr_register_object(&new_fd->iomgr_object, fd_name); - gpr_free(fd_name); #ifdef GRPC_FD_REF_COUNT_DEBUG - gpr_log(GPR_DEBUG, "FD %d %p create %s", fd, r, fd_name); + gpr_log(GPR_DEBUG, "FD %d %p create %s", fd, (void *)new_fd, fd_name); #endif + gpr_free(fd_name); return new_fd; } From eb5e43762be40f78775dfd3728e105daf1169d90 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 23 Jun 2016 16:16:10 -0700 Subject: [PATCH 200/280] Fix ruby tests --- tools/run_tests/run_tests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index c571a81eb28..61cef0a9d57 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -451,7 +451,7 @@ class RubyLanguage(object): _check_compiler(self.args.compiler, ['default']) def test_specs(self): - return [self.config.job_spec(['tools/run_tests/run_ruby.sh'], None, + return [self.config.job_spec(['tools/run_tests/run_ruby.sh'], timeout_seconds=10*60, environ=_FORCE_ENVIRON_FOR_WRAPPERS)] From 2e1a1fe56f4276d670362c5aae1b493df0834c2e Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Thu, 23 Jun 2016 16:18:31 -0700 Subject: [PATCH 201/280] Fixes --- tools/run_tests/dockerjob.py | 4 ++-- tools/run_tests/run_performance_tests.py | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tools/run_tests/dockerjob.py b/tools/run_tests/dockerjob.py index 326c4faed95..e4ca3b7faaf 100755 --- a/tools/run_tests/dockerjob.py +++ b/tools/run_tests/dockerjob.py @@ -104,7 +104,7 @@ class DockerJob: def __init__(self, spec): self._spec = spec - self._job = jobset.Job(spec, bin_hash=None, newline_on_success=True, travis=True, add_env={}) + self._job = jobset.Job(spec, newline_on_success=True, travis=True, add_env={}) self._container_name = spec.container_name def mapped_port(self, port): @@ -118,4 +118,4 @@ class DockerJob: def is_running(self): """Polls a job and returns True if given job is still running.""" - return self._job.state(jobset.NoCache()) == jobset._RUNNING + return self._job.state() == jobset._RUNNING diff --git a/tools/run_tests/run_performance_tests.py b/tools/run_tests/run_performance_tests.py index f037d0d17d9..14901caf07f 100755 --- a/tools/run_tests/run_performance_tests.py +++ b/tools/run_tests/run_performance_tests.py @@ -61,11 +61,11 @@ class QpsWorkerJob: self._spec = spec self.language = language self.host_and_port = host_and_port - self._job = jobset.Job(spec, bin_hash=None, newline_on_success=True, travis=True, add_env={}) + self._job = jobset.Job(spec, newline_on_success=True, travis=True, add_env={}) def is_running(self): """Polls a job and returns True if given job is still running.""" - return self._job.state(jobset.NoCache()) == jobset._RUNNING + return self._job.state() == jobset._RUNNING def kill(self): return self._job.kill() From d6f8f0b7cd9c1e92219027919eb8a562763e029b Mon Sep 17 00:00:00 2001 From: Yuchen Zeng Date: Thu, 23 Jun 2016 17:33:01 -0700 Subject: [PATCH 202/280] Add TODO --- src/objective-c/examples/SwiftSample/ViewController.swift | 1 + 1 file changed, 1 insertion(+) diff --git a/src/objective-c/examples/SwiftSample/ViewController.swift b/src/objective-c/examples/SwiftSample/ViewController.swift index 80d7a47917f..2a95d2de516 100644 --- a/src/objective-c/examples/SwiftSample/ViewController.swift +++ b/src/objective-c/examples/SwiftSample/ViewController.swift @@ -71,6 +71,7 @@ class ViewController: UIViewController { NSLog("2. Response trailers: \(RPC.responseTrailers)") } + // TODO(jcanizales): Revert to using subscript syntax once XCode 8 is released. RPC.requestHeaders.setObject("My value", forKey: "My-Header") RPC.start() From 33b767a7701c373767bf29a49c93c9e515eb17f8 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Fri, 24 Jun 2016 09:13:03 -0700 Subject: [PATCH 203/280] fix build node package --- tools/run_tests/build_package_node.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/run_tests/build_package_node.sh b/tools/run_tests/build_package_node.sh index 6f7211b53fe..ff4cfdb8bf9 100755 --- a/tools/run_tests/build_package_node.sh +++ b/tools/run_tests/build_package_node.sh @@ -86,6 +86,7 @@ for arch in {x86,x64}; do cp $input_dir/protoc* bin/ cp $input_dir/grpc_node_plugin* bin/ mkdir -p bin/google/protobuf + mkdir -p bin/google/protobuf/compiler # needed for plugin.proto for proto in "${well_known_protos[@]}"; do cp $base/third_party/protobuf/src/google/protobuf/$proto.proto bin/google/protobuf/$proto.proto done From e07b83ba9fde162fe0e61f97c657492f4d948e95 Mon Sep 17 00:00:00 2001 From: Sree Kuchibhotla Date: Fri, 24 Jun 2016 09:27:00 -0700 Subject: [PATCH 204/280] generate_projects.sh --- src/node/tools/package.json | 1 + 1 file changed, 1 insertion(+) diff --git a/src/node/tools/package.json b/src/node/tools/package.json index d4849c2e388..c34b259e2e0 100644 --- a/src/node/tools/package.json +++ b/src/node/tools/package.json @@ -34,6 +34,7 @@ "index.js", "bin/protoc.js", "bin/protoc_plugin.js", + "bin/google/protobuf", "LICENSE" ], "main": "index.js" From 886c512832464a5aa5e61b51f1fcd1da356ca847 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Fri, 24 Jun 2016 09:48:58 -0700 Subject: [PATCH 205/280] fix C# nuget build --- src/csharp/Grpc.Core/Grpc.Core.nuspec | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/src/csharp/Grpc.Core/Grpc.Core.nuspec b/src/csharp/Grpc.Core/Grpc.Core.nuspec index fa2c1fbff22..47593f787b1 100644 --- a/src/csharp/Grpc.Core/Grpc.Core.nuspec +++ b/src/csharp/Grpc.Core/Grpc.Core.nuspec @@ -24,11 +24,12 @@ - - - - - - + + + + + + + From d29a3bf004e23eb481b33b9a69fd10ae9221fc98 Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Fri, 24 Jun 2016 10:26:14 -0700 Subject: [PATCH 206/280] Update master branch to 0.16.0-dev --- Makefile | 2 +- build.yaml | 2 +- package.json | 2 +- package.xml | 4 ++-- src/core/lib/surface/version.c | 2 +- src/csharp/Grpc.Auth/project.json | 4 ++-- src/csharp/Grpc.Core/VersionInfo.cs | 4 ++-- src/csharp/Grpc.Core/project.json | 2 +- src/csharp/Grpc.HealthCheck/project.json | 4 ++-- src/csharp/build_packages.bat | 2 +- src/node/tools/package.json | 2 +- src/python/grpcio/grpc_version.py | 2 +- src/ruby/lib/grpc/version.rb | 2 +- src/ruby/tools/version.rb | 2 +- tools/distrib/python/grpcio_tools/grpc_version.py | 2 +- tools/doxygen/Doxyfile.c++ | 2 +- tools/doxygen/Doxyfile.c++.internal | 2 +- tools/doxygen/Doxyfile.core | 2 +- tools/doxygen/Doxyfile.core.internal | 2 +- 19 files changed, 23 insertions(+), 23 deletions(-) diff --git a/Makefile b/Makefile index 9be3e5784cc..2f49e7712fd 100644 --- a/Makefile +++ b/Makefile @@ -414,7 +414,7 @@ E = @echo Q = @ endif -VERSION = 0.15.0-dev +VERSION = 0.16.0-dev CPPFLAGS_NO_ARCH += $(addprefix -I, $(INCLUDES)) $(addprefix -D, $(DEFINES)) CPPFLAGS += $(CPPFLAGS_NO_ARCH) $(ARCH_FLAGS) diff --git a/build.yaml b/build.yaml index 1f06e20cf9b..7d570e4dab9 100644 --- a/build.yaml +++ b/build.yaml @@ -7,7 +7,7 @@ settings: '#3': Use "-preN" suffixes to identify pre-release versions '#4': Per-language overrides are possible with (eg) ruby_version tag here '#5': See the expand_version.py for all the quirks here - version: 0.15.0-dev + version: 0.16.0-dev filegroups: - name: census public_headers: diff --git a/package.json b/package.json index 5bdaa761e28..68a31d794c5 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "grpc", - "version": "0.15.0-dev", + "version": "0.16.0-dev", "author": "Google Inc.", "description": "gRPC Library for Node", "homepage": "http://www.grpc.io/", diff --git a/package.xml b/package.xml index 67e9bb2c282..c1ecae8b806 100644 --- a/package.xml +++ b/package.xml @@ -13,8 +13,8 @@ 2016-05-19 - 0.15.0 - 0.15.0 + 0.16.0 + 0.16.0 beta diff --git a/src/core/lib/surface/version.c b/src/core/lib/surface/version.c index aca76d2bb79..53f3c438541 100644 --- a/src/core/lib/surface/version.c +++ b/src/core/lib/surface/version.c @@ -36,4 +36,4 @@ #include -const char *grpc_version_string(void) { return "0.15.0-dev"; } +const char *grpc_version_string(void) { return "0.16.0-dev"; } diff --git a/src/csharp/Grpc.Auth/project.json b/src/csharp/Grpc.Auth/project.json index 1677565824b..4c5c960204e 100644 --- a/src/csharp/Grpc.Auth/project.json +++ b/src/csharp/Grpc.Auth/project.json @@ -1,5 +1,5 @@ { - "version": "0.15.0-dev", + "version": "0.16.0-dev", "title": "gRPC C# Auth", "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", @@ -13,7 +13,7 @@ "tags": [ "gRPC RPC Protocol HTTP/2 Auth OAuth2" ], }, "dependencies": { - "Grpc.Core": "0.15.0-dev", + "Grpc.Core": "0.16.0-dev", "Google.Apis.Auth": "1.11.1" }, "frameworks": { diff --git a/src/csharp/Grpc.Core/VersionInfo.cs b/src/csharp/Grpc.Core/VersionInfo.cs index e1609341d9a..cb20967680e 100644 --- a/src/csharp/Grpc.Core/VersionInfo.cs +++ b/src/csharp/Grpc.Core/VersionInfo.cs @@ -48,11 +48,11 @@ namespace Grpc.Core /// /// Current AssemblyFileVersion of gRPC C# assemblies /// - public const string CurrentAssemblyFileVersion = "0.15.0.0"; + public const string CurrentAssemblyFileVersion = "0.16.0.0"; /// /// Current version of gRPC C# /// - public const string CurrentVersion = "0.15.0-dev"; + public const string CurrentVersion = "0.16.0-dev"; } } diff --git a/src/csharp/Grpc.Core/project.json b/src/csharp/Grpc.Core/project.json index 7253107e04a..4729a9346cb 100644 --- a/src/csharp/Grpc.Core/project.json +++ b/src/csharp/Grpc.Core/project.json @@ -1,5 +1,5 @@ { - "version": "0.15.0-dev", + "version": "0.16.0-dev", "title": "gRPC C# Core", "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", diff --git a/src/csharp/Grpc.HealthCheck/project.json b/src/csharp/Grpc.HealthCheck/project.json index eb57608957a..c4895c2ad3c 100644 --- a/src/csharp/Grpc.HealthCheck/project.json +++ b/src/csharp/Grpc.HealthCheck/project.json @@ -1,5 +1,5 @@ { - "version": "0.15.0-dev", + "version": "0.16.0-dev", "title": "gRPC C# Healthchecking", "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", @@ -13,7 +13,7 @@ "tags": [ "gRPC health check" ] }, "dependencies": { - "Grpc.Core": "0.15.0-dev", + "Grpc.Core": "0.16.0-dev", "Google.Protobuf": "3.0.0-beta3" }, "frameworks": { diff --git a/src/csharp/build_packages.bat b/src/csharp/build_packages.bat index 63f8c30bc7e..272b30f385d 100644 --- a/src/csharp/build_packages.bat +++ b/src/csharp/build_packages.bat @@ -30,7 +30,7 @@ @rem Builds gRPC NuGet packages @rem Current package versions -set VERSION=0.15.0-dev +set VERSION=0.16.0-dev set PROTOBUF_VERSION=3.0.0-beta3 @rem Packages that depend on prerelease packages (like Google.Protobuf) need to have prerelease suffix as well. diff --git a/src/node/tools/package.json b/src/node/tools/package.json index c34b259e2e0..7c256d7ba0d 100644 --- a/src/node/tools/package.json +++ b/src/node/tools/package.json @@ -1,6 +1,6 @@ { "name": "grpc-tools", - "version": "0.15.0-dev", + "version": "0.16.0-dev", "author": "Google Inc.", "description": "Tools for developing with gRPC on Node.js", "homepage": "http://www.grpc.io/", diff --git a/src/python/grpcio/grpc_version.py b/src/python/grpcio/grpc_version.py index 0c13104d9da..0f4db9d972e 100644 --- a/src/python/grpcio/grpc_version.py +++ b/src/python/grpcio/grpc_version.py @@ -29,4 +29,4 @@ # AUTO-GENERATED FROM `$REPO_ROOT/templates/src/python/grpcio/grpc_version.py.template`!!! -VERSION='0.15.0.dev0' +VERSION='0.16.0.dev0' diff --git a/src/ruby/lib/grpc/version.rb b/src/ruby/lib/grpc/version.rb index 01c8c5ac8f1..5e6aaef2eb5 100644 --- a/src/ruby/lib/grpc/version.rb +++ b/src/ruby/lib/grpc/version.rb @@ -29,5 +29,5 @@ # GRPC contains the General RPC module. module GRPC - VERSION = '0.15.0.dev' + VERSION = '0.16.0.dev' end diff --git a/src/ruby/tools/version.rb b/src/ruby/tools/version.rb index dca7fd7e72c..68c1bf369d2 100644 --- a/src/ruby/tools/version.rb +++ b/src/ruby/tools/version.rb @@ -29,6 +29,6 @@ module GRPC module Tools - VERSION = '0.15.0.dev' + VERSION = '0.16.0.dev' end end diff --git a/tools/distrib/python/grpcio_tools/grpc_version.py b/tools/distrib/python/grpcio_tools/grpc_version.py index 1267d0e45dc..4b1e7fcd584 100644 --- a/tools/distrib/python/grpcio_tools/grpc_version.py +++ b/tools/distrib/python/grpcio_tools/grpc_version.py @@ -29,4 +29,4 @@ # AUTO-GENERATED FROM `$REPO_ROOT/templates/tools/distrib/python/grpcio_tools/grpc_version.py.template`!!! -VERSION='0.15.0.dev0' +VERSION='0.16.0.dev0' diff --git a/tools/doxygen/Doxyfile.c++ b/tools/doxygen/Doxyfile.c++ index 7f9d2df6f6c..de7acd7777e 100644 --- a/tools/doxygen/Doxyfile.c++ +++ b/tools/doxygen/Doxyfile.c++ @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC C++" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.16.0-dev # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a diff --git a/tools/doxygen/Doxyfile.c++.internal b/tools/doxygen/Doxyfile.c++.internal index dcf1a4c8c40..76bb3b6c59e 100644 --- a/tools/doxygen/Doxyfile.c++.internal +++ b/tools/doxygen/Doxyfile.c++.internal @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC C++" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.16.0-dev # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a diff --git a/tools/doxygen/Doxyfile.core b/tools/doxygen/Doxyfile.core index 72102b2fc50..53ae4e4cf4b 100644 --- a/tools/doxygen/Doxyfile.core +++ b/tools/doxygen/Doxyfile.core @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC Core" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.16.0-dev # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal index e1555930e91..1c75c941c27 100644 --- a/tools/doxygen/Doxyfile.core.internal +++ b/tools/doxygen/Doxyfile.core.internal @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC Core" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.16.0-dev # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a From f4c1bff0d8caedf716b460c4ee6114ac126e8872 Mon Sep 17 00:00:00 2001 From: Ken Payson Date: Sun, 12 Jun 2016 15:23:36 -0700 Subject: [PATCH 207/280] Moved grpc_shutdown to end of Py_Finalize() We currently rely on the __del__ method of a module scope object to call grpc_shutdown(). __del__ methods are not guaranteed to be called, and furthermore there are no guarantees about ordering, leading to shutdown race conditions. This moves grpc_shutdown to Py_Finalize(), which gets called after the Python context is completely cleaned up. --- .../grpcio/grpc/_cython/_cygrpc/grpc.pxi | 1 + src/python/grpcio/grpc/_cython/cygrpc.pyx | 32 +++++++------------ src/python/grpcio/grpc/_cython/loader.c | 7 ++++ src/python/grpcio/grpc/_cython/loader.h | 5 +++ 4 files changed, 24 insertions(+), 21 deletions(-) diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi index 05b8886df73..35e394d02f8 100644 --- a/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi +++ b/src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi @@ -37,6 +37,7 @@ cdef extern from "grpc/_cython/loader.h": ctypedef long int64_t int pygrpc_load_core(char*) + int pygrpc_initialize_core() void *gpr_malloc(size_t size) nogil void gpr_free(void *ptr) nogil diff --git a/src/python/grpcio/grpc/_cython/cygrpc.pyx b/src/python/grpcio/grpc/_cython/cygrpc.pyx index cf146f5a048..c92a8d19a77 100644 --- a/src/python/grpcio/grpc/_cython/cygrpc.pyx +++ b/src/python/grpcio/grpc/_cython/cygrpc.pyx @@ -45,30 +45,20 @@ include "grpc/_cython/_cygrpc/security.pyx.pxi" include "grpc/_cython/_cygrpc/server.pyx.pxi" # -# Global state +# initialize gRPC # -cdef class _ModuleState: - cdef bint is_loaded +def _initialize(): + if 'win32' in sys.platform: + filename = pkg_resources.resource_filename( + 'grpc._cython', '_windows/grpc_c.64.python') + if not pygrpc_load_core(filename): + raise ImportError('failed to load core gRPC library') + if not pygrpc_initialize_core(): + raise ImportError('failed to initialize core gRPC library') - def __cinit__(self): - if 'win32' in sys.platform: - filename = pkg_resources.resource_filename( - 'grpc._cython', '_windows/grpc_c.64.python') - if not pygrpc_load_core(filename): - raise ImportError('failed to load core gRPC library') - with nogil: - grpc_init() - self.is_loaded = True - with nogil: - grpc_set_ssl_roots_override_callback( + grpc_set_ssl_roots_override_callback( ssl_roots_override_callback) - def __dealloc__(self): - if self.is_loaded: - with nogil: - grpc_shutdown() - -_module_state = _ModuleState() - +_initialize() diff --git a/src/python/grpcio/grpc/_cython/loader.c b/src/python/grpcio/grpc/_cython/loader.c index b909ad594ed..86b70dbb02d 100644 --- a/src/python/grpcio/grpc/_cython/loader.c +++ b/src/python/grpcio/grpc/_cython/loader.c @@ -31,6 +31,7 @@ * */ +#include #include "loader.h" #ifdef __cplusplus @@ -62,6 +63,12 @@ int pygrpc_load_core(char *path) { return 1; } #endif /* !GPR_WINDOWS */ +// Cython doesn't have Py_AtExit bindings, so we call the C_API directly +int pygrpc_initialize_core(void) { + grpc_init(); + return Py_AtExit(grpc_shutdown) < 0 ? 0 : 1; +} + #ifdef __cplusplus } #endif /* __cpluslus */ diff --git a/src/python/grpcio/grpc/_cython/loader.h b/src/python/grpcio/grpc/_cython/loader.h index 3b8796d39f7..eb4b1a1b018 100644 --- a/src/python/grpcio/grpc/_cython/loader.h +++ b/src/python/grpcio/grpc/_cython/loader.h @@ -46,6 +46,11 @@ extern "C" { /* Attempts to load the core if necessary, and return non-zero upon succes. */ int pygrpc_load_core(char *path); +/* Initializes grpc and registers grpc_shutdown() to be called right before + * interpreter exit. Returns non-zero upon success. + */ +int pygrpc_initialize_core(void); + #ifdef __cplusplus } #endif /* __cpluslus */ From 2dfcf14705f24bdf6d95d1b16265beb52fe33b02 Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Fri, 24 Jun 2016 11:06:28 -0700 Subject: [PATCH 208/280] Fix Node Windows build error --- src/node/ext/node_grpc.cc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/node/ext/node_grpc.cc b/src/node/ext/node_grpc.cc index ce988f9dfa9..745b5023d59 100644 --- a/src/node/ext/node_grpc.cc +++ b/src/node/ext/node_grpc.cc @@ -265,8 +265,8 @@ void InitLogConstants(Local exports) { Nan::Set(log_verbosity, Nan::New("DEBUG").ToLocalChecked(), DEBUG); Local INFO(Nan::New(GPR_LOG_SEVERITY_INFO)); Nan::Set(log_verbosity, Nan::New("INFO").ToLocalChecked(), INFO); - Local ERROR(Nan::New(GPR_LOG_SEVERITY_ERROR)); - Nan::Set(log_verbosity, Nan::New("ERROR").ToLocalChecked(), ERROR); + Local LOG_ERROR(Nan::New(GPR_LOG_SEVERITY_ERROR)); + Nan::Set(log_verbosity, Nan::New("ERROR").ToLocalChecked(), LOG_ERROR); } NAN_METHOD(MetadataKeyIsLegal) { From f519df8147e80b6ef21a576e8e08742e4f957e87 Mon Sep 17 00:00:00 2001 From: murgatroid99 Date: Fri, 24 Jun 2016 11:12:03 -0700 Subject: [PATCH 209/280] Update version to 0.15.0 --- Makefile | 2 +- build.yaml | 2 +- package.json | 2 +- src/core/lib/surface/version.c | 2 +- src/csharp/Grpc.Auth/project.json | 4 ++-- src/csharp/Grpc.Core/VersionInfo.cs | 2 +- src/csharp/Grpc.Core/project.json | 2 +- src/csharp/Grpc.HealthCheck/project.json | 4 ++-- src/csharp/build_packages.bat | 2 +- src/node/tools/package.json | 2 +- src/python/grpcio/grpc_version.py | 2 +- src/ruby/lib/grpc/version.rb | 2 +- src/ruby/tools/version.rb | 2 +- tools/distrib/python/grpcio_tools/grpc_version.py | 2 +- tools/doxygen/Doxyfile.c++ | 2 +- tools/doxygen/Doxyfile.c++.internal | 2 +- tools/doxygen/Doxyfile.core | 2 +- tools/doxygen/Doxyfile.core.internal | 2 +- 18 files changed, 20 insertions(+), 20 deletions(-) diff --git a/Makefile b/Makefile index 9be3e5784cc..d369653be3c 100644 --- a/Makefile +++ b/Makefile @@ -414,7 +414,7 @@ E = @echo Q = @ endif -VERSION = 0.15.0-dev +VERSION = 0.15.0 CPPFLAGS_NO_ARCH += $(addprefix -I, $(INCLUDES)) $(addprefix -D, $(DEFINES)) CPPFLAGS += $(CPPFLAGS_NO_ARCH) $(ARCH_FLAGS) diff --git a/build.yaml b/build.yaml index 1f06e20cf9b..ffc681d9028 100644 --- a/build.yaml +++ b/build.yaml @@ -7,7 +7,7 @@ settings: '#3': Use "-preN" suffixes to identify pre-release versions '#4': Per-language overrides are possible with (eg) ruby_version tag here '#5': See the expand_version.py for all the quirks here - version: 0.15.0-dev + version: 0.15.0 filegroups: - name: census public_headers: diff --git a/package.json b/package.json index 5bdaa761e28..1b2920c6bc4 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "grpc", - "version": "0.15.0-dev", + "version": "0.15.0", "author": "Google Inc.", "description": "gRPC Library for Node", "homepage": "http://www.grpc.io/", diff --git a/src/core/lib/surface/version.c b/src/core/lib/surface/version.c index aca76d2bb79..e4a3358c351 100644 --- a/src/core/lib/surface/version.c +++ b/src/core/lib/surface/version.c @@ -36,4 +36,4 @@ #include -const char *grpc_version_string(void) { return "0.15.0-dev"; } +const char *grpc_version_string(void) { return "0.15.0"; } diff --git a/src/csharp/Grpc.Auth/project.json b/src/csharp/Grpc.Auth/project.json index 1677565824b..ae83c3be2b8 100644 --- a/src/csharp/Grpc.Auth/project.json +++ b/src/csharp/Grpc.Auth/project.json @@ -1,5 +1,5 @@ { - "version": "0.15.0-dev", + "version": "0.15.0", "title": "gRPC C# Auth", "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", @@ -13,7 +13,7 @@ "tags": [ "gRPC RPC Protocol HTTP/2 Auth OAuth2" ], }, "dependencies": { - "Grpc.Core": "0.15.0-dev", + "Grpc.Core": "0.15.0", "Google.Apis.Auth": "1.11.1" }, "frameworks": { diff --git a/src/csharp/Grpc.Core/VersionInfo.cs b/src/csharp/Grpc.Core/VersionInfo.cs index e1609341d9a..d89a2b5c6ee 100644 --- a/src/csharp/Grpc.Core/VersionInfo.cs +++ b/src/csharp/Grpc.Core/VersionInfo.cs @@ -53,6 +53,6 @@ namespace Grpc.Core /// /// Current version of gRPC C# /// - public const string CurrentVersion = "0.15.0-dev"; + public const string CurrentVersion = "0.15.0"; } } diff --git a/src/csharp/Grpc.Core/project.json b/src/csharp/Grpc.Core/project.json index 7253107e04a..c7f2bf4fb12 100644 --- a/src/csharp/Grpc.Core/project.json +++ b/src/csharp/Grpc.Core/project.json @@ -1,5 +1,5 @@ { - "version": "0.15.0-dev", + "version": "0.15.0", "title": "gRPC C# Core", "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", diff --git a/src/csharp/Grpc.HealthCheck/project.json b/src/csharp/Grpc.HealthCheck/project.json index eb57608957a..98ea21a436a 100644 --- a/src/csharp/Grpc.HealthCheck/project.json +++ b/src/csharp/Grpc.HealthCheck/project.json @@ -1,5 +1,5 @@ { - "version": "0.15.0-dev", + "version": "0.15.0", "title": "gRPC C# Healthchecking", "authors": [ "Google Inc." ], "copyright": "Copyright 2015, Google Inc.", @@ -13,7 +13,7 @@ "tags": [ "gRPC health check" ] }, "dependencies": { - "Grpc.Core": "0.15.0-dev", + "Grpc.Core": "0.15.0", "Google.Protobuf": "3.0.0-beta3" }, "frameworks": { diff --git a/src/csharp/build_packages.bat b/src/csharp/build_packages.bat index 63f8c30bc7e..e387efcc2da 100644 --- a/src/csharp/build_packages.bat +++ b/src/csharp/build_packages.bat @@ -30,7 +30,7 @@ @rem Builds gRPC NuGet packages @rem Current package versions -set VERSION=0.15.0-dev +set VERSION=0.15.0 set PROTOBUF_VERSION=3.0.0-beta3 @rem Packages that depend on prerelease packages (like Google.Protobuf) need to have prerelease suffix as well. diff --git a/src/node/tools/package.json b/src/node/tools/package.json index c34b259e2e0..b2cadd3f47a 100644 --- a/src/node/tools/package.json +++ b/src/node/tools/package.json @@ -1,6 +1,6 @@ { "name": "grpc-tools", - "version": "0.15.0-dev", + "version": "0.15.0", "author": "Google Inc.", "description": "Tools for developing with gRPC on Node.js", "homepage": "http://www.grpc.io/", diff --git a/src/python/grpcio/grpc_version.py b/src/python/grpcio/grpc_version.py index 0c13104d9da..c6c07afb443 100644 --- a/src/python/grpcio/grpc_version.py +++ b/src/python/grpcio/grpc_version.py @@ -29,4 +29,4 @@ # AUTO-GENERATED FROM `$REPO_ROOT/templates/src/python/grpcio/grpc_version.py.template`!!! -VERSION='0.15.0.dev0' +VERSION='0.15.0' diff --git a/src/ruby/lib/grpc/version.rb b/src/ruby/lib/grpc/version.rb index 01c8c5ac8f1..7f512e47aab 100644 --- a/src/ruby/lib/grpc/version.rb +++ b/src/ruby/lib/grpc/version.rb @@ -29,5 +29,5 @@ # GRPC contains the General RPC module. module GRPC - VERSION = '0.15.0.dev' + VERSION = '0.15.0' end diff --git a/src/ruby/tools/version.rb b/src/ruby/tools/version.rb index dca7fd7e72c..6a7a1d5bd33 100644 --- a/src/ruby/tools/version.rb +++ b/src/ruby/tools/version.rb @@ -29,6 +29,6 @@ module GRPC module Tools - VERSION = '0.15.0.dev' + VERSION = '0.15.0' end end diff --git a/tools/distrib/python/grpcio_tools/grpc_version.py b/tools/distrib/python/grpcio_tools/grpc_version.py index 1267d0e45dc..9a33c6e5d14 100644 --- a/tools/distrib/python/grpcio_tools/grpc_version.py +++ b/tools/distrib/python/grpcio_tools/grpc_version.py @@ -29,4 +29,4 @@ # AUTO-GENERATED FROM `$REPO_ROOT/templates/tools/distrib/python/grpcio_tools/grpc_version.py.template`!!! -VERSION='0.15.0.dev0' +VERSION='0.15.0' diff --git a/tools/doxygen/Doxyfile.c++ b/tools/doxygen/Doxyfile.c++ index 7f9d2df6f6c..066d29ac001 100644 --- a/tools/doxygen/Doxyfile.c++ +++ b/tools/doxygen/Doxyfile.c++ @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC C++" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.15.0 # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a diff --git a/tools/doxygen/Doxyfile.c++.internal b/tools/doxygen/Doxyfile.c++.internal index dcf1a4c8c40..6a0e8b2129c 100644 --- a/tools/doxygen/Doxyfile.c++.internal +++ b/tools/doxygen/Doxyfile.c++.internal @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC C++" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.15.0 # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a diff --git a/tools/doxygen/Doxyfile.core b/tools/doxygen/Doxyfile.core index 72102b2fc50..fa9fd5a3123 100644 --- a/tools/doxygen/Doxyfile.core +++ b/tools/doxygen/Doxyfile.core @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC Core" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.15.0 # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal index e1555930e91..e4c9f991d31 100644 --- a/tools/doxygen/Doxyfile.core.internal +++ b/tools/doxygen/Doxyfile.core.internal @@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC Core" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 0.15.0-dev +PROJECT_NUMBER = 0.15.0 # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a From be1b9a718a09722e95ab04b08ccaf353dc3a51e9 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Fri, 24 Jun 2016 13:22:11 -0700 Subject: [PATCH 210/280] Fixes --- src/core/lib/iomgr/error.c | 5 ++++- src/core/lib/iomgr/tcp_posix.c | 4 ++-- src/core/lib/surface/call.c | 5 +++-- 3 files changed, 9 insertions(+), 5 deletions(-) diff --git a/src/core/lib/iomgr/error.c b/src/core/lib/iomgr/error.c index 9f5ba76fd6c..aa94bb7fdf7 100644 --- a/src/core/lib/iomgr/error.c +++ b/src/core/lib/iomgr/error.c @@ -248,7 +248,10 @@ static grpc_error *copy_error_and_unref(grpc_error *in) { if (is_special(in)) { if (in == GRPC_ERROR_NONE) return GRPC_ERROR_CREATE("no error"); if (in == GRPC_ERROR_OOM) return GRPC_ERROR_CREATE("oom"); - if (in == GRPC_ERROR_CANCELLED) return GRPC_ERROR_CREATE("cancelled"); + if (in == GRPC_ERROR_CANCELLED) + return grpc_error_set_int(GRPC_ERROR_CREATE("cancelled"), + GRPC_ERROR_INT_GRPC_STATUS, + GRPC_STATUS_CANCELLED); return GRPC_ERROR_CREATE("unknown"); } grpc_error *out = gpr_malloc(sizeof(*out)); diff --git a/src/core/lib/iomgr/tcp_posix.c b/src/core/lib/iomgr/tcp_posix.c index 017f52c3677..1046b60bcc9 100644 --- a/src/core/lib/iomgr/tcp_posix.c +++ b/src/core/lib/iomgr/tcp_posix.c @@ -160,7 +160,7 @@ static void call_read_cb(grpc_exec_ctx *exec_ctx, grpc_tcp *tcp, grpc_error *error) { grpc_closure *cb = tcp->read_cb; - if (false && grpc_tcp_trace) { + if (grpc_tcp_trace) { size_t i; const char *str = grpc_error_string(error); gpr_log(GPR_DEBUG, "read: error=%s", str); @@ -394,7 +394,7 @@ static void tcp_write(grpc_exec_ctx *exec_ctx, grpc_endpoint *ep, grpc_tcp *tcp = (grpc_tcp *)ep; grpc_error *error = GRPC_ERROR_NONE; - if (false && grpc_tcp_trace) { + if (grpc_tcp_trace) { size_t i; for (i = 0; i < buf->count; i++) { diff --git a/src/core/lib/surface/call.c b/src/core/lib/surface/call.c index 77c17a4975b..708ea3502af 100644 --- a/src/core/lib/surface/call.c +++ b/src/core/lib/surface/call.c @@ -407,9 +407,10 @@ static void set_status_code(grpc_call *call, status_source source, static void set_status_details(grpc_call *call, status_source source, grpc_mdstr *status) { if (call->status[source].details != NULL) { - GRPC_MDSTR_UNREF(call->status[source].details); + GRPC_MDSTR_UNREF(status); + } else { + call->status[source].details = status; } - call->status[source].details = status; } static void get_final_status(grpc_call *call, From bd96e8a6a38c0afe01c8bfcb0849d7f10ff634e8 Mon Sep 17 00:00:00 2001 From: Ken Payson Date: Fri, 24 Jun 2016 11:53:54 -0700 Subject: [PATCH 211/280] Fix clang formatting --- .../ext/client_config/channel_connectivity.c | 7 ++++--- src/core/lib/profiling/basic_timers.c | 8 ++++---- .../security/credentials/jwt/jwt_credentials.c | 8 ++++---- src/core/lib/surface/channel.c | 16 +++++++++------- src/core/lib/surface/completion_queue.c | 14 ++++++++------ src/core/lib/transport/transport_op_string.c | 2 +- 6 files changed, 30 insertions(+), 25 deletions(-) diff --git a/src/core/ext/client_config/channel_connectivity.c b/src/core/ext/client_config/channel_connectivity.c index 2fd8e0e1232..c1220e3a8c3 100644 --- a/src/core/ext/client_config/channel_connectivity.c +++ b/src/core/ext/client_config/channel_connectivity.c @@ -189,10 +189,11 @@ void grpc_channel_watch_connectivity_state( GRPC_API_TRACE( "grpc_channel_watch_connectivity_state(" "channel=%p, last_observed_state=%d, " - "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %" PRId64 + ", tv_nsec: %d, clock_type: %d }, " "cq=%p, tag=%p)", - 7, (channel, (int)last_observed_state, deadline.tv_sec, - deadline.tv_nsec, (int)deadline.clock_type, cq, tag)); + 7, (channel, (int)last_observed_state, deadline.tv_sec, deadline.tv_nsec, + (int)deadline.clock_type, cq, tag)); grpc_cq_begin_op(cq, tag); diff --git a/src/core/lib/profiling/basic_timers.c b/src/core/lib/profiling/basic_timers.c index ee464f310f4..51813d04617 100644 --- a/src/core/lib/profiling/basic_timers.c +++ b/src/core/lib/profiling/basic_timers.c @@ -141,11 +141,11 @@ static void write_log(gpr_timer_log *log) { entry->tm = gpr_time_0(entry->tm.clock_type); } fprintf(output_file, - "{\"t\": %"PRId64".%09d, \"thd\": \"%d\", \"type\": \"%c\", \"tag\": " + "{\"t\": %" PRId64 + ".%09d, \"thd\": \"%d\", \"type\": \"%c\", \"tag\": " "\"%s\", \"file\": \"%s\", \"line\": %d, \"imp\": %d}\n", - entry->tm.tv_sec, entry->tm.tv_nsec, entry->thd, - entry->type, entry->tagstr, entry->file, entry->line, - entry->important); + entry->tm.tv_sec, entry->tm.tv_nsec, entry->thd, entry->type, + entry->tagstr, entry->file, entry->line, entry->important); } } diff --git a/src/core/lib/security/credentials/jwt/jwt_credentials.c b/src/core/lib/security/credentials/jwt/jwt_credentials.c index f4dd9208610..1ae9f701fbd 100644 --- a/src/core/lib/security/credentials/jwt/jwt_credentials.c +++ b/src/core/lib/security/credentials/jwt/jwt_credentials.c @@ -149,11 +149,11 @@ grpc_call_credentials *grpc_service_account_jwt_access_credentials_create( "grpc_service_account_jwt_access_credentials_create(" "json_key=%s, " "token_lifetime=" - "gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " + "gpr_timespec { tv_sec: %" PRId64 + ", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", - 5, - (json_key, token_lifetime.tv_sec, token_lifetime.tv_nsec, - (int)token_lifetime.clock_type, reserved)); + 5, (json_key, token_lifetime.tv_sec, token_lifetime.tv_nsec, + (int)token_lifetime.clock_type, reserved)); GPR_ASSERT(reserved == NULL); return grpc_service_account_jwt_access_credentials_create_from_auth_json_key( grpc_auth_json_key_create_from_string(json_key), token_lifetime); diff --git a/src/core/lib/surface/channel.c b/src/core/lib/surface/channel.c index 952a769de1d..2cf6d8890a3 100644 --- a/src/core/lib/surface/channel.c +++ b/src/core/lib/surface/channel.c @@ -223,11 +223,12 @@ grpc_call *grpc_channel_create_call(grpc_channel *channel, "grpc_channel_create_call(" "channel=%p, parent_call=%p, propagation_mask=%x, cq=%p, method=%s, " "host=%s, " - "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %" PRId64 + ", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", - 10, (channel, parent_call, (unsigned)propagation_mask, cq, method, host, - deadline.tv_sec, deadline.tv_nsec, - (int)deadline.clock_type, reserved)); + 10, + (channel, parent_call, (unsigned)propagation_mask, cq, method, host, + deadline.tv_sec, deadline.tv_nsec, (int)deadline.clock_type, reserved)); GPR_ASSERT(!reserved); return grpc_channel_create_call_internal( channel, parent_call, propagation_mask, cq, NULL, @@ -282,11 +283,12 @@ grpc_call *grpc_channel_create_registered_call( "grpc_channel_create_registered_call(" "channel=%p, parent_call=%p, propagation_mask=%x, completion_queue=%p, " "registered_call_handle=%p, " - "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %" PRId64 + ", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", 9, (channel, parent_call, (unsigned)propagation_mask, completion_queue, - registered_call_handle, deadline.tv_sec, - deadline.tv_nsec, (int)deadline.clock_type, reserved)); + registered_call_handle, deadline.tv_sec, deadline.tv_nsec, + (int)deadline.clock_type, reserved)); GPR_ASSERT(!reserved); return grpc_channel_create_call_internal( channel, parent_call, propagation_mask, completion_queue, NULL, diff --git a/src/core/lib/surface/completion_queue.c b/src/core/lib/surface/completion_queue.c index aa35ae02fe2..b5f2f65e5c0 100644 --- a/src/core/lib/surface/completion_queue.c +++ b/src/core/lib/surface/completion_queue.c @@ -316,10 +316,11 @@ grpc_event grpc_completion_queue_next(grpc_completion_queue *cc, GRPC_API_TRACE( "grpc_completion_queue_next(" "cc=%p, " - "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %" PRId64 + ", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", - 5, (cc, deadline.tv_sec, deadline.tv_nsec, - (int)deadline.clock_type, reserved)); + 5, (cc, deadline.tv_sec, deadline.tv_nsec, (int)deadline.clock_type, + reserved)); GPR_ASSERT(!reserved); deadline = gpr_convert_clock_type(deadline, GPR_CLOCK_MONOTONIC); @@ -428,10 +429,11 @@ grpc_event grpc_completion_queue_pluck(grpc_completion_queue *cc, void *tag, GRPC_API_TRACE( "grpc_completion_queue_pluck(" "cc=%p, tag=%p, " - "deadline=gpr_timespec { tv_sec: %"PRId64", tv_nsec: %d, clock_type: %d }, " + "deadline=gpr_timespec { tv_sec: %" PRId64 + ", tv_nsec: %d, clock_type: %d }, " "reserved=%p)", - 6, (cc, tag, deadline.tv_sec, deadline.tv_nsec, - (int)deadline.clock_type, reserved)); + 6, (cc, tag, deadline.tv_sec, deadline.tv_nsec, (int)deadline.clock_type, + reserved)); GPR_ASSERT(!reserved); deadline = gpr_convert_clock_type(deadline, GPR_CLOCK_MONOTONIC); diff --git a/src/core/lib/transport/transport_op_string.c b/src/core/lib/transport/transport_op_string.c index d8fae7805e4..aeaba5339fd 100644 --- a/src/core/lib/transport/transport_op_string.c +++ b/src/core/lib/transport/transport_op_string.c @@ -63,7 +63,7 @@ static void put_metadata_list(gpr_strvec *b, grpc_metadata_batch md) { } if (gpr_time_cmp(md.deadline, gpr_inf_future(md.deadline.clock_type)) != 0) { char *tmp; - gpr_asprintf(&tmp, " deadline=%"PRId64".%09d", md.deadline.tv_sec, + gpr_asprintf(&tmp, " deadline=%" PRId64 ".%09d", md.deadline.tv_sec, md.deadline.tv_nsec); gpr_strvec_add(b, tmp); } From ddf02c1c3ff6efb15a9c47b5da7bd76b3654ad94 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Fri, 24 Jun 2016 14:04:01 -0700 Subject: [PATCH 212/280] Fix bad indentation --- tools/run_tests/jobset.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/run_tests/jobset.py b/tools/run_tests/jobset.py index 40409c43948..4fe77487f96 100755 --- a/tools/run_tests/jobset.py +++ b/tools/run_tests/jobset.py @@ -363,7 +363,7 @@ class Jobset(object): self._running.add(job) if not self.resultset.has_key(job.GetSpec().shortname): self.resultset[job.GetSpec().shortname] = [] - return True + return True def reap(self): """Collect the dead jobs.""" From d3b0dda9b8c5ac7dda843e46b240e99a3973d3b6 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Fri, 24 Jun 2016 15:13:08 -0700 Subject: [PATCH 213/280] update grpc nugets to 0.15 --- examples/csharp/helloworld/.nuget/packages.config | 2 +- examples/csharp/helloworld/Greeter/Greeter.csproj | 10 +++++----- examples/csharp/helloworld/Greeter/packages.config | 6 +++--- .../helloworld/GreeterClient/GreeterClient.csproj | 10 +++++----- .../csharp/helloworld/GreeterClient/packages.config | 6 +++--- .../helloworld/GreeterServer/GreeterServer.csproj | 10 +++++----- .../csharp/helloworld/GreeterServer/packages.config | 6 +++--- examples/csharp/helloworld/generate_protos.bat | 2 +- examples/csharp/route_guide/.nuget/packages.config | 2 +- .../csharp/route_guide/RouteGuide/RouteGuide.csproj | 10 +++++----- examples/csharp/route_guide/RouteGuide/packages.config | 6 +++--- .../RouteGuideClient/RouteGuideClient.csproj | 10 +++++----- .../route_guide/RouteGuideClient/packages.config | 6 +++--- .../RouteGuideServer/RouteGuideServer.csproj | 10 +++++----- .../route_guide/RouteGuideServer/packages.config | 6 +++--- examples/csharp/route_guide/generate_protos.bat | 2 +- 16 files changed, 52 insertions(+), 52 deletions(-) diff --git a/examples/csharp/helloworld/.nuget/packages.config b/examples/csharp/helloworld/.nuget/packages.config index bfd6c6723d9..aa060800c19 100644 --- a/examples/csharp/helloworld/.nuget/packages.config +++ b/examples/csharp/helloworld/.nuget/packages.config @@ -1,4 +1,4 @@  - + \ No newline at end of file diff --git a/examples/csharp/helloworld/Greeter/Greeter.csproj b/examples/csharp/helloworld/Greeter/Greeter.csproj index 0270cc25f7a..20b85db8b60 100644 --- a/examples/csharp/helloworld/Greeter/Greeter.csproj +++ b/examples/csharp/helloworld/Greeter/Greeter.csproj @@ -10,7 +10,7 @@ Greeter Greeter v4.5 - 745ac60f + 2669b4f2 true @@ -33,11 +33,11 @@ False - ..\packages\Google.Protobuf.3.0.0-beta2\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll + ..\packages\Google.Protobuf.3.0.0-beta3\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll False - ..\packages\Grpc.Core.0.14.0\lib\net45\Grpc.Core.dll + ..\packages\Grpc.Core.0.15.0\lib\net45\Grpc.Core.dll @@ -61,11 +61,11 @@ - + This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. - + \ No newline at end of file diff --git a/examples/csharp/helloworld/Greeter/packages.config b/examples/csharp/helloworld/Greeter/packages.config index 617fe6da7be..ff9d6bbf73f 100644 --- a/examples/csharp/helloworld/Greeter/packages.config +++ b/examples/csharp/helloworld/Greeter/packages.config @@ -1,7 +1,7 @@  - - - + + + \ No newline at end of file diff --git a/examples/csharp/helloworld/GreeterClient/GreeterClient.csproj b/examples/csharp/helloworld/GreeterClient/GreeterClient.csproj index 877c450a50e..2b38ce290e9 100644 --- a/examples/csharp/helloworld/GreeterClient/GreeterClient.csproj +++ b/examples/csharp/helloworld/GreeterClient/GreeterClient.csproj @@ -10,7 +10,7 @@ GreeterClient GreeterClient v4.5 - 63b59176 + 5e942a7d true @@ -33,11 +33,11 @@ False - ..\packages\Google.Protobuf.3.0.0-beta2\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll + ..\packages\Google.Protobuf.3.0.0-beta3\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll False - ..\packages\Grpc.Core.0.14.0\lib\net45\Grpc.Core.dll + ..\packages\Grpc.Core.0.15.0\lib\net45\Grpc.Core.dll @@ -59,11 +59,11 @@ - + This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. - + \ No newline at end of file diff --git a/examples/csharp/helloworld/GreeterClient/packages.config b/examples/csharp/helloworld/GreeterClient/packages.config index 617fe6da7be..ff9d6bbf73f 100644 --- a/examples/csharp/helloworld/GreeterClient/packages.config +++ b/examples/csharp/helloworld/GreeterClient/packages.config @@ -1,7 +1,7 @@  - - - + + + \ No newline at end of file diff --git a/examples/csharp/helloworld/GreeterServer/GreeterServer.csproj b/examples/csharp/helloworld/GreeterServer/GreeterServer.csproj index 4d792dcf32e..43c633678b1 100644 --- a/examples/csharp/helloworld/GreeterServer/GreeterServer.csproj +++ b/examples/csharp/helloworld/GreeterServer/GreeterServer.csproj @@ -10,7 +10,7 @@ GreeterServer GreeterServer v4.5 - 25ac2e80 + 9c7b2963 true @@ -33,11 +33,11 @@ False - ..\packages\Google.Protobuf.3.0.0-beta2\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll + ..\packages\Google.Protobuf.3.0.0-beta3\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll False - ..\packages\Grpc.Core.0.14.0\lib\net45\Grpc.Core.dll + ..\packages\Grpc.Core.0.15.0\lib\net45\Grpc.Core.dll @@ -59,11 +59,11 @@ - + This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. - + \ No newline at end of file diff --git a/examples/csharp/helloworld/GreeterServer/packages.config b/examples/csharp/helloworld/GreeterServer/packages.config index 617fe6da7be..ff9d6bbf73f 100644 --- a/examples/csharp/helloworld/GreeterServer/packages.config +++ b/examples/csharp/helloworld/GreeterServer/packages.config @@ -1,7 +1,7 @@  - - - + + + \ No newline at end of file diff --git a/examples/csharp/helloworld/generate_protos.bat b/examples/csharp/helloworld/generate_protos.bat index b9f68d6745a..a952bb46cdc 100644 --- a/examples/csharp/helloworld/generate_protos.bat +++ b/examples/csharp/helloworld/generate_protos.bat @@ -34,7 +34,7 @@ setlocal @rem enter this directory cd /d %~dp0 -set TOOLS_PATH=packages\Grpc.Tools.0.14.0\tools\windows_x86 +set TOOLS_PATH=packages\Grpc.Tools.0.15.0\tools\windows_x86 %TOOLS_PATH%\protoc.exe -I../../protos --csharp_out Greeter ../../protos/helloworld.proto --grpc_out Greeter --plugin=protoc-gen-grpc=%TOOLS_PATH%\grpc_csharp_plugin.exe diff --git a/examples/csharp/route_guide/.nuget/packages.config b/examples/csharp/route_guide/.nuget/packages.config index bfd6c6723d9..aa060800c19 100644 --- a/examples/csharp/route_guide/.nuget/packages.config +++ b/examples/csharp/route_guide/.nuget/packages.config @@ -1,4 +1,4 @@  - + \ No newline at end of file diff --git a/examples/csharp/route_guide/RouteGuide/RouteGuide.csproj b/examples/csharp/route_guide/RouteGuide/RouteGuide.csproj index 4f7222ebba3..601d16ba24d 100644 --- a/examples/csharp/route_guide/RouteGuide/RouteGuide.csproj +++ b/examples/csharp/route_guide/RouteGuide/RouteGuide.csproj @@ -11,7 +11,7 @@ RouteGuide v4.5 512 - 0a9fcb7a + de2137f9 true @@ -33,11 +33,11 @@ False - ..\packages\Google.Protobuf.3.0.0-beta2\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll + ..\packages\Google.Protobuf.3.0.0-beta3\lib\portable-net45+netcore45+wpa81+wp8\Google.Protobuf.dll False - ..\packages\Grpc.Core.0.14.0\lib\net45\Grpc.Core.dll + ..\packages\Grpc.Core.0.15.0\lib\net45\Grpc.Core.dll False @@ -74,12 +74,12 @@ - + This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. - +