Merge pull request #6830 from dgquintas/simplify_compression_interop

Simplify compression interop
pull/6841/head
Jan Tattermusch 9 years ago
commit 129cd65869
  1. 127
      doc/interop-test-descriptions.md
  2. 19
      src/proto/grpc/testing/messages.proto
  3. 97
      test/cpp/interop/interop_client.cc
  4. 24
      test/cpp/interop/server_main.cc

@ -93,26 +93,24 @@ Client asserts:
### large_compressed_unary ### large_compressed_unary
This test verifies compressed unary calls succeed in sending messages. It This test verifies compressed unary calls succeed in sending messages. It
sends one unary request for every combination of compression algorithm and sends one unary request for every payload type, with and without requesting a
payload type. compressed response from the server.
In all scenarios, whether compression was actually performed is determined by In all scenarios, whether compression was actually performed is determined by
the compression bit in the response's message flags. The response's compression the compression bit in the response's message flags.
value indicates which algorithm was used if said compression bit is set.
Server features: Server features:
* [UnaryCall][] * [UnaryCall][]
* [Compressable Payload][] * [Compressable Payload][]
* [Uncompressable Payload][] * [Uncompressable Payload][]
* [Random Payload][]
Procedure: Procedure:
1. Client calls UnaryCall with: 1. Client calls UnaryCall with:
``` ```
{ {
response_compression: <one of {NONE, GZIP, DEFLATE}> request_compressed_response: bool
response_type: COMPRESSABLE response_type: COMPRESSABLE
response_size: 314159 response_size: 314159
payload:{ payload:{
@ -123,11 +121,10 @@ Procedure:
Client asserts: Client asserts:
* call was successful * call was successful
* response payload type is COMPRESSABLE * response payload type is COMPRESSABLE
* response compression is consistent with the requested one. * if `request_compressed_response` is false, the response MUST NOT have the
* if `response_compression == NONE`, the response MUST NOT have the compressed message flag set.
* if `request_compressed_response` is true, the response MUST have the
compressed message flag set. compressed message flag set.
* if `response_compression != NONE`, the response MUST have the compressed
message flag set.
* response payload body is 314159 bytes in size * response payload body is 314159 bytes in size
* clients are free to assert that the response payload body contents are * clients are free to assert that the response payload body contents are
zero and comparing the entire response message against a golden response zero and comparing the entire response message against a golden response
@ -136,7 +133,7 @@ Procedure:
2. Client calls UnaryCall with: 2. Client calls UnaryCall with:
``` ```
{ {
response_compression: <one of {NONE, GZIP, DEFLATE}> request_compressed_response: bool
response_type: UNCOMPRESSABLE response_type: UNCOMPRESSABLE
response_size: 314159 response_size: 314159
payload:{ payload:{
@ -147,29 +144,11 @@ Procedure:
Client asserts: Client asserts:
* call was successful * call was successful
* response payload type is UNCOMPRESSABLE * response payload type is UNCOMPRESSABLE
* response compression is consistent with the requested one. * the response MAY have the compressed message flag set. Some
* the response MUST NOT have the compressed message flag set. implementations will choose to compress the payload even when the output
size if larger than the input.
* response payload body is 314159 bytes in size * response payload body is 314159 bytes in size
* clients are free to assert that the response payload body contents are
identical to the golden uncompressable data at `test/cpp/interop/rnd.dat`.
3. Client calls UnaryCall with:
```
{
response_compression: <one of {NONE, GZIP, DEFLATE}>
response_type: RANDOM
response_size: 314159
payload:{
body: 271828 bytes of zeros
}
}
```
Client asserts:
* call was successful
* response payload type is either COMPRESSABLE or UNCOMPRESSABLE
* the behavior is consistent with the randomly chosen incoming payload type,
as described in their respective sections.
### client_streaming ### client_streaming
@ -245,7 +224,7 @@ Procedure:
size: 31415 size: 31415
} }
response_parameters:{ response_parameters:{
size: 9 size: 59
} }
response_parameters:{ response_parameters:{
size: 2653 size: 2653
@ -272,7 +251,6 @@ Server features:
* [StreamingOutputCall][] * [StreamingOutputCall][]
* [Compressable Payload][] * [Compressable Payload][]
* [Uncompressable Payload][] * [Uncompressable Payload][]
* [Random Payload][]
Procedure: Procedure:
@ -280,13 +258,13 @@ Procedure:
``` ```
{ {
response_compression: <one of {NONE, GZIP, DEFLATE}> request_compressed_response: bool
response_type:COMPRESSABLE response_type:COMPRESSABLE
response_parameters:{ response_parameters:{
size: 31415 size: 31415
} }
response_parameters:{ response_parameters:{
size: 9 size: 59
} }
response_parameters:{ response_parameters:{
size: 2653 size: 2653
@ -301,12 +279,11 @@ Procedure:
* call was successful * call was successful
* exactly four responses * exactly four responses
* response payloads are COMPRESSABLE * response payloads are COMPRESSABLE
* response compression is consistent with the requested one. * if `request_compressed_response` is false, the response's messages MUST
* if `response_compression == NONE`, the response MUST NOT have the NOT have the compressed message flag set.
compressed message flag set. * if `request_compressed_response` is true, the response's messages MUST
* if `response_compression != NONE`, the response MUST have the compressed have the compressed message flag set.
message flag set. * response payload bodies are sized (in order): 31415, 59, 2653, 58979
* response payload bodies are sized (in order): 31415, 9, 2653, 58979
* clients are free to assert that the response payload body contents are * clients are free to assert that the response payload body contents are
zero and comparing the entire response messages against golden responses zero and comparing the entire response messages against golden responses
@ -315,13 +292,13 @@ Procedure:
``` ```
{ {
response_compression: <one of {NONE, GZIP, DEFLATE}> request_compressed_response: bool
response_type:UNCOMPRESSABLE response_type:UNCOMPRESSABLE
response_parameters:{ response_parameters:{
size: 31415 size: 31415
} }
response_parameters:{ response_parameters:{
size: 9 size: 59
} }
response_parameters:{ response_parameters:{
size: 2653 size: 2653
@ -336,40 +313,14 @@ Procedure:
* call was successful * call was successful
* exactly four responses * exactly four responses
* response payloads are UNCOMPRESSABLE * response payloads are UNCOMPRESSABLE
* response compressions are consistent with the requested one. * the response MAY have the compressed message flag set. Some
* the responses MUST NOT have the compressed message flag set. implementations will choose to compress the payload even when the output
* response payload bodies are sized (in order): 31415, 9, 2653, 58979 size if larger than the input.
* response payload bodies are sized (in order): 31415, 59, 2653, 58979
* clients are free to assert that the body of the responses are identical to * clients are free to assert that the body of the responses are identical to
the golden uncompressable data at `test/cpp/interop/rnd.dat`. the golden uncompressable data at `test/cpp/interop/rnd.dat`.
3. Client calls StreamingOutputCall with:
```
{
response_compression: <one of {NONE, GZIP, DEFLATE}>
response_type:RANDOM
response_parameters:{
size: 31415
}
response_parameters:{
size: 9
}
response_parameters:{
size: 2653
}
response_parameters:{
size: 58979
}
}
```
Client asserts:
* call was successful
* response payload type is either COMPRESSABLE or UNCOMPRESSABLE
* the behavior is consistent with the randomly chosen incoming payload type,
as described in their respective sections.
### ping_pong ### ping_pong
This test verifies that full duplex bidi is supported. This test verifies that full duplex bidi is supported.
@ -399,7 +350,7 @@ Procedure:
{ {
response_type: COMPRESSABLE response_type: COMPRESSABLE
response_parameters:{ response_parameters:{
size: 9 size: 59
} }
payload:{ payload:{
body: 8 bytes of zeros body: 8 bytes of zeros
@ -932,9 +883,9 @@ Server implements EmptyCall which immediately returns the empty message.
[UnaryCall]: #unarycall [UnaryCall]: #unarycall
Server implements UnaryCall which immediately returns a SimpleResponse with a Server implements UnaryCall which immediately returns a SimpleResponse with a
payload body of size SimpleRequest.response_size bytes and type as appropriate payload body of size `SimpleRequest.response_size` bytes and type as appropriate
for the SimpleRequest.response_type. If the server does not support the for the `SimpleRequest.response_type`. If the server does not support the
response_type, then it should fail the RPC with INVALID_ARGUMENT. `response_type`, then it should fail the RPC with `INVALID_ARGUMENT`.
### StreamingInputCall ### StreamingInputCall
[StreamingInputCall]: #streaminginputcall [StreamingInputCall]: #streaminginputcall
@ -974,15 +925,7 @@ COMPRESSABLE.
When the client requests UNCOMPRESSABLE payload, the response includes a payload When the client requests UNCOMPRESSABLE payload, the response includes a payload
of the size requested containing uncompressable data and the payload type is of the size requested containing uncompressable data and the payload type is
UNCOMPRESSABLE. A 512 kB dump from /dev/urandom is the current golden data, UNCOMPRESSABLE.
stored at `test/cpp/interop/rnd.dat`
### Random Payload
[Random Payload]: #random-payload
When the client requests RANDOM payload, the response includes either a randomly
chosen COMPRESSABLE or UNCOMPRESSABLE payload. The data and the payload type
will be consistent with this choice.
### Echo Status ### Echo Status
[Echo Status]: #echo-status [Echo Status]: #echo-status
@ -1004,8 +947,8 @@ key and the corresponding value back to the client as trailing metadata.
[Observe ResponseParameters.interval_us]: #observe-responseparametersinterval_us [Observe ResponseParameters.interval_us]: #observe-responseparametersinterval_us
In StreamingOutputCall and FullDuplexCall, server delays sending a In StreamingOutputCall and FullDuplexCall, server delays sending a
StreamingOutputCallResponse by the ResponseParameters's interval_us for that StreamingOutputCallResponse by the ResponseParameters's `interval_us` for that
particular response, relative to the last response sent. That is, interval_us particular response, relative to the last response sent. That is, `interval_us`
acts like a sleep *before* sending the response and accumulates from one acts like a sleep *before* sending the response and accumulates from one
response to the next. response to the next.
@ -1027,13 +970,13 @@ an email address.
#### Echo OAuth scope #### Echo OAuth scope
[Echo OAuth Scope]: #echo-oauth-scope [Echo OAuth Scope]: #echo-oauth-scope
If a SimpleRequest has fill_oauth_scope=true and that request was successfully If a SimpleRequest has `fill_oauth_scope=true` and that request was successfully
authenticated via OAuth, then the SimpleResponse should have oauth_scope filled authenticated via OAuth, then the SimpleResponse should have oauth_scope filled
with the scope of the method being invoked. with the scope of the method being invoked.
Although a general server-side feature, most test servers won't implement this Although a general server-side feature, most test servers won't implement this
feature. The TLS server grpc-test.sandbox.googleapis.com:443 supports this feature. feature. The TLS server `grpc-test.sandbox.googleapis.com:443` supports this
It requires at least the OAuth scope feature. It requires at least the OAuth scope
`https://www.googleapis.com/auth/xapi.zoo` for authentication to succeed. `https://www.googleapis.com/auth/xapi.zoo` for authentication to succeed.
Discussion: Discussion:

@ -41,17 +41,6 @@ enum PayloadType {
// Uncompressable binary format. // Uncompressable binary format.
UNCOMPRESSABLE = 1; UNCOMPRESSABLE = 1;
// Randomly chosen from all other formats defined in this enum.
RANDOM = 2;
}
// Compression algorithms
enum CompressionType {
// No compression
NONE = 0;
GZIP = 1;
DEFLATE = 2;
} }
// A block of data, to simply increase gRPC message size. // A block of data, to simply increase gRPC message size.
@ -88,8 +77,8 @@ message SimpleRequest {
// Whether SimpleResponse should include OAuth scope. // Whether SimpleResponse should include OAuth scope.
bool fill_oauth_scope = 5; bool fill_oauth_scope = 5;
// Compression algorithm to be used by the server for the response (stream) // Whether to request the server to compress the response.
CompressionType response_compression = 6; bool request_compressed_response = 6;
// Whether server should return a given status // Whether server should return a given status
EchoStatus response_status = 7; EchoStatus response_status = 7;
@ -145,8 +134,8 @@ message StreamingOutputCallRequest {
// Optional input payload sent along with the request. // Optional input payload sent along with the request.
Payload payload = 3; Payload payload = 3;
// Compression algorithm to be used by the server for the response (stream) // Whether to request the server to compress the response.
CompressionType response_compression = 6; bool request_compressed_response = 6;
// Whether server should return a given status // Whether server should return a given status
EchoStatus response_status = 7; EchoStatus response_status = 7;

@ -55,8 +55,6 @@
namespace grpc { namespace grpc {
namespace testing { namespace testing {
static const char* kRandomFile = "test/cpp/interop/rnd.dat";
namespace { namespace {
// The same value is defined by the Java client. // The same value is defined by the Java client.
const std::vector<int> request_stream_sizes = {27182, 8, 1828, 45904}; const std::vector<int> request_stream_sizes = {27182, 8, 1828, 45904};
@ -67,30 +65,26 @@ const int kReceiveDelayMilliSeconds = 20;
const int kLargeRequestSize = 271828; const int kLargeRequestSize = 271828;
const int kLargeResponseSize = 314159; const int kLargeResponseSize = 314159;
CompressionType GetInteropCompressionTypeFromCompressionAlgorithm(
grpc_compression_algorithm algorithm) {
switch (algorithm) {
case GRPC_COMPRESS_NONE:
return CompressionType::NONE;
case GRPC_COMPRESS_GZIP:
return CompressionType::GZIP;
case GRPC_COMPRESS_DEFLATE:
return CompressionType::DEFLATE;
default:
GPR_ASSERT(false);
}
}
void NoopChecks(const InteropClientContextInspector& inspector, void NoopChecks(const InteropClientContextInspector& inspector,
const SimpleRequest* request, const SimpleResponse* response) {} const SimpleRequest* request, const SimpleResponse* response) {}
void CompressionChecks(const InteropClientContextInspector& inspector, void CompressionChecks(const InteropClientContextInspector& inspector,
const SimpleRequest* request, const SimpleRequest* request,
const SimpleResponse* response) { const SimpleResponse* response) {
GPR_ASSERT(request->response_compression() == const grpc_compression_algorithm received_compression =
GetInteropCompressionTypeFromCompressionAlgorithm( inspector.GetCallCompressionAlgorithm();
inspector.GetCallCompressionAlgorithm())); if (request->request_compressed_response() &&
if (request->response_compression() == NONE) { received_compression == GRPC_COMPRESS_NONE) {
if (request->request_compressed_response() &&
received_compression == GRPC_COMPRESS_NONE) {
// Requested some compression, got NONE. This is an error.
gpr_log(GPR_ERROR,
"Failure: Requested compression but got uncompressed response "
"from server.");
abort();
}
}
if (!request->request_compressed_response()) {
GPR_ASSERT(!(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS)); GPR_ASSERT(!(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS));
} else if (request->response_type() == PayloadType::COMPRESSABLE) { } else if (request->response_type() == PayloadType::COMPRESSABLE) {
// requested compression and compressable response => results should always // requested compression and compressable response => results should always
@ -211,20 +205,22 @@ bool InteropClient::PerformLargeUnary(SimpleRequest* request,
custom_checks_fn(inspector, request, response); custom_checks_fn(inspector, request, response);
// Payload related checks. // Payload related checks.
if (request->response_type() != PayloadType::RANDOM) {
GPR_ASSERT(response->payload().type() == request->response_type()); GPR_ASSERT(response->payload().type() == request->response_type());
}
switch (response->payload().type()) { switch (response->payload().type()) {
case PayloadType::COMPRESSABLE: case PayloadType::COMPRESSABLE:
GPR_ASSERT(response->payload().body() == GPR_ASSERT(response->payload().body() ==
grpc::string(kLargeResponseSize, '\0')); grpc::string(kLargeResponseSize, '\0'));
break; break;
case PayloadType::UNCOMPRESSABLE: { case PayloadType::UNCOMPRESSABLE: {
std::ifstream rnd_file(kRandomFile); // We don't really check anything: We can't assert that the payload is
GPR_ASSERT(rnd_file.good()); // uncompressed because it's the server's prerogative to decide on that,
for (int i = 0; i < kLargeResponseSize; i++) { // and different implementations decide differently (ie, Java always
GPR_ASSERT(response->payload().body()[i] == (char)rnd_file.get()); // compresses when requested to do so, whereas C core throws away the
} // compressed payload if the output is larger than the input).
// In addition, we don't compare the actual random bytes received because
// asserting that data is sent/received properly isn't the purpose of this
// test. Moreover, different implementations are also free to use
// different sets of random bytes.
} break; } break;
default: default:
GPR_ASSERT(false); GPR_ASSERT(false);
@ -341,13 +337,13 @@ bool InteropClient::DoLargeUnary() {
} }
bool InteropClient::DoLargeCompressedUnary() { bool InteropClient::DoLargeCompressedUnary() {
const CompressionType compression_types[] = {NONE, GZIP, DEFLATE}; const bool request_compression[] = {false, true};
const PayloadType payload_types[] = {COMPRESSABLE, UNCOMPRESSABLE, RANDOM}; const PayloadType payload_types[] = {COMPRESSABLE, UNCOMPRESSABLE};
for (size_t i = 0; i < GPR_ARRAY_SIZE(payload_types); i++) { for (size_t i = 0; i < GPR_ARRAY_SIZE(payload_types); i++) {
for (size_t j = 0; j < GPR_ARRAY_SIZE(compression_types); j++) { for (size_t j = 0; j < GPR_ARRAY_SIZE(request_compression); j++) {
char* log_suffix; char* log_suffix;
gpr_asprintf(&log_suffix, "(compression=%s; payload=%s)", gpr_asprintf(&log_suffix, "(compression=%s; payload=%s)",
CompressionType_Name(compression_types[j]).c_str(), request_compression[j] ? "true" : "false",
PayloadType_Name(payload_types[i]).c_str()); PayloadType_Name(payload_types[i]).c_str());
gpr_log(GPR_DEBUG, "Sending a large compressed unary rpc %s.", gpr_log(GPR_DEBUG, "Sending a large compressed unary rpc %s.",
@ -355,7 +351,7 @@ bool InteropClient::DoLargeCompressedUnary() {
SimpleRequest request; SimpleRequest request;
SimpleResponse response; SimpleResponse response;
request.set_response_type(payload_types[i]); request.set_response_type(payload_types[i]);
request.set_response_compression(compression_types[j]); request.set_request_compressed_response(request_compression[j]);
if (!PerformLargeUnary(&request, &response, CompressionChecks)) { if (!PerformLargeUnary(&request, &response, CompressionChecks)) {
gpr_log(GPR_ERROR, "Large compressed unary failed %s", log_suffix); gpr_log(GPR_ERROR, "Large compressed unary failed %s", log_suffix);
@ -452,23 +448,23 @@ bool InteropClient::DoResponseStreaming() {
} }
bool InteropClient::DoResponseCompressedStreaming() { bool InteropClient::DoResponseCompressedStreaming() {
const CompressionType compression_types[] = {NONE, GZIP, DEFLATE}; const bool request_compression[] = {false, true};
const PayloadType payload_types[] = {COMPRESSABLE, UNCOMPRESSABLE, RANDOM}; const PayloadType payload_types[] = {COMPRESSABLE, UNCOMPRESSABLE};
for (size_t i = 0; i < GPR_ARRAY_SIZE(payload_types); i++) { for (size_t i = 0; i < GPR_ARRAY_SIZE(payload_types); i++) {
for (size_t j = 0; j < GPR_ARRAY_SIZE(compression_types); j++) { for (size_t j = 0; j < GPR_ARRAY_SIZE(request_compression); j++) {
ClientContext context; ClientContext context;
InteropClientContextInspector inspector(context); InteropClientContextInspector inspector(context);
StreamingOutputCallRequest request; StreamingOutputCallRequest request;
char* log_suffix; char* log_suffix;
gpr_asprintf(&log_suffix, "(compression=%s; payload=%s)", gpr_asprintf(&log_suffix, "(compression=%s; payload=%s)",
CompressionType_Name(compression_types[j]).c_str(), request_compression[j] ? "true" : "false",
PayloadType_Name(payload_types[i]).c_str()); PayloadType_Name(payload_types[i]).c_str());
gpr_log(GPR_DEBUG, "Receiving response streaming rpc %s.", log_suffix); gpr_log(GPR_DEBUG, "Receiving response streaming rpc %s.", log_suffix);
request.set_response_type(payload_types[i]); request.set_response_type(payload_types[i]);
request.set_response_compression(compression_types[j]); request.set_request_compressed_response(request_compression[j]);
for (size_t k = 0; k < response_stream_sizes.size(); ++k) { for (size_t k = 0; k < response_stream_sizes.size(); ++k) {
ResponseParameters* response_parameter = ResponseParameters* response_parameter =
@ -483,38 +479,33 @@ bool InteropClient::DoResponseCompressedStreaming() {
size_t k = 0; size_t k = 0;
while (stream->Read(&response)) { while (stream->Read(&response)) {
// Payload related checks. // Payload related checks.
if (request.response_type() != PayloadType::RANDOM) {
GPR_ASSERT(response.payload().type() == request.response_type()); GPR_ASSERT(response.payload().type() == request.response_type());
}
switch (response.payload().type()) { switch (response.payload().type()) {
case PayloadType::COMPRESSABLE: case PayloadType::COMPRESSABLE:
GPR_ASSERT(response.payload().body() == GPR_ASSERT(response.payload().body() ==
grpc::string(response_stream_sizes[k], '\0')); grpc::string(response_stream_sizes[k], '\0'));
break; break;
case PayloadType::UNCOMPRESSABLE: { case PayloadType::UNCOMPRESSABLE:
std::ifstream rnd_file(kRandomFile); break;
GPR_ASSERT(rnd_file.good());
for (int n = 0; n < response_stream_sizes[k]; n++) {
GPR_ASSERT(response.payload().body()[n] == (char)rnd_file.get());
}
} break;
default: default:
GPR_ASSERT(false); GPR_ASSERT(false);
} }
// Compression related checks. // Compression related checks.
GPR_ASSERT(request.response_compression() == if (request.request_compressed_response()) {
GetInteropCompressionTypeFromCompressionAlgorithm( GPR_ASSERT(inspector.GetCallCompressionAlgorithm() >
inspector.GetCallCompressionAlgorithm())); GRPC_COMPRESS_NONE);
if (request.response_compression() == NONE) { if (request.response_type() == PayloadType::COMPRESSABLE) {
GPR_ASSERT(
!(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS));
} else if (request.response_type() == PayloadType::COMPRESSABLE) {
// requested compression and compressable response => results should // requested compression and compressable response => results should
// always be compressed. // always be compressed.
GPR_ASSERT(inspector.GetMessageFlags() & GPR_ASSERT(inspector.GetMessageFlags() &
GRPC_WRITE_INTERNAL_COMPRESS); GRPC_WRITE_INTERNAL_COMPRESS);
} }
} else {
// requested *no* compression.
GPR_ASSERT(
!(inspector.GetMessageFlags() & GRPC_WRITE_INTERNAL_COMPRESS));
}
++k; ++k;
} }

@ -110,14 +110,7 @@ void MaybeEchoMetadata(ServerContext* context) {
} }
} }
bool SetPayload(PayloadType type, int size, Payload* payload) { bool SetPayload(PayloadType response_type, int size, Payload* payload) {
PayloadType response_type;
if (type == PayloadType::RANDOM) {
response_type =
rand() & 0x1 ? PayloadType::COMPRESSABLE : PayloadType::UNCOMPRESSABLE;
} else {
response_type = type;
}
payload->set_type(response_type); payload->set_type(response_type);
switch (response_type) { switch (response_type) {
case PayloadType::COMPRESSABLE: { case PayloadType::COMPRESSABLE: {
@ -141,18 +134,9 @@ bool SetPayload(PayloadType type, int size, Payload* payload) {
template <typename RequestType> template <typename RequestType>
void SetResponseCompression(ServerContext* context, void SetResponseCompression(ServerContext* context,
const RequestType& request) { const RequestType& request) {
switch (request.response_compression()) { if (request.request_compressed_response()) {
case grpc::testing::NONE: // Any level would do, let's go for HIGH because we are overachievers.
context->set_compression_algorithm(GRPC_COMPRESS_NONE); context->set_compression_level(GRPC_COMPRESS_LEVEL_HIGH);
break;
case grpc::testing::GZIP:
context->set_compression_algorithm(GRPC_COMPRESS_GZIP);
break;
case grpc::testing::DEFLATE:
context->set_compression_algorithm(GRPC_COMPRESS_DEFLATE);
break;
default:
abort();
} }
} }

Loading…
Cancel
Save