|
|
|
@ -1070,24 +1070,26 @@ logged and recorded in the latency histogram. |
|
|
|
|
### orca_per_rpc |
|
|
|
|
[orca_per_rpc]: #orca_per_rpc |
|
|
|
|
|
|
|
|
|
The client verifies that a custom LB policy, which is integrated with ORCA APIs, will receive |
|
|
|
|
per-query metric reports from the backend. |
|
|
|
|
|
|
|
|
|
The client will register the custom LB policy named `test_backend_metrics_load_balancer`, which |
|
|
|
|
using ORCA APIs already installed a per-query report listener. The interop-testing client will run with a |
|
|
|
|
service config to select the load balancing config (using argument `--service_config_json`), so that |
|
|
|
|
it effectively uses this newly registered custom LB policy. A load report reference can be passed |
|
|
|
|
from the call to the LB policy through, e.g. CallOptions, to receive metric reports. |
|
|
|
|
The LB policy will fill in the reference with the latest load report from the report listener. |
|
|
|
|
This way, together with server behaviors we can verify the expected metric reports are received. |
|
|
|
|
The client verifies that a custom LB policy, which is integrated with ORCA APIs, |
|
|
|
|
will receive per-query metric reports from the backend. |
|
|
|
|
|
|
|
|
|
The client will register the custom LB policy named |
|
|
|
|
`test_backend_metrics_load_balancer`, which using ORCA APIs already installed a |
|
|
|
|
per-query report listener. The interop-testing client will run with a service |
|
|
|
|
config to select the load balancing config (using argument |
|
|
|
|
`--service_config_json`), so that it effectively uses this newly registered |
|
|
|
|
custom LB policy. A load report reference can be passed from the call to the LB |
|
|
|
|
policy through, e.g. CallOptions, to receive metric reports. The LB policy will |
|
|
|
|
fill in the reference with the latest load report from the report listener. |
|
|
|
|
This way, together with server behaviors we can verify the expected metric |
|
|
|
|
reports are received. |
|
|
|
|
|
|
|
|
|
Server features: |
|
|
|
|
* [UnaryCall][] |
|
|
|
|
* [Backend Metrics Report][] |
|
|
|
|
|
|
|
|
|
Procedures: |
|
|
|
|
* The client sends a unary request to the server. The call request sets `orca_per_rpc_report` to a |
|
|
|
|
test load report. |
|
|
|
|
* The client sends a unary request: |
|
|
|
|
``` |
|
|
|
|
{ |
|
|
|
|
orca_per_rpc_report:{ |
|
|
|
@ -1104,34 +1106,38 @@ test load report. |
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
The call carries a reference to receive the load report, e.g. using CallOptions. |
|
|
|
|
The reference is passed to the custom LB policy as part of the `OrcaPerRequestReportListener` API. |
|
|
|
|
The reference is passed to the custom LB policy as part of the |
|
|
|
|
`OrcaPerRequestReportListener` API. |
|
|
|
|
|
|
|
|
|
Client asserts: |
|
|
|
|
* The call is successful. |
|
|
|
|
* The per-query load report reference contains a metrics report that is identical to the metrics |
|
|
|
|
data sent in the request shown above. |
|
|
|
|
* The per-query load report reference contains a metrics report that is |
|
|
|
|
identical to the metrics data sent in the request shown above. |
|
|
|
|
|
|
|
|
|
### orca_oob |
|
|
|
|
|
|
|
|
|
The client verifies that a custom LB policy, which is integrated with ORCA APIs, will receive |
|
|
|
|
out-of-band metric reports from the backend. |
|
|
|
|
|
|
|
|
|
The client will register the custom LB policy named `test_backend_metrics_load_balancer`. It has |
|
|
|
|
similar and additional functions as described in the [orca_per_rpc][] test. |
|
|
|
|
We use ORCA APIs to install an out-of-band report listener (configure load report interval to be 1s) |
|
|
|
|
in the LB policy. The interop-testing client will run with a service config to select the load |
|
|
|
|
balancing config(using argument `--service_config_json`), so that it effectively uses this newly |
|
|
|
|
registered custom LB policy. A load report reference can be passed from the call to the LB policy |
|
|
|
|
through, e.g. CallOptions, to receive metric reports. |
|
|
|
|
The test framework will fill in the reference with the latest load report from the report listener. |
|
|
|
|
This way, together with server behaviors we can verify the expected metric reports are received. |
|
|
|
|
The client verifies that a custom LB policy, which is integrated with ORCA APIs, |
|
|
|
|
will receive out-of-band metric reports from the backend. |
|
|
|
|
|
|
|
|
|
The client will register the custom LB policy named |
|
|
|
|
`test_backend_metrics_load_balancer`. It has similar and additional functions as |
|
|
|
|
described in the [orca_per_rpc][] test. We use ORCA APIs to install an |
|
|
|
|
out-of-band report listener (configure load report interval to be 1s) in the LB |
|
|
|
|
policy. The interop-testing client will run with a service config to select the |
|
|
|
|
load balancing config(using argument `--service_config_json`), so that it |
|
|
|
|
effectively uses this newly registered custom LB policy. A load report reference |
|
|
|
|
can be passed from the call to the LB policy through, e.g. CallOptions, to |
|
|
|
|
receive metric reports. The test framework will fill in the reference with the |
|
|
|
|
latest load report from the report listener. This way, together with server |
|
|
|
|
behaviors we can verify the expected metric reports are received. |
|
|
|
|
|
|
|
|
|
Server features: |
|
|
|
|
* [UnaryCall][] |
|
|
|
|
* [FullDuplexCall][] |
|
|
|
|
* [Backend Metrics Report][] |
|
|
|
|
|
|
|
|
|
Procedures: |
|
|
|
|
* Client sends a unary call to the server. The call request sets `orca_oob_report` to a test load report. |
|
|
|
|
1. Client starts a full duplex call and sends: |
|
|
|
|
``` |
|
|
|
|
{ |
|
|
|
|
orca_oob_report:{ |
|
|
|
@ -1141,13 +1147,18 @@ Procedures: |
|
|
|
|
util: 0.30499 |
|
|
|
|
} |
|
|
|
|
} |
|
|
|
|
response_parameters:{ |
|
|
|
|
size: 1 |
|
|
|
|
} |
|
|
|
|
} |
|
|
|
|
``` |
|
|
|
|
The call carries a reference to receive the load report, e.g. using CallOptions. |
|
|
|
|
The reference will be passed to the custom LB policy as part of the `OrcaOobReportListener` API. |
|
|
|
|
* Client asserts that, after 1.5 second, the latest OOB load report received is equal to the test load report. |
|
|
|
|
* Client sends another unary call to the server. The call request sets `orca_oob_report` to a |
|
|
|
|
different test load report. |
|
|
|
|
2. After getting a response, client waits up to 5 seconds to receive a OOB load |
|
|
|
|
report that matches the requested load report in step 1. To wait for load |
|
|
|
|
report, client may inject a callback to the custom LB policy, or poll the result |
|
|
|
|
by doing empty unary call that carries a reference, e.g. using |
|
|
|
|
CallOptions, that will be filled in by the custom LB policy as part of the |
|
|
|
|
`OrcaOobReportListener` API. |
|
|
|
|
3. Then client sends: |
|
|
|
|
``` |
|
|
|
|
{ |
|
|
|
|
orca_oob_report:{ |
|
|
|
@ -1157,10 +1168,14 @@ different test load report. |
|
|
|
|
util: 100.2039 |
|
|
|
|
} |
|
|
|
|
} |
|
|
|
|
response_parameters:{ |
|
|
|
|
size: 1 |
|
|
|
|
} |
|
|
|
|
} |
|
|
|
|
``` |
|
|
|
|
The call still carries a reference to receive the load report. |
|
|
|
|
* Client asserts that, after 1.5 second, the latest OOB load report received is equal to the new test load report. |
|
|
|
|
4. After getting a response, client waits up to 5 seconds to receive a OOB load |
|
|
|
|
report that matches the requested load report in step 3. Similar to step 2. |
|
|
|
|
5. Client half closes the stream, and asserts the streaming call is successful. |
|
|
|
|
|
|
|
|
|
### Experimental Tests |
|
|
|
|
|
|
|
|
@ -1391,22 +1406,20 @@ fully communicate metadata. |
|
|
|
|
### Backend metrics report |
|
|
|
|
[Backend Metrics Report]: #backend-metrics-report |
|
|
|
|
|
|
|
|
|
Server reports backend metrics data in both per-query and out-of-band cases, with metrics data |
|
|
|
|
indicated from the unary call request. |
|
|
|
|
|
|
|
|
|
Using ORCA APIs we install the per-query metrics reporting server interceptor, so that it can attach |
|
|
|
|
metrics per RPC. |
|
|
|
|
Also using ORCA APIs we register the `OpenRCAService` implementation to the server, so that it can |
|
|
|
|
report metrics periodically. The minimum report interval in the ORCA service is set to 1 sec. |
|
|
|
|
|
|
|
|
|
During test, the server will receive unary requests from the client that each may contain up to two |
|
|
|
|
test load report, indicating whether it needs to update metrics for the current call or at the OOB server. |
|
|
|
|
Then the server sets the metrics data, echoing the test load report it just received. |
|
|
|
|
Specifically: |
|
|
|
|
1. If the `orca_per_rpc_report` is set in the request, the server test driver will call |
|
|
|
|
`CallMetricRecorder` to record both utilization and |
|
|
|
|
request cost metrics for the current RPC, the metrics data is coped from the received test load |
|
|
|
|
report from the request. |
|
|
|
|
2. And if the `orca_oob_report` is set in the request, the server test driver will call |
|
|
|
|
`MetricRecorder` to first clear all the previous metrics data, and then record the utilization |
|
|
|
|
metrics, the metrics data is coped from the received test load report from the request. |
|
|
|
|
Server reports backend metrics data in both per-query and out-of-band cases, |
|
|
|
|
echoing metrics data from the unary or fullDuplex call. |
|
|
|
|
|
|
|
|
|
Using ORCA APIs we install the per-query metrics reporting server interceptor, |
|
|
|
|
so that it can attach metrics per RPC. We also register the `OpenRCAService` |
|
|
|
|
implementation to the server, so that it can report metrics periodically. |
|
|
|
|
The minimum report interval in the ORCA service is set to 1 sec. |
|
|
|
|
|
|
|
|
|
If `SimpleRequest.orca_per_rpc_report` is set in unary call, the server will add |
|
|
|
|
the metric data from `orca_per_rpc_report` to the RPC using the language's |
|
|
|
|
CallMetricRecorder. |
|
|
|
|
|
|
|
|
|
If `SimpleRequest.orca_oob_report` is set in fullDuplexCall call, the server |
|
|
|
|
will first clear all the previous metrics data, and then add utilization metrics |
|
|
|
|
from `orca_oob_report` to the `OpenRCAService`. |
|
|
|
|
The server implementation should use a lock or similar mechanism to allow only |
|
|
|
|
one client to control the server's out-of-band reports until the end of the RPC. |
|
|
|
|