|
|
|
@ -1056,6 +1056,102 @@ included in that latency measurement (channel teardown semantics differ widely |
|
|
|
|
between languages). This latency measurement should also be the value that is |
|
|
|
|
logged and recorded in the latency histogram. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### orca_per_rpc |
|
|
|
|
[orca_per_rpc]: #orca_per_rpc |
|
|
|
|
|
|
|
|
|
The client verifies that a custom LB policy, which is integrated with ORCA APIs, will receive |
|
|
|
|
per-query metric reports from the backend. |
|
|
|
|
|
|
|
|
|
The client will register the custom LB policy named `test_backend_metrics_load_balancer`, which |
|
|
|
|
using ORCA APIs already installed a per-query report listener. The interop-testing client will run with a |
|
|
|
|
service config to select the load balancing config (using argument `--service_config_json`), so that |
|
|
|
|
it effectively uses this newly registered custom LB policy. A load report reference can be passed |
|
|
|
|
from the call to the LB policy through, e.g. CallOptions, to receive metric reports. |
|
|
|
|
The LB policy will fill in the reference with the latest load report from the report listener. |
|
|
|
|
This way, together with server behaviors we can verify the expected metric reports are received. |
|
|
|
|
|
|
|
|
|
Server features: |
|
|
|
|
* [UnaryCall][] |
|
|
|
|
* [Backend Metrics Report][] |
|
|
|
|
|
|
|
|
|
Procedures: |
|
|
|
|
* The client sends a unary request to the server. The call request sets `orca_per_rpc_report` to a |
|
|
|
|
test load report. |
|
|
|
|
``` |
|
|
|
|
{ |
|
|
|
|
orca_per_rpc_report:{ |
|
|
|
|
cpu_utilization: 0.8210 |
|
|
|
|
memory_utilization: 0.5847 |
|
|
|
|
request_cost: { |
|
|
|
|
cost: 3456.32 |
|
|
|
|
} |
|
|
|
|
utilization: { |
|
|
|
|
util: 0.30499 |
|
|
|
|
} |
|
|
|
|
} |
|
|
|
|
} |
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
The call carries a reference to receive the load report, e.g. using CallOptions. |
|
|
|
|
The reference is passed to the custom LB policy as part of the `OrcaPerRequestReportListener` API. |
|
|
|
|
|
|
|
|
|
Client asserts: |
|
|
|
|
* The call is successful. |
|
|
|
|
* The per-query load report reference contains a metrics report that is identical to the metrics |
|
|
|
|
data sent in the request shown above. |
|
|
|
|
|
|
|
|
|
### orca_oob |
|
|
|
|
|
|
|
|
|
The client verifies that a custom LB policy, which is integrated with ORCA APIs, will receive |
|
|
|
|
out-of-band metric reports from the backend. |
|
|
|
|
|
|
|
|
|
The client will register the custom LB policy named `test_backend_metrics_load_balancer`. It has |
|
|
|
|
similar and additional functions as described in the [orca_per_rpc][] test. |
|
|
|
|
We use ORCA APIs to install an out-of-band report listener (configure load report interval to be 1s) |
|
|
|
|
in the LB policy. The interop-testing client will run with a service config to select the load |
|
|
|
|
balancing config(using argument `--service_config_json`), so that it effectively uses this newly |
|
|
|
|
registered custom LB policy. A load report reference can be passed from the call to the LB policy |
|
|
|
|
through, e.g. CallOptions, to receive metric reports. |
|
|
|
|
The test framework will fill in the reference with the latest load report from the report listener. |
|
|
|
|
This way, together with server behaviors we can verify the expected metric reports are received. |
|
|
|
|
|
|
|
|
|
Server features: |
|
|
|
|
* [UnaryCall][] |
|
|
|
|
* [Backend Metrics Report][] |
|
|
|
|
|
|
|
|
|
Procedures: |
|
|
|
|
* Client sends a unary call to the server. The call request sets `orca_oob_report` to a test load report. |
|
|
|
|
``` |
|
|
|
|
{ |
|
|
|
|
orca_oob_report:{ |
|
|
|
|
cpu_utilization: 0.8210 |
|
|
|
|
memory_utilization: 0.5847 |
|
|
|
|
utilization: { |
|
|
|
|
util: 0.30499 |
|
|
|
|
} |
|
|
|
|
} |
|
|
|
|
} |
|
|
|
|
``` |
|
|
|
|
The call carries a reference to receive the load report, e.g. using CallOptions. |
|
|
|
|
The reference will be passed to the custom LB policy as part of the `OrcaOobReportListener` API. |
|
|
|
|
* Client asserts that, after 1 second, the latest OOB load report received is equal to the test load report. |
|
|
|
|
* Client sends another unary call to the server. The call request sets `orca_oob_report` to a |
|
|
|
|
different test load report. |
|
|
|
|
``` |
|
|
|
|
{ |
|
|
|
|
orca_oob_report:{ |
|
|
|
|
cpu_utilization: 0.29309 |
|
|
|
|
memory_utilization: 0.2 |
|
|
|
|
utilization: { |
|
|
|
|
util: 100.2039 |
|
|
|
|
} |
|
|
|
|
} |
|
|
|
|
} |
|
|
|
|
``` |
|
|
|
|
The call still carries a reference to receive the load report. |
|
|
|
|
* Client asserts that, after 1 second, the latest OOB load report received is equal to the new test load report. |
|
|
|
|
|
|
|
|
|
### Experimental Tests |
|
|
|
|
|
|
|
|
|
These tests are not yet standardized, and are not yet implemented in all |
|
|
|
@ -1281,3 +1377,26 @@ Discussion: |
|
|
|
|
Ideally, this would be communicated via metadata and not in the |
|
|
|
|
request/response, but we want to use this test in code paths that don't yet |
|
|
|
|
fully communicate metadata. |
|
|
|
|
|
|
|
|
|
### Backend metrics report |
|
|
|
|
[Backend Metrics Report]: #backend-metrics-report |
|
|
|
|
|
|
|
|
|
Server reports backend metrics data in both per-query and out-of-band cases, with metrics data |
|
|
|
|
indicated from the unary call request. |
|
|
|
|
|
|
|
|
|
Using ORCA APIs we install the per-query metrics reporting server interceptor, so that it can attach |
|
|
|
|
metrics per RPC. |
|
|
|
|
Also using ORCA APIs we register the `OpenRCAService` implementation to the server, so that it can |
|
|
|
|
report metrics periodically. The minimum report interval in the ORCA service is set to 1 sec. |
|
|
|
|
|
|
|
|
|
During test, the server will receive unary requests from the client that each may contain up to two |
|
|
|
|
test load report, indicating whether it needs to update metrics for the current call or at the OOB server. |
|
|
|
|
Then the server sets the metrics data, echoing the test load report it just received. |
|
|
|
|
Specifically: |
|
|
|
|
1. If the `orca_per_rpc_report` is set in the request, the server test driver will call |
|
|
|
|
`CallMetricRecorder` to record both utilization and |
|
|
|
|
request cost metrics for the current RPC, the metrics data is coped from the received test load |
|
|
|
|
report from the request. |
|
|
|
|
2. And if the `orca_oob_report` is set in the request, the server test driver will call |
|
|
|
|
`MetricRecorder` to first clear all the previous metrics data, and then record the utilization |
|
|
|
|
metrics, the metrics data is coped from the received test load report from the request. |
|
|
|
|