This change adds a mechanism to exclude languages in OSS benchmarks with a one-line change, when necessary.
Currently dotnet is excluded from the benchmarks (https://github.com/grpc/grpc/pull/36759).
This change also formats benchmarking shell scripts to simplify future changes.
Closes#37172
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37172 from paulosjca:languages 4057659c84
PiperOrigin-RevId: 650303856
Pyhton timeout was already set to 240min:
3e8cdb9f39/tools/internal_ci/linux/psm-csm-python.cfg (L19)
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37154
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37154 from XuanWang-Amos:increase_csm_timeout f5caabc06a
PiperOrigin-RevId: 649187094
This moves more code out of the monolithic `grpc_xds_client` BUILD target. We still need more work to split it up completely, but this is a nice step in the right direction -- and it unblocks a subsequent PR that I'm working on for xDS authority rewriting.
Closes#37130
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37130 from markdroth:xds_resource_type_build_refactoring a021d9773c
PiperOrigin-RevId: 648742472
Since these were disabled they stopped working, and we really need to be tracking overheads here.
Closes#37077
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37077 from ctiller:it-must-work aa19a4aa89
PiperOrigin-RevId: 648469428
Move event engine context to be an arena based context, and have party host arena directly.
Now `Party` can directly inject all context necessary without needing to call out to the derived type; this makes `Party` instantiable by itself (no need to inherit from it - though we'll still want to for `CallSpine` to simplify some arena lifetime management), meaning we can start to consider eliminating the base class `Activity` and merging these types.
Closes#37015
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37015 from ctiller:chaotic-party 7f44b37b5d
PiperOrigin-RevId: 646963003
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36942
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36942 from tanvi-jagtap:src_core_lb 524b45fef1
PiperOrigin-RevId: 646311061
[Gpr_To_Absl_Logging] Remove gpr_log_severity_string
This function is not getting used anymore gpr_log_severity_string
Closes#37013
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37013 from tanvi-jagtap:gpr_log_severity_string_remove 3346face4c
PiperOrigin-RevId: 645796077
As we've learned what configuration is needed for our benchmarks the settings have been growing more and more bespoke for each binary. Try to consolidate that into some useful defaults.
Also ensure we always `linkstatic=1`. `cc_binary` defaults to this, so it's reasonable to assume that's the performance our customers see. It also deeply impacts performance for small microbenchmarks, and so enabling it gives us more apples:apples, and saves chasing things that don't matter.
Closes#37012
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37012 from ctiller:benchmark-bin b84cab58c6
PiperOrigin-RevId: 645483183
Previously, metadata mutations were made by the picker directly, which meant that they would be applied even if the channel winds up discarding the pick due to the returned subchannel having been disconnected by the time the pick result is returned. This changes the API such that pickers return metadata mutations along with the pick result, so that the mutations won't get applied unless the pick result is actually used.
Closes#36968
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36968 from markdroth:lb_metadata_api 2765da6121
PiperOrigin-RevId: 645451869
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36979
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36979 from tanvi-jagtap:src_core_util a160d9d7ec
PiperOrigin-RevId: 645397740
[Gpr_To_Absl_Logging] Cleaning up the allow list for deprecated functions.
This file has a list of deprecated functions.
Any new instance of a deprecated function being used in the code will be flagged by the script. If there is a new instance of a deprecated function in a Pull Request, then the Sanity tests will fail.
We are currently working on clearing out the usage of deprecated functions in our code base.
For that reason, while our cleaning is in progress we have a temporary allow list. The allow list has a list of files where clean up of deprecated functions is pending. As we clean up the deprecated function from files , we will remove them from the allow list.
Ideally, it would be wise to do the file clean up and the altering of the allow list int the same PR. This will make sure that any roll back of a clean up PR will alter the allow list and avoid build failures.
Closes#36977
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36977 from tanvi-jagtap:clean_allow_list_2024_06_19 9393244cee
PiperOrigin-RevId: 645325296
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36883
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36883 from tanvi-jagtap:large_file_01 43eb4880cf
PiperOrigin-RevId: 645281490
[Gpr_To_Absl_Logging] Removing incorrect entry from Sanity test
For now - we have decided to continue to keep this function.
So I am removing it from this list.
Closes#36987
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36987 from tanvi-jagtap:gpr_log_verbosity_init_removal cbfb969f34
PiperOrigin-RevId: 645274187
These are substitutes for language features and never warrant a new function being created. Significantly boosts the benchmarks in #36946Closes#36981
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36981 from ctiller:seqqy 1a075be7e9
PiperOrigin-RevId: 645169296
[Gpr_To_Absl_Logging] Remove custom logger from php. Not supported anymore.
While I am writing a custom absl LogSink to fix the issue in the next Pull Request,
I wanted to know if not having this LogSink is a release blocker or not.
Closes#36962
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36962 from tanvi-jagtap:fix_php_logging 1374cbc838
PiperOrigin-RevId: 645056513
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36980
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36980 from tanvi-jagtap:src_core_lib_promise_filter 19ed6234f4
PiperOrigin-RevId: 645050090
Preparation for switching away from `grpc_channel_filter*` to identify channel filters.
Closes#36907
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36907 from ctiller:type-name e7ad4c67a2
PiperOrigin-RevId: 644483948
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36963
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36963 from XuanWang-Amos:add_csm_to_verify_python_release 3e64130771
PiperOrigin-RevId: 644470995
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36941
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36941 from tanvi-jagtap:src_core_xds 9ab8de800d
PiperOrigin-RevId: 644229971
There's something new in the works, so it's time that this unmaintained & broken system got garbage collected.
Closes#36952
PiperOrigin-RevId: 644184198
Do not set `absl::MinLogLevel` if user does not set `GRPC_VERBOSITY`. Only set `absl::MinLogLevel` if the user has specified `GRPC_VERBOSITY`.
Background:
gRPC Core overrides the setting of `absl::MinLogLevel` based on the setting of `GRPC_VERBOSITY`.
`GRPC_VERBOSITY` defaults to the setting of `GPR_DEFAULT_LOG_VERBOSITY_STRING` if that env var is not set, resulting in the consequence that gRPC Core would override the setting of `absl::MinLogLevel` even if the user had previously modified it.
Note that even with this change, the minimum log severity that's printed remains at `ERROR` by default.
Closes#36931
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36931 from yashykt:ChangeDefaultGrpcVerbosity ee92404c35
PiperOrigin-RevId: 644127489
Ensure arena accounting is working, and add a test that a constant call size results in a constant call size estimate.
Closes#36933
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36933 from ctiller:arena-accounts 116c805633
PiperOrigin-RevId: 644102412
This change brings up the direct channel, and inproc promise based transports.
This work exposed a bug that was very difficult to fix with the current call_filters.cc implementation, so I've substantially revamped that - instead of having a pipe-like object per call element, we now have a big ol' combined state machine for the entire call. It's a touch more code, but substantially easier to reason about individual cases, so I much prefer this form (it's also a slight memory improvement: 12 bytes total to track call state, and 10 of those are wakeup bitmasks...).
Closes#36734
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36734 from ctiller:transport-refs-9 3e2a80b40d
PiperOrigin-RevId: 644034593
Paulo Castello da Costa
Tanvi Jagtap
Xuan Wang
Tanvi Jagtap
Eugene Ostroukhov
Mark D. Roth
Craig Tiller
Esun Kim
Easwar Swaminathan
Yash Tibrewal
AJ Heller