[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36703
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36703 from tanvi-jagtap:test_core_gpr_log_01 26c4307b08
PiperOrigin-RevId: 636801504
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - BUILD
In this CL we are just editing the build and bzl files to add dependencies.
This is done to prevent merge conflict and constantly having to re-make the make files using generate_projects.sh for each set of changes.
Closes#36604
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36604 from tanvi-jagtap:build_test_core_misc_01 8995ba4914
PiperOrigin-RevId: 633519619
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36595
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36595 from tanvi-jagtap:regex_test_core_iomgr_transport b0836fda1c
PiperOrigin-RevId: 633456839
Notes:
* The special `on_write` callback was never used, all slices were discarded. I removed that functionality.
Closes#36513
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36513 from drfloob:rewrite-mock-endpoint-to-ee e45a964633
PiperOrigin-RevId: 631187792
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with absl CHECK
These changes have been made using string replacement and regex.
Will not be replacing all instances of CHECK with CHECK_EQ , CHECK_NE etc because there are too many callsites. Only ones which are doable using very simple regex with least chance of failure will be replaced.
Given that we have 5000+ instances of GPR_ASSERT to edit, Doing it manually is too much work for both the author and reviewer.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36408
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36408 from tanvi-jagtap:tjagtap_core_security f9eaf45140
PiperOrigin-RevId: 627264425
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36366
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36366 from tanvi-jagtap:tjagtap_test_01 a74c57ea7e
PiperOrigin-RevId: 625963461
Internally, use `std::vector` instead of `ChunkedVector` to hold extra metadatum.
I'm not totally convinced this is the right move, so it's going to be a try it and monitor for a month or so thing... I might roll back if performance is actually affected (but I think we'll see some wins and losses and overall about a wash).
Closes#36118
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36118 from ctiller:YUPYUPYUP 68e0acd0a2
PiperOrigin-RevId: 620902195
Redacting unknown metadata types in debug logs.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36006
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36006 from tanvi-jagtap:tjagtap_redact_01 94c5738bfe
PiperOrigin-RevId: 611334483
This PR adds the Authority Key Identifier to CertificateInfo. This value _can be_ important in finding the right CRLs to use if there are Issuer name overlaps or a more complicated CA setup with multiple signing keys.
We should observe no behavior change in our `CrlProvider` implementations, this is just adding an important field for users who implement it themselves.
Closes#35931
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35931 from gtcooke94:AkidCheck dd048a53b6
PiperOrigin-RevId: 611143198
This PR is copied from #34276, since I did not have permissions to add commits to it. That PR has been verified to work (see the top-level description). This PR just makes the gRPC tests pass (e.g. adding includes, clang formatting).
Closes#34874
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34874 from matthewstevenson88:pull_34276 d5fb73e5b2
PiperOrigin-RevId: 609107146
…p and add some verification helpers (#35641)"
This reverts commit 310770d61d.
It breaks all portability tests.
```
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CrlUtils
[ RUN ] CrlUtils.HasCrlSignBitExists
/[var/local/git/grpc/test/core/tsi/ssl_transport_security_utils_test.cc:566](https://cs.corp.google.com/piper///depot/google3/var/local/git/grpc/test/core/tsi/ssl_transport_security_utils_test.cc?l=566): Failure
Value of: HasCrlSignBit(root_ca_)
Actual: false
Expected: true
[ FAILED ] CrlUtils.HasCrlSignBitExists (0 ms)
[----------] 1 test from CrlUtils (0 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] CrlUtils.HasCrlSignBitExists
```
Closes#35962
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35962 from drfloob:revert-310770d61d3b25d358d877a074bebf6ae85549d0 e6a2f03309
PiperOrigin-RevId: 609075479
This PR does 2 distinct things, I can unbundle them if desired
1) Add functions in `ssl_transport_security_utils` and associated tests that will eventually be used for additional Crl validation (the logic of actually doing this will be in a future PR), so other than the tests these fns are currently unused.
2) Remove the use of `X509_NAME_oneline` - it is not a guaranteed stable way to get the issuer name for lookups. Instead, use the DER encoding via `i2d_X509_NAME` - the results in a non-human readable string that is stable for lookup, and necessitated some change to the CrlProvider test code that previously used a human readable string for this value.
Neither should result in behavior changes.
Closes#35641
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35641 from gtcooke94:CrlRefactor 2b6f63717c
PiperOrigin-RevId: 607701254
The Oauth2 credentials currently rely on pollsets and polling entities, partially because grpc_core::HttpRequest relies on them as well. This is a temporary solution to get both iomgr and EventEngine clients to pass the oauth2 interop tests.
Tested with the client experiment on and off.
Closes#35866
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35866 from drfloob:oauth2_utils_work_with_ee_hack 91eea0c196
PiperOrigin-RevId: 605433817
This fixes#21619. This experimental ALPN protocol has already been removed from the other gRPC stacks.
Closes#34876
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34876 from matthewstevenson88:remove-grpc-exp 1cb9d084ea
PiperOrigin-RevId: 592080195
This adds the directory reloader implementation of the CrlProvider. This
will periodically reload CRL files in a directory per [gRFC
A69](https://github.com/grpc/proposal/pull/382)
Included in this is the following:
* A public API to create the `DirectoryReloaderCrlProvider`
* A basic directory interface in gprpp and platform specific impls for
getting the list of files in a directory (unfortunately prior C++17,
there is no std::filesystem, so we have to have platform specific impls)
* The implementation of `DirectoryReloaderCrlProvider` takes an
event_engine and a directory interface. This allows us to test using the
fuzzing event engine for time mocking, and to implement a test directory
interface so we avoid having to make temporary directories and files in
the tests. This is notably not in `include`, and the
`CreateDirectoryReloaderCrlProvider` is the only way to construct one
from the public API, so we don't expose the event engine and directory
details to the user.
---------
Co-authored-by: gtcooke94 <gtcooke94@users.noreply.github.com>
Address https://github.com/grpc/grpc/issues/12554
The API for `duplicate_without_call_credentials` says
```
// Creates a version of the channel credentials without any attached call
// credentials. This can be used in order to open a channel to a non-trusted
// gRPC load balancer.
```
As the impl stands right now, because of that description, in the case
of layered composite creds, I think the right behavior would be to call
down until you get the base cred with no call cred.
In discussing with the team, we do wonder if the use-case of layered
composite creds is really something that should be a feature, or if we
should be checking during the creation of composite creds to make sure
we aren't layering composite creds? @markdroth can you give your
thoughts?
The basic APIs for the CRL Reloading features.
This adds external types to represent CRL Providers, CRLs, and
CertificateInfo.
Internally we will use `CrlImpl` - this layer is needed to hide OpenSSL
details from the user.
GRFC - https://github.com/grpc/proposal/pull/382
Things Done
* Add external API for `CrlProvider`, `Crl`, `CertInfo` (`CertInfo` is
used during CRL lookup rather than passing the entire certificate).
* Add code paths in `ssl_transport_security` to utilize CRL providers
* Add `StaticCrlProvider`
* Refactor `crl_ssl_transport_security_test.cc` so it is more extensible
and can be used with providers
Update from gtcooke94:
This PR adds support to build gRPC and it's tests with OpenSSL3. There were some
hiccups with tests as the tests with openssl haven't been built or exercised in a
few months, so they needed some work to fix.
Right now I expect all test files to pass except the following:
- h2_ssl_cert_test
- ssl_transport_security_utils_test
I confirmed locally that these tests fail with OpenSSL 1.1.1 as well,
thus we are at least not introducing regressions. Thus, I've added compiler directives around these tests so they only build when using BoringSSL.
---------
Co-authored-by: Gregory Cooke <gregorycooke@google.com>
Co-authored-by: Esun Kim <veblush@google.com>
Adds access token lifetime configuration for workload identity
federation with service account impersonation for both explicit and
implicit flows.
Changes:
1. Adds a new member "service_account_impersonation" to the
ExternalAccountCredentials class. "token_lifetime_seconds" is a member
of "service_account_impersonation".
2. Adds validation checks, like token_lifetime_seconds should be between
the minimum and maximum accepted value, during the creation of an
ExternalAccountCredentials object.
3. Appends "lifetime" to the body of the service account impersonation
request.
Tests:
1. Modifies a test to check if the default value is passed when
"service_account_impersonation" is empty.
2. Adds tests to check if the token_lifetime_seconds value is propagated
to the request body.
3. Adds tests to verify that an error is thrown when
token_lifetime_seconds is invalid.
This PR does the following: for the TLS server credentials, stops
calling `SSL_CTX_set_client_CA_list` by default in
`ssl_transport_security.cc`, and gives users a knob to re-enable calling
this API.
## What does the `SSL_CTX_set_client_CA_list` API do?
When this API is called, a gRPC TLS server sends the following data in
the ServerHello: for each certificate in the server's trust bundle, the
CA name in the certificate.
This API does not change the set of certificates trusted by the server
in any way. Rather, it is just providing a hint to the client about what
client certificate should be sent to the server.
## Why are we removing the use of `SSL_CTX_set_client_CA_list` by
default for the TLS server credentials?
Removing the use of this API by default has 2 benefits:
1. Calling this API makes gRPC TLS unusable for servers with a
sufficiently large trust bundle. Indeed, if the server trust bundle is
too large, then the server will always fail to build the ServerHello.
2. Calling this API is introducing a huge amount of overhead (1000s of
bytes) to each ServerHello, so removing this feature will improve
connection establishment latency for all users of the TLS server
credentials.
- switch to json_object_loader for config parsing
- use `absl::string_view` instead of `const char*` for cert provider
names
- change cert provider registry to use a map instead of a vector
- remove unused mesh_ca cert provider factory
I generated a new client key and cert where a Spiffe ID is added as the
URI SAN. As such, we are able to test the audit log contains the
principal correctly.
Update: I switched to use the test logger to verify the log content and
removed stdout logger here because one the failure of [RBE Windows Debug
C/C++](https://source.cloud.google.com/results/invocations/c3187f41-bb1f-44b3-b2b1-23f38e47386d).
Update again: Refactored the test logger in a util such that the authz
engine test also uses the same logger. Subsequently, xDS e2e test will
also use it.
---------
Co-authored-by: rockspore <rockspore@users.noreply.github.com>
The logger uses `absl::FPrintF` to write to stdout. After reading a
number of sources online, I got the impression that `std::fwrite` which
is used by `absl::FPrintF` is atomic so there is no locking required
here.
---------
Co-authored-by: rockspore <rockspore@users.noreply.github.com>
To fix this error
```
test/core/security/grpc_authorization_engine_test.cc:88:32: error: unknown type name 'Json'; did you mean 'experimental::Json'?
ParseAuditLoggerConfig(const Json&) override {
^~~~
experimental::Json
```
This makes the JSON API visible as part of the C-core API, but in the
`experimental` namespace. It will be used as part of various
experimental APIs that we will be introducing in the near future, such
as the audit logging API.
1. `GrpcAuthorizationEngine` creates the logger from the given config in
its ctor.
2. `Evaluate()` invokes audit logging when needed.
---------
Co-authored-by: rockspore <rockspore@users.noreply.github.com>
Tanvi Jagtap
Mark D. Roth
AJ Heller
Craig Tiller
Tanvi Jagtap
Gregory Cooke
Matthew Stevenson
Esun Kim
Luwei Ge
aeitzman
apolcyn
jrandolf
Mario Jones Vimal
Matthew Stevenson
Eugene Ostroukhov