[Gpr_To_Absl_Logging] Remove gpr_log_severity_string
This function is not getting used anymore gpr_log_severity_string
Closes#37013
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37013 from tanvi-jagtap:gpr_log_severity_string_remove 3346face4c
PiperOrigin-RevId: 645796077
This commit implements `logger=` for GRPC module. It will allow us to inject a custom logger on the fly easily. Using `self.extend` inside this setter will help us deal with backward compatible for some custom Logger Module.
<!--
Your pull request will be routed to the following person by default for triaging.
If you know who should review your pull request, please remove the mentioning below.
-->
Fixed#24084
cc @sampajano , @apolcyn and @alto-ruby TIA
re: https://github.com/grpc/grpc/pull/24072Closes#36989
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36989 from nvh0412:chore/add-logger-setter-for-GRPC-ruby-2 f5a91225e7
PiperOrigin-RevId: 645189146
### Problem 1
Context :
gpr_log() internally calls gpr_log_message() .
gpr_log_message() may either call gpr_default_log() or a user provided logging function.
gpr_default_log() uses absl LOG. This is the default. Most users will log this way.
For the small percentage of users who have customized the logging function, gpr_log will log to custom this function.
Problem :
We have converted half the instances of gpr_log to absl LOG().
For users who use the defaults - there will be no issue.
For the users who use a customized logging function
1. All the absl LOGs will log to the absl log sink.
2. All the gpr_log statements will log via this user provided function.
This is in-consistent behaviour and will cause confusion and difficulty in debugging.
Solution:
All logs should go to the same sink.
So we decided to make gpr_set_log_function a no op in this release.
The function will be deleted in the next release.
https://github.com/grpc/proposal/pull/425
### Problem 2
Context :
gpr_should_log is used to avoid computing expensive stuff for logging if the log is not going to be visible.
Problem :
gpr_should_log was referencing the GRPC_VERBOSITY flag and values set by gpr_set_log_verbosity .
However, actual logging happens based on the absl settings.
This is incorrect. Because if the old settings are not honoured, they should not be checked and no decision in code should be made based on settings which are not going to get used.
Solution :
Given the above changes in Problem 1, since all custom logging is disabled, all logging from gRPC with honour the absl LOG settings. Hence we modified the gpr_should_log function to refer to absl settings.
### Problem 3
We still have the issue of php using a custom log sink. We will address this in a separate PR.
### Problem 4
Tests related to test/core/end2end/tests/no_logging.cc are broken . These will be fixed in another PR.
Closes#36961
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36961 from tanvi-jagtap:fix_gpr_should_log 70c3224af1
PiperOrigin-RevId: 645096418
Fixes the CBF of `src/ruby/end2end/killed_client_thread_test.rb` (failure mode is a hang of the child process that receives the SIGTERM) that has been happening since https://github.com/grpc/grpc/pull/36724
So far grpc-ruby CQ pluck operations have so far used a 20ms-interval busy poll to check interrupts in case we've received a signal, handle process shutdown, etc. This means ongoing RPCs will not terminate their CQ operations if we need to terminate the process (the loop simply exits without waiting for the CQ op to finish), causing a leak. Those RPCs can leave refs over their corresponding channels preventing [this](8564f72e8e/src/ruby/ext/grpc/rb_channel.c (L653)) from terminating (the channels don't reach state SHUTDOWN after being destroyed).
Fix is to unblock CQ pluck operations by cancelling calls, and thus allowing the CQ pluck to actually complete its operation. For server listening CQ operations, we unblock them by shutting down the server.
A side win here is to remove the [20ms-interval busy poll](8564f72e8e/src/ruby/ext/grpc/rb_completion_queue.c (L44)) on CQ operations, which was only needed to handle shutdown.
Closes#36903
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36903 from apolcyn:fix_ruby_interrupt bed1ee2feb
PiperOrigin-RevId: 643046465
Change was created by the release automation script. See go/grpc-release.
Closes#36892
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36892 from XuanWang-Amos:bump_dev_version_202406112301 9b2898d716
PiperOrigin-RevId: 643027546
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with c assert.
These changes have been made using string replacement and regex.
Closes#36817
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36817 from tanvi-jagtap:gpr_assert_removal_in_php 2222f95413
PiperOrigin-RevId: 642855768
[Gpr_To_Absl_Logging] Move function to test header form log.h
This is not really needed in log.h
Closes#36860
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36860 from tanvi-jagtap:move_function_to_test_header e6494bd06f
PiperOrigin-RevId: 642080756
Change was created by the release automation script. See go/grpc-release.
Closes#36544
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36544 from yashykt:bump_dev_version_202405061956 69ee5c869e
PiperOrigin-RevId: 631187829
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36333
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36333 from yijiem:bump_dev_version_202404101721 0a7cc5b4b7
PiperOrigin-RevId: 623878150
Ref: https://bugs.ruby-lang.org/issues/20311
C global variable that contain references to Ruby object MUST be declared to the Ruby GC to prevent these objects from being collected or moved.
There are a few exceptions to that, such as classes defined using the C APIs such as `rb_define_class` etc.
Up to Ruby 3.4 however, there was a bug that caused classes created from Ruby with the `Struct.new("Name")` API to also be marked as immortal and immovable.
GRPC has been relying on this bug, which I fixed in Ruby 3.4, and now GRPC is crashing when `Struct::Status` is moved around by the GC.
```
-- C level backtrace information -------------------------------------------
ruby(rb_print_backtrace+0x14) [0x5577db219d41] ruby-3.4.0/vm_dump.c:820
ruby(rb_vm_bugreport) ruby-3.4.0/vm_dump.c:1151
ruby(rb_bug_for_fatal_signal+0xfc) [0x5577db3cc60c] ruby-3.4.0/error.c:1066
ruby(sigsegv+0x4d) [0x5577db16358d] ruby-3.4.0/signal.c:926
libc.so.6(0x7f5bacd32520) [0x7f5bacd32520]
ruby(rb_class_superclass+0x32) [0x5577db0a9152] ruby-3.4.0/object.c:2239
ruby(struct_ivar_get+0x2a) [0x5577db194e02] ruby-3.4.0/struct.c:49
ruby(struct_ivar_get) ruby-3.4.0/struct.c:40
ruby(num_members) ruby-3.4.0/struct.c:705
ruby(rb_struct_new+0x56) [0x5577db19a9d6] ruby-3.4.0/struct.c:848
lib/grpc/grpc_c.so(grpc_run_batch_stack_build_result+0xe6) [0x7f5b84bb6b96] /tmp/bundle/ruby/3.4.0+0/bundler/gems/grpc-5ed33ee673e3/src/ruby/ext/grpc/rb_call.c:780
lib/grpc/grpc_c.so(grpc_rb_call_run_batch_try) /tmp/bundle/ruby/3.4.0+0/bundler/gems/grpc-5ed33ee673e3/src/ruby/ext/grpc/rb_call.c:839
ruby(rb_ensure+0x10c) [0x5577db007d3c] ruby-3.4.0/eval.c:1000
/tmp/bundle/ruby/3.4.0+0/bundler/gems/grpc-5ed33ee673e3/src/ruby/lib/grpc/grpc_c.so(grpc_rb_call_run_batch+0xca) [0x7f5b84bb595a] /tmp/bundle/ruby/3.4.0+0/bundler/gems/grpc-5ed33ee673e3/src/ruby/ext/grpc/rb_call.c:893
ruby(vm_call_cfunc_with_frame_+0x117) [0x5577db1f4c77] ruby-3.4.0/vm_insnhelper.c:3524
```
cc @apolcyn@peterzhu2118@k0kubunCloses#36125
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36125 from Shopify:fix-ruby-3.4-compat 7a127599c8
PiperOrigin-RevId: 616152904
In grpc v1.46.2 and later versions, #29155 caused the Ruby client to return `GRPC::Core::CallError`, a non-standard error, with a message: `grpc_call_start_batch failed with outstanding read or write present (code=8)`. However, the actual error should have been `GRPC::DeadlineExceeded`. This occurred because when `GRPC::DeadlineExceeded` is raised, the `@metadata_received` flag doesn't get flipped. When `receive_and_check_status` runs to determine
the error, the `RECV_INITIAL_METADATA` is erroneously sent again.
To avoid this, flip the flag in an `ensure` block whenever the `RECV_INITIAL_METADATA` op is set.
Closes#33283Closes#33565
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/33565 from stanhu:sh-ruby-fix-issue-33283 850889558c
PiperOrigin-RevId: 614088694
Change was created by the release automation script. See go/grpc-release.
Closes#35899
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35899 from eugeneo:bump_dev_version_202402131133 73950425c9
PiperOrigin-RevId: 606777850
As announced in https://discourse.llvm.org/t/clang-16-notice-of-potentially-breaking-changes/65562, clang 16 defaults `-Wincompatible-function-pointer-types` to be on. This causes a number of hard errors due to implicit conversions between `void *` and `void`, such as:
```
../../../../src/ruby/ext/grpc/rb_channel.c:770:47: error: incompatible function pointer types passing 'VALUE (VALUE)' (aka 'unsigned long (unsigned long)') to parameter of type 'VALUE (*)(void *)' (aka 'unsigned long (*)(void *)') [-Wincompatible-function-pointer-types]
g_channel_polling_thread = rb_thread_create(run_poll_channels_loop, NULL);
^~~~~~~~~~~~~~~~~~~~~~
/root/.rbenv/versions/3.1.4/include/ruby-3.1.0/ruby/internal/intern/thread.h:190:32: note: passing argument to parameter 'f' here
VALUE rb_thread_create(VALUE (*f)(void *g), void *g);
^
../../../../src/ruby/ext/grpc/rb_channel.c:780:41: warning: a function declaration without a prototype is deprecated in all versions of C [-Wstrict-prototypes]
void grpc_rb_channel_polling_thread_stop() {
^
void
../../../../src/ruby/ext/grpc/rb_channel.c:786:30: error: incompatible function pointer types passing 'void (void *)' to parameter of type 'void *(*)(void *)' [-Wincompatible-function-pointer-types]
rb_thread_call_without_gvl(run_poll_channels_loop_unblocking_func, NULL, NULL,
```
This commit fixes these pointer types using wrapper functions where necessary.
This issue was also raised on the FreeBSD port of the grpc gem: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271540
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#34481
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34481 from stanhu:sh-ruby-fix-clang-16 63c7424ad0
PiperOrigin-RevId: 600548467
`GPR_BACKWARDS_COMPATIBILITY_MODE` was devised to support old Linux kernel with old glibc but gRPC is now expected to support glibc 2.17 and later (2.17 is derived from the oldest but supported Linux distro, CentOS 7). Effectively, gRPC doesn't need `GPR_BACKWARDS_COMPATIBILITY_MODE` anymore. Hence, let's remove it.
gRPC Python and Ruby that distribute binary artifacts already began to use `manylinux` environment dealing with this issue in a better way so they don't need this flag anymore.
Closes#35602
PiperOrigin-RevId: 599895270
The `grpc_channel_args` is retained on the Ruby object and used for recreating the channel after forking in
grpc_rb_channel_maybe_recreate_channel_after_fork(). Previously, the key for each argument was taken from a Ruby string directly, which could be invalidated if the Ruby string is modified or moved by the GC. Duplicate the string for the key instead, so we own it.
Reproducer in https://github.com/grpc/grpc/issues/35489
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35488
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35488 from Shopify:key-uaf c1813cee01
PiperOrigin-RevId: 599304551
Change was created by the release automation script. See go/grpc-release.
Closes#35580
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35580 from stanley-cheung:bump_dev_version_202401171835 b363888ca5
PiperOrigin-RevId: 599267642
This commit upgrades gRPC to protobuf v25.0 and makes some fixes to
account for upb changes. One major change is that upb has been merged
into the protobuf repo, so we can now drop the separate `@upb`
dependency. Another is that `.upb.c` files no longer exist and there are
new `.upb_minitable.h` and `.upb_minitable.c` files. The longer
filenames exceeded a Windows restriction, so to work around that I
renamed the `upb-generated` directory to just `upb-gen`, and likewise
for `upbdefs-generated`.
Currently it is very easy to use the `TlsCredentialsOptions` in such a
way that it produces a memory leak. For example, the code block
```
{
TlsCredentialsOptions options;
}
```
produces a memory leak. This PR fixes up the ownership bugs in this
class and its `grpc_tls_credentials_options`, the C-core analogue.
Introduce `grpc-native-debug` gems containing debug symbol packages that
complement the shared libraries shipped in pre-compiled binary gems.
src/ruby/nativedebug/README.md has details on usage.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
https://github.com/grpc/grpc/pull/33538 added `-weak_framework
CoreFoundation` in `DLDFLAGS` for only `arm64-darwin` builds, but the
issue reported in https://github.com/grpc/grpc/issues/33483 can also
happen on `x86-darwin` builds. This can happen if:
1. The Ruby interpreter is compiled without
`-Wl,-undefined,dynamic_lookup`.
2. This happens if the Ruby interpreter is built with XCode 14.0 to 14.2
(https://bugs.ruby-lang.org/issues/19005).
Simplify the logic and always include `-weak_framework CoreFoundation`
for macOS builds.
grpc 1.57.0 crashes win ruby and alpine due to no `strdup` in musl libc.
This diff replace `strdup` with `grp_strdup`
```
Thread 1 "ruby" received signal SIGSEGV, Segmentation fault.
0x00000000000a4596 in ?? ()
(gdb) bt
#0 0x00000000000a4596 in ?? ()
#1 0x00007ffff14e298c in grpc_rb_channel_create_in_process_add_args_hash_cb (key=<optimized out>, val=<optimized out>, args_obj=<optimized out>) at rb_channel_args.c:84
#2 0x00007ffff7c2b9ea in hash_ar_foreach_iter (error=0, argp=140737488344784, value=<optimized out>, key=<optimized out>) at hash.c:1341
```
fixes#34044closes#27995
This was done manually due to a problem with
`tools/distrib/python/make_grpcio_tools.py`. ~I fixed it in this PR
(depends on cl/547979185), so there is a fair chance this upgrade will
work normally for the next release.~ The fix may be problematic for
upgrading protobuf on older release branches, so the improvement will be
worked on separately. CC @jtattermusch
This also updates the UPB dep to the latest commit on the 23.x branch.
So far, we've structured global C-core init/shutdown as follows:
1) every grpc-ruby object calls `grpc_init` when allocated, and
`grpc_shutdown` when finalized
2) grpc-ruby background threads are each wrapped in a
`grpc_init/shutdown` pair - for example see
b32d94de05/src/ruby/ext/grpc/rb_event_thread.c (L122)
and
b32d94de05/src/ruby/ext/grpc/rb_event_thread.c (L136)
But because ruby doesn't join ruby threads when the process is
terminating, the `init/shutdown` pairs in 2) are always left open. I.e.,
thus far we have never been invoking the final call to `grpc_shutdown`
which actually does C-core global shutdown. Thus our calls to
`grpc_shutdown` are useless.
So we might as well keep things simple and not even attempt to call
`grpc_shutdown`. Now we just call `grpc_init` before using C-core, and
then never even attempt global shutdown (in non-forking situations).
As a bonus, this fixes the issue with the event thread's
`grpc_ruby_init` racing with `prefork` that is mentioned in
https://github.com/grpc/grpc/pull/33666
Adds experimental fork support to gRPC/Ruby
Works towards https://github.com/grpc/grpc/issues/8798 (see caveats for why this wasn't marked fixed yet)
Works towards https://github.com/grpc/grpc/issues/33578 (see caveats for why this wasn't marked fixed yet)
This leverages existing `pthread_atfork` based C-core support for
forking that python/php use, but there's a bit extra involved mainly
because gRPC/Ruby has additional background threads.
New tests under `src/ruby/end2end` show example usage.
Based on https://github.com/grpc/grpc/pull/33495
Caveats:
- Bidi streams are not yet supported (bidi streams spawn background
threads which are not yet fork safe)
- Servers not supported
- Only linux supported
Remove `src/ruby/ext/grpc/objs/` and strip `src/ruby/ext/grpc/libs/` in
ruby build dir to save some space.
This can reduce the grpc gem size from more than 1G to about 70~80M.
This only affects opt build, dbg build remains the same.
A proper fix maybe use the same makefile to build c-core and the
extension rather than building c-core when generating the extension
makefile.
fixes: https://github.com/grpc/grpc/issues/33412
This PR does the following: for the TLS server credentials, stops
calling `SSL_CTX_set_client_CA_list` by default in
`ssl_transport_security.cc`, and gives users a knob to re-enable calling
this API.
## What does the `SSL_CTX_set_client_CA_list` API do?
When this API is called, a gRPC TLS server sends the following data in
the ServerHello: for each certificate in the server's trust bundle, the
CA name in the certificate.
This API does not change the set of certificates trusted by the server
in any way. Rather, it is just providing a hint to the client about what
client certificate should be sent to the server.
## Why are we removing the use of `SSL_CTX_set_client_CA_list` by
default for the TLS server credentials?
Removing the use of this API by default has 2 benefits:
1. Calling this API makes gRPC TLS unusable for servers with a
sufficiently large trust bundle. Indeed, if the server trust bundle is
too large, then the server will always fail to build the ServerHello.
2. Calling this API is introducing a huge amount of overhead (1000s of
bytes) to each ServerHello, so removing this feature will improve
connection establishment latency for all users of the TLS server
credentials.
Add -weak_framework CoreFoundation to the ruby extension link line on
arm64-darwin, to address "undefined symbols" issues from #33483
This is a variation on #33513 in response to feedback and so I've made
sure @alto-ruby is credited as co-author.
Closes#33483
Supersedes #33513
cc @apolcyn
Co-authored-by: alto-ruby <altorubys@gmail.com>
Tanvi Jagtap
apolcyn
Hoa Nguyen
Xuan Wang
Yash Tibrewal
AJ Heller
Yijie Ma
Jean byroot Boussier
Stan Hu
Esun Kim
Eugene Ostroukhov
Mark D. Roth
Alan Wu
Stanley Cheung
alto-ruby
Luwei Ge
Adam Cozzette
Matthew Stevenson
Richard Belleville
alto-ruby
Mike Dalessio