As announced in https://discourse.llvm.org/t/clang-16-notice-of-potentially-breaking-changes/65562, clang 16 defaults `-Wincompatible-function-pointer-types` to be on. This causes a number of hard errors due to implicit conversions between `void *` and `void`, such as:
```
../../../../src/ruby/ext/grpc/rb_channel.c:770:47: error: incompatible function pointer types passing 'VALUE (VALUE)' (aka 'unsigned long (unsigned long)') to parameter of type 'VALUE (*)(void *)' (aka 'unsigned long (*)(void *)') [-Wincompatible-function-pointer-types]
g_channel_polling_thread = rb_thread_create(run_poll_channels_loop, NULL);
^~~~~~~~~~~~~~~~~~~~~~
/root/.rbenv/versions/3.1.4/include/ruby-3.1.0/ruby/internal/intern/thread.h:190:32: note: passing argument to parameter 'f' here
VALUE rb_thread_create(VALUE (*f)(void *g), void *g);
^
../../../../src/ruby/ext/grpc/rb_channel.c:780:41: warning: a function declaration without a prototype is deprecated in all versions of C [-Wstrict-prototypes]
void grpc_rb_channel_polling_thread_stop() {
^
void
../../../../src/ruby/ext/grpc/rb_channel.c:786:30: error: incompatible function pointer types passing 'void (void *)' to parameter of type 'void *(*)(void *)' [-Wincompatible-function-pointer-types]
rb_thread_call_without_gvl(run_poll_channels_loop_unblocking_func, NULL, NULL,
```
This commit fixes these pointer types using wrapper functions where necessary.
This issue was also raised on the FreeBSD port of the grpc gem: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271540
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#34481
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34481 from stanhu:sh-ruby-fix-clang-16 63c7424ad0
PiperOrigin-RevId: 600548467
`GPR_BACKWARDS_COMPATIBILITY_MODE` was devised to support old Linux kernel with old glibc but gRPC is now expected to support glibc 2.17 and later (2.17 is derived from the oldest but supported Linux distro, CentOS 7). Effectively, gRPC doesn't need `GPR_BACKWARDS_COMPATIBILITY_MODE` anymore. Hence, let's remove it.
gRPC Python and Ruby that distribute binary artifacts already began to use `manylinux` environment dealing with this issue in a better way so they don't need this flag anymore.
Closes#35602
PiperOrigin-RevId: 599895270
The `grpc_channel_args` is retained on the Ruby object and used for recreating the channel after forking in
grpc_rb_channel_maybe_recreate_channel_after_fork(). Previously, the key for each argument was taken from a Ruby string directly, which could be invalidated if the Ruby string is modified or moved by the GC. Duplicate the string for the key instead, so we own it.
Reproducer in https://github.com/grpc/grpc/issues/35489
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35488
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35488 from Shopify:key-uaf c1813cee01
PiperOrigin-RevId: 599304551
Change was created by the release automation script. See go/grpc-release.
Closes#35580
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35580 from stanley-cheung:bump_dev_version_202401171835 b363888ca5
PiperOrigin-RevId: 599267642
This commit upgrades gRPC to protobuf v25.0 and makes some fixes to
account for upb changes. One major change is that upb has been merged
into the protobuf repo, so we can now drop the separate `@upb`
dependency. Another is that `.upb.c` files no longer exist and there are
new `.upb_minitable.h` and `.upb_minitable.c` files. The longer
filenames exceeded a Windows restriction, so to work around that I
renamed the `upb-generated` directory to just `upb-gen`, and likewise
for `upbdefs-generated`.
Currently it is very easy to use the `TlsCredentialsOptions` in such a
way that it produces a memory leak. For example, the code block
```
{
TlsCredentialsOptions options;
}
```
produces a memory leak. This PR fixes up the ownership bugs in this
class and its `grpc_tls_credentials_options`, the C-core analogue.
Introduce `grpc-native-debug` gems containing debug symbol packages that
complement the shared libraries shipped in pre-compiled binary gems.
src/ruby/nativedebug/README.md has details on usage.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
https://github.com/grpc/grpc/pull/33538 added `-weak_framework
CoreFoundation` in `DLDFLAGS` for only `arm64-darwin` builds, but the
issue reported in https://github.com/grpc/grpc/issues/33483 can also
happen on `x86-darwin` builds. This can happen if:
1. The Ruby interpreter is compiled without
`-Wl,-undefined,dynamic_lookup`.
2. This happens if the Ruby interpreter is built with XCode 14.0 to 14.2
(https://bugs.ruby-lang.org/issues/19005).
Simplify the logic and always include `-weak_framework CoreFoundation`
for macOS builds.
grpc 1.57.0 crashes win ruby and alpine due to no `strdup` in musl libc.
This diff replace `strdup` with `grp_strdup`
```
Thread 1 "ruby" received signal SIGSEGV, Segmentation fault.
0x00000000000a4596 in ?? ()
(gdb) bt
#0 0x00000000000a4596 in ?? ()
#1 0x00007ffff14e298c in grpc_rb_channel_create_in_process_add_args_hash_cb (key=<optimized out>, val=<optimized out>, args_obj=<optimized out>) at rb_channel_args.c:84
#2 0x00007ffff7c2b9ea in hash_ar_foreach_iter (error=0, argp=140737488344784, value=<optimized out>, key=<optimized out>) at hash.c:1341
```
fixes#34044closes#27995
This was done manually due to a problem with
`tools/distrib/python/make_grpcio_tools.py`. ~I fixed it in this PR
(depends on cl/547979185), so there is a fair chance this upgrade will
work normally for the next release.~ The fix may be problematic for
upgrading protobuf on older release branches, so the improvement will be
worked on separately. CC @jtattermusch
This also updates the UPB dep to the latest commit on the 23.x branch.
So far, we've structured global C-core init/shutdown as follows:
1) every grpc-ruby object calls `grpc_init` when allocated, and
`grpc_shutdown` when finalized
2) grpc-ruby background threads are each wrapped in a
`grpc_init/shutdown` pair - for example see
b32d94de05/src/ruby/ext/grpc/rb_event_thread.c (L122)
and
b32d94de05/src/ruby/ext/grpc/rb_event_thread.c (L136)
But because ruby doesn't join ruby threads when the process is
terminating, the `init/shutdown` pairs in 2) are always left open. I.e.,
thus far we have never been invoking the final call to `grpc_shutdown`
which actually does C-core global shutdown. Thus our calls to
`grpc_shutdown` are useless.
So we might as well keep things simple and not even attempt to call
`grpc_shutdown`. Now we just call `grpc_init` before using C-core, and
then never even attempt global shutdown (in non-forking situations).
As a bonus, this fixes the issue with the event thread's
`grpc_ruby_init` racing with `prefork` that is mentioned in
https://github.com/grpc/grpc/pull/33666
Adds experimental fork support to gRPC/Ruby
Works towards https://github.com/grpc/grpc/issues/8798 (see caveats for why this wasn't marked fixed yet)
Works towards https://github.com/grpc/grpc/issues/33578 (see caveats for why this wasn't marked fixed yet)
This leverages existing `pthread_atfork` based C-core support for
forking that python/php use, but there's a bit extra involved mainly
because gRPC/Ruby has additional background threads.
New tests under `src/ruby/end2end` show example usage.
Based on https://github.com/grpc/grpc/pull/33495
Caveats:
- Bidi streams are not yet supported (bidi streams spawn background
threads which are not yet fork safe)
- Servers not supported
- Only linux supported
Remove `src/ruby/ext/grpc/objs/` and strip `src/ruby/ext/grpc/libs/` in
ruby build dir to save some space.
This can reduce the grpc gem size from more than 1G to about 70~80M.
This only affects opt build, dbg build remains the same.
A proper fix maybe use the same makefile to build c-core and the
extension rather than building c-core when generating the extension
makefile.
fixes: https://github.com/grpc/grpc/issues/33412
This PR does the following: for the TLS server credentials, stops
calling `SSL_CTX_set_client_CA_list` by default in
`ssl_transport_security.cc`, and gives users a knob to re-enable calling
this API.
## What does the `SSL_CTX_set_client_CA_list` API do?
When this API is called, a gRPC TLS server sends the following data in
the ServerHello: for each certificate in the server's trust bundle, the
CA name in the certificate.
This API does not change the set of certificates trusted by the server
in any way. Rather, it is just providing a hint to the client about what
client certificate should be sent to the server.
## Why are we removing the use of `SSL_CTX_set_client_CA_list` by
default for the TLS server credentials?
Removing the use of this API by default has 2 benefits:
1. Calling this API makes gRPC TLS unusable for servers with a
sufficiently large trust bundle. Indeed, if the server trust bundle is
too large, then the server will always fail to build the ServerHello.
2. Calling this API is introducing a huge amount of overhead (1000s of
bytes) to each ServerHello, so removing this feature will improve
connection establishment latency for all users of the TLS server
credentials.
Add -weak_framework CoreFoundation to the ruby extension link line on
arm64-darwin, to address "undefined symbols" issues from #33483
This is a variation on #33513 in response to feedback and so I've made
sure @alto-ruby is credited as co-author.
Closes#33483
Supersedes #33513
cc @apolcyn
Co-authored-by: alto-ruby <altorubys@gmail.com>
The function grpc_rb_call_run_batch has many places that could raise
errors, including in child functions. Since a raised error will longjump
out of the function, it will cause memory leaks since the function
cannot perform any clean up. This commit fixes the issue by wrapping the
whole function in an rb_ensure, which will ensure that a cleanup
function is ran before the error is propagated upwards.
The function grpc_rb_server_request_call has many places that could
raise errors, including in child functions. Since a raised error will
longjump out of the function, it will cause memory leaks since the
function cannot perform any clean up. This commit fixes the issue by
wrapping the whole function in an rb_ensure, which will ensure that a
cleanup function is ran before the error is propagated upwards.
re2 previously failed to compile if:
1. An old `re2` version is installed with a non-standard system prefix,
such as `/opt/local`.
2. The environment variable is set: `CPPFLAGS=-I/opt/local/include`.
Running `make` would result in function prototype mismatches because the
Makefile would previously attempt to use the headers from
`/opt/local/include/re2` before the `third_party/re2/re2` directory.
https://github.com/grpc/grpc/pull/27660 caused `CPPFLAGS` to inherit
from the environment, but this can cause the Makefile to use external
include files for re2 and other libraries if `-I` flags are defined.
This commit reverts to the original behavior of only using
`RbConfig::CONFIG` values to avoid using the wrong headers.
Let's see if this fixes the "Bus error" flakes that have been happening
in CI.
If it does, then we can narrow things down a bit. If flakes continue,
then we can revert this PR.
* src/proto/grpc/core/stats.proto no longer exists
* remove PHP generated file for proto that no longer exists
* run src/ruby/pb/generate_proto_ruby.sh
In Ruby 3.2 the default allocator warns if used with `Data_Wrap_Struct`. This removed the default allocator in those cases.
Ruby discussion: https://bugs.ruby-lang.org/issues/18007
Previously attempting to build the Ruby C extensionwould result in
undefined symbol:
```
Undefined symbols for architecture arm64:
"_ruby_abi_version", referenced from:
-exported_symbol[s_list] command line option
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
As noted in https://github.com/ruby/ruby/pull/6231,
`_ruby_abi_version` is only available for development
versions. `RUBY_PATCHLEVEL` < 0 denotes a development version, while
>= 0 denotes a released version.
Stan Hu
Esun Kim
Alan Wu
Stanley Cheung
alto-ruby
Luwei Ge
Mark D. Roth
Adam Cozzette
Matthew Stevenson
apolcyn
Richard Belleville
alto-ruby
AJ Heller
Mike Dalessio
Peter Zhu
Yijie Ma
Yash Tibrewal
y.okamoto
Jan Tattermusch
Craig Tiller
Adam Hess
Cheng-Yu Chung