This fixes a v22 regression for Ruby.
Windows builds of the Ruby package are broken in 3.22.0.rc.2. The reason is, `ruby-upb.h` no longer provides the full path when `#include`ing `utf8_range.h`. See https://github.com/protocolbuffers/protobuf/blob/v22.0-rc2/ruby/ext/google/protobuf_c/ruby-upb.h#L10479 (22.0-rc-2), cf. https://github.com/protocolbuffers/protobuf/blob/v21.12/ruby/ext/google/protobuf_c/ruby-upb.h#L5365 (21.12). The `extconf.rb` build configuration tries to compensate by adding `third_party/utf8_range` to the include path, but does not do so on Windows (i.e. `mingw`) platforms. See https://github.com/protocolbuffers/protobuf/blob/v22.0-rc2/ruby/ext/google/protobuf_c/extconf.rb#L9-L10 (22.0-rc-2), cf. https://github.com/protocolbuffers/protobuf/blob/v21.12/ruby/ext/google/protobuf_c/extconf.rb#L9-L10 (21.12).
We could have fixed this by adding another clause to the if statement for the case `RUBY_PLATFORM =~ /mingw/` and adding the appropriate `-I` flag to `CFLAGS`. However, that `CFLAGS` hack is present in the first place due to a related problem: the usage of `$INCFLAGS` is incorrect.
The `$INCFLAGS` constant in Ruby's `mkmf` is a string in a similar format to `CFLAGS`. It's simply appended to compiler invocations. So when you append new flags to it, you have to (1) provide the flag itself, and (2) precede it with a space to delimit it from the previous entry. In 22.0-rc-2 (and indeed in all earlier versions) the usage is incorrect: it's appending a path to the string without the `-I` and without a space. See https://github.com/protocolbuffers/protobuf/blob/v22.0-rc2/ruby/ext/google/protobuf_c/extconf.rb#L22. Hence, not only does the intended include path not get appended correctly, it also clobbers the previous path in the string. Luckily, the previous path is only `-I$(srcdir)` which happens not to matter for our library. But it does mean that the apparent intent of that line, adding `$(srcdir)/third_party/utf8_range` to the include path, isn't working; hence the code that adds it to `CFLAGS` instead.
(Note that the previous line, adding the path to `$VPATH`, _is_ correct as is, because `$VPATH` is an array.)
So what this PR actually does is fix the `$INCFLAGS` usage so `$(srcdir)/third_party/utf8_range` gets added properly to compiler include paths, for all platforms including Windows. Since that should now be working as intended, we also remove the extra `-I` from CFLAGS. Builds for all platforms should now be able to handle the change to `ruby-upb.h`. This has been tested by running the prototype Ruby build Kokoro job against a patched 22.0-rc-2.
This also needs to be backported to the 22.x branch.
/attn @deannagarciaCloses#11882
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11882 from dazuma:pr/ruby-builds-fix ebb18e0004
PiperOrigin-RevId: 508550039
objc_library really will only build for Apple platforms, so tag it as such.
The objc conformance test really will only build/run on macOS, so tag it as
such.
With these two changes, it should be safer for recursive builds that include
this directory no matter what the host platform is.
PiperOrigin-RevId: 508438744
Since the sources can't reference each other, this is what helps keep them in
sync. Move it out to be a stand alone test in bazel instead of having it wired
into the Xcode projects.
PiperOrigin-RevId: 508429454
This change introduces a dedicated StringBlock class that hosts block allocated string instances which are destroyed when the arena gets destroyed.
Experiments have shown this to save 0.40% in memory and 0.10% of CPU in arena heavy use cases.
PiperOrigin-RevId: 508205452
- General naming up to "Objective-C" that some linters want, so just
use it every where for consistence.
- Add a bazel macOS config
- Includes the conformance test, so drop it from the Xcode flows.
- Tweak the naming on the Xcode flows so they are a little more clear.
- CocoaPods flows:
- Give them (hopefully) clear naming
- Use the same `image` as the Xcode tests
- Pin the Xcode version to match the Xcode tests
PiperOrigin-RevId: 508156130
This should substantially reduce network flakes, since most builds will no longer need to download external dependencies. Because these caches actually get pretty expensive (~300 MB just from 1 openjdk dependency), we've taken some precautions:
1) The caches only get updated on push events (i.e. post-submit jobs)
2) Only a single job will push a new cache for any os/branch/commit combination. This will be whichever job finishes its bazel run first, and *also* has updates for the cache. Over time, the cache should expand to the full set of transitive dependencies as the quicker jobs stop pushing updates.
3) A clear_caches workflow has been introduced to reset the caches in case they grow too large. This will run monthly for now, but can also be run manually.
PiperOrigin-RevId: 508133044
This CL also includes some cleanups to the core FieldVars function, and eliminates what had long-since become vestigial functions for setting variables. This needs to be part of the same CL because those functions used string_view-keyed maps.
PiperOrigin-RevId: 508124647
It isn't worth the effort to try and compute the correct minimal set for each
sub file, and the more Swift folks use on top of this, the less it matters
as the modules will need everything anyways.
PiperOrigin-RevId: 508122902
This code is experimental and should not be expected to emit working code, and callers are liable to break without warning.
It is being released now so that development can occur in the open, but users should not expect this to be supported any time soon.
PiperOrigin-RevId: 508095929
Move Field & Extension support for lack of UsesClassRef into the legacy api
so the only things taking the hit are old generate code.
PiperOrigin-RevId: 508074967
Add a new generator interface to:
- provide the message name directly instead of the overhead needed to try and compute it
- provide the data for the GPBFileDescriptor as a struct.
This allows -[GPBMessage fullName] to much more easily compute the fullname.
It also will allow the creation of all GPBFileDescriptors to be done on demand thus reducing the work in the startup codepaths as well as reducing memory requirements.
The generation changes are in a following commit.
PiperOrigin-RevId: 508063366
In order to flip the flag, all downstream projects should be adapted. However, it is hard to fix them all if there are constant regressions. Adding it to the CI will ensure that once the project can build with incompatible_disallow_empty_glob it can keep building like that.
See: bazelbuild/bazel#15327
PiperOrigin-RevId: 507927389
With gcc (tested versions: 11.3.0, 12.2.0) compiled with asan, the
`type_id` variable is warned about maybe being used uninitialized in
the later call:
```
ptr = ParseFieldMaybeLazily(static_cast<uint64_t(type_id) * 8 + 2, ptr,
extendee, metadata, ctx);
```
This change initializes the variable to a defined default.
PiperOrigin-RevId: 507883964
This has the consequence of removing the path filter, but allows us to actually use these test results as presubmits. Since these are relatively cheap tests, this seems like a better situation. A future enhancement may restore path filters to these and other tests.
PiperOrigin-RevId: 507872208
Include option to skip it if it was installed.
https://github.com/tuist/xcbeautify would also be an option (it supports bazel to),
but we'd have to install it.
PiperOrigin-RevId: 507852912
The `BuildInParallel` flag only controls parallelism at the *project* level, while `/MP` enables it at the *compilation unit* level.
PiperOrigin-RevId: 507837073
This slows down the build and hasn't had an issue in a really long time. The
support is still there, so it can be manually run at any time from within Xcode.
PiperOrigin-RevId: 507810543
This uses ccache + github caching to substantially decrease the time it takes to run XCode builds. Due to Bazel caching, these are some of our slowest tests, causing one of the biggest presubmit bottlenecks
PiperOrigin-RevId: 507794241
This uses ccache + github caching to substantially decrease the time it takes to run CMake builds. Due to Bazel caching, these are some of our slowest tests, causing one of the biggest presubmit bottlenecks
PiperOrigin-RevId: 507667813
Protobuf Team Bot
Sandy Zhang
Daniel Azuma
Protobuf Team Bot
Martijn Vels
Mike Kruskal
Xavier Bonaventura
James Brock
Deanna Garcia