Before this change if a field type was defined in a imported .proto file then our codegen would not generate the field. After this change such fields are correctly generated (see tests). This change is rather trivial as all the supporting infra has been implemented as part of the .proto -> crate mapping CLs.
PiperOrigin-RevId: 601443383
issue##15029
As required by https://developer.apple.com/support/third-party-SDK-requirements/
Protos does not collect data.
Required reason APIs searched for using:
```
grep -ER 'NSFileCreationDate|NSFileModificationDate|fileModificationDate|NSURLContentModificationDateKey|NSURLCreationDateKey|getattrlist\(|getattrlistbulk\(|fgetattrlist\(|stat\(|fstat\(|fstatat\(|lstat\(|getattrlistat\(|systemUptime\(|mach_absolute_time\(|NSURLVolumeAvailableCapacityKey|NSURLVolumeAvailableCapacityForImportantUsageKey|NSURLVolumeAvailableCapacityForOpportunisticUsageKey|NSURLVolumeTotalCapacityKey|NSFileSystemFreeSize|NSFileSystemSize|statfs\(|statvfs\(|fstatfs\(|fstatvfs\(|getattrlist\(|fgetattrlist\(|getattrlistat\(|activeInputModes|NSUserDefaults' .
```
```
./third_party/utf8_range/utf8_to_utf16/main.c: if (fstat(fd, &stat) == -1) {
./third_party/utf8_range/main.c: if (fstat(fd, &stat) == -1) {
./src/google/protobuf/map.h: s = mach_absolute_time();
./src/google/protobuf/io/io_win32.cc:int stat(const char* path, struct _stat* buffer) {
./src/google/protobuf/io/io_win32.cc: return ::_wstat(wpath.c_str(), buffer);
./src/google/protobuf/io/io_win32.cc: return ::_stat(path, buffer);
./src/google/protobuf/io/io_win32.h:PROTOBUF_EXPORT int stat(const char* path, struct _stat* buffer);
./src/google/protobuf/testing/file.cc: if (lstat(name.c_str(), &stats) != 0) return;
./src/google/protobuf/compiler/importer.cc: ret = stat(std::string(filename).c_str(), &sb);
```
`./third_party/utf8_range/...` - tool for utf that is not in user binaries
`./src/google/protobuf/map.h` - resolved with pull#15554
`./src/google/protobuf/io/io_win32.cc` - windows code
`./src/google/protobuf/testing/file.cc` - testing code not in user binaries
`./src/google/protobuf/compiler/importer.cc` - compiler code not in user binaries
So no required reason APIs.
Apple has not yet communicated how this is to be put into podspecs so saving for future commit.
Closes#15557
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/15557 from dmaclach:privacy 4c4e570adf
PiperOrigin-RevId: 601362376
Part 3 of 4 (added a stage).
The getter and mut_getter bifurcate based on the kernel.
upb returns Option<RawMessage>, while cpp returns the RawMessage.
This means that we can't have a unified MessageVTable in vtable.rs, so we've split these out in {upb.rs and cpp.rs}.
$field$_entry is now prepped and populated for the $field$_mut swappage in the following CL.
PiperOrigin-RevId: 601230880
Follow up on #10404, the initial description:
> For the sake of performance, the hex2bin call was removed from the generated PHP code in https://github.com/protocolbuffers/protobuf/pull/8006
However, after this PR all autogenerated files contain binary data, which makes it hard or even impossible to see using common tools (git and github included (i.e. [this file](https://github.com/protocolbuffers/protobuf/blob/main/php/src/GPBMetadata/Google/Protobuf/Struct.php)), as well as some code editors) since most software considers such files as binary ones and not as code.
Alexander suggested using a hex representation of string literals and I updated the original patch a bit, not hex-encoding printable characters.
Benchmarks from [#10404#issuecomment-1635939062](https://github.com/protocolbuffers/protobuf/pull/10404#issuecomment-1635939062)
```
The percent of printable chars: 0.83869 (80112 vs 15408)
hex parsing: 3.22371, length=382090
hex2Bin parsing: 1.18489, length=191059
hexAscii parsing: 0.89524, length=142306 (suggested option)
binary parsing: 0.26437, length=95542
```
Closes#13911
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/13911 from mikhainin:do_not_store_binary_data_inside_php_files 490958e165
PiperOrigin-RevId: 601123255
Also hardened the text format printer against invalid UTF-8 in string fields. The output string will always be valid UTF-8, even if string fields contain invalid UTF-8.
PiperOrigin-RevId: 600990001
It's a little bit awkward that we currently have to take a runtime dependency
on rake even though it is a build tool. This CL adds a TODO to explore moving
the Rakefile logic to extconf.rb so that we can drop the rake dependency.
PiperOrigin-RevId: 600936420
We have a rake based extension `ext/google/protobuf_c/Rakefile` (added since 3.25.0), which depends on `rake`. However, when gem is installed by `bundle` command, `rake` is not guaranteed to be activated by `bundle` (e.g. when `BUNDLE_PATH` is not default and rake is not already installed in `BUNDLE_PATH`). Therefore, we have to explicit declare a runtime dependency on `rake` for the extension, or otherwise the installation will fail with `can't find gem rake (>= 0.a) with executable rake (Gem::GemNotFoundException)`.
Here is a reproduction:
```
root@4d9cb6ba41b1:/build# cat Gemfile
source 'https://rubygems.org'
gem 'google-protobuf'
root@4d9cb6ba41b1:/build# bundle config --local path vendor/bundle
root@4d9cb6ba41b1:/build# bundle
Fetching gem metadata from https://rubygems.org/...........
Resolving dependencies...
Installing google-protobuf 3.25.1 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /build/vendor/bundle/ruby/3.3.0/gems/google-protobuf-3.25.1/ext/google/protobuf_c
rake RUBYARCHDIR\=/build/vendor/bundle/ruby/3.3.0/extensions/aarch64-linux/3.3.0/google-protobuf-3.25.1 RUBYLIBDIR\=/build/vendor/bundle/ruby/3.3.0/extensions/aarch64-linux/3.3.0/google-protobuf-3.25.1
/usr/local/lib/ruby/3.3.0/rubygems.rb:259:in `find_spec_for_exe': can't find gem rake (>= 0.a) with executable rake (Gem::GemNotFoundException)
from /usr/local/lib/ruby/3.3.0/rubygems.rb:278:in `activate_bin_path'
from /usr/local/bin/rake:25:in `<main>'
rake failed, exit code 1
Gem files will remain installed in /build/vendor/bundle/ruby/3.3.0/gems/google-protobuf-3.25.1 for inspection.
Results logged to /build/vendor/bundle/ruby/3.3.0/extensions/aarch64-linux/3.3.0/google-protobuf-3.25.1/gem_make.out
/usr/local/lib/ruby/3.3.0/rubygems/ext/builder.rb:125:in `run'
/usr/local/lib/ruby/3.3.0/rubygems/ext/rake_builder.rb:30:in `build'
/usr/local/lib/ruby/3.3.0/rubygems/ext/builder.rb:193:in `build_extension'
/usr/local/lib/ruby/3.3.0/rubygems/ext/builder.rb:227:in `block in build_extensions'
/usr/local/lib/ruby/3.3.0/rubygems/ext/builder.rb:224:in `each'
/usr/local/lib/ruby/3.3.0/rubygems/ext/builder.rb:224:in `build_extensions'
/usr/local/lib/ruby/3.3.0/rubygems/installer.rb:852:in `build_extensions'
/usr/local/lib/ruby/3.3.0/bundler/rubygems_gem_installer.rb:76:in `build_extensions'
/usr/local/lib/ruby/3.3.0/bundler/rubygems_gem_installer.rb:28:in `install'
/usr/local/lib/ruby/3.3.0/bundler/source/rubygems.rb:205:in `install'
/usr/local/lib/ruby/3.3.0/bundler/installer/gem_installer.rb:54:in `install'
/usr/local/lib/ruby/3.3.0/bundler/installer/gem_installer.rb:16:in `install_from_spec'
/usr/local/lib/ruby/3.3.0/bundler/installer/parallel_installer.rb:132:in `do_install'
/usr/local/lib/ruby/3.3.0/bundler/installer/parallel_installer.rb:123:in `block in worker_pool'
/usr/local/lib/ruby/3.3.0/bundler/worker.rb:62:in `apply_func'
/usr/local/lib/ruby/3.3.0/bundler/worker.rb:57:in `block in process_queue'
<internal:kernel>:187:in `loop'
/usr/local/lib/ruby/3.3.0/bundler/worker.rb:54:in `process_queue'
/usr/local/lib/ruby/3.3.0/bundler/worker.rb:90:in `block (2 levels) in create_threads'
An error occurred while installing google-protobuf (3.25.1), and Bundler cannot continue.
In Gemfile:
google-protobuf
```
Closes#15203
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/15203 from ntkme:ruby-rake-dependency 1318863022
PiperOrigin-RevId: 600916613
Stage 2 of 3.
Fleshed out the bodies required by the aforementioned contracts above.
We create MessageVTable and utilize the `unsafe fns` directly in `message.cc` -- a departure from PrimitiveVTables.
In the final followup CL, we will perform the field_entry swapover and update all unit tests.
PiperOrigin-RevId: 600567890
Jakob Buchgraber
Protobuf Team Bot
dmaclach
Marcel Hlopko
Austin Wise
Eric Salo
Joshua Haberman
Sandy Zhang
Hong Shin
Protobuf Team Bot
Mikhail Galanin
Adam Cozzette
なつき
Jie Luo