* Allow pre-compiled binaries for ruby 3.1.0 (#9566)
* Allow pre-compiled binaries for ruby 3.1.1
* add comment
* fix build and use ruby 3.1.0
* add ruby31 to build CI for tests and release
* trying to fix ci
* install ruby 3.1.0 in ruby_build_environment.sh
* use head for rvm to install 3.1.0
* just install master version of rvm in prepare_build_macos_rc
* force install of master rvm in ruby_build_environment.sh
* Use coroutine=universal when compiling ruby31
* use ucontext
* fix filename
* fix coroutine name
* use git head for rake-compiler-dock
* use newest rake-compiler-dock version
* Updated CHANGES.txt for Ruby changes.
* Fixed Ruby 3.1 tests by marking intersect? as unimplemented. (#9645)
* Fixed Ruby 3.1 tests by marking intersect? as unimplemented.
* Updated compatibility tests.
Co-authored-by: Marco Concetto Rudilosso <marcoconcettorudilosso@gmail.com>
The JRuby test runs appear to be fetching the util package from Maven
Central because it is not being installed locally. This causes the tests
to fail when we upgrade to a new version that has not yet been published
to Maven Central. This commit fixes the problem by locally installing
all the protobuf packages, not just util.
* Drop Python versions <3.7.
* Updated README to clarify that Python 3.7 is the minimum.
* Removed more Python 3.5-specific code.
Also changed tests to skip missing interpreters.
* Invoke tox directly instead of through Python.
Hopefully this will pick up python3.
* Updated java_stretch image to bullseye to get Python >= 3.7.
* Use jdk11 instead of jdk8.
* Installed python2 for gtest.
* Use "python3 -m venv" instead of "virtualenv."
* Install python3-venv.
Should prevent future flakiness if unreleased JRuby features are dependent on unreleased Java features by populating the local Maven repository with the artifacts needed when running `tests.sh jruby`.
googletest uses a Python script in its build which is not compatible
with Python 3. Unfortunately we can't easily upgrade googletest right
now, so this commit works around the problem by putting a python symlink
pointing to Python 2 in the $PATH.
* Fix dist install test by ensuring that we use Python 3
Now that we have dropped Python 2 support, we need to make sure this
install test uses Python 3.
* Update Docker image to install Python 3 version of setuptools
* Run pip3 instead of pip
* Fix dist install test by ensuring that we use Python 3
Now that we have dropped Python 2 support, we need to make sure this
install test uses Python 3.
* Update Docker image to install Python 3 version of setuptools
* Run pip3 instead of pip
This is necessary because the Kotlin support requires Java 8 or
higher.
Making this test use Java 8 required switching it to the java_stretch
Docker image and updating that image to include some Python
dependencies.
* Simplified PHP testing setup.
- Consolidated on a single autoloader, created by composer.
- Consolidated on a single phpunit invocation strategy: we run
phpunit on a directory, which will run all tests matching *Test.php
in that directory.
- We now rely on autoloading to import all test protos. require_once()
calls for test protos are removed.
- For now the valgrind tests are removed. A follow-up PR will re-enable
them in a more robust way.
* More improvements to PHP testing.
1. Replace custom PHPUnit-selection logic in test.sh with generic
composer version selection.
2. Optimized both test proto generation and the custom extension
build to avoid unnecessary work when the files are already up
to date.
* Added assertions to verify that the C test doesn't use PHP sources.
* Updated tests.sh for the new PHP testing commands.
* Removed obsolete rules from tests.sh.
* Fixed generate_test_protos.sh for when tmp does not exist.
Also removed undefined_test.php and fixed Makefile.am.
* Added php8.0_all again which is still used.
* Added missing file to Makefile.am.
* Re-added php_all_32 rule which is also still used.
* Updated testing commands for macOS and download composer.
* Use /usr/local/bin on mac instead of /usr/bin, since the latter is not writable.
* WIP.
* WIP.
* WIP.
* WIP.
* WIP.
* WIP.
* Added some missing files.
* WIP.
* WIP.
* Updated upb.
* Extension loads, but crashes immediately.
* Gets through the test suite without SEGV!
Still a lot of bugs to fix, but it is a major step!
214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed
* Test and build for Ruby 3.0
* Fixed a few more bugs, efficient #inspect is almost done.
214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed
* Fixed message hash initialization and encode depth checking.
214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed
* A bunch of fixes to failing tests, now 70% passing.
214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed
* More than 80% of tests are passing now.
214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed
Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.
* Add linux/ruby30 and macos/ruby30
* Use rvm master for 3.0.0-preview2
* Over 90% of tests are passing!
214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed
* Passes all tests!
214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
* A bunch of cleanup.
1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.
* Removed a bunch more stuff from protobuf.h. There is an intermittent assert failure.
Intermittent failure:
ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed
* Removed a few more things from protobuf.h.
* Ruby 3.0.0-preview2 to 3.0.0
* Require rake-compiler-dock >= 1.1.0
* More progress, fighting with the object cache.
* Passes on all Ruby versions!
* Updated and clarified comment regarding WeakMap.
* Fixed the wyhash compile.
* Fixed conformance tests for Ruby.
Conformance results now look like:
RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb
CONFORMANCE TEST BEGIN ====================================
CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.
CONFORMANCE TEST BEGIN ====================================
CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.
Fixes include:
- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
JSON.
- Fixed conformance test to properly report serialization errors.
* Removed debug printf and fixed #inspect for floats.
* Fixed compatibility test to have proper semantics for #to_json.
* Updated Makefile.am with new file list.
* Don't try to copy wyhash when inside Docker.
* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.
* Avoid exporting unneeded symbols and refactored a bit of code.
* Some more refactoring.
* Simplified and added more comments.
* Some more comments and simplification. Added a missing license block.
Co-authored-by: Masaki Hara <hara@wantedly.com>
* Port for php8
* Port php c extension for php8
* Update composer.json
* Drop php7.0 support
* Update phpunit for php7.1 in c extension test
* Add back support for php7.0
* Add badge for php8 continuous build
Using non-versioned scripts for `build_file` means every Python version is
tested (under Tox), and this is multiplied by each Python version running Tox.
For example, when the `python38` config is used to run Tox, the
Tox layer runs on Python 3.8. This then tests all of the Python versions
individually, including, for example, python27 tests.
This change fixes the `build_file` paths to point to the same-pathed build
script. For example, `kokoro/linux/python27/presubmit.cfg` now uses
`kokoro/linux/python27/build.sh`.
Some additional fixes:
* Use `python -m tox` in tests.sh instead of just `tox`. This helps non-site
installations of tox, where the `tox` script may not be on `$PATH`.
* Ensure tox (and other Python build-related packages) are available in
Python testing images. (New images have been pushed.)
* Disable `--warnings_as_errors` due to deprecated function.
* Remove apt lists per [Docker best practices][1].
[1]: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/
This extension allows Python code to be generated from setup.py files, so they
are created as part of a normal Python build. The extension uses an already-existing
protoc binary, which can be explicitly specified if needed.
* Only ported c extension to php8.
* Didn't fixed the issue of throwing warnings for missing arginfo in bundled files.
* Tests not fixed, because syntax of phpunit (<7 vs >9.3) are not compatible.
* In next release, needs to drop php5 and php7.0 support (in order to use phpunit > 7)
This commit cleans up our environment setup for macOS builds.
- Always use `python -m {pip | virtualenv}` form of commands.
- Only upgrade in-place, and never use `--ignore-installed`.
- Use latest tox on macos, and test 3.6 on macos presubmit.
These changes ensure that multibuilder won't break the system-installed Python on the Kokoro macOS builder, and thus won't resolve the ancient system version. For Tox, make sure we test with Python 3.6 (the version of Python in current macOS).
* Added code for the new PHP extension.
* Removed a bunch of unused variables shown by compiler warnings.
* Test protobuf2 in the tests.
* Updated upb version to fix a goldenfile test.
* Added pure-PHP changes. Extension now passes all tests.
* Enabled protobuf2 for all C extension tests.
* Fixed pure=PHP lib: full names no longer start with '.'.
* Added files for new extension to Makefile.am.
* Downgraded make-preload.py to python 2, since python3 isn't available in the php_all Kokoro image.
* Disable tests of new C extension with PHP 5.x.
* Also do not compile the extension for PHP5.
* Accept version 5.*.*, and use /usr/bin/python.
* Addressed PR comments.
* Addressed PR comments.
* Added "const" to a parameter and fixed a memory leak seen in Valgrind.
* Stop testing the C extension for PHP5.
The next release of protobuf will deprecate the C extension
for PHP5, see:
https://github.com/protocolbuffers/protobuf/issues/7525
* Made the PHP5.6 Mac test only exercise pure-PHP.
* Build protoc for PHP 5.6 test.
* Rewrote bundling script in PHP to avoid dependency on Python.
* A few more fixes.
* Fixed int32/int64 behavior for 32-bit builds.
* Match more PHP versions in testing script.
* Use phpize --clean before building extension.
* Force-delete configure.in to avoid phpize problems cross-version.
* Delete both configure.ac and configure.in.
* De-duplicated code to generate PHP protos and install phpunit.
* Removed all references to generate_php_test_proto.
* Replaced with internal_build_cpp.
* Make Timestamp::__construct() static to avoid conflicts with MongoDB.
* Replicated PHPUnit versions and added new script to Makefile.am.
* Fixed filename in Makefile.am.
* Disabled test that SEGV's on macOS.
* Removed extraneous "set -e".
* Make sure generate_protos.sh happens on every test path.
* Removed stray '$' chars.
* Added proper support for aggregate_metadata tests. But now I get a stack overflow.
Stack overflow:
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
namespace GPBMetadata\Proto;
class TestDescriptors
{
public static $is_initialized = false;
public static function initOnce() {
$pool = \Google\Protobuf\Internal\DescriptorPool::getGeneratedPool();
if (static::$is_initialized == true) {
return;
}
\GPBMetadata\Proto\TestDescriptors::initOnce();
$pool->internalAddGeneratedFile(hex2bin(
""
), true);
static::$is_initialized = true;
}
}
* Fixed and verified metadata aggregation testing.
- Remove the `brew install`; the kokoro image already has it, just
update/upgrade instead.
- Remove `prune`, logs had:
"""
Error: Unknown command: prune
"""
- Remove `uninstall`, logs had:
"""
Error: Refusing to uninstall /usr/local/Cellar/cmake/3.16.4 and /usr/local/Cellar/icu4c/64.2
because they are required by ceres-solver, ffmpeg, harfbuzz, libass and opencv, which are currently installed.
You can override this and force removal with:
brew uninstall --ignore-dependencies node icu4c cmake wget
"""
- Skip installing some things since they are already in the base image, logs
had:
"""
Warning: gflags 2.2.2 is already installed and up-to-date
To reinstall 2.2.2, run `brew reinstall gflags`
Warning: openssl@1.1 1.1.1g is already installed and up-to-date
To reinstall 1.1.1g, run `brew reinstall openssl@1.1`
Warning: pcre 8.44 is already installed and up-to-date
To reinstall 8.44, run `brew reinstall pcre`
"""
- Don't install gpg gpg2 as gnupg is already installed, also use gpg instead of
gpg2 for commands (and update the commands), logs had:
"""
kokoro/macos/prepare_build_macos_rc: line 44: gpg2: command not found
kokoro/macos/prepare_build_macos_rc: line 45: gpg2: command not found
"""
- Add env guards to control all the option installs and only request them be
installed in the cases that need it. This avoids having to install/update
the things like ruby when some other tool only needed in some configs is
install differently and could have conflicts.
- Switch to brew for cocoapods to avoid compat issues on the supporting
libraries.
* Simplified PHP testing scripts.
- allow them to be run from any directory.
- remove "VERSION" arg, we can get PHP from $PATH.
* Added a few places I missed.
* Removed redundant "cd `dirname $0`".
Also replaced all backticks with $(), for consistency.
Joshua Haberman
Adam Cozzette
Jason Lunn
Brett McBride
Elliotte Rusty Harold
Bu Sun Kim
Deanna Garcia
Derek Perez
Paul Yang
Rob Widmer
Bu Sun Kim
David L. Jones
Thomas Van Lenten