Using non-versioned scripts for `build_file` means every Python version is
tested (under Tox), and this is multiplied by each Python version running Tox.
For example, when the `python38` config is used to run Tox, the
Tox layer runs on Python 3.8. This then tests all of the Python versions
individually, including, for example, python27 tests.
This change fixes the `build_file` paths to point to the same-pathed build
script. For example, `kokoro/linux/python27/presubmit.cfg` now uses
`kokoro/linux/python27/build.sh`.
Some additional fixes:
* Use `python -m tox` in tests.sh instead of just `tox`. This helps non-site
installations of tox, where the `tox` script may not be on `$PATH`.
* Ensure tox (and other Python build-related packages) are available in
Python testing images. (New images have been pushed.)
* Disable `--warnings_as_errors` due to deprecated function.
* Remove apt lists per [Docker best practices][1].
[1]: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/
This extension allows Python code to be generated from setup.py files, so they
are created as part of a normal Python build. The extension uses an already-existing
protoc binary, which can be explicitly specified if needed.
* Only ported c extension to php8.
* Didn't fixed the issue of throwing warnings for missing arginfo in bundled files.
* Tests not fixed, because syntax of phpunit (<7 vs >9.3) are not compatible.
* In next release, needs to drop php5 and php7.0 support (in order to use phpunit > 7)
This commit cleans up our environment setup for macOS builds.
- Always use `python -m {pip | virtualenv}` form of commands.
- Only upgrade in-place, and never use `--ignore-installed`.
- Use latest tox on macos, and test 3.6 on macos presubmit.
These changes ensure that multibuilder won't break the system-installed Python on the Kokoro macOS builder, and thus won't resolve the ancient system version. For Tox, make sure we test with Python 3.6 (the version of Python in current macOS).
* Added code for the new PHP extension.
* Removed a bunch of unused variables shown by compiler warnings.
* Test protobuf2 in the tests.
* Updated upb version to fix a goldenfile test.
* Added pure-PHP changes. Extension now passes all tests.
* Enabled protobuf2 for all C extension tests.
* Fixed pure=PHP lib: full names no longer start with '.'.
* Added files for new extension to Makefile.am.
* Downgraded make-preload.py to python 2, since python3 isn't available in the php_all Kokoro image.
* Disable tests of new C extension with PHP 5.x.
* Also do not compile the extension for PHP5.
* Accept version 5.*.*, and use /usr/bin/python.
* Addressed PR comments.
* Addressed PR comments.
* Added "const" to a parameter and fixed a memory leak seen in Valgrind.
* Stop testing the C extension for PHP5.
The next release of protobuf will deprecate the C extension
for PHP5, see:
https://github.com/protocolbuffers/protobuf/issues/7525
* Made the PHP5.6 Mac test only exercise pure-PHP.
* Build protoc for PHP 5.6 test.
* Rewrote bundling script in PHP to avoid dependency on Python.
* A few more fixes.
* Fixed int32/int64 behavior for 32-bit builds.
* Match more PHP versions in testing script.
* Use phpize --clean before building extension.
* Force-delete configure.in to avoid phpize problems cross-version.
* Delete both configure.ac and configure.in.
* De-duplicated code to generate PHP protos and install phpunit.
* Removed all references to generate_php_test_proto.
* Replaced with internal_build_cpp.
* Make Timestamp::__construct() static to avoid conflicts with MongoDB.
* Replicated PHPUnit versions and added new script to Makefile.am.
* Fixed filename in Makefile.am.
* Disabled test that SEGV's on macOS.
* Removed extraneous "set -e".
* Make sure generate_protos.sh happens on every test path.
* Removed stray '$' chars.
* Added proper support for aggregate_metadata tests. But now I get a stack overflow.
Stack overflow:
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
namespace GPBMetadata\Proto;
class TestDescriptors
{
public static $is_initialized = false;
public static function initOnce() {
$pool = \Google\Protobuf\Internal\DescriptorPool::getGeneratedPool();
if (static::$is_initialized == true) {
return;
}
\GPBMetadata\Proto\TestDescriptors::initOnce();
$pool->internalAddGeneratedFile(hex2bin(
""
), true);
static::$is_initialized = true;
}
}
* Fixed and verified metadata aggregation testing.
- Remove the `brew install`; the kokoro image already has it, just
update/upgrade instead.
- Remove `prune`, logs had:
"""
Error: Unknown command: prune
"""
- Remove `uninstall`, logs had:
"""
Error: Refusing to uninstall /usr/local/Cellar/cmake/3.16.4 and /usr/local/Cellar/icu4c/64.2
because they are required by ceres-solver, ffmpeg, harfbuzz, libass and opencv, which are currently installed.
You can override this and force removal with:
brew uninstall --ignore-dependencies node icu4c cmake wget
"""
- Skip installing some things since they are already in the base image, logs
had:
"""
Warning: gflags 2.2.2 is already installed and up-to-date
To reinstall 2.2.2, run `brew reinstall gflags`
Warning: openssl@1.1 1.1.1g is already installed and up-to-date
To reinstall 1.1.1g, run `brew reinstall openssl@1.1`
Warning: pcre 8.44 is already installed and up-to-date
To reinstall 8.44, run `brew reinstall pcre`
"""
- Don't install gpg gpg2 as gnupg is already installed, also use gpg instead of
gpg2 for commands (and update the commands), logs had:
"""
kokoro/macos/prepare_build_macos_rc: line 44: gpg2: command not found
kokoro/macos/prepare_build_macos_rc: line 45: gpg2: command not found
"""
- Add env guards to control all the option installs and only request them be
installed in the cases that need it. This avoids having to install/update
the things like ruby when some other tool only needed in some configs is
install differently and could have conflicts.
- Switch to brew for cocoapods to avoid compat issues on the supporting
libraries.
* Simplified PHP testing scripts.
- allow them to be run from any directory.
- remove "VERSION" arg, we can get PHP from $PATH.
* Added a few places I missed.
* Removed redundant "cd `dirname $0`".
Also replaced all backticks with $(), for consistency.
* Add a test suite for ruby 2.7
* Call BigDecimal() instead of BigDecimal.new()
BigDecimal.new was deprecated in ruby 2.6
* Switch FrozenError expectation to a matcher
The error message for FrozenError changed to include more information
about the mutated object. Switch from an exact match to an aproximate
match (equal => match). This does not change the prefix.
* We can safely ignore newest array methods from ruby 2.7
* python: generate documentation with Sphinx and Read the Docs
Background:
Formerly, the Python protobuf reference documentation was built with
[Epydoc](http://epydoc.sourceforge.net/). This package has not been
updated since 2008, and it has inconsistent formatting (see internal
issue 131415575) with most Python documentation. Sphinx is used for the
official docs.python.org docs as well as most other Python packages,
including the Google client libraries and related packages, such as
https://googleapis.dev/python/google-api-core/latest/
To build the docs with Sphinx:
1. Install the needed packages (`sphinx`, `sphinxcontrib-napoleon` for
Google-style docstring support). I've created a conda environment file
to make this easier:
```
conda env create -f python/docs/environment.yml
```
2. (Optional) Generate reference docs files and regenerate index:
```
cd python
python generate_docs.py
cd ..
```
3. Run Sphinx.
```
cd python/docs
make html
```
About this change:
The script at `python/generate_docs.py` creates a ReStructured Text file
for each public module in the protobuf Python package. The script also
updates the table of contents in `python/docs/index.rst` to point to
these module references.
Future work:
Testing the docs build on PRs requires contributors to actually do some
setup work to configure builds on their fork. It'd be better if CI had a
docs build session to verify that the Sphinx docs generation at least
runs.
There are many warnings due to not-quite-correct docstrings in the
actual Python code itself. I'm choosing to ignore these errors to keep
the PR small, but I recommend you fix these and then enable "fail on
warnings" in the docs build on CI.
* add docs to EXTRA_DIST
* add instructions to build documentation to generate_docs.py
* exclude python/odcs from cpp_distcheck
* Add scripts to test multirequest
* chmod ug+x multirequest.sh
* Add continuous test
* Compile c extension
* Class entry is obsolete in the second request
1) Needes to use class name in persistent map
2) Invalidate class entry stored in descriptor
* Add new files to dist
* Fix compile_extension
* Cleanup outputs for phpize
* Make c extension portable for php 7.4
* Fix conformance tests
* Fix comments
* Fix 32-bit
* Update conformance failure list
* Fix compiler warnings
* Cleanup configure created by phpize
The file created in php 7.4 is not recognizable by previous versions
* Fix conformance tests for 64-bit php
* Fix conformance test
* Fix compile warning
* Fix compile warnings
* Add csharp compatibility tests for 3.7.0
* Add compatibility test between last major version with current for C#
* Update last version number
* compatibility tests between last released and the current version
* fix typo
* fix macos build by changing xcode version
* add comments
* fix benchmark build for mac
* fix for gem option and install npm
* add test for node and npm
* fix node
* fix for node
* another attempts to fix node
* another attempts
* clean up
* fix macos build by changing xcode version
* add comments
* fix benchmark build for mac
* fix for gem option and install npm
* add test for node and npm
* fix node
* fix for node
* another attempts to fix node
* another attempts
* clean up
The draconian heapcheck turns up memory leaks outside our control, such
as ones in libc and in the heapchecker itself. I think for an automated
test it makes more sense for us to use the "strict" heapcheck, since
that already passes and should only report leaks that are within our
control.
Bu Sun Kim
Joshua Haberman
Paul Yang
David L. Jones
Thomas Van Lenten
listonjesse
Masaki Hara
Eric Walker
Tim Swast
Brian Wignall
Jie Luo
Tomo Suzuki
Rafi Kamal
David Supplee
Adam Cozzette
Jan Tattermusch
Hao Nguyen
Yilun Chong
Bo Yang