This PR implements lookup of service descriptor and method descriptor objects in Ruby as described in issue https://github.com/protocolbuffers/protobuf/issues/14891.
It contains three implementations - one for the CRuby extension API, one for JRuby, and one for FFI.
With this patch,
* `DescriptorPool#lookup('fully.qualified.service.name')` works and returns a `Google::Protobuf::ServiceDescriptor` object
* You can call `#options` on that to get the service options
* You can call `#methods` on that to get the services' methods as `Google::Protobuf::MethodDescriptor` objects,
* You can call `MethodDescriptor#options` to get method options
* You can also get the streaming flags & input/output types of the method with `#input_type`, `#output_type`, `#client_streaming`, and `#server_streaming`.
In order to make the FFI implementation work, I had to mark some more methods in the UPB header as exported - I guess that's something which will have to be done on the UPB side, like this 01fed1cc1b
CC @dazuma & @haberman from the original issue, and @JasonLunn (since you work on protobuf it seems - small world!)
I apologies for the large volume of copy-pasta'd code from the existing descriptor class implementations into the new ones - I felt this was probably better than designing new abstractions to reduce it off the bat though; this feels like it "fits in" with the existing implementation.
Closes#15817
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/15817 from KJTsanaktsidis:ktsanaktsidis/add_service_method_descriptors 54d7218431
PiperOrigin-RevId: 618221016
This migrates from reusable workflows to composite GHA actions. This has the following advantages:
1) We can split them into smaller, easier to reason about chunks
2) We can reuse them more freely between each other and workflows
3) They don't complicate the job names
4) In theory they'll be easier to test in isolation. While composite actions can't be unit-tested, we can easily break them up into nodejs or docker actions that can be.
As a proof of concept, some of our non-Bazel tests are migrated to GHA as well (CMake + Composer)
Closes#11718
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11718 from protocolbuffers:gha-actions 5403307bc00616e94816aa460813939d8f37a1bd
PiperOrigin-RevId: 506375417
This uses https://github.com/protocolbuffers/rules_ruby to fully Bazelify our ruby runtime code. The Rakefile is left in place for now and is still used by our aarch64 tests. With the current implementation ruby behaves similarly to our python wrapper, which selects whatever version is installed in the system. Future enhancements will allow for more hermetic builds via Bazel flags to pin a specific version
Closes#10525
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/10525 from mkruskal-google:rules_ruby 97fa1f70ab
PiperOrigin-RevId: 499283908
* Bazelfying conformance tests
Adding infrastructure to "Bazelify" languages other than Java and C++
* Delete benchmarks for languages supported by other repositories
* Bazelfying benchmark tests
* Bazelfying python
Use upb's system python rule instead of branching tensorflow
* Bazelfying Ruby
* Bazelfying C#
* Bazelfying Objective-c
* Bazelfying Kokoro mac builds
* Bazelfying Kokoro linux builds
* Deleting all deprecated files from autotools cleanup
This boils down to Makefile.am and tests.sh and all of their remaining references
* Cleanup after PR reorganizing
- Enable 32 bit tests
- Move conformance tests back
- Use select statements to select alternate runtimes
- Add internal prefixes to proto library macros
* Updating READMEs to use bazel instead of autotools.
* Bazelfying Kokoro release builds
* First round of review fixes
* Second round of review fixes
* Third round of review fixes
* Filtering out conformance tests from Bazel on Windows (b/241484899)
* Add version metadata that was previously scraped from configure.ac
* fixing typo from previous fix
* Adding ruby version tests
* Bumping pinned upb version, and adding tests to python CI
* Initial implementation of cmake tests for linux
* Reverting accidental distcheck changes
* Updating gitignore file to include cmake generated files
* Updating existing cmake tests to use newer docker image
* Adding CMake build that uses Ninja as the generator
* Updating CMake documentation to cover Linux support
* Updating to latest cmake image
When following the instructions directly (for developing on Windows) we should end up with a generator that's easy to run for csharp/generate_protos.sh, and we shouldn't end up trying to stage the build output.
This PR tells git to ignore ephemeral files that are created when
developers use clangd.
clangd needs a `compile_commands.json` file at the root of the repo (or
in a `build/` directory) to tell it how to compile each file. CMake can
generate this file with something like the following:
```console
$ cmake -Scmake -B.build \
-DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
-Dprotobuf_BUILD_TESTS=OFF
$ ln -sf .build/compile_commands.json .
```
Then clangd will compile and index files into the `.cache` directory.
* Port for php8
* Port php c extension for php8
* Update composer.json
* Drop php7.0 support
* Update phpunit for php7.1 in c extension test
* Add back support for php7.0
* Add badge for php8 continuous build
* Only ported c extension to php8.
* Didn't fixed the issue of throwing warnings for missing arginfo in bundled files.
* Tests not fixed, because syntax of phpunit (<7 vs >9.3) are not compatible.
* In next release, needs to drop php5 and php7.0 support (in order to use phpunit > 7)
* Fix a typo
* Fix lots of spelling errors
* Fix a few more spelling mistakes
* s/parsable/parseable/
* Don't touch the third party files
* Cloneable is the preferred C# term
* Copyable is the preferred C++ term
* Revert "s/parsable/parseable/"
This reverts commit 534ecf7675.
* Revert unparseable->unparsable corrections
* python: generate documentation with Sphinx and Read the Docs
Background:
Formerly, the Python protobuf reference documentation was built with
[Epydoc](http://epydoc.sourceforge.net/). This package has not been
updated since 2008, and it has inconsistent formatting (see internal
issue 131415575) with most Python documentation. Sphinx is used for the
official docs.python.org docs as well as most other Python packages,
including the Google client libraries and related packages, such as
https://googleapis.dev/python/google-api-core/latest/
To build the docs with Sphinx:
1. Install the needed packages (`sphinx`, `sphinxcontrib-napoleon` for
Google-style docstring support). I've created a conda environment file
to make this easier:
```
conda env create -f python/docs/environment.yml
```
2. (Optional) Generate reference docs files and regenerate index:
```
cd python
python generate_docs.py
cd ..
```
3. Run Sphinx.
```
cd python/docs
make html
```
About this change:
The script at `python/generate_docs.py` creates a ReStructured Text file
for each public module in the protobuf Python package. The script also
updates the table of contents in `python/docs/index.rst` to point to
these module references.
Future work:
Testing the docs build on PRs requires contributors to actually do some
setup work to configure builds on their fork. It'd be better if CI had a
docs build session to verify that the Sphinx docs generation at least
runs.
There are many warnings due to not-quite-correct docstrings in the
actual Python code itself. I'm choosing to ignore these errors to keep
the PR small, but I recommend you fix these and then enable "fail on
warnings" in the docs build on CI.
* add docs to EXTRA_DIST
* add instructions to build documentation to generate_docs.py
* exclude python/odcs from cpp_distcheck
* Add scripts to test multirequest
* chmod ug+x multirequest.sh
* Add continuous test
* Compile c extension
* Class entry is obsolete in the second request
1) Needes to use class name in persistent map
2) Invalidate class entry stored in descriptor
* Add new files to dist
* Fix compile_extension
* Cleanup outputs for phpize
* Avoid initializing primitive fields in layout_init
* Avoid initializing string/bytes/message fields in layout_init
* Lazily create map when needed
* Lazily create repeated fields
* Change layout_init to only do memcpy
* Fix test for php-7.0
* Fix conformance test where default value of string/message map is not encoded
* Fix test for zts
* Clean up
* Fix comments
* Add binary conformance tests for map fields
* Update failure list
* Fix php conformance test
* Fix php conformance test
In 32-bit platform, int64 should be string. However, map iterator returns string key as integer.
* Add more test cases for map
* Update failure list
* Inherit from message instead of implement
When regestering class (implmenets other class) during MINIT, zend_class_implements
would call zend_class_entry->interface_gets_implemented(). In PHP-7.3 interface_gets_implemented shares the same location with create_object.
However, during MINIT, the global object storeage hasn't been initialized. And thus, caused segment fault in php 7.3.
* Use zend_string_init where interned string may be the value.
zend_string_dup will keep using the existing interned string.
In php 7.3, interned string cannot be destroyed from user's code.
* Uncommment debug code
* Use latest phpunit for each php versions
* Revert change in Dockerfile
* Update php test to use the new docker image
* Update composer
* Change docker organization
* Update phpunit
* Debug phpunit
* Store phpunit into bin dir in docker image
* Install valgrind to docker
* Fix compatibility test
* Remove generated_service_test from compatibility c extension test
* Update 32bit php test to the new docker image
* Install bison
* Fix build.sh
* Fix DOCKERIMAGE_PREFIX
* Fix basename
* Add comment to build_and_run_docker2.sh
* Remove commented code
* Fix comments
* Implement json decoding for Any message.
type url may not appear as the first value in json. As a result,
other data cannot be resolved before resolving type url. To solve that,
this change caches the start and end position of unparsed values and
resolve them in end_any_object when type url has been resolved.
* Handle Any in switch
* Update json parser size
* Fix comments
* Sync upstream
* Add dependency on upb_pb for upb_json
* Debug failed test
* Fix cmake
* Update test generated files
* Remove debug tests
* Add support for primitive types in setters
* Update to address PR feedback
* Add tests and fixes for repeated fields
* Remove repeated field code, add getters
* Cleanup, test getters and oneofs
* Move boxing logic into separate class
* Add tests for wrapper type constructor args
* Update to add new setXXXValue methods
* Fix tests for invalid values
* Fix c extension for wrapper accessors
* Fix the bug that well known types didn't call Message_construct
* Address PR comments
* Refactoring init message with array logic
* Add include path to protoc
* Add missing TSRM_LS defintion
* Fix TSRM_LS
* Fix dist check
This change only adds basic proto2 support without advanced features
like extensions, custom options, maps, etc.
The protoc binary now generates ruby code for proto2 syntax.
However, for now, it is restricted to proto2 files without advanced features
like extensions, in which case it still errors out.
This change also modifies the DSL to add proto messages to the DescriptorPool.
There is a new DSL Builder#add_file to create a new FileDescriptor. With this,
the generated ruby DSL looks something like:
Google::Protobuf::DescriptorPool.generated_pool.build do
add_file "test.proto" do
add_message "foo" do
optional :val, :int32, 1
end
end
end
The compiler and util subpackages are created by the build_py class in
setup.py. This has caused an issue in the protobuf package in
conda-forge (https://github.com/conda-forge/protobuf-feedstock/issues/40),
which is fixed by this commit.
KJ Tsanaktsidis
Adam Cozzette
Matt Fowles Kulukundis
Mike Kruskal
Mike Kruskal
Sandy Zhang
Elliotte Rusty Harold
Elliotte Rusty Harold
Joshua Haberman
Jon Skeet
David L. Jones
Greg Miller
Joshua Haberman
Derek Perez
aaron ai
Rob Widmer
Paul Yang
Peter Newman
James Newton-King
listonjesse
Tim Swast
ST-DDT
michaelbausor
Harshit Chopra
Yuzhang Hu
mkosieradzki
Feng Xiao
Thomas Hisch