|
|
|
|
|
|
|
|
|
# μpb: small, fast C protos
|
|
|
|
|
|
|
|
|
|
μpb (often written 'upb') is a small
|
|
|
|
|
[protobuf](https://github.com/protocolbuffers/protobuf) implementation written
|
|
|
|
|
in C.
|
|
|
|
|
|
|
|
|
|
upb is the core runtime for protobuf languages extensions in
|
|
|
|
|
[Ruby](https://github.com/protocolbuffers/protobuf/tree/master/ruby),
|
[protobuf] Upgrade third_party/protobuf to 22.x (#32606)
The very non-trivial upgrade of third_party/protobuf to 22.x
This PR strives to be as small as possible and many changes that were
compatible with protobuf 21.x and didn't have to be merged atomically
with the upgrade were already merged.
Due to the complexity of the upgrade, this PR wasn't created
automatically by a tool, but manually. Subsequent upgraded of
third_party/protobuf with our OSS release script should work again once
this change is merged.
This is best reviewed commit-by-commit, I tried to group changes in
logical areas.
Notable changes:
- the upgrade of third_party/protobuf submodule, the bazel protobuf
dependency itself
- upgrade of UPB dependency to 22.x (in the past, we used to always
upgrade upb to "main", but upb now has release branch as well). UPB
needs to be upgraded atomically with protobuf since there's a de-facto
circular dependency (new protobuf depends on new upb, which depends on
new protobuf for codegen).
- some protobuf and upb bazel rules are now aliases, so `
extract_metadata_from_bazel_xml.py` and `gen_upb_api_from_bazel_xml.py`
had to be modified to be able to follow aliases and reach the actual
aliased targets.
- some protobuf public headers were renamed, so especially
`src/compiler` needed to be updated to use the new headers.
- protobuf and upb now both depend on utf8_range project, so since we
bundle upb with grpc in some languages, we now have to bundle utf8_range
as well (hence changes in build for python, PHP, objC, cmake etc).
- protoc now depends on absl and utf8_range (previously protobuf had
absl dependency, but not for the codegen part), so python's
make_grpcio_tools.py required partial rewrite to be able to handle those
dependencies in the grpcio_tools build.
- many updates and fixes required for C++ distribtests (currently they
all pass, but we'll probably need to follow up, make protobuf's and
grpc's handling of dependencies more aligned and revisit the
distribtests)
- bunch of other changes mostly due to overhaul of protobuf's and upb's
internal build layout.
TODOs:
- [DONE] make sure IWYU and clang_tidy_code pass
- create a list of followups (e.g. work to reenable the few tests I had
to disable and to remove workaround I had to use)
- [DONE in cl/523706129] figure out problem(s) with internal import
---------
Co-authored-by: Craig Tiller <ctiller@google.com>
2 years ago
|
|
|
[PHP](https://github.com/protocolbuffers/protobuf/tree/master/php), and
|
|
|
|
|
[Python](https://github.com/protocolbuffers/upb/tree/main/python).
|
|
|
|
|
|
|
|
|
|
While upb offers a C API, the C API & ABI **are not stable**. For this reason,
|
|
|
|
|
upb is not generally offered as a C library for direct consumption, and there
|
|
|
|
|
are no releases.
|
|
|
|
|
|
|
|
|
|
## Features
|
|
|
|
|
|
|
|
|
|
upb has comparable speed to protobuf C++, but is an order of magnitude smaller
|
|
|
|
|
in code size.
|
|
|
|
|
|
|
|
|
|
Like the main protobuf implementation in C++, it supports:
|
|
|
|
|
|
|
|
|
|
- a generated API (in C)
|
|
|
|
|
- reflection
|
|
|
|
|
- binary & JSON wire formats
|
|
|
|
|
- text format serialization
|
|
|
|
|
- all standard features of protobufs (oneofs, maps, unknown fields, extensions,
|
|
|
|
|
etc.)
|
|
|
|
|
- full conformance with the protobuf conformance tests
|
|
|
|
|
|
|
|
|
|
upb also supports some features that C++ does not:
|
|
|
|
|
|
|
|
|
|
- **optional reflection:** generated messages are agnostic to whether
|
|
|
|
|
reflection will be linked in or not.
|
|
|
|
|
- **no global state:** no pre-main registration or other global state.
|
|
|
|
|
- **fast reflection-based parsing:** messages loaded at runtime parse
|
|
|
|
|
just as fast as compiled-in messages.
|
|
|
|
|
|
|
|
|
|
However there are a few features it does not support:
|
|
|
|
|
|
|
|
|
|
- text format parsing
|
|
|
|
|
- deep descriptor verification: upb's descriptor validation is not as exhaustive
|
|
|
|
|
as `protoc`.
|
|
|
|
|
|
|
|
|
|
## Install
|
|
|
|
|
|
|
|
|
|
For Ruby, use [RubyGems](https://rubygems.org/gems/google-protobuf):
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
$ gem install google-protobuf
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
For PHP, use [PECL](https://pecl.php.net/package/protobuf):
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
$ sudo pecl install protobuf
|
|
|
|
|
```
|
|
|
|
|
|
[protobuf] Upgrade third_party/protobuf to 22.x (#32606)
The very non-trivial upgrade of third_party/protobuf to 22.x
This PR strives to be as small as possible and many changes that were
compatible with protobuf 21.x and didn't have to be merged atomically
with the upgrade were already merged.
Due to the complexity of the upgrade, this PR wasn't created
automatically by a tool, but manually. Subsequent upgraded of
third_party/protobuf with our OSS release script should work again once
this change is merged.
This is best reviewed commit-by-commit, I tried to group changes in
logical areas.
Notable changes:
- the upgrade of third_party/protobuf submodule, the bazel protobuf
dependency itself
- upgrade of UPB dependency to 22.x (in the past, we used to always
upgrade upb to "main", but upb now has release branch as well). UPB
needs to be upgraded atomically with protobuf since there's a de-facto
circular dependency (new protobuf depends on new upb, which depends on
new protobuf for codegen).
- some protobuf and upb bazel rules are now aliases, so `
extract_metadata_from_bazel_xml.py` and `gen_upb_api_from_bazel_xml.py`
had to be modified to be able to follow aliases and reach the actual
aliased targets.
- some protobuf public headers were renamed, so especially
`src/compiler` needed to be updated to use the new headers.
- protobuf and upb now both depend on utf8_range project, so since we
bundle upb with grpc in some languages, we now have to bundle utf8_range
as well (hence changes in build for python, PHP, objC, cmake etc).
- protoc now depends on absl and utf8_range (previously protobuf had
absl dependency, but not for the codegen part), so python's
make_grpcio_tools.py required partial rewrite to be able to handle those
dependencies in the grpcio_tools build.
- many updates and fixes required for C++ distribtests (currently they
all pass, but we'll probably need to follow up, make protobuf's and
grpc's handling of dependencies more aligned and revisit the
distribtests)
- bunch of other changes mostly due to overhaul of protobuf's and upb's
internal build layout.
TODOs:
- [DONE] make sure IWYU and clang_tidy_code pass
- create a list of followups (e.g. work to reenable the few tests I had
to disable and to remove workaround I had to use)
- [DONE in cl/523706129] figure out problem(s) with internal import
---------
Co-authored-by: Craig Tiller <ctiller@google.com>
2 years ago
|
|
|
For Python, use [PyPI](https://pypi.org/project/protobuf/):
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
$ sudo pip install protobuf
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Alternatively, you can build and install upb using
|
|
|
|
|
[vcpkg](https://github.com/microsoft/vcpkg/) dependency manager:
|
|
|
|
|
|
|
|
|
|
git clone https://github.com/Microsoft/vcpkg.git
|
|
|
|
|
cd vcpkg
|
|
|
|
|
./bootstrap-vcpkg.sh
|
|
|
|
|
./vcpkg integrate install
|
|
|
|
|
./vcpkg install upb
|
|
|
|
|
|
|
|
|
|
The upb port in vcpkg is kept up to date by microsoft team members and community
|
|
|
|
|
contributors.
|
|
|
|
|
|
|
|
|
|
If the version is out of date, please
|
|
|
|
|
[create an issue or pull request](https://github.com/Microsoft/vcpkg) on the
|
|
|
|
|
vcpkg repository.
|
|
|
|
|
|
|
|
|
|
## Contributing
|
|
|
|
|
|
|
|
|
|
Please see [CONTRIBUTING.md](CONTRIBUTING.md).
|
