Protocol Buffers - Google's data interchange format (grpc依赖) https://developers.google.com/protocol-buffers/
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

211 lines
5.4 KiB

# Protobuf Global Extension Registry
This file contains a global registry of known extensions for descriptor.proto,
so that any developer who wishes to use multiple 3rd party projects, each with
their own extensions, can be confident that there won't be collisions in
extension numbers.
If you need an extension number for your custom option (see [custom options](
https://developers.google.com/protocol-buffers/docs/proto#customoptions)),
please [send us a pull request](https://github.com/protocolbuffers/protobuf/pulls) to
add an entry to this doc, or [create an issue](https://github.com/protocolbuffers/protobuf/issues)
with info about your project (name and website) so we can add an entry for you.
## Existing Registered Extensions
1. C# port of protocol buffers
* Website: https://github.com/jskeet/protobuf-csharp-port
* Extensions: 1000
1. Perl/XS port of protocol buffers
* Website: http://code.google.com/p/protobuf-perlxs
* Extensions: 1001
1. Objective-C port of protocol buffers
* Website: http://code.google.com/p/protobuf-objc
* Extensions: 1002
1. Google Wave Federation Protocol open-source release (FedOne)
* Website: http://code.google.com/p/wave-protocol
* Extensions: 1003
1. PHP code generator plugin
* Website: ???
* Extensions: 1004
1. GWT code generator plugin (third-party!)
* Website: http://code.google.com/p/protobuf-gwt/
* Extensions: 1005
1. Unix Domain RPC code generator plugin
* Website: http://go/udrpc
* Extensions: 1006
1. Object-C generator plugin (Plausible Labs)
* Website: http://www.plausible.coop
* Extensions: 1007
1. TBD (code42.com)
* Website: ???
* Extensions: 1008
1. Goby Underwater Autonomy Project
* Website: https://github.com/GobySoft/goby
* Extensions: 1009
1. Nanopb
* Website: http://kapsi.fi/~jpa/nanopb
* Extensions: 1010
1. Bluefin AUV Communication Extensions
* Website: http://www.bluefinrobotics.com
* Extensions: 1011
1. Dynamic Compact Control Language
* Website: http://github.com/GobySoft/dccl
* Extensions: 1012
1. ScaleOut StateServer® Native C++ API
* Website: http://www.scaleoutsoftware.com
* Extensions: 1013
1. FoundationDB SQL Layer
* Website: https://github.com/FoundationDB/sql-layer
* Extensions: 1014
1. Fender
* Website: https://github.com/hassox/fender
* Extensions: 1015
1. Vortex
* Website: http://www.prismtech.com/vortex
* Extensions: 1016
1. tresorit
* Website: https://tresorit.com/
* Extensions: 1017
1. CRIU (Checkpoint Restore In Userspace)
* Website: http://criu.org/Main_Page
* Extensions: 1018
1. protobuf-c
* Website: https://github.com/protobuf-c/protobuf-c
* Extensions: 1019
1. ScalePB
* Website: http://trueaccord.github.io/ScalaPB/
* Extensions: 1020
1. protoc-gen-bq-schema
* Website: https://github.com/GoogleCloudPlatform/protoc-gen-bq-schema
* Extensions: 1021
1. grpc-gateway
* Website: https://github.com/gengo/grpc-gateway
* Extensions: 1022
1. Certificate Transparency
* Website: https://github.com/google/certificate-transparency
* Extensions: 1023
1. JUNOS Router Telemetry
* Website: http://www.juniper.net
* Extensions: 1024
1. Spine Event Engine
* Website: https://github.com/SpineEventEngine/core-java
* Extensions: 1025
1. Aruba cloud platform
* Website: ???
* Extensions: 1026 -> 1030
1. Voltha
* Website: ???
* Extensions: 1031 -> 1033
1. gator
* Website: ???
* Extensions: 1034
1. protoc-gen-flowtypes
* Website: https://github.com/tmc/grpcutil/tree/master/protoc-gen-flowtypes
* Extensions: 1035
1. ProfaneDB
* Website: https://gitlab.com/ProfaneDB/ProfaneDB
* Extensions: 1036
1. protobuf-net
* Website: https://github.com/mgravell/protobuf-net
* Extensions: 1037
1. FICO / StreamEngine
* Website: http://www.fico.com/
* Extensions: 1038
1. GopherJS
* Website: https://github.com/johanbrandhorst/protobuf
* Extensions: 1039
1. ygot
* Website: https://github.com/openconfig/ygot
* Extensions: 1040
1. go-grpcmw
* Website: https://github.com/MarquisIO/go-grpcmw
* Extensions: 1041
1. grpc-gateway protoc-gen-swagger
* Website: https://github.com/grpc-ecosystem/grpc-gateway
* Extensions: 1042
1. AN Message
* Website: TBD
* Extensions: 1043
1. protofire
* Website: https://github.com/ribrdb/protofire
* Extensions: 1044
1. Gravity
* Website: https://github.com/aphysci/gravity
* Extensions: 1045
1. SEMI Standards – I&C Technical Committee
* Website: http://downloads.semi.org/web/wstdsbal.nsf/9c2b317e76523cca88257641005a47f5/88a5863a580e323088256e7b00707489!OpenDocument
* Extensions: 1046
1. Elixir plugin
* Website: https://github.com/tony612/grpc-elixir
* Extensions: 1047
1. API client generators
* Website: ???
* Extensions: 1048-1056
1. Netifi Proteus
* Website: https://github.com/netifi-proteus
* Extensions: 1057
1. CGSN Mooring Project
* Website: https://bitbucket.org/ooicgsn/cgsn-mooring
* Extensions: 1058
1. Container Storage Interface
* Website: https://github.com/container-storage-interface/spec
* Extensions: 1059-1069
1. TwirpQL Plugin
* Website: https://twirpql.dev
* Extensions: 1070
1. Protoc-gen-validate
* Website: https://github.com/envoyproxy/protoc-gen-validate
* Extensions: 1071
Requesting extension id 1072 This library supports an idiomatic proto3 protobuf generator for kotlin. The library will be open sourced by Toast Inc under the Apache2 license, and is currently used in production at Toast. The following is the readme.md that will be released with the code by the end of Q4 2019. Supports only the Protocol Buffers language version 3. #### Features - Clean data class generation - Oneof types handled as sealed classes - JavaDoc comments on generated code - Deprecation option pass-through to Kotlin's `@Deprecated` annotation - Protokt-specific options: non-null types, wrapper types, interface implementation, and more - Tight integration with Protobuf's Java library: compatibility with its well-known types and usage of CodedInputStream and CodedOutputStream for best performance #### Not yet implemented - Kotlin native support - Kotlin JS support - Support for gRPC service generation - Protobuf JSON support See examples in [protokt-testing](https://github.com/toasttab/protokt/tree/master/protokt-testing). ### Generated Code Generated code is placed in `<buildDir>/generated-sources/main/protokt`. A simple example: ```proto syntax = "proto3"; package com.protokt.sample; message Sample { string sample_field = 1; } ``` will produce: ```kotlin /* * Generated by protokt. Do not modify. */ package com.protokt.sample import com.toasttab.protokt.rt.* data class Sample( val sampleField: String, val unknown: Map<Int, Unknown> = emptyMap() ) : KtMessage { @Suppress("UNUSED") constructor( sampleField: String = "" ) : this( sampleField, emptyMap() ) override val messageSize by lazy { sizeof() } override fun serialize(serializer: KtMessageSerializer) { if (sampleField.isNotEmpty()) { serializer.write(Tag(10)).write(sampleField) } if (unknown.isNotEmpty()) { serializer.writeUnknown(unknown) } } private fun sizeof(): Int { var res = 0 if (sampleField.isNotEmpty()) { res += sizeof(Tag(1)) + sizeof(sampleField) } res += unknown.entries.sumBy { it.value.sizeof() } return res } companion object Deserializer : KtDeserializer<Sample> { override fun deserialize(deserializer: KtMessageDeserializer): Sample { var sampleField = "" val unknown = mutableMapOf<Int, Unknown>() while (true) { when (deserializer.readTag()) { 0 -> return Sample( sampleField, unknown ) 10 -> sampleField = deserializer.readString() else -> { val unk = deserializer.readUnknown() unknown[unk.fieldNum] = unknown[unk.fieldNum].let { when (it) { null -> unk else -> when (val v = it.value) { is ListVal -> Unknown(unk.fieldNum, ListVal(v.value + unk.value)) else -> Unknown(unk.fieldNum, ListVal(listOf(v, unk.value))) } } } } } } } } } ``` #### Runtime Notes ##### Package The Kotlin package of a generated file can be overridden from protobuf package with the `(protokt).package` option: ```proto syntax = "proto3"; import "protokt.proto"; package com.example; option (protokt).package = "com.package"; ``` ##### Message Each protokt message implements the `KtMessage` interface. `KtMessage` defines the `serialize()` method and its overloads which can serialize to a byte array, a `KtMessageSerializer`, or on the JVM, an `OutputStream`. Each protokt message has a companion object `Deserializer` that implements the `KtDeserializer` interface, which provides the `deserialize()` method and its overloads to construct an instance of the message from a byte array, a Java InputStream, or others. In order to enjoy the full benefits of Kotlin data classes, byte arrays are wrapped in the protokt `Bytes` class, which provides appropriate `equals()` and `hashCode()` implementations. ##### Enums Enum fields are generated as data classes with a single integer field. Kotlin enum classes are closed and cannot retain unknown values, and protobuf requires that unknown enum values are preserved for reserialization. This compromise exposes a constructor taking an integer, but the `from(value: Int)` on an enum's `Deserializer` should be preferred as it avoids instantiation when possible. Other notes: - `optimize_for` is ignored. - `repeated` fields are deserialized to Lists. - `map` fields are deserialized to Maps. - `oneof` fields are represented as data class subtypes of a sealed base class with a single property. ### Extensions See examples of each option in the [protokt-options](https://github.com/toasttab/protokt/tree/master/protokt-testing/protokt-options/src/main/proto) module. All protokt-specific options require importing `protokt.proto` in the protocol file. #### Wrapper Types Sometimes a field on a protobuf message corresponds to a concrete nonprimitive type. In standard protobuf the user would be responsible for this extra transformation, but the protokt wrapper type option allows specification of a converter that will automatically encode and decode custom types to protobuf primitives and well-known types. Some standard types are implemented in [protokt-extensions](https://github.com/toasttab/protokt/tree/master/protokt-extensions/src/main/kotlin/com/toasttab/protokt/ext). Wrap a field by invoking the `(protokt_property).wrap` option: ```proto message DateWrapperMessage { int64 date = 1 [ (protokt_property).wrap = "java.util.Date" ]; } ``` Converters implement the `Converter` interface: ```kotlin interface Converter<S: Any, T: Any> { val wrapper: KClass<S> fun wrap(unwrapped: T): S fun unwrap(wrapped: S): T } ``` and protokt will reference the converter's methods to wrap and unwrap from protobuf primitives: ```kotlin object DateConverter : Converter<Date, Long> { override val wrapper = Date::class override fun wrap(unwrapped: Long) = Date(unwrapped) override fun unwrap(wrapped: Date) = wrapped.time } ``` ```kotlin data class WrapperModel( val date: java.util.Date, ... ) : KtMessage { ... override fun serialize(serializer: KtMessageSerializer) { serializer.write(Tag(10)).write(Int64(DateConverter.unwrap(date))) ... } override fun deserialize(deserializer: KtMessageDeserializer): WrapperModel { var date = 0L while (true) { when (deserializer.readTag()) { 0 -> return WrapperModel( DateConverter.wrap(date), ... ) ... } } } } ``` Converters can also implement the `OptimizedSizeofConverter` interface adding `sizeof()`, which allows them to optimize the calculation of the wrapper's size rather than unwrap the object twice. For example, a UUID is always 16 bytes: ```kotlin object UuidConverter : OptimizedSizeofConverter<UUID, ByteArray> { override val wrapper = UUID::class private val sizeofProxy = ByteArray(16) override fun sizeof(wrapped: UUID) = sizeof(sizeofProxy) override fun wrap(unwrapped: ByteArray): UUID { require(unwrapped.size == 16) { "input must have size 16; had ${unwrapped.size}" } return ByteBuffer.wrap(unwrapped) .run { UUID(long, long) } } override fun unwrap(wrapped: UUID) = ByteBuffer.allocate(16) .putLong(wrapped.mostSignificantBits) .putLong(wrapped.leastSignificantBits) .array() } ``` Rather than convert a UUID to a byte array both for size calculation and for serialization (which is what a naïve implementation would do), UuidConverter always returns the size of a constant 16-byte array. If the wrapper type is in the same package as the generated protobuf message, then it does not need a fully-qualified name. Custom wrapper type converters can be in the same module as protobuf types that reference them. In order to use any wrapper type defined in `protokt-extensions`, the module must be included as a dependency: ```groovy dependencies { implementation 'com.toasttab.protokt:protokt-extensions:0.0.3' } ``` #### Interface implementation To avoid the need to create domain-specific objects from protobuf messages you can declare that a protobuf message implements a custom interface with properties and default methods. ```kotlin package com.protokt.sample interface Model { val id: String } ``` ```proto package com.protokt.sample; message ImplementsSampleMessage { option (protokt_class).implements = "Model"; string id = 1; } ``` If the wrapper interface is in the same package as the generated protobuf message, then it does not need a fully-qualified name. Wrapper interfaces cannot be used by protobuf messages in the same module that defines them; the dependency must be declared with`protoktExtensions` in `build.gradle`: ```groovy dependencies { protoktExtensions project(':api-module') } ``` #### Nonnull fields If there is a message that has no meaning whatsoever when a particular field is missing, you can emulate proto2's `required` key word by using the `(protokt_oneof).non_null` option: ```proto message Sample { } message NonNullSampleMessage { Sample non_null_sample = 1 [ (protokt_property).non_null = true ]; } ``` Generated code will not have a nullable type so the field can be referenced without using Kotlin's `!!`. Oneof fields can also be declared non-null: ```proto message NonNullSampleMessage { oneof non_null_oneof { option (protokt_oneof).non_null = true; string message = 2; } } ``` Note that deserialization of a message with a non-nullable field will fail if the message being decoded does not contain an instance of the required field. #### BytesSlice When reading messages that contain other serialized messages as `bytes` fields, protokt can keep a reference to the originating byte array to prevent a large copy operation on deserialization. This can be desirable when the wrapping message is a thin metadata shim and doesn't include much memory overhead: ```proto message SliceModel { int64 version = 1; bytes encoded_message = 2 [ (protokt_property).bytes_slice = true ]; } ``` ### Usage #### Gradle ```groovy buildscript { dependencies { classpath "com.toasttab.protokt:protokt-gradle-plugin:0.0.3" } } apply plugin: 'com.toasttab.protokt' ``` This will automatically download and install protokt, apply the Google protobuf plugin, and configure all the necessary boilerplate. By default it will also add `protokt-runtime` to the api scope of the project, and `protobuf-java` to the implementation scope. If your project is pure Kotlin you may run into the following error: ``` Execution failed for task ':compileJava'. > error: no source files ``` To work around it, disable all `JavaCompile` tasks in the project: ```groovy tasks.withType(JavaCompile) { enabled = false } ``` or: ```groovy compileJava.enabled = false ``` #### Command line code generation ```bash protokt-codegen$ ./gradlew assemble [OR ./gradlew installDist] protokt-codegen$ ./run-protokt.sh -h protokt-codegen$ ./run-protokt.sh \ -out=../kotlin \ -pkg=com.toasttab.protokt.conformance \ -file=conformance.proto \ -cp=../build/libs/protokt-codegen-0.0.3-SNAPSHOT-all.jar \ -plugin=../bin/protokt.sh ``` ### Contribution To enable rapid development of the code generator, the protobuf conformance tests have been compiled and included in the protokt-testing project. They run on Mac OS 10.14+ and Ubuntu 16.04 x86-64. Publish the plugin to the integration repository: ```bash protokt$ ./gradlew publishToIntegrationRepository ``` Then run the tests from `protokt-testing`: ```bash protokt-testing$ ./gradlew protokt-conformance-tests:test ``` All integration tests can be run with: ``` protokt-testing$ ./gradlew test ```
6 years ago
1. protokt
* Website: https://github.com/toasttab/protokt (Currently Private but will be open soon.)
* Extensions: 1072