Tag:
Branch:
Tree:
1363bf9c05
21.x
21.x-20241125
22.x
22.x-202303072154
22.x-202304122338
23.0-rc2-patch
23.x
23.x-202305051714
23.x-202305081751
23.x-202305171614
24.x
25.x
25.x-202311012212
25.x-allow-compat-test
25.x-compat-tests
25.x-compat-upgrade
26.x
27.x
28.x
28.x-202408082011
28.x-202409182056
29.x
3.0.x
3.1.x
3.10.x
3.11.x
3.12.x
3.13.x
3.14.x
3.15.x
3.16.x
3.17.x
3.18.x
3.19.x
3.2.x
3.20.x
3.3.x
3.5.x
3.6.x
3.7.x
3.8.x
3.9.x
4.0.x
add-support-for-options-in-ruby
add_missing_headers
b7-all
bazel-rules
bazel-rules2
bazel7-deb11
bazel7-macos-cp
bcr
bootstrap_upb_fix
burndown_patches
bzlmod
cherrypickStatic
cherrypicks
cp-bzl
cp-java-feature-bootstrap
cp-java-generator
cp-segv
deannagarcia-patch-1
deannagarcia-patch-10
deannagarcia-patch-11
deannagarcia-patch-12
deannagarcia-patch-2
deannagarcia-patch-3
deannagarcia-patch-4
deannagarcia-patch-5
deannagarcia-patch-6
deannagarcia-patch-7
deannagarcia-patch-8
deannagarcia-patch-9
delete-internal-files
disable-upload-artifacts-action
dmaclach-mach_absolute_time
editions-27
ffi-fix
fix-25.x-staleness
fix_jruby_proto3_optional
gha
gha-actions
gha-migration
gha-migration2
gha-migration3
gha-test
ghaPassFail
java8
jruby_9.4.6.0
main
main-202302161846
main-202303072257
main-202304252156
main-202305082101
main-202310162054
main-202401251548
main-202404180211
main-202407112335
main-202409302244
main-tmp-1
main-tmp-2
main-tmp-3
main-tmp-test-branch
mavenTests
nucleus-upb
php-regen
pkgconfig2-22
pomTrial
reenable_cruby_ffi_tests
regen-upb
revert-12721-main-202305082101
revert-18339-bazel-rules2
rm-php-compat-code
ruby-ffi-freezing
ruby-json-pool-fix
ruby-rm-syntax
ruby_artifact_uploads
ruby_dep
set-ex-test
shaod2-patch-1
support_ruby_3.4.x
test_458711847
test_472529298
test_477475045
test_488676475
test_488736123
test_488863171
test_488996168
test_489037129
test_489075734
test_489259176
test_490412672
test_490480534
test_490544827
test_490775181
test_491410618
test_492007398
test_492047164
test_492051638
test_492365076
test_492380119
test_493086551
test_493088412
test_493584811
test_493743618
test_493886620
test_494027573
test_494043842
test_494271758
test_494398568
test_494723105
test_494824070
test_494835245
test_495003404
test_495019683
test_495043824
test_495125557
test_495293733
test_495294072
test_495352403
test_495412624
test_495507057
test_500390065
test_559560115
test_642287955
test_697418422
test_698410801
test_698580166
test_699885982
test_699987596
test_700079644
test_700089185
test_700734396
test_700755421
test_700786602
test_700787434
test_700826871
test_700829684
test_700864625
test_700867557
test_700867558
test_700867559
test_700914351
test_701852441
test_701959909
test_701963635
test_701976718
test_702043838
test_702046760
test_702052830
test_702066679
test_702103016
test_702103907
test_702108781
test_702127616
test_kfm
test_ruby
win2019-23.x
zhangskz-patch-1
3.15.0-rc1
conformance-build-tag
v16.2
v18.3
v19.5
v2.4.1
v2.5.0
v2.6.0
v2.6.1
v2.6.1rc1
v20.2
v21.0
v21.0-rc1
v21.0-rc2
v21.1
v21.10
v21.11
v21.12
v21.2
v21.3
v21.4
v21.5
v21.6
v21.7
v21.8
v21.9
v22.0
v22.0-rc1
v22.0-rc2
v22.0-rc3
v22.1
v22.2
v22.3
v22.4
v22.5
v23.0
v23.0-rc1
v23.0-rc2
v23.0-rc3
v23.1
v23.2
v23.3
v23.4
v24.0
v24.0-rc1
v24.0-rc2
v24.0-rc3
v24.1
v24.2
v24.3
v24.4
v25.0
v25.0-rc1
v25.0-rc2
v25.1
v25.2
v25.3
v25.4
v25.5
v26-dev
v26.0
v26.0-rc1
v26.0-rc2
v26.0-rc3
v26.1
v27-dev
v27.0
v27.0-rc1
v27.0-rc2
v27.0-rc3
v27.1
v27.2
v27.3
v27.4
v27.5
v28-dev
v28.0
v28.0-rc1
v28.0-rc2
v28.0-rc3
v28.1
v28.2
v28.3
v29-dev
v29.0
v29.0-rc1
v29.0-rc2
v29.0-rc3
v3.0.0
v3.0.0-alpha-1
v3.0.0-alpha-2
v3.0.0-alpha-3
v3.0.0-alpha-3.1
v3.0.0-alpha-4
v3.0.0-alpha-4.1
v3.0.0-beta-1
v3.0.0-beta-1-bzl-fix
v3.0.0-beta-1.1
v3.0.0-beta-2
v3.0.0-beta-3
v3.0.0-beta-3-pre-1
v3.0.0-beta-3.1
v3.0.0-beta-3.2
v3.0.0-beta-3.3
v3.0.0-beta-4
v3.0.0-javalite
v3.0.1-javalite
v3.0.2
v3.1.0
v3.1.0-alpha-1
v3.10.0
v3.10.0-rc1
v3.10.1
v3.11.0
v3.11.0-rc1
v3.11.0-rc2
v3.11.1
v3.11.2
v3.11.3
v3.11.4
v3.12.0
v3.12.0-rc1
v3.12.0-rc2
v3.12.1
v3.12.2
v3.12.3
v3.12.4
v3.13.0
v3.13.0-rc3
v3.13.0.1
v3.14.0
v3.14.0-rc1
v3.14.0-rc2
v3.14.0-rc3
v3.15.0
v3.15.0-rc1
v3.15.0-rc2
v3.15.1
v3.15.2
v3.15.3
v3.15.4
v3.15.5
v3.15.6
v3.15.7
v3.15.8
v3.16.0
v3.16.0-rc1
v3.16.0-rc2
v3.16.1
v3.16.2
v3.16.3
v3.17.0
v3.17.0-rc1
v3.17.0-rc2
v3.17.1
v3.17.2
v3.17.3
v3.18.0
v3.18.0-rc1
v3.18.0-rc2
v3.18.1
v3.18.2
v3.18.3
v3.19.0
v3.19.0-rc1
v3.19.0-rc2
v3.19.1
v3.19.2
v3.19.3
v3.19.4
v3.19.5
v3.19.6
v3.2.0
v3.2.0-alpha-1
v3.2.0-rc.1
v3.2.0rc2
v3.2.1
v3.20.0
v3.20.0-rc1
v3.20.0-rc2
v3.20.0-rc3
v3.20.1
v3.20.1-rc1
v3.20.2
v3.20.3
v3.21.0
v3.21.0-rc2
v3.21.1
v3.21.10
v3.21.11
v3.21.12
v3.21.2
v3.21.3
v3.21.4
v3.21.5
v3.21.6
v3.21.7
v3.21.8
v3.21.9
v3.22.0
v3.22.0-rc1
v3.22.0-rc2
v3.22.0-rc3
v3.22.1
v3.22.2
v3.22.3
v3.22.4
v3.22.5
v3.23.0
v3.23.0-rc1
v3.23.0-rc2
v3.23.0-rc3
v3.23.1
v3.23.2
v3.23.3
v3.23.4
v3.24.0
v3.24.0-rc1
v3.24.0-rc2
v3.24.0-rc3
v3.24.1
v3.24.2
v3.24.3
v3.24.4
v3.25.0
v3.25.0-rc1
v3.25.0-rc2
v3.25.1
v3.25.2
v3.25.3
v3.25.4
v3.25.5
v3.26.0
v3.26.0-rc1
v3.26.0-rc2
v3.26.0-rc3
v3.26.1
v3.27.0
v3.27.0-rc1
v3.27.0-rc2
v3.27.0-rc3
v3.27.1
v3.27.2
v3.27.3
v3.27.4
v3.27.5
v3.28.0
v3.28.0-rc1
v3.28.0-rc2
v3.28.0-rc3
v3.28.1
v3.28.2
v3.28.3
v3.29.0
v3.29.0-rc1
v3.29.0-rc2
v3.29.0-rc3
v3.3.0
v3.3.0rc1
v3.3.1
v3.3.2
v3.4.0
v3.4.0rc1
v3.4.0rc2
v3.4.0rc3
v3.4.1
v3.5.0
v3.5.0.1
v3.5.1
v3.5.1.1
v3.5.2
v3.6.0
v3.6.0.1
v3.6.0rc1
v3.6.0rc2
v3.6.1
v3.6.1.1
v3.6.1.2
v3.6.1.3
v3.7.0
v3.7.0-rc.2
v3.7.0-rc.3
v3.7.0rc1
v3.7.0rc2
v3.7.1
v3.8.0
v3.8.0-rc1
v3.9.0
v3.9.0-rc1
v3.9.1
v3.9.2
v4.22.0
v4.22.0-rc1
v4.22.0-rc2
v4.22.0-rc3
v4.22.1
v4.22.2
v4.22.3
v4.22.4
v4.22.5
v4.23.0
v4.23.0-rc1
v4.23.0-rc2
v4.23.0-rc3
v4.23.1
v4.23.2
v4.23.3
v4.23.4
v4.24.0
v4.24.0-rc1
v4.24.0-rc2
v4.24.0-rc3
v4.24.1
v4.24.2
v4.24.3
v4.24.4
v4.25.0
v4.25.0-rc1
v4.25.0-rc2
v4.25.1
v4.25.2
v4.25.3
v4.25.4
v4.25.5
v5.26.0
v5.26.0-rc1
v5.26.0-rc2
v5.26.0-rc3
v5.26.1
v5.27.0
v5.27.0-rc1
v5.27.0-rc2
v5.27.0-rc3
v5.27.1
v5.27.2
v5.27.3
v5.27.4
v5.27.5
v5.28.0
v5.28.0-rc1
v5.28.0-rc2
v5.28.0-rc3
v5.28.1
v5.28.2
v5.28.3
v5.29.0
v5.29.0-rc1
v5.29.0-rc2
v5.29.0-rc3
${ noResults }
12 Commits (1363bf9c051af36c555c23d734a204246c215697)
Author | SHA1 | Message | Date |
---|---|---|---|
Ben Gordon |
f5362e11fd
|
Requesting extension id 1072
This library supports an idiomatic proto3 protobuf generator for kotlin. The library will be open sourced by Toast Inc under the Apache2 license, and is currently used in production at Toast. The following is the readme.md that will be released with the code by the end of Q4 2019. Supports only the Protocol Buffers language version 3. #### Features - Clean data class generation - Oneof types handled as sealed classes - JavaDoc comments on generated code - Deprecation option pass-through to Kotlin's `@Deprecated` annotation - Protokt-specific options: non-null types, wrapper types, interface implementation, and more - Tight integration with Protobuf's Java library: compatibility with its well-known types and usage of CodedInputStream and CodedOutputStream for best performance #### Not yet implemented - Kotlin native support - Kotlin JS support - Support for gRPC service generation - Protobuf JSON support See examples in [protokt-testing](https://github.com/toasttab/protokt/tree/master/protokt-testing). ### Generated Code Generated code is placed in `<buildDir>/generated-sources/main/protokt`. A simple example: ```proto syntax = "proto3"; package com.protokt.sample; message Sample { string sample_field = 1; } ``` will produce: ```kotlin /* * Generated by protokt. Do not modify. */ package com.protokt.sample import com.toasttab.protokt.rt.* data class Sample( val sampleField: String, val unknown: Map<Int, Unknown> = emptyMap() ) : KtMessage { @Suppress("UNUSED") constructor( sampleField: String = "" ) : this( sampleField, emptyMap() ) override val messageSize by lazy { sizeof() } override fun serialize(serializer: KtMessageSerializer) { if (sampleField.isNotEmpty()) { serializer.write(Tag(10)).write(sampleField) } if (unknown.isNotEmpty()) { serializer.writeUnknown(unknown) } } private fun sizeof(): Int { var res = 0 if (sampleField.isNotEmpty()) { res += sizeof(Tag(1)) + sizeof(sampleField) } res += unknown.entries.sumBy { it.value.sizeof() } return res } companion object Deserializer : KtDeserializer<Sample> { override fun deserialize(deserializer: KtMessageDeserializer): Sample { var sampleField = "" val unknown = mutableMapOf<Int, Unknown>() while (true) { when (deserializer.readTag()) { 0 -> return Sample( sampleField, unknown ) 10 -> sampleField = deserializer.readString() else -> { val unk = deserializer.readUnknown() unknown[unk.fieldNum] = unknown[unk.fieldNum].let { when (it) { null -> unk else -> when (val v = it.value) { is ListVal -> Unknown(unk.fieldNum, ListVal(v.value + unk.value)) else -> Unknown(unk.fieldNum, ListVal(listOf(v, unk.value))) } } } } } } } } } ``` #### Runtime Notes ##### Package The Kotlin package of a generated file can be overridden from protobuf package with the `(protokt).package` option: ```proto syntax = "proto3"; import "protokt.proto"; package com.example; option (protokt).package = "com.package"; ``` ##### Message Each protokt message implements the `KtMessage` interface. `KtMessage` defines the `serialize()` method and its overloads which can serialize to a byte array, a `KtMessageSerializer`, or on the JVM, an `OutputStream`. Each protokt message has a companion object `Deserializer` that implements the `KtDeserializer` interface, which provides the `deserialize()` method and its overloads to construct an instance of the message from a byte array, a Java InputStream, or others. In order to enjoy the full benefits of Kotlin data classes, byte arrays are wrapped in the protokt `Bytes` class, which provides appropriate `equals()` and `hashCode()` implementations. ##### Enums Enum fields are generated as data classes with a single integer field. Kotlin enum classes are closed and cannot retain unknown values, and protobuf requires that unknown enum values are preserved for reserialization. This compromise exposes a constructor taking an integer, but the `from(value: Int)` on an enum's `Deserializer` should be preferred as it avoids instantiation when possible. Other notes: - `optimize_for` is ignored. - `repeated` fields are deserialized to Lists. - `map` fields are deserialized to Maps. - `oneof` fields are represented as data class subtypes of a sealed base class with a single property. ### Extensions See examples of each option in the [protokt-options](https://github.com/toasttab/protokt/tree/master/protokt-testing/protokt-options/src/main/proto) module. All protokt-specific options require importing `protokt.proto` in the protocol file. #### Wrapper Types Sometimes a field on a protobuf message corresponds to a concrete nonprimitive type. In standard protobuf the user would be responsible for this extra transformation, but the protokt wrapper type option allows specification of a converter that will automatically encode and decode custom types to protobuf primitives and well-known types. Some standard types are implemented in [protokt-extensions](https://github.com/toasttab/protokt/tree/master/protokt-extensions/src/main/kotlin/com/toasttab/protokt/ext). Wrap a field by invoking the `(protokt_property).wrap` option: ```proto message DateWrapperMessage { int64 date = 1 [ (protokt_property).wrap = "java.util.Date" ]; } ``` Converters implement the `Converter` interface: ```kotlin interface Converter<S: Any, T: Any> { val wrapper: KClass<S> fun wrap(unwrapped: T): S fun unwrap(wrapped: S): T } ``` and protokt will reference the converter's methods to wrap and unwrap from protobuf primitives: ```kotlin object DateConverter : Converter<Date, Long> { override val wrapper = Date::class override fun wrap(unwrapped: Long) = Date(unwrapped) override fun unwrap(wrapped: Date) = wrapped.time } ``` ```kotlin data class WrapperModel( val date: java.util.Date, ... ) : KtMessage { ... override fun serialize(serializer: KtMessageSerializer) { serializer.write(Tag(10)).write(Int64(DateConverter.unwrap(date))) ... } override fun deserialize(deserializer: KtMessageDeserializer): WrapperModel { var date = 0L while (true) { when (deserializer.readTag()) { 0 -> return WrapperModel( DateConverter.wrap(date), ... ) ... } } } } ``` Converters can also implement the `OptimizedSizeofConverter` interface adding `sizeof()`, which allows them to optimize the calculation of the wrapper's size rather than unwrap the object twice. For example, a UUID is always 16 bytes: ```kotlin object UuidConverter : OptimizedSizeofConverter<UUID, ByteArray> { override val wrapper = UUID::class private val sizeofProxy = ByteArray(16) override fun sizeof(wrapped: UUID) = sizeof(sizeofProxy) override fun wrap(unwrapped: ByteArray): UUID { require(unwrapped.size == 16) { "input must have size 16; had ${unwrapped.size}" } return ByteBuffer.wrap(unwrapped) .run { UUID(long, long) } } override fun unwrap(wrapped: UUID) = ByteBuffer.allocate(16) .putLong(wrapped.mostSignificantBits) .putLong(wrapped.leastSignificantBits) .array() } ``` Rather than convert a UUID to a byte array both for size calculation and for serialization (which is what a naïve implementation would do), UuidConverter always returns the size of a constant 16-byte array. If the wrapper type is in the same package as the generated protobuf message, then it does not need a fully-qualified name. Custom wrapper type converters can be in the same module as protobuf types that reference them. In order to use any wrapper type defined in `protokt-extensions`, the module must be included as a dependency: ```groovy dependencies { implementation 'com.toasttab.protokt:protokt-extensions:0.0.3' } ``` #### Interface implementation To avoid the need to create domain-specific objects from protobuf messages you can declare that a protobuf message implements a custom interface with properties and default methods. ```kotlin package com.protokt.sample interface Model { val id: String } ``` ```proto package com.protokt.sample; message ImplementsSampleMessage { option (protokt_class).implements = "Model"; string id = 1; } ``` If the wrapper interface is in the same package as the generated protobuf message, then it does not need a fully-qualified name. Wrapper interfaces cannot be used by protobuf messages in the same module that defines them; the dependency must be declared with`protoktExtensions` in `build.gradle`: ```groovy dependencies { protoktExtensions project(':api-module') } ``` #### Nonnull fields If there is a message that has no meaning whatsoever when a particular field is missing, you can emulate proto2's `required` key word by using the `(protokt_oneof).non_null` option: ```proto message Sample { } message NonNullSampleMessage { Sample non_null_sample = 1 [ (protokt_property).non_null = true ]; } ``` Generated code will not have a nullable type so the field can be referenced without using Kotlin's `!!`. Oneof fields can also be declared non-null: ```proto message NonNullSampleMessage { oneof non_null_oneof { option (protokt_oneof).non_null = true; string message = 2; } } ``` Note that deserialization of a message with a non-nullable field will fail if the message being decoded does not contain an instance of the required field. #### BytesSlice When reading messages that contain other serialized messages as `bytes` fields, protokt can keep a reference to the originating byte array to prevent a large copy operation on deserialization. This can be desirable when the wrapping message is a thin metadata shim and doesn't include much memory overhead: ```proto message SliceModel { int64 version = 1; bytes encoded_message = 2 [ (protokt_property).bytes_slice = true ]; } ``` ### Usage #### Gradle ```groovy buildscript { dependencies { classpath "com.toasttab.protokt:protokt-gradle-plugin:0.0.3" } } apply plugin: 'com.toasttab.protokt' ``` This will automatically download and install protokt, apply the Google protobuf plugin, and configure all the necessary boilerplate. By default it will also add `protokt-runtime` to the api scope of the project, and `protobuf-java` to the implementation scope. If your project is pure Kotlin you may run into the following error: ``` Execution failed for task ':compileJava'. > error: no source files ``` To work around it, disable all `JavaCompile` tasks in the project: ```groovy tasks.withType(JavaCompile) { enabled = false } ``` or: ```groovy compileJava.enabled = false ``` #### Command line code generation ```bash protokt-codegen$ ./gradlew assemble [OR ./gradlew installDist] protokt-codegen$ ./run-protokt.sh -h protokt-codegen$ ./run-protokt.sh \ -out=../kotlin \ -pkg=com.toasttab.protokt.conformance \ -file=conformance.proto \ -cp=../build/libs/protokt-codegen-0.0.3-SNAPSHOT-all.jar \ -plugin=../bin/protokt.sh ``` ### Contribution To enable rapid development of the code generator, the protobuf conformance tests have been compiled and included in the protokt-testing project. They run on Mac OS 10.14+ and Ubuntu 16.04 x86-64. Publish the plugin to the integration repository: ```bash protokt$ ./gradlew publishToIntegrationRepository ``` Then run the tests from `protokt-testing`: ```bash protokt-testing$ ./gradlew protokt-conformance-tests:test ``` All integration tests can be run with: ``` protokt-testing$ ./gradlew test ``` |
5 years ago |
marwan-at-work | 36cd11a2bf |
Add twirpql ID to global registry
|
6 years ago |
Alex Konradi |
b44a1364ec
|
Add extension number for protoc-gen-validate
This project is used by the [Envoy proxy](https://www.envoyproxy.io/) and several others.
The existing code is using the extension value [919191](
|
6 years ago |
Xiang Dai | e479410564 |
delete all duplicate empty blanks (#5758)
Signed-off-by: Xiang Dai <764524258@qq.com> |
6 years ago |
James DeFelice | 4358133b85 |
options: reserved extension range for CSI
|
6 years ago |
Toby Schneider |
4fd7bdee48
|
Update options.md
Update my projects' (DCCL / Goby) URLs. Propose new extension for CGSN Mooring project. |
6 years ago |
Gohar Aziz |
f3a74befcd
|
Update options.md | fix website url for C# port
Update options.md | fix website url for C# port of protocol buffers.. From `http://github.com/jskeet/dotnet-protobufs` to `https://github.com/jskeet/protobuf-csharp-port` |
6 years ago |
Feng Xiao | afe98de32a |
Replace repo links.
|
6 years ago |
Ryland Degnan | 5b7d0dc8aa |
Update options.md (#4952)
Added Netifi Proteus to Registered Extensions |
6 years ago |
Feng Xiao | ef7052b473 |
Add global extension registry to the repo.
|
7 years ago |