diff --git a/docs/design/editions/README.md b/docs/design/editions/README.md index 2e02f09bff..66671a4afd 100644 --- a/docs/design/editions/README.md +++ b/docs/design/editions/README.md @@ -21,4 +21,5 @@ The following topics are in this repository: * [What are Protobuf Editions?](what-are-protobuf-editions.md) * [Life of an Edition](life-of-an-edition.md) -* [Editions: Life of a FeatureSet](editions-life-of-a-featureset.md) \ No newline at end of file +* [Editions: Life of a FeatureSet](editions-life-of-a-featureset.md) +* [Protobuf Design: Options Attributes](protobuf-design-options-attributes.md) diff --git a/docs/design/editions/protobuf-design-options-attributes.md b/docs/design/editions/protobuf-design-options-attributes.md new file mode 100644 index 0000000000..b4619fd602 --- /dev/null +++ b/docs/design/editions/protobuf-design-options-attributes.md @@ -0,0 +1,247 @@ +# Protobuf Design: Options Attributes + +A proposal to create target and retention attributes to support. + +**Author:** [@kfm](https://github.com/fowles) + +**Approved:** 2022-08-26 + +## Background + +The [Protobuf Editions](what-are-protobuf-editions.md) project plans to use +[custom options](protobuf-editions-design-features.md) to model features and +encourage language bindings to build custom features off options as well. + +This design proposed the specific addition of `target` and `retention` +attributes for options as well as their suggested meaning. + +Both `target` and `retention` attributes are no-ops when applied to fields that +are not options (either from descriptor.proto or custom options). + +## Target Attributes + +Historically, options have only applied to specific entities, but features will +be available on most entities. To allow language specific extensions to restrict +the places where options can bind, we will allow features to explicitly specify +the targets they apply to (similar in concept to the "target" attribute on Java +annotations). `TARGET_TYPE_UNKNOWN` will be treated as absent. + +``` +message FieldOptions { + ... + optional OptionTargetType target = 17; + + enum OptionTargetType { + TARGET_TYPE_UNKNOWN = 0; + TARGET_TYPE_FILE = 1; + TARGET_TYPE_EXTENSION_RANGE = 2; + TARGET_TYPE_MESSAGE = 3; + TARGET_TYPE_FIELD = 4; + TARGET_TYPE_ONEOF = 5; + TARGET_TYPE_ENUM = 6; + TARGET_TYPE_ENUM_VALUE = 7; + TARGET_TYPE_SERVICE = 8; + TARGET_TYPE_METHOD = 9; + }; +} +``` + +If no target is provided, `protoc` will permit the target to apply to any +entity. Otherwise, `protoc` will allow an option to be applied at either the +file level or to its target entity (and will produce a compile error for any +other placement). For example + +``` +message Features { + ... + + enum EnumType { + OPEN = 0; + CLOSED = 1; + } + optional EnumType enum = 2 [ + target = TARGET_TYPE_ENUM + ]; +} +``` + +would allow usage of + +``` +// foo.proto +edition = "tbd" + +option features.enum = OPEN; // allowed at FILE scope + +enum Foo { + option features.enum = CLOSED; // allowed at ENUM scope + A = 2; + B = 4; +} + +message Bar { + option features.enum = CLOSED; // disallowed at Message scope + + enum Baz { + C = 8; + } +} +``` + +## Retention + +To reduce the size of descriptors in protobuf runtimes, features will be +permitted to specify retention rules (again similar in concept to "retention" +attributes on Java annotations). + +``` +enum FeatureRetention { + RETENTION_UNKNOWN = 0; + RETENTION_RUNTIME = 1; + RETENTION_SOURCE = 2; +} +``` + +Options intended to inform code generators or `protoc` itself can be annotated +with `SOURCE` retention. The default retention will be `RUNTIME` as that is the +current behavior for all options. **Code generators that emit generated +descriptors will be required to omit/strip options with `SOURCE` retention from +their generated descriptors.** For example: + +``` +message Cpp { + enum StringType { + STRING = 1; + STRING_VIEW = 0; + CORD = 2; + } + + optional string namespace = 2 [ + retention = RETENTION_SOURCE, + target = TARGET_TYPE_FILE + ]; +} +``` + +## Motivation + +While the proximal motivation for these options is for use with "features" in +"editions", I believe they provide sufficient general utility that adding them +directly to `FieldDescriptorOptions` is warranted. For example, significant +savings in binary sizes could be realized if `ExtensionRangeOptions::Metadata` +had only `SOURCE` retention. Previously, we have specifically special-cased this +behavior on a per-field basis, which does work but does not provide good +extensibility. + +## Discussion + +In the initial design `target` was serving the dual purpose of identifying the +semantic entity, and also the granularity of inheritance for features. After +discussion about concerns around over use of inheritance, we decided for a +slightly refined definition that decouples these concerns. `target` **only** +specifies the semantic entity to which an option can apply. Features will be +able to be set on both the `FILE` level and their semantic entity. Everything in +between will be refused in the initial release. This allows us a clean +forward-compatible way to allow arbitrary feature inheritance, but doesn't +commit us to doing that until we need it. + +Similarly, we will start with `optional` target, because we can safely move to +`repeated` later should the need arise. + +The naming for `target` and `retention` are directly modeled after Java +annotations. Other names were considered, but no better name was found and the +similarity to an existing thing won the day. + +## Alternatives + +### Use a repeated `target` proposed + +This is the proposed alternative. + +#### Pros + +* Allows fine-grained control of target applicability. + +#### Cons + +* Harder to generalize for users (every feature's specification is potentially + unique). + +### Allow hierarchy based on `target` semantic location. + +Rather than having a repeated `target` that specifies all locations, we allow +only the level at which it semantically applies to be specified. The protoc +compiler will implicitly allow the field to be used on entities that can +lexically group that type of entry. For this `target` can be either singular or +repeated. + +#### Pros + +* Enables tooling that understands when a feature is used for grouping vs when + it has semantic value (helpful for minimizing churn in large-scale changes). +* Easier to generalize for users (any `FIELD` feature can apply to a message + as opposed to only the `FIELD` features that explicitly specified an + additional `target`). + +#### Cons + +* Forces all `target` applications to be permitted on scoping entities. + +### Use Custom Options (aka "We Must Go Deeper") + +Rather than building `retention` and `target` directly as fields of +`FieldOptions`, we could use custom options to define an equivalent thing. This +option was rejected because it pushes extra syntax onto users for a fundamental +feature. + +#### Pros + +* Doesn't require modifying `descriptor.proto`. + +#### Cons + +* Requires a less-intuitive spelling in user code. +* Requires an additional import for users. +* Language-level features would have to have a magic syntax or a side table + instead of using the same consistent option as user per code gen features. + +### Hard Code Behaviors in `protoc` + +Rather than building a generic mechanism we could simply hard code the behavior +of protoc and document it. + +#### Pros + +* Can't be misused. + +#### Cons + +* Not extensible for users. +* Requires more special cases users need to learn. + +### Original Approved Proposal + +The proposal as originally approved had some slight differences from what was +ultimately implemented: + +* The retention enum did not have an `UNKNOWN` type. +* The enums were defined at the top level instead of nested inside + `FieldOptions`. +* The enum values did not have a scoping prefix. +* The target enum had a `STREAM` entry, but this turned out to be unnecessary + since the syntax that it applied to was removed. + +### Do Nothing + +We could omit this entirely and get ice cream instead. This was rejected because +the proliferation of features on entities they do not apply to is considered too +high a cost. + +#### Pros + +* Ice cream is awesome. + +#### Cons + +* Doesn't address any of the problems that caused this to come up. +* Some people are lactose intolerant.