am c4a1b547: Merge "Add reftypes field generator option."

* commit 'c4a1b547c91178d79f5d2cc2e00b8be922c22fe8':
  Add reftypes field generator option.
pull/91/head
Ulas Kirazci 11 years ago committed by Android Git Automerger
commit 15d23f93ae
  1. 71
      java/README.txt
  2. 62
      java/src/test/java/com/google/protobuf/NanoTest.java
  3. 14
      src/google/protobuf/compiler/javanano/javanano_enum_field.cc
  4. 10
      src/google/protobuf/compiler/javanano/javanano_generator.cc
  5. 4
      src/google/protobuf/compiler/javanano/javanano_helpers.cc
  6. 11
      src/google/protobuf/compiler/javanano/javanano_params.h
  7. 16
      src/google/protobuf/compiler/javanano/javanano_primitive_field.cc
  8. 116
      src/google/protobuf/unittest_reference_types_nano.proto

@ -487,34 +487,67 @@ java_nano_generate_has={true,false} (default: false)
many cases reading the default works and determining whether the many cases reading the default works and determining whether the
field was received over the wire is irrelevant. field was received over the wire is irrelevant.
optional_field_style={default,accessors} (default: default) optional_field_style={default,accessors,reftypes} (default: default)
Defines the style of the generated code for _optional_ fields only. Defines the style of the generated code for fields.
* default *
In the default style, optional fields translate into public mutable In the default style, optional fields translate into public mutable
Java fields, and the serialization process is as discussed in the Java fields, and the serialization process is as discussed in the
"IMPORTANT" section above. When set to 'accessors', each optional "IMPORTANT" section above.
field is encapsulated behind 4 accessors, namely get<fieldname>(),
set<fieldname>(), has<fieldname>() and clear<fieldname>() methods,
with the standard semantics. The hazzer's return value determines
whether a field is serialized, so this style is useful when you need
to serialize a field with the default value, or check if a field has
been explicitly set to its default value from the wire.
Required fields are still translated to one public mutable Java
field each, and repeated fields are still translated to arrays. No
accessors are generated for them.
optional_field_style=accessors cannot be used together with
java_nano_generate_has=true. If you need the 'has' flag for any
required field (you have no reason to), you can only use
java_nano_generate_has=true.
IMPORTANT: When using the 'accessor' style, ProGuard should always * accessors *
When set to 'accessors', each optional field is encapsulated behind
4 accessors, namely get<fieldname>(), set<fieldname>(), has<fieldname>()
and clear<fieldname>() methods, with the standard semantics. The hazzer's
return value determines whether a field is serialized, so this style is
useful when you need to serialize a field with the default value, or check
if a field has been explicitly set to its default value from the wire.
In the 'accessors' style, required fields are still translated to one
public mutable Java field each, and repeated fields are still translated
to arrays. No accessors are generated for them.
IMPORTANT: When using the 'accessors' style, ProGuard should always
be enabled with optimization (don't use -dontoptimize) and allowing be enabled with optimization (don't use -dontoptimize) and allowing
access modification (use -allowaccessmodification). This removes the access modification (use -allowaccessmodification). This removes the
unused accessors and maybe inline the rest at the call sites, unused accessors and maybe inline the rest at the call sites,
reducing the final code size. reducing the final code size.
TODO(maxtroy): find ProGuard config that would work the best. TODO(maxtroy): find ProGuard config that would work the best.
* reftypes *
When set to 'reftypes', each proto field is generated as a public Java
field. For primitive types, these fields use the Java reference types
such as java.lang.Integer instead of primitive types such as int.
In the 'reftypes' style, fields are initialized to null (or empty
arrays for repeated fields), and their default values are not available.
They are serialized over the wire based on equality to null.
The 'reftypes' mode has some additional cost due to autoboxing and usage
of reference types. In practice, many boxed types are cached, and so don't
result in object creation. However, references do take slightly more memory
than primitives.
The 'reftypes' mode is useful when you want to be able to serialize fields
with default values, or check if a field has been explicitly set to the
default over the wire without paying the extra method cost of the
'accessors' mode.
Note that if you attempt to write null to a required field in the reftypes
mode, serialization of the proto will cause a NullPointerException. This is
an intentional indicator that you must set required fields.
NOTE
optional_field_style=accessors or reftypes cannot be used together with
java_nano_generate_has=true. If you need the 'has' flag for any
required field (you have no reason to), you can only use
java_nano_generate_has=true.
To use nano protobufs: To use nano protobufs:
- Link with the generated jar file - Link with the generated jar file

@ -48,6 +48,7 @@ import com.google.protobuf.nano.NanoAccessorsOuterClass.TestNanoAccessors;
import com.google.protobuf.nano.NanoHasOuterClass.TestAllTypesNanoHas; import com.google.protobuf.nano.NanoHasOuterClass.TestAllTypesNanoHas;
import com.google.protobuf.nano.NanoOuterClass; import com.google.protobuf.nano.NanoOuterClass;
import com.google.protobuf.nano.NanoOuterClass.TestAllTypesNano; import com.google.protobuf.nano.NanoOuterClass.TestAllTypesNano;
import com.google.protobuf.nano.NanoReferenceTypes;
import com.google.protobuf.nano.UnittestImportNano; import com.google.protobuf.nano.UnittestImportNano;
import com.google.protobuf.nano.UnittestMultipleNano; import com.google.protobuf.nano.UnittestMultipleNano;
import com.google.protobuf.nano.UnittestRecursiveNano.RecursiveMessageNano; import com.google.protobuf.nano.UnittestRecursiveNano.RecursiveMessageNano;
@ -2624,6 +2625,67 @@ public class NanoTest extends TestCase {
assertEquals(123, msg.synchronized_); assertEquals(123, msg.synchronized_);
} }
public void testReferenceTypesForPrimitives() throws Exception {
NanoReferenceTypes.TestAllTypesNano message = new NanoReferenceTypes.TestAllTypesNano();
// Base check - when nothing is set, we serialize nothing.
assertHasWireData(message, false);
message.defaultBool = true;
assertHasWireData(message, true);
message.defaultBool = false;
assertHasWireData(message, true);
message.defaultBool = null;
assertHasWireData(message, false);
message.defaultInt32 = 5;
assertHasWireData(message, true);
message.defaultInt32 = null;
assertHasWireData(message, false);
message.defaultInt64 = 123456L;
assertHasWireData(message, true);
message.defaultInt64 = null;
assertHasWireData(message, false);
message.defaultFloat = 1f;
assertHasWireData(message, true);
message.defaultFloat = null;
assertHasWireData(message, false);
message.defaultDouble = 2.1;
assertHasWireData(message, true);
message.defaultDouble = null;
assertHasWireData(message, false);
message.defaultString = "hello";
assertHasWireData(message, true);
message.defaultString = null;
assertHasWireData(message, false);
message.defaultBytes = new byte[] { 1, 2, 3 };
assertHasWireData(message, true);
message.defaultBytes = null;
assertHasWireData(message, false);
}
private void assertHasWireData(MessageNano message, boolean expected) {
int wireLength = MessageNano.toByteArray(message).length;
if (expected) {
assertFalse(wireLength == 0);
} else {
assertEquals(0, wireLength);
}
}
private <T> List<T> list(T first, T... remaining) { private <T> List<T> list(T first, T... remaining) {
List<T> list = new ArrayList<T>(); List<T> list = new ArrayList<T>();
list.add(first); list.add(first);

@ -58,8 +58,16 @@ void SetEnumVariables(const Params& params,
(*variables)["capitalized_name"] = (*variables)["capitalized_name"] =
RenameJavaKeywords(UnderscoresToCapitalizedCamelCase(descriptor)); RenameJavaKeywords(UnderscoresToCapitalizedCamelCase(descriptor));
(*variables)["number"] = SimpleItoa(descriptor->number()); (*variables)["number"] = SimpleItoa(descriptor->number());
if (params.use_reference_types_for_primitives()
&& !descriptor->is_repeated()) {
(*variables)["type"] = "java.lang.Integer";
(*variables)["default"] = "null";
} else {
(*variables)["type"] = "int"; (*variables)["type"] = "int";
(*variables)["default"] = DefaultValue(params, descriptor); (*variables)["default"] = DefaultValue(params, descriptor);
}
(*variables)["repeated_default"] =
"com.google.protobuf.nano.WireFormatNano.EMPTY_INT_ARRAY";
(*variables)["tag"] = SimpleItoa(internal::WireFormat::MakeTag(descriptor)); (*variables)["tag"] = SimpleItoa(internal::WireFormat::MakeTag(descriptor));
(*variables)["tag_size"] = SimpleItoa( (*variables)["tag_size"] = SimpleItoa(
internal::WireFormat::TagSize(descriptor->number(), descriptor->type())); internal::WireFormat::TagSize(descriptor->number(), descriptor->type()));
@ -81,7 +89,7 @@ EnumFieldGenerator::~EnumFieldGenerator() {}
void EnumFieldGenerator:: void EnumFieldGenerator::
GenerateMembers(io::Printer* printer) const { GenerateMembers(io::Printer* printer) const {
printer->Print(variables_, printer->Print(variables_,
"public int $name$ = $default$;\n"); "public $type$ $name$ = $default$;\n");
if (params_.generate_has()) { if (params_.generate_has()) {
printer->Print(variables_, printer->Print(variables_,
@ -233,7 +241,7 @@ RepeatedEnumFieldGenerator::~RepeatedEnumFieldGenerator() {}
void RepeatedEnumFieldGenerator:: void RepeatedEnumFieldGenerator::
GenerateMembers(io::Printer* printer) const { GenerateMembers(io::Printer* printer) const {
printer->Print(variables_, printer->Print(variables_,
"public int[] $name$ = com.google.protobuf.nano.WireFormatNano.EMPTY_INT_ARRAY;\n"); "public $type$[] $name$ = $repeated_default$;\n");
if (descriptor_->options().packed()) { if (descriptor_->options().packed()) {
printer->Print(variables_, printer->Print(variables_,
"private int $name$MemoizedSerializedSize;\n"); "private int $name$MemoizedSerializedSize;\n");
@ -243,7 +251,7 @@ GenerateMembers(io::Printer* printer) const {
void RepeatedEnumFieldGenerator:: void RepeatedEnumFieldGenerator::
GenerateClearCode(io::Printer* printer) const { GenerateClearCode(io::Printer* printer) const {
printer->Print(variables_, printer->Print(variables_,
"$name$ = com.google.protobuf.nano.WireFormatNano.EMPTY_INT_ARRAY;\n"); "$name$ = $repeated_default$;\n");
} }
void RepeatedEnumFieldGenerator:: void RepeatedEnumFieldGenerator::

@ -124,15 +124,21 @@ bool JavaNanoGenerator::Generate(const FileDescriptor* file,
params.set_java_enum_style(options[i].second == "java"); params.set_java_enum_style(options[i].second == "java");
} else if (options[i].first == "optional_field_style") { } else if (options[i].first == "optional_field_style") {
params.set_optional_field_accessors(options[i].second == "accessors"); params.set_optional_field_accessors(options[i].second == "accessors");
params.set_use_reference_types_for_primitives(options[i].second == "reftypes");
} else { } else {
*error = "Ignore unknown javanano generator option: " + options[i].first; *error = "Ignore unknown javanano generator option: " + options[i].first;
} }
} }
// Check illegal parameter combinations // Check illegal parameter combinations
if (params.generate_has() && params.optional_field_accessors()) { // Note: the enum-like optional_field_style generator param ensures
// that we can never have illegal combinations of field styles
// (e.g. reftypes and accessors can't be on at the same time).
if (params.generate_has()
&& (params.optional_field_accessors()
|| params.use_reference_types_for_primitives())) {
error->assign("java_nano_generate_has=true cannot be used in conjunction" error->assign("java_nano_generate_has=true cannot be used in conjunction"
" with optional_field_style=accessors"); " with optional_field_style=accessors or optional_field_style=reftypes");
return false; return false;
} }

@ -357,6 +357,10 @@ string DefaultValue(const Params& params, const FieldDescriptor* field) {
return EmptyArrayName(params, field); return EmptyArrayName(params, field);
} }
if (params.use_reference_types_for_primitives()) {
return "null";
}
// Switch on cpp_type since we need to know which default_value_* method // Switch on cpp_type since we need to know which default_value_* method
// of FieldDescriptor to call. // of FieldDescriptor to call.
switch (field->cpp_type()) { switch (field->cpp_type()) {

@ -60,6 +60,7 @@ class Params {
bool generate_has_; bool generate_has_;
bool java_enum_style_; bool java_enum_style_;
bool optional_field_accessors_; bool optional_field_accessors_;
bool use_reference_types_for_primitives_;
public: public:
Params(const string & base_name) : Params(const string & base_name) :
@ -69,7 +70,8 @@ class Params {
store_unknown_fields_(false), store_unknown_fields_(false),
generate_has_(false), generate_has_(false),
java_enum_style_(false), java_enum_style_(false),
optional_field_accessors_(false) { optional_field_accessors_(false),
use_reference_types_for_primitives_(false) {
} }
const string& base_name() const { const string& base_name() const {
@ -177,6 +179,13 @@ class Params {
bool optional_field_accessors() const { bool optional_field_accessors() const {
return optional_field_accessors_; return optional_field_accessors_;
} }
void set_use_reference_types_for_primitives(bool value) {
use_reference_types_for_primitives_ = value;
}
bool use_reference_types_for_primitives() const {
return use_reference_types_for_primitives_;
}
}; };
} // namespace javanano } // namespace javanano

@ -245,7 +245,12 @@ void SetPrimitiveVariables(const FieldDescriptor* descriptor, const Params param
(*variables)["capitalized_name"] = (*variables)["capitalized_name"] =
RenameJavaKeywords(UnderscoresToCapitalizedCamelCase(descriptor)); RenameJavaKeywords(UnderscoresToCapitalizedCamelCase(descriptor));
(*variables)["number"] = SimpleItoa(descriptor->number()); (*variables)["number"] = SimpleItoa(descriptor->number());
if (params.use_reference_types_for_primitives()
&& !descriptor->is_repeated()) {
(*variables)["type"] = BoxedPrimitiveTypeName(GetJavaType(descriptor));
} else {
(*variables)["type"] = PrimitiveTypeName(GetJavaType(descriptor)); (*variables)["type"] = PrimitiveTypeName(GetJavaType(descriptor));
}
(*variables)["default"] = DefaultValue(params, descriptor); (*variables)["default"] = DefaultValue(params, descriptor);
(*variables)["default_constant"] = FieldDefaultConstantName(descriptor); (*variables)["default_constant"] = FieldDefaultConstantName(descriptor);
// For C++-string types (string and bytes), we might need to have // For C++-string types (string and bytes), we might need to have
@ -254,7 +259,8 @@ void SetPrimitiveVariables(const FieldDescriptor* descriptor, const Params param
// once into a "private static final" field and re-use that from // once into a "private static final" field and re-use that from
// then on. // then on.
if (descriptor->cpp_type() == FieldDescriptor::CPPTYPE_STRING && if (descriptor->cpp_type() == FieldDescriptor::CPPTYPE_STRING &&
!descriptor->default_value_string().empty()) { !descriptor->default_value_string().empty() &&
!params.use_reference_types_for_primitives()) {
string default_value; string default_value;
if (descriptor->type() == FieldDescriptor::TYPE_BYTES) { if (descriptor->type() == FieldDescriptor::TYPE_BYTES) {
default_value = strings::Substitute( default_value = strings::Substitute(
@ -307,6 +313,7 @@ GenerateMembers(io::Printer* printer) const {
printer->Print(variables_, printer->Print(variables_,
"private static final $type$ $default_constant$ = $default_constant_value$;\n"); "private static final $type$ $default_constant$ = $default_constant_value$;\n");
} }
printer->Print(variables_, printer->Print(variables_,
"public $type$ $name$ = $default_copy_if_needed$;\n"); "public $type$ $name$ = $default_copy_if_needed$;\n");
@ -340,6 +347,13 @@ GenerateMergingCode(io::Printer* printer) const {
void PrimitiveFieldGenerator:: void PrimitiveFieldGenerator::
GenerateSerializationConditional(io::Printer* printer) const { GenerateSerializationConditional(io::Printer* printer) const {
if (params_.use_reference_types_for_primitives()) {
// For reference type mode, serialize based on equality
// to null.
printer->Print(variables_,
"if (this.$name$ != null) {\n");
return;
}
if (params_.generate_has()) { if (params_.generate_has()) {
printer->Print(variables_, printer->Print(variables_,
"if (has$capitalized_name$ || "); "if (has$capitalized_name$ || ");

@ -0,0 +1,116 @@
package protobuf_unittest;
option java_package = "com.google.protobuf.nano";
option java_outer_classname = "NanoReferenceTypes";
message TestAllTypesNano {
enum NestedEnum {
FOO = 1;
BAR = 2;
BAZ = 3;
}
message NestedMessage {
optional int32 foo = 1;
}
// Singular
optional int32 optional_int32 = 1;
optional int64 optional_int64 = 2;
optional uint32 optional_uint32 = 3;
optional uint64 optional_uint64 = 4;
optional sint32 optional_sint32 = 5;
optional sint64 optional_sint64 = 6;
optional fixed32 optional_fixed32 = 7;
optional fixed64 optional_fixed64 = 8;
optional sfixed32 optional_sfixed32 = 9;
optional sfixed64 optional_sfixed64 = 10;
optional float optional_float = 11;
optional double optional_double = 12;
optional bool optional_bool = 13;
optional string optional_string = 14;
optional bytes optional_bytes = 15;
optional group OptionalGroup = 16 {
optional int32 a = 17;
}
optional NestedMessage optional_nested_message = 18;
optional NestedEnum optional_nested_enum = 21;
optional string optional_string_piece = 24 [ctype=STRING_PIECE];
optional string optional_cord = 25 [ctype=CORD];
// Repeated
repeated int32 repeated_int32 = 31;
repeated int64 repeated_int64 = 32;
repeated uint32 repeated_uint32 = 33;
repeated uint64 repeated_uint64 = 34;
repeated sint32 repeated_sint32 = 35;
repeated sint64 repeated_sint64 = 36;
repeated fixed32 repeated_fixed32 = 37;
repeated fixed64 repeated_fixed64 = 38;
repeated sfixed32 repeated_sfixed32 = 39;
repeated sfixed64 repeated_sfixed64 = 40;
repeated float repeated_float = 41;
repeated double repeated_double = 42;
repeated bool repeated_bool = 43;
repeated string repeated_string = 44;
repeated bytes repeated_bytes = 45;
repeated group RepeatedGroup = 46 {
optional int32 a = 47;
}
repeated NestedMessage repeated_nested_message = 48;
repeated NestedEnum repeated_nested_enum = 51;
repeated string repeated_string_piece = 54 [ctype=STRING_PIECE];
repeated string repeated_cord = 55 [ctype=CORD];
// Repeated packed
repeated int32 repeated_packed_int32 = 87 [packed=true];
repeated sfixed64 repeated_packed_sfixed64 = 88 [packed=true];
repeated NestedEnum repeated_packed_nested_enum = 89 [packed=true];
// Singular with defaults
optional int32 default_int32 = 61 [default = 41 ];
optional int64 default_int64 = 62 [default = 42 ];
optional uint32 default_uint32 = 63 [default = 43 ];
optional uint64 default_uint64 = 64 [default = 44 ];
optional sint32 default_sint32 = 65 [default = -45 ];
optional sint64 default_sint64 = 66 [default = 46 ];
optional fixed32 default_fixed32 = 67 [default = 47 ];
optional fixed64 default_fixed64 = 68 [default = 48 ];
optional sfixed32 default_sfixed32 = 69 [default = 49 ];
optional sfixed64 default_sfixed64 = 70 [default = -50 ];
optional float default_float = 71 [default = 51.5 ];
optional double default_double = 72 [default = 52e3 ];
optional bool default_bool = 73 [default = true ];
optional string default_string = 74 [default = "hello"];
optional bytes default_bytes = 75 [default = "world"];
optional float default_float_inf = 97 [default = inf];
optional float default_float_neg_inf = 98 [default = -inf];
optional float default_float_nan = 99 [default = nan];
optional double default_double_inf = 100 [default = inf];
optional double default_double_neg_inf = 101 [default = -inf];
optional double default_double_nan = 102 [default = nan];
}
message ForeignMessageNano {
optional int32 c = 1;
}
enum ForeignEnumNano {
FOREIGN_NANO_FOO = 4;
FOREIGN_NANO_BAR = 5;
FOREIGN_NANO_BAZ = 6;
}
Loading…
Cancel
Save