The current implementation of getExtension deserialises the field from bytes
and returns a new object every time. This means that changes to those objects
are reflected when the messages is serialised unless setExtension is called. It
also means that every call to getExtension and setExtension is expensive.
This change introduces a FieldData class that contains everything that's known
about the field at the time. This can be all the tag/byte[] pairs associated
with a given field or an Extension and a value object. This is so that two
messages with a repeated extension can be compared even if the extension
has been deserialised in one of them but not the other.
This change also adds FieldArray class based on SparseArray from the Android
compatibility library. This is used in ExtendableMessageNano to make lookup
of FieldDatas by their field number faster.
Implications:
* calling getExtension multiple times deserialises the field only once and
returns the same object.
* calling setExtension doesn't cause the object to be serialised immediately,
that only happens when the container message is serialised.
* getExtension is no longer a read-only thread-safe operation. README.txt has
been updated to relfect that.
* comparison using equals and hashCode continues to work.
Bug: 10863158
Change-Id: I81c7cb0c73cc0611a1f7c1eabf5eed259738e8bc
If ExtendableMessageNano doesn't have any unknown fields, trying to
clear an extension by setting it to null would throw an NPE.
Change-Id: I6abcdfcc0193de44f97b21dd6cc2f40604938a1a
This CL adds the "parcelable_messages" option. When enabled, all
generated message classes will conform to the Android Parcelable
contract. This is achieved by introducing a new parent class for
generated classes which implements the required functionality.
Since the store_unknown_fields option also makes use of a superclass,
ExtendableMessageNano, we have two versions of the new Parcelable
superclass: one extending MessageNano, and one extending
ExtendableMessageNano. These classes are otherwise identical.
As these classes depend on Android framework jars, they are not
included in the host .jar build of the nanoproto library.
Finally, add a test suite for running tests of Android-specific
functionality, as this cannot be done on a desktop JVM.
Change-Id: Icc2a257f03317e947f7078dbb9857c3286857497
Nano proto compiler normally throws an error if any service is
defined. If --ignore-services=true is set, no error is thrown and the
service is simply skipped.
Change-Id: Id82583555085cc55550d03a485d3f0189885240b
This avoids a race-condition when cachedSize is momentarily set to 0
for non-empty messages if multiple threads call getSerializedSize
(e.g. during serialization).
This is a retry of https://android-review.googlesource.com/#/c/88570/.
getSerializedSize() has been kept non-final so that messages generated
with a previous version of the compiler will not break.
Change-Id: I8d8154a10938cde579ae19c55eae55b1e70e0bda
This avoids a race-condition when cachedSize is momentarily set to 0
for non-empty messages if multiple threads call getSerializedSize
(e.g. during serialization).
Change-Id: I15a8ded92edbf41bf1c8d787960c5bbbc8a323c5
Invalid values from the wire are silently ignored.
Unlike full/lite, the invalid values are not stored into the
unknown fields, because there's no way to get them out from
Nano's unknown fields without a matching Extension.
Edited README and slightly moved it towards a standalone
section for Nano, independent of the Micro section.
Change-Id: I2c1eb07f4d6d8f3aea242b8ddd95b9c966f3f177
Special values for float and double make it inaccurate to test the equality with ==.
The main Java library uses the standard Object.equals() implementation for all fields,
which for floating point fields means Float.equals() or Double.equals(). They define
equality as bitwise equality, with all NaN representations normalized to the same bit
sequence (and therefore equal to each other). This test checks that the nano
implementation complies with Object.equals(), so NaN == NaN and +0.0 != -0.0.
Change-Id: I97bb4a3687223d8a212c70cd736436b9dd80c1d7
- Get rid of TypeLiteral<T>. It was introduced to read the component
type of a List<T> at runtime. But we use arrays everywhere else,
and we can always read the component type of an array type at
runtime.
- Properly read/write "minor" types (e.g. sint32, sfixed32). The old
implementation could only read/write data as the "typical" types
(one per Java type), e.g. java.lang.Integer -> int32, java.lang.Long
-> int64. So if e.g. an extension specifies sfixed32 as the type, it
would be read/written in the totally incompatible int32 format.
- Properly serialize repeated packed fields. The old implementation
doesn't do packed serialization. As an added bonus, and to be more
aligned with the rest of protobuf nano / main, repeated packable
extensions can deserialize both packed and non-packed data.
- Split Extension class into a hierarchy so under typical usage a
large chunk of code dealing with primitive type extensions can be
removed by ProGuard.
Bug: https://code.google.com/p/android/issues/detail?id=62586
Change-Id: I0d692f35cc2a8ad3a5a1cb3ce001282b2356b041
accessors mode switches proto fields away from being public fields (which is
how MessageNanoPrinter found which fields to print via reflection). Add a
pass through the methods looking for generated accessor methods to print
those as well.
Change-Id: I7c47853ecbd5534086f44b25a89dbbe56f63ed03
Class initializers prevent ProGuard from inlining any methods
because it thinks the class initializer may have side effects.
This is true for static methods, but instance methods can still
be inlined, because to have an instance you will have touched
the class and any class initializers would have run. But
ProGuard only starts inlining instance methods of classes with
class initializers from v4.11b6, and Android uses v4.4 now.
This change tries to avoid the class initializers as much as
possible, by delaying the initialization of the empty array and
some fields' saved defaults until when they're needed. However,
if the message hosts any extensions, they must be public static
final and therefore introducing the class initializer. In that
case we won't bother with lazy initialization.
Change-Id: I00d8296f6eb0023112b93ee135cdb28dbd52b0b8
For nested message objects, don't generate accessor methods because they have
a default value that is not a valid value (null), so there is no reason to have
get/set/has/clear methods for them. Clients and protos (while serializing) can
check against the invalid value to see if it's been set.
Change-Id: Ic63400889581271b8cbcd9c45c84519d4921fd4b
It is a requirement for parsing code to handle packed and unpacked
forms on the wire for repeated packable fields. This change aligns
the javanano's behavior with the java's.
Bonus: optimize array length calculation when parsing repeated
fixed-size-element-type fields.
Bonus 2: lose "xMemoizedSerializedSize" for repeated enum fields,
and make the serialized size calculation match that for repeated
int32 fields.
Change-Id: I8a06103d9290234adb46b0971b5ed155544fe86a
The output of toString is now aligned with that used by non-nano and C++
runtimes, with the exception of groups. Groups should be serialized using a
camelized name (e.g. "FooBar" rather than "foo_bar") however the nano runtime
does not have information on which fields are groups.
Changes are:
- bytes fields are output within double-quotes, non-printable characters are
output as octal escape sequences (i.e. \NNN);
- field identifiers are output in underscored format;
- unset fields are not output (rather than printing "null");
- the type name of the root message is not output.
With these changes the nano toString, normal toString, and C++'s DebugString all
produce equivalent output when given the same message. (Provided that message
uses no deprecated features.)
Change-Id: Id4791d73822846db29344db9f7bc3781c3e183a6
- Migrates getCachedSize to the MessageNano parent class to save one method per message.
- Create ExtendableMessageNano parent class for protos with extensions, this saves the
getExtension and setExtension methods on the relevant messages.
- getSerializedSize's default case (with no fields to serialize) also migrate to the
parent class, which saves methods on empty messages.
- Container classes become interfaces to save the constructor.
Change-Id: I81f1a1b6d6a660096835e9df3ea20456655aab4a
Strip the null elements out before serializing the array.
This is helpful in the cases where the user wants to construct
an array of an inexact size for serialization. For example:
User constructs array of size 5 because they anticipate adding
more than 1 element before serialization. Only 3 get added, so
the array looks like [Obj, Obj, Obj, null, null]. This would
curently crash without this CL.
All repeated fields of ref-type elements can contain null
elements: repeated strings, repeated bytes, and repeated
messages/groups.
Change-Id: I117391c868c9a436536d70d6151780e9cc7e8227
Conflicts:
src/google/protobuf/compiler/javanano/javanano_message_field.cc
The option is only called 'generate_equals' because:
- equals() is the main thing; hashCode() is there only to
complement equals();
- it's shorter;
- toString() should not be included in this option because
it's more for debugging and it's more likely to stop
ProGuard from working well.
Also shortened the "has bit" expression; was
((bitField & mask) == mask), now ((bitField & mask) != 0).
Both the Java code and the bytecode are slightly shorter.
Change-Id: Ic309a08a60883bf454eb6612679aa99611620e76
Also pre-inlines set() and has() in serialization code. This could
theoretically help ProGuard: the message class size is usually large,
and because of this only, it may refuse to inline an accessor into
the serialization code, and as a result keeps the accessor intact.
Chances are, after pre-inlining all accessor calls within the message
class, those accessors become unused or single-use, so there are more
reasons for ProGuard to inline and then remove them.
Change-Id: I57decbe0b2533c1be21439de0aad15f49c7024dd
The public doc states that repeated fields are simply concatenated
and doesn't impose a different semantics for packed fields. This
CL fixes this for packed fields and adds tests covering all cases.
Also fixed a bit of missed null-repeated-field treatments.
Change-Id: Ie35277bb1a9f0b8171dc9d07b6adf9b9d3308de2
There's no distinction between a repeated field being null and being
empty. In both cases, nothing is sent on the wire. Clients might for
whatever reason inadvertently set a repeated field to null, so
protect against that and treat it just as if the field was empty.
Change-Id: Ic3846f7f2189d6cfff6f8ef3ca217daecc3c8be7
This option generates fields as reference types, and serializes
based on nullness.
Change-Id: Ic32e0eebff59d14016cc9a19e15a9bb08ae0bba5
Signed-off-by: Brian Duff <bduff@google.com>
When parsing a group, the group's end tag should not be stored within the
message's unknownFieldData. Not only does this waste space, it is also output
the next time the group is serialized, resulting in two end tags for that group.
The resulting bytes are not always a valid protocol buffer and may fail to
parse.
This change ensures that group end tags do not result in an unknownFieldData
entry, and that messages with groups can be roundtripped without corruption.
Change-Id: I240f858a7217a7652b756598c34aacad5dcc3363
Conflicts:
java/src/test/java/com/google/protobuf/NanoTest.java
This javanano_out command line option creates a container interface
at the normal place where the enum constants would reside, per enum
definition. The java_multiple_files flag would now affect the file-
scope enums with the shells. If the flag is true then file-scope
container interfaces are created in their own files.
Change-Id: Id52258fcff8d3dee9db8f3d8022147a811bf3565
This CL implements the 'optional_field_style=accessors' option.
All optional fields will now be 1 Java field and 1 bit in a shared
bitfield behind get/set/has/clear accessor methods. The setter
performs null check for reference types (Strings and byte[]s).
Also decentralized the clear code generation.
Change-Id: I60ac78329e352e76c2f8139fba1f292383080ad3
- File class name is defined as the java_outer_classname option value
or the file name ToCamelCase; never the single message's ClassName.
- File-scope enums are translated to constants in the file class,
regardless of java_multiple_files.
- If java_multiple_files=true, and file's class name equals a message's
class name, no error. This is done by detecting that the outer class
is not needed and skipping the outer class codegen and clash checks.
Note: there is a disparity between java[lite] and the previous
java{micr|nan}o: when generating code for a single-message proto, the
outer class is omitted by java{micr|nan}o if the file does not have
java_outer_classname. This change makes java{micr|nan}o align with
java[lite] codegen and create the outer class, but will print some
info to warn of potential change of code.
- Also fixed the "is_own_file" detection and made all parseX() methods
static. Previously, all messages in a java_multiple_files=true file
are (incorrectly) considered to be in their own files, including
nested messages, causing them to become inner classes (instance-
bound) and forcing the parseX() methods to lose the static modifier.
- This change supersedes c/60164 and c/60086, which causes javanano to
put enum values into enum shell classes if java_multiple_files=true.
We now always use the parent class to host the enum values. A future
change will add a command line option to provide more flexibility.
- Elaborated in java/README.txt.
Change-Id: I684932f90e0a028ef37c662b221def5ffa202439
Remove buggy memoization. Memoization also is too fragile for the api
because the repeated field is public.
Change-Id: I538b8426d274b22df2eeea5935023abbe7df49fe
Imported source files may have different values for the 'java_multiple_files'
option to the main source file's. Whether the fully qualified Java name of an
entity should include the outer class name depends on the flag value in the
file defining the referenced entity, not the main file. This CL loads the
flag values from the main and all transitively imported files into the params,
and generates the fully qualified Java names accordingly.
If the generator option 'java_multiple_files' is set, its value overrides any
in-file values in all source/imported files. This is because this generator
option is typically used on either none or all source files.
Change-Id: Id6a4a42426d68961dc669487d38f35530deb7d8e
Adds support for default values of NaN, infinity and negative
infinity for floats and doubles in both the nano and micro
java compiler.
Change-Id: Ibc43e5ebb073e51d9a8181f3aa23b72e10015dca
You can now do:
MyMessage foo = MessageNano.mergeFrom(new MyMessage(), bytes);
without having to cast the message returned from mergeFrom.
Change-Id: Ibb2ad327f75855d45352ad304c7f054f20dd29c9
You can use the processor option store_unknown_fields to switch
this support on:
aprotoc --javanano_out=store_unknown_fields=true:/tmp/out
A separate option for extensions isn't required. Support
for unknown fields must be turned on to allow storing and
retrieving extensions, because they are just stored as
unknown fields. If unknown fields are switched on, extension
related code will be generated when a proto message includes
an extension range, or an extension is encountered.
By default, store_unknown_fields is false. No additional
code is generated, and the generator will error out if protos
contain extension ranges or extensions.
Change-Id: I1e034c9e8f3305612953f72438189a7da6ed2167
- All of the real work for printing the proto is actually done in
MessageNanoPrinter.
- Uses reflection to find proto-defined fields and prints those.
- Prints all fields, even defaults and nulls.
- Also added a simple test to make sure it handles all proto types well.
Tried not to make the test too brittle (but hey it's testing a toString()
so how flexible can it be)
Change-Id: I3e360ef8b0561041e010c1f3445ec45ecdcd2559
Like micro protobufs except:
- No setter/getter/hazzer functions.
- Has state is not available. Outputs all fields != their default.
- CodedInputStream can only take byte[] (not InputStream).
- Repeated fields are in arrays, not ArrayList or Vector.
- Unset messages/groups are null, not "defaultInstance()".
- Required fields are always serialized.
To use:
- Link libprotobuf-java-2.3.0-nano runtime.
- Use LOCAL_PROTOC_OPTIMIZE_TYPE := nano
Change-Id: I7429015b3c5f7f38b7be01eb2d4927f7a9999c80
Like micro protobufs except:
- No setter/getter/hazzer functions.
- Has state is not available. Outputs all fields != their default.
- CodedInputStream can only take byte[] (not InputStream).
- Repeated fields are in arrays, not ArrayList or Vector.
- Unset messages/groups are null, not "defaultInstance()".
- Required fields are always serialized.
To use:
- Link libprotobuf-java-2.3.0-nano runtime.
- Use LOCAL_PROTOC_OPTIMIZE_TYPE := nano
Change-Id: I7429015b3c5f7f38b7be01eb2d4927f7a9999c80
Juan Silveira
Max Cai
Wink Saville
Jan-Willem Maarse
Jeff Davidson
Jie Dai
Dave Hawkey
Andrew Flynn
Nicholas Seckar
Tom Chao
Aurash Mahbod
Brian Duff
Ulas Kirazci
Chris Smith
Matt Thrailkill
xiaofeng@google.com
liujisi@google.com