descriptor.proto uses proto2 syntax, which is not ready for external
usage. However, some proto3 files import descriptor.proto and cannot be
used. In this PR, all references (We cheated by only removing
extensions, which is enough for now. User should avoid using messages
defined in descriptor.proto as field type.) to content in
descriptor.proto are removed from generated files. Those that import
descriptor.proto can be used like other proto files.
This commit adds a __wrap_memcpy function and a linker flag to use that
in place of memcpy for our Ruby gem C extension. This allows us to
always use the 2.2.5 version of memcpy, making it possible to use the
gem on distributions with pre-2.14 versions of glibc.
Before this change:
$ objdump -T protobuf_c.so | grep memcpy
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.3.4 __memcpy_chk
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.14 memcpy
After:
$ objdump -T protobuf_c.so | grep memcpy
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 memcpy
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.3.4 __memcpy_chk
0000000000042450 g DF .text 0000000000000005 Base __wrap_memcpy
This is based on gRPC's solution to a similar problem:
5098508d2d/src/core/lib/support/wrap_memcpy.c
This fixes issue #2783.
* Changing DOTNET35 framework symbols in preprocessor directives to the default built-in value of NET35.
* Adding extension method StreamExtension.CopyTo for .NET 3.5 because it didn’t exist until .NET 4, and adding associated unit tests.
* Down-integrate internal changes to github.
* Update conformance test failure list.
* Explicitly import used class in nano test to avoid random test fail.
* Update _GNUC_VER to use the correct implementation of atomic operation
on Mac.
* maps_test.js: check whether Symbol is defined before using it (#2524)
Symbol is not yet available on older versions of Node.js and so this
test fails with them. This change just directly checks whether Symbol is
available before we try to use it.
* Added well_known_types_embed.cc to CLEANFILES so that it gets cleaned up
* Updated Makefile.am to fix out-of-tree builds
* Added Bazel genrule for generating well_known_types_embed.cc
In pull request #2517 I made this change for the CMake and autotools
builds but forgot to do it for the Bazel build.
* Update _GNUC_VER to use the correct implementation of atomic operation on Mac.
* Add new js file in extra dist.
* Bump version number to 3.2.0
* Fixed issue with autoloading - Invalid paths (#2538)
* PHP fix int64 decoding (#2516)
* fix int64 decoding
* fix int64 decoding + tests
* Fix int64 decoding on 32-bit machines.
* Fix warning in compiler/js/embed.cc
embed.cc: In function ‘std::string CEscape(const string&)’:
embed.cc:51:32: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (int i = 0; i < str.size(); ++i) {
^
* Fix include in auto-generated well_known_types_embed.cc
Restore include style fix (e3da722) that has been trampled by
auto-generation of well_known_types_embed.cc
* Fixed cross compilations with the Autotools build
Pull request #2517 caused cross compilations to start failing, because
the js_embed binary was being built to run on the target platform
instead of on the build machine. This change updates the Autotools build
to use the AX_PROG_CXX_FOR_BUILD macro to find a suitable compiler for
the build machine and always use that when building js_embed.
* Minor fix for autocreated object repeated fields and maps.
- If setting/clearing a repeated field/map that was objects, check the class
before checking the autocreator.
- Just to be paranoid, don’t mutate within copy/mutableCopy for the autocreated
classes to ensure there is less chance of issues if someone does something
really crazy threading wise.
- Some more tests for the internal AutocreatedArray/AutocreatedDictionary
classes to ensure things are working as expected.
- Add Xcode 8.2 to the full_mac_build.sh supported list.
* Fix generation of extending nested messages in JavaScript (#2439)
* Fix generation of extending nested messages in JavaScript
* Added missing test8.proto to build
* Fix generated code when there is no namespace but there is enum definition.
* Decoding unknown field should succeed.
* Add embed.cc in src/Makefile.am to fix dist check.
* Fixed "make distcheck" for the Autotools build
To make the test pass I needed to fix out-of-tree builds and update
EXTRA_DIST and CLEANFILES.
* Remove redundent embed.cc from src/Makefile.am
* Update version number to 3.2.0-rc.1 (#2578)
* Change protoc-artifacts version to 3.2.0-rc.1
* Update version number to 3.2.0rc2
* Update change logs for 3.2.0 release.
* Update php README
* Update upb, fixes some bugs (including a hash table problem). (#2611)
* Update upb, fixes some bugs (including a hash table problem).
* Ruby: added a test for the previous hash table corruption.
Verified that this triggers the bug in the currently released
version.
* Ruby: bugfix for SEGV.
* Ruby: removed old code for dup'ing defs.
* Reverting deployment target to 7.0 (#2618)
The Protobuf library doesn’t require the 7.1 deployment target so
reverting it back to 7.0
* Fix typo that breaks builds on big-endian (#2632)
* Bump version number to 3.2.0
This consists of:
- Changing the codegen for the fixed set of options protos, to parse unknown fields instead of skipping them
- Add a new CustomOptions type in the C# support library
- Expose CustomOptions properties from the immutable proto wrappers in the support library
Only single-value options are currently supported, and fetching options values requires getting the type right
and knowing the field number. Both of these can be addressed at a later time.
Fixes#2143, at least as a first pass.
- If setting/clearing a repeated field/map that was objects, check the class
before checking the autocreator.
- Just to be paranoid, don’t mutate within copy/mutableCopy for the autocreated
classes to ensure there is less chance of issues if someone does something
really crazy threading wise.
- Some more tests for the internal AutocreatedArray/AutocreatedDictionary
classes to ensure things are working as expected.
- Add Xcode 8.2 to the full_mac_build.sh supported list.
- Check the parent file options for deprecation when deciding to tag Messages
and Enums as deprecated.
- Within the generated source push/pop the warning for implementing deprecated
things around a deprecated class implementation.
- Annotate the methods generated for extension fields as deprecated.
- Add a testing .proto file that covers deprecated fields, messages, enums,
enum values and compile it into the unittests to confirm things compile
cleanly.
- Add a testing .proto file that uses the file level option to make everything
deprecated and compile it into the unittests to confirm things compile
cleanly.
* Factored Conformance test messages into shared test schema.
* Updated benchmarks to use new proto3 message locations.
* Fixed include path.
* Conformance: fixed include of Python test messages.
* Make maven in Rakefile use --batch-mode.
* Revert changes to benchmarks.
On second thought I think a separate schema for
CPU benchmarking makes sense.
* Try regenerating C# protos for new test protos.
* Removed benchmark messages from test proto.
* Added Jon Skeet's fixes for C#.
* Removed duplicate/old test messages C# file.
* C# fixes for test schema move.
* Fixed C# to use the correct TestAllTypes message.
* Fixes for Objective C test schema move.
* Added missing EXTRA_DIST file.
At generation time, walk the file's dependencies to see what really contains
extensions so we can generate more minimal code that only links together the
roots that provided extensions. Gets a bunch of otherwise noop code out of
the call flow when the roots are +initialized.
This takes the code that was sitting in benchmarks/
already and makes it easier for language-specific
benchmarks to consume. Future PRs will enhance this
so that the language-specific benchmarks can report
metrics back that will be tracked over time in PerfKit.
Overview of changes:
- A new C#-specific command-line option, legacy_enum_values to revert to the old behavior
- When legacy_enum_values isn't specified, we strip the enum name as a prefix, and PascalCase the value name
- A new attribute within the C# code so that we can always tell the original in-proto name
Regenerating the C# code with legacy_enum_values leads to code which still compiles and works - but
there's more still to do.
NOTE: This is a binary breaking change as structure sizes have changed size
and/or order.
- Drop capturing field options, no other options were captured and other mobile
targeted languages don't try to capture this sort information (saved 8
bytes for every field defined (in static data and again in field descriptor
instance size data).
- No longer generate/compile in the messages/enums in descriptor.proto. If
developers need it, they should generate it and compile it in. Reduced the
overhead of the core library.
- Compute the number of has_bits actually needs to avoid over reserving.
- Let the boolean single fields store via a has_bit to avoid storage, makes
the common cases of the instance size smaller.
- Reorder some flags and down size the enums to contain the bits needed.
- Reorder the items in the structures to manually ensure they are are packed
better (especially when generating 64bit code - 8 bytes for every field,
16 bytes for every extension, instance sizes 8 bytes also).
- Split off the structure initialization so when the default is zero, the
generated static storage doesn't need to reserve the space. This is batched
at the message level, so all the fields for the message have to have zero
defaults to get the saves. By definition all proto3 syntax files fall into
this case but it also saves space for the proto2 that use the standard
defaults. (saves 8 bytes of static data for every field that had a zero
default)
- Don't track the enums defined by a message. Nothing in the runtime needs it
and it was just generation and runtime overhead. (saves 8 bytes per enum)
- Ensure EnumDescriptors are started up threadsafe in all cases.
- Split some of the Descriptor initialization into multiple methods so the
generated code isn't padded with lots of zero/nil args.
- Change how oneof info is feed to the runtime enabling us to generate less
static data (8 bytes saved per oneof for 64bit).
- Change how enum value informat is capture to pack the data and only decode
it if it ends up being needed. Avoids padding issues causing bloat of 64bit,
and removes the needs for extra pointers in addition to the data (just the
data and one pointer now).
Recently, descriptor.proto gained a GeneratedCodeInfo message, which means the generated code conflicts with our type.
Unfortunately this affects codegen as well, although this is a part of the public API which is very unlikely to affect hand-written code.
Generated code changes in next commit.
Added instructions on what tools to install to compile protobuf from
source. Removed the INSTALL.txt file because it's just a simple copy of
the autoconf documentation and confuses users.
Change-Id: I6fd8aa13495f1238fe5c62451b95ad480b1c4bed
Apple engineers have pointed out that OSSpinLocks are vulnerable to live locking
on iOS in cases of priority inversion:
. http://mjtsai.com/blog/2015/12/16/osspinlock-is-unsafe/
. https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20151214/000372.html
- Use a dispatch_semaphore_t within the extension registry.
- Use a dispatch_semaphore_t for protecting autocreation within messages.
- Drop the custom/internal GPBString class since we don't have really good
numbers to judge the locking replacements and it isn't required. We can
always bring it back with real data in the future.
Paul Yang
Feng Xiao
Adam Cozzette
John Brock
Jon Skeet
Marcus Longmuir
Thomas Van Lenten
Joshua Haberman
Bo Yang
Jason Lunn
Jisi Liu
Jon Skeet
Andy Hochhaus