Php & Ruby Cherry Picks for 3.17.1 (#8632)

* Some more updates to PHP testing infrastructure (#8576)

* WIP.

* Added build config for all of the tests.

* Use ../src/protoc if it is available, for cases where Bazel isn't available.

* Added test_php.sh.

* Fix for the broken macOS tests.

* Move all jobs to use php80 instead of lots of separate jobs.

* Only pass -t flag if we are running in a terminal.

* Updated php_all job to use new Docker stuff.

* Fixed PHP memory leaks and arginfo errors (#8614)

* Fixed a bunch of incorrect arginfo and a few incorrect error messages.

* Passes mem check test with no leaks!

* WIP.

* Fix build warning that was causing Bazel build to fail.

* Added compatibility code for PHP <8.0.

* Added test_valgrind target and made tests Valgrind-clean.

* Updated Valgrind test to fail if memory leaks are detected.

* Removed intermediate shell script so commands are easier to cut, paste, and modify.

* Passing all Valgrind tests!

* Hoist addref into ObjCache_Get().

* Removed special case of map descriptors by keying object map on upb_msgdef.

* Removed all remaining RETURN_ZVAL() macros.

* Removed all explicit reference add/del operations.

* Added REFCOUNTING.md to Makefile.am.

* Updated upb version and fixed PHP to not get unset message field. (#8621)

* Updated upb version and fixed PHP to not get unset message field.

* Updated changelog.

* Fixed preproc test to handle old versions of Clang withot __has_attribute().

* A second try at fixing __has_attribute().

* Copy __has_attribute() fix to cc file also.

* Updated failure list for PHP for fixed test.

* Updated version of upb for Ruby (#8624)

* Updated upb.

* Preserve legacy behavior for unset messages.

* Updated failure list.

* Updated CHANGES.txt.

* Added erroneously-deleted test file.

* Fixed condition on compatibility code.

* Re-introduced deleted file again, and fixed Rakefile to not delete it.

* Fix generation of test protos.
pull/8633/head
Joshua Haberman 4 years ago committed by GitHub
parent 65abb64e73
commit 0b87475592
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 9
      CHANGES.txt
  2. 1
      Makefile.am
  3. 2
      conformance/failure_list_php_c.txt
  4. 2
      conformance/failure_list_ruby.txt
  5. 27
      kokoro/linux/php80/build.sh
  6. 8
      kokoro/linux/php80/continuous.cfg
  7. 8
      kokoro/linux/php80/presubmit.cfg
  8. 28
      kokoro/linux/php_all/build.sh
  9. 112
      php/REFCOUNTING.md
  10. 1
      php/composer.json
  11. 6
      php/ext/google/protobuf/array.c
  12. 13
      php/ext/google/protobuf/convert.c
  13. 5
      php/ext/google/protobuf/convert.h
  14. 202
      php/ext/google/protobuf/def.c
  15. 7
      php/ext/google/protobuf/def.h
  16. 8
      php/ext/google/protobuf/map.c
  17. 31
      php/ext/google/protobuf/message.c
  18. 1569
      php/ext/google/protobuf/php-upb.c
  19. 618
      php/ext/google/protobuf/php-upb.h
  20. 20
      php/ext/google/protobuf/protobuf.c
  21. 14
      php/ext/google/protobuf/protobuf.h
  22. 351
      php/ext/google/protobuf/wkt.inc
  23. 17
      php/generate_test_protos.sh
  24. 18
      php/prepare_c_extension.sh
  25. 8
      php/tests/ArrayTest.php
  26. 1
      php/tests/EncodeDecodeTest.php
  27. 24
      php/tests/GeneratedClassTest.php
  28. 7
      php/tests/compile_extension.sh
  29. 77
      ruby/Rakefile
  30. 8
      ruby/ext/google/protobuf_c/message.c
  31. 1566
      ruby/ext/google/protobuf_c/ruby-upb.c
  32. 615
      ruby/ext/google/protobuf_c/ruby-upb.h
  33. 19
      src/google/protobuf/compiler/php/php_generator.cc

@ -1,3 +1,12 @@
2021-05-07 version 3.17.1 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
PHP
* Fixed JSON parser to allow multiple values from the same oneof as long as
all but one are null.
Ruby
* Fixed JSON parser to allow multiple values from the same oneof as long as
all but one are null.
2021-05-07 version 3.17.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Protocol Compiler

@ -801,6 +801,7 @@ objectivec_EXTRA_DIST= \
php_EXTRA_DIST= \
composer.json \
php/README.md \
php/REFCOUNTING.md \
php/composer.json \
php/ext/google/protobuf/arena.c \
php/ext/google/protobuf/arena.h \

@ -1,4 +1,2 @@
Recommended.Proto2.JsonInput.FieldNameExtension.Validator
Required.Proto2.JsonInput.StoresDefaultPrimitive.Validator
Required.Proto3.JsonInput.OneofFieldNullSecond.JsonOutput
Required.Proto3.JsonInput.OneofFieldNullSecond.ProtobufOutput

@ -56,5 +56,3 @@ Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT32.PackedInput.UnpackedOu
Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT32.UnpackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT64.PackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT64.UnpackedInput.UnpackedOutput.ProtobufOutput
Required.Proto3.JsonInput.OneofFieldNullSecond.JsonOutput
Required.Proto3.JsonInput.OneofFieldNullSecond.ProtobufOutput

@ -1,18 +1,17 @@
#!/bin/bash
#
# This is the top-level script we give to Kokoro as the entry point for
# running the "pull request" project:
#
# This script selects a specific Dockerfile (for building a Docker image) and
# a script to run inside that image. Then we delegate to the general
# build_and_run_docker.sh script.
# This is the entry point for kicking off a Kokoro job. This path is referenced
# from the .cfg files in this directory.
set -ex
cd $(dirname $0)
# Change to repo root
cd $(dirname $0)/../../..
# Most of our tests use a debug build of PHP, but we do one build against an opt
# php just in case that surfaces anything unexpected.
../test_php.sh gcr.io/protobuf-build/php/linux:8.0.5-14a06550010c0649bf69b6c9b803c1ca609bbb6d
export DOCKERHUB_ORGANIZATION=protobuftesting
export DOCKERFILE_DIR=kokoro/linux/dockerfile/test/php80
export DOCKER_RUN_SCRIPT=kokoro/linux/pull_request_in_docker.sh
export OUTPUT_DIR=testoutput
export TEST_SET="php8.0_all"
./kokoro/linux/build_and_run_docker.sh
../test_php.sh gcr.io/protobuf-build/php/linux:7.0.33-dbg-14a06550010c0649bf69b6c9b803c1ca609bbb6d
../test_php.sh gcr.io/protobuf-build/php/linux:7.3.28-dbg-14a06550010c0649bf69b6c9b803c1ca609bbb6d
../test_php.sh gcr.io/protobuf-build/php/linux:7.4.18-dbg-14a06550010c0649bf69b6c9b803c1ca609bbb6d
../test_php.sh gcr.io/protobuf-build/php/linux:8.0.5-dbg-14a06550010c0649bf69b6c9b803c1ca609bbb6d

@ -2,10 +2,4 @@
# Location of the build script in repository
build_file: "protobuf/kokoro/linux/php80/build.sh"
timeout_mins: 120
action {
define_artifacts {
regex: "**/sponge_log.xml"
}
}
timeout_mins: 20

@ -2,10 +2,4 @@
# Location of the build script in repository
build_file: "protobuf/kokoro/linux/php80/build.sh"
timeout_mins: 120
action {
define_artifacts {
regex: "**/sponge_log.xml"
}
}
timeout_mins: 20

@ -1,18 +1,20 @@
#!/bin/bash
#
# This is the top-level script we give to Kokoro as the entry point for
# running the "pull request" project:
#
# This script selects a specific Dockerfile (for building a Docker image) and
# a script to run inside that image. Then we delegate to the general
# build_and_run_docker.sh script.
# This is the entry point for kicking off a Kokoro job. This path is referenced
# from the .cfg files in this directory.
set -ex
# Change to repo root
# Change to repo base.
cd $(dirname $0)/../../..
export DOCKERHUB_ORGANIZATION=protobuftesting
export DOCKERFILE_DIR=kokoro/linux/dockerfile/test/php
export DOCKER_RUN_SCRIPT=kokoro/linux/pull_request_in_docker.sh
export OUTPUT_DIR=testoutput
export TEST_SET="php_all"
./kokoro/linux/build_and_run_docker.sh
docker run $(test -t 0 && echo "-it") -v$PWD:/workspace gcr.io/protobuf-build/php/linux:8.0.5-dbg-14a06550010c0649bf69b6c9b803c1ca609bbb6d "composer test_valgrind"
docker run $(test -t 0 && echo "-it") -v$PWD:/workspace gcr.io/protobuf-build/php/linux:7.0.33-dbg-14a06550010c0649bf69b6c9b803c1ca609bbb6d "composer test && composer test_c"
docker run $(test -t 0 && echo "-it") -v$PWD:/workspace gcr.io/protobuf-build/php/linux:7.3.28-dbg-14a06550010c0649bf69b6c9b803c1ca609bbb6d "composer test && composer test_c"
docker run $(test -t 0 && echo "-it") -v$PWD:/workspace gcr.io/protobuf-build/php/linux:7.4.18-dbg-14a06550010c0649bf69b6c9b803c1ca609bbb6d "composer test && composer test_c"
docker run $(test -t 0 && echo "-it") -v$PWD:/workspace gcr.io/protobuf-build/php/linux:8.0.5-dbg-14a06550010c0649bf69b6c9b803c1ca609bbb6d "composer test && composer test_c"
# Most of our tests use a debug build of PHP, but we do one build against an opt
# php just in case that surfaces anything unexpected.
docker run $(test -t 0 && echo "-it") -v$PWD:/workspace gcr.io/protobuf-build/php/linux:8.0.5-14a06550010c0649bf69b6c9b803c1ca609bbb6d "composer test && composer test_c"

@ -0,0 +1,112 @@
# Refcounting Tips
One of the trickiest parts of the C extension for PHP is getting the refcounting
right. These are some notes about the basics of what you should know,
especially if you're not super familiar with PHP's C API.
These notes cover the same general material as [the Memory Management chapter of
the PHP internal's
book](https://www.phpinternalsbook.com/php7/zvals/memory_management.html), but
calls out some points that were not immediately clear to me.
## Zvals
In the PHP C API, the `zval` type is roughly analogous to a variable in PHP, eg:
```php
// Think of $a as a "zval".
$a = [];
```
The equivalent PHP C code would be:
```c
zval a;
ZVAL_NEW_ARR(&a); // Allocates and assigns a new array.
```
PHP is reference counted, so each variable -- and thus each zval -- will have a
reference on whatever it points to (unless its holding a data type that isn't
refcounted at all, like numbers). Since the zval owns a reference, it must be
explicitly destroyed in order to release this reference.
```c
zval a;
ZVAL_NEW_ARR(&a);
// The destructor for a zval, this must be called or the ref will be leaked.
zval_ptr_dtor(&a);
```
Whenever you see a `zval`, you can assume it owns a ref (or is storing a
non-refcounted type). If you see a `zval*`, which is also quite common, then
this is *pointing to* something that owns a ref, but it does not own a ref
itself.
The [`ZVAL_*` family of
macros](https://github.com/php/php-src/blob/4030a00e8b6453aff929362bf9b25c193f72c94a/Zend/zend_types.h#L883-L1109)
initializes a `zval` from a specific value type. A few examples:
* `ZVAL_NULL(&zv)`: initializes the value to `null`
* `ZVAL_LONG(&zv, 5)`: initializes a `zend_long` (integer) value
* `ZVAL_ARR(&zv, arr)`: initializes a `zend_array*` value (refcounted)
* `ZVAL_OBJ(&zv, obj)`: initializes a `zend_object*` value (refcounted)
Note that all of our custom objects (messages, repeated fields, descriptors,
etc) are `zend_object*`.
The variants that initialize from a refcounted type do *not* increase the
refcount. This makes them suitable for initializing from a newly-created object:
```c
zval zv;
ZVAL_OBJ(&zv, CreateObject());
```
Once in a while, we want to initialize a `zval` while also increasing the
reference count. For this we can use `ZVAL_OBJ_COPY()`:
```c
zend_object *some_global;
void GetGlobal(zval *zv) {
// We want to create a new ref to an existing object.
ZVAL_OBJ_COPY(zv, some_global);
}
```
## Transferring references
A `zval`'s ref must be released at some point. While `zval_ptr_dtor()` is the
simplest way of releasing a ref, it is not the most common (at least in our code
base). More often, we are returning the `zval` back to PHP from C.
```c
zval zv;
InitializeOurZval(&zv);
// Returns the value of zv to the caller and donates our ref.
RETURN_COPY_VALUE(&zv);
```
The `RETURN_COPY_VALUE()` macro (standard in PHP 8.x, and polyfilled in earlier
versions) is the most common way we return a value back to PHP, because it
donates our `zval`'s refcount to the caller, and thus saves us from needing to
destroy our `zval` explicitly. This is ideal when we have a full `zval` to
return.
Once in a while we have a `zval*` to return instead. For example when we parse
parameters to our function and ask for a `zval`, PHP will give us pointers to
the existing `zval` structures instead of creating new ones.
```c
zval *val;
if (zend_parse_parameters(ZEND_NUM_ARGS(), "z", &val) == FAILURE) {
return;
}
// Returns a copy of this zval, adding a ref in the process.
RETURN_COPY(val);
```
When we use `RETURN_COPY`, the refcount is increased; this is perfect for
returning a `zval*` when we do not own a ref on it.

@ -24,6 +24,7 @@
},
"scripts": {
"test_c": "./generate_test_protos.sh && ./tests/compile_extension.sh && php -dextension=ext/google/protobuf/modules/protobuf.so vendor/bin/phpunit --bootstrap tests/force_c_ext.php tests",
"test_valgrind": "./generate_test_protos.sh && ./tests/compile_extension.sh && ZEND_DONT_UNLOAD_MODULES=1 USE_ZEND_ALLOC=0 valgrind --leak-check=full --error-exitcode=1 php -dextension=ext/google/protobuf/modules/protobuf.so vendor/bin/phpunit --bootstrap tests/force_c_ext.php tests",
"test": "./generate_test_protos.sh && vendor/bin/phpunit tests",
"aggregate_metadata_test": "./generate_test_protos.sh --aggregate_metadata && vendor/bin/phpunit tests"
}

@ -337,7 +337,7 @@ PHP_METHOD(RepeatedField, offsetGet) {
msgval = upb_array_get(intern->array, index);
Convert_UpbToPhp(msgval, &ret, intern->type, &intern->arena);
RETURN_ZVAL(&ret, 0, 1);
RETURN_COPY_VALUE(&ret);
}
/**
@ -447,7 +447,7 @@ PHP_METHOD(RepeatedField, count) {
PHP_METHOD(RepeatedField, getIterator) {
zval ret;
RepeatedFieldIter_make(&ret, getThis());
RETURN_ZVAL(&ret, 0, 1);
RETURN_COPY_VALUE(&ret);
}
ZEND_BEGIN_ARG_INFO_EX(arginfo_construct, 0, 0, 1)
@ -579,7 +579,7 @@ PHP_METHOD(RepeatedFieldIter, current) {
msgval = upb_array_get(array, index);
Convert_UpbToPhp(msgval, &ret, field->type, &field->arena);
RETURN_ZVAL(&ret, 0, 1);
RETURN_COPY_VALUE(&ret);
}
/**

@ -76,7 +76,7 @@ PHP_METHOD(Util, checkMapField) {
&val_type, &klass) == FAILURE) {
return;
}
RETURN_ZVAL(val, 1, 0);
RETURN_COPY(val);
}
// The result of checkRepeatedField() is assigned, so we need to return the
@ -89,13 +89,18 @@ PHP_METHOD(Util, checkRepeatedField) {
FAILURE) {
return;
}
RETURN_ZVAL(val, 1, 0);
RETURN_COPY(val);
}
ZEND_BEGIN_ARG_INFO_EX(arginfo_checkPrimitive, 0, 0, 1)
ZEND_ARG_INFO(0, value)
ZEND_END_ARG_INFO()
ZEND_BEGIN_ARG_INFO_EX(arginfo_checkString, 0, 0, 1)
ZEND_ARG_INFO(0, value)
ZEND_ARG_INFO(0, check_utf8)
ZEND_END_ARG_INFO()
ZEND_BEGIN_ARG_INFO_EX(arginfo_checkMessage, 0, 0, 2)
ZEND_ARG_INFO(0, value)
ZEND_ARG_INFO(0, class)
@ -123,7 +128,7 @@ static zend_function_entry util_methods[] = {
ZEND_ACC_PUBLIC|ZEND_ACC_STATIC)
PHP_ME(Util, checkUint64, arginfo_checkPrimitive,
ZEND_ACC_PUBLIC|ZEND_ACC_STATIC)
PHP_ME(Util, checkEnum, arginfo_checkPrimitive,
PHP_ME(Util, checkEnum, arginfo_checkMessage,
ZEND_ACC_PUBLIC|ZEND_ACC_STATIC)
PHP_ME(Util, checkFloat, arginfo_checkPrimitive,
ZEND_ACC_PUBLIC|ZEND_ACC_STATIC)
@ -131,7 +136,7 @@ static zend_function_entry util_methods[] = {
ZEND_ACC_PUBLIC|ZEND_ACC_STATIC)
PHP_ME(Util, checkBool, arginfo_checkPrimitive,
ZEND_ACC_PUBLIC|ZEND_ACC_STATIC)
PHP_ME(Util, checkString, arginfo_checkPrimitive,
PHP_ME(Util, checkString, arginfo_checkString,
ZEND_ACC_PUBLIC|ZEND_ACC_STATIC)
PHP_ME(Util, checkBytes, arginfo_checkPrimitive,
ZEND_ACC_PUBLIC|ZEND_ACC_STATIC)

@ -60,9 +60,10 @@ bool Convert_PhpToUpbAutoWrap(zval *val, upb_msgval *upb_val, TypeInfo type,
upb_arena *arena);
// Converts |upb_val| to a PHP zval according to |type|. This may involve
// creating a PHP wrapper object. If type == UPB_TYPE_MESSAGE, then |desc| must
// be the Descriptor for this message type. Any newly created wrapper object
// creating a PHP wrapper object. Any newly created wrapper object
// will reference |arena|.
//
// The caller owns a reference to the returned value.
void Convert_UpbToPhp(upb_msgval upb_val, zval *php_val, TypeInfo type,
zval *arena);

@ -52,6 +52,9 @@ static zend_object *CreateHandler_ReturnNull(zend_class_entry *class_type) {
return NULL; // Nobody should call this.
}
ZEND_BEGIN_ARG_INFO_EX(arginfo_getByIndex, 0, 0, 1)
ZEND_ARG_INFO(0, index)
ZEND_END_ARG_INFO()
// -----------------------------------------------------------------------------
// EnumValueDescriptor
@ -115,12 +118,19 @@ static zend_function_entry EnumValueDescriptor_methods[] = {
typedef struct {
zend_object std;
const upb_enumdef *enumdef;
void *cache_key;
} EnumDescriptor;
zend_class_entry *EnumDescriptor_class_entry;
static zend_object_handlers EnumDescriptor_object_handlers;
void EnumDescriptor_FromClassEntry(zval *val, zend_class_entry *ce) {
static void EnumDescriptor_destructor(zend_object* obj) {
EnumDescriptor *intern = (EnumDescriptor*)obj;
ObjCache_Delete(intern->cache_key);
}
// Caller owns a ref on the returned zval.
static void EnumDescriptor_FromClassEntry(zval *val, zend_class_entry *ce) {
// To differentiate enums from classes, we pointer-tag the class entry.
void* key = (void*)((uintptr_t)ce | 1);
PBPHP_ASSERT(key != ce);
@ -140,16 +150,14 @@ void EnumDescriptor_FromClassEntry(zval *val, zend_class_entry *ce) {
zend_object_std_init(&ret->std, EnumDescriptor_class_entry);
ret->std.handlers = &EnumDescriptor_object_handlers;
ret->enumdef = e;
ret->cache_key = key;
ObjCache_Add(key, &ret->std);
// Prevent this from ever being collected (within a request).
GC_ADDREF(&ret->std);
ZVAL_OBJ(val, &ret->std);
}
}
void EnumDescriptor_FromEnumDef(zval *val, const upb_enumdef *m) {
// Caller owns a ref on the returned zval.
static void EnumDescriptor_FromEnumDef(zval *val, const upb_enumdef *m) {
if (!m) {
ZVAL_NULL(val);
} else {
@ -199,7 +207,7 @@ PHP_METHOD(EnumDescriptor, getValue) {
EnumValueDescriptor_Make(&ret, upb_enum_iter_name(&iter),
upb_enum_iter_number(&iter));
RETURN_ZVAL(&ret, 0, 1);
RETURN_COPY_VALUE(&ret);
}
/*
@ -220,13 +228,13 @@ PHP_METHOD(EnumDescriptor, getValueCount) {
* the public and private descriptor.
*/
PHP_METHOD(EnumDescriptor, getPublicDescriptor) {
RETURN_ZVAL(getThis(), 1, 0);
RETURN_COPY(getThis());
}
static zend_function_entry EnumDescriptor_methods[] = {
PHP_ME(EnumDescriptor, getPublicDescriptor, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(EnumDescriptor, getValueCount, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(EnumDescriptor, getValue, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(EnumDescriptor, getValue, arginfo_getByIndex, ZEND_ACC_PUBLIC)
ZEND_FE_END
};
@ -242,6 +250,11 @@ typedef struct {
zend_class_entry *OneofDescriptor_class_entry;
static zend_object_handlers OneofDescriptor_object_handlers;
static void OneofDescriptor_destructor(zend_object* obj) {
OneofDescriptor *intern = (OneofDescriptor*)obj;
ObjCache_Delete(intern->oneofdef);
}
static void OneofDescriptor_FromOneofDef(zval *val, const upb_oneofdef *o) {
if (o == NULL) {
ZVAL_NULL(val);
@ -254,10 +267,6 @@ static void OneofDescriptor_FromOneofDef(zval *val, const upb_oneofdef *o) {
ret->std.handlers = &OneofDescriptor_object_handlers;
ret->oneofdef = o;
ObjCache_Add(o, &ret->std);
// Prevent this from ever being collected (within a request).
GC_ADDREF(&ret->std);
ZVAL_OBJ(val, &ret->std);
}
}
@ -302,7 +311,7 @@ PHP_METHOD(OneofDescriptor, getField) {
const upb_fielddef *field = upb_oneof_iter_field(&iter);
FieldDescriptor_FromFieldDef(&ret, field);
RETURN_ZVAL(&ret, 1, 0);
RETURN_COPY_VALUE(&ret);
}
/*
@ -317,7 +326,7 @@ PHP_METHOD(OneofDescriptor, getFieldCount) {
static zend_function_entry OneofDescriptor_methods[] = {
PHP_ME(OneofDescriptor, getName, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(OneofDescriptor, getField, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(OneofDescriptor, getField, arginfo_getByIndex, ZEND_ACC_PUBLIC)
PHP_ME(OneofDescriptor, getFieldCount, arginfo_void, ZEND_ACC_PUBLIC)
ZEND_FE_END
};
@ -334,6 +343,12 @@ typedef struct {
zend_class_entry *FieldDescriptor_class_entry;
static zend_object_handlers FieldDescriptor_object_handlers;
static void FieldDescriptor_destructor(zend_object* obj) {
FieldDescriptor *intern = (FieldDescriptor*)obj;
ObjCache_Delete(intern->fielddef);
}
// Caller owns a ref on the returned zval.
static void FieldDescriptor_FromFieldDef(zval *val, const upb_fielddef *f) {
if (f == NULL) {
ZVAL_NULL(val);
@ -346,10 +361,6 @@ static void FieldDescriptor_FromFieldDef(zval *val, const upb_fielddef *f) {
ret->std.handlers = &FieldDescriptor_object_handlers;
ret->fielddef = f;
ObjCache_Add(f, &ret->std);
// Prevent this from ever being collected (within a request).
GC_ADDREF(&ret->std);
ZVAL_OBJ(val, &ret->std);
}
}
@ -455,7 +466,7 @@ PHP_METHOD(FieldDescriptor, getEnumType) {
}
EnumDescriptor_FromEnumDef(&ret, e);
RETURN_ZVAL(&ret, 1, 0);
RETURN_COPY_VALUE(&ret);
}
/*
@ -466,7 +477,6 @@ PHP_METHOD(FieldDescriptor, getEnumType) {
PHP_METHOD(FieldDescriptor, getMessageType) {
FieldDescriptor *intern = (FieldDescriptor*)Z_OBJ_P(getThis());
Descriptor* desc = Descriptor_GetFromFieldDef(intern->fielddef);
zval ret;
if (!desc) {
zend_throw_exception_ex(
@ -475,8 +485,7 @@ PHP_METHOD(FieldDescriptor, getMessageType) {
return;
}
ZVAL_OBJ(&ret, &desc->std);
RETURN_ZVAL(&ret, 1, 0);
RETURN_OBJ_COPY(&desc->std);
}
static zend_function_entry FieldDescriptor_methods[] = {
@ -502,78 +511,80 @@ static void Descriptor_destructor(zend_object* obj) {
// collected before the end of the request.
}
// C Functions from def.h //////////////////////////////////////////////////////
static zend_class_entry *Descriptor_GetGeneratedClass(const upb_msgdef *m) {
char *classname =
GetPhpClassname(upb_msgdef_file(m), upb_msgdef_fullname(m));
zend_string *str = zend_string_init(classname, strlen(classname), 0);
zend_class_entry *ce = zend_lookup_class(str); // May autoload the class.
// These are documented in the header file.
zend_string_release (str);
void Descriptor_FromClassEntry(zval *val, zend_class_entry *ce) {
if (ce == NULL) {
if (!ce) {
zend_error(E_ERROR, "Couldn't load generated class %s", classname);
}
free(classname);
return ce;
}
void Descriptor_FromMessageDef(zval *val, const upb_msgdef *m) {
if (m == NULL) {
ZVAL_NULL(val);
return;
}
if (!ObjCache_Get(ce, val)) {
const upb_msgdef *msgdef = NameMap_GetMessage(ce);
if (!msgdef) {
ZVAL_NULL(val);
return;
if (!ObjCache_Get(m, val)) {
zend_class_entry *ce = NULL;
if (!upb_msgdef_mapentry(m)) { // Map entries don't have a class.
ce = Descriptor_GetGeneratedClass(m);
if (!ce) {
ZVAL_NULL(val);
return;
}
}
Descriptor* ret = emalloc(sizeof(Descriptor));
zend_object_std_init(&ret->std, Descriptor_class_entry);
ret->std.handlers = &Descriptor_object_handlers;
ret->class_entry = ce;
ret->msgdef = msgdef;
ObjCache_Add(ce, &ret->std);
// Prevent this from ever being collected (within a request).
GC_ADDREF(&ret->std);
ret->msgdef = m;
ObjCache_Add(m, &ret->std);
Descriptors_Add(&ret->std);
ZVAL_OBJ(val, &ret->std);
}
}
Descriptor* Descriptor_GetFromClassEntry(zend_class_entry *ce) {
zval desc;
Descriptor_FromClassEntry(&desc, ce);
if (Z_TYPE_P(&desc) == IS_NULL) {
return NULL;
static void Descriptor_FromClassEntry(zval *val, zend_class_entry *ce) {
if (ce) {
Descriptor_FromMessageDef(val, NameMap_GetMessage(ce));
} else {
return (Descriptor*)Z_OBJ_P(&desc);
ZVAL_NULL(val);
}
}
Descriptor* Descriptor_GetFromMessageDef(const upb_msgdef *m) {
if (m) {
if (upb_msgdef_mapentry(m)) {
// A bit of a hack, since map entries don't have classes.
Descriptor* ret = emalloc(sizeof(Descriptor));
zend_object_std_init(&ret->std, Descriptor_class_entry);
ret->std.handlers = &Descriptor_object_handlers;
ret->class_entry = NULL;
ret->msgdef = m;
// Prevent this from ever being collected (within a request).
GC_ADDREF(&ret->std);
return ret;
}
static Descriptor* Descriptor_GetFromZval(zval *val) {
if (Z_TYPE_P(val) == IS_NULL) {
return NULL;
} else {
zend_object* ret = Z_OBJ_P(val);
zval_ptr_dtor(val);
return (Descriptor*)ret;
}
}
char *classname =
GetPhpClassname(upb_msgdef_file(m), upb_msgdef_fullname(m));
zend_string *str = zend_string_init(classname, strlen(classname), 0);
zend_class_entry *ce = zend_lookup_class(str); // May autoload the class.
// C Functions from def.h //////////////////////////////////////////////////////
zend_string_release (str);
// These are documented in the header file.
if (!ce) {
zend_error(E_ERROR, "Couldn't load generated class %s", classname);
}
Descriptor* Descriptor_GetFromClassEntry(zend_class_entry *ce) {
zval desc;
Descriptor_FromClassEntry(&desc, ce);
return Descriptor_GetFromZval(&desc);
}
free(classname);
return Descriptor_GetFromClassEntry(ce);
} else {
return NULL;
}
Descriptor* Descriptor_GetFromMessageDef(const upb_msgdef *m) {
zval desc;
Descriptor_FromMessageDef(&desc, m);
return Descriptor_GetFromZval(&desc);
}
Descriptor* Descriptor_GetFromFieldDef(const upb_fielddef *f) {
@ -588,7 +599,7 @@ Descriptor* Descriptor_GetFromFieldDef(const upb_fielddef *f) {
* the public and private descriptor.
*/
PHP_METHOD(Descriptor, getPublicDescriptor) {
RETURN_ZVAL(getThis(), 1, 0);
RETURN_COPY(getThis());
}
/*
@ -623,15 +634,8 @@ PHP_METHOD(Descriptor, getField) {
return;
}
upb_msg_field_iter iter;
int i;
for(upb_msg_field_begin(&iter, intern->msgdef), i = 0;
!upb_msg_field_done(&iter) && i < index;
upb_msg_field_next(&iter), i++);
const upb_fielddef *field = upb_msg_iter_field(&iter);
FieldDescriptor_FromFieldDef(&ret, field);
RETURN_ZVAL(&ret, 1, 0);
FieldDescriptor_FromFieldDef(&ret, upb_msgdef_field(intern->msgdef, index));
RETURN_COPY_VALUE(&ret);
}
/*
@ -674,7 +678,7 @@ PHP_METHOD(Descriptor, getOneofDecl) {
const upb_oneofdef *oneof = upb_msg_iter_oneof(&iter);
OneofDescriptor_FromOneofDef(&ret, oneof);
RETURN_ZVAL(&ret, 1, 0);
RETURN_COPY_VALUE(&ret);
}
/*
@ -702,9 +706,9 @@ PHP_METHOD(Descriptor, getClass) {
static zend_function_entry Descriptor_methods[] = {
PHP_ME(Descriptor, getClass, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(Descriptor, getFullName, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(Descriptor, getField, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(Descriptor, getField, arginfo_getByIndex, ZEND_ACC_PUBLIC)
PHP_ME(Descriptor, getFieldCount, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(Descriptor, getOneofDecl, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(Descriptor, getOneofDecl, arginfo_getByIndex, ZEND_ACC_PUBLIC)
PHP_ME(Descriptor, getOneofDeclCount, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(Descriptor, getPublicDescriptor, arginfo_void, ZEND_ACC_PUBLIC)
ZEND_FE_END
@ -781,7 +785,7 @@ upb_symtab *DescriptorPool_GetSymbolTable() {
PHP_METHOD(DescriptorPool, getGeneratedPool) {
zval ret;
ZVAL_COPY(&ret, get_generated_pool());
RETURN_ZVAL(&ret, 0, 1);
RETURN_COPY_VALUE(&ret);
}
/*
@ -810,7 +814,7 @@ PHP_METHOD(DescriptorPool, getDescriptorByClassName) {
}
Descriptor_FromClassEntry(&ret, ce);
RETURN_ZVAL(&ret, 1, 0);
RETURN_COPY_VALUE(&ret);
}
/*
@ -839,7 +843,7 @@ PHP_METHOD(DescriptorPool, getEnumDescriptorByClassName) {
}
EnumDescriptor_FromClassEntry(&ret, ce);
RETURN_ZVAL(&ret, 1, 0);
RETURN_COPY_VALUE(&ret);
}
/*
@ -863,9 +867,7 @@ PHP_METHOD(DescriptorPool, getDescriptorByProtoName) {
m = upb_symtab_lookupmsg(intern->symtab, protoname);
if (m) {
zval ret;
ZVAL_OBJ(&ret, &Descriptor_GetFromMessageDef(m)->std);
RETURN_ZVAL(&ret, 1, 0);
RETURN_OBJ_COPY(&Descriptor_GetFromMessageDef(m)->std);
} else {
RETURN_NULL();
}
@ -1003,6 +1005,10 @@ PHP_METHOD(DescriptorPool, internalAddGeneratedFile) {
upb_arena_free(arena);
}
ZEND_BEGIN_ARG_INFO_EX(arginfo_lookupByName, 0, 0, 1)
ZEND_ARG_INFO(0, name)
ZEND_END_ARG_INFO()
ZEND_BEGIN_ARG_INFO_EX(arginfo_addgeneratedfile, 0, 0, 2)
ZEND_ARG_INFO(0, data)
ZEND_ARG_INFO(0, data_len)
@ -1011,9 +1017,9 @@ ZEND_END_ARG_INFO()
static zend_function_entry DescriptorPool_methods[] = {
PHP_ME(DescriptorPool, getGeneratedPool, arginfo_void,
ZEND_ACC_PUBLIC|ZEND_ACC_STATIC)
PHP_ME(DescriptorPool, getDescriptorByClassName, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(DescriptorPool, getDescriptorByProtoName, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(DescriptorPool, getEnumDescriptorByClassName, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(DescriptorPool, getDescriptorByClassName, arginfo_lookupByName, ZEND_ACC_PUBLIC)
PHP_ME(DescriptorPool, getDescriptorByProtoName, arginfo_lookupByName, ZEND_ACC_PUBLIC)
PHP_ME(DescriptorPool, getEnumDescriptorByClassName, arginfo_lookupByName, ZEND_ACC_PUBLIC)
PHP_ME(DescriptorPool, internalAddGeneratedFile, arginfo_addgeneratedfile, ZEND_ACC_PUBLIC)
ZEND_FE_END
};
@ -1036,9 +1042,7 @@ zend_class_entry *InternalDescriptorPool_class_entry;
* instance.
*/
PHP_METHOD(InternalDescriptorPool, getGeneratedPool) {
zval ret;
ZVAL_COPY(&ret, get_generated_pool());
RETURN_ZVAL(&ret, 0, 1);
RETURN_COPY(get_generated_pool());
}
static zend_function_entry InternalDescriptorPool_methods[] = {
@ -1072,6 +1076,7 @@ void Def_ModuleInit() {
OneofDescriptor_class_entry->create_object = CreateHandler_ReturnNull;
h = &OneofDescriptor_object_handlers;
memcpy(h, &std_object_handlers, sizeof(zend_object_handlers));
h->dtor_obj = &OneofDescriptor_destructor;
INIT_CLASS_ENTRY(tmp_ce, "Google\\Protobuf\\EnumValueDescriptor",
EnumValueDescriptor_methods);
@ -1081,7 +1086,6 @@ void Def_ModuleInit() {
h = &EnumValueDescriptor_object_handlers;
memcpy(h, &std_object_handlers, sizeof(zend_object_handlers));
INIT_CLASS_ENTRY(tmp_ce, "Google\\Protobuf\\EnumDescriptor",
EnumDescriptor_methods);
EnumDescriptor_class_entry = zend_register_internal_class(&tmp_ce);
@ -1089,6 +1093,7 @@ void Def_ModuleInit() {
EnumDescriptor_class_entry->create_object = CreateHandler_ReturnNull;
h = &EnumDescriptor_object_handlers;
memcpy(h, &std_object_handlers, sizeof(zend_object_handlers));
h->dtor_obj = &EnumDescriptor_destructor;
INIT_CLASS_ENTRY(tmp_ce, "Google\\Protobuf\\Descriptor",
Descriptor_methods);
@ -1107,6 +1112,7 @@ void Def_ModuleInit() {
FieldDescriptor_class_entry->create_object = CreateHandler_ReturnNull;
h = &FieldDescriptor_object_handlers;
memcpy(h, &std_object_handlers, sizeof(zend_object_handlers));
h->dtor_obj = &FieldDescriptor_destructor;
INIT_CLASS_ENTRY(tmp_ce, "Google\\Protobuf\\DescriptorPool",
DescriptorPool_methods);

@ -61,13 +61,10 @@ typedef struct Descriptor {
zend_class_entry *class_entry;
} Descriptor;
// Gets or creates a PHP Descriptor object for a |ce| and stores it in |val|.
// If this is not a protobuf generated class, |val| will be set to null.
void Descriptor_FromClassEntry(zval *val, zend_class_entry *ce);
// Gets or creates a Descriptor* for the given class entry, upb_msgdef, or
// upb_fielddef. The returned Descriptor* will live for the entire request,
// so no ref is necessary to keep it alive.
// so no ref is necessary to keep it alive. The caller does *not* own a ref
// on the returned object.
Descriptor* Descriptor_GetFromClassEntry(zend_class_entry *ce);
Descriptor* Descriptor_GetFromMessageDef(const upb_msgdef *m);
Descriptor* Descriptor_GetFromFieldDef(const upb_fielddef *f);

@ -357,7 +357,7 @@ PHP_METHOD(MapField, offsetGet) {
}
Convert_UpbToPhp(upb_val, &ret, intern->type.val_type, &intern->arena);
RETURN_ZVAL(&ret, 0, 1);
RETURN_COPY_VALUE(&ret);
}
/**
@ -444,7 +444,7 @@ PHP_METHOD(MapField, count) {
PHP_METHOD(MapField, getIterator) {
zval ret;
MapFieldIter_make(&ret, getThis());
RETURN_ZVAL(&ret, 0, 1);
RETURN_COPY_VALUE(&ret);
}
ZEND_BEGIN_ARG_INFO_EX(arginfo_construct, 0, 0, 2)
@ -569,7 +569,7 @@ PHP_METHOD(MapFieldIter, current) {
upb_msgval upb_val = upb_mapiter_value(field->map, intern->position);
zval ret;
Convert_UpbToPhp(upb_val, &ret, field->type.val_type, &field->arena);
RETURN_ZVAL(&ret, 0, 1);
RETURN_COPY_VALUE(&ret);
}
/**
@ -583,7 +583,7 @@ PHP_METHOD(MapFieldIter, key) {
upb_msgval upb_key = upb_mapiter_key(field->map, intern->position);
zval ret;
Convert_UpbToPhp(upb_key, &ret, KeyType(field->type), NULL);
RETURN_ZVAL(&ret, 0, 1);
RETURN_COPY_VALUE(&ret);
}
/**

@ -198,8 +198,6 @@ static bool MessageEq(const upb_msg *m1, const upb_msg *m2, const upb_msgdef *m)
!upb_msg_field_done(&i);
upb_msg_field_next(&i)) {
const upb_fielddef *f = upb_msg_iter_field(&i);
upb_msgval val1 = upb_msg_get(m1, f);
upb_msgval val2 = upb_msg_get(m2, f);
if (upb_fielddef_haspresence(f)) {
if (upb_msg_has(m1, f) != upb_msg_has(m2, f)) {
@ -208,6 +206,9 @@ static bool MessageEq(const upb_msg *m1, const upb_msg *m2, const upb_msgdef *m)
if (!upb_msg_has(m1, f)) continue;
}
upb_msgval val1 = upb_msg_get(m1, f);
upb_msgval val2 = upb_msg_get(m2, f);
if (upb_fielddef_ismap(f)) {
if (!MapEq(val1.map_val, val2.map_val, MapType_Get(f))) return false;
} else if (upb_fielddef_isseq(f)) {
@ -268,7 +269,7 @@ static int Message_has_property(PROTO_VAL *obj, PROTO_STR *member,
zend_throw_exception_ex(
NULL, 0,
"Cannot call isset() on field %s which does not have presence.",
ZSTR_VAL(intern->desc->class_entry->name));
upb_fielddef_name(f));
return 0;
}
@ -303,7 +304,7 @@ static void Message_unset_property(PROTO_VAL *obj, PROTO_STR *member,
zend_throw_exception_ex(
NULL, 0,
"Cannot call unset() on field %s which does not have presence.",
ZSTR_VAL(intern->desc->class_entry->name));
upb_fielddef_name(f));
return;
}
@ -596,7 +597,6 @@ PHP_METHOD(Message, __construct) {
return;
}
Message_Initialize(intern, desc);
if (zend_parse_parameters(ZEND_NUM_ARGS(), "|a!", &init_arr) == FAILURE) {
@ -847,7 +847,7 @@ PHP_METHOD(Message, readWrapperValue) {
upb_msgval msgval = upb_msg_get(wrapper, val_f);
zval ret;
Convert_UpbToPhp(msgval, &ret, TypeInfo_Get(val_f), &intern->arena);
RETURN_ZVAL(&ret, 1, 0);
RETURN_COPY_VALUE(&ret);
} else {
RETURN_NULL();
}
@ -1014,7 +1014,7 @@ PHP_METHOD(Message, readOneof) {
Convert_UpbToPhp(msgval, &ret, TypeInfo_Get(f), &intern->arena);
}
RETURN_ZVAL(&ret, 1, 0);
RETURN_COPY_VALUE(&ret);
}
/**
@ -1059,10 +1059,19 @@ PHP_METHOD(Message, writeOneof) {
upb_msg_set(intern->msg, f, msgval, arena);
}
ZEND_BEGIN_ARG_INFO_EX(arginfo_construct, 0, 0, 0)
ZEND_ARG_INFO(0, data)
ZEND_END_ARG_INFO()
ZEND_BEGIN_ARG_INFO_EX(arginfo_mergeFrom, 0, 0, 1)
ZEND_ARG_INFO(0, data)
ZEND_END_ARG_INFO()
ZEND_BEGIN_ARG_INFO_EX(arginfo_mergeFromWithArg, 0, 0, 1)
ZEND_ARG_INFO(0, data)
ZEND_ARG_INFO(0, arg)
ZEND_END_ARG_INFO()
ZEND_BEGIN_ARG_INFO_EX(arginfo_read, 0, 0, 1)
ZEND_ARG_INFO(0, field)
ZEND_END_ARG_INFO()
@ -1078,7 +1087,7 @@ static zend_function_entry Message_methods[] = {
PHP_ME(Message, serializeToString, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(Message, mergeFromString, arginfo_mergeFrom, ZEND_ACC_PUBLIC)
PHP_ME(Message, serializeToJsonString, arginfo_void, ZEND_ACC_PUBLIC)
PHP_ME(Message, mergeFromJsonString, arginfo_mergeFrom, ZEND_ACC_PUBLIC)
PHP_ME(Message, mergeFromJsonString, arginfo_mergeFromWithArg, ZEND_ACC_PUBLIC)
PHP_ME(Message, mergeFrom, arginfo_mergeFrom, ZEND_ACC_PUBLIC)
PHP_ME(Message, readWrapperValue, arginfo_read, ZEND_ACC_PROTECTED)
PHP_ME(Message, writeWrapperValue, arginfo_write, ZEND_ACC_PROTECTED)
@ -1086,7 +1095,7 @@ static zend_function_entry Message_methods[] = {
PHP_ME(Message, readOneof, arginfo_read, ZEND_ACC_PROTECTED)
PHP_ME(Message, writeOneof, arginfo_write, ZEND_ACC_PROTECTED)
PHP_ME(Message, whichOneof, arginfo_read, ZEND_ACC_PROTECTED)
PHP_ME(Message, __construct, arginfo_void, ZEND_ACC_PROTECTED)
PHP_ME(Message, __construct, arginfo_construct, ZEND_ACC_PROTECTED)
ZEND_FE_END
};
@ -1165,13 +1174,14 @@ PHP_METHOD(google_protobuf_Any, unpack) {
if (!upb_decode(value.data, value.size, msg->msg,
upb_msgdef_layout(desc->msgdef), Arena_Get(&msg->arena))) {
zend_throw_exception_ex(NULL, 0, "Error occurred during parsing");
zval_dtor(&ret);
return;
}
// Fuse since the parsed message could alias "value".
upb_arena_fuse(Arena_Get(&intern->arena), Arena_Get(&msg->arena));
RETURN_ZVAL(&ret, 1, 0);
RETURN_COPY_VALUE(&ret);
}
PHP_METHOD(google_protobuf_Any, pack) {
@ -1238,6 +1248,7 @@ PHP_METHOD(google_protobuf_Timestamp, fromDateTime) {
const char *classname = "\\DatetimeInterface";
zend_string *classname_str = zend_string_init(classname, strlen(classname), 0);
zend_class_entry *date_interface_ce = zend_lookup_class(classname_str);
zend_string_release(classname_str);
if (date_interface_ce == NULL) {
zend_error(E_ERROR, "Make sure date extension is enabled.");

File diff suppressed because it is too large Load Diff

@ -1,26 +1,53 @@
/* Amalgamated source file */
#include <stdint.h>/*
* This is where we define macros used across upb.
*
* All of these macros are undef'd in port_undef.inc to avoid leaking them to
* users.
*
* The correct usage is:
*
* #include "upb/foobar.h"
* #include "upb/baz.h"
*
* // MUST be last included header.
* #include "upb/port_def.inc"
*
* // Code for this file.
* // <...>
*
* // Can be omitted for .c files, required for .h.
* #include "upb/port_undef.inc"
*
* This file is private and must not be included by users!
*/
/*
* Copyright (c) 2009-2021, Google LLC
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Google LLC nor the
* names of its contributors may be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
* WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
* DISCLAIMED. IN NO EVENT SHALL Google LLC BE LIABLE FOR ANY
* DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
* ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
* This is where we define macros used across upb.
*
* All of these macros are undef'd in port_undef.inc to avoid leaking them to
* users.
*
* The correct usage is:
*
* #include "upb/foobar.h"
* #include "upb/baz.h"
*
* // MUST be last included header.
* #include "upb/port_def.inc"
*
* // Code for this file.
* // <...>
*
* // Can be omitted for .c files, required for .h.
* #include "upb/port_undef.inc"
*
* This file is private and must not be included by users!
*/
#if !((defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L) || \
(defined(__cplusplus) && __cplusplus >= 201103L) || \
@ -136,9 +163,40 @@
#define UPB_LONGJMP(buf, val) longjmp(buf, val)
#endif
/* UPB_PTRADD(ptr, ofs): add pointer while avoiding "NULL + 0" UB */
#define UPB_PTRADD(ptr, ofs) ((ofs) ? (ptr) + (ofs) : (ptr))
/* Configure whether fasttable is switched on or not. *************************/
#if defined(__x86_64__) && defined(__GNUC__)
#if defined(__has_attribute)
#define UPB_HAS_ATTRIBUTE(x) __has_attribute(x)
#else
#define UPB_HAS_ATTRIBUTE(x) 0
#endif
#if UPB_HAS_ATTRIBUTE(musttail)
#define UPB_MUSTTAIL __attribute__((musttail))
#else
#define UPB_MUSTTAIL
#endif
#undef UPB_HAS_ATTRIBUTE
/* This check is not fully robust: it does not require that we have "musttail"
* support available. We need tail calls to avoid consuming arbitrary amounts
* of stack space.
*
* GCC/Clang can mostly be trusted to generate tail calls as long as
* optimization is enabled, but, debug builds will not generate tail calls
* unless "musttail" is available.
*
* We should probably either:
* 1. require that the compiler supports musttail.
* 2. add some fallback code for when musttail isn't available (ie. return
* instead of tail calling). This is safe and portable, but this comes at
* a CPU cost.
*/
#if (defined(__x86_64__) || defined(__aarch64__)) && defined(__GNUC__)
#define UPB_FASTTABLE_SUPPORTED 1
#else
#define UPB_FASTTABLE_SUPPORTED 0
@ -149,7 +207,7 @@
* for example for testing or benchmarking. */
#if defined(UPB_ENABLE_FASTTABLE)
#if !UPB_FASTTABLE_SUPPORTED
#error fasttable is x86-64 + Clang/GCC only
#error fasttable is x86-64/ARM64 only and requires GCC or Clang.
#endif
#define UPB_FASTTABLE 1
/* Define UPB_TRY_ENABLE_FASTTABLE to use fasttable if possible.
@ -194,54 +252,35 @@ void __asan_unpoison_memory_region(void const volatile *addr, size_t size);
#define UPB_UNPOISON_MEMORY_REGION(addr, size) \
((void)(addr), (void)(size))
#endif
/** upb/decode.h ************************************************************/
/*
** upb_decode: parsing into a upb_msg using a upb_msglayout.
*/
* upb_decode: parsing into a upb_msg using a upb_msglayout.
*/
#ifndef UPB_DECODE_H_
#define UPB_DECODE_H_
/** upb/msg.h ************************************************************/
/*
** Our memory representation for parsing tables and messages themselves.
** Functions in this file are used by generated code and possibly reflection.
**
** The definitions in this file are internal to upb.
**/
* Public APIs for message operations that do not require descriptors.
* These functions can be used even in build that does not want to depend on
* reflection or descriptors.
*
* Descriptor-based reflection functionality lives in reflection.h.
*/
#ifndef UPB_MSG_H_
#define UPB_MSG_H_
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
/*
** upb_table
**
** This header is INTERNAL-ONLY! Its interfaces are not public or stable!
** This file defines very fast int->upb_value (inttable) and string->upb_value
** (strtable) hash tables.
**
** The table uses chained scatter with Brent's variation (inspired by the Lua
** implementation of hash tables). The hash function for strings is Austin
** Appleby's "MurmurHash."
**
** The inttable uses uintptr_t as its key, which guarantees it can be used to
** store pointers or integers of at least 32 bits (upb isn't really useful on
** systems where sizeof(void*) < 4).
**
** The table must be homogeneous (all values of the same type). In debug
** mode, we check this on insert and lookup.
*/
#include <stddef.h>
#ifndef UPB_TABLE_H_
#define UPB_TABLE_H_
#include <stdint.h>
#include <string.h>
/** upb/upb.h ************************************************************/
/*
** This file contains shared definitions that are widely used across upb.
*/
* This file contains shared definitions that are widely used across upb.
*/
#ifndef UPB_H_
#define UPB_H_
@ -399,7 +438,7 @@ typedef struct {
upb_arena *upb_arena_init(void *mem, size_t n, upb_alloc *alloc);
void upb_arena_free(upb_arena *a);
bool upb_arena_addcleanup(upb_arena *a, void *ud, upb_cleanup_func *func);
void upb_arena_fuse(upb_arena *a, upb_arena *b);
bool upb_arena_fuse(upb_arena *a, upb_arena *b);
void *_upb_arena_slowmalloc(upb_arena *a, size_t size);
UPB_INLINE upb_alloc *upb_arena_alloc(upb_arena *a) { return (upb_alloc*)a; }
@ -578,55 +617,134 @@ UPB_INLINE int _upb_lg2ceilsize(int x) {
#endif /* UPB_H_ */
#ifdef __cplusplus
extern "C" {
#endif
typedef void upb_msg;
/* For users these are opaque. They can be obtained from upb_msgdef_layout()
* but users cannot access any of the members. */
struct upb_msglayout;
typedef struct upb_msglayout upb_msglayout;
/* Adds unknown data (serialized protobuf data) to the given message. The data
* is copied into the message instance. */
void upb_msg_addunknown(upb_msg *msg, const char *data, size_t len,
upb_arena *arena);
/* Returns a reference to the message's unknown data. */
const char *upb_msg_getunknown(const upb_msg *msg, size_t *len);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* UPB_MSG_INT_H_ */
/* Must be last. */
#ifdef __cplusplus
extern "C" {
#endif
enum {
/* If set, strings will alias the input buffer instead of copying into the
* arena. */
UPB_DECODE_ALIAS = 1,
};
/* upb_value ******************************************************************/
#define UPB_DECODE_MAXDEPTH(depth) ((depth) << 16)
/* A tagged union (stored untagged inside the table) so that we can check that
* clients calling table accessors are correctly typed without having to have
* an explosion of accessors. */
typedef enum {
UPB_CTYPE_INT32 = 1,
UPB_CTYPE_INT64 = 2,
UPB_CTYPE_UINT32 = 3,
UPB_CTYPE_UINT64 = 4,
UPB_CTYPE_BOOL = 5,
UPB_CTYPE_CSTR = 6,
UPB_CTYPE_PTR = 7,
UPB_CTYPE_CONSTPTR = 8,
UPB_CTYPE_FPTR = 9,
UPB_CTYPE_FLOAT = 10,
UPB_CTYPE_DOUBLE = 11
} upb_ctype_t;
bool _upb_decode(const char *buf, size_t size, upb_msg *msg,
const upb_msglayout *l, upb_arena *arena, int options);
UPB_INLINE
bool upb_decode(const char *buf, size_t size, upb_msg *msg,
const upb_msglayout *l, upb_arena *arena) {
return _upb_decode(buf, size, msg, l, arena, 0);
}
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* UPB_DECODE_H_ */
/** upb/decode_internal.h ************************************************************/
/*
* Internal implementation details of the decoder that are shared between
* decode.c and decode_fast.c.
*/
#ifndef UPB_DECODE_INT_H_
#define UPB_DECODE_INT_H_
#include <setjmp.h>
/** upb/msg_internal.h ************************************************************//*
** Our memory representation for parsing tables and messages themselves.
** Functions in this file are used by generated code and possibly reflection.
**
** The definitions in this file are internal to upb.
**/
#ifndef UPB_MSG_INT_H_
#define UPB_MSG_INT_H_
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
/** upb/table_internal.h ************************************************************/
/*
* upb_table
*
* This header is INTERNAL-ONLY! Its interfaces are not public or stable!
* This file defines very fast int->upb_value (inttable) and string->upb_value
* (strtable) hash tables.
*
* The table uses chained scatter with Brent's variation (inspired by the Lua
* implementation of hash tables). The hash function for strings is Austin
* Appleby's "MurmurHash."
*
* The inttable uses uintptr_t as its key, which guarantees it can be used to
* store pointers or integers of at least 32 bits (upb isn't really useful on
* systems where sizeof(void*) < 4).
*
* The table must be homogeneous (all values of the same type). In debug
* mode, we check this on insert and lookup.
*/
#ifndef UPB_TABLE_H_
#define UPB_TABLE_H_
#include <stdint.h>
#include <string.h>
#ifdef __cplusplus
extern "C" {
#endif
/* upb_value ******************************************************************/
typedef struct {
uint64_t val;
} upb_value;
/* Like strdup(), which isn't always available since it's not ANSI C. */
char *upb_strdup(const char *s, upb_alloc *a);
/* Variant that works with a length-delimited rather than NULL-delimited string,
* as supported by strtable. */
char *upb_strdup2(const char *s, size_t len, upb_alloc *a);
UPB_INLINE char *upb_gstrdup(const char *s) {
return upb_strdup(s, &upb_alloc_global);
}
char *upb_strdup2(const char *s, size_t len, upb_arena *a);
UPB_INLINE void _upb_value_setval(upb_value *v, uint64_t val) {
v->val = val;
}
UPB_INLINE upb_value _upb_value_val(uint64_t val) {
upb_value ret;
_upb_value_setval(&ret, val);
return ret;
}
/* For each value ctype, define the following set of functions:
*
* // Get/set an int32 from a upb_value.
@ -734,14 +852,7 @@ typedef struct {
uint32_t mask; /* Mask to turn hash value -> bucket. */
uint32_t max_count; /* Max count before we hit our load limit. */
uint8_t size_lg2; /* Size of the hashtable part is 2^size_lg2 entries. */
/* Hash table entries.
* Making this const isn't entirely accurate; what we really want is for it to
* have the same const-ness as the table it's inside. But there's no way to
* declare that in C. So we have to make it const so that we can statically
* initialize const hash tables. Then we cast away const when we have to.
*/
const upb_tabent *entries;
upb_tabent *entries;
} upb_table;
typedef struct {
@ -755,8 +866,6 @@ typedef struct {
size_t array_count; /* Array part number of elements. */
} upb_inttable;
#define UPB_ARRAY_EMPTYENT -1
UPB_INLINE size_t upb_table_size(const upb_table *t) {
if (t->size_lg2 == 0)
return 0;
@ -769,48 +878,10 @@ UPB_INLINE bool upb_tabent_isempty(const upb_tabent *e) {
return e->key == 0;
}
/* Used by some of the unit tests for generic hashing functionality. */
uint32_t upb_murmur_hash2(const void * key, size_t len, uint32_t seed);
UPB_INLINE uintptr_t upb_intkey(uintptr_t key) {
return key;
}
UPB_INLINE uint32_t upb_inthash(uintptr_t key) {
return (uint32_t)key;
}
static const upb_tabent *upb_getentry(const upb_table *t, uint32_t hash) {
return t->entries + (hash & t->mask);
}
UPB_INLINE bool upb_arrhas(upb_tabval key) {
return key.val != (uint64_t)-1;
}
/* Initialize and uninitialize a table, respectively. If memory allocation
* failed, false is returned that the table is uninitialized. */
bool upb_inttable_init2(upb_inttable *table, upb_ctype_t ctype, upb_alloc *a);
bool upb_strtable_init2(upb_strtable *table, upb_ctype_t ctype,
size_t expected_size, upb_alloc *a);
void upb_inttable_uninit2(upb_inttable *table, upb_alloc *a);
void upb_strtable_uninit2(upb_strtable *table, upb_alloc *a);
UPB_INLINE bool upb_inttable_init(upb_inttable *table, upb_ctype_t ctype) {
return upb_inttable_init2(table, ctype, &upb_alloc_global);
}
UPB_INLINE bool upb_strtable_init(upb_strtable *table, upb_ctype_t ctype) {
return upb_strtable_init2(table, ctype, 4, &upb_alloc_global);
}
UPB_INLINE void upb_inttable_uninit(upb_inttable *table) {
upb_inttable_uninit2(table, &upb_alloc_global);
}
UPB_INLINE void upb_strtable_uninit(upb_strtable *table) {
upb_strtable_uninit2(table, &upb_alloc_global);
}
bool upb_inttable_init(upb_inttable *table, upb_arena *a);
bool upb_strtable_init(upb_strtable *table, size_t expected_size, upb_arena *a);
/* Returns the number of values in the table. */
size_t upb_inttable_count(const upb_inttable *t);
@ -818,12 +889,6 @@ UPB_INLINE size_t upb_strtable_count(const upb_strtable *t) {
return t->t.count;
}
void upb_inttable_packedsize(const upb_inttable *t, size_t *size);
void upb_strtable_packedsize(const upb_strtable *t, size_t *size);
upb_inttable *upb_inttable_pack(const upb_inttable *t, void *p, size_t *ofs,
size_t size);
upb_strtable *upb_strtable_pack(const upb_strtable *t, void *p, size_t *ofs,
size_t size);
void upb_strtable_clear(upb_strtable *t);
/* Inserts the given key into the hashtable with the given value. The key must
@ -833,26 +898,10 @@ void upb_strtable_clear(upb_strtable *t);
*
* If a table resize was required but memory allocation failed, false is
* returned and the table is unchanged. */
bool upb_inttable_insert2(upb_inttable *t, uintptr_t key, upb_value val,
upb_alloc *a);
bool upb_strtable_insert3(upb_strtable *t, const char *key, size_t len,
upb_value val, upb_alloc *a);
UPB_INLINE bool upb_inttable_insert(upb_inttable *t, uintptr_t key,
upb_value val) {
return upb_inttable_insert2(t, key, val, &upb_alloc_global);
}
UPB_INLINE bool upb_strtable_insert2(upb_strtable *t, const char *key,
size_t len, upb_value val) {
return upb_strtable_insert3(t, key, len, val, &upb_alloc_global);
}
/* For NULL-terminated strings. */
UPB_INLINE bool upb_strtable_insert(upb_strtable *t, const char *key,
upb_value val) {
return upb_strtable_insert2(t, key, strlen(key), val);
}
bool upb_inttable_insert(upb_inttable *t, uintptr_t key, upb_value val,
upb_arena *a);
bool upb_strtable_insert(upb_strtable *t, const char *key, size_t len,
upb_value val, upb_arena *a);
/* Looks up key in this table, returning "true" if the key was found.
* If v is non-NULL, copies the value for this key into *v. */
@ -869,74 +918,21 @@ UPB_INLINE bool upb_strtable_lookup(const upb_strtable *t, const char *key,
/* Removes an item from the table. Returns true if the remove was successful,
* and stores the removed item in *val if non-NULL. */
bool upb_inttable_remove(upb_inttable *t, uintptr_t key, upb_value *val);
bool upb_strtable_remove3(upb_strtable *t, const char *key, size_t len,
upb_value *val, upb_alloc *alloc);
UPB_INLINE bool upb_strtable_remove2(upb_strtable *t, const char *key,
size_t len, upb_value *val) {
return upb_strtable_remove3(t, key, len, val, &upb_alloc_global);
}
/* For NULL-terminated strings. */
UPB_INLINE bool upb_strtable_remove(upb_strtable *t, const char *key,
upb_value *v) {
return upb_strtable_remove2(t, key, strlen(key), v);
}
bool upb_strtable_remove(upb_strtable *t, const char *key, size_t len,
upb_value *val);
/* Updates an existing entry in an inttable. If the entry does not exist,
* returns false and does nothing. Unlike insert/remove, this does not
* invalidate iterators. */
bool upb_inttable_replace(upb_inttable *t, uintptr_t key, upb_value val);
/* Convenience routines for inttables with pointer keys. */
bool upb_inttable_insertptr2(upb_inttable *t, const void *key, upb_value val,
upb_alloc *a);
bool upb_inttable_removeptr(upb_inttable *t, const void *key, upb_value *val);
bool upb_inttable_lookupptr(
const upb_inttable *t, const void *key, upb_value *val);
UPB_INLINE bool upb_inttable_insertptr(upb_inttable *t, const void *key,
upb_value val) {
return upb_inttable_insertptr2(t, key, val, &upb_alloc_global);
}
/* Optimizes the table for the current set of entries, for both memory use and
* lookup time. Client should call this after all entries have been inserted;
* inserting more entries is legal, but will likely require a table resize. */
void upb_inttable_compact2(upb_inttable *t, upb_alloc *a);
UPB_INLINE void upb_inttable_compact(upb_inttable *t) {
upb_inttable_compact2(t, &upb_alloc_global);
}
/* A special-case inlinable version of the lookup routine for 32-bit
* integers. */
UPB_INLINE bool upb_inttable_lookup32(const upb_inttable *t, uint32_t key,
upb_value *v) {
*v = upb_value_int32(0); /* Silence compiler warnings. */
if (key < t->array_size) {
upb_tabval arrval = t->array[key];
if (upb_arrhas(arrval)) {
_upb_value_setval(v, arrval.val);
return true;
} else {
return false;
}
} else {
const upb_tabent *e;
if (t->t.entries == NULL) return false;
for (e = upb_getentry(&t->t, upb_inthash(key)); true; e = e->next) {
if ((uint32_t)e->key == key) {
_upb_value_setval(v, e->val.val);
return true;
}
if (e->next == NULL) return false;
}
}
}
void upb_inttable_compact(upb_inttable *t, upb_arena *a);
/* Exposed for testing only. */
bool upb_strtable_resize(upb_strtable *t, size_t size_lg2, upb_alloc *a);
bool upb_strtable_resize(upb_strtable *t, size_t size_lg2, upb_arena *a);
/* Iterators ******************************************************************/
@ -1032,10 +1028,6 @@ bool upb_inttable_iter_isequal(const upb_inttable_iter *i1,
extern "C" {
#endif
#define PTR_AT(msg, ofs, type) (type*)((const char*)msg + ofs)
typedef void upb_msg;
/** upb_msglayout *************************************************************/
/* upb_msglayout represents the memory layout of a given upb_msgdef. The
@ -1070,7 +1062,7 @@ typedef struct {
_upb_field_parser *field_parser;
} _upb_fasttable_entry;
typedef struct upb_msglayout {
struct upb_msglayout {
const struct upb_msglayout *const* submsgs;
const upb_msglayout_field *fields;
/* Must be aligned to sizeof(void*). Doesn't include internal members like
@ -1082,7 +1074,7 @@ typedef struct upb_msglayout {
/* To constant-initialize the tables of variable length, we need a flexible
* array member, and we need to compile in C99 mode. */
_upb_fasttable_entry fasttable[];
} upb_msglayout;
};
/** upb_msg *******************************************************************/
@ -1137,21 +1129,18 @@ void _upb_msg_discardunknown_shallow(upb_msg *msg);
bool _upb_msg_addunknown(upb_msg *msg, const char *data, size_t len,
upb_arena *arena);
/* Returns a reference to the message's unknown data. */
const char *upb_msg_getunknown(const upb_msg *msg, size_t *len);
/** Hasbit access *************************************************************/
UPB_INLINE bool _upb_hasbit(const upb_msg *msg, size_t idx) {
return (*PTR_AT(msg, idx / 8, const char) & (1 << (idx % 8))) != 0;
return (*UPB_PTR_AT(msg, idx / 8, const char) & (1 << (idx % 8))) != 0;
}
UPB_INLINE void _upb_sethas(const upb_msg *msg, size_t idx) {
(*PTR_AT(msg, idx / 8, char)) |= (char)(1 << (idx % 8));
(*UPB_PTR_AT(msg, idx / 8, char)) |= (char)(1 << (idx % 8));
}
UPB_INLINE void _upb_clearhas(const upb_msg *msg, size_t idx) {
(*PTR_AT(msg, idx / 8, char)) &= (char)(~(1 << (idx % 8)));
(*UPB_PTR_AT(msg, idx / 8, char)) &= (char)(~(1 << (idx % 8)));
}
UPB_INLINE size_t _upb_msg_hasidx(const upb_msglayout_field *f) {
@ -1177,11 +1166,11 @@ UPB_INLINE void _upb_clearhas_field(const upb_msg *msg,
/** Oneof case access *********************************************************/
UPB_INLINE uint32_t *_upb_oneofcase(upb_msg *msg, size_t case_ofs) {
return PTR_AT(msg, case_ofs, uint32_t);
return UPB_PTR_AT(msg, case_ofs, uint32_t);
}
UPB_INLINE uint32_t _upb_getoneofcase(const void *msg, size_t case_ofs) {
return *PTR_AT(msg, case_ofs, uint32_t);
return *UPB_PTR_AT(msg, case_ofs, uint32_t);
}
UPB_INLINE size_t _upb_oneofcase_ofs(const upb_msglayout_field *f) {
@ -1200,7 +1189,7 @@ UPB_INLINE uint32_t _upb_getoneofcase_field(const upb_msg *msg,
}
UPB_INLINE bool _upb_has_submsg_nohasbit(const upb_msg *msg, size_t ofs) {
return *PTR_AT(msg, ofs, const upb_msg*) != NULL;
return *UPB_PTR_AT(msg, ofs, const upb_msg*) != NULL;
}
UPB_INLINE bool _upb_isrepeated(const upb_msglayout_field *field) {
@ -1277,7 +1266,7 @@ UPB_INLINE bool _upb_array_resize(upb_array *arr, size_t size,
UPB_INLINE const void *_upb_array_accessor(const void *msg, size_t ofs,
size_t *size) {
const upb_array *arr = *PTR_AT(msg, ofs, const upb_array*);
const upb_array *arr = *UPB_PTR_AT(msg, ofs, const upb_array*);
if (arr) {
if (size) *size = arr->len;
return _upb_array_constptr(arr);
@ -1289,7 +1278,7 @@ UPB_INLINE const void *_upb_array_accessor(const void *msg, size_t ofs,
UPB_INLINE void *_upb_array_mutable_accessor(void *msg, size_t ofs,
size_t *size) {
upb_array *arr = *PTR_AT(msg, ofs, upb_array*);
upb_array *arr = *UPB_PTR_AT(msg, ofs, upb_array*);
if (arr) {
if (size) *size = arr->len;
return _upb_array_ptr(arr);
@ -1302,7 +1291,7 @@ UPB_INLINE void *_upb_array_mutable_accessor(void *msg, size_t ofs,
UPB_INLINE void *_upb_array_resize_accessor2(void *msg, size_t ofs, size_t size,
int elem_size_lg2,
upb_arena *arena) {
upb_array **arr_ptr = PTR_AT(msg, ofs, upb_array *);
upb_array **arr_ptr = UPB_PTR_AT(msg, ofs, upb_array *);
upb_array *arr = *arr_ptr;
if (!arr || arr->size < size) {
return _upb_array_resize_fallback(arr_ptr, size, elem_size_lg2, arena);
@ -1315,7 +1304,7 @@ UPB_INLINE bool _upb_array_append_accessor2(void *msg, size_t ofs,
int elem_size_lg2,
const void *value,
upb_arena *arena) {
upb_array **arr_ptr = PTR_AT(msg, ofs, upb_array *);
upb_array **arr_ptr = UPB_PTR_AT(msg, ofs, upb_array *);
size_t elem_size = 1 << elem_size_lg2;
upb_array *arr = *arr_ptr;
void *ptr;
@ -1323,7 +1312,7 @@ UPB_INLINE bool _upb_array_append_accessor2(void *msg, size_t ofs,
return _upb_array_append_fallback(arr_ptr, value, elem_size_lg2, arena);
}
ptr = _upb_array_ptr(arr);
memcpy(PTR_AT(ptr, arr->len * elem_size, char), value, elem_size);
memcpy(UPB_PTR_AT(ptr, arr->len * elem_size, char), value, elem_size);
arr->len++;
return true;
}
@ -1470,20 +1459,19 @@ UPB_INLINE void* _upb_map_next(const upb_map *map, size_t *iter) {
}
UPB_INLINE bool _upb_map_set(upb_map *map, const void *key, size_t key_size,
void *val, size_t val_size, upb_arena *arena) {
void *val, size_t val_size, upb_arena *a) {
upb_strview strkey = _upb_map_tokey(key, key_size);
upb_value tabval = {0};
if (!_upb_map_tovalue(val, val_size, &tabval, arena)) return false;
upb_alloc *a = upb_arena_alloc(arena);
if (!_upb_map_tovalue(val, val_size, &tabval, a)) return false;
/* TODO(haberman): add overwrite operation to minimize number of lookups. */
upb_strtable_remove3(&map->table, strkey.data, strkey.size, NULL, a);
return upb_strtable_insert3(&map->table, strkey.data, strkey.size, tabval, a);
upb_strtable_remove(&map->table, strkey.data, strkey.size, NULL);
return upb_strtable_insert(&map->table, strkey.data, strkey.size, tabval, a);
}
UPB_INLINE bool _upb_map_delete(upb_map *map, const void *key, size_t key_size) {
upb_strview k = _upb_map_tokey(key, key_size);
return upb_strtable_remove3(&map->table, k.data, k.size, NULL, NULL);
return upb_strtable_remove(&map->table, k.data, k.size, NULL);
}
UPB_INLINE void _upb_map_clear(upb_map *map) {
@ -1515,7 +1503,7 @@ UPB_INLINE void *_upb_msg_map_next(const upb_msg *msg, size_t ofs,
UPB_INLINE bool _upb_msg_map_set(upb_msg *msg, size_t ofs, const void *key,
size_t key_size, void *val, size_t val_size,
upb_arena *arena) {
upb_map **map = PTR_AT(msg, ofs, upb_map *);
upb_map **map = UPB_PTR_AT(msg, ofs, upb_map *);
if (!*map) {
*map = _upb_map_new(arena, key_size, val_size);
}
@ -1548,8 +1536,7 @@ UPB_INLINE void _upb_msg_map_key(const void* msg, void* key, size_t size) {
UPB_INLINE void _upb_msg_map_value(const void* msg, void* val, size_t size) {
const upb_tabent *ent = (const upb_tabent*)msg;
upb_value v;
_upb_value_setval(&v, ent->val.val);
upb_value v = {ent->val.val};
_upb_map_fromvalue(v, val, size);
}
@ -1612,55 +1599,14 @@ UPB_INLINE bool _upb_sortedmap_next(_upb_mapsorter *s, const upb_map *map,
return true;
}
#undef PTR_AT
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* UPB_MSG_H_ */
/* Must be last. */
#ifdef __cplusplus
extern "C" {
#endif
enum {
/* If set, strings will alias the input buffer instead of copying into the
* arena. */
UPB_DECODE_ALIAS = 1,
};
#define UPB_DECODE_MAXDEPTH(depth) ((depth) << 16)
bool _upb_decode(const char *buf, size_t size, upb_msg *msg,
const upb_msglayout *l, upb_arena *arena, int options);
UPB_INLINE
bool upb_decode(const char *buf, size_t size, upb_msg *msg,
const upb_msglayout *l, upb_arena *arena) {
return _upb_decode(buf, size, msg, l, arena, 0);
}
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* UPB_DECODE_H_ */
/*
** Internal implementation details of the decoder that are shared between
** decode.c and decode_fast.c.
*/
#ifndef UPB_DECODE_INT_H_
#define UPB_DECODE_INT_H_
#include <setjmp.h>
#endif /* UPB_MSG_INT_H_ */
/** upb/upb_internal.h ************************************************************/
#ifndef UPB_INT_H_
#define UPB_INT_H_
@ -1670,7 +1616,10 @@ typedef struct mem_block mem_block;
struct upb_arena {
_upb_arena_head head;
uint32_t *cleanups;
/* Stores cleanup metadata for this arena.
* - a pointer to the current cleanup counter.
* - a boolean indicating if there is an unowned initial block. */
uintptr_t cleanup_metadata;
/* Allocator to allocate arena blocks. We are responsible for freeing these
* when we are destroyed. */
@ -1792,10 +1741,11 @@ bool decode_isdone(upb_decstate *d, const char **ptr) {
}
}
#if UPB_FASTTABLE
UPB_INLINE
const char *fastdecode_tagdispatch(upb_decstate *d, const char *ptr,
upb_msg *msg, intptr_t table,
uint64_t hasbits, uint32_t tag) {
uint64_t hasbits, uint64_t tag) {
const upb_msglayout *table_p = decode_totablep(table);
uint8_t mask = table;
uint64_t data;
@ -1803,8 +1753,10 @@ const char *fastdecode_tagdispatch(upb_decstate *d, const char *ptr,
UPB_ASSUME((idx & 7) == 0);
idx >>= 3;
data = table_p->fasttable[idx].field_data ^ tag;
return table_p->fasttable[idx].field_parser(d, ptr, msg, table, hasbits, data);
UPB_MUSTTAIL return table_p->fasttable[idx].field_parser(d, ptr, msg, table,
hasbits, data);
}
#endif
UPB_INLINE uint32_t fastdecode_loadtag(const char* ptr) {
uint16_t tag;
@ -1837,9 +1789,11 @@ UPB_INLINE void decode_poplimit(upb_decstate *d, const char *ptr,
#endif /* UPB_DECODE_INT_H_ */
/** upb/encode.h ************************************************************/
/*
** upb_encode: parsing into a upb_msg using a upb_msglayout.
*/
* upb_encode: parsing into a upb_msg using a upb_msglayout.
*/
#ifndef UPB_ENCODE_H_
#define UPB_ENCODE_H_
@ -1880,6 +1834,8 @@ UPB_INLINE char *upb_encode(const void *msg, const upb_msglayout *l,
#endif
#endif /* UPB_ENCODE_H_ */
/** upb/decode_fast.h ************************************************************/
// These are the specialized field parser functions for the fast parser.
// Generated tables will refer to these by name.
//
@ -2005,7 +1961,8 @@ TAGBYTES(r)
#undef UPB_PARSE_PARAMS
#endif /* UPB_DECODE_FAST_H_ */
/* This file was generated by upbc (the upb compiler) from the input
/** google/protobuf/descriptor.upb.h ************************************************************//* This file was generated by upbc (the upb compiler) from the input
* file:
*
* google/protobuf/descriptor.proto
@ -3884,18 +3841,20 @@ UPB_INLINE void google_protobuf_GeneratedCodeInfo_Annotation_set_end(google_prot
#endif /* GOOGLE_PROTOBUF_DESCRIPTOR_PROTO_UPB_H_ */
/** upb/def.h ************************************************************/
/*
** Defs are upb's internal representation of the constructs that can appear
** in a .proto file:
**
** - upb_msgdef: describes a "message" construct.
** - upb_fielddef: describes a message field.
** - upb_filedef: describes a .proto file and its defs.
** - upb_enumdef: describes an enum.
** - upb_oneofdef: describes a oneof.
**
** TODO: definitions of services.
*/
* Defs are upb's internal representation of the constructs that can appear
* in a .proto file:
*
* - upb_msgdef: describes a "message" construct.
* - upb_fielddef: describes a message field.
* - upb_filedef: describes a .proto file and its defs.
* - upb_enumdef: describes an enum.
* - upb_oneofdef: describes a oneof.
*
* TODO: definitions of services.
*/
#ifndef UPB_DEF_H_
#define UPB_DEF_H_
@ -3991,9 +3950,6 @@ const upb_msgdef *upb_fielddef_msgsubdef(const upb_fielddef *f);
const upb_enumdef *upb_fielddef_enumsubdef(const upb_fielddef *f);
const upb_msglayout_field *upb_fielddef_layout(const upb_fielddef *f);
/* Internal only. */
uint32_t upb_fielddef_selectorbase(const upb_fielddef *f);
/* upb_oneofdef ***************************************************************/
typedef upb_inttable_iter upb_oneof_iter;
@ -4078,10 +4034,6 @@ UPB_INLINE const upb_fielddef *upb_msgdef_ntofz(const upb_msgdef *m,
return upb_msgdef_ntof(m, name, strlen(name));
}
/* Internal-only. */
size_t upb_msgdef_selectorcount(const upb_msgdef *m);
uint32_t upb_msgdef_submsgfieldcount(const upb_msgdef *m);
/* Lookup of either field or oneof by name. Returns whether either was found.
* If the return is true, then the found def will be set, and the non-found
* one set to NULL. */
@ -4196,7 +4148,8 @@ bool _upb_symtab_loaddefinit(upb_symtab *s, const upb_def_init *init);
#endif /* __cplusplus */
#endif /* UPB_DEF_H_ */
/* This file was generated by upbc (the upb compiler) from the input
/** google/protobuf/descriptor.upbdefs.h ************************************************************//* This file was generated by upbc (the upb compiler) from the input
* file:
*
* google/protobuf/descriptor.proto
@ -4357,6 +4310,7 @@ UPB_INLINE const upb_msgdef *google_protobuf_GeneratedCodeInfo_Annotation_getmsg
#endif /* GOOGLE_PROTOBUF_DESCRIPTOR_PROTO_UPBDEFS_H_ */
/** upb/reflection.h ************************************************************/
#ifndef UPB_REFLECTION_H_
#define UPB_REFLECTION_H_
@ -4438,17 +4392,9 @@ bool upb_msg_next(const upb_msg *msg, const upb_msgdef *m,
const upb_symtab *ext_pool, const upb_fielddef **f,
upb_msgval *val, size_t *iter);
/* Adds unknown data (serialized protobuf data) to the given message. The data
* is copied into the message instance. */
void upb_msg_addunknown(upb_msg *msg, const char *data, size_t len,
upb_arena *arena);
/* Clears all unknown field data from this message and all submessages. */
bool upb_msg_discardunknown(upb_msg *msg, const upb_msgdef *m, int maxdepth);
/* Returns a reference to the message's unknown data. */
const char *upb_msg_getunknown(const upb_msg *msg, size_t *len);
/** upb_array *****************************************************************/
/* Creates a new array on the given arena that holds elements of this type. */
@ -4530,6 +4476,7 @@ void upb_mapiter_setvalue(upb_map *map, size_t iter, upb_msgval value);
#endif /* UPB_REFLECTION_H_ */
/** upb/json_decode.h ************************************************************/
#ifndef UPB_JSONDECODE_H_
#define UPB_JSONDECODE_H_
@ -4552,6 +4499,7 @@ bool upb_json_decode(const char *buf, size_t size, upb_msg *msg,
#endif /* UPB_JSONDECODE_H_ */
/** upb/json_encode.h ************************************************************/
#ifndef UPB_JSONENCODE_H_
#define UPB_JSONENCODE_H_
@ -4586,27 +4534,39 @@ size_t upb_json_encode(const upb_msg *msg, const upb_msgdef *m,
#endif
#endif /* UPB_JSONENCODE_H_ */
/** upb/port_undef.inc ************************************************************/
/* See port_def.inc. This should #undef all macros #defined there. */
#undef UPB_MAPTYPE_STRING
#undef UPB_SIZE
#undef UPB_PTR_AT
#undef UPB_READ_ONEOF
#undef UPB_WRITE_ONEOF
#undef UPB_MAPTYPE_STRING
#undef UPB_INLINE
#undef UPB_ALIGN_UP
#undef UPB_ALIGN_DOWN
#undef UPB_ALIGN_MALLOC
#undef UPB_ALIGN_OF
#undef UPB_LIKELY
#undef UPB_UNLIKELY
#undef UPB_FORCEINLINE
#undef UPB_NOINLINE
#undef UPB_NORETURN
#undef UPB_PRINTF
#undef UPB_MAX
#undef UPB_MIN
#undef UPB_UNUSED
#undef UPB_ASSUME
#undef UPB_ASSERT
#undef UPB_UNREACHABLE
#undef UPB_SETJMP
#undef UPB_LONGJMP
#undef UPB_PTRADD
#undef UPB_MUSTTAIL
#undef UPB_FASTTABLE_SUPPORTED
#undef UPB_FASTTABLE
#undef UPB_FASTTABLE_INIT
#undef UPB_POISON_MEMORY_REGION
#undef UPB_UNPOISON_MEMORY_REGION
#undef UPB_ASAN

@ -74,6 +74,12 @@ ZEND_BEGIN_MODULE_GLOBALS(protobuf)
// Name cache (see interface in protobuf.h).
HashTable name_msg_cache;
HashTable name_enum_cache;
// An array of descriptor objects constructed during this request. These are
// logically referenced by the corresponding class entry, but since we can't
// actually write a class entry destructor, we reference them here, to be
// destroyed on request shutdown.
HashTable descriptors;
ZEND_END_MODULE_GLOBALS(protobuf)
ZEND_DECLARE_MODULE_GLOBALS(protobuf)
@ -164,6 +170,7 @@ static PHP_RINIT_FUNCTION(protobuf) {
zend_hash_init(&PROTOBUF_G(object_cache), 64, NULL, NULL, 0);
zend_hash_init(&PROTOBUF_G(name_msg_cache), 64, NULL, NULL, 0);
zend_hash_init(&PROTOBUF_G(name_enum_cache), 64, NULL, NULL, 0);
zend_hash_init(&PROTOBUF_G(descriptors), 64, NULL, ZVAL_PTR_DTOR, 0);
return SUCCESS;
}
@ -184,6 +191,7 @@ static PHP_RSHUTDOWN_FUNCTION(protobuf) {
zend_hash_destroy(&PROTOBUF_G(object_cache));
zend_hash_destroy(&PROTOBUF_G(name_msg_cache));
zend_hash_destroy(&PROTOBUF_G(name_enum_cache));
zend_hash_destroy(&PROTOBUF_G(descriptors));
return SUCCESS;
}
@ -192,6 +200,15 @@ static PHP_RSHUTDOWN_FUNCTION(protobuf) {
// Object Cache.
// -----------------------------------------------------------------------------
void Descriptors_Add(zend_object *desc) {
// The hash table will own a ref (it will destroy it when the table is
// destroyed), but for some reason the insert operation does not add a ref, so
// we do that here with ZVAL_OBJ_COPY().
zval zv;
ZVAL_OBJ_COPY(&zv, desc);
zend_hash_next_index_insert(&PROTOBUF_G(descriptors), &zv);
}
void ObjCache_Add(const void *upb_obj, zend_object *php_obj) {
zend_ulong k = (zend_ulong)upb_obj;
zend_hash_index_add_ptr(&PROTOBUF_G(object_cache), k, php_obj);
@ -210,8 +227,7 @@ bool ObjCache_Get(const void *upb_obj, zval *val) {
zend_object *obj = zend_hash_index_find_ptr(&PROTOBUF_G(object_cache), k);
if (obj) {
GC_ADDREF(obj);
ZVAL_OBJ(val, obj);
ZVAL_OBJ_COPY(val, obj);
return true;
} else {
ZVAL_NULL(val);

@ -58,9 +58,14 @@ const zval *get_generated_pool();
#if PHP_VERSION_ID < 80000
#define PROTO_VAL zval
#define PROTO_STR zval
#define PROTO_VAL_P(obj) Z_OBJ_P(obj)
#define PROTO_VAL_P(obj) (void*)Z_OBJ_P(obj)
#define PROTO_STRVAL_P(obj) Z_STRVAL_P(obj)
#define PROTO_STRLEN_P(obj) Z_STRLEN_P(obj)
#define ZVAL_OBJ_COPY(z, o) do { ZVAL_OBJ(z, o); GC_ADDREF(o); } while (0)
#define RETVAL_OBJ_COPY(r) ZVAL_OBJ_COPY(return_value, r)
#define RETURN_OBJ_COPY(r) do { RETVAL_OBJ_COPY(r); return; } while (0)
#define RETURN_COPY(zv) do { ZVAL_COPY(return_value, zv); return; } while (0)
#define RETURN_COPY_VALUE(zv) do { ZVAL_COPY_VALUE(return_value, zv); return; } while (0)
#else
#define PROTO_VAL zend_object
#define PROTO_STR zend_string
@ -85,7 +90,7 @@ ZEND_END_ARG_INFO()
// * upb_map*, -> MapField
// * upb_msgdef* -> Descriptor
// * upb_enumdef* -> EnumDescriptor
// * zend_class_entry* -> Descriptor
// * upb_msgdef* -> Descriptor
//
// Each wrapped object should add itself to the map when it is constructed, and
// remove itself from the map when it is destroyed. This is how we ensure that
@ -105,6 +110,11 @@ void NameMap_AddEnum(const upb_enumdef *m);
const upb_msgdef *NameMap_GetMessage(zend_class_entry *ce);
const upb_enumdef *NameMap_GetEnum(zend_class_entry *ce);
// Add this descriptor object to the global list of descriptors that will be
// kept alive for the duration of the request but destroyed when the request
// is ending.
void Descriptors_Add(zend_object *desc);
// We need our own assert() because PHP takes control of NDEBUG in its headers.
#ifdef PBPHP_ENABLE_ASSERTS
#define PBPHP_ASSERT(x) \

File diff suppressed because it is too large Load Diff

@ -4,7 +4,17 @@ set -e
cd `dirname $0`
if [[ -d tmp && -z $(find tests/proto ../src/protoc -newer tmp) ]]; then
./prepare_c_extension.sh
if ../src/protoc --help > /dev/null; then
PROTOC=src/protoc
else
(cd .. && bazel build -c opt :protoc)
PROTOC=bazel-bin/protoc
fi
if [[ -d tmp && -z $(find tests/proto ../$PROTOC -newer tmp) ]]; then
# Generated protos are already present and up to date, so we can skip protoc.
#
# Protoc is very fast, but sometimes it is not available (like if we haven't
@ -16,12 +26,13 @@ fi
rm -rf tmp
mkdir -p tmp
find tests/proto -type f -name "*.proto"| xargs ../src/protoc --php_out=tmp -I../src -Itests
cd ..
find php/tests/proto -type f -name "*.proto"| xargs $PROTOC --php_out=php/tmp -Isrc -Iphp/tests
if [ "$1" = "--aggregate_metadata" ]; then
# Overwrite some of the files to use aggregation.
AGGREGATED_FILES="tests/proto/test.proto tests/proto/test_include.proto tests/proto/test_import_descriptor_proto.proto"
../src/protoc --php_out=aggregate_metadata=foo#bar:tmp -I../src -Itests $AGGREGATED_FILES
$PROTOC --php_out=aggregate_metadata=foo#bar:php/tmp -Isrc -Iphp/tests $AGGREGATED_FILES
fi
echo "Generated test protos from tests/proto -> tmp"

@ -1,6 +1,20 @@
cd $(dirname $0)
if [[ -f ext/google/protobuf/third_party/wyhash/wyhash.h && -z $(find ../third_party/wyhash -newer ext/google/protobuf/third_party) ]]; then
# Generated protos are already present and up to date, so we can skip protoc.
#
# Protoc is very fast, but sometimes it is not available (like if we haven't
# built it in Docker). Skipping it helps us proceed in this case.
echo "wyhash is up to date, skipping."
exit 0
fi
# wyhash has to live in the base third_party directory.
# We copy it into the ext/google/protobuf directory for the build
# (and for the release to PECL).
mkdir -p ../ext/google/protobuf/third_party/wyhash
cp ../../third_party/wyhash/* ../ext/google/protobuf/third_party/wyhash
rm -rf ext/google/protobuf/third_party
mkdir -p ext/google/protobuf/third_party/wyhash
cp ../third_party/wyhash/* ext/google/protobuf/third_party/wyhash
echo "Copied wyhash from ../third_party -> ext/google/protobuf/third_party"

@ -577,6 +577,14 @@ class ArrayTest extends TestBase
public function testCycleLeak()
{
if (getenv("USE_ZEND_ALLOC") === "0") {
// If we are disabling Zend's internal allocator (as we do for
// Valgrind tests, for example) then memory_get_usage() will not
// return a useful value.
$this->markTestSkipped();
return;
}
gc_collect_cycles();
$arr = new RepeatedField(GPBType::MESSAGE, TestMessage::class);
$arr[] = new TestMessage;

@ -57,6 +57,7 @@ class EncodeDecodeTest extends TestBase
{
$m = new TestMessage();
$m->mergeFromJsonString("{\"unknown\":1}", true);
$this->assertEquals("{}", $m->serializeToJsonString());
}
public function testDecodeTopLevelBoolValue()

@ -1755,9 +1755,18 @@ class GeneratedClassTest extends TestBase
#########################################################
public function testUserDefinedClass() {
# This is not allowed, but at least we shouldn't crash.
$this->expectException(Exception::class);
$p = new C();
if (getenv("USE_ZEND_ALLOC") === "0") {
// We're running a memory test. This test appears to leak in a way
// we cannot control, PHP bug?
//
// TODO: investigate further.
$this->markTestSkipped();
return;
}
# This is not allowed, but at least we shouldn't crash.
$this->expectException(Exception::class);
new C();
}
#########################################################
@ -1768,9 +1777,16 @@ class GeneratedClassTest extends TestBase
{
throw new Exception('Intended');
}
public function testNoSegfaultWithError()
{
if (getenv("USE_ZEND_ALLOC") === "0") {
// We're running a memory test. This test appears to leak in a way
// we cannot control, PHP bug?
//
// TODO: investigate further.
$this->markTestSkipped();
return;
}
$this->expectException(Exception::class);
new TestMessage(['optional_int32' => $this->throwIntendedException()]);

@ -10,16 +10,19 @@ pushd ../ext/google/protobuf > /dev/null
CONFIGURE_OPTIONS=("./configure" "--with-php-config=$(which php-config)")
if [ "$1" != "--release" ]; then
CONFIGURE_OPTIONS+=("CFLAGS=-g -O0 -Wall")
CONFIGURE_OPTIONS+=("CFLAGS=-g -O0 -Wall -DPBPHP_ENABLE_ASSERTS")
fi
FINGERPRINT="$(sha256sum $(which php)) ${CONFIGURE_OPTIONS[@]}"
# If the PHP interpreter we are building against or the arguments
# have changed, we must regenerated the Makefile.
if [[ ! -f Makefile ]] || [[ "$(grep ' \$ ./configure' config.log)" != " $ ${CONFIGURE_OPTIONS[@]}" ]]; then
if [[ ! -f BUILD_STAMP ]] || [[ "$(cat BUILD_STAMP)" != "$FINGERPRINT" ]]; then
phpize --clean
rm -f configure.in configure.ac
phpize
"${CONFIGURE_OPTIONS[@]}"
echo "$FINGERPRINT" > BUILD_STAMP
fi
make

@ -18,6 +18,18 @@ well_known_protos = %w[
google/protobuf/wrappers.proto
]
test_protos = %w[
tests/basic_test.proto
tests/basic_test_proto2.proto
tests/generated_code.proto
tests/generated_code_proto2.proto
tests/multi_level_nesting_test.proto
tests/test_import.proto
tests/test_import_proto2.proto
tests/test_ruby_package.proto
tests/test_ruby_package_proto2.proto
]
# These are omitted for now because we don't support proto2.
proto2_protos = %w[
google/protobuf/descriptor.proto
@ -43,6 +55,14 @@ unless ENV['IN_DOCKER'] == 'true'
sh "#{protoc_command} -I../src --ruby_out=lib #{input_file}"
end
end
test_protos.each do |proto_file|
output_file = proto_file.sub(/\.proto$/, "_pb.rb")
genproto_output << output_file
file output_file => proto_file do |file_task|
sh "#{protoc_command} -I../src -I. --ruby_out=. #{proto_file}"
end
end
end
if RUBY_PLATFORM == "java"
@ -100,59 +120,6 @@ else
end
end
# Proto for tests.
genproto_output << "tests/generated_code.rb"
genproto_output << "tests/generated_code_proto2.rb"
genproto_output << "tests/test_import.rb"
genproto_output << "tests/test_import_proto2.rb"
genproto_output << "tests/test_ruby_package.rb"
genproto_output << "tests/test_ruby_package_proto2.rb"
genproto_output << "tests/basic_test.rb"
genproto_output << "tests/basic_test_proto2.rb"
genproto_output << "tests/multi_level_nesting_test.rb"
genproto_output << "tests/wrappers.rb"
file "tests/generated_code.rb" => "tests/generated_code.proto" do |file_task|
sh "#{protoc_command} --ruby_out=. tests/generated_code.proto"
end
file "tests/generated_code_proto2.rb" => "tests/generated_code_proto2.proto" do |file_task|
sh "#{protoc_command} --ruby_out=. tests/generated_code_proto2.proto"
end
file "tests/test_import.rb" => "tests/test_import.proto" do |file_task|
sh "#{protoc_command} --ruby_out=. tests/test_import.proto"
end
file "tests/test_import_proto2.rb" => "tests/test_import_proto2.proto" do |file_task|
sh "#{protoc_command} --ruby_out=. tests/test_import_proto2.proto"
end
file "tests/test_ruby_package.rb" => "tests/test_ruby_package.proto" do |file_task|
sh "#{protoc_command} --ruby_out=. tests/test_ruby_package.proto"
end
file "tests/test_ruby_package_proto2.rb" => "tests/test_ruby_package_proto2.proto" do |file_task|
sh "#{protoc_command} --ruby_out=. tests/test_ruby_package_proto2.proto"
end
file "tests/basic_test.rb" => "tests/basic_test.proto" do |file_task|
sh "#{protoc_command} --experimental_allow_proto3_optional -I../src -I. --ruby_out=. tests/basic_test.proto"
end
file "tests/basic_test_proto2.rb" => "tests/basic_test_proto2.proto" do |file_task|
sh "#{protoc_command} -I../src -I. --ruby_out=. tests/basic_test_proto2.proto"
end
file "tests/multi_level_nesting_test.rb" => "tests/multi_level_nesting_test.proto" do |file_task|
sh "#{protoc_command} -I../src -I. --ruby_out=. tests/multi_level_nesting_test.proto"
end
file "tests/wrappers.rb" => "../src/google/protobuf/wrappers.proto" do |file_task|
sh "#{protoc_command} -I../src -I. --ruby_out=tests ../src/google/protobuf/wrappers.proto"
end
task :genproto => genproto_output
task :clean do
@ -162,7 +129,7 @@ end
Gem::PackageTask.new(spec) do |pkg|
end
Rake::TestTask.new(:test => :build) do |t|
Rake::TestTask.new(:test => [:build, :genproto]) do |t|
t.test_files = FileList["tests/*.rb"].exclude("tests/gc_test.rb", "tests/common_tests.rb")
end
@ -172,7 +139,7 @@ Rake::TestTask.new(:gc_test => :build) do |t|
t.test_files = FileList["tests/gc_test.rb"]
end
task :build => [:clean, :compile, :genproto]
task :build => [:clean, :genproto, :compile]
task :default => [:build]
# vim:sw=2:et

@ -794,6 +794,14 @@ static VALUE Message_CreateHash(const upb_msg *msg, const upb_msgdef *m) {
VALUE msg_value;
VALUE msg_key;
if (!is_proto2 && upb_fielddef_issubmsg(field) &&
!upb_fielddef_isseq(field) && !upb_msg_has(msg, field)) {
// TODO: Legacy behavior, remove when we fix the is_proto2 differences.
msg_key = ID2SYM(rb_intern(upb_fielddef_name(field)));
rb_hash_aset(hash, msg_key, Qnil);
continue;
}
// Do not include fields that are not present (oneof or optional fields).
if (is_proto2 && upb_fielddef_haspresence(field) &&
!upb_msg_has(msg, field)) {

File diff suppressed because it is too large Load Diff

@ -1,26 +1,53 @@
/* Amalgamated source file */
#include <stdint.h>/*
* This is where we define macros used across upb.
*
* All of these macros are undef'd in port_undef.inc to avoid leaking them to
* users.
*
* The correct usage is:
*
* #include "upb/foobar.h"
* #include "upb/baz.h"
*
* // MUST be last included header.
* #include "upb/port_def.inc"
*
* // Code for this file.
* // <...>
*
* // Can be omitted for .c files, required for .h.
* #include "upb/port_undef.inc"
*
* This file is private and must not be included by users!
*/
/*
* Copyright (c) 2009-2021, Google LLC
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Google LLC nor the
* names of its contributors may be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
* WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
* DISCLAIMED. IN NO EVENT SHALL Google LLC BE LIABLE FOR ANY
* DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
* ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
* This is where we define macros used across upb.
*
* All of these macros are undef'd in port_undef.inc to avoid leaking them to
* users.
*
* The correct usage is:
*
* #include "upb/foobar.h"
* #include "upb/baz.h"
*
* // MUST be last included header.
* #include "upb/port_def.inc"
*
* // Code for this file.
* // <...>
*
* // Can be omitted for .c files, required for .h.
* #include "upb/port_undef.inc"
*
* This file is private and must not be included by users!
*/
#if !((defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L) || \
(defined(__cplusplus) && __cplusplus >= 201103L) || \
@ -136,9 +163,40 @@
#define UPB_LONGJMP(buf, val) longjmp(buf, val)
#endif
/* UPB_PTRADD(ptr, ofs): add pointer while avoiding "NULL + 0" UB */
#define UPB_PTRADD(ptr, ofs) ((ofs) ? (ptr) + (ofs) : (ptr))
/* Configure whether fasttable is switched on or not. *************************/
#if defined(__x86_64__) && defined(__GNUC__)
#ifdef __has_attribute
#define UPB_HAS_ATTRIBUTE(x) __has_attribute(x)
#else
#define UPB_HAS_ATTRIBUTE(x) 0
#endif
#if UPB_HAS_ATTRIBUTE(musttail)
#define UPB_MUSTTAIL __attribute__((musttail))
#else
#define UPB_MUSTTAIL
#endif
#undef UPB_HAS_ATTRIBUTE
/* This check is not fully robust: it does not require that we have "musttail"
* support available. We need tail calls to avoid consuming arbitrary amounts
* of stack space.
*
* GCC/Clang can mostly be trusted to generate tail calls as long as
* optimization is enabled, but, debug builds will not generate tail calls
* unless "musttail" is available.
*
* We should probably either:
* 1. require that the compiler supports musttail.
* 2. add some fallback code for when musttail isn't available (ie. return
* instead of tail calling). This is safe and portable, but this comes at
* a CPU cost.
*/
#if (defined(__x86_64__) || defined(__aarch64__)) && defined(__GNUC__)
#define UPB_FASTTABLE_SUPPORTED 1
#else
#define UPB_FASTTABLE_SUPPORTED 0
@ -149,7 +207,7 @@
* for example for testing or benchmarking. */
#if defined(UPB_ENABLE_FASTTABLE)
#if !UPB_FASTTABLE_SUPPORTED
#error fasttable is x86-64 + Clang/GCC only
#error fasttable is x86-64/ARM64 only and requires GCC or Clang.
#endif
#define UPB_FASTTABLE 1
/* Define UPB_TRY_ENABLE_FASTTABLE to use fasttable if possible.
@ -194,54 +252,35 @@ void __asan_unpoison_memory_region(void const volatile *addr, size_t size);
#define UPB_UNPOISON_MEMORY_REGION(addr, size) \
((void)(addr), (void)(size))
#endif
/** upb/decode.h ************************************************************/
/*
** upb_decode: parsing into a upb_msg using a upb_msglayout.
*/
* upb_decode: parsing into a upb_msg using a upb_msglayout.
*/
#ifndef UPB_DECODE_H_
#define UPB_DECODE_H_
/** upb/msg.h ************************************************************/
/*
** Our memory representation for parsing tables and messages themselves.
** Functions in this file are used by generated code and possibly reflection.
**
** The definitions in this file are internal to upb.
**/
* Public APIs for message operations that do not require descriptors.
* These functions can be used even in build that does not want to depend on
* reflection or descriptors.
*
* Descriptor-based reflection functionality lives in reflection.h.
*/
#ifndef UPB_MSG_H_
#define UPB_MSG_H_
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
/*
** upb_table
**
** This header is INTERNAL-ONLY! Its interfaces are not public or stable!
** This file defines very fast int->upb_value (inttable) and string->upb_value
** (strtable) hash tables.
**
** The table uses chained scatter with Brent's variation (inspired by the Lua
** implementation of hash tables). The hash function for strings is Austin
** Appleby's "MurmurHash."
**
** The inttable uses uintptr_t as its key, which guarantees it can be used to
** store pointers or integers of at least 32 bits (upb isn't really useful on
** systems where sizeof(void*) < 4).
**
** The table must be homogeneous (all values of the same type). In debug
** mode, we check this on insert and lookup.
*/
#include <stddef.h>
#ifndef UPB_TABLE_H_
#define UPB_TABLE_H_
#include <stdint.h>
#include <string.h>
/** upb/upb.h ************************************************************/
/*
** This file contains shared definitions that are widely used across upb.
*/
* This file contains shared definitions that are widely used across upb.
*/
#ifndef UPB_H_
#define UPB_H_
@ -399,7 +438,7 @@ typedef struct {
upb_arena *upb_arena_init(void *mem, size_t n, upb_alloc *alloc);
void upb_arena_free(upb_arena *a);
bool upb_arena_addcleanup(upb_arena *a, void *ud, upb_cleanup_func *func);
void upb_arena_fuse(upb_arena *a, upb_arena *b);
bool upb_arena_fuse(upb_arena *a, upb_arena *b);
void *_upb_arena_slowmalloc(upb_arena *a, size_t size);
UPB_INLINE upb_alloc *upb_arena_alloc(upb_arena *a) { return (upb_alloc*)a; }
@ -578,55 +617,134 @@ UPB_INLINE int _upb_lg2ceilsize(int x) {
#endif /* UPB_H_ */
#ifdef __cplusplus
extern "C" {
#endif
typedef void upb_msg;
/* For users these are opaque. They can be obtained from upb_msgdef_layout()
* but users cannot access any of the members. */
struct upb_msglayout;
typedef struct upb_msglayout upb_msglayout;
/* Adds unknown data (serialized protobuf data) to the given message. The data
* is copied into the message instance. */
void upb_msg_addunknown(upb_msg *msg, const char *data, size_t len,
upb_arena *arena);
/* Returns a reference to the message's unknown data. */
const char *upb_msg_getunknown(const upb_msg *msg, size_t *len);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* UPB_MSG_INT_H_ */
/* Must be last. */
#ifdef __cplusplus
extern "C" {
#endif
enum {
/* If set, strings will alias the input buffer instead of copying into the
* arena. */
UPB_DECODE_ALIAS = 1,
};
/* upb_value ******************************************************************/
#define UPB_DECODE_MAXDEPTH(depth) ((depth) << 16)
/* A tagged union (stored untagged inside the table) so that we can check that
* clients calling table accessors are correctly typed without having to have
* an explosion of accessors. */
typedef enum {
UPB_CTYPE_INT32 = 1,
UPB_CTYPE_INT64 = 2,
UPB_CTYPE_UINT32 = 3,
UPB_CTYPE_UINT64 = 4,
UPB_CTYPE_BOOL = 5,
UPB_CTYPE_CSTR = 6,
UPB_CTYPE_PTR = 7,
UPB_CTYPE_CONSTPTR = 8,
UPB_CTYPE_FPTR = 9,
UPB_CTYPE_FLOAT = 10,
UPB_CTYPE_DOUBLE = 11
} upb_ctype_t;
bool _upb_decode(const char *buf, size_t size, upb_msg *msg,
const upb_msglayout *l, upb_arena *arena, int options);
UPB_INLINE
bool upb_decode(const char *buf, size_t size, upb_msg *msg,
const upb_msglayout *l, upb_arena *arena) {
return _upb_decode(buf, size, msg, l, arena, 0);
}
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* UPB_DECODE_H_ */
/** upb/decode_internal.h ************************************************************/
/*
* Internal implementation details of the decoder that are shared between
* decode.c and decode_fast.c.
*/
#ifndef UPB_DECODE_INT_H_
#define UPB_DECODE_INT_H_
#include <setjmp.h>
/** upb/msg_internal.h ************************************************************//*
** Our memory representation for parsing tables and messages themselves.
** Functions in this file are used by generated code and possibly reflection.
**
** The definitions in this file are internal to upb.
**/
#ifndef UPB_MSG_INT_H_
#define UPB_MSG_INT_H_
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
/** upb/table_internal.h ************************************************************/
/*
* upb_table
*
* This header is INTERNAL-ONLY! Its interfaces are not public or stable!
* This file defines very fast int->upb_value (inttable) and string->upb_value
* (strtable) hash tables.
*
* The table uses chained scatter with Brent's variation (inspired by the Lua
* implementation of hash tables). The hash function for strings is Austin
* Appleby's "MurmurHash."
*
* The inttable uses uintptr_t as its key, which guarantees it can be used to
* store pointers or integers of at least 32 bits (upb isn't really useful on
* systems where sizeof(void*) < 4).
*
* The table must be homogeneous (all values of the same type). In debug
* mode, we check this on insert and lookup.
*/
#ifndef UPB_TABLE_H_
#define UPB_TABLE_H_
#include <stdint.h>
#include <string.h>
#ifdef __cplusplus
extern "C" {
#endif
/* upb_value ******************************************************************/
typedef struct {
uint64_t val;
} upb_value;
/* Like strdup(), which isn't always available since it's not ANSI C. */
char *upb_strdup(const char *s, upb_alloc *a);
/* Variant that works with a length-delimited rather than NULL-delimited string,
* as supported by strtable. */
char *upb_strdup2(const char *s, size_t len, upb_alloc *a);
UPB_INLINE char *upb_gstrdup(const char *s) {
return upb_strdup(s, &upb_alloc_global);
}
char *upb_strdup2(const char *s, size_t len, upb_arena *a);
UPB_INLINE void _upb_value_setval(upb_value *v, uint64_t val) {
v->val = val;
}
UPB_INLINE upb_value _upb_value_val(uint64_t val) {
upb_value ret;
_upb_value_setval(&ret, val);
return ret;
}
/* For each value ctype, define the following set of functions:
*
* // Get/set an int32 from a upb_value.
@ -734,14 +852,7 @@ typedef struct {
uint32_t mask; /* Mask to turn hash value -> bucket. */
uint32_t max_count; /* Max count before we hit our load limit. */
uint8_t size_lg2; /* Size of the hashtable part is 2^size_lg2 entries. */
/* Hash table entries.
* Making this const isn't entirely accurate; what we really want is for it to
* have the same const-ness as the table it's inside. But there's no way to
* declare that in C. So we have to make it const so that we can statically
* initialize const hash tables. Then we cast away const when we have to.
*/
const upb_tabent *entries;
upb_tabent *entries;
} upb_table;
typedef struct {
@ -755,8 +866,6 @@ typedef struct {
size_t array_count; /* Array part number of elements. */
} upb_inttable;
#define UPB_ARRAY_EMPTYENT -1
UPB_INLINE size_t upb_table_size(const upb_table *t) {
if (t->size_lg2 == 0)
return 0;
@ -769,48 +878,10 @@ UPB_INLINE bool upb_tabent_isempty(const upb_tabent *e) {
return e->key == 0;
}
/* Used by some of the unit tests for generic hashing functionality. */
uint32_t upb_murmur_hash2(const void * key, size_t len, uint32_t seed);
UPB_INLINE uintptr_t upb_intkey(uintptr_t key) {
return key;
}
UPB_INLINE uint32_t upb_inthash(uintptr_t key) {
return (uint32_t)key;
}
static const upb_tabent *upb_getentry(const upb_table *t, uint32_t hash) {
return t->entries + (hash & t->mask);
}
UPB_INLINE bool upb_arrhas(upb_tabval key) {
return key.val != (uint64_t)-1;
}
/* Initialize and uninitialize a table, respectively. If memory allocation
* failed, false is returned that the table is uninitialized. */
bool upb_inttable_init2(upb_inttable *table, upb_ctype_t ctype, upb_alloc *a);
bool upb_strtable_init2(upb_strtable *table, upb_ctype_t ctype,
size_t expected_size, upb_alloc *a);
void upb_inttable_uninit2(upb_inttable *table, upb_alloc *a);
void upb_strtable_uninit2(upb_strtable *table, upb_alloc *a);
UPB_INLINE bool upb_inttable_init(upb_inttable *table, upb_ctype_t ctype) {
return upb_inttable_init2(table, ctype, &upb_alloc_global);
}
UPB_INLINE bool upb_strtable_init(upb_strtable *table, upb_ctype_t ctype) {
return upb_strtable_init2(table, ctype, 4, &upb_alloc_global);
}
UPB_INLINE void upb_inttable_uninit(upb_inttable *table) {
upb_inttable_uninit2(table, &upb_alloc_global);
}
UPB_INLINE void upb_strtable_uninit(upb_strtable *table) {
upb_strtable_uninit2(table, &upb_alloc_global);
}
bool upb_inttable_init(upb_inttable *table, upb_arena *a);
bool upb_strtable_init(upb_strtable *table, size_t expected_size, upb_arena *a);
/* Returns the number of values in the table. */
size_t upb_inttable_count(const upb_inttable *t);
@ -818,12 +889,6 @@ UPB_INLINE size_t upb_strtable_count(const upb_strtable *t) {
return t->t.count;
}
void upb_inttable_packedsize(const upb_inttable *t, size_t *size);
void upb_strtable_packedsize(const upb_strtable *t, size_t *size);
upb_inttable *upb_inttable_pack(const upb_inttable *t, void *p, size_t *ofs,
size_t size);
upb_strtable *upb_strtable_pack(const upb_strtable *t, void *p, size_t *ofs,
size_t size);
void upb_strtable_clear(upb_strtable *t);
/* Inserts the given key into the hashtable with the given value. The key must
@ -833,26 +898,10 @@ void upb_strtable_clear(upb_strtable *t);
*
* If a table resize was required but memory allocation failed, false is
* returned and the table is unchanged. */
bool upb_inttable_insert2(upb_inttable *t, uintptr_t key, upb_value val,
upb_alloc *a);
bool upb_strtable_insert3(upb_strtable *t, const char *key, size_t len,
upb_value val, upb_alloc *a);
UPB_INLINE bool upb_inttable_insert(upb_inttable *t, uintptr_t key,
upb_value val) {
return upb_inttable_insert2(t, key, val, &upb_alloc_global);
}
UPB_INLINE bool upb_strtable_insert2(upb_strtable *t, const char *key,
size_t len, upb_value val) {
return upb_strtable_insert3(t, key, len, val, &upb_alloc_global);
}
/* For NULL-terminated strings. */
UPB_INLINE bool upb_strtable_insert(upb_strtable *t, const char *key,
upb_value val) {
return upb_strtable_insert2(t, key, strlen(key), val);
}
bool upb_inttable_insert(upb_inttable *t, uintptr_t key, upb_value val,
upb_arena *a);
bool upb_strtable_insert(upb_strtable *t, const char *key, size_t len,
upb_value val, upb_arena *a);
/* Looks up key in this table, returning "true" if the key was found.
* If v is non-NULL, copies the value for this key into *v. */
@ -869,74 +918,21 @@ UPB_INLINE bool upb_strtable_lookup(const upb_strtable *t, const char *key,
/* Removes an item from the table. Returns true if the remove was successful,
* and stores the removed item in *val if non-NULL. */
bool upb_inttable_remove(upb_inttable *t, uintptr_t key, upb_value *val);
bool upb_strtable_remove3(upb_strtable *t, const char *key, size_t len,
upb_value *val, upb_alloc *alloc);
UPB_INLINE bool upb_strtable_remove2(upb_strtable *t, const char *key,
size_t len, upb_value *val) {
return upb_strtable_remove3(t, key, len, val, &upb_alloc_global);
}
/* For NULL-terminated strings. */
UPB_INLINE bool upb_strtable_remove(upb_strtable *t, const char *key,
upb_value *v) {
return upb_strtable_remove2(t, key, strlen(key), v);
}
bool upb_strtable_remove(upb_strtable *t, const char *key, size_t len,
upb_value *val);
/* Updates an existing entry in an inttable. If the entry does not exist,
* returns false and does nothing. Unlike insert/remove, this does not
* invalidate iterators. */
bool upb_inttable_replace(upb_inttable *t, uintptr_t key, upb_value val);
/* Convenience routines for inttables with pointer keys. */
bool upb_inttable_insertptr2(upb_inttable *t, const void *key, upb_value val,
upb_alloc *a);
bool upb_inttable_removeptr(upb_inttable *t, const void *key, upb_value *val);
bool upb_inttable_lookupptr(
const upb_inttable *t, const void *key, upb_value *val);
UPB_INLINE bool upb_inttable_insertptr(upb_inttable *t, const void *key,
upb_value val) {
return upb_inttable_insertptr2(t, key, val, &upb_alloc_global);
}
/* Optimizes the table for the current set of entries, for both memory use and
* lookup time. Client should call this after all entries have been inserted;
* inserting more entries is legal, but will likely require a table resize. */
void upb_inttable_compact2(upb_inttable *t, upb_alloc *a);
UPB_INLINE void upb_inttable_compact(upb_inttable *t) {
upb_inttable_compact2(t, &upb_alloc_global);
}
/* A special-case inlinable version of the lookup routine for 32-bit
* integers. */
UPB_INLINE bool upb_inttable_lookup32(const upb_inttable *t, uint32_t key,
upb_value *v) {
*v = upb_value_int32(0); /* Silence compiler warnings. */
if (key < t->array_size) {
upb_tabval arrval = t->array[key];
if (upb_arrhas(arrval)) {
_upb_value_setval(v, arrval.val);
return true;
} else {
return false;
}
} else {
const upb_tabent *e;
if (t->t.entries == NULL) return false;
for (e = upb_getentry(&t->t, upb_inthash(key)); true; e = e->next) {
if ((uint32_t)e->key == key) {
_upb_value_setval(v, e->val.val);
return true;
}
if (e->next == NULL) return false;
}
}
}
void upb_inttable_compact(upb_inttable *t, upb_arena *a);
/* Exposed for testing only. */
bool upb_strtable_resize(upb_strtable *t, size_t size_lg2, upb_alloc *a);
bool upb_strtable_resize(upb_strtable *t, size_t size_lg2, upb_arena *a);
/* Iterators ******************************************************************/
@ -1032,10 +1028,6 @@ bool upb_inttable_iter_isequal(const upb_inttable_iter *i1,
extern "C" {
#endif
#define PTR_AT(msg, ofs, type) (type*)((const char*)msg + ofs)
typedef void upb_msg;
/** upb_msglayout *************************************************************/
/* upb_msglayout represents the memory layout of a given upb_msgdef. The
@ -1070,7 +1062,7 @@ typedef struct {
_upb_field_parser *field_parser;
} _upb_fasttable_entry;
typedef struct upb_msglayout {
struct upb_msglayout {
const struct upb_msglayout *const* submsgs;
const upb_msglayout_field *fields;
/* Must be aligned to sizeof(void*). Doesn't include internal members like
@ -1082,7 +1074,7 @@ typedef struct upb_msglayout {
/* To constant-initialize the tables of variable length, we need a flexible
* array member, and we need to compile in C99 mode. */
_upb_fasttable_entry fasttable[];
} upb_msglayout;
};
/** upb_msg *******************************************************************/
@ -1137,21 +1129,18 @@ void _upb_msg_discardunknown_shallow(upb_msg *msg);
bool _upb_msg_addunknown(upb_msg *msg, const char *data, size_t len,
upb_arena *arena);
/* Returns a reference to the message's unknown data. */
const char *upb_msg_getunknown(const upb_msg *msg, size_t *len);
/** Hasbit access *************************************************************/
UPB_INLINE bool _upb_hasbit(const upb_msg *msg, size_t idx) {
return (*PTR_AT(msg, idx / 8, const char) & (1 << (idx % 8))) != 0;
return (*UPB_PTR_AT(msg, idx / 8, const char) & (1 << (idx % 8))) != 0;
}
UPB_INLINE void _upb_sethas(const upb_msg *msg, size_t idx) {
(*PTR_AT(msg, idx / 8, char)) |= (char)(1 << (idx % 8));
(*UPB_PTR_AT(msg, idx / 8, char)) |= (char)(1 << (idx % 8));
}
UPB_INLINE void _upb_clearhas(const upb_msg *msg, size_t idx) {
(*PTR_AT(msg, idx / 8, char)) &= (char)(~(1 << (idx % 8)));
(*UPB_PTR_AT(msg, idx / 8, char)) &= (char)(~(1 << (idx % 8)));
}
UPB_INLINE size_t _upb_msg_hasidx(const upb_msglayout_field *f) {
@ -1177,11 +1166,11 @@ UPB_INLINE void _upb_clearhas_field(const upb_msg *msg,
/** Oneof case access *********************************************************/
UPB_INLINE uint32_t *_upb_oneofcase(upb_msg *msg, size_t case_ofs) {
return PTR_AT(msg, case_ofs, uint32_t);
return UPB_PTR_AT(msg, case_ofs, uint32_t);
}
UPB_INLINE uint32_t _upb_getoneofcase(const void *msg, size_t case_ofs) {
return *PTR_AT(msg, case_ofs, uint32_t);
return *UPB_PTR_AT(msg, case_ofs, uint32_t);
}
UPB_INLINE size_t _upb_oneofcase_ofs(const upb_msglayout_field *f) {
@ -1200,7 +1189,7 @@ UPB_INLINE uint32_t _upb_getoneofcase_field(const upb_msg *msg,
}
UPB_INLINE bool _upb_has_submsg_nohasbit(const upb_msg *msg, size_t ofs) {
return *PTR_AT(msg, ofs, const upb_msg*) != NULL;
return *UPB_PTR_AT(msg, ofs, const upb_msg*) != NULL;
}
UPB_INLINE bool _upb_isrepeated(const upb_msglayout_field *field) {
@ -1277,7 +1266,7 @@ UPB_INLINE bool _upb_array_resize(upb_array *arr, size_t size,
UPB_INLINE const void *_upb_array_accessor(const void *msg, size_t ofs,
size_t *size) {
const upb_array *arr = *PTR_AT(msg, ofs, const upb_array*);
const upb_array *arr = *UPB_PTR_AT(msg, ofs, const upb_array*);
if (arr) {
if (size) *size = arr->len;
return _upb_array_constptr(arr);
@ -1289,7 +1278,7 @@ UPB_INLINE const void *_upb_array_accessor(const void *msg, size_t ofs,
UPB_INLINE void *_upb_array_mutable_accessor(void *msg, size_t ofs,
size_t *size) {
upb_array *arr = *PTR_AT(msg, ofs, upb_array*);
upb_array *arr = *UPB_PTR_AT(msg, ofs, upb_array*);
if (arr) {
if (size) *size = arr->len;
return _upb_array_ptr(arr);
@ -1302,7 +1291,7 @@ UPB_INLINE void *_upb_array_mutable_accessor(void *msg, size_t ofs,
UPB_INLINE void *_upb_array_resize_accessor2(void *msg, size_t ofs, size_t size,
int elem_size_lg2,
upb_arena *arena) {
upb_array **arr_ptr = PTR_AT(msg, ofs, upb_array *);
upb_array **arr_ptr = UPB_PTR_AT(msg, ofs, upb_array *);
upb_array *arr = *arr_ptr;
if (!arr || arr->size < size) {
return _upb_array_resize_fallback(arr_ptr, size, elem_size_lg2, arena);
@ -1315,7 +1304,7 @@ UPB_INLINE bool _upb_array_append_accessor2(void *msg, size_t ofs,
int elem_size_lg2,
const void *value,
upb_arena *arena) {
upb_array **arr_ptr = PTR_AT(msg, ofs, upb_array *);
upb_array **arr_ptr = UPB_PTR_AT(msg, ofs, upb_array *);
size_t elem_size = 1 << elem_size_lg2;
upb_array *arr = *arr_ptr;
void *ptr;
@ -1323,7 +1312,7 @@ UPB_INLINE bool _upb_array_append_accessor2(void *msg, size_t ofs,
return _upb_array_append_fallback(arr_ptr, value, elem_size_lg2, arena);
}
ptr = _upb_array_ptr(arr);
memcpy(PTR_AT(ptr, arr->len * elem_size, char), value, elem_size);
memcpy(UPB_PTR_AT(ptr, arr->len * elem_size, char), value, elem_size);
arr->len++;
return true;
}
@ -1470,20 +1459,19 @@ UPB_INLINE void* _upb_map_next(const upb_map *map, size_t *iter) {
}
UPB_INLINE bool _upb_map_set(upb_map *map, const void *key, size_t key_size,
void *val, size_t val_size, upb_arena *arena) {
void *val, size_t val_size, upb_arena *a) {
upb_strview strkey = _upb_map_tokey(key, key_size);
upb_value tabval = {0};
if (!_upb_map_tovalue(val, val_size, &tabval, arena)) return false;
upb_alloc *a = upb_arena_alloc(arena);
if (!_upb_map_tovalue(val, val_size, &tabval, a)) return false;
/* TODO(haberman): add overwrite operation to minimize number of lookups. */
upb_strtable_remove3(&map->table, strkey.data, strkey.size, NULL, a);
return upb_strtable_insert3(&map->table, strkey.data, strkey.size, tabval, a);
upb_strtable_remove(&map->table, strkey.data, strkey.size, NULL);
return upb_strtable_insert(&map->table, strkey.data, strkey.size, tabval, a);
}
UPB_INLINE bool _upb_map_delete(upb_map *map, const void *key, size_t key_size) {
upb_strview k = _upb_map_tokey(key, key_size);
return upb_strtable_remove3(&map->table, k.data, k.size, NULL, NULL);
return upb_strtable_remove(&map->table, k.data, k.size, NULL);
}
UPB_INLINE void _upb_map_clear(upb_map *map) {
@ -1515,7 +1503,7 @@ UPB_INLINE void *_upb_msg_map_next(const upb_msg *msg, size_t ofs,
UPB_INLINE bool _upb_msg_map_set(upb_msg *msg, size_t ofs, const void *key,
size_t key_size, void *val, size_t val_size,
upb_arena *arena) {
upb_map **map = PTR_AT(msg, ofs, upb_map *);
upb_map **map = UPB_PTR_AT(msg, ofs, upb_map *);
if (!*map) {
*map = _upb_map_new(arena, key_size, val_size);
}
@ -1548,8 +1536,7 @@ UPB_INLINE void _upb_msg_map_key(const void* msg, void* key, size_t size) {
UPB_INLINE void _upb_msg_map_value(const void* msg, void* val, size_t size) {
const upb_tabent *ent = (const upb_tabent*)msg;
upb_value v;
_upb_value_setval(&v, ent->val.val);
upb_value v = {ent->val.val};
_upb_map_fromvalue(v, val, size);
}
@ -1612,55 +1599,14 @@ UPB_INLINE bool _upb_sortedmap_next(_upb_mapsorter *s, const upb_map *map,
return true;
}
#undef PTR_AT
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* UPB_MSG_H_ */
/* Must be last. */
#ifdef __cplusplus
extern "C" {
#endif
enum {
/* If set, strings will alias the input buffer instead of copying into the
* arena. */
UPB_DECODE_ALIAS = 1,
};
#define UPB_DECODE_MAXDEPTH(depth) ((depth) << 16)
bool _upb_decode(const char *buf, size_t size, upb_msg *msg,
const upb_msglayout *l, upb_arena *arena, int options);
UPB_INLINE
bool upb_decode(const char *buf, size_t size, upb_msg *msg,
const upb_msglayout *l, upb_arena *arena) {
return _upb_decode(buf, size, msg, l, arena, 0);
}
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* UPB_DECODE_H_ */
/*
** Internal implementation details of the decoder that are shared between
** decode.c and decode_fast.c.
*/
#ifndef UPB_DECODE_INT_H_
#define UPB_DECODE_INT_H_
#include <setjmp.h>
#endif /* UPB_MSG_INT_H_ */
/** upb/upb_internal.h ************************************************************/
#ifndef UPB_INT_H_
#define UPB_INT_H_
@ -1670,7 +1616,10 @@ typedef struct mem_block mem_block;
struct upb_arena {
_upb_arena_head head;
uint32_t *cleanups;
/* Stores cleanup metadata for this arena.
* - a pointer to the current cleanup counter.
* - a boolean indicating if there is an unowned initial block. */
uintptr_t cleanup_metadata;
/* Allocator to allocate arena blocks. We are responsible for freeing these
* when we are destroyed. */
@ -1792,10 +1741,11 @@ bool decode_isdone(upb_decstate *d, const char **ptr) {
}
}
#if UPB_FASTTABLE
UPB_INLINE
const char *fastdecode_tagdispatch(upb_decstate *d, const char *ptr,
upb_msg *msg, intptr_t table,
uint64_t hasbits, uint32_t tag) {
uint64_t hasbits, uint64_t tag) {
const upb_msglayout *table_p = decode_totablep(table);
uint8_t mask = table;
uint64_t data;
@ -1803,8 +1753,10 @@ const char *fastdecode_tagdispatch(upb_decstate *d, const char *ptr,
UPB_ASSUME((idx & 7) == 0);
idx >>= 3;
data = table_p->fasttable[idx].field_data ^ tag;
return table_p->fasttable[idx].field_parser(d, ptr, msg, table, hasbits, data);
UPB_MUSTTAIL return table_p->fasttable[idx].field_parser(d, ptr, msg, table,
hasbits, data);
}
#endif
UPB_INLINE uint32_t fastdecode_loadtag(const char* ptr) {
uint16_t tag;
@ -1837,9 +1789,11 @@ UPB_INLINE void decode_poplimit(upb_decstate *d, const char *ptr,
#endif /* UPB_DECODE_INT_H_ */
/** upb/encode.h ************************************************************/
/*
** upb_encode: parsing into a upb_msg using a upb_msglayout.
*/
* upb_encode: parsing into a upb_msg using a upb_msglayout.
*/
#ifndef UPB_ENCODE_H_
#define UPB_ENCODE_H_
@ -1880,6 +1834,8 @@ UPB_INLINE char *upb_encode(const void *msg, const upb_msglayout *l,
#endif
#endif /* UPB_ENCODE_H_ */
/** upb/decode_fast.h ************************************************************/
// These are the specialized field parser functions for the fast parser.
// Generated tables will refer to these by name.
//
@ -2005,7 +1961,8 @@ TAGBYTES(r)
#undef UPB_PARSE_PARAMS
#endif /* UPB_DECODE_FAST_H_ */
/* This file was generated by upbc (the upb compiler) from the input
/** google/protobuf/descriptor.upb.h ************************************************************//* This file was generated by upbc (the upb compiler) from the input
* file:
*
* google/protobuf/descriptor.proto
@ -3884,18 +3841,20 @@ UPB_INLINE void google_protobuf_GeneratedCodeInfo_Annotation_set_end(google_prot
#endif /* GOOGLE_PROTOBUF_DESCRIPTOR_PROTO_UPB_H_ */
/** upb/def.h ************************************************************/
/*
** Defs are upb's internal representation of the constructs that can appear
** in a .proto file:
**
** - upb_msgdef: describes a "message" construct.
** - upb_fielddef: describes a message field.
** - upb_filedef: describes a .proto file and its defs.
** - upb_enumdef: describes an enum.
** - upb_oneofdef: describes a oneof.
**
** TODO: definitions of services.
*/
* Defs are upb's internal representation of the constructs that can appear
* in a .proto file:
*
* - upb_msgdef: describes a "message" construct.
* - upb_fielddef: describes a message field.
* - upb_filedef: describes a .proto file and its defs.
* - upb_enumdef: describes an enum.
* - upb_oneofdef: describes a oneof.
*
* TODO: definitions of services.
*/
#ifndef UPB_DEF_H_
#define UPB_DEF_H_
@ -3991,9 +3950,6 @@ const upb_msgdef *upb_fielddef_msgsubdef(const upb_fielddef *f);
const upb_enumdef *upb_fielddef_enumsubdef(const upb_fielddef *f);
const upb_msglayout_field *upb_fielddef_layout(const upb_fielddef *f);
/* Internal only. */
uint32_t upb_fielddef_selectorbase(const upb_fielddef *f);
/* upb_oneofdef ***************************************************************/
typedef upb_inttable_iter upb_oneof_iter;
@ -4078,10 +4034,6 @@ UPB_INLINE const upb_fielddef *upb_msgdef_ntofz(const upb_msgdef *m,
return upb_msgdef_ntof(m, name, strlen(name));
}
/* Internal-only. */
size_t upb_msgdef_selectorcount(const upb_msgdef *m);
uint32_t upb_msgdef_submsgfieldcount(const upb_msgdef *m);
/* Lookup of either field or oneof by name. Returns whether either was found.
* If the return is true, then the found def will be set, and the non-found
* one set to NULL. */
@ -4197,6 +4149,7 @@ bool _upb_symtab_loaddefinit(upb_symtab *s, const upb_def_init *init);
#endif /* UPB_DEF_H_ */
/** upb/reflection.h ************************************************************/
#ifndef UPB_REFLECTION_H_
#define UPB_REFLECTION_H_
@ -4278,17 +4231,9 @@ bool upb_msg_next(const upb_msg *msg, const upb_msgdef *m,
const upb_symtab *ext_pool, const upb_fielddef **f,
upb_msgval *val, size_t *iter);
/* Adds unknown data (serialized protobuf data) to the given message. The data
* is copied into the message instance. */
void upb_msg_addunknown(upb_msg *msg, const char *data, size_t len,
upb_arena *arena);
/* Clears all unknown field data from this message and all submessages. */
bool upb_msg_discardunknown(upb_msg *msg, const upb_msgdef *m, int maxdepth);
/* Returns a reference to the message's unknown data. */
const char *upb_msg_getunknown(const upb_msg *msg, size_t *len);
/** upb_array *****************************************************************/
/* Creates a new array on the given arena that holds elements of this type. */
@ -4370,6 +4315,7 @@ void upb_mapiter_setvalue(upb_map *map, size_t iter, upb_msgval value);
#endif /* UPB_REFLECTION_H_ */
/** upb/json_decode.h ************************************************************/
#ifndef UPB_JSONDECODE_H_
#define UPB_JSONDECODE_H_
@ -4392,6 +4338,7 @@ bool upb_json_decode(const char *buf, size_t size, upb_msg *msg,
#endif /* UPB_JSONDECODE_H_ */
/** upb/json_encode.h ************************************************************/
#ifndef UPB_JSONENCODE_H_
#define UPB_JSONENCODE_H_
@ -4426,27 +4373,39 @@ size_t upb_json_encode(const upb_msg *msg, const upb_msgdef *m,
#endif
#endif /* UPB_JSONENCODE_H_ */
/** upb/port_undef.inc ************************************************************/
/* See port_def.inc. This should #undef all macros #defined there. */
#undef UPB_MAPTYPE_STRING
#undef UPB_SIZE
#undef UPB_PTR_AT
#undef UPB_READ_ONEOF
#undef UPB_WRITE_ONEOF
#undef UPB_MAPTYPE_STRING
#undef UPB_INLINE
#undef UPB_ALIGN_UP
#undef UPB_ALIGN_DOWN
#undef UPB_ALIGN_MALLOC
#undef UPB_ALIGN_OF
#undef UPB_LIKELY
#undef UPB_UNLIKELY
#undef UPB_FORCEINLINE
#undef UPB_NOINLINE
#undef UPB_NORETURN
#undef UPB_PRINTF
#undef UPB_MAX
#undef UPB_MIN
#undef UPB_UNUSED
#undef UPB_ASSUME
#undef UPB_ASSERT
#undef UPB_UNREACHABLE
#undef UPB_SETJMP
#undef UPB_LONGJMP
#undef UPB_PTRADD
#undef UPB_MUSTTAIL
#undef UPB_FASTTABLE_SUPPORTED
#undef UPB_FASTTABLE
#undef UPB_FASTTABLE_INIT
#undef UPB_POISON_MEMORY_REGION
#undef UPB_UNPOISON_MEMORY_REGION
#undef UPB_ASAN

@ -1894,8 +1894,8 @@ void GenerateCEnum(const EnumDescriptor* desc, io::Printer* printer) {
"}\n"
"\n"
"static zend_function_entry $c_name$_phpmethods[] = {\n"
" PHP_ME($c_name$, name, arginfo_void, ZEND_ACC_PUBLIC | ZEND_ACC_STATIC)\n"
" PHP_ME($c_name$, value, arginfo_void, ZEND_ACC_PUBLIC | ZEND_ACC_STATIC)\n"
" PHP_ME($c_name$, name, arginfo_lookup, ZEND_ACC_PUBLIC | ZEND_ACC_STATIC)\n"
" PHP_ME($c_name$, value, arginfo_lookup, ZEND_ACC_PUBLIC | ZEND_ACC_STATIC)\n"
" ZEND_FE_END\n"
"};\n"
"\n"
@ -1953,7 +1953,7 @@ void GenerateCMessage(const Descriptor* message, io::Printer* printer) {
" \"$name$\");\n"
" zval ret;\n"
" Message_get(intern, f, &ret);\n"
" RETURN_ZVAL(&ret, 1, 0);\n"
" RETURN_COPY_VALUE(&ret);\n"
"}\n"
"\n"
"static PHP_METHOD($c_name$, set$camel_name$) {\n"
@ -1966,7 +1966,7 @@ void GenerateCMessage(const Descriptor* message, io::Printer* printer) {
" return;\n"
" }\n"
" Message_set(intern, f, val);\n"
" RETURN_ZVAL(getThis(), 1, 0);\n"
" RETURN_COPY(getThis());\n"
"}\n"
"\n",
"c_name", c_name,
@ -2012,7 +2012,7 @@ void GenerateCMessage(const Descriptor* message, io::Printer* printer) {
printer->Print(
"static zend_function_entry $c_name$_phpmethods[] = {\n"
" PHP_ME($c_name$, __construct, arginfo_void, ZEND_ACC_PUBLIC)\n",
" PHP_ME($c_name$, __construct, arginfo_construct, ZEND_ACC_PUBLIC)\n",
"c_name", c_name);
for (int i = 0; i < message->field_count(); i++) {
@ -2111,7 +2111,14 @@ void GenerateCWellKnownTypes(const std::vector<const FileDescriptor*>& files,
printer.Print(
"// This file is generated from the .proto files for the well-known\n"
"// types. Do not edit!\n");
"// types. Do not edit!\n\n");
printer.Print(
"ZEND_BEGIN_ARG_INFO_EX(arginfo_lookup, 0, 0, 1)\n"
" ZEND_ARG_INFO(0, key)\n"
"ZEND_END_ARG_INFO()\n"
"\n"
);
for (auto file : files) {
printer.Print(

Loading…
Cancel
Save