Chiebot-Mirror
/
protobuf
mirror of https://github.com/protocolbuffers/protobuf.git
8
0
Fork 0
Code Issues Projects Releases Wiki Activity
Protocol Buffers - Google's data interchange format (grpc依赖) https://developers.google.com/protocol-buffers/
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
17680 Commits
197 Branches
346 Tags
251 MiB
Tag: Branch: Tree: 6b9a81a042
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '6b9a81a042'
${ noResults }
protobuf/upb/wire/eps_copy_input_stream_test.cc

343 lines
11 KiB
Raw Normal View History Unescape

Reformat copyright headers PiperOrigin-RevId: 554509301
1 year ago
// Protocol Buffers - Google's data interchange format
// Copyright 2023 Google LLC. All rights reserved.
//
Shorten our license headers into an abbreviated form that references LICENSE instead of including it in full. PiperOrigin-RevId: 568239714
1 year ago
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file or at
// https://developers.google.com/open-source/licenses/bsd
Reformat copyright headers PiperOrigin-RevId: 554509301
1 year ago
Reorganize upb file structure This change moves almost everything in the `upb/` directory up one level, so that for example `upb/upb/generated_code_support.h` becomes just `upb/generated_code_support.h`. The only exceptions I made to this were that I left `upb/cmake` and `upb/BUILD` where they are, mostly because that avoids conflict with other files and the current locations seem reasonable for now. The `python/` directory is a little bit of a challenge because we had to merge the existing directory there with `upb/python/`. I made `upb/python/BUILD` into the BUILD file for the merged directory, and it effectively loads the contents of the other BUILD file via `python/build_targets.bzl`, but I plan to clean this up soon. PiperOrigin-RevId: 568651768
1 year ago
#include "upb/wire/eps_copy_input_stream.h"
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
#include <string.h>
#include <string>
Internal change PiperOrigin-RevId: 567664389
1 year ago
#include <gtest/gtest.h>
Reorganize upb file structure This change moves almost everything in the `upb/` directory up one level, so that for example `upb/upb/generated_code_support.h` becomes just `upb/generated_code_support.h`. The only exceptions I made to this were that I left `upb/cmake` and `upb/BUILD` where they are, mostly because that avoids conflict with other files and the current locations seem reasonable for now. The `python/` directory is a little bit of a challenge because we had to merge the existing directory there with `upb/python/`. I made `upb/python/BUILD` into the BUILD file for the merged directory, and it effectively loads the contents of the other BUILD file via `python/build_targets.bzl`, but I plan to clean this up soon. PiperOrigin-RevId: 568651768
1 year ago
#include "upb/mem/arena.hpp"
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// begin:google_only
// #include "testing/fuzzing/fuzztest.h"
// end:google_only
namespace {
TEST(EpsCopyInputStreamTest, ZeroSize) {
upb_EpsCopyInputStream stream;
upb: s/NULL/nullptr in C++ unit tests PiperOrigin-RevId: 572320485
1 year ago
const char* ptr = nullptr;
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
upb_EpsCopyInputStream_Init(&stream, &ptr, 0, false);
upb: s/NULL/nullptr in C++ unit tests PiperOrigin-RevId: 572320485
1 year ago
EXPECT_TRUE(
upb_EpsCopyInputStream_IsDoneWithCallback(&stream, &ptr, nullptr));
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
}
// begin:google_only
//
// // We create a simple, trivial implementation of the stream that we can test
// // our real implementation against.
//
// class FakeStream {
// public:
Added `explicit` to a single-arg constructor to fix a ClangTidy warning. This is the only remaining ClangTidy warning in this directory. PiperOrigin-RevId: 593867029
11 months ago
// explicit FakeStream(const std::string& data) : data_(data), offset_(0) {
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// limits_.push_back(data.size());
// }
//
// // If we reached one or more limits correctly, returns the number of limits
// // ended. If we tried to read beyond the current limit, returns -1.
// // Otherwise, for simple success, returns 0.
// int ReadData(int n, std::string* data) {
// if (n > BytesUntilLimit()) return -1;
//
// data->assign(data_.data() + offset_, n);
// offset_ += n;
//
// int end_limit_count = 0;
//
// while (BytesUntilLimit() == 0) {
// if (PopLimit()) {
// end_limit_count++;
// } else {
// eof_ = true;
// break;
// }
// }
//
// return end_limit_count;
// }
//
// bool TryPushLimit(int limit) {
// if (!CheckSize(limit)) return false;
// limits_.push_back(offset_ + limit);
// return true;
// }
//
// bool IsEof() const { return eof_; }
//
// private:
// int BytesUntilLimit() const { return limits_.back() - offset_; }
// bool CheckSize(int size) const { return BytesUntilLimit() >= size; }
//
// // Return false on EOF.
// bool PopLimit() {
// limits_.pop_back();
// return !limits_.empty();
// }
//
// std::string data_;
// // Limits, specified in absolute stream terms.
// std::vector<int> limits_;
// int offset_;
// bool eof_ = false;
// };
//
// char tmp_buf[kUpb_EpsCopyInputStream_SlopBytes];
//
// class EpsStream {
// public:
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// EpsStream(const std::string& data, bool enable_aliasing)
// : data_(data), enable_aliasing_(enable_aliasing) {
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// ptr_ = data_.data();
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// upb_EpsCopyInputStream_Init(&eps_, &ptr_, data_.size(), enable_aliasing);
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// }
//
// // Returns false at EOF or error.
// int ReadData(int n, std::string* data) {
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// EXPECT_LE(n, kUpb_EpsCopyInputStream_SlopBytes);
Migrated the text format encoder to upb_WireReader. Also fixed a few bugs and added new functionality to upb_WireReader. PiperOrigin-RevId: 498223970
2 years ago
// if (enable_aliasing_) {
// EXPECT_TRUE(upb_EpsCopyInputStream_AliasingAvailable(&eps_, ptr_, n));
// }
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// // We want to verify that we can read kUpb_EpsCopyInputStream_SlopBytes
// // safely, even if we haven't actually been requested to read that much.
// // We copy to a global buffer so the copy can't be optimized away.
// memcpy(&tmp_buf, ptr_, kUpb_EpsCopyInputStream_SlopBytes);
// data->assign(tmp_buf, n);
// ptr_ += n;
Migrated the text format encoder to upb_WireReader. Also fixed a few bugs and added new functionality to upb_WireReader. PiperOrigin-RevId: 498223970
2 years ago
// if (enable_aliasing_) {
// EXPECT_TRUE(upb_EpsCopyInputStream_AliasingAvailable(&eps_, ptr_, 0));
// }
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// return PopLimits();
// }
//
// int ReadString(int n, std::string* data) {
// if (!upb_EpsCopyInputStream_CheckSize(&eps_, ptr_, n)) return -1;
// const char* str_data = ptr_;
Migrated the text format encoder to upb_WireReader. Also fixed a few bugs and added new functionality to upb_WireReader. PiperOrigin-RevId: 498223970
2 years ago
// if (enable_aliasing_) {
// EXPECT_TRUE(upb_EpsCopyInputStream_AliasingAvailable(&eps_, ptr_, n));
// }
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// ptr_ = upb_EpsCopyInputStream_ReadString(&eps_, &str_data, n, arena_.ptr());
// if (!ptr_) return -1;
// if (enable_aliasing_ && n) {
// EXPECT_GE(reinterpret_cast<uintptr_t>(str_data),
// reinterpret_cast<uintptr_t>(data_.data()));
// EXPECT_LT(reinterpret_cast<uintptr_t>(str_data),
// reinterpret_cast<uintptr_t>(data_.data() + data_.size()));
Migrated the text format encoder to upb_WireReader. Also fixed a few bugs and added new functionality to upb_WireReader. PiperOrigin-RevId: 498223970
2 years ago
// EXPECT_TRUE(upb_EpsCopyInputStream_AliasingAvailable(&eps_, ptr_, 0));
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// }
// data->assign(str_data, n);
// return PopLimits();
// }
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
//
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// bool TryPushLimit(int limit) {
// if (!upb_EpsCopyInputStream_CheckSize(&eps_, ptr_, limit)) return false;
// deltas_.push_back(upb_EpsCopyInputStream_PushLimit(&eps_, ptr_, limit));
// return true;
// }
//
// bool IsEof() const { return eof_; }
//
// private:
// int PopLimits() {
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// int end_limit_count = 0;
//
// while (IsAtLimit()) {
// if (error_) return -1;
// if (PopLimit()) {
// end_limit_count++;
// } else {
// eof_ = true; // EOF.
// break;
// }
// }
//
// return error_ ? -1 : end_limit_count;
// }
//
// bool IsAtLimit() {
Created a new upb_WireReader interface for parsing wire data directly. The overall motivation for this interface is to consolidate many places in upb that are parsing wire format data directly. This interface is not yet complete, but this is a good start. We have enough to port the wire format parsing in accessors.c to this interface. We can follow up by porting more places that do wire format parsing. PiperOrigin-RevId: 498109788
2 years ago
// return upb_EpsCopyInputStream_IsDoneWithCallback(
// &eps_, &ptr_, &EpsStream::IsDoneFallback);
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// }
//
// // Return false on EOF.
// bool PopLimit() {
// if (deltas_.empty()) return false;
// upb_EpsCopyInputStream_PopLimit(&eps_, ptr_, deltas_.back());
// deltas_.pop_back();
// return true;
// }
//
// static const char* IsDoneFallback(upb_EpsCopyInputStream* e, const char* ptr,
// int overrun) {
// return _upb_EpsCopyInputStream_IsDoneFallbackInline(
// e, ptr, overrun, &EpsStream::BufferFlipCallback);
// }
//
// static const char* BufferFlipCallback(upb_EpsCopyInputStream* e,
// const char* old_end,
// const char* new_start) {
// EpsStream* stream = reinterpret_cast<EpsStream*>(e);
// if (!old_end) stream->error_ = true;
// return new_start;
// }
//
// upb_EpsCopyInputStream eps_;
// std::string data_;
// const char* ptr_;
// std::vector<int> deltas_;
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// upb::Arena arena_;
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// bool error_ = false;
// bool eof_ = false;
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// bool enable_aliasing_;
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// };
//
// // Reads N bytes from the given position.
// struct ReadOp {
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// int bytes; // Must be <= kUpb_EpsCopyInputStream_SlopBytes.
// };
//
// struct ReadStringOp {
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// int bytes;
// };
//
// // Pushes a new limit of N bytes from the current position.
// struct PushLimitOp {
// int bytes;
// };
//
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// typedef std::variant<ReadOp, ReadStringOp, PushLimitOp> Op;
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
//
// struct EpsCopyTestScript {
// int data_size;
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// bool enable_aliasing;
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// std::vector<Op> ops;
// };
//
// auto ArbitraryEpsCopyTestScript() {
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// using ::fuzztest::Arbitrary;
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// using ::fuzztest::InRange;
// using ::fuzztest::NonNegative;
// using ::fuzztest::StructOf;
// using ::fuzztest::VariantOf;
// using ::fuzztest::VectorOf;
//
// int max_data_size = 512;
//
// return StructOf<EpsCopyTestScript>(
// InRange(0, max_data_size), // data_size
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// Arbitrary<bool>(), // enable_aliasing
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// VectorOf(VariantOf(
// // ReadOp
// StructOf<ReadOp>(InRange(0, kUpb_EpsCopyInputStream_SlopBytes)),
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// // ReadStringOp
// StructOf<ReadStringOp>(NonNegative<int>()),
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// // PushLimitOp
// StructOf<PushLimitOp>(NonNegative<int>()))));
// }
//
// // Run a test that creates both real stream and a fake stream, and validates
// // that they have the same behavior.
// void TestAgainstFakeStream(const EpsCopyTestScript& script) {
// std::string data(script.data_size, 'x');
// for (int i = 0; i < script.data_size; ++i) {
// data[i] = static_cast<char>(i & 0xff);
// }
//
// FakeStream fake_stream(data);
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// EpsStream eps_stream(data, script.enable_aliasing);
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
//
// for (const auto& op : script.ops) {
// if (const ReadOp* read_op = std::get_if<ReadOp>(&op)) {
// std::string data_fake;
// std::string data_eps;
// int fake_result = fake_stream.ReadData(read_op->bytes, &data_fake);
// int eps_result = eps_stream.ReadData(read_op->bytes, &data_eps);
// EXPECT_EQ(fake_result, eps_result);
// if (fake_result == -1) break; // Error
// EXPECT_EQ(data_fake, data_eps);
// EXPECT_EQ(fake_stream.IsEof(), eps_stream.IsEof());
// if (fake_stream.IsEof()) break;
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// } else if (const ReadStringOp* read_op = std::get_if<ReadStringOp>(&op)) {
// std::string data_fake;
// std::string data_eps;
// int fake_result = fake_stream.ReadData(read_op->bytes, &data_fake);
// int eps_result = eps_stream.ReadString(read_op->bytes, &data_eps);
// EXPECT_EQ(fake_result, eps_result);
// if (fake_result == -1) break; // Error
// EXPECT_EQ(data_fake, data_eps);
// EXPECT_EQ(fake_stream.IsEof(), eps_stream.IsEof());
// if (fake_stream.IsEof()) break;
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// } else if (const PushLimitOp* push = std::get_if<PushLimitOp>(&op)) {
// EXPECT_EQ(fake_stream.TryPushLimit(push->bytes),
// eps_stream.TryPushLimit(push->bytes));
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// } else {
// EXPECT_TRUE(false); // Unknown op.
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// }
// }
// }
//
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// // Test with:
// // $ blaze run --config=fuzztest third_party/upb:eps_copy_input_stream_test \
// // -- --gunit_fuzz=
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// FUZZ_TEST(EpsCopyFuzzTest, TestAgainstFakeStream)
// .WithDomains(ArbitraryEpsCopyTestScript());
//
// TEST(EpsCopyFuzzTest, TestAgainstFakeStreamRegression) {
// TestAgainstFakeStream({299,
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// false,
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// {
// PushLimitOp{2},
// ReadOp{14},
// }});
// }
//
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// TEST(EpsCopyFuzzTest, AliasingEnabledZeroSizeReadString) {
// TestAgainstFakeStream({510, true, {ReadStringOp{0}}});
// }
//
// TEST(EpsCopyFuzzTest, AliasingDisabledZeroSizeReadString) {
// TestAgainstFakeStream({510, false, {ReadStringOp{0}}});
// }
//
// TEST(EpsCopyFuzzTest, ReadStringZero) {
// TestAgainstFakeStream({0, true, {ReadStringOp{0}}});
// }
//
// TEST(EpsCopyFuzzTest, ReadZero) {
// TestAgainstFakeStream({0, true, {ReadOp{0}}});
// }
//
// TEST(EpsCopyFuzzTest, ReadZeroTwice) {
// TestAgainstFakeStream({0, true, {ReadOp{0}, ReadOp{0}}});
// }
//
// TEST(EpsCopyFuzzTest, ReadStringZeroThenRead) {
// TestAgainstFakeStream({0, true, {ReadStringOp{0}, ReadOp{0}}});
// }
//
// TEST(EpsCopyFuzzTest, ReadStringOverflowsBufferButNotLimit) {
// TestAgainstFakeStream({351,
// false,
// {
// ReadOp{7},
// PushLimitOp{2147483647},
// ReadStringOp{344},
// }});
// }
//
// TEST(EpsCopyFuzzTest, LastBufferAliasing) {
// TestAgainstFakeStream({27, true, {ReadOp{12}, ReadStringOp{3}}});
// }
//
// TEST(EpsCopyFuzzTest, FirstBufferAliasing) {
// TestAgainstFakeStream({7, true, {ReadStringOp{3}}});
// }
//
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// end:google_only
} // namespace