Chiebot-Mirror
/
protobuf
mirror of https://github.com/protocolbuffers/protobuf.git
8
0
Fork 0
Code Issues Projects Releases Wiki Activity
Protocol Buffers - Google's data interchange format (grpc依赖) https://developers.google.com/protocol-buffers/
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
2915 Commits
201 Branches
346 Tags
254 MiB
Tag: Branch: Tree: 07f6b578ee
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '07f6b578ee'
${ noResults }
protobuf/upb/wire/eps_copy_input_stream_test.cc

335 lines
10 KiB
Raw Normal View History Unescape

Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
#include "upb/wire/eps_copy_input_stream.h"
#include <string.h>
#include <string>
#include "gtest/gtest.h"
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
#include "upb/upb.hpp"
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// begin:google_only
// #include "testing/fuzzing/fuzztest.h"
// end:google_only
namespace {
TEST(EpsCopyInputStreamTest, ZeroSize) {
upb_EpsCopyInputStream stream;
const char* ptr = NULL;
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
upb_EpsCopyInputStream_Init(&stream, &ptr, 0, false);
Created a new upb_WireReader interface for parsing wire data directly. The overall motivation for this interface is to consolidate many places in upb that are parsing wire format data directly. This interface is not yet complete, but this is a good start. We have enough to port the wire format parsing in accessors.c to this interface. We can follow up by porting more places that do wire format parsing. PiperOrigin-RevId: 498109788
2 years ago
EXPECT_TRUE(upb_EpsCopyInputStream_IsDoneWithCallback(&stream, &ptr, NULL));
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
}
// begin:google_only
//
// // We create a simple, trivial implementation of the stream that we can test
// // our real implementation against.
//
// class FakeStream {
// public:
// FakeStream(const std::string& data) : data_(data), offset_(0) {
// limits_.push_back(data.size());
// }
//
// // If we reached one or more limits correctly, returns the number of limits
// // ended. If we tried to read beyond the current limit, returns -1.
// // Otherwise, for simple success, returns 0.
// int ReadData(int n, std::string* data) {
// if (n > BytesUntilLimit()) return -1;
//
// data->assign(data_.data() + offset_, n);
// offset_ += n;
//
// int end_limit_count = 0;
//
// while (BytesUntilLimit() == 0) {
// if (PopLimit()) {
// end_limit_count++;
// } else {
// eof_ = true;
// break;
// }
// }
//
// return end_limit_count;
// }
//
// bool TryPushLimit(int limit) {
// if (!CheckSize(limit)) return false;
// limits_.push_back(offset_ + limit);
// return true;
// }
//
// bool IsEof() const { return eof_; }
//
// private:
// int BytesUntilLimit() const { return limits_.back() - offset_; }
// bool CheckSize(int size) const { return BytesUntilLimit() >= size; }
//
// // Return false on EOF.
// bool PopLimit() {
// limits_.pop_back();
// return !limits_.empty();
// }
//
// std::string data_;
// // Limits, specified in absolute stream terms.
// std::vector<int> limits_;
// int offset_;
// bool eof_ = false;
// };
//
// char tmp_buf[kUpb_EpsCopyInputStream_SlopBytes];
//
// class EpsStream {
// public:
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// EpsStream(const std::string& data, bool enable_aliasing)
// : data_(data), enable_aliasing_(enable_aliasing) {
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// ptr_ = data_.data();
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// upb_EpsCopyInputStream_Init(&eps_, &ptr_, data_.size(), enable_aliasing);
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// }
//
// // Returns false at EOF or error.
// int ReadData(int n, std::string* data) {
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// EXPECT_LE(n, kUpb_EpsCopyInputStream_SlopBytes);
Migrated the text format encoder to upb_WireReader. Also fixed a few bugs and added new functionality to upb_WireReader. PiperOrigin-RevId: 498223970
2 years ago
// if (enable_aliasing_) {
// EXPECT_TRUE(upb_EpsCopyInputStream_AliasingAvailable(&eps_, ptr_, n));
// }
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// // We want to verify that we can read kUpb_EpsCopyInputStream_SlopBytes
// // safely, even if we haven't actually been requested to read that much.
// // We copy to a global buffer so the copy can't be optimized away.
// memcpy(&tmp_buf, ptr_, kUpb_EpsCopyInputStream_SlopBytes);
// data->assign(tmp_buf, n);
// ptr_ += n;
Migrated the text format encoder to upb_WireReader. Also fixed a few bugs and added new functionality to upb_WireReader. PiperOrigin-RevId: 498223970
2 years ago
// if (enable_aliasing_) {
// EXPECT_TRUE(upb_EpsCopyInputStream_AliasingAvailable(&eps_, ptr_, 0));
// }
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// return PopLimits();
// }
//
// int ReadString(int n, std::string* data) {
// if (!upb_EpsCopyInputStream_CheckSize(&eps_, ptr_, n)) return -1;
// const char* str_data = ptr_;
Migrated the text format encoder to upb_WireReader. Also fixed a few bugs and added new functionality to upb_WireReader. PiperOrigin-RevId: 498223970
2 years ago
// if (enable_aliasing_) {
// EXPECT_TRUE(upb_EpsCopyInputStream_AliasingAvailable(&eps_, ptr_, n));
// }
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// ptr_ = upb_EpsCopyInputStream_ReadString(&eps_, &str_data, n, arena_.ptr());
// if (!ptr_) return -1;
// if (enable_aliasing_ && n) {
// EXPECT_GE(reinterpret_cast<uintptr_t>(str_data),
// reinterpret_cast<uintptr_t>(data_.data()));
// EXPECT_LT(reinterpret_cast<uintptr_t>(str_data),
// reinterpret_cast<uintptr_t>(data_.data() + data_.size()));
Migrated the text format encoder to upb_WireReader. Also fixed a few bugs and added new functionality to upb_WireReader. PiperOrigin-RevId: 498223970
2 years ago
// EXPECT_TRUE(upb_EpsCopyInputStream_AliasingAvailable(&eps_, ptr_, 0));
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// }
// data->assign(str_data, n);
// return PopLimits();
// }
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
//
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// bool TryPushLimit(int limit) {
// if (!upb_EpsCopyInputStream_CheckSize(&eps_, ptr_, limit)) return false;
// deltas_.push_back(upb_EpsCopyInputStream_PushLimit(&eps_, ptr_, limit));
// return true;
// }
//
// bool IsEof() const { return eof_; }
//
// private:
// int PopLimits() {
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// int end_limit_count = 0;
//
// while (IsAtLimit()) {
// if (error_) return -1;
// if (PopLimit()) {
// end_limit_count++;
// } else {
// eof_ = true; // EOF.
// break;
// }
// }
//
// return error_ ? -1 : end_limit_count;
// }
//
// bool IsAtLimit() {
Created a new upb_WireReader interface for parsing wire data directly. The overall motivation for this interface is to consolidate many places in upb that are parsing wire format data directly. This interface is not yet complete, but this is a good start. We have enough to port the wire format parsing in accessors.c to this interface. We can follow up by porting more places that do wire format parsing. PiperOrigin-RevId: 498109788
2 years ago
// return upb_EpsCopyInputStream_IsDoneWithCallback(
// &eps_, &ptr_, &EpsStream::IsDoneFallback);
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// }
//
// // Return false on EOF.
// bool PopLimit() {
// if (deltas_.empty()) return false;
// upb_EpsCopyInputStream_PopLimit(&eps_, ptr_, deltas_.back());
// deltas_.pop_back();
// return true;
// }
//
// static const char* IsDoneFallback(upb_EpsCopyInputStream* e, const char* ptr,
// int overrun) {
// return _upb_EpsCopyInputStream_IsDoneFallbackInline(
// e, ptr, overrun, &EpsStream::BufferFlipCallback);
// }
//
// static const char* BufferFlipCallback(upb_EpsCopyInputStream* e,
// const char* old_end,
// const char* new_start) {
// EpsStream* stream = reinterpret_cast<EpsStream*>(e);
// if (!old_end) stream->error_ = true;
// return new_start;
// }
//
// upb_EpsCopyInputStream eps_;
// std::string data_;
// const char* ptr_;
// std::vector<int> deltas_;
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// upb::Arena arena_;
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// bool error_ = false;
// bool eof_ = false;
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// bool enable_aliasing_;
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// };
//
// // Reads N bytes from the given position.
// struct ReadOp {
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// int bytes; // Must be <= kUpb_EpsCopyInputStream_SlopBytes.
// };
//
// struct ReadStringOp {
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// int bytes;
// };
//
// // Pushes a new limit of N bytes from the current position.
// struct PushLimitOp {
// int bytes;
// };
//
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// typedef std::variant<ReadOp, ReadStringOp, PushLimitOp> Op;
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
//
// struct EpsCopyTestScript {
// int data_size;
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// bool enable_aliasing;
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// std::vector<Op> ops;
// };
//
// auto ArbitraryEpsCopyTestScript() {
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// using ::fuzztest::Arbitrary;
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// using ::fuzztest::InRange;
// using ::fuzztest::NonNegative;
// using ::fuzztest::StructOf;
// using ::fuzztest::VariantOf;
// using ::fuzztest::VectorOf;
//
// int max_data_size = 512;
//
// return StructOf<EpsCopyTestScript>(
// InRange(0, max_data_size), // data_size
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// Arbitrary<bool>(), // enable_aliasing
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// VectorOf(VariantOf(
// // ReadOp
// StructOf<ReadOp>(InRange(0, kUpb_EpsCopyInputStream_SlopBytes)),
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// // ReadStringOp
// StructOf<ReadStringOp>(NonNegative<int>()),
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// // PushLimitOp
// StructOf<PushLimitOp>(NonNegative<int>()))));
// }
//
// // Run a test that creates both real stream and a fake stream, and validates
// // that they have the same behavior.
// void TestAgainstFakeStream(const EpsCopyTestScript& script) {
// std::string data(script.data_size, 'x');
// for (int i = 0; i < script.data_size; ++i) {
// data[i] = static_cast<char>(i & 0xff);
// }
//
// FakeStream fake_stream(data);
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// EpsStream eps_stream(data, script.enable_aliasing);
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
//
// for (const auto& op : script.ops) {
// if (const ReadOp* read_op = std::get_if<ReadOp>(&op)) {
// std::string data_fake;
// std::string data_eps;
// int fake_result = fake_stream.ReadData(read_op->bytes, &data_fake);
// int eps_result = eps_stream.ReadData(read_op->bytes, &data_eps);
// EXPECT_EQ(fake_result, eps_result);
// if (fake_result == -1) break; // Error
// EXPECT_EQ(data_fake, data_eps);
// EXPECT_EQ(fake_stream.IsEof(), eps_stream.IsEof());
// if (fake_stream.IsEof()) break;
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// } else if (const ReadStringOp* read_op = std::get_if<ReadStringOp>(&op)) {
// std::string data_fake;
// std::string data_eps;
// int fake_result = fake_stream.ReadData(read_op->bytes, &data_fake);
// int eps_result = eps_stream.ReadString(read_op->bytes, &data_eps);
// EXPECT_EQ(fake_result, eps_result);
// if (fake_result == -1) break; // Error
// EXPECT_EQ(data_fake, data_eps);
// EXPECT_EQ(fake_stream.IsEof(), eps_stream.IsEof());
// if (fake_stream.IsEof()) break;
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// } else if (const PushLimitOp* push = std::get_if<PushLimitOp>(&op)) {
// EXPECT_EQ(fake_stream.TryPushLimit(push->bytes),
// eps_stream.TryPushLimit(push->bytes));
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// } else {
// EXPECT_TRUE(false); // Unknown op.
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// }
// }
// }
//
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// // Test with:
// // $ blaze run --config=fuzztest third_party/upb:eps_copy_input_stream_test \
// // -- --gunit_fuzz=
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// FUZZ_TEST(EpsCopyFuzzTest, TestAgainstFakeStream)
// .WithDomains(ArbitraryEpsCopyTestScript());
//
// TEST(EpsCopyFuzzTest, TestAgainstFakeStreamRegression) {
// TestAgainstFakeStream({299,
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// false,
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// {
// PushLimitOp{2},
// ReadOp{14},
// }});
// }
//
Moved aliasing logic for string field parsing into EpsCopyInputStream. Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality. We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream. Much of this logic is ported directly from the C++ EpsCopyInputStream class. PiperOrigin-RevId: 498091644
2 years ago
// TEST(EpsCopyFuzzTest, AliasingEnabledZeroSizeReadString) {
// TestAgainstFakeStream({510, true, {ReadStringOp{0}}});
// }
//
// TEST(EpsCopyFuzzTest, AliasingDisabledZeroSizeReadString) {
// TestAgainstFakeStream({510, false, {ReadStringOp{0}}});
// }
//
// TEST(EpsCopyFuzzTest, ReadStringZero) {
// TestAgainstFakeStream({0, true, {ReadStringOp{0}}});
// }
//
// TEST(EpsCopyFuzzTest, ReadZero) {
// TestAgainstFakeStream({0, true, {ReadOp{0}}});
// }
//
// TEST(EpsCopyFuzzTest, ReadZeroTwice) {
// TestAgainstFakeStream({0, true, {ReadOp{0}, ReadOp{0}}});
// }
//
// TEST(EpsCopyFuzzTest, ReadStringZeroThenRead) {
// TestAgainstFakeStream({0, true, {ReadStringOp{0}, ReadOp{0}}});
// }
//
// TEST(EpsCopyFuzzTest, ReadStringOverflowsBufferButNotLimit) {
// TestAgainstFakeStream({351,
// false,
// {
// ReadOp{7},
// PushLimitOp{2147483647},
// ReadStringOp{344},
// }});
// }
//
// TEST(EpsCopyFuzzTest, LastBufferAliasing) {
// TestAgainstFakeStream({27, true, {ReadOp{12}, ReadStringOp{3}}});
// }
//
// TEST(EpsCopyFuzzTest, FirstBufferAliasing) {
// TestAgainstFakeStream({7, true, {ReadStringOp{3}}});
// }
//
Separated out buffering code into upb_EpsCopyInputStream. This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class. This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream. After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream. PiperOrigin-RevId: 494400285
2 years ago
// end:google_only
} // namespace