Chiebot-Mirror/protobuf - protobuf - Gitea: Git with a cup of tea

Author	SHA1	Message	Date
Eric Salo	184a4758f2	fix an error path in the (still completely unused) tokenizer PiperOrigin-RevId: 513948062	2 years ago
Eric Salo	4d3998b54b	consolidate some general parsing functions into upb/lex/ There are several other functions which might eventually end up here and ideally become unified across json/ and text/ and io/ so this is just a first step to create the new subdir and get rid of upb/internal/ PiperOrigin-RevId: 488954926	2 years ago
Eric Salo	f6307877d3	move portability stuff into upb/port/ Also delete redundant system #includes that are already pulled in by port/def.inc PiperOrigin-RevId: 486398989	2 years ago
Eric Salo	f3316e2d7d	remove upb_String from the public tokenizer api upb_String is a hack which exists because the original C++ tokenizer got to assume the existence of C++ strings, so at least for now the C tokenizer needs a rough equivalent. But this should be a purely internal implementation detail, not part of the visible surface. PiperOrigin-RevId: 469814074	3 years ago
Eric Salo	0013c936ef	add upb_Status to the tokenizer PiperOrigin-RevId: 469721241	3 years ago
Eric Salo	33114209dc	simplify the tokenizer - remove previous token from the public api - remove upb_Token type PiperOrigin-RevId: 469308543	3 years ago
Eric Salo	922a858e5c	clean up tokenizer options and defaults - Disallow multiline strings. - Disallow a letter immediately following a number without intervening whitespace. - Replace distinct bool option flags with a single options int. PiperOrigin-RevId: 467829817	3 years ago
Eric Salo	6861966501	first stab at a Tokenizer api These functions are not yet part of the upb build but this is a good chunk of work so let's snapshot it now. PiperOrigin-RevId: 467733791	3 years ago