This makes third_party/utf8_range no longer a Git subtree, but instead the
permanent location and source of truth for utf8_range. It is also now
incorporated into the @com_google_protobuf Bazel repo. Utf8_range still has its
own separate CMake build for now, though.
PiperOrigin-RevId: 580682733
There's a test run in test_python.yml that is non-trivial to get working with
Python 3.12 due to some refactoring of our Docker images that would be needed.
But this change updates everything else to add coverage for Python 3.12.
The main changes necessary to get the builds working were to upgrade some Pip
packages via requirements.txt, including in a patch to `rules_fuzzing` that I
plan to upstream soon. I also had to take an explicit dependency on
`setuptools`.
I removed tox.ini, since it was outdated and we have not been actively
maintaining it.
PiperOrigin-RevId: 580548224
This change moves almost everything in the `upb/` directory up one level, so
that for example `upb/upb/generated_code_support.h` becomes just
`upb/generated_code_support.h`. The only exceptions I made to this were that I
left `upb/cmake` and `upb/BUILD` where they are, mostly because that avoids
conflict with other files and the current locations seem reasonable for now.
The `python/` directory is a little bit of a challenge because we had to merge
the existing directory there with `upb/python/`. I made `upb/python/BUILD` into
the BUILD file for the merged directory, and it effectively loads the contents
of the other BUILD file via `python/build_targets.bzl`, but I plan to clean
this up soon.
PiperOrigin-RevId: 568651768
This restores the Python wheel CI runs from the old upb repo with only minor
changes. I had to update a path in one of the `py_wheel` rules and also make a
slight tweak to ensure that the `descriptor.upb_minitable.{h,c}` files make it
into the source wheels. The change in text_format_test.py is not strictly
necessary but is a small simplification I made while I was trying to debug an
issue with CRLF newlines.
I had to update test_util.py to use `importlib` to access the golden files from
the installed `protobuftests` package. I suspect the previous incarnation of
thse test runs was somehow reading the goldens from the repo checkout, but I
think the intention is to read them from `protobuftests` instead. This was a
bit tricky to get working because Python versions before 3.9 do not support
`importlib.resources.files()`. I set up the code to fall back on
`importlib.resources.open_binary()` in that case, but that function does not
support subdirectories, so this required putting an `__init__.py` file inside
the `testdata` directory to make sure it is treated as a Python package.
PiperOrigin-RevId: 567366695
I am getting ready to move almost everything under the upb/ directory up one
level to integrate upb better into its new location in the protobuf repo. This
change makes a few tweaks to prepare for that:
- Delete upb's LICENSE and CONTRIBUTING.md files since we already have similar
files at the top level.
- Rename `//python:python_version` so that it won't conflict later with
`//upb/python:python_version`.
- Move the contents of python/BUILD.bazel out to a Bazel macro to facilitate
merging that BUILD.bazel file with upb/python/BUILD.
PiperOrigin-RevId: 567119840
A couple weeks ago we moved upb into the protobuf Git repo, and this change
continues the merger of the two repos by making them into a single Bazel repo.
This was mostly a matter of deleting upb's WORKSPACE file and fixing up a bunch
of references to reflect the new structure.
Most of the changes are pretty mechanical, but one thing that needed more
invasive changes was the Python script for generating CMakeLists.txt,
make_cmakelists.py. The WORKSPACE file it relied on no longer exists with this
change, so I updated it to hardcode the information it needed from that file.
PiperOrigin-RevId: 564810016
This is the second attempt to fix our Git history. This should allow
"git blame" to work correctly in the upb/ directory even though our
automation unexpectedly blew away that directory.
The file structure was broken only when invoking from protobuf repo (`bazel build @upb//python/dist:source_tarball`) since filegroup files have a path like `protobuf/external/upb/python/dist/setup.py` rather than `protobuf/setup.py`
This was broken by 79af13abde
PiperOrigin-RevId: 553821217
This flag was added to Protobuf and submitted separately due to version skew between repos.
This attempts to fix the following error from local and release settings both matching.
```
ERROR: /workspace/_build/out/external/upb/python/dist/BUILD.bazel:251:9: Illegal ambiguous match on configurable attribute "platform" in @upb//python/dist:binary_wheel:
@upb//python/dist:linux_x86_64_local
@upb//python/dist:windows_x86_64
Multiple matches are not allowed unless one is unambiguously more specialized or they resolve to the same value. See https://bazel.build/reference/be/functions#select.
```
PiperOrigin-RevId: 553226261
The next in a series of CLs to split upb/BUILD into subdirs.
Create mem/internal/
Delete the deprecated upb/arena.h and upb/alloc.h stub headers
PiperOrigin-RevId: 552864952
pkg_files() has some odd quirks, like breaking if a filename exists in multiple directories. filegroup() does everything we need, without these quirks.
PiperOrigin-RevId: 547523447
After this change, `mini_table` only has MiniTable definitions themselves. Everything having to do with the MiniDescriptor wire format is in `mini_descriptor`.
Also rearranged some of the files in mini_table to have better structure for `internal/`.
This CL contains no functional change.
PiperOrigin-RevId: 543529112
Our pkg_tar rule was missing a few template files for python. Using setup.py will guarantee conformance with python package requirements.
Adds MANIFEST.in to include c header/inc files in the source distribution.
PiperOrigin-RevId: 506638326
A while back, the targets were all renamed from "manylinux" to "linux" in order to allow for local testing. However, we still need the release artifacts to use "manylinux" so this conditions the name on whether it is a release or local testing.
PiperOrigin-RevId: 505831116
Addresses https://github.com/protocolbuffers/protobuf/issues/10936.
This requires updating to the newest version of rules_python to use the new py_wheel API that includes a parameter for extra distinfo files
PiperOrigin-RevId: 493060514
The current behavior will crash any Bazel command immediately, due to our declared pip dependencies in WORKSPACE, if python3 can't be found. The new behavior will mock out these workspace dependencies and allow any non-python targets to run. Python targets will be skipped by wildcard expressions if there's no system python3, and will fail when run directly, due to compatibility mismatch.
PiperOrigin-RevId: 492085254
After some debugging I was able to determine that the Win32 DLL was failing to load libgcc_s_sjlj-1.dll. If we statically link libgcc instead, there should be no runtime dependency on libgcc.
PiperOrigin-RevId: 448338965
When I tried the previous `arm64` wheel on macOS, I discovered that `pip` on macOS only supports the `arm64` platform tag in a limited set of circumstances. pip seems to prefer `universal2` wheels.
To build a `universal2` wheel, we must run the `llvm-lipo` tool to bundle multiple `cc_binary()` outputs into a single output wheel. We use a transition to depend on multiple architectures for the extension, if we see that we are building for a multiarch CPU.
PiperOrigin-RevId: 447486256
While I've made some progress debugging b/231485326, I'm not going to be able to get this fully working in time for -rc1.
If I can debug this before the final release, we can re-enable the win32 wheels.
PiperOrigin-RevId: 446849335
This should get us the full set of binary wheels we need for release. Also added acceptance tests for all wheels except aarch64 (on Linux and macos): we cannot test these without emulation, which will require a somewhat more complicated setup.
PiperOrigin-RevId: 446574958
1. For some reason the version script was not working, it was failing to export the main symbol for the Python extension. I fixed this by using the `visibility` attribute instead to export the `PyInit__message` function.
2. We were not properly stripping the `python/dist/` prefix for the C module, which was making the module exported under the name `dist.google._upb` instead of `google._upb`.
3. The `py_library()` rule was failing to actually propagate the module file into the wheel, so I just removed it.
PiperOrigin-RevId: 445446611
Adam Cozzette
Eric Salo
Sandy Zhang
Joshua Haberman
Protobuf Team Bot
Mike Kruskal
Deanna Garcia
Protobuf Team