Users often encounter a Python import error when trying to build Python
protos if protobuf is installed locally on the machine. In this case,
Python ends up looking in the wrong directory when importing files (see
bazelbuild/bazel#1209 and tensorflow/tensorflow#2021). It seems that the
problem is caused by Python getting confused when there are Python
source files that are meant to be part of the same package but are
in separate directories.
Prior to #1233, the Bazel build setup would copy the Python
runtime sources and all generated sources for the builtin protos into
the root directory (assuming that the protobuf tree is vendored in a
google/protobuf directory).
With #1233, the two sets of sources are kept in their respective
directories but both `src/` and `python/` are added to the `PYTHONPATH`
using the new `imports` attribute of the Bazel Python rules. However,
both the runtime sources and the generated sources are under the same
package: `google.protobuf`, causing Python to become confused when
trying to import modules that are in the other directory.
This patch adds a workaround to the Bazel build to add a modified
version of the original `internal_copied_filegroup` macro to copy the
`.proto` files under `src/` to `python/` before building the
`py_proto_library` targets for the builtin protos. This ensures that the
generated sources for the builtin protos will be in the same directory
as the corresponding runtime sources.
This patch was tested with the following:
* All Python tests in protobuf
* All Python tests in tensorflow
* All tests in [Skydoc](https://github.com/bazelbuild/skydoc)
* Importing protobuf as `//google/protobuf`
* Importing and binding targets under `//external`
* Importing protobuf as `//third_party/protobuf`
This change makes use of new imports attribute for Bazel's Python rules, which
enable adding directories to the PYTHONPATH. This allows us to remove
the hack for building protobuf's Python support with Bazel and now
allows projects to include protobuf using a Bazel external repository
rather than requiring it to be imported directly into the source tree as
//google/protobuf.
This change also updates the protobuf BUILD file to use a named
repository, @python//, for including Python headers rather than
//util/python. This allows projects to specify their own package for
Python headers when including protobuf with an external repository.
Fixes#1230
grpc C++ plugin generates additional files, namely .grpc.pb.cc and
.grpc.pb.h. Adding these files to the outs of the _proto_gen rule, so
dependents don't complain about undeclared inclusions. Also, compiling
the .grpc.pb.cc requires additional header files from the grpc library,
so added //external:grpc_lib to the deps of the
cc_library. Clients are expected to declare that in their bazel
WORKSPACE, pointing it to @grpc//:grpc++{_unsecure}.
cc_proto_library now supports use_grpc_plugin flag that passes
--plugin=protoc-gen-grpc=grpc_cpp_plugin to protoc compiler
invocation. grpc_cpp_plugin is assumed to be present as
//external:grpc_cpp_plugin, so clients can setup their WORKSPACE files
appropriately to point to grpc location using bind.
This change make protobuf skylark extension works when using
remote repository.
Note that this make the Skylark extension unusable prior
to Bazel 0.1.4 because the workspace_root is not available
on prior version.
Tested with Bazel 0.1.4rc2.
Fixes https://github.com/bazelbuild/bazel/issues/784.
This enables the use case where all the paths are relative to the
workspace root, e.g.
foo/bar/BUILD
/foo.proto -- package foo.bar
would generate the message correctly.
David Z. Chen
Steven Parkes
Manjunath Kudlur
Damien Martin-Guillerez
Martin Maly
Andrew Harp
Jisi Liu