These will still use MSVC as the compiler, but will no longer generate Visual Studio projects for the builds. Visual Studio is particularly bad at parallelizing builds, and is hostile to ccache.
This change also tweaks the ccache setup to prevent unbounded the growth observed in our github caches. Windows builds have had debug symbols stripped to reduce ccache size by a factor of 2x, and ccache maximum have been tweaked so that we persist fewer older builds.
Before this change, each CMake build took 12 minutes on every run (plus some constant overhead from staleness/gcloud), even with caching or on large multi-core runners. No amount of caching or parallelization made any noticeable difference above noise. With this change, we see the following improvements:
- 12 minutes to build from scratch on normal runners (unchanged)
- 4 minutes on 32-core runners from scratch
- 1 minute with optimal caches available on normal runners.
- 30 seconds on 32-core runners with optimal caches
PiperOrigin-RevId: 508799909
These will still use MSVC as the compiler, but will no longer generate Visual Studio projects for the builds. Visual Studio is particularly bad at parallelizing builds, and is hostile to ccache.
This change also tweaks the ccache setup to prevent unbounded the growth observed in our github caches. Windows builds have had debug symbols stripped to reduce ccache size by a factor of 2x, and ccache maximum have been tweaked so that we persist fewer older builds.
Before this change, each CMake build took 12 minutes on every run (plus some constant overhead from staleness/gcloud), even with caching or on large multi-core runners. No amount of caching or parallelization made any noticeable difference above noise. With this change, we see the following improvements:
- 12 minutes to build from scratch on normal runners (unchanged)
- 4 minutes on 32-core runners from scratch
- 1 minute with optimal caches available on normal runners.
- 30 seconds on 32-core runners with optimal caches
PiperOrigin-RevId: 508799909
This uses ccache + github caching to substantially decrease the time it takes to run CMake builds. Due to Bazel caching, these are some of our slowest tests, causing one of the biggest presubmit bottlenecks
PiperOrigin-RevId: 507667813
This deletes our old branches of internal Abseil code in favor of their newly open-sourced library. Notably, this removes the ability to turn CHECK crashes into exceptions.
PiperOrigin-RevId: 504460562
This leaves the decision of which C++ version to use up to our users. We still have a static_assert in port_def.inc that will prevent pre-C++14 usage.
PiperOrigin-RevId: 501351066
--
cc1f708d75 by Joey Parrish <joeyparrish@google.com>:
Fix misdetection of libatomic with strict compiler flags
With strict flags and -Werror, the detection for libatomic breaks, causing it to be included in places where it is not needed. And on macOS, this leads to a link error when libatomic cannot be found.
Closes#10899
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/10900 from joeyparrish:fix-libatomic cc1f708d75
PiperOrigin-RevId: 487859041
This cl hit an issue during the shared library cmake build from ODR violations, leading to mismatched absl hash seeds. The problem was pre-existing but didn't manifest until now, and can be traced to the fact that in shared library builds we linked Abseil statically. All of the cmake changes here remove the underlying ODR violation.
PiperOrigin-RevId: 485787671
This change enables projects that consume protobuf via
`FetchContent_MakeAvailable()` to set the C++ version to be used. This is
necessary, as linking code compiled for different C++ standards is asking for
trouble (and will simply not work in some cases).
Check that any version that might be set in `CMAKE_CXX_STANDARD` is new
enough (C++14 or later). On Cygwin, check if any `-std=gnu++XX` has
already been set. In all cases, default to C++14.
* Proof of concept for Abseil dependency
* Adding most common Abseil libraries
* Fixing shared library breakages
* Switching to quotes over angled brackets
* Disable install target by default
* Fixing abseil to LTS commit
* Upgrade to latest Abseil LTS
* Turning install back on by default, removing unnecessary export statements
* Add note to future self
* Fixing unsafe globals
* Upgrade third_party/googletest submodule to current main branch
We can finally do this upgrade now that we have dropped our autotools
build. Googletest recommends living at head, so let's go straight to the
most recent commit on main. For some reason the googletest archive is
not present in the Bazel build mirror, so I removed that entry and just
left the GitHub download link in our WORKSPACE file.
Googletest now requires C++14, so I updated all the C++11 flags I could
find to C++14 instead. I added a .bazelrc file to add -std=c++14 for all
our Bazel builds.
* Delete the empty //src/google/protobuf:protobuf_test target
* Avoid building C++ unit tests in aarch64 jobs for Python and Ruby
* Upgrade third_party/googletest submodule to current main branch
We can finally do this upgrade now that we have dropped our autotools
build. Googletest recommends living at head, so let's go straight to the
most recent commit on main. For some reason the googletest archive is
not present in the Bazel build mirror, so I removed that entry and just
left the GitHub download link in our WORKSPACE file.
Googletest now requires C++14, so I updated all the C++11 flags I could
find to C++14 instead. I added a .bazelrc file to add -std=c++14 for all
our Bazel builds.
* Delete the empty //src/google/protobuf:protobuf_test target
* Avoid building C++ unit tests in aarch64 jobs for Python and Ruby
* Bazelfying conformance tests
Adding infrastructure to "Bazelify" languages other than Java and C++
* Delete benchmarks for languages supported by other repositories
* Bazelfying benchmark tests
* Bazelfying python
Use upb's system python rule instead of branching tensorflow
* Bazelfying Ruby
* Bazelfying C#
* Bazelfying Objective-c
* Bazelfying Kokoro mac builds
* Bazelfying Kokoro linux builds
* Deleting all deprecated files from autotools cleanup
This boils down to Makefile.am and tests.sh and all of their remaining references
* Cleanup after PR reorganizing
- Enable 32 bit tests
- Move conformance tests back
- Use select statements to select alternate runtimes
- Add internal prefixes to proto library macros
* Updating READMEs to use bazel instead of autotools.
* Bazelfying Kokoro release builds
* First round of review fixes
* Second round of review fixes
* Third round of review fixes
* Filtering out conformance tests from Bazel on Windows (b/241484899)
* Add version metadata that was previously scraped from configure.ac
* fixing typo from previous fix
* Adding ruby version tests
* Bumping pinned upb version, and adding tests to python CI
* Adding non-docker builds for protoc in our supported build systems
* Tweaking bazel builds
* Fixing compatibility with older CMake used by kokoro
* Fixing configs to point to proper scripts
* Propagate errors up properly
* Reinstall Bazel, point NMake to compiler, fix broken static_assert in test
* Adding common setup, enabling long paths
* Explicitly pick 64-bit compiler
* Removing arch from nmake build
* Try to upgrade cmake for installation
* Enabling 8.3 filename support (https://bazel.build/docs/windows)
* Fixing choco calls
* Making installation use Release
* Switching to batch for common config
* Fixing some install test issues unique to Windows
* Attempt another way to get shorter Bazel paths
* More windows bazel tweaks
* Fixing shared library failure from non-exported function used in tests
* Use commandline flags instead of bazelrc
* Fix line break mismatch
* Add .exe extension for test plugin on windows
* Add temporary logging
* Switch to rootpath over hardcoded system-dependent plugin path
* Enabling symlinks/runfiles to fix broken test on windows
* Initial implementation of cmake tests for linux
* Reverting accidental distcheck changes
* Deleting extract_includes.bat now that it's been replaced with the generated file_lists.cmake
* Removing unnecessary endif conditions
* Revert "Added cmake abseil include guard"
This reverts commit b6ee841d7c.
* Revert "Update CMake configuration to add a dependency on Abseil (#9793)"
This reverts commit e9246cd789.
* Proof of concept for CMake Abseil dependency
* Hooked up Abseil linking
* Adding test binaries
* Reverting absl::string view use added for testing. This will still be broken for automake and bazel
* Adding new cmake config to dist list
* Whitespace fixes and an attempt at fix for Windows Python Release (CMake version < 3.13)
* Hook up LTS to windows cmake
* Fix 'git cd' typo
* Adding clarifying comment for fix
* Adding abseil dependency to protobuf-lite too
* Update submodules instead of cloning abseil-cpp
* Adding explicit pointer to abseil root directory
Co-authored-by: Jason Lunn <jason.lunn@gmail.com>
Co-authored-by: Jorg Brown <jorg.brown@gmail.com>
* Support older versions of CMake.
VERSION_GREATER_EQUAL is supported by CMake >=3.7:
https://cmake.org/cmake/help/latest/command/if.html#version-greater-equal
target_link_options() is supported by CMake >=3.13:
https://cmake.org/cmake/help/latest/command/target_link_options.html
* Use PROTOBUF_USE_DLLS and LIBPROTOC_EXPORTS for libprotoc.so shared library
Similar change for libprotobuf.so and libprotobuf-lite.so was made
in commit 5a0887fc65.
* Make value of PROTOC_EXPORT dependent on LIBPROTOC_EXPORTS instead of LIBPROTOBUF_EXPORTS
This was probably intention, and LIBPROTOC_EXPORTS is also used above
in _MSC_VER branch.
Convenience feature: enable users to test via the familiar `ctest` command rather than making the `check` target.
They would be able to use the familiar CMake pattern:
```
cmake -S source/protobuf -B build/protobuf ...
cmake --build build/protobuf
ctest --test-dir build/protobuf
cmake --install build/protobuf
```
This is a follow-up to 9f447fc9d3da93da29b8301f1a8ca57b1ea812d7
when using CMake >= 3.15, we use CMAKE_MSVC_RUNTIME_LIBRARY
to reproduce the /MT vs /MD when protobuf_MSVC_STATIC_RUNTIME
is TRUE or FALSE
MultiThreaded is for /MT
MultiThreadedDebug is for /MTd
MultiThreadedDLL is for /MD
MultiThreadedDebugDLL is for /MDd
This is based on @haberman's changes in #8257. Now that we're using
std::mutex we no longer need to check whether pthreads are available, so
this commit removes references to HAVE_PTHREAD. I left the autotools
build alone, though, since we are likely to drop support for it soon
anyway.
The `VERSION_GREATER_EQUAL` operator was introduced in CMake 3.7.0
(released in 2016). Use the long form of the check by splitting the
operation into two comparisons to support the older CMake.
When using a new enough CMake (3.15+) prefer to use the
`MSVC_RUNTIME_LIBRARY` property on targets to select the runtime library
variant. This property is automatically set to the value specified by
`CMAKE_MSVC_RUNTIME_LIBRARY`. This property requires that the CMake
Policy 91 is set to new (see CMP0091).
This is useful for Conan recipes that build Protobuf, in which
whatever we want to enable has to be enabled in the initial command line.
Without this, the people maintaining the recipe have to patch the CMake
setup of Protobuf before building the binaries.
Closes#5541
This is useful for Conan recipes that build Protobuf, in which
whatever we want to enable has to be enabled in the initial command line.
Without this, the people maintaining the recipe have to patch the CMake
setup of Protobuf before building the binaries.
Closes#5541
* Fix a typo
* Fix lots of spelling errors
* Fix a few more spelling mistakes
* s/parsable/parseable/
* Don't touch the third party files
* Cloneable is the preferred C# term
* Copyable is the preferred C++ term
* Revert "s/parsable/parseable/"
This reverts commit 534ecf7675.
* Revert unparseable->unparsable corrections
* Update CHANGES.txt with 3.11.0-RC1 release notes (#6909)
* Revert "Make shared libraries be able to link to MSVC static runtime libraries, so that VC runtime is not required." (#6914)
* Marked update_compatibility_version.py as executable (#6916)
This reverts commit 129a7c875f. We are
seeing the following error when building Python release artifacts in Windows:
" error LNK2038: mismatch detected for 'RuntimeLibrary': value
'MD_DynamicRelease' doesn't match value 'MT_StaticRelease' in
descriptor.obj".
Mike Kruskal
Protobuf Team Bot
Joey Parrish
Christian Blichmann
Mike Kruskal
Adam Cozzette
Alexander Neumann
David L. Jones
Arfrever Frehtes Taifersar Arahesis
Florin Crișan
Stephen Kennedy
JCooky
David Geldreich
Shigeo Hashimoto
Saleem Abdulrasool
Saleem Abdulrasool
Jorge López Fueyo
Florian Simon
Daniel Joos
Peter Newman
listonjesse
Chuck Atkins
Rafi Kamal