From 78efa761726dbb88992d686f2872cc0119b44d7d Mon Sep 17 00:00:00 2001 From: Mike Kruskal <62662355+mkruskal-google@users.noreply.github.com> Date: Tue, 26 Jul 2022 13:53:51 -0700 Subject: [PATCH] Add CMake Ninja test (#10314) * Initial implementation of cmake tests for linux * Reverting accidental distcheck changes * Updating gitignore file to include cmake generated files * Updating existing cmake tests to use newer docker image * Adding CMake build that uses Ninja as the generator * Updating CMake documentation to cover Linux support * Updating to latest cmake image --- .gitignore | 18 +++++++ cmake/README.md | 69 +++++++++++++++---------- cmake/conformance.cmake | 13 +++-- kokoro/linux/cmake/build.sh | 4 +- kokoro/linux/cmake_install/build.sh | 9 ++-- kokoro/linux/cmake_ninja/build.sh | 26 ++++++++++ kokoro/linux/cmake_ninja/continuous.cfg | 11 ++++ kokoro/linux/cmake_ninja/presubmit.cfg | 11 ++++ 8 files changed, 121 insertions(+), 40 deletions(-) create mode 100755 kokoro/linux/cmake_ninja/build.sh create mode 100644 kokoro/linux/cmake_ninja/continuous.cfg create mode 100644 kokoro/linux/cmake_ninja/presubmit.cfg diff --git a/.gitignore b/.gitignore index d8b0bd3842..cc881f3f2c 100644 --- a/.gitignore +++ b/.gitignore @@ -19,6 +19,23 @@ m4/ltversion.m4 m4/lt~obsolete.m4 autom4te.cache +# CMake-generated files +.ninja_deps +.ninja_logs +cmake/protobuf/*.cmake +cmake_install.cmake +CMakeCache.txt +CTestTestfile.cmake +CMakeFiles/* +Testing/Temporary/* + +/core +/protoc +/test_plugin +/tests +/lite-test +/protoc-*.* + # downloaded files /gmock @@ -41,6 +58,7 @@ stamp-h1 *.la src/.libs *.so +*.a .dirstamp diff --git a/cmake/README.md b/cmake/README.md index ce3e680234..4a4259e45f 100644 --- a/cmake/README.md +++ b/cmake/README.md @@ -1,15 +1,19 @@ -This directory contains *CMake* files that can be used to build protobuf -with *MSVC* on *Windows*. You can build the project from *Command Prompt* -and using an *Visual Studio* IDE. +This directory contains *CMake* files that can be used to build protobuf. -You need to have [CMake](http://www.cmake.org), [Visual Studio](https://www.visualstudio.com) -and optionally [Git](http://git-scm.com) installed on your computer before proceeding. +You need to have [CMake](http://www.cmake.org) and +[Git](http://git-scm.com) installed on your computer before proceeding. -Most of the instructions will be given to the *Сommand Prompt*, but the same -actions can be performed using appropriate GUI tools. +Most of the instructions will be given using CMake's command-line interface, but +the same actions can be performed using appropriate GUI tools. -Environment Setup -================= +# Windows Builds + +On Windows, you can build the project from *Command Prompt* and using an +*Visual Studio* IDE. You will also need to have +[Visual Studio](https://www.visualstudio.com) installed on your computer before +proceeding. + +## Environment Setup Open the appropriate *Command Prompt* from the *Start* menu. @@ -42,8 +46,7 @@ Optionally, you will want to download [ninja](https://ninja-build.org/) and add Good. Now you are ready to continue. -Getting Sources -=============== +## Getting Sources You can get the latest stable source packages from the release page: @@ -76,8 +79,7 @@ C:\Path\to\src\protobuf> git submodule update --init --recursive Good. Now you are ready for *CMake* configuration. -CMake Configuration -=================== +## CMake Configuration *CMake* supports a lot of different [generators](http://www.cmake.org/cmake/help/latest/manual/cmake-generators.7.html) @@ -137,8 +139,7 @@ The *Visual Studio* generator is multi-configuration: it will generate a single It will generate *Visual Studio* solution file *protobuf.sln* in current directory. -Unit Tests ----------- +### Unit Tests Unit tests are being built along with the rest of protobuf. The unit tests require Google Mock (now a part of Google Test). @@ -178,8 +179,7 @@ For example: -Dprotobuf_BUILD_TESTS=OFF ^ C:\Path\to\src\protobuf -Compiling -========= +## Compiling The standard way to compile a *CMake* project is `cmake --build `. @@ -206,8 +206,7 @@ If you prefer to use the IDE: And wait for the compilation to finish. -Testing -======= +## Testing To run unit-tests, first you must compile protobuf as described above. Then run: @@ -259,8 +258,7 @@ Note that the tests must be run from the source folder. If all tests are passed, safely continue. -Installing -========== +## Installing To install protobuf to the *install* folder you've specified in the configuration step, you need to build the `install` target: @@ -292,8 +290,7 @@ compiling a debug build of your application, you may need to link against a debug build of libprotobufd.lib with "d" postfix. Similarly, release builds should link against release libprotobuf.lib library. -DLLs vs. static linking -======================= +## DLLs vs. static linking Static linking is now the default for the Protocol Buffer libraries. Due to issues with Win32's use of a separate heap for each DLL, as well as binary @@ -318,8 +315,7 @@ recommend that you do NOT expose protocol buffer objects in your library's public interface, and that you statically link protocol buffers into your library. -ZLib support -============ +## ZLib support If you want to include GzipInputStream and GzipOutputStream (google/protobuf/io/gzip_stream.h) in libprotobuf, you will need to do a few @@ -369,8 +365,7 @@ If you already have ZLIB library and headers at some other location on your syst Build and testing protobuf as usual. -Notes on Compiler Warnings -========================== +## Notes on Compiler Warnings The following warnings have been disabled while building the protobuf libraries and compiler. You may have to disable some of them in your own project as @@ -397,3 +392,23 @@ unique, so there should be no problem with this, but MSVC prints warning nevertheless. So, we disable it. Unfortunately, this warning will also be produced when compiling code which merely uses protocol buffers, meaning you may have to disable it in your code too. + +# Linux Builds + +Building with CMake works very similarly on Linux. Instead of Visual Studio, +you will need to have gcc or clang installed to handle the C++ builds. CMake +will generate Makefiles by default, but can also be configured to use Ninja. To +build Protobuf, you will need to run (from the source directory): + + cmake . + cmake --build . --parallel 10 + +Protobuf can be tested and installed with CMake: + + ctest --verbose + sudo cmake --install . + +or directly with the generated Makefiles: + + make VERBOSE=1 test + sudo make install diff --git a/cmake/conformance.cmake b/cmake/conformance.cmake index 13a7346e4a..338b2757a5 100644 --- a/cmake/conformance.cmake +++ b/cmake/conformance.cmake @@ -51,11 +51,10 @@ target_include_directories( target_link_libraries(conformance_test_runner ${protobuf_LIB_PROTOBUF}) target_link_libraries(conformance_cpp ${protobuf_LIB_PROTOBUF}) -add_custom_target(conformance_cpp_test - COMMAND conformance_test_runner - --failure_list conformance/failure_list_cpp.txt - --text_format_failure_list conformance/text_format_failure_list_cpp.txt +add_test(NAME conformance_cpp_test + COMMAND ${CMAKE_CURRENT_BINARY_DIR}/conformance_test_runner + --failure_list ${protobuf_SOURCE_DIR}/conformance/failure_list_cpp.txt + --text_format_failure_list ${protobuf_SOURCE_DIR}/conformance/text_format_failure_list_cpp.txt --output_dir ${protobuf_TEST_XML_OUTDIR} - ${protobuf_BINARY_DIR}/conformance_cpp - DEPENDS conformance_test_runner conformance_cpp - WORKING_DIRECTORY ${protobuf_SOURCE_DIR}) + ${CMAKE_CURRENT_BINARY_DIR}/conformance_cpp + DEPENDS conformance_test_runner conformance_cpp) diff --git a/kokoro/linux/cmake/build.sh b/kokoro/linux/cmake/build.sh index 29d837b865..1b0ebfc5fa 100755 --- a/kokoro/linux/cmake/build.sh +++ b/kokoro/linux/cmake/build.sh @@ -8,7 +8,7 @@ set -eux cd $(dirname $0)/../../.. GIT_REPO_ROOT=`pwd` -CONTAINER_IMAGE=gcr.io/protobuf-build/cmake/linux@sha256:7aaac41a2f06258b967facf2e6afbd17eec01e85fb6a14b44cb03c9372311363 +CONTAINER_IMAGE=gcr.io/protobuf-build/cmake/linux@sha256:79e6ed9d7f3f8e56167a3309a521e5b7e6a212bfb19855c65ee1cbb6f9099671 # Update git submodules git submodule update --init --recursive @@ -19,7 +19,7 @@ docker run \ --cidfile $tmpfile \ -v $GIT_REPO_ROOT:/workspace \ $CONTAINER_IMAGE \ - CMAKE_FLAGS=-Dprotobuf_BUILD_CONFORMANCE=ON /make.sh check conformance_cpp_test + /test.sh -Dprotobuf_BUILD_CONFORMANCE=ON # Save logs for Kokoro docker cp \ diff --git a/kokoro/linux/cmake_install/build.sh b/kokoro/linux/cmake_install/build.sh index b0e2b8d20d..6fdafa557e 100755 --- a/kokoro/linux/cmake_install/build.sh +++ b/kokoro/linux/cmake_install/build.sh @@ -8,7 +8,7 @@ set -eux cd $(dirname $0)/../../.. GIT_REPO_ROOT=`pwd` -CONTAINER_IMAGE=gcr.io/protobuf-build/cmake/linux@sha256:7aaac41a2f06258b967facf2e6afbd17eec01e85fb6a14b44cb03c9372311363 +CONTAINER_IMAGE=gcr.io/protobuf-build/cmake/linux@sha256:79e6ed9d7f3f8e56167a3309a521e5b7e6a212bfb19855c65ee1cbb6f9099671 # Update git submodules git submodule update --init --recursive @@ -19,9 +19,10 @@ docker run \ --cidfile $tmpfile \ -v $GIT_REPO_ROOT:/workspace \ $CONTAINER_IMAGE \ - "/install.sh; \ - CMAKE_FLAGS=\"-Dprotobuf_REMOVE_INSTALLED_HEADERS=ON -Dprotobuf_BUILD_PROTOBUF_BINARIES=OFF -Dprotobuf_BUILD_CONFORMANCE=ON\" \ - /make.sh check conformance_cpp_test" + "/install.sh && /test.sh \ + -Dprotobuf_REMOVE_INSTALLED_HEADERS=ON \ + -Dprotobuf_BUILD_PROTOBUF_BINARIES=OFF \ + -Dprotobuf_BUILD_CONFORMANCE=ON" # Save logs for Kokoro diff --git a/kokoro/linux/cmake_ninja/build.sh b/kokoro/linux/cmake_ninja/build.sh new file mode 100755 index 0000000000..d3a281f9ec --- /dev/null +++ b/kokoro/linux/cmake_ninja/build.sh @@ -0,0 +1,26 @@ +#!/bin/bash +# +# Build file to set up and run tests based on distribution archive + +set -eux + +# Change to repo root +cd $(dirname $0)/../../.. +GIT_REPO_ROOT=`pwd` + +CONTAINER_IMAGE=gcr.io/protobuf-build/cmake/linux@sha256:79e6ed9d7f3f8e56167a3309a521e5b7e6a212bfb19855c65ee1cbb6f9099671 + +# Update git submodules +git submodule update --init --recursive + +tmpfile=$(mktemp -u) + +docker run \ + --cidfile $tmpfile \ + -v $GIT_REPO_ROOT:/workspace \ + $CONTAINER_IMAGE \ + /test.sh -G Ninja -Dprotobuf_BUILD_CONFORMANCE=ON + +# Save logs for Kokoro +docker cp \ + `cat $tmpfile`:/workspace/logs $KOKORO_ARTIFACTS_DIR diff --git a/kokoro/linux/cmake_ninja/continuous.cfg b/kokoro/linux/cmake_ninja/continuous.cfg new file mode 100644 index 0000000000..144fc90a82 --- /dev/null +++ b/kokoro/linux/cmake_ninja/continuous.cfg @@ -0,0 +1,11 @@ +# Config file for running tests in Kokoro + +# Location of the build script in repository +build_file: "protobuf/kokoro/linux/cmake_ninja/build.sh" +timeout_mins: 1440 + +action { + define_artifacts { + regex: "**/sponge_log.*" + } +} diff --git a/kokoro/linux/cmake_ninja/presubmit.cfg b/kokoro/linux/cmake_ninja/presubmit.cfg new file mode 100644 index 0000000000..144fc90a82 --- /dev/null +++ b/kokoro/linux/cmake_ninja/presubmit.cfg @@ -0,0 +1,11 @@ +# Config file for running tests in Kokoro + +# Location of the build script in repository +build_file: "protobuf/kokoro/linux/cmake_ninja/build.sh" +timeout_mins: 1440 + +action { + define_artifacts { + regex: "**/sponge_log.*" + } +}