Using python's xdg-caching can cause flakes

Currently when running python test on linux, we are mounting
/tmp/xdg-cache-home to the docker container that runs the tests
in the attempt to prevent unnecessary downloads of pip packages
(with the theory that more downloads leads to increased flakiness)
- the idea is that while there is a new docker container for each
test suite, the xdg cache remains per-VM.

This approach no longer seems to be useful:
* It turns out that XDG cache doesn't work reliably when multiple
docker containers are using it concurrently (the concurrent run can
see corrupt files). We are using concurrent docker containers in our
multilang test suite to speed up the execution and we are currently
getting flakes seeing flakes caused by this.
* support for caching makes our docker_run scripts more complicated
and we really don't want that.
* since we migrated to kokoro, the caching is limited anyway -
as each run gets a fresh VM, we need to download packages anyway
for every build (and that actually seems to causing way less
flakiness the problem with concurrent XDG caching).
pull/14313/head
Jan Tattermusch 7 years ago
parent 73a50abd1d
commit fb1ad35984
  1. 5
      tools/internal_ci/helper_scripts/prepare_build_linux_rc
  2. 6
      tools/run_tests/dockerize/build_docker_and_run_tests.sh
  3. 4
      tools/run_tests/dockerize/docker_run_tests.sh

@ -27,11 +27,6 @@ echo 'DOCKER_OPTS="${DOCKER_OPTS} --graph=/tmpfs/docker"' | sudo tee --append /e
echo 'DOCKER_OPTS="${DOCKER_OPTS} --registry-mirror=https://mirror.gcr.io"' | sudo tee --append /etc/default/docker
sudo service docker restart
# Populate xdg-cache-home to workaround https://github.com/grpc/grpc/issues/11968
sudo mkdir -p /tmp/xdg-cache-home
PYTHONWARNINGS=ignore XDG_CACHE_HOME=/tmp/xdg-cache-home sudo -E pip install setuptools wheel
PYTHONWARNINGS=ignore XDG_CACHE_HOME=/tmp/xdg-cache-home sudo -E pip install coverage==4.4 pylint==1.6.5
# Download Docker images from DockerHub
export DOCKERHUB_ORGANIZATION=grpctesting

@ -25,10 +25,6 @@ cd -
# Ensure existence of ccache directory
mkdir -p /tmp/ccache
# Ensure existence of the home directory for XDG caches (e.g. what pip uses for
# its cache location now that --download-cache is deprecated).
mkdir -p /tmp/xdg-cache-home
# Inputs
# DOCKERFILE_DIR - Directory in which Dockerfile file is located.
# DOCKER_RUN_SCRIPT - Script to run under docker (relative to grpc repo root)
@ -61,7 +57,6 @@ docker run \
-e "config=$config" \
-e "arch=$arch" \
-e CCACHE_DIR=/tmp/ccache \
-e XDG_CACHE_HOME=/tmp/xdg-cache-home \
-e THIS_IS_REALLY_NEEDED='see https://github.com/docker/docker/issues/14203 for why docker is awful' \
-e HOST_GIT_ROOT="$git_root" \
-e LOCAL_GIT_ROOT=$docker_instance_git_root \
@ -79,7 +74,6 @@ docker run \
-v "$git_root:$docker_instance_git_root" \
-v /tmp/ccache:/tmp/ccache \
-v /tmp/npm-cache:/tmp/npm-cache \
-v /tmp/xdg-cache-home:/tmp/xdg-cache-home \
-w /var/local/git/grpc \
--name="$CONTAINER_NAME" \
"$DOCKER_IMAGE_NAME" \

@ -22,10 +22,6 @@ export CONFIG=$config
export ASAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer
export PATH=$PATH:/usr/bin/llvm-symbolizer
# Ensure that programs depending on current-user-ownership of cache directories
# are satisfied (it's being mounted from outside the image).
chown "$(whoami)" "$XDG_CACHE_HOME"
mkdir -p /var/local/git
git clone /var/local/jenkins/grpc /var/local/git/grpc
# clone gRPC submodules, use data from locally cloned submodules where possible

Loading…
Cancel
Save