Merge branch 'master' into private
* master:
Add docker-compose to the slave-bazel-docker image
Fix task plugin branch configuration
Print chrome version used for PolyGerrit tests to build logs
Build owners plugin on stable-3.4
Add stable-3.{3,4} rate-limiter master tree builds
Add stable-3.{3,4} heartbeat master tree builds
Add stable-3.{3,4} gc-conductor master tree builds
Remove obsolete gerrit branch go-import build lines
Build go-import plugin on stable-3.4 branch
Build/test owners on Java 11 for stable-3.3 onwards
Run tests for the owners' plugin builds
Allow to build all owners-a* artifacts
Build go-import plugin on stable-3.3 branch
Remove builds on stable-3.1 branch
Gatling tests: set Gerrit to 3.4.0-rc5
Build events-rabbitmq plugin on master
Set Gerrit to v3.4.0-rc4
Simplify the execution of the release container
Pass or fail gatling-gerrit-test based on gatling results
Build project-group-structure plugin on stable-3.3 && stable-3.4 branch
Remove unused manual plugin build template
Export logs to S3 when running gatling-gerrit-test
Test Gerrit v3.4.0-rc3 with Gatling
Build multi-site on its own stable-3.4 branch
Build zookeeper-refdb on its own stable-3.4 branch
Build events-kafka on its own stable-3.4 branch
Install python3-distutils package
Build events-aws-kinesis master against gerrit-3.4
Build events-gcloud-pubsub master against gerrit-3.4
Avoid looking for plugin's artifacts in subdirectories
Enable events-kafka build on 3.1, 3.2, 3.3
Verify the events-gcloud-pubsub plugin changes on stable-3.3
Verify the events-aws-kinesis plugin changes on stable-3.3
Remove redundant repo variable in events-aws-kinesis
Build events-kafka plugin on stable-2.16/3.0
Move gcloud-pubsub-events job to events-gcloud-pubsub
Move kinesis-events job to events-aws-kinesis
Build events-broker on its own stable-3.4 branch
Pass GERRIT_WAR_URL to override the version under test
Build account plugin on stable-3.4 branch
Build analytics plugin on stable-3.4 branch
Gatling tests: bump Gerrit to v3.4.0-rc1
Build owners plugin on stable-3.4 branch
Build find-owners plugin on stable-3.4 branch
Build github plugin on stable-3.4 branch
Build its-base plugin on stable-3.4 branch
Build its-jira plugin on stable-3.4 branch
Build kafka-events plugin on stable-3.4 branch
Build menuextender plugin on stable-3.4 branch
Build simple-submit-rules plugin on stable-3.4 branch
Build verify-status plugin on stable-3.4 branch
Build metrics-reporter-prometheus plugin on stable-3.4 branch
Build oauth plugin on stable-3.4 branch
Build reject-private-submit plugin on stable-3.4 branch
Build rename-project plugin on stable-3.4 branch
Build reviewassistant plugin on stable-3.4 branch
Build groovy-provider plugin on stable-3.4 branch
Build slack-integration plugin on stable-3.4 branch
Build websession-broker plugin on stable-3.4 branch
Build websession-flatfile plugin on stable-3.4 branch
Build zookeeper-refdb plugin on stable-3.4 branch
Build zuul-results-summary plugin on stable-3.4 branch
Build serviceuser plugin on stable-3.4 branch
Build ref-protection plugin on stable-3.4 branch
Build login-redirect plugin on stable-3.4 branch
Build events-log plugin on stable-3.4 branch
Build quota plugin on stable-3.4 branch
Build lfs plugin on stable-3.4 branch
Build audit-sl4j plugin on stable-3.4 branch
Build code-owners plugin on stable-3.4 branch
Build branch-network plugin on stable-3.4 branch
Build pull-replication plugin on stable-3.4 branch
Build avatars-external plugin on stable-3.4 branch
Build autosubmitter plugin on stable-3.4 branch
Build automerger plugin on stable-3.4 branch
Build admin-console plugin on stable-3.4 branch
Build module events-broker on stable-3.4 branch
Build module cache-chroniclemap on stable-3.4 branch
Build reviewers-by-blame plugin on stable-3.4 branch
Build readonly plugin on stable-3.4 branch
Build multi-site plugin on stable-3.4 branch
Build high-availability plugin on stable-3.4 branch
Build reviewers plugin on stable-3.4 branch
Build checks plugin on stable-3.4 branch
Build saml plugin on stable-3.4 branch
zuul-results-summary plugin is jar plugin now
Build saml plugin on own 3.2 and 3.3 branches
Build metrics-reporter-cloudwatch plugin on stable-3.4
Build healthcheck plugin on stable-3.4
Build javamelody plugin on stable-3.4
Gatling tests: bump Gerrit to v3.4.0-rc0
Rename gatling test pipeline using kebab-case
Build Gerrit and validate changes on stable-3.4
Build out-of-the-box plugin on stable-3.4
Build avatars-gravatar plugin on stable-3.4
Build uploadvalidator plugin on stable-3.4
Introduce Plugins-stable-3.4 view on Gerrit-CI
Build ref-prottection plugn in 3.3
Build kinesis plugin master against Gerrit stable-3.3
Add gcloud-pubsub-events plugin build
Add kinesis-events plugin build
Bump Gerrit/Gatling tests to v3.3.2
Revert "Move execution of gatling test to 17:15 UTC"
Fully qualify parameters and environment variables
Move execution of gatling test to 17:15 UTC
Build Owner plugin on stable 3.3
Default git http username and password to empty strings
Make gatling-gerrit-test job more resilient
Gatling Gerrit: Generate unique target URL
Adapt to aws-gerrit primary/replica renaming
Build go-import plugin on own 3.2 branch
Build javamelody plugin on own 3.2 branch
Quote and qualify parameters explicitly
Build automerger plugin on recent branches
Adapt checker query
Allow publishing to different Docker registries
Mention that Java 8 will have to stay until v3.4 is EOL
Fix slave-debian Dockerfile setup of OpenJDK 8
web-component-tester: Install babel runtime dependency
Fix Jenkins master image
Remove broken --test_verbose_timeout_warnings option
Remove explicit --build_tests_only option
Remove explicit --test_output errors option
Remove --test_summary=detailed option
Test only Git protocol v2 and Elastic outside RBE
Revert "Revert "Bazel: Activate RBE configuration on GCP""
Default params for gatling against single-master gerrit
Revert "Bazel: Activate RBE configuration on GCP"
Revert "Revert "Bazel: Activate RBE configuration on GCP""
Don't build bazelisk and buildifier from source
Build server-config plugin for stable-3.1
Do not import Google's DNS hosts on Docker images
Bump buildifier version to 4.0.0
Bump bazel version to 4.0.0
Build reject-private-submit plugin against 3.0-3.3
Build zuul-results-summary plugin on stable-3.3
Cross-build Gerrit UI plugins
Build events-broker module on recent branches
Build find-owners plugin against its own 3.2 branch
Build its-base plugin against 3.3
Build its-jira plugin against 3.3
Fix jenkins docker image name
Build lfs plugin against 3.3
Build egit against gerrit 3.3
Build menuextender against gerrit 3.3
Build events-log against gerrit 3.3
Build readonly plugin against gerrit 3.3
Build reviewassistant plugin against gerrit 3.3
Build avatars-external against gerrit 3.3
Build login-redirect against gerrit 3.3
Build admin-console against gerrit 3.3
Fix bazelisk build on Docker build image
Build autosubmitter plugin on stable-3.3 branch
Build metrics-reporter-elasticsearch plugin
Update Docker debian variant for Jenkins master
Build serviceuser plugin on stable-3.3
Build reviewassistant on 3.2 branch
Revert "Bazel: Activate RBE configuration on GCP"
Revert "Revert "Bazel: Activate RBE configuration on GCP""
Revert "Bazel: Activate RBE configuration on GCP"
Bazel: Activate RBE configuration on GCP
Build events-broker module
Remove builds on stable-3.0
Remove obsolete Bazel spawn strategies from plugin branch build
Build autosubmitter plugin on stable-3.2
Multiple targets build for zookeeper-refdb plugin
Fix branch detection for Java 8/11 to build plugins
Build websession-flatfile on stable-3.3
Build quota plugin on stable-3.2 and stable-3.3
Build oauth plugin on stable-3.2
Build saml plugin on recent branches
Remove obsolete Bazel spawn strategies
Remove '[' in the execution of git diff for .bazelversion
Build code-owners plugin on stable-3.3
Build simple-submit-rules plugin on stable-3.2 and stable-3.3
Build reviewassistant on 3.0 and 3.1 branch
Build verify-status plugin on stable-3.1
Build simple-submit-rules plugin on stable-3.1
Remove obsolete Bazel spawn strategies
Build rename-project plugin on stable-3.3
Build zookeeper-refdb plugin on stable-3.3
Verify kafka-events plugin
Build out-of-the-box plugin on stable-3.3
Build kafka-events plugin on stable-3.3
Build reviewers plugin on stable-3.3
Build account plugin on stable-3.3
Build analytics plugin on stable-3.3
Extend maximum build time to 2h
Build code-owners plugin on stable-3.3
Build uploadvalidator plugin on stable-3.3
Build websession-broker plugin on stable-3.3
Build slack-integration plugin on stable-3.3
Build pull-replication plugin on stable-3.3
Build multi-site plugin on stable-3.3
Build groovy-provider on stable-3.3
Build reviewers-by-blame plugin on stable-3.3
Build oauth plugin on stable-3.3
Build find-owners plugin on stable-3.2 and stable-3.3
Build audit-sl4j on stable-3.3
Build github plugin on stable-3.3
Set the correct JDK version for Maven plugins
Build batch plugin on its stable-3.3 branch
Build checks plugin on stable-3.3
Build high-availability plugin on stable-3.3
Build branch-network plugin on stable-3.3
Build avatars-gravatar plugin on stable-3.3
Build autosubmitter plugin on stable-3.3
Revert "Disable elasticsearch tests in Gerrit-CI builds"
Build metrics-reporter-cloudwatch against stable-3.3
Build healthcheck stable-3.3 branch
Build javamelody master branch against stable-3.3
Fixed typo for GERRIT_VOLUME_SNAPSHOT_ID
Expose gerrit version and gerrit patch parameters
Make DOCKER_HOST env variable available to all steps
Adopt gatling-gerrit-test job to the gerrit-ci environment
Change-Id: I3978d1b8a66daf7d2518b628846b74d572a64b9d
diff --git a/jenkins-docker/bazel-cache/Makefile b/jenkins-docker/bazel-cache/Makefile
index 2623506..8fb4d04 100644
--- a/jenkins-docker/bazel-cache/Makefile
+++ b/jenkins-docker/bazel-cache/Makefile
@@ -1,5 +1,6 @@
NO_CACHE=false
-IMAGE=gerritforge/bazel-cache
+PREFIX=gerritforge
+IMAGE=${PREFIX}/bazel-cache
CACHE_ALLOW=0.0.0.0/0
# Targets
diff --git a/jenkins-docker/master/Dockerfile b/jenkins-docker/master/Dockerfile
index 9563800..628067e 100644
--- a/jenkins-docker/master/Dockerfile
+++ b/jenkins-docker/master/Dockerfile
@@ -1,14 +1,8 @@
-FROM jenkins:latest
+FROM jenkins/jenkins:${JENKINS_WAR_VER}
USER root
-ARG JENKINS_WAR_VER=2.204.1
-ARG JENKINS_WAR_SHA=924d2c9fabfdcacee1bae757337a07d7599eaa35
-
-# could use ADD but this one does not check Last-Modified header
-# see https://github.com/docker/docker/issues/8331
-RUN curl -fsSL http://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/${JENKINS_WAR_VER}/jenkins-war-${JENKINS_WAR_VER}.war -o /usr/share/jenkins/jenkins.war \
- && echo "$JENKINS_WAR_SHA /usr/share/jenkins/jenkins.war" | sha1sum -c -
+RUN echo "$JENKINS_WAR_SHA /usr/share/jenkins/jenkins.war" | sha1sum -c -
# Override Jenkins start-up script
RUN mv /usr/local/bin/jenkins.sh /usr/local/bin/run-jenkins.sh
diff --git a/jenkins-docker/master/Makefile b/jenkins-docker/master/Makefile
index 413905b..b7157d3 100644
--- a/jenkins-docker/master/Makefile
+++ b/jenkins-docker/master/Makefile
@@ -53,9 +53,9 @@
chmod a+r id_rsa*
build: id_rsa
+ cat Dockerfile | JENKINS_WAR_VER=${JENKINS_WAR_VER} JENKINS_WAR_SHA=${JENKINS_WAR_SHA} envsubst \$$JENKINS_WAR_VER,\$$JENKINS_WAR_SHA > Dockerfile-subst
docker build --no-cache=$(NO_CACHE) \
- --build-arg JENKINS_WAR_VER=${JENKINS_WAR_VER} \
- --build-arg JENKINS_WAR_SHA=${JENKINS_WAR_SHA} -t ${IMAGE} .
+ -t ${IMAGE} -f Dockerfile-subst .
publish:
docker push ${IMAGE}
@@ -76,4 +76,3 @@
echo -e "\n${NAME} is *RUNNING*\n"
.PHONY: clean image publish
-
diff --git a/jenkins-docker/master/config.xml b/jenkins-docker/master/config.xml
index da5e294..51ebc7a 100644
--- a/jenkins-docker/master/config.xml
+++ b/jenkins-docker/master/config.xml
@@ -74,14 +74,10 @@
</retentionStrategy>
<numExecutors>1</numExecutors>
<dockerTemplateBase>
- <image>gerritforge/gerrit-ci-slave-bazel-docker:debian-stretch-80.0.3987.149</image>
+ <image>gerritforge/gerrit-ci-slave-bazel-docker:debian-buster-80.0.3987.149</image>
<dockerCommand></dockerCommand>
<lxcConfString></lxcConfString>
<hostname></hostname>
- <dnsHosts>
- <string>8.8.8.8</string>
- <string>8.8.4.4</string>
- </dnsHosts>
<volumes>
<string>/dev/urandom:/dev/random</string>
</volumes>
@@ -115,14 +111,10 @@
</retentionStrategy>
<numExecutors>1</numExecutors>
<dockerTemplateBase>
- <image>gerritforge/gerrit-ci-slave-bazel-docker:debian-stretch-69.0.3497.92</image>
+ <image>gerritforge/gerrit-ci-slave-bazel-docker:debian-buster-69.0.3497.92</image>
<dockerCommand></dockerCommand>
<lxcConfString></lxcConfString>
<hostname></hostname>
- <dnsHosts>
- <string>8.8.8.8</string>
- <string>8.8.4.4</string>
- </dnsHosts>
<volumes>
<string>/dev/urandom:/dev/random</string>
</volumes>
@@ -236,10 +228,6 @@
<dockerCommand></dockerCommand>
<lxcConfString></lxcConfString>
<hostname></hostname>
- <dnsHosts>
- <string>8.8.8.8</string>
- <string>8.8.4.4</string>
- </dnsHosts>
<volumes>
<string>/dev/urandom:/dev/random</string>
</volumes>
@@ -283,10 +271,6 @@
<dockerCommand></dockerCommand>
<lxcConfString></lxcConfString>
<hostname></hostname>
- <dnsHosts>
- <string>8.8.8.8</string>
- <string>8.8.4.4</string>
- </dnsHosts>
<volumes>
<string>/dev/urandom:/dev/random</string>
</volumes>
diff --git a/jenkins-docker/slave-aws/Dockerfile b/jenkins-docker/slave-aws/Dockerfile
index 2a2c04d..b72e8dc 100644
--- a/jenkins-docker/slave-aws/Dockerfile
+++ b/jenkins-docker/slave-aws/Dockerfile
@@ -9,4 +9,4 @@
$(lsb_release -cs) \
stable" && \
apt-get update && \
- apt-get install -y docker-ce-cli
+ apt-get install -y docker-ce-cli jq
diff --git a/jenkins-docker/slave-aws/Makefile b/jenkins-docker/slave-aws/Makefile
index d3ddc63..3f4a84c 100644
--- a/jenkins-docker/slave-aws/Makefile
+++ b/jenkins-docker/slave-aws/Makefile
@@ -1,5 +1,6 @@
NO_CACHE=false
-IMAGE=gerritforge/gerrit-ci-slave-aws
+PREFIX=gerritforge
+IMAGE=${PREFIX}/gerrit-ci-slave-aws
# Targets
diff --git a/jenkins-docker/slave-bazel-docker/Dockerfile b/jenkins-docker/slave-bazel-docker/Dockerfile
index 997fe2b..57c6e0c 100644
--- a/jenkins-docker/slave-bazel-docker/Dockerfile
+++ b/jenkins-docker/slave-bazel-docker/Dockerfile
@@ -7,3 +7,8 @@
stable" && \
apt-get update && \
apt-get install -y docker-ce git-
+
+RUN curl -L \
+ "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" \
+ -o /usr/local/bin/docker-compose && \
+ chmod +x /usr/local/bin/docker-compose
diff --git a/jenkins-docker/slave-bazel-docker/Makefile b/jenkins-docker/slave-bazel-docker/Makefile
index e2ee20e..ab3bc4d 100644
--- a/jenkins-docker/slave-bazel-docker/Makefile
+++ b/jenkins-docker/slave-bazel-docker/Makefile
@@ -1,7 +1,8 @@
include ../chrome-vers.mk
NO_CACHE=false
-IMAGE=gerritforge/gerrit-ci-slave-bazel-docker:debian-buster
+PREFIX=gerritforge
+IMAGE=${PREFIX}/gerrit-ci-slave-bazel-docker:debian-buster
# Targets
diff --git a/jenkins-docker/slave-bazel-nocache/Dockerfile b/jenkins-docker/slave-bazel-nocache/Dockerfile
index de5b558..99ee474 100644
--- a/jenkins-docker/slave-bazel-nocache/Dockerfile
+++ b/jenkins-docker/slave-bazel-nocache/Dockerfile
@@ -12,23 +12,16 @@
bash -c 'bazel version'
# Install Buildifier
-RUN cd /tmp && \
- bash -c 'git clone https://github.com/bazelbuild/buildtools.git && \
- cd buildtools && \
- git checkout ${BUILDIFIER_VER} && \
- bazel build --config=release //buildifier && \
- cp bazel-bin/buildifier/linux_amd64_stripped/buildifier /usr/bin && \
- rm -rf /tmp/buildtools && \
- buildifier --version'
+RUN wget -O /tmp/buildifier-${BUILDIFIER_VER}-linux-amd64 https://github.com/bazelbuild/buildtools/releases/download/${BUILDIFIER_VER}/buildifier-linux-amd64 && \
+ chmod a+x /tmp/buildifier-${BUILDIFIER_VER}-linux-amd64 && \
+ cp /tmp/buildifier-${BUILDIFIER_VER}-linux-amd64 /usr/bin/buildifier && \
+ rm -rf /tmp/buildifier-${BUILDIFIER_VER}-linux-amd64 && \
+ buildifier --version
# Install Bazelisk
-RUN cd /tmp && \
- git clone https://github.com/bazelbuild/bazelisk.git && \
- cd bazelisk && \
- git checkout v${BAZELISK_VER} && \
- rm -f .bazelversion && \
- bazel build --stamp --workspace_status_command=`pwd`/stamp.sh --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64 //:bazelisk && \
- cp bazel-bin/linux_amd64_pure_stripped/bazelisk /usr/bin && \
- rm -rf /tmp/bazelisk
+RUN wget -O /tmp/bazelisk-${BAZELISK_VER}-linux-amd64 https://github.com/bazelbuild/bazelisk/releases/download/v${BAZELISK_VER}/bazelisk-linux-amd64 && \
+ chmod a+x /tmp/bazelisk-${BAZELISK_VER}-linux-amd64 && \
+ cp /tmp/bazelisk-${BAZELISK_VER}-linux-amd64 /usr/bin/bazelisk && \
+ rm -rf /tmp/bazelisk-${BAZELISK_VER}-linux-amd64
USER root
diff --git a/jenkins-docker/slave-bazel-nocache/Makefile b/jenkins-docker/slave-bazel-nocache/Makefile
index 30b09f3..5e37d47 100644
--- a/jenkins-docker/slave-bazel-nocache/Makefile
+++ b/jenkins-docker/slave-bazel-nocache/Makefile
@@ -1,9 +1,10 @@
include ../chrome-vers.mk
NO_CACHE=false
-IMAGE=gerritforge/gerrit-ci-slave-bazel:debian-buster-nocache
-BAZEL_VER=3.7.0
-BUILDIFIER_VER=3.5.0
+PREFIX=gerritforge
+IMAGE=${PREFIX}/gerrit-ci-slave-bazel:debian-buster-nocache
+BAZEL_VER=4.0.0
+BUILDIFIER_VER=4.0.0
BAZELISK_VER=1.7.4
NODE_VER=10
WCT_VER=6.8.0
diff --git a/jenkins-docker/slave-bazel/Makefile b/jenkins-docker/slave-bazel/Makefile
index db0d52f..0da3add 100644
--- a/jenkins-docker/slave-bazel/Makefile
+++ b/jenkins-docker/slave-bazel/Makefile
@@ -1,7 +1,8 @@
include ../chrome-vers.mk
NO_CACHE=false
-IMAGE=gerritforge/gerrit-ci-slave-bazel:debian-buster
+PREFIX=gerritforge
+IMAGE=${PREFIX}/gerrit-ci-slave-bazel:debian-buster
# Targets
diff --git a/jenkins-docker/slave-chrome/Makefile b/jenkins-docker/slave-chrome/Makefile
index 127fcb3..e31e9e4 100644
--- a/jenkins-docker/slave-chrome/Makefile
+++ b/jenkins-docker/slave-chrome/Makefile
@@ -2,7 +2,8 @@
NO_CACHE ?= false
DEBIAN_VER ?= buster
-IMAGE=gerritforge/gerrit-ci-slave-chrome:$(DEBIAN_VER)
+PREFIX=gerritforge
+IMAGE=${PREFIX}/gerrit-ci-slave-chrome:$(DEBIAN_VER)
build: Makefile Dockerfile
@for chrome_ver in $(CHROME_VERS); \
diff --git a/jenkins-docker/slave-debian/Dockerfile b/jenkins-docker/slave-debian/Dockerfile
index 51ac131..a85bb6e 100644
--- a/jenkins-docker/slave-debian/Dockerfile
+++ b/jenkins-docker/slave-debian/Dockerfile
@@ -17,6 +17,7 @@
python-pip \
python-yaml \
python-jenkinsapi \
+ python3-distutils \
ant \
autoconf \
automake \
@@ -71,10 +72,10 @@
RUN apt-get update && apt-get install -y lib32stdc++6 lib32z1
# Install OpenJDK 8 for legacy stable Gerrit versions builds, releases and E2E tests
-# TODO: to be removed when Gerrit v3.2 will be EOL
+# TODO: to be removed when Gerrit v3.4 will be EOL
RUN wget -qO - https://adoptopenjdk.jfrog.io/adoptopenjdk/api/gpg/key/public | apt-key add - && \
add-apt-repository --yes https://adoptopenjdk.jfrog.io/adoptopenjdk/deb/ && \
- apt-get update && apt-get install adoptopenjdk-8-hotspot && \
+ apt-get update && apt-get install -y adoptopenjdk-8-hotspot && \
ln -sf /usr/lib/jvm/adoptopenjdk-8-hotspot-amd64 /usr/lib/jvm/java-8-openjdk-amd64 && \
ln -sf /usr/lib/jvm/adoptopenjdk-8-hotspot-amd64/bin/java /etc/alternatives/java
diff --git a/jenkins-docker/slave-debian/Makefile b/jenkins-docker/slave-debian/Makefile
index d89181d..3089f42 100644
--- a/jenkins-docker/slave-debian/Makefile
+++ b/jenkins-docker/slave-debian/Makefile
@@ -1,5 +1,6 @@
NO_CACHE ?= false
-IMAGE=gerritforge/gerrit-ci-slave-debian
+PREFIX=gerritforge
+IMAGE=${PREFIX}/gerrit-ci-slave-debian
MAVEN_VER=3.6.3
GIT_VER=2.26.2
DEBIAN_VER=buster
diff --git a/jenkins-docker/slave-jekyll/Makefile b/jenkins-docker/slave-jekyll/Makefile
index e1948e3..7089017 100644
--- a/jenkins-docker/slave-jekyll/Makefile
+++ b/jenkins-docker/slave-jekyll/Makefile
@@ -1,5 +1,6 @@
NO_CACHE ?= false
-IMAGE=gerritforge/gerrit-ci-slave-jekyll
+PREFIX=gerritforge
+IMAGE=${PREFIX}/gerrit-ci-slave-jekyll
build: Dockerfile Makefile
docker build --no-cache=$(NO_CACHE) -f Dockerfile -t $(IMAGE) .
diff --git a/jenkins-docker/slave-mvn/Makefile b/jenkins-docker/slave-mvn/Makefile
index 03fc47b..217c2c3 100644
--- a/jenkins-docker/slave-mvn/Makefile
+++ b/jenkins-docker/slave-mvn/Makefile
@@ -1,5 +1,6 @@
NO_CACHE=false
-IMAGE=gerritforge/gerrit-ci-slave-mvn
+PREFIX=gerritforge
+IMAGE=${PREFIX}/gerrit-ci-slave-mvn
# Targets
diff --git a/jenkins-docker/slave-node-wct/Dockerfile b/jenkins-docker/slave-node-wct/Dockerfile
index 4f755d8..b5274f9 100644
--- a/jenkins-docker/slave-node-wct/Dockerfile
+++ b/jenkins-docker/slave-node-wct/Dockerfile
@@ -18,5 +18,6 @@
eslint-plugin-promise@"=4.0.1" \
typescript \
fried-twinkie@"^0.2.2" \
- polylint polymer-cli
+ polylint polymer-cli \
+ @babel/runtime@"^7.13.9"
RUN npm install --unsafe-perm -g 'web-component-tester@$WCT_VER'
diff --git a/jenkins-docker/slave-node-wct/Makefile b/jenkins-docker/slave-node-wct/Makefile
index 2fff934..46d3e91 100644
--- a/jenkins-docker/slave-node-wct/Makefile
+++ b/jenkins-docker/slave-node-wct/Makefile
@@ -3,7 +3,8 @@
NO_CACHE ?= false
NODE_VER ?= 10
WCT_VER ?= 6.8.0
-IMAGE=gerritforge/gerrit-ci-slave-node-wct:$(NODE_VER)-$(WCT_VER)
+PREFIX=gerritforge
+IMAGE=${PREFIX}/gerrit-ci-slave-node-wct:$(NODE_VER)-$(WCT_VER)
build: Dockerfile Makefile
@for chrome_ver in $(CHROME_VERS); \
diff --git a/jenkins-docker/slave-python3/Makefile b/jenkins-docker/slave-python3/Makefile
index 5a717ef..0bceabc 100644
--- a/jenkins-docker/slave-python3/Makefile
+++ b/jenkins-docker/slave-python3/Makefile
@@ -1,6 +1,7 @@
NO_CACHE ?= false
DEBIAN_VER ?= buster
-IMAGE=gerritforge/gerrit-ci-slave-python3:$(DEBIAN_VER)
+PREFIX=gerritforge
+IMAGE=${PREFIX}/gerrit-ci-slave-python3:$(DEBIAN_VER)
build: Dockerfile Makefile
cat Dockerfile | DEBIAN_VER=$(DEBIAN_VER) envsubst > Dockerfile-$(DEBIAN_VER)
diff --git a/jenkins-docker/slave-release/Dockerfile b/jenkins-docker/slave-release/Dockerfile
index 2eb3e65..152e63e 100644
--- a/jenkins-docker/slave-release/Dockerfile
+++ b/jenkins-docker/slave-release/Dockerfile
@@ -10,6 +10,10 @@
COPY gerrit-release.sh /usr/local/bin
-ENTRYPOINT /bin/bash
+ENTRYPOINT [ "/usr/local/bin/gerrit-release.sh" ]
+
+CMD [ "--help" ]
USER jenkins
+
+WORKDIR /home/jenkins
diff --git a/jenkins-docker/slave-release/Makefile b/jenkins-docker/slave-release/Makefile
index 42dd7e4..2d0a20b 100644
--- a/jenkins-docker/slave-release/Makefile
+++ b/jenkins-docker/slave-release/Makefile
@@ -1,24 +1,53 @@
NO_CACHE=true
-IMAGE=gerritforge/gerrit-ci-slave-bazel:debian-buster-nocache-80.0.3987.149
+PREFIX=gerritforge
+IMAGE=${PREFIX}/gerrit-release-bazel:debian-buster
CONTAINER=release
-build:
+######################################################
+# Customize the parameters below for running a release
+######################################################
+
+# Branch from where the release will be cut
+BRANCH=stable-3.3
+
+# Release number
+RELEASE=3.3.0
+
+# Next SNAPSHOT after the release
+SNAPSHOT=3.3.1-SNAPSHOT
+
+# Release manager's GNUPG keyring
+GNUPG=${HOME}/.gnupg
+
+# Release manager's local git config for authentication
+GIT_CONFIG=${HOME}/.gitconfig
+
+# Release manager's local git cookies file
+GIT_COOKIES=${HOME}/.gitcookies
+
+######################################################
+# Targets for building the image and making a release
+######################################################
+
+run:
+ docker run --rm -it --name ${CONTAINER} ${IMAGE} ${BRANCH} ${RELEASE} ${SNAPSHOT} \
+ -u jenkins \
+ -v `pwd`/.gerritcodereview:/home/jenkins/.gerritcodereview \
+ -v ${HOME}/.m2:/home/jenkins/.m2 \
+ -v ${GNUPG}:/home/jenkins/.gnupg \
+ -v ${GIT_CONFIG}:/home/jenkins/.gitconfig.template \
+ -v ${GIT_COOKIES}:/home/jenkins/.gitcookies
+
+pull:
+ docker pull gerritforge/gerrit-ci-slave-bazel:debian-buster-nocache-80.0.3987.149
+
+build: pull
docker build --no-cache=$(NO_CACHE) -t ${IMAGE} .
-# Dockerfile shows from where to manually execute the release script once in.
-# E.g.: /usr/local/bin/gerrit-release.sh stable-3.3 3.3.0-rc3 3.3.0-SNAPSHOT
-run:
- docker run -it --name ${CONTAINER} ${IMAGE}
-
-rerun: clean-container run
-
-publish:
+publish: build
docker push ${IMAGE}
-clean-image:
+clean:
docker rmi -f ${IMAGE}
-clean-container:
- docker rm -f ${CONTAINER}
-
-.PHONY: clean-image clean-container publish
+.PHONY: run pull build publish clean
diff --git a/jenkins-docker/slave-release/gerrit-release.sh b/jenkins-docker/slave-release/gerrit-release.sh
index 1a9fa7e..8c49e40 100755
--- a/jenkins-docker/slave-release/gerrit-release.sh
+++ b/jenkins-docker/slave-release/gerrit-release.sh
@@ -1,6 +1,6 @@
#!/bin/bash -e
-if [ "$1" == "" ] || [ "$2" == "" ]
+if [ "$1" == "--help" ] || [ "$1" == "" ] || [ "$2" == "" ] || [ "$3" == "" ]
then
echo "Gerrit Code Review - release automation script"
echo "----------------------------------------------"
@@ -23,7 +23,13 @@
rm -Rf gerrit
fi
+if [ -f $HOME/.gitconfig.template ]
+then
+ cp $HOME/.gitconfig.template $HOME/.gitconfig
+fi
+
echo "Cloning and building Gerrit Code Review on branch $branch ..."
+git config --global credential.helper cache
git clone https://gerrit.googlesource.com/gerrit && (cd gerrit && f=$(git rev-parse --git-dir)/hooks/commit-msg ; curl -Lo "$f" https://gerrit-review.googlesource.com/tools/hooks/commit-msg ; chmod +x "$f")
pushd gerrit
@@ -85,7 +91,7 @@
echo "Pushing gerrit documentation to gerrit-documentation ..."
unzip searchfree.zip
pushd Documentation
-version_no_rc=$(echo "%version" | cut -d '-' -f 1)
+version_no_rc=$(echo "$version" | cut -d '-' -f 1)
gsutil cp -r . gs://gerrit-documentation/Documentation/"$version_no_rc"
popd
diff --git a/jenkins-docker/slave-sbt/Makefile b/jenkins-docker/slave-sbt/Makefile
index ceb6637..d2a811e 100644
--- a/jenkins-docker/slave-sbt/Makefile
+++ b/jenkins-docker/slave-sbt/Makefile
@@ -1,5 +1,6 @@
NO_CACHE=false
-IMAGE=gerritforge/gerrit-ci-slave-bazel-sbt
+PREFIX=gerritforge
+IMAGE=${PREFIX}/gerrit-ci-slave-bazel-sbt
# Targets
diff --git a/jenkins/gatling-test-pipeline.groovy b/jenkins/gatling-test-pipeline.groovy
new file mode 100644
index 0000000..06876f8
--- /dev/null
+++ b/jenkins/gatling-test-pipeline.groovy
@@ -0,0 +1,221 @@
+def accountCookie = ''
+def xsrfToken = ''
+def epochTime = new Date().getTime()
+
+pipeline {
+ agent { label 'aws' }
+
+ parameters {
+ string(name: 'AWS_PREFIX', defaultValue:"jenkins", description: 'A string to prefix stacks and resources with')
+ string(name: 'AWS_REGION', defaultValue:"us-east-1", description: 'Which region to deploy to')
+
+ string(name: 'GERRIT_VERSION', defaultValue:"3.4", description: 'The gerrit version under test')
+ string(name: 'GERRIT_PATCH', defaultValue:"0-rc5", description: 'The gerrit version patch under test')
+ string(name: 'GERRIT_WAR_URL', defaultValue:"https://gerrit-ci.gerritforge.com/job/Gerrit-bazel-java11-stable-3.4/lastSuccessfulBuild/artifact/gerrit/bazel-bin/release.war", description: 'The gerrit.war URL to use as override of the gerrit version under test')
+
+ string(name: 'HOSTED_ZONE_NAME', defaultValue: "gerritforgeaws.com", description: 'Name of the hosted zone')
+ string(name: 'CLUSTER_INSTANCE_TYPE', defaultValue: 'm4.xlarge', description:'The EC2 instance Type used to run the cluster')
+
+ string(name: 'DOCKER_REGISTRY_URI', defaultValue: '117385740707.dkr.ecr.$(AWS_REGION).amazonaws.com', description: 'URI of the Docker registry')
+ string(name: 'SSL_CERTIFICATE_ARN', defaultValue: "arn:aws:acm:us-east-1:117385740707:certificate/33e2c235-a4d1-42b7-b866-18d8d744975c", description: 'ARN of the wildcard SSL Certificate')
+
+ string(name: 'GERRIT_VOLUME_SNAPSHOT_ID', defaultValue: "snap-01c12c75ead9e9cd4", description: 'Id of the EBS volume snapshot')
+
+ string(name: 'METRICS_CLOUDWATCH_NAMESPACE', defaultValue: 'jenkins', description: 'The CloudWatch namespace for Gerrit metrics')
+ string(name: 'BASE_SUBDOMAIN', defaultValue: 'gerrit-demo', description: 'Name of the master sub domain')
+ string(name: 'GERRIT_KEY_PREFIX', defaultValue: 'gerrit_secret', description: 'Secrets prefix')
+
+ string(name: 'GERRIT_SSH_USERNAME', defaultValue: 'gerritadmin', description: 'Gerrit SSH username')
+ string(name: 'GERRIT_SSH_PORT', defaultValue: '29418', description: 'Gerrit SSH port')
+
+ string(name: 'GERRIT_HTTP_SCHEMA', defaultValue: 'https', description: 'Gerrit HTTP schema')
+ string(name: 'GIT_HTTP_USERNAME', defaultValue: '', description: 'Username for Git/HTTP testing, use vault by default')
+ password(name: 'GIT_HTTP_PASSWORD', defaultValue: '', description: 'Password for Git/HTTP testing, use vault by default')
+
+ string(name: 'S3_EXPORT_LOGS_BUCKET_NAME', defaultValue: 'gerritforge-export-logs', description: 'S3 bucket to export logs to')
+
+ string(name: 'GERRIT_PROJECT', defaultValue: 'load-test', description: 'Gerrit project for load test')
+ string(name: 'NUM_USERS', defaultValue: '10', description: 'Number of concurrent user sessions')
+ string(name: 'DURATION', defaultValue: '2 minutes', description: 'Total duration of the test')
+ }
+
+ environment {
+ DOCKER_HOST = """${sh(
+ returnStdout: true,
+ script: '/sbin/ip route|awk \'/default/ {print "tcp://"\$3":2375"}\''
+ )}"""
+ SUBDOMAIN = String.format("%s-%s.%s", "jenkins", epochTime, "${params.BASE_SUBDOMAIN}")
+ BASE_URL = String.format("%s.%s", SUBDOMAIN, "${params.HOSTED_ZONE_NAME}")
+ GERRIT_HTTP_URL = String.format("%s://%s", "${params.GERRIT_HTTP_SCHEMA}", BASE_URL)
+ GERRIT_SSH_URL = String.format("ssh://%s@%s:%s", "${params.GERRIT_SSH_USERNAME}", BASE_URL, "${params.GERRIT_SSH_PORT}")
+ }
+
+ stages{
+ stage("Setup single-primary aws stack") {
+ steps {
+ withCredentials([usernamePassword(usernameVariable: "GS_GIT_USER", passwordVariable: "GS_GIT_PASS", credentialsId: "gerrit.googlesource.com")]) {
+ sh 'echo "machine gerrit.googlesource.com login $GS_GIT_USER password $GS_GIT_PASS">> ~/.netrc'
+ sh 'chmod 600 ~/.netrc'
+ sh 'rm -rf aws-gerrit'
+ sh "git clone -b master https://gerrit.googlesource.com/aws-gerrit"
+ sh "cd aws-gerrit && git fetch origin master && git config user.name jenkins && git config user.email jenkins@gerritforge.com && git merge FETCH_HEAD"
+ }
+ dir ('aws-gerrit/gerrit/etc') {
+ script {
+ def gerritConfig = readFile(file:"gerrit.config.template")
+ gerritConfig = gerritConfig.replace("type = ldap","type = DEVELOPMENT_BECOME_ANY_ACCOUNT")
+ gerritConfig = gerritConfig.replace("smtpUser = {{ SMTP_USER }}\n enable = true","smtpUser = {{ SMTP_USER }}\n enable = false")
+
+ writeFile(file:"gerrit.config.template", text: gerritConfig)
+ }
+ }
+ withCredentials([usernamePassword(usernameVariable: "AWS_ACCESS_KEY_ID",
+ passwordVariable: "AWS_SECRET_ACCESS_KEY",
+ credentialsId: "aws-credentials-id")]) {
+ dir ('aws-gerrit/single-primary') {
+ script {
+ def setupData = readFile(file:"setup.env.template")
+ setupData = resolveParameter(setupData, "HOSTED_ZONE_NAME", "${params.HOSTED_ZONE_NAME}")
+ setupData = resolveParameter(setupData, "CLUSTER_INSTANCE_TYPE", "${params.CLUSTER_INSTANCE_TYPE}")
+ setupData = resolveParameter(setupData, "DOCKER_REGISTRY_URI", "${params.DOCKER_REGISTRY_URI}")
+ setupData = resolveParameter(setupData, "SSL_CERTIFICATE_ARN", "${params.SSL_CERTIFICATE_ARN}")
+
+ setupData = resolveParameter(setupData, "METRICS_CLOUDWATCH_NAMESPACE", "${params.METRICS_CLOUDWATCH_NAMESPACE}")
+ setupData = resolveParameter(setupData, 'SUBDOMAIN', "${env.SUBDOMAIN}")
+
+ setupData = setupData + "\nGERRIT_KEY_PREFIX:= ${params.GERRIT_KEY_PREFIX}"
+ setupData = setupData + "\nGERRIT_VOLUME_SNAPSHOT_ID:= ${params.GERRIT_VOLUME_SNAPSHOT_ID}"
+
+ writeFile(file:"setup.env", text: setupData)
+ }
+ sh 'echo "*** Computed values:"'
+ sh 'echo "* Subdomain: $SUBDOMAIN"'
+ sh 'echo "* Base URL: $BASE_URL"'
+ sh 'echo "* Gerrit HTTP URL: $GERRIT_HTTP_URL"'
+ sh 'echo "* Gerrit SSH URL: $GERRIT_SSH_URL"'
+ sh 'echo "Docker host: $DOCKER_HOST"'
+ sh "make AWS_REGION=${params.AWS_REGION} AWS_PREFIX=${params.AWS_PREFIX} GERRIT_VERSION=${params.GERRIT_VERSION} GERRIT_WAR_URL=${params.GERRIT_WAR_URL} GERRIT_PATCH=${params.GERRIT_PATCH} create-all"
+ }
+ }
+ }
+ }
+ stage('Extract Gatling test user credentials from Gerrit') {
+ steps {
+ retry(50) {
+ sleep(10)
+ sh "curl --fail -L -I '${env.GERRIT_HTTP_URL}/config/server/healthcheck~status' 2>/dev/null"
+ sh "curl -L -c cookies -i -X POST '${env.GERRIT_HTTP_URL}/login/%2Fq%2Fstatus%3Aopen%2B-is%3Awip?account_id=1000000'"
+ }
+ script {
+ def cookies = readFile(file:"cookies")
+ def cookiesMap = cookies
+ .split('\n')
+ .findAll{it.contains('GerritAccount') || it.contains('XSRF_TOKEN')}
+ .inject([:]) { map, token ->
+ def tokens = token.split('\t')
+ map[tokens[5].trim()] = tokens[6].trim()
+ map
+ }
+
+ accountCookie = cookiesMap['GerritAccount']
+ xsrfToken = cookiesMap['XSRF_TOKEN']
+ }
+ }
+ }
+ stage('Pull newest Gatling tests docker image') {
+ steps {
+ sh 'docker pull gerritforge/gatling-sbt-gerrit-test'
+ }
+ }
+
+ stage('Run Gatling tests') {
+ steps {
+ withCredentials([usernamePassword(usernameVariable: "DEFAULT_GIT_HTTP_USERNAME",
+ passwordVariable: "DEFAULT_GIT_HTTP_PASSWORD",
+ credentialsId: "gatlingHttp")]) {
+ script {
+
+ def gitHttpUsername = ("${params.GIT_HTTP_USERNAME}"?.trim()) ?: "${env.DEFAULT_GIT_HTTP_USERNAME}"
+ def gitHttpPassword = ("${params.GIT_HTTP_PASSWORD}"?.trim()) ?: "${env.DEFAULT_GIT_HTTP_PASSWORD}"
+
+ writeFile(file: "simulation.env", text: """
+ GERRIT_HTTP_URL=${env.GERRIT_HTTP_URL}
+ GERRIT_SSH_URL=${env.GERRIT_SSH_URL}
+ ACCOUNT_COOKIE=${accountCookie}
+ GIT_HTTP_USERNAME=${gitHttpUsername}
+ GIT_HTTP_PASSWORD=${gitHttpPassword}
+ XSRF_TOKEN=${xsrfToken}
+ GERRIT_PROJECT=${params.GERRIT_PROJECT}
+ NUM_USERS=${params.NUM_USERS}
+ DURATION=${params.DURATION}
+ GIT_SSH_COMMAND="ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no"
+ """)
+ }
+ sh "mkdir -p ${WORKSPACE}/results"
+ // If Jenkins agent uses Docker remote server mounting local directory will mount directory
+ // from Docker server host not agent host. Gatling reports will not be visible in build workspace.
+ // Use Docker volume to avoid this situation.
+ sh "docker volume create gatling-results"
+ script {
+ for (simulation in ["GerritGitSimulation", "GerritRestSimulation"]) {
+ sh """\
+ docker run --rm --env-file simulation.env -v gatling-results:/opt/gatling/results \
+ gerritforge/gatling-sbt-gerrit-test -s gerritforge.${simulation}
+ """
+ }
+ }
+ //Copy data from Docker volume to Jenkins build workspace
+ sh "docker create -v gatling-results:/data --name gatling-results-container busybox true"
+ sh "docker cp gatling-results-container:/data/. ${WORKSPACE}/results/"
+ // Clean up
+ sh "docker rm gatling-results-container"
+ sh "docker volume rm gatling-results"
+
+ gatlingArchive()
+ }
+ }
+ }
+ stage("Export Cloudwatch logs to S3") {
+ steps {
+ withCredentials([usernamePassword(usernameVariable: "AWS_ACCESS_KEY_ID",
+ passwordVariable: "AWS_SECRET_ACCESS_KEY",
+ credentialsId: "aws-credentials-id")]) {
+ dir ('aws-gerrit/single-primary') {
+ sh "make AWS_REGION=${params.AWS_REGION} AWS_PREFIX=${params.AWS_PREFIX} EXPORT_FROM_MILLIS=${epochTime} S3_EXPORT_LOGS_BUCKET_NAME=${params.S3_EXPORT_LOGS_BUCKET_NAME} export-logs"
+ }
+ }
+ }
+ }
+ stage('Check tests results') {
+ steps {
+ script {
+ def failed_tests = sh(
+ returnStdout: true,
+ script: "for i in `find ${WORKSPACE} -name \"global_stats.json\"`; do cat \$i | jq '.numberOfRequests.ko'| grep -v '0' || true; done;"
+ )
+ if (failed_tests.trim()) {
+ error("Setting build as failed because some gatling tests were not OK")
+ }
+ }
+ }
+ }
+ }
+ post {
+ cleanup {
+ withCredentials([usernamePassword(usernameVariable: "AWS_ACCESS_KEY_ID",
+ passwordVariable: "AWS_SECRET_ACCESS_KEY",
+ credentialsId: "aws-credentials-id")]) {
+ dir ('aws-gerrit/single-primary') {
+ sh "make AWS_REGION=${params.AWS_REGION} AWS_PREFIX=${params.AWS_PREFIX} delete-all"
+ }
+ }
+ }
+ }
+}
+
+def resolveParameter(String text, String paramName, String paramValue) {
+ return text.split('\n').collect { l ->
+ def targetLine = l.trim().startsWith(paramName)
+ targetLine ? "${paramName}:=${paramValue}" : l
+ }.join('\n')
+}
\ No newline at end of file
diff --git a/jenkins/gatlingTestPipeline.yaml b/jenkins/gatling-test-pipeline.yaml
similarity index 84%
rename from jenkins/gatlingTestPipeline.yaml
rename to jenkins/gatling-test-pipeline.yaml
index 21fd1de..861ac31 100644
--- a/jenkins/gatlingTestPipeline.yaml
+++ b/jenkins/gatling-test-pipeline.yaml
@@ -7,6 +7,6 @@
url: https://gerrit.googlesource.com/a/aws-gerrit
branches:
- master
- dsl: !include-raw: gatlingTestPipeline.groovy
+ dsl: !include-raw: gatling-test-pipeline.groovy
triggers:
- timed: "@midnight"
diff --git a/jenkins/gatlingTestPipeline.groovy b/jenkins/gatlingTestPipeline.groovy
deleted file mode 100644
index a743281..0000000
--- a/jenkins/gatlingTestPipeline.groovy
+++ /dev/null
@@ -1,160 +0,0 @@
-def accountCookie = ''
-def xsrfToken = ''
-
-pipeline {
- agent { label 'aws' }
-
- parameters {
- string(name: 'AWS_PREFIX', defaultValue:"jenkins", description: 'A string to prefix stacks and resources with')
- string(name: 'AWS_REGION', defaultValue:"us-east-1", description: 'Which region to deploy to')
-
- string(name: 'HOSTED_ZONE_NAME', description: 'Name of the hosted zone')
- string(name: 'CLUSTER_INSTANCE_TYPE', defaultValue: 'm4.xlarge', description:'The EC2 instance Type used to run the cluster')
-
- string(name: 'DOCKER_REGISTRY_URI', description: 'URI of the Docker registry')
- string(name: 'SSL_CERTIFICATE_ARN', description: 'ARN of the wildcard SSL Certificate')
-
- string(name: 'GERRIT_VOLUME_SHAPSHOT_ID', description: 'Id of the EBS volume snapshot')
-
- string(name: 'METRICS_CLOUDWATCH_NAMESPACE', defaultValue: 'jenkins', description: 'The CloudWatch namespace for Gerrit metrics')
- string(name: 'SUBDOMAIN', defaultValue: '$(AWS_PREFIX)-master-demo', description: 'Name of the master sub domain')
- string(name: 'GERRIT_KEY_PREFIX', defaultValue: 'gerrit_secret', description: 'Secrets prefix')
-
- string(name: 'GERRIT_HTTP_URL', description: 'Gerrit GUI URL')
- string(name: 'GERRIT_SSH_URL', description: 'Gerrit SSH URL')
- string(name: 'GIT_HTTP_USERNAME', description: 'Username for Git/HTTP testing')
- string(name: 'GIT_HTTP_PASSWORD', description: 'Password for Git/HTTP testing')
- string(name: 'GERRIT_PROJECT', defaultValue: 'load-test', description: 'Gerrit project for load test')
- string(name: 'NUM_USERS', defaultValue: '10', description: 'Number of concurrent user sessions')
- string(name: 'DURATION', defaultValue: '2 minutes', description: 'Total duration of the test')
- }
- stages{
- stage("Setup single-master aws stack") {
- steps {
- withCredentials([usernamePassword(usernameVariable: "GS_GIT_USER", passwordVariable: "GS_GIT_PASS", credentialsId: env.GERRIT_CREDENTIALS_ID)]) {
- sh 'echo "machine gerrit.googlesource.com login $GS_GIT_USER password $GS_GIT_PASS">> ~/.netrc'
- sh 'chmod 600 ~/.netrc'
- sh 'rm -rf aws-gerrit'
- sh "git clone -b master https://gerrit.googlesource.com/aws-gerrit"
- sh "cd aws-gerrit && git fetch origin master && git config user.name jenkins && git config user.email jenkins@gerritforge.com && git merge FETCH_HEAD"
- }
- dir ('aws-gerrit/gerrit/etc') {
- script {
- def gerritConfig = readFile(file:"gerrit.config.template")
- gerritConfig = gerritConfig.replace("type = ldap","type = DEVELOPMENT_BECOME_ANY_ACCOUNT")
- gerritConfig = gerritConfig.replace("smtpUser = {{ SMTP_USER }}\n enable = true","smtpUser = {{ SMTP_USER }}\n enable = false")
-
- writeFile(file:"gerrit.config.template", text: gerritConfig)
- }
- }
- withCredentials([usernamePassword(usernameVariable: "AWS_ACCESS_KEY_ID",
- passwordVariable: "AWS_SECRET_ACCESS_KEY",
- credentialsId: "aws-credentials-id")]) {
- dir ('aws-gerrit/single-master') {
- script {
- def setupData = readFile(file:"setup.env.template")
- setupData = resolveParameter(setupData, "HOSTED_ZONE_NAME", HOSTED_ZONE_NAME)
- setupData = resolveParameter(setupData, "CLUSTER_INSTANCE_TYPE", CLUSTER_INSTANCE_TYPE)
- setupData = resolveParameter(setupData, "DOCKER_REGISTRY_URI", DOCKER_REGISTRY_URI)
- setupData = resolveParameter(setupData, "SSL_CERTIFICATE_ARN", SSL_CERTIFICATE_ARN)
-
- setupData = resolveParameter(setupData, "METRICS_CLOUDWATCH_NAMESPACE",METRICS_CLOUDWATCH_NAMESPACE)
- setupData = resolveParameter(setupData, 'SUBDOMAIN', SUBDOMAIN)
-
- setupData = setupData + "\nGERRIT_KEY_PREFIX:= ${GERRIT_KEY_PREFIX}"
- setupData = setupData + "\nGERRIT_VOLUME_SNAPSHOT_ID:= ${GERRIT_VOLUME_SHAPSHOT_ID}"
-
- writeFile(file:"setup.env", text: setupData)
- }
- sh "make AWS_REGION=${AWS_REGION} AWS_PREFIX=${AWS_PREFIX} create-all"
- }
- }
- }
- }
- stage('Extract Gatling test user credentials from Gerrit') {
- steps {
- retry(50) {
- sleep(10)
- sh "curl --fail -L -I '${GERRIT_HTTP_URL}' 2>/dev/null"
- }
- sh "curl -L -c cookies -i -X POST '${GERRIT_HTTP_URL}/login/%2Fq%2Fstatus%3Aopen%2B-is%3Awip?account_id=1000000'"
- script {
- def cookies = readFile(file:"cookies")
- def cookiesMap = cookies
- .split('\n')
- .findAll{it.contains('GerritAccount') || it.contains('XSRF_TOKEN')}
- .inject([:]) { map, token ->
- def tokens = token.split('\t')
- map[tokens[5].trim()] = tokens[6].trim()
- map
- }
-
- accountCookie = cookiesMap['GerritAccount']
- xsrfToken = cookiesMap['XSRF_TOKEN']
- }
- }
- }
- stage('Pull newest Gatling tests docker image') {
- steps {
- sh 'docker pull gerritforge/gatling-sbt-gerrit-test'
- }
- }
-
- stage('Run Gatling tests') {
- steps {
- script {
- writeFile(file:"simulation.env", text: """
- GERRIT_HTTP_URL=${GERRIT_HTTP_URL}
- GERRIT_SSH_URL=${GERRIT_SSH_URL}
- ACCOUNT_COOKIE=${accountCookie}
- GIT_HTTP_USERNAME=${GIT_HTTP_USERNAME}
- GIT_HTTP_PASSWORD=${GIT_HTTP_PASSWORD}
- XSRF_TOKEN=${xsrfToken}
- GERRIT_PROJECT=${GERRIT_PROJECT}
- GIT_SSH_COMMAND="ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no"
- """)
- }
- sh "mkdir -p ${WORKSPACE}/results"
- // If Jenkins agent uses Docker remote server mounting local directory will mount directory
- // from Docker server host not agent host. Gatling reports will not be visible in build workspace.
- // Use Docker volume to avoid this situation.
- sh "docker volume create gatling-results"
- script {
- for (simulation in ["GerritGitSimulation", "GerritRestSimulation"]) {
- sh """\
- docker run --rm --env-file simulation.env -v gatling-results:/opt/gatling/results \
- gerritforge/gatling-sbt-gerrit-test -s gerritforge.${simulation}
- """
- }
- }
- //Copy data from Docker volume to Jenkins build workspace
- sh "docker create -v gatling-results:/data --name gatling-results-container busybox true"
- sh "docker cp gatling-results-container:/data/. ${WORKSPACE}/results/"
- // Clean up
- sh "docker rm gatling-results-container"
- sh "docker volume rm gatling-results"
-
- gatlingArchive()
- }
- }
-
- }
- post {
- cleanup {
- withCredentials([usernamePassword(usernameVariable: "AWS_ACCESS_KEY_ID",
- passwordVariable: "AWS_SECRET_ACCESS_KEY",
- credentialsId: "aws-credentials-id")]) {
- dir ('aws-gerrit/single-master') {
- sh "make AWS_REGION=${AWS_REGION} AWS_PREFIX=${AWS_PREFIX} delete-all"
- }
- }
- }
- }
-}
-
-def resolveParameter(String text, String paramName, String paramValue) {
- return text.split('\n').collect { l ->
- def targetLine = l.trim().startsWith(paramName)
- targetLine ? "${paramName}:=${paramValue}" : l
- }.join('\n')
-}
\ No newline at end of file
diff --git a/jenkins/gerrit-bazel-build.sh b/jenkins/gerrit-bazel-build.sh
index 3a2ef97..d02c40b 100644
--- a/jenkins/gerrit-bazel-build.sh
+++ b/jenkins/gerrit-bazel-build.sh
@@ -4,15 +4,24 @@
cd gerrit
-if [git show --diff-filter=AM --name-only --pretty="" HEAD | grep -q .bazelversion]
+echo "Build with mode=$MODE"
+echo '----------------------------------------------'
+
+if git show --diff-filter=AM --name-only --pretty="" HEAD | grep -q .bazelversion
then
export BAZEL_OPTS=""
fi
-export BAZEL_OPTS="$BAZEL_OPTS --spawn_strategy=standalone --genrule_strategy=standalone"
-
java -fullversion
bazelisk version
-bazelisk build $BAZEL_OPTS plugins:core release api
-tools/maven/api.sh install
-tools/eclipse/project.py --bazel bazelisk
+
+if [[ "$MODE" == *"rbe"* ]]
+then
+ # TODO(davido): Figure out why javadoc part of api-rule doesn't work on RBE.
+ # See: https://github.com/bazelbuild/bazel/issues/12765 for more background.
+ bazelisk build --config=remote --remote_instance_name=projects/api-project-164060093628/instances/default_instance plugins:core release api-skip-javadoc
+else
+ bazelisk build $BAZEL_OPTS plugins:core release api
+ tools/maven/api.sh install
+ tools/eclipse/project.py --bazel bazelisk
+fi
diff --git a/jenkins/gerrit-bazel-cleancache.sh b/jenkins/gerrit-bazel-cleancache.sh
index 59d8fb6..0f9f651 100644
--- a/jenkins/gerrit-bazel-cleancache.sh
+++ b/jenkins/gerrit-bazel-cleancache.sh
@@ -1,6 +1,6 @@
#!/bin/bash -e
-if [git show --diff-filter=AM --name-only --pretty="" HEAD | grep -q .bazelversion]
+if git show --diff-filter=AM --name-only --pretty="" HEAD | grep -q .bazelversion
then
echo "Bazel version upgrade to $(cat .bazelversion) detected => cleaning up all local Bazel caches"
rm -Rf ~/.gerritcodereview ~/.cache
diff --git a/jenkins/gerrit-bazel-verifier-test.sh b/jenkins/gerrit-bazel-verifier-test.sh
index 60eb70c..673cc78 100644
--- a/jenkins/gerrit-bazel-verifier-test.sh
+++ b/jenkins/gerrit-bazel-verifier-test.sh
@@ -7,18 +7,25 @@
echo "Test with mode=$MODE"
echo '----------------------------------------------'
-TEST_TAG_FILTERS="-flaky"
-if [[ "$TARGET_BRANCH" = stable-2.1* ]]
-then
- TEST_TAG_FILTERS="$TEST_TAG_FILTERS,-elastic"
-fi
+case $TARGET_BRANCH$MODE in
+ masterrbe|stable-3.4rbe|stable-3.4-2021-07.sticky-approvalsrbe)
+ TEST_TAG_FILTER="-flaky,-elastic,-git-protocol-v2"
+ BAZEL_OPTS="--config=remote --remote_instance_name=projects/api-project-164060093628/instances/default_instance"
+ ;;
+ masternotedb|stable-3.4notedb)
+ TEST_TAG_FILTER="-flaky,elastic,git-protocol-v2"
+ ;;
+ stable-2.*)
+ TEST_TAG_FILTER="-flaky,-elastic"
+ ;;
+ *)
+ TEST_TAG_FILTER="-flaky"
+esac
-export BAZEL_OPTS="$BAZEL_OPTS --spawn_strategy=standalone --genrule_strategy=standalone \
- --test_output errors \
- --test_summary detailed --flaky_test_attempts 3 \
- --test_verbose_timeout_warnings --build_tests_only \
+export BAZEL_OPTS="$BAZEL_OPTS \
+ --flaky_test_attempts 3 \
--test_timeout 3600 \
- --test_tag_filters=$TEST_TAG_FILTERS \
+ --test_tag_filters=$TEST_TAG_FILTER \
--test_env DOCKER_HOST=$DOCKER_HOST"
export WCT_HEADLESS_MODE=1
@@ -28,7 +35,7 @@
if [[ "$MODE" == *"reviewdb"* ]]
then
GERRIT_NOTEDB="--test_env=GERRIT_NOTEDB=OFF"
- bazelisk test $BAZEL_OPTS //...
+ bazelisk test $GERRIT_NOTEDB $BAZEL_OPTS //...
fi
if [[ "$MODE" == *"notedb"* ]]
@@ -37,13 +44,18 @@
bazelisk test $GERRIT_NOTEDB $BAZEL_OPTS //...
fi
+if [[ "$MODE" == *"rbe"* ]]
+then
+ bazelisk test $BAZEL_OPTS //...
+fi
+
if [[ "$MODE" == *"polygerrit"* ]]
then
echo 'Running Documentation tests...'
bazelisk test $BAZEL_OPTS //tools/bzl:always_pass_test Documentation/...
- echo 'Running local tests...'
+ echo "Running local tests in $(google-chrome --version)"
bash ./polygerrit-ui/app/run_test.sh || touch ~/polygerrit-failed
if [ -z "$SAUCE_USERNAME" ] || [ -z "$SAUCE_ACCESS_KEY" ]
diff --git a/jenkins/gerrit-codestyle-polygerrit-lint.sh b/jenkins/gerrit-codestyle-polygerrit-lint.sh
index 377682e..ebd5e2d 100644
--- a/jenkins/gerrit-codestyle-polygerrit-lint.sh
+++ b/jenkins/gerrit-codestyle-polygerrit-lint.sh
@@ -2,7 +2,7 @@
. set-java.sh 8
-if [git show --diff-filter=AM --name-only --pretty="" HEAD | grep -q .bazelversion]
+if git show --diff-filter=AM --name-only --pretty="" HEAD | grep -q .bazelversion
then
export BAZEL_OPTS=""
fi
@@ -10,6 +10,7 @@
cd gerrit
bazelisk version
if ([ "$TARGET_BRANCH" == "master" ] || \
+ [ "$TARGET_BRANCH" == "stable-3.4" ] || \
[ "$TARGET_BRANCH" == "stable-3.3" ] || \
[ "$TARGET_BRANCH" == "stable-3.2" ]) && \
((git show --diff-filter=AM --name-only --pretty="" HEAD | grep -q polygerrit-ui) || \
@@ -18,6 +19,6 @@
then
echo 'Running PolyGerrit lint check...'
java -fullversion
- bazelisk test //polygerrit-ui/app:lint_test --test_output errors
- bazelisk test //polygerrit-ui/app:polylint_test --test_output errors
+ bazelisk test //polygerrit-ui/app:lint_test
+ bazelisk test //polygerrit-ui/app:polylint_test
fi
diff --git a/jenkins/gerrit-verifier.yaml b/jenkins/gerrit-verifier.yaml
index 6ccd209..ecb07ab 100644
--- a/jenkins/gerrit-verifier.yaml
+++ b/jenkins/gerrit-verifier.yaml
@@ -64,6 +64,7 @@
- reviewdb
- notedb
- polygerrit
+ - rbe
description: 'Type of verification to execute'
properties:
- authorization:
diff --git a/vars/gerritPipeline.groovy b/vars/gerritPipeline.groovy
index 7aef6e8..a30c24b 100644
--- a/vars/gerritPipeline.groovy
+++ b/vars/gerritPipeline.groovy
@@ -45,6 +45,9 @@
}
stage('Report to Gerrit'){
+ resCodeStyle = getLabelValue(1, Builds.codeStyle.result)
+ gerritReview labels: ['Code-Style': resCodeStyle]
+
def verificationResults = Builds.verification.collect { k, v -> v }
def resVerify = verificationResults.inject(1) {
acc, build -> getLabelValue(acc, build.result)
@@ -87,7 +90,8 @@
String consoleUrl
GerritCheck(name, build) {
- this.uuid = "gerritforge:${env.GERRIT_PROJECT}-${name}"
+ this.uuid = "gerritforge:" + name.replaceAll("(bazel/)", "") +
+ Globals.gerritRepositoryNameSha1Suffix
this.build = build
this.consoleUrl = "${build.url}console"
}
@@ -126,11 +130,13 @@
def collectBuildModes() {
Builds.modes = []
- if (env.GERRIT_BRANCH ==~ /stable-3.*/ || env.GERRIT_BRANCH == "master" || env.GERRIT_BRANCH == "stable-3.4-2021-07.sticky-approvals") {
+ if (env.GERRIT_BRANCH == "master" || env.GERRIT_BRANCH == "stable-3.4" || env.GERRIT_BRANCH == "stable-3.4-2021-07.sticky-approvals") {
+ Builds.modes = ["notedb", "rbe"]
+ } else if (env.GERRIT_BRANCH ==~ /stable-3.[0-3]/) {
Builds.modes = ["notedb"]
- } else if (env.GERRIT_BRANCH ==~ /stable-2.16.*/) {
+ } else if (env.GERRIT_BRANCH == "stable-2.16") {
Builds.modes = ["notedb", "reviewdb"]
- } else if (env.GERRIT_BRANCH ==~ /stable-2.1[0-5].*/) {
+ } else if (env.GERRIT_BRANCH ==~ /stable-2.1[0-5]/) {
Builds.modes = ["reviewdb"]
} else {
throw new Exception("Unsupported branch ${env.GERRIT_BRANCH}")
diff --git a/vars/pluginPipeline.groovy b/vars/pluginPipeline.groovy
index 4d36f6a..c168cc0 100644
--- a/vars/pluginPipeline.groovy
+++ b/vars/pluginPipeline.groovy
@@ -37,7 +37,7 @@
def pluginScmUrl = "https://gerrit.googlesource.com/a/${env.GERRIT_PROJECT}"
def gjfVersion = '1.7'
def javaVersion = 11
- if (["stable-2.16", "stable-3.0", "stable-3.1", "stable-3.2"].contains("${env.GERRIT_BRANCH}")) {
+ if (env.GERRIT_BRANCH in ["stable-2.16", "stable-3.2"]) {
javaVersion = 8
}
def bazeliskCmd = "#!/bin/bash\n" + ". set-java.sh ${javaVersion} && bazelisk"
