Merge branch 'stable-2.14' into stable-2.15 * stable-2.14: Sample Gerrit HA with docker-compose Add general introduction to HA Adjust the Dockerfile to use 2.15.17 and stable-2.15. Change-Id: I2e557f10d47c933ae4acc9b0801d22f86686b292
diff --git a/README.md b/README.md new file mode 100644 index 0000000..8e7ef27 --- /dev/null +++ b/README.md
@@ -0,0 +1,196 @@ +# Gerrit high-availability plugin + +This plugin allows deploying a cluster of multiple Gerrit masters +on the same data-center sharing the same ReviewDb and Git repositories. + +Requirements for the Gerrit masters are: + +- Gerrit v2.14.20 or later +- Externally mounted filesystem shared among the cluster +- ReviewDb on an external DataBase Server +- Load-balancer (HAProxy or similar) + +## License + +This plugin is released under the same Apache 2.0 license and copyright holders +as of the Gerrit Code Review project. + +## How to build + +Refer to the [build instructions in the plugin documentation](src/main/resources/Documentation/build.md). + +## Sample configuration for two Gerrit masters in high-availability + +Assuming that the Gerrit masters in the clusters are `gerrit-01.mycompany.com` and +`gerrit-02.mycompany.com`, listening on the HTTP port 8080, with a shared volume +mounted under `/shared`, see below the minimal configuration steps. + +1. Install one Gerrit master on the first node (e.g. `gerrit-01.mycompany.com`) using an external + ReviewDb on a DB server and the repositories location under the shared volume (e.g. `/shared/git`). + Init the site in order to create the DB Schema and the initial repositories. + +2. Copy all the files of the first Gerrit master onto the second node (e.g. `gerrit-02.mycompany.com`) + so that it points to the same ReviewDb and the same repositories location. + +3. Install the high-availability plugin into the `$GERRIT_SITE/plugins` directory of both + the Gerrit servers. + +4. On `gerrit-01.mycompany.com`, create the `$GERRIT_SITE/etc/high-availability.config` with + the following settings: + + ``` + [main] + sharedDirectory = /shared + + [peerInfo] + strategy = static + + [peerInfo "static"] + url = http://gerrit-02.mycompany.com:8080 + ``` + +5. On `gerrit-02.mycompany.com`, create the `$GERRIT_SITE/etc/high-availability.config` with + the following settings: + + ``` + [main] + sharedDirectory = /shared + + [peerInfo] + strategy = static + + [peerInfo "static"] + url = http://gerrit-01.mycompany.com:8080 + ``` + +For more details on the configuration settings, please refer to the +[high-availability configuration documentation](src/main/resources/Documentation/config.md). + +## Load-balancing configuration + +It is possible to distribute the incoming traffic to both Gerrit nodes using any software that can +perform load-balancing of the incoming connections. + +The load-balancing of the HTTP traffic is at L7 (Application) while the +SSH traffic is balanced at L4 (Transport) level. + +### Active-passive configuration + +It is the simplest and safest configuration, where only one Gerrit master at a +time serves the incoming requests. +In case of failure of the primary master, the traffic is forwarded to the backup. + +Assuming a load-balancing implemented using [HAProxy](http://www.haproxy.org/) +associated with the domain name `gerrit.mycompany.com`, exposing Gerrit cluster nodes +on ports, 80 (HTTP) and 29418 (SSH), see below the minimal configuration +steps + +Add to the `haproxy.cfg` the frontend configurations associated with the HTTP +and SSH services: + +``` +frontend gerrit_http + bind *:80 + mode http + default_backend gerrit_http_nodes + +frontend gerrit_ssh + bind *:29418 + mode tcp + default_backend gerrit_ssh_nodes +``` + +Add to the `haproxy.cfg` the backend configurations pointing to the Gerrit cluster +nodes: + +``` +backend gerrit_http_nodes + mode http + balance source + option forwardfor + default-server inter 10s fall 3 rise 2 + option httpchk GET /config/server/version HTTP/1.0 + http-check expect status 200 + server gerrit_http_01 gerrit-01.mycompany.com:8080 check inter 10s + server gerrit_http_02 gerrit-01.mycompany.com:8080 check inter 10s backup + +ackend ssh + mode tcp + option httpchk GET /config/server/version HTTP/1.0 + http-check expect status 200 + balance source + timeout connect 10s + timeout server 5m + server gerrit_ssh_01 gerrit-01.mycompany.com:29418 check port 8080 inter 10s fall 3 rise 2 + server gerrit-ssh_02 gerrit-02.mycompany.com:29418 check port 8080 inter 10s fall 3 rise 2 backup +``` + +### Active-active configuration + +This is an evolution of the previous active-passive configuration, where only one Gerrit master at a +time serves the HTTP write operations (PUT,POST,DELETE) while the remaining HTTP traffic is sent +to both. +In case of failure of one of the nodes, all the traffic is forwarded to the other node. + +With regards to the SSH traffic, it cannot be safely sent to both nodes because it is associated +with a stateful session that can host multiple commands of different nature. + +Assuming an active-passive configuration using HAProxy, see below the changes needed to implement +an active-active scenario. + +Add to the `haproxy.cfg` the extra acl settings into the `gerrit_http` frontend configurations +associated with the HTTP and SSH services: + +``` +frontend gerrit_http + bind *:80 + mode http + acl http_writes method PUT POST DELETE PATCH + use_backend gerrit_http_nodes if http_writes + default_backend gerrit_http_nodes_balanced +``` + +Add to the `haproxy.cfg` a new backend for serving all read-only HTTP operations from both nodes: + +``` +backend gerrit_http_nodes_balanced + mode http + balance source + option forwardfor + default-server inter 10s fall 3 rise 2 + option httpchk GET /config/server/version HTTP/1.0 + http-check expect status 200 + server gerrit_http_01 gerrit-01.mycompany.com:8080 check inter 10s + server gerrit_http_02 gerrit-01.mycompany.com:8080 check inter 10s +``` + +## Gerrit canonical URL and further adjustments + +Both Gerrit masters are now part of the same cluster, accessible through the HAProxy load-balancer. +Set the `gerrit.canoncalWebUrl` on both Gerrit masters to the domain name of HAProxy so that any +location or URL generated by Gerrit would direct the traffic to the balancer and not to the instance +that served the incoming call. + +Example: +``` +[gerrit] + canonicalWebUrl = http://gerrit.mycompany.com +``` + +Secondly, adjust the HTTP listen configuration adding the `proxy-` prefix, to inform Gerrit that +the traffic is getting filtered through a reverse proxy. + +Example: +``` +[httpd] + listenUrl = proxy-http://*:8080/ +``` + +Last adjustment is associated with the session cookies, because they would need to be bound to +the domain rather than the individual node. + +Example: +``` +[auth] + cookiedomain = .mycompany.com +``` \ No newline at end of file
diff --git a/src/test/docker/README.md b/src/test/docker/README.md new file mode 100644 index 0000000..fd6818a --- /dev/null +++ b/src/test/docker/README.md
@@ -0,0 +1,37 @@ +# Gerrit high-availability setup example + +This Docker Compose project contains a simple test environment +of two Gerrit masters in HA configuration. + +## How to build + +The project can be built using docker-compose. + +To build the Docker VMs: +``` + $ docker-compose build +``` + +## How to run + +Use the 'up' target to startup the Docker Compose VMs. + +``` + $ docker-compose up +``` + +## How to stop + +Simply type CTRL+C on the window that started the environment +and all the VMs will stop. Their state will be persisted and the next +run will continue with the same data. + +## How to clean + +If you want to stop and cleanup all the previous state, use the 'down' +target. + +``` + $ docker-compose down +``` +
diff --git a/src/test/docker/docker-compose.yaml b/src/test/docker/docker-compose.yaml new file mode 100644 index 0000000..6ea2438 --- /dev/null +++ b/src/test/docker/docker-compose.yaml
@@ -0,0 +1,74 @@ +version: '2' + +services: + + postgres: + image: postgres:9.5.4 + environment: + - POSTGRES_USER=gerrit + - POSTGRES_PASSWORD=secret + - POSTGRES_DB=reviewdb + networks: + - gerrit-net + volumes: + - ./pgdata:/var/lib/postgresql/data + + gerrit-01: + build: gerrit + ports: + - "8081:8080" + - "29411:29418" + networks: + - gerrit-net + depends_on: + - postgres + volumes: + - /dev/urandom:/dev/random + - ./gitvolume:/var/gerrit/git + - ./shareddir:/var/gerrit/shared/dir + - ./etc/gerrit.config:/var/gerrit/etc/gerrit.config.orig + - ./etc/high-availability.gerrit-01.config:/var/gerrit/etc/high-availability.config + environment: + - HOSTNAME=localhost + + gerrit-02: + build: gerrit + ports: + - "8082:8080" + - "29412:29418" + networks: + - gerrit-net + depends_on: + - postgres + - gerrit-01 + volumes: + - /dev/urandom:/dev/random + - ./gitvolume:/var/gerrit/git + - ./shareddir:/var/gerrit/shared/dir + - ./etc/gerrit.config:/var/gerrit/etc/gerrit.config.orig + - ./etc/high-availability.gerrit-01.config:/var/gerrit/etc/high-availability.config + environment: + - HOSTNAME=localhost + - WAIT_FOR=gerrit-01:8080 + + haproxy: + build: haproxy + ports: + - "80:80" + - "29418:29418" + networks: + - gerrit-net + volumes_from: + - syslog-sidecar + depends_on: + - gerrit-01 + - gerrit-02 + + syslog-sidecar: + build: docker-syslog-ng-stdout + networks: + - gerrit-net + +networks: + gerrit-net: + driver: bridge
diff --git a/src/test/docker/docker-syslog-ng-stdout/Dockerfile b/src/test/docker/docker-syslog-ng-stdout/Dockerfile new file mode 100644 index 0000000..11f3059 --- /dev/null +++ b/src/test/docker/docker-syslog-ng-stdout/Dockerfile
@@ -0,0 +1,11 @@ +FROM alpine:3.4 +MAINTAINER Ryan Schlesinger <ryan@outstand.com> + +RUN apk add --no-cache bash syslog-ng + +RUN mkdir /sidecar +COPY config/* /etc/syslog-ng/ +COPY docker-entrypoint.sh /docker-entrypoint.sh +VOLUME ["/sidecar"] +CMD ["syslog-ng", "-F"] +ENTRYPOINT ["/docker-entrypoint.sh"]
diff --git a/src/test/docker/docker-syslog-ng-stdout/LICENSE b/src/test/docker/docker-syslog-ng-stdout/LICENSE new file mode 100644 index 0000000..8dada3e --- /dev/null +++ b/src/test/docker/docker-syslog-ng-stdout/LICENSE
@@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "{}" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright {yyyy} {name of copyright owner} + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License.
diff --git a/src/test/docker/docker-syslog-ng-stdout/README.md b/src/test/docker/docker-syslog-ng-stdout/README.md new file mode 100644 index 0000000..9442a19 --- /dev/null +++ b/src/test/docker/docker-syslog-ng-stdout/README.md
@@ -0,0 +1,22 @@ +# Supported tags and respective `Dockerfile` links + +- [`latest`, (*Dockerfile*)](https://github.com/outstand/docker-syslog-ng-stdout/blob/master/Dockerfile) + +# Usage + +Add to haproxy config: +``` + log /sidecar/log local0 +``` + +docker-compose.yml: +```yaml +version: '2' +services: + haproxy: + image: haproxy:latest + volumes_from: + - syslog-sidecar + syslog-sidecar: + image: outstand/syslog-ng-stdout:latest +```
diff --git a/src/test/docker/docker-syslog-ng-stdout/config/syslog-ng-destination.out b/src/test/docker/docker-syslog-ng-stdout/config/syslog-ng-destination.out new file mode 100644 index 0000000..170f7e4 --- /dev/null +++ b/src/test/docker/docker-syslog-ng-stdout/config/syslog-ng-destination.out
@@ -0,0 +1 @@ + destination d_stdout { pipe("/dev/stdout"); };
diff --git a/src/test/docker/docker-syslog-ng-stdout/config/syslog-ng-log.out b/src/test/docker/docker-syslog-ng-stdout/config/syslog-ng-log.out new file mode 100644 index 0000000..6901c41 --- /dev/null +++ b/src/test/docker/docker-syslog-ng-stdout/config/syslog-ng-log.out
@@ -0,0 +1 @@ +log { source(s_all); destination(d_stdout); };
diff --git a/src/test/docker/docker-syslog-ng-stdout/config/syslog-ng-plugins.std b/src/test/docker/docker-syslog-ng-stdout/config/syslog-ng-plugins.std new file mode 100644 index 0000000..ef0ce3e --- /dev/null +++ b/src/test/docker/docker-syslog-ng-stdout/config/syslog-ng-plugins.std
@@ -0,0 +1 @@ +@version: 3.7
diff --git a/src/test/docker/docker-syslog-ng-stdout/config/syslog-ng-source.sidecar b/src/test/docker/docker-syslog-ng-stdout/config/syslog-ng-source.sidecar new file mode 100644 index 0000000..1181f78 --- /dev/null +++ b/src/test/docker/docker-syslog-ng-stdout/config/syslog-ng-source.sidecar
@@ -0,0 +1,5 @@ +# sidecar log source for mounting between docker containers + network( + transport("udp") + port("514") + );
diff --git a/src/test/docker/docker-syslog-ng-stdout/config/syslog-ng-source.std b/src/test/docker/docker-syslog-ng-stdout/config/syslog-ng-source.std new file mode 100644 index 0000000..5a50916 --- /dev/null +++ b/src/test/docker/docker-syslog-ng-stdout/config/syslog-ng-source.std
@@ -0,0 +1,8 @@ +# --------------------------------------------------------------------------------- +# Default syslog-ng sources; Do not edit this file! +# append source with line on a file: syslog-ng-source.<package> +# --------------------------------------------------------------------------------- +# message generated by Syslog-NG + internal(); +# standard Linux log source (this is the default place for the syslog() function to send logs to) + unix-dgram("/dev/log");
diff --git a/src/test/docker/docker-syslog-ng-stdout/docker-entrypoint.sh b/src/test/docker/docker-syslog-ng-stdout/docker-entrypoint.sh new file mode 100755 index 0000000..be51a09 --- /dev/null +++ b/src/test/docker/docker-syslog-ng-stdout/docker-entrypoint.sh
@@ -0,0 +1,49 @@ +#!/bin/bash + +# The following two methods are ripped from alpine's syslog-ng package. +# This allows us (and a user) to customize the config with snippets. +grep_syslog_conf_entries() { + local section="$1" FN filelist + grep -v '^#' /etc/syslog-ng/syslog-ng-${section}.std + filelist=$(find /etc/syslog-ng/ -maxdepth 1 -type f -name "syslog-ng-${section}.*" | grep -Ev ".backup|.std|~") + if [ $? -eq 0 ] + then + for FN in ${filelist} + do + grep -v '^#' $FN + done + fi +} + +update() { + local fname='/etc/syslog-ng/syslog-ng.conf' + local f_tmp="/etc/syslog-ng/syslog-ng.conf.$$" + for ng_std in options source destination filter log + do + [ -f /etc/syslog-ng/syslog-ng-${ng_std}.std ] || exit 1 + done + { + # create options entries + grep_syslog_conf_entries plugins + echo "options {" + grep_syslog_conf_entries options + echo "};" + # create source entries + echo "source s_all {" + grep_syslog_conf_entries source + echo "};" + # create destination entries + grep_syslog_conf_entries destination + # create filter entries + grep_syslog_conf_entries filter + # create log entries + grep_syslog_conf_entries log + } > $f_tmp + cp -p $f_tmp $fname + rm -f $f_tmp +} + +update + +echo Starting "$@" +exec "$@"
diff --git a/src/test/docker/etc/gerrit.config b/src/test/docker/etc/gerrit.config new file mode 100644 index 0000000..c110835 --- /dev/null +++ b/src/test/docker/etc/gerrit.config
@@ -0,0 +1,29 @@ +[gerrit] + basePath = git + canonicalWebUrl = http://gerrit:8080/ +[database] + type = postgresql + hostname = postgres + database = reviewdb + username = gerrit + password = secret +[index] + type = LUCENE +[auth] + type = DEVELOPMENT_BECOME_ANY_ACCOUNT + cookiedomain = localhost +[sendemail] + smtpServer = localhost +[sshd] + listenAddress = *:29418 +[httpd] + listenUrl = proxy-http://*:8080/ + requestLog = true +[cache] + directory = cache +[container] + user = gerrit +[download] + scheme = http + scheme = ssh + scheme = anon_http
diff --git a/src/test/docker/etc/high-availability.gerrit-01.config b/src/test/docker/etc/high-availability.gerrit-01.config new file mode 100644 index 0000000..f269f52 --- /dev/null +++ b/src/test/docker/etc/high-availability.gerrit-01.config
@@ -0,0 +1,8 @@ +[main] + sharedDirectory = /var/gerrit/shared/dir + +[peerInfo] + strategy = static + +[peerInfo "static"] + url = http://gerrit-02:8080
diff --git a/src/test/docker/etc/high-availability.gerrit-02.config b/src/test/docker/etc/high-availability.gerrit-02.config new file mode 100644 index 0000000..b49f91b --- /dev/null +++ b/src/test/docker/etc/high-availability.gerrit-02.config
@@ -0,0 +1,8 @@ +[main] + sharedDirectory = /var/gerrit/shared/dir + +[peerInfo] + strategy = static + +[peerInfo "static"] + url = http://gerrit-01:8080 \ No newline at end of file
diff --git a/src/test/docker/gerrit/Dockerfile b/src/test/docker/gerrit/Dockerfile new file mode 100644 index 0000000..27a26ef --- /dev/null +++ b/src/test/docker/gerrit/Dockerfile
@@ -0,0 +1,24 @@ +FROM gerritcodereview/gerrit:2.15.17 + +ENV GERRIT_BRANCH=stable-2.15 + +ENV GERRIT_CI_URL=https://gerrit-ci.gerritforge.com/job + +USER root + +RUN yum install -y iputils-ping netcat postgresql curl lsof gettext moreutils net-tools netcat inetutils-ping + +USER gerrit + +ADD $GERRIT_CI_URL/plugin-javamelody-bazel-$GERRIT_BRANCH/lastSuccessfulBuild/artifact/bazel-bin/plugins/javamelody/javamelody.jar /var/gerrit/plugins/javamelody.jar +ADD $GERRIT_CI_URL/plugin-javamelody-bazel-$GERRIT_BRANCH/lastSuccessfulBuild/artifact/bazel-bin/plugins/javamelody/javamelody-deps_deploy.jar /var/gerrit/lib/javamelody-deps_deploy.jar +ADD $GERRIT_CI_URL/plugin-high-availability-gerritforge-$GERRIT_BRANCH/lastSuccessfulBuild/artifact/bazel-bin/plugins/high-availability/high-availability.jar /var/gerrit/plugins/high-availability.jar + +USER root + +ADD start.sh /bin/ +ADD wait-for-it.sh /bin/ + +RUN rm -Rf /var/gerrit/{git,index,cache}/* + +CMD /bin/start.sh
diff --git a/src/test/docker/gerrit/start.sh b/src/test/docker/gerrit/start.sh new file mode 100755 index 0000000..51bc9bc --- /dev/null +++ b/src/test/docker/gerrit/start.sh
@@ -0,0 +1,25 @@ +#!/bin/bash -e + +wait-for-it.sh postgres:5432 -t 600 -- echo "Postgres is up" + +if [[ ! -z "$WAIT_FOR" ]] +then + wait-for-it.sh $WAIT_FOR -t 600 -- echo "$WAIT_FOR is up" +fi + +sudo -u gerrit cp /var/gerrit/etc/gerrit.config.orig /var/gerrit/etc/gerrit.config + +if [[ ! -f /var/gerrit/git/All-Projects.git/config ]] +then + echo "Initializing Gerrit site ..." + sudo -u gerrit java -jar /var/gerrit/bin/gerrit.war init -d /var/gerrit --batch +fi + +echo "Reindexing Gerrit ..." +sudo -u gerrit java -jar /var/gerrit/bin/gerrit.war reindex -d /var/gerrit +sudo -u gerrit git config -f /var/gerrit/etc/gerrit.config gerrit.canonicalWebUrl http://$HOSTNAME/ + +touch /var/gerrit/logs/{gc_log,error_log,httpd_log,sshd_log,replication_log} && chown -R gerrit: /var/gerrit && tail -f /var/gerrit/logs/* | grep --line-buffered -v 'HEAD /' & + +echo "Running Gerrit ..." +sudo -u gerrit /etc/init.d/gerrit run
diff --git a/src/test/docker/gerrit/wait-for-it.sh b/src/test/docker/gerrit/wait-for-it.sh new file mode 100755 index 0000000..d7b6e3c --- /dev/null +++ b/src/test/docker/gerrit/wait-for-it.sh
@@ -0,0 +1,162 @@ +#!/usr/bin/env bash +# https://github.com/vishnubob/wait-for-it/blob/master/wait-for-it.sh +# Use this script to test if a given TCP host/port are available + +cmdname=$(basename $0) + +echoerr() { if [[ $QUIET -ne 1 ]]; then echo "$@" 1>&2; fi } + +usage() +{ + cat << USAGE >&2 +Usage: + $cmdname host:port [-s] [-t timeout] [-- command args] + -h HOST | --host=HOST Host or IP under test + -p PORT | --port=PORT TCP port under test + Alternatively, you specify the host and port as host:port + -s | --strict Only execute subcommand if the test succeeds + -q | --quiet Don't output any status messages + -t TIMEOUT | --timeout=TIMEOUT + Timeout in seconds, zero for no timeout + -- COMMAND ARGS Execute command with args after the test finishes +USAGE + exit 1 +} + +wait_for() +{ + if [[ $TIMEOUT -gt 0 ]]; then + echoerr "$cmdname: waiting $TIMEOUT seconds for $HOST:$PORT" + else + echoerr "$cmdname: waiting for $HOST:$PORT without a timeout" + fi + start_ts=$(date +%s) + while : + do + (echo > /dev/tcp/$HOST/$PORT) >/dev/null 2>&1 + result=$? + if [[ $result -eq 0 ]]; then + end_ts=$(date +%s) + echoerr "$cmdname: $HOST:$PORT is available after $((end_ts - start_ts)) seconds" + break + fi + sleep 1 + done + return $result +} + +wait_for_wrapper() +{ + # In order to support SIGINT during timeout: http://unix.stackexchange.com/a/57692 + if [[ $QUIET -eq 1 ]]; then + timeout $TIMEOUT $0 --quiet --child --host=$HOST --port=$PORT --timeout=$TIMEOUT & + else + timeout $TIMEOUT $0 --child --host=$HOST --port=$PORT --timeout=$TIMEOUT & + fi + PID=$! + trap "kill -INT -$PID" INT + wait $PID + RESULT=$? + if [[ $RESULT -ne 0 ]]; then + echoerr "$cmdname: timeout occurred after waiting $TIMEOUT seconds for $HOST:$PORT" + fi + return $RESULT +} + +# process arguments +while [[ $# -gt 0 ]] +do + case "$1" in + *:* ) + hostport=(${1//:/ }) + HOST=${hostport[0]} + PORT=${hostport[1]} + shift 1 + ;; + --child) + CHILD=1 + shift 1 + ;; + -q | --quiet) + QUIET=1 + shift 1 + ;; + -s | --strict) + STRICT=1 + shift 1 + ;; + -h) + HOST="$2" + if [[ $HOST == "" ]]; then break; fi + shift 2 + ;; + --host=*) + HOST="${1#*=}" + shift 1 + ;; + -p) + PORT="$2" + if [[ $PORT == "" ]]; then break; fi + shift 2 + ;; + --port=*) + PORT="${1#*=}" + shift 1 + ;; + -t) + TIMEOUT="$2" + if [[ $TIMEOUT == "" ]]; then break; fi + shift 2 + ;; + --timeout=*) + TIMEOUT="${1#*=}" + shift 1 + ;; + --) + shift + CLI="$@" + break + ;; + --help) + usage + ;; + *) + echoerr "Unknown argument: $1" + usage + ;; + esac +done + +if [[ "$HOST" == "" || "$PORT" == "" ]]; then + echoerr "Error: you need to provide a host and port to test." + usage +fi + +TIMEOUT=${TIMEOUT:-15} +STRICT=${STRICT:-0} +CHILD=${CHILD:-0} +QUIET=${QUIET:-0} + +if [[ $CHILD -gt 0 ]]; then + wait_for + RESULT=$? + exit $RESULT +else + if [[ $TIMEOUT -gt 0 ]]; then + wait_for_wrapper + RESULT=$? + else + wait_for + RESULT=$? + fi +fi + +if [[ $CLI != "" ]]; then + if [[ $RESULT -ne 0 && $STRICT -eq 1 ]]; then + echoerr "$cmdname: strict mode, refusing to execute subprocess" + exit $RESULT + fi + exec $CLI +else + exit $RESULT +fi
diff --git a/src/test/docker/haproxy/Dockerfile b/src/test/docker/haproxy/Dockerfile new file mode 100644 index 0000000..72cad4a --- /dev/null +++ b/src/test/docker/haproxy/Dockerfile
@@ -0,0 +1,8 @@ +FROM haproxy:1.8 + +COPY haproxy.cfg /usr/local/etc/haproxy/haproxy.cfg + +RUN mkdir /var/lib/haproxy && \ + mkdir /var/run/haproxy && \ + useradd haproxy && \ + chown haproxy: /var/lib/haproxy /var/run/haproxy
diff --git a/src/test/docker/haproxy/haproxy.cfg b/src/test/docker/haproxy/haproxy.cfg new file mode 100644 index 0000000..e86cdce --- /dev/null +++ b/src/test/docker/haproxy/haproxy.cfg
@@ -0,0 +1,68 @@ +global + chroot /var/lib/haproxy + stats socket /var/run/haproxy/admin.sock mode 660 level admin + stats timeout 30s + user haproxy + group haproxy + daemon + maxconn 128 + + # Default SSL material locations + ca-base /etc/ssl/certs + crt-base /etc/ssl/private + + # Default ciphers to use on SSL-enabled listening sockets. + # For more information, see ciphers(1SSL). + ssl-default-bind-ciphers kEECDH+aRSA+AES:kRSA+AES:+AES256:RC4-SHA:!kEDH:!LOW:!EXP:!MD5:!aNULL:!eNULL + +defaults + timeout connect 5s + timeout client 10s + timeout server 10s + mode http + log syslog-sidecar local0 debug + option log-health-checks + option http-server-close + +frontend gerrit_http + bind *:80 + option httplog + acl http_writes method PUT POST DELETE PATCH + use_backend gerrit_http_nodes if http_writes + default_backend gerrit_http_nodes_balanced + +frontend gerrit_ssh + bind *:29418 + option tcplog + mode tcp + default_backend gerrit_ssh_nodes + +backend gerrit_ssh_nodes + mode tcp + option httpchk GET /config/server/version HTTP/1.0 + http-check expect status 200 + balance source + timeout connect 10s + timeout server 5m + server gerrit_ssh_01 gerrit-01:29418 check port 8080 inter 10s fall 3 rise 2 + server gerrit-ssh_02 gerrit-02:29418 check port 8080 inter 10s fall 3 rise 2 backup + +backend gerrit_http_nodes + mode http + balance source + option forwardfor + default-server inter 10s fall 3 rise 2 + option httpchk GET /config/server/version HTTP/1.0 + http-check expect status 200 + server gerrit_01 gerrit-01:8080 check + server gerrit_02 gerrit-02:8080 check backup + +backend gerrit_http_nodes_balanced + mode http + balance source + option forwardfor + default-server inter 10s fall 3 rise 2 + option httpchk GET /config/server/version HTTP/1.0 + http-check expect status 200 + server gerrit_01 gerrit-01:8080 check + server gerrit_02 gerrit-02:8080 check