Add general introduction to HA Give a general overview of how the high-availability plugin can be used for setting up and configuring a fully working Gerrit cluster. Include reference to the overall design and a simple fully-working Gerrit setup using the high-availability plugin and HAProxy together. Change-Id: I7b81812570cff0006ecf5ca596ec11d70074eb02
diff --git a/README.md b/README.md new file mode 100644 index 0000000..8e7ef27 --- /dev/null +++ b/README.md
@@ -0,0 +1,196 @@ +# Gerrit high-availability plugin + +This plugin allows deploying a cluster of multiple Gerrit masters +on the same data-center sharing the same ReviewDb and Git repositories. + +Requirements for the Gerrit masters are: + +- Gerrit v2.14.20 or later +- Externally mounted filesystem shared among the cluster +- ReviewDb on an external DataBase Server +- Load-balancer (HAProxy or similar) + +## License + +This plugin is released under the same Apache 2.0 license and copyright holders +as of the Gerrit Code Review project. + +## How to build + +Refer to the [build instructions in the plugin documentation](src/main/resources/Documentation/build.md). + +## Sample configuration for two Gerrit masters in high-availability + +Assuming that the Gerrit masters in the clusters are `gerrit-01.mycompany.com` and +`gerrit-02.mycompany.com`, listening on the HTTP port 8080, with a shared volume +mounted under `/shared`, see below the minimal configuration steps. + +1. Install one Gerrit master on the first node (e.g. `gerrit-01.mycompany.com`) using an external + ReviewDb on a DB server and the repositories location under the shared volume (e.g. `/shared/git`). + Init the site in order to create the DB Schema and the initial repositories. + +2. Copy all the files of the first Gerrit master onto the second node (e.g. `gerrit-02.mycompany.com`) + so that it points to the same ReviewDb and the same repositories location. + +3. Install the high-availability plugin into the `$GERRIT_SITE/plugins` directory of both + the Gerrit servers. + +4. On `gerrit-01.mycompany.com`, create the `$GERRIT_SITE/etc/high-availability.config` with + the following settings: + + ``` + [main] + sharedDirectory = /shared + + [peerInfo] + strategy = static + + [peerInfo "static"] + url = http://gerrit-02.mycompany.com:8080 + ``` + +5. On `gerrit-02.mycompany.com`, create the `$GERRIT_SITE/etc/high-availability.config` with + the following settings: + + ``` + [main] + sharedDirectory = /shared + + [peerInfo] + strategy = static + + [peerInfo "static"] + url = http://gerrit-01.mycompany.com:8080 + ``` + +For more details on the configuration settings, please refer to the +[high-availability configuration documentation](src/main/resources/Documentation/config.md). + +## Load-balancing configuration + +It is possible to distribute the incoming traffic to both Gerrit nodes using any software that can +perform load-balancing of the incoming connections. + +The load-balancing of the HTTP traffic is at L7 (Application) while the +SSH traffic is balanced at L4 (Transport) level. + +### Active-passive configuration + +It is the simplest and safest configuration, where only one Gerrit master at a +time serves the incoming requests. +In case of failure of the primary master, the traffic is forwarded to the backup. + +Assuming a load-balancing implemented using [HAProxy](http://www.haproxy.org/) +associated with the domain name `gerrit.mycompany.com`, exposing Gerrit cluster nodes +on ports, 80 (HTTP) and 29418 (SSH), see below the minimal configuration +steps + +Add to the `haproxy.cfg` the frontend configurations associated with the HTTP +and SSH services: + +``` +frontend gerrit_http + bind *:80 + mode http + default_backend gerrit_http_nodes + +frontend gerrit_ssh + bind *:29418 + mode tcp + default_backend gerrit_ssh_nodes +``` + +Add to the `haproxy.cfg` the backend configurations pointing to the Gerrit cluster +nodes: + +``` +backend gerrit_http_nodes + mode http + balance source + option forwardfor + default-server inter 10s fall 3 rise 2 + option httpchk GET /config/server/version HTTP/1.0 + http-check expect status 200 + server gerrit_http_01 gerrit-01.mycompany.com:8080 check inter 10s + server gerrit_http_02 gerrit-01.mycompany.com:8080 check inter 10s backup + +ackend ssh + mode tcp + option httpchk GET /config/server/version HTTP/1.0 + http-check expect status 200 + balance source + timeout connect 10s + timeout server 5m + server gerrit_ssh_01 gerrit-01.mycompany.com:29418 check port 8080 inter 10s fall 3 rise 2 + server gerrit-ssh_02 gerrit-02.mycompany.com:29418 check port 8080 inter 10s fall 3 rise 2 backup +``` + +### Active-active configuration + +This is an evolution of the previous active-passive configuration, where only one Gerrit master at a +time serves the HTTP write operations (PUT,POST,DELETE) while the remaining HTTP traffic is sent +to both. +In case of failure of one of the nodes, all the traffic is forwarded to the other node. + +With regards to the SSH traffic, it cannot be safely sent to both nodes because it is associated +with a stateful session that can host multiple commands of different nature. + +Assuming an active-passive configuration using HAProxy, see below the changes needed to implement +an active-active scenario. + +Add to the `haproxy.cfg` the extra acl settings into the `gerrit_http` frontend configurations +associated with the HTTP and SSH services: + +``` +frontend gerrit_http + bind *:80 + mode http + acl http_writes method PUT POST DELETE PATCH + use_backend gerrit_http_nodes if http_writes + default_backend gerrit_http_nodes_balanced +``` + +Add to the `haproxy.cfg` a new backend for serving all read-only HTTP operations from both nodes: + +``` +backend gerrit_http_nodes_balanced + mode http + balance source + option forwardfor + default-server inter 10s fall 3 rise 2 + option httpchk GET /config/server/version HTTP/1.0 + http-check expect status 200 + server gerrit_http_01 gerrit-01.mycompany.com:8080 check inter 10s + server gerrit_http_02 gerrit-01.mycompany.com:8080 check inter 10s +``` + +## Gerrit canonical URL and further adjustments + +Both Gerrit masters are now part of the same cluster, accessible through the HAProxy load-balancer. +Set the `gerrit.canoncalWebUrl` on both Gerrit masters to the domain name of HAProxy so that any +location or URL generated by Gerrit would direct the traffic to the balancer and not to the instance +that served the incoming call. + +Example: +``` +[gerrit] + canonicalWebUrl = http://gerrit.mycompany.com +``` + +Secondly, adjust the HTTP listen configuration adding the `proxy-` prefix, to inform Gerrit that +the traffic is getting filtered through a reverse proxy. + +Example: +``` +[httpd] + listenUrl = proxy-http://*:8080/ +``` + +Last adjustment is associated with the session cookies, because they would need to be bound to +the domain rather than the individual node. + +Example: +``` +[auth] + cookiedomain = .mycompany.com +``` \ No newline at end of file