Google Git
Sign in
gerrit / plugins / multi-site / 180e2eaecf7dc4040523ec5afc73db4be7f06aff
commit180e2eaecf7dc4040523ec5afc73db4be7f06aff[log] [tgz]
authorLuca Milanesio <luca.milanesio@gmail.com>Tue Jun 25 21:46:27 2019 +0100
committerLuca Milanesio <luca.milanesio@gmail.com>Tue Jun 25 22:58:53 2019 +0100
tree9b226e654467ec7762ddfed1155193882d9a6a10
parent188f4f77b1231984839fa28868a6107c83e6b653 [diff]
parent460f1077a85583253f5d7da99313047cee01d198 [diff]
Merge branch 'stable-3.0'

* stable-3.0:
  Broker publisher metric for number of message published
  Use KafkaConfiguration for Kafka related properties and functionality
  Fix setup.sh script and use bazel-bin directory
  Use bazel-bin to look for plugin's output
  Move ProjectDeletedSharedDbCleanup to global Module
  Decouple Kafka configuration class
  Add LB (HAProxy) to dockerised environment
  Add health-check plugin to dockerised environment
  Add multi-site plugin to dockerised environment
  Move CURATOR_VER constant to right before its use
  Fix plugins URL in the multi-site local environment
  Zookeeper module reads it's own configuration
  Update design proposal to abstract broker and Ref-DB
  Move Zookeeper-specifics into its own Module
  Don't import Id/Key/NameKey directly
  Module: Simplify BufferedReader creation
  DefaultSharedRefEnforcement: Fix MutableConstantField warning
  Fix ClassCanBeStatic warnings
  Fix DefaultCharset warnings
  ChangeCheckerImpl: Fix OperatorPrecedence warning
  Module: Fix DefaultCharset warning
  Configuration: Fix MutableConstantField warning
  KafkaSubscriber: Fix ClassCanBeStatic warning
  Bazel: Drop dependency on commons-lang3 library
  Bazel: Remove unneeded dependencies from test rule
  Bazel: Harmonize library names
  Format build files with buildifier
  Upgrade zookeeper to 3.4.14
  Update test dependencies
  Consolidate source of generated image files
  Allow restart of Gerrit instances
  Add header to the custom Kafka properties

Change-Id: Icb489c3bec2ae02e161d03655518fbe39f5b095b
tree: 9b226e654467ec7762ddfed1155193882d9a6a10
  1. .settings/
  2. dockerised_local_env/
  3. images/
  4. setup_local_env/
  5. src/
  6. tools/
  7. .bazelrc
  8. .gitignore
  9. .mailmap
  10. bazlets.bzl
  11. BUILD
  12. DESIGN.md
  13. docker-compose.kafka-broker.yaml
  14. external_plugin_deps.bzl
  15. LICENSE
  16. README.md
  17. WORKSPACE
README.md

Gerrit multi-site plugin

This plugin allows to deploy a distributed cluster of multiple Gerrit masters each using a separate site without sharing any storage. The alignment between the masters happens using the replication plugin and an external message broker.

Requirements for the Gerrit masters are:

  • Gerrit v2.16.5 or later
  • Migrated to NoteDb
  • Connected to the same message broker
  • Accessible via a load balancer (e.g. HAProxy)

NOTE: The multi-site plugin will not start if Gerrit is not yet migrated to NoteDb.

Currently, the only mode supported is one primary read/write master and multiple read-only masters but eventually the plan is to support multiple read/write masters. The read/write master is handling any traffic while the read-only masters are serving the Gerrit GUI assets, the HTTP GET REST API and git fetch requests (git-upload-pack). The read-only masters are kept synchronized with the read/write master in order to be always ready to become a read/write master.

For more details on the overall multi-site design and roadmap, please refer to the multi-site plugin DESIGN.md document

License

This plugin is released under the same Apache 2.0 license and copyright holders as of the Gerrit Code Review project.

How to build

The multi-site plugin is built like any other Gerrit plugin in tree mode, by cloning Gerrit and the multi-site plugin code, and checking them out on the desired branch.

Example of cloning Gerrit and multi-site for a stable-2.16 build:

git clone -b stable-2.16 https://gerrit.googlesource.com/gerrit
git clone -b stable-2.16 https://gerrit.googlesource.com/plugins/multi-site

cd gerrit/plugins
ln -s ../../multi-site .
rm external_plugin_deps.bzl
ln -s multi-site/external_plugin_deps.bzl .

Example of building the multi-site plugin:

cd gerrit
bazel build plugins/multi-site

The multi-site.jar plugin is generated to bazel-bin/plugins/multi-site/multi-site.jar.

Example of testing the multi-site plugin:

cd gerrit
bazel test plugins/multi-site:multi_site_tests

NOTE: The multi-site tests include also the use of Docker containers for instantiating and using a Kafka/Zookeeper broker. Make sure you have a Docker daemon running (/var/run/docker.sock accessible) or a DOCKER_HOST pointing to a Docker server.

How to configure

Install the multi-site plugin into the $GERRIT_SITE/lib directory of all the Gerrit servers that are part of the multi-site cluster.

Add the multi-site module to $GERRIT_SITE/etc/gerrit.config as follows:

[gerrit]
  installDbModule = com.googlesource.gerrit.plugins.multisite.GitModule
  installModule = com.googlesource.gerrit.plugins.multisite.Module

Create the $GERRIT_SITE/etc/multi-site.config on all Gerrit servers with the following basic settings:

[kafka]
  bootstrapServers = <kafka-host>:<kafka-port>

[kafka "publisher"]
  enabled = true

[kafka "subscriber"]
  enabled = true

[ref-database]
  enabled = true

[ref-database "zookeeper"]
  connectString = "localhost:2181"

For more details on the configuration settings, please refer to the multi-site configuration documentation.

You also need to setup the Git-level replication between nodes, for more details please refer to the replication plugin documentation.