Google Git
Sign in
gerrit / plugins / multi-site / 43a68c765857ca929f6b9bde527ad47b6d431ce2
commit43a68c765857ca929f6b9bde527ad47b6d431ce2[log] [tgz]
authorLuca Milanesio <luca.milanesio@gmail.com>Mon Apr 15 11:20:37 2019 +0100
committerLuca Milanesio <luca.milanesio@gmail.com>Mon Apr 15 17:48:47 2019 +0100
tree3dacdcf3fbb048ab399bbd13f849f66b7423aed0
parentd03900096cc01a4c30936911c424d390fb80a986 [diff]
parent8d48e37ec18b115deeb4cfae73a10e683b997bb6 [diff]
Merge branch 'stable-2.16'

* stable-2.16:
  Skip checks in batch-update when there are no updates
  ZK compareAndPut should alway ignore the ignored refs
  Remove redundant ref-database migration flag
  Disable replication on startup
  Validate replication.config at Gerrit startup
  Write clear text data into ZK znodes
  Differentiate the multi-site Gson from Gerrit's
  Remove forwarded-aware event broker
  Revert "Remove the thread-local event forwarding logic"
  Use prim/backup logic for HAProxy reads
  Deploy multi-site plugin as gerrit lib module
  Ignore immutable and draft-comments Refs
  Remove hardcoded localhost from ZK test
  Wrap batch refupdate for shared refdb validation
  Always run in migration mode
  Fix NPE when creating new refs
  Bind GitRepositoryManager and associated factories
  Turn the multi-site plugin into a libModule
  Reformat with google-java-format
  Remove the thread-local event forwarding logic
  Remove plugin name from logs message
  Wrap Gerrit GitRepositoryManager into a Multisite fashion
  Wrap JGit Repository into a Multisite fashion
  Wrap RefDatabase into a MultiSite fashion
  Validate JGit RefUpdates events
  Add ref-database to local environment
  Remove redundant RefDatabaseConfig
  Always enable the shared ref-database
  Give proper name to the shared ref-database config
  Check for errors during downloading of plugins
  Removed unused obsolete constant
  Format config.md with wrapped lines and spaces
  Add migration mode to support missing entries
  Simplify the Zookeeper-based SharedDB code
  Implement Zookeeper backend for consistency check
  Fix directory creation in Docker test env

Requires Gerrit change I5b5a9432eb3db2559ce6a4393a42a4d227a371b5

Change-Id: Ic465e48a14376bb97e8ec10d69df66a5de87e201
tree: 3dacdcf3fbb048ab399bbd13f849f66b7423aed0
  1. .settings/
  2. dockerised_local_env/
  3. images/
  4. setup_local_env/
  5. src/
  6. tools/
  7. .bazelrc
  8. .gitignore
  9. .mailmap
  10. bazlets.bzl
  11. BUILD
  12. DESIGN.md
  13. docker-compose.kafka-broker.yaml
  14. external_plugin_deps.bzl
  15. LICENSE
  16. README.md
  17. WORKSPACE
README.md

Gerrit multi-site plugin

This plugin allows having multiple Gerrit masters to be deployed across various sites without having to share any storage. The alignment between the masters happens using the replication plugin and an external message broker.

The Gerrit masters requirements are:

  • Gerrit v2.16.5 or later
  • Migrated to NoteDb
  • Connected to the same message broker
  • Accessible behind a load balancer (e.g., HAProxy)

NOTE: The multi-site plugin will not start if Gerrit is not yet migrated to NoteDb.

Currently, the only mode supported is one primary read/write master and multiple read-only masters but eventually the plan is to support N read/write masters. The read/write master is handling any traffic while the read-only masters are serving the Gerrit GUI assets, the HTTP GET REST API and the git-upload-packs. The read-only masters are kept updated to be always ready to become a read/write master.

For more details on the overall multi-site design and roadmap, please refer to the multi-site plugin DESIGN.md document

License

This plugin is released under the same Apache 2.0 license and copyright holders as of the Gerrit Code Review project.

How to build

The multi-site plugin is built like any other Gerrit plugin in tree mode, by cloning Gerrit and the multi-site plugin code, and checking them out on the desired branch.

Example of cloning Gerrit and multi-site for a stable-2.16 build:

git clone -b stable-2.16 https://gerrit.googlesource.com/gerrit
git clone -b stable-2.16 https://gerrit.googlesource.com/plugins/multi-site

cd gerrit/plugins
ln -s ../../multi-site .
rm external_plugin_deps.bzl
ln -s multi-site/external_plugin_deps.bzl .

Example of building the multi-site plugin:

cd gerrit
bazel build plugins/multi-site

The multi-site.jar plugin is generated to bazel-genfiles/plugins/multi-site/multi-site.jar.

Example of testing the multi-site plugin:

cd gerrit
bazel test plugins/multi-site:multi_site_tests

NOTE: The multi-site tests include also the use of Docker containers for instantiating and using a Kafka/Zookeeper broker. Make sure you have a Docker daemon running (/var/run/docker.sock accessible) or a DOCKER_HOST pointing to a Docker server.

How to configure

Install the multi-site plugin into the $GERRIT_SITE/lib directory of all the Gerrit servers that are part of the multi-site cluster.

Add the multi-site module to $GERRIT_SITE/etc/gerrit.config as follows:

[gerrit]
  installModule = com.googlesource.gerrit.plugins.multisite.Module

Create the $GERRIT_SITE/etc/multi-site.config on all Gerrit servers with the following basic settings:

[kafka]
  bootstrapServers = <kafka-host>:<kafka-port>

[kafka "publisher"]
  enabled = true

[kafka "subscriber"]
  enabled = true

[ref-database]
  enabled = true

[ref-database "zookeeper"]
  connectString = "localhost:2181"

For more details on the configuration settings, please refer to the multi-site configuration documentation.

You need also to setup the Git-level replication between nodes, for more details please refer to the replication plugin documentation.