Merge branch 'stable-3.10'

* stable-3.10:
  DRY out shouldNotBeTrackedAnymoreOnGlobalRefDb
  Ignore refs/multi-site/version on global-refdb in push replication filter
  Make the local test environment more realistic
  Make the local test environment more realistic
  Add acceptance test for change up-to-date checker
  Move replication.jar and delete-project.jar to /lib for testing
  Indexing retries: lower normal condition logs to debug
  Check for existence of change's target SHA1 for reindexing
  Do not remove replication.jar from multi-site test sites
  Update local setup script to stable-3.10
  Adapt to replication api changes on stable-3.10
  Adapt BrokerApiWrapper to the newest BrokerApi interface
  Avoid duplicate indexing tasks for the same id
  Refactor help information with additional detail for setup.sh script
  When enabled use draft comments for change up-to-date calculation
  Ignore new project created events which originate from a different node
  Do not propagate internal indexing operations
  Update the refs/multi-site/version/value once per batch-refupdate
  Update refs/multi-site/version for starred changes and sequences
  Remove dead code when updating project version meta-ref
  Fix merge Ib6aa8adbd causing 2x updates of refs/multi-site/version/*
  Fix target branch for the local environment
  Do not take draft comments into account for change index up-to-date
  Update Kafka container to bitnami/kafka:3.6.0
  Remove unused docker compose
  Update Zookeeper container to bitnami/zookeeper:3.8.3
  Remove double-binding of the ProjectVersionRefUpdateImpl to EventListener
  Enhance indexing logging
  Specify cache.threads = 0 in the multi-site config documentation

Change-Id: Ic3b8a892b6481f32391b592b292fe38a8baab023
tree: 026c526139e7aa07f82a44c133addb0245e7f868
  1. .settings/
  2. e2e-tests/
  3. images/
  4. setup_local_env/
  5. src/
  6. .bazelrc
  7. .gitignore
  8. .mailmap
  9. BUILD
  10. DESIGN.md
  11. Jenkinsfile
  12. LICENSE
  13. README.md
README.md

Gerrit multi-site plugin

This plugin allows to deploy a distributed cluster of multiple Gerrit masters each using a separate site without sharing any storage. The alignment between the masters happens using the replication plugin and an external message broker.

Requirements for the Gerrit masters are:

  • Gerrit v2.16.5 or later
  • Migrated to NoteDb
  • Connected to the same message broker
  • Accessible via a load balancer (e.g. HAProxy)

NOTE: The multi-site plugin will not start if Gerrit is not yet migrated to NoteDb.

Supports multiple read/write masters across multiple sites across different geographic locations. The Gerrit nodes are kept synchronized between each other using the replication plugin and a global ref-database in order to detect and prevent split-brains.

For more details on the overall multi-site design and roadmap, please refer to the multi-site plugin DESIGN.md document

License

This plugin is released under the same Apache 2.0 license and copyright holders as of the Gerrit Code Review project.

How to build

The multi-site plugin can only be built in tree mode, by cloning Gerrit and the multi-site plugin code, and checking them out on the desired branch.

Example of cloning Gerrit and multi-site for a stable-2.16 build:

git clone -b stable-2.16 https://gerrit.googlesource.com/gerrit
git clone -b stable-2.16 https://gerrit.googlesource.com/plugins/multi-site

cd gerrit/plugins
ln -s ../../multi-site .

Example of building the multi-site plugin:

cd gerrit
bazel build plugins/multi-site

The multi-site.jar plugin is generated to bazel-bin/plugins/multi-site/multi-site.jar.

Example of testing the multi-site plugin:

cd gerrit
bazel test plugins/multi-site:multi_site_tests

NOTE: The multi-site tests include also the use of Docker containers for instantiating and using a Kafka/Zookeeper broker. Make sure you have a Docker daemon running (/var/run/docker.sock accessible) or a DOCKER_HOST pointing to a Docker server.

Pre-requisites

Each Gerrit server of the cluster must be identified with a globally unique instance-id defined in $GERRIT_SITE/etc/gerrit.config. When migrating from a multi-site configuration with Gerrit v3.3 or earlier, you must reuse the instance-id value stored under $GERRIT_SITE/data/multi-site.

Example:

[gerrit]
  instanceId = 758fe5b7-1869-46e6-942a-3ae0ae7e3bd2

How to configure

Install the multi-site plugin into the $GERRIT_SITE/lib directory of all the Gerrit servers that are part of the multi-site cluster. Create a symbolic link from $GERRIT_SITE/lib/multi-site.jar into the $GERRIT_SITE/plugins.

Add the multi-site module to $GERRIT_SITE/etc/gerrit.config as follows:

[gerrit]
  installDbModule = com.googlesource.gerrit.plugins.multisite.GitModule
  installModule = com.googlesource.gerrit.plugins.multisite.Module

For more details on the configuration settings, please refer to the multi-site configuration documentation.

You also need to setup the Git-level replication between nodes. This can be done with either pull or push replication plugin.

HTTP endpoints

For information about available HTTP endpoints please refer to the documentation.