Google Git
Sign in
gerrit / plugins / multi-site / 9829031ee9dd47600bfc1749d99f2737ac349d64
commit9829031ee9dd47600bfc1749d99f2737ac349d64[log] [tgz]
authorMarcin Czech <maczech@gmail.com>Mon Dec 13 18:17:41 2021 +0100
committerMarcin Czech <maczech@gmail.com>Wed Dec 15 10:17:26 2021 +0000
tree22c4a15d4c72a79ec6be90599be5d7224fa3381d
parent5890e16ec75053c4d82c1d596e415c2401e5d047 [diff]
Fix issue with skipping events from high-availability

ForwarderTask class keeps caller thread object to identify the source
of the event. All events generated by `HTTP POST
/plugins/high-availability/index/change/...` thread are skipped.

The issue is that all threads in HTTP thread pool are named `HTTP-*`
when HTTP call is made thread is renamed to `HTTP POST
/plugins/high-availability/index/change/...`. During the processing
new thread to send the message is created and current HTTP thread is
returned to the pool and it's name is set back to `HTTP-*`.

This means that if the code is not fast enough to check HTTP thread
name before thread is returned to the pool it will get `HTTP-*`
instead of `HTTP POST /plugins/high-availability/index/change/...`
and the event will be forwarded. To avoid this situation keep the
caller thread name instead of thread object.

Bug: Issue 15416
Change-Id: I76f5921793cf844ad638f920127e8f2848a2615e
2 files changed
tree: 22c4a15d4c72a79ec6be90599be5d7224fa3381d
  1. .settings/
  2. dockerised_local_env/
  3. images/
  4. setup_local_env/
  5. src/
  6. .bazelrc
  7. .gitignore
  8. .mailmap
  9. BUILD
  10. DESIGN.md
  11. docker-compose.kafka-broker.yaml
  12. external_plugin_deps.bzl
  13. Jenkinsfile
  14. LICENSE
  15. README.md
README.md

Gerrit multi-site plugin

This plugin allows to deploy a distributed cluster of multiple Gerrit masters each using a separate site without sharing any storage. The alignment between the masters happens using the replication plugin and an external message broker.

Requirements for the Gerrit masters are:

  • Gerrit v2.16.5 or later
  • Migrated to NoteDb
  • Connected to the same message broker
  • Accessible via a load balancer (e.g. HAProxy)

NOTE: The multi-site plugin will not start if Gerrit is not yet migrated to NoteDb.

Currently, the only mode supported is one primary read/write master and multiple read-only masters but eventually the plan is to support multiple read/write masters. The read/write master is handling any traffic while the read-only masters are serving the Gerrit GUI assets, the HTTP GET REST API and git fetch requests (git-upload-pack). The read-only masters are kept synchronized with the read/write master in order to be always ready to become a read/write master.

For more details on the overall multi-site design and roadmap, please refer to the multi-site plugin DESIGN.md document

License

This plugin is released under the same Apache 2.0 license and copyright holders as of the Gerrit Code Review project.

How to build

The multi-site plugin can only be built in tree mode, by cloning Gerrit and the multi-site plugin code, and checking them out on the desired branch.

Example of cloning Gerrit and multi-site for a stable-2.16 build:

git clone -b stable-2.16 https://gerrit.googlesource.com/gerrit
git clone -b stable-2.16 https://gerrit.googlesource.com/plugins/multi-site

cd gerrit/plugins
ln -s ../../multi-site .
rm external_plugin_deps.bzl
ln -s multi-site/external_plugin_deps.bzl .

Example of building the multi-site plugin:

cd gerrit
bazel build plugins/multi-site

The multi-site.jar plugin is generated to bazel-bin/plugins/multi-site/multi-site.jar.

Example of testing the multi-site plugin:

cd gerrit
bazel test plugins/multi-site:multi_site_tests

NOTE: The multi-site tests include also the use of Docker containers for instantiating and using a Kafka/Zookeeper broker. Make sure you have a Docker daemon running (/var/run/docker.sock accessible) or a DOCKER_HOST pointing to a Docker server.

How to configure

Install the multi-site plugin into the $GERRIT_SITE/lib directory of all the Gerrit servers that are part of the multi-site cluster. Create a symbolic link from $GERRIT_SITE/lib/multi-site.jar into the $GERRIT_SITE/plugins.

Add the multi-site module to $GERRIT_SITE/etc/gerrit.config as follows:

[gerrit]
  installModule = com.googlesource.gerrit.plugins.multisite.Module

Create the $GERRIT_SITE/etc/multi-site.config on all Gerrit servers with the following basic settings:

[kafka]
  bootstrapServers = <kafka-host>:<kafka-port>

[kafka "publisher"]
  enabled = true

[kafka "subscriber"]
  enabled = true

[ref-database]
  enabled = true

[ref-database "zookeeper"]
  connectString = "localhost:2181"

For more details on the configuration settings, please refer to the multi-site configuration documentation.

You also need to setup the Git-level replication between nodes, for more details please refer to the replication plugin documentation.