High Availability for Google Cloud Pub/Sub

This plugin extends the Gerrit High Availability plugin by using Google Cloud Pub/Sub as the message broker for forwarding events between Gerrit instances instead of HTTP-based direct communication.

Google Cloud Pub/Sub as Message Broker

Google Cloud Pub/Sub provides:

• Decoupling: Instances do not need to know about each other directly or maintain HTTP connections
• Scalability: Automatically scales to handle high message volume
• Reliability: Messages are persisted and retried automatically
• Ordering: Preserves message ordering per topic
• Dead-letter queues: Messages that exceed max receive count are moved to a dead-letter topic for debugging

Prerequisites

• Google Cloud Project with Pub/Sub API enabled
• Google Cloud service account with the following roles:
  • roles/pubsub.editor - To create and manage topics and subscriptions
  • roles/pubsub.publisher - To publish messages
  • roles/pubsub.subscriber - To consume messages
• Service account credentials (JSON key file or Application Default Credentials)

How It Works

Instead of the base plugin's HTTP-based peer communication model, this GCP extension:

1. Publishes all HA events (cache evictions, index updates, stream events) to a Google Cloud Pub/Sub topic
2. Each Gerrit instance subscribes to the topic via its own subscription
3. Messages are processed asynchronously by each instance's subscription handler
4. Failed messages are automatically retried per Pub/Sub's retry policy
5. Messages exceeding the max receive count are moved to a dead-letter topic

Monitoring and Debugging

Failed messages can be investigated via the dead-letter topic. Use Google Cloud Console or gcloud pubsub CLI commands to:

• View subscription lag and processing rates
• Inspect dead-lettered messages
• Monitor message throughput and latency

For troubleshooting steps, refer to config.md and Google Cloud Pub/Sub documentation.