Google Git
Sign in
gerrit / aws-gerrit / 6495ea99fb40b151e9143c4803fd78a8742653b1 / . / dual-master
tree: b35ea5b503b10cf7666bada79300014da4934abd [path history] [tgz]
  1. haproxy/
  2. syslog-sidecar/
  3. cf-cluster.yml
  4. cf-dns-route.yml
  5. cf-service-lb.yml
  6. cf-service-master.yml
  7. cf-service-slave.yml
  8. Makefile
  9. README.md
  10. setup.env.template
dual-master/README.md

Gerrit dual-master in High-Availability

This set of templates provides all the components to deploy a Gerrit dual-master in HA in ECS. The 2 masters will share the Git repositories via NFS, using EFS.

Architecture

The following templates are provided in this example:

  • cf-cluster: define the ECS cluster and the networking stack
  • cf-service-master: define the service stack running the gerrit master
  • cf-dns-route: define the DNS routing for the service
  • cf-service-slave: define the service stack running the gerrit replica
  • cf-service-lb: define the LBs in front of gerrit masters (this includes haproxy as well as NLB)

Networking

  • Single VPC:
  • CIDR: 10.0.0.0/16
  • Single Availability Zone
  • 1 public Subnets:
  • CIDR: 10.0.0.0/24
  • 1 public NLB exposing:
  • Gerrit master 1 HTTP on port 8080
  • Gerrit master 1 SSH on port 29418
  • 1 public NLB exposing:
  • Gerrit master 2 HTTP on port 8081
  • Gerrit master 2 SSH on port 39418
  • 1 Internet Gateway
  • 2 type A alias DNS entry, for Gerrit master 1 and 2
  • A wildcard SSL certificate available in AWS Certificate Manager

Data persistency

  • EBS volumes for:
    • Indexes
    • Caches
    • Logs
  • EFS volume:
    • Share Git repositories between masters
    • Share Web sessions between masters

NOTE: This stack uses EFS in provisioned mode, which is a better setting for large repos (> 1GB uncompressed) since it provides a lower latency compared to the burst mode. However, it has some costs associated. If you are dealing with small repos, you can switch to burst mode.

Deployment type

  • Latest Gerrit version deployed using the official Docker image
  • Application deployed in ECS on a single EC2 instance

Logging

  • All the logs are forwarded to AWS CloudWatch in the LogGroup with the cluster stack name

Monitoring

  • Standard CloudWatch monitoring metrics for each component
  • Prometheus and Grafana stack is currently not available for dual-master, but a change is in progress to allow this (see Issue 12979)

How to run it

0 - Prerequisites

Follow the steps described in the Prerequisites section

1 - Configuration

Each recipe provides a setup.env.template file which is a template for configuring the Gerrit stacks. Copy that into a setup.env and set the correct values for the provided environment variables.

cp setup.env.template setup.env

This is the list of available parameters:

Environment

Configuration values affecting deployment environment and cluster properties

  • DOCKER_REGISTRY_URI: Mandatory. URI of the Docker registry. See the prerequisites section for more details.
  • SSL_CERTIFICATE_ARN: Mandatory. ARN of the wildcard SSL Certificate, covering both master nodes.
  • CLUSTER_STACK_NAME: Optional. Name of the cluster stack. gerrit-cluster by default.
  • SERVICE_MASTER1_STACK_NAME: Optional. Name of the master 1 service stack. gerrit-service-master-1 by default.
  • SERVICE_MASTER2_STACK_NAME: Optional. Name of the master 2 service stack. gerrit-service-master-2 by default.
  • DNS_ROUTING_STACK_NAME: Optional. Name of the DNS routing stack. gerrit-dns-routing by default.
  • HOSTED_ZONE_NAME: Optional. Name of the hosted zone. mycompany.com by default.
  • MASTER1_SUBDOMAIN: Optional. Name of the master 1 sub domain. gerrit-master-1-demo by default.
  • MASTER2_SUBDOMAIN: Optional. Name of the master 2 sub domain. gerrit-master-2-demo by default.
  • CLUSTER_DESIRED_CAPACITY: Optional. Number of EC2 instances composing the cluster. 1 by default.
  • GERRIT_KEY_PREFIX : Optional. Secrets prefix used during the Import into AWS Secret Manager. gerrit_secret by default.

SPECS

Configuration values to spec up Gerrit containers.

  • GERRIT_RAM: RAM allocated (MiB) to the Gerrit container. 70000 by default.
  • GERRIT_CPU: vCPU units allocated to the Gerrit container. 10240 by default.
  • GERRIT_HEAP_LIMIT: Maximum heap size of the Java process running Gerrit, in bytes. See Gerrit documentation 35g by default.
  • JGIT_CACHE_SIZE: Maximum number of bytes to load and cache in memory from pack files. See Gerrit documentation for more details. 12g by default.

LDAP

Configuration values related to LDAP integration. See more details here

  • LDAP_SERVER: Mandatory. URL of the organization’s LDAP server to query for user information and group membership from See Gerrit documentation
  • LDAP_USERNAME: Mandatory. Username to bind to the LDAP server with See Gerrit documentation
  • LDAP_ACCOUNT_BASE: Mandatory. Root of the tree containing all user accounts See Gerrit documentation
  • LDAP_GROUP_BASE: Mandatory. Root of the tree containing all group objects See Gerrit documentation

SMTP

Configuration values related to SMTP integration. See more details here

  • SMTP_SERVER: Mandatory. Hostname (or IP address) of a SMTP server that will relay messages generated by Gerrit to end users See Gerrit documentation
  • SMTP_USER: Mandatory. User name to authenticate with See Gerrit documentation
  • SMTP_DOMAIN: Mandatory. Domain to be used in the “From” field of any generated email messages See Gerrit documentation

2 - Deploy

  • Create the cluster, services and DNS routing stacks:
make create-all

It might take several minutes to build the stack. You can monitor the creations of the stacks in CloudFormation

  • NOTE: the creation of the cluster needs an EC2 key pair are useful when you need to connect to the EC2 instances for troubleshooting purposes. The key pair is automatically generated and stored in a pem file on the current directory. To use when ssh-ing into your instances as follow: ssh -i cluster-keys.pem ec2-user@<ec2_instance_ip>

Cleaning up

make delete-all

Note that this will not delete:

  • Secrets stored in Secret Manager
  • SSL certificates
  • ECR repositories

Access your Gerrit instances

Get the URL of your Gerrit master instances this way:

aws cloudformation describe-stacks \
  --stack-name <SERVICE_MASTER1_STACK_NAME> \
  | grep -A1 '"OutputKey": "CanonicalWebUrl"' \
  | grep OutputValue \
  | cut -d'"' -f 4

aws cloudformation describe-stacks \
  --stack-name <SERVICE_MASTER2_STACK_NAME> \
  | grep -A1 '"OutputKey": "CanonicalWebUrl"' \
  | grep OutputValue \
  | cut -d'"' -f 4

Gerrit master instance ports:

  • HTTP 8080
  • SSH 29418

External Services

If you need to setup some external services (maybe for testing purposes, such as SMTP or LDAP), you can follow the instructions here

Docker

Refer to the Docker section for information on how to setup docker or how to publish images