Add export-logs operation for all recipes

Allow to export logs to S3 by issuing the command:

```
make eport-logs
```

Bug: Issue 14412
Change-Id: I1574c6969bdef307aae3c88d1cc32063f2fbff84
6 files changed
tree: b11193f572a6df198d0e2a52c606e1703e4ff6d7
  1. common-templates/
  2. dual-primary/
  3. geo-location-routing/
  4. gerrit/
  5. ldap/
  6. load-test-fleet/
  7. maintenance/
  8. monitoring/
  9. operations/
  10. primary-replica/
  11. single-primary/
  12. .gitignore
  13. common.env
  14. Configuration.md
  15. Docker.md
  16. LICENSE
  17. Makefile.common
  18. Prerequisites.md
  19. README.md
  20. Secrets.md
README.md

Gerrit AWS Templates

Those are a collection of AWS CloudFormation templates and scripts to deploy Gerrit in AWS.

The aim is to provide some guidelines and example on how to deploy different Gerrit setups in the Cloud using AWS as provider.

Outline

Overview

The goal of Gerrit AWS Templates is to provide fully-functional Gerrit installations to helps users deploying Gerrit on AWS by providing out-of-the-box templates.

With Gerrit AWS Templates, developers and administrator can create a production-ready installation on the cloud in minutes and in a repeatable way, allowing them to focus on fine tuning of the Gerrit configuration to suit the user needs.

The provided CloudFormation templates automate the entire creation and deployment of the infrastructure and the application.

Pre-requisites

To manage your AWS services via command line you will need to install AWS CLI and set it up to point to your account.

To build gerrit and related-components' images Docker

To manipulate aws cloudformation outputs jq

Templates

External services

This is a list of external services that you might need to setup your stack and some suggestions on how to easily create them.

SMTP Server

If you need to setup a SMTP service Amazon Simple Email Service can be used. Details how setup Amazon SES can be found here.

To correctly setup email notifications Gerrit requires ssl protocol on default port 465 to be enabled on SMTP Server. It is possible to setup Gerrit to talk to standard SMTP port 25 but by default all EC2 instances are blocking it. To enable port 25 please follow this link.

LDAP Server

If you need a testing LDAP server you can find details on how to easily create one in the LDAP folder.

Logging

All recipes stream every log to CloudWatch. This always includes sshd_log, httpd_log and gc_log.

error_log

The ‘error_log’ might or might not be available depending on which version of gerrit is being deployed. From gerrit 3.3 it will always be available. Prior to that it will be available from:

  • stable-3.0 -> starting from 3.0.13
  • stable-3.1 -> starting from 3.1.10
  • stable-3.2 -> starting from 3.2.5

When the error_log is not available, Gerrit will still output the same content to standard error. Refer to the standard error section.

Standard error

Different recipes deploy different services to ECS (please refer to the documentation of each recipe for details on what services are actually deployed).

Every ECS service will stream anything outputted to stderr to cloudwatch, to a stream name that will take the form of:

{environmentName}/{serviceName}/{taskId}

For example, given the gerrit-primary service running task bb21cb504ca44150b770ca05e922e332, on the test environment, the stderr will be streamed to:

test/gerrit-primary/bb21cb504ca44150b770ca05e922e332

The task name can be found in the Amazon ECS console's Task section.

Operations

A set of utilities to perform operational tasks is also provided. Refer to the relevant documentation for details on this.