Add ability to publish cloudwatch metrics to dual-master recipe
dual-master recipe can now be configured so that each gerrit instance
can send metrics to cloudwatch.
Note that metrics are not labelled by which instance actually
published them.
The metrics-reporter-cloudwatch at the moment does not support this
functionality, but a ticket has been created to address this issue [1],
so that this can be addressed at a later stage.
[1] https://bugs.chromium.org/p/gerrit/issues/detail?id=13217
Feature: Issue 13212
Change-Id: Idee8deacea2ad38a8983faf340abc7e3093212c5
diff --git a/dual-master/Makefile b/dual-master/Makefile
index 1800e57..7c03175 100644
--- a/dual-master/Makefile
+++ b/dual-master/Makefile
@@ -50,7 +50,7 @@
ParameterKey=SubnetIdProp,ParameterValue=$(SUBNET_ID) \
$(CLUSTER_OPTIONAL_PARAMS)
-service-master-1:
+service-master-1: set-optional-params-metrics-cloudwatch
$(AWS_FC_COMMAND) create-stack \
--stack-name $(SERVICE_MASTER1_STACK_NAME) \
--capabilities CAPABILITY_IAM \
@@ -86,9 +86,10 @@
ParameterKey=GerritRAM,ParameterValue=$(GERRIT_RAM) \
ParameterKey=GerritCPU,ParameterValue=$(GERRIT_CPU) \
ParameterKey=GerritHeapLimit,ParameterValue=$(GERRIT_HEAP_LIMIT) \
- ParameterKey=JgitCacheSize,ParameterValue=$(JGIT_CACHE_SIZE)
+ ParameterKey=JgitCacheSize,ParameterValue=$(JGIT_CACHE_SIZE) \
+ $(METRICS_CW_OPTIONAL_PARAMS)
-service-master-2:
+service-master-2: set-optional-params-metrics-cloudwatch
$(AWS_FC_COMMAND) create-stack \
--stack-name $(SERVICE_MASTER2_STACK_NAME) \
--capabilities CAPABILITY_IAM \
@@ -125,9 +126,10 @@
ParameterKey=GerritRAM,ParameterValue=$(GERRIT_RAM) \
ParameterKey=GerritCPU,ParameterValue=$(GERRIT_CPU) \
ParameterKey=GerritHeapLimit,ParameterValue=$(GERRIT_HEAP_LIMIT) \
- ParameterKey=JgitCacheSize,ParameterValue=$(JGIT_CACHE_SIZE)
+ ParameterKey=JgitCacheSize,ParameterValue=$(JGIT_CACHE_SIZE) \
+ $(METRICS_CW_OPTIONAL_PARAMS)
-service-slave:
+service-slave: set-optional-params-metrics-cloudwatch
$(AWS_FC_COMMAND) create-stack \
--stack-name $(SERVICE_SLAVE_STACK_NAME) \
--capabilities CAPABILITY_IAM \
@@ -149,7 +151,8 @@
ParameterKey=GerritRAM,ParameterValue=$(GERRIT_RAM) \
ParameterKey=GerritCPU,ParameterValue=$(GERRIT_CPU) \
ParameterKey=GerritHeapLimit,ParameterValue=$(GERRIT_HEAP_LIMIT) \
- ParameterKey=JgitCacheSize,ParameterValue=$(JGIT_CACHE_SIZE)
+ ParameterKey=JgitCacheSize,ParameterValue=$(JGIT_CACHE_SIZE) \
+ $(METRICS_CW_OPTIONAL_PARAMS)
service-lb:
ifdef LOAD_BALANCER_SCHEME
diff --git a/dual-master/README.md b/dual-master/README.md
index c8d3fdb..3e7fc22 100644
--- a/dual-master/README.md
+++ b/dual-master/README.md
@@ -57,6 +57,7 @@
### Monitoring
* Standard CloudWatch monitoring metrics for each component
+* Application level CloudWatch monitoring can be enabled as described [here](#cloudwatch-monitoring)
* Prometheus and Grafana stack is currently not available for dual-master, but a change is in progress to allow this
(see [Issue 12979](https://bugs.chromium.org/p/gerrit/issues/detail?id=12979))
@@ -92,6 +93,27 @@
* `FILESYSTEM_PROVISIONED_THROUGHPUT_IN_MIBPS`: Optional. Only used when `FILESYSTEM_THROUGHPUT_MODE` is set to `provisioned`.
default: `256`.
+#### CloudWatch Monitoring
+
+Application level metrics for CloudWatch are available through the
+[metrics-reporter-cloudwatch](https://gerrit.googlesource.com/plugins/metrics-reporter-cloudwatch/)
+plugin.
+
+* `METRICS_CLOUDWATCH_ENABLED`: Optional - Boolean.
+Whether to publish metrics to CloudWatch. Default: false
+* `METRICS_CLOUDWATCH_NAMESPACE`: Optional - String.
+The CloudWatch namespace for Gerrit metrics. Default: _gerrit_
+* `METRICS_CLOUDWATCH_RATE`: Optional - String.
+The rate at which metrics should be fired to AWS. Default: _60s_
+* `METRICS_CLOUDWATCH_INITIAL_DELAY`: Optional - String.
+The time to delay the first reporting execution. Default: _0_
+* `METRICS_CLOUDWATCH_JVM_ENABLED`: Optional - Boolean.
+Publish JVM metrics. Default: _false_
+* `METRICS_CLOUDWATCH_DRY_RUN`: Optional - Boolean.
+Log.DEBUG the metrics, rather than publishing. Default: _false_
+* `METRICS_CLOUDWATCH_EXCLUDE_METRICS_LIST`: Optional. Comma-separated list.
+ Regex patterns to exclude from publishing. Default: empty string.
+
### 2 - Deploy
* Create the cluster, services and DNS routing stacks:
diff --git a/dual-master/cf-service-master.yml b/dual-master/cf-service-master.yml
index f4ecec5..75e0020 100644
--- a/dual-master/cf-service-master.yml
+++ b/dual-master/cf-service-master.yml
@@ -160,6 +160,37 @@
SMTPDomain:
Description: Domain to be used in the From field
Type: String
+ MetricsCloudwatchEnabled:
+ Description: Whether gerrit metrics should be published to cloudwatch
+ Type: String
+ Default: false
+ AllowedValues: [true, false]
+ MetricsCloudwatchNamespace:
+ Description: The CloudWatch namespace for Gerrit metrics
+ Type: String
+ Default: gerrit
+ MetricsCloudwatchRate:
+ Description: The rate at which metrics should be fired to AWS
+ Type: String
+ Default: 60s
+ MetricsCloudwatchInitialDelay:
+ Description: The time to delay the first reporting execution
+ Type: String
+ Default: 0
+ MetricsCloudwatchJVMEnabled:
+ Description: Whether JVM metrics shoiuld be published to cloudwatch
+ Type: String
+ Default: false
+ AllowedValues: [true, false]
+ MetricsCloudwatchDryRun:
+ Description: The reporter will log.DEBUG the metrics, instead of doing a real POST to CloudWatch
+ Type: String
+ Default: false
+ AllowedValues: [true, false]
+ MetricsCloudwatchExcludeMetrics:
+ Description: Comma separated list of regex patterns to exclude metrics reported to CloudWatch
+ Type: CommaDelimitedList
+ Default: ''
Resources:
Service:
@@ -235,6 +266,20 @@
Value: !Ref GitSSHPort
- Name: SLAVE_SUBDOMAIN
Value: !Ref SlaveSubdomain
+ - Name: METRICS_CLOUDWATCH_ENABLED
+ Value: !Ref MetricsCloudwatchEnabled
+ - Name: METRICS_CLOUDWATCH_NAMESPACE
+ Value: !Ref MetricsCloudwatchNamespace
+ - Name: METRICS_CLOUDWATCH_RATE
+ Value: !Ref MetricsCloudwatchRate
+ - Name: METRICS_CLOUDWATCH_INITIAL_DELAY
+ Value: !Ref MetricsCloudwatchInitialDelay
+ - Name: METRICS_CLOUDWATCH_JVM_ENABLED
+ Value: !Ref MetricsCloudwatchJVMEnabled
+ - Name: METRICS_CLOUDWATCH_DRY_RUN
+ Value: !Ref MetricsCloudwatchDryRun
+ - Name: METRICS_CLOUDWATCH_EXCLUDE_METRICS_LIST
+ Value: !Join [',', !Ref MetricsCloudwatchExcludeMetrics]
MountPoints:
- SourceVolume: !Ref GerritGitVolume
ContainerPath: /var/gerrit/git
diff --git a/dual-master/cf-service-slave.yml b/dual-master/cf-service-slave.yml
index 02ef28c..2d87350 100644
--- a/dual-master/cf-service-slave.yml
+++ b/dual-master/cf-service-slave.yml
@@ -133,6 +133,37 @@
LDAPGroupBase:
Description: Root of the tree containing all group objects
Type: String
+ MetricsCloudwatchEnabled:
+ Description: Whether gerrit metrics should be published to cloudwatch
+ Type: String
+ Default: false
+ AllowedValues: [true, false]
+ MetricsCloudwatchNamespace:
+ Description: The CloudWatch namespace for Gerrit metrics
+ Type: String
+ Default: gerrit
+ MetricsCloudwatchRate:
+ Description: The rate at which metrics should be fired to AWS
+ Type: String
+ Default: 60s
+ MetricsCloudwatchInitialDelay:
+ Description: The time to delay the first reporting execution
+ Type: String
+ Default: 0
+ MetricsCloudwatchJVMEnabled:
+ Description: Whether JVM metrics shoiuld be published to cloudwatch
+ Type: String
+ Default: false
+ AllowedValues: [true, false]
+ MetricsCloudwatchDryRun:
+ Description: The reporter will log.DEBUG the metrics, instead of doing a real POST to CloudWatch
+ Type: String
+ Default: false
+ AllowedValues: [true, false]
+ MetricsCloudwatchExcludeMetrics:
+ Description: Comma separated list of regex patterns to exclude metrics reported to CloudWatch
+ Type: CommaDelimitedList
+ Default: ''
Resources:
GerritService:
@@ -196,6 +227,20 @@
Value: !Ref LDAPAccountBase
- Name: LDAP_GROUP_BASE
Value: !Ref LDAPGroupBase
+ - Name: METRICS_CLOUDWATCH_ENABLED
+ Value: !Ref MetricsCloudwatchEnabled
+ - Name: METRICS_CLOUDWATCH_NAMESPACE
+ Value: !Ref MetricsCloudwatchNamespace
+ - Name: METRICS_CLOUDWATCH_RATE
+ Value: !Ref MetricsCloudwatchRate
+ - Name: METRICS_CLOUDWATCH_INITIAL_DELAY
+ Value: !Ref MetricsCloudwatchInitialDelay
+ - Name: METRICS_CLOUDWATCH_JVM_ENABLED
+ Value: !Ref MetricsCloudwatchJVMEnabled
+ - Name: METRICS_CLOUDWATCH_DRY_RUN
+ Value: !Ref MetricsCloudwatchDryRun
+ - Name: METRICS_CLOUDWATCH_EXCLUDE_METRICS_LIST
+ Value: !Join [',', !Ref MetricsCloudwatchExcludeMetrics]
MountPoints:
- SourceVolume: !Ref GerritGitVolume
ContainerPath: /var/gerrit/git
diff --git a/dual-master/setup.env.template b/dual-master/setup.env.template
index 2c6638a..a4bba00 100644
--- a/dual-master/setup.env.template
+++ b/dual-master/setup.env.template
@@ -30,3 +30,11 @@
SMTP_SERVER:=yoursmtp.yourcompany.com
SMTP_USER:=smtpuser
SMTP_DOMAIN:=mail.yourcompany.com
+
+METRICS_CLOUDWATCH_ENABLED:=true
+METRICS_CLOUDWATCH_NAMESPACE:=gerrit
+METRICS_CLOUDWATCH_RATE:=60s
+METRICS_CLOUDWATCH_INITIAL_DELAY:=0s
+METRICS_CLOUDWATCH_JVM_ENABLED:=true
+METRICS_CLOUDWATCH_DRY_RUN:=false
+METRICS_CLOUDWATCH_EXCLUDE_METRICS_LIST:=foo.*,bar.*
