The events-aws-kinesis plugin is configured by adding a plugin stanza in the gerrit.config file, for example:
[plugin "events-aws-kinesis"] numberOfSubscribers = 6 pollingIntervalMs = 500 maxRecords = 99 region = us-east-1 endpoint = http://localhost:4566 applicationName = instance-1 initialPosition = trim_horizon
plugin.events-aws-kinesis.numberOfSubscribers : Optional. The number of expected kinesis subscribers. This will be used to allocate a thread pool able to run all subscribers. Default: 6
plugin.events-aws-kinesis.pollingIntervalMs : Optional. How often, in milliseconds, to poll Kinesis shards to retrieve records. Please note that setting this value too low might incur in ProvisionedThroughputExceededException. See AWS docs for more details on this. Default: 1000
plugin.events-aws-kinesis.maxRecords : Optional. The maximum number of records to fetch from the kinesis stream Default: 100
plugin.events-aws-kinesis.region : Optional. Which AWS region to connect to. Default: When not specified this value is provided via the default Region Provider Chain, as explained here
plugin.events-aws-kinesis.endpoint : Optional. When defined, it will override the default kinesis endpoint and it will connect to it, rather than connecting to AWS. This is useful when developing or testing, in order to connect locally. See localstack to understand more about how run kinesis stack outside AWS. Default:
plugin.events-aws-kinesis.applicationName : Optional. This value identifies the application and it must be unique within your gerrit cluster to allow different gerrit nodes to have different checkpoints and to consume data from the stream independently. Default: events-aws-kinesis
plugin.events-aws-kinesis.initialPosition : Optional. Which point in the stream the consumer should start consuming from. Note that this only applies to consumers that do not have any persisted checkpoint (i.e. first deployments). When checkpoints exist they always override this value.
Needs to be one of these values:
TRIM_HORIZON: Start streaming at the last untrimmed record in the shard, which is the oldest data record in the shard.
LATEST: Start streaming just after the most recent record in the shard, so that you always read the most recent data in the shard.
Default: “LATEST”
plugin.events-aws-kinesis.publishSingleRequestTimeoutMs : Optional. The maximum total time (milliseconds) elapsed between when a publish request started and the receiving a response. If it goes over, the request will be timed-out and possibly tried again, if publishTimeoutMs allows. Default: 6000
plugin.events-aws-kinesis.publishTimeoutMs : Optional. The maximum total time (milliseconds) waiting for publishing a record to kinesis, including retries. If it goes over, the request will be timed-out and not attempted again. Default: 6000
plugin.events-aws-kinesis.recordMaxBufferedTimeMs : Optional. Maximum amount of time (milliseconds) a record may spend being buffered before it gets sent. Records may be sent sooner than this depending on the other buffering limits.
This setting provides coarse ordering among records - any two records will be reordered by no more than twice this amount (assuming no failures and retries and equal network latency). See AWS docs for more details on this. Default: 100
plugin.events-aws-kinesis.consumerFailoverTimeInMs : Optional. Failover time in milliseconds. A worker which does not renew it‘s lease within this time interval will be regarded as having problems and it’s shards will be assigned to other workers.
See AWS docs for more details on this. Default: 10000
plugin.events-aws-kinesis.shutdownTimeoutMs : Optional. The maximum total time (milliseconds) waiting when shutting down kinesis consumers. Default: 20000
plugin.events-aws-kinesis.checkpointIntervalMs : Optional. The interval between checkpoints (milliseconds). Default: 300000 (5 minutes)
plugin.events-aws-kinesis.awsLibLogLevel : Optional. Which level AWS libraries should log at. This plugin delegates most complex tasks associated to the production and consumption of data to libraries developed and maintained directly by AWS. This configuration specifies how verbose those libraries are allowed to be when logging.
Default: WARN Allowed values:OFF|FATAL|ERROR|WARN|INFO|DEBUG|TRACE|ALL
plugin.events-aws-kinesis.streamEventsTopic : Optional. Name of the kinesis topic for stream events. events-aws-kinesis plugin exposes all stream events under this topic name. Default: gerrit
plugin.events-aws-kinesis.sendStreamEvents : Whether to send stream events to the streamEventsTopic topic. Default: false
plugin.events-aws-kinesis.sendAsync : Optional. Whether to send messages to Kinesis asynchronously, without waiting for the result of the operation. Note that in this case, retries will still be attempted by the producer, but in a separate thread, so that Gerrit will not be blocked waiting for the publishing to terminate. The overall result of the operation, once available, will be logged. Default: true
plugin.events-aws-kinesis.profileName : Optional. The name of the aws configuration and credentials profile used to connect to the Kinesis. See Configuration and credential file settings Default: When not specified credentials are provided via the Default Credentials Provider Chain, as explained here
Note that System properties always override and take priority over the above gerrit.config configuration.
The plugin provides an init step that helps to set up the configuration.
*** events-aws-kinesis plugin *** AWS region (leave blank for default provider chain) : AWS endpoint (dev or testing, not for production) : Should send stream events? [y/N]? y Stream events topic [gerrit]: Number of subscribers [6]: Application name [events-aws-kinesis]: Initial position [latest]: Polling Interval (ms) [1000]: Maximum number of record to fetch [100]: The maximum total time waiting for a publish result (ms) [6000]: The maximum total time waiting for publishing, including retries [6000]: The maximum total time waiting when shutting down (ms) [20000]: Which level AWS libraries should log at [WARN]: Should send messages asynchronously? [Y/n]?