Merge branch 'stable-3.4' into stable-3.5

* stable-3.4:
  Register Kafka consumers with external groupId when plugin starts
  Make KafkaBrokerApi class implement ExtendedBrokerApi interface
  Consume events-broker from source
  Add validation dependency on events-broker module
  Add .gitignore file in the project
  Pass correct amount of arguments to Malformed event log line
  Add Kafka REST API authentication
  Fix the topic events replay Kafka REST-API
  Use Kafka REST Proxy id to subscribe to the correct instance
  Fix Kafka REST Proxy accepts header for topic meta-data
  Kafka REST Client: avoid clashes between clients
  Fix threshold of HTTP wire logging
  Delete subscription at the end of ReceiverJob
  Update kafka-client 2.1.0 -> 2.1.1
  Increase patience to 30s for shouldReplayAllEvents test
  Remove unused RequestConfigProvider
  REST ClientType: Make thread pool and timeouts configuration
  Extract configuration properties into constants
  Manage Kafka clientType when starting session
  Receive messages through Kafka REST API
  Send messages through Kafka REST API
  Abstract Publisher/Subscriber into generic interfaces
  Wait at most for 5s for an empty topic
  Assert that messages are acknowledged in KafkaBrokerApiTest
  Add Kafka REST-API container in test
  Remove access to deprecated poll(long) method
  Use explicit Kafka image:tag in tests
  Do not connect KafkaSession without bootstrap servers

Change-Id: I120c9fffe052195f31b1132e0c6fc0fd35680840
tree: 6bb00e8ecaff8d75fc314a8d180975f0e61c40e9
  1. src/
  2. .gitignore
  3. BUILD
  4. external_plugin_deps.bzl
  5. Jenkinsfile
  6. LICENSE
  7. README.md
README.md

Kafka: Gerrit event producer for Apache Kafka

Build Status

Synopsis

This plugins allows to define a distributed stream of events published by Gerrit.

Events can be anything, from the traditional stream events to the Gerrit metrics.

This plugin requires Gerrit 2.13 or later.

Environments

  • linux
  • java-1.8
  • Bazel

Build

Kafka plugin can be build as a regular ‘in-tree’ plugin. That means that is required to clone a Gerrit source tree first and then to have the Kafka plugin source directory into the /plugins path. The plugin depends on events-broker which is linked directly from source with the same ‘in-tree’ plugin structure.

Additionally, the plugins/external_plugin_deps.bzl file needs to be updated to match the Kafka plugin one.

git clone --recursive https://gerrit.googlesource.com/gerrit
git clone https://gerrit.googlesource.com/plugins/events-kafka gerrit/plugins/events-kafka
git clone https://gerrit.googlesource.com/modules/events-broker gerrit/plugins/events-broker
cd gerrit
rm plugins/external_plugin_deps.bzl
ln -s ./events-kafka/external_plugin_deps.bzl plugins/.

To build the events-kafka plugins, issue the command from the Gerrit source path:

bazel build plugins/events-kafka

The output is created in

bazel-genfiles/plugins/events-kafka/events-kafka.jar
bazel-genfiles/plugins/events-broker/events-broker.jar

Minimum Configuration

Assuming a running Kafka broker on the same Gerrit host, add the following settings to gerrit.config:

  [plugin "events-kafka"]
    bootstrapServers = localhost:9092