Handle fetch tasks gracefully during shutdown
When the pull-replication receives a shutdown request, it currently
interrupts all fetch tasks that are in execution as well as
dropping the pending, scheduled and retrying ones.
This causes those fetch events to be lost and the Gerrit node to be
out-of-sync.
To shutdown gracefully two actions are required:
1. Stop getting new requests (via HTTP and/or via broker)
2. Allow pending, retrying and submitted tasks to terminate
Address point 1. by atomically setting a global shutdown state, so that
the EventsBrokerMessageConsumer can disconnect from the broker (and thus
stop consuming).
Note that the HTTP layer does not require any explicit handling, since
the HTTP servlets and filters exposed by the pull-replication plugin
will unregistered by Gerrit itself, so that no new requests can be
received over HTTP (the client will receive 404s or 403s depending on
the authentication method) as soon as the shutdown request is initiated.
Address point 2. by explicitly waiting until the pending, retrying and
executing fetch tasks are completed.
The maximum amount to wait for is configurable via replication.config
via the new configuration value: `replication.shutDownDrainTimeout`.
Bug: Issue 298084095
Change-Id: Ic672c7df5f52f6cdfcd6e0c8a8cd029c51a88ec5
8 files changed
