Auto-repair replication on missing necessary objects error
When a replication push fails because the remote git-receive-pack
reports 'missing necessary objects', the destination is likely corrupt
and requires manual intervention. Detect that error and, when enabled,
schedule an asynchronous repair.
Repair only runs for plain SSH destinations (not gerrit+ssh, http, or
git protocols) since rsync over SSH is required. Attempts are tracked
per project+destination and gated by three new config options:
* autoRepairMaxAttempts (default 0, disabled): maximum number of
auto-repair attempts per project on each destination URL
* autoRepairInterval (default 3days): minimum spacing between
attempts for the same (project, destination)
* autoRepairConcurrencyLimit (default 2): maximum number of
auto-repair tasks allowed to run concurrently
Tracker state is in-memory and resets on plugin reload or Gerrit
restart, so the cap is best-effort rather than persistent. Enabling
auto-repair in a clustered deployment can lead to redundant repairs as
state is not shared across nodes. This could be improved later if
desired, perhaps by persisting the state using the replication task
storage.
Auto-repair deliberately uses the failing destination URI instead of
round-robin failover, since repairs target corruption errors and are
more likely to succeed on the URL that just reported the problem.
Release-Notes: Auto-repair replication on missing objects errors
Change-Id: I815f620af93b484fa801d0c953d56b6a6b00be66
12 files changed