commit b74517d966577cd92e430ee355097e3097cdfc46
author Luca Milanesio <luca.milanesio@gmail.com> Wed Aug 14 01:36:19 2019 +0100
committer Gerrit Code Review <noreply-gerritcodereview@google.com> Wed Aug 14 00:36:19 2019 +0000
tree fdf2699cd6a74869ae380d45b71d1940521b9e11
parent 3e7cbddfda3e16cf07753cb2c1886cb999c300fe

Update git submodules

* Update plugins/replication from branch 'master'
  to 90253f88e42653a0c3af632beae507847f71adec
  - Merge branch 'stable-3.0'
    
    * stable-3.0:
      Revert "Revert "Do not reload config when queue is not ready""
      Fix auto-reload test by mocking a running replicationQueue
      Add replication refs-filtering before push
      Create missing repository only at the remote we are pushing to
      Remove the NewProjectCreatedListener implementation
      Consistently handle remote repository creation failures
      Revert "Do not reload config when queue is not ready"
      Do not reload config when queue is not ready
      AbstractConfigTest: Import createTempDirectory static
      Fix creation of missing repository when replicating to a Gerrit server
      Revert "Remove replication event from pending when runway is allowed"
      Trace details of the replication events cancelled
      ReplicationFileBasedConfig: Fix setting default sshConnectionTimeout
      SshHelper: Add class javadoc
      Make more classes and fields public to ease extensibility
      Allow to configure timeout for SSH connection and commands
      StartCommand:  Fix synchronization on non-final field
      Destination: Suppress FutureReturnValueIgnored warning
      Cancel auto-reload runnable at configuration shutdown
      Cancel pending replications upon shutdown
      Remove replication event from pending when runway is allowed
    
    Change-Id: Idfa32db6de2c61eb606cacafce3d2127a8def6dc
    
  - Merge branch 'stable-2.16' into stable-3.0
    
    * stable-2.16:
      Revert "Revert "Do not reload config when queue is not ready""
      Add replication refs-filtering before push
    
    Change-Id: I99b0902b8f909745c3eb00490d615935cec62db1
    
  - Revert "Revert "Do not reload config when queue is not ready""
    
    This reverts commit bcb15e00b3bd8804417d7401cff12a941bdb8aee.
    The test that was previously failing on stable-3.0 has been fixed
    with Change-Id: I2bef00e701 and thus does not represent anymore an
    impediment for getting this fix merged in stable-2.16 and stable-3.0.
    
    Change-Id: I936e3547ff4533850d538fe6b9fada7437bf4e49
    
  - Fix auto-reload test by mocking a running replicationQueue
    
    Prepare the mocking of the replicationQueue by simulating a running
    state. This enables the correct merging of the fix Change-Id: I936e3547ff
    done in stable-2.16 onto the stable-3.0 branch, keeping the test
    functionality.
    
    Change-Id: I2bef00e701f20b3c84448f822412768357b32724
    
  - Add replication refs-filtering before push
    
    Add ability to filter out refs before being pushed for replication to
    remote instance. This will help to prevent split-brain issue by
    allowing us to create filter in multi-site plugin to stop the out of sync
    instance from overriding the changes of the instance that is up to date.
    
    Bug: Issue 11175
    Change-Id: I99a7cb4fb2f231626d50abc7d90bb3d79c23861a
    
  - Merge branch 'stable-2.16' into stable-3.0
    
    * stable-2.16:
      Create missing repository only at the remote we are pushing to
      Remove the NewProjectCreatedListener implementation
      Consistently handle remote repository creation failures
      Fix creation of missing repository when replicating to a Gerrit server
    
    Change-Id: I6c1e0ceabb90817fc3938945366b11933f50d93d
    
  - Create missing repository only at the remote we are pushing to
    
    When replication of a ref failed due to missing remote repository at one
    remote site, we then initiated creation of the missing repository on all
    remotes where this repository is replicated. The creation of the missing
    repository is considered to be success if it succeeded for all remotes
    and only in this case the ref replication was rescheduled. This logic
    could cause a ref replication to not be rescheduled when creation of a
    missing repository fails in a remote which is unrelated to the current
    push operation. For example, if we replicate to two remote sites:
    
      [remote "foo"]
        url = https://foo.org/...
        adminUrl = ssh://gerrit@foo.org/...
    
      [remote "bar"]
        url = https://bar.org/...
        adminUrl = ssh://gerrit@bar.org/...
    
    and if during a push to "foo" we find that the repository is missing
    then we will try to create it on both "foo" and "bar". It could happen
    that the creation of the missing repository succeeds on "foo" and then
    fails on "bar" because "bar" is unreachable or down at that moment.
    In this case the replication to "foo" will not be rescheduled but it
    should be as we successfully created the missing repository in "foo".
    
    Create missing repository only under the (admin) URL in the same remote.
    
    Note that there will be one ref replication task for each remote which
    ensures that the missing repository will be created in all remotes.
    
    Change-Id: Idbc3f614df53bb3aabcc4cf6cb0d6540e7000e29
    (cherry picked from commit abafb83c47eb50c0ec5ce9a4b8d04dc355d5a5bf)
    
  - Remove the NewProjectCreatedListener implementation
    
    Initial version of this plugin implemented the NewProjectCreatedListener
    in order to create missing repositories in the replication targets. This
    was the only way how replication plugin created missing repositories.
    
    Since I4e587cdfca09445c9b1c528b2f1edae0944aec68, if during the
    replication of a ref it is found that the repository missing on the
    remote site it will be automatically created. This means, that since
    that change there are two ways a repository is created on the remote
    site:
    1: from the NewProjectCreatedListener.onNewProjectCreated
    2: during ref replication when the repository is missing
    
    There are two major differences in how cases 1 and 2 are invoked. In the
    case 1 the remote repository creation is performed from the calling
    thread, which means from the Gerrit core thread which invokes
    NewProjectCreatedListener(s). In the case 2, the creation of the remote
    repository is done from the replication queue thread which was
    processing the ref replication. Note that replication tasks are created
    with an additional child injector [1] which provides additional bindings
    available for injection into the classes implementing the replication
    and repository creation.  These binding are, however, not available when
    the repository creation is processed from the Gerrit core thread
    invoking NewProjectCreatedListener(s).
    
    Removing the NewProjectCreatedListener implementation removes this
    asymmetry and makes sure that all replication relevant steps, including
    repository creation, are processed from replication queue threads.
    
    [1] https://gerrit.googlesource.com/plugins/replication/+/871da5aa023bdfd03a3b943f7f6b82c7e0f16341/src/main/java/com/googlesource/gerrit/plugins/replication/Destination.java#189
    
    Change-Id: I32e7aa632ffe0e94eb525f7f2c007fbb88569004
    (cherry picked from commit c16fe9c5a6da6c23dbf45a533f788dcc527e209c)
    
  - Consistently handle remote repository creation failures
    
    Failure to create remote repository was handled inconsistently in the
    three existing AdminApi implementations. This issue was present in the
    code even before the refactoring done in
    Ie760bf3e143b1d143b6e81ac6cfa816ef1f8d016. For example, creation of
    remote repository over SSH [1] didn't return a status back to the
    caller, while the Gerrit+SSH implementation [2] did return a boolean
    flag which was afterwards ignored by the caller [3].
    
    Not handling remote repository creation failures had an ugly effect:
    we log the "Missing repository created; ..." [4], even if the repository
    creation failed.
    
    Let all three implementations of the AdminApi return a flag indicating
    the success/failure and make sure to handle it properly.
    
    Further, remove some redundant and potentially confusing parts of the
    log messages as they assume that failure to create missing repository
    can only be caused by using wrong protocol.
    
    [1] https://gerrit.googlesource.com/plugins/replication/+/d557ccc642c59a55750f560ce0d98870e1550d65/src/main/java/com/googlesource/gerrit/plugins/replication/ReplicationQueue.java#290
    [2] https://gerrit.googlesource.com/plugins/replication/+/d557ccc642c59a55750f560ce0d98870e1550d65/src/main/java/com/googlesource/gerrit/plugins/replication/GerritSshApi.java#43
    [3] https://gerrit.googlesource.com/plugins/replication/+/d557ccc642c59a55750f560ce0d98870e1550d65/src/main/java/com/googlesource/gerrit/plugins/replication/ReplicationQueue.java#259
    [4] https://gerrit.googlesource.com/plugins/replication/+/092792edacf9c29732a560a30967b92664cd65f9/src/main/java/com/googlesource/gerrit/plugins/replication/PushOne.java#400
    
    Change-Id: I6566f867138ab73c81ff7a67630772c3734e501b
    (cherry picked from commit b95d921d6db891dc724f619d957ab35e634f7e44)
    
  - Merge branch 'stable-2.15' into stable-2.16
    
    * stable-2.15:
      Fix creation of missing repository when replicating to a Gerrit server
    
    Change-Id: I35712167582fc0a1765707ae13b6aabf85374f33
    
  - Merge branch 'stable-2.16' into stable-3.0
    
    * stable-2.16:
      Revert "Do not reload config when queue is not ready"
      Do not reload config when queue is not ready
      Revert "Remove replication event from pending when runway is allowed"
      ReplicationFileBasedConfig: Fix setting default sshConnectionTimeout
      SshHelper: Add class javadoc
      Make more classes and fields public to ease extensibility
      Allow to configure timeout for SSH connection and commands
      StartCommand:  Fix synchronization on non-final field
      Destination: Suppress FutureReturnValueIgnored warning
    
    Change-Id: I101dbbcfc3f049ebf275f4bff6c4a99e53771416
    
  - Revert "Do not reload config when queue is not ready"
    
    This reverts commit a3cf8a61980a84ae710b15cb93d9a2a7423d93cf.
    
    Breaks the test on stable-3.0. This should be re-done after the
    tests have been backported to stable-2.16.
    
    Change-Id: I147ee42cf3607aa1943a5f57070687a9f0948036
    
  - Merge branch 'stable-2.15' into stable-2.16
    
    * stable-2.15:
      Revert "Remove replication event from pending when runway is allowed"
      ReplicationFileBasedConfig: Fix setting default sshConnectionTimeout
      SshHelper: Add class javadoc
      Make more classes and fields public to ease extensibility
      Allow to configure timeout for SSH connection and commands
      StartCommand:  Fix synchronization on non-final field
      Destination: Suppress FutureReturnValueIgnored warning
    
    Change-Id: I0cbbf0db0bb8d6b44e7f86ec7ad3e0f2067f3a6d
    
  - Do not reload config when queue is not ready
    
    Do not stop/start the replication queue upon a new configuration
    added when the queue is either not running yet or is replaying
    incoming events.
    
    Restarting a partially initialized queue would have unexpected
    consequences as the objects that are needed to schedule and
    serve the requests could be still null and generate random NPEs
    all around.
    
    The replay also is a very delicate phase because it is responsible
    for the replay of the past missed replication events that should
    be re-triggered with highest priority possible. All new events
    and new configuration can wait to be processed once all the
    past events have been replayed and have completed.
    
    Bug: Issue 11055
    Change-Id: Ib154532ec32a50f4cb9ca5553c1f77fa931ae2ac
    
  - AbstractConfigTest: Import createTempDirectory static
    
    Change-Id: I1d8816b1f3f903d7c161377158ae49574fd70b1e
    
  - Fix creation of missing repository when replicating to a Gerrit server
    
    When replicating to a Gerrit server over http, non-existence of a
    repository is reported as:
    
      <repo-name> unavailable
    
    This message wasn't among expected messages and creation of the missing
    repository wasn't executed.
    
    Bug: Issue 11204
    Change-Id: I3d6c3e14573b638b17d54d1777a606b8f356f1f9
    
  - Revert "Remove replication event from pending when runway is allowed"
    
    This reverts commit e780ae61cbbba3f88558a3620065d1fcdc0768c8.
    
    Reason for revert: Creates potentially infinite Heap consumption and
    JVM crash when a PushOp is denied to use the runway.
    
    The problem is caused by the late removal of the PushOp from the
    list of pending operations, which causes the subsequent reschedule
    to find the same operation and double the number of replication states
    associated with it. If the loop happens multiple times, the generated
    consumption grows exponentially causing the JVM to enter a series
    of continuous stop-the-world GC that eventually lead to complete
    block of the JVM activity.
    
    The method `requestRunway` is thus not telling the truth because
    it is actually relying on the fact that the PushOne operation is
    removed also from the pending list, independently from its runway
    concession status.
    
    Change-Id: I6097bc7ad16c8bcc86a7d30af7d2ad331728712d
    
  - Trace details of the replication events cancelled
    
    When a destination is stopped, show in the replication_log
    the number of pending and in-flight events cancelled.
    
    The whole pending and in-flight cancellation is done through
    a stateLock critical region, so that the state of the
    destination object doesn't change in the middle of the
    operation.
    
    Change-Id: Ife2642bd4d1fd0ade627ace01b95bde6a9748d66
    
  - ReplicationFileBasedConfig: Fix setting default sshConnectionTimeout
    
    The default value did not get set when the replication config did not
    exist, was empty, or was invalid. This is not really a problem since
    if there is no config, no SSH connections are made anyway.
    
    A more serious issue is that the value was read with minutes as the
    default time unit, but then converted to milliseconds using seconds as
    the time unit, which resulted in an incorrect value.
    
    Change-Id: I64906e29acb56f0f53b432db61d2707dfe1963d3
    
  - SshHelper: Add class javadoc
    
    Change-Id: I80ab0c2225ad02628b630889c299c474e0087d70
    
  - Make more classes and fields public to ease extensibility
    
    Make classes public, and their constructors protected, to allow
    them to be extended.
    
    Make event type names and fields public.
    
    Change-Id: Iba275f99b7afbf87d57dd44851d043ad0f23fbe1
    
  - Allow to configure timeout for SSH connection and commands
    
    The timeout for SSH command execution is hard-coded to 0 which means
    no timeout, and the timeout for SSH connections is hard-coded to 2
    minutes.
    
    Introduce new configuration options to allow configuring each timeout
    individually:
    
      gerrit.sshCommandTimeout
      gerrit.sshConnectionTimeout
    
    When not set, both default to their previously hard-coded values,
    i.e. no limit and 2 minutes respectively.
    
    Change-Id: Ibd377d45543ef4631082e8d521aeeb99209003f7
    Signed-off-by: Dariusz Luksza <dluksza@collab.net>
    Signed-off-by: David Pursehouse <dpursehouse@collab.net>
    
  - Merge branch 'stable-2.16' into stable-3.0
    
    * stable-2.16:
      Cancel pending replications upon shutdown
      Remove replication event from pending when runway is allowed
    
    Change-Id: Icc8842c36e09e3de926ad585a104539a72980059
    
  - StartCommand:  Fix synchronization on non-final field
    
    When all error prone warnings are enabled the SynchronizeOnNonFinalField
    bug pattern is reported:
    
      plugins/replication/src/main/java/com/googlesource/gerrit/plugins/replication/StartCommand.java:103:
      error: [SynchronizeOnNonFinalField] Synchronizing on non-final fields is not safe:
      if the field is ever updated, different threads may end up locking on different objects.
          synchronized (stdout) {
                       ^
        (see https://errorprone.info/bugpattern/SynchronizeOnNonFinalField)
    
    Change-Id: Ib2df20aa28af4edd36ce5b9dfcf7d82c409dab84
    
  - Destination: Suppress FutureReturnValueIgnored warning
    
    When all error prone warnings are enabled the FutureReturnValueIgnored
    bug pattern is reported, for example:
    
      plugins/replication/src/main/java/com/googlesource/gerrit/plugins/replication/Destination.java:367:
      error: [FutureReturnValueIgnored] Return value of methods returning Future must be checked.
      Ignoring returned Futures suppresses exceptions thrown from the code that completes the Future.
            pool.schedule(e, now ? 0 : config.getDelay(), TimeUnit.SECONDS);
                         ^
        (see https://errorprone.info/bugpattern/FutureReturnValueIgnored)
      Did you mean to remove this line?
    
    Change-Id: I43b5c3c9f9bf8cda5f2d4ee701b2153e7e6f2807
    
  - Merge branch 'stable-2.15' into stable-2.16
    
    * stable-2.15:
      Cancel pending replications upon shutdown
      Remove replication event from pending when runway is allowed
    
    Change-Id: I81209708a42832cbe2be818e2dd50f625362b392
    
  - Cancel auto-reload runnable at configuration shutdown
    
    Avoid having multiple auto-reload runnable when
    the configuration is shut down and restarted.
    The situation happens quite often when the configuration
    is reloaded and generates unnecessary extra concurrency
    on the configuration objects other than increasingly consuming
    additional tasks.
    
    Bug: Issue 11187
    Change-Id: Ic03ce3fceb0df746d20437ae7cf4d2a8f4822d02
    
  - Cancel pending replications upon shutdown
    
    When the replication plugin is stopped or reloaded,
    mark all the current replications as cancelled.
    Having pending and unmanaged replication tasks is quite
    dangerous for two reasons:
    
    1. The result of the replication isn't accounted and
       properly managed, because the associated objects won't
       be there anymore (pending replications, runway, ...)
    
    2. The destination configuration could have changed completely
       (auto-reload use-case) or even not exist anymore: the
       replication event would thus perform an unwanted and
       unconfigured operation.
    
    With regards to the replication events that are already
    on the runway, there is no value in cancelling them. Just flag
    them as cancelled so that, once they finish, can be clearly
    recognized.
    
    Change-Id: Iabc17d375011cbc61f8c642ae07d3d018b49fc69
    
  - Remove replication event from pending when runway is allowed
    
    Consider a replication event not pending anymore *only if* the runway
    is free and allowed to run. Removing it too early would result in a
    ghost event which isn't officially pending but that will be eventually
    run when rescheduled.
    
    Change-Id: Ib0ffa3bec7a6d5d5be56e78a677e8b2133f91e18