Fix markdown formatting of the multi-master replication scalability design
Make the markdown formatting more consistent and use the $GERRIT_SITE
notation for referring to the location of where Gerrit site has been
installed.
The previous notation <site_dir> was confusing the Markdown formatting.
Change-Id: Ib11468b31361a1b48cdf4813375f9c2666fc4cab
diff --git a/pages/design-docs/scaling-multi-master-replication/solution-distribute-replication-tasks-via-shared-FS.md b/pages/design-docs/scaling-multi-master-replication/solution-distribute-replication-tasks-via-shared-FS.md
index 1ac7eb0..372dce8 100644
--- a/pages/design-docs/scaling-multi-master-replication/solution-distribute-replication-tasks-via-shared-FS.md
+++ b/pages/design-docs/scaling-multi-master-replication/solution-distribute-replication-tasks-via-shared-FS.md
@@ -22,14 +22,11 @@
all the nodes in the cluster via sharing the filesystem hosting the persisted replication tasks via
a shared filesystem such as NFS. The high level tasks that will be used to achieve this are:
-### 1) Modifying the persisted task store so that it can be safely shared across multiple nodes via a
-shared file system
+### 1) Modifying the persisted task store so that it can be safely shared across multiple nodes via a shared file system
-### 2) Use a directory lock to ensure replication to the same URI does not occur concurrently in the
-cluster
+### 2) Use a directory lock to ensure replication to the same URI does not occur concurrently in the cluster
-### 3) Distribute the tasks among masters by periodically adding every replication task from the
-persisted store to the replication queue on every node
+### 3) Distribute the tasks among masters by periodically adding every replication task from the persisted store to the replication queue on every node
### 4) Improve task distribution with random delays
@@ -48,7 +45,9 @@
Example:
+```
Event Fired by Gerrit for -> (project="All-Projects", ref="ref/meta/config")
+```
##### iii) The replication plugin receives that project/ref based event
@@ -64,6 +63,7 @@
Example:
+```
url = git://destination-A:9417/${name}.git
-> push git://destination-A:9417/All-Projects.git refs/meta/config
url = git://destination-B:9417/${name}.git
@@ -74,19 +74,22 @@
-> push git://destination-D:9417/All-Projects.git refs/meta/config
url = git://destination-E:9417/${name}.git
-> push git://destination-E:9417/All-Projects.git refs/meta/config
+```
##### v) Persist the specific URI/ref based tasks to the filesystem
Each task is persisted in a json file named after the SHA1 of the json under
-<site_dir>/data/replication/ref-updates/waiting
+$GERRIT_SITE/data/replication/ref-updates/waiting
Example:
- <site_dir>/data/replication/ref-updates/waiting/a564cbd....json
- <site_dir>/data/replication/ref-updates/waiting/77adf32....json
- <site_dir>/data/replication/ref-updates/waiting/07fd222....json
- <site_dir>/data/replication/ref-updates/waiting/57bf2d2....json
- <site_dir>/data/replication/ref-updates/waiting/bbff33d....json
+```
+ $GERRIT_SITE/data/replication/ref-updates/waiting/a564cbd....json
+ $GERRIT_SITE/data/replication/ref-updates/waiting/77adf32....json
+ $GERRIT_SITE/data/replication/ref-updates/waiting/07fd222....json
+ $GERRIT_SITE/data/replication/ref-updates/waiting/57bf2d2....json
+ $GERRIT_SITE/data/replication/ref-updates/waiting/bbff33d....json
+```
##### vi) Scheduled tasks to be run after a configurable replication delay
@@ -103,6 +106,7 @@
Example:
+```
waiting
-> push git://destination-E:9417/All-Projects.git refs/dashboards/mydash
@@ -111,6 +115,7 @@
consolidated
-> push git://destination-E:9417/All-Projects.git refs/dashboards/mydash refs/meta/config
+```
##### ix) Push refs
@@ -121,8 +126,10 @@
Example:
- <site_dir>/data/replication/ref-updates/waiting/bbff33d....json
- -> <site_dir>/data/replication/ref-updates/running/bbff33d....json
+```
+ $GERRIT_SITE/data/replication/ref-updates/waiting/bbff33d....json
+ -> $GERRIT_SITE/data/replication/ref-updates/running/bbff33d....json
+```
##### xi A) Push Fails
@@ -148,8 +155,10 @@
Example:
- <site_dir>/data/replication/ref-updates/building/tmp001232asf.json
- -> <site_dir>/data/replication/ref-updates/waiting/a564cbd....json
+```
+ $GERRIT_SITE/data/replication/ref-updates/building/tmp001232asf.json
+ -> $GERRIT_SITE/data/replication/ref-updates/waiting/a564cbd....json
+```
To avoid resetting timestamps which will be used in step 3, do not overwrite already existing files
in the "waiting" directory.
@@ -178,15 +187,19 @@
Lock dir:
- <site_dir>/data/replication/ref-updates/running/23baf46.../ (SHA1 of URI)
+```
+ $GERRIT_SITE/data/replication/ref-updates/running/23baf46.../ (SHA1 of URI)
+```
Two ref updates for the same URI:
- <site_dir>/data/replication/ref-updates/waiting/a564cbd....json
- -> <site_dir>/data/replication/ref-updates/running/23baf46.../a564cbd....json
+```
+ $GERRIT_SITE/data/replication/ref-updates/waiting/a564cbd....json
+ -> $GERRIT_SITE/data/replication/ref-updates/running/23baf46.../a564cbd....json
- <site_dir>/data/replication/ref-updates/waiting/052b7ca....json
- -> <site_dir>/data/replication/ref-updates/running/23baf46.../052b7ca....json
+ $GERRIT_SITE/data/replication/ref-updates/waiting/052b7ca....json
+ -> $GERRIT_SITE/data/replication/ref-updates/running/23baf46.../052b7ca....json
+```
### 3) Periodically add all persisted tasks to the replication queue
@@ -317,6 +330,7 @@
limit) per node. This requires adding 15 more nodes for a total of 16 nodes each configured like
this:
+```
[remote "All"]
url = git://destination-A:9417/${name}.git
url = git://destination-B:9417/${name}.git
@@ -329,6 +343,7 @@
url = git://destination-I:9417/${name}.git
url = git://destination-J:9417/${name}.git
threads = 5
+```
This give a total of 16 * 5 = 80 threads which should allow 8 threads per destination total. This
solution does not make very good use of the available resources on each node, and every destination
@@ -341,6 +356,7 @@
Node 2:
+```
[remote "F"]
url = git://destination-F:9417/${name}.git
threads = 8
@@ -361,6 +377,7 @@
url = git://destination-J:9417/${name}.git
threads = 8
...
+```
This solution is harder to administer and it suffers from the loss of HA with respect to
replication destinations.
