Persist logs in gerrit-replica chart with multiple replica pods

When using multiple replicas of the apache-git-http-backend and/or
the gerrit-replica pods, persisting the logs did not work, since the
volumes created for this purpose were only supporting read-write-once.

A solution would have been to use StatefulSets instead of Deployments
to ensure a distinct volume for each pod, but this would give the pods
more state than required. E.g. there might be a lot of dangling
volumes, when scaled down. Other features of StatefulSets would not
be used like fixed hostnames. Thus, this solution seemed overly
complex for the Gerrit replica.

Instead now a volume providing read-write-many access, e.g. NFS, is
used to store the logs of all pods. Each pod will create a directory
in the NFS-volume with the pod name as directory name, allowing to
trace the logs to a given pod. The NFS-workaround was enabled for
this additional volume as well. Persisting logs is still optional.
The persisting of the whole Gerrit site in the gerrit-replica pods
was removed. Since the site of a replica is pretty static apart from
the repositories and logs, this should not be required and creating
the site on pod creation should be fast enough.

Change-Id: I924fbd9f9cae9604bfb0af7f703aaf854a14220e
diff --git a/container-images/git-gc/Dockerfile b/container-images/git-gc/Dockerfile
index 4638176..1c469fa 100644
--- a/container-images/git-gc/Dockerfile
+++ b/container-images/git-gc/Dockerfile
@@ -5,6 +5,8 @@
 RUN mkdir -p /var/log/git && \
     chown gerrit:users /var/log/git
 
+USER gerrit
+
 VOLUME ["/var/gerrit/git"]
 
 ENTRYPOINT ["/var/tools/gc-all.sh"]
diff --git a/helm-charts/gerrit-replica/README.md b/helm-charts/gerrit-replica/README.md
index 65d7103..38e1c0a 100644
--- a/helm-charts/gerrit-replica/README.md
+++ b/helm-charts/gerrit-replica/README.md
@@ -125,15 +125,26 @@
 | `storageClasses.shared.reclaimPolicy`  | Whether to `Retain` or `Delete` volumes, when they become unbound | `Delete`                                          |
 | `storageClasses.shared.parameters`     | Parameters for the provisioner                                    | `parameters.mountOptions: vers=4.1`               |
 
+### Workaround for NFS
+
+Kubernetes will not be able to adapt the ownership of the files within NFS
+volumes. Thus, a workaround exists that will add init-containers and jobs to
+adapt file ownership. Also the ID-domain will be configured to ensure correct
+ID-mapping.
+
+| Parameter                | Description                                                                | Default           |
+|--------------------------|----------------------------------------------------------------------------|-------------------|
+| `nfsWorkaround.enabled`  | Whether the volume used is an NFS-volume                                   | `false`           |
+| `nfsWorkaround.idDomain` | The ID-domain that should be used to map user-/group-IDs for the NFS mount | `localdomain.com` |
+
+
 ### Storage for Git repositories
 
-| Parameter                               | Description                                                                | Default                |
-|-----------------------------------------|----------------------------------------------------------------------------|------------------------|
-| `gitRepositoryStorage.externalPVC.use`  | Whether to use a PVC deployed outside the chart                            | `false`                |
-| `gitRepositoryStorage.externalPVC.name` | Name of the external PVC                                                   | `git-repositories-pvc` |
-| `gitRepositoryStorage.size`             | Size of the volume storing the Git repositories                            | `5Gi`                  |
-| `gitRepositoryStorage.nfs.enabled`      | Whether the volume used is an NFS-volume                                   | `false`                |
-| `gitRepositoryStorage.nfs.idDomain`     | The ID-domain that should be used to map user-/group-IDs for the NFS mount | `localdomain.com`      |
+| Parameter                               | Description                                     | Default                |
+|-----------------------------------------|-------------------------------------------------|------------------------|
+| `gitRepositoryStorage.externalPVC.use`  | Whether to use a PVC deployed outside the chart | `false`                |
+| `gitRepositoryStorage.externalPVC.name` | Name of the external PVC                        | `git-repositories-pvc` |
+| `gitRepositoryStorage.size`             | Size of the volume storing the Git repositories | `5Gi`                  |
 
 If the git repositories should be persisted even if the chart is deleted and in
 a way that the volume containing them can be mounted by the reinstalled chart,
@@ -141,12 +152,21 @@
 the external PVC, set `gitRepositoryStorage.externalPVC.enabled` to `true` and
 give the name of the PVC under `gitRepositoryStorage.externalPVC.name`.
 
-Using an NFS mount in Kubernetes can be a bit tricky. To make it work, this chart
-provides a workaround that can be activated by setting `gitRepositoryStorage.nfs.enabled`
-to `true`. One thing it does, is to provide a `idmapd.conf` file that configures
-the ID domain set in `gitRepositoryStorage.nfs.idDomain`. It will also add an
-init-container to the init-job that will ensure that the directory that will
-contain the repositories is owned by the correct user.
+### Storage for Logs
+
+In addition to collecting logs with a log collection tool like Promtail, the logs
+can also be stored in a persistent volume. This volume has to be a read-write-many
+volume to be able to be used by multiple pods.
+
+| Parameter                     | Description                                     | Default           |
+|-------------------------------|-------------------------------------------------|-------------------|
+| `logStorage.enabled`          | Whether to enable persistence of logs           | `false`           |
+| `logStorage.externalPVC.use`  | Whether to use a PVC deployed outside the chart | `false`           |
+| `logStorage.externalPVC.name` | Name of the external PVC                        | `gerrit-logs-pvc` |
+| `logStorage.size`             | Size of the volume                              | `5Gi`             |
+
+Each pod will create a separate folder for its logs, allowing to trace logs to
+the respective pods.
 
 ### Istio
 
@@ -205,8 +225,6 @@
 | `gitBackend.livenessProbe`                 | Configuration of the liveness probe timings                                        | `{initialDelaySeconds: 10, periodSeconds: 5}`                             |
 | `gitBackend.readinessProbe`                | Configuration of the readiness probe timings                                       | `{initialDelaySeconds: 5, periodSeconds: 1}`                              |
 | `gitBackend.credentials.htpasswd`          | `.htpasswd`-file containing username/password-credentials for accessing git        | `git:$apr1$O/LbLKC7$Q60GWE7OcqSEMSfe/K8xU.` (user: git, password: secret) |
-| `gitBackend.logging.persistence.enabled`   | Whether to persist logs                                                            | `true`                                                                    |
-| `gitBackend.logging.persistence.size`      | Storage size for persisted logs                                                    | `1Gi`                                                                     |
 | `gitBackend.tls.secret.create`             | Whether to create a TLS-secret                                                     | `true`                                                                    |
 | `gitBackend.tls.secret.name`               | Name of an external secret that will be used as a TLS-secret                       | `nil`                                                                     |
 | `gitBackend.tls.cert`                      | Public SSL server certificate                                                      | `-----BEGIN CERTIFICATE-----`                                             |
@@ -236,8 +254,6 @@
 |                                     |                                                                  | `requests.memory: 256Mi` |
 |                                     |                                                                  | `limits.cpu: 100m`       |
 |                                     |                                                                  | `limits.memory: 256Mi`   |
-| `gitGC.logging.persistence.enabled` | Whether to persist logs                                          | `true`                   |
-| `gitGC.logging.persistence.size`    | Storage size for persisted logs                                  | `1Gi`                    |
 
 ### Gerrit replica
 
@@ -267,8 +283,6 @@
 |                                               |                                                                                                     | `requests.memory: 5Gi`                                                          |
 |                                               |                                                                                                     | `limits.cpu: 1`                                                                 |
 |                                               |                                                                                                     | `limits.memory: 6Gi`                                                            |
-| `gerritReplica.persistence.enabled`           | Whether to persist the Gerrit site                                                                  | `true`                                                                          |
-| `gerritReplica.persistence.size`              | Storage size for persisted Gerrit site                                                              | `10Gi`                                                                          |
 | `gerritReplica.service.type`                  | Which kind of Service to deploy                                                                     | `NodePort`                                                                      |
 | `gerritReplica.service.http.port`             | Port over which to expose HTTP                                                                      | `80`                                                                            |
 | `gerritReplica.service.ssh.enabled`           | Whether to enable SSH for the Gerrit replica                                                        | `false`                                                                         |
diff --git a/helm-charts/gerrit-replica/templates/gerrit-replica.deployment.yaml b/helm-charts/gerrit-replica/templates/gerrit-replica.deployment.yaml
index c148556..163eb47 100644
--- a/helm-charts/gerrit-replica/templates/gerrit-replica.deployment.yaml
+++ b/helm-charts/gerrit-replica/templates/gerrit-replica.deployment.yaml
@@ -33,6 +33,28 @@
       - name: {{ .Values.images.registry.ImagePullSecret.name }}
       {{- end }}
       initContainers:
+      {{- if .Values.nfsWorkaround.enabled }}
+      - name: nfs-init
+        image: busybox
+        command:
+        - sh
+        - -c
+        args:
+        - |
+          chown -R 1000:100 /var/mnt/logs
+        env:
+        - name: POD_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.name
+        volumeMounts:
+        - name: logs
+          subPathExpr: "gerrit-replica/$(POD_NAME)"
+          mountPath: "/var/mnt/logs"
+        - name: nfs-config
+          mountPath: "/etc/idmapd.conf"
+          subPath: idmapd.conf
+      {{- end }}
       - name: site-cleanup
         image: busybox
         securityContext:
@@ -46,6 +68,7 @@
           # Remove directories that should be mounted rather than exist with the
           # rest of the site
           [ ! -L /var/gerrit/git ] && rm -rf /var/gerrit/git
+          [ ! -L /var/gerrit/logs ] && rm -rf /var/gerrit/logs
           rm -f /var/gerrit/logs/gerrit.pid
         volumeMounts:
         - name: gerrit-site
@@ -65,8 +88,9 @@
             -c /var/config/gerrit-init.yaml \
             -s /var/gerrit
 
-          # The git repositories will be mounted from a volume
+          # The git repositories and logs will be mounted from a volume
           [ -L /var/gerrit/git ] || rm -rf /var/gerrit/git
+          [ -L /var/gerrit/logs ] || rm -rf /var/gerrit/logs
           {{ if .Values.gerritReplica.service.ssh.enabled -}}
           rm -f /var/gerrit/etc/ssh_host*key*
           {{- end }}
@@ -99,13 +123,25 @@
             ln -sf /var/mnt/git /var/gerrit/
           fi
 
+          if [ ! -d /var/gerrit/logs ]; then
+            ln -sf /var/mnt/logs /var/gerrit/
+          fi
+
           /var/tools/validate_notedb.py -s /var/gerrit
+        env:
+        - name: POD_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.name
         volumeMounts:
         - name: gerrit-site
           mountPath: "/var/gerrit"
         - name: git-repositories
           mountPath: "/var/mnt/git"
-        {{- if .Values.gitRepositoryStorage.nfs.enabled }}
+        - name: logs
+          subPathExpr: "gerrit-replica/$(POD_NAME)"
+          mountPath: "/var/mnt/logs"
+        {{- if .Values.nfsWorkaround.enabled }}
         - name: nfs-config
           mountPath: "/etc/idmapd.conf"
           subPath: idmapd.conf
@@ -125,6 +161,11 @@
       - name: gerrit-replica
         image: {{ template "registry" . }}{{ .Values.gerritReplica.images.gerritReplica }}:{{ .Values.images.version }}
         imagePullPolicy: {{ .Values.images.imagePullPolicy }}
+        env:
+        - name: POD_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.name
         ports:
         - name: http
           containerPort: 8080
@@ -137,7 +178,10 @@
           mountPath: "/var/gerrit"
         - name: git-repositories
           mountPath: "/var/mnt/git"
-        {{- if .Values.gitRepositoryStorage.nfs.enabled }}
+        - name: logs
+          subPathExpr: "gerrit-replica/$(POD_NAME)"
+          mountPath: "/var/mnt/logs"
+        {{- if .Values.nfsWorkaround.enabled }}
         - name: nfs-config
           mountPath: "/etc/idmapd.conf"
           subPath: idmapd.conf
@@ -175,29 +219,35 @@
             -config.file=/etc/promtail/promtail.yaml \
             -client.url={{ .Values.promtailSidecar.loki.url }}/loki/api/v1/push \
             -client.external-labels=instance=$HOSTNAME
+        env:
+        - name: POD_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.name
         resources:
 {{ toYaml .Values.promtailSidecar.resources | indent 10 }}
         volumeMounts:
-          - name: promtail-config
-            mountPath: /etc/promtail/promtail.yaml
-            subPath: promtail.yaml
-          - name: promtail-secret
-            mountPath: /etc/promtail/promtail.secret
-            subPath: promtail.secret
-          - name: promtail-secret
-            mountPath: /etc/promtail/promtail.ca.crt
-            subPath: promtail.ca.crt
-          - name: gerrit-site
-            mountPath: "/var/gerrit"
+        - name: promtail-config
+          mountPath: /etc/promtail/promtail.yaml
+          subPath: promtail.yaml
+        - name: promtail-secret
+          mountPath: /etc/promtail/promtail.secret
+          subPath: promtail.secret
+        - name: promtail-secret
+          mountPath: /etc/promtail/promtail.ca.crt
+          subPath: promtail.ca.crt
+        - name: logs
+          subPathExpr: "gerrit-replica/$(POD_NAME)"
+          mountPath: "/var/gerrit/logs"
+        {{- if .Values.nfsWorkaround.enabled }}
+        - name: nfs-config
+          mountPath: "/etc/idmapd.conf"
+          subPath: idmapd.conf
+        {{- end }}
       {{- end }}
       volumes:
       - name: gerrit-site
-        {{ if .Values.gerritReplica.persistence.enabled -}}
-        persistentVolumeClaim:
-          claimName: {{ .Release.Name }}-gerrit-replica-pvc
-        {{ else -}}
         emptyDir: {}
-        {{- end }}
       - name: git-repositories
         persistentVolumeClaim:
           {{- if .Values.gitRepositoryStorage.externalPVC.use }}
@@ -205,12 +255,23 @@
           {{- else }}
           claimName: {{ .Release.Name }}-git-repositories-pvc
           {{- end }}
+      - name: logs
+        {{ if .Values.logStorage.enabled -}}
+        persistentVolumeClaim:
+          {{- if .Values.logStorage.externalPVC.use }}
+          claimName: {{ .Values.logStorage.externalPVC.name }}
+          {{- else }}
+          claimName: {{ .Release.Name }}-log-pvc
+          {{- end }}
+        {{ else -}}
+        emptyDir: {}
+        {{- end }}
       {{- if and .Values.gerritReplica.plugins.cache.enabled .Values.gerritReplica.plugins.downloaded }}
       - name: gerrit-plugin-cache
         persistentVolumeClaim:
           claimName: {{ .Release.Name }}-plugin-cache-pvc
       {{- end }}
-      {{- if .Values.gitRepositoryStorage.nfs.enabled }}
+      {{- if .Values.nfsWorkaround.enabled }}
       - name: nfs-config
         configMap:
           name: {{ .Release.Name }}-nfs-configmap
diff --git a/helm-charts/gerrit-replica/templates/gerrit-replica.storage.yaml b/helm-charts/gerrit-replica/templates/gerrit-replica.storage.yaml
index 28c3dd8..d6ed00c 100644
--- a/helm-charts/gerrit-replica/templates/gerrit-replica.storage.yaml
+++ b/helm-charts/gerrit-replica/templates/gerrit-replica.storage.yaml
@@ -1,23 +1,4 @@
-{{ if .Values.gerritReplica.persistence.enabled -}}
-kind: PersistentVolumeClaim
-apiVersion: v1
-metadata:
-  name: {{ .Release.Name }}-gerrit-replica-pvc
-  labels:
-    app: gerrit-replica
-    chart: {{ template "gerrit-replica.chart" . }}
-    heritage: {{ .Release.Service }}
-    release: {{ .Release.Name }}
-spec:
-  accessModes:
-  - ReadWriteOnce
-  resources:
-    requests:
-      storage: {{ .Values.gerritReplica.persistence.size }}
-  storageClassName: {{ .Values.storageClasses.default.name }}
-{{- end }}
 {{- if and .Values.gerritReplica.plugins.cache.enabled .Values.gerritReplica.plugins.downloaded }}
----
 kind: PersistentVolumeClaim
 apiVersion: v1
 metadata:
diff --git a/helm-charts/gerrit-replica/templates/git-backend.deployment.yaml b/helm-charts/gerrit-replica/templates/git-backend.deployment.yaml
index 61540f6..78906a4 100644
--- a/helm-charts/gerrit-replica/templates/git-backend.deployment.yaml
+++ b/helm-charts/gerrit-replica/templates/git-backend.deployment.yaml
@@ -32,6 +32,29 @@
       imagePullSecrets:
       - name: {{ .Values.images.registry.ImagePullSecret.name }}
       {{- end }}
+      initContainers:
+      {{- if .Values.nfsWorkaround.enabled }}
+      - name: nfs-init
+        image: busybox
+        command:
+        - sh
+        - -c
+        args:
+        - |
+          chown -R 1000:100 /var/mnt/logs
+        env:
+        - name: POD_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.name
+        volumeMounts:
+        - name: logs
+          subPathExpr: "gerrit-replica/$(POD_NAME)"
+          mountPath: "/var/mnt/logs"
+        - name: nfs-config
+          mountPath: "/etc/idmapd.conf"
+          subPath: idmapd.conf
+      {{- end }}
       containers:
       - name: apache-git-http-backend
         imagePullPolicy: {{ .Values.images.imagePullPolicy }}
@@ -45,6 +68,10 @@
         - name: DISABLE_HTTP
           value: "true"
         {{- end }}
+        - name: POD_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.name
         ports:
         {{ if eq .Values.gitBackend.service.http.enabled true -}}
         - name: http-port
@@ -77,7 +104,10 @@
         volumeMounts:
         - name: git-repositories
           mountPath: "/var/gerrit/git"
-        {{- if .Values.gitRepositoryStorage.nfs.enabled }}
+        - name: logs
+          subPathExpr: "apache-git-http-backend/$(POD_NAME)"
+          mountPath: "/var/log/apache2"
+        {{- if .Values.nfsWorkaround.enabled }}
         - name: nfs-config
           mountPath: "/etc/idmapd.conf"
           subPath: idmapd.conf
@@ -96,8 +126,6 @@
           subPath: tls.crt
           mountPath: "/var/apache/credentials/server.crt"
         {{ end }}
-        - name: apache-logs
-          mountPath: "/var/log/apache2"
       volumes:
       - name: git-repositories
         persistentVolumeClaim:
@@ -118,14 +146,18 @@
           secretName: {{ .Values.gitBackend.tls.secret.name }}
           {{- end }}
       {{ end }}
-      - name: apache-logs
-        {{ if .Values.gitBackend.logging.persistence.enabled -}}
+      - name: logs
+        {{ if .Values.logStorage.enabled -}}
         persistentVolumeClaim:
-          claimName: {{ .Release.Name }}-apache-logs-pvc
+          {{- if .Values.logStorage.externalPVC.use }}
+          claimName: {{ .Values.logStorage.externalPVC.name }}
+          {{- else }}
+          claimName: {{ .Release.Name }}-log-pvc
+          {{- end }}
         {{ else -}}
         emptyDir: {}
         {{- end }}
-      {{- if .Values.gitRepositoryStorage.nfs.enabled }}
+      {{- if .Values.nfsWorkaround.enabled }}
       - name: nfs-config
         configMap:
           name: {{ .Release.Name }}-nfs-configmap
diff --git a/helm-charts/gerrit-replica/templates/git-backend.storage.yaml b/helm-charts/gerrit-replica/templates/git-backend.storage.yaml
deleted file mode 100644
index 99848ee..0000000
--- a/helm-charts/gerrit-replica/templates/git-backend.storage.yaml
+++ /dev/null
@@ -1,18 +0,0 @@
-{{ if .Values.gitBackend.logging.persistence.enabled -}}
-kind: PersistentVolumeClaim
-apiVersion: v1
-metadata:
-  name: {{ .Release.Name }}-apache-logs-pvc
-  labels:
-    app: git-backend
-    chart: {{ template "gerrit-replica.chart" . }}
-    heritage: {{ .Release.Service }}
-    release: {{ .Release.Name }}
-spec:
-  accessModes:
-  - ReadWriteOnce
-  resources:
-    requests:
-      storage: {{ .Values.gitBackend.logging.persistence.size }}
-  storageClassName: {{ .Values.storageClasses.default.name }}
-{{- end }}
diff --git a/helm-charts/gerrit-replica/templates/git-gc.cronjob.yaml b/helm-charts/gerrit-replica/templates/git-gc.cronjob.yaml
index fac1eba..46b2cb3 100644
--- a/helm-charts/gerrit-replica/templates/git-gc.cronjob.yaml
+++ b/helm-charts/gerrit-replica/templates/git-gc.cronjob.yaml
@@ -20,23 +20,56 @@
         spec:
           restartPolicy: OnFailure
           securityContext:
-            runAsUser: 1000
             fsGroup: 100
           {{ if .Values.images.registry.ImagePullSecret.name -}}
           imagePullSecrets:
           - name: {{ .Values.images.registry.ImagePullSecret.name }}
           {{- end }}
+          initContainers:
+          {{- if .Values.nfsWorkaround.enabled }}
+          - name: nfs-init
+            image: busybox
+            command:
+            - sh
+            - -c
+            args:
+            - |
+              chown -R 1000:100 /var/mnt/logs
+            env:
+            - name: POD_NAME
+              valueFrom:
+                fieldRef:
+                  fieldPath: metadata.name
+            volumeMounts:
+            - name: logs
+              subPathExpr: "git-gc/$(POD_NAME)"
+              mountPath: "/var/mnt/logs"
+            - name: nfs-config
+              mountPath: "/etc/idmapd.conf"
+              subPath: idmapd.conf
+          {{- end }}
           containers:
           - name: git-gc
             imagePullPolicy: {{ .Values.images.imagePullPolicy }}
             image: {{ template "registry" . }}{{ .Values.gitGC.image }}:{{ .Values.images.version }}
             resources:
 {{ toYaml .Values.gitGC.resources | indent 14 }}
+            env:
+            - name: POD_NAME
+              valueFrom:
+                fieldRef:
+                  fieldPath: metadata.name
             volumeMounts:
             - name: git-repositories
               mountPath: "/var/gerrit/git"
-            - name: git-gc-logs
+            - name: logs
+              subPathExpr: "git-gc/$(POD_NAME)"
               mountPath: "/var/log/git"
+            {{- if .Values.nfsWorkaround.enabled }}
+            - name: nfs-config
+              mountPath: "/etc/idmapd.conf"
+              subPath: idmapd.conf
+            {{- end }}
           volumes:
           - name: git-repositories
             persistentVolumeClaim:
@@ -45,10 +78,19 @@
               {{- else }}
               claimName: {{ .Release.Name }}-git-repositories-pvc
               {{- end }}
-          - name: git-gc-logs
-            {{ if .Values.gitGC.logging.persistence.enabled -}}
+          - name: logs
+            {{ if .Values.logStorage.enabled -}}
             persistentVolumeClaim:
-              claimName: {{ .Release.Name }}-git-gc-logs-pvc
+              {{- if .Values.logStorage.externalPVC.use }}
+              claimName: {{ .Values.logStorage.externalPVC.name }}
+              {{- else }}
+              claimName: {{ .Release.Name }}-log-pvc
+              {{- end }}
             {{ else -}}
             emptyDir: {}
             {{- end }}
+          {{- if .Values.nfsWorkaround.enabled }}
+          - name: nfs-config
+            configMap:
+              name: {{ .Release.Name }}-nfs-configmap
+          {{- end }}
diff --git a/helm-charts/gerrit-replica/templates/git-repositories-init.job.yaml b/helm-charts/gerrit-replica/templates/git-repositories-init.job.yaml
index e2c46aa..9478b18 100644
--- a/helm-charts/gerrit-replica/templates/git-repositories-init.job.yaml
+++ b/helm-charts/gerrit-replica/templates/git-repositories-init.job.yaml
@@ -21,7 +21,7 @@
       imagePullSecrets:
       - name: {{ .Values.images.registry.ImagePullSecret.name }}
       {{- end }}
-      {{- if .Values.gitRepositoryStorage.nfs.enabled }}
+      {{- if .Values.nfsWorkaround.enabled }}
       initContainers:
       - name: nfs-init
         image: busybox
@@ -30,10 +30,12 @@
         - -c
         args:
         - |
-          chown -R 1000:100 /var/mnt/git
+          chown -R 1000:100 /var/mnt
         volumeMounts:
         - name: git-repositories
           mountPath: "/var/mnt/git"
+        - name: logs
+          mountPath: "/var/mnt/logs"
         - name: nfs-config
           mountPath: "/etc/idmapd.conf"
           subPath: idmapd.conf
@@ -60,11 +62,19 @@
           /var/tools/gerrit_init.py \
             -c /var/config/gerrit-init.yaml \
             -s /var/gerrit
+        env:
+        - name: POD_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.name
         volumeMounts:
         - name: gerrit-site
           mountPath: "/var/gerrit"
         - name: git-repositories
           mountPath: "/var/mnt/git"
+        - name: logs
+          subPathExpr: "gerrit-replica/$(POD_NAME)"
+          mountPath: "/var/gerrit/logs"
         - name: gerrit-init-config
           mountPath: "/var/config/gerrit-init.yaml"
           subPath: gerrit-init.yaml
@@ -81,13 +91,24 @@
           {{- else }}
           claimName: {{ .Release.Name }}-git-repositories-pvc
           {{- end }}
+      - name: logs
+        {{ if .Values.logStorage.enabled -}}
+        persistentVolumeClaim:
+          {{- if .Values.logStorage.externalPVC.use }}
+          claimName: {{ .Values.logStorage.externalPVC.name }}
+          {{- else }}
+          claimName: {{ .Release.Name }}-log-pvc
+          {{- end }}
+        {{ else -}}
+        emptyDir: {}
+        {{- end }}
       - name: gerrit-init-config
         configMap:
           name: {{ .Release.Name }}-gerrit-init-configmap
       - name: gerrit-config
         configMap:
           name: {{ .Release.Name }}-gerrit-replica-configmap
-      {{- if .Values.gitRepositoryStorage.nfs.enabled }}
+      {{- if .Values.nfsWorkaround.enabled }}
       - name: nfs-config
         configMap:
           name: {{ .Release.Name }}-nfs-configmap
diff --git a/helm-charts/gerrit-replica/templates/git-repositories.pvc.yaml b/helm-charts/gerrit-replica/templates/git-repositories.pvc.yaml
deleted file mode 100644
index 0788165..0000000
--- a/helm-charts/gerrit-replica/templates/git-repositories.pvc.yaml
+++ /dev/null
@@ -1,13 +0,0 @@
-{{- if not .Values.gitRepositoryStorage.externalPVC.use }}
-kind: PersistentVolumeClaim
-apiVersion: v1
-metadata:
-  name: {{ .Release.Name }}-git-repositories-pvc
-spec:
-  accessModes:
-  - ReadWriteMany
-  resources:
-    requests:
-      storage: {{ .Values.gitRepositoryStorage.size }}
-  storageClassName: {{ .Values.storageClasses.shared.name }}
-{{- end }}
diff --git a/helm-charts/gerrit-replica/templates/nfs.configmap.yaml b/helm-charts/gerrit-replica/templates/nfs.configmap.yaml
index ccd7de9..238bcbc 100644
--- a/helm-charts/gerrit-replica/templates/nfs.configmap.yaml
+++ b/helm-charts/gerrit-replica/templates/nfs.configmap.yaml
@@ -1,4 +1,4 @@
-{{- if .Values.gitRepositoryStorage.nfs.enabled -}}
+{{- if .Values.nfsWorkaround.enabled -}}
 apiVersion: v1
 kind: ConfigMap
 metadata:
@@ -15,7 +15,7 @@
     Verbosity = 0
     Pipefs-Directory = /run/rpc_pipefs
     # set your own domain here, if it differs from FQDN minus hostname
-    Domain = {{ .Values.gitRepositoryStorage.nfs.idDomain }}
+    Domain = {{ .Values.nfsWorkaround.idDomain }}
 
     [Mapping]
 
diff --git a/helm-charts/gerrit-replica/templates/promtail.configmap.yaml b/helm-charts/gerrit-replica/templates/promtail.configmap.yaml
index b5496fb..4ccd869 100644
--- a/helm-charts/gerrit-replica/templates/promtail.configmap.yaml
+++ b/helm-charts/gerrit-replica/templates/promtail.configmap.yaml
@@ -10,7 +10,7 @@
 data:
   promtail.yaml: |-
     positions:
-      filename: /var/gerrit/etc/promtail-positions.yaml
+      filename: /var/gerrit/logs/promtail-positions.yaml
 
     clients:
       - url: {{ .Values.promtailSidecar.loki.url }}/loki/api/v1/push
diff --git a/helm-charts/gerrit-replica/templates/storage.pvc.yaml b/helm-charts/gerrit-replica/templates/storage.pvc.yaml
new file mode 100644
index 0000000..5f8974e
--- /dev/null
+++ b/helm-charts/gerrit-replica/templates/storage.pvc.yaml
@@ -0,0 +1,27 @@
+{{- if not .Values.gitRepositoryStorage.externalPVC.use }}
+kind: PersistentVolumeClaim
+apiVersion: v1
+metadata:
+  name: {{ .Release.Name }}-git-repositories-pvc
+spec:
+  accessModes:
+  - ReadWriteMany
+  resources:
+    requests:
+      storage: {{ .Values.gitRepositoryStorage.size }}
+  storageClassName: {{ .Values.storageClasses.shared.name }}
+{{- end }}
+{{- if and .Values.logStorage.enabled (not .Values.logStorage.externalPVC.use) }}
+---
+kind: PersistentVolumeClaim
+apiVersion: v1
+metadata:
+  name: {{ .Release.Name }}-log-pvc
+spec:
+  accessModes:
+  - ReadWriteMany
+  resources:
+    requests:
+      storage: {{ .Values.logStorage.size }}
+  storageClassName: {{ .Values.storageClasses.shared.name }}
+{{- end }}
diff --git a/helm-charts/gerrit-replica/values.yaml b/helm-charts/gerrit-replica/values.yaml
index 6314753..262b984 100644
--- a/helm-charts/gerrit-replica/values.yaml
+++ b/helm-charts/gerrit-replica/values.yaml
@@ -41,15 +41,24 @@
     parameters:
       mountOptions: vers=4.1
 
+nfsWorkaround:
+  enabled: false
+  idDomain: localdomain.com
+
 
 gitRepositoryStorage:
   externalPVC:
     use: false
     name: git-repositories-pvc
   size: 5Gi
-  nfs:
-    enabled: false
-    idDomain: localdomain.com
+
+
+logStorage:
+  enabled: false
+  externalPVC:
+    use: false
+    name: gerrit-logs-pvc
+  size: 5Gi
 
 
 istio:
@@ -117,11 +126,6 @@
     initialDelaySeconds: 5
     periodSeconds: 1
 
-  logging:
-    persistence:
-      enabled: true
-      size: 1Gi
-
   tls:
     secret:
       # If using an external secret, make sure to name the keys `tls.crt`
@@ -187,11 +191,6 @@
       cpu: 100m
       memory: 256Mi
 
-  logging:
-    persistence:
-      enabled: true
-      size: 1Gi
-
 
 gerritReplica:
   images:
@@ -221,10 +220,6 @@
       cpu: 1
       memory: 6Gi
 
-  persistence:
-    enabled: true
-    size: 10Gi
-
   service:
     type: NodePort
     http: