Disable host key checking in nodepool We're seeing a lot of builds fail with RETRY errors. This is due to the host key on the remote node changing between the time that nodepool collects it (immediately at boot), and Zuul uses it. It appears that the images we're using have an ssh host key baked in to them but then overwrite it at first boot. Because we're using the internal IP (which doesn't need to wait for NAT setup), Nodepool can end up connecting to the VM very quickly and grabbing the initial (as opposed to the newly generated host key). When Zuul tries to talk to the node, it has changed and it throws an error. By disabling this, we allow Zuul to simply accept the host key on first connection. This is low-risk since we're only using the internal cloud network anyway. Change-Id: I2b2347f37527a72866e92164e6779ab6b57ebfa7
diff --git a/nodepool/nodepool.yaml b/nodepool/nodepool.yaml index 2f7dfea..b2f3a6e 100644 --- a/nodepool/nodepool.yaml +++ b/nodepool/nodepool.yaml
@@ -18,6 +18,18 @@ - name: main max-servers: 4 use-internal-ip: True + # Host key checking is disabled because: + # 1) We're using the internal IP so it's slightly less + # valuable (fewer attack vectors). + # 2) The images we're using appear to have a key baked into + # them which is overwritten at boot. Because we're using the + # internal IP, nodepool can end up connecting to the instance + # very quickly and retrieving the original host key rather + # than the new one (which is likely to be written a couple of + # seconds later). By disabling this in nodepool, we let Zuul + # just use the first key it finds (and it's likely to take + # long enough that it will have been updated by then). + host-key-checking: False labels: - name: debian-stretch-8G instance-type: n1-standard-2