Disable host key checking in nodepool
We're seeing a lot of builds fail with RETRY errors. This is due
to the host key on the remote node changing between the time that
nodepool collects it (immediately at boot), and Zuul uses it. It
appears that the images we're using have an ssh host key baked in
to them but then overwrite it at first boot. Because we're using
the internal IP (which doesn't need to wait for NAT setup), Nodepool
can end up connecting to the VM very quickly and grabbing the
initial (as opposed to the newly generated host key). When Zuul
tries to talk to the node, it has changed and it throws an error.
By disabling this, we allow Zuul to simply accept the host key on
first connection. This is low-risk since we're only using the
internal cloud network anyway.
Change-Id: I2b2347f37527a72866e92164e6779ab6b57ebfa7
diff --git a/nodepool/nodepool.yaml b/nodepool/nodepool.yaml
index 2f7dfea..b2f3a6e 100644
--- a/nodepool/nodepool.yaml
+++ b/nodepool/nodepool.yaml
@@ -18,6 +18,18 @@
- name: main
max-servers: 4
use-internal-ip: True
+ # Host key checking is disabled because:
+ # 1) We're using the internal IP so it's slightly less
+ # valuable (fewer attack vectors).
+ # 2) The images we're using appear to have a key baked into
+ # them which is overwritten at boot. Because we're using the
+ # internal IP, nodepool can end up connecting to the instance
+ # very quickly and retrieving the original host key rather
+ # than the new one (which is likely to be written a couple of
+ # seconds later). By disabling this in nodepool, we let Zuul
+ # just use the first key it finds (and it's likely to take
+ # long enough that it will have been updated by then).
+ host-key-checking: False
labels:
- name: debian-stretch-8G
instance-type: n1-standard-2
