nodepool: re-enable host-key-checking to gate node SSH readiness

The main pool set host-key-checking: False, which removes nodepool's
SSH-readiness gate: nodes can be handed to the executor before SSH/host-
key-scan readiness is confirmed, so ~1/3 of builds fail at the first
Ansible connect with RETRY_LIMIT (~15-18s, no console logs). This is the
chronic CI flakiness tracked in issue 502048297.

Serial console of three consecutive failing (RETRY_LIMIT) nodes shows
they boot cleanly and become SSH-ready at a consistent ~31s after VM
creation -- not slow, not resource-starved:

  node 10581: created 14:27:20Z, sshd started 14:27:51Z  (+31s)
  node 10582: created 14:29:44Z, sshd started 14:30:13Z  (+29s)
  node 10583: created 14:32:05Z, sshd started 14:32:38Z  (+33s)

On this Debian image the host keys are regenerated ~2s before sshd
binds, so a post-sshd ssh-keyscan should capture the regenerated key,
not the baked-in one.

Without a readiness gate nodepool marks the node ready when GCE reports
RUNNING (~+10-15s), long before sshd is up at ~+31s. The executor then
does ~19s of merge/checkout and connects at ~+30-34s -- right on the
readiness boundary -- so a few seconds of jitter decides each attempt:
connect before the node is SSH-ready -> RETRY; connect after -> success.
That is the ~1/3 pass rate. (The exact failing state -- sshd not yet
bound vs the zuul authorized_keys not yet written -- is not pinned, as
the retried builds' console is discarded.)

nodepool's GCE driver defaults host-key-checking to True: when true it
ssh-keyscans a node before supplying it and relaunches nodes that never
become reachable, absorbing the wait inside the launch pipeline instead
of racing the executor. It relies on the launcher reaching the node
internal IP, which is how this deployment is configured. It was disabled
to dodge the GCE baked-in-host-key race; the serial evidence above
indicates a post-sshd scan gets the regenerated key, so restoring the
default should not reintroduce it -- validate the launcher log post
merge.

Upstream confirms both mechanism and guidance: the GCE driver docs
(zuul/nodepool: [1]) say host-key-checking defaults to True and "helps
ensure that nodes are ready to receive SSH connections before they ar
supplied to the requestor", to be disabled only for launcher/node
network separation or non-SSH platforms -- neither applies here. The
gating is code-confirmed by zuul/nodepool: [2] (host-key-checking
controls the statemachine/GCE driver's keyscan).

This is a hypothesis-driven change; nodepool config is not speculatively
testable. Validate post-merge: watch the RETRY_LIMIT rate and the
launcher log for ssh-keyscan / host-key failures and relaunches. If
host-key failures appear, roll back.

[1] https://review.opendev.org/c/zuul/nodepool/+/841908
[2] https://review.opendev.org/c/zuul/nodepool/+/821481

Issue: 502048297
Change-Id: I673366b351c74482e7c11f7b4e0c235fd6e330fa
diff --git a/nodepool/nodepool.yaml b/nodepool/nodepool.yaml
index 73199f1..e751f0b 100644
--- a/nodepool/nodepool.yaml
+++ b/nodepool/nodepool.yaml
@@ -23,18 +23,23 @@
       - name: main
         max-servers: 1
         use-internal-ip: True
-        # Host key checking is disabled because:
-        # 1) We're using the internal IP so it's slightly less
-        # valuable (fewer attack vectors).
-        # 2) The images we're using appear to have a key baked into
-        # them which is overwritten at boot.  Because we're using the
-        # internal IP, nodepool can end up connecting to the instance
-        # very quickly and retrieving the original host key rather
-        # than the new one (which is likely to be written a couple of
-        # seconds later).  By disabling this in nodepool, we let Zuul
-        # just use the first key it finds (and it's likely to take
-        # long enough that it will have been updated by then).
-        host-key-checking: False
+        # Gate node readiness on SSH/host-key-scan reachability
+        # (nodepool's GCE default). With this enabled nodepool verifies a node
+        # is SSH/host-key-scan reachable before handing it to the executor, and
+        # relaunches nodes that never become reachable. This relies on
+        # the launcher being able to reach the node internal IP, which
+        # is how this deployment is configured. It was
+        # previously disabled to dodge a GCE baked-in-host-key race
+        # (the image host key is regenerated a few seconds into boot);
+        # that removed the readiness gate and let the executor receive
+        # not-yet-SSH-ready nodes, causing intermittent RETRY_LIMIT
+        # (~1/3 of builds; see issue 502048297). On this Debian image
+        # serial output shows host keys are regenerated before sshd
+        # binds, so a post-sshd ssh-keyscan should capture the
+        # regenerated key, not the baked-in one. Validate the
+        # RETRY_LIMIT rate post-merge; roll back if host-key-scan
+        # failures appear in the launcher log.
+        host-key-checking: True
         labels:
           - name: debian-bookworm-8G
             instance-type: n1-standard-2