Stop the SSH suite's server refusing connections at random
ci / build and test (pull_request) Successful in 2m21s
ci / desktop nightly (pull_request) Skipped
ci / android head (pull_request) Successful in 3m20s
ci / api image (pull_request) Successful in 4s

The suite fails intermittently with SshConnectionException "The connection
was closed by the remote host", within tens of milliseconds, on whichever
test happens to connect first. It has been seen in CI and reproduces
locally. This raises sshd's MaxStartups in the fixture, which is the most
likely cause and is worth doing regardless.

sshd's compiled-in default is 10:30:100: past ten unauthenticated
connections in flight it refuses new ones at random, thirty percent of the
time, rising to always at a hundred. The image ships the line commented
out, so that default was what ran. xUnit runs test classes in parallel and
most of the classes here open a connection, so ten in flight is reachable
during the opening seconds — and a refusal presents to the client exactly
as observed, because a dropped connection and a server that never answered
are indistinguishable from that end.

◆ IT IS A MITIGATION AND NOT A DEMONSTRATED CURE, AND THE COMMENT SAYS SO.

The flake rate could not be measured. On the Windows development machine
the identical unmodified suite ran 85/85 clean and, an hour later, failed
13 runs out of 15; a Linux container gave 30/30 clean and then failed on
the first run of the next batch. Docker throughput on that host swings far
enough to swamp the effect, so every before/after comparison taken there
was noise — including two that were briefly believed.

It is committed on the narrower argument that it is right either way. A
connection throttle is hardening this suite has no interest in
reproducing: it exists to test an SSH client, not to survive a rate limit,
and a test server that drops connections at random is a bad test server
whether or not it is the cause of this particular flake.

The other candidate was the reload window — pkill returns when SIGHUP is
delivered, not when sshd has finished closing its listeners and re-execing,
so a connection immediately afterwards can be refused the same way. A wait
that required three consecutive banner reads before returning was written
and then removed: it could not be shown to change anything either, and a
fixture carrying two unproven fixes for one symptom is worse than one,
because the next person has to disprove both. Both candidates, and how to
tell them apart with sshd's own log, are recorded in the fixture and in
docs/platform-flags.md.
This commit is contained in:
2026-08-10 11:46:14 +02:00
parent 7f77539ba6
commit e41eca01a8
2 changed files with 71 additions and 1 deletions
+13
View File
@@ -388,6 +388,19 @@ agent of our own plus ProxyJump covers the real use cases.
**The SSH suite pulls `linuxserver/openssh-server` from Docker Hub**, which is rate-limited for
unauthenticated pulls. If CI starts failing on image pulls rather than on tests, that is why.
**That suite has an intermittent `The connection was closed by the remote host`**, on whichever test
connects first, within tens of milliseconds. Seen in CI and reproducible locally. *Mitigated, not
solved:* `SshServerFixture` now raises sshd's `MaxStartups` from its compiled-in `10:30:100`, which
refuses connections at random past ten unauthenticated ones in flight — reachable because xUnit runs
test classes in parallel and most of them connect. The fixture comment carries the full argument and
is explicit that the cure is unproven.
**And the reason it is unproven is a measurement trap worth not falling into twice.** Docker
throughput on the Windows development machine swings enough to swamp the effect: the identical
unmodified suite ran 85/85 clean and, an hour later, failed 13 runs out of 15. Any before/after flake
comparison taken there is noise. Measure this class of thing in CI, or make the server say why —
raise sshd's `LogLevel`, disable Ryuk so the container outlives the run, and read `docker logs`.
**MSIX packaging is ruled out, not merely deprioritised.** A packaged app runs WebView2 in an
AppContainer where loopback connections are blocked without a `CheckNetIsolation` exemption. The
terminal data plane *is* a loopback WebSocket, so MSIX would break the product outright. Velopack