Public Access
Stop a flooding remote growing the process, and let an ended session go
The terminal's memory was bounded by an argument that is not true of the SSH library underneath it, and the whole of CreditWindow's remark was that argument: the pump stops reading when the renderer falls behind, SSH's own receive window closes, the remote sshd blocks on write, and the backpressure reaches the process producing the output. The middle step does not happen. Renci.SshNet's Channels.Channel.OnData calls its window adjustment before it raises DataReceived and re-opens the channel window as soon as it drops below one packet, and ShellStream's own handler copies the bytes into a System.Net.ArrayBuffer without blocking and without a ceiling — a buffer that doubles on demand and is not given back until the stream is disposed. So a reader that stopped reading slowed the remote down by nothing at all. It only chose whose buffer the flood landed in, and it chose the one nothing bounds. Measured against the container this repository already runs sshd in: a reader that reads nothing while `yes` runs leaves ShellStream holding 8.9 MB after one second, 26.8 MB after three and 44.3 MB after five, its capacity stepping 16 → 32 → 64 MB. That is the report of "a massive memory leak on Windows"; it is not a Windows fault, it is simply where long sessions with heavy output live. The fix is to read continuously, which is what keeps SSH.NET's buffer empty, and to put the bound where it can actually be enforced. OutputBacklog is that bound: a byte-capped queue of unsent output, 4 MiB by default, which drops its oldest bytes when full and counts what it discarded. Oldest rather than newest, because the tail of a flood is the prompt somebody is waiting for. The two bounds are now separate and each is enforced where it means something. The backlog bounds what is waiting to be sent. The credit window goes on bounding what has been sent and not yet drawn — the queue inside the WebView that nothing on this side can measure — and is reserved against in the flush loop rather than the read loop. Reading is part of neither, because reading was never what the remote was waiting on. Dropping is announced rather than silent. A terminal quietly showing a different transcript from the one the remote sent is worse than one that says it could not keep up, because the second can be believed. The line goes out under its own opcode, TerminalServerOpcode.OutputDropped, rather than spliced into the output stream: what this application says about the transcript is not part of the transcript, and keeping them apart is also what makes the arithmetic hold — every byte read is either in an Output payload or counted in BytesDropped. It opens with an SGR reset, because the cut lands at an arbitrary offset and whatever escape sequence the remote was in the middle of has been left half-said. Frames are now capped at 64 KiB and taken straight into the frame buffer. The cap is the large-object-heap threshold rather than a guess — a quarter-megabyte array allocated sixty times a second is collected only by a generation 2 pass — and the direct take removes an allocation and a copy of every byte the terminal shows. The second half is the one that made the first half's cost persist. Nothing gave a session back when the remote hung up: TerminalWorkspace kept the entry, and with it an undisposed ISshConnection — an authenticated socket, its cipher state, and a keep-alive putting a packet on the wire every thirty seconds — plus the pump and SSH.NET's read buffer at whatever size the session's busiest moment had grown it to, for as long as the tab stayed open. A tab somebody typed `exit` into an hour ago was indistinguishable from a live one in everything except what it drew. ReleaseAsync disposes both, from AnnounceEndedAsync and after the announcement: the tab is told first, because tearing down an SSH connection in front of that would be latency on the one thing the user is watching for. The dictionary entry stays, because LiveSessionCount, IsSessionLive and the tab's own ENDED and RECONNECT all read it. SendInputAsync therefore asks about liveness rather than presence, since the shell channel is gone the moment the remote is. Two faults introduced on the way here and fixed, both worth naming because neither would have been visible from the outside: The read loop left through a condition on its cancellation token, and RunAsync reads a read loop that returns as the remote having closed the channel. A tab the user closed would have reported itself as a shell that exited on its own. It throws instead. With the reader finished and a renderer that had stopped acknowledging, the flush loop span: output to send, no credit to send it with, and a wait for more output that returned at once because there was some. RunAsync never completed, so the tab was never told and the connection was never released — a session invisible and permanent. The loop now leaves for the drain as soon as the backlog is complete, and RunAsync resets the credit window alongside completing it so a flusher already parked on an acknowledgement that will never come is released. Four tests asserted the old contract and now assert its reverse, which is the point rather than a relaxation — WhenCreditRunsOut_ThePumpStopsReading is now WhenCreditRunsOut_ThePumpKeepsReadingAndTheBacklogHoldsTheLine, and has to prove both halves at once. The throughput harness gained a counting transport, because a consumer that copies every frame into a 32 MB list was the slowest thing in the pipeline and a harness whose consumer cannot keep up measures itself. The fake shell gained a pace, so "a renderer that keeps up loses nothing" can be asked of a producer a network could actually deliver rather than of one running at memory bandwidth. The test that would have caught this in the first place is ARunawayRemote_KeepsBeingReadAndStaysInsideTheBacklog, and it needs the real SSH.NET and the real sshd to say anything: a fake shell hands over bytes when it is asked, and a real channel hands them over when they arrive. Under the old code its BytesRead would have stopped at exactly WindowBytes. 121 terminal tests pass over three consecutive runs with no flakes, 105 against the container, and the app, session, layout and remaining suites are unchanged and green. Manual checks 1.10a and 1.10b cover the two symptoms a person can see: a working set that plateaus under `yes`, and a connection that goes when the shell does rather than when the tab does.
This commit is contained in:
1 parent
597e5ae016
commit
9ead42171c
11 files changed
+1150
-144
No files matched your search
@@ -202,6 +202,44 @@ That is what the "is this still the page's socket" guard in `connect()` exists t
|
||||
input while the banner is *clear* is neither: the socket is open and dead, which nothing on this page can
|
||||
currently see — see the note at the end of 11.12a.
|
||||
|
||||
### 1.10a A remote that floods the terminal does not grow the process · **watch Task Manager**
|
||||
|
||||
Open a shell and run `yes dodossh`. Leave it running for a full minute with the tab in front of you, then
|
||||
switch to another screen and leave it another minute, then come back. Watch the application's memory in
|
||||
Task Manager (Details tab, *Memory (active private working set)*) the whole time.
|
||||
|
||||
**Pass:** memory settles at a plateau and stays there — the number moves as the GC breathes, but there is no
|
||||
staircase. Ctrl+C, and the prompt comes back straight away. Somewhere in the scrollback there may be a dim
|
||||
grey line reading `── N MB of output was dropped; the terminal could not keep up ──`; that line is a pass,
|
||||
not a failure. It is this client saying it discarded output it could not draw, which is what keeps the
|
||||
plateau flat, and the number after it tells you how far behind the display got.
|
||||
|
||||
**Failure means:** a working set that climbs steadily for the whole minute and does not come back down after
|
||||
Ctrl+C is the defect this check exists for, and it will not be in this repository's own buffers — it is
|
||||
SSH.NET's. `ShellStream` copies everything the channel delivers into a buffer that doubles and is never
|
||||
given back, and the channel keeps delivering because `Channel.OnData` re-opens the SSH receive window as
|
||||
bytes *arrive* rather than as they are consumed. Measured against the container in
|
||||
`DodoSSH.Client.Ssh.Tests`, a reader that stops entirely leaves that buffer holding 44 MB after five
|
||||
seconds. The only thing that keeps it empty is `TerminalSessionPump` reading continuously and bounding what
|
||||
it holds in `OutputBacklog` instead — so a staircase here means something has put a wait back into that read
|
||||
loop. `ARunawayRemote_KeepsBeingReadAndStaysInsideTheBacklog` is the test that fails first.
|
||||
|
||||
### 1.10b A tab whose shell exited stops costing anything · **needs a debugger or `netstat`**
|
||||
|
||||
Open a shell, type `exit`, and **leave the tab open**. Note the host and port. Then, from a terminal outside
|
||||
the application, run `netstat -ano | findstr <port>` and look for the connection to that host.
|
||||
|
||||
**Pass:** the connection is gone within a second or two of the shell exiting, even though the tab is still on
|
||||
screen showing the session ended and still offering RECONNECT. Clicking that reconnect still works, and
|
||||
closing the tab afterwards still works.
|
||||
|
||||
**Failure means:** a connection that is still ESTABLISHED for as long as the tab is open is the session
|
||||
holding an authenticated socket it has no further use for — with a keep-alive putting a packet on the wire
|
||||
every thirty seconds and SSH.NET's read buffer still at whatever size the session's busiest moment grew it
|
||||
to. It used to do exactly that, until whichever tab it was got closed. `TerminalWorkspace.ReleaseAsync` is
|
||||
what gives it back now, and `ASessionWhoseRemoteHasExited_GivesBackItsConnectionWithoutWaitingForTheTab` is
|
||||
the test that says so.
|
||||
|
||||
### 1.11 The right button copies and pastes, and never opens a menu
|
||||
|
||||
Open a shell and run something that prints a few lines — `ls -la /etc`. Drag a selection across part of the
|
||||
|
||||
Reference in new issue
Block a user