Public Access
The terminal's memory was bounded by an argument that is not true of the SSH library underneath it, and the whole of CreditWindow's remark was that argument: the pump stops reading when the renderer falls behind, SSH's own receive window closes, the remote sshd blocks on write, and the backpressure reaches the process producing the output. The middle step does not happen. Renci.SshNet's Channels.Channel.OnData calls its window adjustment before it raises DataReceived and re-opens the channel window as soon as it drops below one packet, and ShellStream's own handler copies the bytes into a System.Net.ArrayBuffer without blocking and without a ceiling — a buffer that doubles on demand and is not given back until the stream is disposed. So a reader that stopped reading slowed the remote down by nothing at all. It only chose whose buffer the flood landed in, and it chose the one nothing bounds. Measured against the container this repository already runs sshd in: a reader that reads nothing while `yes` runs leaves ShellStream holding 8.9 MB after one second, 26.8 MB after three and 44.3 MB after five, its capacity stepping 16 → 32 → 64 MB. That is the report of "a massive memory leak on Windows"; it is not a Windows fault, it is simply where long sessions with heavy output live. The fix is to read continuously, which is what keeps SSH.NET's buffer empty, and to put the bound where it can actually be enforced. OutputBacklog is that bound: a byte-capped queue of unsent output, 4 MiB by default, which drops its oldest bytes when full and counts what it discarded. Oldest rather than newest, because the tail of a flood is the prompt somebody is waiting for. The two bounds are now separate and each is enforced where it means something. The backlog bounds what is waiting to be sent. The credit window goes on bounding what has been sent and not yet drawn — the queue inside the WebView that nothing on this side can measure — and is reserved against in the flush loop rather than the read loop. Reading is part of neither, because reading was never what the remote was waiting on. Dropping is announced rather than silent. A terminal quietly showing a different transcript from the one the remote sent is worse than one that says it could not keep up, because the second can be believed. The line goes out under its own opcode, TerminalServerOpcode.OutputDropped, rather than spliced into the output stream: what this application says about the transcript is not part of the transcript, and keeping them apart is also what makes the arithmetic hold — every byte read is either in an Output payload or counted in BytesDropped. It opens with an SGR reset, because the cut lands at an arbitrary offset and whatever escape sequence the remote was in the middle of has been left half-said. Frames are now capped at 64 KiB and taken straight into the frame buffer. The cap is the large-object-heap threshold rather than a guess — a quarter-megabyte array allocated sixty times a second is collected only by a generation 2 pass — and the direct take removes an allocation and a copy of every byte the terminal shows. The second half is the one that made the first half's cost persist. Nothing gave a session back when the remote hung up: TerminalWorkspace kept the entry, and with it an undisposed ISshConnection — an authenticated socket, its cipher state, and a keep-alive putting a packet on the wire every thirty seconds — plus the pump and SSH.NET's read buffer at whatever size the session's busiest moment had grown it to, for as long as the tab stayed open. A tab somebody typed `exit` into an hour ago was indistinguishable from a live one in everything except what it drew. ReleaseAsync disposes both, from AnnounceEndedAsync and after the announcement: the tab is told first, because tearing down an SSH connection in front of that would be latency on the one thing the user is watching for. The dictionary entry stays, because LiveSessionCount, IsSessionLive and the tab's own ENDED and RECONNECT all read it. SendInputAsync therefore asks about liveness rather than presence, since the shell channel is gone the moment the remote is. Two faults introduced on the way here and fixed, both worth naming because neither would have been visible from the outside: The read loop left through a condition on its cancellation token, and RunAsync reads a read loop that returns as the remote having closed the channel. A tab the user closed would have reported itself as a shell that exited on its own. It throws instead. With the reader finished and a renderer that had stopped acknowledging, the flush loop span: output to send, no credit to send it with, and a wait for more output that returned at once because there was some. RunAsync never completed, so the tab was never told and the connection was never released — a session invisible and permanent. The loop now leaves for the drain as soon as the backlog is complete, and RunAsync resets the credit window alongside completing it so a flusher already parked on an acknowledgement that will never come is released. Four tests asserted the old contract and now assert its reverse, which is the point rather than a relaxation — WhenCreditRunsOut_ThePumpStopsReading is now WhenCreditRunsOut_ThePumpKeepsReadingAndTheBacklogHoldsTheLine, and has to prove both halves at once. The throughput harness gained a counting transport, because a consumer that copies every frame into a 32 MB list was the slowest thing in the pipeline and a harness whose consumer cannot keep up measures itself. The fake shell gained a pace, so "a renderer that keeps up loses nothing" can be asked of a producer a network could actually deliver rather than of one running at memory bandwidth. The test that would have caught this in the first place is ARunawayRemote_KeepsBeingReadAndStaysInsideTheBacklog, and it needs the real SSH.NET and the real sshd to say anything: a fake shell hands over bytes when it is asked, and a real channel hands them over when they arrive. Under the old code its BytesRead would have stopped at exactly WindowBytes. 121 terminal tests pass over three consecutive runs with no flakes, 105 against the container, and the app, session, layout and remaining suites are unchanged and green. Manual checks 1.10a and 1.10b cover the two symptoms a person can see: a working set that plateaus under `yes`, and a connection that goes when the shell does rather than when the tab does.
857 lines
38 KiB
C#
857 lines
38 KiB
C#
using System.Globalization;
|
|
using System.Net.WebSockets;
|
|
using System.Text;
|
|
using DodoSSH.Client.Ssh;
|
|
|
|
namespace DodoSSH.Client.Terminal.Tests;
|
|
|
|
/// <summary>
|
|
/// What the workspace waits for outside the process, and how long a session lives once it has one.
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// <para>
|
|
/// The renderer gate itself is not in question. <see cref="TerminalDataPlane.SendAsync"/> drops frames when
|
|
/// nothing is attached, so a session opened before the renderer arrives loses its <c>SessionOpened</c> frame
|
|
/// and streams output at a terminal that was never created — and that it opens when a renderer does attach
|
|
/// is covered by <see cref="TerminalDataPlaneTests.TheRenderer_Attaches"/>. What is worth a test here is the
|
|
/// half that used to be missing: waiting for a renderer that never arrives has to end.
|
|
/// </para>
|
|
/// <para>
|
|
/// The session count is here for a different reason. It is shown to a user on the unlock screen, as the
|
|
/// disclosure that locking the vault leaves shells running, and a wrong number there is not a cosmetic bug:
|
|
/// it either hides a live connection or invents one.
|
|
/// </para>
|
|
/// </remarks>
|
|
public sealed class TerminalWorkspaceTests
|
|
{
|
|
/// <summary>How long <see cref="WaitUntilAsync"/> polls before calling it a failure.</summary>
|
|
private static readonly TimeSpan PollTimeout = TimeSpan.FromSeconds(10);
|
|
|
|
[Fact]
|
|
public async Task WaitingForARendererThatNeverAttaches_GivesUp()
|
|
{
|
|
// The shipped failure this stands in for is a WebView2 that never initialises — no Evergreen
|
|
// runtime, an install blocked by policy, an AppContainer that cannot reach loopback. From this
|
|
// side they are identical and all look like the listener being up with nothing ever connecting
|
|
// to it. Before the wait was bounded this test would have hung instead of failing.
|
|
await using var workspace = CreateWorkspace(
|
|
new FakeConnectionFactory(), rendererTimeout: TimeSpan.FromMilliseconds(250));
|
|
|
|
workspace.Start();
|
|
|
|
await Should.ThrowAsync<TimeoutException>(async () =>
|
|
await workspace.WaitForRendererAsync(TestContext.Current.CancellationToken));
|
|
}
|
|
|
|
[Fact]
|
|
public async Task WaitingForARenderer_ObeysItsCancellationToken()
|
|
{
|
|
// The timeout is the backstop; the caller's token is what makes a Connect the user gave up on
|
|
// return at once rather than sitting out the rest of the wait. The timeout here is long enough
|
|
// that only cancellation can end this.
|
|
await using var workspace = CreateWorkspace(
|
|
new FakeConnectionFactory(), rendererTimeout: TimeSpan.FromMinutes(5));
|
|
|
|
workspace.Start();
|
|
|
|
using var cancellation = new CancellationTokenSource();
|
|
var wait = workspace.WaitForRendererAsync(cancellation.Token);
|
|
|
|
await cancellation.CancelAsync();
|
|
|
|
await Should.ThrowAsync<OperationCanceledException>(async () => await wait);
|
|
}
|
|
|
|
[Fact]
|
|
public async Task AnOpenSessionIsReportedAsLive()
|
|
{
|
|
var connections = new FakeConnectionFactory();
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
workspace.LiveSessionCount.ShouldBe(0);
|
|
|
|
await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
workspace.LiveSessionCount.ShouldBe(1);
|
|
|
|
await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
workspace.LiveSessionCount.ShouldBe(2);
|
|
}
|
|
|
|
/// <remarks>
|
|
/// The case a naive <c>sessions.Count</c> gets wrong. Nothing removes the entry when the remote closes
|
|
/// the channel by itself — what it held is given back, but the entry stays so the tab can go on saying
|
|
/// the session ended — so counting entries would report a shell that exited as still running. A user
|
|
/// deciding whether it is safe to walk away is the person that lie is told to.
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task ASessionWhoseRemoteHasExitedIsNotReportedAsLive()
|
|
{
|
|
// A shell with no output to give: its first read returns 0, which is a remote closing the
|
|
// channel, so the pump finishes on its own with nobody asking it to.
|
|
var connections = new FakeConnectionFactory(bytesPerShell: 0);
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
await WaitUntilAsync(() => workspace.LiveSessionCount == 0);
|
|
}
|
|
|
|
/// <remarks>
|
|
/// <para>
|
|
/// <see cref="SshConnectionPhase.OpeningShell"/> is the one phase no connection factory can report,
|
|
/// because by the time it happens the factory has handed back a connection and gone. If this layer did
|
|
/// not report it the card's last step would light only when the whole session opened, which is the one
|
|
/// moment the card is already being taken down — a step nobody would ever see lit.
|
|
/// </para>
|
|
/// <para>
|
|
/// Asserted as the whole sequence rather than as "contains OpeningShell", because the order is the part
|
|
/// that matters: a step list is only readable if what it is told arrives in the order it draws.
|
|
/// </para>
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task OpeningASession_ReportsTheShellPhaseTheFactoryCannot()
|
|
{
|
|
var connections = new FakeConnectionFactory();
|
|
var reported = new List<SshConnectionPhase>();
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
await workspace.OpenSessionAsync(
|
|
Request(),
|
|
TerminalSize.Default,
|
|
new DelegateProgress<SshConnectionPhase>(reported.Add),
|
|
TestContext.Current.CancellationToken);
|
|
|
|
reported.ShouldBe(
|
|
[
|
|
SshConnectionPhase.Reaching,
|
|
SshConnectionPhase.Authenticating,
|
|
SshConnectionPhase.OpeningShell,
|
|
],
|
|
"the factory's own phases, then the one this layer performs itself");
|
|
}
|
|
|
|
/// <remarks>
|
|
/// Nobody watching is the ordinary case — every caller but the connecting card passes null — so it is
|
|
/// worth one test that the null is a null and not a null reference.
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task OpeningASession_WorksWithNobodyWatchingItsPhases()
|
|
{
|
|
var connections = new FakeConnectionFactory();
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
var sessionId = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
workspace.IsSessionLive(sessionId).ShouldBeTrue();
|
|
}
|
|
|
|
[Fact]
|
|
public async Task ClosingASessionEndsItAndDisposesItsConnection()
|
|
{
|
|
var connections = new FakeConnectionFactory();
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
var sessionId = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
await workspace.CloseSessionAsync(sessionId);
|
|
|
|
workspace.LiveSessionCount.ShouldBe(0);
|
|
connections.Connections.ShouldHaveSingleItem().IsDisposed.ShouldBeTrue();
|
|
}
|
|
|
|
/// <remarks>
|
|
/// <para>
|
|
/// Per-session liveness, which the tab strip's status dot is. The aggregate count answers "may I walk
|
|
/// away"; this answers "is <em>this</em> tab still connected", and a tab that went on claiming a shell
|
|
/// it no longer has would be the same lie one row down.
|
|
/// </para>
|
|
/// <para>
|
|
/// The unknown-id case is asserted because it is what a stale tab asks. A session id this workspace
|
|
/// never issued, or has already closed, is not live — it must not throw and must not be optimistic.
|
|
/// </para>
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task LivenessIsAnsweredPerSession()
|
|
{
|
|
var connections = new FakeConnectionFactory();
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
var first = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
var second = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
workspace.IsSessionLive(first).ShouldBeTrue();
|
|
workspace.IsSessionLive(second).ShouldBeTrue();
|
|
workspace.IsSessionLive(9999).ShouldBeFalse("this workspace never issued that id");
|
|
|
|
await workspace.CloseSessionAsync(first);
|
|
|
|
workspace.IsSessionLive(first).ShouldBeFalse();
|
|
workspace.IsSessionLive(second).ShouldBeTrue("closing one tab must not disturb another");
|
|
}
|
|
|
|
/// <remarks>
|
|
/// The shell's connect path reads these back once, right after <see cref="TerminalWorkspace.OpenSessionAsync"/>
|
|
/// returns, to fill in the status bar's cipher and host-key facts — see <c>VaultViewModel.ConnectAndAnnounceAsync</c>.
|
|
/// Asserted the same way <see cref="LivenessIsAnsweredPerSession"/> asserts liveness: per session, and
|
|
/// null rather than thrown for an id this workspace never issued.
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task SessionFactsAreReadPerSession()
|
|
{
|
|
var connections = new FakeConnectionFactory();
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
var sessionId = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
var facts = workspace.GetSessionFacts(sessionId).ShouldNotBeNull();
|
|
var connection = connections.Connections.ShouldHaveSingleItem();
|
|
|
|
facts.Cipher.ShouldBe(connection.Cipher);
|
|
facts.HostKeyAlgorithm.ShouldBe(connection.HostKey.Algorithm);
|
|
|
|
workspace.GetSessionFacts(9999).ShouldBeNull("this workspace never issued that id");
|
|
}
|
|
|
|
/// <remarks>
|
|
/// <para>
|
|
/// Inserting a snippet has to be able to say whether it arrived, and the transport cannot: it drops
|
|
/// frames for a session nothing is listening to, so a send at a dead tab succeeds exactly as loudly as a
|
|
/// send at a live one. That is why <c>PasteAsync</c> answers rather than returning void — and why the
|
|
/// answer is a <see cref="bool"/> and not an exception, since a tab whose remote hung up an hour ago is
|
|
/// still on screen and still clickable.
|
|
/// </para>
|
|
/// <para>
|
|
/// What the renderer does with the frame — bracketed paste, and the Enter deliberately outside it —
|
|
/// cannot be reached from here at all. It is JavaScript inside a WebView, and it is in
|
|
/// <c>docs/manual-checks.md</c> for that reason.
|
|
/// </para>
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task PastingIntoADeadSessionSaysSoRatherThanDroppingIt()
|
|
{
|
|
var connections = new FakeConnectionFactory();
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
var sessionId = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
(await workspace.PasteAsync(
|
|
sessionId, "uptime", execute: false, TestContext.Current.CancellationToken))
|
|
.ShouldBeTrue("the session is live");
|
|
|
|
(await workspace.PasteAsync(
|
|
9999, "uptime", execute: false, TestContext.Current.CancellationToken))
|
|
.ShouldBeFalse("this workspace never issued that id");
|
|
|
|
await workspace.CloseSessionAsync(sessionId);
|
|
|
|
(await workspace.PasteAsync(
|
|
sessionId, "uptime", execute: false, TestContext.Current.CancellationToken))
|
|
.ShouldBeFalse("the tab it names is gone");
|
|
}
|
|
|
|
/// <remarks>
|
|
/// <para>
|
|
/// The right-click paste's answer, and it is a separate frame from the snippet one above rather than the
|
|
/// same frame with a flag: the page brackets this unconditionally and types a single-line snippet, which
|
|
/// is a difference of behaviour and not of degree. See <see cref="TerminalServerOpcode.Clipboard"/>.
|
|
/// </para>
|
|
/// <para>
|
|
/// The payload is the clipboard's text and nothing else — no leading flag byte, because there is no
|
|
/// Enter to ask for. A frame that grew one would put a stray character at the front of every paste, and
|
|
/// this is what would catch it.
|
|
/// </para>
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task TheClipboard_ReachesThePaneThatAskedForIt()
|
|
{
|
|
var connections = new FakeConnectionFactory(blockShellReads: true);
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
workspace.Start();
|
|
|
|
using var renderer = await ConnectRendererAsync(workspace);
|
|
|
|
var sessionId = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
// The session's opening frame, read and discarded so the clipboard frame below is the next one.
|
|
await ReceiveFrameAsync(renderer);
|
|
|
|
(await workspace.SendClipboardAsync(
|
|
sessionId, "sudo systemctl restart sshd", TestContext.Current.CancellationToken))
|
|
.ShouldBeTrue("the session is live");
|
|
|
|
var frame = await ReceiveFrameAsync(renderer);
|
|
|
|
frame.Opcode.ShouldBe((byte)TerminalServerOpcode.Clipboard);
|
|
frame.SessionId.ShouldBe(sessionId);
|
|
Encoding.UTF8.GetString(frame.Payload).ShouldBe("sudo systemctl restart sshd");
|
|
}
|
|
|
|
/// <remarks>
|
|
/// Refused rather than sent and dropped, for the reason <c>PasteAsync</c> is — with the difference that
|
|
/// nothing on screen would have shown it either way, so this answer exists to be honest rather than to
|
|
/// be reported. See <see cref="TerminalWorkspace.SendClipboardAsync"/>.
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task SendingTheClipboardToADeadSession_SaysSoRatherThanDroppingIt()
|
|
{
|
|
var connections = new FakeConnectionFactory();
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
var sessionId = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
(await workspace.SendClipboardAsync(9999, "uptime", TestContext.Current.CancellationToken))
|
|
.ShouldBeFalse("this workspace never issued that id");
|
|
|
|
await workspace.CloseSessionAsync(sessionId);
|
|
|
|
(await workspace.SendClipboardAsync(sessionId, "uptime", TestContext.Current.CancellationToken))
|
|
.ShouldBeFalse("the tab it names is gone");
|
|
}
|
|
|
|
/// <remarks>
|
|
/// The event the tab strip and the phone's keep-alive listen to, so a dot can go out — and a foreground
|
|
/// notification can come down — the moment a shell exits rather than at the next thing that happens to
|
|
/// repaint. The count captured inside the handler is the sharper half of this test: the announcement
|
|
/// used to fire from inside the run's own finally block, where the run task is not yet complete, so
|
|
/// <c>LiveSessionCount</c> read from the handler still said 1 — and the phone's notification went on
|
|
/// claiming a shell that was gone, with nothing left to fire and correct it.
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task ASessionEndingOnItsOwnIsAnnounced_AfterTheCountStoppedIncludingIt()
|
|
{
|
|
// A shell with no output to give: its first read returns 0, which is a remote closing the channel,
|
|
// so the pump finishes with nobody asking it to.
|
|
var connections = new FakeConnectionFactory(bytesPerShell: 0);
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
var ended = new List<uint>();
|
|
var liveAtAnnouncement = -1;
|
|
workspace.SessionEnded += (_, e) =>
|
|
{
|
|
lock (ended)
|
|
{
|
|
liveAtAnnouncement = workspace.LiveSessionCount;
|
|
ended.Add(e.SessionId);
|
|
}
|
|
};
|
|
|
|
var sessionId = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
await WaitUntilAsync(() =>
|
|
{
|
|
lock (ended)
|
|
{
|
|
return ended.Contains(sessionId);
|
|
}
|
|
});
|
|
|
|
lock (ended)
|
|
{
|
|
liveAtAnnouncement.ShouldBe(0, "the announcement must wait for the run to actually complete");
|
|
}
|
|
}
|
|
|
|
/// <remarks>
|
|
/// The reversal of a recorded decision, and the event's own remark carries why: a close used to be
|
|
/// announced to nobody, on the theory that the caller already knew — but the phone's keep-alive is not
|
|
/// the caller, and a close it never heard about left the foreground notification claiming a shell that
|
|
/// was gone. Announced once, after the drain, so the count a handler reads is already honest.
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task ClosingASessionIsAnnounced_OnceItHasDrained()
|
|
{
|
|
var connections = new FakeConnectionFactory();
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
var announcements = 0;
|
|
var liveAtAnnouncement = -1;
|
|
workspace.SessionEnded += (_, _) =>
|
|
{
|
|
liveAtAnnouncement = workspace.LiveSessionCount;
|
|
Interlocked.Increment(ref announcements);
|
|
};
|
|
|
|
var sessionId = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
await workspace.CloseSessionAsync(sessionId);
|
|
|
|
Volatile.Read(ref announcements)
|
|
.ShouldBe(1, "a close is news to the keep-alive even though it is an echo to the closer");
|
|
liveAtAnnouncement.ShouldBe(0, "announced after the drain, so the count already excludes it");
|
|
}
|
|
|
|
/// <remarks>
|
|
/// ◆ The announcement carries how it ended, not just that it did. A subscriber cannot work that out for
|
|
/// itself afterwards — by the time it hears, there is no transport left to interrogate — and the
|
|
/// difference between a shell that exited and a connection that broke is the difference between a tab
|
|
/// that is finished and one worth offering to dial again.
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task ASessionWhoseConnectionBroke_IsAnnouncedAsALostConnection()
|
|
{
|
|
var connections = new FakeConnectionFactory(bytesPerShell: 0)
|
|
{
|
|
TransportFailure = "An established connection was aborted by the server.",
|
|
};
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
TerminalSessionOutcome? outcome = null;
|
|
workspace.SessionEnded += (_, e) => Volatile.Write(ref outcome, e.Outcome);
|
|
|
|
await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
await WaitUntilAsync(() => Volatile.Read(ref outcome) is not null);
|
|
|
|
var announced = Volatile.Read(ref outcome)!;
|
|
|
|
announced.End.ShouldBe(SshSessionEnd.ConnectionLost);
|
|
announced.Description.ShouldBe("An established connection was aborted by the server.");
|
|
}
|
|
|
|
/// <remarks>
|
|
/// The same shell ending with nothing wrong underneath it, which is what <c>exit</c> looks like from
|
|
/// here. Its own test rather than an assertion tacked onto the one above, because the pair is the point:
|
|
/// one fake differs from the other by a latched failure and nothing else, and the two must not report
|
|
/// the same thing.
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task ASessionWhoseShellExited_IsAnnouncedAsTheRemoteClosingIt()
|
|
{
|
|
var connections = new FakeConnectionFactory(bytesPerShell: 0);
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
TerminalSessionOutcome? outcome = null;
|
|
workspace.SessionEnded += (_, e) => Volatile.Write(ref outcome, e.Outcome);
|
|
|
|
await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
await WaitUntilAsync(() => Volatile.Read(ref outcome) is not null);
|
|
|
|
Volatile.Read(ref outcome)!.End.ShouldBe(SshSessionEnd.RemoteClosed);
|
|
}
|
|
|
|
/// <remarks>
|
|
/// A tab being closed, announced as this client's own doing however the pump unwound. It matters for the
|
|
/// shell reusing a tab: a reconnect closes the session it is replacing, and an announcement blaming the
|
|
/// remote for that would draw a dropped connection over an attempt that is already under way.
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task ClosingASession_IsAnnouncedAsThisClientsDoing()
|
|
{
|
|
var connections = new FakeConnectionFactory();
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
TerminalSessionOutcome? outcome = null;
|
|
workspace.SessionEnded += (_, e) => outcome = e.Outcome;
|
|
|
|
var sessionId = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
await workspace.CloseSessionAsync(sessionId);
|
|
|
|
outcome.ShouldNotBeNull();
|
|
outcome.End.ShouldBe(SshSessionEnd.ClosedLocally);
|
|
}
|
|
|
|
/// <remarks>
|
|
/// Disposal is the other path that closes sessions, because it is process shutdown. Asserted so
|
|
/// that the SSH connections are known to be released rather than assumed to be — and that these closes,
|
|
/// unlike a deliberate one, are announced to nobody: shutdown is dismantling every subscriber along
|
|
/// with the sessions, and news nobody is left to hear is not news.
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task DisposingTheWorkspaceClosesEverySession()
|
|
{
|
|
var connections = new FakeConnectionFactory();
|
|
|
|
var workspace = CreateWorkspace(connections);
|
|
|
|
var announcements = 0;
|
|
workspace.SessionEnded += (_, _) => Interlocked.Increment(ref announcements);
|
|
|
|
await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
await workspace.DisposeAsync();
|
|
|
|
workspace.LiveSessionCount.ShouldBe(0);
|
|
connections.Connections.Count.ShouldBe(2);
|
|
connections.Connections.ShouldAllBe(connection => connection.IsDisposed);
|
|
Volatile.Read(ref announcements).ShouldBe(0, "shutdown closes are not announced");
|
|
}
|
|
|
|
// ---- Reattach ----
|
|
|
|
/// <remarks>
|
|
/// <para>
|
|
/// The scenario the whole fix exists for: a page that lost its socket — killed WebView renderer, or
|
|
/// simply a reload — reattaches, and the session that was already running has to come back rather than
|
|
/// sit there forever with its output going nowhere.
|
|
/// </para>
|
|
/// <para>
|
|
/// The session's shell blocks on every read rather than producing output, which is what an idle prompt
|
|
/// looks like and — for this test — is what keeps its <c>Run</c> live without a background read loop
|
|
/// competing with this test over the credit window's exact value.
|
|
/// </para>
|
|
/// <para>
|
|
/// Neither assertion below polls, deliberately. The "before" one does not need to: reserving credit is
|
|
/// a synchronous call, so it is true the instant it returns. The "after" one does not need to either,
|
|
/// for a subtler reason — <see cref="TerminalWorkspace.ReplayAfterAttachAsync"/> calls
|
|
/// <c>Credits.Reset()</c> and only then awaits sending the replay frame for that same session, with no
|
|
/// suspension between the two, so by the time this test has received that frame the reset has
|
|
/// necessarily already happened. A poll here would only have hidden a real ordering bug behind a
|
|
/// generous timeout instead of catching it.
|
|
/// </para>
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task ANewRenderer_ReplaysTheLiveSessionAndResetsItsCredits()
|
|
{
|
|
var connections = new FakeConnectionFactory(blockShellReads: true);
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
workspace.Start();
|
|
|
|
using var first = await ConnectRendererAsync(workspace);
|
|
|
|
var sessionId = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
// The session's own opening frame, sent as soon as the pump starts running. Not a replay, and not
|
|
// what this test is about — read and discarded so it cannot be confused for one below.
|
|
await ReceiveFrameAsync(first);
|
|
|
|
var credits = workspace.CreditsFor(sessionId).ShouldNotBeNull();
|
|
credits.TryReserve(4096);
|
|
credits.Outstanding.ShouldBeGreaterThanOrEqualTo(
|
|
4096, "the pump's own read loop may have reserved a buffer's worth on top of this");
|
|
|
|
using var second = await ConnectRendererAsync(workspace);
|
|
|
|
var replay = await ReceiveFrameAsync(second);
|
|
replay.Opcode.ShouldBe((byte)TerminalServerOpcode.SessionOpened);
|
|
replay.SessionId.ShouldBe(sessionId);
|
|
replay.Payload.ShouldBe(new byte[] { 1 }, "a replay is flagged so the page can tell it apart from a fresh open");
|
|
|
|
credits.Outstanding.ShouldBe(0, "the replay frame above cannot have been sent before the reset that precedes it");
|
|
}
|
|
|
|
/// <remarks>
|
|
/// The other half of a reattach: the workspace has replayed what it owns, and this is the seam the
|
|
/// shell uses to replay what it owns instead — the font size and the selected tab, neither of which a
|
|
/// terminal session knows anything about. <c>MainWindowViewModel</c>'s subscription is what actually
|
|
/// does that; this only asserts that the workspace hands it the chance to.
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task ANewRenderer_RaisesRendererReattached()
|
|
{
|
|
var connections = new FakeConnectionFactory();
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
workspace.Start();
|
|
|
|
using var first = await ConnectRendererAsync(workspace);
|
|
|
|
var reattachedCount = 0;
|
|
workspace.RendererReattached += (_, _) => Interlocked.Increment(ref reattachedCount);
|
|
|
|
using var second = await ConnectRendererAsync(workspace);
|
|
|
|
await WaitUntilAsync(() => Volatile.Read(ref reattachedCount) > 0);
|
|
}
|
|
|
|
// ---- Helpers ----
|
|
|
|
/// <remarks>
|
|
/// <para>
|
|
/// Input that did not come from the keyboard. The Android head's accessory key row is what needs this —
|
|
/// a software keyboard has no Ctrl, Esc, Tab or arrows — and what it sends has to arrive at the remote
|
|
/// byte for byte, because an escape sequence that loses a byte is not a degraded arrow key, it is a
|
|
/// stray character in somebody's shell.
|
|
/// </para>
|
|
/// <para>
|
|
/// Ordinary typing does not come this way and is not what is being tested: that goes from the renderer
|
|
/// down the socket, which the pump's own tests cover.
|
|
/// </para>
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task SendingInput_ReachesTheRemoteUnchanged()
|
|
{
|
|
var connections = new FakeConnectionFactory();
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
var sessionId = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
// The up-arrow, as a PTY expects it. Three bytes, and all three matter.
|
|
byte[] upArrow = [0x1B, (byte)'[', (byte)'A'];
|
|
|
|
await workspace.SendInputAsync(sessionId, upArrow, TestContext.Current.CancellationToken);
|
|
|
|
await WaitUntilAsync(() =>
|
|
connections.Connections.SingleOrDefault()?.Shell?.Written.SequenceEqual(upArrow) == true);
|
|
}
|
|
|
|
/// <remarks>
|
|
/// A tab can close while a key is still in flight, which on a phone is one mis-tap rather than a rare
|
|
/// race — the close cross sits inside the tab and the accessory row is directly under it. Throwing
|
|
/// would turn that into a crash on a keystroke that no longer matters.
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task SendingInputToASessionThatIsGone_IsIgnored()
|
|
{
|
|
await using var workspace = CreateWorkspace(new FakeConnectionFactory());
|
|
|
|
var sessionId = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
await workspace.CloseSessionAsync(sessionId);
|
|
|
|
await Should.NotThrowAsync(async () =>
|
|
await workspace.SendInputAsync(sessionId, "x"u8.ToArray(), TestContext.Current.CancellationToken));
|
|
}
|
|
|
|
/// <summary>
|
|
/// A shell that ended on its own gives its connection back at once, without waiting for the tab.
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// ◆ THE HALF THAT WAS MISSING. Closing a tab disposed the connection and always had; a remote hanging
|
|
/// up did not, so a session that ended went on holding an authenticated socket, a keep-alive putting a
|
|
/// packet on the wire every thirty seconds, and SSH.NET's read buffer at whatever size the session's
|
|
/// busiest moment had grown it to — for as long as the user left the tab open, which for a tab showing
|
|
/// the last thing a job said is indefinitely.
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task ASessionWhoseRemoteHasExited_GivesBackItsConnectionWithoutWaitingForTheTab()
|
|
{
|
|
// A shell with no output to give: its first read returns 0, which is the remote closing the
|
|
// channel, so the pump finishes with nobody having asked it to.
|
|
var connections = new FakeConnectionFactory(bytesPerShell: 0);
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
var sessionId = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
await WaitUntilAsync(() => connections.Connections.TrueForAll(connection => connection.IsDisposed));
|
|
|
|
var connection = connections.Connections.ShouldHaveSingleItem();
|
|
|
|
connection.IsConnected.ShouldBeFalse();
|
|
connection.Shell.ShouldNotBeNull().IsOpen.ShouldBeFalse();
|
|
|
|
// The entry itself stays, and has to: the tab is still on screen, still says it ended, and still
|
|
// offers a reconnect — all of which read this workspace rather than the tab strip's own memory.
|
|
workspace.CreditsFor(sessionId).ShouldNotBeNull();
|
|
workspace.IsSessionLive(sessionId).ShouldBeFalse();
|
|
}
|
|
|
|
/// <remarks>
|
|
/// The consequence of the release above, and the reason <c>SendInputAsync</c> asks about liveness
|
|
/// rather than presence: the shell channel is disposed the moment the remote hangs up, so an accessory
|
|
/// key pressed at a tab that ended reaches a disposed stream. It has to be as ordinary as pressing one
|
|
/// at a tab that has closed, which is to say nothing happens.
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task SendingInputToASessionWhoseRemoteHasExited_IsIgnored()
|
|
{
|
|
var connections = new FakeConnectionFactory(bytesPerShell: 0);
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
var sessionId = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
await WaitUntilAsync(() => !workspace.IsSessionLive(sessionId));
|
|
|
|
await Should.NotThrowAsync(async () =>
|
|
await workspace.SendInputAsync(sessionId, "x"u8.ToArray(), TestContext.Current.CancellationToken));
|
|
}
|
|
|
|
/// <remarks>
|
|
/// Closing a tab over a session that already released itself must still work, because that is the
|
|
/// ordinary way a dead tab goes away. Everything the close does is idempotent by construction — the
|
|
/// entry is removed under the lock, and both disposals are guarded — and this is the test that says so
|
|
/// rather than leaving it to be inferred from three separate classes.
|
|
/// </remarks>
|
|
[Fact]
|
|
public async Task ClosingATabOverASessionThatAlreadyEnded_IsStillSafe()
|
|
{
|
|
var connections = new FakeConnectionFactory(bytesPerShell: 0);
|
|
|
|
await using var workspace = CreateWorkspace(connections);
|
|
|
|
var sessionId = await workspace.OpenSessionAsync(
|
|
Request(), TerminalSize.Default, progress: null, TestContext.Current.CancellationToken);
|
|
|
|
await WaitUntilAsync(() => !workspace.IsSessionLive(sessionId));
|
|
|
|
await Should.NotThrowAsync(() => workspace.CloseSessionAsync(sessionId));
|
|
|
|
workspace.CreditsFor(sessionId).ShouldBeNull();
|
|
}
|
|
|
|
/// <param name="connections">
|
|
/// How connections are made. The renderer-gate tests never reach it — they stop at the gate — but they
|
|
/// take a fake anyway, because reaching a real host from here would make this a network test.
|
|
/// </param>
|
|
/// <param name="rendererTimeout">The gate's bound, or null for the shipped default.</param>
|
|
private static TerminalWorkspace CreateWorkspace(
|
|
ISshConnectionFactory connections,
|
|
TimeSpan? rendererTimeout = null) =>
|
|
new(
|
|
StubAssets(),
|
|
connections,
|
|
TimeProvider.System,
|
|
rendererTimeout is { } timeout
|
|
? new TerminalWorkspaceOptions { RendererTimeout = timeout }
|
|
: null);
|
|
|
|
private static InMemoryTerminalAssetProvider StubAssets() =>
|
|
new(new Dictionary<string, TerminalAsset>(StringComparer.Ordinal)
|
|
{
|
|
// The placeholders, not a token and URL already filled in — the reattach tests below have to
|
|
// connect a real renderer, and doing that by reading them back out of the served page is what
|
|
// proves the workspace serves a page a real renderer could actually attach with, rather than
|
|
// one that merely looks servable.
|
|
[TerminalDataPlane.PagePath] = new(
|
|
"text/html; charset=utf-8",
|
|
Encoding.UTF8.GetBytes(
|
|
$"<html><body data-token=\"{TerminalDataPlane.TokenPlaceholder}\" "
|
|
+ $"data-socket=\"{TerminalDataPlane.SocketUrlPlaceholder}\"></body></html>")),
|
|
});
|
|
|
|
private static SshConnectionRequest Request() =>
|
|
new("host.invalid", 22, "dodo", new SshPasswordCredential("irrelevant"));
|
|
|
|
/// <remarks>
|
|
/// <para>
|
|
/// Attaches the way the real page does: by fetching the served page, reading the token and socket URL
|
|
/// back out of it, and presenting them on the upgrade — rather than reaching into the workspace for a
|
|
/// token it does not expose. A shortcut here would prove only that a socket can be opened, not that the
|
|
/// workspace serves a page a renderer could actually attach with.
|
|
/// </para>
|
|
/// <para>
|
|
/// And it waits for the attach rather than only for the handshake, for the reason
|
|
/// <c>TerminalDataPlaneTests.ConnectAsync</c> sets out at length: the 101 is written before the socket
|
|
/// is attachable, and a frame sent in between is dropped. A caller that opens a session on the socket
|
|
/// this returns and then reads its opening frame is exactly the shape that loses that race.
|
|
/// </para>
|
|
/// <para>
|
|
/// <c>WaitForRendererAsync</c> answers only for the first renderer ever to attach, so a second call
|
|
/// returns immediately without proving anything about the second socket. That is enough here and is
|
|
/// not luck: the only frames a second socket is given are the replay, and the replay is *caused* by the
|
|
/// attach — <see cref="TerminalWorkspace.RendererReattached"/> and the frames before it cannot be sent
|
|
/// early. The day a test sends something else down a reattached socket, this needs the data plane's own
|
|
/// <c>SocketAttached</c>, which the workspace does not forward today.
|
|
/// </para>
|
|
/// </remarks>
|
|
private static async Task<ClientWebSocket> ConnectRendererAsync(TerminalWorkspace workspace)
|
|
{
|
|
using var http = new HttpClient();
|
|
var page = await http.GetStringAsync(workspace.PageUrl, TestContext.Current.CancellationToken);
|
|
|
|
var token = ExtractAttribute(page, "data-token");
|
|
var socketUrl = ExtractAttribute(page, "data-socket");
|
|
|
|
var client = new ClientWebSocket();
|
|
client.Options.AddSubProtocol(TerminalDataPlane.SubProtocol);
|
|
client.Options.AddSubProtocol($"token.{token}");
|
|
client.Options.SetRequestHeader(
|
|
"Origin",
|
|
string.Create(CultureInfo.InvariantCulture, $"http://127.0.0.1:{workspace.PageUrl.Port}"));
|
|
|
|
try
|
|
{
|
|
await client.ConnectAsync(new Uri(socketUrl), TestContext.Current.CancellationToken);
|
|
|
|
await workspace.WaitForRendererAsync(TestContext.Current.CancellationToken);
|
|
}
|
|
catch
|
|
{
|
|
client.Dispose();
|
|
throw;
|
|
}
|
|
|
|
return client;
|
|
}
|
|
|
|
private static string ExtractAttribute(string html, string name)
|
|
{
|
|
var marker = $"{name}=\"";
|
|
var start = html.IndexOf(marker, StringComparison.Ordinal) + marker.Length;
|
|
var end = html.IndexOf('"', start);
|
|
|
|
return html[start..end];
|
|
}
|
|
|
|
private static async Task<(byte Opcode, uint SessionId, byte[] Payload)> ReceiveFrameAsync(
|
|
ClientWebSocket socket)
|
|
{
|
|
var buffer = new byte[64 * 1024];
|
|
|
|
var result = await socket.ReceiveAsync(buffer.AsMemory(), TestContext.Current.CancellationToken);
|
|
|
|
TerminalFrame.TryRead(buffer.AsSpan(0, result.Count), out var opcode, out var sessionId, out var payload)
|
|
.ShouldBeTrue();
|
|
|
|
return (opcode, sessionId, payload.ToArray());
|
|
}
|
|
|
|
/// <remarks>
|
|
/// Polled rather than awaited on a task, because the point is what an observer of the property
|
|
/// sees: the pump ends on a thread of its own, and the count has to catch up without anyone
|
|
/// telling it to.
|
|
/// </remarks>
|
|
private static async Task WaitUntilAsync(Func<bool> condition)
|
|
{
|
|
var deadline = TimeProvider.System.GetUtcNow() + PollTimeout;
|
|
|
|
while (TimeProvider.System.GetUtcNow() < deadline)
|
|
{
|
|
if (condition())
|
|
{
|
|
return;
|
|
}
|
|
|
|
await Task.Delay(20, TestContext.Current.CancellationToken);
|
|
}
|
|
|
|
throw new TimeoutException($"The condition was still false after {PollTimeout}.");
|
|
}
|
|
}
|