DodoTech-Public/DodoSSH
Public Access
3
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Stop a flooding remote growing the process, and let an ended session go
ci / build and test (push) Successful in 3m12s
Details
ci / desktop nightly (push) Successful in 1m16s
Details
ci / api image (push) Successful in 45s
Details
ci / android head (push) Successful in 7m11s
Details

Browse Source 
The terminal's memory was bounded by an argument that is not true of the SSH
library underneath it, and the whole of CreditWindow's remark was that argument:
the pump stops reading when the renderer falls behind, SSH's own receive window
closes, the remote sshd blocks on write, and the backpressure reaches the process
producing the output. The middle step does not happen. Renci.SshNet's
Channels.Channel.OnData calls its window adjustment before it raises DataReceived
and re-opens the channel window as soon as it drops below one packet, and
ShellStream's own handler copies the bytes into a System.Net.ArrayBuffer without
blocking and without a ceiling — a buffer that doubles on demand and is not given
back until the stream is disposed. So a reader that stopped reading slowed the
remote down by nothing at all. It only chose whose buffer the flood landed in,
and it chose the one nothing bounds.

Measured against the container this repository already runs sshd in: a reader
that reads nothing while `yes` runs leaves ShellStream holding 8.9 MB after one
second, 26.8 MB after three and 44.3 MB after five, its capacity stepping 16 → 32
→ 64 MB. That is the report of "a massive memory leak on Windows"; it is not a
Windows fault, it is simply where long sessions with heavy output live.

The fix is to read continuously, which is what keeps SSH.NET's buffer empty, and
to put the bound where it can actually be enforced. OutputBacklog is that bound:
a byte-capped queue of unsent output, 4 MiB by default, which drops its oldest
bytes when full and counts what it discarded. Oldest rather than newest, because
the tail of a flood is the prompt somebody is waiting for.

The two bounds are now separate and each is enforced where it means something.
The backlog bounds what is waiting to be sent. The credit window goes on bounding
what has been sent and not yet drawn — the queue inside the WebView that nothing
on this side can measure — and is reserved against in the flush loop rather than
the read loop. Reading is part of neither, because reading was never what the
remote was waiting on.

Dropping is announced rather than silent. A terminal quietly showing a different
transcript from the one the remote sent is worse than one that says it could not
keep up, because the second can be believed. The line goes out under its own
opcode, TerminalServerOpcode.OutputDropped, rather than spliced into the output
stream: what this application says about the transcript is not part of the
transcript, and keeping them apart is also what makes the arithmetic hold — every
byte read is either in an Output payload or counted in BytesDropped. It opens
with an SGR reset, because the cut lands at an arbitrary offset and whatever
escape sequence the remote was in the middle of has been left half-said.

Frames are now capped at 64 KiB and taken straight into the frame buffer. The cap
is the large-object-heap threshold rather than a guess — a quarter-megabyte array
allocated sixty times a second is collected only by a generation 2 pass — and the
direct take removes an allocation and a copy of every byte the terminal shows.

The second half is the one that made the first half's cost persist. Nothing gave
a session back when the remote hung up: TerminalWorkspace kept the entry, and
with it an undisposed ISshConnection — an authenticated socket, its cipher state,
and a keep-alive putting a packet on the wire every thirty seconds — plus the
pump and SSH.NET's read buffer at whatever size the session's busiest moment had
grown it to, for as long as the tab stayed open. A tab somebody typed `exit` into
an hour ago was indistinguishable from a live one in everything except what it
drew. ReleaseAsync disposes both, from AnnounceEndedAsync and after the
announcement: the tab is told first, because tearing down an SSH connection in
front of that would be latency on the one thing the user is watching for. The
dictionary entry stays, because LiveSessionCount, IsSessionLive and the tab's own
ENDED and RECONNECT all read it. SendInputAsync therefore asks about liveness
rather than presence, since the shell channel is gone the moment the remote is.

Two faults introduced on the way here and fixed, both worth naming because
neither would have been visible from the outside:

The read loop left through a condition on its cancellation token, and RunAsync
reads a read loop that returns as the remote having closed the channel. A tab the
user closed would have reported itself as a shell that exited on its own. It
throws instead.

With the reader finished and a renderer that had stopped acknowledging, the flush
loop span: output to send, no credit to send it with, and a wait for more output
that returned at once because there was some. RunAsync never completed, so the tab
was never told and the connection was never released — a session invisible and
permanent. The loop now leaves for the drain as soon as the backlog is complete,
and RunAsync resets the credit window alongside completing it so a flusher already
parked on an acknowledgement that will never come is released.

Four tests asserted the old contract and now assert its reverse, which is the
point rather than a relaxation — WhenCreditRunsOut_ThePumpStopsReading is now
WhenCreditRunsOut_ThePumpKeepsReadingAndTheBacklogHoldsTheLine, and has to prove
both halves at once. The throughput harness gained a counting transport, because
a consumer that copies every frame into a 32 MB list was the slowest thing in the
pipeline and a harness whose consumer cannot keep up measures itself. The fake
shell gained a pace, so "a renderer that keeps up loses nothing" can be asked of a
producer a network could actually deliver rather than of one running at memory
bandwidth.

The test that would have caught this in the first place is
ARunawayRemote_KeepsBeingReadAndStaysInsideTheBacklog, and it needs the real
SSH.NET and the real sshd to say anything: a fake shell hands over bytes when it
is asked, and a real channel hands them over when they arrive. Under the old code
its BytesRead would have stopped at exactly WindowBytes.

121 terminal tests pass over three consecutive runs with no flakes, 105 against
the container, and the app, session, layout and remaining suites are unchanged and
green. Manual checks 1.10a and 1.10b cover the two symptoms a person can see: a
working set that plateaus under `yes`, and a connection that goes when the shell
does rather than when the tab does.
This commit is contained in:
jaap-jan committed 2026-09-04 17:16:23 +02:00
1 parent 597e5ae016
commit 9ead42171c
11 files changed
+1150 -144
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

src/DodoSSH.Client.Terminal/TerminalSessionPump.cs
+257 -102
View File
@@ -1,6 +1,6 @@
using System.Buffers;
using System.Globalization;
using System.Text;
using System.Threading.Channels;
using DodoSSH.Client.Ssh;
namespace DodoSSH.Client.Terminal;
@@ -49,6 +49,16 @@ public sealed class TerminalPumpOptions
/// <summary>How far behind the renderer may fall, in bytes.</summary>
public int WindowBytes { get; init; } = CreditWindow.DefaultWindowBytes;
/// <summary>
/// How much unsent output to hold for a renderer that is behind, before dropping the oldest.
/// </summary>
/// <remarks>
/// This is the number that bounds a session's memory, and <see cref="OutputBacklog"/> says at length
/// why it rather than <see cref="WindowBytes"/> is the one that does. Larger drops less and shows a
/// flood's tail later; smaller drops sooner and keeps the terminal closer to live.
/// </remarks>
public int BacklogBytes { get; init; } = OutputBacklog.DefaultCapacityBytes;
}
/// <summary>
@@ -56,27 +66,45 @@ public sealed class TerminalPumpOptions
/// </summary>
/// <remarks>
/// <para>
/// Credit is reserved before reading, never after. That ordering is the whole design: because the pump
/// cannot read more than the renderer has room for, the coalescing buffer is bounded by the credit
/// window rather than by how fast the remote can talk. Reserving after reading would leave an
/// unbounded queue between the socket and the screen, which is the failure mode this exists to
/// prevent.
/// ◆ <b>The pump reads the shell channel continuously and never stops, which is deliberate rather than
/// careless.</b> It used to stop as soon as the renderer fell behind, on the reasoning that a reader
/// which does not read closes SSH's receive window, blocks the remote <c>sshd</c> on write, and blocks
/// the process producing the output in turn — backpressure all the way to the source, with no custom
/// protocol. That reasoning is wrong for this client's SSH library, and wrong in the direction that costs
/// memory: SSH.NET re-opens the channel window when bytes <em>arrive</em> rather than when they are
/// consumed, and keeps everything it has been handed in a growable array it does not give back. Not
/// reading slowed nothing down; it only moved the flood into a buffer with no ceiling.
/// <see cref="OutputBacklog"/> carries the full account.
/// </para>
/// <para>
/// When credit runs out the pump stops reading. SSH's own receive window then closes, the remote
/// <c>sshd</c> blocks on write, and the process producing output blocks in turn — backpressure all the
/// way to the source, with no custom protocol.
/// So there are two bounds now, each enforced where it can be. The backlog bounds what is waiting to be
/// sent, by dropping its oldest bytes and saying so. The credit window bounds what has been sent and not
/// yet drawn — the buffer inside the WebView that nothing on this side can measure — and the flush loop
/// reserves against it before every frame, holding off while there is none. Reading is no longer part of
/// either, because reading was never what the remote was waiting on.
/// </para>
/// </remarks>
public sealed class TerminalSessionPump : IAsyncDisposable
{
/// <summary>
/// The most output one frame carries.
/// </summary>
/// <remarks>
/// Under the 85,000-byte threshold at which the runtime allocates on the large object heap, and that is
/// the whole of why the number is here: a frame is allocated per send, and a session doing megabytes a
/// second would otherwise put a quarter-megabyte array on the LOH sixty times a second, which is not
/// compacted and is collected only with a generation 2 pass. Below the threshold the same traffic is
/// generation 0 garbage that costs nothing to collect. It is a ceiling rather than a target — a quiet
/// terminal sends whatever it has, which is a few bytes.
/// </remarks>
private const int MaximumFrameBytes = 64 * 1024;
private readonly uint sessionId;
private readonly ISshShellSession session;
private readonly ITerminalTransport transport;
private readonly TimeProvider clock;
private readonly TerminalPumpOptions options;
private readonly Channel<byte[]> pending = Channel.CreateUnbounded<byte[]>(
new UnboundedChannelOptions { SingleReader = true, SingleWriter = true });
private readonly OutputBacklog backlog;
private readonly CancellationTokenSource lifetime = new();
private int disposed;
@@ -104,6 +132,7 @@ public sealed class TerminalSessionPump : IAsyncDisposable
this.options = options ?? new TerminalPumpOptions();
Credits = new CreditWindow(this.options.WindowBytes);
backlog = new OutputBacklog(this.options.BacklogBytes);
}
/// <summary>This session's flow-control window.</summary>
@@ -112,6 +141,19 @@ public sealed class TerminalSessionPump : IAsyncDisposable
/// <summary>Total bytes read from the remote, for the throughput harness.</summary>
public long BytesRead { get; private set; }
/// <summary>
/// Total bytes this session discarded because the renderer could not keep up with the remote.
/// </summary>
/// <remarks>
/// Zero for every ordinary session, and the number to look at first when somebody reports missing
/// output. The user is told as it happens — a dropped run puts a line in the pane rather than a gap —
/// so this is for a test or a report rather than for the interface.
/// </remarks>
public long BytesDropped => backlog.DroppedTotal;
/// <summary>Bytes read from the remote and not yet sent to the renderer.</summary>
public int BytesPending => backlog.Buffered;
/// <summary>Total frames sent to the renderer, for the throughput harness.</summary>
public long FramesSent { get; private set; }
@@ -152,31 +194,7 @@ public sealed class TerminalSessionPump : IAsyncDisposable
var reader = ReadLoopAsync(linked.Token);
var flusher = FlushLoopAsync(linked.Token);
TerminalSessionOutcome outcome;
try
{
await reader.ConfigureAwait(false);
// The read loop finished of its own accord, which means the shell is over. Which of the two ways
// it is over is not a question asked of the transport now — by now the server has very likely hung
// up behind a clean exit, and asking would call `exit` a dropped connection — but of what the shell
// latched at the moment it happened. See ISshShellSession.TransportFailure.
outcome = session.TransportFailure is { Length: > 0 } failure
? new TerminalSessionOutcome(SshSessionEnd.ConnectionLost, failure)
: new TerminalSessionOutcome(SshSessionEnd.RemoteClosed, "The remote closed the session.");
}
catch (OperationCanceledException)
{
outcome = new TerminalSessionOutcome(SshSessionEnd.ClosedLocally, "The session was closed.");
}
catch (Exception exception)
{
// A read that threw rather than returning zero. Whatever it was, it was not this client asking —
// that arrives as the cancellation above — so it is the connection going out from under the read.
outcome = new TerminalSessionOutcome(
SshSessionEnd.ConnectionLost,
session.TransportFailure is { Length: > 0 } failure ? failure : exception.Message);
}
var outcome = await ReadUntilItEndsAsync(reader).ConfigureAwait(false);
// Before the drain and the last frame, so that anything reading this off a completed run — the
// workspace's own announcement — cannot see the default.
@@ -185,7 +203,17 @@ public sealed class TerminalSessionPump : IAsyncDisposable
// Stop the flusher, but only after draining what the reader already produced — the last thing
// a remote writes is often the most important, and dropping it makes a clean exit look like a
// crash.
pending.Writer.TryComplete();
backlog.Complete();
// ◆ AND RELEASE THE FLUSHER IF IT IS PARKED ON CREDIT, WHICH IS NOT THE SAME SIGNAL. Completing
// the backlog wakes anything waiting for output; it says nothing to a flush loop that has output
// and is waiting for the renderer to make room for it. Against a renderer that has stopped
// acknowledging — the page's process killed, the window gone — that acknowledgement never comes,
// and without this the await below would never return: the shell would have ended, the tab would
// never be told, and the session would never be torn down. Resetting rather than waiting is
// right because the window describes a renderer's remaining room and there is nothing left to
// protect it from: what follows is one backlog's worth of tail and then the session is over.
Credits.Reset();
try
{
@@ -203,6 +231,42 @@ public sealed class TerminalSessionPump : IAsyncDisposable
.ConfigureAwait(false);
}
/// <summary>
/// Waits for the read loop to finish, and says which of the three ways it did.
/// </summary>
/// <remarks>
/// Split out of <see cref="RunAsync"/> for length rather than for reuse, and it is the half of that
/// method with no side effects: three branches, one answer, nothing touched.
/// </remarks>
/// <param name="reader">The read loop's task.</param>
private async Task<TerminalSessionOutcome> ReadUntilItEndsAsync(Task reader)
{
try
{
await reader.ConfigureAwait(false);
// The read loop finished of its own accord, which means the shell is over. Which of the two ways
// it is over is not a question asked of the transport now — by now the server has very likely hung
// up behind a clean exit, and asking would call `exit` a dropped connection — but of what the shell
// latched at the moment it happened. See ISshShellSession.TransportFailure.
return session.TransportFailure is { Length: > 0 } failure
? new TerminalSessionOutcome(SshSessionEnd.ConnectionLost, failure)
: new TerminalSessionOutcome(SshSessionEnd.RemoteClosed, "The remote closed the session.");
}
catch (OperationCanceledException)
{
return new TerminalSessionOutcome(SshSessionEnd.ClosedLocally, "The session was closed.");
}
catch (Exception exception)
{
// A read that threw rather than returning zero. Whatever it was, it was not this client asking —
// that arrives as the cancellation above — so it is the connection going out from under the read.
return new TerminalSessionOutcome(
SshSessionEnd.ConnectionLost,
session.TransportFailure is { Length: > 0 } failure ? failure : exception.Message);
}
}
/// <summary>Forwards keystrokes to the remote.</summary>
public ValueTask WriteInputAsync(ReadOnlyMemory<byte> data, CancellationToken cancellationToken) =>
session.WriteAsync(data, cancellationToken);
@@ -227,44 +291,37 @@ public sealed class TerminalSessionPump : IAsyncDisposable
// Unblocks anything waiting on credit that will now never be acknowledged.
Credits.Reset();
pending.Writer.TryComplete();
backlog.Complete();
lifetime.Dispose();
await session.DisposeAsync().ConfigureAwait(false);
}
/// <remarks>
/// No credit is consulted here, and the class remark says at length why: the remote is not waiting on
/// this loop, so pausing it buys nothing and costs a buffer inside SSH.NET that grows without bound.
/// What this loop owes is to keep that buffer empty, which means reading as fast as the channel
/// delivers and putting the result somewhere with a ceiling.
/// </remarks>
private async Task ReadLoopAsync(CancellationToken cancellationToken)
{
var buffer = ArrayPool<byte>.Shared.Rent(options.ReadBufferBytes);
try
{
while (!cancellationToken.IsCancellationRequested)
while (true)
{
await Credits.WaitForCreditAsync(cancellationToken).ConfigureAwait(false);
// Thrown rather than tested in the loop condition, and the difference is the whole of how
// this session's ending is described. RunAsync reads a read loop that RETURNS as
// the remote having closed the channel, and one that throws OperationCanceledException as
// this client having ended it. A condition on the token would leave by the first door for
// a reason that belongs to the second, and a tab the user closed would report itself as a
// shell that exited on its own.
cancellationToken.ThrowIfCancellationRequested();
var granted = Credits.TryReserve(options.ReadBufferBytes);
if (granted == 0)
{
continue;
}
int read;
try
{
read = await session
.ReadAsync(buffer.AsMemory(0, granted), cancellationToken)
.ConfigureAwait(false);
}
catch
{
Credits.Return(granted);
throw;
}
// Hand back what was reserved but not used, so a short read does not permanently
// shrink the window.
Credits.Return(granted - read);
var read = await session
.ReadAsync(buffer.AsMemory(0, options.ReadBufferBytes), cancellationToken)
.ConfigureAwait(false);
if (read == 0)
{
@@ -272,7 +329,10 @@ public sealed class TerminalSessionPump : IAsyncDisposable
}
BytesRead += read;
await pending.Writer.WriteAsync(buffer[..read], cancellationToken).ConfigureAwait(false);
// Copied out of the rented buffer, because the backlog holds it until the renderer has
// room and the next read is already about to overwrite this one.
backlog.Append(buffer[..read]);
}
}
finally
@@ -283,65 +343,160 @@ public sealed class TerminalSessionPump : IAsyncDisposable
private async Task FlushLoopAsync(CancellationToken cancellationToken)
{
var segments = new List<byte[]>();
while (await pending.Reader.WaitToReadAsync(cancellationToken).ConfigureAwait(false))
while (await backlog.WaitForOutputAsync(cancellationToken).ConfigureAwait(false))
{
if (!pending.Reader.TryRead(out var first))
if (backlog.IsCompleted)
{
// The reader has finished and what is left is the tail, which goes out regardless of
// credit. Leaving by this door rather than going round again is load-bearing: with the
// backlog complete, SendWhatCreditAllowsAsync declines to wait for an acknowledgement
// that may never come, so a renderer that has stopped would put this loop into a spin —
// output to send, no credit to send it with, and a wait for more output that returns at
// once because there is some.
break;
}
// Coalesce for one frame's worth of time before the first send, so a burst arriving at an idle
// terminal becomes a single write. Under sustained output the loop below never comes back
// here, and does not need to: credit is what paces it then, and a frame is as large as the
// renderer has room for.
await Task.Delay(options.FlushInterval, clock, cancellationToken).ConfigureAwait(false);
await SendWhatCreditAllowsAsync(cancellationToken).ConfigureAwait(false);
}
await DrainAsync().ConfigureAwait(false);
}
/// <summary>Sends until the backlog is empty or the renderer has no room left.</summary>
private async Task SendWhatCreditAllowsAsync(CancellationToken cancellationToken)
{
while (true)
{
var wanted = Math.Min(backlog.Buffered, MaximumFrameBytes);
if (wanted == 0)
{
return;
}
var granted = Credits.TryReserve(wanted);
if (granted == 0)
{
// The renderer is behind. Nothing more goes out until it acknowledges, and meanwhile the
// backlog holds what it can and drops its oldest for what it cannot — which is the whole
// of how this session's memory stays bounded.
if (backlog.IsCompleted)
{
// Except at the end, where waiting on an acknowledgement that may never come would
// hang the session's teardown behind a renderer that has already gone.
return;
}
await Credits.WaitForCreditAsync(cancellationToken).ConfigureAwait(false);
continue;
}
segments.Clear();
segments.Add(first);
var sent = await SendFromBacklogAsync(granted, cancellationToken).ConfigureAwait(false);
// Coalesce for one frame's worth of time, then send everything at once. Whatever the
// remote produced in that window becomes a single write to the terminal.
await Task.Delay(options.FlushInterval, clock, cancellationToken).ConfigureAwait(false);
while (pending.Reader.TryRead(out var more))
if (granted > sent)
{
segments.Add(more);
// Whatever was reserved and not used, so a backlog that shrank between the reservation
// and the take does not permanently narrow the window. Guarded rather than called with a
// zero, because returning credit wakes every waiter and replaces the signal they were
// waiting on, and this is the hot path.
Credits.Return(granted - sent);
}
await SendOutputAsync(segments, cancellationToken).ConfigureAwait(false);
}
// The channel completed. Anything the reader wrote before finishing still has to go out.
segments.Clear();
while (pending.Reader.TryRead(out var trailing))
{
segments.Add(trailing);
}
if (segments.Count > 0)
{
await SendOutputAsync(segments, CancellationToken.None).ConfigureAwait(false);
}
}
private async ValueTask SendOutputAsync(List<byte[]> segments, CancellationToken cancellationToken)
/// <summary>
/// Puts the last of the output out, regardless of credit.
/// </summary>
/// <remarks>
/// The session is over by the time this runs, so there is no renderer left to protect from a flood —
/// and what remains is bounded by the backlog's own capacity rather than by anything the remote can
/// still do. Waiting on credit here instead would mean a shell that exited against a renderer which
/// had gone never finished tearing down.
/// </remarks>
private async Task DrainAsync()
{
var total = 0;
foreach (var segment in segments)
while (await SendFromBacklogAsync(MaximumFrameBytes, CancellationToken.None)
.ConfigureAwait(false) > 0)
{
total += segment.Length;
// Until it is empty.
}
}
/// <summary>
/// Moves up to <paramref name="maxBytes"/> out of the backlog and onto the wire.
/// </summary>
/// <returns>How many bytes of output were sent, which is zero once the backlog is empty.</returns>
private async ValueTask<int> SendFromBacklogAsync(int maxBytes, CancellationToken cancellationToken)
{
// Taken straight into the frame, header space and all. A second buffer here would be an
// allocation and a copy of every byte the terminal ever shows.
var frame = new byte[TerminalFrame.HeaderLength + maxBytes];
var taken = backlog.TakeInto(frame.AsSpan(TerminalFrame.HeaderLength), out var dropped);
if (dropped > 0)
{
// Sent immediately before the bytes it explains, and under its own opcode — see
// TerminalServerOpcode.OutputDropped for why this is not simply spliced into the output.
await SendAsync(TerminalServerOpcode.OutputDropped, DroppedNotice(dropped), cancellationToken)
.ConfigureAwait(false);
}
if (taken == 0)
{
return 0;
}
var frame = new byte[TerminalFrame.HeaderLength + total];
TerminalFrame.Write(frame, (byte)TerminalServerOpcode.Output, sessionId, default);
var offset = TerminalFrame.HeaderLength;
foreach (var segment in segments)
{
segment.CopyTo(frame, offset);
offset += segment.Length;
}
FramesSent++;
await transport.SendAsync(frame, cancellationToken).ConfigureAwait(false);
await transport
.SendAsync(frame.AsMemory(0, TerminalFrame.HeaderLength + taken), cancellationToken)
.ConfigureAwait(false);
return taken;
}
/// <summary>
/// The line a user sees where output was dropped.
/// </summary>
/// <remarks>
/// <para>
/// It opens with a reset rather than the colour, and that is the load-bearing part: the bytes were cut
/// at an arbitrary offset, so whatever escape sequence the remote was in the middle of has been left
/// half-said. Resetting first means the notice reads as a notice and the output after it starts from a
/// known state, instead of inheriting a colour or a mode from a sequence nobody finished.
/// </para>
/// <para>
/// Dim grey, matching the renderer's own banners for a session that closed and a view that
/// reconnected — this is the same kind of thing, a sentence from the application rather than from the
/// remote, and it should not read like part of the transcript.
/// </para>
/// </remarks>
private static byte[] DroppedNotice(long dropped) =>
Encoding.UTF8.GetBytes(string.Create(
CultureInfo.InvariantCulture,
$"\u001b[0m\r\n\u001b[38;5;244m── {Sized(dropped)} of output was dropped; "
+ $"the terminal could not keep up ──\u001b[0m\r\n"));
/// <remarks>
/// One decimal place from a megabyte up, because the number is an order of magnitude rather than an
/// accounting: what a person needs from it is whether they lost a screenful or a session's worth.
/// </remarks>
private static string Sized(long bytes) => bytes switch
{
< 1024 => string.Create(CultureInfo.CurrentCulture, $"{bytes} bytes"),
< 1024 * 1024 => string.Create(CultureInfo.CurrentCulture, $"{bytes / 1024.0:0.#} KB"),
< 1024L * 1024 * 1024 => string.Create(CultureInfo.CurrentCulture, $"{bytes / (1024.0 * 1024):0.#} MB"),
_ => string.Create(CultureInfo.CurrentCulture, $"{bytes / (1024.0 * 1024 * 1024):0.#} GB"),
};
private async ValueTask SendAsync(
TerminalServerOpcode opcode,
ReadOnlyMemory<byte> payload,