DodoTech-Public/DodoSSH
Public Access
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Say when a vault has moved, so nobody waits out the minute

Browse Source 
The delta pull was cheap enough to run on a timer and the client did, once a
minute. That is fine for a machine and wrong for two people: an edit a colleague
makes is up to a minute stale, which is long enough for both of them to make it
and produce a conflict neither needed to have. Shortening the interval is the
obvious answer and the wrong one — it costs a request per client per interval
whether or not anything happened, and it converges on a busier server that is
still late.

So the server now says so. A client holds a WebSocket open at GET /api/v1/events,
subprotocol dodossh.events.v1, and gets a line down it when something it can read
has changed. ADR 0012 has the reasoning; three parts of it are worth repeating
here, because they are what everything else rests on.

**What crosses the socket is a notice, never data.** A frame names a vault and
how far its change log has got. No item, no ciphertext, not even which item it
was. The client's answer is the delta pull it would have run anyway, so there is
still exactly one code path that applies a change to a keychain, and it is not
this one. Pushing the items themselves would save a round trip and fork that path
in two, with the cursor, the merge and the tombstone rules duplicated across both
— ADR 0003 put every mutation through one write path for that reason, and this
keeps every read on one for the same one. It also makes a dropped notice
harmless, which is what lets the fan-out below be as simple as it is.

**Polling stays, and is what guarantees a pass.** The minute timer is unchanged.
A network that eats WebSockets, a server with Events:Enabled off, an older
server, a proxy that will not upgrade, a notice dropped under backpressure —
every one of those leaves a client behaving exactly as it did before this commit.
Nothing is reachable only over the socket and nothing is meant to become so;
VaultViewModel's AutoSyncInterval remark now says that where somebody changing it
will read it.

**The bearer token authorises the upgrade, unlike the relay's ticket.** Not an
inconsistency with ADR 0004: the relay's socket is a byte pipe whose whole
authorization decision — which host, which IPs, which port — is made before it
opens and never revisited, and it is the extraction seam for a process that must
hold no ACL code. This one is a view of the caller's own vault list and has to
keep answering "what may this account read" for as long as it is held. A ticket
would carry that answer in a token and be wrong the moment the account's access
changed. The two bounds that arrangement needs are met rather than waved at: the
socket is closed at the token's exp with close code 4401 and the client comes
straight back with a fresh one, and the vault set is re-resolved every few
minutes as well as on the changes known to affect it. Both bound *metadata*,
because a notice contains nothing else and reading a vault still needs a key this
server has never held.

**The fan-out.** VaultEventHub is a singleton holding the sockets this node
accepted; publishing walks them and asks each whether it cares, rather than
keeping a vault-to-subscriber index that every re-subscription would have to move
entries between under a lock publishing also takes. At a few hundred sockets per
node and an event rate bounded by how often people edit keychains, the walk is
not measurable and its races are obvious. Per-connection queues are bounded and
drop the *oldest*: a notice means "pull vault X, which is at least at sequence
N", so the newest subsumes what it displaces and the client's answer is identical
either way — which is what lets the publish path be void, never block, and never
fail.

Announced from the endpoint rather than from SyncService, and that placement is
the point: by then the push has committed and released the per-vault advisory
lock. From inside it would name a sequence no reader can see yet and would hold
the lock that serialises writers across a socket write. Only the highest
*applied* sequence, so a batch of pure conflicts announces nothing, and a
duplicate — already announced when it first landed — announces nothing either.

Grants and membership publish too, and those take the *recipient* rather than the
actor. This is what AdmitNewVaultsAsync has been apologising for since sharing
shipped — "the recipient is handed nothing, there is no push channel" — and the
README with it. A vault shared with somebody now turns up as it is shared. The
comment and the README paragraph both say what is true now, and both keep saying
that the pass is what *discovers* the vault, because a client with no socket has
to arrive at the same place.

**On the client**, VaultEventStream is really a reconnection policy wrapped round
a ClientWebSocket: a dropped socket is the ordinary case here — laptops sleep,
proxies time out, tokens expire, servers are redeployed — so nothing in it treats
a failure as exceptional, and every path ends in "wait, then dial again". A
connection that lived long enough to say hello resets the backoff, so a laptop
that woke, worked, and lost its network an hour later does not inherit a
minute-long wait it has already proved it need not take. A 4401 close skips the
backoff entirely and asks the token provider again, which is the whole reason
that close code is distinct. A server that does not advertise the events feature
gets IdleVaultEventStream, which never delivers — so IVaultServer.Events is never
null and every caller stays on one shape, because the correct behaviour without a
socket is the behaviour with a silent one.

The shell's background loop now selects between the timer and a notice, and both
waits are held across iterations. That is load-bearing rather than tidy:
PeriodicTimer permits one outstanding WaitForNextTickAsync and throws on a
second, and an abandoned channel read stays registered and consumes the next
notice written. Either defect leaves the first notice working and every one after
it silently lost, which is why NoticesKeepWakingTheLoop_NotJustTheFirst pushes
three and not one. Notices are coalesced over a quarter of a second, so one
person's save — a host and its log entry are two items — and a colleague clearing
a folder each cost one pass rather than a dozen.

**The kind is a string, not an enum**, and that is a compatibility decision.
UseStringEnumConverter throws on a value it does not know, so a newer server
sending a kind an older client had never heard of would not add an unreadable
frame — it would break that client's socket outright. A string is ignored
instead. ProblemCodes is the same shape for the same reason.

**Tested on both sides, through the real pipeline.** The endpoint suite opens a
genuine socket against TestServer and proves a push produces a notice, that
another account's push does not reach it, that a ping is answered, and that a
frame this server cannot parse does not end the connection. Two of those assert
on *ordering* rather than on absence within a timeout — the stranger's write goes
first, so a socket that leaked would have announced it before the one the test
waits for — because "nothing arrived in two seconds" is a test that passes on a
slow machine for the wrong reason. And ANoticeCarriesNoCiphertext asserts on the
bytes that crossed the wire rather than on the record's fields, since the latter
would only prove that this type has no payload member, which is a tautology; the
former is what catches a field added later without anybody thinking about
disclosure.

The client suite drives VaultEventStream through an injected connector, because
the one thing a test cannot do to a real network is make it fail on cue — and
failure is the entire subject. The shell suite proves a notice produces a pull
inside ten seconds against a sixty-second timer, so the timer cannot be what
caused it.

**Two limits, stated rather than left to be discovered.** Fan-out is in-process,
so a deployment running more than one API replica only pushes for writes its own
replica handled and the rest arrive on the timer. IVaultEventPublisher is the
seam a PostgreSQL LISTEN/NOTIFY backplane implements and it is deliberately not
implemented: an untested backplane is worse than a documented gap, and multiple
replicas degrade to the behaviour before this commit rather than breaking. And a
client is notified of its own writes; it pushed, so it already pulled, and the
extra pass finds nothing. Suppressing that echo correctly needs a per-device
identity on the socket, and the same user's other machines must still be told.

Manual checks phase 15 covers what no test here can reach, which is the network
in between: a proxy that will not upgrade, one that drops an idle socket without
telling either end, a laptop lid, a token expiring. Every one of those is
invisible inside a test host, and every check there passes only if the change
arrives quickly *and* still arrives with the socket taken away.

ADR 0012 also fixes one thing about the shared terminal session this is the
transport for, so it need not be renegotiated later: session data will be binary
frames on this same socket, because base64 in a JSON envelope is the wrong shape
for the one payload here that is continuous rather than occasional. Two questions
it explicitly does not answer by implication — whether those bytes go through the
API at all, and what end-to-end encryption means when the second party watches a
stream rather than holding a key — are ADR 0001 questions and get their own
decision.

1512 tests pass. DodoSSH.SystemTests was not run — it needs the whole compose
stack — so the end-to-end path is unverified for this change beyond what the
manual checks describe.
This commit is contained in:
jaap-jan
2026-08-04 16:37:41 +02:00
parent 176df67861
commit 4b706bc3c3
31 changed files with 3285 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/DodoSSH.Client.Api/VaultEventStream.cs
+552
View File
@@ -0,0 +1,552 @@
using System.Net.WebSockets;
using System.Security.Cryptography;
using System.Text.Json;
using System.Threading.Channels;
using DodoSSH.Contracts;
namespace DodoSSH.Client.Api;
/// <summary>
/// A server's "pull now" notices, as everything above the transport needs them.
/// </summary>
/// <remarks>
/// <para>
/// A queue to read from rather than an event to subscribe to, and that shape is the point: the one
/// consumer is a synchronisation loop that already waits on a timer, so it can wait on this the same
/// way and keep every continuation on the thread it started from. An event would deliver on whichever
/// thread the socket happened to complete on, which in a user interface is the difference between
/// working and an intermittent rendering fault nobody can reproduce.
/// </para>
/// <para>
/// <b>Reading this is never how a change is applied.</b> A notice says which vault moved and nothing
/// else; the answer to it is the ordinary delta pull. See ADR 0012.
/// </para>
/// </remarks>
public interface IVaultEventStream : IDisposable
{
/// <summary>Whether a socket is currently established.</summary>
/// <remarks>
/// For the interface to say whether it is live, not for a caller to branch on before reading:
/// synchronising is correct whether or not this is true, because the timer is the fallback.
/// </remarks>
bool IsConnected { get; }
/// <summary>
/// Waits for the next notice.
/// </summary>
/// <remarks>
/// Connects on the first call and reconnects for as long as it is read, so a caller neither starts
/// nor restarts anything. A server that cannot be reached is not an error here — it is a wait that
/// has not finished — because the caller's alternative is the timer it is already running.
/// </remarks>
ValueTask<VaultEvent> ReadAsync(CancellationToken cancellationToken);
/// <summary>
/// Takes a notice if one is already waiting, without blocking.
/// </summary>
/// <returns>Whether there was one.</returns>
/// <remarks>
/// How a caller coalesces a burst. Five people saving at once produces five notices whose answer
/// is a single synchronisation pass, so the loop reads one, waits a moment, and swallows the rest
/// rather than running the same pull five times.
/// </remarks>
bool TryRead(out VaultEvent notice);
}
/// <summary>
/// A stream that never delivers anything.
/// </summary>
/// <remarks>
/// For a server that does not advertise the <c>events</c> feature, and for tests. Deliberately waits
/// for ever rather than completing: a caller selecting between this and a timer must fall through to
/// the timer, and a read that returned immediately would spin that loop as fast as the machine allows.
/// </remarks>
public sealed class IdleVaultEventStream : IVaultEventStream
{
/// <summary>The one instance. It holds nothing.</summary>
public static IdleVaultEventStream Instance { get; } = new();
/// <inheritdoc />
public bool IsConnected => false;
/// <inheritdoc />
public async ValueTask<VaultEvent> ReadAsync(CancellationToken cancellationToken)
{
await Task.Delay(System.Threading.Timeout.Infinite, cancellationToken).ConfigureAwait(false);
// Unreachable: the delay above only ever ends by throwing.
return new VaultEvent(VaultEventKinds.Ping);
}
/// <inheritdoc />
public bool TryRead(out VaultEvent notice)
{
notice = new VaultEvent(VaultEventKinds.Ping);
return false;
}
/// <inheritdoc />
public void Dispose()
{
// Nothing is held.
}
}
/// <summary>Tuning for <see cref="VaultEventStream"/>.</summary>
/// <remarks>
/// Every value here bounds a reconnection rather than a feature. With the socket permanently
/// unavailable the client synchronises on its timer, so the cost of getting these wrong is latency,
/// never correctness.
/// </remarks>
public sealed record VaultEventStreamOptions
{
/// <summary>The defaults.</summary>
public static VaultEventStreamOptions Default { get; } = new();
/// <summary>How long to wait before the first reconnection attempt.</summary>
public TimeSpan InitialBackoff { get; init; } = TimeSpan.FromSeconds(1);
/// <summary>The longest the backoff may grow to.</summary>
/// <remarks>
/// A minute, which is the polling interval: past that point reconnecting sooner buys nothing,
/// because the timer has already done the work the socket would have prompted.
/// </remarks>
public TimeSpan MaxBackoff { get; init; } = TimeSpan.FromMinutes(1);
/// <summary>
/// How long a socket may be silent before it is presumed dead.
/// </summary>
/// <remarks>
/// The server pings on an interval it states in its <c>hello</c>, so silence past a multiple of
/// that means the connection is gone rather than idle — which is otherwise indistinguishable, and
/// is exactly what a reverse proxy that quietly drops idle sockets produces. Used only until a
/// <c>hello</c> arrives; after that the server's own figure is trusted.
/// </remarks>
public TimeSpan InitialSilenceTimeout { get; init; } = TimeSpan.FromSeconds(90);
/// <summary>How many notices may be waiting before the oldest are dropped.</summary>
/// <remarks>
/// Small on purpose. A notice means "pull that vault", so a newer one subsumes the one it
/// displaces; a backlog would only make the loop pull repeatedly for work it has already done.
/// </remarks>
public int QueueDepth { get; init; } = 32;
}
/// <summary>
/// Holds a socket to one server open, and hands over what it says.
/// </summary>
/// <remarks>
/// <para>
/// The whole class is a reconnection policy. A dropped socket is the ordinary case — laptops sleep,
/// proxies time out, tokens expire, servers are redeployed — so nothing here treats a failure as
/// exceptional: it backs off and dials again, for as long as somebody is reading.
/// </para>
/// <para>
/// It is safe to have no server at all. Every failure path ends in "wait, then try again", and the
/// caller's synchronisation timer runs regardless, which is what makes it correct for this class to
/// stay silent about problems rather than surface them.
/// </para>
/// </remarks>
public sealed class VaultEventStream : IVaultEventStream, IAsyncDisposable
{
private readonly Uri endpoint;
private readonly IAccessTokenProvider tokens;
private readonly TimeProvider clock;
private readonly VaultEventStreamOptions options;
private readonly Func<Uri, string, CancellationToken, Task<WebSocket>> connect;
private readonly Channel<VaultEvent> notices;
private readonly CancellationTokenSource closing = new();
private readonly Lock starting = new();
private Task? pump;
private bool disposed;
/// <summary>Creates a stream against one server.</summary>
/// <param name="serverUrl">The server's base URL, as an ordinary <c>http</c> or <c>https</c> address.</param>
/// <param name="tokens">Supplies a bearer token, refreshing it when it is due.</param>
/// <param name="clock">Time source, for the backoff and the silence timeout.</param>
/// <param name="options">Tuning, or null for the defaults.</param>
public VaultEventStream(
Uri serverUrl,
IAccessTokenProvider tokens,
TimeProvider clock,
VaultEventStreamOptions? options = null)
: this(serverUrl, tokens, clock, DialAsync, options)
{
}
/// <remarks>
/// The connector is injected so the suite can drive this against a test host's in-memory socket.
/// Reconnection is the entire behaviour of this class, and testing it against a real network would
/// mean testing it against the one thing that cannot be made to fail on demand.
/// </remarks>
internal VaultEventStream(
Uri serverUrl,
IAccessTokenProvider tokens,
TimeProvider clock,
Func<Uri, string, CancellationToken, Task<WebSocket>> connect,
VaultEventStreamOptions? options = null)
{
ArgumentNullException.ThrowIfNull(serverUrl);
ArgumentNullException.ThrowIfNull(tokens);
ArgumentNullException.ThrowIfNull(clock);
ArgumentNullException.ThrowIfNull(connect);
endpoint = EventsUrl(serverUrl);
this.tokens = tokens;
this.clock = clock;
this.connect = connect;
this.options = options ?? VaultEventStreamOptions.Default;
notices = Channel.CreateBounded<VaultEvent>(new BoundedChannelOptions(this.options.QueueDepth)
{
FullMode = BoundedChannelFullMode.DropOldest,
SingleReader = true,
SingleWriter = true,
});
}
/// <inheritdoc />
public bool IsConnected { get; private set; }
/// <inheritdoc />
public ValueTask<VaultEvent> ReadAsync(CancellationToken cancellationToken)
{
ObjectDisposedException.ThrowIf(disposed, this);
Start();
return notices.Reader.ReadAsync(cancellationToken);
}
/// <inheritdoc />
/// <remarks>
/// Does not start the connection, unlike <see cref="ReadAsync"/>: a caller draining a burst has
/// already read one notice, and "is there another right now" is not a reason to dial a server.
/// </remarks>
public bool TryRead(out VaultEvent notice) => notices.Reader.TryRead(out notice!);
/// <summary>
/// Ends the connection, without waiting for it to unwind.
/// </summary>
/// <remarks>
/// What a shell calls when a connection is dropped, from a synchronous path that must not block —
/// <c>IVaultServer</c> is <see cref="IDisposable"/>, and blocking on a socket teardown from the
/// user-interface thread is exactly the sync-over-async this repository bans. Cancelling is enough:
/// every loop reads the token, and the pump has nothing to flush.
/// </remarks>
public void Dispose()
{
if (disposed)
{
return;
}
disposed = true;
closing.Cancel();
// The source is deliberately left undisposed. The pump may still be inside a linked token
// source derived from this one, and disposing a parent out from under a live child is how a
// clean shutdown becomes an ObjectDisposedException on a background thread. It holds no timer
// and no handle once cancelled; DisposeAsync is the path that cleans it up properly.
}
/// <summary>Ends the connection and waits for it to unwind.</summary>
/// <remarks>The deterministic form, for a caller that can await one — tests, mostly.</remarks>
public async ValueTask DisposeAsync()
{
if (disposed)
{
return;
}
disposed = true;
await closing.CancelAsync().ConfigureAwait(false);
if (pump is not null)
{
try
{
await pump.ConfigureAwait(false);
}
catch (OperationCanceledException)
{
// The point of the cancel above.
}
}
closing.Dispose();
}
/// <summary>
/// Turns a server's base URL into its event socket's.
/// </summary>
/// <remarks>
/// The path is replaced rather than appended, matching every other call in this client: request
/// paths here are absolute — <c>/api/v1/…</c> — so a deployment behind a path prefix is already
/// unsupported, and pretending otherwise in this one place would be a difference nobody could act
/// on.
/// </remarks>
private static Uri EventsUrl(Uri serverUrl) =>
new UriBuilder(serverUrl)
{
Scheme = string.Equals(serverUrl.Scheme, Uri.UriSchemeHttps, StringComparison.OrdinalIgnoreCase)
? "wss"
: "ws",
Path = VaultEvents.Path,
Query = string.Empty,
Fragment = string.Empty,
}.Uri;
private static async Task<WebSocket> DialAsync(Uri url, string token, CancellationToken cancellationToken)
{
var socket = new ClientWebSocket();
try
{
socket.Options.AddSubProtocol(VaultEvents.SubProtocol);
// A header rather than the Sec-WebSocket-Protocol smuggling ADR 0004 needs for the relay:
// this client is a native application and can set one, and the token here is the ordinary
// bearer credential rather than a ticket.
socket.Options.SetRequestHeader("Authorization", $"Bearer {token}");
await socket.ConnectAsync(url, cancellationToken).ConfigureAwait(false);
return socket;
}
catch
{
socket.Dispose();
throw;
}
}
private void Start()
{
if (pump is not null)
{
return;
}
lock (starting)
{
pump ??= Task.Run(() => RunAsync(closing.Token), closing.Token);
}
}
/// <summary>Connects, reads until it cannot, waits, and does it again.</summary>
private async Task RunAsync(CancellationToken cancellationToken)
{
var backoff = options.InitialBackoff;
while (!cancellationToken.IsCancellationRequested)
{
var outcome = await AttemptAsync(cancellationToken).ConfigureAwait(false);
// A socket that lived long enough to say hello proves the server is there and willing, so
// the next failure starts from the bottom again rather than inheriting the backoff that
// got us here. Without this a laptop that woke, connected, and then lost its network an
// hour later would wait a full minute before trying, having already proved it need not.
if (outcome == Outcome.Established)
{
backoff = options.InitialBackoff;
}
// The server said this token is spent, which the token provider can fix without waiting.
// Reconnecting at once is the whole reason that close code is distinct.
var wait = outcome == Outcome.TokenExpired ? TimeSpan.Zero : Jitter(backoff);
if (wait > TimeSpan.Zero)
{
try
{
await Task.Delay(wait, clock, cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException)
{
return;
}
backoff = backoff < options.MaxBackoff
? Shorter(backoff * 2, options.MaxBackoff)
: options.MaxBackoff;
}
}
}
/// <summary>One connection, from dial to close.</summary>
private async Task<Outcome> AttemptAsync(CancellationToken cancellationToken)
{
WebSocket? socket = null;
try
{
var token = await tokens.GetAccessTokenAsync(cancellationToken).ConfigureAwait(false);
socket = await connect(endpoint, token, cancellationToken).ConfigureAwait(false);
IsConnected = true;
return await PumpAsync(socket, cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException)
{
return Outcome.Cancelled;
}
catch (Exception exception) when (exception is not OutOfMemoryException)
{
// Every failure this can meet — no network, a refused upgrade, a server that has not been
// deployed with this feature, a token that cannot be refreshed — has the same remedy, and
// none of them is worth telling a user about. The synchronisation timer is still running.
return Outcome.Failed;
}
finally
{
IsConnected = false;
socket?.Dispose();
}
}
/// <summary>Reads frames until the socket ends or goes quiet.</summary>
private async Task<Outcome> PumpAsync(WebSocket socket, CancellationToken cancellationToken)
{
var buffer = new byte[8 * 1024];
var silence = options.InitialSilenceTimeout;
var established = false;
while (socket.State == WebSocketState.Open && !cancellationToken.IsCancellationRequested)
{
// Rebuilt per frame rather than reset, because a linked source cannot be un-cancelled and
// the deadline is what detects a socket that has silently gone away.
using var deadline = new CancellationTokenSource(silence, clock);
using var quiet = CancellationTokenSource.CreateLinkedTokenSource(
cancellationToken, deadline.Token);
WebSocketReceiveResult received;
try
{
received = await socket.ReceiveAsync(buffer, quiet.Token).ConfigureAwait(false);
}
catch (OperationCanceledException) when (!cancellationToken.IsCancellationRequested)
{
// Silent for longer than the server said it would be. The socket is gone in a way that
// only reconnecting can discover, which is what a proxy dropping an idle connection
// looks like from this end.
return Ended(established);
}
if (received.MessageType == WebSocketMessageType.Close)
{
return (int?)received.CloseStatus == VaultEvents.TokenExpiredCloseCode
? Outcome.TokenExpired
: Ended(established);
}
// Binary is reserved by ADR 0012 for shared-session data, and text that arrived in pieces
// is longer than anything this protocol defines. Skipped rather than fatal, so a newer
// server does not cost this client its push for the whole session.
if (received.MessageType != WebSocketMessageType.Text || !received.EndOfMessage)
{
continue;
}
if (Parse(buffer.AsSpan(0, received.Count)) is not { } frame)
{
continue;
}
established = true;
silence = await AbsorbAsync(socket, frame, silence, cancellationToken).ConfigureAwait(false);
}
return Ended(established);
}
/// <summary>Deals with one frame, and says how long the socket may now stay quiet.</summary>
private async Task<TimeSpan> AbsorbAsync(
WebSocket socket,
VaultEvent frame,
TimeSpan silence,
CancellationToken cancellationToken)
{
if (frame.HeartbeatSeconds is > 0 and var seconds)
{
// Three missed heartbeats. Two is within one paused thread of a false positive, and a
// false positive here costs a reconnection rather than anything a user sees.
silence = TimeSpan.FromSeconds(seconds * 3);
}
if (string.Equals(frame.Kind, VaultEventKinds.Ping, StringComparison.Ordinal))
{
await socket.SendAsync(
JsonSerializer.SerializeToUtf8Bytes(
new VaultEvent(VaultEventKinds.Pong), DodoSshJsonContext.Default.VaultEvent),
WebSocketMessageType.Text,
endOfMessage: true,
cancellationToken)
.ConfigureAwait(false);
return silence;
}
// Everything else, including a kind this build has never heard of, goes to the reader — which
// is what makes the frame table extensible. An unrecognised kind is one the caller ignores;
// refusing it here would be this class deciding what a newer server may say.
notices.Writer.TryWrite(frame);
return silence;
}
private static Outcome Ended(bool established) =>
established ? Outcome.Established : Outcome.Failed;
private static VaultEvent? Parse(ReadOnlySpan<byte> utf8)
{
try
{
return JsonSerializer.Deserialize(utf8, DodoSshJsonContext.Default.VaultEvent);
}
catch (JsonException)
{
return null;
}
}
/// <summary>
/// Spreads reconnections out, so a server that restarts is not met by every client at once.
/// </summary>
/// <remarks>
/// <c>RandomNumberGenerator</c> because <c>System.Random</c> is banned repo-wide. Nothing here is
/// security-relevant — the ban exists so that nothing key-, token- or nonce-adjacent can reach for
/// the weak one by habit, and paying a few microseconds to keep that rule absolute is the cheaper
/// side of the trade.
/// </remarks>
private static TimeSpan Jitter(TimeSpan delay)
{
var milliseconds = (int)Math.Clamp(delay.TotalMilliseconds, 1, int.MaxValue / 2);
return TimeSpan.FromMilliseconds(
milliseconds + RandomNumberGenerator.GetInt32(0, Math.Max(1, milliseconds / 2)));
}
private static TimeSpan Shorter(TimeSpan left, TimeSpan right) => left < right ? left : right;
/// <summary>How one connection attempt ended.</summary>
private enum Outcome
{
/// <summary>Never got as far as a frame. Back off.</summary>
Failed,
/// <summary>Ran, and then ended. Back off, but from the bottom.</summary>
Established,
/// <summary>The server closed it because the token expired. Reconnect at once with a new one.</summary>
TokenExpired,
/// <summary>The stream is being disposed.</summary>
Cancelled,
}
}