Let a host marked for the relay actually be reached through it #19

Merged
jaap-jan merged 1 commits from claude/server-relay-94b02f into main 2026-09-28 12:41:30 +00:00
51 changed files with 6455 additions and 88 deletions

No files matched your search

+7
View File
@@ -873,6 +873,13 @@ keychain plus a terminal — and the spike that gates all of it.
[Moving files](#moving-files) for the two things about it worth knowing before you use it, both of which
are consequences rather than choices.
*Relay done:* a host with "Connect through the server relay" ticked is reached through the deployment,
for a machine this one cannot dial. The server issues a single-use, thirty-second ticket for that host
alone, resolves its address and checks it against a deny list with no configuration escape — loopback,
link-local, metadata endpoints — and then pipes raw TCP; the SSH session still ends on your machine, so
the relay sees ciphertext. It is off by default (`Relay:Enabled`), and it only helps when the server can
reach what you cannot. See [ADR 0004](docs/adr/0004-relay-authorization.md).
*Organising done:* hosts can be filed into groups, and commands can be saved as snippets. Both are ordinary
synced items — encrypted, merged and pushed like every other — and both are invisible until used: a
keychain with no groups draws the flat host list it always did. Two things about them are deliberate.
+19
View File
@@ -99,3 +99,22 @@ real ones.
- **Client-supplied target address.** One line of convenience, and the relay becomes an open
proxy into the operator's network.
- **Allow-listing by hostname.** Defeated by DNS rebinding.
## Implementation notes
Built in `DodoSSH.Api/Features/Relay` and, on the client, `RelayBridge` and `ServerRelay`; see
`docs/reaching-a-host-you-cannot-dial.md`, "What step 3 built". Three places differ from the text above:
- **Backpressure is an awaited copy loop, not `System.IO.Pipelines`.** Each direction reads one buffer and
writes it before reading again. The property is the same — a slow peer stops the read, and TCP throttles
the origin — and the bound is tighter: two 32 KiB buffers per session.
- **There is no `?ticket=` fallback.** Both clients can set a subprotocol, and the fallback's cost — a bearer
credential in the URL — was the reason it needed scrubbing at all. It can be added if a constrained
client ever needs it; the scrubbing requirement stands.
- **The ticket carries no device id.** The server does not see a device on an ordinary request, so there was
nothing to put there. `PortForwardId` on the request is refused: a forward's port is ciphertext, and
honouring one would mean taking the port from the client.
The loopback bridge serves the relay only. SSH.NET's `ForwardedPortDynamic` already is the jump-host path's
listener, so "one mechanism, two features" is true one level down: both features reach their target through
a loopback SOCKS5 proxy named on the connection request.
+1 -1
View File
@@ -254,7 +254,7 @@ the chrome, hosts and terminals, file transfer, the vault, teams, and preference
> | v5 element | What ships instead |
> | --- | --- |
> | The status dot's third, amber state | ◆ **Still two states, on the same reasoning v3 and v4 both gave and v5 was asked to reconsider.** Green means a terminal is open on that host, grey means one is not; nothing here pings a machine, so the amber "reachable but not connected" state has no fact behind it to draw. |
> | The relay checkbox's mock copy | The pre-existing sentence, unchanged, "(not wired up yet)" included — restyled into the mock's own nested-card shape (radius 12, a title beside the box rather than under it) but not reworded. The mock's own copy implies a relay this client can dial; it cannot, for the reason `HostDrawer.axaml`'s own remarks give. |
> | The relay checkbox's mock copy | Restyled into the mock's own nested-card shape (radius 12, a title beside the box rather than under it). The "(not wired up yet)" it carried while this client could not dial the relay came out when it could; the sentence under it now says what ticking it does — the server dials the host, so the server has to be able to reach it, and the address is stored in plain text. |
> | The host card as a link straight to a terminal | Click still selects, double-click still connects, and the pencil still opens the pane — the mock's card-as-link is not adopted, because multi-select (Ctrl, Shift, the marquee band) depends on a plain click meaning "choose this one" rather than "go". |
> | Quick connect's SSH/SFTP kind column | The auth word — credential, key, or password — see above. |
> | A collapsed section staying collapsed after a restart | In memory only, for the running session. `settings.json` holds two scalars by decision — the terminal's text size and whether this machine checks for updates on its own — and collapse state is not judged worth a third. |
+71 -10
View File
@@ -1,26 +1,30 @@
# Reaching a host you cannot dial
Some machines do not answer from where the user is sitting. This product has **two** answers to that, and
neither of them works.
one of them works.
- The **relay** pipes raw TCP through the deployment. The server half is built and shipped; the client
half does not exist, and both heads offer a checkbox that promises it.
- The **relay** pipes raw TCP through the deployment. Both halves are built: the server issues a ticket and
pipes a WebSocket to the host, and the client reaches it through a loopback SOCKS5 bridge. Step 3 below.
- A **jump host** reaches the target through a machine already in the keychain. `HostSecret.JumpHostIds`
stores the chain; nothing writes it and nothing reads it.
◆ **An earlier version of this page said the relay's server half was "built and shipped". It was not.** What
existed was ADR 0004, the contract records, the options and the CHECK constraint — and `RelayOptions` said,
in its own remarks, that there was no endpoint and no dialler. Both halves were built together at step 3.
This document is the comparison between them, which is the thing that has to be settled before either is
built, and then the plan. It replaces an earlier draft of `docs/jump-hosts.md` that recommended deleting the
jump chain — see the last section for why that was wrong.
> **Status: step 0 done, the rest planned.**
> **Status: steps 0, 1 and 3 done, and step 4 for the relay. Jump hosts are next.**
>
> | Step | State | Notes |
> | --- | --- | --- |
> | 0. Stop promising the relay | **Done** | Both heads say the box is not wired up and that ticking it stores the address and changes nothing. Left tickable, so a host already carrying the flag can lose it |
> | 0. Stop promising the relay | **Done, then retired at step 3** | The "not wired up yet" sentence came out of both heads when the relay started working |
> | 1. Dial through a loopback proxy | **Done** | `SshLoopbackProxy` on the request, honoured in `BuildConnectionInfo`. **Not the bridge** — see below |
> | 2. Jump hosts over it | Not started | No server change, and no bridge either: SSH.NET's own forward is the proxy |
> | 3. The relay: the bridge, then the ticket | Not started | The bridge's only consumer, so it lands with the feature that uses it |
> | 4. File transfer parity | Not started | The transfers screen opens its own connection. Already half-done — `OpenSftpAsync` shares `BuildConnectionInfo`, so it honours a proxy today; what is missing is anything passing it one |
> | 3. The relay: the bridge, then the ticket | **Done** | Server: `Features/Relay`. Client: `RelayBridge`, `ServerRelay`, and `VaultViewModel.OpenThroughAsync`. See "What step 3 built" |
> | 4. File transfer parity | **Done for the relay** | `TransfersViewModel` opens a relay for a relayed host exactly as the shell does. A jump chain will need the same line when step 2 lands |
## ◆ The relay's checkbox is a false promise, and that is a defect
@@ -171,7 +175,64 @@ the sentence step 0 added comes out of both heads.
**4. File transfer.** `ISftpSessionFactory.OpenSftpAsync` opens its own second connection, so a host that
needs a chain or a relay to reach needs it there too, or SFTP silently fails for exactly the hosts this
work exists for.
work exists for. ✅ for the relay.
## What step 3 built
**The server**, in `DodoSSH.Api/Features/Relay`:
- `POST /api/v1/relay/tickets` makes every decision. It finds the host, requires Connect on its vault and
the relay switched on, resolves the stored name, and runs every answer through `RelayAddressPolicy` —
the non-overridable list first, then the operator's denials, allow-list and private-network switch. What
passes goes into a Data Protection ticket; the name is not resolved again. **Every target refusal is one
answer**, 403 `relay-target-rejected` with one sentence, because anybody who can write a host can point
it anywhere, and a distinct "resolves, but refused" would be a scanner for the operator's DNS. The real
reason is logged (event 2301).
- `GET /api/v1/relay/connect` is anonymous at the HTTP layer and takes the ticket as the second
subprotocol. It verifies, checks expiry against `TimeProvider`, and spends the ticket with a unique
insert into `relay_ticket_use` before accepting the upgrade — so a refused ticket is a 401 and no socket.
After the upgrade, a refusal the client could not have avoided is a close code: 4429 for a limit, 4502
for a host the server could not reach, 4408 idle, 4409 lifetime, 1001 at the end of a shutdown's drain.
- `relay_session` is the audit row: opened before the dial, closed after, each from a scope of its own.
**The client:**
- `RelayBridge` (Client.Ssh) is the loopback SOCKS5 listener. It binds `127.0.0.1`, accepts once, answers
one CONNECT without acting on its target, and pipes to the WebSocket. It is handed to the connection as
the new `SshLoopbackProxy.Lifetime`, so the connection disposes it on every way a session ends, and a
failed connect disposes it before throwing.
- `ServerRelay` (Client.Session) asks for the ticket, opens the socket and starts the bridge, and turns
every refusal into a sentence. It is `IVaultServer.Relay`, decided from `/api/v1/meta` the way `Events`
is; a server without the feature supplies `UnavailableServerRelay`.
- `VaultViewModel` opens a relay for a host with the box ticked and **never falls back to dialling it
directly** — the user ticked it because that path does not work, or goes somewhere they do not mean. A
relay that closed with a reason replaces SSH.NET's transport error with that reason, and the
reachability probe is skipped for a relayed host, because it would test the direct path.
### ◆ Two things that are not what ADR 0004 wrote
- **No `System.IO.Pipelines`.** Each direction reads one buffer and does not read again until the write
has completed. That is the same end-to-end backpressure — a slow peer stops the read, which fills the
kernel's window, which throttles the far sender — with a tighter bound: two 32 KiB buffers a session
rather than ~2 MiB.
- **No `?ticket=` fallback, and no device id in the ticket.** The fallback exists for clients that cannot
set a subprotocol, which neither head is, and it would put a bearer credential in the one place — the
URL — that access logs keep. A device is never visible to the server on an ordinary request.
### ◆ A WebSocket operation is never cancelled, on either side
Cancelling a pending receive or send on a .NET WebSocket aborts it, and an aborted socket sends no close
frame. The close frame is the only way the client learns that the server could not reach the host, so both
the server's pump and the client's bridge stop a direction by closing and waiting a bounded time instead.
`AnIdleSession_IsClosedWithTheIdleCode` holds the server to it: it asserts the code arrives, not merely that
the socket ended.
### What a developer meets
The relay refuses loopback with no configuration escape, so the development stack's sshd on
`localhost:2222` cannot be reached through it. Point a relayed host at an address the API process reaches
on a real interface — the machine's own LAN address works — or at the container's address on the Docker
network.
## Traps already known
@@ -226,8 +287,8 @@ is right, sharing a lifetime policy is not.
row, and the status bar's `via bastion-eu` row, all of which say jump hosts are data-only.
- `Client.Import/ImportedHost.cs` — the note written onto every imported host with a `ProxyJump`, and the
remark above it.
- `README.md` and `docs/android-port.md` wherever the relay is described as available.
- ADR 0004 gains a note that its last consequence was built, and how.
- `README.md` and `docs/android-port.md` wherever the relay is described as available. ✅ at step 3.
- ADR 0004 gains a note that its last consequence was built, and how. ✅ at step 3.
## What the first draft of this document got wrong
@@ -26,9 +26,6 @@ internal sealed class GetMetaEndpoint(
/// <summary>Sync semantics version. Bumped when push or pull behaviour changes.</summary>
internal const int SyncProtocolVersion = 1;
/// <summary>Feature flag for the relay.</summary>
internal const string RelayFeature = "relay";
/// <inheritdoc />
public override void Configure()
{
@@ -50,7 +47,7 @@ internal sealed class GetMetaEndpoint(
List<string> features = ["teams"];
if (relay.Value.Enabled)
{
features.Add(RelayFeature);
features.Add(RelayProtocol.Feature);
}
// Advertised so a client knows whether to hold a socket open or rely on its timer. Absence is
@@ -0,0 +1,192 @@
using System.Net;
using System.Net.Sockets;
using DodoSSH.Api.Setup;
namespace DodoSSH.Api.Features.Relay;
/// <summary>
/// Decides which addresses and ports the relay may dial. ADR 0004 §3, in the order it is written.
/// </summary>
/// <remarks>
/// <para>
/// This runs on <em>addresses</em>, never on names. The name is resolved once, at ticket issue, and what
/// this permits goes into the ticket; the socket dials those addresses and nothing else. A check on the
/// name would be defeated by DNS rebinding — answer with a public address for the check and a loopback
/// one for the dial — which is the attack ADR 0004 is built around.
/// </para>
/// <para>
/// Three layers, and the order is the security property. The <see cref="Refused"/> list comes first and
/// has no configuration at all: nothing an operator sets, and nothing a host row says, reaches a
/// loopback, link-local, unspecified, multicast or metadata address. Then the operator's own denials,
/// then their allow-list, then the private-network switch.
/// </para>
/// <para>
/// <b>An address that embeds another is judged as the one it embeds.</b> <c>::ffff:127.0.0.1</c> is
/// loopback however it is spelled, and a check that compared it against IPv6 loopback alone would pass
/// it — the classic bypass the ADR names. The NAT64 well-known prefix is unwrapped the same way, because
/// on a network with a NAT64 gateway <c>64:ff9b::a9fe:a9fe</c> <em>is</em> the metadata endpoint.
/// </para>
/// </remarks>
internal sealed class RelayAddressPolicy
{
/// <summary>
/// Addresses the relay never dials, whatever it is configured to do.
/// </summary>
/// <remarks>
/// Broader than the ADR's list in two places, both deliberately. <c>240.0.0.0/4</c> — reserved, and
/// containing the limited broadcast address — has no host worth reaching. And IPv6 <c>::/96</c>, the
/// deprecated IPv4-compatible block, covers the unspecified address and loopback while also refusing
/// the spellings of IPv4 that a stack may or may not map; nothing legitimate lives there.
/// </remarks>
private static readonly IPNetwork[] Refused =
[
// IPv4: "this network", loopback, link-local (which holds 169.254.169.254), multicast, reserved.
IPNetwork.Parse("0.0.0.0/8"),
IPNetwork.Parse("127.0.0.0/8"),
IPNetwork.Parse("169.254.0.0/16"),
IPNetwork.Parse("224.0.0.0/4"),
IPNetwork.Parse("240.0.0.0/4"),
// Alibaba Cloud's metadata endpoint, which sits inside carrier-grade NAT space and so would
// otherwise be permitted as private — the range a Tailnet also uses, which is why the whole
// block cannot simply be refused.
IPNetwork.Parse("100.100.100.200/32"),
// IPv6: IPv4-compatible (with unspecified and loopback), link-local, multicast, and the AWS
// metadata endpoint's IPv6 address.
IPNetwork.Parse("::/96"),
IPNetwork.Parse("fe80::/10"),
IPNetwork.Parse("ff00::/8"),
IPNetwork.Parse("fd00:ec2::254/128"),
];
/// <summary>
/// Ranges <see cref="RelayOptions.AllowPrivateNetworks"/> governs.
/// </summary>
/// <remarks>
/// Carrier-grade NAT is here rather than treated as public, because it is where a Tailnet lives and
/// "the server is on the same Tailnet" is one of the shapes the relay exists for. Site-local
/// <c>fec0::/10</c> is deprecated and was private while it lived.
/// </remarks>
private static readonly IPNetwork[] Private =
[
IPNetwork.Parse("10.0.0.0/8"),
IPNetwork.Parse("172.16.0.0/12"),
IPNetwork.Parse("192.168.0.0/16"),
IPNetwork.Parse("100.64.0.0/10"),
IPNetwork.Parse("fc00::/7"),
IPNetwork.Parse("fec0::/10"),
];
/// <summary>The NAT64 well-known prefix, RFC 6052, whose low 32 bits are an IPv4 address.</summary>
private static readonly IPNetwork Nat64 = IPNetwork.Parse("64:ff9b::/96");
/// <summary>
/// Ports the relay never dials.
/// </summary>
/// <remarks>
/// SMTP and nothing else, per ADR 0004: a relay that reached port 25 is a spam cannon with the
/// operator's address on it. Database ports stay open, because forwarding to one is a headline use,
/// and the control that matters there is who may dial the host at all.
/// </remarks>
private static readonly int[] RefusedPorts = [25, 465, 587];
private readonly bool allowPrivate;
private readonly IPNetwork[] allowed;
private readonly IPNetwork[] denied;
public RelayAddressPolicy(RelayOptions options)
{
ArgumentNullException.ThrowIfNull(options);
allowPrivate = options.AllowPrivateNetworks;
allowed = ParseAll(options.AllowedNetworks);
denied = ParseAll(options.DeniedNetworks);
}
/// <summary>Whether a port may be dialled at all.</summary>
internal static bool PermitsPort(int port) =>
port is >= 1 and <= 65535 && !RefusedPorts.Contains(port);
/// <summary>
/// Whether every entry is a network this policy can read.
/// </summary>
/// <remarks>
/// For startup validation. A typo in a deny list that was silently skipped would be a hole the
/// operator believes is closed, so an unreadable entry stops the server instead.
/// <para>
/// Host bits are refused as well, although <see cref="IPNetwork.TryParse(string?, out IPNetwork)"/>
/// would mask them off quietly. <c>10.20.5.7/16</c> could be a /16 written from memory or a /32 with
/// the wrong suffix, and in a deny list those differ by sixty-five thousand addresses.
/// </para>
/// </remarks>
internal static bool AreNetworks(IEnumerable<string> entries) =>
entries.All(entry =>
IPNetwork.TryParse(entry, out var network)
&& IPAddress.TryParse(entry.AsSpan(0, entry.IndexOf('/', StringComparison.Ordinal)), out var written)
&& written.Equals(network.BaseAddress));
/// <summary>
/// The address the relay would actually be talking to, with any embedding unwrapped.
/// </summary>
internal static IPAddress Normalise(IPAddress address)
{
ArgumentNullException.ThrowIfNull(address);
if (address.IsIPv4MappedToIPv6)
{
return address.MapToIPv4();
}
if (address.AddressFamily == AddressFamily.InterNetworkV6 && Nat64.Contains(address))
{
var bytes = address.GetAddressBytes();
return new IPAddress(bytes.AsSpan(12, 4));
}
// A scope id is how an address names an interface on this machine, which is not something a
// host row gets to choose. Dropped, so what is judged is what is dialled.
return address.AddressFamily == AddressFamily.InterNetworkV6 && address.ScopeId != 0
? new IPAddress(address.GetAddressBytes())
: address;
}
/// <summary>Whether the relay may dial this address.</summary>
/// <remarks>
/// Judges the normalised address. Callers should dial <see cref="Normalise"/>'s answer too, so that
/// the address checked and the address connected to are the same object rather than two spellings
/// the operating system is trusted to agree about.
/// </remarks>
internal bool Permits(IPAddress address)
{
var target = Normalise(address);
if (Refused.Any(network => Within(network, target)))
{
return false;
}
if (denied.Any(network => Within(network, target)))
{
return false;
}
if (allowed.Length > 0)
{
return allowed.Any(network => Within(network, target));
}
return allowPrivate || !Private.Any(network => Within(network, target));
}
/// <remarks>
/// <see cref="IPNetwork.Contains"/> answers false across address families rather than throwing, so
/// a list mixing IPv4 and IPv6 needs no sorting — but it is spelled out here because the one-line
/// alternative reads as though it compared across families.
/// </remarks>
private static bool Within(IPNetwork network, IPAddress address) =>
network.BaseAddress.AddressFamily == address.AddressFamily && network.Contains(address);
private static IPNetwork[] ParseAll(IEnumerable<string> entries) =>
[.. entries.Select(entry => IPNetwork.Parse(entry))];
}
@@ -0,0 +1,722 @@
using System.Buffers;
using System.Net;
using System.Net.WebSockets;
using DodoSSH.Api.Setup;
using DodoSSH.Contracts;
using DodoSSH.Domain;
using DodoSSH.Infrastructure;
using FastEndpoints;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Options;
using Npgsql;
namespace DodoSSH.Api.Features.Relay;
/// <summary>
/// The relay socket: a ticket in, raw bytes to one host and back.
/// </summary>
/// <remarks>
/// <para>
/// Anonymous at the HTTP layer, because the ticket is the authorization and it is the whole of it. A
/// ticket carries no API authority — it names one host, one set of addresses and one session — so this
/// endpoint can accept it from a client that cannot set an <c>Authorization</c> header on an upgrade,
/// which is the constraint ADR 0004 was written around. It is verified, checked for freshness and
/// redeemed before the upgrade is accepted, so a refused ticket costs a problem document and no socket.
/// </para>
/// <para>
/// <b>Nothing here is scoped to the request.</b> A session lasts as long as a shell does, which is hours,
/// and anything scoped held across it is a pooled database connection held across it. The database is
/// touched three times — redeem, open the audit row, close it — each from a scope of its own that is
/// disposed before the next byte moves. That is ADR 0004's consequence about <c>DbContext</c> lifetime.
/// </para>
/// <para>
/// Backpressure is the await. Each direction reads one buffer, writes it, and does not read again until
/// the write has completed, so a slow reader on either side stops this side reading, which fills the
/// kernel's receive window, which throttles the sender at the far end. Nothing is queued in between, and
/// a session costs its two buffers whatever the peers do.
/// </para>
/// </remarks>
internal sealed class RelayConnectEndpoint(
IServiceScopeFactory scopes,
IRelayNetwork network,
RelayTicketCodec codec,
RelaySessionRegistry sessions,
IOptions<RelayOptions> options,
TimeProvider clock,
IHostApplicationLifetime lifetime,
ILogger<RelayConnectEndpoint> logger)
: EndpointWithoutRequest
{
/// <summary>
/// One direction's buffer.
/// </summary>
/// <remarks>
/// Room for an SSH packet at the size OpenSSH sends them, so a bulk transfer moves in one read and
/// one write per packet rather than several.
/// </remarks>
private const int BufferBytes = 32 * 1024;
/// <summary>How long a close handshake, or a peer's last write, may take before the socket is dropped.</summary>
private static readonly TimeSpan CloseTimeout = TimeSpan.FromSeconds(5);
/// <inheritdoc />
public override void Configure()
{
Get(RelayProtocol.ConnectPath);
// The ticket is the authorization. See the remarks on this class, and ADR 0004.
AllowAnonymous();
Description(b => b
.WithName("RelayConnect")
.WithSummary("Pipes raw bytes between a WebSocket and the host a relay ticket names.")
.WithTags("Relay"));
}
/// <inheritdoc />
public override async Task HandleAsync(CancellationToken ct)
{
var ticket = await AdmitTicketAsync(ct).ConfigureAwait(false);
if (ticket is null)
{
return;
}
var lease = sessions.TryAdmit(ticket.UserId);
try
{
using var socket = await HttpContext.WebSockets
.AcceptWebSocketAsync(new WebSocketAcceptContext { SubProtocol = RelayProtocol.SubProtocol })
.ConfigureAwait(false);
if (lease is null)
{
RelayLog.LimitReached(logger, ticket.UserId, sessions.Count);
await CloseAsync(
socket,
new Ending(
RelayCloseReason.Unspecified,
RelayProtocol.TooManySessionsCloseCode,
"Too many relay sessions are open."))
.ConfigureAwait(false);
return;
}
if (lifetime.ApplicationStopping.IsCancellationRequested)
{
await CloseAsync(socket, ShuttingDown).ConfigureAwait(false);
return;
}
await RelayAsync(socket, ticket, ct).ConfigureAwait(false);
}
finally
{
// Inside a try that starts before the upgrade, for the reason VaultEventsEndpoint gives: an
// accept that throws would otherwise hold a place against this user's cap forever.
lease?.Dispose();
}
}
private static Ending ShuttingDown { get; } = new(
RelayCloseReason.ServerShutdown,
(int)WebSocketCloseStatus.EndpointUnavailable,
"The server is shutting down.");
/// <summary>
/// Answers everything that is not a redeemable ticket, as ordinary HTTP, before any socket exists.
/// </summary>
/// <returns>The redeemed ticket, or null when a response has been sent.</returns>
/// <remarks>
/// A ticket that is forged, expired, spent, or simply absent all answer 401
/// <c>relay-ticket-invalid</c>, with one remedy — ask for another — and no hint of which it was.
/// </remarks>
private async Task<RelayTicket?> AdmitTicketAsync(CancellationToken cancellationToken)
{
if (!options.Value.Enabled)
{
await Send.ResultAsync(Problems.Coded(
StatusCodes.Status404NotFound,
ProblemCodes.RelayUnavailable,
"This server does not relay. GET /api/v1/meta lists the features it does offer."))
.ConfigureAwait(false);
return null;
}
var offered = HttpContext.WebSockets.WebSocketRequestedProtocols;
if (!HttpContext.WebSockets.IsWebSocketRequest
|| !offered.Contains(RelayProtocol.SubProtocol, StringComparer.Ordinal))
{
await Send.ResultAsync(Problems.Coded(
StatusCodes.Status400BadRequest,
ProblemCodes.MalformedRequest,
"This endpoint is a WebSocket. Send an upgrade request offering the "
+ $"'{RelayProtocol.SubProtocol}' subprotocol and the ticket as a second one, "
+ $"'{RelayProtocol.TicketProtocolPrefix}<ticket>'."))
.ConfigureAwait(false);
return null;
}
var sealedTicket = offered
.FirstOrDefault(protocol => protocol.StartsWith(RelayProtocol.TicketProtocolPrefix, StringComparison.Ordinal))
?[RelayProtocol.TicketProtocolPrefix.Length..];
var ticket = sealedTicket is { Length: > 0 } ? codec.TryOpen(sealedTicket) : null;
if (ticket is null || !await RedeemAsync(ticket, cancellationToken).ConfigureAwait(false))
{
RelayLog.TicketRefused(logger, ticket?.SessionId);
await Send.ResultAsync(Problems.Coded(
StatusCodes.Status401Unauthorized,
ProblemCodes.RelayTicketInvalid,
"This relay ticket is not valid: it is expired, already used, or was not issued here. "
+ $"Ask for another from POST {RelayProtocol.TicketsPath}."))
.ConfigureAwait(false);
return null;
}
return ticket;
}
/// <summary>
/// Spends the ticket, or answers false when it has been spent already.
/// </summary>
/// <remarks>
/// <para>
/// A unique insert, which is the only form of "once" that holds across nodes: two redemptions racing
/// on two nodes both attempt the insert and the database lets exactly one of them win.
/// </para>
/// <para>
/// Rows for tickets that can no longer verify are swept on the way. They guard nothing, and sweeping
/// here — on a path that runs once per session rather than once per request — keeps the table the
/// size of the last thirty seconds without a background job to forget about.
/// </para>
/// </remarks>
private async Task<bool> RedeemAsync(RelayTicket ticket, CancellationToken cancellationToken)
{
var scope = scopes.CreateAsyncScope();
await using var _ = scope.ConfigureAwait(false);
var database = scope.ServiceProvider.GetRequiredService<DodoDbContext>();
var now = clock.GetUtcNow();
await database.RelayTicketUses
.Where(use => use.ExpiresAtUtc < now)
.ExecuteDeleteAsync(cancellationToken)
.ConfigureAwait(false);
database.RelayTicketUses.Add(new RelayTicketUse
{
TicketId = ticket.TicketId,
ExpiresAtUtc = ticket.ExpiresAt,
UsedAtUtc = now,
});
try
{
await database.SaveChangesAsync(cancellationToken).ConfigureAwait(false);
return true;
}
catch (DbUpdateException exception)
when (string.Equals(
(exception.InnerException as PostgresException)?.SqlState,
PostgresErrorCodes.UniqueViolation,
StringComparison.Ordinal))
{
return false;
}
}
/// <summary>Dials the target, pipes until something ends it, and writes down what happened.</summary>
private async Task RelayAsync(WebSocket socket, RelayTicket ticket, CancellationToken requestAborted)
{
var started = clock.GetUtcNow();
await OpenAuditAsync(ticket, started).ConfigureAwait(false);
var dialled = await DialAsync(ticket, requestAborted).ConfigureAwait(false);
if (dialled is not var (target, address))
{
RelayLog.TargetUnreachable(logger, ticket.SessionId, ticket.HostId);
var unreachable = new Ending(
RelayCloseReason.TargetUnreachable,
RelayProtocol.TargetUnreachableCloseCode,
"The relay could not reach the host.");
await CloseAsync(socket, unreachable).ConfigureAwait(false);
await CloseAuditAsync(ticket.SessionId, null, new Counters(clock, started), unreachable.Reason).ConfigureAwait(false);
return;
}
var counters = new Counters(clock, started);
Ending ending;
await using (target.ConfigureAwait(false))
{
RelayLog.SessionOpened(logger, ticket.SessionId, ticket.UserId, ticket.HostId, sessions.Count);
ending = await PumpAsync(socket, target, counters, started, requestAborted).ConfigureAwait(false);
}
RelayLog.SessionClosed(
logger,
ticket.SessionId,
ending.Reason,
counters.ClientToTarget,
counters.TargetToClient);
await CloseAuditAsync(ticket.SessionId, address, counters, ending.Reason).ConfigureAwait(false);
}
/// <summary>
/// Connects to the first of the ticket's addresses that answers, within the connect timeout overall.
/// </summary>
/// <remarks>
/// Overall rather than per address, so a name with sixteen dead addresses costs five seconds rather
/// than eighty. In the order resolved, which is the order the operating system's resolver prefers.
/// </remarks>
private async Task<(Stream Target, IPAddress Address)?> DialAsync(
RelayTicket ticket,
CancellationToken cancellationToken)
{
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
timeout.CancelAfter(options.Value.ConnectTimeout);
foreach (var address in ticket.TargetAddresses)
{
try
{
var target = await network
.ConnectAsync(address, ticket.TargetPort, timeout.Token)
.ConfigureAwait(false);
return (target, address);
}
catch (OperationCanceledException)
{
return null;
}
catch (Exception exception) when (exception is IOException or System.Net.Sockets.SocketException)
{
// The next address, if there is one.
}
}
return null;
}
/// <summary>
/// Runs both directions and the clock until one of them ends the session, then ends it politely.
/// </summary>
/// <remarks>
/// <para>
/// <b>No operation on the WebSocket is ever cancelled.</b> Cancelling a pending receive or send on a
/// .NET WebSocket aborts it, and an aborted socket cannot send the close frame that is the only way
/// a client learns <em>why</em> its session ended — which, for an idle timeout or a drain, is the
/// difference between "reconnect" and "something is broken". So the token stops the target side and
/// the clock; the sender is let finish its last write; the close goes out; and the receiver ends on
/// the client's answering close. Each wait is bounded, and the socket is aborted only when a peer
/// has stopped answering altogether.
/// </para>
/// </remarks>
private async Task<Ending> PumpAsync(
WebSocket socket,
Stream target,
Counters counters,
DateTimeOffset started,
CancellationToken requestAborted)
{
using var pump = CancellationTokenSource.CreateLinkedTokenSource(requestAborted);
var upstream = ClientToTargetAsync(socket, target, counters, pump.Token);
var downstream = TargetToClientAsync(target, socket, counters, pump.Token);
var minding = MindAsync(counters, started, pump.Token);
var first = await Task.WhenAny(upstream, downstream, minding).ConfigureAwait(false);
var ending = await first.ConfigureAwait(false) ?? new Ending(
RelayCloseReason.ClientClosed, (int)WebSocketCloseStatus.NormalClosure, string.Empty);
await pump.CancelAsync().ConfigureAwait(false);
// The only sender must be done before the close frame is written: a WebSocket permits one send
// at a time and faults permanently on a second.
if (!await SettlesAsync(downstream).ConfigureAwait(false))
{
socket.Abort();
}
await CloseAsync(socket, ending).ConfigureAwait(false);
if (!await SettlesAsync(upstream).ConfigureAwait(false))
{
socket.Abort();
await SettlesAsync(upstream).ConfigureAwait(false);
}
await SettlesAsync(minding).ConfigureAwait(false);
return ending;
}
/// <summary>Client to target: every binary message is the next bytes of the stream.</summary>
private static async Task<Ending?> ClientToTargetAsync(
WebSocket socket,
Stream target,
Counters counters,
CancellationToken cancellationToken)
{
var buffer = ArrayPool<byte>.Shared.Rent(BufferBytes);
try
{
while (true)
{
ValueWebSocketReceiveResult received;
try
{
// Not cancellable, deliberately. See PumpAsync.
received = await socket.ReceiveAsync(buffer.AsMemory(), CancellationToken.None)
.ConfigureAwait(false);
}
catch (Exception exception) when (exception is WebSocketException or OperationCanceledException)
{
return ClientGone;
}
if (received.MessageType == WebSocketMessageType.Close)
{
return ClientGone;
}
// A byte pipe has no text. A client sending some has the wrong idea about this endpoint,
// and forwarding it into an SSH stream would corrupt that stream at best.
if (received.MessageType != WebSocketMessageType.Binary)
{
return new Ending(
RelayCloseReason.Error,
(int)WebSocketCloseStatus.InvalidMessageType,
"The relay carries binary messages only.");
}
try
{
await target.WriteAsync(buffer.AsMemory(0, received.Count), cancellationToken)
.ConfigureAwait(false);
}
catch (OperationCanceledException)
{
return null;
}
catch (IOException)
{
return TargetGone;
}
counters.AddClientToTarget(received.Count);
}
}
finally
{
ArrayPool<byte>.Shared.Return(buffer);
}
}
/// <summary>Target to client: every read becomes one binary message.</summary>
private static async Task<Ending?> TargetToClientAsync(
Stream target,
WebSocket socket,
Counters counters,
CancellationToken cancellationToken)
{
var buffer = ArrayPool<byte>.Shared.Rent(BufferBytes);
try
{
while (true)
{
int read;
try
{
read = await target.ReadAsync(buffer.AsMemory(0, BufferBytes), cancellationToken)
.ConfigureAwait(false);
}
catch (OperationCanceledException)
{
return null;
}
catch (IOException)
{
return TargetGone;
}
if (read == 0)
{
return TargetGone;
}
try
{
// Not cancellable, deliberately. See PumpAsync.
await socket
.SendAsync(
buffer.AsMemory(0, read),
WebSocketMessageType.Binary,
endOfMessage: true,
CancellationToken.None)
.ConfigureAwait(false);
}
catch (Exception exception) when (exception is WebSocketException or ObjectDisposedException)
{
return ClientGone;
}
counters.AddTargetToClient(read);
}
}
finally
{
ArrayPool<byte>.Shared.Return(buffer);
}
}
/// <summary>
/// Ends the session at the idle timeout, the lifetime cap, or the end of a shutdown's drain.
/// </summary>
/// <remarks>
/// <para>
/// A shutdown does not end a session; it starts a countdown. ADR 0004 asks that a
/// <c>docker compose up -d</c> not guillotine live shells, so a session in progress when the server is
/// told to stop is given the drain timeout to finish — most will, because the user was already
/// closing it or the deployment waits — and is closed with 1001 only when that runs out, which tells
/// the client to reconnect rather than to report a fault. The host's own shutdown timeout is set
/// longer than the drain for this to be possible at all.
/// </para>
/// <para>
/// Idle means no bytes in either direction. WebSocket pings are not bytes of the stream and do not
/// count; SSH's own keepalives do, which is how a client that wants an idle shell kept open says so.
/// </para>
/// </remarks>
private async Task<Ending?> MindAsync(Counters counters, DateTimeOffset started, CancellationToken cancellationToken)
{
var settings = options.Value;
var hardDeadline = started + settings.MaxSessionDuration;
DateTimeOffset? drainDeadline = null;
while (!cancellationToken.IsCancellationRequested)
{
var now = clock.GetUtcNow();
if (drainDeadline is null && lifetime.ApplicationStopping.IsCancellationRequested)
{
drainDeadline = now + settings.DrainTimeout;
}
if (now >= hardDeadline)
{
return new Ending(
RelayCloseReason.DurationLimit,
RelayProtocol.DurationLimitCloseCode,
"This relay session reached its maximum lifetime.");
}
if (drainDeadline is { } drain && now >= drain)
{
return ShuttingDown;
}
var idleDeadline = counters.LastActivity + settings.IdleTimeout;
if (now >= idleDeadline)
{
return new Ending(
RelayCloseReason.IdleTimeout,
RelayProtocol.IdleTimeoutCloseCode,
"This relay session carried nothing for the idle timeout.");
}
var next = new[] { hardDeadline, idleDeadline, drainDeadline ?? DateTimeOffset.MaxValue }.Min();
// Woken early by the start of a shutdown, so the drain is measured from when it began rather
// than from whenever the idle clock next came round.
using var wake = drainDeadline is null
? CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, lifetime.ApplicationStopping)
: CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
try
{
await Task.Delay(next - now, clock, wake.Token).ConfigureAwait(false);
}
catch (OperationCanceledException) when (!cancellationToken.IsCancellationRequested)
{
// The shutdown began. Round again to start the drain.
}
catch (OperationCanceledException)
{
return null;
}
}
return null;
}
private static Ending ClientGone { get; } =
new(RelayCloseReason.ClientClosed, (int)WebSocketCloseStatus.NormalClosure, string.Empty);
private static Ending TargetGone { get; } =
new(RelayCloseReason.TargetClosed, (int)WebSocketCloseStatus.NormalClosure, "The host closed the connection.");
/// <summary>
/// Opens the audit row, before the target is dialled, so a session that never reached its host is
/// still one the operator can see.
/// </summary>
private async Task OpenAuditAsync(RelayTicket ticket, DateTimeOffset started)
{
var scope = scopes.CreateAsyncScope();
await using var _ = scope.ConfigureAwait(false);
var database = scope.ServiceProvider.GetRequiredService<DodoDbContext>();
database.RelaySessions.Add(new RelaySession
{
Id = ticket.SessionId,
UserId = ticket.UserId,
HostId = ticket.HostId,
VaultId = ticket.VaultId,
TargetHost = ticket.TargetHost,
TargetPort = ticket.TargetPort,
ClientAddress = HttpContext.Connection.RemoteIpAddress?.ToString(),
StartedAtUtc = started,
});
// Not the request's token: the row has to be written even for a client that has already gone,
// or a session that was cut off would be the one with no record.
await database.SaveChangesAsync(CancellationToken.None).ConfigureAwait(false);
}
/// <summary>
/// Closes the audit row. A failure is logged and swallowed: the bytes have already moved, and
/// throwing here would only turn a finished session into an unhandled exception.
/// </summary>
private async Task CloseAuditAsync(
Guid sessionId,
IPAddress? address,
Counters counters,
RelayCloseReason reason)
{
try
{
var scope = scopes.CreateAsyncScope();
await using var _ = scope.ConfigureAwait(false);
var database = scope.ServiceProvider.GetRequiredService<DodoDbContext>();
var ended = clock.GetUtcNow();
var dialled = address?.ToString();
await database.RelaySessions
.Where(session => session.Id == sessionId)
.ExecuteUpdateAsync(
set => set
.SetProperty(session => session.EndedAtUtc, ended)
.SetProperty(session => session.TargetAddress, dialled)
.SetProperty(session => session.BytesClientToTarget, counters.ClientToTarget)
.SetProperty(session => session.BytesTargetToClient, counters.TargetToClient)
.SetProperty(session => session.CloseReason, reason),
CancellationToken.None)
.ConfigureAwait(false);
}
catch (Exception exception) when (exception is DbUpdateException or NpgsqlException or InvalidOperationException)
{
RelayLog.AuditCloseFailed(logger, sessionId, exception);
}
}
private static async Task CloseAsync(WebSocket socket, Ending ending)
{
if (socket.State is not (WebSocketState.Open or WebSocketState.CloseReceived))
{
return;
}
using var timeout = new CancellationTokenSource(CloseTimeout);
try
{
await socket
.CloseOutputAsync((WebSocketCloseStatus)ending.Code, ending.Description, timeout.Token)
.ConfigureAwait(false);
}
catch (Exception exception)
when (exception is OperationCanceledException or WebSocketException or ObjectDisposedException)
{
// The peer is already gone. There is nothing to tell it.
}
}
/// <summary>Waits for a loop to finish, for at most the close timeout.</summary>
/// <returns>Whether it finished.</returns>
private static async Task<bool> SettlesAsync(Task loop)
{
try
{
await loop.WaitAsync(CloseTimeout).ConfigureAwait(false);
return true;
}
catch (TimeoutException)
{
return false;
}
catch (Exception exception)
when (exception is OperationCanceledException or WebSocketException or IOException or ObjectDisposedException)
{
return true;
}
}
/// <summary>How a session ended, and what the client is told.</summary>
private sealed record Ending(RelayCloseReason Reason, int Code, string Description);
/// <summary>
/// Byte counts and the time of the last byte, written by the two directions and read by the clock.
/// </summary>
private sealed class Counters(TimeProvider clock, DateTimeOffset started)
{
private long clientToTarget;
private long targetToClient;
private long lastActivityTicks = started.UtcTicks;
internal long ClientToTarget => Interlocked.Read(ref clientToTarget);
internal long TargetToClient => Interlocked.Read(ref targetToClient);
internal DateTimeOffset LastActivity => new(Interlocked.Read(ref lastActivityTicks), TimeSpan.Zero);
internal void AddClientToTarget(int count)
{
Interlocked.Add(ref clientToTarget, count);
Interlocked.Exchange(ref lastActivityTicks, clock.GetUtcNow().UtcTicks);
}
internal void AddTargetToClient(int count)
{
Interlocked.Add(ref targetToClient, count);
Interlocked.Exchange(ref lastActivityTicks, clock.GetUtcNow().UtcTicks);
}
}
}
@@ -0,0 +1,94 @@
using DodoSSH.Domain;
namespace DodoSSH.Api.Features.Relay;
/// <summary>Source-generated log messages for the relay.</summary>
/// <remarks>
/// <para>
/// Ids, counts and reasons. Never a ticket, which is a bearer credential for thirty seconds, and never
/// the target's address: that is in the audit table, where retention is the operator's decision, and
/// copying it into a log would make it the log pipeline's decision too.
/// </para>
/// <para>
/// <see cref="TargetRejected"/> is the one place the reason for a refused target is written down. The
/// client is told nothing specific, deliberately — see <c>RelayTicketEndpoint</c> — so this line is how
/// an operator answers "why will the relay not reach this host".
/// </para>
/// </remarks>
internal static partial class RelayLog
{
[LoggerMessage(
EventId = 2301,
Level = LogLevel.Information,
Message = "Relay ticket refused for user {UserId} and host {HostId}: {Reason}.")]
internal static partial void TargetRejected(ILogger logger, Guid userId, Guid hostId, string reason);
[LoggerMessage(
EventId = 2302,
Level = LogLevel.Debug,
Message = "Relay ticket issued to user {UserId} for host {HostId}, session {SessionId}, with "
+ "{AddressCount} permitted address(es).")]
internal static partial void TicketIssued(
ILogger logger,
Guid userId,
Guid hostId,
Guid sessionId,
int addressCount);
/// <remarks>
/// The session id is known only for a ticket that verified and had already been spent, which is the
/// case worth an operator's attention: a replay, or a client retrying a redemption it thinks failed.
/// </remarks>
[LoggerMessage(
EventId = 2303,
Level = LogLevel.Information,
Message = "Relay ticket refused at the socket (session {SessionId}): forged, expired or already used.")]
internal static partial void TicketRefused(ILogger logger, Guid? sessionId);
[LoggerMessage(
EventId = 2304,
Level = LogLevel.Information,
Message = "Relay session refused for user {UserId}: a session limit is met, with {SessionCount} "
+ "open on this node.")]
internal static partial void LimitReached(ILogger logger, Guid userId, int sessionCount);
[LoggerMessage(
EventId = 2305,
Level = LogLevel.Information,
Message = "Relay session {SessionId} could not reach host {HostId} on any permitted address.")]
internal static partial void TargetUnreachable(ILogger logger, Guid sessionId, Guid hostId);
[LoggerMessage(
EventId = 2306,
Level = LogLevel.Debug,
Message = "Relay session {SessionId} opened for user {UserId} to host {HostId}; {SessionCount} "
+ "open on this node.")]
internal static partial void SessionOpened(
ILogger logger,
Guid sessionId,
Guid userId,
Guid hostId,
int sessionCount);
[LoggerMessage(
EventId = 2307,
Level = LogLevel.Debug,
Message = "Relay session {SessionId} closed ({Reason}) after {ClientToTarget} byte(s) up and "
+ "{TargetToClient} down.")]
internal static partial void SessionClosed(
ILogger logger,
Guid sessionId,
RelayCloseReason reason,
long clientToTarget,
long targetToClient);
/// <remarks>
/// A warning rather than an error: the session itself is over and was fine. What is lost is the
/// audit row's second half, which leaves it looking like a session still open.
/// </remarks>
[LoggerMessage(
EventId = 2308,
Level = LogLevel.Warning,
Message = "Relay session {SessionId} ended but its audit record could not be closed.")]
internal static partial void AuditCloseFailed(ILogger logger, Guid sessionId, Exception exception);
}
@@ -0,0 +1,61 @@
using System.Net;
using System.Net.Sockets;
namespace DodoSSH.Api.Features.Relay;
/// <summary>
/// The two things the relay asks of the network: what a name resolves to, and a connection to an address.
/// </summary>
/// <remarks>
/// <para>
/// A seam for tests, and a narrow one on purpose. <see cref="RelayAddressPolicy"/> is not behind it:
/// whatever this answers, the policy judges, and a test that replaces this still runs every address
/// through the real rules. What a test cannot do without the seam is dial anything, because every
/// address a test machine can reach on its own — loopback — is one the policy refuses outright.
/// </para>
/// </remarks>
internal interface IRelayNetwork
{
/// <summary>Resolves a stored hostname, which may be an address literal.</summary>
Task<IPAddress[]> ResolveAsync(string host, CancellationToken cancellationToken);
/// <summary>Opens a TCP connection to one address.</summary>
/// <returns>A stream that owns the connection.</returns>
Task<Stream> ConnectAsync(IPAddress address, int port, CancellationToken cancellationToken);
}
/// <summary>The operating system's resolver and sockets.</summary>
internal sealed class SystemRelayNetwork : IRelayNetwork
{
/// <inheritdoc />
public Task<IPAddress[]> ResolveAsync(string host, CancellationToken cancellationToken) =>
// A literal parses to itself without a lookup, which is what an address typed into a host
// editor should do — and what makes "10.0.0.5" mean the same thing on every node.
IPAddress.TryParse(host, out var literal)
? Task.FromResult<IPAddress[]>([literal])
: Dns.GetHostAddressesAsync(host, cancellationToken);
/// <inheritdoc />
public async Task<Stream> ConnectAsync(IPAddress address, int port, CancellationToken cancellationToken)
{
var socket = new Socket(address.AddressFamily, SocketType.Stream, ProtocolType.Tcp)
{
// An interactive shell is a stream of keystrokes; Nagle would hold each one back for the
// acknowledgement of the last, which is the latency the user feels as a sticky terminal.
NoDelay = true,
};
try
{
await socket.ConnectAsync(new IPEndPoint(address, port), cancellationToken).ConfigureAwait(false);
return new NetworkStream(socket, ownsSocket: true);
}
catch
{
socket.Dispose();
throw;
}
}
}
@@ -0,0 +1,111 @@
using DodoSSH.Api.Setup;
using Microsoft.Extensions.Options;
namespace DodoSSH.Api.Features.Relay;
/// <summary>
/// Counts the relay sessions open on this node, and refuses the ones over ADR 0004's limits.
/// </summary>
/// <remarks>
/// <para>
/// Per node, and so per user <em>per node</em>: a user behind a round-robin balancer with three nodes
/// could hold three times the per-user cap. That is accepted rather than fixed, because fixing it means
/// shared state on the hot path of every connect, and the cap exists to stop one account exhausting a
/// node — which a per-node count does exactly.
/// </para>
/// <para>
/// A singleton, because a session outlives the request that opened it. It holds counts and nothing else;
/// the sockets belong to the handlers that opened them.
/// </para>
/// </remarks>
internal sealed class RelaySessionRegistry(IOptions<RelayOptions> options)
{
private readonly Lock gate = new();
private readonly Dictionary<Guid, int> perUser = [];
private int total;
/// <summary>How many sessions are open on this node.</summary>
internal int Count
{
get
{
lock (gate)
{
return total;
}
}
}
/// <summary>Whether a session for this user would be admitted now.</summary>
/// <remarks>
/// Advisory, for the ticket endpoint: it lets a caller already at the cap hear so as an HTTP status
/// before it has spent a ticket, but only <see cref="TryAdmit"/> decides, since the answer here can
/// be stale by the time the socket arrives — possibly at another node.
/// </remarks>
internal bool HasRoomFor(Guid userId)
{
var settings = options.Value;
lock (gate)
{
return total < settings.MaxConcurrentSessionsTotal
&& perUser.GetValueOrDefault(userId) < settings.MaxConcurrentSessionsPerUser;
}
}
/// <summary>Admits a session, or answers null when a limit is met.</summary>
/// <returns>A lease that gives the place back when disposed.</returns>
internal Lease? TryAdmit(Guid userId)
{
var settings = options.Value;
lock (gate)
{
var held = perUser.GetValueOrDefault(userId);
if (total >= settings.MaxConcurrentSessionsTotal || held >= settings.MaxConcurrentSessionsPerUser)
{
return null;
}
perUser[userId] = held + 1;
total++;
return new Lease(this, userId);
}
}
private void Release(Guid userId)
{
lock (gate)
{
total--;
// Removed at zero rather than left behind, so the dictionary is the size of the users with a
// session open and not of every user who ever had one.
if (perUser[userId] <= 1)
{
perUser.Remove(userId);
}
else
{
perUser[userId]--;
}
}
}
/// <summary>One admitted session's place. Idempotent to dispose.</summary>
internal sealed class Lease(RelaySessionRegistry registry, Guid userId) : IDisposable
{
private int released;
/// <inheritdoc />
public void Dispose()
{
if (Interlocked.Exchange(ref released, 1) == 0)
{
registry.Release(userId);
}
}
}
}
@@ -0,0 +1,171 @@
using System.Net;
using System.Security.Cryptography;
using Microsoft.AspNetCore.DataProtection;
using Microsoft.AspNetCore.WebUtilities;
namespace DodoSSH.Api.Features.Relay;
/// <summary>
/// Everything the relay socket needs to know, decided by the ticket endpoint.
/// </summary>
/// <remarks>
/// The addresses are the resolved ones and the socket dials nothing else. There is no hostname for it
/// to resolve again — <see cref="TargetHost"/> is carried for the audit row only — which is what keeps a
/// DNS answer that changes between issue and connect from changing where the bytes go.
/// </remarks>
/// <param name="TicketId">The replay guard's key.</param>
/// <param name="SessionId">The audit row's id, already given to the client.</param>
/// <param name="UserId">Who asked.</param>
/// <param name="HostId">The host it is for.</param>
/// <param name="VaultId">The host's vault at issue.</param>
/// <param name="TargetHost">The hostname as stored, for the audit row.</param>
/// <param name="TargetAddresses">The permitted addresses the name resolved to, in the order to try.</param>
/// <param name="TargetPort">The port.</param>
/// <param name="ExpiresAt">When the ticket stops verifying.</param>
internal sealed record RelayTicket(
Guid TicketId,
Guid SessionId,
Guid UserId,
Guid HostId,
Guid VaultId,
string TargetHost,
IReadOnlyList<IPAddress> TargetAddresses,
int TargetPort,
DateTimeOffset ExpiresAt);
/// <summary>
/// Seals and opens relay tickets.
/// </summary>
/// <remarks>
/// <para>
/// A Data Protection payload, per ADR 0004: authenticated and encrypted, with key rotation handled by
/// the framework, and verifiable on any node that shares the key ring. Encrypted rather than merely
/// signed because the ticket travels in a header every proxy on the way can read, and what it holds is
/// the resolved address of a machine on the operator's network.
/// </para>
/// <para>
/// The expiry is inside the payload and checked against <see cref="TimeProvider"/>, rather than left to
/// the framework's time-limited protector, which reads the system clock directly — the thing this
/// repository bans so that time can be faked in tests.
/// </para>
/// <para>
/// The layout is a version byte and fixed fields, written by hand. Small enough that a serialiser would
/// be the larger half of the code, and a hand layout has no reflection for trimming to break.
/// </para>
/// </remarks>
internal sealed class RelayTicketCodec(IDataProtectionProvider protection, TimeProvider clock)
{
/// <summary>The purpose string, which isolates these payloads from every other protector's.</summary>
private const string Purpose = "DodoSSH.Relay.Ticket";
private const byte LayoutVersion = 1;
/// <summary>More addresses than any real name resolves to, and a bound on what a ticket can carry.</summary>
internal const int MaxAddresses = 16;
private readonly IDataProtector protector = protection.CreateProtector(Purpose);
internal string Seal(RelayTicket ticket)
{
ArgumentNullException.ThrowIfNull(ticket);
using var buffer = new MemoryStream();
using (var writer = new BinaryWriter(buffer))
{
writer.Write(LayoutVersion);
writer.Write(ticket.TicketId.ToByteArray());
writer.Write(ticket.SessionId.ToByteArray());
writer.Write(ticket.UserId.ToByteArray());
writer.Write(ticket.HostId.ToByteArray());
writer.Write(ticket.VaultId.ToByteArray());
writer.Write(ticket.ExpiresAt.ToUnixTimeMilliseconds());
writer.Write(ticket.TargetPort);
writer.Write(ticket.TargetHost);
writer.Write((byte)ticket.TargetAddresses.Count);
foreach (var address in ticket.TargetAddresses)
{
var bytes = address.GetAddressBytes();
writer.Write((byte)bytes.Length);
writer.Write(bytes);
}
}
// Base64url, whose alphabet is all token characters, so it can stand in a subprotocol name.
return WebEncoders.Base64UrlEncode(protector.Protect(buffer.ToArray()));
}
/// <summary>Opens a ticket, or answers null for one that is forged, damaged or expired.</summary>
/// <remarks>
/// One answer for all three, because the caller's remedy is the same — ask for another — and a
/// distinct "expired" would tell a forger which of their attempts had verified.
/// </remarks>
internal RelayTicket? TryOpen(string sealedTicket)
{
byte[] payload;
try
{
payload = protector.Unprotect(WebEncoders.Base64UrlDecode(sealedTicket));
}
catch (Exception exception) when (exception is CryptographicException or FormatException)
{
return null;
}
RelayTicket ticket;
try
{
ticket = Read(payload);
}
catch (Exception exception)
when (exception is EndOfStreamException or ArgumentException or FormatException)
{
return null;
}
return ticket.ExpiresAt > clock.GetUtcNow() ? ticket : null;
}
private static RelayTicket Read(byte[] payload)
{
using var reader = new BinaryReader(new MemoryStream(payload));
if (reader.ReadByte() != LayoutVersion)
{
throw new FormatException("Unknown ticket layout.");
}
var ticketId = new Guid(reader.ReadBytes(16));
var sessionId = new Guid(reader.ReadBytes(16));
var userId = new Guid(reader.ReadBytes(16));
var hostId = new Guid(reader.ReadBytes(16));
var vaultId = new Guid(reader.ReadBytes(16));
var expiresAt = DateTimeOffset.FromUnixTimeMilliseconds(reader.ReadInt64());
var port = reader.ReadInt32();
var host = reader.ReadString();
var count = reader.ReadByte();
if (count is 0 or > MaxAddresses)
{
throw new FormatException("A ticket names between one and sixteen addresses.");
}
var addresses = new IPAddress[count];
for (var index = 0; index < count; index++)
{
var length = reader.ReadByte();
if (length is not (4 or 16))
{
throw new FormatException("An address is four or sixteen bytes.");
}
addresses[index] = new IPAddress(reader.ReadBytes(length));
}
return new RelayTicket(ticketId, sessionId, userId, hostId, vaultId, host, addresses, port, expiresAt);
}
}
@@ -0,0 +1,231 @@
using System.Net;
using System.Net.Sockets;
using DodoSSH.Api.Authorization;
using DodoSSH.Api.Setup;
using DodoSSH.Contracts;
using DodoSSH.Domain.Authorization;
using DodoSSH.Infrastructure;
using FastEndpoints;
using Microsoft.AspNetCore.Http.HttpResults;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Options;
namespace DodoSSH.Api.Features.Relay;
/// <summary>
/// Issues the ticket that lets exactly one relay socket reach exactly one host.
/// </summary>
/// <remarks>
/// <para>
/// Every decision the relay makes is made here, with the full bearer token and the full access model in
/// hand: which host, whether the caller may connect to it, which addresses its name resolves to and
/// which of those the policy permits. The socket that follows decides nothing. It checks that the ticket
/// is genuine, fresh and unused, and dials what the ticket says. See ADR 0004.
/// </para>
/// <para>
/// <b>Every refusal about the target is the same refusal.</b> A host that does not exist, one in a vault
/// the caller cannot reach, one without the relay switched on, one whose name does not resolve, and one
/// whose every address the policy refuses all answer 403 <c>relay-target-rejected</c> with the same
/// sentence. Anyone who can write a host can set its address to anything, so a distinct answer for
/// "resolves, but refused" would turn this endpoint into a scanner for the operator's internal DNS. The
/// real reason goes to the server log, where the operator can read it.
/// </para>
/// </remarks>
internal sealed class RelayTicketEndpoint(
ICurrentUserContext currentUser,
IVaultAccessService vaultAccess,
DodoDbContext database,
IRelayNetwork network,
RelayTicketCodec codec,
RelaySessionRegistry sessions,
IOptions<RelayOptions> options,
TimeProvider clock,
ILogger<RelayTicketEndpoint> logger)
: Endpoint<RelayTicketRequest, Results<Ok<RelayTicketResponse>, ProblemHttpResult>>
{
/// <summary>What every target refusal says, whatever the reason was.</summary>
private const string RejectedDetail =
"The relay will not connect to this host. Check that the relay is switched on for it and that its "
+ "address is one this server may reach; the server's log records which.";
/// <inheritdoc />
public override void Configure()
{
Post(RelayProtocol.TicketsPath);
// Enrolled, matching sync. A relayed host is still a host whose name, username and key are
// ciphertext, so a caller with no identity key has no way to have learned which host to ask for.
Policies(Auth.EnrolledPolicy);
Description(b => b
.WithName("IssueRelayTicket")
.WithSummary("Issues a single-use ticket for one relay connection to one host.")
.WithTags("Relay"));
}
/// <inheritdoc />
public override async Task<Results<Ok<RelayTicketResponse>, ProblemHttpResult>> ExecuteAsync(
RelayTicketRequest req,
CancellationToken ct)
{
var settings = options.Value;
if (!settings.Enabled)
{
return Problems.Coded(
StatusCodes.Status404NotFound,
ProblemCodes.RelayUnavailable,
"This server does not relay. Connect to the host directly; GET /api/v1/meta lists the "
+ "features this server does offer.");
}
var user = await currentUser.GetOrProvisionAsync(ct).ConfigureAwait(false);
var (target, refusal) = await FindTargetAsync(user.Id, req, ct).ConfigureAwait(false);
if (target is not var (hostname, port, vaultId))
{
return Rejected(user.Id, req.HostId, refusal!);
}
// Before resolving rather than after: a caller already at the cap is told so without this server
// spending a DNS query on a ticket that could not be redeemed.
if (!sessions.HasRoomFor(user.Id))
{
RelayLog.LimitReached(logger, user.Id, sessions.Count);
return Problems.Coded(
StatusCodes.Status429TooManyRequests,
ProblemCodes.RelayLimitReached,
"You have as many relay sessions open as this server allows. Close one and try again.");
}
var addresses = await ResolvePermittedAsync(hostname, ct).ConfigureAwait(false);
if (addresses is not { Length: > 0 })
{
return Rejected(user.Id, req.HostId, addresses is null
? "its name did not resolve"
: "every address its name resolved to is refused by policy");
}
var ticket = new RelayTicket(
TicketId: Guid.CreateVersion7(),
SessionId: Guid.CreateVersion7(),
UserId: user.Id,
HostId: req.HostId,
VaultId: vaultId,
TargetHost: hostname,
TargetAddresses: addresses,
TargetPort: port,
ExpiresAt: clock.GetUtcNow() + settings.TicketLifetime);
RelayLog.TicketIssued(logger, user.Id, req.HostId, ticket.SessionId, addresses.Length);
return TypedResults.Ok(new RelayTicketResponse(
Ticket: codec.Seal(ticket),
WebSocketUrl: new Uri(settings.WebSocketUrl!, UriKind.Absolute),
SubProtocol: RelayProtocol.SubProtocol,
SessionId: ticket.SessionId,
ExpiresAt: ticket.ExpiresAt));
}
/// <summary>
/// Finds the host, and decides whether this caller may be relayed to it at all.
/// </summary>
/// <returns>The stored target, or the reason there is none — for the log, never for the caller.</returns>
private async Task<((string Hostname, int Port, Guid VaultId)? Target, string? Refusal)> FindTargetAsync(
Guid userId,
RelayTicketRequest req,
CancellationToken cancellationToken)
{
if (req.PortForwardId is not null)
{
return (null, "a port forward was requested, which v1 does not relay");
}
var host = await database.Hosts
.AsNoTracking()
.SingleOrDefaultAsync(h => h.Id == req.HostId && h.DeletedAtUtc == null, cancellationToken)
.ConfigureAwait(false);
if (host is null)
{
return (null, "no such host");
}
var access = await vaultAccess.ResolveAsync(userId, host.VaultId, cancellationToken).ConfigureAwait(false);
// Connect rather than Read, because Connect is the permission that says what this is. Every role
// that reads today also connects — see VaultAccessService.ForRole — but a role that ever split
// them should be refused a relay here without anybody remembering to look.
if (!access.Granted || !access.Permissions.HasFlag(PermissionFlags.Connect))
{
return (null, "the caller may not connect to this host's vault");
}
// The CHECK constraint makes the second half unreachable while the first is true. Tested anyway,
// because the cost of trusting it is dialling a null.
if (!host.RelayEnabled || host.Hostname is not { } hostname || host.Port is not { } port)
{
return (null, "the relay is not switched on for this host");
}
return RelayAddressPolicy.PermitsPort(port)
? ((hostname, port, host.VaultId), null)
: (null, $"port {port} is refused by policy");
}
/// <summary>
/// Resolves the stored name, and keeps the addresses the policy permits.
/// </summary>
/// <returns>The permitted addresses, possibly none; or null when the name did not resolve.</returns>
/// <remarks>
/// <para>
/// Filtered rather than refused as a whole when one answer is bad. A name that resolves to a public
/// address and a link-local one is dialled on the public address, and never on the other — the
/// ticket carries only what passed, so there is nothing else for the socket to try.
/// </para>
/// <para>
/// The addresses kept are the normalised ones, so what is dialled is exactly what was judged rather
/// than a second spelling of it.
/// </para>
/// </remarks>
private async Task<IPAddress[]?> ResolvePermittedAsync(string hostname, CancellationToken cancellationToken)
{
var policy = new RelayAddressPolicy(options.Value);
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
timeout.CancelAfter(options.Value.ConnectTimeout);
IPAddress[] resolved;
try
{
resolved = await network.ResolveAsync(hostname, timeout.Token).ConfigureAwait(false);
}
catch (SocketException)
{
return null;
}
catch (OperationCanceledException) when (!cancellationToken.IsCancellationRequested)
{
return null;
}
return
[
.. resolved
.Where(policy.Permits)
.Select(RelayAddressPolicy.Normalise)
.Distinct()
.Take(RelayTicketCodec.MaxAddresses),
];
}
private ProblemHttpResult Rejected(Guid userId, Guid hostId, string reason)
{
RelayLog.TargetRejected(logger, userId, hostId, reason);
return Problems.Coded(StatusCodes.Status403Forbidden, ProblemCodes.RelayTargetRejected, RejectedDetail);
}
}
+3
View File
@@ -49,6 +49,9 @@ builder.Services.AddSingleton<VaultEventHub>();
builder.Services.AddSingleton<IVaultEventPublisher>(
provider => provider.GetRequiredService<VaultEventHub>());
// The relay's services, its ticket key ring, and a shutdown long enough for its drain. See ADR 0004.
builder.Services.AddDodoRelay();
// Scoped rather than the AddAuthorization default of singleton: the handler reads the request's
// DbContext, and a singleton would capture one for the lifetime of the process.
builder.Services.AddScoped<IAuthorizationHandler, EnrolledHandler>();
+27 -3
View File
@@ -54,6 +54,13 @@ internal static class Configuration
+ "setting exists to provide.")
.ValidateOnStart();
services.AddRelayOptions();
return services;
}
/// <summary>The relay's options, which carry more rules than any other section. See ADR 0004.</summary>
private static void AddRelayOptions(this IServiceCollection services) =>
services.AddOptions<RelayOptions>()
.BindConfiguration(RelayOptions.SectionName)
.ValidateDataAnnotations()
@@ -63,11 +70,28 @@ internal static class Configuration
.Validate(
options => options.IdleTimeout < options.MaxSessionDuration,
"Relay:IdleTimeout must be shorter than Relay:MaxSessionDuration.")
.Validate(
options => options.TicketLifetime > TimeSpan.Zero
&& options.TicketLifetime <= TimeSpan.FromSeconds(30),
"Relay:TicketLifetime must be positive and at most 30 seconds. A ticket is a bearer "
+ "credential in a header; ADR 0004 bounds how long one may be worth stealing.")
.Validate(
options => options.ConnectTimeout > TimeSpan.Zero && options.DrainTimeout >= TimeSpan.Zero,
"Relay:ConnectTimeout must be positive and Relay:DrainTimeout must not be negative.")
.Validate(
options => Features.Relay.RelayAddressPolicy.AreNetworks(options.AllowedNetworks)
&& Features.Relay.RelayAddressPolicy.AreNetworks(options.DeniedNetworks),
"Relay:AllowedNetworks and Relay:DeniedNetworks must hold CIDR networks such as "
+ "10.20.0.0/16, with no host bits set. An entry this server cannot read would be a gap "
+ "the operator believes is closed.")
.Validate(
options => !options.Enabled
|| (Uri.TryCreate(options.WebSocketUrl, UriKind.Absolute, out var url)
&& (string.Equals(url.Scheme, Uri.UriSchemeWs, StringComparison.Ordinal)
|| string.Equals(url.Scheme, Uri.UriSchemeWss, StringComparison.Ordinal))),
"Relay:WebSocketUrl must be an absolute ws:// or wss:// URL.")
.ValidateOnStart();
return services;
}
/// <summary>
/// Warns about configuration that is individually valid but dangerous in combination.
/// </summary>
+34 -7
View File
@@ -121,17 +121,44 @@ public sealed class RelayOptions
/// the ACL: only hosts in a vault the caller holds Connect on can be dialled at all.
/// </para>
/// <para>
/// <b>Nothing reads this setting yet, and that is the point of saying so here.</b> The relay is
/// an ADR and a contract; there is no endpoint and no dialler. This option exists so the shape
/// is settled before the code is, which makes it exactly the kind of comment that gets read as a
/// description of an implemented control. It is not one. Whoever builds the dialler owes ADR 0004
/// §3 in full — loopback, link-local, <c>0.0.0.0/8</c>, multicast and the cloud metadata addresses
/// denied with no configuration escape, and IPv4-mapped IPv6 normalised and re-checked — and this
/// flag governs RFC1918 only, never that list.
/// Governs the private ranges only — RFC1918, carrier-grade NAT (which is where a Tailnet lives) and
/// IPv6 unique-local — and never the list <c>RelayAddressPolicy</c> refuses with no configuration
/// escape: loopback, link-local, <c>0.0.0.0/8</c>, multicast and the cloud metadata addresses.
/// Nothing here can re-open those, which is what ADR 0004 §3 means by non-overridable.
/// </para>
/// </remarks>
public bool AllowPrivateNetworks { get; set; } = true;
/// <summary>
/// Networks, in CIDR form, that a relay target must fall inside. Empty means any network the other
/// rules permit.
/// </summary>
/// <remarks>
/// When set, this is the whole answer for the ranges it names: a target inside one is permitted even
/// with <see cref="AllowPrivateNetworks"/> off, because "only 10.20.0.0/16" is the precise thing an
/// operator means by setting both. It still cannot reach the non-overridable list, and
/// <see cref="DeniedNetworks"/> still wins over it.
/// </remarks>
public IList<string> AllowedNetworks { get; } = [];
/// <summary>Networks, in CIDR form, that a relay target may never fall inside.</summary>
/// <remarks>
/// Checked before <see cref="AllowedNetworks"/>, so a deny inside an allow is a hole in it — the way
/// round that lets an operator carve the database subnet out of an otherwise permitted range.
/// </remarks>
public IList<string> DeniedNetworks { get; } = [];
/// <summary>
/// A directory to keep the ticket key ring in. Unset means the platform default.
/// </summary>
/// <remarks>
/// Losing the key ring is harmless in the way losing <c>Sync:CursorSigningKey</c> is: a ticket lives
/// thirty seconds, so a restart invalidates at most the connects in flight and the client asks
/// again. What does need it is more than one node — every node must be able to read every other's
/// tickets for round-robin to work, so a multi-node deployment points this at shared storage.
/// </remarks>
public string? KeyRingDirectory { get; set; }
/// <summary>Maximum concurrent sessions per user.</summary>
[Range(1, 1000)]
public int MaxConcurrentSessionsPerUser { get; set; } = 10;
@@ -1,6 +1,7 @@
using DodoSSH.Api.Features.Events;
using DodoSSH.Api.Features.Identity;
using DodoSSH.Api.Features.Meta;
using DodoSSH.Api.Features.Relay;
using DodoSSH.Api.Features.Sync;
using DodoSSH.Api.Features.Teams;
using DodoSSH.Contracts;
@@ -45,6 +46,8 @@ internal static class EndpointRegistration
typeof(SyncPullEndpoint),
typeof(SyncPushEndpoint),
typeof(VaultEventsEndpoint),
typeof(RelayTicketEndpoint),
typeof(RelayConnectEndpoint),
typeof(CreateTeamEndpoint),
typeof(ListTeamsEndpoint),
typeof(UpdateTeamEndpoint),
@@ -65,7 +68,6 @@ internal static class EndpointRegistration
// Registered as each feature lands:
// Identity — key rotation, passphrase change
// Vaults — per-item ACLs
// Relay — tickets and the WebSocket
// Audit, Admin
});
+59
View File
@@ -0,0 +1,59 @@
using DodoSSH.Api.Features.Relay;
using Microsoft.AspNetCore.DataProtection;
using Microsoft.AspNetCore.DataProtection.KeyManagement;
using Microsoft.AspNetCore.DataProtection.Repositories;
using Microsoft.Extensions.Options;
namespace DodoSSH.Api.Setup;
/// <summary>The relay's services. See ADR 0004.</summary>
internal static class RelaySetup
{
/// <summary>
/// How much longer than the drain the host waits before it stops regardless.
/// </summary>
/// <remarks>
/// Room for the close handshakes and the audit rows' final writes once the drain has run out, which
/// is the part of a shutdown an operator would otherwise lose without noticing.
/// </remarks>
private static readonly TimeSpan ShutdownMargin = TimeSpan.FromSeconds(10);
internal static IServiceCollection AddDodoRelay(this IServiceCollection services)
{
// Singletons: a session outlives the request that opened it, a protector is safe to share, and
// the network seam holds no state.
services.AddSingleton<RelayTicketCodec>();
services.AddSingleton<RelaySessionRegistry>();
services.AddSingleton<IRelayNetwork, SystemRelayNetwork>();
// The application name isolates this deployment's payloads from any other ASP.NET application
// sharing a key directory, and — unlike the default, which is derived from the content root —
// is the same on every node, which is what lets any node verify any node's ticket.
services.AddDataProtection().SetApplicationName("DodoSSH");
services.AddOptions<KeyManagementOptions>()
.Configure<IOptions<RelayOptions>, ILoggerFactory>((keys, relay, loggers) =>
{
if (relay.Value.KeyRingDirectory is { Length: > 0 } directory)
{
keys.XmlRepository = new FileSystemXmlRepository(new DirectoryInfo(directory), loggers);
}
});
// Longer than the drain, or the host would stop waiting for live sessions before the drain ADR 0004
// promises them had run. Harmless with the relay off: nothing else holds a request open that long,
// and the host stops as soon as the last request has.
services.AddOptions<HostOptions>()
.Configure<IOptions<RelayOptions>>((host, relay) =>
{
var wanted = relay.Value.DrainTimeout + ShutdownMargin;
if (wanted > host.ShutdownTimeout)
{
host.ShutdownTimeout = wanted;
}
});
return services;
}
}
@@ -1199,19 +1199,18 @@
port into plaintext columns the server can read, which is the single deliberate concession in
the design — see ADR 0004 — so it says so rather than being a switch labelled "relay".
◆ AND IT SAYS, FIRST, THAT IT BUYS NOTHING TODAY. The relay's server half is built and this
client has no path to it — the connect path dials the address directly whether this is ticked or
not — so the box as it stood spent that concession and delivered nothing for it. Left tickable
rather than disabled, so a host already carrying the flag can lose it. See
It also says the two things a user would otherwise find out by failing: the server, not this
phone, has to be able to reach the address, and a relayed host does not fall back to dialling
directly, so it connects only while the server is reachable. See
docs/reaching-a-host-you-cannot-dial.md.
-->
<CheckBox IsChecked="{Binding EditorRelayEnabled}" MinHeight="44">
<TextBlock Classes="mono" FontSize="11.5"
Text="Reach this host through the server relay (not wired up yet)"
Text="Reach this host through the server relay"
TextWrapping="Wrap" />
</CheckBox>
<TextBlock Classes="body"
Text="Not built yet: this app always dials the host itself, so ticking this stores the address on the server and changes nothing about how the host is reached. When it works, the relay will dial on your behalf — which is why the address and port have to be stored in the clear. Everything else about the host stays encrypted either way, and a relayed host needs a port of its own rather than its group's." />
Text="The server dials this host on your behalf, for a machine this phone cannot reach — so the server has to be able to reach it. That is why the address and port are stored on the server in the clear; everything else about the host stays encrypted. A relayed host needs a port of its own rather than its group's, and connects only while you are online." />
<!--
This page covers the whole screen while it is open, which is why it needs a Status line of its
+31 -1
View File
@@ -259,6 +259,22 @@ public interface ISyncApi
CancellationToken cancellationToken);
}
/// <summary>Asks the server for leave to relay one connection to one host.</summary>
/// <remarks>
/// The request names a host and nothing else. Where it is, and whether this caller may reach it, is the
/// server's decision, made from its own copy of the host's relay address — which is what keeps the relay
/// from being an open proxy that a client points wherever it likes. See ADR 0004.
/// </remarks>
public interface IRelayApi
{
/// <summary>Issues a single-use ticket, good for thirty seconds, for one relay socket to this host.</summary>
/// <exception cref="DodoSshApiException">
/// The server refused: <c>relay-unavailable</c>, <c>relay-target-rejected</c> or
/// <c>relay-limit-reached</c>, each with its own remedy.
/// </exception>
Task<RelayTicketResponse> IssueRelayTicketAsync(Guid hostId, CancellationToken cancellationToken);
}
/// <summary>
/// The typed client for one DodoSSH server.
/// </summary>
@@ -275,7 +291,7 @@ public interface ISyncApi
/// </para>
/// </remarks>
public sealed class DodoSshApiClient(HttpClient http, IAccessTokenProvider tokens)
: IAccountApi, ISyncApi, ITeamApi, IDirectoryApi, IVaultGrantApi
: IAccountApi, ISyncApi, ITeamApi, IDirectoryApi, IVaultGrantApi, IRelayApi
{
private const string MetaPath = "/api/v1/meta";
private const string ConfigurationPath = "/.well-known/dodossh-configuration";
@@ -310,6 +326,20 @@ public sealed class DodoSshApiClient(HttpClient http, IAccessTokenProvider token
DodoSshJsonContext.Default.DodoSshConfiguration,
cancellationToken);
/// <inheritdoc />
/// <remarks>
/// Never cached, and never fetched ahead of need: a ticket is spent by the socket it opens and is
/// worthless thirty seconds after issue, so the only right moment to ask for one is the connect that
/// will use it.
/// </remarks>
public Task<RelayTicketResponse> IssueRelayTicketAsync(Guid hostId, CancellationToken cancellationToken) =>
SendAsync(
HttpMethod.Post,
RelayProtocol.TicketsPath,
JsonContent.Create(new RelayTicketRequest(hostId, PortForwardId: null), DodoSshJsonContext.Default.RelayTicketRequest),
DodoSshJsonContext.Default.RelayTicketResponse,
cancellationToken);
/// <summary>
/// Reads the caller's profile, unlock material and reachable vaults.
/// </summary>
+9 -16
View File
@@ -636,16 +636,15 @@
<!--
◆ THE RELAY CARD, restyled to the mock's nested-card shape — radius 12, a checkbox with the
title beside it rather than under it — but NOT to the mock's copy. The sentence stays
exactly what it was, "(not wired up yet)" included: the design's own wording implies a relay
this client can dial, and it cannot. See the remark under the hint below, which is unchanged
and still the reason this box says what it says.
title beside it rather than under it. The "(not wired up yet)" this title carried while the
client had no path to the relay is gone, because it has one: a relayed host is connected
through a ticket, a WebSocket and a loopback bridge. See docs/reaching-a-host-you-cannot-dial.md.
-->
<Border CornerRadius="12" Background="{StaticResource Field}"
BorderBrush="{StaticResource Border}" BorderThickness="1" Padding="12">
<StackPanel Spacing="6">
<CheckBox IsChecked="{Binding EditorRelayEnabled}" VerticalContentAlignment="Center">
<TextBlock Text="Connect through the server relay (not wired up yet)" FontSize="13"
<TextBlock Text="Connect through the server relay" FontSize="13"
FontWeight="SemiBold" Foreground="{StaticResource Text}"
TextWrapping="Wrap" />
</CheckBox>
@@ -655,19 +654,13 @@
to resolve the target, or it becomes an authenticated open proxy into the operator's
network.
◆ AND IT SAYS, FIRST, THAT TICKING IT BUYS NOTHING TODAY. The server half of the relay
is built and this client has no path to it: VaultViewModel dials the address directly
whether this is ticked or not. So the box as it stood spent the one deliberate privacy
concession in the design — the address in plaintext columns — and delivered a
connection that failed exactly as it had before, with nothing saying why.
Left tickable rather than disabled, because a host that already carries the flag has to
be able to lose it, and a disabled control would trap the concession on. See
docs/reaching-a-host-you-cannot-dial.md, which is the plan that makes this sentence
removable.
It also says the two things a user would otherwise find out by failing: the server,
not this machine, has to be able to reach the address — the relay is no help against a
deployment outside the target's network — and a relayed host does not fall back to
dialling directly, so it connects only while the server is reachable.
-->
<TextBlock Classes="hint" FontSize="11.5"
Text="Not built yet: this client always dials the host itself, so ticking this stores the address on the server and changes nothing about how the host is reached. When it does work, the relay will dial on your behalf — which is why the address and port have to be stored in plain text. Everything else about the host stays encrypted either way." />
Text="The server dials this host on your behalf, for a machine this computer cannot reach — so the server has to be able to reach it. That is why the address and port are stored on the server in plain text; everything else about the host stays encrypted. A relayed host connects only while you are online." />
</StackPanel>
</Border>
@@ -36,6 +36,8 @@
<ItemGroup>
<InternalsVisibleTo Include="DodoSSH.Client.Session.Tests" />
<!-- The shell suite drives a real ServerRelay over an in-process socket through its test constructor. -->
<InternalsVisibleTo Include="DodoSSH.Client.App.Tests" />
</ItemGroup>
</Project>
@@ -161,6 +161,15 @@ public interface IVaultServer : IDisposable
/// </remarks>
IVaultEventStream Events { get; }
/// <summary>
/// Reaches a host through the server, for a host set to connect that way.
/// </summary>
/// <remarks>
/// Never null, for the reason <see cref="Events"/> is not: a server without a relay supplies
/// <see cref="UnavailableServerRelay"/>, which refuses in words. See ADR 0004.
/// </remarks>
IServerRelay Relay { get; }
/// <summary>Obtains the identity provider's signature over a key statement.</summary>
IKeyBindingAuthorizer KeyBinding { get; }
@@ -226,6 +235,12 @@ public sealed class ServerConnection : IVaultServer
Events = meta.Features.Contains(VaultEvents.Feature, StringComparer.Ordinal)
? new VaultEventStream(serverUrl, tokens, clock)
: IdleVaultEventStream.Instance;
// Decided the same way and for the same reason. Configuration.Relay says the same thing twice;
// meta is what the server says it does, and is what Events is decided from.
Relay = meta.Features.Contains(RelayProtocol.Feature, StringComparer.Ordinal)
? new ServerRelay(api)
: UnavailableServerRelay.Instance;
}
/// <summary>The server this is connected to.</summary>
@@ -261,6 +276,9 @@ public sealed class ServerConnection : IVaultServer
/// <inheritdoc />
public IVaultEventStream Events { get; }
/// <inheritdoc />
public IServerRelay Relay { get; }
/// <inheritdoc />
public IKeyBindingAuthorizer KeyBinding => Oidc;
+262
View File
@@ -0,0 +1,262 @@
using System.Net;
using System.Net.WebSockets;
using DodoSSH.Client.Api;
using DodoSSH.Client.Ssh;
using DodoSSH.Contracts;
namespace DodoSSH.Client.Session;
/// <summary>
/// Reaches a host through the server, for a host this machine cannot dial itself.
/// </summary>
/// <remarks>
/// Always present on <see cref="IVaultServer"/>, like <see cref="IVaultServer.Events"/>: a server that does
/// not relay supplies <see cref="UnavailableServerRelay"/>, whose answer is a sentence saying so. That keeps
/// the connect path on one shape, and it means the refusal a user reads is the same whether the server said
/// no before being asked or when it was.
/// </remarks>
public interface IServerRelay
{
/// <summary>Whether this server advertises a relay at all.</summary>
bool IsOffered { get; }
/// <summary>
/// Opens a relay to one host, ready for a connection to be made through it.
/// </summary>
/// <remarks>
/// Asks for a ticket, opens the socket and starts the bridge, in that order and at once — the ticket is
/// spent by the socket and dead thirty seconds after issue, so this is called by the connect that will
/// use it and never ahead of one.
/// </remarks>
/// <exception cref="ServerRelayException">The relay could not be opened; the message says why.</exception>
Task<RelayRoute> OpenAsync(Guid hostId, CancellationToken cancellationToken);
}
/// <summary>
/// An open relay, one connection long.
/// </summary>
/// <remarks>
/// Handing <see cref="Proxy"/> to a connection hands the bridge over with it, so the connection's own
/// disposal is what closes the relay. <see cref="DisposeAsync"/> is for a caller that ends up not
/// connecting; after a hand-over it does nothing, because the bridge's disposal is idempotent.
/// </remarks>
public sealed class RelayRoute : IAsyncDisposable
{
private readonly RelayBridge bridge;
internal RelayRoute(RelayBridge bridge) => this.bridge = bridge;
/// <summary>The loopback proxy to connect through, which owns the relay from then on.</summary>
public SshLoopbackProxy Proxy => bridge.Proxy;
/// <summary>
/// What the relay said when it closed, in words a user can act on; null when it has not closed, or
/// closed with nothing to say.
/// </summary>
/// <remarks>
/// For a connect that failed. SSH.NET reports a relay that hung up as a stream that ended — which reads
/// exactly like the direct connection that failed and sent the user to the relay — so the relay's own
/// reason, when it gave one, replaces it.
/// </remarks>
public string? Failure => bridge.CloseStatus is { } status ? Describe((int)status) : null;
/// <inheritdoc />
public ValueTask DisposeAsync() => bridge.DisposeAsync();
private static string? Describe(int code) => code switch
{
RelayProtocol.TargetUnreachableCloseCode =>
"The server relay could not reach this host. The server dials it on your behalf, so the "
+ "address has to be one the server can reach — check it from the server's side.",
RelayProtocol.TooManySessionsCloseCode =>
"The server relay has as many sessions open as it allows. Close one and try again.",
(int)WebSocketCloseStatus.EndpointUnavailable =>
"The server relay is restarting. Try again in a moment.",
RelayProtocol.IdleTimeoutCloseCode => "The server relay closed the connection for being idle.",
RelayProtocol.DurationLimitCloseCode =>
"The server relay closed the connection: it reached the longest session the server allows.",
_ => null,
};
}
/// <summary>A relay that could not be opened, with the reason in its message.</summary>
/// <remarks>
/// Its own type so the connect path can tell a relay refusal from a connection failure. The first has an
/// exact reason already, and running the reachability probe over it would dial the direct address — the one
/// the user chose the relay because they cannot reach — and append a finding about the wrong path.
/// </remarks>
public sealed class ServerRelayException : Exception
{
/// <summary>A refusal with nothing underneath it.</summary>
public ServerRelayException(string message)
: base(message)
{
}
/// <summary>A refusal and what caused it.</summary>
public ServerRelayException(string message, Exception innerException)
: base(message, innerException)
{
}
}
/// <summary>The relay of a server that does not offer one.</summary>
public sealed class UnavailableServerRelay : IServerRelay
{
private UnavailableServerRelay()
{
}
/// <summary>The one instance.</summary>
public static UnavailableServerRelay Instance { get; } = new();
/// <inheritdoc />
public bool IsOffered => false;
/// <inheritdoc />
public Task<RelayRoute> OpenAsync(Guid hostId, CancellationToken cancellationToken) =>
Task.FromException<RelayRoute>(new ServerRelayException(ServerRelay.NotOffered));
}
/// <summary>
/// The relay of a server that offers one: a ticket from the API, a socket, and a bridge.
/// </summary>
/// <remarks>
/// See ADR 0004 for the protocol and <c>docs/reaching-a-host-you-cannot-dial.md</c> for why the bridge
/// speaks SOCKS5. Every way this can fail is turned into a sentence here, because the codes are the
/// server's and the one place that should know what each means is the one that asked.
/// </remarks>
public sealed class ServerRelay : IServerRelay
{
/// <summary>What a user is told when the server they are signed in to does not relay.</summary>
internal const string NotOffered =
"This host is set to connect through the server relay, and this server does not offer one. "
+ "Untick “Connect through the server relay” in the host's settings to dial it directly.";
private readonly IRelayApi api;
private readonly Func<Uri, IReadOnlyList<string>, CancellationToken, Task<WebSocket>> connect;
/// <summary>A relay reached through this API client.</summary>
public ServerRelay(IRelayApi api)
: this(api, DialAsync)
{
}
/// <summary>A relay whose socket is opened by <paramref name="connect"/>, for tests.</summary>
internal ServerRelay(
IRelayApi api,
Func<Uri, IReadOnlyList<string>, CancellationToken, Task<WebSocket>> connect)
{
this.api = api;
this.connect = connect;
}
/// <inheritdoc />
public bool IsOffered => true;
/// <inheritdoc />
public async Task<RelayRoute> OpenAsync(Guid hostId, CancellationToken cancellationToken)
{
RelayTicketResponse ticket;
try
{
ticket = await api.IssueRelayTicketAsync(hostId, cancellationToken).ConfigureAwait(false);
}
catch (DodoSshApiException exception)
{
throw new ServerRelayException(Describe(exception), exception);
}
catch (HttpRequestException exception)
{
throw new ServerRelayException(
$"The server could not be reached to open the relay: {exception.Message}",
exception);
}
// The ticket as a second subprotocol, which is how ADR 0004 carries it: a header every WebSocket
// client can set. The server echoes the first alone.
var socket = await connect(
ticket.WebSocketUrl,
[RelayProtocol.SubProtocol, RelayProtocol.TicketProtocolPrefix + ticket.Ticket],
cancellationToken)
.ConfigureAwait(false);
return new RelayRoute(RelayBridge.Start(socket));
}
private static string Describe(DodoSshApiException exception) => exception.Code switch
{
ProblemCodes.RelayUnavailable => NotOffered,
// Deliberately unspecific on the server's side, so that it cannot be used to map the operator's
// network. The likeliest causes from here are the two this machine can check.
ProblemCodes.RelayTargetRejected =>
"The server refused to relay to this host. Check that the change turning the relay on has "
+ "synchronised, and that the address is one the server is allowed to reach; the server's log "
+ "records which.",
ProblemCodes.RelayLimitReached =>
"You have as many relay sessions open as the server allows. Close one and try again.",
_ => $"The server would not open the relay: {exception.Message}",
};
/// <summary>Opens the relay socket, and translates a refused upgrade into its reason.</summary>
/// <remarks>
/// The status of a refused upgrade is collected, which a constrained client could not do and this one
/// can: a 401 is a ticket the relay would not take — spent, expired, or issued by a node whose key ring
/// this one does not share — and asking again is the remedy.
/// </remarks>
private static async Task<WebSocket> DialAsync(
Uri url,
IReadOnlyList<string> protocols,
CancellationToken cancellationToken)
{
var socket = new ClientWebSocket();
try
{
foreach (var protocol in protocols)
{
socket.Options.AddSubProtocol(protocol);
}
socket.Options.CollectHttpResponseDetails = true;
await socket.ConnectAsync(url, cancellationToken).ConfigureAwait(false);
}
catch (WebSocketException exception)
{
var status = socket.HttpStatusCode;
socket.Dispose();
throw new ServerRelayException(
status == HttpStatusCode.Unauthorized
? "The server relay refused its ticket. Try connecting again."
: $"The server relay could not be reached: {exception.Message}",
exception);
}
catch
{
socket.Dispose();
throw;
}
if (!string.Equals(socket.SubProtocol, RelayProtocol.SubProtocol, StringComparison.Ordinal))
{
socket.Abort();
socket.Dispose();
throw new ServerRelayException(
"The server relay answered in a protocol this app does not speak. Update the app or the server.");
}
return socket;
}
}
@@ -1795,25 +1795,9 @@ internal sealed partial class TransfersViewModel : ObservableObject, IAsyncDispo
{
await CloseSessionAsync().ConfigureAwait(true);
ISftpSession opened;
try
if (await OpenHostSessionAsync(open, row, request, cancellationToken).ConfigureAwait(true)
is not { } opened)
{
opened = await sftp.OpenSftpAsync(request, cancellationToken).ConfigureAwait(true);
}
catch (SshHostKeyUnknownException exception)
{
// First contact, decided here rather than inherited from a terminal. File transfer is
// its own connection, so it makes its own trust decision — and the pin it writes is the
// same pin a shell would then find.
PendingHostKey = exception.Presentation;
Status = "This host has not been seen before.";
return;
}
catch (SshHostKeyMismatchException exception)
{
HostKeyMismatch = exception.Message;
Status = "The host key has changed. Nothing was connected.";
return;
}
@@ -1831,6 +1815,55 @@ internal sealed partial class TransfersViewModel : ObservableObject, IAsyncDispo
}).ConfigureAwait(true);
}
/// <summary>
/// Opens the session on a host, through the relay when the host uses it, or says why not.
/// </summary>
/// <returns>The session, or null with <see cref="Status"/> saying why there is none.</returns>
private async Task<ISftpSession?> OpenHostSessionAsync(
VaultViewModel open,
HostRowViewModel row,
SshConnectionRequest request,
CancellationToken cancellationToken)
{
RelayRoute? relay = null;
try
{
// Through the relay for a host set to use it, exactly as a shell to it would be. The
// session owns the relay from here, on success and on failure alike.
relay = row.Host.RelayEnabled
? await open.OpenRelayAsync(row.Label, row.EntityId, cancellationToken).ConfigureAwait(true)
: null;
return await sftp
.OpenSftpAsync(relay is null ? request : request with { Proxy = relay.Proxy }, cancellationToken)
.ConfigureAwait(true);
}
catch (Exception exception)
when (relay?.Failure is { } failure
&& exception is not (OperationCanceledException or SshHostKeyUnknownException or SshHostKeyMismatchException))
{
// See VaultViewModel.OpenThroughAsync: the relay's reason is the true one.
Status = failure;
return null;
}
catch (SshHostKeyUnknownException exception)
{
// First contact, decided here rather than inherited from a terminal. File transfer is
// its own connection, so it makes its own trust decision — and the pin it writes is the
// same pin a shell would then find.
PendingHostKey = exception.Presentation;
Status = "This host has not been seen before.";
return null;
}
catch (SshHostKeyMismatchException exception)
{
HostKeyMismatch = exception.Message;
Status = "The host key has changed. Nothing was connected.";
return null;
}
}
/// <summary>
/// Records that a host connection just succeeded: the address, the negotiated facts, and the ticket the
/// disconnect log closes out later.
@@ -11671,7 +11671,10 @@ internal sealed partial class VaultViewModel(
PhaseReporter reporter,
CancellationToken cancellationToken)
{
if (reporter.Furthest != SshConnectionPhase.Reaching)
// A relay refusal already says exactly what happened, and a relayed host is reached by a path the
// probe does not take: it would dial the host directly — the route the user chose the relay because
// they cannot use — and report on that instead.
if (reporter.Furthest != SshConnectionPhase.Reaching || exception is ServerRelayException || target.Relayed)
{
return exception.Message;
}
@@ -11776,9 +11779,7 @@ internal sealed partial class VaultViewModel(
authentication.Username,
authentication.Credential);
var sessionId = await workspace
.OpenSessionAsync(request, TerminalSize.Default, reporter, cancellationToken)
.ConfigureAwait(true);
var sessionId = await OpenThroughAsync(target, request, reporter, cancellationToken).ConfigureAwait(true);
// The workspace has already opened a ticket for this session, with the address and the moment it
// connected. What it could not know is which keychain item this was — an SshConnectionRequest has no
@@ -11823,6 +11824,90 @@ internal sealed partial class VaultViewModel(
}
}
/// <summary>
/// Opens a relay to a host set to connect through the server, or says why there is none.
/// </summary>
/// <remarks>
/// <para>
/// <b>There is no falling back to a direct connection.</b> A user who ticked the relay did so because
/// the direct path does not work, or goes somewhere they do not mean — and dialling it anyway on a
/// failure would be the old defect in reverse: the box would say one thing and the connection do
/// another. So a relay that cannot be opened is the failure, stated as such.
/// </para>
/// <para>
/// The server is resolved the way a synchronisation resolves it, which comes back online quietly when a
/// sign-in is remembered. Offline is its own sentence, because it is the one case the user can fix from
/// this machine without touching the host.
/// </para>
/// <para>
/// Internal because file transfer opens a connection of its own and a relayed host needs the relay
/// there too — or the transfers screen would fail for exactly the hosts the relay exists for.
/// </para>
/// </remarks>
internal async Task<RelayRoute> OpenRelayAsync(string label, Guid hostId, CancellationToken cancellationToken)
{
if (await ResolveServerAsync(cancellationToken).ConfigureAwait(true) is not { } server)
{
throw new ServerRelayException(
$"{label} connects through the server relay, and this machine is not connected to the "
+ "server right now. Try again once it is back online.");
}
return await server.Relay.OpenAsync(hostId, cancellationToken).ConfigureAwait(true);
}
/// <summary>
/// Opens the session, through the relay when there is one, and puts the relay's reason on a failure.
/// </summary>
/// <remarks>
/// <para>
/// A relay that hung up reaches SSH.NET as a stream that ended, and it reports that the way it reports
/// every lost transport — a sentence that reads exactly like the direct failure the relay was ticked to
/// get around. When the relay said why it closed, that sentence is the true one and replaces it. The
/// host key refusals are left alone: a key was presented, so the relay did its job.
/// </para>
/// <para>
/// The relay is disposed on every failure. The connection owns it once it is handed over and disposes
/// it on its own failure paths too, but a throw before the factory is reached would otherwise leave a
/// bridge listening until its own timeout; disposing twice is safe.
/// </para>
/// </remarks>
private async Task<uint> OpenThroughAsync(
ConnectionTarget target,
SshConnectionRequest request,
PhaseReporter reporter,
CancellationToken cancellationToken)
{
// After the renderer rather than before it: a ticket is dead thirty seconds after issue, and the
// renderer's wait is the one step of a connect with no bound a user would recognise.
var relay = target.Relayed
? await OpenRelayAsync(target.Label, target.HostId!.Value, cancellationToken).ConfigureAwait(true)
: null;
try
{
return await workspace
.OpenSessionAsync(
relay is null ? request : request with { Proxy = relay.Proxy },
TerminalSize.Default,
reporter,
cancellationToken)
.ConfigureAwait(true);
}
catch (Exception exception) when (relay is not null)
{
await relay.DisposeAsync().ConfigureAwait(true);
if (relay.Failure is { } failure
&& exception is not (OperationCanceledException or SshHostKeyUnknownException or SshHostKeyMismatchException))
{
throw new ServerRelayException(failure, exception);
}
throw;
}
}
/// <summary>
/// Turns the password that just worked into a keychain credential bound to this host.
/// </summary>
@@ -12041,6 +12126,15 @@ internal sealed partial class VaultViewModel(
{
/// <summary>The keychain item, or null for a machine that is not in it.</summary>
internal Guid? HostId => Row?.EntityId;
/// <summary>
/// Whether this is reached through the server relay rather than dialled.
/// </summary>
/// <remarks>
/// Only a keychain host can be: the relay dials the address the server holds for an item, and a
/// machine typed into the manual box has no item and therefore nothing on the server to dial.
/// </remarks>
internal bool Relayed => Row?.Host.RelayEnabled is true;
}
/// <summary>
+369
View File
@@ -0,0 +1,369 @@
using System.Buffers;
using System.Net;
using System.Net.Sockets;
using System.Net.WebSockets;
namespace DodoSSH.Client.Ssh;
/// <summary>
/// A loopback SOCKS5 proxy that carries exactly one connection over one relay WebSocket.
/// </summary>
/// <remarks>
/// <para>
/// SSH.NET cannot be handed a stream, only an address — so the relay, which is a WebSocket, is reached by
/// giving SSH.NET an address that leads into one. This listens on <c>127.0.0.1</c> at a port the system
/// chooses, answers one SOCKS5 CONNECT, and from then on moves bytes: what SSH.NET writes goes out as binary
/// messages, and what the relay sends comes back as the stream SSH.NET reads. That is ADR 0004's loopback
/// bridge, and step 3 of <c>docs/reaching-a-host-you-cannot-dial.md</c>.
/// </para>
/// <para>
/// <b>SOCKS5 rather than a plain pipe, so the host stays the host.</b> The connection made through this is
/// told the target's real name and port, and pins its host key under them; the CONNECT carrying that name
/// arrives here and is answered without being acted on, because where the bytes go was decided by the
/// server when it issued the ticket, and nothing this side says can change it.
/// </para>
/// <para>
/// Three properties are the security of this, and each is held by a test. It binds loopback and nothing
/// else — asserted on the bound address, because a bridge on every interface would be a proxy into the
/// relay for anyone on the network and nothing would report it. It accepts once and stops listening, so a
/// second process on this machine cannot take a turn. And it gives up if nobody arrives within
/// <see cref="AcceptTimeout"/>, so a connect that failed before dialling cannot leave it waiting.
/// </para>
/// <para>
/// <b>No operation on the WebSocket is cancelled.</b> Cancelling one aborts a .NET WebSocket, and an aborted
/// one never reads the close frame that says why the relay ended — which is the one thing the user needs to
/// be told when the server could not reach the host. The bridge stops a direction by closing, and waits a
/// bounded time for the other side to agree, as the relay's own endpoint does.
/// </para>
/// </remarks>
public sealed class RelayBridge : IAsyncDisposable
{
/// <summary>How long the bridge waits for its one connection before closing the relay unused.</summary>
/// <remarks>
/// The connection is made the moment the bridge exists, so this bounds a caller that failed between the
/// two. Longer than any SSH connect timeout, so it never ends a connect that is merely slow.
/// </remarks>
public static readonly TimeSpan AcceptTimeout = TimeSpan.FromSeconds(60);
private const int BufferBytes = 32 * 1024;
private static readonly TimeSpan CloseTimeout = TimeSpan.FromSeconds(5);
private readonly TcpListener listener;
private readonly WebSocket socket;
private readonly CancellationTokenSource stopping = new();
private readonly Task running;
private int disposed;
private RelayBridge(WebSocket socket, TimeSpan acceptTimeout)
{
this.socket = socket;
// The address rather than "localhost", so there is no resolver and no hosts file between this and
// the interface it means. See the remarks on SshLoopbackProxy.
listener = new TcpListener(IPAddress.Loopback, 0);
listener.Start(backlog: 1);
Endpoint = (IPEndPoint)listener.LocalEndpoint;
running = RunAsync(acceptTimeout);
}
/// <summary>Where the bridge is listening. Loopback, always.</summary>
public IPEndPoint Endpoint { get; }
/// <summary>The proxy to hand the connection, which then owns this bridge.</summary>
public SshLoopbackProxy Proxy => new(Endpoint.Port, this);
/// <summary>
/// How the relay closed the socket, once it has; null while it is open or when it vanished without
/// saying.
/// </summary>
/// <remarks>
/// Read after a connect through the bridge fails. The relay says in a close code what went wrong on its
/// side — the server could not reach the host, or has too many sessions open — and that is the sentence
/// the user needs in place of the transport error SSH.NET reports for a stream that ended.
/// </remarks>
public WebSocketCloseStatus? CloseStatus { get; private set; }
/// <summary>The relay's own words about why it closed, alongside <see cref="CloseStatus"/>.</summary>
public string? CloseDescription { get; private set; }
/// <summary>Completes when the bridge has finished: its one connection is over, or nobody came.</summary>
public Task Completion => running;
/// <summary>Starts a bridge over an open relay socket, which it then owns.</summary>
public static RelayBridge Start(WebSocket socket) => Start(socket, AcceptTimeout);
/// <summary>Starts a bridge with a shorter wait for its connection, for tests.</summary>
internal static RelayBridge Start(WebSocket socket, TimeSpan acceptTimeout)
{
ArgumentNullException.ThrowIfNull(socket);
return new RelayBridge(socket, acceptTimeout);
}
/// <inheritdoc />
/// <remarks>
/// Idempotent, because two owners may reach it: the connection that was handed the proxy, and a caller
/// tidying up after a connect that never got that far.
/// </remarks>
public async ValueTask DisposeAsync()
{
if (Interlocked.Exchange(ref disposed, 1) != 0)
{
return;
}
await stopping.CancelAsync().ConfigureAwait(false);
listener.Stop();
if (!await SettlesAsync(running, CloseTimeout * 2).ConfigureAwait(false))
{
socket.Abort();
await SettlesAsync(running, CloseTimeout).ConfigureAwait(false);
}
socket.Dispose();
stopping.Dispose();
}
private async Task RunAsync(TimeSpan acceptTimeout)
{
TcpClient? client = null;
try
{
using (var accepting = CancellationTokenSource.CreateLinkedTokenSource(stopping.Token))
{
accepting.CancelAfter(acceptTimeout);
client = await listener.AcceptTcpClientAsync(accepting.Token).ConfigureAwait(false);
}
// Once. A second connection to this port is somebody else's, and there is nothing for it here.
listener.Stop();
client.NoDelay = true;
var stream = client.GetStream();
if (await Socks5.AcceptConnectAsync(stream, stopping.Token).ConfigureAwait(false))
{
await PipeAsync(stream).ConfigureAwait(false);
}
}
catch (Exception exception)
when (exception is OperationCanceledException or SocketException or IOException or ObjectDisposedException)
{
// Nobody came, the bridge was disposed, or the connection went away mid-handshake. The relay is
// closed below either way.
}
finally
{
listener.Stop();
client?.Dispose();
await CloseSocketAsync(WebSocketCloseStatus.NormalClosure).ConfigureAwait(false);
}
}
/// <summary>Moves bytes both ways until either side ends, then lets the other finish.</summary>
private async Task PipeAsync(NetworkStream stream)
{
var upstream = TcpToRelayAsync(stream);
var downstream = RelayToTcpAsync(stream);
var first = await Task.WhenAny(upstream, downstream).ConfigureAwait(false);
if (first == upstream)
{
// SSH.NET closed its end. Say so to the relay, then give it a moment to answer, which is what
// ends the receive in progress without aborting it.
await CloseSocketAsync(WebSocketCloseStatus.NormalClosure).ConfigureAwait(false);
if (!await SettlesAsync(downstream, CloseTimeout).ConfigureAwait(false))
{
socket.Abort();
}
}
else
{
// The relay ended. Closing the TCP side is how SSH.NET hears it, as the end of its stream.
stream.Socket.Shutdown(SocketShutdown.Both);
await SettlesAsync(upstream, CloseTimeout).ConfigureAwait(false);
}
}
private async Task TcpToRelayAsync(NetworkStream stream)
{
var buffer = ArrayPool<byte>.Shared.Rent(BufferBytes);
try
{
int read;
while ((read = await stream.ReadAsync(buffer.AsMemory(0, BufferBytes), stopping.Token).ConfigureAwait(false)) > 0)
{
// Not cancellable, deliberately. See the remarks on this class.
await socket
.SendAsync(buffer.AsMemory(0, read), WebSocketMessageType.Binary, endOfMessage: true, CancellationToken.None)
.ConfigureAwait(false);
}
}
catch (Exception exception)
when (exception is OperationCanceledException or IOException or WebSocketException or ObjectDisposedException)
{
// Either end went away. PipeAsync decides what that means.
}
finally
{
ArrayPool<byte>.Shared.Return(buffer);
}
}
private async Task RelayToTcpAsync(NetworkStream stream)
{
var buffer = ArrayPool<byte>.Shared.Rent(BufferBytes);
try
{
while (true)
{
// Not cancellable, deliberately. See the remarks on this class.
var received = await socket.ReceiveAsync(buffer.AsMemory(), CancellationToken.None)
.ConfigureAwait(false);
if (received.MessageType == WebSocketMessageType.Close)
{
CloseStatus = socket.CloseStatus;
CloseDescription = socket.CloseStatusDescription;
return;
}
await stream.WriteAsync(buffer.AsMemory(0, received.Count), stopping.Token).ConfigureAwait(false);
}
}
catch (Exception exception)
when (exception is OperationCanceledException or IOException or WebSocketException or ObjectDisposedException)
{
// Either end went away. PipeAsync decides what that means.
}
finally
{
ArrayPool<byte>.Shared.Return(buffer);
}
}
/// <summary>Sends a close, if one can still be sent, bounded so a vanished peer cannot hold it.</summary>
private async Task CloseSocketAsync(WebSocketCloseStatus status)
{
if (socket.State is not (WebSocketState.Open or WebSocketState.CloseReceived))
{
return;
}
using var timeout = new CancellationTokenSource(CloseTimeout);
try
{
await socket.CloseOutputAsync(status, string.Empty, timeout.Token).ConfigureAwait(false);
}
catch (Exception exception)
when (exception is OperationCanceledException or WebSocketException or ObjectDisposedException)
{
// Gone already.
}
}
private static async Task<bool> SettlesAsync(Task task, TimeSpan within)
{
try
{
await task.WaitAsync(within).ConfigureAwait(false);
return true;
}
catch (TimeoutException)
{
return false;
}
}
/// <summary>
/// The server half of SOCKS5 (RFC 1928), as much of it as SSH.NET speaks: no authentication, CONNECT.
/// </summary>
private static class Socks5
{
private const byte Version = 5;
private const byte NoAuthentication = 0;
private const byte NoAcceptableMethod = 0xFF;
private const byte Connect = 1;
private const byte Succeeded = 0;
private const byte CommandNotSupported = 7;
private const byte AddressTypeNotSupported = 8;
/// <summary>
/// Answers the greeting and one CONNECT, whatever it names.
/// </summary>
/// <returns>Whether the stream is now a tunnel; false when the client asked for something else.</returns>
/// <remarks>
/// The target in the request is read and discarded. It is the host the connection is <em>about</em>,
/// which is why SSH.NET sends it, and it is not where the bytes go, which the relay decided. The bound
/// address in the reply is all zeroes: nothing on the far side of a relay has an address the client
/// could use, and SSH.NET does not read it.
/// </remarks>
internal static async Task<bool> AcceptConnectAsync(Stream stream, CancellationToken cancellationToken)
{
var header = await ReadAsync(stream, 2, cancellationToken).ConfigureAwait(false);
if (header[0] != Version)
{
return false;
}
var methods = await ReadAsync(stream, header[1], cancellationToken).ConfigureAwait(false);
if (!methods.Contains(NoAuthentication))
{
await stream.WriteAsync(new byte[] { Version, NoAcceptableMethod }, cancellationToken).ConfigureAwait(false);
return false;
}
await stream.WriteAsync(new byte[] { Version, NoAuthentication }, cancellationToken).ConfigureAwait(false);
var request = await ReadAsync(stream, 4, cancellationToken).ConfigureAwait(false);
if (request[0] != Version || request[1] != Connect)
{
await ReplyAsync(stream, CommandNotSupported, cancellationToken).ConfigureAwait(false);
return false;
}
var addressLength = request[3] switch
{
1 => 4,
4 => 16,
3 => (await ReadAsync(stream, 1, cancellationToken).ConfigureAwait(false))[0],
_ => -1,
};
if (addressLength < 0)
{
await ReplyAsync(stream, AddressTypeNotSupported, cancellationToken).ConfigureAwait(false);
return false;
}
// The address, then the port. Neither is used; see the remarks above.
await ReadAsync(stream, addressLength + 2, cancellationToken).ConfigureAwait(false);
await ReplyAsync(stream, Succeeded, cancellationToken).ConfigureAwait(false);
return true;
}
private static Task ReplyAsync(Stream stream, byte status, CancellationToken cancellationToken) =>
stream.WriteAsync(new byte[] { Version, status, 0, 1, 0, 0, 0, 0, 0, 0 }, cancellationToken).AsTask();
private static async Task<byte[]> ReadAsync(Stream stream, int count, CancellationToken cancellationToken)
{
var buffer = new byte[count];
await stream.ReadExactlyAsync(buffer, cancellationToken).ConfigureAwait(false);
return buffer;
}
}
}
+11 -4
View File
@@ -38,12 +38,19 @@ public sealed record SshPrivateKeyCredential(byte[] PrivateKeyPem, string? Passp
/// stand-in. See the gate in <c>SshNetConnectionFactory</c>, which pins what this request names.
/// </para>
/// <para>
/// Nothing in this assembly opens one. The proxy is somebody else's — a forward on a bastion connection,
/// or the relay bridge — and its lifetime belongs to whoever opened it, which must outlast the connection
/// made through it. See <c>docs/reaching-a-host-you-cannot-dial.md</c>.
/// <b>Whoever opens a proxy may hand it to the connection made through it.</b> A proxy with no
/// <see cref="Lifetime"/> belongs to its opener, which must keep it listening for as long as the connection
/// lives. One with a lifetime is given away: the connection disposes it after itself, and a connect that
/// fails disposes it before throwing — so a proxy opened for one connection, which is what the relay bridge
/// is, cannot outlive it on any of the paths a session ends by. See
/// <c>docs/reaching-a-host-you-cannot-dial.md</c>.
/// </para>
/// </remarks>
public sealed record SshLoopbackProxy(int Port);
/// <param name="Port">The loopback port the proxy listens on.</param>
/// <param name="Lifetime">
/// What keeps the proxy listening, when the connection is to own it; null when its opener does.
/// </param>
public sealed record SshLoopbackProxy(int Port, IAsyncDisposable? Lifetime = null);
/// <summary>Everything needed to reach one host.</summary>
/// <param name="Host">Hostname or address.</param>
@@ -75,12 +75,22 @@ public sealed class SshNetConnectionFactory(IKnownHostStore knownHosts)
{
ArgumentNullException.ThrowIfNull(request);
var client = new SshClient(BuildConnectionInfo(request)) { KeepAliveInterval = KeepAliveInterval };
var lifetime = request.Proxy?.Lifetime;
var gate = await ConnectThroughHostKeyGateAsync(client, request, progress, cancellationToken)
.ConfigureAwait(false);
try
{
var client = new SshClient(BuildConnectionInfo(request)) { KeepAliveInterval = KeepAliveInterval };
return new SshNetConnection(client, gate.Presented!);
var gate = await ConnectThroughHostKeyGateAsync(client, request, progress, cancellationToken)
.ConfigureAwait(false);
return new SshNetConnection(client, gate.Presented!, lifetime);
}
catch
{
await DisposeAsync(lifetime).ConfigureAwait(false);
throw;
}
}
/// <inheritdoc />
@@ -95,6 +105,40 @@ public sealed class SshNetConnectionFactory(IKnownHostStore knownHosts)
{
ArgumentNullException.ThrowIfNull(request);
var lifetime = request.Proxy?.Lifetime;
try
{
return await OpenSftpCoreAsync(request, lifetime, cancellationToken).ConfigureAwait(false);
}
catch
{
await DisposeAsync(lifetime).ConfigureAwait(false);
throw;
}
}
/// <summary>
/// Disposes a proxy the connection was given, when the connection it was given for did not happen.
/// </summary>
/// <remarks>
/// Every failure path, including the ones before a socket exists — an unreadable key throws from
/// <see cref="BuildConnectionInfo"/> — because a relay bridge left listening after a failed connect is a
/// loopback SOCKS proxy with an open relay session behind it, and nothing would ever close it.
/// </remarks>
private static async ValueTask DisposeAsync(IAsyncDisposable? lifetime)
{
if (lifetime is not null)
{
await lifetime.DisposeAsync().ConfigureAwait(false);
}
}
private async Task<ISftpSession> OpenSftpCoreAsync(
SshConnectionRequest request,
IAsyncDisposable? lifetime,
CancellationToken cancellationToken)
{
// Keep-alive here for the reason it is on the shell client: a transfer connection left open while
// somebody browses is idle in exactly the same way, and a file listing against a machine that has
// since rebooted would otherwise wait on a socket nothing will ever answer.
@@ -125,7 +169,7 @@ public sealed class SshNetConnectionFactory(IKnownHostStore knownHosts)
throw;
}
return new SshNetSftpSession(client, gate.Presented!, home);
return new SshNetSftpSession(client, gate.Presented!, home, lifetime);
}
/// <summary>
@@ -326,7 +370,15 @@ public sealed class SshNetConnectionFactory(IKnownHostStore knownHosts)
}
/// <summary>An SSH.NET-backed connection.</summary>
internal sealed class SshNetConnection(SshClient client, HostKeyPresentation hostKey) : ISshConnection
/// <remarks>
/// Owns the proxy it was reached through when it was given one, and disposes it after the client — the
/// order that lets SSH.NET close its end of the loopback socket before the thing on the other end goes.
/// </remarks>
internal sealed class SshNetConnection(
SshClient client,
HostKeyPresentation hostKey,
IAsyncDisposable? proxyLifetime = null)
: ISshConnection
{
/// <inheritdoc />
public bool IsConnected => client.IsConnected;
@@ -366,10 +418,14 @@ internal sealed class SshNetConnection(SshClient client, HostKeyPresentation hos
}
/// <inheritdoc />
public ValueTask DisposeAsync()
public async ValueTask DisposeAsync()
{
client.Dispose();
return ValueTask.CompletedTask;
if (proxyLifetime is not null)
{
await proxyLifetime.DisposeAsync().ConfigureAwait(false);
}
}
}
+12 -2
View File
@@ -14,13 +14,23 @@ namespace DodoSSH.Client.Ssh;
internal sealed class SshNetSftpSession : ISftpSession
{
private readonly SftpClient client;
private readonly IAsyncDisposable? proxyLifetime;
private string? transportFailure;
private int announced;
internal SshNetSftpSession(SftpClient client, HostKeyPresentation hostKey, string homeDirectory)
/// <param name="client">The connected client.</param>
/// <param name="hostKey">What the server presented.</param>
/// <param name="homeDirectory">Where the session starts.</param>
/// <param name="proxyLifetime">The proxy this session was reached through, when it owns it.</param>
internal SshNetSftpSession(
SftpClient client,
HostKeyPresentation hostKey,
string homeDirectory,
IAsyncDisposable? proxyLifetime = null)
{
this.client = client;
this.proxyLifetime = proxyLifetime;
HostKey = hostKey;
HomeDirectory = homeDirectory;
@@ -243,7 +253,7 @@ internal sealed class SshNetSftpSession : ISftpSession
client.Dispose();
return ValueTask.CompletedTask;
return proxyLifetime?.DisposeAsync() ?? ValueTask.CompletedTask;
}
/// <remarks>
+11
View File
@@ -76,6 +76,17 @@ public static class ProblemCodes
/// </remarks>
public const string EventsUnavailable = "events-unavailable";
/// <summary>
/// This deployment does not relay.
/// </summary>
/// <remarks>
/// Its own code rather than <see cref="RelayTargetRejected"/>, because it is the one relay refusal
/// whose remedy is not about the host: nothing the user changes on this machine will make it work
/// here, and saying so is not an oracle for anything. <c>GET /api/v1/meta</c> says the same thing
/// without asking.
/// </remarks>
public const string RelayUnavailable = "relay-unavailable";
/// <summary>The relay refused the requested target. Never states why, to avoid a probe oracle.</summary>
public const string RelayTargetRejected = "relay-target-rejected";
@@ -966,3 +966,14 @@ static DodoSSH.Contracts.VaultKeyWrap.operator !=(DodoSSH.Contracts.VaultKeyWrap
static DodoSSH.Contracts.VaultKeyWrap.operator ==(DodoSSH.Contracts.VaultKeyWrap? left, DodoSSH.Contracts.VaultKeyWrap? right) -> bool
static DodoSSH.Contracts.VaultSummary.operator !=(DodoSSH.Contracts.VaultSummary? left, DodoSSH.Contracts.VaultSummary? right) -> bool
static DodoSSH.Contracts.VaultSummary.operator ==(DodoSSH.Contracts.VaultSummary? left, DodoSSH.Contracts.VaultSummary? right) -> bool
const DodoSSH.Contracts.ProblemCodes.RelayUnavailable = "relay-unavailable" -> string!
const DodoSSH.Contracts.RelayProtocol.ConnectPath = "/api/v1/relay/connect" -> string!
const DodoSSH.Contracts.RelayProtocol.DurationLimitCloseCode = 4409 -> int
const DodoSSH.Contracts.RelayProtocol.Feature = "relay" -> string!
const DodoSSH.Contracts.RelayProtocol.IdleTimeoutCloseCode = 4408 -> int
const DodoSSH.Contracts.RelayProtocol.SubProtocol = "dodossh.relay.v1" -> string!
const DodoSSH.Contracts.RelayProtocol.TargetUnreachableCloseCode = 4502 -> int
const DodoSSH.Contracts.RelayProtocol.TicketProtocolPrefix = "ticket." -> string!
const DodoSSH.Contracts.RelayProtocol.TicketsPath = "/api/v1/relay/tickets" -> string!
const DodoSSH.Contracts.RelayProtocol.TooManySessionsCloseCode = 4429 -> int
DodoSSH.Contracts.RelayProtocol
+77 -2
View File
@@ -1,5 +1,78 @@
namespace DodoSSH.Contracts;
/// <summary>
/// The relay's routes, subprotocol and close codes.
/// </summary>
/// <remarks>
/// <para>
/// Two steps, per ADR 0004. <see cref="TicketsPath"/> is an ordinary authenticated request that decides
/// everything — which host, which addresses, whether the caller may — and answers with a ticket.
/// <see cref="ConnectPath"/> is a WebSocket that carries no API authority at all: it accepts the ticket
/// and nothing else, and pipes raw bytes to the addresses the ticket names. There is no framing of the
/// relay's own inside the socket. SSH is already a byte stream, so every binary message is simply the
/// next bytes of it, in order, and message boundaries mean nothing.
/// </para>
/// <para>
/// A refusal the client could not have avoided by asking differently — a limit, a target that did not
/// answer — is sent as a close code after the upgrade rather than as an HTTP status before it, for the
/// same reason as <see cref="VaultEvents"/>: a constrained WebSocket client cannot read the status of a
/// failed upgrade, and these are exactly the cases where the user needs to be told something specific.
/// </para>
/// </remarks>
public static class RelayProtocol
{
/// <summary>The <c>/api/v1/meta</c> feature flag advertising that this server relays at all.</summary>
public const string Feature = "relay";
/// <summary>Where a ticket is asked for, with a <see cref="RelayTicketRequest"/>.</summary>
public const string TicketsPath = "/api/v1/relay/tickets";
/// <summary>The path the relay socket is served from.</summary>
/// <remarks>
/// A client dials <see cref="RelayTicketResponse.WebSocketUrl"/> rather than building this itself,
/// because a deployment may publish the relay under a different host than the API.
/// </remarks>
public const string ConnectPath = "/api/v1/relay/connect";
/// <summary>The only subprotocol this version speaks.</summary>
public const string SubProtocol = "dodossh.relay.v1";
/// <summary>
/// The prefix of the second subprotocol an upgrade offers, which carries the ticket.
/// </summary>
/// <remarks>
/// The header rather than the query string, because a URL is what proxies, access logs and traces
/// keep. The server echoes <see cref="SubProtocol"/> alone and never the ticket.
/// </remarks>
public const string TicketProtocolPrefix = "ticket.";
/// <summary>
/// Close code for a caller, or a node, already holding as many relay sessions as it may.
/// </summary>
/// <remarks>
/// The same number as <see cref="VaultEvents.TooManyConnectionsCloseCode"/>, deliberately, for the
/// same meaning. Retrying at once is what caused it.
/// </remarks>
public const int TooManySessionsCloseCode = 4429;
/// <summary>
/// Close code for a target that did not accept the connection.
/// </summary>
/// <remarks>
/// The one failure a user needs told in so many words, because from the laptop it looks exactly like
/// the direct connection that failed and sent them to the relay in the first place. It says that the
/// <em>server</em> could not reach the machine, which is a different problem with a different owner.
/// </remarks>
public const int TargetUnreachableCloseCode = 4502;
/// <summary>Close code for a session that carried no bytes for the idle timeout.</summary>
public const int IdleTimeoutCloseCode = 4408;
/// <summary>Close code for a session that reached the maximum lifetime this server allows.</summary>
public const int DurationLimitCloseCode = 4409;
}
/// <summary>
/// A request for permission to relay to a host.
/// </summary>
@@ -11,8 +84,10 @@ namespace DodoSSH.Contracts;
/// </remarks>
/// <param name="HostId">The host to reach.</param>
/// <param name="PortForwardId">
/// A configured port forward to use instead of the host's own SSH port. Resolved server-side the
/// same way.
/// A configured port forward to use instead of the host's own SSH port. Reserved, and refused when
/// set: a port forward lives inside the host's encrypted payload, so the server has no plaintext port
/// to resolve it to, and taking one from the client is the open proxy this record exists to prevent.
/// Serving it needs a plaintext concession of its own, argued the way ADR 0004 argues the host's.
/// </param>
public sealed record RelayTicketRequest(
Guid HostId,
+123
View File
@@ -0,0 +1,123 @@
namespace DodoSSH.Domain;
/// <summary>
/// A relay ticket that has been redeemed.
/// </summary>
/// <remarks>
/// <para>
/// The replay guard, and the only state the relay shares between nodes — see ADR 0004. A ticket is
/// verified without the database; what the database adds is that it is verified <em>once</em>, by a
/// unique insert on <see cref="TicketId"/> that a second redemption loses whichever node it reaches.
/// </para>
/// <para>
/// A row is worth keeping only while its ticket could still verify, so <see cref="ExpiresAtUtc"/> is
/// stored to let redemption sweep the rows that can no longer matter. Nothing else reads this table,
/// and it carries nothing about the target: the audit record is <see cref="RelaySession"/>.
/// </para>
/// </remarks>
public sealed class RelayTicketUse
{
/// <summary>The ticket's own id. Primary key, and the whole of the guard.</summary>
public Guid TicketId { get; set; }
/// <summary>When the ticket stops verifying, after which this row guards nothing.</summary>
public DateTimeOffset ExpiresAtUtc { get; set; }
/// <summary>When it was redeemed.</summary>
public DateTimeOffset UsedAtUtc { get; set; }
}
/// <summary>
/// The audit record of one relay session.
/// </summary>
/// <remarks>
/// <para>
/// What ADR 0004 says remains auditable in relay mode, and nothing more: who, to where, for how long,
/// how many bytes each way and why it ended. The relay forwards SSH ciphertext, so there is no content
/// to record and no column that pretends otherwise.
/// </para>
/// <para>
/// Written twice — once when the target answers, once when the session ends — each time from a scope
/// opened for that write alone. A session lasts hours and a scope held across it is a pooled database
/// connection held across it, which is the failure ADR 0004 records.
/// </para>
/// <para>
/// No foreign keys to the host or the vault, deliberately. An audit row has to outlive what it is
/// about: a host deleted a minute after a session is exactly the host an operator will want the
/// session for.
/// </para>
/// </remarks>
public sealed class RelaySession
{
/// <summary>The session id, minted with the ticket and returned to the client. Primary key.</summary>
public Guid Id { get; set; }
/// <summary>Who opened it.</summary>
public Guid UserId { get; set; }
/// <summary>The host it reached.</summary>
public Guid HostId { get; set; }
/// <summary>The vault the host was in when the ticket was issued.</summary>
public Guid VaultId { get; set; }
/// <summary>The hostname as the host row carried it at issue.</summary>
public string TargetHost { get; set; } = string.Empty;
/// <summary>The address actually dialled, which is the one that answered.</summary>
/// <remarks>Null only for a session whose target never answered.</remarks>
public string? TargetAddress { get; set; }
/// <summary>The port dialled.</summary>
public int TargetPort { get; set; }
/// <summary>The client's address as this node saw it, for the audit ADR 0004 promises.</summary>
public string? ClientAddress { get; set; }
/// <summary>When the socket was accepted.</summary>
public DateTimeOffset StartedAtUtc { get; set; }
/// <summary>When it closed, or null while it is open — or if the node died under it.</summary>
public DateTimeOffset? EndedAtUtc { get; set; }
/// <summary>Bytes forwarded from the client to the target.</summary>
public long BytesClientToTarget { get; set; }
/// <summary>Bytes forwarded from the target to the client.</summary>
public long BytesTargetToClient { get; set; }
/// <summary>Why it ended. <see cref="RelayCloseReason.Unspecified"/> while open.</summary>
public RelayCloseReason CloseReason { get; set; }
}
/// <summary>Why a relay session ended.</summary>
/// <remarks>
/// Numbered as <c>RelaySessionCloseReason</c> on the wire is, and held to it by a test rather than by
/// hope: the endpoint converts between the two by value.
/// </remarks>
public enum RelayCloseReason
{
/// <summary>Not a legal value for a closed session; what an open one carries.</summary>
Unspecified = 0,
/// <summary>The client closed the connection.</summary>
ClientClosed = 1,
/// <summary>The target closed the connection.</summary>
TargetClosed = 2,
/// <summary>No traffic within the idle timeout.</summary>
IdleTimeout = 3,
/// <summary>The maximum session duration was reached.</summary>
DurationLimit = 4,
/// <summary>The server is shutting down and drained the session.</summary>
ServerShutdown = 5,
/// <summary>The connection to the target failed.</summary>
TargetUnreachable = 6,
/// <summary>An error ended the session.</summary>
Error = 7,
}
@@ -0,0 +1,61 @@
using DodoSSH.Domain;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Metadata.Builders;
namespace DodoSSH.Infrastructure.Configurations;
/// <summary>Maps <see cref="RelayTicketUse"/>.</summary>
public sealed class RelayTicketUseConfiguration : IEntityTypeConfiguration<RelayTicketUse>
{
/// <inheritdoc />
public void Configure(EntityTypeBuilder<RelayTicketUse> builder)
{
ArgumentNullException.ThrowIfNull(builder);
builder.ToTable("relay_ticket_use");
// The key is the guard. A second redemption of the same ticket is a unique violation, on whichever
// node it reaches, and nothing weaker — a lookup followed by an insert — would be one.
builder.HasKey(u => u.TicketId);
builder.Property(u => u.TicketId).ValueGeneratedNever();
// For the sweep, which deletes by expiry.
builder.HasIndex(u => u.ExpiresAtUtc);
}
}
/// <summary>Maps <see cref="RelaySession"/>.</summary>
public sealed class RelaySessionConfiguration : IEntityTypeConfiguration<RelaySession>
{
/// <inheritdoc />
public void Configure(EntityTypeBuilder<RelaySession> builder)
{
ArgumentNullException.ThrowIfNull(builder);
builder.ToTable("relay_session");
builder.HasKey(s => s.Id);
// Minted with the ticket and handed to the client before the row exists, so the client can name
// the session it opened.
builder.Property(s => s.Id).ValueGeneratedNever();
builder.Property(s => s.TargetHost).HasMaxLength(255).IsRequired();
// An address's text form: 45 characters is the longest IPv6 spelling with an embedded IPv4.
builder.Property(s => s.TargetAddress).HasMaxLength(45);
builder.Property(s => s.ClientAddress).HasMaxLength(45);
builder.Property(s => s.CloseReason).HasConversion<int>();
builder.HasIndex(s => new { s.UserId, s.StartedAtUtc });
builder.HasIndex(s => new { s.HostId, s.StartedAtUtc });
builder.ToTable(t => t.HasCheckConstraint(
"ck_relay_session_port_range",
"target_port BETWEEN 1 AND 65535"));
builder.ToTable(t => t.HasCheckConstraint(
"ck_relay_session_bytes",
"bytes_client_to_target >= 0 AND bytes_target_to_client >= 0"));
}
}
@@ -87,6 +87,12 @@ public class DodoDbContext(DbContextOptions<DodoDbContext> options) : DbContext(
/// <summary>Applied-operation receipts, for exactly-once retries.</summary>
public DbSet<SyncOperationReceipt> SyncOperationReceipts => Set<SyncOperationReceipt>();
/// <summary>Redeemed relay tickets: the replay guard ADR 0004 shares between nodes.</summary>
public DbSet<RelayTicketUse> RelayTicketUses => Set<RelayTicketUse>();
/// <summary>The relay's audit record, one row per session.</summary>
public DbSet<RelaySession> RelaySessions => Set<RelaySession>();
/// <inheritdoc />
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,92 @@
using System;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace DodoSSH.Infrastructure.Migrations
{
/// <summary>
/// Adds the relay's two tables: the replay guard for its tickets, and its audit record.
/// </summary>
/// <remarks>
/// Both are new and empty, so this is additive and safe to roll back. Neither table references a
/// host or a vault by foreign key — an audit row has to outlive what it is about — and neither holds
/// anything the host row did not already hold in plaintext for a relay-enabled host. See ADR 0004.
/// </remarks>
public partial class AddRelaySessions : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable(
name: "relay_session",
schema: "dodo",
columns: table => new
{
id = table.Column<Guid>(type: "uuid", nullable: false),
user_id = table.Column<Guid>(type: "uuid", nullable: false),
host_id = table.Column<Guid>(type: "uuid", nullable: false),
vault_id = table.Column<Guid>(type: "uuid", nullable: false),
target_host = table.Column<string>(type: "character varying(255)", maxLength: 255, nullable: false),
target_address = table.Column<string>(type: "character varying(45)", maxLength: 45, nullable: true),
target_port = table.Column<int>(type: "integer", nullable: false),
client_address = table.Column<string>(type: "character varying(45)", maxLength: 45, nullable: true),
started_at_utc = table.Column<DateTimeOffset>(type: "timestamp with time zone", nullable: false),
ended_at_utc = table.Column<DateTimeOffset>(type: "timestamp with time zone", nullable: true),
bytes_client_to_target = table.Column<long>(type: "bigint", nullable: false),
bytes_target_to_client = table.Column<long>(type: "bigint", nullable: false),
close_reason = table.Column<int>(type: "integer", nullable: false)
},
constraints: table =>
{
table.PrimaryKey("pk_relay_session", x => x.id);
table.CheckConstraint("ck_relay_session_bytes", "bytes_client_to_target >= 0 AND bytes_target_to_client >= 0");
table.CheckConstraint("ck_relay_session_port_range", "target_port BETWEEN 1 AND 65535");
});
migrationBuilder.CreateTable(
name: "relay_ticket_use",
schema: "dodo",
columns: table => new
{
ticket_id = table.Column<Guid>(type: "uuid", nullable: false),
expires_at_utc = table.Column<DateTimeOffset>(type: "timestamp with time zone", nullable: false),
used_at_utc = table.Column<DateTimeOffset>(type: "timestamp with time zone", nullable: false)
},
constraints: table =>
{
table.PrimaryKey("pk_relay_ticket_use", x => x.ticket_id);
});
migrationBuilder.CreateIndex(
name: "ix_relay_session_host_id_started_at_utc",
schema: "dodo",
table: "relay_session",
columns: new[] { "host_id", "started_at_utc" });
migrationBuilder.CreateIndex(
name: "ix_relay_session_user_id_started_at_utc",
schema: "dodo",
table: "relay_session",
columns: new[] { "user_id", "started_at_utc" });
migrationBuilder.CreateIndex(
name: "ix_relay_ticket_use_expires_at_utc",
schema: "dodo",
table: "relay_ticket_use",
column: "expires_at_utc");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(
name: "relay_session",
schema: "dodo");
migrationBuilder.DropTable(
name: "relay_ticket_use",
schema: "dodo");
}
}
}
@@ -135,6 +135,104 @@ namespace DodoSSH.Infrastructure.Migrations
b.ToTable("key_log", "dodo");
});
modelBuilder.Entity("DodoSSH.Domain.RelaySession", b =>
{
b.Property<Guid>("Id")
.HasColumnType("uuid")
.HasColumnName("id");
b.Property<long>("BytesClientToTarget")
.HasColumnType("bigint")
.HasColumnName("bytes_client_to_target");
b.Property<long>("BytesTargetToClient")
.HasColumnType("bigint")
.HasColumnName("bytes_target_to_client");
b.Property<string>("ClientAddress")
.HasMaxLength(45)
.HasColumnType("character varying(45)")
.HasColumnName("client_address");
b.Property<int>("CloseReason")
.HasColumnType("integer")
.HasColumnName("close_reason");
b.Property<DateTimeOffset?>("EndedAtUtc")
.HasColumnType("timestamp with time zone")
.HasColumnName("ended_at_utc");
b.Property<Guid>("HostId")
.HasColumnType("uuid")
.HasColumnName("host_id");
b.Property<DateTimeOffset>("StartedAtUtc")
.HasColumnType("timestamp with time zone")
.HasColumnName("started_at_utc");
b.Property<string>("TargetAddress")
.HasMaxLength(45)
.HasColumnType("character varying(45)")
.HasColumnName("target_address");
b.Property<string>("TargetHost")
.IsRequired()
.HasMaxLength(255)
.HasColumnType("character varying(255)")
.HasColumnName("target_host");
b.Property<int>("TargetPort")
.HasColumnType("integer")
.HasColumnName("target_port");
b.Property<Guid>("UserId")
.HasColumnType("uuid")
.HasColumnName("user_id");
b.Property<Guid>("VaultId")
.HasColumnType("uuid")
.HasColumnName("vault_id");
b.HasKey("Id")
.HasName("pk_relay_session");
b.HasIndex("HostId", "StartedAtUtc")
.HasDatabaseName("ix_relay_session_host_id_started_at_utc");
b.HasIndex("UserId", "StartedAtUtc")
.HasDatabaseName("ix_relay_session_user_id_started_at_utc");
b.ToTable("relay_session", "dodo", t =>
{
t.HasCheckConstraint("ck_relay_session_bytes", "bytes_client_to_target >= 0 AND bytes_target_to_client >= 0");
t.HasCheckConstraint("ck_relay_session_port_range", "target_port BETWEEN 1 AND 65535");
});
});
modelBuilder.Entity("DodoSSH.Domain.RelayTicketUse", b =>
{
b.Property<Guid>("TicketId")
.HasColumnType("uuid")
.HasColumnName("ticket_id");
b.Property<DateTimeOffset>("ExpiresAtUtc")
.HasColumnType("timestamp with time zone")
.HasColumnName("expires_at_utc");
b.Property<DateTimeOffset>("UsedAtUtc")
.HasColumnType("timestamp with time zone")
.HasColumnName("used_at_utc");
b.HasKey("TicketId")
.HasName("pk_relay_ticket_use");
b.HasIndex("ExpiresAtUtc")
.HasDatabaseName("ix_relay_ticket_use_expires_at_utc");
b.ToTable("relay_ticket_use", "dodo");
});
modelBuilder.Entity("DodoSSH.Domain.SshHost", b =>
{
b.Property<Guid>("Id")
+25 -1
View File
@@ -1,9 +1,11 @@
using System.Net.Http.Headers;
using System.Net.WebSockets;
using DodoSSH.Api.Features.Relay;
using DodoSSH.Contracts;
using DodoSSH.Infrastructure;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.AspNetCore.TestHost;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
@@ -70,7 +72,17 @@ public sealed class ApiFixture : WebApplicationFactory<Program>, IAsyncLifetime
// The stub serves plaintext HTTP on a loopback port.
["DODOSSH_Oidc__RequireHttpsMetadata"] = "false",
["DODOSSH_Relay__Enabled"] = "false",
// On, so the relay's endpoints can be tested at all. Nothing dials a real network: the seam
// below answers every resolve and connect, and the address policy still judges what it says.
["DODOSSH_Relay__Enabled"] = "true",
["DODOSSH_Relay__WebSocketUrl"] = "ws://localhost" + RelayProtocol.ConnectPath,
// Small, so a test can reach the cap with two sockets rather than ten.
["DODOSSH_Relay__MaxConcurrentSessionsPerUser"] = "2",
// Short enough that a test can watch a session time out, long enough that no test which is
// moving bytes comes near it.
["DODOSSH_Relay__IdleTimeout"] = "00:00:03",
// Fixed so cursors stay valid for the lifetime of the test host.
["DODOSSH_Sync__CursorSigningKey"] = Convert.ToBase64String(new byte[32]),
@@ -80,8 +92,20 @@ public sealed class ApiFixture : WebApplicationFactory<Program>, IAsyncLifetime
{
Environment.SetEnvironmentVariable(name, value);
}
builder.ConfigureTestServices(services => services.AddSingleton<IRelayNetwork>(RelayNetwork));
}
/// <summary>
/// What the relay resolves and dials, in place of the machine's own resolver and sockets.
/// </summary>
/// <remarks>
/// Every address a test machine can reach by itself is loopback, which the relay refuses outright, so
/// a relay test needs a network that pretends. This one maps the addresses a test names onto loopback
/// listeners the test owns, and remembers what it was asked to dial.
/// </remarks>
internal FakeRelayNetwork RelayNetwork { get; } = new();
/// <summary>Creates a client carrying a valid token for the given subject.</summary>
/// <remarks>
/// There is no verified-address knob here, and its absence is worth a sentence. It used to exist so
@@ -62,6 +62,13 @@ public sealed class EndpointInventoryTests(ApiFixture fixture)
// the handshake, unlike the relay's ticket. See ADR 0012.
"GET /api/v1/events name=VaultEvents tags=Events policies=Enrolled anon=False",
// The relay's two halves, and the split between them is the design. The ticket is Enrolled, as sync
// is, and every decision is made there with the full access model; the socket is anonymous because
// the ticket it carries is the whole of its authorization — single-use, thirty seconds, one host —
// and a constrained client cannot put a bearer token on an upgrade. See ADR 0004.
"POST /api/v1/relay/tickets name=IssueRelayTicket tags=Relay policies=Enrolled anon=False",
"GET /api/v1/relay/connect name=RelayConnect tags=Relay policies= anon=True",
// Enrolled, because the answer exists to be wrapped to and a caller with no key of their own has
// nothing to wrap and no signature to attribute it with. There is no search here — see
// DirectoryService for why an exact-match-only directory is a decision rather than a shortcut.
@@ -0,0 +1,68 @@
using System.Collections.Concurrent;
using System.Net;
using System.Net.Sockets;
using DodoSSH.Api.Features.Relay;
namespace DodoSSH.Api.Tests;
/// <summary>
/// A resolver and a dialler that reach only what a test has arranged.
/// </summary>
/// <remarks>
/// Names resolve to what <see cref="Resolve"/> says, and address literals to themselves, as the real
/// resolver does. An address dials the loopback endpoint <see cref="Route"/> mapped it to; anything
/// else is refused the way a closed port is, so a test that forgot to arrange a target sees the relay's
/// real "unreachable" path rather than a hang.
/// </remarks>
internal sealed class FakeRelayNetwork : IRelayNetwork
{
private readonly ConcurrentDictionary<string, IPAddress[]> names = new(StringComparer.OrdinalIgnoreCase);
private readonly ConcurrentDictionary<IPEndPoint, IPEndPoint> routes = new();
private readonly ConcurrentQueue<IPEndPoint> dialled = new();
/// <summary>Every endpoint the relay asked to connect to, in order.</summary>
internal IReadOnlyCollection<IPEndPoint> Dialled => dialled;
internal void Resolve(string name, params IPAddress[] addresses) => names[name] = addresses;
internal void Route(IPEndPoint target, IPEndPoint loopback) => routes[target] = loopback;
/// <inheritdoc />
public Task<IPAddress[]> ResolveAsync(string host, CancellationToken cancellationToken)
{
if (IPAddress.TryParse(host, out var literal))
{
return Task.FromResult<IPAddress[]>([literal]);
}
return names.TryGetValue(host, out var addresses)
? Task.FromResult(addresses)
: Task.FromException<IPAddress[]>(new SocketException((int)SocketError.HostNotFound));
}
/// <inheritdoc />
public async Task<Stream> ConnectAsync(IPAddress address, int port, CancellationToken cancellationToken)
{
var endpoint = new IPEndPoint(address, port);
dialled.Enqueue(endpoint);
if (!routes.TryGetValue(endpoint, out var loopback))
{
throw new SocketException((int)SocketError.ConnectionRefused);
}
var client = new TcpClient();
try
{
await client.ConnectAsync(loopback, cancellationToken);
return client.GetStream();
}
catch
{
client.Dispose();
throw;
}
}
}
@@ -0,0 +1,125 @@
using System.Net;
using DodoSSH.Api.Features.Relay;
using DodoSSH.Api.Setup;
namespace DodoSSH.Api.Tests;
/// <summary>
/// ADR 0004 §3, address by address.
/// </summary>
/// <remarks>
/// The refused list is tested against the most permissive configuration this server accepts — every
/// network allowed, private ranges on — because "no configuration escape" is a claim about exactly that
/// case. A test that ran it only under the defaults would pass for a policy an operator could open up.
/// </remarks>
public sealed class RelayAddressPolicyTests
{
[Theory]
[InlineData("127.0.0.1")]
[InlineData("127.255.255.254")]
[InlineData("0.0.0.0")]
[InlineData("0.1.2.3")]
[InlineData("169.254.169.254")]
[InlineData("169.254.0.1")]
[InlineData("224.0.0.251")]
[InlineData("255.255.255.255")]
[InlineData("100.100.100.200")]
[InlineData("::")]
[InlineData("::1")]
[InlineData("fe80::1")]
[InlineData("ff02::1")]
[InlineData("fd00:ec2::254")]
// The same addresses, spelled as IPv6 — the bypass ADR 0004 names, and its NAT64 cousin.
[InlineData("::ffff:127.0.0.1")]
[InlineData("::ffff:169.254.169.254")]
[InlineData("64:ff9b::a9fe:a9fe")]
[InlineData("64:ff9b::7f00:1")]
[InlineData("::127.0.0.1")]
public void TheRefusedList_HasNoConfigurationEscape(string address)
{
var wideOpen = new RelayOptions { AllowPrivateNetworks = true };
wideOpen.AllowedNetworks.Add("0.0.0.0/0");
wideOpen.AllowedNetworks.Add("::/0");
new RelayAddressPolicy(wideOpen).Permits(IPAddress.Parse(address)).ShouldBeFalse();
new RelayAddressPolicy(new RelayOptions()).Permits(IPAddress.Parse(address)).ShouldBeFalse();
}
[Fact]
public void ALinkLocalAddressWithAScope_IsStillLinkLocal()
{
var scoped = IPAddress.Parse("fe80::1%7");
new RelayAddressPolicy(new RelayOptions()).Permits(scoped).ShouldBeFalse();
}
[Theory]
[InlineData("10.0.0.5")]
[InlineData("172.16.4.4")]
[InlineData("192.168.1.10")]
[InlineData("100.101.102.103")] // A Tailnet address: carrier-grade NAT, beside the refused metadata one.
[InlineData("fd12:3456::1")]
[InlineData("::ffff:10.0.0.5")]
public void PrivateNetworks_ArePermittedByDefault_AndRefusedWhenSwitchedOff(string address)
{
new RelayAddressPolicy(new RelayOptions()).Permits(IPAddress.Parse(address)).ShouldBeTrue();
new RelayAddressPolicy(new RelayOptions { AllowPrivateNetworks = false })
.Permits(IPAddress.Parse(address))
.ShouldBeFalse();
}
[Theory]
[InlineData("203.0.113.10")]
[InlineData("2001:db8::10")]
public void PublicAddresses_ArePermitted(string address) =>
new RelayAddressPolicy(new RelayOptions { AllowPrivateNetworks = false })
.Permits(IPAddress.Parse(address))
.ShouldBeTrue();
[Fact]
public void AnAllowList_IsTheWholeAnswer_EvenWithPrivateNetworksOff()
{
var options = new RelayOptions { AllowPrivateNetworks = false };
options.AllowedNetworks.Add("10.20.0.0/16");
var policy = new RelayAddressPolicy(options);
policy.Permits(IPAddress.Parse("10.20.3.4")).ShouldBeTrue();
policy.Permits(IPAddress.Parse("10.21.3.4")).ShouldBeFalse();
policy.Permits(IPAddress.Parse("203.0.113.10")).ShouldBeFalse();
}
[Fact]
public void ADenial_IsAHoleInAnAllowance()
{
var options = new RelayOptions();
options.AllowedNetworks.Add("10.20.0.0/16");
options.DeniedNetworks.Add("10.20.5.0/24");
var policy = new RelayAddressPolicy(options);
policy.Permits(IPAddress.Parse("10.20.4.1")).ShouldBeTrue();
policy.Permits(IPAddress.Parse("10.20.5.1")).ShouldBeFalse();
}
[Theory]
[InlineData(25, false)]
[InlineData(465, false)]
[InlineData(587, false)]
[InlineData(0, false)]
[InlineData(65536, false)]
[InlineData(22, true)]
[InlineData(5432, true)]
public void Ports_RefuseSmtpAndNothingElse(int port, bool permitted) =>
RelayAddressPolicy.PermitsPort(port).ShouldBe(permitted);
[Theory]
[InlineData("10.0.0.0/8", true)]
[InlineData("fd00::/8", true)]
[InlineData("10.0.0.1/8", false)]
[InlineData("10.0.0.0", false)]
[InlineData("ten dot oh", false)]
public void NetworksAreReadStrictly(string entry, bool readable) =>
RelayAddressPolicy.AreNetworks([entry]).ShouldBe(readable);
}
@@ -0,0 +1,559 @@
using System.Net;
using System.Net.Sockets;
using System.Net.WebSockets;
using System.Text;
using DodoSSH.Contracts;
using DodoSSH.Domain;
using DodoSSH.Infrastructure;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;
namespace DodoSSH.Api.Tests;
/// <summary>
/// The relay, over a real socket, through the real ticket, against a target the test owns.
/// </summary>
/// <remarks>
/// <para>
/// The network is the only thing replaced — see <see cref="FakeRelayNetwork"/> — and it is replaced
/// <em>under</em> the address policy, so every address these tests arrange is still judged by the real
/// rules. That is what lets the refusals here be meaningful: a loopback target is refused because the
/// policy refused it, not because the fake had nothing to route it to, and the tests say which by
/// asserting that nothing was dialled.
/// </para>
/// <para>
/// The refusals share one answer on purpose, and a test holds them to it. A distinct response for "that
/// name resolves internally, but no" is a scanner for the operator's DNS in the hands of anyone who can
/// edit a host.
/// </para>
/// </remarks>
[Collection(ApiCollection.Name)]
public sealed class RelayEndpointTests(ApiFixture fixture)
{
private static readonly DateTimeOffset Now = new(2026, 9, 28, 12, 0, 0, TimeSpan.Zero);
/// <summary>How long a test waits for a frame. Generous, because it bounds a broken build, not a measurement.</summary>
private static readonly TimeSpan FrameTimeout = TimeSpan.FromSeconds(30);
// ---- The ticket ----
[Fact]
public async Task ATicket_NamesTheSocketAndExpiresWithinThirtySeconds()
{
var (subject, hostId) = await SeedRelayHostAsync("10.40.0.1", 22);
var ticket = await IssueAsync(subject, hostId);
ticket.SubProtocol.ShouldBe(RelayProtocol.SubProtocol);
ticket.WebSocketUrl.AbsolutePath.ShouldBe(RelayProtocol.ConnectPath);
ticket.ExpiresAt.ShouldBeLessThanOrEqualTo(TimeProvider.System.GetUtcNow().AddSeconds(31));
// It has to stand in a Sec-WebSocket-Protocol list, so it must be an HTTP token: no padding, no
// slash, no plus — which is what base64url is for.
ticket.Ticket.ShouldAllBe(c => char.IsAsciiLetterOrDigit(c) || c == '-' || c == '_');
}
[Fact]
public async Task EveryTargetRefusal_IsTheSameAnswer()
{
var (subject, relayOff) = await SeedHostAsync(relay: null);
var (_, others) = await SeedRelayHostAsync("10.40.0.2", 22);
var (_, loopback) = await SeedRelayHostAsync("127.0.0.1", 22, subject);
var (_, smtp) = await SeedRelayHostAsync("10.40.0.3", 25, subject);
var (_, unresolvable) = await SeedRelayHostAsync("nowhere.invalid", 22, subject);
var client = fixture.CreateClientFor(subject);
var answers = new List<JsonProblem>();
foreach (var hostId in new[] { relayOff, others, loopback, smtp, unresolvable, Guid.CreateVersion7() })
{
var response = await client.PostContractAsync(RelayProtocol.TicketsPath, new RelayTicketRequest(hostId, null));
response.StatusCode.ShouldBe(HttpStatusCode.Forbidden);
answers.Add((await response.Content.ReadProblemAsync()).ShouldNotBeNull());
}
answers.ShouldAllBe(problem => problem.Code == ProblemCodes.RelayTargetRejected);
answers.Select(problem => problem.Detail).Distinct(StringComparer.Ordinal).Count().ShouldBe(1);
}
[Theory]
[InlineData("127.0.0.1")]
[InlineData("::1")]
[InlineData("::ffff:127.0.0.1")]
[InlineData("169.254.169.254")]
[InlineData("::ffff:169.254.169.254")]
[InlineData("0.0.0.0")]
public async Task AnAddressThePolicyRefuses_IsNeverDialled(string address)
{
var (subject, hostId) = await SeedRelayHostAsync(address, 22);
var dialledBefore = fixture.RelayNetwork.Dialled.Count;
var response = await fixture.CreateClientFor(subject)
.PostContractAsync(RelayProtocol.TicketsPath, new RelayTicketRequest(hostId, null));
response.StatusCode.ShouldBe(HttpStatusCode.Forbidden);
fixture.RelayNetwork.Dialled.Count.ShouldBe(dialledBefore);
}
[Fact]
public async Task ANameResolvingToABadAndAGoodAddress_IsDialledOnlyOnTheGoodOne()
{
// The rebinding shape: one answer the policy likes and one it must never reach. The ticket carries
// only what passed, so the socket has nothing else it could try.
using var target = LoopbackTarget.Echo();
var good = new IPEndPoint(IPAddress.Parse("10.40.1.1"), 22);
fixture.RelayNetwork.Resolve("mixed.test", IPAddress.Parse("169.254.169.254"), good.Address);
fixture.RelayNetwork.Route(good, target.Endpoint);
var (subject, hostId) = await SeedRelayHostAsync("mixed.test", 22);
using var timeout = Timeout();
using var socket = await ConnectAsync(await IssueAsync(subject, hostId), timeout.Token);
await RoundTripAsync(socket, "hello", timeout.Token);
fixture.RelayNetwork.Dialled.ShouldNotContain(endpoint => endpoint.Address.Equals(IPAddress.Parse("169.254.169.254")));
}
[Fact]
public async Task APortForward_IsRefused()
{
// Its port lives in ciphertext, so honouring one would mean taking the port from the client.
var (subject, hostId) = await SeedRelayHostAsync("10.40.0.4", 22);
var response = await fixture.CreateClientFor(subject)
.PostContractAsync(RelayProtocol.TicketsPath, new RelayTicketRequest(hostId, Guid.CreateVersion7()));
response.StatusCode.ShouldBe(HttpStatusCode.Forbidden);
(await response.Content.ReadProblemAsync()).ShouldNotBeNull().Code.ShouldBe(ProblemCodes.RelayTargetRejected);
}
// ---- The socket ----
[Fact]
public async Task BytesGoBothWays_AndTheSessionIsAudited()
{
using var target = LoopbackTarget.Echo();
var (subject, hostId) = await SeedRoutedHostAsync(target, "10.40.2.1");
var ticket = await IssueAsync(subject, hostId);
using var timeout = Timeout();
using (var socket = await ConnectAsync(ticket, timeout.Token))
{
socket.SubProtocol.ShouldBe(RelayProtocol.SubProtocol);
await RoundTripAsync(socket, "SSH-2.0-test", timeout.Token);
await RoundTripAsync(socket, "a second message", timeout.Token);
await socket.CloseAsync(WebSocketCloseStatus.NormalClosure, string.Empty, timeout.Token);
}
var session = await WaitForClosedAuditAsync(ticket.SessionId, timeout.Token);
session.HostId.ShouldBe(hostId);
session.TargetAddress.ShouldBe("10.40.2.1");
session.TargetPort.ShouldBe(22);
session.CloseReason.ShouldBe(RelayCloseReason.ClientClosed);
var sent = Encoding.UTF8.GetByteCount("SSH-2.0-test") + Encoding.UTF8.GetByteCount("a second message");
session.BytesClientToTarget.ShouldBe(sent);
session.BytesTargetToClient.ShouldBe(sent);
}
[Fact]
public async Task ATicket_IsSpentByItsFirstUse()
{
using var target = LoopbackTarget.Echo();
var (subject, hostId) = await SeedRoutedHostAsync(target, "10.40.3.1");
var ticket = await IssueAsync(subject, hostId);
using var timeout = Timeout();
using var first = await ConnectAsync(ticket, timeout.Token);
var replay = await Should.ThrowAsync<InvalidOperationException>(ConnectAsync(ticket, timeout.Token));
replay.Message.ShouldContain("401");
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData("not-a-ticket")]
public async Task AnUpgradeWithoutAGenuineTicket_IsRefused(string? ticket)
{
var client = fixture.Server.CreateWebSocketClient();
client.SubProtocols.Add(RelayProtocol.SubProtocol);
if (ticket is not null)
{
client.SubProtocols.Add(RelayProtocol.TicketProtocolPrefix + ticket);
}
var refused = await Should.ThrowAsync<InvalidOperationException>(client.ConnectAsync(
new Uri(fixture.Server.BaseAddress, RelayProtocol.ConnectPath.TrimStart('/')),
TestContext.Current.CancellationToken));
refused.Message.ShouldContain("401");
}
[Fact]
public async Task APlainGet_SaysItIsAWebSocket()
{
var response = await fixture.CreateClient().GetAsync(
new Uri(RelayProtocol.ConnectPath, UriKind.Relative),
TestContext.Current.CancellationToken);
response.StatusCode.ShouldBe(HttpStatusCode.BadRequest);
(await response.Content.ReadProblemAsync()).ShouldNotBeNull().Code.ShouldBe(ProblemCodes.MalformedRequest);
}
[Fact]
public async Task ATargetThatDoesNotAnswer_ClosesWithItsOwnCode()
{
// Routed nowhere, so the fake refuses the connect the way a closed port does. The user has to be
// told the *server* could not reach the machine, which is not what a direct failure means.
var (subject, hostId) = await SeedRelayHostAsync("10.40.4.1", 22);
var ticket = await IssueAsync(subject, hostId);
using var timeout = Timeout();
using var socket = await ConnectAsync(ticket, timeout.Token);
await ReadUntilCloseAsync(socket, timeout.Token);
socket.CloseStatus.ShouldBe((WebSocketCloseStatus)RelayProtocol.TargetUnreachableCloseCode);
(await WaitForClosedAuditAsync(ticket.SessionId, timeout.Token)).CloseReason
.ShouldBe(RelayCloseReason.TargetUnreachable);
}
[Fact]
public async Task ATargetThatHangsUp_EndsTheSession()
{
using var target = LoopbackTarget.SayThenHangUp("bye");
var (subject, hostId) = await SeedRoutedHostAsync(target, "10.40.5.1");
var ticket = await IssueAsync(subject, hostId);
using var timeout = Timeout();
using var socket = await ConnectAsync(ticket, timeout.Token);
(await ReadTextAsync(socket, timeout.Token)).ShouldBe("bye");
await ReadUntilCloseAsync(socket, timeout.Token);
socket.CloseStatus.ShouldBe(WebSocketCloseStatus.NormalClosure);
(await WaitForClosedAuditAsync(ticket.SessionId, timeout.Token)).CloseReason
.ShouldBe(RelayCloseReason.TargetClosed);
}
[Fact]
public async Task AnIdleSession_IsClosedWithTheIdleCode()
{
// The close code has to arrive, not merely the close. Cancelling a pending receive aborts a .NET
// WebSocket, and an aborted one sends nothing — so a relay that stopped its reader by cancelling
// it would pass a test that only checked the socket ended.
using var target = LoopbackTarget.Echo();
var (subject, hostId) = await SeedRoutedHostAsync(target, "10.40.6.1");
var ticket = await IssueAsync(subject, hostId);
using var timeout = Timeout();
using var socket = await ConnectAsync(ticket, timeout.Token);
await ReadUntilCloseAsync(socket, timeout.Token);
socket.CloseStatus.ShouldBe((WebSocketCloseStatus)RelayProtocol.IdleTimeoutCloseCode);
(await WaitForClosedAuditAsync(ticket.SessionId, timeout.Token)).CloseReason
.ShouldBe(RelayCloseReason.IdleTimeout);
}
[Fact]
public async Task AUserAtTheCap_IsRefusedATicket_AndASocketOverItIsClosed()
{
using var target = LoopbackTarget.Echo();
var (subject, hostId) = await SeedRoutedHostAsync(target, "10.40.7.1");
// Three tickets while nothing is open, so the third reaches the socket and is refused there —
// the authoritative check, which the ticket endpoint's is only an early warning of.
var tickets = new[]
{
await IssueAsync(subject, hostId),
await IssueAsync(subject, hostId),
await IssueAsync(subject, hostId),
};
using var timeout = Timeout();
using var first = await ConnectAsync(tickets[0], timeout.Token);
using var second = await ConnectAsync(tickets[1], timeout.Token);
await RoundTripAsync(first, "one", timeout.Token);
await RoundTripAsync(second, "two", timeout.Token);
var early = await fixture.CreateClientFor(subject)
.PostContractAsync(RelayProtocol.TicketsPath, new RelayTicketRequest(hostId, null));
early.StatusCode.ShouldBe(HttpStatusCode.TooManyRequests);
(await early.Content.ReadProblemAsync()).ShouldNotBeNull().Code.ShouldBe(ProblemCodes.RelayLimitReached);
using var third = await ConnectAsync(tickets[2], timeout.Token);
await ReadUntilCloseAsync(third, timeout.Token);
third.CloseStatus.ShouldBe((WebSocketCloseStatus)RelayProtocol.TooManySessionsCloseCode);
}
// ---- Helpers ----
private static CancellationTokenSource Timeout()
{
var source = CancellationTokenSource.CreateLinkedTokenSource(TestContext.Current.CancellationToken);
source.CancelAfter(FrameTimeout);
return source;
}
private static string NewSubject() => $"relay-{Guid.CreateVersion7():N}";
private async Task<RelayTicketResponse> IssueAsync(string subject, Guid hostId)
{
var response = await fixture.CreateClientFor(subject)
.PostContractAsync(RelayProtocol.TicketsPath, new RelayTicketRequest(hostId, null));
response.StatusCode.ShouldBe(HttpStatusCode.OK);
return (await response.Content.ReadContractAsync<RelayTicketResponse>()).ShouldNotBeNull();
}
private Task<WebSocket> ConnectAsync(RelayTicketResponse ticket, CancellationToken cancellationToken)
{
var client = fixture.Server.CreateWebSocketClient();
client.SubProtocols.Add(RelayProtocol.SubProtocol);
client.SubProtocols.Add(RelayProtocol.TicketProtocolPrefix + ticket.Ticket);
return client.ConnectAsync(
new Uri(fixture.Server.BaseAddress, RelayProtocol.ConnectPath.TrimStart('/')),
cancellationToken);
}
private static async Task RoundTripAsync(WebSocket socket, string text, CancellationToken cancellationToken)
{
await socket.SendAsync(Encoding.UTF8.GetBytes(text), WebSocketMessageType.Binary, true, cancellationToken);
(await ReadTextAsync(socket, cancellationToken, Encoding.UTF8.GetByteCount(text))).ShouldBe(text);
}
/// <summary>
/// Reads until at least <paramref name="expected"/> bytes have arrived, because message boundaries
/// carry no meaning on the relay and an echo may come back split.
/// </summary>
private static async Task<string> ReadTextAsync(
WebSocket socket,
CancellationToken cancellationToken,
int expected = 1)
{
var buffer = new byte[64 * 1024];
var received = 0;
while (received < expected)
{
var result = await socket.ReceiveAsync(buffer.AsMemory(received), cancellationToken);
result.MessageType.ShouldBe(WebSocketMessageType.Binary);
received += result.Count;
}
return Encoding.UTF8.GetString(buffer, 0, received);
}
private static async Task ReadUntilCloseAsync(WebSocket socket, CancellationToken cancellationToken)
{
var buffer = new byte[64 * 1024];
while (true)
{
var result = await socket.ReceiveAsync(buffer.AsMemory(), cancellationToken);
if (result.MessageType == WebSocketMessageType.Close)
{
return;
}
}
}
/// <summary>
/// Waits for the audit row's second write, which lands after the close frame has gone out.
/// </summary>
private async Task<RelaySession> WaitForClosedAuditAsync(Guid sessionId, CancellationToken cancellationToken)
{
while (true)
{
await using (var scope = fixture.CreateScope())
{
var database = scope.ServiceProvider.GetRequiredService<DodoDbContext>();
var session = await database.RelaySessions
.AsNoTracking()
.SingleOrDefaultAsync(s => s.Id == sessionId, cancellationToken);
if (session?.EndedAtUtc is not null)
{
return session;
}
}
await Task.Delay(TimeSpan.FromMilliseconds(50), cancellationToken);
}
}
private async Task<(string Subject, Guid HostId)> SeedRoutedHostAsync(LoopbackTarget target, string address)
{
fixture.RelayNetwork.Route(new IPEndPoint(IPAddress.Parse(address), 22), target.Endpoint);
return await SeedRelayHostAsync(address, 22);
}
private Task<(string Subject, Guid HostId)> SeedRelayHostAsync(
string hostname,
int port,
string? subject = null) =>
SeedHostAsync((hostname, port), subject);
/// <summary>A user, their personal vault, and one host in it — relayed when an address is given.</summary>
private async Task<(string Subject, Guid HostId)> SeedHostAsync(
(string Hostname, int Port)? relay,
string? subject = null)
{
await using var scope = fixture.CreateScope();
var database = scope.ServiceProvider.GetRequiredService<DodoDbContext>();
var user = await FindOrAddUserAsync(database, subject);
var vault = await database.Vaults.FirstOrDefaultAsync(
v => v.OwnerUserId == user.Id,
TestContext.Current.CancellationToken)
?? database.Vaults.Add(new Vault
{
Id = Guid.CreateVersion7(),
Name = "Personal",
OwnerKind = VaultOwnerKind.Personal,
OwnerUserId = user.Id,
KeyGeneration = 1,
CreatedAtUtc = Now,
UpdatedAtUtc = Now,
}).Entity;
var host = new SshHost
{
Id = Guid.CreateVersion7(),
VaultId = vault.Id,
Payload = [1, 2, 3],
KeyGeneration = 1,
PayloadAadVersion = 1,
RelayEnabled = relay is not null,
Hostname = relay?.Hostname,
Port = relay?.Port,
Version = 1,
CreatedAtUtc = Now,
UpdatedAtUtc = Now,
CreatedByUserId = user.Id,
UpdatedByUserId = user.Id,
};
database.Hosts.Add(host);
await database.SaveChangesAsync(TestContext.Current.CancellationToken);
return (user.Subject, host.Id);
}
private async Task<UserAccount> FindOrAddUserAsync(DodoDbContext database, string? subject)
{
if (subject is not null
&& await database.Users.SingleOrDefaultAsync(
u => u.Subject == subject,
TestContext.Current.CancellationToken) is { } existing)
{
return existing;
}
var user = new UserAccount
{
Id = Guid.CreateVersion7(),
Issuer = fixture.IdentityProvider.Authority,
Subject = NewSubject(),
Status = UserStatus.Active,
CreatedAtUtc = Now,
UpdatedAtUtc = Now,
};
database.Users.Add(user);
database.UserKeys.Add(Seed.CurrentKey(user.Id, Now));
return user;
}
/// <summary>A TCP server on loopback, standing in for the machine the relay reaches.</summary>
private sealed class LoopbackTarget : IDisposable
{
private readonly TcpListener listener = new(IPAddress.Loopback, 0);
private readonly CancellationTokenSource stopping = new();
private LoopbackTarget(Func<NetworkStream, CancellationToken, Task> serve)
{
listener.Start();
_ = AcceptAsync(serve);
}
internal IPEndPoint Endpoint => (IPEndPoint)listener.LocalEndpoint;
/// <summary>Writes back everything it reads, on every connection.</summary>
internal static LoopbackTarget Echo() => new(static async (stream, token) =>
{
var buffer = new byte[16 * 1024];
int read;
while ((read = await stream.ReadAsync(buffer, token)) > 0)
{
await stream.WriteAsync(buffer.AsMemory(0, read), token);
}
});
/// <summary>Writes one line and closes, as a server does that refuses the client.</summary>
internal static LoopbackTarget SayThenHangUp(string text) => new(async (stream, token) =>
await stream.WriteAsync(Encoding.UTF8.GetBytes(text), token));
public void Dispose()
{
stopping.Cancel();
listener.Stop();
stopping.Dispose();
}
private async Task AcceptAsync(Func<NetworkStream, CancellationToken, Task> serve)
{
try
{
while (!stopping.IsCancellationRequested)
{
var client = await listener.AcceptTcpClientAsync(stopping.Token);
_ = Task.Run(
async () =>
{
using (client)
{
try
{
await serve(client.GetStream(), stopping.Token);
}
catch (Exception exception) when (exception is IOException or OperationCanceledException or ObjectDisposedException)
{
// The relay hung up, or the test is over.
}
}
},
CancellationToken.None);
}
}
catch (Exception exception) when (exception is SocketException or OperationCanceledException or ObjectDisposedException)
{
// Stopped.
}
}
}
}
@@ -73,4 +73,20 @@ public sealed class TeamEnumAlignmentTests
((int)PermissionFlags.Read).ShouldBe(1 << 0);
((int)PermissionFlags.Write).ShouldBe(1 << 1);
}
/// <remarks>
/// Not a team enum, and here because this is where the wire-against-domain pairs live. The relay
/// writes its audit row's reason by value, so a renumbering would file every idle timeout as
/// something else.
/// </remarks>
[Fact]
public void EveryWireRelayCloseReason_HasADomainReasonWithTheSameValue()
{
Enum.GetValues<RelaySessionCloseReason>().Length.ShouldBe(Enum.GetValues<RelayCloseReason>().Length);
foreach (var reason in Enum.GetValues<RelaySessionCloseReason>())
{
((int)Enum.Parse<RelayCloseReason>(reason.ToString())).ShouldBe((int)reason);
}
}
}
@@ -222,6 +222,43 @@ public sealed class DodoSshApiClientTests : IDisposable
exception.Code.ShouldBe(ProblemCodes.VaultConflict);
}
[Fact]
public async Task ARelayTicket_IsAskedForByHostAlone()
{
// The request names a host and nothing else. An address in it would be the open proxy ADR 0004
// exists to prevent, and the server would be right to ignore it — so it is not sent.
var hostId = Guid.CreateVersion7();
var issued = new RelayTicketResponse(
"dGlja2V0",
new Uri("wss://relay.example" + RelayProtocol.ConnectPath),
RelayProtocol.SubProtocol,
Guid.CreateVersion7(),
new DateTimeOffset(2026, 9, 28, 12, 0, 30, TimeSpan.Zero));
server.StubRelayTicket(issued);
var ticket = await Client().IssueRelayTicketAsync(hostId, TestContext.Current.CancellationToken);
ticket.ShouldBe(issued);
server.LastAuthorization(RelayProtocol.TicketsPath).ShouldBe("Bearer test-access-token");
var body = System.Text.Json.Nodes.JsonNode.Parse(server.LastBody(RelayProtocol.TicketsPath))!.AsObject();
body["hostId"]!.GetValue<Guid>().ShouldBe(hostId);
body.Where(property => property.Key is not ("hostId" or "portForwardId")).ShouldBeEmpty();
}
[Fact]
public async Task ARefusedRelayTicket_CarriesItsCode()
{
server.StubProblem(
RelayProtocol.TicketsPath, "POST", 403, ProblemCodes.RelayTargetRejected, "The relay will not connect.");
var exception = await Should.ThrowAsync<DodoSshApiException>(async () =>
await Client().IssueRelayTicketAsync(Guid.CreateVersion7(), TestContext.Current.CancellationToken));
exception.Code.ShouldBe(ProblemCodes.RelayTargetRejected);
}
private DodoSshApiClient Client() => new(http, new StubTokenProvider());
private static MeResponse UnenrolledMe() =>
@@ -51,6 +51,10 @@ internal sealed class StubServer : IDisposable
response, DodoSshJsonContext.Default.SyncPullResponse));
/// <summary>Stubs an RFC 9457 problem response.</summary>
internal void StubRelayTicket(RelayTicketResponse response) =>
StubJson(RelayProtocol.TicketsPath, "POST", 200, JsonSerializer.Serialize(
response, DodoSshJsonContext.Default.RelayTicketResponse));
internal void StubProblem(string path, string method, int statusCode, string code, string detail)
{
var problem = new JsonObject
@@ -68,6 +68,9 @@ internal sealed class StubTeamServer : IVaultServer, ITeamApi, IVaultGrantApi
/// </remarks>
public IVaultEventStream Events => IdleVaultEventStream.Instance;
/// <inheritdoc />
public IServerRelay Relay => UnavailableServerRelay.Instance;
/// <inheritdoc />
public SyncOptions SyncOptions => new();
+15
View File
@@ -29,6 +29,16 @@ internal sealed class FakeSshConnectionFactory : ISshConnectionFactory, ISftpSes
/// <summary>Requests this factory was asked for, in order.</summary>
internal List<SshConnectionRequest> Requests { get; } = [];
/// <summary>
/// Runs against each shell request before it succeeds or fails, when set.
/// </summary>
/// <remarks>
/// How a test plays the part of SSH.NET arriving at a proxy: the relay tests use it to speak SOCKS to the
/// bridge the request names, so that what the relay then says reaches the bridge the way it would in a
/// real connect, before <see cref="Failure"/> is thrown.
/// </remarks>
internal Func<SshConnectionRequest, Task>? BeforeConnect { get; set; }
/// <summary>Every shell this factory has opened, in order.</summary>
/// <remarks>
/// Exists so a test can reach one and drop it — see <see cref="FakeSshShellSession.Drop"/>. There was no
@@ -93,6 +103,11 @@ internal sealed class FakeSshConnectionFactory : ISshConnectionFactory, ISftpSes
await gate.Task.WaitAsync(cancellationToken).ConfigureAwait(false);
}
if (BeforeConnect is { } before)
{
await before(request).ConfigureAwait(false);
}
if (Failure is { } failure)
{
throw failure;
@@ -122,6 +122,12 @@ internal sealed partial class FakeVaultServer : IVaultServer, IAccountApi, ISync
/// <inheritdoc />
public IVaultEventStream Events => Notices;
/// <summary>
/// The relay a relayed host is reached through. Unavailable unless a test sets one, which is what a
/// server that does not advertise the feature supplies.
/// </summary>
public IServerRelay Relay { get; set; } = UnavailableServerRelay.Instance;
/// <inheritdoc />
public SyncOptions SyncOptions => SyncOptions.Default;
@@ -0,0 +1,244 @@
using System.Net;
using System.Net.Sockets;
using System.Net.WebSockets;
using System.Text;
using DodoSSH.Client.Api;
using DodoSSH.Client.Session;
using DodoSSH.Client.Shell.ViewModels;
using DodoSSH.Client.Ssh;
using DodoSSH.Contracts;
namespace DodoSSH.Client.App.Tests;
/// <summary>
/// A host set to connect through the server relay, from the Connect button to the request that reaches SSH.
/// </summary>
/// <remarks>
/// <para>
/// The relay here is the real <see cref="ServerRelay"/> — ticket, subprotocols and bridge — over an
/// in-process WebSocket pair, with only the API call and the far end of the socket stood in for. What these
/// hold is the promise the checkbox makes and used not to keep: ticking it changes how the host is reached,
/// and a relay that cannot be used is a stated failure rather than a quiet direct dial.
/// </para>
/// <para>
/// The bridge's own behaviour against a real sshd is <c>RelayBridgeTests</c>, and the server's is
/// <c>RelayEndpointTests</c>.
/// </para>
/// </remarks>
public sealed partial class ShellFlowTests
{
[Fact]
public async Task ARelayedHost_IsDialledThroughTheBridge_AndStillAsItself()
{
var tickets = new FakeRelayApi();
server.Relay = new ServerRelay(tickets, ConnectPairedAsync);
var vault = await ReadyToConnectRelayedAsync();
await using var renderer = await FakeRenderer.AttachAsync(workspace, Token);
await vault.ConnectCommand.ExecuteAsync(null);
tickets.HostIds.ShouldHaveSingleItem().ShouldBe(vault.SelectedHost.ShouldNotBeNull().EntityId);
// Through a loopback proxy — and still named as the host, because the name is what the host key is
// pinned under. The bridge carries the name in its CONNECT; it does not replace it.
var request = ssh.Requests.ShouldHaveSingleItem();
request.Proxy.ShouldNotBeNull().Lifetime.ShouldBeOfType<RelayBridge>();
request.Host.ShouldBe("db.internal");
request.Port.ShouldBe(22);
// The ticket travels as the second subprotocol and never as anything else.
offeredProtocols.ShouldBe([RelayProtocol.SubProtocol, RelayProtocol.TicketProtocolPrefix + FakeRelayApi.Ticket]);
shell.IsTerminalShowing.ShouldBeTrue();
await request.Proxy.Lifetime.DisposeAsync();
}
[Fact]
public async Task ARelayedHost_OnAServerWithoutARelay_IsRefusedWithoutDialling()
{
// The fake server's default: a server that does not advertise the feature.
var vault = await ReadyToConnectRelayedAsync();
await using var renderer = await FakeRenderer.AttachAsync(workspace, Token);
await vault.ConnectCommand.ExecuteAsync(null);
// No fallback to the direct address. The user ticked the relay because that path does not work, or
// goes somewhere they do not mean.
ssh.Requests.ShouldBeEmpty();
vault.Status.ShouldNotBeNull().ShouldContain("does not offer one");
}
[Theory]
[InlineData(ProblemCodes.RelayTargetRejected, "refused to relay")]
[InlineData(ProblemCodes.RelayLimitReached, "as many relay sessions")]
[InlineData(ProblemCodes.RelayUnavailable, "does not offer one")]
public async Task ARefusedTicket_IsExplained_AndNothingIsDialled(string code, string explanation)
{
server.Relay = new ServerRelay(
new FakeRelayApi { Refusal = new DodoSshApiException(HttpStatusCode.Forbidden, code, "no") },
ConnectPairedAsync);
var vault = await ReadyToConnectRelayedAsync();
await using var renderer = await FakeRenderer.AttachAsync(workspace, Token);
await vault.ConnectCommand.ExecuteAsync(null);
ssh.Requests.ShouldBeEmpty();
vault.Status.ShouldNotBeNull().ShouldContain(explanation);
}
[Fact]
public async Task ARelayThatCouldNotReachTheHost_SaysSo_InsteadOfTheTransportError()
{
// What the server does when it cannot dial the host: accept the socket, then close it with 4502.
server.Relay = new ServerRelay(new FakeRelayApi(), async (url, protocols, cancellationToken) =>
{
var socket = await ConnectPairedAsync(url, protocols, cancellationToken);
await relayEnd!.CloseOutputAsync(
(WebSocketCloseStatus)RelayProtocol.TargetUnreachableCloseCode,
"unreachable",
cancellationToken);
return socket;
});
// SSH.NET arriving at the bridge and then losing its stream, which is how a relay that hung up looks
// from inside the connect.
ssh.BeforeConnect = request => ThroughSocksToEndOfStreamAsync(request.Proxy.ShouldNotBeNull().Port);
ssh.Failure = new InvalidOperationException("An established connection was aborted by the server.");
var vault = await ReadyToConnectRelayedAsync();
await using var renderer = await FakeRenderer.AttachAsync(workspace, Token);
await vault.ConnectCommand.ExecuteAsync(null);
vault.Status.ShouldNotBeNull().ShouldContain("could not reach this host");
vault.Status.ShouldNotContain("aborted");
}
[Fact]
public async Task ARelayedHostsFileTransfer_GoesThroughTheRelayToo()
{
// Its own connection, so it needs its own relay — or the transfers screen would fail for exactly the
// hosts the relay exists for.
server.Relay = new ServerRelay(new FakeRelayApi(), ConnectPairedAsync);
var vault = await ReadyToConnectRelayedAsync();
shell.Transfers.Attach(vault, knownHosts);
shell.Transfers.SelectedHost = shell.Transfers.Hosts[0];
await shell.Transfers.ConnectCommand.ExecuteAsync(null);
var request = ssh.SftpRequests.ShouldHaveSingleItem();
request.Proxy.ShouldNotBeNull().Lifetime.ShouldBeOfType<RelayBridge>();
await request.Proxy.Lifetime.DisposeAsync();
}
/// <summary>The protocols the last relay socket was opened offering.</summary>
private IReadOnlyList<string>? offeredProtocols;
/// <summary>The far end of the last relay socket, which a test may close as the server would.</summary>
private WebSocket? relayEnd;
private async Task<VaultViewModel> ReadyToConnectRelayedAsync()
{
await UnlockedAsync();
var vault = shell.Vault!;
vault.NewHostCommand.Execute(null);
vault.EditorLabel = "behind-the-server";
vault.EditorHostname = "db.internal";
vault.EditorUsername = "deploy";
// A relayed host needs a port of its own; the server stores it and dials it.
vault.EditorPort = 22;
vault.EditorRelayEnabled = true;
await vault.SaveHostCommand.ExecuteAsync(null);
vault.SelectedHost = vault.Hosts.Single(row => row.Host.RelayEnabled);
return vault;
}
/// <summary>Opens both ends of a WebSocket over loopback, and keeps the far one.</summary>
private async Task<WebSocket> ConnectPairedAsync(
Uri url,
IReadOnlyList<string> protocols,
CancellationToken cancellationToken)
{
url.ShouldBe(FakeRelayApi.SocketUrl);
offeredProtocols = protocols;
using var listener = new TcpListener(IPAddress.Loopback, 0);
listener.Start();
var dialling = new TcpClient();
var connecting = dialling.ConnectAsync((IPEndPoint)listener.LocalEndpoint, cancellationToken);
var accepted = await listener.AcceptTcpClientAsync(cancellationToken);
await connecting;
relayEnd = WebSocket.CreateFromStream(
accepted.GetStream(),
new WebSocketCreationOptions { IsServer = true, KeepAliveInterval = TimeSpan.Zero });
return WebSocket.CreateFromStream(
dialling.GetStream(),
new WebSocketCreationOptions { IsServer = false, KeepAliveInterval = TimeSpan.Zero });
}
/// <summary>
/// Speaks SOCKS5 to the bridge as SSH.NET would, then reads until the bridge closes the stream — which
/// it does once the relay has closed, so the relay's reason is recorded by the time this returns.
/// </summary>
private static async Task ThroughSocksToEndOfStreamAsync(int port)
{
using var client = new TcpClient();
await client.ConnectAsync(IPAddress.Loopback, port, Token);
var stream = client.GetStream();
var name = Encoding.ASCII.GetBytes("db.internal");
await stream.WriteAsync(new byte[] { 5, 1, 0 }, Token);
await stream.ReadExactlyAsync(new byte[2], Token);
await stream.WriteAsync((byte[])[5, 1, 0, 3, (byte)name.Length, .. name, 0, 22], Token);
await stream.ReadExactlyAsync(new byte[10], Token);
var rest = new byte[256];
while (await stream.ReadAsync(rest, Token) > 0)
{
}
}
/// <summary>The ticket half of the API, answering or refusing as a test arranges.</summary>
private sealed class FakeRelayApi : IRelayApi
{
internal const string Ticket = "dGlja2V0";
internal static readonly Uri SocketUrl = new("ws://relay.test" + RelayProtocol.ConnectPath);
internal List<Guid> HostIds { get; } = [];
internal DodoSshApiException? Refusal { get; init; }
public Task<RelayTicketResponse> IssueRelayTicketAsync(Guid hostId, CancellationToken cancellationToken)
{
HostIds.Add(hostId);
return Refusal is { } refusal
? Task.FromException<RelayTicketResponse>(refusal)
: Task.FromResult(new RelayTicketResponse(
Ticket,
SocketUrl,
RelayProtocol.SubProtocol,
Guid.CreateVersion7(),
TimeProvider.System.GetUtcNow().AddSeconds(30)));
}
}
}
@@ -0,0 +1,221 @@
using System.Net;
using System.Net.Sockets;
using System.Net.WebSockets;
namespace DodoSSH.Client.Ssh.Tests;
/// <summary>
/// The relay bridge: a real SSH connection, through a real SOCKS5 handshake, over a real WebSocket.
/// </summary>
/// <remarks>
/// <para>
/// The relay server is stood in for by the other end of an in-process WebSocket pair that pipes to the
/// fixture's sshd — which is the whole of what the server's socket does once a ticket is spent, so the
/// bytes on the wire here are the bytes a relayed session carries. What is not here is the ticket, which
/// is the server's half and is tested there.
/// </para>
/// <para>
/// Three of these are the security properties the bridge has to keep, from
/// <c>docs/reaching-a-host-you-cannot-dial.md</c>: it binds loopback only, it accepts once, and a
/// connection that goes away takes the bridge — and so the relay session — with it. The failure each
/// guards against is silent: an open proxy, or a relay session left running behind a closed tab.
/// </para>
/// </remarks>
[Collection(SshCollection.Name)]
public sealed class RelayBridgeTests(SshServerFixture fixture)
{
private static readonly TimeSpan Settles = TimeSpan.FromSeconds(20);
private static CancellationToken Token => TestContext.Current.CancellationToken;
[Fact]
public async Task TheBridge_ListensOnLoopbackOnly()
{
var (client, _) = await PairAsync();
await using var bridge = RelayBridge.Start(client);
// Asserted on the address rather than trusted to the constructor: a bridge on every interface
// would be a proxy into the relay for anyone on the network, and nothing would report it.
bridge.Endpoint.Address.ShouldBe(IPAddress.Loopback);
bridge.Proxy.Port.ShouldBe(bridge.Endpoint.Port);
bridge.Proxy.Lifetime.ShouldBeSameAs(bridge);
}
[Fact]
public async Task AShell_ThroughTheBridge_ReachesTheHost_AndGoesWithTheConnection()
{
var knownHosts = await TrustedStoreAsync();
var (client, server) = await PairAsync();
var relaying = RelayToSshdAsync(server);
var bridge = RelayBridge.Start(client);
// The host as the request names it, so the pin is the host's — the bridge is a SOCKS proxy, and
// the CONNECT carries this name through it rather than 127.0.0.1.
var connection = await new SshNetConnectionFactory(knownHosts)
.ConnectAsync(Request(bridge.Proxy), progress: null, Token);
connection.IsConnected.ShouldBeTrue();
connection.HostKey.Host.ShouldBe(fixture.Host);
// The connection owns the bridge. Closing the one is what closes the other, and the relay hears it
// as a close rather than as a socket that stopped answering.
await connection.DisposeAsync();
await bridge.Completion.WaitAsync(Settles, Token);
(await relaying.WaitAsync(Settles, Token)).ShouldBe(WebSocketMessageType.Close);
}
[Fact]
public async Task TheBridge_AcceptsOnce()
{
var (client, _) = await PairAsync();
await using var bridge = RelayBridge.Start(client);
using var first = new TcpClient();
await first.ConnectAsync(bridge.Endpoint, Token);
// A greeting answered means the first connection has been accepted, which is when the listener
// stops — so what the second attempt meets is a closed port, not a race with the backlog.
var stream = first.GetStream();
await stream.WriteAsync(new byte[] { 5, 1, 0 }, Token);
var reply = new byte[2];
await stream.ReadExactlyAsync(reply, Token);
reply.ShouldBe(new byte[] { 5, 0 });
using var second = new TcpClient();
await Should.ThrowAsync<SocketException>(second.ConnectAsync(bridge.Endpoint, Token).AsTask());
}
[Fact]
public async Task ARelayThatClosesWithAReason_FailsTheConnect_AndKeepsTheReason()
{
// What the server does when it cannot reach the host: accept the socket, then close it with 4502.
var (client, server) = await PairAsync();
await server.CloseOutputAsync((WebSocketCloseStatus)4502, "unreachable", Token);
var bridge = RelayBridge.Start(client);
var failure = await Should.ThrowAsync<Exception>(async () =>
await new SshNetConnectionFactory(new InMemoryKnownHostStore())
.ConnectAsync(Request(bridge.Proxy), progress: null, Token));
// A connection error, not a question about a host key nobody presented.
failure.ShouldNotBeOfType<SshHostKeyUnknownException>();
bridge.CloseStatus.ShouldBe((WebSocketCloseStatus)4502);
// And the failed connect disposed the bridge it was handed, rather than leaving it listening.
await bridge.Completion.WaitAsync(Settles, Token);
}
[Fact]
public async Task NobodyArriving_ClosesTheRelay()
{
var (client, server) = await PairAsync();
await using var bridge = RelayBridge.Start(client, TimeSpan.FromMilliseconds(200));
await bridge.Completion.WaitAsync(Settles, Token);
var received = await server.ReceiveAsync(new byte[16], Token);
received.MessageType.ShouldBe(WebSocketMessageType.Close);
}
/// <summary>Two ends of one WebSocket, over a loopback TCP connection this test owns.</summary>
private static async Task<(WebSocket Client, WebSocket Server)> PairAsync()
{
using var listener = new TcpListener(IPAddress.Loopback, 0);
listener.Start();
var dialling = new TcpClient();
var connecting = dialling.ConnectAsync((IPEndPoint)listener.LocalEndpoint, Token);
var accepted = await listener.AcceptTcpClientAsync(Token);
await connecting;
var client = WebSocket.CreateFromStream(
dialling.GetStream(),
new WebSocketCreationOptions { IsServer = false, KeepAliveInterval = TimeSpan.Zero });
var server = WebSocket.CreateFromStream(
accepted.GetStream(),
new WebSocketCreationOptions { IsServer = true, KeepAliveInterval = TimeSpan.Zero });
return (client, server);
}
/// <summary>
/// The relay's half: bytes from the socket to sshd and back, until one side ends.
/// </summary>
/// <returns>The type of the last message the relay received — a close, when the client said goodbye.</returns>
private Task<WebSocketMessageType> RelayToSshdAsync(WebSocket server) => Task.Run(async () =>
{
using var target = new TcpClient();
await target.ConnectAsync(fixture.Host, fixture.Port, Token);
var stream = target.GetStream();
var down = Task.Run(async () =>
{
var buffer = new byte[32 * 1024];
int read;
try
{
while ((read = await stream.ReadAsync(buffer, Token)) > 0)
{
await server.SendAsync(buffer.AsMemory(0, read), WebSocketMessageType.Binary, true, Token);
}
}
catch (Exception exception) when (exception is IOException or WebSocketException or ObjectDisposedException)
{
// The test's connection is over.
}
});
var inbound = new byte[32 * 1024];
while (true)
{
var received = await server.ReceiveAsync(inbound.AsMemory(), Token);
if (received.MessageType == WebSocketMessageType.Close)
{
await server.CloseOutputAsync(WebSocketCloseStatus.NormalClosure, string.Empty, Token);
target.Close();
await down;
return received.MessageType;
}
await stream.WriteAsync(inbound.AsMemory(0, received.Count), Token);
}
});
private SshConnectionRequest Request(SshLoopbackProxy proxy) =>
new(
fixture.Host,
fixture.Port,
SshServerFixture.Username,
new SshPasswordCredential(SshServerFixture.Password),
ConnectTimeout: null,
proxy);
/// <summary>A store that already trusts the container's host key, learned directly.</summary>
private async Task<InMemoryKnownHostStore> TrustedStoreAsync()
{
var knownHosts = new InMemoryKnownHostStore();
var unknown = await Should.ThrowAsync<SshHostKeyUnknownException>(async () =>
await new SshNetConnectionFactory(knownHosts).ConnectAsync(
new SshConnectionRequest(
fixture.Host,
fixture.Port,
SshServerFixture.Username,
new SshPasswordCredential(SshServerFixture.Password)),
progress: null,
Token));
await knownHosts.TrustAsync(unknown.Presentation, Token);
return knownHosts;
}
}