Public Access
jaap-jan
b20f7cae04
The relay checkbox said it was not wired up, and the plan said the server half was built and shipped. It was not: there was ADR 0004, the contract records, RelayOptions and a CHECK constraint, and RelayOptions said in its own remarks that there was no endpoint and no dialler. So both halves are built here, as step 3 of docs/reaching-a-host-you-cannot-dial.md. Server, in Features/Relay: - POST /api/v1/relay/tickets makes every decision: the host, Connect on its vault, the relay switched on, and the stored name resolved once and judged by RelayAddressPolicy. Loopback, link-local, 0.0.0.0/8, multicast and the metadata endpoints are refused with no configuration escape, with IPv4-mapped and NAT64 spellings unwrapped first; then the operator's denials, allow-list and private-network switch. What passes goes into a Data Protection ticket, and nothing resolves the name again. Every target refusal is the same 403, so the endpoint cannot be used to map the operator's DNS; the real reason is logged. - GET /api/v1/relay/connect takes the ticket as a second subprotocol, checks its expiry against TimeProvider and spends it with a unique insert into relay_ticket_use before accepting the upgrade. After the upgrade a refusal is a close code: 4502 for a host the server could not reach, 4429 for a limit, 4408 idle, 4409 lifetime, 1001 after a drain. - relay_session is the audit row, written from short-lived scopes. Client: - RelayBridge is a loopback SOCKS5 listener that accepts once and pipes to the relay socket. It rides on the new SshLoopbackProxy.Lifetime, so the connection disposes it on every way a session ends and a failed connect disposes it before throwing. - ServerRelay, exposed as IVaultServer.Relay from /api/v1/meta, asks for the ticket, opens the socket, starts the bridge and turns every refusal into a sentence. - VaultViewModel and TransfersViewModel reach a relayed host through it, never fall back to dialling it directly, and replace SSH.NET's transport error with the relay's own reason when it gave one. Neither side ever cancels a WebSocket operation. Cancelling one aborts the socket, and an aborted socket sends no close frame, which is the only way the client learns the server could not reach the host. AnIdleSession_IsClosedWithTheIdleCode asserts that the code arrives. Three departures from ADR 0004, recorded in it: awaited copy loops rather than Pipelines (same backpressure, two 32 KiB buffers), no ?ticket= fallback, and no device id in the ticket. Tests: API 245, shell 582 (5 skipped), layout 170, SSH 110 including a real login through the bridge, API client 30, contracts 25. Not run: the Android build (its change is copy only) and the Alpine layout oracle.
121 lines
6.9 KiB
Markdown
121 lines
6.9 KiB
Markdown
# ADR 0004 — Relay as a raw TCP tunnel, and how its targets are authorized
|
|
|
|
- Status: accepted
|
|
- Date: 2026-07-28
|
|
|
|
## Context
|
|
|
|
Some hosts are not reachable from a user's laptop. The obvious answer is a server-side jump
|
|
host — but that conflicts directly with [ADR 0001](0001-e2ee-trust-model.md): a server that
|
|
terminates SSH needs plaintext credentials for the target.
|
|
|
|
A server that dials arbitrary addresses on an authenticated user's behalf is also a textbook
|
|
SSRF and lateral-movement primitive, aimed at the operator's own network.
|
|
|
|
## Decision
|
|
|
|
### The relay forwards bytes, it does not terminate SSH
|
|
|
|
The client opens a WebSocket to the backend; the backend pipes raw bytes to `host:port`. The
|
|
SSH handshake still terminates on the client, so the relay sees only SSH ciphertext.
|
|
Zero-knowledge survives. Subprotocol `dodossh.relay.v1`; no framing of our own, because SSH is
|
|
already a byte stream.
|
|
|
|
### Two steps, so the WebSocket carries no API authority
|
|
|
|
1. `POST /relay/tickets` — full bearer JWT, full ACL context. Takes **`{ hostId }`, never a
|
|
client-supplied address.**
|
|
2. `GET /relay/connect` — WebSocket upgrade with the ticket only, zero API access.
|
|
|
|
Tickets are `IDataProtectionProvider` payloads (key management and rotation for free) carrying
|
|
`jti, userId, deviceId, hostId, targetIps[], targetPort, exp ≤ 30s`, single-use via a unique
|
|
insert into `relay_ticket_use`. The ticket travels in
|
|
`Sec-WebSocket-Protocol: dodossh.relay.v1, ticket.<token>`, since constrained WebSocket clients
|
|
cannot set arbitrary headers; a `?ticket=` fallback exists and **requires query scrubbing in
|
|
access logs and OTel spans**.
|
|
|
|
### Anti-SSRF, in order of importance
|
|
|
|
1. **The server resolves the target from `host_id`.** `relay_enabled = true` and non-null
|
|
`hostname`/`port` are enforced by a database CHECK constraint. Arbitrary targets are not
|
|
supported in v1 at all.
|
|
2. **DNS resolves at ticket-issue time and the resolved IPs go into the ticket; the relay
|
|
dials the IP, never the name.** This is what defeats DNS rebinding, which otherwise breaks
|
|
naive allow-listing.
|
|
3. **A non-overridable deny list with no configuration escape:** loopback, link-local,
|
|
`0.0.0.0/8`, multicast, cloud metadata (`169.254.169.254`, `100.100.100.200`,
|
|
`fd00:ec2::254`), and **IPv4-mapped IPv6 normalised and then re-checked** — a classic
|
|
bypass.
|
|
4. **A configurable layer:** `Relay:AllowPrivateNetworks` defaults **true**, because RFC1918 is
|
|
the primary use case for a self-hosted SSH tool, plus allow/deny CIDRs. Ports allow
|
|
everything except SMTP. Database ports stay open, because forwarding to 5432 is a headline
|
|
feature — **the real control is the ACL**: you can only dial hosts in a vault you hold
|
|
`Connect` on.
|
|
5. **Limits:** 10 concurrent sessions per user, 200 per node, 12 h maximum, 10 min idle, 5 s
|
|
connect.
|
|
|
|
### Why host addresses are plaintext
|
|
|
|
Encrypting hostnames sounds strictly better and is not. The relay must resolve its target
|
|
server-side or point 1 above collapses and the relay becomes an authenticated open TCP proxy —
|
|
a worse risk than the metadata it would protect. Plaintext address and port are therefore
|
|
stored **only when the user opts that host into relay**, enforced by the CHECK constraint.
|
|
Everything else about a host — username, notes, jump chain, options — is always ciphertext, and
|
|
there is no plaintext host label at all, because ACL administration runs on the client, which
|
|
can decrypt names.
|
|
|
|
Searchability is explicitly *not* the argument: vaults hold thousands of items, not millions,
|
|
so the client syncs everything and searches in memory. The relay and audit arguments are the
|
|
real ones.
|
|
|
|
## Consequences
|
|
|
|
- **No session recording or command auditing is possible in relay mode.** The relay sees
|
|
ciphertext. Do not promise otherwise. Recording would need a separate, explicitly
|
|
non-zero-knowledge "recorded bastion" mode opted into per host.
|
|
- Useful audit remains: `target_host:port`, duration, byte counts, close reason, client IP.
|
|
- Backpressure comes from `System.IO.Pipelines` with `pauseWriterThreshold: 1 MiB`. When the
|
|
WebSocket peer is slow, `FlushAsync` stops completing, which stops reading the socket, which
|
|
lets TCP's own receive window throttle the origin end-to-end. No custom flow control, ~2 MiB
|
|
bounded per session — which is what makes 200 sessions per node viable.
|
|
- **The relay must not hold a `DbContext` for the session lifetime.** Insert the row, dispose
|
|
the scope, relay, then open a fresh scope to write the close row. Otherwise the connection
|
|
pool dies around 100 concurrent sessions.
|
|
- Tickets are stateless-verifiable and `relay_ticket_use` is the only shared state, so any node
|
|
accepts any ticket: plain round-robin, no session affinity. That table is also the extraction
|
|
seam — a standalone relay process needs the Data Protection key ring and one table, no ACL
|
|
code. Extraction is worthwhile eventually because long-lived connections and short requests
|
|
have opposite scaling and rollout profiles.
|
|
- Graceful shutdown sends close 1001 and drains for 30 s, so a `docker compose up -d` does not
|
|
guillotine live shells.
|
|
- On the client, SSH.NET cannot be handed a pre-connected stream, so the relay is reached via a
|
|
loopback TCP bridge. The same bridge provides ProxyJump via a SOCKS5 dynamic forward — one
|
|
mechanism, two features.
|
|
|
|
### Rejected
|
|
|
|
- **Server terminates SSH (a true bastion).** Enables recording and central credential
|
|
control; destroys zero-knowledge. Out of scope, possibly a separate product line.
|
|
- **Client-supplied target address.** One line of convenience, and the relay becomes an open
|
|
proxy into the operator's network.
|
|
- **Allow-listing by hostname.** Defeated by DNS rebinding.
|
|
|
|
## Implementation notes
|
|
|
|
Built in `DodoSSH.Api/Features/Relay` and, on the client, `RelayBridge` and `ServerRelay`; see
|
|
`docs/reaching-a-host-you-cannot-dial.md`, "What step 3 built". Three places differ from the text above:
|
|
|
|
- **Backpressure is an awaited copy loop, not `System.IO.Pipelines`.** Each direction reads one buffer and
|
|
writes it before reading again. The property is the same — a slow peer stops the read, and TCP throttles
|
|
the origin — and the bound is tighter: two 32 KiB buffers per session.
|
|
- **There is no `?ticket=` fallback.** Both clients can set a subprotocol, and the fallback's cost — a bearer
|
|
credential in the URL — was the reason it needed scrubbing at all. It can be added if a constrained
|
|
client ever needs it; the scrubbing requirement stands.
|
|
- **The ticket carries no device id.** The server does not see a device on an ordinary request, so there was
|
|
nothing to put there. `PortForwardId` on the request is refused: a forward's port is ciphertext, and
|
|
honouring one would mean taking the port from the client.
|
|
|
|
The loopback bridge serves the relay only. SSH.NET's `ForwardedPortDynamic` already is the jump-host path's
|
|
listener, so "one mechanism, two features" is true one level down: both features reach their target through
|
|
a loopback SOCKS5 proxy named on the connection request.
|