Public Access
f43436218ac932f5a9613fb9fd7b24447a06ea43
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 jaap-jan
|
b20f7cae04 |
Let a host marked for the relay actually be reached through it
The relay checkbox said it was not wired up, and the plan said the server half was built and shipped. It was not: there was ADR 0004, the contract records, RelayOptions and a CHECK constraint, and RelayOptions said in its own remarks that there was no endpoint and no dialler. So both halves are built here, as step 3 of docs/reaching-a-host-you-cannot-dial.md. Server, in Features/Relay: - POST /api/v1/relay/tickets makes every decision: the host, Connect on its vault, the relay switched on, and the stored name resolved once and judged by RelayAddressPolicy. Loopback, link-local, 0.0.0.0/8, multicast and the metadata endpoints are refused with no configuration escape, with IPv4-mapped and NAT64 spellings unwrapped first; then the operator's denials, allow-list and private-network switch. What passes goes into a Data Protection ticket, and nothing resolves the name again. Every target refusal is the same 403, so the endpoint cannot be used to map the operator's DNS; the real reason is logged. - GET /api/v1/relay/connect takes the ticket as a second subprotocol, checks its expiry against TimeProvider and spends it with a unique insert into relay_ticket_use before accepting the upgrade. After the upgrade a refusal is a close code: 4502 for a host the server could not reach, 4429 for a limit, 4408 idle, 4409 lifetime, 1001 after a drain. - relay_session is the audit row, written from short-lived scopes. Client: - RelayBridge is a loopback SOCKS5 listener that accepts once and pipes to the relay socket. It rides on the new SshLoopbackProxy.Lifetime, so the connection disposes it on every way a session ends and a failed connect disposes it before throwing. - ServerRelay, exposed as IVaultServer.Relay from /api/v1/meta, asks for the ticket, opens the socket, starts the bridge and turns every refusal into a sentence. - VaultViewModel and TransfersViewModel reach a relayed host through it, never fall back to dialling it directly, and replace SSH.NET's transport error with the relay's own reason when it gave one. Neither side ever cancels a WebSocket operation. Cancelling one aborts the socket, and an aborted socket sends no close frame, which is the only way the client learns the server could not reach the host. AnIdleSession_IsClosedWithTheIdleCode asserts that the code arrives. Three departures from ADR 0004, recorded in it: awaited copy loops rather than Pipelines (same backpressure, two 32 KiB buffers), no ?ticket= fallback, and no device id in the ticket. Tests: API 245, shell 582 (5 skipped), layout 170, SSH 110 including a real login through the bridge, API client 30, contracts 25. Not run: the Android build (its change is copy only) and the Alpine layout oracle. |
||
|
jaap-jan
|
4b706bc3c3 |
Say when a vault has moved, so nobody waits out the minute
The delta pull was cheap enough to run on a timer and the client did, once a minute. That is fine for a machine and wrong for two people: an edit a colleague makes is up to a minute stale, which is long enough for both of them to make it and produce a conflict neither needed to have. Shortening the interval is the obvious answer and the wrong one — it costs a request per client per interval whether or not anything happened, and it converges on a busier server that is still late. So the server now says so. A client holds a WebSocket open at GET /api/v1/events, subprotocol dodossh.events.v1, and gets a line down it when something it can read has changed. ADR 0012 has the reasoning; three parts of it are worth repeating here, because they are what everything else rests on. **What crosses the socket is a notice, never data.** A frame names a vault and how far its change log has got. No item, no ciphertext, not even which item it was. The client's answer is the delta pull it would have run anyway, so there is still exactly one code path that applies a change to a keychain, and it is not this one. Pushing the items themselves would save a round trip and fork that path in two, with the cursor, the merge and the tombstone rules duplicated across both — ADR 0003 put every mutation through one write path for that reason, and this keeps every read on one for the same one. It also makes a dropped notice harmless, which is what lets the fan-out below be as simple as it is. **Polling stays, and is what guarantees a pass.** The minute timer is unchanged. A network that eats WebSockets, a server with Events:Enabled off, an older server, a proxy that will not upgrade, a notice dropped under backpressure — every one of those leaves a client behaving exactly as it did before this commit. Nothing is reachable only over the socket and nothing is meant to become so; VaultViewModel's AutoSyncInterval remark now says that where somebody changing it will read it. **The bearer token authorises the upgrade, unlike the relay's ticket.** Not an inconsistency with ADR 0004: the relay's socket is a byte pipe whose whole authorization decision — which host, which IPs, which port — is made before it opens and never revisited, and it is the extraction seam for a process that must hold no ACL code. This one is a view of the caller's own vault list and has to keep answering "what may this account read" for as long as it is held. A ticket would carry that answer in a token and be wrong the moment the account's access changed. The two bounds that arrangement needs are met rather than waved at: the socket is closed at the token's exp with close code 4401 and the client comes straight back with a fresh one, and the vault set is re-resolved every few minutes as well as on the changes known to affect it. Both bound *metadata*, because a notice contains nothing else and reading a vault still needs a key this server has never held. **The fan-out.** VaultEventHub is a singleton holding the sockets this node accepted; publishing walks them and asks each whether it cares, rather than keeping a vault-to-subscriber index that every re-subscription would have to move entries between under a lock publishing also takes. At a few hundred sockets per node and an event rate bounded by how often people edit keychains, the walk is not measurable and its races are obvious. Per-connection queues are bounded and drop the *oldest*: a notice means "pull vault X, which is at least at sequence N", so the newest subsumes what it displaces and the client's answer is identical either way — which is what lets the publish path be void, never block, and never fail. Announced from the endpoint rather than from SyncService, and that placement is the point: by then the push has committed and released the per-vault advisory lock. From inside it would name a sequence no reader can see yet and would hold the lock that serialises writers across a socket write. Only the highest *applied* sequence, so a batch of pure conflicts announces nothing, and a duplicate — already announced when it first landed — announces nothing either. Grants and membership publish too, and those take the *recipient* rather than the actor. This is what AdmitNewVaultsAsync has been apologising for since sharing shipped — "the recipient is handed nothing, there is no push channel" — and the README with it. A vault shared with somebody now turns up as it is shared. The comment and the README paragraph both say what is true now, and both keep saying that the pass is what *discovers* the vault, because a client with no socket has to arrive at the same place. **On the client**, VaultEventStream is really a reconnection policy wrapped round a ClientWebSocket: a dropped socket is the ordinary case here — laptops sleep, proxies time out, tokens expire, servers are redeployed — so nothing in it treats a failure as exceptional, and every path ends in "wait, then dial again". A connection that lived long enough to say hello resets the backoff, so a laptop that woke, worked, and lost its network an hour later does not inherit a minute-long wait it has already proved it need not take. A 4401 close skips the backoff entirely and asks the token provider again, which is the whole reason that close code is distinct. A server that does not advertise the events feature gets IdleVaultEventStream, which never delivers — so IVaultServer.Events is never null and every caller stays on one shape, because the correct behaviour without a socket is the behaviour with a silent one. The shell's background loop now selects between the timer and a notice, and both waits are held across iterations. That is load-bearing rather than tidy: PeriodicTimer permits one outstanding WaitForNextTickAsync and throws on a second, and an abandoned channel read stays registered and consumes the next notice written. Either defect leaves the first notice working and every one after it silently lost, which is why NoticesKeepWakingTheLoop_NotJustTheFirst pushes three and not one. Notices are coalesced over a quarter of a second, so one person's save — a host and its log entry are two items — and a colleague clearing a folder each cost one pass rather than a dozen. **The kind is a string, not an enum**, and that is a compatibility decision. UseStringEnumConverter throws on a value it does not know, so a newer server sending a kind an older client had never heard of would not add an unreadable frame — it would break that client's socket outright. A string is ignored instead. ProblemCodes is the same shape for the same reason. **Tested on both sides, through the real pipeline.** The endpoint suite opens a genuine socket against TestServer and proves a push produces a notice, that another account's push does not reach it, that a ping is answered, and that a frame this server cannot parse does not end the connection. Two of those assert on *ordering* rather than on absence within a timeout — the stranger's write goes first, so a socket that leaked would have announced it before the one the test waits for — because "nothing arrived in two seconds" is a test that passes on a slow machine for the wrong reason. And ANoticeCarriesNoCiphertext asserts on the bytes that crossed the wire rather than on the record's fields, since the latter would only prove that this type has no payload member, which is a tautology; the former is what catches a field added later without anybody thinking about disclosure. The client suite drives VaultEventStream through an injected connector, because the one thing a test cannot do to a real network is make it fail on cue — and failure is the entire subject. The shell suite proves a notice produces a pull inside ten seconds against a sixty-second timer, so the timer cannot be what caused it. **Two limits, stated rather than left to be discovered.** Fan-out is in-process, so a deployment running more than one API replica only pushes for writes its own replica handled and the rest arrive on the timer. IVaultEventPublisher is the seam a PostgreSQL LISTEN/NOTIFY backplane implements and it is deliberately not implemented: an untested backplane is worse than a documented gap, and multiple replicas degrade to the behaviour before this commit rather than breaking. And a client is notified of its own writes; it pushed, so it already pulled, and the extra pass finds nothing. Suppressing that echo correctly needs a per-device identity on the socket, and the same user's other machines must still be told. Manual checks phase 15 covers what no test here can reach, which is the network in between: a proxy that will not upgrade, one that drops an idle socket without telling either end, a laptop lid, a token expiring. Every one of those is invisible inside a test host, and every check there passes only if the change arrives quickly *and* still arrives with the socket taken away. ADR 0012 also fixes one thing about the shared terminal session this is the transport for, so it need not be renegotiated later: session data will be binary frames on this same socket, because base64 in a JSON envelope is the wrong shape for the one payload here that is continuous rather than occasional. Two questions it explicitly does not answer by implication — whether those bytes go through the API at all, and what end-to-end encryption means when the second party watches a stream rather than holding a key — are ADR 0001 questions and get their own decision. 1512 tests pass. DodoSSH.SystemTests was not run — it needs the whole compose stack — so the end-to-end path is unverified for this change beyond what the manual checks describe. |
||
 jaap-janandClaude Opus 5
|
093f3904c1 |
Apply pending migrations at startup instead of asking for a second command
The API deliberately never migrated: it failed readiness while a migration was pending and named it, and a separate step applied them. That is the right split for a deployment with a release pipeline and the wrong one for a self-hosted server, where it means an image that boots, refuses traffic, and waits for somebody to know that dotnet ef exists. The schema and the code that expects it ship in the same image, so the image is where the two are reconciled now. Before RunAsync rather than in the background. A migration racing the first requests would let them through against a half-applied schema, and the first authenticated request is the one that provisions accounts. Failing to migrate therefore fails to start, which is the loudest signal available and the one an orchestrator already acts on. Concurrent starts take a Postgres advisory lock first. Without it two replicas rolled out together read the same empty history table, both apply the same migration, and the second dies on an object that already exists — a crash loop on the day of a schema change, which is the worst day to have one. The lock is held on a connection of its own because EF opens and closes one per command, and a session lock belongs to the connection that took it. The exception is a database that does not exist yet: there is nothing to hold a lock in, so that path migrates without one and says so. Two instances creating it at once still converges — one wins, the other restarts into the ordinary locked path — and refusing to start would leave a fresh deployment stuck on the step this removes. Database:AutoMigrate turns it off for the deployments that own their schema: a migrator job, a rollout where new code must run against the old schema first, or a database user denied DDL. With it off the behaviour is exactly what it was, and the health check now explains which of the two situations a pending migration means. Verified against a throwaway PostgreSQL container: an empty database gets all seven migrations applied before the port opens, the tables land in the dodo schema, and a second start logs the schema up to date and serves. The API suite passes, which exercises the startup path once per assembly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
jaap-jan
|
d3b14e6bc0 |
Add configuration, OIDC auth wiring and discovery endpoints (M1)
Options, JWT bearer validation, the /meta and .well-known endpoints, and a dev compose stack with Keycloak. Verified end to end: compose up, migrate, run, both discovery endpoints return correct payloads, and readiness reports the schema current. Configuration: - Strongly-typed options for Server, Oidc, Relay and Sync, all ValidateOnStart. A self-hosted server that boots half-configured and fails later per-request is far harder to diagnose than one that refuses to start and names the bad setting. - Cross-field validation the annotations cannot express: relay needs a WebSocketUrl when enabled, idle timeout must be under max session duration, item payload cap under batch cap. - Startup warnings for combinations that are individually valid but dangerous together: RequireHttpsMetadata false outside Development, and AllowEmailLinking (which turns any token bearing a victim's email into account takeover, hence default false). Auth: - JwtBearer with ClockSkew cut to 30s from the 5-minute default; five minutes of slack on a credential granting vault ciphertext access is more than any clock needs. - IncludeErrorDetails off, and a FallbackPolicy so an endpoint without an explicit policy still requires a caller rather than silently being public. Discovery, per ADR 0002: - /api/v1/meta reports versions, features and push caps. - /.well-known/dodossh-configuration is the onboarding story: the user types one server URL and the client discovers OIDC authority, client id, scopes and relay endpoint. Two environment problems found by actually running the stack: - PostgreSQL 18 changed its data mount point. Mounting /var/lib/postgresql/data — correct through 17 — makes the image refuse to start; 18+ wants a single mount at /var/lib/postgresql with the cluster in a subdirectory. - Keycloak moved to host port 18080. An unrelated Apache Tomcat on this machine holds 127.0.0.1:8080, and a loopback-specific bind beats Docker's 0.0.0.0 publish for "localhost". It presents as Keycloak 404ing every realm while its own log says the import succeeded, which is a genuinely misleading failure. Also: CA1848 is enforced, not advisory — warnings are errors, so the .editorconfig comment claiming otherwise was wrong. Startup and health logging now uses [LoggerMessage]. And a clean rebuild is back to zero warnings; the incremental build had been hiding 40 in test projects (banned Guid.NewGuid, an obsolete Testcontainers constructor, and two analyzer families that are genuinely noise under a test host). Verified: 0 warnings on a clean rebuild, 122 tests pass, format clean. |