Files
DodoSSH/docs/adr/0003-sync-protocol.md
T
jaap-jan ce43f397a6 Add ADRs 0001-0006 and README (M0)
Records the decisions the milestone plan already made, with their costs stated rather
than only their benefits:

- 0001 e2ee-trust-model: key hierarchy, the AAD-to-row binding that stops the server
  moving ciphertext between rows, and the four-layer public-key trust story. States
  plainly that revocation is not retroactive, that Connect cannot be a security
  boundary, and that the IdP becomes a key-distribution trust root.
- 0002 minimal-apis: feature modules with explicit registration; capability negotiation
  instead of Asp.Versioning, since client and server upgrade independently when
  self-hosted.
- 0003 sync-protocol: single write path, revision cursors, and the bigserial
  pre-commit sequence gap that silently corrupts sync — plus the per-vault advisory
  lock that fixes it and the test that must prove it.
- 0004 relay-authorization: relay forwards bytes rather than terminating SSH, so
  zero-knowledge survives; server-resolved target IPs in the ticket to defeat DNS
  rebinding; why host addresses must be plaintext when relay is enabled.
- 0005 no-application-layer: why the usual Application/mediator layer earns nothing
  here, with the trigger that would make us revisit it.
- 0006 observability-stack: OTel plus built-in ILogger; liveness excludes dependencies
  so a database blip cannot restart the container and kill live SSH sessions.

Also adds a README covering layout, build, enforced conventions and milestones.
2026-07-28 12:28:44 +02:00

4.1 KiB

ADR 0003 — Revision-based delta sync through a single write path

  • Status: accepted
  • Date: 2026-07-28

Context

Clients must work fully offline and reconcile on reconnect, across multiple devices per user. The server holds ciphertext, so it cannot merge, validate or inspect item contents. Every conflict resolution decision therefore has to happen on a client.

Decision

One write path

All vault mutations go through POST /vaults/{vaultId}/sync/push. There are no per-entity POST/PUT/DELETE endpoints. Reads are separate, list/get only, keyset-paginated.

Change log and cursors

A per-vault sync_change(seq bigserial, vault_id, entity_type, entity_id, operation, revision, actor_user_id, occurred_at_utc) log. Each entity denormalises change_seq so a delta pull joins straight to the row.

Cursors are opaque and HMAC-tagged — base64url("v1|{vaultId}|{seq}") — so a tampered cursor is rejected rather than silently mis-serving someone else's data.

Push semantics

expectedVersion per operation. HTTP 200 even on partial failure, with a per-operation status of Applied | Conflict | Forbidden | Invalid | Duplicate. Conflicting operations are skipped, not aborted, and a Conflict returns the server's current row so the client can merge and re-push. opId deduplication via sync_operation_receipt makes a retried push exactly-once at operation granularity.

Caps enforced before the transaction opens: 500 operations per push, 8 MiB per batch, 256 KiB per item.

The bigserial cursor gap — the reason for the advisory lock

bigserial hands out values before commit. Transaction A takes seq 5, B takes 6 and commits first; a reader that advances its cursor to 6 permanently misses 5. This is silent sync corruption that only manifests under concurrent writes to a single vault, which is exactly the case least likely to be exercised by hand.

Mitigation: every push takes, as its first statement,

SELECT pg_advisory_xact_lock(hashtextextended(@vaultId::text, 0))

This serialises writers per vault, so sequence order equals commit order. Contention is per-vault and a push batch is already one transaction. It requires Multiplexing=false in the Npgsql connection string — the default; do not enable multiplexing.

Consequences

  • One place enforces revision, change-log and ACL invariants. That halves both the endpoint count and the authorization surface, which is the main reason for the single write path.
  • Delta pull makes frequent polling cheap, so multi-device feels live; push notification over SSE or the existing WebSocket can layer on with polling as the fallback.
  • Conflict resolution is entirely client-side. The client retains a BaseCiphertext common ancestor and performs a field-level three-way merge for structured items, or creates a visible conflicted copy for opaque ones. It must never silently drop a key or a host.
  • Deletes are revisioned tombstones, garbage-collected after 90 days. Sync must therefore be able to read tombstones, which is the one place that legitimately bypasses the soft-delete query filter — guarded by an explicit permission check.
  • An Infrastructure.Tests case must prove cursor ordering under N concurrent pushes. Without it this ADR's central bug is invisible until production.
  • Two concurrency mechanisms, deliberately: version integer is the client-visible monotonic item version used for conflict detection; xmin is the server-side optimistic guard and is never exposed, because it is not stable across VACUUM FREEZE and must not become a client cursor.

Rejected

  • Snapshot-watermark cursors (pg_snapshot_xmin(pg_current_snapshot())). Correct without locking, but materially harder to reason about and to test. Revisit only if per-vault lock contention shows up in practice.
  • Last-writer-wins. Cheap, and it loses credentials. Unacceptable for this data.
  • Full pull on every sync. Simple, but rules out the frequent polling that makes multi-device sync feel immediate.
  • Server-side merge. Impossible by construction: the server cannot read the payloads.