Public Access
jaap-jan
858e36279d
The server now forgets a deletion past its retention and refuses cursors from before it. The client then reads everything again, but everything again says nothing about what went. So a machine away for longer than the retention kept those items for good, and its next edit of one put it back on the server for everyone. A read from the beginning now marks each item it reads, keeping the mark in the sync state because the read can span passes. When it finishes, it removes the items of the synced types it never marked. Two kinds are kept: items newer than the read's bound, which this machine pushed during it, and items with a change still waiting, which are the user's work. For those, the push finds the server without the item and sends it as a create, as before, but now records a RecreatedAfterCollection conflict and counts it, so the sync message says so instead of undoing someone's deletion silently. The bound is the highest sequence the cache held when the read began, or anything higher it has read since. Taking only what the read returned would have left the last item of a vault in place for ever.
96 lines
5.2 KiB
Markdown
96 lines
5.2 KiB
Markdown
# ADR 0003 — Revision-based delta sync through a single write path
|
|
|
|
- Status: accepted
|
|
- Date: 2026-07-28
|
|
|
|
## Context
|
|
|
|
Clients must work fully offline and reconcile on reconnect, across multiple devices per user.
|
|
The server holds ciphertext, so **it cannot merge, validate or inspect item contents**. Every
|
|
conflict resolution decision therefore has to happen on a client.
|
|
|
|
## Decision
|
|
|
|
### One write path
|
|
|
|
All vault mutations go through `POST /vaults/{vaultId}/sync/push`. There are no per-entity
|
|
POST/PUT/DELETE endpoints. Reads are separate, list/get only, keyset-paginated.
|
|
|
|
### Change log and cursors
|
|
|
|
A per-vault `sync_change(seq bigserial, vault_id, entity_type, entity_id, operation,
|
|
revision, actor_user_id, occurred_at_utc)` log. Each entity denormalises `change_seq` so a
|
|
delta pull joins straight to the row.
|
|
|
|
Cursors are opaque and HMAC-tagged — `base64url("v1|{vaultId}|{seq}")` — so a tampered cursor
|
|
is rejected rather than silently mis-serving someone else's data.
|
|
|
|
### Push semantics
|
|
|
|
`expectedVersion` per operation. **HTTP 200 even on partial failure**, with a per-operation
|
|
status of `Applied | Conflict | Forbidden | Invalid | Duplicate`. Conflicting operations are
|
|
skipped, not aborted, and a `Conflict` returns the server's current row so the client can
|
|
merge and re-push. `opId` deduplication via `sync_operation_receipt` makes a retried push
|
|
exactly-once at operation granularity.
|
|
|
|
Caps enforced before the transaction opens: 500 operations per push, 8 MiB per batch,
|
|
256 KiB per item.
|
|
|
|
### The `bigserial` cursor gap — the reason for the advisory lock
|
|
|
|
`bigserial` hands out values **before** commit. Transaction A takes seq 5, B takes 6 and
|
|
commits first; a reader that advances its cursor to 6 **permanently misses 5**. This is silent
|
|
sync corruption that only manifests under concurrent writes to a single vault, which is
|
|
exactly the case least likely to be exercised by hand.
|
|
|
|
Mitigation: every push takes, as its first statement,
|
|
|
|
```sql
|
|
SELECT pg_advisory_xact_lock(hashtextextended(@vaultId::text, 0))
|
|
```
|
|
|
|
This serialises writers per vault, so sequence order equals commit order. Contention is
|
|
per-vault and a push batch is already one transaction. **It requires `Multiplexing=false` in
|
|
the Npgsql connection string** — the default; do not enable multiplexing.
|
|
|
|
## Consequences
|
|
|
|
- One place enforces revision, change-log and ACL invariants. That halves both the endpoint
|
|
count and the authorization surface, which is the main reason for the single write path.
|
|
- Delta pull makes frequent polling cheap, so multi-device feels live; push notification over
|
|
SSE or the existing WebSocket can layer on with polling as the fallback. **That has since been
|
|
built — see [ADR 0012](0012-realtime-push.md)** — and nothing in this ADR changed to accommodate
|
|
it. The socket carries a notice naming a vault and a sequence, whose answer is the delta pull
|
|
above, so there is still exactly one path that applies a change; and polling is still what
|
|
guarantees a pass rather than a legacy route kept for old clients.
|
|
- Conflict resolution is entirely client-side. The client retains a `BaseCiphertext` common
|
|
ancestor and performs a field-level three-way merge for structured items, or creates a
|
|
visible conflicted copy for opaque ones. **It must never silently drop a key or a host.**
|
|
- Deletes are revisioned tombstones, garbage-collected after 90 days. Sync must therefore be
|
|
able to read tombstones, which is the one place that legitimately bypasses the soft-delete
|
|
query filter — guarded by an explicit permission check. **The collection has since been built**
|
|
(`TombstoneCollector`), and it costs a machine away for longer than the retention its news of
|
|
those deletions. So a collection raises the vault's sync horizon, and a cursor from before it is
|
|
refused. The client then reads from the beginning, and when that read finishes it removes the
|
|
items it held that the read never mentioned. An item with an unsent local edit is kept: the
|
|
push finds the server without it, puts it back, and records a `RecreatedAfterCollection`
|
|
conflict. A cursor carries the horizon it was issued under. Without that, the read from the
|
|
beginning would be refused on its own early pages, which sit behind the horizon whenever an
|
|
older item survives.
|
|
- **An `Infrastructure.Tests` case must prove cursor ordering under N concurrent pushes.**
|
|
Without it this ADR's central bug is invisible until production.
|
|
- Two concurrency mechanisms, deliberately: `version integer` is the client-visible monotonic
|
|
item version used for conflict detection; `xmin` is the server-side optimistic guard and is
|
|
**never exposed**, because it is not stable across `VACUUM FREEZE` and must not become a
|
|
client cursor.
|
|
|
|
### Rejected
|
|
|
|
- **Snapshot-watermark cursors** (`pg_snapshot_xmin(pg_current_snapshot())`). Correct without
|
|
locking, but materially harder to reason about and to test. Revisit only if per-vault lock
|
|
contention shows up in practice.
|
|
- **Last-writer-wins.** Cheap, and it loses credentials. Unacceptable for this data.
|
|
- **Full pull on every sync.** Simple, but rules out the frequent polling that makes
|
|
multi-device sync feel immediate.
|
|
- **Server-side merge.** Impossible by construction: the server cannot read the payloads.
|