Add the encrypted local cache and the sync client

Three new client projects, and the wire-contract fix they needed.

DodoSSH.Client.Domain holds the decrypted item model and the three-way
merge, with no I/O at all — so the suite that decides whether a
credential can be lost runs in milliseconds with nothing to mock.
Scalars defer to the server on a genuine clash so every replica resolves
the same triple identically and two clients cannot ping-pong; directives
merge per name so two people each adding one both keep theirs; the jump
chain merges as a whole value because its order is the route. Whatever
loses is returned rather than dropped.

DodoSSH.Client.Storage is EF Core on SQLite, no SQLCipher: the rows are
already ciphertext, so an encrypted file would protect protected bytes
at the cost of a native dependency. It keeps the server's state and the
outbox in separate tables, which is what preserves the common ancestor a
merge needs. One pending operation per item, enforced by a unique index.

DodoSSH.Client.Sync is the pull/apply/push loop. Pulling never decrypts
— a change with no local work pending is plumbed as ciphertext — so a
first sync of thousands of items does not run twice as many AEAD
operations for nothing.

Contracts: EncryptedPayload gains WrappedDataKey and DataKeyId. The
specification has required a per-item data key since crypto.md §3, the
columns have existed since the first migration and DshAad.ItemPayload
binds the id, but this record had nowhere to put either — so a
spec-compliant item could not be transmitted at all. Found by writing
the client that has to produce one. Also closes a hole in
AadResourceType, which had no value for the HostTag and HostCredential
that SyncEntityType has always listed.

Four bugs the tests found, not review:

- SQLite refuses to order or compare its own DateTimeOffset mapping, and
  throws at execution rather than model build. Collecting tombstones and
  listing conflicts are both that shape, so this was a crash waiting for
  the first user with a deleted host. Timestamps are integers now, by
  convention so a later field cannot be the one left unconverted.
- SQLitePCLRaw 2.1.11, which EF resolves, is covered by
  GHSA-2m69-gcr7-jv3q. Pinned forward as a family.
- Resurrecting content from a remote deletion cleared the original
  before queueing the copy. Two transactions, so a crash between them
  lost the work; reversed, and the rescued id is derived from the
  tombstone so a replay coalesces instead of duplicating.
- Several equality assertions went through Shouldly's ShouldBe, which
  compares IEnumerable element-wise and so tested nothing about the
  Equals these types exist to provide. Corrected; the falsification that
  caught it went from 2 failures to 6.

The push response's cursor is deliberately ignored. It sits after this
client's own writes, so adopting it skips anything another client
committed at a lower sequence in the window between a pull and a push —
permanently. Re-reading one's own writes is idempotent and costs a page.
The Contracts doc that invited the shortcut now says so.

593 tests, up from 448. The delete-versus-edit rules, the ancestor
retention, the fresh operation id on coalesce and the cursor safeguard
were each verified by breaking them and watching the right test fail.
This commit is contained in:
jaap-jan committed 2026-07-29 10:27:37 +02:00
1 parent a878c2b6bb
commit 8d2416a602
72 files changed
+11313 -30

No files matched your search

+403
View File
@@ -0,0 +1,403 @@
using DodoSSH.Contracts;
using Microsoft.EntityFrameworkCore;
namespace DodoSSH.Client.Storage;
/// <summary>A local change to queue.</summary>
/// <param name="VaultId">Owning vault.</param>
/// <param name="EntityType">Kind of item.</param>
/// <param name="EntityId">The item. Client-generated UUIDv7, so items can be made offline.</param>
/// <param name="Operation">Upsert or delete.</param>
/// <param name="ExpectedVersion">The version the client believes the server holds; null to create.</param>
/// <param name="Payload">Ciphertext. Required for an upsert.</param>
/// <param name="Fields">Plaintext columns the server needs.</param>
/// <param name="Ancestor">
/// The version this edit branched from, so a conflict can be merged rather than arbitrated. Null when
/// creating, where there is nothing to have branched from.
/// </param>
public sealed record QueuedChange(
Guid VaultId,
SyncEntityType EntityType,
Guid EntityId,
SyncOperation Operation,
int? ExpectedVersion,
EncryptedPayload? Payload,
SyncPlaintextFields? Fields,
StoredAncestor? Ancestor);
/// <summary>
/// Changes made here that the server has not yet accepted.
/// </summary>
/// <remarks>
/// <para>
/// One row per item, and that is a database constraint rather than a convention. Two queued edits to
/// one item would have to be pushed in order, and the second's <c>expectedVersion</c> is the version
/// the first will produce — which is not known when it is queued. Coalescing sidesteps that instead of
/// managing it, and the row holds a desired end state rather than a delta, so coalescing loses nothing.
/// </para>
/// <para>
/// <b>Why a coalesced row gets a new operation id.</b> The id is the server's exactly-once key. If a
/// push has already gone out and the user edits again, keeping the id would let the server answer
/// <c>Duplicate</c> for an operation whose contents have since changed — silently discarding the newer
/// edit. A fresh id means the newer state is offered on its own terms: if the earlier push did land,
/// the version has moved on, the push comes back <c>Conflict</c>, and the merge resolves it against an
/// ancestor that is this client's own earlier edit. That merge finds no disagreement, so it converges
/// on the newest state with nothing for the user to arbitrate.
/// </para>
/// </remarks>
public sealed class OutboxStore(
IDbContextFactory<ClientCacheContext> contexts,
LocalCacheProtector protector,
TimeProvider clock)
{
/// <summary>
/// Queues a change the user just made, coalescing into any row already pending for the item.
/// </summary>
/// <remarks>
/// A coalesced row keeps the ancestor and <c>expectedVersion</c> of the row it replaces, because
/// the new state is still a descendant of that same base. Taking the caller's values instead would
/// throw away the common ancestor after the first edit, and with it the ability to merge.
/// </remarks>
public async Task<PendingOperation> QueueAsync(
QueuedChange change,
CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(change);
Validate(change);
var context = contexts.CreateDbContext();
await using var scope = context.ConfigureAwait(false);
var row = await FindRowAsync(context, change.VaultId, change.EntityType, change.EntityId, cancellationToken)
.ConfigureAwait(false);
if (row is null)
{
row = new OutboxRow
{
VaultId = change.VaultId,
EntityType = change.EntityType,
EntityId = change.EntityId,
ExpectedVersion = change.ExpectedVersion,
};
SetAncestor(row, change.EntityType, change.EntityId, change.Ancestor);
context.Add(row);
}
row.OperationId = Guid.CreateVersion7();
row.Operation = change.Operation;
row.QueuedAtUtc = clock.GetUtcNow();
row.Attempts = 0;
row.LastError = null;
row.IsParked = false;
SetPayload(row, change.EntityType, change.EntityId, change.Payload, change.Fields);
await context.SaveChangesAsync(cancellationToken).ConfigureAwait(false);
return ToPending(row);
}
/// <summary>
/// Replaces a pending operation with the outcome of a merge.
/// </summary>
/// <remarks>
/// <para>
/// Distinct from <see cref="QueueAsync"/> because the intent is opposite: this <em>does</em> move
/// the ancestor forward, to the server version the merge was performed against. Without that the
/// re-push would conflict against the same base for ever.
/// </para>
/// <para>
/// It also <b>keeps the attempt count</b>, where queueing resets it. That difference is what makes
/// the retry bound real: a row that has conflicted five times needs a person to look at it whether
/// or not each attempt carried a freshly merged payload, whereas a user making a new edit has
/// genuinely started over.
/// </para>
/// </remarks>
public async Task<PendingOperation?> ReviseAsync(
long sequence,
SyncOperation operation,
int? expectedVersion,
EncryptedPayload? payload,
SyncPlaintextFields? fields,
StoredAncestor? ancestor,
CancellationToken cancellationToken)
{
var context = contexts.CreateDbContext();
await using var scope = context.ConfigureAwait(false);
var row = await context.Set<OutboxRow>()
.SingleOrDefaultAsync(r => r.Sequence == sequence, cancellationToken)
.ConfigureAwait(false);
if (row is null)
{
return null;
}
row.OperationId = Guid.CreateVersion7();
row.Operation = operation;
row.ExpectedVersion = expectedVersion;
row.LastError = null;
row.IsParked = false;
SetPayload(row, row.EntityType, row.EntityId, payload, fields);
SetAncestor(row, row.EntityType, row.EntityId, ancestor);
await context.SaveChangesAsync(cancellationToken).ConfigureAwait(false);
return ToPending(row);
}
/// <summary>Reads the next operations to push, oldest first, skipping parked ones.</summary>
public async Task<IReadOnlyList<PendingOperation>> TakeAsync(
Guid vaultId,
int limit,
CancellationToken cancellationToken)
{
ArgumentOutOfRangeException.ThrowIfLessThan(limit, 1);
var context = contexts.CreateDbContext();
await using var scope = context.ConfigureAwait(false);
var rows = await context.Set<OutboxRow>()
.AsNoTracking()
.Where(r => r.VaultId == vaultId && !r.IsParked)
.OrderBy(r => r.Sequence)
.Take(limit)
.ToListAsync(cancellationToken)
.ConfigureAwait(false);
return [.. rows.Select(ToPending)];
}
/// <summary>Reads the operation pending for one item, if any.</summary>
public async Task<PendingOperation?> FindAsync(
Guid vaultId,
SyncEntityType entityType,
Guid entityId,
CancellationToken cancellationToken)
{
var context = contexts.CreateDbContext();
await using var scope = context.ConfigureAwait(false);
var row = await FindRowAsync(context, vaultId, entityType, entityId, cancellationToken)
.ConfigureAwait(false);
return row is null ? null : ToPending(row);
}
/// <summary>
/// Reads every pending operation for a vault, parked ones included.
/// </summary>
/// <remarks>
/// What the interface needs, as opposed to what the pusher needs. A parked change is still the
/// user's current intent for that item and must be what they see; hiding it because the server
/// refused it would show them the old values and look like their edit was lost.
/// </remarks>
public async Task<IReadOnlyList<PendingOperation>> ListAllAsync(
Guid vaultId,
CancellationToken cancellationToken)
{
var context = contexts.CreateDbContext();
await using var scope = context.ConfigureAwait(false);
var rows = await context.Set<OutboxRow>()
.AsNoTracking()
.Where(r => r.VaultId == vaultId)
.OrderBy(r => r.Sequence)
.ToListAsync(cancellationToken)
.ConfigureAwait(false);
return [.. rows.Select(ToPending)];
}
/// <summary>Reads operations the server refused, which need a person.</summary>
public async Task<IReadOnlyList<PendingOperation>> ListParkedAsync(
Guid vaultId,
CancellationToken cancellationToken)
{
var context = contexts.CreateDbContext();
await using var scope = context.ConfigureAwait(false);
var rows = await context.Set<OutboxRow>()
.AsNoTracking()
.Where(r => r.VaultId == vaultId && r.IsParked)
.OrderBy(r => r.Sequence)
.ToListAsync(cancellationToken)
.ConfigureAwait(false);
return [.. rows.Select(ToPending)];
}
/// <summary>Records that an operation has been sent, so a repeated failure can be noticed.</summary>
public Task MarkDispatchedAsync(long sequence, CancellationToken cancellationToken) =>
UpdateAsync(sequence, row => row.Attempts++, cancellationToken);
/// <summary>Removes an operation the server accepted.</summary>
public async Task<bool> CompleteAsync(long sequence, CancellationToken cancellationToken)
{
var context = contexts.CreateDbContext();
await using var scope = context.ConfigureAwait(false);
var removed = await context.Set<OutboxRow>()
.Where(r => r.Sequence == sequence)
.ExecuteDeleteAsync(cancellationToken)
.ConfigureAwait(false);
return removed > 0;
}
/// <summary>
/// Stops retrying an operation and records why.
/// </summary>
/// <remarks>
/// For the answers that will not change on retry — the server calling an operation structurally
/// invalid, or the caller no longer having permission. Retrying either would spin forever and, far
/// worse, would block every change queued behind it in a vault the user can still write to.
/// </remarks>
public Task ParkAsync(long sequence, string reason, CancellationToken cancellationToken) =>
UpdateAsync(
sequence,
row =>
{
row.IsParked = true;
row.LastError = reason;
},
cancellationToken);
/// <summary>Records a transient failure without parking the operation.</summary>
public Task RecordFailureAsync(long sequence, string reason, CancellationToken cancellationToken) =>
UpdateAsync(sequence, row => row.LastError = reason, cancellationToken);
private static Task<OutboxRow?> FindRowAsync(
ClientCacheContext context,
Guid vaultId,
SyncEntityType entityType,
Guid entityId,
CancellationToken cancellationToken) =>
context.Set<OutboxRow>()
.SingleOrDefaultAsync(
r => r.VaultId == vaultId && r.EntityType == entityType && r.EntityId == entityId,
cancellationToken);
private static void Validate(QueuedChange change)
{
if (change.Operation == SyncOperation.Upsert && change.Payload is null)
{
throw new ArgumentException("An upsert requires a payload.", nameof(change));
}
if (change.Operation == SyncOperation.Unspecified)
{
throw new ArgumentException("An operation is required.", nameof(change));
}
if (change.EntityId == Guid.Empty)
{
throw new ArgumentException("An entity id is required.", nameof(change));
}
}
private async Task UpdateAsync(
long sequence,
Action<OutboxRow> mutate,
CancellationToken cancellationToken)
{
var context = contexts.CreateDbContext();
await using var scope = context.ConfigureAwait(false);
var row = await context.Set<OutboxRow>()
.SingleOrDefaultAsync(r => r.Sequence == sequence, cancellationToken)
.ConfigureAwait(false);
if (row is null)
{
return;
}
mutate(row);
await context.SaveChangesAsync(cancellationToken).ConfigureAwait(false);
}
private void SetPayload(
OutboxRow row,
SyncEntityType entityType,
Guid entityId,
EncryptedPayload? payload,
SyncPlaintextFields? fields)
{
row.Payload = payload?.Envelope;
row.WrappedDataKey = payload?.WrappedDataKey;
row.DataKeyId = payload?.DataKeyId;
row.KeyGeneration = payload?.KeyGeneration ?? 0;
row.AadVersion = payload?.AadVersion ?? 0;
row.ProtectedFields = Seal(entityType, entityId, fields);
}
private void SetAncestor(
OutboxRow row,
SyncEntityType entityType,
Guid entityId,
StoredAncestor? ancestor)
{
row.AncestorVersion = ancestor?.Version;
row.AncestorPayload = ancestor?.Payload.Envelope;
row.AncestorWrappedDataKey = ancestor?.Payload.WrappedDataKey;
row.AncestorDataKeyId = ancestor?.Payload.DataKeyId;
row.AncestorKeyGeneration = ancestor?.Payload.KeyGeneration;
row.AncestorAadVersion = ancestor?.Payload.AadVersion;
row.AncestorProtectedFields = Seal(entityType, entityId, ancestor?.Fields);
}
private byte[]? Seal(SyncEntityType entityType, Guid entityId, SyncPlaintextFields? fields) =>
fields is null
? null
: protector.Protect(
AadResourceTypes.For(entityType), entityId, PlaintextFieldsCodec.Encode(fields));
private SyncPlaintextFields? Open(SyncEntityType entityType, Guid entityId, byte[]? sealedFields)
{
if (sealedFields is null)
{
return null;
}
var plaintext = protector.TryUnprotect(
AadResourceTypes.For(entityType), entityId, sealedFields);
return plaintext is null ? null : PlaintextFieldsCodec.TryDecode(plaintext);
}
private PendingOperation ToPending(OutboxRow row)
{
var ancestorPayload = CacheMapping.ToAncestorPayload(row);
var ancestor = ancestorPayload is null || row.AncestorVersion is null
? null
: new StoredAncestor(
row.AncestorVersion.Value,
ancestorPayload,
Open(row.EntityType, row.EntityId, row.AncestorProtectedFields));
return new PendingOperation(
row.Sequence,
row.OperationId,
row.VaultId,
row.EntityType,
row.EntityId,
row.Operation,
row.ExpectedVersion,
CacheMapping.ToPayload(
row.Payload, row.WrappedDataKey, row.DataKeyId, row.KeyGeneration, row.AadVersion),
Open(row.EntityType, row.EntityId, row.ProtectedFields),
ancestor,
row.QueuedAtUtc,
row.Attempts,
row.LastError,
row.IsParked);
}
}