Public Access
Add the encrypted local cache and the sync client
Three new client projects, and the wire-contract fix they needed. DodoSSH.Client.Domain holds the decrypted item model and the three-way merge, with no I/O at all — so the suite that decides whether a credential can be lost runs in milliseconds with nothing to mock. Scalars defer to the server on a genuine clash so every replica resolves the same triple identically and two clients cannot ping-pong; directives merge per name so two people each adding one both keep theirs; the jump chain merges as a whole value because its order is the route. Whatever loses is returned rather than dropped. DodoSSH.Client.Storage is EF Core on SQLite, no SQLCipher: the rows are already ciphertext, so an encrypted file would protect protected bytes at the cost of a native dependency. It keeps the server's state and the outbox in separate tables, which is what preserves the common ancestor a merge needs. One pending operation per item, enforced by a unique index. DodoSSH.Client.Sync is the pull/apply/push loop. Pulling never decrypts — a change with no local work pending is plumbed as ciphertext — so a first sync of thousands of items does not run twice as many AEAD operations for nothing. Contracts: EncryptedPayload gains WrappedDataKey and DataKeyId. The specification has required a per-item data key since crypto.md §3, the columns have existed since the first migration and DshAad.ItemPayload binds the id, but this record had nowhere to put either — so a spec-compliant item could not be transmitted at all. Found by writing the client that has to produce one. Also closes a hole in AadResourceType, which had no value for the HostTag and HostCredential that SyncEntityType has always listed. Four bugs the tests found, not review: - SQLite refuses to order or compare its own DateTimeOffset mapping, and throws at execution rather than model build. Collecting tombstones and listing conflicts are both that shape, so this was a crash waiting for the first user with a deleted host. Timestamps are integers now, by convention so a later field cannot be the one left unconverted. - SQLitePCLRaw 2.1.11, which EF resolves, is covered by GHSA-2m69-gcr7-jv3q. Pinned forward as a family. - Resurrecting content from a remote deletion cleared the original before queueing the copy. Two transactions, so a crash between them lost the work; reversed, and the rescued id is derived from the tombstone so a replay coalesces instead of duplicating. - Several equality assertions went through Shouldly's ShouldBe, which compares IEnumerable element-wise and so tested nothing about the Equals these types exist to provide. Corrected; the falsification that caught it went from 2 failures to 6. The push response's cursor is deliberately ignored. It sits after this client's own writes, so adopting it skips anything another client committed at a lower sequence in the window between a pull and a push — permanently. Re-reading one's own writes is idempotent and costs a page. The Contracts doc that invited the shortcut now says so. 593 tests, up from 448. The delete-versus-edit rules, the ancestor retention, the fresh operation id on coalesce and the cursor safeguard were each verified by breaking them and watching the right test fail.
This commit is contained in:
@@ -0,0 +1,429 @@
|
||||
using System.Buffers.Binary;
|
||||
using System.Security.Cryptography;
|
||||
using DodoSSH.Client.Domain;
|
||||
using DodoSSH.Client.Storage;
|
||||
using DodoSSH.Contracts;
|
||||
|
||||
namespace DodoSSH.Client.Sync;
|
||||
|
||||
/// <summary>
|
||||
/// Derives the id a resurrected item takes.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Deterministic, from the original id and the version of the tombstone that displaced it. That matters
|
||||
/// because applying a pulled change is at-least-once: the cursor is saved after the changes are applied,
|
||||
/// so a process that dies in between re-applies them on the next start. A random id would resurrect the
|
||||
/// same host twice and leave the user with duplicates to sort out; this way the second attempt produces
|
||||
/// the same id and coalesces into the same outbox row.
|
||||
/// <para>
|
||||
/// Not a UUIDv7, and that is fine — the server treats item ids as opaque, and the time ordering a v7 id
|
||||
/// carries is meaningless for a copy created to rescue content from a deletion.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
internal static class ResurrectionId
|
||||
{
|
||||
internal static Guid For(Guid entityId, int tombstoneVersion)
|
||||
{
|
||||
Span<byte> input = stackalloc byte[19 + 16 + sizeof(int)];
|
||||
|
||||
"dsh1/resurrect/v1"u8.CopyTo(input);
|
||||
var offset = 17;
|
||||
|
||||
input[offset++] = 0;
|
||||
input[offset++] = 0;
|
||||
|
||||
if (!entityId.TryWriteBytes(input[offset..], bigEndian: true, out _))
|
||||
{
|
||||
throw new InvalidOperationException("Failed to write the entity id.");
|
||||
}
|
||||
|
||||
offset += 16;
|
||||
BinaryPrimitives.WriteInt32BigEndian(input[offset..], tombstoneVersion);
|
||||
|
||||
Span<byte> digest = stackalloc byte[32];
|
||||
SHA256.HashData(input, digest);
|
||||
|
||||
return new Guid(digest[..16], bigEndian: true);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Decides what happens when a remote change collides with an unpushed local one.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// Shared by the pull and the push paths, because both meet the same six situations and must answer them
|
||||
/// identically — a pull that merged one way and a push that merged the other would make the outcome
|
||||
/// depend on which side happened to notice first.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>The governing rule is that nothing is discarded silently.</b> Where the two sides can be
|
||||
/// reconciled field by field, they are. Where they cannot, one value survives, the other is written to
|
||||
/// the conflict log verbatim, and the user is told. Where a deletion meets an edit, the edit survives:
|
||||
/// re-deleting costs a click, while a discarded edit may be the only copy of something the user cannot
|
||||
/// reconstruct.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
internal sealed class ItemReconciler(
|
||||
ItemStore items,
|
||||
OutboxStore outbox,
|
||||
ConflictStore conflicts,
|
||||
VaultKeyring keyring)
|
||||
{
|
||||
/// <summary>Reconciles a remote change against the operation pending for the same item.</summary>
|
||||
internal Task ReconcileAsync(
|
||||
Guid vaultId,
|
||||
SyncChange remote,
|
||||
PendingOperation pending,
|
||||
SyncReportBuilder report,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
if (remote.Operation == SyncOperation.Delete)
|
||||
{
|
||||
return pending.Operation == SyncOperation.Delete
|
||||
// Both sides deleted it. Nothing to arbitrate and nothing to tell the user.
|
||||
? outbox.CompleteAsync(pending.Sequence, cancellationToken)
|
||||
: ResurrectAsync(vaultId, remote, pending, report, cancellationToken);
|
||||
}
|
||||
|
||||
return pending.Operation == SyncOperation.Delete
|
||||
? AbandonLocalDeleteAsync(vaultId, remote, pending, report, cancellationToken)
|
||||
: MergeAsync(vaultId, remote, pending, report, cancellationToken);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reconciles a pending create that the server says already exists.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// In practice this means an earlier push of the same create did land and its acknowledgement was
|
||||
/// lost — a timeout, a dropped connection — after which the local row may also have been edited. The
|
||||
/// resolution adopts the server's row as the base and re-offers the local content as an update, so
|
||||
/// the newer local state wins and no duplicate host appears. A genuine id collision between two
|
||||
/// clients is the other reading, and is not achievable with UUIDv7; if it happened, the server's
|
||||
/// values would be in the conflict log rather than gone.
|
||||
/// </remarks>
|
||||
internal async Task AdoptRemoteAsBaseAsync(
|
||||
Guid vaultId,
|
||||
SyncChange remote,
|
||||
PendingOperation pending,
|
||||
SyncReportBuilder report,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
var opened = await OpenPairAsync(vaultId, remote, pending, report, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
|
||||
if (opened is null)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
var (local, remoteHost, vaultKey, generation) = opened.Value;
|
||||
|
||||
if (local == remoteHost)
|
||||
{
|
||||
// Byte-for-byte the same host: this is our own create coming back. Nothing to do but stop
|
||||
// trying to send it again.
|
||||
await outbox.CompleteAsync(pending.Sequence, cancellationToken).ConfigureAwait(false);
|
||||
return;
|
||||
}
|
||||
|
||||
await ReviseAsUpdateAsync(
|
||||
vaultId, remote, pending, local, vaultKey, generation, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
|
||||
await conflicts.RecordAsync(
|
||||
vaultId,
|
||||
SyncEntityType.Host,
|
||||
remote.EntityId,
|
||||
ConflictKind.FieldOverridden,
|
||||
ConflictDetailCodec.Encode(
|
||||
$"An item with this id already existed on the server at version {remote.Version}. "
|
||||
+ "The version from this machine was kept; the server's values are recorded here."),
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
|
||||
report.Merged++;
|
||||
}
|
||||
|
||||
/// <summary>Merges two divergent edits of the same item.</summary>
|
||||
private async Task MergeAsync(
|
||||
Guid vaultId,
|
||||
SyncChange remote,
|
||||
PendingOperation pending,
|
||||
SyncReportBuilder report,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
if (pending.Ancestor is null)
|
||||
{
|
||||
await AdoptRemoteAsBaseAsync(vaultId, remote, pending, report, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
return;
|
||||
}
|
||||
|
||||
var opened = await OpenPairAsync(vaultId, remote, pending, report, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
|
||||
if (opened is null)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
var (local, remoteHost, vaultKey, generation) = opened.Value;
|
||||
|
||||
var ancestor = HostCipher.TryOpen(
|
||||
pending.Ancestor.Payload, vaultKey.Span, remote.EntityId, pending.Ancestor.Version);
|
||||
|
||||
if (ancestor is null)
|
||||
{
|
||||
// The base is unreadable, so a three-way merge is not possible. Falling back to a two-way
|
||||
// one would have to guess which side changed what, so the honest move is to keep the local
|
||||
// state as an update over the server's and record what was overridden.
|
||||
await AdoptRemoteAsBaseAsync(vaultId, remote, pending, report, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
return;
|
||||
}
|
||||
|
||||
var merged = HostSecretMerge.Merge(ancestor.Host, local, remoteHost);
|
||||
|
||||
await ReviseAsUpdateAsync(
|
||||
vaultId, remote, pending, merged.Merged, vaultKey, generation, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
|
||||
if (merged.HasConflicts)
|
||||
{
|
||||
await conflicts.RecordAsync(
|
||||
vaultId,
|
||||
SyncEntityType.Host,
|
||||
remote.EntityId,
|
||||
ConflictKind.FieldOverridden,
|
||||
ConflictDetailCodec.Encode(
|
||||
$"'{merged.Merged.Label}' was edited in two places at once. "
|
||||
+ $"{merged.Conflicts.Count} field(s) could not be reconciled automatically.",
|
||||
merged.Conflicts),
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
report.Merged++;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Keeps local content that a remote deletion would otherwise take with it.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// The tombstone is accepted — arguing with it would conflict for ever, since a delete beats a late
|
||||
/// upsert on the server — and the local content is re-offered under a fresh id, labelled so the user
|
||||
/// can see what happened. That is the whole of "never silently drop a host": the original goes, the
|
||||
/// work does not.
|
||||
/// </remarks>
|
||||
private async Task ResurrectAsync(
|
||||
Guid vaultId,
|
||||
SyncChange remote,
|
||||
PendingOperation pending,
|
||||
SyncReportBuilder report,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
if (!keyring.TryGet(vaultId, out var vaultKey, out var generation))
|
||||
{
|
||||
await ParkAsync(vaultId, remote.EntityId, pending, report, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
return;
|
||||
}
|
||||
|
||||
var local = pending.Payload is null
|
||||
? null
|
||||
: HostCipher.TryOpen(
|
||||
pending.Payload,
|
||||
vaultKey.Span,
|
||||
remote.EntityId,
|
||||
SyncVersions.NextVersion(pending.ExpectedVersion));
|
||||
|
||||
if (local is null || local.IsReadOnly)
|
||||
{
|
||||
await ParkAsync(vaultId, remote.EntityId, pending, report, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
return;
|
||||
}
|
||||
|
||||
var restoredId = ResurrectionId.For(remote.EntityId, remote.Version);
|
||||
var restored = local.Host with { Label = $"{local.Host.Label} (restored)" };
|
||||
|
||||
// Queued before the original is cleared, and that order matters. These are two separate
|
||||
// transactions, so a process that dies between them has to fail in the direction that keeps the
|
||||
// work: this way the original stays pending and the next pass resurrects again — landing on the
|
||||
// same deterministic id, which coalesces into the row already queued. The other order would
|
||||
// leave the tombstone accepted and the local content gone.
|
||||
await outbox.QueueAsync(
|
||||
new QueuedChange(
|
||||
vaultId,
|
||||
SyncEntityType.Host,
|
||||
restoredId,
|
||||
SyncOperation.Upsert,
|
||||
ExpectedVersion: null,
|
||||
HostCipher.Seal(restored, vaultKey.Span, restoredId, generation, itemVersion: 1),
|
||||
HostFields.From(restored),
|
||||
Ancestor: null),
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
|
||||
// Now the tombstone can stand.
|
||||
await outbox.CompleteAsync(pending.Sequence, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
await conflicts.RecordAsync(
|
||||
vaultId,
|
||||
SyncEntityType.Host,
|
||||
remote.EntityId,
|
||||
ConflictKind.RemoteDeleteResurrected,
|
||||
ConflictDetailCodec.Encode(
|
||||
$"'{local.Host.Label}' was deleted elsewhere while this machine had unsaved changes. "
|
||||
+ $"The deletion stands and the local version was kept as '{restored.Label}'."),
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
|
||||
report.Resurrected++;
|
||||
}
|
||||
|
||||
/// <summary>Drops a local deletion because the other side edited the item instead.</summary>
|
||||
private async Task AbandonLocalDeleteAsync(
|
||||
Guid vaultId,
|
||||
SyncChange remote,
|
||||
PendingOperation pending,
|
||||
SyncReportBuilder report,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
await outbox.CompleteAsync(pending.Sequence, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
await conflicts.RecordAsync(
|
||||
vaultId,
|
||||
SyncEntityType.Host,
|
||||
remote.EntityId,
|
||||
ConflictKind.LocalDeleteOverridden,
|
||||
ConflictDetailCodec.Encode(
|
||||
"This host was edited elsewhere after it was deleted here, so the deletion was not "
|
||||
+ "applied. Delete it again if that is still what you want."),
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
|
||||
report.DeletesAbandoned++;
|
||||
}
|
||||
|
||||
/// <summary>Re-offers a host as an update against the server's current version.</summary>
|
||||
private async Task ReviseAsUpdateAsync(
|
||||
Guid vaultId,
|
||||
SyncChange remote,
|
||||
PendingOperation pending,
|
||||
HostSecret host,
|
||||
ReadOnlyMemory<byte> vaultKey,
|
||||
uint generation,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
var nextVersion = SyncVersions.NextVersion(remote.Version);
|
||||
|
||||
await outbox.ReviseAsync(
|
||||
pending.Sequence,
|
||||
SyncOperation.Upsert,
|
||||
expectedVersion: remote.Version,
|
||||
HostCipher.Seal(host, vaultKey.Span, remote.EntityId, generation, nextVersion),
|
||||
HostFields.From(host),
|
||||
new StoredAncestor(remote.Version, remote.Payload!, remote.PlaintextFields),
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
/// <summary>Opens both sides of a collision, parking the operation if either will not open.</summary>
|
||||
private async Task<(HostSecret Local, HostSecret Remote, ReadOnlyMemory<byte> VaultKey, uint Generation)?>
|
||||
OpenPairAsync(
|
||||
Guid vaultId,
|
||||
SyncChange remote,
|
||||
PendingOperation pending,
|
||||
SyncReportBuilder report,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
if (!keyring.TryGet(vaultId, out var vaultKey, out var generation)
|
||||
|| pending.Payload is null
|
||||
|| remote.Payload is null)
|
||||
{
|
||||
await ParkAsync(vaultId, remote.EntityId, pending, report, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
return null;
|
||||
}
|
||||
|
||||
var local = HostCipher.TryOpen(
|
||||
pending.Payload,
|
||||
vaultKey.Span,
|
||||
remote.EntityId,
|
||||
SyncVersions.NextVersion(pending.ExpectedVersion));
|
||||
|
||||
var remoteHost = HostCipher.TryOpen(
|
||||
remote.Payload, vaultKey.Span, remote.EntityId, remote.Version);
|
||||
|
||||
if (local is null || remoteHost is null)
|
||||
{
|
||||
await ParkAsync(vaultId, remote.EntityId, pending, report, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
return null;
|
||||
}
|
||||
|
||||
if (local.IsReadOnly || remoteHost.IsReadOnly)
|
||||
{
|
||||
// A newer client wrote fields this build cannot represent. Re-encoding would drop them, so
|
||||
// the item is left alone until this client is updated.
|
||||
await outbox.ParkAsync(
|
||||
pending.Sequence,
|
||||
"Written by a newer version of DodoSSH; update before editing this host.",
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
|
||||
await conflicts.RecordAsync(
|
||||
vaultId,
|
||||
SyncEntityType.Host,
|
||||
remote.EntityId,
|
||||
ConflictKind.TooNewToEdit,
|
||||
ConflictDetailCodec.Encode(
|
||||
"This host was written by a newer version of DodoSSH. It can be read but not "
|
||||
+ "merged here, because saving it would discard fields this version does not know "
|
||||
+ "about."),
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
|
||||
report.Parked++;
|
||||
return null;
|
||||
}
|
||||
|
||||
return (local.Host, remoteHost.Host, vaultKey, generation);
|
||||
}
|
||||
|
||||
private async Task ParkAsync(
|
||||
Guid vaultId,
|
||||
Guid entityId,
|
||||
PendingOperation pending,
|
||||
SyncReportBuilder report,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
await outbox.ParkAsync(
|
||||
pending.Sequence,
|
||||
"The local or the server copy of this host could not be decrypted.",
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
|
||||
await conflicts.RecordAsync(
|
||||
vaultId,
|
||||
SyncEntityType.Host,
|
||||
entityId,
|
||||
ConflictKind.Undecryptable,
|
||||
ConflictDetailCodec.Encode(
|
||||
"This host could not be decrypted, so the change made here could not be merged. "
|
||||
+ "The vault key may have been rotated, or the stored payload may not belong to this "
|
||||
+ "item."),
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
|
||||
report.Unreadable++;
|
||||
report.Parked++;
|
||||
}
|
||||
|
||||
/// <summary>Writes the server's version of an item into the local mirror.</summary>
|
||||
internal Task MirrorAsync(Guid vaultId, SyncChange change, CancellationToken cancellationToken) =>
|
||||
items.SaveAsync(
|
||||
new StoredItem(
|
||||
vaultId,
|
||||
change.EntityType,
|
||||
change.EntityId,
|
||||
change.Version,
|
||||
change.ChangeSequence,
|
||||
change.Payload,
|
||||
change.PlaintextFields,
|
||||
change.Operation == SyncOperation.Delete,
|
||||
change.UpdatedAt),
|
||||
cancellationToken);
|
||||
}
|
||||
Reference in New Issue
Block a user