DodoTech-Public/DodoSSH
Public Access
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Add the encrypted local cache and the sync client

Browse Source 
Three new client projects, and the wire-contract fix they needed.

DodoSSH.Client.Domain holds the decrypted item model and the three-way
merge, with no I/O at all — so the suite that decides whether a
credential can be lost runs in milliseconds with nothing to mock.
Scalars defer to the server on a genuine clash so every replica resolves
the same triple identically and two clients cannot ping-pong; directives
merge per name so two people each adding one both keep theirs; the jump
chain merges as a whole value because its order is the route. Whatever
loses is returned rather than dropped.

DodoSSH.Client.Storage is EF Core on SQLite, no SQLCipher: the rows are
already ciphertext, so an encrypted file would protect protected bytes
at the cost of a native dependency. It keeps the server's state and the
outbox in separate tables, which is what preserves the common ancestor a
merge needs. One pending operation per item, enforced by a unique index.

DodoSSH.Client.Sync is the pull/apply/push loop. Pulling never decrypts
— a change with no local work pending is plumbed as ciphertext — so a
first sync of thousands of items does not run twice as many AEAD
operations for nothing.

Contracts: EncryptedPayload gains WrappedDataKey and DataKeyId. The
specification has required a per-item data key since crypto.md §3, the
columns have existed since the first migration and DshAad.ItemPayload
binds the id, but this record had nowhere to put either — so a
spec-compliant item could not be transmitted at all. Found by writing
the client that has to produce one. Also closes a hole in
AadResourceType, which had no value for the HostTag and HostCredential
that SyncEntityType has always listed.

Four bugs the tests found, not review:

- SQLite refuses to order or compare its own DateTimeOffset mapping, and
  throws at execution rather than model build. Collecting tombstones and
  listing conflicts are both that shape, so this was a crash waiting for
  the first user with a deleted host. Timestamps are integers now, by
  convention so a later field cannot be the one left unconverted.
- SQLitePCLRaw 2.1.11, which EF resolves, is covered by
  GHSA-2m69-gcr7-jv3q. Pinned forward as a family.
- Resurrecting content from a remote deletion cleared the original
  before queueing the copy. Two transactions, so a crash between them
  lost the work; reversed, and the rescued id is derived from the
  tombstone so a replay coalesces instead of duplicating.
- Several equality assertions went through Shouldly's ShouldBe, which
  compares IEnumerable element-wise and so tested nothing about the
  Equals these types exist to provide. Corrected; the falsification that
  caught it went from 2 failures to 6.

The push response's cursor is deliberately ignored. It sits after this
client's own writes, so adopting it skips anything another client
committed at a lower sequence in the window between a pull and a push —
permanently. Re-reading one's own writes is idempotent and costs a page.
The Contracts doc that invited the shortcut now says so.

593 tests, up from 448. The delete-versus-edit rules, the ancestor
retention, the fresh operation id on coalesce and the cursor safeguard
were each verified by breaking them and watching the right test fail.
This commit is contained in:
jaap-jan
2026-07-29 10:27:37 +02:00
parent a878c2b6bb
commit 8d2416a602
72 changed files with 11313 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/DodoSSH.Client.Sync/DodoSSH.Client.Sync.csproj
+25
View File
@@ -0,0 +1,25 @@
<Project Sdk="Microsoft.NET.Sdk">
<!--
The pull / apply / push loop, and the conflict policy it enforces.
This is the layer where data gets lost if it is wrong, so the shape is deliberate: the merge rules
live in Client.Domain with no I/O, persistence lives in Client.Storage with no policy, and this
project is the only place that decides what to do when two people edited the same host. Everything
it talks to is an interface or a store, so the conflict matrix runs against an in-memory server that
enforces real version checks rather than against HTTP.
-->
<ItemGroup>
<ProjectReference Include="../DodoSSH.Contracts/DodoSSH.Contracts.csproj" />
<ProjectReference Include="../DodoSSH.Crypto/DodoSSH.Crypto.csproj" />
<ProjectReference Include="../DodoSSH.Client.Api/DodoSSH.Client.Api.csproj" />
<ProjectReference Include="../DodoSSH.Client.Domain/DodoSSH.Client.Domain.csproj" />
<ProjectReference Include="../DodoSSH.Client.Storage/DodoSSH.Client.Storage.csproj" />
</ItemGroup>
<ItemGroup>
<InternalsVisibleTo Include="DodoSSH.Client.Sync.Tests" />
</ItemGroup>
</Project>
src/DodoSSH.Client.Sync/HostCipher.cs
+176
View File
@@ -0,0 +1,176 @@
using System.Security.Cryptography;
using DodoSSH.Client.Domain;
using DodoSSH.Contracts;
using DodoSSH.Crypto;
namespace DodoSSH.Client.Sync;
/// <summary>
/// Turns a host into an item payload and back.
/// </summary>
/// <remarks>
/// <para>
/// Every operation binds to the item's identity, its key generation <em>and</em> its version, because
/// that is what <c>DshAad.ItemPayload</c> requires. The version part has a consequence worth stating
/// plainly: a payload must be sealed at the version the server will assign, not the version it is
/// replacing. See <see cref="SyncVersions"/>.
/// </para>
/// <para>
/// The data key is fresh per call and is zeroed before returning, as is the encoded plaintext. Neither
/// is ever handed to a caller: a data key that escaped this class would be a data key some other layer
/// could forget to clear.
/// </para>
/// </remarks>
public static class HostCipher
{
private const CryptoSpec.AadResourceType Resource = CryptoSpec.AadResourceType.Host;
/// <summary>
/// Encrypts a host.
/// </summary>
/// <param name="host">The host. Must be valid for storage.</param>
/// <param name="vaultKey">The vault key, which the data key is wrapped under.</param>
/// <param name="entityId">The item id, which the AAD binds.</param>
/// <param name="keyGeneration">The vault's current key generation.</param>
/// <param name="itemVersion">
/// The version this payload will hold once the server accepts it — one more than the version being
/// replaced. Sealing at the version being replaced would produce a payload that authenticates
/// against a row that no longer exists, and the item would read as corrupt from then on.
/// </param>
public static EncryptedPayload Seal(
HostSecret host,
ReadOnlySpan<byte> vaultKey,
Guid entityId,
uint keyGeneration,
int itemVersion)
{
ArgumentNullException.ThrowIfNull(host);
ArgumentOutOfRangeException.ThrowIfLessThan(itemVersion, 1);
var plaintext = HostSecretCodec.Encode(host);
var dataKey = ItemKeys.CreateDataKey();
try
{
var dataKeyId = Guid.CreateVersion7();
var wrappedDataKey = ItemKeys.WrapDataKey(
dataKey, vaultKey, Resource, entityId, keyGeneration, (uint)itemVersion);
var envelope = ItemKeys.SealPayload(
dataKey, plaintext, Resource, entityId, dataKeyId, keyGeneration, (uint)itemVersion);
return new EncryptedPayload(
envelope, wrappedDataKey, dataKeyId, keyGeneration, CryptoSpec.CurrentAadVersion);
}
finally
{
CryptographicOperations.ZeroMemory(dataKey);
CryptographicOperations.ZeroMemory(plaintext);
}
}
/// <summary>
/// Decrypts a host.
/// </summary>
/// <returns>
/// The host and the schema version it was written at, or <see langword="null"/> if the payload does
/// not belong to this item, version or generation, or does not parse.
/// <para>
/// A null is a meaningful outcome, not an error to be thrown past. It is what a server relocating
/// ciphertext between rows looks like from here, and it is also what an ordinary rekey looks like
/// before new grants arrive. The caller distinguishes them by comparing generations; either way one
/// unreadable item must not abort a sync pass and strand every change behind it.
/// </para>
/// </returns>
public static HostSecretDocument? TryOpen(
EncryptedPayload payload,
ReadOnlySpan<byte> vaultKey,
Guid entityId,
int itemVersion)
{
ArgumentNullException.ThrowIfNull(payload);
if (itemVersion < 1 || payload.WrappedDataKey.Length == 0)
{
return null;
}
var dataKey = ItemKeys.TryUnwrapDataKey(
vaultKey,
payload.WrappedDataKey,
Resource,
entityId,
payload.KeyGeneration,
(uint)itemVersion);
if (dataKey is null)
{
return null;
}
try
{
var plaintext = ItemKeys.TryOpenPayload(
dataKey,
payload.Envelope,
Resource,
entityId,
payload.DataKeyId,
payload.KeyGeneration,
(uint)itemVersion);
if (plaintext is null)
{
return null;
}
try
{
return HostSecretCodec.TryDecode(plaintext, out var document) ? document : null;
}
finally
{
CryptographicOperations.ZeroMemory(plaintext);
}
}
finally
{
CryptographicOperations.ZeroMemory(dataKey);
}
}
}
/// <summary>
/// The one place that decides which item version a payload is sealed at.
/// </summary>
/// <remarks>
/// The payload's AAD binds the item version, so the sealing side has to predict what the server will
/// assign. That prediction is safe because it is checked: the server applies an upsert only when
/// <c>expectedVersion</c> matches, and then increments by exactly one. A mismatch is a conflict, not a
/// silently mis-sealed row. Both the sealing and the opening sides go through here, so they cannot
/// drift — the failure if they did would be an item that encrypts fine and never decrypts again.
/// </remarks>
internal static class SyncVersions
{
/// <summary>The version an accepted upsert will produce.</summary>
/// <param name="expectedVersion">The version being replaced, or null for a create.</param>
internal static int NextVersion(int? expectedVersion) => (expectedVersion ?? 0) + 1;
}
/// <summary>
/// Derives the plaintext columns the server needs from a host.
/// </summary>
/// <remarks>
/// The single point at which a hostname can leave the encrypted payload, which is the whole reason it
/// is a function rather than something each call site assembles. The address is emitted only when the
/// user has turned the relay on for that host; with relay off, the server learns nothing but that an
/// item exists. See ADR 0004 for why the relay cannot work any other way.
/// </remarks>
internal static class HostFields
{
internal static SyncPlaintextFields From(HostSecret host) =>
host.RelayEnabled
? new SyncPlaintextFields(RelayEnabled: true, Hostname: host.Hostname, Port: host.Port)
: new SyncPlaintextFields();
}
src/DodoSSH.Client.Sync/HostRepository.cs
+299
View File
@@ -0,0 +1,299 @@
using DodoSSH.Client.Domain;
using DodoSSH.Client.Storage;
using DodoSSH.Contracts;
namespace DodoSSH.Client.Sync;
/// <summary>A host as the interface should show it.</summary>
/// <param name="EntityId">The item id.</param>
/// <param name="Host">The decrypted host.</param>
/// <param name="Version">
/// The server version this is based on. Zero for an item that has never been accepted.
/// </param>
/// <param name="HasUnsyncedChanges">
/// Whether this reflects a local edit the server has not accepted yet. Worth showing: it is the
/// difference between "saved" and "saved here".
/// </param>
/// <param name="IsBlocked">
/// Whether the pending change was refused and is waiting on a person, so it will not retry on its own.
/// </param>
/// <param name="IsReadOnly">
/// Whether this host was written by a newer client and so must not be edited here, because re-encoding
/// it would drop fields this build cannot represent.
/// </param>
public sealed record VaultHost(
Guid EntityId,
HostSecret Host,
int Version,
bool HasUnsyncedChanges,
bool IsBlocked,
bool IsReadOnly);
/// <summary>The hosts in a vault, and what could not be read.</summary>
/// <param name="Hosts">The readable hosts, newest change last.</param>
/// <param name="Unreadable">
/// How many items would not decrypt. Surfaced rather than swallowed: a non-zero count here after a
/// rekey is the signal that new grants are needed.
/// </param>
public sealed record HostListing(IReadOnlyList<VaultHost> Hosts, int Unreadable);
/// <summary>
/// Reading and writing hosts, as the interface sees them.
/// </summary>
/// <remarks>
/// <para>
/// The view is the mirror of the server's state with the outbox laid over it, which is what makes the
/// application feel local: an edit appears immediately and a delete disappears immediately, whether or
/// not the network is there. Nothing here talks to the server; the sync engine reconciles later.
/// </para>
/// <para>
/// Writes never touch the mirror. That separation is load-bearing — the mirror is the common ancestor a
/// three-way merge needs, and a repository that updated it on save would destroy the very state that
/// lets a conflict be merged instead of arbitrated.
/// </para>
/// </remarks>
public sealed class HostRepository(ItemStore items, OutboxStore outbox, VaultKeyring keyring)
{
/// <summary>Reads every host the user should see in a vault.</summary>
public async Task<HostListing> ListAsync(Guid vaultId, CancellationToken cancellationToken)
{
if (!keyring.TryGet(vaultId, out var vaultKey, out _))
{
throw new VaultUnreadableException(vaultId);
}
var mirrored = await items
.ListAsync(vaultId, SyncEntityType.Host, includeDeleted: true, cancellationToken)
.ConfigureAwait(false);
var pending = await outbox.ListAllAsync(vaultId, cancellationToken).ConfigureAwait(false);
var pendingByEntity = pending
.Where(operation => operation.EntityType == SyncEntityType.Host)
.ToDictionary(operation => operation.EntityId);
var hosts = new List<VaultHost>();
var unreadable = 0;
foreach (var item in mirrored)
{
if (pendingByEntity.Remove(item.EntityId, out var local))
{
AddPending(hosts, ref unreadable, vaultKey, local);
continue;
}
if (item.IsDeleted || item.Payload is null)
{
continue;
}
var opened = HostCipher.TryOpen(item.Payload, vaultKey.Span, item.EntityId, item.Version);
if (opened is null)
{
unreadable++;
continue;
}
hosts.Add(new VaultHost(
item.EntityId, opened.Host, item.Version, false, false, opened.IsReadOnly));
}
// Whatever is left has no mirror row yet: items created here and not yet accepted.
foreach (var local in pendingByEntity.Values)
{
AddPending(hosts, ref unreadable, vaultKey, local);
}
return new HostListing(hosts, unreadable);
}
/// <summary>
/// Adds a host, returning the id it was given.
/// </summary>
/// <remarks>
/// The id is generated here, not by the server, which is what lets a host be created with no network
/// at all — the point of the whole outbox. UUIDv7 so that ids sort by creation time, which keeps
/// index locality reasonable on the server side.
/// </remarks>
public async Task<Guid> CreateAsync(
Guid vaultId,
HostSecret host,
CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(host);
Validate(host);
var (vaultKey, generation) = Key(vaultId);
var entityId = Guid.CreateVersion7();
await outbox.QueueAsync(
new QueuedChange(
vaultId,
SyncEntityType.Host,
entityId,
SyncOperation.Upsert,
ExpectedVersion: null,
HostCipher.Seal(host, vaultKey.Span, entityId, generation, itemVersion: 1),
HostFields.From(host),
Ancestor: null),
cancellationToken).ConfigureAwait(false);
return entityId;
}
/// <summary>
/// Replaces a host's contents.
/// </summary>
/// <remarks>
/// The base is taken from the pending operation when there is one, and from the mirror otherwise.
/// Reading it the other way round would seal the payload at a version that does not match the
/// <c>expectedVersion</c> the coalesced row keeps — and because the AAD binds the item version, the
/// result would encrypt cleanly and never decrypt again.
/// </remarks>
public async Task UpdateAsync(
Guid vaultId,
Guid entityId,
HostSecret host,
CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(host);
Validate(host);
var (vaultKey, generation) = Key(vaultId);
var pending = await outbox
.FindAsync(vaultId, SyncEntityType.Host, entityId, cancellationToken)
.ConfigureAwait(false);
var expectedVersion = pending is not null
? pending.ExpectedVersion
: await MirrorVersionAsync(vaultId, entityId, cancellationToken).ConfigureAwait(false);
var ancestor = pending?.Ancestor
?? await MirrorAncestorAsync(vaultId, entityId, cancellationToken).ConfigureAwait(false);
await outbox.QueueAsync(
new QueuedChange(
vaultId,
SyncEntityType.Host,
entityId,
SyncOperation.Upsert,
expectedVersion,
HostCipher.Seal(
host, vaultKey.Span, entityId, generation, SyncVersions.NextVersion(expectedVersion)),
HostFields.From(host),
ancestor),
cancellationToken).ConfigureAwait(false);
}
/// <summary>
/// Deletes a host.
/// </summary>
/// <remarks>
/// Queued as a tombstone, never a local removal. An offline client that simply forgot the row would
/// be unable to tell the server anything, and the item would come back on the next pull.
/// </remarks>
public async Task DeleteAsync(Guid vaultId, Guid entityId, CancellationToken cancellationToken)
{
var pending = await outbox
.FindAsync(vaultId, SyncEntityType.Host, entityId, cancellationToken)
.ConfigureAwait(false);
var expectedVersion = pending is not null
? pending.ExpectedVersion
: await MirrorVersionAsync(vaultId, entityId, cancellationToken).ConfigureAwait(false);
var ancestor = pending?.Ancestor
?? await MirrorAncestorAsync(vaultId, entityId, cancellationToken).ConfigureAwait(false);
await outbox.QueueAsync(
new QueuedChange(
vaultId,
SyncEntityType.Host,
entityId,
SyncOperation.Delete,
expectedVersion,
Payload: null,
Fields: null,
ancestor),
cancellationToken).ConfigureAwait(false);
}
private static void Validate(HostSecret host)
{
if (!host.TryValidate(out var error))
{
throw new ArgumentException(error, nameof(host));
}
}
private static void AddPending(
List<VaultHost> hosts,
ref int unreadable,
ReadOnlyMemory<byte> vaultKey,
PendingOperation local)
{
if (local.Operation == SyncOperation.Delete)
{
// Gone as far as this machine is concerned, even before the server agrees.
return;
}
if (local.Payload is null)
{
unreadable++;
return;
}
var version = SyncVersions.NextVersion(local.ExpectedVersion);
var opened = HostCipher.TryOpen(local.Payload, vaultKey.Span, local.EntityId, version);
if (opened is null)
{
unreadable++;
return;
}
hosts.Add(new VaultHost(
local.EntityId,
opened.Host,
local.ExpectedVersion ?? 0,
HasUnsyncedChanges: true,
local.IsParked,
opened.IsReadOnly));
}
private (ReadOnlyMemory<byte> VaultKey, uint Generation) Key(Guid vaultId) =>
keyring.TryGet(vaultId, out var vaultKey, out var generation)
? (vaultKey, generation)
: throw new VaultUnreadableException(vaultId);
private async Task<int?> MirrorVersionAsync(
Guid vaultId,
Guid entityId,
CancellationToken cancellationToken)
{
var item = await items
.FindAsync(vaultId, SyncEntityType.Host, entityId, cancellationToken)
.ConfigureAwait(false);
// A null means the server has never seen this item, which is exactly what "create" is.
return item?.Version;
}
private async Task<StoredAncestor?> MirrorAncestorAsync(
Guid vaultId,
Guid entityId,
CancellationToken cancellationToken)
{
var item = await items
.FindAsync(vaultId, SyncEntityType.Host, entityId, cancellationToken)
.ConfigureAwait(false);
return item?.Payload is null
? null
: new StoredAncestor(item.Version, item.Payload, item.Fields);
}
}
src/DodoSSH.Client.Sync/ItemReconciler.cs
+429
View File
@@ -0,0 +1,429 @@
using System.Buffers.Binary;
using System.Security.Cryptography;
using DodoSSH.Client.Domain;
using DodoSSH.Client.Storage;
using DodoSSH.Contracts;
namespace DodoSSH.Client.Sync;
/// <summary>
/// Derives the id a resurrected item takes.
/// </summary>
/// <remarks>
/// Deterministic, from the original id and the version of the tombstone that displaced it. That matters
/// because applying a pulled change is at-least-once: the cursor is saved after the changes are applied,
/// so a process that dies in between re-applies them on the next start. A random id would resurrect the
/// same host twice and leave the user with duplicates to sort out; this way the second attempt produces
/// the same id and coalesces into the same outbox row.
/// <para>
/// Not a UUIDv7, and that is fine — the server treats item ids as opaque, and the time ordering a v7 id
/// carries is meaningless for a copy created to rescue content from a deletion.
/// </para>
/// </remarks>
internal static class ResurrectionId
{
internal static Guid For(Guid entityId, int tombstoneVersion)
{
Span<byte> input = stackalloc byte[19 + 16 + sizeof(int)];
"dsh1/resurrect/v1"u8.CopyTo(input);
var offset = 17;
input[offset++] = 0;
input[offset++] = 0;
if (!entityId.TryWriteBytes(input[offset..], bigEndian: true, out _))
{
throw new InvalidOperationException("Failed to write the entity id.");
}
offset += 16;
BinaryPrimitives.WriteInt32BigEndian(input[offset..], tombstoneVersion);
Span<byte> digest = stackalloc byte[32];
SHA256.HashData(input, digest);
return new Guid(digest[..16], bigEndian: true);
}
}
/// <summary>
/// Decides what happens when a remote change collides with an unpushed local one.
/// </summary>
/// <remarks>
/// <para>
/// Shared by the pull and the push paths, because both meet the same six situations and must answer them
/// identically — a pull that merged one way and a push that merged the other would make the outcome
/// depend on which side happened to notice first.
/// </para>
/// <para>
/// <b>The governing rule is that nothing is discarded silently.</b> Where the two sides can be
/// reconciled field by field, they are. Where they cannot, one value survives, the other is written to
/// the conflict log verbatim, and the user is told. Where a deletion meets an edit, the edit survives:
/// re-deleting costs a click, while a discarded edit may be the only copy of something the user cannot
/// reconstruct.
/// </para>
/// </remarks>
internal sealed class ItemReconciler(
ItemStore items,
OutboxStore outbox,
ConflictStore conflicts,
VaultKeyring keyring)
{
/// <summary>Reconciles a remote change against the operation pending for the same item.</summary>
internal Task ReconcileAsync(
Guid vaultId,
SyncChange remote,
PendingOperation pending,
SyncReportBuilder report,
CancellationToken cancellationToken)
{
if (remote.Operation == SyncOperation.Delete)
{
return pending.Operation == SyncOperation.Delete
// Both sides deleted it. Nothing to arbitrate and nothing to tell the user.
? outbox.CompleteAsync(pending.Sequence, cancellationToken)
: ResurrectAsync(vaultId, remote, pending, report, cancellationToken);
}
return pending.Operation == SyncOperation.Delete
? AbandonLocalDeleteAsync(vaultId, remote, pending, report, cancellationToken)
: MergeAsync(vaultId, remote, pending, report, cancellationToken);
}
/// <summary>
/// Reconciles a pending create that the server says already exists.
/// </summary>
/// <remarks>
/// In practice this means an earlier push of the same create did land and its acknowledgement was
/// lost — a timeout, a dropped connection — after which the local row may also have been edited. The
/// resolution adopts the server's row as the base and re-offers the local content as an update, so
/// the newer local state wins and no duplicate host appears. A genuine id collision between two
/// clients is the other reading, and is not achievable with UUIDv7; if it happened, the server's
/// values would be in the conflict log rather than gone.
/// </remarks>
internal async Task AdoptRemoteAsBaseAsync(
Guid vaultId,
SyncChange remote,
PendingOperation pending,
SyncReportBuilder report,
CancellationToken cancellationToken)
{
var opened = await OpenPairAsync(vaultId, remote, pending, report, cancellationToken)
.ConfigureAwait(false);
if (opened is null)
{
return;
}
var (local, remoteHost, vaultKey, generation) = opened.Value;
if (local == remoteHost)
{
// Byte-for-byte the same host: this is our own create coming back. Nothing to do but stop
// trying to send it again.
await outbox.CompleteAsync(pending.Sequence, cancellationToken).ConfigureAwait(false);
return;
}
await ReviseAsUpdateAsync(
vaultId, remote, pending, local, vaultKey, generation, cancellationToken)
.ConfigureAwait(false);
await conflicts.RecordAsync(
vaultId,
SyncEntityType.Host,
remote.EntityId,
ConflictKind.FieldOverridden,
ConflictDetailCodec.Encode(
$"An item with this id already existed on the server at version {remote.Version}. "
+ "The version from this machine was kept; the server's values are recorded here."),
cancellationToken).ConfigureAwait(false);
report.Merged++;
}
/// <summary>Merges two divergent edits of the same item.</summary>
private async Task MergeAsync(
Guid vaultId,
SyncChange remote,
PendingOperation pending,
SyncReportBuilder report,
CancellationToken cancellationToken)
{
if (pending.Ancestor is null)
{
await AdoptRemoteAsBaseAsync(vaultId, remote, pending, report, cancellationToken)
.ConfigureAwait(false);
return;
}
var opened = await OpenPairAsync(vaultId, remote, pending, report, cancellationToken)
.ConfigureAwait(false);
if (opened is null)
{
return;
}
var (local, remoteHost, vaultKey, generation) = opened.Value;
var ancestor = HostCipher.TryOpen(
pending.Ancestor.Payload, vaultKey.Span, remote.EntityId, pending.Ancestor.Version);
if (ancestor is null)
{
// The base is unreadable, so a three-way merge is not possible. Falling back to a two-way
// one would have to guess which side changed what, so the honest move is to keep the local
// state as an update over the server's and record what was overridden.
await AdoptRemoteAsBaseAsync(vaultId, remote, pending, report, cancellationToken)
.ConfigureAwait(false);
return;
}
var merged = HostSecretMerge.Merge(ancestor.Host, local, remoteHost);
await ReviseAsUpdateAsync(
vaultId, remote, pending, merged.Merged, vaultKey, generation, cancellationToken)
.ConfigureAwait(false);
if (merged.HasConflicts)
{
await conflicts.RecordAsync(
vaultId,
SyncEntityType.Host,
remote.EntityId,
ConflictKind.FieldOverridden,
ConflictDetailCodec.Encode(
$"'{merged.Merged.Label}' was edited in two places at once. "
+ $"{merged.Conflicts.Count} field(s) could not be reconciled automatically.",
merged.Conflicts),
cancellationToken).ConfigureAwait(false);
}
report.Merged++;
}
/// <summary>
/// Keeps local content that a remote deletion would otherwise take with it.
/// </summary>
/// <remarks>
/// The tombstone is accepted — arguing with it would conflict for ever, since a delete beats a late
/// upsert on the server — and the local content is re-offered under a fresh id, labelled so the user
/// can see what happened. That is the whole of "never silently drop a host": the original goes, the
/// work does not.
/// </remarks>
private async Task ResurrectAsync(
Guid vaultId,
SyncChange remote,
PendingOperation pending,
SyncReportBuilder report,
CancellationToken cancellationToken)
{
if (!keyring.TryGet(vaultId, out var vaultKey, out var generation))
{
await ParkAsync(vaultId, remote.EntityId, pending, report, cancellationToken)
.ConfigureAwait(false);
return;
}
var local = pending.Payload is null
? null
: HostCipher.TryOpen(
pending.Payload,
vaultKey.Span,
remote.EntityId,
SyncVersions.NextVersion(pending.ExpectedVersion));
if (local is null || local.IsReadOnly)
{
await ParkAsync(vaultId, remote.EntityId, pending, report, cancellationToken)
.ConfigureAwait(false);
return;
}
var restoredId = ResurrectionId.For(remote.EntityId, remote.Version);
var restored = local.Host with { Label = $"{local.Host.Label} (restored)" };
// Queued before the original is cleared, and that order matters. These are two separate
// transactions, so a process that dies between them has to fail in the direction that keeps the
// work: this way the original stays pending and the next pass resurrects again — landing on the
// same deterministic id, which coalesces into the row already queued. The other order would
// leave the tombstone accepted and the local content gone.
await outbox.QueueAsync(
new QueuedChange(
vaultId,
SyncEntityType.Host,
restoredId,
SyncOperation.Upsert,
ExpectedVersion: null,
HostCipher.Seal(restored, vaultKey.Span, restoredId, generation, itemVersion: 1),
HostFields.From(restored),
Ancestor: null),
cancellationToken).ConfigureAwait(false);
// Now the tombstone can stand.
await outbox.CompleteAsync(pending.Sequence, cancellationToken).ConfigureAwait(false);
await conflicts.RecordAsync(
vaultId,
SyncEntityType.Host,
remote.EntityId,
ConflictKind.RemoteDeleteResurrected,
ConflictDetailCodec.Encode(
$"'{local.Host.Label}' was deleted elsewhere while this machine had unsaved changes. "
+ $"The deletion stands and the local version was kept as '{restored.Label}'."),
cancellationToken).ConfigureAwait(false);
report.Resurrected++;
}
/// <summary>Drops a local deletion because the other side edited the item instead.</summary>
private async Task AbandonLocalDeleteAsync(
Guid vaultId,
SyncChange remote,
PendingOperation pending,
SyncReportBuilder report,
CancellationToken cancellationToken)
{
await outbox.CompleteAsync(pending.Sequence, cancellationToken).ConfigureAwait(false);
await conflicts.RecordAsync(
vaultId,
SyncEntityType.Host,
remote.EntityId,
ConflictKind.LocalDeleteOverridden,
ConflictDetailCodec.Encode(
"This host was edited elsewhere after it was deleted here, so the deletion was not "
+ "applied. Delete it again if that is still what you want."),
cancellationToken).ConfigureAwait(false);
report.DeletesAbandoned++;
}
/// <summary>Re-offers a host as an update against the server's current version.</summary>
private async Task ReviseAsUpdateAsync(
Guid vaultId,
SyncChange remote,
PendingOperation pending,
HostSecret host,
ReadOnlyMemory<byte> vaultKey,
uint generation,
CancellationToken cancellationToken)
{
var nextVersion = SyncVersions.NextVersion(remote.Version);
await outbox.ReviseAsync(
pending.Sequence,
SyncOperation.Upsert,
expectedVersion: remote.Version,
HostCipher.Seal(host, vaultKey.Span, remote.EntityId, generation, nextVersion),
HostFields.From(host),
new StoredAncestor(remote.Version, remote.Payload!, remote.PlaintextFields),
cancellationToken).ConfigureAwait(false);
}
/// <summary>Opens both sides of a collision, parking the operation if either will not open.</summary>
private async Task<(HostSecret Local, HostSecret Remote, ReadOnlyMemory<byte> VaultKey, uint Generation)?>
OpenPairAsync(
Guid vaultId,
SyncChange remote,
PendingOperation pending,
SyncReportBuilder report,
CancellationToken cancellationToken)
{
if (!keyring.TryGet(vaultId, out var vaultKey, out var generation)
|| pending.Payload is null
|| remote.Payload is null)
{
await ParkAsync(vaultId, remote.EntityId, pending, report, cancellationToken)
.ConfigureAwait(false);
return null;
}
var local = HostCipher.TryOpen(
pending.Payload,
vaultKey.Span,
remote.EntityId,
SyncVersions.NextVersion(pending.ExpectedVersion));
var remoteHost = HostCipher.TryOpen(
remote.Payload, vaultKey.Span, remote.EntityId, remote.Version);
if (local is null || remoteHost is null)
{
await ParkAsync(vaultId, remote.EntityId, pending, report, cancellationToken)
.ConfigureAwait(false);
return null;
}
if (local.IsReadOnly || remoteHost.IsReadOnly)
{
// A newer client wrote fields this build cannot represent. Re-encoding would drop them, so
// the item is left alone until this client is updated.
await outbox.ParkAsync(
pending.Sequence,
"Written by a newer version of DodoSSH; update before editing this host.",
cancellationToken).ConfigureAwait(false);
await conflicts.RecordAsync(
vaultId,
SyncEntityType.Host,
remote.EntityId,
ConflictKind.TooNewToEdit,
ConflictDetailCodec.Encode(
"This host was written by a newer version of DodoSSH. It can be read but not "
+ "merged here, because saving it would discard fields this version does not know "
+ "about."),
cancellationToken).ConfigureAwait(false);
report.Parked++;
return null;
}
return (local.Host, remoteHost.Host, vaultKey, generation);
}
private async Task ParkAsync(
Guid vaultId,
Guid entityId,
PendingOperation pending,
SyncReportBuilder report,
CancellationToken cancellationToken)
{
await outbox.ParkAsync(
pending.Sequence,
"The local or the server copy of this host could not be decrypted.",
cancellationToken).ConfigureAwait(false);
await conflicts.RecordAsync(
vaultId,
SyncEntityType.Host,
entityId,
ConflictKind.Undecryptable,
ConflictDetailCodec.Encode(
"This host could not be decrypted, so the change made here could not be merged. "
+ "The vault key may have been rotated, or the stored payload may not belong to this "
+ "item."),
cancellationToken).ConfigureAwait(false);
report.Unreadable++;
report.Parked++;
}
/// <summary>Writes the server's version of an item into the local mirror.</summary>
internal Task MirrorAsync(Guid vaultId, SyncChange change, CancellationToken cancellationToken) =>
items.SaveAsync(
new StoredItem(
vaultId,
change.EntityType,
change.EntityId,
change.Version,
change.ChangeSequence,
change.Payload,
change.PlaintextFields,
change.Operation == SyncOperation.Delete,
change.UpdatedAt),
cancellationToken);
}
src/DodoSSH.Client.Sync/SyncEngine.cs
+487
View File
@@ -0,0 +1,487 @@
using System.Globalization;
using System.Runtime.InteropServices;
using DodoSSH.Client.Api;
using DodoSSH.Client.Storage;
using DodoSSH.Contracts;
namespace DodoSSH.Client.Sync;
/// <summary>
/// One vault's synchronisation pass: pull, reconcile, push, pull again.
/// </summary>
/// <remarks>
/// <para>
/// Pull first, so a local change is merged against the newest server state before it is offered — which
/// turns most would-be conflicts into ordinary merges and keeps the push round count down. Push second.
/// Pull once more at the end only if something was pushed, so the mirror reflects the versions the
/// server actually assigned.
/// </para>
/// <para>
/// <b>Pulling does not decrypt.</b> A change with no local work pending is copied into the mirror as
/// ciphertext and nothing more. Decryption happens when a merge needs it, or when the interface reads an
/// item. For a five-thousand-item first sync that is the difference between plumbing bytes and running
/// ten thousand AEAD operations for nothing.
/// </para>
/// </remarks>
public sealed class SyncEngine
{
private readonly ISyncApi api;
private readonly ItemStore items;
private readonly OutboxStore outbox;
private readonly SyncStateStore syncState;
private readonly ConflictStore conflicts;
private readonly VaultKeyring keyring;
private readonly TimeProvider clock;
private readonly SyncOptions options;
private readonly ItemReconciler reconciler;
/// <summary>Creates the engine.</summary>
public SyncEngine(
ISyncApi api,
ItemStore items,
OutboxStore outbox,
SyncStateStore syncState,
ConflictStore conflicts,
VaultKeyring keyring,
TimeProvider clock,
SyncOptions? options = null)
{
ArgumentNullException.ThrowIfNull(api);
ArgumentNullException.ThrowIfNull(items);
ArgumentNullException.ThrowIfNull(outbox);
ArgumentNullException.ThrowIfNull(syncState);
ArgumentNullException.ThrowIfNull(conflicts);
ArgumentNullException.ThrowIfNull(keyring);
ArgumentNullException.ThrowIfNull(clock);
this.api = api;
this.items = items;
this.outbox = outbox;
this.syncState = syncState;
this.conflicts = conflicts;
this.keyring = keyring;
this.clock = clock;
this.options = options ?? SyncOptions.Default;
reconciler = new ItemReconciler(items, outbox, conflicts, keyring);
}
/// <summary>Runs a full pass over one vault.</summary>
public async Task<SyncReport> SyncAsync(Guid vaultId, CancellationToken cancellationToken)
{
var report = new SyncReportBuilder(vaultId);
await PullAsync(vaultId, report, cancellationToken).ConfigureAwait(false);
var pushedAnything = false;
for (var round = 1; ; round++)
{
var outcome = await DrainAsync(vaultId, report, cancellationToken).ConfigureAwait(false);
if (outcome.Sent == 0)
{
break;
}
pushedAnything = true;
if (!outcome.NeedsAnotherRound)
{
break;
}
if (round >= options.MaxPushRounds)
{
report.RoundsExhausted = true;
break;
}
}
if (pushedAnything)
{
await PullAsync(vaultId, report, cancellationToken).ConfigureAwait(false);
}
return report.Build();
}
/// <summary>
/// Reads every change available and applies it.
/// </summary>
/// <remarks>
/// The cursor is saved after each page's changes are applied, which makes applying at-least-once
/// rather than exactly-once: a process that dies between the two re-reads that page next time. That
/// is deliberate and safe, because applying a change is a blind overwrite of a mirror row and a
/// resurrection takes a deterministic id. The other ordering — save the cursor first — would lose
/// changes outright, which no amount of idempotence can repair.
/// </remarks>
private async Task PullAsync(
Guid vaultId,
SyncReportBuilder report,
CancellationToken cancellationToken)
{
var state = await syncState.ReadAsync(vaultId, cancellationToken).ConfigureAwait(false);
for (var page = 0; page < options.MaxPullPages; page++)
{
var response = await api.SyncPullAsync(
vaultId,
new SyncPullRequest(state.Cursor, options.PullPageSize, [SyncEntityType.Host]),
cancellationToken).ConfigureAwait(false);
foreach (var change in response.Changes)
{
await ApplyAsync(vaultId, change, report, cancellationToken).ConfigureAwait(false);
report.Pulled++;
}
var advanced = !string.Equals(state.Cursor, response.NextCursor, StringComparison.Ordinal);
state = Record(vaultId, state, response, report);
await syncState.SaveAsync(state, cancellationToken).ConfigureAwait(false);
if (!response.HasMore)
{
break;
}
// A server that claims more but neither returns a change nor moves the cursor would spin
// this loop for ever. Stopping is the only safe reading of that answer.
if (!advanced && response.Changes.Count == 0)
{
break;
}
}
}
private StoredSyncState Record(
Guid vaultId,
StoredSyncState state,
SyncPullResponse response,
SyncReportBuilder report)
{
var now = clock.GetUtcNow();
var skew = (long)(response.ServerTime - now).TotalMilliseconds;
report.ServerKeyGeneration = response.CurrentKeyGeneration;
report.ServerTimeSkewMs = skew;
// A generation ahead of the key this client holds means the vault was rekeyed and this client's
// grant has not been re-wrapped. Items pulled meanwhile are stored but cannot be read.
keyring.TryGet(vaultId, out _, out var held);
report.RekeyRequired = response.CurrentKeyGeneration > held;
return state with
{
Cursor = response.NextCursor,
KeyGeneration = response.CurrentKeyGeneration,
LastPulledAt = now,
ServerTimeSkewMs = skew,
};
}
private async Task ApplyAsync(
Guid vaultId,
SyncChange change,
SyncReportBuilder report,
CancellationToken cancellationToken)
{
if (change.EntityType != SyncEntityType.Host)
{
// Reserved in the contract but not yet syncable. Ignoring it keeps a newer server's extra
// entity types from breaking an older client's pull.
return;
}
await reconciler.MirrorAsync(vaultId, change, cancellationToken).ConfigureAwait(false);
var pending = await outbox
.FindAsync(vaultId, change.EntityType, change.EntityId, cancellationToken)
.ConfigureAwait(false);
if (pending is null || pending.IsParked)
{
return;
}
await reconciler.ReconcileAsync(vaultId, change, pending, report, cancellationToken)
.ConfigureAwait(false);
}
/// <summary>What one push round achieved.</summary>
[StructLayout(LayoutKind.Auto)]
private readonly record struct DrainOutcome(int Sent, int Conflicts, bool BatchWasFull)
{
internal bool NeedsAnotherRound => Conflicts > 0 || BatchWasFull;
}
/// <summary>
/// Sends one batch and acts on each per-operation answer.
/// </summary>
/// <remarks>
/// <b>The cursor in the push response is deliberately ignored.</b> It sits after this push's own
/// changes, so adopting it would skip any change another client committed at a lower sequence
/// between this client's last pull and this push — permanently. Continuing from the cursor this
/// client already holds re-reads its own writes, which costs one redundant page and is idempotent.
/// </remarks>
private async Task<DrainOutcome> DrainAsync(
Guid vaultId,
SyncReportBuilder report,
CancellationToken cancellationToken)
{
var pending = await outbox
.TakeAsync(vaultId, options.MaxOperationsPerPush, cancellationToken)
.ConfigureAwait(false);
if (pending.Count == 0)
{
return default;
}
var operations = new List<SyncPushOperation>(pending.Count);
var byOperationId = new Dictionary<Guid, PendingOperation>(pending.Count);
foreach (var operation in pending)
{
operations.Add(new SyncPushOperation(
operation.OperationId,
operation.EntityType,
operation.EntityId,
operation.Operation,
operation.ExpectedVersion,
operation.Payload,
operation.Fields));
byOperationId[operation.OperationId] = operation;
await outbox.MarkDispatchedAsync(operation.Sequence, cancellationToken)
.ConfigureAwait(false);
}
var response = await api
.SyncPushAsync(vaultId, new SyncPushRequest(operations), cancellationToken)
.ConfigureAwait(false);
var conflicted = 0;
foreach (var result in response.Results)
{
if (!byOperationId.TryGetValue(result.OperationId, out var operation))
{
// An id this client did not send. Nothing sane to do with it.
continue;
}
// The attempt count was incremented above, so the bound is read from the fresh value.
var attempts = operation.Attempts + 1;
if (await HandleAsync(vaultId, operation with { Attempts = attempts }, result, report, cancellationToken)
.ConfigureAwait(false))
{
conflicted++;
}
}
return new DrainOutcome(
pending.Count, conflicted, pending.Count == options.MaxOperationsPerPush);
}
/// <returns>Whether this answer warrants another push round.</returns>
private async Task<bool> HandleAsync(
Guid vaultId,
PendingOperation operation,
SyncPushResult result,
SyncReportBuilder report,
CancellationToken cancellationToken)
{
switch (result.Status)
{
case SyncOperationStatus.Applied:
case SyncOperationStatus.Duplicate:
// Duplicate means an earlier push of this exact operation id already landed, so the
// stored state is what this operation intended. Treated as success on purpose: that is
// what makes a retry after a timeout exactly-once rather than merely at-least-once.
await AcceptAsync(vaultId, operation, result, cancellationToken).ConfigureAwait(false);
report.Pushed++;
return false;
case SyncOperationStatus.Conflict:
return await ResolveAsync(vaultId, operation, result, report, cancellationToken)
.ConfigureAwait(false);
case SyncOperationStatus.Forbidden:
await RejectAsync(
vaultId,
operation,
"You no longer have permission to change this item.",
report,
cancellationToken).ConfigureAwait(false);
return false;
case SyncOperationStatus.Invalid:
await RejectAsync(
vaultId,
operation,
result.Detail ?? "The server rejected this change as invalid.",
report,
cancellationToken).ConfigureAwait(false);
return false;
default:
await outbox.RecordFailureAsync(
operation.Sequence,
$"Unexpected push status {result.Status}.",
cancellationToken).ConfigureAwait(false);
return false;
}
}
/// <summary>Records an accepted operation and clears it from the outbox.</summary>
private async Task AcceptAsync(
Guid vaultId,
PendingOperation operation,
SyncPushResult result,
CancellationToken cancellationToken)
{
var expected = SyncVersions.NextVersion(operation.ExpectedVersion);
var version = result.Version ?? expected;
var isDelete = operation.Operation == SyncOperation.Delete;
// The payload was sealed at the version this client predicted, and the AAD binds that version.
// If the server assigned a different one — which its own version check should make impossible —
// storing the payload would leave a mirror row that never decrypts. Skip the write; the pull at
// the end of the pass brings the authoritative row.
var canMirror = isDelete || version == expected;
if (canMirror)
{
await items.SaveAsync(
new StoredItem(
vaultId,
operation.EntityType,
operation.EntityId,
version,
result.ChangeSequence ?? 0,
isDelete ? null : operation.Payload,
isDelete ? null : operation.Fields,
isDelete,
clock.GetUtcNow()),
cancellationToken).ConfigureAwait(false);
}
await outbox.CompleteAsync(operation.Sequence, cancellationToken).ConfigureAwait(false);
}
private async Task<bool> ResolveAsync(
Guid vaultId,
PendingOperation operation,
SyncPushResult result,
SyncReportBuilder report,
CancellationToken cancellationToken)
{
if (operation.Attempts >= options.MaxAttemptsBeforeParking)
{
await RejectAsync(
vaultId,
operation,
string.Create(
CultureInfo.InvariantCulture,
$"Could not be reconciled after {operation.Attempts} attempts."),
report,
cancellationToken).ConfigureAwait(false);
return false;
}
if (result.ServerEntity is null)
{
// The version check failed but the server has no such row. Re-offer it as a create.
return await RetryAsCreateAsync(vaultId, operation, report, cancellationToken)
.ConfigureAwait(false);
}
await reconciler.MirrorAsync(vaultId, result.ServerEntity, cancellationToken)
.ConfigureAwait(false);
await reconciler
.ReconcileAsync(vaultId, result.ServerEntity, operation, report, cancellationToken)
.ConfigureAwait(false);
return true;
}
private async Task<bool> RetryAsCreateAsync(
Guid vaultId,
PendingOperation operation,
SyncReportBuilder report,
CancellationToken cancellationToken)
{
if (operation.Operation == SyncOperation.Delete)
{
// Nothing there to delete, so the intent is already satisfied.
await outbox.CompleteAsync(operation.Sequence, cancellationToken).ConfigureAwait(false);
return false;
}
if (!keyring.TryGet(vaultId, out var vaultKey, out var generation)
|| operation.Payload is null)
{
await RejectAsync(
vaultId, operation, "This item has no usable vault key.", report, cancellationToken)
.ConfigureAwait(false);
return false;
}
var local = HostCipher.TryOpen(
operation.Payload,
vaultKey.Span,
operation.EntityId,
SyncVersions.NextVersion(operation.ExpectedVersion));
if (local is null)
{
await RejectAsync(
vaultId,
operation,
"The queued change could not be decrypted, so it could not be re-offered.",
report,
cancellationToken).ConfigureAwait(false);
return false;
}
// Re-sealed at version 1, because that is what the server assigns to a create and the AAD binds
// the version.
await outbox.ReviseAsync(
operation.Sequence,
SyncOperation.Upsert,
expectedVersion: null,
HostCipher.Seal(local.Host, vaultKey.Span, operation.EntityId, generation, itemVersion: 1),
HostFields.From(local.Host),
ancestor: null,
cancellationToken).ConfigureAwait(false);
return true;
}
/// <summary>Parks an operation the server will never accept, and says why.</summary>
private async Task RejectAsync(
Guid vaultId,
PendingOperation operation,
string reason,
SyncReportBuilder report,
CancellationToken cancellationToken)
{
await outbox.ParkAsync(operation.Sequence, reason, cancellationToken).ConfigureAwait(false);
await conflicts.RecordAsync(
vaultId,
operation.EntityType,
operation.EntityId,
ConflictKind.Rejected,
ConflictDetailCodec.Encode(reason),
cancellationToken).ConfigureAwait(false);
report.Parked++;
}
}
src/DodoSSH.Client.Sync/SyncReport.cs
+196
View File
@@ -0,0 +1,196 @@
using System.Text.Json;
using System.Text.Json.Serialization;
using DodoSSH.Client.Domain;
namespace DodoSSH.Client.Sync;
/// <summary>Tuning for a sync pass.</summary>
/// <remarks>
/// The push batch default is well below the server's own cap. A smaller batch means a conflict is
/// discovered and merged sooner, and it bounds how much work one rejected batch delays.
/// </remarks>
public sealed record SyncOptions
{
/// <summary>Changes requested per pull. The server clamps this.</summary>
public int PullPageSize { get; init; } = 500;
/// <summary>Operations sent per push. Must not exceed the server's advertised maximum.</summary>
public int MaxOperationsPerPush { get; init; } = 100;
/// <summary>
/// How many push rounds one pass may take.
/// </summary>
/// <remarks>
/// A bound rather than a loop until clear. Each round either applies, parks, or merges and advances
/// a version, so it does terminate — but against a vault someone else is writing to continuously it
/// could keep finding new conflicts, and a sync pass that never returns is worse than one that stops
/// and says so.
/// </remarks>
public int MaxPushRounds { get; init; } = 8;
/// <summary>
/// How many times an operation may be dispatched before it is parked for a person to look at.
/// </summary>
public int MaxAttemptsBeforeParking { get; init; } = 5;
/// <summary>
/// How many pages one pull may read.
/// </summary>
/// <remarks>
/// A backstop against a server that keeps saying there is more. At the default page size this is
/// half a million changes, well past any real vault, so reaching it means something is wrong rather
/// than merely large.
/// </remarks>
public int MaxPullPages { get; init; } = 1000;
/// <summary>The defaults.</summary>
public static SyncOptions Default { get; } = new();
}
/// <summary>What one sync pass did.</summary>
/// <param name="VaultId">The vault.</param>
/// <param name="Pulled">Changes received.</param>
/// <param name="Pushed">Operations the server accepted.</param>
/// <param name="Merged">Items where local and remote edits were reconciled.</param>
/// <param name="Resurrected">
/// Items someone else deleted while this machine had unpushed edits. The tombstone stands and the local
/// content survives under a new id; nothing is discarded.
/// </param>
/// <param name="DeletesAbandoned">
/// Local deletions dropped because the other side edited the item instead. An edit outlives a removal:
/// re-deleting costs a click, and a discarded edit may be irrecoverable.
/// </param>
/// <param name="Parked">Operations the server refused, now waiting on a person.</param>
/// <param name="Unreadable">Items whose payload would not decrypt.</param>
/// <param name="ServerKeyGeneration">The generation the server reports for this vault.</param>
/// <param name="RekeyRequired">
/// True when the server's generation is ahead of the key this client holds, so items cannot be read
/// until new grants arrive.
/// </param>
/// <param name="ServerTimeSkewMs">
/// Difference between the server's clock and this machine's. Recorded, never acted on — the merge uses
/// versions and a retained ancestor, so a skewed clock must not be able to decide which edit wins.
/// </param>
/// <param name="RoundsExhausted">
/// True when the push loop hit <see cref="SyncOptions.MaxPushRounds"/> with work still outstanding. Not
/// a failure: the next pass continues from here.
/// </param>
public sealed record SyncReport(
Guid VaultId,
int Pulled,
int Pushed,
int Merged,
int Resurrected,
int DeletesAbandoned,
int Parked,
int Unreadable,
uint ServerKeyGeneration,
bool RekeyRequired,
long ServerTimeSkewMs,
bool RoundsExhausted)
{
/// <summary>Whether anything happened that a user should be told about.</summary>
public bool NeedsAttention =>
Resurrected > 0 || DeletesAbandoned > 0 || Parked > 0 || Unreadable > 0 || RekeyRequired;
}
/// <summary>Accumulates a <see cref="SyncReport"/> while a pass runs.</summary>
internal sealed class SyncReportBuilder(Guid vaultId)
{
internal int Pulled { get; set; }
internal int Pushed { get; set; }
internal int Merged { get; set; }
internal int Resurrected { get; set; }
internal int DeletesAbandoned { get; set; }
internal int Parked { get; set; }
internal int Unreadable { get; set; }
internal uint ServerKeyGeneration { get; set; }
internal bool RekeyRequired { get; set; }
internal long ServerTimeSkewMs { get; set; }
internal bool RoundsExhausted { get; set; }
internal SyncReport Build() =>
new(
vaultId,
Pulled,
Pushed,
Merged,
Resurrected,
DeletesAbandoned,
Parked,
Unreadable,
ServerKeyGeneration,
RekeyRequired,
ServerTimeSkewMs,
RoundsExhausted);
}
/// <summary>The record written to the conflict log when a merge had to override something.</summary>
/// <param name="Field">Which field, as a path.</param>
/// <param name="DiscardedSide">Whose intent was overridden: <c>Local</c> or <c>Remote</c>.</param>
/// <param name="Kept">The value that survives.</param>
/// <param name="Discarded">The value that lost.</param>
/// <param name="DiscardedWasRemoval">Whether what lost was a deletion rather than a value.</param>
public sealed record ConflictDetailEntry(
string Field,
string DiscardedSide,
string? Kept,
string? Discarded,
bool DiscardedWasRemoval);
/// <summary>A conflict log entry.</summary>
/// <param name="Summary">One line for a person to read.</param>
/// <param name="Fields">Everything the merge overrode.</param>
public sealed record ConflictDetail(string Summary, IReadOnlyList<ConflictDetailEntry> Fields);
/// <summary>
/// Serialises what a merge discarded, for the conflict log.
/// </summary>
/// <remarks>
/// The bytes crossing into <c>ConflictStore</c> are plaintext vault content and are sealed there under
/// the LocalCacheKey. Deliberately its own format rather than the item payload's: this is local
/// bookkeeping and is never pushed, so it has no compatibility obligation to any other client.
/// </remarks>
internal static class ConflictDetailCodec
{
internal static byte[] Encode(string summary, IReadOnlyList<HostFieldConflict> conflicts) =>
JsonSerializer.SerializeToUtf8Bytes(
new ConflictDetail(
summary,
[.. conflicts.Select(c => new ConflictDetailEntry(
c.Field,
c.DiscardedSide.ToString(),
c.Kept,
c.Discarded,
c.DiscardedWasRemoval))]),
ConflictJsonContext.Default.ConflictDetail);
internal static byte[] Encode(string summary) => Encode(summary, []);
/// <summary>Reads a detail back, for display.</summary>
internal static ConflictDetail? TryDecode(ReadOnlySpan<byte> utf8)
{
try
{
return JsonSerializer.Deserialize(utf8, ConflictJsonContext.Default.ConflictDetail);
}
catch (JsonException)
{
return null;
}
}
}
[JsonSourceGenerationOptions(PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase)]
[JsonSerializable(typeof(ConflictDetail))]
internal sealed partial class ConflictJsonContext : JsonSerializerContext;
src/DodoSSH.Client.Sync/VaultKeyring.cs
+149
View File
@@ -0,0 +1,149 @@
using System.Diagnostics.CodeAnalysis;
using System.Security.Cryptography;
using DodoSSH.Client.Storage;
using DodoSSH.Crypto;
namespace DodoSSH.Client.Sync;
/// <summary>
/// The vault keys held for the duration of an unlocked session.
/// </summary>
/// <remarks>
/// <para>
/// One place that holds plaintext vault keys, so there is one place that clears them. Every store and
/// every cipher call borrows a key from here rather than keeping a copy, which is what makes
/// "the keys exist only while unlocked" a property of the code and not of everyone's discipline.
/// </para>
/// <para>
/// A grant that will not open is not an error: it means the vault has been rekeyed and this client's
/// grant has not been re-wrapped yet, or the grant was fabricated. Both leave the vault temporarily
/// unreadable and both are reported rather than thrown, so one bad grant does not take the other vaults
/// down with it.
/// </para>
/// </remarks>
public sealed class VaultKeyring : IDisposable
{
private readonly Dictionary<Guid, byte[]> keys = [];
private readonly Dictionary<Guid, uint> generations = [];
private bool disposed;
private VaultKeyring()
{
}
/// <summary>Vaults whose grant could not be opened, and which are therefore unreadable.</summary>
public IReadOnlyList<Guid> Unopened { get; private set; } = [];
/// <summary>
/// Opens every grant the bundle can.
/// </summary>
/// <param name="bundle">The unlocked identity keys.</param>
/// <param name="vaults">The cached vault list, each with its wrapped key.</param>
public static VaultKeyring Open(UserSecretBundle bundle, IReadOnlyList<StoredVault> vaults)
{
ArgumentNullException.ThrowIfNull(bundle);
ArgumentNullException.ThrowIfNull(vaults);
var keyring = new VaultKeyring();
var unopened = new List<Guid>();
try
{
foreach (var vault in vaults)
{
if (vault.WrappedVaultKey is null)
{
// The server said so itself: a grant awaiting re-wrap after a rekey.
unopened.Add(vault.VaultId);
continue;
}
var key = VaultKeys.TryUnwrap(
bundle.EncryptionKey,
vault.WrappedVaultKey,
vault.VaultId,
vault.KeyGeneration);
if (key is null)
{
unopened.Add(vault.VaultId);
continue;
}
keyring.keys[vault.VaultId] = key;
keyring.generations[vault.VaultId] = vault.KeyGeneration;
}
keyring.Unopened = unopened;
return keyring;
}
catch
{
keyring.Dispose();
throw;
}
}
/// <summary>
/// Borrows a vault's key.
/// </summary>
/// <remarks>
/// The returned memory is the keyring's own buffer, not a copy, and is zeroed when the keyring is
/// disposed. Callers must not retain it past the operation they borrowed it for.
/// </remarks>
public bool TryGet(Guid vaultId, out ReadOnlyMemory<byte> vaultKey, out uint keyGeneration)
{
ObjectDisposedException.ThrowIf(disposed, this);
if (keys.TryGetValue(vaultId, out var key))
{
vaultKey = key;
keyGeneration = generations[vaultId];
return true;
}
vaultKey = default;
keyGeneration = 0;
return false;
}
/// <summary>Whether this vault can be read at all.</summary>
public bool CanRead(Guid vaultId) => !disposed && keys.ContainsKey(vaultId);
/// <inheritdoc />
public void Dispose()
{
if (disposed)
{
return;
}
disposed = true;
foreach (var key in keys.Values)
{
CryptographicOperations.ZeroMemory(key);
}
keys.Clear();
generations.Clear();
}
}
/// <summary>Thrown when an operation needs a vault key the keyring does not hold.</summary>
/// <remarks>
/// An exception rather than a silent no-op, because every caller that reaches this point has already
/// been given the chance to check <see cref="VaultKeyring.CanRead"/>. Continuing without the key would
/// mean writing an item nobody can open.
/// </remarks>
[SuppressMessage(
"Design",
"CA1032:Implement standard exception constructors",
Justification = "The vault id is required context; a message-only constructor would lose it.")]
public sealed class VaultUnreadableException(Guid vaultId)
: InvalidOperationException(
$"Vault {vaultId} has no usable key. Its grant is missing or awaiting re-wrap after a rekey.")
{
/// <summary>The vault that cannot be read.</summary>
public Guid VaultId { get; } = vaultId;
}
src/DodoSSH.Client.Sync/packages.lock.json
+257
View File
@@ -0,0 +1,257 @@
{
"version": 2,
"dependencies": {
"net10.0": {
"Meziantou.Analyzer": {
"type": "Direct",
"requested": "[3.0.134, )",
"resolved": "3.0.134",
"contentHash": "tTYCcYKyOko3TMNxmxmA9nakbcHVUgglENmCMIhzIjl9y9FBZO/0tWSxTGC74Sp198FmWih5S5KkjQRBg5ePkQ=="
},
"Microsoft.CodeAnalysis.BannedApiAnalyzers": {
"type": "Direct",
"requested": "[5.6.0, )",
"resolved": "5.6.0",
"contentHash": "Kcobt3pnOdO0A+6CKiMHZdTEluJpsfxiV20axtZdmfBQnDmiWTKPJADlgAfdTuKNAnVarrkJa0UEGwuOo91muw=="
},
"Microsoft.Data.Sqlite.Core": {
"type": "Transitive",
"resolved": "10.0.10",
"contentHash": "TPCs0ldm7AWqcKmp6f/Xr+14sat7hx4rHfRlS4RgCURBH2thEWbAKEyX7cCWr63zVJVOJIJZTg2cBiUXa8ys6g==",
"dependencies": {
"SQLitePCLRaw.core": "2.1.11"
}
},
"Microsoft.EntityFrameworkCore.Abstractions": {
"type": "Transitive",
"resolved": "10.0.10",
"contentHash": "bOzrFCl6uZCjaSh2bG1ToRQRdx+iXvxosCg9hFyG9OWeAzOFI4xev9OqKeWfKf/kAHyox2JnbcvLVf2ceA7sqA=="
},
"Microsoft.EntityFrameworkCore.Analyzers": {
"type": "Transitive",
"resolved": "10.0.10",
"contentHash": "2gLDordUCGf3aNOOuqtTbP5mxhiP9nk6TnvGiE3RnqT891O+Zf/qKu1PIREubs1M16A0SImr4vULBfU5BTDs1Q=="
},
"Microsoft.EntityFrameworkCore.Sqlite.Core": {
"type": "Transitive",
"resolved": "10.0.10",
"contentHash": "YbVWMIouzwTKBiLms8boa7xeRT88wI14R1msv3XExFk9n0/sa8nU7MwDa1CKtfLGMJs7O7QWuS9/xhcQ72AD2A==",
"dependencies": {
"Microsoft.Data.Sqlite.Core": "10.0.10",
"Microsoft.EntityFrameworkCore.Relational": "10.0.10",
"Microsoft.Extensions.Caching.Memory": "10.0.10",
"Microsoft.Extensions.Configuration.Abstractions": "10.0.10",
"Microsoft.Extensions.DependencyModel": "10.0.10",
"Microsoft.Extensions.Logging": "10.0.10",
"SQLitePCLRaw.core": "2.1.11"
}
},
"Microsoft.Extensions.Caching.Abstractions": {
"type": "Transitive",
"resolved": "10.0.10",
"contentHash": "4ZFBNE+jzR+CrWWlhOesnmywCW7pYKT0dxyAQRdL11yJwxe4jvcAu31eorFtEkoFeCDcUTeNssgPv2yaRRptaQ==",
"dependencies": {
"Microsoft.Extensions.Primitives": "10.0.10"
}
},
"Microsoft.Extensions.Caching.Memory": {
"type": "Transitive",
"resolved": "10.0.10",
"contentHash": "N1w5H7uK6gCTnCBZAWzE0/EQYSPysij/uYwDqntqBVvBa6bjMmBKitsnEFd6yh/SX3wLm67nO6+OnZ84K+gZWg==",
"dependencies": {
"Microsoft.Extensions.Caching.Abstractions": "10.0.10",
"Microsoft.Extensions.DependencyInjection.Abstractions": "10.0.10",
"Microsoft.Extensions.Logging.Abstractions": "10.0.10",
"Microsoft.Extensions.Options": "10.0.10",
"Microsoft.Extensions.Primitives": "10.0.10"
}
},
"Microsoft.Extensions.Configuration.Abstractions": {
"type": "Transitive",
"resolved": "10.0.10",
"contentHash": "5Vnd2I75DmZCVEjSynIdJ/0EGafgnLQwgR3t2C2/fkjx/nRG+cLwxLLdInoHeCEpkD5K4Ov/g9ZCRYrl4TRsaA==",
"dependencies": {
"Microsoft.Extensions.Primitives": "10.0.10"
}
},
"Microsoft.Extensions.DependencyInjection": {
"type": "Transitive",
"resolved": "10.0.10",
"contentHash": "ANyvsgkNBRvcJh2XLgn8veGmajf+8m0AbKK+HPWdRL1yraSNVVSmQhFntLtdz/C795jxqqup+k05cs/3jZQPOA==",
"dependencies": {
"Microsoft.Extensions.DependencyInjection.Abstractions": "10.0.10"
}
},
"Microsoft.Extensions.DependencyInjection.Abstractions": {
"type": "Transitive",
"resolved": "10.0.10",
"contentHash": "z/2xXlFw2aLGjHyEm6E0tQ+In6VfzQzTrtArbQ2c0TQE16ZbyDCMGPvaUT9I0s8rgy9sRWlU2P9waW37qV04qA=="
},
"Microsoft.Extensions.DependencyModel": {
"type": "Transitive",
"resolved": "10.0.10",
"contentHash": "rfZA1RjR021RPqSmIPovfz2aOd79TGqJ9BengbjnzIISOVwjLmuSDnhCMmiY/1c6iYvGolQ1iNGzkav0u11XEA=="
},
"Microsoft.Extensions.Logging": {
"type": "Transitive",
"resolved": "10.0.10",
"contentHash": "Tf6z5HsL0VDYRTfvsoNrTGHGheCwkTsZBA2FFh5ATJUbkAwug+FFNISJK2gjpUNemlAOoWllAK52HOWCjto3EQ==",
"dependencies": {
"Microsoft.Extensions.DependencyInjection": "10.0.10",
"Microsoft.Extensions.Logging.Abstractions": "10.0.10",
"Microsoft.Extensions.Options": "10.0.10"
}
},
"Microsoft.Extensions.Logging.Abstractions": {
"type": "Transitive",
"resolved": "10.0.10",
"contentHash": "zkFxGYUvdxAvIKTyXHrmW+Sux53D4SezD9dMyZ6hrwwzPQJNuwCRy1f5W7AvYTqacEGhWF2XderRQG1OvbV8og==",
"dependencies": {
"Microsoft.Extensions.DependencyInjection.Abstractions": "10.0.10"
}
},
"Microsoft.Extensions.Options": {
"type": "Transitive",
"resolved": "10.0.10",
"contentHash": "srnhnk7nE8krBiIXp71LvBmKBtraBONWSRzdjJgRv1Ko9Mp8IVNqv4vIS9hGeVteBig8aQkva9ZG+sC+o5sVcA==",
"dependencies": {
"Microsoft.Extensions.DependencyInjection.Abstractions": "10.0.10",
"Microsoft.Extensions.Primitives": "10.0.10"
}
},
"Microsoft.Extensions.Primitives": {
"type": "Transitive",
"resolved": "10.0.10",
"contentHash": "5wu/GrYVd8mG2DVUw3vFJzF+O336TyTGg/Kmcgw9bfwYhCoFiV5lR5QeEmKecJyrW4W54nMfD3p3589E8a7czQ=="
},
"dodossh.client.api": {
"type": "Project",
"dependencies": {
"DodoSSH.Client.Auth": "[1.0.0, )",
"DodoSSH.Contracts": "[1.0.0, )",
"DodoSSH.Crypto": "[1.0.0, )"
}
},
"dodossh.client.auth": {
"type": "Project"
},
"dodossh.client.domain": {
"type": "Project"
},
"dodossh.client.storage": {
"type": "Project",
"dependencies": {
"DodoSSH.Contracts": "[1.0.0, )",
"DodoSSH.Crypto": "[1.0.0, )",
"EFCore.NamingConventions": "[10.0.1, )",
"Microsoft.EntityFrameworkCore.Sqlite": "[10.0.10, )"
}
},
"dodossh.contracts": {
"type": "Project"
},
"dodossh.crypto": {
"type": "Project",
"dependencies": {
"NSec.Cryptography": "[26.4.0, )"
}
},
"EFCore.NamingConventions": {
"type": "CentralTransitive",
"requested": "[10.0.1, )",
"resolved": "10.0.1",
"contentHash": "Xs5k8XfNKPkkQSkGmZkmDI1je0prLTdxse+s8PgTFZxyBrlrTLzTBUTVJtQKSsbvu4y+luAv8DdtO5SALJE++A==",
"dependencies": {
"Microsoft.EntityFrameworkCore": "[10.0.1, 11.0.0)",
"Microsoft.EntityFrameworkCore.Relational": "[10.0.1, 11.0.0)",
"Microsoft.Extensions.DependencyInjection.Abstractions": "10.0.1"
}
},
"libsodium": {
"type": "CentralTransitive",
"requested": "[1.0.22, )",
"resolved": "1.0.22",
"contentHash": "KPD9SloJFclrsjnhABu7dzWrcyYkwPbvx5l1gRSPAX/0n+OBtSiVCKtGFv4n+ecWUHU0tCG9LSSwoZZx673zBQ=="
},
"Microsoft.EntityFrameworkCore": {
"type": "CentralTransitive",
"requested": "[10.0.10, )",
"resolved": "10.0.10",
"contentHash": "a0V7zj/VbYP6dTdWpUgE/r2PuLKtUGe2aJ0lVKkn/wP9ZhaxUz2kQydVfvOjCv2SKxlrqdBfHhPD4Cvlf+4ffA==",
"dependencies": {
"Microsoft.EntityFrameworkCore.Abstractions": "10.0.10",
"Microsoft.EntityFrameworkCore.Analyzers": "10.0.10",
"Microsoft.Extensions.Caching.Memory": "10.0.10",
"Microsoft.Extensions.Logging": "10.0.10"
}
},
"Microsoft.EntityFrameworkCore.Relational": {
"type": "CentralTransitive",
"requested": "[10.0.10, )",
"resolved": "10.0.10",
"contentHash": "wNonj40aZxia+GtuBiiD6ZqVh4h6y5Nje1bGdmzZ8/ui0QRsAN+S0SIrLHFCEGbG9cDbeaE40sh+Lr7o9rRs6g==",
"dependencies": {
"Microsoft.EntityFrameworkCore": "10.0.10",
"Microsoft.Extensions.Caching.Memory": "10.0.10",
"Microsoft.Extensions.Configuration.Abstractions": "10.0.10",
"Microsoft.Extensions.Logging": "10.0.10"
}
},
"Microsoft.EntityFrameworkCore.Sqlite": {
"type": "CentralTransitive",
"requested": "[10.0.10, )",
"resolved": "10.0.10",
"contentHash": "kzg9MuQNJvZQxAU+piSkEzc7/1tpW6n1nVSGGMObu2GgxLK8Nf+6fvZundaznTZ+O2KhfPZ8HFNCzMH3PWDUmA==",
"dependencies": {
"Microsoft.EntityFrameworkCore.Sqlite.Core": "10.0.10",
"Microsoft.Extensions.Caching.Memory": "10.0.10",
"Microsoft.Extensions.Configuration.Abstractions": "10.0.10",
"Microsoft.Extensions.DependencyModel": "10.0.10",
"Microsoft.Extensions.Logging": "10.0.10",
"SQLitePCLRaw.bundle_e_sqlite3": "2.1.11",
"SQLitePCLRaw.core": "2.1.11"
}
},
"NSec.Cryptography": {
"type": "CentralTransitive",
"requested": "[26.4.0, )",
"resolved": "26.4.0",
"contentHash": "0vsCtY5f+YgQROiWNqzgWp+l2pddfk9FkWoGV/bEo0MuEYPKlJWuoA8aOfO6qp3f+EnObKE3zSJhn1PspJeJVg==",
"dependencies": {
"libsodium": "[1.0.22, 1.0.23)"
}
},
"SQLitePCLRaw.bundle_e_sqlite3": {
"type": "CentralTransitive",
"requested": "[2.1.12, )",
"resolved": "2.1.12",
"contentHash": "mAgscpQMLw5/nfA1Q5oJVAT29yROUo1ifZGbbTpx/lwZpSxMUGoYbKfmvdm8oXER+RzxqBmmQzeBEVKfeHv2nw==",
"dependencies": {
"SQLitePCLRaw.lib.e_sqlite3": "2.1.12",
"SQLitePCLRaw.provider.e_sqlite3": "2.1.12"
}
},
"SQLitePCLRaw.core": {
"type": "CentralTransitive",
"requested": "[2.1.12, )",
"resolved": "2.1.12",
"contentHash": "ETpNw9DY3ckWLgRRAeCHj+GKOuPi61aeczkXhgHexUvqoZBAYg8RYESE2J7O1M7+o6QbdSEZwrw9bfqztUVWXg=="
},
"SQLitePCLRaw.lib.e_sqlite3": {
"type": "CentralTransitive",
"requested": "[2.1.12, )",
"resolved": "2.1.12",
"contentHash": "fWi8Dbknuhgg72fWinIdjXVaqO1hHL4YBBwVLnr7e1c9TAZwJ0QE38j9syW1hwx6HaqEVTwI+O07WPdZn8Rp0w=="
},
"SQLitePCLRaw.provider.e_sqlite3": {
"type": "CentralTransitive",
"requested": "[2.1.12, )",
"resolved": "2.1.12",
"contentHash": "W3oH4XIfCzFrgUSDKHhN6N+dgzA5YHOR2VxX8GB6Qy7CyrJJgxPEG8NirgYWlPQC5P2jz2knSsexWu4tDUL33g==",
"dependencies": {
"SQLitePCLRaw.core": "2.1.12"
}
}
}
}
}