Sync: after a read from the beginning, remove what the server no longer has, and flag an edit that brings a forgotten item back
ci / build and test (push) Successful in 3m38s
ci / android head (push) Successful in 3m46s
ci / desktop nightly (push) Failing after 37s
ci / api image (push) Successful in 48s

The server now forgets a deletion past its retention and refuses cursors
from before it. The client then reads everything again, but everything
again says nothing about what went. So a machine away for longer than the
retention kept those items for good, and its next edit of one put it back
on the server for everyone.

A read from the beginning now marks each item it reads, keeping the mark in
the sync state because the read can span passes. When it finishes, it
removes the items of the synced types it never marked. Two kinds are kept:
items newer than the read's bound, which this machine pushed during it, and
items with a change still waiting, which are the user's work. For those,
the push finds the server without the item and sends it as a create, as
before, but now records a RecreatedAfterCollection conflict and counts it,
so the sync message says so instead of undoing someone's deletion silently.

The bound is the highest sequence the cache held when the read began, or
anything higher it has read since. Taking only what the read returned would
have left the last item of a vault in place for ever.
This commit is contained in:
jaap-jan committed 2026-10-02 17:36:11 +02:00
1 parent 55e5de0ce4
commit 858e36279d
13 files changed
+1048 -63

No files matched your search

+88
View File
@@ -177,6 +177,94 @@ public sealed class ItemStore(
await context.SaveChangesAsync(cancellationToken).ConfigureAwait(false);
}
/// <summary>Marks the items a pull from nothing has just read as ones the server still has.</summary>
/// <remarks>One update per type in the page; see <see cref="SweepAsync"/>.</remarks>
public async Task MarkSeenAsync(
Guid vaultId,
long mark,
IReadOnlyCollection<(SyncEntityType Type, Guid Id)> seen,
CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(seen);
if (seen.Count == 0)
{
return;
}
var context = contexts.CreateDbContext();
await using var scope = context.ConfigureAwait(false);
foreach (var group in seen.GroupBy(item => item.Type))
{
var type = group.Key;
var ids = group.Select(item => item.Id).Distinct().ToList();
await context.Set<CachedItemRow>()
.Where(row => row.VaultId == vaultId && row.EntityType == type && ids.Contains(row.EntityId))
.ExecuteUpdateAsync(set => set.SetProperty(row => row.SweepMark, mark), cancellationToken)
.ConfigureAwait(false);
}
}
/// <summary>The highest change sequence held for a vault, or zero for none.</summary>
/// <remarks>Where a pull from nothing starts its sweep's upper bound; see <see cref="SweepAsync"/>.</remarks>
public async Task<long> HighestSequenceAsync(Guid vaultId, CancellationToken cancellationToken)
{
var context = contexts.CreateDbContext();
await using var scope = context.ConfigureAwait(false);
return await context.Set<CachedItemRow>()
.Where(row => row.VaultId == vaultId)
.MaxAsync(row => (long?)row.ChangeSequence, cancellationToken)
.ConfigureAwait(false) ?? 0;
}
/// <summary>
/// Removes the items a completed pull from nothing did not see, which are the ones the server no longer has.
/// </summary>
/// <remarks>
/// <para>
/// The other half of the server's tombstone collection. A machine that missed a deletion for longer than
/// the retention is refused its cursor and pulls everything again — and everything again does not include
/// the item, so its copy here would otherwise last for ever, and its next edit would push it back.
/// </para>
/// <para>
/// ◆ Two kinds of row are not evidence and are kept. One newer than <paramref name="seenUpTo"/> was
/// written by this machine's own push during the pull: the bound is the higher of what the cache held
/// when the pull began and what it has read since, and the server's head is past both, so a push lands
/// above it. One with a change still waiting to go is the user's work: it is
/// left for the push, which finds the server without the item and says so — see
/// <c>SyncEngine.RetryAsCreateAsync</c> — rather than being thrown away here unseen.
/// </para>
/// </remarks>
/// <returns>How many items went.</returns>
public async Task<int> SweepAsync(
Guid vaultId,
IReadOnlyCollection<SyncEntityType> types,
long mark,
long seenUpTo,
CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(types);
var context = contexts.CreateDbContext();
await using var scope = context.ConfigureAwait(false);
// Only the types the pull asked for: a row of any other type was never going to be marked.
return await context.Set<CachedItemRow>()
.Where(row => row.VaultId == vaultId
&& types.Contains(row.EntityType)
&& (row.SweepMark == null || row.SweepMark != mark)
&& row.ChangeSequence <= seenUpTo
&& !context.Set<OutboxRow>().Any(pending =>
pending.VaultId == row.VaultId
&& pending.EntityType == row.EntityType
&& pending.EntityId == row.EntityId))
.ExecuteDeleteAsync(cancellationToken)
.ConfigureAwait(false);
}
/// <summary>Removes a tombstone whose change has been seen by everything that needed it.</summary>
/// <remarks>
/// Only ever called for a row that is already a tombstone. Collecting a live item here would make