DodoTech-Public/DodoSSH
Public Access
2
0
Fork 0
Code Issues Pull Requests 2 Actions 1 Packages Projects Releases 2 Wiki Activity

Move files to and from a host over SFTP

Browse Source 
M2's file transfer, built bottom-up: an SFTP session on the SSH layer, a
transfer queue in a project of its own, and the two-pane browser the design
asked for replacing the screen that said it did not exist. Remote listings
carry names, sizes, modification times and a real drwxr-xr-x — nothing in this
repository could render a POSIX mode before — and the queue moves one file at a
time with progress, throughput and resume.

The design import assumed this would be an SFTP subsystem channel on
ISshConnection, beside the shell on a transport that is already up. SSH.NET
does not offer that: SftpClient derives from BaseClient and owns its own
transport, and there is no supported way to hand it an SshClient's session. So
file transfer opens a second authenticated connection, and it is named for
that rather than dressed up as a channel — OpenSftpAsync is on
ISftpSessionFactory, not on a connection. The difference is visible to a user:
the host records a second login, and a host whose password is typed each time
asks for it again on this screen. It goes through the same host key gate, the
same pin and the same two refusals a shell does, so a fingerprint approved for
a terminal is approved here and one approved here reaches the other machines
with the next sync. docs/design-import-gaps.md is corrected, and marked as the
one row where what shipped differs from what it predicted.

Nothing is written at its final name until it is complete. Every transfer goes
to a .dodossh-part file beside its destination and is renamed into place at the
end, so an interrupted transfer can never be mistaken for a finished one —
which matters most for what this screen is actually for, which is copying a
build artefact onto a server and then running it. A destination that already
exists is refused outright rather than overwritten: the queue has no way to
ask, and silently replacing a file somebody's process is serving is the worse
of the two failures. The remote pane has DELETE and MKDIR so that refusal is
not a dead end. A test against the container pins the assumption underneath all
of this — that SFTP's rename does not clobber.

Resume works within a run of the application and not across a restart, and the
limit is deliberate rather than unfinished. Nothing records which source wrote
a part file, and resuming one on the strength of its name matching is how a
corrupt artefact gets delivered with nothing reporting a failure; a part file
found at startup is started over. Making it survive a restart needs the
preferences store this client still has not got. The offset a resume starts at
is the part file's own length rather than the transfer's recorded progress: a
cancellation can land between a write completing and the counter moving, and
only one of those two is a fact about the bytes that are there.

The queue and its connection outlive a lock, as shells do. LockAsync already
argues that locking must not destroy work in flight — it is what somebody does
when they walk away from the machine, which is exactly when a long transfer is
most likely to be running — so TransfersViewModel is created once and the vault
is attached on unlock and detached on lock. What locking takes is the host
list, and it has to: those rows carry decrypted secrets.

DodoSSH.Client.Transfer is a new project rather than more of Client.Ssh. The
two answer different questions — one is about reaching a host, the other about
moving bytes and what to do when moving them stops halfway — and this is the
only client project that deliberately touches the local filesystem.

Three defects the tests found, none of which review would have. SftpPath.Name
answered an empty string for the root. NavigateRemoteAsync wrapped itself in
the busy guard, so navigating from inside another command did nothing at all
and the remote pane simply stayed empty after connecting, with no failure
anywhere to explain it. And opening an SFTP session per test made two
handshakes per test — this client learns a host key by being refused — which
pushed the SSH assembly past sshd's MaxStartups and failed a different few
unrelated tests each run; the session is shared through the fixture now, with
the reason written where the next person will hit it.

1004 tests green across 18 projects, 24 of them new: the SFTP subsystem against
the OpenSSH container, the queue against a real temporary directory and a fake
host, and three more layout measurements because a screen this window has never
laid out is a screen never checked.

Not verified: the screen has not been looked at running. The layout harness
measures it at the window's minimum in three shapes, which is the class of
defect that has shipped here before, but reaching it in the application needs
the compose stack, the migrations, the API and a browser sign-in. What is still
absent — the status bar's transfer count, dragging between the panes,
transferring a directory, and sftp over a bastion — is in
docs/design-import-gaps.md.
This commit is contained in:
jaap-jan
2026-07-31 11:07:29 +02:00
parent 94e11f5e38
commit 04faef6597
32 changed files with 4933 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/DodoSSH.Client.App.Layout.Tests/ScreenLayoutTests.cs
+126
View File
@@ -7,6 +7,7 @@ using DodoSSH.Client.Session.Tests;
using DodoSSH.Client.Ssh;
using DodoSSH.Client.Storage;
using DodoSSH.Client.Terminal;
using DodoSSH.Client.Transfer;
using DodoSSH.Crypto;
using NSubstitute;
@@ -54,6 +55,14 @@ public sealed class ScreenLayoutTests : IAsyncLifetime
private VaultSession session = null!;
private VaultViewModel vault = null!;
/// <remarks>
/// Over a substitute factory that is never asked for a session. Every shape measured here is one the
/// screen is in before a connection exists or after one has failed, which is deliberate: the two panes
/// are at their widest with the local one full and the remote one carrying its explanation, and a
/// connected pane is the same template with shorter names in it.
/// </remarks>
private TransfersViewModel transfers = null!;
private static CancellationToken Token => TestContext.Current.CancellationToken;
/// <inheritdoc />
@@ -82,12 +91,20 @@ public sealed class ScreenLayoutTests : IAsyncLifetime
// every sync pass out of a suite that is only measuring rectangles.
vault = new VaultViewModel(session, workspace, knownHosts, static () => null);
transfers = new TransfersViewModel(
Substitute.For<ISftpSessionFactory>(), TimeProvider.System);
await SeedAsync();
// Attached after seeding, so the host picker has something in it and the local pane has listed this
// machine's home directory — which is what puts real names of real length into the row template.
transfers.Attach(vault, knownHosts);
}
/// <inheritdoc />
public async ValueTask DisposeAsync()
{
await transfers.DisposeAsync();
await vault.DisposeAsync();
knownHosts.Close();
await workspace.DisposeAsync();
@@ -271,6 +288,73 @@ public sealed class ScreenLayoutTests : IAsyncLifetime
});
}
// ---- The transfers screen ----
/// <remarks>
/// <para>
/// The widest thing in this window and the one with the least room to give: two file listings side by
/// side, each with four columns, and a queue underneath — all inside 826 pixels once the nav rail has
/// taken its column. The header row is the tight part, because it holds a host picker, a password box,
/// a button and a chip on one line.
/// </para>
/// <para>
/// Measured disconnected, which is the state the screen opens in and the one where the local pane is at
/// its fullest: it lists this machine's home directory, so the row template is exercised with real names
/// of real length rather than with fixtures chosen to fit.
/// </para>
/// </remarks>
[Fact]
public async Task TheTransfersScreenFitsBeforeAnythingIsConnected()
{
await MeasureTransfersAsync(faults => faults.ShouldBeEmpty());
}
/// <remarks>
/// <para>
/// The queue is the half of this screen that only exists once something has been asked for, so a shape
/// nothing puts a row into is a shape never laid out. Three rows, because the row template changes with
/// the state: a running one shows a bar and a STOP, a stopped one shows RESUME and DISCARD, and a failed
/// one carries the server's own sentence in the column the other two put a byte count in.
/// </para>
/// <para>
/// The rows are placed directly rather than driven through the queue. What is being measured is the
/// template at each state, and running a real transfer to reach those states would put a thread-pool
/// hand-off and a filesystem in the middle of a test about rectangles. What the queue does is measured in
/// <c>DodoSSH.Client.Transfer.Tests</c>.
/// </para>
/// </remarks>
[Fact]
public async Task TheTransfersScreenFitsWithTransfersInTheQueue()
{
Enqueue(TransferDirection.Download, "artefact.tar.gz", 402_653_184, 149_000_000,
TransferState.Running, bytesPerSecond: 6_500_000);
Enqueue(TransferDirection.Upload, "site-backup-2026-07-30.sql.gz", 8_100_000_000, 3_200_000_000,
TransferState.Cancelled);
Enqueue(TransferDirection.Upload, "deploy.sh", 4_096, 0, TransferState.Failed,
failure: "deploy.sh is already in that directory on the host. Rename or remove it first — "
+ "nothing here overwrites a file that is already there.");
transfers.Transfers.Count.ShouldBe(3);
await MeasureTransfersAsync(faults => faults.ShouldBeEmpty());
}
/// <remarks>
/// The trust card covers the whole screen, and it is the one thing here a user cannot get past without
/// pressing something — so a button of its own that fell outside the window would leave the screen
/// permanently blocked.
/// </remarks>
[Fact]
public async Task TheTransfersScreenFitsWithTheHostKeyCardShowing()
{
transfers.PendingHostKey = new HostKeyPresentation(
"db.internal", 22, "ssh-ed25519", "SHA256:0123456789abcdefghijklmnopqrstuvwxyzABCDEFG");
await MeasureTransfersAsync(faults => faults.ShouldBeEmpty());
}
// ---- The chrome ----
/// <remarks>
@@ -389,6 +473,48 @@ public sealed class ScreenLayoutTests : IAsyncLifetime
},
Token);
/// <summary>Lays the transfers screen out at the width it gets beside the nav rail.</summary>
private Task MeasureTransfersAsync(Action<IReadOnlyList<string>> assert) =>
LayoutHarness.OnTheUiThreadAsync(
() =>
{
var screen = new TransfersScreen { DataContext = transfers };
var window = LayoutHarness.HostAtMinimumSize(
screen, LayoutHarness.ScreenWidth, LayoutHarness.ScreenHeight);
try
{
assert(LayoutHarness.Unreachable(window));
}
finally
{
window.Close();
}
},
Token);
/// <summary>Puts one transfer on the queue in a given state, without moving a byte.</summary>
private void Enqueue(
TransferDirection direction,
string name,
long length,
long transferred,
TransferState state,
double bytesPerSecond = 0,
string? failure = null) =>
transfers.Transfers.Add(new TransferRowViewModel(new TransferSnapshot(
Guid.CreateVersion7(),
direction,
name,
Path.Combine(Path.GetTempPath(), name),
SftpPath.Combine("/srv/releases", name),
length,
transferred,
state,
bytesPerSecond,
failure)));
/// <summary>Lays the vault screen out at the width it gets once the nav rail has taken its column.</summary>
private Task MeasureVaultAsync(Action<IReadOnlyList<string>> assert) =>
OnTheVaultAsync((_, window) => assert(LayoutHarness.Unreachable(window)));
tests/DodoSSH.Client.App.Layout.Tests/packages.lock.json
+8 -1
View File
@@ -487,7 +487,8 @@
"CommunityToolkit.Mvvm": "[8.4.2, )",
"DodoSSH.Client.Session": "[1.0.0, )",
"DodoSSH.Client.Ssh": "[1.0.0, )",
"DodoSSH.Client.Terminal": "[1.0.0, )"
"DodoSSH.Client.Terminal": "[1.0.0, )",
"DodoSSH.Client.Transfer": "[1.0.0, )"
}
},
"dodossh.client.auth": {
@@ -538,6 +539,12 @@
"DodoSSH.Client.Ssh": "[1.0.0, )"
}
},
"dodossh.client.transfer": {
"type": "Project",
"dependencies": {
"DodoSSH.Client.Ssh": "[1.0.0, )"
}
},
"dodossh.contracts": {
"type": "Project"
},
tests/DodoSSH.Client.App.Tests/FakeSsh.cs
+84 -1
View File
@@ -10,7 +10,7 @@ namespace DodoSSH.Client.App.Tests;
/// sshd — which <c>DodoSSH.Client.Ssh.Tests</c> already covers against a container. What this makes
/// testable is everything the connect path does <em>around</em> the connection.
/// </remarks>
internal sealed class FakeSshConnectionFactory : ISshConnectionFactory
internal sealed class FakeSshConnectionFactory : ISshConnectionFactory, ISftpSessionFactory
{
/// <summary>Thrown instead of connecting, when set. Used for the host-key paths.</summary>
internal Exception? Failure { get; set; }
@@ -18,6 +18,14 @@ internal sealed class FakeSshConnectionFactory : ISshConnectionFactory
/// <summary>Requests this factory was asked for, in order.</summary>
internal List<SshConnectionRequest> Requests { get; } = [];
/// <summary>Requests for a file-transfer session, in order.</summary>
/// <remarks>
/// Kept apart from <see cref="Requests"/> deliberately: file transfer is a separate connection, and a
/// test asserting that opening a terminal did not also open one would have nothing to look at if the two
/// shared a list.
/// </remarks>
internal List<SshConnectionRequest> SftpRequests { get; } = [];
/// <inheritdoc />
public Task<ISshConnection> ConnectAsync(
SshConnectionRequest request,
@@ -29,6 +37,81 @@ internal sealed class FakeSshConnectionFactory : ISshConnectionFactory
? Task.FromException<ISshConnection>(failure)
: Task.FromResult<ISshConnection>(new FakeSshConnection(request));
}
/// <inheritdoc />
public Task<ISftpSession> OpenSftpAsync(
SshConnectionRequest request,
CancellationToken cancellationToken)
{
SftpRequests.Add(request);
return Failure is { } failure
? Task.FromException<ISftpSession>(failure)
: Task.FromResult<ISftpSession>(new FakeSftpSession(request));
}
}
/// <summary>A remote filesystem with one directory in it.</summary>
/// <remarks>
/// Enough for the shell suite, which is about what the screen does around a session rather than about
/// moving bytes. The queue's own behaviour is covered against a fuller fake in
/// <c>DodoSSH.Client.Transfer.Tests</c>, and the real subsystem against a container in
/// <c>DodoSSH.Client.Ssh.Tests</c>.
/// </remarks>
internal sealed class FakeSftpSession(SshConnectionRequest request) : ISftpSession
{
/// <inheritdoc />
public bool IsConnected { get; private set; } = true;
/// <inheritdoc />
public HostKeyPresentation HostKey { get; } =
new(request.Host, request.Port, "ssh-ed25519", "SHA256:fake");
/// <inheritdoc />
public string HomeDirectory => $"/home/{request.Username}";
/// <inheritdoc />
public Task<IReadOnlyList<SftpEntry>> ListAsync(string path, CancellationToken cancellationToken) =>
Task.FromResult<IReadOnlyList<SftpEntry>>(
[
new SftpEntry(
"notes.txt",
SftpPath.Combine(path, "notes.txt"),
SftpEntryKind.File,
12,
DateTimeOffset.UnixEpoch,
"-rw-r--r--"),
]);
/// <inheritdoc />
public Task<SftpEntry?> StatAsync(string path, CancellationToken cancellationToken) =>
Task.FromResult<SftpEntry?>(null);
/// <inheritdoc />
public Task<Stream> OpenReadAsync(string path, long offset, CancellationToken cancellationToken) =>
Task.FromResult<Stream>(new MemoryStream("hello there\n"u8.ToArray(), writable: false));
/// <inheritdoc />
public Task<Stream> OpenWriteAsync(string path, long offset, CancellationToken cancellationToken) =>
Task.FromResult<Stream>(new MemoryStream());
/// <inheritdoc />
public Task CreateDirectoryAsync(string path, CancellationToken cancellationToken) =>
Task.CompletedTask;
/// <inheritdoc />
public Task DeleteAsync(string path, CancellationToken cancellationToken) => Task.CompletedTask;
/// <inheritdoc />
public Task RenameAsync(string fromPath, string toPath, CancellationToken cancellationToken) =>
Task.CompletedTask;
/// <inheritdoc />
public ValueTask DisposeAsync()
{
IsConnected = false;
return ValueTask.CompletedTask;
}
}
internal sealed class FakeSshConnection(SshConnectionRequest request) : ISshConnection
tests/DodoSSH.Client.App.Tests/ShellFlowTests.cs
+87 -2
View File
@@ -110,6 +110,7 @@ public sealed class ShellFlowTests : IAsyncLifetime
deviceKeys,
SignInAsync,
TimeProvider.System,
ssh,
CheapProfile);
return ValueTask.CompletedTask;
@@ -293,6 +294,7 @@ public sealed class ShellFlowTests : IAsyncLifetime
new UnavailableDeviceKeyStore(),
(_, _) => throw new InvalidOperationException("The shell went to the network to unlock."),
TimeProvider.System,
ssh,
CheapProfile);
await using var _ = offline.ConfigureAwait(false);
@@ -738,6 +740,7 @@ public sealed class ShellFlowTests : IAsyncLifetime
new UnavailableDeviceKeyStore(),
(_, _) => throw new InvalidOperationException("unreachable"),
TimeProvider.System,
ssh,
CheapProfile);
await using var _ = offline.ConfigureAwait(false);
@@ -1966,7 +1969,8 @@ public sealed class ShellFlowTests : IAsyncLifetime
await AddHostAsync(vault, "prod-web-01");
await AddHostAsync(vault, "prod-web-02");
vault.SelectedHost = vault.Hosts.Single(host => host.Label == "prod-web-01");
vault.SelectedHost = vault.Hosts.Single(
host => string.Equals(host.Label, "prod-web-01", StringComparison.Ordinal));
vault.HostFilter = "prod";
@@ -1999,7 +2003,8 @@ public sealed class ShellFlowTests : IAsyncLifetime
await AddHostAsync(vault, "prod-db");
await AddHostAsync(vault, "stage-web");
vault.SelectedHost = vault.Hosts.Single(host => host.Label == "stage-web");
vault.SelectedHost = vault.Hosts.Single(
host => string.Equals(host.Label, "stage-web", StringComparison.Ordinal));
await vault.LoadAsync(Token);
@@ -2153,6 +2158,7 @@ public sealed class ShellFlowTests : IAsyncLifetime
deviceKeys,
(_, _) => throw new InvalidOperationException("The shell went to the network."),
TimeProvider.System,
ssh,
CheapProfile);
await using var _ = offline.ConfigureAwait(false);
@@ -2210,6 +2216,85 @@ public sealed class ShellFlowTests : IAsyncLifetime
shell.State.ShouldBe(ShellState.Unlocked, shell.StatusMessage);
}
// ---- File transfer ----
[Fact]
public async Task TheTransfersScreen_TakesItsHostListFromTheUnlockedVault()
{
await UnlockedAsync();
await AddHostAsync(shell.Vault!, "prod-db");
// Attached at unlock, after the vault has loaded. Before that ordering was right the picker was
// empty until something else happened to reload it.
shell.Transfers.Hosts.ShouldBeEmpty("the vault had no hosts when it was attached");
await shell.LockCommand.ExecuteAsync(null);
shell.Passphrase = Passphrase;
await shell.UnlockCommand.ExecuteAsync(null);
shell.Transfers.Hosts.Select(host => host.Label).ShouldBe(["prod-db"]);
}
/// <remarks>
/// The lock policy, applied to the other thing that can be in flight. <c>LockAsync</c> argues that
/// locking must not destroy work — it is what somebody does when they walk away from the machine, which
/// is exactly when a long transfer is most likely to be running. A screen rebuilt per unlock would have
/// dropped the session and with it whatever was moving.
/// </remarks>
[Fact]
public async Task Locking_LeavesAFileTransferConnectionOpenAndOnlyTakesTheHostListAway()
{
await UnlockedAsync();
var vault = shell.Vault!;
await AddHostAsync(vault, "prod-db");
shell.Transfers.Attach(vault, knownHosts);
shell.Transfers.SelectedHost = shell.Transfers.Hosts[0];
await shell.Transfers.ConnectCommand.ExecuteAsync(null);
shell.Transfers.IsConnected.ShouldBeTrue(shell.Transfers.Status);
await shell.LockCommand.ExecuteAsync(null);
shell.Transfers.IsConnected.ShouldBeTrue("locking the vault is not a disconnect");
// What it does take is the host list, and it has to: those rows carry decrypted secrets and the
// vault they came from has just been disposed.
shell.Transfers.Hosts.ShouldBeEmpty();
shell.Transfers.SelectedHost.ShouldBeNull();
}
/// <remarks>
/// The consequence of SSH.NET having no way to open an SFTP subsystem on an existing transport, made
/// visible: browsing a host's files authenticates again rather than reusing the terminal's connection.
/// It is asserted rather than merely written down because the host's audit log shows a second login,
/// and somebody will eventually be asked to explain it.
/// </remarks>
[Fact]
public async Task ConnectingTheTransfersScreen_OpensItsOwnConnectionRatherThanReusingATerminals()
{
var vault = await ReadyToConnectAsync();
await ConnectWithRendererAsync(vault);
ssh.Requests.Count.ShouldBe(1);
ssh.SftpRequests.ShouldBeEmpty("opening a terminal must not open a file-transfer session");
shell.Transfers.Attach(vault, knownHosts);
shell.Transfers.SelectedHost = shell.Transfers.Hosts[0];
await shell.Transfers.ConnectCommand.ExecuteAsync(null);
ssh.SftpRequests.Count.ShouldBe(1);
ssh.Requests.Count.ShouldBe(1, "and it must not open a shell either");
// It opens on the account's home directory, which is the only path the layer knows without asking.
shell.Transfers.RemotePath.ShouldBe("/home/deploy");
shell.Transfers.RemoteEntries.Select(entry => entry.Name).ShouldBe(["notes.txt"]);
}
private async Task UnlockedAsync()
{
await EnrolledAndConfirmedAsync();
tests/DodoSSH.Client.App.Tests/packages.lock.json
+8 -1
View File
@@ -476,7 +476,8 @@
"CommunityToolkit.Mvvm": "[8.4.2, )",
"DodoSSH.Client.Session": "[1.0.0, )",
"DodoSSH.Client.Ssh": "[1.0.0, )",
"DodoSSH.Client.Terminal": "[1.0.0, )"
"DodoSSH.Client.Terminal": "[1.0.0, )",
"DodoSSH.Client.Transfer": "[1.0.0, )"
}
},
"dodossh.client.auth": {
@@ -527,6 +528,12 @@
"DodoSSH.Client.Ssh": "[1.0.0, )"
}
},
"dodossh.client.transfer": {
"type": "Project",
"dependencies": {
"DodoSSH.Client.Ssh": "[1.0.0, )"
}
},
"dodossh.contracts": {
"type": "Project"
},
tests/DodoSSH.Client.Ssh.Tests/RemotePathTests.cs
+99
View File
@@ -0,0 +1,99 @@
namespace DodoSSH.Client.Ssh.Tests;
/// <summary>
/// Remote paths, permission bits and byte counts — the three things the file browser renders.
/// </summary>
/// <remarks>
/// Not in the SSH collection, so this class needs no container. Every case here is one where the obvious
/// implementation is wrong on Windows, at a filesystem root, or on a number that has just crossed a
/// threshold — which is to say, one that a listing of somebody's home directory would not reveal.
/// </remarks>
public sealed class RemotePathTests
{
[Theory]
[InlineData("/var/log", "syslog", "/var/log/syslog")]
[InlineData("/", "etc", "/etc")]
[InlineData("/home/dodo/", "notes", "/home/dodo/notes")]
public void Combine_JoinsWithExactlyOneSeparator(string directory, string name, string expected)
{
// Deliberately not Path.Combine, which on Windows would yield "/var/log\syslog" — a path the remote
// cannot resolve, failing as "no such file" somewhere the backslash is invisible.
SftpPath.Combine(directory, name).ShouldBe(expected);
}
[Theory]
[InlineData("/var/log/syslog", "/var/log")]
[InlineData("/var/log", "/var")]
[InlineData("/var", "/")]
[InlineData("/", "/")]
[InlineData("/var/log/", "/var")]
public void Parent_StopsAtTheRoot(string path, string expected)
{
// The root is its own parent rather than null, which is what lets the breadcrumb's "up" be a plain
// navigation with nothing above it to special-case.
SftpPath.Parent(path).ShouldBe(expected);
}
[Theory]
[InlineData("/var/log/syslog", "syslog")]
[InlineData("/var/log/", "log")]
[InlineData("/", "/")]
public void Name_IsTheLastSegment(string path, string expected)
{
SftpPath.Name(path).ShouldBe(expected);
}
[Fact]
public void Trail_NamesEverySegmentWithThePathThatReachesIt()
{
SftpPath.Trail("/var/log/nginx").ShouldBe(
[("var", "/var"), ("log", "/var/log"), ("nginx", "/var/log/nginx")]);
}
[Fact]
public void Trail_IsEmptyAtTheRoot()
{
// The root has no segment to name. The breadcrumb draws it as a leading separator, so a trail with a
// phantom empty crumb in it would render as a button with no label.
SftpPath.Trail("/").ShouldBeEmpty();
}
[Fact]
public void PosixMode_RendersTheKindCharacterAndThreeTriples()
{
PosixMode.Format(
SftpEntryKind.Directory,
ownerRead: true, ownerWrite: true, ownerExecute: true,
groupRead: true, groupWrite: false, groupExecute: true,
othersRead: true, othersWrite: false, othersExecute: true)
.ShouldBe("drwxr-xr-x");
PosixMode.Format(
SftpEntryKind.File,
ownerRead: true, ownerWrite: true, ownerExecute: false,
groupRead: true, groupWrite: false, groupExecute: false,
othersRead: false, othersWrite: false, othersExecute: false)
.ShouldBe("-rw-r-----");
PosixMode.Format(
SftpEntryKind.SymbolicLink,
ownerRead: true, ownerWrite: true, ownerExecute: true,
groupRead: true, groupWrite: true, groupExecute: true,
othersRead: true, othersWrite: true, othersExecute: true)
.ShouldBe("lrwxrwxrwx");
}
[Theory]
[InlineData(0, "0 B")]
[InlineData(1023, "1023 B")]
[InlineData(1024, "1.0 KB")]
[InlineData(10 * 1024, "10 KB")]
[InlineData(1536 * 1024, "1.5 MB")]
[InlineData(5L * 1024 * 1024 * 1024, "5.0 GB")]
public void ByteSize_KeepsOneDecimalOnlyWhileItMeansSomething(long bytes, string expected)
{
// The boundary at ten is the whole rule: "9.4 MB" and "512 MB" carry the same information, and
// "512.3 MB" claims a precision the figure does not have by the time it is that large.
ByteSize.Format(bytes).ShouldBe(expected);
}
}
tests/DodoSSH.Client.Ssh.Tests/SftpSessionTests.cs
+262
View File
@@ -0,0 +1,262 @@
using System.Text;
namespace DodoSSH.Client.Ssh.Tests;
/// <summary>
/// The SFTP subsystem, against a real sshd.
/// </summary>
/// <remarks>
/// <para>
/// Everything this suite is about is behaviour of a server rather than of this code: whether a listing
/// carries the permission bits the design's <c>PERMS</c> column needs, whether opening at an offset really
/// starts there, and whether a rename over a name that already exists fails rather than silently replacing —
/// which the transfer queue's part-file scheme depends on. None of it can be established against a mock.
/// </para>
/// <para>
/// One directory per test, named after it, because the container is shared with every other suite in the
/// assembly and a test that cleaned up by emptying the home directory would take another test's fixture with
/// it. The <em>session</em> is shared too, and that is a limit of the server rather than tidiness — see
/// <see cref="SshServerFixture.SftpAsync"/>, which explains what opening one per test did to the rest of the
/// assembly.
/// </para>
/// </remarks>
[Collection(SshCollection.Name)]
public sealed class SftpSessionTests(SshServerFixture fixture)
{
private static CancellationToken Token => TestContext.Current.CancellationToken;
[Fact]
public async Task AnOpenedSession_StartsInTheAccountsHomeDirectory()
{
var sftp = await fixture.SftpAsync(Token);
// Absolute, because the server canonicalises it during the handshake. A relative answer would make
// every path the browser builds relative too, and the breadcrumb trail meaningless.
SftpPath.IsAbsolute(sftp.HomeDirectory).ShouldBeTrue(sftp.HomeDirectory);
sftp.IsConnected.ShouldBeTrue();
}
[Fact]
public async Task AListing_CarriesKindSizeAndPermissions()
{
var sftp = await fixture.SftpAsync(Token);
var directory = await MakeDirectoryAsync(sftp, nameof(AListing_CarriesKindSizeAndPermissions));
var content = "the quick brown fox"u8.ToArray();
await WriteAsync(sftp, SftpPath.Combine(directory, "a-file"), content);
await sftp.CreateDirectoryAsync(SftpPath.Combine(directory, "a-directory"), Token);
var entries = await sftp.ListAsync(directory, Token);
// Directories first: the order this interface promises, and what the file browser shows without
// sorting again.
entries.Select(entry => entry.Name).ShouldBe(["a-directory", "a-file"]);
var file = entries[1];
file.Kind.ShouldBe(SftpEntryKind.File);
file.Length.ShouldBe(content.Length);
file.FullPath.ShouldBe(SftpPath.Combine(directory, "a-file"));
// The one column nothing in this repository could render before. The exact bits depend on the
// server's umask, so what is pinned is the shape and the kind character rather than the mode.
file.Permissions.Length.ShouldBe(10);
file.Permissions[0].ShouldBe('-');
file.Permissions.ShouldStartWith("-rw");
entries[0].Kind.ShouldBe(SftpEntryKind.Directory);
entries[0].Permissions[0].ShouldBe('d');
}
[Fact]
public async Task AListing_DropsTheDotEntries()
{
var sftp = await fixture.SftpAsync(Token);
var directory = await MakeDirectoryAsync(sftp, nameof(AListing_DropsTheDotEntries));
// Empty, which is the case where "." and ".." are the whole listing — so a browser that showed them
// would present an empty directory as one holding two things.
var entries = await sftp.ListAsync(directory, Token);
entries.ShouldBeEmpty();
}
[Fact]
public async Task ListingSomethingThatIsNotADirectory_FailsWithThePath()
{
var sftp = await fixture.SftpAsync(Token);
var directory = await MakeDirectoryAsync(
sftp, nameof(ListingSomethingThatIsNotADirectory_FailsWithThePath));
var file = SftpPath.Combine(directory, "not-a-directory");
await WriteAsync(sftp, file, "x"u8.ToArray());
// The path is the whole point of the translation. SSH.NET's own exception for this carries the
// server's message and nothing about which path was asked for, and the browser has to say which
// row failed.
var failure = await Should.ThrowAsync<SftpPathException>(async () =>
await sftp.ListAsync(file, Token));
failure.Path.ShouldBe(file);
}
[Fact]
public async Task Stat_AnswersNullForSomethingThatIsNotThere()
{
var sftp = await fixture.SftpAsync(Token);
var directory = await MakeDirectoryAsync(
sftp, nameof(Stat_AnswersNullForSomethingThatIsNotThere));
// Absent is an answer rather than a failure: the transfer queue asks this before every upload to
// find out whether it would be overwriting something, and that question has a "no".
(await sftp.StatAsync(SftpPath.Combine(directory, "nothing-here"), Token)).ShouldBeNull();
}
[Fact]
public async Task OpeningAtAnOffset_ReadsFromThere()
{
var sftp = await fixture.SftpAsync(Token);
var directory = await MakeDirectoryAsync(sftp, nameof(OpeningAtAnOffset_ReadsFromThere));
var path = SftpPath.Combine(directory, "resumable");
await WriteAsync(sftp, path, "0123456789"u8.ToArray());
// The whole of resume, in one call. If the server ignored the offset this would read the file from
// the start and a resumed download would silently duplicate its first half.
var stream = await sftp.OpenReadAsync(path, 4, Token);
await using var scope = stream.ConfigureAwait(false);
using var reader = new StreamReader(stream, Encoding.UTF8);
(await reader.ReadToEndAsync(Token)).ShouldBe("456789");
}
[Fact]
public async Task WritingAtAnOffset_LeavesWhatWasAlreadyThere()
{
var sftp = await fixture.SftpAsync(Token);
var directory = await MakeDirectoryAsync(sftp, nameof(WritingAtAnOffset_LeavesWhatWasAlreadyThere));
var path = SftpPath.Combine(directory, "appended");
await WriteAsync(sftp, path, "0123"u8.ToArray());
var stream = await sftp.OpenWriteAsync(path, 4, Token);
await using (stream.ConfigureAwait(false))
{
await stream.WriteAsync("456789"u8.ToArray(), Token);
await stream.FlushAsync(Token);
}
// The other half of resume: an upload that carried on from an offset must not have truncated the
// bytes an earlier attempt already delivered.
(await ReadAllAsync(sftp, path)).ShouldBe("0123456789");
}
[Fact]
public async Task Rename_RefusesToReplaceSomethingThatIsAlreadyThere()
{
var sftp = await fixture.SftpAsync(Token);
var directory = await MakeDirectoryAsync(
sftp, nameof(Rename_RefusesToReplaceSomethingThatIsAlreadyThere));
var part = SftpPath.Combine(directory, "artefact.dodossh-part");
var destination = SftpPath.Combine(directory, "artefact");
await WriteAsync(sftp, part, "new"u8.ToArray());
await WriteAsync(sftp, destination, "old"u8.ToArray());
// The transfer queue's last step, and the assumption underneath its promise never to overwrite: it
// checks the destination before starting, and this is what stops a file that appeared in the
// meantime from being replaced anyway. SFTP's rename is specified not to clobber, and this is the
// check that the server this project tests against actually behaves that way.
await Should.ThrowAsync<SftpPathException>(async () =>
await sftp.RenameAsync(part, destination, Token));
(await ReadAllAsync(sftp, destination)).ShouldBe("old");
}
[Fact]
public async Task DeletingANonEmptyDirectory_Fails()
{
var sftp = await fixture.SftpAsync(Token);
var directory = await MakeDirectoryAsync(sftp, nameof(DeletingANonEmptyDirectory_Fails));
await WriteAsync(sftp, SftpPath.Combine(directory, "occupant"), "x"u8.ToArray());
// Deliberate, and the reason this interface has no recursive delete: the one destructive operation
// on the transfers screen must not be able to take a directory tree with it.
await Should.ThrowAsync<SftpPathException>(async () => await sftp.DeleteAsync(directory, Token));
}
[Fact]
public async Task AnUntrustedHost_IsRefusedExactlyAsAShellWouldBe()
{
var factory = new SshNetConnectionFactory(new InMemoryKnownHostStore());
// File transfer opens its own connection, so it makes its own first-contact decision. The failure
// that matters is the one this asserts is *not* raised: a bare connection error would send the user
// looking at the network for what is a fingerprint they have not approved.
var refusal = await Should.ThrowAsync<SshHostKeyUnknownException>(async () =>
await factory.OpenSftpAsync(Request(), Token));
refusal.Presentation.Host.ShouldBe(fixture.Host);
refusal.Presentation.Fingerprint.ShouldStartWith("SHA256:");
}
private SshConnectionRequest Request() => new(
fixture.Host,
fixture.Port,
SshServerFixture.Username,
new SshPasswordCredential(SshServerFixture.Password));
/// <summary>A directory of this test's own, under the account's home.</summary>
private static async Task<string> MakeDirectoryAsync(ISftpSession sftp, string name)
{
var path = SftpPath.Combine(sftp.HomeDirectory, $"sftp-{name}");
if (await sftp.StatAsync(path, Token) is not null)
{
// A previous run of the same test in a container that outlived it. Emptying it is enough:
// nothing here creates nested directories.
foreach (var entry in await sftp.ListAsync(path, Token))
{
await sftp.DeleteAsync(entry.FullPath, Token);
}
return path;
}
await sftp.CreateDirectoryAsync(path, Token);
return path;
}
private static async Task WriteAsync(ISftpSession sftp, string path, byte[] content)
{
var stream = await sftp.OpenWriteAsync(path, 0, Token);
await using (stream.ConfigureAwait(false))
{
await stream.WriteAsync(content, Token);
await stream.FlushAsync(Token);
}
}
private static async Task<string> ReadAllAsync(ISftpSession sftp, string path)
{
var stream = await sftp.OpenReadAsync(path, 0, Token);
await using var scope = stream.ConfigureAwait(false);
using var reader = new StreamReader(stream, Encoding.UTF8);
return await reader.ReadToEndAsync(Token);
}
}
tests/DodoSSH.Client.Ssh.Tests/SshServerFixture.cs
+63
View File
@@ -29,7 +29,10 @@ public sealed class SshServerFixture : IAsyncLifetime
private const int SshPort = 2222;
private readonly SemaphoreSlim sftpGate = new(1, 1);
private IContainer? container;
private ISftpSession? sftp;
/// <summary>Host port the container's sshd is published on.</summary>
public ushort Port => container!.GetMappedPublicPort(SshPort);
@@ -68,9 +71,69 @@ public sealed class SshServerFixture : IAsyncLifetime
await container.StartAsync();
}
/// <summary>
/// One file-transfer session, opened on first use and shared by every test that wants one.
/// </summary>
/// <remarks>
/// <para>
/// Shared rather than opened per test, and that is a limit of the server rather than an optimisation.
/// sshd's <c>MaxStartups</c> drops connections at random once enough are part-way through a handshake,
/// and this client's first contact with an unknown host is a connection deliberately <em>refused</em> at
/// the host key — so a suite that opened its own session per test made two handshakes per test and
/// pushed the whole assembly over the threshold. What that looks like is unrelated tests failing with
/// "the connection was closed by the remote host", a different few each run.
/// </para>
/// <para>
/// Safe to share because an SFTP session holds no per-test state: every test here works in a directory
/// named after itself. See <c>ISftpSession</c>, which is one channel and is used by one caller at a
/// time.
/// </para>
/// </remarks>
public async ValueTask<ISftpSession> SftpAsync(CancellationToken cancellationToken)
{
await sftpGate.WaitAsync(cancellationToken);
try
{
if (sftp is not null)
{
return sftp;
}
var knownHosts = new InMemoryKnownHostStore();
var factory = new SshNetConnectionFactory(knownHosts);
var request = new SshConnectionRequest(
Host, Port, Username, new SshPasswordCredential(Password));
try
{
// Learned by being refused, which is the only way this client learns a host key.
return sftp = await factory.OpenSftpAsync(request, cancellationToken);
}
catch (SshHostKeyUnknownException unknown)
{
await knownHosts.TrustAsync(unknown.Presentation, cancellationToken);
}
return sftp = await factory.OpenSftpAsync(request, cancellationToken);
}
finally
{
sftpGate.Release();
}
}
/// <inheritdoc />
public async ValueTask DisposeAsync()
{
if (sftp is not null)
{
await sftp.DisposeAsync();
}
sftpGate.Dispose();
if (container is not null)
{
await container.DisposeAsync();
tests/DodoSSH.Client.Transfer.Tests/DodoSSH.Client.Transfer.Tests.csproj
+16
View File
@@ -0,0 +1,16 @@
<Project Sdk="Microsoft.NET.Sdk">
<!--
The queue, against a real temporary directory and a fake host. Real files on the local side
because the part-file scheme is about what is on disk when a transfer stops halfway, and a
filesystem abstraction would let that be right in the test and wrong in the product; a fake on
the remote side because the failures worth pinning here — a read that dies mid-file, a
destination that appears while a transfer is queued — are ones no real server can be asked for
on cue. What the real server does answer is in DodoSSH.Client.Ssh.Tests.
-->
<ItemGroup>
<ProjectReference Include="../../src/DodoSSH.Client.Transfer/DodoSSH.Client.Transfer.csproj" />
</ItemGroup>
</Project>
tests/DodoSSH.Client.Transfer.Tests/FakeSftpSession.cs
+258
View File
@@ -0,0 +1,258 @@
using DodoSSH.Client.Ssh;
namespace DodoSSH.Client.Transfer.Tests;
/// <summary>
/// A remote filesystem in a dictionary.
/// </summary>
/// <remarks>
/// Here rather than a container because what this suite is about is the queue's behaviour when a transfer
/// goes wrong halfway — a read that dies after so many bytes, a destination that appears while a transfer is
/// queued — and neither can be asked of a real server on cue. That the real server behaves as this fake
/// pretends is established separately, against sshd, in <c>SftpSessionTests</c>.
/// </remarks>
internal sealed class FakeSftpSession : ISftpSession
{
private readonly Dictionary<string, byte[]> files = new(StringComparer.Ordinal);
private readonly HashSet<string> directories = new(StringComparer.Ordinal) { "/", "/home/dodo" };
private readonly Lock gate = new();
/// <inheritdoc />
public bool IsConnected => true;
/// <inheritdoc />
public HostKeyPresentation HostKey { get; } = new("host.internal", 22, "ssh-ed25519", "SHA256:fake");
/// <inheritdoc />
public string HomeDirectory => "/home/dodo";
/// <summary>Throws once, this many bytes into the next read, and then stops doing so.</summary>
/// <remarks>
/// One-shot on purpose: every resume test is "it broke, then it did not", and a fake that kept failing
/// would need turning off at exactly the point the assertion is about.
/// </remarks>
public int? FailReadAfter { get; set; }
/// <summary>The offsets <see cref="OpenReadAsync"/> was asked to start at, in order.</summary>
/// <remarks>
/// The only direct evidence that a resume resumed. A test can see the right bytes on disk at the end
/// whether the second attempt started at the offset or at zero.
/// </remarks>
public List<long> ReadOffsets { get; } = [];
/// <summary>Puts a file on the fake host.</summary>
public void Seed(string path, byte[] content)
{
lock (gate)
{
files[path] = content;
directories.Add(SftpPath.Parent(path));
}
}
/// <summary>What is at a path, or null.</summary>
public byte[]? Read(string path)
{
lock (gate)
{
return files.GetValueOrDefault(path);
}
}
/// <summary>Whether anything is at a path.</summary>
public bool Exists(string path)
{
lock (gate)
{
return files.ContainsKey(path) || directories.Contains(path);
}
}
/// <inheritdoc />
public Task<IReadOnlyList<SftpEntry>> ListAsync(string path, CancellationToken cancellationToken)
{
lock (gate)
{
if (!directories.Contains(path))
{
throw new SftpPathException(path, $"{path} is not there.");
}
IReadOnlyList<SftpEntry> entries =
[
.. files
.Where(file => string.Equals(SftpPath.Parent(file.Key), path, StringComparison.Ordinal))
.Select(file => Describe(file.Key, file.Value.Length)),
];
return Task.FromResult(entries);
}
}
/// <inheritdoc />
public Task<SftpEntry?> StatAsync(string path, CancellationToken cancellationToken)
{
lock (gate)
{
if (files.TryGetValue(path, out var content))
{
return Task.FromResult<SftpEntry?>(Describe(path, content.Length));
}
return Task.FromResult<SftpEntry?>(
directories.Contains(path)
? new SftpEntry(
SftpPath.Name(path),
path,
SftpEntryKind.Directory,
0,
DateTimeOffset.UnixEpoch,
"drwxr-xr-x")
: null);
}
}
/// <inheritdoc />
public Task<Stream> OpenReadAsync(string path, long offset, CancellationToken cancellationToken)
{
lock (gate)
{
ReadOffsets.Add(offset);
if (!files.TryGetValue(path, out var content))
{
throw new SftpPathException(path, $"{path} is not there.");
}
var failAfter = FailReadAfter;
FailReadAfter = null;
return Task.FromResult<Stream>(
new BrittleStream(content.AsSpan((int)offset).ToArray(), failAfter));
}
}
/// <inheritdoc />
public Task<Stream> OpenWriteAsync(string path, long offset, CancellationToken cancellationToken)
{
lock (gate)
{
var existing = files.GetValueOrDefault(path, []);
return Task.FromResult<Stream>(new CommittingStream(
existing.AsSpan(0, (int)Math.Min(offset, existing.Length)).ToArray(),
content =>
{
lock (gate)
{
files[path] = content;
}
}));
}
}
/// <inheritdoc />
public Task CreateDirectoryAsync(string path, CancellationToken cancellationToken)
{
lock (gate)
{
directories.Add(path);
}
return Task.CompletedTask;
}
/// <inheritdoc />
public Task DeleteAsync(string path, CancellationToken cancellationToken)
{
lock (gate)
{
if (!files.Remove(path) && !directories.Remove(path))
{
throw new SftpPathException(path, $"{path} is not there.");
}
}
return Task.CompletedTask;
}
/// <inheritdoc />
public Task RenameAsync(string fromPath, string toPath, CancellationToken cancellationToken)
{
lock (gate)
{
if (!files.TryGetValue(fromPath, out var content))
{
throw new SftpPathException(fromPath, $"{fromPath} is not there.");
}
// SFTP's rename does not replace, and the queue's promise never to overwrite rests on it. A fake
// that clobbered would make the one test about that promise pass for the wrong reason.
if (files.ContainsKey(toPath))
{
throw new SftpPathException(toPath, $"{toPath} is already there.");
}
files.Remove(fromPath);
files[toPath] = content;
}
return Task.CompletedTask;
}
/// <inheritdoc />
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
private static SftpEntry Describe(string path, int length) => new(
SftpPath.Name(path),
path,
SftpEntryKind.File,
length,
DateTimeOffset.UnixEpoch,
"-rw-r--r--");
/// <summary>A read that dies partway through, the way a dropped connection does.</summary>
private sealed class BrittleStream(byte[] content, int? failAfter) : MemoryStream(content, writable: false)
{
public override int Read(Span<byte> buffer)
{
Guard();
return base.Read(buffer);
}
public override ValueTask<int> ReadAsync(
Memory<byte> buffer,
CancellationToken cancellationToken = default)
{
Guard();
return base.ReadAsync(buffer, cancellationToken);
}
private void Guard()
{
if (failAfter is { } limit && Position >= limit)
{
throw new IOException("The connection dropped.");
}
}
}
/// <summary>A write that lands on the fake host when it is disposed.</summary>
private sealed class CommittingStream(byte[] prefix, Action<byte[]> commit) : MemoryStream()
{
private bool committed;
public override void Close()
{
if (!committed)
{
committed = true;
commit([.. prefix, .. ToArray()]);
}
base.Close();
}
}
}
tests/DodoSSH.Client.Transfer.Tests/FileTransferQueueTests.cs
+365
View File
@@ -0,0 +1,365 @@
using DodoSSH.Client.Ssh;
namespace DodoSSH.Client.Transfer.Tests;
/// <summary>
/// The transfer queue: what ends up on disk, and what happens when a transfer stops halfway.
/// </summary>
/// <remarks>
/// Every assertion here is about a promise the queue makes in prose — nothing is written at its final name
/// until it is complete, a destination that already exists is refused rather than overwritten, an
/// interrupted transfer carries on rather than starting again, and a partial file left by something else is
/// never resumed from. Those are the four ways a file transfer can quietly deliver the wrong bytes.
/// </remarks>
public sealed class FileTransferQueueTests : IDisposable
{
private const string RemoteDirectory = "/home/dodo";
private readonly FakeSftpSession host = new();
private readonly string workspace = Directory.CreateTempSubdirectory("dodossh-transfer-").FullName;
private static CancellationToken Token => TestContext.Current.CancellationToken;
/// <inheritdoc />
public void Dispose()
{
try
{
Directory.Delete(workspace, recursive: true);
}
catch (IOException)
{
// A handle the runtime has not released yet. The directory is under the system temporary path
// and failing a passing test over it would be the wrong trade.
}
}
[Fact]
public async Task ADownload_LandsAtItsFinalNameWithNoPartFileLeftBehind()
{
var content = Bytes(300_000);
host.Seed(Remote("artefact.tar"), content);
await using var queue = NewQueue();
var finished = await RunToCompletionAsync(
queue, TransferDirection.Download, Local("artefact.tar"), Remote("artefact.tar"), content.Length);
finished.State.ShouldBe(TransferState.Completed);
finished.Transferred.ShouldBe(content.Length);
(await File.ReadAllBytesAsync(Local("artefact.tar"), Token)).ShouldBe(content);
File.Exists(Local("artefact.tar") + FileTransferQueue.PartSuffix).ShouldBeFalse();
}
[Fact]
public async Task ADownloadOntoAFileThatIsAlreadyThere_IsRefusedAndLeavesItAlone()
{
host.Seed(Remote("artefact.tar"), Bytes(1_000));
await File.WriteAllTextAsync(Local("artefact.tar"), "something else entirely", Token);
await using var queue = NewQueue();
var finished = await RunToCompletionAsync(
queue, TransferDirection.Download, Local("artefact.tar"), Remote("artefact.tar"), 1_000);
finished.State.ShouldBe(TransferState.Failed);
finished.Failure.ShouldNotBeNull();
// The whole point of the refusal: what was there is still there, unread and unreplaced.
(await File.ReadAllTextAsync(Local("artefact.tar"), Token)).ShouldBe("something else entirely");
}
[Fact]
public async Task AnInterruptedDownload_CarriesOnFromWhereItStopped()
{
var content = Bytes(300_000);
host.Seed(Remote("artefact.tar"), content);
host.FailReadAfter = 100_000;
await using var queue = NewQueue();
var broken = await RunToCompletionAsync(
queue, TransferDirection.Download, Local("artefact.tar"), Remote("artefact.tar"), content.Length);
broken.State.ShouldBe(TransferState.Failed);
broken.Transferred.ShouldBeInRange(1, content.Length - 1);
broken.CanResume.ShouldBeTrue();
// The part file is what makes this a resume rather than a restart, and it is deliberately not at the
// destination's name — nothing may look like a finished download until it is one.
var part = Local("artefact.tar") + FileTransferQueue.PartSuffix;
File.Exists(part).ShouldBeTrue();
File.Exists(Local("artefact.tar")).ShouldBeFalse();
var partLength = new FileInfo(part).Length;
var resumed = await AwaitFinishAsync(queue, () => queue.Retry(broken.Id));
resumed.State.ShouldBe(TransferState.Completed);
// Asked for the second time at the offset the part file had reached, which is the only direct
// evidence that the bytes already moved were not moved again.
host.ReadOffsets.ShouldBe([0, partLength]);
(await File.ReadAllBytesAsync(Local("artefact.tar"), Token)).ShouldBe(content);
}
[Fact]
public async Task AFreshDownload_WillNotResumeFromAPartFileItDidNotWrite()
{
var content = Bytes(200_000);
host.Seed(Remote("artefact.tar"), content);
// Litter from something else entirely — an earlier run of the application, or an earlier attempt at
// a file of the same name that has since changed. Resuming on the strength of the name matching is
// how a corrupt artefact gets delivered with nothing reporting a failure.
await File.WriteAllBytesAsync(
Local("artefact.tar") + FileTransferQueue.PartSuffix, Bytes(50_000, seed: 7), Token);
await using var queue = NewQueue();
var finished = await RunToCompletionAsync(
queue, TransferDirection.Download, Local("artefact.tar"), Remote("artefact.tar"), content.Length);
finished.State.ShouldBe(TransferState.Completed);
host.ReadOffsets.ShouldBe([0]);
(await File.ReadAllBytesAsync(Local("artefact.tar"), Token)).ShouldBe(content);
}
[Fact]
public async Task AnUpload_LandsAtItsFinalNameOnTheHost()
{
var content = Bytes(150_000);
await File.WriteAllBytesAsync(Local("artefact.tar"), content, Token);
await using var queue = NewQueue();
var finished = await RunToCompletionAsync(
queue, TransferDirection.Upload, Local("artefact.tar"), Remote("artefact.tar"), content.Length);
finished.State.ShouldBe(TransferState.Completed);
host.Read(Remote("artefact.tar")).ShouldBe(content);
host.Exists(Remote("artefact.tar") + FileTransferQueue.PartSuffix).ShouldBeFalse();
}
[Fact]
public async Task AnUploadOntoAFileThatIsAlreadyThere_IsRefusedAndLeavesItAlone()
{
var existing = "the running deployment"u8.ToArray();
host.Seed(Remote("artefact.tar"), existing);
await File.WriteAllBytesAsync(Local("artefact.tar"), Bytes(1_000), Token);
await using var queue = NewQueue();
var finished = await RunToCompletionAsync(
queue, TransferDirection.Upload, Local("artefact.tar"), Remote("artefact.tar"), 1_000);
finished.State.ShouldBe(TransferState.Failed);
host.Read(Remote("artefact.tar")).ShouldBe(existing);
}
[Fact]
public async Task TransfersRunOneAtATime()
{
for (var i = 0; i < 4; i++)
{
host.Seed(Remote($"file-{i}"), Bytes(200_000, seed: i));
}
await using var queue = NewQueue();
var running = new HashSet<Guid>();
var most = 0;
var gate = new Lock();
queue.Changed += (_, e) =>
{
lock (gate)
{
if (e.Transfer.State is TransferState.Running)
{
running.Add(e.Transfer.Id);
}
else
{
running.Remove(e.Transfer.Id);
}
most = Math.Max(most, running.Count);
}
};
var ids = new List<Guid>();
for (var i = 0; i < 4; i++)
{
ids.Add(queue.Enqueue(
TransferDirection.Download, Local($"file-{i}"), Remote($"file-{i}"), 200_000));
}
await WaitForQuietAsync(queue);
queue.Snapshot().ShouldAllBe(transfer => transfer.State == TransferState.Completed);
// The claim the whole design rests on: one channel, one transfer, so the throughput on a row is the
// throughput of the link rather than a share of it.
lock (gate)
{
most.ShouldBe(1);
}
ids.Count.ShouldBe(4);
}
[Fact]
public async Task CancellingAQueuedTransfer_TakesItOutOfTheQueueWithoutStartingIt()
{
host.Seed(Remote("slow"), Bytes(2_000_000));
host.Seed(Remote("never"), Bytes(1_000));
await using var queue = NewQueue();
queue.Enqueue(TransferDirection.Download, Local("slow"), Remote("slow"), 2_000_000);
var second = queue.Enqueue(TransferDirection.Download, Local("never"), Remote("never"), 1_000);
queue.Cancel(second);
await WaitForQuietAsync(queue);
var cancelled = queue.Snapshot().Single(transfer => transfer.Id == second);
cancelled.State.ShouldBe(TransferState.Cancelled);
cancelled.Transferred.ShouldBe(0);
File.Exists(Local("never")).ShouldBeFalse();
}
[Fact]
public async Task DiscardingAStoppedDownload_TakesItsPartFileWithIt()
{
var content = Bytes(200_000);
host.Seed(Remote("artefact.tar"), content);
host.FailReadAfter = 60_000;
await using var queue = NewQueue();
var broken = await RunToCompletionAsync(
queue, TransferDirection.Download, Local("artefact.tar"), Remote("artefact.tar"), content.Length);
var part = Local("artefact.tar") + FileTransferQueue.PartSuffix;
File.Exists(part).ShouldBeTrue();
(await queue.DiscardAsync(broken.Id, Token)).ShouldBeTrue();
// The row was the only thing that knew the part file existed, so removing one without the other
// would leave bytes on disk nobody could attribute.
queue.Snapshot().ShouldBeEmpty();
File.Exists(part).ShouldBeFalse();
}
[Fact]
public async Task ATransferWithNoSessionToRunOn_FailsOnItsOwnRowRatherThanSilently()
{
await using var queue = new FileTransferQueue(
_ => Task.FromException<ISftpSession>(new IOException("the host is not reachable")),
TimeProvider.System);
var finished = await RunToCompletionAsync(
queue, TransferDirection.Download, Local("artefact.tar"), Remote("artefact.tar"), 10);
finished.State.ShouldBe(TransferState.Failed);
finished.Failure.ShouldBe("the host is not reachable");
}
private FileTransferQueue NewQueue() =>
new(_ => Task.FromResult<ISftpSession>(host), TimeProvider.System);
private string Local(string name) => Path.Combine(workspace, name);
private static string Remote(string name) => SftpPath.Combine(RemoteDirectory, name);
/// <remarks>
/// Repeatable rather than random, so a failure can be reproduced, and not all one byte, so a resumed
/// transfer that started from the wrong offset produces a file that differs rather than one that happens
/// to match.
/// </remarks>
private static byte[] Bytes(int length, int seed = 0)
{
var content = new byte[length];
for (var i = 0; i < length; i++)
{
content[i] = (byte)((i + seed) % 251);
}
return content;
}
private static Task<TransferSnapshot> RunToCompletionAsync(
FileTransferQueue queue,
TransferDirection direction,
string localPath,
string remotePath,
long length) =>
AwaitFinishAsync(queue, () => queue.Enqueue(direction, localPath, remotePath, length));
/// <summary>
/// Subscribes, starts a transfer, and completes when a transfer reaches a state it will not leave.
/// </summary>
/// <remarks>
/// <para>
/// The subscription goes on <em>before</em> the transfer starts, because the pump runs on a thread-pool
/// thread: a small transfer can be finished before <c>Enqueue</c> has returned its id, so subscribing
/// afterwards would wait for an event that has already happened.
/// </para>
/// <para>
/// Which is also why it does not match on the id — there is nothing to match against yet. Every caller
/// has exactly one transfer in flight, which is what makes "a transfer finished" and "this transfer
/// finished" the same statement. The one test with several running uses
/// <see cref="WaitForQuietAsync"/> instead.
/// </para>
/// </remarks>
private static async Task<TransferSnapshot> AwaitFinishAsync(FileTransferQueue queue, Action start)
{
var finished = new TaskCompletionSource<TransferSnapshot>(
TaskCreationOptions.RunContinuationsAsynchronously);
void OnChanged(object? sender, TransferChangedEventArgs e)
{
if (e.Transfer.IsFinished)
{
finished.TrySetResult(e.Transfer);
}
}
queue.Changed += OnChanged;
try
{
start();
return await finished.Task.WaitAsync(TimeSpan.FromSeconds(30), Token);
}
finally
{
queue.Changed -= OnChanged;
}
}
/// <summary>Waits until nothing is queued or running.</summary>
private static async Task WaitForQuietAsync(FileTransferQueue queue)
{
var deadline = TimeSpan.FromSeconds(30);
var waited = TimeSpan.Zero;
while (queue.IsBusy && waited < deadline)
{
await Task.Delay(20, Token);
waited += TimeSpan.FromMilliseconds(20);
}
queue.IsBusy.ShouldBeFalse("the queue should have drained");
}
}
tests/DodoSSH.Client.Transfer.Tests/packages.lock.json
+240
View File
@@ -0,0 +1,240 @@
{
"version": 2,
"dependencies": {
"net10.0": {
"Meziantou.Analyzer": {
"type": "Direct",
"requested": "[3.0.137, )",
"resolved": "3.0.137",
"contentHash": "48n8WsokyefLOZt2qGPpk+hjfF9rQRa7MGK+8hMS6sO22QuIDtBztfbXV7RtVfX7JKjUwDI9e06I8aeCc5XZzQ=="
},
"Microsoft.CodeAnalysis.BannedApiAnalyzers": {
"type": "Direct",
"requested": "[5.6.0, )",
"resolved": "5.6.0",
"contentHash": "Kcobt3pnOdO0A+6CKiMHZdTEluJpsfxiV20axtZdmfBQnDmiWTKPJADlgAfdTuKNAnVarrkJa0UEGwuOo91muw=="
},
"NSubstitute": {
"type": "Direct",
"requested": "[6.0.0, )",
"resolved": "6.0.0",
"contentHash": "0gvKMbiJ+/WrfbcfBfqRZZrvfLJcd3rqkqVMjjlY5dtmLRVzMY+o/K/rJUStofQ2haSr9Vd04YDfvZtVVGS3/A==",
"dependencies": {
"Castle.Core": "5.1.1"
}
},
"Shouldly": {
"type": "Direct",
"requested": "[4.3.0, )",
"resolved": "4.3.0",
"contentHash": "sDetrWXrl6YXZ4HeLsdBoNk3uIa7K+V4uvIJ+cqdRa5DrFxeTED7VkjoxCuU1kJWpUuBDZz2QXFzSxBtVXLwRQ==",
"dependencies": {
"DiffEngine": "11.3.0",
"EmptyFiles": "4.4.0"
}
},
"xunit.v3": {
"type": "Direct",
"requested": "[3.2.2, )",
"resolved": "3.2.2",
"contentHash": "L+4/4y0Uqcg8/d6hfnxhnwh4j9FaeULvefTwrk30rr1o4n/vdPfyUQ8k0yzH8VJx7bmFEkDdcRfbtbjEHlaYcA==",
"dependencies": {
"xunit.v3.mtp-v1": "[3.2.2]"
}
},
"Castle.Core": {
"type": "Transitive",
"resolved": "5.1.1",
"contentHash": "rpYtIczkzGpf+EkZgDr9CClTdemhsrwA/W5hMoPjLkRFnXzH44zDLoovXeKtmxb1ykXK9aJVODSpiJml8CTw2g==",
"dependencies": {
"System.Diagnostics.EventLog": "6.0.0"
}
},
"DiffEngine": {
"type": "Transitive",
"resolved": "11.3.0",
"contentHash": "k0ZgZqd09jLZQjR8FyQbSQE86Q7QZnjEzq1LPHtj1R2AoWO8sjV5x+jlSisL7NZAbUOI4y+7Bog8gkr9WIRBGw==",
"dependencies": {
"EmptyFiles": "4.4.0",
"System.Management": "6.0.1"
}
},
"EmptyFiles": {
"type": "Transitive",
"resolved": "4.4.0",
"contentHash": "gwJEfIGS7FhykvtZoscwXj/XwW+mJY6UbAZk+qtLKFUGWC95kfKXnj8VkxsZQnWBxJemM/q664rGLN5nf+OHZw=="
},
"Microsoft.ApplicationInsights": {
"type": "Transitive",
"resolved": "2.23.0",
"contentHash": "nWArUZTdU7iqZLycLKWe0TDms48KKGE6pONH2terYNa8REXiqixrMOkf1sk5DHGMaUTqONU2YkS4SAXBhLStgw=="
},
"Microsoft.Bcl.AsyncInterfaces": {
"type": "Transitive",
"resolved": "6.0.0",
"contentHash": "UcSjPsst+DfAdJGVDsu346FX0ci0ah+lw3WRtn18NUwEqRt70HaOQ7lI72vy3+1LxtqI3T5GWwV39rQSrCzAeg=="
},
"Microsoft.Extensions.DependencyInjection.Abstractions": {
"type": "Transitive",
"resolved": "8.0.2",
"contentHash": "3iE7UF7MQkCv1cxzCahz+Y/guQbTqieyxyaWKhrRO91itI9cOKO76OHeQDahqG4MmW5umr3CcCvGmK92lWNlbg=="
},
"Microsoft.Extensions.Logging.Abstractions": {
"type": "Transitive",
"resolved": "8.0.3",
"contentHash": "dL0QGToTxggRLMYY4ZYX5AMwBb+byQBd/5dMiZE07Nv73o6I5Are3C7eQTh7K2+A4ct0PVISSr7TZANbiNb2yQ==",
"dependencies": {
"Microsoft.Extensions.DependencyInjection.Abstractions": "8.0.2"
}
},
"Microsoft.Testing.Extensions.Telemetry": {
"type": "Transitive",
"resolved": "1.9.1",
"contentHash": "No5AudZMmSb+uNXjlgL2y3/stHD2IT4uxqc5yHwkE+/nNux9jbKcaJMvcp9SwgP4DVD8L9/P3OUz8mmmcvEIdQ==",
"dependencies": {
"Microsoft.ApplicationInsights": "2.23.0",
"Microsoft.Testing.Platform": "1.9.1"
}
},
"Microsoft.Testing.Extensions.TrxReport.Abstractions": {
"type": "Transitive",
"resolved": "1.9.1",
"contentHash": "AL46Xe1WBi85Ntd4mNPvat5ZSsZ2uejiVqoKCypr8J3wK0elA5xJ3AN4G/Q4GIwzUFnggZoH/DBjnr9J18IO/g==",
"dependencies": {
"Microsoft.Testing.Platform": "1.9.1"
}
},
"Microsoft.Testing.Platform": {
"type": "Transitive",
"resolved": "1.9.1",
"contentHash": "QafNtNSmEI0zazdebnsIkDKmFtTSpmx/5PLOjURWwozcPb3tvRxzosQSL8xwYNM1iPhhKiBksXZyRSE2COisrA=="
},
"Microsoft.Testing.Platform.MSBuild": {
"type": "Transitive",
"resolved": "1.9.1",
"contentHash": "oTUtyR4X/s9ytuiNA29FGsNCCH0rNmY5Wdm14NCKLjTM1cT9edVSlA+rGS/mVmusPqcP0l/x9qOnMXg16v87RQ==",
"dependencies": {
"Microsoft.Testing.Platform": "1.9.1"
}
},
"Microsoft.Win32.Registry": {
"type": "Transitive",
"resolved": "5.0.0",
"contentHash": "dDoKi0PnDz31yAyETfRntsLArTlVAVzUzCIvvEDsDsucrl33Dl8pIJG06ePTJTI3tGpeyHS9Cq7Foc/s4EeKcg=="
},
"System.CodeDom": {
"type": "Transitive",
"resolved": "6.0.0",
"contentHash": "CPc6tWO1LAer3IzfZufDBRL+UZQcj5uS207NHALQzP84Vp/z6wF0Aa0YZImOQY8iStY0A2zI/e3ihKNPfUm8XA=="
},
"System.Diagnostics.EventLog": {
"type": "Transitive",
"resolved": "6.0.0",
"contentHash": "lcyUiXTsETK2ALsZrX+nWuHSIQeazhqPphLfaRxzdGaG93+0kELqpgEHtwWOlQe7+jSFnKwaCAgL4kjeZCQJnw=="
},
"System.Management": {
"type": "Transitive",
"resolved": "6.0.1",
"contentHash": "10J1D0h/lioojphfJ4Fuh5ZUThT/xOVHdV9roGBittKKNP2PMjrvibEdbVTGZcPra1399Ja3tqIJLyQrc5Wmhg==",
"dependencies": {
"System.CodeDom": "6.0.0"
}
},
"xunit.analyzers": {
"type": "Transitive",
"resolved": "1.27.0",
"contentHash": "y/pxIQaLvk/kxAoDkZW9GnHLCEqzwl5TW0vtX3pweyQpjizB9y3DXhb9pkw2dGeUqhLjsxvvJM1k89JowU6z3g=="
},
"xunit.v3.assert": {
"type": "Transitive",
"resolved": "3.2.2",
"contentHash": "BPciBghgEEaJN/JG00QfCYDfEfnLgQhfnYEy+j1izoeHVNYd5+3Wm8GJ6JgYysOhpBPYGE+sbf75JtrRc7jrdA=="
},
"xunit.v3.common": {
"type": "Transitive",
"resolved": "3.2.2",
"contentHash": "Hj775PEH6GTbbg0wfKRvG2hNspDCvTH9irXhH4qIWgdrOSV1sQlqPie+DOvFeigsFg2fxSM3ZAaaCDQs+KreFA==",
"dependencies": {
"Microsoft.Bcl.AsyncInterfaces": "6.0.0"
}
},
"xunit.v3.core.mtp-v1": {
"type": "Transitive",
"resolved": "3.2.2",
"contentHash": "Ga5aA2Ca9ktz+5k3g5ukzwfexwoqwDUpV6z7atSEUvqtd6JuybU1XopHqg1oFd78QdTfZgZE9h5sHpO4qYIi5w==",
"dependencies": {
"Microsoft.Testing.Extensions.Telemetry": "1.9.1",
"Microsoft.Testing.Extensions.TrxReport.Abstractions": "1.9.1",
"Microsoft.Testing.Platform": "1.9.1",
"Microsoft.Testing.Platform.MSBuild": "1.9.1",
"xunit.v3.extensibility.core": "[3.2.2]",
"xunit.v3.runner.inproc.console": "[3.2.2]"
}
},
"xunit.v3.extensibility.core": {
"type": "Transitive",
"resolved": "3.2.2",
"contentHash": "srY8z/oMPvh/t8axtO2DwrHajhFMH7tnqKildvYrVQIfICi8fOn3yIBWkVPAcrKmHMwvXRJ/XsQM3VMR6DOYfQ==",
"dependencies": {
"xunit.v3.common": "[3.2.2]"
}
},
"xunit.v3.mtp-v1": {
"type": "Transitive",
"resolved": "3.2.2",
"contentHash": "O41aAzYKBT5PWqATa1oEWVNCyEUypFQ4va6K0kz37dduV3EKzXNMaV2UnEhufzU4Cce1I33gg0oldS8tGL5I0A==",
"dependencies": {
"xunit.analyzers": "1.27.0",
"xunit.v3.assert": "[3.2.2]",
"xunit.v3.core.mtp-v1": "[3.2.2]"
}
},
"xunit.v3.runner.common": {
"type": "Transitive",
"resolved": "3.2.2",
"contentHash": "/hkHkQCzGrugelOAehprm7RIWdsUFVmIVaD6jDH/8DNGCymTlKKPTbGokD5czbAfqfex47mBP0sb0zbHYwrO/g==",
"dependencies": {
"Microsoft.Win32.Registry": "[5.0.0]",
"xunit.v3.common": "[3.2.2]"
}
},
"xunit.v3.runner.inproc.console": {
"type": "Transitive",
"resolved": "3.2.2",
"contentHash": "ulWOdSvCk+bPXijJZ73bth9NyoOHsAs1ZOvamYbCkD4DNLX/Bd29Ve2ZNUwBbK0MqfIYWXHZViy/HKrdEC/izw==",
"dependencies": {
"xunit.v3.extensibility.core": "[3.2.2]",
"xunit.v3.runner.common": "[3.2.2]"
}
},
"dodossh.client.ssh": {
"type": "Project",
"dependencies": {
"SSH.NET": "[2025.1.0, )"
}
},
"dodossh.client.transfer": {
"type": "Project",
"dependencies": {
"DodoSSH.Client.Ssh": "[1.0.0, )"
}
},
"BouncyCastle.Cryptography": {
"type": "CentralTransitive",
"requested": "[2.6.2, )",
"resolved": "2.6.2",
"contentHash": "7oWOcvnntmMKNzDLsdxAYqApt+AjpRpP2CShjMfIa3umZ42UQMvH0tl1qAliYPNYO6vTdcGMqnRrCPmsfzTI1w=="
},
"SSH.NET": {
"type": "CentralTransitive",
"requested": "[2025.1.0, )",
"resolved": "2025.1.0",
"contentHash": "jrnbtf0ItVaXAe6jE8X/kSLa6uC+0C+7W1vepcnRQB/rD88qy4IxG7Lf1FIbWmkoc4iVXv0pKrz+Wc6J4ngmHw==",
"dependencies": {
"BouncyCastle.Cryptography": "2.6.2",
"Microsoft.Extensions.Logging.Abstractions": "8.0.3"
}
}
}
}
}