Notice when a shell ends, say which way it ended, and offer the way back

A terminal whose machine rebooted stayed green for ever. Not "went stale and
recovered late" — for ever: the tab claimed CONNECTED, the elapsed timer went
on counting, and the only thing that ever corrected it was closing the
application. That is the defect this fixes, and it turned out to be two
separate failures wearing one symptom.

◆ THE HANG IS IN SSH.NET, AND KEEP-ALIVE ALONE WOULD NOT HAVE FIXED IT.

ShellStream ends a blocked read from exactly two places: its channel being
closed, and the session raising Disconnected. A shell that exits gives the
first. But Session raises Disconnected from one method only — OnDisconnectReceived,
for a server that politely says goodbye — and a socket that simply fails goes
down a different path entirely: RaiseError fires the error event, sets the
disconnecting flag and disposes the socket, without ever raising Disconnected.
So the stream was never disposed, its Monitor.Wait was never pulsed, and the
read blocked until the process ended. The transport was gone, the client knew
the transport was gone, and nothing woke the one thread that was waiting.

That is why the first attempt at a test for this sat at thirty seconds and
failed. Both halves are needed and neither is sufficient:

  - Keep-alive makes the failure HAPPEN. An idle SSH session is a socket
    nobody writes to, and TCP has no way to discover that the far end has gone
    without writing to it — nothing was asked, so nothing could fail. Thirty
    seconds, matching OpenSSH's own ServerAliveInterval examples. It is the
    write that matters and not a reply: SSH_MSG_IGNORE need not be answered,
    so this is traffic rather than a heartbeat with a deadline, and there is
    no timeout invented here to be wrong about a slow link.

  - Disposing the stream from the error handler makes it ARRIVE. Queued to the
    thread pool and not done inline, which is not tidiness: this runs on
    SSH.NET's message-listener thread, and closing a channel waits for the
    server's close — processed by that same thread — so an inline dispose is
    the listener thread waiting on itself.

SFTP gets the keep-alive too. Its operations are request/response and bounded
by ConnectionInfo.Timeout already, so it needs no equivalent wake-up.

◆ exit AND reboot ARE TOLD APART BY A LATCH, BECAUSE THE OBVIOUS WAY IS A RACE.

Wait for the read to end and then ask whether the transport is still up, and
against a real server the answer is wrong about half the time: this was the
only channel on the connection, so sshd hangs up a millisecond after closing
it. By the time anything could ask, the transport is gone either way — and a
poll would report every ordinary `exit` as a dropped connection.

So ISshShellSession.TransportFailure is set at the moment SSH.NET raises the
failure, which its own ordering puts strictly before the disconnect that wakes
the read. A read that has just returned zero consults a settled value: set if
the transport failed, still null if the remote merely closed the channel. The
pump turns that into a TerminalSessionOutcome — RemoteClosed, ConnectionLost,
ClosedLocally — carried on SessionEnded, which used to hand over a session id
and nothing else. A subscriber cannot work this out afterwards; by the time it
hears, there is no transport left to interrogate.

Deliberately no fourth value for "the remote rebooted". Nothing on the wire
distinguishes a reboot from a cable being pulled, and a name this client
cannot check would be a guess dressed as a fact.

◆ WHAT IS DRAWN, AND WHY IT IS NOT DRAWN OVER THE PANE.

The status bar says DISCONNECTED or SHELL EXITED where it used to say NOT
CONNECTED for both — a word that is honest about a tab still dialling and
useless about one whose shell has stopped. Red for the first and grey for the
second: one is a machine that stopped answering and the other is work
finished, and crying wolf on the ordinary end of every session is its own bug.
The tab strip's dot gains the same fourth state, on both heads.

Under the terminal, a strip carrying the reason and two buttons. It takes its
own Auto grid row rather than overlaying the renderer, and that is forced
rather than chosen: the WebView is a native child view composited above
everything Avalonia draws, so the alternative is collapsing the terminal — and
the scrollback is precisely what the user wants to read, because the last
thing the remote said is usually why it went. terminal.js already writes the
same sentence into the pane; this is the copy that does not scroll away.

RECONNECT reuses the tab in place, which is what the word means to somebody
who pressed it: the terminal they were working in comes back, where it was.
It dials through the vault's ordinary connect path, so every refusal — a key
that has left the keychain, an unknown host key, a group default that has
changed since — is decided in the one place that already decides them; what
this adds is which tab the attempt belongs to. The old session is closed as
part of it, because a dead shell still holds an authenticated connection this
process has stopped reading, and its pane still holds an xterm, a scrollback
and a WebGL context, of which a browser hands out about sixteen.

Two refusals it makes rather than pretending: a host reached by typing an
address has no keychain entry to redial, and one that wants a password typed
goes to the keychain with the sheet raised rather than being dialled with an
empty box. Both say so on the strip where the button was pressed, not only in
the vault's status on a screen that surface is not.

TRY AGAIN appears on a refused connection for the same reason, and it is worth
as much: the commonest refusal is a host that was not up yet.

◆ WHAT IS ACTUALLY VERIFIED.

Two tests against a real sshd in a container. `exit` is reported as the remote
closing the session — the one a polling implementation fails, and a fake shell
cannot show it because a fake has no server behind it deciding to hang up. And
a live shell losing its transport is reported as a lost connection, produced
by dialling through a plain TCP relay the test owns and then cutting it; the
container is shared by the collection, so killing sshd was never an option.

The first draft of that second test used LoopbackProxyTests' SOCKS forward and
timed out: disconnecting the carrier of a ForwardedPortDynamic does not close
the loopback socket it accepted. Recorded in the test, because it is a
property of SSH.NET's forwarder rather than of anything here, and it is not
what a dying machine looks like.

What no test reproduces is the waiting. A relay that closes sends a FIN, so
the socket reports itself at once; a machine switched off sends nothing, and
finding that out is the keep-alive's business and the OS's retransmit budget's
— neither of which a test can hurry along without a clock this code does not
own. A reboot is prompt in practice because the machine comes back and RSTs.

Beside those: three pump outcomes, three workspace announcements, and seven
shell-flow tests covering the drop, the clean exit, reuse of the tab, retry
after a refusal, and both ways a reconnect can honestly come to nothing.

Also fixed in passing: SshNetShellSession.IsOpen read ShellStream.CanRead,
which is documented as always true — the stream stays readable after disposal
so buffered output can be drained — so it had answered "yes" for every shell
that ever existed. It reads CanWrite now. And MainWindowViewModel's session-
ended handler moved from Dispatcher.UIThread.Post to the injected post
delegate, which is what makes any of the shell half testable at all; the
remark listing the three unreachable posts is one shorter.

src/DodoSSH.Client.App/App.axaml

@@ -1282,6 +1282,17 @@
       <Setter Property="Fill" Value="{StaticResource Warn}" />
     </Style>
 
+    <!--
+      Red, and it is the fourth state this dot has rather than a recolouring of the grey one. Grey is a
+      shell that ended — somebody typed `exit`, and nothing is wrong with the machine. This is a connection
+      that went out from under a session that had not finished, which is the one worth looking at: the two
+      used to be the same dot, so a tab that had been cut off and a tab somebody had closed down properly
+      were indistinguishable in the strip. See TerminalTabViewModel.EndedSummary.
+    -->
+    <Style Selector="Ellipse.dot.dropped">
+      <Setter Property="Fill" Value="{StaticResource Danger}" />
+    </Style>
+
     <!--
       The accent strip a selected row carries, drawn by the row template rather than by the item, because the
       item's presenter is the thing the theme keeps repainting.

src/DodoSSH.Client.App/Views/ConnectingCard.axaml

@@ -181,6 +181,19 @@
         in the strip and the card is still here on the way back.
       -->
       <StackPanel Orientation="Horizontal" Spacing="10" HorizontalAlignment="Center">
+        <!--
+          TRY AGAIN rather than RECONNECT, because nothing was ever connected: this tab is a handshake that
+          was refused. It runs the same command the disconnect strip's RECONNECT does — the vault's own
+          connect path, into this same tab — and it is worth having here for the reason the commonest
+          refusal is worth retrying at all: a host that was not up yet is a host that will be in a minute.
+
+          Disabled rather than hidden when there is nothing to dial, exactly as the strip's is. See
+          MainWindowViewModel.CanReconnectSelectedTab.
+        -->
+        <Button Classes="ghost" Content="TRY AGAIN" Command="{Binding ReconnectTabCommand}"
+                CommandParameter="{Binding SelectedTab}"
+                IsEnabled="{Binding CanReconnectSelectedTab, FallbackValue=False}"
+                IsVisible="{Binding SelectedTab.IsFailed}" />
         <Button Classes="ghost" Content="SHOW LOGS" Command="{Binding ShowScreenCommand}"
                 CommandParameter="{x:Static vm:ShellScreen.Logs}" />
         <Button Classes="ghost" Content="GIVE UP" Command="{Binding CloseTabCommand}"

src/DodoSSH.Client.App/Views/MainWindow.axaml

@@ -279,7 +279,14 @@
                   this Grid is only drawn on the terminal surface, and the surface with no tab open already
                   answers for itself in the tab row's own "+" and in the connecting card below.
                 -->
-                <Grid Grid.Column="0" RowDefinitions="*,Auto">
+                <!--
+                  Three rows since the disconnect strip arrived, where v5c-4 left two. The extra one is Auto
+                  and empty except while the selected tab's shell has stopped, so it costs the terminal
+                  nothing in the ordinary case — and when it is up it takes its 34 pixels off the pane rather
+                  than drawing over it, which is the only way Avalonia content can sit near the renderer at
+                  all. See SessionEndedBar.axaml.
+                -->
+                <Grid Grid.Column="0" RowDefinitions="*,Auto,Auto">
 
                   <Panel Grid.Row="0" Background="{StaticResource Pane}">
 
@@ -306,7 +313,8 @@
 
                   </Panel>
 
-                  <views:SessionStatusBar Grid.Row="1" ShowsEncoding="True" />
+                  <views:SessionEndedBar Grid.Row="1" />
+                  <views:SessionStatusBar Grid.Row="2" ShowsEncoding="True" />
                 </Grid>
                 <!--
                   Hides when no session is active — see ShowsQuickAccessSidebar — rather than always drawn:

src/DodoSSH.Client.App/Views/SessionEndedBar.axaml

@@ -0,0 +1,97 @@
+<UserControl xmlns="https://github.com/avaloniaui"
+             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
+             xmlns:vm="using:DodoSSH.Client.Shell.ViewModels"
+             x:Class="DodoSSH.Client.App.Views.SessionEndedBar"
+             x:DataType="vm:MainWindowViewModel">
+
+  <!--
+    ── What a terminal says once its shell has stopped ──────────────────────────────────────────────────
+    A strip between the pane and the status bar, shown only while the selected tab has a session that has
+    ended. It names which of the two endings it was, prints the sentence that came with it, and offers the
+    one thing worth offering: dialling the machine again.
+
+    ◆ IT IS A ROW OF ITS OWN RATHER THAN SOMETHING DRAWN OVER THE PANE, and that is not a style choice.
+    The renderer is a native child view that composites above everything Avalonia draws, so a banner
+    overlapping its rectangle would either be invisible or would have to collapse the terminal — and
+    collapsing it is exactly wrong here, because the scrollback is what the user is reading. A dead shell's
+    last words are usually why it died. So this takes its own Auto row in the grid the WebView's Panel sits
+    in, and the pane simply gets 34 fewer pixels while it is up. See MainWindow.axaml.
+
+    ◆ THE SENTENCE IS HERE AS WELL AS IN THE PANE ON PURPOSE. terminal.js writes the same string into the
+    scrollback as a grey rule, which is right and is not enough: a machine that printed a page of shutdown
+    messages on its way down has pushed that line off the screen, and a session that ended while the user
+    was on another tab was never on screen at all. This copy does not scroll.
+
+    ── the two endings, and why they are not one colour ─────────────────────────────────────────────────
+    SHELL EXITED is grey. Somebody typed `exit`, or a login shell ended; nothing is wrong and an alarm
+    would be noise. DISCONNECTED is the palette's red, because the connection went out from under a session
+    that had not finished — a reboot, a network drop, an sshd that was killed — and it is the one of the two
+    that means the machine may still be there and worth dialling. The dot beside the word says the same
+    thing a second way, for anybody who cannot tell this design's red from its grey.
+
+    ── the buttons ──────────────────────────────────────────────────────────────────────────────────────
+    RECONNECT is disabled rather than hidden when there is nothing to dial — a tab typed in by hand, or one
+    whose host has left the keychain — because the answer to "why can I not reconnect this" is better given
+    by a control that is visibly refusing than by one that is not there. See
+    MainWindowViewModel.CanReconnectSelectedTab, which is what draws that line, and ReconnectTabAsync for
+    what the button actually does: the vault's own connect path, into this same tab.
+
+    CLOSE TAB is the same command the tab's cross runs. It is here because this is the moment the other
+    reasonable answer is "I have read it, take it away", and hunting for a 16-pixel cross in the strip above
+    is not the shape of that answer.
+  -->
+
+  <UserControl.Styles>
+
+    <Style Selector="TextBlock.endword">
+      <Setter Property="FontFamily" Value="Montserrat" />
+      <Setter Property="FontWeight" Value="SemiBold" />
+      <Setter Property="FontSize" Value="9.5" />
+      <Setter Property="LetterSpacing" Value="1.2" />
+      <Setter Property="VerticalAlignment" Value="Center" />
+      <Setter Property="Foreground" Value="{StaticResource TextFaint}" />
+    </Style>
+    <Style Selector="TextBlock.endword.dropped">
+      <Setter Property="Foreground" Value="{StaticResource DangerText}" />
+    </Style>
+
+  </UserControl.Styles>
+
+  <Border Height="34" Background="{StaticResource DeepChrome}"
+          BorderBrush="{StaticResource Border}" BorderThickness="0,1,0,0"
+          IsVisible="{Binding SessionEndedText, Converter={x:Static StringConverters.IsNotNullOrEmpty}, FallbackValue=False}">
+    <Grid ColumnDefinitions="Auto,*,Auto" Margin="24,0">
+
+      <StackPanel Grid.Column="0" Orientation="Horizontal" Spacing="7" VerticalAlignment="Center">
+        <Ellipse Classes="dot" Width="7" Height="7" Classes.dropped="{Binding IsSessionDropped}"
+                 VerticalAlignment="Center" />
+        <TextBlock Classes="endword" Classes.dropped="{Binding IsSessionDropped}"
+                   Text="{Binding SessionEndedText}" />
+      </StackPanel>
+
+      <!--
+        Trimmed rather than wrapped: this row is one line tall and the sentences it draws are short, but a
+        transport error can hand over whatever the operating system called it, and a bar that grew to fit
+        one would take the height out of the terminal above it.
+      -->
+      <SelectableTextBlock Grid.Column="1" Classes="mono" FontSize="11.5" Margin="16,0"
+                           Foreground="{StaticResource TextGhost}" VerticalAlignment="Center"
+                           TextTrimming="CharacterEllipsis"
+                           Text="{Binding SessionEndedDetail}"
+                           ToolTip.Tip="{Binding SessionEndedDetail}" />
+
+      <StackPanel Grid.Column="2" Orientation="Horizontal" Spacing="10" VerticalAlignment="Center">
+        <Button Classes="ghost" Content="RECONNECT"
+                Command="{Binding ReconnectTabCommand}"
+                CommandParameter="{Binding SelectedTab}"
+                IsEnabled="{Binding CanReconnectSelectedTab, FallbackValue=False}"
+                ToolTip.Tip="Opens a new shell on this machine, in this tab. The scrollback above it goes." />
+        <Button Classes="ghost" Content="CLOSE TAB"
+                Command="{Binding CloseTabCommand}"
+                CommandParameter="{Binding SelectedTab}" />
+      </StackPanel>
+
+    </Grid>
+  </Border>
+
+</UserControl>

src/DodoSSH.Client.App/Views/SessionEndedBar.axaml.cs

@@ -0,0 +1,14 @@
+using Avalonia.Controls;
+
+namespace DodoSSH.Client.App.Views;
+
+/// <summary>What the terminal surface says once the selected tab's shell has stopped.</summary>
+/// <remarks>
+/// No code of its own: the word, the sentence and both buttons are the shell's. In its own file for the
+/// reason every other card here is — nothing inside <c>MainWindow</c> can be laid out by a test, because
+/// WebView2's adapter refuses the headless session's thread. See the markup.
+/// </remarks>
+internal sealed partial class SessionEndedBar : UserControl
+{
+    public SessionEndedBar() => InitializeComponent();
+}

src/DodoSSH.Client.App/Views/SessionStatusBar.axaml

@@ -42,14 +42,29 @@
         <StackPanel Orientation="Horizontal" Spacing="7" VerticalAlignment="Center"
                     IsVisible="{Binding SessionAddress, Converter={x:Static StringConverters.IsNotNullOrEmpty}}">
           <Ellipse Classes="dot" Width="7" Height="7" Classes.live="{Binding IsSessionConnected}"
+                   Classes.dropped="{Binding IsSessionDropped}"
                    VerticalAlignment="Center" />
 
           <TextBlock FontFamily="Montserrat" FontWeight="SemiBold" FontSize="9.5" LetterSpacing="1.2"
                      VerticalAlignment="Center" Text="CONNECTED" Foreground="{StaticResource Live}"
                      IsVisible="{Binding IsSessionConnected}" />
+
+          <!--
+            Three words where there were two, and the third replaces rather than joins the second — see
+            MainWindowViewModel.IsSessionIdle, which is what keeps them from ever being drawn together.
+            NOT CONNECTED is honest about a tab still dialling and useless about one whose shell has
+            stopped: it says nothing about which of the two endings it was, and the difference between them
+            is the difference between work that is finished and a machine that stopped answering. The strip
+            below carries the sentence and the way back; this is the same fact in the one line that is
+            always on screen.
+          -->
           <TextBlock FontFamily="Montserrat" FontWeight="SemiBold" FontSize="9.5" LetterSpacing="1.2"
                      VerticalAlignment="Center" Text="NOT CONNECTED" Foreground="{StaticResource TextFaint}"
-                     IsVisible="{Binding !IsSessionConnected}" />
+                     IsVisible="{Binding IsSessionIdle}" />
+          <TextBlock FontFamily="Montserrat" FontWeight="SemiBold" FontSize="9.5" LetterSpacing="1.2"
+                     VerticalAlignment="Center" Text="{Binding SessionEndedText}"
+                     Foreground="{StaticResource TextFaint}"
+                     IsVisible="{Binding SessionEndedText, Converter={x:Static StringConverters.IsNotNullOrEmpty}}" />
         </StackPanel>
 
         <TextBlock Classes="mono" FontSize="11.5" Foreground="{StaticResource TextGhost}"

src/DodoSSH.Client.App/Views/SessionTabRow.axaml

@@ -74,9 +74,17 @@
                     Worth the third colour because the two it replaces were the same one: a tab still
                     dialling and a tab whose shell has exited both drew grey, which are the two states in
                     this strip with the least in common — one is worth waiting for and the other is over.
+
+                    ── and a fourth since, splitting that grey ────────────────────────────────────────────
+                    A shell that ended and a connection that broke were still one dot, and they are the two
+                    dead states with the least in common: one is work finished, the other is a machine that
+                    stopped answering and may well still be there. Red for the second, matching the strip
+                    under the terminal — see TerminalTabViewModel.IsDropped and SessionEndedBar.axaml, which
+                    is where the sentence and the way back live.
                   -->
                   <Ellipse Classes="dot" Width="8" Height="8" Classes.live="{Binding IsLive}"
                            Classes.connecting="{Binding IsConnecting}"
+                           Classes.dropped="{Binding IsDropped}"
                            VerticalAlignment="Center" />
                   <TextBlock Text="{Binding Label}" VerticalAlignment="Center" />