Operate and Recover SSH on the Raspberry Pi 5
This runbook explicitly assumes Starlink’s default IPv4 CGNAT, a Netgate 2100 LAN4 DMZ at 192.168.60.0/24, and a Raspberry Pi 5 at 192.168.60.10. Nginx listens only on loopback, cloudflared serves public web traffic only, and Tailscale is the private administration path. There is no WAN SSH forward and no SSH through the Cloudflare public web hostname.
Record the Pi’s operating contract
- LAN4 gateway is
192.168.60.1; the Pi is192.168.60.10/24. Netgate LAN4 rules describe packets entering LAN4 from this DMZ client and are stateful egress rules, not WAN inbound rules. - Nginx is bound to
127.0.0.1; cloudflared’s approved tunnel maps the public web hostname to that local origin. Neither service carries SSH. - Tailscale policy grants named administrators TCP/22 to this Pi. The Pi host firewall permits TCP/22 on
tailscale0, with optional UDP/41641 direct transport only after review. - A trusted-LAN source or local console is the break-glass path. It is not a broad trusted-LAN allow and is tested whenever the SSH policy changes.
cat /etc/os-release
uname -m
uname -r
ip -br address
ip route
ss -lntp
systemctl is-active ssh 2>/dev/null || systemctl is-active sshd
tailscale status
Use commands that exist on the installed distribution; Debian and Raspberry Pi OS commonly call the service ssh, while other systems use sshd. Confirm the exact package and service before applying a change.
Run SSH with a narrow policy
Use an included drop-in so package upgrades and local policy remain reviewable. The final values must match the account recovery design:
# /etc/ssh/sshd_config.d/30-pi-private-admin.conf
PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin no
X11Forwarding no
AllowUsers pi-admin
Replace pi-admin with the actual dedicated administrator account, or use AllowGroups where local group management is the reviewed control. Do not disable a required console recovery account without testing its alternative. Use a short-lived certificate or rotated Ed25519 key, verify the host fingerprint out of band, and keep the private key off the Pi.
sudo sshd -t
sudo sshd -T | grep -E '^(allowusers|allowgroups|passwordauthentication|kbdinteractiveauthentication|permitrootlogin|pubkeyauthentication|x11forwarding|trustedusercakeys|revokedkeys)'
sudo systemctl reload ssh || sudo systemctl reload sshd
Reload, rather than restart, preserves current sessions while the new connection is tested. Never close the known-good session until a new Tailscale session and the trusted-LAN recovery test are complete.
Maintain the Tailscale and firewall boundary
In the Pi’s existing nftables inet input chain, preserve state and add only:
ct state established,related accept
iifname "tailscale0" tcp dport 22 accept
If direct peer transport is desired and the LAN4-facing interface is confirmed as eth0, the optional separate rule is:
iifname "eth0" udp dport 41641 accept
UDP/41641 is encrypted Tailscale WireGuard transport, not SSH and not an authorization grant. Starlink CGNAT may force an encrypted DERP relay over TCP/443. Permit the documented outbound Tailscale control, STUN, and relay behavior through Netgate LAN4 egress as required, but do not add a WAN pass or port forward. Keep the grants or ACL policy limited to named administrators, this Pi, and TCP/22.
Test both paths:
tailscale netcheck
tailscale ping pi-dmz
ssh -o ConnectTimeout=10 pi-admin@<tailnet-address-or-name>
A relay is an expected connectivity result, not a reason to allow public SSH. Test that a non-member tailnet device, an unapproved trusted-LAN host, the Netgate WebConfigurator, the loopback origin from outside, and every unlisted port remain inaccessible.
Logs, health, and privacy
Review only the time range needed for the incident. SSH logs contain account names, source addresses, certificate IDs, and authentication outcomes; restrict access and retention, protect forwarding, and redact copies:
sudo journalctl -u ssh --since "1 hour ago" --no-pager
sudo journalctl _COMM=sshd --since "1 hour ago" --no-pager
sudo journalctl -u tailscaled --since "1 hour ago" --no-pager
sudo systemctl status ssh --no-pager 2>/dev/null || sudo systemctl status sshd --no-pager
df -h
timedatectl status
Check CPU, memory, disk, power, temperature, and clock health using the Pi OS tools available on the installed release. A full filesystem can prevent logins and updates; do not delete logs blindly. Preserve evidence, follow the retention policy, and rotate or archive through the configured log service.
Recovery procedure
- Stop making network changes. Keep any existing session open, connect a local display/keyboard or serial-capable console, and record the last known-good SSH, nftables, Tailscale, and Netgate changes.
- Check link, address, route, time, disk, service state, and listening sockets. Confirm that
192.168.60.10is not duplicated and that the cable is really on LAN4. - Validate configuration with
sshd -t. Restore the reviewed drop-in or nftables policy from backup if a syntax or authorization change caused the outage. Do not enable password login or create a WAN rule. - Reload the SSH service through the console, then test the trusted-LAN break-glass host. Next test Tailscale enrollment, grant/ACL,
tailscale0rule, and a fresh key or certificate. - If a key, certificate, auth key, or device was exposed, revoke it, rotate it, remove the old device, inspect successful-login logs, and record the event. Revoke OpenSSH certificate serials with the KRL process rather than deleting unrelated keys.
For a CA or KRL emergency, use Use OpenSSH Certificates and Key Revocation. For diagnostic evidence and rate limits, use Log Audit Troubleshoot and Recover SSH.
Backups, rotation, and test matrix
Keep encrypted, versioned backups of the Pi’s SSH drop-ins, authorized keys or principals, CA public material, KRL, host keys, nftables rules, Tailscale policy, service unit overrides, Netgate export, cloudflared configuration, and recovery notes. Do not put private keys or tunnel tokens in an ordinary Pi image. Test a restore onto spare storage or a disposable host; verify file ownership and permissions after restore.
| Test | Expected result | Record |
|---|---|---|
| Normal Tailscale administrator | Key or certificate SSH to Pi succeeds on TCP/22. | Tailnet policy, host rule, sshd log. |
| Expired, revoked, wrong-principal, or old key | SSH fails; existing sessions are terminated when required. | Certificate/KRL inspection and denial log. |
| Trusted-LAN break glass | Named source succeeds when Tailscale is stopped; another LAN host fails. | Source, firewall counter, and recovery timestamp. |
| Direct Tailscale transport | Optional UDP/41641 direct path works if available. | tailscale netcheck and Netgate counter. |
| DERP fallback | Blocking direct UDP still permits encrypted relay administration. | tailscale ping, latency, and no policy broadening. |
| Public boundary | Public web hostname reaches web only; no SSH via hostname or WAN address. | External test and Cloudflare/Netgate evidence. |
| DMZ isolation | Pi cannot reach trusted, management, storage, backup, or Netgate control-plane destinations. | LAN4 block counters and logs. |
Rotate user keys before expiry and after any loss; rotate certificates frequently with short validity; review Tailscale device and policy membership; and periodically change the break-glass key through a tested overlap. Rollback by restoring the last known-good files and Netgate/Tailscale policies, validating syntax, reloading through console or an open session, and repeating the full matrix. Remove temporary debug logging and transport exceptions after validation.
Continue the SSH sequence
For the preceding private-overlay procedure, return to Use Tailscale and OpenSSH for Private Pi Administration (SSH order 15). For the security-first reverse-tunnel progression, continue with Understand Reverse SSH Tunnels and Their Risks (SSH order 17). The links below are related Homelab references, not alternate SSH entry points.
Related Homelab references
- Reference Architecture: Starlink, Netgate 2100, and Raspberry Pi 5
- Test the Complete Homelab from Outside
Official references
- Raspberry Pi OS configuration documentation source ↗ (the rendered Raspberry Pi documentation site may challenge automated requests)
- Debian OpenSSH sshd_config manual ↗
- Tailscale firewalls and NAT traversal ↗
- Netgate 2100 switch ports ↗
dispelled