Master Linux SSH Troubleshooting with Simple Steps

SSH troubleshooting is one of the most important skills every Linux administrator needs. A simple connection attempt can fail because of authentication issues, incorrect permissions, network problems, host key mismatches, firewall rules, or server-side configuration errors. The challenge is that many SSH errors look similar at first glance, even though the root causes can be completely different.

Knowing how to troubleshoot SSH methodically is an essential Linux administration skill. Instead of guessing or repeatedly changing settings, experienced administrators follow a structured process to identify where the connection is failing and why. Understanding what each error message actually means can save hours of troubleshooting and help avoid changes that create bigger problems.

This guide walks through the most common SSH connection errors encountered on Ubuntu, Rocky Linux, RHEL, and other Linux distributions. You will learn how to diagnose authentication failures, permission problems, host key warnings, network connectivity issues, and server-side SSH configuration mistakes using practical examples and proven troubleshooting techniques.

Prerequisites :

Operating System         :   Ubuntu 22.04/24.04, Rocky Linux 8/9, RHEL 8/9, Debian 11/12
Packages and Dependencies:   openssh-client (client side), openssh-server (server side)
User Account             :   root or a user account with sudo privileges
Recommended to run diagnostic and config commands with sudo instead of logging in as root directly.

Difficulties setting up a sudo user on your server? Click here to find the steps before you go further.

If you are still getting familiar with basic SSH usage, our SSH client commands guide is worth a quick read before diving into the flowchart below.

My Lab Setup :

SSH Client and Server (used for this guide):

Client Operating System  :   Ubuntu 24.04 LTS
Server Operating System  :   Rocky Linux 9.4
Server Hostname           :   sshs02.linuxteck
Server IP Address         :   192.168.1.150
SSH Port (custom)         :   22 (default, noted where changed)

Step 1: Read the Exact Error Before Doing Anything Else

Run the connection with verbose mode on so you get real diagnostic detail instead of a one line error.

Step 2: Fix Connection Refused

First, confirm sshd is actually running on the server. You will need console access or an existing session for this.

If it shows inactive, start it and enable it so it survives a reboot.

Next, check which port sshd is actually bound to. Plenty of people forget the server was hardened to a custom port and keep hammering port 22.

That output shows sshd is on port 2222, not 22. You would need to connect with ssh -p 2222 john@192.168.1.150 for this to work. If you have networking basics that feel shaky, our Linux networking commands guide covers tools like this one in more depth.

On Ubuntu/Debian with ufw, the equivalent is covered in detail in our firewall-cmd commands article, which also explains the difference between firewalld zones if you are managing both distro types.

Step 3: Fix Connection Timeout

Start with basic reachability. If ping fails, stop troubleshooting SSH and start troubleshooting the network or the server itself.

If ping itself times out, that could just mean ICMP is blocked, which is common and not a real problem on its own. Test the SSH port specifically with netcat instead, that is a more accurate test.

That confirms it is the port specifically, not just ICMP. If you are on a cloud VM, this is where most people get stuck, because the operating system firewall is not the only thing in the way. Check your cloud provider's security group or network ACL too, since those silently drop traffic before it ever reaches the instance. If you are running production workloads, this kind of layered firewall check should really be part of your server hardening checklist from day one, not something you discover during an outage.

Step 4: Fix Permission Denied

Run verbose mode again and look specifically at which authentication methods were offered and which keys were tried.

That tells you the client offered a key but the server rejected it. Check whether the matching public key actually exists in authorized_keys on the server.

If your public key is missing from that file, that is your answer. If it is present, the next most common cause is file permissions. SSH is strict about this and will silently ignore keys with permissions it considers unsafe.

If permissions and authorized_keys both check out and you are still denied, check the server config for what auth methods are actually permitted.

If you are trying to log in as root and PermitRootLogin is set to no, that explains the denial right there, and it is intentional hardening, not a bug. Create or use a regular sudo user instead. Our guide on installing and securing an SSH server walks through exactly why root login gets disabled and how to set up a proper sudo account in its place.

Step 5: Fix Host Key Changed Warnings

Here is what the warning looks like in full.

Before removing anything, verify with whoever manages the server whether it was actually rebuilt, reimaged, or had its IP reassigned recently. If the answer is yes, this is expected, and you can safely remove the old key. If nobody touched the server and you are not on a trusted network, stop and investigate further before connecting.

That removes just the one stale entry instead of wiping the entire known_hosts file. Connect again and you will be asked to accept the new key fingerprint, which gets recorded fresh.

Step 6: Fix Wrong Key Type or Passphrase Issues

Check what key types the server actually accepts before assuming your key is wrong.

If that line is missing entirely, newer OpenSSH versions reject old RSA-SHA1 keys by default, an Ed25519 key is the safer bet going forward. Generate a fresh one if needed.

If you are sure the key type is fine and it is a passphrase problem instead, confirm the key is actually loaded in your agent.

An empty agent means it forgot your passphrase protected key after a reboot or logout. Add it back manually.

Step 7: Fix Too Many Authentication Failures

Here is exactly what this looks like when it happens.

Check how many keys your agent is currently holding, this is usually the root cause if you use 1Password, GNOME Keyring, or a similar SSH agent integration.

Four keys being offered against a server with a low MaxAuthTries setting is a recipe for exactly this error. Fix it permanently by pointing this specific host at one identity file only.

Now ssh sshs02 uses only the one key you specified, regardless of how many others sit in your agent.

Step 8: Fix Slow SSH Login Before the Prompt Appears

Confirm it is DNS related by checking the timing difference with verbose output.

If login is fast with GSSAPI off but slow without that flag, you have found your cause. Disable reverse DNS lookups permanently on the server side instead of adding flags every time you connect.

Step 9: Fix SSH Connections That Drop With Broken Pipe

The error usually looks like this right in the middle of a command.

Add keepalive settings on the client side so SSH sends a small signal regularly, which stops routers and firewalls from treating the session as idle.

If you control the server too, set the matching server side option so both ends agree on keeping idle sessions alive.

Step 10: Fix Permission Denied at the Account Level

Check the account status directly on the server, this rules out keys and passwords as the cause entirely.

An expired account explains a denial with no error about keys at all. Extend or remove the expiry as needed.

Also confirm the account has a real login shell, not /sbin/nologin or /bin/false, which exist specifically to block interactive SSH sessions.

That shell setting blocks interactive login entirely regardless of how correct your key is. Fix it with usermod if this account is meant to log in.

Step 11: Fix Algorithm or Cipher Mismatch Errors

The error is usually very explicit about what is missing.

You can confirm which algorithms your client supports for comparison against what the old device is offering.

Notice diffie-hellman-group1-sha1 is not in that list, which is exactly why negotiation failed. If you genuinely need to reach this one legacy device, add the old algorithm back for just that connection instead of weakening your client globally.

Step 12: Fix SCP and SFTP Transfer Failures

Test SFTP directly to see the actual error instead of guessing from a failed scp command.

Check the Subsystem line in sshd_config to confirm the path it points to actually exists on this server.

On Rocky Linux and RHEL the binary actually lives under /usr/libexec/openssh/sftp-server, while the config still points at the older /usr/lib/ssh path from a different distro template. Fix the mismatch directly.

For a full walkthrough of setting this up correctly from scratch on Rocky Linux, see our dedicated SFTP server setup guide.

Step 13: Fix Port Forwarding and Tunneling Issues

Try a basic local forward and see what comes back.

You still get a shell, but the forward itself silently failed. Check whether forwarding is disabled on the server side, this is a common hardening setting that gets left on by accident when it is actually needed.

That setting blocks all forwarding regardless of how correct your ssh -L command is. Enable it deliberately if you need this feature.

Conclusion:

Every SSH error you will run into in real work fits into one of these branches, refused, timeout, denied in its various forms, host key changes, algorithm mismatches, or the transfer and tunneling issues covered above. Once things are stable, lock the server down properly with our SSH server hardening guide, and keep the official OpenSSH client config documentation bookmarked for anything not covered here. Drop me your feedback/comments. Feel free to share this article with others if you like it.