If you manage multiple Linux servers, using SSH public key authentication can make your life a whole lot easier. No more typing passwords or running scripts that break whenever credentials change. Just seamless, secure SSH logins.
But it does take some initial work to set up SSH key-based auth properly. There are keys to generate, configurations to update, and testing to be done.
In this comprehensive guide, I‘ll walk you step-by-step through the entire process of exchanging SSH keys between two Linux servers. I‘ll share tips and tricks I‘ve learned from nearly a decade of being a Linux system administrator.
Here‘s what I‘ll cover:
- What exactly SSH key authentication is and why it‘s more secure than passwords
- How to generate brand new SSH key pairs on your client machine
- How to securely copy public keys to your remote Linux servers
- Step-by-step instructions to configure SSH for key-based auth
- How to test that your keys work properly and troubleshoot errors
- Extra security precautions you can take like using a passphrase
I‘ll also explain the core components of SSH key-based authentication works under the hood.
My goal is to provide you a deep understanding of how to exchange SSH keys, not just copy/paste commands.
So if you‘re ready to heighten your server security, avoid password prompts, and be more efficient managing Linux machines, let‘s get started!
How SSH Key-Based Authentication Works
SSH utilizes public key cryptography to verify your identity and securely log you into remote servers. This process happens without ever sending your private key over the internet.
Here‘s how it works at a high level:
You generate a keypair consisting of a private key (stored locally) and a public key (copied to servers).
When attempting to SSH into a server, your client sends your public key to the server.
The server checks if your public key is in the
authorized_keysfile for that user account.
If found, the server generates a random number and uses your public key to encrypt it.
This encrypted number gets sent back to the client along with the server‘s public key.
Your local SSH client decrypts this number using your private key.
If decryption succeeds, this proves you hold the associated private key, and SSH access is granted!
This handshake verifies that you own the private key without ever sending it over the internet. Pretty clever!
Now let‘s look at the benefits you get from SSH public key authentication…
Why SSH Keys are More Secure Than Passwords
Passwords alone for SSH logins have some weaknesses and risks:
- Users often create weak passwords that are easy to guess or crack with brute force.
- People tend to reuse passwords across systems, so one compromised system can expose many.
- Passwords can be phished or spied on by malware if entered on a compromised machine.
- Automating SSH logins and file transfers require plaintext passwords in scripts.
SSH key pairs overcome many of these issues:
- Private keys have 2048+ bits of entropy making them near impossible to crack.
- Keypairs are unique to each client and server, avoiding password reuse.
- Private keys never get transmitted or stored anywhere but your local machine.
- Automation with keys is seamless as there‘s no password prompt.
With SSH keys, you can enforce very strong passphrases on private keys for added protection. Or you can securely store unlocked keys on local machines.
According to a recent Venafi study, [] organizations using SSH keys reduced authentication attacks by 90% compared to those using only passwords.
So moving beyond just passwords to use SSH key pairs is one of the best ways to strengthen your Linux server security.
Next, let‘s dive into generating brand new SSH keys…
Step 1 – Generate New SSH Key Pairs
The first step is to generate a public and private SSH keypair that will be used for authentication.
You generate keys on your local computer which will be the SSH client accessing your remote Linux servers. For example your own Linux/Mac workstation or a CI/CD pipeline node.
ssh-keygen command handles all of the key generation steps for us.
On your local machine, open a terminal and run:
ssh-keygen -t rsa
- Create a 2048 bit RSA keypair (very secure).
- Save the private and public key files into
When prompted for a passphrase, you can either:
A. Enter a secure passphrase to add an extra layer of security. You‘ll need to enter this each time you use the key.
B. Or just press
Enter to create an unlocked key for fully automated logins.
I suggest starting with an unlocked key for testing, and later enforcing a strong passphrase before moving the key to production use.
ssh-keygen will output the fingerprint and a randomart image representing your new keypair:
Your identification has been saved in /home/john/.ssh/id_rsa Your public key has been saved in /home/john/.ssh/id_rsa.pub The key fingerprint is: SHA256:YjtctIjT8eRGiLjr1Vn8hnMGrdmQgzvjjBC1+ix3f3E john@client The key‘s randomart image is: +---[RSA 2048]----+ | ooo+. | | =o=o | | . =O o | | + B = . | | S = = o | | o + .. | | . o .. | | ..o ..| | ..o.| +----[SHA256]-----+
The randomart gives you a unique visual fingerprint that can help identify keys.
Your new SSH keypair is now ready to start using!
Next let‘s look at securely copying the public key to your remote Linux servers.
Step 2 – Copy Public Key to Remote Servers
Now that you‘ve created an SSH keypair, you need to distribute the public key to any Linux server you want to log into using key-based authentication.
The public key needs to be copied into a special
authorized_keys file on the remote servers. This file resides within
.ssh directory of the user account you‘ll be logging into.
For example, to SSH into remote servers as the user
john, you‘ll copy your public key into
Here are the detailed steps to copy your public key securely:
Log into your remote Linux server as the user you intend to eventually SSH into. If you want to access the server as user
john, log in as
johnnow using password authentication.
Navigate to the user‘s
.sshdirectory. If it doesn‘t exist yet, create it:
cd ~ mkdir .ssh
catand output redirection to append your public key into the user‘s
cat ~/path/to/id_rsa.pub >> ~/.ssh/authorized_keys
This will add your public key to the end of
authorized_keysif it already exists. Or create a new file if not.
Set the most restrictive permissions possible on both the
chmod 700 ~/.ssh chmod 600 ~/.ssh/authorized_keys
This prevents unauthorized users from accessing private keys if they get access to the server.
You‘ll need to repeat this process for each remote server and user account you wish to enable key authentication for.
The same public key can be used across all servers and accounts. But I suggest creating a unique keypair per server or per user for compartmentalization.
Next I‘ll show you how to configure the SSH client and servers to actually use your new keys…
Step 3 – Configure SSH for Public Key Authentication
To ensure SSH connections only use your public/private keypair for authentication, you need to disable password authentication on both the client and server sides.
This prevents any fallback to insecure password logins.
A. Configure Your SSH Client
On your local SSH client machine, edit the SSH client configuration file located at
Add or modify the following lines:
Host * PubkeyAuthentication yes PasswordAuthentication no
This disables password auth and enables public key authentication globally for all SSH connections from your client machine.
B. Configure Your SSH Servers
On each of your remote Linux servers, edit the SSH daemon configuration file located at
Set the following values:
PubkeyAuthentication yes PasswordAuthentication no
Then reload the sshd service:
sudo systemctl reload sshd
Your Linux servers will now reject all password-based login attempts for that user. Only public key authentication will be allowed.
This provides an enormous security benefit against brute force password attacks.
Step 4 – Test SSH Key Authentication
With all keys and configurations in place, let‘s test that key-based authentication is working properly.
From your SSH client machine, connect to one of your remote servers as the user you configured:
If your keypair is set up correctly, you should log in immediately without any password prompt.
The first time you connect, you may see a warning like:
The authenticity of host ‘192.168.1.5 (192.168.1.5)‘ can‘t be established. RSA key fingerprint is SHA256:401kInIFOxmgFx31BjJHJ6kXYEmnkfXZceM7VRNFkWM. Are you sure you want to continue connecting (yes/no)?
This is because your client is connecting to that server‘s public key for the first time. Type
yes to continue, and you won‘t see the message on subsequent connections.
After logging in, feel free to run some commands or checks to verify everything is working:
ssh [email protected] Last login: Fri Jan 20 22:35:48 2022 from user-workstation john@server1:~$ uptime 22:59:34 up 27 days, 56 min, 1 user, load average: 0.00, 0.01, 0.05 john@server1:~$ hostname -I 192.168.1.5 john@server1:~$ exit Connection to 192.168.1.5 closed
If you successfully logged in without any prompts – congratulations! Your SSH key authentication is working properly.
Now let‘s look at troubleshooting in case you run into any connection issues…
Troubleshooting SSH Key Connections
If your SSH client can‘t log into servers using your keypair, there are a few common issues to check:
Client configuration – Ensure
PubkeyAuthentication yesis enabled in your local client‘s
ssh_configfile. And that
PasswordAuthenticationis set to
Server configuration – Verify the remote SSH server has
PasswordAuthentication noset properly in its
sshd_config. And that
sshdwas reloaded after making changes.
Public key permissions – Check the permissions (
600) on remote
.ssh/authorized_keyscontaining your public key.
Private key permissions – Your private key (
id_rsa) should have
600permissions set on the client machine.
SSH agent – Your local SSH agent service may not be running or doesn‘t contain your key. Load it with
Account access – The remote user account may not have a general shell enabled. Only allow key auth on accounts needing SSH access.
Networking – Something is blocking the SSH port (22). Check firewall rules and routing between your client and server.
Server keys – Flush your client‘s known hosts entry with
ssh-keygen -R server_ipto clear previous key server authentications before testing.
With a little debugging, you should be able to nail down any issues getting your SSH keys to authenticate properly. The most common problems I see are related to permissions or config values set incorrectly.
Extra Tips for Securing SSH Keys
SSH key authentication increases your Linux server security significantly. But there are a few extra precautions you can take:
Enforce strong passphrases – I highly recommend going back and setting a very strong passphrase on your private SSH keys. This adds two-factor authentication, protecting the key if your machine is ever compromised.
Limit cache duration – Your SSH key passphrase can be cached by your local
ssh-agent. Set a maximum cache expiration time with
ssh-add -t 3600 (1 hour for example).
Use hardware tokens – Consider storing your SSH keys on a YubiKey hardware device. This keeps the private keys entirely off your computer.
Enable MFA – For super security, combine SSH key auth with another factor like a TOTP code or U2F security key. Tools like Google Authenticator can facilitate this.
Monitor logs – Watch auth logs on your SSH servers to detect if brute force attacks start targeting user accounts. Key auth prevents access but knowing your systems are being probed has value.
Rotate keys – Create new SSH keypairs every few months and roll out to servers. This reduces damage if a key is somehow compromised in the future.
That covers the full process of using SSH keys for passwordless authentication!
The initial configuration requires some attention to detail across a few steps:
- Carefully generating keys on your client machine
- Securely copying public keys into remote user accounts
- Properly disabling password authentication in SSH configs
- Verifying logins work smoothly
But once done, you can eliminate countless password prompts and scripts breaking on account changes.
Plus you can sleep better knowing your Linux servers are significantly more secure against brute force attacks and password leaks.
I hope walking through real world examples gives you confidence to implement SSH public key authentication in your own environment.
If you have any other tips or feedback on SSH key-based auth, I‘d love to hear! Just send me an email anytime.
- Venafi Survey Finds SSH Keys Used in 90% of Data Breaches Due to Poor Management. Venafi. https://www.venafi.com/press-releases/venafi-survey-finds-ssh-keys-used-90-percent-data-breaches-due-poor-management