SSH Key Setup and Management - Complete Guide

Published: September 25, 2024 | Reading time: 16 minutes

SSH Security Overview

Secure Shell (SSH) provides encrypted communication between client and server:

SSH Components

# SSH Key Components
- Private Key (id_rsa) - Keep secure, never share
- Public Key (id_rsa.pub) - Can be shared safely
- Authorized Keys - Server-side key storage
- SSH Agent - Key management and caching

# Security Benefits
- Passwordless authentication
- Stronger than passwords
- Encrypted communication
- Audit trail capabilities

Generating SSH Keys

Basic Key Generation

Generate RSA Key

# Generate RSA key pair
ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

# Generate with custom filename
ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa_github -C "github@example.com"

# Generate Ed25519 key (recommended)
ssh-keygen -t ed25519 -C "your_email@example.com"

# Generate with passphrase
ssh-keygen -t ed25519 -C "your_email@example.com" -N "your_passphrase"

# List existing keys
ls -la ~/.ssh/

Key Types Comparison

RSA Keys

Widely supported
Minimum 2048 bits
Recommended 4096 bits
Larger key size
Legacy compatibility

Ed25519 Keys

Modern and secure
Smaller key size
Faster performance
Better security
Recommended for new keys

SSH Key Management

SSH Agent

SSH Agent Commands

# Start SSH agent
eval "$(ssh-agent -s)"

# Add key to agent
ssh-add ~/.ssh/id_rsa

# Add all keys
ssh-add

# List loaded keys
ssh-add -l

# Remove specific key
ssh-add -d ~/.ssh/id_rsa

# Remove all keys
ssh-add -D

# Add key with timeout
ssh-add -t 3600 ~/.ssh/id_rsa

SSH Config File

~/.ssh/config

# Global settings
Host *
    AddKeysToAgent yes
    UseKeychain yes
    IdentitiesOnly yes

# GitHub configuration
Host github.com
    HostName github.com
    User git
    IdentityFile ~/.ssh/id_rsa_github
    IdentitiesOnly yes

# Production server
Host production
    HostName 192.168.1.100
    User deploy
    IdentityFile ~/.ssh/id_rsa_production
    Port 22
    ServerAliveInterval 60

# Development server
Host dev
    HostName dev.example.com
    User developer
    IdentityFile ~/.ssh/id_rsa_dev
    Port 2222
    LocalForward 8080 localhost:8080

Server Configuration

Adding Public Key to Server

Copy Public Key

# Copy public key to clipboard (macOS)
pbcopy < ~/.ssh/id_rsa.pub

# Copy public key to clipboard (Linux)
xclip -sel clip < ~/.ssh/id_rsa.pub

# Copy public key to clipboard (Windows)
clip < ~/.ssh/id_rsa.pub

# Display public key
cat ~/.ssh/id_rsa.pub

# Copy key to server manually
ssh-copy-id user@server.com

# Copy specific key
ssh-copy-id -i ~/.ssh/id_rsa.pub user@server.com

Server-Side Configuration

Server Setup

# Create .ssh directory
mkdir -p ~/.ssh
chmod 700 ~/.ssh

# Add public key to authorized_keys
echo "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQ..." >> ~/.ssh/authorized_keys

# Set proper permissions
chmod 600 ~/.ssh/authorized_keys

# Verify permissions
ls -la ~/.ssh/

# Test SSH connection
ssh user@server.com

SSH Security Best Practices

Server Security Configuration

/etc/ssh/sshd_config

# Disable password authentication
PasswordAuthentication no

# Disable root login
PermitRootLogin no

# Change default port
Port 2222

# Limit authentication attempts
MaxAuthTries 3

# Disable empty passwords
PermitEmptyPasswords no

# Enable key authentication
PubkeyAuthentication yes

# Disable X11 forwarding
X11Forwarding no

# Set login grace time
LoginGraceTime 30

# Disable unused authentication methods
ChallengeResponseAuthentication no
KerberosAuthentication no
GSSAPIAuthentication no

# Restart SSH service
sudo systemctl restart sshd

Key Security Practices

Key Protection

Use strong passphrases
Store keys securely
Use SSH agent
Rotate keys regularly
Use different keys per service

Access Control

Limit key access
Use key restrictions
Monitor key usage
Revoke unused keys
Use key expiration

Multiple Key Management

Service-Specific Keys

Generate Multiple Keys

# GitHub key
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_github -C "github@example.com"

# GitLab key
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_gitlab -C "gitlab@example.com"

# Production server key
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_production -C "production@example.com"

# Development server key
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_dev -C "dev@example.com"

# Add all keys to SSH agent
ssh-add ~/.ssh/id_ed25519_github
ssh-add ~/.ssh/id_ed25519_gitlab
ssh-add ~/.ssh/id_ed25519_production
ssh-add ~/.ssh/id_ed25519_dev

SSH Config for Multiple Keys

Multiple Key Configuration

# GitHub
Host github.com
    HostName github.com
    User git
    IdentityFile ~/.ssh/id_ed25519_github
    IdentitiesOnly yes

# GitLab
Host gitlab.com
    HostName gitlab.com
    User git
    IdentityFile ~/.ssh/id_ed25519_gitlab
    IdentitiesOnly yes

# Production server
Host production
    HostName prod.example.com
    User deploy
    IdentityFile ~/.ssh/id_ed25519_production
    Port 22
    IdentitiesOnly yes

# Development server
Host dev
    HostName dev.example.com
    User developer
    IdentityFile ~/.ssh/id_ed25519_dev
    Port 2222
    IdentitiesOnly yes

SSH Key Restrictions

Key Restrictions

Restricted Keys

# Key with command restriction
command="rsync --server --daemon .",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQ...

# Key with IP restriction
from="192.168.1.0/24" ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQ...

# Key with time restriction
from="192.168.1.0/24",no-port-forwarding,no-X11-forwarding ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQ...

# Key with environment restriction
environment="BACKUP_USER=backup" ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQ...

Troubleshooting SSH Issues

Common Problems

Debug SSH Connection

# Verbose SSH connection
ssh -v user@server.com

# More verbose
ssh -vv user@server.com

# Maximum verbosity
ssh -vvv user@server.com

# Test with specific key
ssh -i ~/.ssh/id_rsa user@server.com

# Check SSH agent
ssh-add -l

# Test key fingerprint
ssh-keygen -lf ~/.ssh/id_rsa.pub

# Check server key
ssh-keyscan server.com

Permission Issues

Fix Permissions

# Fix SSH directory permissions
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_rsa
chmod 644 ~/.ssh/id_rsa.pub
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/config

# Check permissions
ls -la ~/.ssh/

# Common permission errors:
# - .ssh directory: 755 (should be 700)
# - Private key: 644 (should be 600)
# - authorized_keys: 644 (should be 600)

SSH Key Rotation

Key Rotation Process

Rotate Keys

# 1. Generate new key
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_new -C "new_key@example.com"

# 2. Add new key to server
ssh-copy-id -i ~/.ssh/id_ed25519_new.pub user@server.com

# 3. Test new key
ssh -i ~/.ssh/id_ed25519_new user@server.com

# 4. Update SSH config
# Replace old key with new key in ~/.ssh/config

# 5. Remove old key from server
ssh user@server.com "sed -i '/old_key_fingerprint/d' ~/.ssh/authorized_keys"

# 6. Backup old key
mv ~/.ssh/id_ed25519_old ~/.ssh/id_ed25519_old.backup

SSH Tunneling

Port Forwarding

SSH Tunnels

# Local port forwarding
ssh -L 8080:localhost:80 user@server.com

# Remote port forwarding
ssh -R 8080:localhost:80 user@server.com

# Dynamic port forwarding (SOCKS proxy)
ssh -D 1080 user@server.com

# Background tunnel
ssh -f -N -L 8080:localhost:80 user@server.com

# Tunnel with specific key
ssh -i ~/.ssh/id_rsa -L 8080:localhost:80 user@server.com

# Kill tunnel
pkill -f "ssh.*-L.*8080"

SSH Key Backup and Recovery

Backup Strategy

Backup Keys

# Create backup directory
mkdir -p ~/ssh_backup

# Backup all SSH keys
cp -r ~/.ssh ~/ssh_backup/

# Encrypt backup
tar -czf ssh_backup.tar.gz ~/ssh_backup
gpg --symmetric ssh_backup.tar.gz

# Store encrypted backup securely
# - External drive
# - Cloud storage (encrypted)
# - Password manager

# Restore from backup
gpg --decrypt ssh_backup.tar.gz.gpg | tar -xzf -
cp -r ~/ssh_backup/.ssh ~/

Summary

SSH key management is essential for secure server access:

Generate strong keys: Use Ed25519 or RSA 4096-bit keys
Use SSH agent: Manage keys securely and efficiently
Configure SSH config: Simplify connections and key management
Implement security: Disable password auth, use key restrictions
Rotate keys regularly: Maintain security over time
Backup keys securely: Encrypt and store safely

Need More Help?

Struggling with SSH key setup or need help securing your server access? Our SSH experts can help you implement proper key management and security practices.

Get SSH Help