Configure SSH Servers and Clients in Linux

Series: Learning Linux Basics — Networking | Topic: Configure SSH Servers and Clients

---

The Gateway to Remote Linux Administration

SSH — the Secure Shell protocol — is the most important tool in a Linux administrator's toolkit. Every time you manage a server that isn't physically in front of you, SSH is how you get there. It creates an encrypted tunnel between your local machine and a remote server, letting you run commands, transfer files, and manage systems as securely as if you were sitting right at the keyboard.

But SSH is far more than just a way to open a terminal on a remote machine. It's a configurable system with fine-grained controls for authentication methods, access restrictions, port configurations, key-based login, and client shortcuts that can save enormous amounts of time when managing many servers.

This lesson covers both sides of SSH: the server (the SSH daemon that listens for incoming connections on your remote machine) and the client (the program you run locally to connect to it). We'll walk through configuration files, authentication hardening, SSH key generation, and the known_hosts fingerprint system — everything you need to set up, secure, and efficiently use SSH in real environments.

---

Part 1: The Two Halves of an SSH Connection

Every SSH connection involves two participants:

Role	Program	Config File	Purpose
SSH Server (remote)	sshd (OpenSSH Daemon)	/etc/ssh/sshd_config	Listens for incoming connections, authenticates users
SSH Client (local)	ssh	/etc/ssh/ssh_config (global) or ~/.ssh/config (per-user)	Initiates connections to remote servers

A Linux machine typically runs both — it has an SSH daemon accepting connections from others, and an SSH client for connecting to other servers. This is why two very similarly named config files exist:

• /etc/ssh/sshd_config — the SSH Daemon (server) config — note the d

• /etc/ssh/ssh_config — the SSH client config — no d

Mixing them up is a surprisingly common mistake. The d is your cue: if you're configuring what incoming connections are allowed, you want sshd_config.

---

Part 2: Configuring the SSH Server (sshd_config)

sudo vim /etc/ssh/sshd_config

The file is heavily commented — almost every setting appears as a commented-out line showing its default value. This means: if you see #Port 22, the actual active default is port 22, but the line isn't doing anything because it's commented. To change a setting, uncomment the line (remove the #) and modify the value.

Key Setting 1: Port Number

#Port 22

SSH listens on port 22 by default. Changing this to a non-standard port (e.g., Port 2222) is a common approach to reduce noise from automated bots that constantly probe port 22 across the internet. However, it's a minor security-through-obscurity measure — not a replacement for proper authentication hardening.

To find out what any setting means, consult the manual:

man sshd_config

Use / to search within the manual. For example, typing /AddressFamily jumps straight to its description.

Key Setting 2: Address Family

#AddressFamily any

Controls which IP protocol versions SSH accepts connections on:

Value	Accepts
any	Both IPv4 and IPv6 (default)
inet	IPv4 only
inet6	IPv6 only

Key Setting 3: Listen Address

#ListenAddress 0.0.0.0

By default, SSH listens on all available network interfaces. If a server has two interfaces — say, 203.0.113.1 connected to the internet and 10.11.12.9 connected to an internal office network — you might want SSH accessible only from inside the office:

ListenAddress 10.11.12.9

Now SSH only accepts connections arriving on that internal IP. External connections to 203.0.113.1 on port 22 will be silently refused, even if the firewall allows them through.

Key Setting 4: PermitRootLogin

PermitRootLogin prohibit-password

This setting controls whether the root user can log in via SSH:

Value	Behaviour
yes	Root can log in with password or key
prohibit-password	Root can log in with SSH keys only, not passwords (default)
no	Root cannot log in at all via SSH

The default prohibit-password is a reasonable balance: root can still access the machine remotely using keys (which are much more secure than passwords), but password-based root logins are blocked.

For maximum security on a hardened server:

PermitRootLogin no

This forces administrators to log in as a regular user and use sudo for elevated commands.

Key Setting 5: Password Authentication

#PasswordAuthentication yes

Controls whether users can authenticate with a password. Many security-conscious administrators set this to no once SSH key authentication is configured — passwords are far more vulnerable to brute-force attacks than keys.

Key Setting 6: KbdInteractiveAuthentication

#KbdInteractiveAuthentication yes

This is subtly different from PasswordAuthentication:

• PasswordAuthentication yes — allows the SSH client to automatically send a password (can be scripted)

• KbdInteractiveAuthentication yes — requires the user to actually type the password at the keyboard interactively

To completely disable all password-based login and force key-only authentication, set both to no:

PasswordAuthentication no
KbdInteractiveAuthentication no

Key Setting 7: X11Forwarding

X11Forwarding yes

X11 forwarding lets you run a graphical application on the remote server but display its window on your local screen — similar in concept to TeamViewer, but for a single application rather than an entire desktop. If you don't need this (most servers don't), it can be set to no to reduce attack surface.

Per-User Setting Overrides

SSH allows global settings to be overridden for specific users. This is useful when you want a strict global policy but need an exception for one account:

# Global setting at top of file:
PasswordAuthentication no

# Per-user exception at bottom of file:
Match User anu
    PasswordAuthentication yes

User anu can now log in with a password, even though no other user can. The Match block must be at the end of the file — settings after a Match block only apply to that matched context.

Reloading After Changes

Every time you modify sshd_config, the daemon must reload to apply the changes:

sudo systemctl reload ssh.service
# or on Red Hat-based systems:
sudo systemctl reload sshd.service

The Override Directory: /etc/ssh/sshd_config.d/

There's an important gotcha: files in /etc/ssh/sshd_config.d/ are loaded after the main config file and can override your settings. On cloud instances, you may find a file like 50-cloud-init.conf that contains:

PasswordAuthentication yes

This will re-enable password authentication even if you've disabled it in the main file — because the directory file is processed later and wins.

Always check this directory before assuming your settings took effect:

ls /etc/ssh/sshd_config.d/
cat /etc/ssh/sshd_config.d/50-cloud-init.conf

If a conflicting file exists, either remove it, edit it, or override it with a higher-numbered file of your own.

---

Part 3: Configuring the SSH Client

The client side of SSH is equally configurable. SSH client configuration lives in two places:

File	Scope
~/.ssh/config	Per-user configuration (only for the current user)
/etc/ssh/ssh_config	Global configuration (all users on this machine)

Per-User Client Configuration: ~/.ssh/config

SSH clients are used on all major platforms:

• Linux/macOS: Open a terminal and type ssh

• Windows 10+: Open Command Prompt or PowerShell and type ssh

The per-user SSH directory lives at:

• Linux/macOS: ~/.ssh/ (e.g., /home/Sudheer/.ssh/)

• Windows: C:\Users\Sudheer\.ssh\

No client config file exists by default — you create it manually:

mkdir -p ~/.ssh
vim ~/.ssh/config
chmod 600 ~/.ssh/config

⚠️ Permissions matter: The SSH client refuses to use a config file that's readable by other users. chmod 600 (owner read/write only) is required.

Why Use a Client Config?

Without a config file, connecting to a server looks like:

ssh Sudheer@10.0.0.186

That's manageable for one server. But if you manage ten servers with different usernames, ports, and key files, you'll be memorising connection strings constantly.

A ~/.ssh/config file solves this with named shortcuts:

Host myserver
    HostName 10.0.0.186
    User Sudheer
    Port 22
    IdentityFile ~/.ssh/id_ed25519

Host workdb
    HostName 192.168.1.50
    User dbadmin
    Port 2222

Now connecting is simply:

ssh myserver
ssh workdb

The SSH client looks up the matching Host block and fills in all the details automatically.

To discover all available client config options:

man ssh_config

Global Client Configuration: /etc/ssh/ssh_config

The global client config applies to all users on the machine. However, just like sshd_config, direct edits risk being overwritten by software updates.

The recommended approach is to add a new file in the config directory:

sudo vim /etc/ssh/ssh_config.d/custom.conf

Port 229

Now every SSH connection made from this machine defaults to port 229 instead of 22 — useful if all your internal servers use a non-standard port.

---

Part 4: SSH Key Authentication

Password authentication is convenient but vulnerable — passwords can be guessed, leaked, or phished. SSH key authentication is far more secure and, once set up, actually easier to use.

How SSH Keys Work

SSH key authentication uses a cryptographic key pair:

Private Key  ← stays on your local machine, NEVER shared
     │
     │  mathematically linked
     ▼
Public Key   ← copied to the remote server's authorized_keys file

When you connect, the server sends a challenge that only the holder of the matching private key can answer. No password is transmitted. Even if someone intercepts the network traffic, they get nothing usable.

Step 1 — Generate a Key Pair

Run this on your local machine (or any machine you want to connect from):

ssh-keygen

You'll be prompted for:

1. File location — press Enter for the default (~/.ssh/id_ed25519)

2. Passphrase — optionally encrypt the private key file itself

💡 What is a passphrase for? The passphrase encrypts your private key file on disk. If someone steals the file, they still can't use it without the passphrase. In production environments, always use a passphrase. For development/testing where convenience matters more, you can leave it empty.

This creates two files:

• ~/.ssh/id_ed25519 — your private key (guard this carefully, never share it)

• ~/.ssh/id_ed25519.pub — your public key (safe to share freely)

Step 2 — Copy the Public Key to the Server

Option A: The easy way (recommended)

ssh-copy-id Sudheer@10.0.0.186

This command:

1. Connects to the server using your password (for this one last time)

2. Appends your public key to ~/.ssh/authorized_keys on the server

3. Creates the file and sets correct permissions automatically

Option B: Manual copy (when ssh-copy-id isn't available)

# On your local machine, display the public key:
cat ~/.ssh/id_ed25519.pub

# SSH to the server (with password), then:
mkdir -p ~/.ssh
vim ~/.ssh/authorized_keys
# Paste the public key content here and save

# Set required permissions:
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

⚠️ Permissions are enforced: SSH will refuse to use an authorized_keys file that's too permissive. chmod 600 (owner read/write only) is required.

Step 3 — Test Key-Based Login

ssh Sudheer@10.0.0.186

If the key is set up correctly, you'll connect without being prompted for a password (or just for your key's passphrase, if you set one).

Adding Multiple Keys

The authorized_keys file can contain many public keys — one per line. This is how you grant multiple people access to the same account:

ssh-ed25519 AAAA... alice@laptop
ssh-ed25519 AAAA... bob@workstation
ssh-ed25519 AAAA... carol@homepc

Alice, Bob, and Carol each have their own private key. All three can log into the Sudheer account because their public keys are listed in authorized_keys.

---

Part 5: Host Fingerprints and known_hosts

The first time you connect to a new SSH server, you see:

The authenticity of host '10.0.0.186 (10.0.0.186)' can't be established.
ED25519 key fingerprint is SHA256:xxxxxxxxxxxxxxxxxxx
Are you sure you want to continue connecting (yes/no/[fingerprint])?

This is SSH protecting you from man-in-the-middle attacks. It's asking: "I've never seen this server before. Are you sure you trust it?"

When you type yes, the server's fingerprint is saved to:

~/.ssh/known_hosts

On every subsequent connection, your SSH client compares the server's current fingerprint against what's stored. If they match — you connect instantly, no prompt. If they don't match:

WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!

Two scenarios cause this warning:

1. Someone is impersonating the server (take this seriously)

2. The server was reinstalled and got a new host key (common and innocent)

If you know the server was legitimately reinstalled, remove the old fingerprint:

ssh-keygen -R 10.0.0.186

This removes the entry for 10.0.0.186 from known_hosts. The next connection will prompt you to accept the new fingerprint.

To remove all stored fingerprints (nuclear option):

rm ~/.ssh/known_hosts

---

Complete Reference: Files and Commands

Configuration Files

File	Purpose
/etc/ssh/sshd_config	SSH server (daemon) settings
/etc/ssh/sshd_config.d/	Override files for SSH server
/etc/ssh/ssh_config	Global SSH client settings
/etc/ssh/ssh_config.d/	Override files for SSH client (preferred)
~/.ssh/config	Per-user SSH client shortcuts
~/.ssh/authorized_keys	Public keys allowed to log in as this user
~/.ssh/known_hosts	Fingerprints of previously connected servers
~/.ssh/id_ed25519	Your private key (never share)
~/.ssh/id_ed25519.pub	Your public key (safe to share)

Essential Commands

# --- Server Management ---
# Edit SSH server config
sudo vim /etc/ssh/sshd_config

# Reload SSH daemon after config changes
sudo systemctl reload ssh.service

# Check SSH daemon status
systemctl status ssh.service

# --- Key Generation and Distribution ---
# Generate an SSH key pair
ssh-keygen

# Copy public key to remote server (easy way)
ssh-copy-id username@server_ip

# Remove a stale server fingerprint
ssh-keygen -R 10.0.0.186

# --- Connecting ---
# Basic connection
ssh username@10.0.0.186

# Connect using a config file shortcut (after setting up ~/.ssh/config)
ssh myserver

# Connect to a non-standard port
ssh -p 2222 username@10.0.0.186

# Connect using a specific private key
ssh -i ~/.ssh/id_ed25519 username@10.0.0.186

# --- Checking Manuals ---
man sshd_config    # SSH server config options
man ssh_config     # SSH client config options
man ssh-keygen     # Key generation options

---

Common Questions Answered

Q: What's the difference between ssh_config and sshd_config? The d makes all the difference: sshd_config (with d) configures the SSH daemon — the server side that receives connections. ssh_config (without d) configures the SSH client — the program that initiates connections. On a typical Linux machine, both files exist because the machine can both receive connections and initiate them.

Q: I set PasswordAuthentication no in sshd_config but I can still log in with a password. Why? Check /etc/ssh/sshd_config.d/ for override files. Cloud providers often ship a file like 50-cloud-init.conf that re-enables password authentication. Your file change is overridden by this later-processed file. Edit or remove that file, then reload SSH.

Q: What algorithm should I use when generating SSH keys?ssh-keygen defaults to ed25519, which is excellent — modern, secure, and fast. Avoid RSA keys shorter than 4096 bits. The default ed25519 is the recommended choice for new keys on all modern systems.

Q: Is it safe to store the private key without a passphrase? For personal development use on a machine only you access, it's convenient and the risk is low. For production access or any key that grants access to important systems, always use a passphrase. If someone steals an unencrypted private key file, they have immediate access to everything that key unlocks.

Q: Can I use the same SSH key pair for multiple servers? Yes. Generate one key pair and add your public key to ~/.ssh/authorized_keys on as many servers as you like. This is the common approach for a single administrator managing many servers. Different team members each have their own key pairs, and all their public keys are added to the appropriate authorized_keys files.

---

What's Next?

You now have complete control over SSH configuration on both sides of the connection:

1. ✅ SSH server config (sshd_config) — port, listen address, authentication methods, per-user overrides

2. ✅ The override directory (sshd_config.d/) — understanding cloud provider conflicts

3. ✅ SSH client config — per-user shortcuts (~/.ssh/config) and global settings

4. ✅ Key authentication — generating keys, copying public keys, authorized_keys

5. ✅ Host fingerprints — known_hosts, why the warning appears, removing stale entries

Coming up in the Networking chapter: Firewall management with iptables and ufw — controlling exactly which network connections are allowed to reach your servers, including your SSH service.

---

This post is part of the Learning Linux Basics series — a beginner-friendly journey through the essential skills every Linux user and administrator should know.