How to secure a new Linux server: a beginner's hardening checklist

Most servers aren't broken into by clever attacks. They're broken into by programs that scan the whole internet, try common passwords and known vulnerabilities on every address they find, and move on to the next. Put a new server on a public IP and its login log starts filling with attempts for root, admin and ubuntu soon after. Even the large breaches usually start this simply. At Equifax, in 2017, attackers came in through a vulnerability that had been public, with a patch available, for two months, and stayed undetected for 76 days.

A new server from a cloud provider or an installer isn't insecure by design. It's set up to be easy to reach and to work out of the box, and that leaves a few doors open until you close them. This guide closes them in about an hour, in the order that won't lock you out. The commands are for Ubuntu and Debian; other distributions differ in package names, not in principle.

Before you start

You need a new server, its IP address, and the root password or the SSH key the provider set up. Keep the provider's web console within reach: if something goes wrong with SSH, it's how you get back in.

One rule for the whole guide: when you change anything about SSH, keep your current session open and test from a new one. An open session isn't affected by the change, so it's your way back.

1. Install the updates

Start from current packages, so you're not hardening software that already has known holes:

apt update && apt full-upgrade -y
# If this file exists, a new kernel or library needs a restart
[ -f /var/run/reboot-required ] && reboot

2. Create a user for yourself

Working as root means every typo runs with full power, and every attacker already knows the username. Create your own user and give it sudo:

adduser deploy                 # choose a strong password; sudo will ask for it
usermod -aG sudo deploy

deploy is just an example. Use any name that isn't easy to guess.

3. Log in with a key instead of a password

A password can be guessed. An SSH key can't, in any practical sense. On your own computer, create a key if you don't have one, and copy it to the server:

ssh-keygen -t ed25519                  # accept the default path; set a passphrase
ssh-copy-id deploy@203.0.113.10        # your server's IP
ssh deploy@203.0.113.10                # should log in without the user's password

If the provider set up the server with your key for root and password login is already off, ssh-copy-id won't get in as the new user. Copy root's key across instead, on the server: rsync --archive --chown=deploy:deploy ~/.ssh /home/deploy.

Check that sudo works for the new user (sudo -v) before you go on. The next step takes root's SSH access away.

4. Turn off passwords and root login

On current Ubuntu and Debian, SSH reads extra settings from /etc/ssh/sshd_config.d/. Put yours in a file there rather than editing the main one:

# /etc/ssh/sshd_config.d/00-hardening.conf
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
AllowUsers deploy

The 00 in the name matters. SSH keeps the first value it reads for each setting, and reads these files in name order. Some cloud images include a file such as 50-cloud-init.conf that turns password login back on. A file named 60-hardening.conf would lose to it, silently, and the server would still accept passwords while your configuration says it doesn't.

So don't trust the file. Ask SSH what it will actually do, then reload it:

sudo sshd -t     # checks the syntax; no output means it's fine
sudo sshd -T | grep -Ei 'permitrootlogin|passwordauthentication|kbdinteractive|allowusers'
sudo systemctl reload ssh

Now, from a new terminal, log in as your user (it should work) and try ssh root@203.0.113.10 (it should be refused). Only then close the old session.

Moving SSH to a port other than 22 is optional. It cuts the noise in the logs, because most scanners only try 22, but it doesn't make anything harder to break into. Turning passwords off does that.

5. Turn on the firewall

A firewall should allow what the server is for and nothing else. ufw is the simplest way on Ubuntu, where it's already installed; on Debian, install it first with sudo apt install ufw. Allow SSH before you enable it, or the firewall will cut your own connection:

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow OpenSSH
sudo ufw allow 80,443/tcp      # only if this server serves websites
sudo ufw enable
sudo ufw status verbose

One catch, if you'll run Docker on this server: ports that containers publish bypass ufw. Docker writes its own firewall rules, and its documentation says plainly that Docker and ufw are incompatible in this respect. A database published as "5432:5432" is open to the internet even though ufw status doesn't show it. Publish only what must be public, and bind everything else to the server itself, as "127.0.0.1:5432:5432".

6. Install security updates automatically

New vulnerabilities are found every week. If updating depends on someone remembering, the server falls behind as soon as that person is busy. unattended-upgrades installs security updates on its own, every day:

sudo apt install unattended-upgrades
sudo dpkg-reconfigure --priority=low unattended-upgrades    # answer Yes
sudo unattended-upgrade --dry-run --debug                    # shows what it would do

It installs updates but doesn't restart the server, and a new kernel only takes effect after a restart. Either reboot when /var/run/reboot-required appears, or let it reboot by itself at a quiet hour:

# /etc/apt/apt.conf.d/52unattended-upgrades-local
Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-Time "04:00";

Note that APT works the other way round from SSH: here the last value read wins, so a file whose name sorts after 50unattended-upgrades overrides it.

7. Block repeated login attempts

With passwords off, guessing can't succeed, but the attempts still fill the logs and use resources. fail2ban watches for repeated failures and blocks the source in the firewall for a while:

sudo apt install fail2ban python3-systemd

# /etc/fail2ban/jail.local
[sshd]
enabled  = true
backend  = systemd
maxretry = 5
findtime = 10m
bantime  = 1h

backend = systemd tells it to read the system journal. Minimal installs, such as a default Debian 12, keep their logs only there and have no /var/log/auth.log, so without this line fail2ban has nothing to read and may not start at all. Restart it and check:

sudo systemctl restart fail2ban
sudo fail2ban-client status sshd

8. Check your work with an audit

Lynis checks a server against hundreds of hardening rules and lists what it found:

sudo apt install lynis
sudo lynis audit system

At the end it prints a hardening index and a list of warnings and suggestions. Don't try to fix everything: some suggestions don't fit every server, and a few can break things. Read the warnings first, fix what applies, and run it again. The point is to have a repeatable check, so that next month you can see whether the server has drifted.

The checklist

• Updates close known vulnerabilities in the installed software.
• Your own user with sudo ends working as root, under a username everyone knows.
• An SSH key replaces a password that can be guessed.
• No passwords and no root login close the scanners' main way in.
• The firewall hides services that were never meant to be public.
• Automatic security updates keep you from falling behind on the next vulnerability.
• fail2ban stops endless attempts from the same sources.
• The audit shows what you missed, and later, what changed.

Going further

These eight steps are the floor, not the ceiling. On servers I set up for my own projects, I built on the same base with existing Ansible hardening roles that I adapted: kernel and network restrictions through sysctl, AppArmor, and CrowdSec next to fail2ban. I audited the result with both Lynis and OpenSCAP, which check against different rule sets, and fixed what they found. On top of that came detection: Wazuh, with decoders and correlation rules for the SSH and Traefik logs that recognise credential stuffing, scanners enumerating paths, path traversal and probes for known vulnerabilities, with CrowdSec blocking those sources automatically, and file integrity monitoring that reports when system files change.

The step that matters most for more than one server is the one this guide does by hand: every command above has to be repeated on every new server, exactly the same way. That's the job of a configuration tool such as Ansible, and the subject of the next article. If you're setting up servers and want a second pair of eyes, tell me what you're running.

Sources: The Equifax Data Breach (US House Committee on Oversight and Government Reform); sshd_config manual (OpenBSD); Packet filtering and firewalls (Docker); Unattended upgrades (Debian Wiki); Fail2ban; Lynis (CISOfy).