Hardening a Small Server Without Making It Unusable

A single virtual machine running a couple of small sites does not need a security programme. It needs about a dozen specific changes, most of which take a minute each.

Two of them look finished when they are not, and those two are worth more attention than the rest combined. They are at the end.

Start with the front door

Key-only SSH, no root login, and a firewall that rejects by default.

# /etc/ssh/sshd_config
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no

Moving SSH off port 22 is worth doing, but be honest about what it buys. It does not stop a targeted attacker, who will find the port in one scan. It removes almost all of the background noise, which makes the logs readable, and readable logs are what let you notice the targeted attempt.

Keep 22 open until the new port is proven. Lock yourself out of a remote box and the recovery is a console session you may not have.

iptables -A INPUT -p tcp --dport 22    -j ACCEPT
iptables -A INPUT -p tcp --dport 57000 -j ACCEPT   # your new port
iptables -A INPUT -p tcp --dport 80    -j ACCEPT
iptables -A INPUT -p tcp --dport 443   -j ACCEPT
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -j REJECT --reject-with icmp-host-prohibited

Close 22 once you have logged in on the new port from a machine you are not currently connected from. Add fail2ban and leave it at defaults.

Turn off what you are not using

A default cloud image runs services you will never touch. Every one is attack surface for nothing in return.

ss -ltnp                       # what is actually listening
systemctl list-units --type=service --state=running

rpcbind is the usual offender. It listens on 111, it is enabled by default on several images, and nothing on a web server needs it.

systemctl disable --now rpcbind rpcbind.socket
systemctl mask rpcbind

mask rather than disable, because a dependency can pull a merely disabled unit back up.

Give a small box swap

A 1 GB machine running a database and an application server will eventually hit a moment where it needs 1.1 GB. Without swap the kernel kills something, and what it kills is usually the database.

fallocate -l 2G /swapfile && chmod 600 /swapfile
mkswap /swapfile && swapon /swapfile
echo '/swapfile none swap sw 0 0' >> /etc/fstab
sysctl -w vm.swappiness=10     # swap as a safety net, not as memory

Swap is not free performance. It is the difference between slow and dead.

Answer nothing to unknown hostnames

Point any name at your address and it reaches your web server. Without a default virtual host, whichever site happens to be first answers, which means every scanner in the world gets a copy of your site under whatever hostname it invented.

server {
    listen 80 default_server;
    listen 443 ssl default_server;
    server_name _;
    ssl_certificate     /path/to/any/cert.pem;
    ssl_certificate_key /path/to/any/key.pem;
    return 444;                      # close without responding
}

444 is nginx’s own code for “drop the connection”. A scanner gets an empty reply instead of a page to fingerprint.

Test it from outside, with a hostname you never configured:

curl -s -o /dev/null -w '%{http_code}\n' -H 'Host: nothing.example.com' http://YOUR.IP/
# curl exits 52, "empty reply from server". That is the 444.

Headers, and the one that is hard

The easy ones go in the server block and take a minute:

add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
add_header X-Content-Type-Options    "nosniff" always;
add_header X-Frame-Options           "SAMEORIGIN" always;
add_header Referrer-Policy           "strict-origin-when-cross-origin" always;

Content-Security-Policy is the one that takes real work, because a useful CSP means removing every inline <script> from your pages. script-src 'self' with 'unsafe-inline' still present is close to worthless: 'unsafe-inline' is exactly the thing an injected script needs.

Moving inline scripts into a file is mechanical. Find them:

grep -rc '<script>' your-built-site/ | grep -v ':0'

Then move each block into your existing bundle, guarded so it stays inert on pages that do not need it:

function initThing() {
  if (!document.querySelector('[data-thing]')) return;
  /* ... the code that used to be inline ... */
}

Verify against the built output rather than a dev server. A dev server injects its own scripts and will show you failures that do not exist, and hide successes that do.

DNS is part of the server

Four records decide whether somebody else can send mail as you.

Three things about these are worth knowing before you touch them.

Do not publish a personal address in DMARC. The rua and ruf fields sit in a public TXT record that harvesters read. Use an alias you can discard.

Skip ruf entirely. Aggregate reports (rua) are XML counts. Forensic reports (ruf) are copies of failing messages, headers and often body, sent to you unencrypted.

Go to -all last. Softfail (~all) while you confirm every real sender is listed, then harden. Removing an include: for a provider that does send puts your own mail in spam, and you will find out slowly.

For a domain that sends no mail at all, publish that fact:

example.com.        TXT  "v=spf1 -all"
_dmarc.example.com. TXT  "v=DMARC1; p=reject; sp=reject; adkim=s; aspf=s"
*._domainkey        TXT  "v=DKIM1; p="

Note that a CDN may append its own CAs to your CAA record so its certificates keep working. Check what resolves publicly after you add one; it may be less restrictive than what you wrote.

Trap one: flexible TLS

If you put a CDN in front of your origin, its SSL mode decides how it talks to you. “Flexible” means it fetches over plain HTTP and serves the visitor over HTTPS. The padlock appears. The traffic between the proxy and your server is cleartext.

Set it to full, strict, before the proxy is ever switched on. The setting is inert while the records are unproxied, which is exactly why that is the moment to fix it - there is nothing to break.

Trap two: rate limits behind a proxy

This is the one that looks completely fine and is not.

An application behind a reverse proxy sees the proxy as its client. Every request appears to come from 127.0.0.1. So a per-IP rate limiter keyed on the socket address gives the entire internet one shared bucket.

The failure is quiet in both directions. It never fires for the attacker you meant to stop, because they share the allowance with everyone. And it fires for everybody at once when any single person exhausts it, which is a denial of service anyone can trigger by getting their own password wrong ten times.

The proxy already sends the real address:

proxy_set_header X-Real-IP $remote_addr;

The application has to read it, and trust it only from the proxy:

fn client_ip(headers: &HeaderMap, addr: &SocketAddr) -> IpAddr {
    let peer = addr.ip();
    if !peer.is_loopback() {
        return peer;           // not via the proxy: never trust the header
    }
    headers.get("x-real-ip")
        .and_then(|v| v.to_str().ok())
        .and_then(|v| v.trim().parse().ok())
        .unwrap_or(peer)
}

The loopback check is the whole security of it. If your application also listens on a public interface, a request arriving directly can claim any address it likes, and a limiter that believes it is worse than none.

Test it with two source addresses rather than reading the code. Exhaust the allowance from one, then make a single request from another. If the second is refused, every client shares a bucket.

If you later add a CDN, the same collapse happens one layer up. The proxy then needs to learn the real address from the CDN before it can pass it on:

set_real_ip_from 173.245.48.0/20;      # one line per CDN range
real_ip_header CF-Connecting-IP;

Back up in a way that can fail loudly

A backup you have never restored is a file.

pg_dump -Fc -d mydb > "$out"
pg_restore -l "$out" >/dev/null || { rm -f "$out"; exit 1; }

That second line is the point. A truncated or empty dump is the failure that stays hidden until the day it matters, and listing the archive catches it for free. Delete the bad dump and exit non-zero so the timer reports a failure.

Then actually restore one, into a scratch database, and compare row counts against production. Do it once when you set it up, and again whenever the schema changes.

The checklist

What this does not solve

None of it helps if the application itself is broken. A server can be perfectly configured and still hand out other people’s data because a query forgot a WHERE user_id = ?.

It also does not survive a stolen SSH key. Everything above assumes the key is safe. Put a passphrase on it, and keep the number of machines holding it small.