Connection tracking
A server that suddenly refuses new connections while the CPU is idle and the network is fine is very often a full conntrack table.
What it is
When your firewall has a rule like ct state established,related accept,
the kernel keeps a record of every connection so it can recognise the return traffic.
That table is finite. When it fills, new connections are dropped and a line appears
in the kernel log:
nf_conntrack: table full, dropping packet
dmesg -T | grep conntrack is the first place to look, and the message
is the whole diagnosis.
See where you stand
cat /proc/sys/net/netfilter/nf_conntrack_count — current
cat /proc/sys/net/netfilter/nf_conntrack_max — the ceiling
conntrack -L | wc -l — the same, with detail available
Put the ratio of those two in your monitoring with an alert at 80% — see Prometheus and Grafana. It is one of the cheapest alerts you can have and it catches a failure that is otherwise baffling.
Raising it
In /etc/sysctl.d/99-conntrack.conf:
net.netfilter.nf_conntrack_max = 1048576
net.netfilter.nf_conntrack_buckets = 262144
Keep buckets at roughly a quarter of max, or lookups slow down as the table grows. Each entry costs a few hundred bytes, so a million entries is a few hundred megabytes of kernel memory — fine on a server, worth knowing before you set ten million.
Timeouts matter more than the ceiling
The default for an established TCP connection is five days. On a busy server that means entries for connections that ended long ago:
net.netfilter.nf_conntrack_tcp_timeout_established = 3600
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 30
An hour is plenty for almost everything, and it frees far more space than doubling the table would.
Or stop tracking what you do not need
The cleanest fix for a high-rate public service is not to track it at all. For a web server that has no stateful rules on port 443:
nft add rule inet raw prerouting tcp dport 443 notrack
You then need explicit stateless rules for that traffic, which is more work and removes the problem entirely. Worth it on a delivery server or anything handling tens of thousands of connections.
The usual causes
- A genuine traffic increase — see capacity planning
- An attack, including a small one aimed at exhausting exactly this table — what you can do about DDoS
- A machine doing NAT for virtual machines or containers; every guest connection is an entry too — see Proxmox
- UDP-heavy traffic with long timeouts, which is the quiet one
If the table is filling because of traffic you did not expect, tell us — we can see the port from our side and say whether it is an attack. See understanding your traffic.
Still stuck? Mail support@novogara.com — an engineer answers, at any hour. Back to the knowledge base