ulimits and process limits
A service that works in testing and falls over under load is often hitting a limit, not a bug. The ceilings are invisible until you reach them.
The one that bites first
Open file descriptors. Every connection and every open file counts, and busy network services hit the default ceiling and start refusing connections with confusing errors. Raising the open-files limit for the service is one of the most common fixes for "it stops accepting connections at a few thousand users".
Set them where they stick
Shell ulimit settings do not reliably reach a daemon. For a service, set the limits in its systemd unit, where they apply every time it starts, rather than relying on login shells. Check what a running process actually has, not what you think you set.
The others worth knowing
- Maximum processes and threads — a fork bomb or a thread leak hits this.
- Locked memory, for databases that pin pages.
- Kernel-wide ceilings on connection tracking and sockets — see conntrack and TCP tuning.
When a server misbehaves only at scale, check limits early — it is faster than chasing a phantom bug. A slow server covers the wider diagnosis.
Still stuck? Mail support@novogara.com — an engineer answers, at any hour. Back to the knowledge base