Spreading traffic across two servers

Sort out the state first

Before any balancer, answer three questions:

• Sessions. If they are files on disk, a user bounced to the other machine is logged out. Move them to Redis or the database.
• Uploads. A file written on one machine does not exist on the other. Move them to shared storage or an object store.
• The database. One database, reachable privately from both application servers — see connecting your own servers. Replicating it is a later and much larger decision.

Skip this and you will spend your first week blaming the balancer for logouts that are your application's doing.

The simplest balancer: DNS

Two A records for the same name, and clients spread themselves across them. It costs nothing and needs no extra machine.

The catch is failure: DNS does not know a server is down, so a share of your visitors keep being sent to a dead one until you change the record and caches expire. Keep the TTL short, and treat this as load spreading rather than high availability. For the latter, see high availability.

A real balancer in front

nginx is adequate and you probably already run it:

upstream app {
  server 10.0.0.11:8080 max_fails=3 fail_timeout=10s;
  server 10.0.0.12:8080 max_fails=3 fail_timeout=10s;
}

HAProxy is better at the job proper: real active health checks, far more detailed statistics, and queueing that behaves sensibly when the backends are saturated. Choose it if balancing is the machine's purpose.

Health checks are the point

Balancing without health checks just spreads your errors evenly. Give your application an endpoint that returns 200 only when it can really serve — database reachable, cache reachable, migrations applied — and have the balancer poll it every couple of seconds. A backend that fails is removed and returns when it passes again.

Keep the check cheap. One that runs an expensive query every two seconds becomes its own load problem.

Sticky sessions: last resort

ip_hash in nginx, or a cookie in HAProxy, pins a visitor to one backend. It works, and it means that losing a machine logs those users out and that load spreads unevenly. Use it when you cannot fix the session storage, not instead of fixing it.

Where to terminate TLS

At the balancer, almost always. One place to manage certificates, and the backends speak plain HTTP over the private network. Pass the original client address through with X-Forwarded-For and make your application trust it only from the balancer — otherwise anyone can claim any address, which breaks your rate limiting and your logs.

The balancer is now the single point

You have moved the problem rather than solved it. Two balancers with a floating address is the next step, and that is what high availability covers. Be honest about whether you need it: for many businesses, one well-built machine with fast hardware replacement is more available in practice than a complicated cluster nobody fully understands.

---

Still stuck? Mail support@novogara.com — an engineer answers, at any hour. Back to the knowledge base