Nginx load balancing: upstream blocks, methods and failover

When one application server is not enough, Nginx can act as a load balancer, spreading requests across a group of backends defined in an upstream block. It also gives you failover: if a backend dies, Nginx stops sending it traffic. This guide covers the load balancing methods in open-source Nginx and the settings that make failover reliable.

Before you start

• Two or more backend servers running the same app, reachable from the Nginx server (ideally over a private network).
• Nginx as a reverse proxy: Nginx reverse proxy setup.
• Shared state handled outside the app servers: sessions in Redis or the database, uploads on shared storage or object storage.

A basic upstream

upstream app_servers {
    server 10.0.0.11:8080;
    server 10.0.0.12:8080;
    server 10.0.0.13:8080;
    keepalive 32;
}

server {
    listen 443 ssl;
    http2 on;
    server_name app.example.com;
    # ssl_certificate lines here

    location / {
        proxy_pass http://app_servers;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

keepalive with HTTP/1.1 and an empty Connection header reuses backend connections, which lowers latency.

Load balancing methods

Weights, backups and draining

upstream app_servers {
    least_conn;
    server 10.0.0.11:8080 weight=3;
    server 10.0.0.12:8080;
    server 10.0.0.13:8080 backup;
    server 10.0.0.14:8080 down;
}

• weight=3: gets three times the share; useful for a bigger server.
• backup: used only when all primary servers are unavailable.
• down: temporarily removed, e.g. while you patch it. Reload Nginx to apply.

Passive health checks and failover

Open-source Nginx marks a server as failed based on real traffic:

server 10.0.0.11:8080 max_fails=3 fail_timeout=30s;

After 3 failures within 30 seconds the server is skipped for 30 seconds, then tried again. What counts as a failure, and whether Nginx retries on another server, is controlled by:

proxy_next_upstream error timeout http_502 http_503 http_504;
proxy_next_upstream_tries 2;
proxy_connect_timeout 3s;

A short proxy_connect_timeout makes failover fast. Be careful retrying non-idempotent requests: by default Nginx does not retry POST after the request was sent, which prevents duplicate orders.

Active health checks (probing /health on a schedule) are part of the commercial NGINX Plus. With open-source Nginx, combine passive checks with external monitoring, or use HAProxy, which has active checks built in: HAProxy load balancer setup.

Sticky sessions

If your app keeps sessions in local memory or files, users must return to the same backend. Options:

• ip_hash: simple, but users behind one corporate NAT all hit one server, and mobile users switching networks lose their session.
• hash $cookie_sessionid consistent;: sticks by session cookie.
• Best: move sessions to Redis or the database so any backend can serve any user, and keep round robin or least_conn.

Test failover

1. Add a header showing which backend answered: add_header X-Upstream $upstream_addr always;
2. Request the site several times and watch the header rotate.
3. Stop the app on one backend and confirm requests continue without errors.
4. Check the error log for "upstream timed out" or "no live upstreams".

Troubleshooting

• "no live upstreams while connecting to upstream": every server is marked failed. Check backends and the firewall between them.
• Users get logged out randomly: local sessions without stickiness.
• One server gets everything: ip_hash with most traffic from one IP (e.g. a CDN). Restore real IPs or use another method.

FAQ: Nginx load balancing

Is the load balancer itself a single point of failure?

Yes. Run two with a floating IP: Keepalived floating IP failover.

Can Nginx balance TCP or UDP services?

Yes, with the stream module, e.g. for databases or DNS.

How many backends can I add?

As many as you need; the upstream block has no practical limit for typical setups.

Next steps

• High availability basics
• Nginx proxy_cache
• Proxies and load balancers explained