Load balance backends with Nginx upstream

When one app process or one server is not enough, Nginx can share requests between several of them. You list the backends in an upstream block, pick a balancing method, and Nginx stops sending traffic to any backend that keeps failing. This guide covers the methods available in open source Nginx, health checks, sticky sessions and connection reuse.

Before you start

• A Linux VPS running Nginx as the front end. See Set up Nginx as a reverse proxy for the basics.
• Two or more backends serving the same app. They can be several processes on one server (ports 3001, 3002, 3003) or other VPS servers.
• If backends are on other servers, connect them over a private network or restrict their app port to the load balancer's IP with a firewall.
• Your app must not keep user state in memory only. Sessions should be in a shared store such as Redis or a database, or you need sticky sessions.

A basic upstream

Create a config file:

sudo nano /etc/nginx/conf.d/app-upstream.conf

upstream app_pool {
    server 10.0.0.11:8080;
    server 10.0.0.12:8080;
    server 10.0.0.13:8080;
}

Then point your site's location at it:

location / {
    proxy_pass http://app_pool;
    proxy_set_header Host              $host;
    proxy_set_header X-Real-IP         $remote_addr;
    proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

With no method set, Nginx uses round robin: each request goes to the next server in turn.

Choose a balancing method

Add one of these as the first line inside upstream:

• Round robin (default): fine when every request costs about the same.
• least_conn; sends each request to the backend with the fewest active connections. Better when some requests are slow, such as reports or uploads.
• ip_hash; always sends the same client IP to the same backend. A simple form of sticky sessions.
• hash $cookie_sessionid consistent; picks a backend by any value you choose, here a session cookie. consistent limits reshuffling when you add or remove a server.

Example:

upstream app_pool {
    least_conn;
    server 10.0.0.11:8080;
    server 10.0.0.12:8080;
}

Weights and backup servers

Give stronger servers a bigger share with weight:

upstream app_pool {
    server 10.0.0.11:8080 weight=3;
    server 10.0.0.12:8080 weight=1;
    server 10.0.0.13:8080 backup;
}

Here the first server gets roughly three requests for every one the second gets. The backup server only receives traffic when all the others are unavailable. backup cannot be used together with ip_hash or hash.

To take a server out for maintenance without deleting the line, add down and reload.

Passive health checks

Open source Nginx does not probe backends on a schedule. Instead it watches real requests. Two parameters control this:

upstream app_pool {
    least_conn;
    server 10.0.0.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.12:8080 max_fails=3 fail_timeout=30s;
}

If a server fails 3 times within 30 seconds, Nginx skips it for 30 seconds, then tries it again with a live request.

By default only connection errors and timeouts count as failures. To also retry on bad responses from a sick backend, set proxy_next_upstream in the location:

location / {
    proxy_pass http://app_pool;
    proxy_next_upstream error timeout http_502 http_503 http_504;
    proxy_next_upstream_tries 2;
    proxy_connect_timeout 3s;
}

Active health checks (Nginx probing a URL on a timer) are part of the commercial NGINX Plus. With open source Nginx, use an external monitor to alert you when a backend goes down.

Keepalive to backends

Opening a new TCP connection to the backend for every request wastes time. Keep a pool of idle connections open:

upstream app_pool {
    least_conn;
    server 10.0.0.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.12:8080 max_fails=3 fail_timeout=30s;
    keepalive 32;
}

And in the location:

proxy_http_version 1.1;
proxy_set_header Connection "";

keepalive 32 is the number of idle connections each Nginx worker keeps per upstream, not a limit on total connections.

Apply the change

sudo nginx -t && sudo systemctl reload nginx

On AlmaLinux or Rocky Linux, allow Nginx to open network connections if you have not already:

sudo setsebool -P httpd_can_network_connect 1

Check it works

Add a header that shows which backend answered. Put this in the location temporarily:

add_header X-Upstream $upstream_addr always;

Then send a few requests:

for i in 1 2 3 4 5 6; do curl -sI https://example.com/ | grep -i x-upstream; done

You should see the backend addresses change. Stop one backend and repeat: after a few failures Nginx should only show the healthy ones. Check the error log for upstream... failed lines, then remove the debug header.

Troubleshooting

• Users get logged out at random: sessions are stored per backend. Move them to Redis or a database, or use ip_hash or a cookie hash.
• One backend gets almost all traffic: you are using ip_hash behind a proxy, so every visitor has the same IP. Fix real IPs or switch methods.
• 502 errors while one backend is down: failures are not retried. Add proxy_next_upstream and a short proxy_connect_timeout.
• nginx -t says "invalid parameter backup": backup is not allowed with hash methods. Remove one or the other.
• Backends overloaded after a restart: recovered servers rejoin at full weight at once. Stagger restarts and keep spare capacity.

Next steps

• Keep each backend process alive: Keep Node.js apps running with PM2.
• Watch backend health: Monitor Linux server resources.
• Add more backends with extra Linux VPS servers.