Set up an HAProxy load balancer for web servers

A load balancer spreads incoming traffic across several servers so no single one is overwhelmed, and it stops sending traffic to a server that fails. HAProxy is a fast, free and widely used load balancer. This guide explains the basics and sets up an HAProxy load balancer in front of two web servers.

Before you start

• Three Linux servers on the same private network if possible: one for HAProxy and two web servers (for example 10.0.0.11 and 10.0.0.12) serving the same site.
• Your domain's A record pointing to the HAProxy server.
• A shared place for uploads and sessions (shared storage, object storage, or a database/Redis session store) so both web servers behave the same.

Load balancing basics

• Layer 4 (TCP) – forwards connections without reading HTTP; fast and protocol-agnostic.
• Layer 7 (HTTP) – reads requests, so it can route by host or path, add headers and terminate HTTPS.

Install HAProxy

1. Ubuntu/Debian: sudo apt install -y haproxy. AlmaLinux/Rocky: sudo dnf install -y haproxy.
2. Enable it: sudo systemctl enable haproxy.

Configure the frontend and backend

1. Edit /etc/haproxy/haproxy.cfg and add below the existing global and defaults sections:
   Copy

   frontend web_in
       bind *:80
       mode http
       option forwardfor
       default_backend web_servers
   
   backend web_servers
       mode http
       balance roundrobin
       option httpchk GET /health
       http-check expect status 200
       server web1 10.0.0.11:80 check
       server web2 10.0.0.12:80 check
   
   listen stats
       bind 127.0.0.1:8404
       mode http
       stats enable
       stats uri /stats

2. Create a /health page on each web server that returns 200 when the app and database work.
3. Validate and start:
   Copy

   sudo haproxy -c -f /etc/haproxy/haproxy.cfg && sudo systemctl restart haproxy

option forwardfor adds the X-Forwarded-For header so the web servers can log real visitor IPs.

Add sticky sessions (optional)

If your app keeps sessions in local files, keep each visitor on one server with a cookie:

backend web_servers
    cookie SRV insert indirect nocache
    server web1 10.0.0.11:80 check cookie web1
    server web2 10.0.0.12:80 check cookie web2

Shared session storage is still the better long-term fix.

Terminate HTTPS on HAProxy

1. Get a certificate (for example with Certbot standalone) and combine the full chain and key into one file: cat fullchain.pem privkey.pem > /etc/haproxy/certs/example.com.pem.
2. Add to the frontend:
   Copy

       bind *:443 ssl crt /etc/haproxy/certs/example.com.pem alpn h2,http/1.1
       http-request redirect scheme https unless { ssl_fc }
       http-request set-header X-Forwarded-Proto https if { ssl_fc }

3. Reload HAProxy and renew the combined file after each certificate renewal with a Certbot deploy hook.

What to check afterwards

• View the stats page through an SSH tunnel to 127.0.0.1:8404/stats; both servers should be green.
• Stop the web server on one node and confirm the site still loads.
• Check web server logs show real client IPs.

Troubleshooting

• 503 Service Unavailable: all backends failed health checks – test curl http://10.0.0.11/health from the HAProxy server.
• Users logged out randomly: sessions are stored locally; add stickiness or shared sessions.
• SELinux denies connections: run setsebool -P haproxy_connect_any 1 on RHEL-family systems.

FAQ about HAProxy load balancers

Is the load balancer a single point of failure?

Yes, unless you run two HAProxy nodes with a floating IP managed by keepalived (VRRP).

HAProxy or Nginx for load balancing?

Both work. HAProxy has richer health checks, stats and balancing options; Nginx is handy if you already use it as a web server.

Can HAProxy balance MySQL?

Yes, in TCP mode, but database replication and write routing need careful planning.

Next steps

• Set up Nginx as a reverse proxy
• TCP vs UDP and ports explained
• SSH tunnels and port forwarding