Load balancer
One address in front of many servers, handing each request to whichever one is healthy and least busy. Visitors just see a single site.
See it
What it is
A load balancer takes one public address and hands each incoming request to one of several identical backends, picking by round robin, least connections, or a hash of something stable. It polls each backend with a health check and quietly stops sending traffic to any that stops answering. Layer 4 balancers move TCP and are fast and incurious; layer 7 balancers read HTTP, so they can route by host or path, terminate TLS, and rewrite headers. AWS ALB, Google Cloud Load Balancing, and Cloudflare Load Balancing are the managed versions; nginx, HAProxy, Caddy, and Traefik are the ones you run yourself. On a PaaS there is already one in front of you and you never touch it.
You need one the moment you run more than a single instance: for surviving one machine dying, for zero-downtime deploys (drain connections from the old instances, then swap), and for shifting a slice of traffic during a canary release.
The classic gotcha is sticky sessions. If your app keeps login state in process memory, every request that lands on a different instance looks logged out, and the tempting fix is to pin each visitor to one backend. Do the other fix: move sessions to Redis or a signed cookie so any instance can serve anyone. Second gotcha is a health check that only proves the process is alive, so a broken instance with a dead database connection keeps getting traffic. Third: WebSockets and long uploads die at the default idle timeout unless you raise it.
Ask AI for it
Put a layer 7 load balancer in front of this app running on three instances. Terminate TLS at the balancer, route by host and path, use least-connections, and enable HTTP/2 to clients. Add a /healthz endpoint that actually checks the database connection, and configure the health check to pull an instance after two failures and return it after two passes. Enable connection draining with a 30 second grace period so deploys finish in-flight requests, raise the idle timeout to 300 seconds for the WebSocket route, and forward the real client IP via X-Forwarded-For. No sticky sessions: refactor session state into Redis so any instance can serve any user.