Autoscaling

Autoscaling is a controller moving the number of running instances between a floor and a ceiling you set. It can react to one signal or several (CPU, requests per instance, concurrency, queue depth, a custom business metric), follow a schedule you know in advance, or on some platforms run a predictive policy trained on past traffic. The usual meaning is horizontal: more identical copies behind a load balancer. Vertical scaling means giving one machine more CPU or RAM, which normally needs a restart and has a hard ceiling. Serverless platforms take the idea further, though not uniformly: some run one request per instance, others let a single instance handle many at once, and 'scales to zero' is a common default rather than a law, since most platforms let you hold a minimum warm and will happily charge you for it.

Reach for it when load is spiky and predictable only in shape: launches, campaign emails, business hours, background job bursts. If your traffic is flat and small, two fixed instances are cheaper, calmer, and easier to reason about than a policy that flaps.

Autoscaling mostly reacts, and reacting takes time. Between the metric window, the cooldown, the image pull, and framework warm-up you are often minutes late to a spike that lasted ninety seconds, so raise the floor before a known event instead of trusting the policy. Scaling the app tier also just moves the bottleneck downstream: 50 instances opening 20 database connections each will exhaust the database long before the app breaks a sweat, which is what a connection pooler such as PgBouncer is for. And set a real maximum, because a retry storm or a runaway loop will happily scale you into a five-figure invoice.