Golden signals
Latency, traffic, errors, and saturation: the four charts that tell you whether a service is healthy, from the Google SRE book.
See it
What it is
From the Google SRE book: if you can only watch four things about a service, watch these. Latency, how long requests take. Traffic, how many are arriving. Errors, what fraction fail. Saturation, how full the most constrained resource is (memory, connection pool, queue depth, disk). Most outages announce themselves in one of the four before anyone files a ticket.
Two well-known repackagings exist and both are fine: RED (rate, errors, duration) for request-driven services, and USE (utilization, saturation, errors) for machines and resources. Reach for the golden signals when you are staring at a blank dashboard, or at the opposite problem: 40 panels of CPU graphs that nobody has looked at since they were made. Four panels per service, same layout everywhere, so anyone can read any service at 3am.
Two traps. Measure latency for successful and failed requests separately, because a fast 500 drags the average down and makes an outage look like a speed improvement. And do not skip saturation just because it is the awkward one to define; it is the only signal that warns you before the other three go bad.
Ask AI for it
Build a golden signals dashboard for this service, one row of four panels. Latency: p50, p95, and p99 request duration, with successful and failed requests plotted separately. Traffic: requests per second, broken down by route template rather than raw URL. Errors: 5xx rate as a percentage of total requests, plus a separate series for explicit application errors. Saturation: the most constrained resource for this workload (database connection pool usage, queue depth, memory headroom) against its limit. Use consistent colors and time ranges across all four, and label each panel with the threshold that should worry someone.