Three dashboards instead of thirty

Almost every environment we take over has Grafana. And almost every one has dashboards nobody has opened in months. The problem is not too little observation, it is too much of it with no question behind it. Observability you actually read therefore starts not with the tool but with the question of which three statements about the system have to be answerable at any moment: is it running, is it getting slower, and where is it stuck right now. Anything that does not answer one of those three is decoration, however pretty the panel.

Every dashboard answers exactly one question

Before a panel exists, it has to be clear which decision it supports. “Is the service healthy?” is a question. “CPU usage across all pods” is not — it is a number.

We almost always get by with three views:

1. Are users all right? Error rate, latency at the edge, successful core actions per minute.
2. Is the system all right? Saturation of the scarce resources: connections, queue length, memory.
3. What changed recently? Deployments, configuration changes, feature flags — as markers on the timeline.

Everything beyond that is diagnosis. Diagnosis belongs in ad-hoc queries, not on a dashboard.

Alerts you can take seriously

An alert that misfires weekly gets muted. At that point it is no longer an alert. Our rule: every alert has a name, a threshold derived from the promise made to users, and a runbook. If one of those is missing, the alert is deleted rather than extended.

An alert is a request for human attention. You send those sparingly.

Logs with context, not with volume

Structured, with a request ID, with the why rather than the what. payment declined: limit exceeded, limit=500 amount=620 says everything. error in handler says nothing, and it is nonetheless in every second log we see.

What is left at the end

Fewer panels, fewer alerts, fewer log lines — and a team that during an incident knows within thirty seconds where to look. That is observability. The rest is disk space.