System Design · Architecture under constraints
What makes a system observable?
Short answer
Observability is the ability to understand internal state from outputs: metrics, logs and traces, correlated by request id. It requires deliberate instrumentation and control of cost and cardinality.
Monitoring answers known questions with dashboards and alerts. Observability lets you ask new questions during an incident without shipping code. The three pillars are metrics, logs and traces, but the value comes from correlating them around a request, a user or a deployment.
The pillars
- Metrics: cheap time series for rates, errors, durations and saturation, ideal for alerting
- Logs: detailed events with context, useful for debugging but expensive at high volume
- Traces: a request's path across services with span timings and attributes
- Profiles: continuous CPU and memory samples that show where time is spent
- Events: deploys, feature-flag changes and config updates that explain a change in behavior
Useful conventions reduce guesswork. RED metrics cover rate, errors and duration for services; USE covers utilization, saturation and errors for resources. Include a correlation id at the edge, propagate it through every call, and attach it to logs and traces. Record the deploy version and region on every signal so a spike can be tied to a rollout.
Cardinality is the budget. A metric labeled by user id or request id explodes the time-series count and can take down the monitoring system. Keep high-cardinality dimensions in logs or traces, not metric labels. Sample traces at the edge with a strategy that keeps all errors and a representative share of successes.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1How do you control metric cardinality?
- 2What would you sample in a tracing pipeline?
- 3How do you correlate a log line with a trace?
- 4Why alert on symptoms rather than causes?
Say it in the interview
Observability means I can answer new questions about a running system without changing code. I combine metrics for alerting, logs for detail and traces for cross-service timing, all correlated by a request id and tagged with deploy version and region. I keep high-cardinality data out of metric labels and sample traces deliberately. Alerts fire on user-visible symptoms against an SLO, not on every resource spike.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live System Design mock interview and get scored feedback on depth, clarity and confidence.