System Design · Architecture under constraints
How does service discovery work in a dynamic environment?
Short answer
Service discovery maps a service name to healthy instance addresses. A registry tracks instances, and clients or a proxy resolve and load-balance among them, with health checks and caching to handle churn.
In a static environment, a service address might live in a config file. In containers and autoscaling groups, instances start and stop constantly, so addresses change. Service discovery provides a stable name and a current list of healthy endpoints behind it.
Client-side versus server-side
- Client-side: the caller queries a registry and chooses an instance, which gives fine control but spreads discovery logic into every client
- Server-side: a load balancer or proxy resolves the service, so clients only know a stable virtual address
- DNS-based: service names resolve to records, simple but constrained by TTL caching and limited health information
- Registry-based: Consul, etcd or a platform control plane stores instances and health status
Health checks determine what the registry advertises. A process can be running but unable to serve because its database connection pool is exhausted. Readiness should reflect serving ability, while liveness should reflect whether the process needs a restart. Deregistration must be prompt, or callers will keep sending traffic to dead instances.
Caching is unavoidable because querying the registry on every request is too slow. The cache introduces staleness: a newly started instance may not receive traffic yet, and a dead one may linger. Clients need timeouts, retries with jitter and a way to refresh, while the registry needs a lease or heartbeat so crashed instances expire.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1What is the difference between liveness and readiness in discovery?
- 2How does DNS TTL affect failover?
- 3Why is client-side discovery harder to operate?
- 4How do you avoid routing to an instance that is shutting down?
Say it in the interview
Service discovery gives callers a stable name and a current set of healthy instances. A registry tracks heartbeats and readiness, and either the client or a proxy resolves and balances across the result. Caching is necessary for performance but creates staleness, so I combine short leases, readiness checks and retries with jitter. On Kubernetes I would reuse the built-in service and endpoints machinery rather than add a second registry.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live System Design mock interview and get scored feedback on depth, clarity and confidence.