Design a rate limiter for a public API.

Assume 50,000 requests per second at peak, limits per API key and per endpoint, a burst allowance, and a requirement that one abusive tenant cannot degrade others. The limiter must add only a few milliseconds to the request path and must be observable.

Token bucket

on request:
  refill tokens by elapsed_time * rate
  cap tokens at burst_size
  if tokens >= cost:
    tokens -= cost
    allow
  else:
    reject with retry_after = (cost - tokens) / rate

The counter must be updated atomically. A read, compute and write across separate Redis calls races when many requests arrive together. Use a Lua script or a single conditional operation so the refill, check and decrement happen as one step. The key should include the tenant and the limit dimension, such as ratelimit:tenant:endpoint:minute.

Distribute enforcement. A gateway can pre-check a local token bucket to shed obvious excess, then confirm with the shared store. For very large tenants, shard the counter and sum the shards, accepting a small over-admission window. Set a short TTL on idle keys so memory does not grow without bound.

Decide the failure policy before an incident. If Redis is unavailable, failing closed protects the backend but rejects legitimate traffic; failing open preserves availability but removes protection. Most public APIs choose a degraded local limit and an alert. Return 429 with Retry-After, and include limit and remaining headers so clients can adapt.