Design a Rate Limiter

Requirements to clarify

Functional: limit requests per user/IP/API key to N per time window; return 429 when exceeded.
Non-functional: must add negligible latency, must work correctly across multiple servers (distributed), should be configurable per endpoint/client tier.

Core components

Algorithm choice: token bucket (allows bursts, simple), sliding window log/counter (accurate, more memory), fixed window (simplest, allows boundary bursts) — see Rate Limiting for the tradeoffs.
Storage: a fast shared store (Redis) holding counters/tokens per client key, with atomic increment-and-check (e.g., Redis INCR + EXPIRE, or a Lua script for atomicity).
Placement: typically enforced at the API gateway/edge, before requests hit application servers.

Key tradeoffs

In-memory per-server counters are fast but wrong in a multi-server deployment (a client can exceed the limit by hitting different servers) — a centralized store fixes correctness at the cost of a network hop per request.
Precision (sliding window) vs cost (fixed window / token bucket) — most systems accept fixed-window’s boundary imprecision for its simplicity and speed.

Approach / Notes

The clean interview answer is to separate policy from enforcement. The policy decides who is limited, by how much, and over what window. The enforcement layer executes that policy as cheaply and atomically as possible, ideally before the request reaches your app servers.

Reference design

flowchart LR
	C[Client] --> G[API Gateway / Edge]
	G --> L[(Rate-limit store)]
	G --> A[Application service]
	L -->|allow| A
	L -->|reject 429| R[Retry-After + limit headers]

The gateway receives the request and computes a key such as user_id, api_key, or ip.
It checks the key in a shared store like Redis using an atomic algorithm.
If the request is over limit, it returns HTTP 429 and rate-limit headers.
Otherwise the request proceeds to the application.

Choosing an algorithm

Token bucket when bursts should be allowed up to a cap, which matches real client behavior well.
Sliding window counter when you want a cheap approximation that avoids the worst fixed-window boundary burst.
Fixed window only when you want the simplest implementation and can tolerate some burstiness.

flowchart TD
	Q{"Need burst tolerance?"} -->|yes| T[Token bucket]
	Q -->|somewhat, but cheap| S[Sliding window counter]
	Q -->|simplest possible| F[Fixed window]

The token-bucket default is the best general-purpose answer because it matches how traffic actually looks: a page load or mobile app can burst briefly, but shouldn’t sustain abuse indefinitely.

Distributed implementation

For a multi-server deployment, use Redis and make the check-and-update atomic.

Token bucket example

Store tokens and last_refill_ts per key. On each request:

Refill based on elapsed time.
Clamp at capacity.
If tokens remain, decrement and allow.
Otherwise reject.

That logic belongs in a Lua script or equivalent atomic operation so two app servers cannot both spend the same token.

Boundary burst example

Limit = 100 requests/minute.

sequenceDiagram
	participant C as Client
	participant L as Limiter

	C->>L: 100 requests at 12:00:59.9
	L-->>C: allowed
	C->>L: 100 requests at 12:01:00.1
	L-->>C: allowed (fixed window)

That is why fixed windows are easy but imperfect: a client can fit 200 requests into a tiny span straddling the boundary. Sliding-window or token-bucket policies reduce or eliminate that artifact.

Practical response contract

Return 429 Too Many Requests when the client is over limit.
Include Retry-After so well-behaved clients know when to try again.
Include rate-limit headers so the client can self-throttle before hitting the hard limit.

Practical answer shape

If asked live, say: I would enforce limits at the gateway, use Redis for shared state, implement the policy atomically with Lua, choose token bucket for burst-friendly limits, and layer per-user, per-IP, and global caps so one client cannot starve the system.

Notes

Explorer