medium general · part of Practice Questions · Senior SWE Roadmap · topic form: Rate Limiting
Requirements to clarify
- Functional: limit requests per user/IP/API key to N per time window; return 429 when exceeded.
- Non-functional: must add negligible latency, must work correctly across multiple servers (distributed), should be configurable per endpoint/client tier.
Core components
- Algorithm choice: token bucket (allows bursts, simple), sliding window log/counter (accurate, more memory), fixed window (simplest, allows boundary bursts) — see Rate Limiting for the tradeoffs.
- Storage: a fast shared store (Redis) holding counters/tokens per client key, with atomic increment-and-check (e.g., Redis
INCR+EXPIRE, or a Lua script for atomicity). - Placement: typically enforced at the API gateway/edge, before requests hit application servers.
Key tradeoffs
- In-memory per-server counters are fast but wrong in a multi-server deployment (a client can exceed the limit by hitting different servers) — a centralized store fixes correctness at the cost of a network hop per request.
- Precision (sliding window) vs cost (fixed window / token bucket) — most systems accept fixed-window’s boundary imprecision for its simplicity and speed.