Rate limiting a public API

Design an API Rate Limiter

A public API must hold every customer to their plan across thirty stateless servers, absorb honest bursts, stop abuse, and never let the limiter itself become the outage.

Intermediate, about 40 minutes, 10 stages

Start the investigation Design it from a blank page See the full walkthrough

The situation

You run the public REST API of a developer platform. Customers authenticate with API keys and buy plans: Free at 60 requests a minute, Pro at 600, and Enterprise at whatever sales negotiated. Last month a Pro customer shipped a client with a tight retry loop, and the flood of requests pinned the primary database for forty minutes. Every customer was affected.

The API runs as 20-60 stateless instances behind a load balancer that cannot do custom limiting. A Redis cluster is available in the same region. Your task is a rate limiter that the business can trust and customers can understand.

What it has to do

Functional

Enforce per-API-key limits according to each customer's plan.
Reject over-limit requests with 429, a Retry-After header, and remaining-quota headers.
Apply stricter limits to the login endpoint, per IP and per account.
Change a customer's plan without a deploy.

Non-functional

The limiter adds under 2 ms at p99.
A client sending at twice its limit is held near its limit, not allowed thirty times it by the fleet.
If the limiter's state store fails, the API stays up.
Honest bursts, such as a dashboard firing 20 requests on load, are not punished.

Constraints and assumptions

20-60 API instances (autoscaled), about 40,000 requests a second at peak across ~50,000 active keys.
Redis cluster in-region with ~0.3 ms round trips.
The API's p99 latency budget is 150 ms; the primary database is the resource being protected.
Authenticated requests carry an API key; unauthenticated ones can only be identified by IP.
IPs are shared (corporate NAT, mobile carriers) and can be rotated by attackers.
Instance clocks are NTP-synchronized to within milliseconds but can still disagree.

Interview questions it prepares you for

“Design a rate limiter.”
“How would you enforce API quotas across a fleet of stateless servers?”
“Design protection against credential stuffing on a login endpoint.”
“Your rate limiter's Redis goes down. What happens to your API?”

Read and practise next

How Cloudflare built it · Rate limiting at the edge, in their engineers' own words

Concepts to know first: Rate limiting, Concurrency control.