Rate limiting

A rate limiter caps how many requests one client can make in a span of time, like 100 per minute.

Why limit

• Abuse: a bot guessing passwords or copying every page.
• Overload: one buggy client in a loop can slow the app for everyone.
• Fairness: each customer gets a fair share.

Saying no to an extra request is cheap; serving it may not be.

Who to limit

Count requests per user or per API key when you know who is calling. Use the IP address for visitors who are not logged in. An IP is only a rough guess: a whole office can share one IP.

A request over the limit gets status 429 Too Many Requests. A Retry-After header can tell the client how long to wait. With many servers, keep the counts in a shared store that every server sees.

Fixed window

Split time into blocks, like each clock minute, and count requests in the current block. When a new minute starts, the count goes back to zero.

It is simple, but has a flaw at the edge. With 100 per minute, a client can send 100 requests at 0:59 and 100 more at 1:00. That is 200 requests in about two seconds.

Sliding window

A sliding window counts requests in the last 60 seconds, measured from now. It moves with time, so there is no edge to burst across. Keeping the time of every request is exact, but uses more memory.

Token bucket

A bucket holds up to 10 tokens and gets 1 new token each second. Each request takes one token. If the bucket is empty, the request is rejected. The bucket never holds more than its size.

The bucket size sets the biggest burst. The refill rate sets the average over time.

Remember

• Over the limit, answer 429 Too Many Requests.
• Limit per user or API key; an IP is a rough fallback.
• A fixed window can let twice the limit through around an edge.
• A sliding window has no edge.
• Token bucket: size sets the burst, refill rate sets the average.