# System Design Drill Day 7: Rate Limiting APIs, Logins and OTPs with Redis Someone finds your OTP endpoint and starts hammering it. Every call sends an SMS, and you pay for each one. Rate limiting fits into a half-hour interview and still touches algorithms, shared state and failure modes. In this drill I design limits for an online marketplace running on 3 Laravel servers with a shared Redis, covering the general API, login, OTP SMS and partner API keys. ## Core concepts A rate limit caps how many requests a client can make in a time window. It blocks brute-force logins and scrapers, keeps expensive calls like SMS and payments from burning money, and stops one noisy client from starving the rest. ### The five algorithms to compare The simplest approach counts requests per key per minute (`rl:user:42:202610081030`) and rejects anything past the limit. That fixed window counter is cheap, but it has a boundary problem: 60 requests at 10:30:59 and 60 more at 10:31:00 make 120 in two seconds. A sliding window log fixes that by storing a timestamp for every request in a Redis sorted set, dropping old entries and counting the rest. It's exact, but memory grows with traffic. The sliding window counter sits between the two. It weights the previous window against the current one, so 30% into this minute the count is prev × 0.7 + current. That's nearly exact with fixed-window memory, and it's a good default answer in an interview. A token bucket holds up to N tokens and refills at R per second, and each request spends one. Short bursts are allowed as long as the average holds, which suits API plans. Stripe wrote in 2017 that its API rate limiter uses a token bucket ([Scaling your API with rate limiters](https://stripe.com/blog/rate-limiters)), and AWS documents [API Gateway throttling](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-request-throttling.html) the same way. A leaky bucket queues requests and drains them at a fixed rate. Its output is smooth, but bursts wait or get dropped, so it behaves more like a traffic shaper than an API limiter. ### Distributed concerns With several app servers, the counter has to live in shared state, which here means Redis. If each server counts in its own memory, a client effectively gets three times the limit across 3 servers. The update also has to be atomic. Reading the count, checking it and incrementing it in separate calls leaves room for a race. For a fixed window, `INCR` and `EXPIRE` sent as two separate calls aren't atomic either: if the process dies between them, the key never expires and that user stays limited. Run both inside one Lua script, or create the key with its expiry first using `SET key 0 EX 60 NX` and then `INCR` it, so the TTL is in place before any count exists. For anything more involved than a counter, a Lua script runs the check and the update as one step inside Redis. Rejected requests get `429` with a `Retry-After` header, set to the number of seconds until the client can succeed again. For a fixed window that's the time until the window resets, which is simply the key's remaining `TTL`. For a token bucket it's the time until the next whole token arrives: `ceil((1 - tokens) / refill_per_sec)`. For each limit you also decide whether to fail open or fail closed when Redis is down, and the right choice differs between limits. > **Pause and think:** Each of your 3 servers counts requests in its own memory with a limit of 60 per minute. What limit does a client actually get? **Answer:** Up to 180 per minute, because the load balancer spreads the client's requests across all 3 servers and each one counts separately. That's why the counter has to live in shared Redis. All three servers end up sharing one bucket per key in Redis. ![Three Laravel servers sharing one Redis, updating the rate limit bucket atomically with a single Lua script and returning 429 when over the limit](https://cdn.hashnode.com/uploads/covers/5fdde6221485f22e24b34a2c/4edcff55-6160-43d7-ad7a-c2db1167eb91.png align="center") *Figure 1: One shared bucket in Redis, updated in a single Lua call, so all servers enforce one limit.* ## The design problem The marketplace runs on 3 Laravel app servers with a shared Redis. Design limits for the general API per user, for login against brute-force attacks, for OTP SMS where every message costs money, and for partner API keys on Basic and Pro tiers. For each limit, say what happens when Redis is down. > **Try the problem before reading the solution.** ## Model solution ### 1 and 2. Per-user API and login Use shared Redis counters with an atomic check-and-increment, so all 3 servers enforce a single limit instead of three separate ones. ### 3\. OTP SMS Limit sends per phone number, and add a global daily SMS budget (say 5,000 a day) that trips like a circuit breaker and alerts you. A per-number limit alone does nothing against an attacker cycling through thousands of numbers. Send the SMS from a queued job (Day 3) so a slow gateway doesn't hold up the request. If Redis is down, I'd fail closed, because a short outage costs less than an SMS-pumping attack. ### 4\. Partner API keys Partners get a token bucket keyed by `partner:{key}`. Basic has capacity 100, refilling at 100/60 tokens per second, and Pro has 1,000 at 1,000/60. Store `tokens` and the last refill time in a Redis hash and update them in one Lua script, so refill, check and spend are atomic across all 3 servers. This script refills, checks and spends a token in one call: ```lua -- KEYS[1]=bucket ARGV: capacity, refill_per_sec, now local b = redis.call('HMGET', KEYS[1], 'tokens', 'ts') local tokens = tonumber(b[1]) or tonumber(ARGV[1]) local ts = tonumber(b[2]) or tonumber(ARGV[3]) tokens = math.min(tonumber(ARGV[1]), tokens + (ARGV[3]-ts)*ARGV[2]) local ok = tokens >= 1 if ok then tokens = tokens - 1 end redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', ARGV[3]) redis.call('EXPIRE', KEYS[1], 3600) -- also return the remaining tokens (as a string, so Redis keeps the fraction) return {ok and 1 or 0, tostring(tokens)} ``` Call it from custom middleware with `Redis::eval(...)`. It returns whether the request is allowed plus the tokens left, and when it's rejected the middleware sets `Retry-After` to `ceil((1 - tokens) / refill_per_sec)` seconds. The servers' clocks need to agree for `now` to mean anything, so keep them in sync or read the time from Redis itself. Return `429` with `Retry-After` and the `X-RateLimit-*` headers. If Redis is down here, I'd fail open with a local fallback where each server enforces about a third of the limit in its own memory. A paying partner shouldn't lose access because your Redis is down. ![Fail-open vs fail-closed when Redis is down: OTP SMS fails closed, partner API keys fail open with a local per-server limit](https://cdn.hashnode.com/uploads/covers/5fdde6221485f22e24b34a2c/0783b06f-3ebd-4db0-8fcc-1941330d3ed3.png align="center") *Figure 2: Same outage, opposite policies, because the cost of being wrong is different.* ### Interview bonus points Say where the limiter lives (edge plus app). Mention that the Redis check adds about 1 ms per request, so it should stay at one round trip. Point out that a hot partner key is also a hot Redis key, the Day 6 hot-spot problem again. And log every 429 so limits can be tuned from real data. ### Something to try next How would you rate limit across two regions that each have their own Redis? **Answer:** The simplest option is to split each limit between the regions, for example half each or in proportion to normal traffic, and let each Redis enforce its share. It needs no cross-region calls, but a client whose traffic all lands in one region hits the limit early. If that's a problem, count locally and sync the counts between regions in the background. That's closer to the real limit but can overshoot a little while the sync catches up. For limits where overshooting costs real money, like the global SMS budget, keep one authoritative counter in a single home region and accept the extra latency, or set each region's share low enough that the total stays safe. ## Interview one-liner > "Pick the algorithm per use case (sliding window counter as the default, token bucket for API plans), keep counters in shared Redis updated atomically with Lua, return 429 with Retry-After, and choose fail-open or fail-closed per limit: closed for SMS, open with a local fallback for paying partners." ## Conclusion Choosing the algorithm is the easy part; shared state, atomic updates and the Redis-down policy are where rate limiters fail. Decide fail-open or fail-closed separately for each limit, based on what being wrong would cost. ## Try it yourself Load the Lua script above with `Redis::eval()` in a small Laravel middleware and give a test key a capacity of 5, refilling at 1 token per second. Hit the endpoint 10 times quickly with `curl` in a loop. You should see 5 successes and then `429`s. A fast loop of 10 calls finishes well within a second, so it won't show any refill; wait a couple of seconds (or add a `sleep 1` before the last few calls) and run it again to see tokens come back. Then compare it with Laravel's built-in `RateLimiter::for()`, which is a fixed-window counter: it counts hits in the cache and resets them when the decay period ends. --- This is part of my daily [System Design Drills](https://blog.anishansari.com.np/series/system-design-drills) series.