Rate Limiting Your API the Right Way: Token Bucket vs Sliding Window

The first rate limiter I ever shipped was a naive fixed window counter, reset a count to zero every minute, reject requests once the count crosses a threshold, and it took about a week in production before someone found the exact flaw every rate limiting tutorial warns you about but that feels abstract until you actually get burned by it: a client could send their full quota of requests in the last second of one window and their full quota again in the first second of the next window, getting twice the intended rate through in a two-second span straddling the boundary. Understanding why that happens, and which algorithms actually avoid it, is worth more than memorizing any specific implementation.

Fixed window: simple, and simply wrong at the edges

const counts = new Map();

function isAllowed(clientId: string, limit: number, windowMs: number): boolean {

const now = Date.now();

const entry = counts.get(clientId);

if (!entry || now - entry.windowStart >= windowMs) {

counts.set(clientId, { count: 1, windowStart: now });

return true;

}

if (entry.count < limit) {

entry.count++;

return true;

}

return false;

}

This is trivial to implement and trivial to reason about, which is exactly why it shows up so often in quick implementations. The boundary problem I described above is real and exploitable, and for anything where the rate limit is protecting against actual abuse rather than just rough fairness among well-behaved clients, it is not sufficient on its own.

Sliding window log: correct, but expensive at scale

The theoretically precise fix is to track the exact timestamp of every request within the trailing window and count how many fall inside it.

async function isAllowedSlidingLog(clientId: string, limit: number, windowMs: number): Promise {

const now = Date.now();

const key = 'ratelimit:' + clientId;

await redis.zremrangebyscore(key, 0, now - windowMs);

const count = await redis.zcard(key);

if (count < limit) {

await redis.zadd(key, now, now.toString());

await redis.expire(key, Math.ceil(windowMs / 1000));

return true;

}

return false;

}

Using Redis sorted sets, this is exact, there is no boundary exploit because you are always looking at a genuinely trailing window of real timestamps, not a fixed calendar-aligned bucket. The cost is memory and computation that scales with request volume, since you are storing a timestamp entry per request per client rather than a single counter, which matters if you have high-volume clients or a very large number of distinct clients to track simultaneously.

Sliding window counter: the practical middle ground

The algorithm I actually reach for most often approximates the sliding window using two fixed counters and a weighted interpolation between them, giving you nearly the correctness of the sliding log at nearly the cost of the fixed window.

async function isAllowedSlidingCounter(clientId: string, limit: number, windowMs: number): Promise {

const now = Date.now();

const currentWindow = Math.floor(now / windowMs);

const elapsedInWindow = now - currentWindow * windowMs;

const weight = elapsedInWindow / windowMs;

const currentKey = 'rl:' + clientId + ':' + currentWindow;

const previousKey = 'rl:' + clientId + ':' + (currentWindow - 1);

const [currentCount, previousCount] = await redis.mget(currentKey, previousKey);

const estimated = (Number(previousCount) || 0) * (1 - weight) + (Number(currentCount) || 0);

if (estimated < limit) {

await redis.multi().incr(currentKey).expire(currentKey, Math.ceil(windowMs / 1000) * 2).exec();

return true;

}

return false;

}

The weighting formula estimates the current sliding window's count by taking a proportional share of the previous window's count based on how far into the current window you are, plus the full current window's count so far. It is an approximation, not exact like the sorted-set log approach, but it is accurate enough for the overwhelming majority of real rate limiting needs, and it only requires two simple counters per client rather than a growing set of individual timestamps, which is a meaningfully cheaper storage and computation profile at scale.

Token bucket: the right choice when you want to allow bursts

Everything above assumes you want a hard, roughly even cap on request rate. Token bucket is a genuinely different model, suited for situations where you want to allow short bursts above the average rate as long as the long-run average stays under the limit, which is closer to how a lot of real client behavior actually looks, mostly quiet with occasional bursts of legitimate activity.

async function isAllowedTokenBucket(clientId: string, capacity: number, refillRatePerSec: number): Promise {

const key = 'bucket:' + clientId;

const now = Date.now() / 1000;

const bucket = await redis.hgetall(key);

let tokens = bucket.tokens ? Number(bucket.tokens) : capacity;

const lastRefill = bucket.lastRefill ? Number(bucket.lastRefill) : now;

const elapsed = now - lastRefill;

tokens = Math.min(capacity, tokens + elapsed * refillRatePerSec);

if (tokens >= 1) {

tokens -= 1;

await redis.hset(key, { tokens, lastRefill: now });

return true;

}

await redis.hset(key, { tokens, lastRefill: now });

return false;

}

Each client has a bucket that refills continuously at a steady rate up to some capacity, and each request costs one token. A client that has been quiet can burst up to the full bucket capacity all at once, then has to wait for tokens to refill for further requests, while a client sending a steady trickle of requests below the refill rate never gets throttled at all. This is the model most major API providers actually use publicly, because it matches real usage patterns better than a hard window, letting a client catch up on a batch of queued work without being punished for the burst as long as their sustained average stays reasonable.

What I actually recommend, situationally

For protecting a login endpoint against credential stuffing, where you genuinely want a hard, unforgiving cap with no burst allowance, the sliding window counter is the right fit, cheap and close enough to exact that the boundary exploit is no longer practically meaningful. For a general-purpose API where legitimate clients naturally have bursty usage patterns, batch operations, retries after a network blip, token bucket is the better fit because it does not punish normal, well-behaved burst behavior. I would reach for the exact sliding log with Redis sorted sets only when you have a specific compliance or fairness requirement precise enough to justify the extra memory and computation cost, which in my experience is rarer than people assume when they reach for it by default. And I would never ship a bare fixed window counter for anything actually protecting against abuse, only for rough, low-stakes fairness limits where an occasional double-rate burst at a boundary genuinely does not matter.

One implementation detail that has bitten more teams than the choice of algorithm itself: rate limiting state needs to live somewhere shared if you are running more than one application instance, which almost everyone is once they have any real traffic. An in-memory counter, of any of the algorithms above, works perfectly in local development and then silently fails to enforce the intended limit in production, because each instance behind your load balancer has its own separate counter, effectively multiplying your actual limit by the number of running instances. This is an easy mistake to ship, because it passes every test written against a single running instance, and it only becomes visible under real production traffic spread across multiple servers. Centralizing rate limit state in Redis, as all the examples above do, is not just a nice-to-have for the sliding window and token bucket approaches, it is the actual fix for this specific, easy-to-miss production surprise, and it is worth testing explicitly against a multi-instance setup before trusting any rate limiter is doing what you think it is doing.