Rate limiting and caching: start without Redis, scale when needed

When a project starts growing, two problems arrive almost at the same time: someone floods your API with requests (deliberately or by accident), and the database runs the same heavy query over and over. Rate limiting answers the first, caching answers the second. Most people, on hearing these two words, immediately think "so we need Redis". I'll argue the opposite: you can start both on a single Node.js instance, with no external infrastructure at all — and for many projects that stage is enough.

Why these two go together

At first glance rate limiting (protection) and caching (speed) are separate topics. But at their core both answer the same question: do we really have to fully process every incoming request? Rate limiting asks "do we accept this request at all?". Caching says "we accepted it, but do we compute the response again or serve a ready one?". Both save the server's most expensive resources — CPU and the database — from wasted work.

That's why the architecture questions are the same too: where do we keep the state (inside the process or in a shared store), when does it expire, and what happens when the number of instances grows. Understand one properly and the other becomes obvious.

Rate limiting algorithms in plain language

Fixed window — the simplest: "60 requests per minute". A counter opens for each minute; hit 60 and you get a 429. The weakness is at the boundary: if a user sends 60 requests at 11:00:59 and another 60 at 11:01:00, 120 requests pass in two seconds — the limit is effectively broken twice over.

Sliding window — fixes that problem: it counts over a sliding "last 60 seconds" window. More accurate, but you have to store each request's timestamp or work with an approximation formula — slightly more expensive.

Token bucket — my favorite. Imagine every user has a bucket that receives 5 tokens per second, with a capacity of 10. Each request eats one token. Got a token — go ahead; none left — wait. This model matches real traffic best: a user can do a short burst (8 parallel requests when a page opens is normal), but their average rate stays capped. In practice, token bucket covers most API protection cases — knowing the other two mainly helps you understand the settings of off-the-shelf libraries.

Start in-memory: a token bucket with a Map

For a single Node.js instance a plain Map is enough. Here is a fully working Express middleware:

// rate-limit.js
const buckets = new Map();

const CAPACITY = 10;    // bucket capacity (burst limit)
const REFILL_RATE = 5;  // tokens added per second

function isAllowed(key) {
  const now = Date.now();
  let bucket = buckets.get(key);

  if (!bucket) {
    bucket = { tokens: CAPACITY, updatedAt: now };
    buckets.set(key, bucket);
  }

  // refill tokens proportionally to elapsed time
  const elapsedSec = (now - bucket.updatedAt) / 1000;
  bucket.tokens = Math.min(CAPACITY, bucket.tokens + elapsedSec * REFILL_RATE);
  bucket.updatedAt = now;

  if (bucket.tokens >= 1) {
    bucket.tokens -= 1;
    return true;
  }
  return false;
}

// clean up stale buckets — otherwise the Map grows forever
setInterval(() => {
  const now = Date.now();
  for (const [key, bucket] of buckets) {
    if (now - bucket.updatedAt > 60_000) buckets.delete(key);
  }
}, 30_000);

module.exports = function rateLimit(req, res, next) {
  if (!isAllowed(req.ip)) {
    res.set('Retry-After', '1');
    return res.status(429).json({ error: 'Too many requests' });
  }
  next();
};

Note: there is no separate timer per bucket — tokens are refilled "lazily", at request time, proportionally to the elapsed time. These 40 lines run fine in production; the popular express-rate-limit library uses exactly this kind of in-memory store by default.

When in-memory stops being enough

With one instance, everything is fine. Now suppose you start 4 workers in PM2 cluster mode: each worker has its own Map, requests get distributed among them, and the real limit becomes roughly 240 instead of 60. Two servers behind a load balancer — same picture.

Here's the important question: how much does the limit's precision matter? If the goal is plain protection ("one IP must not take down the server"), an approximate limit still does the job — each worker polices its own share, and you can live with that. But if the limit is a product rule — "1000 requests per day on the Free plan" — you now need an exact shared counter, and this is exactly where Redis enters the stage. Notice: Redis is needed here not as a cache but as a shared counter across instances. The rate-limiter-flexible library supports both modes: you start with the memory store, and when needed you swap one constructor parameter to Redis — the rest of the code doesn't change.

HTTP caching: write a header before writing code

The cheapest request is the one that never reaches your server. For that you need a header, not code:

res.set('Cache-Control', 'public, max-age=60, stale-while-revalidate=300');

max-age=60 — the browser and CDN keep the response for 60 seconds and don't contact the server at all during that time. stale-while-revalidate=300 — even after expiry, for the next 5 minutes the stale response is served instantly while a fresh one is fetched in the background: the user never waits.

ETag works differently: the server sends a content hash along with the response, and next time the browser asks with If-None-Match. If the content hasn't changed, the server returns 304 — no body is sent. That saves traffic, but the server still processes the request — so ETag saves bandwidth, not CPU. CDNs like Cloudflare read these same headers: one line, and thousands of requests get answered without ever reaching the server. Just be careful with personal data — use private instead of public, or one user's response may be served to another via the CDN.

Application cache: LRU + TTL

A database query result, an external API response, a heavy computation — you can't cache those with an HTTP header; you need a cache inside the application. Here is a minimal LRU + TTL cache:

class LruCache {
  constructor(maxSize = 500, ttlMs = 60_000) {
    this.maxSize = maxSize;
    this.ttlMs = ttlMs;
    this.map = new Map();
  }

  get(key) {
    const entry = this.map.get(key);
    if (!entry) return undefined;
    if (Date.now() > entry.expiresAt) {
      this.map.delete(key);
      return undefined;
    }
    // LRU: move the used entry to the end
    this.map.delete(key);
    this.map.set(key, entry);
    return entry.value;
  }

  set(key, value) {
    if (this.map.size >= this.maxSize && !this.map.has(key)) {
      // Map preserves insertion order — the first entry is the oldest
      this.map.delete(this.map.keys().next().value);
    }
    this.map.set(key, { value, expiresAt: Date.now() + this.ttlMs });
  }

  delete(key) {
    this.map.delete(key);
  }
}

const cache = new LruCache(1000, 30_000);

async function getProduct(id) {
  const cached = cache.get(`product:${id}`);
  if (cached) return cached;

  const product = await db.products.findById(id);
  cache.set(`product:${id}`, product);
  return product;
}

async function updateProduct(id, data) {
  const product = await db.products.update(id, data);
  cache.delete(`product:${id}`); // event-based invalidation
  return product;
}

There are two safeguards here: TTL bounds how stale the data can get, and maxSize bounds memory — when the limit is reached, the least recently used entry is evicted.

Now the hard part. Cache invalidation is one of the two famously hard problems in programming, and I approach it with three practical strategies. First — live with TTL: if the data can tolerate being 30–60 seconds stale (product lists, statistics, blog posts), do nothing — TTL handles it. Most data actually is like that. Second — delete on events: like updateProduct above, remove the entry from the cache whenever the record changes. Precise, but you must not forget a single write path — so gather your writes into one service layer. Third — both together: delete on events, but keep a TTL anyway. Wherever you forget to delete, the TTL acts as insurance. In production I almost always pick the third.

Next.js ISR is a member of the same family

If you use Next.js, you've already seen all of this. revalidate: 60 is page-level stale-while-revalidate: the old HTML is served instantly while the new one is built in the background. And revalidatePath is exactly event-based invalidation: when content changes, you clear the cache by hand. My own site (Notion CMS + Next.js) works on precisely this principle: a post changes in Notion, the site keeps answering fast from cache, and refreshes in the background. You don't need new theory to understand ISR — it's the same strategies from this article, lifted to the framework level.

Conclusion: measure first, Redis later

Redis is a great tool, but it should be an answer, not a default. My order is this: monitoring first — which endpoint is slow, how many identical queries hit the database, what's the cache hit rate. Without measuring, you won't even know what you gained after adding Redis. Then the in-memory solution: a token bucket on a Map, an LRU + TTL cache, correct Cache-Control headers. And only when a clear signal appears — you need an exact quota across several instances, the cache no longer fits in one process's memory, losing the cache on every deploy has become painful — you add Redis. Even then the code barely changes: the store is swapped, while the logic remains the same token bucket and the same TTL you already understand.