The Reality of Traffic Spikes in Modern Web Platforms
When engineering modern web applications, the primary bottleneck is rarely the CPU of your application server. Rather, it is almost invariably the downstream database connection pool and relational query latency under concurrent write and read pressure.
In this article, we examine an end-to-end architecture combining Next.js Server Components, Redis distributed caching, single-flight request deduplication, and atomic cache tag invalidation.
The Problem: Cache Stampedes (Thundering Herd)
Under normal load, cached endpoints return responses within 5 to 15 milliseconds. However, when a high-traffic cache key expires simultaneously for 5,000 active concurrent requests:
// The dangerous naive pattern:
async function getProductData(productId: string) {
const cached = await redis.get(`product:${productId}`);
if (cached) return JSON.parse(cached);
// If 2,000 users arrive here at once, 2,000 DB queries trigger
const fresh = await db.product.findUnique({ where: { id: productId } });
await redis.set(`product:${productId}`, JSON.stringify(fresh), "EX", 300);
return fresh;
}Solution 1: Single-Flight Request Deduplication (Mutex / Flight Group)
By wrapping asynchronous cache misses in an in-flight promise registry, we guarantee that only one query ever reaches the database for any given key. Subsequent concurrent requests simply attach to the single pending promise.
class FlightGroup<T> {
private inFlight = new Map<string, Promise<T>>();
async do(key: string, fn: () => Promise<T>): Promise<T> {
const existing = this.inFlight.get(key);
if (existing) {
return existing;
}
const promise = fn().finally(() => {
this.inFlight.delete(key);
});
this.inFlight.set(key, promise);
return promise;
}
}
export const productFlightGroup = new FlightGroup<Product>();Solution 2: Stale-While-Revalidate with Probabilistic Early Expiration (XFetch)
Instead of hard TTL expirations, we can use the XFetch algorithm (probabilistic early refreshing). As a key approaches its expiration time, requests have an increasing probability of triggering a non-blocking background refresh while continuing to serve the stale cache immediately:
interface CacheEnvelope<T> {
data: T;
ttl: number; // in seconds
computedAt: number; // timestamp
}
function shouldRefreshEarly(computedAt: number, ttl: number, beta = 1.0): boolean {
const delta = (Date.now() - computedAt) / 1000;
const remaining = ttl - delta;
// Probabilistic calculation based on exponential distribution
const random = Math.random();
return -beta * Math.log(random) > remaining;
}