Thundering Herd

A thundering herd is a failure pattern in which many clients, workers or waiting tasks act at nearly the same time and overwhelm a shared resource. The average request rate may look acceptable before and after the event, but the synchronized burst can exceed capacity by orders of magnitude.

The pattern appears in distributed services, caches, locks, operating systems, message queues, packet systems, telemetry gateways and recovery workflows. Common triggers include cache expiry, service restart, lock release, broadcast wakeup, synchronized retries, batch schedules, leader failover, DNS or connection recovery and clients reconnecting after an outage.

Burst Window

If:

clients or tasks act inside a burst window:

then the burst arrival rate is:

This rate, not the long-term average, controls whether queues, locks, caches, databases or network paths survive the event.

Capacity Condition

For protected capacity:

the burst is safe only if:

Capacity margin during the burst is:

When M_C is strongly negative, the system needs spreading, admission control, load shedding, bulkhead isolation or single-flight behavior before release.

Jittered Spreading

If the same N actors are spread randomly across a jitter window:

the average spread arrival rate is:

This is a first screen. Real arrivals are random, so validation should still check tails and clustering. Jitter reduces synchronization; it does not create capacity.

Cache Stampede

A cache stampede is a common thundering-herd case. Many clients observe the same expired or missing cached value and all regenerate it at once. If regeneration cost is high, the origin store or computation tier can overload.

Single-flight or request coalescing changes the effective regeneration count from:

to:

for one key and one regeneration window, while other clients wait, receive stale data or get a controlled degraded response.

Worked Example

After a short dependency outage, a client fleet retries together. The number of clients is:

Most retries arrive inside:

The burst rate is:

The recovering service can process:

Burst overload is:

The service is not merely busy. It is exposed to more than thirteen times its recovery capacity:

Now spread retries across:

The average jittered retry rate is:

If normal traffic during recovery is:

then total average demand is:

Capacity margin becomes:

The jittered design is still not proof of safety, but it changes the recovery problem from an impossible synchronized burst to a load level that can be tested and protected.

Boundary With Retry Storm

A retry storm is about repeated attempts multiplying load during failure. A thundering herd is about many actors acting at the same time. The two often combine: deterministic retry delays create synchronized waves, and each wave can be amplified by retry count.

A retry budget limits the amount of retry work. Jitter, admission control and load shedding control when that work arrives and whether it should be accepted.

Boundary With Backpressure

Backpressure reacts to downstream pressure and asks producers to slow or reshape work. A thundering herd may arrive too fast for cooperative backpressure to take effect unless producers already implement jitter, retry budgets, queue limits or admission rules.

The practical design should avoid waking every producer at the same instant. Stagger cache refresh, randomize retry delays, cap reconnection attempts, use single-flight regeneration, and protect shared resources with explicit limits.

Validation Evidence

Useful evidence includes burst-size estimates, retry-delay distribution, cache-expiry distribution, reconnect rate, queue depth, lock wait, origin-store load, admitted rate, shed rate, latency percentiles and recovery time.

Validation should deliberately synchronize clients or workers to reproduce the bad case. A steady load test can miss the failure entirely. The acceptance test should show bounded queue growth and recovery without manual traffic draining.

Common Mistakes

Do not validate only average traffic. Do not set identical retry delays across clients. Do not let all cache keys expire at the same absolute time. Do not wake all workers for one unit of work. Do not assume autoscaling can react inside a subsecond herd.

A good thundering-herd review states the synchronized trigger, actor count, burst window, protected resource, spreading method, admission or shedding behavior, and evidence that recovery remains bounded.