Hedging when the hedges compete for the same service

A hedge cuts the tail when the slow call was bad luck, but when every attempt waits in the same queue it can make the tail far worse: a simulation of @Hedge against one shared service, at loads from 30 % to 95 %.

5 November 2026 · by fabiangftech, author of resilience4cata

The question

After the release candidate was announced, a reader asked how hedging behaves under tail latency when the second call contends for the same downstream. It is the right question. A hedge sends a second request when the first one is slow. If the first one is slow because of bad luck (a GC pause, a cold cache, one slow disk), the second one usually wins and the tail shrinks. If the first one is slow because the service is busy, the second request makes it busier, for every caller.

The hedge page already says not to hedge a saturated dependency, and that the hedge budget "limits the damage, it does not undo it". This article puts numbers on that sentence.

How it was simulated

The question is about queues, not about the cost of the policy, so this is not a JMH benchmark. JMH measures what code costs in a closed loop, where a slow service slows the callers down too and hides the collapse this article is about. Instead, a simulation runs the library's own HedgePolicy and AdaptiveLimitPolicy, through Pipeline, on the testkit's ManualClock, against a simulated service:

• The service: 16 workers, and one FIFO queue shared by every attempt of every call, so a hedge waits behind other callers' first attempts.
• Service time: log-normal with a 10 ms median; 2 % of requests take 100 to 300 ms instead. Each request draws on its own, so a hedge usually does not hit the same hiccup.
• Arrivals: open loop (Poisson), at a rate that would keep the workers busy 30 % to 95 % of the time without hedges. A slow service does not slow the arrivals down, as with real users.
• The hedge: a second attempt after 20 ms, a little above the service's p95.
• Each point: 200,000 calls, the first 10 % not measured. Two seeds gave the same picture; the tables show seed 1.

These are the setups compared:

NO_HEDGE("no hedge", null),
HEDGE_NO_BUDGET("hedge, no budget", b -> b.hedge(h -> h.delay(DELAY).budgetPercent(0))),
HEDGE_DEFAULT_BUDGET("hedge, default budget", b -> b.hedge(h -> h.delay(DELAY))),
HEDGE_BUDGET_5("hedge, budget 5 %", b -> b.hedge(h -> h.delay(DELAY).budgetPercent(5))),
HEDGE_BUDGET_ADAPTIVE("hedge, default budget, adaptive limit", b -> b.hedge(h -> h.delay(DELAY)).adaptiveLimit()),
THREE_ATTEMPTS_NO_BUDGET("3 attempts, no budget", b -> b.hedge(h -> h.delay(DELAY).maxAttempts(3).budgetPercent(0)));

The default budget allows hedges for 10 % of the calls plus 1 per second over 10 seconds. The adaptive limit has its defaults (Gradient2). The one thing the simulation changes is whether a cancelled attempt stops its work on the server.

Note These are the results of a model, deterministic for a seed, not wall-clock measurements. They show how the policies react to load; a real network, real timers and a real server add noise and costs the model leaves out. The code is in examples/src/test/java/dev/resilience4cata/examples/hedgesim, and ./gradlew :examples:hedgeSimulation writes the full report.

When the server ignores the cancellation

This is the usual case for an HTTP call: when one attempt wins, the hedge cancels the other one, but cancelling only stops the client from waiting. The server finishes the work anyway.

p99 / p99.9 latency in ms, by load without hedges:

"Collapse" means the queue never drains: every call waits long enough to send its hedges, so the load doubles (triples with three attempts) and stays there. Latency then grows for as long as the run lasts, here to minutes.

What the numbers say:

• With room to spare, hedging works as promised. Up to 50 % load, the p99.9 falls from 290 ms to 36 ms for 2.2 % more requests, with or without a budget.
• Without a budget, it collapses. Once queueing pushes ordinary calls past the 20 ms delay, they hedge too, and the extra load keeps the queue long. It does not recover by itself: it is a stable state. With the second seed, three attempts collapsed already at 50 %, after a burst of arrivals.
• The budget prevents the collapse, but not the damage. The requests never exceed 1.10 per call. But hedges that fire because of the queue, not because of a hiccup, spend the budget and add load where there is little room. From 70 % load, the tail with the default budget is worse than with no hedge at all. At 95 %, 10 % more load does not fit in 5 % of headroom.
• A smaller budget helps. Here 2.2 % of calls needed a hedge, and 5 % was enough for them. It does better than 10 % at every load and beats no hedge at 70 %, but at 85 % and 95 % it is still worse than no hedge.
• The adaptive limit keeps the tail short. As latency rises, it lowers the number of attempts allowed in flight and turns the others away, first attempts and hedges alike; a rejected hedge also stops further ones. At 95 % load the p99 is 164 ms instead of 237 ms without hedges, but 7.1 % of calls are rejected with AdaptiveLimitFullException, which the caller must handle (a fallback, for example).

When the cancellation reaches the server

With cancellation that reaches the server (gRPC propagates it, for example), a queued loser leaves the queue and one in service frees its worker. The picture changes completely:

Every hedge setup gives the same result, at every load. Cancelling the slow attempt saves the rest of its work: at 95 % load, the workers are busy 74 % of the time instead of 94 %. Hedging takes load off the service here, because most hedges cut short a request that would have kept its worker for 100 to 300 ms.

What to take from it

• Know whether your cancellation reaches the server. If it does, hedging is safe across a wide range of loads. If it does not, hedging needs headroom.
• Keep the budget on. Without it, a burst can tip the service into a state it does not leave. resilience4cata turns it on by default for this reason.
• Size the budget to the hedges you expect. If 2 % of calls are slow by bad luck, a 10 % budget leaves 8 % for hedges caused by queueing, which only add load. budgetPercent is the knob.
• Put an adaptive limit inside the hedge when the service may run hot. It turns overload into fast rejections instead of a long queue.
• Set the delay above the latency that queueing alone causes. A delay close to the median hedges ordinary calls as soon as the service gets busy.

Caveats

• One service with one queue. Real services have several replicas; a hedge sent to a less busy replica can help more than here, and a load balancer that sends it to the same busy one, less.
• The hiccups are independent. When every replica shares the same slow path (a slow database behind them all), a hedge waits twice and never helps; the simulation does not cover that case.
• No network, no real timers, no thread scheduling. A confirmation on real clocks, with a load generator and a server on separate machines, is possible future work.
• Two seeds, one service-time model, one delay. The shape held across both seeds, but the numbers at 85 % and 95 % moved a lot between them (up to three times, for the 5 % budget at 95 %).

Try it

The simulation runs in about a minute:

./gradlew :examples:hedgeSimulation -Phedge.calls=200000 -Phedge.seed=1

To try hedging in your own service, start with the hedge page and the adaptive limit. Questions and results from your own services are welcome in the discussions.