Why Your P99 Looks Fine but the Tail Still Hurts Production

You stare at the dashboard. P99 latency: 200 ms. Looks fine. But your pager just went off. A downstream service reports connection pool exhaustion. Users complain about a spinning wheel that only happens sometimes. That's the tail.

Here's the reality: P99 hides the worst 1% of requests. If you serve 10,000 requests per second, that's 100 requests every second that take longer than 200 ms. In a 10-second window, that's 1,000 slow requests. Enough to cascade. This article walks through why your P99 lulls you into false confidence, and what to actually track.

Where Tail Latency Bites in Practice

Payment processing: one slow call holds up the entire checkout flow

A customer clicks 'Place Order' and waits. Behind that button, a chain of services fires: auth, fraud check, inventory hold, payment gateway. I have seen this exact setup—P99 at 400ms, dashboards green, alerts silent—while users were abandoning carts at 12%. The median checkout took 180ms, but one payment gateway call, the one hitting a degraded upstream, stretched to 2.3 seconds. That single slow leg blocked the entire transaction. The rest of the services had already finished; they just sat there, threads parked, waiting on the straggler. Worse, the gateway didn't timeout—it eventually succeeded. So the P99 stayed clean. The P99.9, the one nobody graphs? It told the real story. But teams don't watch P99.9 because it looks noisy. Noise that signals lost revenue.

The catch is that checkout flows rarely retry on their own. They hold a lock—order reservation, inventory deduction—and the slow call pins that lock open. Other customers hitting the same product? They get 'item unavailable' or a spinning wheel. One slow payment call can crater product availability lookups across the entire catalog. That looks like a caching bug. Or a database lock. Teams chase those ghosts for days while the real culprit sits in a single outbound HTTP client with a default timeout of 5 seconds. We fixed this at a previous company by breaking checkout into two phases: synchronous minimum (auth + fraud) and deferred async for the rest. The tail didn't disappear. But the user stopped waiting on it.

Ad serving: tail latency means lost revenue from timeouts

Ad exchanges run on 100-millisecond deadlines. Cross that line and your bid simply vanishes—no second chance, no partial credit. I once consulted for a supply-side platform where the average bid response was 45ms and the P99 was 92ms. Looks fine. Then they mapped revenue against response-time bins and found that bids arriving after 70ms won 40% fewer auctions. The platform's own timeout was 120ms, but the exchange's effective cutoff was tighter—something buried in their SLA fine print. The P99 masked a cliff: 99% of bids made it under 100ms, but the 1% that didn't? Those were the high-value ones, the bids that needed extra computation for audience segmentation. The cheap bids flew through. The expensive ones got dropped. That's not a latency problem—that's a revenue leak dressed up as a tail metric.

'We were optimizing for the wrong percentile. The exchange didn't care about our P99. It cared about the 99th bidder in a 30-bidder auction.'

— engineering lead, SSP postmortem

The fix is not just faster code. It's priority-aware scheduling: expensive bids get more CPU time earlier, or they get pre-computed and cached against user cohorts. Most teams skip this because it sounds like premature optimization. Then they watch a 3% timeout rate slice their margins in half. The trade-off is complexity—you trade a clean, stateless bidder for a stateful scheduler with backpressure. But if your revenue lives on the edge of a strict deadline, the complexity pays for itself in two weeks.

Microservice cascades: one slow service degrades others

Service A calls B, B calls C, C calls D. D is having a bad Tuesday—disk compaction, maybe a noisy neighbor. D's responses drift from 5ms to 350ms, but only for 0.8% of requests. B's thread pool starts accumulating these slow calls. Ten threads stuck waiting on D. Then twenty. Then B's thread pool saturates and starts queueing. Now every call to B, even the fast ones, has to wait in line. Latency that was 10ms becomes 200ms just from queuing. A sees B slowing down and opens more connections—aggressive retry logic. That makes B's queue even longer. The tail in D, a service with a 0.8% slow rate, becomes a 60% timeout disaster at A's edge. The dashboard shows A's P99 at 2.1 seconds, and everyone blames A. But A was just the victim.

What usually breaks first is the shared resource: connection pools, thread pools, database connections. These are invisible in standard latency histograms. You can have perfect P99 across all services and still cascade into a deadlock because one pool exhausted its slots waiting on a single slow peer. The fix is brutal but necessary: separate thread pools for upstream and downstream calls, impose strict per-call timeouts (not per-service timeouts), and implement circuit breakers that trip on P99.9 degradation, not just error rate. That last part trips up most teams—they set circuit breakers for 50% errors, but a service can be catastrophically slow without returning a single error. Slow is not an error in most monitoring. It should be.

Foundations: What Most Engineers Get Wrong

Tail vs. Jitter: Not the Same Thing

I once watched a team celebrate p99 dropping under 50ms. Their dashboard looked clean — green bars, happy alerts. Production still hurt. Customers complained about timeouts every few minutes. The problem? They were measuring the wrong thing. Tail latency is not jitter. Jitter is a measurement of variation between consecutive packets or requests. Tail latency is the worst-case behavior experienced by the unlucky few — the requests that land on saturated threads, garbage-collecting nodes, or slow disks. Most engineers conflate the two. They slap a p99 threshold on a Grafana panel and call it done. That misses the point entirely.

Averaging Percentiles Hides Spikes

P99 hides the horror. Consider a service that handles 10,000 requests per second. Your p99 sits at 80ms. Looks fine — except that p99 averages the top 1% across a rolling window. A burst of 200ms latencies gets smoothed out by the thousand quiet milliseconds around it. The catch? The tail is not uniform. That 200ms spike might hit the same database shard, the same upstream timeout, the same GC pause. Users on that shard see repeated failures. Your dashboard shows a gentle curve. This is why teams ship a shiny graph to stakeholders and still get paged at 3 a.m. — the average hides the poison.

“Averaging percentiles is like measuring a river’s depth by its surface. You miss the rocks that tear the hull.”

— paraphrased from a production post-mortem I sat through, 2023

Why p99.9 Is Often More Important Than p99

Run the numbers on a typical checkout flow. p99 at 120ms feels safe. Your p99.9, however, hits 1.4 seconds. That’s the request that fails a payment gateway timeout — or the one that keeps a user staring at a spinner while their cart expires. In a system with 100,000 daily checkouts, p99.9 punishes 100 people every single day. Those 100 people tweet. They open support tickets. They churn. The practical trap: chasing p99.9 amplifies every noisy neighbor in your system. A cold start, a cache miss, a network retransmission — each blip gets magnified. Teams often revert because p99.9 optimization feels like fighting ghosts. One day the fix works, the next a new library version reintroduces the same spike. That’s not a reason to ignore it — but it’s a reason to measure differently. Bury p99 in your team dashboard. Surface p99.9 to your on-call rotation. Let the tail tell you who is bleeding.

The tricky bit is that p99.9 optimization demands a different mindset. You stop asking “what’s the average?” and start asking “what does the worst 0.1% share?” Often it’s a single resource: a contended lock, a misconfigured timeout, a hotspot partition. I fixed one by moving a background batch job off the same CPU core as the request handler. Two lines of config. Tail dropped 400ms. The team had been tuning connection pools for weeks. Wrong order. Start with the seam, not the surface.

Patterns That Tame the Tail

Request Hedging: Send Duplicate Requests to Multiple Replicas

You wait 200ms for a single replica. Meanwhile, another idle replica could have answered in 10ms. That gap—that unnecessary wait—is where the tail hides. Hedging solves this by firing the same request to two or three replicas simultaneously, then taking the first response and cancelling the rest. I have seen teams at a mid-size ad platform drop their P99.9 from 1.2 seconds to 340ms simply by hedging reads on their object-store tier. The catch is cost: you now do 2x–3x the work for the same user action. That sounds fine until your traffic doubles and your infrastructure bill spikes.

Worse—hedging can amplify system load during cascading failures. If one replica is slow because it's overloaded, sending extra requests to other replicas that are also strained can trigger a pile-on. Smart hedging uses a small initial delay: send the primary request, wait a few milliseconds, then send the hedge only if no response arrived. This avoids redundant work when the primary is fast. The trade-off is real: you trade compute dollars for latency stability. Most teams I have seen start with a hedge-after-5ms policy for read paths and scale back when costs exceed 15% overhead.

One more pitfall: hedging against a shared bottleneck. If all replicas hit the same overloaded database shard, duplicate requests just compound the misery. Hedging works best when replicas are truly independent—different availability zones, separate connection pools, distinct cache lines. Otherwise you're just shouting louder into a crowded room.

“The fastest response wins, but the cost is every replica that lost the race. Hedge only when the tail hurts more than the bill.”

— SRE lead, large CDN provider, internal postmortem

Timeout Tuning: Adaptive vs. Fixed Timeouts

Most engineers set a fixed timeout—say 500ms—and call it done. That works until a brief network blip pushes latency to 480ms and your service starts timing out healthy requests. Fixed timeouts are brittle. Adaptive timeouts, in contrast, adjust based on recent latency distributions. The idea is simple: measure the p99 of recent successful requests (last 30 seconds), then set your timeout to 1.5x that value. When the system is healthy, timeouts shrink. When a hiccup occurs, the timeout stretches—but never beyond a hard ceiling.

The tricky bit is choosing the window. Too short (5 seconds) and transient spikes cause tight timeouts that cascade. Too long (5 minutes) and you mask real degradation. I have landed on a 30-second sliding window with a 2x multiplier as a sane default, then tuned from there. One team I worked with forgot to cap the hard ceiling—their p99 crept to 3 seconds, the adaptive timeout hit 4.5 seconds, and user-facing requests piled up waiting for slow responses. The fix: a hard limit at 1 second, adaptive below that only.

What usually breaks first is the interaction between hedging and timeouts. If your hedge fires after 5ms but your timeout is 500ms, the primary request might still be in-flight when the timeout fires—cancelling both the primary and the hedge response. That hurts. Sequence matters: cancel the primary immediately once the hedge wins, and let the timeout kill only stragglers that haven't been hedged yet. A common tactic is to set the per-request timeout to half the client-facing timeout, so the first response arrives before the user gives up.

Queue Sizing: Controlling Admission to Prevent Overload

Queues hide latency—until they amplify it. Picture this: a service with a 1000-item queue suddenly receives 1500 requests. The first 1000 enter the queue, the next 500 get rejected. Those 1000 queued requests now wait, on average, for the processing time of 500 items ahead of them. That's a 500ms delay added to every queued request. The tail blows out not because the service is slow, but because it accepted work it could not start. Admission control—rejecting excess requests early—keeps the queue shallow and the tail tight.

Most teams skip this: set a hard queue depth of 10–20 items per worker thread. When the queue fills, return a 503 or a retry hint to the caller. This is brutally simple. No adaptive magic. Yet I have watched three separate projects revert to unbounded queues because “rejection feels wrong”. The alternative? A small, fast queue that protects the service's head-of-line blocking, paired with client-side retry with jitter. That pattern keeps system throughput stable and the tail predictable.

The trade-off is throughput. Rejecting requests means you lose work—but you keep the work you do accept fast. In a system where users retry, a fast rejection (10ms) beats a slow timeout (2s) every time. One trick: instrument the queue depth as a gauge metric and alert when it exceeds 80% of capacity. That's often the first sign of an impending tail spike, hours before users complain.

Anti-Patterns: Why Teams Revert

Over-hedging: doubling load without benefit

The most painful rollback I ever witnessed happened two hours after a team deployed probabilistic hedging to every read path. P99 dropped by 40 milliseconds — beautiful. Then the database connection pool collapsed. The team had hedged requests that were already sub-millisecond, firing duplicate queries for operations that never tailed. The result? Twice the read load, one dead replica, and a production incident that erased any latency win. That sounds fine until you realize the hedging logic treated all latencies equally. Wrong move.

The trap is seductive: if some hedging helps, more hedging must help more. But every duplicate request consumes resources — CPU, disk I/O, network buffers — that could serve other users. Most teams skip this: measuring the effective load multiplier before enabling hedging at scale. I have seen services where aggressive hedging created self-inflicted congestion collapse. The replicas got so busy processing hedged copies that primary requests queued behind them. Your p90 actually worsens. The fix? Run a small canary with and without hedging, measure both tail latency and request throughput per core. If throughput drops more than 5%, you're over-hedging.

Static timeouts that don't adapt to traffic changes

Hard-coded timeouts feel like safety rails — until traffic shifts and those rails become prison bars. Consider a service that sets a 200ms timeout on all external calls. At 9 AM, when traffic is light, the tail sits at 80ms. Fine. At 2 PM, a sudden spike in requests pushes the natural 95th percentile to 190ms — still under the timeout. But now the 99.9th percentile jumps to 210ms. Every one of those requests gets cancelled, triggering retries from upstream callers. The retries pile on, and the tail blossoms into a full-blown timeout storm. The team reverts the "optimization" because production feels worse, even though the timeout itself never changed.

The catch is that static values bake assumptions about traffic distribution into your infrastructure. Those assumptions rot. What usually breaks first is the coordination overhead: each timed-out request wastes connection state, memory for response buffers, and the cost of error handling on both sides. I fixed one system by swapping a static 150ms timeout for a percentile-based adaptive timeout — it watches the rolling p99 of response times and sets the timeout to p99 × 1.5, re-evaluated every 30 seconds. That single change stopped the rollback cycle. The team finally kept the timeout enabled because it breathed with the traffic, not against it.

Ignoring the coordination overhead of hedging

Hedging isn't free — you pay in coordination. Every duplicate request needs deduplication logic on the receiving end, or you process the same work twice. I watched a team deploy hedged reads to a cache layer that had no idempotency guarantees. The first copy decremented inventory. The second copy, arriving 8ms later, decremented inventory again. The result? Overselling by 12% before anyone noticed. That revert hurt because the latency win was real — the cost just showed up in a different metric.

'We shaved 30ms off the tail, and then the nightly reconciliation job failed for the first time in two years.'

— Site reliability lead, describing the moment they reverted a hedging rollout

The coordination overhead compounds when hedged requests hit different partitions, different datacenters, or different consistency levels. A simple unique-request-id check works for one call, but fails when the hedged copy reaches a node that hasn't replicated the first write yet. Now you have two conflicting states, and the reconciliation logic becomes a distributed transaction. That complexity is why teams revert: the mental model breaks, and the operational cost exceeds the latency improvement. Honest question — is a 15ms tail reduction worth weekly on-call pages for consistency bugs?

Maintenance and Drift: The Hidden Cost

Tail Latency Shifts as Traffic Patterns Change

You ship the fix. P99 drops from 200ms to 45ms. Everyone high-fives. Three weeks later the same endpoint starts throwing 500s at 3 AM. What changed? Not your code — your traffic pattern. A new client rolled out aggressive pre-fetching, doubling request concurrency during off-peak hours. Your carefully tuned connection pool now queues requests differently, and that once-eliminated 99.9th percentile spike is back. Most teams skip this: tail latency is a moving target because the load that triggers it's also moving. I have watched teams burn two sprints cutting P99 from 120ms to 40ms, only to see it creep back to 90ms within a month — not because the optimization degraded, but because the request distribution shifted.

The catch is that standard dashboards hide this. You look at the p99 line over seven days, see it flat, and call it stable. Meanwhile the tail is thrashing inside that average. One team I worked with used a heatmap of latency by request size — small payloads were fine, but a new mobile SDK suddenly sent 40% larger payloads. That tiny drift in distribution resurrected a tail problem they thought they had killed.

Code Changes Can Reintroduce Tail Spikes

A single innocent commit. Someone adds a synchronous DNS lookup in a hot path, or a junior engineer imports a logging library that blocks on flush. You won't catch it in code review because the diff is three lines. But the tail remembers. The worst example I encountered was a cache key change: a team swapped SHA-256 for a faster hash, but the new implementation used a thread-local buffer that was not cleared between calls. Under load, stale data poisoned lookups, causing cascading retries. P99 jumped from 50ms to 3 seconds — and the commit passed all unit tests.

That sounds fine until you realize your deployment pipeline has no latency regression gate. Most CI systems test throughput, not tail behavior. So the spike ships to production on a Tuesday, users complain on Wednesday, and by Friday someone is reverting the change while arguing it "couldn't be the issue." Wrong order. The real cost is not the fix — it's the three days of degraded user experience and the context switch for the on-call engineer. Honestly, you can design the best tail-taming architecture in the world, but one careless line can undo it.

Continuous Monitoring of Tail Metrics Requires Investment

You need more than a p99 line on a dashboard. You need per-percentile histograms sliced by service, by endpoint, by payload size, by client version. That's not free. Storage costs go up. Query latency on your metrics backend increases. Teams often revert to high-level aggregation because the detailed view becomes too expensive to keep. Trade-off alert: you can either pay for the data or pay for the outages.

'We had seventeen latency dashboards and still missed the spike because we were averaging across all customers.'

— Senior SRE, post-mortem for a streaming platform outage

What usually breaks first is retention: teams keep seven days of fine-grained histograms and thirty days of rolled-up p50/p99. That thirty-day gap hides slow drifts. A 2% increase in tail latency per week is invisible at that granularity — until week eight when your p99 crosses the threshold you promised the business. The fix is not just better tooling. It's a rotation: one engineer per sprint owns tail metrics as a *live concern*, not a post-mortem artifact. They watch the distribution shift, correlate it with recent deploys, and flag regressions before users feel them. That costs engineering time. But compare it to the cost of a production incident that wakes up four people at 2 AM — the investment looks cheap.

Next: when you shouldn't chase the tail at all. Because sometimes the smartest move is to let it burn.

When You Shouldn't Chase the Tail

When the budget doesn't match the blast radius

Last year I watched a team spend six weeks rewriting their caching layer — all to shave 300 microseconds off a P99 that was already sitting at 1.2 milliseconds. Their actual production fires? A 30-second query timeout in the billing service that hit every Friday afternoon. The tail fix felt good. The billing problem was eating revenue. That gap — between what feels like engineering excellence and what actually protects the business — is where chasing the tail becomes a tax, not an investment.

When p99 is already sub-millisecond

Some systems genuinely need sub-100µs P99s. High-frequency trading, real-time video pipelines, certain ad exchanges. Yours probably isn't one of them. If your P99 is already under a millisecond and your SLO is 99.9% at 10ms you're optimizing past the point of diminishing return. The tricky bit is admitting that. Engineers love a tight loop. I have seen teams sink two months into kernel-bypass networking for an API that served a dashboard used by twelve internal people. The real p99 of user frustration was the render time in their browser — three seconds, untouched.

Here is the trade-off: every hour you spend on tail optimization for an already-fast endpoint is an hour you're not fixing the 3-second p90 that actually loses users. The catch is that sub-millisecond work feels like mastery. The 3-second mess feels like debt. You have to fight the ego.

When hardware anomalies dominate the tail

What about GC pauses? Or a noisy neighbor container stealing your CPU? Or the network switch that drops packets every 90 minutes like clockwork? You can tune your application logic until your fingers bleed — the tail will still belong to the garbage collector. Most teams skip this: they treat a hardware-driven tail as a software problem. It isn't.

You can't code your way out of a hardware problem. You just buy yourself a nicer view of the crash.

— senior SRE, after three weeks of kernel parameter tuning

If your P99 spikes correlate with GC logs, or with cron jobs on the same host, or with traffic shifts that trigger autoscaling — fix the infrastructure first, not the code. One concrete anecdote: a team spent two months rewriting their hot-path handler to avoid allocations. The tail barely moved. Turned out a single misconfigured NUMA binding was causing remote memory access on every third request. They fixed it in an afternoon. The tail dropped by 40%. Wrong order, massive cost.

When the business impact is invisible

Honestly — not every tail matters. If your endpoint serves a batch job that runs overnight and completes at 4:02 AM instead of 3:58 AM, the business doesn't care. If the p99 tail of your email-sending service is 12 seconds but the user already closed the tab, you're optimizing a ghost. The question to ask: does this latency cause a user to leave, a transaction to fail, or a cost to spike? If no, walk away.

What usually breaks first is the team's discipline. They optimize because the chart is ugly, not because the business hurts. I have seen teams revert optimized code after six months because the maintenance cost (special-cased serialization, pinned JVM flags, fragile thread pools) outweighed the latency savings. The seam blows out when the engineer who built it leaves. Then the next person inherits a mysterious code path that saves 200 microseconds but breaks every deploy.

So when should you stop? When the effort to shave the next microsecond exceeds the effort to keep the system running — or when the tail you're chasing belongs to something else entirely. Next time you see a P99 spike, check the hardware, check the business logic, check your ego. One of those three is lying.

Open Questions: Can You Ever Kill the Tail?

Is zero tail latency even possible?

Short answer: no. Longer answer: chasing absolute zero is how you burn budget and goodwill. I have watched teams spend three months shaving 2ms off their P99.9 while their P50 drifted up by 4ms because the fix added queueing overhead. That's a bad trade. The tail is not a knob you zero out — it's a distribution with physics in it. Network hops, garbage collection cycles, kernel scheduler jitter: these are not bugs, they're features of the hardware. You can compress them, hedge against them, or reroute around them, but you can't erase them. The question is not can we kill the tail but which slice of the tail matters enough to fight.

How to decide which percentile to optimize

Most teams default to P99 because dashboards show it. That's lazy. The real decider is your user-facing contract. A search autocomplete that returns in 200ms P50 but 900ms P99.9 — users feel the slow ones as stutter, so you fight the P99.9. A batch payment processor? P99.5 might be fine; the system retries anyway. The trap is optimizing a percentile nobody experiences. I once saw a team obsess over P99.99 on a metrics pipeline that fed a dashboard refreshed every sixty seconds. The seam blew out because they ignored P50 latency during a cache stampede. Choose your percentile by asking: what does the slowest request that actually annoys a human look like? That number is rarely the one in your monitoring tooltip.

When to accept the tail as irreducible

There is a point where further cuts cost more than the outage itself. The catch is — most teams discover this after they have already shipped the over-engineered solution. Things that make the tail irreducible: shared tenancy, cold-start functions, physical distance. You can't geo-distribute your way out of a law of physics. A good heuristic: if your P99 is below your business timeout and the engineering effort to cut it further equals more than the annualized cost of those slow requests — stop. Put that energy into circuit breakers or client-side retry logic instead. That sounds boring. It's also how you keep your pager quiet. The tail is a fact; your response to it's a choice.

“We spent six months making our P99 bulletproof. Then a DNS resolver hiccup took us down for twelve minutes.”

— Staff engineer, after a postmortem that shifted the team from tail optimization to fault isolation

What usually breaks first is not the long tail — it's the unexpected spike that bypasses your percentile entirely. I keep a running list of incidents where the team that killed the P99 tail got blindsided by a P100 timeout caused by a dependency they forgot to model. The honest next step: instrument for shape, not just rank. Track full distributions, not buckets. Your P99 can look fine while your p50 creeps up week over week. That drift is the tail you should worry about — it's the one that grows while you stare at the far end. Next time you debate which percentile to optimize, run an experiment: measure the gap between your P99 and your actual timeout. If that gap is wider than your sprint cycle, shift your attention. The tail you can't kill is the one you stop feeding.

Next Experiments for Your System

Start measuring p99.9 and p99.99

You're flying blind if your highest resolution is p99. I have debugged production fires where p99 looked clean — under 50ms — while p99.99 was sitting at 14 seconds. That was the difference between one customer hitting a cold cache path and dozens silently rage-quitting. The fix isn't complicated: instrument every percentile bucket from p99 up through p99.999. Most APM tools let you set custom percentiles. Do it today. The catch is cost — higher-percentile data generates more metrics, and your observability bill might spike. Start with a two-week sampling window on one critical endpoint. You will likely find a shape you weren't expecting: a sudden cliff at p99.9 where a single lock contention pattern eats 300ms. That shape is real. That shape is the tail that hurts.

Run chaos experiments with injected latency

Stop guessing. Inject 50ms of artificial jitter into one random request per minute and watch what breaks. I ran this on a payment service — within an hour three downstream timeouts cascaded into a full retry storm. The team thought they had circuit breakers. They had settings that looked correct. Wrong order. The experiment revealed that the breaker opened at 5 seconds, but the upstream client timeout was 4.5 seconds — so requests piled up before the safety mechanism kicked in. Cheap fix, expensive lesson. Run this on your own systems: pick Tuesday morning, inject 100ms of latency into a core database read path, measure the ripple. Most teams skip this because it feels reckless. Honestly — the production chaos is already happening; you just haven't timed it yet.

'We added hedging to one service and p99 dropped 40%. Then we forgot to monitor the extra load. Two weeks later the database crashed.'

— SRE lead, after a post-mortem I attended

Implement a single hedging pattern and measure the trade-off

Hedging — sending two requests in parallel and keeping the fastest response — is the single most effective pattern for tail variance. But it comes with a hidden price: amplified load. One team I worked with hedged every read request to their primary cache cluster. P99 looked beautiful. Then the cluster saturated at 2x normal traffic and the whole thing collapsed. The fix was to hedge only requests that had already waited 200ms — a small gate that catches the outliers without doubling baseline pressure. That sounds fine until the gate itself becomes a concurrency bottleneck. The pattern works, but you need to measure the tail of the hedging mechanism. Did your p99 improve? Good. Did your p99.9 of the gate logic blow out? Check that. If it did, you traded one tail for another. Run this experiment for one week, compare p99.9 of hedged vs unhedged paths, and watch the median shift upward. Some degradation at the median is acceptable if the p99.999 collapses from 8s to 400ms. But you have to measure that trade-off — nobody will hand it to you.