What Your Cache Eviction Rate Tells You About Pipeline Health

Your cache eviction rate is like a check-engine light—sometimes it's a loose gas cap, sometimes a cracked head gasket. In production pipelines, a sudden jump in evictions can mean memory pressure, a misconfigured TTL, or just a normal traffic spike. Ignoring it could crash your downstream services. Overreacting and clearing the whole cache might cause a thundering herd. So how do you tell the difference?

This article walks you through the decision: who needs to care, what options exist, how to compare them, and what happens if you choose wrong. No fluff, no fake benchmarks—just patterns from real pipelines.

Who Must Decide on Eviction Policy — and by When

The Owner of the Metric — and Why It’s Often the Wrong Person

Eviction rate lives in a weird no-man’s-land. Production engineers stare at the graph, but they rarely have authority to change the policy. Architects write the policy, but they’ve already moved to the next design doc. So who actually decides? The honest answer is: whoever gets paged at 3 AM when the cache starts thrashing. I have watched a senior engineer waste an entire sprint arguing that LRU (Least Recently Used) was killing their pipeline — only to discover the architect had left the company six months prior. That metric, the eviction rate, sat unowned. Meanwhile, the cache kept throwing out data that the next query desperately needed.

“Eviction rate is the canary. Nobody feeds the canary. They just wonder why it stopped singing.”

— Sarah, production engineer at a mid-stage fintech, during a postmortem I attended

The tricky bit is ownership. Architects tend to view eviction policy as a static design choice — set it and forget it. Production engineers see it as a dynamic signal, something that shifts with traffic patterns, new feature rollouts, and data volume spikes. The gap between those two perspectives is where latency deaths happen. If you're a production engineer, you probably own the *reaction* to eviction rate changes. If you're an architect, you own the *policy* — but only until the next deploy cycle. After that, the code runs on its own, and the eviction rate becomes an operations concern, not a design one.

Signals That Demand Immediate Action — Not Just a Jira Ticket

Not every eviction rate blip means panic. A healthy pipeline might spike to 8% during a batch job and recover. What breaks first is usually latency. When the cache starts evicting hot data, the database takes the hit — queries slow from microseconds to seconds. Error rates follow. And then the OOM killer shows up. That progression is a signature, not a coincidence. I have fixed three separate production incidents where the root cause was a misconfigured TTL, not a code bug — each time, the eviction rate had been climbing for days before anyone acted.

What should trigger a decision? Three things:

• Latency increases by more than 2x at P99 — that pulsing red line tells you the cache is missing its job.
• Application error rate jumps, even briefly — evicted data can cause cascading failures when retries overwhelm the origin.
• OOM kills appear in the container logs — the cache might be using memory to store metadata for evicted items, or the policy itself forces excessive object retention.

The catch is time pressure. Most teams wait until the postmortem to debate policy changes. That's too late. The decision window opens *before* the next deploy — because changing eviction policy mid-cycle on a production cluster, without a rollback plan, is how you turn a small spike into a full outage. Wrong order. Act during the incident, and you risk making things worse. Act after the postmortem, and you lose the week. The right moment is between incident response and the next code push — maybe four hours. Honest experience: you won't get four hours. You will get thirty minutes, a panicked Slack thread, and a senior engineer saying “just use LFU” without checking whether the workload is even reference-friendly.

One rhetorical question worth asking: would you rather decide on eviction policy now, calmly, with data — or during the next page, when your judgment is shot and your coffee is cold? Most teams I have worked with pick the second option by default. That hurts. The decision-maker is whoever shows up first, and the deadline is the moment the latency graph turns red. No pressure.

Three Common Eviction Policies — and What They Cost

LRU: simple, but scan-resistant workloads hurt

Least-Recently-Used is the default for good reason: it’s cheap to implement, requires minimal configuration, and works decently for most pipelines. Every cache implementation I have touched ships with LRU out of the box. The mechanism is straightforward — when space runs out, evict the item that hasn’t been touched in the longest time. That sounds fine until someone runs a full-table scan against your cache. I watched a team at a mid-sized ad platform lose 40% of their cache hit rate in under three minutes because a nightly ETL job touched every key in order. LRU dutifully evicted the real hot data to make room for the scan’s one-hit wonders. The hidden cost: LRU can't distinguish between a genuine popularity spike and a batch process walking through your keys alphabetically. You pay for this in blown-out tail latency and surprised on-call rotations.

The catch is that scan resistance isn’t an optional upgrade — it has to be built in. Some eviction algorithms add a second clock hand or a probationary segment, but vanilla LRU? Not so much. Most teams skip this because they test with synthetic workloads. Then production hits, and the seam blows out. If your pipeline has periodic bulk loads or anyone who says “I’ll just dump it all at once,” LRU will eat your budget alive. That hurts.

LFU: better for skewed access, but memory overhead

Least-Frequently-Used flips the logic: track how often each item is accessed, then evict the coldest by count. For workloads with a clear head-and-tail distribution — think a recommendation engine where 10% of the products get 90% of the views — LFU is the sweet spot. The tricky bit is that “how often” is a surprisingly expensive question to answer. Each key needs a counter, and those counters consume memory even for data that hasn’t been touched in hours. I once consulted on a pipeline where the LFU metadata consumed 22% of the total cache cluster memory. That’s not a rounding error; that’s a 22% capacity tax you don’t see in the dashboard. The counter values also age poorly — a previously hot seasonal item that goes cold still holds a high frequency score, so it clogs the cache for days before being evicted. Most teams fix this by adding a decay factor or a sliding window, but that pushes complexity into the eviction path. Wrong order? Your cache becomes a landfill of once-popular data. Not good.

So when does LFU pay for itself? When your access pattern is so lopsided that the extra memory overhead is dwarfed by the hit rate gain. The hidden cost is operational: you can't tune LFU blind. You need to know your key access distribution before deployment, or you will burn memory on counters that deliver no benefit. That said, for skewed workloads, I have seen LFU push hit rates from 78% to 94% — a genuine win, provided you measure the overhead first.

‘We switched to LFU for our user-activity cache. Hit rate went up 15 points. Then our memory usage went up 18%. The net latency win was a few milliseconds — barely measurable.’

— Senior SRE, e-commerce platform

Flag this for redis: shortcuts cost a day.

TTL-based: predictable, but wastes space on cold data

Time-To-Live eviction doesn’t care about access patterns at all. You set an expiry, and the cache drops the entry when the clock runs out. Predictable. Deterministic. No scan resistance worries, no counter bloat. The catch is that you're promising to hold cold data until its timer expires, even if nobody asks for it. Most teams overestimate how long data stays relevant. I see pipelines with default TTLs of 24 hours when the actual useful lifetime is 90 minutes. The result: the cache fills with cold data that consumes space while hot data gets evicted early because it arrived later. You waste capacity on stuff nobody will ever touch again. That's a hidden cost that compounds under write-heavy workloads — every cache miss forces a recompute, which delays the next write, which pushes the system into a slow-burn death spiral.

What usually breaks first is the TTL policy itself. Engineers set it once during initial design and never revisit it. The data changes, the traffic pattern shifts, but the TTL stays at 86400 seconds forever. I have fixed this exact problem by adding adaptive TTLs based on actual access recency, but that requires per-key tracking — which circles back to the LFU overhead problem. TTL-based eviction is great for session data and short-lived tokens. For anything else, expect to throw away 15–30% of your cache capacity on items that expired before they were ever requested again. That's the trade-off you accept for simplicity.

How to Compare Eviction Policies for Your Pipeline

Hit Ratio vs. Latency vs. Memory Footprint

Pick two. Seriously—that’s the hard truth I’ve watched teams rediscover every quarter. A policy that delivers a 99% hit ratio on paper can double your p99 latency because eviction itself becomes the bottleneck. I once saw a pipeline swap from LRU to LFU, hit ratio jumped from 87% to 94%, and throughput *collapsed* by 40%. The eviction scan on every insert ate cycles the hot path needed. The real measure isn’t “what’s the highest hit ratio”—it’s “what ratio can your pipeline sustain without degrading write latency or blowing the memory budget?” Track all three metrics side by side. The catch is that most dashboards only show hit ratio. You have to instrument eviction CPU cost and per-key memory overhead yourself. That hurts—but only once.

Workload Pattern: Read-Heavy, Write-Heavy, Scan-Heavy

Your eviction policy doesn’t live in a vacuum. It lives in the rhythm of your access stream.

Read-heavy workloads (product catalog, static reference data) love LRU with a large tail—most keys never get touched after the first burst, and you want them gone fast. Write-heavy workloads (event ingestion, session recordings) punish LRU because every new key evicts a recently-written one before it’s ever read. That scenario screams for LFU or a hybrid like TinyLFU (what Caffeine uses). Scan-heavy workloads—bulk exports, nightly report generation—are the silent killer. Each scan floods the cache with one-hit wonders, poisoning the eviction order for hours. What breaks first is the hit ratio for your *real* users. We fixed this by adding a separate scan buffer with its own FIFO eviction, so the main LRU stays clean. That felt hacky. It worked.

‘We picked LFU because it looked better on a spreadsheet. Three days later our CDN bill doubled from cache misses on cold keys.’

— Lead data engineer, mid-stage fintech pipeline post-mortem

The mistake is assuming one policy fits every workload phase. Most pipelines cycle between read spikes (9 AM product launch) and write bursts (midnight batch load). Your policy needs to survive both—or you need dynamic policy switching, which is rare and fragile.

Data Lifetime: Session Cache, Product Catalog, Rate Limits

How long should a key *live*? Not its TTL—its natural lifespan in your business logic.

Session caches (user login tokens, shopping cart state) are ephemeral by design: a session lasts 15–30 minutes, then the key becomes dead weight. LRU evicts dead sessions naturally; LFU keeps them around because they were hit five times during that 20-minute window. That’s wasted memory. Product catalogs, by contrast, change slowly—a SKU might stay active for months. LFU shines here because it protects steady-traffic items from being kicked out by a flash sale on a different product. Rate-limit counters (per-user request counts) are the weird cousin: keys reset every minute, eviction isn’t the problem, but memory churn is. For rate limits, use a fixed-size circular buffer with TTL expiration, *not* eviction policy at all—most teams skip this, then wonder why their Redis memory climbs linearly with active users.

The decision framework boils down to one question: How predictable is your data’s useful life? Predictable (sessions, rate limits) → pick policy by memory footprint, not hit ratio. Unpredictable (catalogs, mixed loads) → optimize for hit ratio, but budget 20% extra memory to absorb policy overhead. That’s the safety margin nobody budgets for until the first production eviction storm. Budget it now.

Trade-Offs at a Glance: Eviction Policy Comparison

LRU vs. LFU vs. TTL: Hit Rate, Memory, Complexity

LRU kicks out the item you haven't touched longest. LFU evicts the item that gets called least frequently. TTL just kills entries after a fixed wall-clock timeout — no tracking, no fuss. The trade-offs hit different parts of your pipeline, and they bite in different ways. LRU gives you decent hit rates for most web workloads, but it chokes on scan-heavy traffic: a single bulk read of ten thousand keys poisons the entire cache, evicting hot items you actually need. LFU avoids that scan problem — frequency counters protect popular content from one-off bursts — but it burns memory storing those counters, and it adapts slowly when user behavior shifts overnight. TTL is stupid-simple: zero overhead per access, no promotion logic. The catch? Your hit rate collapses if you guess the timeout wrong by even 30 seconds.

When to Use Hybrid Approaches (2Q, ARC, Clock)

Pure LRU and LFU each have a blind spot. 2Q — two queues, one for a single pass, one for repeat visitors — catches scan spikes without tanking your hit rate. ARC adjusts its balance between recency and frequency on the fly, which sounds magical until you see it thrash under bursty traffic. I fixed a pipeline once where ARC oscillated between evicting everything then keeping everything, every 90 seconds. We switched to Clock — a simple approximation of LRU with a reference bit — and the hit rate stabilized at 71%. Not perfect, but predictable. The hybrid cost is complexity: you now tune three knobs instead of one, and misconfiguring the ratio between queues can make your eviction rate worse than random.

Most teams skip this part. They pick LRU because Redis defaults to it, then wonder why their cache-miss spike hits 40% every nightly batch job. The real question isn't which algorithm wins a benchmark — it's which one degrades gracefully when your workload lies. Hybrids buy you that grace. The price is debugging time when they don't behave as documented.

'We spent two weeks chasing a phantom memory leak. Turned out ARC was evicting the same keys it just promoted — our access pattern was too cyclic.'

— senior engineer, post-mortem on a pipeline that averaged 23% eviction rate for six months before they looked at the policy

Field note: redis plans crack at handoff.

Workload-Specific Recommendations — and a Trap

Read-heavy dashboard? LRU works fine — most dashboards cycle through the same 200 queries. Content delivery with mixed popularity (some viral, most cold)? LFU keeps the viral items warm, but you'll burn 8 bytes per entry for the frequency counter. Time-series data that streams in and dies after five minutes? TTL is the only sane choice — don't waste CPU tracking something you know will expire. The trap: teams copy a policy from a blog post without checking their own access distribution. I have seen a team run LFU on a cache that rotated keys every three hours — the frequency counters never reached meaningful values, so eviction was essentially random. Their eviction rate hit 68%. They blamed the database.

Measure your working set size first. Sample the request frequency histogram for a week. If 80% of your hits land on 20% of keys, LRU or a hybrid works. If the distribution is flat — every key gets called roughly the same number of times — TTL saves you complexity. Wrong pick here doesn't just waste memory. It amplifies latency spikes, burns database connections, and makes your operations team hate the on-call rotation. That hurts more than a bad algorithm choice ever will.

Implementing Your Chosen Policy — Step by Step

Backward Compatibility: Existing Cache Keys and TTLs

You have decided on a policy. Now the real work begins — and it's rarely a clean install. Most pipelines carry a decade of cached artifacts, each stamped with old TTLs and eviction metadata. What happens when you flip the switch? The seam blows out. I have seen teams deploy LRU into a system built for FIFO; the new eviction logic immediately purged every long-lived key because the access-timestamps were missing. Suddenly, cold cache for eight hours. The fix is boring but mandatory: write a migration script that parses existing keys and assigns them a plausible access-time (creation time minus one second, for instance). Not elegant. But a cache stampede on day one will kill any rollout.

Test the migration against a full production snapshot. Run it in staging with the same key volume. If your new policy relies on frequency counters (LFU variants) and your old keys have none, inject a baseline count of 1 — otherwise, every resurrected key appears as a newcomer and gets evicted immediately. That hurts. A single unplanned eviction wave can spike p99 latency by 300 ms for hours.

Rollout Strategy: Canary, Feature Flag, Gradual Migration

Don't push the new policy to every cache node at once. Instead, use a feature flag that controls which node group uses the new eviction logic. Start with one canary — a low-traffic shard that holds non-critical data. Monitor eviction rate and hit ratio side-by-side. The tricky bit is duration: run the canary for at least one full business cycle (24–48 hours). Why? Because a weekend traffic pattern looks nothing like a Monday morning payroll burst. I fixed a client's pipeline by catching exactly that: their Tuesday rollout looked great until Friday's batch jobs triggered a 4× eviction spike under the new policy.

Gradually expand the flag to 10%, then 50%, then full. Each step requires a rollback exit: if hit ratio drops more than 2%, flip the flag back and investigate. Most teams skip this, assume the policy will work universally, and end up debugging a production incident at 3 AM. Don't be that team.

We rolled LRU to 100% in one deploy. Within an hour, eviction rate tripled and the database connection pool saturated. Lesson learned: canary every cache policy change for at least 24 hours.

— Senior infrastructure engineer, media streaming platform

Monitoring: Eviction Rate, Hit Ratio, p99 Latency

Three metrics matter after deployment. Eviction rate tells you if the new policy is purging too aggressively — a 5× spike means keys are dying before the next read. Hit ratio reveals the real cost: every miss cascades to the database. And p99 latency is the final judge because a 99% hit ratio is worthless if the 1% miss takes 2 seconds. What usually breaks first is the eviction rate: it climbs silently for an hour before anyone notices the latency graph. Set an alert at 2× the baseline eviction rate. Not 1.5× — too noisy — but 3× is already disaster territory.

Watch for oscillating patterns. A sawtooth eviction curve — sharp spike, plateau, then another spike — often means your policy is repeatedly purging the same hot keys. That's a cache stampede symptom. Drop the new policy early if you see it. One last thing: log eviction reasons. Understanding why a key was removed (TTL expiry vs. memory pressure vs. policy decision) turns a black box into actionable data. Your future self will thank you.

What Happens If You Pick the Wrong Policy — or Skip Steps

The thundering herd — when eviction itself becomes the outage

I once watched a pipeline collapse because someone set TTL to 60 seconds on a cache that fed 40 parallel workers. Every minute, every worker found a cold slot simultaneously. They all hammered the origin database at once. That seam blew out in under three seconds — connection pool exhausted, queries queued, then the database just gave up. The cache had done its job too well until it hadn't. What killed us wasn't cache miss rate — it was the synchronization of misses.

You see this pattern often with LRU eviction under uniform access. The policy evicts the least-recently-used entries, and if your workload has periodic bursts across the same key set, every node evicts at roughly the same moment. The herd thunders. The fix is ugly but effective: stagger eviction windows per node, or add jitter to TTLs. Most teams skip this because it complicates the deployment. Then they get paged at 3 AM because Redis CPU spiked to 100% and the application layer started returning 503s.

That said — jitter isn't a silver bullet. If you stagger too widely, older nodes hold stale data while fresher nodes have already evicted. You trade one failure mode for another. The trick is measuring eviction synchronization across your cluster, not just per-instance rates. A 5% eviction rate on each box might hide that all boxes evict the same keys at the same second.

Cache poisoning from stale data — the silent corruption

Wrong policy choice number two: picking LRU when your data has hard freshness requirements but your pipeline doesn't enforce write-invalidation. Imagine a product catalog cache that evicts based purely on access recency. A popular listing stays warm for hours even after the price changes. Every downstream consumer sees the old price. Orders route incorrectly. Customer service gets flooded.

The eviction rate looks healthy — maybe 2-3% — because hot keys never leave memory. But the cache is quietly poisoning every reader. I have seen teams celebrate low eviction rates while their data accuracy metrics were in the gutter. They had optimized for the wrong number. Eviction rate tells you how often you discard data, not whether the data you keep is still correct.

The root cause is usually skipping the step where you classify data by staleness tolerance. Production pipelines rarely have uniform data — some keys expire gracefully, others turn toxic after ten seconds. A single eviction policy applied blindly across all keys is a promise you can't keep. — infrastructure engineer at a payments platform, describing a three-hour outage from cached authorization tokens that shouldn't have been cached at all

Flag this for redis: shortcuts cost a day.

— paraphrased from a post-mortem I read; the fix was per-key TTL override, not a policy swap

Memory pressure and OOM kills — the crash you earned

Here's the pattern nobody admits to: you set maxmemory on Redis or memcached, pick allkeys-lru, and let it run. Eviction rate climbs to 15%, then 25%, then 40%. You think the cache is working — it's aggressively making room. What's actually happening is your working set far exceeds available memory, so the cache spends more CPU on eviction bookkeeping than on serving hits. Response times degrade. Connections time out. Then the OOM killer arrives.

Honestly — that's not a cache failure. That's a sizing failure disguised as an eviction problem. The eviction rate was your early warning, but you read it as a success metric. Most teams skip the step where they model working set size against memory budget. They just tune eviction policies and hope. Hope doesn't survive production traffic at scale.

A concrete fix we applied once: after the third OOM in two weeks, we graphed eviction rate against memory allocation over 24 hours. The correlation was obvious — every time eviction rate passed 30%, memory fragmentation spiked and latency doubled. We doubled the instance memory, set maxmemory-policy to volatile-lru (keys with explicit TTLs only), and the eviction rate dropped to 6%. The OOMs stopped. The lesson: don't treat eviction rate as a knob. Treat it as a symptom. If it's above 10%, your first question shouldn't be "which policy?" — it should be "do I have enough memory for my actual workload?"

Mini-FAQ: Eviction Rate Questions from the Trenches

Should I evict aggressively or conservatively?

Aggressive eviction keeps memory lean—but it punishes every repeat visitor. Conservative eviction hoards cold data—and starves hot keys under pressure. I have watched pipelines where teams cranked LRU to maximum churn because 'free memory felt safe.' The seam blew out: hit rate dropped 14 points, and upstream workers started timing out waiting for recomputation. The catch is that aggression buys you a flat memory graph at the cost of a wildly spiking load profile. Conservative buys you comfort on cache warmth—right up to the moment you run out of heap and the garbage collector locks your pipeline for three seconds at a time. There is no universal answer. What matters is whether you can afford the latency of a cache miss more than you can afford a memory-pressure stall. Measure both under real traffic before you decide.

Why is my eviction rate spiking every hour?

That hourly spike is almost never random. Most teams I have debugged found a cron job—a batch report, a nightly aggregation, a data sync—that sweeps through keys it doesn't own. The spike hits, evictions jump 3x, then the real production traffic suffers because half the warm set was just kicked out. Wrong order. The fix is isolating batch workloads onto a separate cache pool or shifting their TTLs so they expire naturally, not through collision. If the spike is precisely sixty minutes apart, check your TTL distribution first. Second, check whether your eviction policy penalizes the newest entries; that turns a gentle hour-old item into a sudden massacre.

The most dangerous eviction rate is the one that looks normal at minute resolution but hides a cycle of cold-start pain.

— field note from debugging a fraud-detection pipeline that lost 40% of its hot set every hour

How do I distinguish normal churn from a problem?

Track two numbers: eviction rate and miss rate, side by side. A high eviction rate with a low miss rate means your cache is healthy—you're recycling items that would not have been reused anyway. That's fine. A moderate eviction rate with a rising miss rate? That's the early warning. The trend matters more than the absolute number. One false move is watching evictions alone; many dashboards alarm at a 20% eviction rate, but if those evicted items were written once and never re-read, you're not losing anything. First, segregate temporary data from long-lived data in the same cache—that alone halves false alarms. Second, set a threshold that compares evictions to total insertions. Anything above 60% evictions-to-insertions deserves a root-cause chat, not just a resizing.

What if my eviction rate drops to zero?

That sounds like a win. It's often a trap. A zero eviction rate typically means one of three things: your cache is overprovisioned (money down the drain), your TTLs are expiring entries before the cache reaches capacity (so evictions never trigger), or—worst case—your pipeline has stopped writing new data entirely. I have seen all three. The first is wasteful but stable. The second masks that you're paying for storage you don't use. The third is a silent pipeline failure disguised as a perfect metric. The fix: track allocation percentage alongside evictions. If evictions are zero and allocation is below 40%, you're burning cloud spend. If allocation is above 80% and evictions are still zero, congratulations—but verify TTLs are not doing the ejection work for your eviction policy. That's cheating, and it hides a real eviction crisis under expired keys.

No-Hype Recommendation: Which Policy for Your Workload

Read-heavy pipelines: LFU or TTL with long lifetime

If your pipeline serves more cache hits than it writes — think product catalog, reference data, or precomputed aggregates — the eviction policy choice is deceptively simple. I have seen teams slap LRU on a read-heavy feed and wonder why their hit rate craters at 72%. The problem: LRU kicks out items that used to be hot but still get pulled every few hours. Wrong order. For read-dominant workloads, LFU (Least Frequently Used) or a TTL with a long lifetime usually wins — but each carries a trap. LFU tracks frequency counters per key, which chews memory and can starve newer content that suddenly goes viral. TTL avoids that memory tax but forces you to guess the right expiration. Guess too short and you evict data that was still earning its keep. Guess too long and stale records poison downstream joins. The catch: you can't set TTL once and forget it — revisit every quarter as query patterns drift.

One concrete fix we applied on a read-heavy API layer: start with LFU, cap the frequency counter at uint8 (255 max), and let the oldest entries decay by 5% every minute. That decay step — small, but deliberate — prevents the staleness problem without needing a global TTL. Hit rate jumped from 78% to 91% within two days. Not magic; just matching eviction math to actual access distribution.

Write-heavy pipelines: LRU with a large pool

Write-heavy pipelines — ingestion streams, session logs, sensor data — dump new keys faster than readers consume them. Here, LFU collapses because no single key lives long enough to build a frequency count. LRU with a generous pool is the workhorse, but teams routinely undersize the pool. I have watched a team allocate 4 GB to a write-heavy cache that flushed 30 GB per hour. That hurts: each eviction triggers a disk write or a network retry, and soon the eviction rate becomes your bottleneck. The rule of thumb I use: size the pool to hold at least 2× your peak write window — every key written in the last two busy minutes stays resident. That keeps the read-after-write pattern healthy. What usually breaks first is the tail latency on cache misses, not the memory cost itself.

A pitfall specific to write-heavy workloads: eviction storms. When LRU's pool fills up and every new write evicts an old entry, the eviction rate spikes vertically — sometimes 10× steady state. That's a sign your pool is too small, not that LRU is wrong. Add capacity before tuning the policy.

Mixed workloads: adaptive policies (ARC, 2Q, or S3-FIFO)

Mixed workloads — the messy majority — see bursts of both reads and writes, often in unpredictable waves. Static policies choke here. LFU starves new writes; LRU forgets old reads too fast. You need something that adapts. ARC (Adaptive Replacement Cache) and 2Q dynamically balance a ghost list of recently evicted entries to guess whether a read or write pattern is shifting. The trade-off is complexity: tuning ARC's four internal lists under real traffic takes effort. S3-FIFO, a newer contender, offers a simpler two-queue model with lower memory overhead — I have seen it outperform ARC by 8% on workloads with high temporal locality. But no adaptive policy is a silver bullet. A rhetorical question worth sitting with: does your team have the monitoring to spot when the policy starts favoring the wrong pattern? If you can't graph eviction rate per queue, you will tune blind.

“We switched from LRU to ARC and our eviction rate dropped 40 % — but only because our read-to-write ratio shifted every Tuesday afternoon. The policy saved us; the dashboard saved Tuesday.”

— Lead data engineer, mid-size ad-tech pipeline

Start with ARC if your team can invest a week of tuning. Start with S3-FIFO if you need something that works after an afternoon of testing. Start with neither if you lack per-queue metrics — pick LRU with a large pool and add better monitoring first. The best policy is the one your team can measure and adjust.