מושגי ליבה
Increasing the cache hit ratio can actually hurt the throughput (and response time) for many caching algorithms, contrary to the intuitive expectation.
תקציר
The paper investigates the counterintuitive phenomenon where increasing the cache hit ratio can lead to decreased throughput for DRAM-based software caches. The authors take a three-pronged approach:
Queueing modeling and analysis: The authors develop queueing models for various cache eviction algorithms, including LRU, FIFO, Probabilistic LRU, and CLOCK. The analysis provides an upper bound on throughput as a function of the hit ratio.
Simulation: The authors simulate the queueing models to obtain the exact throughput as a function of hit ratio. The simulation results match the implementation within 5%.
Implementation: The authors implement a prototype of the caching system and measure the throughput. The implementation results align closely with the simulation.
The key insights are:
For LRU, when the hit ratio is high, the delink operation becomes the bottleneck, leading to longer delays and lower throughput.
For FIFO, increasing the hit ratio always improves throughput, as the bottleneck is in the miss path, not the hit path.
For Probabilistic LRU, the behavior depends heavily on the probability parameter 'q'. Only when 'q' is extremely high (close to 1) does the algorithm exhibit FIFO-like behavior where increasing hit ratio always helps.
The phenomenon of throughput decreasing at higher hit ratios is likely to be more pronounced in future systems with faster disks and higher numbers of CPU cores.
סטטיסטיקה
The mean disk latency E[Zdisk] is 100 μs.
The mean cache lookup time E[Zcache] is 0.51 μs.
The mean delink time E[Sdelink] is 0.7 μs.
The mean head update time E[Shead] is 0.59 μs.
The mean tail update time E[Stail] is less than 0.59 μs.
ציטוטים
"What if increasing the hit ratio actually hurts performance?"
"Increasing the hit ratio can actually hurt the throughput (and response time) for many caching algorithms."