GEAR: Efficient KV Cache Compression for Near-Lossless Generative Inference of LLM

GEAR proposes an efficient KV cache compression framework for near-lossless high-ratio compression, improving system throughput and reducing memory size.