Abstract
While caching key-value (KV) states accelerates Large Language Model (LLM) decoding, this cache can dominate memory usage at long context lengths. One solution is to compress this memory by exploiting inter-layer cache similarities. However, most existing techniques necessitate architectural changes to LLMs and incur substantial overhead. In this work, we propose a symmetry-aware value cache merging strategy that reduces cache memory while avoiding both harmful performance degradation and architectural overhead during decoding. Furthermore, we show that this approach can be exploited alongside existing cache compression techniques, composing with high-ratio quantization or key cache pruning to reach compression ratios that neither method reaches alone, with minimal additional cost. Ultimately, our findings reveal a major source of underutilized capacity in the value cache, offering a simple yet highly effective direction for scaling context windows under memory constraints.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Verma, N., Kim, S., Murray, K., & Duh, K. (2026). VFold: Symmetry-Aware Cross-Layer Value Cache Compression. https://omanscience.com/en/articles/vfold-symmetry-aware-cross-layer-value-cache-compression
MLA 9
Verma, Neha, et al. "VFold: Symmetry-Aware Cross-Layer Value Cache Compression." https://omanscience.com/en/articles/vfold-symmetry-aware-cross-layer-value-cache-compression.
Chicago (author–date)
Verma, Neha, Sungwon Kim, Kenton Murray, and Kevin Duh. 2026. "VFold: Symmetry-Aware Cross-Layer Value Cache Compression." https://omanscience.com/en/articles/vfold-symmetry-aware-cross-layer-value-cache-compression.
Harvard
Verma, N., Kim, S., Murray, K. and Duh, K. (2026) 'VFold: Symmetry-Aware Cross-Layer Value Cache Compression', Available at: https://omanscience.com/en/articles/vfold-symmetry-aware-cross-layer-value-cache-compression.
Vancouver
Verma N, Kim S, Murray K, Duh K. VFold: Symmetry-Aware Cross-Layer Value Cache Compression. https://omanscience.com/en/articles/vfold-symmetry-aware-cross-layer-value-cache-compression
IEEE
N. Verma, S. Kim, K. Murray, and K. Duh, "VFold: Symmetry-Aware Cross-Layer Value Cache Compression," https://omanscience.com/en/articles/vfold-symmetry-aware-cross-layer-value-cache-compression.