الملخص
Indexer-based sparse attention reduces the cost of core attention by passing only a fixed, small number of important tokens to it. However, the indexer must still score the entire KV cache at every decoding step. This scoring overhead becomes a major bottleneck as the context length grows. We propose SPIN (Shadow Predictive Indexer) to reduce this indexer overhead. SPIN uses lightweight, history-based prediction to identify important KV blocks, avoiding the need to score the full KV cache at every decoding step. SPIN treats KV blocks and speculative decoding as first-class design and implementation considerations. Across extensive evaluations on long-context and agentic benchmarks, SPIN achieves 30-40% sparsity while preserving task quality. In end-to-end vLLM serving, SPIN improves output throughput by up to 14.9% and reduces median inter-token latency by up to 13.2%.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Fu, Y., Chang, J., Zhao, R., Long, B., Li, Y., Kamani, M., Jain, S., Raman, R., Safavi, T., Gupta, S., Fashi, P. A., Lee, M., Demouth, J., & Rouhani, B. D. (2026). SPIN: Shadow Predictive Indexer for Sparse Attention. https://omanscience.com/ar/articles/spin-shadow-predictive-indexer-for-sparse-attention
MLA 9
Fu, Yao, et al. "SPIN: Shadow Predictive Indexer for Sparse Attention." https://omanscience.com/ar/articles/spin-shadow-predictive-indexer-for-sparse-attention.
شيكاغو (المؤلف–التاريخ)
Fu, Yao, Jiahan Chang, Ritchie Zhao, Bryce Long, Yueying Li, Mahdi Kamani, Samkit Jain, Rahul Raman, Tara Safavi, Shreya Gupta, Parsa Ashrafi Fashi, Minseok Lee, Julien Demouth, and Bita Darvish Rouhani. 2026. "SPIN: Shadow Predictive Indexer for Sparse Attention." https://omanscience.com/ar/articles/spin-shadow-predictive-indexer-for-sparse-attention.
هارفارد
Fu, Y., Chang, J., Zhao, R., Long, B., Li, Y., Kamani, M., Jain, S., Raman, R., Safavi, T., Gupta, S., Fashi, P. A., Lee, M., Demouth, J. and Rouhani, B. D. (2026) 'SPIN: Shadow Predictive Indexer for Sparse Attention', Available at: https://omanscience.com/ar/articles/spin-shadow-predictive-indexer-for-sparse-attention.
فانكوفر
Fu Y, Chang J, Zhao R, Long B, Li Y, Kamani M, et al. SPIN: Shadow Predictive Indexer for Sparse Attention. https://omanscience.com/ar/articles/spin-shadow-predictive-indexer-for-sparse-attention
IEEE
Y. Fu, J. Chang, R. Zhao, B. Long, Y. Li, M. Kamani, S. Jain, R. Raman, T. Safavi, S. Gupta, P. A. Fashi, M. Lee, J. Demouth, and B. D. Rouhani, "SPIN: Shadow Predictive Indexer for Sparse Attention," https://omanscience.com/ar/articles/spin-shadow-predictive-indexer-for-sparse-attention.