الملخص
State Space Models (SSMs) compress sequence history into a bounded recurrent state, making the resulting memory law a central architectural choice for long-context performance. Most modern SSMs rely on ODE-based dynamics that lead to exponential forgetting, limiting their ability to retain information over broad temporal ranges. We introduce FRAC, a selective SSM architecture derived from fractional dynamics that replaces this exponential decay with power-law long memory. To make fractional dynamics practical, FRAC approximates the heavy-tailed target kernel with a finite-state, log-spaced sum of exponential modes. This construction turns fractional memory into an efficient recurrent module with parallel training and prefill, while retaining bounded-state autoregressive decoding. Extensive experiments, including 1.3B-parameter language modeling, demonstrate that FRAC consistently improves long-context performance over state-of-the-art SSM baselines while staying competitive on short-context. These results show that fractional dynamics provide a practical and effective prior for long-context SSMs.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Kobyzev, I., Ghaddar, A., Nasiri-Sarvi, A., Shang, L., & Cui, Y. (2026). Fractional State Space Transition for Long Sequence Modeling. https://omanscience.com/ar/articles/fractional-state-space-transition-for-long-sequence-modeling
MLA 9
Kobyzev, Ivan, et al. "Fractional State Space Transition for Long Sequence Modeling." https://omanscience.com/ar/articles/fractional-state-space-transition-for-long-sequence-modeling.
شيكاغو (المؤلف–التاريخ)
Kobyzev, Ivan, Abbas Ghaddar, Ali Nasiri-Sarvi, Lifeng Shang, and Yufei Cui. 2026. "Fractional State Space Transition for Long Sequence Modeling." https://omanscience.com/ar/articles/fractional-state-space-transition-for-long-sequence-modeling.
هارفارد
Kobyzev, I., Ghaddar, A., Nasiri-Sarvi, A., Shang, L. and Cui, Y. (2026) 'Fractional State Space Transition for Long Sequence Modeling', Available at: https://omanscience.com/ar/articles/fractional-state-space-transition-for-long-sequence-modeling.
فانكوفر
Kobyzev I, Ghaddar A, Nasiri-Sarvi A, Shang L, Cui Y. Fractional State Space Transition for Long Sequence Modeling. https://omanscience.com/ar/articles/fractional-state-space-transition-for-long-sequence-modeling
IEEE
I. Kobyzev, A. Ghaddar, A. Nasiri-Sarvi, L. Shang, and Y. Cui, "Fractional State Space Transition for Long Sequence Modeling," https://omanscience.com/ar/articles/fractional-state-space-transition-for-long-sequence-modeling.