الملخص
Tabular foundation models using in-context learning have recently surpassed gradient-boosted trees on predictive tabular tasks. However, recent mechanistic insights suggest that parameters in these models are largely redundant. We introduce LoopICL, a looped transformer whose core design decouples parameter count from computational depth. LoopICL consists of a single block, processing data through two coupled streams: a cell stream capturing per-cell feature representations and a row stream capturing in-context example representations, jointly refined through within-column and cross-column attention. During pre-training, we vary loop counts, allowing the block to be unrolled for a varying number of iterations at test-time and use a learned exit-gate to automatically exit. In its standard setting, LoopICL performs competitively with TabICLv2 on TabArena and TALENT at the same computational cost (FLOPs), while using nearly 90% fewer parameters. Furthermore, its recurrent design enables users to also trade off inference cost and performance, providing a resource-aware TFM.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Balef, A. R., & Eggensperger, K. (2026). LoopICL: Looping a single transformer block to solve tabular tasks. https://omanscience.com/ar/articles/loopicl-looping-a-single-transformer-block-to-solve-tabular-tasks
MLA 9
Balef, Amir Rezaei, and Katharina Eggensperger. "LoopICL: Looping a single transformer block to solve tabular tasks." https://omanscience.com/ar/articles/loopicl-looping-a-single-transformer-block-to-solve-tabular-tasks.
شيكاغو (المؤلف–التاريخ)
Balef, Amir Rezaei, and Katharina Eggensperger. 2026. "LoopICL: Looping a single transformer block to solve tabular tasks." https://omanscience.com/ar/articles/loopicl-looping-a-single-transformer-block-to-solve-tabular-tasks.
هارفارد
Balef, A. R. and Eggensperger, K. (2026) 'LoopICL: Looping a single transformer block to solve tabular tasks', Available at: https://omanscience.com/ar/articles/loopicl-looping-a-single-transformer-block-to-solve-tabular-tasks.
فانكوفر
Balef AR, Eggensperger K. LoopICL: Looping a single transformer block to solve tabular tasks. https://omanscience.com/ar/articles/loopicl-looping-a-single-transformer-block-to-solve-tabular-tasks
IEEE
A. R. Balef, and K. Eggensperger, "LoopICL: Looping a single transformer block to solve tabular tasks," https://omanscience.com/ar/articles/loopicl-looping-a-single-transformer-block-to-solve-tabular-tasks.