الملخص
Modern neural networks scale predictably, yet the mechanisms behind these regularities remain unclear. Neural redundancy is typically characterized by component importance or representational similarity, both indirect proxies. We view redundancy as an input-conditioned, dynamic relation: intermediate computational states are functionally redundant when they induce similar downstream responses. We introduce Conditional Functional Substitutability (CFS) to directly characterize such functional substitution. CFS exposes functional relations and reduction potential missed by conventional importance- and similarity-based measures. Across modalities and Transformer families, CFS reveals systematic functional reorganization with scale. Controlled scaling further shows that performance gains need not track growth in substitutability, while fixed-capacity models with more independent functional structure perform better, providing a functional account of diminishing returns. Predicted CFS further enables dynamic computation with a better performance--computation trade-off than importance-based component selection, suggesting new directions for redundancy-aware computation and more efficient model scaling.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Chen, J., Li, J., Xiao, Y., Cai, X., Bu, J., Yu, L., & Zhang, T. (2026). Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers. https://omanscience.com/ar/articles/effective-does-not-mean-useful-conditional-functional-substitutability-for-redundancy-and-scaling-in-transformers
MLA 9
Chen, Jiaheng, et al. "Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers." https://omanscience.com/ar/articles/effective-does-not-mean-useful-conditional-functional-substitutability-for-redundancy-and-scaling-in-transformers.
شيكاغو (المؤلف–التاريخ)
Chen, Jiaheng, Jiaxing Li, Yucheng Xiao, Xinyong Cai, Juncheng Bu, Lan Yu, and Tinghe Zhang. 2026. "Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers." https://omanscience.com/ar/articles/effective-does-not-mean-useful-conditional-functional-substitutability-for-redundancy-and-scaling-in-transformers.
هارفارد
Chen, J., Li, J., Xiao, Y., Cai, X., Bu, J., Yu, L. and Zhang, T. (2026) 'Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers', Available at: https://omanscience.com/ar/articles/effective-does-not-mean-useful-conditional-functional-substitutability-for-redundancy-and-scaling-in-transformers.
فانكوفر
Chen J, Li J, Xiao Y, Cai X, Bu J, Yu L, et al. Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers. https://omanscience.com/ar/articles/effective-does-not-mean-useful-conditional-functional-substitutability-for-redundancy-and-scaling-in-transformers
IEEE
J. Chen, J. Li, Y. Xiao, X. Cai, J. Bu, L. Yu, and T. Zhang, "Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers," https://omanscience.com/ar/articles/effective-does-not-mean-useful-conditional-functional-substitutability-for-redundancy-and-scaling-in-transformers.