Abstract
Modern neural networks scale predictably, yet the mechanisms behind these regularities remain unclear. Neural redundancy is typically characterized by component importance or representational similarity, both indirect proxies. We view redundancy as an input-conditioned, dynamic relation: intermediate computational states are functionally redundant when they induce similar downstream responses. We introduce Conditional Functional Substitutability (CFS) to directly characterize such functional substitution. CFS exposes functional relations and reduction potential missed by conventional importance- and similarity-based measures. Across modalities and Transformer families, CFS reveals systematic functional reorganization with scale. Controlled scaling further shows that performance gains need not track growth in substitutability, while fixed-capacity models with more independent functional structure perform better, providing a functional account of diminishing returns. Predicted CFS further enables dynamic computation with a better performance--computation trade-off than importance-based component selection, suggesting new directions for redundancy-aware computation and more efficient model scaling.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Chen, J., Li, J., Xiao, Y., Cai, X., Bu, J., Yu, L., & Zhang, T. (2026). Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers. https://omanscience.com/en/articles/effective-does-not-mean-useful-conditional-functional-substitutability-for-redundancy-and-scaling-in-transformers
MLA 9
Chen, Jiaheng, et al. "Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers." https://omanscience.com/en/articles/effective-does-not-mean-useful-conditional-functional-substitutability-for-redundancy-and-scaling-in-transformers.
Chicago (author–date)
Chen, Jiaheng, Jiaxing Li, Yucheng Xiao, Xinyong Cai, Juncheng Bu, Lan Yu, and Tinghe Zhang. 2026. "Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers." https://omanscience.com/en/articles/effective-does-not-mean-useful-conditional-functional-substitutability-for-redundancy-and-scaling-in-transformers.
Harvard
Chen, J., Li, J., Xiao, Y., Cai, X., Bu, J., Yu, L. and Zhang, T. (2026) 'Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers', Available at: https://omanscience.com/en/articles/effective-does-not-mean-useful-conditional-functional-substitutability-for-redundancy-and-scaling-in-transformers.
Vancouver
Chen J, Li J, Xiao Y, Cai X, Bu J, Yu L, et al. Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers. https://omanscience.com/en/articles/effective-does-not-mean-useful-conditional-functional-substitutability-for-redundancy-and-scaling-in-transformers
IEEE
J. Chen, J. Li, Y. Xiao, X. Cai, J. Bu, L. Yu, and T. Zhang, "Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers," https://omanscience.com/en/articles/effective-does-not-mean-useful-conditional-functional-substitutability-for-redundancy-and-scaling-in-transformers.