Preprint Open access
Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers
Modern neural networks scale predictably, yet the mechanisms behind these regularities remain unclear. Neural redundancy is typically characterized by component importance or representational similarity, both indirect proxies. We view redundancy as an input-conditioned, dynamic relation: intermediate computational stat …