Abstract
Large language models are able to solve tasks whose answers depend on not only individual input tokens, but also on relations among them. How is such relational information represented and processed across transformer layers? We study activations from ensembles of prompts that require inferring relationships between three tokens corresponding to a cyclic concept (months, hours, weekdays, and musical notes) to correctly predict the next token. Across model families (Llama, Qwen, Gemma, and Mistral) and cyclic concepts, we find a consistent layerwise progression in how the joint dependence among the tokens is geometrically organized and causally used: intermediate layers use a joint representation based on the inferred relationship between two tokens, while later layers use a joint representation associated with all three tokens to correctly complete the task. We also find other relationships between tokens that are geometrically structured but remain causally inert in the next-token prediction. Crucially, when taken together, these geometric and causal investigations reveal the representation-level mechanism that progressively organizes and composes the relational information to form the answer. More surprisingly, restricting the models to such causally relevant joint representations improves next-token prediction accuracy.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Arora, G., Liu, T. J. B., Bao, J., Sarfati, R., & Earls, C. J. (2026). Causal and Interpretable Structures in LLM Compositional Tasks. https://omanscience.com/en/articles/causal-and-interpretable-structures-in-llm-compositional-tasks
MLA 9
Arora, Gurbir, et al. "Causal and Interpretable Structures in LLM Compositional Tasks." https://omanscience.com/en/articles/causal-and-interpretable-structures-in-llm-compositional-tasks.
Chicago (author–date)
Arora, Gurbir, Toni J. B. Liu, Jiajun Bao, Raphaël Sarfati, and Christopher J. Earls. 2026. "Causal and Interpretable Structures in LLM Compositional Tasks." https://omanscience.com/en/articles/causal-and-interpretable-structures-in-llm-compositional-tasks.
Harvard
Arora, G., Liu, T. J. B., Bao, J., Sarfati, R. and Earls, C. J. (2026) 'Causal and Interpretable Structures in LLM Compositional Tasks', Available at: https://omanscience.com/en/articles/causal-and-interpretable-structures-in-llm-compositional-tasks.
Vancouver
Arora G, Liu TJB, Bao J, Sarfati R, Earls CJ. Causal and Interpretable Structures in LLM Compositional Tasks. https://omanscience.com/en/articles/causal-and-interpretable-structures-in-llm-compositional-tasks
IEEE
G. Arora, T. J. B. Liu, J. Bao, R. Sarfati, and C. J. Earls, "Causal and Interpretable Structures in LLM Compositional Tasks," https://omanscience.com/en/articles/causal-and-interpretable-structures-in-llm-compositional-tasks.