Preprint Open access
Causal and Interpretable Structures in LLM Compositional Tasks
Large language models are able to solve tasks whose answers depend on not only individual input tokens, but also on relations among them. How is such relational information represented and processed across transformer layers? We study activations from ensembles of prompts that require inferring relationships between th …