Abstract
Spherical data arise in many scientific applications. Often spherical transformers disregard the geometry of the underlying spherical domain, causing distortions and coordinate singularities near the poles. We introduce SO(3)-RoPE, a relative positional embedding that incorporates spherical geometry into transformer attention through unitary SO(3) representations. Our formulation is SO(3)-equivariant and compatible with FlashAttention, retaining efficiency of vanilla transformers. On shallow water dynamics prediction over a rotating sphere, our SO3ViT outperforms an S2Transformer baseline with lower errors and reduced runtime.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Libner, C., van de Geijn, C., Ecker, A. S., & Weiler, M. (2026). SO(3)-RoPE for Spherical Transformers. https://omanscience.com/en/articles/so-3-rope-for-spherical-transformers
MLA 9
Libner, Christian, et al. "SO(3)-RoPE for Spherical Transformers." https://omanscience.com/en/articles/so-3-rope-for-spherical-transformers.
Chicago (author–date)
Libner, Christian, Chase van de Geijn, Alexander S. Ecker, and Maurice Weiler. 2026. "SO(3)-RoPE for Spherical Transformers." https://omanscience.com/en/articles/so-3-rope-for-spherical-transformers.
Harvard
Libner, C., van de Geijn, C., Ecker, A. S. and Weiler, M. (2026) 'SO(3)-RoPE for Spherical Transformers', Available at: https://omanscience.com/en/articles/so-3-rope-for-spherical-transformers.
Vancouver
Libner C, van de Geijn C, Ecker AS, Weiler M. SO(3)-RoPE for Spherical Transformers. https://omanscience.com/en/articles/so-3-rope-for-spherical-transformers
IEEE
C. Libner, C. van de Geijn, A. S. Ecker, and M. Weiler, "SO(3)-RoPE for Spherical Transformers," https://omanscience.com/en/articles/so-3-rope-for-spherical-transformers.