Preprint Open access
Improving Video Sparse Attention with Fine-grained Router and Sparse Rebasing
We present VSA2, a frontier trainable sparse attention for video DiTs. VSA2 includes a variety of new architectural features and training procedures that we apply across all stages of the DiT development cycle, including pretraining, RL, and inference, to produce a DiT with comparable or better quality than a full atte …