Preprint Open access
Understanding Clustering in Slot Attention via Particle Dynamics
Studying attention through the lens of interacting particle dynamics has shown how token clustering can emerge from the underlying dynamics. We extend this perspective to slot attention, a method for object-centric image segmentation and representation learning in which learned components obscure how much of the cluste …