Preprint Open access
ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs
Learned rotations play an important role in enabling low-bit weight and activation quantization of large language models by smoothing outliers in the activation distribution. State-of-the-art approaches include gradient-based procedures such as SpinQuant and computationally friendlier gradient-free approaches such as D …