Authors

James Hensman

Publications 2

Preprint Open access

Activation Denoising: A Robustness View on Parallel vs Sequential LLM Quantization

Yan Scholten, Rachel Lawrence, James Hensman et al. · 2026

Post-training quantization is a powerful tool for compressing large language models. The most scalable methods quantize every layer in parallel, but quantization errors then compound through the residual stream, as no layer corrects for the errors of the layers before it. Sequential quantization accounts for this error …

Preprint Open access

Rotated Manifold Optimization for Low-Rank Adaptation

Yuhui Ding, Javier Zazo, James Hensman · 2026

We propose a novel optimizer for low-rank adaptation (LoRA) that explicitly incorporates the gauge symmetry of low-rank factorization. Our optimizer extends recent matrix optimizers for full-parameter training to the manifold of fixed-rank matrices by interpreting them as normalization under a rotated basis. We show ho …

Co-authors