الباحثون

Ngai Wong

المنشورات 2

نسخة أولية وصول مفتوح

QuantMLA: Function-Aligned Dual-Path Quantization for Low-Bit MLA KV Caching

Zunhai Su, Yuxuan Sun, Jianchao Tan وآخرون · 2026

Multi-Head Latent Attention (MLA) enables expressive multi-head attention with compact caches for its content and decoupled RoPE paths, yet cache memory still scales linearly with context length and batch size. In this work, we establish a systematic model of MLA's dual-path quantization errors, characterizing their di …

نسخة أولية وصول مفتوح

Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidance

Hengyuan Zhang, Chenming Shang, Zunhai Su وآخرون · 2026

Best-of-$N$ is a widely used inference strategy for complex reasoning, whose effectiveness depends on whether sampled candidates can cover diverse and high-quality reasoning paths. However, post-trained reasoning models often suffer from \emph{exploration collapse}, where independent rollouts repeatedly follow similar …

المؤلفون المشاركون