Preprint Open access
AlignQuant: Tile-Aligned Mixed-Precision Quantization for Efficient LLM Generation
Fine-grained mixed-precision quantization promises efficient large language model inference, but local precision choices can conflict with regular GPU storage and computation units. This precision-boundary mismatch limits the translation of compression into practical acceleration. We introduce AlignQuant, a post-traini …