Abstract
Sink-aware deployment may identify important first-token attention heads before a model is quantized, then reuse that map at the edge. We test when this shortcut is safe for 4-bit NF4 weight-only post-training quantization (PTQ). Our Sink Topology Consistency (STC) metrics separate global rank preservation, top-$k$ set overlap, and layerwise sink-mass shift, and distinguish per-input sensitivity from calibration-map transfer. Across Qwen2.5-0.5B, Qwen2.5-1.5B, and Llama-3.2-1B, global bf16-to-4-bit ranks remain high at 4,096 tokens ($ρ_s \geq 0.980$), yet top-$k$ Jaccard overlap is only 0.619-0.793, corresponding to 76.5-88.5% membership retention. The global statistic also masks local failures: terminal Qwen layers shift by 6.2-7.9x their model means, whereas Llama-3.2-1B shows low, nearly uniform drift. Under a C4-to-LongBench shift, cross-domain overlap degrades more than the within-domain precision comparison for both Qwen models, but not for Llama-3.2-1B. Matched-domain 4-bit recalibration reaches 90% of a split-half stability plateau at the smallest tested $n=8$ for both Qwen models and $n=32$ for Llama-3.2-1B, though not as a sharp threshold; for the two Qwen models, updating only selected layers does not reach the full-map stability criterion. On Jetson Orin NX, the 16-sample workload takes seconds for the two models with valid on-device sink measurements. The practical message is precise: global rankings often transfer, but discrete head sets, layer-local policies, and cross-domain calibration should be revalidated after quantization.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Chen, K., Kuo, C. W., & Ou, C. E. (2026). Global Ranks Survive, Selected Heads Shift: BOS-Sink Topology under 4-bit Weight-Only Quantization. https://omanscience.com/en/articles/global-ranks-survive-selected-heads-shift-bos-sink-topology-under-4-bit-weight-only-quantization
MLA 9
Chen, Kuanlin, et al. "Global Ranks Survive, Selected Heads Shift: BOS-Sink Topology under 4-bit Weight-Only Quantization." https://omanscience.com/en/articles/global-ranks-survive-selected-heads-shift-bos-sink-topology-under-4-bit-weight-only-quantization.
Chicago (author–date)
Chen, Kuanlin, Chen-Wei Kuo, and Cheng-En Ou. 2026. "Global Ranks Survive, Selected Heads Shift: BOS-Sink Topology under 4-bit Weight-Only Quantization." https://omanscience.com/en/articles/global-ranks-survive-selected-heads-shift-bos-sink-topology-under-4-bit-weight-only-quantization.
Harvard
Chen, K., Kuo, C. W. and Ou, C. E. (2026) 'Global Ranks Survive, Selected Heads Shift: BOS-Sink Topology under 4-bit Weight-Only Quantization', Available at: https://omanscience.com/en/articles/global-ranks-survive-selected-heads-shift-bos-sink-topology-under-4-bit-weight-only-quantization.
Vancouver
Chen K, Kuo CW, Ou CE. Global Ranks Survive, Selected Heads Shift: BOS-Sink Topology under 4-bit Weight-Only Quantization. https://omanscience.com/en/articles/global-ranks-survive-selected-heads-shift-bos-sink-topology-under-4-bit-weight-only-quantization
IEEE
K. Chen, C. W. Kuo, and C. E. Ou, "Global Ranks Survive, Selected Heads Shift: BOS-Sink Topology under 4-bit Weight-Only Quantization," https://omanscience.com/en/articles/global-ranks-survive-selected-heads-shift-bos-sink-topology-under-4-bit-weight-only-quantization.