Abstract
Post-training quantization (PTQ) enables efficient deployment of large language models, and PTQ methods are usually optimized and evaluated with generic reconstruction, perplexity, or answer accuracy. But in explanation-critical domains, preserving only the final answer may be insufficient, since users may also inspect generated rationales to judge whether a prediction is trustworthy. We study this issue in medical multiple-choice question answering, where rationales should provide evidence that supports the selected answer. We propose an explanation-aware objective for transformation-based PTQ. Our method builds an offline faithfulness cache from full-precision teacher rationales and uses it during optimization to preserve answer-supporting evidence tokens and evidence-conditioned answer behavior. We instantiate it on OSTQuant under W4A4KV4 quantization and evaluate four 7B--8B medical and instruction-tuned LLMs on MedExQA, MedExpQA, and ChallengeClinicalQA. While a same-calibration OSTQuant baseline preserves task accuracy, it can substantially weaken answer-supporting rationales. Our objective is to preserve the full-precision model's answer-supporting behavior rather than improve gold-label accuracy, and our method better preserves the full-precision model's answer behavior and rationale-to-answer support. These results suggest that PTQ for explanation-critical settings should evaluate preservation of answer-supporting evidence, not only answer accuracy. Code and evaluation scripts are available at https://github.com/dut0817/EAQuant.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Kim, Y., Kim, M. Y., & Goebel, R. (2026). When Quantization Preserves Accuracy but Not Evidence: Explanation-Aware Post-Training Quantization for Medical LLMs. https://omanscience.com/en/articles/when-quantization-preserves-accuracy-but-not-evidence-explanation-aware-post-training-quantization-for-medical-llms
MLA 9
Kim, Yeji, et al. "When Quantization Preserves Accuracy but Not Evidence: Explanation-Aware Post-Training Quantization for Medical LLMs." https://omanscience.com/en/articles/when-quantization-preserves-accuracy-but-not-evidence-explanation-aware-post-training-quantization-for-medical-llms.
Chicago (author–date)
Kim, Yeji, Mi-Young Kim, and Randy Goebel. 2026. "When Quantization Preserves Accuracy but Not Evidence: Explanation-Aware Post-Training Quantization for Medical LLMs." https://omanscience.com/en/articles/when-quantization-preserves-accuracy-but-not-evidence-explanation-aware-post-training-quantization-for-medical-llms.
Harvard
Kim, Y., Kim, M. Y. and Goebel, R. (2026) 'When Quantization Preserves Accuracy but Not Evidence: Explanation-Aware Post-Training Quantization for Medical LLMs', Available at: https://omanscience.com/en/articles/when-quantization-preserves-accuracy-but-not-evidence-explanation-aware-post-training-quantization-for-medical-llms.
Vancouver
Kim Y, Kim MY, Goebel R. When Quantization Preserves Accuracy but Not Evidence: Explanation-Aware Post-Training Quantization for Medical LLMs. https://omanscience.com/en/articles/when-quantization-preserves-accuracy-but-not-evidence-explanation-aware-post-training-quantization-for-medical-llms
IEEE
Y. Kim, M. Y. Kim, and R. Goebel, "When Quantization Preserves Accuracy but Not Evidence: Explanation-Aware Post-Training Quantization for Medical LLMs," https://omanscience.com/en/articles/when-quantization-preserves-accuracy-but-not-evidence-explanation-aware-post-training-quantization-for-medical-llms.