Abstract
As large language models (LLMs) are increasingly used in clinical settings, it is critical to evaluate their reliability under realistic variation in clinical text. We study this question in clinical triage, comparing LLMs to practicing physicians under text perturbations that preserve the underlying clinical setting. We introduce a benchmark of over 6,000 clinical scenarios, 7,000 physician annotations, and 225,000 model responses. Using this benchmark, we make two key observations. First, LLMs are more likely than physicians to recommend unnecessary care at baseline, and this tendency increases under perturbed inputs. Further, we find that LLM recommendations are more sensitive to gender and tone perturbations than human recommendations. Together, these results demonstrate that LLMs can vary under clinically irrelevant textual changes, highlighting the need for deployment-oriented evaluations grounded in expert physician behavior.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Gourabathina, A., Zhang, H., Hao, Y., Gerych, W., & Ghassemi, M. (2026). Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts. https://omanscience.com/en/articles/sense-and-sensitivity-benchmarking-llm-clinical-triage-recommendations-with-physician-experts
MLA 9
Gourabathina, Abinitha, et al. "Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts." https://omanscience.com/en/articles/sense-and-sensitivity-benchmarking-llm-clinical-triage-recommendations-with-physician-experts.
Chicago (author–date)
Gourabathina, Abinitha, Haoran Zhang, Yuexing Hao, Walter Gerych, and Marzyeh Ghassemi. 2026. "Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts." https://omanscience.com/en/articles/sense-and-sensitivity-benchmarking-llm-clinical-triage-recommendations-with-physician-experts.
Harvard
Gourabathina, A., Zhang, H., Hao, Y., Gerych, W. and Ghassemi, M. (2026) 'Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts', Available at: https://omanscience.com/en/articles/sense-and-sensitivity-benchmarking-llm-clinical-triage-recommendations-with-physician-experts.
Vancouver
Gourabathina A, Zhang H, Hao Y, Gerych W, Ghassemi M. Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts. https://omanscience.com/en/articles/sense-and-sensitivity-benchmarking-llm-clinical-triage-recommendations-with-physician-experts
IEEE
A. Gourabathina, H. Zhang, Y. Hao, W. Gerych, and M. Ghassemi, "Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts," https://omanscience.com/en/articles/sense-and-sensitivity-benchmarking-llm-clinical-triage-recommendations-with-physician-experts.