Abstract
Clinical prediction models estimate risk from patient measurements, while large language models support medical text understanding and question answering. Yet their language capabilities do not ensure accurate prediction from structured, high-dimensional clinical time series. Improving this ability would connect risk estimation with flexible questions about a patient's evolving condition. We introduce ViSTA, a compact adapter that incorporates irregular numerical measurements into a pretrained vision-language model's chart representations. It learns corrections to visual tokens while leaving all pretrained parameters unchanged. On MIMIC-IV, ViSTA has the highest mean scores among the compared adaptations on all four metrics for acute kidney injury and mortality prediction across models with 2-9 billion parameters. With 0.516 million trainable parameters, the 2-billion-parameter model reaches an area under the ROC curve of 0.7376 for acute kidney injury, compared with GPT-5.6 Sol's 0.7380 with text input and high reasoning effort. Training for temporal question answering yields 69.27% accuracy at 4 billion parameters with over 90% fewer trainable parameters than low-rank adaptation using charts or numerical text, at a 2.82-4.88 percentage-point accuracy gap. ViSTA extends pretrained language models to numerical prediction and temporal questions.
Keywords
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Gao, J., Shi, Y., Hu, P., & Harrison, E. M. (2026). ViSTA: A Simple Bridge Extends Visual Alignment to Clinical Time-Series Understanding in Multimodal LLMs. https://omanscience.com/en/articles/vista-a-simple-bridge-extends-visual-alignment-to-clinical-time-series-understanding-in-multimodal-llms
MLA 9
Gao, Junyi, et al. "ViSTA: A Simple Bridge Extends Visual Alignment to Clinical Time-Series Understanding in Multimodal LLMs." https://omanscience.com/en/articles/vista-a-simple-bridge-extends-visual-alignment-to-clinical-time-series-understanding-in-multimodal-llms.
Chicago (author–date)
Gao, Junyi, Yu Shi, Pingzhao Hu, and Ewen M Harrison. 2026. "ViSTA: A Simple Bridge Extends Visual Alignment to Clinical Time-Series Understanding in Multimodal LLMs." https://omanscience.com/en/articles/vista-a-simple-bridge-extends-visual-alignment-to-clinical-time-series-understanding-in-multimodal-llms.
Harvard
Gao, J., Shi, Y., Hu, P. and Harrison, E. M. (2026) 'ViSTA: A Simple Bridge Extends Visual Alignment to Clinical Time-Series Understanding in Multimodal LLMs', Available at: https://omanscience.com/en/articles/vista-a-simple-bridge-extends-visual-alignment-to-clinical-time-series-understanding-in-multimodal-llms.
Vancouver
Gao J, Shi Y, Hu P, Harrison EM. ViSTA: A Simple Bridge Extends Visual Alignment to Clinical Time-Series Understanding in Multimodal LLMs. https://omanscience.com/en/articles/vista-a-simple-bridge-extends-visual-alignment-to-clinical-time-series-understanding-in-multimodal-llms
IEEE
J. Gao, Y. Shi, P. Hu, and E. M. Harrison, "ViSTA: A Simple Bridge Extends Visual Alignment to Clinical Time-Series Understanding in Multimodal LLMs," https://omanscience.com/en/articles/vista-a-simple-bridge-extends-visual-alignment-to-clinical-time-series-understanding-in-multimodal-llms.