Abstract
Latent diffusion models now dominate medical image generation, and every such pipeline rests on a \emph{tokenizer} that compresses images into the latent codes for image generation to operate on. Thereby, the tokenizer choice bounds every downstream task from reconstruction fidelity and generation quality to the representations available for downstream analysis. Yet, medical imaging pipelines routinely utilize tokenizers from natural imaging on the hypothesis that their behavior carries over. However, this is an assumption never tested in the medical imaging regime, where datasets are orders of magnitude smaller and images exhibit far lower inter-sample variance. We present a systematic evaluation of medical image tokenizers evaluating thirty configurations across ten model families on twelve datasets at three compression factors, spanning reconstruction, generation, latent geometry, downstream classification, and memorization. We find that (1) performance on image reconstruction and generation strongly correlate, unlike prior reports on natural images; (2) modern tokenizers use nearly all of their codebook entries, but still leave most of the latent space unused; (3) training-set memorization is mild and is further suppressed by stronger latent space compression; and (4) discrete quantization can largely preserve downstream classification, with lookup-free schemes being the main exception.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Bubeck, N., Zhang, Y., Sideri-Lampretsa, V., McGinnis, J., Yang, J., Rueckert, D., & Pan, J. (2026). What Makes a Good Medical Image Tokenizer? Rethinking Reconstruction and Generation in Medical Image Tokenization. https://omanscience.com/en/articles/what-makes-a-good-medical-image-tokenizer-rethinking-reconstruction-and-generation-in-medical-image-tokenization
MLA 9
Bubeck, Niklas, et al. "What Makes a Good Medical Image Tokenizer? Rethinking Reconstruction and Generation in Medical Image Tokenization." https://omanscience.com/en/articles/what-makes-a-good-medical-image-tokenizer-rethinking-reconstruction-and-generation-in-medical-image-tokenization.
Chicago (author–date)
Bubeck, Niklas, Yundi Zhang, Vasiliki Sideri-Lampretsa, Julian McGinnis, Jiancheng Yang, Daniel Rueckert, and Jiazhen Pan. 2026. "What Makes a Good Medical Image Tokenizer? Rethinking Reconstruction and Generation in Medical Image Tokenization." https://omanscience.com/en/articles/what-makes-a-good-medical-image-tokenizer-rethinking-reconstruction-and-generation-in-medical-image-tokenization.
Harvard
Bubeck, N., Zhang, Y., Sideri-Lampretsa, V., McGinnis, J., Yang, J., Rueckert, D. and Pan, J. (2026) 'What Makes a Good Medical Image Tokenizer? Rethinking Reconstruction and Generation in Medical Image Tokenization', Available at: https://omanscience.com/en/articles/what-makes-a-good-medical-image-tokenizer-rethinking-reconstruction-and-generation-in-medical-image-tokenization.
Vancouver
Bubeck N, Zhang Y, Sideri-Lampretsa V, McGinnis J, Yang J, Rueckert D, et al. What Makes a Good Medical Image Tokenizer? Rethinking Reconstruction and Generation in Medical Image Tokenization. https://omanscience.com/en/articles/what-makes-a-good-medical-image-tokenizer-rethinking-reconstruction-and-generation-in-medical-image-tokenization
IEEE
N. Bubeck, Y. Zhang, V. Sideri-Lampretsa, J. McGinnis, J. Yang, D. Rueckert, and J. Pan, "What Makes a Good Medical Image Tokenizer? Rethinking Reconstruction and Generation in Medical Image Tokenization," https://omanscience.com/en/articles/what-makes-a-good-medical-image-tokenizer-rethinking-reconstruction-and-generation-in-medical-image-tokenization.