Abstract
Diffusion language models (DLMs) enable fast generation by predicting multiple tokens in parallel, but their practical adoption remains limited by a persistent quality gap relative to comparably sized autoregressive (AR) models. We attribute this gap to a computation-difficulty mismatch: within a partially observed sequence, some unknown tokens are easy to predict, while others require substantially more computation. Existing DLMs nevertheless apply uniform computational depth to all unknown positions at each denoising step. We introduce ALoDLM, which replaces uniform computation with token-adaptive latent recurrence. At each denoising step, ALoDLM iteratively refines latent representations and allocates computation according to token difficulty. Tokens ready to commit are fed back as discrete context, while unresolved tokens retain and further refine their latent states through additional recurrent passes. To learn token prediction and computation allocation jointly, we formulate token-wise computation schedules as latent variables and derive a conditional negative evidence lower bound (NELBO). We train ALoDLM at 1.7B and 8B parameter scales. Across eleven benchmarks, ALoDLM outperforms all evaluated DLMs and the corresponding AR baselines in average benchmark score at both scales. ALoDLM also retains fast parallel decoding, yielding a strong quality-efficiency trade-off among evaluated autoregressive and diffusion models under optimized inference engines.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Fang, L., Li, Z., Kim, Y., Zhao, T., Koner, R., Wu, J., Xu, L., Chen, X., Xu, X., Zhang, Z., Zablocki, J., Sankaran, N., & Xing, Y. (2026). ALoDLM: Adaptively Looped Diffusion Language Models. https://omanscience.com/en/articles/alodlm-adaptively-looped-diffusion-language-models
MLA 9
Fang, Liancheng, et al. "ALoDLM: Adaptively Looped Diffusion Language Models." https://omanscience.com/en/articles/alodlm-adaptively-looped-diffusion-language-models.
Chicago (author–date)
Fang, Liancheng, Zhuowei Li, Youngeun Kim, Tianchen Zhao, Rajat Koner, Jiaye Wu, Linghan Xu, Xuanbai Chen, Xiang Xu, Zheng Zhang, Jakub Zablocki, Nishant Sankaran, and Yifan Xing. 2026. "ALoDLM: Adaptively Looped Diffusion Language Models." https://omanscience.com/en/articles/alodlm-adaptively-looped-diffusion-language-models.
Harvard
Fang, L., Li, Z., Kim, Y., Zhao, T., Koner, R., Wu, J., Xu, L., Chen, X., Xu, X., Zhang, Z., Zablocki, J., Sankaran, N. and Xing, Y. (2026) 'ALoDLM: Adaptively Looped Diffusion Language Models', Available at: https://omanscience.com/en/articles/alodlm-adaptively-looped-diffusion-language-models.
Vancouver
Fang L, Li Z, Kim Y, Zhao T, Koner R, Wu J, et al. ALoDLM: Adaptively Looped Diffusion Language Models. https://omanscience.com/en/articles/alodlm-adaptively-looped-diffusion-language-models
IEEE
L. Fang, Z. Li, Y. Kim, T. Zhao, R. Koner, J. Wu, L. Xu, X. Chen, X. Xu, Z. Zhang, J. Zablocki, N. Sankaran, and Y. Xing, "ALoDLM: Adaptively Looped Diffusion Language Models," https://omanscience.com/en/articles/alodlm-adaptively-looped-diffusion-language-models.