Abstract
Single-image nutrition estimation can fail silently when visible foods are missed. Even when a food is correctly identified, its proposed region may not support portion estimation. We propose a framework that uses multimodal large language models (MLLMs) to inventory visible foods and separately verify food identity and whether each proposed 2D region supports portion estimation. One whole-image review uses these verification results to identify unresolved gaps and omitted foods, triggering at most one targeted recovery pass. Recovered regions are re-verified without access to the recovery prompt, then reconciled into a final item set for nutrition estimation. The framework requires no task-specific fine-tuning. Matched evaluation on common valid-output samples shows that item-level grounding improves mass accuracy across all tested settings and energy accuracy relative to an adapted retrieval baseline, with item-identity precision and recall also improving, while post-recovery visual coverage is assessed separately at inference time without ground-truth annotations.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Yue, J., Coburn, B., Ma, J., Chi, J. F., & Zhu, F. (2026). Improving Image-Based Nutrition Estimation Through Multimodal Food-Item Verification and Recovery. https://omanscience.com/en/articles/improving-image-based-nutrition-estimation-through-multimodal-food-item-verification-and-recovery
MLA 9
Yue, Jingbo, et al. "Improving Image-Based Nutrition Estimation Through Multimodal Food-Item Verification and Recovery." https://omanscience.com/en/articles/improving-image-based-nutrition-estimation-through-multimodal-food-item-verification-and-recovery.
Chicago (author–date)
Yue, Jingbo, Bruce Coburn, Jinge Ma, Jui-Feng Chi, and Fengqing Zhu. 2026. "Improving Image-Based Nutrition Estimation Through Multimodal Food-Item Verification and Recovery." https://omanscience.com/en/articles/improving-image-based-nutrition-estimation-through-multimodal-food-item-verification-and-recovery.
Harvard
Yue, J., Coburn, B., Ma, J., Chi, J. F. and Zhu, F. (2026) 'Improving Image-Based Nutrition Estimation Through Multimodal Food-Item Verification and Recovery', Available at: https://omanscience.com/en/articles/improving-image-based-nutrition-estimation-through-multimodal-food-item-verification-and-recovery.
Vancouver
Yue J, Coburn B, Ma J, Chi JF, Zhu F. Improving Image-Based Nutrition Estimation Through Multimodal Food-Item Verification and Recovery. https://omanscience.com/en/articles/improving-image-based-nutrition-estimation-through-multimodal-food-item-verification-and-recovery
IEEE
J. Yue, B. Coburn, J. Ma, J. F. Chi, and F. Zhu, "Improving Image-Based Nutrition Estimation Through Multimodal Food-Item Verification and Recovery," https://omanscience.com/en/articles/improving-image-based-nutrition-estimation-through-multimodal-food-item-verification-and-recovery.