Abstract
Vision language action (VLA) policies continue to grow in parameter count, making deployment on resource-constrained robot platforms difficult. The central goal is to reduce the number of LLM-side parameters retained in the deployed policy while preserving downstream task performance. Our approach, AdaDE, adapts selected dense feed forward blocks into mixture of experts (MoE) layers and derives expert retention masks from router statistics during fine tuning. The Dense2MoE conversion preserves the original dense FFN function at initialization, so expert deactivation can start without a separate recovery stage. Instead of using a fixed shutdown rule, expert masks are updated dynamically from router usage statistics, with staged training and expert protection to avoid early collapse. With 40% of the LLM parameters deactivated, AdaDE retains 95.7% average success in LIBERO and 42.0% average success across all 50 RobotWin2.0 tasks. These results suggest that dense to MoE adaptation with dynamic expert deactivation is a practical direction for reducing active VLA model size without severe performance loss.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Niu, M., Chen, S., Wu, Y., Tu, X., Wang, Y., Ren, J., Zhang, X., & Zhang, L. (2026). Dense to MoE Adaptation for Compact Vision Language Action Policies. https://omanscience.com/en/articles/dense-to-moe-adaptation-for-compact-vision-language-action-policies
MLA 9
Niu, Muchun, et al. "Dense to MoE Adaptation for Compact Vision Language Action Policies." https://omanscience.com/en/articles/dense-to-moe-adaptation-for-compact-vision-language-action-policies.
Chicago (author–date)
Niu, Muchun, Shuang Chen, Yuzhou Wu, Xiaobing Tu, Yinggui Wang, Jinkui Ren, Xiantao Zhang, and Linfeng Zhang. 2026. "Dense to MoE Adaptation for Compact Vision Language Action Policies." https://omanscience.com/en/articles/dense-to-moe-adaptation-for-compact-vision-language-action-policies.
Harvard
Niu, M., Chen, S., Wu, Y., Tu, X., Wang, Y., Ren, J., Zhang, X. and Zhang, L. (2026) 'Dense to MoE Adaptation for Compact Vision Language Action Policies', Available at: https://omanscience.com/en/articles/dense-to-moe-adaptation-for-compact-vision-language-action-policies.
Vancouver
Niu M, Chen S, Wu Y, Tu X, Wang Y, Ren J, et al. Dense to MoE Adaptation for Compact Vision Language Action Policies. https://omanscience.com/en/articles/dense-to-moe-adaptation-for-compact-vision-language-action-policies
IEEE
M. Niu, S. Chen, Y. Wu, X. Tu, Y. Wang, J. Ren, X. Zhang, and L. Zhang, "Dense to MoE Adaptation for Compact Vision Language Action Policies," https://omanscience.com/en/articles/dense-to-moe-adaptation-for-compact-vision-language-action-policies.