الملخص
Vision-Language-Action (VLA) models rely strongly on language for describing task information, despite having multimodal inputs. We hypothesize that other modalities in the state space may present opportunities for supplemental task conditioning, which may be particularly relevant in cluttered or otherwise ambiguous scenes. We introduce two tuned models to test this hypothesis: (1) an electrophysiology-conditioned VLA (EC-VLA) that incorporates 8-channel electromyography envelopes as continuous conditioning input concatenated to the proprioceptive vector, and (2) a visually-annotated VLA (VA-VLA) that incorporates visual segmentation annotations to the image inputs. On a cube-selection task evaluated across three participants, EC-VLA matches a language-prompted baseline in uncluttered, in-distribution conditions and substantially outperforms it in cluttered, out-of-distribution scenes. Similarly, VA-VLA shows modest improvements over a language-prompted baseline in in-distribution scenes with substantial improvement in cluttered, out-of-distribution trials. Together, these results provide strong evidence for the potential benefit of task-conditioning beyond language.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Staley, E. W., Pyles, C. O., Hingorani, R., Camargo, F., Milsap, G., Markowitz, J., Fifer, M. S., & Wolmetz, M. (2026). Continuous Conditioning of VLAs with Augmenting EMG and Visual Task Descriptors. https://omanscience.com/ar/articles/continuous-conditioning-of-vlas-with-augmenting-emg-and-visual-task-descriptors
MLA 9
Staley, Edward W., et al. "Continuous Conditioning of VLAs with Augmenting EMG and Visual Task Descriptors." https://omanscience.com/ar/articles/continuous-conditioning-of-vlas-with-augmenting-emg-and-visual-task-descriptors.
شيكاغو (المؤلف–التاريخ)
Staley, Edward W., Connor O. Pyles, Rahul Hingorani, Frank Camargo, Griffin Milsap, Jared Markowitz, Matthew S. Fifer, and Michael Wolmetz. 2026. "Continuous Conditioning of VLAs with Augmenting EMG and Visual Task Descriptors." https://omanscience.com/ar/articles/continuous-conditioning-of-vlas-with-augmenting-emg-and-visual-task-descriptors.
هارفارد
Staley, E. W., Pyles, C. O., Hingorani, R., Camargo, F., Milsap, G., Markowitz, J., Fifer, M. S. and Wolmetz, M. (2026) 'Continuous Conditioning of VLAs with Augmenting EMG and Visual Task Descriptors', Available at: https://omanscience.com/ar/articles/continuous-conditioning-of-vlas-with-augmenting-emg-and-visual-task-descriptors.
فانكوفر
Staley EW, Pyles CO, Hingorani R, Camargo F, Milsap G, Markowitz J, et al. Continuous Conditioning of VLAs with Augmenting EMG and Visual Task Descriptors. https://omanscience.com/ar/articles/continuous-conditioning-of-vlas-with-augmenting-emg-and-visual-task-descriptors
IEEE
E. W. Staley, C. O. Pyles, R. Hingorani, F. Camargo, G. Milsap, J. Markowitz, M. S. Fifer, and M. Wolmetz, "Continuous Conditioning of VLAs with Augmenting EMG and Visual Task Descriptors," https://omanscience.com/ar/articles/continuous-conditioning-of-vlas-with-augmenting-emg-and-visual-task-descriptors.