Abstract
People write and sketch while speaking to explain, organize, and develop content together. Inspired by these practices, we investigate how voice agents can use a canvas alongside speech in multi-turn conversations with users. We conducted a two-part formative study: an observational study of how pairs coordinated speech and boardwork, followed by a design workshop that informed a design space for speech-canvas interaction with voice agents. Building on these insights, we developed VoCa, a voice agent that coordinates speech with visual object creation, annotation, and attention guidance. A five-day deployment with 18 participants examined usability, experiences of speech-canvas interaction, patterns of use, and desired improvements. Participants' experiences highlighted opportunities for speech-canvas interaction in learning, work, and daily life, alongside challenges in coordinating what agents say and show in ways users can follow and influence. These findings inform how voice agents can use a canvas alongside speech in conversation.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Ge, Y., Yuan, R., Qi, Y., He, W., Mo, J., Chen, Y., Zuo, W., Sun, X., Guo, W., & Wang, Q. (2026). VoCa: Designing Speech-Canvas Interaction for Voice-Based Conversational Agents. https://omanscience.com/en/articles/voca-designing-speech-canvas-interaction-for-voice-based-conversational-agents
MLA 9
Ge, Yate, et al. "VoCa: Designing Speech-Canvas Interaction for Voice-Based Conversational Agents." https://omanscience.com/en/articles/voca-designing-speech-canvas-interaction-for-voice-based-conversational-agents.
Chicago (author–date)
Ge, Yate, Run Yuan, Yueran Qi, Wenjie He, Jiaqi Mo, Yangshuo Chen, Wenbin Zuo, Xiaohua Sun, Weiwei Guo, and Qi Wang. 2026. "VoCa: Designing Speech-Canvas Interaction for Voice-Based Conversational Agents." https://omanscience.com/en/articles/voca-designing-speech-canvas-interaction-for-voice-based-conversational-agents.
Harvard
Ge, Y., Yuan, R., Qi, Y., He, W., Mo, J., Chen, Y., Zuo, W., Sun, X., Guo, W. and Wang, Q. (2026) 'VoCa: Designing Speech-Canvas Interaction for Voice-Based Conversational Agents', Available at: https://omanscience.com/en/articles/voca-designing-speech-canvas-interaction-for-voice-based-conversational-agents.
Vancouver
Ge Y, Yuan R, Qi Y, He W, Mo J, Chen Y, et al. VoCa: Designing Speech-Canvas Interaction for Voice-Based Conversational Agents. https://omanscience.com/en/articles/voca-designing-speech-canvas-interaction-for-voice-based-conversational-agents
IEEE
Y. Ge, R. Yuan, Y. Qi, W. He, J. Mo, Y. Chen, W. Zuo, X. Sun, W. Guo, and Q. Wang, "VoCa: Designing Speech-Canvas Interaction for Voice-Based Conversational Agents," https://omanscience.com/en/articles/voca-designing-speech-canvas-interaction-for-voice-based-conversational-agents.