الملخص
Audio large language models (AudioLLMs) operate reactively, responding only when queried. We introduce proactive audio assistance, where an AudioLLM monitors an audio stream and autonomously decides when to alert the user from a single natural-language intent, motivated by wearable applications for Deaf and Hard of Hearing users. We propose Interrupt and Silent Modeling (ISM), a model-agnostic paradigm that embeds proactive decisions into LLM decoding via two special tokens: \texttt{ } and \texttt{ }, capturing four states: onset detection, sustained-relevance triggering, irrelevance suppression, and de-duplication. Applied to Qwen2-Audio-7B, ISM achieves 99.6\% interrupt F1 and perfect de-duplication recall on ESC-50. On noisy Epic-Sounds kitchen audio, ISM achieves the highest interrupt F1 without domain-specific training, the only method maintaining strong onset detection without over-triggering or over-suppression. Streaming evaluation confirms real-time viability with 3.5-second average latency.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Yadav, A. K. S., Shrivastava, R., Zhang, X., Moon, S., Jain, S., Donmez, P., & Damavandi, B. (2026). I'll Keep an Ear Out: Teaching AudioLLMs Proactive Audio Assistance. https://omanscience.com/ar/articles/i-ll-keep-an-ear-out-teaching-audiollms-proactive-audio-assistance
MLA 9
Yadav, Amit Kumar Singh, et al. "I'll Keep an Ear Out: Teaching AudioLLMs Proactive Audio Assistance." https://omanscience.com/ar/articles/i-ll-keep-an-ear-out-teaching-audiollms-proactive-audio-assistance.
شيكاغو (المؤلف–التاريخ)
Yadav, Amit Kumar Singh, Ritvik Shrivastava, Xuan Zhang, Seungwhan Moon, Shashank Jain, Pinar Donmez, and Babak Damavandi. 2026. "I'll Keep an Ear Out: Teaching AudioLLMs Proactive Audio Assistance." https://omanscience.com/ar/articles/i-ll-keep-an-ear-out-teaching-audiollms-proactive-audio-assistance.
هارفارد
Yadav, A. K. S., Shrivastava, R., Zhang, X., Moon, S., Jain, S., Donmez, P. and Damavandi, B. (2026) 'I'll Keep an Ear Out: Teaching AudioLLMs Proactive Audio Assistance', Available at: https://omanscience.com/ar/articles/i-ll-keep-an-ear-out-teaching-audiollms-proactive-audio-assistance.
فانكوفر
Yadav AKS, Shrivastava R, Zhang X, Moon S, Jain S, Donmez P, et al. I'll Keep an Ear Out: Teaching AudioLLMs Proactive Audio Assistance. https://omanscience.com/ar/articles/i-ll-keep-an-ear-out-teaching-audiollms-proactive-audio-assistance
IEEE
A. K. S. Yadav, R. Shrivastava, X. Zhang, S. Moon, S. Jain, P. Donmez, and B. Damavandi, "I'll Keep an Ear Out: Teaching AudioLLMs Proactive Audio Assistance," https://omanscience.com/ar/articles/i-ll-keep-an-ear-out-teaching-audiollms-proactive-audio-assistance.