الملخص
End-to-end speech-to-speech dialogue models listen and speak simultaneously, so a continuously open acoustic channel is exposed to adversarial manipulation. We formalize imperceptible attacks on full-duplex agents as optimization over additive perturbations confined beneath the psychoacoustic masking threshold of the carrier speech, under three goals: targeted semantic hijacking, response suppression, and policy jailbreaking. Against an undefended Moshi-style agent, white-box attacks succeed in up to 91.7% of trials. We then introduce psychoacoustically aligned latent smoothing (PALS), which injects anisotropic Gaussian noise shaped by local codebook covariance at the residual-vector-quantized latent interface, with input noise shaped by the masking threshold constraining the attacker and trained by a Kullback--Leibler consistency objective. Deployed with no inference-time cost, PALS reduces hijack to 8.3%, mute to 11.2%, and jailbreak to 9.1% at clean quality within 2.3%. A Monte Carlo-smoothed variant certifies an ellipsoidal latent radius up to 0.616, a guaranteed floor that the empirical robustness far exceeds.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Shamsaie, K., & Modarressi, I. (2026). Psychoacoustically Aligned Latent Smoothing for Adversarial Robustness of Full-Duplex Speech-to-Speech Dialogue Models. https://omanscience.com/ar/articles/psychoacoustically-aligned-latent-smoothing-for-adversarial-robustness-of-full-duplex-speech-to-speech-dialogue-models
MLA 9
Shamsaie, Kian, and Iman Modarressi. "Psychoacoustically Aligned Latent Smoothing for Adversarial Robustness of Full-Duplex Speech-to-Speech Dialogue Models." https://omanscience.com/ar/articles/psychoacoustically-aligned-latent-smoothing-for-adversarial-robustness-of-full-duplex-speech-to-speech-dialogue-models.
شيكاغو (المؤلف–التاريخ)
Shamsaie, Kian, and Iman Modarressi. 2026. "Psychoacoustically Aligned Latent Smoothing for Adversarial Robustness of Full-Duplex Speech-to-Speech Dialogue Models." https://omanscience.com/ar/articles/psychoacoustically-aligned-latent-smoothing-for-adversarial-robustness-of-full-duplex-speech-to-speech-dialogue-models.
هارفارد
Shamsaie, K. and Modarressi, I. (2026) 'Psychoacoustically Aligned Latent Smoothing for Adversarial Robustness of Full-Duplex Speech-to-Speech Dialogue Models', Available at: https://omanscience.com/ar/articles/psychoacoustically-aligned-latent-smoothing-for-adversarial-robustness-of-full-duplex-speech-to-speech-dialogue-models.
فانكوفر
Shamsaie K, Modarressi I. Psychoacoustically Aligned Latent Smoothing for Adversarial Robustness of Full-Duplex Speech-to-Speech Dialogue Models. https://omanscience.com/ar/articles/psychoacoustically-aligned-latent-smoothing-for-adversarial-robustness-of-full-duplex-speech-to-speech-dialogue-models
IEEE
K. Shamsaie, and I. Modarressi, "Psychoacoustically Aligned Latent Smoothing for Adversarial Robustness of Full-Duplex Speech-to-Speech Dialogue Models," https://omanscience.com/ar/articles/psychoacoustically-aligned-latent-smoothing-for-adversarial-robustness-of-full-duplex-speech-to-speech-dialogue-models.