Abstract
Warning: This paper contains examples of stereotypes and social bias. LLMs are increasingly used in interactive settings by the general public, making the evaluation of model behavior in multi-turn conversational scenarios important for safety, including stereotyping-related harms. However, existing multi-turn social bias evaluations often rely on pre-specified or template-based user inputs that do not adapt to model responses and typically assume a fixed dialogue length in advance. In this paper, we study social bias dynamics in response-conditioned multi-turn interactions using a controlled evaluation protocol that generates follow-up user queries from the evolving dialogue history and allows evaluation over variable numbers of turns. Experimental results show that LLMs exhibit social bias even in coherent, response-conditioned multi-turn interactions, revealing late-emerging bias, non-monotonic bias patterns, and bias re-emergence. These results motivate evaluations that extend beyond fixed-turn, pre-scripted protocols. Our findings highlight the importance of analyzing social bias as a turn-level dynamic phenomenon.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Hida, R., Kaneko, M., Oba, D., Bollegala, D., & Okazaki, N. (2026). DyMT-ESB: Dynamic Multi-Turn Evaluation of Social Bias in User-LLM Interactions. https://omanscience.com/en/articles/dymt-esb-dynamic-multi-turn-evaluation-of-social-bias-in-user-llm-interactions
MLA 9
Hida, Rem, et al. "DyMT-ESB: Dynamic Multi-Turn Evaluation of Social Bias in User-LLM Interactions." https://omanscience.com/en/articles/dymt-esb-dynamic-multi-turn-evaluation-of-social-bias-in-user-llm-interactions.
Chicago (author–date)
Hida, Rem, Masahiro Kaneko, Daisuke Oba, Danushka Bollegala, and Naoaki Okazaki. 2026. "DyMT-ESB: Dynamic Multi-Turn Evaluation of Social Bias in User-LLM Interactions." https://omanscience.com/en/articles/dymt-esb-dynamic-multi-turn-evaluation-of-social-bias-in-user-llm-interactions.
Harvard
Hida, R., Kaneko, M., Oba, D., Bollegala, D. and Okazaki, N. (2026) 'DyMT-ESB: Dynamic Multi-Turn Evaluation of Social Bias in User-LLM Interactions', Available at: https://omanscience.com/en/articles/dymt-esb-dynamic-multi-turn-evaluation-of-social-bias-in-user-llm-interactions.
Vancouver
Hida R, Kaneko M, Oba D, Bollegala D, Okazaki N. DyMT-ESB: Dynamic Multi-Turn Evaluation of Social Bias in User-LLM Interactions. https://omanscience.com/en/articles/dymt-esb-dynamic-multi-turn-evaluation-of-social-bias-in-user-llm-interactions
IEEE
R. Hida, M. Kaneko, D. Oba, D. Bollegala, and N. Okazaki, "DyMT-ESB: Dynamic Multi-Turn Evaluation of Social Bias in User-LLM Interactions," https://omanscience.com/en/articles/dymt-esb-dynamic-multi-turn-evaluation-of-social-bias-in-user-llm-interactions.