Abstract
Advances in large language models (LLMs) have enabled AI-driven code generation from natural language specifications, introducing new attack surfaces for injecting vulnerabilities into software. Prior work has studied this problem only in benign settings where vulnerabilities are introduced inadvertently, or under unconventional threat models where the LLM itself is malicious (backdooring) or the user is the attacker (jailbreaking). In this paper, we study a more realistic threat model: a third-party adversary, with capabilities comparable to existing cybercriminals, compromises the AI code generation pipeline to deliberately introduce vulnerabilities. We call this the EviLLM attack. We have implemented two instances of EviLLM, each of which only requires the underlying LLM to be accessed through a compromised account or browser, and can inject vulnerabilities from 13 CWE classes. As we show in our feasibility study, both attack vectors are already used to implement many existing cyberattacks. Our user study shows that 7 out of 8 and 10 out of 13 participants did not notice the vulnerabilities injected by the two instances of EviLLM, and 13 out of 21 participants "rarely" or "never" considered the risk of an attack like EviLLM.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Ryu, Z., Chung, S., Karim, M. F., Raymaker, A., Jodha, K. S., Chaturvedi, Y., & Mertoguno, S. (2026). Beware EviLLM: Enabling Vulnerability Injection via Large Language Models. https://omanscience.com/en/articles/beware-evillm-enabling-vulnerability-injection-via-large-language-models
MLA 9
Ryu, Zeezoo, et al. "Beware EviLLM: Enabling Vulnerability Injection via Large Language Models." https://omanscience.com/en/articles/beware-evillm-enabling-vulnerability-injection-via-large-language-models.
Chicago (author–date)
Ryu, Zeezoo, Simon Chung, Muhammad Faraz Karim, Anna Raymaker, Karan Singh Jodha, Yash Chaturvedi, and Sukarno Mertoguno. 2026. "Beware EviLLM: Enabling Vulnerability Injection via Large Language Models." https://omanscience.com/en/articles/beware-evillm-enabling-vulnerability-injection-via-large-language-models.
Harvard
Ryu, Z., Chung, S., Karim, M. F., Raymaker, A., Jodha, K. S., Chaturvedi, Y. and Mertoguno, S. (2026) 'Beware EviLLM: Enabling Vulnerability Injection via Large Language Models', Available at: https://omanscience.com/en/articles/beware-evillm-enabling-vulnerability-injection-via-large-language-models.
Vancouver
Ryu Z, Chung S, Karim MF, Raymaker A, Jodha KS, Chaturvedi Y, et al. Beware EviLLM: Enabling Vulnerability Injection via Large Language Models. https://omanscience.com/en/articles/beware-evillm-enabling-vulnerability-injection-via-large-language-models
IEEE
Z. Ryu, S. Chung, M. F. Karim, A. Raymaker, K. S. Jodha, Y. Chaturvedi, and S. Mertoguno, "Beware EviLLM: Enabling Vulnerability Injection via Large Language Models," https://omanscience.com/en/articles/beware-evillm-enabling-vulnerability-injection-via-large-language-models.