Abstract
Indirect prompt injection embeds malicious instructions within external content retrieved by LLM-based agents, altering target behavior without user authorization. We introduce pikit, a research toolkit designed to systematically evaluate these threats across three core dimensions: attacks (13 methods), channels (16 carriers across text and file modes), and defenses (9 prevention strategies and 3 offline detection baselines). Built on a decorator-based registry, pikit enables seamless extension of custom components without modifying core code, while a unified craft() API composes arbitrary attacks and channels in a single call. We evaluated the toolkit on the pi coding agent powered by an anonymized LLM in a production-like environment. Benchmarking 9 prevention strategies against high-risk attacks yields a 71.8\% relative reduction in attack success rate, with few\_shot\_warning and instruction\_hierarchy providing the strongest protection. Offline detection baselines achieve perfect precision but low recall, demonstrating that heuristic detectors complement rather than replace prompt-level defenses. To ensure reproducibility, each run automatically logs full prompts, agent event traces, session transcripts, and verdict records. Our code is available at https://github.com/Tencent/AI-Infra-Guard/tree/main/Research/pikit.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Ying, Z., Wu, X., Yang, B., Wu, H., Zheng, X., Cheng, H., Shi, X., & Guo, J. (2026). pikit: A Composable Toolkit for Indirect Prompt Injection Research and Evaluation. https://omanscience.com/en/articles/pikit-a-composable-toolkit-for-indirect-prompt-injection-research-and-evaluation
MLA 9
Ying, Zonghao, et al. "pikit: A Composable Toolkit for Indirect Prompt Injection Research and Evaluation." https://omanscience.com/en/articles/pikit-a-composable-toolkit-for-indirect-prompt-injection-research-and-evaluation.
Chicago (author–date)
Ying, Zonghao, Xiangfan Wu, Bo Yang, Huiyu Wu, Xing Zheng, Huangsheng Cheng, Xiaorong Shi, and Jing Guo. 2026. "pikit: A Composable Toolkit for Indirect Prompt Injection Research and Evaluation." https://omanscience.com/en/articles/pikit-a-composable-toolkit-for-indirect-prompt-injection-research-and-evaluation.
Harvard
Ying, Z., Wu, X., Yang, B., Wu, H., Zheng, X., Cheng, H., Shi, X. and Guo, J. (2026) 'pikit: A Composable Toolkit for Indirect Prompt Injection Research and Evaluation', Available at: https://omanscience.com/en/articles/pikit-a-composable-toolkit-for-indirect-prompt-injection-research-and-evaluation.
Vancouver
Ying Z, Wu X, Yang B, Wu H, Zheng X, Cheng H, et al. pikit: A Composable Toolkit for Indirect Prompt Injection Research and Evaluation. https://omanscience.com/en/articles/pikit-a-composable-toolkit-for-indirect-prompt-injection-research-and-evaluation
IEEE
Z. Ying, X. Wu, B. Yang, H. Wu, X. Zheng, H. Cheng, X. Shi, and J. Guo, "pikit: A Composable Toolkit for Indirect Prompt Injection Research and Evaluation," https://omanscience.com/en/articles/pikit-a-composable-toolkit-for-indirect-prompt-injection-research-and-evaluation.