Preprint Open access
ReproBench: Benchmarking LLM Agents on Reproducing Vulnerability From Scratch
Large language model (LLM) agents are increasingly evaluated on cybersecurity tasks such as vulnerability reproduction, exploitation, and patching. However, existing cybersecurity benchmarks predominantly operate under a post-environment evaluation paradigm, i.e., handing the agent source code, a container, or an execu …