Abstract
Interactive CAPTCHAs remain challenging for computer-use agents, while existing datasets face trade-offs among type coverage, interaction fidelity, and trajectory supervision. To address these gaps, we present CaptchaArena, the first large-scale, fine-grained training dataset for interactive CAPTCHA solving. It contains 50K puzzles across 20 CAPTCHA types and 5 interaction modes, with every solution verified through execution. CaptchaArena provides 50K screenshot-action trajectories, including 46K with step-by-step reasoning annotations. It also includes fine-grained pixel-mask annotations for irregular targets. Using CaptchaArena, we train CaptchaAgent, a single 9B policy for all 20 CAPTCHA types, with supervised fine-tuning followed by reinforcement learning. The environment verifier directly provides the RL reward. Supervised fine-tuning reaches 70.5 Pass@1, and reinforcement learning further improves it to 71.7, while also improving performance on two external benchmarks. These results demonstrate the value of large-scale, fine-grained computer-use supervision for training interactive CAPTCHA agents. We release CaptchaArena and CaptchaAgent at https://github.com/X0X0X00/CaptchaArena.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Zhang, Z., Fan, Z., Ying, H., Hu, J., Fan, H., Zhou, J., Chen, Z., & Zhu, L. (2026). CaptchaArena: A Large-Scale, Fine-Grained Dataset for Training Computer-Use Agents on Interactive CAPTCHAs. https://omanscience.com/en/articles/captchaarena-a-large-scale-fine-grained-dataset-for-training-computer-use-agents-on-interactive-captchas
MLA 9
Zhang, Zhenhao, et al. "CaptchaArena: A Large-Scale, Fine-Grained Dataset for Training Computer-Use Agents on Interactive CAPTCHAs." https://omanscience.com/en/articles/captchaarena-a-large-scale-fine-grained-dataset-for-training-computer-use-agents-on-interactive-captchas.
Chicago (author–date)
Zhang, Zhenhao, Zhaoyu Fan, Haohan Ying, Jingwen Hu, Hancen Fan, Junhao Zhou, Zitian Chen, and Linchao Zhu. 2026. "CaptchaArena: A Large-Scale, Fine-Grained Dataset for Training Computer-Use Agents on Interactive CAPTCHAs." https://omanscience.com/en/articles/captchaarena-a-large-scale-fine-grained-dataset-for-training-computer-use-agents-on-interactive-captchas.
Harvard
Zhang, Z., Fan, Z., Ying, H., Hu, J., Fan, H., Zhou, J., Chen, Z. and Zhu, L. (2026) 'CaptchaArena: A Large-Scale, Fine-Grained Dataset for Training Computer-Use Agents on Interactive CAPTCHAs', Available at: https://omanscience.com/en/articles/captchaarena-a-large-scale-fine-grained-dataset-for-training-computer-use-agents-on-interactive-captchas.
Vancouver
Zhang Z, Fan Z, Ying H, Hu J, Fan H, Zhou J, et al. CaptchaArena: A Large-Scale, Fine-Grained Dataset for Training Computer-Use Agents on Interactive CAPTCHAs. https://omanscience.com/en/articles/captchaarena-a-large-scale-fine-grained-dataset-for-training-computer-use-agents-on-interactive-captchas
IEEE
Z. Zhang, Z. Fan, H. Ying, J. Hu, H. Fan, J. Zhou, Z. Chen, and L. Zhu, "CaptchaArena: A Large-Scale, Fine-Grained Dataset for Training Computer-Use Agents on Interactive CAPTCHAs," https://omanscience.com/en/articles/captchaarena-a-large-scale-fine-grained-dataset-for-training-computer-use-agents-on-interactive-captchas.