الملخص
Modern AI systems are designed to refuse hazardous requests. A jailbreak is a prompt crafted to bypass those safeguards and elicit outputs that the system would normally refuse to provide. The MLCommons Jailbreak Benchmark v1.0 provides an end-to-end methodology for evaluating the robustness of large language models to single-turn, text-based jailbreak attacks. It combines criteria-driven system and attack selection, paired baseline and adversarial evaluation, human annotation, automated evaluator calibration, scoring, grading, and risk-calibrated disclosure within a single benchmarking pipeline. The benchmark evaluates eight open-weight systems using 264 seed prompts spanning eleven hazard categories and representative attacks drawn from the MLCommons Jailbreak Taxonomy. Responses are assessed using the AILuminate Assessment Standard v1.4, and robustness is measured through the Resilience Gap: the change in safety performance between baseline and adversarial conditions. Across all evaluated systems and attacks, the unsafe-response rate increased from 11.08% under baseline conditions to 18.65% under jailbreak conditions, producing an average Resilience Gap of 7.57%. Accessible systems showed a larger mean gap, while attack effectiveness varied substantially across attack categories and hazards. The benchmark also examines evaluator reliability and sources of measurement error. Beyond reporting results, Jailbreak Benchmark v1.0 establishes a reproducible methodological foundation for comparative jailbreak evaluation and for future expansion across systems, attacks, hazards, and evaluation methods.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Maple, C., Yucel, C., Holeman, I., Knotz, C., Mattson, P., Goel, J., Petit, J., McGregor, S., Ezick, J., Kumar, A., Parrish, A., Emani, M., Iyer, K., Khattak, F. K., Mbonu, W., Machlab, D., Long, E., Ghosh, S., Varghese, J., Lutz, R., Gruen, A., Hillenbrand, B., Gupta, P., Serrhini, M., Nagasubramanian, D., Gupta, A., Jun, Lu, Bollacker, K., Liu, C., Chen, C., Monteuuis, J. P., Miller, B., Verma, A., Eng, R., & Foundjem, A. (2026). MLCommons Jailbreak Benchmark v1.0. https://omanscience.com/ar/articles/mlcommons-jailbreak-benchmark-v1-0
MLA 9
Maple, Carsten, et al. "MLCommons Jailbreak Benchmark v1.0." https://omanscience.com/ar/articles/mlcommons-jailbreak-benchmark-v1-0.
شيكاغو (المؤلف–التاريخ)
Maple, Carsten, Cagatay Yucel, Isaac Holeman, Chris Knotz, Peter Mattson, James Goel, Jonathan Petit, Sean McGregor, James Ezick, Abhishek Kumar, Alicia Parrish, Murali Emani, Kashyap Iyer, Faiza Khan Khattak, Washington Mbonu, Daniel Machlab, Eileen Long, Shaona Ghosh, Jibin Varghese, Roman Lutz, Andrew Gruen, Bennett Hillenbrand, Prabal Gupta, Mohammed Serrhini, Dhivya Nagasubramanian, Aakash Gupta, Jun, Lu, Kurt Bollacker, Chang Liu, Cong Chen, Jean-Philippe Monteuuis, Brent Miller, Apurv Verma, Roman Eng, and Armstrong Foundjem. 2026. "MLCommons Jailbreak Benchmark v1.0." https://omanscience.com/ar/articles/mlcommons-jailbreak-benchmark-v1-0.
هارفارد
Maple, C., Yucel, C., Holeman, I., Knotz, C., Mattson, P., Goel, J., Petit, J., McGregor, S., Ezick, J., Kumar, A., Parrish, A., Emani, M., Iyer, K., Khattak, F. K., Mbonu, W., Machlab, D., Long, E., Ghosh, S., Varghese, J., Lutz, R., Gruen, A., Hillenbrand, B., Gupta, P., Serrhini, M., Nagasubramanian, D., Gupta, A., Jun, Lu, Bollacker, K., Liu, C., Chen, C., Monteuuis, J. P., Miller, B., Verma, A., Eng, R. and Foundjem, A. (2026) 'MLCommons Jailbreak Benchmark v1.0', Available at: https://omanscience.com/ar/articles/mlcommons-jailbreak-benchmark-v1-0.
فانكوفر
Maple C, Yucel C, Holeman I, Knotz C, Mattson P, Goel J, et al. MLCommons Jailbreak Benchmark v1.0. https://omanscience.com/ar/articles/mlcommons-jailbreak-benchmark-v1-0
IEEE
C. Maple, C. Yucel, I. Holeman, C. Knotz, P. Mattson, J. Goel, J. Petit, S. McGregor, J. Ezick, A. Kumar, A. Parrish, M. Emani, K. Iyer, F. K. Khattak, W. Mbonu, D. Machlab, E. Long, S. Ghosh, J. Varghese, R. Lutz, A. Gruen, B. Hillenbrand, P. Gupta, M. Serrhini, D. Nagasubramanian, A. Gupta, Jun, Lu, K. Bollacker, C. Liu, C. Chen, J. P. Monteuuis, B. Miller, A. Verma, R. Eng, and A. Foundjem, "MLCommons Jailbreak Benchmark v1.0," https://omanscience.com/ar/articles/mlcommons-jailbreak-benchmark-v1-0.