Preprint Open access
Risk-Aware Adaptive Evaluation: Finding High-Impact Failures Under Limited Budgets
Evaluating interactive agents is expensive. Agent behavior is stochastic, so reliability must be measured over repeated trials, but failures are rare and differ widely in how much they matter. Standard benchmarks spend this budget uniformly: a read-only lookup is sampled as often as an irreversible payment action. We i …