Abstract
Benchmark scores license claims about the capabilities being evaluated. We audit the inference licensed by an evaluation protocol, rather than the predictive model alone. Using procedural action anticipation as a controlled case study, we study a broader evaluation-validity failure mode: a protocol can remain temporally causal and free of classical target leakage while still supplying a privileged intermediate representation. The failure is not merely optimistic accuracy, but a mismatch between the capability claimed and the construct actually measured. On Breakfast, matched recognizer-history and GT-history regimes score 30.3% and 62.3% (Delta_PH(R_BA) = +32.0, 95% Student-t interval [+24.1, +39.8]). A fixed-checkpoint 2 x 2 intervention isolates a +14.6-point test-time oracle contrast; a matched no-video provenance probe yields a +26.8-point gap; and a boundary-independent query/history control retains a +19.4-point gap. We refer to this discrepancy as a privileged-information gap, defined relative to a specified non-oracle recovery pipeline, and propose a reusable four-condition audit. Across five evaluation settings the gap is heterogeneous; a standard 50 Salads long-term-anticipation re-implementation provides cautious support beyond the audit-specific next-action construction.
Keywords
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Mohammadi, M., & Rowlands, S. (2026). Are We Measuring Anticipation? Auditing Privileged Information in Procedural Video Evaluation. https://omanscience.com/en/articles/are-we-measuring-anticipation-auditing-privileged-information-in-procedural-video-evaluation
MLA 9
Mohammadi, Mahsa, and Sareh Rowlands. "Are We Measuring Anticipation? Auditing Privileged Information in Procedural Video Evaluation." https://omanscience.com/en/articles/are-we-measuring-anticipation-auditing-privileged-information-in-procedural-video-evaluation.
Chicago (author–date)
Mohammadi, Mahsa, and Sareh Rowlands. 2026. "Are We Measuring Anticipation? Auditing Privileged Information in Procedural Video Evaluation." https://omanscience.com/en/articles/are-we-measuring-anticipation-auditing-privileged-information-in-procedural-video-evaluation.
Harvard
Mohammadi, M. and Rowlands, S. (2026) 'Are We Measuring Anticipation? Auditing Privileged Information in Procedural Video Evaluation', Available at: https://omanscience.com/en/articles/are-we-measuring-anticipation-auditing-privileged-information-in-procedural-video-evaluation.
Vancouver
Mohammadi M, Rowlands S. Are We Measuring Anticipation? Auditing Privileged Information in Procedural Video Evaluation. https://omanscience.com/en/articles/are-we-measuring-anticipation-auditing-privileged-information-in-procedural-video-evaluation
IEEE
M. Mohammadi, and S. Rowlands, "Are We Measuring Anticipation? Auditing Privileged Information in Procedural Video Evaluation," https://omanscience.com/en/articles/are-we-measuring-anticipation-auditing-privileged-information-in-procedural-video-evaluation.