Abstract
Self-consistency samples many reasoning trajectories and aggregates their final answers, treating the LLM as a black box that returns one answer per trajectory. Yet the final answer of each trajectory is sampled from a softmax vector that is available from the model's log-probabilities. We refer to this as the grey-box setting in which each trajectory reveals this answer distribution rather than a single draw from it. We formulate efficient inference in this setting as sequential mode identification with distribution-valued observations: sample trajectories one at a time and stop as soon as the LLM's modal answer is identified at a prescribed confidence level. We characterize the asymptotic stopping rate of mode identification with distribution-valued observations exactly and show that it is never worse than the black-box rate. We then propose the ASC-D algorithm, a betting stopping rule that attains this asymptotic stopping rate. On MMLU-Redux, ASC-D uses $46.4$--$95.6\%$ fewer trajectories than answer-only adaptive self-consistency baselines and achieves the highest fixed-budget correct-certification rate across three open-source models.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Huang, J., Zhang, Y., Ma, W., Zhou, W., & Zhou, Z. (2026). Adaptive Self-Consistency: From Black-Box Sampling to Distribution-Valued Feedback. https://omanscience.com/en/articles/adaptive-self-consistency-from-black-box-sampling-to-distribution-valued-feedback
MLA 9
Huang, Jingkai, et al. "Adaptive Self-Consistency: From Black-Box Sampling to Distribution-Valued Feedback." https://omanscience.com/en/articles/adaptive-self-consistency-from-black-box-sampling-to-distribution-valued-feedback.
Chicago (author–date)
Huang, Jingkai, Yunfan Zhang, Will Ma, Weihua Zhou, and Zhengyuan Zhou. 2026. "Adaptive Self-Consistency: From Black-Box Sampling to Distribution-Valued Feedback." https://omanscience.com/en/articles/adaptive-self-consistency-from-black-box-sampling-to-distribution-valued-feedback.
Harvard
Huang, J., Zhang, Y., Ma, W., Zhou, W. and Zhou, Z. (2026) 'Adaptive Self-Consistency: From Black-Box Sampling to Distribution-Valued Feedback', Available at: https://omanscience.com/en/articles/adaptive-self-consistency-from-black-box-sampling-to-distribution-valued-feedback.
Vancouver
Huang J, Zhang Y, Ma W, Zhou W, Zhou Z. Adaptive Self-Consistency: From Black-Box Sampling to Distribution-Valued Feedback. https://omanscience.com/en/articles/adaptive-self-consistency-from-black-box-sampling-to-distribution-valued-feedback
IEEE
J. Huang, Y. Zhang, W. Ma, W. Zhou, and Z. Zhou, "Adaptive Self-Consistency: From Black-Box Sampling to Distribution-Valued Feedback," https://omanscience.com/en/articles/adaptive-self-consistency-from-black-box-sampling-to-distribution-valued-feedback.