Preprint Open access
Nucleus Speculative Decoding: Plausibility-Aware Verification Beyond Exact Distribution
Speculative decoding accelerates autoregressive generation by using a lightweight draft model to propose multiple tokens that are verified by a target model in parallel. However, the standard acceptance rule focuses on exact distribution correction and rejects tokens that remain highly plausible under the target model …