Abstract
Long chain-of-thought reasoning increases sequential decoding cost while creating a growing history of potentially reusable continuations. We investigate when this history supplies useful drafts and complements an existing drafter. Controlled source comparisons reveal trajectory-specific reuse, motivating our method Trajectory-Local Adaptive Retrieval (TLAR). TLAR retrieves approximately matched continuations from the current trajectory and uses recent verification outcomes to adapt retrieval activation and candidate width. TLAR combines retrieved continuations with model-generated drafts in a shared candidate tree, preserving the target model's output distribution through exact verification. Across code debugging, mathematics, and open-ended writing, our evaluation connects source reuse, incremental acceptance, and execution cost. Combining TLAR with strong retrieval baselines improves token acceptance under matched verification budgets and increases end-to-end throughput over the draft-model baseline. These findings support generated trajectories as runtime memory for adaptive inference.
Keywords
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Dai, Y., & Dong, Y. (2026). Trajectory-Retrieval Speculative Decoding: When Does a Model's Own History Help? https://omanscience.com/en/articles/trajectory-retrieval-speculative-decoding-when-does-a-model-s-own-history-help
MLA 9
Dai, Yuyang, and Yushun Dong. "Trajectory-Retrieval Speculative Decoding: When Does a Model's Own History Help?" https://omanscience.com/en/articles/trajectory-retrieval-speculative-decoding-when-does-a-model-s-own-history-help.
Chicago (author–date)
Dai, Yuyang, and Yushun Dong. 2026. "Trajectory-Retrieval Speculative Decoding: When Does a Model's Own History Help?" https://omanscience.com/en/articles/trajectory-retrieval-speculative-decoding-when-does-a-model-s-own-history-help.
Harvard
Dai, Y. and Dong, Y. (2026) 'Trajectory-Retrieval Speculative Decoding: When Does a Model's Own History Help?', Available at: https://omanscience.com/en/articles/trajectory-retrieval-speculative-decoding-when-does-a-model-s-own-history-help.
Vancouver
Dai Y, Dong Y. Trajectory-Retrieval Speculative Decoding: When Does a Model's Own History Help? https://omanscience.com/en/articles/trajectory-retrieval-speculative-decoding-when-does-a-model-s-own-history-help
IEEE
Y. Dai, and Y. Dong, "Trajectory-Retrieval Speculative Decoding: When Does a Model's Own History Help?," https://omanscience.com/en/articles/trajectory-retrieval-speculative-decoding-when-does-a-model-s-own-history-help.