Preprint Open access
BitNest: Bit-Nested Speculative Decoding for Memory-Efficient LLM Inference Acceleration
Speculative decoding accelerates autoregressive generation by using a lightweight draft to propose multiple tokens for parallel verification. However, existing methods often require an additional draft model or weight representation, introducing non-negligible memory overhead on resource-constrained devices. Self-specu …