Authors

Rex Ying

Publications 5

Preprint Open access

Component-Weighted Centroid Search for Exact Incremental BPE

Exact incremental BPE maintains the canonical tokenization state after every appended byte. The recent algorithm of Jiang and Gong (2026) does this in $O(\log^2 t)$ worst-case time, where $t$ is the maximum canonical token length. Its centroid search visits $O(\log t)$ components and can pay another $O(\log t)$ for ord …

Co-authors