Abstract
We consider estimating the one-step-ahead conditional distribution of a multivariate stochastic process. Many existing approaches rely on assumptions such as finite-range memory, sparsity, or additivity, which can be poorly suited to processes with long-range nonlinear interactions. However, without such structural assumptions, nonparametric estimation is challenging due to the curse of dimensionality. To address this challenge, we introduce a new estimation approach based on the predictive states of a process, possibly with infinite-range memory. We show that our estimator achieves fast convergence rates when the past history can be compressed into a low-dimensional statistic that is sufficient for predicting the future. Specifically, we show that the statistical complexity of the estimation problem is determined by the intrinsic dimension of the predictive state space. We establish guarantees for an instantiation of our method based on deep neural network estimators, and we support these theoretical results with experiments.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Wieck-Sosa, M., & Shalizi, C. R. (2026). Generative sequence modeling for infinite memory processes via predictive states. https://omanscience.com/en/articles/generative-sequence-modeling-for-infinite-memory-processes-via-predictive-states
MLA 9
Wieck-Sosa, Michael, and Cosma Rohilla Shalizi. "Generative sequence modeling for infinite memory processes via predictive states." https://omanscience.com/en/articles/generative-sequence-modeling-for-infinite-memory-processes-via-predictive-states.
Chicago (author–date)
Wieck-Sosa, Michael, and Cosma Rohilla Shalizi. 2026. "Generative sequence modeling for infinite memory processes via predictive states." https://omanscience.com/en/articles/generative-sequence-modeling-for-infinite-memory-processes-via-predictive-states.
Harvard
Wieck-Sosa, M. and Shalizi, C. R. (2026) 'Generative sequence modeling for infinite memory processes via predictive states', Available at: https://omanscience.com/en/articles/generative-sequence-modeling-for-infinite-memory-processes-via-predictive-states.
Vancouver
Wieck-Sosa M, Shalizi CR. Generative sequence modeling for infinite memory processes via predictive states. https://omanscience.com/en/articles/generative-sequence-modeling-for-infinite-memory-processes-via-predictive-states
IEEE
M. Wieck-Sosa, and C. R. Shalizi, "Generative sequence modeling for infinite memory processes via predictive states," https://omanscience.com/en/articles/generative-sequence-modeling-for-infinite-memory-processes-via-predictive-states.