Abstract
We develop a model-free policy gradient method for discrete-time mean-field control (MFC). In MFC, the policy affects the objective both through the controlled dynamics and through the population distribution. Standard REINFORCE estimators capture the first effect but not the second. We introduce Transport REINFORCE, a transport map-based approach that perturbs a suitable transformation of the population distribution to estimate this missing mean-field contribution. The method applies to both finite and continuous state spaces. In finite state spaces, we perturb the population distribution directly on the probability simplex through a convex combination of the current population weights and random weights. In continuous state spaces, we project the population distribution onto the manifold of Gaussian mixtures, and then randomize it via a transport map that ensures the perturbed law remains within this manifold. We prove consistency of the perturbed objective and gradient as the perturbation vanishes, and derive bias and mean-square error bounds for the resulting sample-based gradient estimator. Numerical experiments on several MFC benchmarks show that Transport REINFORCE improves over standard REINFORCE.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Jamal, A., Mekkaoui, S., Hafsi, Y., & Pham, H. (2026). Randomized Transport Maps for Model-Free Policy-Gradient Mean-Field Control. https://omanscience.com/en/articles/randomized-transport-maps-for-model-free-policy-gradient-mean-field-control
MLA 9
Jamal, Adonis, et al. "Randomized Transport Maps for Model-Free Policy-Gradient Mean-Field Control." https://omanscience.com/en/articles/randomized-transport-maps-for-model-free-policy-gradient-mean-field-control.
Chicago (author–date)
Jamal, Adonis, Samy Mekkaoui, Yadh Hafsi, and Huyên Pham. 2026. "Randomized Transport Maps for Model-Free Policy-Gradient Mean-Field Control." https://omanscience.com/en/articles/randomized-transport-maps-for-model-free-policy-gradient-mean-field-control.
Harvard
Jamal, A., Mekkaoui, S., Hafsi, Y. and Pham, H. (2026) 'Randomized Transport Maps for Model-Free Policy-Gradient Mean-Field Control', Available at: https://omanscience.com/en/articles/randomized-transport-maps-for-model-free-policy-gradient-mean-field-control.
Vancouver
Jamal A, Mekkaoui S, Hafsi Y, Pham H. Randomized Transport Maps for Model-Free Policy-Gradient Mean-Field Control. https://omanscience.com/en/articles/randomized-transport-maps-for-model-free-policy-gradient-mean-field-control
IEEE
A. Jamal, S. Mekkaoui, Y. Hafsi, and H. Pham, "Randomized Transport Maps for Model-Free Policy-Gradient Mean-Field Control," https://omanscience.com/en/articles/randomized-transport-maps-for-model-free-policy-gradient-mean-field-control.