Abstract
In cooperative Multi-Agent Reinforcement Learning (MARL), agents are often trained under concurrent participation, while in many tasks some agents act earlier and leave task-relevant information that becomes useful to agents participating later. We study this setting as staggered participation (SP), which introduces a cross-time, cross-agent learning dependency because an early action may affect the return through the information it provides and the later policy that uses it. Learning under SP therefore requires both identifying what information is useful for future decisions and learning how later agents should use it. We propose Staggered Participation Learning (SPL), a training-time augmentation that addresses these two parts with prospective acquisition supervision for earlier agents and outcome-supervised receiver learning for later agents. We evaluate SPL across multiple policy-based MARL backbones, environments, and staggered-participation patterns. Across 60 MPE/RWARE backbone setting comparisons, SPL achieves higher observed mean task completion in every case, with an average difference of 14.1%. The gains also extend to eight-agent teams and a physics-based UAV-UGV environment in Isaac Lab, providing evidence across algorithmic, temporal, and embodied settings.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Qiao, J., Hu, S., Nguyen, T. H., Cao, Z., & Sukkarieh, S. (2026). Cooperating with Future Collaborators: Multi-Agent RL under Staggered Participation. https://omanscience.com/en/articles/cooperating-with-future-collaborators-multi-agent-rl-under-staggered-participation
MLA 9
Qiao, Jianglin, et al. "Cooperating with Future Collaborators: Multi-Agent RL under Staggered Participation." https://omanscience.com/en/articles/cooperating-with-future-collaborators-multi-agent-rl-under-staggered-participation.
Chicago (author–date)
Qiao, Jianglin, Siyi Hu, Thien Hoang Nguyen, Zehong Cao, and Salah Sukkarieh. 2026. "Cooperating with Future Collaborators: Multi-Agent RL under Staggered Participation." https://omanscience.com/en/articles/cooperating-with-future-collaborators-multi-agent-rl-under-staggered-participation.
Harvard
Qiao, J., Hu, S., Nguyen, T. H., Cao, Z. and Sukkarieh, S. (2026) 'Cooperating with Future Collaborators: Multi-Agent RL under Staggered Participation', Available at: https://omanscience.com/en/articles/cooperating-with-future-collaborators-multi-agent-rl-under-staggered-participation.
Vancouver
Qiao J, Hu S, Nguyen TH, Cao Z, Sukkarieh S. Cooperating with Future Collaborators: Multi-Agent RL under Staggered Participation. https://omanscience.com/en/articles/cooperating-with-future-collaborators-multi-agent-rl-under-staggered-participation
IEEE
J. Qiao, S. Hu, T. H. Nguyen, Z. Cao, and S. Sukkarieh, "Cooperating with Future Collaborators: Multi-Agent RL under Staggered Participation," https://omanscience.com/en/articles/cooperating-with-future-collaborators-multi-agent-rl-under-staggered-participation.