This is the pre-proceedings for the RLC 2026. You may expect minor changes.
Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
Will be presented at the Reinforcement Learning Conference (RLC), Montréal, Quebec, Canada, August 15–17, 2026.
Keywords: Multi-Agent System, Reinforcement Learning, Centralized Training with Decentralized
In partially observable multi-agent systems following the centralized training with decentralized execution (CTDE) paradigm, agents have access only to local observations during execution, while global state information is available during training but inaccessible at deployment. To mitigate this information gap, we introduce a novel framework, State Inference with Diffusion Models (SIDIFF). Drawing inspiration from image outpainting in computer vision, SIDIFF leverages diffusion models as state generators to reconstruct the complete global state from local observations, under the assumption that the global structure is recoverable from partial views. In addition, the global state may contain redundant components, necessitating the introduction of an additional state extractor to derive decision-relevant information from the reconstructed global state. By jointly conditioning on the reconstructed global state and local observations, SIDIFF enables agents to select more appropriate actions during decentralized execution, thereby narrowing the training-execution information discrepancy. Furthermore, SIDIFF can be effortlessly incorporated into current multi-agent reinforcement learning algorithms to improve their performance. Extensive evaluations on widely used partially observable benchmarks, as well as on the newly proposed Multi-Agent Battle City (MABC) environment, demonstrate consistent performance improvements over baselines.
Zhiwei Xu, Hangyu Mao, ZHANG NIANMIN, Shengtao Zhang, Xin Xin, Pengjie Ren, Dapeng Li, Bin Zhang, Guoliang Fan, Zhumin Chen, Changwei Wang, and Jiangjin Yin. "Beyond Local Views: Global State Inference with Diffusion Models for Cooperative MARL." Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
BibTeX:@article{xu2026beyond,
title={Beyond Local Views: Global State Inference with Diffusion Models for Cooperative MARL},
author={Zhiwei Xu and Hangyu Mao and ZHANG NIANMIN and Shengtao Zhang and Xin Xin and Pengjie Ren and Dapeng Li and Bin Zhang and Guoliang Fan and Zhumin Chen and Changwei Wang and Jiangjin Yin},
journal={Reinforcement Learning Journal},
volume={7},
pages={},
year={2026}
}