A Finite-Time Analysis of Distributed Q-Learning

By Han-Dong Lim, and Donghwan Lee

Reinforcement Learning Journal, vol. TBD, 2025, pp. TBD.

Presented at the Reinforcement Learning Conference (RLC), Edmonton, Alberta, Canada, August 5–9, 2025.

Download:

Abstract:

Multi-agent reinforcement learning (MARL) has witnessed a remarkable surge in interest, fueled by the empirical success achieved in applications of single-agent reinforcement learning (RL). In this study, we consider a distributed Q-learning scenario, wherein a number of agents cooperatively solve a sequential decision making problem without access to the central reward function which is an average of the local rewards. In particular, we study finite-time analysis of a distributed Q-learning algorithm, and provide a new sample complexity result under tabular lookup setting for Markovian observation model.

Citation Information:

Han-Dong Lim and Donghwan Lee. "A Finite-Time Analysis of Distributed Q-Learning." Reinforcement Learning Journal, vol. TBD, 2025, pp. TBD.

BibTeX:
@article{lim2025finite,
    title={A Finite-Time Analysis of Distributed {Q-Learning}},
    author={Lim, Han-Dong and Lee, Donghwan},
    journal={Reinforcement Learning Journal},
    year={2025}
}