This is the pre-proceedings for the RLC 2026. You may expect minor changes.
Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
Will be presented at the Reinforcement Learning Conference (RLC), Montréal, Quebec, Canada, August 15–17, 2026.
Keywords: reinforcement learning, multi-goal, diverse goal coverage.
Reinforcement Learning algorithms are primarily focused on learning a policy that maximizes expected return. As a result, the learned policy can exploit one or few reward sources. However, in many natural situations, it is desirable to learn a policy that induces a dispersed marginal state distribution over rewarding states, while maximizing the expected return which is typically tied to visiting a goal state. This aspect remains relatively unexplored. Existing techniques based on entropy regularization and intrinsic rewards use stochasticity for encouraging exploration to find an optimal policy which may not necessarily lead to dispersed marginal state distribution over rewarding states. Other RL algorithms which match a target distribution assume the latter to be available apriori. This may be infeasible in large scale systems where enumeration of all states is not possible and a state is determined to be a goal state only upon visiting it. We formalize the problem of frequently and uniformly visiting the goal states as Multi Goal RL, and propose a novel algorithm that learns a high-return policy mixture with marginal state distribution dispersed over the set of goal states. Our algorithm is based on optimizing a custom RL reward which is computed - based on the current policy mixture - at each iteration for a set of sampled trajectories. The latter are used via an offline RL algorithm to update the policy mixture. We prove performance guarantees for our algorithm, showing efficient convergence bounds for optimizing a natural objective which captures the expected return as well as the dispersion of the marginal state distribution over the goal states. We design and perform experiments on a synthetic MDP and MuJoCo environments to evaluate the effectiveness of our algorithm.
Sagalpreet Singh, Rishi Saket, and Aravindan Raghuveer. "Dense and Diverse Goal Coverage in Multi Goal Reinforcement Learning." Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
BibTeX:@article{singh2026dense,
title={Dense and Diverse Goal Coverage in Multi Goal Reinforcement Learning},
author={Sagalpreet Singh and Rishi Saket and Aravindan Raghuveer},
journal={Reinforcement Learning Journal},
volume={7},
pages={},
year={2026}
}