This is the pre-proceedings for the RLC 2026. You may expect minor changes.
Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
Will be presented at the Reinforcement Learning Conference (RLC), Montréal, Quebec, Canada, August 15–17, 2026.
Keywords: Categorical Distributional Reinforcement Learning, Cardinal B-splines, Distributional
This paper proposes a regularization method for \emph{categorical distributional reinforcement learning} by generalizing the standard projection step to redistribute probability mass across multiple neighboring support locations using higher-order cardinal B-splines. The resulting operator induces a parameterized regularization of the categorical return representation. We establish convergence guarantees for the corresponding projected distributional Bellman operator in the policy evaluation setting. The proposed approach yields a more dispersed representation of the return distribution. This can be interpreted as an uncertainty-aware modeling choice or as an explicit regularization mechanism that may improve training performance. We demonstrate these effects empirically in both evaluation and control settings
André Inge, Jonas Nordqvist, Björn Lindenberg, and Karl-Olof Lindahl. "Representation Regularization in Distributional Reinforcement Learning." Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
BibTeX:@article{inge2026representation,
title={Representation Regularization in Distributional Reinforcement Learning},
author={André Inge and Jonas Nordqvist and Björn Lindenberg and Karl-Olof Lindahl},
journal={Reinforcement Learning Journal},
volume={7},
pages={},
year={2026}
}