This is the pre-proceedings for the RLC 2026. You may expect minor changes.
Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
Will be presented at the Reinforcement Learning Conference (RLC), Montréal, Quebec, Canada, August 15–17, 2026.
Keywords: MBRL, latent dynamics models, epistemic uncertainty, RSSM
Model-based reinforcement learning distinguishes between dynamics models operating on proprioceptive states and latent dynamics models typically operating on high-dimensional image observations. Among the latter, Dreamer's Recurrent State Space Model (RSSM) has emerged as a dominant architecture. While ensemble-based epistemic uncertainty has proven effective in proprioceptive dynamics for mitigating model exploitation, guiding exploration, or promoting caution, its behavior in latent dynamics remains largely unexplored. Our experiments reveal that although ensemble disagreement captures local epistemic uncertainty, it does not reliably reflect global compounding model error accumulated over prolonged RSSM latent rollouts. We provide evidence for an attractor behavior that draws rollouts toward well-supported latent regions, where uncertainty diminishes despite increasing discrepancies from the true environment dynamics. This can cause the model to overestimate returns when attractor regions correspond to high-reward behaviors. Our findings reveal a structural limitation of epistemic uncertainty estimation in RSSMs and challenge the assumption that epistemic uncertainty estimation transfers directly from proprioceptive to latent dynamics.
Julia Berger, Bernd Frauenknecht, Sebastian Trimpe, and Bastian Leibe. "Biased Dreams: Limitations to Epistemic Uncertainty Quantification in Latent Dynamics Models." Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
BibTeX:@article{berger2026biased,
title={Biased Dreams: Limitations to Epistemic Uncertainty Quantification in Latent Dynamics Models},
author={Julia Berger and Bernd Frauenknecht and Sebastian Trimpe and Bastian Leibe},
journal={Reinforcement Learning Journal},
volume={7},
pages={},
year={2026}
}