This is the pre-proceedings for the RLC 2026. You may expect minor changes.
Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
Will be presented at the Reinforcement Learning Conference (RLC), Montréal, Quebec, Canada, August 15–17, 2026.
Keywords: predictive safety filter, model-based reinforcement learning, probabilistic ensembles,
Predictive safety filters (PSFs) leverage model predictive control to enforce constraint satisfaction during deep reinforcement learning (RL) exploration, yet their reliance on first-principles models or Gaussian processes limits scalability and broader applicability. Meanwhile, model-based RL (MBRL) methods routinely employ probabilistic ensemble (PE) neural networks to capture complex, high-dimensional dynamics from data with minimal prior knowledge. However, existing attempts to integrate PEs into PSFs lack rigorous uncertainty quantification. We introduce the Uncertainty-Aware Predictive Safety Filter (UPSi), a PSF that provides rigorous safety verification using PE dynamics models by formulating future outcomes as reachable sets. UPSi introduces an explicit certainty constraint that prevents model exploitation and integrates seamlessly into common MBRL frameworks. We evaluate UPSi under practical simplifications within Dyna-style MBRL on standard safe RL benchmarks and report substantial improvements in exploration safety over prior neural network PSFs while maintaining performance on par with standard MBRL. UPSi bridges the gap between the scalability and generality of modern MBRL and the safety guarantees of predictive safety filters.
Bernd Frauenknecht, Lukas Kesper, Daniel Mayfrank, Henrik Hose, and Sebastian Trimpe. "Uncertainty-Aware Predictive Safety Filters for Probabilistic Neural Network Dynamics." Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
BibTeX:@article{frauenknecht2026uncertaintyaware,
title={Uncertainty-Aware Predictive Safety Filters for Probabilistic Neural Network Dynamics},
author={Bernd Frauenknecht and Lukas Kesper and Daniel Mayfrank and Henrik Hose and Sebastian Trimpe},
journal={Reinforcement Learning Journal},
volume={7},
pages={},
year={2026}
}