This is the pre-proceedings for the RLC 2026. You may expect minor changes.
Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
Will be presented at the Reinforcement Learning Conference (RLC), Montréal, Quebec, Canada, August 15–17, 2026.
Keywords: Safe reinforcement learning, offline reinforcement learning, flow matching,
Offline safe reinforcement learning (RL) seeks reward-maximizing policies from static datasets under strict safety constraints. Existing methods often rely on soft expected-cost objectives or iterative generative inference, which can be insufficient for safety-critical real-time control. We propose Safe Flow Q-Learning (SafeFQL), which extends FQL to safe offline RL by combining a Hamilton--Jacobi reachability-inspired safety value function with an efficient one-step flow policy. SafeFQL learns the safety value via a self-consistency Bellman recursion, trains a flow policy by behavioral cloning, and distills it into a one-step actor for reward-maximizing safe action selection without rejection sampling at deployment. Empirically, SafeFQL trades modestly higher offline training cost for substantially lower inference latency than diffusion-style safe generative baselines, which is advantageous for real-time safety-critical deployment. Across boat navigation, and Safety Gymnasium MuJoCo tasks, SafeFQL matches or exceeds prior offline safe RL performance while substantially reducing constraint violations.
Mumuksh Tayal, Manan Tayal, and Ravi Prakash. "Safe Flow Q-Learning: Offline Safe Reinforcement Learning with Reachability-Based Flow Policies." Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
BibTeX:@article{tayal2026safe,
title={Safe Flow Q-Learning: Offline Safe Reinforcement Learning with Reachability-Based Flow Policies},
author={Mumuksh Tayal and Manan Tayal and Ravi Prakash},
journal={Reinforcement Learning Journal},
volume={7},
pages={},
year={2026}
}