This is the pre-proceedings for the RLC 2026. You may expect minor changes.
Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
Will be presented at the Reinforcement Learning Conference (RLC), Montréal, Quebec, Canada, August 15–17, 2026.
Keywords: Monitored-MDPs, Partially observable rewards, Generalization, Function
Reinforcement learning (RL) typically models the interaction between the agent and environment as a Markov decision process (MDP), assuming fully observable rewards. In many real-world settings, this assumption fails, motivating the monitored Markov decision process (Mon-MDP), where rewards may be unobserved. Prior work on Mon-MDPs has been limited to simple, tabular cases, restricting their applicability to real-world problems. This work explores Mon-MDPs using function approximation and investigates the challenges. We show that combining function approximation with a learned reward model enables agents to generalize from monitored states with observable rewards to unmonitored states with unobservable rewards. Therefore, we demonstrate that such generalization with a reward model achieves near-optimal policies in environments formally defined as unsolvable. However, we also uncover a critical limitation: agents may incorrectly extrapolate rewards due to overgeneralization, which can lead to undesirable behaviors. To mitigate overgeneralization, we propose a cautious policy optimization method leveraging reward uncertainty. This work serves as a step towards bridging the gap between Mon-MDP theory and real-world applications.
Montaser Mohammedalamen and Michael Bowling. "Generalization in Monitored Markov Decision Processes (Mon-MDPs)." Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
BibTeX:@article{mohammedalamen2026generalization,
title={Generalization in Monitored Markov Decision Processes (Mon-MDPs)},
author={Montaser Mohammedalamen and Michael Bowling},
journal={Reinforcement Learning Journal},
volume={7},
pages={},
year={2026}
}