This is the pre-proceedings for the RLC 2026. You may expect minor changes.
Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
Will be presented at the Reinforcement Learning Conference (RLC), Montréal, Quebec, Canada, August 15–17, 2026.
Keywords: Online Robust Reinforcement Learning, Linear Function Approximation
We study off-dynamics reinforcement learning (RL), where the policy training and deployment environments are different. To deal with this environmental perturbation, we focus on learning policies robust to uncertainties in transition dynamics under the framework of distributionally robust Markov decision processes (DRMDPs), where the nominal and perturbed dynamics are linear Markov Decision Processes. We propose a novel algorithm, We-DRIVE-U, that enjoys an average suboptimality $\widetilde{\mathcal{O}}\big({d H \cdot \min (1/{\rho}, H)/\sqrt{K} }\big)$, where $K$ is the number of episodes, $H$ is the horizon length, $d$ is the feature dimension and $\rho$ is the uncertainty level. This result improves the state-of-the-art by $\mathcal{O}(dH/\min(1/\rho,H))$. We also construct a novel hard instance and derive the first information-theoretic lower bound in this setting. In stark contrast with standard linear MDPs, our lower bound depends on the uncertainty level $\rho$, revealing the unique feature of DRMDPs. Our algorithm also enjoys a `rare-switching' design, requiring only $\mathcal{O}(dH\log(1+H^2K))$ policy switches and $\mathcal{O}(d^2H\log(1+H^2K))$ calls of the dual optimization oracle, which significantly improves the computational efficiency of existing algorithms, whose policy switch and oracle complexities are both $\mathcal{O}(K)$.
Zhishuai Liu, Weixin Wang, and Pan Xu. "Near-Optimal Reinforcement Learning for Linear Distributionally Robust Markov Decision Processes." Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
BibTeX:@article{liu2026nearoptimal,
title={Near-Optimal Reinforcement Learning for Linear Distributionally Robust Markov Decision Processes},
author={Zhishuai Liu and Weixin Wang and Pan Xu},
journal={Reinforcement Learning Journal},
volume={7},
pages={},
year={2026}
}