This is the pre-proceedings for the RLC 2026. You may expect minor changes.
Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
Will be presented at the Reinforcement Learning Conference (RLC), Montréal, Quebec, Canada, August 15–17, 2026.
Keywords: imitation learning, reward assignment, optimal transport, offline RL, online RL
Reward assignment from scarce demonstrations is a key challenge in both offline and online imitation learning. A common and intuitive strategy assigns rewards according to how closely learner trajectories match expert demonstrations. Although this principle underlies many existing methods, the core ingredients that drive performance remain systematically underexplored. We therefore ask: what is the minimal structure that reward assignment must encode to achieve effective downstream RL performance across settings? We approach this question along two design axes: _proximity approximation_ and _temporal alignment_. Across 32 benchmarks spanning offline and online settings, and with three downstream RL algorithms, our empirical findings suggest: (1) In offline regimes, proximity alone captures the reward structure necessary for effective offline RL, while (2) lightweight temporal correspondence provides consistent gains that are modest offline but essential online or in the presence of multiple demonstrations. We further complement our offline results with a lightweight theory characterizing when simple proximity approximation suffices. Overall, these findings advocate algorithmic minimalism in reward design before introducing complex schemes in both offline and online imitation learning.
Zixuan Dong, Yumi Omori, and Keith W. Ross. "Minimal Ingredients for Reward Assignment from Expert Demonstrations." Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
BibTeX:@article{dong2026minimal,
title={Minimal Ingredients for Reward Assignment from Expert Demonstrations},
author={Zixuan Dong and Yumi Omori and Keith W. Ross},
journal={Reinforcement Learning Journal},
volume={7},
pages={},
year={2026}
}