This is the pre-proceedings for the RLC 2026. You may expect minor changes.
Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
Will be presented at the Reinforcement Learning Conference (RLC), Montréal, Quebec, Canada, August 15–17, 2026.
Keywords: Linear bandits, offline-to-online, regret minimization
We study online learning with an additional offline dataset in the stochastic linear bandit setting. Although this problem arises frequently in practice, the offline-to-online tradeoff remains poorly understood in structured environments. We propose a linear bandit algorithm that balances this tradeoff: it relies on offline data during early rounds, and increasingly favors exploration as the horizon grows. We establish regret bounds showing that our method is simultaneously competitive with both purely online and purely offline solutions. In particular, it achieves sublinear regret relative to the optimal action in the number of online interactions, while its regret relative to an offline reference decreases as the number of offline samples grows. Empirical results further demonstrate its effectiveness across various problem parameters.
Kushagra Chandak, Toshinori Kitamura, and Xiaoqi Tan. "Offline-to-Online Learning in Linear Bandits." Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
BibTeX:@article{chandak2026offlinetoonline,
title={Offline-to-Online Learning in Linear Bandits},
author={Kushagra Chandak and Toshinori Kitamura and Xiaoqi Tan},
journal={Reinforcement Learning Journal},
volume={7},
pages={},
year={2026}
}