This is the pre-proceedings for the RLC 2026. You may expect minor changes.
Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
Will be presented at the Reinforcement Learning Conference (RLC), Montréal, Quebec, Canada, August 15–17, 2026.
Keywords: Continual RL, VLA Models, Robotics.
Continual Reinforcement Learning (CRL) for Vision-Language-Action (VLA) models is a promising direction toward self-improving embodied agents that can adapt in open-ended, evolving environments. However, conventional wisdom from continual learning suggests that naive Sequential Finetuning (Seq FT) leads to catastrophic forgetting, necessitating complex CRL strategies. In this work, we take a step back and conduct a systematic study of parameter-efficient CRL for large pretrained VLAs across three models and five challenging lifelong RL benchmarks. Our findings are surprising: contrary to established belief, simple Seq FT with low-rank adaptation (LoRA) is remarkably strong: it achieves high plasticity, exhibits little to no forgetting, and retains strong zero-shot generalization, frequently outperforming more sophisticated CRL methods. Through detailed analysis, we show that this robustness arises from a synergy between large-scale pretraining, parameter-efficient adaptation, and on-policy RL. Together, these components reshape the stability–plasticity trade-off, making continual adaptation both stable and scalable. Our results position sequential finetuning as a powerful method for continual RL with VLAs and provide new insights into lifelong learning in the large model era.
Jiaheng Hu, Jay Shim, Chen Tang, Yoonchang Sung, Bo Liu, Peter Stone, and Roberto Martín-Martín. "Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learning." Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
BibTeX:@article{hu2026simple,
title={Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learning},
author={Jiaheng Hu and Jay Shim and Chen Tang and Yoonchang Sung and Bo Liu and Peter Stone and Roberto Martín-Martín},
journal={Reinforcement Learning Journal},
volume={7},
pages={},
year={2026}
}