This is the pre-proceedings for the RLC 2026. You may expect minor changes.
Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
Will be presented at the Reinforcement Learning Conference (RLC), Montréal, Quebec, Canada, August 15–17, 2026.
Keywords: Reinforcement Learning, Robotics, Embodied Intelligence, Research Platform
Reinforcement learning (RL) research has demonstrated success in both physical and simulated domains, though empirically, the predominant methodology is firmly rooted in simulations. The predominance of simulations makes translating research to physical reality uncertain for both algorithms and researchers. We propose a physical platform that is designed to simplify the transition. In this paper, we present the Open Ant: a physical variant of the commonly used Gymnasium Ant environment, along with a simulation. We demonstrate that competent walking policies can be learned from scratch in approximately one hour directly from the physical robot's experience, for two substantially different RL algorithms: SARSA($\lambda$) and Soft Actor-Critic (SAC). Separately, we show policies that were learned in simulation transfer to reality. We also examine how well the platform supports a nimble experimental ecosystem. Specifically, we observe the speed with which new users from diverse backgrounds achieve their first success with the platform, and how easily the platform can be repaired and updated when hardware issues arise. Both the hardware design and software are available as open-source for ease of customization. In summary, we advocate for the use of the Open Ant for RL researchers who frequently use simulated environments, so they can more easily include robot experiments in their evaluations.
Elena Sorina Lupu, Patrick Spieler, Khurram Javed, Kris De Asis, John D Martin, Martha Steenstrup, and Joseph Varughese Modayil. "The Open Ant: A Robot Platform for Reinforcement Learning Research." Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
BibTeX:@article{lupu2026the,
title={The Open Ant: A Robot Platform for Reinforcement Learning Research},
author={Elena Sorina Lupu and Patrick Spieler and Khurram Javed and Kris De Asis and John D Martin and Martha Steenstrup and Joseph Varughese Modayil},
journal={Reinforcement Learning Journal},
volume={7},
pages={},
year={2026}
}