Reinforcement Learning Journal, vol. TBD, 2025, pp. TBD.
Presented at the Reinforcement Learning Conference (RLC), Edmonton, Alberta, Canada, August 5–9, 2025.
Curriculum learning has been a quiet, yet crucial component of many high-profile successes of reinforcement learning. Despite this, it is still a niche topic that is not directly supported by any of the major reinforcement learning libraries. These methods can improve the capabilities and generalization of RL agents, but often require complex changes to training code. We introduce Syllabus, a portable curriculum learning library, as a solution to this problem. Syllabus provides a universal API for curriculum learning, modular implementations of popular automatic curriculum learning methods, and infrastructure that allows them to be easily integrated with asynchronous training code in nearly any RL library. Syllabus provides a minimal API for core curriculum learning components, making it easier to design new algorithms and adapt existing ones to new environments. We demonstrate this by evaluating the algorithms in Syllabus on several new environments, each using agents written in a different RL library. We present the first examples of automatic curriculum learning in NetHack and Neural MMO, two of the most challenging RL benchmarks, and find evidence that existing methods do not easily transfer to new environments.
Ryan Sullivan, Ryan Pégoud, Ameen Ur Rehman, Xinchen Yang, Junyun Huang, Aayush Verma, Nistha Mitra, and John P Dickerson. "Syllabus: Portable Curricula for Reinforcement Learning Agents." Reinforcement Learning Journal, vol. TBD, 2025, pp. TBD.
BibTeX:@article{sullivan2025syllabus,
title={Syllabus: {P}ortable Curricula for Reinforcement Learning Agents},
author={Sullivan, Ryan and P{\'{e}}goud, Ryan and Rehman, Ameen Ur and Yang, Xinchen and Huang, Junyun and Verma, Aayush and Mitra, Nistha and Dickerson, John P},
journal={Reinforcement Learning Journal},
year={2025}
}