PufferLib 2.0: Reinforcement Learning at 1M steps/s

By Joseph Suarez

Reinforcement Learning Journal, vol. TBD, 2025, pp. TBD.

Presented at the Reinforcement Learning Conference (RLC), Edmonton, Alberta, Canada, August 5–9, 2025.

Download:

Abstract:

PufferLib is an open-source reinforcement learning project built around efficient and broadly compatible simulation. Our first-party suite of 12 environments each run at 1M steps/second. For existing environments, PufferLib provides one-line wrappers that eliminate common compatibility problems and fast vectorization to accelerate training. With PufferLib, you can use familiar libraries like CleanRL and SB3 to scale from classic benchmarks like Atari and Procgen to complex simulators like NetHack and Neural MMO 3.

Citation Information:

Joseph Suarez. "PufferLib 2.0: Reinforcement Learning at 1M steps/s." Reinforcement Learning Journal, vol. TBD, 2025, pp. TBD.

BibTeX:
@article{suarez2025pufferlib,
    title={{PufferLib} 2.0: {R}einforcement Learning at 1M steps/s},
    author={Suarez, Joseph},
    journal={Reinforcement Learning Journal},
    year={2025}
}