This is the pre-proceedings for the RLC 2026. You may expect minor changes.
Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
Will be presented at the Reinforcement Learning Conference (RLC), Montréal, Quebec, Canada, August 15–17, 2026.
Keywords: deep RL, state abstractions, inductive biases
The ability to identify and ignore task irrelevant environment variables is central to intelligent behavior. In reinforcement learning (RL), existing methods for learning such state abstractions typically rely on auxiliary objectives and such objectives tend to add significant complexity to the base RL algorithm. In this work, we take a step back and ask: can task-specific abstraction emerge from return optimization alone, without any additional objectives? We introduce a surprisingly simple neural network architecture change: a learnable, state-independent attention mask applied to the inputs of the policy and value networks and trained end-to-end using only the RL objective. Despite its simplicity, this architectural modification consistently improves sample efficiency and learns to mask out distracting input variables across 12 continuous control tasks. We analyze the dynamics of gradient descent using this method on a linear regression task and demonstrate suppression of distracting input features. Finally, we conduct experiments on toy MDPs and show that the attention mask increases the accuracy of action-values and induces a soft abstraction over a factored state space. Our findings challenge the need for complex auxiliary objectives to learn state abstractions in this setting and suggest a simple baseline for future research.
Anshuman Senapati and Josiah P. Hanna. "A Simple Baseline for Learning Approximate State Abstractions in Factored State Spaces." Reinforcement Learning Journal, vol. 7, 2026, pp. TBD.
BibTeX:@article{senapati2026a,
title={A Simple Baseline for Learning Approximate State Abstractions in Factored State Spaces},
author={Anshuman Senapati and Josiah P. Hanna},
journal={Reinforcement Learning Journal},
volume={7},
pages={},
year={2026}
}