Reinforcement Learning Journal, vol. 1, 2024, pp. 108–129.
Presented at the Reinforcement Learning Conference (RLC), Amherst Massachusetts, August 9–12, 2024.
In key real-world problems, full state information can sometimes be obtained but only at a high cost, such as by activating more precise yet energy-intensive sensors, or by consulting a human, thereby compelling the agent to operate under partial observability. For this scenario, we propose AEMS-SR (Anytime Error Minimization Search with State Requests), a principled online planning algorithm tailored for POMDPs with state requests. By representing the search space as a graph instead of a tree, AEMS-SR avoids the exponential growth of the search space originating from state requests. Theoretical analysis demonstrates AEMS-SR's $\varepsilon$-optimality, ensuring solution quality, while empirical evaluations illustrate its effectiveness compared with AEMS and POMCP, two SOTA online planning algorithms. AEMS-SR enables efficient planning in domains characterized by partial observability and costly state requests offering practical benefits across various applications.
Raphaël Avalos, Eugenio Bargiacchi, Ann Nowe, Diederik Roijers, and Frans A Oliehoek. "Online Planning in POMDPs with State-Requests." Reinforcement Learning Journal, vol. 1, 2024, pp. 108–129.
BibTeX:@article{avalos2024online,
title={Online Planning in {POMDP}s with State-Requests},
author={Avalos, Rapha{\"{e}}l and Bargiacchi, Eugenio and Nowe, Ann and Roijers, Diederik and Oliehoek, Frans A},
journal={Reinforcement Learning Journal},
volume={1},
pages={108--129},
year={2024}
}