Efficient Reinforcement Learning with Local Simulator Access: Unlocking Sample-Efficient Learning for Challenging MDPs

Local simulator access enables sample-efficient reinforcement learning for MDPs with low coverability, including challenging settings like Exogenous Block MDPs, using only realizability of the optimal state-value function.