The Fallacy of Minimizing Local Regret in Sequential Task Settings: A Study on Reinforcement Learning

Optimal regret rates in early tasks may lead to worse rates in subsequent ones due to unanticipated changes, necessitating additional exploration.