Core Concepts
This paper proposes a framework to design optimal policies for stochastic shortest path problems that consider the probability of task failure, by expanding the search range beyond policies that solely minimize failure probability.
Abstract
The paper addresses limitations of the standard stochastic shortest path (SSP) problem, which cannot handle cases where a catastrophic event may occur during an episode for any policy. To address this, the authors introduce the concept of "dead-ends" to express catastrophic events.
The key contributions are:
- Formulation of a constrained SSP problem that considers the task failure probability as a constraint, in addition to minimizing the expected total cost.
- Approximation of the original problem by treating it as a combination of a Bayesian adaptive Markov decision process (BAMDP) and a two-person zero-sum game.
- Derivation of the optimal policy, which is shown to be a mixed policy that stochastically selects from a set of deterministic semi-Markov policies.
- Demonstration of the effectiveness of the proposed methods through a motion planning problem with obstacle avoidance for a mobile robot.
The authors show that by appropriately setting the parameters (c, ε, γ), the optimal policy for the approximation problem can be made to closely approximate the optimal policy for the original problem.
Stats
There are no key metrics or important figures used to support the author's key logics.
Quotes
There are no striking quotes supporting the author's key logics.