The paper investigates a novel approach to obtain data-driven abstractions of discrete-time stochastic processes as richer discrete stochastic models, where the nondeterminism in the probability space is captured by a collection of Markov Processes. The key aspects are:
The data-driven component of the methodology lies in the fact that only samples from an unknown probability distribution are assumed, while the model of the underlying dynamics is used to build the abstraction through backward reachability computations.
The nondeterminism in the probability space is represented by a Robust Markov Decision Process (RMDP), where the transition probability function is an uncertain set rather than a single probability distribution. This allows searching for policies over a larger action space and synthesizing richer controllers for a wider variety of scenarios.
The connection between the discrete abstraction and the underlying dynamics is formalized through the use of the scenario approach theory, providing probably approximately correct (PAC) guarantees of correctness.
Numerical experiments illustrate the advantages of the proposed RMDP-based abstraction compared to existing MDP-based approaches, particularly in cases where the dynamics are not well-aligned with the chosen partition of the state space.
Naar een andere taal
vanuit de broninhoud
arxiv.org
Belangrijkste Inzichten Gedestilleerd Uit
by Rudi Coppola... om arxiv.org 04-15-2024
https://arxiv.org/pdf/2404.08344.pdfDiepere vragen