Core Concepts
To establish a certification framework for deep learning systems in safety-critical applications, this work proposes principles and methods for (1) inherently safe design through disentangled representation learning and (2) run-time error detection via uncertainty quantification, out-of-distribution detection, and adversarial robustness.
Abstract
This work aims to establish a certification framework for deep learning systems in safety-critical applications. It starts by reviewing current progress in AI certification from both industry and research perspectives.
The key assumptions made are:
Semantic representation and hidden generative model: The input data can be represented by a set of semantic content and style variables, generated by an unknown stochastic process.
Full content disentanglement and numerical representation: The content variables, which represent the target quantities to be predicted, are disentangled from each other and have a numerical representation.
Disentanglement between content and style: The content variables are disentangled from the style variables.
Known prior and complete coverage of content variables: The range of possible content variable realizations is known, and the training data covers this range.
Unimodal mapping between input and semantics: There is a unique mapping from the input to the content variables, up to some uncertainty.
Based on these assumptions, the work proposes a certification framework with two main components:
Inherently Safe Design:
Recovering and representing semantic features through disentangled representation learning
Ensuring disentanglement between content and style features
Fulfilling priors for a "good" representation, such as interpretability and robustness
Providing model transparency in the failure case
Run-time Error Detection:
Calibrated uncertainty quantification
Principled out-of-distribution detection
Avoiding feature collapse
Defending against adversarial attacks
The work concludes by proposing a novel deep learning model architecture that aims to fulfill the principles established in the certification framework.