Conceptos Básicos
A novel post-hoc concept-based XAI framework that combines local and global decision-making strategies via prototypes to enable a clearer understanding of model behavior and detect outlier predictions.
Resumen
The paper presents a novel eXplainable AI (XAI) framework called Prototypical Concept-based Explanations (PCX) that aims to validate and understand the decision-making process of deep neural networks (DNNs).
Key highlights:
PCX leverages concept-based explanations to provide local (instance-wise) insights into model predictions. It computes relevance scores and visualizations for human-understandable concepts used by the model.
To gain global (class-wise) understanding, PCX models the distribution of concept relevances across the training data using Gaussian Mixture Models (GMMs). This allows it to discover prototypical prediction strategies for each class.
By comparing individual predictions to these prototypes, PCX can quantify how (un-)ordinary a prediction is, highlighting over- or underused concepts. This enables the detection of outlier predictions, data quality issues, and spurious model behavior.
Experiments on ImageNet, CUB-200, and CIFAR-10 datasets demonstrate the effectiveness of PCX for model validation, OOD detection, and understanding global model behavior.
Overall, PCX provides a comprehensive framework to validate DNN decisions in a more objective and interpretable manner, reducing the reliance on human assessment.
Estadísticas
DNNs can learn shortcuts from spurious data artifacts, leading to unreliable predictions on out-of-distribution (OOD) samples.
Existing XAI methods often rely heavily on human assessment, hindering practical deployment in critical applications.
The proposed PCX framework combines local and global explanations via prototypes to enable more objective model validation.
Citas
"Ensuring both transparency and safety is critical when deploying Deep Neural Networks (DNNs) in high-risk applications, such as medicine."
"Only few XAI methods are suitable of ensuring safety in practice as they heavily rely on repeated labor-intensive and possibly biased human assessment."
"Quantifying the deviation from prototypical behavior not only allows to associate predictions with specific model sub-strategies but also to detect outlier behavior."