Long-Tailed Anomaly Detection with Learnable Class Names for Scalable and Robust Defect Identification

Основные понятия

LTAD combines anomaly detection by reconstruction and semantic anomaly detection to detect defects across multiple and long-tailed image classes, without relying on dataset class names. It learns pseudo-class names and uses a VAE-based data augmentation to address the long-tailed distribution of real-world applications.

Аннотация

The content discusses the problem of long-tailed anomaly detection (AD), where different image classes have vastly different sample sizes in real-world applications. It introduces several long-tailed AD datasets and performance metrics, and proposes a novel method called LTAD to address this challenge.
LTAD combines two approaches:

AD by reconstruction: A transformer-based reconstruction module projects image patches onto the manifold of normal images, and the reconstruction error is used as an anomaly score.
Semantic AD: A binary classifier in the semantic space of a pretrained foundation model (ALIGN) is used to detect anomalies, leveraging learned pseudo-class names to make the classifier class-sensitive.

To address the long-tailed distribution, LTAD has a two-phase training process:

Phase 1 learns the pseudo-class names and a VAE-based data augmentation module to synthesize features for minority classes.
Phase 2 then trains the reconstruction and classification modules using a mix of real and synthetic data.
Extensive experiments show that LTAD outperforms state-of-the-art methods on various long-tailed AD datasets and configurations. The ablation studies confirm the efficacy of LTAD's components, including the semantic AD module and the data augmentation strategy.

Статистика

"Anomaly detection aims to identify defective images and localize their defects (if any)."
"Various methods have shown that this problem can be solved with high accuracy; e.g., [3, 20, 31, 36, 41, 44, 69, 77, 79, 84] have success rates >95% for anomaly detection and localization on the MVTec dataset [5]."
"However, as illustrated in Fig. 1, these methods require a different model per image category, which compromises scalability to many classes."

Цитаты

"Anomaly detection (AD) aims to identify defective images and localize their defects (if any)."
"Ideally, AD models should be able to detect defects over many image classes; without relying on hard-coded class names that can be uninformative or inconsistent across datasets; learn without anomaly supervision; and be robust to the long-tailed distributions of real-world applications."

Ключевые выводы из

Long-Tailed Anomaly Detection with Learnable Class Names

by Chih-Hui Ho,... в arxiv.org 04-01-2024

https://arxiv.org/pdf/2403.20236.pdf

Дополнительные вопросы

How can LTAD be extended to handle open-set anomaly detection, where the test-time anomalies may belong to completely unseen classes

To extend LTAD for open-set anomaly detection, where test-time anomalies may belong to completely unseen classes, we can incorporate a few key strategies. One approach is to implement a mechanism for dynamically updating the class prototypes or pseudo class names based on the incoming data. This adaptive learning process can help the model adjust to novel classes and anomalies that were not present during training. Additionally, integrating a few-shot learning framework, such as meta-learning or prototypical networks, can enable the model to quickly adapt to new classes with limited samples. By leveraging the meta-knowledge learned from the known classes, the model can generalize better to unseen anomalies. Furthermore, incorporating a mechanism for outlier detection or novelty detection can help identify instances that do not fit into any of the learned classes, signaling potential unseen anomalies. By combining these approaches, LTAD can be extended to handle open-set anomaly detection effectively.

What are the potential limitations of the VAE-based data augmentation approach used in LTAD, and how could it be further improved to better capture the diversity of anomalies in long-tailed distributions

While the VAE-based data augmentation approach used in LTAD is effective in synthesizing additional training examples to address the data scarcity in long-tailed distributions, it may have some limitations. One potential limitation is the risk of generating unrealistic or irrelevant synthetic features that do not accurately represent the diversity of anomalies present in the dataset. To improve this approach, one possible enhancement could be to incorporate a more sophisticated generative model, such as a generative adversarial network (GAN), to generate more realistic and diverse anomaly samples. By leveraging the discriminative capabilities of a GAN, the data augmentation process can produce more authentic anomalies that better capture the variability in the long-tailed distribution. Additionally, introducing diversity-promoting techniques, such as diversity regularization or diversity-aware loss functions, can encourage the VAE to generate a broader range of anomaly representations, enhancing the model's ability to detect and generalize to diverse anomalies in the dataset.

Given the importance of the text prompts for the semantic AD module, how could LTAD be adapted to automatically generate or optimize these prompts, rather than relying on manual selection

To automate the generation or optimization of text prompts for the semantic AD module in LTAD, we can explore several approaches. One method is to incorporate a reinforcement learning framework where the model learns to generate or adapt the text prompts based on the feedback received during training. By rewarding the model for selecting informative and discriminative prompts that lead to better anomaly detection performance, it can learn to optimize the prompts automatically. Another approach is to leverage natural language processing techniques, such as transformer models or recurrent neural networks, to generate text prompts based on the characteristics of the input images. These models can learn to generate contextually relevant prompts that capture the distinguishing features of normal and abnormal images. Additionally, employing active learning strategies where the model iteratively selects and refines the text prompts based on the uncertainty or informativeness of the data samples can further enhance the adaptability and effectiveness of the semantic AD module in LTAD.