The paper presents a novel heterogeneous autoencoder that combines conventional and quadratic neurons, which is the first of its kind in the autoencoder family. The authors first prove a theoretical result showing that heterogeneous networks can approximate certain functions more efficiently than homogeneous networks of either conventional or quadratic neurons alone.
Motivated by this theoretical insight, the authors develop three heterogeneous autoencoder designs that arrange conventional and quadratic layers in a symmetric fashion in the encoder and decoder. They apply these heterogeneous autoencoders to unsupervised anomaly detection tasks, which face challenges such as data unknownness, anomaly feature heterogeneity, and feature unnoticeability. The authors argue that the high feature representation ability of heterogeneous autoencoders can effectively address these challenges, as they can characterize a variety of anomaly data, discriminate anomalies from normal instances, and accurately learn the distribution of normal samples.
Experimental results demonstrate that the proposed heterogeneous autoencoders achieve competitive performance compared to state-of-the-art models on tabular data anomaly detection and real-world bearing fault detection tasks. The work provides a novel perspective on neural network design by introducing neuronal diversity, with both theoretical and empirical evidence confirming the benefits of heterogeneous networks.
לשפה אחרת
מתוכן המקור
arxiv.org
תובנות מפתח מזוקקות מ:
by Jing-Xiao Li... ב- arxiv.org 04-26-2024
https://arxiv.org/pdf/2204.01707.pdfשאלות מעמיקות