แนวคิดหลัก
Periodic model retraining can significantly improve the performance of anomaly detection models over time, but the choice of retraining technique (blind vs informed) and data (full-history vs sliding window) depends on the specific anomaly detection model.
บทคัดย่อ
The study evaluates the performance of state-of-the-art anomaly detection models on operational data from the Yahoo and NAB datasets. It then investigates the impact of different model retraining techniques on the performance of these anomaly detectors over time.
Key highlights:
- The more complex anomaly detection models (LSTM-AE and SR-CNN) perform significantly better than simpler models (FFT, PCI, SR) on the operational datasets.
- The performance of SR-CNN is highly sensitive to the size of the testing data, while LSTM-AE and SR are more robust.
- Periodically retraining the anomaly detection models can improve their performance over time, but the choice of retraining technique matters:
- LSTM-AE benefits more from a sliding window retraining approach, while SR and SR-CNN perform better with a full-history approach.
- Blind (periodic) retraining generally achieves better results than informed retraining based on a concept drift detector.
The study provides guidance for AIOps practitioners on selecting and maintaining anomaly detection models in the face of evolving operational data.