General In-Network Unsupervised Intrusion Detection by Rule Extraction

Core Concepts

Genos, a general in-network framework for unsupervised anomaly-based network intrusion detection, achieves high throughput, interpretability, and trivial updating overhead by extracting model-agnostic rules.

Abstract

The paper proposes Genos, a general in-network framework for unsupervised anomaly-based network intrusion detection (A-NIDS). Genos consists of three modules:

1. Model Compiler:

• Adopts a divide-and-conquer approach to extract model-agnostic rules from the A-NIDS source model.
• Utilizes a Score Clustering Tree to partition the feature space into subspaces based on the source model's anomaly scores.
• Designs a Decision Boundary Estimation method to approximate the decision boundaries of the source model in each subspace using axis-aligned rules.
• Translates the extracted rules into P4 tables for efficient in-network deployment.

2. Model Interpreter:

• Provides interpretable explanations for anomaly detections by analyzing the feature deviations from the extracted rules.
• Outperforms a state-of-the-art interpretation method (LIME) in terms of efficiency and accuracy.

3. Model Debugger:

• Identifies and updates the rules responsible for false positives through two modes: patching mode and excluding mode.
• Enables incremental updates by only fine-tuning the affected rules, reducing the overhead compared to retraining the source model.

Genos is implemented on a commodity programmable switch, achieving a throughput of around 100 Gbps, high interpretability, and trivial updating overhead, outperforming several prior works.

Stats

The network traffic datasets used are CIC-IDS and TON-IoT, containing a wide range of realistic attack traffic.
The source A-NIDS models (autoencoder, variational autoencoder, one-class SVM, isolation forest) achieve AUC scores ranging from 0.9879 to 0.9998 on the datasets.

Quotes

None

Key Insights Distilled From

Genos

by Ruoyu Li,Qin... at arxiv.org 03-29-2024

https://arxiv.org/pdf/2403.19248.pdf

Deeper Inquiries

How can Genos be extended to support other types of unsupervised models beyond the four evaluated in the paper

Genos can be extended to support other types of unsupervised models beyond the four evaluated in the paper by enhancing the rule extraction algorithm to accommodate the specific characteristics of different models. For instance, if a new unsupervised model requires a different way of partitioning the feature space or estimating decision boundaries, the algorithm can be adapted accordingly. Additionally, the Model Compiler module can be modified to translate the output of these new models into P4 tables effectively. By ensuring flexibility in the rule extraction process and the translation to P4 tables, Genos can seamlessly integrate various unsupervised models for in-network deployment.

What are the potential limitations or challenges of the divide-and-conquer approach used by Genos in handling highly complex decision boundaries of the source model

The divide-and-conquer approach used by Genos may face potential limitations or challenges when handling highly complex decision boundaries of the source model. One challenge could be the scalability of the approach when dealing with extremely intricate decision boundaries that require a large number of subspaces for accurate approximation. As the complexity of the decision boundaries increases, the number of rules extracted for each subspace may also grow significantly, leading to potential performance issues in terms of rule management and deployment. Moreover, the divide-and-conquer strategy may struggle to capture intricate relationships between features that span multiple subspaces, potentially impacting the overall accuracy of the rule extraction process.

How can the feature extraction mechanism in Genos be further optimized to support an even wider range of flow-level statistics without relying on workarounds for arithmetic constraints on programmable switches

To further optimize the feature extraction mechanism in Genos to support a wider range of flow-level statistics without relying on workarounds for arithmetic constraints on programmable switches, several strategies can be implemented. One approach could involve enhancing the feature extractor on the data plane to incorporate more advanced arithmetic operations that are commonly used in calculating complex flow-level statistics. This enhancement could include optimizing the hardware capabilities of the programmable switches to support a broader set of mathematical operations, enabling the direct computation of a wider range of flow-level features without the need for workarounds. Additionally, leveraging techniques such as data preprocessing and feature engineering can help in transforming complex flow-level statistics into formats that are compatible with the arithmetic constraints of the programmable switches, thereby expanding the range of supported features in Genos.