核心概念
This study provides a comprehensive benchmarking of various text anonymization methodologies, focusing on the comparative analysis of modern transformer-based models, Large Language Models (LLMs), and traditional architectures.
要約
The paper presents a detailed evaluation of different text anonymization techniques, including:
-
Traditional Models:
- Conditional Random Fields (CRF)
- Long Short-Term Memory (LSTM) networks
- ELMo for Named Entity Recognition (NER)
-
Transformer-based Models:
- BERT
- ELECTRA
- Custom Transformer model
-
Microsoft Presidio Model
-
Large Language Model (LLM): GPT-2
The evaluation is conducted using the CoNLL-2003 dataset, known for its robustness and diversity. The results showcase the strengths and weaknesses of each approach, offering insights into the efficacy of modern versus traditional methods for text anonymization.
Key findings:
- The custom Transformer model outperformed other models, achieving the highest precision, recall, and F1 score.
- Traditional models like CRF and LSTM also demonstrated strong performance, comparable to the top-performing transformer model.
- Microsoft Presidio exhibited robust capabilities, balancing accuracy and comprehensive coverage in the anonymization task.
- The GPT-2 LLM model performed reasonably well, but there is room for improvement, especially in increasing precision without significantly sacrificing recall.
The study aims to guide researchers and practitioners in selecting the most suitable model for their anonymization needs, while also shedding light on potential paths for future advancements in the field.
統計
The CRF model achieved a precision, recall, and F1 score of 0.93.
The LSTM model achieved a precision of 0.93, a recall of 0.92, and an F1 score of 0.92.
The custom Transformer model achieved a precision of 0.94, a recall of 0.95, and an F1 score of 0.95.
Microsoft Presidio achieved a precision of 0.83, a recall of 0.88, and an F1 score of 0.85.
The GPT-2 model achieved a precision of 0.70, a recall of 0.79, and an F1 score of 0.71.
引用
"The custom Transformer Model surpassed both with metrics of 0.94 across precision, recall, and F1 score, indicating an almost optimal balance between prediction accuracy and retrieval capability."
"While traditional models and specialised solutions like Presidio have showcased strong capabilities, the custom Transformer Model stood out, reinforcing the transformative power and efficiency of advanced transformer architectures in the domain of data anonymisation."