Recovering Sensitive Information from Pretrained Speech Models through Noise Masking Attacks

Centrala begrepp

Noise masking attacks can recover sensitive information from the pretraining data of speech models, even when the models were trained on audio-only data without access to transcripts.

Sammanfattning

The authors extend the noise masking attack introduced by Amid et al. [1] to target modern large-scale pretrained speech encoders. Their key finding is that by fine-tuning the pretrained encoder to build an ASR model, they can successfully perform noise masking attacks to recover sensitive information from the pretraining data, even though the original encoder was trained on audio-only data without access to transcripts.

The authors first describe their attack pipeline, which involves fine-tuning the pretrained encoder to produce an ASR model, and then performing noise masking on this fine-tuned model. They also introduce techniques to improve the precision of the noise masking attacks by allowing the adversary to abstain from low-confidence predictions.

The authors then evaluate their attacks on the LibriLight and LibriSpeech datasets. They find that it is indeed possible to perform noise masking attacks on pretraining data, recovering exact sensitive information (e.g., names) in up to 2% of cases, and leaking any sensitive information in up to 14% of cases. The authors also experiment with various mitigations, including data sanitization, modified pretraining, and data deduplication, finding that data sanitization and a combination of silence masking and MTR are the most effective at reducing the risk of these attacks.

Statistik

Noise masking attacks can recover the exact sensitive information (e.g., names) in up to 2% of cases, and leak any sensitive information in up to 14% of cases, when fine-tuning on the LS-NoName dataset.
Precision of the noise masking attacks can be improved to over 12% by allowing the adversary to abstain from low-confidence predictions.

Citat

"Our key finding is that this approach leads to successful noise masking, paralleling privacy attacks which have been shown on the pretraining data for image models [7, 8]."
"Noise masking attacks are possible on the pretraining data. When finetuning on both LS and LS-NoName, we find our attacks can correctly recover the exact name from roughly 1-2% of training utterances, even without any overlap with the finetuning set (for LS-NoName), and leakage of any name is much higher."

Viktiga insikter från

Noise Masking Attacks and Defenses for Pretrained Speech Models

by Matthew Jagi... på arxiv.org 04-03-2024

https://arxiv.org/pdf/2404.02052.pdf

Djupare frågor

How might the noise masking attack performance and mitigation effectiveness change if the pretraining dataset was more diverse or contained more sensitive information?

If the pretraining dataset were more diverse or contained additional sensitive information, the performance of noise masking attacks could potentially increase in terms of successful extraction of sensitive data. A more diverse dataset would provide a broader range of contexts for the model to learn from, potentially making it more adept at recognizing and completing sensitive information during attacks. This could lead to higher accuracy rates in recovering private data. On the other hand, the effectiveness of mitigations such as data sanitization or deduplication might decrease if the dataset contains a wider array of sensitive information. Mitigations that rely on specific patterns or types of data for removal or modification may struggle to adequately protect against a more varied set of sensitive information.

What other types of sensitive information, beyond names, could be extracted from the pretraining data using noise masking attacks?

In addition to names, various other types of sensitive information could potentially be extracted from the pretraining data using noise masking attacks. Examples of such information include but are not limited to:

Addresses: Extracting addresses from audio data could pose a significant privacy risk, especially if the model has been trained on a dataset containing personal or location-specific information.
Financial Data: Details such as credit card numbers, bank account information, or financial transactions could be vulnerable to extraction through noise masking attacks.
Medical Information: Sensitive medical conditions, treatments, or patient details could be inadvertently revealed if the model memorizes and reproduces such information during attacks.
Legal Information: Confidential legal matters, case details, or privileged communications could also be at risk of exposure through successful noise masking attacks.

How could the noise masking attack be further improved to increase the precision and actionability of the recovered sensitive information?

To enhance the precision and actionability of the recovered sensitive information in noise masking attacks, several strategies could be employed:

Fine-tuning Techniques: Implement more sophisticated fine-tuning methods to optimize the model for recognizing and completing sensitive information accurately during attacks.
Advanced Noise Generation: Utilize advanced noise generation techniques to create more realistic and contextually appropriate noise, increasing the likelihood of successful information extraction.
Multi-Modal Data: Incorporate multi-modal data sources (e.g., text, images) during fine-tuning to provide additional context for the model, potentially improving the precision of recovered information.
Adversarial Training: Employ adversarial training methods to make the model more robust against noise masking attacks, reducing the chances of successful information leakage.
Dynamic Abstention: Implement dynamic abstention strategies that adaptively determine when to abstain from predictions based on the confidence levels of the model, enhancing precision by filtering out uncertain or inaccurate results.