The authors extend the noise masking attack introduced by Amid et al. [1] to target modern large-scale pretrained speech encoders. Their key finding is that by fine-tuning the pretrained encoder to build an ASR model, they can successfully perform noise masking attacks to recover sensitive information from the pretraining data, even though the original encoder was trained on audio-only data without access to transcripts.
The authors first describe their attack pipeline, which involves fine-tuning the pretrained encoder to produce an ASR model, and then performing noise masking on this fine-tuned model. They also introduce techniques to improve the precision of the noise masking attacks by allowing the adversary to abstain from low-confidence predictions.
The authors then evaluate their attacks on the LibriLight and LibriSpeech datasets. They find that it is indeed possible to perform noise masking attacks on pretraining data, recovering exact sensitive information (e.g., names) in up to 2% of cases, and leaking any sensitive information in up to 14% of cases. The authors also experiment with various mitigations, including data sanitization, modified pretraining, and data deduplication, finding that data sanitization and a combination of silence masking and MTR are the most effective at reducing the risk of these attacks.
إلى لغة أخرى
من محتوى المصدر
arxiv.org
الرؤى الأساسية المستخلصة من
by Matthew Jagi... في arxiv.org 04-03-2024
https://arxiv.org/pdf/2404.02052.pdfاستفسارات أعمق