מושגי ליבה
Flow-based multi-modal adversarial attack induces garbling and incorrect responses in video-based LLMs.
תקציר
The content introduces FMM-Attack, the first adversarial attack tailored for video-based LLMs. It crafts imperceptible perturbations on video frames to induce incorrect answers and garbling in model output. Extensive experiments demonstrate effectiveness and safety-related feature alignment importance.
Introduction
Large multi-modal models vulnerable to attacks.
Video-based LLMs enhance video understanding but lack robustness.
Methodology
Threat model includes imperceptible perturbations for incorrect responses.
Flow-based temporal mask selects effective frames for attack.
Experiments
Quantitative evaluation shows FMM-Attack's effectiveness.
Qualitative examples illustrate garbled responses induced by the attack.
Discussions
Importance of flow-based masks in precise video manipulation.
Garbling effect observed in model outputs due to FMM-Attack.
Conclusion
FMM-Attack disrupts video-based LLMs effectively with minimal perturbations.
Insights into cross-modal feature attacks contribute to understanding multi-modal robustness.
סטטיסטיקה
著者らは、ビデオフレームの20%未満に微視的な攪乱を加えることで、ビデオベースのLLMが不正確な応答を生成することを示した。
ציטוטים
"Our attack can effectively induce video-based LLMs to generate either garbled nonsensical sequences or incorrect semantic sequences."