Exploiting Text Classifiers' Horizontal Vision: VertAttack, a Novel Adversarial Attack

Core Concepts

VertAttack exploits the inability of current text classifiers to recognize vertically written words, allowing an attacker to significantly reduce the accuracy of these classifiers while preserving the meaning for human readers.

Abstract

The paper presents VertAttack, a novel adversarial attack that exploits the limitation of current text classifiers in processing vertically written text. Unlike humans, who can easily read text written both horizontally and vertically, most state-of-the-art text classifiers are only able to process text in a horizontal manner, failing to recognize vertically written words.
VertAttack works in two main steps: 1) Word Selection, where it identifies the most important words for the classifier's decision, and 2) Word Transformation, where it rewrites these words vertically. This approach allows VertAttack to significantly reduce the accuracy of four different transformer-based text classifiers across five datasets, with drops of up to 90 percentage points.
The paper also examines the transferability of VertAttack, finding that it can effectively attack classifiers even when the feedback classifier differs from the target classifier. Additionally, a human study confirms that humans can still understand the meaning of the perturbed texts, with 77% accuracy compared to 81% for the original texts.
The authors also investigate initial defenses against VertAttack, such as whitespace removal and text segmentation, and find that a more sophisticated "reverse" defense can mitigate the attack, but is hindered by VertAttack's ability to add chaff characters to further disguise the text.
Overall, VertAttack demonstrates a significant vulnerability in current text classifiers and highlights the need for more robust algorithms that can process text in a manner closer to human understanding.

Stats

VertAttack is able to drop RoBERTa's accuracy on the SST2 dataset from 94% to 13%.
VertAttack causes up to 90 percentage point drops in classification accuracy across the tested datasets and classifiers.
Humans are able to correctly classify 77% of the perturbed texts, compared to 81% of the original texts.

Quotes

"VertAttack exploits the current limitation of classifiers' inability to read text vertically. Specifically, VertAttack perturbs input text by changing information rich words from horizontally to vertically written."
"We find that VertAttack is able to greatly drop the accuracy of 4 different transformer models on 5 datasets. For example, on the SST2 dataset, VertAttack is able to drop RoBERTa's accuracy from 94 to 13%."
"Furthermore, since VertAttack does not replace the word, meaning is easily preserved. We verify this via a human study and find that crowdworkers are able to correctly label 77% perturbed texts perturbed, compared to 81% of the original texts."

Key Insights Distilled From

VertAttack: Taking advantage of Text Classifiers' horizontal vision

by Jonathan Rus... at arxiv.org 04-15-2024

https://arxiv.org/pdf/2404.08538.pdf

Deeper Inquiries

How could text classifiers be designed to better handle vertically written text, while maintaining their performance on horizontally written text?

To enhance text classifiers' ability to handle vertically written text, designers could implement a few key strategies. Firstly, incorporating bidirectional processing mechanisms within the classifier architecture can help it understand text in multiple orientations. By allowing the model to consider context from both directions, it can better interpret vertically written words. Additionally, introducing specific training data that includes vertically written text samples can help the classifier learn to recognize and interpret such variations. Augmenting the pre-processing steps to include transformations that convert vertically written text to a horizontal format before feeding it into the classifier can also improve its performance on such inputs. By integrating these approaches, classifiers can become more versatile in handling various text orientations without compromising their effectiveness on horizontally written text.

What other types of text perturbations, beyond vertical writing, could human adversaries leverage to evade current text classifiers?

Apart from vertical writing, human adversaries can employ various other text perturbations to evade current text classifiers. Some common techniques include character-level manipulations such as inserting, deleting, or substituting characters to introduce noise and confuse the classifier. Adversaries can also leverage syntactic changes like altering sentence structures, changing word orders, or introducing grammatical errors to mislead the classifier. Semantic perturbations, such as synonym substitutions, paraphrasing, or context modifications, can also be effective in evading classifiers. Additionally, adversarial attacks can target specific vulnerabilities in the classifier's understanding of context, sentiment, or intent to craft deceptive text inputs. By combining these diverse perturbation strategies, adversaries can create challenging inputs that exploit the limitations of current text classifiers.

How might the insights from VertAttack be applied to improve the robustness of text classifiers in other domains, such as content moderation or fake news detection?

The insights from VertAttack offer valuable lessons that can be applied to enhance the robustness of text classifiers in various domains like content moderation and fake news detection. By understanding the vulnerabilities exposed by VertAttack, developers can implement targeted defenses to mitigate similar adversarial threats. For content moderation, integrating mechanisms to detect and handle vertically written text can improve the classifier's ability to accurately process diverse text formats. In fake news detection, the learnings from VertAttack can inform the development of more resilient models that can identify and counter deceptive text manipulations. Techniques like incorporating diverse training data, implementing robust pre-processing steps, and deploying adversarial training methods can help classifiers better handle sophisticated attacks. By leveraging the insights from VertAttack, developers can fortify text classifiers against adversarial challenges and enhance their effectiveness in critical applications like content moderation and fake news detection.