Enhancing Stance Detection on Social Media by Incorporating Moral Foundations

Core Concepts

Incorporating individuals' moral foundations can enhance the performance of stance detection models on social media data.

Abstract

This study investigates how moral foundation dimensions can contribute to predicting an individual's stance on a given target on social media. The authors incorporate moral foundation features extracted from text, along with message semantic features, to classify stances at both message- and user-levels using traditional machine learning models, fine-tuned language models, and large language models.
The key findings are:

Encoding moral foundations can enhance the performance of stance detection tasks, with improvements in F1 score up to 23.7 points, depending on the choice of model and dataset.

The predictive performance of moral foundation-based models varies across tasks (message-level vs. user-level stance detection), datasets, stance targets, and classifiers.

The study highlights interesting associations between moral foundations and the stances towards specific targets, such as people against the "Climate Change is a Real Concern" target displaying moral violations towards Care, Fairness, Authority and Sanctity foundations.

The incorporation of moral foundations improves F1 score on stance detection tasks on average by 1.06 points for traditional machine learning models, 5.91 points for Fine-tuned Language Models, and 15.82 points for Large Language Models, suggesting that the addition of such psychological attributes might be particularly fruitful for LLM-based stance detection models.

Stats

Moral foundations can enhance stance detection performance by up to 23.7 F1 score points.
Incorporating moral foundations improves F1 score on average by 1.06 points for traditional ML models, 5.91 points for Fine-tuned Language Models, and 15.82 points for Large Language Models.

Quotes

"Encoding moral foundations can enhance the performance of stance detection tasks, with improvements in F1 score up to 23.7 points, depending on the choice of model and dataset."
"The incorporation of moral foundations improves F1 score on stance detection tasks on average by 1.06 points for traditional machine learning models, 5.91 points for Fine-tuned Language Models, and 15.82 points for Large Language Models."

Key Insights Distilled From

Enhancing Stance Classification on Social Media Using Quantified Moral Foundations

by Hong Zhang, ... at arxiv.org 09-26-2024

https://arxiv.org/pdf/2310.09848.pdf

Deeper Inquiries

How can the insights from this study be leveraged to design psychologically-rooted large language models for related tasks beyond stance detection?

The insights from this study can significantly inform the design of psychologically-rooted large language models (LLMs) by emphasizing the integration of moral foundations as a core component of model architecture. By understanding that moral foundations—such as care/harm, fairness/cheating, authority/subversion, sanctity/degradation, and loyalty/betrayal—play a crucial role in shaping user behavior and opinions, developers can create LLMs that are more attuned to the psychological underpinnings of human communication.
To leverage these insights, future LLMs could incorporate moral foundation features during the training phase, allowing the models to learn nuanced moral cues from diverse datasets. This could involve using moral embeddings derived from the extended Moral Foundation Dictionary (eMFD) or FrameAxis techniques, which capture the bias and intensity of moral language. Additionally, the models could be fine-tuned on tasks that require understanding moral implications, such as sentiment analysis, ethical reasoning, and conflict resolution in social discourse.
Moreover, the design of LLMs could benefit from a multi-dimensional approach that includes not only moral foundations but also other psychological attributes, such as personality traits, emotional intelligence, and cultural values. By creating a more holistic representation of users, these models could enhance their performance in various applications, including mental health support, personalized content recommendations, and social media moderation, ultimately leading to more empathetic and context-aware interactions.

What other user-level attributes beyond moral foundations could be incorporated to further enhance stance detection performance?

In addition to moral foundations, several other user-level attributes could be integrated to enhance stance detection performance. These attributes include:

Personality Traits: Utilizing frameworks like the Big Five personality traits (openness, conscientiousness, extraversion, agreeableness, and neuroticism) can provide insights into how individual differences influence stance expression. For instance, users with high openness may express more progressive stances, while those high in conscientiousness may lean towards traditional views.

Emotional States: Incorporating emotional intelligence and sentiment analysis can help models understand the emotional context behind stances. By analyzing the emotional tone of a user's posts, models can better predict their stance based on the emotional resonance of the content.

Cultural Background: Understanding the cultural context of users can significantly impact stance detection. Different cultures may prioritize different moral foundations or have varying norms around expressing opinions. Incorporating cultural indicators can help models adapt to these differences.

Social Influence: User interactions, such as the influence of peers or social networks, can shape stances. Analyzing the social dynamics of users, including their connections and the stances of their social circles, can provide valuable context for stance detection.

Historical Behavior: Tracking a user's historical stances and changes over time can help models identify patterns and predict future stances more accurately. This longitudinal approach can reveal shifts in opinion that are influenced by external events or personal experiences.

By integrating these attributes, stance detection models can achieve a more nuanced understanding of user behavior, leading to improved accuracy and relevance in predicting stances on various topics.

How do the associations between moral foundations and stances vary across different cultural and sociopolitical contexts?

The associations between moral foundations and stances can vary significantly across different cultural and sociopolitical contexts due to the diverse ways in which moral values are prioritized and interpreted. For instance:

Cultural Variability: Different cultures may emphasize certain moral foundations over others. For example, collectivist cultures may prioritize loyalty and authority more than individualistic cultures, which might emphasize fairness and care. This cultural lens can shape how individuals express their stances on social media, leading to variations in moral framing.

Sociopolitical Climate: The prevailing sociopolitical environment can influence which moral foundations are salient in public discourse. In politically polarized contexts, such as during elections or social movements, individuals may invoke moral foundations that align with their ideological beliefs. For instance, during debates on climate change, those opposing the consensus may emphasize authority and loyalty to national interests, while supporters may focus on care for future generations.

Religious Influences: In societies where religion plays a significant role, moral foundations may be heavily influenced by religious teachings. For example, sanctity and purity may be more pronounced in discussions around issues like abortion or LGBTQ+ rights, leading to distinct stances that reflect these moral concerns.

Historical Context: Historical events, such as wars, social movements, or economic crises, can shape collective moral narratives within a society. These narratives can influence how moral foundations are perceived and how they relate to stances on contemporary issues, such as immigration or social justice.

Media Representation: The portrayal of moral issues in media can also affect public perception and stance expression. Media narratives that frame issues through specific moral lenses can reinforce or challenge existing moral foundations, leading to shifts in public opinion.

Overall, understanding these cultural and sociopolitical nuances is essential for accurately interpreting the associations between moral foundations and stances, as they provide critical context for the moral language used in social media discourse.