Addressing Non-Consensual Intimate Media: Sociotechnical Opportunities for Computing Research

Core Concepts

Non-consensual intimate media (NCIM) is a prevalent form of online abuse that disproportionately targets women, causing severe trauma. Technological components across the sociotechnical stack enable and perpetuate these harms, necessitating a comprehensive research agenda to address the problem.

Abstract

The paper introduces the sociotechnical stack, a conceptual framework that maps social impacts to the technical components that facilitate them. It then applies this framework to analyze how different layers of the technical stack, from user interfaces to hardware, contribute to the harms of non-consensual intimate media (NCIM). At the user interface level, the lack of control over shared content and the public visibility of social networks enable NCIM harms. The application layer facilitates impersonation and the hosting of abusive platforms. Algorithms, including content recommendation and search engines, can inadvertently promote and surface non-consensual content. The network layer enables the widespread duplication and broadcasting of NCIM, making it nearly impossible to remove. The storage and operating system layers lack robust consent mechanisms, while hardware like cameras can be exploited for surreptitious recordings. The paper proposes a research agenda to address NCIM across these sociotechnical layers. Recommendations include developing better ways to communicate and enforce consent, improving content moderation practices, designing trauma-informed social computing systems, enhancing hashing algorithms, and exploring consent-preserving protocols for data transmission and storage. The goal is to leverage computing research to significantly reduce the harms of NCIM.

Stats

"1 in 6 adults have had their intimate content shared without consent" "93% of NCIM victims experience considerable emotional distress" "82% encounter disruptions in their social and professional lives" "42% seek psychological treatment" "51% have contemplated suicide"

Quotes

"There will never be a day in my entire lifetime that all the images of me could ever be deleted ...It's a crime that doesn't just happen and then that's done. It's something that is continual, and this could continue for I don't know how long." "[It's become] a hidden obsession to always check my phone...and that's kind of become the way that I've coped with it, constantly checking my phone, to the detriment of my work." "People were messaging me because they found me on a porn website ...my name and my face were on there ...my school and my hometown were also on there ...oh my God, these videos had thousands of views on them. Hearing that the next time someone's in your hometown, they're going to fuck you. It's just terrifying."

Key Insights Distilled From

The Sociotechnical Stack: Opportunities for Social Computing Research in Non-consensual Intimate Media

by Li Qiwei,All... at arxiv.org 05-07-2024

https://arxiv.org/pdf/2405.03585.pdf

Deeper Inquiries

How can we design consent-preserving protocols for data transmission and storage that can be effectively implemented across centralized and decentralized platforms?

To design consent-preserving protocols for data transmission and storage that are effective across centralized and decentralized platforms, several key considerations must be taken into account: Transparency and Control: Users should have clear visibility and control over how their data is transmitted and stored. This includes providing options for users to specify their preferences regarding the handling of their data, such as setting privacy levels and consent parameters. Encryption and Authentication: Implement robust encryption mechanisms to secure data transmission and storage. Utilize strong authentication methods to ensure that only authorized parties can access and interact with the data. Blockchain Technology: Consider leveraging blockchain technology to create immutable records of consent and data transactions. Blockchain can provide a decentralized and transparent ledger for tracking data interactions while maintaining user privacy and control. Smart Contracts: Smart contracts can be used to automate and enforce consent agreements between parties. These self-executing contracts can ensure that data is only transmitted and stored according to predefined consent parameters. Interoperability: Ensure that the consent-preserving protocols are interoperable across different platforms and systems. This interoperability is crucial for seamless data transmission and storage processes, especially in decentralized environments. User Education and Empowerment: Educate users about the importance of consent and data privacy. Empower them with tools and resources to actively manage and monitor their data interactions, promoting a culture of informed consent. By incorporating these principles and technologies, consent-preserving protocols can be designed to effectively safeguard data transmission and storage while respecting user privacy and autonomy in both centralized and decentralized platforms.

How can we balance content removal with free speech rights, considering the potential unintended consequences of more robust content moderation practices?

Balancing content removal with free speech rights is a complex and delicate task that requires careful consideration of various factors. Here are some strategies to achieve this balance while mitigating unintended consequences: Clear Content Moderation Policies: Establish clear and transparent content moderation policies that outline the criteria for removing content. These policies should be consistently applied and take into account legal standards and community guidelines. Appeals and Transparency: Provide avenues for users to appeal content removal decisions and ensure transparency in the moderation process. Users should understand why their content was removed and have the opportunity to challenge the decision. Proportionality and Context: Consider the context in which content is shared and the potential impact of its removal. Moderation practices should be proportionate to the harm caused by the content and take into account the broader societal implications. Human Oversight: While automated moderation tools can assist in flagging inappropriate content, human moderators should review and make final decisions on contentious cases. Human judgment can provide nuance and context that automated systems may lack. Collaboration and Consultation: Involve stakeholders, including legal experts, civil society organizations, and user representatives, in the development of content moderation policies. Collaboration can help ensure that decisions are fair, balanced, and respectful of free speech rights. Continuous Evaluation and Improvement: Regularly evaluate the impact of content moderation practices on free speech rights and adjust policies as needed. Continuous monitoring and feedback mechanisms can help identify and address unintended consequences. By implementing these strategies and maintaining a commitment to upholding free speech rights while addressing harmful content, platforms can navigate the complexities of content moderation with greater sensitivity and effectiveness.

How can we leverage emerging technologies like blockchain and differential privacy to empower victims-survivors and deter NCIM perpetration without compromising individual privacy and autonomy?

Leveraging emerging technologies like blockchain and differential privacy can offer innovative solutions to empower victims-survivors and deter NCIM perpetration while safeguarding individual privacy and autonomy: Blockchain for Consent Management: Utilize blockchain to create secure and immutable records of consent for data sharing and storage. Victims-survivors can have greater control over how their intimate content is used and shared, with transparent audit trails to track consent. Secure Data Storage with Differential Privacy: Implement differential privacy techniques to protect sensitive information while allowing for meaningful analysis. By adding noise to data sets, individual privacy is preserved, and victim-survivors can share their experiences without compromising their identities. Decentralized Platforms for Secure Communication: Develop decentralized platforms that prioritize security and privacy for victim-survivors. Blockchain-based messaging systems can offer end-to-end encryption, self-destructing messages, and anonymous communication channels to prevent unauthorized access to intimate content. Smart Contracts for Content Removal: Use smart contracts on blockchain networks to automate the removal of non-consensual intimate content. Victims-survivors can trigger the removal of harmful material by providing evidence of lack of consent, ensuring swift and effective response to NCIM incidents. Community Reporting and Verification: Implement community-driven reporting mechanisms on decentralized platforms to verify and address NCIM incidents. Blockchain can facilitate transparent reporting processes and incentivize community members to flag and remove abusive content. Education and Support Services: Leverage blockchain for secure and transparent distribution of educational resources and support services for victim-survivors. Differential privacy can protect the identities of individuals seeking help while ensuring the effectiveness of interventions. By integrating these technologies into comprehensive strategies for victim empowerment and NCIM prevention, platforms can create safer and more supportive environments for individuals affected by non-consensual intimate media, all while upholding privacy and autonomy rights.