Adapting an Artificial Intelligence Tool to Screen for Monkeypox Symptoms: The HeHealth Experience

Core Concepts

The HeHealth team leveraged an existing AI-powered sexually transmitted disease symptom checker tool to rapidly develop and validate a digital screening test for symptomatic Monkeypox during the global outbreak.

Abstract

The HeHealth team developed an AI-powered smartphone app (HeHealth) that allows users to take pictures of their own genitals to screen for sexually transmitted diseases (STDs). The initial AI model was trained on 5,000 cases and used a modified convolutional neural network (CNN) to output prediction scores across visually diagnosable penis pathologies including Syphilis, Herpes Simplex Virus (HSV), and Human Papilloma Virus (HPV). When the Monkeypox (Mpox) outbreak was declared a public health emergency, the team quickly adapted the existing tool to screen for Mpox symptoms. They went through a five-stage process: Formative research: Initial discussions with medical, sociobehavioral, community, and statistical experts to understand the Mpox outbreak. Stakeholder engagement: Collaborated with healthcare institutions, communities, and searched for Mpox data to train the AI tool. Image consolidation: Rapidly consolidated Mpox images from various sources to train the initial tool and refined the user interface based on stakeholder feedback. Validation study: Conducted a validation study using 100 patient observations to assess the accuracy of the Mpox symptom checker tool. Launch and refinement: Launched the Mpox symptom checker tool and continued to refine it through patient data obtained via the web app. The final Mpox symptom checker tool showed an accuracy of 87% to rule in Mpox and 90% to rule out symptomatic Mpox. The team faced several challenges, including data privacy concerns, lack of initial Mpox data, and ensuring the generalizability of the tool across skin tones and symptom presentations. They offer lessons learned, such as engaging a wide range of stakeholders, having a multidisciplinary team, prioritizing pragmatism, and recognizing that "big data" is often composed of "small data" that can be consolidated incrementally.

Stats

From June 2022 to October 2022, a total of about 22,000 users had downloaded the HeHealth app, and about 21,000 images have been analyzed using HeHealth AI technology. A total of 1,000 Mpox-related images have been used to train the Mpox symptom checker tool. The Mpox symptom checker tool showed an accuracy of 87% to rule in Mpox and 90% accuracy to rule out the symptomatic infection.

Quotes

"Our digital symptom checker tool showed accuracy of 87% to rule in Mpox and 90% to rule out symptomatic Mpox." "We ensured this was done through several means. First, we tried to build a dataset that represented multiple skin colours from a global population. At the same time, our core technology was built in a way skin results will not be affected by skin color by training the AI to focus on colour-neutral skin abnormalities."

Key Insights Distilled From

Adapting an Artificial Intelligence Sexually Transmitted Diseases Symptom Checker Tool for Mpox Detection: The HeHealth Experience

by Rayner Kay J... at arxiv.org 04-29-2024

https://arxiv.org/pdf/2404.16885.pdf

Deeper Inquiries

How can the HeHealth team further improve the generalizability of their Mpox symptom checker tool to ensure it is inclusive of diverse skin tones and symptom presentations?

To enhance the generalizability of the Mpox symptom checker tool developed by the HeHealth team, several strategies can be implemented: Diverse Dataset Inclusion: The team should actively seek out and include a more diverse dataset that represents various skin tones and symptom presentations related to Mpox. This can be achieved by collaborating with healthcare institutions globally to gather a wide range of images that reflect the diversity of affected populations. Data Augmentation Techniques: Utilize data augmentation techniques to artificially increase the diversity of the dataset. This can involve techniques such as image rotation, flipping, scaling, and color adjustments to create a more comprehensive dataset that covers a broader spectrum of skin tones and symptom variations. Inclusive AI Training: Ensure that the AI model is trained on a balanced dataset that includes images from individuals with different skin tones and various symptom presentations. This will help the AI algorithm learn to accurately identify Mpox symptoms across diverse populations. Continuous Monitoring and Feedback: Implement a system for continuous monitoring and feedback from users and healthcare providers to identify any biases or limitations in the tool's performance related to skin tones or symptom presentations. This feedback can be used to fine-tune the AI model and improve its generalizability over time. Collaboration with Dermatologists and Experts: Collaborate with dermatologists and experts in infectious diseases to validate the tool's performance across different skin tones and symptom presentations. Their insights and feedback can help refine the tool and ensure its effectiveness in diverse populations. By implementing these strategies, the HeHealth team can enhance the generalizability of their Mpox symptom checker tool and ensure it is inclusive of diverse skin tones and symptom presentations.

What are the potential ethical and privacy concerns around the use of AI-powered symptom checker tools, and how can the HeHealth team address these issues to build user trust?

The use of AI-powered symptom checker tools raises several ethical and privacy concerns that need to be addressed to build user trust. Some potential concerns include: Data Privacy: Users may be concerned about the privacy and security of their personal health information when using the tool. The HeHealth team should ensure that all data collected is encrypted, anonymized, and stored securely to protect user privacy. Informed Consent: Users should be informed about how their data will be used, who will have access to it, and the purpose of collecting their information. Obtaining explicit consent from users before collecting any data is essential to ensure transparency and trust. Bias and Fairness: AI algorithms can be biased, leading to inaccurate or discriminatory results. The HeHealth team should regularly monitor the tool for biases and ensure that it provides fair and accurate assessments for all users, regardless of demographic factors. Algorithm Transparency: Users may be skeptical of AI algorithms and their decision-making processes. The team should strive to make the algorithm transparent by explaining how it works, what data it uses, and how it arrives at its conclusions. User Control: Providing users with control over their data, such as the ability to delete their information or opt-out of data collection, can help build trust and confidence in the tool. To address these ethical and privacy concerns and build user trust, the HeHealth team can: Implement robust data protection measures and comply with relevant data privacy regulations. Conduct regular audits and assessments to ensure the tool's fairness and accuracy. Provide clear and accessible information about data usage and privacy policies to users. Engage with stakeholders, including privacy experts and user representatives, to gather feedback and address concerns proactively. By prioritizing data privacy, transparency, fairness, and user control, the HeHealth team can address ethical and privacy concerns and build trust among users of their AI-powered symptom checker tool.

How can the lessons learned from the development of the HeHealth Mpox symptom checker tool be applied to the design and deployment of AI-powered tools for other emerging infectious diseases or public health crises?

The lessons learned from the development of the HeHealth Mpox symptom checker tool can be valuable in designing and deploying AI-powered tools for other emerging infectious diseases or public health crises. Here are some key takeaways that can be applied: Stakeholder Engagement: Engage a wide range of stakeholders early in the development process, including medical experts, community representatives, and industry partners. Collaborating with diverse stakeholders can provide valuable insights and perspectives that contribute to the tool's effectiveness and acceptance. Multidisciplinary Team: Assemble a multidisciplinary team with expertise in various fields such as medicine, data science, public health, and community engagement. A diverse team can offer different perspectives and skills that are essential for developing comprehensive and effective AI-powered tools. Pragmatism over Elegance: Prioritize pragmatism in the design and deployment of AI tools, focusing on practical solutions that address immediate needs. While research and validation are important, it is crucial to balance scientific rigor with the urgency of public health crises. Data Accessibility and Collaboration: Actively seek out partnerships and collaborations to access diverse and comprehensive datasets for training AI models. Sharing data securely and ethically with other institutions and researchers can accelerate the development of AI tools for emerging infectious diseases. Continuous Improvement and Validation: Conduct regular validation studies and feedback sessions to assess the tool's performance and user satisfaction. Iterative improvements based on feedback and real-world data can enhance the tool's accuracy and relevance in addressing public health challenges. By applying these lessons learned, future AI-powered tools for emerging infectious diseases or public health crises can be developed and deployed more effectively, ensuring their impact and usability in addressing critical healthcare challenges.