Основные понятия
Effective real-time monitoring systems are essential to safeguard participants and ensure data quality when using online decision-making algorithms in digital health interventions.
Аннотация
This paper provides guidelines and case studies for building real-time monitoring systems for online decision-making algorithms used in digital health interventions.
The key highlights are:
Monitoring systems should categorize potential issues into three severity levels (red, yellow, green) to prioritize addressing critical issues that compromise participant experience or data quality.
Fallback methods are pre-specified procedures executed when issues occur, ensuring the system defaults to baseline functionality and minimizing negative impacts.
The Oralytics trial faced constraints like reliance on an external data source, leading to a more broad monitoring system. The MiWaves trial had more control over data, allowing for more detailed monitoring.
Both trials encountered various yellow severity issues, such as communication failures and algorithm crashes, which were resolved using the fallback methods to prevent participant harm and data quality issues.
Green severity issues involved documenting all incidents to properly adjust statistical analyses, highlighting the importance of coordination between software development teams.
The monitoring systems described safeguarded participants and ensured high-quality data, giving digital health intervention teams the confidence to incorporate online decision-making algorithms.
Статистика
The Oralytics trial ran for 70 days with 79 participants, while the MiWaves trial ran for 30 days with 122 participants.
Цитаты
"Without these algorithm monitoring systems, critical issues would have gone undetected and unresolved. Instead, these monitoring systems safeguarded participants and ensured the quality of the resulting data for updating the intervention and facilitating scientific discovery."