Core Concepts
Despite the rapid development of AI models in medical image analysis, their validation in real-world clinical settings remains limited. This study introduces a generic framework for deploying image-based AI models in clinical settings and evaluates the deployment of a deep learning model for fetal ultrasound standard plane detection.
Abstract
The authors introduce a generic framework for deploying image-based AI models in real-world clinical settings. The framework is designed to address key challenges such as device output compatibility, low prediction latency, local processing, wireless display, video recording, and ease of use for research code.
Using this framework, the authors deployed a trained deep learning model for fetal ultrasound standard plane detection and evaluated it in real-time sessions with both novice and expert users. The key findings include:
- Novice users found the model's explanations on the presence of anatomical landmarks helpful, but requested more navigational guidance to reach the standard planes.
- Expert users used the model's predictions for confirmation rather than relying on it for guidance, highlighting the different use cases for AI tools between novice and expert users.
- Participants expressed a desire for the prediction results to be displayed directly on the ultrasound machine, and requested a higher frame rate to better match their scanning speed.
These findings underscore the importance of early deployment of AI models in real-world settings, as it can provide valuable insights to guide the refinement of the model and system based on actual user feedback.
Stats
The authors measured the time taken to process one video frame using different computational devices, both by running the native research code and using their framework. The results show that the framework introduces a small additional latency of around 0.05-0.06 seconds compared to running the native code directly.
Quotes
"Almost all participants expressed a desire for the prediction results to be displayed directly on the ultrasound machine."
"P1, P3 & P4 expressed their wish for more navigational support. They acknowledged that a higher frame rate might be helpful, but ultimately it would be ideal if the tool could tell them the direction they should move the probe if they wanted to reach a certain standard plane."
"P7 used the tool for confirmation of thoughts, while P8 tended to rely on self-judgement rather than relying on feedback from PCBM."