Enhancing Unsupervised Cross-Modality Adaptation for 3D Medical Image Segmentation with Filtered Pseudo Labels

Core Concepts

The authors propose an enhanced Filtered Pseudo Label (FPL+) framework to effectively adapt a medical image segmentation model from one modality to another without requiring labeled data in the target domain. The key innovations include cross-domain data augmentation, a dual-domain pseudo label generator, and joint training with image-level and pixel-level weighting to mitigate the impact of noisy pseudo labels.

Abstract

The paper presents a novel unsupervised domain adaptation (UDA) framework, FPL+, for cross-modality 3D medical image segmentation. The main components are:

Cross-Domain Data Augmentation (CDDA): The authors augment labeled source-domain images into a dual-domain training set, consisting of a pseudo source-domain set and a pseudo target-domain set with shared labels. This enhances the diversity of the training data and reduces the risk of overfitting to structure distortions.

Dual-Domain pseudo label Generator (DDG): The authors propose a pseudo label generator that uses dual batch normalization to effectively learn from the augmented dual-domain training set, generating high-quality pseudo labels for the target-domain images.

Pseudo Label Filtering: To mitigate the impact of noisy pseudo labels, the authors introduce image-level weighting based on size-aware uncertainty estimation and pixel-level weighting based on dual-domain consensus. This helps the final segmentor focus on reliable pseudo labels during training.

Joint Training: The final segmentor is trained jointly on the labeled source-domain images and target-domain images with filtered pseudo labels. This allows the model to leverage knowledge from both domains to improve performance in the target domain.

The authors validate their method on three public multi-modal medical image datasets for Vestibular Schwannoma, brain tumor, and whole heart segmentation. The results show that FPL+ outperforms ten state-of-the-art UDA methods and even achieves better performance than fully supervised learning in some cases.

Stats

The authors used the following key metrics and figures in the paper:
"The Dice scores of "w/o DA" for different cardiac structures, including AA, LAC, LVC and MYO, were 15.20%, 53.16%, 5.96%, and 6.05%, respectively, and the average Dice score (20.09%) was significantly lower than the "labeled target" (84.84%)."
"For "FLAIR to T2", the Dice scores of "w/o DA" were 47.16%, indicating a certain domain gap between the two modalities."
"For "ceT1 to hrT2", the "w/o DA" method obtained an average Dice of 0.00% and 2.65% in "hrT2 to ceT1", respectively, indicating a significant domain shift between the two modalities."

Quotes

"Our FPL+ was inferior to "strong upbound", but was even slightly better than "labeled target" (91.98% vs 90.72% in terms of Dice), which was mainly due to that our segmentor leverages images from both domains for learning."
"FPL+ achieved an average Dice of 84.81% with an average ASSD of 2.72 mm, surpassing "labeled target" and falling slightly behind the "strong upper bound" that achieved a Dice of 86.69% and ASSD of 2.30 mm."

Key Insights Distilled From

FPL+

by Jianghao Wu,... at arxiv.org 04-09-2024

https://arxiv.org/pdf/2404.04971.pdf

Deeper Inquiries

How can the proposed FPL+ framework be extended to handle more than two modalities for medical image segmentation

The FPL+ framework can be extended to handle more than two modalities for medical image segmentation by incorporating additional image translators and pseudo label generators for each new modality. For each additional modality, a new set of image translators can be trained to translate images from the source domain to the new target domain and back. Similarly, a new pseudo label generator can be trained to generate high-quality pseudo labels for the images in the new target domain. By repeating this process for each new modality, the FPL+ framework can adapt to multiple modalities simultaneously, allowing for more comprehensive cross-modality adaptation in medical image segmentation.

What other types of domain shifts, beyond imaging modalities, could the FPL+ framework be applied to (e.g., different scanners, institutions, patient populations)

The FPL+ framework can be applied to various types of domain shifts beyond imaging modalities in medical image segmentation. For example, it can be used to adapt segmentation models to different scanners, institutions, patient populations, or imaging protocols. By training the pseudo label generator and segmentor on data from different scanners or institutions, the framework can learn to generalize across different imaging settings and capture the variations in image appearance and quality. Additionally, by incorporating patient-specific information or demographic data into the training process, the framework can adapt to different patient populations and account for variations in anatomy and pathology.

Given the promising results, how could the FPL+ framework be integrated into a clinical workflow to assist radiologists and clinicians in segmenting medical images across different modalities

To integrate the FPL+ framework into a clinical workflow for medical image segmentation, radiologists and clinicians can use it as a tool to assist in segmenting images across different modalities. The framework can be used to automatically generate high-quality pseudo labels for unlabeled images in new modalities, reducing the manual annotation burden and improving efficiency. Radiologists can then review and refine the segmentations generated by the framework, leveraging their expertise to ensure accuracy and clinical relevance. By incorporating the FPL+ framework into existing segmentation pipelines, clinicians can streamline the segmentation process, improve consistency across modalities, and enhance the overall quality of medical image analysis.