Kernkonzepte
A method is proposed to map input volumetric measurements to a latent space where overlapping signal components are disentangled, enabling their isolation and quantification through the application of bandpass filters.
Zusammenfassung
The authors present a novel approach called "Latent Unmixing" for processing mixed images into their individual contributing components. The method uses a 3D U-Net convolutional neural network to combine the spatial and time/spectral dimensions of the input data, which is essential when each dimension alone does not contain enough information to allow correct separation of the components.
The key highlights and insights are:
- The 3D U-Net is used to map the input volumetric data to a latent space where the different signal components are disentangled.
- Predefined bandpass filters are then applied to the latent space to pool the disentangled components into separate output channels.
- The method is demonstrated on three test cases: a variation of the MNIST dataset with simulated decay, fluorescence lifetime microscopy (FLIM), and mode decomposition in optical fibers.
- For the MNIST dataset, the Latent Unmixing method outperforms the established Maximum Likelihood Estimation (MLE) approach, achieving high Pearson correlation between true and predicted pixel intensities.
- In the FLIM experiments, Latent Unmixing shows better robustness to low photon counts and close lifetime values compared to the phasor analysis method.
- For mode decomposition in multi-mode fibers, Latent Unmixing can retrieve the spatial distributions of the modes, even when the sampling frequency is below the Nyquist criterion.
- The method is shown to generalize well to different types of input distributions, demonstrating its broad applicability.
Statistiken
The 3D-MNIST dataset was generated with the following parameters:
Signal-to-noise ratio (R) = 5
Decay constants: τ4 = 0.1, τ8 = 0.2, τ9 = 0.3, τb = 0.4
Time dimension histogram of L = 28 bins
The STED-FLIM dataset consists of 30 real microscopy images of four different proteins tagged with fluorescent markers in fixed cortical neurons. The images were split into 256 x 256 pixel crops, resulting in 1806 crops for training.
The S2-MMF dataset consists of simulated images of the intensity measured at the output of optical fibers for different input wavelengths. The dataset includes measurements with various combinations of up to 6 propagating modes, four different fiber geometries, and different mode-specific intensities. 1500 fiber measurements were used for training and 175 for testing.
Zitate
"Our method, called Latent Unmixing, transforms the overlapping input contributions to a latent space where they are untangled and can be separated by applying filters directly in the latent space."
"We chose to use a 3D U-Net for unmixing undersampled data so that the 3D kernels can combine essential information from neighboring pixels and neighboring time- or spectral-bins, since 3D convolution kernels allow the processing of all dimensions of a 3D input volume simultaneously."
"Our Latent Unmixing approach accurately predicts the pixel-wise contributions of 4 components on the test images in conditions where MLE and phasor approaches fail at correctly separating each channel."