Temel Kavramlar
High-quality fMRI-to-image reconstructions achieved with minimal data using shared-subject models.
Özet
The article introduces MindEye2, a novel approach for reconstructing visual perception from brain activity using only 1 hour of fMRI training data. By pretraining a model across multiple subjects and fine-tuning on limited data from a new subject, high-quality reconstructions are achieved. The method involves mapping brain data to a shared-subject latent space and then to CLIP image space, improving generalization and state-of-the-art image retrieval metrics. MindEye2 innovates upon previous approaches by incorporating functional alignment procedures and refining reconstructions through Stable Diffusion XL unCLIP models. The study showcases the potential for accurate reconstructions of perception from single MRI visits, enabling clinical applications and brain-computer interfaces.
İstatistikler
"1 hour of fMRI training data"
"7 subjects pretraining"
"40 hours of training data per subject"
"State-of-the-art image retrieval metrics"
"4096-dim latent space"