The article introduces MindEye2, a novel approach for reconstructing visual perception from brain activity using only 1 hour of fMRI training data. By pretraining a model across multiple subjects and fine-tuning on limited data from a new subject, high-quality reconstructions are achieved. The method involves mapping brain data to a shared-subject latent space and then to CLIP image space, improving generalization and state-of-the-art image retrieval metrics. MindEye2 innovates upon previous approaches by incorporating functional alignment procedures and refining reconstructions through Stable Diffusion XL unCLIP models. The study showcases the potential for accurate reconstructions of perception from single MRI visits, enabling clinical applications and brain-computer interfaces.
To Another Language
from source content
arxiv.org
Key Insights Distilled From
by Paul S. Scot... at arxiv.org 03-19-2024
https://arxiv.org/pdf/2403.11207.pdfDeeper Inquiries