Our framework integrates 3D brain structures with visual semantics using Vision Transformer 3D, enabling efficient visual reconstruction and multimodal interaction from single-trial fMRI data without the need for subject-specific models.
DREAM is a visual decoding method that mirrors the forward pathways of the human visual system to decipher semantics, color, and depth cues from fMRI data, and then uses these cues to guide the reconstruction of the viewed images.