MulCogBench introduces a multi-modal cognitive benchmark dataset for Chinese and English language models. The dataset includes eye-tracking, fMRI, MEG, and semantic ratings. Results indicate that language models share similarities with human cognitive data across languages.
The study explores the relationship between computational models and cognitive data. Context-aware models outperform context-independent ones as linguistic stimulus complexity increases. Different layers of context-aware models align differently with brain representations.
Comparisons are made between Chinese and English datasets, linguistic units, modalities of cognitive data, and regions of interest in the brain. The findings suggest generalizability across languages and variations in model performance based on linguistic complexity.
The research highlights the importance of cognitive data in evaluating computational language models' similarity to human cognition. Further analysis is needed to understand the mechanisms underlying these similarities.
In un'altra lingua
dal contenuto originale
arxiv.org
Approfondimenti chiave tratti da
by Yunhao Zhang... alle arxiv.org 03-05-2024
https://arxiv.org/pdf/2403.01116.pdfDomande più approfondite