MulCogBench introduces a multi-modal cognitive benchmark dataset for Chinese and English language models. The dataset includes eye-tracking, fMRI, MEG, and semantic ratings. Results indicate that language models share similarities with human cognitive data across languages.
The study explores the relationship between computational models and cognitive data. Context-aware models outperform context-independent ones as linguistic stimulus complexity increases. Different layers of context-aware models align differently with brain representations.
Comparisons are made between Chinese and English datasets, linguistic units, modalities of cognitive data, and regions of interest in the brain. The findings suggest generalizability across languages and variations in model performance based on linguistic complexity.
The research highlights the importance of cognitive data in evaluating computational language models' similarity to human cognition. Further analysis is needed to understand the mechanisms underlying these similarities.
Egy másik nyelvre
a forrásanyagból
arxiv.org
Mélyebb kérdések