The paper introduces an automated, data-driven method to characterize the creative process in humans and large language models (LLMs) by analyzing their semantic exploration patterns. The method uses sentence embeddings to categorize responses and compute semantic similarities, which are then used to generate "jump profiles" that signal transitions between semantic spaces.
The authors find that human participants exhibit both persistent (deep search within few semantic spaces) and flexible (broad search across multiple semantic spaces) pathways to creativity, with both leading to similar creativity scores. In contrast, LLMs are biased towards either persistent or flexible paths, which varies across tasks. While LLMs as a population match the variability in human response sequences on the Alternate Uses Task (AUT), their relationship with creativity is different - the more flexible LLMs score higher on originality.
The paper also discusses the reliability and validity of the proposed method, as well as its implications for using LLMs as artificial participants in cognitive science research and for human-AI co-creativity.
To Another Language
from source content
arxiv.org
Key Insights Distilled From
by Surabhi S. N... at arxiv.org 05-03-2024
https://arxiv.org/pdf/2405.00899.pdfDeeper Inquiries