核心概念
Understanding user motives and model naturalness in ChatGPT conversations.
要約
The study explores ChatGPT behavior in natural and role-play settings.
Dataset CRD annotated for user motives and model naturalness.
Statistical analysis reveals differences in conversation length and question frequency.
ChatGPT responses analyzed for naturalness and alignment with user motives.
Perplexity and sentiment analysis provide insights into model performance and user sentiment.
Findings suggest users prioritize human-like interactions with ChatGPT.
統計
"CRD consists of conversations conducted in two settings: vanilla, boss, and classmate."
"Participants largely comparable in age and prior experience using text chatbots."
"CRD dataset includes 57 participants, 85 unique conversations, and 1742 utterances."
"Average conversation length in vanilla: 29.59 turns, boss: 14.57 turns, classmate: 17.11 turns."
"Average utterance length (Human) in vanilla: 12.18 words, boss: 20.58 words, classmate: 19.06 words."
"Average utterance length (ChatGPT) in vanilla: 77.66 words, boss: 35.78 words, classmate: 46.10 words."
"Correlation between human and ChatGPT utterance lengths: 0.20 (vanilla), 0.14 (boss), 0.25 (classmate)."
"Questions as percentage of conversation (Human) in vanilla: 26.34%, boss: 21.32%, classmate: 21.29%."
"Questions as percentage of conversation (ChatGPT) in vanilla: 14.69%, boss: 20.34%, classmate: 32.57%."
"Correlation between human questions and number of turns: 0.87 (vanilla), 0.68 (boss), 0.51 (classmate)."
"Correlation between ChatGPT questions and number of turns: 0.65 (vanilla), 0.77 (boss), 0.83 (classmate)."