Alapfogalmak
Large language models can strategically adjust conversation length to improve user satisfaction, especially for questions with high conversational potential.
Kivonat
This study investigates the impact of conversation length on user satisfaction when interacting with LLM-powered chatbots. The key findings are:
-
For questions with high conversational potential, longer conversations led to increased user satisfaction and perceived helpfulness. As the number of conversational turns increased from 3 to 7, user satisfaction scores rose.
-
However, the MTurk evaluation suggests that beyond a certain point, more conversation does not necessarily lead to higher effectiveness. Helpfulness scores improved as the number of turns increased for high-conversability questions, but declined when the number of turns reached 7.
-
Participants had mixed reactions - some found longer conversations more engaging and nuanced, while others considered them repetitive and not useful. The ideal conversation length appears to be dynamic and context-dependent.
-
The study demonstrates LLMs' ability to change conversation length, but cautions that changes in text form may not necessarily imply changes in quality or content. Strategically adjusting conversation formats to user situations can offer benefits, but requires careful design.
Statisztikák
"As the conversation length increased, satisfaction levels for high-conversability questions also rose."
"The helpfulness of responses to high-conversability questions increased with increasing conversation length."
"Participants may believe high-conversability questions necessitate more questions from the assistant."
Idézetek
"I don't think either of the bots particularly are better or worse than the other one so this is why I chose a 3. It felt like [SlackVanilla]'s direct responses to my questions were appropriate for the question types (typically ones that have factual/objective answers). For [MultiSlack], the questions were more focused on opinion/subjective topics, and I think its ability to provide follow-up questions is good for this case."
"Time matters. if I'm in a rush to get a quick answer from a robot who does not have any follow-up question or empathy/emotion, and then I would prefer [SlackVanilla]. However, if I would take some time to enjoy a one-on-one text conversation or seek for actual suggestion in a particular real life scenario (hypothetically), and then I would prefer [MultiSlack] in general."