Khái niệm cốt lõi
Choice-75 challenges intelligent systems to make decisions based on descriptive scenarios, highlighting the need for multi-hop reasoning in script learning.
Tóm tắt
Abstract:
Script learning studies stereotypical events unfolding for machine reasoning.
Choice-75 introduces decision branching in scripts with 75 examples and over 600 scenarios.
Introduction:
Events are fundamental, requiring understanding of event-to-event relationships.
Script learning focuses on how stereotypical events unfold.
Goal & Options:
Dataset includes goals, options, scenarios, and ground-truth choices.
Scenarios categorized as easy, medium, hard, or either based on complexity.
Manual Scenario Annotation:
Scenarios written in verb phrase format.
Difficulty levels determined by the number of reasoning steps required.
Human-in-the-Loop Generation:
Two subsets of hard scenarios generated using human-in-the-loop paradigm.
Verb phrase and user profile formats used for scenario generation.
Method and Experiments:
Task formulated as an in-context learning task for predicting optimal choices.
Results and Analysis:
Human judgment aligns with model performance across difficulty levels.
Related Work:
Event-centric reasoning and script learning explored alongside human decision-making studies.
Limitations:
Dataset distribution limited to daily procedures from proScript.
For more detailed information, refer to the original content.
Thống kê
Choice-75は、人工知能システムに記述的なシナリオに基づいて意思決定を行うよう挑戦し、スクリプト学習におけるマルチホップ推論の必要性を強調しています。