The paper proposes a method called Generate-on-Graph (GoG) to address the task of Incomplete Knowledge Graph Question Answering (IKGQA). IKGQA differs from conventional Knowledge Graph Question Answering (KGQA) in that the given KG in IKGQA does not contain all the factual triples required to answer the questions.
The key highlights of the paper are:
Motivation: In real-world scenarios, KGs are often incomplete, and LLMs contain rich knowledge and reasoning abilities. Therefore, evaluating LLMs' ability to integrate internal and external knowledge is important.
Approach: GoG adopts a selecting-generating-answering framework. It treats the LLM as both an agent to explore the KG and a knowledge source to generate new factual triples based on the explored subgraph and its inherent knowledge.
Experiments: GoG outperforms previous methods, including semantic parsing and retrieval augmented approaches, on two IKGQA datasets (WebQSP and CWQ). The results demonstrate that even an incomplete KG can still help LLMs answer complex questions by providing related structured information.
Ablation Study: The paper analyzes the impact of the explored subgraph and the number of related triples on GoG's performance. Utilizing the subgraph information and an appropriate number of related triples can significantly improve the model's ability to generate new knowledge.
Overall, the paper proposes a novel method that effectively leverages the strengths of LLMs and incomplete KGs to address the IKGQA task, which is closer to real-world scenarios and can better evaluate LLMs' reasoning abilities.
In un'altra lingua
dal contenuto originale
arxiv.org
Approfondimenti chiave tratti da
by Yao Xu,Shizh... alle arxiv.org 04-24-2024
https://arxiv.org/pdf/2404.14741.pdfDomande più approfondite