Основні поняття
A framework called GAME-KG that leverages human computation gaming to modify and validate knowledge graphs, enabling their use to augment large language models in accuracy-critical domains.
Анотація
The GAME-KG framework is a six-step approach that leverages human computation gaming (HCG) to modify and validate knowledge graphs (KGs) for use in accuracy-critical generative AI applications.
The key steps are:
Discover Data: Identify and collect the documents that will be used to develop the domain model.
Parse Text: Extract entities and their relationships from the documents.
Build KG and Generate Narrative: Construct the KG from the parsed text, and use a language model to generate a fictionalized narrative based on the KG.
Identify Graph: Determine which parts of the KG may benefit most from human feedback, e.g. based on connection strength.
Inject Graph and Narrative: Incorporate the KG and narrative into the HCG.
Collect Player Feedback: Players provide feedback to modify and validate the KG, which is then used to update the graph.
The framework is demonstrated through two examples:
A Unity-based video game called "Dark Shadows" that collects player feedback to modify KGs parsed from human trafficking press releases.
An experiment where GPT-4 is prompted to answer questions based on the original KG and a human-modified KG, showing how the modified KG can provide more explainable responses.
The initial results suggest that GAME-KG can be an effective way to enhance KGs with human feedback, enabling their use to augment large language models in accuracy-critical domains like human trafficking data analysis.
Статистика
Kizer trafficked victims across state borders.
Villaman was an accomplice to Kizer.
Цитати
"Providing LLMs with external, structured representations of facts in the form of knowledge graphs (KGs) has proven useful for addressing this limitation [of explainability]."
"Crowdsourcing has been used to collect knowledge that may be obvious to a human but not a machine."