The NUMTEMP dataset is a comprehensive collection of 15,514 real-world numerical claims from various fact-checking domains. The dataset addresses the challenge of verifying claims involving numerical quantities and temporal expressions, which are prevalent in political discourse but often overlooked by existing fact-checking datasets.
The dataset construction process involves:
The dataset is divided into training, validation, and test sets, with a distribution of 'True', 'False', and 'Conflicting' claims. The authors also categorize the numerical claims into four types: temporal, statistical, interval, and comparison.
The authors evaluate various fact-checking approaches on the NUMTEMP dataset, including claim decomposition, pre-trained models for numerical understanding, and different NLI models. The results show that NUMTEMP poses a significant challenge for fact-checking, with the best approach achieving a weighted-F1 of 64.89 for unified evidence and 69.79 for gold evidence. The authors also find that claim decomposition and models pre-trained on numerical understanding tasks can improve performance on numerical claims.
To Another Language
from source content
arxiv.org
Key Insights Distilled From
by Venktesh V,A... at arxiv.org 03-27-2024
https://arxiv.org/pdf/2403.17169.pdfDeeper Inquiries