Core Concepts
Introducing ChartInstruct, a dataset and models for instruction tuning in chart comprehension and reasoning.
Abstract
Charts are essential for data analysis, but understanding them can be challenging. Various tasks like question-answering and summarization have emerged. Existing models may not be optimal for chart-specific tasks. ChartInstruct introduces a novel dataset with instructions for chart understanding. Two systems are presented: an end-to-end model and a pipeline model. The models achieve state-of-the-art results on four downstream tasks, expanding the applicability of models to new tasks.
Stats
"191K instructions generated with 71K charts."
"Models achieve state-of-the-art performance."
"End-to-end system utilizes LLaVA architecture."
"Pipeline system skips alignment step."
"Human evaluation confirms effectiveness of instruction-tuning approach."
Quotes
"Our main contributions include: A new instruction-following corpus with real-world charts and a wide range of tasks by utilizing LLMs."
"ChartInstruct surpasses previous state-of-the-art models on various benchmarks."
"Human evaluation further suggests the effectiveness of our instruction-tuning approach in supporting a wide array of real-world chart comprehension and reasoning scenarios."