Core Concepts
Instruction-tuned LLMs like Jellyfish enhance DP performance and generalizability.
Abstract
The paper explores the use of large language models (LLMs) for data preprocessing (DP), focusing on instruction-tuning local LLMs to serve as universal DP task solvers. The Jellyfish dataset is introduced, enabling manual crafting of instructions for DP tasks and enhancing model interpretability. Experiments show that Jellyfish models outperform state-of-the-art methods on seen and unseen tasks, showcasing their competitiveness and generalizability. The impact of tuning with different datasets on DP performance is analyzed, highlighting the importance of multi-task tuning in improving overall performance.
Stats
Mistral-7B and OpenOrca-Platypus2-13B models deliver competitiveness compared to state-of-the-art DP methods.
Jellyfish-13B consistently outperforms non-LLM methods on seen datasets.
Jellyfish models exhibit strong generalizability to unseen tasks beyond what they are tuned for.