LayoutLLM: Enhancing Large Language Models for Document Understanding through Layout Instruction Tuning

LayoutLLM is an LLM/MLLM based method that integrates a document pre-trained model as encoder and employs a novel layout instruction tuning strategy to enhance the comprehension and utilization of document layouts for improved zero-shot document understanding.