Improving Generalization and Consistency of Factual Knowledge Extraction from Language Models through Debiasing

Simultaneous debiasing of the misalignment between pre-training and downstream tuning objectives can improve the generalization and consistency of factual knowledge extraction from language models.