Investigating Knowledge Updating Behaviors of Large Language Models in Realistic Scenarios

Large language models often fail to update their incorrect parametric knowledge when presented with factual contextual information, due to a parametric bias where the appearance of the incorrect parametric answer in the context makes the knowledge update more likely to fail.