Limitations of the Knowledge Neuron Thesis in Explaining Language Model Capabilities

The Knowledge Neuron (KN) thesis, which proposes that facts are stored in the MLP weights of language models, is an oversimplification. Language models exhibit complex mechanisms for processing both linguistic and factual information that cannot be fully explained by the KN thesis.