Identifying Neurons Encoding Speech Properties in Self-Supervised Transformer Models

Neurons in the feedforward layers of self-supervised speech Transformer models encode specific properties of speech, such as phones, gender, and pitch. These "property neurons" can be identified and leveraged for model editing and pruning.