Efficiently rectify language models to mitigate toxicity, bias, and privacy issues while preserving overall model performance.