The author explores the impact of simplicity bias on neural networks and proposes Spare as a method to identify and mitigate spurious correlations early in training.