Random feature ridge regression exhibits a trade-off between approximation and generalization power in high-dimensional polynomial scaling.