Optimizing Reservoir Weights for Efficient Approximation of Linear Time-Invariant Systems using Recurrent Neural Networks

The core message of this work is that randomly generating the recurrent weights (reservoir weights) of a recurrent neural network, specifically an echo state network, can provide an optimal approximation of a general linear time-invariant system. The authors derive the optimal probability distribution for configuring these reservoir weights and show that this distribution is also optimal for approximating higher-order linear time-invariant systems.