LORS introduces a low-rank residual structure to reduce parameters in stacked modules while maintaining performance.