Scalable Distributed Multi-Agent Reinforcement Learning Using Graph-Induced Local Value Functions

The core message of this paper is to develop a distributed and scalable reinforcement learning framework for large-scale cooperative multi-agent systems by leveraging the structures of graphs involved in the problem, including state graph, observation graph, reward graph, and communication graph. The proposed approach constructs local value functions for each agent that can effectively capture the global objective while significantly reducing the sample complexity and computational complexity compared to centralized and consensus-based distributed RL algorithms.