Core Concepts
This paper proposes a hierarchical learning scheme, specifically a Hierarchical Deep-Q-Network (h-DQN) framework, to enable efficient and intelligent traffic steering in Open Radio Access Networks (O-RAN).
Abstract
The paper provides background on traffic steering in O-RAN and surveys various machine learning techniques that can be applied to this problem, including deep learning, reinforcement learning, federated learning, and hierarchical learning.
The key contributions are:
- Introduction of a hierarchical learning scheme for traffic steering in O-RAN, which decomposes the problem into a bi-level architecture with a meta-controller and a controller.
- The meta-controller in the non-RT-RIC sets high-level goals for traffic steering, while the controller in the near-RT-RIC makes real-time traffic steering decisions to achieve these goals.
- A case study demonstrating the implementation of the proposed h-DQN framework and its performance advantages over baseline algorithms in terms of improved throughput and reduced network delay.
- Discussion on the integration of the hierarchical learning scheme with the O-RAN architecture and the AI/ML workflow.
The hierarchical learning approach allows for flexible decision-making at different levels of the network hierarchy, leading to better exploration efficiency, faster convergence, and improved network performance compared to standalone machine learning algorithms.
Stats
"The proposed h-DQN-based traffic steering scheme gains a significant performance increase based on our simulation results compared to the DRL and heuristic baselines. It achieves an average increase in throughput of 15.55% and 6.46% over the threshold-based heuristic and DQN algorithms, respectively. Furthermore, the h-DQN scheme demonstrates a lower network delay of 27.74% and 58.96% compared to the same baselines."
Quotes
"Decomposing the traffic steering problem using h-DQN can bring higher exploration efficiency, faster convergence, and better network performance."
"The hierarchical structure of h-DQN allows for flexible decision-making at two distinct levels of the network hierarchy via two different agents interacting with the environment."