Multiple quality-of-services optimization in space–air–ground integrated network: Centralized and decentralized deep reinforcement learning approaches

초록

The objective of the study is to develop efficient machine learning (ML) algorithms to enhance the multi-quality of service (QoS) in space–air–ground integrated networks (SAGIN), particularly in terms of coverage, data rate fairness, and energy consumption. In this work, we focus on developing efficient ML algorithms to enhance the multi QoS in SAGIN. Considering a network scenario involving low-Earth orbit (LEO) satellites and aerial vehicles such as high-altitude platforms (HAPs) and unmanned aerial vehicles (UAVs) equipped with array antennas, we formulate a QoS optimization problem that incorporates trajectory planning, bandwidth allocation, power control, and beam direction. To tackle this problem, we first design three centralized deep reinforcement learning (DRL) models: single-agent deep Q-learning (SA-DQL), single-agent deep actor–critic (SA-DAC), and single-agent deep deterministic policy gradient (SA-DDPG). Due to scalability issues in centralized approaches, we propose a new decentralized DRL algorithm called multi-agent DAC with experienced deep neural networks (MA-DAC-DNN), which improves training efficiency and convergence by distributing the learning process across multiple agents. In order to evaluate the performance of the proposed MA-DAC-DNN algorithm, we conduct a near-practical simulations scenario by using the street map of Jeju Island, South Korea, captured by Google Map. Simulations show that MA-DAC-DNN outperforms centralized DRL and convex programming in both optimality and convergence speed.

키워드