Maximizing Throughput of Aerial Base Stations via Resources-based Multi-Agent Proximal Policy Optimization: A Deep Reinforcement Learning Approach

Fifth-generation (5G) networks use millimeter-wave (mmWave) technology to process high-speed and capacity data services. However, wireless communication losses occur due to mmWave limitations, i.e., penetration, rain attenuation, and coverage range. Furthermore, many base stations (BSs) are needed to support stable wireless communications and overcome coverage distances in rural and suburban areas. Therefore, a new wireless communication platform that supports communication services at the aerial level is required. Furthermore, this aerial platform enables line-of-sight (LoS) communications rather than non-LoS (NLoS), which is advantageous in overcoming ground-level losses. Thus, an unmanned aerial vehicle (UAV) or an unmanned aerial platform (UAP) that can be rapidly and dynamically deployed at the point of interest is considered. Despite these benefits, UAV-BSs (also known as aerial BSs) still have optimization problems to solve, i.e., resource allocation and trajectory optimization. Thus, this study considered resource-based multi-agent deep reinforcement learning (MADRL) to solve the resource allocation and trajectory optimization problems of UAV-BSs at the same time. However, our proposed optimization problem is non-convex. Thus we proposed an algorithm based on multi-agent proximal policy optimization (MAPPO) DRL. The proposed algorithm treats each agent as a resource variable to perform optimization more effectively. As a result, the proposed algorithm achieved faster convergence and higher rewards than the baselines.