Multiobjective Deep Reinforcement Learning based Joint Beamforming and Power Allocation in UAV assisted Cellular Communication

Abstract In order to provide spectrum and energy efficient communication for unmanned aerial vehicle (UAV) assisted cellular network, the problem of joint beamforming and power allocation (JBPA) in aerial multicell scenario is addressed. The JBPA multiobjective optimization model which would simultaneously maximize the achievable spectrum and energy efficiency is first developed. In view of the model, the centralized deep reinforcement learning (DRL) algorithm, i.e., upper confidence bound based Dueling deep Q network (UCB DDQN) with Mish activation function, is proposed to solve the multiobjective optimization problem and we make use of this learning algorithm to design joint beamforming and power allocation strategy. Furthermore, a federated UCB DDQN learning based JBPA is to proposed tackle the challenge of centralized DRL would require excessive data exchange. Simulation results validate that the faster convergence speed and the total weighted energy-spectrum efficiency (TWESE) achieved by the joint beamforming and power allocation based on UCB DDQN is greater than conventional DQN based resource allocation approach, and show the superior TWESE performance federated UCB DDQN achieve compared to centralized UCB DDQN.