A Reinforcement Learning based Computing Offloading and Resource Allocation Scheme in F-RAN

Abstract This paper investigates a computing offloading policy and the allocation of computational resource for multiple user equipments (UEs) in Device-to-Device (D2D) aided fog radio access networks (F-RANs). Concerning the dynamically changing wireless environment where the channel state information (CSI) is difficult to predict and know exactly, we formulate the problem of task offloading and resource optimization as a mixed-integer nonlinear programming problem to maximize the total utility of all UEs. Concerning the non-convex property of the formulated problem, we decouple the original problem into two phases to solve. Firstly, a centralized deep reinforcement learning (DRL) algorithm called Dueling Deep Q-Network (DDQN) is utilized to obtain the most suitable offloading mode for each UE. Particularly, to reduce the complexity of the proposed offloading scheme based DDQN algorithm, a pre-processing procedure is adopted. Then a distributed Deep Q-Network (DQN)algorithm based on the training result of the DDQN algorithm is further proposed to allocate the appropriate computational resource for each UE. Combining these two phases, the optimal offloading policy and resource allocation for each UE are finally achieved. Simulation results demonstrate the performance gains of the proposed scheme compared with other existing baseline schemes.