Reinforced Variable Selection via Natural Policy Gradient

Variable selection identifies the best subset of covariates when building the prediction model, among all possible subsets. In this paper, we propose a novel reinforced variable selection method, called “Actor-Critic-Predictor”. The actor takes an action to choose variables and the predictor evaluates the action based on a well-designed reward function, where the reward baseline is learned by the critic. We model the variable selection process as a multi-armed bandit and update the subset of variables being selected using a natural policy gradient algorithm. We provide an analytical framework on how different errors impact the performance of our method theoretically. Large amounts of experiments on both synthetic and real datasets show that the proposed framework is easy-implemented and outperforms classical variable selection methods in a wide range of scenarios.