WD3QNE: A value-based deep reinforcement learning model with human expertise in optimal treatment of sepsis

Deep Reinforcement Learning (DRL) has been increasingly attempted in assisting clinicians for real-time treatment of sepsis. While a value function quantifies the performance of policies in such decision-making processes, most value-based DRL algorithms cannot evaluate the target value function precisely and are not as safe as clinical experts. In this study, we propose a Weighted Dueling Double Deep Q-Network with embedded human Expertise (WD3QNE) that incorporates a target Q value function with adaptive dynamic weight for enhanced policy improvement and human expertise in decision-making for sepsis treatment. In addition, the random forest algorithm is employed for feature selection to improve model interpretability. We test our algorithm against state-of-the-art value function methods in terms of expected return, survival rate and action distribution. The results demonstrate that WD3QNE obtains the highest survival rate of 97.81%. Our proposed method is capable of providing reliable treatment decisions with embedded clinician expertise.