Enhanced Mode-Locked Laser Control with Deep Reinforcement Learning: A Comparative Study with DDQN, TD3 and SAC Algorithms

Abstract Deep reinforcement learning algorithms, i.e. Double Deep Q Network (DDQN), Twin Delayed Deep Deterministic policy gradient (TD3), and Soft Actor-Critic (SAC), have recently found their applications in laser mode-locking. However, their performances have not yet been compared in detail. We show the implementation of the three AI algorithms on a simulated fiber laser mode-locked by nonlinear polarization rotation. The reward function is specifically designed to maximize the tendency of mode-locking by introducing kurtosis in frequency domain. The training curves indicate that TD3 and SAC outperform DDQN in multi-input context. DDQN has difficulty in handling multiple inputs and tends to have issues with convergence. While TD3 is most stable and efficient, SAC is more advantageous in searching various states. The simulation pulse evolution diagrams of these states are also listed to confirm the feasibility of training a mode-locked laser model through the three AI algorithms.