Reinforcement Learning-Based PID for Nonlinear Temperature Control with Delayed Feedback and Measurement Noise

Temperature control systems in practice are often affected by nonlinear dynamics, delayed feedback, and measurement noise, which makes reliable controller tuning challenging. This study examines how different PID tuning strategies perform under such realistic conditions, considering both classical PID and nonlinear PID controller structures. Traditional tuning approaches are compared with optimization-based and reinforcement learning--based methods within a unified simulation framework, allowing a fair assessment of their robustness and efficiency. In addition to standard tracking and transient performance measures, control effort is included as an energy-related indicator to reflect practical operating concerns. The results show that classical and evolutionary tuning methods remain effective when disturbances are moderate, while reinforcement learning--based tuning provides superior robustness in highly nonlinear and noisy scenarios, albeit at the cost of increased control effort. These findings offer practical guidance on selecting appropriate PID tuning strategies for nonlinear temperature control systems operating under uncertainty.