Performance comparison of PI and DQN-PI controller over hydraulic cylinder system

Abstract

Electro-hydraulic servo systems are widely used in industry, but their trajectory tracking performance is easily affected by nonlinear flow dynamics, load mass variations, and damping uncertainties. Traditional PID/PI controllers are simple and practical, but their fixed gains often struggle to balance multiple performance metrics. To improve tracking performance, this thesis proposes and evaluates a PI control framework based on the Deep Q-Network (DQN-PI), in which the PI controller serves as the low-level feedback loop and the DQN agent adjusts the PI gains online. A MATLAB/Simulink model of a valve-controlled asymmetric hydraulic cylinder is developed as the simulation object. Based on state observations, gain adjustment actions, and the reward function, a reinforcement learning (RL) environment is constructed for online parameter tuning. After online training, the control performance of the DQN-PI controller is compared with that of various fixed-gain PI controllers under the same simulation conditions. The results show that the DQN-PI controller achieves a better trade-off in multi-step trajectory tracking. The DQN-PI controller reduces the RMSE from 6.86 mm to 5.84 mm and shortens the settling time from 0.27 s to 0.07 s relative to the manually tuned PI controller. It also lowers the overshoot from 9.91% to 4.47% compared with the grid-optimal PI controller, while maintaining similar tracking accuracy. Furthermore, the DQN-PI controller also maintains stable tracking performance under conditions involving an untrained sinusoidal reference, load mass variations, and damping uncertainties. These results indicate that the DQN-based online PI gain tuning can improve the trajectory tracking performance of the hydraulic cylinder system.