Deep learning based simulators for the phosphorus removal process control in wastewater treatment via deep reinforcement learning algorithms

Phosphorus removal is vital in wastewater treatment to reduce reliance on limited resources. Deep reinforcement learning (DRL) can be used to optimize the processes in wastewater treatment plants by learning control policies through trial and error. However, applying DRL to chemical and biological processes is challenging due to the need for accurate simulators. This study trained six models to identify the phosphorus removal process and used them to create a simulator for the DRL environment. While achieving high accuracy (>97%) in one-step ahead prediction of the test dataset, these models struggled as simulators over longer horizons, showing uncertainty and incorrect predictions when using their own outputs for multi-step simulations. Compounding errors in the models’ predictions were identified as one of the causes of this problem. This approach for improving process control involves creating simulation environments for DRL algorithms, using data from supervisory control and data acquisition (SCADA) systems with a sufficient historical horizon without complex system modeling or parameter estimation.