Reinforcement Learning Control for Water Distribution Networks with Periodic Disturbances

Cost efficient management of Water Distribution Networks with storage units requires of extensive knowledge of the water network. However, the network models are not always available or the calibration costs are too high for most of small water utilities. This paper proposes a model-free control solution based on Q-learning methods that provides a policy for the operation of the network. This supervisory controller must guarantee the water supply despite of the uncertainty of the daily water consumption and reduce the operation cost. The function approximation proposed for the Q-learning controller uses Fourier Basis Functions which provide an accurate approximation of the periodic disturbances. This paper presents results of the control validation in a simulation framework as well as experimental evidence of the advantages and limitations of the proposed design.