Reinforcement Learning Control for Water Distribution Networks with Periodic Disturbances

Jorge Val Ledesma*, Rafal Wisniewski, Carsten Kallesøe

*Corresponding author for this work

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review


Cost efficient management of Water Distribution Networks with storage units requires of extensive knowledge of the water network. However, the network models are not always available or the calibration costs are too high for most of small water utilities. This paper proposes a model-free control solution based on Q-learning methods that provides a policy for the operation of the network. This supervisory controller must guarantee the water supply despite of the uncertainty of the daily water consumption and reduce the operation cost. The function approximation proposed for the Q-learning controller uses Fourier Basis Functions which provide an accurate approximation of the periodic disturbances. This paper presents results of the control validation in a simulation framework as well as experimental evidence of the advantages and limitations of the proposed design.

Original languageEnglish
Title of host publication2021 Annual American Control Conference, ACC 2021
Number of pages6
Publication date2021
Article number9482787
ISBN (Print)978-1-7281-9704-3
ISBN (Electronic)978-1-6654-4197-1
Publication statusPublished - 2021
Event2021 American Control Conference (ACC) - New Orleans, United States
Duration: 25 May 202128 May 2021


Conference2021 American Control Conference (ACC)
Country/TerritoryUnited States
CityNew Orleans
SeriesAmerican Control Conference


  • Reinforcement Learning (RL)
  • Disturbance
  • Optimal Control
  • water distribution network
  • Water Supply


Dive into the research topics of 'Reinforcement Learning Control for Water Distribution Networks with Periodic Disturbances'. Together they form a unique fingerprint.

Cite this