Control of HVAC-systems with Slow Thermodynamic Using Reinforcement Learning

Christian Blad, Søren Koch, Sajuran Ganeswarathas, Carsten Kallesøe, Simon Bøgh

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

26 Downloads (Pure)

Abstract

This paper proposes an adaptive controller based on Reinforcement Learning (RL), which copes with HVAC-systems consisting of slow thermodynamics. Two different RL algorithms with Q-Networks (QNs) are investigated. The HVAC-system is in this study an underfloor heating system. Underfloor heating is of great interest because it is very common in Scandinavia, but this research can be applied to a wide range of HVAC-systems, industrial processes and other control applications that are dominated by very slow dynamics. The environments consist of one, two, and four zones within a house in a simulation environment meaning that agents will be exposed to gradually more complex environments separated into test levels. The novelty of this paper is the incorporation of two different RL algorithms for industrial process control; a QN and a QN + Eligibility Trace (QN+ET). The reason for using eligibility trace is that an underfloor heating environment is dominated by slow dynamics and by using eligibility trace the agent can find correlations between the reward and actions taken in earlier iterations.
Original languageEnglish
Title of host publication29th International Conference on Flexible Automation and Intelligent Manufacturing : FAIM 2019
Number of pages8
Volume38
PublisherElsevier
Publication date2019
Pages1308-1315
DOIs
Publication statusPublished - 2019
Event29th International Conference on Flexible Automation and Intelligent Manufacturing (FAIM2019) - Limerick, Ireland
Duration: 24 Jun 201928 Jun 2019
https://faim2019.org

Conference

Conference29th International Conference on Flexible Automation and Intelligent Manufacturing (FAIM2019)
CountryIreland
CityLimerick
Period24/06/201928/06/2019
Internet address
SeriesProcedia Manufacturing
ISSN2351-9789

Keywords

  • Reinforcement Learning
  • Artificial Intelligence (AI)
  • Machine Learning
  • Deep Learning
  • HVAC Systems

Fingerprint Dive into the research topics of 'Control of HVAC-systems with Slow Thermodynamic Using Reinforcement Learning'. Together they form a unique fingerprint.

  • Equipment

    AI Cloud Service

    CLAAUDIA

    Facility: Equipment

  • Cite this

    Blad, C., Koch, S., Ganeswarathas, S., Kallesøe, C., & Bøgh, S. (2019). Control of HVAC-systems with Slow Thermodynamic Using Reinforcement Learning. In 29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019 (Vol. 38, pp. 1308-1315). Elsevier. Procedia Manufacturing https://doi.org/10.1016/j.promfg.2020.01.159