Control of HVAC-systems with Slow Thermodynamic Using Reinforcement Learning

Christian Blad, Søren Koch, Sajuran Ganeswarathas, Carsten Kallesøe, Simon Bøgh

Publikation: Bidrag til bog/antologi/rapport/konference proceedingKonferenceartikel i proceedingForskningpeer review

Resumé

This paper proposes an adaptive controller based on Reinforcement Learning (RL), which copes with HVAC-systems consisting of slow thermodynamics. Two different RL algorithms with Q-Networks (QNs) are investigated. The HVAC-system is in this study an underfloor heating system. Underfloor heating is of great interest because it is very common in Scandinavia, but this research can be applied to a wide range of HVAC-systems, industrial processes and other control applications that are dominated by very slow dynamics. The environments consist of one, two, and four zones within a house in a simulation environment meaning that agents will be exposed to gradually more complex environments separated into test levels. The novelty of this paper is the incorporation of two different RL algorithms for industrial process control; a QN and a QN + Eligibility Trace (QN+ET). The reason for using eligibility trace is that an underfloor heating environment is dominated by slow dynamics and by using eligibility trace the agent can find correlations between the reward and actions taken in earlier iterations.
OriginalsprogEngelsk
Titel29th International Conference on Flexible Automation and Intelligent Manufacturing : FAIM 2019
Antal sider8
Publikationsdato2019
StatusUdgivet - 2019

Emneord

  • Reinforcement Learning
  • Artificial intelligence
  • Sustainable Manufacturing Engineering
  • HVAC Systems

Citer dette

Blad, C., Koch, S., Ganeswarathas, S., Kallesøe, C., & Bøgh, S. (2019). Control of HVAC-systems with Slow Thermodynamic Using Reinforcement Learning. I 29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019
Blad, Christian ; Koch, Søren ; Ganeswarathas, Sajuran ; Kallesøe, Carsten ; Bøgh, Simon. / Control of HVAC-systems with Slow Thermodynamic Using Reinforcement Learning. 29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019. 2019.
@inproceedings{9b151558c7604570b4cf17a7b0792f8b,
title = "Control of HVAC-systems with Slow Thermodynamic Using Reinforcement Learning",
abstract = "This paper proposes an adaptive controller based on Reinforcement Learning (RL), which copes with HVAC-systems consisting of slow thermodynamics. Two different RL algorithms with Q-Networks (QNs) are investigated. The HVAC-system is in this study an underfloor heating system. Underfloor heating is of great interest because it is very common in Scandinavia, but this research can be applied to a wide range of HVAC-systems, industrial processes and other control applications that are dominated by very slow dynamics. The environments consist of one, two, and four zones within a house in a simulation environment meaning that agents will be exposed to gradually more complex environments separated into test levels. The novelty of this paper is the incorporation of two different RL algorithms for industrial process control; a QN and a QN + Eligibility Trace (QN+ET). The reason for using eligibility trace is that an underfloor heating environment is dominated by slow dynamics and by using eligibility trace the agent can find correlations between the reward and actions taken in earlier iterations.",
keywords = "Reinforcement Learning, Artificial intelligence, Sustainable Manufacturing Engineering, HVAC Systems, Reinforcement Learning, Artificial Intelligence (AI), Machine Learning, Deep Learning, HVAC Systems",
author = "Christian Blad and S{\o}ren Koch and Sajuran Ganeswarathas and Carsten Kalles{\o}e and Simon B{\o}gh",
year = "2019",
language = "English",
booktitle = "29th International Conference on Flexible Automation and Intelligent Manufacturing",

}

Blad, C, Koch, S, Ganeswarathas, S, Kallesøe, C & Bøgh, S 2019, Control of HVAC-systems with Slow Thermodynamic Using Reinforcement Learning. i 29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019.

Control of HVAC-systems with Slow Thermodynamic Using Reinforcement Learning. / Blad, Christian; Koch, Søren; Ganeswarathas, Sajuran; Kallesøe, Carsten; Bøgh, Simon.

29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019. 2019.

Publikation: Bidrag til bog/antologi/rapport/konference proceedingKonferenceartikel i proceedingForskningpeer review

TY - GEN

T1 - Control of HVAC-systems with Slow Thermodynamic Using Reinforcement Learning

AU - Blad, Christian

AU - Koch, Søren

AU - Ganeswarathas, Sajuran

AU - Kallesøe, Carsten

AU - Bøgh, Simon

PY - 2019

Y1 - 2019

N2 - This paper proposes an adaptive controller based on Reinforcement Learning (RL), which copes with HVAC-systems consisting of slow thermodynamics. Two different RL algorithms with Q-Networks (QNs) are investigated. The HVAC-system is in this study an underfloor heating system. Underfloor heating is of great interest because it is very common in Scandinavia, but this research can be applied to a wide range of HVAC-systems, industrial processes and other control applications that are dominated by very slow dynamics. The environments consist of one, two, and four zones within a house in a simulation environment meaning that agents will be exposed to gradually more complex environments separated into test levels. The novelty of this paper is the incorporation of two different RL algorithms for industrial process control; a QN and a QN + Eligibility Trace (QN+ET). The reason for using eligibility trace is that an underfloor heating environment is dominated by slow dynamics and by using eligibility trace the agent can find correlations between the reward and actions taken in earlier iterations.

AB - This paper proposes an adaptive controller based on Reinforcement Learning (RL), which copes with HVAC-systems consisting of slow thermodynamics. Two different RL algorithms with Q-Networks (QNs) are investigated. The HVAC-system is in this study an underfloor heating system. Underfloor heating is of great interest because it is very common in Scandinavia, but this research can be applied to a wide range of HVAC-systems, industrial processes and other control applications that are dominated by very slow dynamics. The environments consist of one, two, and four zones within a house in a simulation environment meaning that agents will be exposed to gradually more complex environments separated into test levels. The novelty of this paper is the incorporation of two different RL algorithms for industrial process control; a QN and a QN + Eligibility Trace (QN+ET). The reason for using eligibility trace is that an underfloor heating environment is dominated by slow dynamics and by using eligibility trace the agent can find correlations between the reward and actions taken in earlier iterations.

KW - Reinforcement Learning

KW - Artificial intelligence

KW - Sustainable Manufacturing Engineering

KW - HVAC Systems

KW - Reinforcement Learning

KW - Artificial Intelligence (AI)

KW - Machine Learning

KW - Deep Learning

KW - HVAC Systems

M3 - Article in proceeding

BT - 29th International Conference on Flexible Automation and Intelligent Manufacturing

ER -

Blad C, Koch S, Ganeswarathas S, Kallesøe C, Bøgh S. Control of HVAC-systems with Slow Thermodynamic Using Reinforcement Learning. I 29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019. 2019