15 Citations (Scopus)

Abstract

Building performance simulations (BPS) are used to test different designs and systems with the intention of reducing building costs and energy demand while ensuring a comfortable indoor climate. Unfortunately, software for BPS is computationally intensive. This makes it impractical to run thousands of simulations for sensitivity analysis and optimization. Worse yet, millions of simulations may be necessary for a thorough exploration of the high-dimensional design space formed by the many design parameters. This computational issue may be overcome by the creation of fast metamodels. In this paper, we aim to find suitable metamodeling techniques for diverse outputs from BPS. We consider five indicators of building performance and eight test problems for the comparison six popular metamodeling techniques – linear regression with ordinary least squares (OLS), random forest (RF), support vector regression (SVR), multivariate adaptive regression splines, Gaussian process regression (GPR), and neural network (NN). The methods are compared with respect to accuracy, efficiency, ease-of-use, robustness, and interpretability. To conduct a fair and in-depth comparison, a methodological approach is pursued using exhaustive grid searches for model selection assisted by sensitivity analysis. The comparison shows that GPR produces the most accurate metamodels, followed by NN and MARS. GPR is robust and easy to implement but becomes inefficient for large training sets compared to NN and MARS. A coefficient of determination, R 2, larger than 0.9 have been obtained for the BPS outputs using between 128 and 1024 training points. In contrast, accurate metamodels with R 2 values larger than 0.99 can be achieved for all eight test problems using only 32–256 training points.

Original languageEnglish
JournalApplied Energy
Volume211
Pages (from-to)89-103
Number of pages15
ISSN0306-2619
DOIs
Publication statusPublished - 1 Feb 2018

Fingerprint

Neural networks
Sensitivity analysis
simulation
sensitivity analysis
Linear regression
Splines
comparison
software
Costs
climate
cost
test

Keywords

  • Gaussian process regression (kriging)
  • Random forest
  • Neural network
  • Support vector regression
  • Sensitivity analysis
  • Supervised learning

Cite this

@article{29a2fcf61b7b4573b64f6eb5c940c133,
title = "A comparison of six metamodeling techniques applied to building performance simulations",
abstract = "Building performance simulations (BPS) are used to test different designs and systems with the intention of reducing building costs and energy demand while ensuring a comfortable indoor climate. Unfortunately, software for BPS is computationally intensive. This makes it impractical to run thousands of simulations for sensitivity analysis and optimization. Worse yet, millions of simulations may be necessary for a thorough exploration of the high-dimensional design space formed by the many design parameters. This computational issue may be overcome by the creation of fast metamodels. In this paper, we aim to find suitable metamodeling techniques for diverse outputs from BPS. We consider five indicators of building performance and eight test problems for the comparison six popular metamodeling techniques – linear regression with ordinary least squares (OLS), random forest (RF), support vector regression (SVR), multivariate adaptive regression splines, Gaussian process regression (GPR), and neural network (NN). The methods are compared with respect to accuracy, efficiency, ease-of-use, robustness, and interpretability. To conduct a fair and in-depth comparison, a methodological approach is pursued using exhaustive grid searches for model selection assisted by sensitivity analysis. The comparison shows that GPR produces the most accurate metamodels, followed by NN and MARS. GPR is robust and easy to implement but becomes inefficient for large training sets compared to NN and MARS. A coefficient of determination, R 2, larger than 0.9 have been obtained for the BPS outputs using between 128 and 1024 training points. In contrast, accurate metamodels with R 2 values larger than 0.99 can be achieved for all eight test problems using only 32–256 training points.",
keywords = "Gaussian process regression (kriging), Random forest, Neural network, Support vector regression, Sensitivity analysis, Supervised learning, Gaussian process regression (kriging), Random forest, Neural network, Support vector regression, Sensitivity analysis, Supervised learning",
author = "Torben {\O}sterg{\aa}rd and Jensen, {Rasmus Lund} and Maagaard, {Steffen Enersen}",
year = "2018",
month = "2",
day = "1",
doi = "10.1016/j.apenergy.2017.10.102",
language = "English",
volume = "211",
pages = "89--103",
journal = "Applied Energy",
issn = "0306-2619",
publisher = "Pergamon Press",

}

A comparison of six metamodeling techniques applied to building performance simulations. / Østergård, Torben; Jensen, Rasmus Lund; Maagaard, Steffen Enersen.

In: Applied Energy, Vol. 211, 01.02.2018, p. 89-103.

Research output: Contribution to journalJournal articleResearchpeer-review

TY - JOUR

T1 - A comparison of six metamodeling techniques applied to building performance simulations

AU - Østergård, Torben

AU - Jensen, Rasmus Lund

AU - Maagaard, Steffen Enersen

PY - 2018/2/1

Y1 - 2018/2/1

N2 - Building performance simulations (BPS) are used to test different designs and systems with the intention of reducing building costs and energy demand while ensuring a comfortable indoor climate. Unfortunately, software for BPS is computationally intensive. This makes it impractical to run thousands of simulations for sensitivity analysis and optimization. Worse yet, millions of simulations may be necessary for a thorough exploration of the high-dimensional design space formed by the many design parameters. This computational issue may be overcome by the creation of fast metamodels. In this paper, we aim to find suitable metamodeling techniques for diverse outputs from BPS. We consider five indicators of building performance and eight test problems for the comparison six popular metamodeling techniques – linear regression with ordinary least squares (OLS), random forest (RF), support vector regression (SVR), multivariate adaptive regression splines, Gaussian process regression (GPR), and neural network (NN). The methods are compared with respect to accuracy, efficiency, ease-of-use, robustness, and interpretability. To conduct a fair and in-depth comparison, a methodological approach is pursued using exhaustive grid searches for model selection assisted by sensitivity analysis. The comparison shows that GPR produces the most accurate metamodels, followed by NN and MARS. GPR is robust and easy to implement but becomes inefficient for large training sets compared to NN and MARS. A coefficient of determination, R 2, larger than 0.9 have been obtained for the BPS outputs using between 128 and 1024 training points. In contrast, accurate metamodels with R 2 values larger than 0.99 can be achieved for all eight test problems using only 32–256 training points.

AB - Building performance simulations (BPS) are used to test different designs and systems with the intention of reducing building costs and energy demand while ensuring a comfortable indoor climate. Unfortunately, software for BPS is computationally intensive. This makes it impractical to run thousands of simulations for sensitivity analysis and optimization. Worse yet, millions of simulations may be necessary for a thorough exploration of the high-dimensional design space formed by the many design parameters. This computational issue may be overcome by the creation of fast metamodels. In this paper, we aim to find suitable metamodeling techniques for diverse outputs from BPS. We consider five indicators of building performance and eight test problems for the comparison six popular metamodeling techniques – linear regression with ordinary least squares (OLS), random forest (RF), support vector regression (SVR), multivariate adaptive regression splines, Gaussian process regression (GPR), and neural network (NN). The methods are compared with respect to accuracy, efficiency, ease-of-use, robustness, and interpretability. To conduct a fair and in-depth comparison, a methodological approach is pursued using exhaustive grid searches for model selection assisted by sensitivity analysis. The comparison shows that GPR produces the most accurate metamodels, followed by NN and MARS. GPR is robust and easy to implement but becomes inefficient for large training sets compared to NN and MARS. A coefficient of determination, R 2, larger than 0.9 have been obtained for the BPS outputs using between 128 and 1024 training points. In contrast, accurate metamodels with R 2 values larger than 0.99 can be achieved for all eight test problems using only 32–256 training points.

KW - Gaussian process regression (kriging)

KW - Random forest

KW - Neural network

KW - Support vector regression

KW - Sensitivity analysis

KW - Supervised learning

KW - Gaussian process regression (kriging)

KW - Random forest

KW - Neural network

KW - Support vector regression

KW - Sensitivity analysis

KW - Supervised learning

UR - http://www.scopus.com/inward/record.url?scp=85033502240&partnerID=8YFLogxK

U2 - 10.1016/j.apenergy.2017.10.102

DO - 10.1016/j.apenergy.2017.10.102

M3 - Journal article

VL - 211

SP - 89

EP - 103

JO - Applied Energy

JF - Applied Energy

SN - 0306-2619

ER -