A comparison of six metamodeling techniques applied to building performance simulations

Research output: Contribution to journalJournal articleResearchpeer-review

Abstract

Building performance simulations (BPS) are used to test different designs and systems with the intention of reducing building costs and energy demand while ensuring a comfortable indoor climate. Unfortunately, software for BPS is computationally intensive. This makes it impractical to run thousands of simulations for sensitivity analysis and optimization. Worse yet, millions of simulations may be necessary for a thorough exploration of the high-dimensional design space formed by the many design parameters. This computational issue may be overcome by the creation of fast metamodels. In this paper, we aim to find suitable metamodeling techniques for diverse outputs from BPS. We consider five indicators of building performance and eight test problems for the comparison six popular metamodeling techniques – linear regression with ordinary least squares (OLS), random forest (RF), support vector regression (SVR), multivariate adaptive regression splines, Gaussian process regression (GPR), and neural network (NN). The methods are compared with respect to accuracy, efficiency, ease-of-use, robustness, and interpretability. To conduct a fair and in-depth comparison, a methodological approach is pursued using exhaustive grid searches for model selection assisted by sensitivity analysis. The comparison shows that GPR produces the most accurate metamodels, followed by NN and MARS. GPR is robust and easy to implement but becomes inefficient for large training sets compared to NN and MARS. A coefficient of determination, R 2, larger than 0.9 have been obtained for the BPS outputs using between 128 and 1024 training points. In contrast, accurate metamodels with R 2 values larger than 0.99 can be achieved for all eight test problems using only 32–256 training points.

Close

Details

Building performance simulations (BPS) are used to test different designs and systems with the intention of reducing building costs and energy demand while ensuring a comfortable indoor climate. Unfortunately, software for BPS is computationally intensive. This makes it impractical to run thousands of simulations for sensitivity analysis and optimization. Worse yet, millions of simulations may be necessary for a thorough exploration of the high-dimensional design space formed by the many design parameters. This computational issue may be overcome by the creation of fast metamodels. In this paper, we aim to find suitable metamodeling techniques for diverse outputs from BPS. We consider five indicators of building performance and eight test problems for the comparison six popular metamodeling techniques – linear regression with ordinary least squares (OLS), random forest (RF), support vector regression (SVR), multivariate adaptive regression splines, Gaussian process regression (GPR), and neural network (NN). The methods are compared with respect to accuracy, efficiency, ease-of-use, robustness, and interpretability. To conduct a fair and in-depth comparison, a methodological approach is pursued using exhaustive grid searches for model selection assisted by sensitivity analysis. The comparison shows that GPR produces the most accurate metamodels, followed by NN and MARS. GPR is robust and easy to implement but becomes inefficient for large training sets compared to NN and MARS. A coefficient of determination, R 2, larger than 0.9 have been obtained for the BPS outputs using between 128 and 1024 training points. In contrast, accurate metamodels with R 2 values larger than 0.99 can be achieved for all eight test problems using only 32–256 training points.

Original languageEnglish
JournalApplied Energy
Volume211
Pages (from-to)89-103
Number of pages15
ISSN0306-2619
DOI
Publication statusPublished - 1 Feb 2018
Publication categoryResearch
Peer-reviewedYes

    Research areas

  • Gaussian process regression (kriging), Random forest, Neural network, Support vector regression, Sensitivity analysis, Supervised learning
ID: 265086489