Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data

Seshu Tirupathi; Dhaval Salwala; Giulio Zizzo; Ambrish Rawat; Mark Purcell; Søren Kejser Jensen; Christian Thomsen; Nguyen Ho; Carlos E. Muniz Cuza; Jonas Brusokas; Torben Bach Pedersen; Giorgos Alexiou; Giorgos Giannopoulos; Panagiotis Gidarakos; Alexandros Kalimeris; Stavros Maroulis; George Papastefanatos; Ioannis Psarros; Vassilis Stamatopoulos; Manolis Terrovitis

doi:10.1109/BigData55660.2022.10020540

Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data

Seshu Tirupathi, Dhaval Salwala, Giulio Zizzo, Ambrish Rawat, Mark Purcell, Søren Kejser Jensen, Christian Thomsen, Nguyen Ho, Carlos E. Muniz Cuza, Jonas Brusokas, Torben Bach Pedersen, Giorgos Alexiou, Giorgos Giannopoulos, Panagiotis Gidarakos, Alexandros Kalimeris, Stavros Maroulis, George Papastefanatos, Ioannis Psarros, Vassilis Stamatopoulos, Manolis Terrovitis

Research output: Contribution to book/anthology/report/conference proceeding › Article in proceeding › Research › peer-review

4 Citations (Scopus)

Abstract

With the lowering costs of sensors, high-volume and high-velocity data are increasingly being generated and analyzed, especially in IoT domains like energy and smart homes. Consequently, applications that require accurate short-term forecasts and predictions are also steadily increasing. In this paper, we provide an overview of a novel end-to-end platform that provides efficient ingestion, compression, transfer, query processing, and machine learning-based analytics for high-frequency and high-volume time series from IoT. The performance of the platform is evaluated using real-world dataset from RES installations. The results show the importance of high-frequency analytics and the surprisingly positive impact of error bounded lossy compression on machine learning in the form of AutoML. For example, when detecting yaw misalignments in wind turbines, an improvement of 9% in accuracy was observed for AutoML models on lossy compressed data compared to the current industry standard of 10-minute aggregated data. Thus, these small-scale experiments show the potential of the platform, and larger pilots are planned.

Original language	English
Title of host publication	2022 IEEE International Conference on Big Data (Big Data)
Editors	Shusaku Tsumoto, Yukio Ohsawa, Lei Chen, Dirk Van den Poel, Xiaohua Hu, Yoichi Motomura, Takuya Takagi, Lingfei Wu, Ying Xie, Akihiro Abe, Vijay Raghavan
Number of pages	7
Publisher	IEEE Communications Society
Publication date	20 Dec 2022
Pages	3179-3185
Article number	10020540
ISBN (Print)	978-1-6654-8046-8
ISBN (Electronic)	9781665480451
DOIs	https://doi.org/10.1109/BigData55660.2022.10020540
Publication status	Published - 20 Dec 2022
Event	2022 IEEE International Conference on Big Data (Big Data) - Osaka, Japan Duration: 17 Dec 2022 → 20 Dec 2022

Conference

Conference	2022 IEEE International Conference on Big Data (Big Data)
Location	Osaka, Japan
Period	17/12/2022 → 20/12/2022

Keywords

Big Data
Data models
Industries
Machine learning
Query processing
Smart homes
Time series analysis
Lossy Data Compression
Machine Learning
Cloud
Lossless Data Compression
Edge
Renewable Energy Sources

Access to Document

10.1109/BigData55660.2022.10020540

https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10020540

AUB Link

Search for the material in Aalborg University Library's search engine

Cite this

Tirupathi, S., Salwala, D., Zizzo, G., Rawat, A., Purcell, M., Jensen, S. K., Thomsen, C., Ho, N., Cuza, C. E. M., Brusokas, J., Pedersen, T. B., Alexiou, G., Giannopoulos, G., Gidarakos, P., Kalimeris, A., Maroulis, S., Papastefanatos, G., Psarros, I., Stamatopoulos, V., & Terrovitis, M. (2022). Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data. In S. Tsumoto, Y. Ohsawa, L. Chen, D. Van den Poel, X. Hu, Y. Motomura, T. Takagi, L. Wu, Y. Xie, A. Abe, & V. Raghavan (Eds.), 2022 IEEE International Conference on Big Data (Big Data) (pp. 3179-3185). Article 10020540 IEEE Communications Society. https://doi.org/10.1109/BigData55660.2022.10020540

Tirupathi, Seshu ; Salwala, Dhaval ; Zizzo, Giulio et al. / Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data. 2022 IEEE International Conference on Big Data (Big Data). editor / Shusaku Tsumoto ; Yukio Ohsawa ; Lei Chen ; Dirk Van den Poel ; Xiaohua Hu ; Yoichi Motomura ; Takuya Takagi ; Lingfei Wu ; Ying Xie ; Akihiro Abe ; Vijay Raghavan. IEEE Communications Society, 2022. pp. 3179-3185

@inproceedings{771ac3a11afc44d3b4424f64c86ec4e0,

title = "Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data",

abstract = "With the lowering costs of sensors, high-volume and high-velocity data are increasingly being generated and analyzed, especially in IoT domains like energy and smart homes. Consequently, applications that require accurate short-term forecasts and predictions are also steadily increasing. In this paper, we provide an overview of a novel end-to-end platform that provides efficient ingestion, compression, transfer, query processing, and machine learning-based analytics for high-frequency and high-volume time series from IoT. The performance of the platform is evaluated using real-world dataset from RES installations. The results show the importance of high-frequency analytics and the surprisingly positive impact of error bounded lossy compression on machine learning in the form of AutoML. For example, when detecting yaw misalignments in wind turbines, an improvement of 9% in accuracy was observed for AutoML models on lossy compressed data compared to the current industry standard of 10-minute aggregated data. Thus, these small-scale experiments show the potential of the platform, and larger pilots are planned.",

keywords = "Big Data, Data models, Industries, Machine learning, Query processing, Smart homes, Time series analysis, Lossy Data Compression, Machine Learning, Cloud, Lossless Data Compression, Edge, Renewable Energy Sources",

author = "Seshu Tirupathi and Dhaval Salwala and Giulio Zizzo and Ambrish Rawat and Mark Purcell and Jensen, {S{\o}ren Kejser} and Christian Thomsen and Nguyen Ho and Cuza, {Carlos E. Muniz} and Jonas Brusokas and Pedersen, {Torben Bach} and Giorgos Alexiou and Giorgos Giannopoulos and Panagiotis Gidarakos and Alexandros Kalimeris and Stavros Maroulis and George Papastefanatos and Ioannis Psarros and Vassilis Stamatopoulos and Manolis Terrovitis",

year = "2022",

month = dec,

day = "20",

doi = "10.1109/BigData55660.2022.10020540",

language = "English",

isbn = "978-1-6654-8046-8",

pages = "3179--3185",

editor = "Shusaku Tsumoto and Yukio Ohsawa and Lei Chen and {Van den Poel}, Dirk and Xiaohua Hu and Yoichi Motomura and Takuya Takagi and Lingfei Wu and Ying Xie and Akihiro Abe and Vijay Raghavan",

booktitle = "2022 IEEE International Conference on Big Data (Big Data)",

publisher = "IEEE Communications Society",

address = "United States",

note = "2022 IEEE International Conference on Big Data (Big Data) ; Conference date: 17-12-2022 Through 20-12-2022",

}

Tirupathi, S, Salwala, D, Zizzo, G, Rawat, A, Purcell, M, Jensen, SK , Thomsen, C , Ho, N , Cuza, CEM , Brusokas, J , Pedersen, TB, Alexiou, G, Giannopoulos, G, Gidarakos, P, Kalimeris, A, Maroulis, S, Papastefanatos, G, Psarros, I, Stamatopoulos, V & Terrovitis, M 2022, Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data. in S Tsumoto, Y Ohsawa, L Chen, D Van den Poel, X Hu, Y Motomura, T Takagi, L Wu, Y Xie, A Abe & V Raghavan (eds), 2022 IEEE International Conference on Big Data (Big Data)., 10020540, IEEE Communications Society, pp. 3179-3185, 2022 IEEE International Conference on Big Data (Big Data), 17/12/2022. https://doi.org/10.1109/BigData55660.2022.10020540

Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data. / Tirupathi, Seshu; Salwala, Dhaval; Zizzo, Giulio et al.
2022 IEEE International Conference on Big Data (Big Data). ed. / Shusaku Tsumoto; Yukio Ohsawa; Lei Chen; Dirk Van den Poel; Xiaohua Hu; Yoichi Motomura; Takuya Takagi; Lingfei Wu; Ying Xie; Akihiro Abe; Vijay Raghavan. IEEE Communications Society, 2022. p. 3179-3185 10020540.

Research output: Contribution to book/anthology/report/conference proceeding › Article in proceeding › Research › peer-review

TY - GEN

T1 - Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data

AU - Tirupathi, Seshu

AU - Salwala, Dhaval

AU - Zizzo, Giulio

AU - Rawat, Ambrish

AU - Purcell, Mark

AU - Jensen, Søren Kejser

AU - Thomsen, Christian

AU - Ho, Nguyen

AU - Cuza, Carlos E. Muniz

AU - Brusokas, Jonas

AU - Pedersen, Torben Bach

AU - Alexiou, Giorgos

AU - Giannopoulos, Giorgos

AU - Gidarakos, Panagiotis

AU - Kalimeris, Alexandros

AU - Maroulis, Stavros

AU - Papastefanatos, George

AU - Psarros, Ioannis

AU - Stamatopoulos, Vassilis

AU - Terrovitis, Manolis

PY - 2022/12/20

Y1 - 2022/12/20

N2 - With the lowering costs of sensors, high-volume and high-velocity data are increasingly being generated and analyzed, especially in IoT domains like energy and smart homes. Consequently, applications that require accurate short-term forecasts and predictions are also steadily increasing. In this paper, we provide an overview of a novel end-to-end platform that provides efficient ingestion, compression, transfer, query processing, and machine learning-based analytics for high-frequency and high-volume time series from IoT. The performance of the platform is evaluated using real-world dataset from RES installations. The results show the importance of high-frequency analytics and the surprisingly positive impact of error bounded lossy compression on machine learning in the form of AutoML. For example, when detecting yaw misalignments in wind turbines, an improvement of 9% in accuracy was observed for AutoML models on lossy compressed data compared to the current industry standard of 10-minute aggregated data. Thus, these small-scale experiments show the potential of the platform, and larger pilots are planned.

AB - With the lowering costs of sensors, high-volume and high-velocity data are increasingly being generated and analyzed, especially in IoT domains like energy and smart homes. Consequently, applications that require accurate short-term forecasts and predictions are also steadily increasing. In this paper, we provide an overview of a novel end-to-end platform that provides efficient ingestion, compression, transfer, query processing, and machine learning-based analytics for high-frequency and high-volume time series from IoT. The performance of the platform is evaluated using real-world dataset from RES installations. The results show the importance of high-frequency analytics and the surprisingly positive impact of error bounded lossy compression on machine learning in the form of AutoML. For example, when detecting yaw misalignments in wind turbines, an improvement of 9% in accuracy was observed for AutoML models on lossy compressed data compared to the current industry standard of 10-minute aggregated data. Thus, these small-scale experiments show the potential of the platform, and larger pilots are planned.

KW - Big Data

KW - Data models

KW - Industries

KW - Machine learning

KW - Query processing

KW - Smart homes

KW - Time series analysis

KW - Lossy Data Compression

KW - Machine Learning

KW - Cloud

KW - Lossless Data Compression

KW - Edge

KW - Renewable Energy Sources

UR - http://www.scopus.com/inward/record.url?scp=85147976313&partnerID=8YFLogxK

U2 - 10.1109/BigData55660.2022.10020540

DO - 10.1109/BigData55660.2022.10020540

M3 - Article in proceeding

SN - 978-1-6654-8046-8

SP - 3179

EP - 3185

BT - 2022 IEEE International Conference on Big Data (Big Data)

A2 - Tsumoto, Shusaku

A2 - Ohsawa, Yukio

A2 - Chen, Lei

A2 - Van den Poel, Dirk

A2 - Hu, Xiaohua

A2 - Motomura, Yoichi

A2 - Takagi, Takuya

A2 - Wu, Lingfei

A2 - Xie, Ying

A2 - Abe, Akihiro

A2 - Raghavan, Vijay

PB - IEEE Communications Society

T2 - 2022 IEEE International Conference on Big Data (Big Data)

Y2 - 17 December 2022 through 20 December 2022

ER -

Tirupathi S, Salwala D, Zizzo G, Rawat A, Purcell M, Jensen SK et al. Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data. In Tsumoto S, Ohsawa Y, Chen L, Van den Poel D, Hu X, Motomura Y, Takagi T, Wu L, Xie Y, Abe A, Raghavan V, editors, 2022 IEEE International Conference on Big Data (Big Data). IEEE Communications Society. 2022. p. 3179-3185. 10020540 doi: 10.1109/BigData55660.2022.10020540

Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data

Abstract

Conference

Keywords

Access to Document

AUB Link

Other files and links

Fingerprint

ModelarDB: Integrated Model-Based Management of Time Series from Edge to Cloud

Extreme-Scale Model-Based Time Series Management with ModelarDB (Invited Talk)

Scalable Model-Based Management of Correlated Dimensional Time Series in ModelarDB+

Cite this