Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data

Seshu Tirupathi; Dhaval Salwala; Giulio Zizzo; Ambrish Rawat; Mark Purcell; Søren Kejser Jensen; Christian Thomsen; Nguyen Ho; Carlos E. Muniz Cuza; Jonas Brusokas; Torben Bach Pedersen; Giorgos Alexiou; Giorgos Giannopoulos; Panagiotis Gidarakos; Alexandros Kalimeris; Stavros Maroulis; George Papastefanatos; Ioannis Psarros; Vassilis Stamatopoulos; Manolis Terrovitis

doi:10.1109/BigData55660.2022.10020540

Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data

Seshu Tirupathi, Dhaval Salwala, Giulio Zizzo, Ambrish Rawat, Mark Purcell, Søren Kejser Jensen, Christian Thomsen, Nguyen Ho, Carlos E. Muniz Cuza, Jonas Brusokas, Torben Bach Pedersen, Giorgos Alexiou, Giorgos Giannopoulos, Panagiotis Gidarakos, Alexandros Kalimeris, Stavros Maroulis, George Papastefanatos, Ioannis Psarros, Vassilis Stamatopoulos, Manolis Terrovitis

Publikation: Bidrag til bog/antologi/rapport/konference proceeding › Konferenceartikel i proceeding › Forskning › peer review

1 Citationer (Scopus)

Abstract

With the lowering costs of sensors, high-volume and high-velocity data are increasingly being generated and analyzed, especially in IoT domains like energy and smart homes. Consequently, applications that require accurate short-term forecasts and predictions are also steadily increasing. In this paper, we provide an overview of a novel end-to-end platform that provides efficient ingestion, compression, transfer, query processing, and machine learning-based analytics for high-frequency and high-volume time series from IoT. The performance of the platform is evaluated using real-world dataset from RES installations. The results show the importance of high-frequency analytics and the surprisingly positive impact of error bounded lossy compression on machine learning in the form of AutoML. For example, when detecting yaw misalignments in wind turbines, an improvement of 9% in accuracy was observed for AutoML models on lossy compressed data compared to the current industry standard of 10-minute aggregated data. Thus, these small-scale experiments show the potential of the platform, and larger pilots are planned.

Originalsprog	Engelsk
Titel	2022 IEEE International Conference on Big Data (Big Data)
Redaktører	Shusaku Tsumoto, Yukio Ohsawa, Lei Chen, Dirk Van den Poel, Xiaohua Hu, Yoichi Motomura, Takuya Takagi, Lingfei Wu, Ying Xie, Akihiro Abe, Vijay Raghavan
Antal sider	7
Forlag	IEEE Communications Society
Publikationsdato	20 dec. 2022
Sider	3179-3185
Artikelnummer	10020540
ISBN (Trykt)	978-1-6654-8046-8
ISBN (Elektronisk)	9781665480451
DOI	https://doi.org/10.1109/BigData55660.2022.10020540
Status	Udgivet - 20 dec. 2022
Begivenhed	2022 IEEE International Conference on Big Data (Big Data) - Osaka, Japan Varighed: 17 dec. 2022 → 20 dec. 2022

Konference

Konference	2022 IEEE International Conference on Big Data (Big Data)
Lokation	Osaka, Japan
Periode	17/12/2022 → 20/12/2022

Adgang til dokumentet

10.1109/BigData55660.2022.10020540

https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10020540

AUB Link

Søg efter materialet i Aalborg Universitetsbiblioteks søgemaskine

Andre filer og links

Link to publication in Scopus

1 Citationer
3 Konferenceartikel i proceeding
1 Bidrag til bog/antologi
1 Tidsskriftartikel
1 Ph.d.-afhandling

ModelarDB: Integrated Model-Based Management of Time Series from Edge to Cloud
Jensen, S. K., Thomsen, C. & Pedersen, T. B., 9 feb. 2023, Transactions on Large-Scale Data- and Knowledge-Centered Systems LIII. Springer, s. 1-33 33 s. (Transactions on Large-Scale Data- and Knowledge-Centered Systems). (Lecture Notes in Computer Science, Bind 13840).
Publikation: Bidrag til bog/antologi/rapport/konference proceeding › Bidrag til bog/antologi › Forskning › peer review

Åben adgang
Fil
5 Downloads (Pure)
Extreme-Scale Model-Based Time Series Management with ModelarDB (Invited Talk)
Pedersen, T. B., sep. 2021, 28th International Symposium on Temporal Representation and Reasoning, TIME 2021, September 27-29, 2021, Klagenfurt, Austria.. Schloss Dagstuhl. Leibniz-Zentrum für Informatik, s. 2:1-2:2 (Leibniz International Proceedings in Informatics, Bind 206).
Publikation: Bidrag til bog/antologi/rapport/konference proceeding › Konferenceartikel i proceeding › Forskning › peer review

Åben adgang
Fil
8 Downloads (Pure)
Scalable Model-Based Management of Correlated Dimensional Time Series in ModelarDB+
Jensen, S. K., Pedersen, T. B. & Thomsen, C., 22 apr. 2021, Proceedings of the 37th IEEE International Conference on Data Engineering. IEEE, s. 1380-1391 12 s. 9458830. (Proceedings of the International Conference on Data Engineering).
Publikation: Bidrag til bog/antologi/rapport/konference proceeding › Konferenceartikel i proceeding › Forskning › peer review

Åben adgang
5 Citationer (Scopus)

Citationsformater

Tirupathi, S., Salwala, D., Zizzo, G., Rawat, A., Purcell, M., Jensen, S. K., Thomsen, C., Ho, N., Cuza, C. E. M., Brusokas, J., Pedersen, T. B., Alexiou, G., Giannopoulos, G., Gidarakos, P., Kalimeris, A., Maroulis, S., Papastefanatos, G., Psarros, I., Stamatopoulos, V., & Terrovitis, M. (2022). Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data. I S. Tsumoto, Y. Ohsawa, L. Chen, D. Van den Poel, X. Hu, Y. Motomura, T. Takagi, L. Wu, Y. Xie, A. Abe, & V. Raghavan (red.), 2022 IEEE International Conference on Big Data (Big Data) (s. 3179-3185). Artikel 10020540 IEEE Communications Society. https://doi.org/10.1109/BigData55660.2022.10020540

Tirupathi, Seshu ; Salwala, Dhaval ; Zizzo, Giulio et al. / Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data. 2022 IEEE International Conference on Big Data (Big Data). red. / Shusaku Tsumoto ; Yukio Ohsawa ; Lei Chen ; Dirk Van den Poel ; Xiaohua Hu ; Yoichi Motomura ; Takuya Takagi ; Lingfei Wu ; Ying Xie ; Akihiro Abe ; Vijay Raghavan. IEEE Communications Society, 2022. s. 3179-3185

@inproceedings{771ac3a11afc44d3b4424f64c86ec4e0,

title = "Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data",

abstract = "With the lowering costs of sensors, high-volume and high-velocity data are increasingly being generated and analyzed, especially in IoT domains like energy and smart homes. Consequently, applications that require accurate short-term forecasts and predictions are also steadily increasing. In this paper, we provide an overview of a novel end-to-end platform that provides efficient ingestion, compression, transfer, query processing, and machine learning-based analytics for high-frequency and high-volume time series from IoT. The performance of the platform is evaluated using real-world dataset from RES installations. The results show the importance of high-frequency analytics and the surprisingly positive impact of error bounded lossy compression on machine learning in the form of AutoML. For example, when detecting yaw misalignments in wind turbines, an improvement of 9% in accuracy was observed for AutoML models on lossy compressed data compared to the current industry standard of 10-minute aggregated data. Thus, these small-scale experiments show the potential of the platform, and larger pilots are planned.",

keywords = "Big Data, Data models, Industries, Machine learning, Query processing, Smart homes, Time series analysis, Lossy Data Compression, Machine Learning, Cloud, Lossless Data Compression, Edge, Renewable Energy Sources",

author = "Seshu Tirupathi and Dhaval Salwala and Giulio Zizzo and Ambrish Rawat and Mark Purcell and Jensen, {S{\o}ren Kejser} and Christian Thomsen and Nguyen Ho and Cuza, {Carlos E. Muniz} and Jonas Brusokas and Pedersen, {Torben Bach} and Giorgos Alexiou and Giorgos Giannopoulos and Panagiotis Gidarakos and Alexandros Kalimeris and Stavros Maroulis and George Papastefanatos and Ioannis Psarros and Vassilis Stamatopoulos and Manolis Terrovitis",

year = "2022",

month = dec,

day = "20",

doi = "10.1109/BigData55660.2022.10020540",

language = "English",

isbn = "978-1-6654-8046-8",

pages = "3179--3185",

editor = "Shusaku Tsumoto and Yukio Ohsawa and Lei Chen and {Van den Poel}, Dirk and Xiaohua Hu and Yoichi Motomura and Takuya Takagi and Lingfei Wu and Ying Xie and Akihiro Abe and Vijay Raghavan",

booktitle = "2022 IEEE International Conference on Big Data (Big Data)",

publisher = "IEEE Communications Society",

address = "United States",

note = "2022 IEEE International Conference on Big Data (Big Data) ; Conference date: 17-12-2022 Through 20-12-2022",

}

Tirupathi, S, Salwala, D, Zizzo, G, Rawat, A, Purcell, M, Jensen, SK , Thomsen, C , Ho, N , Cuza, CEM , Brusokas, J , Pedersen, TB, Alexiou, G, Giannopoulos, G, Gidarakos, P, Kalimeris, A, Maroulis, S, Papastefanatos, G, Psarros, I, Stamatopoulos, V & Terrovitis, M 2022, Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data. i S Tsumoto, Y Ohsawa, L Chen, D Van den Poel, X Hu, Y Motomura, T Takagi, L Wu, Y Xie, A Abe & V Raghavan (red), 2022 IEEE International Conference on Big Data (Big Data)., 10020540, IEEE Communications Society, s. 3179-3185, 2022 IEEE International Conference on Big Data (Big Data), 17/12/2022. https://doi.org/10.1109/BigData55660.2022.10020540

Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data. / Tirupathi, Seshu; Salwala, Dhaval; Zizzo, Giulio et al.
2022 IEEE International Conference on Big Data (Big Data). red. / Shusaku Tsumoto; Yukio Ohsawa; Lei Chen; Dirk Van den Poel; Xiaohua Hu; Yoichi Motomura; Takuya Takagi; Lingfei Wu; Ying Xie; Akihiro Abe; Vijay Raghavan. IEEE Communications Society, 2022. s. 3179-3185 10020540.

Publikation: Bidrag til bog/antologi/rapport/konference proceeding › Konferenceartikel i proceeding › Forskning › peer review

TY - GEN

T1 - Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data

AU - Tirupathi, Seshu

AU - Salwala, Dhaval

AU - Zizzo, Giulio

AU - Rawat, Ambrish

AU - Purcell, Mark

AU - Jensen, Søren Kejser

AU - Thomsen, Christian

AU - Ho, Nguyen

AU - Cuza, Carlos E. Muniz

AU - Brusokas, Jonas

AU - Pedersen, Torben Bach

AU - Alexiou, Giorgos

AU - Giannopoulos, Giorgos

AU - Gidarakos, Panagiotis

AU - Kalimeris, Alexandros

AU - Maroulis, Stavros

AU - Papastefanatos, George

AU - Psarros, Ioannis

AU - Stamatopoulos, Vassilis

AU - Terrovitis, Manolis

PY - 2022/12/20

Y1 - 2022/12/20

N2 - With the lowering costs of sensors, high-volume and high-velocity data are increasingly being generated and analyzed, especially in IoT domains like energy and smart homes. Consequently, applications that require accurate short-term forecasts and predictions are also steadily increasing. In this paper, we provide an overview of a novel end-to-end platform that provides efficient ingestion, compression, transfer, query processing, and machine learning-based analytics for high-frequency and high-volume time series from IoT. The performance of the platform is evaluated using real-world dataset from RES installations. The results show the importance of high-frequency analytics and the surprisingly positive impact of error bounded lossy compression on machine learning in the form of AutoML. For example, when detecting yaw misalignments in wind turbines, an improvement of 9% in accuracy was observed for AutoML models on lossy compressed data compared to the current industry standard of 10-minute aggregated data. Thus, these small-scale experiments show the potential of the platform, and larger pilots are planned.

AB - With the lowering costs of sensors, high-volume and high-velocity data are increasingly being generated and analyzed, especially in IoT domains like energy and smart homes. Consequently, applications that require accurate short-term forecasts and predictions are also steadily increasing. In this paper, we provide an overview of a novel end-to-end platform that provides efficient ingestion, compression, transfer, query processing, and machine learning-based analytics for high-frequency and high-volume time series from IoT. The performance of the platform is evaluated using real-world dataset from RES installations. The results show the importance of high-frequency analytics and the surprisingly positive impact of error bounded lossy compression on machine learning in the form of AutoML. For example, when detecting yaw misalignments in wind turbines, an improvement of 9% in accuracy was observed for AutoML models on lossy compressed data compared to the current industry standard of 10-minute aggregated data. Thus, these small-scale experiments show the potential of the platform, and larger pilots are planned.

KW - Big Data

KW - Data models

KW - Industries

KW - Machine learning

KW - Query processing

KW - Smart homes

KW - Time series analysis

KW - Lossy Data Compression

KW - Machine Learning

KW - Cloud

KW - Lossless Data Compression

KW - Edge

KW - Renewable Energy Sources

UR - http://www.scopus.com/inward/record.url?scp=85147976313&partnerID=8YFLogxK

U2 - 10.1109/BigData55660.2022.10020540

DO - 10.1109/BigData55660.2022.10020540

M3 - Article in proceeding

SN - 978-1-6654-8046-8

SP - 3179

EP - 3185

BT - 2022 IEEE International Conference on Big Data (Big Data)

A2 - Tsumoto, Shusaku

A2 - Ohsawa, Yukio

A2 - Chen, Lei

A2 - Van den Poel, Dirk

A2 - Hu, Xiaohua

A2 - Motomura, Yoichi

A2 - Takagi, Takuya

A2 - Wu, Lingfei

A2 - Xie, Ying

A2 - Abe, Akihiro

A2 - Raghavan, Vijay

PB - IEEE Communications Society

T2 - 2022 IEEE International Conference on Big Data (Big Data)

Y2 - 17 December 2022 through 20 December 2022

ER -

Tirupathi S, Salwala D, Zizzo G, Rawat A, Purcell M, Jensen SK et al. Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data. I Tsumoto S, Ohsawa Y, Chen L, Van den Poel D, Hu X, Motomura Y, Takagi T, Wu L, Xie Y, Abe A, Raghavan V, red., 2022 IEEE International Conference on Big Data (Big Data). IEEE Communications Society. 2022. s. 3179-3185. 10020540 doi: 10.1109/BigData55660.2022.10020540

Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data

Abstract

Konference

Adgang til dokumentet

AUB Link

Andre filer og links

Fingeraftryk

Publikation

ModelarDB: Integrated Model-Based Management of Time Series from Edge to Cloud

Extreme-Scale Model-Based Time Series Management with ModelarDB (Invited Talk)

Scalable Model-Based Management of Correlated Dimensional Time Series in ModelarDB+

Citationsformater