Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data

Seshu Tirupathi, Dhaval Salwala, Giulio Zizzo, Ambrish Rawat, Mark Purcell, Søren Kejser Jensen, Christian Thomsen, Nguyen Ho, Carlos E. Muniz Cuza, Jonas Brusokas, Torben Bach Pedersen, Giorgos Alexiou, Giorgos Giannopoulos, Panagiotis Gidarakos, Alexandros Kalimeris, Stavros Maroulis, George Papastefanatos, Ioannis Psarros, Vassilis Stamatopoulos, Manolis Terrovitis

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review


With the lowering costs of sensors, high-volume and high-velocity data are increasingly being generated and analyzed, especially in IoT domains like energy and smart homes. Consequently, applications that require accurate short-term forecasts and predictions are also steadily increasing. In this paper, we provide an overview of a novel end-to-end platform that provides efficient ingestion, compression, transfer, query processing, and machine learning-based analytics for high-frequency and high-volume time series from IoT. The performance of the platform is evaluated using real-world dataset from RES installations. The results show the importance of high-frequency analytics and the surprisingly positive impact of error bounded lossy compression on machine learning in the form of AutoML. For example, when detecting yaw misalignments in wind turbines, an improvement of 9% in accuracy was observed for AutoML models on lossy compressed data compared to the current industry standard of 10-minute aggregated data. Thus, these small-scale experiments show the potential of the platform, and larger pilots are planned.
Original languageEnglish
Title of host publication2022 IEEE International Conference on Big Data (Big Data)
EditorsShusaku Tsumoto, Yukio Ohsawa, Lei Chen, Dirk Van den Poel, Xiaohua Hu, Yoichi Motomura, Takuya Takagi, Lingfei Wu, Ying Xie, Akihiro Abe, Vijay Raghavan
Number of pages7
PublisherIEEE Communications Society
Publication date20 Dec 2022
Article number10020540
ISBN (Print)978-1-6654-8046-8
ISBN (Electronic)9781665480451
Publication statusPublished - 20 Dec 2022
Event2022 IEEE International Conference on Big Data (Big Data) - Osaka, Japan
Duration: 17 Dec 202220 Dec 2022


Conference2022 IEEE International Conference on Big Data (Big Data)
LocationOsaka, Japan


  • Big Data
  • Data models
  • Industries
  • Machine learning
  • Query processing
  • Smart homes
  • Time series analysis
  • Lossy Data Compression
  • Machine Learning
  • Cloud
  • Lossless Data Compression
  • Edge
  • Renewable Energy Sources


Dive into the research topics of 'Machine Learning Platform for Extreme Scale Computing on Compressed IoT Data'. Together they form a unique fingerprint.

Cite this