OLAP over Probabilistic Data Cubes II: Parallel Materialization and Extended Aggregates

X. Xie, K. Zou, X. Hao, T. B. Pedersen, Peiquan Jin, W. Yang

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

8 Downloads (Pure)

Abstrakt

On-Line Analytical Processing (OLAP) enables powerful analytics by quickly computing aggregate values of numerical measures over multiple hierarchical dimensions for massive datasets. However, many types of source data, e.g., from GPS, sensors, and other measurement devices, are intrinsically inaccurate (imprecise and/or uncertain) and thus OLAP cannot be readily applied. In this paper, we address the resulting data veracity problem in OLAP by proposing the concept of probabilistic data cubes. Such a cube is comprised of a set of probabilistic cuboids which summarize the aggregated values in the form of probability mass functions (pmfs in short) and thus offer insights into the underlying data quality and enable confidence-aware query evaluation and analysis. However, the probabilistic nature of data poses computational challenges, since a probabilistic database can have exponential number of possible worlds under the possible world semantics. Even worse, it is hard to share computations among different cuboids, as aggregation functions that are distributive for traditional data cubes, e.g., \tt SUMSUM, become holistic in probabilistic settings. In this paper, we propose a complete set of techniques for probabilistic data cubes, from cuboid aggregation, over cube materialization, to query evaluation. We study two types of aggregation: convolution and sketch-based, which take polynomial time complexities for aggregation and jointly enable efficient query processing. Also, our proposal is versatile in terms of: 1) its capability of supporting common aggregation functions, i.e., \tt SUMSUM, \tt COUNTCOUNT, \tt MAXMAX, and \tt AVGAVG; 2) its adaptivity to different materialization strategies, e.g., full versus partial materialization, with support of our devised cost models and parallelization framework; 3) its coverage of common OLAP operations, i.e., probabilistic slicing and dicing queries. Extensive experiments over real and synthetic datasets show that our techniques are effective and scalable.

OriginalsprogEngelsk
Artikelnummer8700285
TidsskriftIEEE Transactions on Knowledge and Data Engineering
Vol/bind32
Udgave nummer10
Sider (fra-til)1966-1981
Antal sider16
ISSN1041-4347
DOI
StatusUdgivet - 1 okt. 2020

Emneord

  • Probabilistic logic
  • Aggregates
  • Sensors
  • Temperature measurement
  • Query processing
  • Convolution
  • Time measurement
  • Probabilistic Databases
  • OLAP
  • Data Warehousing

Fingeraftryk Dyk ned i forskningsemnerne om 'OLAP over Probabilistic Data Cubes II: Parallel Materialization and Extended Aggregates'. Sammen danner de et unikt fingeraftryk.

Citationsformater