Research output per year
Research output per year
Abstract
Extract–Transform–Load (ETL) programs are used to load data
into data warehouses (DWs). An ETL program must extract data
from sources, apply different transformations to it, and use the DW
to look up/insert the data. It is both time consuming to develop and
to run an ETL program. It is, however, typically the case that the
ETL program can exploit both task parallelism and data parallelism
to run faster. This, on the other hand, makes the development time
longer as it is complex to create a parallel ETL program. To remedy
this situation, we propose efficient ways to parallelize typical ETL
tasks and we implement these new constructs in an ETL framework.
The constructs are easy to apply and do only require few
modifications to an ETL program to parallelize it. They support
both task and data parallelism and give the programmer different
possibilities to choose from. An experimental evaluation shows
that by using a little more CPU time, the (wall-clock) time to run
an ETL program can be greatly reduced.
into data warehouses (DWs). An ETL program must extract data
from sources, apply different transformations to it, and use the DW
to look up/insert the data. It is both time consuming to develop and
to run an ETL program. It is, however, typically the case that the
ETL program can exploit both task parallelism and data parallelism
to run faster. This, on the other hand, makes the development time
longer as it is complex to create a parallel ETL program. To remedy
this situation, we propose efficient ways to parallelize typical ETL
tasks and we implement these new constructs in an ETL framework.
The constructs are easy to apply and do only require few
modifications to an ETL program to parallelize it. They support
both task and data parallelism and give the programmer different
possibilities to choose from. An experimental evaluation shows
that by using a little more CPU time, the (wall-clock) time to run
an ETL program can be greatly reduced.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the ACM 14th International Workshop on Data warehousing and OLAP |
| Number of pages | 8 |
| Place of Publication | New York, NY, USA |
| Publisher | Association for Computing Machinery |
| Publication date | 2011 |
| Pages | 37-44 |
| ISBN (Electronic) | 978-1-4503-0963-9 |
| DOIs | |
| Publication status | Published - 2011 |
| Event | ACM 14th International Workshop on Data Warehousing and OLAP - Glasgow, United Kingdom Duration: 28 Oct 2011 → … Conference number: 14 |
Conference
| Conference | ACM 14th International Workshop on Data Warehousing and OLAP |
|---|---|
| Number | 14 |
| Country/Territory | United Kingdom |
| City | Glasgow |
| Period | 28/10/2011 → … |
Fingerprint
Dive into the research topics of 'Easy and Effective Parallel Programmable ETL'. Together they form a unique fingerprint.Research output
- 15 Citations
- 1 Book chapter
-
pygrametl: A Powerful Programming Framework for Easy Creation and Testing of ETL Flows
Jensen, S. K., Thomsen, C., Pedersen, T. B. & Andersen, O., 18 May 2021, Transactions on Large-Scale Data- and Knowledge-Centered Systems XLVIII: Special Issue In Memory of Univ. Prof. Dr. Roland Wagner. Springer, Vol. XLVIII. p. 45-84 40 p. (Lecture Notes in Computer Science, Vol. 12670).Research output: Contribution to book/anthology/report/conference proceeding › Book chapter › Research › peer-review
Open AccessFile2 Citations (Scopus)364 Downloads (Pure)