pygrametl: A Powerful Programming Framework for Extract–Transform–Load Programmers

Publikation: Bog/antologi/afhandling/rapportRapportForskning

26 Citationer (Scopus)
5087 Downloads (Pure)

Resumé

    Extract-Transform-Load (ETL) processes are used for extracting data, transforming it and loading it into data
warehouses (DWs). Many tools for creating ETL processes exist. The dominating tools all use graphical user
interfaces (GUIs) where the developer visually defines the data flow and operations. In this paper, we challenge
this approach and propose to do ETL programming by writing code. To make the programming easy, we present
the (Python-based) framework pygrametl which offers commonly used functionality for ETL development.
By using the framework, the developer can efficiently create effective ETL solutions from which the full power
of programming can be exploited. Our experiments show that when pygrametl is used, both the development
time and running time are short when compared to an existing GUI-based tool
OriginalsprogEngelsk
Udgivelses stedAalborg
ForlagDepartment of Computer Science, Aalborg University
Vol/bind25
Antal sider18
StatusUdgivet - 2009
Navn1DB Technical Report
Nummer25

Fingerprint

Mathematical transformations
Experiments

Citer dette

Thomsen, C., & Pedersen, T. B. (2009). pygrametl: A Powerful Programming Framework for Extract–Transform–Load Programmers. Aalborg: Department of Computer Science, Aalborg University. 1DB Technical Report, Nr. 25
Thomsen, Christian ; Pedersen, Torben Bach. / pygrametl: A Powerful Programming Framework for Extract–Transform–Load Programmers. Aalborg : Department of Computer Science, Aalborg University, 2009. 18 s. (1DB Technical Report; Nr. 25).
@book{de9d76d0eb0c11deb63d000ea68e967b,
title = "pygrametl: A Powerful Programming Framework for Extract–Transform–Load Programmers",
abstract = "    Extract-Transform-Load (ETL) processes are used for extracting data, transforming it and loading it into datawarehouses (DWs). Many tools for creating ETL processes exist. The dominating tools all use graphical userinterfaces (GUIs) where the developer visually defines the data flow and operations. In this paper, we challengethis approach and propose to do ETL programming by writing code. To make the programming easy, we presentthe (Python-based) framework pygrametl which offers commonly used functionality for ETL development.By using the framework, the developer can efficiently create effective ETL solutions from which the full powerof programming can be exploited. Our experiments show that when pygrametl is used, both the developmenttime and running time are short when compared to an existing GUI-based tool",
author = "Christian Thomsen and Pedersen, {Torben Bach}",
year = "2009",
language = "English",
volume = "25",
publisher = "Department of Computer Science, Aalborg University",

}

Thomsen, C & Pedersen, TB 2009, pygrametl: A Powerful Programming Framework for Extract–Transform–Load Programmers. 1DB Technical Report, nr. 25, bind 25, Department of Computer Science, Aalborg University, Aalborg.

pygrametl: A Powerful Programming Framework for Extract–Transform–Load Programmers. / Thomsen, Christian; Pedersen, Torben Bach.

Aalborg : Department of Computer Science, Aalborg University, 2009. 18 s. (1DB Technical Report; Nr. 25).

Publikation: Bog/antologi/afhandling/rapportRapportForskning

TY - RPRT

T1 - pygrametl: A Powerful Programming Framework for Extract–Transform–Load Programmers

AU - Thomsen, Christian

AU - Pedersen, Torben Bach

PY - 2009

Y1 - 2009

N2 -     Extract-Transform-Load (ETL) processes are used for extracting data, transforming it and loading it into datawarehouses (DWs). Many tools for creating ETL processes exist. The dominating tools all use graphical userinterfaces (GUIs) where the developer visually defines the data flow and operations. In this paper, we challengethis approach and propose to do ETL programming by writing code. To make the programming easy, we presentthe (Python-based) framework pygrametl which offers commonly used functionality for ETL development.By using the framework, the developer can efficiently create effective ETL solutions from which the full powerof programming can be exploited. Our experiments show that when pygrametl is used, both the developmenttime and running time are short when compared to an existing GUI-based tool

AB -     Extract-Transform-Load (ETL) processes are used for extracting data, transforming it and loading it into datawarehouses (DWs). Many tools for creating ETL processes exist. The dominating tools all use graphical userinterfaces (GUIs) where the developer visually defines the data flow and operations. In this paper, we challengethis approach and propose to do ETL programming by writing code. To make the programming easy, we presentthe (Python-based) framework pygrametl which offers commonly used functionality for ETL development.By using the framework, the developer can efficiently create effective ETL solutions from which the full powerof programming can be exploited. Our experiments show that when pygrametl is used, both the developmenttime and running time are short when compared to an existing GUI-based tool

M3 - Report

VL - 25

BT - pygrametl: A Powerful Programming Framework for Extract–Transform–Load Programmers

PB - Department of Computer Science, Aalborg University

CY - Aalborg

ER -

Thomsen C, Pedersen TB. pygrametl: A Powerful Programming Framework for Extract–Transform–Load Programmers. Aalborg: Department of Computer Science, Aalborg University, 2009. 18 s. (1DB Technical Report; Nr. 25).