Multi-schema-version data management: data independence in the twenty-first century

Kai Herrmann, Hannes Voigt, Torben Bach Pedersen, Wolfgang Lehner

Research output: Contribution to journalJournal articleResearchpeer-review

25 Downloads (Pure)

Abstract

Agile software development allows us to continuously evolve and run a software system. However, this is not possible in databases, as established methods are very expensive, error-prone, and far from agile. We present InVerDa, a multi-schema-version database management system (MSVDB) for agile database development. MSVDBs realize co-existing schema versions within one database, where each schema version behaves like a regular single-schema database and write operations are propagated between schema versions. Developers use a relationally complete and bidirectional database evolution language (BiDEL) to easily evolve existing schema versions to new ones. BiDEL scripts are more robust, orders of magnitude shorter, and cause only a small performance overhead compared to handwritten SQL scripts. We formally guarantee data independence: no matter how the data of the co-existing schema versions is physically materialized, each schema version is guaranteed to behave like a regular database. Since, the chosen physical materialization significantly determines the overall performance, we equip database administrators with an advisor that proposes an optimized materialization for the current workload, which can improve the performance by orders of magnitude compared to naïve solutions. To our best knowledge, we are the first to facilitate agile evolution of production databases with full support of co-existing schema versions and formally guaranteed data independence.
Original languageEnglish
JournalV L D B Journal
Volume27
Issue number4
Pages (from-to)547-571
Number of pages25
ISSN1066-8888
DOIs
Publication statusPublished - Aug 2018

Fingerprint

Information management
Software engineering

Keywords

  • Database
  • Evolution
  • Data independence

Cite this

Herrmann, Kai ; Voigt, Hannes ; Pedersen, Torben Bach ; Lehner, Wolfgang. / Multi-schema-version data management : data independence in the twenty-first century. In: V L D B Journal. 2018 ; Vol. 27, No. 4. pp. 547-571.
@article{98eb40dbf39142b8a0507d82d30fa21e,
title = "Multi-schema-version data management: data independence in the twenty-first century",
abstract = "Agile software development allows us to continuously evolve and run a software system. However, this is not possible in databases, as established methods are very expensive, error-prone, and far from agile. We present InVerDa, a multi-schema-version database management system (MSVDB) for agile database development. MSVDBs realize co-existing schema versions within one database, where each schema version behaves like a regular single-schema database and write operations are propagated between schema versions. Developers use a relationally complete and bidirectional database evolution language (BiDEL) to easily evolve existing schema versions to new ones. BiDEL scripts are more robust, orders of magnitude shorter, and cause only a small performance overhead compared to handwritten SQL scripts. We formally guarantee data independence: no matter how the data of the co-existing schema versions is physically materialized, each schema version is guaranteed to behave like a regular database. Since, the chosen physical materialization significantly determines the overall performance, we equip database administrators with an advisor that proposes an optimized materialization for the current workload, which can improve the performance by orders of magnitude compared to na{\"i}ve solutions. To our best knowledge, we are the first to facilitate agile evolution of production databases with full support of co-existing schema versions and formally guaranteed data independence.",
keywords = "Database, Evolution, Data independence",
author = "Kai Herrmann and Hannes Voigt and Pedersen, {Torben Bach} and Wolfgang Lehner",
year = "2018",
month = "8",
doi = "10.1007/s00778-018-0508-7",
language = "English",
volume = "27",
pages = "547--571",
journal = "V L D B Journal",
issn = "1066-8888",
publisher = "Springer",
number = "4",

}

Multi-schema-version data management : data independence in the twenty-first century. / Herrmann, Kai; Voigt, Hannes; Pedersen, Torben Bach; Lehner, Wolfgang.

In: V L D B Journal, Vol. 27, No. 4, 08.2018, p. 547-571.

Research output: Contribution to journalJournal articleResearchpeer-review

TY - JOUR

T1 - Multi-schema-version data management

T2 - data independence in the twenty-first century

AU - Herrmann, Kai

AU - Voigt, Hannes

AU - Pedersen, Torben Bach

AU - Lehner, Wolfgang

PY - 2018/8

Y1 - 2018/8

N2 - Agile software development allows us to continuously evolve and run a software system. However, this is not possible in databases, as established methods are very expensive, error-prone, and far from agile. We present InVerDa, a multi-schema-version database management system (MSVDB) for agile database development. MSVDBs realize co-existing schema versions within one database, where each schema version behaves like a regular single-schema database and write operations are propagated between schema versions. Developers use a relationally complete and bidirectional database evolution language (BiDEL) to easily evolve existing schema versions to new ones. BiDEL scripts are more robust, orders of magnitude shorter, and cause only a small performance overhead compared to handwritten SQL scripts. We formally guarantee data independence: no matter how the data of the co-existing schema versions is physically materialized, each schema version is guaranteed to behave like a regular database. Since, the chosen physical materialization significantly determines the overall performance, we equip database administrators with an advisor that proposes an optimized materialization for the current workload, which can improve the performance by orders of magnitude compared to naïve solutions. To our best knowledge, we are the first to facilitate agile evolution of production databases with full support of co-existing schema versions and formally guaranteed data independence.

AB - Agile software development allows us to continuously evolve and run a software system. However, this is not possible in databases, as established methods are very expensive, error-prone, and far from agile. We present InVerDa, a multi-schema-version database management system (MSVDB) for agile database development. MSVDBs realize co-existing schema versions within one database, where each schema version behaves like a regular single-schema database and write operations are propagated between schema versions. Developers use a relationally complete and bidirectional database evolution language (BiDEL) to easily evolve existing schema versions to new ones. BiDEL scripts are more robust, orders of magnitude shorter, and cause only a small performance overhead compared to handwritten SQL scripts. We formally guarantee data independence: no matter how the data of the co-existing schema versions is physically materialized, each schema version is guaranteed to behave like a regular database. Since, the chosen physical materialization significantly determines the overall performance, we equip database administrators with an advisor that proposes an optimized materialization for the current workload, which can improve the performance by orders of magnitude compared to naïve solutions. To our best knowledge, we are the first to facilitate agile evolution of production databases with full support of co-existing schema versions and formally guaranteed data independence.

KW - Database

KW - Evolution

KW - Data independence

U2 - 10.1007/s00778-018-0508-7

DO - 10.1007/s00778-018-0508-7

M3 - Journal article

VL - 27

SP - 547

EP - 571

JO - V L D B Journal

JF - V L D B Journal

SN - 1066-8888

IS - 4

ER -