Adaptive Network Coded Clouds

High Speed Downloads and Cost-Effective Version Control

Marton A. Sipos, Janus Heide, Daniel Enrique Lucani Roetter, Morten Videbæk Pedersen, Frank Hanns Paul Fitzek, Hassan Charaf

Research output: Contribution to journalJournal articleResearchpeer-review

1 Citation (Scopus)

Abstract

Although cloud systems provide a reliable and flexible storage solution, the use of a single cloud service constitutes a single point of failure, which can compromise data availability, download speed, and security. To address these challenges, we advocate for the use of multiple cloud storage providers simultaneously using network coding as the key enabling technology. Our goal is to study two challenges of network coded storage systems. First, the efficient update of the number of coded fragments per cloud in a system aggregating multiple clouds in order to boost the download speed of files. We developed a novel scheme using recoding with limited packets to trade-off storage space, reliability, and data retrieval speed. Implementation and measurements with commercial cloud providers show that up to 9x less network use is needed compared to other network coding schemes, while maintaining similar download speeds and reliability. Second, the ability to update coded fragments from a linear erasure code when the original file is modified. We exploit code structure to provide efficient representations of the evolution of the file. Evaluations using file changes on software library repositories show that a five-order of magnitude reduction in network and storage use is possible compared to
state-of-the-art.
Original languageEnglish
Article number7274724
JournalIEEE Transactions on Cloud Computing
Volume7
Issue number1
Pages (from-to)19-33
Number of pages15
DOIs
Publication statusPublished - Mar 2019

Fingerprint

Network coding
Costs
Availability

Keywords

  • Distributed systems
  • error control codes
  • information storage and retrieval

Cite this

Sipos, M. A., Heide, J., Roetter, D. E. L., Pedersen, M. V., Fitzek, F. H. P., & Charaf, H. (2019). Adaptive Network Coded Clouds: High Speed Downloads and Cost-Effective Version Control. IEEE Transactions on Cloud Computing, 7(1), 19-33. [7274724]. https://doi.org/10.1109/TCC.2015.2481433
Sipos, Marton A. ; Heide, Janus ; Roetter, Daniel Enrique Lucani ; Pedersen, Morten Videbæk ; Fitzek, Frank Hanns Paul ; Charaf, Hassan. / Adaptive Network Coded Clouds : High Speed Downloads and Cost-Effective Version Control. In: IEEE Transactions on Cloud Computing. 2019 ; Vol. 7, No. 1. pp. 19-33.
@article{d20ce9cb1fe6455f8400e77c2a3516cc,
title = "Adaptive Network Coded Clouds: High Speed Downloads and Cost-Effective Version Control",
abstract = "Although cloud systems provide a reliable and flexible storage solution, the use of a single cloud service constitutes a single point of failure, which can compromise data availability, download speed, and security. To address these challenges, we advocate for the use of multiple cloud storage providers simultaneously using network coding as the key enabling technology. Our goal is to study two challenges of network coded storage systems. First, the efficient update of the number of coded fragments per cloud in a system aggregating multiple clouds in order to boost the download speed of files. We developed a novel scheme using recoding with limited packets to trade-off storage space, reliability, and data retrieval speed. Implementation and measurements with commercial cloud providers show that up to 9x less network use is needed compared to other network coding schemes, while maintaining similar download speeds and reliability. Second, the ability to update coded fragments from a linear erasure code when the original file is modified. We exploit code structure to provide efficient representations of the evolution of the file. Evaluations using file changes on software library repositories show that a five-order of magnitude reduction in network and storage use is possible compared tostate-of-the-art.",
keywords = "Distributed systems, error control codes, information storage and retrieval",
author = "Sipos, {Marton A.} and Janus Heide and Roetter, {Daniel Enrique Lucani} and Pedersen, {Morten Videb{\ae}k} and Fitzek, {Frank Hanns Paul} and Hassan Charaf",
year = "2019",
month = "3",
doi = "10.1109/TCC.2015.2481433",
language = "English",
volume = "7",
pages = "19--33",
journal = "IEEE Transactions on Cloud Computing",
publisher = "IEEE",
number = "1",

}

Sipos, MA, Heide, J, Roetter, DEL, Pedersen, MV, Fitzek, FHP & Charaf, H 2019, 'Adaptive Network Coded Clouds: High Speed Downloads and Cost-Effective Version Control', IEEE Transactions on Cloud Computing, vol. 7, no. 1, 7274724, pp. 19-33. https://doi.org/10.1109/TCC.2015.2481433

Adaptive Network Coded Clouds : High Speed Downloads and Cost-Effective Version Control. / Sipos, Marton A.; Heide, Janus; Roetter, Daniel Enrique Lucani; Pedersen, Morten Videbæk; Fitzek, Frank Hanns Paul; Charaf, Hassan.

In: IEEE Transactions on Cloud Computing, Vol. 7, No. 1, 7274724, 03.2019, p. 19-33.

Research output: Contribution to journalJournal articleResearchpeer-review

TY - JOUR

T1 - Adaptive Network Coded Clouds

T2 - High Speed Downloads and Cost-Effective Version Control

AU - Sipos, Marton A.

AU - Heide, Janus

AU - Roetter, Daniel Enrique Lucani

AU - Pedersen, Morten Videbæk

AU - Fitzek, Frank Hanns Paul

AU - Charaf, Hassan

PY - 2019/3

Y1 - 2019/3

N2 - Although cloud systems provide a reliable and flexible storage solution, the use of a single cloud service constitutes a single point of failure, which can compromise data availability, download speed, and security. To address these challenges, we advocate for the use of multiple cloud storage providers simultaneously using network coding as the key enabling technology. Our goal is to study two challenges of network coded storage systems. First, the efficient update of the number of coded fragments per cloud in a system aggregating multiple clouds in order to boost the download speed of files. We developed a novel scheme using recoding with limited packets to trade-off storage space, reliability, and data retrieval speed. Implementation and measurements with commercial cloud providers show that up to 9x less network use is needed compared to other network coding schemes, while maintaining similar download speeds and reliability. Second, the ability to update coded fragments from a linear erasure code when the original file is modified. We exploit code structure to provide efficient representations of the evolution of the file. Evaluations using file changes on software library repositories show that a five-order of magnitude reduction in network and storage use is possible compared tostate-of-the-art.

AB - Although cloud systems provide a reliable and flexible storage solution, the use of a single cloud service constitutes a single point of failure, which can compromise data availability, download speed, and security. To address these challenges, we advocate for the use of multiple cloud storage providers simultaneously using network coding as the key enabling technology. Our goal is to study two challenges of network coded storage systems. First, the efficient update of the number of coded fragments per cloud in a system aggregating multiple clouds in order to boost the download speed of files. We developed a novel scheme using recoding with limited packets to trade-off storage space, reliability, and data retrieval speed. Implementation and measurements with commercial cloud providers show that up to 9x less network use is needed compared to other network coding schemes, while maintaining similar download speeds and reliability. Second, the ability to update coded fragments from a linear erasure code when the original file is modified. We exploit code structure to provide efficient representations of the evolution of the file. Evaluations using file changes on software library repositories show that a five-order of magnitude reduction in network and storage use is possible compared tostate-of-the-art.

KW - Distributed systems

KW - error control codes

KW - information storage and retrieval

UR - http://www.scopus.com/inward/record.url?scp=85032613150&partnerID=8YFLogxK

U2 - 10.1109/TCC.2015.2481433

DO - 10.1109/TCC.2015.2481433

M3 - Journal article

VL - 7

SP - 19

EP - 33

JO - IEEE Transactions on Cloud Computing

JF - IEEE Transactions on Cloud Computing

IS - 1

M1 - 7274724

ER -