A Performance Evaluation of Erasure Coding Libraries for Cloud-Based Data Stores - Distributed Applications and Interoperable Systems (DAIS 2016)
Conference Papers Year : 2016

A Performance Evaluation of Erasure Coding Libraries for Cloud-Based Data Stores

Abstract

Erasure codes have been widely used over the last decade to implement reliable data stores. They offer interesting trade-offs between efficiency, reliability, and storage overhead. Indeed, a distributed data store holding encoded data blocks can tolerate the failure of multiple nodes while requiring only a fraction of the space necessary for plain replication, albeit at an increased encoding and decoding cost. There exists nowadays a number of libraries implementing several variations of erasure codes, which notably differ in terms of complexity and implementation-specific optimizations.Seven years ago, Plank et al. [14] have conducted a comprehensive performance evaluation of open-source erasure coding libraries available at the time to compare their raw performance and measure the impact of different parameter configurations. In the present experimental study, we take a fresh perspective at the state of the art of erasure coding libraries. Not only do we cover a wider set of libraries running on modern hardware, but we also consider their efficiency when used in realistic settings for cloud-based storage, namely when deployed across several nodes in a data centre. Our measurements therefore account for the end-to-end costs of data accesses over several distributed nodes, including the encoding and decoding costs, and shed light on the performance one can expect from the various libraries when deployed in a real system. Our results reveal important differences in the efficiency of the different libraries, notably due to the type of coding algorithm and the use of hardware-specific optimizations.
Fichier principal
Vignette du fichier
416479_1_En_13_Chapter.pdf (470.81 Ko) Télécharger le fichier
Origin Files produced by the author(s)
Loading...

Dates and versions

hal-01434792 , version 1 (13-01-2017)

Licence

Identifiers

Cite

Dorian Burihabwa, Pascal Felber, Hugues Mercier, Valerio Schiavoni. A Performance Evaluation of Erasure Coding Libraries for Cloud-Based Data Stores. 16th IFIP WG 6.1 International Conference on Distributed Applications and Interoperable Systems (DAIS), Jun 2016, Heraklion, Crete, Greece. pp.160-173, ⟨10.1007/978-3-319-39577-7_13⟩. ⟨hal-01434792⟩
118 View
803 Download

Altmetric

Share

More