The HDFS Replica Placement Policies: A Comparative Experimental Investigation - Distributed Applications and Interoperable Systems
Conference Papers Year : 2022

The HDFS Replica Placement Policies: A Comparative Experimental Investigation

Abstract

The Hadoop Distributed File System (HDFS) is a robust and flexible file system designed for reliably storing large volumes of data in distributed environments. Its storage model relies upon data replication and one of its central features is to optimize the placement of the replicas across the cluster for fault tolerance, availability, and performance. To this end, the Replica Placement Policy selects which nodes will store the data blocks. This work presents an experimental investigation of the different placement strategies available in HDFS. For a broader analysis, we consider different stages where the placement of the replicas is necessary, such as writing files in the system, re-replicating blocks among the nodes, and balancing the replica distribution in the cluster. The evaluation results allowed a deeper understanding of the behavior of the policies, in addition to highlighting the advantages and drawbacks of the replica placement concerning optimizations in data availability, data locality, write and read throughput, and in the overall performance of the HDFS.
Embargoed file
Embargoed file
0 0 10
Year Month Jours
Avant la publication
Wednesday, January 1, 2025
Embargoed file
Wednesday, January 1, 2025
Please log in to request access to the document

Dates and versions

hal-03855562 , version 1 (09-12-2024)

Licence

Identifiers

Cite

Rhauani Weber Aita Fazul, Patrícia Pitthan Barcelos. The HDFS Replica Placement Policies: A Comparative Experimental Investigation. 17th IFIP International Conference on Distributed Applications and Interoperable Systems (DAIS), Jun 2022, Lucca, Italy. pp.151-166, ⟨10.1007/978-3-031-16092-9_10⟩. ⟨hal-03855562⟩
9 View
0 Download

Altmetric

Share

More