PDRM: A Probability Distribution Based Resource Management for Batch Workloads in Heterogeneous Cluster - Network and Parallel Computing Access content directly
Conference Papers Year : 2019

PDRM: A Probability Distribution Based Resource Management for Batch Workloads in Heterogeneous Cluster

Abstract

Resource consumption prediction and dynamic resource provision based on historical consumption are common methods to improve cluster resource utilization, however they have to face the challenge of fluctuation in resource consumption for accurate prediction. We propose PDRM, an efficient resource management scheme based on resource consumption probability distribution for batch workloads to deal with this dilemma. Based on the common sense that the same type of tasks have similar resource consumption on the same node, we get the resource consumption probability distribution of each type of task to describe the fluctuations in its resource consumption. Based on the resource consumption distribution function, we can allocate resources precisely for tasks. Experimental results demonstrate that PDRM achieves good performance for various application in the heterogeneous cluster. PDRM can effectively improve resource utilization and reduce job completion time.
Fichier principal
Vignette du fichier
486810_1_En_34_Chapter.pdf (255.52 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03770531 , version 1 (06-09-2022)

Licence

Attribution

Identifiers

Cite

Jun Zhou, Dan Feng, Fang Wang. PDRM: A Probability Distribution Based Resource Management for Batch Workloads in Heterogeneous Cluster. 16th IFIP International Conference on Network and Parallel Computing (NPC), Aug 2019, Hohhot, China. pp.361-365, ⟨10.1007/978-3-030-30709-7_34⟩. ⟨hal-03770531⟩
12 View
43 Download

Altmetric

Share

Gmail Facebook X LinkedIn More