DROAllocator: A Dynamic Resource-Aware Operator Allocation Framework in Distributed Streaming Processing - Network and Parallel Computing
Conference Papers Year : 2021

DROAllocator: A Dynamic Resource-Aware Operator Allocation Framework in Distributed Streaming Processing

Abstract

With the rapid development of Internet services and the Internet of Things (IoT), many studies focus on operator allocation to enhance the DSPAs’ (data stream processing applications) performance and resource utilization. However, the existing approaches ignore the dynamic changes of the node resources to allocate the operator instances to guarantee the performance, which increasing the number of migration leads to the waste of resources and the instability. To address these issues, we propose a framework named DROAllocator to select the appropriate nodes to allocate the operator instances. By capturing the change tendency of the node resources and the operator performance, our allocation mechanism decreases the number of migration to enhance the performance. The experimental results show the DROAllocator not only decrease the number of migrations to allocate the operator instances to ensure the end-to-end throughput and the latency, but also enhance the resource utilization.
Fichier principal
Vignette du fichier
511910_1_En_30_Chapter.pdf (809.33 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03768731 , version 1 (04-09-2022)

Licence

Identifiers

Cite

Fan Liu, Zongze Jin, Weimin Mu, Weilin Zhu, Yun Zhang, et al.. DROAllocator: A Dynamic Resource-Aware Operator Allocation Framework in Distributed Streaming Processing. 17th IFIP International Conference on Network and Parallel Computing (NPC), Sep 2020, Zhengzhou, China. pp.349-360, ⟨10.1007/978-3-030-79478-1_30⟩. ⟨hal-03768731⟩
32 View
37 Download

Altmetric

Share

More