GraphScSh: Efficient I/O Scheduling and Graph Sharing for Concurrent Graph Processing - Network and Parallel Computing
Conference Papers Year : 2019

GraphScSh: Efficient I/O Scheduling and Graph Sharing for Concurrent Graph Processing

Zhan Shi
  • Function : Author
  • PersonId : 1161695
Dan Feng
  • Function : Author
  • PersonId : 1161624
Yamei Peng
  • Function : Author
  • PersonId : 1161697

Abstract

With the increasing need for analyzing graph data, graph systems have to efficiently deal with concurrent graph processing (CGP) jobs. However, existing platforms are inherently designed for a single job, they incur the high cost when CGP jobs are executed. In this work, we observed that existing systems do not allow CGP jobs to share graph structure data of each iteration, introducing redundant accesses to same graph. Moreover, all the graphs are real-world graphs with highly skewed power-law degree distributions. The gain from extending multiple external storage devices is diminishing rapidly, which needs reasonable schedulings to balance I/O pressure into each storage. Following this direction, we propose GraphScSh that handles CGP jobs efficiently on a single machine, which focuses on reducing I/O conflict and sharing graph structure data among CGP jobs. We apply a CGP balanced partition method to break graphs into multiple partitions that are stored in multiple external storage devices. Additionally, we present a CGP I/O scheduling method, so that I/O conflict can be reduced and graph data can be shared among multiple jobs. We have implemented GraphScSh in C++ and the experiment shows that GraphScSh outperforms existing out-of-core systems by up to 82%.
Fichier principal
Vignette du fichier
486810_1_En_1_Chapter.pdf (540.83 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03770571 , version 1 (06-09-2022)

Licence

Identifiers

Cite

Shang Liu, Zhan Shi, Dan Feng, Shuo Chen, Fang Wang, et al.. GraphScSh: Efficient I/O Scheduling and Graph Sharing for Concurrent Graph Processing. 16th IFIP International Conference on Network and Parallel Computing (NPC), Aug 2019, Hohhot, China. pp.3-15, ⟨10.1007/978-3-030-30709-7_1⟩. ⟨hal-03770571⟩
40 View
80 Download

Altmetric

Share

More