Compiler-Assisted Operator Template Library for DNN Accelerators - Network and Parallel Computing Access content directly
Conference Papers Year : 2021

Compiler-Assisted Operator Template Library for DNN Accelerators


Despite many dedicated accelerators are gaining popularity for their performance and energy efficiency in the deep neural network (DNN) domain, high-level programming support for these accelerators remains thin. In contrast to existing researches targeting the whole DNNs, we choose to dive into details and review this problem from a finer-grained level, operators. Due to performance concerns, operator programmers may have to take hand-written assembly as their first choice, which is error-prone and involves many programming chores. To alleviate this problem, we propose TOpLib, a compiler-assisted template library. By providing a unified user-view abstraction, TOpLib allows programmers to express computational kernels with high-level tensor primitives, which will be automatically lowered into low-level intrinsic primitives via expression templates. Moreover, considering memory management is performance critical and the optimization strategy of expression template is limited to enumeration based rewriting rules, we implement TOpLib with a compiler-assisted approach. We address the memory reuse challenges into the compiler, which allows TOpLib to make full use of on-chip buffers and result in better performance. Experiments over 55 typical DNN operators demonstrate that TOpLib can generate scalable code with performance faster than or on par with hand-written assembly versions.
Fichier principal
Vignette du fichier
511910_1_En_1_Chapter.pdf (1.46 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03768760 , version 1 (04-09-2022)





Jiansong Li, Wei Cao, Xiao Dong, Guangli Li, Xueying Wang, et al.. Compiler-Assisted Operator Template Library for DNN Accelerators. 17th IFIP International Conference on Network and Parallel Computing (NPC), Sep 2020, Zhengzhou, China. pp.3-16, ⟨10.1007/978-3-030-79478-1_1⟩. ⟨hal-03768760⟩
14 View
6 Download



Gmail Facebook X LinkedIn More