Tensor-Based CUDA Optimization for ANN Inferencing Using Parallel Acceleration on Embedded GPU - Artificial Intelligence Applications and Innovations Access content directly
Conference Papers Year : 2020

Tensor-Based CUDA Optimization for ANN Inferencing Using Parallel Acceleration on Embedded GPU

Abstract

With image processing, robots acquired visual perception skills; enabling them to become autonomous. Since the emergence of Artificial Intelligence (AI), sophisticated tasks such as object identification have become possible through inferencing Artificial Neural Networks (ANN). Be that as it may, Autonomous Mobile Robots (AMR) are Embedded Systems (ESs) with limited on-board resources. Thus, efficient techniques in ANN inferencing are required for real-time performance. This paper presents the process of optimizing ANNs inferencing using tensor-based optimization on embedded Graphical Processing Unit (GPU) with Computer Unified Device Architecture (CUDA) platform for parallel acceleration on ES. This research evaluates renowned network, namely, You-Only-Look-Once (YOLO), on NVIDIA Jetson TX2 System-On-Module (SOM). The findings of this paper display a significant improvement in inferencing speed in terms of Frames-Per-Second (FPS) up to 3.5 times the non-optimized inferencing speed. Furthermore, the current CUDA model and TensorRT optimization techniques are studied, comments are made on its implementation for inferencing, and improvements are proposed based on the results acquired. These findings will contribute to ES developers and industries will benefit from real-time performance inferencing for AMR automation solutions.
Fichier principal
Vignette du fichier
497040_1_En_25_Chapter.pdf (413.58 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04050601 , version 1 (29-03-2023)

Licence

Attribution

Identifiers

Cite

Ahmed Al Ghadani, Waleeja Mateen, Rameshkumar G. Ramaswamy. Tensor-Based CUDA Optimization for ANN Inferencing Using Parallel Acceleration on Embedded GPU. 16th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2020, Neos Marmaras, Greece. pp.291-302, ⟨10.1007/978-3-030-49161-1_25⟩. ⟨hal-04050601⟩
20 View
30 Download

Altmetric

Share

Gmail Facebook X LinkedIn More