FPVRGame: Deep Learning for Hand Pose Recognition in Real-Time Using Low-End HMD - Entertainment Computing and Serious Games Access content directly
Conference Papers Year : 2019

FPVRGame: Deep Learning for Hand Pose Recognition in Real-Time Using Low-End HMD

Abstract

Head Mounted Display (HMD) became a popular device, drastically increasing the usage of Virtual, Mixed, and Augmented Reality. While the systems’ visual resources are accurate and immersive, precise interfaces require depth cameras or special joysticks, requiring either complex devices or not following the natural body expression. This work presents an approach for the usage of bare hands to control an immersive game from an egocentric perspective and built from a proposed case study methodology. We used a DenseNet Convolutional Neural Network (CNN) architecture to perform the recognition in real-time, from both indoor and outdoor environments, not requiring any image segmentation process. Our research also generated a vocabulary, considering users’ preferences, seeking a set of natural and comfortable hand poses and evaluated users’ satisfaction and performance for an entertainment setup. Our recognition model achieved an accuracy of 97.89%. The user’s studies show that our method outperforms the classical controllers in regards to natural interactions. We demonstrate our results using commercial low-end HMD’s and compare our solution with state-of-the-art methods.
Fichier principal
Vignette du fichier
491829_1_En_6_Chapter.pdf (1.89 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03652048 , version 1 (26-04-2022)

Licence

Attribution

Identifiers

Cite

Eder De Oliveira, Esteban Clua, Cristina Nader Vasconcelos, Bruno Marques, Daniela Trevisan, et al.. FPVRGame: Deep Learning for Hand Pose Recognition in Real-Time Using Low-End HMD. 1st Joint International Conference on Entertainment Computing and Serious Games (ICEC-JCSG), Nov 2019, Arequipa, Peru. pp.70-84, ⟨10.1007/978-3-030-34644-7_6⟩. ⟨hal-03652048⟩
17 View
54 Download

Altmetric

Share

Gmail Facebook X LinkedIn More