RemoAct: Portable Projected Interface with Hand Gesture Interaction

Abstract

RemoAct is a wearable depth sensing and projection system that enables interaction on many surfaces. It makes interaction with the environment more intuitive through sharing and sending data with surrounding devices by applying certain gestures. This system offers a mobile and intuitive solution for interacting using a projected surface on habitual flat surfaces. Every user has their public and private areas, where the user can create tiles on the fly and share it with others and these public tiles are shown to other users through augmented reality. Interaction is made through hand gestures, finger tracking and hand tracking. This gives the user more freedom in movement. Different experiments were conducted to calculate the accuracy and RemoAct ran against different conventional methods to compare its accuracy, time and user experience. RemoAct takes less time for two users to draw one chart. As the system enables the users to work simultaneously, it reduces the needed time, short compared to successive drawing. For gesture recognition, accuracy reached 90-95\%. Object recognition and face identification accuracy varied with the variation of light.

Systematic literature review of hand gestures used in human computer interaction interfaces

“… Interaction with a display/projection (Bolt, 1980, Choi et al, 2007, Foehrenbach et al, 2009, Beyer and Meier, 2011, Asadzadeh et al, 2012, Cauchard et al, 2012, Xie and Xu, 2013, Rossol et al, 2014, Saxen et al, 2014, Adeen et al, 2015, Braun et al, 2017, Osti et al, 2017, Dondi et al, 2018, Ma et al, 2018.  Interaction with augmented reality (Reifinger et al, 2007, Lu et al, 2012, Bai et al, 2013, Hürst and van Wezel, 2013, Gangman and Yen, 2014, Adhikarla et al, 2015, Hernoux and Christmann, 2015, Shim et al, 2016, Saxen et al, 2014, Kim and Lee, 2016, Memo and Zanuttigh, 2018.…”