X-Vision: An Augmented Vision Tool with Real-Time Sensing Ability in Tagged Environments

Armengol-Urpi

2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)

et al. 2019

Self Cite

We present an Augmented Reality (AR) visualization and interaction tool for users to control Internet of Things (IoT) devices with hand gestures. Today, smart IoT devices are becoming increasingly ubiquitous with diverse forms and functions, yet most user controls over them are still limited to mobile devices and web interfaces. Recently, AR has been developed rapidly, and provided immersive solutions to enhance user experience of applications in many fields. Its capability to create immersive interactions allows AR to improve the way smart devices are controlled via more direct visual feedback. In this paper, we create a functional prototype of one such system, enabling seamless interactions with sound and lighting systems through the use of augmented hand-controlled interaction panels. To interpret users' intentions, we implement a standard 2D convolution neural network (CNN) for real-time hand gesture recognition and deploy it within our system. Our prototype is also equipped with a simple but effective object detector which can identify target devices within a proper range by analyzing visual and geometric features. We evaluate the performance of our system qualitatively and quantitatively and demonstrate it on two smart devices.

Section: Ar Enhances User Experiencementioning

confidence: 99%

MagicHand: Interact with IoT Devices in Augmented Reality Environment

Armengol-Urpi

2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)

et al. 2019

Self Cite

“…Augmented reality and computer vision applications can also leverage information from tags to create immersive experiences for users by connecting even low-cost everyday objects such as cups, water bottles, etc. to the network in smart spaces [15].…”

Section: B Activity Sensing With Rfid Tagsmentioning

confidence: 99%

Learning Gestures Using A Passive Data-Glove With RFID Tags

2019 IEEE International Conference on RFID Technology and Applications (RFID-TA)

Bhattacharyya

2019

Self Cite

Hand gesture recognition enables non-tactile interfaces for human-machine interactions. Cameras are currently powerful tools to recognize these gestures, however, use of cameras is constrained by privacy concerns and need for welllit, line of sight implementation. In this study, we propose an alternate method to recognize gestures using a passive data-glove augmented with passive RFID tags. We envision passive tagsbased gesture recognition will have applications in improving operator safety around machines, activity monitoring in factories and sign to speech recognition, etc. Low-level reader information (RSSI, Phase and Doppler frequency) can be used to capture changes to the tags in the environment, therefore generating enough information to infer gestures. In this paper, we present a technique to enable fast feature recognition using low-level reader data by correcting for inconsistencies in phase data due to frequency hopping. We experimented with four different classifiers on the low-level reader data and our Fully-Connected Neural Network (FCCN) classifier is able to learn gestures from tag-data with 98% accuracy.

“…Fusing IoT and AR is not entirely novel, and has been demonstrated in [3]- [5]. However, AR has not been adopted to the fullest in a manufacturing context.…”

Section: Introductionmentioning

confidence: 99%

Towards Industrial IoT-AR Systems using Deep Learning-Based Object Pose Estimation

2019 IEEE 38th International Performance Computing and Communications Conference (IPCCC)

Siegel

et al. 2019

Self Cite

Augmented Reality (AR) is known to enhance user experience, however, it remains under-adopted in industry. We present an AR interaction system improving human-machine coordination in Internet of Things (IoT) and Industry 4.0 applications including manufacturing and assembly, maintenance and safety, and other highly-interactive functions. A driver of slow adoption is the computational complexity and inaccuracy in localization and rendering digital content. AR systems may render digital content close to the associated physical objects, but traditional object recognition and localization modules perform poorly when tracking texture-less objects and complex shapes, presenting a need for robust and efficient digital content rendering techniques. We propose a method of improving IoT-AR by integrating Deep Learning with AR to increase accuracy and robustness of the target object localization module, taking both color and depth images as input and outputting the target's pose parameters. Quantitative and qualitative experiments prove this system's efficacy and show potential for fusing these emerging technologies in real-world applications.