Smart Assistive Architecture for the Integration of IoT Devices, Robotic Systems, and Multimodal Interfaces in Healthcare Environments

Abstract

This paper presents a new architecture that integrates Internet of Things (IoT) devices, service robots, and users in a smart assistive environment. A new intuitive and multimodal interaction system supporting people with disabilities and bedbound patients is presented. This interaction system allows the user to control service robots and devices inside the room in five different ways: touch control, eye control, gesture control, voice control, and augmented reality control. The interaction system is comprised of an assistive robotic arm holding a tablet PC. The robotic arm can place the tablet PC in front of the user. A demonstration of the developed technology, a prototype of a smart room equipped with home automation devices, and the robotic assistive arm are presented. The results obtained from the use of the various interfaces and technologies are presented in the article. The results include user preference with regard to eye-base control (performing clicks, and using winks or gaze) and the use of mobile phones over augmented reality glasses, among others.

Adapting to progressive paralysis: A tongue-brain hybrid robot interface for individuals with amyotrophic lateral sclerosis

“…Similar multimodal control frameworks have previously been suggested, but have not been used for continuous control of a robot arm [62]- [64]. These frameworks do not directly combine different modalities to improve control, they instead make multiple modality control options available to the user.…”

IoT Smart Flooring Supporting Active and Healthy Lifestyles

“…IoT is widely exploited for health-oriented applications [ 2 , 3 , 4 , 5 ], including highly specific solutions, supporting the needs of people with disabilities [ 6 , 7 ], or techniques for non-intrusive, continuous monitoring of relevant physiological parameters (e.g., blood pressure [ 8 ], heart rate [ 9 ]). Wearable sensors are widely used, with the non-wearable approach being sought for further reducing the burden of the user [ 10 , 11 ].…”

Voice-based control system for smart hospital wards: a pilot study of patient acceptance

“…Voice-based control, a specific IoT technology application used in healthcare environments, could improve healthcare quality and experience [ 3 – 5 ]. The voiced-based control system was also an effective way to reduce contamination of surfaces, which could decrease the spread of healthcare-associated infections (HCAIs) with touchless computer interfaces [ 5 ].…”

“…Since patients are consumers under this condition, studying their points before the voice-based control system's actual implementation is essential. Previous studies conducted in laboratory environments showed that users considered positive with different interaction modes in the wards [ 3 , 6 ]. Here, we conducted an experimental study in the real clinical workflow.…”