Toward Building Safer Smart Homes for the People with Disabilities

Abstract

Situational awareness is a critical foundation for the protection of human life/properties and is challenging to maintain for people with disabilities (i.e., visual impairments and limited mobility). In this paper, we present a dialog enabled end-to-end assistive solution called SafeAccess to build a safer smart home by providing situational awareness. The key functions of SafeAccess are: -1) monitoring homes and identifying incoming persons; 2) helping users in assessing incoming threats (e.g., burglary, robbery, gun violence); and, 3) allowing users to grant safe access to homes for friends/families. In this work, we focus on building a robust model for detecting and recognizing person, generating image descriptions, and designing a prototype for the smart door. To interact with the system, we implemented a dialog enabled smartphone app, especially for creating a personalized profile from face images or videos of friends/families. A Raspberry pi connected to the home monitoring cameras captures the video frames and performs change detection to identify frames with activities. Then, we detect human presence using Faster r-cnn and extract faces using Multi-task Cascaded Convolutional Networks (MTCNN). Subsequently, we match the detected faces using FaceNet/support vector machine (SVM) classifiers. The system notifies users with an MMS containing the name of incoming persons or as unknown, scene image, facial description, and contextual information. The users can grant access or call emergency services using the SafeAccess app based on the received notification. Our system identifies persons with an F-score 0.97 and recognizes items to generate image description with an average F-score 0.97.

Security and privacy concerns in assisted living environments

“…As advocated by [19], technology-rich environments must be implemented to respect the right of users to protect their privacy. Due to the amount of sensitive data collected by IoT devices, as well as by fall detection systems [23,32,50,51] and monitoring systems [6,12,35,48,58], a secure assisted living environment should be developed to avoid data leaks.…”

“…Intelligent environments, which combine different technologies, have been used to assist individuals with and without limitations in accomplishing their daily activities. Although many advances in this area have been made, with the development of alternative interaction methods [18,19,25,52,59] and monitoring systems [6,12,23,32,35,48,50,51,58], there are many challenges to overcome in developing a secure and safe assisted living environment. It is necessary to guarantee the privacy and data protection of the inhabitants in any intelligent environment to avoid disclosure or misuse of their sensitive information.…”

“…Through these interviews, we concluded that all of the participants are well aware about the potential risks related to use of domotic technologies, including surveillance and monitoring systems. 6 They expressed concerns about potential hacker intrusions and information leaks related to the use of these technologies. Thus, although the proposed solution allows Paulo and his family members to turn on/off all systems, they suggested that we should design a mechanism that would allow both inhabitants and visitors to activate a temporary privacy mode, in which all sensitive technologies are deactivated.…”

“…Furthermore, Paulo's friends might have also been afraid that such devices could be misused or hacked. 6 Please note that this group may not be a representative sample of the entire population due to their previous knowledge regardings certain risks related to use of IoT devices, acquired through their contact with Paulo's family. 7 Such functionality would be integrated with our proposed robotic solution, allowing anyone to be able to control that mechanism.…”

“…Fortunately, there are several assistive technologies that allow them to have a more independent life [1,4,17,18,26,[29][30][31]33,59,61,65]. Some of these technologies have been used to enable people with limitations 1 to use intelligent envi-ronments, namely easy-to-use interaction methods [7,18,25,52], assistive robotic solutions [19,20], fall detection systems [23,32,50,51] and monitoring systems [6,12,35,48,58].…”