Activity Recognition in Smart Homes via Feature-Rich Visual Extraction of Locomotion Traces

Abstract

The proliferation of sensors in smart homes makes it possible to monitor human activities, routines, and complex behaviors in an unprecedented way. Hence, human activity recognition has gained increasing attention over the last few years as a tool to improve healthcare and well-being in several applications. However, most existing activity recognition systems rely on cameras or wearable sensors, which may be obtrusive and may invade the user’s privacy, especially at home. Moreover, extracting expressive features from a stream of data provided by heterogeneous smart-home sensors is still an open challenge. In this paper, we investigate a novel method to detect activities of daily living by exploiting unobtrusive smart-home sensors (i.e., passive infrared position sensors and sensors attached to everyday objects) and vision-based deep learning algorithms, without the use of cameras or wearable sensors. Our method relies on depicting the locomotion traces of the user and visual clues about their interaction with objects on a floor plan map of the home, and utilizes pre-trained deep convolutional neural networks to extract features for recognizing ongoing activity. One additional advantage of our method is its seamless extendibility with additional features based on the available sensor data. Extensive experiments with a real-world dataset and a comparison with state-of-the-art approaches demonstrate the effectiveness of our method.

Unobtrusive Cognitive Assessment in Smart-Homes: Leveraging Visual Encoding and Synthetic Movement Traces Data Mining

“…To enhance trajectory classification and cognitive assessment, this sub-module is in charge of visually encoding segmented trajectories and events of interest (EOI) in images, inspired by the method presented in [ 27 ]. In this context, EOIs refer to object interactions, the resident’s position, and movement indicators within the smart-home, which can be gathered through the sensor events.…”

“…This system transforms movement traces within an indoor environment into interpretable images, incorporating visual cues associated with speed of movement, sensor activations, and interactions with objects into the image-encoding process. This innovative approach was inspired by the earlier works of Zolfaghari et al in cognitive assessment, as presented in [ 12 , 26 ], as well as their contributions to human activity recognition outlined in [ 27 ].…”

Application of Deep Learning and Intelligent Sensing Analysis in Smart Home

“…Along with the continuous maturity of the Internet of Things, artificial intelligence, big data and other technologies, the intelligent innovation and development of optical sensors have been promoted. Based on cloud computing technology, deep learning algorithms such as convolutional neural networks and recurrent neural networks are applied to realize intelligent processing and the analysis of optical sensor data, which can effectively improve the data processing and computing ability, model training ability, hard disk storage ability and information transmission speed of optical sensors [24]. At the same time, the security and privacy of smart homes, as the user needs, are key to improving the user experience.…”

AtomGID: An Atomic Gesture Identifier for Qualitative Spatial Reasoning

“…Very often, smart homes still rely on very simple sensors due to the cost and the complexity of processing the data [23]. With the recent deep learning architectures, researchers are now including more advanced sensors such as passive RFID [24], ultrawideband (UWB) radars [25], and cameras [26]. While deep learning can readily extract features and transform complex data to learn meaningful patterns, in smart homes, for some specific tasks, it does not cover all scenarios and all needs.…”