Entropy-based abnormal activity detection fusing RGB-D and domotic sensors

Abstract

The automatic detection of anomalies in Active and Assisted Living (AAL) environments is important for monitoring the wellbeing and safety of the elderly at home. The integration of smart domotic sensors (e.g. presence detectors) and those ones equipping modern mobile robots (e.g. RGB-D cameras) provides new opportunities for addressing this challenge. In this paper, we propose a novel solution to combine local activity levels detected by a single RGB-D camera with the global activity perceived by a network of domotic sensors. Our approach relies on a new method for computing such a global activity using various presence detectors, based on the concept of entropy from information theory. This entropy effectively shows how active a particular room or environment's area is. The solution includes also a new application of Hybrid Markov Logic Networks (HMLNs) to merge different information sources for local and global anomaly detection. The system has been tested with a comprehensive dataset of RGB-D and domotic data containing data entries from 37 different domotic sensors (presence, temperature, light, energy consumption, door contact), which is made publicly available. The experimental results show the effectiveness of our approach and its potential for complex anomaly detection in AAL settings.

Wavelet-based Temporal Models of Human Activity for Anomaly Detection in Smart Robot-assisted Environments

“…For example, Xu et al [27] used a multiple one-class Support Vector Machine (SVM) models to predict anomaly scores, while Leyva et al [28] used Markov Chains to detect abnormal events on a video stream. Compared to camera-based systems, smart-home sensors offer a cheaper alternative for anomaly detection [5].…”

“…repeatedly opening/closing the fridge in the morning, but only on weekdays). In particular, we adopt the anomaly definition in [5], which considers the amount of motion in specific locations as a normalized entropy beyond some given thresholds. Note that the term "motion" is used in a broad sense to include the activation of various binary sensors, such as passive infrared (PIR) motion detectors or contact sensors on doors, cupboards, etc.…”

“…motion detectors and contact sensors), we build accurate temporal models to represent and forecast their expected output. Then, using an entropy-based method [5], we estimate the current and expected levels of human activity. These two are finally compared by an original inference system based on an Hybrid Markov Logic Network (HMLN) [6], to detect potential anomalies.…”

“…• Finally, we present extensive experimental results based on two large datasets, one previously recorded in an office environment [5] and a new one from a real elderly home, which we also made publicly available. These datasets were recorded in MongoDB format [7] for easy access and re-usability by the scientific community.…”

Research on Action Recognition and Content Analysis in videos based on DNN and MLN

“…Finally, the cloud server is used to perform collaborative calculation and feedback on the above two modules. Bellotto et al [Fernandez-Carmona, Cosar, Coppola et al (2017)] collected information from both local and global levels. The local information is obtained by the RGB-D camera, and the global information is defined by the normalized entropy.…”