Human Behavior Analysis by Means of Multimodal Context Mining

Abstract

There is sufficient evidence proving the impact that negative lifestyle choices have on people’s health and wellness. Changing unhealthy behaviours requires raising people’s self-awareness and also providing healthcare experts with a thorough and continuous description of the user’s conduct. Several monitoring techniques have been proposed in the past to track users’ behaviour; however, these approaches are either subjective and prone to misreporting, such as questionnaires, or only focus on a specific component of context, such as activity counters. This work presents an innovative multimodal context mining framework to inspect and infer human behaviour in a more holistic fashion. The proposed approach extends beyond the state-of-the-art, since it not only explores a sole type of context, but also combines diverse levels of context in an integral manner. Namely, low-level contexts, including activities, emotions and locations, are identified from heterogeneous sensory data through machine learning techniques. Low-level contexts are combined using ontological mechanisms to derive a more abstract representation of the user’s context, here referred to as high-level context. An initial implementation of the proposed framework supporting real-time context identification is also presented. The developed system is evaluated for various realistic scenarios making use of a novel multimodal context open dataset and data on-the-go, demonstrating prominent context-aware capabilities at both low and high levels.

Improving environment detection by behavior association for context‐adaptive navigation

“…As investigated in Gao and Groves, 30 the feature comprising the total C/N 0 values summed across the satellite signals above 25 dB-Hz was proven to be more useful rather than the single summed C/N 0 values used alone in indoor/outdoor classification for a pedestrian. Therefore, in addition to the two features described above, it is adopted as a third feature for pedestrian-based environment classification, denoted as sumCNR 25 . The similar feature metric optimized for vehicle context will be discussed in Section 5.2.…”

Multi-sensor fusion based on multiple classifier systems for human activity identification

“…Feature concatenation methods that involve the fusion of vision based sensors and inertial sensors have also been proposed using machine learning for human activity detection and health monitoring. The fusion methods enable identification of mobility changes, complex and concurrent activity details and behaviour tracking [43]. However, each sensor modalities provide different statistical properties for recognition of particular activity details and maybe not be optimal to aggregate these features before applying learning algorithms [44].…”

A Novel Framework for the Holistic Monitoring and Analysis of Human Behaviour

“…Gaggioli et al [2] presented a platform to gather users' psychological, physiological and activity information for describing mental health. Banos et al [3] proposed a machine-learning-based framework to recognize behaviour elements, such as physical activities, emotions and locations, which are further combined through ontological methods into more abstract representations of user context. Furthermore, there is an increasing number of commercial frameworks, such as Apple HealthKit, GoogleFit and Samsung SAMI, that use different types of behaviour-related contextual information, while they rely on third-party applications and systems for inferring the behaviour information [3].…”