Activity Recognition Based on Intra and Extra Manipulation of Everyday Objects

Abstract

Abstract.Recognizing activities based on an actor's interaction with everyday objects is an important research approach within ubiquitous computing. We present a recognition approach which complement objects grabbed or released information with the object's internal state changes (as an effect of intra manipulation) and the object's external state changes with reference to other objects (as an effect of extra manipulation). The concept of Intra manipulation is inspired by the fact that many everyday objects change their internal state when manipulated by the human actor, while extra manipulation is motivated by the fact that humans commonly rearrange the spatial relations between everyday objects as part of their activities. A detailed evaluation of our prototype activity recognition system in virtual reality (VR) environment is presented as a "proof of concept". We have obtained a recognition precision of 92% on the activity-level and 81% on the action-level among 15 everyday home activities. Virtual reality was used as a test-bed in order to speed up the design process of our activity recognition system, allowing us to compensate for the limitations with currently available sensing technologies and to compare the contributions of intra manipulation and extra manipulation for activity recognition.

An Activity-Centered Wearable Computing Infrastructure for Intelligent Environment Applications

“…For many intelligent environment applications, knowing the inhabitant's activity information alone is not sufficient [4]. This introduces a necessity to recognize activities and actions related to those activities with fine granularity; to recognize those activities during the initial phase of an activity before the activity reaches an irreversible state; and to determine the end of an activity [7], [8]. According to Activity Theory [9], human activities have an objective and are mediated through tools.…”

“…We consider the objects present in the occupant's environment as tools for accomplishing his/her activities and recognize the activities based on the occupant's usage of those objects. For more information about the two activity recognition systems that we have built based on the occupant's usage of everyday objects, we refer to [7], [8].…”

“…The first issue is addressed by considering the action the occupant is currently performing within an activity and based on the status of an activity (initiated, interrupted or completed). For more information about how activities are modelled, we refer to [8]. The second issue is addressed by considering interactive devices around the occupant's body and based on the occupant's perceptual capabilities (explicit output devices within the observable space and explicit input devices within the manipulable space).…”

“…are embedded on complex everyday objects to sense their internal states and state changes. RFID readers are embedded on container objects to sense the objects present within the containers and when an object enters or leaves a container [8]. The instrumented sensors communicate with the object manager using ZigBee communication protocol [22].…”

“…For container objects, it stores the identities of the set of objects the container contains. It also stores the event that is generated when an object enters or leaves a container object [8]. Situative space monitor maintains a real-time model of the environment by capturing and storing information about the objects present in the environment from the occupant's egocentric perspective based on what the occupant can see and not see, touch and not touch at any given moment in time [7], [10].…”