A representation for human gesture recognition and beyond

Abstract

A highly general and centralized reasoning framework which combines first-order-logic with Markov networks proposed to recognize both simple and complex activities. The generality and systematicity of the reasoning framework is characterized by a newly defined set of spatio-temporal and spatial semantic free low level event predicates(LLEs). With the new low level event predicates any human activity represented by trajectories can be described without domain knowledge thus can be applied across domains. High-level events (HLEs) of interest across different domains can be described by encoding the newly defined HLEs and temporal logic (Allen's interval logic) in a first-order-logic presentation. The main contribution is the proposed reasoning framework represented by a new set of semantics free LLEs which can be utilized across different domains. The human action Kinect dataset from Microsoft Research(MSR) is used to evaluate the proposed gesture representation and recognition framework. The capacity of performing across different domains is validated on both MSR dataset and one synthetic interation dataset.

MIFTel: a multimodal interactive framework based on temporal logic rules

“…When two sensors collaborate to recognize the same event, it could be not necessary to consider the combination of single results. The proposed grammar is inspired by other multimodal spatio-temporal logic based frameworks [123,15]. A rule is composed by two main sections: the condition and the actuation.…”