Sign Language Recognition Using Cluster and Chunk-Based Feature Extraction and Symbolic Representation

Abstract

This paper focuses on sign language recognition with respect to the hand movement trajectories at a sentence level. This is achieved by applying two proposed methods namely Chunk-based and Cluster-based feature representation techniques in order to extract the desired keyframes. The features are extracted based on hands and head local centroid characteristics such as velocity, magnitude and orientation. A set of experiments are conducted on a large self-curated sign language sentence data set (UOM-SL2020) in order to evaluate the performance of the proposed methods. The results clearly show the high recognition rate of 75.51% in terms of F-measure which is achieved by combining the proposed method with symbolic interval-based representation and validation of feature sets.