Irish Sign Language Recognition Using Principal Component Analysis and Convolutional Neural Networks

Abstract

Abstract-Hand-shape recognition is an important problem in computer vision with significant societal impact. In this work, we introduce a new image dataset for Irish Sign Language (ISL) recognition and we compare between two recognition approaches. The dataset was collected by filming human subjects performing ISL hand-shapes and movements. Then, we extracted frames from the videos. This produced a total of 52,688 images for the 23 common hand-shapes from ISL. Afterwards, we filter the redundant images with an iterative image selection process that selects the images which keep the dataset diverse. For classification, we use Principal Component Analysis (PCA) with with K-Nearest Neighbours (k-NN) and Convolutional Neural Networks (CNN). We obtain a recognition accuracy of 0.95 for our PCA model and 0.99 for our CNN model. We show that image blurring improves PCA results to 0.98. In addition, we compare times for classification.

Hand orientation redundancy filter applied to hand-shapes dataset

“…In order to avoid redundancy in very similar frames, a method to filter frames with very low difference was proposed in [7]. This method is called the Iterative Redundancy Filter (IRF).…”

“…The dataset is divided into a training set and a testing set. Selection were made by random selection, then each of training and testing dataset consist of 25,000 images [7].…”

“…We did not consider the shapes with movement such as 'J', 'X' and 'Z'. Figure 3 show the occurrence by shape before filtering, using the filter proposed -IRF -in [7] and our new filter using pose angle -HORF -as well as the total of images proposed in the original ISL dataset [6].…”

“…A new ISL dataset with 58,114 images corresponding to the 23 ISL hand-shapes was introduced in [6]. Later on, a filter to reduce the frames similarity was proposed in [7], which we call the IRF dataset.…”

A comparison between end-to-end approaches and feature extraction based approaches for Sign Language recognition

“…We take this subset as the final dataset for subsequent experiments. In order to run experiment in a CPU we resized each image to 160 × 120 pixels [8].…”