Exploring the Importance of Sign Language Phonology for a Deep Neural Network

Abstract

We conduct an initial investigation to gain insight into whether a deep neural network learns phonological aspects of sign language when classifying video recordings of isolated signs from a continuous signing scenario. We train a series of neural networks to distinguish pairs of signs in Dutch Sign Language, controlling the phonological difference between the signs in each pair. Our results suggest that the intrinsic dimension of the final hidden layer of a network is surprisingly insensitive to the phonological difference between the signs in a pair. However, the ability of the network to discriminate two signs shows a clear trend towards increasing with increasing phonological distinctiveness. Related WorkIsolated sign language recognition. Sign language recognition (SLR) is the problem of recognizing and identifying a particular sign in a video clip. In this paper, we study isolated SLR, also known as word-level SLR, which