BSL-1K: Scaling Up Co-articulated Sign Language Recognition Using Mouthing Cues

Sign Language Segmentation with Temporal Convolutional Networks

“…We calculate the F1 score for the boundaries as the harmonic mean of precision and recall, given this definition of a correct boundary detection. We use all integer-valued thresholds that fall within the closed interval [1,4] and report the mean across thresholds, which we refer to as mF1B. The quality of the sign segments is also evaluated with the F1 score, where sign segments with an IoU higher than a given threshold are defined as correct.…”

“…Although the task of automatically identifying temporal boundaries between signs has received attention in the literature, it has typically been tackled with methods that require Fig. 2: Datasets: We provide samples from each dataset we use in this work: BSLCORPUS [27,28], BSL-1K [1], PHOENIX14 [20]. access to a semantic labelling of the signed content (e.g., in the form of glosses or free-form sentence translations) [18,21,26].…”

“…Training. Motivated by its effectiveness for human action recognition and recently for sign language recognition [1,16,22] and sign spotting [24], our approach adopts the spatiotemporal convolutional I3D architecture [7] and couples it with the Multi-Stage Temporal Convolutional Network (MS-TCN) module proposed by [13]. Each stage of the latterknown as a Single-Stage TCN (SS-TCN)-comprises a stack of dilated residual layers of 1D temporal convolutions, followed by a linear classifier which is used to predict framelevel labels, (y 1 , .…”

“…BSL-1K. We apply our segmentation model on the BSL-1K dataset and complement the sparse sign annotations provided by [1] with timing information for start and end frames. While both our training data BSLCORPUS and test data BSL-1K include the same sign language, there is a considerable domain gap in appearance as shown in Fig.…”

“…We make the following contributions: (1) We demonstrate the effectiveness of coupling robust 3D spatio-temporal convolutional neural network (CNN) representations with an iter-…”