Speech and sliding text aided sign retrieval from hearing impaired sign news videos

Abstract

The objective of this study is to automatically extract annotated sign data from the broadcast news recordings for the hearing impaired. These recordings present an excellent source for automatically generating annotated data: In news for the hearing impaired, the speaker also signs with the hands as she talks. On top of this, there is also corresponding sliding text superimposed on the video. The video of the signer can be segmented via the help of either the speech or both the speech and the text, generating segmented, and annotated sign videos. We call this application O. Aran ( ) · I. Ari · L. Akarun as Signiary, and aim to use it as a sign dictionary where the users enter a word as text and retrieve sign videos of the related sign. This application can also be used to automatically create annotated sign databases that can be used for training recognizers.

Towards Automatic Annotation of Sign Language Dictionary Corpora

“…We threshold and scale the probability of the model to obtain a 256 × 256 × 256 look-up-table with values from 0 to 255. We were inspired by the work [15] which we refer to for further details.…”

Automatic fingersign-to-speech translation system

“…Therefore studies in the literature make use of different cues and restrictions to perform hand segmentation. Popular methods for hand segmentation include using skin color cues [5], motion cues [37], shape cues [9] or depth information [39].…”

HMM-based sliding video text recognition for Turkish broadcast news

“…We convert the first frame of each video clip into a grey-level image and use its horizontal projection to detect the vertical position of the 20 pixel high 352 pixel wide text band. As proposed in [4], this band lies between the hills of the horizontal projection which is a simple summation of the rows of the grey-level image. Text band hosts a logo towards the left end of the region.…”

“…[3] proposed multi-frame combination technique as a robust video text feature extraction method. [4], [5] finds the text location from the horizontal projection of the grey-level image.…”