Recognition of deaf gestures based on a bio-inspired neural network

Grif, Mikhail G.; Kugaevskikh, A. V.

doi:10.1088/1742-6596/1661/1/012038

J. Phys.: Conf. Ser.

2020

DOI: 10.1088/1742-6596/1661/1/012038

|View full text |Cite

Recognition of deaf gestures based on a bio-inspired neural network

Mikhail G. Grif

A. V. Kugaevskikh

Abstract: In this paper discusses the current situation in Russia and the world in the field of development of sign languages translation system. The main problems are formulated, and ways to solve them are given. One of the most important unresolved tasks is the task of recognizing the gestures of the deaf. To effectively solve it, an approach based on the development of bio-inspired neural networks is proposed. The architecture of a bio-inspired neural network, including four types of neurons, is described. New simple… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2022

Publication Types

Select...

Article1

Relationship

Self Cite0

Independent1

Authors

Journals

Cited by 1 publication

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Russian Sign Language Database for Clinical Use: Data and Annotation Peculiarities

Kagirov

Ryumin

2022

jour

View full text Add to dashboard Cite

This work has been carried out by the members of the Laboratory of Speech and Multimodal Interfaces of the St. Petersburg Federal Research Center of the Russian Academy of Sciences within an interdisciplinary research project aimed at the creation of an automatic Russian sign language translation system. The paper presents the design of a Russian sign language digital database for a specific subject area, namely, «The first visit to doctor».Our database is meant to be used first of all a dataset for training neural network-based systems of automatic translation from Russian sign language. But also, it can be of interest for linguistics of sign languages in general since the approach elucidates solution of a number of distorting phenomena typical for continuous sign language tracking, such as epenthesis, assimilation, reduction, and hold deletion.The principal difference between the presented video data and other datasets developed for similar purposes is the use of continuous sign utterances and elements of Russian sign language proper instead of the so-called “signed Russian” (that is, the visual form of the Russian spoken language), popular in deaf schooling.One of the most challenging problems, dealt with in the paper, is an automatic segmentation of sign speech into separate meaningful units with clearly defined boundaries. Its solution is hampered by signs’ deformations in continuous speech. In this paper, the segmentation is carried out within the framework of the movement-hold model. This approach allows extraction of their functional core as well as annotation of possible changes likely to appear in different signs’ segments. Accordingly, each utterance was subdivided into segments of motion and hold, and the resulting sign schemes were then entered into a separate directory containing information about the main parameters of each hold, including the possibilities of change due to immediate environment. The result of this work is a full decomposition of the signs, forming the database which can find its application in different statistical models of Russian Sign Language.Among other linguistic peculiarities of the database should be noted lexical variability of signs, mouthing and code switching between Russian sign language and “signed Russian”. Also, our signers learned Russian sign language in different regions of Russia, thus the collected data is potentially a source for research in dialectal variability of Russian sign language.

show abstract

Russian Sign Language Database for Clinical Use: Data and Annotation Peculiarities

Kagirov

Ryumin

2022

jour

View full text Add to dashboard Cite

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Recognition of deaf gestures based on a bio-inspired neural network

Cited by 1 publication

References 4 publications

Russian Sign Language Database for Clinical Use: Data and Annotation Peculiarities

Russian Sign Language Database for Clinical Use: Data and Annotation Peculiarities

Contact Info

Product

Resources

About