Web-Based Automation Speech-to-Text Application using Audio Recording for Meeting Speech

Yasir, Muhammad; Nababan, Marlince; Laia, Yonata; Purba, Windania; Robin, .; Gea, Asaziduhu

doi:10.1088/1742-6596/1230/1/012081

Cited by 9 publications

(7 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Effective communication is vital to human society, and spoken language is the primary tool we use to convey our ideas and beliefs [9]. Crafting a reliable platform for converting speech into sign language requires a multitude of intricate steps, as elucidated in Section II.…”

Section: Methodsmentioning

confidence: 99%

“…The review highlights the specific areas that require attention. For example, speech recognition systems necessitate distinct enunciation [9], while sign language generation has lexicon constraints [6]. Avatars currently lack genuine emotional expression [7], and natural language processing encounters challenges with tokenization techniques.…”

Section: Related Workmentioning

confidence: 99%

“…2) Feature Extraction: In the feature extraction phase, each frame audio frequency feature is extracted. There are various techniques for feature extraction, some of them are Shorttime Fourier Transform (STFT) [13], Mel Frequency Cepstral Coefficients [9], Chroma feature [22], and Spectral Contrast [23]. Our Listen model leverages Mel Frequency Cepstral Coefficients (MFCC) to accurately capture audio features.…”

Section: A Speech To Textmentioning

confidence: 99%

“…Hidden Markov Models (HMMs) are commonly used to model the temporal dependencies in speech signals. HMMs consist of several matrices that define the model's parameters [9] [13]. The outputted text is then fed to the Subsection III-B for conversion of text using Natural Language Processing(NLP) which is explained next.…”

Section: A Speech To Textmentioning

confidence: 99%

See 3 more Smart Citations

Harnessing AI to Generate Indian Sign Language from Natural Speech and Text for Digital Inclusion and Accessibility

Yadav,

Sharma,

Khanna

et al. 2024

IJACSA

View full text Add to dashboard Cite

Sign language is the fundamental mode of communication for those who are deaf and mute, as well as for individuals with hearing impairments. Regrettably, there has been a dearth of research on Indian Sign Language, primarily due to the lack of adequate grammar and regional variations in such language. Consequently, research in this area has been limited. The primary objective of our research is to develop a sophisticated speech/ text-to-Indian sign language conversion system that employs advanced 3D modeling techniques to display sign language motions. Our research is motivated by our desire to promote effective communication between hearing and hearingimpaired individuals in India. The proposed model integrates Automatic Speech Recognition (ASR) technology, which effectively transforms spoken words into text, and leverages 3D modeling techniques to generate corresponding sign language motions. We have conducted a comprehensive study of the grammar of Indian Sign Language, which includes identifying sentence structure and signs that represent the tense of the subject. It is noteworthy that the sentence structure of Indian Sign Language follows the Subject-Object-Verb sequence, in contrast to spoken language, which follows the Subject-Verb-Object structure. To enhance user experience as well as digital inclusion and accessibility, the research incorporates user-friendly and simple interfaces that allow individuals to interact effortlessly with the system intuitively. The model/ system is equipped to receive speech input through a microphone/ text and provide immediate feedback through 3D-modeled videos that display the generated sign language gestures and has achieved 99.2% accuracy. Our main goal is to promote digital inclusion and improve accessibility and enhance the user experience.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: A Speech To Textmentioning

confidence: 99%

Section: A Speech To Textmentioning

confidence: 99%

See 2 more Smart Citations

Harnessing AI to Generate Indian Sign Language from Natural Speech and Text for Digital Inclusion and Accessibility

Yadav,

Sharma,

Khanna

et al. 2024

IJACSA

View full text Add to dashboard Cite

show abstract

“…audio sections with a single speaker label. Likewise, some approaches process the audio, turning it into text [35]. As overlapping speech can lead to miss detections or recognition errors, audio detection may not be efficient enough, and the visual detection of the person may lead to some improvements.…”

Section: Introductionmentioning

confidence: 99%

Audio-video fusion strategies for active speaker detection in meetings

Pibre¹,

Madrigal²,

Equoy³

et al. 2022

Preprint

View full text Add to dashboard Cite

Meetings are a common activity in professional contexts, and it remains challenging to endow vocal assistants with advanced functionalities to facilitate meeting management. In this context, a task like active speaker detection can provide useful insights to model interaction between meeting participants. Detection of the active speaker can be performed using only video based on the movements of the participants of a meeting. Depending on the assistant design and each participant position regarding the device, active speaker detection can benefit from information coming from visual and audio modalities. Motivated by our application context related to advanced meeting assistant, we want to combine audio and visual information to achieve the best possible performance. In this paper, we propose two different types of fusion (naive fusion and attention-based fusion) for the detection of the active speaker, combining two visual modalities and an audio modality through neural networks. In addition, the audio modality is mainly processed using neural networks. For comparison purpose, classical unsupervised approaches for audio feature extraction are also used. We expect visual data centered on the face of each participant to be very appropriate for detecting voice activity, based on the detection of lip and facial gestures. Thus, our baseline system uses visual data (video) and we chose a 3D Convolutional Neural Network (CNN) architecture, which is effective for simultaneously encoding appearance and movement. To improve this system, we supplemented the visual information by processing the audio stream with a CNN or an unsupervised speaker diarization system. We have further improved this system by adding visual modality information using motion through optical flow. We evaluated our proposal with a public and state-ofthe-art benchmark: the AMI corpus. We analysed the contribution of each system to the merger carried out in order to determine if a 1

show abstract

From Text to Voice: A Comparative Study of Machine Learning Techniques for Podcast Synthesis

Chandre,

Vanarote,

Mande

et al. 2024

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

Web-Based Automation Speech-to-Text Application using Audio Recording for Meeting Speech

Cited by 9 publications

References 3 publications

Harnessing AI to Generate Indian Sign Language from Natural Speech and Text for Digital Inclusion and Accessibility

Harnessing AI to Generate Indian Sign Language from Natural Speech and Text for Digital Inclusion and Accessibility

Audio-video fusion strategies for active speaker detection in meetings

From Text to Voice: A Comparative Study of Machine Learning Techniques for Podcast Synthesis

Contact Info

Product

Resources

About