Utilizing Machine Learning for Detecting Harmful Situations by Audio and Text

Abstract

Children with special needs may struggle to identify uncomfortable and unsafe situations. In this study, we aimed at developing an automated system that can detect such situations based on audio and text cues to encourage children’s safety and prevent situations of violence toward them. We composed a text and audio database with over 1891 sentences extracted from videos presenting real-world situations, and categorized them into three classes: neutral sentences, insulting sentences, and sentences indicating unsafe conditions. We compared insulting and unsafe sentence-detection abilities of various machine-learning methods. In particular, we found that a deep neural network that accepts the text embedding vectors of bidirectional encoder representations from transformers (BERT) and audio embedding vectors of Wav2Vec as input attains the highest accuracy in detecting unsafe and insulting situations. Our results indicate that it may be applicable to build an automated agent that can detect unsafe and unpleasant situations that children with special needs may encounter, given the dialogue contexts conducted with these children.

Text-Based Detection of On-Hold Scripts in Contact Center Calls

“…Paper [23] uses the NLU model to classify each utterance of a transcribed call into a finite set of labels. In [24], the authors construct a system for children with special needs that detects insulting and harmful speech in the context of a dialogue. They classified such speech into three types of sentences: insulting, harmful, and neutral.…”

“…Paper [27] extensively reviews related methods. Moreover, both textual and audio features can be used, as described in [24,[28][29][30].…”

“…Following [24], we used BERT trained on the target language and fine-tuned it on our dataset using the "transformers" library [32]. We used DeepPavlov's "Conversational RuBERT" [33] because it best suits our domain.…”

One-Class Learning for AI-Generated Essay Detection

“…The proposed architecture was shown to be effective in learning rich semantic and contextual representations of news for the detection task. The authors in [16] devised a text and audio database with sentences extracted from videos presenting real-world situations and categorized them into three classes: neutral sentences, insulting sentences, and sentences representing unsafe conditions. A deep neural network was proposed to jointly consider text and audio embedding vectors, which resulted in accurate detections of unsafe and insulting situations.…”