Analysis of Multimodal Features for Speaking Proficiency Scoring in an Interview Dialogue

Saeki, Mao; Matsuyama, Yoichi; Kobashikawa, Satoshi; Ogawa, Tetsuji; Kobayashi, Tetsuo

doi:10.1109/slt48900.2021.9383590

Cited by 6 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, a Softmax classifier is used to classify and identify the test sample image's expression by carefully combining the Fer-2013 facial expression database with the LFW data set; a simulation experiment is created to test the resilience of this approach for facial expression identification against a complicated context. According to [7] A language learner's conversational ability may be assessed using multimodal activities, including speech content, prosody, and visual cues. Although linguistic and auditory components have been well studied, less attention has been paid to visual factors, including eye contact and facial expressions.…”

Section: IImentioning

confidence: 99%

Hybrid Features Extraction for Emotion Detection using Deep Convolutional Neural Network

Rathod¹,

Musale²,

Haribhakta³

et al. 2023

IJRASET

View full text Add to dashboard Cite

These Human facial expressions communicate much information more visually than verbally. Recognition of facial expressions is essential for human-machine interaction. Applications for automatic facial expression recognition include but are not limited to, comprehending human behaviour, spotting mental illnesses, and creating artificial human emotions. Humancomputer interaction, autonomous cars, and a wide range of multimedia applications all rely heavily on facial expression detection. In this study, we present a reusable architecture for recognizing human emotions based on facial expressions. Two machine learning algorithms that may be learned in advance for use in real-time applications make up the framework. At first, we use AdaBoost cascade classifiers to find instances of faces in the photos. The extracted neighborhood difference characteristics then serve as a representation of a face's features according to its localized appearance data. Instead of only looking at intensity data, the NDF predicts multiple patterns depending on the connections between surrounding areas. Despite a high identification rate, facial expression recognition by computers remains difficult. Based on geometry and appearance, two widely used approaches for automated FER systems are often used in the literature. Pre-processing, face detection, feature extraction, and expression classification are the four steps that typically make up facial expression recognition. Various deep learning techniques (convolutional neural networks) were used in this effort to recognize the primary seven human emotions: anger, fear, happiness, surprise, and neutrality

show abstract

Section: IImentioning

confidence: 99%

Hybrid Features Extraction for Emotion Detection using Deep Convolutional Neural Network

Rathod¹,

Musale²,

Haribhakta³

et al. 2023

IJRASET

View full text Add to dashboard Cite

show abstract

“…Butt [7] 28 EEG, GSR, PPG Chen [8] 14 Lexical, Speech, Visual Saeki [10] 210 Lexical, Speech, Visual Suen [30] 120 Visual Hsiao [31] 128 Lexical, Speech, Visual Xu [35] 13,347 Visual Wörtwein [43] 45 Speech, Visual Ramanarayanan [44] 24 Speech, Visual Rasipuram [45] 106 Lexical, Speech, Visual Gavrilescu [46] 128 Visual Giritlio glu [47] 60 Lexical, Speech, Visual…”

Section: Author Participants Classes Of Human Behavior Signalmentioning

confidence: 99%

“…More recently, several multimodal sensing systems were directed at developing applications for personality trait recognition, such as for job interviews [4,5], work stress tests [6], public speaking [7,8], consumer behavior [9], and verbal rating systems [10]. These applications achieved good results in various fields in terms of analyzing personality traits using individual behavioral characteristics.…”

Section: Introductionmentioning

confidence: 99%

A Multimodal Affective Sensing Model for Constructing a Personality-Based Financial Advisor System

Lee

Yang²,

Su³

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

To achieve successful investments, in addition to financial expertise and knowledge of market information, a further critical factor is an individual’s personality. Decisive people tend to be able to quickly judge when to invest, while calm people can analyze the current situation more carefully and make appropriate decisions. Therefore, in this study, we developed a multimodal personality-recognition system to understand investors’ personality traits. The system analyzes the personality traits of investors when they share their investment experiences and plans, allowing them to understand their own personality traits before investing. To perform system functions, we collected digital human behavior data through video-recording devices and extracted human behavior features using video, speech, and text data. We then used data fusion to fuse human behavior features from heterogeneous data to address the problem of learning only one-sided information from a single modality. Through several experiments, we demonstrated that multimodal (i.e., three different signal inputs) personality trait analysis is more accurate than unimodal models. We also used statistical methods and questionnaires to evaluate the correlation between the investor’s personality traits and risk tolerance. It was found that investors with higher openness, extraversion, and lower neuroticism personality traits took higher risks, which is similar to research findings in the field of behavioral finance. Experimental results show that, in a case study, our multimodal personality prediction system exhibits high performance with highly accurate prediction scores in various metrics.

show abstract

A WoZ Study for an Incremental Proficiency Scoring Interview Agent Eliciting Ratable Samples

Saeki

Demkow

Kobayashi

et al. 2022

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

Analysis of Multimodal Features for Speaking Proficiency Scoring in an Interview Dialogue

Cited by 6 publications

References 14 publications

Hybrid Features Extraction for Emotion Detection using Deep Convolutional Neural Network

Hybrid Features Extraction for Emotion Detection using Deep Convolutional Neural Network

A Multimodal Affective Sensing Model for Constructing a Personality-Based Financial Advisor System

A WoZ Study for an Incremental Proficiency Scoring Interview Agent Eliciting Ratable Samples

Contact Info

Product

Resources

About