Using Temporal Association Rules for the Synthesis of Embodied Conversational Agents with a Specific Stance

Janssoone, Thomas; Clavel, Chloé; Bailly, Kévin; Richard, Gaël

doi:10.1007/978-3-319-47665-0_16

Cited by 13 publications

(8 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More complex metrics can be explored based on the outcomes of traditional ARM. Recent work such as temporal association rule mining [44,85] and graph association rule mining [14,60], have the potential to take temporal information into account. In addition, sequential pattern mining [15] and sequential rule mining [34] can also be employed to investigate temporal sequences or behavior sequences.…”

Section: Leveraging Association Rule Mining and Other Algorithmsmentioning

confidence: 99%

Leveraging Routine Behavior and Contextually-Filtered Features for Depression Detection among College Students

Chikersal

Doryab

et al. 2019

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

The rate of depression in college students is rising, which is known to increase suicide risk, lower academic performance and double the likelihood of dropping out of school. Existing work on fnding relationships between passively sensed behavior and depression, as well as detecting depression, mainly derives relevant unimodal features from a single sensor. However, co-occurrence of values in multiple sensors may provide better features, because such features can describe behavior in context. We present a new method to extract contextually fltered features from passively collected, time-series mobile data via association rule mining. After calculating traditional unimodal features from the data, we extract rules that relate unimodal features to each other using association rule mining. We extract rules from each class separately (e.g., depression vs. nondepression). We introduce a new metric to select a subset of rules that distinguish between the two classes. From these rules, which capture the relationship between multiple unimodal features, we automatically extract contextually fltered features. These features are then fed into a traditional machine learning pipeline to detect the class of interest (in our case, depression), defned by whether a student has a high BDI-II score at the end of the semester. The behavior rules generated by our methods are highly interpretable representations of diferences between classes. Our best model uses contextually-fltered features to signifcantly outperform a standard model that uses only unimodal features, by an average of 9.7% across a variety of metrics. We further verifed the generalizability of our approach on a second dataset, and achieved very similar results. CCS Concepts: • Human-centered computing Ubiquitous and mobile computing; • Applied computing Life and medical sciences.

show abstract

Section: Leveraging Association Rule Mining and Other Algorithmsmentioning

confidence: 99%

Leveraging Routine Behavior and Contextually-Filtered Features for Depression Detection among College Students

Chikersal

Doryab

et al. 2019

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

show abstract

“…To measure the perception of attitudes, previous researches relied on either IAS [36], [64], [29], [42], [32] or ICL [65]. In our work, we choose to use a combination of both IAS and ICL.…”

Section: Methodsmentioning

confidence: 99%

“…For example, this patterns the tutor violates social norms while being gazed at by the tutee, and their speech overlaps within the next minute characterizes a decrease in interpersonal rapport. The TITARL algorithm has also been used in [42] to extract temporal association rules related to attitude from the SEMAINE database [43]. More precisely, Janssoone et al investigated the correlation between nonverbal behavior (like eyebrow movements and prosody), and two attitudes: friendliness and hostility.…”

Section: Sequence-based Multimodal Behavior Modelingmentioning

confidence: 99%

Leveraging the Dynamics of Non-Verbal Behaviors For Social Attitude Modeling

Dermouche¹,

Pélachaud²

2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

An Embodied Conversational Agent (ECA) is a virtual character designed to interact with humans in the most natural way. In the recent years, ECAs have been deployed in various contexts, such as commercial consulting and social training. In the context of social training, the virtual agent should be able to express different social attitudes in order to train the user in different situations, likely to occur in real life. Previous studies from psychology underlined the importance of considering the non-verbal behavior as well as its evolution over time, for efficient modeling of interpersonal attitudes. Inspired by these works as well as by advances from sequence mining, we propose to model attitude variation as a sequence of non-verbal signals, each being described by its starting time and duration. We demonstrate the efficiency of our model by integrating the sequences representing attitude variation in an ECA and assessing the obtained results based on the interpersonal circumplex, statistical tests and accuracy measures. To the best of our knowledge, this is the first attempt to study the relationship, in term of perception, between different attitude variations.

show abstract

“…mean, standard deviation, etc). However, the lack of sequence modeling can lead to the loss of some important social signals such as emphasis by raising one's eyebrows followed by a smile (Janssoone et al 2016). Moreover co-occurrences of events are not captured by this representation.…”

Section: Machine Learning Approaches For Automatic Analysis Of Video mentioning

confidence: 99%

HireNet: A Hierarchical Attention Model for the Automatic Analysis of Asynchronous Video Job Interviews

Hemamou

Felhi

Vandenbussche

et al. 2019

AAAI

Self Cite

View full text Add to dashboard Cite

New technologies drastically change recruitment techniques. Some research projects aim at designing interactive systems that help candidates practice job interviews. Other studies aim at the automatic detection of social signals (e.g. smile, turn of speech, etc...) in videos of job interviews. These studies are limited with respect to the number of interviews they process, but also by the fact that they only analyze simulated job interviews (e.g. students pretending to apply for a fake position). Asynchronous video interviewing tools have become mature products on the human resources market, and thus, a popular step in the recruitment process. As part of a project to help recruiters, we collected a corpus of more than 7000 candidates having asynchronous video job interviews for real positions and recording videos of themselves answering a set of questions. We propose a new hierarchical attention model called HireNet that aims at predicting the hirability of the candidates as evaluated by recruiters. In HireNet, an interview is considered as a sequence of questions and answers containing salient socials signals. Two contextual sources of information are modeled in HireNet: the words contained in the question and in the job position. Our model achieves better F1-scores than previous approaches for each modality (verbal content, audio and video). Results from early and late multimodal fusion suggest that more sophisticated fusion schemes are needed to improve on the monomodal results. Finally, some examples of moments captured by the attention mechanisms suggest our model could potentially be used to help finding key moments in an asynchronous job interview.

show abstract

Using Temporal Association Rules for the Synthesis of Embodied Conversational Agents with a Specific Stance

Cited by 13 publications

References 20 publications

Leveraging Routine Behavior and Contextually-Filtered Features for Depression Detection among College Students

Leveraging Routine Behavior and Contextually-Filtered Features for Depression Detection among College Students

Leveraging the Dynamics of Non-Verbal Behaviors For Social Attitude Modeling

HireNet: A Hierarchical Attention Model for the Automatic Analysis of Asynchronous Video Job Interviews

Contact Info

Product

Resources

About