EmpathBERT: A BERT-based Framework for Demographic-aware Empathy Prediction

Abstract

Affect preferences vary with user demographics, and tapping into demographic information provides important cues about the users' language preferences. In this paper, we utilize the user demographics, and propose EMPATH-BERT, a demographic-aware framework for empathy prediction based on BERT. Through several comparative experiments, we show that EMPATHBERT surpasses traditional machine learning and deep learning models, and illustrate the importance of user demographics to predict empathy and distress in user responses to stimulative news articles. We also highlight the importance of affect information in the responses by developing affect-aware models to predict user demographic attributes.

Toward Transformer Fusions for Chinese Sentiment Intensity Prediction in Valence-Arousal Dimensions

“…The RoBERTa multi-task model and the vanilla ELECTRA model was combined to predict empathy scores [36]. A demographic-aware EmpathBERT architecture was presented to infuse demographic information for empathy prediction [37]. The BERT transformer was used to recognize the emotion intensity scores of Japanese tweets on the topics of vaccinations [38].…”

“…This section describes the existing methods for sentiment intensity prediction, including lexicon-based [4,[10][11][12][13][14][15][16], regression-based [17][18][19][20][21][22], neural-network-based [23][24][25][26][27][28][29][30][31][32][33] and transformer-based [34][35][36][37][38][39][40][41][42][43][49][50][51] approaches.…”

“…In general, sentiment intensity prediction methods can be summarized into four types: lexicon-based [4,[10][11][12][13][14][15][16], regression-based [17][18][19][20][21][22], neural-network-based [23][24][25][26][27][28][29][30][31][32][33], and transformer-based [34][35][36][37][38][39][40][41][42][43]. The lexicon-based approaches provide baseline results for reference, while transformer-based models have usually achieved promising results in the valence-arousal dimensions [9].…”

Nationality Bias in Text Generation

“…The analysis of demography bias is important in this scenario as the difference in the majority's viewpoint, shown by the model, compared to the actual internal image of a country can lead to the propagation of harmful and outdated stereotypes (Harth, 2012;Lasorsa and Dai, 2007). Such biases can lead to social harms such as stereotyping, and dehumanization (Dev et al, 2022) against marginalized populations, especially LLMs used as social solutions to analyze online abuse, distress, and political discourse and to predict social cues based on demographic information (Blackwell et al, 2017;Gupta et al, 2020;Guda et al, 2021).…”

Can Demographic Factors Improve Text Classification? Revisiting Demographic Adaptation in the Age of Transformers

“…Accordingly, several studies have leveraged demographic information (e.g., gender, age, education) to investigate the effect of encoded sociodemographic knowledge in the representations of PLMs or obtain better language representations for various NLP tasks (Volkova et al, 2013;Garimella et al, 2017). Recent research studies on demographic adaptation mainly focus on (1) learning demographicaware word embeddings and do not work with large PLMs (Hovy, 2015) or (2) leveraging demographic information with special PLM architectures specifically designed for certain downstream tasks (e.g., empathy prediction (Guda et al, 2021)). The latter, however, do not consider a task-agnostic approach to injecting demographic knowledge into language models, and also focus on a monolingual setup only.…”