PANDORA Talks: Personality and Demographics on Reddit

Abstract

Personality and demographics are important variables in social sciences, whilein NLP they can aid in interpretability and removal of societal biases.However, datasets with both personality and demographic labels are scarce. Toaddress this, we present PANDORA, the first large-scale dataset of Reddit commentslabeled with three personality models (including the well-established Big 5 model) and demographics (age, gender, and location) for more than 10k users. Weshowcase the usefulness of this dataset on three experiments, where we leveragethe more readily available data from other personality models to predict theBig 5 traits, analyze gender classification biases arising frompsycho-demographic variables, and carry out a confirmatory and exploratoryanalysis based on psychological theories. Finally, we present benchmarkprediction models for all personality and demographic variables.

Psycholinguistic Tripartite Graph Network for Personality Detection

“…Jiang et al (2020) simply concatenate all the utterances from a single user into a document and encode it with BERT (Devlin et al, 2019) and RoBERTa (Liu et al, 2019). Gjurković et al (2020) first encode each post by BERT and then use CNN (LeCun et al, 1998) to aggregate the post representations. Most of them focus on how to obtain more effective contextual representations, with only several exceptions that try to introduce psycholinguistic features into DNNs, such as Majumder et al (2017) and Xue et al (2018).…”

“…Personality detection can be formulated as a multidocument multi-label classification task (Lynn et al, 2020;Gjurković et al, 2020). Formally, each user has a set P = {p 1 , p 2 , .…”

“…We choose two public MBTI datasets for evaluations, which have been widely used in recent studies (Tadesse et al, 2018;Hernandez and Knight, 2017;Majumder et al, 2017;Jiang et al, 2020;Gjurković et al, 2020). The Kaggle dataset 4 is collected from PersonalityCafe, 5 where people share their personality types and discussions about health, behavior, care, etc.…”

“…As a major psycholinguistic instrument, Linguistic Inquiry and Word Count (LIWC) (Tausczik and Pennebaker, 2010) divides words into psychologically relevant categories (e.g., Function, Affect, and Social as shown in Figure 1) and is commonly used to extract psycholinguistic features in conventional methods (Golbeck et al, 2011;Sumner et al, 2012). Nevertheless, most recent works (Hernandez and Knight, 2017;Jiang et al, 2020;Keh et al, 2019;Lynn et al, 2020;Gjurković et al, 2020) tend to adopt deep neural networks (DNNs) to represent the posts and build predictive models in a data-driven manner. They first encode each post separately and then aggregate the post representations into a user representation.…”

How to Obtain Reliable Labels for MBTI Classification from Texts?

“…To detect MBTI typologies in a more natural way and without necessity for trained human assessors, many studies have attempted at building systems for automatic detection of MBTI personality types from text in the last several years. Attempts have been made for automatic detection of MBTI personality types from: tweets written in English (Plank and Hovy, 2015), six other Western European languages (Ver-hoeven et al, 2016), and Japanese (Yamada et al, 2019); English posts collected from Personality Cafe forum 6 available in Kaggle; 7 and English Reddit comments (Gjurković and Šnajder, 2018;Gjurković et al, 2020). Despite being trained on large amounts of textual data (over one million), and modelled as four binary classification tasks, the best systems performed only slightly better than the random and majority-class baselines, regardless of the architecture used.…”