Examining Analytic Practices in Latent Dirichlet Allocation Within Psychological Science: Scoping Review

Abstract

Background Topic modeling approaches allow researchers to analyze and represent written texts. One of the commonly used approaches in psychology is latent Dirichlet allocation (LDA), which is used for rapidly synthesizing patterns of text within “big data,” but outputs can be sensitive to decisions made during the analytic pipeline and may not be suitable for certain scenarios such as short texts, and we highlight resources for alternative approaches. This review focuses on the complex analytical practices specific to LDA, which existing practical guides for training LDA models have not addressed. Objective This scoping review used key analytical steps (data selection, data preprocessing, and data analysis) as a framework to understand the methodological approaches being used in psychology research using LDA. Methods A total of 4 psychology and health databases were searched. Studies were included if they used LDA to analyze written words and focused on a psychological construct or issue. The data charting processes were constructed and employed based on common data selection, preprocessing, and data analysis steps. Results A total of 68 studies were included. These studies explored a range of research areas and mostly sourced their data from social media platforms. Although some studies reported on preprocessing and data analysis steps taken, most studies did not provide sufficient detail for reproducibility. Furthermore, the debate surrounding the necessity of certain preprocessing and data analysis steps is revealed. Conclusions Our findings highlight the growing use of LDA in psychological science. However, there is a need to improve analytical reporting standards and identify comprehensive and evidence-based best practice recommendations. To work toward this, we developed an LDA Preferred Reporting Checklist that will allow for consistent documentation of LDA analytic decisions and reproducible research outcomes.

Using machine learning and topic modeling to examine manifestos of violent attackers.

“…The exploration of concepts such as semantic meaning, syntactic complexity, and text length has been a focal point in psychology, linguistics, and, to a certain extent, in forensic linguistics as well. This line of inquiry has focused on analyzing communication patterns for varied purposes, from comprehending language processing disorders and investigating the interplay between linguistic processes and cognitive functions to delving into the complexities of cognitive processes that guide behavior (Bradley & Meeds, 2002; Hagg et al, 2022; Müller & Schwarz, 2021; Simons & Skillicorn, 2020). The evolution of technology, particularly the advent of the internet and social media, has led to a significant expansion in the volume, speed, and reach of information (Alam & Kabir, 2023; Mir et al, 2023; Pasquini et al, 2021).…”

A Statistical Approach for Quantifying Group Difference in Topic Distributions Using Clinical Discourse Samples

“…The application of topic modeling methods in clinical research has become more common in recent years (Hagg et al, 2022;Boyd-Graber et al, 2017;Jelodar et al, 2019). While topic modeling approaches have advanced significantly over the last twenty years, evaluation methods have lagged behind (see Hoyle et al, 2021 for a recent survey of methods).…”