The Meaning Extraction Method: An Approach to Evaluate Content Patterns From Large-Scale Language Data

Abstract

Qualitative content analyses often rely on a top-down approach to understand themes in a collection of texts. A codebook prescribes how humans should judge if a text fits a theme based on rules and judgment criteria. Qualitative approaches are challenging because they require many resources (e.g., coders, training, rounds of coding), can be affected by researcher or coder bias, may miss meaningful patterns that deviate from the codebook, and often use a subsample of the data. A complementary, bottom-up approach—the Meaning Extraction Method—has been popular in social psychology but rarely applied to communication research. This paper outlines the value of the Meaning Extraction Method, concluding with a guide to conduct analyses of content and themes from massive and complete datasets, quantitatively. The Meaning Extraction Method is performed on a public and published archive of pet adoption profiles to demonstrate the approach. Considerations for communication research are offered.

Using crowdsourced medicine to manage uncertainty on Reddit: The case of COVID-19 long-haulers

“…As part of a larger study of COVID-19 long-haulers on Reddit, a topic modeling text analysis approach called the Meaning Extraction Method [28] was used to identify groups of words that mathematically group together across a number of text observations Reddit. Similar to other topic modeling approaches, such as Latent Dirichlet Allocation (LDA), researchers have used the MEM to glean information about people’s communication patterns [29] , [30] , romantic relationships [31] , health behaviors [32] and psychological health [33] . Given that technology is providing researchers access to large amounts of organic textual data, a software system called the Meaning Extraction Helper program was created as an automated companion tool for MEM [34] and we used it to perform topic modeling analyses on natural language extracted from the Reddit community (i.e., subreddit) r/covidlonghaulers.…”

“…Given that technology is providing researchers access to large amounts of organic textual data, a software system called the Meaning Extraction Helper program was created as an automated companion tool for MEM [34] and we used it to perform topic modeling analyses on natural language extracted from the Reddit community (i.e., subreddit) r/covidlonghaulers. The MEM first identifies the most common content (e.g., nouns, verbs) words across a corpus per user while simultaneously removing low-frequency words and stop words (i.e., function words, low base rate words) [30] . A percentage of use score is calculated for each text observation by dividing the number of times an n-gram appears in their text divided by total word count [34] .…”

Age-Related Differences in Experiences With Social Distancing at the Onset of the COVID-19 Pandemic: A Computational and Content Analytic Investigation of Natural Language From a Social Media Survey

“…LIWC counts the number of words in a variety of psychological (eg, positive or negative emotion terms), topical (eg, family-related terms, work-related terms), and part of speech (eg, pronouns, adverbs) categories that appear in a given text relative to all the words in that text. To further explore topical focus in people's descriptions of the impact of COVID-19, we identified themes in open-ended responses for each age group using the meaning extraction method, which relies on principal component analysis (PCA) of content words in language corpuses [16]. Data were processed with the Meaning Extraction Helper software to remove function words (ie, prepositions) and words with low base rates (present in <5% of responses), and calculate whether content words (ie, nouns, verbs) were present (coded as "1") or absent (coded as "0") within a response [17].…”

“…The extraction of qualitative themes from participants' responses through the meaning extraction method allowed us to gain deeper insight into what topics people of different age groups focused on during the pandemic onset. For each of the five factors analyzed, content words were retained if their loadings were over or equal to the absolute value of .30 [16]. As seen in Table 1, people of different age groups focused on distinct aspects of their experiences.…”

How Experts React: The World Health Organization’s Appraisal of COVID-19 via Communication Patterns

“…2. Note, the Meaning Extraction Helper collapses terms into their "basic form" when counting words (Markowitz, 2021). Therefore, results for the word country in Table 1 would also capture the word countries, and the word support would capture the words supported, supports, and supporting as well.…”