Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond

Abstract

A fundamental goal of scientific research is to learn about causal relationships. However, despite its critical role in the life and social sciences, causality has not had the same importance in Natural Language Processing (NLP), which has traditionally placed more emphasis on predictive tasks. This distinction is beginning to fade, with an emerging area of interdisciplinary research at the convergence of causal inference and language processing. Still, research on causality in NLP remains scattered across domains without unified definitions, benchmark datasets and clear articulations of the challenges and opportunities in the application of causal inference to the textual domain, with its unique properties. In this survey, we consolidate research across academic areas and situate it in the broader NLP landscape. We introduce the statistical challenge of estimating causal effects with text, encompassing settings where text is used as an outcome, treatment, or to address confounding. In addition, we explore potential uses of causal inference to improve the robustness, fairness, and interpretability of NLP models. We thus provide a unified overview of causal inference for the NLP community.1

From Text Signals to Simulations: A Review and Complement to Text as Data by Grimmer, Roberts & Stewart (PUP 2022)

“…Grimmer, Roberts and Stewarts’ chapter on causal analysis highlights critical principles for causal identification and articulates challenges associated with the accuracy of causal modeling assumptions, like the Stable Unit Treatment Value Assumption (SUTVA), which requires that the response of a particular unit depends on the treatment to which it was assigned, not the treatments of those around it. They emphasize the importance of experiments, which they illustrate with some of their own pathbreaking work in this area (Egami et al 2018; Grimmer 2015). I argue below that modern transformer methods could contribute most to this area, by enabling the automated contruction of text manifesting precise semantic variation with generative models, and the simulation of text understanding and reception with encoding models.…”

“…Perhaps the most unique contribution from Text as Data is in the penultimate section on inference, or drawing reasoned insights from observed text to that which is unobserved (e.g., written in the future, from other places or for other purposes). The authors, leaders in causal analysis with text (Feder et al 2021), clearly distinguish between prediction, or the anticipation of future events, and causal inference that requires conditional or counterfactual prediction. Their principles of prediction are clear and elegant (e.g., "predictive features do not have to cause the outcome", "it can be difficult to apply prediction to policy-making").…”

“…The book directly addresses concerns held by social scientists, humanists, and data scientists seeking to use established natural language processing techniques to enrich their descriptive and causal analyses. The book comes from three top research performers who have individually and together published leading work with these methods on topics that range across several areas of American and international politics (Feder et al 2021; Grimmer 2009; Grimmer and Stewart 2013; King, Pan, and Roberts 2013). With these methods, they have generated and validated a number of new insights regarding the character, causes and consequences of political communication that demonstrate the power of text as data to enrich many fields of social inquiry.…”

Continual Causal Effect Estimation: Challenges and Opportunities

“…In addition, the combination of causal inference and other research fields also exhibits complementary strengths, such as computer vision (Tang et al 2020;Liu et al 2022a), graph learning (Ma et al 2022;Chu, Rathbun, and Li 2021), natural language processing (Feder et al 2022;Liu et al 2022b), and so on. The involved causal analysis helps improve the model's capability of discovering and resolving the underlying system beyond the statistical relationships learned from observational data.…”

Seeded Topic Models in Digital Archives: Analyzing the Swedish Understanding of Immigration, 1945--2019

“…We demonstrate that widely recognized, emotion-laden events have immediate but no lasting effects on interpretations of immigration and that it takes prolonged periods of rupture to change them. This event analysis exemplifies how researchers can use the output of computational text models to trace causal claims within, for example, the traditional regression model framework (Egami et al 2018, Roberts et al 2020, Fong & Grimmer 2019, Feder et al 2021, Gencoglu & Gruber 2020.…”