Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models

Abstract

Targeted syntactic evaluations have demonstrated the ability of language models to perform subject-verb agreement given difficult contexts. To elucidate the mechanisms by which the models accomplish this behavior, this study applies causal mediation analysis to pre-trained neural language models. We investigate the magnitude of models' preferences for grammatical inflections, as well as whether neurons process subject-verb agreement similarly across sentences with different syntactic structures. We uncover similarities and differences across architectures and model sizesnotably, that larger models do not necessarily learn stronger preferences. We also observe two distinct mechanisms for producing subject-verb agreement depending on the syntactic structure of the input sentence. Finally, we find that language models rely on similar sets of neurons when given sentences with similar syntactic structure. * Equal contribution. † Work done while visiting Google Research. ‡ Supported by the Viterbi Fellowship in the Center for Computer Engineering at the Technion.

A Causal View of Entity Bias in (Large) Language Models

“…In this context, utilize total direct effect and total effect to debias the relation extraction. Apart from debiasing, causal mediation analysis can be used to analyze biases in LLMs (Vig et al, 2020;Finlayson et al, 2021).…”