Facebook as an Avenue to News: A Comparison and Validation of Approaches to Identify Facebook Referrals

Abstract

Given that Facebook is still the most widely used social networking site in the world, its influence on democratic processes is under constant scrutiny. Academics have put a special focus on Facebook's role in inhibiting or enhancing citizens' news exposure. Recent studies using digital behavioral data have analyzed the prevalence and effects of "Facebook news referrals". Using a web tracking tool that captures general browsing behavior as well as public posts seen on Facebook, this paper lays the groundwork for the field by assessing the validity of previously proposed operationalizations. We validate news referrals by investigating whether different measures actually reflect exposure to a news URL a user saw on Facebook. We furthermore assess the effects of news referrals on central outcomes in extant literature, contingent on different operationalizations. The results show that the most precise measure of news referrals are referral IDs attached as parameters to news URLs by Facebook. Still, the substantive findings are broadly comparable across approaches, lending further credibility to published research, despite measurement errors. The paper demonstrates the need for academics to constantly innovate in order to measure citizens' online behavior in an ecologically valid manner.