The Engagement-Diversity Connection: Evidence from a Field Experiment on Spotify

Abstract

It remains unknown whether personalized recommendations increase or decrease the diversity of content people consume. We present results from a randomized field experiment on Spotify testing the effect of personalized recommendations on consumption diversity. In the experiment, both control and treatment users were given podcast recommendations, with the sole aim of increasing podcast consumption. Treatment users' recommendations were personalized based on their music listening history, whereas control users were recommended popular podcasts among users in their demographic group. We find that, on average, the treatment increased podcast streams by 28.90%. However, the treatment also decreased the average individual-level diversity of podcast streams by 11.51%, and increased the aggregate diversity of podcast streams by 5.96%, indicating that personalized recommendations have the potential to create patterns of consumption that are homogenous within and diverse across users, a pattern reflecting Balkanization. Our results provide evidence of an "engagement-diversity trade-off" when recommendations are optimized solely to drive consumption: while personalized recommendations increase user engagement, they also affect the diversity of consumed content. This shift in consumption diversity can affect user retention and lifetime value, and impact the optimal strategy for content producers. We also observe evidence that our treatment affected streams from sections of Spotify's app not directly affected by the experiment, suggesting that exposure to personalized recommendations can affect the content that users consume organically. We believe these findings highlight the need for academics and practitioners to continue investing in personalization methods that explicitly take into account the diversity of content recommended.

The music that people use to sleep: universal and subgroup characteristics

“…These features describe not only basic features of recorded music, such as its tempo and loudness, but also compound measures indicative of, for instance, a particular track's Danceability and Acousticness. This allows us to use Spotify as a platform for investigating sleep music in a representative industrialised population 27,28 .…”

Understanding citizen engagement and concerns about the COVID-19 pandemic in China: a thematic analysis of government social media

“…The average number of likes received on each user's comments is regarded as an indicator of information amplification. Holtz et al (2020) found that personalized recommendation increase user engagement after finishing a field experiment on Spotify. Thematic analysis is used to analyze engagement behavior on social platforms, and some content topics have been concluded.…”

Social Influence Leads to the Formation of Diverse Local Trends

“…Recent work has shed light on design features that moderate the relationship between social influence and diversity. In the context of music listening, Holtz et al [28] found that personalized recommendation decreased diversity within users, but increased diversity across users. Pescetelli et al [50] found that increased diversity increases collective intelligence for large groups (≈ 25 people) but decreases it for small groups (≈ 5 people).…”

“…Social influence occurs when the decisions of a user are impacted by those of other users [10], and has been shown to be a key design dimension for contexts as varied as health behavior [9], political engagement [5] , collective behavior [41], online book purchasing [8], food ordering [30], and digital news engagement [46,62]. The ubiquity of social influence suggests how crucial a factor it is for platform designers seeking to jointly optimize for the quality and diversity of content online [28].…”