Summarizing User-generated Textual Content: Motivation and Methods for Fairness in Algorithmic Summaries

Abstract

As the amount of user-generated textual content grows rapidly, text summarization algorithms are increasingly being used to provide users a quick overview of the information content. Traditionally, summarization algorithms have been evaluated only based on how well they match human-written summaries (e.g. as measured by ROUGE scores). In this work, we propose to evaluate summarization algorithms from a completely new perspective that is important when the user-generated data to be summarized comes from different socially salient user groups, e.g. men or women, Caucasians or African-Americans, or different political groups (Republicans or Democrats). In such cases, we check whether the generated summaries fairly represent these different social groups. Specifically, considering that an extractive summarization algorithm selects a subset of the textual units (e.g. microblogs) in the original data for inclusion in the summary, we investigate whether this selection is fair or not. Our experiments over real-world microblog datasets show that existing summarization algorithms often represent the socially salient user-groups very differently compared to their distributions in the original data. More importantly, some groups are frequently under-represented in the generated summaries, and hence get far less exposure than what they would have obtained in the original data. To reduce such adverse impacts, we propose novel fairness-preserving summarization algorithms which produce high-quality summaries while ensuring fairness among various groups. To our knowledge, this is the first attempt to produce fair text summarization, and is likely to open up an interesting research direction.

Fair and Representative Subset Selection from Data Streams

“…Celis et al [5] proposed a determinantal point process (DPP) based sampling method for fair data summarization. Dash et al [11] considered the fairness issue on summarizing user-generated textual content. Although these studies adopt similar definitions of fairness constraints to ours, their proposed methods cannot be applied to the FSM problem since the objective functions of the problems they study are not submodular.…”

“…Despite the extensive studies on streaming submodular maximization, unfortunately, it seems that none of the existing methods consider the fairness issue of the subsets extracted from data streams. In fact, recent studies [5,10,11,20] reveal that data summaries automatically generated by algorithms might be biased with respect to sensitive attributes such as gender, race, or ethnicity, and the biases in summaries could be passed to data-driven decisionmaking processes in education, recruitment, banking, and judiciary systems. Thus, it is necessary to introduce fairness constraints into submodular maximization problems so that the selected subset can fairly represent each sensitive attribute in the dataset.…”

When the Umpire is also a Player: Bias in Private Label Product Recommendations on E-commerce Marketplaces

“…We believe any slight variation to 𝜖 will not change our observations significantly. While this threshold has been used in multiple prior studies [13,18], we acknowledge that the choice remains context dependent and can change based on the application and prior established regulations.…”

Fairness for Whom? Understanding the Reader's Perception of Fairness in Text Summarization

“…Facebook, Twitter, user-generated content constitutes a large chunk of the textual information generated on the Web today. On social media, different user groups discuss different socio-political issues, and it has been observed that they often have very different opinions on the same topic or event [3], [4]. Hence, the textual information to be summarised has gradually become heterogeneous.…”

“…Hence, the textual information to be summarised has gradually become heterogeneous. In our prior work [3], we have shown that such text often contains very different opinions from people of different ideologies, social groups, etc. In many downstream applications, algorithm-generated summaries are consumed by people and hence they often play a vital role in shaping their opinion in different socio-political issues.…”