Automatic identification of personal insults on social news sites

Abstract

As online communities grow and the volume of usergenerated content increases, the need for community management also rises. Community management has three main purposes: to create a positive experience for existing participants, to promote appropriate, socionormative behaviors, and to encourage potential participants to make contributions. Research indicates that the quality of content a potential participant sees on a site is highly influential; off-topic, negative comments with malicious intent are a particularly strong boundary to participation or set the tone for encouraging similar contributions. A problem for community managers, therefore, is the detection and elimination of such undesirable content. As a community grows, this undertaking becomes more daunting. Can an automated system aid community managers in this task? In this paper, we address this question through a machine learning approach to automatic detection of inappropriate negative user contributions. Our training corpus is a set of comments from a news commenting site that we tasked Amazon Mechanical Turk workers with labeling. Each comment is labeled for the presence of profanity, insults, and the object of the insults. Support vector machines trained on these data are combined with relevance and valence analysis systems in a multistep approach to the detection of inappropriate negative user contributions. The system shows great potential for semiautomated community management.

Verbal Venting in the Social Web: Effects of Anonymity and Group Norms on Aggressive Language Use in Online Comments

“…great potential to escalate and set the tone in online discussions (Sood, Churchill, & Antin, 2012), the exerted social influence of other online users and the power of social norms might be another cause of aggressive language use in online communication. According to social influence theories, individuals affect each other's opinions and behaviors in social context and tend to conform to prevalent social norms of a common social group-especially when they identify with the group (cf.…”

Detecting Offensive Statements towards Foreigners in Social Media

“…Compared to lexicon approaches, the classification performance is only slightly better [28,29,31]. Additionally, the collection of an adequate amount of training data is cumbersome due to the lack of annotated datasets [28,31].…”

“…A major advantage of these approaches is their simplicity and independence of training data as well as easy adoption in other languages by providing adequate lexica by experts. However, their practical applicability is limited, especially in the context of online harassment detection as they achieve only reasonable to moderate classification performance [28]. As a consequence, they are often used to preselect potential offending messages to perform subsequent analyses [31].…”

“…These features are derived by experts from characteristics of hate speech messages and include, for example, the presence of offending words defined in a lexicon [14,30] and the presence of words typically referring to persons [30]. Compared to lexicon approaches, the classification performance is only slightly better [28,29,31]. Additionally, the collection of an adequate amount of training data is cumbersome due to the lack of annotated datasets [28,31].…”

“…The majority of the publications apply either lexicon [28,31] or machine learning approaches [14,29,30]. Lexicon approaches entirely rely on a lexicon containing offensive words typically used in hate speech.…”