Defining and Detecting Toxicity on Social Media: Context and Knowledge are Key

Abstract

As the role of online platforms has become increasingly prominent for communication, toxic behaviors, such as cyberbullying and harassment, have been rampant in the last decade. On the other hand, online toxicity is multi-dimensional and sensitive in nature, which makes its detection challenging. As the impact of exposure to online toxicity can lead to serious implications for individuals and communities, reliable models and algorithms are required for detecting and understanding such communications. In this paper, we define toxicity to provide a foundation drawing social theories. Then, we provide an approach that identifies multiple dimensions of toxicity and incorporates explicit knowledge in a statistical learning algorithm to resolve ambiguity across such dimensions.

Identifying Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction

“…For example, abusers may misuse Wi-Fi enabled devices, including locking out and controlling devices to terrorize and harass users [38,149], or use technology to generate or share non-consensual sexually explicit images [15,123]. Beyond gender violence, other facets of technology-facilitated violence, include doxxing [69], trolling [14], cyberstalking [14], cyberbullying [14,88,179], monitoring and control [38], and online harassment and intimidation [88,169,174,200], under the broader banner of online toxicity [88,117]. Technology-facilitated violence leads to co-occurring harms, including feeling of distress, fear, and humiliation [38], while often infringing personal and bodily integrity, dignity, and privacy and inhibiting autonomy and expression [123].…”