Down the Rabbit Hole: Detecting Online Extremism, Radicalisation, and Politicised Hate Speech

Govers, Jarod; Feldman, Philip S.; Dant, Aaron; Patros, Panos

doi:10.1145/3583067

Cited by 19 publications

(9 citation statements)

References 83 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We may hypothesize that in order to enhance long-term engagement, recommender systems may steer users towards regions where highly engaging content can be recommended in the future. This investigation may shed light on the mechanisms behind radicalization 44,45 , and so-called "rabbit holes" 46 , which are not captured in the present description, the maximization of engagement being session-wise.…”

Section: /22mentioning

confidence: 85%

Skewed perspectives: examining the influence of engagement maximization on content diversity in social media feeds

Bouchaud

2024

J Comput Soc Sc

View full text Add to dashboard Cite

This article investigates the information landscape shaped by curation algorithms that seek to maximize user engagement. Leveraging unique behavioral data, we trained machine learning models to predict user engagement with tweets. Our study reveals how the pursuit of engagement maximization skews content visibility, favoring posts similar to previously engaged content while downplaying alternative perspectives. The empirical grounding of our work contributes to the understanding of human-machine interactions and provides a basis for evidence-based policies aimed at promoting responsible social media platforms.

show abstract

Section: /22mentioning

confidence: 85%

Skewed perspectives: examining the influence of engagement maximization on content diversity in social media feeds

Bouchaud

2024

J Comput Soc Sc

View full text Add to dashboard Cite

show abstract

“…They have become a central part of modern society, enabling people to connect with each other regardless of their location, share their thoughts and opinions, and participate in online communities (Osman, Barbaro, and Skumanich 2023). However, the rise of social networks has also led to a number of challenges, including the deliberate spread of disinformation (Barbaro and Skumanich 2023), cyberbullying (Giumetti and Kowalski 2022), and the propagation of hate speech and extremist ideologies (Govers et al 2023), along with in general what we call "mal-info". In particular the latter, is being deployed for deliberate disruption of society with the intent to cause conflict and discord.…”

Section: Introductionmentioning

confidence: 99%

“…While these networks offer a space for so-called free expression, they also raise questions about the impact of social networks in shaping perceptions and attitudes, and the potential consequences of this influence on societal well-being. Despite these many concerns, very little research has focused on these new social networks (Peucker and Fisher 2023), instead mostly concentrating on X/Twitter (Govers et al 2023). This paper presents a preliminary step to setting a foundation for monitoring the mal-info which can cause social degradation.…”

Section: Introductionmentioning

confidence: 99%

Modes of Tracking Mal-Info in Social Media with AI/ML Tools to Help Mitigate Harmful GenAI for Improved Societal Well Being

Skumanich,

Kim

2024

AAAI-SS

View full text Add to dashboard Cite

A rapidly developing threat to societal well-being is from misinformation widely spread on social media. Even more concerning is ”mal-info” (malicious) which is amplified on certain social networks. Now there is an additional dimension to that threat, which is the use of Generative AI to deliberately augment the mis-info and mal-info. This paper highlights some of the ”fringe” social media channels which have a high level of mal-info as characterized by our AI/ML algorithms. We discuss various channels and focus on one in particular, ”GAB”, as representative of the potential negative impacts. We outline some of the current mal-info as an example. We capture elements, and observe the trends in time. We provide a set of AI/ML modes which can characterize the mal-info and allow for capture, tracking, and potentially for responding or for mitigation. We highlight the concern about malicious agents using GenAI for deliberate mal-info messaging specifically to disrupt societal well being. We suggest the characterizations presented as a methodology for initiating a more deliberate and quantitative approach to address these harmful aspects of social media which would adversely impact societal well being. The article highlights the potential for ”mal-info,” including disinfo, cyberbullying, and hate speech, to disrupt segments of society. The amplification of mal-info can result in serious real-world consequences such as mass shootings. Despite attempts to introduce moderation on major platforms like Facebook and to some extent on X/Twitter, there are now growing social networks such as Gab, Gettr, and Bitchute that offer completely unmoderated spaces. This paper presents an introduction to these platforms and the initial results of a semiquantitative analysis of Gab’s posts. The paper examines several characterization modes using text analysis. The paper emphasizes the developing dangerous use of generative AI algorithms by Gab and other fringe platforms, highlighting the risks to societal well being. This article aims to lay the foundation for capturing, monitoring, and mitigating these risks.

show abstract

“…Historically, cyberbullying detection predominantly relied on rule-based systems and traditional machine learning algorithms [2]. These approaches, albeit beneficial to some extent, have proven inadequate in understanding the multifaceted nature of human language, especially given the eclectic mixture of colloquialisms, slang, and indirect innuendos that characterize online communication [3]. The challenges are compounded by the dynamic nature of online discourse, which continually evolves, often eluding static algorithmic formulations.…”

Section: Introductionmentioning

confidence: 99%

Hybrid Convolutional Recurrent Neural Network for Cyberbullying Detection on Textual Data

Baiganova,

Toxanova,

Yerekesheva

et al. 2024

ijacsa

View full text Add to dashboard Cite

With the burgeoning use of social media platforms, online harassment and cyberbullying have become significant concerns. Traditional mechanisms often falter, necessitating advanced methodologies for efficient detection. This study presents an innovative approach to identifying cyberbullying incidents on social media sites, employing a hybrid neural network architecture that amalgamates Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN). By harnessing the sequential processing capabilities of LSTM to analyze the temporal progression of textual data, and the spatial discernment of CNN to pinpoint bullying keywords and patterns, the model demonstrates substantial improvement in detection accuracy compared to extant methods. A diverse dataset, encompassing multiple social media platforms and linguistic styles, was utilized to train and test the model, ensuring robustness. Results evince that the LSTM-CNN amalgamation can adeptly handle varied sentence structures and contextual nuances, outstripping traditional machine learning classifiers in both specificity and sensitivity. This research underscores the potential of hybrid neural networks in addressing contemporary digital challenges, urging further exploration into blended architectures for nuanced problem-solving in cyber realms.

show abstract

Down the Rabbit Hole: Detecting Online Extremism, Radicalisation, and Politicised Hate Speech

Cited by 19 publications

References 83 publications

Skewed perspectives: examining the influence of engagement maximization on content diversity in social media feeds

Skewed perspectives: examining the influence of engagement maximization on content diversity in social media feeds

Modes of Tracking Mal-Info in Social Media with AI/ML Tools to Help Mitigate Harmful GenAI for Improved Societal Well Being

Hybrid Convolutional Recurrent Neural Network for Cyberbullying Detection on Textual Data

Contact Info

Product

Resources

About