Big data meets storytelling: using machine learning to predict popular fanfiction

Abstract

Fanfictions are a popular literature genre in which writers reuse a universe, for example to transform heteronormative relationships with queer characters or to bring romance into shows focused on horror and adventure. Fanfictions have been the subject of numerous studies in text mining and network analysis, which used Natural Language Processing (NLP) techniques to compare fanfictions with the original scripts or to make various predictions. In this paper, we use NLP to predict the popularity of a story and examine which features contribute to popularity. This endeavor is important given the rising use of AI assistants and the ongoing interest in generating text with desirable characteristics. We used the main two websites to collect fan stories (Fanfiction.net and Archives Of Our Own) on Supernatural, which has been the subject of numerous scholarly works. We extracted high-level features such as the main character and sentiments from 79,288 of these stories and used the features in a binary classification supported by tree-based methods, ensemble methods (random forest), neural networks, and Support Vector Machines. Our optimized classifiers correctly identified popular stories in four out of five cases. By relating features to classification outcomes using SHAP values, we found that fans prefer longer stories with a wider vocabulary, which can inform the prompts of AI chatbots to continue generating such successful stories. However, we also observed that fans wanted stories unlike the original material (e.g., favoring romance and disliking when characters are hurt), hence AI-powered stories may be less popular if they strictly follow the original material of a show.