Revolution of Shakespearean Plays' Genre Research: Exploring New Avenues through Machine Learning and Shapley Value Analysis

Abstract

In the realm of literary research, the challenges of being confined to narrow niches and disconnected from broader contexts have been long-standing. In response, the integration of digital research methods into literary studies has emerged as a compelling area of exploration. Among the classic subjects of literary research, the classification of Shakespearean drama genres holds particular significance. In this paper, we present a case study focused on introducing a promising predictive and analytical method, which leverages Linear Discriminant Analysis (LDA) and the Shapley value. Our methodology begins by employing decision trees to reduce the dimensionality of textual data. Subsequently, a LDA based on Bayesian optimization algorithm is applied to predict the genres of texts. Finally, we utilize the Shapley value to analyze the important words within the texts and unveil their profound literary associations with respective genres. By adopting this approach, our research contributes to the widespread adoption and digital transformation of literary studies, thereby pioneering new avenues in Shakespearean drama research.