Towards a Transparent and an Environmental-Friendly Approach for Short Text Topic Detection: A Review of Methods for Performance, Transparency and Carbon Footprint

Abstract

<p>Online social media platforms have contributed significantly to the dissemination of user-generated information. Many studies have proposed various techniques to analyse the publicly available short texts to automatically extract topics. The majority of these works have mainly focused on the competitive performance of the proposed approaches. In this paper our main focus is on how to tackle this problem by incorporating two other important qualities: Transparency and Carbon Footprint. These two pillars are cornerstones to fulfil the emerging international demands and to adhere to the new regulations, such as “Right to Explanation” and “Green AI”. Based on these three qualities, this paper compares the most prominent algorithms in this field, such as: Latent Dirichlet Allocation, Non-Negative Matrix Factorization and KMeans as well as two most recent approaches, such as: BERTopic and Contextual Analysis. By using two different corpuses, the methods were evaluated for Performance. On average, the results show that BERTopic is the best performing approach overall in terms of Performance. However, Contextual Analysis achieves the best Performance in one of the two corpuses used. When considering the three qualities together, the results demonstrate the effectiveness and the benefits of the Contextual Analysis method towards a more transparent and a greener approach for the topic detection task.</p>