Automated Multidimensional Analysis of Global Events With Entity Detection, Sentiment Analysis and Anomaly Detection

A Novel Method of Generating Geospatial Intelligence from Social Media Posts of Political Leaders

“…The existing studies in political science, as seen in [4][5][6][7]21,22], have directly used location fields from Twitter feeds without applying a NER algorithm. Since this is the first study to apply a NER-based algorithm in the domain of political science, this study observed unique cases that have never been explored in previous studies [4][5][6]11,12,21,22,30]. During our experimentation, we witnessed several terms being incorrectly classified as location entities (i.e., false positives) and several valid location entities not being recognized as location entities (i.e., false negatives) by our implementation of NER algorithms.…”

“…NERs have been previously used in multiple domains, including the biomedical sector, as shown in [30]. In [30], researchers identified 14 articles that used NER exclusively within the biomedical In our previous studies, NER was used to generate locations of global events in a generic domain and not particularly focused on political science [11,12]. Moreover, in [17], our implementation of NER-based techniques was focused on the social science domain.…”

“…The system proposed in this paper can work with any other social media platforms, such as Facebook, Instagram, and LinkedIn, via supported APIs with Microsoft Power Platform, as depicted in Figure 1. Moreover, this system can also extract dynamic content from websites by using web scraping technology supported by Microsoft Power Platform (i.e., by using M language) [11,12]. As seen from Figure 1, MS Power Automate [28] and MS Power Query were used for acquiring tweets, data cleansing and transformation, language detection and language translation [26,27], sentiment analysis, and finally, NER.…”

“…SQL queries were used for data exploration, further analysis, and serving dashboards with filtered queries. Finally, Microsoft Power BI was used for data visualization, analysis, generating insights with AI and NLP [9][10][11][12]17] To assess the end-to-end performance of our algorithm, we used several evaluation metrics, including accuracy (i.e., the fraction of the total samples that were correctly classified by a classifier), precision (i.e., the fraction of predictions as positive categories that were actually positive by the classifier), recall (i.e., the fraction of all positive samples that were correctly predicted as positive by the classifier), and the F1 score (i.e., precision and recall combined into a single measure) as represented by Equation (1) to Equation ( 4).…”

“…It is methodologically possible to perform behavioral analysis of Twitter users by harnessing big data extracted from their social media accounts [8] and then using various artificial intelligence (AI)-based techniques reported in our earlier studies [9][10][11][12][13][14][15][16][17][18]. Existing studies in the political science domain either used geotagged tweets or completely skipped usage of location extraction algorithms, as observed in [4][5][6][7]19,20].…”