SocialTERM-Extractor: Identifying and Predicting Social-Problem-Specific Key Noun Terms from a Large Number of Online News Articles Using Text Mining and Machine Learning Techniques

Abstract

In the digital age, the abundant unstructured data on the Internet, particularly online news articles, provide opportunities for identifying social problems and understanding social systems for sustainability. However, the previous works have not paid attention to the social-problem-specific perspectives of such big data, and it is currently unclear how information technologies can use the big data to identify and manage the ongoing social problems. In this context, this paper introduces and focuses on social-problem-specific key noun terms, namely SocialTERMs, which can be used not only to search the Internet for social-problem-related data, but also to monitor the ongoing and future events of social problems. Moreover, to alleviate time-consuming human efforts in identifying the SocialTERMs, this paper designs and examines the SocialTERM-Extractor, which is an automatic approach for identifying the key noun terms of social-problem-related topics, namely SPRTs, in a large number of online news articles and predicting the SocialTERMs among the identified key noun terms. This paper has its novelty as the first trial to identify and predict the SocialTERMs from a large number of online news articles, and it contributes to literature by proposing three types of text-mining-based features, namely temporal weight, sentiment, and complex network structural features, and by comparing the performances of such features with various machine learning techniques including deep learning. Particularly, when applied to a large number of online news articles that had been published in South Korea over a 12-month period and mostly written in Korean, the experimental results showed that Boosting Decision Tree gave the best performances with the full feature sets. They showed that the SocialTERMs can be predicted with high performances by the proposed SocialTERM-Extractor. Eventually, this paper can be beneficial for individuals or organizations who want to explore and use social-problem-related data in a systematical manner for understanding and managing social problems even though they are unfamiliar with ongoing social problems.

Big data as a value generator in decision support systems: a literature review

“…We also found application in route optimization models through DSS to view traffic volume and weather interactions applicable to transport planners, traffic control rooms and urban infrastructure DSS (Sathiaraj, Punkasem, Wang & Seedah, 2018) and through an online route generation system with Dijkstra algorithm, which resulted in changes of route paradigms that were determined together (Rönnqvist, Svenson, Flisberg & Jö). Finally, we found an application for the purpose of identifying and predicting social issues through online news analytics (Suh, 2019).…”

“…We also found application in route optimization models through DSS to view traffic volume and weather interactions applicable to transport planners, traffic control rooms and urban infrastructure DSS (Sathiaraj, Punkasem, Wang & Seedah, 2018) and through an online route generation system with Dijkstra algorithm, which resulted in changes of route paradigms that were determined together (R€ onnqvist, Svenson, Flisberg & J€ o). Finally, we found an application for the purpose of identifying and predicting social issues through online news analytics (Suh, 2019). Other findings were with vector machine for decision-making optimization through patient similarity (Tashkandi, Wiese & Wiese, 2018) and in optimizing data collection for cancer classification and online critic sentiments (Ghaddar & Naoum-Sawaya, 2018), the fuzzy rule to determine the health status of cattle to predict nutritional intake (Sivamani, Choi & Cho, 2018), Mehrabian-Russell model for forecasting consumer purchases using climate parameters (Tian, Zhang & Zhang, 2018), the use of a set of attribute reduction and data set analysis in information systems (Li, Yang, Jin & Guo, 2017) and rapid safety feedback used to identify child maltreatment (Gillingham, 2019a).…”

Forecasting Spare Parts Demand of Military Aircraft: Comparisons of Data Mining Techniques and Managerial Features from the Case of South Korea

“…Data mining, among many qualitative methods of demand forecasting, is a casual method that helps decision support analytically by extracting features, relationships, patterns, and rules through exploration, analysis, and modeling of large-scale data [12,13]. Data mining has emerged as an alternative tool for modeling and forecasting due to its ability to capture the non-linearity in the data.…”

“…For each of the data mining techniques, we performed a 10-fold cross validation as an experiment, and repeated the same experiments 30 times. For each of the experimental repetitions, we used a different random seed, but the random seed was kept identical for the same iteration of different data mining techniques, by referring to the previous studies [13,54]. To implement the four data mining techniques, we programmed JAVA codes based on the data Regarding the Surion, the 2013-2018 data used for this study were collected from the DELIIS, which manages information related to maintenance and supply of the weapon systems in South Korea.…”

Constructive Approach for Early Extraction of Viral Spreading Social Issues from Twitter

“…Jeong et al extracted sentences that explained the social issues by using topic modeling to Korean newspapers and news channels [15]. While many previous studies used newspapers and news data, Suh focused on the fact that articles on world events circulate over the Internet and proposed a method to extract keywords related to social issues from web news [30]. In comparison to these study, our study aimed at the early extraction of viral spreading social issues and proposed the use of Twitter because news channels and newspapers are not the epicenters of viral spreading social issues.…”