Detecting Weak Signals of the Future: A System Implementation Based on Text Mining and Natural Language Processing

Abstract

Organizations, companies and start-ups need to cope with constant changes on the market which are difficult to predict. Therefore, the development of new systems to detect significant future changes is vital to make correct decisions in an organization and to discover new opportunities. A system based on business intelligence techniques is proposed to detect weak signals, that are related to future transcendental changes. While most known solutions are based on the use of structured data, the proposed system quantitatively detects these signals using heterogeneous and unstructured information from scientific, journalistic and social sources, applying text mining to analyze the documents and natural language processing to extract accurate results. The main contributions are that the system has been designed for any field, using different input datasets of documents, and with an automatic classification of categories for the detected keywords. In this research paper, results from the future of remote sensors are presented. Remote sensing services are providing new applications in observation and analysis of information remotely. This market is projected to witness a significant growth due to the increasing demand for services in commercial and defense industries. The system has obtained promising results, evaluated with two different methodologies, to help experts in the decision-making process and to discover new trends and opportunities.

Contextualizing Emerging Trends in Financial News Articles

“…A refined version of TF-IDF, proposed by Zhu et al (2019), aims to discover "hot" topics according to "hot" terms based on time distribution information, user attention, and K-means clustering. Unlike previous work that tackled trend detection in official documents and newspapers, others focused on proposing new approaches using research and scientific papers, documents that generally contain citations and bibliographies that could be considered as additional features (Nie and Sun, 2017;Xu et al, 2019;He et al, 2009;Griol-Barres et al, 2020). However, exploiting bibliographies can have disadvantages in timeliness and content analysis, as discussed by (Dridi et al, 2019).…”

BEAM: A Network Topology Framework to Detect Weak Signals

“…In addition, keywords related to weak signals are, in general, isolated terms and the absence of relationships and context limits the information in further interpretation. Griol-Barres et al [12], [13] use a third measure, the degree of transmission (DoT) to assess the importance of keywords. DoT takes into account the documents' h-index where keywords appear.…”

Evolution of the Electricity Distribution Networks—Active Management Architecture Schemes and Microgrid Control Functionalities

“…The prerequisite for the scenario building or anticipating and envisioning tomorrow is understanding the prevailing facts, which are then summarized by imagination [8]. In addition, the weak signals [9][10][11] are used individually since they are indicators of changing or emerging topics. The weak signals may be related, for example, to technologies, behaviors, markets, and regulations.…”