Combining semantic and term frequency similarities for text clustering

Soares, Victor Hugo Andrade; Campello, Ricardo J. G. B.; Nourashrafeddin, Seyednaser; Milios, Evangelos; Naldi, Murilo Coelho

doi:10.1007/s10115-018-1278-7

Cited by 18 publications

(7 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Text clustering has a wide range of applications, such as topic detection and tracking [ 46 ], document summary [ 47 ], and search results clustering [ 48 ]. A wealth of techniques has been proposed for text clustering, including spectral methods [ 49 ], matrix factorization [ 50 ], hierarchical methods [ 51 ], partitional approaches [ 52 ], and model-based methods [ 53 ], in addition to further approaches based on semantic similarity [ 54 ], evolutionary algorithms [ 55 ] and concept factorization [ 56 ].…”

Section: Literature Reviewmentioning

confidence: 99%

Analysis of Risk Factors of Coal Chemical Enterprises Based on Text Mining

Yao

Luo

et al. 2023

Journal of Environmental and Public Health

View full text Add to dashboard Cite

Coal chemical enterprises have many risk factors, and the causes of accidents are complex. The traditional risk assessment methods rely on expert experience and previous literature to determine the causes of accidents, which has the problems such as lack of objectivity and low interpretation ability. Analyzing the accident report helps to identify typical accident risk factors and determines the accident evolution rule. However, experts usually judge this work manually, which is subjective and time-consuming. This paper developed an improved approach to identify safety risk factors from a volume of coal chemical accident reports using text mining (TM) technology. Firstly, the accident report was preprocessed, and the Term Frequency Inverse Document Frequency (TF-IDF) was used for feature extraction. Then, the K-means algorithm and apriori algorithm were developed to cluster and for the association rule analysis of the vectorized documents in the TF-IDF matrix, respectively to quickly identify the hidden risk factors and the relationship between risk factors in the accident report and to propose targeted safety management measures. Using the sample data of 505 accidents in a large coal chemical enterprise in Western China in the past seven years, the enterprise accident reports were analyzed by text clustering analysis and association rule analysis methods. Through the analysis, six accident clusters and 13 association rules were obtained, and the main risk factors of each accident cluster were further mined, and the corresponding management suggestions were put forward for the enterprise. This method provides a new idea for coal chemical enterprises to make safety management decisions and helps to prevent safety accidents.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Analysis of Risk Factors of Coal Chemical Enterprises Based on Text Mining

Yao

Luo

et al. 2023

Journal of Environmental and Public Health

View full text Add to dashboard Cite

show abstract

“…Text mining refers to the computer processing technology used to extract valuable information from text data, which can also be called knowledge discovery in a database [ 23 ]. In recent years, there has been a rising trend in the achievements related to text mining, and it is already becoming one of the most effective methods for studying the relationships between elements in various disciplines [ 24 , 25 , 26 , 27 ]. This paper mainly uses the function of text segmentation in text mining to extract the manifestations with the characteristics of 883 coal mine accidents.…”

Section: Preliminaries and Research Frameworkmentioning

confidence: 99%

Correlation in Causality: A Progressive Study of Hierarchical Relations within Human and Organizational Factors in Coal Mine Accidents

Liu

et al. 2021

IJERPH

View full text Add to dashboard Cite

It has been revealed in numerous investigation reports that human and organizational factors (HOFs) are the fundamental causes of coal mine accidents. However, with various kinds of accident-causing factors in coal mines, the lack of systematic analysis of causality within specific HOFs could lead to defective accident precautions. Therefore, this study centered on the data-driven concept and selected 883 coal mine accident reports from 2011 to 2020 as the original data to discover the influencing paths of specific HOFs. First, 55 manifestations with the characteristics of the coal mine accidents were extracted by text segmentation. Second, according to their own attributes, all manifestations were mapped into the Human Factors Analysis and Classification System (HFACS), forming a modified HFACS-CM framework in China’s coal-mining industry with 5 categories, 19 subcategories and 42 unsafe factors. Finally, the Apriori association algorithm was applied to discover the causal association rules among external influences, organizational influences, unsafe supervision, preconditions for unsafe acts and direct unsafe acts layer by layer, exposing four clear accident-causing “trajectories” in HAFCS-CM. This study contributes to the establishment of a systematic causation model for analyzing the causes of coal mine accidents and helps form corresponding risk prevention measures directly and objectively.

show abstract

“…The mixed measures took advantage of thesaurus‐based and corpus‐based semantic methods to obtain better performance. Afterward, more and more semantic similarity technologies have been applied to a document or text clustering, and have achieved good results 73,74 …”

Section: Applicationsmentioning

confidence: 99%

A survey on the techniques, applications, and performance of short text semantic similarity

Han

Zhang

Yuan

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

Summary Short text similarity plays an important role in natural language processing (NLP). It has been applied in many fields. Due to the lack of sufficient context in the short text, it is difficult to measure the similarity. The use of semantics similarity to calculate textual similarity has attracted the attention of academia and industry and achieved better results. In this survey, we have conducted a comprehensive and systematic analysis of semantic similarity. We first propose three categories of semantic similarity: corpus‐based, knowledge‐based, and deep learning (DL)‐based. We analyze the pros and cons of representative and novel algorithms in each category. Our analysis also includes the applications of these similarity measurement methods in other areas of NLP. We then evaluate state‐of‐the‐art DL methods on four common datasets, which proved that DL‐based can better solve the challenges of the short text similarity, such as sparsity and complexity. Especially, bidirectional encoder representations from transformer model can fully employ scarce information of short texts and semantic information and obtain higher accuracy and F1 value. We finally put forward some future directions.

show abstract

Combining semantic and term frequency similarities for text clustering

Cited by 18 publications

References 33 publications

Analysis of Risk Factors of Coal Chemical Enterprises Based on Text Mining

Analysis of Risk Factors of Coal Chemical Enterprises Based on Text Mining

Correlation in Causality: A Progressive Study of Hierarchical Relations within Human and Organizational Factors in Coal Mine Accidents

A survey on the techniques, applications, and performance of short text semantic similarity

Contact Info

Product

Resources

About