Term Weighting: Novel Fuzzy Logic Based Method vs. Classical Tf-Idf Method for Web Information Extraction

Ropero, Jorge; Gómez, Ariel; León, Carlos; Carrasco, Alejandro

doi:10.5220/0001982901300137

Cited by 5 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Following, the basic formula TF * IDF (Bounabi et al , 2017) has known the additional factors, e.g. the length of document N. for normalization reasons and to adjust the formula according to the case study (Ropero et al , 2009). Different versions of TF-IDF were adopted in the following, and the probabilistic TF-IDF was mentioned in several works (Bounabi et al , 2018; Aizawa, 2003), which marked a significant improvement in the text classification field.…”

Section: Related Workmentioning

confidence: 99%

A new neutrosophic TF-IDF term weighting for text mining tasks: text classification use case

Bounabi

Elmoutaouakil

Satori

2021

IJWIS

View full text Add to dashboard Cite

Purpose This paper aims to present a new term weighting approach for text classification as a text mining task. The original method, neutrosophic term frequency – inverse term frequency (NTF-IDF), is an extended version of the popular fuzzy TF-IDF (FTF-IDF) and uses the neutrosophic reasoning to analyze and generate weights for terms in natural languages. The paper also propose a comparative study between the popular FTF-IDF and NTF-IDF and their impacts on different machine learning (ML) classifiers for document categorization goals. Design/methodology/approach After preprocessing textual data, the original Neutrosophic TF-IDF applies the neutrosophic inference system (NIS) to produce weights for terms representing a document. Using the local frequency TF, global frequency IDF and text N's length as NIS inputs, this study generate two neutrosophic weights for a given term. The first measure provides information on the relevance degree for a word, and the second one represents their ambiguity degree. Next, the Zhang combination function is applied to combine neutrosophic weights outputs and present the final term weight, inserted in the document's representative vector. To analyze the NTF-IDF impact on the classification phase, this study uses a set of ML algorithms. Findings Practicing the neutrosophic logic (NL) characteristics, the authors have been able to study the ambiguity of the terms and their degree of relevance to represent a document. NL's choice has proven its effectiveness in defining significant text vectorization weights, especially for text classification tasks. The experimentation part demonstrates that the new method positively impacts the categorization. Moreover, the adopted system's recognition rate is higher than 91%, an accuracy score not attained using the FTF-IDF. Also, using benchmarked data sets, in different text mining fields, and many ML classifiers, i.e. SVM and Feed-Forward Network, and applying the proposed term scores NTF-IDF improves the accuracy by 10%. Originality/value The novelty of this paper lies in two aspects. First, a new term weighting method, which uses the term frequencies as components to define the relevance and the ambiguity of term; second, the application of NL to infer weights is considered as an original model in this paper, which also aims to correct the shortcomings of the FTF-IDF which uses fuzzy logic and its drawbacks. The introduced technique was combined with different ML models to improve the accuracy and relevance of the obtained feature vectors to fed the classification mechanism.

show abstract

Section: Related Workmentioning

confidence: 99%

A new neutrosophic TF-IDF term weighting for text mining tasks: text classification use case

Bounabi

Elmoutaouakil

Satori

2021

IJWIS

View full text Add to dashboard Cite

show abstract

“…Information extraction (IE) is the task of transformation a document collection into easier to analyze information [2], it tries to get relevant facts from documents. Whereas, Information retrieval (IR) deals with the representation, storage and organization of, and access to information items [3].…”

Section: Relate Workmentioning

confidence: 99%

A Term Weighting Scheme Using Fuzzy Logic for Enhancing Candidate Screening Task

Habous

Nfaoui

2021

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

The candidate screening is an essential task in the recruitment process. It is about choosing a suitable candidate that satisfies the recruiter requirements for a given job position. The evolution of information technologies leads to an increase in the use of the recruitment web portals by the candidates that apply for the job positions published in the job boards. Thus the candidate screening process automation becomes necessary to handle the enormous volume of CVs applying for the job positions. In Information Technology (IT) domain, the technology skills are the key competencies to identify the job profile; Consequently, they have priority to the candidate screening task. In this paper, we enhance the candidate screening task in the IT field. For this purpose, we propose a fuzzy-based weighting scheme using domain ontology for Information Retrieval (IR). Experimental results on a recruiter company data show the effective results of our proposed solution.

show abstract

“…In particular, a considerable amount of research has been devoted to discriminating between the representativeness of keywords, which represents the importance of the extracted keyword, thereby attaching a weight to the keyword [5 –8]. The use of keywords with weight led to association rule mining being performed [9–11]; moreover, some research was performed to compare a fuzzy logic-based weight technique and TF-IDF (Term Frequency–Inverse Document Frequency) [12 –15]. In addition, other research applied the clustering method with fuzzy weighting to symbolic interval data [16].…”

Section: Related Workmentioning

confidence: 99%

Proposal reviewer recommendation system based on big data for a national research management institute

Shon

Han

Kim

et al. 2016

Journal of Information Science

View full text Add to dashboard Cite

National research management organizations need to ensure that research proposals are reviewed fairly and efficiently, which requires the selection of suitable reviewers. In particular, reviewing research proposals in a particular area necessitates the selection of a group with the most reasonable standard for recommending an expert in that area. In this study, we develop an automatic matching system that matches a research proposal with a reviewer who can evaluate it most effectively, using keywords with fuzzy weights based on databases in the corresponding field of research. All functions that we developed were based on the MapReduce framework created by Hadoop, which was verified to enhance matching performance and ensure expandability. This enabled us to select suitable researchers from existing research projects, papers and research reviewer databases. Our system can influence the operation of the national research management system and contribute to academic development.

show abstract

Term Weighting: Novel Fuzzy Logic Based Method vs. Classical Tf-Idf Method for Web Information Extraction

Cited by 5 publications

References 12 publications

A new neutrosophic TF-IDF term weighting for text mining tasks: text classification use case

A new neutrosophic TF-IDF term weighting for text mining tasks: text classification use case

A Term Weighting Scheme Using Fuzzy Logic for Enhancing Candidate Screening Task

Proposal reviewer recommendation system based on big data for a national research management institute

Contact Info

Product

Resources

About