Experiments on the Use of Feature Selection and Negative Evidence in Automated Text Categorization

Galavotti, Luigi; Sebastiani, Fabrizio; Simi, Maria

doi:10.1007/3-540-45268-0_6

Cited by 165 publications

(99 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Numerous feature selection/reduction approaches have been proposed [6] in order to solve this problem. The successfully used feature selection approaches include Document Frequency (DF), Mutual Information (MI), Information Gain (IG), Chi-square test or Gallavotti, Sebastiani & Simi metric [7,8]. Furthermore, a better document representation may lead to decreasing the feature vector dimension, e.g.…”

Section: Related Workmentioning

confidence: 99%

“…6 Olsson presents in [18] a Czech-English cross-language classification on the MALACH 7 data set. Wu et al deals in [19] with a bilingual topic aspect classification of English and Chinese news articles from the Topic Detection and Tracking (TDT) 8 collection. Unfortunatelly, only few work about the classification of the Czech documents exits.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Named Entities as New Features for Czech Document Classification

Král

2014

Computational Linguistics and Intelligent Text Processing

View full text Add to dashboard Cite

Abstract. This paper is focused on automatic document classification. The results will be used to develop a real application for the Czech News Agency. The main goal of this work is to propose new features based on the Named Entities (NEs) for this task. Five different approaches to employ NEs are suggested and evaluated on a Czech newspaper corpus. We show that these features do not improve significantly the score over the baseline word-based features. The classification error rate improvement is only about 0.42% when the best approach is used.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Named Entities as New Features for Czech Document Classification

Král

2014

Computational Linguistics and Intelligent Text Processing

View full text Add to dashboard Cite

show abstract

“…Among these are the DIA association factor (Fuhr and Buckley 1991), chi-square (Yang and Pedersen 1997;Sebastiani, Sperduti et al 2000;Caropreso, Matwin et al 2001), NGL coefficient (Ng, Goh et al 1997;Ruiz and Srinivasan 1999), information gain Lewis and Ringuette 1994;Moulinier, Raskinis et al 1996;Yang and Pedersen 1997;Larkey 1998;Mladenic and Grobelnik 1998;Caropreso, Matwin et al 2001), mutual information (Larkey and Croft 1996;Wai and Fan 1997;Dumais, Platt et al 1998;Taira and Haruno 1999) odds ratio (Mladenic and Grobelnik 1998;Ruiz and Srinivasan 1999;Caropreso, Matwin et al 2001), relevancy score (Wiener, Pedersen et al 1995) and GSS coefficient (Galavotti, Sebastiani et al 2000). Three of the most popular methods are descrivbed briefly below.…”

Section: This Leads To the Term Frequency/inverse Document Frequency mentioning

confidence: 99%

“…Various experimental comparisons of feature selection functions applied to TC contexts have been carried out (Yang and Pedersen 1997;Mladenic and Grobelnik 1998;Galavotti, Sebastiani et al 2000;Caropreso, Matwin et al 2001). In these experiments most functions have improved on the results for basic document frequency thresholding.…”

Section: Experimental Comparisonsmentioning

confidence: 99%

Text and Hypertext Categorization

Benbrahim

Bramer

2009

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Automatic categorization of text documents has become an important area of research in the last two decades, with features that make it significantly more difficult than the traditional classification tasks studied in machine learning. A more recent development is the need to classify hypertext documents, most notably web pages. These have features that add further complexity to the categorization task but also offer the possibility of using information that is not available in standard text classification, such as metadata and the content of the web pages that point to and are pointed at by a web page of interest. This chapter surveys the state of the art in text categorization and hypertext categorization, focussing particularly on issues of representation that differentiate them from 'conventional' classification tasks and from each other.

show abstract

“…• GSSC: The GSS (Galavotti⋅Sebastiani⋅Simi) Coefficient defined in [13] represents the core calculation as well as a simplified variant of both the Chisquare Statistics (χ 2 ) and the Correlation Coefficient (CC) statistical FS mechanisms. In [27,30], the authors state: (i) the well-established χ 2 statistic can be applied to measure the lack of independence between a term u h and a predefined class C i ; and (ii) if the feature/term and the class are independent, the calculated χ 2 score has a natural value 0.…”

Section: Statistical Feature Selectionmentioning

confidence: 99%

A Hybrid Statistical Data Pre-processing Approach for Language-Independent Text Classification

Wang¹,

Coenen

Sanderson

2009

Advanced Data Mining and Applications

View full text Add to dashboard Cite

Abstract. Data pre-processing is an important topic in Text Classification (TC).It aims to convert the original textual data in a data-mining-ready structure, where the most significant text-features that serve to differentiate between textcategories are identified. Broadly speaking, textual data pre-processing techniques can be divided into three groups: (i) linguistic, (ii) statistical, and (iii) hybrid (i) & (ii). With regard to language-independent TC, our study relates to the statistical aspect only. The nature of textual data pre-processing includes: Document-base Representation (DR) and Feature Selection (FS). In this paper, we propose a hybrid statistical FS approach that integrates two existing (statistical FS) techniques, DIAAF (Darmstadt Indexing Approach Association Factor) and GSSC (Galavotti⋅Sebastiani⋅Simi Coefficient). Our proposed approach is presented under a statistical "bag of phrases" DR setting. The experimental results, based on the well-established associative text classification approach, demonstrate that our proposed technique outperforms existing mechanisms with respect to the accuracy of classification.

show abstract

Experiments on the Use of Feature Selection and Negative Evidence in Automated Text Categorization

Cited by 165 publications

References 6 publications

Named Entities as New Features for Czech Document Classification

Named Entities as New Features for Czech Document Classification

Text and Hypertext Categorization

A Hybrid Statistical Data Pre-processing Approach for Language-Independent Text Classification

Contact Info

Product

Resources

About