Statistical Approach for Term Weighting in Very Short Documents for Text Categorization

Timonen, Mika; Kasari, Melissa

doi:10.1007/978-3-642-54105-6_1

Cited by 3 publications

(9 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to limited input space (character limitations) and severe time-pressure under which data is collected, maintenance records are frequently much shorter than that typically considered in literature (Mukherjee and Chakraborty, 2007;Chen and Nayak, 2007). Most text-mining research is focussed on documents with more than 100 words, such as the common Reuters-21578 dataset with an average document length of 160 words (Timonen, 2012). In contrast, Chen and Nayak (2007) report records ranging from 1-50 words and even more extreme Mukherjee and Chakraborty (2007) with 5-10 words.…”

Section: Maintenance Specific Propertiesmentioning

confidence: 99%

“…Until recently, Twitter messages were limited to 140 characters with an average length of 34 characters per tweet (Perez, 2018), making it much shorter than more typical text-mining corpora. The growing impact of these social networking sites has made the classification of such short-form corpora an increasingly important research topic (Timonen, 2012;Bermingham and Smeaton, 2010).…”

Section: Jqme 293mentioning

confidence: 99%

“…The authors investigating such short-from corpora have noted significant differences in the effectiveness of various techniques used in the broader text classification literature. The most notable of these include the choice of algorithm, use of n-grams, feature representation scheme, stemming, stop-word removal and document length normalisation (Bermingham and Smeaton, 2010;Wang and Manning, 2012;Timonen, 2012). These are discussed from three perspectives: the domain specific maintenance literature, the broader short-form literature and the general text classification literature.…”

Section: Jqme 293mentioning

confidence: 99%

“…They also state that while the performance of the MNB is typically superior to the Bernoulli variant, the performance becomes more comparable for short documents (Wang and Manning, 2012). However, Timonen (2012), who also compared the performance of various algorithms on short-form corpora, found SVM the clear winner followed by Naı €ve Bayes. But, in support of Baharudin et al (2010)…”

Section: Choice Of Algorithmsmentioning

confidence: 99%

“…Because there are so few words per document, each word typically occurs only once per document, regardless of its importance, making traditional frequency-based feature weighting schemes such as TF and TFIDF ineffective. Timonen (2012) calls this the TF 5 1 challenge. If the majority of words occur only once, all words will be given the same importance weighting in the term frequency weighting scheme regardless of its information content.…”

Section: Feature Representation Schemementioning

confidence: 99%

See 4 more Smart Citations

Text mining techniques for identifying failure modes

Malan

Jooste

2023

JQME

View full text Add to dashboard Cite

PurposeThe purpose of this paper is to compare the effectiveness of the various text mining techniques that can be used to classify maintenance work-order records into their respective failure modes, focussing on the choice of algorithm and preprocessing transforms. Three algorithms are evaluated, namely Bernoulli Naïve Bayes, multinomial Naïve Bayes and support vector machines.Design/methodology/approachThe paper has both a theoretical and experimental component. In the literature review, the various algorithms and preprocessing techniques used in text classification is considered from three perspectives: the domain-specific maintenance literature, the broader short-form literature and the general text classification literature. The experimental component consists of a 5 × 2 nested cross-validation with an inner optimisation loop performed using a randomised search procedure.FindingsFrom the literature review, the aspects most affected by short document length are identified as the feature representation scheme, higher-order n-grams, document length normalisation, stemming, stop-word removal and algorithm selection. However, from the experimental analysis, the selection of preprocessing transforms seemed more dependent on the particular algorithm than on short document length. Multinomial Naïve Bayes performs marginally better than the other algorithms, but overall, the performances of the optimised models are comparable.Originality/valueThis work highlights the importance of model optimisation, including the selection of preprocessing transforms. Not only did the optimisation improve the performance of all the algorithms substantially, but it also affects model comparisons, with multinomial Naïve Bayes going from the worst to the best performing algorithm.

show abstract

Section: Maintenance Specific Propertiesmentioning

confidence: 99%

Section: Jqme 293mentioning

confidence: 99%

Section: Jqme 293mentioning

confidence: 99%

Section: Choice Of Algorithmsmentioning

confidence: 99%

Section: Feature Representation Schemementioning

confidence: 99%

See 3 more Smart Citations

Text mining techniques for identifying failure modes

Malan

Jooste

2023

JQME

View full text Add to dashboard Cite

show abstract

Multi-view Representation Induced Kernel Ensemble Support Vector Machine

Quayson

Ganaa²,

Zhu

et al. 2023

Neural Process Lett

View full text Add to dashboard Cite

IDF for Word N-grams

Shirakawa

Hara

Nishio

2017

ACM Trans. Inf. Syst.

View full text Add to dashboard Cite

Inverse Document Frequency (IDF) is widely accepted term weighting scheme whose robustness is supported by many theoretical justifications. However, applying IDF to word N-grams (or simply N-grams) of any length without relying on heuristics has remained a challenging issue. This article describes a theoretical extension of IDF to handle N-grams. First, we elucidate the theoretical relationship between IDF and information distance, a universal metric defined by the Kolmogorov complexity. Based on our understanding of this relationship, we propose N-gram IDF, a new IDF family that gives fair weights to words and phrases of any length. Based only on the magnitude relation of N-gram IDF weights, dominant N-grams among overlapping N-grams can be determined. We also propose an efficient method to compute the N-gram IDF weights of all N-grams by leveraging the enhanced suffix array and wavelet tree. Because the exact computation of N-gram IDF provably requires significant computational cost, we modify it to a fast approximation method that can estimate weight errors analytically and maintain application-level performance. Empirical evaluations with unsupervised/supervised key term extraction and web search query segmentation with various experimental settings demonstrate the robustness and language-independent nature of the proposed N-gram IDF.

show abstract

Statistical Approach for Term Weighting in Very Short Documents for Text Categorization

Cited by 3 publications

References 7 publications

Text mining techniques for identifying failure modes

Text mining techniques for identifying failure modes

Multi-view Representation Induced Kernel Ensemble Support Vector Machine

IDF for Word N-grams

Contact Info

Product

Resources

About