Feature selection methods for event detection in Twitter: a text mining approach

Hossny, Ahmad Hany; Mitchell, Lewis; Lothian, Nick; Osborne, Grant

doi:10.1007/s13278-020-00658-3

Cited by 28 publications

(17 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Peng & Fan [52] 2017 By optimizing lower bound of conditional mutual information SFR [54] 2018 Uses subspace feature clustering to identify feature clusters CFS [55] 2018 Similar to MRMR and uses composition of feature relevancy Wang et al [59] 2019 Uses rough set theory based relative neighborhood self-information on both lower and upper approximations. PRFS [60] 2020 Proportional Rough Feature Selection based on rough set for regional distinction Liu et al [61] 2020 Independent feature space search using relative doc-term frequency difference for class correlation and redundancy Hossny et al [62] 2020 Uses text mining specifics e.g., word count, word forms such as n-gram, skip-gram, etc. Gao et al [65] 2020 min-redundancy and max-dependency (MRMD) using relevancy with a class given selected features…”

Section: Selection Methods Year Key Idea/advantage/applicationmentioning

confidence: 99%

“…We restrict ourselves to those methods using mutual information or related scores, and omit other approaches such as [56], [57] which are based on search using evolutionary strategies. More recent methods have also been proposed, particularly those using information theory and suitable for high-dimensional data such as in [58]- [62].…”

Section: ) Composition Of Feature Relevancy (Cfs)mentioning

confidence: 99%

See 1 more Smart Citation

A Fast Non-Redundant Feature Selection Technique for Text Data

et al. 2020

View full text Add to dashboard Cite

Feature selection is critical in reducing the size of data and improving classifier accuracy by selecting an optimum subset of the overall features. Traditionally, each feature is given a score against a particular category (such as using Mutual Information) and the task of feature selection comes down to choosing the top k ranked features with the best average score across all categories. However, this approach has two major drawbacks. Firstly, the maximum or average score of a feature with a class might not necessarily determine its discriminating strength among samples of other classes. Secondly, most feature selection methods only use the scores to select the discriminating features from the corpus without taking into account the redundancy of information provided by the selected features. In this paper, we propose a new feature ranking score measure called the Discriminative Mutual Information (DMI) score. This score helps to select features that distinguish samples of one category against all other categories. Moreover, Non-Redundant Feature Selection (NRFS) heuristic is also proposed that explicitly takes the problem of feature redundancy into account when selecting the features set. The performance of our approach is investigated and compared with other feature selection techniques on datasets derived from high-dimensional text corpora using multiple classification algorithms. The results show that the proposed method leads to better classification micro-F1 score as compared to other state-of-the-art methods. In particular, the proposed method shows great improvement when the number of selected features are small as well as an overall higher robustness to label noise.

show abstract

Section: Selection Methods Year Key Idea/advantage/applicationmentioning

confidence: 99%

Section: ) Composition Of Feature Relevancy (Cfs)mentioning

confidence: 99%

A Fast Non-Redundant Feature Selection Technique for Text Data

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Automatic keyphrase extraction is the process of identifying representative phrases in a document that summarize its content. Keyphrases are important pieces of information for many applications, including information retrieval (Ji et al, 2019;Boudin et al, 2020), text classification (Meng et al, 2019), text summarization (Song et al, 2019), entity recognition (Du et al, 2018) and event detection (Hossny et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

Keyphrase Extraction from Scientific Articles via Extractive Summarization

Kontoulis¹,

Papagiannopoulou²,

Tsoumakas³

2021

Proceedings of the Second Workshop on Scholarly Document Processing

View full text Add to dashboard Cite

Automatically extracting keyphrases from scholarly documents leads to a valuable concise representation that humans can understand and machines can process for tasks, such as information retrieval, article clustering and article classification. This paper is concerned with the parts of a scientific article that should be given as input to keyphrase extraction methods. Recent deep learning methods take titles and abstracts as input due to the increased computational complexity in processing long sequences, whereas traditional approaches can also work with full-texts. Titles and abstracts are dense in keyphrases, but often miss important aspects of the articles, while full-texts on the other hand are richer in keyphrases but much noisier. To address this trade-off, we propose the use of extractive summarization models on the full-texts of scholarly documents. Our empirical study on 3 article collections using 3 keyphrase extraction methods shows promising results.

show abstract

“…Automatic keyphrase extraction is the process of identifying representative phrases in a document that summarize its content. Keyphrases are important pieces of information for many applications, including information retrieval (Ji et al, 2019;Boudin et al, 2020), text classification (Meng et al, 2019), text summarization , entity recognition (Du et al, 2018) and event detection (Hossny et al, 2020). This work focuses on keyphrase extraction from scholarly documents.…”

Section: Introductionmentioning

confidence: 99%

Proceedings of the Second Workshop on Scholarly Document Processing

2021

View full text Add to dashboard Cite

Most work on scholarly document processing assumes that the information processed is trustworthy and factually correct. However, this is not always the case. There are two core challenges, which should be addressed: 1) ensuring that scientific publications are credible -e.g. that claims are not made without supporting evidence, and that all relevant supporting evidence is provided; and 2) that scientific findings are not misrepresented, distorted or outright misreported when communicated by journalists or the general public. I will present some first steps towards addressing these problems and outline remaining challenges. BiologyWood Frogs (Rana sylvatica) are a charismatic species of frog common in much of North America. They breed in explosive choruses over a few nights in late winter to early spring. The incidence in Wood Frogs was associated with a die-off of frogs during the breeding chorus in the Sylamore District of the Ozark National Forest in Arkansas (Trauth et al., 2000). Computer ScienceLand use or cover change is a direct reflection of human activity, such as land use, urban expansion, and architectural planning, on the earth's surface caused by urbanization [1]. Remote sensing images are important data sources that can efficiently detect land changes. Meanwhile, remote sensing image-based change detection is the change identification of surficial objects or geographic phenomena through the remote observation of two or more different phases [2].

show abstract

Feature selection methods for event detection in Twitter: a text mining approach

Cited by 28 publications

References 50 publications

A Fast Non-Redundant Feature Selection Technique for Text Data

A Fast Non-Redundant Feature Selection Technique for Text Data

Keyphrase Extraction from Scientific Articles via Extractive Summarization

Proceedings of the Second Workshop on Scholarly Document Processing

Contact Info

Product

Resources

About