Term Weighting Based on Index of Genre for Web Page Genre Classification

Abstract

Automating the identification of the genre of web pages becomes an important area in web pages

Open-set web genre identification

“…In addition, there are some schemes specifically designed for WGI tasks like Term Frequency -Inverted Genre Frequency (TF-IGF). This is an extension of TF-IDF that is based on the frequencies of a term in the documents of particular genre rather than the whole corpus (Sugiyanto et al, 2014).…”

“…In general, genre is related to form and communicative purpose of texts rather than their theme. It is closely related to style and Genus 1 (Sugiyanto et al, 2014). Approaches to define text genre start mainly from two directions: linguistics and computational analysis of language (e.g.…”

“…Although TF-IDF is a popular choice in many text mining studies, the study of (Sugiyanto et al, 2014) demonstrates that it is not the best choice for WGI tasks. On the contrary, they propose a genre-specific weighting scheme, called TF-IGF.…”

“…where f (t, g) is the frequency of term t in genre g and g f (t) is the genre frequency of t. Since TF-IGF depends on genre, the average value over all genres in a given palette is finally used. The TF-IGF score can be used to select the most informative features that highlight genre-related information and reported results show that it is a better criterion for feature selection in comparison to regular TF-IDF (Sugiyanto et al, 2014).…”

“…Instead of learning a simple model, ensemble methods attempt to extract several base models and then combine them. One main direction is to use well-known ensemble learning methods such as AdaBoost, Bagging and Random Forests (Sugiyanto et al, 2014;Onan, 2018;Worsham and Kalita, 2018). This approach can easily handle high-dimensional representations and heterogeneous features.…”