A Pure URL-Based Genre Classification of Web Pages

Abstract

In this paper, we propose a new approach for multi-label genre classification of web pages that exploits character n-grams extracted from the URL of the web page rather than its content. Using only the URL reduces the time needed for feature extraction since it does not need to download the content of the web page. Our approach deals with the complexity of web pages because it uses a multi-label classification where each web page can be assigned to more than one genre. Moreover, our approach implements a new weighting technique that exploits the structure of the URL. Experiments conducted on a known multi-label dataset show that our approach achieves encouraging results.

Discovering Advertisement Links by Using URL Text

“…The URL attribute refers to some simple properties of the URL on the structure and content. It mainly contains three key parts below [6]. Firstly, the URL structure attribute is formatted by the structure "protocol://host:port/path?parameter#infomation-fragment".…”

Open-set web genre identification

“…Most previous studies in WGI consider the simple case where all web pages should belong to a predefined taxonomy of genres (Lim, 2005;Santini, 2007;Kanaris and Stamatatos, 2009;Jebari, 2014). This is known as closed-set classification.…”

“…In some cases, it has been reported that the web-pages's URL alone is sufficient for predicting its genre (Abramson and Aha, 2012;Jebari, 2014;Priyatam et al, 2013;Zhu, Zhou, and Fung, 2011). Concerning available hyperlinks in web-pages there are two parts than can provide useful information: the URL of the hyperlink itself handled as a string of characters and its anchor text.…”

“…Alternatively, the structure of the graph which is formed by the hyperlinks and information found in neighboring pages can also be used. Usually, the neighbouring pages can contribute by amplifying the signals for the correct genre classification using either information extracted from their text or based on the assumption that pages of the same genre tend to be inter-linked (Abramson and Aha, 2012;Asheghi, Markert, and Sharoff, 2014;Jebari, 2014;Priyatam et al, 2013;Zhu, Zhou, and Fung, 2011).…”

“…Special characters such as { , ., ?, $, %}, top-level domains {.gr, .uk, .com, etc}, and file suffixes such as ".html", ".pdf" are usually discarded and then character n-grams are extracted from the URL counterparts. WGI experiments using only the hyperlink information combined (or not) with other web-page information seems to be a promising researching path especially for performance oriented WGI applications such as genre-based focused-crawling where only the URLs are available (Jebari, 2014;Jebari, 2015;Abramson and Aha, 2012;Priyatam et al, 2013)…”