Sample-based XPath Ranking for Web Information Extraction

Abstract

Web information extraction typically relies on a wrapper, i.e., program code or a configuration that specifies how to extract some information from web pages at a specific website. Manually creating and maintaining wrappers is a cumbersome and error-prone task. It may even be prohibitive as some applications require information extraction from previously unseen websites. This paper targets automatic on-the-fly wrapper creation for websites that provide attribute data for objects in a 'search -search result page -detail page' setup. It is a wrapper induction approach which uses a small and easily obtainable set of sample data for ranking XPaths on their suitability for extracting the wanted attribute data. Experiments show that the automatically generated top-ranked XPaths indeed extract the wanted data. Moreover, it appears that 20 to 25 input samples suffice for finding a suitable XPath for an attribute.

Probabilistic Data Integration

“…A probabilistic approach may lead to more robust navigation. Subtasks like finding search results (Trieschnigg et al 2012) or finding target fields (Jundt and van Keulen 2013) are typically based on ranking "possible actions". By executing not only one but a top-k of possible actions and representing resulting data probabilistically, consequences of imperfect ranking are reduced.…”

Geosocial recommender systems

“…Alternative model mapping scrapelets could be introduced to support also websites that make less use of HTML attributes. Jundt and Van Keulen for example used a manually annotated training set to detect the respective XPaths in [51]. We used NeoGeo for a POI scraping task of 603 POI categories for the city of Enschede.…”