Understanding query interfaces by statistical parsing

Su, Weifeng; Hejun, Wu; Li, Yafei; Zhao, Jing; Lochovsky, Frederick H.; Cai, Hongming; Huang, Tianqiang

doi:10.1145/2460383.2460387

Cited by 11 publications

(9 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since search forms are designed for human-computer interactions [27], the access to the deep Web requires simulating such interactions with an automated process. Any agent that interacts with search forms needs to fulfil two tasks [86]: form modelling and query selection. Form modelling involves understanding the search form to be able to discern the semantics associated to each field, the type and domain of the values that are expected in each field, and the relationship between fields.…”

Section: Form Fillingmentioning

confidence: 99%

“…Figure 1 Deep Web crawling workflow access to the deep Web requires simulating these interactions, hopefully without the intervention from the user. A human user that interacts with search forms needs to understand the search form to be able to discern the semantics associated to each field, the type and domain of the values that are expected in each field, and the relationship between fields (form modelling), and provide a combination of values of the proper type and domain to fill in each field, so that submitting the form using those values yields as a response results that are relevant for the user (query selection) [86]. In some cases, a crawler needs to go further, navigating from those results pages into the deep Web site, by following a crawling path that allows it to reach the deep Web pages.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Web crawling: a survey

2018

View full text Add to dashboard Cite

Deep Web crawling refers to the problem of traversing the collection of pages in a deep Web site, which are dynamically generated in response to a particular query that is submitted using a search form. To achieve this, crawlers need to be endowed with some features that go beyond merely following links, such as the ability to automatically discover search forms that are entry points to the deep Web, fill in such forms, and follow certain paths to reach the deep Web pages with relevant information. Current surveys that analyse the state of the art in deep Web crawling do not provide a framework that allows comparing the most up-to-date proposals regarding all the different aspects involved in the deep Web crawling process. In this article, we propose a framework that analyses the main features of existing deep Web crawling-related techniques, including the most recent proposals, and provides an overall picture regarding deep Web crawling, including novel features that to the present day had not been analysed by previous surveys. Our main conclusion is that crawler evaluation is an immature research area due to the lack of a standard set of performance measures, or a benchmark or publicly available dataset to evaluate the crawlers. In addition, we conclude that the future work in this area should be focused on devising crawlers to deal with ever-evolving Web technologies and improving the crawling efficiency and scalability, in order to create effective crawlers that can operate in real-world contexts.

show abstract

Section: Form Fillingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Deep Web crawling: a survey

2018

View full text Add to dashboard Cite

show abstract

“…And our aim is to find WVoptimal from the solution space of Inequalities (2). Intuitively, we have to solve these n(n -1) inequalities, a right (not optimal) WV can be the output.…”

Section: Inequalities-based Metricsmentioning

confidence: 99%

“…In another word, the solution space continues shrinking during the process and a WV in the solution space of Inequalities (3) has more probability to be in the solution space of Inequalities (2). Actually, there is more than one inequality which does not satisfy Inequalities (2), but only one inequality (Inequality (4)) is appended at every iteration, due to the consideration of efficiency improvement.…”

Section: Step 1 Computing Wv In Inequalitiesmentioning

confidence: 99%

See 1 more Smart Citation

Duplicate Literature Detection for Cross-Library Search

Liu

Jianxun

2016

Cybernetics and Information Technologies

View full text Add to dashboard Cite

Schema Extraction for Deep Web Query Interfaces Using Heuristics Rules

Jou¹

2018

Inf Syst Front

View full text Add to dashboard Cite

Along with the popularity of the world wide web, data volumes inside web databases have been increasing tremendously. These deep web contents, hidden behind the query interfaces, are of much better quality than those in the surface web. Internet users need to fill in query conditions in the HTML query interface and click the submit button to obtain deep web data. Many deep web contents related applications, like named entity attribute collection, topic-focused crawling, and heterogeneous data integration, are based on understanding schema of these query interfaces. The schema needs to cover mappings of input elements and labels, data types of valid input values, and range constraints of the input values. Additionally, to extract these hidden data, the schema needs to include many form submission related information, like cookies and action types. We design and implement a Heuristics-based deep web query interface Schema Extraction system (HSE). In HSE, texts surrounding elements are collected as candidate labels. We propose a string similarity function and use a dynamic similarity threshold to cleanse candidate labels. In HSE, elements, candidate labels, and new lines in the query interface are streamlined to produce its Interface Expression (IEXP). By combining the user's view and the designer's view, with the aid of semantic information, we build heuristic rules to extract schema from IEXP of query interfaces in the ICQ dataset. These rules are constructed through utilizing (1) the characteristics of labels and elements, and (2) the spatial, group, and range relationships of labels and elements. Supplemented with form submission related information, the extracted schemas are then stored in the XML format, so that they could be utilized in further applications, like schema matching and merging for federated query interface integration. The experimental results on the TEL-8 dataset illustrate that HSE produces effective performance.

show abstract

Understanding query interfaces by statistical parsing

Cited by 11 publications

References 33 publications

Deep Web crawling: a survey

Deep Web crawling: a survey

Duplicate Literature Detection for Cross-Library Search

Schema Extraction for Deep Web Query Interfaces Using Heuristics Rules

Contact Info

Product

Resources

About