Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment

Wang, Pei; Shea, Ryan; Wang, Jiannan; Wu, Eugene

doi:10.1145/3299869.3319899

Cited by 9 publications

(2 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Madhusudan and Poonam (2017) discuss how the challenge of crawling the deep Web can be approached since it makes up approximately 96% of all Web content. Recently, Wang et al (2019) proposed SmartCrawl , which is designed to maximize the return of hidden records from a database, given a set of queries and a fixed budget (e.g., number of API calls per day). Meschenmoser et al (2016) outline and provide solutions for the common challenges when crawling scientific resources like pagination (splitting the results into pages), dynamic contents (page updates when scrolling to the bottom), and access barriers like obfuscated URL parameters and robot detection mechanisms.…”

Section: Componentsmentioning

confidence: 99%

The Search as Learning Spaceship: Toward a Comprehensive Model of Psychological and Technological Facets of Search as Learning

et al. 2022

View full text Add to dashboard Cite

Using a Web search engine is one of today’s most frequent activities. Exploratory search activities which are carried out in order to gain knowledge are conceptualized and denoted as Search as Learning (SAL). In this paper, we introduce a novel framework model which incorporates the perspective of both psychology and computer science to describe the search as learning process by reviewing recent literature. The main entities of the model are the learner who is surrounded by a specific learning context, the interface that mediates between the learner and the information environment, the information retrieval (IR) backend which manages the processes between the interface and the set of Web resources, that is, the collective Web knowledge represented in resources of different modalities. At first, we provide an overview of the current state of the art with regard to the five main entities of our model, before we outline areas of future research to improve our understanding of search as learning processes.

show abstract

Section: Componentsmentioning

confidence: 99%

The Search as Learning Spaceship: Toward a Comprehensive Model of Psychological and Technological Facets of Search as Learning

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Matching revolves around providing correspondences between concepts describing the meaning of data in various heterogeneous, distributed data sources, such as SQL and XML schemata, entity-relationship diagrams, ontology descriptions, interface definitions, etc. The need for schema matching arises in a variety of domains including linking datasets and entities for data discovery [27,57,58], finding related tables in data lakes [63], data enrichment [59], aligning ontologies and relational databases for the Semantic Web [24], and document format merging (e.g., orders and invoices in e-commerce) [52]. As an example, a shopping comparison app that supports queries such as "the cheapest computer among retailers" or "the best rate for a flight to Boston in September" requires integrating and matching several data sources of product orders and airfare forms.…”

Section: Introductionmentioning

confidence: 99%

PoWareMatch: a Quality-aware Deep Learning Approach to Improve Human Schema Matching

Shraga¹,

Gal²

2021

Preprint

View full text Add to dashboard Cite

Schema matching is a core task of any data integration process. Being investigated in the fields of databases, AI, Semantic Web and data mining for many years, the main challenge remains the ability to generate quality matches among data concepts (e.g., database attributes). In this work, we examine a novel angle on the behavior of humans as matchers, studying match creation as a process. We analyze the dynamics of common evaluation measures (precision, recall, and f-measure), with respect to this angle and highlight the need for unbiased matching to support this analysis. Unbiased matching, a newly defined concept that describes the common assumption that human decisions represent reliable assessments of schemata correspondences, is, however, not an inherent property of human matchers. In what follows, we design PoWareMatch that makes use of a deep learning mechanism to calibrate and filter human matching decisions adhering the quality of a match, which are then combined with algorithmic matching to generate better match results. We provide an empirical evidence, established based on an experiment with more than 200 human matchers over common benchmarks, that PoWareMatch predicts well the benefit of extending the match with an additional correspondence and generates high quality matches. In addition, PoWareMatch outperforms state-of-the-art matching algorithms.

show abstract

In Situ Neural Relational Schema Matcher

Du,

Yuan,

et al. 2024

2024 IEEE 40th International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment

Cited by 9 publications

References 34 publications

The Search as Learning Spaceship: Toward a Comprehensive Model of Psychological and Technological Facets of Search as Learning

The Search as Learning Spaceship: Toward a Comprehensive Model of Psychological and Technological Facets of Search as Learning

PoWareMatch: a Quality-aware Deep Learning Approach to Improve Human Schema Matching

In Situ Neural Relational Schema Matcher

Contact Info

Product

Resources

About