Robust Web Data Extraction Based on Unsupervised Visual Validation

Potvin, Benoit; Villemaire, Roger

doi:10.1007/978-3-030-14799-0_7

Cited by 6 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The idea here is how different types of content are formatted 'relative to each other' on a product listing web page can be consistent across many different domains and websites. This has been validated in different contexts such as [72]. The abundance of already annotated product listing pages in the form of semantic markup data can create an opportunity to train such taggers in a self-supervised way.…”

Section: Discussionmentioning

confidence: 99%

An Exploratory Study on Utilising the Web of Linked Data for Product Data Mining

Zhang

Song

2022

SN COMPUT. SCI.

View full text Add to dashboard Cite

The Linked Open Data practice has led to a significant growth of structured data on the Web. While this has created an unprecedented opportunity for research in the field of Natural Language Processing, there is a lack of systematic studies on how such data can be used to support downstream NLP tasks. This work focuses on the e-commerce domain and explores how we can use such structured data to create language resources for product data mining tasks. To do so, we process billions of structured data points in the form of RDF n-quads, to create multi-million words of product-related corpora that are later used in three different ways for creating language resources: training word-embedding models, continued pre-training of BERT-like language models, and training machine translation models that are used as a proxy to generate product-related keywords. These language resources are then evaluated in three downstream tasks, product classification, linking, and fake review detection using an extensive set of benchmarks. Our results show word embeddings to be the most reliable and consistent method to improve the accuracy on all tasks (with up to 6.9% points in macro-average F1 on some datasets). Contrary to some earlier studies that suggest a rather simple but effective approach such as building domain-specific language models by pre-training using in-domain corpora, our work serves a lesson that adapting these methods to new domains may not be as easy as it seems. We further analyse our datasets and reflect on how our findings can inform future research and practice.

show abstract

Section: Discussionmentioning

confidence: 99%

An Exploratory Study on Utilising the Web of Linked Data for Product Data Mining

Zhang

Song

2022

SN COMPUT. SCI.

View full text Add to dashboard Cite

show abstract

“…The idea here is how different types of content are formatted 'relative to each other' on a product listing web page can be consistent across many different domains and websites. This has been validated in different contexts such as Potvin and Villemaire (2019). The abundance of already annotated product listing pages in the form of semantic markup data can create an opportunity to train such taggers in a self-supervised way.…”

Section: Discussionmentioning

confidence: 99%

An Exploratory Study on Utilising the Web of Linked Data for Product Data Mining

Zhang¹,

Song²

2021

Preprint

View full text Add to dashboard Cite

The Linked Open Data practice has led to a significant growth of structured data on the Web in the last decade. Such structured data describe real-world entities in a machine-readable way, and have created an unprecedented opportunity for research in the field of Natural Language Processing. However, there is a lack of studies on how such data can be used, for what kind of tasks, and to what extent they can be useful for these tasks. This work focuses on the e-commerce domain to explore methods of utilising such structured data to create language resources that may be used for product classification and linking. We process billions of structured data points in the form of RDF n-quads, to create multi-million words of product-related corpora that are later used in three different ways for creating of language resources: training word embedding models, continued pretraining of BERT-like language models, and training Machine Translation models that are used as a proxy to generate product-related keywords. Our evaluation on an extensive set of benchmarks shows word embeddings to be the most reliable and consistent method to improve the accuracy on both tasks (with up to 6.9 percentage points in macro-average F1 on some datasets). The other two methods however, are not as useful. Our analysis shows that this could be due to a number of reasons, including the biased domain representation in the structured data and lack of vocabulary coverage. We share our datasets and discuss how our lessons learned could be taken forward to inform future research in this direction.Keywords linked data • web of data • schema.org • natural language processing • nlp • data mining • product mining

show abstract

“…To overcome the challenges associated with web data extraction, an extractor is presented based on supervised and unsupervised methods to extract the specific product description from the websites [39]. An unsupervised technique called visual validation was combined with a supervised classifier to produce a versatile extractor that works on a variety of websites.…”

Section: Related Workmentioning

confidence: 99%

Exploring the Topological Properties of the Tor Dark Web

et al. 2021

View full text Add to dashboard Cite

The web graph can be used to get insight into the internal structure and connectivity of the Tor dark web. This paper analyzes the internal structure of the Tor dark web graph and examines the presence of bow-tie structure as found in the World Wide Web. The web graph is generated from the data collected by the Python crawler customized to scrape data from the Tor dark web. Each of the nodes in the graph represents an individual Tor hidden service, and an edge denotes the hyperlink from one hidden service to the other. Various graph metrics are then computed and analyzed for both directed and undirected graphs using the Python NetworkX package. It was found that most of the nodes of the graph have in-degree and out-degree less than ten. The presence of power-law in degree distribution could neither be confirmed nor denied. The Tor web graph is sparse with a few connected pairs of nodes. Like the surface web, the dark web can also be decomposed into a bow-tie structure though with small component sizes. Several important and well-known websites on the surface web have incoming links from the dark web. Moreover, the Tor network also shows the characteristics of small-world and scale-free networks.INDEX TERMS Bow tie, dark web, graph analysis, Tor, web graph.

show abstract

Robust Web Data Extraction Based on Unsupervised Visual Validation

Cited by 6 publications

References 20 publications

An Exploratory Study on Utilising the Web of Linked Data for Product Data Mining

An Exploratory Study on Utilising the Web of Linked Data for Product Data Mining

An Exploratory Study on Utilising the Web of Linked Data for Product Data Mining

Exploring the Topological Properties of the Tor Dark Web

Contact Info

Product

Resources

About