Classification-Based Adaptive Web Scraper

Ujwal, B V S; Gaind, Bharat; Kundu, Amit; Holla, Anusha; Rungta, Mukund

doi:10.1109/icmla.2017.0-168

Cited by 8 publications

(2 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some of the available web data extraction techniques, that are used in such adaptive systems are discussed below. Web data extraction is explored using repetitive blocks [11] , with their respective attributes obtained from classification-based approaches. This data extraction technique demonstrates good accuracy and adaptability to layout changes in websites.…”

Section: Introductionmentioning

confidence: 99%

Intelligent and adaptive web data extraction system using convolutional and long short-term memory deep learning networks

Patnaik

Babu

Bhave³

2021

Big Data Min. Anal.

View full text Add to dashboard Cite

Data are crucial to the growth of e-commerce in today's world of highly demanding hyper-personalized consumer experiences, which are collected using advanced web scraping technologies. However, core data extraction engines fail because they cannot adapt to the dynamic changes in website content. This study investigates an intelligent and adaptive web data extraction system with convolutional and Long Short-Term Memory (LSTM) networks to enable automated web page detection using the You only look once (Yolo) algorithm and Tesseract LSTM to extract product details, which are detected as images from web pages. This state-of-the-art system does not need a core data extraction engine, and thus can adapt to dynamic changes in website layout. Experiments conducted on real-world retail cases demonstrate an image detection (precision) and character extraction accuracy (precision) of 97% and 99%, respectively. In addition, a mean average precision of 74%, with an input dataset of 45 objects or images, is obtained.

show abstract

Section: Introductionmentioning

confidence: 99%

Intelligent and adaptive web data extraction system using convolutional and long short-term memory deep learning networks

Patnaik

Babu

Bhave³

2021

Big Data Min. Anal.

View full text Add to dashboard Cite

show abstract

“…Another use-case could be to filter all the unnecessary offers received by the user (as SMS messages on his phone) to give him/her personalized offers and avoid clutter. Yet another use-case could be a continuation of the work done by Ujwal et al [1], which proposes a method to scrape offers from offer-aggregator websites. The Hybrid Model we propose could be used to extract meaningful entities from these scraped offers.…”

Section: Introductionmentioning

confidence: 99%

Hybrid NER System for Multi-Source Offer Feeds

Holla¹,

Gaind²,

Katta³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Data available across the web is largely unstructured. Offers published by multiple sources like banks, digital wallets, merchants, etc., are one of the most accessed advertising data in today's world. This data gets accessed by millions of people on a daily basis and is easily interpreted by humans, but since it is largely unstructured and diverse, using an algorithmic way to extract meaningful information out of these offers is hard. Identifying the essential offer entities (for instance, its amount, the product on which the offer is applicable, the merchant providing the offer, etc.) from these offers plays a vital role in targeting the right customers to improve sales. This work presents and evaluates various existing Named Entity Recognizer (NER) models which can identify the required entities from offer feeds. We also propose a novel Hybrid NER model constructed by two-level stacking of Conditional Random Field, Bidirectional LSTM and Spacy models at the first level and an SVM classifier at the second. The proposed hybrid model has been tested on offer feeds collected from multiple sources and has shown better performance in the offer domain when compared to the existing models.

show abstract