RETRACTED ARTICLE: Efficiently harvesting deep web interfaces based on adaptive learning using two-phase data crawler framework

Murugudu, Madhusudhan Rao; Reddy, L. S. S.

doi:10.1007/s00500-021-05816-z

Cited by 13 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among many programming languages, Python is the most widely used scripting language for writing web crawler programs. Based on Python language, there are many excellent libraries and crawler frameworks, such as scratch, beautiful soup, Crawley, Python goose, and mechanize [15,16]. Python, an assembly language, will provide a web page interface that can be used directly to facilitate the operation of crawler code.…”

Section: Methodsmentioning

confidence: 99%

Media Content Mining Based on Artificial Intelligence and Network Interaction

Kang

Wang

Zhang

2022

Mobile Information Systems

View full text Add to dashboard Cite

In recent years, due to the explosive growth of social media information, mining hot information in social media has become a research direction of great concern. In this paper, Python crawler technology is used to crawl the semi-structured text data of food safety news from static web pages and dynamic web pages. After preprocessing, the structured text data required to establish a document clustering algorithm (CASC) based on a convolutional neural network is obtained. Using the feature extraction ability of convolutional neural network and self-encoder, while preserving the internal structure of the original data to the greatest extent, it is embedded into the low-dimensional potential space for clustering. Finally, it is compared with the performance of the K-means algorithm and spectral clustering algorithm. The experimental results show that the CASC algorithm reduces the running time and time complexity of the algorithm on the premise of ensuring clustering accuracy. The CASC algorithm is superior to the k-means algorithm and spectral clustering algorithm in precision, recall, and composite index. At the same time, the running time is 91 seconds faster than the K-Means algorithm and 5 seconds faster than the spectral clustering algorithm.

show abstract

Section: Methodsmentioning

confidence: 99%

Media Content Mining Based on Artificial Intelligence and Network Interaction

Kang

Wang

Zhang

2022

Mobile Information Systems

View full text Add to dashboard Cite

show abstract

“…Each online platform is composed of different degrees of information producers, such as organized OGC (professional content production), relatively specialised PGC (professional content production), and individual UGC (user produced content), and information creators of all kinds have more diverse information sources and information presentation methods than traditional media in the past, but rigour and standardisation are seriously lacking. In an era, when the flow is king, public figures and "big V" focus on the number of clicks, retweets, likes, and favorites, all hoping to get a share of the "attention" resources [16]. Against this backdrop, there were several problems with the coverage of Chinese people during the epidemic control period: first is about the amplification of individual incidents.…”

Section: Communication Factorsmentioning

confidence: 99%

Analysis of Chinese Image Discourse Based on Crawler Algorithms

Zhang¹

2022

Mobile Information Systems

View full text Add to dashboard Cite

The study explores the construction of China's image in the context of a major public health event such as the Newcastle pneumonia epidemic by crawling social media data in China during the epidemic prevention and control period. The study argues that, in the era of new media, especially self-media, the image of China in Chinese social media during the prevention and control of the Newcastle pneumonia epidemic presents multiple types of images, ranging from a “dual” image of those fighting and preventing the epidemic in their home country and their country of residence to a “dual” image of those being questioned and even those being “questioned” in their home country and their country of residence. “It is worth noting that in March 2020, the number of people who have been affected by the epidemic in their home countries will be increased to 3,000. It is worth mentioning that in mid to late March 2020, Chinese people in China had a serious negative image crisis, but it was quickly repaired through mechanisms such as government “stabilisers,” mainstream media “gatekeepers,” and audience “self-purification.” One of the lessons is that image repair relies on the “stabiliser,” the “gatekeeper” of mainstream media, and the “self-purification” of the audience. One of the lessons is that image repair relies on the triple factor of “real society—audience” working together. The second lesson is that while the open Internet environment can generate a lot of exaggerated and false information; at the same time, audiences, including Chinese people, can remove misunderstandings and bridge differences through active voices and debates and activate rational and healthy positive energy. The epidemic is a “danger” and an “opportunity” for China’s image.

show abstract

“…In comparison to the usual design, the implemented technique crawls, allowing for more equitable data distribution through plug-and-plug. When compared with the existing methods such as IHCM [16], SVM [17], NTPDCF [18], and SIMHAR [21], the implemented hamming distance method achieves better performance values of 99.8%, 99.9%, 98%, and 99% in terms of accuracy, precision, recall, and f-measure.…”

Section: Discussionmentioning

confidence: 97%

“…This section provides a discussion of the implemented hamming distance and compares those results with existing methods such as IHCM [16], SVM [17], NTPDCF [18], and SIMHAR [21] in the comparative analysis section. The main goal of the hamming distance is to avoid becoming the bottleneck in the pipeline, these algorithms must be fast and efficient.…”

Section: Discussionmentioning

confidence: 99%

“…However, SVM-based classifier's heterogeneous behavior and insufficient labeled data, compiling a representative sample of malicious web crawlers was difficult. Murugudu and Reddy [18] implemented a novel and efficient two-phase deep learning data crawler framework (NTPDCF), for intelligent crawling of search data to produce a large variety of effectively matched content. The twophase data crawler architecture improves data harvesting and makes it possible to successfully extract pertinent information from a variety of web sources by effectively collecting targeted data from deep web interfaces.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Efficient intelligent crawler for hamming distance based on prioritization of web documents

Dange,

Byranahalli Eraiah,

Rao

et al. 2024

IJECE

View full text Add to dashboard Cite

Search engines play a crucial role in today's Internet landscape, especially with the exponential increase in data storage. Ranking models are used in search engines to locate relevant pages and rank them in decreasing order of relevance. They are an integral component of a search engine. The offline gathering of the document is crucial for providing the user with more accurate and pertinent findings. With the web’s ongoing expansions, the number of documents that need to be crawled has grown enormously. It is crucial to wisely prioritize the documents that need to be crawled in each iteration for any academic or mid-level organization because the resources for continuous crawling are fixed. The advantages of prioritization are implemented by algorithms designed to operate with the existing crawling pipeline. To avoid becoming the bottleneck in pipeline, these algorithms must be fast and efficient. A highly efficient and intelligent web crawler has been developed, which employs the hamming distance method for prioritizing the pages to be downloaded in each iteration. This cutting-edge search engine is specifically designed to make the crawling process more streamlined and effective. When compared with other existing methods, the implemented hamming distance method achieves a high value of 99.8% accuracy.

show abstract

RETRACTED ARTICLE: Efficiently harvesting deep web interfaces based on adaptive learning using two-phase data crawler framework

Cited by 13 publications

References 13 publications

Media Content Mining Based on Artificial Intelligence and Network Interaction

Media Content Mining Based on Artificial Intelligence and Network Interaction

Analysis of Chinese Image Discourse Based on Crawler Algorithms

Efficient intelligent crawler for hamming distance based on prioritization of web documents

Contact Info

Product

Resources

About