Focused crawling with ontology using semi-automatic tagging for relevancy

Gaur, Risha; Sharma, Dilip Kumar

doi:10.1109/ic3.2014.6897224

Cited by 6 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By retrieving important or thematically relevant pages early, a focused crawler [21,22] can improve the quality of the downloaded pages. The studies concerning this category explore how a crawler can discover and identify important webpages or pages relevant to a topic [23] or user interest [10,[24][25][26][27], and propose various algorithms to achieve this goal.…”

Section: Preferential (Focused/topical)mentioning

confidence: 99%

inTIME: A Machine Learning-Based Framework for Gathering and Leveraging Web Data to Cyber-Threat Intelligence

et al. 2021

View full text Add to dashboard Cite

In today’s world, technology has become deep-rooted and more accessible than ever over a plethora of different devices and platforms, ranging from company servers and commodity PCs to mobile phones and wearables, interconnecting a wide range of stakeholders such as households, organizations and critical infrastructures. The sheer volume and variety of the different operating systems, the device particularities, the various usage domains and the accessibility-ready nature of the platforms creates a vast and complex threat landscape that is difficult to contain. Staying on top of these evolving cyber-threats has become an increasingly difficult task that presently relies heavily on collecting and utilising cyber-threat intelligence before an attack (or at least shortly after, to minimize the damage) and entails the collection, analysis, leveraging and sharing of huge volumes of data. In this work, we put forward inTIME, a machine learning-based integrated framework that provides an holistic view in the cyber-threat intelligence process and allows security analysts to easily identify, collect, analyse, extract, integrate, and share cyber-threat intelligence from a wide variety of online sources including clear/deep/dark web sites, forums and marketplaces, popular social networks, trusted structured sources (e.g., known security databases), or other datastore types (e.g., pastebins). inTIME is a zero-administration, open-source, integrated framework that enables security analysts and security stakeholders to (i) easily deploy a wide variety of data acquisition services (such as focused web crawlers, site scrapers, domain downloaders, social media monitors), (ii) automatically rank the collected content according to its potential to contain useful intelligence, (iii) identify and extract cyber-threat intelligence and security artifacts via automated natural language understanding processes, (iv) leverage the identified intelligence to actionable items by semi-automatic entity disambiguation, linkage and correlation, and (v) manage, share or collaborate on the stored intelligence via open standards and intuitive tools. To the best of our knowledge, this is the first solution in the literature to provide an end-to-end cyber-threat intelligence management platform that is able to support the complete threat lifecycle via an integrated, simple-to-use, yet extensible framework.

show abstract

Section: Preferential (Focused/topical)mentioning

confidence: 99%

inTIME: A Machine Learning-Based Framework for Gathering and Leveraging Web Data to Cyber-Threat Intelligence

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Author Gaur and Sharma (2014a), Gaur and Sharma (2014b) used an approach semi-automatic tagging to enhance the performance of the focused crawler. They generate the tags on the basis of the contents of the web pages and assigned the tags to the URLs.…”

Section: Figure 3 Generalized Framework Of the Ontology Learningmentioning

confidence: 99%

Self-Adaptive Ontology based Focused Crawler for Social Bookmarking Sites

Khan

Sharma

2017

International Journal of Information Retrieval Research

Self Cite

View full text Add to dashboard Cite

It is not possible for one person to explore or surf all the relevant websites pre-training to his/her topic. A user might not be able to get the results that he/she expects from the search engine but another user might have some knowledge about some website containing the information about the first user's topical query. Users share their information on a common sharing platform known as SBS (Social Bookmarking Sites). In SBS a user posts a question seeking some knowledge about a certain topic, and then the people who have some knowledge about any website related to the query topic post the URLs of the website. This paper presents a novel method to verify the authenticity and validity of the URL posted in the SBS. The performance of our method is further increased by using a dictionary based learning methodology that finds the contextually similar words that are added to the Ontology.

show abstract

Self-adaptive ontology-based focused crawling: A literature survey

Khan

Sharma

2016

2016 5th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO)

View full text Add to dashboard Cite

Focused crawling with ontology using semi-automatic tagging for relevancy

Cited by 6 publications

References 11 publications

inTIME: A Machine Learning-Based Framework for Gathering and Leveraging Web Data to Cyber-Threat Intelligence

inTIME: A Machine Learning-Based Framework for Gathering and Leveraging Web Data to Cyber-Threat Intelligence

Self-Adaptive Ontology based Focused Crawler for Social Bookmarking Sites

Self-adaptive ontology-based focused crawling: A literature survey

Contact Info

Product

Resources

About