Geo-identification of web users through logs using ELK stack

Prakash, T. Bharani; Kakkar, Misha; Patel, Kritika

doi:10.1109/confluence.2016.7508191

Cited by 37 publications

(11 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The ELK stack, i.e., Elasticsearch, Logstash, and Kibana, automatically collects, indexes, aggregates, and visualizes logs [18]. Prakash et al efficiently geo-identified the website user traffic through logs using ELK stack [19]. Bagnasco et al used the Elasticsearch ecosystem to monitor the Infrastructure as a Service (IaaS) and scientific application on the cloud, keeping track of usage and dynamic allocation of resources [20].…”

Section: Related Workmentioning

confidence: 99%

A Cloud-Based Framework for Large-Scale Log Mining through Apache Spark and Elasticsearch

Jiang

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

The volume, variety, and velocity of different data, e.g., simulation data, observation data, and social media data, are growing ever faster, posing grand challenges for data discovery. An increasing trend in data discovery is to mine hidden relationships among users and metadata from the web usage logs to support the data discovery process. Web usage log mining is the process of reconstructing sessions from raw logs and finding interesting patterns or implicit linkages. The mining results play an important role in improving quality of search-related components, e.g., ranking, query suggestion, and recommendation. While researches were done in the data discovery domain, collecting and analyzing logs efficiently remains a challenge because (1) the volume of web usage logs continues to grow as long as users access the data; (2) the dynamic volume of logs requires on-demand computing resources for mining tasks; (3) the mining process is compute-intensive and time-intensive. To speed up the mining process, we propose a cloud-based log-mining framework using Apache Spark and Elasticsearch. In addition, a data partition paradigm, logPartitioner, is designed to solve the data imbalance problem in data parallelism. As a proof of concept, oceanographic data search and access logs are chosen to validate performance of the proposed parallel log-mining framework.

show abstract

Section: Related Workmentioning

confidence: 99%

A Cloud-Based Framework for Large-Scale Log Mining through Apache Spark and Elasticsearch

Jiang

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…However, most of the log management products require technical expertise to process and interpret logs data. A study of performance testing was done by comparing three log analytics systems namely Graylog2, ELK, and ELSA (Prakash & Patel, 2016). In the same way, various web log analyzer tools namely Webalizer, Piwik, Open Web Analytics, Deep Log Analyzer, Fire Stats, Go Access, Web Forensic, AW Log Analyzer, WebLog Expert, and Google Analytic were listed based on its core features such as languages, current stable version, specialization, strength, including some criticism (Brian Jackson, 2017;Čegan & Filip, 2017;Kumar & Thakur, 2017;Shakti & Garg, 2017;Valency Networks, 2016).…”

Section: Comparative Study Of Log Analysis Domainsmentioning

confidence: 99%

Visualizing Web Server Logs Insights With Elastic Stack– A Case Study of Ummail’s Access Logs

Muhammad¹,

Zain²

2018

MJoC

View full text Add to dashboard Cite

One of the most significant information resources that often overlooked and it is mostly owned by the modern organization today is logs data. Likewise, logs data analytics is practised in many industries for different purposes, including website/system performance improvement, web development, information architecture, web-based campaigns/programs, network traffic monitoring, e-commerce optimization, marketing/advertising, etc. Many tools or approaches are available for this purpose, some are proprietary and some are open source. Studying the nature of these tools in finding the suitable and the right log analyzer in order to perform log analytics economically, efficiently and effectively will give advantages to the organization towards utilizing the primary source of information for identifying the system threats and problems that occur in the system at any time through Visualizing Insights of source using Elastic Stack. These kinds of threats and problems which existed in the system can be identified by analyzing the log file and finding the patterns for possible suspicious behaviour. A case study of UMMAIL’s access logs is proposed to visualise web server logs. The system administrator's concern can then be furnished with an appropriate infographics representation regarding these security threats and problems in the system, which are generated after the log files, are analysed. Based on this signs the administrator can take appropriate actions.

show abstract

“…Big Data can bring vital information and value to the organizations, if data is well processed in real-time. [4] describes suggests ELK stack (Elasticsearch, Logstash and Kibana) to handle Big Data.…”

Section: Background and Related Workmentioning

confidence: 99%

Pipeline for Real-time Anomaly Detection in Log Data Streams using Apache Kafka and Apache Spark

Poojitha¹,

Sowmyarani²

2018

IJCA

View full text Add to dashboard Cite

Anomaly detection is a standout amongst the most critical assignments so as to construct a system that is trustworthy and secure. The aim of anomaly detection is to detect significant deviation of the system behavior from that of the normal behavior. This approach is broadly used on static data, for instance on dumps of log data. Most systems require a real-time detection of anomalies with a specific end goal to lessen the harm that can be caused by the ignorance of an anomaly or detection at a later time. The recent implementations of the anomaly detection are mostly based on self-learning methods. Machine learning has brought about a significant transformation in the field of anomaly detection. One of the methodologies for anomaly detection depends on clustering algorithms. The implementation discussed in this paper utilizes a timeseries evaluation approach for anomaly detection. The paper explains the pipeline built for anomaly detection and the visualization of the results.

show abstract

Geo-identification of web users through logs using ELK stack

Cited by 37 publications

References 1 publication

A Cloud-Based Framework for Large-Scale Log Mining through Apache Spark and Elasticsearch

A Cloud-Based Framework for Large-Scale Log Mining through Apache Spark and Elasticsearch

Visualizing Web Server Logs Insights With Elastic Stack– A Case Study of Ummail’s Access Logs

Pipeline for Real-time Anomaly Detection in Log Data Streams using Apache Kafka and Apache Spark

Contact Info

Product

Resources

About