Hierarchical Multi-log Cloud-Based Search Engine

Abstract

Having become the leading trend in IT infrastructure, service delivering, and multi-layered resource sharing, cloud services typically include SaaS (Software as a service), PaaS (Platform as a service) and Iaas (Infrastructure as a service). With the increasing popularity of cloud computing, users store large amounts of data as documents, text files, databases, and more relevant to this work, system logs. Current cloud services are getting more decoupled with each layer in the cloud stack generating different logs for network, applications, database, and programming interfaces on different machines. At any point in time, cloud providers, users, or application developers arguably require to understand the status of different components, monitor business processes, and analyse machine logs in real time. However, there are no specialised search engines for the systematic analysis of logs by different cloud providers. Hence, this paper presents Simha, an agent-based document search service for cloud platforms. It implements a proof of concept system to analyse user documents, logs, and folders in real time from different virtual machines. Based on an Elasticsearch server, our overall search process has been extended to distributively search data stored into cloud. So, we propose an application which looks for data in private cloud and public clouds. In this paper, we describe its design and implementation. We have obtained initial encouraging results, and we further discuss how to extend our scheme in several ways.