Today's real time big data applications mostly rely on map-reduce (M-R) framework of Hadoop File System (HDFS). Hadoop makes the complexity of such applications in a simpler manner. This paper works on two goals: maximizing resource utilization and reducing the overall job completion time. Based on the goals proposed, we have developed Agent Centric Enhanced Reinforcement Learning Algorithm (AGERL) .The algorithm concentrates in four dimensions: variable partitioning of tasks, calculation of progress ratio of processing tasks including delays, XMPP based multi attribute query posting and Hopkins statistics assessment based dynamic cluster restructuring. An Enhanced Reinforcement Learning Process with the above features is employed to achieve the proposed goal. Finally performance gain is theoretically proved.

In the heterogeneous parallel and distributed computing environments like cloud there were many related approaches proposed for fault tolerant execution of workflows. Most of the earlier works involved does not depend on failure prediction of the resources that is really hard to achieve with the tracing of historic failure data over years of the desired environment. In this paper, to solve the software fault prediction, unavailability of the resources and monitoring problems we propose a failure prediction model that involves two different methods. In order to predict the failures at the nodes we propose a method using Intelligent Platform Management Interface (IPMI), that monitor the failure at nodes and provide the respective data that is useful for determining likely imminent failures. The other method is to predict the Unavailability of the resources from past behavior that generates some initial results that indicate that nodes are different from one another and their failure is somewhat predictable and monitoring is performed which intimates about the failure.

No abstract

No abstract