Utility-Driven Proactive Management of Availability in Enterprise-Scale Information Flows

Abstract

Abstract. Enterprises rely critically on the timely and sustained delivery of information. To support this need, we augment information flow middleware with new functionality that provides high levels of availability to distributed applications while at the same time maximizing the utility end users derive from such information. Specifically, the paper presents utility-driven 'proactive availabilitymanagement' techniques to offer (1) information flows that dynamically selfdetermine their availability requirement based on high-level utility specifications, (2) flows that can trade recovery time for performance based on the 'perceived' stability of and failure predictions (early alarm) for the underlying system, and (3) methods, based on real-world case studies, to deal with both transient and non-transient failures. Utility-driven 'proactive availability-management' is integrated into information flow middleware and used with representative applications. Experiments reported in the paper demonstrate middleware capability to self-determine availability guarantees, to offer improved performance versus a statically configured system, and to be resilient to a wide range of faults.

Replica placement for high availability in distributed stream processing systems

“…Handling failures not during the placement but during the stream processing application execution is a different and rather complicated problem, considering the request rates and the real-time requirements of the applications. Several mechanisms have been proposed to address this problem, including checkpointing [10,16], masking [32], logging [15], and trading-off consistency [5]. Since our work focuses on placement to minimize failure probability and not on handling failures during execution, existing solutions for the latter can be integrated in our architecture.…”

“…Existing research in the area of high availability for distributed stream processing systems [5,10,15,16,32] has focused on efficient replica state maintenance to mask component failures. To this extent, recovery mechanisms [15], failure masking [32], consistency trade-offs [5], and checkpoint scheduling [10,16] have been explored.…”

“…Existing work in this area has focused on tolerating failures during application execution despite the continuous arrival of data and on fast recovery despite the significant state maintenance overhead. In this context recovery mechanisms [15], failure masking [32], consistency trade-offs [5], and scheduling of checkpoints [10,16] have been investigated.…”

Flexible and Dynamic Replication Control for Interdependent Distributed Real-Time Embedded Systems

“…Even if errors are detected in some of the replicas, the non-erroneous ones will still be able to produce results within the deadlines. On the negative side, running several replicas simultaneously is costly and can be infeasible or undesirable in distributed embedded systems [5] due to the limited resource availability and excessive overhead. Thus, a different approach is needed.…”

“…Software replication has some advantages over other fault tolerance solutions in distributed environments, providing the shortest recovery delays, it is less intrusive with respect to execution time, it scales much better, and is relatively generic and transparent to the application domain [4]. However, actively replicating all software components, independently of their significance to the overall system, may be infeasible in some embedded systems due to the scale of their timing, cost, and resource constraints [5].…”