Adaptive, scalable and reliable monitoring of big data on clouds

Andreolini, Mauro; Colajanni, Michele; Pietri, Marcello; Tosi, Stefania

doi:10.1016/j.jpdc.2014.08.007

Cited by 42 publications

(16 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Real-time monitoring of cloud resources is crucial for a variety of tasks including performance analysis, workload management, capacity planning and fault detection (Andreolini et al 2015). Progress has been made in cloud system monitoring and tracking.…”

Section: Cloud Monitoring and Trackingmentioning

confidence: 99%

“…For example, Yang et al (2015a) investigated the challenges posed by industrial Big Data and complex machine working conditions and proposed a framework for implementing cloud-based, machine health prognostics. To limit computational and communication costs and guarantee high reliability in capturing relevant load changes, Andreolini et al (2015) presented an adaptive algorithm for monitoring Big Data applications that adapts the intervals of sampling and frequency of updates to data characteristics and administrator needs. Bae et al (2014) proposed an intrusive analyzer that detects interesting events (such as task failure) occurring in the Hadoop system.…”

Section: Cloud Monitoring and Trackingmentioning

confidence: 99%

“…Developing tools to precisely measure usage of resources, including computing and data for pricing purposes (Gui et al 2014); . Monitoring of Big Data in the cloud to optimized selection and activation (Andreolini et al 2015…”

Section: Selection Activation and Pricingmentioning

confidence: 99%

See 2 more Smart Citations

Big Data and cloud computing: innovation opportunities and challenges

Yang

Huang

et al. 2016

International Journal of Digital Earth

645

289

View full text Add to dashboard Cite

Big Data has emerged in the past few years as a new paradigm providing abundant data and opportunities to improve and/or enable research and decision-support applications with unprecedented value for digital earth applications including business, sciences and engineering. At the same time, Big Data presents challenges for digital earth to store, transport, process, mine and serve the data. Cloud computing provides fundamental support to address the challenges with shared computing resources including computing, storage, networking and analytical software; the application of these resources has fostered impressive Big Data advancements. This paper surveys the two frontiers-Big Data and cloud computingand reviews the advantages and consequences of utilizing cloud computing to tackling Big Data in the digital earth and relevant science domains. From the aspects of a general introduction, sources, challenges, technology status and research opportunities, the following observations are offered: (i) cloud computing and Big Data enable science discoveries and application developments; (ii) cloud computing provides major solutions for Big Data; (iii) Big Data, spatiotemporal thinking and various application domains drive the advancement of cloud computing and relevant technologies with new requirements; (iv) intrinsic spatiotemporal principles of Big Data and geospatial sciences provide the source for finding technical and theoretical solutions to optimize cloud computing and processing Big Data; (v) open availability of Big Data and processing capability pose social challenges of geospatial significance and (vi) a weave of innovations is transforming Big Data into geospatial research, engineering and business values. This review introduces future innovations and a research agenda for cloud computing supporting the transformation of the volume, velocity, variety and veracity into values of Big Data for local to global digital earth science and applications.

show abstract

Section: Cloud Monitoring and Trackingmentioning

confidence: 99%

Section: Cloud Monitoring and Trackingmentioning

confidence: 99%

See 1 more Smart Citation

Big Data and cloud computing: innovation opportunities and challenges

Yang

Huang

et al. 2016

International Journal of Digital Earth

645

289

View full text Add to dashboard Cite

show abstract

“…Real-time monitoring, gating, filtering, and throttling of streaming data requires new approaches due to the "variety of tasks, such as performance analysis, workload management, capacity planning, and fault detection. Applications producing Big Data make the monitoring task very difficult at high-sampling frequencies because of high computational and communication overheads [13]." • Provisioning and package management activities to support automated deployment and configuration of software and services.…”

Section: System Managementmentioning

confidence: 99%

NIST Big Data Interoperability Framework:

Subgroup.¹

2019

View full text Add to dashboard Cite

NIST Special Publication series 1500 is intended to capture external perspectives related to NIST standards, measurement, and testing-related efforts. These external perspectives can come from industry, academia, government, and others. These reports are intended to document external perspectives and do not represent official NIST positions. Certain commercial entities, equipment, or materials may be identified in this document to describe an experimental procedure or concept adequately. Such identification is not intended to imply recommendation or endorsement by NIST, nor is it intended to imply that the entities, materials, or equipment are necessarily the best available for the purpose. There may be references in this publication to other publications currently under development by NIST in accordance with its assigned statutory responsibilities. The information in this publication, including concepts and methodologies, may be used by federal agencies even before the completion of such companion publications. Thus, until each publication is completed, current requirements, guidelines, and procedures, where they exist, remain operative. For planning and transition purposes, federal agencies may wish to closely follow the development of these new publications by NIST. Organizations are encouraged to review all publications during public comment periods and provide feedback to NIST. All NIST publications are available at http://www.nist.gov/publication-portal.cfm.

show abstract

“…Scholars including computer scientists, physicists, economists, mathematicians, political scientists, bio-informaticists, and sociologists are clamoring for access to the massive quantities of Big Data produced by and about people, things, and their interactions (Boyd & Crawford, 2012). Research on Big Data related to computer science has focused on system performance and scalability technology, such as virtualization, Hadoop, MapReduce, and security for increases in heterogeneous environments (Andreolini, Colajanni, Pietri, & Tosi, 2015;Kshetri, 2014). Few studies have focused on Big Data for system availability, DR in particular (Clitherow et al, 2008;Serrelis & Alexandris, 2006).…”

Section: Big Data and Disaster Recoverymentioning

confidence: 99%

Disaster Recovery Site Evaluations and Selections for Information Systems of Academic Big Data

Yang

Huang²,

Kao

et al. 2017

EURASIA J MATH SCI T

View full text Add to dashboard Cite

The most dramatic factor shaping the future of higher education is Big Data and analytics. In the Big Data era, the explosive growth of massive data manipulations imposes a heavy burden on computation, storage, and communication in data centers. Increasing uncertainties in information system availability have become a daily serious problem. An appropriate evaluation and selection of the right information system disaster recovery (DR) site can ensure business continuity and investment optimization. However, most academic institutes always neglect the importance of DR. Not to mention the DR sites in the era of Big Data. Existing research results do not evaluate or select DR sites in general or those for academic Big Data applications in particular. Therefore, this research aims to establish an analytic framework for evaluating, selecting DR sites for academic Big Data. The proposed analytic framework is consisting of the Decision-Making Trial and Evaluation Laboratory (DEMATEL), DEMATEL-based network process (DNP) and VIšekriterijumsko KOmpromisno Rangiranje (VIKOR) methods. An empirical study based on a real Big Data DR application of an Asian high-performance computer center's evaluation and selection of DR sites for academic Big Data is used to illustrate the feasibility of the proposed framework. The analytic results can serve as a foundation for information technology administrators' strategies to reduce the performance gaps of a DR site for Big Data manipulations in general, and academic Big Data manipulations in special.

show abstract

Adaptive, scalable and reliable monitoring of big data on clouds

Cited by 42 publications

References 22 publications

Big Data and cloud computing: innovation opportunities and challenges

Big Data and cloud computing: innovation opportunities and challenges

NIST Big Data Interoperability Framework:

Disaster Recovery Site Evaluations and Selections for Information Systems of Academic Big Data

Contact Info

Product

Resources

About