Big Data and Cloud Computing: A Survey of the State-of-the-Art and Research Challenges

Skourletopoulos, Georgios; Mavromoustakis, Constandinos X.; Mastorakis, George; Batalla, Jordi Mongay; Dobre, Ciprian; Panagiotakis, Spyros; Pallis, Evangelos

doi:10.1007/978-3-319-45145-9_2

Cited by 50 publications

(18 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If it is needed to collect huge amounts of data but also with the use of real-time data stream mining, a complex architecture solution for data ingestion and distributed computing is developed. In such cases, special attention is paid to the usage of cloud technologies capable of processing and storing large amounts of data or local data centers [22]. In parallel, the processes of building an Operational Intelligence solution must occur.…”

Section: Operational Intelligence Analytics and Engineering In A Technology Data-based Projectmentioning

confidence: 99%

Real-Time High-Load Infrastructure Transaction Status Output Prediction Using Operational Intelligence and Big Data Technologies

2020

View full text Add to dashboard Cite

An approach to use Operational Intelligence with mathematical modeling and Machine Learning to solve industrial technology projects problems are very crucial for today’s IT (information technology) processes and operations, taking into account the exponential growth of information and the growing trend of Big Data-based projects. Monitoring and managing high-load data projects require new approaches to infrastructure, risk management, and data-driven decision support. Key difficulties that might arise when performing IT Operations are high error rates, unplanned downtimes, poor infrastructure KPIs and metrics. The methods used in the study include machine learning models, data preprocessing, missing data imputation, SRE (site reliability engineering) indicators computation, quantitative research, and a qualitative study of data project demands. A requirements analysis for the implementation of an Operational Intelligence solution with Machine learning capabilities has been conducted and represented in the study. A model based on machine learning algorithms for transaction status code and output predictions, in order to execute system load testing, risks identification and, to avoid downtimes, is developed. Metrics and indicators for determining infrastructure load are given in the paper to obtain Operational intelligence and Site reliability insights. It turned out that data mining among the set of Operational Big Data simplifies the task of getting an understanding of what is happening with requests within the data acquisition pipeline and helps identify errors before a user faces them. Transaction tracing in a distributed environment has been enhanced using machine learning and mathematical modelling. Additionally, a step-by-step algorithm for applying the application monitoring solution in a data-based project, especially when it is dealing with Big Data is described and proposed within the study.

show abstract

Section: Operational Intelligence Analytics and Engineering In A Technology Data-based Projectmentioning

confidence: 99%

Real-Time High-Load Infrastructure Transaction Status Output Prediction Using Operational Intelligence and Big Data Technologies

2020

View full text Add to dashboard Cite

show abstract

“…Since the emergence of Cloud computing, more efforts are directed towards exploiting and leveraging cloud computing for the range of benefits and advantages it offers, mostly as services; and this is now evident in a range of services springing up e.g. Big Data-as-a-Service, analytics-as-a-service, and a host of other service offerings in the industry [36].…”

Section: Overview Of Cloud Computingmentioning

confidence: 99%

Cloud Based Collaborative Software Development: A Review, Gap Analysis and Future Directions

Ewenike

Benkhelifa

Chibelushi

2017

2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA)

View full text Add to dashboard Cite

Organizations who have transitioned their development environments to the Cloud have started realizing benefits such as: cost reduction in hardware; relatively accelerated development process via reduction of time and effort to set up development and testing environments; unified management; service and functionality expansion; on-demand provisioning and access to resources and development environments. These benefits represent only a fraction of the full potential that could be achieved via leveraging Cloud Computing for the collaborative software development process. Related efforts in this area have been mainly in the areas of: asynchronous collaboration; collaboration in isolated aspects of the Software Development process, such as coding activities; use of open-source tools for contributing, improving, and managing code, etcetera. Although these efforts represent valid contributions and important enablers, they are still missing important aspects which enable a more holistic process, with solid theoretical foundation. This paper reviews this research area, in order to better assess factors and gaps creating the need to enhance the collaborative software development process in the Cloud, to better meet the pressure to collaboratively create better cloud-agnostic applications.

show abstract

“…Cloud computing provides an information and technological pay-per-use type services [3,5,16,59]. According to the demand of users, software applications, infrastructure, and platforms are served using the Internet.…”

Section: Introductionmentioning

confidence: 99%

A Double Threshold-Based Power-Aware Honey Bee Cloud Load Balancing Algorithm

et al. 2021

View full text Add to dashboard Cite

Present-day advancement in cloud computing provides ICT infrastructure as a service on a pay per use. Cloud computing provides this infrastructure as a service and as service demand increases, service providers organize large-scale data centers with a lot of resources, and cause of huge greenhouse gases' emission. This data center's huge power demand necessitates the balancing of cloud load. To attain the optimum resource utilization, least processing time of CPU, minimal average response time, and avoiding over-load, cloud load balancing algorithms distributes workload across virtual machines. The key challenge here is to develop such a load balancing algorithm which consumes the least resources to fulfill the service demands. In this paper, a double threshold-based power-aware honey bee load balancing algorithm is proposed for the fair and even distribution of the incoming task requests to all the virtual machines. This paper compares the proposed algorithm with five widely used existing load balancing algorithms. Moreover, we have done the performance analysis using the popular CloudAnalyst simulation toolkit. Results of simulation showed that the proposed algorithm gives a note-worthy outcome for average response time, CPU cost, storage cost, memory cost, and energy consumption in cloud computing to show the resource utilization.

show abstract

Big Data and Cloud Computing: A Survey of the State-of-the-Art and Research Challenges

Cited by 50 publications

References 44 publications

Real-Time High-Load Infrastructure Transaction Status Output Prediction Using Operational Intelligence and Big Data Technologies

Real-Time High-Load Infrastructure Transaction Status Output Prediction Using Operational Intelligence and Big Data Technologies

Cloud Based Collaborative Software Development: A Review, Gap Analysis and Future Directions

A Double Threshold-Based Power-Aware Honey Bee Cloud Load Balancing Algorithm

Contact Info

Product

Resources

About