Big Data prediction framework for weather Temperature based on MapReduce algorithm

Ismail, Khalid Adam; Majid, Mazlina Abdul; Zain, Jasni Mohamad; Bakar, Noor Akma Abu

doi:10.1109/icos.2016.7881981

Cited by 20 publications

(8 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…What is instead required is advanced distributed management systems, where parts of the data are stored in several warehouses and can be accessed through special software such as Hadoop. Ismail et al [32] proposed a new prediction framework for Big Data analytics based on the Hadoop MapReduce algorithm. Hadoop is considered an effective platform that offers efficient functionalities for processing and storing of large amounts of data.…”

Section: Big 'Weather' Data and Challengesmentioning

confidence: 99%

A Dynamic Neural Network Architecture with Immunology Inspired Optimization for Weather Data Forecasting

et al. 2018

View full text Add to dashboard Cite

Recurrent neural networks are dynamical systems that provide for memory capabilities to recall past behaviour, which is necessary in the prediction of time series. In this paper, a novel neural network architecture inspired by the immune algorithm is presented and used in the forecasting of naturally occurring signals, including weather big data signals. Big Data Analysis is a major research frontier, which attracts extensive attention from academia, industry and government, particularly in the context of handling issues related to complex dynamics due to changing weather conditions. Recently, extensive deployment of IoT, sensors, and ambient intelligence systems led to an exponential growth of data in the climate domain. In this study, we concentrate on the analysis of big weather data by using the Dynamic Self Organized Neural Network Inspired by the Immune Algorithm. The learning strategy of the network focuses on the local properties of the signal using a self-organised hidden layer inspired by the immune algorithm, while the recurrent links of the network aim at recalling previously observed signal patterns. The proposed network exhibits improved performance when compared to the feedforward multilayer neural network and state-of-the-art recurrent networks, e.g., the Elman and the Jordan networks. Three non-linear and non-stationary weather signals are used in our experiments. Firstly, the signals are transformed into stationary, followed by 5-steps ahead prediction. Improvements in the prediction results are

show abstract

Section: Big 'Weather' Data and Challengesmentioning

confidence: 99%

A Dynamic Neural Network Architecture with Immunology Inspired Optimization for Weather Data Forecasting

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Similarly big data analytics has brought about significant impact in many other domains such as climate change analysis, weather pattern analysis, real estate markets, etc. (Barkham, Bokhari, & Saiz, ; Faghmous & Vipin, ; Ismail, Majid, Zain, & Bakar, ).…”

Section: Big Data Analyticsmentioning

confidence: 99%

Smart city big data analytics: An advanced review

Soomro

Bhutta

Khan

et al. 2019

WIREs Data Min & Knowl

View full text Add to dashboard Cite

With the increasing role of ICT in enabling and supporting smart cities, the demand for big data analytics solutions is increasing. Various artificial intelligence, data mining, machine learning and statistical analysis-based solutions have been successfully applied in thematic domains like climate science, energy management, transport, air quality management and weather pattern analysis. In this paper, we present a systematic review of the literature on smart city big data analytics. We have searched a number of different repositories using specific keywords and followed a structured data mining methodology for selecting material for the review. We have also performed a technological and thematic analysis of the shortlisted literature, identified various data mining/machine learning techniques and presented the results. Based on this analysis we also present a classification model that studies four aspects of research in this domain. These include data models, computing models, security and privacy aspects and major market drivers in the smart cities domain. Moreover, we present a gap analysis and identify future directions for research. For the thematic analysis we identified the themes smart city governance, economy, environment, transport and energy. We present the major challenges in these themes, the major research work done in the field of data analytics to address these challenges and future research directions.

show abstract

“…Big data made immense business and interpersonal opportunities in each area, empowering the disclosure of formerly invisible patterns and the advancements of new insights for decision making [9]. The keyword Big Data is currently utilized wherever in our regular day to day existence and it is a recent development and besides going to control the world in future and has grown in light of the way that people and various associations makes growing use of data raised advances [10]. Present scenario big data sizes are running from a TB to ZB in individual data set.…”

Section: Big Datamentioning

confidence: 99%

An Adaptive Model for Forecasting Seasonal Rainfall Using Predictive Analytics

Reddy¹,

Sureshbabu²

2019

IJIES

View full text Add to dashboard Cite

India is a country which has exemplary climate circumstances comprising of different seasons and topographical conditions like high temperatures, cold atmosphere, and drought, heavy rainfall seasonal wise. These utmost varieties in climate make us exact weather prediction is a challenging task. Majority people of the country depend on agriculture. Farmers require climate information to decide the planting. Weather prediction turns into orientation in the farming sector to decide the start of the planting season and furthermore quality and amount of their harvesting. One of the variables are influencing agriculture is rainfall. The main goal of this project is early and proper rainfall forecasting, that helpful to people who live in regions which are inclined natural calamities such as floods and it helps agriculturists for decision making in their crop and water management using big data analytics which produces high in terms of profit and production for farmers. In this project, we proposed an advanced automated framework called Enhanced Multiple Linear Regression Model (EMLRM) with MapReduce algorithm and the Hadoop file system. In the proposed model (EMLRM) first, we stored the unstructured weather data in hadoop distributed file system (HDFS), process that stored data by using MapReduce Algorithm and build the rainfall prediction model by utilizing Multiple Linear Regression.We used climate data from IMD (Indian Metrological Department, Hyderabad) in 1901 to 2002 period. The experimental outcomes show that the EMLRM provided the lowest value of Root Mean Square Error (RMSE= 0.274) and Mean Absolute Error (MAE= 0.0745) compared with existing methods. The results of the analysis will help the farmers to adopt effective modeling approach for predicting long-term seasonal rainfall.

show abstract

Big Data prediction framework for weather Temperature based on MapReduce algorithm

Cited by 20 publications

References 4 publications

A Dynamic Neural Network Architecture with Immunology Inspired Optimization for Weather Data Forecasting

A Dynamic Neural Network Architecture with Immunology Inspired Optimization for Weather Data Forecasting

Smart city big data analytics: An advanced review

An Adaptive Model for Forecasting Seasonal Rainfall Using Predictive Analytics

Contact Info

Product

Resources

About