Large scale real-time water quality monitoring system usually produces vast amounts of high frequency data, and it is difficult for traditional water quality monitoring system to process such large and high frequency data generated by wireless sensor network. A real-time processing and early warning system framework is proposed to solve this problem, Apache Storm is used as the big data processing platform, and Kafka message queue is applied to classify the sample data into several data streams so as to reserve the time series data property of a sensor. In storm platform, Daubechies Wavelet is used to decompose the data series to obtain the trend of the series, then Long Short Term Memory Network (LSTM) model is used to model and predict the trend of the data. This paper provides a detailed description concerning the distribution mechanism of aggregated data in Storm, data storage format in HBase, the process of wavelet decomposition, model training and the application of mode for prediction. The application results in Xin’an River in Yantai City reveal that the prosed system framework has a very good ability to model big data with high prediction accuracy and robust processing capability.

No abstract

With the development of the Internet of Things and the popularity of smart terminal devices, mobile crowdsourcing systems are receiving more and more attention. However, the information overload of crowdsourcing platforms makes workers face difficulties in task selection. This paper proposes a task recommendation model based on the prediction of workers’ mobile trajectories. A recurrent neural network is used to obtain the movement pattern of workers and predict the next destination. In addition, an attention mechanism is added to the task recommendation model in order to capture records that are similar to candidate tasks and to obtain task selection preferences. Finally, we conduct experiments on two real datasets, Foursquare and AMT (Amazon Mechanical Turk), to verify the effectiveness of the proposed recommendation model.