Bigdata logs analysis based on seq2seq networks for cognitive Internet of Things

Wu, Pin; Lu, Zhihui; Zhou, Quan; Lei, Zhidan; Li, Xiaoqiang; Qiu, Meikang; Hung, Patrick C. K.

doi:10.1016/j.future.2018.08.021

Cited by 40 publications

(9 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…VI. BACKGROUND Wu et al [11] presented a method for analyzing the bigdata collected through tools e.g. Hadoop, Mapreduce, Hive and others based on seq2seq-predictive models.…”

Section: B Results and Analysismentioning

confidence: 99%

See 1 more Smart Citation

Collaborative adversary nodes learning on the logs of IoT devices in an IoT network

Aneja¹,

Xuan²,

Aneja³

2021

Preprint

View full text Add to dashboard Cite

Artificial Intelligence (AI) development has encouraged many new research areas, including AI-enabled Internet of Things (IoT) network. AI analytics and intelligent paradigms greatly improve learning efficiency and accuracy. Applying these learning paradigms to network scenarios provide technical advantages of new networking solutions. In this paper, we propose an improved approach for IoT security from data perspective. The network traffic of IoT devices can be analyzed using AI techniques. The Adversary Learning (AdLIoTLog) model is proposed using Recurrent Neural Network (RNN) with attention mechanism on sequences of network events in the network traffic. We define network events as a sequence of the time series packets of protocols captured in the log. We have considered different packets TCP packets, UDP packets, and HTTP packets in the network log to make the algorithm robust. The distributed IoT devices can collaborate to cripple our world which is extending to Internet of Intelligence. The time series packets are converted into structured data by removing noise and adding timestamps. The resulting data set is trained by RNN and can detect the node pairs collaborating with each other. We used the BLEU score to evaluate the model performance. Our results show that the predicting performance of the AdLIoTLog model trained by our method degrades by 3-4% in the presence of attack in comparison to the scenario when the network is not under attack. AdLIoTLog can detect adversaries because when adversaries are present the model gets duped by the collaborative events and therefore predicts the next event with a biased event rather than a benign event. We conclude that AI can provision ubiquitous learning for the new generation of Internet of Things.

show abstract

“…VI. BACKGROUND Wu et al [11] presented a method for analyzing the bigdata collected through tools e.g. Hadoop, Mapreduce, Hive and others based on seq2seq-predictive models.…”

Section: B Results and Analysismentioning

confidence: 99%

“…Log analysis using the Recurrent Neural Network (RNN) method has been [4,11] studied to predict future events. In ns-2 network simulator, two network scenarios were set up to generate data for the proposed study.…”

Section: Introductionmentioning

confidence: 99%

Collaborative adversary nodes learning on the logs of IoT devices in an IoT network

Aneja¹,

Xuan²,

Aneja³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“… The datasets will help researchers in optimizing and analyzing network communication used by IoT devices in an IoT network, e.g. device identification [1 , 2] , and device log prediction [3] are some of the techniques developed for IoT security by analyzing the network communication behaviour. The given IoT datasets can be analyzed by researchers in the context of network bandwidth and storage capability required for home or office automation since data were recorded in a separate file every day.…”

Section: Value Of the Datamentioning

confidence: 99%

Section: Value Of the Datamentioning

confidence: 99%

Packet-level and IEEE 802.11 MAC frame-level network traffic traces data of the D-Link IoT devices

Chowdhury

Aneja

et al. 2021

Data in Brief

View full text Add to dashboard Cite

With the growth of wireless network technology-based devices, identifying the communication behaviour of wireless connectivity enabled devices, e.g. Internet of Things (IoT) devices, is one of the vital aspects, in managing and securing IoT networks. Initially, devices use frames to connect to the access point on the local area network and then, use packets of typical communication protocols through the access point to communicate over the Internet. Toward this goal, network packet and IEEE 802.11 media access control (MAC) frame analysis may assist in managing IoT networks efficiently, and allow investigation of inclusive behaviour of IoT devices. This paper presents network traffic traces data of D-Link IoT devices from packet and frame levels. Data collection experiment has been conducted in the Network Systems and Signal Processing (NSSP) laboratory at Universiti Brunei Darussalam (UBD). All the required devices, such as IoT devices, workstation, smartphone, laptop, USB Ethernet adapter, and USB WiFi adapter, have been configured accordingly, to capture and store network traffic traces of the 14 IoT devices in the laboratory. These IoT devices were from the same manufacture (D-Link) with different types, such as camera, home-hub, door-window sensor, and smart-plug.

show abstract

“…A seq2seq model-based approach was used to improve query focused summarization performance [19]. Wu et al described a novel method to create, store, and convert logs of Internet of ings big data systems to be later processed through their proposed seq2seq algorithm [20]. e method has also been applied in manufacturing systems.…”

Section: Introductionmentioning

confidence: 99%

Event‐Tree Based Sequence Mining Using LSTM Deep‐Learning Model

2021

View full text Add to dashboard Cite

During the operation of modern technical systems, the use of the LSTM model for the prediction of process variable values and system states is commonly widespread. The goal of this paper is to expand the application of the LSTM-based models upon obtaining information based on prediction. In this method, by predicting transition probabilities, the output layer is interpreted as a probability model by creating a prediction tree of sequences instead of just a single sequence. By further analyzing the prediction tree, we can take risk considerations into account, extract more complex prediction, and analyze what event trees are yielded from different input sequences, that is, with a given state or input sequence, the upcoming events and the probability of their occurrence are considered. In the case of online application, by utilizing a series of input events and the probability trees, it is possible to predetermine subsequent event sequences. The applicability and performance of the approach are demonstrated via a dataset in which the occurrence of events is predetermined, and further datasets are generated with a higher-order decision tree-based model. The case studies simply and effectively validate the performance of the created tool as the structure of the generated tree, and the determined probabilities reflect the original dataset.

show abstract

Bigdata logs analysis based on seq2seq networks for cognitive Internet of Things

Cited by 40 publications

References 16 publications

Collaborative adversary nodes learning on the logs of IoT devices in an IoT network

Collaborative adversary nodes learning on the logs of IoT devices in an IoT network

Packet-level and IEEE 802.11 MAC frame-level network traffic traces data of the D-Link IoT devices

Event‐Tree Based Sequence Mining Using LSTM Deep‐Learning Model

Contact Info

Product

Resources

About