Big data classification

Abstract

This paper focuses on the specific problem of Big Data classification of network intrusion traffic. It discusses the system challenges presented by the Big Data problems associated with network intrusion prediction. The prediction of a possible intrusion attack in a network requires continuous collection of traffic data and learning of their characteristics on the fly. The continuous collection of traffic data by the network leads to Big Data problems that are caused by the volume, variety and velocity properties of Big Data. The learning of the network characteristics require machine learning techniques that capture global knowledge of the traffic patterns. The Big Data properties will lead to significant system challenges to implement machine learning frameworks. This paper discusses the problems and challenges in handling Big Data classification using geometric representation-learning techniques and the modern Big Data networking technologies. In particular this paper discusses the issues related to combining supervised learning techniques, representation-learning techniques, machine lifelong learning techniques and Big Data technologies (e.g. Hadoop, Hive and Cloud) for solving network traffic classification problems.

A survey of cloud-based network intrusion detection analysis

“…overhead vs. speed), there are several major research challenges of utilizing MLA in cloud-based network intrusion analysis. There are three salient challenges facing widespread MLA implementation in cloudbased network intrusion detection techniques [15]. First, MLAs trained on a particular datasets may not be suitable for other datasets, and that classification may not be robust over different datasets or domains.…”

“…To address this limitation, researchers have started designing scalable performant machine learning applications that run in the cloud computing environment [9][10][11][12][13][14]. In addition, many researchers emphasized the importance of machine learning algorithms (MLAs) to intrusion detection analysis using cloud computing technology [15][16][17][18]. Although the integration of MLAs to intrusion detection is important, it has not been studied broadly, primarily due to problems associated with leveraging optimal algorithms in a cloud computing environment.…”

A Big Data Analytics Approach for Construction Firms Failure Prediction Models

“…A dataset can be taken to be Big Data when its velocity, volume and variety become so much that current technological tools make it hard to store and/or process it [34], [35]. Its size is such that it forces a search for new approaches away from the known and trusted ones [36].…”

“…In the past, say around the 80s, it would have been a data size that required 'tape monkeys'; presently, it is a data size that will require clusters of computer and/or cloud running concurrently and in a parallel mode to be analysed [37]. Big Data Analytics can be defined as involving analysis of huge data in order to unmask valuable patterns/information [35].…”

Understanding big data themes from scientific biomedical literature through topic modeling

“…In total 17 definitions of big data were considered from the following sources [3,5,6,14,15,[32][33][34][35][36][37][38][39][40][41][42][43]. Table 1 presents the results of our analysis listing the found themes, their description, and respective sources.…”