Big Data Systems Meet Machine Learning Challenges: Towards Big Data Science as a Service

Elshawi, Radwa; Sakr, Sherif; Talia, Domenico; Trunfio, Paolo

doi:10.1016/j.bdr.2018.04.004

Cited by 117 publications

(70 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…There is a substantial difference between the recall percentage of two-class SVM and two-class locally deep SVM (89.6% and 99.3% respectively). Two class SVM"s capability to detect false alarms has not been impressive since its FAR is reported to be as high as 16.5%. On the other hand, locally deep SVM has been comparatively better in reducing false alarms due to the application of sigmoid kernel.…”

Section: Logistic Regressionmentioning

confidence: 99%

“…Performance and scalability are the two major considerations for conducting network intrusion detection study. Big data processing platforms like Pig [13], Spark machine learning [14] and Azure machine learning [15] are the preferred choices in ISSN: 2088-8708  Performance analysis of binary and multiclass models using azure machine learning (Smitha Rajagopal) 979 the modern scenario given their ability to uphold memory requirements and implementation essentials [16]. Going by these considerations, it is imperative to introduce radical advancements to intrusion detection infrastructure.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Performance analysis of binary and multiclass models using azure machine learning

Rajagopal

Hareesha

Kundapur

2020

IJECE

View full text Add to dashboard Cite

Network data is expanding and that too at an alarming rate. Besides, the sophisticated attack tools used by hackers lead to capricious cyber threat landscape. Traditional models proposed in the field of network intrusion detection using machine learning algorithms emphasize more on improving attack detection rate and reducing false alarms but time efficiency is often overlooked. Therefore, in order to address this limitation, a modern solution has been presented using Machine Learning-as-a-Service platform. The proposed work analyses the performance of eight two-class and three multiclass algorithms using UNSW NB-15, a modern intrusion detection dataset. 82,332 testing samples were considered to evaluate the performance of algorithms. The proposed two class decision forest model exhibited 99.2% accuracy and took 6 seconds to learn 1,75,341 network instances. Multiclass classification task was also undertaken wherein attack types like generic, exploits, shellcode and worms were classified with a recall percentage of 99%, 94.49%, 91.79% and 90.9% respectively by the multiclass decision forest model that also leapfrogged others in terms of training and execution time.

show abstract

Section: Logistic Regressionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Performance analysis of binary and multiclass models using azure machine learning

Rajagopal

Hareesha

Kundapur

2020

IJECE

View full text Add to dashboard Cite

show abstract

“…The concept of big data generated an ever-increasing pressure for scalable data processing solutions, leading to the implementation of several data processing and management systems recently. A summary of the major features of big data analysis frameworks is presented in Table 1 [29]. The increasing data analysis requirements in virtually all application areas have necessitated need to design and build a new set of big data science tools which can seamlessly analyze huge data volumes, extract vital information, and establish important patterns and knowledge from such datasets [30].…”

Section: Big Data Analysis Frameworkmentioning

confidence: 99%

A novel approach for big data processing using message passing interface based on memory mapping

2019

View full text Add to dashboard Cite

The analysis and processing of big data are one of the most important challenges that researchers are working on to find the best approaches to handle it with high performance, low cost and high accuracy. In this paper, a novel approach for big data processing and management was proposed that differed from the existing ones; the proposed method employs not only the memory space to reads and handle big data, it also uses space of memory-mapped extended from memory storage. From a methodological viewpoint, the novelty of this paper is the segmentation stage of big data using memory mapping and broadcasting all segments to a number of processors using a parallel message passing interface. From an application viewpoint, the paper presents a high-performance approach based on a homogenous network which works parallelly to encrypt-decrypt big data using AES algorithm. This approach can be done on Windows Operating System using .NET libraries.

show abstract

“…In this literature reviews, we would like to focus on data science frameworks as follows: [17] captured data science, data source, data scale, data story, and data scientists; [18] explained about data science expertise, venn diagrams, goals and deliverables, process, skills and education, data analysts and data engineers, also the data scientist's toolbox; [19] described data science definition, comparing data science with data analysis, process of data science, tools, skills, scope, advantages, how data science is different from big data; [20] explained data insight, data product, the skill set requirement, analytics and machine learning; [21] emphasized the life cycle of data science and data scientist profile; [22] presented structured and unstructured data, business intelligence and data science, life cycle, model planning and building tools; [23] reviewed about statistics and associated data science methods in bioimage informatics; [24] emphasized software for supporting big data science for data scientists and big data analytics frameworks based on clouds;…”

Section: B Data Sciencementioning

confidence: 99%

A New framework of e-Participation using data science to support community empowerment

Yusuf¹,

Herawati²,

Imamah³

2018

Proceedings of the International Conference on Science and Technology (ICST 2018)

View full text Add to dashboard Cite

e-Participation is an interesting field capturing how technology can support community empowerment in public decision making. Data Science is also an emerging field that uses methods, processes, algorithms and systems to develop pattern, knowledge and insights from tructured and unstructured data. It is still limited research capturing and integrating e-participation and data science into one framework. This paper aims to develop a new framework that integrate e-participation and data science frameworks. Hopefully, this research contributes a new framework of e-Participation using data science approach. The novel framework is collaborating technological and nontechnological aspects, multidisciplinary approaches and accommodating the emerging technologies nowadays. This research has implications for theory by adding the new framework into e-Participation, e-Government, Information Systems, Data Science fields. Another important practical implication is that practitioners and decision makers in government could consider elements in the framework to get successful in the technology implementation for supporting citizen participation

show abstract

Big Data Systems Meet Machine Learning Challenges: Towards Big Data Science as a Service

Cited by 117 publications

References 43 publications

Performance analysis of binary and multiclass models using azure machine learning

Performance analysis of binary and multiclass models using azure machine learning

A novel approach for big data processing using message passing interface based on memory mapping

A New framework of e-Participation using data science to support community empowerment

Contact Info

Product

Resources

About