Characterizing big data software architectures

Sena, Bruno; Allian, Ana Paula; Nakagawa, Elisa Yumi

doi:10.1145/3132498.3132510

Cited by 16 publications

(13 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bolster [16] achieved the highest score, whereas SOLID [14], the RA by Pääkkönen and Pakkala [4], Lambda, and Kappa achieved good results. Also, RAs were systematically studied based on literature, and analysed in terms of architectural requirements, modules, layers, and patterns [5]. Additionally, RAs for big data systems have been developed by including security aspects [17], targeting plant genotyping and phenotyping systems [18], and government enterprise architecture frameworks [19].…”

Section: Related Workmentioning

confidence: 99%

“…Security, monitoring, and management layers are among the aspects [5], which are missing from the developed RA. Also, a lacking metadata layer has been seen as a drawback of the developed RA [6].…”

Section: Future Workmentioning

confidence: 99%

“…

…”

mentioning

confidence: 99%

See 2 more Smart Citations

Extending reference architecture of big data systems towards machine learning in edge computing environments

Pääkkönen

Pakkala

2020

J Big Data

View full text Add to dashboard Cite

IntroductionMany big data systems have been developed and realised to provide end user services (Netflix, Facebook, Twitter, LinkedIn etc.). Also, underlying architectures and technologies of the enabling systems have been published [1-3], and RAs have been designed and proposed [4][5][6]. Edge/5G computing is an emerging technological field [7], and the first products are being shipped to the markets. However, the utilisation of machine learning (ML) as part of the edge computing infrastructure is still an area for further research [8]. Particularly, it should be understood, how data is collected, and how models are Abstract Background: Augmented reality, computer vision and other (e.g. network functions, Internet-of-Things (IoT)) use cases can be realised in edge computing environments with machine learning (ML) techniques. For realisation of the use cases, it has to be understood how data is collected, stored, processed, analysed, and visualised in big data systems. In order to provide services with low latency for end users, often utilisation of ML techniques has to be optimized. Also, software/service developers have to understand, how to develop and deploy ML models in edge computing environments. Therefore, architecture design of big data systems to edge computing environments may be challenging. Findings:The contribution of this paper is reference architecture (RA) design of a big data system utilising ML techniques in edge computing environments. An earlier version of the RA has been extended based on 16 realised implementation architectures, which have been developed to edge/distributed computing environments. Also, deployment of architectural elements in different environments is described. Finally, a system view is provided of the software engineering aspects of ML model development and deployment. Conclusions:The presented RA may facilitate concrete architecture design of use cases in edge computing environments. The value of RAs is reduction of development and maintenance costs of systems, reduction of risks, and facilitation of communication between different stakeholders.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Future Workmentioning

confidence: 99%

“…

…”

mentioning

confidence: 99%

See 1 more Smart Citation

Extending reference architecture of big data systems towards machine learning in edge computing environments

Pääkkönen

Pakkala

2020

J Big Data

View full text Add to dashboard Cite

show abstract

“…Sena et al [13] have explored a basic issue in the area of software architectures for BD-namely how software architectures of these systems can be characterized in terms of their modules and organizations, basic requirements, and main characteristics. In order to present the state of the art for BD software architectures, they conducted a systematic mapping study.…”

Section: Software Architecture For Big Datamentioning

confidence: 99%

Big Data and software engineering: prospects for mutual enrichment

Arndt

2017

Iran J Comput Sci

View full text Add to dashboard Cite

Software engineering has evolved over the last 50 years, initially as a response to the so-called software crisis (the problems that organizations had producing quality software systems on time and on budget) of the 1960s and 1970s. Software engineering (SE) has been defined as "the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software; that is, the application of engineering to software". Software engineering has developed a number of approaches to areas such as software requirements, software design, software testing, and software maintenance. Software development processes such as the waterfall model, incremental development, and the spiral model have been successfully applied to produce high-quality software on time and under budget. More recently, agile software development has gained popularity as an alternative to the more traditional development methods for development of complex systems. Within the last decade or so, advances in technologies such as mobile computing, social networks, cloud computing, and the Internet of things have given rise to massive datasets which have been given the name Big Data (BD). Big Data has been defined as data with 3Vs-high volume, velocity, and variety. Big Data contains so much data that low probability events are captured in the data. These events can be discovered using analytics methods and turned into actionable intelligence which can be used by businesses to gain a competitive advantage. Unfortunately, the very scale of BD often renders inadequate SQL-based relational database systems which have formed the backbone of data intensive systems for the last 30 years, requiring new NoSQL technologies to be effective. In this paper, we will explore how well-established SE technology can be adapted to support successful development of BD projects, as well as how BD techniques can be used to increase the utility of SE processes and techniques. Thus, BD and SE may mutually support and enrich each other.

show abstract

“…Problem Statement focusing on the industry, Wirth and Hipp (2000) proposed CRISP-DM, a more dynamical process focused on the business aspects of the data discovery and composed of six steps: (i) business understanding; (ii) data understanding; (iii) data preparation; (iv) modeling; (v) evaluation; ad (vi) deployment. Furthermore, some architectural approaches have been proposed to handle the big data context and knowledge discovery in an architectural level, i.e., trying to support the design of systems which enable knowledge discovery (SENA et al, 2017); however, few of these approaches show mature results to support such knowledge discovery in SoS.…”

mentioning

confidence: 99%

KnowSoS: A Software Architecture for Knowledge Discovery in Systems-of-Systems

Silva¹

View full text Add to dashboard Cite

Before starting this Master's program, I thought it would be easy. I thought I knew everything. I thought I was ready. I thought it was going to be simple. I thought it was going to be quick. But, over time, I realized that this journey was much more than just developing a project. This journey was about personal growth. It was about intellectual development. It was about the humility of accepting that I don't know anything. It was about learning to deal with different types of people. It was about receiving "no" and not giving up. But above all, it was about strengthening bonds with those who were by my side, and it is to these people that I offer my sincere gratitude.First of all, to God, for giving me wisdom to embrace opportunities, patience to await victories, and strength to face difficulties. My heart will be forever grateful for your compassion.To my mother, Rosemeire, for being by my side at all times, for laughing and crying with me, for the advice when I needed it and for believing that one day I would be here. You're everything to me! To my whole family, and especially, to Talita Fontanezi and Feliphi Sena, for the company, love and words of affection.To my advisor, Elisa Yumi Nakagawa, for adopting me when I knew nothing, and teaching me much of what I know today.To my best friend, Carolina "Buzz", for comforting me in every moment of crisis, for not letting me give up at any time, and for always using the right words to support me. You are the exactly definition of what a best friend should be.To the friends and co-supervisors that this journey gave me, Maria Lydia Fioravanti and Ana Paula Allian, who advised me, guided me, and supported me throughout my (eternal) drama. I couldn't have done it without you.

show abstract

Characterizing big data software architectures

Cited by 16 publications

References 34 publications

Extending reference architecture of big data systems towards machine learning in edge computing environments

Extending reference architecture of big data systems towards machine learning in edge computing environments

Big Data and software engineering: prospects for mutual enrichment

KnowSoS: A Software Architecture for Knowledge Discovery in Systems-of-Systems

Contact Info

Product

Resources

About