CCoDaMiC: A framework for Coherent Coordination of Data Migration and Computation platforms

Dehury, Chinmaya Kumar; Srirama, Satish Narayana; Chhetri, Tek Raj

doi:10.1016/j.future.2020.03.029

Cited by 20 publications

(8 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From the loosely coupled architecture of the tool and the use of scalable technologies, such as Docker and serverless, we can observe the scalability features, as this allows scaling of the individual components. Numerous studies, such as [ 108 , 109 , 110 ], have demonstrated the scalability of the loosely coupled architecture and the support provided by technologies such as Docker and serverless. Moreover, to the observed scalability of the tool from the loosely coupled architecture, we also performed a scalability testing with the Kubernetes platform, as discussed in Section 6.1 .…”

Section: Discussionmentioning

confidence: 99%

Data Protection by Design Tool for Automated GDPR Compliance Verification Based on Semantically Modeled Informed Consent

Chhetri

Kurteva

DeLong

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

The enforcement of the GDPR in May 2018 has led to a paradigm shift in data protection. Organizations face significant challenges, such as demonstrating compliance (or auditability) and automated compliance verification due to the complex and dynamic nature of consent, as well as the scale at which compliance verification must be performed. Furthermore, the GDPR’s promotion of data protection by design and industrial interoperability requirements has created new technical challenges, as they require significant changes in the design and implementation of systems that handle personal data. We present a scalable data protection by design tool for automated compliance verification and auditability based on informed consent that is modeled with a knowledge graph. Automated compliance verification is made possible by implementing a regulation-to-code process that translates GDPR regulations into well-defined technical and organizational measures and, ultimately, software code. We demonstrate the effectiveness of the tool in the insurance and smart cities domains. We highlight ways in which our tool can be adapted to other domains.

show abstract

Section: Discussionmentioning

confidence: 99%

Data Protection by Design Tool for Automated GDPR Compliance Verification Based on Semantically Modeled Informed Consent

Chhetri

Kurteva

DeLong

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Dehury et al [21] designed a framework known as CCo-DaMiC, which aims to ensure data accuracy, trustworthiness, and validation in SDP. This work directly relates to our proposed DFT based SDP approach.…”

Section: Related Workmentioning

confidence: 99%

Serverless data pipeline approaches for IoT data in fog and cloud computing

Poojara¹,

Dehury²,

Jakovits³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

With the increasing number of Internet of Things (IoT) devices, massive amounts of raw data is being generated. The latency, cost, and other challenges in cloud-based IoT data processing have driven the adoption of Edge and Fog computing models, where some data processing tasks are moved closer to data sources. Properly dealing with the flow of such data requires building data pipelines, to control the complete life cycle of data streams from data acquisition at the data source, edge and fog processing, to Cloud side storage and analytics. Data analytics tasks need to be executed dynamically at different distances from the data sources and often on very heterogeneous hardware devices. This can be streamlined by the use of a Serverless (or FaaS) cloud computing model, where tasks are defined as virtual functions, which can be migrated from edge to cloud (and vice versa) and executed in an eventdriven manner on data streams. In this work, we investigate the benefits of building Serverless data pipelines (SDP) for IoT data analytics and evaluate three different approaches for designing SDPs: 1) Off-the-shelf data flow tool (DFT) based, 2) Object storage service (OSS) based and 3) MQTT based. Further, we applied these strategies on three fog applications (Aeneas, PocketSphinx, and custom Video processing application) and evaluated the performance by comparing their processing time (computation time, network communication and disk access time), and resource utilization. Results show that DFT is unsuitable for compute-intensive applications such as video or image processing, whereas OSS is best suitable for this task. However, DFT is nicely fit for bandwidthintensive applications due to the minimum use of network resources. On the other hand, MQTT-based SDP is observed with increase in CPU and Memory usage as the number of users rose, and experienced a drop in data units in the pipeline for PocketSphinx and custom video processing applications, however it performed well for Aeneas which had low size data units.

show abstract

“…However, the building blocks of TOSCA modelling language mainly focus on automatic deployment and orchestration of generic cloud applications [6] and do not deal with controlling the flow of data inside such systems [7]. The challenge arises when data-intensive applications need to be designed and orchestrated.…”

Section: Motivation and Goalmentioning

confidence: 99%

“…This has also been extended to support the scaling [30], load balancing, monitoring, and other aspects of cloud computing. Dehury et al in [7,8] developed the TOSCA profile for serverless and data pipeline-based cloud applications.…”

Section: Literature Survey On Toscamentioning

confidence: 99%

TOSCAdata: Modelling data pipeline applications in TOSCA

Dehury,

Jakovits,

Srirama

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The serverless platform allows a customer to effectively use cloud resources and pay for the exact amount of used resources. A number of dedicated open source and commercial cloud data management tools are available to handle the massive amount of data. Such modern cloud data management tools are not enough matured to integrate the generic cloud application with the serverless platform due to the lack of mature and stable standards. One of the most popular and mature standards, TOSCA (Topology and Orchestration Specification for Cloud Applications), mainly focuses on application and service portability and automated management of the generic cloud application components. This paper proposes the extension of the TOSCA standard, TOSCAdata, that focuses on the modeling of data pipeline-based cloud applications. Keeping the requirements of modern data pipeline cloud applications, TOSCAdata provides a number of TOSCA models that are independently deployable, schedulable, scalable, and re-usable, while effectively handling the flow and transformation of data in a pipeline manner. We also demonstrate the applicability of proposed TOSCAdata models by taking a web-based cloud application in the context of tourism promotion as a use case scenario.

show abstract

CCoDaMiC: A framework for Coherent Coordination of Data Migration and Computation platforms

Cited by 20 publications

References 9 publications

Data Protection by Design Tool for Automated GDPR Compliance Verification Based on Semantically Modeled Informed Consent

Data Protection by Design Tool for Automated GDPR Compliance Verification Based on Semantically Modeled Informed Consent

Serverless data pipeline approaches for IoT data in fog and cloud computing

TOSCAdata: Modelling data pipeline applications in TOSCA

Contact Info

Product

Resources

About