A Minimalistic Dataflow Programming Library for Python

Alves, Tiago A. O.; Goldstein, Brunno F.; França, Felipe M. G.; Marzulo, Leandro A. J.

doi:10.1109/sbac-padw.2014.20

Cited by 16 publications

(11 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This work investigates the reuse potential in applications written for the Sucuri dataflow execution model. The DF‐DTM reuse is inspired on the DTM reuse mechanism.…”

Section: Related Workmentioning

confidence: 99%

“…In this paper, we use the Sucuri Dataflow runtime as basis for implementing and evaluating our redundant task reuse approach. In this section, we present our proposed reuse mechanism, the DF‐DTM, and explain the basic concepts of dataflow and Sucuri.…”

Section: Memoization and Reuse In Dataflowmentioning

confidence: 99%

“…Sucuri is a minimalistic dataflow runtime for Python. In Sucuri, the developer implements functions of his program and associates them to node objects , which are in turn connected according to their dependencies.…”

Section: Memoization and Reuse In Dataflowmentioning

confidence: 99%

“…Redundant sequences of tasks, ie, subgraphs, can be identified during the execution of a dataflow program and both tasks and subgraphs can be reused. The Sucuri dataflow runtime was modified in order to implement DF‐DTM and identify in which situations there would be reuse and how implementation aspects can affect it.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

DF‐DTM: Dynamic Task Memoization and reuse in dataflow

Rouberte

Sena

Nery

et al. 2018

Concurrency and Computation

View full text Add to dashboard Cite

Summary Instruction Reuse is a technique adopted in Von Neumann architectures that improves performance by avoiding redundant execution of instructions when the result to be produced can be obtained by searching an input/output memoization table for such instruction. Trace reuse can be applied to traces of instructions in a similar fashion. However, those techniques are yet to be studied in the context of the Dataflow model, which has been gaining traction in the high performance computing community due to its inherent parallelism. Dataflow programs are represented by directed graphs where nodes are instructions or tasks and edges denote data dependencies between tasks. This work presents Dataflow Dynamic Task Memoization (DF‐DTM), a technique that allows the reuse of both nodes and subgraphs in dataflow, which are analogous to instructions and traces, respectively. The potential of DF‐DTM is evaluated by a series of experiments that analyze the behavior of redundant tasks in five relevant benchmarks, where up to 99.70% of the instantiated tasks could be reused. Moreover, this paper evaluates how reuse rates can be affected by limiting subgraph size, memoization table size, task granularity, and problem size, showing that DF‐DTM can yield good reuse rates in more realistic environments.

show abstract

“…This work investigates the reuse potential in applications written for the Sucuri dataflow execution model. The DF‐DTM reuse is inspired on the DTM reuse mechanism.…”

Section: Related Workmentioning

confidence: 99%

Section: Memoization and Reuse In Dataflowmentioning

confidence: 99%

Section: Memoization and Reuse In Dataflowmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

DF‐DTM: Dynamic Task Memoization and reuse in dataflow

Rouberte

Sena

Nery

et al. 2018

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…Neste trabalho iremos tratar da aplicação do iDMDGP na reconstrução de estruturas tridimensionais de proteínas, através do emprego de dados (intervalares) que simulam medidas experimentais obtidas por Ressonância Magnética Nuclear. Empregaremos, para isto, uma abordagem paralela para o iDMDGP, baseada no modelo de execução Dataflow, o qual, tem se mostrado uma boa opção para a programação paralela [6].…”

Section: Introductionunclassified

Explorando Paralelismo Dataflow em Geometria de Distâncias Moleculares Intervalares

Vilela¹,

Marzulo²,

França³

2020

Proceeding Series of the Brazilian Society of Computational and Applied Mathematics

View full text Add to dashboard Cite

Resumo. Geometria de Distâncias corresponde a uma classe de problemas onde as posições dos pontos no espaço são determinadas através da informação de algumas distâncias entre estes. Um destes problemas, muito importante, consiste na reconstrução de estruturas tridimensionais de proteínas a partir de algumas distâncias intervalares obtidas através de experimentos de Ressonância Magnética Nuclear. Este trabalho propõe um abordagem paralela por Dataflow para o tratamento destes problemas aplicadosà reconstrução de estruturas moleculares. Palavras-chave. Geometria de Distâncias Moleculares, iDMDGP, Dataflow

show abstract

Gamma — General Abstract Model for Multiset mAnipulation and dynamic dataflow model: An equivalence study

Mello

Araújo

Alves

et al. 2021

Concurrency and Computation

View full text Add to dashboard Cite

With the increase of the search for computational models where the expression of parallelism occurs naturally, some paradigms arise as options for the current generation of computers. In this context, dynamic dataflow and Gamma—General Abstract Model for Multiset mAnipulation—emerge as interesting computational model choices. In dynamic dataflow model, operations are performed as soon as their associated operands are available, without rely on a Program Counter to dictate the execution order of instructions. The Gamma paradigm is based on a parallel multiset rewriting scheme. It provides a nondeterministic execution model inspired by an abstract chemical machinemetaphor, where operations are formulated as reactions that occur freely among matching elements belonging to the multiset. In this work, equivalence relations between the dynamic dataflow and Gamma paradigms are exposed and explored, while methods to convert from dataflow to Gamma paradigm and vice versa are provided. It is shown that vertices and edges of a dynamic dataflow graph can correspond, respectively, to reactions and multiset elements in the Gamma paradigm. This work provides the scientific community with the possibility of taking profit of both parallel programming models, contributing with a versatility component to researchers and developers.

show abstract

A Minimalistic Dataflow Programming Library for Python

Cited by 16 publications

References 8 publications

DF‐DTM: Dynamic Task Memoization and reuse in dataflow

DF‐DTM: Dynamic Task Memoization and reuse in dataflow

Explorando Paralelismo Dataflow em Geometria de Distâncias Moleculares Intervalares

Gamma — General Abstract Model for Multiset mAnipulation and dynamic dataflow model: An equivalence study

Contact Info

Product

Resources

About