Timely Dataflow: A Model

Abstract

This paper studies timely dataflow, a model for data-parallel computing in which each communication event is associated with a virtual time. It defines and investigates the could-result-in relation which is central to this model, then the semantics of timely dataflow graphs.

A computational model for TensorFlow: an introduction

“…In the course of such an execution, each node fires exactly once, and the execution ends with non-EMPTY output edges. 3 The order in which nodes fire is not necessarily unique, as mentioned above. Furthermore, the TF program executes in the context of an assignment of values to variables; these values can be read during the execution and can be updated repeatedly during the execution, under program control.…”

“…The rewriting optimizations mentioned above constitute one class of examples. In our past work on timely dataflow [16], we found that having a semantics [3] was crucial for the design of correct techniques for fault-tolerance [4]; a semantics could play an analogous role in future work on Tensor-Flow.…”

“…InTransit(d) = x for some x in s, 2. InTransit(d) = EMPTY in s ,3. InTransit(e) = VarValue(x) in s .• If L(n) = Assign-f, then n's inputs must be some variable x and some tensor v, and it consumes these inputs (so the incoming edges are EMPTY afterwards), and updates the value of x to be the result of applying f to the current value of x and to v. Formally, this means that, if d1 is n's incoming variable edge and d2 is its incoming tensor edge, then1.…”

Incremental, iterative data processing with timely dataflow

“…a As a result, we had to invent suitable restrictions on the structure of timely dataflow graphs, and on the timestamps that can be affixed to messages, to make the notification guarantee hold. The general model is described in detail elsewhere 1,19 but the restrictions that we adopted in the Naiad system are easy to explain informally. A Naiad dataflow graph is acyclic apart from structurally nested cycles that correspond to loops in the program.…”