A proposal for parallel self-adjusting computation

Hammer, Matthew A.; Acar, Umut A.; Rajagopalan, Mohan; Ghuloum, Anwar

doi:10.1145/1248648.1248651

Cited by 27 publications

(23 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such a parallelization would require parallelizing the underlying self-adjusting computation techniques. There has been some research on this problem, but existing solutions work in certain domains and/or use a sub-optimal algorithms for parallel change propagation [5,6,18].…”

Section: Related Workmentioning

confidence: 99%

Functional programming for dynamic and large data with self-adjusting computation

2014

Self Cite

View full text Add to dashboard Cite

Combining type theory, language design, and empirical work, we present techniques for computing with large and dynamically changing datasets. Based on lambda calculus, our techniques are suitable for expressing a diverse set of algorithms on large datasets and, via self-adjusting computation, enable computations to respond automatically to changes in their data. Compared to prior work, this work overcomes the main challenge of reducing the space usage of self-adjusting computation without disproportionately decreasing performance. To this end, we present a type system for precise dependency tracking that minimizes the time and space for storing dependency metadata. The type system eliminates an important assumption of prior work that can lead to recording of spurious dependencies. We give a new type-directed translation algorithm that generates correct self-adjusting programs without relying on this assumption. We then show a probabilistic chunking technique to further decrease space usage by controlling the fundamental space-time tradeoff in self-adjusting computation. We implement and evaluate these techniques, showing very promising results on challenging benchmarks and large graphs.

show abstract

Section: Related Workmentioning

confidence: 99%

Functional programming for dynamic and large data with self-adjusting computation

2014

Self Cite

View full text Add to dashboard Cite

show abstract

“…Another line of research realized an interesting duality between incremental and parallel computation-both benefit from identifying independent computations-and proposed techniques for parallel self-adjusting computation. Some earlier work considered techniques for performing efficient parallel updates in the context of a lambda calculus extended with fork-join style parallelism [29]. Follow-up work considered the technique in the context of a more sophisticated problem showing both theoretical and empirical results of its effectiveness [7].…”

Section: Related Workmentioning

confidence: 99%

Functional programming for dynamic and large data with self-adjusting computation

Chen

Acar

Tangwongsan

2014

Proceedings of the 19th ACM SIGPLAN International Conference on Functional Programming

Self Cite

View full text Add to dashboard Cite

Combining type theory, language design, and empirical work, we present techniques for computing with large and dynamically changing datasets. Based on lambda calculus, our techniques are suitable for expressing a diverse set of algorithms on large datasets and, via self-adjusting computation, enable computations to respond automatically to changes in their data. To improve the scalability of self-adjusting computation, we present a type system for precise dependency tracking that minimizes the time and space for storing dependency metadata. The type system eliminates an important assumption of prior work that can lead to recording spurious dependencies. We present a type-directed translation algorithm that generates correct self-adjusting programs without relying on this assumption. We then show a probabilistic-chunking technique to further decrease space usage by controlling the fundamental spacetime tradeoff in self-adjusting computation. We implement and evaluate these techniques, showing promising results on challenging benchmarks involving large graphs.

show abstract

“…Self-adjusting computation has been shown to be effective for a wide variety of problems and has led to progress on and sometimes solutions to open problems in several domains including computational geometry, machine learning, and checking of data structural invariants (e.g., [36,8,5,37,9]). Some earlier [7,23] and more recent work [13] realized that many parallel algorithms are amenable to self-adjusting computation and developed techniques for taking advantage of both simultaneously.…”

Section: Related Workmentioning

confidence: 99%

Distributed Programming for the Cloud: Models, Challenges, and Analytics Engines

Bhatotia¹,

Wieder²,

Acar³

et al. 2014

Large Scale and Big Data

View full text Add to dashboard Cite

A proposal for parallel self-adjusting computation

Cited by 27 publications

References 10 publications

Functional programming for dynamic and large data with self-adjusting computation

Functional programming for dynamic and large data with self-adjusting computation

Functional programming for dynamic and large data with self-adjusting computation

Distributed Programming for the Cloud: Models, Challenges, and Analytics Engines

Contact Info

Product

Resources

About