Coalescing executions for fast uncertainty analysis

Abstract

Uncertain data processing is critical in a wide range of applications such as scientific computation handling data with inevitable errors and financial decision making relying on human provided parameters. While increasingly studied in the area of databases, uncertain data processing is often carried out by software, and thus software based solutions are attractive. In particular, Monte Carlo (MC) methods execute software with many samples from the uncertain inputs and observe the statistical behavior of the output. In this paper, we propose a technique to improve the cost-effectiveness of MC methods. Assuming only part of the input is uncertain, the certain part of the input always leads to the same execution across multiple sample runs. We remove such redundancy by coalescing multiple sample runs in a single run. In the coalesced run, the program operates on a vector of values if uncertainty is present and a single value otherwise. We handle cases where control flow and pointers are uncertain. Our results show that we can speed up the execution time of 30 sample runs by an average factor of 2.3 without precision lost or by up to 3.4 with negligible precision lost.

Faster variational execution with transparent bytecode transformation

“…• Finally, variational execution can be used to speed up similar computations if there is sufficient sharing to offset the overhead. For example, Sumner et al [2011] shares similarities among executions of simulation workloads and computes with several values in parallel. Wang et al [2017] shares executions of mutated programs with equivalence modulo states in the same process and forks new processes only if there are differences in program states after executing mutated statements.…”

“…Researchers have applied these techniques to various scenarios, such as testing highly configurable systems [Nguyen et al 2014], understanding feature interactions [Meinicke et al 2016] and configuration faults [Su et al 2007], monitoring information flow of sensitive data [Austin and Flanagan 2012;Austin et al 2013;Devriese and Piessens 2010;Kim et al 2015;Kolbitsch et al 2012;Kwon et al 2016], and detecting inconsistent updates [Hosek and Cadar 2013;Maurer and Brumley 2012;Tucek et al 2009]. These techniques are similar, and often called differently in different communities, such as variability-aware execution [Kästner et al 2012;Meinicke et al 2016;Nguyen et al 2014], faceted execution [Austin and Flanagan 2012;Austin et al 2013], coalescing execution [Sumner et al 2011], shared execution [Kim et al 2012], and multi-execution [De Groef et al 2012;Devriese and Piessens 2010]. Our work is built on these ideas, and we use the name variational execution in this work, as we target primarily analyzing and testing configuration options in programs.…”

“…Their prototype implementations in JavaScript and Jeeves can prevent cross-site scripting attacks and handle complex information flow in a conference management system. Sumner et al [2011] showed that a form of variational execution can exploit similarities among data processing of similar inputs and gain a speedup of 2.3 without precision lost. Although variational execution has been explored before with promising results, a universal and scalable implementation is still missing.…”

White box sampling in uncertain data processing enabled by program analysis

“…Several techniques are proposed to improve the efficiency of MC methods by parallelizing MC trials [2,4]. In [22], an execution coalescing technique was proposed to pack multiple MC trials in a single run, using vectors. These techniques do not aim at guiding the sampling process to expose critical points using a small number of samples.…”