V2: Fast Detection of Configuration Drift in Python

Abstract

Code snippets are prevalent, but are hard to reuse because they often lack an accompanying environment configuration. Most are not actively maintained, allowing for drift between the most recent possible configuration and the code snippet as the snippet becomes out-of-date over time. Recent work has identified the problem of validating and detecting out-of-date code snippets as the most important consideration for code reuse. However, determining if a snippet is correct, but simply out-of-date, is a non-trivial task. In the best case, breaking changes are well documented, allowing developers to manually determine when a code snippet contains an out-of-date API usage. In the worst case, determining if and when a breaking change was made requires an exhaustive search through previous dependency versions.We present V2, a strategy for determining if a code snippet is out-of-date by detecting discrete instances of configuration drift, where the snippet uses an API which has since undergone a breaking change. Each instance of configuration drift is classified by a failure encountered during validation and a configuration patch, consisting of dependency version changes, which fixes the underlying fault. V2 uses feedback-directed search to explore the possible configuration space for a code snippet, reducing the number of potential environment configurations that need to be validated. When run on a corpus of public Python snippets from prior research, V2 identifies 248 instances of configuration drift.

Conflict-aware Inference of Python Compatible Runtime Environments with Domain Knowledge Graph

“…• DockerizeMe [10], which infers the runtime environments based on a pre-built knowledge base, without considering Python versions nor specific dependency versions. • V2 [11], which reuses the inference results of DockerzieMe as the starting environments in Python 2 and Python 3, and guides the version changes by the error messages of code execution until finding a working environment. • SnifferDog [21], which uses a pre-built API bank to restore the execution environments of Jupyter notebooks.…”

“…com/nju-websoft/PyCRE) and evaluate it with 10,250 realworld Python code snippets on Gistable [9]. Our experiments show that PyCRE efficiently resolves dependency issues for both Python 2 and Python 3, leaving only 1,524 ImportError, which is significantly superior to 2,654 ImportError of the state-of-the-art approach [11].…”

“…DockerizeMe [10] builds an offline knowledge base semi-automatically to infer the environment dependencies for Python code snippets. V2 [11] searches working environments with proper versions based on the error messages of code execution. SnifferDog [21] builds an API bank of Python packages to infer the specific versions and restore the execution environments of Jupyter notebooks.…”

“…dependencies of dependencies) are compatible with the rest of inferred environment. To the best of our knowledge, all previous studies [10,11,21] have not considered the compatibility of the inferred Python environments yet. Due to the NP-completeness of dependency solving [14], we propose a heuristic graph traversal algorithm to infer a compatible environment, which efficiently selects the newer versions and prunes the traversal paths.…”

Restoring Execution Environments of Jupyter Notebooks

“…Second, DockerizeMe does not execute code to validate its findings, let alone compare results against published results-whereas SnifferDog automatically determines the configuration that makes the notebook executable, ideally even reproducing the results. The authors further proposed the tool V2 that takes the program crashes information to guide the search for correct environment dependencies [21]. However this approach relies on repeated execution of code snippets and does not handle the case when no crash happen and dependencies are incorrect.…”