Refining interaction search through signed iterative Random Forests

Abstract

Advances in supervised learning have enabled accurate prediction in biological systems governed by complex interactions among biomolecules. However, state-of-the-art predictive algorithms are typically "black-boxes," learning statistical interactions that are difficult to translate into testable hypotheses. The iterative Random Forest (iRF) algorithm took a step towards bridging this gap by providing a computationally tractable procedure to identify the stable, high-order feature interactions that drive the predictive accuracy of Random Forests (RF). Here we refine the interactions identified by iRF to explicitly map responses as a function of interacting features. Our method, signed iRF (s-iRF), describes "subsets" of rules that frequently occur on RF decision paths. We refer to these "rule subsets" as signed interactions. Signed interactions share not only the same set of interacting features but also exhibit similar thresholding behavior, and thus describe a consistent functional relationship between interacting features and responses. We describe stable and predictive importance metrics (SPIMs) to rank signed interactions in terms of their stability, predictive accuracy, and strength of interaction. For each SPIM, we define null importance metrics that characterize its expected behavior under known structure. We evaluate our proposed approach in biologically inspired simulations and two case studies: predicting enhancer activity and spatial gene expression patterns. In the case of enhancer activity, s-iRF recovers one of the few experimentally validated high-order interactions and suggests novel enhancer elements where this interaction may be active. In the case of spatial gene expression patterns, s-iRF recovers all 11 reported links in the gap gene network. By refining the process of interaction recovery, our approach has the potential to guide mechanistic inquiry into systems whose scale and complexity is beyond human comprehension.

Definitions, methods, and applications in interpretable machine learning

“…Interactions are important as ML models are often highly nonlinear and learn complex interactions between features. Methods exist to extract interactions from many ML models, including random forests (21,57,58) and neural networks (59,60). In the below example, the descriptive accuracy of random forests is increased by extracting Boolean interactions (a problem-relevant form of interpretation) from a trained model.…”

“…A previous line of work considers the problem of searching for biological interactions associated with important biological processes (21,57). To identify candidate biological interactions, the authors train a series of iteratively reweighted random forests (RFs) and search for stable combinations of features that frequently co-occur along the predictive RF decision paths.…”

Veridical data science

“…Dropout in neural networks is a form of algorithm perturbation that leverages stability to improve generalizability (43). Our previous work (34) stabilizes random forests to interpret decision rules in tree ensembles (34,44), which are perturbed using random feature selection (model perturbation) and bootstrap (data perturbation).…”

iRF: extracting interactions from random forests

“…The iterative Random Forest algorithm (iRF), and corresponding iRF R package, take a step towards addressing these issues with a computationally tractable approach to search for important interactions in a fitted random forest ,Kumbier, Basu, Brown, Celniker, & Yu (2018). Our algorithm grows a series of feature weighted random forests (Amaratunga, Cabrera, & Lee, 2008) to perform soft regularization on the model based on predictive features.…”