Architectural Optimization and Feature Learning for High-Dimensional Time Series Datasets

Abstract

As our ability to sense increases, we are experiencing a transition from data-poor problems, in which the central issue is a lack of relevant data, to data-rich problems, in which the central issue is to identify a few relevant features in a sea of observations. Motivated by applications in gravitational-wave astrophysics, we study a problem in which the goal is to predict the presence of transient noise artifacts in a gravitational wave detector from a rich collection of measurements from the detector and its environment.We argue that feature learning-in which relevant features are optimized from data-is critical to achieving high accuracy. We introduce models that reduce the error rate by over 60% compared to the previous state of the art, which used fixed, hand-crafted features. Feature learning is useful not only because it can improve performance on prediction tasks; the results provide valuable information about patterns associated with phenomena of interest that would otherwise be impossible to discover. In our motivating application, features found to be associated with transient noise provide diagnostic information about its origin and suggest mitigation strategies.Learning in such a high-dimensional setting is challenging. Through experiments with a variety of architectures, we identify two key factors in high-performing models: sparsity, for selecting relevant variables within the high-dimensional observations; and depth, which confers flexibility for handling complex interactions and robustness with respect to temporal variations. We illustrate their significance through a systematic series of experiments on real gravitational-wave detector data. Our results provide experimental corroboration of common assumptions in the machine-learning community and have direct applicability to improving our ability to sense gravitational waves, as well as to a wide variety of problem settings with similarly high-dimensional, noisy, or partly irrelevant data.

Detecting and Diagnosing Terrestrial Gravitational-Wave Mimics Through Feature Learning

“…Notably, the models discussed here are explicitly tuned to learn to ignore input channels that are not useful for making its predictions-the vast majority of them, in this setting. Such a practice has previously been demonstrated (e.g., in [42,43]) to improve both interpretability and accuracy. These methods achieve excellent performance while ignoring all but a few dozen to few hundred channels, which can then be further investigated manually by detector domain experts.…”

“…Machine learning models have been demonstrated in recent works [42,43] to be able to learn to predict the presence or absence of glitches in high-dimensional gravitational-wave astronomy data by considering only auxiliary channel information, without looking at the gravitational-wave strain data stream in which the glitches themselves appear. In this section, we describe one such model initially proposed in [43], which we refer to as LF. The model is highly interpretable and wellsuited to efficient strain-independent glitch detection, offering several useful features including:…”

“…These methods achieve excellent performance while ignoring all but a few dozen to few hundred channels, which can then be further investigated manually by detector domain experts. [Section IV B] For the experiments presented here, we employ a slightly modified version of the LF model introduced in [43]. The model is a flat convolutional binary classifier parameterized by a learned filter f p ∈ R T of desired length T 1 for each of the P input time series (auxiliary channels) as well as a single learned bias term b sp for all channels of each sample rate s p .…”

“…As in [43], we employ a sparsifying regularizer-the elastic net [51]-during training to encourage most of the 1 We set the filter length T to 96 samples for all channels based on the finding of [43] that six seconds of data for 16-Hz channels performed well. As discussed in Sec.…”

“…Recent machine learning-based approaches to glitch understanding and mitigation include works that investigate the association between glitches and LIGO's auxiliary data channels by casting glitch detection as a classification problem using only non-astrophysicallysensitive auxiliary channel data as input [42,43]. Many other glitch detection methods directly analyze the gravitational-wave strain [21, 23, 25, 30-32, 34-39, 44-49].…”