Anomaly detection in scientific data using joint statistical moments

Aditya, Konduri; Kolla, Hemanth; Kegelmeyer, W. Philip; Shead, Timothy M.; Ling, Julia; Davis, Warren L.

doi:10.1016/j.jcp.2019.03.003

Cited by 25 publications

(21 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…and is of interest in different contexts. [16][17][18][19][20] Expanding the numerator for n = m = 2 yields…”

Section: Pairwise Correlationmentioning

confidence: 99%

“…The cokurtosis [ 14,15 ] of two variables is defined as

C o k false(x_{1}, x_{2} false) = \frac{E [{false(x_{1} - ⟨ x_{1} ⟩ false)}^{n} {false(x_{2} - ⟨ x_{2} ⟩ false)}^{m}]}{σ_{x_{1}}^{n} σ_{x_{2}}^{m}}

and is of interest in different contexts. [ 16–20 ] Expanding the numerator for

n = m = 2

yields

\begin{matrix} true \\ left ⟨ x_{1}^{2} x_{2}^{2} ⟩ - C o v (x_{1}^{2}, x_{2}) ⟨ x_{2} ⟩ - C o v (x_{1}, x_{2}^{2}) ⟨ x_{1} ⟩ + 4 C o v (x_{1}, x_{2}) ⟨ x_{1} ⟩ ⟨ x_{2} ⟩ \\ right - ⟨ x_{1}^{2} x_{2} ⟩ ⟨ x_{2} ⟩ - ⟨ x_{1} x_{2}^{2} ⟩ ⟨ x_{1} ⟩ + {false⟨ x_{1} false⟩}^{2} {false⟨ x_{2} false⟩}^{2} \end{matrix}

For symmetrical distributions, the first term is the only one which survives

C o k false(x_{1}, x_{2}

…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Statistical Correlation Measures From Higher‐Order Moments in Quantum Oscillator Systems

Salazar¹,

Laguna²,

Sagar³

2021

Advcd Theory and Sims

View full text Add to dashboard Cite

Triple‐wise and higher‐order correlation measures are proposed using analogies from information theory. Two‐ and three‐variable statistical correlation measures, obtained from higher‐order moments, are examined in three‐particle continuous variable quantum oscillator systems. It is illustrated how the mutual information and cokurtosis measures exhibit similar behaviors, as the strength of the attractive or repulsive interparticle potential is varied.

show abstract

“…and is of interest in different contexts. [16][17][18][19][20] Expanding the numerator for n = m = 2 yields…”

Section: Pairwise Correlationmentioning

confidence: 99%

“…The cokurtosis [ 14,15 ] of two variables is defined as

C o k false(x_{1}, x_{2} false) = \frac{E [{false(x_{1} - ⟨ x_{1} ⟩ false)}^{n} {false(x_{2} - ⟨ x_{2} ⟩ false)}^{m}]}{σ_{x_{1}}^{n} σ_{x_{2}}^{m}}

and is of interest in different contexts. [ 16–20 ] Expanding the numerator for

n = m = 2

yields

\begin{matrix} true \\ left ⟨ x_{1}^{2} x_{2}^{2} ⟩ - C o v (x_{1}^{2}, x_{2}) ⟨ x_{2} ⟩ - C o v (x_{1}, x_{2}^{2}) ⟨ x_{1} ⟩ + 4 C o v (x_{1}, x_{2}) ⟨ x_{1} ⟩ ⟨ x_{2} ⟩ \\ right - ⟨ x_{1}^{2} x_{2} ⟩ ⟨ x_{2} ⟩ - ⟨ x_{1} x_{2}^{2} ⟩ ⟨ x_{1} ⟩ + {false⟨ x_{1} false⟩}^{2} {false⟨ x_{2} false⟩}^{2} \end{matrix}

For symmetrical distributions, the first term is the only one which survives

C o k false(x_{1}, x_{2}

…”

Section: Introductionmentioning

confidence: 99%

Statistical Correlation Measures From Higher‐Order Moments in Quantum Oscillator Systems

Salazar¹,

Laguna²,

Sagar³

2021

Advcd Theory and Sims

View full text Add to dashboard Cite

show abstract

“…Blockwise distribution based visualization and analysis has been applied in multiple contexts, including feature tracking [17], anomaly data [1], and in situ summarization in several different domains [16,52]. The blockwise distribution driven analysis package computes distribution functions for a given set of variables, over each subblock of the domain decomposition of a distributed Cartesian volume.…”

Section: Blockwise Distribution Driven Analysismentioning

confidence: 99%

Diva: A Declarative and Reactive Language for In-Situ Visualization

Neuroth

Igouchkine

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

The use of adaptive workflow management for in situ visualization and analysis has been a growing trend in large-scale scientific simulations. However, coordinating adaptive workflows with traditional procedural programming languages can be difficult, because system flow is determined by unpredictable scientific phenomena, which often appear in an unknown order and can evade event handling. This makes the implementation of adaptive workflows tedious and errorprone. Recently, reactive and declarative programming paradigms have been recognized as well-suited solutions to similar problems in other domains. Not much work has been done to adopt these methods into the field of in situ visualization and analysis. With this paper, we present a proof-of-concept language, parser and runtime system for designing adaptive systems through a declarative and reactive programming paradigm. We illustrate how an adaptive workflow system is implemented using our approach, and demonstrate it with use cases from real scientific simulations.

show abstract

“…Tensor decomposition of moment and cumulant tensors are used in a variety of statistical and data science applications, including independent component analysis and blind source separation [10,13,15], clustering [34,12], learning Gaussian mixture models [23,4,20,18,33], latent variable models [2,3], outlier detection [16,1], feature extraction in hyperspectral imagery [19], and multireference alignment [32]. In these cases, it is assumed that the empirical higher-order moment is already computed.…”

mentioning

confidence: 99%

“…For d " 3 and n " 1000, X requires 8 GB of storage; for d " 4 and n " 200, X requires 12 GB of storage. 1 The per iteration storage and floating point operations per iteration cost to compute a rank r " 10 approximation are shown in Figure 1.2 for different values of d and n.…”

mentioning

confidence: 99%

Estimating Higher-Order Moments Using Symmetric Tensor Decomposition

Sherman,

Kolda

2019

Preprint

View full text Add to dashboard Cite

We consider the problem of decomposing higher-order moment tensors, i.e., the sum of symmetric outer products of data vectors. Such a decomposition can be used to estimate the means in a Gaussian mixture model and for other applications in machine learning. The dth-order empirical moment tensor of a set of p observations of n variables is a symmetric d-way tensor. Our goal is to find a low-rank tensor approximation comprising r ! p symmetric outer products. The challenge is that forming the empirical moment tensors costs Oppn d q operations and Opn d q storage, which may be prohibitively expensive; additionally, the algorithm to compute the low-rank approximation costs Opn d q per iteration. Our contribution is avoiding formation of the moment tensor, computing the low-rank tensor approximation of the moment tensor implicitly using Oppnrq operations per iteration and no extra memory. This advance opens the door to more applications of higher-order moments since they can now be efficiently computed. We present numerical evidence of the computational savings and show an example of estimating the means for higher-order moments.

show abstract

Anomaly detection in scientific data using joint statistical moments

Cited by 25 publications

References 24 publications

Statistical Correlation Measures From Higher‐Order Moments in Quantum Oscillator Systems

Statistical Correlation Measures From Higher‐Order Moments in Quantum Oscillator Systems

Diva: A Declarative and Reactive Language for In-Situ Visualization

Estimating Higher-Order Moments Using Symmetric Tensor Decomposition

Contact Info

Product

Resources

About