Robust PCA for skewed data and its outlier map

Change point detection in multivariate time series is a complex task since next to the mean, the correlation structure of the monitored variables may also alter when change occurs. DeCon was recently developed to detect such changes in mean and\or correlation by combining a moving windows approach and robust PCA. However, in the literature, several other methods have been proposed that employ other non-parametric tools: E-divisive, Multirank, and KCP. Since these methods use different statistical approaches, two issues need to be tackled. First, applied researchers may find it hard to appraise the differences between the methods. Second, a direct comparison of the relative performance of all these methods for capturing change points signaling correlation changes is still lacking. Therefore, we present the basic principles behind DeCon, E-divisive, Multirank, and KCP and the corresponding algorithms, to make them more accessible to readers. We further compared their performance through extensive simulations using the settings of Bulteel et al. (Biological Psychology, 98 (1), 29-42, 2014) implying changes in mean and in correlation structure and those of Matteson and James (Journal of the American Statistical Association, 109 (505), 334-345, 2014) implying different numbers of (noise) variables. KCP emerged as the best method in almost all settings. However, in case of more than two noise variables, only DeCon performed adequately in detecting correlation changes.

show abstract

“…2 If the normality assumption may not hold for a specific data set, a variant of ROBPCA for skewed data (Hubert, Rousseeuw & Verdonck 2009) …”

Section: Apply Robust Pca In Each Time Window and Determinementioning

confidence: 99%

Detecting correlation changes in multivariate time series: A comparison of four non-parametric change point detection methods

et al. 2016

View full text Add to dashboard Cite

show abstract

“…Appropriate cutoff values to separate the different type of observations are derived in Hubert et al (2005Hubert et al ( , 2009). Here, we follow the method of Hubert et al (2009) and define the cutoffs as the upper whiskers of the adjusted boxplot computed on the respective distances.…”

Section: Methodsmentioning

confidence: 99%

Analysis of travel activity determinants using robust statistics

et al. 2016

Self Cite

View full text Add to dashboard Cite

This study investigates travel behavior determinants based on a multiday travel survey conducted in the region of Ghent, Belgium. Due to the limited data reliability of the data sample and the influence of outliers exerted on classical principal component analysis, robust principal component analysis (ROBPCA) is employed in order to reveal the explanatory variables responsible for most of the variability. Interpretation of the results is eased by utilizing ROSPCA. The application of ROSPCA reveals six distinct principal components where each is determined by a few variables. Among others, our results suggest a key role of variable categories such as journey purpose-related impedance and journey inherent constraints. Surprisingly, the variables associated with journey timing turn out to be less important. Finally, our findings reveal the critical role of outliers in travel behavior analysis. This suggests that a systematic understanding of how outliers contribute to observed mobility behavior patterns, as derived from travel surveys, is needed. In this regard, the proposed methods serve for processing raw data typically used in activity-based modelling.

show abstract

“…Simulation is often performed to assess selected features of algorithms (Josse-Husson [2012]), Brechmann-Joe [2014]). Its results can be considered reliable because the problem that factor analysis results are sensitive to outliers (as described, for example, by SerneelsVerdonck [2008] and Hubert-Rousseeuw-Verdonck [2009]), cannot be regarded serious in calculation, due to the distributional assumptions of this paper. Accordingly, principal component analysis (and the spectral decomposition of the ordinary correlation matrix) is used in the following to identify latent factors.…”

Section: The Theoretical Modelmentioning

confidence: 97%

Comparison of goodness measures for linear factor structures

Szüle¹

2017

Statisztikai szemle

View full text Add to dashboard Cite

Linear factor structures often exist in empirical data, and they can be mapped by factor analysis. It is, however, not straightforward how to measure the goodness of a factor analysis solution since its results should correspond to various requirements. Instead of a unique indicator, several goodness measures can be defined that all contribute to the evaluation of the results.This paper aims to find an answer to the question whether factor analysis outputs can meet several goodness criteria at the same time. Data aggregability (measured by the determinant of the correlation matrix and the proportion of explained variance) and the extent of latency (defined by the determinant of the antiimage correlation matrix, the maximum partial correlation coefficient and the Kaiser-Meyer-Olkin measure of sampling adequacy) are studied. According to the theoretical and simulation results, it is not possible to meet simultaneously these two criteria when the correlation matrices are relatively small. For larger correlation matrices, however, there are linear factor structures that combine good data aggregability with a high extent of latency.

show abstract

Robust PCA for skewed data and its outlier map

Cited by 98 publications

References 23 publications

Detecting correlation changes in multivariate time series: A comparison of four non-parametric change point detection methods

Detecting correlation changes in multivariate time series: A comparison of four non-parametric change point detection methods

Analysis of travel activity determinants using robust statistics

Comparison of goodness measures for linear factor structures

Contact Info

Product

Resources

About