Universal Dependency Analysis

Abstract

Finding patterns from binary data is a classical problem in data mining, dating back to at least frequent itemset mining. More recently, approaches such as tiling and Boolean matrix factorization (BMF), have been proposed to find sets of patterns that aim to explain the full data well. These methods, however, are not robust against non-trivial destructive noise, i.e. when relatively many 1s are removed from the data: tiling can only model additive noise while BMF assumes approximately equal amounts of additive and destructive noise. Most real-world binary datasets, however, exhibit mostly destructive noise. In presence/absence data, for instance, it is much more common to fail to observe something than it is to observe a spurious presence. To address this problem, we take the recent approach of employing the Minimum Description Length (MDL) principle for BMF and introduce a new algorithm, Nassau, that directly optimizes the description length of the factorization instead of the reconstruction error. In addition, unlike the previous algorithms, it can adjust the factors it has discovered during its search. Empirical evaluation on synthetic data shows that Nassau excels at datasets with high destructive noise levels and its performance on real-world datasets confirms our hypothesis of the high numbers of missing observations in the real-world data

Discovering Reliable Correlations in Categorical Data

“…The regret is defined as r n (τ, p (i) ) = E w(X * i ) − w(X * i,j,n,τ ) , where X * i represents the true maximizer of population p (i) , and X * i,j,n,τ the maximizer in D (i) n,j according to an estimator τ = {ŵ,ŵ 0 ,ŵ0,ŵ0}, for which we use exhaustive search to obtain. 4 The expected value is with respect to j ∈ [1,500]. We average regrets across the different p (i) to obtain r n (τ, P [u,v] [a,b] ), e.g., r n (τ, P [2,3] [0,0.5] ) would be the average regret of estimator τ across all p (i) ∈ P 3 [0,0.5] and p (i) ∈ P 4 [0,0.5] .…”

“…We start with Fig. 3 and plot r n (τ, P [2,4] [0.1,0.5] ), i.e. the average regret across all p (i) .…”

“…Finally, in Fig. 5 we plot the regrets averaged over two "strengths" of correlation, low with p (i) ∈ r n (τ, P [2,4] [0.1,0.3) ) (left) and relatively high p (i) ∈ r n (τ, P [2,4] [0.3,0.5] ) (right). Again, the corrected estimators have better regret curves.…”

“…Figure 5: Regret curves averaged over "low" and "high" correlation. Average regret r n (τ, P [2,4] [0.1,0.3) ) (left), and r n (τ, P [2,4] [0.3,0.5] ) (right), for sample sizes n = {10, . .…”

“…BNB: Given a set of input variables I, function w0, bounding functionw0 ref , branching operator r H , and α ∈ (0, 1], the algorithm returns the X * ⊆ I satisfyingŵ0(X * ) ≥ α max{ŵ0(X ) : X ⊆ I} 1: function BNB(Q, S) = r H (top(Q)) 6: X * = arg max{ŵ0(X ) : X ∈ R ∪ {S}} 7:R = {X ∈ R : αw0 ref (X ) >ŵ0(X * )} 8: Q = (Q \ top(Q)) ∪ R 9:return BNB(Q , X * ) 10: X * = BNB({∅}, ∅) Average regret. Regret r n (τ, P[2,4] [0.1,0.5] ) for sample sizes n = {10, . .…”