Unsupervised extra trees: a stochastic approach to compute similarities in heterogeneous data

Dalleau, Kévin; Couceiro, Miguel; Smaïl-Tabbone, Malika

doi:10.1007/s41060-020-00214-4

Cited by 4 publications

(6 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The number of trees was set to 4000, the number of features randomly selected at each node was √ K and the max depth was reached when the node was pure or when it had less than two units; all other tunable parameters were set to their default values.6.UET: UET is based on ET with labels that are randomly generated [10]. A single feature without replacement was selected at each node and n tree was set to 10,000.…”

Section: Evaluation Of Tarf In Real‐world Datasetsmentioning

confidence: 99%

“…In particular, when k = 1, the structure of the tree and the labels in the sample data are independent. Dalleau et al [10,11] adapted ET to a setting in which labels are unknown, calling their method UET. The source code and installation instructions can be found on Gitlab (https://gitlab.inria.fr/kdalleau/uetcpp).…”

Section: Background: Et and Uetmentioning

confidence: 99%

“…6. UET: UET is based on ET with labels that are randomly generated [10]. A single feature without replacement was selected at each node and n tree was set to 10,000.…”

Section: Evaluating the Performance Of Tarf For Clustering In Eight R...mentioning

confidence: 99%

“…Dalleau et al [10, 11] adapted ET to a setting in which labels are unknown , calling their method UET . The source code and installation instructions can be found on Gitlab (https://gitlab.inria.fr/kdalleau/uetcpp).…”

Section: Background: Et and Uetmentioning

confidence: 99%

See 3 more Smart Citations

A treeless absolutely random forest with closed‐form estimators of expected proximities

Laska,

Lin,

Siegel

et al. 2024

Statistical Analysis

View full text Add to dashboard Cite

We introduce a simple variant of a purely random forest, called an absolute random forest (ARF) used for clustering. At every node, splits of units are determined by a randomly chosen feature and a random threshold drawn from a uniform distribution whose support, the range of the selected feature in the root node, does not change. This enables closed‐form estimators of parameters, such as pairwise proximities, to be obtained without having to grow a forest. The probabilistic structure corresponding to an ARF is called a treeless absolute random forest (TARF). With high probability, the algorithm will split units whose feature vectors are far apart and keep together units whose feature vectors are similar. Thus, the underlying structure of the data drives the growth of the tree. The expected value of pairwise proximities is obtained for three pathway functions. One, a completely common pathway function, is an indicator of whether a pair of units follow the same path from the root to the leaf node. The properties of TARF‐based proximity estimators for clustering and classification are compared to other methods in eight real‐world datasets and in simulations. Results show substantial performance and computing efficiencies of particular value for large datasets.

show abstract

Section: Evaluation Of Tarf In Real‐world Datasetsmentioning

confidence: 99%

Section: Background: Et and Uetmentioning

confidence: 99%

“…6. UET: UET is based on ET with labels that are randomly generated [10]. A single feature without replacement was selected at each node and n tree was set to 10,000.…”

Section: Evaluating the Performance Of Tarf For Clustering In Eight R...mentioning

confidence: 99%

Section: Background: Et and Uetmentioning

confidence: 99%

See 2 more Smart Citations

A treeless absolutely random forest with closed‐form estimators of expected proximities

Laska,

Lin,

Siegel

et al. 2024

Statistical Analysis

View full text Add to dashboard Cite

show abstract

“…Those methods optimize similarity measures by linear or non-linear aggregation functions in a data-independent manner, which cannot depict the complex attribute coupling relationships and heterogeneity among attributes [5]. Accordingly, a more promising but challenging approach is to capture the heterogeneity of attributes [11,26,82] and learn data-aware similarity metrics [47,64] to capture feature couplings [9,50,85].…”

Section: Challenges Of High-dimensional and Heterogeneous Datamentioning

confidence: 99%

Supervised Deep Hashing for High-dimensional and Heterogeneous Case-based Reasoning

Zhang¹,

Liang²,

Shi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Case-based Reasoning (CBR) on high-dimensional and heterogeneous data is a trending yet challenging and computationally expensive task in the real world. A promising approach is to obtain low-dimensional hash codes representing cases and perform a similarity retrieval of cases in a Hamming space. However, previous methods based on data-independent hashing rely on random projections or manual construction, inapplicable to address specific data issues (e.g., high-dimensionality and heterogeneity) due to their insensitivity to data characteristics. To address these issues, this work introduces a novel deep hashing network to learn similarity-preserving compact hash codes for efficient case retrieval and proposes a deep-hashing-enabled CBR model HeCBR. Specifically, we introduce position embedding to represent heterogeneous features and utilize a multilinear interaction layer to obtain case embeddings, which effectively filtrates zero-valued features to tackle high-dimensionality and sparsity and captures inter-feature couplings. Then, we feed the case embeddings into fully-connected layers, and subsequently a hash layer generates hash codes with a quantization regularizer to control the quantization loss during relaxation. To cater for incremental learning of CBR, we further propose an adaptive learning strategy to update the hash function. Extensive experiments on public datasets show HeCBR greatly reduces storage and significantly accelerates the case retrieval. HeCBR achieves desirable performance compared with the state-of-the-art CBR methods and performs significantly better than hashing-based CBR methods in classification.

show abstract

Head-to-head comparison of clustering methods for heterogeneous data: a simulation-driven benchmark

Preud’homme

Duarte

Dalleau

et al. 2021

Sci Rep

View full text Add to dashboard Cite

The choice of the most appropriate unsupervised machine-learning method for “heterogeneous” or “mixed” data, i.e. with both continuous and categorical variables, can be challenging. Our aim was to examine the performance of various clustering strategies for mixed data using both simulated and real-life data. We conducted a benchmark analysis of “ready-to-use” tools in R comparing 4 model-based (Kamila algorithm, Latent Class Analysis, Latent Class Model [LCM] and Clustering by Mixture Modeling) and 5 distance/dissimilarity-based (Gower distance or Unsupervised Extra Trees dissimilarity followed by hierarchical clustering or Partitioning Around Medoids, K-prototypes) clustering methods. Clustering performances were assessed by Adjusted Rand Index (ARI) on 1000 generated virtual populations consisting of mixed variables using 7 scenarios with varying population sizes, number of clusters, number of continuous and categorical variables, proportions of relevant (non-noisy) variables and degree of variable relevance (low, mild, high). Clustering methods were then applied on the EPHESUS randomized clinical trial data (a heart failure trial evaluating the effect of eplerenone) allowing to illustrate the differences between different clustering techniques. The simulations revealed the dominance of K-prototypes, Kamila and LCM models over all other methods. Overall, methods using dissimilarity matrices in classical algorithms such as Partitioning Around Medoids and Hierarchical Clustering had a lower ARI compared to model-based methods in all scenarios. When applying clustering methods to a real-life clinical dataset, LCM showed promising results with regard to differences in (1) clinical profiles across clusters, (2) prognostic performance (highest C-index) and (3) identification of patient subgroups with substantial treatment benefit. The present findings suggest key differences in clustering performance between the tested algorithms (limited to tools readily available in R). In most of the tested scenarios, model-based methods (in particular the Kamila and LCM packages) and K-prototypes typically performed best in the setting of heterogeneous data.

show abstract

Unsupervised extra trees: a stochastic approach to compute similarities in heterogeneous data

Cited by 4 publications

References 24 publications

A treeless absolutely random forest with closed‐form estimators of expected proximities

A treeless absolutely random forest with closed‐form estimators of expected proximities

Supervised Deep Hashing for High-dimensional and Heterogeneous Case-based Reasoning

Head-to-head comparison of clustering methods for heterogeneous data: a simulation-driven benchmark

Contact Info

Product

Resources

About