A Siamese Deep Forest

Utkin, Lev V.; Ryabinin, M. A.

doi:10.1016/j.knosys.2017.10.006

Cited by 54 publications

(22 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…for j = c, which implicates that P c,j , 1 − P c,j , and p(t c,j ) for j = c, j = c, and all j, respectively, are all O(exp(−1/σ 2 j ) as σ j → 0. Applying these on (34) and (35), it proves (27)- (29). Meanwhile, (26) is proved by observing |ρ(ŷ j ) − ρ(ŷ j )| ≤ |ŷ j −ŷ j | and the expectation of |ŷ−ŷ |, provided that x,x ∈ C c , is 2 j σ 2 j .…”

Section: B Layer-wise Intra/interclass Distance Dynamics In Danmentioning

confidence: 71%

“…To assure high degree of diversity leading to performance gain in consequence, each ensemble is instantiated with RFs of different types. Other DF-driven S-DNNs include [33] and [34]. However, the performance evaluation of these DF networks is limited to only handwritten digit recognition.…”

Section: A Related Workmentioning

confidence: 99%

“…Stacking-based deep neural network (S-DNN) emerges as a DNN-alike resemblance, e.g., [10]- [13], [15], [24]- [27], [29], [30], [32]- [34], etc., for pattern classification. Architecturally, S-DNN is in line with stacked generalization [7], [8], which aggregates a chain of independent self-learnable modules.…”

mentioning

confidence: 99%

“…Different from that of convolutional, the nonconvolutional S-DNN is topologically fully connected (FC) for both images and nonimages such as the pre-extracted baseline features and also the handcrafted structured features. The most representative FC S-DNNs include deep convex networks (DCNs) [24]- [27]; deep ELMs (D-ELMs) [29], [30]; and deep forests (DFs) [32]- [34]. We group all the abovementioned networks under the S-DNN umbrella term in Fig.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Stacking-Based Deep Neural Network: Deep Analytic Network for Pattern Classification

Low

Park

Teoh

2020

IEEE Trans. Cybern.

View full text Add to dashboard Cite

Stacking-based deep neural network (S-DNN) is aggregated with pluralities of basic learning modules, one after another, to synthesize a deep neural network (DNN) alternative for pattern classification. Contrary to the DNNs trained from end to end by backpropagation (BP), each S-DNN layer, that is, a selflearnable module, is to be trained decisively and independently without BP intervention. In this paper, a ridge regression-based S-DNN, dubbed deep analytic network (DAN), along with its kernelization (K-DAN), are devised for multilayer feature relearning from the pre-extracted baseline features and the structured features. Our theoretical formulation demonstrates that DAN/K-DAN relearn by perturbing the intra/interclass variations, apart from diminishing the prediction errors. We scrutinize the DAN/K-DAN performance for pattern classification on datasets of varying domains-faces, handwritten digits, generic objects, to name a few. Unlike the typical BP-optimized DNNs to be trained from gigantic datasets by GPU, we reveal that DAN/K-DAN are trainable using only CPU even for small-scale training sets. Our experimental results show that DAN/K-DAN outperform the present S-DNNs and also the BP-trained DNNs, including multiplayer perceptron, deep belief network, etc., without data augmentation applied. Index Terms-Deep analytic network (DAN), face recognition, object recognition, pattern classification, stacking-based deep neural network (S-DNN).

show abstract

Section: B Layer-wise Intra/interclass Distance Dynamics In Danmentioning

confidence: 71%

Section: A Related Workmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Stacking-Based Deep Neural Network: Deep Analytic Network for Pattern Classification

Low

Park

Teoh

2020

IEEE Trans. Cybern.

View full text Add to dashboard Cite

show abstract

“…Challenges such as local optimality or gradient vanishing also accompany this advantage, and regularizations similar to early stopping, weight decay, dropout [141] and batch normalization [142] remain to be discovered. In addition to networks, deep structures have recently been extended to ensembles of decision trees, such as gcForest [143] and related variants [144,145]. It can be asserted that deep learning aggregated with other learning frameworks will be promising, and deeper models not limited to networks will emerge in the coming decades to satisfy the need to fit complex datasets.…”

Section: Deep Models Short Models and Ensemble Modelsmentioning

confidence: 99%

Addressing Complexities of Machine Learning in Big Data: Principles, Trends and Challenges from Systematical Perspectives

Q¹,

Zhao²,

Huang³

et al. 2017

Preprint

View full text Add to dashboard Cite

The concept of 'big data' has been widely discussed, and its value has been illuminated throughout a variety of domains. To quickly mine potential values and alleviate the ever-increasing volume of information, machine learning is playing an increasingly important role and faces more challenges than ever. Because few studies exist regarding how to modify machine learning techniques to accommodate big data environments, we provide a comprehensive overview of the history of the evolution of big data, the foundations of machine learning, and the bottlenecks and trends of machine learning in the big data era. More specifically, based on learning principals, we discuss regularization to enhance generalization. The challenges of quality in big data are reduced to the curse of dimensionality, class imbalances, concept drift and label noise, and the underlying reasons and mainstream methodologies to address these challenges are introduced. Learning model development has been driven by domain specifics, dataset complexities, and the presence or absence of human involvement. In this paper, we propose a robust learning paradigm by aggregating the aforementioned factors. Over the next few decades, we believe that these perspectives will lead to novel ideas and encourage more studies aimed at incorporating knowledge and establishing data-driven learning systems that involve both data quality considerations and human interactions.

show abstract