Incremental Hierarchical Discriminant Regression

Abstract

This paper presents Incremental Hierarchical Discriminant Regression (IHDR) which incrementally builds a decision tree or regression tree for very high dimensional regression or decision spaces by an online, real-time learning system. Biologically motivated, it is an approximate computational model for automatic development of associative cortex, with both bottom-up sensory inputs and top-down motor projections. At each internal node of the IHDR tree, information in the output space is used to automatically derive the local subspace spanned by the most discriminating features. Embedded in the tree is a hierarchical probability distribution model used to prune very unlikely cases during the search. The number of parameters in the coarse-to-fine approximation is dynamic and data-driven, enabling the IHDR tree to automatically fit data with unknown distribution shapes (thus, it is difficult to select the number of parameters up front). The IHDR tree dynamically assigns long-term memory to avoid the loss-of-memory problem typical with a global-fitting learning algorithm for neural networks. A major challenge for an incrementally built tree is that the number of samples varies arbitrarily during the construction process. An incrementally updated probability model, called sample size dependent negative-log-likelihood (SDNLL) metric is used to deal with large-sample size cases, small-sample size cases, and unbalanced-sample size cases, measured among different internal nodes of the IHDR tree. We report experimental results for four types of data: synthetic data to visualize the behavior of the algorithms, large face image data, continuous video stream from robot navigation, and publicly available data sets that use human defined features.

Online-learned classifiers for robust multitarget tracking

“…methods in MATLAB to classify the extracted window images (64×32) as target identities: (1) K-Nearest Neighbor (K-NN), with K=1, and using a L1 distance metric for baseline performance; (2) Incremental Support Vector Machines (I-SVM) [4]; (3) Incremental Hierarchical Discriminant Regression (IHDR) [14] and (4) the proposed network described in this paper. We used a linear kernel for I-SVM, as is suggested for high-dimensional problems [10].…”

Real-time Accurate Runway Detection based on Airborne Multi-sensors Fusion

“…Existing works on airport runway detection can be broadly divided into two categories: feature-based [11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28] and template-based 11,29,30 . The first category relies on the detection of such features as intensity edges, high-contrast corners, or texture primitives.…”

“…In addition, the SIFT feature-based methods apply SIFT features to detect airport runway. Wang 17 , et al extract SIFT feature from regions of candidates and classify them by trained hierarchical discriminant regression (HDR) tree to recognise the airport runway 18 . Tao 19 , et al obtain a set of SIFT key points and use an improved SIFT matching strategy to detect runway.…”

Knowledge-driven path planning for mobile robots: relative state tree

“…However, due to the continuous high-dimensional feature of the actual application environment, this method performs poorly in real-time applications. Besides, another learning method relies on the similar distance that is calculated between the testing and stored samples, such as k-nearest searching (Yershova and LaValle 2007), k-d tree (Zhang et al 2011), Incremental Hierarchical Discriminant Regression (Weng and Hwang 2007;Zu et al 2007). The first two calculate the Euclidean distance between samples and the last one calculates a weighted Euclidean distance in a probabilistic manner, i.e., size-dependent negative-log-likelihood.…”