BYY Harmony Learning on Finite Mixture: Adaptive Gradient Implementation and A Floating RPCL Mechanism

Abstract

In tackling the learning problem on a set of finite samples, Bayesian Ying-Yang (BYY) harmony learning has developed a new learning mechanism that makes model selection implemented either automatically during parameter learning or in help of evaluating a new class of model selection criteria. In this paper, parameter learning with automated model selection has been studied for finite mixture model via an adaptive gradient learning algorithm for BYY harmony learning on a specific bidirectional architecture (BI-architecture). Via theoretical analysis, it has shown that the adaptive gradient learning implements a mechanism of floating rival penalized competitive learning (RPCL) among the components in the mixture. Also, the simulation results are demonstrated well for the adaptive gradient algorithm on the sample data sets from Gaussian mixtures with certain degree of overlap. Moreover, the adaptive gradient algorithm is applied to classification of the Iris data and unsupervised color image segmentation.

Dynamically regularized harmony learning of Gaussian mixtures

“…According to the theoretical and experimental results on this Bi-architecture of the BYY harmony learning system for Gaussian mixtures [20,17,18,19], the maximization of J(Θ k ) is capable of making model selection adaptively during parameter learning when the actual Gaussians or clusters are separated in a certain degree. That is, if we choose k to be larger than the number (k * ) of actual Gausians or clusters in the sample data, the maximization of the harmony function can make k * Gaussians to match the actual ones and simultaneously eliminate k−k * extra ones.…”

“…It has already been implemented on Guassian mixture learning and several BYY harmony learning algorithms have also been established for Gaussian mixtures [17,18,19]. Although the BYY harmony learning owns the ability of adaptive model selction, its parameter estimation has a notable deviation from the ML estimation which is consistent with true parameters.…”

Detecting the fuzzy clusters of complex networks

“…The typical example of such case is aðnÞ ¼ a 0 =n, where a 0 is a positive constant [25]. Another choice is to fix the learning rate a as a positive constant [26,27], which we utilize here, since the initial partition is good enough that the objective function Eq.…”

“…To further test the validity of the algorithms, we apply them to a sample network generated from a Gaussian mixture model [25][26][27]. This model is quite related the concept of random geometric graph proposed by Penrose [29] except that we take Gaussian mixture here compared with uniform distribution in [29].…”

“…To verify this fact,we denote e m r and e 1 r as the mean and maximal L 1 -error of r in Eq. (25), with respect to r FCM for our methods compared with the traditional FCM algorithm. Now we implement the algorithms by setting K ¼ 3, TOL ¼ 10 À6 , a ¼ 32, b ¼ 0:22.…”