A hybrid gradient‐based and differential evolution algorithm for infinite impulse response adaptive filtering

Abstract

Global optimization algorithms (GO) had been applied to solve the adaptive infinite impulse response filtering problem, which is known to have multimodal error surface under certain conditions. However, although GO may be able to search multimodal surfaces, they have certain disadvantages. They may not converge to any minimum point, the convergence speed is reduced as the solution vectors move closer, and tracking ability for non-stationary environment is lacking. The traditional gradient descent method does not have these limitation but is not able to search multimodal surfaces. In this work, we propose a hybrid algorithm combining gradient descent and differential evolution (DE) for adapting the coefficients of infinite impulse response adaptive filters. DE is run in a block-based manner. The coefficient vector with the lowest error surface value (the best member) of the current block is updated via gradient descent for the duration of the next block. Thus combining the ability to search multimodal surface of DE and fast local search of gradient descent. As with all GO, global search capacity is gradually lost as the coefficient vectors converge together. Thus, re-initialization is also incorporated into the hybrid algorithm to provide continuous global search capacity for non-stationary environment. All the coefficient vectors except the best member are reinitialized when the normalized mean Euclidean distance between each pair of vectors falls below a threshold value. Simulation results show that the proposed algorithm achieves better solution quality and convergence speed than classic DE and GO for stationary and non-stationary environments. The poor performance of ABC compared with other algorithm is not due to bad choice of parameter, the algorithm seem to be sensitive to small block length, and hence noisy cost function than others. ABC becomes comparable with others when L is significantly larger than 30 Further work includes incorporating adaptive parameters for DE and adaptive stepsize for gradient descent, experimenting with block gradient update, and testing other measures that may be more appropriate than the normalized mean Euclidean distance for re-initialization checking.

An Improved Teaching-Learning-Based Optimization with Differential Learning and Its Application

“…PSO-wFIPS achieves better results than DLTLBO on 5 functions, while DLTLBO performs better for the remaining 19 functions. FDR-PSO only obtains better performance than DLTLBO on 5 , 20 , and 21…”

“…where V = [ 0 1 ⋅ ⋅ ⋅ 1 2 ⋅ ⋅ ⋅ ] denotes the digital IIR filter coefficient vector, is the number of samples used for the computation of the time-averaged cost function, and ( ) and ( ) are the filter's ideal and actual responses, respectively. 0.00e + 000 ± 0.00e + 000 0.00e + 000 ± 0.00e + 000 8.30 − 001 ± 8.07 − 001 1.99 + 000 ± 9.95 − 001 5.88 − 001 ± 5.49 − 001 2.19 + 000 ± 1.30 + 000 0.00e + 000 ± 0.00e + 000 8 0.00e + 000 ± 0.00e + 000 0.00e + 000 ± 0.00e + 000 2.07 − 010 ± 1.90 − 010 0.00e + 000 ± 0.00e + 000 0.00e + 000 ± 0.00e + 000 0.00e + 000 ± 0.00e + 000 0.00e + 000 ± 0.00e + 000 9 0.00e + 000 ± 0.00e + 000 0.00e + 000 ± 0.00e + 000 2.62 − 002 ± 1.29 − 002 6.93 − 002 ± 4.92 − 002 1.48 − 003 ± 3.31 − 003 4.44 − 017 ± 9.93 − 017 0.00e + 000 ± 0.00e + 000 10 1.27e − 004 ± 0.00e + 000 1.27e − 004 ± 0.00e + 000 2.41 + 002 ± 7.65 + 001 5.65 + 002 ± 1.88 + 002 4.82 + 002 ± 2.04 + 002 4.80 + 002 ± 1.81 + 002 5.92 + 001 ± 8.37 + 001 11 1.55 − 070 ± 3.46 − 070 4.39 − 066 ± 9.81 − 066 1.30 − 019 ± 1.51 − 019 2.85 − 102 ± 6.37 − 102 2.92 − 209 ± 0.00 + 000 8.70 − 190 ± 0.00 + 000 3.00e − 250 ± 0.00e + 000 12 14 3.55 − 015 ± 0.00 + 000 2.84 − 015 ± 1.59 − 015 5.76 − 011 ± 2.92 − 011 3.55 − 015 ± 0.00 + 000 3.55 − 015 ± 0.00 + 000 2.84 − 015 ± 1.59 − 015 1.78e − 015 ± 2.51e − 015 15 3.87 + 000 ± 1.28 + 000 3.66 + 000 ± 1.55 + 000 3.97 + 000 ± 3.24 + 000 7.56 + 000 ± 3.76 + 000 2.42 + 000 ± 1.81 + 000 2.98 + 000 ± 1.41 + 000 1.40e − 004 ± 1.97e − 004 16 0.00e + 000 ± 0.00e + 000 0.00e + 000 ± 0.00e + 000 1.48 − 007 ± 2.50 − 007 7.13 − 002 ± 1.26 − 001 0.00e + 000 ± 0.00e + 000 0.00e + 000 ± 0.00e + 000 0.00e + 000 ± 0.00e + 000 17 1.48 − 003 ± 3.31 − 003 3.94 − 003 ± 5.67 − 003 1.04 − 001 ± 8.74 − 002 1.13 − 001 ± 5.80 − 002 5.41 − 003 ± 8.59 − 003 1.48 − 003 ± 3.31 − 003 0.00e + 000 ± 0.00e + 000 18 1.38 + 002 ± 1.22 + 002 1.07e + 002 ± 1.21e + 002 2.80 + 002 ± 1.28 + 002 5.29 + 002 ± 2.00 + 002 5.46 + 002 ± 3.23 + 002 5.44 + 002 ± 2.81 + 002 1.18 + 002 ± 0.00 + 000 19 0.00e + 000 ± 0.00e + 000 0.00e + 000 ± 0.00e + 000 2.21 − 020 ± 2.43 − 020 3.03 + 000 ± 4.14 + 000 0.00e + 000 ± 0.00e + 000 1.01 − 029 ± 2.26 − 029 2.52 − 029 ± 3.57 − 029 20 0.00e + 000 ± 0.00e + 000 0.00e + 000 ± 0.00e + 000 5.87 − 008 ± 2.46 − 008 0.00e + 000 ± 0.00e + 000 3.06 − 011 ± 5.16 − 011 2.47 − 010 ± 5.33 − 010 8.77 − 001 ± 1.23 + 000 21 1.83e − 017 ± 3.99e − 017 1.50 − 001 ± 2.13 − 001 4.50 + 000 ± 1.57 − 001 1.53 + 000 ± 2.06 + 000 1.05 − 001 ± 1.49 − 001 8.73 + 000 ± 1.78 + 001 4.55 + 000 ± 8.73 − 001 22 0.00e + 000 ± 0.00e + 000 0.00e + 000 ± 0.00e + 000 2.26 + 000 ± 9.23 − 001 3.58 + 000 ± 2.06 + 000 7.76 + 000 ± 3.02 + 000 9.55 + 000 ± 4.37 + 000 3.48 + 000 ± 7.04 − 001 23 0.00e + 000 ± 0.00e + 000 0.00e + 000 ± 0.00e + 000 2.32 − 002 ± 1.71 − 002 6.88 − 001 ± 5.35 − 001 5.86 − 002 ± 2.77 − 002 6.75 − 002 ± 3.32 − 002 5.04 − 002 ± 3.65 − 002 24 0.00e + 000 ± 0.00e + 000 0.00e + 000 ± 0.00e + 000 5.73 − 011 ± 2.17 − 011 3.54 + 000 ± 1.98 + 000 6.34 − 001 ± 9. To achieve higher stop band attenuation and to have better control on the transition width, the frequency cost function [27] to be minimized is defined as…”

“…Figure 4 shows the comparison of convergence curves, magnitude responses, and gain responses of five algorithms. In addition, a statistical t-test [21] is given in Table 4.…”

“…Example 3. In the second example (taken from [17,18,21,22,26]), a second-order IIR filter is estimated by a first-order IIR filter. The unknown plant and the filter had the following transfer functions:…”

“…This error surface is generally nonquadratic and multimodal with respect to the filter coefficients, and hence optimization algorithms are required to find out better global solution. In order to avoid the local optima problem encountered by gradient descent methods in IIR system identification, considering IIR system identification as an optimization problem, recently, evolutionary algorithm and swarm intelligence, such as Genetic Algorithms (GA) [14][15][16], Simulated Annealing (SA) [17], Tabu Search (TS) [18], Differential Evolution (DE) [19][20][21], Ant Colony Optimization (ACO) [22], Artificial Bee Colony (ABC) algorithm [23], Particle Swarm Optimizer (PSO) [24,25], Gravitation Search Algorithm (GSA) [26,27], and Cuckoo search optimization [28], have been made for studying alternative structures and algorithms for adaptive digital IIR filters.…”