System Identification Based on Tensor Decompositions: A Trilinear Approach

Abstract

The theory of nonlinear systems can currently be encountered in many important fields, while the nonlinear behavior of electronic systems and devices has been studied for a long time. However, a global approach for dealing with nonlinear systems does not exist and the methods to address this problem differ depending on the application and on the types of nonlinearities. An interesting category of nonlinear systems is one that can be regarded as an ensemble of (approximately) linear systems. Some popular examples in this context are nonlinear electronic devices (such as acoustic echo cancellers, which are used in applications for two-party or multi-party voice communications, e.g., videoconferencing), which can be modeled as a cascade of linear and nonlinear systems, similar to the Hammerstein model. Multiple-input/single-output (MISO) systems can also be regarded as separable multilinear systems and be treated using the appropriate methods. The high dimension of the parameter space in such problems can be addressed with methods based on tensor decompositions and modelling. In recent work, we focused on a particular type of multilinear structure—namely the bilinear form (i.e., two-dimensional decompositions)—in the framework of identifying spatiotemporal models. In this paper, we extend the work to the decomposition of more complex systems and we propose an iterative Wiener filter tailored for the identification of trilinear forms (where third-order tensors are involved), which can then be further extended to higher order multilinear structures. In addition, we derive the least-mean-square (LMS) and normalized LMS (NLMS) algorithms tailored for such trilinear forms. Simulations performed in the context of system identification (based on the MISO system approach) indicate the good performance of the proposed solution, as compared to conventional approaches.

Bilinear Models for Machine Learning

“…The idea of bilinear structures to replace the generic linear vector-based processing is not entirely new. For example, it was already adopted in the context of acoustic signal processing to identify the response of the accoustic channel [2,4].…”

“…Our work is closest in the spirit to [2,4] which were mainly concerned with tracking of time-varying models, while we deal with static data for classification which allows us to devise new efficient alternate optimization algorithms for high-rank BLR.…”

Low-Complexity Recursive Least-Squares Adaptive Algorithm Based on Tensorial Forms

“…When implementing system identification tasks, the problems can be reformulated to split the arithmetic workload into multiple smaller system identification problems. Considerable recent research was dedicated to identify multilinear forms [4][5][6] with two, three, or even more components using the Wiener filter [7][8][9] and adaptive algorithms based on the least-meansquare (LMS) method [10][11][12][13] or on the recursive least-squares (RLS) systems [10,14,15]. Despite the attractive performance of the latter, its classical implementations are considered too costly for most practical applications and the preferred workhorse remains the LMS or the normalized-LMS (NLMS) method, which is not sensitive to the scaling of its input signal.…”

A Kalman Filter for Multilinear Forms and Its Connection with Tensorial Adaptive Filters

“…Compared to the conventional Wiener solution, the iterative version can obtain a good accuracy of the solution, even when a few data are available for the estimation of the statistics. Furthermore, in [19], this solution was extended to the identification of trilinear forms, based on the decomposition of third-order tensors (of rank-1). Since there are inherent limitations to the Wiener filters (time-invariant framework, matrix inversion operation, etc.…”