A fundamental problem in neural network research, as well as in many other disciplines, is finding a suitable representation of multivariate data, i.e. random vectors. For reasons of computational and conceptual simplicity, the representation is often sought as a linear transformation of the original data. In other words, each component of the representation is a linear combination of the original variables. Well-known linear transformation methods include principal component analysis, factor analysis, and projection pursuit. Independent component analysis (ICA) is a recently developed method in which the goal is to find a linear representation of non-Gaussian data so that the components are statistically independent, or as independent as possible. Such a representation seems to capture the essential structure of the data in many applications, including feature extraction and signal separation. In this paper, we present the basic theory and applications of ICA, and our recent work on the subject. ᭧ 2000 Published by Elsevier Science Ltd.Keywords: Independent component analysis; Projection pursuit; Blind signal separation; Source separation; Factor analysis; Representation
MotivationImagine that you are in a room where two people are speaking simultaneously. You have two microphones, which you hold in different locations. The microphones give you two recorded time signals, which we could denote by x 1 (t) and x 2 (t), with x 1 and x 2 the amplitudes, and t the time index. Each of these recorded signals is a weighted sum of the speech signals emitted by the two speakers, which we denote by s 1 (t) and s 2 (t). We could express this as a linear equation:where a 11 , a 12 , a 21 , and a 22 are some parameters that depend on the distances of the microphones from the speakers. It would be very useful if you could now estimate the two original speech signals s 1 (t) and s 2 (t), using only the recorded signals x 1 (t) and x 2 (t). This is called the cocktailparty problem. For the time being, we omit any time delays or other extra factors from our simplified mixing model. As an illustration, consider the waveforms in Figs. 1 and 2. These are, of course, not realistic speech signals, but suffice for this illustration. The original speech signals could look something like those in Fig. 1 and the mixed signals could look like those in Fig. 2. The problem is to recover the data in Fig. 1 using only the data in Fig. 2.Actually, if we knew the parameters a ij , we could solve the linear equation in (1) by classical methods. The point is, however, that if you do not know the a ij , the problem is considerably more difficult.One approach to solving this problem would be to use some information on the statistical properties of the signals s i (t) to estimate the a ii . Actually, and perhaps surprisingly, it turns out that it is enough to assume that s 1 (t) and s 2 (t), at each time instant t, are statistically independent. This is not an unrealistic assumption in many cases, and it need not be exactly true in practice. ...
