Latent Code-Based Fusion: A Volterra Neural Network Approach

Abstract

We propose a deep structure encoder using the recently introduced Volterra Neural Networks (VNNs) to seek a latent representation of multi-modal data whose features are jointly captured by a union of subspaces. The so-called selfrepresentation embedding of the latent codes leads to a simplified fusion which is driven by a similarly constructed decoding. The Volterra Filter architecture achieved reduction in parameter complexity is primarily due to controlled non-linearities being introduced by the higher-order convolutions in contrast to generalized activation functions. Experimental results on two different datasets have shown a significant improvement in the clustering performance for VNNs auto-encoder over conventional Convolutional Neural Networks (CNNs) auto-encoder. In addition, we also show that the proposed approach demonstrates a muchimproved sample complexity over CNN-based auto-encoder with a superb robust classification performance.

Scaling Subspace-Driven Approaches Using Information Fusion

“…Uncovering the principles and laying out the fundamentals for multi-modal data has become an important topic in research in light of many applications in diverse fields including image fusion [8], target recognition [9][10][11][12], speaker recognition [13], and handwriting analysis [14]. Convolutional neural networks have been widely used on multi-modal data as in [15][16][17][18][19].…”

Decision-Based Fusion for Vehicle Matching

“…The topic of multi-modal data fusion has been extensively studied in computer vision. Laying out the fundamentals for data fusion has become crucial for many applications, including target recognition [ 7 , 8 , 9 ], handwriting analysis [ 10 ], and image fusion [ 11 ]. A comprehensive survey of data fusion is provided in [ 12 , 13 ].…”