Deep learning on multi-view sequential data: a survey

Abstract

With the progress of human daily interaction activities and the development of industrial society, a large amount of media data and sensor data become accessible. Humans collect these multi-source data in chronological order, called multi-view sequential data (MvSD). MvSD has numerous potential application domains, including intelligent transportation, climate science, health care, public safety and multimedia, etc. However, as the volume and scale of MvSD increases, the traditional machine learning methods become difficult to withstand such large-scale data, and it is no longer appropriate to use hand-craft features to represent these complex data. In addition, there is no general framework in the process of mining multi-view relationships and integrating multi-view information. In this paper, We first introduce four common data types that constitute MvSD, including point data, sequence data, graph data, and raster data. Then, we summarize the technical challenges of MvSD. Subsequently, we review the recent progress in deep learning technology applied to MvSD. Meanwhile, we discuss how the network represents and learns features of MvSD. Finally, we summarize the applications of MvSD in different domains and give potential research directions.

Multi‐view synergistic enhanced fault recording data for transmission line fault classification

“…This is because traditional data enhancement algorithms that introduce random perturbations purely from one perspective (e.g. time domain views or frequency domain views) without considering the synergy between different views [12] may introduce noise or data distortion, resulting in inconsistencies in the physical properties and statistical distributions of the synthesized data and the actual recorded waveform data. This can lead to a loss of realism and confidence in the enhanced data.…”