No abstract

In recent years, with the rapid development of stereoscopic display technology, its applications have become increasingly popular in many fields, and, meanwhile, the number of audiences is also growing. The problem of visual fatigue is becoming more and more prominent. Visual fatigue is mainly caused by vergence‐accommodation conflicts. An evaluation experiment was conducted, and the electroencephalogram (EEG) data of the subjects were collected when they were watching stereoscopic content, and then the stereoscopic fatigue state of the subjects during the viewing process was analyzed. As deep learning is proved to be an effective end-to-end learning method and multi-task learning can alleviate the problem of lacking annotated data, the authors establish a user visual fatigue assessment model based on EEG by using multi-task learning, which can effectively obtain the user’s visual fatigue status, so as to make the comfort designs to avoid the harm caused by user’s visual fatigue.

Three-dimension (3D) display has become increasingly popular in many fields. However, watching 3D content continuously can lead to visual fatigue that is harmful to users' vision system. Visual fatigue assessment aims at monitoring users' brain states based on the electroencephalogram (EEG) signals to identify different fatigue levels and avoid severe fatigue. Most of existing studies on the modeling of visual fatigue assessment rely on manual features extracted from EEG, which is time-consuming and needs prior knowledge. Convolutional Neural Networks (CNNs) which have been used in computer vision, speech recognition have attracted increasing interest. There is still a lack of attempts to employ endto-end EEG analysis on visual fatigue assessment. In this paper, we propose a deep learning model DeepFatigueNet to perform automatic feature extraction and classification from raw singlechannel EEG. The DeepFatigueNet is evaluated on our own visual fatigue dataset and compared with the state-of-the-art deep learning methods for EEG-based tasks. The overall accuracy of DeepFatigueNet reaches 75.9% on the three-classification task exceeding other models. The experimental results demonstrate the effectiveness of our model and show the potential of deep convolutional neural networks for end-to-end visual fatigue assessment.

The explosive growth of video data has brought great challenges to video retrieval, which aims to find out related videos from a video collection. Most users are usually not interested in all the content of retrieved videos but have a more fine-grained need. In the meantime, most existing methods can only return a ranked list of retrieved videos lacking a proper way to present the video content. In this paper, we introduce a distinctively new task, namely One-Stop Video Delivery (OSVD) aiming to realize a comprehensive retrieval system with the following merits: it not only retrieves the relevant videos but also filters out irrelevant information and presents compact video content to users, given a natural language query and video collection. To solve this task, we propose an end-to-end Hierarchical Video Graph Reasoning framework (HVGR) , which considers relations of different video levels and jointly accomplishes the one-stop delivery task. Specifically, we decompose the video into three levels, namely the video-level, moment-level, and the clip-level in a coarse-to-fine manner, and apply Graph Neural Networks (GNNs) on the hierarchical graph to model the relations. Furthermore, a pairwise ranking loss named Progressively Refined Loss is proposed based on prior knowledge that there is a relative order of the similarity of query-video, query-moment, and query-clip due to the different granularity of matched information. Extensive experimental results on benchmark datasets demonstrate that the proposed method achieves superior performance compared with baseline methods.

No abstract