360-degree video provides an immersive experience to end-users through Virtual Reality (VR) Head-Mounted-Displays (HMDs). However, it is not trivial to understand the Quality of Experience (QoE) of 360-degree video since user experience is influenced by various factors that affect QoE when watching a 360-degree video in VR. This manuscript presents a machine learning-based QoE prediction of 360degree video in VR, considering the two key QoE aspects: perceptual quality and cybersickness. In addition, we proposed two new QoE-affecting factors: user's familiarity with VR and user's interest in 360-degree video for the QoE evaluation. To aim this, we first conduct a subjective experiment on 96 video samples and collect datasets from 29 users for perceptual quality and cybersickness. We design a new Logistic Regression (LR) based model for QoE prediction in terms of perceptual quality. The prediction accuracy of the proposed model is compared against well-known supervised machine-learning algorithms such as k-Nearest Neighbors (kNN), Support Vector Machine (SVM), and Decision Tree (DT) with respect to accuracy rate, recall, f1-score, precision, and mean absolute error (MAE). LR performs well with 86% accuracy, which is in close agreement with subjective opinion. The prediction accuracy of the proposed model is then compared with existing QoE models in terms of perceptual quality. Finally, we build a Neural Network-based model for the QoE prediction in terms of cybersickness. The proposed model performs well against the state of the art QoE prediction methods in terms of cybersickness.

Binaural sound source localization is an important and widely used perceptually based method and it has been applied to machine learning studies by many researchers based on head-related transfer function (HRTF). Because the HRTF is closely related to human physiological structure, the HRTFs vary between individuals. Related machine learning studies to date tend to focus on binaural localization in reverberant or noisy environments, or in conditions with multiple simultaneously active sound sources. In contrast, mismatched HRTF condition, in which the HRTFs used to generate the training and test sets are different, is rarely studied. This mismatch leads to a degradation of localization performance. A basic solution to this problem is to introduce more data to improve generalization performance, which requires a lot. However, simply increasing the data volume will result in data-inefficiency. In this paper, we propose a data-efficient method based on deep neural network (DNN) and clustering to improve binaural localization performance in the mismatched HRTF condition. Firstly, we analyze the relationship between binaural cues and the sound source localization with a classification DNN. Different HRTFs are used to generate training and test sets, respectively. On this basis, we study the localization performance of DNN model trained by each training set on different test sets. The result shows that the localization performance of the same model on different test sets is different, while the localization performance of different models on the same test set may be similar. The result also shows a clustering trend. Secondly, different HRTFs are divided into several clusters. Finally, the corresponding HRTFs of each cluster center are selected to generate a new training set and to train a more generalized DNN model. The experimental results show that the proposed method achieves better generalization performance than the baseline methods in the mismatched HRTF condition and has almost equal performance to the DNN trained with a large number of HRTFs, which means the proposed method is data-efficient.

No abstract

No abstract

With the appearance of a large amount of audio data, people have a higher demand for audio retrieval, which can quickly and accurately find the required information. Audio fingerprint retrieval is a popular choice because of its excellent performance. However, there is a problem about the large amount of audio fingerprint data in the existing audio fingerprint retrieval method which takes up more storage space and affects the retrieval speed. Aiming at the problem, this paper presents a novel audio fingerprinting method based on locally linear embedding (LLE) that has smaller fingerprints and the retrieval is more efficient. The proposed audio fingerprint extraction divides the bands around each peak in the frequency domain into four groups of sub-regions and the energy of every sub-region is computed. Then the LLE is performed for each group, respectively, and the audio fingerprint is encoded by comparing adjacent energies. To solve the distortion of linear speed changes, a matching strategy based on dynamic time warping (DTW) is adopted in the retrieval part which can compare two audio segments with different lengths. To evaluate the retrieval performance of the proposed method, the experiments are carried out under different conditions of single and multiple groups’ dimensionality reduction. Both of them can achieve a high recall and precision rate and has a better retrieval efficiency with less data compared with some state-of-the-art methods.