No abstract

Given the ongoing COVID-19 pandemic, remote interviews have become an increasingly popular approach in many fields. For example, a survey by the HR Research Institute (PCR Institute in Survey on hiring activities for graduates of 2021 and 2022. https://www.hrpro.co.jp/research_detail.php?r_no=273. Accessed 03 Oct 2021) shows that more than 80% of job interviews are conducted remotely, particularly in large companies. However, for some reason, an interviewee might attempt to deceive an interviewer or feel difficult to tell the truth. Although the ability of interviewers to detect deception among interviewees is significant for their company or organization, it still strongly depends on their individual experience and cannot be automated. To address this issue, in this study, we propose a machine learning approach to aid in detecting whether a person is attempting to deceive the interlocutor by associating the features of their facial expressions with those of their pulse rate. We also constructed a more realistic dataset for the task of deception detection by asking subjects not to respond artificially, but rather to improvise natural responses using a web camera and wearable device (smartwatch). The results of an experimental evaluation of the proposed approach with 10-fold cross-validation using random forests classifier show that the accuracy and the F1 value were in the range between 0.75 and 0.8 for each subject, and the highest values were 0.87 and 0.88, respectively. Through the analysis of the importance of the features the trained models, we revealed the crucial features of each subject during deception, which differed among the subjects.

According to a survey on the cause of death among Japanese people, lifestyle-related diseases (such as malignant neoplasms, cardiovascular diseases, and pneumonia) account for 55.8% of all deaths. Three habits, namely, drinking, smoking, and sleeping, are considered the most important factors associated with lifestyle-related diseases, but it is difficult to measure these habits autonomously and regularly. Here, we propose a machine learning-based approach for detecting these lifestyle habits using voice data. We used classifiers and probabilistic linear discriminant analysis based on acoustic features, such as mel-frequency cepstrum coefficients (MFCCs) and jitter, extracted from a speech dataset we developed, and an X-vector from a pre-trained ECAPA-TDNN model. For training models, we used several classifiers implemented in MATLAB 2021b, such as support vector machines, K-nearest neighbors (KNN), and ensemble methods with some feature-projection options. Our results show that a cubic KNN method using acoustic features performs well on the sleep habit classification, while X-vector-based models perform well on smoking and drinking habit classifications. These results suggest that X-vectors may help estimate factors directly affecting the vocal cords and vocal tracts of the users (e.g., due to smoking and drinking), while acoustic features may help classify chronotypes, which might be informative with respect to the individuals’ vocal cord and vocal tract ultrastructure.

No abstract