Digital Phenotyping of Mental Health using multimodal sensing of multiple situations of interest: A Systematic Literature Review

Moura, Ivan; Teles, Ariel Soares; Viana, Davi; Marques, Jean P. M.; Coutinho, Luciano Galvão; Silva, Francisco

doi:10.1016/j.jbi.2022.104278

Cited by 30 publications

(23 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…is the signal of the layer hypergraph， ( 1) l X + is the signal of the 1 l + layer hypergraph, () l  is the learnable filter matrix, v D is the diagonal matrix of the vertex degrees， e D is the diagonal matrix of edge degrees in equation (11). Let (0) X X = ，and  is the nonlinear activation function.…”

Section: Predictive Modeling(mh-hgnn)mentioning

confidence: 99%

“…Consequently, this phenomenon introduces substantial deviations in the measurement outcomes of psychopathological indicators. Second, in the realm of psychological research, it has been demonstrated by previous studies [7][8][9][10][11] that demographic information, health-related data, and personal, contextual information encompassing significant life transitions during adolescence serve as crucial indicators for assessing the mental health of middle-aged and older adults. However, existing investigations predominantly focus on establishing associations between mental health and individual factors, lacking a comprehensive evaluation of multiple variables.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Prediction Model of mental health of Middle-aged and older adults in China

Kong

2023

Preprint

View full text Add to dashboard Cite

The phenomenon of aging has emerged as a prominent concern within the context of China's economic and social progress. The middle-aged and elderly population suffers the dual burden of psychological and physiological challenges due to the decline in their physiological functions. However, the intricate nature of mental health diagnostic analysis poses difficulties in early predicting and intervening in mental health issues affecting middle-aged and elderly individuals. We attempt to establish a benchmark for evaluating the mental health of middle-aged and older adults based on psychological theories. Additionally, we propose a hypergraph-based mental health prediction model, termed MH-HGNN, specifically designed for this demographic. MH-HGNN incorporates a hypergraph structure to capture and process complex high-order correlation relationships by representing middle-aged and elderly individuals as nodes. By applying Laplace polynomial spectral convolution representation learning on these relationships, the model forecasts the mental health status of this population segment. Empirical findings indicate that MH-HGNN achieves an 82.7% accuracy rate in predicting mental health outcomes, surpassing the performance of prevalent deep learning baseline techniques like GNN, GAT, and GraphSAGE. Notably, the MH-HGNN model exhibits an improvement of up to 9.17% in accuracy compared to the baseline model.

show abstract

Section: Predictive Modeling(mh-hgnn)mentioning

confidence: 99%

mentioning

confidence: 99%

Prediction Model of mental health of Middle-aged and older adults in China

Kong

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Their hybrid model demonstrated an accuracy of 85% when tested on the HAM10000 dataset [1] [3]. Khan et al [12] introduced a deep learning model specifically designed for screening skin disease lesions effectively [10], [11]. They conducted experiments utilizing a mask recurrent neural network (MASK-RNN) and combined it with a pyramid network using ResNet50 to extract and classify features with a SoftMax classifier [10], [12].…”

Section: Introductionmentioning

confidence: 99%

Integration of Multimodal Data Sources for Enhanced Skin Disease Classification and Cancer Prediction: A Study Leveraging Pre-Trained Models on HAM_10000 Metadata and Squamous Cell Carcinoma (SCC) Images.

Khan,

S.,

Gangodkar

2024

Preprint

View full text Add to dashboard Cite

Skin diseases, disorders, and deficiencies are fundamental issues in the human body. Over the last few decades, they have become widely prevalent, often leading to cancer. This spectrum of ailments ranges from benign conditions to potentially life-threatening cancers, posing significant challenges to global health. Accurate classification of these diseases and timely prediction of malignant transformations are crucial for effective diagnosis and treatment planning. In our research paper, we propose an innovative approach that levitation multimodal data integration to enhance the precision of skin disease classification and improve the prediction of cancerous transformations. The study utilizes the HAM_10000 Metadata, a dataset comprising diverse skin lesions, along with 100 high-resolution images of Squamous Cell Carcinoma (SCC). Pre-trained models, specifically multi-model CNNs, are employed to extract intricate patterns and features. By amalgamating information from these heterogeneous sources and implementing advanced machine learning techniques, the results highlight the effectiveness of this integrative methodology. This approach can revolutionize the fields of dermatology and oncology by providing patients with skin diseases accurate, efficient, and early diagnoses of skin disease. Through this method, it was discovered that SCC images might hold a higher priority for skin cancer identification, achieving an accuracy rate of 92% on the original dataset.

show abstract

“…and neural signals (electroencephalogram [34], [35], functional magnetic resonance imaging [36], [37] and functional near-infrared spectroscopy [38]). The multimodal approach, or the combination of multiple types of signals, has been widely adopted to improve the accuracy and robustness of those automated assessments [39], [40]. For example, [41], [42] combined behavioral signals, including cues from video, audio, and text, while others [24], [43] found the combination and interaction between physiological and behavioral signals were also useful in evaluating disorders.…”

Section: Introductionmentioning

confidence: 99%

Multimodal mental health assessment with remote interviews using facial, vocal, linguistic, and cardiovascular patterns

Jiang,

Seyedi,

Griner

et al. 2023

Preprint

View full text Add to dashboard Cite

ObjectiveThe current clinical practice of psychiatric evaluation suffers from subjectivity and bias, and requires highly skilled professionals that are often unavailable or unaffordable. Objective digital biomarkers have shown the potential to address these issues. In this work, we investigated whether behavioral and physiological signals, extracted from remote interviews, provided complimentary information for assessing psychiatric disorders.MethodsTime series of multimodal features were derived from four conceptual modes: facial expression, vocal expression, linguistic expression, and cardiovascular modulation. The features were extracted from simultaneously recorded audio and video of remote interviews using task-specific and foundation models. Averages, standard deviations, and hidden Markov model-derived statistics of these features were computed from 73 subjects. Four binary classification tasks were defined: detecting 1) any clinically-diagnosed psychiatric disorder, 2) major depressive disorder, 3) self-rated depression, and 4) self-rated anxiety. Each modality was evaluated individually and in combination.ResultsStatistically significant feature differences were found between controls and subjects with mental health conditions. Correlations were found between features and self-rated depression and anxiety scores. Visual heart rate dynamics achieved the best unimodal performance with areas under the receiver-operator curve (AUROCs) of 0.68-0.75 (depending on the classification task). Combining multiple modalities achieved AUROCs of 0.72-0.82. Features from task-specific models outperformed features from foundation models.ConclusionMultimodal features extracted from remote interviews revealed informative characteristics of clinically diagnosed and self-rated mental health status.SignificanceThe proposed multimodal approach has the potential to facilitate objective, remote, and low-cost assessment for low-burden automated mental health services.

show abstract

Digital Phenotyping of Mental Health using multimodal sensing of multiple situations of interest: A Systematic Literature Review

Cited by 30 publications

References 69 publications

Prediction Model of mental health of Middle-aged and older adults in China

Prediction Model of mental health of Middle-aged and older adults in China

Integration of Multimodal Data Sources for Enhanced Skin Disease Classification and Cancer Prediction: A Study Leveraging Pre-Trained Models on HAM_10000 Metadata and Squamous Cell Carcinoma (SCC) Images.

Multimodal mental health assessment with remote interviews using facial, vocal, linguistic, and cardiovascular patterns

Contact Info

Product

Resources

About